skip-connection
时间: 2023-09-14 22:04:58 浏览: 44
Skip connections are a type of connection used in neural networks, particularly in convolutional neural networks (CNNs) and residual neural networks (ResNets). Skip connections allow information to bypass one or more layers in the network, allowing the network to learn both low-level and high-level features simultaneously.
In a skip connection, the output of one layer is added to the output of a previous layer, and the sum is passed on to the next layer. This allows the network to learn not only from the immediate input, but also from previous inputs, which can help improve performance and reduce the risk of vanishing gradients.
Skip connections are particularly useful in deep networks, where the gradient can become very small as it is propagated back through the network, making it difficult to train. By allowing information to bypass certain layers, skip connections can help alleviate this problem and improve the overall performance of the network.