帮我解释一下这段话：The connection growth algorithm greedily activates useful, but currently ‘dormant,’ connections. We incorporate it in the following learning policy: Policy 1: Add a connection w iff it can quickly reduce the value of loss function L. The DNN seed contains only a small fraction of active connections to propagate gradients. To locate the ‘dormant’ connections that can reduce L effectively, we evaluate ∂L/∂w for all the ‘dormant’ connections w (computed either using the whole training set or a large batch). Policy 1 activates ‘dormant’ connections iff they are the most efficient at reducing L. This can also assist with avoiding local minima and achieving higher accuracy [28]. To illustrate this policy, we plot the connections grown from the input to the first layer of LeNet-300-100 [7] (for the MNIST dataset) in Fig. 3. The image center has a much higher grown density than the margins, consistent with the fact that the MNIST digits are centered. From a neuroscience perspective, our connection growth algorithm coincides with the Hebbian theory: “Neurons that fire together wire together [29]." We define the stimulation magnitude of the mth presynaptic neuron in the (l + 1)th layer and the n th postsynaptic neuron in the l th layer as ∂L/∂ul+1 m and x l n , respectively. The connections activated based on Hebbian theory would have a strong correlation between presynaptic and postsynaptic cells, thus a large value of (∂L/∂ul+1 m )x l n . This is also the magnitude of the gradient of L with respect to w (w is the weight that connects u l+1 m and x l n ): |∂L/∂w| = (∂L/∂ul+1 m )x l n (1) Thus, this is mathematically equivalent to Policy 1.

这段话描述了一种神经网络训练策略，即通过激活当前处于“休眠状态”的连接来提高训练效果。该策略被称为“连接增长算法”，其核心思想是在保证训练误差下降的前提下，尽可能地激活那些能够快速降低误差的“休眠连接”。在训练开始时，神经网络中只有少量的活跃连接用于梯度传播，因此需要寻找那些“休眠连接”并计算它们对误差的贡献，以此来确定哪些连接应该被激活。这个策略有助于避免局部最小值，并提高准确性。从神经科学的角度来看，这个策略与赫布理论相吻合，即“一起激活的神经元会相互连接”。在具体实现中，通过计算某个神经元的前后连接之间的梯度大小来确定是否激活该连接。这个梯度大小也是连接增长算法中用来衡量连接重要性的指标。