Fast-MoCo：组合补丁加速自监督学习的对比学习

174 浏览量更新于2024-06-19 收藏 19.36MB PDF 举报

"Fast-MoCo：基于组合补丁的对比学习提速自监督学习" 自监督学习是当前计算机视觉领域中的一个重要研究方向，尤其在对比学习方面取得了显著的成就。对比学习的方法通过区分不同实例的嵌入，使得正样本对之间的距离更接近，而负样本对之间的距离更远，以此来学习表示。MoCo（Momentum Contrast）是一种对比学习框架，它通过动量编码器保持一个大的负样本库，以增强模型的学习能力。然而，现有的自监督学习方法通常需要大量的训练周期，例如MoCo v3可能需要800个训练周期才能达到理想的效果。这不仅对学术研究造成负担，也限制了自监督学习的快速迭代和发展。针对这一问题，Fast-MoCo提出了一种新的解决方案。 Fast-MoCo的核心创新在于利用组合补丁来构建多个正样本对。传统的对比学习通常从两个不同的增强视图中仅生成一个正样本对，而Fast-MoCo通过组合来自这两个视图的不同补丁，生成了丰富的正样本对，从而在几乎不增加计算成本的情况下，提高了监督信号的多样性。这种方法显著提升了学习效率。实验结果显示，Fast-MoCo在仅100个训练周期后就能达到73.5%的线性评估准确性，这与MoCo v3经过800个周期训练后的性能相当。继续训练200个周期后，准确率进一步提升至75.1%，与当前最先进的方法相比肩。此外，Fast-MoCo在多个下游任务上的表现也验证了其有效性。 Fast-MoCo的提出，不仅为自监督学习的训练速度设定了新的标准，也为未来的研究开辟了新的方向。它展示了如何通过改进对比学习的采样策略，提高模型的训练效率，同时也降低了对大规模训练资源的需求。这使得研究人员能够更快地探索和优化自监督学习算法，推动整个领域的进步。 Fast-MoCo的源代码和预训练模型已在GitHub上公开，供研究者参考和使用，进一步促进了自监督学习的社区合作和研究发展。

Fast-MoCo: Boost Momentum-based Contrastive Learning 5

Divide

Momentum

update

Combine

Grad.

encoder 





encoder 

Combine

Target

Branch

Online

Branch

Momentum

update



Contrastive

Loss



Fig. 2: Overview of Fast-MoCo framework. It consists of four steps: 1) Divide

step, where the input image in the online branch is divided into multiple patches;

2) Encode step, which the encoder f encodes the features of the patches sepa-

rately; 3) Combine step, which combines the encoded features (at the last layer

of the neural network); 4) the combined features are fed into projector g, pre-

dictor q, and contrastive loss for contrastive learning. Compared with MoCo,

we add the Divide step and Combine Step in the online branch, with details in

Section 3.2. The target branch is the same as MoCo.

3.2 Fast-MoCo

In this section, we introduce Fast-MoCo, a simple method that can greatly im-

prove the training eﬃciency of self-supervised learning with negligible extra cost.

An overview of Fast-MoCo is shown in Fig.2. With MoCo v3 as the baseline,

Fast-MoCo only makes three modiﬁcations, 1) add a Divide step to divide an

image into multiple patches before sending the patches to the encoder

‡

of the

online branch, 2) insert a Combine step (e.g., Combine) immediately behind

the encoder to combine patches, and 3) a slightly modiﬁed deﬁnition of positive

and negative pairs corresponding to the divide and combine operations. In the

following, we illustrate the Divide step, Combine step, and the modiﬁed loss

function in detail.

Divide Step. For the online branch, instead of directly feed the given the

augmented image x

into the encoder, we ﬁrst divide it into a m × m grid of

patches {x

|p ∈ {1, . . . , m

}} as shown in Fig.2, with p denotes the set of patch

index {p}. The inﬂuence of m in will be analyzed in Section 5.4.

Combine Step. Instead of directly using the encoded embedding of each

patch individually for further step, we combine multiple (less than m

) patch

embeddings v

to form combined embeddings c before sending them to further

step, i.e., the projector.

To form a combined embedding, we take a subset of n indices from the

patch index set p, noted as p

(⊆ p), and collect their corresponding features

= {v

|p ∈ p

}. While there could be diverse options to combine multiple

embeddings (e.g., concatenate, sum), we empirically found that simply averag-

‡在本文中，我们仅探索了ResNet50作为编码器，而将ViT版本的MoCo

v3的评估作为我们未来的工作。

+v:mala2255获取更多论文

剩余21页未读，继续阅读

cpongm

粉丝: 5
资源: 2万+

Fast-MoCo：组合补丁加速自监督学习的对比学习

自监督学习：生成和对比方法综述

自监督的对比学习框架都有哪些

一文梳理无监督对比学习（MoCo/SimCLR/SwAV/BYOL/SimSiam）

以 SimCLR、InfoLoss、MOCO、BYOL为关键词讲解对比学习

class MoCo_ResNet(MoCo):

moco训练自己的数据集

MoCo是怎么自监督的

MoCo queue

simclr和moco

最新资源