![](https://csdnimg.cn/release/download_crawler_static/84997636/bga.jpg)
Contrastive Learning Objectives
• A common pattern:
• is the encoder, e.g., CNN and GNN.
• measures similarity between two embeddings.
• Usually implemented with an -way softmax function:
• Commonly referred to as the InfoNCE loss.
• The critic function can be simply implemented as .
Feburary 27, 2021 Graph Contrastive Learning: Recent Progress and Outlook
10
s(f(x),f(x
+
)) s(f(x),f(x
))
<latexit sha1_base64="+qd3YVUoxL5eCBAGY02kS3eKGfg=">AAACsXicbZHNThsxEMedLS00/SCUIxeLqFKittFuiwS9IXHpkUoNH0rSdOzMbqy11yvbS4lW+yg8Ddf2Afo2eEMqsYGRLP39+89o7BmWS2FdGP5rBc82nr/Y3HrZfvX6zdvtzs67M6sLw3HItdTmgoFFKTIcOuEkXuQGQTGJ5yw9qf3zKzRW6OyHW+Q4UZBkIhYcnEfTzqHtxb0xU+V11f/4X/38QPv9cZLQJ81P3mxPO91wEC6DPhbRSnTJKk6nOy02nmleKMwcl2DtKApzNynBOMElVu1xYTEHnkKCIy8zUGgn5fKHFX3vyYzG2viTObqkDytKUNYuFPOZCtzcrns1fMobFS4+mpQiywuHGb9vFBeSOk3rcdGZMMidXHgB3Aj/VsrnYIA7P9RGF6aq5l3r1AGzTZoYyOeCXzepLVgskiZThXTC6N9rFFLkKGWTzmw9lqreSrS+g8fi7PMgOhh8/X7QPT5a7WeL7JF90iMROSTH5Bs5JUPCyQ25JX/I3+BLcBn8Cth9atBa1eySRgTpHQhp15k=</latexit>
<latexit sha1_base64="4gJbZZWsygSJKBUoJMmIHSUkqHE=">AAACgHicbZFLSwMxEMfT9V3fevSyWIR6qbsi+DgJXjwqWC20RSbZ2TY02SzJrFpKv4RX/WJ+G9Pag1sdCPznNzPMZIbnSjqKoq9KsLC4tLyyulZd39jc2t7Z3Xt0prACm8IoY1scHCqZYZMkKWzlFkFzhU98cDOJP72gddJkDzTMsauhl8lUCiCPWmm9IxJDx887tagRTS38K+KZqLGZ3T3vVngnMaLQmJFQ4Fw7jnLqjsCSFArH1U7hMAcxgB62vcxAo+uOpgOPwyNPkjA11r+Mwin9XTEC7dxQc5+pgfpuPjaB/8XaBaUX3ZHM8oIwEz+N0kKFZMLJ78NEWhSkhl6AsNLPGoo+WBDkd1TqwvW47BszIOCuTHsW8r4Ub2XqCp7KXpnpQpG05nWOwgAFKlWmiZusZVz1V4nnb/BXPJ424rPG5f1Z7fpidp9VdsAOWZ3F7Jxds1t2x5pMMMXe2Qf7DIKgHpwE8U9qUJnV7LOSBVffQETGfw==</latexit>
<latexit sha1_base64="NsI7dPCvTIFCM7edLe3yTEk8Auk=">AAAChnicbZHNahtBDMfH26Z1naRJmmMvS03BgWB2Q0qSm6GXHF2oE4PtGs2s1h48s7PMaJMYs+/Ra/tWeZuMPw5ZO4IZ/vpJQkLiuZKOoui5Frx7v/fhY/1TY//g8PPR8cmXO2cKK7AnjDK2z8Ghkhn2SJLCfm4RNFd4z2c/l/H7B7ROmuw3zXMcaZhkMpUCyKM/rjUUiaHz1X82Pm5G7Whl4a6IN6LJNtYdn9T4MDGi0JiRUODcII5yGi3AkhQKy8awcJiDmMEEB15moNGNFquxy/C7J0mYGutfRuGKvq5YgHZurrnP1EBTtx1bwrdig4LS69FCZnlBmIl1o7RQIZlwuYMwkRYFqbkXIKz0s4ZiChYE+U1VunBdVn1jZgTcVenEQj6V4qlKXcFTOakyXSiS1jxuUZihQKWqNHHLtZQNf5V4+wa74u6iHV+2b35dNjvXm/vU2Vf2jbVYzK5Yh92yLusxwSz7y/6x/0E9aAc/gqt1alDb1JyyigWdF+XpyPo=</latexit>
[Oord et al., 2018] A. van den Oord, Y. Li, and O. Vinyals, Representation Learning with Contrastive Predictive Coding, arXiv.org, vol.
cs.LG. 2018.
L = E
X
"
log
exp(s(x, x
+
))
exp(s(x, x
+
))+
P
n1
j=1
exp(s(x, x
j
))
#
<latexit sha1_base64="FiaSDcgkOqJB/nkF+rsmpoVBVHs=">AAADO3icbVFNixQxEE23X+v4NatHL8FBnGXdoVsWXA8LCyJ4UFjB+YDp2SbJpHuyk3SaJK0zhPw0D/4Mz97Eq3fTPYNsz1oQ8urVK6qoh0vOtImiH0F44+at23f27nbu3X/w8FF3//FIy0oROiSSSzXBSFPOCjo0zHA6KRVFAnM6xsu3dX38hSrNZPHZrEs6EygvWMYIMp5Ku98TgcyCIG4/OHgKj5oUY/vOpROYcJqZqf9kDpNMIWITuir7NmnmWkXnDup+goVduZdw818cHrgD1whhS/lPeEUHD2GiK5Hay9PYXdjiKHaw3Yl5RXdb08t6BEwUyxdmlnZ70SBqAl4H8Rb0wDbO0/0AJ3NJKkELQzjSehpHpZlZpAwjnLpOUmlaIrJEOZ16WCBB9cw2+zj43DNzmEnlX2Fgw17tsEhovRbYK+tT6t1aTf6vNq1MdjKzrCgrQwuyGZRVHBoJa+PgnClKDF97gIhifldIFsibYry9rSlYuHYu5dIgrNtsrlC5YGTVZnWFM5a3OVFxw5T8usOiJSWU8zY71/VZXMe7Eu96cB2MXg3i48GbT8e9s5OtP3vgKXgG+iAGr8EZeA/OwRCQ4EXwMRgF4/Bb+DP8Ff7eSMNg2/MEtCL88xddPA8I</latexit>
s(x, y)=g( x)
>
g(y)
<latexit sha1_base64="60bfl27Equ1dtUaVr9TQghB53Sw=">AAACpXicbZHNbhMxEMed5auErxSOXCwiRCtQtIsqUQ5IlbhwQSpSk1ZKQjR2ZjdW7PXKnm0brfYheBqu8Bi8Dd50D2zKSJb/85sZjT0jCq08xfGfXnTn7r37D/Ye9h89fvL02WD/+cTb0kkcS6utuxDgUascx6RI40XhEIzQeC7Wn5v4+SU6r2x+RpsC5wayXKVKAgW0GLz1BzNhquv6XXNt6kP+iWctOvw+I1u0bggtBsN4FG+N3xZJK4astdPFfk/MllaWBnOSGryfJnFB8wocKamx7s9KjwXINWQ4DTIHg35ebX9V89eBLHlqXTg58S39t6IC4/3GiJBpgFZ+N9bA/8WmJaXH80rlRUmYy5tGaak5Wd6MiC+VQ0l6EwRIp8JbuVyBA0lhkJ0uwtRd39o1gfBdmjkoVkped6kvRaqyLjOlJuXs1Q6FNUrUukuXvhlL3Q9bSXZ3cFtM3o+So9HHb0fDk+N2P3vsJXvFDljCPrAT9oWdsjGT7Af7yX6x39Gb6Gt0Fk1uUqNeW/OCdSxa/AWxd9TJ</latexit>
<latexit sha1_base64="iha3f1TWL80na5z9J9cVcqkzlwE=">AAACj3icbZFLSwNBDMen66vWZ/XoZbEInsquKPUgUvCitwpWhVokO822Q+exzGTVUvoJvOqH89s4rT241cDAP78kJJMkmRSOouirFCwtr6yuldcrG5tb2zu71b17Z3LLsc2NNPYxAYdSaGyTIImPmUVQicSHZHg1jT+8oHXC6DsaZdhV0NciFRzIo1v9vFuL6tHMwr8inosam1vruVp6e+oZnivUxCU414mjjLpjsCS4xEnlKXeYAR9CHztealDouuPZpJPwyJNemBrrn6ZwRn9XjEE5N1KJz1RAA7cYm8L/Yp2c0vPuWOgsJ9T8p1Gay5BMOP122BMWOcmRF8Ct8LOGfAAWOPnlFLokalL0jRkSJK5I+xaygeBvReryJBX9IlO5JGHN6wKFIXKUskh7brqWItOCY+oHnVT8reLFy/wV9yf1+Kwe3Z7Wmhfzq5XZATtkxyxmDdZk16zF2owzZO/sg30G1aARXAbNn9SgNK/ZZwULbr4B2GnNHA==</latexit>
n