![](https://csdnimg.cn/release/download_crawler_static/87323494/bg3.jpg)
Contrastive Multi-View Representation Learning on Graphs
Figure 1.
The proposed model for contrastive multi-view representation learning on both node and graph levels. A graph diffusion is used
to generate an additional structural view of a sample graph which along with a regular view are sub-sampled and fed to two dedicated
GNNs followed by a shared MLP to learn node representations. The learned features are then fed to a graph pooling layer followed by a
shared MLP to learn graph representations. A discriminator contrasts node representations from one view with graph representation of
another view and vice versa, and scores the agreement between representations which is used as the training signal.
3. Method
Inspired by recent advances in multi-view contrastive learn-
ing for visual representation learning, our approach learns
node and graph representations by maximizing MI between
node representations of one view and graph representation
of another view and vice versa which achieves better results
compared to contrasting global or multi-scale encodings on
both node and graph classification tasks (see section 4.4).
As shown in Figure 1, our method consists of the following
components:
•
An augmentation mechanism that transforms a sample
graph into a correlated view of the same graph. We only
apply the augmentation to the structure of the graphs
and not the initial node features. This is followed by
a sampler that sub-samples identical nodes from both
views, i.e., similar to cropping in visual domain.
•
Two dedicated GNNs, i.e., graph encoders, one for
each view, followed by a shared MLP, i.e., projection
head, to learn node representations for both views.
•
A graph pooling layer, i.e., readout function, followed
by a shared MLP, i.e., projection head, to learn graph
representations for both views.
•
A discriminator that contrasts node representations
from one view with graph representation from another
view and vice versa, and scores the agreement between
them.
3.1. Augmentations
Recent works in self-supervised visual representation learn-
ing suggest that contrasting congruent and incongruent
views of images allows encoders to learn rich represen-
tations (Tian et al., 2019; Bachman et al., 2019). Unlike
images in which views are generated by standard augmenta-
tions, e.g., cropping, rotating, distorting colors, etc., defining
views on graphs is not a trivial task. We can consider two
types of augmentations on graphs: (1) feature-space aug-
mentations operating on initial node features, e.g., masking
or adding Gaussian noise, and (2) structure-space augmenta-
tions and corruptions operating on graph structure by adding
or removing connectivities, sub-sampling, or generating
global views using shortest distances or diffusion matri-
ces. The former augmentation can be problematic as many
benchmarks do not carry initial node features. Moreover,
we observed that masking or adding noise on either spaces
degrades the performance. Hence, we opted for generating
a global view followed by sub-sampling.
We empirically show that in most cases the best results are
achieved by transforming an adjacency matrix to a diffusion
matrix and treating the two matrices as two congruent views
of the same graph’s structure (see section 4.4). We specu-
late that because adjacency and diffusion matrices provide
local and global views of a graph structure, respectively,
maximizing agreement between representation learned from
these two views allows the model to simultaneously encode
rich local and global information.