![](https://csdnimg.cn/release/download_crawler_static/22389436/bg3.jpg)
(a) (b)
(c)
Figure 2. Multi-resolution block: (a) multi-resolution group con-
volution and (b) multi-resolution convolution. (c) A normal con-
volution (left) is equivalent to fully-connected multi-branch con-
volutions (right).
and interlinked CNNs [132], lack careful design on when
to start low-resolution parallel streams and how and when
to exchange information across parallel streams, and do not
use batch normalization and residual connections, thus not
showing satisfactory performance.
GridNet [30] is like a combination of multiple U-Nets
and includes two symmetric information exchange stages:
the first stage only passes information from high-resolution
to low-resolution, and the second stage only passes infor-
mation from low-resolution to high-resolution. This limits
its segmentation quality.
3. Learning High-Resolution Representations
The high-resolution network [91], which we named
HRNetV1 for convenience, maintains high-resolution rep-
resentations by connecting high-to-low resolution convolu-
tions in parallel, where there are repeated multi-scale fu-
sions across parallel convolutions.
Architecture. The architecture is illustrated in Figure 1.
There are four stages, and the 2nd, 3rd and 4th stages are
formed by repeating modularized multi-resolution blocks.
A multi-resolution block consists of a multi-resolution
group convolution and a multi-resolution convolution which
is illustrated in Figure 2 (a) and (b). The multi-resolution
group convolution is a simple extension of the group convo-
lution, which divides the input channels into several subsets
of channels and performs a regular convolution over each
subset over different spatial resolutions separately.
The multi-resolution convolution is depicted in Figure 2
(b). It resembles the multi-branch full-connection manner
of the regular convolution, illustrated in in Figure 2 (c). A
regular convolution can be divided as multiple small con-
volutions as explained in [122]. The input channels are
divided into several subsets, and the output channels are
also divided into several subsets. The input and output sub-
sets are connected in a fully-connected fashion, and each
connection is a regular convolution. Each subset of output
channels is a summation of the outputs of the convolutions
over each subset of input channels.
The differences lie in two-fold. (i) In a multi-resolution
convolution each subset of channels is over a different res-
olution. (ii) The connection between input channels and
output channels needs to handle The resolution decrease is
implemented in [91] by using several 2-strided 3 × 3 con-
volutions. The resolution increase is simply implemented
in [91] by bilinear (nearest neighbor) upsampling.
Modification. In the original approach HRNetV1, only the
representation (feature maps) from the high-resolution con-
volutions in [91] are outputted, which is illustrated in Fig-
ure 3 (a). This means that only a subset of output channels
from the high-resolution convolutions is exploited and other
subsets from low-resolution convolutions are lost.
We make a simple yet effective modification by exploit-
ing other subsets of channels outputted from low-resolution
convolutions. The benefit is that the capacity of the multi-
resolution convolution is fully explored. This modification
only adds a small parameter and computation overhead.
We rescale the low-resolution representations through
bilinear upsampling to the high resolution, and concate-
nate the subsets of representations, illustrated in Figure 3
(b), resulting in the high-resolution representation, which
we adopt for estimating segmentation maps/facial landmark
heatmaps. In application to object detection, we construct
a multi-level representation by downsampling the high-
resolution representation with average pooling to multiple
levels, which is depicted in Figure 3 (c). We name the two
modifications as HRNetV2 and HRNetV2p, respectively,
and empirically compare them in Section 4.4.
Instantiation We instantiate the network using a similar
manner as HRNetV1 [91]
2
. The network starts from a stem
that consists of two strided 3 × 3 convolutions decreasing
the resolution to 1/4. The 1st stage contains 4 residual units
where each unit is formed by a bottleneck with the width 64,
and is followed by one 3× 3 convolution reducing the width
of feature maps to C. The 2nd, 3rd, 4th stages contain 1, 4,
3 multi-resolution blocks, respectively. The widths (number
of channels) of the convolutions of the four resolutions are
C, 2C, 4C, and 8C, respectively. Each branch in the multi-
resolution group convolution contains 4 residual units and
each unit contains two 3×3 convolutions in each resolution.
In applications to semantic segmentation and facial land-
mark detection, we mix the output representations (Figure 3
(b)), from all the four resolutions through a 1 × 1 convolu-
tion, and produce a 15C-dimensional representation. Then,
we pass the mixed representation at each position to a lin-
ear classifier/regressor with the softmax/MSE loss to pre-
dict the segmentation maps/facial landmark heatmaps. For
semantic segmentation, the segmentation maps are upsam-
pled (4 times) to the input size by bilinear upsampling for
both training and testing. In application to object detection,
2
https://github.com/leoxiaobin/
deep-high-resolution-net.pytorch