H. Zheng et al.: Multi-Contrast Brain MRI Image Super-Resolution With Gradient-Guided Edge Enhancement
FIGURE 2. Illustration for SRGR model in a local pattern. (a) is the target
HR image
˜
X; (b) is the HR reference image
˜
R with different contrast; (c) is
the pre-interpolated image X obtained from the low observation of
˜
X;
(d) An example of the gradient direction l in a local pattern and the edge
direction. Note: The gradient direction is denoted by the vector l.
In estimation, a more accurate pixel, e.g. X
i,j
+ δ
i,j
, indicated by dark
circles within (c) is used to replace the pre-interpolated HR pixel,
e.g. X
i,j
, to generate a better HR image.
where X
i+1,j−1
and X
i−1,j+1
are the neighborhood pixels of
X
i,j
along the direction l, while X
i−1,j−1
and X
i+1,j+1
are the
nearest neighbors of X
i,j
along the direction l
⊥
, respectively.
Second, the λ and λ
⊥
in Eq. (
3) are parameters that adjusting
intensity in two directions. Too smaller λ and λ
⊥
will lead to
blur edges. On the contrary, too larger λ and λ
⊥
will cause
artificial edges [19]. In SRGR, their values are estimated as
λ = (X
i+1,j−1
− X
i−1,j+1
)/(
˜
R
i+1,j−1
−
˜
R
i−1,j+1
)
λ
⊥
= (X
i−1,j−1
− X
i+1,j+1
)/(
˜
R
i−1,j−1
−
˜
R
i+1,j+1
). (5)
Overall, the SRGR model in Eq. (
3) tries to refine the pixels
of pre-interpolated image by minimizing the consistent sec-
ond order gradient that is parallel and perpendicular to the
edges. Since each term in this model is quadratic, Eq. (
3) can
be solved fast by forcing the first derivative of the objective
function to be zero. More details of implementation can be
found in [19].
However, the basic SRGR may not suit well for recovering
MRI since the local-contrast may be more complex than the
remote sensing images presented in [19]. How to modify the
SRGR to fit for multi-contrast MRI will be the main focus of
our work.
III. PROPOSED METHOD
An overview of the proposed approach is summarized
in Fig. 3. First, the LR image is pre-interpolated by the
CGI method. Second, the HR reference image is registered
to pre-interpolated image (Benefit of registration is shown in
Appendix C). Then, the HR image of interest will be restored
by using the gradient information of the registered HR ref-
erence image in another contrast. Finally, the HR image will
be enhanced with an IBP filter. In the following, the essential
parts of the proposed method will be presented.
A. MEASURING GRADIENT DIFFERENCE ON SAME
CONTRAST IMAGES
The essence of SRGR model in Eq. (3) is to estimate the
relationship of the second gradient order between the pre-
interpolated image X and the HR reference image
˜
R. Now,
let us consider an ideal step. Assume that an ideal pre-
interpolation can generate a HR image X that is as good as
the target HR image
˜
X. Accordingly, there may exist a pre-
interpolated reference image R that is as good as
˜
R. Then,
according to Eq. (2), the following relationship holds true
g(X
i,j
)
g(R
i,j
)
≈ λ
0
, (6)
where R
i,j
is the (i, j) pixel of R. Through the elementary
mathematical manipulation of Eqs. (2) and (
6), one obtains
λ
0
′
=
g(
˜
X
i,j
) − g(X
i,j
)
g(
˜
R
i,j
) − g(R
i,j
)
≈ λ
0
, (7)
which is the ratio of gradient difference estimated from the
same contrast images.
The Eq. (
7) implies that one may turn to measure the
ratio of the gradient difference for the pairs of target/pre-
interpolated images and reference/pre-interpolated reference
images. This modification is important since the gradient
difference is computed within one contrast, but not across
two contrasts in the original SRGR in Eq. (2). Thus, Eq. (
7)
is able to reduce the effect of large contrast variations on
degrading the multi-contrast MRI super-resolution, which
will be systematically analyzed in the following section.
B. SUPER-RESOLUTION MODEL
Based on Eq. (7), our model can be expressed as
min
δ
i,j
{[(g(X
i,j
+ δ
i,j
) − g(X
i,j
)) − λ(g(
˜
R
i,j
) − g(R
i,j
))]
2
+ [(g
⊥
(X
i,j
+ δ
i,j
) − g
⊥
(X
i,j
))
− λ
⊥
(g
⊥
(
˜
R
i,j
) − g
⊥
(R
i,j
))]
2
} (8)
which finds refinements on pixels to minimize the gradient
difference between the HR images and HR reference images.
In this model, the accuracy of λ and λ
⊥
is essential for
image reconstruction. Thus, error analysis on λ (same anal-
ysis for λ
⊥
is omitted) and edge pixels will be briefly given
below (More details can be found in Appendix A). The fol-
lowing analysis will show that, comparing with the SRGR,
the new method will obtain lower reconstruction error on
edge pixels.
In our model, the error on λ is denoted by 1λ
′
defined as
1λ
′
= λ
0
′
− λ, (9)
57858 VOLUME 6, 2018