图像修复与插值的新型扩散模型研究

需积分: 9 110 浏览量更新于2024-07-17 收藏 13.09MB PDF 举报

"《 Novel Diffusion-Based Models for Image Restoration and Interpolation》一书探讨了基于扩散方程的新型模型在图像修复与插值领域的应用。该论文发表于2019年的《信号与通信技术》期刊，由Tudor Barbu撰写，他来自罗马尼亚科学院计算机科学研究所。本文献深入研究了长期以来，在信号处理、图像分析和计算机视觉等工程领域广泛应用的偏微分方程（PDE）模型，这些模型因其表达连续变化的能力而被用于描述动态现象。在过去几十年里，基于PDE的数学模型在图像处理任务中取得了显著成功，推动了这一科学领域的子领域发展。这些模型的优势在于它们的灵活性，强大的数学基础，以及其数值近似能够解释和再现多种经典的图像处理技术。例如，扩散模型可以有效地平滑图像噪声，恢复细节，同时保持边缘的清晰度，这对于图像去噪、超分辨率重建、以及图像插值至关重要。文中可能涵盖了具体的技术细节，如Perona-Malik扩散模型、Laplacian金字塔方法或非局部均值滤波器等，它们如何通过演化方程实现图像的自适应平滑和细节保留。此外，文章可能还讨论了如何利用这些模型进行插值，比如在图像缩放、缺失数据填充或视频帧间预测时，通过扩散过程预测缺失的像素或帧内容。值得注意的是，版权信息强调了所有权利保留，包括翻译、复制、再版、图像重用、朗诵、广播、数字化存取等任何形式的使用都必须得到出版商的许可。这意味着研究者和从业者在引用或使用这些模型时需要遵循版权规定。《Novel Diffusion-Based Models for Image Restoration and Interpolation》是一篇深入探讨了如何利用PDE理论解决实际图像处理问题的高质量学术论文，对于那些关注图像处理和计算机视觉领域的研究人员来说，具有重要的参考价值。"

2.1 Related Works 9

approximation of the model (2.6) would produce an iterative algorithm that evolves

the noised image to the denoised version.

This linear diffusion based smoothing process is equivalent to the 2D Gaussian

ﬁltering [4]. Thus, the parabolic Eq. (2.6) is characterized by a unique solution that

has the following form:

(

x, y, t

)





(

x, y

)

, t  0

(

∗ I

)(

x, y

)

, t > 0

(2.7)

where σ 

√

2t and G

denotes the two-dimensional Gaussian ﬁlter kernel that is

expressed as:

(

x, y

)



2πσ

−

2σ

(2.8)

The parameter σ > 0 determines the spatial size of the image details which are

removed by this 2D ﬁlter: the bigger σ, the smoother the result and the lesser details

are kept. The convolution operation in Eq. (2.7) is performed as following:

(

x, y

)

∗ I

(

x, y

)





(

x − ξ, y − η

)

· I

(

ξ,η

)

d ξ dη (2.9)

where this convolution is efﬁciently computed by applying the Fast Fourier Transform

(FFT).

This demonstrates that performing a linear diffusion for a time t with α  1is

exactly equivalent to performing the Gaussian restoration with σ 

√

2t. The 2D

Gaussian ﬁlter provided by Eq. (2.8) constitutes a low-pass ﬁlter that attenuates high

frequencies in a monotone way, as it results from the analysis of its behaviour in the

frequency domain [4].

The PDE-based linear ﬁltering approaches have some important advantages, such

as the easyness of handling, the fast execution and the low processing time. Also, the

PDE formulation in terms of a diffusion equation is much more natural and posesses

a higher generalization potential than the convolution with 2D Gaussians.

The main disadvantage of the linear diffusion models is the undesired image

blurring effect. The isotropic diffusion schemes perform a homogeneous denois-

ing, blurring equally in an all directions. These PDE denoising techniques remove

successfully the Gaussian noise, but often blur some important image details and

features, such as boundaries and textures, since they generate diffusion across the

edges.

A linear diffusion-based image denoising example is displayed in the next ﬁgure.

The denoised form of the noisy image from Fig. 2.1a, which is displayed in Fig. 2.1b,

is affected severely by the blurring effect.

Also, the linear PDE-based ﬁltering approaches could dislocate the edges when

moving from ﬁner to coarser scales [5]. Another drawback is that linear diffusion

2.1 Related Works 11

We have also proposed some improved linear diffusion models that overcome the

drawbacks of the existing linear PDE-based schemes. They are described in detail

in the next section of this chapter.

2.2 Second-Order Hyperbolic PDE-Based Restoration

Solutions

In this section and the next one we describe some of our contributions in this linear

PDE-based image restoration domain. They represent some improved linear diffusion

schemes that aim to solve the main drawbacks of the existing i sotropic linear PDE-

based denoising models.

The proposed restoration approaches are based on second-order hyperbolic equa-

tions [8–10]. Thus, we consider a class of hyperbolic diffusion-based restoration

models. It is composed of a linear second-order hyperbolic partial differential equa-

tion and a set of boundary conditions [10]. So, we have:

⎧

⎪

⎨

⎪

⎩

∂

∂t

+ γ

∂u

∂t

− α∇

u + ζ

(

u − u

)

 0

(

0, x, y

)

 u

(

x, y

)

∂u

∂t

(

0, x, y

)

 u

(

x, y

)

(

t, x, y

)

 0, ∀t ≥ 0,

(

x, y

)

∈ ∂

(

x, y

)

∈  (2.12)

where  ⊆

(

0, ∞

)

× R

represents the image domain, λ, γ, α ∈

(

0, 3

]

, ζ ∈

(

0, 0.5

]

and u

represents the initial image, which is corrupted by Gaussian noise.

The 2nd—order dynamic PDE model expressed by Eq. (2.12) is well-posed.

Assuming that u

∈ L





, then this isotropic diffusion-based equation is having

a unique weak solution that is continuous in t with values in L





. Moreover, if

∈ H





, then u

(

)

∈ H





, ∀t ≥ 0[9].

Also, this PDE scheme represents a non-Fourier model for the heat propaga-

tion, since its unique and weak solution is propagating with ﬁnite speed [10, 11].

Thus, one may demonstrate that if the support of u

is in the ball of radius r,





(

x, y

)

; x

+ y

≤ r



, then the support of u

(

t, x, y

)

⊆ B

r+ ρ t

,forsome

ρ>0. See [12], at page 260 for more proof. That means, the proposed differential

model possesses the localization property [5], unlike others linear PDEs, representing

Fourier models of heat propagation and having solutions propagating with inﬁnite

speed, therefore not having this important property.

The component ζ

(

u − u

)

from the Eq. (2.12) is introduced to stabilize this opti-

mal denoising solution, preventing the further degradation of the ﬁltered image. This

weak solution of the linear diffusion model is approximated numerically by using

the ﬁnite-difference method.

12 2 Linear PDE-Based Image Denoising Schemes

So, a consistent explicit numerical approximation scheme is developed for the

proposed PDE model [13, 14]. One considers a space grid size of h and a time step

t. The space and time coordinates are quantized as follows:

x  ih, y  jh, t  nt, ∀i ∈

{

0, 1,...,I

}

, j ∈

{

0, 1,...,J

}

, n ∈

{

0, 1,...,N

}

(2.13)

The second-order equation f rom (2.12) is equivalent to the second-order PDE

∂

∂t

+ γ

∂u

∂t

− α



∂

∂x

∂

∂y



+ ζ

(

u − u

)

 0, which can be discretized, by using

the ﬁnite differences, as:

n+t

(

i, j

)

n−t

(

i, j

)

−2u

(

i, j

)

t

+ γ

n+t

(

i, j

)

−u

n−t

(

i, j

)

2t

−

(

i+h, j

)

(

i−h, j

)

(

i, j +h

)

(

i, j −h

)

−4u

(

i, j

)

+ ζ



(

i, j

)

− u

(

i, j

)



 0

(2.14)

We may take h  1 and t  1, so (2.14) leads to the next explicit numerical

approximation scheme of the differential model:

n+1

(

i, j

)



2ζ −4λ

2λ+γ

(

i, j

)

−2λ

2λ+γ

n−1

(

i, j

)

+2ζ u

(

i, j

)

2α

(

i +1, j

)

+ u

(

i − 1, j

)

+ u

(

i, j +1

)

+ u

(

i, j − 1

)

− 4u

(

i, j

))

(2.15)

where u

(

i, j

)

 u

(

i, j

)

and n >0.

The obtained iterative restoration algorithm receives a

[

I × J

]

noisy image as

input and applies repeatedly the operation (2.15), for n  1, 2,…, N. The num-

ber of iterations of our numerical approximating scheme, N, is quite low, since

this discretization procedure converges fast to the solution representing the optimal

denoising.

The described linear hyperbolic diffusion technique has been successfully tested

on hundreds of images corrupted by various levels of Gaussian noise. Our restoration

model achieves satisfactory smoothing results while preserving the image features,

such as edges and other important details, quite well, although the image blurring is

not completely avoided. As one can see in the next ﬁgure, our approach overcomes

other unintended effects, such as the image staircasing [15] and speckle noise [16].

We have identiﬁed the following set of PDE model’s parameters that provide the

optimal results:

λ  2.4,γ  1.5,α  1.8,ζ  0.15,t  1, N  12 (2.16)

So, the optimal image smoothing is achieved after a low number of iterations,

N  12. That means our denoising technique runs very fast, its execution time being

less than 1 s.

2.2 Second-Order Hyperbolic PDE-Based Restoration Solutions 13

Table 2.1 PSNR values for several noise removal techniques

Our ﬁlter Linear

PDE-based ﬁlter

Average Wiener Perona-Malik TV

27.1 (dB) 24.3 (dB) 23.2 (dB) 24.5 (dB) 25.8 (dB) 24.3 (dB)

Method comparison were also performed and we found that our restoration tech-

nique outperforms not only the 2D conventional ﬁlters, but also numerous linear

and nonlinear PDE-based approaches. The smoothing performance of the proposed

approach is assessed using the Peak Signal-to-Noise Ratio (PSNR) measure [17].

Our denoising scheme obtains higher PSNR values than popular 2D classic ﬁl-

ters, such as two-dimension Gaussian, Average and Wiener [4], the linear PDE-

based denoising (heat equation) and inﬂuential nonlinear diffusion-based models,

like Perona-Malik scheme and TV Denoising (see more about them in the next

chapter). One can see the results in Table 2.1, which registers the average PSNR

values in decibels.

The denoising results produced by these techniques are displayed in Fig. 2.2,

which contains: (a) the original

[

512 × 512

]

Peppers image; (b) the image corrupted

by Gaussian noise with parameters μ  0.21 and variance  0.023; (c) the image

restored by our scheme; (d) Linear PDE-based ﬁltering (based on heat equation); (e)

and (f) the denoising results achieved by the

[

3 × 3

]

2D ﬁlter kernels (two-dimension

Average and Wiener); (g) Perona-Malik restoration; (h) TV denoising.

Therefore, our linear isotropic PDE restoration model achieves a better enhance-

ment than conventional ﬁlters and existing linear diffusion models, removing a greater

amount of Gaussian noise and reducing the blurring effect. It also provides better

restoration results than some nonlinear PDE and variational models, executes much

faster, by converging to the optimal solution in fewer iterations, and overcomes better

other unintended effects, such as the staircasing and speckling.

While some improved second-order nonlinear PDE models achieve better ﬁlter-

ing results (higher PSNR) than our linear hyperbolic scheme and a better deblurring,

our technique possesses some advantages over them, too. It avoids totally the stair-

case effect and executes considerably faster than those approaches, having a lower

computational cost.

Also, this proposed linear PDE-based model is important not only because it

operates successfully, produces satisfactory noise reduction results and outperforms

other smoothing algorithms, but also because some effective nonlinear diffusion

based image enhancement schemes can be derived from it. For example, a second-

order nonlinear hyperbolic PDE model that would provide much better deblurring

results can be obtained by transforming the Eq. (2.12)asfollowing:

∂

∂t

+ γ

∂u

∂t

− α

(

u

)

u + ζ

(

u − u

)

 0 (2.17)

剩余132页未读，继续阅读

drjiachen

粉丝: 172
资源: 2138

图像修复与插值的新型扩散模型研究

Diffusion-Based Generative Models.pdf

CVPR2023 AIGC应用论文和部分代码汇总整理

stable-diffusion-webui codeformer.pth

介绍下Palette-Image-to-Image-Diffusion-Models

diffusion-based sparse subspace clustering

stable-diffusion-webui安装

stable-diffusion-webui docker

安装stable-diffusion-webui

stable-diffusion-webui使用

服务器搭建stable-diffusion-webui

最新资源