Nonlocal means methods comparison:

INTRODUCTION:
Nonlocal means filtering consists in finding the similarities between sets of pixels (patches) that are present
in the image without restricting the research to a limited portion of it (nonlocal); this procedure finds
redundancy of the information contained in the image: by recombining these patches, it is possible to
achieve noise reduction, super resolution, image inpainting, compression and interpolation.

GENERAL IMPLEMENTATION FOR DENOISING:

Nonlocal means technique has been implemented for the first time as a denoising algorithm.

Considering a neighborhood B(q, r) with size (2𝑟 + 1) × (2𝑟 + 1) centered on a pixel ′𝑞′

The denoising of a pixel ‘p’ belonging to an image is be given by:

1
𝑢̂(𝑝) = ∑ 𝑢(𝑞)𝜔(𝑝, 𝑞) 𝐶(𝑝) = ∑ 𝑢(𝑞)𝜔(𝑝, 𝑞)
𝐶(𝑝)
𝑞∈𝐵(𝑝,𝑟) 𝑞∈𝐵(𝑝,𝑟)

Where the weighs 𝜔(𝑝, 𝑞) exponentially decay with the squared Euclidean distance between the pixels
neighborhoods.
‖𝐵(𝑝,𝑟)−𝐵(𝑞,𝑟)‖
−
𝜔(𝑝, 𝑞) = 𝑒 ℎ2

Here, 'h' is a filtering constant used to compensate for the variance of the noise that is affecting the image,
for higher values the filtering becomes less selective and vice-versa.

The physical meaning of this equation can be explained as a ‘scan’ of the image searching for patches that
are similar to each other.

When two patches are similar, their distance will be low, thus the decaying exponential will be closer to 1
and therefore that similar patch will contribute meaningfully in an averaging process represented by the
sum and the normalization factor 𝐶(𝑝).

Assuming the noise to be a random Gaussian process, the averaging will grant a noise reduction level that
increases with a higher amount of similarities.

NONLOCAL MEANS FOR SUPER-RESOLUTION:

The concept behind nonlocal means can be extended to achieve super-resolution of an image.

Starting with the premise that a given image may be considered as a down-sampled version of a better quality
one, super-resolution presents itself as a technique to solve the inverse problem of recovering the hi-res version
from the low-res.

The problem mentioned above can be modeled as:

𝑋𝑙𝑟 = 𝐷𝐻𝐹𝑋ℎ𝑟 + 𝑛
Where 𝑋𝑙𝑟 and 𝑋ℎ𝑟 denote the low resolution and the high resolution images, ‘H’ is the decimation operator, F is
a warping matrix, ‘H’ is a blurring matrix and ‘n’ is Gaussian white noise.

Considering an up-sampled version of the low res image 𝑥𝑢𝑝 the interpolated image would be:

1
𝑥̂(𝑝) = ∑ 𝑥𝑢𝑝 (𝑞)𝜔𝑢𝑝 (𝑝, 𝑞) 𝐶(𝑝) = ∑ 𝑥𝑢𝑝 (𝑞)𝜔𝑢𝑝 (𝑝, 𝑞)
𝐶(𝑝)
𝑞∈𝐵(𝑝,𝑟) 𝑞∈𝐵(𝑝,𝑟)

The up-sampling process is required because the resulting image, the weigh matrix and the starting image
should all have the same size.

A possible approach is to use wavelet decomposition, this can be implemented via a 6 step algorithm:

1. Decompose the LR image using CDF 9/7 wavelet.

2. Apply wavelet interpolation in the sub-bands HL and LH to obtain the up-sampled version.
3. Calculate the weigh matrix for every coefficient in the scaled-up sub-bands.
4. Estimate the sub-band coefficient by weighed-averaging the relevant coefficients with the weights.
5. Fill the sub-band HH with zeros and use the original LR for sub-band LL’.
6. Apply the inverse wavelet transform to get a HR image.

In short, the wavelet decomposition of the LR image is computed, up-scaling and nonlocal means filtering is
applied to the HL and LH coefficients, then the HR image is built with inverse wavelet transform using the
computed LH’ and HL’ along with the LR image as LL coefficients and zeros as HH coefficients.

MRI UPSAMPLING:
The concept behind nonlocal means filtering can be extended from the 2D domain to the 3D domain: pixel
become voxels, while 2D patches become 3D patches.

To increase the regularization power of the nonlocal means reconstruction, the weights computation is modified
to increment selectivity:

‖𝐵(𝑝,𝑟)−𝐵(𝑞,𝑟)‖
− 𝑖𝑓 |𝜇𝑝 −𝜇𝑞 |<3ℎ/√𝑁
𝜔(𝑝, 𝑞) = { 𝑒 ℎ2
0 𝑜𝑡ℎ𝑒𝑟𝑤𝑖𝑠𝑒
Where 𝜇𝑝 𝑎𝑛𝑑 𝜇𝑞 are the local averages of the patches centered on p and q, N is the number of elements
contained in a patch and ‘h’ is the selectivity parameter.

This modification will exclude dissimilar patches from the averaging process allowing a higher fidelity of
reconstruction.

To improve consistency over the structure of the up-sampled 3d image we can impose a new constraint:
N
1
𝑦𝑝 − ∑ x̌i = 0 ∀𝑝 ∈ 𝑦
N
i=1

This constraint is built upon the initial hypothesis that the LR image is the result of a down-sampling of the HR
image we want to calculate: it physically means that we want the down-sampled version of the resulting HR
image 𝑥̌to be equal to the mean of the starting LR image ‘y’.
The down-sampling is simulated by the local averaging operation of the reconstructed image.

The implementation of this further constraint needs an iterative process:

After each cycle of reconstruction via nonlocal means, the intensity difference between the LR image and the
computed image is added back to the interpolated data.

The reconstruction is then recalculated with the corrected down-sampled image from the previous step halving
the ‘h’ parameter, these 2 steps are repeated until no meaningful difference is found between the
downsampling of the reconstructed image and the original LR.

FEATURE BASED NONLOCAL UPSAMPLING:

A limitation of the nonlocal means technique is the lack of sensitivity to geometric transformation