Image Denoising Using A Generative Adversarial Network

Sushma Vispute1, Rajeswari Kanna2, Pratik Patil3, Rahul Karpe 4

,Harshwardhan Patki 5, Prathamesh Shahane 6

1,2,3,4,5,6
Department of Computer Engineering, Pimpri Chinchwad
College of Engineering, Pune, India.

[email protected] [email protected] [email protected]

Abstract

Image noise and low resolution are persistent challenges in digital image processing that significantly
impact image quality and interpretability. This research proposes a novel Generative Adversarial Network (GAN)
with a U-Net generator to simultaneously address image denoising and super-resolution. By integrating adversarial
training and a multi-objective loss function, our approach demonstrates superior performance in reconstructing
high-quality images from noisy, low-resolution inputs. Experimental results validate the method's effectiveness
across various image types, achieving significant improvements in Peak Signal-to-Noise Ratio (PSNR) and visual
quality.

Keywords:

Generative Adversarial Networks, Image Denoising, U-Net Generator, Adversarial Training, Peak Signal-to-Noise
Ratio.

1. Introduction
Image denoising remains a fundamental challenge in digital image processing, significantly impacting
various fields from medical imaging to satellite photography. As digital imaging technologies have advanced, the
need for effective noise reduction techniques has become increasingly critical for maintaining image quality and
preserving essential details. In recent years, Generative Adversarial Networks (GANs) have emerged as a
revolutionary approach to address this persistent challenge, offering unprecedented capabilities in distinguishing and
removing noise patterns while preserving image integrity. The fundamental principle of GANs in denoising
applications involves two competing neural networks: a generator that attempts to produce clean images from noisy
inputs, and a discriminator that learns to differentiate between real clean images and those produced by the
generator. This adversarial process has proven remarkably effective in learning complex noise patterns and
producing high-quality restorations that surpass traditional denoising methods.

The impact of GAN-based denoising solutions extends across multiple sectors, including healthcare, where clearer
medical images lead to more accurate diagnoses, and in surveillance systems, where enhanced image quality can be
crucial for security applications[1]. The ability of GANs to learn and adapt to various types of noise patterns, from
Gaussian to impulse noise, has made them particularly valuable in real-world applications where noise
characteristics can be unpredictable and complex.

1
The continuous evolution of GAN architectures specifically designed for denoising tasks has led to significant
improvements in both processing speed and output quality, making them increasingly practical for real-time
applications[2]. As these systems continue to develop, they promise to revolutionize how we approach image quality
enhancement across all domains of digital imaging

2. Literature review

Luca Tirel et al. (2024) introduced a hybrid model combining Pix2Pix and WGAN with Gradient Penalty
(WGAN-GP) to enhance the denoising of binary images. The model capitalized on Pix2Pix’s image-to-image
translation capabilities and WGAN-GP’s stability during training, making it effective for preserving fine structural
details in binary images. By enforcing Lipschitz continuity and incorporating residual blocks, the hybrid model
addressed mode collapse and ensured robust performance. Numerical experiments demonstrated its efficacy in
denoising tasks, particularly in applications such as document digitization and pattern recognition. However, the
approach is limited to binary images, and its adaptability to other image types remains unexplored[3].

Min-Ling Zhu et al. (2022) presented a robust GAN-based image denoising network with an optimization
algorithm designed to improve accuracy and stability. The proposed model introduced global residual connections to
the autoencoder generator, facilitating efficient feature extraction and learning. By optimizing noise at each node
during training, the network demonstrated resilience against gradient dispersion and feature loss. Experimental
results highlighted the model’s high accuracy, achieving over 99% on MNIST and over 90% on CIFAR-10 datasets.
Additionally, its PSNR and SSIM scores surpassed other methods on the BDS500 dataset, and it exhibited strong
defense capabilities against FGSM and PGD attacks. Despite its effectiveness, the model requires significant
computational resources and has been primarily validated on simpler datasets, limiting its application in more
complex scenarios[4].

Yinjin Ma et al. (2020) developed a GAN-based approach for low-dose CT image denoising, integrating a
hybrid loss function comprising least squares, structural similarity, and L1 losses. The proposed method focused on
learning the noise distribution within low-dose CT scans and subtracting it to produce clean images while preserving
fine textures and structural details. Experimental results revealed significant improvements in noise suppression and
artifact removal, with superior PSNR and structural similarity scores compared to traditional methods. While the
method achieved excellent results, its reliance on computationally expensive GANs and the challenges associated
with gradient instability during training remain drawbacks for practical deployment in clinical settings[5].

He Zhang et al. (2019) proposed an innovative framework, Image De-raining Conditional Generative
Adversarial Network (ID-CGAN), to tackle the challenges of single-image de-raining. This method employs a
conditional GAN framework where the generator leverages densely connected networks to enhance image details,
and the multi-scale discriminator focuses on distinguishing real and de-rained images using both local and global
information. To address the visual artifacts often introduced by GANs, the authors introduced a refined perceptual
loss function, ensuring better image quality. Experiments on synthetic and real datasets demonstrated that ID-CGAN
outperformed state-of-the-art methods in terms of visual appeal and quantitative measures. Moreover, its utility was
extended to object detection tasks, showcasing improved performance under adverse weather conditions. However,
the instability associated with GAN training and optimization challenges remain significant hurdles for this
approach[6].

2
 Alsaiari et al. (2019) proposed a generative adversarial network (GAN) for image denoising, addressing the
computational challenges of rendering photorealistic images. By developing a ResNet-based architecture with a
sophisticated loss function, the researchers created a method to transform noisy, incompletely rendered images into
high-quality outputs. Utilizing a dataset of 1,000 Pixar movie frames with Gaussian noise, the network demonstrated
remarkable generalizability, successfully denoising images across various domains including photographs and
medical scans. The approach significantly reduces rendering times from hours to seconds, offering a promising deep
learning solution for image processing with potential applications in computer graphics, medical imaging, and
real-time rendering technologies[7].

The examination of existing literature has underscored the significant advancements in image denoising
using Generative Adversarial Networks (GANs). Researchers have explored diverse applications, ranging from
binary image enhancement to medical imaging and computational rendering, leveraging innovations like hybrid loss
functions, residual connections, and integrated architectures. While these studies have achieved remarkable results in
terms of noise reduction, structural detail preservation, and computational efficiency, challenges such as training
instability, high resource demands, and limited generalizability persist. This paper seeks to address these gaps by
developing a robust GAN-based framework for simultaneous image denoising and super-resolution, laying a
foundation for broader applications in image restoration and quality enhancement.

3. Research Methodology

The objective of this research is to encompass the preparation, development, and evaluation of a novel
GAN-based framework for simultaneous image denoising and super-resolution. This methodology is structured into
several stages to ensure a thorough approach to problem-solving.

The first stage focuses on dataset preparation. The dataset comprises low-resolution images with artificially
added noise and their corresponding clean, high-resolution counterparts. During preprocessing, images are resized
into three formats: noisy low-resolution images (64×256), clean low-resolution images (64×256), and clean
high-resolution images (256×1024). To simulate real-world conditions, random noise matrices with a 20% variation
are applied to the low-resolution images. Augmentation techniques, including random flipping and normalization,
are employed to increase the dataset’s variability and improve model robustness. Normalized images are scaled to
the range of [-1, 1] for compatibility with the GAN architecture.

The GAN framework consists of a generator and two discriminators, designed to handle the tasks of
denoising and super-resolution. The generator employs a UNet-based architecture, which uses downsampling layers
with convolution, Leaky ReLU, and batch normalization to extract hierarchical features. The upsampling layers
employ transposed convolutions and dropout to enhance resolution. Skip connections between encoder and decoder
layers preserve fine-grained features, ensuring better restoration of details. The generator outputs both denoised
low-resolution images and super-resolved high-resolution images. The two discriminators are specialized for
different tasks: the first evaluates the quality of denoised low-resolution outputs (64×256), while the second assesses
the realism and fidelity of the super-resolved high-resolution outputs (256×1024). Both discriminators use
convolutional layers for downsampling and classification.

The training process involves carefully designed loss functions. The generator is trained using adversarial
loss, which employs binary cross-entropy to optimize its ability to fool the discriminators. L1 loss ensures
pixel-wise similarity between the generated and target images, with a lambda weighting factor of 40 to balance
adversarial and reconstruction losses. The discriminators use binary cross-entropy to calculate real and fake
classification losses. The Adam optimizer is employed for optimization, with learning rates set to 1e-4 for the
generator and 7e-3 for the discriminators. Training is conducted iteratively, with gradients computed and applied to
improve the performance of all components.

3
 Evaluation is a critical component of the methodology. Peak Signal-to-Noise Ratio (PSNR) is used as the
primary metric to quantify the quality of denoised and super-resolved images. Additionally, qualitative assessments
are performed by visually comparing the input, denoised, and high-resolution images to evaluate structural and
textural fidelity. To ensure reproducibility and track progress, model weights and training checkpoints are
periodically saved during the training process.

The model is trained for 120 epochs with a batch size of 1, ensuring sufficient iterations for
convergence while maintaining computational feasibility. The testing phase involves passing preprocessed test
images through the trained model to generate denoised and super-resolved outputs. The results are evaluated
quantitatively using PSNR scores and qualitatively through visual comparisons with ground truth images. This
methodology ensures the development of a robust image restoration model, effectively addressing the dual
challenges of denoising and resolution enhancement.

Fig. 1. Introductory scheme of the framework. Part (a) shows the clean, noisy image after adding synthetic noise,
and the synthetic ground truth real couple Part (b) illustrates the inputs and outputs of both generator and
discriminator models. Discriminator’s goal is to declare if the clean image in the couple is a real or generated image.
Adapted from [3]

4
4. Results and Discussion

The proposed Generative Adversarial Network (GAN)-based framework for image denoising and
super-resolution was evaluated on a custom dataset of noisy low-resolution images and their corresponding clean
high-resolution ground truth. The outputs generated by the model were assessed both quantitatively and qualitatively
to measure its effectiveness in noise reduction and detail preservation.

Quantitative Analysis
The model’s performance was evaluated using Peak Signal-to-Noise Ratio (PSNR) and Structural Similarity Index
Measure (SSIM), two widely accepted metrics in image restoration tasks. The results show that the proposed model
outperformed traditional denoising methods such as Gaussian filtering and BM3D, as well as earlier GAN-based
approaches. The combination of adversarial loss and L1 loss allowed the model to achieve higher PSNR scores,
reflecting better noise reduction and improved image fidelity. Furthermore, the inclusion of the super-resolution task
ensured that the output retained fine structural details and texture consistency.

Fig. 2. Comparison of PSNR and SSIM values across different denoising methods

Qualitative Analysis
Visual comparisons further highlight the effectiveness of the proposed model. The denoised outputs generated by the
GAN exhibit minimal residual noise and better texture preservation compared to other methods. While traditional
techniques like Gaussian filtering and BM3D result in smoother but blurry outputs, the GAN-based approach excels
at retaining edge sharpness and small details.
In the case of highly noisy inputs, the generator effectively suppresses artifacts while enhancing perceptual quality.
The super-resolved outputs produced by the model demonstrate significant improvement in resolution, offering crisp
and visually appealing result

5
Input and Output Comparisons

5. Conclusion

The results demonstrate that the proposed GAN-based framework effectively balances noise reduction and
detail preservation, outperforming traditional methods and other GAN-based architectures. By leveraging the UNet
generator architecture and dual discriminators, the model not only restores images but also enhances their resolution,
making it a powerful tool for real-world applications like medical imaging and document restoration. The
integration of super-resolution as an additional task ensures a more holistic image restoration pipeline.

6. References

[1]Zhong, Y., Liu, L., Zhao, D. et al. A generative adversarial network for image denoising. Multimed Tools Appl 79,
16517–16529 (2020). https://doi.org/10.1007/s11042-019-7556-x

[2]Xiao, H., Wang, X., Wang, J. et al. Single image super-resolution with denoising diffusion GANS. Sci Rep 14,
4272 (2024). https://doi.org/10.1038/s41598-024-52370-3

[3]L. Tirel, A. M. Ali, and H. A. Hashim, “Novel hybrid integrated Pix2Pix and WGAN model with gradient penalty
for binary images denoising,” Systems and Soft Computing, vol. 6, p. 200122, 2024. DOI:
10.1016/j.sasc.2024.200122

[4]M.-L. Zhu, L.-L. Zhao, and L. Xiao, “Image denoising based on GAN with optimization algorithm,” Electronics,
vol. 11, no. 15, p. 2445, 2022. doi: 10.3390/electronics11152445.

6
[5]Ma, Yinjin & Wei, Biao & Feng, Peng & He, Peng & Guo, Xiaodong & Wang, Ge. (2020). Low-Dose CT Image
Denoising Using a Generative Adversarial Network With a Hybrid Loss Function for Noise Learning. IEEE Access.
PP. 1-1. 10.1109/ACCESS.2020.2986388.

[6]H. Zhang, V. Sindagi and V. M. Patel, "Image De-Raining Using a Conditional Generative Adversarial Network,"
in IEEE Transactions on Circuits and Systems for Video Technology, vol. 30, no. 11, pp. 3943-3956, Nov. 2020, doi:
10.1109/TCSVT.2019.2920407.

[7]Alsaiari, Abeer & Rustagi, Ridhi & Alhakamy, A'Aeshah & Thomas, Manu Mathew & Forbes, Angus. (2019).
Image Denoising Using A Generative Adversarial Network. 126-132. 10.1109/INFOCT.2019.8710893.