Arun Solanki

Mohd Naved Editors

GANs for Data

Augmentation
in Healthcare
GANs for Data Augmentation in Healthcare
Arun Solanki • Mohd Naved
Editors

GANs for Data

Augmentation in Healthcare
Editors
Arun Solanki Mohd Naved
Gautam Buddha University SOIL School of Business Design
Greater Noida, India Greater Noida, India

ISBN 978-3-031-43204-0 ISBN 978-3-031-43205-7 (eBook)

https://doi.org/10.1007/978-3-031-43205-7

© The Editor(s) (if applicable) and The Author(s), under exclusive license to Springer Nature
Switzerland AG 2023
This work is subject to copyright. All rights are solely and exclusively licensed by the Publisher, whether
the whole or part of the material is concerned, specifically the rights of translation, reprinting, reuse of
illustrations, recitation, broadcasting, reproduction on microfilms or in any other physical way, and
transmission or information storage and retrieval, electronic adaptation, computer software, or by similar
or dissimilar methodology now known or hereafter developed.
The use of general descriptive names, registered names, trademarks, service marks, etc. in this publication
does not imply, even in the absence of a specific statement, that such names are exempt from the relevant
protective laws and regulations and therefore free for general use.
The publisher, the authors, and the editors are safe to assume that the advice and information in this book
are believed to be true and accurate at the date of publication. Neither the publisher nor the authors or the
editors give a warranty, expressed or implied, with respect to the material contained herein or for any
errors or omissions that may have been made. The publisher remains neutral with regard to jurisdictional
claims in published maps and institutional affiliations.

This Springer imprint is published by the registered company Springer Nature Switzerland AG
The registered company address is: Gewerbestrasse 11, 6330 Cham, Switzerland

Paper in this product is recyclable.

Contents

Role of Machine Learning in Detection and Classification

of Leukemia: A Comparative Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
Ruchi Garg, Harsh Garg, Harshita Patel, Gayathri Ananthakrishnan,
and Suvarna Sharma
1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
2 Methodology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
3 Literature Review . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
3.1 Diagnosis by Using SVM, KNN, K-Means, and Naive Bayes . . . . . . . 4
3.2 Diagnosis by Using ANN and CNN Algorithms . . . . . . . . . . . . . . . . . . 7
3.3 Diagnosis by Using Random Forest and Decision
Tree Algorithms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
4 Results and Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
4.1 K-Means . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
4.2 Naive Bayes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
4.3 Support Vector Means . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
4.4 Logistic Regression . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
4.5 XG-Boost . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
4.6 Accuracy Achieved in Algorithms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
5 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
6 Declarations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
A Review on Mode Collapse Reducing GANs with GAN’s
Algorithm and Theory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
Shivani Tomar and Ankit Gupta
1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
2 Literature Survey. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
3 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38

v
vi Contents

Medical Image Synthesis Using Generative Adversarial Networks . . . . . . 41

Vishal Raner, Amit Joshi, and Suraj Sawant
1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41
1.1 Retinal Image Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42
2 Related Work. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43
3 Proposed Methodology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46
3.1 Generator Architecture. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47
3.2 Discriminator Architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48
4 Results and Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49
4.1 Experimental Setup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49
4.2 Dataset Description . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49
4.3 Performance Metrics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49
4.4 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50
5 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52
Chest X-Ray Data Augmentation with Generative
Adversarial Networks for Pneumonia and COVID-19 Diagnosis . . . . . . . 55
Beena Godbin A and Graceline Jasmine S
1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55
1.1 Related Works . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55
2 GAN Architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57
2.1 Generator Architecture. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58
2.2 Discriminator Architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59
2.3 Classifier . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59
2.4 Batch Size . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60
2.5 Transforms of Training Samples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60
2.6 Hyperparameters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60
2.7 Evaluation Metrics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61
3 Materials and Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63
3.1 Dataset . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63
4 Experimental Details and Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64
4.1 Synthetic Images Generated from GAN . . . . . . . . . . . . . . . . . . . . . . . . 64
5 Discussion and Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72
State of the Art Framework-Based Detection of GAN-Generated
Face Images . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75
Swati Shilaskar, Shripad Bhatlawande, Siddharth Nahar,
Mohammed Daanish Shaikh, Vishwesh Meher, and Rajesh Jalnekar
1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75
2 Related Work. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 76
3 Methodology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77
3.1 Dataset . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77
3.2 Models Used . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78
3.3 Hardware and Software Setup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82
3.4 Algorithms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82
Contents vii

4 Results and Discussions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 84

5 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 86
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87
Data Augmentation in Classifying Chest Radiograph
Images (CXR) Using DCGAN-CNN . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91
C. Rajeev and Karthika Natarajan
1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91
1.1 Data Augmentation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91
1.2 Augmented Versus Synthetic Data. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92
1.3 Why Data Augmentation Is Important Now? . . . . . . . . . . . . . . . . . . . . 93
1.4 How Does Data Augmentation Work? . . . . . . . . . . . . . . . . . . . . . . . . . . 93
1.5 Advanced Techniques for Data Augmentation . . . . . . . . . . . . . . . . . . . 94
1.6 Data Augmentation in Health Care . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95
1.7 Benefits of Data Augmentation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 96
1.8 Challenges of Data Augmentation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 96
2 GANs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97
2.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97
2.2 Why GANs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97
2.3 Components of GANs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 98
2.4 GAN Loss Function . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 100
2.5 Training and Prediction of GANs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 100
2.6 Challenges Faced by GANs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101
2.7 Different Types of GANs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 102
2.8 Steps to Implement Basic GAN . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103
3 Augmentation of Chest Radiograph Images
for Covid-19 Classification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103
3.1 Methodology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 104
3.2 DCGAN-CNN Model’s Architecture . . . . . . . . . . . . . . . . . . . . . . . . . . 106
4 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 108
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 109
Data Augmentation Approaches Using Cycle Consistent Adversarial
Networks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111
Agrawal Surbhi, Patil Mallanagouda, and Malini M. Patil
1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111
1.1 GANs Application in Healthcare . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 113
2 Data Augmentation in Deep Learning Models . . . . . . . . . . . . . . . . . . . . . . . 113
3 Data Augmentation in Healthcare . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 116
3.1 Basic Augmentation Techniques . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 117
3.2 Deformable Augmentation Techniques . . . . . . . . . . . . . . . . . . . . . . . . . 118
3.3 GAN-Based Augmentation Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . 118
4 Cycle Consistent Generated Adversarial Networks Architecture . . . . . . . . . 119
4.1 Introduction to Cycle Consistent GANs . . . . . . . . . . . . . . . . . . . . . . . . 119
5 Building Cycle Consistent GANs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 122
5.1 Generator . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 122
5.2 Discriminator . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 123
viii Contents

5.3 The Loss Function . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 123

6 Applications of Cycle Consistent GANs . . . . . . . . . . . . . . . . . . . . . . . . . . . 125
7 Cycle Consistent GAN in Healthcare . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 125
8 Future Scope and Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 129
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 130
Geometric Transformations-Based Medical Image Augmentation . . . . . . 133
S. Kalaivani, N. Asha, and A. Gayathri
1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 133
2 Geometric Transformations: Basic Manipulation . . . . . . . . . . . . . . . . . . . . . 136
3 Test-Time Augmentation (TTA) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 137
4 Synchronous Medical Image Augmentation (SMIA) Framework . . . . . . . . 138
5 Random Local Rotation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 139
6 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 139
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 140
Generative Adversarial Learning for Medical Thermal
Imaging Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 143
Prasant K. Mahapatra, Neelesh Kumar, Manjeet Singh,
Hemlata Saini, and Satyam Gupta
1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 143
2 What is a GAN (Generative Adversarial Network)?. . . . . . . . . . . . . . . . . . . 143
2.1 Overview of GAN Structure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 144
2.2 Mathematical Equation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 144
2.3 Major Applications of GAN . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 146
3 Self-supervised Generative Adversarial Learning . . . . . . . . . . . . . . . . . . . . 147
4 Conditional and Unconditional GANs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 147
5 Thermal Imaging Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 148
5.1 Why are Thermal Imaging Devices Beneficial? . . . . . . . . . . . . . . . . . . 148
6 Need of Data Augmentation in GANs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 149
7 Improved Medical Image Generation via Self-supervised Learning . . . . . . 150
8 Methods. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 150
8.1 Training Dataset . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 150
8.2 Results and Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 151
8.3 GAN Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 152
8.4 Outlook and Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 152
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 154
Improving Performance of a Brain Tumor Detection
on MRI Images Using DCGAN-Based Data Augmentation
and Vision Transformer (ViT) Approach . . . . . . . . . . . . . . . . . . . . . . . . . . . 157
Md. Momenul Haque, Subrata Kumer Paul, Rakhi Rani Paul,
Nurnama Islam, Mirza A. F. M. Rashidul Hasan,
and Md. Ekramul Hamid
1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 157
2 Related Works . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 158
2.1 Existing Data Augmentation Algorithms . . . . . . . . . . . . . . . . . . . . . . . . 160
Contents ix

2.2 Existing Brain Tumor Detection Algorithms . . . . . . . . . . . . . . . . . . . . . 161

3 Proposed Methodology’s . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 162
3.1 Proposed Workflow . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 163
3.2 Model Evolution Metrics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 170
4 Experimental Results and Discussions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 170
4.1 Dataset Description . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 170
4.2 Data Augmentation Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 171
4.3 Classification Performance Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . 173
4.4 Comparative Assessment of the Proposed Approach
and Established Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 183
5 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 184
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 184
Combining Super-Resolution GAN and DC GAN for Enhancing
Medical Image Generation: A Study on Improving
CNN Model Performance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 187
Mahesh Vasamsetti, Poojita Kaja, Srujan Putta, and Rupesh Kumar
1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 187
2 Related Work. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 188
3 Methodology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 190
3.1 Dataset . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 190
3.2 Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 192
3.3 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 196
3.4 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 202
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 203
GAN for Augmenting Cardiac MRI Segmentation . . . . . . . . . . . . . . . . . . . 207
Pawan Whig, Pavika Sharma, Rahul Reddy Nadikattu,
Ashima Bhatnagar Bhatia, and Yusuf Jibrin Alkali
1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 207
1.1 Overview of Cardiac MRI Segmentation . . . . . . . . . . . . . . . . . . . . . . . 208
1.2 Challenges in Cardiac MRI Segmentation. . . . . . . . . . . . . . . . . . . . . . . 209
1.3 Introduction to GANs and Data Augmentation . . . . . . . . . . . . . . . . . . . 209
2 Literature Review of Cardiac MRI Segmentation . . . . . . . . . . . . . . . . . . . . 210
2.1 GAN Image Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 212
2.2 Data Augmentation Techniques for Medical Image Segmentation . . . . 213
3 Methodology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 213
3.1 Dataset Description . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 213
3.2 Network Architecture of GAN. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 214
3.3 Training Procedure of GAN for Augmentation . . . . . . . . . . . . . . . . . . . 214
3.4 Segmentation Network Architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . 215
3.5 Segmentation Training Procedure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 215
4 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 217
4.1 Comparison of Performance of Segmentation Network
with and Without Augmented Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . 217
x Contents

4.2 Comparison of Performance of GAN Augmentation with

Other Data Augmentation Techniques . . . . . . . . . . . . . . . . . . . . . . . . . . 218
5 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 219
5.1 Impact of GAN Augmentation on Cardiac MRI Segmentation. . . . . . . 219
5.2 Limitations of the Study . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 219
5.3 Future Directions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 220
6 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 220
7 Future Scope . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 221
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 221
WGAN for Data Augmentation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 223
Mallanagouda Patil, Malini M. Patil, and Surbhi Agrawal
1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 223
1.1 Generative Adversarial Networks (GANs) . . . . . . . . . . . . . . . . . . . . . . 223
1.2 Architecture of GANs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 224
1.3 Probability Theory Behind the Generator and Discriminator . . . . . . . . 227
1.4 Advantages and Limitations of GANs . . . . . . . . . . . . . . . . . . . . . . . . . . 228
2 Wasserstein Generative Adversarial Networks (WGANs) . . . . . . . . . . . . . . 229
2.1 Motivation for WGANs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 231
2.2 Architecture of WGANs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 231
2.3 WGANs for Data Augmentation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 232
3 Pros and Cons of WGANs Over Other Data Augmentation
Techniques . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 235
4 Data Augmentation Using WGANs: A Case Study . . . . . . . . . . . . . . . . . . . 237
5 Conclusion and Future Scope . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 240
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 241
Image Segmentation in Medical Images by Using
Semi-Supervised Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 243
S. Selva Kumar, S. P. Siddique Ibrahim, and S. Kalaivani
1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 243
1.1 Semi-Supervised Learning. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 244
2 Self-Training Semi-Supervised Learning (SSL) Methods . . . . . . . . . . . . . . 245
2.1 Network-Based Cardiac MR Image Segmentation . . . . . . . . . . . . . . . . 245
2.2 Self-Paced and Self-Consistent Co-training Method . . . . . . . . . . . . . . . 246
3 Adversarial Training Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 247
3.1 Deep Adversarial Network (DAN) . . . . . . . . . . . . . . . . . . . . . . . . . . . . 248
3.2 SGNet Image Segmentation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 249
3.3 Multi-path and Progressive Upscaling GAN-Based Method. . . . . . . . . 249
4 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 250
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 250
Generative Adversarial Learning
for Medical Thermal Imaging Analysis

Prasant K. Mahapatra, Neelesh Kumar, Manjeet Singh, Hemlata Saini,

and Satyam Gupta

1 Introduction

Currently, most medical practitioners diagnose disorders using computer-aided

imagery. Generally speaking, low-resolution photos made it difficult to diagnose
several of the disorders. In order to create artificial images as well as their seg-
mented images, the deep convolutional network will be used. These synthesized
photos have a high resolution.
Generative adversarial networks (GANs) have emerged, offering new technolo-
gies and a framework for the use of medical pictures. GANs are quickly becoming
a cutting-edge foundation as a result of achieving increased performances in a num-
ber of medical applications. The technical characteristics of common GAN
approaches utilized in the medical imaging domain are extensively elucidated.
Unsupervised learning is accomplished using sophisticated neural networks called
generative adversarial networks (GANs).

2 What is a GAN (Generative Adversarial Network)?

Generative adversarial networks (GANs), a method for deep learning, allow com-
puters to synthesize new, artificial data from collections of pre-existing data. In
particular, a GAN can produce high-quality data with little to no labeling through
competition between the generator and discriminator networks [1, 2].

P. K. Mahapatra ( ) · N. Kumar · M. Singh · H. Saini · S. Gupta

Biomedical Applications (BMA) Group, CSIR-CSIO, Chandigarh, UT, India
e-mail: [email protected]; [email protected]; [email protected]

© The Author(s), under exclusive license to Springer Nature Switzerland AG 2023 143
A. Solanki, M. Naved (eds.), GANs for Data Augmentation in Healthcare,
https://doi.org/10.1007/978-3-031-43205-7_9
144 P. K. Mahapatra et al.

There are two competing neural network models in GAN. Using the noise vector
(usually a low-dimensional random vector sampled from a normal or uniform dis-
tribution typically between 50 and 512 dimensions, and is randomly generated for
each sample during training) as an input, one creates samples (and so named gen-
erator). The purpose of the noise vector is to introduce randomness into the genera-
tor network and to allow it to produce a diverse set of 2 outputs. By providing
different random vectors as input to the generator network, we can generate a wide
range of new data. In order to ensure that the generated outputs are diverse and not
just copies of the training data, the noise vector is an important factor in the success
of GANs.
The second model, referred to as the discriminator, is given samples from the
generator and training data [3]. The generator has been trained to make images that
closely resemble actual data, while the discriminator has been trained to completely
distinguish between produced data and true data. The adversarial network’s genera-
tor and discriminator compete against one another until symmetry is established, at
which point the network is trained.

2.1 Overview of GAN Structure

GANs compete two neural networks against each other to establish the probability
distribution of a dataset. GAN has two neural networks in it:
• Generator, G.
• Discriminator, D.
A generative network seeks to create artificial images that appear realistic. It
accepts a random vector as input (let us say a 100-dimensional array of numbers
from a Gaussian distribution) and produces a highly realistic image that appears to
be a part of our training set.
On the other hand, the discriminator network accurately determines if an image
is fake (i.e., created by the generator) or real (i.e., direct from the source of the
input). These processes are repeated many times, so that the generator and the
discriminator get better and better at their respective roles with each iteration. Fig. 1
will help you understand how it works.

2.2 Mathematical Equation

The discriminator examines generated images and real images (i.e., training sam-
ples) separately. It determines if the discriminator’s input image is fake or real. The
probability that the input x is real is represented by the output D(x). The discrimina-
tor is trained in the same manner as a deep network classifier. We want D(x) = 1 if
the input is true, that is, image is real. It should be zero if it is a generated image.
Generative Adversarial Learning for Medical Thermal Imaging Analysis 145

Fig. 1 An illustration of a generative adversarial network (GAN)

The discriminator finds qualities that contribute to realistic images through this
method. On the other hand, we want the generator to produce images that are identi-
cal to the true image, with D(x) = 1. It backpropagates the desired value all the way
back to the generator in order to train the generator to generate images that are more
similar to what the discriminator recognizes as real.
The generator becomes stronger at producing realistic images that the discrimi-
nator cannot tell apart from actual ones as the training goes on. The discriminator
also grows stronger at picking up even the smallest variations between the two sorts
of images. The generator eventually creates visuals that are similar to real images as
the two models converge.
The following formula can be used to mathematically explain it [4]:

min max V D,G (1)

G D

V D,G E x~ p ª¬ log D x º¼  E z ~ p z ª log 1  D G z º

data
x
z ¬ ¼

where,
x = real data,
z = noise vector,
G(z;θ_g) = The generator network operates by performing a mapping function from
the noise vector to a synthetic data point, with the parameters of the generator
network denoted as θ_g.
D(x;θ_d) = The discriminator network is a function that receives a data point as its
input and generates a scalar output that denotes whether the input is authentic or
artificial. The parameters of the discriminator network are represented by θ_d.
p_data(x) = The probability distribution of the actual data.
p_z(z) = The probability distribution function of the noise vector z.
The principal objective of GANs is to enhance the discriminative capacity of the
discriminator in discerning genuine and synthesized images. The aforementioned
146 P. K. Mahapatra et al.

process is accomplished via a minimization-maximization methodology, wherein

the generator endeavors to minimize its objective, while the discriminator strives to
maximize it. The primary aim of the objective function is to enhance the likelihood
of detecting artificially generated images as counterfeit and genuine images as
authentic, thereby optimizing the likelihood of observed data. The cross-entropy
function is a widely adopted method for computing the loss in deep learning, which
involves the calculation of p multiplied by the natural logarithm of q. In the context
of real images, the appropriate label to assign is p, which has a value of 1. In the case
of generated images, the label is inverted, specifically by subtracting it from one.
GANs are commonly characterized as a minimax game in which the objective of the
generator is to minimize the value of V, while the discriminator aims to maximize it.

2.3 Major Applications of GAN

Wherever new, plausible data is required, GANs can be used in a wide range of
applications. GANs are specifically used to produce new images and videos.
Image Generation: GANs are commonly used for generating realistic images. For
example, they can be used to generate realistic-looking faces, landscapes, or
even artwork.
Style Transfer: GANs have the potential to facilitate the transfer of style from one
image to another, thereby enabling the creation of an entirely new image that
incorporates the content of one image and the style of another.
Data Augmentation: GANs can be used for generating new data from existing data,
which can be useful for training machine learning models with limited datasets.
Disease Diagnosis and Prediction: GANs can be used for identifying patterns in
medical data and predicting the likelihood of a patient developing a certain
disease.
Medical Image Analysis: GANs can be used for generating synthetic medical
images, such as CT or MRI scans, which can be used for training machine learn-
ing models. GANs can also be used for image segmentation, enhancing the qual-
ity of medical images, and reducing image noise.
Medical Data Augmentation: GANs have the potential to generate synthetic medi-
cal data, thereby serving as a means of augmenting limited datasets and enhanc-
ing the precision of machine learning models.
Overall, GANs have the potential to revolutionize the field of medicine, by
improving the accuracy of disease diagnosis, speeding up drug discovery, and
enabling personalized treatment plans.
Generative Adversarial Learning for Medical Thermal Imaging Analysis 147

3 Self-supervised Generative Adversarial Learning

We will first define the term “Self-Supervised Learning” and then discuss how it
enhances GANs. Self-supervised is the most similar to unsupervised learning when
compared to the prominent families of supervised and unsupervised learning. An
effective method for learning representations from unlabeled data is self-supervised
(SS) learning [5]. Self-supervised learning algorithm learns from data itself, with no
data labeled examples. The algorithm must identify patterns within the dataset to
facilitate the process of acquiring knowledge from it [4].
With the help of pseudo-labels [5], self-supervised approaches enable the classi-
fier to learn better feature representation [6]. These methods specifically suggest
learning the model to recognize a geometric transformation that has been done to
the input image in order to learn an image feature.
There exist several approaches to the implementation of self-supervised learn-
ing. One approach to comprehending the attributes of the data is to employ a neural
network. Subsequently, the neural network can be employed to forecast the designa-
tions of novel data. The identification of data structure can also be accomplished
through the utilization of a Convolutional Neural Network (CNN). A CNN can be
utilized to forecast the outcomes of novel data.
There are some situations where self-supervised learning is superior to super-
vised learning. For example, a CNN trained just through self-supervised learning
can classify images more accurately than a CNN taught only through supervised
learning. This is due to the fact that a CNN that is learned only through supervised
learning is limited by the training set that is made accessible to it. A CNN that has
been trained only through self-supervised learning can understand the data’s struc-
ture from scratch, improving its ability to generalize to new data [6, 7].

4 Conditional and Unconditional GANs

The issues with training GANs will now be linked to self-supervised learning.
GANs are a type of unsupervised generative modelling in which you may just input
data and let the model generate false data from it. Modern GANs, on the other hand,
use a method called conditional-GANs [8], which convert the generative modelling
challenge into a supervised learning task that needs labeled data. For easier genera-
tive modelling, conditional-GANs incorporate class labels within the generator and
discriminator [9].
The term “unconditional GANs” eliminates the necessity for class labels in gen-
erative modelling. This chapter will demonstrate how self-supervised learning tasks
can do away with labeled data when using GANs.
148 P. K. Mahapatra et al.

5 Thermal Imaging Systems

The surface skin temperature [10] can be measured using thermal imaging devices.
These systems might contain a temperature reference source in addition to an
infrared thermal camera [11, 12].
The surface skin temperature of a subject may typically be measured reliably by
thermal imaging devices without being in immediate contact to the subject under
evaluation [13]. Thermal imaging systems [14] have advantages over other tech-
niques of measuring temperature since they require a closer proximity or touch
(Fig. 2).

5.1 Why are Thermal Imaging Devices Beneficial?

There are various advantages of using thermal imaging systems/cameras, which

are listed below:
1. 100% non-invasive: The proximity of the evaluator to the subject under scrutiny
is not a requisite for the operation of thermal imaging devices.
2. Speed and accuracy: In contrast to traditional forehead or oral thermometers that
require close proximity or physical contact with the subject under evaluation,
thermal imaging devices have the potential to provide more rapid and precise
monitoring of surface skin temperature.
3. Flexible and cost-effective diagnostic approach: Thermal camera should be
readily available on the market at reasonable price and the same equipment is
used to record both thermal and geometric data.

Fig. 2 Shows how to set

up thermal imaging
properly to analyze persons
individually
Generative Adversarial Learning for Medical Thermal Imaging Analysis 149

6 Need of Data Augmentation in GANs

Generating annotated medical imaging data is a challenging and costly task. The
creation of deep learning models that can be generalized necessitates the acquisition
of substantial amounts of data. Standard data augmentation is a commonly employed
technique aimed at enhancing the generalizability of machine learning models.
Generative adversarial networks offer a novel method for data augmentation [6, 7].
Insufficient data during the training of GANs often leads to the issue of discrimi-
nator overfitting [15] which in turn causes the training process to diverge. Our pro-
posed approach involves utilizing an adaptive discriminator augmentation technique
that effectively enhances the stability of training in scenarios where data availability
is limited [16, 17]. This approach is applicable for both initial training and does not
necessitate modifications to either loss functions or network architectures. The uti-
lization of unlabeled data holds significant value in the improvement of deep learn-
ing efficacy. GANs are a potent category of neural networks capable of generating
lifelike novel images based on unannotated source images [15, 18]. GANs have
been employed in the past to augment data, including the creation of supplementary
training images for classification purposes and the enhancement of synthetic
images [19].
In order to overcome overfitting and underfitting [2], data augmentation with
GANs was demonstrated to boost model accuracy and decrease model loss, hence
enhancing the generalizability of the model [20] (Fig. 3).

Fig. 3 Thermal image of knee osteoarthritis patient (a) and its augmented GAN-generated
images (b–e)
150 P. K. Mahapatra et al.

7 Improved Medical Image Generation via

Self-supervised Learning

In the domain of deep learning, it is customary to utilize extensive labeled datasets

to effectively train a deep neural network. Various self-supervised learning tech-
niques have been suggested as a means to acquire universal visual characteristics in
an automated manner, thereby circumventing the laborious and time-intensive pro-
cess of manually annotating vast quantities of data. Self-supervised generative
adversarial neural networks, also known as unconditional GANs, are utilized for the
purpose of generating synthetic thermal images.
The widespread use of deep CNNs in computer vision applications can be attrib-
uted to their remarkable ability to extract features from visual data. These applica-
tions include but are not limited to image classification, semantic and instance
segmentation, object recognition, and image captioning. The efficacy of deep learn-
ing models is notably impacted by the quantity of data utilized during the training
process, as they have the ability to expand and enhance in intricacy with the incor-
poration of supplementary training data.

8 Methods

Despite the prevalence of comprehensive color image databases for diverse objects
in the public sphere, there exists a dearth of comparable databases for thermal
images, with either a lack of availability or restricted representation of object
categories. The synthesis of thermal images is of great significance due to the ardu-
ous nature and high expenses associated with obtaining authentic data. The process
of gathering and annotating extensive datasets comprising millions of images is
arduous, costly, and time-intensive [21].
GANs have demonstrated remarkable efficacy in generating diverse images
through the use of pre-existing images and stochastic noise, a widely acknowledged
fact. Currently, unconditional GANs have the ability to generate images that exhibit
a high degree of realism, diversity, and quality.

8.1 Training Dataset

The selection of an appropriate unlabeled dataset is an essential part of transfer

learning via self-supervised pre-training.
Our training dataset [22] is based on the knee areas [23] of the human body
which are captured with a FLIR thermal camera. While diagnosing arthritis, ther-
mography is frequently used to examine deep-bodily joints that are challenging to
evaluate with a standard X-ray [19, 24]. The size of all thermal images is 312 KB
with dimensions of 320 × 240 pixels [25] (Fig. 4).
Generative Adversarial Learning for Medical Thermal Imaging Analysis 151

Fig. 4 Generator and discriminator models used in this technique

8.2 Results and Conclusion

The application of thermal imaging technology [26, 27] is employed for the purpose
of diagnosing infectious skin conditions and investigating a wide range of disorders,
wherein alterations in body temperature may indicate the presence of inflammation
in injured tissues or clinical abnormalities that result in changes in blood circula-
tion [23].
The results of the current study indicate that thermal imaging has the potential to
serve as a dependable diagnostic modality for detecting measurable patterns in skin
temperatures [28]. It has been shown that changes in pain intensity associated with
arthritic, repetitive strain, muscular, and circulatory issues can be correlated with
temperature variance [29–31].
152 P. K. Mahapatra et al.

We believe that this non-intrusive method makes it possible to find the earliest
clinical features, with high reliability [32].

8.3 GAN Results

The GAN generated fake images from the given thermal images of Knee dataset and
comparison of generator & discriminator loss on a trained GAN architecture are
visualised (Figs. 5 and 6).

8.4 Outlook and Conclusions

This chapter has explored various techniques for producing simulated thermal
images using the provided knee dataset. Future research in the field of data augmen-
tation will focus on various topics, including the development of a taxonomy of
augmentation methods. To improve the quality of GAN samples, researchers may
explore novel combinations of meta-learning and data augmentation techniques,
investigate the correlation between data augmentation and classifier architecture,

Fig. 5 (a) Different thermal images of knee pair and its lateral view (right and left); (b) Matrix of
input real image and the GAN-generated fake image matrix
Generative Adversarial Learning for Medical Thermal Imaging Analysis 153

Fig. 6 Comparison of the generator and discriminator loss on a GAN architecture that trained on
knee dataset

and apply these concepts to diverse data types. Furthermore, the integration of inno-
vative data augmentation techniques can enhance the variety and magnitude of the
training dataset, consequently augmenting the efficacy of the GAN model [33].
In our upcoming study, we intend to investigate performance benchmarks for
geometric and color space augmentations on numerous datasets from various image
recognition tasks. To show how well these augmentations work in situations when
there isn’t a lot of data, we are going to impose these dataset’s size restrictions. The
qualities of the temperature profile that is connected with a thermal image have not
yet been investigated while creating synthetic thermal images, which may be a
future course of action.
The GAN framework has undergone several modifications in various research
articles, utilizing diverse network designs, loss functions, evolutionary techniques,
and other methodologies. The study has led to a significant improvement in the
quality of samples generated by GANs. An important avenue for further investiga-
tion pertains to the augmentation of GANs’ sample quality, as well as the assess-
ment of their efficacy across diverse datasets. To advance the exploration of GAN
sample combinatorics, we aim to employ supplementary augmentation techniques,
including the transfer of diverse styles onto GAN-generated samples.
154 P. K. Mahapatra et al.

Future research in generative models with data augmentation should also focus
on StyleGAN2, StyleGAN2-ADA, DiffAugment, and Variational Autoencoder
(VAE). Trying to produce high-resolution outputs from GAN samples is one of the
main challenges. It will be interesting to explore how we might utilize these GAN
networks to produce high-resolution images as a result.

Acknowledgments This work is made possible by the core grant (HCP-0026-3.2) of the Medical
Mission project, which was sponsored by the Government of India, DSIR, Ministry of Science &
Technology, CSIR, India, and CSIR-Central Scientific Instruments Organization, Sector-30,
Chandigarh.

References

1. Goodfellow, I., et al. (2020). Generative adversarial networks. Communications of the ACM,
63(11), 139–144. https://doi.org/10.1145/3422622
2. Tian, L., Wang, Z., Liu, W., Cheng, Y., Alsaadi, F. E., & Liu, X. (2021). A new GAN-based
approach to data augmentation and image segmentation for crack detection in thermal imaging
tests. Cognitive Computation, 13(5), 1263–1273. https://doi.org/10.1007/s12559-021-09922-w
3. Chen, H. (2021). Challenges and corresponding solutions of Generative Adversarial Networks
(GANs): A survey study. Journal of Physics: Conference Series, 1827(1). https://doi.
org/10.1088/1742-6596/1827/1/012066
4. Treneska, S., Zdravevski, E., Pires, I. M., Lameski, P., & Gievska, S. (2022). GAN-based
image colorization for self-supervised visual feature learning. Sensors, 22(4). https://doi.
org/10.3390/s22041599
5. Taherkhani, F., Dabouei, A., Soleymani, S., Dawson, J., & Nasrabadi, N. M. (2021). Self-
supervised Wasserstein pseudo-labeling for semi-supervised image classification. In
Proceedings of the IEEE/CVF conference on computer vision and pattern recognition
(pp. 12267–12277).
6. Araslanov, N., & Roth, S. (2021). Self-supervised augmentation consistency for adapting
semantic segmentation. In Proceedings of the IEEE/CVF conference on computer vision and
pattern recognition (pp. 15384–15394). Available: https://github.com/visinf/da-sac.
7. Song, J., Li, P., Fang, Q., Xia, H., & Guo, R. (2022). Data augmentation by an additional self-
supervised CycleGAN-based for shadowed pavement detection. Sustain., 14(21). https://doi.
org/10.3390/su142114304
8. Elaraby, N., Barakat, S., & Rezk, A. (2022). A conditional GAN-based approach for enhanc-
ing transfer learning performance in few-shot HCR tasks. Scientific Reports, 12(1). https://doi.
org/10.1038/s41598-022-20654-1
9. Gauthier, J. (2014). Conditional generative adversarial nets for convolutional face generation.
Class project for Stanford CS231N: Convolutional neural networks for visual recognition.
Winter Semester, 2014(5), 2.
10. Ludwig, N., Formenti, D., Gargano, M., & Alberti, G. (2014). Skin temperature evalua-
tion by infrared thermography: Comparison of image analysis methods. Infrared Physics &
Technology, 62, 1–6. https://doi.org/10.1016/j.infrared.2013.09.011
11. Snekhalatha, U., Rajalakshmi, T., & Gobikrishnan, M. (2018). Automated segmentation of
knee thermal imaging and X-ray in evaluation of rheumatoid arthritis. International Journal of
Engineering & Technology, 7, 326–330.
12. GiziĔska, M., Rutkowski, R., Szymczak-Bartz, L., Romanowski, W., & StraburzyĔska-
Lupa, A. (2021). Thermal imaging for detecting temperature changes within the rheumatoid
foot. Journal of Thermal Analysis and Calorimetry, 145(1), 77–85. https://doi.org/10.1007/
s10973-020-09665-0
Generative Adversarial Learning for Medical Thermal Imaging Analysis 155

13. Fernández-Cuevas, I., et al. (2015). Classification of factors influencing the use of infrared
thermography in humans: A review. Infrared Physics and Technology, 71, 28–55. https://doi.
org/10.1016/j.infrared.2015.02.007
14. Mishra, P., & Pathak, K. (2019). A research paper on thermal imaging system. [Online].
Available: www.jetir.org.
15. Zhao, S., Liu, Z., Lin, J., Zhu, J. Y., & Han, S. (2020). Differentiable augmentation for data-
efficient GAN training. Advances in Neural Information Processing Systems, 33, 7559–7570.
16. Zhang, X., Wang, Z., Liu, D., & Ling, Q. (2018). Dada: Deep adversarial data augmentation
for extremely low data regime classification. In IEEE International Conference on Acoustics,
Speech and Signal Processing (ICASSP) (pp. 2807–2811). IEEE.
17. Maack, L., Holstein, L., & Schlaefer, A. (2022). GANs for generation of synthetic ultrasound
images from small datasets. Current Directions in Biomedical Engineering, 8(1), 17–20.
https://doi.org/10.1515/cdbme-2022-0005
18. Patel, M., Wang, X., & Mao, S. (2020). Data augmentation with conditional GAN for automatic
modulation classification. In WiseML 2020 – Proceedings of the 2nd ACM workshop on wire-
less security and machine learning (pp. 31–36), doi: https://doi.org/10.1145/3395352.3402622.
19. Mizginov, V. A., Kniaz, V. V., & Fomin, N. A. (2021). A method for synthesizing thermal
images using GAN multi-layered approach. The International Archives of the Photogrammetry,
Remote Sensing and Spatial Information Sciences, 44, 155–162.
20. Shin, H. C., Tenenholtz, N. A., Rogers, J. K., Schwarz, C. G., Senjem, M. L., Gunter, J. L., …
& Michalski, M. (2018). Medical image synthesis for data augmentation and anonymization
using generative adversarial networks. In Simulation and synthesis in medical imaging: Third
international workshop, SASHIMI 2018, Held in Conjunction with MICCAI 2018, Granada,
Spain (pp. 1–11). Springer International Publishing.
21. Selfe, J., Hardaker, N., Thewlis, D., & Karki, A. (2006). An accurate and reliable method
of thermal data analysis in thermal imaging of the anterior knee for use in cryotherapy
research. Archives of Physical Medicine and Rehabilitation, 87(12), 1630–1635. https://doi.
org/10.1016/j.apmr.2006.08.346
22. Cueva, J. H., Castillo, D., Espinós-Morató, H., Durán, D., Díaz, P., & Lakshminarayanan,
V. (2022). Detection and classification of knee osteoarthritis. Diagnostics, 12(10). https://doi.
org/10.3390/diagnostics12102362
23. Bardhan, S., Nath, S., Debnath, T., Bhattacharjee, D., & Bhowmik, M. K. (2022). Designing of
an inflammatory knee joint thermogram dataset for arthritis classification using deep convolu-
tion neural network. Quantitative InfraRed Thermography Journal, 19(3), 145–171. https://
doi.org/10.1080/17686733.2020.1855390
24. Lubkowska, A., & Pluta, W. (2022). Infrared thermography as a non-invasive tool in muscu-
loskeletal disease rehabilitation – The control variables in applicability – A systematic review.
Applied Sciences (Switzerland), 12(9). https://doi.org/10.3390/app12094302
25. Jin, C., Yang, Y., Xue, Z. J., Liu, K. M., & Liu, J. (2013). Automated analysis method for
screening knee osteoarthritis using medical infrared thermography. Journal of Medical and
Biological Engineering, 33(5), 471–477. https://doi.org/10.5405/jmbe.1054
26. Frize, M., Adéa, C., Payeur, P., Gina Di Primio, M. D., Karsh, J., & Ogungbemile, A. (2011).
Detection of rheumatoid arthritis using infrared imaging. In Medical imaging 2011: Image
processing (Vol. 7962, pp. 205–215). SPIE.
27. Umapathy, S., Vasu, S., & Gupta, N. (2018). Computer aided diagnosis based hand thermal
image analysis: A potential tool for the evaluation of rheumatoid arthritis. Journal of Medical
and Biological Engineering, 38(4), 666–677. https://doi.org/10.1007/s40846-017-0338-x
28. Fokam, D., & Lehmann, C. (2019). Clinical assessment of arthritic knee pain by infrared
thermography. Journal of Basic and Clinical Physiology and Pharmacology, 30(3). https://doi.
org/10.1515/jbcpp-2017-0218
29. Snekhalatha, U., Anburajan, M., Sowmiya, V., Venkatraman, B., & Menaka, M. (2015).
Automated hand thermal image segmentation and feature extraction in the evaluation of rheu-
matoid arthritis. Proceedings of the Institution of Mechanical Engineers, Part H: Journal of
Engineering in Medicine, 229(4), 319–331. https://doi.org/10.1177/0954411915580809
156 P. K. Mahapatra et al.

30. Snekhalatha, U., Rajalakshmi, T., Gopikrishnan, M., & Gupta, N. (2017). Computer-based
automated analysis of X-ray and thermal imaging of knee region in evaluation of rheuma-
toid arthritis. Proceedings of the Institution of Mechanical Engineers, Part H: Journal of
Engineering in Medicine, 231(12), 1178–1187. https://doi.org/10.1177/0954411917737329
31. Suma, A. B., Snekhalatha, U., & Rajalakshmi, T. (2016). Automated thermal image segmenta-
tion of knee rheumatoid arthritis. In 2016 International Conference on Communication and
Signal Processing (ICCSP) (pp. 0535–0539). IEEE.
32. Denoble, A. E., Hall, N., Pieper, C. F., & Kraus, V. B. (2010). Patellar skin surface temperature
by thermography reflects knee osteoarthritis severity. Clinical Medicine Insights: Arthritis and
Musculoskeletal Disorders, 3, 69–75. https://doi.org/10.4137/CMAMD.S5916
33. Shorten, C., & Khoshgoftaar, T. M. (2019). A survey on image data augmentation for deep
learning. Journal of Big Data, 6(1). https://doi.org/10.1186/s40537-019-0197-0