See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/341565298

Medical Image Generation using Generative Adversarial Networks

Preprint · May 2020

CITATIONS READS

0 2,750

2 authors:

Nripendra Kumar Singh Khalid Raza

Jamia Millia Islamia Jamia Millia Islamia
8 PUBLICATIONS 168 CITATIONS 155 PUBLICATIONS 1,633 CITATIONS

SEE PROFILE SEE PROFILE

Some of the authors of this publication are also working on these related projects:

Drug repurposing against key COVID-19 Drug Targets using advance Machine Learning based algorithms (CVD/2020/000447) View project

Review Paper View project

All content following this page was uploaded by Khalid Raza on 14 February 2021.

The user has requested enhancement of the downloaded file.

Medical Image Generation Using
Generative Adversarial Networks:
A Review

Nripendra Kumar Singh and Khalid Raza

Abstract Generative adversarial networks (GANs) are unsupervised deep learning

approach in the computer vision community which has gained significant attention
from the last few years in identifying the internal structure of multimodal medical
imaging data. The adversarial network simultaneously generates realistic medical
images and corresponding annotations, which proven to be useful in many cases
such as image augmentation, image registration, medical image generation, image
reconstruction, and image-to-image translation. These properties bring the attention
of the researcher in the field of medical image analysis and we are witness of rapid
adaption in many novel and traditional applications. This chapter provides state-of-
the-art progress in GANs based clinical application in medical image generation and
cross-modality synthesis. The various framework of GANs which gained popularity
in the interpretation of medical images, such as deep convolutional GAN (DCGAN),
Laplacian GAN (LAPGAN), pix2pix, CycleGAN, and unsupervised image-to-image
translation model (UNIT),continue to improve their performance by incorporating
additional hybrid architecture, has been discussed. Further, some of the recent appli-
cations of these frameworks for image reconstruction, and synthesis, and future
research directions in the area have been covered.

Keywords Unsupervised deep learning · Image processing · Medical image

translation

1 Introduction

Medical imaging plays a pivotal role in capturing high-quality images of almost all
the visceral organs like the brain, heart, lungs, kidneys, bones, soft tissues, etc. For
image acquisition, there is a plethora of techniques used by a variety of imaging

N. K. Singh · K. Raza (B)

Department of Computer Science, Jamia Millia Islamia, New Delhi 110025, India
e-mail: [email protected]

© The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2021 77
R. Patgiri et al. (eds.), Health Informatics: A Computational Perspective in Healthcare,
Studies in Computational Intelligence 932,
https://doi.org/10.1007/978-981-15-9735-0_5
78 N. K. Singh and K. Raza

modalities including ultrasonography, computed tomography (CT), positron emis-

sion tomography (PET), and magnetic resonance imaging (MRI). However, the basic
principles behind every modality are different for image acquisition, data processing,
and complexity [1]. For example, images from CT, PET, and MR images differ from
each other in terms of complexity and dimensionality to incorporate modality-specific
information which ultimately assists in better diagnosis. However, these diversi-
ties create a major constrain when it comes to cross-modality image synthesis. For
instance, hybrid imaging involves simultaneous imaging from two modalities like
MRI/PET, CT/PET imaging. The extraction of information of one modality from the
hybrid images is usually a tough exercise.
For automated analysis of the medical image, it needs to meet certain criteria
like high-quality images, preserved low and high-level features, etc. A framework
that performs translation of images from one modality to the other can be very
promising by eliminating the need for multimodality scanning of the patient and
help reduce time and expenditure. Generative adversarial network (GAN) is one
such unsupervised framework that has carried out cross-modality image synthesis
with significant accuracy and reliability [2].
This chapter is organized in the four sections and a discussion at the end. The first
section is an introduction, the second section describes a brief background of GAN
and its variant widely used the reconstruction and cross-modality translation. In the
third section, we elaborate on some powerful variants of the GAN framework which
got popularity in the medical image generation with desired output image resolu-
tion and cross-modality image-to-image translation. The fourth section contributes
application of GAN in the medical image reconstruction and synthesis of different
modality.

2 Generative Adversarial Network

The generative adversarial networks (GAN), introduced by Ian J. Goodfellow and

collaborators in 2014 [3], is one of the recently developed approachesto ‘generative
model’ using a flexible unsupervised deep learning architecture [2]. Generative model
is an important task carried out using unsupervised learning to automatically generate
and learn the patterns in input data so that the model can be utilized for generating
new examples (output) that could have been drawn using the original dataset. The
ability of GAN to generate new content makes it more popular and useful in real-life
image generation.

Vanilla GAN
Vanilla GAN is the initial variant for the synthesis of artificial images proposed by
Goodfellow and colleagues [3]. The building block of this network, shown in Fig. 1,
is generative (G) and discriminative (D) models which use fully connected layers of
the neural network and due to this, it has limited performance. It performs generative
approach for generating samples directly taking input z from noise distribution p(z),
Medical Image Generation Using Generative Adversarial Networks … 79

Fig. 1 Typical structure of vanilla GAN

without any conditional information. The output of the generator is x g ∼ G(z) and at
the same time, instance of the real distribution xr ∼ pdata (x) input to the discriminator
model D, which produces single value output indicating the probability of generated
sample is real or fake. In the case of the real image, the generator gets a reward in terms
of positive gradient learning from discriminator and punishes the generator when the
sample (image) is not close to real. However, the objective of D’s is like a binary
classifier to distinguish pair of real or fake samples, while generator G is trained
much to generate a variety of realistic samples to confuse the discriminator model.
The objective of competing models G&D shown in the following mathematical
representation [3],
   
min max V (D, G) = E xr ∼ pdata(x) log D(xr ) + E x g ∼ pz (z) 1 − log D(G(z)) (1)
G D

The challenges in GANs optimization are as follows:

(i) Mode collapse: A complete mode collapse is not arising commonly but happens
partially, which is the most difficult and unreliable problem to solve in GAN.
As the name suggests, it is a situation when the generator continually produces
a similar image and discriminator unable to consider the difference in generated
samples, and in this way, generator easily fools the discriminator. It restricts
the learning of the generator and focuses on a limited set of images instead
of producing verities of images. The problem of collapse might be arising in
inter-class or intra-class mode collapse.
(ii) Vanishing gradients: When discriminator becomes too strong, then in that situ-
ation, discriminator does not provide enough corresponding gradients as feed-
back to the generator (when gradient loss function (1 − log D(G(z)) ≈ 0, and
learning of the generator stopped) and finally, generator is unable to produce
a competitive sample. Another condition is that when D’s become too weak,
then loss represents nothing to the generator.
80 N. K. Singh and K. Raza

(iii) Problem with counting: Early GANs framework was unable to distinguish
several objects at particular location in generated image, and sometimes, GAN
produce more or lesser number of nose holes, eye’s and ear’s even fails to place
the right location.
(iv) Problem with perspective: Sometimes, GANs are unable to understand the
perspective and failed to differentiate front view and back view objects, hence
GANs could not work well at the conversion of 3D representation into 2D
representation [4].

3 GANs Framework for Medical Image Translation

In the initial setup, GAN was used as fully connected layers and no restrictions
on data generation but, later on, it was replaced by fully convolutional down-
sampling/up-sampling layers and conditional images constraints to get images
with desired properties. Many different variants of the GAN framework were
proposed to meet the desired output, which are Deep Convolutional generative adver-
sarial networks (DCGAN), Laplacian generative adversarial networks (LAPGAN),
pix2pix, CycleGAN, UNIT, CatGAN, BiGAN, InfoGAN, VAEGAN, StyleGAN,
and more. However, our main objective of this paper is to discuss GAN for image
generation framework especially those which are popular among computer vision
scientists for medical image-to-image translation. In this paper, we discuss DCGAN,
LAPGAN, pix2pix, CycleGAN, and UNIT framework.

3.1 DCGAN

Deep convolutional generative adversarial networks (DCGAN) [5] produce better

and stable training results when a fully connected layer is replaced by a fully convo-
lutional layer. The architecture of the generator in DCGAN is illustrated in Fig. 2a. In
the core of the framework, pooling layers replaced with fractional-stride convolutions
that allowed it to learn from random input noise vector by own spatial up-sampling
to generate an image from it. There are two important changes adopted to modify
the architecture of CNN are Batch Normalization (BatchNorm) and LeakyReLU-
activation. BatchNorm [6] for regulating the poor initialization to prevent the deep
generator from mode collapse which is a major drawback in the early GAN frame-
work. LeakyReLU [7] activation introduced at the place of maxout activation(present
in the vanilla GAN) all layers of a discriminator which improve higher-resolution
image output.
Medical Image Generation Using Generative Adversarial Networks … 81

Fig. 2 A schematic representation of a variant of GANs framework for image synthesis. a Represent
architecture of DCGAN, where generator consists convolutional neural network followed by ReLU
activation while discriminator uses another neural network followed by Leaky ReLU activation
b Shows sampling procedure of LAPGAN c shows pix2pix framework takes input xa and xb as
aligned training sample d shows the architecture of CycleGAN which takes input xa and xb as
unaligned training sample e represents the architecture of the UNIT framework that contains two
autoencoders with shared-latent space
82 N. K. Singh and K. Raza

3.2 LAPGAN

It is very difficult to generate a high-resolution image directly from the noise vector.
To solve this problem, Denton et al. [8] proposed Laplacian generative adversarial
networks (LAPGAN) (Fig. 2b) is a stack of conditional GAN model with Laplacian
pyramid representation, each of layers adds higher frequency into a generated image.
The merit of this model is successive sampling procedure used to get full resolution
image, for the training, a set of generative models {G 0 , G 1 , . . . G k } at each level k
of the Laplacian Pyramid L(I) to captures the distribution coefficients h k for given
input images I , h k is calculated by difference of adjacent levels in Gaussian Pyramid
G k (I ) and up-sampled value U (I ), i.e.,

h k = L k (I ) = G k (I ) − U (G k+1 (I )) = Ik − U (Ik+1 ) (2)

At the end of the level, it simply represents a low-frequency residual which is

h k = Ik .
Following equation is used for sampling procedure with initializing I˜k=1 = 0,
      
I˜k = U I˜k=1 + h̃ k = U I˜k=1 + G k Z k , U I˜k=1 (3)

Figure 2b illustrates the sampling procedure for LAPGAN model. Suppose we

need to sample a 64 × 64 image, for the number of layers K = 3 required four
generator models. Sampling started with a noise z 3 and generative model G 3 to
generate I˜3 , which is up-sampled to the next level l2 for the generative model G 2 .
This process is repeated across successive levels to get the final full resolution image
Io .

3.3 Pix2pix

The pix2pix is a supervised image-to-image translation model proposed by Isola

et al. [9] as depicted in Fig. 2c. It has received a multi-domain user acceptance in
the computer vision community for image synthesis, whose merit is to combine the
loss of conditional GAN (CGAN) [10] with L1 regularizer loss, so that network not
only learns the mapping from the input image to output image but also learn the
loss function to generate the image as close to ground truth. The loss of CGAN is
expressed as:
   
L CGAN (G, D) = E x,y log D(x, y) + E x,z log(1 − D(x, G(x, z))) (4)

where z ~ p(z) is random noise. The L1 regularization loss can be computed as,
 
L L1 (G) = E x,y∼Pdata(x,y),z∼P(z) ||y − G(x, z)||1 (5)
Medical Image Generation Using Generative Adversarial Networks … 83

The final objective function can be obtained by combining the above two equations
as:

G ∗ , D ∗ = arg min max L CGAN (G, D) + λL L1 (G) (6)

G D

where λ is a hyper-parameter coefficient introduced to balance the losses. Isola et al.

propose two choices for training the pix2pix model [9], first for generator architecture
based on U-Net [11] have an encoder/decoder with skip connections (concatenate
all channels between two layers), so it allows until a bottleneck layer to gather low-
level information like the location of edges. The second scheme for discriminator
architecture is PatchGAN [12] which tries to classify N × N patches of images
instead of the whole image.

3.4 CycleGAN

The cyclic adversarial generative network (CycleGAN) is proposed to perform

higher-resolution image-to-image translation using unpaired data [13]. Figure 2d
illustrates the architecture of CycleGAN, which preserves the history of input training
image after the cycle of transformation and adding reconstruction loss. It consists
of two generators,G AB : transfer an image of the domain from A to B and another
generator G B A : doing the opposite transformation of G AB . This cyclefacilitates dual
learning to the model. Also, the model consists of two discriminators, namely D A a
and D B that decide the domain of an image. The adversarial loss function for G AB
and D B pair is expressed as,
   
L GAN (G AB , D B ) = E b∼PB (b) log D B (b) + E a∼PA (a) 1 − log(D B (G AB (a))) (7)

And similarly, the adversarial loss for another pair G B A and D A is represented as
L GAN (G B A , D A ). Another loss is a cyclic-consistency loss to minimize the recon-
struction error when an image in one domain translates to another domain and
reverses back to the original domain. The cyclic-consistency loss is represented by
the equation,

L cyc (G AB , G B A ) = E a∼PA (a) [a − G B A (G AB (a))||1 ] + E b∼PB (b) [b − G AB (G B A (b))||1 ]

(8)

After combining the above two Eqs. (7) and (8) to obtain overall loss of the model,
it would be,

L(G AB , G B A , D A , D B ) = L GAN (G AB , D B ) + L GAN (G B A , D A ) + L cyc (G AB , G B A )

(9)
84 N. K. Singh and K. Raza

The objective function of the CycleGAN can be represented by the following

expression,

G ∗AB , G ∗B A = arg min max L(G AB , G B A , D A , D B ) (10)

G AB ,G B A D A ,D B

Even CycleGAN generates high-quality cross-domain medical images, pix2pix

outperforms CycleGAN with a significant margin. Although, both CycleGAN and
pix2pix are unable to construct reverse geometric transformation (for example, Zebra
↔ Horse). Visit CycleGAN project website (https://junyanz.github.io/CycleGAN/)
for more examples and readings.

3.5 UNIT

An unsupervised image-to-image translation model (UNIT) was proposed by Liu

and collaborators in 2017 [14]. The model is hybrid in terms of the sharing of the
weight of VAE (variational autoencoder) to coupled GAN (CGAN) [15]. Assumed
that x1 and x2 be the same input image of different domains X A and X B , then the
encoders E 1 and E 2 share the same latent space, i.e., E 1 (X A ) = E 2 (X B ). The UNIT
framework is depicted in Fig. 2e.
UNIT framework implements the shared-latent space assumption using weight
sharing between last few layers of autoencoders and first few layers of generators.
Due to shared-latent space, the objective function of UNIT is a combination of GAN
and VAE objective function which implies the cycle-consistency constraints [16,
17]. Therefore, the result of processing stream is called cycle-reconstruction stream
represented by the following mathematical equation,

min max L VAE1 (E 1 , G 1 ) + L GAN1 (E 1 , G 1 , D1 ) + L CC1 (E 1 , G 1 , E 2 , D2 )

E 1 ,E 2 ,G 1 ,G 2 D1 ,D2

L VAE2 (E 2 , G 2 ) + L GAN2 (E 2 , G 2 , D2 ) + L CC2 (E 2 , G 2 , E 1 , D1 ) (11)

where L VAE represents objective for minimizing variational upper bond, L GAN is
theobjective function of GAN, and L CC is the objective function like VAE’s to model
the cycle-consistency constraint. Although, the UNIT framework performs superior
to the CGANon MNIST datasets [15], and there is no comparison available with
another unsupervised model such as CycleGAN [13].

4 Applications of GANs in Medical Imaging

Medical imaging exploits GAN in two different ways, which are generative and
discriminative aspects. Generative aspects deal with the generation of new images
Medical Image Generation Using Generative Adversarial Networks … 85

Fig. 3 Application of GAN in medical image synthesis. All the figures are adapted from corre-
sponding articles. a shows MR reconstruction from given reference image [18] b Low dose CT
denoising [19] c shows input brain MRI used to generated equivalent CT image close to ground
truth [20] d generation of synthetic eye fundus image from corresponding synthetic eye vessels [21]

using underlying structural information present in the training data. On the contrary,
descriptive aspects of GAN learns structural feature from the natural or original
images and rule out the anomalies present in the abnormal generated image. Most
of the literature reviewed in this section has applied conditional GAN methods for
image-to-image translation that suffers in certain forms such as under sampling,
noise in the output image, and low spatial resolution. Figure 3 shows the example
of GANs application for the generative and discriminative aspect of medical image
generation.

4.1 Reconstruction

The medical image reconstruction is an essential step to obtain high-quality images

for diagnosis with minimal patient discomfort, but the quality of such images limited
by noise and artifact due to clinical constraints like the concentration of contrast
media, amount of radiation administered to a patient in undergoing contrast MRI and
PET, respectively, during the acquisition of images. Therefore, the objective to get
reduced noise and another factor, analytical and iterative methods of reconstruction
paradigm shifted to data-driven-based machine learning methods [22].
During literature survey, we have observed that pix2pix and CycleGAN frame-
work is frequently used in magnetic resonance imaging (MRI) reconstruction [18],
[23–26], low dose CT denoising [19], [27], [28], optimization of the pre-trained
86 N. K. Singh and K. Raza

network for sharpness detection and highlighting low contrast region in CT image
[29], and synthesizing full dose equivalent PET from low dose concentration using
skip connection introduced in the generator of conditional GAN(cGAN). Similarly,
Liao et al. [30] explore sparse view CBCT reconstruction for artifact reduction. They
propose feature pyramid networks for the specialized discriminator and compute
modulated focus map to reconstruct the output while preserving anatomical struc-
ture. Besides the reconstruction from lower sampling, it must ensure the domain
data accuracy. MR reconstruction also imposes under-sample k-space data in the
frequency domain [19, 31, 32]. In image reconstruction, different types of losses
have been used to distinguish local image structures, such as cycle-consistency and
identity loss together [33] in the denoising of cardiac CT. Wolterinket al. proposed
low dose CT denoising after removing some domain loss but the final result compro-
mised with local image structure [28]. Reconstruction in MR is an exceptional case as
it has a very much characterized forward and backward formula, for example, Fourier
transformation. We summarized some studies related to medical image reconstruc-
tion in Table 1, and various losses to improve reconstruction accuracy described in
Table 5.

4.2 Medical Image Synthesis

GANs framework provides a collective solution for augmenting training samples

with sound results compared to traditional transformation, thus it is widely accepted
for medical image synthesis and successfully overcomes the problem lacking in
the volume of diagnostic imaging dataset of the positive or negative instance of
each pathology. Another problem is lacking expertise in the annotation of diagnostic
images which might be a big hurdle in the selection of supervised methods. Although,
the multiple numbers of healthcare organization across the world and collaborative
effort to build an open-access dataset of different modalities and pathology have
been done, for example, The Cancer Imaging Archive (TCIA), National Biomed-
ical Imaging Archive (NBIA), Radiologist Society of North America (RSNA), and
Biobank. Researchers can access these image datasets with certain constraints.
Unconditional Synthesis: Unconditional synthesis simply generates an image from
the latent space of a real sample before any conditional information. Commonly, GAN
models, DCGAN, PGGAN, and LAPGAN are adopted in medical image synthesis
due to exceptional training stability. Where DCGAN generates limited output image
quality, that could be up to 256 × 256 image size. However, DCGAN has been used
to generate high-quality image samples of lungs nodule and liver lesion which easily
deceive radiologists [41, 42]. Other methods used for generating higher-resolution
images are the iterative method, sharing weights among generators, but the hier-
archical method may not do the same. Progressive growing generative adversarial
networks (PGGAN) performs the progressive growing techniques to get the desired
realistic image. For example, Beers et al. can produce a synthetic image of MRI and
Medical Image Generation Using Generative Adversarial Networks … 87

Table 1 Summary of contributions in the medical image reconstruction over different modalities,
Modalities Methods Losses Remarks References
MRI Pix2pix L1, 2 3D super-resolution from single [18]
low-resolution input image using multi-level
densely connected super-resolution network
(mDCSRN)
MRI Pix2pix L1, 2, 7, 9 Inpainting [22]
MRI Pix2pix L1, 2 Super-resolution [23]
MRI Pix2pix L1, 2 Under-sampled K-space reconstruction for [24]
accelerated MRI scan
MRI Pix2pix L1, 2 Under-sampled K-space [25]
MRI Pix2pix L1, 2,7 Two-stage [26]
MRI Pix2pix L1, 2, 11 Under-sampled K-space [31]
MRI Pix2pix L1, 2, 11 Reconstruction into high-quality image [32]
under sampled K-space
MRI Pix2pix L1, 2, 7, 11 Under-sampled K-space [19]
MRI Pix2pix L1, 2 Motion correction [34]
MRI Pix2pix L1, 2, 7, 10 Directly in complex-valued k-space data [35]
MRI Pix2pix L1, 2, 7, 9 Motion correction [36]
PET cGAN L1, 2 3D high-resolution image synthesizes [29]
equivalent to full dose PET image
CT Pix2pix L1, 7 3D denoising, transfer from 2D [19]
CT Pix2pix L1, 2, 5 Denoising [27]
CT Pix2pix L1, 2 3D denoising [28]
CT Pix2pix L1, 2, 7 Sparse view CT reconstruction [30]
CT Pix2pix L1, 2, 8 3D denoising [37]
CT SGAN L1, 2, 7 Denoising, contrast enhance [38]
CT CycleGAN L1, 2, 10 Super-resolution, denoising [39]
CT CycleGAN L1, 3, 12 Denoising CT [33]
CT Pix2pix L1, 2, 7 Denoising using the adjacent slice [40]
The method for basic training network architecture, or with some modification with losses incurred
(more description about losses described in Table 5) and the last column discuss remark for each
contribution

retinal fundus of size up to 1024 × 1024 using the PGGAN model which is unprece-
dented for the previous model [43]. The summary of articles related to unconditional
synthesis is presented in Table 2.
Conditional Synthesis: Availability of right samples of medical imaging data is
going to be a challenge especially when pathologies are involved. They raise two
factors, like scarcity of the number of cases and large variation in anatomical loca-
tion, appearance, and scale. Therefore, it is useful to synthesize artificially generated
medical images in uncommon conditions by constraints on locations, segmentation
88 N. K. Singh and K. Raza

Table 2 Summarize articles of unconditional synthesis of medical images

Modalities Methods Remark References
CT PGGAN Segmentation mapping using joint learning in [44]
augmenting brain image
CT DCGAN Synthesizing liver lesion of each class using [42]
DCGAN, then classifying different class of
lesion
CT DCGAN Generate realistic lung nodule separately benign [41]
and malignant nodules
MRI Semi-Coupled GAN Two-stage semi-supervised methods for [45]
detection of missing features from cardiac MR
image
MRI LAPGAN Generating synthetic brain MR image slices [46]
MRI DCGAN* Semi-supervised achieve better than fully [47]
supervised learning with labelled and unlabelled
3D image segmentation
MRI DCGAN Manifold learning for image synthesis and [48]
denoising
MRI PGGAN Generating high-resolution Multimodal MR [43]
image of glioma and retinal fundus using
progressive training,
X-ray DCGAN Artificial chest X-ray augmentation and [49]
pathology classification
X-ray DCGAN Semi-supervised learning for abnormal cardiac [50]
classification
Retinal DCGAN Vessel segmentation in the retinal fundus image [51]
Dermo LAPGAN Generating high-resolution skin lesion image [52]

maps or text, etc. Jin et al. [53] augmented the lung CT data set with artificially
synthesize nodules touching the lung border to improve pathological lung segmenta-
tion of CT. An adversarial auto encoder for a conditional segmentation map has been
used to generate a retinal color image from a synthetic vessel tree which is at two-
stage process [21]. Moreover, generating brain MR by conditioned segmentation map
used conditional GAN to learn automatic augmentation as well training samples for
brain tumor segmentation [54], and trained CycleGAN network to correct geometric
distortion in diffusion MR [55]. Some contributions to conditional synthesis are
summarized in Table 3.
Cross-Modality Synthesis: Cross-modality synthesis, for example, creating CT
equivalent image dependents on MR images is esteemed to be helpful for different
reasons. Consider a case study, when two or more imaging modalities say CT and
MR require providing supplementary information in diagnostic planning, in this
case, the separate acquisition is required for the complete diagnosis, which increases
the cost and time in the acquisition. So, cross-modality synthesis provides artifi-
cially generated samples of target modality from available modality with preserving
Medical Image Generation Using Generative Adversarial Networks … 89

Table 3 Summarize articles of conditional image synthesis of different modalities

Modalities Methods Conditional information References
CT Pix2pix VOI with a removed central region [53]
MRI CycleGAN MR [55]
MRI Cascade cGAN Segmentation map [56]
MRI Pix2pix Segmentation map [57]
MRI cGAN Segmentation map [54]
Ultrasound Cascade cGAN Segmentation map [58]
Retinal cGAN Vessel map [59]
Retinal VAE + cGAN Vessel map [21]
Retinal cGAN Vessel map [60]
X-ray Pix2pix Segmentation map [61]
Representing modification in the given method either in network architecture or induced losses

anatomical structure or features, without separate medical image acquisition. Most

of the methods are used in this section are similar to previous sections such as the
CycleGAN-based method used where registration of images is more challenging.
The pix2pix-based method is another well-accepted model used where imaging data
registration ensures the data fidelity. Articles related to cross-modality synthesis
summarized in Table 4. Summary of various losses used in the literature cited and
losses introduced in different variants of GANs framework to get the desired output
result is shown in Tables 4 and 5.

5 Conclusion and Future Research Directions

GANs framework has achieved great success in the field of medical image generation
and image-to-image translation. We have discussed the weightiness of a significant
rise in the study of medical imaging during the past 2–3 years. A detailed litera-
ture survey of GANs in medical imaging reported that about 46% of these articles
are related to cross-modality image synthesis [76]. A large section of research has
focussed on the application of GANs in medical image synthesis of MRI imaging.
The probable reason for the synthesis of MRI images is that it takes longer scan
time for multiple sequence acquisition. Conversely, GAN effectively generates the
next sequence from the acquired one, which saves time slots for another patient. The
second reason may be the large number of MRI data set available in the public domain
allowing researchers to have a surplus sample size for better model training. Further,
a large fraction of studies conducted in the area of reconstruction and segmentation
applications are due to better adversarial training and regulation on the generator’s
output of the GAN model for image-to-image translation framework. Although,
conditional generation provides flexibility over augmentation and high resolution
90 N. K. Singh and K. Raza

Table 4 Summary of articles for cross-modality synthesis among different modality

Modality Methods Losses Remark References
MR ↔ CT Cascade GAN L1, 2, 4 Context-aware network for the [20]
multi-subject synthesis
MR ↔ CT CycleGAN L1, 3 Unpaired training in [62]
synthesizing the cardiac image
MR ↔ CT Cycle GAN L1, 3 Training of unpaired 2D images [63]
to synthesis cross imaging
modality
MR ↔ CT cGAN L1, 2 Brain cancer analyzed to [64]
generate synCT
MR ↔ CT CycleGAN* L1, 3, 6 Generic synthesis of unpaired [65]
3D cardiovascular data
MR ↔ CT CycleGAN* L1, 3, 4 Unpaired training to synthesize [66]
musculoskeletal image
MR ↔ CT Pix2pix L1, 2 Paired training of 2D image to [67]
analyse prostate cancer for the
complete pelvic region
MR ↔ CT CycleGAN L1, 3, 6 Two-stage training and [32]
synthesis for abdominal image
MR ↔ CT 3D U-Net – 3D patch-based network for [68]
pelvic bone synCT generation
CT ↔ MR CycleGAN* L1, 2, 3, 6, 7 A two-step process to synthesis [69]
synMR in lung tumor
segmentation
CT ↔ MR CycleGAN L1, 2 Both paired and unpaired [70]
training for brain tumour
CT ↔ PET FCN + cGAN L1, 2 Synthesize paired training for [71]
liver lesion detection
PET ↔ CT cGAN L1, 2, 7, 9 Paired training for motion [72]
artifact and PET denoising
MR ↔ PET Cascade cGAN L1, 2 Brain anatomical feature from [73]
the sketch-refinement process
used in the synthesis
MR ↔ PET 3D Cycle GAN L1, 2, 3 Two stages paired training for [74]
Alzheimer’s disease diagnosis
PET ↔ MR Pix2pix L1, 2 Paired templet-based training [75]
for brain imaging data
The arrow → represents one-way synthesis, and a dual arrow represents two-way synthesis, *
represents modification in the given method either in network architecture or induced losses
Medical Image Generation Using Generative Adversarial Networks … 91

Table 5 Summary of various losses used in the literature cited and introduced in different variants
of the pix2pix framework to get the desired output result
Abbreviation Losses Remark
L1 L adversarial Discriminators introduced adversarial loss to maximize the
probability of real or fake images
L2 L image Element-wise loss in the structural similarity between real or fake in
the image domain when aligned training is provided
L3 L cycle Loss during cycle transformation to ensure self-similarity when
unaligned training is provided
L4 L gradient Loss in the gradient domain to focus on edges
L5 L sharp Element-wise loss in which low contrast region computed to be
image sharpness using a pre-trained network
L6 L shape Shape loss is also to be segment loss in the reconstruction of the
specified region
L7 L perceptual Element-wise loss to get a visual perception in a feature domain
L8 L structure Structural loss is the patch-wise loss to get a human visual
perception in the image domain
L9 L style-content Style-content loss resembles style and content, where style is a
correlation of low-level features
L10 L self-reg Element-wise loss in identifying among similar structure or in
denoising in the image domain
L11 L frequency Data fidelity loss in the frequency domain (K-space) especially in
MRI reconstructions
L12 L regulation Regulation loss in the generation of image contrast while keeping
the balance across the row and column

for training data. Some studies have synthesized chest X-rays for the classification
of cardiac abnormalities and pathology [49, 50]. However, a very limited number of
studies have been reported for the detection and registration of medical images.
Despite the diverse use of GANs, it has faced many challenges on the way for the
adaptation of generated medical images directly into clinical diagnosis and decision
making. Most of the work for image translation and reconstruction uses traditional
methods of the metric system for quantitative evaluation of proposed methods. Espe-
cially, when GAN incorporate additional loss, there arises difficulty in the optimiza-
tion of the visual standard of an image in the absence of a dedicated reference metric.
Recently, Armanious et al. have proposed MedGAN adopted perceptual study along
with subjective measure, conducted by the domain expert (experienced radiologist)
for an extensive evaluation of GAN generated image, but the downside is that it
bears high-cost, time-consuming and difficult to generalize [72]. So, it is required
to explore the validity of the metrics. Another big challenge is the absence of data
fidelity loss in case of unpaired training. Therefore, it is unable to retain the infor-
mation of the minor abnormality region during the cross-domain image-to-image
translation process. Due to these problems, Cohen et al. suggested that the GANs
generated image should not be straightaway used in clinical practice [77]. Cohen
92 N. K. Singh and K. Raza

et al. further experimented to confirm that unpaired training of CycleGAN subjected

to bias in generated data from the target domain [77]. In a similar study of Mirsky
et al. proven the possibility of intervening 3D medical imaging and bias only exists
when the model was trained with normal standard data and tested with abnormal
data [78].
Finally, we conclude that, even though there are many promising outcomes
announced, the appropriation of GANs in clinical imaging is still in its early stages
and further research in the area is needed to achieve a level of advancements essential
for reliable application of GANs based imaging techniques in the clinical setup.

References

1. Wani, N. & Raza, K. (2018). Multiple Kernel Learning Approach for Medical Image Anal-
ysis. In: Dey N, Ashour A, Shi F, Balas E (eds), Soft Computing Based Medical Image
Analysis, Elsevier, 31-47. https://doi.org/10.1016/B978-0-12-813087-2.00002-6.
2. Raza, K., & Singh, N. K. (2018). A tour of unsupervised deep learning for medical image
analysis (pp. 1–29), 2018, [Online]. Available: http://arxiv.org/abs/1812.07715.
3. Goodfellow, I. J., et al. (2014). Generative adversarial nets. In Advances in neural information
processing systems.
4. Yadav, A., Shah, S., Xu, Z., Jacobs, D., & Goldstein, T. (2018). Stabilizing adversarial nets
with prediction methods. In 6th International Conference on Learning Representations, ICLR
2018 - Conference Track Proceedings.
5. Radford, A., Metz, L., & Chintala, S. (2016). Unsupervised representation learning with deep
convolutional generative adversarial networks. In 4th International; Conference on Learning
Representations ICLR 2016—Conference Track Proceedings (pp. 1–16).
6. Ioffe, S., & Szegedy, C. (2015). Batch normalization: Accelerating deep network training by
reducing internal covariate shift. In 32nd International Conference on Machine Learning, ICML
2015.
7. Maas, A. L., Hannun, A. Y., & Ng, A. Y. (2013). Rectifier nonlinearities improve neural network
acoustic models. In ICML Workshop on Deep Learning for Audio, Speech and Language
Processing.
8. Denton, E., Chintala, S., Szlam, A., & Fergus, R. (2015). Deep generative image models using
a laplacian pyramid of adversarial networks. In Advances in neural information processing
systems.
9. Isola, P., Zhu, J. Y., Zhou, T., & Efros, A. A. (2017). Image-to-image translation with condi-
tional adversarial networks. In Proceedings–30th IEEE Conference on Computer Visualization
Pattern Recognition, CVPR 2017, vol. 2017-January, pp. 5967–5976. https://doi.org/10.1109/
cvpr.2017.632.
10. Mirza, M., & Osindero, S. (2017). Conditional generative adversarial nets (pp. 1–7), [Online].
Available: http://arxiv.org/abs/1411.1784.
11. Ronneberger, O., Fischer, P., & Brox, T. (2015) U-net: Convolutional networks for biomedical
image segmentation. In Lecture notes in computer science (including subseries lecture notes in
artificial intelligence and lecture notes in bioinformatics). https://doi.org/10.1007/978-3-319-
24574-4_28.
12. Li C., & Wand, M. (2016). Precomputed real-time texture synthesis with markovian generative
adversarial networks. In Lecture notes in computer science (including subseries lecture notes in
artificial intelligence and lecture notes in bioinformatics). https://doi.org/10.1007/978-3-319-
46487-9_43.
Medical Image Generation Using Generative Adversarial Networks … 93

13. Zhu, J. Y., Park, T., Isola, P., & Efros, A. A. (2017). Unpaired image-to-image translation using
cycle-consistent adversarial networks. In Proceedings of the IEEE International Conference on
Computer Vision, vol. 2017-October, pp. 2242–2251. https://doi.org/10.1109/iccv.2017.244.
14. Liu, M. Y., Breuel, T., & Kautz, J. (2017). Unsupervised image-to-image translation networks.
In Advances in neural information processing systems.
15. Liu, M. Y., & Tuzel, O. (2016). Coupled generative adversarial networks. Advances in neural
information processing systems, no. Nips, (pp. 469–477).
16. Kim, T., Cha, M., Kim, J., Lee, J. K., & Kim, J. (2017). Learning to discover cross-domain
relations with generative adversarial networks. In 34th International Conference on Machine
Learning (ICML 2017) (vol. 4, pp. 2941–2949).
17. Yi, Z., Zhang, H., Tan, P., & Gong, M. (2017). DualGAN: Unsupervised dual learning for
image-to-image translation. In Proceedings of the IEEE International Conference on Computer
Vision. https://doi.org/10.1109/iccv.2017.310.
18. Chen, Y., Shi, F., Christodoulou, A. G., Xie, Y., Zhou, Z., & Li, D. (2018). Efficient and accu-
rate MRI super-resolution using a generative adversarial network and 3D multi-level densely
connected network. In Lecture notes in computer science (including subseries lecture notes in
artificial intelligence and lecture notes in bioinformatics), 2018. https://doi.org/10.1007/978-
3-030-00928-1_11.
19. Shan, H., et al. (2018). 3-D convolutional encoder-decoder network for low-dose CT via transfer
learning from a 2-D trained network. IEEE Transactions on Medical Imaging, 37, 0001. https://
doi.org/10.1109/TMI.2018.2832217.
20. Nie, D., et al. (2017). Medical image synthesis with context-aware generative adversarial
networks. In Lecture notes in computer science (including subseries lecture notes in arti-
ficial intelligence and lecture notes in bioinformatics), 2017. https://doi.org/10.1007/978-3-
319-66179-7_48.
21. Costa, P., et al. (2018). End-to-end adversarial retinal image synthesis. IEEE Transactions on
Medical Imaging, 37(3), 0001. https://doi.org/10.1109/TMI.2017.2759102.
22. Armanious, K., Mecky, Y., Gatidis, S., & Yang, B. (2019). Adversarial inpainting of medical
image modalities. In ICASSP, IEEE international conference on acoustics, speech and signal
processing—proceedings, 2019. https://doi.org/10.1109/icassp.2019.8682677.
23. Kim, K. H., Do, W. J., & Park, S. H. (2018). Improving resolution of MR images with an
adversarial network incorporating images with different contrast. Medical Physics, 47, 0001.
https://doi.org/10.1002/mp.12945.
24. Shitrit, O., & Riklin Raviv, T. (2017). Accelerated magnetic resonance imaging by adversarial
neural network. In Lecture notes in computer science (including subseries lecture notes in
artificial intelligence and lecture notes in bioinformatics). https://doi.org/10.1007/978-3-319-
67558-9_4.
25. Ran, M., et al. (2019). Denoising of 3D magnetic resonance images using a residual encoder–
decoder Wasserstein generative adversarial network. Medical Image Analysis, 55, 0001. https://
doi.org/10.1016/j.media.2019.05.001.
26. Seitzer, M., et al. (2018). Adversarial and perceptual refinement for compressed sensing MRI
reconstruction. In Lecture notes in computer science (including subseries lecture notes in
artificial intelligence and lecture notes in bioinformatics). https://doi.org/10.1007/978-3-030-
00928-1_27.
27. Yi, X., & Babyn, P. (2018). Sharpness-aware low-dose ct denoising using conditional generative
adversarial network. Journal of Digital Imaging, 55, 0001. https://doi.org/10.1007/s10278-018-
0056-0.
28. Wolterink, J. M., Leiner, T., Viergever, M. A., & Išgum, I. (2017). Generative adversarial
networks for noise reduction in low-dose CT. IEEE Transactions on Medical Imaging, 36,
0001. https://doi.org/10.1109/TMI.2017.2708987.
29. Wang, Y., et al. (2018). 3D conditional generative adversarial networks for high-quality PET
image estimation at low dose. Neuroimage, 174, 0001. https://doi.org/10.1016/j.neuroimage.
2018.03.045.
94 N. K. Singh and K. Raza

30. Liao, H., Huo, Z., Sehnert, W. J., Zhou, S. K., & Luo, J. (2018). Adversarial sparse-view CBCT
artifact reduction. In Lecture notes in computer science (including subseries lecture notes in
artificial intelligence and lecture notes in bioinformatics).
31. Quan, T. M., Nguyen-Duc, T., & Jeong, W. K. (2018). Compressed sensing mri reconstruction
using a generative adversarial network with a cyclic Loss. In IEEE Transactions on Medical
Imaging.
32. Mardani, M., et al. (2019). Deep generative adversarial neural networks for compressive sensing
MRI. IEEE Transactions on Medical Imaging, 38, 0001. https://doi.org/10.1109/TMI.2018.285
8752.
33. Kang, E., Koo, H. J., Yang, D. H., Seo, J. B., & Ye, J. C. (2019). Cycle-consistent adversarial
denoising network for multiphase coronary CT angiography. Med Phys.
34. Oksuz, I. et al. (2018) Cardiac mr motion artefact correction from k-space using deep learning-
based reconstruction. In Lecture notes in computer science (including subseries lecture notes
in artificial intelligence and lecture notes in bioinformatics).
35. Zhang, P., Wang, F., Xu, W., & Li, Y. (2018) Multi-channel generative adversarial network for
parallel magnetic resonance image reconstruction in k-space. In Lecture notes in computer
science (including subseries lecture notes in artificial intelligence and lecture notes in
bioinformatics). https://doi.org/10.1007/978-3-030-00928-1_21.
36. Armanious, K., Gatidis, S., Nikolaou, K., Yang, B., & Kustner, T. (2019). Retrospective correc-
tion of rigid and non-rigid mr motion artifacts using gans. In Proceedings—International
Symposium on Biomedical Imaging, 2019. https://doi.org/10.1109/isbi.2019.8759509.
37. You, C., et al. (2018). Structurally-sensitive multi-scale deep neural network for low-dose CT
denoising. IEEE Access, 6, 0001. https://doi.org/10.1109/ACCESS.2018.2858196.
38. Tang, T. et al. (2018). CT image enhancement using stacked generative adversarial networks and
transfer learning for lesion segmentation improvement. In Lecture notes in computer science
(including subseries lecture notes in artificial intelligence and lecture notes in bioinformatics).
https://doi.org/10.1007/978-3-030-00919-9_6.
39. You, C., et al. (2018) CT Super-resolution GAN constrained by the identical, residual, and
cycle learning ensemble(GAN-CIRCLE). IEEE Transactions on Computational Imaging.
40. Liu, Z., Bicer, T., Kettimuthu, R., Gursoy, D., De Carlo, F., & Foster, I. (2020). TomoGAN: low-
dose synchrotron x-ray tomography with generative adversarial networks: Discussion. Journal
of the Optical Society of America A. Optics and Image Science, 37, 442. https://doi.org/10.
1364/josaa.375595.
41. Chuquicusma, M. J. M., Hussein, S., Burt, J., & Bagci, U. (2018). How to fool radiolo-
gists with generative adversarial networks? A visual turing test for lung cancer diagnosis.
In Proceedings—International Symposium on Biomedical Imaging.
42. Frid-Adar, M., Diamant, I., Klang, E., Amitai, M., Goldberger, J., & Greenspan, H. (2018).
GAN-based synthetic medical image augmentation for increased CNN performance in liver
lesion classification. Neurocomputing, 321, 321–331. https://doi.org/10.1016/j.neucom.2018.
09.013.
43. Beers, A., et al. (2018). High-resolution medical image synthesis using progressively grown
generative adversarial networks, 2018, [Online]. Available: http://arxiv.org/abs/1805.03144.
44. Bowles, C., et al. (2018). GAN augmentation: augmenting training data using generative
adversarial networks, 2018, [Online]. Available: http://arxiv.org/abs/1810.10863.
45. Zhang, L., Gooya, A., & Frangi, A. F. (2017). Semi-supervised assessment of incomplete LV
coverage in cardiac MRI using generative adversarial nets. In Lecture notes in computer science
(including subseries lecture notes in artificial intelligence and lecture notes in bioinformatics),
https://doi.org/10.1007/978-3-319-68127-6_7.
46. Calimeri, F., Marzullo, A., Stamile, C., & Terracina, G. (2017). Biomedical data augmentation
using generative adversarial neural networks. Lecture Notes in Computer Science (including
subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol.
10614, LNCS, no. 690974, pp. 626–634, https://doi.org/10.1007/978-3-319-68612-7_71.
47. Mondal, A. K., Dolz, J., & Desrosiers, C. (2018). Few-shot 3D multi-modal medical image
segmentation using generative adversarial learning (pp. 1–10), 2018, [Online]. Available: http://
arxiv.org/abs/1810.12241.
Medical Image Generation Using Generative Adversarial Networks … 95

48. Plassard, A. J., Davis, L. T., Newton, A. T., Resnick, S. M., Landman, B. A., & Bermudez, C.
(2018). Learning implicit brain MRI manifolds with deep learning (p. 56).
49. Salehinejad, H., Valaee, S., Dowdell, T., Colak, E., & Barfett, J. (2018). Generalization of
deep neural networks for chest pathology classification in x-rays using generative adver-
sarial networks. In ICASSP, IEEE International Conference on Acoustics, Speech and Signal
Processing—Proceedings, (vol. 2018-April, pp. 990–994).
50. Madani, A., Moradi, M., Karargyris, A., & Syeda-mahmood, T. (2018). Semi-supervised
learning with generative adversarial networks for chest x-ray classification with ability of
data domain adaptation. no. Isbi, pp. 1038–1042.
51. Lahiri, A., Jain, V., Mondal, A., Biswas, P. K., & Member, S. Retinal vessel segmentation under
extreme low annotation : a generative adversarial network approach (pp. 1–9).
52. Baur, C., & Navab, N. MelanoGANs : High resolution skin lesion synthesis with GANs.
53. Jin, D., Xu, Z., Tang, Y., Harrison, A. P., & Mollura, D. J. (2018). CT-realistic lung nodule
simulation from 3D conditional generative adversarial networks for robust lung segmentation.
In Lecture notes in computer science (including subseries lecture notes in artificial intelligence
and lecture notes in bioinformatics). https://doi.org/10.1007/978-3-030-00934-2_81.
54. Mok, T. C. W., & Chung, A. C. S. (2019). Learning data augmentation for brain tumor segmen-
tation with coarse-to-fine generative adversarial networks. In Lecture notes in computer science
(including subseries lecture notes in artificial intelligence and lecture notes in bioinformatics).
https://doi.org/10.1007/978-3-030-11723-8_7.
55. Gu, X., Knutsson, H., Nilsson, M., & Eklund, A. (2019). Generating diffusion MRI scalar
maps from T1 weighted images using generative adversarial networks. In Lecture notes in
computer science (including subseries lecture notes in artificial intelligence and lecture notes
in bioinformatics). https://doi.org/10.1007/978-3-030-20205-7_40.
56. Lau, F., Hendriks, T., Lieman-sifry, J., & Sall, S. (2017). ScarGAN: chained generative adver-
sarial networks to simulate pathological tissue on cardiovascular MR scans. In Deep learning
in medical image analysis and multimodal learning for clinical decision support. https://doi.
org/10.1007/978-3-319-67558-9.
57. Shin H. C., et al. (2018). Medical image synthesis for data augmentation and anonymiza-
tion using generative adversarial networks. In Lecture notes in computer science (including
subseries lecture notes in artificial intelligence and lecture notes in bioinformatics).
58. Tom, F., & Sheet, D. (2018). Simulating patho-realistic ultrasound images using deep generative
networks with adversarial learning. In Proceedings–International Symposium on Biomedical
Imaging, 2018. https://doi.org/10.1109/isbi.2018.8363780.
59. Zhao, H., Li, H., Maurer-Stroh, S., & Cheng, L. (2018). Synthesizing retinal and neuronal
images with generative adversarial nets. Medical Image Analysis, 49, 14–26. https://doi.org/
10.1016/j.media.2018.07.001.
60. Iqbal, T., & Ali, H. (2018). Generative adversarial network for medical images (MI-GAN).
Journal of Medical Systems, 49, 14–26. https://doi.org/10.1007/s10916-018-1072-9.
61. Mahapatra, D., Bozorgtabar, B., Thiran, J. P., & Reyes, M. (2018). Efficient active learning
for image classification and segmentation using a sample selection and conditional generative
adversarial network. In Lecture notes in computer science (including subseries lecture notes
in artificial intelligence and lecture notes in bioinformatics).
62. Chartsias, A., Joyce, T., Dharmakumar, R., & Tsaftaris, S. A. (2017) Adversarial image
synthesis for unpaired multi-modal cardiac data. In Lecture notes in computer science
(including subseries lecture notes in artificial intelligence and lecture notes in bioinformatics).
63. Wolterink, J. M., Dinkla, A. M., Savenije, M. H. F., Seevinck, P. R., van den Berg, C. A.
T., & Išgum, I. (2017). Deep MR to CT synthesis using unpaired data. In Lecture notes in
computer science (including subseries lecture notes in artificial intelligence and lecture notes
in bioinformatics). https://doi.org/10.1007/978-3-319-68127-6_2.
64. Emami, H., Dong, M., Nejad-Davarani, S. P., & Glide-Hurst, C. K. (2018). Generating synthetic
CTs from magnetic resonance images using generative adversarial networks. Medical Physics.
65. Zhang, Z., Yang, L., & Zheng, Y. (2018). Translating and segmenting multimodal medical
volumes with cycle- and shape-consistency generative adversarial network. In Proceedings of
the IEEE Computer Society Conference on Computer Vision and Pattern Recognition.
 96 N. K. Singh and K. Raza

66. Hiasa, Y., et al. (2018). Cross-modality image synthesis from unpaired data using cyclegan:
Effects of gradient consistency loss and training data size. In Lecture notes in computer science
(including subseries lecture notes in artificial intelligence and lecture notes in bioinformatics).
https://doi.org/10.1007/978-3-030-00536-8_4.
67. Maspero, M., et al. (2018). Dose evaluation of fast synthetic-CT generation using a generative
adversarial network for general pelvis MR-only radiotherapy. Physics in Medicine & Biology,
63, 185001. https://doi.org/10.1088/1361-6560/aada6d.
68. Florkow M. C., et al. (2020) Deep learning–based MR-to-CT synthesis: The influence of
varying gradient echo–based MR images as input channels. Magnetic Resonance in Medicine.
69. Jiang, J., et al. (2018). Tumor-aware, adversarial domain adaptation from CT to MRI for lung
cancer segmentation. In Lecture notes in computer science (including subseries lecture notes
in artificial intelligence and lecture notes in bioinformatics).
70. Bin Jin, C., et al. (2019) Deep CT to MR synthesis using paired and unpaired data. In Sensors.
Switzerland.
71. Ben-Cohen, A., et al. (2019). Cross-modality synthesis from CT to PET using FCN and
GAN networks for improved automated lesion detection. Engineering Applications of Artificial
Intelligence, 78, 186–194. https://doi.org/10.1016/j.engappai.2018.11.013.
72. Armanious, K., et al. (2020). MedGAN: Medical image translation using GANs. Computerized
Medical Imaging and Graphics.
73. Wei, W., et al. (2018) Learning myelin content in multiple sclerosis from multimodal MRI
through adversarial training. In Lecture notes in computer science (including subseries lecture
notes in artificial intelligence and lecture notes in bioinformatics). https://doi.org/10.1007/978-
3-030-00931-1_59.
74. Pan, Y., Liu, M., Lian, C., Zhou, T., Xia, T., & Shen, D. (2018). Synthesizing missing PET from
MRI with cycle-consistent generative adversarial networks for Alzheimer’s disease diagnosis.
In Lecture notes in computer science (including subseries lecture notes in artificial intelligence
and lecture notes in bioinformatics). https://doi.org/10.1007/978-3-030-00931-1_52.
75. Choi, H., & Lee, D. S. (2018). Generation of structural MR images from amyloid PET: Appli-
cation to MR-less quantification. Journal of Nuclear Medicine, 59, 0001. https://doi.org/10.
2967/jnumed.117.199414.
76. Yi, X., Walia, E., & Babyn, P. (2019). Generative adversarial network in medical imaging: A
review. Medical Image Analysis, 58. https://doi.org/10.1016/j.media.2019.101552.
77. Cohen, J. P., Luck, M., & Honari, S. (2018). Distribution matching losses can hallucinate
features in medical image translation. In Lecture notes in computer science (including subseries
lecture notes in artificial intelligence and lecture notes in bioinformatics). https://doi.org/10.
1007/978-3-030-00928-1_60.
78. Mirsky, Y., Mahler, T., Shelef, I., & Elovici, Y. (2019) CT-GAN: Malicious tampering of 3D
medical imagery using deep learning. In Proceedings of the 28th USENIX Security Symposium.

View publication stats