SPECIAL SECTION ON ARTIFICIAL INTELLIGENCE FOR PHYSICAL-LAYER

WIRELESS COMMUNICATIONS

Received November 19, 2019, accepted December 3, 2019, date of publication December 19, 2019, date of current version January 3, 2020.
Digital Object Identifier 10.1109/ACCESS.2019.2960775

Data Augmentation for Deep Learning-Based

Radio Modulation Classification
LIANG HUANG 1 , (Member, IEEE), WEIJIAN PAN 1 , YOU ZHANG 1,

LIPING QIAN 1,5 , (Senior Member, IEEE), NAN GAO 2 ,

AND YUAN WU 3,4 , (Senior Member, IEEE)
1 College of Information Engineering, Zhejiang University of Technology, Hangzhou 310058, China
2 College of Computer Science and Technology, Zhejiang University of Technology, Hangzhou 310058, China
3 StateKey Laboratory of Internet of Things for Smart City, University of Macau, Macau 999078, China
4 Department of Computer and Information Science, University of Macau, Macau 999078, China
5 National Mobile Communications Research Laboratory, Southeast University, Nanjing 210096, China

Corresponding author: Nan Gao ([email protected])

This work was supported in part by the National Natural Science Foundation of China under Project 61572440 and Project 61702456,
in part by the Zhejiang Provincial Natural Science Foundation of China under Project LR16F010003 and Project LY19F020033, in part by
the Open Research Fund of National Mobile Communications Research Laboratory, Southeast University under Project 2019D11, and in
part by the Research Grant of University of Macau under Project SRG2019-00168-IOTSC.

ABSTRACT Deep learning has recently been applied to automatically classify the modulation categories
of received radio signals without manual experience. However, training deep learning models requires
massive volume of data. An insufficient training data will cause serious overfitting problem and degrade
the classification accuracy. To cope with small dataset, data augmentation has been widely used in image
processing to expand the dataset and improve the robustness of deep learning models. However, in wireless
communication areas, the effect of different data augmentation methods on radio modulation classification
has not been studied yet. In this paper, we evaluate different data augmentation methods via a state-of-
the-art deep learning-based modulation classifier. Based on the characteristics of modulated signals, three
augmentation methods are considered, i.e., rotation, flip, and Gaussian noise, which can be applied in both
training phase and inference phase of the deep learning-based classifier. Numerical results show that all three
augmentation methods can improve the classification accuracy. Among which, the rotation augmentation
method outperforms the flip method, both of which achieve higher classification accuracy than the Gaussian
noise method. Given only 12.5% of training dataset, a joint rotation and flip augmentation policy can achieve
even higher classification accuracy than the baseline with initial 100% training dataset without augmentation.
Furthermore, with data augmentation, radio modulation categories can be successfully classified using
shorter radio samples, leading to a simplified deep learning model and a shorter classification response
time.

INDEX TERMS Data augmentation, deep learning, modulation classification, wireless communication.

I. INTRODUCTION Deep learning-based modulation classification automati-

Benefiting from the improvement of computing power and cally and efficiently classify received signals without prior
big data, deep learning has achieved unprecedented develop- knowledge. Modulation classification is a fundamental step
ment in many applications, i.e., speech and audio process- for many applications in wireless communication sys-
ing [1], natural language processing [2], object detection [3], tems, such as spectrum management in cognitive com-
and so on. In recent years, it also achieves dramatic devel- munication systems [8] and unauthorized signal detection
opment in the field of wireless communications, e.g., mod- in secure communications [9], [10]. Traditional modula-
ulation classification [4], symbol detection [5], end-to-end tion classification method either requires high computational
communication [6], and mobile edge computing [7]. complexity or greatly depends on manual operations [9].
The associate editor coordinating the review of this manuscript and Recently, deep learning is successfully introduced to clas-
approving it for publication was Guan Gui . sify signals [11]–[16], which feeds raw signal data or its

This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see http://creativecommons.org/licenses/by/4.0/
1498 VOLUME 8, 2020
L. Huang et al.: Data Augmentation for Deep Learning-Based Radio Modulation Classification

transforms into a deep neural network and instantly obtains (−1, −1) are generated as shown Fig. 1(b), which are also
the modulation category at the network output. It achieves positively phase shifted. Therefore, for the radio modulation
higher classification accuracy than traditional methods for classification task considered in this paper, rotating the mod-
automatic modulation classification based on expert features ulated radio signal is similar to rotating an image, without
such as higher order cumulants based features [17], while losing features for classification. However, flipping the radio
requiring a little extra computational overhead computation signal generates two new QPSK modulated symbols whose
time. phases are negatively shifted in the clockwise direction,
Although deep learning-based approaches can greatly as shown in Fig. 1(c). Although both rotation and flip aug-
improve the performance of the modulation classifier, mentation methods achieve similar accuracy improvements
it requires a large volume of training radio samples. How- for image classification [22], [23], it is an open question about
ever, in practice, collecting a large amount of high quality which one is preferred for radio modulation classification.
and reliable training radio samples sometimes is costly and After the Gaussian noise augmentation, the image is full
difficult. Data augmentation has been widely used to deal of ’snow’ and the received radio symbols are deviated as
with lack of training data by artificially expanding the train- shown in Fig. 1(d). Can all these three augmentation methods
ing dataset with label preserving transformation. Different improve the classification accuracy for deep learning-based
data augmentation methods have been proposed in the liter- radio modulation classification? To the best of our knowl-
ature, i.e., random cropping, rotation and mirroring in image edge, the effect of different data augmentation methods on
classification [18], [19] and pitch shifting, time stretching radio modulation classification has not been evaluated yet.
and random frequency filtering in speech recognition [20]. In this paper, we study data augmentation methods for deep
For deep learning-based radio modulation classification, data learning-based radio modulation classification. Specifically,
augmentation can improve its invariant, especially for small a state-of-the-art deep learning-based modulation classifier,
radio signal dataset. is used to automatically classify the modulation category of
each radio signal sample. Based on the characteristics of
the modulated signal, we study three augmentation methods,
i.e., rotation, flip, and Gaussian noise. After extensive numer-
ical evaluations on an open radio signal dataset, we obtain the
following contributions:
(1) We propose algorithms to augment radio signals at
both training phase and inference phase of the deep learning
algorithm, which achieves around 2.5% improvement on the
baseline in terms of classification accuracy.
(2) We discover that the rotation augmentation method
outperforms the flip method, both of which achieve higher
classification accuracy than the Gaussian noise method.
(3) We propose a joint augmentation policy with both
FIGURE 1. Different data augmentation methods for both image and rotation and flip methods for insufficient training dataset.
modulated signal: (a) raw data, (b) rotation, (c) flip, (d) Gaussian noise.
Given only 12.5% of training dataset, the joint augmenta-
tion method expands the dataset to be a size of 75% of
Augmenting modulated radio signal is similar to augment the initial dataset and achieves an even higher classification
images as shown in Fig. 1. Specifically, we consider three accuracy than the baseline with 100% training dataset without
basic augmentation methods, i.e. rotation, flip, and Gaussian augmentation.
noise, for both an image and a quadrature phase-shift key- (4) With data augmentation, we successfully classify radio
ing (QPSK) modulated radio signal sample illustrated in samples by using only one half of the sampling points.
constellation diagram. For the image, after rotation or flip Therefore, the deep learning model can be simplified with
augmentation, the same cat is displayed but from differ- a significantly reduced inference complexity. Furthermore,
ent viewpoints. In the constellation diagram of the QPSK in the future field deployment, the modulation category can
modulated radio signal, the black circles indicate four ideal be successfully classified upon receiving only half number of
reference points, and the red crosses are the received sym- radio sampling points, which greatly reduces the classifica-
bols which are shifted due to the imperfection of transmit- tion response time.
ter/receiver hardware and wireless channel [21]. In Fig. 1, The remainder of this paper is organized as follows.
we consider two received symbols with positive phase shift Section II presents related work. Section III provides an
(1, 1) and (−1, 1), which are counter-clockwise shifted overview of the studied radio signal dataset and the deep
from their reference points. In wireless communication, each learning-based modulation classifier. We introduce three data
received symbol will be demodulated and mapped to one augmentation methods in Section IV and propose an algo-
of the reference points based on the transmitted content. rithm to augment signals at both deep learning phases in
After rotation augmentation, two new symbols (−1, 1) and Section V. In Section VI, we present the simulation setup and

VOLUME 8, 2020 1499

 L. Huang et al.: Data Augmentation for Deep Learning-Based Radio Modulation Classification

the final experimental results. We finally conclude this paper lower than on real dataset with the same amount of signal
in Section VII. samples. Therefore, an efficient augmentation method for
insufficient radio signal dataset is still absent.
II. RELATED WORK
A. DEEP LEARNING IN RADIO MODULATION III. PRELIMINARIES
CLASSIFICATION In this section, we introduce the radio signal dataset and the
Deep learning has been applied to automatically classify architecture of the state-of-the-art LSTM model [32], which
radio modulation categories in recent literature. By convert- will be used to evaluate different data augmentation methods
ing radio signals into images, two convolutional neural net- presented in Sec. IV.
work (CNN)-based deep learning models, GoogleNet [24]
and AlexNet [18], originally developed for image classifica- A. RADIO SIGNAL DATASET
tion, are used for modulation classification [13], [14]. The We evaluate the radio signal modulation classification based
modulation classification accuracy is further improved by on an open radio signal dataset, RadioML2016.10a [31].
a modified deep residual network (ResNet) [12], which is The radio signals in the dataset consider sample rate off-
fed with modulated in-phase (I) and quadrature phase (Q) set, center frequency offset, multi-path fading and additive
signals. Considering channel interference, the CNN structure white Gaussian noise. Specifically, there are 220,000 modu-
also achieves a considerable classification accuracy [11]. lated radio signal segments belonging to 11 different mod-
In addition to the CNN-based models, the Long Short-Term ulation categories, i.e., binary phase-shift keying (BPSK),
Memory (LSTM) architecture with time-dependent ampli- QPSK, eight phase-shift keying (8PSK), continuous phase
tude and phase information can achieve the state-of-the-art frequency-shift keying (CPFSK), Gauss frequency-shift key-
classification accuracy [16]. To reduce the training time of ing (GFSK), pulse-amplitude modulation four (PAM4),
deep learning models, different subsampling techniques are quadrature amplitude modulation 16 (QAM16), quadra-
investigated in [15] to reduce the dimensions of input signals. ture amplitude modulation 64 (QAM64), double-sideband
AM (AM-DSB), single-sideband AM (AM-SSB) and wide-
B. DATA AUGMENTATION IN DEEP LEARNING band FM (WB-FM). Each radio signal sample is composed
Data augmentation is widely used in deep learning algo- of 128 consecutive modulated in-phase (I) signal and quadra-
rithms to increase the diversity of training dataset, prevent ture phase (Q) signal. The labels of each signal sample
model overfitting, and improve the robustness of the model. include its value of signal-to-noise ratio (SNR) and its cor-
For image classification tasks, generic data augmentation responding modulation category. There are total 20 differ-
methods include flip, rotation, cropping, color jittering, edge ent SNRs ranging from −20dB to 18dB with a step size
enhancement, and Fancy PCA [22]. Other complex data aug- of 2dB. In the dataset, these 220,000 signal samples are
mentation methods synthesize a new image from two training uniformly distributed among 11 modulation categories and
images [25] or from Generative Adversarial Nets (GAN) [26]. 20 SNRs. In other words, there are 1,000 signal samples for
Although there are many augmentation methods for images, each modulation category at each SNR. In Fig. 2, we plot
AutoAugment [27] is proposed to automatically search for examples of 11 modulation categories in forms of con-
augmentation policies based on the dataset. In addition to stellation diagrams under different SNRs. In the following
images, augmentation methods such as synonym replace- subsection, we introduce a deep learning algorithm which
ment, random insertion, random swap, and random deletion automatically predicts the radio’s modulation category based
are used for text classification [28], where the same accuracy on its raw I/Q signals.
as normal in all training data is achieved when only half of
the training data is available. For speech recognition tasks, B. LSTM NETWORK ARCHITECTURE
training audio is augmented by changing the audio speed [29], LSTM is a special category of Recurrent neural network
warping features, masking blocks of frequency channels, and (RNN), which is widely used to process time series data.
masking blocks of time steps [30]. Benefited from a specific LSTM memory cell mechanism,
There are few related works on data augmentation for radio LSTM effectively solves the exploding and vanishing gradi-
modulation classification in the literature. The most related ent problem of traditional RNN during training process and
work is a GAN based data augmentation method proposed learns long-term dependencies in sequential data. The LSTM
in [14]. The authors first converted the signal samples into memory cell mainly consists of a forget gate, an input gate
Contour Stellar Images which were further used to train the and a update gate [33], which implement selective retention
GAN network so as to generate new signal training samples. and discard of input information.
With GAN-based augmentation, the modulation classifica- The LSTM network takes each data sample with con-
tion accuracy is improved by no more than 6%. However, secutive modulated in-phase (I) and quadrature phase (Q)
training GAN network still requires sufficient signal samples signals as input and maps them to a specific modulation
to guarantee the convergence. Moreover, as reported in [14], category, whose architecture is shown in Fig. 3. Specifically,
the classification accuracy based on augmented dataset is the modulated I/Q signals are first converted into amplitudes

1500 VOLUME 8, 2020

L. Huang et al.: Data Augmentation for Deep Learning-Based Radio Modulation Classification

FIGURE 2. Constellation diagrams of 11 modulated signals [31] under different SNRs.

and alleviates overfitting. In this section, we describe in

detail three data augmentation methods for modulation signal
recognition, including rotation, flip, and Gaussian noise. The
dataset is expanded by a scale factor N .

A. ROTATION
By rotating a modulated radio signal (I , Q) around its origin,
we obtain augmented signal sample (I 0 , Q0 ) as follows:

cos θ −sinθ
 0    
I I
= ,
Q0 sin θ cosθ Q

where θ is the angle of rotation. In this paper, the radio signal

is rotated in the counter-clockwise direction by 0, π /2, π,
FIGURE 3. The architecture of LSTM network. and 3π /2. In Fig. 4(a), we plot the constellation diagram of

and phases [16], as:

( p
A = I 2 + Q2
φ = arc tan(Q/I ),
where A and φ represent the amplitude and phase of the
modulated signal, respectively. The obtained signals are then
fed into a two-layer LSTM network to extract characteristic
features, where each layer has 128 LSTM cells. Finally,
a fully connected layer with Softmax function is used to
map the radio signal sample to one of these 11 modulation
categories. Adam optimizer [34] with dynamic learning rate
is used to minimize the cross-entropy loss as follows:
K
X
`=− yk log(ŷk ),
k=1
where K is the number of classes, yk represents the ground
truth label, and ŷk denotes the probability that the input
sample will be predicted as k−th class.

IV. DATA AUGMENTATION METHODS

Data augmentation is a method widely used in deep learning FIGURE 4. Constellation diagram of an QPSK radio signal sample with
because it improves the generalization ability of the model different data augmentation methods.

VOLUME 8, 2020 1501

 L. Huang et al.: Data Augmentation for Deep Learning-Based Radio Modulation Classification

an QPSK sample where one set of raw data is augmented into B. TEST-TIME AUGMENTATION
four radio signal samples. Test-time augmentation fuses features of all augmented radio
signal samples in inference phase. In the inference phase, one
B. FLIP radio signal sample (I , Q) in the test dataset is augmented
For a given modulated radio signal (I , Q), we define the into N samples {(I 0 , Q0 )n |n ∈ N }. Then each augmented
horizontal flip by switching the I value to its opposite, as: sample (I 0 , Q0 )n is fed into the LSTM network, and we obtain
 0    a vector of corresponding predicted probabilities ŷnk . The
I −I predicted modulation category is decided through summing
= ,
Q0 Q the predicted probabilities ŷnk over all N augmented samples
and choosing the one with maximum conference [35], as:
and define the vertical flip by switching the Q value to its
N
opposite, as: X
arg max ŷnk .
 0    1≤k≤K n=1
I I
= ,
Q0 −Q C. TRAIN-TEST-TIME AUGMENTATION
to augment the radio signals. We can perform horizontal flip, Train-test-time augmentation conducts both train-time aug-
vertical flip, or both flips at the same time such that the mentation and test-time augmentation, where both training
signal dataset is expanded by a scale factor N = 4, as shown and test datasets are augmented and expanded by a factor N .
in Fig. 4(b).

C. GAUSSIAN NOISE
By adding a Gaussian noise N (0, σ 2 ) to the modulated
radio signal (I , Q), we obtain the augmented signal sample
(I 0 , Q0 ) as:
 0   
I I
= + N (0, σ 2 ),
Q0 Q

where σ 2 is the variance of noise. In Fig. 4(c), we show the

augmented signal samples by adding Gaussian noise with
different standard deviations σ = 0, σ = 0.0005 σ = 0.001
and σ = 0.002. For each data augmentation method, the orig-
inal radio signal dataset is expanded by a default scale factor
N = 4, as illustrated in Fig. 4. Note that the Gaussian noise
FIGURE 5. Classification accuracy under different augmentation times.
data augmentation is supposed to significantly expand the
dataset by choosing enough different values of σ . However,
in the next section, we show that the Gaussian noise data
augmentation is not preferred for radio data augmentation.

V. DATA AUGMENTATION TIME

The execution of a deep learning algorithm includes train-
ing phase and inference phase. Data augmentation can be
performed in both phases, resulting in three possible com-
binations of augmentations, i.e., test-time augmentation,
train-time augmentation, and train-test-time augmentation.

A. TRAIN-TIME AUGMENTATION
Train-time augmentation performs data augmentation during
the training stage of the model. That is the training dataset
is augmented and expanded by a scale factor N while the
test dataset remains the same. Taking the rotation data aug- FIGURE 6. Classification accuracy under different augmentation methods.
mentation as an example, the training dataset is expanded
from 110,000 radio signal samples to 440,000 samples after In Fig. 5, we numerically study the performance of the
train-time augmentation. In general, a larger size of training data augmentation at different phases, where the rotation
dataset leads to a higher modulation classification accuracy. augmentation with a scale factor N = 4 is considered.

1502 VOLUME 8, 2020

L. Huang et al.: Data Augmentation for Deep Learning-Based Radio Modulation Classification

FIGURE 7. Confusion matrices under different data augmentation methods with 100% training dataset when SNR is −2dB.

FIGURE 8. Confusion matrices under different data augmentation methods with 100% training dataset when SNR is 18dB.

Comparing with the baseline without augmentation, augmen-

tations at different phases all improve the classification accu-
racy when the SNR is greater than −10 dB. The train-time
augmentation achieves better performance than test-time aug-
mentation, and the train-test-time augmentation generates the
highest accuracy. Specifically, comparing with the baseline,
the train-test-time augmentation improves the modulation
classification accuracy by 8.87% when SNR is −6dB and by
about 2.2% when SNR is greater than 4 dB. In the following
numerical studies, we use the train-test-time augmentation by
default.

VI. AUGMENTATION PERFORMANCE

In this section, we numerically study the performance FIGURE 9. Classification accuracy versus different data augmentation
of different radio data augmentation methods in terms methods under different SNRs with 12.5% training dataset.

of modulation classification accuracy. The open dataset,

RadioML2016.10a, is divided equally into a training dataset
and a test dataset, each containing 110,000 radio signal A. AUGMENTATIONS ON FULL DATASET
samples. In order to avoid overfitting, we set dropout rate In Fig. 6, we study the modulation classification accuracies of
to be 0.5 at both two LSTM layers. The number of train- the LSTM model after deploying all three data augmentation
ing epoch is 80 and the mini-batch size is 128. The value methods presented in Sec. VI. Comparing with the baseline
of the learning rate is initially set as 0.001 and is halved without augmentation, all augmentation methods improve the
when the training accuracy is not improved during three classification accuracy when the SNR is greater than −10dB,
consecutive epochs. The model is implemented based on especially for the rotation data augmentation and flip data
PyTorch [36]. augmentation. In particular, the rotation data augmentation

VOLUME 8, 2020 1503

 L. Huang et al.: Data Augmentation for Deep Learning-Based Radio Modulation Classification

FIGURE 10. Confusion matrices under different data augmentation methods with 12.5% training dataset when SNR is 18dB.

method achieves the greatest improvement by 8% when SNR data augmentation methods, the classification accuracy is
is between −6dB and −2dB and by about 2% at higher SNR improved. As expected, both the rotation augmentation and
(≥4dB). Meanwhile, the Gaussian noise data augmentation the flip augmentation outperform the Gaussian noise data
performs better at lower SNR when it is between −16dB augmentation. Interestingly, while the training sub-dataset is
and −10dB. Intuitively, adding Gaussian noise reduces the expanded by a scale factor N = 4 after augmentation, in the
SNR of the original data sample which in turn generates more same size of 50% of the initial dataset, the rotation/flip aug-
signal samples with low SNR. However, the improvement is mentation achieves a higher classification accuracy, around
trivial since the resulting classification accuracy is too small, 0.04%-4.03%, than the baseline by training the LSTM with
less than 2% when SNR is smaller than 10 dB. Therefore, 50% of the initial training dataset without augmentation.
rotation data augmentation and flip data augmentation are We further consider a joint augmentation policy with both
more preferred for radio signals in modulation classification. rotation and flip methods, which expands the dataset by a
To further evaluate the improvements of different aug- scale factor N = 6 (with 2 redundant augmented radio signal
mentation methods on classification accuracy, we present the samples) as shown in Fig. 4(a-b). After this joint augmenta-
corresponding confusion matrices these at low SNR (−2dB) tion, the size of the training dataset is expanded from 12.5% to
and high SNR (18dB) in Fig. 7 and Fig. 8, respectively. Most be 75% of the initial training dataset. Interestingly, we obtain
values at diagonal entries of these matrices are increased after similar classification accuracies at different SNRs as the
argumentation, which means the modulation classification baseline with 100% training dataset without augmentation,
accuracy is improved. Specifically, the proposed augmen- as plotted in Fig. 9. Note that such a classification accuracy
tation methods successfully reduce the confusion between is achieved by using 25% less training data.
QAM16 and QAM64 and solve the short-time observation To further evaluate the advantages of joint rotation and
problem presented in [37]. At low SNR, the LSTM model flip augmentation, we present confusion matrices in differ-
is difficult to classify 8PSK and QPSK, whose classification ent augmentation methods with 12.5% training dataset at
accuracy is greatly improved after rotation augmentation as 18dB in Fig. 10. When training dataset is insufficient, it is
shown in Fig. 7. At high SNR, the accuracy of the LSTM difficult to classify BPSK, WBFM, QAM16 and QAM64,
model is mainly limited by the confusion between AM-DSB whose classification accuracies are significantly improved
and WBFM, which dues to frequent radio samples without after joint augmentation. Specifically, in reducing the con-
information in the dataset [37]. In general, rotation and flip fusion between QAM16 and QAM64, the joint augmentation
achieve better classification accuracy than Gaussian noise for performs better than both the rotation augmentation and the
all modulation categories. flip augmentation.
We have also evaluated another joint augmentation with
all three augmentation methods. However, adding Gaussian
B. AUGMENTATIONS ON PARTIAL DATASET noise method to the joint rotation and flip augmenta-
In Fig. 9, we further study the performance of different tion slightly reduces the classification accuracy. Therefore,
data augmentation methods with insufficient training dataset. we conclude that both rotation and flip methods are preferred
To form new training sub-dataset, we randomly sample partial for radio data augmentation and they can be jointly applied to
radio signal samples from the initial 110,000 radio signal further improve the augmentation performance.
training samples, i.e., 12.5% of the initial training dataset.
Then, the LSTM network is trained by feeding the obtained
training sub-dataset and is tested with the initial 110,000 radio C. AUGMENTATIONS ON SHORT SAMPLE
signal testing samples. Note that 12.5% of the training dataset We further evaluate data augmentation methods for modu-
is insufficient to train the LSTM network, resulting a low lated radio signals with fewer sampling points. We halve each
modulation classification accuracy around 45% under high original 128-point radio signal sample into two new samples
SNR, as shown in Fig. 9. After deploying different radio and obtained a new dataset consisting of 440,000 entries

1504 VOLUME 8, 2020

L. Huang et al.: Data Augmentation for Deep Learning-Based Radio Modulation Classification

of 64-point radio signal samples. Similar to previous evalua- initial training dataset, the joint augmentation method
tions, we randomly choose half of them to the LSTM network, expands the dataset to be a size of 75% of the initial dataset
which is further tested with the remaining half dataset. With and obtains even higher than the baseline with 100% training
a shorter radio signal sample, the number of LSTM cells datasets without augmentation. Furthermore, after deploying
in each LSTM layer in Fig. 3 is reduced from 128 to 64, data augmentation, a radio sample can be classified based
resulting a simpler inference model. Specifically, the number on only one half of the radio sampling points, resulting in
of parameters of the LSTM network is reduced from 201.1K a simplified deep learning model and a shorter classification
to 54.1K and the inference complexity in FLOPs (floating- response time.
point operations) is reduced from 2.8K to 1.4K.
REFERENCES
[1] T. Afouras, J. S. Chung, A. Senior, O. Vinyals, and A. Zisserman, ‘‘Deep
audio-visual speech recognition,’’ IEEE Trans. Pattern Anal. Mach. Intell.,
to be published, doi: 10.1109/TMC.2019.2928811.
[2] M. M. Lopez and J. Kalita, ‘‘Deep learning applied to NLP,’’ 2017,
arXiv:1703.03091. [Online]. Available: https://arxiv.org/abs/1703.03091
[3] X. Li, L. Zhao, L. Wei, M.-H. Yang, F. Wu, Y. Zhuang, H. Ling, and
J. Wang, ‘‘DeepSaliency: Multi-task deep neural network model for
salient object detection,’’ IEEE Trans. Image Process., vol. 25, no. 8,
pp. 3919–3930, Aug. 2016.
[4] A. Ali and F. Yangyu, ‘‘Automatic modulation classification using deep
learning based on sparse autoencoders with nonnegativity constraints,’’
IEEE Signal Process. Lett., vol. 24, no. 11, pp. 1626–1630, Nov. 2017.
[5] H. Ye, G. Y. Li, and B.-H. Juang, ‘‘Power of deep learning for channel esti-
mation and signal detection in OFDM systems,’’ IEEE Wireless Commun.
Lett., vol. 7, no. 1, pp. 114–117, Feb. 2018.
[6] A. Anderson, S. R. Young, T. P. Karnowski, and J. M. Vann, ‘‘Deepmod:
An over-the-air trainable machine modem for resilient PHY layer commu-
nications,’’ in Proc. IEEE Mil. Commun. Conf., Oct. 2018, pp. 213–218.
[7] L. Huang, S. Bi, and Y. J. Zhang, ‘‘Deep reinforcement learning
FIGURE 11. Classification accuracy versus different data augmentation for online computation offloading in wireless powered mobile-edge
methods under different SNRs with 64-length signal. computing networks,’’ IEEE Trans. Mobile Comput., to be published,
doi: 10.1109/TMC.2019.2928811.
In Fig. 11, we evaluate modulation classifications with [8] A. Ali and W. Hamouda, ‘‘Advances on spectrum sensing for cognitive
radio networks: Theory and applications,’’ IEEE Commun. Surveys Tutr.,
64-point radio samples. Without augmentation, 64-point vol. 19, no. 2, pp. 1277–1304, 2nd Quart., 2016.
modulated radio samples always lead to lower classification [9] O. A. Dobre, A. Abdi, Y. Bar-Ness, and W. Su, ‘‘Survey of automatic mod-
accuracy than the baseline with 128-point, around an 8% ulation classification techniques: Classical approaches and new trends,’’
IET Commun., vol. 1, no. 2, pp. 137–156, Apr. 2007.
reduction when SNR is greater than 0 dB. The classification [10] H. S. Jang and B. C. Jung, ‘‘Performance analysis of reactive symbol-
accuracy is improved after deploying either rotation or flip level jamming techniques,’’ IEEE Trans. Veh. Technol., vol. 67, no. 12,
augmentation. Especially, the joint rotation and flip augmen- pp. 12432–12437, Dec. 2018.
[11] P. Triantaris, E. Tsimbalo, W. H. Chin, and D. Gündüz, ‘‘Automatic mod-
tation can achieve 1% higher classification accuracy than the
ulation classification in the presence of interference,’’ in Proc. Eur. Conf.
baseline under high SNR. Therefore, with data augmentation, Netw. Commun., Jun. 2019, pp. 549–553.
the radio signal modulations can be successfully classified [12] T. J. O’Shea, T. Roy, and T. C. Clancy, ‘‘Over-the-air deep learning based
upon receiving only half number of sampling points, which radio signal classification,’’ IEEE J. Sel. Topics Signal Process., vol. 12,
no. 1, pp. 168–179, Feb. 2018.
significantly reduces the classification response time. [13] S. Peng, H. Jiang, H. Wang, H. Alwageed, Y. Zhou, M. M. Sebdani, and
Y.-D. Yao, ‘‘Modulation classification based on signal constellation dia-
VII. CONCLUSION grams and deep learning,’’ IEEE Trans. Neural Netw. Learn. Syst., vol. 30,
no. 3, pp. 718–727, Mar. 2019.
In this paper, we studied radio data augmentation methods for [14] B. Tang, Y. Tu, Z. Zhang, and Y. Lin, ‘‘Digital signal modulation classifica-
deep learning-based modulation classification. Specifically, tion with data augmentation using generative adversarial nets in cognitive
three typical augmentation methods, i.e., rotation, flip, and radio networks,’’ IEEE Access, vol. 6, pp. 15713–15722, 2018.
[15] S. Ramjee, S. Ju, D. Yang, X. Liu, A. E. Gamal, and Y. C. Eldar,
Gaussian noise, were studied based on a well-known LSTM ‘‘Fast deep learning for automatic modulation classification,’’ 2019,
model. We first studied radio data augmentations at training arXiv:1901.05850. [Online]. Available: https://arxiv.org/abs/1901.05850
and inference phases and revealed that train-test-time aug- [16] S. Rajendran, W. Meert, D. Giustiniano, V. Lenders, and S. Pollin, ‘‘Deep
learning models for wireless signal classification with distributed low-
mentation achieves the highest accuracy. Then, we numeri- cost spectrum sensors,’’ IEEE Trans. Cogn. Commun. Netw., vol. 4, no. 3,
cally evaluated all three augmentation methods based on the pp. 433–445, Sep. 2018.
full and partial training dataset. All numerical results show [17] T. J. O’Shea, J. Corgan, and T. C. Clancy, ‘‘Convolutional radio modulation
recognition networks,’’ in Proc. Int. Conf. Eng. Appl. Neural Netw., 2016,
that both the rotation and the flip methods achieve higher clas-
pp. 213–226.
sification accuracy than the Gaussian noise method and the [18] A. Krizhevsky, I. Sutskever, and G. E. Hinton, ‘‘Imagenet classification
rotation method achieves the highest accuracy. Meanwhile, with deep convolutional neural networks,’’ in Proc. Adv. Neural Inf. Pro-
a joint augmentation policy with both rotation and flip meth- cess. Syst. (NIPS), 2012, pp. 1097–1105.
[19] K. He, X. Zhang, S. Ren, and J. Sun, ‘‘Deep residual learning for image
ods can further improve the classification accuracy, espe- recognition,’’ in Proc. IEEE Comput. Soc. Conf. Comput. Vis. Pattern
cially with insufficient training samples. Given only 12.5% of Recognit, Jun. 2016, pp. 770–778.

VOLUME 8, 2020 1505

 L. Huang et al.: Data Augmentation for Deep Learning-Based Radio Modulation Classification

[20] J. Salamon and J. P. Bello, ‘‘Deep convolutional neural networks and WEIJIAN PAN received the B.E. degree from the
data augmentation for environmental sound classification,’’ IEEE Signal College of Mechanical and Electrical Engineer-
Process. Lett., vol. 24, no. 3, pp. 279–283, Mar. 2017. ing, China Jiliang University, Hangzhou, China,
[21] K. Sankhe, M. Belgiovine, F. Zhou, S. Riyaz, S. Ioannidis, and in June 2018. He is currently pursuing the master’s
K. Chowdhury, ‘‘ORACLE: Optimized radio classification through con- degree with the Information Engineering College,
volutional neural networks,’’ in Proc. IEEE INFOCOM, Apr./May 2019, Zhejiang University of Technology. His research
pp. 370–378. interests are deep learning and pattern recognition.
[22] L. Taylor and G. Nitschke, ‘‘Improving deep learning with generic data
augmentation,’’ in Proc. IEEE Symp. Ser. Comput. Intell., Nov. 2018,
pp. 1542–1547.
[23] J. Shijie, W. Ping, J. Peiyi, and H. Siping, ‘‘Research on data augmentation
for image classification based on convolution neural networks,’’ in Proc.
Chin. Automat. Congr., Oct. 2017, pp. 4165–4170.
[24] C. Szegedy, W. Liu, Y. Jia, P. Sermanet, S. Reed, D. Anguelov, D. Erhan, YOU ZHANG received the B.E. degree in electri-
V. Vanhoucke, and A. Rabinovich, ‘‘Going deeper with convolutions,’’ in cal engineering and automation from the Luoyang
Proc. IEEE Conf. Comput. Vis. Pattern Recognit., Jun. 2015, pp. 1–9. Institute of Science and Technology, China,
[25] H. Inoue, ‘‘Data augmentation by pairing samples for images in June 2018. He is currently pursuing the mas-
classification,’’ 2018, arXiv:1801.02929. [Online]. Available: ter’s degree with the Information Engineering
https://arxiv.org/abs/1801.02929 College, Zhejiang University of Technology. His
[26] A. Antoniou, A. Storkey, and H. Edwards, ‘‘Data augmentation genera- research interests are deep learning and signal
tive adversarial networks,’’ 2017, arXiv:1711.04340. [Online]. Available: communication.
https://arxiv.org/abs/1711.04340
[27] E. D. Cubuk, B. Zoph, D. Mane, V. Vasudevan, and Q. V. Le,
‘‘Autoaugment: Learning augmentation policies from data,’’ 2018,
arXiv:1805.09501. [Online]. Available: https://arxiv.org/abs/1805.09501
[28] J. W. Wei and K. Zou, ‘‘EDA: Easy data augmentation techniques for boost- LIPING QIAN received the Ph.D. degree in infor-
ing performance on text classification tasks,’’ 2019, arXiv:1901.11196. mation engineering from The Chinese University
[Online]. Available: https://arxiv.org/abs/1901.11196 of Hong Kong, in 2010. She is currently a Profes-
[29] T. Ko, V. Peddinti, D. Povey, and S. Khudanpur, ‘‘Audio augmentation for sor with the College of Information Engineering,
speech recognition,’’ in Proc. Annu. Conf. Int. Speech Commun. Assoc., Zhejiang University of Technology, China. She is
2015. also with the National Mobile Communications
[30] D. S. Park, W. Chan, Y. Zhang, C.-C. Chiu, B. Zoph, E. D. Cubuk, and
Research Laboratory, Southeast University, China.
Q. V. Le, ‘‘Specaugment: A simple data augmentation method for auto-
Her research interests include wireless communi-
matic speech recognition,’’ 2019, arXiv:1904.08779. [Online]. Available:
https://arxiv.org/abs/1904.08779
cation and networking, resource management in
[31] T. J. O’shea and N. West, ‘‘Radio machine learning dataset generation with wireless networks, the massive IoTs, mobile edge
gnu radio,’’ in Proc. GNU Radio Conf., 2016, vol. 1, no. 1. computing, emerging multiple access techniques, and machine learning ori-
[32] S. Hochreiter and J. Schmidhuber, ‘‘Long short-term memory,’’ Neural ented toward wireless communications. She was a co-recipient of the IEEE
Comput., vol. 9, no. 8, pp. 1735–1780, 1997. Marconi Prize Paper Award in Wireless Communications, in 2011.
[33] F. A. Gers, J. Schmidhuber, and F. Cummins, ‘‘Learning to forget:
Continual prediction with LSTM,’’ in Proc. ICANN, vol. 2, Sep. 1999,
pp. 850–855.
[34] D. P. Kingma and J. Ba, ‘‘Adam: A method for stochastic NAN GAO received the master’s degree in safety
optimization,’’ 2014, arXiv:1412.6980. [Online]. Available: technology and engineering from Tsinghua Uni-
https://arxiv.org/abs/1412.6980 versity, in 2008, and the Ph.D. degree in computer
[35] S. Zheng, P. Qi, S. Chen, and X. Yang, ‘‘Fusion methods for science from the University of South Carolina,
cnn-based automatic modulation classification,’’ IEEE Access, vol. 7, in 2014. She is currently an Assistant Profes-
pp. 66496–66504, 2019. sor with the College of Computer Science and
[36] A. Paszke, S. Gross, S. Chintala, G. Chanan, E. Yang, Z. DeVito, Z. Lin, Technology, Zhejiang University of Technology,
A. Desmaison, L. Antiga, and A. Lerer, ‘‘Automatic differentiation in and the Director of the Natural Language Pro-
pytorch,’’ in Proc. Adv. Neural Inf. Process. Syst., 2017. cessing Institute of Guoxin Big Data Research
[37] T. O’Shea and J. Hoydis, ‘‘An introduction to deep learning for the physical Institute. Her research interests include algorithm
layer,’’ IEEE Trans. Cogn. Commun. Netw., vol. 3, no. 4, pp. 563–575, optimization, data mining, and artificial intelligence.
Dec. 2017.

YUAN WU (S’08–M’10–SM’16) received the

Ph.D. degree in electronic and computer engineer-
ing from The Hong Kong University of Science
and Technology, Hong Kong, in 2010. He is cur-
rently an Associate Professor with the State Key
Laboratory of Internet of Things for Smart City,
LIANG HUANG (M’16) received the B.Eng. University of Macau, Macao, China, and also with
degree in communications engineering from the Department of Computer and Information Sci-
Zhejiang University, Hangzhou, China, in 2009, ence, University of Macau, Macao. Prior to that,
and the Ph.D. degree in information engineer- he was a Full Professor with the College of Infor-
ing from The Chinese University of Hong Kong, mation Engineering, Zhejiang University of Technology, China. His research
Hong Kong, in 2013. He is currently an Assis- interests focus on resource management for wireless networks, green com-
tant Professor with the College of Information munications and computing, mobile edge computing, and smart grid. He was
Engineering, Zhejiang University of Technology, a recipient of the Best Paper Award of the IEEE International Conference on
China. His research interest includes the areas Communications, in 2016, and the Best Paper Award of the IEEE Technical
of queuing and scheduling in communication Committee on Green Communications and Computing, in 2017.
systems and networks.

1506 VOLUME 8, 2020