Performance Analysis of QAM Modulation and Demodul
Performance Analysis of QAM Modulation and Demodul
1. Introduction
1.1. Research Background
The demand for high data rates in modern communication systems has led to the development of
several modulation schemes, among which M-QAM is one of the most popular techniques. M-QAM
can send more than one bit per symbol by changing both the amplitude and phase of the carrier signal.
This results in higher spectral efficiency, thus making it suitable for high-throughput applications
such as wireless communications, broadband internet, and digital television.
However, despite such merits, M-QAM systems remain susceptible to various channel
impairments, most notably AWGN that is usually modeled in analytical work. In practical
environments, further channel distortions, such as frequency offset and partial band interference, can
also be present. The former is due to a mismatch between the transmitter and receiver frequencies,
which will potentially cause symbol misalignments. Partial band interference involves interference
in some frequency bands that could degrade the signal quality and increase the BER [1,2]. With the
evolution in the communication system, it has been of prime importance to know the performance of
M-QAM under these impairments.
1.2. Research Focus and Objectives
This research focuses on the performance evaluation of M-QAM systems in the presence of
AWGN, frequency offset, and partial band interference. The simulator models the behavior of a
system subjected to these degradations by calculating the BER as the main performance metric.
Different M-QAM schemes, such as 16QAM [3] and 64QAM, are considered to observe how
increasing the modulation order affects system performance in noisy and distorted channels [4,5,6,7].
While it seeks to determine the performance of channel impairments in M-QAM systems and more
importantly, how higher-order modulations, which will yield higher data rates, necessarily trade off
against increased sensitivity to noise and interference, this paper attempts to be practical in the choice
of modulation schemes in realistic environments [8].
114
Highlights in Science, Engineering and Technology ESAC 2025
Volume 134 (2025)
Equation (1) is the theoretical expression for the minimum Euclidean distance between points in
the constellation diagram. In the above formula, M is the number of symbols in the constellation and
log 2 M indicates the number of bits per symbol. 𝐸𝑏 is the energy per bit, which is one of the main
parameters characterizing system performance in terms of power efficiency. 𝑀 − 1 in the
denominator normalizes for the average energy of the constellation. This distance d has a direct
impact on system performance since a higher value of d decreases the probability of symbol errors in
an additive noise environment. Therefore, the formula presents the trade-off between constellation
size M and the system's robustness to noise as a basic measure for the study of modulation schemes
such as M-QAM.
2.3. Constellation Diagrams for QAM Modulation of Different Orders
Modulation is done on the binary bits. It is basically the process of mapping the binary data onto
a specific modulation constellation, such as QAM. During this process, each group of binary bits is
mapped to a complex symbol, which corresponds to a point in the modulation constellation on the
complex plane. This process effectively transforms the original binary data into complex symbols
suitable for wireless transmission. The system reshapes the modulated symbols into the length of the
frame and the number of symbols, respectively, in three-dimensional matrix form for further
processing. Each frame has a certain number of symbols; the whole sequence of signals is rearranged
in that structure for further transmission or processing tasks. Finally, the system visualizes the
modulated signal by plotting a scatter diagram, showing the distribution of the symbols on the
complex plane. This allows for a visual inspection of the modulation constellation, where the symbol
spacing and phase differences can be analyzed.
This article uses constellation diagrams generated by the models for QPSK, 16QAM, and 64QAM
modulation as examples which can be seen in Fig. 2. Upon comparison, it is evident that as the
modulation order increases, the distance between points in the constellation diagram becomes smaller,
which also implies a weaker ability to resist channel noise and distortion.
115
Highlights in Science, Engineering and Technology ESAC 2025
Volume 134 (2025)
(a) QPSK
where M is the modulation order, log 2 M represents the number of bits per symbol, Q(x) is the
Q-function that characterizes the tail probability of the Gaussian distribution, Eb denotes the energy
per bit, and N0 is the noise power spectral density. The formula consists of three parts: a scaling
4 1
factor log (1 − ) , a summation term related to the constellation points ∑√M/2
i=1 Q(⋅) , and a
2 M √M
b 2 3E log M
coefficient √ (M−1)N which reflects the relationship between the SNR and system performance.
0
This formula provides a theoretical estimation of the SER for M-QAM modulation in an AWGN
channel, effectively capturing the trade-offs between modulation order, signal energy, and noise
strength.
116
Highlights in Science, Engineering and Technology ESAC 2025
Volume 134 (2025)
3.1.2. Constellation Diagrams under Different QAM Modulation Schemes with Added Noise
QPSK (Quadrature Phase Shift Keying), 16QAM (16-Quadrature Amplitude Modulation), and
64QAM (64-Quadrature Amplitude Modulation) are commonly used modulation schemes in digital
communication systems, each offering different trade-offs between data rate and robustness [10].
QPSK is a phase modulation technique that has each symbol carrying 2 bits of information by
using four different phase shifts to encode the signal. This will provide better spectral efficiency
compared to BPSK, with fairly good noise immunity, hence it is suitable for low SNR environments.
16QAM combines both amplitude and phase modulation, where each symbol represents 4 bits of
information attained by mapping 16 different symbols on the complex plane. It gives higher data rates
compared to QPSK; a factor that makes it effective in applications where bandwidth is limited.
However, 16QAM has a greater vulnerability to noise, hence a higher BER under poor channel
conditions.64QAM extends this further, where in this case, each symbol represents 6 bits of
information, mapped across 64 distinct points on the complex plane. While 64QAM offers the highest
data rate, it demands a high SNR for reliable communication, as the symbols are closely spaced,
increasing the likelihood of errors in noisy environments [11,12].
In order to investigate the impact of noise with different signal-to-noise ratios (SNR) on various
QAM modulation schemes we add the Gaussian white noise with SNR ranging from -10 dB to 20 dB
to the channel. The constellation diagrams for QPSK, 16QAM and 64QAM modulation schemes are
recorded accordingly. The noise levels with SNR of -10 dB, 0 dB, 10 dB and 20 dB are used as
representatives, as shown in the Fig.3 below.
117
Highlights in Science, Engineering and Technology ESAC 2025
Volume 134 (2025)
almost negligible distance between them. This makes the system vulnerable to noise, as even small
variations in the signal can result in the misinterpretation of a symbol, thus increasing the BER. In
contrast, the 16QAM constellation has more widely spaced symbols, thus greater separation between
them. The increased spacing of the symbols, in turn, provides improved error resilience since noise-
induced deviations are less likely to cause symbol misclassification.
This observation points to one of the fundamental trade-offs in modulation schemes: while the
modulation order increases, more bits are transmitted per symbol, which results in a higher data rate.
However, this also means that the signal's sensitivity to noise increases. The reduced distance between
adjacent symbols in higher-order modulation schemes like 64QAM means that the system's ability to
resist channel noise is compromised, particularly in environments with lower signal-to-noise ratios.
In contrast, lower-order schemes like 16QAM, while offering lower data rates, tend to be more robust
against noise and, therefore, are preferable in conditions where the signal quality is less stable or the
SNR is lower. Thus, the choice of modulation scheme must carefully balance the need for high data
rates with the system's robustness to noise, depending on the specific requirements of the
communication system and the quality of the transmission channel.
3.1.3. BER under the Influence of Noise
The theoretical BER of the system can be computed using Equation (2). By storing the actual
simulation data of SNR values ranging from -10 to 20 and plotting the results, we can see the BER
trends for different QAM modulation orders. We then compare the theoretical BER, derived from the
formulas, with the actual simulated BER for QPSK, 16QAM, and 64QAM modulation schemes on
the same linear graph Fig. 4.
118
Highlights in Science, Engineering and Technology ESAC 2025
Volume 134 (2025)
119
Highlights in Science, Engineering and Technology ESAC 2025
Volume 134 (2025)
120
Highlights in Science, Engineering and Technology ESAC 2025
Volume 134 (2025)
121
Highlights in Science, Engineering and Technology ESAC 2025
Volume 134 (2025)
The goal of JSR is the scaling of the amplitude of the interference signal by using a gain factor. In
order to apply the JSR, the code uses the following formula to calculate the gain factor and adjust the
interference signal power
JSR
. (5)
PowerAdjustmentFactor=10 20
This formula converts the JSR value from dB to a linear gain. By performing element-wise
multiplication on the modulated interference symbol matrix, the amplitude of the interference signal
is scaled by the corresponding multiple of the original effective signal's amplitude. Particularly,
the gain factor scales the interference signal's power due to a given JSR value. For instance, if the
JSR is 10 dB, the gain factor will be around 3.16, which means that the power of the interference
signal will be 3.16 times the effective signal power. In the case where the JSR is -10 dB, the gain
factor will be approximately 0.316, meaning that the interference signal power will be 0.316 times
the effective signal power. This approach allows the code to simulate interference with varying
strengths and adjust the interference signal's impact on the system as needed.
3) Frequency Domain Mapping and Inverse Transformation
The program arranges the modulated interference signal in a frequency domain matrix in a certain
way and then applies the Inverse Fast Fourier Transform to this matrix. The inverse transform serves
to convert the frequency domain signal back into the time domain, generating the final interference
waveform. During this process, the signal is appropriately normalized to ensure that the resulting
time-domain signal has the correct amplitude.
4) Frequency Domain Mapping and Inverse Transformation
Finally, the interference signal, after undergoing the IFFT process, is stored in the interference
variable and returned as the output. At this point, the interference signal is ready in the time domain
for further analysis or for combination with other signals. The whole process is shown in the Fig 6.
Where:
Bi is the bandwidth of the interference signal, i.e., the width of the frequency band occupied by
the interference signal.
122
Highlights in Science, Engineering and Technology ESAC 2025
Volume 134 (2025)
Bs is the operational bandwidth of the effective signal, i.e., the bandwidth used by the
communication system to transmit the valid signal.
Interference factor γ is one of the important parameters that characterizes the share of the frequency
the spectrum occupied by the interference signal. It is directly involved in system performance
regarding interference strength, BER, JSR, and spectral efficiency. A higher interference factor γ
indicates that the interference signal has a greater impact on the system and is typically associated
with degradation in communications performance, such as an increased bit error rate. By analyzing
γ, the performance of the system in interference can be evaluated and optimized. Particularly in
various modulation schemes, like QPSK, 16QAM, and 64QAM.
3) The BER Results of Different QAM Under Different JSR
250
Experiments were conducted with the parameterr = 1024 = 24.4%and without other impairments.
The BER results of M-QAM under different JSR conditions are shown in Fig 7.
123
Highlights in Science, Engineering and Technology ESAC 2025
Volume 134 (2025)
more susceptible to interference, leading to an increased bit error rate. Therefore, under the same
interference conditions, the performance degradation in higher-order QAM systems is more
pronounced, while lower-order QAM schemes (e.g., QPSK) exhibit relatively stronger resistance to
interference.
4. Conclusion
4.1. Summary of Research Findings and Conclusions
This study investigates the performance of baseband modulation systems utilizing M-QAM, with
a particular focus on the impact of higher modulation orders on system performance. The results
indicate that as the order of QAM increases, the distance between constellation points decreases,
leading to reduced resistance to noise and other channel impairments. Although higher-order QAM
schemes, such as 64QAM, provide higher data transmission rates, their performance under noise,
frequency offset, and band interference significantly deteriorates, resulting in a substantial increase
in BER. Thus, selecting the appropriate modulation order requires balancing data transmission speed
with system robustness. Advanced algorithms, such as carrier synchronization, symbol timing
recovery, channel estimation, and equalization, will also be necessary in practical communications to
mitigate channel impairments.
4.2. Implications and Significance
These results have an important meaning for the design and optimization of communication
systems. The modern tendency in communication systems is the continuous increase in data
transmission rates. In this context, the most important challenge is the need to find an appropriate
balance between high-speed transmission and the possibility of the system's resistance to noise and
interference. In such conditions, the analysis of the M-QAM system with different channel conditions
gives significant insight into the choice of modulation schemes for various applications. While
higher-order M-QAM modulations yield increased data rates, it makes them more susceptible to noise
and impairments, therefore, requiring sophisticated compensation techniques. It reassures that this
study emphasizes how this understanding of the trade-off between transmission speed and reliability
may guide the design of robust communication systems in realistic environmental conditions.
4.3. Future Research Directions
In this respect, future research work should further consider the performance of M-QAM systems
under more complex and dynamic channel conditions. As communication technologies are
continuously evolving, traditional modulation schemes and compensation algorithms may struggle to
handle new types of channel impairments introduced by high-speed mobility or multi-path fading. It
thus follows that future work needs to shift attention to algorithm development, especially deep
learning-based adaptive channel estimation, real-time modulation scheme optimizations, and hence
enhance robustness in demanding environments. Hybrid modulation techniques, combined with
MIMO, shall also be another interesting topic for further investigations, regarding their application
on emerging 5G/6G networks, hence opening further perspectives for enhancement of the efficiency
and reliability of a communication system.
References
[1] Breed, Gary. "Bit error rate: Fundamental concepts and measurement issues." High Frequency Electronics
2.1 (2003): 46 - 47.
[2] Jeruchim, Michel. "Techniques for estimating the bit error rate in the simulation of digital communication
systems." IEEE Journal on selected areas in communications 2.1 (1984): 153 - 170.
[3] El-Nahal, Fady. "Coherent 16 quadrature amplitude modulation (16QAM) optical communication
systems." Photonics letters of Poland 10.2 (2018): 57 - 59.
124
Highlights in Science, Engineering and Technology ESAC 2025
Volume 134 (2025)
[4] Hong, Sungnam, et al. "Frequency and quadrature-amplitude modulation for downlink cellular OFDMA
networks." IEEE journal on selected areas in communications 32.6 (2014): 1256 - 1267.
[5] Park, Sung-Joon. "Performance analysis of triangular quadrature amplitude modulation in AWGN
channel." IEEE communications letters 16.6 (2012): 765 - 768.
[6] Pappi, Koralia N., Athanasios S. Lioumpas, and George K. Karagiannidis. "θ-QAM: A parametric
quadrature amplitude modulation family and its performance in AWGN and fading channels." IEEE
Transactions on Communications 58.4 (2010): 1014 - 1019.
[7] RANHOTRA, SARVRAJ SINGH. "Analytical performance evaluation of OFDM in presence of channel
estimation errors and carrier frequency offset." (2015).
[8] Delmas, J-P., and Habti Abeida. "Cramer-Rao bounds of DOA estimates for BPSK and QPSK modulated
signals." IEEE Transactions on Signal Processing 54.1 (2005): 117 - 126.
[9] Ali, Irfan. "Bit-error-rate (BER) simulation using MATLAB." International Journal of Engineering
Research and Applications 3.1 (2013): 706 - 711.
[10] Singya, Praveen Kumar, et al. "A survey on design and performance of higher-order QAM constellations."
arXiv preprint arXiv:2004.14708 (2020).
[11] Youssef, Tamer, and Eman Abdelfattah. "Performance evaluation of different QAM techniques using
Matlab/Simulink." 2013 IEEE Long Island Systems, Applications and Technology Conference (LISAT).
IEEE, 2013.
[12] Zhendong, Chou, et al. "Modulation recognition based on constellation diagram for M-QAM signals."
2013 IEEE 11th International Conference on Electronic Measurement & Instruments. Vol. 1. IEEE, 2013.
[13] Singya, Praveen Kumar, et al. "A survey on higher-order QAM constellations: Technical challenges,
recent advances, and future trends." IEEE Open Journal of the Communications Society 2 (2021): 617 -
655.
125