Quickest Spectrum Detection Using Hidden Markov Model

for Cognitive Radio

Zhe Chen, Zhen Hu, Robert C. Qiu
Department of Electrical and Computer Engineering
Center for Manufacturing Research
Tennessee Technological University
Cookville, TN 38505, USA
Email: {zchen42, zhu21, rqiu}@tntech.edu

Abstract—The prerequisite of accessing white spectrum is to Unused Spectrum

Segments
find and locate it. Our work deals with spectrum detection

Time
and recognition under the umbrella of cognitive radio. In the
procedure of spectrum recognition, a frequency sweeping device
sweeps the wideband spectrum and the samples of the wideband
power spectrum density (PSD) are fed into different Hidden
Markov Models (HMMs) sequentially. The core idea of sequential
detection or quickest detection is borrowed and utilized here from Spectrum
the classical detection theory. In our proposed approach, forward Segment
variables from different HMMs are sequentially exploited to Holes
generate the decision statistics. The decision can be made any
time as long as the condition is met. The motivation of our
work is to detect the availability of spectra and recognize the
spectra as quickly as possible and thus shorten the time delay
of detection so as to improve the spectrum utilization. The PSDs Used Spectrum
of Wi-Fi signal, CDMA signal and GSM signal are measured Segments
using a Spectrum Analyzer (SA). These acquired data are used
to train HMMs beforehand. Meanwhile, a fourth HMM is trained Frequency
by the PSD of blank spectrum. Experimental results shows this
proposed approach is effective. Fig. 1. Unused spectrum segments and spectrum segment holes.

I. I NTRODUCTION
The Federal Communications Commission (FCC) opened putation resources to obtain the spectral correlation function.
free white space spectrum on November 4, 2008 [1]. Cognitive Since it is a block transform based technique, it introduces
radio has been put forward as a more efficient way to share more delay. A cyclostationary approach for signal detection
the radio frequency spectrum. Although there have been a and classification using Hidden Markov Model (HMM) is
few years since the concept of cognitive radio came out, introduced in [4].
its functions, algorithms, and implementations are still under The major contribution of this paper is introducing the
exploration. One of the key capabilities of cognitive radio is combination of HMM and quickest detection to cognitive
spectrum sensing and management. The first step of spectrum radio. In this paper, an approach based on HMM and quickest
sensing and management is spectrum detection, which detects detection for spectrum detection is proposed, by which the
if the spectrum segments of interest are occupied by primary radio frequency spectrum is swept continuously and spectrum
users, other secondary users, or even jammers. Spectrum detection results are output as quickly as possible. Since
detection is the cornerstone of cognitive radio. The accuracy the proposed approach is based on frequency sweeping and
and rapidity of spectrum detection algorithms are vital to quickest detection, it enables reducing the time delay between
cognitive radio. when spectrum detection result is output and when the spec-
Traditionally, there are three techniques that can be used trum is sampled. Moreover, it considers the information of
for spectrum detection [2], [3]. The most popular technique a whole spectrum segment instead of that of independent
is energy detection. Although it is simple in terms of imple- spectrum samples. Thus it enables an increased accuracy of
mentation, it has several drawbacks. Firstly, it takes time to do spectrum detection. Besides a detection result, the proposed
average. Secondly, it is not easy to set the threshold. Thirdly, approach also outputs a recognition result with the category
it does not distinguish interference from signal and noise. of detected spectrum segment, as well as its starting point
Moreover, it lacks consideration of joint information. The and ending point. With the information of detected spectrum
second technique is matched filter, which maximizes received segment, more features are enabled for cognitive radio, such as
signal-to-noise ratio. However, it requires demodulation of prediction of the time of primary user accessing the detected
received signal and dedicated receiver. The third technique spectrum segment, and simultaneous access to the detected
is cyclostationary feature detection, which needs more com- spectrum segment for both primary user and secondary user.
 Fig. 1 illustrates unused spectrum segments and spectrum 2) Iteration from 2 ≤ t ≤ T and 1 ≤ j ≤ N .
segment holes. Since at a given time spectrum segment hole
is equivalent to unused spectrum segment, in the rest of this δt (j) = max [δt−1 (i) aij ] bj (ot ) (5)
1≤i≤N
paper both unused spectrum segment and spectrum segment
3) Termination.
hole are called unused spectrum segment for convenience.
This paper is organized as follows. Quickest pattern recog- pviterbi = max [δT (i)] (6)
1≤i≤N
nition using HMM are addressed in Section II. Details of
frequency sweeping based spectrum detection and recognition Meanwhile, Forward-Backward algorithm can be used to
are presented in Section III. Experimental results are reported calculate the likelihood function pf (o|λ), in which αt (i) is
in Section IV, followed by a conclusion in Section V. called forward variable [6].
1) Initialization.
II. C OMBINING HMM WITH Q UICKEST D ETECTION
α1 (i) = πi bi (o1 )
A. HMM (7)
i = 1, 2, . . . , N
It is well known that HMM is used for pattern recognition,
2) Iteration from 2 ≤ t ≤ T and 1 ≤ j ≤ N .
such as speech recognition [5], for a long time. Take a simple N 
case as an example. Given a pattern set p = [p1 , p2 , . . . , pQ ] P
αt (j) = αt−1 (i)aij bj (ot ) (8)
and an observation sequence o = [o1 , o2 , . . . , oT ], find a i=1
pattern pq , q = 1, 2, . . . Q, that most likely generates the
3) Termination.
observation sequence. The first step is to employ training
algorithm like Baum-Welch algorithm to derive a set of HMM N
X
parameters λ = {π, A, B} for each pattern pq ∈ p, q = pf (o|λ) = αT (i) (9)
1, 2, . . . Q. The inputs for training a HMM′ can be multiple i=1

observation sequences O = [o1 , o2 , . . . , oL ] and each obser- B. Combining HMM with Quickest Detection
vation sequence ol contains multiple observations ot ∈ ol , Now the question is that if the observation sequence o to
where ol = [o1 , o2 , . . . , oT ], l = 1, 2, . . . L, t = 1, 2, . . . T . be recognized is embedded in a longer observation sequence
Note that the effective length T for each observation sequence x = [x1 , x2 , . . . , xK ], how can we find the appearance of a
is not necessarily to be equal. Assume there are N states predefined pattern as quickly as possible? And how can we
s = {s1 , s2 , . . . , sN } and M possible observation values in the determine the category of the pattern?
observation space v = {v1 , v2 , . . . , vM } after quantization. Suppose the hypothesis testing problem is,
In λ, A is defined as the state transition matrix,
H0 : xk = nk 1≤k≤K (10)
(A)i,j = Pr (sj at t + 1 |si at t )
(1) 
i, j = 1, 2, . . . , N  xk = nk 1 ≤ k < k0
H1 : xk = ok−k0 +1 k0 ≤ k < k0 + T (11)
where (•)i,j denotes the entry at the i-th row and the j-th 
xk = nk k0 + T ≤ k ≤ K
column of matrix or vector, and Pr (•) means the probability.
B is defined as the observation matrix, where nk is additive white Gaussian noise and k0 is the
starting position of o in x.
(B)i,j = Pr (vj at t |si at t ) Quickest detection can be introduced to answer the first
i = 1, 2, . . . , N (2) question, while HMM can be exploited to answer the second
j = 1, 2, . . . , M question. A beautiful bridge between quickest detection and
Because HMM considered here is static, the value of A and B forward variable in HMM was built in [7], [8]. Thus, this
is independent of t. π is defined as the initial state probability problem can be solved by using the following procedure.
vector, 1) Initialization of variables. Set threshold Sth . Let k = 0
(π)i,1 = Pr (si at t = 1) (3) and n = 1.
2) Initialize the Forward-Backward algorithm for HMM.
For simplicity, aij = (A)i,j , bi (vj ) = (B)i,j and πi = And let S0 = 0.
(π)i,1 . 3) Iteration. k = k + 1,
After HMM parameters are obtained through training, the
likelihood probability pviterbi of an input observation sequence Sk = max {0, Sk−1 + g (k; n)} (12)
o with a given set of HMM parameters λ can be calculated where
using Viterbi algorithm shown as below.  
PrH1 (xk |xk−1 , . . . , xn )
1) Initialization. g (k; n) = ln (13)
PrH0 (xk |xk−1 , . . . , xn )
δ1 (i) = πi bi (o1 ) If Sk is forced to be zero by the max operation, then
(4)
i = 1, 2, . . . , N n = k + 1 and goto step 2).
 Detection &
Received Frequency Observations Forward
recognition
signals samples for HMM variables
V1
results

VB
Sk Calculation · Arg Max
Index of kind of
used spectrum
HMM Training
Quickest (1)
Frequency & Forward segment
Quantification Detection &
Sweeping Variables
Recognition
Calculation

V2 Starting point and

Fig. 2. Overall architecture and dataflow of the proposed approach. Sk Calculation · ending point of
VB
(2) · Rising Detection
used spectrum
· segment

...
Forward Reference
Variables V1 VQ
HMM indicator for
Calculation Sk Calculation Comparison
parameters (spectrum 1)
VB occupancy of
(Q) with Threshold
(spectrum 1)

Forward
· spectrum
segment
Variables V2
HMM
Calculation
HMM Training parameters
(spectrum 2) Threshold in
(spectrum 2)
Page’s Test

...
Quickest Detection &
Forward
...

Recognition
Variables VQ
HMM
Calculation
parameters
(spectrum Q)
(spectrum Q)
Fig. 4. Architecture of the “quickest detection and recognition”
Forward
Variables VB
module.
HMM Calculation
parameters
(blank
( blank
spectrum)
HMM Training & Forward spectrum)
Variables Calculation segment is unavailable for cognitive radio, what kind of pri-
mary user is occupying this spectrum segment? For answering
Fig. 3. Architecture of the “HMM training and calculation of forward these two questions, an approach for spectrum detection and
variables” module. recognition is proposed in this section.
Quickest detection and HMM have been applied to this
approach. Different spectra of primary users or secondary
4) Termination. If Sk > Sth , then the procedure is termi-
users are treated as different patterns. For each pattern, an
nated and the appearance of the pattern is announced.
HMM is built. The whole spectrum under continuous sweeping
Otherwise, goto step 3).
is considered as a long observation sequence. Fig. 2 shows the
Based on the definition of forward variable in [6], we have overall architecture and dataflow of the proposed frequency
Pr (xk , xk−1 , . . . x1 ) sweeping based spectrum detection and recognition approach.
Pr (xk |xk−1 , . . . , x1 ) = (14) A frequency sweeping device outputs the frequency spec-
Pr (xk−1 , . . . x1 )
PN trum samples of received time-domain signals in real-time.
αk (i) Then the frequency spectrum samples are fed into a quantifier
i=1
= (15) to limit the values of the frequency spectrum samples to
N
P
αk−1 (i) a certain subset of integers, i.e., observation space. These
i=1 quantified frequency spectrum samples are called observations
In order to avoid numerical underflow as k in (14) becomes for HMMs, which are inputs for HMMs.
larger, a scaling operation is applied to forward variables [7]: There are two phases for HMM in this approach, i.e., train-
ing phase and recognition phase. In training phase, observation
α̂1 (i) = α1 (i) sequences calculated from known spectrum segments are used
(16)
i = 1, 2, . . . , N to train different HMMs, which is a learning process. Baum-
and for 2 ≤ t ≤ T and 1 ≤ j ≤ N , Welch algorithm can be employed to process input observation
N  sequences and generate parameters of HMMs. Training is
P
α̂t−1 (i) aij bj (ot ) usually done offline. As shown in Fig. 3, parameters of
multiple HMMs are obtained after training phase and stored
α̂t (j) = i=1 N (17)
P for future use. In the proposed approach, an additional HMM
α̂t−1 (i) is built in training phase, which models the “blank” spectrum.
i=1
It means that no particular primary user or secondary user but
III. F REQUENCY S WEEPING BASED S PECTRUM thermal noise is present. No forward variables are calculated
D ETECTION AND R ECOGNITION in training phase.
Detecting available spectrum segments is a basis of cog- Once HMMs for known spectrum segments are bulit, recog-
nitive radio. The information of the availability of spectrum nition phase becomes ready. In recognition phase, the switch
segments and the category of primary user who is accessing in Fig. 3 is switched up and thus training process is bypassed.
the spectrum segment is important for cognitive radio to utilize Conventionally, Viterbi algorithm is applied for HMM based
unused spectrum segments. How to detect the availability of pattern recognition and the likelihood probabilities of input
spectrum segments as quickly as possible? And if a spectrum observation sequences is given by (4), (5), (6). While in the
 −40 CDMA PSD −50 Wi−Fi PSD
Marking lines Marking lines

−50
−60

−60
−70
−70
PSD (dBm)

PSD (dBm)
−80
−80

−90 −90

−100
−100
−110
−110
−120
805 810 815 820 825 830 835 840 845 850 855 2440 2450 2460 2470 2480 2490
Frequency (MHz) Frequency (MHz)

Fig. 5. PSD of CDMA. Fig. 7. PSD of Wi-Fi.

−30 −92
GSM PSD
Marking lines
−94
−40
−96
−50
−98

−60 −100
PSD (dBm)
PSD (dBm)

−102
−70
−104
−80
−106
−90
−108

−100 −110

−112
−110
1870 1880 1890 1900 1910 1920 50 100 150 200 250 300 350 400 450 500
Frequency (MHz) Frequency Sample

Fig. 6. PSD of GSM. Fig. 8. PSD of blank spectrum.

proposed approach, Forward-Backward algorithm is employed detection submodule. The rising detection submodule records
to calculate forward variables, as shown in (7), (8), (9). Each the starting points and the ending points of the rising parts
forward variables calculation submodule in Figure 3 calculates of Sk curves. Once it receives the enabling-signal sent by
forward variables sequentially based on input observation the comparison submodule and all the Sk curves begin to go
sequences and the parameters of corresponding HMMs. Then down, it outputs the positions of starting point and ending
the calculated forward variables are delivered to the “Quickest point that it records. And an enabling-signal is sent to the Arg
Detection and Recognition” module, as shown in Fig. 4. Max submodule from the rising detection submodule. Then the
In this module, Page’s Test is employed [7], [8], [9]. Arg Max submodule outputs the index of the Sk curve that
Each Sk calculation module in Fig. 4 takes the forward achieves the maximum value at the ending point. After that,
variables calculated both from its corresponding forward vari- all the Sk s are reset to zero and forward variables calculation
ables calculation submodule in Fig. 3 and from the forward submodules are reinitialized.
variables calculation submodule for “blank” spectrum as in- The process of spectrum detection and recognition in the
puts. (12), (13), (14), (16), (17) present the calculation proce- proposed approach is summarized as follows.
dure. Then the calculated Sk s are all fed to the comparison 1) Initialization. k = 0. Si,0 = 0, for i = 1, 2, ..., Q,
submodule, the rising detection submodule, and the Arg Max where Q is the number of kinds of known spectra, and
submodule in Fig. 4. Si,k represents the k th sample of S for the ith kind of
The comparison submodule compares each Sk with a given known spectrum. Initialize forward variables calculation
threshold. If any Sk is greater than the threshold, a reference submodules.
indicator for occupancy of spectrum segment is output, which 2) Iteration. k = k + 1. Calculate Si,k , for i = 1, 2, ..., Q.
indicates that the spectrum segment being swept is occupied. If Si,k is forced to be set to zero by the max operation
At the same time, an enabling-signal is sent to the rising in 12, then goto step 1).
 2500 4000
WiFi WiFi
CDMA CDMA
3500
GSM GSM
2000 Threshold Threshold
3000

2500
1500
Sk

Sk
2000

1000
1500

1000
500
500

0 0
805 810 815 820 825 830 835 840 845 850 855 1870 1880 1890 1900 1910 1920
Frequency (MHz) Frequency (MHz)

Fig. 9. Sk for CDMA spectrum detection and recognition. Fig. 10. Sk for GSM spectrum detection and recognition.

14000
3) If Si,k is great than the threshold Sth , for any i = WiFi
1, 2, ..., Q, then the comparison submodule outputs a 12000
CDMA
GSM
reference indicator for occupancy of spectrum segment Threshold

and sends an enabling-signal to the rising detection 10000

submodule.
8000
4) If the Sk curve that later goes the highest begins to rise, Sk

the position of the starting point of rising is recorded by 6000

the rising detection submodule. If all of the Sk curves
begin to drop, the position of the ending point of rising 4000

is recorded.
2000
5) If the rising detection submodule receives an enabling-
signal and all of the Sk curves begin to drop, then the 0
2440 2450 2460 2470 2480 2490
positions of recorded starting point and ending point are Frequency (MHz)
output, and an enabling-signal is sent to the Arg Max
submodule. Fig. 11. Sk for Wi-Fi spectrum detection and recognition.
6) If the Arg Max submodule receives an enabling-signal,
then it outputs the index of the Sk curve that achieves
the maximum value at the ending point, and goto step Using the proposed approach, the Sk curves for detection and
1). recognition of CDMA, GSM, and Wi-Fi spectra are shown
7) Goto step 2). in Fig. 9, Fig. 10, Fig. 11. From these figures it can be
observed that where no primary user’s spectrum is present,
IV. E XPERIMENTAL R ESULTS the curves of Sk are close to zero. And where primary
Experimental testing has been done in the Wireless Net- user’s spectrum exists, the corresponding parts of Sk curves
working Systems Lab of Tennessee Technological University. go up sharply and beyond the threshold. Given a proper
A Spectrum Analyzer (SA) is used to sweep the spectrum threshold, quickest detection of the existence of primary user’s
and output the measured power spectrum densities (PSDs) of spectrum can be achieved. Since the curve that goes the highest
spectrum segments. The model of the SA is Rohde & Schwarz indicates the final recognition result, according to the figures,
FSEM20 with 10 dB RF attenuation and 500 points per sweep. the recognition results are all correct for CDMA, GSM, and
Three categories of primary user’s spectrum are chosen for Wi-Fi.
testing the proposed approach, i.e., Code Division Multiple After the category of detected spectrum segment is obtained,
Access (CDMA), Global System for Mobile communications the bandwidth of the spectrum segment can be determined.
(GSM), and Wi-Fi. A blank spectrum is needed for calculating The bandwidth that this detected spectrum segment occupies
Sk s. The PSDs of their spectra have been measured, as shown depends on the turning points of the highest curve in the
in Fig. 5, Fig. 6, Fig. 7, and Fig. 8. Sk figure. The span between the turning point that the curve
Then the samples of these PSDs are quantified respectively begins to go up sharply and the turning point that the curve
with 64 levels. The HMMs are trained in advance with 5 states begins to drop suddenly is the bandwidth of the detected
using the quantified PSDs. And the whole quantified PSDs spectrum segment. The bandwidths, starting points and ending
are regarded as observation sequences for HMM recognition. points detected by the proposed approach for the spectra of
 0 0

−1000
−1000
−2000
−2000
−3000
Likelihood (Log10)

Likelihood (Log10)
−3000
−4000

−4000 −5000

−6000
−5000
−7000
−6000
−8000
WiFi WiFi
−7000
CDMA −9000 CDMA
GSM GSM
−8000 −10000
805 810 815 820 825 830 835 840 845 850 855 2440 2450 2460 2470 2480 2490
Frequency (MHz) Frequency (MHz)

Fig. 12. pf for CDMA spectrum recognition. Fig. 14. pf for Wi-Fi spectrum recognition.

0
illustrations that the exact starting points and ending points of
−1000
spectrum segments are unknown. Conventionally, the curve
−2000 that goes above other curves corresponds to the recognition
result. From the figures it can be observed that the recognition
Likelihood (Log10)

−3000
results are totally unexpected. That is why Viterbi algorithm
−4000 is not adopted for recognition in this situation.
−5000
V. C ONCLUSIONS
−6000
An approach for quickest spectrum detection and recogni-
WiFi
−7000
CDMA tion for cognitive radio has been proposed in this paper. HMM
GSM
−8000 and quickest detection are utilized. The proposed architecture
1870 1880 1890 1900 1910 1920
Frequency (MHz) and algorithm have been presented. The spectra of CDMA,
GSM, Wi-Fi and blank spectrum have been measured and
Fig. 13. pf for GSM spectrum recognition. used for performance evaluation of the proposed approach.
Experimental results have demonstrated that the proposed
approach is effective.
CDMA, GSM, and Wi-Fi are listed in Table I, which are also In this paper, PSDs are used as the inputs of HMM. In the
marked as red dash-dot lines in Fig. 5, Fig. 6, and Fig. 7, for future, more features extracted from measured signals besides
the purpose of comparison with the actual spectrum peaks. PSD should be explored.
T ABLE I. Detected Starting Points, Ending Points and Bandwidths.
Kind of spectrum segment Starting point Ending point Bandwidth ACKNOWLEDGMENT
CDMA 828.3 MHz 831.8 MHz 3.5 MHz
GSM 1897.9 MHz 1903.1 MHz 5.2 MHz This work is funded by National Science Founda-
Wi-Fi 2453.9 MHz 2468.8 MHz 14.9 MHz tion through grants (ECCS-0901420), (ECCS-0821658), and
(ECCS-0622125), and Office of Naval Research through a
Conventional HMM recognition using Viterbi algorithm contract (N00014-07-1-0529). The authors would like to thank
does not work well for recognizing the swept spectrum. Amanpreet S. Saini for assistance in the measurement of the
Because when the spectrum is swept, exact starting points and spectrum.
ending points of spectrum segments are hard to be determined.
Thus errors will accumulate outside of the range of the spec- R EFERENCES
trum segments, which affects the final decision of recognition.
Moreover, the recognition process using Viterbi algorithm does [1] A. S. Saini, Z. Hu, and R. C. Qiu, “Spectrum Sensing and Reconstruction
not contain a “reset” step. So the highest likelihood curves out- for Cognitive Radio,” in IEEE SSST’09, March 2009.
[2] D. Cabric, S. Mishra, and R. Brodersen, “Implementation Issues in
put by Viterbi algorithm can not represent recognition results Spectrum Sensing for Cognitive Radios,” in Conference Record of the
correctly. In addition, for Viterbi algorithm, the presence of Thirty-Eighth Asilomar Conference on Signals, Systems and Computers.,
recognized spectrum segment will not be announced until the 2004.
[3] T. Yucek and H. Arslan, “A Survey of Spectrum Sensing Algorithms
whole spectrum segment is swept, which introduces much time for Cognitive Radio Applications,” IEEE Communications Surveys &
delay for spectrum detection. Fig. 12, Fig. 13 and Fig. 14 give Tutorials, vol. 11, no. 1, pp. 116–130, 2009.
[4] K. Kim, I. A. Akbar, K. K. Bae, J. Urn, C. M. Spooner, and J. H. Reed,
“Cyclostationary Approaches to Signal Detection and Classification in
Cognitive Radio,” in 2nd IEEE International Symposium on New Frontiers
in Dynamic Spectrum Access Networks (DySPAN), April 2007, pp. 212–
215.
[5] Z. Chen, “A Study and Implementation of Speech Recognition System
for Mobile Communication Terminals,” Master’s thesis, Hangzhou Dianzi
University, February 2003.
[6] W. Tranter, K. Shanmugan, T. Rappaport, and K. Kosbar, Principles of
Communication Systems Simulation with Wireless Applications. Prentice
Hall Press Upper Saddle River, NJ, USA, 2003.
[7] B. Chen and P. Willett, “Detection of Hidden Markov Model Tran-
sient Signals,” IEEE Transactions on Aerospace and Electronic systems,
vol. 36, no. 4, pp. 1253–1268, 2000.
[8] ——, “Quickest Detection of Hidden Markov Models,” in Proceedings
of the 36th IEEE Conference on Decision and Control, vol. 4, 1997, pp.
3984–3989.
[9] E. Page, “Continuous Inspection Schemes,” Biometrika, vol. 41, pp. 100–
115, 1954.