Spare Optimization Algorithms for Compressed Sensing
Compressed Sensing Recovery Algorithms
He Liu
[email protected] Southwest Jiaotong University
Sparse optimization algorithm is the key of compressed sensing technology.
At present, there are many compressed sensing (CS) recovery algorithms have
been proposed. The problem of compressed sensing recovery is actually a
minimization problem. Solving norm minimize using combinatorial search is
NP-hard, two frequently used sparse recovery algorithms in compressive sensing
(CS) are matching pursuit (MP) and norm minimization based algorithm
known as basis pursuit (BP). In order to obtain the results of approximate
minimization, and to avoid the NP-Hard problem, many sparse recovery
algorithms based approximate and norm based minimization problem with
have been proposed. Here I list some of them, and provide the
corresponding experimental results. The corresponding Matlab Codes of the
following algorithms can be downloaded from my GitHub page,
https://github.com/aresmiki.
The signal could be estimated from noisy measurement by solving the
convex minimization problem, called second-order cone program (SOCP), as
follows.
where is a bound of the amount of noise in the data. For notation convenience,
we rephrase the minimization problem for noise free case as
The following recovery algorithms are considered for two optimization
models, Please refer to the relevant literature for details.
1. Orthogonal Matching Pursuit (OMP) [1]
Algorithm 1: CS recovery using OMP
Southwest Jiaotong University He Liu [email protected]
Spare Optimization Algorithms for Compressed Sensing
Input: The CS observation , and a measurement matrix where
and .
Initialization: Index , residual , sparse representaton .
Iteration:
while(stopping criterion false)
;
end while
Output: Sparse representation , and the original signal .
Figure.1. Recovery result by using OMP
2. Compressive Sampling Matching Pursuit (CoSaMP) [2-3]
Southwest Jiaotong University He Liu [email protected]
Spare Optimization Algorithms for Compressed Sensing
Algorithm 2: CS recovery using CoSaMP
Input: The CS observation , a measurement matrix where
and , and the sparsity of signal based on is .
Initialization: Residual , sparse representation .
Iteration:
while(stopping criterion false)
;
end while
Output: Sparse representation , and the original signal .
Southwest Jiaotong University He Liu [email protected]
Spare Optimization Algorithms for Compressed Sensing
Figure.2. Recovery result by using CoSaMP
3. Fast Iterative Shrinkage-Thresholding Algorithm (FISTA) [4]
Algorithm 3: CS recovery using FISTA
Input: ---A Lipschitz constant of .
Step 0: Take
Step : ( ) Compute
(i) by solving the problem:
(ii)
(iii)
Southwest Jiaotong University He Liu [email protected]
Spare Optimization Algorithms for Compressed Sensing
Figure.4. Recovery result by using FISTA
4. Iterative Hard Thresholding algorithms for compressive sensing
(IHT) [5-6]
Algorithm 4: CS recovery using IHT
Input: observation , and a measurement matrix .
Initialization:
Step : ( ) Compute
(i)
(ii)
(iii) set all but the largest K elements to zero, where K is the sparsity of the estimate
(iv)
Figure.4. Recovery result by using IHT
5. Iteratively Reweighted Least Square (IRLS) [7]
Algorithm 5: CS recovery using IRLS
Southwest Jiaotong University He Liu [email protected]
Spare Optimization Algorithms for Compressed Sensing
Input: The CS observation , and a measurement matrix where
and .
Initialization: sparse representation , , .
Iteration:
while (stopping criterion false)
for from 1 to end
;
end
;
end whlie
Output: Sparse representation , and original signal .
Figure.5. Recovery result by using IRLS
6. Iterative Shrinkage-Thresholding Algorithm (ISTA) [8]
Algorithm 6: CS recovery using ISTA
Input: ---A Lipschitz constant of .
Step 0: Take
Step : ( ) Compute
(i) by solving the problem:
Southwest Jiaotong University He Liu [email protected]
Spare Optimization Algorithms for Compressed Sensing
Figure.6. Recovery result by using ISTA
7. Null-Space Reweigthted Approximate l0-Pseudonorm Algorithm
(NSRAL0) [9]
Algorithm 7: CS recovery using NSRAL0
Step 1
Input: , , , , , and .
Step 2
Set , , , and .
Step 3
Perform the QR decomposition and construct using the last columns of Q.
Step 4
With and using as an initial point, apply the BFGS algorithm to solve the problem
in (10), where re-weighting with parameter is applied using in each
Southwest Jiaotong University He Liu [email protected]
Spare Optimization Algorithms for Compressed Sensing
iteration. Denote the solution as .
Step 5
Compute and update weight vector to using .
Step 6
If . stop and output as solution; otherwise, set , , , and
repeat from Step 4.
Figure.7. Recovery result by using NSRAL0
8. Reweighted L1 Minimization Algorithm (RL1) [10]
Algorithm 8: CS recovery using RL1
Step 1
Set and for .
Step 2
Solve the weighted minimization problem
Southwest Jiaotong University He Liu [email protected]
Spare Optimization Algorithms for Compressed Sensing
Step 3
Updata the weights; let and for each ,define
Step 4
Terminate on convergence or when attains a specified maximum number of iterations
.Otherwise, increment and go to Step 2.
Figure.8. Recovery result by using RL1
9. Robust Smoothed l0-Pseudonorm Algorithm (RSL0) [11]
Algorithm 9: CS recovery using RSL0
Step 1
Let equal to the minimum norm solution of .
Step 2
Choose a suitable decreasing sequence for , .
Step 3
Let .
Maximize (approximately) the function on the feasible set ;
Initialization:
For
Southwest Jiaotong University He Liu [email protected]
Spare Optimization Algorithms for Compressed Sensing
Let .
Let .
If projiect on with:
Set .
Step 4
Final answer is .
Figure.9. Recovery result by using RSL0
10. L1_SplitBregmanIteration (SBIL1) [12]
Algorithm 10: CS recovery using SBIL1
While
For to
Southwest Jiaotong University He Liu [email protected]
Spare Optimization Algorithms for Compressed Sensing
, use a wide variety of optimization techniques to
solve this problem.
, compute the optimal value of using shrinkage
operators:
End
End
Figure.10. Recovery result by using SBIL1
11. Smoothed l0-Pseudonorm Algorithm (SL0) [13]
Algorithm 11: CS recovery using SL0
Step 1
Set . Choose a suitable decreasing sequence for .
Step 2
For
Let .
Maximize subject to , using iteration of steepest ascent:
Initialization: .
For
Southwest Jiaotong University He Liu [email protected]
Spare Optimization Algorithms for Compressed Sensing
Let .
Project s back onto the feasible set
Set .
Step 3
Final answer is .
Figure.11. Recovery result by using SL0
12. Minimization of Approximate Lp Pseudonorm Using a Quasi-
Newton Algorithm (UALP) [14]
Algorithm 12: CS recovery using UALP
Step 1
Input: and .
Southwest Jiaotong University He Liu [email protected]
Spare Optimization Algorithms for Compressed Sensing
set .
Step 2
Compute for using where .
Step 3
Use to compute and .
Step 4
Repeat for
Set and use as an initial point. Apply the BFGS algorithm to solve the problem in
Denote the solution as .
Step 5
Set , and terminate.
Figure.12. Recovery result by using URLP
Reference
[1] Joel A. Tropp and Anna C. Gilbert Signal Recovery From Random Measurements Via
Orthogonal Matching Pursuit ， IEEE TRANSACTIONS ON INFORMATION THEORY, VOL.
53, NO. 12.
[2] Needell D ， Tropp J A CoSaMP ： Iterative signal recovery from incomplete and inaccurate
samples[J]．Applied and Computation Harmonic Analysis，2009，26：301-321.
[3] D.Needell, J.A. Tropp ． CoSaMP: Iterative signal recoveryfrom incomplete and inaccurate
samples[J]. Communications of theACM，2010，53(12)：93-100.
[4] A. Beck and M. Teboulle, “A fast iterative shrinkage-thresholding algorithm for linear inverse
Southwest Jiaotong University He Liu [email protected]
Spare Optimization Algorithms for Compressed Sensing
problems,” SIAM J. Imaging Sciences, vol. 2, no. 1, pp. 183-202, 2009.
[5] Blumensath T, Davies M E. Iterative hard thresholding for compressed sensing[J]. Applied &
Computational Harmonic Analysis, 2009, 27(3):265-274.
[6] Blumensath T, Davies M E. Iterative Thresholding for Sparse Approximations[J]. Journal of
Fourier Analysis and Applications, 2008, 14(5):629-654.
[7] Chartrand and W. Yin, “Iteratively Reweighted Algorithms for Compressed Sensing,” 2008.
[8] I. Daubechies, M. Defrise, and C. D. Mol, “An iterative thresholding algorithm for linear
inverse problems with a sparsity constraint,” Comm. Pure Appl. Math., vol. 57, pp. 1413-1457,
2004.
[9] J. K. Pant, W.-S. Lu, and A. Antoniou,“Reconstruction of sparse signals by minimizing a re-
weighted approximate l0-norm in the null space of the measurement matrix,” IEEE Inter. Midwest
Symp. on Circuits-Syst, pp. 430–433, 2010.
[10] Candès E J, Wakin M B, Boyd S P. Enhancing sparsity by reweighted L1 minimization.[J].
Journal of Fourier Analysis & Applications, 2007, 14(5):877-905.
[11] H. Mohimani, M. Babie-Zadeh, and C. Jutten,“A fast approach for overcomplete sparse
decomposition based on smoothed l0-norm,” IEEE Trans. Signal Process., vol. 57, no. 1, pp. 289-
301, Jan. 2009.
[12] Yin W, Osher S, Goldfarb D, et al.Bregman Iterative Algorithms for L1 Minimization with
Applications to Compressed Sensing[J]. Siam Journal on Imaging Sciences, 2008, 1(1):143-168.
[13] H. Mohimani, M. Babie-Zadeh, and C. Jutten,“A fast approach for overcomplete sparse
decomposition based on smoothed l0-norm,” IEEE Trans. Signal Process., vol. 57, no. 1, pp. 289-
301, Jan. 2009.
[14] Pant J K, Lu W S, Antoniou A.Unconstrained regularized Lp -norm based algorithm for the
reconstruction of sparse signals[C] IEEE International Symposium on Circuits and Systems.
IEEE, 2011:1740-1743.
Southwest Jiaotong University He Liu [email protected]
