Showing 1–42 of 42 results for author: Palomar, D

Polynomial Graphical Lasso: Learning Edges from Gaussian Graph-Stationary Signals

Authors: Andrei Buciulea, Jiaxi Ying, Antonio G. Marques, Daniel P. Palomar

Abstract: This paper introduces Polynomial Graphical Lasso (PGL), a new approach to learning graph structures from nodal signals. Our key contribution lies in modeling the signals as Gaussian and stationary on the graph, enabling the development of a graph-learning formulation that combines the strengths of graphical lasso with a more encompassing model. Specifically, we assume that the precision matrix can… ▽ More This paper introduces Polynomial Graphical Lasso (PGL), a new approach to learning graph structures from nodal signals. Our key contribution lies in modeling the signals as Gaussian and stationary on the graph, enabling the development of a graph-learning formulation that combines the strengths of graphical lasso with a more encompassing model. Specifically, we assume that the precision matrix can take any polynomial form of the sought graph, allowing for increased flexibility in modeling nodal relationships. Given the resulting complexity and nonconvexity of the resulting optimization problem, we (i) propose a low-complexity algorithm that alternates between estimating the graph and precision matrices, and (ii) characterize its convergence. We evaluate the performance of PGL through comprehensive numerical simulations using both synthetic and real data, demonstrating its superiority over several alternatives. Overall, this approach presents a significant advancement in graph learning and holds promise for various applications in graph-aware signal analysis and beyond. △ Less

Submitted 3 April, 2024; originally announced April 2024.

FDR-Controlled Portfolio Optimization for Sparse Financial Index Tracking

Authors: Jasin Machkour, Daniel P. Palomar, Michael Muma

Abstract: In high-dimensional data analysis, such as financial index tracking or biomedical applications, it is crucial to select the few relevant variables while maintaining control over the false discovery rate (FDR). In these applications, strong dependencies often exist among the variables (e.g., stock returns), which can undermine the FDR control property of existing methods like the model-X knockoff m… ▽ More In high-dimensional data analysis, such as financial index tracking or biomedical applications, it is crucial to select the few relevant variables while maintaining control over the false discovery rate (FDR). In these applications, strong dependencies often exist among the variables (e.g., stock returns), which can undermine the FDR control property of existing methods like the model-X knockoff method or the T-Rex selector. To address this issue, we have expanded the T-Rex framework to accommodate overlapping groups of highly correlated variables. This is achieved by integrating a nearest neighbors penalization mechanism into the framework, which provably controls the FDR at the user-defined target level. A real-world example of sparse index tracking demonstrates the proposed method's ability to accurately track the S&P 500 index over the past 20 years based on a small number of stocks. An open-source implementation is provided within the R package TRexSelector on CRAN. △ Less

Submitted 30 January, 2024; v1 submitted 26 January, 2024; originally announced January 2024.

Sparse PCA with False Discovery Rate Controlled Variable Selection

Authors: Jasin Machkour, Arnaud Breloy, Michael Muma, Daniel P. Palomar, Frédéric Pascal

Abstract: Sparse principal component analysis (PCA) aims at mapping large dimensional data to a linear subspace of lower dimension. By imposing loading vectors to be sparse, it performs the double duty of dimension reduction and variable selection. Sparse PCA algorithms are usually expressed as a trade-off between explained variance and sparsity of the loading vectors (i.e., number of selected variables). A… ▽ More Sparse principal component analysis (PCA) aims at mapping large dimensional data to a linear subspace of lower dimension. By imposing loading vectors to be sparse, it performs the double duty of dimension reduction and variable selection. Sparse PCA algorithms are usually expressed as a trade-off between explained variance and sparsity of the loading vectors (i.e., number of selected variables). As a high explained variance is not necessarily synonymous with relevant information, these methods are prone to select irrelevant variables. To overcome this issue, we propose an alternative formulation of sparse PCA driven by the false discovery rate (FDR). We then leverage the Terminating-Random Experiments (T-Rex) selector to automatically determine an FDR-controlled support of the loading vectors. A major advantage of the resulting T-Rex PCA is that no sparsity parameter tuning is required. Numerical experiments and a stock market data example demonstrate a significant performance improvement. △ Less

Submitted 16 January, 2024; originally announced January 2024.

Comments: Published in ICASSP 2024 - 2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), scheduled for 14-19 April 2024 in Seoul, Korea

Joint Signal Recovery and Graph Learning from Incomplete Time-Series

Authors: Amirhossein Javaheri, Arash Amini, Farokh Marvasti, Daniel P. Palomar

Abstract: Learning a graph from data is the key to taking advantage of graph signal processing tools. Most of the conventional algorithms for graph learning require complete data statistics, which might not be available in some scenarios. In this work, we aim to learn a graph from incomplete time-series observations. From another viewpoint, we consider the problem of semi-blind recovery of time-varying grap… ▽ More Learning a graph from data is the key to taking advantage of graph signal processing tools. Most of the conventional algorithms for graph learning require complete data statistics, which might not be available in some scenarios. In this work, we aim to learn a graph from incomplete time-series observations. From another viewpoint, we consider the problem of semi-blind recovery of time-varying graph signals where the underlying graph model is unknown. We propose an algorithm based on the method of block successive upperbound minimization (BSUM), for simultaneous inference of the signal and the graph from incomplete data. Simulation results on synthetic and real time-series demonstrate the performance of the proposed method for graph learning and signal recovery. △ Less

Submitted 28 December, 2023; originally announced December 2023.

Discerning and Enhancing the Weighted Sum-Rate Maximization Algorithms in Communications

Authors: Zepeng Zhang, Ziping Zhao, Kaiming Shen, Daniel P. Palomar, Wei Yu

Abstract: Weighted sum-rate (WSR) maximization plays a critical role in communication system design. This paper examines three optimization methods for WSR maximization, which ensure convergence to stationary points: two block coordinate ascent (BCA) algorithms, namely, weighted sum-minimum mean-square error (WMMSE) and WSR maximization via fractional programming (WSR-FP), along with a minorization-maximiza… ▽ More Weighted sum-rate (WSR) maximization plays a critical role in communication system design. This paper examines three optimization methods for WSR maximization, which ensure convergence to stationary points: two block coordinate ascent (BCA) algorithms, namely, weighted sum-minimum mean-square error (WMMSE) and WSR maximization via fractional programming (WSR-FP), along with a minorization-maximization (MM) algorithm, WSR maximization via MM (WSR-MM). Our contributions are threefold. Firstly, we delineate the exact relationships among WMMSE, WSR-FP, and WSR-MM, which, despite their extensive use in the literature, lack a comprehensive comparative study. By probing the theoretical underpinnings linking the BCA and MM algorithmic frameworks, we reveal the direct correlations between the equivalent transformation techniques, essential to the development of WMMSE and WSR-FP, and the surrogate functions pivotal to WSR-MM. Secondly, we propose a novel algorithm, WSR-MM+, harnessing the flexibility of selecting surrogate functions in MM framework. By circumventing the repeated matrix inversions in the search for optimal Lagrange multipliers in existing algorithms, WSR-MM+ significantly reduces the computational load per iteration and accelerates convergence. Thirdly, we reconceptualize WSR-MM+ within the BCA framework, introducing a new equivalent transform, which gives rise to an enhanced version of WSR-FP, named as WSR-FP+. We further demonstrate that WSR-MM+ can be construed as the basic gradient projection method. This perspective yields a deeper understanding into its computational intricacies. Numerical simulations corroborate the connections between WMMSE, WSR-FP, and WSR-MM and confirm the efficacy of the proposed WSR-MM+ and WSR-FP+ algorithms. △ Less

Submitted 8 November, 2023; originally announced November 2023.

Learning Large-Scale MTP$_2$ Gaussian Graphical Models via Bridge-Block Decomposition

Authors: Xiwen Wang, Jiaxi Ying, Daniel P. Palomar

Abstract: This paper studies the problem of learning the large-scale Gaussian graphical models that are multivariate totally positive of order two ($\text{MTP}_2$). By introducing the concept of bridge, which commonly exists in large-scale sparse graphs, we show that the entire problem can be equivalently optimized through (1) several smaller-scaled sub-problems induced by a \emph{bridge-block decomposition… ▽ More This paper studies the problem of learning the large-scale Gaussian graphical models that are multivariate totally positive of order two ($\text{MTP}_2$). By introducing the concept of bridge, which commonly exists in large-scale sparse graphs, we show that the entire problem can be equivalently optimized through (1) several smaller-scaled sub-problems induced by a \emph{bridge-block decomposition} on the thresholded sample covariance graph and (2) a set of explicit solutions on entries corresponding to bridges. From practical aspect, this simple and provable discipline can be applied to break down a large problem into small tractable ones, leading to enormous reduction on the computational complexity and substantial improvements for all existing algorithms. The synthetic and real-world experiments demonstrate that our proposed method presents a significant speed-up compared to the state-of-the-art benchmarks. △ Less

Submitted 29 September, 2023; v1 submitted 23 September, 2023; originally announced September 2023.

Adaptive Estimation of Graphical Models under Total Positivity

Authors: Jiaxi Ying, José Vinícius de M. Cardoso, Daniel P. Palomar

Abstract: We consider the problem of estimating (diagonally dominant) M-matrices as precision matrices in Gaussian graphical models. These models exhibit intriguing properties, such as the existence of the maximum likelihood estimator with merely two observations for M-matrices \citep{lauritzen2019maximum,slawski2015estimation} and even one observation for diagonally dominant M-matrices \citep{truell2021max… ▽ More We consider the problem of estimating (diagonally dominant) M-matrices as precision matrices in Gaussian graphical models. These models exhibit intriguing properties, such as the existence of the maximum likelihood estimator with merely two observations for M-matrices \citep{lauritzen2019maximum,slawski2015estimation} and even one observation for diagonally dominant M-matrices \citep{truell2021maximum}. We propose an adaptive multiple-stage estimation method that refines the estimate by solving a weighted $\ell_1$-regularized problem at each stage. Furthermore, we develop a unified framework based on the gradient projection method to solve the regularized problem, incorporating distinct projections to handle the constraints of M-matrices and diagonally dominant M-matrices. A theoretical analysis of the estimation error is provided. Our method outperforms state-of-the-art methods in precision matrix estimation and graph edge identification, as evidenced by synthetic and financial time-series data sets. △ Less

Submitted 8 June, 2023; v1 submitted 27 October, 2022; originally announced October 2022.

Comments: 26 pages

Fast Projected Newton-like Method for Precision Matrix Estimation under Total Positivity

Authors: Jian-Feng Cai, José Vinícius de M. Cardoso, Daniel P. Palomar, Jiaxi Ying

Abstract: We study the problem of estimating precision matrices in Gaussian distributions that are multivariate totally positive of order two ($\mathrm{MTP}_2$). The precision matrix in such a distribution is an M-matrix. This problem can be formulated as a sign-constrained log-determinant program. Current algorithms are designed using the block coordinate descent method or the proximal point algorithm, whi… ▽ More We study the problem of estimating precision matrices in Gaussian distributions that are multivariate totally positive of order two ($\mathrm{MTP}_2$). The precision matrix in such a distribution is an M-matrix. This problem can be formulated as a sign-constrained log-determinant program. Current algorithms are designed using the block coordinate descent method or the proximal point algorithm, which becomes computationally challenging in high-dimensional cases due to the requirement to solve numerous nonnegative quadratic programs or large-scale linear systems. To address this issue, we propose a novel algorithm based on the two-metric projection method, incorporating a carefully designed search direction and variable partitioning scheme. Our algorithm substantially reduces computational complexity, and its theoretical convergence is established. Experimental results on synthetic and real-world datasets demonstrate that our proposed algorithm provides a significant improvement in computational efficiency compared to the state-of-the-art methods. △ Less

Submitted 22 October, 2023; v1 submitted 3 December, 2021; originally announced December 2021.

Algorithms for Learning Graphs in Financial Markets

Authors: José Vinícius de Miranda Cardoso, Jiaxi Ying, Daniel Perez Palomar

Abstract: In the past two decades, the field of applied finance has tremendously benefited from graph theory. As a result, novel methods ranging from asset network estimation to hierarchical asset selection and portfolio allocation are now part of practitioners' toolboxes. In this paper, we investigate the fundamental problem of learning undirected graphical models under Laplacian structural constraints fro… ▽ More In the past two decades, the field of applied finance has tremendously benefited from graph theory. As a result, novel methods ranging from asset network estimation to hierarchical asset selection and portfolio allocation are now part of practitioners' toolboxes. In this paper, we investigate the fundamental problem of learning undirected graphical models under Laplacian structural constraints from the point of view of financial market times series data. In particular, we present natural justifications, supported by empirical evidence, for the usage of the Laplacian matrix as a model for the precision matrix of financial assets, while also establishing a direct link that reveals how Laplacian constraints are coupled to meaningful physical interpretations related to the market index factor and to conditional correlations between stocks. Those interpretations lead to a set of guidelines that practitioners should be aware of when estimating graphs in financial markets. In addition, we design numerical algorithms based on the alternating direction method of multipliers to learn undirected, weighted graphs that take into account stylized facts that are intrinsic to financial data such as heavy tails and modularity. We illustrate how to leverage the learned graphs into practical scenarios such as stock time series clustering and foreign exchange network estimation. The proposed graph learning algorithms outperform the state-of-the-art methods in an extensive set of practical experiments. Furthermore, we obtain theoretical and empirical convergence results for the proposed algorithms. Along with the developed methodologies for graph learning in financial markets, we release an R package, called fingraph, accommodating the code and data to obtain all the experimental results. △ Less

Submitted 30 December, 2020; originally announced December 2020.

Comments: 62 pages, 25 figures

Does the $\ell_1$-norm Learn a Sparse Graph under Laplacian Constrained Graphical Models?

Authors: Jiaxi Ying, José Vinícius de M. Cardoso, Daniel P. Palomar

Abstract: We consider the problem of learning a sparse graph under the Laplacian constrained Gaussian graphical models. This problem can be formulated as a penalized maximum likelihood estimation of the Laplacian constrained precision matrix. Like in the classical graphical lasso problem, recent works made use of the $\ell_1$-norm regularization with the goal of promoting sparsity in Laplacian constrained p… ▽ More We consider the problem of learning a sparse graph under the Laplacian constrained Gaussian graphical models. This problem can be formulated as a penalized maximum likelihood estimation of the Laplacian constrained precision matrix. Like in the classical graphical lasso problem, recent works made use of the $\ell_1$-norm regularization with the goal of promoting sparsity in Laplacian constrained precision matrix estimation. However, we find that the widely used $\ell_1$-norm is not effective in imposing a sparse solution in this problem. Through empirical evidence, we observe that the number of nonzero graph weights grows with the increase of the regularization parameter. From a theoretical perspective, we prove that a large regularization parameter will surprisingly lead to a complete graph, i.e., every pair of vertices is connected by an edge. To address this issue, we introduce the nonconvex sparsity penalty, and propose a new estimator by solving a sequence of weighted $\ell_1$-norm penalized sub-problems. We establish the non-asymptotic optimization performance guarantees on both optimization error and statistical error, and prove that the proposed estimator can recover the edges correctly with a high probability. To solve each sub-problem, we develop a projected gradient descent algorithm which enjoys a linear convergence rate. Finally, an extension to learn disconnected graphs is proposed by imposing additional rank constraint. We propose a numerical algorithm based on based on the alternating direction method of multipliers, and establish its theoretical sequence convergence. Numerical experiments involving synthetic and real-world data sets demonstrate the effectiveness of the proposed method. △ Less

Submitted 5 September, 2023; v1 submitted 26 June, 2020; originally announced June 2020.

Structured Graph Learning Via Laplacian Spectral Constraints

Authors: Sandeep Kumar, Jiaxi Ying, Jos'e Vin'icius de M. Cardoso, Daniel P. Palomar

Abstract: Learning a graph with a specific structure is essential for interpretability and identification of the relationships among data. It is well known that structured graph learning from observed samples is an NP-hard combinatorial problem. In this paper, we first show that for a set of important graph families it is possible to convert the structural constraints of structure into eigenvalue constraint… ▽ More Learning a graph with a specific structure is essential for interpretability and identification of the relationships among data. It is well known that structured graph learning from observed samples is an NP-hard combinatorial problem. In this paper, we first show that for a set of important graph families it is possible to convert the structural constraints of structure into eigenvalue constraints of the graph Laplacian matrix. Then we introduce a unified graph learning framework, lying at the integration of the spectral properties of the Laplacian matrix with Gaussian graphical modeling that is capable of learning structures of a large class of graph families. The proposed algorithms are provably convergent and practically amenable for large-scale semi-supervised and unsupervised graph-based learning tasks. Extensive numerical experiments with both synthetic and real data sets demonstrate the effectiveness of the proposed methods. An R package containing code for all the experimental results is available at https://cran.r-project.org/package=spectralGraphTopology. △ Less

Submitted 24 September, 2019; originally announced September 2019.

Comments: 12 Pages, Accepted for NIPS 2019. arXiv admin note: substantial text overlap with arXiv:1904.09792

Distributed Inexact Successive Convex Approximation ADMM: Analysis-Part I

Authors: Sandeep Kumar, Ketan Rajawat, Daniel P. Palomar

Abstract: In this two-part work, we propose an algorithmic framework for solving non-convex problems whose objective function is the sum of a number of smooth component functions plus a convex (possibly non-smooth) or/and smooth (possibly non-convex) regularization function. The proposed algorithm incorporates ideas from several existing approaches such as alternate direction method of multipliers (ADMM), s… ▽ More In this two-part work, we propose an algorithmic framework for solving non-convex problems whose objective function is the sum of a number of smooth component functions plus a convex (possibly non-smooth) or/and smooth (possibly non-convex) regularization function. The proposed algorithm incorporates ideas from several existing approaches such as alternate direction method of multipliers (ADMM), successive convex approximation (SCA), distributed and asynchronous algorithms, and inexact gradient methods. Different from a number of existing approaches, however, the proposed framework is flexible enough to incorporate a class of non-convex objective functions, allow distributed operation with and without a fusion center, and include variance reduced methods as special cases. Remarkably, the proposed algorithms are robust to uncertainties arising from random, deterministic, and adversarial sources. The part I of the paper develops two variants of the algorithm under very mild assumptions and establishes first-order convergence rate guarantees. The proof developed here allows for generic errors and delays, paving the way for different variance-reduced, asynchronous, and stochastic implementations, outlined and evaluated in part II. △ Less

Submitted 21 July, 2019; originally announced July 2019.

A Unified Framework for Structured Graph Learning via Spectral Constraints

Authors: Sandeep Kumar, Jiaxi Ying, José Vinícius de M. Cardoso, Daniel Palomar

Abstract: Graph learning from data represents a canonical problem that has received substantial attention in the literature. However, insufficient work has been done in incorporating prior structural knowledge onto the learning of underlying graphical models from data. Learning a graph with a specific structure is essential for interpretability and identification of the relationships among data. Useful stru… ▽ More Graph learning from data represents a canonical problem that has received substantial attention in the literature. However, insufficient work has been done in incorporating prior structural knowledge onto the learning of underlying graphical models from data. Learning a graph with a specific structure is essential for interpretability and identification of the relationships among data. Useful structured graphs include the multi-component graph, bipartite graph, connected graph, sparse graph, and regular graph. In general, structured graph learning is an NP-hard combinatorial problem, therefore, designing a general tractable optimization method is extremely challenging. In this paper, we introduce a unified graph learning framework lying at the integration of Gaussian graphical models and spectral graph theory. To impose a particular structure on a graph, we first show how to formulate the combinatorial constraints as an analytical property of the graph matrix. Then we develop an optimization framework that leverages graph learning with specific structures via spectral constraints on graph matrices. The proposed algorithms are provably convergent, computationally efficient, and practically amenable for numerous graph-based tasks. Extensive numerical experiments with both synthetic and real data sets illustrate the effectiveness of the proposed algorithms. The code for all the simulations is made available as an open source repository. △ Less

Submitted 22 April, 2019; originally announced April 2019.

Optimization of MIMO Device-to-Device Networks via Matrix Fractional Programming: A Minorization-Maximization Approach

Authors: Kaiming Shen, Wei Yu, Licheng Zhao, Daniel P. Palomar

Abstract: Interference management is a fundamental issue in device-to-device (D2D) communications whenever the transmitter-and-receiver pairs are located in close proximity and frequencies are fully reused, so active links may severely interfere with each other. This paper devises an optimization strategy named FPLinQ to coordinate the link scheduling decisions among the interfering links, along with power… ▽ More Interference management is a fundamental issue in device-to-device (D2D) communications whenever the transmitter-and-receiver pairs are located in close proximity and frequencies are fully reused, so active links may severely interfere with each other. This paper devises an optimization strategy named FPLinQ to coordinate the link scheduling decisions among the interfering links, along with power control and beamforming. The key enabler is a novel optimization method called matrix fractional programming (FP) that generalizes previous scalar and vector forms of FP in allowing multiple data streams per link. From a theoretical perspective, this paper provides a deeper understanding of FP by showing a connection to the minorization-maximization (MM) algorithm. From an application perspective, this paper shows that as compared to the existing methods for coordinating scheduling in the D2D network, such as FlashLinQ, ITLinQ, and ITLinQ+, the proposed FPLinQ approach is more general in allowing multiple antennas at both the transmitters and the receivers, and further in allowing arbitrary and multiple possible associations between the devices via matching. Numerical results show that FPLinQ significantly outperforms the previous state-of-the-art in a typical D2D communication environment. △ Less

Submitted 29 March, 2019; v1 submitted 16 August, 2018; originally announced August 2018.

Comments: 14 pages, 8 figures

Sparse Reduced Rank Regression With Nonconvex Regularization

Authors: Ziping Zhao, Daniel P. Palomar

Abstract: In this paper, the estimation problem for sparse reduced rank regression (SRRR) model is considered. The SRRR model is widely used for dimension reduction and variable selection with applications in signal processing, econometrics, etc. The problem is formulated to minimize the least squares loss with a sparsity-inducing penalty considering an orthogonality constraint. Convex sparsity-inducing fun… ▽ More In this paper, the estimation problem for sparse reduced rank regression (SRRR) model is considered. The SRRR model is widely used for dimension reduction and variable selection with applications in signal processing, econometrics, etc. The problem is formulated to minimize the least squares loss with a sparsity-inducing penalty considering an orthogonality constraint. Convex sparsity-inducing functions have been used for SRRR in literature. In this work, a nonconvex function is proposed for better sparsity inducing. An efficient algorithm is developed based on the alternating minimization (or projection) method to solve the nonconvex optimization problem. Numerical simulations show that the proposed algorithm is much more efficient compared to the benchmark methods and the nonconvex function can result in a better estimation accuracy. △ Less

Submitted 20 March, 2018; originally announced March 2018.

Comments: 13 pages, 5 figures

MIMO Transmit Beampattern Matching Under Waveform Constraints

Authors: Ziping Zhao, Daniel P. Palomar

Abstract: In this paper, the multiple-input multiple-output (MIMO) transmit beampattern matching problem is considered. The problem is formulated to approximate a desired transmit beampattern (i.e., an energy distribution in space and frequency) and to minimize the cross-correlation of signals reflected back to the array by considering different practical waveform constraints at the same time. Due to the no… ▽ More In this paper, the multiple-input multiple-output (MIMO) transmit beampattern matching problem is considered. The problem is formulated to approximate a desired transmit beampattern (i.e., an energy distribution in space and frequency) and to minimize the cross-correlation of signals reflected back to the array by considering different practical waveform constraints at the same time. Due to the nonconvexity of the objective function and the waveform constraints, the optimization problem is highly nonconvex. An efficient one-step method is proposed to solve this problem based on the majorization-minimization (MM) method. The performance of the proposed algorithms compared to the state-of-art algorithms is shown through numerical simulations. △ Less

Submitted 19 February, 2018; originally announced February 2018.

Comments: 15 pages, 3 figures, to appear in Proc. of the 2018 43rd IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

Effective Low-Complexity Optimization Methods for Joint Phase Noise and Channel Estimation in OFDM

Authors: Zhongju Wang, Prabhu Babu, Daniel P. Palomar

Abstract: Phase noise correction is crucial to exploit full advantage of orthogonal frequency division multiplexing (OFDM) in modern high-data-rate communications. OFDM channel estimation with simultaneous phase noise compensation has therefore drawn much attention and stimulated continuing efforts. Existing methods, however, either have not taken into account the fundamental properties of phase noise or ar… ▽ More Phase noise correction is crucial to exploit full advantage of orthogonal frequency division multiplexing (OFDM) in modern high-data-rate communications. OFDM channel estimation with simultaneous phase noise compensation has therefore drawn much attention and stimulated continuing efforts. Existing methods, however, either have not taken into account the fundamental properties of phase noise or are only able to provide estimates of limited applicability owing to considerable computational complexity. In this paper, we have reformulated the joint estimation problem in the time domain as opposed to existing frequency-domain approaches, which enables us to develop much more efficient algorithms using the majorization-minimization technique. In addition, we propose a method based on dimensionality reduction and the Bayesian Information Criterion (BIC) that can adapt to various phase noise levels and accomplish much lower mean squared error than the benchmarks without incurring much additional computational cost. Several numerical examples with phase noise generated by free-running oscillators or phase-locked loops demonstrate that our proposed algorithms outperform existing methods with respect to both computational efficiency and mean squared error within a large range of signal-to-noise ratios. △ Less

Submitted 5 October, 2016; originally announced October 2016.

Undersampled Phase Retrieval via Majorization-Minimization

Authors: Tianyu Qiu, Daniel P. Palomar

Abstract: In the undersampled phase retrieval problem, the goal is to recover an $N$-dimensional complex signal $\mathbf{x}$ from only $M<N$ noisy intensity measurements without phase information. This problem has drawn a lot of attention to reduce the number of required measurements since a recent theory established that $M\approx4N$ intensity measurements are necessary and sufficient to recover a generic… ▽ More In the undersampled phase retrieval problem, the goal is to recover an $N$-dimensional complex signal $\mathbf{x}$ from only $M<N$ noisy intensity measurements without phase information. This problem has drawn a lot of attention to reduce the number of required measurements since a recent theory established that $M\approx4N$ intensity measurements are necessary and sufficient to recover a generic signal $\mathbf{x}$. In this paper, we propose to exploit the sparsity in the original signal and develop low-complexity algorithms with superior performance based on the majorization-minimization (MM) framework. The proposed algorithms are preferred to existing benchmark methods since at each iteration a simple surrogate problem is solved with a closed-form solution that monotonically decreases the original objective function. Experimental results validate that our algorithms outperform existing up-to-date methods in terms of recovery probability and accuracy, under the same settings. △ Less

Submitted 9 September, 2016; originally announced September 2016.

Distributed Nonconvex Multiagent Optimization Over Time-Varying Networks

Authors: Ying Sun, Gesualdo Scutari, Daniel Palomar

Abstract: We study nonconvex distributed optimization in multiagent networks where the communications between nodes is modeled as a time-varying sequence of arbitrary digraphs. We introduce a novel broadcast-based distributed algorithmic framework for the (constrained) minimization of the sum of a smooth (possibly nonconvex and nonseparable) function, i.e., the agents' sum-utility, plus a convex (possibly n… ▽ More We study nonconvex distributed optimization in multiagent networks where the communications between nodes is modeled as a time-varying sequence of arbitrary digraphs. We introduce a novel broadcast-based distributed algorithmic framework for the (constrained) minimization of the sum of a smooth (possibly nonconvex and nonseparable) function, i.e., the agents' sum-utility, plus a convex (possibly nonsmooth and nonseparable) regularizer. The latter is usually employed to enforce some structure in the solution, typically sparsity. The proposed method hinges on Successive Convex Approximation (SCA) techniques coupled with i) a tracking mechanism instrumental to locally estimate the gradients of agents' cost functions; and ii) a novel broadcast protocol to disseminate information and distribute the computation among the agents. Asymptotic convergence to stationary solutions is established. A key feature of the proposed algorithm is that it neither requires the double-stochasticity of the consensus matrices (but only column stochasticity) nor the knowledge of the graph sequence to implement. To the best of our knowledge, the proposed framework is the first broadcast-based distributed algorithm for convex and nonconvex constrained optimization over arbitrary, time-varying digraphs. Numerical results show that our algorithm outperforms current schemes on both convex and nonconvex problems. △ Less

Submitted 14 December, 2016; v1 submitted 1 July, 2016; originally announced July 2016.

Comments: Copyright 2001 SS&C. Published in the Proceedings of the 50th annual Asilomar conference on signals, systems, and computers, Nov. 6-9, 2016, CA, USA

Joint Optimization of Power and Data Transfer in Multiuser MIMO Systems

Authors: Javier Rubio, Antonio Pascual-Iserte, Daniel P. Palomar, Andrea Goldsmith

Abstract: We present an approach to solve the nonconvex optimization problem that arises when designing the transmit covariance matrices in multiuser multiple-input multiple-output (MIMO) broadcast networks implementing simultaneous wireless information and power transfer (SWIPT). The MIMO SWIPT problem is formulated as a general multi-objective optimization problem, in which data rates and harvested powers… ▽ More We present an approach to solve the nonconvex optimization problem that arises when designing the transmit covariance matrices in multiuser multiple-input multiple-output (MIMO) broadcast networks implementing simultaneous wireless information and power transfer (SWIPT). The MIMO SWIPT problem is formulated as a general multi-objective optimization problem, in which data rates and harvested powers are optimized simultaneously. Two different approaches are applied to reformulate the (nonconvex) multi-objective problem. In the first approach, the transmitter can control the specific amount of power to be harvested by power transfer whereas in the second approach the transmitter can only control the proportion of power to be harvested among the different harvesting users. The computational complexity will also be different, with higher computational resources required in the first approach. In order to solve the resulting formulations, we propose to use the majorization-minimization (MM) approach. The idea behind this approach is to obtain a convex function that approximates the nonconvex objective and, then, solve a series of convex subproblems that will converge to a locally optimal solution of the general nonconvex multi-objective problem. The solution obtained from the MM approach is compared to the classical block-diagonalization (BD) strategy, typically used to solve the nonconvex multiuser MIMO network by forcing no interference among users. Simulation results show that the proposed approach improves over the BD approach both the system sum rate and the power harvested by users. Additionally, the computational times needed for convergence of the proposed methods are much lower than the ones required for classical gradient-based approaches. △ Less

Submitted 1 April, 2016; originally announced April 2016.

Design of PAR-Constrained Sequences for MIMO Channel Estimation via Majorization-Minimization

Authors: Zhongju Wang, Prabhu Babu, Daniel P. Palomar

Abstract: PAR-constrained sequences are widely used in communication systems and radars due to various practical needs; specifically, sequences are required to be unimodular or of low peak-to-average power ratio (PAR). For unimodular sequence design, plenty of efforts have been devoted to obtaining good correlation properties. Regarding channel estimation, however, sequences of such properties do not necess… ▽ More PAR-constrained sequences are widely used in communication systems and radars due to various practical needs; specifically, sequences are required to be unimodular or of low peak-to-average power ratio (PAR). For unimodular sequence design, plenty of efforts have been devoted to obtaining good correlation properties. Regarding channel estimation, however, sequences of such properties do not necessarily help produce optimal estimates. Tailored unimodular sequences for the specific criterion concerned are desirable especially when the prior knowledge of the channel is taken into account as well. In this paper, we formulate the problem of optimal unimodular sequence design for minimum mean square error estimation of the channel impulse response and conditional mutual information maximization, respectively. Efficient algorithms based on the majorization-minimization framework are proposed for both problems with guaranteed convergence. As the unimodular constraint is a special case of the low PAR constraint, optimal sequences of low PAR are also considered. Numerical examples are provided to show the performance of the proposed training sequences, with the efficiency of the derived algorithms demonstrated. △ Less

Submitted 29 February, 2016; originally announced February 2016.

Journal ref: IEEETrans.SignalProcess. 64 (2016) 6132-6144

Orthogonal Sparse PCA and Covariance Estimation via Procrustes Reformulation

Authors: Konstantinos Benidis, Ying Sun, Prabhu Babu, Daniel P. Palomar

Abstract: The problem of estimating sparse eigenvectors of a symmetric matrix attracts a lot of attention in many applications, especially those with high dimensional data set. While classical eigenvectors can be obtained as the solution of a maximization problem, existing approaches formulate this problem by adding a penalty term into the objective function that encourages a sparse solution. However, the r… ▽ More The problem of estimating sparse eigenvectors of a symmetric matrix attracts a lot of attention in many applications, especially those with high dimensional data set. While classical eigenvectors can be obtained as the solution of a maximization problem, existing approaches formulate this problem by adding a penalty term into the objective function that encourages a sparse solution. However, the resulting methods achieve sparsity at the expense of sacrificing the orthogonality property. In this paper, we develop a new method to estimate dominant sparse eigenvectors without trading off their orthogonality. The problem is highly non-convex and hard to handle. We apply the MM framework where we iteratively maximize a tight lower bound (surrogate function) of the objective function over the Stiefel manifold. The inner maximization problem turns out to be a rectangular Procrustes problem, which has a closed form solution. In addition, we propose a method to improve the covariance estimation problem when its underlying eigenvectors are known to be sparse. We use the eigenvalue decomposition of the covariance matrix to formulate an optimization problem where we impose sparsity on the corresponding eigenvectors. Numerical experiments show that the proposed eigenvector extraction algorithm matches or outperforms existing algorithms in terms of support recovery and explained variance, while the covariance estimation algorithms improve significantly the sample covariance estimator. △ Less

Submitted 12 February, 2016; originally announced February 2016.

PRIME: Phase Retrieval via Majorization-Minimization

Authors: Tianyu Qiu, Prabhu Babu, Daniel P. Palomar

Abstract: This paper considers the phase retrieval problem in which measurements consist of only the magnitude of several linear measurements of the unknown, e.g., spectral components of a time sequence. We develop low-complexity algorithms with superior performance based on the majorization-minimization (MM) framework. The proposed algorithms are referred to as PRIME: Phase Retrieval vIa the Majorization-m… ▽ More This paper considers the phase retrieval problem in which measurements consist of only the magnitude of several linear measurements of the unknown, e.g., spectral components of a time sequence. We develop low-complexity algorithms with superior performance based on the majorization-minimization (MM) framework. The proposed algorithms are referred to as PRIME: Phase Retrieval vIa the Majorization-minimization techniquE. They are preferred to existing benchmark methods since at each iteration a simple surrogate problem is solved with a closed-form solution that monotonically decreases the original objective function. In total, four algorithms are proposed using different majorization-minimization techniques. Experimental results validate that our algorithms outperform existing methods in terms of successful recovery and mean square error under various settings. △ Less

Submitted 13 November, 2015; v1 submitted 5 November, 2015; originally announced November 2015.

Optimization Methods for Designing Sequences with Low Autocorrelation Sidelobes

Authors: Junxiao Song, Prabhu Babu, Daniel P. Palomar

Abstract: Unimodular sequences with low autocorrelations are desired in many applications, especially in the area of radar and code-division multiple access (CDMA). In this paper, we propose a new algorithm to design unimodular sequences with low integrated sidelobe level (ISL), which is a widely used measure of the goodness of a sequence's correlation property. The algorithm falls into the general framewor… ▽ More Unimodular sequences with low autocorrelations are desired in many applications, especially in the area of radar and code-division multiple access (CDMA). In this paper, we propose a new algorithm to design unimodular sequences with low integrated sidelobe level (ISL), which is a widely used measure of the goodness of a sequence's correlation property. The algorithm falls into the general framework of majorization-minimization (MM) algorithms and thus shares the monotonic property of such algorithms. In addition, the algorithm can be implemented via fast Fourier transform (FFT) operations and thus is computationally efficient. Furthermore, after some modifications the algorithm can be adapted to incorporate spectral constraints, which makes the design more flexible. Numerical experiments show that the proposed algorithms outperform existing algorithms in terms of both the quality of designed sequences and the computational complexity. △ Less

Submitted 26 December, 2014; originally announced January 2015.

A Parallel Stochastic Approximation Method for Nonconvex Multi-Agent Optimization Problems

Authors: Yang Yang, Gesualdo Scutari, Daniel P. Palomar, Marius Pesavento

Abstract: Consider the problem of minimizing the expected value of a (possibly nonconvex) cost function parameterized by a random (vector) variable, when the expectation cannot be computed accurately (e.g., because the statistics of the random variables are unknown and/or the computational complexity is prohibitive). Classical sample stochastic gradient methods for solving this problem may empirically suffe… ▽ More Consider the problem of minimizing the expected value of a (possibly nonconvex) cost function parameterized by a random (vector) variable, when the expectation cannot be computed accurately (e.g., because the statistics of the random variables are unknown and/or the computational complexity is prohibitive). Classical sample stochastic gradient methods for solving this problem may empirically suffer from slow convergence. In this paper, we propose for the first time a stochastic parallel Successive Convex Approximation-based (best-response) algorithmic framework for general nonconvex stochastic sum-utility optimization problems, which arise naturally in the design of multi-agent systems. The proposed novel decomposition enables all users to update their optimization variables in parallel by solving a sequence of strongly convex subproblems, one for each user. Almost surely convergence to stationary points is proved. We then customize our algorithmic framework to solve the stochastic sum rate maximization problem over Single-Input-Single-Output (SISO) frequency-selective interference channels, multiple-input-multiple-output (MIMO) interference channels, and MIMO multiple-access channels. Numerical results show that our algorithms are much faster than state-of-the-art stochastic gradient schemes while achieving the same (or better) sum-rates. △ Less

Submitted 21 October, 2014; v1 submitted 19 October, 2014; originally announced October 2014.

Comments: Part of this work has been presented at IEEE SPAWC 2013

Sparse Generalized Eigenvalue Problem via Smooth Optimization

Authors: Junxiao Song, Prabhu Babu, Daniel P. Palomar

Abstract: In this paper, we consider an $\ell_{0}$-norm penalized formulation of the generalized eigenvalue problem (GEP), aimed at extracting the leading sparse generalized eigenvector of a matrix pair. The formulation involves maximization of a discontinuous nonconcave objective function over a nonconvex constraint set, and is therefore computationally intractable. To tackle the problem, we first approxim… ▽ More In this paper, we consider an $\ell_{0}$-norm penalized formulation of the generalized eigenvalue problem (GEP), aimed at extracting the leading sparse generalized eigenvector of a matrix pair. The formulation involves maximization of a discontinuous nonconcave objective function over a nonconvex constraint set, and is therefore computationally intractable. To tackle the problem, we first approximate the $\ell_{0}$-norm by a continuous surrogate function. Then an algorithm is developed via iteratively majorizing the surrogate function by a quadratic separable function, which at each iteration reduces to a regular generalized eigenvalue problem. A preconditioned steepest ascent algorithm for finding the leading generalized eigenvector is provided. A systematic way based on smoothing is proposed to deal with the "singularity issue" that arises when a quadratic function is used to majorize the nondifferentiable surrogate function. For sparse GEPs with special structure, algorithms that admit a closed-form solution at every iteration are derived. Numerical experiments show that the proposed algorithms match or outperform existing algorithms in terms of computational complexity and support recovery. △ Less

Submitted 18 November, 2014; v1 submitted 28 August, 2014; originally announced August 2014.

Convex separable problems with linear and box constraints in signal processing and communications

Authors: Antonio A. D'Amico, Luca Sanguinetti, Daniel P. Palomar

Abstract: In this work, we focus on separable convex optimization problems with box constraints and a set of triangular linear constraints. The solution is given in closed-form as a function of some Lagrange multipliers that can be computed through an iterative procedure in a finite number of steps. Graphical interpretations are given casting valuable insights into the proposed algorithm and allowing to ret… ▽ More In this work, we focus on separable convex optimization problems with box constraints and a set of triangular linear constraints. The solution is given in closed-form as a function of some Lagrange multipliers that can be computed through an iterative procedure in a finite number of steps. Graphical interpretations are given casting valuable insights into the proposed algorithm and allowing to retain some of the intuition spelled out by the water-filling policy. It turns out that it is not only general enough to compute the solution to different instances of the problem at hand but also remarkably simple in the way it operates. We also show how some power allocation problems in signal processing and communications can be solved with the proposed algorithm. △ Less

Submitted 16 July, 2014; originally announced July 2014.

Comments: submitted to IEEE Trans. Signal Process

Convex separable problems with linear and box constraints

Authors: Antonio A. D'Amico, Luca Sanguinetti, Daniel P. Palomar

Abstract: In this work, we focus on separable convex optimization problems with linear and box constraints and compute the solution in closed-form as a function of some Lagrange multipliers that can be easily computed in a finite number of iterations. This allows us to bridge the gap between a wide family of power allocation problems of practical interest in signal processing and communications and their ef… ▽ More In this work, we focus on separable convex optimization problems with linear and box constraints and compute the solution in closed-form as a function of some Lagrange multipliers that can be easily computed in a finite number of iterations. This allows us to bridge the gap between a wide family of power allocation problems of practical interest in signal processing and communications and their efficient implementation in practice. △ Less

Submitted 22 March, 2014; originally announced March 2014.

Comments: 5 pages, 2 figures. Published at IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP 2014)

Decomposition by Partial Linearization: Parallel Optimization of Multi-Agent Systems

Authors: Gesualdo Scutari, Francisco Facchinei, Peiran Song, Daniel P. Palomar, Jong-Shi Pang

Abstract: We propose a novel decomposition framework for the distributed optimization of general nonconvex sum-utility functions arising naturally in the system design of wireless multiuser interfering systems. Our main contributions are: i) the development of the first class of (inexact) Jacobi best-response algorithms with provable convergence, where all the users simultaneously and iteratively solve a su… ▽ More We propose a novel decomposition framework for the distributed optimization of general nonconvex sum-utility functions arising naturally in the system design of wireless multiuser interfering systems. Our main contributions are: i) the development of the first class of (inexact) Jacobi best-response algorithms with provable convergence, where all the users simultaneously and iteratively solve a suitably convexified version of the original sum-utility optimization problem; ii) the derivation of a general dynamic pricing mechanism that provides a unified view of existing pricing schemes that are based, instead, on heuristics; and iii) a framework that can be easily particularized to well-known applications, giving rise to very efficient practical (Jacobi or Gauss-Seidel) algorithms that outperform existing adhoc methods proposed for very specific problems. Interestingly, our framework contains as special cases well-known gradient algorithms for nonconvex sum-utility problems, and many blockcoordinate descent schemes for convex functions. △ Less

Submitted 19 September, 2013; v1 submitted 4 February, 2013; originally announced February 2013.

Comments: submitted to IEEE Transactions on Signal Processing

Real and Complex Monotone Communication Games

Authors: Gesualdo Scutari, Francisco Facchinei, Jong-Shi Pang, Daniel P. Palomar

Abstract: Noncooperative game-theoretic tools have been increasingly used to study many important resource allocation problems in communications, networking, smart grids, and portfolio optimization. In this paper, we consider a general class of convex Nash Equilibrium Problems (NEPs), where each player aims to solve an arbitrary smooth convex optimization problem. Differently from most of current works, we… ▽ More Noncooperative game-theoretic tools have been increasingly used to study many important resource allocation problems in communications, networking, smart grids, and portfolio optimization. In this paper, we consider a general class of convex Nash Equilibrium Problems (NEPs), where each player aims to solve an arbitrary smooth convex optimization problem. Differently from most of current works, we do not assume any specific structure for the players' problems, and we allow the optimization variables of the players to be matrices in the complex domain. Our main contribution is the design of a novel class of distributed (asynchronous) best-response- algorithms suitable for solving the proposed NEPs, even in the presence of multiple solutions. The new methods, whose convergence analysis is based on Variational Inequality (VI) techniques, can select, among all the equilibria of a game, those that optimize a given performance criterion, at the cost of limited signaling among the players. This is a major departure from existing best-response algorithms, whose convergence conditions imply the uniqueness of the NE. Some of our results hinge on the use of VI problems directly in the complex domain; the study of these new kind of VIs also represents a noteworthy innovative contribution. We then apply the developed methods to solve some new generalizations of SISO and MIMO games in cognitive radios and femtocell systems, showing a considerable performance improvement over classical pure noncooperative schemes. △ Less

Submitted 14 December, 2013; v1 submitted 26 December, 2012; originally announced December 2012.

Comments: to appear on IEEE Transactions in Information Theory

On MMSE Properties and I-MMSE Implications in Parallel MIMO Gaussian Channels

Authors: Ronit Bustin, Miquel Payaro, Daniel Palomar, Shlomo Shamai

Abstract: The scalar additive Gaussian noise channel has the "single crossing point" property between the minimum-mean square error (MMSE) in the estimation of the input given the channel output, assuming a Gaussian input to the channel, and the MMSE assuming an arbitrary input. This paper extends the result to the parallel MIMO additive Gaussian channel in three phases: i) The channel matrix is the identit… ▽ More The scalar additive Gaussian noise channel has the "single crossing point" property between the minimum-mean square error (MMSE) in the estimation of the input given the channel output, assuming a Gaussian input to the channel, and the MMSE assuming an arbitrary input. This paper extends the result to the parallel MIMO additive Gaussian channel in three phases: i) The channel matrix is the identity matrix, and we limit the Gaussian input to a vector of Gaussian i.i.d. elements. The "single crossing point" property is with respect to the snr (as in the scalar case). ii) The channel matrix is arbitrary, the Gaussian input is limited to an independent Gaussian input. A "single crossing point" property is derived for each diagonal element of the MMSE matrix. iii) The Gaussian input is allowed to be an arbitrary Gaussian random vector. A "single crossing point" property is derived for each eigenvalue of the MMSE matrix. These three extensions are then translated to new information theoretic properties on the mutual information, using the fundamental relationship between estimation theory and information theory. The results of the last phase are also translated to a new property of Fisher's information. Finally, the applicability of all three extensions on information theoretic problems is demonstrated through: a proof of a special case of Shannon's vector EPI, a converse proof of the capacity region of the parallel degraded MIMO broadcast channel (BC) under per-antenna power constrains and under covariance constraints, and a converse proof of the capacity region of the compound parallel degraded MIMO BC under covariance constraint. △ Less

Submitted 26 March, 2012; originally announced March 2012.

Comments: Submitted to the IEEE Transactions on Information Theory

Performance analysis and optimal selection of large mean-variance portfolios under estimation risk

Authors: Francisco Rubio, Xavier Mestre, Daniel P. Palomar

Abstract: We study the consistency of sample mean-variance portfolios of arbitrarily high dimension that are based on Bayesian or shrinkage estimation of the input parameters as well as weighted sampling. In an asymptotic setting where the number of assets remains comparable in magnitude to the sample size, we provide a characterization of the estimation risk by providing deterministic equivalents of the po… ▽ More We study the consistency of sample mean-variance portfolios of arbitrarily high dimension that are based on Bayesian or shrinkage estimation of the input parameters as well as weighted sampling. In an asymptotic setting where the number of assets remains comparable in magnitude to the sample size, we provide a characterization of the estimation risk by providing deterministic equivalents of the portfolio out-of-sample performance in terms of the underlying investment scenario. The previous estimates represent a means of quantifying the amount of risk underestimation and return overestimation of improved portfolio constructions beyond standard ones. Well-known for the latter, if not corrected, these deviations lead to inaccurate and overly optimistic Sharpe-based investment decisions. Our results are based on recent contributions in the field of random matrix theory. Along with the asymptotic analysis, the analytical framework allows us to find bias corrections improving on the achieved out-of-sample performance of typical portfolio constructions. Some numerical simulations validate our theoretical findings. △ Less

Submitted 16 October, 2011; originally announced October 2011.

On MMSE Properties and I-MMSE Implications in Parallel MIMO Gaussian Channels

Authors: Ronit Bustin, Miquel Payaró, Daniel P. Palomar, Shlomo Shamai

Abstract: This paper extends the "single crossing point" property of the scalar MMSE function, derived by Guo, Shamai and Verdú (first presented in ISIT 2008), to the parallel degraded MIMO scenario. It is shown that the matrix Q(t), which is the difference between the MMSE assuming a Gaussian input and the MMSE assuming an arbitrary input, has, at most, a single crossing point for each of its eigenvalues.… ▽ More This paper extends the "single crossing point" property of the scalar MMSE function, derived by Guo, Shamai and Verdú (first presented in ISIT 2008), to the parallel degraded MIMO scenario. It is shown that the matrix Q(t), which is the difference between the MMSE assuming a Gaussian input and the MMSE assuming an arbitrary input, has, at most, a single crossing point for each of its eigenvalues. Together with the I-MMSE relationship, a fundamental connection between Information Theory and Estimation Theory, this new property is employed to derive results in Information Theory. As a simple application of this property we provide an alternative converse proof for the broadcast channel (BC) capacity region under covariance constraint in this specific setting. △ Less

Submitted 26 April, 2010; originally announced April 2010.

Comments: 5 pages, to appear in ISIT 2010, Austin, Texas

On optimal precoding in linear vector Gaussian channels with arbitrary input distribution

Authors: Miquel Payaró, Daniel P. Palomar

Abstract: The design of the precoder the maximizes the mutual information in linear vector Gaussian channels with an arbitrary input distribution is studied. Precisely, the precoder optimal left singular vectors and singular values are derived. The characterization of the right singular vectors is left, in general, as an open problem whose computational complexity is then studied in three cases: Gaussian… ▽ More The design of the precoder the maximizes the mutual information in linear vector Gaussian channels with an arbitrary input distribution is studied. Precisely, the precoder optimal left singular vectors and singular values are derived. The characterization of the right singular vectors is left, in general, as an open problem whose computational complexity is then studied in three cases: Gaussian signaling, low SNR, and high SNR. For the Gaussian signaling case and the low SNR regime, the dependence of the mutual information on the right singular vectors vanishes, making the optimal precoder design problem easy to solve. In the high SNR regime, however, the dependence on the right singular vectors cannot be avoided and we show the difficulty of computing the optimal precoder through an NP-hardness analysis. △ Less

Submitted 30 April, 2009; originally announced April 2009.

Comments: Proceedings of the 2009 IEEE International Symposium on Information Theory, Seoul, Korea, June 28 - July 3, 2009

MSC Class: 94A15; 94A05

Hessian and concavity of mutual information, differential entropy, and entropy power in linear vector Gaussian channels

Authors: M. Payaró, D. P. Palomar

Abstract: Within the framework of linear vector Gaussian channels with arbitrary signaling, closed-form expressions for the Jacobian of the minimum mean square error and Fisher information matrices with respect to arbitrary parameters of the system are calculated in this paper. Capitalizing on prior research where the minimum mean square error and Fisher information matrices were linked to information-the… ▽ More Within the framework of linear vector Gaussian channels with arbitrary signaling, closed-form expressions for the Jacobian of the minimum mean square error and Fisher information matrices with respect to arbitrary parameters of the system are calculated in this paper. Capitalizing on prior research where the minimum mean square error and Fisher information matrices were linked to information-theoretic quantities through differentiation, closed-form expressions for the Hessian of the mutual information and the differential entropy are derived. These expressions are then used to assess the concavity properties of mutual information and differential entropy under different channel conditions and also to derive a multivariate version of the entropy power inequality due to Costa. △ Less

Submitted 11 March, 2009; originally announced March 2009.

Comments: 33 pages, 2 figures. A shorter version of this paper is to appear in IEEE Transactions on Information Theory

MSC Class: 94A15; 94A05

The MIMO Iterative Waterfilling Algorithm

Authors: Gesualdo Scutari, Daniel P. Palomar, Sergio Barbarossa

Abstract: This paper considers the non-cooperative maximization of mutual information in the vector Gaussian interference channel in a fully distributed fashion via game theory. This problem has been widely studied in a number of works during the past decade for frequency-selective channels, and recently for the more general MIMO case, for which the state-of-the art results are valid only for nonsingular… ▽ More This paper considers the non-cooperative maximization of mutual information in the vector Gaussian interference channel in a fully distributed fashion via game theory. This problem has been widely studied in a number of works during the past decade for frequency-selective channels, and recently for the more general MIMO case, for which the state-of-the art results are valid only for nonsingular square channel matrices. Surprisingly, these results do not hold true when the channel matrices are rectangular and/or rank deficient matrices. The goal of this paper is to provide a complete characterization of the MIMO game for arbitrary channel matrices, in terms of conditions guaranteeing both the uniqueness of the Nash equilibrium and the convergence of asynchronous distributed iterative waterfilling algorithms. Our analysis hinges on new technical intermediate results, such as a new expression for the MIMO waterfilling projection valid (also) for singular matrices, a mean-value theorem for complex matrix-valued functions, and a general contraction theorem for the multiuser MIMO watefilling mapping valid for arbitrary channel matrices. The quite surprising result is that uniqueness/convergence conditions in the case of tall (possibly singular) channel matrices are more restrictive than those required in the case of (full rank) fat channel matrices. We also propose a modified game and algorithm with milder conditions for the uniqueness of the equilibrium and convergence, and virtually the same performance (in terms of Nash equilibria) of the original game. △ Less

Submitted 12 December, 2008; originally announced December 2008.

Comments: IEEE Transactions on Signal Processing (accepted)

Cognitive MIMO Radio: A Competitive Optimality Design Based on Subspace Projections

Authors: Gesualdo Scutari, Daniel P. Palomar, Sergio Barbarossa

Abstract: Cognitive MIMO Radio: A Competitive Optimality Design Based on Subspace Projections Cognitive MIMO Radio: A Competitive Optimality Design Based on Subspace Projections △ Less

Submitted 7 August, 2008; originally announced August 2008.

Journal ref: IEEE Signal Processing Magazine Special Issue on Signal Processing for Cognitive Radio Networks, November 2008

Competitive Design of Multiuser MIMO Systems based on Game Theory: A Unified View

Authors: Gesualdo Scutari, Daniel P. Palomar, Sergio Barbarossa

Abstract: This paper considers the noncooperative maximization of mutual information in the Gaussian interference channel in a fully distributed fashion via game theory. This problem has been studied in a number of papers during the past decade for the case of frequency-selective channels. A variety of conditions guaranteeing the uniqueness of the Nash Equilibrium (NE) and convergence of many different di… ▽ More This paper considers the noncooperative maximization of mutual information in the Gaussian interference channel in a fully distributed fashion via game theory. This problem has been studied in a number of papers during the past decade for the case of frequency-selective channels. A variety of conditions guaranteeing the uniqueness of the Nash Equilibrium (NE) and convergence of many different distributed algorithms have been derived. In this paper we provide a unified view of the state-of-the-art results, showing that most of the techniques proposed in the literature to study the game, even though apparently different, can be unified using our recent interpretation of the waterfilling operator as a projection onto a proper polyhedral set. Based on this interpretation, we then provide a mathematical framework, useful to derive a unified set of sufficient conditions guaranteeing the uniqueness of the NE and the global convergence of waterfilling based asynchronous distributed algorithms. The proposed mathematical framework is also instrumental to study the extension of the game to the more general MIMO case, for which only few results are available in the current literature. The resulting algorithm is, similarly to the frequency-selective case, an iterative asynchronous MIMO waterfilling algorithm. The proof of convergence hinges again on the interpretation of the MIMO waterfilling as a matrix projection, which is the natural generalization of our results obtained for the waterfilling mapping in the frequency-selective case. △ Less

Submitted 9 June, 2008; originally announced June 2008.

Comments: To appear on IEEE Journal on Selected Areas in Communications (JSAC), September 2008

A multivariate generalization of Costa's entropy power inequality

Authors: M. Payaró, D. P. Palomar

Abstract: A simple multivariate version of Costa's entropy power inequality is proved. In particular, it is shown that if independent white Gaussian noise is added to an arbitrary multivariate signal, the entropy power of the resulting random variable is a multidimensional concave function of the individual variances of the components of the signal. As a side result, we also give an expression for the Hes… ▽ More A simple multivariate version of Costa's entropy power inequality is proved. In particular, it is shown that if independent white Gaussian noise is added to an arbitrary multivariate signal, the entropy power of the resulting random variable is a multidimensional concave function of the individual variances of the components of the signal. As a side result, we also give an expression for the Hessian matrix of the entropy and entropy power functions with respect to the variances of the signal components, which is an interesting result in its own right. △ Less

Submitted 28 April, 2008; originally announced April 2008.

Comments: Proceedings of the 2008 IEEE International Symposium on Information Theory, Toronto, ON, Canada, July 6 - 11, 2008

Asynchronous Iterative Waterfilling for Gaussian Frequency-Selective Interference Channels

Authors: Gesualdo Scutari, Daniel P. Palomar, Sergio Barbarossa

Abstract: This paper considers the maximization of information rates for the Gaussian frequency-selective interference channel, subject to power and spectral mask constraints on each link. To derive decentralized solutions that do not require any cooperation among the users, the optimization problem is formulated as a static noncooperative game of complete information. To achieve the so-called Nash equili… ▽ More This paper considers the maximization of information rates for the Gaussian frequency-selective interference channel, subject to power and spectral mask constraints on each link. To derive decentralized solutions that do not require any cooperation among the users, the optimization problem is formulated as a static noncooperative game of complete information. To achieve the so-called Nash equilibria of the game, we propose a new distributed algorithm called asynchronous iterative waterfilling algorithm. In this algorithm, the users update their power spectral density in a completely distributed and asynchronous way: some users may update their power allocation more frequently than others and they may even use outdated measurements of the received interference. The proposed algorithm represents a unified framework that encompasses and generalizes all known iterative waterfilling algorithms, e.g., sequential and simultaneous versions. The main result of the paper consists of a unified set of conditions that guarantee the global converge of the proposed algorithm to the (unique) Nash equilibrium of the game. △ Less

Submitted 16 January, 2008; originally announced January 2008.

Comments: Submitted to IEEE Transactions on Information Theory, August 22, 2006. Revised September 25, 2007. Accepted January 14, 2008. To appear on IEEE Transactions on Information Theory, 2008

Optimal Linear Precoding Strategies for Wideband Non-Cooperative Systems based on Game Theory-Part II: Algorithms

Authors: Gesualdo Scutari, Daniel P. Palomar, Sergio Barbarossa

Abstract: In this two-part paper, we address the problem of finding the optimal precoding/multiplexing scheme for a set of non-cooperative links sharing the same physical resources, e.g., time and bandwidth. We consider two alternative optimization problems: P.1) the maximization of mutual information on each link, given constraints on the transmit power and spectral mask; and P.2) the maximization of the… ▽ More In this two-part paper, we address the problem of finding the optimal precoding/multiplexing scheme for a set of non-cooperative links sharing the same physical resources, e.g., time and bandwidth. We consider two alternative optimization problems: P.1) the maximization of mutual information on each link, given constraints on the transmit power and spectral mask; and P.2) the maximization of the transmission rate on each link, using finite order constellations, under the same constraints as in P.1, plus a constraint on the maximum average error probability on each link. Aiming at finding decentralized strategies, we adopted as optimality criterion the achievement of a Nash equilibrium and thus we formulated both problems P.1 and P.2 as strategic noncooperative (matrix-valued) games. In Part I of this two-part paper, after deriving the optimal structure of the linear transceivers for both games, we provided a unified set of sufficient conditions that guarantee the uniqueness of the Nash equilibrium. In this Part II, we focus on the achievement of the equilibrium and propose alternative distributed iterative algorithms that solve both games. Specifically, the new proposed algorithms are the following: 1) the sequential and simultaneous iterative waterfilling based algorithms, incorporating spectral mask constraints; 2) the sequential and simultaneous gradient projection based algorithms, establishing an interesting link with variational inequality problems. Our main contribution is to provide sufficient conditions for the global convergence of all the proposed algorithms which, although derived under stronger constraints, incorporating for example spectral mask constraints, have a broader validity than the convergence conditions known in the current literature for the sequential iterative waterfilling algorithm. △ Less

Submitted 5 July, 2007; originally announced July 2007.

Comments: Paper submitted to IEEE Transactions on Signal Processing, February 22, 2006. Revised March 26, 2007. Accepted June 5, 2007. To appear on IEEE Transactions on Signal Processing, 2007

Optimal Linear Precoding Strategies for Wideband Non-Cooperative Systems based on Game Theory-Part I: Nash Equilibria

Authors: Gesualdo Scutari, D. P. Palomar, S. Barbarossa

Abstract: In this two-parts paper we propose a decentralized strategy, based on a game-theoretic formulation, to find out the optimal precoding/multiplexing matrices for a multipoint-to-multipoint communication system composed of a set of wideband links sharing the same physical resources, i.e., time and bandwidth. We assume, as optimality criterion, the achievement of a Nash equilibrium and consider two… ▽ More In this two-parts paper we propose a decentralized strategy, based on a game-theoretic formulation, to find out the optimal precoding/multiplexing matrices for a multipoint-to-multipoint communication system composed of a set of wideband links sharing the same physical resources, i.e., time and bandwidth. We assume, as optimality criterion, the achievement of a Nash equilibrium and consider two alternative optimization problems: 1) the competitive maximization of mutual information on each link, given constraints on the transmit power and on the spectral mask imposed by the radio spectrum regulatory bodies; and 2) the competitive maximization of the transmission rate, using finite order constellations, under the same constraints as above, plus a constraint on the average error probability. In Part I of the paper, we start by showing that the solution set of both noncooperative games is always nonempty and contains only pure strategies. Then, we prove that the optimal precoding/multiplexing scheme for both games leads to a channel diagonalizing structure, so that both matrix-valued problems can be recast in a simpler unified vector power control game, with no performance penalty. Thus, we study this simpler game and derive sufficient conditions ensuring the uniqueness of the Nash equilibrium. Interestingly, although derived under stronger constraints, incorporating for example spectral mask constraints, our uniqueness conditions have broader validity than previously known conditions. Finally, we assess the goodness of the proposed decentralized strategy by comparing its performance with the performance of a Pareto-optimal centralized scheme. To reach the Nash equilibria of the game, in Part II, we propose alternative distributed algorithms, along with their convergence conditions. △ Less

Submitted 4 July, 2007; originally announced July 2007.

Comments: Paper submitted to IEEE Transactions on Signal Processing, September 22, 2005. Revised March 14, 2007. Accepted June 5, 2007. To be published on IEEE Transactions on Signal Processing, 2007. To appear on IEEE Transactions on Signal Processing, 2007