Showing 1–50 of 62 results for author: Qin, J

Shape-restricted transfer learning analysis for generalized linear regression model

Authors: Pengfei Li, Tao Yu, Chixiang Chen, Jing Qin

Abstract: Transfer learning has emerged as a highly sought-after and actively pursued research area within the statistical community. The core concept of transfer learning involves leveraging insights and information from auxiliary datasets to enhance the analysis of the primary dataset of interest. In this paper, our focus is on datasets originating from distinct yet interconnected distributions. We assume… ▽ More Transfer learning has emerged as a highly sought-after and actively pursued research area within the statistical community. The core concept of transfer learning involves leveraging insights and information from auxiliary datasets to enhance the analysis of the primary dataset of interest. In this paper, our focus is on datasets originating from distinct yet interconnected distributions. We assume that the training data conforms to a standard generalized linear model, while the testing data exhibit a connection to the training data based on a prior probability shift assumption. Ultimately, we discover that the two-sample conditional means are interrelated through an unknown, nondecreasing function. We integrate the power of generalized estimating equations with the shape-restricted score function, creating a robust framework for improved inference regarding the underlying parameters. We theoretically establish the asymptotic properties of our estimator and demonstrate, through simulation studies, that our method yields more accurate parameter estimates compared to those based solely on the testing or training data. Finally, we apply our method to a real-world example. △ Less

Submitted 31 July, 2024; originally announced July 2024.

Comments: 35 pages, 2 tables, and 1 figure

Efficient estimation of partially linear additive Cox models and variance estimation under shape restrictions

Authors: Junjun Lang, Yukun Liu, Jing Qin

Abstract: Shape-restricted inferences have exhibited empirical success in various applications with survival data. However, certain works fall short in providing a rigorous theoretical justification and an easy-to-use variance estimator with theoretical guarantee. Motivated by Deng et al. (2023), this paper delves into an additive and shape-restricted partially linear Cox model for right-censored data, wher… ▽ More Shape-restricted inferences have exhibited empirical success in various applications with survival data. However, certain works fall short in providing a rigorous theoretical justification and an easy-to-use variance estimator with theoretical guarantee. Motivated by Deng et al. (2023), this paper delves into an additive and shape-restricted partially linear Cox model for right-censored data, where each additive component satisfies a specific shape restriction, encompassing monotonic increasing/decreasing and convexity/concavity. We systematically investigate the consistencies and convergence rates of the shape-restricted maximum partial likelihood estimator (SMPLE) of all the underlying parameters. We further establish the aymptotic normality and semiparametric effiency of the SMPLE for the linear covariate shift. To estimate the asymptotic variance, we propose an innovative data-splitting variance estimation method that boasts exceptional versatility and broad applicability. Our simulation results and an analysis of the Rotterdam Breast Cancer dataset demonstrate that the SMPLE has comparable performance with the maximum likelihood estimator under the Cox model when the Cox model is correct, and outperforms the latter and Huang (1999)'s method when the Cox model is violated or the hazard is nonsmooth. Meanwhile, the proposed variance estimation method usually leads to reliable interval estimates based on the SMPLE and its competitors. △ Less

Submitted 9 July, 2024; originally announced July 2024.

Time-Varying Graph Signal Recovery Using High-Order Smoothness and Adaptive Low-rankness

Authors: Weihong Guo, Yifei Lou, Jing Qin, Ming Yan

Abstract: Time-varying graph signal recovery has been widely used in many applications, including climate change, environmental hazard monitoring, and epidemic studies. It is crucial to choose appropriate regularizations to describe the characteristics of the underlying signals, such as the smoothness of the signal over the graph domain and the low-rank structure of the spatial-temporal signal modeled in a… ▽ More Time-varying graph signal recovery has been widely used in many applications, including climate change, environmental hazard monitoring, and epidemic studies. It is crucial to choose appropriate regularizations to describe the characteristics of the underlying signals, such as the smoothness of the signal over the graph domain and the low-rank structure of the spatial-temporal signal modeled in a matrix form. As one of the most popular options, the graph Laplacian is commonly adopted in designing graph regularizations for reconstructing signals defined on a graph from partially observed data. In this work, we propose a time-varying graph signal recovery method based on the high-order Sobolev smoothness and an error-function weighted nuclear norm regularization to enforce the low-rankness. Two efficient algorithms based on the alternating direction method of multipliers and iterative reweighting are proposed, and convergence of one algorithm is shown in detail. We conduct various numerical experiments on synthetic and real-world data sets to demonstrate the proposed method's effectiveness compared to the state-of-the-art in graph signal recovery. △ Less

Submitted 15 May, 2024; originally announced May 2024.

Power of $\ell_1$-Norm Regularized Kaczmarz Algorithms for High-Order Tensor Recovery

Authors: Katherine Henneberger, Jing Qin

Abstract: Tensors serve as a crucial tool in the representation and analysis of complex, multi-dimensional data. As data volumes continue to expand, there is an increasing demand for developing optimization algorithms that can directly operate on tensors to deliver fast and effective computations. Many problems in real-world applications can be formulated as the task of recovering high-order tensors charact… ▽ More Tensors serve as a crucial tool in the representation and analysis of complex, multi-dimensional data. As data volumes continue to expand, there is an increasing demand for developing optimization algorithms that can directly operate on tensors to deliver fast and effective computations. Many problems in real-world applications can be formulated as the task of recovering high-order tensors characterized by sparse and/or low-rank structures. In this work, we propose novel Kaczmarz algorithms with a power of the $\ell_1$-norm regularization for reconstructing high-order tensors by exploiting sparsity and/or low-rankness of tensor data. In addition, we develop both a block and an accelerated variant, along with a thorough convergence analysis of these algorithms. A variety of numerical experiments on both synthetic and real-world datasets demonstrate the effectiveness and significant potential of the proposed methods in image and video processing tasks, such as image sequence destriping and video deconvolution. △ Less

Submitted 13 May, 2024; originally announced May 2024.

Comments: arXiv admin note: text overlap with arXiv:2311.00783

Hyperspectral Band Selection based on Generalized 3DTV and Tensor CUR Decomposition

Authors: Katherine Henneberger, Jing Qin

Abstract: Hyperspectral Imaging (HSI) serves as an important technique in remote sensing. However, high dimensionality and data volume typically pose significant computational challenges. Band selection is essential for reducing spectral redundancy in hyperspectral imagery while retaining intrinsic critical information. In this work, we propose a novel hyperspectral band selection model by decomposing the d… ▽ More Hyperspectral Imaging (HSI) serves as an important technique in remote sensing. However, high dimensionality and data volume typically pose significant computational challenges. Band selection is essential for reducing spectral redundancy in hyperspectral imagery while retaining intrinsic critical information. In this work, we propose a novel hyperspectral band selection model by decomposing the data into a low-rank and smooth component and a sparse one. In particular, we develop a generalized 3D total variation (G3DTV) by applying the $\ell_1^p$-norm to derivatives to preserve spatial-spectral smoothness. By employing the alternating direction method of multipliers (ADMM), we derive an efficient algorithm, where the tensor low-rankness is implied by the tensor CUR decomposition. We demonstrate the effectiveness of the proposed approach through comparisons with various other state-of-the-art band selection techniques using two benchmark real-world datasets. In addition, we provide practical guidelines for parameter selection in both noise-free and noisy scenarios. △ Less

Submitted 1 May, 2024; originally announced May 2024.

On the Choice of Loss Function in Learning-based Optimal Power Flow

Authors: Ge Chen, Junjie Qin

Abstract: We analyze and contrast two ways to train machine learning models for solving AC optimal power flow (OPF) problems, distinguished with the loss functions used. The first trains a mapping from the loads to the optimal dispatch decisions, utilizing mean square error (MSE) between predicted and optimal dispatch decisions as the loss function. The other intends to learn the same mapping, but directly… ▽ More We analyze and contrast two ways to train machine learning models for solving AC optimal power flow (OPF) problems, distinguished with the loss functions used. The first trains a mapping from the loads to the optimal dispatch decisions, utilizing mean square error (MSE) between predicted and optimal dispatch decisions as the loss function. The other intends to learn the same mapping, but directly uses the OPF cost of the predicted decisions, referred to as decision loss, as the loss function. In addition to better aligning with the OPF cost which results in reduced suboptimality, the use of decision loss can circumvent feasibility issues that arise with MSE when the underlying mapping from loads to optimal dispatch is discontinuous. Since decision loss does not capture the OPF constraints, we further develop a neural network with a specific structure and introduce a modified training algorithm incorporating Lagrangian duality to improve feasibility.} This result in an improved performance measured by feasibility and suboptimality as demonstrated with an IEEE 39-bus case study. △ Less

Submitted 1 February, 2024; originally announced February 2024.

Comments: 5 pages, Accepted by PESGM2024

Neural Risk Limiting Dispatch in Power Networks: Formulation and Generalization Guarantees

Authors: Ge Chen, Junjie Qin

Abstract: Risk limiting dispatch (RLD) has been proposed as an approach that effectively trades off economic costs with operational risks for power dispatch under uncertainty. However, how to solve the RLD problem with provably near-optimal performance still remains an open problem. This paper presents a learning-based solution to this challenge. We first design a data-driven formulation for the RLD problem… ▽ More Risk limiting dispatch (RLD) has been proposed as an approach that effectively trades off economic costs with operational risks for power dispatch under uncertainty. However, how to solve the RLD problem with provably near-optimal performance still remains an open problem. This paper presents a learning-based solution to this challenge. We first design a data-driven formulation for the RLD problem, which aims to construct a decision rule that directly maps day-ahead observable information to cost-effective dispatch decisions for the future delivery interval. Unlike most existing works that follow a predict-then-optimize paradigm, this end-to-end rule bypasses the additional suboptimality introduced by separately handling prediction and optimization. We then propose neural RLD, a novel solution method to the data-driven formulation. This method leverages an L2-regularized neural network to learn the decision rule, thereby transforming the data-driven formulation into a neural network training task that can be efficiently completed by stochastic gradient descent. A theoretical performance guarantee is further established to bound the suboptimality of our method, which implies that its suboptimality approaches to zero with high probability as more samples are utilized. Simulation tests across various systems demonstrate our method's superior performance in convergence, suboptimality, and computational efficiency compared with benchmarks. △ Less

Submitted 1 February, 2024; originally announced February 2024.

Comments: 10 pages

Log-Sum Regularized Kaczmarz Algorithms for High-Order Tensor Recovery

Authors: Katherine Henneberger, Jing Qin

Abstract: Sparse and low rank tensor recovery has emerged as a significant area of research with applications in many fields such as computer vision. However, minimizing the $\ell_0$-norm of a vector or the rank of a matrix is NP-hard. Instead, their convex relaxed versions are typically adopted in practice due to the computational efficiency, e.g., log-sum penalty. In this work, we propose novel log-sum re… ▽ More Sparse and low rank tensor recovery has emerged as a significant area of research with applications in many fields such as computer vision. However, minimizing the $\ell_0$-norm of a vector or the rank of a matrix is NP-hard. Instead, their convex relaxed versions are typically adopted in practice due to the computational efficiency, e.g., log-sum penalty. In this work, we propose novel log-sum regularized Kaczmarz algorithms for recovering high-order tensors with either sparse or low-rank structures. We present block variants along with convergence analysis of the proposed algorithms. Numerical experiments on synthetic and real-world data sets demonstrate the effectiveness of the proposed methods. △ Less

Submitted 19 April, 2024; v1 submitted 1 November, 2023; originally announced November 2023.

Probabilistic Reduced-Dimensional Vector Autoregressive Modeling for Dynamics Prediction and Reconstruction with Oblique Projections

Authors: Yanfang Mo, Jiaxin Yu, S. Joe Qin

Abstract: In this paper, we propose a probabilistic reduced-dimensional vector autoregressive (PredVAR) model with oblique projections. This model partitions the measurement space into a dynamic subspace and a static subspace that do not need to be orthogonal. The partition allows us to apply an oblique projection to extract dynamic latent variables (DLVs) from high-dimensional data with maximized predictab… ▽ More In this paper, we propose a probabilistic reduced-dimensional vector autoregressive (PredVAR) model with oblique projections. This model partitions the measurement space into a dynamic subspace and a static subspace that do not need to be orthogonal. The partition allows us to apply an oblique projection to extract dynamic latent variables (DLVs) from high-dimensional data with maximized predictability. We develop an alternating iterative PredVAR algorithm that exploits the interaction between updating the latent VAR dynamics and estimating the oblique projection, using expectation maximization (EM) and a statistical constraint. In addition, the noise covariance matrices are estimated as a natural outcome of the EM method. A simulation case study of the nonlinear Lorenz oscillation system illustrates the advantages of the proposed approach over two alternatives. △ Less

Submitted 3 September, 2023; originally announced September 2023.

The Brown-Halmos theorems on the Fock space

Authors: Jie Qin

Abstract: In this paper, we extend the Brown-Halmos theorems to the Fock space and investigate the range of the Berezin transform. We observe that there are non-pluriharmonic functions $u$ that can be written as a finite sum $B(u)=\sum_lf_l\overline{g_l}$, where $f_l,g_l$ are holomorphic functions belonging to the class $\mathrm{Sym}(\mathbb{C}^n)$. In addition, we solve an open question about the zero prod… ▽ More In this paper, we extend the Brown-Halmos theorems to the Fock space and investigate the range of the Berezin transform. We observe that there are non-pluriharmonic functions $u$ that can be written as a finite sum $B(u)=\sum_lf_l\overline{g_l}$, where $f_l,g_l$ are holomorphic functions belonging to the class $\mathrm{Sym}(\mathbb{C}^n)$. In addition, we solve an open question about the zero product of Toeplitz operators, which was posed by Bauer et al. in 2015. Our results reveal that the Brown-Halmos theorems on the Fock space are more complicated than that on the classical Bergman space. △ Less

Submitted 24 September, 2023; v1 submitted 31 August, 2023; originally announced August 2023.

Comments: 19 pages

MSC Class: 30H20; 47B35

Alternating minimization for simultaneous estimation of a latent variable and identification of a linear continuous-time dynamic system

Authors: Pierre-Cyril Aubin-Frankowski, Alain Bensoussan, S. Joe Qin

Abstract: We propose an optimization formulation for the simultaneous estimation of a latent variable and the identification of a linear continuous-time dynamic system, given a single input-output pair. We justify this approach based on Bayesian maximum a posteriori estimators. Our scheme takes the form of a convex alternating minimization, over the trajectories and the dynamic model respectively. We prove… ▽ More We propose an optimization formulation for the simultaneous estimation of a latent variable and the identification of a linear continuous-time dynamic system, given a single input-output pair. We justify this approach based on Bayesian maximum a posteriori estimators. Our scheme takes the form of a convex alternating minimization, over the trajectories and the dynamic model respectively. We prove its convergence to a local minimum which verifies a two point-boundary problem for the (latent) state variable and a tensor product expression for the optimal dynamics. △ Less

Submitted 28 June, 2023; originally announced June 2023.

MSC Class: 49N10; 93B30; 62F15

Learning Unnormalized Statistical Models via Compositional Optimization

Authors: Wei Jiang, Jiayu Qin, Lingyu Wu, Changyou Chen, Tianbao Yang, Lijun Zhang

Abstract: Learning unnormalized statistical models (e.g., energy-based models) is computationally challenging due to the complexity of handling the partition function. To eschew this complexity, noise-contrastive estimation~(NCE) has been proposed by formulating the objective as the logistic loss of the real data and the artificial noise. However, as found in previous works, NCE may perform poorly in many t… ▽ More Learning unnormalized statistical models (e.g., energy-based models) is computationally challenging due to the complexity of handling the partition function. To eschew this complexity, noise-contrastive estimation~(NCE) has been proposed by formulating the objective as the logistic loss of the real data and the artificial noise. However, as found in previous works, NCE may perform poorly in many tasks due to its flat loss landscape and slow convergence. In this paper, we study it a direct approach for optimizing the negative log-likelihood of unnormalized models from the perspective of compositional optimization. To tackle the partition function, a noise distribution is introduced such that the log partition function can be written as a compositional function whose inner function can be estimated with stochastic samples. Hence, the objective can be optimized by stochastic compositional optimization algorithms. Despite being a simple method, we demonstrate that it is more favorable than NCE by (1) establishing a fast convergence rate and quantifying its dependence on the noise distribution through the variance of stochastic estimators; (2) developing better results for one-dimensional Gaussian mean estimation by showing our objective has a much favorable loss landscape and hence our method enjoys faster convergence; (3) demonstrating better performance on multiple applications, including density estimation, out-of-distribution detection, and real image generation. △ Less

Submitted 12 June, 2023; originally announced June 2023.

Stochastic Natural Thresholding Algorithms

Authors: Rachel Grotheer, Shuang Li, Anna Ma, Deanna Needell, Jing Qin

Abstract: Sparse signal recovery is one of the most fundamental problems in various applications, including medical imaging and remote sensing. Many greedy algorithms based on the family of hard thresholding operators have been developed to solve the sparse signal recovery problem. More recently, Natural Thresholding (NT) has been proposed with improved computational efficiency. This paper proposes and disc… ▽ More Sparse signal recovery is one of the most fundamental problems in various applications, including medical imaging and remote sensing. Many greedy algorithms based on the family of hard thresholding operators have been developed to solve the sparse signal recovery problem. More recently, Natural Thresholding (NT) has been proposed with improved computational efficiency. This paper proposes and discusses convergence guarantees for stochastic natural thresholding algorithms by extending the NT from the deterministic version with linear measurements to the stochastic version with a general objective function. We also conduct various numerical experiments on linear and nonlinear measurements to demonstrate the performance of StoNT. △ Less

Submitted 7 June, 2023; originally announced June 2023.

Iterative Singular Tube Hard Thresholding Algorithms for Tensor Recovery

Authors: Rachel Grotheer, Shuang Li, Anna Ma, Deanna Needell, Jing Qin

Abstract: Due to the explosive growth of large-scale data sets, tensors have been a vital tool to analyze and process high-dimensional data. Different from the matrix case, tensor decomposition has been defined in various formats, which can be further used to define the best low-rank approximation of a tensor to significantly reduce the dimensionality for signal compression and recovery. In this paper, we c… ▽ More Due to the explosive growth of large-scale data sets, tensors have been a vital tool to analyze and process high-dimensional data. Different from the matrix case, tensor decomposition has been defined in various formats, which can be further used to define the best low-rank approximation of a tensor to significantly reduce the dimensionality for signal compression and recovery. In this paper, we consider the low-rank tensor recovery problem when the tubal rank of the underlying tensor is given or estimated a priori. We propose a novel class of iterative singular tube hard thresholding algorithms for tensor recovery based on the low-tubal-rank tensor approximation, including basic, accelerated deterministic and stochastic versions. Convergence guarantees are provided along with the special case when the measurements are linear. Numerical experiments on tensor compressive sensing and color image inpainting are conducted to demonstrate convergence and computational efficiency in practice. △ Less

Submitted 26 December, 2023; v1 submitted 10 April, 2023; originally announced April 2023.

Federated Gradient Matching Pursuit

Authors: Halyun Jeong, Deanna Needell, Jing Qin

Abstract: Traditional machine learning techniques require centralizing all training data on one server or data hub. Due to the development of communication technologies and a huge amount of decentralized data on many clients, collaborative machine learning has become the main interest while providing privacy-preserving frameworks. In particular, federated learning (FL) provides such a solution to learn a sh… ▽ More Traditional machine learning techniques require centralizing all training data on one server or data hub. Due to the development of communication technologies and a huge amount of decentralized data on many clients, collaborative machine learning has become the main interest while providing privacy-preserving frameworks. In particular, federated learning (FL) provides such a solution to learn a shared model while keeping training data at local clients. On the other hand, in a wide range of machine learning and signal processing applications, the desired solution naturally has a certain structure that can be framed as sparsity with respect to a certain dictionary. This problem can be formulated as an optimization problem with sparsity constraints and solving it efficiently has been one of the primary research topics in the traditional centralized setting. In this paper, we propose a novel algorithmic framework, federated gradient matching pursuit (FedGradMP), to solve the sparsity constrained minimization problem in the FL setting. We also generalize our algorithms to accommodate various practical FL scenarios when only a subset of clients participate per round, when the local model estimation at clients could be inexact, or when the model parameters are sparse with respect to general dictionaries. Our theoretical analysis shows the linear convergence of the proposed algorithms. A variety of numerical experiments are conducted to demonstrate the great potential of the proposed framework -- fast convergence both in communication rounds and computation time for many important scenarios without sophisticated parameter tuning. △ Less

Submitted 20 February, 2023; originally announced February 2023.

Comments: Submitted to a journal

MSC Class: 65-xx; 68W15; 68W20

Semi-commutants of Toeplitz Operators on Fock-Sobolev space of Nonnegative orders

Authors: Jie Qin

Abstract: We make a progress towards describing the semi-commutants of Toeplitz operators on Fock-Sobolev spaces of nonnegative orders. We generalize the results in \cite{Bauer1,Qin}. For the certain symbol spaces, we obtain two Toeplitz operators can semi-commute only in the trivial cases, which is different from what is known for the classical Fock spaces. As an application, we consider the conjecture whi… ▽ More We make a progress towards describing the semi-commutants of Toeplitz operators on Fock-Sobolev spaces of nonnegative orders. We generalize the results in \cite{Bauer1,Qin}. For the certain symbol spaces, we obtain two Toeplitz operators can semi-commute only in the trivial cases, which is different from what is known for the classical Fock spaces. As an application, we consider the conjecture which was shown to be false for Fock space in \cite{MA}. The main results of this paper say that there is the fundamental difference between the geometries of Fock and Fock-Sobolev space. △ Less

Submitted 16 February, 2023; v1 submitted 27 November, 2022; originally announced November 2022.

Comments: 25pages

MSC Class: 47B35

Optimal Ordering Policies for Multi-Echelon Supply Networks

Authors: Jose I. Caiza, Ian Walter, Jitesh H. Panchal, Junjie Qin, Philip E. Pare

Abstract: In this paper, we formulate an optimal ordering policy as a stochastic control problem where each firm decides the amount of input goods to order from their upstream suppliers based on the current inventory level of its output good. For this purpose, we provide a closed-form solution for the optimal request of the raw materials for given a fixed production policy. We implement the proposed policy… ▽ More In this paper, we formulate an optimal ordering policy as a stochastic control problem where each firm decides the amount of input goods to order from their upstream suppliers based on the current inventory level of its output good. For this purpose, we provide a closed-form solution for the optimal request of the raw materials for given a fixed production policy. We implement the proposed policy on a 15-firm acyclic network based on a real product supply chain. We first simulate ideal demand situations, and then we implement demand-side shocks (i.e., demand levels outside of those considered in the policy formulation) and supply-side shocks (i.e., halts in production for some suppliers) to evaluate the robustness of the proposed policies. △ Less

Submitted 11 September, 2022; originally announced September 2022.

Tuning-parameter-free optimal propensity score matching approach for causal inference

Authors: Yukun Liu, Jing Qin

Abstract: Propensity score matching (PSM) is a pseudo-experimental method that uses statistical techniques to construct an artificial control group by matching each treated unit with one or more untreated units of similar characteristics. To date, the problem of determining the optimal number of matches per unit, which plays an important role in PSM, has not been adequately addressed. We propose a tuning-pa… ▽ More Propensity score matching (PSM) is a pseudo-experimental method that uses statistical techniques to construct an artificial control group by matching each treated unit with one or more untreated units of similar characteristics. To date, the problem of determining the optimal number of matches per unit, which plays an important role in PSM, has not been adequately addressed. We propose a tuning-parameter-free PSM method based on the nonparametric maximum-likelihood estimation of the propensity score under the monotonicity constraint. The estimated propensity score is piecewise constant, and therefore automatically groups data. Hence, our proposal is free of tuning parameters. The proposed estimator is asymptotically semiparametric efficient for the univariate case, and achieves this level of efficiency in the multivariate case when the outcome and the propensity score depend on the covariate in the same direction. We conclude that matching methods based on the propensity score alone cannot, in general, be efficient. △ Less

Submitted 26 May, 2022; originally announced May 2022.

Comments: 28 pages, 2 figures, 3 tables

Robust Dual-Graph Regularized Moving Object Detection

Authors: Jing Qin, Ruilong Shen, Ruihan Zhu, Biyun Xie

Abstract: Moving object detection and its associated background-foreground separation have been widely used in a lot of applications, including computer vision, transportation and surveillance. Due to the presence of the static background, a video can be naturally decomposed into a low-rank background and a sparse foreground. Many regularization techniques, such as matrix nuclear norm, have been imposed on… ▽ More Moving object detection and its associated background-foreground separation have been widely used in a lot of applications, including computer vision, transportation and surveillance. Due to the presence of the static background, a video can be naturally decomposed into a low-rank background and a sparse foreground. Many regularization techniques, such as matrix nuclear norm, have been imposed on the background. In the meanwhile, sparsity or smoothness based regularizations, such as total variation and $\ell_1$, can be imposed on the foreground. Moreover, graph Laplacians are further imposed to capture the complicated geometry of background images. Recently, weighted regularization techniques including the weighted nuclear norm regularization have been proposed in the image processing community to promote adaptive sparsity while achieving efficient performance. In this paper, we propose a robust dual-graph regularized moving object detection model based on the weighted nuclear norm regularization, which is solved by the alternating direction method of multipliers (ADMM). Numerical experiments on body movement data sets have demonstrated the effectiveness of this method in separating moving objects from background, and the great potential in robotic applications. △ Less

Submitted 25 April, 2022; originally announced April 2022.

Optimal Online Algorithms for Peak-Demand Reduction Maximization with Energy Storage

Authors: Yanfang Mo, Qiulin Lin, Minghua Chen, Si-Zhao Joe Qin

Abstract: The high proportions of demand charges in electric bills motivate large-power customers to leverage energy storage for reducing the peak procurement from the outer grid. Given limited energy storage, we expect to maximize the peak-demand reduction in an online fashion, challenged by the highly uncertain demands and renewable injections, the non-cumulative nature of peak consumption, and the coupli… ▽ More The high proportions of demand charges in electric bills motivate large-power customers to leverage energy storage for reducing the peak procurement from the outer grid. Given limited energy storage, we expect to maximize the peak-demand reduction in an online fashion, challenged by the highly uncertain demands and renewable injections, the non-cumulative nature of peak consumption, and the coupling of online decisions. In this paper, we propose an optimal online algorithm that achieves the best competitive ratio, following the idea of maintaining a constant ratio between the online and the optimal offline peak-reduction performance. We further show that the optimal competitive ratio can be computed by solving a linear number of linear-fractional programs. Moreover, we extend the algorithm to adaptively maintain the best competitive ratio given the revealed inputs and actions at each decision-making round. The adaptive algorithm retains the optimal worst-case guarantee and attains improved average-case performance. We evaluate our proposed algorithms using real-world traces and show that they obtain up to 81% peak reduction of the optimal offline benchmark. Additionally, the adaptive algorithm achieves at least 20% more peak reduction against baseline alternatives. △ Less

Submitted 10 May, 2021; originally announced May 2021.

Comments: 12 pages, 6 figures

Hand Gesture Recognition Based on a Nonconvex Regularization

Authors: Jing Qin, Joshua Ashley, Biyun Xie

Abstract: Recognition of hand gestures is one of the most fundamental tasks in human-robot interaction. Sparse representation based methods have been widely used due to their efficiency and low demands on the training data. Recently, nonconvex regularization techniques including the $\ell_{1-2}$ regularization have been proposed in the image processing community to promote sparsity while achieving efficient… ▽ More Recognition of hand gestures is one of the most fundamental tasks in human-robot interaction. Sparse representation based methods have been widely used due to their efficiency and low demands on the training data. Recently, nonconvex regularization techniques including the $\ell_{1-2}$ regularization have been proposed in the image processing community to promote sparsity while achieving efficient performance. In this paper, we propose a vision-based hand gesture recognition model based on the $\ell_{1-2}$ regularization, which is solved by the alternating direction method of multipliers (ADMM). Numerical experiments on binary and gray-scale data sets have demonstrated the effectiveness of this method in identifying hand gestures. △ Less

Submitted 25 April, 2022; v1 submitted 29 April, 2021; originally announced April 2021.

Bounded Hankel products on Fock-Sobolev spaces

Authors: Jie Qin

Abstract: Let $F^{2,m}$ denote the Fock-Sobolev space of complex plane. The purpose of this paper is to study the conjecture which was shown to be false for Fock space by Ma-Yan-Zheng-Zhu in 2019. For the certain symbol space, the main result of the paper says that the conjecture is actually true in $F^{2,m}$ if $m$ is a positive integer. Let $F^{2,m}$ denote the Fock-Sobolev space of complex plane. The purpose of this paper is to study the conjecture which was shown to be false for Fock space by Ma-Yan-Zheng-Zhu in 2019. For the certain symbol space, the main result of the paper says that the conjecture is actually true in $F^{2,m}$ if $m$ is a positive integer. △ Less

Submitted 27 November, 2022; v1 submitted 6 April, 2021; originally announced April 2021.

Comments: I Have a new paper, which recover the result in this paper

MSC Class: 47B35

Semi-commuting Toeplitz Operators on Fock-Sobolev spaces

Authors: Jie Qin

Abstract: Let $F^{2,m}(\mathbb{C})$ denote the Fock-Sobolev space of complex plane. In this paper, we characterize the semi-commutator of two Toeplitz operators on $F^{2,m}(\mathbb{C})$ is zero. The result is different from the result of Bauer, Choe, and Koo. (J. Funct. Anal., 268 (2015): 3017-3060.) Let $F^{2,m}(\mathbb{C})$ denote the Fock-Sobolev space of complex plane. In this paper, we characterize the semi-commutator of two Toeplitz operators on $F^{2,m}(\mathbb{C})$ is zero. The result is different from the result of Bauer, Choe, and Koo. (J. Funct. Anal., 268 (2015): 3017-3060.) △ Less

Submitted 22 May, 2022; v1 submitted 29 March, 2021; originally announced March 2021.

Comments: to appear in Bulletin des Sciences Math$\acute{e}$matiques

MSC Class: 47B35

Maximum pairwise-rank-likelihood-based inference for the semiparametric transformation model

Authors: Tao Yu, Pengfei Li, Baojiang Chen, Ao Yuan, Jing Qin

Abstract: In this paper, we study the linear transformation model in the most general setup. This model includes many important and popular models in statistics and econometrics as special cases. Although it has been studied for many years, the methods in the literature are based on kernel-smoothing techniques or make use of only the ranks of the responses in the estimation of the parametric components. The… ▽ More In this paper, we study the linear transformation model in the most general setup. This model includes many important and popular models in statistics and econometrics as special cases. Although it has been studied for many years, the methods in the literature are based on kernel-smoothing techniques or make use of only the ranks of the responses in the estimation of the parametric components. The former approach needs a tuning parameter, which is not easily optimally specified in practice; and the latter is computationally expensive and may not make full use of the information in the data. In this paper, we propose two methods: a pairwise rank likelihood method and a score-function-based method based on this pairwise rank likelihood. We also explore the theoretical properties of the proposed estimators. Via extensive numerical studies, we demonstrate that our methods are appealing in that the estimators are not only robust to the distribution of the random errors but also lead to mean square errors that are in many cases comparable to or smaller than those of existing methods. △ Less

Submitted 24 March, 2021; originally announced March 2021.

Comments: 6 tables and 2 figures

MSC Class: 62J02; 62J05; 62J86

Optimal Online Peak Minimization Using Energy Storage

Authors: Yanfang Mo, Qiulin Lin, Minghua Chen, Si-Zhao Joe Qin

Abstract: The significant presence of demand charges in electric bills motivates large-load customers to utilize energy storage to reduce the peak procurement from the grid. We herein study the problem of energy storage allocation for peak minimization, under the online setting where irrevocable decisions are sequentially made without knowing future demands. The problem is uniquely challenging due to (i) th… ▽ More The significant presence of demand charges in electric bills motivates large-load customers to utilize energy storage to reduce the peak procurement from the grid. We herein study the problem of energy storage allocation for peak minimization, under the online setting where irrevocable decisions are sequentially made without knowing future demands. The problem is uniquely challenging due to (i) the coupling of online decisions across time imposed by the inventory constraints and (ii) the noncumulative nature of the peak procurement. We apply the CR-Pursuit framework and address the challenges unique to our minimization problem to design an online algorithm achieving the optimal competitive ratio (CR) among all online algorithms. We show that the optimal CR can be computed in polynomial time by solving a linear number of linear-fractional problems. More importantly, we generalize our approach to develop an \emph{anytime-optimal} online algorithm that achieves the best possible CR at any epoch, given the inputs and online decisions so far. The algorithm retains the optimal worst-case performance and attains adaptive average-case performance. Trace-driven simulations show that our algorithm can decrease the peak demand by an extra 19% compared to baseline alternatives under typical settings. △ Less

Submitted 17 September, 2022; v1 submitted 26 February, 2021; originally announced March 2021.

Regularized Kaczmarz Algorithms for Tensor Recovery

Authors: Xuemei Chen, Jing Qin

Abstract: Tensor recovery has recently arisen in a lot of application fields, such as transportation, medical imaging and remote sensing. Under the assumption that signals possess sparse and/or low-rank structures, many tensor recovery methods have been developed to apply various regularization techniques together with the operator-splitting type of algorithms. Due to the unprecedented growth of data, it be… ▽ More Tensor recovery has recently arisen in a lot of application fields, such as transportation, medical imaging and remote sensing. Under the assumption that signals possess sparse and/or low-rank structures, many tensor recovery methods have been developed to apply various regularization techniques together with the operator-splitting type of algorithms. Due to the unprecedented growth of data, it becomes increasingly desirable to use streamlined algorithms to achieve real-time computation, such as stochastic optimization algorithms that have recently emerged as an efficient family of methods in machine learning. In this work, we propose a novel algorithmic framework based on the Kaczmarz algorithm for tensor recovery. We provide thorough convergence analysis and its applications from the vector case to the tensor one. Numerical results on a variety of tensor recovery applications, including sparse signal recovery, low-rank tensor recovery, image inpainting and deconvolution, illustrate the enormous potential of the proposed methods. △ Less

Submitted 12 February, 2021; originally announced February 2021.

MSC Class: 68Q25; 68R10; 68U05

Off-Street Parking for TNC Vehicles to Reduce Cruising Traffic

Authors: Sen Li, Junjie Qin, Hai Yang, Kameshwar Poolla, Pravin Varaiya

Abstract: This paper considers off-street parking for the cruising vehicles of transportation network companies (TNCs) to reduce the traffic congestion. We propose a novel business that integrates the shared parking service into the TNC platform. In the proposed model, the platform (a) provides interfaces that connect passengers, drivers and garage operators (commercial or private garages); (b) determines t… ▽ More This paper considers off-street parking for the cruising vehicles of transportation network companies (TNCs) to reduce the traffic congestion. We propose a novel business that integrates the shared parking service into the TNC platform. In the proposed model, the platform (a) provides interfaces that connect passengers, drivers and garage operators (commercial or private garages); (b) determines the ride fare, driver payment, and parking rates; (c) matches passengers to TNC vehicles for ride-hailing services; and (d) matches vacant TNC vehicles to unoccupied parking garages to reduce the cruising cost. A queuing-theoretic model is proposed to capture the matching process of passengers, drivers, and parking garages. A market-equilibrium model is developed to capture the incentives of the passengers, drivers, and garage operators. An optimization-based model is formulated to capture the optimal pricing of the TNC platform. Through a realistic case study, we show that the proposed business model will offer a Pareto improvement that benefits all stakeholders, which leads to higher passenger surplus, higher drivers surplus, higher garage operator surplus, higher platform profit, and reduced traffic congestion. △ Less

Submitted 31 July, 2020; originally announced July 2020.

A Novel Regularization Based on the Error Function for Sparse Recovery

Authors: Weihong Guo, Yifei Lou, Jing Qin, Ming Yan

Abstract: Regularization plays an important role in solving ill-posed problems by adding extra information about the desired solution, such as sparsity. Many regularization terms usually involve some vector norm, e.g., $L_1$ and $L_2$ norms. In this paper, we propose a novel regularization framework that uses the error function to approximate the unit step function. It can be considered as a surrogate funct… ▽ More Regularization plays an important role in solving ill-posed problems by adding extra information about the desired solution, such as sparsity. Many regularization terms usually involve some vector norm, e.g., $L_1$ and $L_2$ norms. In this paper, we propose a novel regularization framework that uses the error function to approximate the unit step function. It can be considered as a surrogate function for the $L_0$ norm. The asymptotic behavior of the error function with respect to its intrinsic parameter indicates that the proposed regularization can approximate the standard $L_0$, $L_1$ norms as the parameter approaches to $0$ and $\infty,$ respectively. Statistically, it is also less biased than the $L_1$ approach. We then incorporate the error function into either a constrained or an unconstrained model when recovering a sparse signal from an under-determined linear system. Computationally, both problems can be solved via an iterative reweighted $L_1$ (IRL1) algorithm with guaranteed convergence. A large number of experimental results demonstrate that the proposed approach outperforms the state-of-the-art methods in various sparse recovery scenarios. △ Less

Submitted 6 July, 2020; originally announced July 2020.

MSC Class: 49N45; 90C05; 90C26

An immersed interface-lattice Boltzmann method for fluid-structure interaction

Authors: Jianhua Qin, Ebrahim M. Kolahdouz, Boyce E. Griffith

Abstract: An immersed interface-lattice Boltzmann method (II-LBM) is developed for modelling fluid-structure systems. The key element of this approach is the determination of the jump conditions that are satisfied by the distribution functions within the framework of the lattice Boltzmann method when forces are imposed along a surface immersed in an incompressible fluid. In this initial II-LBM, the disconti… ▽ More An immersed interface-lattice Boltzmann method (II-LBM) is developed for modelling fluid-structure systems. The key element of this approach is the determination of the jump conditions that are satisfied by the distribution functions within the framework of the lattice Boltzmann method when forces are imposed along a surface immersed in an incompressible fluid. In this initial II-LBM, the discontinuity related to the normal portion of the interfacial force is sharply resolved by imposing the relevant jump conditions using an approach that is analogous to imposing the corresponding pressure jump condition in the incompressible Navier-Stokes equations. We show that the jump conditions for the distribution functions are the same in both single-relaxation-time and multi-relaxation-time LBM formulations. Tangential forces are treated using the immersed boundary-lattice Boltzmann method (IB-LBM). The performance of the II-LBM method is compared to both the direct forcing IB-LBM for rigid-body fluid-structure interaction, and the classical IB-LBM for elastic interfaces. Higher order accuracy is observed with the II-LBM as compared to the IB-LBM for selected benchmark problems. Because the jump conditions of the distribution function also satisfy the continuity of the velocity field across the interface, the error in the velocity field is much smaller for the II-LBM than the IB-LBM. The II-LBM is also demonstrated to provide superior volume conservation when simulating flexible boundaries. △ Less

Submitted 26 March, 2020; originally announced March 2020.

Journal ref: Journal of Computational Physics, Volume 428, 1 March 2021, 109807

Hierarchical Low-rank Structure of Parameterized Distributions

Authors: Jun Qin, Lexing Ying

Abstract: This note shows that the matrix forms of several one-parameter distribution families satisfy a hierarchical low-rank structure. Such families of distributions include binomial, Poisson, and $χ^2$ distributions. The proof is based on a uniform relative bound of a related divergence function. Numerical results are provided to confirm the theoretical findings. This note shows that the matrix forms of several one-parameter distribution families satisfy a hierarchical low-rank structure. Such families of distributions include binomial, Poisson, and $χ^2$ distributions. The proof is based on a uniform relative bound of a related divergence function. Numerical results are provided to confirm the theoretical findings. △ Less

Submitted 29 November, 2019; originally announced November 2019.

MSC Class: 41A35; 62E17; 62H10; 94A15

Stochastic Iterative Hard Thresholding for Low-Tucker-Rank Tensor Recovery

Authors: Rachel Grotheer, Shuang Li, Anna Ma, Deanna Needell, Jing Qin

Abstract: Low-rank tensor recovery problems have been widely studied in many applications of signal processing and machine learning. Tucker decomposition is known as one of the most popular decompositions in the tensor framework. In recent years, researchers have developed many state-of-the-art algorithms to address the problem of low-Tucker-rank tensor recovery. Motivated by the favorable properties of the… ▽ More Low-rank tensor recovery problems have been widely studied in many applications of signal processing and machine learning. Tucker decomposition is known as one of the most popular decompositions in the tensor framework. In recent years, researchers have developed many state-of-the-art algorithms to address the problem of low-Tucker-rank tensor recovery. Motivated by the favorable properties of the stochastic algorithms, such as stochastic gradient descent and stochastic iterative hard thresholding, we aim to extend the well-known stochastic iterative hard thresholding algorithm to the tensor framework in order to address the problem of recovering a low-Tucker-rank tensor from its linear measurements. We have also developed linear convergence analysis for the proposed method and conducted a series of experiments with both synthetic and real data to illustrate the performance of the proposed method. △ Less

Submitted 16 July, 2020; v1 submitted 22 September, 2019; originally announced September 2019.

Iterative Hard Thresholding for Low CP-rank Tensor Models

Authors: Rachel Grotheer, Shuang Li, Anna Ma, Deanna Needell, Jing Qin

Abstract: Recovery of low-rank matrices from a small number of linear measurements is now well-known to be possible under various model assumptions on the measurements. Such results demonstrate robustness and are backed with provable theoretical guarantees. However, extensions to tensor recovery have only recently began to be studied and developed, despite an abundance of practical tensor applications. Rece… ▽ More Recovery of low-rank matrices from a small number of linear measurements is now well-known to be possible under various model assumptions on the measurements. Such results demonstrate robustness and are backed with provable theoretical guarantees. However, extensions to tensor recovery have only recently began to be studied and developed, despite an abundance of practical tensor applications. Recently, a tensor variant of the Iterative Hard Thresholding method was proposed and theoretical results were obtained that guarantee exact recovery of tensors with low Tucker rank. In this paper, we utilize the same tensor version of the Restricted Isometry Property (RIP) to extend these results for tensors with low CANDECOMP/PARAFAC (CP) rank. In doing so, we leverage recent results on efficient approximations of CP decompositions that remove the need for challenging assumptions in prior works. We complement our theoretical findings with empirical results that showcase the potential of the approach. △ Less

Submitted 22 August, 2019; originally announced August 2019.

Full-semiparametric-likelihood-based inference for non-ignorable missing data

Authors: Yukun Liu, Pengfei Li, Jing Qin

Abstract: During the past few decades, missing-data problems have been studied extensively, with a focus on the ignorable missing case, where the missing probability depends only on observable quantities. By contrast, research into non-ignorable missing data problems is quite limited. The main difficulty in solving such problems is that the missing probability and the regression likelihood function are tang… ▽ More During the past few decades, missing-data problems have been studied extensively, with a focus on the ignorable missing case, where the missing probability depends only on observable quantities. By contrast, research into non-ignorable missing data problems is quite limited. The main difficulty in solving such problems is that the missing probability and the regression likelihood function are tangled together in the likelihood presentation, and the model parameters may not be identifiable even under strong parametric model assumptions. In this paper we discuss a semiparametric model for non-ignorable missing data and propose a maximum full semiparametric likelihood estimation method, which is an efficient combination of the parametric conditional likelihood and the marginal nonparametric biased sampling likelihood. The extra marginal likelihood contribution can not only produce efficiency gain but also identify the underlying model parameters without additional assumptions. We further show that the proposed estimators for the underlying parameters and the response mean are semiparametrically efficient. Extensive simulations and a real data analysis demonstrate the advantage of the proposed method over competing methods. △ Less

Submitted 3 August, 2019; originally announced August 2019.

Comments: 45 pages

Test for homogeneity with unordered paired observations

Authors: Jiahua Chen, Pengfei Li, Jing Qin, Tao Yu

Abstract: In some applications, an experimental unit is composed of two distinct but related subunits. The response from such a unit is $(X_{1}, X_{2})$ but we observe only $Y_1 = \min\{X_{1},X_{2}\}$ and $Y_2 = \max\{X_{1},X_{2}\}$, i.e., the subunit identities are not observed. We call $(Y_1, Y_2)$ unordered paired observations. Based on unordered paired observations $\{(Y_{1i}, Y_{2i})\}_{i=1}^n$, we are… ▽ More In some applications, an experimental unit is composed of two distinct but related subunits. The response from such a unit is $(X_{1}, X_{2})$ but we observe only $Y_1 = \min\{X_{1},X_{2}\}$ and $Y_2 = \max\{X_{1},X_{2}\}$, i.e., the subunit identities are not observed. We call $(Y_1, Y_2)$ unordered paired observations. Based on unordered paired observations $\{(Y_{1i}, Y_{2i})\}_{i=1}^n$, we are interested in whether the marginal distributions for $X_1$ and $X_2$ are identical. Testing methods are available in the literature under the assumptions that $Var(X_1) = Var(X_2)$ and $Cov(X_1, X_2) = 0$. However, by extensive simulation studies, we observe that when one or both assumptions are violated, these methods have inflated type I errors or much lower powers. In this paper, we study the likelihood ratio test statistics for various scenarios and explore their limiting distributions without these restrictive assumptions. Furthermore, we develop Bartlett correction formulae for these statistics to enhance their precision when the sample size is not large. Simulation studies and real-data examples are used to illustrate the efficacy of the proposed methods. △ Less

Submitted 3 May, 2019; originally announced May 2019.

Comments: 30 pages, 1 figure

Closed-Loop Hierarchical Operation for Optimal Unit Commitment and Dispatch in Microgrids: A Hybrid System Approach

Authors: Fangyuan Li, Jiahu Qin, Yu Kang

Abstract: To contribute to a smart grid, the unit commitment and dispatch (UCD) is a very important problem to be revisited in the future microgrid environment. In this paper, the UCD problem is studied by taking account of both traditional thermal units and distributed generation units (DGs) while considering demand response (DR) and carbon emissions trading (CET). Then, the UCD problem is reformulated as… ▽ More To contribute to a smart grid, the unit commitment and dispatch (UCD) is a very important problem to be revisited in the future microgrid environment. In this paper, the UCD problem is studied by taking account of both traditional thermal units and distributed generation units (DGs) while considering demand response (DR) and carbon emissions trading (CET). Then, the UCD problem is reformulated as the optimal switching problem of a hybrid system. The reformulation allows convenient handling of time-dependent start-up cost for dynamic programming (DP). Afterwards, a closed-loop hierarchical operation (CLHO) algorithm is developed which is capable of producing the optimal schedule in the closed-loop behavior. Theoretical analysis is also presented to show the equivalence of the reformulation and the effectiveness of the CLHO algorithm. Finally, the closed-loop behavior and the effectiveness of the algorithm are further verified by simulation studies. It is also shown that low emission quotas and high emission trading price help reduce the total carbon emissions. △ Less

Submitted 24 December, 2018; originally announced December 2018.

Distributed Least Squares Solver for Network Linear Equations

Authors: Tao Yang, Jemin George, Jiahu Qin, Xinlei Yi, Junfeng Wu

Abstract: In this paper, we study the problem of finding the least square solutions of over-determined linear algebraic equations over networks in a distributed manner. Each node has access to one of the linear equations and holds a dynamic state. We first propose a distributed least square solver over connected undirected interaction graphs and establish a necessary and sufficient on the step-size under wh… ▽ More In this paper, we study the problem of finding the least square solutions of over-determined linear algebraic equations over networks in a distributed manner. Each node has access to one of the linear equations and holds a dynamic state. We first propose a distributed least square solver over connected undirected interaction graphs and establish a necessary and sufficient on the step-size under which the algorithm exponentially converges to the least square solution. Next, we develop a distributed least square solver over strongly connected directed graphs and show that the proposed algorithm exponentially converges to the least square solution provided the step-size is sufficiently small. Moreover, we develop a finite-time least square solver by equipping the proposed algorithms with a finite-time decentralized computation mechanism. The theoretical findings are validated and illustrated by numerical simulation examples. △ Less

Submitted 9 September, 2019; v1 submitted 29 September, 2018; originally announced October 2018.

Compressed Anomaly Detection with Multiple Mixed Observations

Authors: Natalie Durgin, Rachel Grotheer, Chenxi Huang, Shuang Li, Anna Ma, Deanna Needell, Jing Qin

Abstract: We consider a collection of independent random variables that are identically distributed, except for a small subset which follows a different, anomalous distribution. We study the problem of detecting which random variables in the collection are governed by the anomalous distribution. Recent work proposes to solve this problem by conducting hypothesis tests based on mixed observations (e.g. linea… ▽ More We consider a collection of independent random variables that are identically distributed, except for a small subset which follows a different, anomalous distribution. We study the problem of detecting which random variables in the collection are governed by the anomalous distribution. Recent work proposes to solve this problem by conducting hypothesis tests based on mixed observations (e.g. linear combinations) of the random variables. Recognizing the connection between taking mixed observations and compressed sensing, we view the problem as recovering the "support" (index set) of the anomalous random variables from multiple measurement vectors (MMVs). Many algorithms have been developed for recovering jointly sparse signals and their support from MMVs. We establish the theoretical and empirical effectiveness of these algorithms at detecting anomalies. We also extend the LASSO algorithm to an MMV version for our purpose. Further, we perform experiments on synthetic data, consisting of samples from the random variables, to explore the trade-off between the number of mixed observations per sample and the number of samples required to detect anomalies. △ Less

Submitted 19 June, 2018; v1 submitted 30 January, 2018; originally announced January 2018.

Comments: 27 pages, 9 figures. Incorporates reviewer feedback, additional experiments, and additional figures

Sparse Randomized Kaczmarz for Support Recovery of Jointly Sparse Corrupted Multiple Measurement Vectors

Authors: Natalie Durgin, Rachel Grotheer, Chenxi Huang, Shuang Li, Anna Ma, Deanna Needell, Jing Qin

Abstract: While single measurement vector (SMV) models have been widely studied in signal processing, there is a surging interest in addressing the multiple measurement vectors (MMV) problem. In the MMV setting, more than one measurement vector is available and the multiple signals to be recovered share some commonalities such as a common support. Applications in which MMV is a naturally occurring phenomeno… ▽ More While single measurement vector (SMV) models have been widely studied in signal processing, there is a surging interest in addressing the multiple measurement vectors (MMV) problem. In the MMV setting, more than one measurement vector is available and the multiple signals to be recovered share some commonalities such as a common support. Applications in which MMV is a naturally occurring phenomenon include online streaming, medical imaging, and video recovery. This work presents a stochastic iterative algorithm for the support recovery of jointly sparse corrupted MMV. We present a variant of the Sparse Randomized Kaczmarz algorithm for corrupted MMV and compare our proposed method with an existing Kaczmarz type algorithm for MMV problems. We also showcase the usefulness of our approach in the online (streaming) setting and provide empirical evidence that suggests the robustness of the proposed method to the distribution of the corruption and the number of corruptions occurring. △ Less

Submitted 14 June, 2018; v1 submitted 7 November, 2017; originally announced November 2017.

Comments: 13 pages, 6 figures

Stochastic Greedy Algorithms For Multiple Measurement Vectors

Authors: Jing Qin, Shuang Li, Deanna Needell, Anna Ma, Rachel Grotheer, Chenxi Huang, Natalie Durgin

Abstract: Sparse representation of a single measurement vector (SMV) has been explored in a variety of compressive sensing applications. Recently, SMV models have been extended to solve multiple measurement vectors (MMV) problems, where the underlying signal is assumed to have joint sparse structures. To circumvent the NP-hardness of the $\ell_0$ minimization problem, many deterministic MMV algorithms solve… ▽ More Sparse representation of a single measurement vector (SMV) has been explored in a variety of compressive sensing applications. Recently, SMV models have been extended to solve multiple measurement vectors (MMV) problems, where the underlying signal is assumed to have joint sparse structures. To circumvent the NP-hardness of the $\ell_0$ minimization problem, many deterministic MMV algorithms solve the convex relaxed models with limited efficiency. In this paper, we develop stochastic greedy algorithms for solving the joint sparse MMV reconstruction problem. In particular, we propose the MMV Stochastic Iterative Hard Thresholding (MStoIHT) and MMV Stochastic Gradient Matching Pursuit (MStoGradMP) algorithms, and we also utilize the mini-batching technique to further improve their performance. Convergence analysis indicates that the proposed algorithms are able to converge faster than their SMV counterparts, i.e., concatenated StoIHT and StoGradMP, under certain conditions. Numerical experiments have illustrated the superior effectiveness of the proposed algorithms over their SMV counterparts. △ Less

Submitted 22 August, 2020; v1 submitted 4 November, 2017; originally announced November 2017.

MSC Class: 68W20; 94A12; 47N10

Flexible Market for Smart Grid: Coordinated Trading of Contingent Contracts

Authors: Junjie Qin, Ram Rajagopal, Pravin Varaiya

Abstract: A coordinated trading process is proposed as a design for an electricity market with significant uncertainty, perhaps from renewables. In this process, groups of agents propose to the system operator (SO) a contingent buy and sell trade that is balanced, i.e. the sum of demand bids and the sum of supply bids are equal. The SO accepts the proposed trade if no network constraint is violated or curta… ▽ More A coordinated trading process is proposed as a design for an electricity market with significant uncertainty, perhaps from renewables. In this process, groups of agents propose to the system operator (SO) a contingent buy and sell trade that is balanced, i.e. the sum of demand bids and the sum of supply bids are equal. The SO accepts the proposed trade if no network constraint is violated or curtails it until no violation occurs. Each proposed trade is accepted or curtailed as it is presented. The SO also provides guidance to help future proposed trades meet network constraints. The SO does not set prices, and there is no requirement that different trades occur simultaneously or clear at uniform prices. Indeed, there is no price-setting mechanism. However, if participants exploit opportunities for gain, the trading process will lead to an efficient allocation of energy and to the discovery of locational marginal prices (LMPs). The great flexibility in the proposed trading process and the low communication and control burden on the SO may make the process suitable for coordinating producers and consumers in the distribution system. △ Less

Submitted 24 August, 2017; v1 submitted 22 January, 2017; originally announced January 2017.

Comments: Longer version of a paper submitted to IEEE Transactions on Control of Network Systems

A Nonparametric Likelihood Approach for Inference in Instrumental Variable Models

Authors: Kwonsang Lee, Bhaswar B. Bhattacharya, Jing Qin, Dylan S. Small

Abstract: Instrumental variable methods allow for inference about the treatment effect by controlling for unmeasured confounding in randomized experiments with noncompliance. However, many studies do not consider the observed compliance behavior in the testing procedure, which can lead to a loss of power. In this paper, we propose a novel nonparametric likelihood approach, referred to as the binomial likeli… ▽ More Instrumental variable methods allow for inference about the treatment effect by controlling for unmeasured confounding in randomized experiments with noncompliance. However, many studies do not consider the observed compliance behavior in the testing procedure, which can lead to a loss of power. In this paper, we propose a novel nonparametric likelihood approach, referred to as the binomial likelihood (BL) method, that incorporates information on compliance behavior while overcoming several limitations of previous techniques and utilizing the advantages of likelihood methods. Our proposed method produces proper estimates of the counterfactual distribution functions by maximizing the binomial likelihood over the space of distribution functions. Using this we propose two versions of a binomial likelihood ratio test for the null hypothesis of no treatment effect. We show that both versions are more powerful to detect any distributional change than existing methods in finite sample cases, and are asymptotically equivalent to the two-sample Anderson-Darling test. We also develop an efficient algorithm for computing our estimates, and apply the binomial likelihood method to a study of the effect of Medicaid coverage on mental health using the Oregon Health Insurance Experiment. △ Less

Submitted 10 June, 2020; v1 submitted 12 May, 2016; originally announced May 2016.

Comments: Major changes. Updated BL method. New theorems and data analysis added

Group sparse optimization via $\ell_{p,q}$ regularization

Authors: Yaohua Hu, Chong Li, Kaiwen Meng, Jing Qin, Xiaoqi Yang

Abstract: In this paper, we investigate a group sparse optimization problem via $\ell_{p,q}$ regularization in three aspects: theory, algorithm and application. In the theoretical aspect, by introducing a notion of group restricted eigenvalue condition, we establish some oracle property and a global recovery bound of order $O(λ^\frac{2}{2-q})$ for any point in a level set of the $\ell_{p,q}$ regularization… ▽ More In this paper, we investigate a group sparse optimization problem via $\ell_{p,q}$ regularization in three aspects: theory, algorithm and application. In the theoretical aspect, by introducing a notion of group restricted eigenvalue condition, we establish some oracle property and a global recovery bound of order $O(λ^\frac{2}{2-q})$ for any point in a level set of the $\ell_{p,q}$ regularization problem, and by virtue of modern variational analysis techniques, we also provide a local analysis of recovery bound of order $O(λ^2)$ for a path of local minima. In the algorithmic aspect, we apply the well-known proximal gradient method to solve the $\ell_{p,q}$ regularization problems, either by analytically solving some specific $\ell_{p,q}$ regularization subproblems, or by using the Newton method to solve general $\ell_{p,q}$ regularization subproblems. In particular, we establish the linear convergence rate of the proximal gradient method for solving the $\ell_{1,q}$ regularization problem under some mild conditions. As a consequence, the linear convergence rate of proximal gradient method for solving the usual $\ell_{q}$ regularization problem ($0<q<1$) is obtained. Finally in the aspect of application, we present some numerical results on both the simulated data and the real data in gene transcriptional regulation. △ Less

Submitted 28 January, 2016; originally announced January 2016.

Comments: 48 pages, 7 figures

MSC Class: 65K05; 49M37; 90C26 ACM Class: B.1.5

Control of Generalized Energy Storage Networks

Authors: Junjie Qin, Yinlam Chow, Jiyan Yang, Ram Rajagopal

Abstract: The integration of intermittent and volatile renewable energy resources requires increased flexibility in the operation of the electric grid. Storage, broadly speaking, provides the flexibility of shifting energy over time; network, on the other hand, provides the flexibility of shifting energy over geographical locations. The optimal control of general storage networks in uncertain environments i… ▽ More The integration of intermittent and volatile renewable energy resources requires increased flexibility in the operation of the electric grid. Storage, broadly speaking, provides the flexibility of shifting energy over time; network, on the other hand, provides the flexibility of shifting energy over geographical locations. The optimal control of general storage networks in uncertain environments is an important open problem. The key challenge is that, even in small networks, the corresponding constrained stochastic control problems with continuous spaces suffer from curses of dimensionality, and are intractable in general settings. For large networks, no efficient algorithm is known to give optimal or near-optimal performance. This paper provides an efficient and provably near-optimal algorithm to solve this problem in a very general setting. We study the optimal control of generalized storage networks, i.e., electric networks connected to distributed generalized storages. Here generalized storage is a unifying dynamic model for many components of the grid that provide the functionality of shifting energy over time, ranging from standard energy storage devices to deferrable or thermostatically controlled loads. An online algorithm is devised for the corresponding constrained stochastic control problem based on the theory of Lyapunov optimization. We prove that the algorithm is near-optimal, and construct a semidefinite program to min- imize the sub-optimality bound. The resulting bound is a constant that depends only on the parameters of the storage network and cost functions, and is independent of uncertainty realizations. Numerical examples are given to demonstrate the effectiveness of the algorithm. △ Less

Submitted 22 April, 2015; originally announced April 2015.

Comments: This report, written in January 2014, is a longer version of the conference paper [1] (See references in the report). This version contains a somewhat more general treatment for the cases with sub-differentiable objective functions and Markov disturbance. arXiv admin note: substantial text overlap with arXiv:1405.7789

A Multiphase Image Segmentation Based on Fuzzy Membership Functions and L1-norm Fidelity

Authors: Fang Li, Stanley Osher, Jing Qin, Ming Yan

Abstract: In this paper, we propose a variational multiphase image segmentation model based on fuzzy membership functions and L1-norm fidelity. Then we apply the alternating direction method of multipliers to solve an equivalent problem. All the subproblems can be solved efficiently. Specifically, we propose a fast method to calculate the fuzzy median. Experimental results and comparisons show that the L1-n… ▽ More In this paper, we propose a variational multiphase image segmentation model based on fuzzy membership functions and L1-norm fidelity. Then we apply the alternating direction method of multipliers to solve an equivalent problem. All the subproblems can be solved efficiently. Specifically, we propose a fast method to calculate the fuzzy median. Experimental results and comparisons show that the L1-norm based method is more robust to outliers such as impulse noise and keeps better contrast than its L2-norm counterpart. Theoretically, we prove the existence of the minimizer and analyze the convergence of the algorithm. △ Less

Submitted 12 February, 2016; v1 submitted 9 April, 2015; originally announced April 2015.

Comments: 28 pages, 8 figures, 3 tables

Journal ref: Journal of Scientific Computing, 69 (2016), 82-106

Weighted Difference Approximation of Value Functions for Slow-Discounting Markov Decision Processes

Authors: Yin-Lam Chow, Junjie Qin

Abstract: Processes (MDPs) often require frequent decision making, that is, taking an action every microsecond, second, or minute. Infinite horizon discount reward formulation is still relevant for a large portion of these applications, because actual time span of these problems can be months or years, during which discounting factors due to e.g. interest rates are of practical concern. In this paper, we sh… ▽ More Processes (MDPs) often require frequent decision making, that is, taking an action every microsecond, second, or minute. Infinite horizon discount reward formulation is still relevant for a large portion of these applications, because actual time span of these problems can be months or years, during which discounting factors due to e.g. interest rates are of practical concern. In this paper, we show that, for such MDPs with discount rate $α$ close to $1$, under a common ergodicity assumption, a weighted difference between two successive value function estimates obtained from the classical value iteration (VI) is a better approximation than the value function obtained directly from VI. Rigorous error bounds are established which in turn show that the approximation converges to the actual value function in a rate $(αβ)^k$ with $β<1$. This indicates a geometric convergence even if discount factor $α\to 1$. Furthermore, we explicitly link the convergence speed to the system behaviors of the MDP using the notion of $ε-$mixing time and extend our result to Q-functions. Numerical experiments are conducted to demonstrate the convergence properties of the proposed approximation scheme. △ Less

Submitted 16 December, 2014; originally announced December 2014.

Two-stage Geometric Information Guided Image Reconstruction

Authors: Jing Qin, Weihong Guo

Abstract: In compressive sensing, it is challenging to reconstruct image of high quality from very few noisy linear projections. Existing methods mostly work well on piecewise constant images but not so well on piecewise smooth images such as natural images, medical images that contain a lot of details. We propose a two-stage method called GeoCS to recover images with rich geometric information from very li… ▽ More In compressive sensing, it is challenging to reconstruct image of high quality from very few noisy linear projections. Existing methods mostly work well on piecewise constant images but not so well on piecewise smooth images such as natural images, medical images that contain a lot of details. We propose a two-stage method called GeoCS to recover images with rich geometric information from very limited amount of noisy measurements. The method adopts the shearlet transform that is mathematically proven to be optimal in sparsely representing images containing anisotropic features such as edges, corners, spikes etc. It also uses the weighted total variation (TV) sparsity with spatially variant weights to preserve sharp edges but to reduce the staircase effects of TV. Geometric information extracted from the results of stage I serves as an initial prior for stage II which alternates image reconstruction and geometric information update in a mutually beneficial way. GeoCS has been tested on incomplete spectral Fourier samples. It is applicable to other types of measurements as well. Experimental results on various complicated images show that GeoCS is efficient and generates high-quality images. △ Less

Submitted 16 February, 2021; v1 submitted 25 September, 2014; originally announced September 2014.

Maximum Smoothed Likelihood Component Density Estimation in Mixture Models with Known Mixing Proportions

Authors: Tao Yu, Pengfei Li, Jing Qin

Abstract: In this paper, we propose a maximum smoothed likelihood method to estimate the component density functions of mixture models, in which the mixing proportions are known and may differ among observations. The proposed estimates maximize a smoothed log likelihood function and inherit all the important properties of probability density functions. A majorization-minimization algorithm is suggested to c… ▽ More In this paper, we propose a maximum smoothed likelihood method to estimate the component density functions of mixture models, in which the mixing proportions are known and may differ among observations. The proposed estimates maximize a smoothed log likelihood function and inherit all the important properties of probability density functions. A majorization-minimization algorithm is suggested to compute the proposed estimates numerically. In theory, we show that starting from any initial value, this algorithm increases the smoothed likelihood function and further leads to estimates that maximize the smoothed likelihood function. This indicates the convergence of the algorithm. Furthermore, we theoretically establish the asymptotic convergence rate of our proposed estimators. An adaptive procedure is suggested to choose the bandwidths in our estimation procedure. Simulation studies show that the proposed method is more efficient than the existing method in terms of integrated squared errors. A real data example is further analyzed. △ Less

Submitted 11 July, 2014; originally announced July 2014.

Distributed Online Modified Greedy Algorithm for Networked Storage Operation under Uncertainty

Authors: Junjie Qin, Yinlam Chow, Jiyan Yang, Ram Rajagopal

Abstract: The integration of intermittent and stochastic renewable energy resources requires increased flexibility in the operation of the electric grid. Storage, broadly speaking, provides the flexibility of shifting energy over time; network, on the other hand, provides the flexibility of shifting energy over geographical locations. The optimal control of storage networks in stochastic environments is an… ▽ More The integration of intermittent and stochastic renewable energy resources requires increased flexibility in the operation of the electric grid. Storage, broadly speaking, provides the flexibility of shifting energy over time; network, on the other hand, provides the flexibility of shifting energy over geographical locations. The optimal control of storage networks in stochastic environments is an important open problem. The key challenge is that, even in small networks, the corresponding constrained stochastic control problems on continuous spaces suffer from curses of dimensionality, and are intractable in general settings. For large networks, no efficient algorithm is known to give optimal or provably near-optimal performance for this problem. This paper provides an efficient algorithm to solve this problem with performance guarantees. We study the operation of storage networks, i.e., a storage system interconnected via a power network. An online algorithm, termed Online Modified Greedy algorithm, is developed for the corresponding constrained stochastic control problem. A sub-optimality bound for the algorithm is derived, and a semidefinite program is constructed to minimize the bound. In many cases, the bound approaches zero so that the algorithm is near-optimal. A task-based distributed implementation of the online algorithm relying only on local information and neighbor communication is then developed based on the alternating direction method of multipliers. Numerical examples verify the established theoretical performance bounds, and demonstrate the scalability of the algorithm. △ Less

Submitted 3 November, 2014; v1 submitted 18 June, 2014; originally announced June 2014.

Comments: arXiv admin note: text overlap with arXiv:1405.7789

Online Modified Greedy Algorithm for Storage Control under Uncertainty

Authors: Junjie Qin, Yinlam Chow, Jiyan Yang, Ram Rajagopal

Abstract: This paper studies the general problem of operating energy storage under uncertainty. Two fundamental sources of uncertainty are considered, namely the uncertainty in the unexpected fluctuation of the net demand process and the uncertainty in the locational marginal prices. We propose a very simple algorithm termed Online Modified Greedy (OMG) algorithm for this problem. A stylized analysis for th… ▽ More This paper studies the general problem of operating energy storage under uncertainty. Two fundamental sources of uncertainty are considered, namely the uncertainty in the unexpected fluctuation of the net demand process and the uncertainty in the locational marginal prices. We propose a very simple algorithm termed Online Modified Greedy (OMG) algorithm for this problem. A stylized analysis for the algorithm is performed, which shows that comparing to the optimal cost of the corresponding stochastic control problem, the sub-optimality of OMG is bounded and approaches zero in various scenarios. This suggests that, albeit simple, OMG is guaranteed to have good performance in some cases; and in other cases, OMG together with the sub-optimality bound can be used to provide a lower bound for the optimal cost. Such a lower bound can be valuable in evaluating other heuristic algorithms. For the latter cases, a semidefinite program is derived to minimize the sub-optimality bound of OMG. Numerical experiments are conducted to verify our theoretical analysis and to demonstrate the use of the algorithm. △ Less

Submitted 13 June, 2015; v1 submitted 30 May, 2014; originally announced May 2014.

Comments: 14 page version of a paper submitted to IEEE trans on Power Systems

Risk Limiting Dispatch with Ramping Constraints

Authors: Junjie Qin, Baosen Zhang, Ram Rajagopal

Abstract: Reliable operation in power systems is becoming more difficult as the penetration of random renewable resources increases. In particular, operators face the risk of not scheduling enough traditional generators in the times when renewable energies becomes lower than expected. In this paper we study the optimal trade-off between system and risk, and the cost of scheduling reserve generators. We expl… ▽ More Reliable operation in power systems is becoming more difficult as the penetration of random renewable resources increases. In particular, operators face the risk of not scheduling enough traditional generators in the times when renewable energies becomes lower than expected. In this paper we study the optimal trade-off between system and risk, and the cost of scheduling reserve generators. We explicitly model the ramping constraints on the generators. We model the problem as a multi-period stochastic control problem, and we show the structure of the optimal dispatch. We then show how to efficiently compute the dispatch using two methods: i) solving a surrogate chance constrained program, ii) a MPC-type look ahead controller. Using real world data, we show the chance constrained dispatch outperforms the MPC controller and is also robust to changes in the probability distribution of the renewables. △ Less

Submitted 26 May, 2013; v1 submitted 19 May, 2013; originally announced May 2013.

Comments: Shorter version submitted to smartgrid comm 2013