Showing 1–32 of 32 results for author: Khoo, Y

Solving cluster moment relaxation with hierarchical matrix

Authors: Yi Wang, Yuehaw Khoo

Abstract: Convex relaxation methods are powerful tools for studying the lowest energy of many-body problems. By relaxing the representability conditions for marginals to a set of local constraints, along with a global semidefinite constraint, a polynomial-time solvable semidefinite program (SDP) that provides a lower bound for the energy can be derived. In this paper, we propose accelerating the solution of… ▽ More Convex relaxation methods are powerful tools for studying the lowest energy of many-body problems. By relaxing the representability conditions for marginals to a set of local constraints, along with a global semidefinite constraint, a polynomial-time solvable semidefinite program (SDP) that provides a lower bound for the energy can be derived. In this paper, we propose accelerating the solution of such an SDP relaxation by imposing a hierarchical structure on the positive semidefinite (PSD) primal and dual variables. Furthermore, these matrices can be updated efficiently using the algebra of the compressed representations within an augmented Lagrangian method. We achieve quadratic and even near-linear time per-iteration complexity. Through experimentation on the quantum transverse field Ising model, we showcase the capability of our approach to provide a sufficiently accurate lower bound for the exact ground-state energy. △ Less

Submitted 31 July, 2024; originally announced August 2024.

Temporal label recovery from noisy dynamical data

Authors: Yuehaw Khoo, Xin T. Tong, Wanjie Wang, Yuguan Wang

Abstract: Analyzing dynamical data often requires information of the temporal labels, but such information is unavailable in many applications. Recovery of these temporal labels, closely related to the seriation or sequencing problem, becomes crucial in the study. However, challenges arise due to the nonlinear nature of the data and the complexity of the underlying dynamical system, which may be periodic or… ▽ More Analyzing dynamical data often requires information of the temporal labels, but such information is unavailable in many applications. Recovery of these temporal labels, closely related to the seriation or sequencing problem, becomes crucial in the study. However, challenges arise due to the nonlinear nature of the data and the complexity of the underlying dynamical system, which may be periodic or non-periodic. Additionally, noise within the feature space complicates the theoretical analysis. Our work develops spectral algorithms that leverage manifold learning concepts to recover temporal labels from noisy data. We first construct the graph Laplacian of the data, and then employ the second (and the third) Fiedler vectors to recover temporal labels. This method can be applied to both periodic and aperiodic cases. It also does not require monotone properties on the similarity matrix, which are commonly assumed in existing spectral seriation algorithms. We develop the $\ell_{\infty}$ error of our estimators for the temporal labels and ranking, without assumptions on the eigen-gap. In numerical analysis, our method outperforms spectral seriation algorithms based on a similarity matrix. The performance of our algorithms is further demonstrated on a synthetic biomolecule data example. △ Less

Submitted 19 June, 2024; originally announced June 2024.

Comments: 20 pages, 4 figures

S-SOS: Stochastic Sum-Of-Squares for Parametric Polynomial Optimization

Authors: Richard L. Zhu, Mathias Oster, Yuehaw Khoo

Abstract: Global polynomial optimization is an important tool across applied mathematics, with many applications in operations research, engineering, and physical sciences. In various settings, the polynomials depend on external parameters that may be random. We discuss a stochastic sum-of-squares (S-SOS) algorithm based on the sum-of squares hierarchy that constructs a series of semidefinite programs to jo… ▽ More Global polynomial optimization is an important tool across applied mathematics, with many applications in operations research, engineering, and physical sciences. In various settings, the polynomials depend on external parameters that may be random. We discuss a stochastic sum-of-squares (S-SOS) algorithm based on the sum-of squares hierarchy that constructs a series of semidefinite programs to jointly find strict lower bounds on the global minimum and extract candidates for parameterized global minimizers. We prove quantitative convergence of the hierarchy as the degree increases and use it to solve unconstrained and constrained polynomial optimization problems parameterized by random variables. By employing $n$-body priors from condensed matter physics to induce sparsity, we can use S-SOS to produce solutions and uncertainty intervals for sensor network localization problems containing up to 40 variables and semidefinite matrix sizes surpassing $800 \times 800$. △ Less

Submitted 13 June, 2024; originally announced June 2024.

Augmented Lagrangian method for coupled-cluster

Authors: Fabian M. Faulstich, Yuehaw Khoo, Kangbo Li

Abstract: We propose to improve the convergence properties of the single-reference coupled cluster (CC) method through an augmented Lagrangian formalism. The conventional CC method changes a linear high-dimensional eigenvalue problem with exponential size into a problem of determining the roots of a nonlinear system of equations that has a manageable size. However, current numerical procedures for solving t… ▽ More We propose to improve the convergence properties of the single-reference coupled cluster (CC) method through an augmented Lagrangian formalism. The conventional CC method changes a linear high-dimensional eigenvalue problem with exponential size into a problem of determining the roots of a nonlinear system of equations that has a manageable size. However, current numerical procedures for solving this system of equations to get the lowest eigenvalue suffer from two practical issues: First, solving the CC equations may not converge, and second, when converging, they may converge to other -- potentially unphysical -- states, which are stationary points of the CC energy expression. We show that both issues can be dealt with when a suitably defined energy is minimized in addition to solving the original CC equations. We further propose an augmented Lagrangian method for coupled cluster (alm-CC) to solve the resulting constrained optimization problem. We numerically investigate the proposed augmented Lagrangian formulation showing that the convergence towards the ground state is significantly more stable and that the optimization procedure is less susceptible to local minima. Furthermore, the computational cost of alm-CC is comparable to the conventional CC method. △ Less

Submitted 24 March, 2024; originally announced March 2024.

Comments: 17 pages, 8 figures

Nonparametric Density Estimation via Variance-Reduced Sketching

Authors: Yifan Peng, Yuehaw Khoo, Daren Wang

Abstract: Nonparametric density models are of great interest in various scientific and engineering disciplines. Classical density kernel methods, while numerically robust and statistically sound in low-dimensional settings, become inadequate even in moderate higher-dimensional settings due to the curse of dimensionality. In this paper, we introduce a new framework called Variance-Reduced Sketching (VRS), sp… ▽ More Nonparametric density models are of great interest in various scientific and engineering disciplines. Classical density kernel methods, while numerically robust and statistically sound in low-dimensional settings, become inadequate even in moderate higher-dimensional settings due to the curse of dimensionality. In this paper, we introduce a new framework called Variance-Reduced Sketching (VRS), specifically designed to estimate multivariable density functions with a reduced curse of dimensionality. Our framework conceptualizes multivariable functions as infinite-size matrices, and facilitates a new sketching technique motivated by numerical linear algebra literature to reduce the variance in density estimation problems. We demonstrate the robust numerical performance of VRS through a series of simulated experiments and real-world data applications. Notably, VRS shows remarkable improvement over existing neural network estimators and classical kernel methods in numerous density models. Additionally, we offer theoretical justifications for VRS to support its ability to deliver nonparametric density estimation with a reduced curse of dimensionality. △ Less

Submitted 7 July, 2024; v1 submitted 21 January, 2024; originally announced January 2024.

Comments: 62 pages, 12 figures

Robust Point Matching with Distance Profiles

Authors: YoonHaeng Hur, Yuehaw Khoo

Abstract: While matching procedures based on pairwise distances are conceptually appealing and thus favored in practice, theoretical guarantees for such procedures are rarely found in the literature. We propose and analyze matching procedures based on distance profiles that are easily implementable in practice, showing these procedures are robust to outliers and noise. We demonstrate the performance of the… ▽ More While matching procedures based on pairwise distances are conceptually appealing and thus favored in practice, theoretical guarantees for such procedures are rarely found in the literature. We propose and analyze matching procedures based on distance profiles that are easily implementable in practice, showing these procedures are robust to outliers and noise. We demonstrate the performance of the proposed method using a real data example and provide simulation studies to complement the theoretical findings. △ Less

Submitted 15 May, 2024; v1 submitted 19 December, 2023; originally announced December 2023.

Convex Relaxation for Fokker-Planck

Authors: Yian Chen, Yuehaw Khoo, Lek-Heng Lim

Abstract: We propose an approach to directly estimate the moments or marginals for a high-dimensional equilibrium distribution in statistical mechanics, via solving the high-dimensional Fokker-Planck equation in terms of low-order cluster moments or marginals. With this approach, we bypass the exponential complexity of estimating the full high-dimensional distribution and directly solve the simplified parti… ▽ More We propose an approach to directly estimate the moments or marginals for a high-dimensional equilibrium distribution in statistical mechanics, via solving the high-dimensional Fokker-Planck equation in terms of low-order cluster moments or marginals. With this approach, we bypass the exponential complexity of estimating the full high-dimensional distribution and directly solve the simplified partial differential equations for low-order moments/marginals. Moreover, the proposed moment/marginal relaxation is fully convex and can be solved via off-the-shelf solvers. We further propose a time-dependent version of the convex programs to study non-equilibrium dynamics. We show the proposed method can recover the meanfield approximation of an equilibrium density. Numerical results are provided to demonstrate the performance of the proposed algorithm for high-dimensional systems. △ Less

Submitted 3 December, 2023; v1 submitted 5 June, 2023; originally announced June 2023.

MSC Class: 65M22; 90C25; 49M20; 82C23

Combining Monte Carlo and Tensor-network Methods for Partial Differential Equations via Sketching

Authors: Yian Chen, Yuehaw Khoo

Abstract: In this paper, we propose a general framework for solving high-dimensional partial differential equations with tensor networks. Our approach uses Monte-Carlo simulations to update the solution and re-estimates the new solution from samples as a tensor-network using a recently proposed tensor train sketching technique. We showcase the versatility and flexibility of our approach by applying it to tw… ▽ More In this paper, we propose a general framework for solving high-dimensional partial differential equations with tensor networks. Our approach uses Monte-Carlo simulations to update the solution and re-estimates the new solution from samples as a tensor-network using a recently proposed tensor train sketching technique. We showcase the versatility and flexibility of our approach by applying it to two specific scenarios: simulating the Fokker-Planck equation through Langevin dynamics and quantum imaginary time evolution via auxiliary-field quantum Monte Carlo. We also provide convergence guarantees and numerical experiments to demonstrate the efficacy of the proposed method. △ Less

Submitted 10 October, 2023; v1 submitted 29 May, 2023; originally announced May 2023.

MSC Class: 15A69; 62G07; 65M75; 65M22; 82B05; 82B10

Deep Neural-network Prior for Orbit Recovery from Method of Moments

Authors: Yuehaw Khoo, Sounak Paul, Nir Sharon

Abstract: Orbit recovery problems are a class of problems that often arise in practice and various forms. In these problems, we aim to estimate an unknown function after being distorted by a group action and observed via a known operator. Typically, the observations are contaminated with a non-trivial level of noise. Two particular orbit recovery problems of interest in this paper are multireference alignme… ▽ More Orbit recovery problems are a class of problems that often arise in practice and various forms. In these problems, we aim to estimate an unknown function after being distorted by a group action and observed via a known operator. Typically, the observations are contaminated with a non-trivial level of noise. Two particular orbit recovery problems of interest in this paper are multireference alignment and single-particle cryo-EM modelling. In order to suppress the noise, we suggest using the method of moments approach for both problems while introducing deep neural network priors. In particular, our neural networks should output the signals and the distribution of group elements, with moments being the input. In the multireference alignment case, we demonstrate the advantage of using the NN to accelerate the convergence for the reconstruction of signals from the moments. Finally, we use our method to reconstruct simulated and biological volumes in the cryo-EM setting. △ Less

Submitted 30 January, 2024; v1 submitted 27 April, 2023; originally announced April 2023.

Journal ref: J. Comput. Appl. Math. 115782 (2024)

Generative Modeling via Hierarchical Tensor Sketching

Authors: Yifan Peng, Yian Chen, E. Miles Stoudenmire, Yuehaw Khoo

Abstract: We propose a hierarchical tensor-network approach for approximating high-dimensional probability density via empirical distribution. This leverages randomized singular value decomposition (SVD) techniques and involves solving linear equations for tensor cores in this tensor network. The complexity of the resulting algorithm scales linearly in the dimension of the high-dimensional density. An analy… ▽ More We propose a hierarchical tensor-network approach for approximating high-dimensional probability density via empirical distribution. This leverages randomized singular value decomposition (SVD) techniques and involves solving linear equations for tensor cores in this tensor network. The complexity of the resulting algorithm scales linearly in the dimension of the high-dimensional density. An analysis of estimation error demonstrates the effectiveness of this method through several numerical experiments. △ Less

Submitted 11 April, 2023; originally announced April 2023.

MSC Class: 15A69; 62Gxx

High-dimensional density estimation with tensorizing flow

Authors: Yinuo Ren, Hongli Zhao, Yuehaw Khoo, Lexing Ying

Abstract: We propose the tensorizing flow method for estimating high-dimensional probability density functions from the observed data. The method is based on tensor-train and flow-based generative modeling. Our method first efficiently constructs an approximate density in the tensor-train form via solving the tensor cores from a linear system based on the kernel density estimators of low-dimensional margina… ▽ More We propose the tensorizing flow method for estimating high-dimensional probability density functions from the observed data. The method is based on tensor-train and flow-based generative modeling. Our method first efficiently constructs an approximate density in the tensor-train form via solving the tensor cores from a linear system based on the kernel density estimators of low-dimensional marginals. We then train a continuous-time flow model from this tensor-train density to the observed empirical distribution by performing a maximum likelihood estimation. The proposed method combines the optimization-less feature of the tensor-train with the flexibility of the flow-based generative models. Numerical results are included to demonstrate the performance of the proposed method. △ Less

Submitted 1 December, 2022; originally announced December 2022.

Autocorrelation analysis for cryo-EM with sparsity constraints: Improved sample complexity and projection-based algorithms

Authors: Tamir Bendory, Yuehaw Khoo, Joe Kileel, Oscar Mickelin, Amit Singer

Abstract: The number of noisy images required for molecular reconstruction in single-particle cryo-electron microscopy (cryo-EM) is governed by the autocorrelations of the observed, randomly-oriented, noisy projection images. In this work, we consider the effect of imposing sparsity priors on the molecule. We use techniques from signal processing, optimization, and applied algebraic geometry to obtain new t… ▽ More The number of noisy images required for molecular reconstruction in single-particle cryo-electron microscopy (cryo-EM) is governed by the autocorrelations of the observed, randomly-oriented, noisy projection images. In this work, we consider the effect of imposing sparsity priors on the molecule. We use techniques from signal processing, optimization, and applied algebraic geometry to obtain new theoretical and computational contributions for this challenging non-linear inverse problem with sparsity constraints. We prove that molecular structures modeled as sums of Gaussians are uniquely determined by the second-order autocorrelation of their projection images, implying that the sample complexity is proportional to the square of the variance of the noise. This theory improves upon the non-sparse case, where the third-order autocorrelation is required for uniformly-oriented particle images and the sample complexity scales with the cube of the noise variance. Furthermore, we build a computational framework to reconstruct molecular structures which are sparse in the wavelet basis. This method combines the sparse representation for the molecule with projection-based techniques used for phase retrieval in X-ray crystallography. △ Less

Submitted 1 May, 2023; v1 submitted 21 September, 2022; originally announced September 2022.

Comments: 31 pages, 5 figures, 1 movie

Journal ref: Proceedings of the National Academy of Sciences 120.18 (2023): e2216507120

Generative modeling via tensor train sketching

Authors: YH. Hur, J. G. Hoskins, M. Lindsey, E. M. Stoudenmire, Y. Khoo

Abstract: In this paper, we introduce a sketching algorithm for constructing a tensor train representation of a probability density from its samples. Our method deviates from the standard recursive SVD-based procedure for constructing a tensor train. Instead, we formulate and solve a sequence of small linear systems for the individual tensor train cores. This approach can avoid the curse of dimensionality t… ▽ More In this paper, we introduce a sketching algorithm for constructing a tensor train representation of a probability density from its samples. Our method deviates from the standard recursive SVD-based procedure for constructing a tensor train. Instead, we formulate and solve a sequence of small linear systems for the individual tensor train cores. This approach can avoid the curse of dimensionality that threatens both the algorithmic and sample complexities of the recovery problem. Specifically, for Markov models under natural conditions, we prove that the tensor cores can be recovered with a sample complexity that scales logarithmically in the dimensionality. Finally, we illustrate the performance of the method with several numerical experiments. △ Less

Submitted 23 June, 2023; v1 submitted 23 February, 2022; originally announced February 2022.

MSC Class: 15A69; 62Gxx

Committor functions via tensor networks

Authors: Yian Chen, Jeremy Hoskins, Yuehaw Khoo, Michael Lindsey

Abstract: We propose a novel approach for computing committor functions, which describe transitions of a stochastic process between metastable states. The committor function satisfies a backward Kolmogorov equation, and in typical high-dimensional settings of interest, it is intractable to compute and store the solution with traditional numerical methods. By parametrizing the committor function in a matrix… ▽ More We propose a novel approach for computing committor functions, which describe transitions of a stochastic process between metastable states. The committor function satisfies a backward Kolmogorov equation, and in typical high-dimensional settings of interest, it is intractable to compute and store the solution with traditional numerical methods. By parametrizing the committor function in a matrix product state/tensor train format and using a similar representation for the equilibrium probability density, we solve the variational formulation of the backward Kolmogorov equation with linear time and memory complexity in the number of dimensions. This approach bypasses the need for sampling the equilibrium distribution, which can be difficult when the distribution has multiple modes. Numerical results demonstrate the effectiveness of the proposed method for high-dimensional problems. △ Less

Submitted 2 August, 2021; v1 submitted 23 June, 2021; originally announced June 2021.

MSC Class: 15A69; 82Cxx

Scalable semidefinite programming approach to variational embedding for quantum many-body problems

Authors: Yuehaw Khoo, Michael Lindsey

Abstract: In quantum embedding theories, a quantum many-body system is divided into localized clusters of sites which are treated with an accurate `high-level' theory and glued together self-consistently by a less accurate `low-level' theory at the global scale. The recently introduced variational embedding approach for quantum many-body problems combines the insights of semidefinite relaxation and quantum… ▽ More In quantum embedding theories, a quantum many-body system is divided into localized clusters of sites which are treated with an accurate `high-level' theory and glued together self-consistently by a less accurate `low-level' theory at the global scale. The recently introduced variational embedding approach for quantum many-body problems combines the insights of semidefinite relaxation and quantum embedding theory to provide a lower bound on the ground-state energy that improves as the cluster size is increased. The variational embedding method is formulated as a semidefinite program (SDP), which can suffer from poor computational scaling when treated with black-box solvers. We exploit the interpretation of this SDP as an embedding method to develop an algorithm which alternates parallelizable local updates of the high-level quantities with updates that enforce the low-level global constraints. Moreover, we show how translation invariance in lattice systems can be exploited to reduce the complexity of projecting a key matrix to the positive semidefinite cone. △ Less

Submitted 4 June, 2021; originally announced June 2021.

Multiscale semidefinite programming approach to positioning problems with pairwise structure

Authors: Yian Chen, Yuehaw Khoo, Michael Lindsey

Abstract: We consider the optimization of pairwise objective functions, i.e., objective functions of the form $H(\mathbf{x}) = H(x_1,\ldots,x_N) = \sum_{1\leq i<j \leq N} H_{ij}(x_i,x_j)$ for $x_i$ in some continuous state spaces $\mathcal{X}_i$. Global optimization in this setting is generally confounded by the possible existence of spurious local minima and the impossibility of global search due to the cu… ▽ More We consider the optimization of pairwise objective functions, i.e., objective functions of the form $H(\mathbf{x}) = H(x_1,\ldots,x_N) = \sum_{1\leq i<j \leq N} H_{ij}(x_i,x_j)$ for $x_i$ in some continuous state spaces $\mathcal{X}_i$. Global optimization in this setting is generally confounded by the possible existence of spurious local minima and the impossibility of global search due to the curse of dimensionality. In this paper, we approach such problems via convex relaxation of the marginal polytope considered in graphical modeling, proceeding in a multiscale fashion which exploits the smoothness of the cost function. We show theoretically that, compared with existing methods, such an approach is advantageous even in simple settings for sensor network localization (SNL). We successfully apply our method to SNL problems, particularly difficult instances with high noise. We also validate performance on the optimization of the Lennard-Jones potential, which is plagued by the existence of many near-optimal configurations. We demonstrate that in MMR allows us to effectively explore these configurations. △ Less

Submitted 17 December, 2020; originally announced December 2020.

A semigroup method for high dimensional committor functions based on neural network

Authors: Haoya Li, Yuehaw Khoo, Yinuo Ren, Lexing Ying

Abstract: This paper proposes a new method based on neural networks for computing the high-dimensional committor functions that satisfy Fokker-Planck equations. Instead of working with partial differential equations, the new method works with an integral formulation based on the semigroup of the differential operator. The variational form of the new formulation is then solved by parameterizing the committor… ▽ More This paper proposes a new method based on neural networks for computing the high-dimensional committor functions that satisfy Fokker-Planck equations. Instead of working with partial differential equations, the new method works with an integral formulation based on the semigroup of the differential operator. The variational form of the new formulation is then solved by parameterizing the committor function as a neural network. There are two major benefits of this new approach. First, stochastic gradient descent type algorithms can be applied in the training of the committor function without the need of computing any mixed second-order derivatives. Moreover, unlike the previous methods that enforce the boundary conditions through penalty terms, the new method takes into account the boundary conditions automatically. Numerical results are provided to demonstrate the performance of the proposed method. △ Less

Submitted 5 May, 2021; v1 submitted 12 December, 2020; originally announced December 2020.

Comments: 21 pages, 14 figures; Final version accepted at MSML 2021

NMR Assignment through Linear Programming

Authors: Jose F. S. Bravo-Ferreira, David Cowburn, Yuehaw Khoo, Amit Singer

Abstract: Nuclear Magnetic Resonance (NMR) Spectroscopy is the second most used technique (after X-ray crystallography) for structural determination of proteins. A computational challenge in this technique involves solving a discrete optimization problem that assigns the resonance frequency to each atom in the protein. This paper introduces LIAN (LInear programming Assignment for NMR), a novel linear progra… ▽ More Nuclear Magnetic Resonance (NMR) Spectroscopy is the second most used technique (after X-ray crystallography) for structural determination of proteins. A computational challenge in this technique involves solving a discrete optimization problem that assigns the resonance frequency to each atom in the protein. This paper introduces LIAN (LInear programming Assignment for NMR), a novel linear programming formulation of the problem which yields state-of-the-art results in simulated and experimental datasets. △ Less

Submitted 7 September, 2021; v1 submitted 8 August, 2020; originally announced August 2020.

Comments: 28 pages, 10 figures

Maximizing robustness of point-set registration by leveraging non-convexity

Authors: Cindy Orozco Bohorquez, Yuehaw Khoo, Lexing Ying

Abstract: Point-set registration is a classical image processing problem that looks for the optimal transformation between two sets of points. In this work, we analyze the impact of outliers when finding the optimal rotation between two point clouds. The presence of outliers motivates the use of least unsquared deviation, which is a non-smooth minimization problem over non-convex domain. We compare approach… ▽ More Point-set registration is a classical image processing problem that looks for the optimal transformation between two sets of points. In this work, we analyze the impact of outliers when finding the optimal rotation between two point clouds. The presence of outliers motivates the use of least unsquared deviation, which is a non-smooth minimization problem over non-convex domain. We compare approaches based on non-convex optimization over special orthogonal group and convex relaxations. We show that if the fraction of outliers is larger than a certain threshold, any naive convex relaxation fails to recover the ground truth rotation regardless of the sample size and dimension. In contrast, minimizing the least unsquared deviation directly over the special orthogonal group exactly recovers the ground truth rotation for any level of corruption as long as the sample size is large enough. These theoretical findings are supported by numerical simulations. △ Less

Submitted 7 October, 2020; v1 submitted 19 April, 2020; originally announced April 2020.

Comments: 38 pages, 6 figures; Fix typos

MSC Class: 65D1; 68T45

Method of moments for 3-D single particle ab initio modeling with non-uniform distribution of viewing angles

Authors: Nir Sharon, Joe Kileel, Yuehaw Khoo, Boris Landa, Amit Singer

Abstract: Single-particle reconstruction in cryo-electron microscopy (cryo-EM) is an increasingly popular technique for determining the 3-D structure of a molecule from several noisy 2-D projections images taken at unknown viewing angles. Most reconstruction algorithms require a low-resolution initialization for the 3-D structure, which is the goal of ab initio modeling. Suggested by Zvi Kam in 1980, the me… ▽ More Single-particle reconstruction in cryo-electron microscopy (cryo-EM) is an increasingly popular technique for determining the 3-D structure of a molecule from several noisy 2-D projections images taken at unknown viewing angles. Most reconstruction algorithms require a low-resolution initialization for the 3-D structure, which is the goal of ab initio modeling. Suggested by Zvi Kam in 1980, the method of moments (MoM) offers one approach, wherein low-order statistics of the 2-D images are computed and a 3-D structure is estimated by solving a system of polynomial equations. Unfortunately, Kam's method suffers from restrictive assumptions, most notably that viewing angles should be distributed uniformly. Often unrealistic, uniformity entails the computation of higher-order correlations, as in this case first and second moments fail to determine the 3-D structure. In the present paper, we remove this hypothesis, by permitting an unknown, non-uniform distribution of viewing angles in MoM. Perhaps surprisingly, we show that this case is statistically easier than the uniform case, as now first and second moments generically suffice to determine low-resolution expansions of the molecule. In the idealized setting of a known, non-uniform distribution, we find an efficient provable algorithm inverting first and second moments. For unknown, non-uniform distributions, we use non-convex optimization methods to solve for both the molecule and distribution. △ Less

Submitted 23 November, 2019; v1 submitted 11 July, 2019; originally announced July 2019.

Comments: 41 pages. v2: additional numerical experiments, appendices edited, other updates

Semidefinite relaxation of multi-marginal optimal transport for strictly correlated electrons in second quantization

Authors: Yuehaw Khoo, Lin Lin, Michael Lindsey, Lexing Ying

Abstract: We consider the strictly correlated electron (SCE) limit of the fermionic quantum many-body problem in the second-quantized formalism. This limit gives rise to a multi-marginal optimal transport (MMOT) problem. Here the marginal state space for our MMOT problem is the binary set $\{0,1\}$, and the number of marginals is the number $L$ of sites in the model. The costs of storing and computing the e… ▽ More We consider the strictly correlated electron (SCE) limit of the fermionic quantum many-body problem in the second-quantized formalism. This limit gives rise to a multi-marginal optimal transport (MMOT) problem. Here the marginal state space for our MMOT problem is the binary set $\{0,1\}$, and the number of marginals is the number $L$ of sites in the model. The costs of storing and computing the exact solution of the MMOT problem both scale exponentially with respect to $L$. We propose an efficient convex relaxation which can be solved by semidefinite programming (SDP). In particular, the semidefinite constraint is only of size $2L\times 2L$. Moreover, the SDP-based method yields an approximation of the dual potential needed to the perform self-consistent field iteration in the so-called Kohn-Sham SCE framework, which, once converged, yields a lower bound for the total energy of the system. We demonstrate the effectiveness of our methods on spinless and spinful Hubbard-type models. Numerical results indicate that our relaxation methods yield tight lower bounds for the optimal cost, in the sense that the error due to the semidefinite relaxation is much smaller than the intrinsic modeling error of the Kohn-Sham SCE method. We also describe how our relaxation methods generalize to arbitrary MMOT problems with pairwise cost functions. △ Less

Submitted 15 September, 2020; v1 submitted 20 May, 2019; originally announced May 2019.

Comments: 28 pages, 9 figures

MSC Class: 49M20 ACM Class: G.1.6; G.1.10; J.2

SwitchNet: a neural network model for forward and inverse scattering problems

Authors: Yuehaw Khoo, Lexing Ying

Abstract: We propose a novel neural network architecture, SwitchNet, for solving the wave equation based inverse scattering problems via providing maps between the scatterers and the scattered field (and vice versa). The main difficulty of using a neural network for this problem is that a scatterer has a global impact on the scattered wave field, rendering typical convolutional neural network with local con… ▽ More We propose a novel neural network architecture, SwitchNet, for solving the wave equation based inverse scattering problems via providing maps between the scatterers and the scattered field (and vice versa). The main difficulty of using a neural network for this problem is that a scatterer has a global impact on the scattered wave field, rendering typical convolutional neural network with local connections inapplicable. While it is possible to deal with such a problem using a fully connected network, the number of parameters grows quadratically with the size of the input and output data. By leveraging the inherent low-rank structure of the scattering problems and introducing a novel switching layer with sparse connections, the SwitchNet architecture uses much fewer parameters and facilitates the training process. Numerical experiments show promising accuracy in learning the forward and inverse maps between the scatterers and the scattered wave field. △ Less

Submitted 23 October, 2018; originally announced October 2018.

Comments: 19 pages, 7 figures

Convex relaxation approaches for strictly correlated density functional theory

Authors: Yuehaw Khoo, Lexing Ying

Abstract: In this paper, we introduce methods from convex optimization to solve the multimarginal transport type problems arise in the context of density functional theory. Convex relaxations are used to provide outer approximation to the set of $N$-representable 2-marginals and 3-marginals, which in turn provide lower bounds to the energy. We further propose rounding schemes to obtain upper bound to the en… ▽ More In this paper, we introduce methods from convex optimization to solve the multimarginal transport type problems arise in the context of density functional theory. Convex relaxations are used to provide outer approximation to the set of $N$-representable 2-marginals and 3-marginals, which in turn provide lower bounds to the energy. We further propose rounding schemes to obtain upper bound to the energy. Numerical experiments demonstrate a gap of the order of $10^{-3}$ to $10^{-2}$ between the upper and lower bounds. The Kantorovich potential of the multi-marginal transport problem is also approximated with a similar accuracy. △ Less

Submitted 13 August, 2018; originally announced August 2018.

Comments: 22 pages, 6 figures

MSC Class: 49M20 ACM Class: G.1.6; G.1.10; J.2

Solving for high dimensional committor functions using artificial neural networks

Authors: Yuehaw Khoo, Jianfeng Lu, Lexing Ying

Abstract: In this note we propose a method based on artificial neural network to study the transition between states governed by stochastic processes. In particular, we aim for numerical schemes for the committor function, the central object of transition path theory, which satisfies a high-dimensional Fokker-Planck equation. By working with the variational formulation of such partial differential equation… ▽ More In this note we propose a method based on artificial neural network to study the transition between states governed by stochastic processes. In particular, we aim for numerical schemes for the committor function, the central object of transition path theory, which satisfies a high-dimensional Fokker-Planck equation. By working with the variational formulation of such partial differential equation and parameterizing the committor function in terms of a neural network, approximations can be obtained via optimizing the neural network weights using stochastic algorithms. The numerical examples show that moderate accuracy can be achieved for high-dimensional problems. △ Less

Submitted 28 February, 2018; originally announced February 2018.

Comments: 12 pages, 6 figures

MSC Class: 65Nxx

Efficient construction of tensor ring representations from sampling

Authors: Yuehaw Khoo, Jianfeng Lu, Lexing Ying

Abstract: In this paper we propose an efficient method to compress a high dimensional function into a tensor ring format, based on alternating least-squares (ALS). Since the function has size exponential in $d$ where $d$ is the number of dimensions, we propose efficient sampling scheme to obtain $O(d)$ important samples in order to learn the tensor ring. Furthermore, we devise an initialization method for A… ▽ More In this paper we propose an efficient method to compress a high dimensional function into a tensor ring format, based on alternating least-squares (ALS). Since the function has size exponential in $d$ where $d$ is the number of dimensions, we propose efficient sampling scheme to obtain $O(d)$ important samples in order to learn the tensor ring. Furthermore, we devise an initialization method for ALS that allows fast convergence in practice. Numerical examples show that to approximate a function with similar accuracy, the tensor ring format provided by the proposed method has less parameters than tensor-train format and also better respects the structure of the original function. △ Less

Submitted 27 June, 2019; v1 submitted 2 November, 2017; originally announced November 2017.

MSC Class: 65D15; 33F05; 15A69 ACM Class: G.1.3; G.1.10

Solving parametric PDE problems with artificial neural networks

Authors: Yuehaw Khoo, Jianfeng Lu, Lexing Ying

Abstract: The curse of dimensionality is commonly encountered in numerical partial differential equations (PDE), especially when uncertainties have to be modeled into the equations as random coefficients. However, very often the variability of physical quantities derived from a PDE can be captured by a few features on the space of the coefficient fields. Based on such an observation, we propose using a neur… ▽ More The curse of dimensionality is commonly encountered in numerical partial differential equations (PDE), especially when uncertainties have to be modeled into the equations as random coefficients. However, very often the variability of physical quantities derived from a PDE can be captured by a few features on the space of the coefficient fields. Based on such an observation, we propose using a neural-network (NN) based method to parameterize the physical quantity of interest as a function of input coefficients. The representability of such quantity using a neural-network can be justified by viewing the neural-network as performing time evolution to find the solutions to the PDE. We further demonstrate the simplicity and accuracy of the approach through notable examples of PDEs in engineering and physics. △ Less

Submitted 22 May, 2018; v1 submitted 11 July, 2017; originally announced July 2017.

Comments: 17 pages, 4 figures, 2 tables

MSC Class: 65Nxx

Journal ref: Eur. J. Appl. Math 32 (2021) 421-435

Semidefinite Programming Approach for the Quadratic Assignment Problem with a Sparse Graph

Authors: Jose F. S. Bravo Ferreira, Yuehaw Khoo, Amit Singer

Abstract: The matching problem between two adjacency matrices can be formulated as the NP-hard quadratic assignment problem (QAP). Previous work on semidefinite programming (SDP) relaxations to the QAP have produced solutions that are often tight in practice, but such SDPs typically scale badly, involving matrix variables of dimension $n^2$ where n is the number of nodes. To achieve a speed up, we propose a… ▽ More The matching problem between two adjacency matrices can be formulated as the NP-hard quadratic assignment problem (QAP). Previous work on semidefinite programming (SDP) relaxations to the QAP have produced solutions that are often tight in practice, but such SDPs typically scale badly, involving matrix variables of dimension $n^2$ where n is the number of nodes. To achieve a speed up, we propose a further relaxation of the SDP involving a number of positive semidefinite matrices of dimension $\mathcal{O}(n)$ no greater than the number of edges in one of the graphs. The relaxation can be further strengthened by considering cliques in the graph, instead of edges. The dual problem of this novel relaxation has a natural three-block structure that can be solved via a convergent Augmented Direction Method of Multipliers (ADMM) in a distributed manner, where the most expensive step per iteration is computing the eigendecomposition of matrices of dimension $\mathcal{O}(n)$. The new SDP relaxation produces strong bounds on quadratic assignment problems where one of the graphs is sparse with reduced computational complexity and running times, and can be used in the context of nuclear magnetic resonance spectroscopy (NMR) to tackle the assignment problem. △ Less

Submitted 27 March, 2017; originally announced March 2017.

Comments: 31 pages

Integrating NOE and RDC using sum-of-squares relaxation for protein structure determination

Authors: Yuehaw Khoo, Amit Singer, David Cowburn

Abstract: We revisit the problem of protein structure determination from geometrical restraints from NMR, using convex optimization. It is well-known that the NP-hard distance geometry problem of determining atomic positions from pairwise distance restraints can be relaxed into a convex semidefinite program. Often the NOE distance restraints are too imprecise and sparse for accurate structure determination.… ▽ More We revisit the problem of protein structure determination from geometrical restraints from NMR, using convex optimization. It is well-known that the NP-hard distance geometry problem of determining atomic positions from pairwise distance restraints can be relaxed into a convex semidefinite program. Often the NOE distance restraints are too imprecise and sparse for accurate structure determination. Residual dipolar coupling (RDC) measurements provide additional geometric information on the angles between atom-pair directions and axes of the principal-axis-frame. The optimization problem involving RDC is highly non-convex and requires a good initialization even within the simulated annealing framework. In this paper, we model the protein backbone as an articulated structure composed of rigid units. Determining the rotation of each rigid unit gives the full protein structure. We propose solving the non-convex optimization problems using the sum-of-squares (SOS) hierarchy. The two algorithms - RDC-SOS and RDC-NOE-SOS, have polynomial time complexity in the number of amino-acid residues and run efficiently on a standard desktop. In many instances, the proposed methods exactly recover the solution to the original non-convex optimization problem. We introduce a statistical tool, the Cramer-Rao bound (CRB), to provide an information theoretic bound on the highest resolution one can hope to achieve when determining protein structure from noisy measurements using any methodology. Our simulation results show that when the RDC measurements are corrupted by Gaussian noise of realistic variance, both SOS based algorithms attain the CRB. We successfully apply our method in a divide-and-conquer fashion to determine the structure of ubiquitin from experimental NOE and RDC measurements, achieving more accurate and faster reconstructions compared to the current state of the art. △ Less

Submitted 26 February, 2017; v1 submitted 6 April, 2016; originally announced April 2016.

Comments: 41 pages, 5 figures

MSC Class: 90C90; 90C22; 68T40; 92C40 ACM Class: I.4.9; G.1.6

Non-iterative rigid 2D/3D point-set registration using semidefinite programming

Authors: Yuehaw Khoo, Ankur Kapoor

Abstract: We describe a convex programming framework for pose estimation in 2D/3D point-set registration with unknown point correspondences. We give two mixed-integer nonlinear program (MINP) formulations of the 2D/3D registration problem when there are multiple 2D images, and propose convex relaxations for both of the MINPs to semidefinite programs (SDP) that can be solved efficiently by interior point met… ▽ More We describe a convex programming framework for pose estimation in 2D/3D point-set registration with unknown point correspondences. We give two mixed-integer nonlinear program (MINP) formulations of the 2D/3D registration problem when there are multiple 2D images, and propose convex relaxations for both of the MINPs to semidefinite programs (SDP) that can be solved efficiently by interior point methods. Our approach to the 2D/3D registration problem is non-iterative in nature as we jointly solve for pose and correspondence. Furthermore, these convex programs can readily incorporate feature descriptors of points to enhance registration results. We prove that the convex programs exactly recover the solution to the original nonconvex 2D/3D registration problem under noiseless condition. We apply these formulations to the registration of 3D models of coronary vessels to their 2D projections obtained from multiple intra-operative fluoroscopic images. For this application, we experimentally corroborate the exact recovery property in the absence of noise and further demonstrate robustness of the convex programs in the presence of noise. △ Less

Submitted 6 April, 2016; v1 submitted 3 January, 2015; originally announced January 2015.

Comments: 15 pages, 7 figures

MSC Class: 90C22; 92C55 ACM Class: G.1.6; I.4.9

Open problem: Tightness of maximum likelihood semidefinite relaxations

Authors: Afonso S. Bandeira, Yuehaw Khoo, Amit Singer

Abstract: We have observed an interesting, yet unexplained, phenomenon: Semidefinite programming (SDP) based relaxations of maximum likelihood estimators (MLE) tend to be tight in recovery problems with noisy data, even when MLE cannot exactly recover the ground truth. Several results establish tightness of SDP based relaxations in the regime where exact recovery from MLE is possible. However, to the best o… ▽ More We have observed an interesting, yet unexplained, phenomenon: Semidefinite programming (SDP) based relaxations of maximum likelihood estimators (MLE) tend to be tight in recovery problems with noisy data, even when MLE cannot exactly recover the ground truth. Several results establish tightness of SDP based relaxations in the regime where exact recovery from MLE is possible. However, to the best of our knowledge, their tightness is not understood beyond this regime. As an illustrative example, we focus on the generalized Procrustes problem. △ Less

Submitted 9 April, 2014; originally announced April 2014.

Large-Scale Sensor Network Localization via Rigid Subnetwork Registration

Authors: Kunal N. Chaudhury, Yuehaw Khoo, Amit Singer

Abstract: In this paper, we describe an algorithm for sensor network localization (SNL) that proceeds by dividing the whole network into smaller subnetworks, then localizes them in parallel using some fast and accurate algorithm, and finally registers the localized subnetworks in a global coordinate system. We demonstrate that this divide-and-conquer algorithm can be used to leverage existing high-precision… ▽ More In this paper, we describe an algorithm for sensor network localization (SNL) that proceeds by dividing the whole network into smaller subnetworks, then localizes them in parallel using some fast and accurate algorithm, and finally registers the localized subnetworks in a global coordinate system. We demonstrate that this divide-and-conquer algorithm can be used to leverage existing high-precision SNL algorithms to large-scale networks, which could otherwise only be applied to small-to-medium sized networks. The main contribution of this paper concerns the final registration phase. In particular, we consider a least-squares formulation of the registration problem (both with and without anchor constraints) and demonstrate how this otherwise non-convex problem can be relaxed into a tractable convex program. We provide some preliminary simulation results for large-scale SNL demonstrating that the proposed registration algorithm (together with an accurate localization scheme) offers a good tradeoff between run time and accuracy. △ Less

Submitted 15 January, 2015; v1 submitted 29 October, 2013; originally announced October 2013.

Comments: 5 pages, 8 figures, 1 table. To appear in Proc. IEEE International Conference on Acoustics, Speech, and Signal Processing, April 19-24, 2015

Global registration of multiple point clouds using semidefinite programming

Authors: Kunal N. Chaudhury, Yuehaw Khoo, Amit Singer

Abstract: Consider $N$ points in $\mathbb{R}^d$ and $M$ local coordinate systems that are related through unknown rigid transforms. For each point we are given (possibly noisy) measurements of its local coordinates in some of the coordinate systems. Alternatively, for each coordinate system, we observe the coordinates of a subset of the points. The problem of estimating the global coordinates of the $N$ poi… ▽ More Consider $N$ points in $\mathbb{R}^d$ and $M$ local coordinate systems that are related through unknown rigid transforms. For each point we are given (possibly noisy) measurements of its local coordinates in some of the coordinate systems. Alternatively, for each coordinate system, we observe the coordinates of a subset of the points. The problem of estimating the global coordinates of the $N$ points (up to a rigid transform) from such measurements comes up in distributed approaches to molecular conformation and sensor network localization, and also in computer vision and graphics. The least-squares formulation of this problem, though non-convex, has a well known closed-form solution when $M=2$ (based on the singular value decomposition). However, no closed form solution is known for $M\geq 3$. In this paper, we demonstrate how the least-squares formulation can be relaxed into a convex program, namely a semidefinite program (SDP). By setting up connections between the uniqueness of this SDP and results from rigidity theory, we prove conditions for exact and stable recovery for the SDP relaxation. In particular, we prove that the SDP relaxation can guarantee recovery under more adversarial conditions compared to earlier proposed spectral relaxations, and derive error bounds for the registration error incurred by the SDP relaxation. We also present results of numerical experiments on simulated data to confirm the theoretical findings. We empirically demonstrate that (a) unlike the spectral relaxation, the relaxation gap is mostly zero for the semidefinite program (i.e., we are able to solve the original non-convex least-squares problem) up to a certain noise threshold, and (b) the semidefinite program performs significantly better than spectral and manifold-optimization methods, particularly at large noise levels. △ Less

Submitted 23 December, 2014; v1 submitted 21 June, 2013; originally announced June 2013.

Comments: 33 pages, 12 figures. To appear in SIAM Journal on Optimization

MSC Class: 90C22; 52C25; 05C50