Computation
See recent articles
- [1] arXiv:2409.03094 [pdf, html, other]
-
Title: A Bayesian Optimization through Sequential Monte Carlo and Statistical Physics-Inspired TechniquesComments: 8 pages, 7 figures, accepted to the International Conference on Computer Science 2023Journal-ref: Lecture Notes in Computer Science, vol 14077 (2023)Subjects: Computation (stat.CO); Distributed, Parallel, and Cluster Computing (cs.DC); Data Analysis, Statistics and Probability (physics.data-an)
In this paper, we propose an approach for an application of Bayesian optimization using Sequential Monte Carlo (SMC) and concepts from the statistical physics of classical systems. Our method leverages the power of modern machine learning libraries such as NumPyro and JAX, allowing us to perform Bayesian optimization on multiple platforms, including CPUs, GPUs, TPUs, and in parallel. Our approach enables a low entry level for exploration of the methods while maintaining high performance. We present a promising direction for developing more efficient and effective techniques for a wide range of optimization problems in diverse fields.
New submissions for Friday, 6 September 2024 (showing 1 of 1 entries )
- [2] arXiv:2409.03136 (cross-list from stat.ME) [pdf, html, other]
-
Title: A New Forward Discriminant Analysis Framework Based On Pillai's Trace and ULDASubjects: Methodology (stat.ME); Computation (stat.CO); Machine Learning (stat.ML)
Linear discriminant analysis (LDA), a traditional classification tool, suffers from limitations such as sensitivity to noise and computational challenges when dealing with non-invertible within-class scatter matrices. Traditional stepwise LDA frameworks, which iteratively select the most informative features, often exacerbate these issues by relying heavily on Wilks' $\Lambda$, potentially causing premature stopping of the selection process. This paper introduces a novel forward discriminant analysis framework that integrates Pillai's trace with Uncorrelated Linear Discriminant Analysis (ULDA) to address these challenges, and offers a unified and stand-alone classifier. Through simulations and real-world datasets, the new framework demonstrates effective control of Type I error rates and improved classification accuracy, particularly in cases involving perfect group separations. The results highlight the potential of this approach as a robust alternative to the traditional stepwise LDA framework.
- [3] arXiv:2409.03495 (cross-list from stat.ML) [pdf, html, other]
-
Title: Maximum likelihood inference for high-dimensional problems with multiaffine variable relationsSubjects: Machine Learning (stat.ML); Machine Learning (cs.LG); Systems and Control (eess.SY); Computation (stat.CO)
Maximum Likelihood Estimation of continuous variable models can be very challenging in high dimensions, due to potentially complex probability distributions. The existence of multiple interdependencies among variables can make it very difficult to establish convergence guarantees. This leads to a wide use of brute-force methods, such as grid searching and Monte-Carlo sampling and, when applicable, complex and problem-specific algorithms. In this paper, we consider inference problems where the variables are related by multiaffine expressions. We propose a novel Alternating and Iteratively-Reweighted Least Squares (AIRLS) algorithm, and prove its convergence for problems with Generalized Normal Distributions. We also provide an efficient method to compute the variance of the estimates obtained using AIRLS. Finally, we show how the method can be applied to graphical statistical models. We perform numerical experiments on several inference problems, showing significantly better performance than state-of-the-art approaches in terms of scalability, robustness to noise, and convergence speed due to an empirically observed super-linear convergence rate.
Cross submissions for Friday, 6 September 2024 (showing 2 of 2 entries )
- [4] arXiv:2307.12438 (replaced) [pdf, html, other]
-
Title: Multifidelity Covariance Estimation via Regression on the Manifold of Symmetric Positive Definite MatricesComments: To appear in the SIAM Journal on Mathematics of Data Science (SIMODS)Subjects: Computation (stat.CO); Machine Learning (cs.LG); Numerical Analysis (math.NA)
We introduce a multifidelity estimator of covariance matrices formulated as the solution to a regression problem on the manifold of symmetric positive definite matrices. The estimator is positive definite by construction, and the Mahalanobis distance minimized to obtain it possesses properties enabling practical computation. We show that our manifold regression multifidelity (MRMF) covariance estimator is a maximum likelihood estimator under a certain error model on manifold tangent space. More broadly, we show that our Riemannian regression framework encompasses existing multifidelity covariance estimators constructed from control variates. We demonstrate via numerical examples that the MRMF estimator can provide significant decreases, up to one order of magnitude, in squared estimation error relative to both single-fidelity and other multifidelity covariance estimators. Furthermore, preservation of positive definiteness ensures that our estimator is compatible with downstream tasks, such as data assimilation and metric learning, in which this property is essential.
- [5] arXiv:2409.01547 (replaced) [pdf, html, other]
-
Title: The R package psvmSDR: A Unified Algorithm for Sufficient Dimension Reduction via Principal MachinesComments: version 2.0Subjects: Computation (stat.CO); Machine Learning (stat.ML)
Sufficient dimension reduction (SDR), which seeks a lower-dimensional subspace of the predictors containing regression or classification information has been popular in a machine learning community. In this work, we present a new R software package psvmSDR that implements a new class of SDR estimators, which we call the principal machine (PM) generalized from the principal support vector machine (PSVM). The package covers both linear and nonlinear SDR and provides a function applicable to realtime update scenarios. The package implements the descent algorithm for the PMs to efficiently compute the SDR estimators in various situations. This easy-to-use package will be an attractive alternative to the dr R package that implements classical SDR methods.
- [6] arXiv:2401.04156 (replaced) [pdf, html, other]
-
Title: LASPATED: a Library for the Analysis of SPAtio-TEmporal Discrete dataSubjects: Methodology (stat.ME); Statistics Theory (math.ST); Computation (stat.CO)
We describe methods, tools, and a software library called LASPATED, available on GitHub (at this https URL) to fit models using spatio-temporal data and space-time discretization. A video tutorial for this library is available on YouTube. We consider two types of methods to estimate a non-homogeneous Poisson process in space and time. The methods approximate the arrival intensity function of the Poisson process by discretizing space and time, and estimating arrival intensity as a function of subregion and time interval. With such methods, it is typical that the dimension of the estimator is large relative to the amount of data, and therefore the performance of the estimator can be improved by using additional data. The first method uses additional data to add a regularization term to the likelihood function for calibrating the intensity of the Poisson process. The second method uses additional data to estimate arrival intensity as a function of covariates. We describe a Python package to perform various types of space and time discretization. We also describe two packages for the calibration of the models, one in Matlab and one in C++. We demonstrate the advantages of our methods compared to basic maximum likelihood estimation with simulated and real data. The experiments with real data calibrate models of the arrival process of emergencies to be handled by the Rio de Janeiro emergency medical service.