Machine Learning

Authors and titles for February 2024

[1] arXiv:2402.00168 [pdf, html, other]

Title: Continuous Treatment Effects with Surrogate Outcomes

Zhenghao Zeng, David Arbour, Avi Feller, Raghavendra Addanki, Ryan Rossi, Ritwik Sinha, Edward H. Kennedy

Comments: 30 pages, 7 figures

Subjects: Machine Learning (stat.ML); Machine Learning (cs.LG); Methodology (stat.ME)

[2] arXiv:2402.00501 [pdf, other]

Title: Equivalence of the Empirical Risk Minimization to Regularization on the Family of f-Divergences

Francisco Daunas, Iñaki Esnaola, Samir M. Perlaza, H. Vincent Poor

Comments: Submitted to the IEEE Symposium in Information Theory 2024. arXiv admin note: text overlap with arXiv:2306.07123

Subjects: Machine Learning (stat.ML); Information Theory (cs.IT); Machine Learning (cs.LG)

[3] arXiv:2402.00623 [pdf, other]

Title: Bayesian Causal Inference with Gaussian Process Networks

Enrico Giudice, Jack Kuipers, Giusi Moffa

Subjects: Machine Learning (stat.ML); Machine Learning (cs.LG); Methodology (stat.ME)

[4] arXiv:2402.00645 [pdf, other]

Title: Spectrally Transformed Kernel Regression

Runtian Zhai, Rattana Pukdee, Roger Jin, Maria-Florina Balcan, Pradeep Ravikumar

Comments: ICLR 2024 spotlight. 36 pages

Subjects: Machine Learning (stat.ML); Machine Learning (cs.LG)

[5] arXiv:2402.01000 [pdf, html, other]

Title: Multivariate Probabilistic Time Series Forecasting with Correlated Errors

Vincent Zhihao Zheng, Lijun Sun

Comments: This paper extends the work presented in arXiv:2305.17028 to a multivariate setting

Subjects: Machine Learning (stat.ML); Machine Learning (cs.LG)

[6] arXiv:2402.01050 [pdf, html, other]

Title: Distributed MCMC inference for Bayesian Non-Parametric Latent Block Model

Reda Khoufache, Anisse Belhadj, Hanene Azzag, Mustapha Lebbah

Comments: Accepted to PaKDD 2024

Subjects: Machine Learning (stat.ML); Machine Learning (cs.LG); Computation (stat.CO)

[7] arXiv:2402.01089 [pdf, html, other]

Title: No Free Prune: Information-Theoretic Barriers to Pruning at Initialization

Tanishq Kumar, Kevin Luo, Mark Sellke

Subjects: Machine Learning (stat.ML); Machine Learning (cs.LG)

[8] arXiv:2402.01090 [pdf, html, other]

Title: Scalable Higher-Order Tensor Product Spline Models

David Rügamer

Comments: Accepted at the 27th International Conference on Artificial Intelligence and Statistics (AISTATS) 2024. arXiv admin note: substantial text overlap with arXiv:2205.14515

Subjects: Machine Learning (stat.ML); Machine Learning (cs.LG); Computation (stat.CO)

[9] arXiv:2402.01092 [pdf, html, other]

Title: A Dynamical Model of Neural Scaling Laws

Blake Bordelon, Alexander Atanasov, Cengiz Pehlevan

Comments: ICML Camera Ready. Included online SGD section with additional simulations and its connection to large sample limit of our gradient flow theory. Fixed typo in Appendix eq 112

Subjects: Machine Learning (stat.ML); Disordered Systems and Neural Networks (cond-mat.dis-nn); Machine Learning (cs.LG)

[10] arXiv:2402.01139 [pdf, html, other]

Title: Online conformal prediction with decaying step sizes

Anastasios N. Angelopoulos, Rina Foygel Barber, Stephen Bates

Subjects: Machine Learning (stat.ML); Machine Learning (cs.LG); Methodology (stat.ME)

[11] arXiv:2402.01258 [pdf, html, other]

Title: Transformers Learn Nonlinear Features In Context: Nonconvex Mean-field Dynamics on the Attention Landscape

Juno Kim, Taiji Suzuki

Comments: ICML 2024 Oral

Subjects: Machine Learning (stat.ML); Machine Learning (cs.LG)

[12] arXiv:2402.01382 [pdf, other]

Title: Emergence of heavy tails in homogenized stochastic gradient descent

Zhe Jiao, Martin Keller-Ressel

Subjects: Machine Learning (stat.ML); Machine Learning (cs.LG)

[13] arXiv:2402.01400 [pdf, html, other]

Title: Query-Efficient Correlation Clustering with Noisy Oracle

Yuko Kuroki, Atsushi Miyauchi, Francesco Bonchi, Wei Chen

Comments: Accepted to NeurIPS 2024

Subjects: Machine Learning (stat.ML); Data Structures and Algorithms (cs.DS); Machine Learning (cs.LG)

[14] arXiv:2402.01434 [pdf, other]

Title: Conditioning non-linear and infinite-dimensional diffusion processes

Elizabeth Louise Baker, Gefan Yang, Michael L. Severinsen, Christy Anna Hipsley, Stefan Sommer

Subjects: Machine Learning (stat.ML); Machine Learning (cs.LG); Computation (stat.CO)

[15] arXiv:2402.01460 [pdf, other]

Title: Deep conditional distribution learning via conditional F\"ollmer flow

Jinyuan Chang, Zhao Ding, Yuling Jiao, Ruoxuan Li, Jerry Zhijian Yang

Comments: The original title of this paper is "Deep Conditional Generative Learning: Model and Error Analysis"

Subjects: Machine Learning (stat.ML); Machine Learning (cs.LG)

[16] arXiv:2402.01493 [pdf, html, other]

Title: Sliced-Wasserstein Estimation with Spherical Harmonics as Control Variates

Rémi Leluc, Aymeric Dieuleveut, François Portier, Johan Segers, Aigerim Zhuman

Comments: Accepted to ICML 2024

Subjects: Machine Learning (stat.ML); Machine Learning (cs.LG)

[17] arXiv:2402.01502 [pdf, html, other]

Title: Why do Random Forests Work? Understanding Tree Ensembles as Self-Regularizing Adaptive Smoothers

Alicia Curth, Alan Jeffares, Mihaela van der Schaar

Subjects: Machine Learning (stat.ML); Machine Learning (cs.LG)

[18] arXiv:2402.01810 [pdf, html, other]

Title: Parameter uncertainties for imperfect surrogate models in the low-noise regime

Thomas D Swinburne, Danny Perez

Subjects: Machine Learning (stat.ML); Machine Learning (cs.LG); Data Analysis, Statistics and Probability (physics.data-an)

[19] arXiv:2402.01855 [pdf, other]

Title: SPDE priors for uncertainty quantification of end-to-end neural data assimilation schemes

Maxime Beauchamp, Nicolas Desassis, J. Emmanuel Johnson, Simon Benaichouche, Pierre Tandeo, Ronan Fablet

Subjects: Machine Learning (stat.ML); Machine Learning (cs.LG); Image and Video Processing (eess.IV)

[20] arXiv:2402.01887 [pdf, html, other]

Title: On $f$-Divergence Principled Domain Adaptation: An Improved Framework

Ziqiao Wang, Yongyi Mao

Comments: Accepted at NeurIPS 2024

Subjects: Machine Learning (stat.ML); Computer Vision and Pattern Recognition (cs.CV); Machine Learning (cs.LG)

[21] arXiv:2402.01900 [pdf, other]

Title: Distributional Off-policy Evaluation with Bellman Residual Minimization

Sungee Hong, Zhengling Qi, Raymond K. W. Wong

Subjects: Machine Learning (stat.ML); Machine Learning (cs.LG)

[22] arXiv:2402.01972 [pdf, other]

Title: Combining T-learning and DR-learning: a framework for oracle-efficient estimation of causal contrasts

Lars van der Laan, Marco Carone, Alex Luedtke

Subjects: Machine Learning (stat.ML); Machine Learning (cs.LG); Methodology (stat.ME)

[23] arXiv:2402.02041 [pdf, html, other]

Title: $\alpha$-Divergence Loss Function for Neural Density Ratio Estimation

Yoshiaki Kitazawa

Comments: $\mathcal{T}_{\text{Lip}}$ in Theorem 7.1 (Theorem B.15.) was changed to the set of all locally Lipschitz continuous functions. In the previous version, $\mathcal{T}_{\text{Lip}}$ was defined as the set of all Lipschitz continuous functions, which is unsuitable for the statement of case (ii) in the theorem

Subjects: Machine Learning (stat.ML); Machine Learning (cs.LG)

[24] arXiv:2402.02098 [pdf, html, other]

Title: Self-attention Networks Localize When QK-eigenspectrum Concentrates

Han Bao, Ryuichiro Hataya, Ryo Karakida

Subjects: Machine Learning (stat.ML); Machine Learning (cs.LG)

[25] arXiv:2402.02111 [pdf, html, other]

Title: Accelerating Look-ahead in Bayesian Optimization: Multilevel Monte Carlo is All you Need

Shangda Yang, Vitaly Zankin, Maximilian Balandat, Stefan Scherer, Kevin Carlberg, Neil Walton, Kody J. H. Law

Comments: Preprint ICML 2024

Subjects: Machine Learning (stat.ML); Machine Learning (cs.LG); Optimization and Control (math.OC); Probability (math.PR); Computation (stat.CO); Methodology (stat.ME)

[26] arXiv:2402.02162 [pdf, other]

Title: A Bayesian cluster validity index

Nathakhun Wiroonsri, Onthada Preedasawakul

Comments: 23 pages

Subjects: Machine Learning (stat.ML); Machine Learning (cs.LG); Statistics Theory (math.ST)

[27] arXiv:2402.02171 [pdf, html, other]

Title: Off-Policy Evaluation of Slate Bandit Policies via Optimizing Abstraction

Haruka Kiyohara, Masahiro Nomura, Yuta Saito

Comments: WWW2024

Subjects: Machine Learning (stat.ML); Machine Learning (cs.LG)

[28] arXiv:2402.02190 [pdf, html, other]

Title: Continuous Tensor Relaxation for Finding Diverse Solutions in Combinatorial Optimization Problems

Yuma Ichikawa, Hiroaki Iwashita

Comments: 20 pages, 12 figures. arXiv admin note: text overlap with arXiv:2309.16965

Subjects: Machine Learning (stat.ML); Machine Learning (cs.LG); Computation (stat.CO); Methodology (stat.ME)

[29] arXiv:2402.02459 [pdf, html, other]

Title: On Minimum Trace Factor Analysis -- An Old Song Sung to a New Tune

C. Li, A. Shkolnik

Subjects: Machine Learning (stat.ML); Machine Learning (cs.LG); Methodology (stat.ME)

[30] arXiv:2402.02817 [pdf, other]

Title: Bayes-Optimal Fair Classification with Linear Disparity Constraints via Pre-, In-, and Post-processing

Xianli Zeng, Guang Cheng, Edgar Dobriban

Comments: This paper replaces the preprint "Bayes-optimal classifiers under group fairness" by Xianli Zeng, Edgar Dobriban, and Guang Cheng (arXiv:2202.09724)

Subjects: Machine Learning (stat.ML); Computers and Society (cs.CY); Machine Learning (cs.LG)

[31] arXiv:2402.02857 [pdf, other]

Title: Non-asymptotic Analysis of Biased Adaptive Stochastic Approximation

Sobihan Surendran (LPSM (UMR\_8001)), Antoine Godichon-Baggioni (LPSM (UMR\_8001)), Adeline Fermanian, Sylvain Le Corff (LPSM (UMR\_8001))

Subjects: Machine Learning (stat.ML); Machine Learning (cs.LG)

[32] arXiv:2402.02862 [pdf, other]

Title: Graph Neural Machine: A New Model for Learning with Tabular Data

Giannis Nikolentzos, Siyun Wang, Johannes Lutzeyer, Michalis Vazirgiannis

Subjects: Machine Learning (stat.ML); Machine Learning (cs.LG)

[33] arXiv:2402.02952 [pdf, html, other]

Title: On Least Square Estimation in Softmax Gating Mixture of Experts

Huy Nguyen, Nhat Ho, Alessandro Rinaldo

Comments: Accepted to ICML 2024, 29 pages, 2 figures, 2 tables

Subjects: Machine Learning (stat.ML); Machine Learning (cs.LG)

[34] arXiv:2402.02969 [pdf, html, other]

Title: Towards Understanding the Word Sensitivity of Attention Layers: A Study via Random Features

Simone Bombari, Marco Mondelli

Comments: Revision after ICML2024 reviews

Subjects: Machine Learning (stat.ML); Computation and Language (cs.CL); Machine Learning (cs.LG)

[35] arXiv:2402.03008 [pdf, html, other]

Title: Diffusive Gibbs Sampling

Wenlin Chen, Mingtian Zhang, Brooks Paige, José Miguel Hernández-Lobato, David Barber

Comments: Accepted for publication at ICML 2024. Code available: this https URL

Subjects: Machine Learning (stat.ML); Machine Learning (cs.LG); Computation (stat.CO)

[36] arXiv:2402.03104 [pdf, other]

Title: High-dimensional Bayesian Optimization via Covariance Matrix Adaptation Strategy

Lam Ngo, Huong Ha, Jeffrey Chan, Vu Nguyen, Hongyu Zhang

Comments: 31 pages, 17 figures

Journal-ref: Transactions on Machine Learning Research 2024

Subjects: Machine Learning (stat.ML); Machine Learning (cs.LG)

[37] arXiv:2402.03169 [pdf, other]

Title: A Random Matrix Approach to Low-Multilinear-Rank Tensor Approximation

Hugo Lebeau, Florent Chatelain, Romain Couillet

Subjects: Machine Learning (stat.ML); Machine Learning (cs.LG); Probability (math.PR)

[38] arXiv:2402.03220 [pdf, html, other]

Title: The Benefits of Reusing Batches for Gradient Descent in Two-Layer Networks: Breaking the Curse of Information and Leap Exponents

Yatin Dandi, Emanuele Troiani, Luca Arnaboldi, Luca Pesce, Lenka Zdeborová, Florent Krzakala

Comments: Accepted at the International Conference on Machine Learning (ICML), 2024

Journal-ref: Proceedings of the 41st International Conference on Machine Learning, PMLR 235:9991-10016, 2024

Subjects: Machine Learning (stat.ML); Machine Learning (cs.LG)

[39] arXiv:2402.03254 [pdf, other]

Title: Minimum Description Length and Generalization Guarantees for Representation Learning

Milad Sefidgaran, Abdellatif Zaidi, Piotr Krasnowski

Comments: Accepted and presented at NeurIPS 2023

Subjects: Machine Learning (stat.ML); Information Theory (cs.IT); Machine Learning (cs.LG)

[40] arXiv:2402.03447 [pdf, html, other]

Title: Challenges in Variable Importance Ranking Under Correlation

Annie Liang, Thomas Jemielita, Andy Liaw, Vladimir Svetnik, Lingkang Huang, Richard Baumgartner, Jason M. Klusowski

Subjects: Machine Learning (stat.ML); Machine Learning (cs.LG); Methodology (stat.ME)

[41] arXiv:2402.03460 [pdf, html, other]

Title: Approximation Rates and VC-Dimension Bounds for (P)ReLU MLP Mixture of Experts

Anastasis Kratsios, Haitz Sáez de Ocáriz Borde, Takashi Furuya, Marc T. Law

Subjects: Machine Learning (stat.ML); Machine Learning (cs.LG); Neural and Evolutionary Computing (cs.NE); Combinatorics (math.CO); Numerical Analysis (math.NA)

[42] arXiv:2402.03485 [pdf, html, other]

Title: Attention Meets Post-hoc Interpretability: A Mathematical Perspective

Gianluigi Lopardo, Frederic Precioso, Damien Garreau

Comments: Accepted at ICML 2024

Subjects: Machine Learning (stat.ML); Computation and Language (cs.CL); Machine Learning (cs.LG)

[43] arXiv:2402.03527 [pdf, html, other]

Title: Consistent Validation for Predictive Methods in Spatial Settings

David R. Burt, Yunyi Shen, Tamara Broderick

Comments: 52 pages, 14 figures

Subjects: Machine Learning (stat.ML); Machine Learning (cs.LG); Methodology (stat.ME)

[44] arXiv:2402.03724 [pdf, html, other]

Title: Statistical Test for Anomaly Detections by Variational Auto-Encoders

Daiki Miwa, Tomohiro Shiraishi, Vo Nguyen Le Duy, Teruyuki Katsuoka, Ichiro Takeuchi

Subjects: Machine Learning (stat.ML); Machine Learning (cs.LG)

[45] arXiv:2402.03779 [pdf, other]

Title: EERO: Early Exit with Reject Option for Efficient Classification with limited budget

Florian Valade (LAMA), Mohamed Hebiri (LAMA), Paul Gay

Subjects: Machine Learning (stat.ML); Machine Learning (cs.LG)

[46] arXiv:2402.03819 [pdf, html, other]

Title: Do we need rebalancing strategies? A theoretical and empirical study around SMOTE and its variants

Abdoulaye Sakho (LPSM (UMR\_8001)), Emmanuel Malherbe, Erwan Scornet (LPSM (UMR\_8001))

Subjects: Machine Learning (stat.ML); Machine Learning (cs.LG)

[47] arXiv:2402.03838 [pdf, html, other]

Title: Gaussian process regression with Sliced Wasserstein Weisfeiler-Lehman graph kernels

Raphaël Carpintero Perez (CMAP), Sébastien da Veiga (ENSAI, CREST), Josselin Garnier (CMAP), Brian Staber

Subjects: Machine Learning (stat.ML); Machine Learning (cs.LG)

[48] arXiv:2402.03990 [pdf, html, other]

Title: Subsampling is not Magic: Why Large Batch Sizes Work for Differentially Private Stochastic Optimisation

Ossi Räisä, Joonas Jälkö, Antti Honkela

Comments: After the publication of this work (ICML 2024), the Conjecture 6.3 has been proven by Kalinin (2024). Kalinin, N. P. Notes on Sampled Gaussian Mechanism. arXiv:2409.04636, 2024

Subjects: Machine Learning (stat.ML); Cryptography and Security (cs.CR); Machine Learning (cs.LG)

[49] arXiv:2402.04022 [pdf, other]

Title: A General Theory for Kernel Packets: from state space model to compactly supported basis

Liang Ding, Rui Tuo

Subjects: Machine Learning (stat.ML); Machine Learning (cs.LG)

[50] arXiv:2402.04038 [pdf, html, other]

Title: PAC-Bayesian Adversarially Robust Generalization Bounds for Graph Neural Network

Tan Sun, Junhong Lin

Comments: 38pages

Subjects: Machine Learning (stat.ML); Machine Learning (cs.LG)