Showing 1–50 of 53 results for author: Chopin, N

Gradient-free optimization via integration

Authors: Christophe Andrieu, Nicolas Chopin, Ettore Fincato, Mathieu Gerber

Abstract: In this paper we propose a novel, general purpose, algorithm to optimize functions $l\colon \mathbb{R}^d \rightarrow \mathbb{R}$ not assumed to be convex or differentiable or even continuous. The main idea is to sequentially fit a sequence of parametric probability densities, possessing a concentration property, to $l$ using a Bayesian update followed by a reprojection back onto the chosen paramet… ▽ More In this paper we propose a novel, general purpose, algorithm to optimize functions $l\colon \mathbb{R}^d \rightarrow \mathbb{R}$ not assumed to be convex or differentiable or even continuous. The main idea is to sequentially fit a sequence of parametric probability densities, possessing a concentration property, to $l$ using a Bayesian update followed by a reprojection back onto the chosen parametric sequence. Remarkably, with the sequence chosen to be from the exponential family, reprojection essentially boils down to the computation of expectations. Our algorithm therefore lends itself to Monte Carlo approximation, ranging from plain to Sequential Monte Carlo (SMC) methods. The algorithm is therefore particularly simple to implement and we illustrate performance on a challenging Machine Learning classification problem. Our methodology naturally extends to the scenario where only noisy measurements of $l$ are available and retains ease of implementation and performance. At a theoretical level we establish, in a fairly general scenario, that our framework can be viewed as implicitly implementing a time inhomogeneous gradient descent algorithm on a sequence of smoothed approximations of $l$. This opens the door to establishing convergence of the algorithm and provide theoretical guarantees. Along the way, we establish new results for inhomogeneous gradient descent algorithms of independent interest. △ Less

Submitted 1 August, 2024; originally announced August 2024.

Logarithmic Smoothing for Pessimistic Off-Policy Evaluation, Selection and Learning

Authors: Otmane Sakhi, Imad Aouali, Pierre Alquier, Nicolas Chopin

Abstract: This work investigates the offline formulation of the contextual bandit problem, where the goal is to leverage past interactions collected under a behavior policy to evaluate, select, and learn new, potentially better-performing, policies. Motivated by critical applications, we move beyond point estimators. Instead, we adopt the principle of pessimism where we construct upper bounds that assess a… ▽ More This work investigates the offline formulation of the contextual bandit problem, where the goal is to leverage past interactions collected under a behavior policy to evaluate, select, and learn new, potentially better-performing, policies. Motivated by critical applications, we move beyond point estimators. Instead, we adopt the principle of pessimism where we construct upper bounds that assess a policy's worst-case performance, enabling us to confidently select and learn improved policies. Precisely, we introduce novel, fully empirical concentration bounds for a broad class of importance weighting risk estimators. These bounds are general enough to cover most existing estimators and pave the way for the development of new ones. In particular, our pursuit of the tightest bound within this class motivates a novel estimator (LS), that logarithmically smooths large importance weights. The bound for LS is provably tighter than all its competitors, and naturally results in improved policy selection and learning strategies. Extensive policy evaluation, selection, and learning experiments highlight the versatility and favorable performance of LS. △ Less

Submitted 23 May, 2024; originally announced May 2024.

Towards a turnkey approach to unbiased Monte Carlo estimation of smooth functions of expectations

Authors: Nicolas Chopin, Francesca R. Crucinio, Sumeetpal S. Singh

Abstract: Given a smooth function $f$, we develop a general approach to turn Monte Carlo samples with expectation $m$ into an unbiased estimate of $f(m)$. Specifically, we develop estimators that are based on randomly truncating the Taylor series expansion of $f$ and estimating the coefficients of the truncated series. We derive their properties and propose a strategy to set their tuning parameters -- which… ▽ More Given a smooth function $f$, we develop a general approach to turn Monte Carlo samples with expectation $m$ into an unbiased estimate of $f(m)$. Specifically, we develop estimators that are based on randomly truncating the Taylor series expansion of $f$ and estimating the coefficients of the truncated series. We derive their properties and propose a strategy to set their tuning parameters -- which depend on $m$ -- automatically, with a view to make the whole approach simple to use. We develop our methods for the specific functions $f(x)=\log x$ and $f(x)=1/x$, as they arise in several statistical applications such as maximum likelihood estimation of latent variable models and Bayesian inference for un-normalised models. Detailed numerical studies are performed for a range of applications to determine how competitive and reliable the proposed approach is. △ Less

Submitted 12 April, 2024; v1 submitted 29 March, 2024; originally announced March 2024.

A connection between Tempering and Entropic Mirror Descent

Authors: Nicolas Chopin, Francesca R. Crucinio, Anna Korba

Abstract: This paper explores the connections between tempering (for Sequential Monte Carlo; SMC) and entropic mirror descent to sample from a target probability distribution whose unnormalized density is known. We establish that tempering SMC corresponds to entropic mirror descent applied to the reverse Kullback-Leibler (KL) divergence and obtain convergence rates for the tempering iterates. Our result mot… ▽ More This paper explores the connections between tempering (for Sequential Monte Carlo; SMC) and entropic mirror descent to sample from a target probability distribution whose unnormalized density is known. We establish that tempering SMC corresponds to entropic mirror descent applied to the reverse Kullback-Leibler (KL) divergence and obtain convergence rates for the tempering iterates. Our result motivates the tempering iterates from an optimization point of view, showing that tempering can be seen as a descent scheme of the KL divergence with respect to the Fisher-Rao geometry, in contrast to Langevin dynamics that perform descent of the KL with respect to the Wasserstein-2 geometry. We exploit the connection between tempering and mirror descent iterates to justify common practices in SMC and derive adaptive tempering rules that improve over other alternative benchmarks in the literature. △ Less

Submitted 16 June, 2024; v1 submitted 18 October, 2023; originally announced October 2023.

Fast Slate Policy Optimization: Going Beyond Plackett-Luce

Authors: Otmane Sakhi, David Rohde, Nicolas Chopin

Abstract: An increasingly important building block of large scale machine learning systems is based on returning slates; an ordered lists of items given a query. Applications of this technology include: search, information retrieval and recommender systems. When the action space is large, decision systems are restricted to a particular structure to complete online queries quickly. This paper addresses the o… ▽ More An increasingly important building block of large scale machine learning systems is based on returning slates; an ordered lists of items given a query. Applications of this technology include: search, information retrieval and recommender systems. When the action space is large, decision systems are restricted to a particular structure to complete online queries quickly. This paper addresses the optimization of these large scale decision systems given an arbitrary reward function. We cast this learning problem in a policy optimization framework and propose a new class of policies, born from a novel relaxation of decision functions. This results in a simple, yet efficient learning algorithm that scales to massive action spaces. We compare our method to the commonly adopted Plackett-Luce policy class and demonstrate the effectiveness of our approach on problems with action space sizes in the order of millions. △ Less

Submitted 29 December, 2023; v1 submitted 3 August, 2023; originally announced August 2023.

Comments: Transactions on Machine Learning Research

Debiasing Piecewise Deterministic Markov Process samplers using couplings

Authors: Adrien Corenflos, Matthew Sutton, Nicolas Chopin

Abstract: Monte Carlo methods - such as Markov chain Monte Carlo (MCMC) and piecewise deterministic Markov process (PDMP) samplers - provide asymptotically exact estimators of expectations under a target distribution. There is growing interest in alternatives to this asymptotic regime, in particular in constructing estimators that are exact in the limit of an infinite amount of computing processors, rather… ▽ More Monte Carlo methods - such as Markov chain Monte Carlo (MCMC) and piecewise deterministic Markov process (PDMP) samplers - provide asymptotically exact estimators of expectations under a target distribution. There is growing interest in alternatives to this asymptotic regime, in particular in constructing estimators that are exact in the limit of an infinite amount of computing processors, rather than in the limit of an infinite number of Markov iterations. In particular, Jacob et al. (2020) introduced coupled MCMC estimators to remove the non-asymptotic bias, resulting in MCMC estimators that can be embarrassingly parallelised. In this work, we extend the estimators of Jacob et al. (2020) to the continuous-time context and derive couplings for the bouncy, the boomerang and the coordinate samplers. Some preliminary empirical results are included that demonstrate the reasonable scaling of our method with the dimension of the target. △ Less

Submitted 27 June, 2023; originally announced June 2023.

Comments: 30 pages, 3 figures. This is a preliminary version which does not include all the experiments

Scalable Bayesian bi-level variable selection in generalized linear models

Authors: Younès Youssfi, Nicolas Chopin

Abstract: Motivated by a real-world application in cardiology, we develop an algorithm to perform Bayesian bi-level variable selection in a generalized linear model, for datasets that may be large both in terms of the number of individuals and the number of predictors. Our algorithm relies on the waste-free SMC Sequential Monte Carlo methodology of Dau and Chopin (2022), a new proposal mechanism to deal wit… ▽ More Motivated by a real-world application in cardiology, we develop an algorithm to perform Bayesian bi-level variable selection in a generalized linear model, for datasets that may be large both in terms of the number of individuals and the number of predictors. Our algorithm relies on the waste-free SMC Sequential Monte Carlo methodology of Dau and Chopin (2022), a new proposal mechanism to deal with the constraints specific to bi-level selection (which forbid to select an individual predictor if its group is not selected), and the ALA (approximate Laplace approximation) approach of Rossell et al. (2021). We show in our numerical study that the algorithm may offer reliable performance on large datasets within a few minutes, on both simulated data and real data related to the aforementioned cardiology application. △ Less

Submitted 22 March, 2023; originally announced March 2023.

PAC-Bayesian Offline Contextual Bandits With Guarantees

Authors: Otmane Sakhi, Pierre Alquier, Nicolas Chopin

Abstract: This paper introduces a new principled approach for off-policy learning in contextual bandits. Unlike previous work, our approach does not derive learning principles from intractable or loose bounds. We analyse the problem through the PAC-Bayesian lens, interpreting policies as mixtures of decision rules. This allows us to propose novel generalization bounds and provide tractable algorithms to opt… ▽ More This paper introduces a new principled approach for off-policy learning in contextual bandits. Unlike previous work, our approach does not derive learning principles from intractable or loose bounds. We analyse the problem through the PAC-Bayesian lens, interpreting policies as mixtures of decision rules. This allows us to propose novel generalization bounds and provide tractable algorithms to optimize them. We prove that the derived bounds are tighter than their competitors, and can be optimized directly to confidently improve upon the logging policy offline. Our approach learns policies with guarantees, uses all available data and does not require tuning additional hyperparameters on held-out sets. We demonstrate through extensive experiments the effectiveness of our approach in providing performance guarantees in practical scenarios. △ Less

Submitted 27 May, 2023; v1 submitted 24 October, 2022; originally announced October 2022.

Comments: Accepted to ICML 2023

Higher-order stochastic integration through cubic stratification

Authors: Nicolas Chopin, Mathieu Gerber

Abstract: We propose two novel unbiased estimators of the integral $\int_{[0,1]^{s}}f(u) du$ for a function $f$, which depend on a smoothness parameter $r\in\mathbb{N}$. The first estimator integrates exactly the polynomials of degrees $p<r$ and achieves the optimal error $n^{-1/2-r/s}$ (where $n$ is the number of evaluations of $f$) when $f$ is $r$ times continuously differentiable. The second estimator is… ▽ More We propose two novel unbiased estimators of the integral $\int_{[0,1]^{s}}f(u) du$ for a function $f$, which depend on a smoothness parameter $r\in\mathbb{N}$. The first estimator integrates exactly the polynomials of degrees $p<r$ and achieves the optimal error $n^{-1/2-r/s}$ (where $n$ is the number of evaluations of $f$) when $f$ is $r$ times continuously differentiable. The second estimator is computationally cheaper but it is restricted to functions that vanish on the boundary of $[0,1]^s$. The construction of the two estimators relies on a combination of cubic stratification and control ariates based on numerical derivatives. We provide numerical evidence that they show good performance even for moderate values of $n$. △ Less

Submitted 9 June, 2023; v1 submitted 4 October, 2022; originally announced October 2022.

Comments: Revision according to referees' comments and suggestions

MSC Class: 65D32; 65C05 ACM Class: G.1.4

On backward smoothing algorithms

Authors: Hai-Dang Dau, Nicolas Chopin

Abstract: In the context of state-space models, skeleton-based smoothing algorithms rely on a backward sampling step which by default has a $\mathcal O(N^2)$ complexity (where $N$ is the number of particles). Existing improvements in the literature are unsatisfactory: a popular rejection sampling -- based approach, as we shall show, might lead to badly behaved execution time; another rejection sampler with… ▽ More In the context of state-space models, skeleton-based smoothing algorithms rely on a backward sampling step which by default has a $\mathcal O(N^2)$ complexity (where $N$ is the number of particles). Existing improvements in the literature are unsatisfactory: a popular rejection sampling -- based approach, as we shall show, might lead to badly behaved execution time; another rejection sampler with stopping lacks complexity analysis; yet another MCMC-inspired algorithm comes with no stability guarantee. We provide several results that close these gaps. In particular, we prove a novel non-asymptotic stability theorem, thus enabling smoothing with truly linear complexity and adequate theoretical justification. We propose a general framework which unites most skeleton-based smoothing algorithms in the literature and allows to simultaneously prove their convergence and stability, both in online and offline contexts. Furthermore, we derive, as a special case of that framework, a new coupling-based smoothing algorithm applicable to models with intractable transition densities. We elaborate practical recommendations and confirm those with numerical experiments. △ Less

Submitted 6 March, 2023; v1 submitted 3 July, 2022; originally announced July 2022.

Comments: revised version

MSC Class: 62M05; 65C05; 65Y20

De-biasing particle filtering for a continuous time hidden Markov model with a Cox process observation model

Authors: Ruiyang Jin, Sumeetpal S. Singh, Nicolas Chopin

Abstract: We develop a (nearly) unbiased particle filtering algorithm for a specific class of continuous-time state-space models, such that (a) the latent process $X_t$ is a linear Gaussian diffusion; and (b) the observations arise from a Poisson process with intensity $λ(X_t)$. The likelihood of the posterior probability density function of the latent process includes an intractable path integral. Our algo… ▽ More We develop a (nearly) unbiased particle filtering algorithm for a specific class of continuous-time state-space models, such that (a) the latent process $X_t$ is a linear Gaussian diffusion; and (b) the observations arise from a Poisson process with intensity $λ(X_t)$. The likelihood of the posterior probability density function of the latent process includes an intractable path integral. Our algorithm relies on Poisson estimates which approximate unbiasedly this integral. We show how we can tune these Poisson estimates to ensure that, with large probability, all but a few of the estimates generated by the algorithm are positive. Then replacing the negative estimates by zero leads to a much smaller bias than what would obtain through discretisation. We quantify the probability of negative estimates for certain special cases and show that our particle filter is effectively unbiased. We apply our method to a challenging 3D single molecule tracking example with a Born and Wolf observation model. △ Less

Submitted 30 June, 2022; v1 submitted 21 June, 2022; originally announced June 2022.

Comments: 34 pages, 14 figures

Computational Doob's h-transforms for Online Filtering of Discretely Observed Diffusions

Authors: Nicolas Chopin, Andras Fulop, Jeremy Heng, Alexandre H. Thiery

Abstract: This paper is concerned with online filtering of discretely observed nonlinear diffusion processes. Our approach is based on the fully adapted auxiliary particle filter, which involves Doob's $h$-transforms that are typically intractable. We propose a computational framework to approximate these $h$-transforms by solving the underlying backward Kolmogorov equations using nonlinear Feynman-Kac form… ▽ More This paper is concerned with online filtering of discretely observed nonlinear diffusion processes. Our approach is based on the fully adapted auxiliary particle filter, which involves Doob's $h$-transforms that are typically intractable. We propose a computational framework to approximate these $h$-transforms by solving the underlying backward Kolmogorov equations using nonlinear Feynman-Kac formulas and neural networks. The methodology allows one to train a locally optimal particle filter prior to the data-assimilation procedure. Numerical experiments illustrate that the proposed approach can be orders of magnitude more efficient than state-of-the-art particle filters in the regime of highly informative observations, when the observations are extreme under the model, or if the state dimension is large. △ Less

Submitted 30 May, 2023; v1 submitted 7 June, 2022; originally announced June 2022.

Comments: 20 pages

Journal ref: ICML 2023

On resampling schemes for particle filters with weakly informative observations

Authors: Nicolas Chopin, Sumeetpal S. Singh, Tomás Soto, Matti Vihola

Abstract: We consider particle filters with weakly informative observations (or `potentials') relative to the latent state dynamics. The particular focus of this work is on particle filters to approximate time-discretisations of continuous-time Feynman--Kac path integral models -- a scenario that naturally arises when addressing filtering and smoothing problems in continuous time -- but our findings are ind… ▽ More We consider particle filters with weakly informative observations (or `potentials') relative to the latent state dynamics. The particular focus of this work is on particle filters to approximate time-discretisations of continuous-time Feynman--Kac path integral models -- a scenario that naturally arises when addressing filtering and smoothing problems in continuous time -- but our findings are indicative about weakly informative settings beyond this context too. We study the performance of different resampling schemes, such as systematic resampling, SSP (Srinivasan sampling process) and stratified resampling, as the time-discretisation becomes finer and also identify their continuous-time limit, which is expressed as a suitably defined `infinitesimal generator.' By contrasting these generators, we find that (certain modifications of) systematic and SSP resampling `dominate' stratified and independent `killing' resampling in terms of their limiting overall resampling rate. The reduced intensity of resampling manifests itself in lower variance in our numerical experiment. This efficiency result, through an ordering of the resampling rate, is new to the literature. The second major contribution of this work concerns the analysis of the limiting behaviour of the entire population of particles of the particle filter as the time discretisation becomes finer. We provide the first proof, under general conditions, that the particle approximation of the discretised continuous-time Feynman--Kac path integral models converges to a (uniformly weighted) continuous-time particle system. △ Less

Submitted 9 July, 2022; v1 submitted 18 March, 2022; originally announced March 2022.

Comments: 36 pages, 9 figures

MSC Class: Primary 65C35; secondary 65C05; 65C60; 60J25

De-Sequentialized Monte Carlo: a parallel-in-time particle smoother

Authors: Adrien Corenflos, Nicolas Chopin, Simo Särkkä

Abstract: Particle smoothers are SMC (Sequential Monte Carlo) algorithms designed to approximate the joint distribution of the states given observations from a state-space model. We propose dSMC (de-Sequentialized Monte Carlo), a new particle smoother that is able to process $T$ observations in $\mathcal{O}(\log T)$ time on parallel architecture. This compares favourably with standard particle smoothers, th… ▽ More Particle smoothers are SMC (Sequential Monte Carlo) algorithms designed to approximate the joint distribution of the states given observations from a state-space model. We propose dSMC (de-Sequentialized Monte Carlo), a new particle smoother that is able to process $T$ observations in $\mathcal{O}(\log T)$ time on parallel architecture. This compares favourably with standard particle smoothers, the complexity of which is linear in $T$. We derive $\mathcal{L}_p$ convergence results for dSMC, with an explicit upper bound, polynomial in $T$. We then discuss how to reduce the variance of the smoothing estimates computed by dSMC by (i) designing good proposal distributions for sampling the particles at the initialization of the algorithm, as well as by (ii) using lazy resampling to increase the number of particles used in dSMC. Finally, we design a particle Gibbs sampler based on dSMC, which is able to perform parameter inference in a state-space model at a $\mathcal{O}(\log(T))$ cost on parallel hardware. △ Less

Submitted 4 February, 2022; originally announced February 2022.

Comments: 31 pages, 6 figures

Improved Gibbs samplers for Cosmic Microwave Background power spectrum estimation

Authors: Gabriel Ducrocq, Nicolas Chopin, Josquin Errard, Radek Stompor

Abstract: We study different variants of the Gibbs sampler algorithm from the perspective of their applicability to the estimation of power spectra of the cosmic microwave background (CMB) anisotropies. These include approaches studied earlier in the CMB literature as well as new ones which are proposed in this work. We demonstrate all these variants on full and cut sky simulations and compare their perform… ▽ More We study different variants of the Gibbs sampler algorithm from the perspective of their applicability to the estimation of power spectra of the cosmic microwave background (CMB) anisotropies. These include approaches studied earlier in the CMB literature as well as new ones which are proposed in this work. We demonstrate all these variants on full and cut sky simulations and compare their performance, assessing both their computational and statistical efficiency. For this we employ a consistent comparison metric, an effective sample size (ESS) per second, commonly used in this context in the statistical literature. We show that one of the proposed approaches, referred to as Centered overrelax, which capitalizes on additional, auxiliary variables to minimize computational time needed per sample, and uses overrelaxation to decorrelate subsequent samples, performs better than the standard Gibbs sampler by a factor between one and two orders of magnitude in the nearly full-sky, satellite-like cases. It therefore potentially provides an interesting alternative to the currently favored approaches. △ Less

Submitted 12 May, 2022; v1 submitted 15 November, 2021; originally announced November 2021.

Fast compression of MCMC output

Authors: Nicolas Chopin, Gabriel Ducrocq

Abstract: We propose cube thinning, a novel method for compressing the output of a MCMC (Markov chain Monte Carlo) algorithm when control variates are available. It amounts to resampling the initial MCMC sample (according to weights derived from control variates), while imposing equality constraints on averages of these control variates, using the cube method of \cite{Deville2004}. Its main advantage is tha… ▽ More We propose cube thinning, a novel method for compressing the output of a MCMC (Markov chain Monte Carlo) algorithm when control variates are available. It amounts to resampling the initial MCMC sample (according to weights derived from control variates), while imposing equality constraints on averages of these control variates, using the cube method of \cite{Deville2004}. Its main advantage is that its CPU cost is linear in $N$, the original sample size, and is constant in $M$, the required size for the compressed sample. This compares favourably to Stein thinning \citep{Riabiz2020}, which has complexity $\mathcal{O}(NM^2)$, and which requires the availability of the gradient of the target log-density (which automatically implies the availability of control variates). Our numerical experiments suggest that cube thinning is also competitive in terms of statistical error. △ Less

Submitted 19 July, 2021; v1 submitted 9 July, 2021; originally announced July 2021.

Metropolis-Hastings with Averaged Acceptance Ratios

Authors: Christophe Andrieu, Sinan Yıldırım, Arnaud Doucet, Nicolas Chopin

Abstract: Markov chain Monte Carlo (MCMC) methods to sample from a probability distribution $π$ defined on a space $(Θ,\mathcal{T})$ consist of the simulation of realisations of Markov chains $\{θ_{n},n\geq1\}$ of invariant distribution $π$ and such that the distribution of $θ_{i}$ converges to $π$ as $i\rightarrow\infty$. In practice one is typically interested in the computation of expectations of functio… ▽ More Markov chain Monte Carlo (MCMC) methods to sample from a probability distribution $π$ defined on a space $(Θ,\mathcal{T})$ consist of the simulation of realisations of Markov chains $\{θ_{n},n\geq1\}$ of invariant distribution $π$ and such that the distribution of $θ_{i}$ converges to $π$ as $i\rightarrow\infty$. In practice one is typically interested in the computation of expectations of functions, say $f$, with respect to $π$ and it is also required that averages $M^{-1}\sum_{n=1}^{M}f(θ_{n})$ converge to the expectation of interest. The iterative nature of MCMC makes it difficult to develop generic methods to take advantage of parallel computing environments when interested in reducing time to convergence. While numerous approaches have been proposed to reduce the variance of ergodic averages, including averaging over independent realisations of $\{θ_{n},n\geq1\}$ simulated on several computers, techniques to reduce the "burn-in" of MCMC are scarce. In this paper we explore a simple and generic approach to improve convergence to equilibrium of existing algorithms which rely on the Metropolis-Hastings (MH) update, the main building block of MCMC. The main idea is to use averages of the acceptance ratio w.r.t. multiple realisations of random variables involved, while preserving $π$ as invariant distribution. The methodology requires limited change to existing code, is naturally suited to parallel computing and is shown on our examples to provide substantial performance improvements both in terms of convergence to equilibrium and variance of ergodic averages. In some scenarios gains are observed even on a serial machine. △ Less

Submitted 29 December, 2020; originally announced January 2021.

Comments: arXiv admin note: text overlap with arXiv:1803.09527

Waste-free Sequential Monte Carlo

Authors: Hai-Dang Dau, Nicolas Chopin

Abstract: A standard way to move particles in a SMC sampler is to apply several steps of a MCMC (Markov chain Monte Carlo) kernel. Unfortunately, it is not clear how many steps need to be performed for optimal performance. In addition, the output of the intermediate steps are discarded and thus wasted somehow. We propose a new, waste-free SMC algorithm which uses the outputs of all these intermediate MCMC s… ▽ More A standard way to move particles in a SMC sampler is to apply several steps of a MCMC (Markov chain Monte Carlo) kernel. Unfortunately, it is not clear how many steps need to be performed for optimal performance. In addition, the output of the intermediate steps are discarded and thus wasted somehow. We propose a new, waste-free SMC algorithm which uses the outputs of all these intermediate MCMC steps as particles. We establish that its output is consistent and asymptotically normal. We use the expression of the asymptotic variance to develop various insights on how to implement the algorithm in practice. We develop in particular a method to estimate, from a single run of the algorithm, the asymptotic variance of any particle estimate. We show empirically, through a range of numerical examples, that waste-free SMC tends to outperform standard SMC samplers, and especially so in situations where the mixing of the considered MCMC kernels decreases across iterations (as in tempering or rare event problems). △ Less

Submitted 23 August, 2021; v1 submitted 4 November, 2020; originally announced November 2020.

Comments: revised version

Adaptive Tuning Of Hamiltonian Monte Carlo Within Sequential Monte Carlo

Authors: Alexander Buchholz, Nicolas Chopin, Pierre E. Jacob

Abstract: Sequential Monte Carlo (SMC) samplers form an attractive alternative to MCMC for Bayesian computation. However, their performance depends strongly on the Markov kernels used to rejuvenate particles. We discuss how to calibrate automatically (using the current particles) Hamiltonian Monte Carlo kernels within SMC. To do so, we build upon the adaptive SMC approach of Fearnhead and Taylor (2013), and… ▽ More Sequential Monte Carlo (SMC) samplers form an attractive alternative to MCMC for Bayesian computation. However, their performance depends strongly on the Markov kernels used to rejuvenate particles. We discuss how to calibrate automatically (using the current particles) Hamiltonian Monte Carlo kernels within SMC. To do so, we build upon the adaptive SMC approach of Fearnhead and Taylor (2013), and we also suggest alternative methods. We illustrate the advantages of using HMC kernels within an SMC sampler via an extensive numerical study. △ Less

Submitted 12 February, 2020; v1 submitted 23 August, 2018; originally announced August 2018.

On the utility of Metropolis-Hastings with asymmetric acceptance ratio

Authors: Christophe Andrieu, Arnaud Doucet, Sinan Yıldırım, Nicolas Chopin

Abstract: The Metropolis-Hastings algorithm allows one to sample asymptotically from any probability distribution $π$. There has been recently much work devoted to the development of variants of the MH update which can handle scenarios where such an evaluation is impossible, and yet are guaranteed to sample from $π$ asymptotically. The most popular approach to have emerged is arguably the pseudo-marginal MH… ▽ More The Metropolis-Hastings algorithm allows one to sample asymptotically from any probability distribution $π$. There has been recently much work devoted to the development of variants of the MH update which can handle scenarios where such an evaluation is impossible, and yet are guaranteed to sample from $π$ asymptotically. The most popular approach to have emerged is arguably the pseudo-marginal MH algorithm which substitutes an unbiased estimate of an unnormalised version of $π$ for $π$. Alternative pseudo-marginal algorithms relying instead on unbiased estimates of the MH acceptance ratio have also been proposed. These algorithms can have better properties than standard PM algorithms. Convergence properties of both classes of algorithms are known to depend on the variability of the estimators involved and reduced variability is guaranteed to decrease the asymptotic variance of ergodic averages and will shorten the burn-in period, or convergence to equilibrium, in most scenarios of interest. A simple approach to reduce variability, amenable to parallel computations, consists of averaging independent estimators. However, while averaging estimators of $π$ in a pseudo-marginal algorithm retains the guarantee of sampling from $π$ asymptotically, naive averaging of acceptance ratio estimates breaks detailed balance, leading to incorrect results. We propose an original methodology which allows for a correct implementation of this idea. We establish theoretical properties which parallel those available for standard PM algorithms and discussed above. We demonstrate the interest of the approach on various inference problems. In particular we show that convergence to equilibrium can be significantly shortened, therefore offering the possibility to reduce a user's waiting time in a generic fashion when a parallel computing architecture is available. △ Less

Submitted 26 March, 2018; originally announced March 2018.

Improving approximate Bayesian computation via quasi-Monte Carlo

Authors: Alexander Buchholz, Nicolas Chopin

Abstract: ABC (approximate Bayesian computation) is a general approach for dealing with models with an intractable likelihood. In this work, we derive ABC algorithms based on QMC (quasi- Monte Carlo) sequences. We show that the resulting ABC estimates have a lower variance than their Monte Carlo counter-parts. We also develop QMC variants of sequential ABC algorithms, which progressively adapt the proposal… ▽ More ABC (approximate Bayesian computation) is a general approach for dealing with models with an intractable likelihood. In this work, we derive ABC algorithms based on QMC (quasi- Monte Carlo) sequences. We show that the resulting ABC estimates have a lower variance than their Monte Carlo counter-parts. We also develop QMC variants of sequential ABC algorithms, which progressively adapt the proposal distribution and the acceptance threshold. We illustrate our QMC approach through several examples taken from the ABC literature. Keywords: Approximate Bayesian computation, Likelihood-free inference, Quasi Monte Carlo, Randomized Quasi-Monte Carlo, Adaptive importance sampling △ Less

Submitted 7 May, 2018; v1 submitted 3 October, 2017; originally announced October 2017.

Negative association, ordering and convergence of resampling methods

Authors: Mathieu Gerber, Nicolas Chopin, Nick Whiteley

Abstract: We study convergence and convergence rates for resampling schemes. Our first main result is a general consistency theorem based on the notion of negative association, which is applied to establish the almost-sure weak convergence of measures output from Kitagawa's (1996) stratified resampling method. Carpenter et al's (1999) systematic resampling method is similar in structure but can fail to conv… ▽ More We study convergence and convergence rates for resampling schemes. Our first main result is a general consistency theorem based on the notion of negative association, which is applied to establish the almost-sure weak convergence of measures output from Kitagawa's (1996) stratified resampling method. Carpenter et al's (1999) systematic resampling method is similar in structure but can fail to converge depending on the order of the input samples. We introduce a new resampling algorithm based on a stochastic rounding technique of Srinivasan (2001), which shares some attractive properties of systematic resampling, but which exhibits negative association and therefore converges irrespective of the order of the input samples. We confirm a conjecture made by Kitagawa (1996) that ordering input samples by their states in $\mathbb{R}$ yields a faster rate of convergence; we establish that when particles are ordered using the Hilbert curve in $\mathbb{R}^d$, the variance of the resampling error is ${\scriptscriptstyle\mathcal{O}}(N^{-(1+1/d)})$ under mild conditions, where $N$ is the number of particles. We use these results to establish asymptotic properties of particle algorithms based on resampling schemes that differ from multinomial resampling. △ Less

Submitted 17 January, 2020; v1 submitted 6 July, 2017; originally announced July 2017.

Comments: 54 pages, including 30 pages of supplementary materials (a typo in Algorithm 1 has been corrected)

Sequential quasi-Monte Carlo: Introduction for Non-Experts, Dimension Reduction, Application to Partly Observed Diffusion Processes

Authors: Nicolas Chopin, Mathieu Gerber

Abstract: SMC (Sequential Monte Carlo) is a class of Monte Carlo algorithms for filtering and related sequential problems. Gerber and Chopin (2015) introduced SQMC (Sequential quasi-Monte Carlo), a QMC version of SMC. This paper has two objectives: (a) to introduce Sequential Monte Carlo to the QMC community, whose members are usually less familiar with state-space models and particle filtering; (b) to exte… ▽ More SMC (Sequential Monte Carlo) is a class of Monte Carlo algorithms for filtering and related sequential problems. Gerber and Chopin (2015) introduced SQMC (Sequential quasi-Monte Carlo), a QMC version of SMC. This paper has two objectives: (a) to introduce Sequential Monte Carlo to the QMC community, whose members are usually less familiar with state-space models and particle filtering; (b) to extend SQMC to the filtering of continuous-time state-space models, where the latent process is a diffusion. A recurring point in the paper will be the notion of dimension reduction, that is how to implement SQMC in such a way that it provides good performance despite the high dimension of the problem. △ Less

Submitted 16 June, 2017; originally announced June 2017.

Comments: To be published in the proceedings of MCMQMC 2016

MSC Class: 65C05

Modelling dependency completion in sentence comprehension as a Bayesian hierarchical mixture process: A case study involving Chinese relative clauses

Authors: Shravan Vasishth, Nicolas Chopin, Robin Ryder, Bruno Nicenboim

Abstract: We present a case-study demonstrating the usefulness of Bayesian hierarchical mixture modelling for investigating cognitive processes. In sentence comprehension, it is widely assumed that the distance between linguistic co-dependents affects the latency of dependency resolution: the longer the distance, the longer the retrieval time (the distance-based account). An alternative theory, direct-acces… ▽ More We present a case-study demonstrating the usefulness of Bayesian hierarchical mixture modelling for investigating cognitive processes. In sentence comprehension, it is widely assumed that the distance between linguistic co-dependents affects the latency of dependency resolution: the longer the distance, the longer the retrieval time (the distance-based account). An alternative theory, direct-access, assumes that retrieval times are a mixture of two distributions: one distribution represents successful retrievals (these are independent of dependency distance) and the other represents an initial failure to retrieve the correct dependent, followed by a reanalysis that leads to successful retrieval. We implement both models as Bayesian hierarchical models and show that the direct-access model explains Chinese relative clause reading time data better than the distance account. △ Less

Submitted 5 May, 2017; v1 submitted 2 February, 2017; originally announced February 2017.

Comments: 6 pages, 2 figures. To appear in the Proceedings of the Cognitive Science Conference 2017, London, UK

Divide and conquer in ABC: Expectation-Progagation algorithms for likelihood-free inference

Authors: Simon Barthelmé, Nicolas Chopin, Vincent Cottet

Abstract: ABC algorithms are notoriously expensive in computing time, as they require simulating many complete artificial datasets from the model. We advocate in this paper a "divide and conquer" approach to ABC, where we split the likelihood into n factors, and combine in some way n "local" ABC approximations of each factor. This has two advantages: (a) such an approach is typically much faster than standa… ▽ More ABC algorithms are notoriously expensive in computing time, as they require simulating many complete artificial datasets from the model. We advocate in this paper a "divide and conquer" approach to ABC, where we split the likelihood into n factors, and combine in some way n "local" ABC approximations of each factor. This has two advantages: (a) such an approach is typically much faster than standard ABC and (b) it makes it possible to use local summary statistics (i.e. summary statistics that depend only on the data-points that correspond to a single factor), rather than global summary statistics (that depend on the complete dataset). This greatly alleviates the bias introduced by summary statistics, and even removes it entirely in situations where local summary statistics are simply the identity function. We focus on EP (Expectation-Propagation), a convenient and powerful way to combine n local approximations into a global approximation. Compared to the EP- ABC approach of Barthelmé and Chopin (2014), we present two variations, one based on the parallel EP algorithm of Cseke and Heskes (2011), which has the advantage of being implementable on a parallel architecture, and one version which bridges the gap between standard EP and parallel EP. We illustrate our approach with an expensive application of ABC, namely inference on spatial extremes. △ Less

Submitted 1 December, 2015; originally announced December 2015.

Comments: To appear in the forthcoming Handbook of Approximate Bayesian Computation (ABC), edited by S. Sisson, L. Fan, and M. Beaumont

Leave Pima Indians alone: binary regression as a benchmark for Bayesian computation

Authors: Nicolas Chopin, James Ridgway

Abstract: Abstract. Whenever a new approach to perform Bayesian computation is introduced, a common practice is to showcase this approach on a binary regression model and datasets of moderate size. This paper discusses to which extent this practice is sound. It also reviews the current state of the art of Bayesian computation, using binary regression as a running example. Both sampling-based algorithms (imp… ▽ More Abstract. Whenever a new approach to perform Bayesian computation is introduced, a common practice is to showcase this approach on a binary regression model and datasets of moderate size. This paper discusses to which extent this practice is sound. It also reviews the current state of the art of Bayesian computation, using binary regression as a running example. Both sampling-based algorithms (importance sampling, MCMC and SMC) and fast approximations (Laplace and EP) are covered. Extensive numerical results are provided, some of which might go against conventional wisdom regarding the effectiveness of certain algorithms. Implications for other problems (variable selection) and other models are also discussed. △ Less

Submitted 29 June, 2015; originally announced June 2015.

Convergence of Sequential Quasi-Monte Carlo Smoothing Algorithms

Authors: Mathieu Gerber, Nicolas Chopin

Abstract: Gerber and Chopin (2015) recently introduced Sequential quasi-Monte Carlo (SQMC) algorithms as an efficient way to perform filtering in state-space models. The basic idea is to replace random variables with low-discrepancy point sets, so as to obtain faster convergence than with standard particle filtering. Gerber and Chopin (2015) describe briefly several ways to extend SQMC to smoothing, but do… ▽ More Gerber and Chopin (2015) recently introduced Sequential quasi-Monte Carlo (SQMC) algorithms as an efficient way to perform filtering in state-space models. The basic idea is to replace random variables with low-discrepancy point sets, so as to obtain faster convergence than with standard particle filtering. Gerber and Chopin (2015) describe briefly several ways to extend SQMC to smoothing, but do not provide supporting theory for this extension. We discuss more thoroughly how smoothing may be performed within SQMC, and derive convergence results for the so-obtained smoothing algorithms. We consider in particular SQMC equivalents of forward smoothing and forward filtering backward sampling, which are the most well-known smoothing techniques. As a preliminary step, we provide a generalization of the classical result of Hlawka and Mück (1972) on the transformation of QMC point sets into low discrepancy point sets with respect to non uniform distributions. As a corollary of the latter, we note that we can slightly weaken the assumptions to prove the consistency of SQMC. △ Less

Submitted 19 June, 2015; originally announced June 2015.

Comments: 37 pages, 3 figures

On the properties of variational approximations of Gibbs posteriors

Authors: Pierre Alquier, James Ridgway, Nicolas Chopin

Abstract: The PAC-Bayesian approach is a powerful set of techniques to derive non- asymptotic risk bounds for random estimators. The corresponding optimal distribution of estimators, usually called the Gibbs posterior, is unfortunately intractable. One may sample from it using Markov chain Monte Carlo, but this is often too slow for big datasets. We consider instead variational approximations of the Gibbs p… ▽ More The PAC-Bayesian approach is a powerful set of techniques to derive non- asymptotic risk bounds for random estimators. The corresponding optimal distribution of estimators, usually called the Gibbs posterior, is unfortunately intractable. One may sample from it using Markov chain Monte Carlo, but this is often too slow for big datasets. We consider instead variational approximations of the Gibbs posterior, which are fast to compute. We undertake a general study of the properties of such approximations. Our main finding is that such a variational approximation has often the same rate of convergence as the original PAC-Bayesian procedure it approximates. We specialise our results to several learning tasks (classification, ranking, matrix completion),discuss how to implement a variational approximation in each case, and illustrate the good properties of said approximation on real datasets. △ Less

Submitted 15 June, 2015; v1 submitted 12 June, 2015; originally announced June 2015.

Towards automatic calibration of the number of state particles within the SMC$^2$ algorithm

Authors: Nicolas Chopin, James Ridgway, Mathieu Gerber, Omiros Papaspiliopoulos

Abstract: SMC$^2$ is an efficient algorithm for sequential estimation and state inference of state-space models. It generates $N_θ$ parameter particles $θ^{m}$, and, for each $θ^{m}$, it runs a particle filter of size $N_{x}$ (i.e. at each time step, $N_{x}$ particles are generated in the state space $\mathcal{X}$). We discuss how to automatically calibrate $N_{x}$ in the course of the algorithm. Our approa… ▽ More SMC$^2$ is an efficient algorithm for sequential estimation and state inference of state-space models. It generates $N_θ$ parameter particles $θ^{m}$, and, for each $θ^{m}$, it runs a particle filter of size $N_{x}$ (i.e. at each time step, $N_{x}$ particles are generated in the state space $\mathcal{X}$). We discuss how to automatically calibrate $N_{x}$ in the course of the algorithm. Our approach relies on conditional Sequential Monte Carlo updates, monitoring the state of the pseudo random number generator and on an estimator of the variance of the unbiased estimate of the likelihood that is produced by the particle filters, which is obtained using nonparametric regression techniques. We observe that our approach is both less CPU intensive and with smaller Monte Carlo errors than the initial version of SMC$^2$. △ Less

Submitted 1 June, 2015; originally announced June 2015.

MSC Class: 62-06

Application of Sequential Quasi-Monte Carlo to Autonomous Positioning

Authors: Nicolas Chopin, Mathieu Gerber

Abstract: Sequential Monte Carlo algorithms (also known as particle filters) are popular methods to approximate filtering (and related) distributions of state-space models. However, they converge at the slow $1/\sqrt{N}$ rate, which may be an issue in real-time data-intensive scenarios. We give a brief outline of SQMC (Sequential Quasi-Monte Carlo), a variant of SMC based on low-discrepancy point sets propo… ▽ More Sequential Monte Carlo algorithms (also known as particle filters) are popular methods to approximate filtering (and related) distributions of state-space models. However, they converge at the slow $1/\sqrt{N}$ rate, which may be an issue in real-time data-intensive scenarios. We give a brief outline of SQMC (Sequential Quasi-Monte Carlo), a variant of SMC based on low-discrepancy point sets proposed by Gerber and Chopin (2015), which converges at a faster rate, and we illustrate the greater performance of SQMC on autonomous positioning problems. △ Less

Submitted 5 March, 2015; originally announced March 2015.

Comments: 5 pages, 4 figures

On Particle Methods for Parameter Estimation in State-Space Models

Authors: Nikolas Kantas, Arnaud Doucet, Sumeetpal S. Singh, Jan Maciejowski, Nicolas Chopin

Abstract: Nonlinear non-Gaussian state-space models are ubiquitous in statistics, econometrics, information engineering and signal processing. Particle methods, also known as Sequential Monte Carlo (SMC) methods, provide reliable numerical approximations to the associated state inference problems. However, in most applications, the state-space model of interest also depends on unknown static parameters that… ▽ More Nonlinear non-Gaussian state-space models are ubiquitous in statistics, econometrics, information engineering and signal processing. Particle methods, also known as Sequential Monte Carlo (SMC) methods, provide reliable numerical approximations to the associated state inference problems. However, in most applications, the state-space model of interest also depends on unknown static parameters that need to be estimated from the data. In this context, standard particle methods fail and it is necessary to rely on more sophisticated algorithms. The aim of this paper is to present a comprehensive review of particle methods that have been proposed to perform static parameter estimation in state-space models. We discuss the advantages and limitations of these methods and illustrate their performance on simple models. △ Less

Submitted 10 September, 2015; v1 submitted 30 December, 2014; originally announced December 2014.

Comments: Published at http://dx.doi.org/10.1214/14-STS511 in the Statistical Science (http://www.imstat.org/sts/) by the Institute of Mathematical Statistics (http://www.imstat.org)

Report number: IMS-STS-STS511

Journal ref: Statistical Science 2015, Vol. 30, No. 3, 328-351

Control functionals for Monte Carlo integration

Authors: Chris J. Oates, Mark Girolami, Nicolas Chopin

Abstract: A non-parametric extension of control variates is presented. These leverage gradient information on the sampling density to achieve substantial variance reduction. It is not required that the sampling density be normalised. The novel contribution of this work is based on two important insights; (i) a trade-off between random sampling and deterministic approximation and (ii) a new gradient-based fu… ▽ More A non-parametric extension of control variates is presented. These leverage gradient information on the sampling density to achieve substantial variance reduction. It is not required that the sampling density be normalised. The novel contribution of this work is based on two important insights; (i) a trade-off between random sampling and deterministic approximation and (ii) a new gradient-based function space derived from Stein's identity. Unlike classical control variates, our estimators achieve super-root-$n$ convergence, often requiring orders of magnitude fewer simulations to achieve a fixed level of precision. Theoretical and empirical results are presented, the latter focusing on integration problems arising in hierarchical models and models based on non-linear ordinary differential equations. △ Less

Submitted 1 April, 2016; v1 submitted 9 October, 2014; originally announced October 2014.

Comments: Accepted for publication in J. R. Statist. Soc. B

PAC-Bayesian AUC classification and scoring

Authors: James Ridgway, Pierre Alquier, Nicolas Chopin, Feng Liang

Abstract: We develop a scoring and classification procedure based on the PAC-Bayesian approach and the AUC (Area Under Curve) criterion. We focus initially on the class of linear score functions. We derive PAC-Bayesian non-asymptotic bounds for two types of prior for the score parameters: a Gaussian prior, and a spike-and-slab prior; the latter makes it possible to perform feature selection. One important a… ▽ More We develop a scoring and classification procedure based on the PAC-Bayesian approach and the AUC (Area Under Curve) criterion. We focus initially on the class of linear score functions. We derive PAC-Bayesian non-asymptotic bounds for two types of prior for the score parameters: a Gaussian prior, and a spike-and-slab prior; the latter makes it possible to perform feature selection. One important advantage of our approach is that it is amenable to powerful Bayesian computational tools. We derive in particular a Sequential Monte Carlo algorithm, as an efficient method which may be used as a gold standard, and an Expectation-Propagation algorithm, as a much faster but approximate method. We also extend our method to a class of non-linear score functions, essentially leading to a nonparametric procedure, by considering a Gaussian process prior. △ Less

Submitted 13 October, 2014; v1 submitted 7 October, 2014; originally announced October 2014.

Comments: Accepted at NIPS 2014

MSC Class: 62H30

The Poisson transform for unnormalised statistical models

Authors: Simon Barthelmé, Nicolas Chopin

Abstract: Contrary to standard statistical models, unnormalised statistical models only specify the likelihood function up to a constant. While such models are natural and popular, the lack of normalisation makes inference much more difficult. Here we show that inferring the parameters of a unnormalised model on a space $Ω$ can be mapped onto an equivalent problem of estimating the intensity of a Poisson po… ▽ More Contrary to standard statistical models, unnormalised statistical models only specify the likelihood function up to a constant. While such models are natural and popular, the lack of normalisation makes inference much more difficult. Here we show that inferring the parameters of a unnormalised model on a space $Ω$ can be mapped onto an equivalent problem of estimating the intensity of a Poisson point process on $Ω$. The unnormalised statistical model now specifies an intensity function that does not need to be normalised. Effectively, the normalisation constant may now be inferred as just another parameter, at no loss of information. The result can be extended to cover non-IID models, which includes for example unnormalised models for sequences of graphs (dynamical graphs), or for sequences of binary vectors. As a consequence, we prove that unnormalised parameteric inference in non-IID models can be turned into a semi-parametric estimation problem. Moreover, we show that the noise-contrastive divergence of Gutmann & Hyvärinen (2012) can be understood as an approximation of the Poisson transform, and extended to non-IID settings. We use our results to fit spatial Markov chain models of eye movements, where the Poisson transform allows us to turn a highly non-standard model into vanilla semi-parametric logistic regression. △ Less

Submitted 27 November, 2014; v1 submitted 11 June, 2014; originally announced June 2014.

MSC Class: 62F99 ACM Class: G.3

Bayesian matrix completion: prior specification

Authors: Pierre Alquier, Vincent Cottet, Nicolas Chopin, Judith Rousseau

Abstract: Low-rank matrix estimation from incomplete measurements recently received increased attention due to the emergence of several challenging applications, such as recommender systems; see in particular the famous Netflix challenge. While the behaviour of algorithms based on nuclear norm minimization is now well understood, an as yet unexplored avenue of research is the behaviour of Bayesian algorithm… ▽ More Low-rank matrix estimation from incomplete measurements recently received increased attention due to the emergence of several challenging applications, such as recommender systems; see in particular the famous Netflix challenge. While the behaviour of algorithms based on nuclear norm minimization is now well understood, an as yet unexplored avenue of research is the behaviour of Bayesian algorithms in this context. In this paper, we briefly review the priors used in the Bayesian literature for matrix completion. A standard approach is to assign an inverse gamma prior to the singular values of a certain singular value decomposition of the matrix of interest; this prior is conjugate. However, we show that two other types of priors (again for the singular values) may be conjugate for this model: a gamma prior, and a discrete prior. Conjugacy is very convenient, as it makes it possible to implement either Gibbs sampling or Variational Bayes. Interestingly enough, the maximum a posteriori for these different priors is related to the nuclear norm minimization problems. We also compare all these priors on simulated datasets, and on the classical MovieLens and Netflix datasets. △ Less

Submitted 22 October, 2014; v1 submitted 5 June, 2014; originally announced June 2014.

Sequential Quasi-Monte Carlo

Authors: Mathieu Gerber, Nicolas Chopin

Abstract: We derive and study SQMC (Sequential Quasi-Monte Carlo), a class of algorithms obtained by introducing QMC point sets in particle filtering. SQMC is related to, and may be seen as an extension of, the array-RQMC algorithm of L'Ecuyer et al. (2006). The complexity of SQMC is $O(N \log N)$, where $N$ is the number of simulations at each iteration, and its error rate is smaller than the Monte Carlo r… ▽ More We derive and study SQMC (Sequential Quasi-Monte Carlo), a class of algorithms obtained by introducing QMC point sets in particle filtering. SQMC is related to, and may be seen as an extension of, the array-RQMC algorithm of L'Ecuyer et al. (2006). The complexity of SQMC is $O(N \log N)$, where $N$ is the number of simulations at each iteration, and its error rate is smaller than the Monte Carlo rate $O_P(N^{-1/2})$. The only requirement to implement SQMC is the ability to write the simulation of particle $x_t^n$ given $x_{t-1}^n$ as a deterministic function of $x_{t-1}^n$ and a fixed number of uniform variates. We show that SQMC is amenable to the same extensions as standard SMC, such as forward smoothing, backward smoothing, unbiased likelihood evaluation, and so on. In particular, SQMC may replace SMC within a PMCMC (particle Markov chain Monte Carlo) algorithm. We establish several convergence results. We provide numerical evidence that SQMC may significantly outperform SMC in practical scenarios. △ Less

Submitted 28 November, 2014; v1 submitted 17 February, 2014; originally announced February 2014.

Comments: 55 pages, 10 figures (final version)

On particle Gibbs sampling

Authors: Nicolas Chopin, Sumeetpal S. Singh

Abstract: The particle Gibbs sampler is a Markov chain Monte Carlo (MCMC) algorithm to sample from the full posterior distribution of a state-space model. It does so by executing Gibbs sampling steps on an extended target distribution defined on the space of the auxiliary variables generated by an interacting particle system. This paper makes the following contributions to the theoretical study of this algo… ▽ More The particle Gibbs sampler is a Markov chain Monte Carlo (MCMC) algorithm to sample from the full posterior distribution of a state-space model. It does so by executing Gibbs sampling steps on an extended target distribution defined on the space of the auxiliary variables generated by an interacting particle system. This paper makes the following contributions to the theoretical study of this algorithm. Firstly, we present a coupling construction between two particle Gibbs updates from different starting points and we show that the coupling probability may be made arbitrarily close to one by increasing the number of particles. We obtain as a direct corollary that the particle Gibbs kernel is uniformly ergodic. Secondly, we show how the inclusion of an additional Gibbs sampling step that reselects the ancestors of the particle Gibbs' extended target distribution, which is a popular approach in practice to improve mixing, does indeed yield a theoretically more efficient algorithm as measured by the asymptotic variance. Thirdly, we extend particle Gibbs to work with lower variance resampling schemes. A detailed numerical study is provided to demonstrate the efficiency of particle Gibbs and the proposed variants. △ Less

Submitted 28 July, 2015; v1 submitted 6 April, 2013; originally announced April 2013.

Comments: Published at http://dx.doi.org/10.3150/14-BEJ629 in the Bernoulli (http://isi.cbs.nl/bernoulli/) by the International Statistical Institute/Bernoulli Society (http://isi.cbs.nl/BS/bshome.htm)

Report number: IMS-BEJ-BEJ629

Journal ref: Bernoulli 2015, Vol. 21, No. 3, 1855-1883

Bayesian learning of noisy Markov decision processes

Authors: Sumeetpal S. Singh, Nicolas Chopin, Nick Whiteley

Abstract: We consider the inverse reinforcement learning problem, that is, the problem of learning from, and then predicting or mimicking a controller based on state/action data. We propose a statistical model for such data, derived from the structure of a Markov decision process. Adopting a Bayesian approach to inference, we show how latent variables of the model can be estimated, and how predictions about… ▽ More We consider the inverse reinforcement learning problem, that is, the problem of learning from, and then predicting or mimicking a controller based on state/action data. We propose a statistical model for such data, derived from the structure of a Markov decision process. Adopting a Bayesian approach to inference, we show how latent variables of the model can be estimated, and how predictions about actions can be made, in a unified framework. A new Markov chain Monte Carlo (MCMC) sampler is devised for simulation from the posterior distribution. This step includes a parameter expansion step, which is shown to be essential for good convergence properties of the MCMC sampler. As an illustration, the method is applied to learning a human controller. △ Less

Submitted 26 November, 2012; originally announced November 2012.

Computational aspects of Bayesian spectral density estimation

Authors: Nicolas Chopin, Judith Rousseau, Brunero Liseo

Abstract: Gaussian time-series models are often specified through their spectral density. Such models present several computational challenges, in particular because of the non-sparse nature of the covariance matrix. We derive a fast approximation of the likelihood for such models. We propose to sample from the approximate posterior (that is, the prior times the approximate likelihood), and then to recover… ▽ More Gaussian time-series models are often specified through their spectral density. Such models present several computational challenges, in particular because of the non-sparse nature of the covariance matrix. We derive a fast approximation of the likelihood for such models. We propose to sample from the approximate posterior (that is, the prior times the approximate likelihood), and then to recover the exact posterior through importance sampling. We show that the variance of the importance sampling weights vanishes as the sample size goes to infinity. We explain why the approximate posterior may typically multi-modal, and we derive a Sequential Monte Carlo sampler based on an annealing sequence in order to sample from that target distribution. Performance of the overall approach is evaluated on simulated and real datasets. In addition, for one real world dataset, we provide some numerical evidence that a Bayesian approach to semi-parametric estimation of spectral density may provide more reasonable results than its Frequentist counter-parts. △ Less

Submitted 19 November, 2012; originally announced November 2012.

MSC Class: 62F15; 65C05

In praise of the referee

Authors: Nicolas Chopin, Andrew Gelman, Kerrie L. Mengersen, Christian P. Robert

Abstract: There has been a lively debate in many fields, including statistics and related applied fields such as psychology and biomedical research, on possible reforms of the scholarly publishing system. Currently, referees contribute so much to improve scientific papers, both directly through constructive criticism and indirectly through the threat of rejection. We discuss ways in which new approaches to… ▽ More There has been a lively debate in many fields, including statistics and related applied fields such as psychology and biomedical research, on possible reforms of the scholarly publishing system. Currently, referees contribute so much to improve scientific papers, both directly through constructive criticism and indirectly through the threat of rejection. We discuss ways in which new approaches to journal publication could continue to make use of the valuable efforts of peer reviewers. △ Less

Submitted 19 May, 2012; originally announced May 2012.

Comments: 13 pages

Fast simulation of truncated Gaussian distributions

Authors: Nicolas Chopin

Abstract: We consider the problem of simulating a Gaussian vector X, conditional on the fact that each component of X belongs to a finite interval [a_i,b_i], or a semi-finite interval [a_i,+infty). In the one-dimensional case, we design a table-based algorithm that is computationally faster than alternative algorithms. In the two-dimensional case, we design an accept-reject algorithm. According to our calcu… ▽ More We consider the problem of simulating a Gaussian vector X, conditional on the fact that each component of X belongs to a finite interval [a_i,b_i], or a semi-finite interval [a_i,+infty). In the one-dimensional case, we design a table-based algorithm that is computationally faster than alternative algorithms. In the two-dimensional case, we design an accept-reject algorithm. According to our calculations and our numerical studies, the acceptance rate of this algorithm is bounded from below by 0.5 for semi-finite truncation intervals, and by 0.47 for finite intervals. Extension to 3 or more dimensions is discussed. △ Less

Submitted 30 January, 2012; originally announced January 2012.

MSC Class: 65C10 ACM Class: G.3

Journal ref: Statistics and Computing 2011, Volume 21, Number 2, 275-288

Some discussions of D. Fearnhead and D. Prangle's Read Paper "Constructing summary statistics for approximate Bayesian computation: semi-automatic approximate Bayesian computation"

Authors: Christophe Andrieu, Simon Barthelme, Nicolas Chopin, Julien Cornebise, Arnaud Doucet, Mark Girolami, Ioannis Kosmidis, Ajay Jasra, Anthony Lee, Jean-Michel Marin, Pierre Pudlo, Christian P. Robert, Mohammed Sedki., Sumeetpal S. Singh

Abstract: This report is a collection of comments on the Read Paper of Fearnhead and Prangle (2011), to appear in the Journal of the Royal Statistical Society Series B, along with a reply from the authors. This report is a collection of comments on the Read Paper of Fearnhead and Prangle (2011), to appear in the Journal of the Royal Statistical Society Series B, along with a reply from the authors. △ Less

Submitted 5 January, 2012; originally announced January 2012.

Comments: 10 pages

Expectation-Propagation for Likelihood-Free Inference

Authors: Simon Barthelmé, Nicolas Chopin

Abstract: Many models of interest in the natural and social sciences have no closed-form likelihood function, which means that they cannot be treated using the usual techniques of statistical inference. In the case where such models can be efficiently simulated, Bayesian inference is still possible thanks to the Approximate Bayesian Computation (ABC) algorithm. Although many refinements have been suggested,… ▽ More Many models of interest in the natural and social sciences have no closed-form likelihood function, which means that they cannot be treated using the usual techniques of statistical inference. In the case where such models can be efficiently simulated, Bayesian inference is still possible thanks to the Approximate Bayesian Computation (ABC) algorithm. Although many refinements have been suggested, ABC inference is still far from routine. ABC is often excruciatingly slow due to very low acceptance rates. In addition, ABC requires introducing a vector of "summary statistics", the choice of which is relatively arbitrary, and often require some trial and error, making the whole process quite laborious for the user. We introduce in this work the EP-ABC algorithm, which is an adaptation to the likelihood-free context of the variational approximation algorithm known as Expectation Propagation (Minka, 2001). The main advantage of EP-ABC is that it is faster by a few orders of magnitude than standard algorithms, while producing an overall approximation error which is typically negligible. A second advantage of EP-ABC is that it replaces the usual global ABC constraint on the vector of summary statistics computed on the whole dataset, by n local constraints of the form that apply separately to each data-point. As a consequence, it is often possible to do away with summary statistics entirely. In that case, EP-ABC approximates directly the evidence (marginal likelihood) of the model. Comparisons are performed in three real-world applications which are typical of likelihood-free inference, including one application in neuroscience which is novel, and possibly too challenging for standard ABC techniques. △ Less

Submitted 18 July, 2012; v1 submitted 29 July, 2011; originally announced July 2011.

Comments: Revised version following peer-review

Sequential Monte Carlo on large binary sampling spaces

Authors: Christian Schäfer, Nicolas Chopin

Abstract: A Monte Carlo algorithm is said to be adaptive if it automatically calibrates its current proposal distribution using past simulations. The choice of the parametric family that defines the set of proposal distributions is critical for good performance. In this paper, we present such a parametric family for adaptive sampling on high-dimensional binary spaces. A practical motivation for this problem… ▽ More A Monte Carlo algorithm is said to be adaptive if it automatically calibrates its current proposal distribution using past simulations. The choice of the parametric family that defines the set of proposal distributions is critical for good performance. In this paper, we present such a parametric family for adaptive sampling on high-dimensional binary spaces. A practical motivation for this problem is variable selection in a linear regression context. We want to sample from a Bayesian posterior distribution on the model space using an appropriate version of Sequential Monte Carlo. Raw versions of Sequential Monte Carlo are easily implemented using binary vectors with independent components. For high-dimensional problems, however, these simple proposals do not yield satisfactory results. The key to an efficient adaptive algorithm are binary parametric families which take correlations into account, analogously to the multivariate normal distribution on continuous spaces. We provide a review of models for binary data and make one of them work in the context of Sequential Monte Carlo sampling. Computational studies on real life data with about a hundred covariates suggest that, on difficult instances, our Sequential Monte Carlo approach clearly outperforms standard techniques based on Markov chain exploration. △ Less

Submitted 9 November, 2011; v1 submitted 31 January, 2011; originally announced January 2011.

SMC^2: an efficient algorithm for sequential analysis of state-space models

Authors: Nicolas Chopin, Pierre E. Jacob, Omiros Papaspiliopoulos

Abstract: We consider the generic problem of performing sequential Bayesian inference in a state-space model with observation process y, state process x and fixed parameter theta. An idealized approach would be to apply the iterated batch importance sampling (IBIS) algorithm of Chopin (2002). This is a sequential Monte Carlo algorithm in the theta-dimension, that samples values of theta, reweights iterative… ▽ More We consider the generic problem of performing sequential Bayesian inference in a state-space model with observation process y, state process x and fixed parameter theta. An idealized approach would be to apply the iterated batch importance sampling (IBIS) algorithm of Chopin (2002). This is a sequential Monte Carlo algorithm in the theta-dimension, that samples values of theta, reweights iteratively these values using the likelihood increments p(y_t|y_1:t-1, theta), and rejuvenates the theta-particles through a resampling step and a MCMC update step. In state-space models these likelihood increments are intractable in most cases, but they may be unbiasedly estimated by a particle filter in the x-dimension, for any fixed theta. This motivates the SMC^2 algorithm proposed in this article: a sequential Monte Carlo algorithm, defined in the theta-dimension, which propagates and resamples many particle filters in the x-dimension. The filters in the x-dimension are an example of the random weight particle filter as in Fearnhead et al. (2010). On the other hand, the particle Markov chain Monte Carlo (PMCMC) framework developed in Andrieu et al. (2010) allows us to design appropriate MCMC rejuvenation steps. Thus, the theta-particles target the correct posterior distribution at each iteration t, despite the intractability of the likelihood increments. We explore the applicability of our algorithm in both sequential and non-sequential applications and consider various degrees of freedom, as for example increasing dynamically the number of x-particles. We contrast our approach to various competing methods, both conceptually and empirically through a detailed simulation study, included here and in a supplement, and based on particularly challenging examples. △ Less

Submitted 27 January, 2012; v1 submitted 7 January, 2011; originally announced January 2011.

Comments: 27 pages, 4 figures; supplementary material available on the second author's web page

MSC Class: 62F15; 65C05

Discussions on "Riemann manifold Langevin and Hamiltonian Monte Carlo methods"

Authors: Simon Barthelme, Magali Beffy, Nicolas Chopin, Arnaud Doucet, Pierre Jacob, Adam M. Johansen, Jean-Michel Marin, Christian P. Robert

Abstract: This is a collection of discussions of `Riemann manifold Langevin and Hamiltonian Monte Carlo methods" by Girolami and Calderhead, to appear in the Journal of the Royal Statistical Society, Series B. This is a collection of discussions of `Riemann manifold Langevin and Hamiltonian Monte Carlo methods" by Girolami and Calderhead, to appear in the Journal of the Royal Statistical Society, Series B. △ Less

Submitted 3 November, 2010; originally announced November 2010.

Comments: 6 pages, one figure

Bayesian nonparametric estimation of the spectral density of a long or intermediate memory Gaussian process

Authors: Judith Rousseau, Nicolas Chopin, Brunero Liseo

Abstract: A stationary Gaussian process is said to be long-range dependent (resp., anti-persistent) if its spectral density $f(λ)$ can be written as $f(λ)=|λ|^{-2d}g(|λ|)$, where $0<d<1/2$ (resp., $-1/2<d<0$), and $g$ is continuous and positive. We propose a novel Bayesian nonparametric approach for the estimation of the spectral density of such processes. We prove posterior consistency for both $d$ and… ▽ More A stationary Gaussian process is said to be long-range dependent (resp., anti-persistent) if its spectral density $f(λ)$ can be written as $f(λ)=|λ|^{-2d}g(|λ|)$, where $0<d<1/2$ (resp., $-1/2<d<0$), and $g$ is continuous and positive. We propose a novel Bayesian nonparametric approach for the estimation of the spectral density of such processes. We prove posterior consistency for both $d$ and $g$, under appropriate conditions on the prior distribution. We establish the rate of convergence for a general class of priors and apply our results to the family of fractionally exponential priors. Our approach is based on the true likelihood and does not resort to Whittle's approximation. △ Less

Submitted 23 July, 2012; v1 submitted 22 July, 2010; originally announced July 2010.

Comments: Published in at http://dx.doi.org/10.1214/11-AOS955 the Annals of Statistics (http://www.imstat.org/aos/) by the Institute of Mathematical Statistics (http://www.imstat.org)

Report number: IMS-AOS-AOS955

Journal ref: Annals of Statistics 2012, Vol. 40, No. 2, 964-995

Free energy Sequential Monte Carlo, application to mixture modelling

Authors: Nicolas Chopin, Pierre Jacob

Abstract: We introduce a new class of Sequential Monte Carlo (SMC) methods, which we call free energy SMC. This class is inspired by free energy methods, which originate from Physics, and where one samples from a biased distribution such that a given function $ξ(θ)$ of the state $θ$ is forced to be uniformly distributed over a given interval. From an initial sequence of distributions $(π_t)$ of interest, an… ▽ More We introduce a new class of Sequential Monte Carlo (SMC) methods, which we call free energy SMC. This class is inspired by free energy methods, which originate from Physics, and where one samples from a biased distribution such that a given function $ξ(θ)$ of the state $θ$ is forced to be uniformly distributed over a given interval. From an initial sequence of distributions $(π_t)$ of interest, and a particular choice of $ξ(θ)$, a free energy SMC sampler computes sequentially a sequence of biased distributions $(\tildeπ_{t})$ with the following properties: (a) the marginal distribution of $ξ(θ)$ with respect to $\tildeπ_{t}$ is approximatively uniform over a specified interval, and (b) $\tildeπ_{t}$ and $π_{t}$ have the same conditional distribution with respect to $ξ$. We apply our methodology to mixture posterior distributions, which are highly multimodal. In the mixture context, forcing certain hyper-parameters to higher values greatly faciliates mode swapping, and makes it possible to recover a symetric output. We illustrate our approach with univariate and bivariate Gaussian mixtures and two real-world datasets. △ Less

Submitted 15 June, 2010; originally announced June 2010.

Comments: presented at "Bayesian Statistics 9" (Valencia meetings, 4-8 June 2010, Benidorm)

MSC Class: 62F15

On Particle Learning

Authors: Nicolas Chopin, Alessandra Iacobucci, Jean-Michel Marin, Kerrie Mengersen, Christian P. Robert, Robin Ryder, Christian Schäfer

Abstract: This document is the aggregation of six discussions of Lopes et al. (2010) that we submitted to the proceedings of the Ninth Valencia Meeting, held in Benidorm, Spain, on June 3-8, 2010, in conjunction with Hedibert Lopes' talk at this meeting, and of a further discussion of the rejoinder by Lopes et al. (2010). The main point in those discussions is the potential for degeneracy in the particle le… ▽ More This document is the aggregation of six discussions of Lopes et al. (2010) that we submitted to the proceedings of the Ninth Valencia Meeting, held in Benidorm, Spain, on June 3-8, 2010, in conjunction with Hedibert Lopes' talk at this meeting, and of a further discussion of the rejoinder by Lopes et al. (2010). The main point in those discussions is the potential for degeneracy in the particle learning methodology, related with the exponential forgetting of the past simulations. We illustrate in particular the resulting difficulties in the case of mixtures. △ Less

Submitted 19 November, 2010; v1 submitted 3 June, 2010; originally announced June 2010.

Comments: 14 pages, 9 figures, discussions on the invited paper of Lopes, Carvalho, Johannes, and Polson, for the Ninth Valencia International Meeting on Bayesian Statistics, held in Benidorm, Spain, on June 3-8, 2010. To appear in Bayesian Statistics 9, Oxford University Press (except for the final discussion)

Free Energy Methods for Bayesian Inference: Efficient Exploration of Univariate Gaussian Mixture Posteriors

Authors: Nicolas Chopin, Tony Lelievre, Gabriel Stoltz

Abstract: Because of their multimodality, mixture posterior distributions are difficult to sample with standard Markov chain Monte Carlo (MCMC) methods. We propose a strategy to enhance the sampling of MCMC in this context, using a biasing procedure which originates from computational Statistical Physics. The principle is first to choose a "reaction coordinate", that is, a "direction" in which the target di… ▽ More Because of their multimodality, mixture posterior distributions are difficult to sample with standard Markov chain Monte Carlo (MCMC) methods. We propose a strategy to enhance the sampling of MCMC in this context, using a biasing procedure which originates from computational Statistical Physics. The principle is first to choose a "reaction coordinate", that is, a "direction" in which the target distribution is multimodal. In a second step, the marginal log-density of the reaction coordinate with respect to the posterior distribution is estimated; minus this quantity is called "free energy" in the computational Statistical Physics literature. To this end, we use adaptive biasing Markov chain algorithms which adapt their targeted invariant distribution on the fly, in order to overcome sampling barriers along the chosen reaction coordinate. Finally, we perform an importance sampling step in order to remove the bias and recover the true posterior. The efficiency factor of the importance sampling step can easily be estimated \emph{a priori} once the bias is known, and appears to be rather large for the test cases we considered. A crucial point is the choice of the reaction coordinate. One standard choice (used for example in the classical Wang-Landau algorithm) is minus the log-posterior density. We discuss other choices. We show in particular that the hyper-parameter that determines the order of magnitude of the variance of each component is both a convenient and an efficient reaction coordinate. We also show how to adapt the method to compute the evidence (marginal likelihood) of a mixture model. We illustrate our approach by analyzing two real data sets. △ Less

Submitted 18 April, 2011; v1 submitted 1 March, 2010; originally announced March 2010.