-
Angular Combining of Forecasts of Probability Distributions
Authors:
James W. Taylor,
Xiaochun Meng
Abstract:
When multiple forecasts are available for a probability distribution, forecast combining enables a pragmatic synthesis of the available information to extract the wisdom of the crowd. A linear opinion pool has been widely used, whereby the combining is applied to the probability predictions of the distributional forecasts. However, it has been argued that this will tend to deliver overdispersed di…
▽ More
When multiple forecasts are available for a probability distribution, forecast combining enables a pragmatic synthesis of the available information to extract the wisdom of the crowd. A linear opinion pool has been widely used, whereby the combining is applied to the probability predictions of the distributional forecasts. However, it has been argued that this will tend to deliver overdispersed distributional forecasts, prompting the combination to be applied, instead, to the quantile predictions of the distributional forecasts. Results from different applications are mixed, leaving it as an empirical question whether to combine probabilities or quantiles. In this paper, we present an alternative approach. Looking at the distributional forecasts, combining the probability forecasts can be viewed as vertical combining, with quantile forecast combining seen as horizontal combining. Our alternative approach is to allow combining to take place on an angle between the extreme cases of vertical and horizontal combining. We term this angular combining. The angle is a parameter that can be optimized using a proper scoring rule. We show that, as with vertical and horizontal averaging, angular averaging results in a distribution with mean equal to the average of the means of the distributions that are being combined. We also show that angular averaging produces a distribution with lower variance than vertical averaging, and, under certain assumptions, greater variance than horizontal averaging. We provide empirical support for angular combining using weekly distributional forecasts of COVID-19 mortality at the national and state level in the U.S.
△ Less
Submitted 26 May, 2023;
originally announced May 2023.
-
A Comparison of Aggregation Methods for Probabilistic Forecasts of COVID-19 Mortality in the United States
Authors:
Kathryn S. Taylor,
James W. Taylor
Abstract:
The COVID-19 pandemic has placed forecasting models at the forefront of health policy making. Predictions of mortality and hospitalization help governments meet planning and resource allocation challenges. In this paper, we consider the weekly forecasting of the cumulative mortality due to COVID-19 at the national and state level in the U.S. Optimal decision-making requires a forecast of a probabi…
▽ More
The COVID-19 pandemic has placed forecasting models at the forefront of health policy making. Predictions of mortality and hospitalization help governments meet planning and resource allocation challenges. In this paper, we consider the weekly forecasting of the cumulative mortality due to COVID-19 at the national and state level in the U.S. Optimal decision-making requires a forecast of a probability distribution, rather than just a single point forecast. Interval forecasts are also important, as they can support decision making and provide situational awareness. We consider the case where probabilistic forecasts have been provided by multiple forecasting teams, and we aggregate the forecasts to extract the wisdom of the crowd. With only limited information available regarding the historical accuracy of the forecasting teams, we consider aggregation (i.e. combining) methods that do not rely on a record of past accuracy. In this empirical paper, we evaluate the accuracy of aggregation methods that have been previously proposed for interval forecasts and predictions of probability distributions. These include the use of the simple average, the median, and trimming methods, which enable robust estimation and allow the aggregate forecast to reduce the impact of a tendency for the forecasting teams to be under- or overconfident. We use data that has been made publicly available from the COVID-19 Forecast Hub. While the simple average performed well for the high mortality series, we obtained greater accuracy using the median and certain trimming methods for the low and medium mortality series. It will be interesting to see if this remains the case as the pandemic evolves.
△ Less
Submitted 20 August, 2020; v1 submitted 21 July, 2020;
originally announced July 2020.
-
Probabilistic Forecasting of Patient Waiting Times in an Emergency Department
Authors:
Siddharth Arora,
James W. Taylor,
Ho-Yin Mak
Abstract:
We study the estimation of the probability distribution of individual patient waiting times in an emergency department (ED). Our feature-rich modelling allows for dynamic updating and refinement of waiting time estimates as patient- and ED-specific information (e.g., patient condition, ED congestion levels) is revealed during the waiting process. Aspects relating to communicating forecast uncertai…
▽ More
We study the estimation of the probability distribution of individual patient waiting times in an emergency department (ED). Our feature-rich modelling allows for dynamic updating and refinement of waiting time estimates as patient- and ED-specific information (e.g., patient condition, ED congestion levels) is revealed during the waiting process. Aspects relating to communicating forecast uncertainty to patients, and implementing this methodology in practice, are also discussed.
△ Less
Submitted 30 May, 2020;
originally announced June 2020.
-
Scores for Multivariate Distributions and Level Sets
Authors:
Xiaochun Meng,
James W. Taylor,
Souhaib Ben Taieb,
Siran Li
Abstract:
Forecasts of multivariate probability distributions are required for a variety of applications. Scoring rules enable the evaluation of forecast accuracy, and comparison between forecasting methods. We propose a theoretical framework for scoring rules for multivariate distributions, which encompasses the existing quadratic score and multivariate continuous ranked probability score. We demonstrate h…
▽ More
Forecasts of multivariate probability distributions are required for a variety of applications. Scoring rules enable the evaluation of forecast accuracy, and comparison between forecasting methods. We propose a theoretical framework for scoring rules for multivariate distributions, which encompasses the existing quadratic score and multivariate continuous ranked probability score. We demonstrate how this framework can be used to generate new scoring rules. In some multivariate contexts, it is a forecast of a level set that is needed, such as a density level set for anomaly detection or the level set of the cumulative distribution as a measure of risk. This motivates consideration of scoring functions for such level sets. For univariate distributions, it is well-established that the continuous ranked probability score can be expressed as the integral over a quantile score. We show that, in a similar way, scoring rules for multivariate distributions can be decomposed to obtain scoring functions for level sets. Using this, we present scoring functions for different types of level set, including density level sets and level sets for cumulative distributions. To compute the scores, we propose a simple numerical algorithm. We perform a simulation study to support our proposals, and we use real data to illustrate usefulness for forecast combining and CoVaR estimation.
△ Less
Submitted 21 June, 2023; v1 submitted 21 February, 2020;
originally announced February 2020.
-
Rule-based Autoregressive Moving Average Models for Forecasting Load on Special Days: A Case Study for France
Authors:
Siddharth Arora,
James W. Taylor
Abstract:
This paper presents a case study on short-term load forecasting for France, with emphasis on special days, such as public holidays. We investigate the generalisability to French data of a recently proposed approach, which generates forecasts for normal and special days in a coherent and unified framework, by incorporating subjective judgment in univariate statistical models using a rule-based meth…
▽ More
This paper presents a case study on short-term load forecasting for France, with emphasis on special days, such as public holidays. We investigate the generalisability to French data of a recently proposed approach, which generates forecasts for normal and special days in a coherent and unified framework, by incorporating subjective judgment in univariate statistical models using a rule-based methodology. The intraday, intraweek, and intrayear seasonality in load are accommodated using a rule-based triple seasonal adaptation of a seasonal autoregressive moving average (SARMA) model. We find that, for application to French load, the method requires an important adaption. We also adapt a recently proposed SARMA model that accommodates special day effects on an hourly basis using indicator variables. Using a rule formulated specifically for the French load, we compare the SARMA models with a range of different benchmark methods based on an evaluation of their point and density forecast accuracy. As sophisticated benchmarks, we employ the rule-based triple seasonal adaptations of Holt-Winters-Taylor (HWT) exponential smoothing and artificial neural networks (ANNs). We use nine years of half-hourly French load data, and consider lead times ranging from one half-hour up to a day ahead. The rule-based SARMA approach generated the most accurate forecasts.
△ Less
Submitted 26 March, 2018;
originally announced March 2018.
-
Forecasting Electricity Smart Meter Data Using Conditional Kernel Density Estimation
Authors:
Siddharth Arora,
James W. Taylor
Abstract:
The recent advent of smart meters has led to large micro-level datasets. For the first time, the electricity consumption at individual sites is available on a near real-time basis. Efficient management of energy resources, electric utilities, and transmission grids, can be greatly facilitated by harnessing the potential of this data. The aim of this study is to generate probability density estimat…
▽ More
The recent advent of smart meters has led to large micro-level datasets. For the first time, the electricity consumption at individual sites is available on a near real-time basis. Efficient management of energy resources, electric utilities, and transmission grids, can be greatly facilitated by harnessing the potential of this data. The aim of this study is to generate probability density estimates for consumption recorded by individual smart meters. Such estimates can assist decision making by helping consumers identify and minimize their excess electricity usage, especially during peak times. For suppliers, these estimates can be used to devise innovative time-of-use pricing strategies aimed at their target consumers. We consider methods based on conditional kernel density (CKD) estimation with the incorporation of a decay parameter. The methods capture the seasonality in consumption, and enable a nonparametric estimation of its conditional density. Using eight months of half-hourly data for one thousand meters, we evaluate point and density forecasts, for lead times ranging from one half-hour up to a week ahead. We find that the kernel-based methods outperform a simple benchmark method that does not account for seasonality, and compare well with an exponential smoothing method that we use as a sophisticated benchmark. To gauge the financial impact, we use density estimates of consumption to derive prediction intervals of electricity cost for different time-of-use tariffs. We show that a simple strategy of switching between different tariffs, based on a comparison of cost densities, delivers significant cost savings for the great majority of consumers.
△ Less
Submitted 9 September, 2014;
originally announced September 2014.
-
Short-term Forecasting of Anomalous Load Using Rule-based Triple Seasonal Methods
Authors:
Siddharth Arora,
James W. Taylor
Abstract:
Numerous methods have been proposed for forecasting load for normal days. Modeling of anomalous load, however, has often been ignored in the research literature. Occurring on special days, such as public holidays, anomalous load conditions pose considerable modeling challenges due to their infrequent occurrence and significant deviation from normal load. To overcome these limitations, we adopt a r…
▽ More
Numerous methods have been proposed for forecasting load for normal days. Modeling of anomalous load, however, has often been ignored in the research literature. Occurring on special days, such as public holidays, anomalous load conditions pose considerable modeling challenges due to their infrequent occurrence and significant deviation from normal load. To overcome these limitations, we adopt a rule-based approach, which allows incorporation of prior expert knowledge of load profiles into the statistical model. We use triple seasonal Holt-Winters-Taylor (HWT) exponential smoothing, triple seasonal autoregressive moving average (ARMA), artificial neural networks (ANNs), and triple seasonal intraweek singular value decomposition (SVD) based exponential smoothing. These methods have been shown to be competitive for modeling load for normal days. The methodological contribution of this paper is to demonstrate how these methods can be adapted to model load for special days, when used in conjunction with a rule-based approach. The proposed rule-based method is able to model normal and anomalous load in a unified framework. Using nine years of half-hourly load for Great Britain, we evaluate point forecasts, for lead times from one half-hour up to a day ahead. A combination of two rule-based methods generated the most accurate forecasts.
△ Less
Submitted 6 September, 2014;
originally announced September 2014.
