Modeling and Analysis of Commercial Building Electrical Loads for Demand Side

Management

A Thesis

Submitted to the Faculty

of

Drexel University

by

Jonathan Berardino

in partial fulfillment of the

requirements for the degree

of

Doctor of Philosophy

December 2016
 © Copyright 2016
Jonathan Berardino. All Rights Reserved.
 iii
ACKNOWLEDGMENTS

First, I would like to thank my advisor Dr. Chika Nwankpa for your time,

guidance, and support in performing the research that led to this thesis. I am forever

grateful to you for accepting me as a student and providing me with the opportunity to

work in this lab.

I would like to thank Dr. Miu, Dr. Niebur, Dr. Benson, and Dr. Kwatny for

serving on my thesis committee. Your comments have greatly improved the final version

of this thesis and I appreciate the advice and insights you have provided that will enhance

this work going forward.

Thank you to my fellow members of the Center for Electric Power Engineering. I

have been very fortunate to work with a group of people who have been valuable

colleagues and (more importantly) friends. I wish you all the best and look forward to

when our paths cross in the future.

Lastly, and only because the last paragraph is always the most important, I would

like to thank my wife Lindsay. Pursuing this degree has taken up enormous amounts of

my time and nobody has felt that impact more than you. Thank you for your patience and

selflessness over the years. This would not have happened without you.
 iv
TABLE OF CONTENTS

LIST OF TABLES ........................................................................................................... viii

LIST OF FIGURES ............................................................................................................ x

ABSTRACT .................................................................................................................. xxxii

1. INTRODUCTION ....................................................................................................... 1

1.1. OVERVIEW ....................................................................................................... 1

1.2. BACKGROUND AND MOTIVATION............................................................ 1

1.3. OBJECTIVES .................................................................................................... 3

1.4. CONTRIBUTIONS ............................................................................................ 4

1.5. ORGANIZATION OF THESIS ......................................................................... 4

2. ANALYSIS OF COMMERCIAL BUILDING LOADS AS DEMAND SIDE

RESOURCES ..................................................................................................................... 6

2.1. OVERVIEW ....................................................................................................... 6

2.2. DEMAND SIDE MANAGEMENT PROGRAMS ............................................ 6

2.3. COMMERCIAL BUILDING ELECTRICITY USE AND BEHAVIOR .......... 9

2.4. REVIEW OF COMMERCIAL BUILDING DEMAND SIDE

MANAGEMENT RESEARCH................................................................................. 13

2.4.1. BUILDING-SPECIFIC LOAD FORECASTING................................ 13

2.4.2. COMMERCIAL BUILDING LOAD MODELING AND DISPATCH

16

3. BUILDING-SPECIFIC LOAD FORECASTING STUDIES ................................... 18

3.1. OVERVIEW ..................................................................................................... 18

3.2. LOAD FORECASTING METHOD REVIEW ................................................ 18

 v
3.3. LOAD FORECASTING PROBLEM FORMULATION ................................ 20

3.4. DREXEL UNIVERSITY CASE STUDIES..................................................... 23

3.4.1. DATA SET DESCRIPTION ................................................................ 23

3.4.2. SELECTED FORECASTING METHODS ......................................... 25

3.4.3. MODEL BUILDING AND FORECASTING PROCEDURE ............ 33

3.4.4. FORECAST PERFORMANCE EVALUATION FRAMEWORK ..... 35

3.4.5. RESULTS ............................................................................................. 38

3.4.6. REMARKS ........................................................................................... 67

4. ESTIMATION OF BUILDING-SPECIFIC LOAD FORECASTING PREDICTION

INTERVALS .................................................................................................................... 69

4.1. OVERVIEW ..................................................................................................... 69

4.2. PROBABILISITIC FORECASTING REVIEW .............................................. 69

4.3. PREDICTION INTERVAL FORMULATION AND ESTIMATION

PROCEDURE............................................................................................................ 72

4.4. PREDICTION INTERVAL EVALUATION FRAMEWORK ....................... 79

4.5. RESULTS ......................................................................................................... 82

4.6. REMARKS ....................................................................................................... 90

5. CONTROLLABLE BUILDING ELECTRICAL LOAD MODELING ................... 92

5.1. OVERVIEW ..................................................................................................... 92

5.2. MODEL DEVELOPMENT ............................................................................. 92

5.2.1. TEST DESCRIPTION ......................................................................... 93

5.2.2. MODEL DERIVATION ...................................................................... 95

5.2.3. DREXEL-SPECIFIC EXAMPLE ........................................................ 99

 vi
5.3. REMARKS ..................................................................................................... 102

6. OPTIMAL SCHEDULING OF BUILDING DEMAND RESOURCES ................ 104

6.1. OVERVIEW ................................................................................................... 104

6.2. BUILDING LOAD SCHEDULING PROBLEM DESCRIPTION ............... 104

6.2.1. GOALS ............................................................................................... 104

6.2.2. BUILDING DEMAND TERMINOLOGY ........................................ 105

6.2.3. SCENARIOS ...................................................................................... 107

6.2.4. ASSUMPTIONS ................................................................................ 109

6.3. OPTIMIZATION PROBLEM FORMULATIONS ....................................... 110

6.3.1. SCENARIO 1 & 2 .............................................................................. 110

6.3.2. SCENARIO 3 & 4 .............................................................................. 113

6.3.3. SCENARIO 1 & 2 WITH IMPERFECT LOAD FORECASTS ....... 115

6.3.4. SCENARIO 3 & 4 WITH IMPERFECT LOAD FORECASTS ........ 116

6.4. SIMULATIONS ............................................................................................. 117

6.4.1. TEST SYSTEM DESCRIPTION ................................................................... 117

6.4.2. SIMULATION RESULTS AND DISCUSSION........................................... 121

7. CONCLUSIONS AND FUTURE WORK .............................................................. 134

7.1. OVERVIEW ................................................................................................... 134

7.2. SUMMARY OF RESEARCH CONTRIBUTIONS ...................................... 134

7.3. FUTURE WORK ........................................................................................... 135

References ....................................................................................................................... 137

APPENDICES ................................................................................................................ 145

Appendix A. LIST OF NOMENCLATURE .............................................................. 146

 vii
Appendix B. FORECASTING CASE STUDY RESULTS ....................................... 150

VITA ............................................................................................................................... 216

 viii

LIST OF TABLES

Table 3.1. Indicator variables representing the day of the week ...................................... 26

Table 3.2. General and Building-specific predictor variables .......................................... 34

Table 3.3. Quantiles of day-ahead MAPE performance across all out-of-sample forecasts

........................................................................................................................................... 38

Table 3.4. Monthly MAPE (%) performance ................................................................... 39

Table 3.5. MAPE (%) performance broken up by weekdays and weekends/holidays ..... 39

Table 3.6. Hourly MAPE (%) across all out-of-sample forecasts .................................... 41

Table 3.7. Hourly bias (kW) across all out-of-sample forecasts....................................... 48

Table 3.8. Hourly SDE (kW) Across all out-of-sample forecasts .................................... 51

Table 3.9. Hourly skewness across all out-of-sample forecasts ....................................... 61

Table 3.10. Hourly kurtosis across all out-of-sample forecasts ........................................ 64

Table 4.1. % Deviation in empirical coverage compared to the nominal coverage rate . 85

Table 5.1 Parameter estimation results for the test described in Section 5.2.1............... 102

Table 6.1. Summary of DSM scenarios .......................................................................... 108

Table 6.2 Building load footprint definitions ................................................................ 118

Table 6.3 Building controllable load flexibility definitions............................................ 118

Table 6.4. Load footprint and flexibility for each building used in this case study........ 118

Table 6.5 Base building parameters values used in the simulations ............................... 120

Table 6.6 Optimal load schedule in kW for Test 1 (Scenario 1) .................................... 123

Table 6.7. Optimal load schedule for in kW Test 2 (Scenario 1 with reduced load

flexibility) ....................................................................................................................... 123

Table 6.8. Optimal load schedule in kW for Test 4 (Scenario 2) ................................... 127
 ix
thresh
Table 6.9 Optimal load schedule in kW for Test 5 (Scenario 3 – P =150kW) ......... 127

Table 6.10 Optimal load schedule in kW for Test 6 (Scenario 3 – Pthresh=75kW) ......... 132

Table 6.11 Optimal load schedule in kW for Test 6 (Scenario 4) .................................. 132

Table 6.12 Optimal load schedule in kW for Test 8: +10kW uniform load forecast

uncertainty....................................................................................................................... 133

Table B.1 MAPE results at 5 minute resolution for all 116 day-ahead forecasts ........... 150
 x

LIST OF FIGURES

Figure 2.1. Topology of existing DSM programs ............................................................... 7

Figure 2.2. Breakdown of commercial building energy use in the United States (2014)

[19] ...................................................................................................................................... 9

Figure 2.3. Linear correlation between demand and several variables: OAT (top), OAT-

Temp (middle), and OAT-Stpt (bottom) ........................................................................... 11

Figure 2.4. Temporal correlation for 24 hours of building demand. Sample size is 4

months of weekdays (88 days – 2011).............................................................................. 12

Figure 2.5. Example of a baseline demand profile during a DSM event .......................... 14

Figure 3.1. Sample load forecast showing time indices.................................................... 22

Figure 3.2. Breakup of data set and how each portion is used in the forecasting process 24

Figure 3.3. Daily building demand curves for the 116 days used for out of sample

testing. The thicker line shows the mean daily profile .................................................... 25

Figure 3.4. General artificial neuron model ...................................................................... 27

Figure 3.5. A general multi-layer, feed-forward artificial neural network with N hidden

layers ................................................................................................................................. 28

Figure 3.6. Diagram of the STLF process......................................................................... 33

Figure 3.7. Hourly MAPE for all forecasts generated by the MLR models ..................... 42

Figure 3.8. Hourly MAPE for all forecasts generated by the NN models ........................ 43

Figure 3.9. Hourly MAPE for all forecasts generated by the SVM models ..................... 43

Figure 3.10. Hourly MAPE for all forecasts generated by the SAM models ................... 44

Figure 3.11. Box plots of hourly MAPE results. (top) MLR without building

measurements (bottom) MLR with building measurements ............................................. 45

 xi
Figure 3.12. Box plots of hourly MAPE results. (top) NN without building measurements

(bottom) NN with building measurements ....................................................................... 45

Figure 3.13. Box plots of hourly MAPE results. (top) SVM without building

measurements (bottom) SVM with building measurements ............................................. 46

Figure 3.14. Box plots of hourly MAPE results. (top) SAM without building

measurements (bottom) SAM with building measurements ............................................. 46

Figure 3.15. Hourly Bias for all forecasts generated by the MLR models ....................... 49

Figure 3.16. Hourly Bias for all forecasts generated by the NN models .......................... 49

Figure 3.17. Hourly Bias for all forecasts generated by the SVM models ....................... 50

Figure 3.18. Hourly Bias for all forecasts generated by the SAM models ....................... 50

Figure 3.19. Hourly standard deviation of the forecast error for all forecasts generated by

the MLR models ............................................................................................................... 52

Figure 3.20. Hourly standard deviation of the forecast error for all forecasts generated by

the NN models .................................................................................................................. 53

Figure 3.21. Hourly standard deviation of the forecast error for all forecasts generated by

the SVM models ............................................................................................................... 53

Figure 3.22. Hourly standard deviation of the forecast error for all forecasts generated by

the SAM models ............................................................................................................... 54

Figure 3.23. Empirical distribution of forecast errors for the NN model without building

measurements included (09:00-10:00).............................................................................. 55

Figure 3.24. Empirical distribution of forecast errors for the NN model with building

measurements included (09:00-10:00).............................................................................. 55

 xii
Figure 3.25. Frequency of forecast errors within a ±10 kW error margin for the MLR

models ............................................................................................................................... 56

Figure 3.26. Frequency of forecast errors within a ±30 kW error margin for the MLR

models ............................................................................................................................... 57

Figure 3.27. Frequency of forecast errors within a ±10 kW error margin for the NN

models ............................................................................................................................... 57

Figure 3.28. Frequency of forecast errors within a ±30 kW error margin for the NN

models ............................................................................................................................... 58

Figure 3.29. Frequency of forecast errors within a ±10 kW error margin for the SVM

models ............................................................................................................................... 58

Figure 3.30. Frequency of forecast errors within a ±30 kW error margin for the SVM

models ............................................................................................................................... 59

Figure 3.31. Frequency of forecast errors within a ±10 kW error margin for the SAM

models ............................................................................................................................... 59

Figure 3.32. Frequency of forecast errors within a ±30 kW error margin for the SAM

models ............................................................................................................................... 60

Figure 3.33. Hourly skew of the forecast error for all forecasts generated by the MLR

models ............................................................................................................................... 62

Figure 3.34. Hourly skew of the forecast error for all forecasts generated by the NN

models ............................................................................................................................... 62

Figure 3.35. Hourly skew of the forecast error for all forecasts generated by the SVM

models ............................................................................................................................... 63
 xiii
Figure 3.36. Hourly skew of the forecast error for all forecasts generated by the SAM

models ............................................................................................................................... 63

Figure 3.37. Hourly kurtosis of the forecast error for all forecasts generated by the MLR

models ............................................................................................................................... 65

Figure 3.38. Hourly kurtosis of the forecast error for all forecasts generated by the NN

models ............................................................................................................................... 65

Figure 3.39. Hourly kurtosis of the forecast error for all forecasts generated by the SVM

models ............................................................................................................................... 66

Figure 3.40. Hourly kurtosis of the forecast error for all forecasts generated by the SAM

models ............................................................................................................................... 66

Figure 4.1. Example of the difference between a 95% confidence interval (CI) and 95%

prediction interval (PI) ...................................................................................................... 70

Figure 4.2. Seasonal block segmentation (a) and double seasonal block segmentation (b)

........................................................................................................................................... 76

Figure 4.3. Block bootstrapped time series (a) and double seasonal block bootstrapped

time series (b) .................................................................................................................... 76

Figure 4.4. Residual bootstrapping procedure .................................................................. 78

Figure 4.5. Reliability diagram for the MLR models ....................................................... 83

Figure 4.6. Reliability diagram for the NN models .......................................................... 83

Figure 4.7. Reliability diagram for the SVM models ....................................................... 84

Figure 4.8. Reliability diagram for the SAM models ....................................................... 84

Figure 4.9. Average hourly skill score for the MLR models. Nominal coverage rate =

25% ................................................................................................................................... 86
 xiv
Figure 4.10. Average hourly skill score for the MLR models. Nominal coverage rate =

75% ................................................................................................................................... 86

Figure 4.11. Average hourly skill score for the NN models. Nominal coverage rate = 25%

........................................................................................................................................... 87

Figure 4.12. Average hourly skill score for the NN models. Nominal coverage rate = 75%

........................................................................................................................................... 87

Figure 4.13. Average hourly skill score for the SVM models. Nominal coverage rate =

25% ................................................................................................................................... 88

Figure 4.14. Average hourly skill score for the SVM models. Nominal coverage rate =

75% ................................................................................................................................... 88

Figure 4.15. Average hourly skill score for the SAM models. Nominal coverage rate =

25% ................................................................................................................................... 89

Figure 4.16. Average hourly skill score for the SAM models. Nominal coverage rate =

75% ................................................................................................................................... 89

Figure 5.1. Electrical load data in %FLA and chilled water outlet temperature data in oF.

Data was collected over a 12 hour window ...................................................................... 94

Figure 5.2. Electric load (blue line) and temperature (red line) response to raising the chill

water temp. ........................................................................................................................ 95

Figure 5.3. Response to a step change in the chilled water temperature setpoint value

according to Equations (5.4)-(5.5) .................................................................................... 98

Figure 5.4. Application of the curve fit approach on the collected data for parameter

estimation ........................................................................................................................ 101

Figure 6.1. Sample building load forecast and dispatched load ..................................... 105
 xv
Figure 6.2. Relationship between load margin and building load forecast uncertainty

(expected worst case) ...................................................................................................... 107

Figure 6.3. Forecasted building load profiles ................................................................. 119

Figure 6.4. LMP data for Monday July 21st, 2014 .......................................................... 120

Figure 6.5. Dispatch schedule for buildings 7 with decreasing load flexibility (No

dispatch during the intervals not shown) ........................................................................ 124

Figure 6.6 Dispatch schedule for buildings 7 with decreasing load footprint (No dispatch

during the intervals not shown ........................................................................................ 125

Figure 6.7 Cost savings as a function of load uncertainty. Break even line shown in blue

......................................................................................................................................... 130

Figure 6.8. Decreasing number of intervals where the DR revenue constraint can be met

......................................................................................................................................... 131

Figure B.1 MLR Empirical error distribution (L) No building measurements (R) With

building measurements (00:00-01:00) ............................................................................ 158

Figure B.2 MLR Empirical error distribution (L) No building measurements (R) With

building measurements (01:00-02:00) ............................................................................ 158

Figure B.3 MLR Empirical error distribution (L) No building measurements (R) With

building measurements (02:00-03:00) ............................................................................ 158

Figure B.4 MLR Empirical error distribution (L) No building measurements (R) With

building measurements (03:00-04:00) ............................................................................ 159

Figure B.5 MLR Empirical error distribution (L) No building measurements (R) With

building measurements (04:00-05:00) ............................................................................ 159

 xvi
Figure B.6 Empirical error distribution (L) No building measurements (R) With building

measurements (05:00-06:00) .......................................................................................... 159

Figure B.7 MLR Empirical error distribution (L) No building measurements (R) With

building measurements (06:00-07:00) ............................................................................ 160

Figure B.8 MLR Empirical error distribution (L) No building measurements (R) With

building measurements (07:00-08:00) ............................................................................ 160

Figure B.9 MLR Empirical error distribution (L) No building measurements (R) With

building measurements (08:00-09:00) ............................................................................ 160

Figure B.10 MLR Empirical error distribution (L) No building measurements (R) With

building measurements (09:00-10:00) ............................................................................ 161

Figure B.11 MLR Empirical error distribution (L) No building measurements (R) With

building measurements (10:00-11:00) ............................................................................ 161

Figure B.12 MLR Empirical error distribution (L) No building measurements (R) With

building measurements (11:00-12:00) ............................................................................ 161

Figure B.13 MLR Empirical error distribution (L) No building measurements (R) With

building measurements (12:00-13:00) ............................................................................ 162

Figure B.14 MLR Empirical error distribution (L) No building measurements (R) With

building measurements (13:00-14:00) ............................................................................ 162

Figure B.15 MLR Empirical error distribution (L) No building measurements (R) With

building measurements (14:00-15:00) ............................................................................ 162

Figure B.16 MLR Empirical error distribution (L) No building measurements (R) With

building measurements (15:00-16:00) ............................................................................ 163

 xvii
Figure B.17 MLR Empirical error distribution (L) No building measurements (R) With

building measurements (16:00-17:00) ............................................................................ 163

Figure B.18 MLR Empirical error distribution (L) No building measurements (R) With

building measurements (17:00-18:00) ............................................................................ 163

Figure B.19 MLR Empirical error distribution (L) No building measurements (R) With

building measurements (18:00-19:00) ............................................................................ 164

Figure B.20 MLR Empirical error distribution (L) No building measurements (R) With

building measurements (19:00-20:00) ............................................................................ 164

Figure B.21 MLR Empirical error distribution (L) No building measurements (R) With

building measurements (20:00-21:00) ............................................................................ 164

Figure B.22 MLR Empirical error distribution (L) No building measurements (R) With

building measurements (21:00-22:00) ............................................................................ 165

Figure B.23 MLR Empirical error distribution (L) No building measurements (R) With

building measurements (22:00-23:00) ............................................................................ 165

Figure B.24 MLR Empirical error distribution (L) No building measurements (R) With

building measurements (23:00-24:00) ............................................................................ 165

Figure B.25 NN Empirical error distribution (L) No building measurements (R) With

building measurements (00:00-01:00) ............................................................................ 166

Figure B.26 NN Empirical error distribution (L) No building measurements (R) With

building measurements (01:00-02:00) ............................................................................ 166

Figure B.27 NN Empirical error distribution (L) No building measurements (R) With

building measurements (02:00-03:00) ............................................................................ 166

 xviii
Figure B.28 NN Empirical error distribution (L) No building measurements (R) With

building measurements (03:00-04:00) ............................................................................ 167

Figure B.29 NN Empirical error distribution (L) No building measurements (R) With

building measurements (04:00-05:00) ............................................................................ 167

Figure B.30 NN Empirical error distribution (L) No building measurements (R) With

building measurements (05:00-06:00) ............................................................................ 167

Figure B.31 NN Empirical error distribution (L) No building measurements (R) With

building measurements (06:00-07:00) ............................................................................ 168

Figure B.32 NN Empirical error distribution (L) No building measurements (R) With

building measurements (07:00-08:00) ............................................................................ 168

Figure B.33 NN Empirical error distribution (L) No building measurements (R) With

building measurements (08:00-09:00) ............................................................................ 168

Figure B.34 NN Empirical error distribution (L) No building measurements (R) With

building measurements (09:00-10:00) ............................................................................ 169

Figure B.35 NN Empirical error distribution (L) No building measurements (R) With

building measurements (10:00-11:00) ............................................................................ 169

Figure B.36 NN Empirical error distribution (L) No building measurements (R) With

building measurements (11:00-12:00) ............................................................................ 169

Figure B.37 NN Empirical error distribution (L) No building measurements (R) With

building measurements (12:00-13:00) ............................................................................ 170

Figure B.38 NN Empirical error distribution (L) No building measurements (R) With

building measurements (13:00-14:00) ............................................................................ 170

 xix
Figure B.39 NN Empirical error distribution (L) No building measurements (R) With

building measurements (14:00-15:00) ............................................................................ 170

Figure B.40 NN Empirical error distribution (L) No building measurements (R) With

building measurements (15:00-16:00) ............................................................................ 171

Figure B.41 NN Empirical error distribution (L) No building measurements (R) With

building measurements (16:00-17:00) ............................................................................ 171

Figure B.42 NN Empirical error distribution (L) No building measurements (R) With

building measurements (17:00-18:00) ............................................................................ 171

Figure B.43 NN Empirical error distribution (L) No building measurements (R) With

building measurements (18:00-19:00) ............................................................................ 172

Figure B.44 NN Empirical error distribution (L) No building measurements (R) With

building measurements (19:00-20:00) ............................................................................ 172

Figure B.45 NN Empirical error distribution (L) No building measurements (R) With

building measurements (20:00-21:00) ............................................................................ 172

Figure B.46 NN Empirical error distribution (L) No building measurements (R) With

building measurements (21:00-22:00) ............................................................................ 173

Figure B.47 NN Empirical error distribution (L) No building measurements (R) With

building measurements (22:00-23:00) ............................................................................ 173

Figure B.48 NN Empirical error distribution (L) No building measurements (R) With

building measurements (23:00-24:00) ............................................................................ 173

Figure B.49 SVM Empirical error distribution (L) No building measurements (R) With

building measurements (00:00-01:00) ............................................................................ 174

 xx
Figure B.50 SVM Empirical error distribution (L) No building measurements (R) With

building measurements (01:00-02:00) ............................................................................ 174

Figure B.51 SVM Empirical error distribution (L) No building measurements (R) With

building measurements (02:00-03:00) ............................................................................ 174

Figure B.52 SVM Empirical error distribution (L) No building measurements (R) With

building measurements (03:00-04:00) ............................................................................ 175

Figure B.53 SVM Empirical error distribution (L) No building measurements (R) With

building measurements (04:00-05:00) ............................................................................ 175

Figure B.54 SVM Empirical error distribution (L) No building measurements (R) With

building measurements (05:00-06:00) ............................................................................ 175

Figure B.55 SVM Empirical error distribution (L) No building measurements (R) With

building measurements (06:00-07:00) ............................................................................ 176

Figure B.56 SVM Empirical error distribution (L) No building measurements (R) With

building measurements (07:00-08:00) ............................................................................ 176

Figure B.57 SVM Empirical error distribution (L) No building measurements (R) With

building measurements (08:00-09:00) ............................................................................ 176

Figure B.58 SVM Empirical error distribution (L) No building measurements (R) With

building measurements (09:00-10:00) ............................................................................ 177

Figure B.59 SVM Empirical error distribution (L) No building measurements (R) With

building measurements (10:00-11:00) ............................................................................ 177

Figure B.60 SVM Empirical error distribution (L) No building measurements (R) With

building measurements (11:00-12:00) ............................................................................ 177

 xxi
Figure B.61 SVM Empirical error distribution (L) No building measurements (R) With

building measurements (12:00-13:00) ............................................................................ 178

Figure B.62 SVM Empirical error distribution (L) No building measurements (R) With

building measurements (13:00-14:00) ............................................................................ 178

Figure B.63 SVM Empirical error distribution (L) No building measurements (R) With

building measurements (14:00-15:00) ............................................................................ 178

Figure B.64 SVM Empirical error distribution (L) No building measurements (R) With

building measurements (15:00-16:00) ............................................................................ 179

Figure B.65 SVM Empirical error distribution (L) No building measurements (R) With

building measurements (16:00-17:00) ............................................................................ 179

Figure B.66 SVM Empirical error distribution (L) No building measurements (R) With

building measurements (17:00-18:00) ............................................................................ 179

Figure B.67 SVM Empirical error distribution (L) No building measurements (R) With

building measurements (18:00-19:00) ............................................................................ 180

Figure B.68 SVM Empirical error distribution (L) No building measurements (R) With

building measurements (19:00-20:00) ............................................................................ 180

Figure B.69 SVM Empirical error distribution (L) No building measurements (R) With

building measurements (20:00-21:00) ............................................................................ 180

Figure B.70 SVM Empirical error distribution (L) No building measurements (R) With

building measurements (21:00-22:00) ............................................................................ 181

Figure B.71 SVM Empirical error distribution (L) No building measurements (R) With

building measurements (22:00-23:00) ............................................................................ 181

 xxii
Figure B.72 SVM Empirical error distribution (L) No building measurements (R) With

building measurements (23:00-24:00) ............................................................................ 181

Figure B.73 SAM Empirical error distribution (L) No building measurements (R) With

building measurements (00:00-01:00) ............................................................................ 182

Figure B.74 SAM Empirical error distribution (L) No building measurements (R) With

building measurements (01:00-02:00) ............................................................................ 182

Figure B.75 SAM Empirical error distribution (L) No building measurements (R) With

building measurements (02:00-03:00) ............................................................................ 182

Figure B.76 SAM Empirical error distribution (L) No building measurements (R) With

building measurements (03:00-04:00) ............................................................................ 183

Figure B.77 SAM Empirical error distribution (L) No building measurements (R) With

building measurements (04:00-05:00) ............................................................................ 183

Figure B.78 SAM Empirical error distribution (L) No building measurements (R) With

building measurements (05:00-06:00) ............................................................................ 183

Figure B.79 SAM Empirical error distribution (L) No building measurements (R) With

building measurements (06:00-07:00) ............................................................................ 184

Figure B.80 SAM Empirical error distribution (L) No building measurements (R) With

building measurements (07:00-08:00) ............................................................................ 184

Figure B.81 SAM Empirical error distribution (L) No building measurements (R) With

building measurements (08:00-09:00) ............................................................................ 184

Figure B.82 SAM Empirical error distribution (L) No building measurements (R) With

building measurements (09:00-10:00) ............................................................................ 185

 xxiii
Figure B.83 SAM Empirical error distribution (L) No building measurements (R) With

building measurements (10:00-11:00) ............................................................................ 185

Figure B.84 SAM Empirical error distribution (L) No building measurements (R) With

building measurements (11:00-12:00) ............................................................................ 185

Figure B.85 SAM Empirical error distribution (L) No building measurements (R) With

building measurements (12:00-13:00) ............................................................................ 186

Figure B.86 SAM Empirical error distribution (L) No building measurements (R) With

building measurements (13:00-14:00) ............................................................................ 186

Figure B.87 SAM Empirical error distribution (L) No building measurements (R) With

building measurements (14:00-15:00) ............................................................................ 186

Figure B.88 SAM Empirical error distribution (L) No building measurements (R) With

building measurements (15:00-16:00) ............................................................................ 187

Figure B.89 SAM Empirical error distribution (L) No building measurements (R) With

building measurements (16:00-17:00) ............................................................................ 187

Figure B.90 SAM Empirical error distribution (L) No building measurements (R) With

building measurements (17:00-18:00) ............................................................................ 187

Figure B.91 SAM Empirical error distribution (L) No building measurements (R) With

building measurements (18:00-19:00) ............................................................................ 188

Figure B.92 SAM Empirical error distribution (L) No building measurements (R) With

building measurements (19:00-20:00) ............................................................................ 188

Figure B.93 SAM Empirical error distribution (L) No building measurements (R) With

building measurements (20:00-21:00) ............................................................................ 188

 xxiv
Figure B.94 SAM Empirical error distribution (L) No building measurements (R) With

building measurements (21:00-22:00) ............................................................................ 189

Figure B.95 SAM Empirical error distribution (L) No building measurements (R) With

building measurements (22:00-23:00) ............................................................................ 189

Figure B.96 SAM Empirical error distribution (L) No building measurements (R) With

building measurements (23:00-24:00) ............................................................................ 189

Figure B.97 Average hourly skill score for the MLR Models. Nominal coverage rate =

5% ................................................................................................................................... 190

Figure B.98 Average hourly skill score for the MLR Models. Nominal coverage rate =

10% ................................................................................................................................. 190

Figure B.99 Average hourly skill score for the MLR Models. Nominal coverage rate =

15% ................................................................................................................................. 190

Figure B.100 Average hourly skill score for the MLR Models. Nominal coverage rate =

20% ................................................................................................................................. 191

Figure B.101 Average hourly skill score for the MLR Models. Nominal coverage rate =

25% ................................................................................................................................. 191

Figure B.102 Average hourly skill score for the MLR Models. Nominal coverage rate =

30% ................................................................................................................................. 191

Figure B.103 Average hourly skill score for the MLR Models. Nominal coverage rate =

35% ................................................................................................................................. 192

Figure B.104 Average hourly skill score for the MLR Models. Nominal coverage rate =

40% ................................................................................................................................. 192

 xxv
Figure B.105 Average hourly skill score for the MLR Models. Nominal coverage rate =

45% ................................................................................................................................. 192

Figure B.106 Average hourly skill score for the MLR Models. Nominal coverage rate =

50% ................................................................................................................................. 193

Figure B.107 Average hourly skill score for the MLR Models. Nominal coverage rate =

55% ................................................................................................................................. 193

Figure B.108 Average hourly skill score for the MLR Models. Nominal coverage rate =

60% ................................................................................................................................. 193

Figure B.109 Average hourly skill score for the MLR Models. Nominal coverage rate =

65% ................................................................................................................................. 194

Figure B.110 Average hourly skill score for the MLR Models. Nominal coverage rate =

70% ................................................................................................................................. 194

Figure B.111 Average hourly skill score for the MLR Models. Nominal coverage rate =

75% ................................................................................................................................. 194

Figure B.112 Average hourly skill score for the MLR Models. Nominal coverage rate =

80% ................................................................................................................................. 195

Figure B.113 Average hourly skill score for the MLR Models. Nominal coverage rate =

85% ................................................................................................................................. 195

Figure B.114 Average hourly skill score for the MLR Models. Nominal coverage rate =

90% ................................................................................................................................. 195

Figure B.115 Average hourly skill score for the MLR Models. Nominal coverage rate =

95% ................................................................................................................................. 196

 xxvi
Figure B.116 Average hourly skill score for the NN Models. Nominal coverage rate =

5% ................................................................................................................................... 196

Figure B.117 Average hourly skill score for the NN Models. Nominal coverage rate =

10% ................................................................................................................................. 196

Figure B.118 Average hourly skill score for the NN Models. Nominal coverage rate =

15% ................................................................................................................................. 197

Figure B.119 Average hourly skill score for the NN Models. Nominal coverage rate =

20% ................................................................................................................................. 197

Figure B.120 Average hourly skill score for the NN Models. Nominal coverage rate =

25% ................................................................................................................................. 197

Figure B.121 Average hourly skill score for the NN Models. Nominal coverage rate =

30% ................................................................................................................................. 198

Figure B.122 Average hourly skill score for the NN Models. Nominal coverage rate =

35% ................................................................................................................................. 198

Figure B.123 Average hourly skill score for the NN Models. Nominal coverage rate =

40% ................................................................................................................................. 198

Figure B.124 Average hourly skill score for the NN Models. Nominal coverage rate =

45% ................................................................................................................................. 199

Figure B.125 Average hourly skill score for the NN Models. Nominal coverage rate =

50% ................................................................................................................................. 199

Figure B.126 Average hourly skill score for the NN Models. Nominal coverage rate =

55% ................................................................................................................................. 199

 xxvii
Figure B.127 Average hourly skill score for the NN Models. Nominal coverage rate =

60% ................................................................................................................................. 200

Figure B.128 Average hourly skill score for the NN Models. Nominal coverage rate =

65% ................................................................................................................................. 200

Figure B.129 Average hourly skill score for the NN Models. Nominal coverage rate =

70% ................................................................................................................................. 200

Figure B.130 Average hourly skill score for the NN Models. Nominal coverage rate =

75% ................................................................................................................................. 201

Figure B.131 Average hourly skill score for the NN Models. Nominal coverage rate =

80% ................................................................................................................................. 201

Figure B.132 Average hourly skill score for the NN Models. Nominal coverage rate =

85% ................................................................................................................................. 201

Figure B.133 Average hourly skill score for the NN Models. Nominal coverage rate =

90% ................................................................................................................................. 202

Figure B.134 Average hourly skill score for the NN Models. Nominal coverage rate =

95% ................................................................................................................................. 202

Figure B.135 Average hourly skill score for the SVM Models. Nominal coverage rate =

5% ................................................................................................................................... 202

Figure B.136 Average hourly skill score for the SVM Models. Nominal coverage rate =

10% ................................................................................................................................. 203

Figure B.137 Average hourly skill score for the SVM Models. Nominal coverage rate =

15% ................................................................................................................................. 203

 xxviii
Figure B.138 Average hourly skill score for the SVM Models. Nominal coverage rate =

20% ................................................................................................................................. 203

Figure B.139 Average hourly skill score for the SVM Models. Nominal coverage rate =

25% ................................................................................................................................. 204

Figure B.140 Average hourly skill score for the SVM Models. Nominal coverage rate =

30% ................................................................................................................................. 204

Figure B.141 Average hourly skill score for the SVM Models. Nominal coverage rate =

35% ................................................................................................................................. 204

Figure B.142 Average hourly skill score for the SVM Models. Nominal coverage rate =

40% ................................................................................................................................. 205

Figure B.143 Average hourly skill score for the SVM Models. Nominal coverage rate =

45% ................................................................................................................................. 205

Figure B.144 Average hourly skill score for the SVM Models. Nominal coverage rate =

50% ................................................................................................................................. 205

Figure B.145 Average hourly skill score for the SVM Models. Nominal coverage rate =

55% ................................................................................................................................. 206

Figure B.146 Average hourly skill score for the SVM Models. Nominal coverage rate =

60% ................................................................................................................................. 206

Figure B.147 Average hourly skill score for the SVM Models. Nominal coverage rate =

65% ................................................................................................................................. 206

Figure B.148 Average hourly skill score for the SVM Models. Nominal coverage rate =

70% ................................................................................................................................. 207

 xxix
Figure B.149 Average hourly skill score for the SVM Models. Nominal coverage rate =

75% ................................................................................................................................. 207

Figure B.150 Average hourly skill score for the SVM Models. Nominal coverage rate =

80% ................................................................................................................................. 207

Figure B.151 Average hourly skill score for the SVM Models. Nominal coverage rate =

85% ................................................................................................................................. 208

Figure B.152 Average hourly skill score for the SVM Models. Nominal coverage rate =

90% ................................................................................................................................. 208

Figure B.153 Average hourly skill score for the SVM Models. Nominal coverage rate =

95% ................................................................................................................................. 208

Figure B.154 Average hourly skill score for the SAM Models. Nominal coverage rate =

5% ................................................................................................................................... 209

Figure B.155 Average hourly skill score for the SAM Models. Nominal coverage rate =

10% ................................................................................................................................. 209

Figure B.156 Average hourly skill score for the SAM Models. Nominal coverage rate =

15% ................................................................................................................................. 209

Figure B.157 Average hourly skill score for the SAM Models. Nominal coverage rate =

20% ................................................................................................................................. 210

Figure B.158 Average hourly skill score for the SAM Models. Nominal coverage rate =

25% ................................................................................................................................. 210

Figure B.159 Average hourly skill score for the SAM Models. Nominal coverage rate =

30% ................................................................................................................................. 210

 xxx
Figure B.160 Average hourly skill score for the SAM Models. Nominal coverage rate =

35% ................................................................................................................................. 211

Figure B.161 Average hourly skill score for the SAM Models. Nominal coverage rate =

40% ................................................................................................................................. 211

Figure B.162 Average hourly skill score for the SAM Models. Nominal coverage rate =

45% ................................................................................................................................. 211

Figure B.163 Average hourly skill score for the SAM Models. Nominal coverage rate =

50% ................................................................................................................................. 212

Figure B.164 Average hourly skill score for the SAM Models. Nominal coverage rate =

55% ................................................................................................................................. 212

Figure B.165 Average hourly skill score for the SAM Models. Nominal coverage rate =

60% ................................................................................................................................. 212

Figure B.166 Average hourly skill score for the SAM Models. Nominal coverage rate =

65% ................................................................................................................................. 213

Figure B.167 Average hourly skill score for the SAM Models. Nominal coverage rate =

70% ................................................................................................................................. 213

Figure B.168 Average hourly skill score for the SAM Models. Nominal coverage rate =

75% ................................................................................................................................. 213

Figure B.169 Average hourly skill score for the SAM Models. Nominal coverage rate =

80% ................................................................................................................................. 214

Figure B.170 Average hourly skill score for the SAM Models. Nominal coverage rate =

85% ................................................................................................................................. 214

 xxxi
Figure B.171 Average hourly skill score for the SAM Models. Nominal coverage rate =

90% ................................................................................................................................. 214

Figure B.172 Average hourly skill score for the SAM Models. Nominal coverage rate =

95% ................................................................................................................................. 215

 xxxii

ABSTRACT
Modeling and Analysis of Commercial Building Electrical Loads for Demand Side
Management
Jonathan Berardino
Chikaodinaka Nwankpa, Ph.D.

In recent years there has been a push in the electric power industry for more

customer involvement in the electricity markets. Traditionally the end user has played a

passive role in the planning and operation of the power grid. However, many energy

markets have begun opening up opportunities to consumers who wish to commit a certain

amount of their electrical load under various demand side management programs. The

potential benefits of more demand participation include reduced operating costs and new

revenue opportunities for the consumer, as well as more reliable and secure operations for

the utilities. The management of these load resources creates challenges and

opportunities to the end user that were not present in previous market structures.

This work examines the behavior of commercial-type building electrical loads and

their capacity for supporting demand side management actions. This work is motivated

by the need for accurate and dynamic tools to aid in the advancement of demand side

operations. A dynamic load model is proposed for capturing the response of controllable

building loads. Building-specific load forecasting techniques are developed, with

particular focus paid to the integration of building management system (BMS)

information. These approaches are tested using Drexel University building data. The

application of building-specific load forecasts and dynamic load modeling to the optimal

scheduling of multi-building systems in the energy market is proposed. Sources of

 xxxiii
potential load uncertainty are introduced in the proposed energy management problem

formulation in order to investigate the impact on the resulting load schedule.

 1

1. INTRODUCTION

1.1. OVERVIEW

This thesis presents a study of commercial buildings as potential resources for demand

side management. Within this work, methods for modeling and forecasting building electric load

behavior are presented. These problems are examined through extensive case studies involving

Drexel University building data. The resulting modeling and forecasting approaches are applied

to the problem of optimally scheduling building loads as part of several demand side

management programs. Simulations demonstrate the potential value in optimally dispatching

building loads, subject to real-world building operational constraints and building load

variability.

In this chapter, the following topics are presented:

 The background and motivation for the work

 A summary of the objectives and main contributions of this research

 An overview of the thesis organization

1.2. BACKGROUND AND MOTIVATION

Power system operators are tasked with ensuring adequate generation is available to meet

projected demand. Traditionally this requirement is met through controlling the supply side (i.e.

- generators) of the power system; limited priority has been placed on involving the demand side.

However, increased investment in the development of a “Smart Grid” [1] is driving new

opportunities for the demand side to take a more active role in power system planning and

operations. Demand Side Management (DSM), also referred to often as Demand Response

(DR), is a class of programs that are designed to motivate end-use customers to modify their
 2
electricity usage. This could be through the shifting of electricity use to another time (e.g. – off

peak hours) or shedding electric load temporarily.

The idea of managing demand side resources has been discussed since the 1890’s, as

described in detail in [2]. However, it was not until the restructuring and deregulation of electric

utilities in the 1990’s and subsequent issues that began to arise in the new wholesale markets that

a concerted effort was made to include DSM as an essential aspect of these new markets [3]. In

support of this, the United States government has issued a number of policies in an effort to

remove potential barriers to DSM participants [4] [5] [6] [7]. These programs have been

identified as having the potential to provide a wide range of benefits to both the power system

operator and the end-user [8] [9] [10]. Potential benefits include:

 Reduction in grid demand during peak hours and subsequent reduction in the

reliance on expensive peaking units [11]. In the United States, DSM potential is

estimated to be between 38GW and 188GW by 2019 [8]. Peak reduction also has

the added potential to defer transmission system infrastructure upgrades that may

otherwise be required to expand system capacity [12].

 Reduction in wholesale energy prices and decreased price volatility. Even a small

increase in demand elasticity can offset the extreme increase in generation cost at

high demand levels [3], [13].

 Increased system reliability. DSM resources can be scheduled in the ancillary

services market for regulation, spinning reserve, or to support the integration of

renewable resources [14] [15] [16] [17] [18].

 Reduced electricity costs and new revenue opportunities through the local

electricity market operator for the end-user.

 3
Building electrical loads have been identified as important potential resources for

supporting power system operations in owing to a high load footprint [19] and controllability that

is at least on par with that of generators [20]. The future involvement of building loads in

various DSM programs is buoyed by technological advancements such as advanced metering

infrastructure (AMI). Such advancements bring a new level of two-way communication and

controllability between the end-user and the utility.

The motivation for this thesis is the need for improved methods to assess building-

specific DSM opportunities and support increased involvement of building loads in various DSM

programs. In order for building participation in DSM programs to continue to evolve, the

understanding of end-user capabilities and limitations must continue to progress. Specifically to

that end, this thesis will consider what tools are needed from the perspective of the end-user in

order to understand and plan their own building loads as part of a DSM program.

1.3. OBJECTIVES

The objectives of this work include:

 Describe the existing standard of DSM programs and the potential for using

commercial building electric loads as controllable resources

 Evaluate how building-specific information can improve predictions of building

electrical load and better characterize controllable building load behavior

 Study the problem of optimally scheduling building loads on a multi-building

campus. This study will include:

o Multiple DSM scenarios

o Realistic building operational constraints

o Building load variability

 4
1.4. CONTRIBUTIONS

The main contributions of this research can be summarized as follows:

 An examination of conventional load forecasting methods enhanced with

building-specific information. This includes:

o Demonstration of improved accuracy and consistency of point forecasts

generated via enhanced conventional load forecasting methods

o Evaluation of demand side planning risk through both enhanced building-

specific point forecasts and probabilistic forecasts

 Development of a dynamic load model for a controllable HVAC chiller that

captures the thermal-electrical behavior of the load.

 Formulation of several optimization problems for finding the optimal building

load dispatch schedule of a multi-building campus. Conditions under which the

optimization problems are formulated include:

o Real-time pricing structures

o Opportunities for demand response revenue

o Building electrical and thermal operating constraints

o Controllable load dynamic behavior

o Varying levels of load forecast uncertainty

1.5. ORGANIZATION OF THESIS

This thesis is organized as follows

 Chapter 2 reviews the existing state of commercial buildings as resources for

demand side management programs. Also, the role of building-specific

information in describing commercial building electrical load behavior is covered.

 5
 In Chapter 3, the problem of building-specific load forecasting is considered.

The main topics of interest include:

o Application of building-specific variables to the load forecasting problem

o Drexel specific load forecasting case study with results and observations

 Chapter 4 extends the results of Chapter 3 to the probabilistic load forecasting

problem. Methods of estimating prediction intervals of building demand are

presented, with an emphasis on developing and evaluating prediction intervals

conditioned on building-specific variables.

 In Chapter 5 a model of controllable building electrical loads is developed. In

this work the load is considered to be the building HVAC chiller.

 Chapter 6 presents a methodology of optimally scheduling building loads for a

multi-building campus. Simulation results for several problem formulations are

presented and discussed.

 Chapter 7 will summarize the contributions of this research and present a

discussion on related future work.

 6
2. ANALYSIS OF COMMERCIAL BUILDING LOADS AS DEMAND SIDE

RESOURCES

2.1. OVERVIEW

This chapter examines the potential of using commercial building loads as controllable

load resources in DSM programs. Specifically this includes the following:

 An introduction to available DSM programs and considerations for integrating

building loads into the programs moving forward

 A review of commercial building energy use in the United States and a

breakdown of building energy consumption

 An exploration of the relationship between building-level information and

building demand

 A review of the existing research specific to employing commercial buildings as

demand side resources. Special attention is paid to methods for forecasting

building loads and evaluating building DSM capabilities. These methods will be

contrasted with the approach taken in this thesis.

2.2. DEMAND SIDE MANAGEMENT PROGRAMS

As mentioned previously, there are a wide variety of DSM programs that may be

available to the end-user. These programs are generally grouped into two basic types: price-

based and incentive based programs [10]. Figure 2.1 shows this classification of various DSM

programs. Price-based programs, as the name would imply, use variable electricity pricing

structures to encourage modification of electricity use. Time-of-use (TOU) pricing programs

provide variable rate plans to customers where the rates are defined well in advance of pre-
 7
defined usage periods [21]. These usage periods can include, for example, daily peak or off

peak hours or seasonal time windows where each period is priced differently.

Figure 2.1. Topology of existing DSM programs

In contrast to TOU pricing, dynamic pricing sets electricity rates that are not known to

the end-user so far in advance. These rates may be set on a day-ahead or an hour-ahead time

scale. Dynamic pricing is a key factor in both Critical Peak Pricing (CPP) and Real Time Pricing

(RTP) programs. During CPP events which occur during high system demand days several

times a year, enrolled participants receive higher electricity rates to encourage reduced

electricity. In exchange, program participants receive a discount on normal operating electricity

rates [22]. In RTP programs, the cost of electricity is variable from day to day or hour to hour.

At present time, RTP programs are not very common. These types of programs rely on two-way

communication between the utility and end-user for pricing information. As more AMI projects

and other communication and control advancements are put into operation, RTP programs

should become more widespread. It is arguable that the evolution of DSM will rely heavily on

the universal availability of real-time price information.

 8
Incentive-based DSM programs can be grouped into two categories: direct control and

market based programs. Direct control refers to programs where the end-user allows the utility

to control their load in exchange for an agreed upon payment or credit. Interruptible Load

Management (ILM), which targets mainly large industrial customers, and Direct Load Control

(DLC) programs, which focus mainly on residential customers, have been around since the

1970’s [2] to support peak load management. These programs continue to be employed by

utilities today.

Market based programs are a more recent expansion of DSM opportunities which allow

the end-user to participate in the wholesale electricity market. In Demand Bidding and Capacity

Market programs, the end-user bids demand reductions and, if the bids are accepted, must

provide demand reductions at the specified time [23]. In these programs the demand-side bids

are optimized with the supply-side bids, effectively treating loads and generators as equal players

in the market. Participation in the Ancillary services market includes, for example, registering

controllable demand as resources for regulation or spinning reserve applications [14] [15] [16]

[17] [18].

The DSM programs described above continue to grow in participation as time goes on. It

is expected these markets will continue to advance and that building loads will be increasingly

utilized as DSM sources in all programs [8]. While building loads make attractive DSM

participants there are still practical challenges that remain in order to evaluate participation in

DSM applications. Several qualities of the loads themselves need to be better understood: what

factors drive building demand, what temporal behavior do they present, can the behavior be

described accurately in a form conducive to automated decision making, and what uncertainties

arise when controlling building loads.

 9
2.3. COMMERCIAL BUILDING ELECTRICITY USE AND BEHAVIOR

One of the most appealing aspects of using commercial building loads for DSM purposes

is the large electrical footprint and potential for controllability. In 2014, commercial building

energy use made up 34% of the total United States electricity demand [19]. Within these

buildings, over 50% of the electricity was consumed by the HVAC and lighting systems as

shown in Figure 2.2.

Figure 2.2. Breakdown of commercial building energy use in the United States (2014) [19]

It stands to reason given this breakdown that improved operation of HVAC systems has

been identified as a means of potential energy savings and increased energy efficiency in

commercial buildings [24]. It also indicates a strongly coupled relationship between the building

thermal mass and building demand. By exploring this relationship, important information can be

obtained in regards to factors driving building load and the dynamics of building load behavior.

The approach taken in this thesis is to focus on information available from building

management systems (BMS) typical of commercial buildings that influence building electrical

behavior. These measurements are directly linked to the thermal-electrical behavior that drives a

large portion of the building demand. Additionally, this approach can lead to the development of
 10
data-driven building models that are well suited for integration in a DSM scheduling tool

deployed at the BMS level.

Many medium to large commercial and industrial buildings operate a BMS that is

responsible for the monitoring and control of various building mechanical and electrical system

performance, such as the heating and cooling systems. These BMS are typically configured as a

distributed control system, with a software layer managing the functions of hardware

components distributed throughout a building or a campus consisting of multiple buildings. In

order to study building load features, electrical and thermal data has been collected using the

Drexel University BMS over the last several years. Three main variables are considered from

this data:

1. Outside Air Temperature (OAT) as measured directly at the building

2. Temperature Gradient: Outside Air Temp. – Indoor Air Temp. (OAT-Temp)

3. Target Temperature Gradient: Outside Air Temp. – Indoor Air Temp. Setpoint (OAT-

Stpt)

For the studies presented in this chapter, the data used is from the summer months (May-

August) of 2011 and 2012. A large portion of the load for campus buildings is the HVAC

system, making them good candidates as controllable loads during the typically warm and humid

summers. Consequently, that makes these good months for studying building behavior to

support DSM planning activities. It should be noted that Drexel does not have electric heating in

the majority of buildings on campus (and none where data recording capabilities exist).

However the approach employed in this thesis could easily be employed in characterizing the

building load behavior if heating loads were also a potential controllable resource.

Figure 2.3 shows the linear correlation between demand and each BMS measured

variable from above for 2011 and 2012.

 11

Figure 2.3. Linear correlation between demand and several variables: OAT (top), OAT-Temp (middle), and
OAT-Stpt (bottom)

Particularly strong correlation is noted between demand and the target temperature

gradient. This relationship makes sense intuitively, as the building HVAC system needs to draw

more energy to maintain a given setpoint inside the building when temperature outside increases.

These results indicate a reasonable correlation, particularly when considering the relationship

between demand and temperature is inherently nonlinear. Additional correlation studies have

been performed and the results are available in [25].

In addition to correlation with measured BMS variables, there are temporal dependencies

that arise due to the thermal-electrical coupling present in large HVAC electrical loads [26] [27],

which make up a considerable portion of the load base for commercial buildings [28] [19].

Figure 2.4 shows an example of the correlation between measured demand values with future

demand levels.
 12
Linear Correlation Between Demand Readings (Summer 2011)
0 1

3 0.9

0.8
6

0.7
9

0.6
12

0.5

15
0.4

18
0.3

21
0.2

24
0 3 6 9 12 15 18 21 24

Figure 2.4. Temporal correlation for 24 hours of building demand. Sample size is 4 months of weekdays (88
days – 2011)

Figure 2.4 depicts how the demand at each point in time is correlated to the demand at a

future point in time over 24 hours, with samples taken every 5 minutes. For this example there

are large pockets of high correlation shown, particularly between 4am and 8am and 1pm and

8pm. This serial correlation between demand observations is representative of the temporal

dependencies found in building load behavior. The HVAC consumption, and indeed that of

other load components, is driven not only by environmental factors but also building operational

schedules and occupant behaviors. When compared to the residential sector, commercial

buildings experience less variance day in and day out due to more consistent schedules and

occupant behaviors. However, from the demand usage patterns, important intraday

characteristics related to said behavioral determinants can be extracted and characterized to

develop improved, data-informed building demand profiles [29]. These patterns are inherently

linked to the measurements captured via the BMS.

It has now been established that this thesis will focus on applying information available

from building management systems typical of commercial buildings to describe building

 13
electrical behavior for DSM planning. In the next section, similar research efforts studying

building resources for DSM are reviewed and contrasted with the approach taken in this thesis.

2.4. REVIEW OF COMMERCIAL BUILDING DEMAND SIDE MANAGEMENT

RESEARCH

Before buildings can be deployed as part of a DSM program, two questions must be

addressed: how accurately can the load of the building be predicted and how can a building load

be integrated in a DSM application. This thesis argues that the answer to both questions is more

realistic, and consequently more accurate, if measurements from the BMS play a role in the

DSM planning process.

Given the potential of buildings as DSM resources it is not surprising that there is

considerable research that has been conducted in attempting to answer these questions. The next

two subsections look at the state-of-the-art in predicting building loads and modeling and

scheduling building loads for DSM.

2.4.1. BUILDING-SPECIFIC LOAD FORECASTING

The load shed during a DSM event must be measured against a forecast of what the

demand would be if no adjustment occurred. This forecast is often referred to as the baseline

demand. Figure 2.5 shows an example of the baseline demand and actual metered load during a

DSM event. The area between the magenta and blue vertical lines is the period of time in which

an adjustment to the building load is made.

 14
240

Baseline

Metered Load
220

200

Demand (kW) 180

160

140

120
0 4 8 12 16 20 24
Time of Day (hours)

Figure 2.5. Example of a baseline demand profile during a DSM event

Utilities generally use simple baseline models that involve averaging the daily demand

over several days (excluding DSM event days) [30]. For example, CAISO introduced a “3-in-

10” method where the baseline is based on the hourly average of the three highest energy usages

in the past ten similar days [31]. PJM used the same approach except they look at four out of the

past 5 similar days [23]. Recent adjustments to both the CAISO and PJM models have been

made to improve the accuracy. Observations of metered demand are collected several hours

prior to the beginning of a DSM event and the original baseline calculations are adjusted up or

down based on these more recent demand readings. This adjustment has improved the accuracy

in quantifying demand behavior for economic settlement post event. However, this adjustment

provides no benefit for forecasting building loads except in near real-time applications since it

depends on very recent demand readings.

Several regression-based approaches have been proposed in the literature as alternatives

to the averaging methods discussed above [32] [33] [34] [35]. These methods include predictor

variables in the model such as weather and calendar information in an attempt to better predict

load. However, a comprehensive analysis of baseline calculation methods prepared for PJM
 15
showed only 2 of 13 unadjusted methods applied a regression approach [36] and only one

method was actually employed by a utility (ERCOT). The results of this study also indicated the

regression models, as currently designed, did not necessarily outperform the other models in the

tests that were performed.

Beyond the averaging and regression-based baseline calculation approaches that have

been discussed, there is a surprising lack of more sophisticated approaches in the technical

literature [37]. This can be attributed more to terminology rather than research effort however.

The DSM community tends to use the term baseline as opposed to forecast. Baseline has the

very specific connotation as the load profile against which incentive-based DSM performance is

measured when the utility and end-user reach settlement following a DSM event. There are

other research efforts that focus on building-specific forecasts for studying building load

behavior. The models in these works tend to be more complex and less intuitive than the

baseline models but they more thoroughly capture building load behavior and produce more

consistently accurate load forecasts. These works are not solely concerned with quantifying

building behavior for DSM, although that is still often a motivation. Going forward, this thesis

will use the term forecast to refer to any approach concerned with predicting the future building

electricity demand.

Methods for forecasting end-user facility demand are presented in [30] [25] [38] [39] [40]

[41] [42] [43] [44] [45] [46] [47] [48] [49]. These works have applied a wide variety of methods

that are often used in short term load forecasting (STLF) studies but at a campus or building

level. In [30] [25] [38] [39] regression models are used. [40] [41] [42] [43] [44] employ several

different forms of neural network models. [45] proposes a new day-ahead probabilistic model

based on Gaussian processes for an industrial facility. Support vector machines (SVM) are used

in [46] to forecast monthly demand at four commercial sites in Singapore and in a short-term
 16
application in [47]. In [48] and [49], several forecasting models are combined to create

ensemble models for forecasting building demand. All of these works recognize that forecasting

at the building level must deal with a higher degree of variability, a feature that becomes washed

out to a degree when forecasting at a transmission substation level with aggregated loads.

However, of these references, only [47] included internal building measurements in their forecast

models. The measurements were collected from a limited sensor network installed for testing

purposes. These measurements included occupancy information from sensors at the two

building entrances and four temperature sensors distributed through the building.

In contrast to the works above, the approach taken in this thesis is to focus on including

information available from building management systems (BMS) typical of commercial

buildings that influence building electrical behavior. As discussed earlier in this chapter, these

measurements are directly linked to the thermal-electrical behavior that drives a large portion of

the building demand. The goal is to enhance existing STLF techniques with these building-

specific measurements and compare the performance against similarly trained models that do not

include this information. This work is presented in detail in Chapters 3 and 4.

2.4.2. COMMERCIAL BUILDING LOAD MODELING AND DISPATCH

When surveying the literature on building load modeling, contributions predominantly

come from the HVAC engineer community. This research tends to provide highly detailed

models of the building thermal dynamic behavior [50]. Often these models require a thorough

knowledge of the building construction and equipment profiles. In general these models are far

too complex to be used for evaluating DSM applications for a given building, let alone a group

of buildings that might be under the end-user’s control.

Power systems researchers are increasingly interested in how building loads can be

integrated in the electric grid through DSM. However, the majority this research has centered
 17
around the integration of bulk demand response by the independent system operator (ISO)

when scheduling system resources and performing system security analysis [51] [52] [53] [54]

[55]. It is assumed that building loads can play a role as DSM resources in these scenarios but

these methods include limited characteristics of actual building behavior. Further, no

information can be derived from these processes that would help the end-user achieve in practice

the demand modifications that would be required.

From the perspective of the end-user it is crucial to have a method of evaluating one’s

own potential for DSM activity; this includes models of building behavior and formal method of

determining when and how to dispatch loads. This is particularly true when demand

modifications are driven through control of the HVAC system given the potential for significant

impact to the comfort levels of the building. Modeling the behavior of thermostatically

controllable loads (TCL) for DSM has been studied in [14] [56] [57] [58]. While there is

potential applications for these models they are better suited for the aggregation of many

residential or small commercial loads [14] and overlook the fact that cycling HVAC systems

between on and off states is undesirable for reliability and efficiency reasons [59].

There remains a need for a model that captures the thermal-electrical dynamics of

commercial building loads with an eye towards controllability and simple implementation in a

DSM scheduling for the consumer. In this thesis such a model is developed in Chapter 5 using

methods applied previously for the development of power system dynamic load models and

again leveraging information collected from the BMS. Chapter 6 evaluates how this model,

combined with the building-specific forecasts from Chapters 3 and 4, can be used in a method of

planning commercial building load resources for DSM directed at the end-user.
 18

3. BUILDING-SPECIFIC LOAD FORECASTING STUDIES

3.1. OVERVIEW

In this chapter, the problem of building-specific load forecasting is studied. The main

focus is incorporating building-specific information into conventional load forecasting

techniques and evaluating the performance. This chapter includes the following:

 A brief overview of state-of-the-art load forecasting methods and the definition of

the load forecasting problem

 A case study where conventional forecasting models are trained on data collected

from the Drexel University BMS and a series of day-ahead load forecasts are

performed.

 Results from the case study are used to evaluate building-specific forecasting

performance and characterize the uncertainty in building-specific forecasts

3.2. LOAD FORECASTING METHOD REVIEW

Load forecasting is an essential process in electrical utility planning and operation.

Historically, load forecasting (in particular short term load forecasting (STLF) ) has played a

critical role in ensuring power system dispatchers schedule adequate generating capacity in the

most economical way possible. The importance of having access to accurate and consistent

methods of load forecasting is obvious. Consequently, a considerable amount of work has been

done to develop and examine methods of forecasting power system loads over varying time

horizons [60] [61]. In this thesis the focus is on day-ahead forecasts, which fall into the STLF

window.
 19
Load forecasting approaches generally fall into two categories: statistical approaches

such as classical time series analysis and regression–based models, and artificial intelligence

(AI) models such as artificial neural networks and fuzzy logic models [62]. Methods from both

of these categories have been applied to solving the STLF problem. A very thorough review and

critique of STLF techniques introduced prior to 2010 is presented in [63]. The popularity of AI

methods has led to a lot of research resources being devoted to applying newly developed AI

techniques to STLF. Far less attention is placed on applying modern statistical methods to this

problem. This is despite, as noted in [63], that statistical methods are much more widely used to

develop the candidate models employed by utilities.

An important contribution from [63] is the focus is not only on the variety of techniques

but also what predictor variables are employed in the forecasting methods and how said variables

can lead to conclusions regarding the causality of load consumption. Nearly all of the research

surveyed included some predictor variables and this thesis will consider their impact on

forecasting heavily. Predictor variables include environmental factors (“weather variables”),

lagged observations of demand or variables, as well as human or operational factors (“calendar

variables”). One of the conclusions is that successful inclusion of such variables depends on an

understanding of the geography of the power system under study and time frame of interest as

much as the method used to integrate these variables into a model. Without understanding this

behavior the chosen method will not successfully capture the link between demand and any

applied external variables.

In the years since the review in [63], the technical literature related to STLF has explored

additional methods for capturing the relationship between demand and additional predictor

variables. Artificial neural network models continue to be widely used owing to the inherent

capability of being able to learn and capture complex linear and nonlinear relationships from the
 20
data to be modeled [64]. Such models have been recently applied in a variety of forms [65]

[66] [67] [68] [69] [70] to take advantage of this property. Additional AI techniques such as

Support Vector Machines (SVM) and other supervised learning algorithms [71] [72] [73] [74]

have also been used to achieve suitable results. On the statistical method front, semi-parametric

additive models have recently shown very good results at both the transmission and distribution

system level [75] [76] [77] [78]. These works employ a regression-based structure with smooth,

nonlinear functions used to capture the link between demand and a number of covariates.

3.3. LOAD FORECASTING PROBLEM FORMULATION

The goal of any forecasting problem is to predict future observations of a particular

variable given a set of information about said variable. This information may include only

historical observations of the variable of interest (univariate case) or observations and predictions

of related predictor variables as well (multivariate case). There is an implicit assumption that

identified historical behavior will continue in the future.

For this work, we are interested in predicting the demand level p for a given building.

Although electric power is consumed by a building continuously, the observations are discrete

based on the sampling of the electric meter. Therefore, demand is represented by a discrete time-

series  pt  t 1,..,  where pt is the metered demand at time t and T is the collection of time

indices where (in general)    0 .

The general load forecasting problem can be defined as a function f of the available

information as shown in (3.1) and (3.2) below:

pˆ t k|t  f  Φt  (3.1)

where
 21
K: Forecast horizon
k 1,..., K : Time index within forecast horizon
pˆ t  k |t : Estimate of p given observations up to time t looking
k steps ahead
Φt : Collection of historical demand and predictor variables

with

Φt   pt , pt 1 ,..., pt  m , xt , xt 1 ,..., xt  m , xˆ t  k |t  (3.2)

where

m: Time in the past to which historical observations are included

pt  m : Historical observations of p from time m to time t
x: Set of all available predictor variables
xt  m : Historical observations of x from time m to time t
xˆ t  k |t : Estimate of x given observations up to time t looking
k steps ahead

Figure 3.1 shows a sample load forecast. In the plot, the division is shown between

historical observations pt-m (blue line) and estimated demand pˆ t k|t (grey line), also noting the

time indices as defined in above.

 22

Figure 3.1. Sample load forecast showing time indices

The forecast horizon K, also referred to as the look-ahead time in the forecasting

literature, defines how far into the future one is looking to forecast. If K > 1, the forecast is

called a multiple step ahead forecast. The estimated model remains fixed for the duration of the

forecast horizon. The predictor variable estimates xˆ t  k |t are used as inputs to the model in order

to generate the final forecast pˆ t k|t . An iterative method can also be used which effectively turns

the problem into a single step ahead forecast. At each step k, the estimate pˆ t k|t is found. This

estimate is then treated as a previous observation, becoming an input to the same model in order

to forecast the subsequent point. The process continues in this manner until reaching the end of

the forecast horizon. This approach is more often used for univariate forecasts and has the

disadvantage that the errors in the predicted values are accumulated into the next predictions.
 23
The predictor intervals contained in Φ will ideally include only variables that have a

causal relationship with the demand p. Identification of such variables can be done prior to

estimation of the function f by observing correlation plots such as the one shown in Figure 2.3.

Variable identification can also be done by training several forecast models with different

variables included in Φ, observing the forecast performance on a sample of actual demand

observations, and selecting the “best” model. In this thesis, two groups of predictor variables

will be used: one which includes building-specific measurements and one that does not. The

specifics of this are described in the model building procedure in subsection 3.4.3.

3.4. DREXEL UNIVERSITY CASE STUDIES

In order to test the idea of enhancing conventional load forecasting methods with

building-specific information, several STLF techniques will be used to forecast the day-ahead

demand of a building on Drexel University’s campus. The following subsections will describe

the data set used in this study, introduce the selected forecasting methods and describe the model

building procedure.

3.4.1. DATA SET DESCRIPTION

Data has been collected using the Drexel University building management system (BMS)

over the last several years. For the studies presented in this thesis, the data used is from the

summer months (May-August) of 2011, 2012, and 2014. As mentioned previously, the summer

weather conditions and large HVAC building load make these good months to study load

forecasting performance in support of demand side management activities. This case study will

focus on the Hagerty Library however the approach is generic to any similar commercial-type

building.
 24
The raw data includes four variables, all recorded at 5 minute intervals:

1. Building Demand (kW)

2. Outside Air Temperature (oF)

3. Indoor Air Temperature (oF)

4. Indoor Air Temperature Setpoint (oF)

The summer months (May-August) of 2011 and 2012 are used as the training set and

2014 held out to be used for out-of-sample testing. This distinction is shown in Figure 3.2

below. By training and testing the models on separate data sets the problem of overfitting can be

avoided. The training set can be broken into a sample used for estimating model parameters and

a sample used to validate these parameters. This is not required for all methods in this study.

Instances where this is the case will be discussed in the methods section 3.4.2.

Figure 3.2. Breakup of data set and how each portion is used in the forecasting process

Unfortunately the 2013 data had to be excluded from this study due to a large number of

bad measurements in this data set. These issues are attributed to BMS data collection

functionality problems. However, the building involved in this study is known to have

undergone no equipment upgrades or operational changes (University library). It is therefore

acceptable to leave out this data set for the purposes of this research effort.

Figure 3.3 below shows the daily building demand measurements for the test set portion

of the data (2014). The superimposed dark line represents the mean load profile for the test set.

It is these measurements against which the forecasts will be compared in this case study.
 25
Daily Building Demand (116 days) with Mean Profile
300

250

200
Demand (kW)

150

100

50
0 4 8 12 16 20 24
Hours

Figure 3.3. Daily building demand curves for the 116 days used for out of sample testing. The thicker line
shows the mean daily profile

3.4.2. SELECTED FORECASTING METHODS

This work examines the performance of four popular forecasting methods used in a

building-specific application. The selected methods were chosen to represent a broad cross

section of techniques. Detailed descriptions of each approach are well established in other works

but the general problem formulations are outlined below. Information specific to how these

problems are implemented in the chosen software packages is also discussed.

 MULTIPLE LINEAR REGRESSION (MLR)

A multiple linear regression model relates a dependent (or response) variable p to two or

more independent (or predictor) variables x. The general model is shown in (3.3) [79]:

pi  0  x1i 1  x2i 2  ...  xki k   i

(3.3)
i  1,..., N

where, for k predictor variables:

 26
N: Total number of observations
pi : Observation of the dependent variable p at time i
xki : Observation of predictor variable k at time i
k : Model parameter corresponding to the k th explanatory variable
i : Model error at time i

In many cases, the predictor variables are quantitative such as temperature or wind speed.

However, the formulation in (3.3) does not limit the predictor variables to quantitative ones.

Qualitative predictor variables, often referred to as indicator or dummy variables, can also be

included in the model and are particularly important in forecasting applications. Indicator

variables with values 0 and 1 can be used to identify the category of a quantitative variable, for

example indicating a weekday versus a weekend. A qualitative variable with c categories must

be represented by at most c-1 indicator variables. For example, a qualitative variable

representing the day of the week has 7 categories (i.e. - Sunday, Monday, … , Saturday) and is

represented by 6 indicator variables as shown in Table 3.1. Choosing to use c variables will

result in too many parameters to estimate and subsequent regression failure.

Table 3.1. Indicator variables representing the day of the week

Day X1 X2 X3 X4 X5 X6
Mon 1 0 0 0 0 0
Tue 0 1 0 0 0 0
Wed 0 0 1 0 0 0
Thur 0 0 0 1 0 0
Fri 0 0 0 0 1 0
Sat 0 0 0 0 0 1
Sun 0 0 0 0 0 0

The coefficients β represent the partial effect of one predictor variable when all others are

held constant [79]. In other words they represent the marginal effect of each predictor variable.

Given a set of training data, the β values are estimated using the ordinary least squares estimation

method. The models used in this thesis were implemented in “R” using the ‘lm’ function. This

package handles MLR model fitting as described here.

 27
 ARTIFICIAL NEURAL NETWORKS (NN)

Artificial neural networks (NN) are mathematical tools inspired by the way the human

brain processes information. The most basic computational unit of an NN is the neuron. The

neuron receives information, processes it internally, and provides a response. Figure 3.4 shows a

general schematic of an artificial neuron.

Figure 3.4. General artificial neuron model

In general, the information is processed in two stages. First, the input values are linearly

combined. Each value of the input array is associated with a weight value wi. An additional

input, a constant bias term θ, with a weight value equal to 1 is also applied. Second, this

combination becomes the argument of a non-linear activation function. There are a number of

possible functions but a very common choice (and the one implemented in this thesis) is the

logistic sigmoid function shown in (3.4).

1
f  x  (3.4)
1  e x

The organization of the neurons defines the architecture of the NN. A feed-forward

multilayer perceptron (MLP) neural network is a typical NN architecture employed for STLF. In

this architecture, the neurons are organized in layers where no neuron in a given layer is

connected to another neuron in the same layer, though they can share inputs. The term feed-
 28
forward means that the outputs of one layer become the inputs to the following layer. An

example N-layer feed-forward MLP NN is shown in Figure 3.5.

Figure 3.5. A general multi-layer, feed-forward artificial neural network with N hidden layers

The parameters of this network are the matrices of weights between each neuron and its

associated input. Input in this case also means connections between neurons in different layers.

Estimation of these weight matrices is referred to as “training” the network. The most widely

applied approach to training in STLF is supervised learning. For this approach, sets of inputs

and matched outputs are used as teaching patterns and the network weights that provide the best

fit between the network output and teaching output are found. Best fit is determined through

minimization of a loss function. The available training algorithms and loss functions are varied,

but historically the back-propagation method and mean squared error criterion are common in

STLF and will be used in this thesis.

The neural network used in this thesis is a three-layer feed-forward MLP implemented

using the Matlab Neural Network Toolbox. The network architecture and training method used

are as described in the preceding paragraphs. The model is used to forecast multiple steps ahead;

that is to say 288 point forecasts, corresponding to a single day-ahead forecast at 5 minute
 29
resolution. This results in a large neural network model that may be computationally

undesirable for implementation in a building level energy management system. However for the

purposes of this study it is sufficient and more computationally sensitive techniques for building

the models can be explored in future work.

One other item of note when constructing the NN model used in this work is that there

are no hard and fast rules guiding the selection of the number of hidden neurons [64]. The

hidden neurons are the neurons in the layer (or layers) between the input and output layers. The

model in this work uses 25 hidden neurons. This number was determined by estimating the

model on a portion of the training set as shown in Figure 3.2 with a varying the number of

neurons and observing the accumulated error on the validation set. Varying the number of

neurons did not appear to significantly affect the error results, which is not uncommon [64], and

25 was selected.

 SUPPORT VECTOR REGRESSION (SVR)

Support vectors and support vector machines (SVM) are a machine learning technique

used for data classification and regression [80]. Assume given training data (x1 , y1),…,(xn , yn),

where xi are the inputs and yi the corresponding outputs, the support vector regression solves the

following optimization problem in (3.5)-(3.8).

1 n

min * 2

wT w  C   i   i*  (3.5)
w , b , , i 1

subject to


yi  wT   xi   b     i*  (3.6)

w   x   b  y
T
i i
   i (3.7)
 30
i ,   0, i  1,.., n
i
*
(3.8)

where xi is mapped to a higher dimensional space by the function ϕ, i* is the upper

training error and  i is the lower limit subject to the δ-insensitive tube y   wT   x   b    .

The parameters which control regression quality are the error cost C, the width of the tube δ, and

the mapping function ϕ.

As noted in [81], since the function ϕ can map xi to a high or even infinite dimensional

space the dual of (2) is often solved instead.

1 n n

   *  Q    *      i   i* + yi  i   i* 
T

min 2
(3.9)
 , * i 1 i 1

subject to

  i 
  i*  0 (3.10)
i 1

0   i ,  i*  C , i  1,.., n (3.11)

where Qij= ϕ(xi)Tϕ(xj). To solve this inner product, which may be computationally

difficult, a “kernel trick” is implemented to do the mapping implicitly. In other words, the

application of special forms which are inner products in a higher dimensional space yet can be

calculated in the original space. There are many options for kernel functions but for this work

the radial basis function (RBF) kernel is used. The expression for this kernel is shown in (3.13).

  xi    x j   e  
2
T  x  x i j
(3.12)
 31
The RBF kernel can handle nonlinear relationships between xi and yi unlike the linear

kernel yet has fewer parameters than a polynomial kernel, reducing model complexity. This

kernel selection has been used to great effect in STLF applications [80] [73].

There are two key parameters that must be found when training the SVM models: the error

cost C and the RBF parameter γ. To decide the proper parameter values, the training set is

segmented such that the model performance can be evaluated. A portion of the training set is

used for updating the model parameters while the validation set is used to observe the

corresponding model performance. To determine suitable values for C and γ, v-fold cross

validation [82] is performed by dividing the training set into v equally sized subsets. With one

subset held out for validation, the model is trained on the other v-1 subsets. This process is

repeated using each subset as a validation set and the model performance aggregated. In this

thesis the models are implemented using the LIBSVM extension in R [81]. This package

automatically handles the parameter estimation procedure by efficiently solving (3.9) while

performing the CV procedure.

 GENERALIZED ADDITIVE MODELS (GAM)

In a manner similar to the regression structure established for the multiple linear

regression model, the semi-parametric additive models capture nonlinear relationships using the

framework established in [83].

The general statistical model is shown in (3.13):

pi  f1  x1i   f 2  x2 i   ...  f k  xki    i ,

(3.13)
i  1,..., N

where, for k predictor variables:

 32
pi : Observation of the dependent variable p at time i
N: Total number of observations
xki : Observation of predictor variable k at time i
fk : Smooth, nonlinear function related to predictor variable k
i : Model error at time i

The functions fk are non-linear, smooth functions that can be well estimated from

observed data. These functions can be multivariate (e.g. – f(x1,x2) ) but for this thesis they will

only be a function of a single variable. It is common for these functions to be estimated via

penalized regression in a spline basis. If bi(x) is the ith basis function, then f can be represented

as in (3.14).

d
f k  xk    bi  xk   i (3.14)
i 1

where d is the dimension of the spline basis and bi(xk) are the corresponding spline

functions. There are many possible spline functions available. This work uses cubic regression

splines as was applied in previous STLF works [ [75] [76]]. In order to estimate f a penalized

regression problem is solved via (3.15):

kv
min p  Bβ   q βT S q β
2

β ,
(3.15)
q 1

where

kv : Total number of predictor variables

p: All collected observations of the dependent variable p
B: Matrix formed by concatenation of the spline functions
bi  x 
β: Vector of the the unknown regression parameters 
q : Smoothing paramter parameter
S: Smoothing matrix of known coefficients
 33
The solution to this problem will attempt to balance the tradeoff between model fit and

model smoothness. The models used in this work are implemented in R using the mgcv package

[84]. The problem in (3.15) is solved using the methodology presented in [85] [86] which

involves minimization of the Generalize Cross Validation (GCV) criteria. This is done

automatically in mgcv with the appropriate spline function and training set specified.

3.4.3. MODEL BUILDING AND FORECASTING PROCEDURE

The overall STLF process is summarized in Figure 3.6. (Modified from [63]).

Figure 3.6. Diagram of the STLF process

The first step is to use historical information to estimate the model. The predictor

variables contained in Φ includes the data required to estimate the necessary model parameters.

When training each model, the considered predictor variables are shown in Table 3.2.
 34
Table 3.2. General and Building-specific predictor variables

For each forecasting method tested in this thesis, two models are built: one which

includes the general variables and building-specific variables and one that only includes the

general variables. The 2011 and 2012 data is used for model estimation. For those forecasting

methods that use a validation set (refer to section 3.4.2 for details), the August 2012 data is set

aside for model validation.

Once the models are built the forecasts can be generated. Each model is used to forecast

the day-ahead demand for the 2014 test set. The forecasts are generated in 5 minute steps

matching the sample rate of the historical data. A total of 116 day-ahead forecasts were

performed (May 8th - Aug. 31st 2014). The first week of May was excluded as no historical

lagged measurement observations were available prior to May 1st to support forecast generation.

In real-world forecasting applications the forecasts are generated by feeding estimates of

the predictor variables xˆ t  k |t into the model. For example, a weather forecast for the next day

would be used as an input as opposed to actual temperature information. Using only information

known in advance, these forecasts are known as ex-ante forecasts. These are the only “true”

forecasts since all future information is unknown. In this thesis, because the goal is model
 35
analysis, estimates of the predictor variables xˆ t  k |t are replaced with observed values xt  k |t , and

the generated forecast pˆ t  k |t is compared to the known value pt  k |t . These types of forecasts are

known as ex-post forecasts and are useful in demonstrating the potential accuracy of a model and

are the standard in the forecasting literature.

3.4.4. FORECAST PERFORMANCE EVALUATION FRAMEWORK

Performance evaluation focuses on three questions: did the building-specific

measurements improve the forecast accuracy, how consistent is this performance, and how does

the forecast uncertainty change when including building information. Most of the forecasting

literature is focused solely on the forecast accuracy. The majority of papers follow the same

general formula: develop a forecast model and then precede to use an accuracy metric that

demonstrates how great a particular model is for that given test. This type of analysis is fine but

a bit limited. It struggles to address consistency of the forecast performance and provides little

to no characterization of the forecast errors. The forecast performance evaluation framework in

this thesis will address not only the overall accuracy for the forecasts but focus particularly on

the performance consistency.

Forecast error is defined as the difference between the predicted power value and the

actual power, as shown in (3.16).

et k|t  pt k|t  pˆ t k|t (3.16)

The error is composed of a systematic part te k |t and a random part te k |t .

et  k |t  te k |t  te k |t (3.17)

Ideally, the systematic error is zero and the random part is white noise (zero mean,

Gaussian random variable). However, in practice these conditions are rarely the case and
 36
examining each part of the error is necessary to understand the impact of building

measurements on the forecast performance.

The standard metric for defining the forecast accuracy in the load forecasting literature is

the Mean Absolute Percent Error (MAPE), shown in (3.18).

 1 N et k |t 
MAPE     100 (3.18)
 N k 1 pt  k |t 
 

where N is the number of forecasted points in the interval of interest. By varying the timeframe

over which the MAPE is calculated certain behaviors of the forecast method of interest can be

observed. This metric is an overall accuracy measure and both the systematic and random errors

contribute to the calculated value.

Another basic metric is the forecast method’s bias, given by the mean error over a

specified interval.

N
1
bias  ke 
N
e
k 1
t  k |t (3.19)

where N is the number of forecasted points in the interval of interest. The bias shows if the

method tends to under- or overestimate the forecast. It corresponds to the systematic part of the

forecast error et+k|t. Ideally the forecasts will be unbiased but in practice this is often unrealistic.

The variability of the forecast performance can observed by calculating the standard

deviation of the errors.

1 N
  et  k |t  ke 
2
 ke  (3.20)
N k 1

where N is the number of forecasted points in the interval of interest. The standard deviation

corresponds to the random part of the forecast error et+k|t.

 37
In addition to the MAPE (3.18), bias (3.19), and standard deviation (3.20) metrics a

distribution-based approach to performance evaluation will be employed. Histograms of the

forecast errors represent the empirical distributions of these errors. This type of analysis is

important for characterizing the consistency of the forecast performance by answering the

question “How often does a given forecasting method result in a specific error level?”. For

example, two methods might result in nearly identical MAPE values over a given time window

but have radically different error distribution shapes, thus different frequencies of large errors.

Evaluating the moments of the forecast error distributions will shed light on several important

characteristics:

 The mean ke is a measure of the central tendency of the error distribution. As mentioned

above, this is equivalent to the bias metric.

 The standard deviation  ke reflects the dispersion of the distribution.

 The skewness ske describes the lack of symmetry of a distribution, indicating the most

likely direction of expected forecast errors. The skewness is calculated using (3.21) [87]:

N 3
1
N
 e k  t |t  e
k 
s 
e
k
k 1
(3.21)
N 3
1
N
 e
k 1
k  t |t  e
k 

 The kurtosis represents the “tailedness” of the distribution, indicating the propensity for

outliers in the forecast errors. The kurtosis is calculated using (3.21) [87]:
 38
4
1 N
  ek t|t  ke 
N k 1
 ke  2 (3.22)
1 N 2

   k t|t   k 
e
e

 N k 1 

Error margin plots will be used to shed more light on the error histograms. These plots

show what proportion of forecast errors fall above a certain threshold for a given window of

time. This will help answer the question of how often the forecasts result in unacceptable errors

and if building measurements can improve this performance.

The following subsection will use the above metrics and distribution-based methods to

evaluate the 116 day-ahead building-specific load forecasts. The forecasts correspond to 33408

total point predictions. Comparisons are made between the models that include building

measurements and those that do not. The forecasts are observed over several different time

windows to try and better characterize the performance.

3.4.5. RESULTS

Before taking a more detailed look at the forecast performance it is typical of forecasting

studies to observe the overall accuracy results. First, the day-ahead MAPE for each of the 116

forecasts is calculated. These results are summarized in Table 3.3.

Table 3.3. Quantiles of day-ahead MAPE performance across all out-of-sample forecasts
MLR NN SVM SAM
quantile No Bldg Bldg No Bldg Bldg No Bldg Bldg No Bldg Bldg
10% 3.06 2.87 3.88 3.58 2.72 2.53 3.36 3.04
25% 3.82 3.64 4.74 4.25 3.15 2.97 4.01 3.94
50% 5.03 4.52 5.65 5.36 4.18 4.08 5.34 5.29
75% 6.74 5.93 8.55 6.68 6.44 6.39 7.72 7.62
90% 8.75 7.49 11.99 9.02 9.26 9.60 11.78 10.86
 39
The universal improvement when building-specific measurements are included is

apparent for all 4 forecasting methods. The only deviation is for the SVM model in the 90th

percentile of forecasted days. It is important to note that the MAPE values of the models without

building measurements are already consistently good (and in some cases excellent) by today’s

forecasting standards. Improving upon a poorly performing model by feeding it new variables

would say little about the actual impact. That the models with building information consistently

outperformed the other set in this case study demonstrates the predictive capacity of these

variables.

Other breakdowns of forecast performance are shown in Table 3.4 and Table 3.5. Table

3.4 shows the MAPE calculated on a monthly basis. Table 3.5 shows the MAPE calculated for

weekdays and weekends/holidays.

Table 3.4. Monthly MAPE (%) performance

MLR NN SVM SAM
Month No Bldg Bldg No Bldg Bldg No Bldg Bldg No Bldg Bldg
May 9.04 8.78 10.91 8.60 8.87 8.86 11.40 9.55
June 5.42 5.40 6.40 5.29 4.67 4.80 5.96 5.90
July 4.89 4.93 5.91 5.03 4.19 3.99 5.93 5.49
August 3.92 4.00 5.48 5.00 3.81 3.68 4.76 4.45

Table 3.5. MAPE (%) performance broken up by weekdays and weekends/holidays

MLR NN SVM SAM
No Bldg Bldg No Bldg Bldg No Bldg Bldg No Bldg Bldg
Wkdy 5.75 5.68 6.98 5.58 5.29 5.15 6.69 6.06
Wknd/Hol. 5.34 5.42 6.89 6.44 4.91 5.05 6.91 6.40

The improvement when building-specific measurements are included is again

demonstrated for all methods. It is worth noting the improvement in July and August vs. May

and June. Although classes at Drexel University are in session year-round, the period between
 40
July and September tends to have fewer people on campus. The forecast improvement during

these months could be attributed to diminished volatility of occupancy effects that are more

pronounced in May and June.

One of the objectives of this thesis is to study a tool for improving end-user energy

management decisions. While the results in Table 3.3 – Table 3.5 indicate solid performance

improvement, good daily or monthly performance is not sufficient to confidently inform DSM

operations and could in fact be misleading. The performance during hourly (or even shorter)

intervals must be explored to provide value for demand side management planning.

For this study the forecasts have been generated in 5 minute steps as stated previously.

The error statistics presented going forward in this chapter are aggregated as to be reported

hourly. The decision to present error statistics hourly is motivated by typical settlement period

lengths for day-ahead demand response programs [23]. This decision will also make

observations of the forecast performance a little more straightforward, but no less accurate, with

a condensed set of results. A full set of results at 5 minute resolution can be found in Appendix

A2.

Table 3.6 shows the hourly MAPE for all 4 methods across all 116 day-ahead forecasts.

When looking at the hourly performance we again see an overall measurable improvement when

building measurements are included in the models. There are also a number of additional

observations that can be made about the consistency of forecast performance from these results.
 41
Table 3.6. Hourly MAPE (%) across all out-of-sample forecasts
MLR NN SVM SAM
Hour No Bldg Bldg No Bldg Bldg No Bldg Bldg No Bldg Bldg
1 5.16 5.12 8.06 6.54 5.49 5.55 7.03 6.31
2 5.37 5.33 7.85 5.62 6.86 6.81 7.51 6.62
3 8.53 8.61 8.61 7.95 6.47 6.24 10.80 9.39
4 8.37 8.56 12.50 8.08 6.72 7.02 11.52 10.16
5 8.18 8.26 10.23 6.85 6.11 6.35 10.56 9.38
6 10.01 9.61 10.38 7.73 8.66 9.18 10.01 10.31
7 6.55 6.61 6.77 6.14 5.95 5.93 7.26 6.66
8 5.27 5.31 6.43 5.23 5.09 5.17 6.05 5.76
9 5.47 5.46 6.94 5.52 6.30 6.15 6.19 6.17
10 5.25 5.19 5.96 5.40 5.65 5.55 5.94 5.86
11 5.00 4.89 5.99 5.25 5.49 5.23 5.42 5.43
12 4.93 4.84 6.37 4.89 4.87 4.84 5.62 5.36
13 4.41 4.33 5.64 4.81 4.43 4.60 5.06 4.86
14 4.43 4.42 6.24 5.12 4.08 4.01 5.05 4.70
15 4.89 4.71 5.59 4.70 3.99 3.83 5.30 5.03
16 5.06 4.86 5.77 5.06 4.20 3.93 5.41 5.13
17 4.89 4.78 5.68 5.64 4.23 3.88 5.58 4.99
18 4.56 4.53 5.06 5.11 4.00 3.35 5.24 4.61
19 4.41 4.22 4.83 6.98 4.15 3.72 5.32 4.64
20 4.68 4.48 4.83 6.63 4.13 3.87 5.75 5.03
21 4.57 4.62 5.05 4.77 3.96 3.85 5.80 4.85
22 4.98 5.30 7.62 5.18 4.41 4.48 6.51 5.27
23 5.09 5.36 8.25 5.20 4.38 4.49 6.62 5.46
24 4.98 5.10 6.27 5.52 4.70 4.94 6.58 5.78

Table 3.6 describes the day-ahead MAPE on an hourly interval. The hourly MAPE is

generally higher during the early morning hours. This is a time of lower building demand (see

Figure 3.3) so any error in the forecast will reflect more heavily on a percentage metric. During

a number of these early morning time intervals the MLR and SVM models actually perform

slightly better without the building measurements included. This is in contrast to the neural

network and additive models, and also to how all of the models generally behave over the rest of

the day.

The hourly MAPE values tend to drop as the day progresses and the building demand

increases considerably (Figure 3.3). More importantly, during this time window the models

which include building measurements improve the forecast in nearly all intervals, for all
 42
methods, and by a greater amount. This is important for DSM planning purposes. This is the

time window when building demand resources will most often be dispatched and having a better

estimate of the building load is essential. Figure 3.7 - Figure 3.10 below show the hourly MAPE

for each forecasting method to visualize the results from Table 3.6.

13
W/O Bldg Meas.
12 WITH Bldg Meas

11

10

9
MAPE (%)

3
0 5 10 15 20 25
Hours

Figure 3.7. Hourly MAPE for all forecasts generated by the MLR models
 43
13
W/O Bldg Meas
12 WITH Bldg Meas

11

10

9
MAPE (%)

3
0 5 10 15 20 25
Hours

Figure 3.8. Hourly MAPE for all forecasts generated by the NN models

13
W/O Bldg Meas
12 WITH Bldg Meas

11

10

9
MAPE (%)

3
0 5 10 15 20 25
Hours

Figure 3.9. Hourly MAPE for all forecasts generated by the SVM models
 44
13
W/O Bldg Meas
12 WITH Bldg Meas

11

10

9
MAPE (%)

3
0 5 10 15 20 25
Hours

Figure 3.10. Hourly MAPE for all forecasts generated by the SAM models

To better understand the inter-hour and intra-hour forecast accuracy variability it is

helpful to visualize the range of MAPE results on an hourly basis. The box plots in Figure 3.11 -

Figure 3.14 look at the hourly MAPE across all out-of-sample forecasts. Each box plot displays

a notched mark at the median value with the box edges showing the 25th and 75th percentiles

and the whiskers capturing the most extreme results. These plots help identify during what

intervals of the day the models perform less consistently (in terms of MAPE values) and how this

consistency varies throughout the day.

 45
Multiple Linear Regression Model without Building Measurements
35
30
25

MAPE (%)
20
15
10
5
0
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24
Hour

Multiple Linear Regression Model with Building Measurements

35
30
25
MAPE (%)

20
15
10
5
0
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24
Hour

Figure 3.11. Box plots of hourly MAPE results. (top) MLR without building measurements (bottom) MLR
with building measurements

Neural Network Model without Building Measurements

35
30
25
MAPE (%)

20
15
10
5
0
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24
Hour

Neural Network Model with Building Measurements

35
30
25
MAPE (%)

20
15
10
5
0
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24
Hour

Figure 3.12. Box plots of hourly MAPE results. (top) NN without building measurements (bottom) NN with
building measurements
 46
Support Vector Machine Model without Building Measurements
35
30
25

MAPE (%)
20
15
10
5
0
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24
Hour

Support Vector Machine Model with Building Measurements

35
30
25
MAPE (%)

20
15
10
5
0
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24
Hour

Figure 3.13. Box plots of hourly MAPE results. (top) SVM without building measurements (bottom) SVM
with building measurements

Semi-parametric Additive Model without Building Measurements

35
30
25
MAPE (%)

20
15
10
5
0
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24
Hour

Semi-parametric Additive Model with Building Measurements

35
30
25
MAPE (%)

20
15
10
5
0
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24
Hour

Figure 3.14. Box plots of hourly MAPE results. (top) SAM without building measurements (bottom) SAM
with building measurements

The plots in Figure 3.11 - Figure 3.14 reinforce the results from Table 3.6. Including the

building measurements generally improves the forecast performance overall with a couple

exceptions. Again, the hourly MAPE performance is more variable in the early morning hours
 47
for all models and improves later in the day. Of the four methods studied in this work, the

neural network model overall performance is the most greatly improved with building

measurements included in the model. This is especially true in terms of reducing the day-ahead

forecast variability. The SVM models are generally better when BMS measurements are

included but far less consistently compared to the other approaches. It would be interesting in

future work to try and identify what methods can be employed to optimize how building-specific

measurements can be included in forecast models. The variability in Figure 3.11 - Figure 3.14

will be discussed further in regards to the standard deviation of errors below.

While MAPE is the most commonly applied metric for quantifying forecast accuracy, it

is also helpful to look at other error metrics. Table 3.7 shows the hourly bias for all 4 methods

across all 116 day-ahead forecasts. As stated earlier, the ideal bias value is zero.
 48
Table 3.7. Hourly bias (kW) across all out-of-sample forecasts
MLR NN SVM SAM
Hour No Bldg Bldg No Bldg Bldg No Bldg Bldg No Bldg Bldg
1 ‐2.53 ‐2.97 ‐7.15 ‐8.06 ‐5.50 ‐6.23 ‐1.49 ‐1.02
2 ‐2.67 ‐3.08 ‐7.61 0.59 ‐8.94 ‐9.06 ‐1.20 ‐1.07
3 ‐2.03 ‐1.01 0.04 4.53 0.20 ‐1.17 2.10 ‐0.35
4 ‐3.26 ‐0.66 ‐12.57 ‐5.76 ‐3.47 ‐3.98 2.59 ‐2.41
5 ‐4.85 ‐2.93 ‐7.32 ‐1.93 ‐1.89 ‐2.64 1.17 ‐2.78
6 ‐6.50 ‐6.57 ‐2.65 2.93 ‐5.03 ‐7.63 ‐2.78 ‐4.72
7 0.77 0.59 1.75 3.08 ‐1.28 ‐1.94 2.35 1.08
8 ‐0.56 ‐0.70 ‐0.30 1.26 ‐4.42 ‐4.95 0.87 ‐0.50
9 ‐1.73 ‐1.69 ‐3.85 ‐2.16 ‐5.64 ‐5.78 0.57 ‐0.81
10 ‐0.78 ‐0.64 ‐0.57 2.27 ‐4.22 ‐4.73 1.80 0.98
11 ‐2.10 ‐2.25 ‐0.81 ‐0.19 ‐5.42 ‐5.35 0.56 0.28
12 1.08 0.74 ‐6.88 2.42 ‐3.68 ‐4.34 3.48 3.01
13 0.91 0.11 ‐1.67 3.42 ‐3.63 ‐5.39 2.70 2.51
14 1.81 0.67 ‐3.39 3.84 ‐2.75 ‐3.83 3.69 3.27
15 3.40 1.67 0.46 3.10 0.38 ‐1.50 4.80 5.17
16 4.24 2.15 ‐2.79 6.63 0.98 ‐0.71 5.45 5.98
17 3.70 1.32 ‐3.33 8.51 2.06 0.47 4.63 5.26
18 2.76 0.23 ‐3.50 6.68 3.26 0.53 3.21 3.63
19 4.10 1.74 ‐2.06 12.39 5.20 3.29 3.79 4.77
20 3.65 1.79 ‐1.41 10.41 3.82 2.67 3.06 4.31
21 0.76 ‐1.00 ‐4.29 3.58 ‐0.39 ‐1.00 ‐0.14 1.29
22 ‐2.17 ‐4.15 ‐12.89 3.17 ‐3.07 ‐3.85 ‐3.57 ‐1.55
23 ‐3.89 ‐4.91 ‐13.94 ‐2.91 ‐2.25 ‐3.35 ‐4.35 ‐2.99
24 ‐3.71 ‐4.46 ‐9.54 2.83 ‐3.00 ‐4.76 ‐3.82 ‐2.46

Figure 3.15 - Figure 3.18 below show the hourly bias for each forecasting method to

visualize the results from Table 3.7.

 49
15
W/O Bldg Meas
WITH Bldg Meas

10

Bias (kW) 5

-5

-10

-15
0 5 10 15 20 25
Hours

Figure 3.15. Hourly Bias for all forecasts generated by the MLR models

15
W/O Bldg Meas
WITH Bldg Meas

10

5
Bias (kW)

-5

-10

-15
0 5 10 15 20 25
Hours

Figure 3.16. Hourly Bias for all forecasts generated by the NN models
 50
15
W/O Bldg Meas
WITH Bldg Meas

10

Bias (kW) 5

-5

-10

-15
0 5 10 15 20 25
Hours

Figure 3.17. Hourly Bias for all forecasts generated by the SVM models

15
W/O Bldg Meas
WITH Bldg Meas

10

5
Bias (kW)

-5

-10

-15
0 5 10 15 20 25
Hours

Figure 3.18. Hourly Bias for all forecasts generated by the SAM models

These results do not provide any conclusive evidence that building measurements affect

the bias of the forecasts. In fact, the shape of the bias as it evolves over time is basically the
 51
same whether building measurements are included or not. The only exception to this is a few

intervals for the neural network models. This implies that the bias is a function of the model

building process more so than a function of the input variables. It is also interesting that the

jump in bias for the NN model without building measurements seen between 18:00 and 21:00 in

Figure 3.16 corresponds to the diminished accuracy during the same time period seen in the

hourly MAPE plot Figure 3.8.

The next metric presented in this section is the standard deviation of the forecast errors

(SDE). This metric is very important for understanding the consistency of the forecast

performance. Table 3.8 shows the hourly SDE for all 4 methods across all 116 day-ahead

forecasts.

Table 3.8. Hourly SDE (kW) Across all out-of-sample forecasts

MLR NN SVM SAM
Hour No Bldg Bldg No Bldg Bldg No Bldg Bldg No Bldg Bldg
1 10.34 10.08 15.44 10.61 9.80 9.51 14.71 12.89
2 10.96 10.77 14.77 11.62 10.27 9.95 15.88 13.57
3 10.36 11.22 13.32 10.70 10.59 10.03 16.15 14.02
4 11.70 12.55 13.25 10.17 9.81 10.19 16.98 15.24
5 11.25 11.69 14.40 10.41 9.55 9.64 16.56 14.66
6 13.58 13.27 16.11 13.28 13.28 12.64 16.18 16.13
7 12.43 12.42 13.04 11.87 11.18 10.99 14.11 13.06
8 11.38 11.37 14.10 11.48 10.16 9.85 13.07 12.18
9 12.54 12.40 15.39 12.08 12.95 12.50 14.19 13.63
10 12.79 12.70 14.58 12.53 13.17 12.57 14.13 13.53
11 12.05 11.78 14.35 12.65 13.18 12.63 13.62 13.11
12 13.24 13.09 14.28 12.61 12.24 11.91 15.73 13.51
13 11.70 11.65 14.02 11.80 11.09 10.71 13.71 12.47
14 11.75 11.73 15.01 12.52 10.33 9.89 13.52 12.13
15 12.93 12.79 14.64 12.00 10.65 10.31 14.28 12.99
16 13.10 13.05 14.82 11.73 11.00 10.49 14.43 13.04
17 13.00 12.99 14.48 12.16 11.10 10.29 14.90 13.14
18 12.35 12.30 12.51 11.55 10.46 9.37 14.38 12.30
19 11.22 11.23 11.97 10.84 9.91 9.29 13.36 11.34
20 11.70 11.71 12.22 11.61 10.23 9.88 14.09 11.90
21 11.45 11.48 12.29 10.89 10.07 9.66 13.74 11.77
22 11.74 11.79 11.22 11.87 10.15 9.98 14.34 12.17
23 11.23 11.27 10.95 12.10 10.27 10.20 13.90 12.12
24 10.67 10.62 11.10 12.20 10.28 10.14 14.01 12.69
 52

Figure 3.19 - Figure 3.22 below show the hourly SDE for each forecasting method to

visualize the results from Table 3.8.

17
W/O Bldg Meas
WITH Bldg Meas
16

15
Standard Deviation (kW)

14

13

12

11

10

9
0 5 10 15 20 25
Hours

Figure 3.19. Hourly standard deviation of the forecast error for all forecasts generated by the MLR models
 53
17
W/O Bldg Meas
WITH Bldg Meas
16

15

Standard Deviation (kW)

14

13

12

11

10

9
0 5 10 15 20 25
Hours

Figure 3.20. Hourly standard deviation of the forecast error for all forecasts generated by the NN models

17
W/O Bldg Meas
WITH Bldg Meas
16

15
Standard Deviation (kW)

14

13

12

11

10

9
0 5 10 15 20 25
Hours

Figure 3.21. Hourly standard deviation of the forecast error for all forecasts generated by the SVM models
 54
17
W/O Bldg Meas
WITH Bldg Meas
16

15

Standard Deviation (kW)

14

13

12

11

10

9
0 5 10 15 20 25
Hours

Figure 3.22. Hourly standard deviation of the forecast error for all forecasts generated by the SAM models

Unlike with the bias, the building measurements have a definite overall positive impact

on the SDE. This improvement is most pronounced in the NN models where there is a

significant decrease over the majority of the day. The SDE for the SVM and SAM models also

decreases but more substantially later in the day than during the morning hours. Only the MLR

models showed no meaningful difference.

The next step is to analyze the shape of the error distributions. As an example, the

following two histograms show the forecast errors during the 09:00-10:00 time window for the

neural network models. The model in Figure 3.23 does not include building measurements while

the model in Figure 3.24 does include building measurements.

 55
Neural Network Model without Building Measurments
140

120

100
Number of Events

80

60

40

20

0
-80 -60 -40 -20 0 20 40 60 80
Error (kW)

Figure 3.23. Empirical distribution of forecast errors for the NN model without building measurements
included (09:00-10:00)

Neural Network Model with Building Measurments

140

120

100
Number of Events

80

60

40

20

0
-80 -60 -40 -20 0 20 40 60 80
Error (kW)

Figure 3.24. Empirical distribution of forecast errors for the NN model with building measurements
included (09:00-10:00)
 56
The overall change in shape of the distribution is noticeable: the tail ends are `reduced

in both directions in Figure 3.24, as are a number of the peaks. All hourly plots of the empirical

distributions for each forecast model are located in Appendix A2. It is not necessary to show all

the distributions here as the characteristics of these distributions will be measured in several

ways below.

First, the error margin plots corresponding to two error thresholds are shown in Figure

3.25 - Figure 3.26. The definition of “unacceptable error” is subjective and must be determined

on a case by case basis. This study will look at the 10kW and 30kW thresholds. These plots are

shown in Figure 3.25 - Figure 3.32 below.

80
W/O Bldg Meas
WITH Bldg Meas
75

70

65

60
Frequency (%)

55

50

45

40

35

30
0 5 10 15 20 25
Hours

Figure 3.25. Frequency of forecast errors within a ±10 kW error margin for the MLR models
 57
100
W/O Bldg Meas
WITH Bldg Meas

98

Frequency (%) 96

94

92

90

88
0 5 10 15 20 25
Hours

Figure 3.26. Frequency of forecast errors within a ±30 kW error margin for the MLR models

80
W/O Bldg Meas
WITH Bldg Meas
75

70

65

60
Frequency (%)

55

50

45

40

35

30
0 5 10 15 20 25
Hours

Figure 3.27. Frequency of forecast errors within a ±10 kW error margin for the NN models
 58
100
W/O Bldg Meas
WITH Bldg Meas

98

Frequency (%) 96

94

92

90

88
0 5 10 15 20 25
Hours

Figure 3.28. Frequency of forecast errors within a ±30 kW error margin for the NN models

80
W/O Bldg Meas
WITH Bldg Meas
75

70

65

60
Frequency (%)

55

50

45

40

35

30
0 5 10 15 20 25
Hours

Figure 3.29. Frequency of forecast errors within a ±10 kW error margin for the SVM models
 59
100
W/O Bldg Meas
WITH Bldg Meas

98

Frequency (%) 96

94

92

90

88
0 5 10 15 20 25
Hours

Figure 3.30. Frequency of forecast errors within a ±30 kW error margin for the SVM models

80
W/O Bldg Meas
WITH Bldg Meas
75

70

65

60
Frequency (%)

55

50

45

40

35

30
0 5 10 15 20 25
Hours

Figure 3.31. Frequency of forecast errors within a ±10 kW error margin for the SAM models
 60
100
W/O Bldg Meas
WITH Bldg Meas

98

Frequency (%) 96

94

92

90

88
0 5 10 15 20 25
Hours

Figure 3.32. Frequency of forecast errors within a ±30 kW error margin for the SAM models

The error margin plots reflect similar performance as in the previous tests. The MLR

models perform nearly identically regardless of the inclusion of building measurements. The

improvement is most pronounced in the NN models where there is a significant increase in

frequency at both the 10kW and 30kW thresholds. One observation of note is a sharp decrease

in frequency at the 10kW threshold in Figure 3.27, corresponding to the jump in bias for the NN

model with building measurements seen previously between 18:00 and 21:00. Again the SVM

and SAM models also perform better with building measurements but more substantially later in

the day than during the morning hours.

The next metric presented in this section is the skewness of the forecast error

distributions. Table 3.9 shows the hourly skewness for all 4 methods across all 116 day-ahead

forecasts.
 61
Table 3.9. Hourly skewness across all out-of-sample forecasts
MLR NN SVM SAM
Hour No Bldg Bldg No Bldg Bldg No Bldg Bldg No Bldg Bldg
1 -0.84 -0.74 -0.97 -0.84 -0.75 -0.90 0.41 -0.41
2 -0.95 -0.89 -0.99 -0.92 -0.42 -0.64 0.53 -0.20
3 0.00 -0.25 -0.39 -0.04 -0.01 -0.26 -0.14 -0.39
4 -0.13 -0.05 -1.19 -0.22 -0.69 -0.75 -0.02 -0.60
5 0.03 0.30 -1.02 0.40 0.10 -0.10 0.39 -0.59
6 -0.11 -0.12 -0.16 -0.46 0.61 0.62 0.26 -0.46
7 -0.55 -0.54 -0.10 0.11 -0.20 -0.25 0.06 -0.44
8 -0.38 -0.33 0.30 -0.03 -0.02 -0.12 0.30 -0.52
9 0.26 0.31 -0.32 0.27 -0.08 -0.22 0.49 -0.11
10 0.15 0.20 0.51 0.06 -0.02 -0.01 0.17 -0.32
11 -0.43 -0.41 -0.57 -0.70 -1.18 -1.22 -0.10 -0.84
12 0.51 0.59 0.33 -0.18 -0.14 -0.04 1.23 -0.18
13 -0.23 -0.15 -0.22 -0.53 -0.71 -0.61 0.31 -0.32
14 -0.12 -0.03 -0.02 -0.47 -0.47 -0.55 0.33 -0.09
15 0.03 0.15 -0.13 -0.22 -0.34 -0.59 0.50 0.24
16 -0.07 0.10 0.01 -0.38 -0.71 -0.65 0.47 0.13
17 0.04 0.15 -0.14 -0.45 -0.47 -0.52 0.49 0.23
18 -0.03 0.16 -0.20 -0.29 0.04 -0.26 0.56 0.08
19 -0.08 0.02 0.06 -0.72 0.12 -0.11 0.27 -0.14
20 -0.91 -0.78 -0.84 -1.65 -0.74 -1.11 -0.36 -0.75
21 -0.66 -0.54 -0.85 -0.99 -0.70 -0.99 -0.21 -0.54
22 -0.50 -0.35 -0.51 -1.00 -0.40 -0.68 0.02 -0.37
23 -0.46 -0.29 -0.19 -1.20 -0.35 -0.53 0.11 -0.24
24 -0.66 -0.54 -0.89 -1.01 -0.12 -0.37 0.06 -0.10

Figure 3.33 - Figure 3.36 below show the hourly skew for each forecasting method to

visualize the results from Table 3.9.

 62
1.5
W/O Bldg Meas
WITH Bldg Meas
1

0.5

0
Skew

-0.5

-1

-1.5

-2
0 5 10 15 20 25
Hours

Figure 3.33. Hourly skew of the forecast error for all forecasts generated by the MLR models

1.5
W/O Bldg Meas
WITH Bldg Meas
1

0.5

0
Skew

-0.5

-1

-1.5

-2
0 5 10 15 20 25
Hours

Figure 3.34. Hourly skew of the forecast error for all forecasts generated by the NN models
 63
1.5
W/O Bldg Meas
WITH Bldg Meas
1

0.5

0
Skew

-0.5

-1

-1.5

-2
0 5 10 15 20 25
Hours

Figure 3.35. Hourly skew of the forecast error for all forecasts generated by the SVM models

1.5
W/O Bldg Meas
WITH Bldg Meas
1

0.5

0
Skew

-0.5

-1

-1.5

-2
0 5 10 15 20 25
Hours

Figure 3.36. Hourly skew of the forecast error for all forecasts generated by the SAM models

Much like the bias measurement, there is little that can be drawn from the skewness of

the error distributions. The evolution of skewness over the course of the day is roughly the same

whether or not the forecast models included building measurements. One interesting observation
 64
from Figure 3.36 is that the SAM model without building measurements tends to overestimate

the load while the SAM model with building measurements tends to underestimate the load, both

by roughly the same amount.

The last metric presented in this section is the kurtosis of the forecast error distributions.

If the errors are normally distributed then the kurtosis is equal to 3. Table 3.10 shows the hourly

kurtosis for all 4 methods across all 116 day-ahead forecasts.

Table 3.10. Hourly kurtosis across all out-of-sample forecasts

MLR NN SVM SAM
Hour No Bldg Bldg No Bldg Bldg No Bldg Bldg No Bldg Bldg
1 4.08 3.88 5.62 4.77 4.46 5.06 4.15 3.56
2 4.76 4.78 4.56 4.89 4.82 5.30 4.90 3.48
3 2.62 2.79 4.02 2.73 3.95 3.81 3.05 3.75
4 2.94 2.83 5.71 3.32 4.08 3.92 3.00 3.61
5 3.02 3.07 6.45 3.84 4.65 4.47 3.57 4.04
6 2.97 2.90 3.00 5.05 4.82 4.90 3.26 3.35
7 3.51 3.32 3.69 3.55 3.58 3.37 3.32 3.55
8 5.05 4.98 5.00 5.38 4.94 4.81 4.47 3.82
9 7.15 7.03 5.00 5.91 5.66 5.51 5.54 4.77
10 5.28 5.35 6.68 4.66 5.13 5.07 4.31 3.49
11 3.96 3.95 3.60 3.76 5.53 5.60 4.63 4.01
12 5.83 6.07 3.87 5.11 4.57 4.86 8.95 4.58
13 3.70 3.78 2.58 3.47 3.51 3.37 4.53 3.35
14 3.41 3.54 2.56 3.39 3.32 3.62 3.97 3.23
15 3.41 3.41 3.03 3.18 3.92 4.04 3.89 3.40
16 3.14 3.09 2.94 3.44 3.66 3.53 3.73 3.16
17 3.36 3.21 2.84 3.97 3.64 3.30 4.00 3.43
18 3.90 3.75 3.14 4.06 3.94 3.95 4.68 4.03
19 4.07 3.87 2.90 4.50 3.72 3.67 3.69 4.05
20 6.73 6.23 6.20 9.75 6.77 8.19 4.25 5.52
21 4.93 4.59 5.39 5.86 5.69 6.59 3.51 4.18
22 4.23 3.80 4.17 6.01 4.95 5.31 3.21 3.44
23 3.54 3.20 3.65 5.75 4.78 4.49 2.94 3.17
24 3.33 3.07 4.13 4.64 3.03 2.95 3.26 3.29

Figure 3.37 - Figure 3.40 below show the hourly skew for each forecasting method to

visualize the results from Table 3.10.

 65
10
W/O Bldg Meas
WITH Bldg Meas
9

7
Kurtosis

2
0 5 10 15 20 25
Hours

Figure 3.37. Hourly kurtosis of the forecast error for all forecasts generated by the MLR models

10
W/O Bldg Meas
WITH Bldg Meas
9

7
Kurtosis

2
0 5 10 15 20 25
Hours

Figure 3.38. Hourly kurtosis of the forecast error for all forecasts generated by the NN models
 66
10
W/O Bldg Meas
WITH Bldg Meas
9

7
Kurtosis

2
0 5 10 15 20 25
Hour

Figure 3.39. Hourly kurtosis of the forecast error for all forecasts generated by the SVM models

10
W/O Bldg Meas
WITH Bldg Meas
9

7
Kurtosis

2
0 5 10 15 20 25
Hours

Figure 3.40. Hourly kurtosis of the forecast error for all forecasts generated by the SAM models
 67
For the MLR and SVM models the kurtosis of the error distributions is nearly identical

throughout the day whether models include building measurements or not. The only method

where building measurements tend to make the errors more “normal” is the SAM method.

3.4.6. REMARKS

There are a number of observations that can be made from the analysis above. Most

prominently, there is a clear trend in forecast performance throughout the day. Early in the day

the forecast error is higher and more variable. Heading into the afternoon and evening hours the

forecasts become more accurate and with a greater frequency. This is true in general for all

methods and all models. The impact of building measurements to both the accuracy and

consistency also follows this performance trend. Improvements are lower during the morning

hours and more pronounced later on. Knowing the operational patterns of the building can make

more sense of the forecast performance during these time windows.

For energy efficiency reasons, large commercial HVAC systems operate under multiple

control schemes [88]. The Hagerty Library operates with a “nighttime setback” where the

building setpoint is reset (e.g. – raised) from 02:00-06:00 which is considered an “unoccupied”

time period. Referring back to Figure 3.3, this time window also corresponds to the low point in

the daily load profile. This low level of demand approaches the base load of the building which

is almost completely thermally insensitive [89]. It stands to reason that including BMS

measurements tied directly to the HVAC system will have less of an impact on the forecasts

during this period. Conversely, later in the day the building HVAC equipment is required to

maintain the building internal temperature in the face of much higher ambient temperature and

increased internal heat gains related to occupancy. This increased cooling load corresponds to

the jump in improvement seen in the enhanced forecasting models.

 68
Although this thesis is not concerned with selecting the best method for building-

specific forecasting applications, several interesting observations can be made about the

performance of each model. The characteristics of the MLR forecasts are impacted less heavily

by the inclusion of building measurements than the other methods. It is however the one of the

most accurate and consistent models overall. In contrast, the NN forecasts when the models

include building measurements are the most greatly improved by nearly every metric observed.

The SAM models also see a similar level of improvement. It makes sense that the only linear

model of those tested (MLR) would see the smallest impact from the addition of these variables.

The good overall MLR model forecast performance is a result of well-developed models. The

nonlinear models are better able to capture the underlying relationship between demand and the

building measurements but they are more complicated to build. It will be an area of future

work to fine tune the model building approach to try and determine if there is an ideal model for

building specific applications. The examination of several models in this thesis is an important

step in that direction.

 69

4. ESTIMATION OF BUILDING-SPECIFIC LOAD FORECASTING PREDICTION

INTERVALS

4.1. OVERVIEW

The previous chapter focused on the generation of point forecasts. No matter how

accurate the selected method is, point forecasts will always be incorrect to a certain degree. The

estimation of forecasting distributions, and the prediction intervals that are obtained from them,

can provide information that is more relevant to risk assessment for the end-user than simple

point forecasts. This Chapter will apply a nonparametric method of estimating prediction

intervals for building-specific load forecasting. The same case study from Chapter 3 will provide

the basis for assessing the impact of building measurements on the resulting prediction intervals.

4.2. PROBABILISITIC FORECASTING REVIEW

Traditionally, load forecasts are presented in the form of point forecasts. This form has

the advantage of being very easy to interpret: each number corresponds directly to an estimate of

the load at a future time. The majority of the forecasting literature focuses on refining the

models and methods that generate these point forecasts. However, one shortcoming of point

forecasts is the lack of any information about the forecast uncertainty. The error analysis

performed in Section 3.4.5 can describe the historical performance of a given method but cannot

fully describe the uncertainty related to a given prediction. Conservative demand side planning

activities require some measure of understanding of the forecast uncertainty. This uncertainty

can be expressed in the form of a prediction interval, also referred to as an interval forecast. The

prediction interval is a range in which a future observation is expected to lie with a pre-assigned

probability [90].
 70
It bears mentioning here that there is some confusion surrounding the use of

confidence intervals versus prediction intervals. While these quantities are often used

interchangeably in the literature, they do not describe the same thing. A confidence interval (CI)

is an interval associated with a parameter estimated from a sample population. A prediction

interval (PI) is an interval associated with future observations of a random variable, where said

observations lie within the interval with a specified probability. Figure 4.1 below shows an

example of the difference between a 95% CI and 95% PI for a simple regression example

(modified from [91]).

Figure 4.1. Example of the difference between a 95% confidence interval (CI) and 95% prediction interval
(PI)

The confidence interval tells us with a 95% probability that the fitted line lies within the

interval. The prediction interval tells us with a 95% probability that the data points will be

within the given interval. Figure 4.1 shows the prediction interval encloses the confidence

interval and it was proven in [92] this is necessarily the case. This thesis is interested in the

accuracy of building-specific forecasts in relation to the actual metered load. Therefore this
 71
thesis will look at prediction intervals rather than confidence intervals to assess the impact of

building measurements on the forecasting accuracy.

Although traditionally the forecasting community has focused on point forecasts, there

has been a more recent push towards methods that can provide valuable formation about the

forecast uncertainty. Uncertain forecasts are a necessary input to a number of power systems

analyses such as unit commitment [93] [94] and reliability planning [95] [96] [97]. This type of

forecasting research falls into the area broadly classified as probabilistic forecasting. In [98], a

very recent and thorough review of probabilistic load forecasting (PLF) is presented. Related

review papers have been written on probabilistic wind forecasting [99] and probabilistic

electricity price forecasting [100]. Similar concepts are shared among all three applications.

As defined in [98] there are a number of techniques employed in the PLF problem. The

PLF process addresses the stochastic nature of the problem by focusing on one of three parts: the

input, the model, or the output. Approaches that focus on the input are concerned with scenario

generation with simulated predictors. These types of works are most often concerned with long-

term forecasting (LTF). Modeling techniques for producing PLFs either extend existing point

forecasting models to generate interval forecasts or are designed specifically to express

probabilistic forecasts. The last category includes PLF methods that post-process the outputs of

point forecasts. This is done either by producing density functions of the point forecast residuals

or combining several point forecasts through quantile regression.

This thesis will employ a modified version of the method from [75] where forecasting

residuals are simulated via a modified bootstrap method in order to produce density functions of

the electrical demand. The details of this procedure are laid out in the following section. There

are several reasons why this method is chosen. By focusing on the out-of-sample test residuals

this method takes advantage of the existing accurate point forecasting methods developed in the
 72
previous chapter. Additionally this method assumes no underlying distribution of the forecast

errors. Typical parametric methods assume an underlying, often Gaussian, distribution. It has

been shown in [101] [75] and many other works that this assumption is usually incorrect. The

results in Chapter 3 indicate that a non-Gaussian distribution is more appropriate. Additionally,

for methods that rely on forecasts of predictor variables, the exact distribution of the load

forecast depends on the distribution of the predictor variables. This makes such distributions

extremely difficult to determine [75]. Bootstrapping is a non-parametric approach that can be

used to construct empirical forecasting distributions and the desired prediction intervals can be

obtained from these distributions. The next section will describe what defines a prediction

interval applied to the load forecasting problem and the details of the chosen method.

4.3. PREDICTION INTERVAL FORMULATION AND ESTIMATION PROCEDURE

The formal definition of a forecast prediction interval estimated at time t looking k steps

ahead is presented in (4.1).

 
   

P pt  k  ˆIt  k |t  P pt  k   Lˆt  k |t ,Uˆ t k|t   1   (4.1)

where

pt  k |t : True value of p given observations up to time t looking

k steps ahead
Ît k |t : Prediction interval estimated at time t looking k steps
ahead
Lˆt k |t : Prediction interval lower bound
Uˆ tk|t : Prediction interval upper bound
1 : Nominal coverage rate

A prediction interval is defined by its lower and upper bounds. The probability (1-α)

within which the observed value is expected to lie is called the nominal confidence rate [90].
 73
This term is preferred to confidence level which is often used in the literature but may lead to

confusion with the confidence interval. The lower and upper bounds are defined in (4.2) - (4.3)

below.

 
Lˆt k |t  qˆtk/|t2  , P pt  k  Lˆt k |t   / 2 (4.2)

 
Uˆ tk|t  qˆt1k|t / 2  , P pt  k  Uˆ tk|t  1   / 2 (4.3)

The boundaries correspond to the quantiles qˆt k|t of the estimated prediction distribution

Fˆt pk|t . This defines a central prediction interval centered on the median of Fˆt pk|t . If a Gaussian

distribution is assumed for Fˆt pk|t then a central prediction interval is sufficient since the mean and

median are the same, and thus the point forecast will lie within the prediction interval. However,

for an asymmetric distribution the mean and median can differ significantly. For small nominal

confidence rates the point forecasts may not even lie within the prediction interval. Therefore,

when the distribution is expected to be non-Gaussian, prediction intervals can be constructed that

are centered on the point prediction pˆ t k|t itself as shown in (4.4).

   
P pt  k   Lˆt  k |t , pˆ t  k |t   P pt  k   pˆ t  k |t ,Uˆ t k|t   1    / 2
 
(4.4)

In order to construct the desired prediction intervals it is necessary to collect a set of k-

step ahead forecast errors denoted St,k. It is assumed that these error samples are representative

of the true predictive distribution Ft pk |t . For a well-developed point forecasting method this is a

reasonable assumption. Thus there is an implicit assumption that the future uncertainty can be

expressed based on the recent performance of the point prediction method. To find an estimate

Fˆt pk|t of the distribution, the cumulative distribution function is introduced in (4.5).
 74
1
Gˆ t,k     #  i  St , k |  i    (4.5)
n

where the cumulative distribution function gives the fraction of collected errors εi less

than a given error level ε.

Recall from (4.2) and (4.3) that the prediction interval bounds with nominal coverage rate

(1-α) correspond to the quantiles of the estimated prediction distribution Fˆt pk|t . Thus the final

form of the prediction interval upper and lower bounds are defined in (4.6) and (4.7) below.

Lˆt k |t  pˆ t  k |t  Gˆ t,k 1  / 2  (4.6)

Uˆ tk|t  pˆ t  k |t  Gˆ t,k 1 1   / 2  (4.7)

where Gˆ t,k 1 are the quantile functions. Effectively, the prediction interval bounds are

calculated by “dressing” the point predictions with estimates of the prediction distribution.

In order to construct the prediction intervals as defined in (4.6) and (4.7), this thesis will

generally employ the modified bootstrap method from [75] as stated previously. The main idea

is to bootstrap the out-of-sample forecast residuals to construct the empirical forecast

distributions from which prediction intervals will be obtained. The steps for this process are

outlined in Figure 4.4. Details of each step are provided below.

Step 1: Calculate the out-of-sample load forecasts

In practice, a rolling window of forecasts and corresponding measured load values would

be collected. For this analysis the out-of-sample load forecasts that are described in section 3.4.3

are used. Of these 116 day-ahead forecasts, the first 3 months (May-July) are collected to

estimate the prediction intervals. The August forecasts are held out to evaluate the prediction

intervals. Data from the Chapter 3 training set (2011 and 2012 data) cannot be used in order to

avoid biasing the results.

 75
Step 2: Collect the out-of-sample load forecast residuals

The residuals are calculated as per equation (3.16). This forms the collection of residuals

St,k that will be bootstrapped to obtain the simulated forecast errors.

Step 3: Parse the residuals into double seasonal blocks.

The standard method for bootstrapping time series data is the block bootstrap [102]. This

process involves taking random segments of the historical time series and rearranging them to

form new artificial series. The length of each random segment (or block) must be long enough to

capture any serial correlations in the data but short enough to allow for a large number of

possible simulated series. At this step of the process this thesis will deviate slightly from the

procedure used in [75]. Rather than use a single block bootstrap, a double seasonal block

bootstrap as used. Seasonality in this case refers to periodic correlations in the data, such as

annual or daily patterns. This method was introduced in [102] to address the double seasonality

(annual and daily) contained in historical temperature observations. Building loads express

similar double seasonality, both daily and hourly, which makes this approach more appropriate.

The difference in how the residual time series are segmented is highlighted in Figure 4.2. The

diagram (a) shows a single seasonal block segmentation and (b) the double seasonal block

segmentation.
 76

Figure 4.2. Seasonal block segmentation (a) and double seasonal block segmentation (b)

Step 4: Bootstrap residuals to obtain simulated forecasting residuals

The original residual time series is double seasonal bootstrapped to obtain simulated

forecast errors. The rearranged time series are shown in Figure 4.3. The diagram (a) shows an

example of a single seasonal block bootstrap and (b) the double seasonal block bootstrap. Note

for this work only (b) is implemented. A total of 500 bootstrap replications are made.

Figure 4.3. Block bootstrapped time series (a) and double seasonal block bootstrapped time series (b)
 77

Step 5: Estimate the empirical forecast distributions

Using the accumulated simulated forecast errors, empirical forecasting distributions can

be constructed. Distributions for each hour of the day are estimated via kernel density

estimation. Using the definition of the lower and upper bounds of the prediction interval from

(4.6) and (4.7), the prediction interval can be determined from the constructed distributions. The

nominal coverage rate 1- α is varied in order to evaluate the forecasts over a range of coverages.
 78

Figure 4.4. Residual bootstrapping procedure

 79
4.4. PREDICTION INTERVAL EVALUATION FRAMEWORK

It is simple to evaluate a point forecast: how large was the deviation between the

estimated value and the true value defines whether or not a forecast is acceptable. Evaluating

probabilistic forecasts is far more difficult. Research into establishing methods for evaluating

probabilistic load forecasts is almost nonexistent [98]. There are a few commonly used attributes

for probabilistic wind forecasting evaluation, which have been borrowed from the meteorological

forecast world, that are of interest to the probabilistic load forecasting problem and will be

applied in this thesis.

Three main attributes are used to characterize the prediction intervals developed using the

procedure laid out in the previous section. These attributes are called reliability, sharpness, and

resolution. Reliability refers to how close the predicted distribution is to the actual. In other

words it is a comparison of the empirical coverage to the nominal coverage rate 1-α. To check

the reliability of a given prediction interval, it is first necessary to introduce an indicator variable

shown in (4.8) below.

 1, if pt  k   Lˆt k |t ,Uˆ tk|t 

    
 t ,k  (4.8)
 0, otherwise

If the true power value pt+k lies in the prediction interval for a given α, then the indicator

value equals 1. Otherwise it is zero. The collection of “hits” and “misses” over the interval of

interest are used to assess the reliability of the estimated prediction intervals.

NT
nk ,1    t,k 
 
(4.9)
t 1

nk,0  NT  nk,1 (4.10)

 80
where

nk ,0 :

Number of "misses" over the interval of interest
nk,1 : Number of "hits" over the interval of interest
NT : Number of realizations over the interval of interest

Given the above expressions, the reliability of a prediction interval is obtained using

(4.11).

1 NT
nk,1
aˆk  
NT
 t,k  
t 1 nk,0  nk,1
(4.11)

Sharpness and resolution are concerned with describing the shape of the forecast

distributions. Sharpness refers to how tightly the estimated distribution covers the actual one.

Narrower distributions are preferable from a decision making context (provided they still

maintain the desirable reliability). For example, if the lower and upper bounds of the prediction

interval are very close to the predicted 99% interval then the interval is said to be very sharp.

The size of the prediction interval is shown in (4.12).

 t,k   Uˆ tk|t  Lˆt k |t (4.12)

The sharpness of the forecasting distribution is calculated by averaging  t,k  of the

prediction intervals estimated at time t for a future time k [103] [98].

NT
1
t ,k  
NT
   
t 1
t ,k (4.13)

As a complement to the sharpness (the mean prediction interval size) the variation of the

interval size over time (e.g. – resolution) can be expressed as the standard deviation of the

interval size for a future time k and a given nominal coverage 1-α.
 81


1 N T  

2
 k     t ,k   k 
NT  1 t 1
(4.14)

In order to easily communicate the uncertain performance of a forecasting method, it is

beneficial to have a single numerical value that can be calculated that includes all relevant

information about the probabilistic forecast. Using an appropriate skill score provides a

comprehensive measure for comparing rival forecasting approaches. As noted previously, this

thesis is focused on comparing forecast performance with and without building measurements

included in the models so such a metric will be a helpful tool. In addition to providing a

convenient measure for comparison, a skill score presented in tandem with observations of the

reliability allows for the assessment of both the sharpness and resolution without having to

calculate these values separately [90]. A separate analysis of the reliability is required to divorce

its influence on the skill score from that of the sharpness and resolution.

As noted in [98], the examples of applying a comprehensive measure of performance is

extremely limited in the load forecasting literature. The pinball loss function was used as a

criteria in the GEFCom 2014 competition [104]. This skill score is formulated specifically to

assess quantile regression forecasts. More appropriate for this work is the Winkler score [105]

shown in (4.15), which has been used for evaluating wind power forecast prediction intervals in

[90] and for probabilistic load forecasting in a slightly modified form in [106]. It can be seen in

(4.15) that the Winkler score will reward narrow prediction intervals while penalizing

observations that do not fall within the estimated interval.

 82

 
 2 t,k   4 Lˆt k |t  pt  k , if pt  k  Lˆt k |t


Sct ,k   2 t,k  , if pt  k  Iˆtk|t (4.15)
  
 ˆ    ˆ  
2 t ,k  4 pt  k  U t  k |t , if pt  k  U t  k |t


To summarize, the building-specific prediction intervals will be evaluated using the

following approach. First, the reliability will be calculated. Results will be presented as

reliability diagrams. These plots will show the deviations from the nominal coverage as a

function of the nominal coverage, indicating whether the given method over or underestimates

the uncertainty. In addition, the skill score defined in (4.15) will be calculated and presented

hourly for a range of nominal coverage rates. This analysis will again focus on evaluating the

impact to the forecast when building measurements are included in the forecast models.

Between the reliability and skill score metrics a comprehensive comparison of the forecast

uncertainty can be achieved.

4.5. RESULTS

The reliability results for the August 2014 test data is presented first. To demonstrate the

impact of building measurements on reliability, reliability diagrams are presented in Figure 4.5 -

Figure 4.8.
 83
100

90

80

W/O Bldg Meas

70 WITH Bldg Meas
Empirical Coverage (%)

60

50

40

30

20

10

0
0 10 20 30 40 50 60 70 80 90 100
Required Probability (%)

Figure 4.5. Reliability diagram for the MLR models

100

90

80
W/O Bldg Meas
WITH Bldg Meas
70
Empirical Coverage (%)

60

50

40

30

20

10

0
0 10 20 30 40 50 60 70 80 90 100
Required Probability (%)

Figure 4.6. Reliability diagram for the NN models

 84
100

90

80
W/O Bldg Meas
70 WITH Bldg Meas
Empirical Coverage (%)

60

50

40

30

20

10

0
0 10 20 30 40 50 60 70 80 90 100
Required Probability (%)

Figure 4.7. Reliability diagram for the SVM models

100

90

80
W/O Bldg Meas
WITH Bldg Meas
70
Empirical Coverage (%)

60

50

40

30

20

10

0
0 10 20 30 40 50 60 70 80 90 100
Required Probability (%)

Figure 4.8. Reliability diagram for the SAM models

For the above reliability diagrams, the x-axis gives the required probability (i.e. the

nominal coverage rate of the prediction intervals). The y-axis shows the empirical coverage

calculated from (4.11). The deviation from the dashed blue line represents the deviation from
 85
the ‘perfect reliability’ situation for which the empirical coverage would equal the nominal

one. For example a 1% deviation for a required 50% coverage rate means the empirical

coverage rate is 51%. These plots indicate a systematic overestimation of the forecast

uncertainty across all methods. There is also a general trend where the deviation is greater for

mid-range coverage rates. Inclusion of building-specific variables in the models does not stray

too much from these observed trends. However, for all but the MLR models, the inclusion of

building-specific measurements drives the reliability closer to the nominal coverage. This effect

is most pronounced for the NN and SAM models. Table 4.1 below quantifies the deviation in %

from the ideal for each model.

Table 4.1. % Deviation in empirical coverage compared to the nominal coverage rate
MLR NN SVM SAM
Coverage No Bldg Bldg No Bldg Bldg No Bldg Bldg No Bldg Bldg
5% 3.92 4.06 2.72 3.12 4.33 4.19 4.06 4.06
10% 7.31 7.31 5.43 4.76 7.31 7.31 6.37 7.04
15% 9.89 10.03 5.59 5.05 8.68 8.68 8.68 9.49
20% 11.26 11.26 6.56 5.35 9.25 8.98 10.86 10.73
25% 12.50 12.50 7.53 5.24 11.16 9.95 13.44 11.96
30% 12.80 12.66 7.02 5.27 12.12 11.45 15.35 11.85
35% 12.96 13.49 8.25 5.43 11.88 10.94 15.78 12.42
40% 14.60 14.19 7.34 5.46 11.77 11.64 15.13 12.04
45% 14.76 15.03 6.69 5.48 11.26 12.20 16.51 12.07
50% 13.71 15.05 5.24 5.78 11.29 10.75 17.47 12.37
55% 14.01 14.27 4.60 4.06 9.30 9.84 19.11 12.12
60% 11.75 13.76 5.56 2.88 9.33 5.83 17.39 9.19
65% 11.64 12.04 5.86 2.37 8.28 5.59 15.00 8.28
70% 10.05 12.07 5.62 2.53 5.48 6.29 14.35 8.17
75% 7.80 9.95 3.23 2.69 5.24 4.03 11.42 6.72
80% 5.94 7.82 2.45 2.18 3.52 4.06 9.70 5.13
85% 4.49 6.24 2.47 1.26 1.40 3.15 6.91 4.35
90% 3.44 5.19 1.02 0.62 1.69 1.02 5.59 2.77
95% 1.32 2.26 0.65 0.38 0.78 0.24 3.20 1.45

The average hourly skill score for the August test data is calculated for each nominal

coverage rate. A total of 19 coverage rates are considered (5% to 95% in 5% increments). A full
 86
set of the results is located in Appendix A2. A subset is shown here in Figure 4.9 - Figure

4.16. Each forecast method is shown for nominal coverage rates of 25% and 75%.

0
W/O Bldg Meas
WITH Bldg Meas
-10

-20

-30
Skill Score

-40

-50

-60

-70

-80
0 5 10 15 20 25
Hours

Figure 4.9. Average hourly skill score for the MLR models. Nominal coverage rate = 25%

-5
W/O Bldg Meas
WITH Bldg Meas

-10

-15

-20
Skill Score

-25

-30

-35

-40
0 5 10 15 20 25
Hours

Figure 4.10. Average hourly skill score for the MLR models. Nominal coverage rate = 75%
 87
0

-10

-20

-30
Skill Score

-40

-50

-60

-70 W/O Bldg Meas

WITH Bldg Meas

-80
0 5 10 15 20 25
Hours

Figure 4.11. Average hourly skill score for the NN models. Nominal coverage rate = 25%

-5

-10

-15

-20
Skill Score

-25

-30

-35
W/O Bldg Meas
WITH Bldg Meas

-40
0 5 10 15 20 25
Hours

Figure 4.12. Average hourly skill score for the NN models. Nominal coverage rate = 75%
 88
0

-10

-20

-30
Skill Score

-40

-50

-60

W/O Bldg Meas

-70
WITH Bldg Meas

-80
0 5 10 15 20 25
Hours

Figure 4.13. Average hourly skill score for the SVM models. Nominal coverage rate = 25%

-5

-10

-15

-20
Skill Score

-25

-30

-35
W/O Bldg Meas
WITH Bldg Meas

-40
0 5 10 15 20 25
Hours

Figure 4.14. Average hourly skill score for the SVM models. Nominal coverage rate = 75%
 89
0

W/O Bldg Meas

-10 WITH Bldg Meas

-20

-30
Skill Score

-40

-50

-60

-70

-80
0 5 10 15 20 25
Hours

Figure 4.15. Average hourly skill score for the SAM models. Nominal coverage rate = 25%

-5

W/O Bldg Meas

WITH Bldg Meas
-10

-15

-20
Skill Score

-25

-30

-35

-40
0 5 10 15 20 25
Hours

Figure 4.16. Average hourly skill score for the SAM models. Nominal coverage rate = 75%

The closer to zero the skill score, the better the performance. Figure 4.9 and Figure 4.10

demonstrate that the skill of the MLR model forecasts is not sensitive to inclusion of building-

specific measurements. The other three methods however are measurably improved. One
 90
notable characteristic common between the point forecast accuracy results and the skill scores

above is the drop in average performance during the morning hours compared to later in the day.

4.6. REMARKS

Predictive distributions of building-specific load have been constructed by dressing

recent point forecast errors in order to assess the forecast uncertainty. Applying the same

method for generating probabilistic forecasts to a set of different point forecasting methods

allows for a comprehensive statistical comparison of the resulting prediction intervals. There are

a number of observations that can be made from this analysis. First, the reliability diagrams

demonstrate the systematic overestimation of the forecast uncertainty. Overestimating (and

underestimating as well) the uncertainty is not a bad thing in and of itself. That determination

must be made by the forecaster and is subjective. However it is necessary to characterize the

overall performance trend to support such a determination. A more objective approach

evaluating reliability based on statistical hypothesis testing has been considered in [103], [107].

These tests rely on the assumption of independence in the forecasting residuals. It is widely

understood in load forecasting (as well as wind and electricity price forecasting) that this

assumption generally does not hold. Therefore the reliability as presented in this thesis provides

a more practical evaluation of the forecasting performance.

The reliability of a probabilistic forecast is similar to the bias metric for point forecasts.

It is interesting then to compare these reliability results to the bias results presented in Section

3.4.5 for point forecasts. The bias of the point forecasts followed no particular trend across each

of the four methods. This is in contrast to the reliability plots which show a distinct trend that is

common across each of the four methods. Notably for both the bias and reliability results the

inclusion of building specific measurements had very little impact. It is reasonable to conclude
 91
then that the point forecasting method drives the error bias and the method of prediction

interval estimation drives the reliability, not the input variables.

Unlike with the reliability results, the skill score is measurably improved when building

measurements are included for all methods except the MLR forecasts. Similar to the conclusions

made in Section 3.4.5 for point forecasts, the skill of the probabilistic forecasts is more positively

impacted for the nonlinear models. As stated in Section 4.4, the combination of the reliability

and skill score metrics fully describes a comprehensive measure of the uncertain forecast

performance. The reliability has been shown to be driven by the method employed to construct

the probabilistic forecast. Thus the improvement in skill observed for the models including

building measurements can be attributed to these variables. This improvement in skill reflects

tighter, more consistently sized prediction intervals (i.e. – improved sharpness and resolution).

Forecasts that produce narrower intervals are more valuable in a decision making context. This

is important where building-specific load forecasts are used to assess demand side management

capabilities.

It stands to reason that the quality of the method for constructing prediction intervals is

tied to the quality of the related point forecasts themselves. The sharper the error distributions,

the higher the quality of the resulting probabilistic forecasts. It is then an area of future interest

to develop direct probabilistic forecasts of building demand, removing the reliance on point

forecasts in order to possibly improve the quality of the probabilistic forecasts. For this thesis

however it is an important first contribution to the area of building-specific forecasting that the

uncertainty of point forecasting methods is initially considered.

 92

5. CONTROLLABLE BUILDING ELECTRICAL LOAD MODELING

5.1. OVERVIEW

In this chapter a dynamic load model for the controllable load of a building is developed.

In this work the controllable load is considered to be the HVAC chiller. The coupling of a

building’s electrical and thermal characteristics is not adequately captured using existing static

models. Therefore a dynamic exponential load model is proposed. This allows for more

accurate planning of the dispatching of load for DSM. The model in this thesis is derived from

tests performed on an actual HVAC system at Drexel University.

5.2. MODEL DEVELOPMENT

Traditionally in power systems, loads are represented in aggregate. Loads are grouped by

bus, or substation, and modeled as one complex power injection (for a given bus) of the form

shown in (5.1).

S inj  P inj  jQinj (5.1)

While this structure is convenient in that it reduces model complexity when performing

power system analysis, studies over the last 20 years have shown such static models are not

appropriate for studies where load dynamics have critical impacts [108] [109]. In terms of DSM

applications, the model in (5.1) neglects dynamic behavior and lacks the resolution required to

analyze and optimize control of individual components. An alternative approach is one where a

direct mathematical model is developed based on measurements. The advantages of this model

will be two-fold:
 93
 The increased granularity will provide more opportunities for customers to

control and dispatch loads in DSM programs

 The dynamic response of HVAC loads due to the natural coupling of the electric

demand and the thermal response of the building can be clearly understood.

At the power system level, a measurement-based approach is used in [110] [111] [112]

[113] to study the impact of voltage changes on real power and reactive power respectively.

Measured data is fit to an assumed model of the dynamic behavior. The model used in these

works is a dynamic exponential load model. In this thesis the same model will be used to

capture the thermal electrical behavior of an HVAC chiller. As noted in [110] [111] this model

was proposed as a means of capturing the unique effects of large electrical heating loads on the

Swedish power system.

If the end purpose is to use this model for DSM, characterizing the electrical load response

to a control action is essential. In the case of an HVAC chiller, raising or lowering the outlet

chilled water temperature is a straightforward action a facilities manager can take to increase or

decrease the electric power drawn by the machine. Therefore, in the following sections, a

mathematical model for the HVAC chiller electrical load response to a change in outlet water

temperature setpoint is presented and derived.

5.2.1. TEST DESCRIPTION

Data was collected during tests performed in the fall of 2009 for a single HVAC chiller

on Drexel University’s campus whose cooling load was carried on a single chiller. For this

particular chiller, the nominal outlet water temperature is 44oF. At approximately 09:00, the

temperature setpoint was raised to 51oF and around 15:00 it was lowered back to 44oF. These

setpoint changes were the only change made to the HVAC system. All other system operations

were allowed to operate as normal (i.e. fan speeds, damper positions, etc.). While this may not
 94
be the most effective action in terms of a DSM activity since the overall power draw of the

system might not decrease as much as desired, this allowed us to observe the chiller electrical

response to the setpoint change isolated from any other HVAC actions. Chiller load data and

outlet water temperature was recorded. These values are shown in Figure 5.1.
Full Load Amps (%)

100

80

60

40

6:08 8:58 11:48 14:35 18:28

Chilled Water Temp. (oF)

54

52

50

48

46

44

42
6:08 8:58 11:48 14:35 18:28
Time
Figure 5.1. Electrical load data in %FLA and chilled water outlet temperature data in oF. Data was collected
over a 12 hour window

Note that this data was recorded by the Drexel BMS which measures the chiller electrical

load in terms of percentage of full load amps (FLA). Because there are no local voltage

measurements available in the BMS, it is assumed that there is a constant voltage profile and the

% FLA reading is a direct reflection of the real power drawn by the chiller motor. Another

advantage of plotting the electrical power in this manner is that we can easily view the electrical

response and outlet temperature response on the same graph. This is demonstrated more clearly

in Figure 5.2. This plot shows the electrical response versus the temperature response, and in

looking to develop a mathematical model for this component it is more telling to examine this

time period.
 95

Figure 5.2. Electric load (blue line) and temperature (red line) response to raising the chill water temp.

5.2.2. MODEL DERIVATION

Several important points can be drawn from observing the response in Figure 5.2. The

recovery of the electrical load does not exhibit the “fast” response normally associated with

electrical systems. Due to the dependence of the electrical load on the temperature of the system,

there are long time delays involved in the load recovery. While the speed of the response is

unique to this type of system, the general electrical power response has been exhibited before in

dynamic load studies of transmission systems as noted previously [110] [113] [114]. The

exponential recovery model used in these can be used for the HVAC chiller model as well.

To express the electric power response of the chiller, a similar set of equations as in [110]

[113] [114] is applied, the difference being the chiller is a function of time and outlet water

temperature change in lieu of time and voltage. A set of equations describing this model is

presented in (5.4) – (5.5) below.

 96
  tc 
s
(5.2)
Ps    Po  
 o 

 t 
t
(5.3)
Pt    Po  c 
 o 

where

tc : time delay between temperature change and load response

Po : initial power value (nominal operating point)
o : initial chilled water temperature value
s : static load model exponential index
t : transient load model exponential index

with

dPd  t  dPt  
Tr  Pd  t   Ps    Tr (5.4)
dt dt

where

Tr : dynamic load time constant

: chilled water temperature value
Pd   : dynamic load function chilled water outlet temperature
Pd  0   P0

From [110], a closed form solution may be derived to Equation (5.4) in the event of a

step change in temperature (change in setpoint level):

 t t 
 c 
Pd  t   Ps     Pt    Ps    e  Tr  (5.5)

The equations (5.4)-(5.5) describing this response are plotted in Figure 5.3 for an

arbitrary increase in the chilled water temperature setpoint. Subplots (a) and (b) show the

temperature response. Of note is the time delay tc between when the temperature setpoint is
 97
raised and when the actual chilled water temperature begins to increase, as well as the time

required for the temperature to reach its new value. The electric power response is shown in

subplots (c) – (e). The transient model Pt and the static model Ps show the minimum power

value and new steady state operating power value respectively. The dynamic load function Pd

shows the full dynamic power response. It is important to observe that the power level recovers

to a new steady state power value that is less than the initial power value Po. The level of steady-

state power recovery is dictated by the static load model index βs.
 98

Figure 5.3. Response to a step change in the chilled water temperature setpoint value according to Equations
(5.4)-(5.5)
 99
5.2.3. DREXEL-SPECIFIC EXAMPLE

Given the mathematical model presented here, the next step is to estimate the model

parameters based on the observed test data. The specific method of parameter estimation is not a

focus of this thesis. As noted in [111] and [115] the model parameters can be estimated from

measured data using the method of least squares, as shown in (5.6).

N
min   p     y  k   yˆ  k  
2

pZ
(5.6)
k 1

where

p: Vector of model parameters to be estimated

Z: Feasible parameter space
 : Least squares error function
N: Number of measurements
y k  : Measured value
yˆ  k  : Model output value

Another approach is to use curve fitting and manual inspection [113] [116]. While this

approach is not well suited for automation, when considering a single test a greater context as to

the physical meaning of each parameter can be attained.

The set of model parameters from (5.4)-(5.5) that need to be estimated includes θo, θs, Po,

tc, TR, s, and t. These parameters will be solved for by directly observing the data to measure

the values, as well as applying appropriate curve fits. The parameters will be solved for in the

following order:

1. Determination of tc

The time delay tc is measured as the time delay between raising the chilled water

temperature setpoint and when the actual temperature begins to rise. This delay is measured

directly from the data. The time delay is measured as the point in time after the setpoint change
 100
where the observed temperature is greater than the maximum temp. from the previous minute

in order to distinguish the response from measurement noise. Through several tests performed at

Drexel, this time delay was observed to be on the order of 100 – 1000 seconds. These numbers

seem reasonable but more testing or research into the chiller control system is warranted to better

understand the source(s) and characteristics driving this delay.

2. Determination of θo, θs, Po, and TR

Each of these parameters was estimated by curve fitting the collected data. Figure 5.4

shows the curve fit approach to the measured test data. As shown in this figure, the fit can be

broken into three sections. Line segment 1 is applied to the initial load behavior prior to raising

the temperature setpoint. This defines the initial operating power Po and temperature θo. Line

segment 2 is applied to the drop in load following the setpoint change. This line segment runs

between the initial power Po and the minimum transient power Pt. The load recovery response is

fit using an exponential regression of the general form Y  A 1  e BX  . The recovery time is an

estimate of TR and this segment settles out to the new steady-state power level Ps. This

regression analysis was performed using both the Excel regression tool as well as the Matlab

Curve fitting toolbox.

 101

Figure 5.4. Application of the curve fit approach on the collected data for parameter estimation

3. Determination of s and t

The parameter βs can be estimated directly using (5.7):

P   
 s  ln  s  / ln  s  (5.7)
 Po   o 

Similarly, the parameter βt can be estimated directly using (5.8):

 P   
t  ln  t  / ln  s  (5.8)
 Po   o 

The parameter estimation method outlined in this section was applied to the test data shown in Figure 5.4.
Table 5.1 shows the resulting parameter estimates for the Drexel test described in Section

5.2.1. Note that power values are presented as a percentage of full load.
 102
Table 5.1 Parameter estimation results for the test described in Section 5.2.1
Parameter Estimated Value
Po 49.75%
Ps 46.81%
Pt 28.25%
o
θo 44.11 F
o
θs 51.27 F
s -0.405
t -3.762
tc 143 sec.
TR 1690 sec.

One important observation from Table 5.1 shows that the load power recovery time TR is

approximately 28 minutes. This demonstrates the effects of the thermal time constant of the

building on the electrical response. Looking back at Figure 5.1, Figure 5.2, and Figure 5.4 there

is a significant amount of noise contained in the data. There is a concern that this amount of

noise can lead to inaccurate parameter estimates regardless of the parameter estimation method

used. In addition to the noise, there is a known hunting issue with this particular chiller. This

issue is believed to be a considerable source of noise in this data. Correcting this problem prior

to future testing may help achieve test data reflecting more regular behavior.

5.3. REMARKS

The HVAC chiller dynamic load model presented above can be used as a tool for

implementing DSM plans for any customer where their HVAC load is a significant portion of

their total electrical load. The model allows a good amount of flexibility to the end-user

depending on the time window of interest. In a simple case, the model can be used to accurately

predict a specific amount of a facility’s electric load being shed through a temperature setpoint

adjustment. Provided the dispatch interval is sufficiently long the static load model in (5.2) can

be used to model the reduction in load, as shown in (5.9) below.

 103
P  Ps  o   Ps  s  (5.9)

Even when using the static model, the impact of the load dynamics can still be applied.

Knowing the recovery time TR, the time-varying properties can be programmed into ramp

constraints that ensure load recovery response times meet the required scheduling windows. In

a more dynamic case, a customer can use the full dynamic exponential load model to look at the

behavior during the transient period in addition to how long it will take to reach a new steady

state load level. In the next section of this thesis, this load model will be used in a method of

planning commercial building load resources for DSM. The end goal is to determine the optimal

dispatch schedule for a group of buildings and including the load dynamics established here is

essential.
 104

6. OPTIMAL SCHEDULING OF BUILDING DEMAND RESOURCES

6.1. OVERVIEW

The previous chapters of this thesis focused on modeling and predicting building

electrical demand behavior. The study of these topics is an essential piece in determining how to

use building load resources for demand side management. Given a method to predict building

load and a model of the controllable HVAC load, we can now develop a systematic way of

scheduling the demand levels of multiple buildings to meet certain objectives.

This chapter of the thesis will focus on the following:

 Define the optimal load scheduling problem, including the variables, assumptions,

and scenarios to be considered.

 Develop the mathematical framework for several optimization problems. These

problems will address the impacts of real-time pricing, demand response revenue,

building operational constraints, and varying levels of forecast uncertainty.

 Provide simulations and interpret results.

6.2. BUILDING LOAD SCHEDULING PROBLEM DESCRIPTION

6.2.1. GOALS

The goal of the building load scheduling problem is to find the optimal day-ahead, hourly

load dispatch schedule for a multi-building campus, subject to a set of operational constraints. In

this work optimal refers to a load schedule that minimizes total electricity cost through usage

reduction and (in some cases) demand response (DR) participation. The constraints will include

electrical and thermal operating limits of the buildings. The output of the scheduling problem is
 105
the optimal reduction of the controllable load for each building. Simulations will focus on

the impact of several key practical features of this problem. This includes the role of several

DSM programs and sensitivity to a number of model parameters defined for this problem.

6.2.2. BUILDING DEMAND TERMINOLOGY

Several terms need to be defined in order to formulate the load scheduling problem. This

work is concerned with finding optimal schedules that will deviate from the normal demand

levels of the building. As discussed in Chapter 2, a baseline prediction of building load is

required to assess the performance resulting from any control actions. The baseline is a forecast

of the building load assuming no demand side management actions are taken and this is the load

profile against which DSM activities are measured. The controllable load for this work is

considered to be the building HVAC chiller. This is the only portion of the load that can be

adjusted. Figure 6.1 below shows the relationship between the forecast PF and dispatched

controllable load ∆PC.

Figure 6.1. Sample building load forecast and dispatched load

Often a campus may have a variety of building stock available as potential DSM

resources. To characterize several features that describe potential variety of available buildings

three parameters are introduced.

1) Load Footprint

First, the size (electrically speaking) of a given building is referred to as the load

footprint. The load footprint of a building is defined by the maximum power level of the
 106
building. In practice this value can be derived from observed historical data or the electrical

size of the main service feed to the building.

2) Load Flexibility

Some buildings will act as more responsive load resources. This responsiveness is

defined as the load flexibility. Load flexibility refers to the amount of the controllable load ∆PC

 P c 
changes to a building temperature setpoint change   . This responsiveness is captured by
  

the controllable load model exponent β.

3) Load Margin

The last term that needs to be defined for this problem is the load margin. Load margin

refers to the amount of load that is available for dispatch during a given interval. This parameter

captures two important issues with dispatching building loads. First, the controllable load

margin is defined by how close the controllable nominal operating level is to any operating

limits. This margin will constrain the amount of load that can be dispatched at any given time.

This allowable range is defined in (6.1) where it is assumed the building is designed such that the

nominal operating level is within this range.

margin C   PC ,nom  PC ,min , PC ,max  PC ,nom  (6.1)

The overall building load margin is also impacted by any uncertainty in the load forecast.

The margin as defined above constrains how much the controllable load can be changed.

However, the total load change ∆P observed in relation to the forecast may be much lower.

Recall that this comparison is how settlement is done in determining DSM savings. This

situation is shown in Figure 6.2. The bounds of the load forecast Û and L̂ are depicted with

dashed lines. For identical amounts of ∆PC, the amount of load shed that qualifies as a DSM

action can be quite different. This distinction in available load margin becomes particularly
 107
important in situations where the utility may impose penalties for not achieving the promised

load dispatch amounts.

Figure 6.2. Relationship between load margin and building load forecast uncertainty (expected worst case)

The influence of these 3 new parameters (load footprint, load flexibility, and load

margin) on the optimal load dispatch schedule will be studied in this Chapter. Each will be

addressed within the context of several DSM scenarios defined in the next section.

6.2.3. SCENARIOS

Four different scenarios are considered in this work. Each scenario corresponds to a

different demand side management program. All four scenarios are defined below:

Scenario 1 – Cost Minimization with Constant Pricing

In this scenario the price of electricity is assumed to be constant. This is common

among large electricity users who typically negotiate annually a constant cost per kWh,

provided the user stays within agreed upon minimum and maximum total electrical

energy limits. In this scenario, the controllable load for each building will be dispatched

during each interval as determined to achieve the greatest electricity cost savings.
 108
Scenario 2 – Cost Minimization with Variable Pricing

As discussed in Chapter 2, there has been a push for dynamic pricing structures to

be implemented in the retail electricity market. These changes will serve to incentive

customers to take active control of their electricity usage. The only difference between

Scenario 2 and Scenario 1 is that electricity cost is allowed to vary throughout the day.

Minimizing electricity costs over the day by dispatching controllable building loads

remains the goal.

Scenario 3 – Cost Minimization with Constant Pricing and Demand Response Revenue

Many electricity markets now allow large end-users or demand aggregators to bid

a certain amount of load into the market. Generally a threshold exists for the amount of

load that needs to be shed in order to qualify for a market payout [23]. In this scenario

the goal remains to find the load schedule that minimizes total daily electricity cost. For

each interval the possibility of earning revenue is considered as in input to the overall

cost function.

Scenario 4 – Cost Minimization with Variable Pricing and Demand Response

The only difference between Scenario 4 and Scenario 3 is that electricity cost is

again allowed to vary thought the day. The distinctions between each scenario are

summarized in Table 6.1 below.

Table 6.1. Summary of DSM scenarios

Scenario Pricing DR Revenue
1 Constant No
2 Dynamic No
3 Constant Yes
4 Dynamic Yes
 109

6.2.4. ASSUMPTIONS

Several assumptions are made for this scheduling problem.

A1: Any losses related to a campus-wide distribution network will be ignored. It is

assumed that the electric meters where demand is measured are located directly at

the building service entry and thus the effects of the network do not need to be

considered.

A2: Day-ahead load forecasts and price forecasts are available for all buildings. It is

assumed that the end-user has access to an accurate method of building-specific

load forecasting (such as those in Chapter 3) and that day-ahead price forecasts

are provided and they are correct. Price volatility is not considered explicitly in

this work, though the role of price plays in the solution will be examined.

A3: The nominal operating conditions (controllable load level and temperature) for all

buildings are known. The nominal operating level PC,Nom of the building HVAC

chiller varies throughout the day as a function of the cooling requirements of the

building. Methods exist for accurately estimating this value and this work will

assume the value is readily available [89]. Therefore these quantities are assumed

as known values in this study.

A4: Load variability within the dispatch intervals is ignored. All analysis is concerned

with the day-ahead, hourly scheduling of building demand. Such intra-interval

behavior is of interest during real-time operations but is outside the scope of this

study.

A5: The dispatch interval is sufficiently long to ignore controllable load dynamics.

This assumption does not mean the dynamics of the load are ignored entirely. As
 110
will be shown in the optimization problem formulations in the next section,

load recovery rates are accounted by applying a ramp rate constraint. This

assumptions, like A5 before it, means that the intra-interval transient behavior of

the load is ignored.

6.3. OPTIMIZATION PROBLEM FORMULATIONS

6.3.1. SCENARIO 1 & 2

The optimization problem for scenario 1 and 2 is constructed as a mixed integer linear

programming (MILP) problem and is presented below. This formulation assumes for now a

perfect load forecast is available for each building. This assumption will be relaxed later on.

T M
min  t , m Pt C,m (6.2)
t 1 m 1

s.t. t 1,..., T m  1,..., M

PtC,m  st ,m  PtC,m,min  PtC,m,nom   0 (6.3)

PtC,m  st ,m  PtC,m,max  PtC,m,nom   0 (6.4)

PtC,m  PtC1,m  tdown C, nom

, m + Pt , m  PtC1,,nom
m (6.5)

Pt C, m  Pt C1, m  tup, m  Pt C, m, nom  Pt C1,, nom

m (6.6)

s
t 1
t ,m   mon (6.7)

where
 111
PtC,m : Dispatched controllable load at time t for building m
t ,m : Price of electricity at time t for building m
st ,m : State (on/off) of controllable load at time t for building m
PtC,m,nom : Nominal controllable load level at time t for building m
PtC,m,min : Minimum controllable load level at time t for building m
PtC,m,max : Maximum controllable load level at time t for building m
tdown
,m : Ramp rate limit (down) at time t for building m
t ,m :
up
Ramp rate limit (up) at time t for building m
on
m : Limit on number of intervals PC can deviate from PC ,nom
for building m
M  0 Total number of buildings
T  0 Total number of dispatch intervals

with

Pt C,m,min  max  Pt C,m tmax

, m  , Pt , m 
min
 (6.8)

Pt C,m,max  min  Pt C,m tmin

, m  , Pt , m 
max
 (6.9)

where

tmax
,m : Maximum allowable temperature at time t for building m
t , m :
min
Minimum allowable temperature at time t for building m
Pt max
,m : Maximum allowable load level at time t for building m
Pt min
,m : Minimum allowable load level at time t for building m

 Objective function – Equation (6.2)

This problem seeks to minimize the cost associated with dispatching building demand for

all buildings M over every time interval T. The optimal value will be less than or equal to zero,

indicating a reduction in electricity costs in relation to the normal operating schedule.

 112
The price of electricity t ,m is kept constant  t,m in scenario 1. In scenario 2 the

price varies at each interval t but is the same for each building m.

 Minimum/Maximum power constraints – Equations (6.3) - (6.4)

These constraints do two things. First, the quantity of dispatched load must not cause the

building to operate above or below practical limits (i.e.- controllable load margin). These

minimum and maximum values (determined via (6.8) - (6.9)) capture both the electrical limits

(e.g. – HVAC machine ratings) and thermal comfort limits. Whichever is more restrictive at

each interval is applied.

The second function of these constraints is to force Pt C,m =0 during intervals when the

load is not being dispatched. Inserting the operating state st , m in these inequality constraints

avoids the situation where Pt C,m and st , m would be multiplied in the objective function, thus

simplifying the problem.

 Ramp rate constraints – Equations (6.5) - (6.6)

These linear constraints capture the controllable load recovery dynamics. The ramp rate

limits  tdown
,m and  upt, m are derived from the recovery time of the nonlinear load model. These

constraints ensure raising or lowering of the load to a new operating point is achievable within

the required interval length.

 Run time constraint – Equation (6.7)

It is necessary to limit the number of times the controllable load is operated off of its

normal operating point. Even though the thermal comfort issue is managed by the min/max

constraints, there is a practical need to limit the time the HVAC system operates off its normal

level. As was discussed earlier, repeated cycling of the HVAC system is not good for reliability

reasons. Additionally, HVAC systems are designed for specific operating conditions for
 113
efficiency reasons. Therefore it is prudent to limit the total number of times a day the HVAC

system is committed for dispatching load.

 Decision variables

The decision variables are Pt C, m and st , m . There are (M  T) of each variable: one for each

building at every time step. The dispatched power Pt C, m is a real valued number and the load

state is a binary on/off indicator {0,1}. The total number of decision variables is therefore 2

(MT).

 Total problem size

Number of decision variables: 2 (MT)

Number of constraints: 2 (MT)+ 2 (M(T-1))+T

6.3.2. SCENARIO 3 & 4

The optimization problem for scenario 3 and 4 is constructed as a mixed integer linear

programming (MILP) problem and is presented below:

 T M T M

min   t , m Pt C, m    t , m Pt ,DR
m  (6.10)
 t 1 m 1 t 1 m 1 

s.t. t 1,..., T m  1,..., M

Equations (6.2) - (6.7)

 P
m 1
C
t ,m  BstDR  PtThresh (6.11)

Pt C, m  Pt ,DR

m  Bst
DR
0 (6.12)

Pt C, m  Pt ,DR

m  Bst
DR
0 (6.13)
 114
m  B 1  st 
Pt ,DR DR (6.14)

m  B 1  st 
Pt ,DR DR (6.15)

where

t ,m : Locational marginal price (LMP) at time t for building m

P : DR
t ,m Dispatched controllable load for DR revenue at time t for building m
B : 0
Large positive integer
stDR : State (on/off) of DR dispatched load at time t
PtThresh : Load threshold required to earn DR revenue at time t

 Objective function – Equation (6.10)

An additional term is added to the cost function to capture potential revenue earned

through demand response (DR) programs. A new decision variable Pt ,DRm is introduced to

quantify load shed that qualifies for DR revenue. This variable equals Pt C, m if the required

threshold is met and equals zero otherwise.

The price of electricity t ,m is kept constant  t,m in scenario 3. In scenario 4 the price

varies at each interval t but is the same for each building m. For both scenarios, the LMP t ,m

varies at each interval t but is the same for each building m.

 Minimum/Maximum power constraints – Equations (6.3) - (6.4)

These are the same constraints applied in scenarios 1 and 2.

 Ramp rate constraints – Equations (6.5) - (6.6)

These are the same constraints applied in scenarios 1 and 2.

 Run time constraint – Equation (6.7)

This is the same constraint applied in scenarios 1 and 2.

 115
 DR threshold constraint – Equation (6.11)

In order to qualify for DR revenue the amount of load shed must exceed a certain

threshold PtThresh . Another decision variable for the state of DR revenue is introduced ( stDR ). This

constraint ensures that stDR takes the value of 1 if the threshold is met and zero otherwise.

 DR equivalence and switching constraints – Equations (6.12) - (6.15)

The variable Pt ,DRm does not represent a real quantity of load separate from Pt C, m . It is a

M
variable that is used solely to account for DR revenue if the condition  P
m 1
C
t ,m  PtThresh exists. To

that end, these constraints ensure that if the above condition is true, Pt ,DRm  Pt C, m . If it does not,

then Pt ,DRm  0 .

 Decision variables

The decision variables Pt C, m and st , m from scenario 1 and 2 are the same here. The two

new decision variables introduced in scenario 3 and 4 are Pt ,DRm and stDR . There are (M x T)

variables for Pt ,DRm - one for each building at every time interval. There are T stDR variables – one

for each time interval. The total number of decision variables is therefore 3 (MT)+T.

 Total problem size

Number of decision variables: 3 (MT)+T

Number of constraints: 6 (MT)+2(M(T-1))+2T

6.3.3. SCENARIO 1 & 2 WITH IMPERFECT LOAD FORECASTS

The goal remains finding the optimal Pt C, m that will lead to the greatest electricity cost

savings. However now the difference between the dispatched controllable load Pt C, m and the

deviation from the total predicted load Pt , m becomes relevant. An error parameter ξ which in
 116
practice can be derived from a probabilistic forecast will be added to the constraints to

provide a worst-case bound to the total available load margin.

The modified optimization problem for scenario 1 and 2 is presented below.

T M
min  t , m Pt C,m (6.16)
t 1 m 1

s.t. t 1,..., T m  1,..., M

PtC,m  t ,m  st ,m  PtC,m,min  PtC,m,nom   0 (6.17)

PtC,m  t ,m  st ,m  PtC,m,max  PtC,m,nom   0 (6.18)

PtC,m  t ,m PtC1,m  t 1,m  tdown C, nom

, m + Pt , m  PtC1,,nom
m (6.19)

Pt C, m  t , m  Pt C1, m  t 1, m  up

t , m  Pt , m
C , nom
 Pt C1,, nom
m (6.20)

s
t 1
t ,m   mon (6.21)

Notice that the objective function remains unchanged. There is no need to add an

additional cost term associated with the load uncertainty. As the objective is written, the

resulting optimal day-ahead dispatch schedule will still minimize costs with respect to how much

uncertainty is expected in the forecast.

6.3.4. SCENARIO 3 & 4 WITH IMPERFECT LOAD FORECASTS

Just like with scenario 1 and 2, a new parameter ξ is introduced. Following the same

procedure, the original problem becomes:

 T M T M

min   t , m Pt C, m    t , m Pt ,DR
m  (6.22)
 t 1 m 1 t 1 m 1 
 117
s.t. t 1,..., T m  1,..., M

Equations (6.17) - (6.21)

  P
m 1
C
t ,m  t , m   BstDR  PtThresh (6.23)

Pt C, m   t , m  Pt ,DR

m  Bst
DR
0 (6.24)

Pt C, m   t , m  Pt ,DR

m  Bst
DR
0 (6.25)

m  B 1  st 
Pt ,DR DR (6.26)

m  B 1  st 
Pt ,DR DR (6.27)

As with the deterministic case, the variable Pt ,DRm does not represent a real quantity of

load separate from Pt C, m . It is a quantity that is used solely to account for DR revenue if the

M
condition  P
m 1
C
t ,m  PtThresh exists. Therefore no uncertainty term is associated with that variable.

6.4. SIMULATIONS

6.4.1. TEST SYSTEM DESCRIPTION

The proposed load scheduling methods are tested on a fictitious 10 building system. The

relevant parameters for each building are based on observations of Drexel University buildings

so while the test system does not actually exist it is representative of a real world campus. It is

desirable to have a variety of building stock to observe the impact each building type has in the

proposed load scheduling approaches. Variety in this case refers to both the load footprint as

well as load flexibility. For this study the following range values are presented. Building load

footprint classification is shown in Table 6.2. Building controllable load flexibility is shown in
 118
Table 6.3. A summary of the building types that form the base of this study are presented in

Table 6.4.

Table 6.2 Building load footprint definitions

Footprint Max. Power (kW)
Small < 250
Medium 250-500
Large > 500

Table 6.3 Building controllable load flexibility definitions

Flexibility β
Low < -0.4
Medium -0.4 < β < -0.9
High > -0.9

Table 6.4. Load footprint and flexibility for each building used in this case study
Building # Footprint Flexibility
1 Small Medium
2 Medium Medium
3 Medium High
4 Small High
5 Large Medium
6 Large Low
7 Large High
8 Medium Low
9 Medium High
10 Large Medium

The forecasted building load profiles are shown in Figure 6.3. These profiles define the

nominal operating behavior against which the optimal dispatched load is compared.
 119
900
Bldg 1
Bldg 2
800 Bldg 3
Bldg 4
Bldg 5
Bldg 6
700
Bldg 7
Bldg 8
Bldg 9
600
Bldg 10
Demand (kW)

500

400

300

200

100
0 5 10 15 20 25
Hour

Figure 6.3. Forecasted building load profiles

Electricity price information for a single day has been obtained from the PJM website

[117] for Monday, July 21, 2014. These prices are representative of the LMP prices at the nodes

in the system where Drexel University connects to the grid. For the variable pricing scenarios

and for the DR revenue pricing, these LMP prices are used as the cost of electricity. For the

constant pricing scenarios, the price of electricity is assumed to be the average of the daily LMP.

This pricing information is shown in Figure 6.4.

 120

65

60 LMP
mean(LMP)

55

50
LMP ($/MWh)

45

40

35

30

25

20
0 5 10 15 20 25
Hour

Figure 6.4. LMP data for Monday July 21st, 2014

Finally, the parameters that define the building operating behavior and constraints is

shown in Table 6.5.

Table 6.5 Base building parameters values used in the simulations

Building # θmin (oF) max o
θ ( F) ∆
up
(kW) ∆
down
(kW)
o
θo ( F) β
up
β
down min
P (kW) P
max
(kW)
on
τ
1 65 78 70 70 72 -0.9 -0.9 50 200 4
2 65 75 70 70 72 -0.7 -0.7 50 300 2
3 65 75 100 100 70 -1.1 -1.1 50 450 5
4 65 75 100 100 70 -1.3 -1.3 50 150 4
5 65 75 70 70 70 -0.6 -0.6 100 600 5
6 65 75 40 40 72 -0.4 -0.4 100 700 6
7 65 75 100 100 72 -1 -1 50 1050 3
8 65 75 40 40 70 -0.4 -0.4 50 500 3
9 65 75 100 100 72 -1.25 -1.25 50 450 6
10 65 75 70 70 70 -0.5 -0.5 50 1200 4
 121
The information presented here defines the base values for the following simulations.

The impact to the load schedule solution from changing some of these parameters is one of the

main goals of this work. Therefore some of the values presented here may be changed during

various tests. Any deviations from these numbers will be specifically noted.

It bears mentioning that the following simulations will not compare the performance of

the proposed load scheduling algorithms against any established methods. This is partly because

we are more interested in observing the impact of various DSM scenarios and model parameters

on the solution and less on the value of the solution itself. It is also partly because there is no

established benchmark for the consumer-focused load scheduling problem.

All of the following simulations are carried out using the OPTI Toolbox for Matlab

[118]. Specifically the GLPK solver is used to solve each MILP problem. This solver employs a

version of the simplex method. All simulations are performed on a personal computer with 2

processors clocking at 2.0 GHz and 8 GB of RAM. The computational aspects of the problem

are not considered for this work but it should be noted that none of these simulations required

more than 30s to complete. This is perfectly suitable for a day-ahead scheduling algorithm.

6.4.2. SIMULATION RESULTS AND DISCUSSION

 Test 1: Find the optimal dispatch schedule for Scenario 1

The optimal dispatch schedule is given in Table 6.6 on the next page. The total savings

achieved through this dispatch schedule is $33212.00. Several key observations can be made

about this optimal schedule:

o Since the number of times a building is allowed to be dispatched is limited during

the day, the most potential value occurs where the greatest amount of load margin

exists. Considering the load forecasts in Figure 6.3, it makes sense that most of
 122
the dispatch is scheduled during the middle of the day when building load is

highest.

o The amount of load dispatched per building is clearly driven by the size of the

load footprint as well as the load flexibility. The next two tests will look at how

varying these parameters affects the solution.

 Test 2: Examine the impact of load sensitivity to the Scenario 1 dispatch schedule

For this test the β value for building 7 is varied from -1 to -0.3 in steps of -0.1. This is a

major reduction in load flexibility. No other changes are made. The optimal dispatch

schedule for β = -0.3 is given in Table 6.7 on the previous page. Several key observations

can be made about this optimal schedule:

o The savings is reduced by $4832 when compared to the schedule in Table 6.6

(Building 7 β=-1). This reduction is expected. By lowering the load flexibility of

building 7 by such a large amount the capacity for shedding load (and hence

saving on cost) is greatly diminished.

o It is notable that the rest of the dispatch schedule is unchanged for each building.

This includes which intervals where building 7 is dispatched as shown in Figure

6.5. The load sensitivity does not impact the timing of the schedule in this

Scenario, only the amount of power that is dispatched and only for a specific

building. This result should be intuitive. Load will be dispatched when the

largest margin for shedding exists. This is driven by the nominal operating level

PC in relation to the min/max and ramp constraints. Load flexibility, for a

constant pricing DSM scenario, impacts how much load is dispatched for a

specific building only, and no impact not when the load is dispatched.
 123

Table 6.6 Optimal load schedule in kW for Test 1 (Scenario 1)

Interval Number
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24
1 0 0 0 0 0 0 0 -10.94 0 -10.94 0 0 -10.94 0 0 -10.94 0 0 0 0 0 0 0 0
2 0 0 0 0 0 0 0 0 0 0 0 0 0 -11.91 -11.57 0 0 0 0 0 0 0 0 0
Building Number

3 0 0 0 0 0 0 0 0 -23.57 -27.50 -31.43 -31.43 0 0 0 0 0 0 -23.57 0 0 0 0 0

4 0 0 0 0 0 0 0 0 0 0 0 0 -8.82 -9.29 -8.82 -8.82 0 0 0 0 0 0 0 0
5 0 0 0 0 0 0 0 0 0 0 0 -14.14 -15.00 -15.86 -15.00 -15.00 0 0 0 0 0 0 0 0
6 0 0 0 0 0 0 0 0 0 0 0 -17.89 -18.67 -19.44 -18.67 -18.67 0 0 -17.31 0 0 0 0 0
7 0 0 0 0 0 0 0 0 0 0 0 0 -62.22 -68.06 -62.22 0 0 0 0 0 0 0 0 0
8 0 0 0 0 0 0 0 0 0 0 0 0 -8.37 -8.57 -8.37 0 0 0 0 0 0 0 0 0
9 0 0 0 0 0 0 0 0 0 0 -32.81 -32.81 -34.76 -36.46 -34.76 -34.76 0 0 0 0 0 0 0 0
10 0 0 0 0 0 0 0 0 0 0 0 0 -20.00 -20.71 -20.00 -20.00 0 0 0 0 0 0 0 0

Table 6.7. Optimal load schedule for in kW Test 2 (Scenario 1 with reduced load flexibility)
Interval Number
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24
1 0 0 0 0 0 0 0 -10.94 0 -10.94 0 0 -10.94 0 0 -10.94 0 0 0 0 0 0 0 0
2 0 0 0 0 0 0 0 0 0 0 0 0 0 -11.91 -11.57 0 0 0 0 0 0 0 0 0
Building Number

3 0 0 0 0 0 0 0 0 -23.57 -27.50 -31.43 -31.43 0 0 0 0 0 0 -23.57 0 0 0 0 0

4 0 0 0 0 0 0 0 0 0 0 0 0 -8.82 -9.29 -8.82 -8.82 0 0 0 0 0 0 0 0
5 0 0 0 0 0 0 0 0 0 0 0 -14.14 -15.00 -15.86 -15.00 -15.00 0 0 0 0 0 0 0 0
6 0 0 0 0 0 0 0 0 0 0 0 -17.89 -18.67 -19.44 -18.67 -18.67 0 0 -17.31 0 0 0 0 0
7 0 0 0 0 0 0 0 0 0 0 0 0 -18.67 -20.42 -18.67 0 0 0 0 0 0 0 0 0
8 0 0 0 0 0 0 0 0 0 0 0 0 -8.37 -8.57 -8.37 0 0 0 0 0 0 0 0 0
9 0 0 0 0 0 0 0 0 0 0 -32.81 -32.81 -34.76 -36.46 -34.76 -34.76 0 0 0 0 0 0 0 0
10 0 0 0 0 0 0 0 0 0 0 0 0 -20.00 -20.71 -20.00 -20.00 0 0 0 0 0 0 0 0
 124

Figure 6.5. Dispatch schedule for buildings 7 with decreasing load flexibility (No dispatch during the
intervals not shown)

 Test 3: Examine the impact of load footprint to the Scenario 1 dispatch schedule

For this test the load footprint of building 7 is varied. The nominal operating point across the

whole day is reduced to 75%, 50%, and 25% of the normal level. No other changes are

made. The complete load schedule is omitted for this test as the overall results are very

similar to Table 6.7. Several observations about the impact of varying the load footprint are

made:

o The savings is reduced from $33212.00 to $28512 as the load footprint goes

down. The response is similar to Test 2 where load flexibility is decreased. This

is demonstrated in Figure 6.6

 125

Figure 6.6 Dispatch schedule for buildings 7 with decreasing load footprint (No dispatch during the intervals
not shown

 Test 4: Compare dynamic pricing (Scenario 2) vs constant pricing (Scenario 1)

The optimal dispatch schedule for scenario 2 is shown in Table 6.9. The total savings

achieved through this dispatch schedule is $47164.00. Several key observations can be made

about this schedule:

o Comparing the optimal schedule under dynamic pricing (Table 6.9) to the

optimal schedule under constant pricing (Table 6.6) the savings increases by

$28496 with dynamic pricing. Simply observing the price curve in Figure 6.4

explains this savings.

o More interesting than the overall savings, the timing of the dispatch schedule

changes drastically with dynamic pricing. This shift in dispatch timing can be

seen in Table 6.9. Load is shed during periods of higher prices even though
 126
these periods often have lower load available for dispatch. In fact, the

total demand shed for Scenario 2 is 881.38 kW compared to 931.00 kW for

Scenario 1. This demonstrates one way dynamic pricing can incentive

consumer participation.

 Test 5: Find the optimal dispatch schedule for Scenario 3 in the deterministic case

This is the first test where a market entry constraint is present. In this test, PtThresh  150kW .

The optimal dispatch schedule for scenario 3 is shown in Table 6.8. The total savings

achieved through this dispatch schedule is $71417. Several key observations can be made

about this optimal schedule:

o As expected, the addition of DR revenue greatly increases the cost savings.

Breaking down the savings, it can be seen that the overall load reduction saves

$31564 and DR revenue totals $39853.

o The incentive to meet the DR revenue threshold is reflected in the load

schedule. Looking at Table 6.8 compared to Table 6.6 it is apparent that the

optimal solution shifts load reductions to intervals where the total can meet

the threshold and earn revenue.

o Similar to the results for Test 4, the overall load reduction is less for Scenario

3 than Scenario 1 (880kW vs 931.00 kW). This underlines the point that

additional cost saving incentives drive the optimal load schedule.

 127
Table 6.8. Optimal load schedule in kW for Test 4 (Scenario 2)
Interval Number
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24
1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 -10.06 -10.94 -9.63 -9.19 0 0 0 0 0 0
2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 -11.23 -10.55 0 0 0 0 0 0
Building Number

3 0 0 0 0 0 0 0 0 0 0 -31.43 -31.43 0 0 0 0 0 -23.6 -23.57 -22.39 0 0 0 0

4 0 0 0 0 0 0 0 0 0 0 0 0 0 0 -8.82 -8.82 -8.45 -7.89 0 0 0 0 0 0
5 0 0 0 0 0 0 0 0 0 0 0 0 0 -15.86 -15.00 -15.00 -14.14 -12.86 0 0 0 0 0 0
6 0 0 0 0 0 0 0 0 0 0 0 0 0 -19.44 -18.67 -18.67 -17.89 -17.31 -17.31 0 0 0 0 0
7 0 0 0 0 0 0 0 0 0 0 0 0 0.0 0 0 -62.22 -59.31 -50.56 0 0 0 0 0 0
8 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 -8.37 -8.00 -7.29 0 0 0 0 0 0
9 0 0 0 0 0 0 0 0 0 0 0 0 -34.76 -36.46 -34.76 -34.76 -32.8 -29.5 0 0 0 0 0 0
10 0 0 0 0 0 0 0 0 0 0 0 0 0.00 0.00 -20.00 -20.00 -19.5 -17.9 0 0 0 0 0 0

Table 6.9 Optimal load schedule in kW for Test 5 (Scenario 3 – Pthresh=150kW)

Interval Number
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24
1 0 0 0 0 0 0 0 0 0 0 -10.50 -10.50 0 0 0 -10.94 0 -9.19 0 0 0 0 0 0
2 0 0 0 0 0 0 0 0 0 0 -11.23 -11.23 0 0 0 0 0 0 0 0 0 0 0 0
Building Number

3 0 0 0 0 0 0 0 0 0 -27.50 -31.43 -31.43 0 0 0 0 0 -23.57 -23.57 0 0 0 0 0

4 0 0 0 0 0 0 0 0 0 0 -8.26 -8.45 0 0 0 0 -8.45 -7.89 0 0 0 0 0 0
5 0 0 0 0 0 0 0 0 0 0 -14.14 -14.14 0 0 0 -15.00 -14.14 -12.86 0 0 0 0 0 0
6 0 0 0 0 0 0 0 0 0 0 -17.89 -17.89 0 0 0 -18.67 -17.89 -17.31 -17.31 0 0 0 0 0
7 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 -62.22 -59.31 -50.56 0 0 0 0 0 0
8 0 0 0 0 0 0 0 0 0 0 -8.00 -8.00 0 0 0 0 -8.00 0 0 0 0 0 0 0
9 0 0 0 0 0 0 0 0 0 0 -32.81 -32.81 0 -36.46 0 -34.76 -32.8 -29.53 0 0 0 0 0 0
10 0 0 0 0 0 0 0 0 0 0 -19.46 -19.46 0 0 0 -20.00 -19.46 0 0 0 0 0 0 0
 128
 Test 6: Find the impact of changing the market entry constraint threshold

In this test, the market entry constraint threshold is cut in half ( PtThresh  75kW ). The optimal

dispatch schedule is shown in Table 6.10. The total savings achieved through this dispatch

schedule is $77538. Reducing the market entry constraint threshold allows for more

opportunities to qualify for DR revenue (7 intervals vs 5 intervals). Overall savings increases

as the dispatch schedule shifts to make the most of the lower market entry threshold.

 Test 7: Compare Dynamic pricing (Scenario 4) vs Constant Pricing (Scenario 3) with

DR revenue

The optimal dispatch schedule for scenario 4 is shown in Table 6.11. The total savings

achieved through this dispatch schedule is $83666.66. Several key observations can be made

about this optimal schedule:

o This DSM scenario results in the largest savings. The combination of

dynamic pricing and DR revenue expectedly leads to the greatest savings.

o The optimal schedule is nearly identical between scenario 3 and scenario 4.

This can be attributed to the fact that higher prices occur during intervals

where it is lucrative to try and make the DR threshold.

 Test 8: Find the optimal dispatch schedule for Scenario 1 in the uncertain case

This is the first test to include uncertainty related to the load forecast. As a first cut, the error

parameter ξ will be held constant at 10kW for all buildings and all intervals. The resulting

load dispatch schedule is shown in Table 6.12.

o In this simple example, the dispatch schedule differs from Table 6.6 in all

intervals by +10kW (in other words, 10kW less load shed). While this seems

a trivial result, it is important to note that the results are showing the load shed

in relation to the baseline forecast, not just dispatched controllable load. This
 129
shows the expected total building load reduction ∆P. The load reduction

decreases by 410kW (from 931 to 521kW) compared to Scenario 1.

o For building 4 and 8, which have very small load footprints, the forecast

uncertainty results in an expected increase in total building load. This is an

important result that demonstrates the potential impact of high forecast

uncertainty.

 Test 9: Observe cost savings as forecast uncertainty varies from best case to worst case

In order to see how the overall expected cost saving are impacted with increasing forecast

uncertainty the error parameter ξ will be varied over a range of values. Unlike the previous

test, the load footprint of the building will be taken into account by scaling ξ. For each

building m, ξ will be varied at each interval t from -10% to 10% of the forecasted building

demand. Total campus expected cost savings are plotted against the range of ξ values in

Figure 6.7

o Expected savings decreases linearly (due to the objective function). There is a

point at slightly less than 5% uncertainty where the expected savings becomes

an expected expenditure.

o This plot shows both the worst case and best case in total building load

dispatch. When the forecast overestimates the load (ξ<0) then the consumer

“sheds” extra load. However in the worst case, the amount of dispatchable

controllable load cannot overcome the underestimates of the load.

 130
4
x 10
12

10

8
Total Expected Campus ($)

Savings
0

Expenditure
-2

-4
-10 -8 -6 -4 -2 0 2 4 6 8 10
Load Uncertainty (% of Forecast)

Figure 6.7 Cost savings as a function of load uncertainty. Break even line shown in blue

 Test 11: Examine the impact of forecast uncertainty to the Scenario 3 dispatch

schedule

For each building m, the load uncertainty parameter ξ will be varied at each interval t from -

10% to 10% of the forecasted building demand. The impact of the load uncertainty is

reflected in the number of intervals where the DR revenue threshold can be met

( PtThresh  100kW ). These results are shown in Figure 6.8. A small deviation from the perfect

forecast can results in a great misunderstanding of how often the DR revenue constraint can

be met. A 3% underestimation of the forecast gives the impression that the DR revenue

constraint can never be met, while a 3% overestimation makes it look as though it can be met

during all intervals.

 131

25

Increasing underestimation
DR Constraint Met (# of Intervals) 20

15

10

Increasing overestimation

-10 -8 -6 -4 -2 0 2 4 6 8 10
Load Uncertainty (% of Forecast)

Figure 6.8. Decreasing number of intervals where the DR revenue constraint can be met
 132

Table 6.10 Optimal load schedule in kW for Test 6 (Scenario 3 – Pthresh=75kW)

Interval Number
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24
1 0 0 0 0 0 0 0 0 0 0 0 0 -10.9 0 0 -10.94 -9.63 0 -8.75 0 0 0 0 0
2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 -11.57 -11.23 0 0 0 0 0 0 0
Building Number

3 0 0 0 0 0 0 0 0 -23.57 0 0 0 0 0 0 0 -31.43 -23.57 -23.57 -22.39 0 0 0 0

4 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 -8.82 -8.45 -7.89 -7.89 0 0 0 0 0
5 0 0 0 0 0 0 0 0 0 0 0 0 -15.00 0 -15.00 -15.00 -14.14 -12.86 0 0 0 0 0 0
6 0 0 0 0 0 0 0 0 0 0 0 0 -18.67 0 -18.67 -18.67 -17.89 -17.31 -17.31 0 0 0 0 0
7 0 0 0 0 0 0 0 0 0 0 0 0 0 -68.06 0 -62.22 -59.31 0.00 0 0 0 0 0 0
8 0 0 0 0 0 0 0 0 0 0 0 0 0 0 -8.37 -8.37 -8.00 0 0 0 0 0 0 0
9 0 0 0 0 0 0 0 0 0 0 0 0 -34.76 -36.46 -34.76 -34.76 -32.81 -29.53 0 0 0 0 0 0
10 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 -20.00 -19.46 -17.86 -17.86 0 0 0 0 0

Table 6.11 Optimal load schedule in kW for Test 6 (Scenario 4)

Interval Number
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24
1 0 0 0 0 0 0 0 0 0 0 -10.50 -10.50 0 0 0 -10.94 -9.63 0 0 0 0 0 0 0
2 0 0 0 0 0 0 0 0 0 0 -11.23 -11.23 0 0 0 0 0 0 0 0 0 0 0 0
Building Number

3 0 0 0 0 0 0 0 0 0 0 -31.43 -31.43 0 0 0 0 0 -23.57 -23.57 -22.39 0 0 0 0

4 0 0 0 0 0 0 0 0 0 0 -8.26 -8.45 0 0 0 -8.82 -8.45 0 0 0 0 0 0 0
5 0 0 0 0 0 0 0 0 0 0 -14.14 -14.14 0 0 0 -15.00 -14.14 -12.86 0 0 0 0 0 0
6 0 0 0 0 0 0 0 0 0 0 -17.89 -17.89 0 0 0 -18.67 -17.89 -17.31 -17.31 0 0 0 0 0
7 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 -62.22 -59.31 -50.56 0 0 0 0 0 0
8 0 0 0 0 0 0 0 0 0 0 -8.00 -8.00 0 0 0 0 -8.00 0 0 0 0 0 0 0
9 0 0 0 0 0 0 0 0 0 0 -32.81 -32.81 0 0 -34.76 -34.76 -32.81 -29.53 0 0 0 0 0 0
10 0 0 0 0 0 0 0 0 0 0 -19.46 -19.46 0 0 0 0 -19.46 -17.86 0 0 0 0 0 0
 133

Table 6.12 Optimal load schedule in kW for Test 8: +10kW uniform load forecast uncertainty
Interval Number
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24
1 0 0 0 0 0 0 0 -0.94 0 -0.94 0 0 -0.94 0 0 -0.94 0 0 0 0 0 0 0 0
2 0 0 0 0 0 0 0 0 0 0 0 0 0 -1.91 -1.57 0 0 0 0 0 0 0 0 0
Building Number

3 0 0 0 0 0 0 0 0 -13.57 -17.50 -21.43 -21.43 0 0 0 0 0 0 -13.57 0 0 0 0 0

4 0 0 0 0 0 0 0 0 0 0 0 0 1.18 0.71 1.18 1.18 0 0 0 0 0 0 0 0
5 0 0 0 0 0 0 0 0 0 0 0 -4.14 -5.00 -5.86 -5.00 -5.00 0 0 0 0 0 0 0 0
6 0 0 0 0 0 0 0 0 0 0 0 -7.89 -8.67 -9.44 -8.67 -8.67 0 0 -17.31 0 0 0 0 0
7 0 0 0 0 0 0 0 0 0 0 0 0 -52.22 -58.06 0.00 -52.2 0 0 0 0 0 0 0 0
8 0 0 0 0 0 0 0 0 0 0 0 0 1.629 1.429 0 1.629 0 0 0 0 0 0 0 0
9 0 0 0 0 0 0 0 0 0 0 -22.8 -22.81 -24.76 -26.46 -24.76 -24.76 0 0 0 0 0 0 0 0
10 0 0 0 0 0 0 0 0 0 0 0 0 -10.00 -10.71 -10.00 -10.00 0 0 0 0 0 0 0 0
 134

7. CONCLUSIONS AND FUTURE WORK

7.1. OVERVIEW

This concluding chapter will cover two final items:

 A summary of the research contributions of this work will be presented

 A discussion of how the work presented in this thesis can be extended in the

future

7.2. SUMMARY OF RESEARCH CONTRIBUTIONS

The overall vision for this work was to study demand side management (DSM) using

commercial buildings with a focus on addressing practical issues of interest to the end-user.

Specifically these contributions fall into two main areas. The first focuses on the forecasting of

building electrical demand. The second is concerned with methods for scheduling commercial

loads as demand side resources.

In this work a case study involving Drexel University building demand provides the basis

for studying building specific load forecasting techniques. Four popular methods for load

forecasting are implemented and tested on actual building data. These traditional approaches are

enhanced by measurements collected via a BMS which is a new contribution to this area. In

addition to point forecasts and method for generating probabilistic forecasts of building demand

is presented. These studies demonstrate the overall improved performance (in terms of accuracy,

consistency, and overall skill) when building measurements are included in the models.

Additionally these tests have examined some performance characteristics that are important

understand how building-specific behavior may be valuable in future efforts to better forecast

building loads.
 135
In addition to the forecasting studies, the problem of scheduling building electrical

loads for DSM has been presented. A new model for the electrical demand of an HVAC chiller

was derived from Drexel building data, leveraging existing dynamic load modeling techniques.

This model captures the dynamics exhibited by the HVAC system that arises from the strong link

between the thermal and electrical behavior of the building and has the flexibility to be used

across a number of time windows (i.e.- static vs transient studies). This model has the added

benefit of being well suited for integration in a load scheduling algorithm. To that end, several

load scheduling optimizations problems have been presented for determining the optimal

dispatch of controllable building loads across a multi-building campus. These formulations

cover several different potential DSM programs, including simple minimization of usage,

dynamic pricing, and the potential to bid DR resources into the market. Three parameters, load

footprint, load flexibility, and load margin, have been introduced to help analyze building

dispatch capabilities. The analysis performed has identified the important sensitivities of the

solution to variations in model parameters and DSM scenarios. Lastly, the uncertainty related to

building forecasts is introduced into the problem. To get a feel for the impact such uncertainty

has on the dispatch schedule, potential best and worst case uncertainty scenarios are presented.

7.3. FUTURE WORK

This thesis has laid the foundation for several areas of future research. First, it would be

beneficial to develop a load forecasting model that is “best” for building-specific applications.

This model would ideally account for the building-specific characteristics that have been

demonstrated in this thesis as being highly impactful. Some preliminary work has been done in

this area in [119]. In that work, a forecast model is developed that considers resampling from

established covariance structures. Models of temporal dependencies constructed from both

Gaussian and non-Gaussian copulas are used in tandem with a boosting procedure to improve
 136
forecast accuracy. This approach captures some of the temporal dependencies of building

loads but could be further enhanced with building-specific information.

Additionally it would be helpful to study occupancy behavior as a possible input to the

load forecast model. Data on occupancy levels and other building usage information may lead to

better load characterization. This information would need to be balanced with keeping the

models from becoming too complex to use.

There is also a number of aspects of the load scheduling problem that can enhanced.

First is a more systematic way of including forecast uncertainty in the optimization formulation.

The work in this thesis has laid the groundwork for a scheduling method that provides practical

schedules that are sensitive to real-world building constraints. However the uncertainty of the

forecast can be better balanced in terms of the conservativeness of the solution. In addition to

the forecast uncertainty it would be good to consider price forecast uncertainty. This thesis

showed how price-dependent the dispatch schedule is under various DSM programs. However a

forecast of day-ahead prices will, just like the load forecast, include some uncertainty. Lastly, it

would be interesting to explore extensions of these formulations into other DSM scenarios. For

example, this can include the ancillary services market, where the model could try and balance

the benefits of load reduction with potential rewards for raising building load to support the

power system operation. Real-time load dispatch is also an important possible extension where

the full dynamics of the load would need to be incorporated to model the scenario accurately.
 137

List of References
[1] "H.R. 6 (110th): Energy Independence and Security Act of 2007," 2007. [Online]. Available:
https://www.govtrack.us/congress/bills/110/hr6. [Accessed 2010].

[2] P. Cappers, C. Goldman and D. Kathan, "Demand response in U.S. electricity markets: Empirical
evidence," Energy, vol. 35, no. 4, pp. 1526-1535, 2010.

[3] S. Borenstein, "The trouble with electricity markets: Understanding California's restructuring
disaster," Journal of Economic Perspectives, vol. 16, no. 1, pp. 191-211, 2002.

[4] U.S. Congress, "Energy Policy Act of 2005," 2005. [Online]. Available:
https://www.gpo.gov/fdsys/pkg/BILLS-109hr6enr/pdf/BILLS-109hr6enr.pdf. [Accessed 2016].

[5] FERC, "Order No. 719," 2008. [Online]. Available: https://www.ferc.gov/whats-new/comm-

meet/2008/101608/E-1.pdf. [Accessed 2016].

[6] FERC, "Order No. 755," 2011. [Online]. Available: https://www.ferc.gov/whats-new/comm-

meet/2011/102011/E-28.pdf. [Accessed 2016].

[7] FERC, "Order No. 745," 2011. [Online]. Available:

https://www.ferc.gov/EventCalendar/Files/20110315105757-RM10-17-000.pdf. [Accessed 2016].

[8] FERC, "A national assessment of demand response potential," Prepared by the Brattle Group;
Freeman, Sullivan & Co.; and Global Energy Partners, 2009. [Online]. Available:
https://www.ferc.gov/legal/staff-reports/06-09-demand-response.pdf. [Accessed 2016].

[9] U.S. DOE, "Benefits of demand response in electricity markets and recommendations for
achieving them," Technical report to the United States Congress, 2006. [Online]. Available:
http://energy.gov/sites/prod/files/oeprod/DocumentsandMedia/DOE_Benefits_of_Demand_Response_in_E
lectricity_Markets_and_Recommendations_for_Achieving_Them_Report_to_Congress.pdf. [Accessed
2016].

[10] M. Albadi and E. El-Saadany, "A Summary of Demand Response in Electricity Markets," Electric
Power Systems Research, vol. 78, pp. 1989-1996, 2008.

[11] EPRI, "The green grid: Energy savings and carbon reductions enabled by a smart grid.," Electric
Power Research Institute, 2008.

[12] D. Chassin and J. Fuller, "On the Equilibrium Dynamics of Demand Response in Thermostatic
Loads," in Hawaii International Conference on System Sciences (HICSS), 2011.

[13] D. Kirschen, "Demand-side View of Electricity Markets," IEEE Transactions on Power Systems,
vol. 100, no. 2, pp. 1104-1118, 2003.

[14] N. Liu, "An Evaluation of the HVAC Load Potential for Providing Load Balancing Service,"
IEEE Transactions on Smart Grid, vol. 3, no. 3, pp. 1263-1270, 2012.

[15] D. S. Callaway, "Tapping the energy storage potential in electric loads to deliver load following
and regulation, with application to wind energy," Energy Conversion and Management, vol. 50, no. 5, pp.
1389-1400, 2009.

[16] NERC, "Long-term reliability assessment 2008-2017," North American Electric Reliability
Corporation, 2008.

[17] C. De Jonghe and B. F. B. R. Hobbs, "Optimal Generation Mix With Short-Term Demand
Response and Wind Penetration," IEEE Transactions on Power Systems, vol. 27, no. 2, pp. 830-839, 2012.
 138

[18] J. Eto, "Innovative developments in load as a reliability resource," in Proceedings of the IEEE PES
Winter Meeting, 2002.

[19] DOE/EIA, "Annual Energy Outlook 2015 with projections to 2040," U.S. Energy Information
Administration (EIA), 2015.

[20] D. Callaway and I. Hiskens, "Achieving controllability of electric loads," Proceedings of the
IEEE, vol. 99, no. 1, pp. 184-199, 2011.

[21] PG&E, "Explore the PG&E Time-of-Use plans," 2016. [Online]. Available:
https://www.pge.com/en_US/residential/rate-plans/rate-plan-options/time-of-use-base-plan/time-of-use-
plan.page. [Accessed 2016].

[22] PG&E, "Find out if Peak Day Pricing is right for your business," PG&E, 2016. [Online].
Available: https://www.pge.com/en_US/business/rate-plans/rate-plans/peak-day-pricing/peak-day-
pricing.page. [Accessed 2016].

[23] PJM, "PJM Manual 11: Energy & Ancillary Services Market Operations," August 2016. [Online].
Available: http://www.pjm.com/~/media/documents/manuals/m11.ashx. [Accessed 2016].

[24] N. Motegi, M. W. D. Piette and S. X. P. Kiliccote, "Introduction to Commercial Building Control

Strategies and Techniques for Demand Response," California Energy Commission, PIER. LBNL Report
Number 59975, 2006.

[25] J. Berardino and C. Nwankpa, "Statistics of building-specific load forecasting models," in

Proceedings of the North American Power Symposium (NAPS), 2013.

[26] J. Berardino and C. Nwankpa, "Dynamic Load Modeling of an HVAC Chiller for Demand
Response Applications," in Proceedings of the IEEE Smart Grid Communications Conference, 2010.

[27] D. Hill, "Nonlinear Dynamic Load Models with Recovery for voltage Stability Studies," IEEE
Transactions on Power Systems, vol. 8, no. 1, pp. 166-176, 1993.

[28] J. Lam and e. al, "An analysis of electricity end-use in air-conditioned office buildings in Hong
Kong," Building and Environment, vol. 38, no. 3, pp. 493-498, 2003.

[29] A. R. Florita, L. J. Brackney, O. T. P. and J. Robertson, "Classification of Commercial Building

Electrical Demand Profiles for Energy Storage Applications," Journal of Solar Energy Engineering, vol.
135, no. 3, 2013.

[30] J. Mathieu, P. N. Price, S. Kiliccote and M. A. Piette, "Quantifying Changes in Building

Electricity Use, With Application to Demand Response," IEEE Transactions on Smart Grid, vol. 2, no. 3,
pp. 507-518, 2011.

[31] CAISO, "Baselines for Retail Demand Response," 2009. [Online]. Available:
https://www.caiso.com/Documents/Presentation-Baselines_RetailDemandResponsePrograms.pdf.
[Accessed 2016].

[32] M. Goldberg and G. Agnew, "Protocol development for demand response calculation - findings
and recommendations," California Energy Commission CEC-400-02-017F, 2003.

[33] K. Coughlin, M. Piette, C. Goldman and S. Kiliccote, "Statistical analysis of baseline load models
for non-residential buildings," Energy and Buildings, vol. 41, no. 4, 2009.

[34] S. Kiliccote, M. Piette and D. Hansen, "Advanced controls and com- munications for demand
response and energy efficiency in commercial buildings," in Proceedings of Second Carnegie Mellon
Conference in Electric Power Systems, 2006.
 139

[35] S. Braithwait, D. Hansen and J. Reaser, "2009 load impact evaluation of California statewide
critical-peak pricing rates for non-residential customers: Ex post and ex ante report," Christensen
Associates Energy Consulting CALMAC Study ID: SDG0244.01, 2010.

[36] KEMA, "PJM Empirical Analysis of Demand Response Baseline Methods," Prepared for the PJM
Market Implementation Committee, 2011.

[37] J. L. Mathieu, D. S. Callaway and S. Kiliccote, "Examining Uncertainty in Demand Response

Baseline Models and Variability in Automated Responses to Dynamic Pricing," in Proceedings of the 50th
IEEE Conference on Decision and Control and European Control Conference, 2011.

[38] J. Berardino and C. Nwankpa, "Interval-specific building load models for demand resouce
planning," in Proceedings of the IEEE PowerCon 2012 Conference, Auckland, 2012.

[39] J. Berardino and C. Nwankpa, "Inclusion of temporal effects in forecasting building electrical
loads for demand resource planning," in Proceedings of the IEEE PowerTech 2013 Conference, Grenoble,
2013.

[40] D. Palchak, S. Suryanarayanan and D. Zimmerle, "An Artificial Neural Network in Short-Term
Electrical Load Forecasting of a University Campus: A Case Study," Journal of Energy Resources
Technology, vol. 135, no. 3, 2013.

[41] W. Mai, C. Y. Chung, T. Wu and H. Huang, "Electric load forecasting for large office building
based on radial basis function neural network," in Proceedings of the 2014 IEEE PES General Meeting,
National Harbor, MD, 2014.

[42] H. Chitsaz, H. Shker, H. Zareipour, D. Wood and N. Amjady, "Short-term electricity load
forecasting of buildings in microgrids," Energy and Buildings, vol. 99, pp. 50-60, 2015.

[43] P. A. Gonzalez and J. M. Zamarreno, "Prediction of hourly energy consumption in buildings based
on a feedback artificial neural network," Energy and Buildings, vol. 37, no. 6, pp. 595-601, 2005.

[44] S. Karatasou, M. Santamouris and V. Geros, "Modeling and predicting building's energy use with
artificial neural networks: Methods and results," Energy and Buildings, vol. 38, no. 8, pp. 949-958, 2006.

[45] P. Kou and Gao, "A sparse heteroscedastic model for the probabilistic load forecasting in energy-
intensive enterprises," International Journal of Electric Power Energy Systems, vol. 55, pp. 144-154, 2014.

[46] B. Dong, C. Cao and S. E. Lee, "Applying support vector machines to predict building energy
consumption in tropical region," Energy and Buildings, vol. 37, no. 5, pp. 545-553, 2005.

[47] J. Massana, C. Pous, L. Burgas, J. Melendez and J. Colomer, "Short-term load forecasting in a
non-residential building contrasting models and attributes," Energy and Buildings, vol. 92, pp. 322-330,
2015.

[48] C. E. Borges, Y. K. Penya and I. Fernandez, "Optimal combined short-term building load
forecasting," in Proceedings of the 2011 IEEE PES Innovative Smart Grid Technologies (ISGT)
Conference, 2001.

[49] J. G. Jetcheva, M. Majidpour and W.-P. Chenwei, "Neural Network Model Ensembles for
Building-Level Electricity Load Forecasts," Energy and Buildings, vol. 84, pp. 214-223, 2014.

[50] A. Afram and F. Janabi-Sharifi, "Review of modeling methods for HVAC systems," Applied
Thermal Engineering, vol. 67, no. 1-2, pp. 507-519, 2014.
 140

[51] A. Khodaei, M. Shahidehpour and S. Bahramirad, "SCUC With Hourly Demand Response
Considering Intertemporal Load Characteristics," IEEE Transactions on Smart Grid, vol. 2, no. 2, pp. 564-
571, 2011.

[52] M. Parvania and M. Fotuhi-firuzabad, "Demand Response Scheduling by Stochastic SCUC,"

IEEE Transactions on Smart Grid, vol. 1, no. 1, pp. 89-98, 2010.

[53] P. Palensky and D. Dietrich, "Demand Side Management : Demand Response , Intelligent Energy
Systems , and Smart Loads," IEEE Transactions on Industrial Informatics, vol. 7, no. 3, pp. 381-388, 2011.

[54] J. S. Vardakas, N. Zorba and C. V. Verikoukis, "A Survey on demand response programs in smart
grids: pricing methods and optimization algorithms," IEEE Communications Surveys & Tutorials, vol. 17,
no. 1, pp. 152-178, 2014.

[55] R. Deng, Z. Yang, M.-Y. Chow and J. Chen, "A Survey on Demand Response in Smart Grids:
Mathematical Models and Approaches," IEEE Transactions on Industrial Informatics, vol. 11, no. 3, 2015.

[56] S. Bashash and H. Fathy, ""Modeling and control insights into demand-side energy management
through setpoint control of thermostatic loads," in Proceedings of the American Control Conference
(ACC), 2011.

[57] N. Lu and D. Chassin, "A state queuing model of thermostatically controlled appliances," IEEE
Transactions on Power Systems, vol. 19, no. 3, pp. 1666-1673, 2004.

[58] J. Mathieu and D. Callaway, "State estimation and control of heterogeneous thermostatically
controlled loads for load following," in Proceedings of the Hawaii International Conference on Systems
Science, 2012.

[59] W. Burke and D. Auslander, "Development of an HVAC Load Model for Aggregates of Homes,"
California Energy Commission Multi-Campus Award No. C-08-19, 2010.

[60] I. Moghram and S. Rahman, "Analysis of five short-term load forecasting techniques," IEEE
Transactions on Power Systems, vol. 4, no. 4, pp. 1484-1491, 1989.

[61] H. Alfares and M. Nazeeruddin, "Electric load forecasting: literature survey and classification of
methods," International Journal of System Science, vol. 33, no. 1, pp. 23-34, 2002.

[62] H. Hahn, S. Meyer-Nieburg and S. Pickl, "Electric load forecasting methods: tools for decision
making," European Journal of Operational Research, vol. 199, no. 3, pp. 902-907, 2009.

[63] T. Hong, "Short term electric load forecasting," Ph.D. dissertation, North Carolina State
University, 2010.

[64] H. Hippert, C. Pedreira and R. Souza, "Neural networks for short-term load forecasting: a review
and evaluation," IEEE Transactions on Power Systems, vol. 16, no. 1, pp. 44-55, 2001.

[65] K. Nose-Filho, A. D. P. Lotufo and C. R. Minussi, "Short-Term Multinodal Load Forecasting

Using a Modified General Regression Neural Network," IEEE Transactions on Power Delivery, vol. 26, no.
4, pp. 2862-2869, 2011.

[66] S. Li, P. Wang and L. Goel, "A Novel Wavelet-Based Ensemble Method for Short-Term Load
Forecasting with Hybrid Neural Networks and Feature Selection," IEEE Transactions on Power Systems,
vol. 31, no. 3, pp. 1788 - 1798, 2016.

[67] A. Deihimi and H. Showkati, "Application of echo state networks in short-term electric load
forecasting," Energy, vol. 39, no. 1, pp. 327-340, 2012.
 141

[68] W.-C. Chu, Y.-P. Chen, Z.-W. Xu and W.-J. Lee, "Multiregion Short-Term Load Forecasting in
Consideration of HI and Load/Weather Diversity," IEEE Transactions on Industry Applications, vol. 47,
no. 1, pp. 232-237, 2011.

[69] N. Amjady, "Short-Term Bus Load Forecasting of Power Systems by a New Hybrid Method,"
IEEE Transactions on Power Systems, vol. 22, no. 1, pp. 333-341, 2007.

[70] R.-J. Wai, Y.-C. Chen and Y.-R. Chang, "Short-term load forecasting via fuzzy neural network
with varied learning rates," in Proceedings of the 2011 IEEE International Conference on Fuzzy Systems
(FUZZ-IEEE 2011), 2011.

[71] B.-J. Chen, M.-W. Chang and C.-J. Lin, "Load Forecasting Using Support Vector Machines: A
Study on EUNITE Competition 2001," IEEE Transactions on Power Systems, vol. 19, no. 4, pp. 1821-
1830, 2004.

[72] D. Niu, Y. Wang and D. D. Wu, "Power load forecasting using support vector machine and ant
colony optimization," Expert Systems with Applications, vol. 37, no. 3, pp. 2531-2539, 2010.

[73] S. Fan and L. Chen, "Short-term load forecasting based on an adaptive hybrid method," IEEE
Transactions on Power Systems, vol. 21, no. 1, pp. 392-401, 2006.

[74] A. Hinojosa and V. Hoese, "Short-term load forecasting using fuzzy inductive reasoning and
evolutionary algorithms," IEEE Transactions on Power Systems, vol. 25, p. 565–574, 2010.

[75] S. Fan and R. J. Hyndman, "Short-term load forecasting based on a semi-parametric additive
model," IEEE Transactions on Power Systems, vol. 27, no. 1, pp. 134-141, 2012.

[76] Y. Goude, R. Nedellec and N. Kong, "Local short and middle term electricity load forecasting
with semi-parametric additive models," IEEE Transactions on Smart Grid, vol. 5, no. 1, pp. 440-446, 2014.

[77] R. Nedellec, J. Cugliari and Y. Goude, "Gefcom2012: Electricity load forecasting and backcasting
with semi-parametric models," International Journal of Forecasting, vol. 30, no. 2, p. 375–381, 2014.

[78] T. Hong and P. F. S. Pinson, "Global Energy Forecasting Competition 2012," International Journal
of Forecasting, vol. 30, no. 2, pp. 357-363, 2014.

[79] M. Kutner, C. Nachtsheim, J. Neter and W. Li, Applied Linear Statistical Models, McGraw-
Hill/Irwin, 2004.

[80] C. Cortes and V. Vapnik, "Support-vector Network," Machine Learning, vol. 20, pp. 273-297,
1995.

[81] C.-C. Chang and C.-J. Lin, "LIBSVM: A Library for Support Vector Machines," 2001. [Online].
Available: https://www.csie.ntu.edu.tw/~cjlin/libsvm/. [Accessed 2016].

[82] S. Arlot and A. Celisse, "A survey of cross-validation procedures for model selection," Statistical
Surveys, vol. 4, pp. 40-79, 2010.

[83] T. Hastie and R. Tibshirani, Generalized Additive Models, London, U.K.: Chapman & Hall, 2006.

[84] S. Wood, "mgcv: GAMs and generalized ridge regression of R," R News, vol. 1, no. 2, pp. 20-25,
2001.

[85] S. Wood, "Stable and efficient multiple smoothing parameter estimation for generalized additive
models," Journal of the American Statistical Association, vol. 99, p. 673–686, 2004.
 142

[86] S. Wood, "Fast stable restricted maximum likelihood and marginal likelihood estimation of
semiparametric generalized linear models," Journal of the Royal Statistical Society B, vol. 73, no. 1, p. 3–
36, 2011.

[87] The MathWorks Inc., MATLAB R2012b, Natick, MA, 2012.

[88] S. Katipamula, R. Underhill, J. Goddard, G. Liu and S. Eaton, "Instructor Manual: Re-tuning
Large Laboratory (PNNL-SA-82896)," U. S. Department of Energy, Pacific Northwest National
Laboratory , 2011.

[89] M. Muthalib and C. Nwankpa, "Physically Based Building Load Model for Electric Grid
Operation and Planning," IEEE Transactions on Smart Grid, 2016.

[90] P. Pinson, "Estimation of the uncertainty in wind power forecasting," Ph.D. dissertation, Centre
Energétique et Procédés–Ecole des Mines de Paris, Paris, France, 2006.

[91] G. Verschuuren, "Confidence Intervals vs. Prediction Intervals," 2014. [Online]. Available:
https://www.youtube.com/watch?v=o0UESA3UZss. [Accessed 2016].

[92] T. Heskes, "Practical Confidence and Prediction Intervals," in Advances in Neural Information
Processing Systems 9, M. Mozer, M. Jordan and T. Petsche, Eds., Cambridge, MIT Press, 1997.

[93] L. Wu, M. Shahidehpour and T. Li, "Stochastic security-constrained unit commitment," IEEE
Transactions on Power Systems, vol. 22, no. 2, pp. 800-811, 2007.

[94] Y. Wang, Q. Xia and C. Kang, "Unit commitment with volatile node injections by using interval
optimization," IEEE Transactions on Power Systems, vol. 26, no. 3, pp. 1705-1713, 2011.

[95] J. Hoffer and P. Dorfner, "Reliability and production cost calculation with peak load forecast
uncertainty," International Journal of Electrical Power & Energy Systems, vol. 13, no. 4, pp. 223-229,
1991.

[96] G. Hamoud, "Probabilistic assessment of interconnection assistance between power systems,"

IEEE Transactions on Power Systems, vol. 13, no. 2, pp. 535-542, 1998.

[97] R. Billinton and D. Huang, "Effects of load forecast uncertainty on bulk electric system reliability
evaluation," IEEE Transactions on Power Systems, vol. 23, no. 2, pp. 418-425, 2008.

[98] T. Hong and S. Fan, "Probabilistic electric load forecasting: A tutorial review," International
Journal of Forecasting, vol. 32, pp. 914-938, 2016.

[99] P. Pinson, H. Nielsen, J. Moller, H. Madsen and G. Kariniotakis, "Non-parametric probabilisitic

forecasts of wind power: Required properties and evaluation," Wind Energy, vol. 10, no. 6, pp. 497-516,
2007.

[100] R. Weron, "Electricity price forecasting: A review of the state-of-the-art with a look into the
future," International Journal of Forecasting, vol. 30, no. 4, pp. 1030-1081, 2014.

[101] T. Hong, J. Wilson and J. Xie, "Long term probabilistic forecasting and normalization with hourly
information," IEEE Transactions on Smart Grid, vol. 5, no. 1, pp. 456-462, 2014.

[102] R. Hyndman and S. Fan, "Density Forecasting for long-term peak electricity demand," IEEE
Transactions on Power Systems, vol. 25, pp. 1142-1153, 2010.

[103] J. Bremnes, "Probabilistic wind power forecasts using local quantile regression," Wind Energy,
vol. 7, no. 1, pp. 47-54, 2004.
 143

[104] T. Hong, P. Pinson, S. Fan, H. Zareipour, A. Troccoli and R. Hyndman, "Probabilistic energy
forecasting: Global Energy Forecasting Competition 2014 and beyond," International Journal of
Forecasting, vol. 32, no. 3, pp. 896-913, 2016.

[105] R. Winkler, "A decision-theoretic approach to interval estimation," Journal of the American
Statistical Association, vol. 67, pp. 187-191, 1972.

[106] B. Liu, J. Nowotarski, T. Hong and R. Weron, "Probabilistic load forecasting via quantile
regression averaging on sister forecasts," IEEE Transactions on Smart Grid, In Press..

[107] P. F. Christoffersen, "Evaluating interval forecasts," International Economic Review, vol. 39, no.
4, p. 841–862, 1998.

[108] "IEEE Stability Special: Voltage Stability of Power Systems: Concepts, Analytical Tools and
Industry Experience," IEEE Special Publication, 1990.

[109] C. Taylor, Power System Voltage Stability, Electric Power Research Institute, McGraw-Hill,
1994.

[110] D. Hill, "Nonlinear dynamic load models with recovery for voltage stability studies," IEEE
Transactions on Power Systems, vol. 8, no. 1, pp. 166- 176, 1993.

[111] D. Karlsson and D. Hill, "Modeling and identification of nonlinear dynamic loads in power
systems," IEEE Transactions on Power Systems, vol. 9, no. 1, pp. 157-166, 1994.

[112] S. Casper, C. Nwankpa, R. Fischl, A. Devito and S. Readinger, "On substation tests for load
modeling," in Proceedings of the 26th North American Power Symposium (NAPS) Part II, Manhattan, KS,
1995.

[113] Y. Liang, R. Fischl, A. DeVito and S. Readinger, "Dynamic reactive load model," IEEE
Transactions on Power Systems, vol. 13, no. 4, pp. 1365-1372, 1998.

[114] IEEE Task Force on Load Model Representation for Voltage Stability Studies, "Bibliography on
Load Models for Power Flow and Dynamic Performance Simulation," IEEE Transactions on Power
Systems, vol. 10, no. 1, pp. 523-538, 1995.

[115] B.-K. Choi, H.-D. Chiang, Y. Li, H. Li, Y.-T. Chen, D.-H. Huang and M. Lauby, "Measurement-
Based Dynamic Load Models: derivation, comparison, and validation," IEEE Transactions on Power
Systems, vol. 21, no. 3, pp. 1276 - 1283, 2006.

[116] I. R. Navarro, "Dynamic Load Models for Power Systems: Estimation of Time-Varying
Parameters During Normal Operation," Ph.D dissertation, Department of Industrial Electrical Engineering
and Automation, Lund University, Lund, Sweden, 2002.

[117] PJM, "Hourly Real-Time & Day-Ahead LMP," [Online]. Available:

http://www.pjm.com/markets-and-operations/energy/real-time/monthlylmp.aspx. [Accessed 2016].

[118] J. Currie and D. Wilson, "Opti: Lowering the Barrier Between Open Source Optimizers and the
Industrial MATLAB User," in Foundations of Computer-Aided Process Operations, Savannah, Georgia,
2012.

[119] B. Stephen, J. Shaw, J. Berardino, S. Galloway and C. Nwankpa, "Intra-day Error Dependency
Boosting for Short Term Building and LV Network Load Forecast Models," IEEE Transactions on Power
Systems.
144
 145
APPENDICES
 146
Appendix A. LIST OF NOMENCLATURE

In order of appearance:

Chapter 3: BUILDING-SPECIFIC LOAD FORECASTING STUDIES

pt : Observed demand at time t

T: Collection of time indicies
K: Forecast horizon
k 1,..., K : Time index within forecast horizon
pˆ t  k |t : Estimate of p given observations up to time t looking k steps ahead
Φt : Collection of historical demand and predictor variables
m: Time in the past to which historical observations are included
pt  m : Historical observations of p from time m to time t
x: Set of all available predictor variables
xt  m : Historical observations of x from time m to time t
xˆ t  k |t : Estimate of x given observations up to time t looking k steps ahead
N: Total number of observations
pi : Observation of the dependent variable p at time i
xki : Observation of predictor variable k at time i
k : Model regression parameter corresponding to the k th explanatory variable
i : Model error at time i
: SVM mapping function
i : SVM lower training error limit
 i* : SVM upper training error limit
C: SVM error cost
: SVM tube width
: Radial basis function parameter
fk : Smooth, nonlinear function related to predictor variable k
bi  xk  : i th spline basis function
d: Dimension of the spline basis
kv : Total number of predictor variables
 147
p: All collected observations of the dependent variable p
B: Matrix formed by concatenation of the spline functions bi  x 
β: Vector of the the unknown regression parameters 
q : Smoothing parameter
S: Smoothing matrix of known coefficients
et  k |t : Forecast error given observations up to time t looking k steps ahead
te k |t : Systematic part of the forecast error
te k |t : Random part of the forecast error
 ke : Standard deviation of the forecaast error
e
s :k Skewness of the forecast errors
 : e
k Kurtosis of the forecast errors

Chapter 4: ESTIMATION OF BUILDING-SPECIFIC LOAD FORECASTING

PREDICTION INTERVALS

Ît  k |t : Prediction interval estimated at time t looking k steps

ahead
ˆ 
Lt  k |t : Prediction interval lower bound
Uˆ t k|t :

Prediction interval upper bound
1 : Nominal coverage rate
qˆt  k |t : Quantiles of the estimated prediction error distribution
Fˆt pk |t : Estimated prediction error distribution
St , k : Set of k-step ahead forecast errors
Gˆ t,k    : Cumulative distribution function giving the fraction of collected errors  i less than a
given error level 
 t,k  : Reliability indicator variable
 
nk ,0 : Number of "misses" over the interval of interest
 
nk ,1 : Number of "hits" over the interval of interest
NT : Number of realizations over the interval of interest
 
aˆk : Prediction interval reliability
 
 t ,k : Size of a given prediction interval
 t ,k  : Prediciton interval sharpness
 k  : Prediction interval resolution
Sct ,k : Winkler Skill score
 148
Chapter 5: CONTROLLABLE BUILDING ELECTRICAL LOAD MODELING

tc : Time delay between temperature change and load response

Po : Initial power value (nominal operating point)
o : Initial chilled water temperature value
s : Static load model exponential index
t : Transient load model exponential index
Ps   : Static load function
Pt   : Transient load functin
Tr : Dynamic load time constant
: Chilled water temperature value
Pd   : Dynamic load function chilled water outlet temperature
p: Vector of model parameters to be estimated
Z: Feasible parameter space
: Least squares error function
N: Number of measurements
y k  : Measured value
yˆ  k  : Model output value

Chapter 6: OPTIMAL SCHEDULING OF BUILDING DEMAND RESOURCES

P c : Dispatched amount of controllable load

P : Dispatched amount of total building load
margin C : Available load margin
C , nom
P : Nominal operating controllable load level
C ,min
P : Minimum controllable load level
C ,max
P : Maximum controllable load level
PtC,m : Dispatched controllable load at time t for building m
t ,m : Price of electricity at time t for building m
st ,m : State (on/off) of controllable load at time t for building m
PtC,m,nom : Nominal controllable load level at time t for building m
PtC,m,min : Minimum controllable load level at time t for building m
PtC,m,max : Maximum controllable load level at time t for building m
tdown
,m : Ramp rate limit (down) at time t for building m
up
t ,m : Ramp rate limit (up) at time t for building m
 :
on
m Limit on number of intervals PC can deviate from PC ,nom for building m
 149
M Total number of buildings
T Total number of dispatch intervals
tmax
,m : Maximum allowable temperature at time t for building m
tmin
,m : Minimum allowable temperature at time t for building m
Pt max
,m : Maximum allowable load level at time t for building m
Pt min
,m : Minimum allowable load level at time t for building m
 t ,m : Locational marginal price (LMP) at time t for building m
Pt ,DR
m : Dispatched controllable load for DR revenue at time t for building m
B: Large positive integer
DR
s :
t State (on/off) of DR dispatched load at time t
Thresh
P t : Load threshold required to earn DR revenue at time t
t ,m : Forecast error parameter for building m at time t
 150
Appendix B. FORECASTING CASE STUDY RESULTS

This Appendix presents the additional forecasting case study results from Chapter

3 and Chapter 4. These results are presented here without commentary. Refer to the

Chapters for discussion and observations regarding the results.

Table B.1 MAPE results at 5 minute resolution for all 116 day-ahead forecasts
MLR NN SVM SAM
Time No Bldg Bldg No Bldg Bldg No Bldg Bldg No Bldg Bldg
0:00 4.46 4.34 6.45 4.50 3.80 3.62 5.64 5.42
0:05 4.14 3.99 6.62 5.02 3.88 3.81 5.67 5.34
0:10 4.35 4.22 7.00 5.29 4.13 4.14 5.92 5.35
0:15 4.51 4.36 7.21 5.51 4.39 4.39 6.13 5.44
0:20 4.94 4.86 7.84 6.28 5.09 5.13 6.77 6.25
0:25 5.09 5.04 7.93 6.67 5.35 5.38 7.12 6.43
0:30 5.34 5.35 8.28 7.08 5.87 6.02 7.46 6.70
0:35 5.54 5.53 8.35 7.11 5.96 6.10 7.78 6.88
0:40 5.79 5.76 8.84 7.48 6.57 6.69 7.85 7.02
0:45 6.13 6.16 9.31 7.97 7.04 7.18 8.19 7.25
0:50 5.72 5.84 9.47 7.76 6.89 7.05 7.90 6.79
0:55 5.89 5.96 9.42 7.79 6.94 7.14 7.98 6.85
1:00 5.54 5.48 8.44 5.81 7.08 7.13 7.59 6.70
1:05 5.45 5.40 8.37 5.80 6.95 6.98 7.61 6.74
1:10 5.38 5.32 7.96 5.84 6.80 6.80 7.53 6.74
1:15 5.25 5.18 8.00 5.74 6.69 6.66 7.29 6.49
1:20 5.39 5.29 7.75 5.74 6.74 6.71 7.41 6.62
1:25 5.23 5.19 7.78 5.74 6.68 6.63 7.38 6.60
1:30 5.32 5.25 7.74 5.58 6.74 6.71 7.49 6.60
1:35 5.24 5.21 7.75 5.54 6.73 6.70 7.37 6.47
1:40 5.16 5.16 7.68 5.28 6.78 6.66 7.21 6.32
1:45 5.60 5.57 7.56 5.51 7.01 6.92 7.70 6.88
1:50 5.36 5.33 7.40 5.33 6.79 6.73 7.41 6.57
1:55 5.52 5.62 7.80 5.51 7.39 7.12 8.09 6.70
2:00 10.40 7.66 8.46 10.10 6.40 5.65 9.85 9.41
2:05 8.49 9.87 8.33 10.66 6.48 5.98 11.37 8.48
2:10 6.93 7.99 8.01 9.32 6.10 5.44 10.42 8.05
2:15 5.87 6.59 7.77 7.89 5.76 5.38 9.89 8.04
2:20 6.20 6.46 7.77 7.09 5.53 5.29 9.61 8.07
2:25 7.37 7.55 8.02 6.80 6.03 5.93 10.17 9.05
2:30 8.76 8.81 8.57 7.02 6.60 6.59 10.72 9.56
2:35 9.24 9.18 8.63 7.20 6.75 6.70 11.16 10.12
 151
MLR NN SVM SAM
Time No Bldg Bldg No Bldg Bldg No Bldg Bldg No Bldg Bldg
2:40 9.40 9.49 9.35 7.19 6.71 6.63 11.34 10.45
2:45 9.73 9.64 9.31 7.29 6.93 7.02 11.62 10.41
2:50 9.98 10.16 9.66 7.62 7.21 7.26 11.90 10.69
2:55 9.98 9.90 9.47 7.25 7.15 7.01 11.53 10.39
3:00 8.22 8.41 12.92 7.99 6.95 7.27 11.52 10.00
3:05 8.09 8.43 12.53 7.89 6.69 7.01 11.92 10.10
3:10 8.33 8.45 12.93 8.39 7.14 7.45 11.94 10.46
3:15 8.90 8.91 13.10 8.21 7.07 7.34 12.10 10.50
3:20 8.72 8.76 12.74 8.05 6.97 7.18 11.64 10.27
3:25 8.78 8.70 12.51 8.13 6.99 7.24 11.79 10.36
3:30 8.50 8.71 12.25 7.92 6.57 6.79 11.78 10.32
3:35 8.58 8.60 12.85 8.29 6.86 7.18 11.22 10.17
3:40 8.16 8.30 12.19 7.93 6.48 6.76 11.14 9.91
3:45 8.00 8.33 11.83 7.60 6.15 6.49 11.24 9.78
3:50 8.15 8.44 12.23 8.24 6.47 6.93 11.06 9.96
3:55 8.06 8.67 11.87 8.25 6.24 6.65 10.91 10.05
4:00 8.38 7.81 10.36 6.20 5.87 5.93 10.59 9.54
4:05 8.19 7.77 9.73 5.97 5.62 5.67 10.69 9.38
4:10 8.24 8.02 9.73 6.17 5.87 5.88 10.80 9.52
4:15 8.32 7.98 9.94 6.46 6.03 6.03 10.45 9.14
4:20 8.14 7.89 9.85 6.56 6.02 6.05 10.35 8.87
4:25 8.09 7.95 9.62 6.60 5.98 6.18 10.50 8.78
4:30 8.33 8.10 10.25 6.83 6.27 6.46 10.61 8.99
4:35 8.61 8.54 10.46 7.16 6.32 6.55 11.00 9.40
4:40 8.37 8.72 10.63 7.30 6.57 6.97 10.62 9.36
4:45 8.02 8.83 10.64 7.51 6.17 6.73 10.43 9.49
4:50 7.75 8.72 10.63 7.56 6.22 6.81 10.30 9.90
4:55 7.75 8.75 10.97 7.90 6.41 6.99 10.41 10.17
5:00 13.51 11.53 13.30 7.89 11.77 12.03 11.70 12.91
5:05 12.43 10.96 12.42 7.64 10.96 11.55 10.91 12.04
5:10 11.73 10.64 11.59 7.52 10.26 11.06 10.54 11.37
5:15 11.03 10.30 10.95 7.56 9.90 10.58 10.33 11.24
5:20 10.42 10.04 10.59 7.32 9.07 9.81 10.26 10.90
5:25 10.05 9.94 10.25 7.49 8.73 9.41 9.83 10.72
5:30 9.53 9.65 9.97 7.52 8.10 8.91 9.80 10.24
5:35 8.92 9.18 9.22 7.77 7.77 8.43 9.89 9.87
5:40 8.76 8.98 9.21 7.75 7.41 7.99 9.68 9.49
5:45 8.28 8.50 8.92 7.51 6.92 7.36 9.20 8.86
5:50 7.88 8.04 8.88 7.97 6.62 6.80 9.08 8.23
5:55 7.63 7.59 9.19 8.75 6.44 6.21 8.87 7.88
6:00 8.56 8.77 9.13 7.74 9.07 9.34 8.84 8.32
 152
MLR NN SVM SAM
Time No Bldg Bldg No Bldg Bldg No Bldg Bldg No Bldg Bldg
6:05 8.01 8.14 8.27 7.09 7.98 8.20 8.25 8.00
6:10 7.66 7.75 7.29 6.75 7.14 7.26 7.75 7.58
6:15 6.70 6.75 6.27 5.92 5.93 6.02 7.11 6.77
6:20 6.42 6.45 6.21 5.90 5.43 5.42 7.03 6.39
6:25 6.33 6.38 6.50 6.08 5.45 5.34 7.04 6.30
6:30 6.06 6.11 6.65 6.18 5.28 5.15 7.17 6.36
6:35 6.07 6.13 6.58 6.02 5.37 5.20 7.23 6.44
6:40 5.77 5.81 6.12 5.65 5.09 4.95 7.00 6.09
6:45 5.70 5.71 6.18 5.68 5.01 4.85 6.83 5.99
6:50 5.66 5.65 5.95 5.36 4.84 4.74 6.54 5.99
6:55 5.61 5.66 6.04 5.36 4.84 4.72 6.35 5.73
7:00 5.27 5.26 6.33 5.25 5.20 5.18 5.96 5.47
7:05 5.35 5.34 6.41 5.48 5.51 5.55 6.33 5.69
7:10 5.45 5.40 6.50 5.52 5.58 5.64 6.29 5.89
7:15 5.45 5.51 6.39 5.43 5.59 5.62 6.18 5.83
7:20 5.08 5.12 6.12 5.17 5.15 5.17 5.94 5.55
7:25 5.10 5.11 6.23 5.15 5.05 5.15 5.94 5.65
7:30 5.38 5.46 6.32 5.38 5.20 5.30 6.30 5.93
7:35 5.07 5.12 6.36 4.95 4.77 4.95 5.79 5.64
7:40 5.35 5.41 6.58 5.30 4.96 5.06 5.90 5.91
7:45 5.25 5.30 6.72 5.05 4.84 5.00 5.95 5.90
7:50 5.20 5.26 6.45 4.95 4.62 4.72 6.00 5.81
7:55 5.36 5.45 6.78 5.14 4.64 4.71 6.04 5.89
8:00 5.32 5.42 7.49 5.59 7.20 7.05 5.90 5.83
8:05 5.66 5.73 7.66 5.99 7.30 7.07 6.39 6.36
8:10 5.24 5.36 7.30 5.56 6.66 6.48 6.34 6.07
8:15 5.32 5.33 7.31 5.70 6.82 6.62 6.38 6.16
8:20 5.43 5.44 7.45 5.65 6.74 6.57 6.04 6.20
8:25 5.15 5.10 6.79 5.44 6.18 6.02 5.74 5.84
8:30 5.27 5.28 6.52 5.30 5.79 5.65 5.83 5.92
8:35 5.14 5.14 6.17 5.26 5.58 5.44 5.77 5.76
8:40 5.57 5.48 6.55 5.37 5.89 5.78 6.15 6.17
8:45 5.92 5.75 6.90 5.58 5.91 5.83 6.44 6.56
8:50 5.69 5.62 6.66 5.37 5.73 5.65 6.61 6.57
8:55 5.86 5.82 6.43 5.49 5.81 5.66 6.71 6.64
9:00 5.97 5.87 6.35 5.82 6.74 6.61 6.53 6.36
9:05 5.78 5.65 6.40 5.65 6.51 6.42 6.28 6.20
9:10 5.46 5.32 6.06 5.36 6.08 6.00 6.07 5.97
9:15 5.27 5.16 5.78 5.25 5.92 5.88 6.04 5.88
9:20 5.07 5.02 5.82 5.31 5.67 5.53 5.65 5.49
9:25 5.17 5.15 6.06 5.26 5.73 5.64 5.79 5.68
 153
MLR NN SVM SAM
Time No Bldg Bldg No Bldg Bldg No Bldg Bldg No Bldg Bldg
9:30 4.97 4.95 5.98 5.25 5.49 5.37 5.78 5.71
9:35 4.99 5.01 6.11 5.31 5.56 5.49 5.79 5.87
9:40 5.11 5.09 6.06 5.41 5.36 5.26 5.80 5.94
9:45 5.04 5.00 5.88 5.30 5.13 5.04 5.88 5.84
9:50 4.99 5.02 5.59 5.49 4.97 4.83 5.77 5.64
9:55 5.13 5.05 5.39 5.34 4.67 4.55 5.88 5.79
10:00 5.37 5.29 6.49 5.75 6.44 6.14 5.43 5.58
10:05 5.30 5.18 6.19 5.46 6.08 5.83 5.45 5.44
10:10 5.17 5.10 6.05 5.54 5.95 5.66 5.40 5.53
10:15 4.95 4.86 6.11 5.19 5.65 5.35 5.38 5.43
10:20 5.03 4.93 6.19 5.20 5.55 5.33 5.41 5.42
10:25 4.85 4.76 6.06 5.06 5.41 5.14 5.27 5.22
10:30 4.83 4.74 5.87 4.88 5.18 4.90 5.34 5.20
10:35 4.72 4.60 5.76 4.95 5.15 4.86 5.24 5.31
10:40 4.70 4.59 5.65 5.09 5.04 4.79 5.37 5.34
10:45 4.79 4.68 5.80 5.16 5.05 4.81 5.40 5.38
10:50 5.01 4.90 5.91 5.39 5.23 5.02 5.52 5.52
10:55 5.22 5.09 5.82 5.32 5.16 4.93 5.82 5.74
11:00 5.43 5.34 6.82 5.11 5.66 5.71 5.97 5.72
11:05 5.39 5.26 6.77 5.06 5.46 5.47 5.85 5.61
11:10 5.10 4.98 6.40 4.90 5.09 5.12 5.81 5.56
11:15 4.97 4.86 6.44 4.79 4.97 4.97 5.54 5.30
11:20 5.02 4.97 6.39 5.03 4.87 4.89 5.71 5.48
11:25 4.95 4.90 6.42 5.01 4.69 4.70 5.64 5.47
11:30 4.89 4.84 6.44 4.86 4.76 4.74 5.58 5.38
11:35 4.84 4.82 6.09 4.95 4.69 4.62 5.68 5.37
11:40 4.77 4.72 6.42 4.94 4.78 4.71 5.58 5.26
11:45 4.78 4.68 6.28 4.76 4.56 4.50 5.57 5.28
11:50 4.53 4.38 5.99 4.68 4.33 4.21 5.29 4.99
11:55 4.45 4.31 6.02 4.57 4.56 4.42 5.19 4.92
12:00 4.50 4.41 5.82 4.82 4.35 4.54 5.15 5.00
12:05 4.41 4.31 5.82 4.83 4.46 4.69 4.99 4.89
12:10 4.51 4.45 5.62 4.75 4.49 4.68 5.07 4.88
12:15 4.49 4.40 5.80 4.82 4.52 4.75 5.14 4.95
12:20 4.45 4.34 5.78 4.77 4.46 4.60 4.95 4.79
12:25 4.38 4.31 5.57 4.88 4.35 4.49 5.07 4.94
12:30 4.25 4.15 5.48 4.66 4.49 4.66 4.87 4.75
12:35 4.42 4.29 5.65 4.87 4.39 4.59 4.89 4.85
12:40 4.21 4.06 5.55 4.79 4.30 4.45 4.90 4.83
12:45 4.46 4.41 5.52 5.00 4.40 4.58 5.20 4.95
12:50 4.37 4.34 5.43 4.76 4.35 4.55 5.38 4.81
 154
MLR NN SVM SAM
Time No Bldg Bldg No Bldg Bldg No Bldg Bldg No Bldg Bldg
12:55 4.46 4.45 5.67 4.73 4.54 4.68 5.11 4.71
13:00 4.31 4.33 6.12 5.25 4.13 4.20 5.03 4.76
13:05 4.43 4.46 6.29 5.20 4.11 4.07 5.04 4.76
13:10 4.45 4.50 6.41 5.02 4.07 4.04 5.00 4.72
13:15 4.46 4.47 6.35 5.04 4.17 4.10 4.97 4.74
13:20 4.44 4.42 6.22 5.10 4.13 4.00 5.13 4.80
13:25 4.44 4.44 6.12 4.88 3.89 3.78 5.25 4.70
13:30 4.49 4.49 6.25 5.24 4.12 3.98 5.09 4.64
13:35 4.55 4.57 6.16 5.16 4.16 4.04 5.09 4.66
13:40 4.42 4.36 6.12 5.12 4.12 4.01 5.01 4.64
13:45 4.35 4.33 6.23 5.06 4.05 3.96 5.00 4.60
13:50 4.43 4.33 6.31 5.10 4.02 3.99 4.93 4.63
13:55 4.44 4.34 6.29 5.21 3.98 3.89 5.09 4.71
14:00 4.66 4.59 5.79 4.60 4.02 3.86 5.18 4.86
14:05 4.79 4.64 5.72 4.70 4.01 3.80 5.21 4.96
14:10 4.65 4.51 5.46 4.65 4.02 3.95 5.21 4.90
14:15 4.91 4.74 5.63 4.67 4.05 3.93 5.40 5.03
14:20 4.94 4.79 5.66 4.64 4.09 3.90 5.32 5.14
14:25 4.84 4.59 5.46 4.50 3.91 3.72 5.30 5.04
14:30 4.90 4.69 5.53 4.59 3.81 3.63 5.25 4.97
14:35 4.83 4.72 5.48 4.69 3.87 3.71 5.31 4.83
14:40 4.89 4.76 5.59 4.99 4.09 3.95 5.28 5.04
14:45 4.91 4.68 5.58 4.81 4.18 3.96 5.18 5.08
14:50 5.14 4.86 5.47 4.66 3.89 3.77 5.35 5.24
14:55 5.20 4.95 5.69 4.91 3.98 3.75 5.56 5.31
15:00 4.84 4.71 5.67 4.99 4.27 3.96 5.23 5.06
15:05 4.86 4.65 5.65 5.07 4.08 3.80 5.26 4.99
15:10 4.93 4.67 5.74 5.13 4.12 3.93 5.31 4.96
15:15 5.07 4.80 5.65 5.07 4.06 3.81 5.40 5.07
15:20 5.08 4.88 5.63 5.22 4.30 3.94 5.55 5.24
15:25 5.22 5.04 5.60 5.21 4.37 4.05 5.57 5.38
15:30 5.26 5.07 5.83 5.10 4.19 4.00 5.52 5.21
15:35 5.12 4.96 5.91 5.06 4.54 4.24 5.44 5.26
15:40 5.18 5.02 5.81 5.06 4.23 3.93 5.48 5.17
15:45 5.14 4.99 5.87 5.07 4.26 3.97 5.50 5.22
15:50 5.05 4.80 5.93 4.92 4.10 3.86 5.35 5.04
15:55 4.92 4.68 5.90 4.76 3.86 3.72 5.34 4.97
16:00 4.85 4.69 5.59 5.57 4.10 3.78 5.34 4.87
16:05 5.01 4.80 5.52 5.70 4.10 3.76 5.47 4.95
16:10 5.05 4.86 5.44 5.70 4.27 3.92 5.58 4.99
16:15 4.80 4.64 5.41 5.57 4.22 3.82 5.57 4.91
 155
MLR NN SVM SAM
Time No Bldg Bldg No Bldg Bldg No Bldg Bldg No Bldg Bldg
16:20 4.81 4.72 5.57 5.57 4.17 3.81 5.58 4.88
16:25 4.77 4.74 5.68 5.61 4.28 3.95 5.60 4.89
16:30 4.83 4.72 5.83 5.57 4.37 3.96 5.65 5.08
16:35 4.91 4.82 5.88 5.89 4.40 4.02 5.60 5.23
16:40 4.96 4.93 5.96 5.87 4.37 4.00 5.75 5.08
16:45 4.94 4.88 5.79 5.62 4.30 3.96 5.69 5.05
16:50 4.88 4.75 5.77 5.55 4.07 3.70 5.66 5.04
16:55 4.89 4.78 5.67 5.42 4.14 3.88 5.48 4.92
17:00 5.03 4.89 4.93 5.77 4.41 3.71 5.42 4.92
17:05 4.64 4.45 4.85 5.58 4.22 3.51 5.34 4.75
17:10 4.75 4.55 4.90 5.43 4.06 3.42 5.23 4.76
17:15 4.70 4.60 4.93 5.43 4.04 3.41 5.33 4.73
17:20 4.69 4.59 5.06 5.17 4.21 3.57 5.20 4.65
17:25 4.60 4.53 4.92 5.18 3.99 3.21 5.30 4.50
17:30 4.49 4.52 4.96 5.01 3.93 3.29 5.26 4.52
17:35 4.41 4.38 5.00 4.88 3.80 3.06 5.14 4.49
17:40 4.51 4.54 5.36 4.92 3.91 3.29 5.34 4.65
17:45 4.36 4.42 5.22 4.83 3.88 3.26 5.18 4.53
17:50 4.39 4.57 5.27 4.52 3.79 3.21 5.22 4.51
17:55 4.18 4.31 5.29 4.56 3.78 3.31 4.93 4.37
18:00 4.55 4.22 5.01 7.36 4.56 3.93 5.32 4.74
18:05 4.59 4.27 5.02 7.26 4.65 4.07 5.21 4.62
18:10 4.37 4.07 4.89 7.32 4.57 4.03 4.99 4.46
18:15 4.49 4.23 4.70 7.12 4.34 3.87 5.03 4.53
18:20 4.51 4.29 4.85 7.29 4.29 3.83 5.30 4.62
18:25 4.45 4.16 4.36 7.04 4.07 3.52 5.18 4.62
18:30 4.57 4.42 4.67 6.99 4.21 3.74 5.49 4.92
18:35 4.32 4.17 4.66 6.91 3.94 3.56 5.47 4.81
18:40 4.37 4.37 4.90 6.66 3.96 3.72 5.59 4.79
18:45 4.10 4.00 4.64 6.60 3.65 3.35 5.31 4.47
18:50 4.28 4.16 4.96 6.65 3.75 3.43 5.40 4.56
18:55 4.31 4.30 5.26 6.53 3.79 3.59 5.56 4.61
19:00 4.60 4.28 4.47 6.87 4.18 3.84 5.54 4.87
19:05 4.65 4.39 4.79 6.83 4.22 3.90 5.67 5.06
19:10 4.58 4.29 4.62 7.02 4.06 3.80 5.63 4.86
19:15 4.92 4.66 4.80 6.75 4.26 3.93 5.98 5.24
19:20 4.79 4.51 4.73 6.68 4.08 3.82 5.78 5.14
19:25 4.88 4.64 4.95 6.79 4.30 4.01 6.05 5.21
19:30 4.83 4.65 5.03 6.77 4.23 3.96 6.02 5.24
19:35 4.80 4.67 5.04 6.55 4.17 3.98 5.90 5.11
19:40 4.69 4.54 4.97 6.58 4.14 3.90 5.67 5.02
 156
MLR NN SVM SAM
Time No Bldg Bldg No Bldg Bldg No Bldg Bldg No Bldg Bldg
19:45 4.58 4.48 4.86 6.42 4.16 3.95 5.58 4.92
19:50 4.46 4.32 4.79 6.29 3.93 3.71 5.58 4.79
19:55 4.40 4.33 4.87 6.05 3.81 3.67 5.62 4.91
20:00 4.60 4.53 4.86 5.06 3.89 3.73 5.60 5.04
20:05 4.35 4.25 4.58 5.01 3.85 3.58 5.44 4.83
20:10 4.49 4.48 4.70 5.00 3.72 3.55 5.68 4.86
20:15 4.60 4.57 4.79 4.87 3.62 3.45 5.82 4.88
20:20 4.50 4.49 4.86 4.82 3.70 3.58 5.73 4.79
20:25 4.62 4.64 5.17 4.78 3.95 3.85 5.66 4.83
20:30 4.69 4.74 4.96 4.61 4.06 3.99 5.97 4.91
20:35 4.50 4.60 5.06 4.66 4.01 4.03 5.85 4.88
20:40 4.51 4.66 5.10 4.51 3.83 3.78 5.76 4.76
20:45 4.64 4.80 5.40 4.65 4.24 4.19 5.96 4.82
20:50 4.74 4.98 5.58 4.70 4.41 4.35 6.14 4.82
20:55 4.56 4.69 5.54 4.57 4.27 4.11 5.95 4.83
21:00 4.78 4.95 6.44 5.41 4.12 4.17 6.19 5.09
21:05 4.98 5.15 6.75 5.53 4.20 4.23 6.36 5.24
21:10 5.10 5.24 6.71 5.54 4.40 4.48 6.44 5.40
21:15 4.92 5.10 6.79 5.08 4.05 4.17 6.43 5.31
21:20 4.97 5.19 6.96 5.16 4.15 4.26 6.40 5.25
21:25 4.86 5.19 7.85 4.98 4.32 4.41 6.46 5.27
21:30 4.74 5.06 7.60 5.24 4.19 4.33 6.34 5.26
21:35 4.90 5.32 8.05 4.85 4.44 4.52 6.55 5.11
21:40 5.16 5.60 8.38 4.92 4.82 4.85 6.72 5.26
21:45 4.99 5.46 8.42 5.05 4.66 4.68 6.62 5.23
21:50 5.17 5.58 8.75 5.12 4.82 4.84 6.78 5.43
21:55 5.21 5.71 8.81 5.31 4.77 4.79 6.83 5.40
22:00 5.07 5.25 7.48 5.11 4.30 4.40 6.49 5.17
22:05 4.97 5.24 7.71 5.04 4.15 4.23 6.51 5.27
22:10 5.09 5.32 7.62 5.23 4.21 4.28 6.58 5.21
22:15 5.22 5.50 8.03 5.20 4.24 4.34 6.77 5.38
22:20 4.99 5.20 7.79 4.84 4.17 4.24 6.56 5.37
22:25 5.00 5.25 7.91 5.08 4.25 4.41 6.65 5.41
22:30 5.20 5.51 8.38 5.10 4.44 4.57 6.87 5.56
22:35 5.06 5.35 8.28 5.07 4.47 4.50 6.51 5.40
22:40 5.04 5.34 8.83 5.26 4.57 4.65 6.63 5.68
22:45 5.15 5.46 8.77 5.27 4.66 4.78 6.68 5.62
22:50 5.19 5.49 9.03 5.50 4.60 4.78 6.54 5.69
22:55 5.10 5.46 9.18 5.68 4.53 4.68 6.69 5.71
23:00 4.85 4.94 5.74 5.65 4.59 4.81 6.39 5.57
23:05 4.84 4.97 5.85 5.32 4.54 4.74 6.50 5.55
 157
MLR NN SVM SAM
Time No Bldg Bldg No Bldg Bldg No Bldg Bldg No Bldg Bldg
23:10 5.03 5.07 6.14 5.75 4.70 5.00 6.49 5.83
23:15 5.01 5.05 6.03 5.53 4.53 4.76 6.56 5.83
23:20 5.15 5.22 6.25 5.63 4.72 4.99 6.85 5.93
23:25 4.86 4.96 6.16 5.33 4.69 4.83 6.53 5.66
23:30 4.80 4.95 6.22 5.54 4.69 4.94 6.54 5.60
23:35 5.00 5.11 6.68 5.62 4.75 5.13 6.49 5.87
23:40 4.97 5.09 6.38 5.60 4.67 4.85 6.41 5.69
23:45 5.11 5.27 6.45 5.54 4.86 5.05 6.73 5.96
23:50 5.17 5.38 6.47 5.28 4.87 5.03 6.87 5.88
23:55 5.04 5.20 6.89 5.44 4.81 5.11 6.67 5.94
 158
 Hourly empirical error distributions for the MLR models

Figure B.1 MLR Empirical error distribution (L) No building measurements (R) With building
measurements (00:00-01:00)

Figure B.2 MLR Empirical error distribution (L) No building measurements (R) With building
measurements (01:00-02:00)

Figure B.3 MLR Empirical error distribution (L) No building measurements (R) With building
measurements (02:00-03:00)
 159

Figure B.4 MLR Empirical error distribution (L) No building measurements (R) With building
measurements (03:00-04:00)

Figure B.5 MLR Empirical error distribution (L) No building measurements (R) With building
measurements (04:00-05:00)

Figure B.6 Empirical error distribution (L) No building measurements (R) With building
measurements (05:00-06:00)
 160

Figure B.7 MLR Empirical error distribution (L) No building measurements (R) With building
measurements (06:00-07:00)

Figure B.8 MLR Empirical error distribution (L) No building measurements (R) With building
measurements (07:00-08:00)

Figure B.9 MLR Empirical error distribution (L) No building measurements (R) With building
measurements (08:00-09:00)
 161

Figure B.10 MLR Empirical error distribution (L) No building measurements (R) With building
measurements (09:00-10:00)

Figure B.11 MLR Empirical error distribution (L) No building measurements (R) With building
measurements (10:00-11:00)

Figure B.12 MLR Empirical error distribution (L) No building measurements (R) With building
measurements (11:00-12:00)
 162

Figure B.13 MLR Empirical error distribution (L) No building measurements (R) With building
measurements (12:00-13:00)

Figure B.14 MLR Empirical error distribution (L) No building measurements (R) With building
measurements (13:00-14:00)

Figure B.15 MLR Empirical error distribution (L) No building measurements (R) With building
measurements (14:00-15:00)
 163

Figure B.16 MLR Empirical error distribution (L) No building measurements (R) With building
measurements (15:00-16:00)

Figure B.17 MLR Empirical error distribution (L) No building measurements (R) With building
measurements (16:00-17:00)

Figure B.18 MLR Empirical error distribution (L) No building measurements (R) With building
measurements (17:00-18:00)
 164

Figure B.19 MLR Empirical error distribution (L) No building measurements (R) With building
measurements (18:00-19:00)

Figure B.20 MLR Empirical error distribution (L) No building measurements (R) With building
measurements (19:00-20:00)

Figure B.21 MLR Empirical error distribution (L) No building measurements (R) With building
measurements (20:00-21:00)
 165

Figure B.22 MLR Empirical error distribution (L) No building measurements (R) With building
measurements (21:00-22:00)

Figure B.23 MLR Empirical error distribution (L) No building measurements (R) With building
measurements (22:00-23:00)

Figure B.24 MLR Empirical error distribution (L) No building measurements (R) With building
measurements (23:00-24:00)
 166
 Hourly empirical error distributions for the NN models

Figure B.25 NN Empirical error distribution (L) No building measurements (R) With building
measurements (00:00-01:00)

Figure B.26 NN Empirical error distribution (L) No building measurements (R) With building
measurements (01:00-02:00)

Figure B.27 NN Empirical error distribution (L) No building measurements (R) With building
measurements (02:00-03:00)
 167

Figure B.28 NN Empirical error distribution (L) No building measurements (R) With building
measurements (03:00-04:00)

Figure B.29 NN Empirical error distribution (L) No building measurements (R) With building
measurements (04:00-05:00)

Figure B.30 NN Empirical error distribution (L) No building measurements (R) With building
measurements (05:00-06:00)
 168

Figure B.31 NN Empirical error distribution (L) No building measurements (R) With building
measurements (06:00-07:00)

Figure B.32 NN Empirical error distribution (L) No building measurements (R) With building
measurements (07:00-08:00)

Figure B.33 NN Empirical error distribution (L) No building measurements (R) With building
measurements (08:00-09:00)
 169

Figure B.34 NN Empirical error distribution (L) No building measurements (R) With building
measurements (09:00-10:00)

Figure B.35 NN Empirical error distribution (L) No building measurements (R) With building
measurements (10:00-11:00)

Figure B.36 NN Empirical error distribution (L) No building measurements (R) With building
measurements (11:00-12:00)
 170

Figure B.37 NN Empirical error distribution (L) No building measurements (R) With building
measurements (12:00-13:00)

Figure B.38 NN Empirical error distribution (L) No building measurements (R) With building
measurements (13:00-14:00)

Figure B.39 NN Empirical error distribution (L) No building measurements (R) With building
measurements (14:00-15:00)
 171

Figure B.40 NN Empirical error distribution (L) No building measurements (R) With building
measurements (15:00-16:00)

Figure B.41 NN Empirical error distribution (L) No building measurements (R) With building
measurements (16:00-17:00)

Figure B.42 NN Empirical error distribution (L) No building measurements (R) With building
measurements (17:00-18:00)
 172

Figure B.43 NN Empirical error distribution (L) No building measurements (R) With building
measurements (18:00-19:00)

Figure B.44 NN Empirical error distribution (L) No building measurements (R) With building
measurements (19:00-20:00)

Figure B.45 NN Empirical error distribution (L) No building measurements (R) With building
measurements (20:00-21:00)
 173

Figure B.46 NN Empirical error distribution (L) No building measurements (R) With building
measurements (21:00-22:00)

Figure B.47 NN Empirical error distribution (L) No building measurements (R) With building
measurements (22:00-23:00)

Figure B.48 NN Empirical error distribution (L) No building measurements (R) With building
measurements (23:00-24:00)
 174
 Hourly empirical error distributions for the SVM models

Figure B.49 SVM Empirical error distribution (L) No building measurements (R) With building
measurements (00:00-01:00)

Figure B.50 SVM Empirical error distribution (L) No building measurements (R) With building
measurements (01:00-02:00)

Figure B.51 SVM Empirical error distribution (L) No building measurements (R) With building
measurements (02:00-03:00)
 175

Figure B.52 SVM Empirical error distribution (L) No building measurements (R) With building
measurements (03:00-04:00)

Figure B.53 SVM Empirical error distribution (L) No building measurements (R) With building
measurements (04:00-05:00)

Figure B.54 SVM Empirical error distribution (L) No building measurements (R) With building
measurements (05:00-06:00)
 176

Figure B.55 SVM Empirical error distribution (L) No building measurements (R) With building
measurements (06:00-07:00)

Figure B.56 SVM Empirical error distribution (L) No building measurements (R) With building
measurements (07:00-08:00)

Figure B.57 SVM Empirical error distribution (L) No building measurements (R) With building
measurements (08:00-09:00)
 177

Figure B.58 SVM Empirical error distribution (L) No building measurements (R) With building
measurements (09:00-10:00)

Figure B.59 SVM Empirical error distribution (L) No building measurements (R) With building
measurements (10:00-11:00)

Figure B.60 SVM Empirical error distribution (L) No building measurements (R) With building
measurements (11:00-12:00)
 178

Figure B.61 SVM Empirical error distribution (L) No building measurements (R) With building
measurements (12:00-13:00)

Figure B.62 SVM Empirical error distribution (L) No building measurements (R) With building
measurements (13:00-14:00)

Figure B.63 SVM Empirical error distribution (L) No building measurements (R) With building
measurements (14:00-15:00)
 179

Figure B.64 SVM Empirical error distribution (L) No building measurements (R) With building
measurements (15:00-16:00)

Figure B.65 SVM Empirical error distribution (L) No building measurements (R) With building
measurements (16:00-17:00)

Figure B.66 SVM Empirical error distribution (L) No building measurements (R) With building
measurements (17:00-18:00)
 180

Figure B.67 SVM Empirical error distribution (L) No building measurements (R) With building
measurements (18:00-19:00)

Figure B.68 SVM Empirical error distribution (L) No building measurements (R) With building
measurements (19:00-20:00)

Figure B.69 SVM Empirical error distribution (L) No building measurements (R) With building
measurements (20:00-21:00)
 181

Figure B.70 SVM Empirical error distribution (L) No building measurements (R) With building
measurements (21:00-22:00)

Figure B.71 SVM Empirical error distribution (L) No building measurements (R) With building
measurements (22:00-23:00)

Figure B.72 SVM Empirical error distribution (L) No building measurements (R) With building
measurements (23:00-24:00)
 182
 Hourly empirical error distributions for the SAM models

Figure B.73 SAM Empirical error distribution (L) No building measurements (R) With building
measurements (00:00-01:00)

Figure B.74 SAM Empirical error distribution (L) No building measurements (R) With building
measurements (01:00-02:00)

Figure B.75 SAM Empirical error distribution (L) No building measurements (R) With building
measurements (02:00-03:00)
 183

Figure B.76 SAM Empirical error distribution (L) No building measurements (R) With building
measurements (03:00-04:00)

Figure B.77 SAM Empirical error distribution (L) No building measurements (R) With building
measurements (04:00-05:00)

Figure B.78 SAM Empirical error distribution (L) No building measurements (R) With building
measurements (05:00-06:00)
 184

Figure B.79 SAM Empirical error distribution (L) No building measurements (R) With building
measurements (06:00-07:00)

Figure B.80 SAM Empirical error distribution (L) No building measurements (R) With building
measurements (07:00-08:00)

Figure B.81 SAM Empirical error distribution (L) No building measurements (R) With building
measurements (08:00-09:00)
 185

Figure B.82 SAM Empirical error distribution (L) No building measurements (R) With building
measurements (09:00-10:00)

Figure B.83 SAM Empirical error distribution (L) No building measurements (R) With building
measurements (10:00-11:00)

Figure B.84 SAM Empirical error distribution (L) No building measurements (R) With building
measurements (11:00-12:00)
 186

Figure B.85 SAM Empirical error distribution (L) No building measurements (R) With building
measurements (12:00-13:00)

Figure B.86 SAM Empirical error distribution (L) No building measurements (R) With building
measurements (13:00-14:00)

Figure B.87 SAM Empirical error distribution (L) No building measurements (R) With building
measurements (14:00-15:00)
 187

Figure B.88 SAM Empirical error distribution (L) No building measurements (R) With building
measurements (15:00-16:00)

Figure B.89 SAM Empirical error distribution (L) No building measurements (R) With building
measurements (16:00-17:00)

Figure B.90 SAM Empirical error distribution (L) No building measurements (R) With building
measurements (17:00-18:00)
 188

Figure B.91 SAM Empirical error distribution (L) No building measurements (R) With building
measurements (18:00-19:00)

Figure B.92 SAM Empirical error distribution (L) No building measurements (R) With building
measurements (19:00-20:00)

Figure B.93 SAM Empirical error distribution (L) No building measurements (R) With building
measurements (20:00-21:00)
 189

Figure B.94 SAM Empirical error distribution (L) No building measurements (R) With building
measurements (21:00-22:00)

Figure B.95 SAM Empirical error distribution (L) No building measurements (R) With building
measurements (22:00-23:00)

Figure B.96 SAM Empirical error distribution (L) No building measurements (R) With building
measurements (23:00-24:00)
 190
 Average hourly skill score for the MLR models
0
W/O Bldg Meas
WITH Bldg Meas
-10

-20

-30

Skill Score
-40

-50

-60

-70

-80

-90
0 5 10 15 20 25
Hours

Figure B.97 Average hourly skill score for the MLR Models. Nominal coverage rate = 5%
0
W/O Bldg Meas
WITH Bldg Meas
-10

-20

-30
Skill Score

-40

-50

-60

-70

-80

-90
0 5 10 15 20 25
Hours

Figure B.98 Average hourly skill score for the MLR Models. Nominal coverage rate = 10%
0
W/O Bldg Meas
WITH Bldg Meas
-10

-20

-30
Skill Score

-40

-50

-60

-70

-80

-90
0 5 10 15 20 25
Hours

Figure B.99 Average hourly skill score for the MLR Models. Nominal coverage rate = 15%
 191
0
W/O Bldg Meas
WITH Bldg Meas
-10

-20

-30

Skill Score
-40

-50

-60

-70

-80

-90
0 5 10 15 20 25
Hours

Figure B.100 Average hourly skill score for the MLR Models. Nominal coverage rate = 20%
0
W/O Bldg Meas
WITH Bldg Meas
-10

-20

-30
Skill Score

-40

-50

-60

-70

-80

-90
0 5 10 15 20 25
Hours

Figure B.101 Average hourly skill score for the MLR Models. Nominal coverage rate = 25%
0
W/O Bldg Meas
WITH Bldg Meas
-10

-20

-30
Skill Score

-40

-50

-60

-70

-80

-90
0 5 10 15 20 25
Hours

Figure B.102 Average hourly skill score for the MLR Models. Nominal coverage rate = 30%
 192
0
W/O Bldg Meas
WITH Bldg Meas
-10

-20

-30

Skill Score
-40

-50

-60

-70

-80

-90
0 5 10 15 20 25
Hours

Figure B.103 Average hourly skill score for the MLR Models. Nominal coverage rate = 35%
0
W/O Bldg Meas
WITH Bldg Meas
-10

-20

-30
Skill Score

-40

-50

-60

-70

-80

-90
0 5 10 15 20 25
Hours

Figure B.104 Average hourly skill score for the MLR Models. Nominal coverage rate = 40%
0
W/O Bldg Meas
WITH Bldg Meas
-10

-20

-30
Skill Score

-40

-50

-60

-70

-80

-90
0 5 10 15 20 25
Hours

Figure B.105 Average hourly skill score for the MLR Models. Nominal coverage rate = 45%
 193
0
W/O Bldg Meas
WITH Bldg Meas
-10

-20

-30

Skill Score
-40

-50

-60

-70

-80

-90
0 5 10 15 20 25
Hours

Figure B.106 Average hourly skill score for the MLR Models. Nominal coverage rate = 50%
0
W/O Bldg Meas
WITH Bldg Meas
-10

-20

-30
Skill Score

-40

-50

-60

-70

-80

-90
0 5 10 15 20 25
Hours

Figure B.107 Average hourly skill score for the MLR Models. Nominal coverage rate = 55%
0
W/O Bldg Meas
WITH Bldg Meas
-10

-20

-30
Skill Score

-40

-50

-60

-70

-80

-90
0 5 10 15 20 25
Hours

Figure B.108 Average hourly skill score for the MLR Models. Nominal coverage rate = 60%
 194
0
W/O Bldg Meas
WITH Bldg Meas
-10

-20

-30

Skill Score
-40

-50

-60

-70

-80

-90
0 5 10 15 20 25
Hours

Figure B.109 Average hourly skill score for the MLR Models. Nominal coverage rate = 65%
0
W/O Bldg Meas
WITH Bldg Meas
-10

-20

-30
Skill Score

-40

-50

-60

-70

-80

-90
0 5 10 15 20 25
Hours

Figure B.110 Average hourly skill score for the MLR Models. Nominal coverage rate = 70%
0
W/O Bldg Meas
WITH Bldg Meas
-10

-20

-30
Skill Score

-40

-50

-60

-70

-80

-90
0 5 10 15 20 25
Hours

Figure B.111 Average hourly skill score for the MLR Models. Nominal coverage rate = 75%
 195
0
W/O Bldg Meas
WITH Bldg Meas
-10

-20

-30

Skill Score
-40

-50

-60

-70

-80

-90
0 5 10 15 20 25
Hours

Figure B.112 Average hourly skill score for the MLR Models. Nominal coverage rate = 80%
0
W/O Bldg Meas
WITH Bldg Meas
-10

-20

-30
Skill Score

-40

-50

-60

-70

-80

-90
0 5 10 15 20 25
Hours

Figure B.113 Average hourly skill score for the MLR Models. Nominal coverage rate = 85%
0

-10

W/O Bldg Meas

-20
WITH Bldg Meas

-30
Skill Score

-40

-50

-60

-70

-80

-90
0 5 10 15 20 25
Hours

Figure B.114 Average hourly skill score for the MLR Models. Nominal coverage rate = 90%
 196
0

-10

W/O Bldg Meas

-20 WITH Bldg Meas

-30

Skill Score
-40

-50

-60

-70

-80

-90
0 5 10 15 20 25
Hours

Figure B.115 Average hourly skill score for the MLR Models. Nominal coverage rate = 95%

 Average hourly skill score for the NN models

0

-10

-20

-30
Skill Score

-40

-50

-60

-70

W/O Bldg Meas

-80
WITH Bldg Meas

-90
0 5 10 15 20 25
Hours

Figure B.116 Average hourly skill score for the NN Models. Nominal coverage rate = 5%
0

-10

-20

-30
Skill Score

-40

-50

-60

-70

W/O Bldg Meas

-80
WITH Bldg Meas

-90
0 5 10 15 20 25
Hours

Figure B.117 Average hourly skill score for the NN Models. Nominal coverage rate = 10%
 197
0

-10

-20

-30

Skill Score
-40

-50

-60

-70

W/O Bldg Meas

-80
WITH Bldg Meas

-90
0 5 10 15 20 25
Hours

Figure B.118 Average hourly skill score for the NN Models. Nominal coverage rate = 15%
0

-10

-20

-30
Skill Score

-40

-50

-60

-70

-80 W/O Bldg Meas

WITH Bldg Meas

-90
0 5 10 15 20 25
Hours

Figure B.119 Average hourly skill score for the NN Models. Nominal coverage rate = 20%
0

-10

-20

-30
Skill Score

-40

-50

-60

-70

-80 W/O Bldg Meas

WITH Bldg Meas

-90
0 5 10 15 20 25
Hours

Figure B.120 Average hourly skill score for the NN Models. Nominal coverage rate = 25%
 198
0

-10

-20

-30

Skill Score
-40

-50

-60

-70

-80 W/O Bldg Meas

WITH Bldg Meas

-90
0 5 10 15 20 25
Hours

Figure B.121 Average hourly skill score for the NN Models. Nominal coverage rate = 30%
0

-10

-20

-30
Skill Score

-40

-50

-60

-70

W/O Bldg Meas

-80
WITH Bldg Meas

-90
0 5 10 15 20 25
Hours

Figure B.122 Average hourly skill score for the NN Models. Nominal coverage rate = 35%
0

-10

-20

-30
Skill Score

-40

-50

-60

-70

-80 W/O Bldg Meas

WITH Bldg Meas

-90
0 5 10 15 20 25
Hours

Figure B.123 Average hourly skill score for the NN Models. Nominal coverage rate = 40%
 199
0

-10

-20

-30

Skill Score
-40

-50

-60

-70

W/O Bldg Meas

-80 WITH Bldg Meas

-90
0 5 10 15 20 25
Hours

Figure B.124 Average hourly skill score for the NN Models. Nominal coverage rate = 45%
0

-10

-20

-30
Skill Score

-40

-50

-60

-70

-80 W/O Bldg Meas

WITH Bldg Meas

-90
0 5 10 15 20 25
Hours

Figure B.125 Average hourly skill score for the NN Models. Nominal coverage rate = 50%
0

-10

-20

-30
Skill Score

-40

-50

-60

-70

W/O Bldg Meas

-80
WITH Bldg Meas

-90
0 5 10 15 20 25
Hours

Figure B.126 Average hourly skill score for the NN Models. Nominal coverage rate = 55%
 200
0

-10

-20

-30

Skill Score
-40

-50

-60

-70

W/O Bldg Meas

-80 WITH Bldg Meas

-90
0 5 10 15 20 25
Hours

Figure B.127 Average hourly skill score for the NN Models. Nominal coverage rate = 60%
0

-10

-20

-30
Skill Score

-40

-50

-60

-70

-80 W/O Bldg Meas

WITH Bldg Meas

-90
0 5 10 15 20 25
Hours

Figure B.128 Average hourly skill score for the NN Models. Nominal coverage rate = 65%
0

-10

-20

-30
Skill Score

-40

-50

-60

-70

-80 W/O Bldg Meas

WITH Bldg Meas

-90
0 5 10 15 20 25
Hours

Figure B.129 Average hourly skill score for the NN Models. Nominal coverage rate = 70%
 201
0

-10

-20

-30

Skill Score
-40

-50

-60

-70

W/O Bldg Meas

-80
WITH Bldg Meas

-90
0 5 10 15 20 25
Hours

Figure B.130 Average hourly skill score for the NN Models. Nominal coverage rate = 75%
0

-10

-20

-30
Skill Score

-40

-50

-60

-70

-80 W/O Bldg Meas

WITH Bldg Meas

-90
0 5 10 15 20 25
Hours

Figure B.131 Average hourly skill score for the NN Models. Nominal coverage rate = 80%
0

-10

-20

-30
Skill Score

-40

-50

-60

-70

-80 W/O Bldg Meas

WITH Bldg Meas

-90
0 5 10 15 20 25
Hours

Figure B.132 Average hourly skill score for the NN Models. Nominal coverage rate = 85%
 202
0

-10

-20

-30

Skill Score
-40

-50

-60

-70

W/O Bldg Meas

-80
WITH Bldg Meas

-90
0 5 10 15 20 25
Hours

Figure B.133 Average hourly skill score for the NN Models. Nominal coverage rate = 90%
0

-10

-20

-30
Skill Score

-40

-50

-60

-70

W/O Bldg Meas

-80
WITH Bldg Meas

-90
0 5 10 15 20 25
Hours

Figure B.134 Average hourly skill score for the NN Models. Nominal coverage rate = 95%

 Average hourly skill score for the SVM models

0
W/O Bldg Meas
WITH Bldg Meas
-10

-20

-30
Skill Score

-40

-50

-60

-70

-80

-90
0 5 10 15 20 25
Hours

Figure B.135 Average hourly skill score for the SVM Models. Nominal coverage rate = 5%
 203
0
W/O Bldg Meas
WITH Bldg Meas
-10

-20

-30

Skill Score
-40

-50

-60

-70

-80

-90
0 5 10 15 20 25
Hours

Figure B.136 Average hourly skill score for the SVM Models. Nominal coverage rate = 10%
0
W/O Bldg Meas
WITH Bldg Meas
-10

-20

-30
Skill Score

-40

-50

-60

-70

-80

-90
0 5 10 15 20 25
Hours

Figure B.137 Average hourly skill score for the SVM Models. Nominal coverage rate = 15%
0
W/O Bldg Meas
WITH Bldg Meas
-10

-20

-30
Skill Score

-40

-50

-60

-70

-80

-90
0 5 10 15 20 25
Hours

Figure B.138 Average hourly skill score for the SVM Models. Nominal coverage rate = 20%
 204
0
W/O Bldg Meas
WITH Bldg Meas
-10

-20

-30

Skill Score
-40

-50

-60

-70

-80

-90
0 5 10 15 20 25
Hours

Figure B.139 Average hourly skill score for the SVM Models. Nominal coverage rate = 25%
0
W/O Bldg Meas
WITH Bldg Meas
-10

-20

-30
Skill Score

-40

-50

-60

-70

-80

-90
0 5 10 15 20 25
Hours

Figure B.140 Average hourly skill score for the SVM Models. Nominal coverage rate = 30%
0
W/O Bldg Meas
WITH Bldg Meas
-10

-20

-30
Skill Score

-40

-50

-60

-70

-80

-90
0 5 10 15 20 25
Hours

Figure B.141 Average hourly skill score for the SVM Models. Nominal coverage rate = 35%
 205
0
W/O Bldg Meas
WITH Bldg Meas
-10

-20

-30

Skill Score
-40

-50

-60

-70

-80

-90
0 5 10 15 20 25
Hours

Figure B.142 Average hourly skill score for the SVM Models. Nominal coverage rate = 40%
0
W/O Bldg Meas
WITH Bldg Meas
-10

-20

-30
Skill Score

-40

-50

-60

-70

-80

-90
0 5 10 15 20 25
Hours

Figure B.143 Average hourly skill score for the SVM Models. Nominal coverage rate = 45%
0
W/O Bldg Meas
WITH Bldg Meas
-10

-20

-30
Skill Score

-40

-50

-60

-70

-80

-90
0 5 10 15 20 25
Hours

Figure B.144 Average hourly skill score for the SVM Models. Nominal coverage rate = 50%
 206
0
W/O Bldg Meas
WITH Bldg Meas
-10

-20

-30

Skill Score
-40

-50

-60

-70

-80

-90
0 5 10 15 20 25
Hours

Figure B.145 Average hourly skill score for the SVM Models. Nominal coverage rate = 55%
0
W/O Bldg Meas
WITH Bldg Meas
-10

-20

-30
Skill Score

-40

-50

-60

-70

-80

-90
0 5 10 15 20 25
Hours

Figure B.146 Average hourly skill score for the SVM Models. Nominal coverage rate = 60%
0
W/O Bldg Meas
WITH Bldg Meas
-10

-20

-30
Skill Score

-40

-50

-60

-70

-80

-90
0 5 10 15 20 25
Hours

Figure B.147 Average hourly skill score for the SVM Models. Nominal coverage rate = 65%
 207
0
W/O Bldg Meas
WITH Bldg Meas
-10

-20

-30

Skill Score
-40

-50

-60

-70

-80

-90
0 5 10 15 20 25
Hours

Figure B.148 Average hourly skill score for the SVM Models. Nominal coverage rate = 70%
0
W/O Bldg Meas
WITH Bldg Meas
-10

-20

-30
Skill Score

-40

-50

-60

-70

-80

-90
0 5 10 15 20 25
Hours

Figure B.149 Average hourly skill score for the SVM Models. Nominal coverage rate = 75%
0

-10

-20
W/O Bldg Meas
WITH Bldg Meas
-30
Skill Score

-40

-50

-60

-70

-80

-90
0 5 10 15 20 25
Hours

Figure B.150 Average hourly skill score for the SVM Models. Nominal coverage rate = 80%
 208
0

-10

W/O Bldg Meas

-20
WITH Bldg Meas

-30

Skill Score
-40

-50

-60

-70

-80

-90
0 5 10 15 20 25
Hours

Figure B.151 Average hourly skill score for the SVM Models. Nominal coverage rate = 85%
0

-10

W/O Bldg Meas

-20
WITH Bldg Meas

-30
Skill Score

-40

-50

-60

-70

-80

-90
0 5 10 15 20 25
Hours

Figure B.152 Average hourly skill score for the SVM Models. Nominal coverage rate = 90%
0

-10
W/O Bldg Meas
WITH Bldg Meas
-20

-30
Skill Score

-40

-50

-60

-70

-80

-90
0 5 10 15 20 25
Hours

Figure B.153 Average hourly skill score for the SVM Models. Nominal coverage rate = 95%
 209
 Average hourly skill score for the SAM models
0
W/O Bldg Meas
WITH Bldg Meas
-10

-20

-30

Skill Score
-40

-50

-60

-70

-80

-90
0 5 10 15 20 25
Hours

Figure B.154 Average hourly skill score for the SAM Models. Nominal coverage rate = 5%
0
W/O Bldg Meas
WITH Bldg Meas
-10

-20

-30
Skill Score

-40

-50

-60

-70

-80

-90
0 5 10 15 20 25
Hours

Figure B.155 Average hourly skill score for the SAM Models. Nominal coverage rate = 10%
0
W/O Bldg Meas
WITH Bldg Meas
-10

-20

-30
Skill Score

-40

-50

-60

-70

-80

-90
0 5 10 15 20 25
Hours

Figure B.156 Average hourly skill score for the SAM Models. Nominal coverage rate = 15%
 210
0
W/O Bldg Meas
WITH Bldg Meas
-10

-20

-30

Skill Score
-40

-50

-60

-70

-80

-90
0 5 10 15 20 25
Hours

Figure B.157 Average hourly skill score for the SAM Models. Nominal coverage rate = 20%
0
W/O Bldg Meas
WITH Bldg Meas
-10

-20

-30
Skill Score

-40

-50

-60

-70

-80

-90
0 5 10 15 20 25
Hours

Figure B.158 Average hourly skill score for the SAM Models. Nominal coverage rate = 25%
0
W/O Bldg Meas
WITH Bldg Meas
-10

-20

-30
Skill Score

-40

-50

-60

-70

-80

-90
0 5 10 15 20 25
Hours

Figure B.159 Average hourly skill score for the SAM Models. Nominal coverage rate = 30%
 211
0
W/O Bldg Meas
WITH Bldg Meas
-10

-20

-30

Skill Score
-40

-50

-60

-70

-80

-90
0 5 10 15 20 25
Hours

Figure B.160 Average hourly skill score for the SAM Models. Nominal coverage rate = 35%
0
W/O Bldg Meas
WITH Bldg Meas
-10

-20

-30
Skill Score

-40

-50

-60

-70

-80

-90
0 5 10 15 20 25
Hours

Figure B.161 Average hourly skill score for the SAM Models. Nominal coverage rate = 40%
0
W/O Bldg Meas
WITH Bldg Meas
-10

-20

-30
Skill Score

-40

-50

-60

-70

-80

-90
0 5 10 15 20 25
Hours

Figure B.162 Average hourly skill score for the SAM Models. Nominal coverage rate = 45%
 212
0
W/O Bldg Meas
WITH Bldg Meas
-10

-20

-30

Skill Score
-40

-50

-60

-70

-80

-90
0 5 10 15 20 25
Hours

Figure B.163 Average hourly skill score for the SAM Models. Nominal coverage rate = 50%
0
W/O Bldg Meas
WITH Bldg Meas
-10

-20

-30
Skill Score

-40

-50

-60

-70

-80

-90
0 5 10 15 20 25
Hours

Figure B.164 Average hourly skill score for the SAM Models. Nominal coverage rate = 55%
0
W/O Bldg Meas
WITH Bldg Meas
-10

-20

-30
Skill Score

-40

-50

-60

-70

-80

-90
0 5 10 15 20 25
Hours

Figure B.165 Average hourly skill score for the SAM Models. Nominal coverage rate = 60%
 213
0
W/O Bldg Meas
WITH Bldg Meas
-10

-20

-30

Skill Score
-40

-50

-60

-70

-80

-90
0 5 10 15 20 25
Hours

Figure B.166 Average hourly skill score for the SAM Models. Nominal coverage rate = 65%
0
W/O Bldg Meas
WITH Bldg Meas
-10

-20

-30
Skill Score

-40

-50

-60

-70

-80

-90
0 5 10 15 20 25
Hours

Figure B.167 Average hourly skill score for the SAM Models. Nominal coverage rate = 70%
0
W/O Bldg Meas
WITH Bldg Meas
-10

-20

-30
Skill Score

-40

-50

-60

-70

-80

-90
0 5 10 15 20 25
Hours

Figure B.168 Average hourly skill score for the SAM Models. Nominal coverage rate = 75%
 214
0
W/O Bldg Meas
WITH Bldg Meas
-10

-20

-30

Skill Score
-40

-50

-60

-70

-80

-90
0 5 10 15 20 25
Hours

Figure B.169 Average hourly skill score for the SAM Models. Nominal coverage rate = 80%
0

-10

-20

W/O Bldg Meas

-30 WITH Bldg Meas
Skill Score

-40

-50

-60

-70

-80

-90
0 5 10 15 20 25
Hours

Figure B.170 Average hourly skill score for the SAM Models. Nominal coverage rate = 85%
0

-10

-20 W/O Bldg Meas

WITH Bldg Meas

-30
Skill Score

-40

-50

-60

-70

-80

-90
0 5 10 15 20 25
Hours

Figure B.171 Average hourly skill score for the SAM Models. Nominal coverage rate = 90%
 215
0

-10

W/O Bldg Meas

-20 WITH Bldg Meas

-30

Skill Score
-40

-50

-60

-70

-80

-90
0 5 10 15 20 25
Hours

Figure B.172 Average hourly skill score for the SAM Models. Nominal coverage rate = 95%
 216

VITA

FULL NAME: Jonathan Berardino

DATE/PLACE OF BIRTH: 03/03/1982, Peabody, MA

EDUCATION: PhD. Electrical Engineering Drexel University, PA, USA 2016

B.S. Electrical Engineering Bucknell University, PA, USA 2004

SELECT PUBLICATIONS:
[J1]. Jonathan Berardino, Chika Nwankpa, “Enhanced Demand Forecasting Through
Inclusion of Building-Specific Measurements”, Submitted to IEEE Transactions on
Power Systems
[J2]. Jonathan Berardino and Chika Nwankpa, "Demand Scheduling for Multi-
Building Campuses Considering Building Dynamic Behavior", (In preparation)
[J3]. Bruce Stephen, Jennifer Shaw, Jonathan Berardino, Stuart Galloway, Chika
Nwankpa, “Intra-day Error Dependency Boosting for Short Term Building and LV
Network Load Forecast Models” Submitted to IEEE Transactions on Smart Grid
[C1]. Jonathan Berardino and Chika Nwankpa, "Statistics of Building-Specific Load
Forecasting Models", Proceedings of the North American Power Symposium 2013,
September 2013, Manhattan, KS
[C2]. Jonathan Berardino and Chika Nwankpa, " Inclusion of Temporal Effects in
Forecasting Building Electrical Loads for Demand Resource Planning", Proceedings of
the IEEE PowerTech 2013 Conference, June 2013, Grenoble, France
[C3]. Jonathan Berardino and Chika Nwankpa, "Economic Demand Dispatch of
Controllable Building Electrical Loads Incorporating Delayed Response Times",
Innovative Smart Grid Technologies (ISGT), 2013, Washington DC
[C4]. Jonathan Berardino and Chika Nwankpa, "Interval-Specific Building Load
Forecasting Models for Demand Resource Planning", Proceedings of the IEEE
PowerCon 2012 Conference, October 2012, Auckland, New Zealand
[C5]. Jonathan Berardino, Mohammed Muthalib, Chika Nwankpa, “Network
Constrained Economic Demand Dispatch of Controllable Building Electric Loads”,
Innovative Smart Grid Technologies (ISGT), 2012, Washington DC
[C6]. Jonathan Berardino, Chika Nwankpa, Karen Miu, “Economic Demand Dispatch
of Controllable Building Electric Loads for Demand Response”, Proceedings of the IEEE
PowerTech 2011 Conference, June 2011, Trondheim, Norway
[C7] Jonathan Berardino and Chika Nwankpa, “Dynamic Load Modeling of an
HVAC Chiller for Demand Response Applications”, IEEE International Conference on
Smart Grid Communications, 2010, Gaithersburg, MD