Environmental Life Cycle
Assessment
CEE 12-714 / EPP 19-714
Lecture 11: Modeling Uncertainty in LCA
February 21, 2018
Administrivia
2
• HW3 due tomorrow, Feb 22nd
Significant figures are important
Watch your formatting – make it easy for grading!
Comma format large numbers
$ signs if relevant
Page numbers and headers
Last assignment with formatting points
• HW4 will be released tomorrow
2
More administrivia
• Monday, we start a 4-part series of lectures on
input output modeling, including an important
(fun) class exercise
See Canvas readings
Be on time!
• Tuesday, office hours will be an optional lecture
on stochastic modeling (more details than today)
3
Group projects
4
• Proposals due no later than Monday, Feb 26,
midnight
Submit sooner if you want earlier feedback
I will provide feedback to you ASAP
Canvas project deliverables opened, see for additional
details and rubrics
• Schedule team meetings with me to discuss
your draft proposals (optional)
Team 4, today, 1 pm
4
Course trajectory
1. Introductions 10. Uncertainty
2. Life cycle thinking 11. Input-output LCA
3. Quantitative methods and 12. Process-matrix LCA
life cycle cost analysis
13. Hybrid LCA
4. ISO LCA framework
14. Impact assessment
5. Critical review
15. Structural path analysis
6. LCA data sources
16. Professional responsibility
7. Life cycle inventory
17. Carbon footprinting
8. SimaPro
18. LCA for big decisions
9. Handling multifunction
19. Project presentations
systems
5
ISO 14040: Figure 1
Phases of an LCA
6
• Goal and scope
definition
• Inventory
• Impact assessment
• Interpretation
6
Uncertainty
Types, sources,
how will we deal
with it?
7
“A decision made without taking uncertainty
into account is barely worth calling a
decision.” Wilson (1985)
8
From Lecture 3
Uncertainty vs. Variability
9
• Uncertainty: exists because of ignorance or lack
of data
Likely reducible with further study
• Variability: exists because of heterogeneity or
diversity
Unlikely to be reducible with more study
• We assume they are the same, and call them
both uncertainty
• And do “uncertainty analysis”
Adapted from Margrit von Braun, Univ of Idaho 9
Deterministic Context in LCA
• Most studies still deterministic (point
estimates, no uncertainty)
Result: a single LCI value (or list of single values)
• Typical use: hot spot or comparison
Hot spot: which LCI value is bigger than all others?
Comparison: which is lower, A or B?
• Typical analysis: Simply A < B
• Typical result: It depends
10
How are comparisons done in other
domains with data?
• How do we check whether A < B?
We set up a hypothesis and apply statistical tests (e.g.,
t-test) and assess significance
• Why doesn’t LCA do this?
Easy answer: there hasn’t been sufficient data, or
sufficient demand for this level of technical complexity.
That is changing.
11
ISO 14040 says…
• LCA addresses potential environmental
impacts; LCA does not predict absolute or
precise environmental impacts due to:
relative expression of potential environmental
impacts to a reference unit
integration of environmental data over space and
time
inherent uncertainty in modelling environmental
impacts
some possible environmental impacts are clearly
future impacts
12
ISO 14044 says…
• Data quality requirements should address
uncertainty of information
data, models, and assumptions
• “Uncertainty analysis and sensitivity analysis
shall be done for comparative studies intended
for public release.”
13
Text from an actual LCA
How well does this statement
handle uncertainty?
14
Paper vs. plastic
• Lave et al compared energy use (electricity only) of
plastic and paper cups (1995)
• Plastic cup consumed ~50% less electricity
plastic cup: 4,400 kWh
paper cup: 8,600 kWh
• Updated to consider uncertainty (and total energy)
(Chen, 2017):
plastic cup: 0.3 – 0.7 TJ
paper cup: 0.3 – 0.4 TJ
Original conclusions change
Overlapping range suggest high potential for ~ same
energy use, or for lower energy use for paper cup
15
Back to the decision context
• Use care in using simplistic methods to support
decisions
• “A decision made without taking uncertainty into
account is barely worth calling a decision.”
(Wilson, 1985)
• Would you really want to redesign a process
around the result of a deterministic LCA?
16
Decisions that matter
• Presumed focus of book and course
We’re doing LCA for big decisions
Hybrid cars? Green energy? Change-the-world policy
questions
• Thus analysis should be as good as possible
• “It depends” studies perhaps just not “complete”
studies
• LCA is a robust methodology that requires
careful choice of methods and leverage of
quantitative tools.
17
Biofuels and land use changes
• Growing crops for food or fuel requires land
• Converting land use creates significant GHG
emissions
e.g., forests to farms
Biomass decomposition, disturbance of soils and other
activities
• Promoting biofuels causes crop price increases
Triggers increased production and conversion of land
More GHG emissions from increased biofuels
• Results of studies of indirect land use change (ILUC)
emissions vary
18
Biofuels and ILUC
• No way to measure ILUC directly
• No objective way to tell which estimate is correct
• Researchers have conservatively considered
ILUC effects in biofuel systems, concluding that
biofuels can reduce GHG emissions
• However using higher estimates of ILUC effects
on GHG emissions would reverse conclusion
19
Biofuels and ILUC
• Conclusion ‘depends’ highly on which ILUC
values are considered
• Use of any one of the ILUC estimates creates
very narrow (misleading?) conclusions
• The uncertainty matters.
20
Measurement vs Accounting
• Measurement: observable quantity with an
ideal way to measure; limits to precision
Uncertainty range might be described as +/- 1%
Could be improved with better measurement tools
• LCI accounting: may lack primary data, raw
data leveraged to estimate flows
Uncertainty ranges likely to be appreciable
Results roll up many, many processes
• Do stakeholders expect measurement-quality
results?
21
General Overview of a Model
22
Data or input uncertainty
• Measurement uncertainty
• Parameter uncertainty
Survey errors
Incomplete and missing data
Unit conversions
• Geospatial uncertainty
• Temporal uncertainty
Old data
Forecasting
23
Data or input uncertainty:
Measurement
• Imperfect measurement technologies (or
measurers)
24
Data or input uncertainty: Parameter
• Survey data
Coverage: sample or census
Reporting errors
Compiling errors
Data suppressed due to confidentiality claims
• Incomplete and missing data
Flows that are not included
Sectors that are not studied
Data unavailable for smaller businesses
25
Data or input uncertainty: Geospatial
• Different processes used:
Titanium dioxide is made by a sulfide process in
China, and a chloride process in the U.S.
• Different electricity grids
Regional natural or built resources
Coal, gas, hydro, nuclear, renewables?
Age and efficiency
Pollution controls
National vs. micro
26
Foreign production?
• Are inputs manufactured in same region as
target product?
• Production in other regions may be similar OR
substantially different
Different levels of environmental regulation
Or environmental protection
27
Data or input uncertainty:
Temporal
• Old data
Technology changes
All US titanium dioxide sulfide processes have halted
Three major US titanium dioxide chloride processes have
dramatically reduced dioxin/furan loads
Computing: Desktops, laptops, tablets, handheld
Pricing changes: competition, innovation, resource
constraints
• Forecasting
Distribution of USLCI basis years 28
Temporal uncertainty
29
Example: Iron and Steel in US, Japan, China
Williams, Weber, Hawkins, 2009 29
Model or method uncertainty
• Process-based model
• Cutoff
• Software tool
• Database
• Allocation
• Others…
Aggregation
Input-output model
Impact assessment method
30
Model or method uncertainty:
Process-based models
• How good is the model’s resolution (number of
processes in the model)?
• How well characterized are the input and output
flows?
• Presumed to be linear
31
Model or method uncertainty:
Cutoff error
• Cutoff or truncation error in
process LCA: not all
processes are included
Cutoff or dummy flows in
USLCI, AgLCI, Ecoinvent
• Matthews et al (2008):
average industry captures
only 25% of upstream
cradle-to-gate using carbon
footprint methods
32
Model or method uncertainty:
Cutoff error
• Leaving out disposal/recycling stage
Often done due to scenario uncertainty
• Landfill, incineration
Depends on product/scope
Not usually important for energy
Incineration might dominate for dioxin
• Recycling
Can be significant
For buildings, embodied materials energy is 30,000
GJ, out of 360,000 GJ over 50 year life cycle
energy
(Junnila, Horvath, Guggemos, Journal Of
Infrastructure Systems, 11, March, 2006) 33
Model or method uncertainty:
Software and databases
• Internal differences in software setups (SimaPro
versus Gabi versus OpenLCA)
• Default database setups
Default scope boundaries
Default flows
• Read the metadata
34
Model or method uncertainty:
Allocation
• Allocation methods yield different results
No apriori or consensus
• How important is allocation in an LCA?
• Example: how should we assess personal
transport impacts associated with purchasing a
good?
100% of impacts?
Allocate by number of passengers, by number of items
purchased, other?
Assess using scenario-based sensitivity analysis
35
Uncertainty in results
• Results are uncertain due to accumulated effect
of all data, inputs, assumptions, and methods
• No unique type of result uncertainty
• Rolled up uncertainties in results are most
visible and tangible expressions of uncertainty
to both practitioner and audience
This is the uncertainty that matters the most
36
Methods to address uncertainty
• Qualitative
• Semi-quantitative
Significance heuristics
Pedigree matrix
• Quantitative
Ranges
Sensitivity analysis
Probabilistic methods and simulation
37
Qualitative
• Discus sources of uncertainty
• Textual summary without quantification
• Reliability: “All data for key processes are based
on measurements (primary data), so uncertainty
is deemed to be relatively low for this category.”
• Completeness: “Various processes include only
effects of direct production, leading to some
cutoff uncertainty.”
See Fig 7-6 for more examples 38
Semi-quantitative: Heuristics
• Rule of thumb, preset “rule” for comparisons
Example: uncertainties of energy and carbon emissions
~20%; other LCI categories more uncertain
Differences <20% … inconclusive
No solid science behind “20%”, but a useful screening
tool
• True quantitative better!
Maintain enough significant
digits for comparison:
0.6 and 1.4, not 1 and 1
39
Quantitative methods
• Ranges
• Sensitivity analysis
• Probabilistic methods and simulation
40
Using ranges to understand uncertainty
• Use ranges for inputs or outputs
• Recommended: use multiple data sources, not
single values
41
Using ranges to understand uncertainty
• Ranges quantitatively represents effects of
different assumptions/boundaries in underlying
data
helping to show when they matter
• Appropriate graphical range representation:
“uncertainty bars”
Linear representation of ranges of results with upper
and lower bounds
AKA “error bars” in Excel
42
See also Lecture 3
43
Process Flow Diagram-based
Example with Ranges
• See E-resource for Ch. 7, ‘Complex Models from
USLCI with Uncertainty’
• Template for comparing modules to develop LCI
uncertainty ranges
• Example compares:
Electricity from bituminous coal
Electricity from lignite coal
Electricity from anthracite coal
Includes upstream (scaled) processes for coal mining
(by type) and transport (by type)
44
Note scale is logarithmic
45
Covered also in
Sensitivity analysis Lecture 3, and
discussed in
Chapter 2 and 4
• Quantitative method
• Assess effect on results from changing a single
input
Change one variable at a time; hold all others constant
• Other methods, such as multi-way sensitivity
analysis and simulation, can show effects of
changing more than one variable at a time
46
ISO and Sensitivity Analysis
14040:3.31 sensitivity analysis – systematic
procedures for estimating the effects of the choices
made regarding methods and data on the outcome
of a study
47
Sensitivity analysis
14044:4.3.3.4 Refining the system boundary –
“Reflecting the iterative nature of LCA, decisions
regarding the data to be included shall be based on a
sensitivity analysis to determine their significance…”
“The sensitivity analysis may result in
exclusion of life cycle stages or unit processes when lack of
significance can be shown by the sensitivity analysis,
exclusion of inputs and outputs that lack significance to the
results of the study, or
inclusion of new unit processes, inputs and outputs that are
shown to be significant in the sensitivity analysis.”
48
ISO and Sensitivity Analysis
14044:4.3.4.1: Allocation “…Whenever several
alternative allocation procedures seem applicable,
a sensitivity analysis shall be conducted to
illustrate the consequences of the departure from
the selected approach.”
• And in LCIA to show impact of different modeling
choices (normalization, weighting)
To be covered later in the semester
• Required for “comparative assertions intended
to be disclosed to the public”
49
Sensitivity analysis
• “By hand” or using sensitivity analysis tools in
Matlab, @Risk, Excel (What-If analysis),
SimaPro (if you have a higher level license than
the course license), etc.
• Applies to inputs and assumptions (parameter
choices, allocation methods, etc.)
• Choose appropriate sensitivity ranges
What makes sense?
Not just automatic acceptance of +/-50% defaults
50
Depicting sensitivity analysis
• Simple hi/lo chart
(Excel or Matlab)
• @Risk, TopRank or
other software to
generate tornado or
spider type graphs
51
Case study – Uncertainty in Process-
based flows
• Examine USLCI for all
US regional electricity
flows to show range of
potential CO2e
emissions per kWh
52
Leslie Abrahams, PhD
CEE/EPP Doctoral research
Spotlight on
LCA research
53
LNG Supply Chain
54
LNG Supply Chain
Currently, the export approval
process only requires an assessment
of localized environmental impacts
from the liquefaction facility
How would an increase in U.S. LNG
exports would impact global GHG
emissions from a life cycle perspective?
55
Electricity Generation
Exporting natural gas results
in 11% higher emissions than
domestic combustion
56
Displacing Traditional Energy
Sources for Electricity Generation
45%
13%
57
LNG Exports LCA
• Debate over fugitive emissions rate for
upstream natural gas life cycle stages. Handled
three ways:
“Most likely” range commonly cited in literature
represented as triangular distribution (minimum 2%,
maximum 4%, most likely value 3%)
Sensitivity analysis showing effects of fugitive
emissions rates across range encompassing most
values discussed in the literature (1−9%)
Discussion of break-even fugitive emissions rate that
would change result of analysis
58
Sensitivity of life cycle emissions of LNG exports
for electricity generation to fugitive emissions
rates
59
LNG Exports LCA Conclusions with
uncertainty and sensitivity
• “…emissions from the liquefaction, shipping, and
regasification segments of the LNG life cycle are
fewer than 11% of the total life cycle emissions
of LNG exports for electricity generation based
on a 100-year GWP and 3% average fugitive
emission rate.”
• “This percentage would continue to decrease as
a result of increased methane leakage rates
and/or a 20-year GWP assumption.”
60
LNG Exports LCA conclusions with
uncertainty and sensitivity considerations
• “Based on a sensitivity analysis of these results, the
key model parameters that can have a significant
impact on the LNG life cycle emissions are
the end use efficiency,
the GWP (both time horizon used, and value within a given
time horizon probability distribution), and
the fugitive emissions rate.”
• “Other uncertain parameters, however, such as
liquefaction plant efficiency, tanker capacity, tanker
fuel, and shipping distance can vary widely without
materially affecting the overall life cycle emissions.”
61
LNG Exports
Figure 2. Life cycle emissions for electricity generation from natural gas
exports (A) by life cycle stage for a 100-year GWP, and (B) for the complete
life cycle for both a 100-year and 20-year GWP.
62
Uncertainty … iterations
• Adding uncertainty allows for screening tools
that ‘screen’ based on:
Magnitude of effect (likely most important effects)
Uncertainty
…Indicators of where additional data needed in
subsequent effort.
63
Environmental Life Cycle
Assessment
CEE 12-714 / EPP 19-714
Lecture 11a: Stochastic modeling in LCA (optional)
February 27, 2018
Uncertainty and probabilistic
methods
• We know we have uncertainty in…
Inputs
Outputs
Model
• Goal: to understand and model appropriately
uncertainty so we can make better decisions
• Simulation can advance that objective
65
Uncertainty Basics
• Uncertainty is inherent in everything
There are no right answers (just poor assumptions)
• We can analyze uncertainty using sensitivity
analysis
• We can model uncertainty by using ranges or
distributions for our input variables
66
Probability versus ranges
• Ranges imply at least a sort
of distribution
But ranges quickly get
prohibitively complex
5 variables only evaluated at
max/min points creates 32
different combinations to
evaluate if we abandon moving
only one value at a time
• Probabilistic modeling allows
us to move beyond the
limitations of ranges
67
Using probabilistic methods to
understand uncertainty
• Use data from various sources to generate
probability distributions for inputs
• Use spreadsheets or other techniques to track
the effect of these probabilities through the
model
• Generate results that also have probability
distributions
68
Using probabilistic methods to
understand uncertainty: INPUTS
• Not point estimates or ranges
• Distributions defined based on multiple data
sources and methods like curve fitting
• Actual data used to make distributions
Fit, or assume a particular type of distribution given
data properties
69
2 Types of Probability Distributions
• Discrete distribution
The uncertain quantity can only be a finite or countable
number of values
Binary events (0 or 1), number of chocolate chips in a
cookie
Binomial, Poisson
• Continuous distribution
The uncertain quantity can take on any value in a
given range
Temperature, age
Normal, uniform, beta
70
Discrete and continuous distributions
Discrete Distributions Continuous Distributions
71
Choosing the right distributions
Normal Distribution
• Central tendency
common to many
physical and social
phenomena
• Beware of long tails!
• Not suitable for
variables that are
inherently positive or
strongly skewed
(weight, stock price)
72
Choosing the right distributions
Lognormal
Distribution
• Only positive values,
well defined
minimum limit
• Mean can be very
distant from mode
• Traditional
distribution for
reliability analyses
73
Choosing the right distributions
Uniform Distribution
• Reflective of truly
random, but
constrained quantities
• Good when
justification for
distribution
parameters is lacking
74
Choosing the right distributions
Triangle
Distribution
• Good medium
between normal and
uniform distributions
• Can be too restrictive
on values near the
max/min limits
• When in doubt… not
a bad choice.
75
Choosing the right distributions
Poisson
Distribution
• Discrete distribution
common to
queuing/queuing
problems
• cars arriving at a
traffic light
• telephone calls
arriving in a system
• photons arriving at a
telescope
76
STOP !
Do you really need a distribution?
• What doesn’t need a distribution?
User criteria
Discount rates, safety factors, etc.
User decisions
Which investment to select, etc.
Anything that might be causally related to a variable
already in the model…
• Another individual’s criteria or decisions may be
correctly modeled as a randomized distribution
77
Justify distribution parameters
• Distribution parameters must be justified
Can come from existing data
From personal experience
From similar known mechanisms in analogous
situations
• Pay attention to independence
Is it valid to consider your inputs independent?
78
Depicting Probability Distributions
• Probability Mass or Density Function
Describe the density of probability of a random
variable taking on a value in the sample space
PMF for discrete distributions (similar to a histogram)
PDF for continuous distributions
X
PMF PDF
P(X = x)
Binomial Distribution (Discrete) Normal Distribution (Continuous) 79
Depicting Probability Distributions
• Cumulative Distribution Function
Probability that a random variable is less than or equal
to a given value in the sample space
Used for both discrete and continuous distributions
CDF
P(X ≤ x)
CDF
Binomial Distribution (Discrete) Normal Distribution (Continuous)
80
Relationship between PMF/PDF and CDF
Binomial Distribution (Discrete)
PMF CDF
P(X = x)
P(X ≤ x)
Add
X Normal Distribution (Continuous) X
PDF
Integrate
CDF
81
Know how to read PDFs & CDFs
• Mean?
• Minimum, maximum?
• 5-95%?
CDF
82
When variables are distributions, how
can we use them?
• LCA models are based on mathematical
manipulation
What are the rules when your variables are distributions?
• Some cases are easy
Adding Normal distributions
N(μ1,σ1) + N(μ2,σ2) = N(μ1+ μ2,√(σ12+σ22))
Multiplying Lognormal distributions
LN(a1,b1) x LN(a2,b2) = LN(a1+ a2,√(b12+b22))
• Very difficult for models of complex systems
N(μ1,σ1) x LN(a1,b1) = ?
Binomial(n,p) x [Exponential(λ) + Uniform(a,b)] = ?
83
Solution:
Stochastic (or Monte Carlo) modeling
• Allows for modeling of uncertain parameters
across their ranges of probabilities
• Output represents the resultant range of
possibilities…
12-706 84
How does the simulation work?
• A model is a set of calculations
• Variables are defined as distributions
• The model is run many times, each time taking
random samples from the variable distributions
• The resultant set of model results (outputs) is
evaluated as an output distribution
• Let’s see an example.
85
Intro Resilience Trips Efficiency Air Conclusions
Annual Waterways Shipments, 2012
Nationwide: 2.3 billion tons
Port of Pittsburgh: 35 million tons
Port of Huntington: 53 million tons
(USACE)
Source: Port of Pittsburgh Commission
86
Regional Inland Waterway
Dataset from
Corps of
Engineers
56,000 records of
each lockage in
2010
Derive time to
travel between
locks, delays
before locking,
time locking
Source: USACE
87
Issue: Long delayed infrastructure
improvement causes trip delay
• Most Corps analyses focus on delays at one
lock
• Most trips traverse multiple locks, with
compounding delays
• Infrastructure improvements reduce trip time
• Challenge: how to measure a “trip”?
• Dataset yielded distributions of
– Time to travel between locks
– Delays before locking
– Time in lock
88
Charleroi Locks and Dam
90
91 of
25
Modeling transit time
Coal to Fort Martin Powerplant
(EIA)
2.75 million tons in 2010 92
Shipping Time Components
• Stochastic model of time savings from
completion of the Lower Mon Project
Total travel time (hours)= Time underwayhours +
Probability of delay * Delayhours +
Lockage timehours + Reconfigurationhours
• 15-barge tow shipment from Powhatan No. 1
mine to Fort Martin
– 200 miles, 11 locks, 164,000 tons in 2010
– 17,200 to 22,100 tons per tow
Setting up a travel time model
Total travel time (hours)=
Time underwayhours
+ (Probability of delay * Delayhours )
+ Lockage timehours
+ Reconfigurationhours
+[ * ]+ +4
Pearson Binomial Lognormal Loglogistic
Monte Carlo (Stochastic) Simulation
Take K random samples of each distribution
Fit a distribution and
Uncertain Variables Random Random
… determine the relevant
(with parameters) Sample 1 Sample K parameters
4.1 … 3.2
Plot a histogram
1.2 … 0.3 and/or
a PDF
50 … 76
Calculated output
for each set of 1209 … 1638 Produce K output values
random samples
95
Model results
Stochastic dominance
Jumbo barge shows
measurable
improvement
Mixed tows show little
improvement
LCA of the Yellow Pages:
Does Pittsburgh Still Need the Big
Book?
Functional unit: one
volume of “Yellow
Pages,” including its
production,
distribution, and
disposal
System boundary: all
households in Allegheny
County; excludes
commercial and
institutional
establishments 97
Yellow Pages LCA Stages
• Phonebook production
• Freight of phonebooks from manufacturing plant
to a Pittsburgh warehouse
• Warehouse storage prior to distribution
• Distribution
• Production of fuel used in distribution
• End of life
98
Sample Distributions Used in
Yellow Pages LCA
99
Distribution fitting with @Risk
100
Presenting LCA results considering
uncertainty
101
Stochastic modeling tools
• Simple models – Excel
• Palisades DecisionTools Suite (@Risk)
Excel add-on makes it easy (and Help Content
is super)
Cluster computers
Download from IT Services
https://userguide.its.ce.cmu.edu/computing-
help/software/
• Other courses cover the underlying math
and more sophisticated application in much
more depth…
102
Using probabilistic methods to
understand uncertainty: RESULTS
• Result of simulation is a probability distribution
• Greater iterations of simulation, resulting output
distribution should converge to actual
distribution of results
103
Using probabilistic methods to
understand uncertainty: RESULTS
• Results as probability distributions (e.g., pdf/cdf)
let us leverage probabilities to provide decision
support beyond what is possible with ranges
Range-based analyses only show where ranges
overlap
Probability distributions can compare distributions to
determine the likelihood that performance of A is
superior to B
104
Summary
• LCA involves considerable uncertainty
• Can be reduced with careful analysis of
underlying data and production processes
• Relative desirability of design alternatives can
be assessed with greater confidence than
overall impact of a single alternative
• Unrealistic to expect that we could support a big
decision without accounting for uncertainty.
105
Questions?
106
Next
• HW3 due tomorrow, Feb 22nd
• Monday, project proposals (goal and scope) due by
midnight
• Monday, we start a 4-part series of lectures on input
output modeling, including an important (fun) class
exercise
Chapter 8 thru Ex. 8-2.
See additional Canvas readings
Be on time!
• Tuesday, office hours will be an optional lecture on
stochastic modeling (more details than today)
107