2/18/2020

CHENG 111
Introduction to Chemical
Engineering
Dr. Ajaz Rashid

Chapter 2

Introduction to Engineering
Calculations

1
 2/18/2020

Chapter Objectives

▪ Convert a quantity expressed in one set of units into another set of units.
▪ Identify the units commonly used to express both mass and weight in SI, CGS, and U.S.
customary units.
▪ Identify the number of significant figures in a given value expressed in either decimal or
scientific notation and determine the correct number of significant figures in the result of
a series of arithmetic operations (adding, subtracting, multiplying, and dividing).
▪ Validate a quantitative problem solution by applying back-substitution, order-of-
magnitude estimation, and the test of reasonableness.
▪ Given a set of measured values, able to do statistical analysis and plot/analyze the data
using interpolation and data fitting tools.

2.1 Units and Dimensions

▪ A dimension is a property that can be

▪ measured, such as length, time, mass, or temperature, or
▪ calculated by multiplying or dividing other dimensions, such as length/time
(velocity), length3 (volume), or mass/length3 (density).

▪ A unit is a way to assign a numerical value to the measurement of that

dimension
▪ Mass is a dimension
▪ Kilogram (kg) is a unit
▪ kilogram helps us to assign a numerical value to mass (e.g. mass of a bottle of HCl is
measured to be 1.2 kg)
▪ Using a different unit will result in a different numerical value, generally

2
 2/18/2020

2.1 Units and Dimensions

▪ Units can be treated like algebraic variables when quantities are added,
subtracted, multiplied, or divided
▪ Two quantities may be added or subtracted only if their units are the same

4 cm – 1 cm = 3 cm 3 cm + 2 mm = ??
4𝑥 – 𝑥 = 3𝑥 3𝑥 + 2𝑦 = ??

2.1 Units and Dimensions

▪ Numerical values and their corresponding units may always be combined by

multiplication or division.

3 N × 4 m = 12 N.m 2 m × 2 m = 4 m2
3𝑥 × 4𝑦 = 12 𝑥.𝑦 2𝑦 × 2𝑦 = 4 𝑦2
5 m / 1 s = 5 m/s
6 m / 3 m/s = 2 s
2 g / 2 g = 1 (dimensionless) 5𝑦 ÷ 5𝑦 = 1

3
 2/18/2020

2.2 Conversion of Units

▪ A measured quantity can be expressed in terms of any units having the

appropriate dimension.
▪ Velocity maybe expressed in m/s, ft/s, miles/h, cm/min, km/yr or any other ratio
of a length unit to a time unit.
▪ The numerical value of the velocity will depend on the units chosen.

10 m/s = 32.8 ft/s = 22.4 mph = 60000 cm/min = 315360 km/yr

2.2 Conversion of Units

▪ The equivalence between two expressions of the same quantity may be defined
in terms of a ratio:
1 cm 10 mm 1 hr 1 kg 2.20462 lb𝑚
10 mm 1 cm 60 min 2.20462 lb𝑚 1 kg
▪ These ratios are known as conversion factors.
▪ To convert a quantity expressed in terms of one unit to its equivalent in terms of
another unit, multiply the given quantity by the conversion factor (new unit/old
unit). For example, to convert 1.2 m to its equivalent in cm, write

100 cm
1.2 m × = 120 cm
1m

4
 2/18/2020

2.2 Conversion of Units

▪ TEST YOURSELF (page 7)

1. What is a conversion factor?

2. What is the conversion factor for s/min?
3. What is the conversion factor for min2/s2?
4. What is the conversion factor for m3/cm3?

Quantity Equivalent values

Mass 1 kg = 1000 g = 0.001 metric ton = 2.20462 lbm = 35.27392 oz
1 lbm = 16 oz = 5 x 10-4 ton = 453.593 g = 0.453593 kg
2.2 Conversion of Units
Conversion Factors Table

Length 1m = 100 cm = 1000 mm = 106 microns = 1010 angstroms

= 39.37 in = 3.2808 ft = 1.0936 yd = 0.0006214 mile
1 ft = 12 in. = 1/3 yd = 0.3048 m = 30.48 cm
Volume 1m3 = 1000 L = 106 cm3 = 106 mL
= 35.3145 ft3 = 220.83 imperial gallons = 264.17 gal
= 1056.68 qt
1 ft3 = 1728 in.3 = 7.4805 gal = 0.028317 m3 = 28.317 L
= 28317 cm3
Force 1N = 1 kgm/s2 = 105 dynes = 105 gcm/s2 = 0.22481 lbf
1 lbf = 32.174 lbmft/s2 = 4.4482 N = 4.4482 x 105 dynes
1 atm = 1.01325 x 105 N/m2 (Pa) = 101.325 kPa = 1.01325 bar
Pressure = 1.01325 x 106 dynes/cm2 = 760 mm Hg at 0oC (torr)
= 10.333 m H2O at 4oC = 14.696 lbf/in.2 (psi)

Energy 1J = 1 Nm = 107 ergs = 107 dyne.cm = 2.778 x 10-7 kWh

= 0.23901 cal = 0.7376 ft-lbf = 9.486 x 10-4 Btu
Power 1W = 1 J/s = 0.23901 cal/s = 0.7376 ft.lbf/s = 9.486 x 10-4 Btu/s
1.341 x 10-3 hp

10

5
 2/18/2020

2.2 Conversion of Units

▪ Example 2.2-1

▪ Convert an acceleration of 1 cm/s2 to km/yr2.

1 cm 36002 s 2 242 h2 3652 day 2 1 m 1 km

2 2 2 2 2 2 2
= 9.95 × 109 km/yr 2
s 1 h 1 day 1 yr 100 cm 1000 m

11

2.3 Systems of Units

Common Systems of Units

SI Units CGS system English units

Centimeter Gram US Customary

International
Second system system of units
system of units
of units (ft, s, lb)
(m, s, kg)
(cm, s, g) also called FPS

12

6
 2/18/2020

2.3 Systems of Units

A system of Mass, length, time, temperature, electrical current, light

Base units intensity, quantity of matter
units has the
following
components
Multiple units Multiples or fractions of base units

mm cm m dm km

Derived units Obtained in one of the two ways:

Compound unit: by multiplying and Defined equivalent of compound unit:

dividing base or multiple e.g. 1 erg = 1 g.cm/s2,
units; e.g. cm2, ft/min, m/s2 1 lbf = 32.174 lbm.ft/s2

13

2.3 Systems of Units

14

7
 2/18/2020

2.3 Systems of Units

Base Units
Quantity SI CGS English

Length meter (m) centimeter (cm) foot (ft)

Mass kilogram (kg) gram (g) pound (lb)

Moles gram-mole (gmol) gram-mole (gmol) pound-mole (lb-mol)

Time second (s) second (s) second (s)

Temperature Kelvin (K) Kelvin (K) Rankine (oR)

Electric current Ampere (A) Ampere (A) Ampere (A)

Light intensity Candela (cd) Candela (cd) Candela (cd)

15

2.3 Systems of Units

Multiple Unit Preferences

Factor Name Symbol Factor Name Symbol
1024 yotta Y 10-1 deci d
1021 zetta Z 10-2 centi c
1018 exa E 10-3 milli m
1015 peta P 10-6 micro 
1012 tera T 10-9 nano n
109 giga G 10-12 pico p
106 mega M 10-15 femto f
103 kilo k 10-18 atto a
102 hecto h 10-21 zepto z
101 deca da 10-24 yocto y
Examples: 1 GJ = 109 J 1 mg = 10-3 g

16

8
 2/18/2020

2.3 Systems of Units

▪ TEST YOURSELF (page 9)

1. What are the factors (numerical values and units) needed to convert
a) meters to millimeters?
b) nanoseconds to seconds?
c) square centimeters to square meters?
d) cubic feet to cubic meters (use the conversion factor table on the inside front
cover)?
e) horsepower to British thermal units per second?
2. What is the derived SI unit for velocity? The velocity unit in the CGS system? In
U.S. customary units?

17

2.3 Systems of Units

▪ Example 2.3-1

▪ Convert 23 lbm.ft/min2 to its equivalent in kg.cm/s2.

Answer: 0.088 kg.cm/s2

18

9
 2/18/2020

2.4 Force and Weight

▪ According to Newton’s Second Law of Motion

𝐹 = 𝑚. 𝑎

▪ Natural force units are therefore:

▪ SI units: kg.m/s2
▪ cgs units: g.cm/s2
▪ American units: lbm.ft/s2
▪ Since these units are too long, Derived Force Units have been defined

19

2.4 Force and Weight

▪ SI Units: 1 Newton (1 N) = 1 kg.m/s2

▪ 1 N is the force required to accelerate a mass of 1 kg at the rate of 1 m/s2
▪ cgs Units: 1 dyne = 1 g.cm/s2
▪ 1 dyne is the force required to accelerate a mass of 1 g at the rate of 1 cm/s2
▪ American Engineering Units: pound force (lbf)
▪ 1 pound (lbf) is the force exerted by earth on an object that has a mass of 1 lbm
and falling at acceleration of gravity at sea level and 45o latitude (which is 32.174
ft/s2).
▪ 1 pound force (lbf) = 32.174 lbm.ft/s2

20

10
 2/18/2020

2.4 Force and Weight

Example:

▪ What is the force required to accelerate a mass of 4.0 lbm at a rate of

80.435 ft/s2. Express the force in the unit of lbf.

𝐹 = 𝑚. 𝑎
4.0 𝑙𝑏𝑚 80.435 𝑓𝑡
=
𝑠2
321.74 𝑙𝑏𝑚 . 𝑓𝑡
=
𝑠2
321.74 𝑙𝑏𝑚 . 𝑓𝑡 1𝑙𝑏𝑓
=
𝑠2 32.174 𝑙𝑏𝑚 𝑓𝑡Τ𝑠 2
= 10𝑙𝑏𝑓

21

2.4 Force and Weight

Mass vs Weight
▪ The weight of an object is the force exerted on the object by gravitational attraction.
Example: If an object has a mass of 25 kg then what is its weight?

𝑭= 𝒎×𝒂
or
𝑾= 𝒎×𝒈

25 kg 9.81 m 245.25 kg. m

𝑊= ቤ = = 245.25 N
s2 s2

22

11
 2/18/2020

2.4 Force and Weight

▪ The value of 𝑔 at sea level and 45o latitude in different units is:
▪𝑔 = 9.8066 m/s2
▪ = 980.66 cm/s2
▪ = 32.174 ft/s2

▪ If the value of 𝑔 is not specified, then you may assume the above values

23

2.4 Force and Weight

▪ Example 2.4-1
Water has a density of 62.4 lbm/ft3. How much does 2.000 ft3 of water weigh

▪ (1) at sea level and 45o latitude and

▪ (2) in Denver, Colorado, where the altitude is 5374 ft and the gravitational
acceleration is 32.139 ft/s2?

Answer: (1) 124.8 lbf

(2) 124.7 lbf

24

12
 2/18/2020

2.4 Force and Weight

▪ TEST YOURSELF (page 11)

1. What is a force of 2 kg m/s2 equivalent to in newtons? What is a force of 2 lbm ft/s2

equivalent to in lbf?
2. If the acceleration of gravity at a point is 𝑔 = 9.8 m/s2 and an object is resting on the
ground at this point, is this object accelerating at a rate of 9.8 m/s2?
3. Suppose an object weighs 9.8 N at sea level. What is its mass? Would its mass be
greater, less, or the same on the moon? How about its weight?
4. Suppose an object weighs 2 lbf at sea level. What is its mass? Would its mass be
greater, less, or the same at the center of the earth? How about its weight? (Careful!)

25

2.6 Dimensional Homogeneity and Dimensionless Quantities

▪ All quantities can be added and subtracted ONLY if

their units are the same.
▪ Dimensional Homogeneity in Equations:
Every valid equation must be dimensionally
homogeneous; that is all additive terms on both sides of
the equation must have the same dimensions.
𝑆 𝑚 = 𝑢 𝑚Τ𝑠 × 𝑡 𝑠 + ½ 𝑎 𝑚Τ𝑠 2 × 𝑡 2 (𝑠 2 )

26

13
 2/18/2020

2.6 Dimensional Homogeneity and Dimensionless Quantities

▪ In this equation,
𝑆 𝑚 = 𝑢 𝑚Τ𝑠 × 𝑡 𝑠 + ½ 𝑎 𝑚Τ𝑠 2 × 𝑡 2 (𝑠 2 )
▪ Each additive term has dimension of Length
▪ Hence, we say it is dimensionally homogenous
▪ Also, each additive term has a unit of meter (m)
▪ Hence, we say it is consistent in units

27

2.6 Dimensional Homogeneity and Dimensionless Quantities

▪ Consider the same equation but in different units,

𝑆(𝑓𝑡) = 𝑢 𝑚Τ𝑠 × 𝑡 𝑚𝑖𝑛 + ½ 𝑎 𝑐𝑚Τ𝑠 2 × 𝑡 2 (𝑠 2 )
▪ Each additive term has dimension of Length
▪ Hence, we say it is dimensionally homogenous
▪ But the additive terms have different units
▪ Hence, we say it is inconsistent in units
▪ We can fix such an equation by converting the units
28

14
 2/18/2020

2.6 Dimensional Homogeneity and Dimensionless Quantities

▪ Consider the same equation but in different units,

𝑆(𝑚) = 𝑢 𝑚Τ𝑠 + ½ 𝑎 𝑐𝑚Τ𝑠 2 × 𝑡 2 (𝑠 2 )
▪ Additive terms have different dimensions
▪ Hence, we say it is NOT dimensionally homogenous
▪ Such an equation cannot be valid
▪ For an equation to be valid it must be dimensionally
homogenous

29

2.6 Dimensional Homogeneity and Dimensionless Quantities

▪ A dimensionless quantity can be a pure number

(2, 1.5, 3/2 )

▪ or a multiplicative combination of variables with no net dimensions:

▪ Relative Values: e.g. relative densities: (the ratio of density of a substance

to a density of a standard)

▪ Dimensionless Group: e.g. Reynolds Number:

𝜌 𝑔Τ𝑐𝑚3 𝐷(𝑐𝑚)𝑣(𝑐𝑚Τ𝑠)
𝑅𝑒 =
𝜇 𝑔Τ𝑐𝑚. 𝑠

30

15
 2/18/2020

2.6 Dimensional Homogeneity and Dimensionless Quantities

▪ Exponents: e.g. 2 in 𝑦2

▪ Transcendental Functions: e.g. log, ln, exp or e, sin

▪ Arguments of Transcendental Functions: e.g. 𝑥 in log(𝑥) or sin (𝑥)

RIGHT WRONG

102 ft 102ft
log (10) log (10 kg)
sin(0.5) sin(0.5 N)

31

2.6 Dimensional Homogeneity and Dimensionless Quantities

▪ Example 2.6-1
▪ Consider the equation
𝐷 𝑓𝑡 = 3𝑡 𝑠 + 4
▪ If the equation is valid, what are the dimensions of the constants 3 and 4?
▪ If the equation is consistent in its units, what are the units of 3 and 4?
▪ Derive an equation for distance in meters and time in minutes.

32

16
 2/18/2020

2.6 Dimensional Homogeneity and Dimensionless Quantities

▪ For the equation to be valid, it must be dimensionally homogeneous, so

that each term must have the dimension of length. The constant 3 must
therefore have the dimension length/time , and 4 must have the
dimension length.

▪ For consistency, the constants must be 3 ft/s and 4 ft .

33

2.6 Dimensional Homogeneity and Dimensionless Quantities

▪ Define new variables D’(m) and t’(min). The equivalence relations

between the old and new variables are:

𝐷′ m 3.2808 ft
𝐷 ft = อ = 3.2808 𝐷′
1m

𝑡 ′ min 60 𝑠
𝑡 s = ቤ = 60 𝑡′
1 min

34

17
 2/18/2020

2.6 Dimensional Homogeneity and Dimensionless Quantities

▪ Substitute these expressions in the given equation

3.2808𝐷′ = 3 60𝑡′ + 4
▪ Or we can simplify it by dividing both sides by 3.2808:

𝐷′ 𝑚 = 54.86𝑡 ′ 𝑚𝑖𝑛 + 1.22

35

2.6 Dimensional Homogeneity and Dimensionless Quantities

▪ Example 2.6-2
▪ A quantity 𝑘 depends on the temperature 𝑇 in the following manner:
𝑚𝑜𝑙 5
20000
𝑘 = 1.2 × 10 𝑒𝑥𝑝 −
𝑐𝑚3 . 𝑠 1.987𝑇

▪ The units of the quantity 20000 are cal/mol and 𝑇 is in K (kelvin). What are
the units of 1.2x105 and 1.987?

36

18
 2/18/2020

2.6 Dimensional Homogeneity and Dimensionless Quantities

▪ Since the equation must be consistent in its units and exp is

dimensionless, 1.2×105 should have the same units as 𝑘, mol/(cm3.s).
Moreover, since the argument of exp must be dimensionless, we can write

20000 1 mol. K
=1
𝑇 (K) 1.987 cal

1 1
20000 = = = cal/mol
1 mol. K mol/cal
Kቚ cal

37

2.6 Dimensional Homogeneity and Dimensionless Quantities

▪ TEST YOURSELF (page 21)

1. What is a dimensionally homogeneous equation? If an equation is

dimensionally homogeneous, is it necessarily valid? If an equation is valid, is it
necessarily dimensionally homogeneous?
2. If 𝑦 𝑚/𝑠 2 = 𝑎 𝑧 𝑚3 , what are the units of 𝑎?
3. What is a dimensionless group? What multiplicative combination of 𝑟(𝑚), 𝑠(𝑚
/𝑠 2 ) and 𝑡(𝑠) would constitute a dimensionless group?
4. If 𝑧 𝑙𝑏𝑓 = 𝑎 sin(𝑄), what are the units of 𝑎 and 𝑄?

38

19
 2/18/2020

2.5 Numerical Calculation and Estimation

2.5a Scientific Notation, Significant Figures, and Precision

▪ In process calculations, we come across very large numbers as well as very

small numbers

▪ A convenient way to express these numbers is using the scientific notation

▪ In Scientific notation, a number is expressed as the product of another number
(usually between 0.1 and 10) and a power of 10.
▪ Examples:
123,000,000 = 1.23 × 108

0.000028 = 2.8 × 10−4

39

2.5 Numerical Calculation and Estimation

2.5a Scientific Notation, Significant Figures, and Precision

▪ Significant Figures of a number are the digits from the first nonzero digit on the
left of the number to either:

▪ the last digit (zero or nonzero) on the right if there is a decimal point
▪ (e.g. 86.030 or 8.6030 X 101 has 5 significant figures)

or

▪ the last nonzero digit of the number if there is no decimal point

▪ (e.g. 12,000 has 2 significant figures)

40

20
 2/18/2020

2.5 Numerical Calculation and Estimation

2.5a Scientific Notation, Significant Figures, and Precision

Number Scientific Notation Significant Figures

6.3528
34.020
0.005708
0.00007835
850950
2500
250.0
41

2.5 Numerical Calculation and Estimation

2.5a Scientific Notation, Significant Figures, and Precision

Number Scientific Notation Significant Figures

6.3528 6.3528×100 5
34.020 3.4020×101 5
0.005708 5.708×10-3 4
0.00007835 7.835×10-5 4
850950 8.5095×105 5
2500 2.5×103 2
250.0 2.500×102 4
42

21
 2/18/2020

2.5 Numerical Calculation and Estimation

2.5a Scientific Notation, Significant Figures, and Precision

▪ The rule for significant figures applies only to measured

quantities or numbers calculated from measured
quantities
▪ If a quantity is known precisely — like a pure integer (2)
or a counted rather than measured quantity (16
oranges)—its value implicitly contains an infinite
number of significant figures

43

2.5 Numerical Calculation and Estimation

2.5a Scientific Notation, Significant Figures, and Precision

▪ Accuracy is how close a measurement is to the correct value for

that measurement.
▪ For example, let us say that you are measuring the length of a
paper that is 11.0 inches long.
▪ You measure the length of the paper three times and obtain the
following measurements: 11.1 in., 11.2 in., and 10.9 in.
▪ These measurements are quite accurate because they are very
close to the correct value of 11.0 inches.
▪ In contrast, if you had obtained a measurement of 12 inches, your
measurement would not be very accurate.

44

22
 2/18/2020

2.5 Numerical Calculation and Estimation

2.5a Scientific Notation, Significant Figures, and Precision

▪ The precision of a measurement system refers to how

close the agreement is between repeated
measurements (which are repeated under the same
conditions).
▪ Consider the example of the paper measurements. The
precision of the measurements refers to the spread of
the measured values.

45

2.5 Numerical Calculation and Estimation

46

23
 2/18/2020

2.5 Numerical Calculation and Estimation

2.5a Scientific Notation, Significant Figures, and Precision

▪ The number of significant figures in a given value is an

indication of the precision with which the quantity is
known.
▪ The more significant figures, the more precise is the
value

47

2.5 Numerical Calculation and Estimation

2.5a Scientific Notation, Significant Figures, and Precision
28.3 mL

28 mL

30 mL

48

24
 2/18/2020

2.5 Numerical Calculation and Estimation

2.5a Scientific Notation, Significant Figures, and Precision

Mathematical Operation and Significant Figures

4.30 x 2.444 = 10.5, 10.51, 10.509, or 10.5092 ?

4.30 + 2.444 = 6.7, 6.74, 6.744, or 6.7440 ?

The Precision of the final answer cannot be greater than the least
precise measurement

49

2.5 Numerical Calculation and Estimation

2.5a Scientific Notation, Significant Figures, and Precision

Mathematical Operation and Significant Figures

Multiplication and Division:

4.30 x 2.444 = 10.5092

10.5

3 s.f 4 s.f 3 s.f

50

25
 2/18/2020

2.5 Numerical Calculation and Estimation

2.5a Scientific Notation, Significant Figures, and Precision

Mathematical Operation and Significant Figures

Multiplication and Division:

Example:

10.523 x10-3 x 11 x103 ÷ 2.50 = 46

51

2.5 Numerical Calculation and Estimation

2.5a Scientific Notation, Significant Figures, and Precision

Mathematical Operation and Significant Figures

Addition and Subtraction:

When two or more numbers are added or subtracted, the

positions of the last significant figures of each number relative
to the decimal point should be compared. Of these positions,
the one farthest to the left is the position of the last
permissible significant figure of the sum or difference.

52

26
 2/18/2020

2.5 Numerical Calculation and Estimation

2.5a Scientific Notation, Significant Figures, and Precision

Mathematical Operation and Significant Figures

Addition and Subtraction:

Example position of last

s.f.

1530 – 2.56 = 1527.44 1530

– 2.56

1530

position of last
s.f.

53

2.5 Numerical Calculation and Estimation

2.5a Scientific Notation, Significant Figures, and Precision

Mathematical Operation and Significant Figures

Addition and Subtraction:

Example position of last

s.f.

4.30 + 2.444 = 6.744 4.30

+ 2.444

6.74
position of last
s.f.

54

27
 2/18/2020

2.5 Numerical Calculation and Estimation

2.5a Scientific Notation, Significant Figures, and Precision

Mathematical Operation and Significant Figures

Addition and Subtraction:

Example

1.2 + 2.03 = 3.2

1230 + 2.03 = 1230

55

2.5 Numerical Calculation and Estimation

2.5a Scientific Notation, Significant Figures, and Precision

▪ A small area in a chemical plant which is 20.5 meters by 8.5

meters needs to be covered by waterproofing material. A tarp is
available, which is 5.50 meters by 10.00 meters. If the tarp is
laid over part of this area, how much of the area remains
uncovered?

▪ Keep in mind the significant figures

56

28
 2/18/2020

2.5 Numerical Calculation and Estimation

2.5a Scientific Notation, Significant Figures, and Precision

▪ TEST YOURSELF (page 13)

1. Express the following quantities in scientific notation and indicate how many
significant figures each has.
a) 12,200
b) 12,200.0
c) 0.003040
2. Express the following quantities in standard decimal form and indicate how many
significant figures each has.
a) 1.34×105
b) 1.340×10-2
c) 0.00420×106

57

2.5 Numerical Calculation and Estimation

2.5a Scientific Notation, Significant Figures, and Precision

▪ TEST YOURSELF (page 13)

3. How many significant figures would the solution of each of the following problems
have? What are the solutions of (c) and (d)?
a) (5.74)(38.27) / (0.001250)
b) 1.000 + 10.2
c) (1.76×104)(0.12×10-6)
d) 18.76 – 7
4. Round off each of the following numbers to three significant figures.
a) 1465
b) 13.35
c) 1.765×10-7

58

29
 2/18/2020

2.5 Numerical Calculation and Estimation

2.5a Scientific Notation, Significant Figures, and Precision

▪ TEST YOURSELF (page 13)

5. When the value of a number is given, the significant figures provide an

indication of the uncertainty in the value; for example, a value of 2.7 indicates
that the number lies between 2.65 and 2.75. Give ranges within which each of
the following values lie.
a) 4.3
b) 4.30
c) 2.778×10-3
d) 2500
e) 2.500×103

59

2.5 Numerical Calculation and Estimation

2.5b Validating Results

▪ The two most important questions to ask when solving

problems

1. How do I get a solution?

2. When I get one, how do I know it’s right?

60

30
 2/18/2020

2.5 Numerical Calculation and Estimation

2.5b Validating Results

▪ You can validate your answers in the following ways:

1. Back Substitution
2. Order-of-Magnitude Estimation
3. Test of Reasonableness

61

2.5 Numerical Calculation and Estimation

2.5b Validating Results

▪ Back substitution
▪ After you solve a set of equations, substitute your solution back into the equations and
make sure it works:
3𝑥 2 – 12 = 0
▪ I solve the equation and find the solution as ±2.
▪ I substitute this value in the equation and check:
3(±2)2 – 12
= 3 × 4 – 12
= 12 – 12
= 0
Hence my solution is right

62

31
 2/18/2020

2.5 Numerical Calculation and Estimation

2.5b Validating Results

▪ Order of magnitude estimation

▪ The procedure for checking an arithmetic calculation by order-of-
magnitude estimation is as follows:

1. Substitute simple integers for all numerical quantities, using

powers of 10 (scientific notation) for very small and very large
numbers.
27.36 → 20 or 30 (whatever makes calculation easier)
63472 → 6x104

63

2.5 Numerical Calculation and Estimation

2.5b Validating Results

2. Do the resulting arithmetic calculations by hand,

continuing to round off intermediate answers.

(36720)(0.0624) (4 × 104 )(5 × 10−2 )

= = 4 × 10(4−2+4) = 4 × 106
0.000478 5 × 10−4
3. If a number is added to a second, much smaller, number,
drop the second number in the approximation.

1 1 1
≈ ≈ = 0.25
4.13 + 0.04762 4.13 4

64

32
 2/18/2020

2.5 Numerical Calculation and Estimation

2.5b Validating Results

▪ Test of Reasonableness
▪ You calculate the diameter of a distillation column to be 2.3x105
meters Does this look right?

▪ You calculate the amount of chlorine required for treatment of

drinking water for a city to be 0.32 mL Does this make sense?

▪ You calculate the temperature of superheated steam to be

-10 oC Is this reasonable?

65

2.6 Dimensional Homogeneity and Dimensionless Quantities

▪ Example 2.5-1
▪ The calculation of a process stream volumetric flow rate has led to the
following formula:
254 13 1
𝑉ሶ = + ×
(0.879)(62.4) (0.866)(62.4) (31.3145)(60)

Estimate the answer without using a calculator.

250 10 1 5
𝑉ሶ ≈ + × ≈ ≈ 0.2 × 10−2 = 0.002
50 60 4 × 101 6 × 101 25 × 102

66

33
 2/18/2020

2.5 Numerical Calculation and Estimation

2.5c Estimation of Measured Values: Sample Mean

▪ Suppose we carry out a chemical reaction of the form A

→ Products, starting with pure A in the reactor and
keeping the reactor temperature constant at 45 oC.
▪ After two minutes we draw a sample from the reactor
and analyze it to determine X, the percentage of the A
fed that has reacted.

67

2.5 Numerical Calculation and Estimation

2.5c Estimation of Measured Values: Sample Mean

▪ In theory should have a unique value;

▪ However, in a real reactor X is a changing in an
unpredictable manner from one run to another at the
same experimental conditions.
▪ The values of X obtained after 10 successive runs might
be as follows:
Run 1 2 3 4 5 6 7 8 9 10
X (%) 67.1 73.1 69.6 67.4 71.0 68.2 69.4 68.2 68.7 70.2

68

34
 2/18/2020

2.5 Numerical Calculation and Estimation

2.5c Estimation of Measured Values: Sample Mean

▪ Why don’t we get the same value X of in each run? There are several
reasons.
▪ It is impossible to replicate experimental conditions exactly in
successive experiments. If the temperature in the reactor varies by as
little as 0.1 degree from one run to another, it could be enough to
change the measured value of X
▪ Even if conditions were identical in two runs, we could not possibly
draw our sample at exactly 2.000 minutes both times, and a difference
of a second could make a measurable difference in X
▪ Variations in sampling and chemical analysis procedures invariably
introduce scatter in measured values.

69

2.5 Numerical Calculation and Estimation

2.5c Estimation of Measured Values: Sample Mean

▪ What is the true value of X?

▪ Theoretically there exists a True Value for X.
▪ Practically, there is no means to determine it.
▪ The best we can do is estimate the true value.
▪ How can we estimate of the true value of X?
▪ The most common estimate is the “Sample Mean” (or the
“arithmetic mean”)

70

35
 2/18/2020

2.5 Numerical Calculation and Estimation

2.5c Estimation of Measured Values: Sample Mean

▪ Sample Mean:
𝑁
1 1
𝑋ത = 𝑋1 + 𝑋2 + 𝑋3 + ⋯ + 𝑋𝑁 = ෍ 𝑋𝑗
𝑁 𝑁
𝑗=1

▪ For the given data:

Run 1 2 3 4 5 6 7 8 9 10
X (%) 67.1 73.1 69.6 67.4 71.0 68.2 69.4 68.2 68.7 70.2

1
𝑋ത = 67.1 + 73.1 + 69.6 + ⋯ + 70.2 = 69.3%
10

71

2.5 Numerical Calculation and Estimation

2.5c Estimation of Measured Values: Sample Mean

▪ Graphically we can represent this as:

80

75

70
Xmean
65

60
1 3 5 7 9
Run

72

36
 2/18/2020

2.5 Numerical Calculation and Estimation

2.5c Estimation of Measured Values: Sample Mean

▪ The more measurements of a random variable, the

better the estimated value based on the sample mean.
▪ However, even with a huge number of measurements
the sample mean is at best an approximation of the
true value.

73

2.5 Numerical Calculation and Estimation

2.5c Estimation of Measured Values: Sample Mean

▪ TEST YOURSELF (page 16)

▪ The weekly production rates of a pharmaceutical product over the past six weeks
have been 37, 17, 39, 40, 40, and 40 batches per week.
1. Think of several possible explanations for the observed variation in the weekly
production rate.
2. If you used the sample mean of the given data as a basis, what would you
predict the next weekly production rate to be?
3. Come up with a better prediction, and explain your reasoning.

74

37
 2/18/2020

2.5 Numerical Calculation and Estimation

2.5c Estimation of Measured Values: Sample Mean

▪ How scattered is the data?

75

2.5 Numerical Calculation and Estimation

2.5c Estimation of Measured Values: Sample Mean

▪ Range:
𝑅 = 𝑋𝑚𝑎𝑥 − 𝑋𝑚𝑖𝑛

▪ For Data set (a), 𝑅 = 72 – 68 = 4

▪ For Data set (b), 𝑅 = 95 – 52 = 43
▪ The range is the crudest measure of scatter: it involves only two of
the measured values and gives no indication of whether or not
most of the values cluster close to the mean or scatter widely
around it.

76

38
 2/18/2020

2.5 Numerical Calculation and Estimation

2.5c Estimation of Measured Values: Sample Mean

▪ Sample Variance:
▪ The sample variance is a much better measure of the scatter of data:

1
𝑆𝑋2 = 𝑋1 − 𝑋ത 2 + 𝑋2 − 𝑋ത 2 + ⋯ + 𝑋𝑁 − 𝑋ത 2
𝑁−1

▪ Sample Standard Deviation:

𝑆𝑋 = 𝑆𝑋2

77

2.5 Numerical Calculation and Estimation

2.5c Estimation of Measured Values: Sample Mean

▪ The smaller the Standard Deviation for a given set of data

the less scattered the data
▪ The smaller the Standard Deviation, the more reliable is the
set of data
▪ For typical random variables, roughly two-thirds of all
measured values fall within one standard deviation of the
mean;
▪ about 95% fall within two standard deviations;
▪ and about 99% fall within three standard deviations.

78

39
 2/18/2020

2.5 Numerical Calculation and Estimation

2.5c Estimation of Measured Values: Sample Mean

𝑋ത − 3𝑆𝑋 𝑋ത − 𝑆𝑋 𝑋ത 𝑋ത + 𝑆𝑋 𝑋ത + 3𝑆𝑋
𝑋ത − 2𝑆𝑋 𝑋ത + 2𝑆𝑋

79

2.5 Numerical Calculation and Estimation

2.5c Estimation of Measured Values: Sample Mean

▪ TEST YOURSELF (page 18)

▪ The volumetric flow rate of a process fluid, 𝑉ሶ (cm3/s), is measured five times,
with the following results: Measurement 1 2 3 4 5
𝑽ሶ (cm3/s) 232 248 227 241 239
ത range, sample variance (𝑆𝑉2 ), and sample
1. Calculate the sample mean (𝑉),
standard deviation (𝑆𝑉 ).
2. There is a high probability (above 90%) that a measured value of 𝑉ሶ will fall
ሶ the form
within two standard deviations of the mean. Report the value of 𝑉in
ሶ𝑉 = 𝑎 ± 𝑏, choosing the values of 𝑎 and 𝑏 to define this range.

80

40
 2/18/2020

2.7 Process Data Representation and Analysis

▪ The operation of any chemical process is ultimately

based on the measurement of process variables like
temperatures, pressures, flow rates, concentrations,
and so on.
▪ It is sometimes possible to measure these variables
directly, but more commonly indirect techniques are
used.

81

2.7 Process Data Representation and Analysis

▪ Suppose, for example, that you wish to measure the

concentration, 𝐶, of a solute in a solution.
▪ To do so, you normally measure a quantity 𝑋, such as a
thermal or electrical conductivity, a light absorbance, or the
volume of a titer, that varies in a known manner with 𝐶,
and then calculate 𝐶 from the measured value of 𝑋.
▪ The relationship between 𝐶 and 𝑋 is determined in a
separate calibration experiment in which solutions of
known concentration are prepared and is measured for
each solution.

82

41
 2/18/2020

2.7 Process Data Representation and Analysis

▪ Concentration of a solution is measured by using a light

absorbance technique
▪ Calibration experiment gives the following data:
Calibration Experiment Data
30
Concentration
Absorption
mol/L 25

Concentration (mol/L)
3 0.106 20

9 0.295 15

15 0.512 10

5
20 0.720
0
28 0.880 0 0.2 0.4 0.6 0.8 1
Absorption

83

2.7 Process Data Representation and Analysis

▪ What if we need concentration for an absorption of

0.20?
▪ What if we need concentration for an absorption of
0.95?
▪ Interpolation: Estimation of values at points between
the tabulated data
▪ Extrapolation: Estimation of values at points that lie
outside the range of the tabulated data

84

42
 2/18/2020

2.7 Process Data Representation and Analysis

▪ A number of interpolation and extrapolation methods

are commonly used, including two-point linear
interpolation, graphical interpolation, and curve fitting.
▪ Which one is most appropriate?
▪ This depends on the nature of the relationship between
the variables under consideration.

85

2.7 Process Data Representation and Analysis

2.7a Two-Point Linear Interpolation

▪ Two Point Linear Interpolation:

▪ Slope of straight line is same between any two points
▪ Slope = 𝑦 / 𝑥

𝑦2 − 𝑦1 𝑦 − 𝑦1
=
𝑥2 − 𝑥1 𝑥 − 𝑥1

𝑥 − 𝑥1
𝑦 = 𝑦1 + (𝑦 − 𝑦1 )
𝑥2 − 𝑥1 2

86

43
 2/18/2020

2.7 Process Data Representation and Analysis

2.7a Two-Point Linear Interpolation

▪ What if we need concentration for an absorption of 0.20?

𝒚 𝒙
𝑥 − 𝑥1
Concentration
Absorption 𝑦 = 𝑦1 + (𝑦 − 𝑦1 )
mol/L 𝑥2 − 𝑥1 2
𝒚𝟏 3 𝒙𝟏 0.106
𝒚 =? ? ← ← 𝒙 = 𝟎. 𝟐𝟎
𝒚𝟐 9 𝒙𝟐 0.295 0.2 − 0.106
15 0.512 𝑦=3+ 9−3
0.295 − 0.106
20 0.720
28 0.880
= 5.984 𝑚𝑜𝑙/𝐿
Dependent Independent
Variable Variable

87

2.7 Process Data Representation and Analysis

2.7a Two-Point Linear Interpolation

▪ TEST YOURSELF (page 22)

1. Values of a variable (𝑓) are measured at several times (𝑡):
𝑓 1 4 8
𝑡 1 2 3

Show that if two-point linear interpolation is used:

(a)𝑓(𝑡 = 1.3) ≈ 1.9
(b) 𝑡(𝑓 = 5) ≈ 2.25

88

44
 2/18/2020

2.7 Process Data Representation and Analysis

2.7a Two-Point Linear Interpolation

▪ TEST YOURSELF (page 22)

1. If a function 𝑦(𝑥) appears as shown in each of the diagrams shown below,
would two-point linear interpolation yield estimates of 𝑦 that are too high, too
low, or correct? If the two-point linear interpolation formula were used to
estimate 𝑦 𝑥3 from the tabulated values of (𝑥1 , 𝑦1 ) and (𝑥2 , 𝑦2 ) in plot (b),
would the estimated value be too high or too low?

89

2.7 Process Data Representation and Analysis

2.7b Fitting a Straight Line

▪ Fitting a straight line:

▪ The equation for a straight line is:
𝑦 = 𝑎𝑥 + 𝑏
𝑎 is the slope of the line 𝑏 𝑎

𝑏 is the intercept of the line

90

45
 2/18/2020

2.7 Process Data Representation and Analysis

2.7b Fitting a Straight Line

▪ Fitting a straight line:

▪ The equation for a straight line is:
𝑦 = 𝑎𝑥 + 𝑏 Concentration
Absorption
(mol/L)
Fit the given data to a straight line
3 0.106
9 0.295
∆𝑦 𝑦2 − 𝑦1 28 − 3 15 0.512
𝑎= = = = 32.3
∆𝑥 𝑥2 − 𝑥1 0.88 − 0.106 20 0.720
28 0.880
𝑏 = 𝑦 − 𝑎𝑥 = 3 − 32.3 0.106 = −0.4238 𝑪 = 𝟑𝟐. 𝟑𝑨 − 𝟎. 𝟒𝟐𝟑𝟖
𝑏 = 𝑦 − 𝑎𝑥 = 28 − 32.3 0.880 = −0.4238

91

2.7 Process Data Representation and Analysis

2.7b Fitting a Straight Line

▪ Fitting a straight line:

▪ We can now use this equation
𝐶 = 32.3 𝐴 − 0.4238 Concentration
Absorption
(mol/L)
to determine values of 𝐶 for any given 3 0.106
Absorption value. 9 0.295
e.g. Find 𝐶 when 𝐴 = 0.6 15 0.512
20 0.720
𝐶 = 32.3 (0.6) – 0.4238 28 0.880
= 18.96 mol/L 𝑪 = 𝟑𝟐. 𝟑𝑨 − 𝟎. 𝟒𝟐𝟑𝟖

92

46
 2/18/2020

2.7 Process Data Representation and Analysis

2.7b Fitting a Straight Line
▪ Fitting non-linear data:
If the data looks like this:

▪ I want to find a value of 𝑦 using a value of 𝑥 that is not in the Data Table
Can I use a fitted linear equation?
Can I use linear interpolation or extrapolation?

93

2.7 Process Data Representation and Analysis

2.7c Fitting Nonlinear Data

▪ Fitting a nonlinear equation to data is usually much harder than fitting a line
▪ However, with some nonlinear equations you can still use straight-line fitting if
you plot the data in a suitable manner.
▪ Suppose, for example, that 𝑥 and 𝑦 are related by the equation 𝑦 2 = 𝑎𝑥 3 + 𝑏.
▪ A plot of measured 𝑦 versus 𝑥 data would clearly be curved
▪ However, a plot of 𝑦2 versus 𝑥 3 would be a straight line with a slope 𝑎 and
intercept 𝑏.

94

47
 2/18/2020

2.7 Process Data Representation and Analysis

2.7c Fitting Nonlinear Data
▪ Here are some examples of plots that yield straight lines:
𝒚 = 𝒂𝒙𝟐 + 𝒃 → plot 𝒚 𝐯𝐬 𝒙𝟐
𝒂 𝟏
𝒚𝟐 = + 𝒃 → plot 𝒚𝟐 𝐯𝐬
𝒙 𝒙
𝟏 𝟏
=𝒂 𝒙+𝟑 +𝒃 → plot 𝐯𝐬 (𝒙 + 𝟑)
𝒚 𝒚
𝐬𝐢𝐧 𝒚 = 𝒂(𝒙𝟐 − 𝟒) → plot 𝐬𝐢𝐧 𝒚 𝐯𝐬 (𝒙𝟐 − 𝟒)

𝟏 𝟏
𝒚= → plot 𝐯𝐬 𝒙
𝑪𝟏 𝒙 − 𝑪𝟐 𝒚

𝒚−𝟏 𝟐
𝟏Τ𝟐
𝒚 = 𝟏 + 𝒙 𝒎𝒙𝟐 + 𝒏 → plot 𝐯𝐬 𝒙𝟐
𝒙𝟐

95

2.7 Process Data Representation and Analysis

2.7c Fitting Nonlinear Data

▪ Example 2.7-2
▪ A mass flow rate 𝑚ሶ (g/s) is measured as a function of temperature T (oC)
T 10 20 40 80
𝑚ሶ 14.76 20.14 27.73 38.47

▪ There is reason to believe that 𝑚ሶ varies linearly with the square root of T :
𝑚ሶ = 𝑎𝑇 0.5 + 𝑏
▪ Use a straight-line plot to verify this formula and determine 𝒂 and 𝒃

96

48
 2/18/2020

2.7 Process Data Representation and Analysis

2.7c Fitting Nonlinear Data

▪ We find the values of T 0.5 from the given data

T 10 20 40 80
T 0.5 3.162 4.472 6.325 8.944
𝑚ሶ 14.76 20.14 27.73 38.47

▪ We now plot 𝑚ሶ vs T 0.5 which should be a straight line, with a

slope = 𝒂 and an intercept = 𝒃

97

2.7 Process Data Representation and Analysis

2.7c Fitting Nonlinear Data

40

30

20

10

0
0 2 4 6 8 10

▪ We now plot 𝑚ሶ vs T 0.5 which should be a straight line, with a

slope = 𝒂 and an intercept = 𝒃

98

49
 2/18/2020

2.7 Process Data Representation and Analysis

2.7c Fitting Nonlinear Data

T 10 20 40 80
T 0.5 3.162 4.472 6.325 8.944
𝑚ሶ 14.76 20.14 27.73 38.47

▪ To find a and b, we can use the first and last points

38.47 − 14.76 23.71
𝑎= = = 4.1
8.944 − 3.162 5.782

𝑏 = 𝑚ሶ − 𝑎𝑇 0.5 = 14.76 − 4.1 3.162 = 1.8

▪ Verification: At last point, 𝑚ሶ = 4.1 800.5 + 1.8 = 38.47

99

2.7 Process Data Representation and Analysis

2.7c Fitting Nonlinear Data

▪ TEST YOURSELF (page 26)

1. The following plot is generated from experimental (𝑥, 𝑦) data:

What equation would you use to relate 𝑥 and 𝑦?

100

50
 2/18/2020

2.7 Process Data Representation and Analysis

2.7c Fitting Nonlinear Data

▪ TEST YOURSELF (page 26)

2. How would you plot (𝑥, 𝑦) data to get a straight line, and how would you
determine 𝑎 and 𝑏 for each of the following functions?
a) y = 𝑎 𝑥 + 𝑏
b) 1/y = 𝑎 𝑥 − 3 3 + 𝑏
c) y = 𝑎𝑥 2 − 𝑏 1/3
d) sin (y) = 𝑥 𝑎𝑥 + 𝑏 −2
e) y = 𝑎𝑒 𝑏𝑥
f) y = 𝑎𝑥 𝑏

101

2.7 Process Data Representation and Analysis

2.7d Logarithmic Coordinates

▪ Suppose you wish to fit an exponential function 𝑦 = 𝑎𝑒 𝑏𝑥 to measured

(𝑥, 𝑦) data.
▪ The determination of 𝑎 and 𝑏 is simplified by plotting ln 𝑦 as a function of
𝑥.
▪ There are two plotting procedures you might use:
1. Calculate ln 𝑦 for each value of 𝑦, and plot ln 𝑦 versus 𝑥 on
rectangular axes,
2. Plot 𝑦 directly on a logarithmic scale versus 𝑥 on a rectangular scale.

102

51
 2/18/2020

2.7 Process Data Representation and Analysis

2.7d Logarithmic Coordinates

▪ Logarithmic Coordinates:
Parallel axis

No need to
calculate ln(y)
Plot directly
the y values

103

2.7 Process Data Representation and Analysis

2.7d Logarithmic Coordinates

▪ Semi-log plot
▪ For a function of the form: 𝑦 = 𝑎𝑒 𝑏𝑥
1200 10000

1000
1000
800

600 100

400
10
200

0 1
0 1 2 3 4 5 6 7 8 0 1 2 3 4 5 6 7 8

Rectangular Plot Semi-log Plot

104

52
 2/18/2020

2.7 Process Data Representation and Analysis

2.7d Logarithmic Coordinates

▪ Example

▪ A plot of 𝑦 versus 𝑥 yields a straight line on a semilog plot

(the vertical axis is a logarithmic scale while the horizontal
axis is a rectangular scale). The line passes through
(8.68,0.01) and (2,1). Sketch the plot and calculate the
equation 𝑦(𝑥).

105

2.7 Process Data Representation and Analysis

2.7d Logarithmic Coordinates

10

0.1

0.01
2 4 6 8 10

106

53
 2/18/2020

2.7 Process Data Representation and Analysis

2.7d Logarithmic Coordinates
A plot of 𝑦 versus 𝑥 yields a straight line on a semilog plot (the vertical axis is a logarithmic scale while the horizontal axis is a
rectangular scale). The line passes through (8.68,0.01) and (2,1). Sketch the plot and calculate the equation for 𝑦(𝑥).

ln 𝑦 = ln 𝑎 + 𝑏𝑥

ln 1 − ln 0.01 4.608
𝑏= = = −0.69
2 − 8.68 −6.68

ln 𝑎 = ln 𝑦 − 𝑏𝑥 = ln 1 + 0.69 2 = 1.38
𝑎 = 𝑒 1.38 = 3.97

𝑦 = 3.97𝑒 −0.69𝑥

107

2.7 Process Data Representation and Analysis

2.7d Logarithmic Coordinates

▪ If one axis is logarithmic and the other is rectangular then it is called a

semi-log plot

▪ If both axis are logarithmic, then it is called a log-log or simply a log plot

▪ If 𝑦 versus 𝑥 data appear linear on a log plot, then ln 𝑦 versus ln 𝑥 would

be linear on a rectangular plot, and the data can therefore be correlated
by a power law:
𝑦 = 𝑎𝑥 𝑏

108

54
 2/18/2020

2.7 Process Data Representation and Analysis

2.7d Logarithmic Coordinates

▪ Log Paper:

109

2.7 Process Data Representation and Analysis

2.7c Fitting Nonlinear Data

▪ TEST YOURSELF (page 29)

1. The following plots yield straight lines. What are the equations that
relate the variables?
a) 𝑃 versus t on rectangular coordinates.
b) 𝑃 (logarithmic axis) versus t on a semilog paper.
c) 𝑃 versus t on rectangular coordinates."
d) 𝑦 2 − 3 (logarithmic axis) versus 1/𝑥 2 on a semlog paper.
e) 1/𝐹 versus 𝑡 2 − 4 on a log paper.

110

55
 2/18/2020

2.7 Process Data Representation and Analysis

2.7c Fitting Nonlinear Data

▪ TEST YOURSELF (page 29)

2. What would you plot against what on what kind of axes to get a straight line for
the following relationships (a and b are constants)? 𝑃 versus t on rectangular
coordinates.
a) 𝑃 = 𝑎𝑒 𝑏𝑡
b) 𝑃 = 𝑎𝑡 𝑏
c) 𝑃2 = exp 𝑎𝑡 3 + 𝑏
d) 1/𝑃 = 𝑎 𝑡 − 4 −𝑏

111

2.7 Process Data Representation and Analysis

2.7d Logarithmic Coordinates

▪ Fitting a Line to Scattered Data

Easy to fit a straight line to

such data

most common technique = least squares method

scattered data
More realistic data

Many lines can be drawn, so

which one to use?

112

56