Introduction and Classical Optimization Techniques

Unit I

Mr.S.Ganapathi
Assistant Professor
Department of Mechanical Engineering
Aditya University
 Course Outcomes

• CO1: State and formulate the optimization problem, without and with
constraints, by using design variables from an engineering design problem
• Syllabus: Introduction and Classical Optimization Techniques: Statement
of an Optimization problem, design vector, design constraints, constraint
surface, objective function, objective function surfaces, classification of
Optimization problems.
• Classical Optimization Techniques: Single variable Optimization, multi
variable Optimization without constraints, necessary and sufficient
conditions for minimum/maximum, multivariable Optimization with
equality constraints. Solution by method of Lagrange multipliers,
multivariable Optimization with inequality constraints, Kuhn – Tucker
conditions.
26/05/2025 Optimization Techniques S.Ganapathi, Asst.Professor, Dept.of ME
2
 • Optimization is the act of obtaining the best result under given
circumstances. In design, construction, and maintenance of any
engineering system, engineers have to take many technological and
managerial decisions at several stages.
• The ultimate goal of all such decisions is either to minimize the effort
required or to maximize the desired benefit. Since the effort required
or the benefit desired in any practical situation can be expressed as a
function of certain decision variables, optimization can be defined as
the process of finding the conditions that give the maximum or
minimum value of a function.

26/05/2025 Optimization Techniques S.Ganapathi, Asst.Professor, Dept.of ME 3

 • There is no single method available for solving all optimization problems efficiently. Hence a
number of optimization methods have been developed for solving different types of optimization
problems. The optimum seeking methods are also known as mathematical programming
techniques and are generally studied as a part of operations research.
• Operations research is a branch of mathematics concerned with the application of scientific
methods and techniques to decision making problems and with establishing the best or optimal
solutions.
• The beginnings of the subject of operations research can be traced to the early period of World
War II. During the war, the British military faced the problem of allocating very scarce and limited
resources (such as fighter airplanes, radars, and submarines) to several activities (deployment to
numerous targets and destinations). Because there were no systematic methods available to
solve resource allocation problems, the military called upon a team of mathematicians to develop
methods for solving the problem in a scientific manner. The methods devel opted by the team
were instrumental in the winning of the Air Battle by Britain. These methods, such as linear
programming, which were developed as a result of research on (military) operations,
subsequently became known as the methods of operations research.
26/05/2025 Optimization Techniques S.Ganapathi, Asst.Professor, Dept.of ME
4
 • STATEMENT OF AN OPTIMIZATION PROBLEM

26/05/2025 Optimization Techniques S.Ganapathi, Asst.Professor, Dept.of ME 5

 • Design Vector
Any engineering system or component is defined by a set of quantities some of which are
viewed as variables during the design process. In general, certain quantities are usually fixed at the
outset and these are called preassigned parameters. All the other quantities are treated as variables
in the design process and are called design or decision variables xi, i=1,2,...,n. The design variables
are collectively represented as a design vector X={x1,x2,...,xn}.
Design Constraints
In many practical problems, the design variables cannot be chosen arbitrarily; rather, they
have to satisfy certain specified functional and other requirements. The restrictions that must be
satisfied to produce an acceptable design are collectively called design constraints. Constraints that
represent limitations on the behavior or performance of the system are termed behavior or
functional constraints. Constraints that represent physical limitations on design variables, such as
availability, fabricability, and trans portability, are known as geometric or side constraints.

26/05/2025 Optimization Techniques S.Ganapathi, Asst.Professor, Dept.of ME

6
 • Constraint Surface For illustration, consider an optimization problem with only inequality constraints gj(X) ≤ 0. The
set of values of X that satisfy the equation gj(X) = 0 forms a hyper surface in the design space and is called a
constraint surface. Note that this is an (n −1)-dimensional subspace, where n is the number of design variables. The
constraint surface divides the design space into two regions: one in which gj(X) < 0 and the other in which gj(X)>0.
Thus the points lying on the hypersurface will satisfy the constraint gj(X) critically, whereas the points lying in the
region where gj(X)>0 are infeasible or unacceptable, and the points lying in the region where gj(X) < 0 are feasible or
acceptable. The collection of all the constraint surfaces gj(X) = 0, j = 1,2,...,m, which separates the acceptable region
is called the composite constraint surface.
• Figure shows a hypothetical two-dimensional design space where the infeasible region is indicated by hatched lines.
A design point that lies on one or more than one constraint surface is called a bound point, and the associated
constraint is called an active constraint. Design points that do not lie on any constraint surface are known as free
points. Depending on whether a particular design point belongs to the acceptable or unacceptable region, it can be
identified as one of the following four types:
• 1. Free and acceptable point
• 2. Free and unacceptable point
• 3. Bound and acceptable point
• 4. Bound and unacceptable point

26/05/2025 Optimization Techniques S.Ganapathi, Asst.Professor, Dept.of ME

7
26/05/2025 Optimization Techniques S.Ganapathi, Asst.Professor, Dept.of ME
8
 • Objective Function
The conventional design procedures aim at finding an acceptable or adequate
design that merely satisfies the functional and other requirements of the problem. In
general, there will be more than one acceptable design, and the purpose of optimization
is to choose the best one of the many acceptable designs available. Thus a criterion has
to be chosen for comparing the different alternative acceptable designs and for
selecting the best one. The criterion with respect to which the design is optimized,
when expressed as a function of the design variables, is known as the criterion or
merit or objective function.
In some situations, there may be more than one criterion to be satisfied simultaneously.
For example, a gear pair may have to be designed for minimum weight and maximum
efficiency while transmitting a specified horsepower. An optimization problem involving
multiple objective functions is known as a Multi-objective programming problem.

26/05/2025 Optimization Techniques S.Ganapathi, Asst.Professor, Dept.of ME

9
26/05/2025 Optimization Techniques S.Ganapathi, Asst.Professor, Dept.of ME
10
 • CLASSIFICATION OF OPTIMIZATION PROBLEMS
Optimization problems can be classified in several ways, as described below
• Classification Based on the Existence of Constraints
As indicated earlier, any optimization problem can be classified as constrained or
unconstrained ,depending on whether constraints exist in the problem.
• Classification Based on the Nature of the Design Variables
In the first category, the problem is to find values to a set of design parameters that make some prescribed
function of these parameters minimum subject to certain constraints. Such problems are called parameter or
static optimization problems.
In the second category of problems, the objective is to find a set of design parameters, which are all
continuous functions of some other parameter, that minimizes an objective function subject to a set of
constraints. This type of problem, where each design variable is a function of one or more parameters, is known
as a trajectory or dynamic optimization problem.
• Classification Based on the Physical Structure of the Problem
Depending on the physical structure of the problem, optimization problems can be classified as optimal
control and nonoptimal control problems.
26/05/2025 Optimization Techniques S.Ganapathi, Asst.Professor, Dept.of ME
11
 • Classification Based on the Nature of the Equations Involved
optimization problems can be classified as linear, nonlinear,
geometric, and quadratic programming problems. This classification is
extremely useful from the com putational point of view since there are
many special methods available for the efficient solution of a particular
class of problems.
• Classification Based on the Permissible Values of the Design Variables
Depending on the values permitted for the design variables,
optimization problems can be classified as integer and real-valued
programming problems.

26/05/2025 Optimization Techniques S.Ganapathi, Asst.Professor, Dept.of ME

12
 • Classification Based on the Deterministic Nature of the Variables
Based on the deterministic nature of the variables involved, optimization
problems can be classified as deterministic and stochastic programming problems.
• Classification Based on the Separability of the Functions
Optimization problems can be classified as separable and non-separable
programming problems based on the separability of the objective and constraint
functions.
• Classification Based on the Number of Objective Functions
Depending on the number of objective functions to be minimized,
optimization problems can be classified as single-and multi-objective
programming problems.

26/05/2025 Optimization Techniques S.Ganapathi, Asst.Professor, Dept.of ME

13
 • SINGLE-VARIABLE OPTIMIZATION
Asingle-variable optimization problem is one in which the value of x = x∗ is
to be found in the interval [a,b] such that x∗ minimizes f(x). The following two
theorems provide the necessary and sufficient conditions for the relative minimum
of a function of a single variable
(i)Necessary Condition If a function f(x) is defined in the interval a ≤ x ≤b and has a
relative minimum at x = x∗, where a < x∗ < b, and if the derivative df(x)/dx = f′(x)
exists as a finite number at x = x∗, then f′(x∗) = 0.

26/05/2025 Optimization Techniques S.Ganapathi, Asst.Professor, Dept.of ME

14
 • Determine the maximum and minimum values of the function

26/05/2025 Optimization Techniques S.Ganapathi, Asst.Professor, Dept.of ME

15
26/05/2025 Optimization Techniques S.Ganapathi, Asst.Professor, Dept.of ME
16
 • MULTI VARIABLE OPTIMIZATION WITH NO CONSTRAINTS
In this section we consider the necessary and sufficient conditions for the minimum or
maximum of an unconstrained function of several variables.

Theorem 2.4 Sufficient Condition A sufficient condition for a stationary point X∗ to be an

extreme point is that the matrix of second partial derivatives (Hessian matrix) of f(X)
evaluated at X∗ is
(i) positive definite when X∗ is a relative minimum point, and
(ii) negative definite when X∗ is a relative maximum point.
26/05/2025 Optimization Techniques S.Ganapathi, Asst.Professor, Dept.of ME
17
26/05/2025 Optimization Techniques S.Ganapathi, Asst.Professor, Dept.of ME
18
26/05/2025 Optimization Techniques S.Ganapathi, Asst.Professor, Dept.of ME
19
26/05/2025 Optimization Techniques S.Ganapathi, Asst.Professor, Dept.of ME
20
26/05/2025 Optimization Techniques S.Ganapathi, Asst.Professor, Dept.of ME
21
 • MULTIVARIABLE OPTIMIZATION WITH EQUALITY CONSTRAINTS
• In this section we consider the optimization of continuous functions
subjected to equality constraints:
Here m is less than or equal to n;
otherwise (if m>n), the problem
becomes over defined and, in
general, there will be no solution.
There are several methods
available for the solution of this
problem. The methods of direct
substitution, constrained variation,
and Lagrange multipliers.

26/05/2025 Optimization Techniques S.Ganapathi, Asst.Professor, Dept.of ME 22

 • Solution by Direct Substitution
• For a problem with n variables and m equality constraints, it is
theoretically possible to solve simultaneously the m equality
constraints and express any set of m variables in terms of the
remaining n − m variables. When these expressions are substituted
into the original objective function, there results a new objective
function involving only n −m variables. The new objective function is
not subjected to any constraint, and hence its optimum can be found
by using the unconstrained optimization techniques

26/05/2025 Optimization Techniques S.Ganapathi, Asst.Professor, Dept.of ME

23
 • Solution by the Method of Lagrange Multipliers
• The basic features of the Lagrange multiplier method is given initially for a simple problem of two
variables with one constraint.

26/05/2025 Optimization Techniques S.Ganapathi, Asst.Professor, Dept.of ME

24
26/05/2025 Optimization Techniques S.Ganapathi, Asst.Professor, Dept.of ME
25
26/05/2025 Optimization Techniques S.Ganapathi, Asst.Professor, Dept.of ME
26
26/05/2025 Optimization Techniques S.Ganapathi, Asst.Professor, Dept.of ME
27
26/05/2025 Optimization Techniques S.Ganapathi, Asst.Professor, Dept.of ME
28
26/05/2025 Optimization Techniques S.Ganapathi, Asst.Professor, Dept.of ME
29
26/05/2025 Optimization Techniques S.Ganapathi, Asst.Professor, Dept.of ME
30
26/05/2025 Optimization Techniques S.Ganapathi, Asst.Professor, Dept.of ME
31
26/05/2025 Optimization Techniques S.Ganapathi, Asst.Professor, Dept.of ME
32
26/05/2025 Optimization Techniques S.Ganapathi, Asst.Professor, Dept.of ME
33
26/05/2025 Optimization Techniques S.Ganapathi, Asst.Professor, Dept.of ME
34
26/05/2025 Optimization Techniques S.Ganapathi, Asst.Professor, Dept.of ME
35
 • Example 2.14 A manufacturing firm producing small refrigerators has
entered into a contract to supply 50 refrigerators at the end of the
first month, 50 at the end of the second month, and 50 at the end of
the third. The cost of producing x refrigerators in any month is given
by $(x2 +1000). The firm can produce more refrigerators in any month
and carry them to a subsequent month. However, it costs $20 per unit
for any refrigerator carried over from one month to the next.
Assuming that there is no initial inventory, determine the number of
refrigerators to be produced in each month to minimize the total cost.

26/05/2025 Optimization Techniques S.Ganapathi, Asst.Professor, Dept.of ME

36
26/05/2025 Optimization Techniques S.Ganapathi, Asst.Professor, Dept.of ME
37
26/05/2025 Optimization Techniques S.Ganapathi, Asst.Professor, Dept.of ME
38
26/05/2025 Optimization Techniques S.Ganapathi, Asst.Professor, Dept.of ME
39
26/05/2025 Optimization Techniques S.Ganapathi, Asst.Professor, Dept.of ME
40
26/05/2025 Optimization Techniques S.Ganapathi, Asst.Professor, Dept.of ME
41
 Thank You

26/05/2025 Optimization Techniques S.Ganapathi, Asst.Professor, Dept.of ME 42