Logspace Sequential Quadratic Programming for Design

Optimization

Cody J. Karcher ∗
Massachusetts Institute of Technology, Cambridge, MA, 02139

A novel approach to exploiting the log-convex structure present in many design problems is
arXiv:2105.14441v1 [cs.CE] 30 May 2021

developed by modifying the classical Sequential Quadratic Programming (SQP) algorithm. The

modified algorithm, Logspace Sequential Quadratic Programming (LSQP), inherits some of the

computational efficiency exhibited by log-convex methods such as Geometric Programing and

Signomial Programing, but retains the the natural integration of black box analysis methods

from SQP. As a result, significant computational savings is achieved without the need to

invasively modify existing black box analysis methods prevalent in practical design problems.

In the cases considered here, the LSQP algorithm shows a 40-70% reduction in number of

iterations compared to SQP.

I. Introduction and Motivation

A Geometric Program (GP) is a specific type of non-linear optimization problem that becomes convex upon a

transformation to logspace∗ [1]. GPs have received a great deal of attention in many industries including chemical

engineering[2], environment quality control[3], digital circuit design[4], analog and RF circuit design[5–7], transformer

design[8], communication systems [9–11], biotechnology[12, 13], epidemiology[14], optimal gas flow[15], and tree-

water-network control[16] (compilation from Agrawal[17]), but the most recent surge has occurred in the field of aircraft

design[18–29].

As convex programs GPs can be solved rapidly and with guaranteed convergence to a global optimum, but the GP

formulation is narrow, only allowing an objective and constraints which take a specific form. Signomial Programming

(SP), also referred to as Generalized Geometric Programming (GGP) in the literature, offers a natural extension to the

GP formulation and enables a much wider variety of models to be included in the objective and constraints[1]. The

price of this added flexibility is the loss of convexity and Signomial Programs must therefore be solved by a series of

convex GP approximations. In practice, Signomial Programs are often an order of magnitude more expensive to solve

than similar Geometric Programs.

Despite the enhanced modeling flexibility, the Signomial Programming formulation often remains too limiting for
∗ Graduate Student, Department of Aeronautics and Astronautics, [email protected], AIAA Member
∗ In some of the literature, the transformation considered in this work is referred to as a log-log transformation since both dependent and
independent variables are transformed. In all cases here, logspace, log-convexity, log transformation etcetera could equivalently be called log-log
space, log-log convexity, log-log transformation and similar.
designers. Hall[29] specifically notes that both GP and SP formulations require all models to be explicitly written as

constraints in the optimization formulation, which effectively eliminates the use of black box analysis models that are

prevalent in practical design problems[30]. Thus, committing to a GP or SP approach to design significantly increases

the modeling effort required by the designer and drastically limits the practicality of these methods.

Due to the existence of these black boxes, Sequential Quadratic Programming (SQP) has become one of the more

popular algorithms for aircraft design applications[31, 32]. With SQP, the objective and each constraint are approximated

using only a function evaluation and the gradient vector at a series of candidate points. For black boxes constructed to

return gradients (as has become standard practice), the integration with SQP is quite natural.

This work builds on these two fundamental pillars. First, efforts in Geometric and Signomial Programming have

revealed an exploitable underlying structure in engineering design, but these formulations do not conform to the existing

modeling approach. Furthermore, modifying existing models would require an investment of time and money that

would likely be prohibitive. Second, the SQP algorithm conforms with existing modeling approaches but ignores known

structure in the optimization formulation, causing valuable computational effort to be wasted in the form of unnecessary

iterations. The modification to SQP proposed in this work, Logspace Sequential Quadratic Programming (LSQP),

bridges this divide to exploit underlying design space structure while remaining practical to designers who wish to

continue utilizing their existing methods and practices.

II. Foundations in Established Optimization Methods

A. Sequential Quadratic Programming

Consider the general non-linear constrained optimization problem:

minimize 𝑓 (x)
x

subject to g𝑖 (x) ≤ 0, 𝑖 = 1, . . . , 𝑁 (1)

h 𝑗 (x) = 0, 𝑗 = 1, . . . , 𝑀

The Sequential Quadratic Programming (SQP) algorithm solves this general non-linear program (NLP) by iteratively

considering candidate solutions x 𝑘 and solving the Quadratic Programming (QP) sub-problem:

1
minimize 𝑓 (x 𝑘 ) + ∇ 𝑓 (x 𝑘 )𝑇 d + d𝑇 ∇2 L (x 𝑘 )d
d 2
subject to 𝑔𝑖 (x 𝑘 ) + ∇𝑔𝑖 (x 𝑘 )𝑇 d ≤ 0, 𝑖 = 1, . . . , 𝑁
(2)
ℎ 𝑗 (x 𝑘 ) + ∇ℎ 𝑗 (x 𝑘 )𝑇 d = 0, 𝑗 = 1, . . . , 𝑀

d = x − x𝑘

2
at each iteration 𝑘 [33–35] until some convergence criteria is reached. The key assumption of SQP is that the QP

sub-problem (Equation 2) is an accurate local approximation of the original problem (Equation 1). Note that the use of

the Hessian of the Lagrangian ∇2 L has been shown to be superior to directly using the Hessian of the objective function

∇2 𝑓 [33].

The sub-problem is constructed by approximating the objective as a second order Taylor Series about the candidate

solution x 𝑘 and each constraint as a first order Taylor Series about the same x 𝑘 . This process results in a convex QP

sub-problem that can be readily solved for a global optimum d∗ , the direction vector that points from current point x 𝑘 to

the next point x 𝑘+1 . Note that this sub-problem can be fully constructed given only the function evaluations 𝑓 (x 𝑘 ),

𝑔𝑖 (x 𝑘 ), and ℎ 𝑗 (x 𝑘 ) and the gradients ∇ 𝑓 (x 𝑘 ), ∇𝑔𝑖 (x 𝑘 ), and ∇ℎ 𝑗 (x 𝑘 ), which makes apparent the value of this method

when black box analyses are present.

B. Geometric Programming

Geometric Programs are built upon two fundamental building blocks: monomial and posynomial functions. A

monomial function is defined as the product of a leading constant with each variable raised to a real power[1]:

𝑁
Ö
𝑚(x) = 𝑐𝑥1 𝑎1 𝑥2 𝑎2 ... 𝑥 𝑛 𝑎𝑛 = 𝑐 𝑥𝑖𝑎𝑖 (3)
𝑖=1

A posynomial is simply the sum of monomials[1], which can be defined in notation as:

𝐾
∑︁ 𝑁
Ö
𝑝(x) = 𝑚 1 (x) + 𝑚 2 (x) + ... + 𝑚 𝑛 (x) = 𝑐𝑘 𝑥𝑖𝑎𝑖𝑘 (4)
𝑘=1 𝑖=1

From these two building blocks, it is possible to construct the definition of a GP in standard form[1]:

minimize 𝑝 0 (x)
x

subject to 𝑚 𝑖 (x) = 1, 𝑖 = 1, . . . , 𝑁 (5)

𝑝 𝑗 (x) ≤ 1, 𝑗 = 1, . . . , 𝑀

When constraints and objectives can be written in the form specified in Equation 5 it is said that the problem is GP

compatible.

In general, the formulation defined by Equation 5 is a non-linear and non-convex optimization problem, making it

extremely difficult to solve[1], however a GP can be transformed into a convex optimization problem by undergoing a

logarithmic transformation. As convex programs GPs can be solved by a wide variety of algorithms, but most are now

solved using primal/dual methods[1] and solvers such as CVXOPT[36] are readily available and return reliable results

under most circumstances.

3
C. Signomial Programming

Signomal Programs (SPs) are a logical extension of Geometric Programs that enable the inclusion of negative leading

constants and a broader set of equality constraints. Of interest here is that SPs are not convex upon transformation to

logspace unlike their GP counterparts, but still benefit from an underlying structure which is well approximated by a

log-convex formulation. This property is what is referred to here has having a high degree of log-convexity.

The key building blocks of the Signomial Programing are signomials, which are the difference between two

posynomials 𝑝(x) and 𝑛(x):

𝐾
∑︁ 𝑁
Ö 𝑃
∑︁ 𝑁
Ö
𝑔
𝑠(x) = 𝑝(x) − 𝑛(x) = 𝑐𝑘 𝑥𝑖𝑎𝑖𝑘 − 𝑑𝑝 𝑥 𝑖 𝑖𝑘 (6)
𝑘=1 𝑖=1 𝑝=1 𝑖=1

where posynomial 𝑝(x) represents the convex portion of the signomial and ‘neginomial’ 𝑛(x) is the concave portion.

With this definition, it is now possible to write the standard form for a Signomial Program [21]:

𝑝 0 (x)
minimize
x 𝑛0 (x)
subject to 𝑠𝑖 (x) = 0, 𝑖 = 1, . . . , 𝑁 (7)

𝑠 𝑗 (x) ≤ 0, 𝑗 = 1, . . . , 𝑀

however, another useful form is:

𝑝 0 (x)
minimize
x 𝑛0 (x)
𝑝 𝑖 (x)
subject to = 1, 𝑖 = 1, . . . , 𝑁 (8)
𝑛𝑖 (x)
𝑝 𝑗 (x)
≤ 1, 𝑗 = 1, . . . , 𝑀
𝑛 𝑗 (x)

In this alternative form, the neginomial is added to both sides, and then used as a divisor to construct an expression

either equal to or constrained by a value of one.

The Difference of Convex Algorithm (DCA) is used to solve SPs. In this method, the neginomials are replaced by

their monomial approximation[1]:

𝑁  𝑎
Ö 𝑥𝑖 𝑖
¯
𝑛(x)| x 𝑘 = 𝑛(x 𝑘 )
𝑥 𝑘𝑖
𝑖=1 (9)
𝑥 𝑘𝑖 𝜕𝑛
𝑎𝑖 =
𝑛(x 𝑘 ) 𝜕𝑥 𝑖

4
thus yielding a GP at each iteration:

𝑝 0 (x)
minimize
x 𝑛¯ 0 (x)
𝑝 𝑖 (x)
subject to = 1, 𝑖 = 1, . . . , 𝑁 (10)
𝑛¯ 𝑖 (x)
𝑝 𝑗 (x)
≤ 1, 𝑗 = 1, . . . , 𝑀
𝑛¯ 𝑗 (x)

It has been shown in many test cases[21–27] that extremely complex Signomial Programs can be constructed which

exhibit a high degree of log-convexity and can therefore be solved in very few iterations. For example, Kirschen[21]

proposes a full aircraft design problem with 824 variables that solves in only six iterations. Similarly, the design case

presented by York[27] has 628 variables and also solves in only six iterations.

III. Building to Logspace Sequential Quadratic Programming

A. The Transformed Formulation of Logspace Sequential Quadratic Programming

Consider now a slight adjustment to Equation 1:

minimize 𝑓 (x)
x

subject to g𝑖 (x) ≤ 1, 𝑖 = 1, . . . , 𝑁 (11)

h 𝑗 (x) = 1, 𝑗 = 1, . . . , 𝑀

reformulated so that a value of 1 appears on the right hand side of the constraints.

From Equation 11, the problem can now be transformed in a similar fashion to that of a Geometric Program [1]:

minimize log 𝑓 (𝑒 y )
y

subject to log g𝑖 (𝑒 y ) ≤ 0, 𝑖 = 1, . . . , 𝑁 (12)

log h 𝑗 (𝑒 y ) = 0, 𝑗 = 1, . . . , 𝑀

If foreknowledge of log-convexity is assumed, Equation 12 is expected to be well approximated as a log-convex

optimization problem based on the similar properties of geometric and signomial programs. For example, in the case

where the original NLP is a geometric program, monomial constraints will be exactly represented as lines in logspace,

while posynomials be represented as log-sum-exp functions which are known to be convex for positive leading constants.

Consider then that rather than implementing the SQP algorithm on the original problem (Equation 1), that the same

SQP algorithm can be implemented on this transformed problem (Equation 12). This is Logspace Sequential Quadratic

Programming (LSQP).

5
B. Expressions for the Gradients of the LSQP Functions

LSQP can be implemented in two ways. First, one can directly construct the formulation in Equation 12 and

then implement any existing SQP method, an approach demonstrated by Kirschen[37] in a limited form. The log

transformation is essentially treated as a pre-processing step, and a final post-processing step can transform the variables

y𝑖 back into the original variables x𝑖 . However, this approach is only available if the objective and constraints are

known explicitly to the solver and violates the desire to utilize the same function evaluations 𝑓 (x 𝑘 ), 𝑔𝑖 (x 𝑘 ), ℎ 𝑗 (x 𝑘 ) and

gradients ∇ 𝑓 (x 𝑘 ), ∇𝑔𝑖 (x 𝑘 ), ∇ℎ 𝑗 (x 𝑘 ) that are generally expected. Using calculus, it is possible to derive expressions for

the gradients of the transformed functions as functions of the original gradients, thereby allowing black box analyses to

be readily implemented just like traditional SQP.

For the sake of simplicity, take the log transformed formulation (Equation 12) in the following condensed form:

minimize 𝐹 (𝑒 y )
y

subject to 𝐺 𝑖 (𝑒 y ) ≤ 0, 𝑖 = 1, . . . , 𝑁 (13)

𝐻 𝑗 (𝑒 y ) = 0, 𝑗 = 1, . . . , 𝑀

with capital letters representing the corresponding transformed functions. The relevant derivatives can now be readily

obtained from Boyd[5]:

𝜕𝐹 𝑥𝑖 𝜕 𝑓
= (14)
𝜕𝑦 𝑖 𝑓 (x) 𝜕𝑥 𝑖
𝑥𝑖
Thus, the gradients of the original functions can simply be multiplied by the value of 𝑓 (x) (or similar for g𝑖 (x) and

h 𝑗 (x)) and used in the LSQP modification.

IV. Formal Definition of LSQP

A. Problem and Subproblem Definition

With this information, a formal definition of the Logspace Sequential Quadratic Program can be constructed. The

LSQP approach solves the general non-linear constrained optimization problem:

minimize 𝑓 (x)
x

subject to g𝑖 (x) ≤ 1, 𝑖 = 1, . . . , 𝑁 (15)

h 𝑗 (x) = 1, 𝑗 = 1, . . . , 𝑀

6
by solving a series of approximate sub-problems† :

1 1
minimize log 𝑓 (x 𝑘 ) + (x 𝑘 ∇ 𝑓 (x 𝑘 ))𝑇 d + d𝑇 ∇2 L (y 𝑘 )d
d 𝑓 (x 𝑘 ) 2
1
subject to log 𝑔𝑖 (x 𝑘 ) + (x 𝑘 ∇𝑔𝑖 (x 𝑘 ))𝑇 d ≤ 0, 𝑖 = 1, . . . , 𝑁
𝑔𝑖 (x 𝑘 )
1
𝑇 (16)
log ℎ 𝑗 (x 𝑘 ) + x 𝑘 ∇ℎ 𝑗 (x 𝑘 ) d = 0, 𝑗 = 1, . . . , 𝑀
ℎ 𝑗 (x 𝑘 )

d = y − log x 𝑘

y = log x

Note that the term 12 d𝑇 ∇2 L (y 𝑘 )d has been left as a function of transformed variables y 𝑘 , since this matrix is rarely ever

explicitly computed and is instead constructed via an approximation method such as BFGS[34]. However it is possible

to derive this expression as a function of the second derivatives of the original objective and constraints. This exercise

was omitted for brevity.

B. How to Construct Standard From

Typically, standard form in constrained optimization is defined by Equation 1, with constraints being bound by a

value of zero. Constructing the typical standard form is generally a simple matter of subtracting the necessary terms

until zero is attained on the correct side of the constraint. However, the standard form for LSQP (Equation 11) is not as

simple to construct.

By far the most accurate sub-problem is obtained if standard form is derived via the same steps used in constructing

the standard form of Signomial Programs: separate the concave contributions of the constraints and divide to obtain a

unity constraint. This method results in a constraint that is naturally scaled and has a higher likelihood of being well

approximated in the sub-problem.

Unless the constraint is explicitly written, as is the case with Signomial Programing, such separation may not be

possible and an alternative approach is needed. A typical constraint has two sides which are approximately equal in

magnitude. Dividing the two sides of the constraint to construct an appropriate fraction and using the correct operator to

constrain this fraction with a value of one will result in an equivalent constraint in LSQP standard form, though the lack

of concave separation makes it inferior to the first approach.

At worst, it is always possible to construct the typical standard form (bound by zero) and then add 1 to both sides,

though this approach gives no consideration to underlying structure and will likely not be well scaled.

In the work presented here, constraints which are GP compatible are written in GP standard form. However,

signomial constraints are constructed by first writing 𝑠𝑖 (x) = 0 (or similar) and then adding one to both sides, yielding
† The symbol represents the Hadamard Product, or element-wise product, of two matrices

7
𝑠𝑖 (x) + 1 = 1. This approach was taken to present the hardest challenge for the proposed LSQP algorithm, though future

efforts could be improved by taking one of the first two approaches described here.

C. Non-Positive Values

In cases where a variable 𝑥𝑖 is not strictly positive, an additive shift can be implemented to create the transformed

variable 𝑥 𝑖,𝑠𝑢𝑏 that is then substituted in place of the original variable:

𝑥 𝑖,𝑠𝑢𝑏 = 𝑥𝑖 + 𝐾 (17)

where 𝐾 is some constant that ensures 𝑥 𝑖,𝑠𝑢𝑏 is strictly positive. The constraints can be transformed as:

g𝑖 (x) + 𝐾
≤1 (18)
1+𝐾

and similar, where the left hand side becomes the new function implemented in LSQP.

Excessive use of these transformations will likely change underlying structure and it is therefore prudent to limit

their use. In general, problems with many negative variables or constraints are likely to be more efficiently solved with

traditional SQP, but this remains an area for further exploration not fully considered in this work.

V. The LSQP Algorithm

At a high level, the LSQP algorithm presented here is simply an SQP which instead utilizes the gradients presented in

Section III.B to construct the QP sub-problem and approximate the Hessian via a BFGS update. As a result, the existing

SQP literature is heavily utilized in constructing the LSQP algorithm. The proposed modified algorithm (Algorithm

1) is a combination of methods proposed by Nocedal and Wright[34], Kraft[35], and the Matlab documentation[38].

These sources seem to form the core of the some of the more commonly used algorithms, such as those implemented in

Matlab and Python’s Scipy package. Note the use of condensed notation in the algorithm definition (see Equation 13).

The following sections outline relevant implementation details. For brevity, the elements of the algorithm shared

with traditional SQP will not be revisited, but citations have been provided in Algorithm 1 that provide a detailed

discussion of these shared elements.

A. Implications of the Log Transformation

A handful of specific failure modes have been introduced due to the log transformation. First, the function evaluations

of the objectives and all constraints must be strictly positive at the initial guess x0 , and at each subsequent iteration x 𝑘 .

Second, while iteration points x 𝑘 need not be feasible, the conditions 𝑓 (x 𝑘 ) > 0, 𝑔𝑖 (x 𝑘 ) > 0, and ℎ 𝑗 (x 𝑘 ) > 0 must all

hold true at the initial condition and at each subsequent x 𝑘 .

8
 One potential solution to both of these issues would be to modify the step size computation (Line 12 of Al-

gorithm 1) to ensure that any step taken results in a new x 𝑘+1 that has strictly positive values and returns strictly

positive values for all 𝑓 (x 𝑘 ), 𝑔𝑖 (x 𝑘 ), and ℎ 𝑗 (x 𝑘 ), though this would require additional evaluations of the constraints at

each iteration and may add excessive computational expense. As a result, such measures have not been implemented here.

Algorithm 1 Logspace Sequential Quadratic Programming (LSQP)

1: Given x0
2: Construct standard form (if necessary, see Section IV.B above)
3: Compute y0 = log(x0 )
4: Initialize logspace Lagrange multipliers, 𝜇0 ← 1
5: Initialize the matrix B ← I the approximation of ∇2 L (𝐹, 𝐺, 𝐻, y, 𝜇) [34]
6: Compute 𝑓 (𝑥0 ), 𝑔𝑖 (𝑥 0 ), and ℎ 𝑗 (𝑥0 )
7: Compute 𝐹 (𝑦 0 ), 𝐺 𝑖 (𝑦 0 ), and 𝐻 𝑗 (𝑦 0 ) via log of the appropriate scalar
8: Compute ∇ 𝑓 (𝑥0 ), ∇𝑔𝑖 (𝑥0 ), and ∇ℎ 𝑗 (𝑥0 )
9: Compute ∇𝐹 (𝑦 0 ), ∇𝐺 𝑖 (𝑦 0 ), and ∇𝐻 𝑗 (𝑦 0 ) via Equation 14
10: for 𝑘 = 0 to maxIter do
11: Solve the QP sub-problem to obtain 𝑑 𝑦 and 𝑑 𝜇 [34]
12: Compute the step size 𝛼 𝑘 via inexact line search [34, 38]
13: y 𝑘+1 ← y 𝑘 + 𝛼 𝑘 𝑑 𝑦
14: x 𝑘+1 ← exp(y 𝑘+1 )
15: 𝜇 𝑘+1 ← 𝜇 𝑘 + 𝛼 𝑘 𝑑 𝜇
16: Compute 𝑓 (𝑥 𝑘+1 ), 𝑔𝑖 (𝑥 𝑘+1 ), and ℎ 𝑗 (𝑥 𝑘+1 )
17: Compute 𝐹 (𝑦 𝑘+1 ), 𝐺 𝑖 (𝑦 𝑘+1 ), and 𝐻 𝑗 (𝑦 𝑘+1 ) via log of the appropriate scalar
18: Compute ∇ 𝑓 (𝑥 𝑘+1 ), ∇𝑔𝑖 (𝑥 𝑘+1 ), and ∇ℎ 𝑗 (𝑥 𝑘+1 )
19: Compute ∇𝐹 (𝑦 𝑘+1 ), ∇𝐺 𝑖 (𝑦 𝑘+1 ), and ∇𝐻 𝑗 (𝑦 𝑘+1 ) via Equation 14
20: Compute ∇L 𝑘 (𝐹, 𝐺, 𝐻, y 𝑘 , 𝜇 𝑘+1 ) [34]
21: Compute ∇L 𝑘+1 (𝐹, 𝐺, 𝐻, y 𝑘+1 , 𝜇 𝑘+1 ) [34]
22: if ∇L 𝑘+1 (𝐹, 𝐺, 𝐻, y 𝑘+1 , 𝜇 𝑘+1 ) < 𝜀 𝐺𝐿 [34] then
23: return 𝑥 𝑘+1
24: else if ||𝑑 𝑥 || < 𝜀 𝑑 𝑥 [38] then
25: return 𝑥 𝑘+1
26: else
27: Perform a damped BFGS update on matrix B [34]
28: 𝑘 ← 𝑘 +1
29: end if
30: end for
31: return 𝑥 𝑘 , maximum iteration count reached

9
B. Generating and Solving the QP Sub-Problem

Solving the QP sub-problem proposed in Equation 16 often fails in practice due to inconsistent constraints. This

method instead solves the sub-problem:

+𝑀
𝑁∑︁
1 1
minimize log 𝑓 (x 𝑘 ) + (x 𝑘 ∇ 𝑓 (x 𝑘 ))𝑇 d + d𝑇 ∇2 L (y 𝑘 )d + 𝐾 𝜎𝑖2
d 𝑓 (x 𝑘 ) 2 𝑖=1
1
subject to log 𝑔𝑖 (x 𝑘 ) + (x 𝑘 ∇𝑔𝑖 (x 𝑘 ))𝑇 d ≤ 𝜎𝑖 , 𝑖 = 1, . . . , 𝑁
𝑔𝑖 (x 𝑘 )
1
𝑇 (19)
log ℎ 𝑗 (x 𝑘 ) + x 𝑘 ∇ℎ 𝑗 (x 𝑘 ) d = 𝜎 𝑗+𝑁 , 𝑗 = 1, . . . , 𝑀
ℎ 𝑗 (x 𝑘 )

d = y − log x 𝑘

y = log x

where 𝐾 is some arbitrarily large constant. If all constraints are consistent, the QP optimizer will push all 𝜎𝑖 = 0,

thus yielding an equivalent sub-problem to Equation 16. A post processing check is implemented to ensure all 𝜎𝑖 are

sufficiently close to zero at the final iteration.

C. Trust Bounding in Areas with Large Hessian Uncertainty

For the first few iterations (ie 𝑘 = 1, 2, 3), the curvature matrix ∇2 L (y 𝑘 ) is an exceedingly poor approximation of

the actual curvature in the search space. Thus, a limit in these early iterations has been imposed according to:

𝑑 𝑥𝑖
≤ 𝑏𝑘 (20)
𝑥𝑖

where 𝑏 𝑘 follows the schedule [0.2, 0.5, 1.0] for 𝑘 = 0, 1, 2 and becomes 𝑏 𝑘 = ∞ for 𝑘 ≥ 3. While not necessary for

successful convergence, this bounding in early iterations was observed to improve both SQP and LSQP solutions for the

problems considered here.

Note that at each of the iterations 𝑘, each variable bounds the step size independently, rendering relevant only the

variable with the most relative change. This simple trust bounding allows the approximation of the curvature matrix to

increase in quality before attempting to move too rapidly to a poor candidate solution. This trust bounding is the only

area in which this approach knowingly deviated substantially from the Matlab SQP algorithm used for comparison.

VI. Comparison of the Performance of the SQP and LSQP Algorithms

A. Methodology

Three algorithms were implemented to study the potential benefits of the LSQP modification: a Python implementation

of LSQP (as described in Section V), a similar Python implementation of SQP (implemented to be identical to the

10
LSQP algorithm without the log transformation), and the Matlab implementation of SQP (fmincon with the ‘SQP’

flag). Comparing the Python SQP and the Matlab SQP provides evidence that the Python implementation is consistent

with the performance of a well established algorithm. If the Python SQP performs within acceptable tolerance of the

Matlab algorithm, it is sufficient foundation to compare the Python SQP to the Python LSQP and draw conclusions as

to the performance differences. The SQP algorithm in Python’s Scipy package was also considered for comparison

(an implementation of the algorithm presented by Kraft[35]), but lack of control on the termination condition meant a

meaningful comparison was not possible.

Considering these three algorithms yields a total of 24 cases: 4 problems (Boyd, Rosenbrock, Floudas, Kirschen-

Ozturk), 2 guess categories (Good and Poor, where initial guess values were bound to +/- 10% and +/- 80% from the

known optimum respectively), and 3 algorithms (LSQP, SQP, Matlab SQP).

For each of the 24 cases, 1000 trials were conducted from a random initial starting point. For a given test problem

and guess category, the random starting points were kept the same across the three algorithms so that no algorithm

benefited from a random cluster of favorable guesses.

Due to the computational expense of 24000 trials, the computational resources of the MIT SuperCloud were

utilized[39], and a limit of 500 iterations was placed on all 3 algorithms.

Note that none of these cases contains a true black boxed analysis model in order to allow for comparison between

known optima and the computed result, however the objective and constraints are treated by the algorithm (Algorithm 1)

as black boxes regardless of actual form and therefore any equivalent black box could be equivalently substituted in

without affecting the optimizer in any way.

B. Results from the Boyd Geometric Program

The first test case is a toy problem proposed by Boyd as an example of a simple Geometric Program[1]. While this

is not a complicated problem, it is a design case in that it seeks maximize the volume of some box given constraints on

11
the surface areas and aspect ratios of various faces. The problem is given here in LSQP standard form:

minimize 1/(ℎ𝑤𝑑)
ℎ,𝑤 ,𝑑
ℎ𝑤 ℎ𝑑
subject to 2 +2 ≤1
𝐴 𝑤 𝑎𝑙𝑙 𝐴 𝑤 𝑎𝑙𝑙
𝑤𝑑
≤1
𝐴 𝑓 𝑙𝑜𝑜𝑟
𝛼𝑤 (21)
≤1
ℎ
ℎ
≤1
𝛽𝑤
𝛾𝑤
≤1
𝑑
𝑑
≤1
𝛿𝑤

Solving this problem with the LSQP modification requires significantly fewer iterations than traditional SQP. Figure

1 shows that all of the LSQP trials converged within 10 iterations, while the best SQP case converges in the same number

of iterations less than 60% of the time. Table 1 provides a more detailed report on the average number of iterations for

each case over all 1000 trials.

1.0 1.0
Probability of Convergence

Probability of Convergence

0.8 0.8
0.6 0.6
0.4 0.4
0.2 Matlab SQP 0.2 Matlab SQP
SQP SQP
0.0 LSQP 0.0 LSQP
0 10 20 30 40 50 60 0 10 20 30 40 50 60
Number of Iterations Number of Iterations
(a) A Good Initial Guess (b) A Poor Initial Guess

Fig. 1 Probability of Convergence vs. Iteration Count for the Boyd Problem

Table 1 Comparison of the Number of Iterations Required to Solve the Boyd Geometric Program

Algorithm Average Number of Iterations (% change)

Good Guess Poor Guess
LSQP 4.65 (-72.67%) 6.60 (-69.50%)
SQP 17.00 (-) 21.64 (-)
Matlab SQP 10.72 (-36.95%) 18.43 (-14.86%)

The Boyd Geometric Program demonstrates a clear win for the LSQP modification: the number of required iterations

12
is decreased by approximately 70% at no additional computational cost or sacrifice to solution quality. This benefit

is unsurprising due to the high number of monomials which become exactly represented under the transformation to

logspace, and the high degree of log-convexity exhibited by all geometric programs. However, subsequent cases will

become more complex and represent greater challenges to the LSQP algorithm.

C. Results from the Constrained Rosenbrock Problem

The Rosenbrock function is a common test case for optimization methods. It is unique in that the optimum resides

in an extremely shallow local valley, making it an excellent test problem for gradient based methods. Consider the

following constrained problem:

minimize (1 − 𝑥) 2 + 100(𝑦 − 𝑥 2 ) 2 + 1
𝑥,𝑦

subject to (𝑥 − 1) 3 − 𝑦 + 2 ≤ 1

𝑥+𝑦−1 ≤ 1 (22)
𝑥
≤1
1.5
𝑦
≤1
2.5
Note that a constant of 1 has been added to the objective to shift up the optimum and enable the log transformation of

LSQP. The variables are also bound to be greater than a small positive constant to aid the construction of sub-problems.

Rosenbrock’s problem is not a GP or SP as formulated, but can be reformulated by setting an intermediate variable 𝑧

equal to the objective. However, an attempt to solve the problem in this modified form with the Difference of Convex

Algorithm does not succeed. Fortunately, the optimum is known to be 𝑓 (1, 1) = 1.

Unlike Boyd’s Geometric Program, the Rosenbrock problem is a clear win for traditional SQP. Figure 2 shows good

agreement between the Matlab and Python SQP, but a 50-60% increase in iteration count for the LSQP modification.

1.0 1.0
Probability of Convergence

Probability of Convergence

0.8 0.8
0.6 0.6
0.4 0.4
0.2 Matlab SQP 0.2 Matlab SQP
SQP SQP
0.0 LSQP 0.0 LSQP
0.0 2.5 5.0 7.5 10.0 12.5 15.0 17.5 20.0 0.0 2.5 5.0 7.5 10.0 12.5 15.0 17.5 20.0
Number of Iterations Number of Iterations
(a) A Good Initial Guess (b) A Poor Initial Guess

Fig. 2 Probability of Convergence vs. Iteration Count for the Rosenbrock Problem

13
 In the case of good initial guesses, all three algorithms reach 100% success. However, for poor initial guess values,

all three algorithms have some percentage of cases which converge instead to the local optimum at 𝑓 (0, 0) = 2, which is

why Figure 2(b) shows ultimate success probabilities between 0.7 and 0.85.

So why does the traditional SQP outperform the LSQP modification? As was mentioned in Section II.C, just because

the problem can be constructed as a Signomial Program does not imply underlying log-convexity, and the Rosenbrock

Problem exhibits almost no underlying log-convexity.

This can be seen by visualizing the original Rosenbrock Problem and the Rosenbrock Problem under log transformation

directly in Figures 3 and 4.

(a) Rosenbrock Objective Function (b) Quadratic Approximation

Fig. 3 A quadratic approximation of the Rosenbrock objective function in untransformed space. Approxima-
tion referenced about the global optimum indicated by a red dot.

(a) Log Transformation (b) Quadratic Approximation

Fig. 4 A quadratic approximation of the Rosenbrock objective function in log transformed space. Approxima-
tion referenced about the global optimum indicated by a red dot. Note the large white area in (b) represents a
region where the value of the function exceeds the contour range.

The traditional SQP algorithm takes QP approximations in the space of Figure 3(a), resulting in a QP objective

14
function that appears like Figure 3(b). Compare this visually to Figure 4, which represents the LSQP modification. In

order to capture the local region around the reference point, the quadratic approximation in Figure 4(b) creates a deep

narrow valley that clearly is a poor approximation of the objective globally, as indicated by the large white area where the

contour range is exceeded. In other words, there is generally far more agreement between the original objective function

Figure 3(a) and its quadratic approximations Figure 3(b) than there is between the transformed objective Figure 4(a) and

its quadratic approximations Figure 4(b). Since the traditional SQP algorithm has superior sub-problem representations,

it is no surprise that it outperforms the LSQP modification in this case.

D. Results from the Floudas Heat Exchanger Problem

Floudas[40] provides the following example for the design of a heat exchanger:

minimize 𝑥 1 + 𝑥2 + 𝑥 3
𝑥1 ,..., 𝑥8
833.33252𝑥4 100 83333.333
subject to + − ≤1
𝑥2 𝑥6 𝑥6 𝑥1 𝑥6
1250𝑥5 𝑥4 1250𝑥4
+ − ≤1
𝑥2 𝑥7 𝑥7 𝑥2 𝑥7
1250000 𝑥5 2500𝑥5 (23)
+ − ≤1
𝑥3 𝑥8 𝑥8 𝑥3 𝑥8
0.0025𝑥 4 + 0.0025𝑥6 ≤ 1

− 0.0025𝑥 4 + 0.0025𝑥5 + 0.0025𝑥7 ≤ 1

− 0.01𝑥5 + 0.01𝑥 8 ≤ 1

The problem has 5 signomial constraints, and only a single GP-compatible posynomial (the 4th constraint). In contrast

with the Rosenbrock Problem, which has signomials but little underlying log-convexity, the Floudas Problem does

indeed exhibit some degree of log-convexity. This underlying structure is apparent in the results presented in Figure 5(a)

as the LSQP modification achieves 100% success before reaching 20 iterations, while the SQP algorithms take twice as

many iterations to achieve the same success rate.

15
 1.0
Probability of Convergence 1.0

Probability of Convergence
0.8 0.8
0.6 0.6
0.4 0.4
0.2 Matlab SQP 0.2 Matlab SQP
SQP SQP
0.0 LSQP 0.0 LSQP
0 20 40 60 80 100 0 20 40 60 80 100
Number of Iterations Number of Iterations
(a) A Good Initial Guess (b) A Poor Initial Guess

Fig. 5 Probability of Convergence vs. Iteration Count for the Floudas Problem

Figure 5(b) is a more nuanced result. For the majority of trials where LSQP does converge, it converges in fewer

iterations. However 145 (14.5%) of the LSQP trials failed. These failures are a result of one or more of the constraints

violating the conditions 𝑔𝑖 (x 𝑘 ) > 0 and ℎ 𝑗 (x 𝑘 ) > 0, thus causing the log transformation to fail at one of the iterations.

A proposal to correct these failures in future work was proposed in Section V.A, but here that modification was not

implemented specifically to demonstrate this failure mode and can be corrected in future implementations.

Tables 2 and 3 provide a more detailed insight into the averages over all trials for the cases with good and poor

initial guesses. It is noteworthy that with a poor initial guess, the LSQP modification exhibited a 0.07% error in the

average objective value (Table 3), but this is still well within acceptable bounds.

Table 2 Results of Solving the Floudas Problem With a Good Initial Guess

Optimum LSQP SQP Matlab SQP

Obj 7049.2 7049.2 (-0.00%) 7049.2 (-0.00%) 7049.2 (-0.00%)
𝑥1 579.3 579.3 (-0.00%) 579.3 (-0.00%) 579.3 (-0.00%)
𝑥2 1360.0 1360.0 (+0.00%) 1360.0 (-0.00%) 1360.0 (-0.00%)
𝑥3 5110.0 5110.0 (-0.00%) 5110.0 (-0.00%) 5110.0 (-0.00%)
𝑥4 182.0 182.0 (-0.00%) 182.0 (-0.00%) 182.0 (-0.00%)
𝑥5 295.6 295.6 (+0.00%) 295.6 (+0.00%) 295.6 (+0.00%)
𝑥6 218.0 218.0 (+0.00%) 218.0 (+0.00%) 218.0 (+0.00%)
𝑥7 286.4 286.4 (-0.00%) 286.4 (-0.00%) 286.4 (-0.00%)
𝑥8 395.6 395.6 (+0.00%) 395.6 (+0.00%) 395.6 (+0.00%)
Iterations - 11.89 (-54.40%) 26.07 (0.00%) 25.65 (-1.60%)
Failures - 0 (0.00%) 0 (0.00%) 0 (0.00%)

On the whole, the Floudas problem is another win for LSQP, as significant computational savings is achieved by

implementing the log transformation. Work remains to be done in the robustness of the LSQP algorithm as presented

here (see Section V.A), but the successful exploitation of the underlying problem structure is clear.

16
 Table 3 Results of Solving the Floudas Problem With a Poor Initial Guess

Optimum LSQP SQP Matlab SQP

Obj 7049.2 7053.9 (+0.07%) 7049.2 (-0.00%) 7049.2 (-0.00%)
𝑥1 579.3 577.6 (-0.30%) 579.3 (-0.00%) 579.3 (-0.00%)
𝑥2 1360.0 1358.3 (-0.12%) 1360.0 (-0.00%) 1360.0 (+0.00%)
𝑥3 5110.0 5118.0 (+0.16%) 5110.0 (-0.00%) 5110.0 (-0.00%)
𝑥4 182.0 181.8 (-0.11%) 182.0 (-0.00%) 182.0 (-0.00%)
𝑥5 295.6 295.4 (-0.06%) 295.6 (+0.00%) 295.6 (+0.00%)
𝑥6 218.0 218.2 (+0.09%) 218.0 (+0.00%) 218.0 (+0.00%)
𝑥7 286.4 286.4 (-0.00%) 286.4 (-0.00%) 286.4 (-0.00%)
𝑥8 395.6 395.4 (-0.05%) 395.6 (+0.00%) 395.6 (+0.00%)
Iterations - 20.48 (-39.49%) 33.85 (0.00%) 32.83 (-3.03%)
Failures - 145 (14.50%) 0 (0.00%) 0 (0.00%)

E. Results from the Kirschen-Ozturk Aircraft Design Problem

Kirschen[37] proposes a signomial program representing a low fidelity aircraft sizing, originally attributed to Ozturk.

This problem is an extension of one of the more simple cases proposed by Hoburg[18]. It is GP compatible with the

exception of a single fuel volume constraint, making it a non-convex Signomial Program. It has been reformulated here

to conform to LSQP standard form:

minimize 𝑊 𝑓
𝑐𝑇 𝑡𝐷
subject to ≤1
𝑊𝑓
𝑅
≤1
𝑉𝑡
1 2
2 𝜌𝑉 𝑆𝐶 𝐷
≤1
𝐷
𝐴𝐶𝐷0 𝑘𝐶 𝑓 𝑆 𝑤𝑒𝑡 𝐶 𝐿2
+ + ≤1
𝑆𝐶𝐷 𝐶𝐷 𝑆 𝜋 𝐴𝑒𝐶𝐷
0.074𝑅𝑒 −0.02
≤1
𝐶𝑓
𝜇𝑅𝑒
√︁ ≤1
𝜌𝑉 𝑆/𝐴
1
𝑊0 𝑊𝑤 2𝑊 𝑓
1
+ 1
+ 1
≤1
2 2 2
2 𝜌𝑉 𝑆𝐶 𝐿 2 𝜌𝑉 𝑆𝐶 𝐿 2 𝜌𝑉 𝑆𝐶 𝐿
𝑊
1 2
≤1 (24)
2 𝜌𝑉𝑚𝑖𝑛 𝑆𝐶 𝐿𝑚𝑎𝑥
𝑊0 𝑊 𝑤 𝑊 𝑓
+ + ≤1
𝑊 𝑊 𝑊
𝑊 𝑤𝑠𝑢𝑟 𝑓 𝑊 𝑤𝑠𝑡𝑟 𝑐
+ ≤1
𝑊𝑤 𝑊𝑤

17
 𝐶𝑊𝑤 ,1 𝑆
≤1
𝑊 𝑤𝑠𝑢𝑟 𝑓
3
√︃
𝑁𝑢𝑙𝑡 𝐴 2 (𝑊0 + 𝑉 𝑓 𝑓 𝑢𝑠𝑒 𝑔𝜌 𝑓 )𝑊 𝑆
𝐶𝑊𝑤 ,2 ≤1
𝑊 𝑤𝑠𝑡𝑟 𝑐 𝜏
𝑉𝑓
≤1
𝑉 𝑓𝑎𝑣 𝑎𝑖𝑙
𝑉 𝑓 𝑔𝜌 𝑓
=1
𝑊𝑓

𝑉 𝑓𝑎𝑣 𝑎𝑖𝑙 − 𝑉 𝑓𝑤𝑖𝑛𝑔 − 𝑉 𝑓 𝑓 𝑢𝑠𝑒 + 1 ≤ 1

𝑉 2𝑓𝑤𝑖𝑛𝑔 𝐴
≤1
0.0009𝑆 3 𝜏 2
𝑉 𝑓 𝑓 𝑢𝑠𝑒
≤1
𝐴𝐶𝐷0 10[𝑚]

Variables were also constrained to be greater than a small positive constant in order to assist in the construction of

sub-problems.

Kirschen showed an approximately 40% decrease in the number of iterations between the log transformed problem

and the original problem[37], but only considered a single initial guess that yielded an interpretable result. Table 4

confirms that result with a good initial guess. However the trials with poor initial guesses demonstrate a far more

significant win for LSQP, as shown in Table 5 and Figure 6(b).

1.0 1.0
Probability of Convergence

Probability of Convergence

0.8 0.8
0.6 0.6
0.4 0.4
0.2 Matlab SQP 0.2 Matlab SQP
SQP SQP
0.0 LSQP 0.0 LSQP
0 100 200 300 400 500 0 100 200 300 400 500
Number of Iterations Number of Iterations
(a) A Good Initial Guess (b) A Poor Initial Guess

Fig. 6 Probability of Convergence vs. Iteration Count for the Kirschen-Ozturk Problem

This test problem is the first one where notable error occurs in the average SQP objective value (greater than 0.1%).

While notable, these are not significant, with the exception being the Matlab SQP case with poor initial guesses where

the average error in the objective was 3.75%. To explain this discrepancy, consider the histogram of error in the objective,

Figure 7. It is apparent that despite the Matlab SQP claiming convergence, the much wider distribution of error implies

that the solution quality is not nearly as good as the other two algorithms. Notably, approximately 10% of trials had an

18
Table 4 Results of Solving the Kirschen-Ozturk Problem With a Good Initial Guess

Optimum LSQP SQP Matlab SQP

Obj [𝑛𝑒𝑤𝑡𝑜𝑛] 755.5 755.9 (+0.05%) 756.3 (+0.11%) 756.6 (+0.15%)
𝐴 [−] 6.57 6.51 (-0.86%) 6.53 (-0.67%) 6.59 (+0.34%)
𝐶𝐷 [−] 0.013 0.013 (-0.08%) 0.013 (-0.01%) 0.013 (+0.24%)
𝐶𝐿 [−] 0.235 0.234 (-0.47%) 0.235 (-0.22%) 0.237 (+0.69%)
𝐶 𝑓 [−] 3.281e-3 3.277e-3 (-0.12%) 3.278e-3 (-0.11%) 3.283e-3 (+0.05%)
𝐷 [𝑁 ] 367.7 368.8 (+0.28%) 368.8 (+0.30%) 367.5 (-0.07%)
𝑅𝑒 [−] 5.83e6 5.87e6 (+0.61%) 5.87e6 (+0.67%) 5.83e6 (-0.05%)
𝑆 [𝑚2 ] 16.00 15.99 (-0.10%) 16.00 (-0.03%) 16.03 (+0.19%)
𝑉 [𝑚/𝑠] 54.1 54.2 (+0.23%) 54.2 (+0.19%) 54.0 (-0.21%)
𝑉 𝑓 [𝑚3 ] 0.096 0.096 (+0.05%) 0.096 (+0.11%) 0.096 (+0.15%)
𝑉 𝑓𝑎𝑣 𝑎𝑖𝑙 [𝑚3 ] 0.110 0.0958 (-13%) 0.0959 (-13%) 0.0959 (-13%)
𝑉 𝑓 𝑓 𝑢𝑠𝑒 [𝑚3 ] 5.69e-4 5.66e-3 (+894%) 5.64e-3 (+890%) 5.80e-3 (+919%)
𝑉 𝑓𝑤𝑖𝑛𝑔 [𝑚3 ] 0.0723 0.0902 (+25%) 0.0903 (+25%) 0.0901 (+25%)
𝑊 [𝑁 ] 7145.8 7138.6 (-0.10%) 7143.7 (-0.03%) 7159.2 (+0.19%)
𝑊 𝑓 [𝑁 ] 755.5 755.9 (+0.05%) 756.3 (+0.11%) 756.6 (+0.15%)
𝑊𝑤 [𝑁 ] 1450.3 1442.7 (-0.52%) 1447.4 (-0.20%) 1462.6 (+0.85%)
𝑊𝑤𝑠𝑡𝑟 𝑐 [𝑁 ] 726.2 719.4 (-0.94%) 723.5 (-0.37%) 737.2 (+1.51%)
𝑊𝑤𝑠𝑢𝑟 𝑓 [𝑁 ] 724.0 723.3 (-0.10%) 723.8 (-0.03%) 725.4 (+0.19%)
𝑡 [𝑚𝑖𝑛] 308.2 307.5 (-0.23%) 307.8 (-0.13%) 309.1 (+0.30%)
Iterations - 27.37 (-45.22%) 49.96 (0.00%) 44.25 (-11.43%)
Failures - 0 (0.00%) 4 (0.40%) 29 (2.92%)

Table 5 Results of Solving the Kirschen-Ozturk Problem With a Poor Initial Guess

Optimum LSQP SQP Matlab SQP

Obj [𝑛𝑒𝑤𝑡𝑜𝑛] 755.5 755.9 (+0.05%) 758.1 (+0.35%) 783.9 (+3.75%)
𝐴 [−] 6.57 6.51 (-0.86%) 6.31 (-3.82%) 5.83 (-11%)
𝐶𝐷 [−] 0.013 0.013 (-0.08%) 0.013 (-0.71%) 0.013 (-1.84%)
𝐶𝐿 [−] 0.235 0.234 (-0.47%) 0.229 (-2.78%) 0.218 (-7.38%)
𝐶 𝑓 [−] 3.281e-3 3.277e3 (-0.12%) 3.259e-3 (-0.69%) 3.28e-3 (+0.01%)
𝐷 [𝑁 ] 367.7 368.8 (+0.28%) 375.8 (+2.20%) 413.0 (+12%)
𝑅𝑒 [−] 5.83e6 5.87e6 (+0.61%) 6.09e6 (+4.48%) 6.43e6 (+10%)
𝑆 [𝑚2 ] 16.00 15.99 (-0.10%) 15.92 (-0.53%) 19.27 (+20%)
𝑉 [𝑚/𝑠] 54.1 54.2 (+0.23%) 55.1 (+1.79%) 61.2 (+13%)
𝑉 𝑓 [𝑚3 ] 0.096 0.096 (+0.05%) 0.096 (+0.35%) 0.152 (+58%)
𝑉 𝑓𝑎𝑣 𝑎𝑖𝑙 [𝑚3 ] 0.110 0.0958 (-13%) 0.0963 (-13%) 0.1014 (-8.22%)
𝑉 𝑓 𝑓 𝑢𝑠𝑒 [𝑚3 ] 5.69e-4 5.66e-3 (+894%) 5.18e-3 (+810%) 1.09e-2 (+1822%)
𝑉 𝑓𝑤𝑖𝑛𝑔 [𝑚3 ] 0.0723 0.0902 (+25%) 0.0912 (+26%) 0.1163 (+61%)
𝑊 [𝑁 ] 7145.8 7138.6 (-0.10%) 7108.1 (-0.53%) 7451.8 (+4.28%)
𝑊 𝑓 [𝑁 ] 755.5 755.9 (+0.05%) 758.1 (+0.35%) 783.9 (+3.75%)
𝑊𝑤 [𝑁 ] 1450.3 1442.7 (-0.52%) 1409.9 (-2.78%) 1684.6 (+16%)
𝑊𝑤𝑠𝑡𝑟 𝑐 [𝑁 ] 726.2 719.4 (-0.94%) 689.7 (-5.03%) 664.7 (-8.47%)
𝑊𝑤𝑠𝑢𝑟 𝑓 [𝑁 ] 724.0 723.3 (-0.10%) 720.2 (-0.53%) 1016.7 (+40%)
𝑡 [𝑚𝑖𝑛] 308.2 307.5 (-0.23%) 303.6 (-1.50%) 297.6 (-3.44%)
Iterations - 30.11 (-74.81%) 119.50 (0.00%) 125.20 (+4.77%)
Failures - 3 (0.30%) 560 (56.00%) 375 (48.14%)

19
 100
Percentage of Converged Cases 100

Percentage of Converged Cases

Matlab SQP Matlab SQP
80 SQP 80 SQP
LSQP LSQP
60 60
40 40
20 20
08 7 6 5 4 3 2 1 0 08 7 6 5 4 3 2 1 0
Log of Percent Error Log of Percent Error
(a) A Good Initial Guess (b) A Poor Initial Guess

Fig. 7 Error Histogram for the Kirschen-Ozturk Problem

error above 1%.

It is also notable that none of the algorithms finds the true global optimum, which is apparent from the discrepancy

in the 𝑉 𝑓 𝑓 𝑢𝑠𝑒 variable. Either some local optimum is being reached, or the fact that some of these variables are so small

in magnitude means the termination criteria can be met without driving all the way to the true optimal value. However,

both the LSQP modification and the traditional SQP algorithm attain the same solution and so neither is considered to

have superior performance here.

This test problem is another clear win for the LSQP modification, and is perhaps the most significant result of this

work. The best result is shown in Figure 6(b): an aircraft design problem is solved with a 74% reduction in number of

iterations and with a 45% higher success rate when using the LSQP modification as opposed to traditional SQP.

F. When Should LSQP be Used Over SQP?

In light of the evidence presented here, the question remains when LSQP should be used in place of SQP. First,

three tenants must hold true: variables are expected to be strictly positive, the objective function is expected to be

strictly positive, and constraints in LSQP standard form are expected to be strictly positive. If these tenants hold, then

LSQP should be considered. In the case of the Floudas problem, these tenants alone are sufficient to show a significant

improvement in the number of iterations required for convergence. But as was shown in the Rosenbrock Problem, these

tenants can hold true and still not imply log-convexity. Thus, one additional indicator for use of LSQP is a significant

percentage of GP-compatible constraints (monomials and posynomials). If GP compatible constraints are present in

large numbers, then some degree of log-convexity can be expected, and LSQP should be strongly considered.

20
 VII. Conclusions
This work demonstrates that LSQP solves some engineering design problems faster than classical SQP, and is

capable of solving problems that are not solvable by classical SQP. Thus, LSQP is a new tool that both improves and

extends existing capability.

While previous research efforts in Geometric and Signomial Programming have exposed the log-convex structure

present in many engineering design problems, the adoption of these methodologies has been slow. To adopt a GP or SP

approach requires total commitment to a new process and a rewriting of existing models, a barrier too substantial for

most designers. LSQP provides a middle ground, enabling the exploitation of log-convexity while requiring no change

to existing processes or discarding of trusted black box models. As a result of the work done here, LSQP can be viewed

as a direct substitute for traditional SQP and the potential computational savings obtained from this simple switch are

significant.

Funding Sources
This material is based on research sponsored by the U.S. Air Force under agreement number FA8650-20-2-2002.

The U.S. Government is authorized to reproduce and distribute reprints for Governmental purposes notwithstanding any

copyright notation thereon. The views and conclusions contained herein are those of the authors and should not be

interpreted as necessarily representing the official policies or endorsements, either expressed or implied, of the U.S. Air

Force or the U.S. Government.

Acknowledgments
The author would like to thank Bob Haimes and Mark Drela for their mentorship and support and for their notes on

this paper, the EnCAPS Technical Monitor Ryan Durscher, Philippe Kirschen who laid some of the foundation for this

work, Berk Ozturk who originally authored the fourth test problem, and Devon Jedamski for his review and comments.

The author also acknowledges the MIT SuperCloud and Lincoln Laboratory Supercomputing Center for providing

HPC, database, and consultation resources that have contributed to the research results reported within this paper.

References
[1] Boyd, S., Kim, S.-J., Vandenberghe, L., and Hassibi, A., “A Tutorial on Geometric Programming,” Optimization and Engineering,

Vol. 8, No. 1, 2007, pp. 67–127.

[2] Clasen, R. J., “The solution of the chemical equilibrium programming problem with generalized benders decomposition,”

Operations Research, Vol. 32, No. 1, 1984, pp. 70–79.

[3] Greenberg, H. J., “Mathematical programming models for environmental quality control,” Operations Research, Vol. 43, No. 4,

1995, pp. 578–622.

21
 [4] Boyd, S. P., Kim, S.-J., Patil, D. D., and Horowitz, M. A., “Digital circuit optimization via geometric programming,” Operations

research, Vol. 53, No. 6, 2005, pp. 899–932.

[5] Boyd, S. P., Lee, T. H., et al., “Optimal design of a CMOS op-amp via geometric programming,” IEEE Transactions on

Computer-aided design of integrated circuits and systems, Vol. 20, No. 1, 2001, pp. 1–21.

[6] Li, X., Gopalakrishnan, P., Xu, Y., and Pileggi, T., “Robust analog/RF circuit design with projection-based posynomial

modeling,” IEEE/ACM International Conference on Computer Aided Design, 2004. ICCAD-2004., IEEE, 2004, pp. 855–862.

[7] Xu, Y., Pileggi, L. T., and Boyd, S. P., “ORACLE: optimization with recourse of analog circuits including layout extraction,”

Proceedings of the 41st annual Design Automation Conference, 2004, pp. 151–154.

[8] Jabr, R. A., “Application of geometric programming to transformer design,” IEEE Transactions on Magnetics, Vol. 41, No. 11,

2005, pp. 4261–4269.

[9] Chiang, M., Geometric programming for communication systems, Now Publishers Inc, 2005.

[10] Chiang, M., Tan, C. W., Palomar, D. P., O’neill, D., and Julian, D., “Power control by geometric programming,” IEEE

transactions on wireless communications, Vol. 6, No. 7, 2007, pp. 2640–2651.

[11] Kandukuri, S., and Boyd, S., “Optimal power control in interference-limited fading wireless channels with outage-probability

specifications,” IEEE transactions on wireless communications, Vol. 1, No. 1, 2002, pp. 46–55.

[12] Marin-Sanguino, A., Voit, E. O., Gonzalez-Alcon, C., and Torres, N. V., “Optimization of biotechnological systems through

geometric programming,” Theoretical Biology and Medical Modelling, Vol. 4, No. 1, 2007, p. 38.

[13] Vera, J., González-Alcón, C., Marín-Sanguino, A., and Torres, N., “Optimization of biochemical systems through mathematical

programming: Methods and applications,” Computers & Operations Research, Vol. 37, No. 8, 2010, pp. 1427–1438.

[14] Preciado, V. M., Zargham, M., Enyioha, C., Jadbabaie, A., and Pappas, G., “Optimal resource allocation for network protection:

A geometric programming approach,” IEEE Transactions on Control of Network Systems, Vol. 1, No. 1, 2014, pp. 99–108.

[15] Misra, S., Fisher, M. W., Backhaus, S., Bent, R., Chertkov, M., and Pan, F., “Optimal compression in natural gas networks: A

geometric programming approach,” IEEE transactions on control of network systems, Vol. 2, No. 1, 2014, pp. 47–56.

[16] Sela Perelman, L., and Amin, S., “Control of tree water networks: A geometric programming approach,” Water Resources

Research, Vol. 51, No. 10, 2015, pp. 8409–8430.

[17] Agrawal, A., Diamond, S., and Boyd, S., “Disciplined geometric programming,” Optimization Letters, Vol. 13, No. 5, 2019, pp.

961–976.

[18] Hoburg, W., and Abbeel, P., “Geometric Programming for Aircraft Design Optimization,” AIAA Journal, Vol. 52, No. 11, 2014,

pp. 2414–2426.

22
[19] Torenbeek, E., Advanced Aircraft Design: Conceptual Design, Analysis and Optimization of Subsonic Civil Airplanes, 2nd ed.,

John Wiley & Sons, Ltd., Chichester, United Kingdom, 2013.

[20] Hoburg, W., and Abbeel, P., “Fast Wind Turbine Design via Geometric Programming,” 54th AIAA/ASME/ASCE/AHS/ASC

Structures, Structural Dynamics, and Materials Conference, 2013.

[21] Kirschen, P. G., Burnell, E. E., and Hoburg, W. W., “Signomial Programming Models for Aircraft Design,” 54th AIAA Aerospace

Sciences Meeting, 2016.

[22] Brown, A., and Harris, W., “A vehicle design and optimization model for on-demand aviation,” 2018 AIAA/ASCE/AHS/ASC

Structures, Structural Dynamics, and Materials Conference, 2018, p. 0105.

[23] York, M. A., Öztürk, B., Burnell, E., and Hoburg, W. W., “Efficient Aircraft Multidisciplinary Design Optimization and

Sensitivity Analysis via Signomial Programming,” AIAA Journal, Vol. 56, No. 11, 2018, pp. 4546–4561.

[24] Burton, M., and Hoburg, W., “Solar and gas powered long-endurance unmanned aircraft sizing via geometric programming,”

Journal of Aircraft, Vol. 55, No. 1, 2018, pp. 212–225.

[25] Lin, B., Carpenter, M., and de Weck, O., “Simultaneous Vehicle and Trajectory Design using Convex Optimization,” AIAA

Scitech 2020 Forum, 2020, p. 0160.

[26] Kirschen, P. G., York, M. A., Ozturk, B., and Hoburg, W. W., “Application of Signomial Programming to Aircraft Design,”

Journal of Aircraft, Vol. 55, No. 3, 2018, pp. 965–987.

[27] York, M. A., Hoburg, W. W., and Drela, M., “Turbofan engine sizing and tradeoff analysis via signomial programming,” Journal

of Aircraft, Vol. 55, No. 3, 2018, pp. 988–1003.

[28] Saab, A., Burnell, E., and Hoburg, W. W., “Robust Designs via Geometric Programming,” arXiv, 2018.

[29] Hall, D. K., Dowdle, A., Gonzalez, J., Trollinger, L., and Thalheimer, W., “Assessment of a boundary layer ingesting turboelectric

aircraft configuration using signomial programming,” 2018 Aviation Technology, Integration, and Operations Conference,

2018, p. 3973.

[30] Martins, J. R., and Lambe, A. B., “Multidisciplinary Design Optimization: A Survey of Architectures,” AIAA Journal, Vol. 51,

No. 9, 2013, pp. 2049–2075.

[31] Wakayama, S., “Multidisciplinary Design Optimization of the Blended-Wing-Body,” AIAA Paper AIAA-98-4938, 1998.

[32] Kroo, I., and Takai, M., “A Quasi-Procedural, Knowledge-Based System for Aircraft Design,” AIAA Paper AIAA-88-6502, 1988.

[33] Boggs, P. T., and Tolle, J. W., “Sequential Quadratic Programming,” Acta Numerica, Vol. 4, 1996, pp. 1–51.

[34] Nocedal, J., and Wright, S., Numerical Optimization, Springer Science & Business Media, 2006.

[35] Kraft, D., “A software package for sequential quadratic programming,” 1988.

23
[36] Andersen, M. S., Dahl, J., and Vandenberghe, L., “CVXOPT: A Python package for convex optimization,” 2013.

[37] Kirschen, P. G., and Hoburg, W. W., “The power of log transformation: A comparison of geometric and signomial programming

with general nonlinear programming techniques for aircraft design optimization,” 2018 AIAA/ASCE/AHS/ASC Structures,

Structural Dynamics, and Materials Conference, 2018, p. 0655.

[38] “Constrained Nonlinear Optimization Algorithms,” https://www.mathworks.com/help/optim/ug/constrained-

nonlinear-optimization-algorithms.html, July 2020.

[39] Reuther, A., Kepner, J., Byun, C., Samsi, S., Arcand, W., Bestor, D., Bergeron, B., Gadepally, V., Houle, M., Hubbell, M., et al.,

“Interactive supercomputing on 40,000 cores for machine learning and data analysis,” 2018 IEEE High Performance extreme

Computing Conference (HPEC), IEEE, 2018, pp. 1–6.

[40] Floudas, C. A., Pardalos, P. M., Adjiman, C., Esposito, W. R., Gümüs, Z. H., Harding, S. T., Klepeis, J. L., Meyer, C. A., and

Schweiger, C. A., Handbook of test problems in local and global optimization, Vol. 33, Springer Science & Business Media,

2013.

24