Computers & Operations Research 73 (2016) 165–173

Contents lists available at ScienceDirect

Computers & Operations Research

journal homepage: www.elsevier.com/locate/caor

Mixed Integer Programming models for job shop scheduling: A

computational analysis
Wen-Yang Ku n, J. Christopher Beck
Department of Mechanical & Industrial Engineering, University of Toronto, 5 King's College Rd., Toronto, ON, Canada M5S 3G8

art ic l e i nf o a b s t r a c t

Article history: In both industry and the research literature, Mixed Integer Programming (MIP) is often the default ap-
Received 11 November 2014 proach for solving scheduling problems. In this paper we present and evaluate four MIP formulations for
Received in revised form the classical job shop scheduling problem (JSP). While MIP formulations for the JSP have existed since the
4 October 2015
1960s, it appears that comprehensive computational studies have not been performed since then. Due to
Accepted 8 April 2016
Available online 9 April 2016
substantial improvements in MIP technology in recent years, it is of interest to compare the standard JSP
models using modern optimization software. We perform a fully crossed empirical study of four MIP
Keywords: models using CPLEX, GUROBI and SCIP, focusing on both the number of instances that can be proved
Job shop scheduling optimal and the solution quality over time. Our results demonstrate that modern MIP solvers are able to
Mixed Integer Programming
prove optimality for moderate-sized problems very quickly. Comparing the four MIP models, the dis-
Constraint programming
junctive formulation proposed by Manne performs best on both performance measures. We also in-
vestigate the performance of MIP with multi-threading and parameter tuning using CPLEX. Noticeable
performance gain is observed when compared to the results using only single thread and default
parameter settings. Our results serve as a snapshot of the performance of modern MIP solvers for an
important, well-studied scheduling problem. Finally, the results of MIP is compared to constraint pro-
gramming (CP), another common approach for scheduling, and the best known complete algorithm to
provide a broad view among different approaches.
& 2016 Elsevier Ltd. All rights reserved.

1. Introduction branch and bound algorithm based on the disjunctive formulation

and concluded that the JSP is computationally challenging even for
Mixed Integer Programming (MIP) has been widely applied to moderate-sized problems.
scheduling problems and it is often the initial approach to attack a In this paper, we compare the performance of four MIP models
new scheduling problem. For example, of the 40 research papers for the classical job shop scheduling problem (JSP). In addition to
published in the Journal of Scheduling in 2014, 14 use MIP, more the three standard JSP formulations, we include a second dis-
than any other technology. Given this popularity, together with the junctive formulation claimed by Liao [8] to be superior to the
improvements in commercial MIP technology [1], it is valuable to original one. We perform experiments with three different solvers:
understand how various MIP models for scheduling compare with IBM ILOG CPLEX v12.6.2 [9], GUROBI v6.0.4 [10], and SCIP v3.1.1
each other in the context of modern solvers. [11]. CPLEX and GUROBI are both regarded as the state-of-the-art
There are three widely used general MIP formulations for commercial MIP solvers [1] while SCIP is the fastest non-com-
scheduling problems: the time-indexed formulation [2,3], the mercial solver [12]. Our goal is to provide a complete empirical
rank-based formulation [4], and the disjunctive formulation [5]. A comparison of the MIP models for the JSP using different MIP
theoretical comparison among these formulations claims that the solvers and identify the most efficient model.
disjunctive formulation as best and the time-indexed formulation Our experimental results using CPLEX and GUROBI show, con-
worst [6]. However, despite the long history of these formulations, trary to Pan [6], that the time-indexed model is able to perform
there does not appear to have been a complete computational better than the rank-based model for small problems but fails to
scale to larger problems due to the size of the model. In addition,
study comparing the performance of different formulations using
contrary to Liao's finding, our results show that the original dis-
modern MIP solvers. In the most recent computational study we
junctive model is more efficient than Liao's disjunctive model for
have found (from 1991), Applegate et al. [7] proposed an efficient
both CPLEX and GUROBI. However, the experiments using SCIP
provide the opposite conclusions: first, the rank-based model
n
Corresponding author. outperforms the time-indexed model for small problems and

http://dx.doi.org/10.1016/j.cor.2016.04.006
0305-0548/& 2016 Elsevier Ltd. All rights reserved.
166 W.-Y. Ku, J.C. Beck / Computers & Operations Research 73 (2016) 165–173

second, while at first glance Liao's disjunctive formulation seems

more efficient than the original disjunctive formulation, careful
investigation reveals that these differences are due to different
preprocessing techniques undertaken by SCIP, and the phenom-
enon of erraticism [13] in the search process of MIP solvers. De-
spite these differences, across all tested solvers the disjunctive
model outperforms the rank-based and the time-indexed models.
We then investigate two crucial aspects of modern MIP solvers:
multi-threading and parameter tuning. Multi-threading allows the
search to be executed in parallel and therefore may drastically
improve performance. Parameter tuning aims at finding the best
set of parameters for a specific class of problems. We perform
experiments with the best MIP model, the disjunctive model,
using CPLEX. Results show that running 8 threads in parallel im-
proves the performance by about a factor of three. With multi-
Fig. 1. Job shop scheduling problem. Three jobs J1, J2, and J3 are to be scheduled on
threading enabled, CPLEX's parameter tuning tool can further three machines M1, M2 and M3. The graph on the top represents the precedence
improve the performance by about a factor of 1.5 for problems that constraints. The Gantt Chart on the bottom displays a feasible schedule which
can be proved optimal, which is 4.5 times faster than the single- satisfies the precedence constraints.
threaded default search. For the problems for which the optimal
solution could not be found and proved within one hour, the so-
2.2. Literature review
lution quality over time is improved by about 30%.
We also provide a comparison of the best MIP results with two
As noted, the three standard MIP formulations for scheduling
complete approaches: constraint programming (CP) [14,15] and
problems are the time-indexed formulation [2,3], the rank-based
iSTS-SGS [16], with the latter being the state-of-the-art exact al- formulation [4], and the disjunctive formulation [5]. Other models
gorithm. Our motivations of comparing CP and MIP are twofold. are typically combinations or variations of these formulations. We
First, since MIP and CP are widely used by practitioners as “out-of- define each model formally in the next section.
the-box” technology for scheduling, it is valuable to understand Pan [6] performed a theoretical comparison of these three
the differences between their performance. Second, due to the models for the JSP, the flow shop scheduling problem, and the
substantial improvements of MIP and CP solvers over the last permutation flow shop scheduling problem. He concluded that the
decade [1], it is of interest to investigate the advances of MIP and disjunctive model is most efficient since it has the fewest binary
CP, and how they compete with a state-of-the-art algorithm. We variables, followed by the rank-based model, and then the time-
perform an empirical study using CPLEX for the disjunctive MIP indexed model. While model size is an important aspect of the
model, with both multi-threading and parameter tuning and IBM quality of a formulation, it is well-known that other characteristics
ILOG CP Optimizer for the CP model. Our experimental results such as the tightness of the linear relaxation can be equally or
demonstrate that MIP performs similarly to CP for problems with more important. Pan's evaluation of the models was challenged for
moderate size and the goal of proving optimality. However, CP the flow shop scheduling problem by Ronconi [18], who empiri-
dominates MIP for the larger JSP instances and it is competitive cally compared the performance of two MIP models and showed
with iSTS-SGS. that a model with half the number binary variables performed
In summary, the contribution of this paper lies in the empirical much worse that a larger model.
analysis of the MIP models for the job shop scheduling problem. Liao [8] proposed a modified disjunctive formulation and
The rest of the paper is organized as follows. In Section 2 we give showed empirically that this new model performs better than the
the definition of the JSP and review relevant literature. Section 3 original disjunctive model. The reduction in the computation ef-
describes the MIP models. Section 4 presents the computational fort was attributed to the fact that the new model had fewer linear
results and the discussions. We conclude in Section 5. constraints. While Liao showed that the computation time is sig-
nificantly reduced with this new formulation, the experiments
used problem instances with fewer than 5 jobs and 10 machines.
The literature on the JSP is vast with Google Scholar returning
2. Background
over 60,000 references (accessed September 28, 2015). Most ap-
proaches to optimization have been applied to JSP at some point,
2.1. Problem definition
including heuristics [19], metaheuristics [20], genetic and evolu-
tionary algorithms [21], customized branch-and-bound [7], con-
The JSP is defined by a finite set J of n jobs and a finite set M of
straint programming [22], and decomposition procedures [16].
m machines (Fig. 1). For convenience we refer an n × m problem.
For each job j ∈ J , we are given a list ( σ1j, … , σhj, … , σmj ) of the
machines which represents the processing order of j through the 3. MIP models
machines. Note that shj is called the h-th operation of job j and smj
is the last operation of job j. In addition, for each job j and machine In this section, we present the four MIP models considered in
i, we are given a non-negative integer pij, which represents the this paper.
processing time of j on i. Each machine can process at most one job
at a time, and once a job starts on a given machine, it must 3.1. Disjunctive model
complete processing on that machine without interruption. The
objective is to find a schedule of J on M that minimizes the ma- Our disjunctive model is based on Manne [5]. The decision
kespan, i.e., the maximum completion time of the last operation of variables are defined as follows:
any job in J. Makespan minimization for the JSP is NP-hard for
n ≥ 3 and m ≥ 2 [17].
xij is the integer start time of job j on machine i
 W.-Y. Ku, J.C. Beck / Computers & Operations Research 73 (2016) 165–173 167

Fig. 2. Disjunctive model [5].

Fig. 3. Time-indexed model [3].

zijk is equal to 1 if job j precedes job k on machine i.
3.4. Rank-based model
The disjunctive MIP model is stated in Fig. 2.
The objective function is stated in (1). Constraint (2) ensures The rank-based model is due to Wagner [4]. The decision
that the start time of each job is greater or equal to 0. Constraint variables are defined as follows:
(3) is the precedence constraint. It ensures that all operations of a
job are executed in the given order. The disjunctive constraints
xijk is equal to 1 if job j is scheduled at the k-th position on
(4) and (5) ensure that no two jobs can be scheduled on the same machine i
machine at the same time. V has to be assigned to a large enough
hik denotes the start time of the job at the k-th position of
value to ensure the correctness of (4) and (5). In our model, we machine i.
assign V = ∑j ∈ J ∑i ∈ M pij , since the completion time of any opera-
The parameter rijk is 1 if the k-th operation of job j requires ma-
tion cannot exceed the summation of the processing times from all
chine i. The rank-based model is given in Fig. 4.
the operations. We use the same V value in the rest of the models
The objective function is stated in (16). Constraint (17) ensures
throughout the paper. Constraint (6) ensures that the makespan is
that each position on each machine is assigned to exactly one job.
at least the largest completion time of the last operation of all jobs.
Constraint (18) ensures that each job only gets one position on a
We note that modern MIP solvers allow direct modeling of the
machine. Constraint (19) states that the start time of a job on a
disjunctive constraints with solvers specific constraints. For ex-
machine should be larger than the completion time of the job
ample, Constraints (4) and (5) can be modeled with the “indicator”
scheduled at the previous position. Constraint (20) is the pre-
constraints in CPLEX. We investigate the performance of the in-
cedence constraint. It ensures that all operations of a job are
dicator constraints in the experiments section.
executed in the given order. Constraint (21) ensures that the ma-
kespan is at least the largest completion time of the last job on all
3.2. Liao's disjunctive model machines. Constraint (22) ensures that the start time of all jobs at
all positions are greater or equal to 0.
In his disjunctive model, Liao [8] added continuous, surplus The number of variables and constraints for each model is
variables qijk to constraint (4). Constraints (4) and (5) therefore summarized in Table 1.
become:
V ·zijk + (xij − xik ) − pik = qijk , ∀ j, k ∈ J , i ∈ M (8)
4. Experiments and discussion

qijk ≤ V − pij − pik , ∀ ∀ j, k ∈ J , i ∈ M (9) All experiments were performed on a Intel Core i7 3.40 GHz
The addition of the new decision variables reduces the number machine (in 64 bit mode) with 8 GB memory running Red Hat
of linear constraints, but introduces additional variables and upper Enterprise 6.2. We perform experiments with CPLEX Optimization
bounds on these variables. Liao claimed that this transformation
can improve performance, since the bounds on the variables are
easier to handle than the linear constraints.

3.3. Time-indexed model

Although the idea of using time-indexed variables was origin-

ally proposed by Bowman [2], we adopt Kondili's model [3] here
because our preliminary experiments shows that it is computa-
tionally more efficient than Bowman's model. The decision vari-
able, xijt, is equal to 1 if job j starts at time t at machine i. The time-
indexed MIP model can be written as in Fig. 3.
The objective function is stated in (10). Constraint (11) ensures
that each job starts exactly once on each machine. Constraint (12)
ensures that the makespan is at least the largest completion time
of the last operation of all jobs. Constraint (13) ensures that the
machine is not over-capacitated at any time point. Constraint (14)
is the precedence constraint. It ensures that all operations of a job
are executed in the given order. Fig. 4. Rank-based model [4].
168 W.-Y. Ku, J.C. Beck / Computers & Operations Research 73 (2016) 165–173

Table 1 proportional to the numeric value of processing time). When the

Comparison of the MIP models for the JSP. J , M , and H denote the number of jobs, problem scales up or the number of jobs is increased, the mod-
the number of machines, and the number of total time points, respectively. eling time becomes dominant, and the model does not fit into a
Model Number of bin- Number of constraints Number of other
reasonable amount of memory. However, even for problems where
ary variables integer variables that memory is not an issue, the disjunctive models perform better
than the time-indexed model.
Time-Indexed J M H M H + 3M J − J 1 Table 2 shows that with CPLEX the performance of Liao's dis-
Rank-Based J2 M J3 M + 4J M + M J M junctive model is worse than the original disjunctive model,
Disjunctive J2 M J2 M J M +1 contradicting the results of Liao [8], where the performance of the
new formulation significantly outperforms the old formulation for
problems with size 3 × 3, 4 × 3, 5 × 3, 3 × 6, 3 × 8, and 3 × 10. Our
results are consistent with Ronconi [18] who found that Liao's
Studio v12.6.2, GUROBI v6.0.4, and SCIP v3.1.1.
disjunctive model is not better than the original model for the flow
The CPU time limit for each run on each problem instance is
shop scheduling problem.
3600 s. All the solvers are executed in their default settings with
Fig. 5(a) provides a more detailed look at the 15 × 15 results by
one thread unless specified otherwise.
plotting the mean relative error (MRE) of the disjunctive MIP
models over the run time. The relative error for each instance is
4.1. Problem instances
computed as

The experiments consist of ten problem sets of with problem best incumbent − best known solution
RE = × 100,
sizes of {3 × 3, 4 × 3, 5 × 3, 3 × 6, 3 × 8, 3 × 10, 5 × 5, 8 × best known solution
8, 10 × 10, 15 × 15, 20 × 15, 20 × 20} . Each set consists of 10 where the best incumbent is the best solution found at a given
problem instances. In addition to the first six problem sizes that time point. MRE is then the arithmetic average of the relative error
were originally used by Liao, in this paper we generate the over the problem instances in the relevant set. Fig. 5(a) shows that
5 × 5, 8 × 8, 10 × 10, 15 × 15, 20 × 15 and 20 × 20 problems, and the solution quality of Manne's model improves much faster than
use Taillard's 15 × 15 and 20 × 15 benchmark instances [23] for that of Liao's model.
additional comparison (specifically those labeled ta01 through As neither disjunctive model can solve all of the Taillard's
ta20). All instances that we created are generated using Taillard's 15 × 15 problem instances (ta01 through ta10) to optimality, we
JSP problem generator [24]. In accordance with Liao's experi- again compare their solution quality over time. Consistent with
mental setup, the processing times are randomly generated from a the 15 × 15 problems, Fig. 5b shows that Manne's model finds
discrete uniform distribution over [1,20]. The order of the opera- better feasible solutions much faster than Liao's model. However,
tions in each job is assigned randomly. the Taillard's instances are more difficult: in a given run-time
lower MREs are achieved for the randomly generated problem
4.2. Comparison of MIP models instances. Judging from the flattening of the MIP curves, it seems
unlikely that Liao's model will achieve solution quality equal to
We first reproduce Liao's experiments [8], adding the time- Manne's model at any reasonable run-time. The results of the
indexed and the rank-based models. An overview of the results of 20 × 15 problem instances and the Taillard's 20 × 15 problem in-
CPLEX, GUROBI, and SCIP are given in Table 2. For each problem stances (ta11 through ta20) show similar trends as the smaller
size we report the arithmetic mean CPU running time “arith”, the instances, reinforcing our results.
shifted geometric mean time “geo” of the 10 problem instances Results of GUROBI. The results of GUROBI are consistent with
and the number of instances proved to optimality “Opt”. The those of CPLEX, showing that Manne's disjunctive formulation is
shifted geometric mean time is computed as follows: most efficient. The time-indexed model is also able to outperform
∏ (ti + s)1/ n − s, the rank-based model for smaller problems. Detailed results are
presented in Table 2. We do not plot the MRE figures as the results
where ti is the actual running time, n is the number of instances, have a very similar relative performance as CPLEX.
and s is chosen as 10. Using geometric mean can decrease the Results of SCIP. The results of SCIP in Table 2 show different
influence of the outliers of data [25]. behavior from CPLEX and GUROBI. Liao's disjunctive formulation
Results of CPLEX. The results of the time-indexed model, the performs better than Manne's model for small problems up to size
rank-based model, and the disjunctive model in Table 2 are par- 12 × 12. For larger problems there is no clear winner (Fig. 6(a) and
tially consistent with Pan's theoretical analysis [6]. Our results (b). The results also show that the rank-based model is more ef-
indicate that both disjunctive models perform much better than ficient than the time-indexed model for all the problems.
the rank-based and time-indexed models. They are able to solve The reason that SCIP performs differently is due to the different
the 10 × 10 problems very quickly, whereas the rank-based and presolving techniques compared to CPLEX and GUROBI, and pos-
time-indexed models cannot solve these problems to optimality in sibly the consequence of erraticism in search. Recall that Liao's
3600 s. Partially contrary to Pan's conclusion that the rank-based formulation reduces the number of linear constraints at the ex-
model is more efficient than the time-indexed model, the latter is pense of introducing new variables. We examined Liao's for-
able to outperform the rank-based model for problems with size mulation after presolving and found that SCIP eliminated these
less than 8 × 8. However, the time-indexed model has great dif- introduced variables in the presolving stage, transforming the
ficulties finding a feasible solution for problems with size more model into Manne's formulation. However, although the presolved
than 10 × 10. The time-indexed model also performs the worst Liao's model is mathematically the same as the original disjunctive
among these four models when problem size reaches 15 × 15, model, the transformation does not achieve the exact same or-
where the time-indexed model does not fit into 8 GB of memory. dering of the constraints and the coefficients of the constraints are
The disadvantage of the time-indexed formulation is that the slightly different. This leads to the phenomenon called “erraticism”
number of variables and constraints is proportional to the number [13]: that minor changes in the initial condition of a MIP solver can
of time-points. The space complexity of the time-indexed model is effect the search process, thus resulting in a different search and a
therefore pseudo-polynomial (i.e., the memory required is differing number of nodes in the search tree.
 W.-Y. Ku, J.C. Beck / Computers & Operations Research 73 (2016) 165–173 169

Table 2
Comparison of MIP Models for the three solvers used. Bold numbers indicate the best MIP model for each solver for the given problem size. The symbol ‘-’ means that none of
the 10 problem instances were solved to optimality within 3600 s. The symbol ‘#’ means the models do not fit into 8 GB memory. The superscript numbers indicate the
number of instances for which no feasible solution was found.

CPLEX Results

Problem Disjunctive Disjunctive (Liao) Rank-based Time-Indexed

Time (geo/arith) Opt Time (geo/arith) Opt Time (geo/arith) Opt Time (geo/arith) Opt

3×3 0.01/0.01 10 0.00/0.00 10 0.01/0.02 10 0.04/0.04 10

4×3 0.01/0.01 10 0.01/0.01 10 0.05/0.05 10 0.07/0.07 10
5×3 0.02/0.02 10 0.01/0.02 10 0.18/0.18 10 0.22/0.22 10
3×6 0.01/0.01 10 0.01/0.01 10 0.15/0.15 10 0.16/0.17 10
3×8 0.01/0.01 10 0.01/0.01 10 0.80/0.81 10 0.23/0.24 10
3 × 10 0.01/0.01 10 0.01/0.01 10 3.01/3.03 10 0.55/0.56 10
5×5 0.02/0.02 10 0.02/0.02 0 21.18/31.98 10 1063.03/2003.15 5
8×8 0.60/0.61 10 0.76/0.77 10 - - 1986.61/2518.23 5
10 × 10 3.19/3.48 10 7.62/8.84 10 - - -9 -
12 × 12 99.43/212.76 10 386.58/1041.44 8 - - -10 -
15 × 15 1568.89/2272.75 5 3463.38/3484.45 1 -1 - # #
20 × 15 - - - - -5 - # #

GUROBI Results
3×3 0.00/0.00 10 0.00/0.00 10 0.02/0.02 10 0.05/0.05 10
4×3 0.00/0.00 10 0.00 /0.00 10 0.04/0.05 10 0.14/0.14 10
5×3 0.00/0.01 10 0.01/0.01 10 0.06/0.06 10 0.57/0.58 10
3×6 0.00/0.00 10 0.00/0.01 10 0.19/0.19 10 0.70/0.77 10
3×8 0.00/0.00 10 0.00/0.01 10 0.54/0.54 10 0.75/0.76 10
3 × 10 0.00/0.00 10 0.00/0.00 10 2.04/2.06 10 1.05/1.07 10
5×5 0.02/0.02 10 0.00/0.00 10 12.82/16.50 10 117.85/188.81 10
8×8 0.40/0.41 10 0.69/0.70 10 - - 2058.77/2722.754 3
10 × 10 2.43/2.76 10 6.63 / 8.84 10 -2 - -10 -
12 × 12 186.52/705.90 10 375.56 / 1019.21 8 -6 - -10 -
15 × 15 2142.14/2784.49 3 2940.86 / 3287.14 1 -6 - # #
20 × 15 - - - - -8 - # #

SCIP Results
3×3 0.00/0.00 10 0.00/0.01 10 0.06/0.06 10 0.43/0.43 10
4×3 0.04/0.04 10 0.02/0.02 10 0.33/0.33 10 1.93/1.95 10
5×3 0.08/0.08 10 0.04/0.04 10 0.99/1.00 10 4.64/4.81 10
3×6 0.01/0.01 10 0.01/0.01 10 0.59/0.59 10 7.69/8.48 10
3×8 0.01/0.01 10 0.02/0.02 10 2.44/2.47 10 25.26/26.72 10
3 × 10 0.01/0.01 10 0.01/0.02 10 10.01/10.2 10 45.24/53.98 10
5×5 0.17/0.17 10 0.16/0.16 10 67.74/83.21 10 3363.01/3422.09 1
8×8 4.04/4.14 10 1.48/1.52 10 -5 - -8 -
10 × 10 30.45/41.54 10 18.93/28.71 10 -8 - -10 -
12 × 12 669.33/1202.90 8 626.59/1156.67 8 -10 - -10 -
15 × 15 3498.00/3510.04 1 3227.29/3360.35 1 -10 - # #
20 × 15 - - - - -10 - # #

Fig. 5. MRE Performance Over Time for the Disjunctive MIP Models using CPLEX (a)15  15 problems, (b) ta01–ta10.
170 W.-Y. Ku, J.C. Beck / Computers & Operations Research 73 (2016) 165–173

Fig. 6. MRE Performance Over Time for the Disjunctive MIP Models using SCIP (a)15  15 problems, (b)ta01–ta10.

To further investigate the erraticism phenomenon, we per- allows modeling the disjunctive constraints directly with its so-
turbed the order of the constraints of Manne's model and found called indicator constraints from which CPLEX derives an opti-
that the performance of Liao's model was within the variance mized formulation and search strategy. We investigate two dif-
observed, i.e., we were able to achieve better performance than ferent methods of modeling the indicator constraints as follows:
Liao's model by simply changing the order of the constraints in the
original disjunctive model. This demonstrates that the perfor- 1. The “IF” formulation: We replace constraints (4) and (5) with
mance gain does not come from Liao's model (as it is mathema- IloIfThen (zijk = 0, x ij ≥ x ik + pik ) and IloIfThen (zijk = 1, x ik ≥ x ij + pij ).
tically equivalent to Manne's model after SCIP's presolving), but 2. The “OR” formulation: We remove the variable zijk and replace
the erraticism phenomenon of MIP solver. Note that CPLEX and constraints (4) and (5) with IloOr (xij ≥ xik + pik , xik ≥ xij + pij ).
GUROBI do not remove the introduced surplus variables from
Liao's model. As Table 3 shows, both formulations using the indicator con-
straints perform worse than the original formulation. The reason
4.3. Comparison of different MIP optimization software may be because that the chosen V value in our model is not too
large. As CPLEX's website [9] suggests: “Use indicator constraints
The overall performance of CPLEX and GUROBI is very close for instead of Big M1 when Big M values in the formulation cannot be
the tested problems, with the former performing slightly better. reduced”. In our case, V is the summation of the processing times
GUROBI is able to prove optimality faster with Liao's formulation, from all the operations, which is still well bounded in our
and it performs better for the rank-based model and the time- experiments.
indexed model for small problems up to size 5 × 5. SCIP, as the
only non-commercial solver, is less efficient than CPLEX and 4.5. Multi-threading and parameter tuning
GUROBI.
Comparing our MIP results with that published by Liao in 1992, We investigate the performance gain of the best MIP model, the
it is observed, unsurprisingly, that the performance of MIP solvers disjunctive model, by enabling multi-threading and parameter
has been significantly improved. In Liao's experiments, it takes tuning using CPLEX.
more than 1000 s to solve the 5 × 3 problems with Manne's Table 4 shows that the performance is improved by about a
model, whereas these problems are solved almost instantly with factor of three with 8 threads compared to that using only one
modern MIP solvers. This demonstrates the ability and strength of thread.
We use CPLEX's parameter tuning tool as follows. First, we
modern MIP solvers and shows that MIP is now a competitive
classified the instances into two categories based on the single-
approach for solving scheduling problems.
threaded, untuned CPLEX results: 1. Optimality provable in 3600 s.
Summarizing the above results, we conclude that the disjunctive
2. Optimality not provable in 3600 s. For the second category, we
model is the most efficient MIP model for the JSP for all problem
further divide the instances into two sets. The first set consists of
sizes. Manne's original formulation is more efficient for CPLEX and
the 15 × 15, 20 × 15 and 20 × 20 instances and the second set
GUROBI, while the preprocessing in SCIP actually transforms Liao's
consists of the Taillard's instances. We assume that the problem
formulation to Manne's with the accompanying similar perfor-
characteristic are different between these two sets therefore
mance. It should be noted that judging the performance of the MIP
avoiding potential performance loss when using the tuning tool.
models based on the number of binary variables and constraints is
We set a total tuning time of two days for both categories, with
error prone and should only be taken into account as the first step of
3600 seconds per run in the first category and 1200 s per run in
analyzing the performance of MIP models. Careful empirical eva-
the second category. Since a majority of instances cannot be pro-
luation may be required to fully understand the behaviour of these
ven optimal for the second category, we choose a shorter running
models, especially when using modern MIP solvers.
time per run to balance between the number of runs and the time
spent at each run. All the training instances are also used as the
4.4. Comparison of disjunctive constraints
testing instances. This is obviously not methodologically sound,
As noted in Section 3.1, we investigate the capability of the
modern MIP solver CPLEX to directly handle disjunctions. CPLEX 1
M corresponds to V in our model.
 W.-Y. Ku, J.C. Beck / Computers & Operations Research 73 (2016) 165–173 171

Table 3
Comparison of the disjunctive formulations using indicator constraints (using CPLEX). Bold numbers indicate the best formulation for each solver for the given problem size.
The symbol ‘-’ means that none of the 10 problem instances were solved to optimality within 3600 s.

Problem Disjunctive Indicator (OR) Indicator (IF)

Time (geo/ Opt Time (geo/ Opt Time (geo/ Opt

arith) arith) arith)

3×3 0.01/0.01 10 0.01/0.01 10 0.00/0.00 10

4×3 0.01/0.01 10 0.01/0.01 10 0.01/0.01 10
5×3 0.02/0.02 10 0.02/0.02 10 0.01/0.01 10
3×6 0.01/0.01 10 0.01/0.01 10 0.01/0.01 10
3×8 0.01/0.01 10 0.01/0.01 10 0.01/0.01 10
3 × 10 0.01/0.01 10 0.01/0.01 10 0.01/0.01 10
5×5 0.02/0.02 10 0.03/0.03 10 0.04/0.04 10
8×8 0.60/0.61 10 1.38/1.39 10 1.16/1.17 10
10 × 10 3.19/3.48 10 8.97/10.99 10 7.64/9.65 10
12 × 12 99.43/212.76 10 274.24/779.03 10 273.36/919.04 8
15 × 15 1568.89/ 5 3593.73/ 1 - -
2272.75 3594.47

Table 4
Comparison of the best MIP model with multi-threading and parameter tuning (using CPLEX) and the CP model. Bold numbers indicate the best model for the given problem
size. The symbol ‘-’ means that none of the 10 problem instances were solved to optimality within 3600 s.

Problem Disjunctive Disjunctive (multi) Disjunctive (multi þ tune) CP

Time (geo/arith) Opt Time (geo/arith) Opt Time (geo/arith) Opt Time (geo/arith) Opt

3×3 0.01/0.01 10 0.02/0.02 10 0.02/0.02 10 0.00/0.00 10

4×3 0.01/0.01 10 0.03/0.03 10 0.13/0.13 10 0.00/0.00 10
5×3 0.02/0.02 10 0.07/0.07 10 0.03/0.03 10 0.00/0.00 10
3×6 0.01/0.01 10 0.04/0.04 10 0.02/0.02 10 0.00/0.00 10
3×8 0.01/0.01 10 0.03/0.03 10 0.07/0.07 10 0.00/0.00 10
3 × 10 0.01/0.01 10 0.04/0.04 10 0.07/0.07 10 0.00/0.00 10
5×5 0.02/0.02 10 0.04/0.04 10 0.07/0.08 10 0.00/0.00 10
8×8 0.60/0.61 10 0.30/0.30 10 0.32/0.32 10 0.15/0.15 10
10 × 10 3.19/3.48 10 1.65/1.70 10 1.72/1.76 10 0.67/0.68 10
12 × 12 99.43/212.76 10 37.51/71.74 10 33.19/47.67 10 3.58/3.88 10
15 × 15 1568.89/2272.75 5 865.29/1595.79 8 926.83/1920.19 7 39.57/47.71 10
20 × 15 - - - - - - 1037.81/1924.35 6

but however biases the results in favor of the tuned parameters. Table 4 shows that both MIP and CP solve the problems very
We did this as we were seeking to evaluate the maximum im- quickly up to size 10  10. MIP still solves the 12  12 problems in a
provement we could expect from tuning. reasonable amount of time but only solves 80% of the 15  15
Table 4 shows that CPLEX's tuning tool reduces the mean problems, while CP is able to prove optimality for all the 15  15
running time for the 12 × 12 instances by a factor of 1.5 on top of problems. As problems become even bigger, CP is still able to solve
multi-threading. For larger instances, Fig. 7b shows that the MRE is 60% of the 20  15 problems.
reduced by about 30%. However, for the Taillard's instances, the For additional comparison, Fig. 7(a) and b presents results of
tuning tool does not find parameter settings different from the the 20  20 problems and Taillard's 20  15 problem instances
default. Therefore, the performance remains unchanged. (ta11 through ta20). Both figures show that CP finds better feasible
Though both multi-threading and parameter tuning are effec- solutions much faster than the MIP model, but the solutions im-
tive techniques, our results indicate that neither, even used to- prove only marginally with additional run time, plateauing around
gether, is likely to make a significant portion of the non-solvable 500 s. The MIP model performs worse than CP throughout the
problems solvable. time horizon, though the solutions tend to improve steadily.
However, judging from the flattening of the MIP curves, it seems
4.6. Comparison of MIP and CP unlikely that the MIP model will achieve performance equal to CP
without an impractical run time. Additional comparison on the
As another commonly used approach for solving scheduling Taillard's 20  15 problem instances show similar trends with
problems, Constraint Programming (CP) has proven to be suc- higher MRE for both approaches.
cessful over the last decade [14,15]. Constraint-based solving In summary, our comparison of MIP and CP for the job shop
techniques allow CP to outperform MIP in a variety of scheduling scheduling problem indicates that for medium size problems (i.e.,
problems [14]. In this section we compare the best MIP results around size 12 × 12) both MIP and CP can find and prove optim-
using CPLEX (best of the three MIP variations for each individual ality under a reasonable time limit. However, for larger problems,
problem instance in Table 4) with the CP model using one thread CP dominates MIP regardless of the time limit.
only.2 The complete CP model is presented in the Appendix.
4.7. Comparing to the state of the art
2
Using all the 8 threads actually increases the running time of CP by about a
factor of 1.6. One argument for using MIP (or CP) models for scheduling as
172 W.-Y. Ku, J.C. Beck / Computers & Operations Research 73 (2016) 165–173

Fig. 7. MRE Performance Over Time of the MIP Models, CP Model and iSTS-SGS. Note that for the ta11–ta20 problems, the tuning tool does not find parameter settings
different from the default. Therefore we do not plot the curve for multi þ tune, (a)20  20 problems, (b)ta11–ta20.

opposed to hand-crafted, specialized algorithms is that the “model parameter tuning. Comparing the best MIP results with that of CP,
and solve” paradigm is less time consuming for users, requires less results show that MIP performs similarly to CP for smaller problems
knowledge of specialized optimization algorithms, and is more in terms of proving optimality. However, CP dominates MIP for lar-
flexible in that new constraints can be easily added to existing ger problems both in terms of proving optimality and solution
models.3 The cost of general models, however, is that modeling quality. Finally, we demonstrate that if the goal is to find a high
itself can be challenging and time consuming and the resultant quality solution within a given time-limit for large problems, the off-
performance is unlikely to match the state-of-the-art specialized the-shelf CP model is highly competitive with the state-of-the-art.
algorithms.
While it is difficult to measure the effort of modeling, we can
compare the performance of standard models with specialized Appendix A. CP model
algorithms. Here, we compare the disjunctive MIP and CP models
used above with, iSTS-SGS, a state-of-the-art exact algorithm for Using the same decision variables as the disjunctive MIP model
JSP [16]. iSTS-SGS is a two-phase hybridization of a JSP-specific (3.1), the CP model can be stated in Fig. A.8.
tabu search (iSTS) with a specialized CP search (SGS). For SGS, we The objective function is stated in (A.1). Constraint (A.2) en-
perform experiments with ILOG Scheduler 6.5, now out-dated sures that the start time of each job is greater or equal to 0.
software. Constraint (A.3) is the precedence constraint. It ensures that all
On the randomly generated 20 × 20 instances, Fig. 7(a) shows
operations of a job are executed in order. Constraint (A.4) ensures
that MIP finds solutions on average only 2.3% worse than the iSTS-
that the makespan is at least the largest completion time of the
SGS, depending on the time limit. The CP model achieves solutions
last operation of all jobs. Constraint (A.5) is a global constraint
within about 0.6% of the iSTS-SGS. On the 20 × 15 instances from
Taillard's benchmark set, Fig. 7(b) shows a larger difference be-
tween iSTS-SGS and MIP at about 6.2%, while CP still maintains a
relatively small gap at about 0.5%.

5. Conclusion
Fig. A.8. CP model.
In this paper, we evaluated four MIP models for the classical job
shop scheduling problems and compared their performance using which ensures that no two jobs can be scheduled on the same
CPLEX, GUROBI and SCIP. The results of CPLEX and GUROBI de- machine at the same time.
monstrated that, first, contrary to the claim of Liao [8], Manne's
original disjunctive model is the best performing MIP model for the
JSP. In addition, in partial contradiction to Pan's claims [6], the time-
indexed model is able to outperform the rank-based model for References
smaller problem instances. In contrast, experimental results using
SCIP showed that the rank-based model outperforms the time-in- [1] Koch T, Achterberg T, Andersen E, Bastert O, Berthold T, Bixby R, Danna E,
Gamrath G, Gleixner A, Heinz S, et al., MIPLIB 2010, Mathematical Program-
dexed model for small problems, while Liao's formulation is trans- ming Computation; 2011. p. 1–61.
formed to Manne's in presolving so similar performance is observed. [2] Bowman EH. The schedule-sequencing problem. Oper Res 1959:621–4.
Second, due to the advances of modern MIP solvers, we show that [3] Kondili E, Pantelides C, Sargent R. A general algorithm for scheduling batch
operations. In: PSE'88: third international symposium on process systems
MIP is now able to solve the JSP with moderate size very quickly. We
engineering: in affiliation with CHEMECA 88, a Bicentennial Event: Sydney,
also demonstrate the effectiveness of multi-threading and Australia, 28 August–2 September; 1998. Preprints of Papers, Institution of
Engineers, Australia; 1988. p. 62.
[4] Wagner HM. An integer linear-programming model for machine scheduling.
3
Though unfortunately often the performance of the modified models is not Naval Res Logist Q 1959;6(2):131–40.
easily predictable. [5] Manne AS. On the job-shop scheduling problem. Oper Res 1960:219–23.
 W.-Y. Ku, J.C. Beck / Computers & Operations Research 73 (2016) 165–173 173

[6] Pan C-H. A study of integer programming formulations for scheduling pro- search for job-shop scheduling. INFORMS J Comput 2011;23(1):1–14.
blems. Int J Syst Sci 1997;28(1):33–41. [17] Garey M, Johnson D. Computers and intractability.San Francisco, CA: Freeman;
[7] Applegate D, Cook W. A computational study of the job-shop scheduling 1979.
problem. ORSA J Comput 1991;3(2):149–56. [18] Ronconi D, Birgin E. Mixed-integer programming models for flowshop sche-
[8] Liao C, You C. An improved formulation for the job-shop scheduling problem. J duling problems minimizing the total earliness and tardiness. Just-in-Time
Oper Res Soc 1992:1047–54. Syst 2012:91–105.
[9] IBM ILOG CPLEX optimization studio. 〈http://www-01.ibm.com/software/com [19] Adams J, Balas E, Zawack D. The shifting bottleneck procedure for job shop
merce/optimization/cplex-optimizer/index.html〉 [accessed 20.08.15]. scheduling. Manag Sci 1988;34(3):391–401.
[10] Gurobi optimization. 〈http://www.gurobi.com〉 [accessed 20.08.15]. [20] Nowicki E, Smutnicki C. An advanced tabu search algorithm for the job shop
[11] Solving constraint integer programs. 〈http://scip.zib.de〉 [accessed 20.08.15]. problem. J Sched 2005;8(2):145–59.
[12] Mittelmann H. Mixed integer linear programming benchmark. 〈http://plato. [21] Della Croce F, Tadei R, Volta G. A genetic algorithm for the job shop problem.
asu.edu/ftp/milpc.html〉 [accessed 20.08.15]. Comput Oper Res 1995;22(1):15–24.
[13] Fischetti M, Monaci M. Exploiting erraticism in search. Oper Res 2014;62 [22] Beck JC. Solution-guided multi-point constructive search for job shop sche-
(1):114–22. duling. J Artif Intell Res 2007:49–77.
[14] Baptiste P, Le Pape C, Nuijten W. Constraint-based scheduling: applying con- [23] Taillard E. Taillard's instances, job shop scheduling problem. 〈http://optimizi
straint programming to scheduling problems, vol. 39. Germany: Springer; zer.com/TA.php〉 [accessed 20.08.15].
2001. [24] Taillard E. Benchmarks for basic scheduling problems. Eur J Oper Res 1993;64
[15] Barták R, Salido M, Rossi F. New trends in constraint satisfaction, planning, and (2):278–85.
scheduling: a survey. Knowl Eng Rev 2010;25(03):249–79. [25] Achterberg T. Conflict analysis in mixed integer programming. Discret Optim
[16] Beck JC, Feng T, Watson JP. Combining constraint programming and local 2007;4(1):4–20.