Computers & Operations Research 28 (2001) 955}971

A new adaptive neural network and heuristics hybrid

approach for job-shop scheduling6
Shengxiang Yangª›*, Dingwei Wangª
ªDepartment of Computer Science, King's College London, University of London, London, WC2R 2LS, UK
ªDepartment of Systems Engineering, Northeastern University, Shenyang 110006, People's Republic of China
Received 1 May 1998; received in revised form 1 April 1999

Abstract

A new adaptive neural network and heuristics hybrid approach for job-shop schedulingis presented. The
neural network has the property of adapting its connection weights and biases of neural units while solving
the feasible solution. Two heuristics are presented, which can be combined with the neural network. One
heuristic is used to accelerate the solving process of the neural network and guarantee its convergence, the
other heuristic is used to obtain non-delay schedules from the feasible solutions gained by the neural
network. Computer simulations have shown that the proposed hybrid approach is of high speed and
e$ciency. The strategy for solving practical job-shop scheduling problems is provided. 2001 Elsevier
Science Ltd. All rights reserved.

Scope and purpose

Job-shop scheduling is usually a strongly NP-complete problem of combinatorial optimization problems

and is the most typical one of the production schedulingproblems. It is usually very hard to "nd its optimal
solution. Practically researchers turn to search its near-optimal solutions with all kind of heuristic algo-
rithms. The scope of this paper is to present a new hybrid approach in dealingwith this job-shop scheduling
problem based on adaptive neural network and heuristics.

Keywords: Job-shop scheduling; Adaptive neural network; Heuristics

6
This research was supported by the National Nature Science Foundation (No. 69684005) and National High-Tech
Program of People's Republic of China (No. 863-511-9609-003) and was done when Shengxiang Yang was pursuing
his Ph.D. degree.
* Corresponding author. Tel.: #44-207 848 2009; fax: #44-207 848 2851.
E-mail address: [email protected] (S. Yang)

0305-0548/01/$ - see front matter 2001 Elsevier Science Ltd. All rights reserved.
PII: S 0 3 0 5 - 0 5 4 8 ( 0 0 ) 0 0 0 1 8 - 6
1. Introduction

It is well known, the job-shop schedulingproblem is the most complicated and typical problem
of all kinds of production scheduling problems, the allocation of resources over time to perform
a collection of tasks [1]. Job-shop schedulingcan be stated as follows [2]: given n jobs that have
to be processed on m machines in a prescribed order under certain restrictive assumptions, the
objective is to decide how to arrange the processing orders and starting times of operations
sharing the same machine for each machine, in order to optimize certain criteria. Manufacturing
systems with di!erent objectives require di!erent optimization criteria [3], such as stock size, due-
date reliability, mean lead time and makespan.
Traditionally, there are three kinds of approaches to solve job-shop scheduling problems:
priority rules, combinatorial optimization and constraints analysis [4]. The "rst kind of method
has the merit of beingcomputationally very e$cient and easy to be applied to real cases, but there
is no guarantee with respect to the quality of the obtained solution. Especially if some temporary
constraints should be respected [5]. The optimization methods are much more rigorous but are not
tractable in large size problems if the optimal solution is required [6]. The third method,
originated from Erschler et al. [7], looks for a set of feasible solutions that meet several
technological constraints for the user to choose the "nal solution.
It has been demonstrated [8] that job-shop schedulingis usually an NP-complete (nondetermin-
istic polynomial time complete) problem. Because of the NP-complete characteristics of job-shop
scheduling, it is usually very hard to "nd its optimal solution, and an optimal solution in the
mathematical sense is not always necessary in practices [6]. Researchers turned to search its near-
optimal solutions with all kind of heuristic algorithms [9]. Fortunately, the searched near- optimal
solutions usually meet requirements of practical problems very well. Recently, several knowledge-
based scheduling systems have been presented [10,11], which are much general than the above
traditional methods because of its using constraints systematically, its implementing heuristic
knowledge and its generality as a framework for stating and solving combinatorial optimization
problems.
Since Hop"eld [12] "rst used a neural network to solve an optimization problem, Hop"eld
networks have been successfully applied to solving a variety of problems, such as the analog-to-
digital conversation problem [13], the traveling-salesman problem [14], the combinatorial optim-
ization problem [15], the linear and non-linear programming problems [16]. However, Hop"eld
networks have the drawbacks of non-convergence to valid solutions, inability to locate the global
minimum and poor scaling properties due to the use of quadratic energy functions, as pointed out
by DARPA [17]. Since Foo and Takefuji [18,19] "rst used neural networks to solve job-shop
scheduling problems, several neural network architectures have been presented to solve job-shop
scheduling (see e.g., Foo and Takefuji [20], Foo et al. [21], Zhou et al. [22] and Willems and
Brandts [23]). All the above-mentioned neural networks are basicaly non-adaptive networks with
the connection weights and biases prescribed in advance before the networks begin to work.
In Yang and Wang [24] we have proposed an e$cient constraint satisfaction adaptive neural
network (CSANN) and heuristics combined approach for job-shop schedulingproblems. CSANN
di!ers itself from the above-mentioned networks in its adaptivity. CSANN has the property of
adaptively adjusting its weights of connections and biases of neural units according to the actual
constraint violations during its processing to remove these violations for obtaining feasible
 S. Yang, D. Wang / Computers & Operations Research 28 (2001) 955}971 957

solutions. In order to improve the performance of CSANN several heuristics are presented in
Yang and Wang [24].
In this paper we present a new heuristic based on the property of non-delay schedules. This
new heuristic together with one of the heuristics presented in Yang and Wang [24] can be
combined with CSANN to form a new hybrid approach for job-shop scheduling problems. In
the hybrid approach, CSANN is used to obtain feasible solutions, the heuristics from
Yangand Wang[24] is used to accelerate the solving process of CSANN and guarantee
feasible solutions, the new heuristic is used to obtain the non-delay solution from the feasible
solution obtained by CSANN with determined orders of operations. The new hybrid approach
presented in this paper is simpler and equivalently e$cient (see e.g. Yang and Wang [24]).
The computational simulations have shown that the proposed hybrid approach has good
performance with respect to the quality of solution and the speed of computation.
This paper is organized as follows. Section 2 presents a mathematical formulation of the job-
shop scheduling problem. The model of CSANN is presented in Section 3. In Section 4 the
heuristics used are described, the hybrid approach is also described in this section. Section
5 presents the computer simulation results with two examples to show the performance of the
proposed new hybrid approach for job-shop scheduling. Finally, the conclusions about the hybrid
approach are presented in Section 6.

2. Formulation of the job-shop scheduling problem

Generally for the job-shop scheduling problem there are two types of constraints: sequence
constraint and resource constraint. The "rst type states that two operations of a job cannot be
processed at the same time. The second type states that no more than one job can be handled on
a machine at the same time. Job-shop scheduling can be viewed as an optimization problem,
bounded by both sequence and resource constraints. For a job-shop schedulingproblem, each job
may consist of di!erent number of operations, subjected to some precedence restrictions. Com-
monly the processingorders of each job by all machines and the processing time of each
operation are known and "xed. Once started operations cannot be interrupted (non-preemption).
This kind of scheduling is usually called deterministic and static scheduling. In this paper we
consider the deterministic and static job-shop scheduling problem.
Denote N"{1,2, n} and M"{1,2, m} as the job set and the machine set, where n and m
are the numbers of jobs and machines. Let n be the operation number of job i. O
fi fi’R
represents
operation k of job i to be processed on machine q, ¹ and P represent the starting time and
fi’R fi’R
processing time (which is known in advance) of O
fi’R , respectively, fiªfi and P fiªfi represent the
¹ R R
starting time and processing time of the last operation of job i, respectively. Denote r and d as
the release date (earliest starting time) and due date (latest ending time) of job i. Let S fidenote fithe
fi
set of operation pairs [O , O ] with precedence restriction of job i, where operation O must
fi’P fi’R fi’P
precede operation O . Let R be the set of operations O that will be processed on machine q.
fi’R R fi’R
Commonly, the starting time and the processing time of an operation are assumed to be integers.
We use the pure integer representation model to transfer the sequence constraints, resource
constraints, the release date and due date constraints of jobs into integer linear inequalities.
Taking minimizing the makespan as the optimization criterion, the mathematical
formulation of the
job-shop scheduling problem considered is presented as follows:

Minimize E"Max (¹ # )
fiªfi P fiªfi R
subject to fi²’

¹ !¹ *P , [O , O ]3S , k, l3{1,2, n }, i3N, (1)

fi’R fi’P fi’P fi’P fi’R fi fi
¹ !¹ *P or ¹ !¹ *P , O , O 3R , i, j3N, q3M, (2)
fi’R fi’R fi’R fi’R fi’R fi’R fi’R fi’R R
r )¹ )d !P , i3N, j3{1,2, n }, q3M, (3)
fi fifiR fi fifiR fi
where the cost function is the endingtime of the latest operation, i.e., maximal complete time of
the job-shop scheduling problem. Minimizing the cost function means minimizing the makespan.
Eq. (1) represents the sequence constraint; Eq. (2), in a disjunctive type, represents resource
constraints; Eq. (3) represents the release date and due date constraints.
For an n/m/J/C (notation system of Conway [2]) problem, there are at most n(m!1)
sequence constraint ª’ inequalities of Eq. (1) type, at most mn(n!1) resource constraint inequalities
of Eq. (2) type, at most mn starting time constraint inequalities of Eq. (3) type, resulting in a total
number of at most n(mn#m!1) constraint inequalities. There are also at most mn number of
variables ¹ s. The objective of job-shop scheduling is to solve these variables so that they satisfy
fi’R
all the constraint inequalities while minimizing the makespan.

3. Model of CSANN

To solve the job-shop schedulingproblem, the previous pure integer representation model has to
be mapped onto the CSANN. The proposed CSANN will be discussed in detail with respect to its
basic components of units and connections, its architecture and its solving process for job-shop
scheduling.

3.1. Neural units of CSANN

Generally a neural unit consists of a linear summator and a nonlinear activation function which are
serialized [25] (see e.g., Fig. 1). The summator of unit i receives all activations A (j"1,2, n)
fi
from connected units and sums the received activations, weighted with connection weight = ,
fifi
together with a bias B . The output of summator is the net input N , this net input N is passed
through an activation fifunction f (.), resulting in the activation A of unit
fi i. The summator
fi and the
fi
activation function are de"ned as follows:
’
A "f (N )"f Σ (
)
fi fi fi"¹ (= ×A )#B , (4)
fifi fi fi
where = is the connection weight from unit j to unit i.
Usually, fifi for neural units to perform di!erent functional behaviors, there are several types of
activation functions, such as linear threshold function, linear-segmented function and S-shaped
 Fig. 1. General neural unit model.

Fig. 2. Linear-segmented activation functions.

function [26]. In this paper two kinds of linear-segmented function A and B (see e.g., Fig. 2(a)
and (b)) are used as the activation functions of neural units.
Based on the general neural unit, CSANN contains three kinds of units: ST-units, SC-units and
RC-units. The "rst kind of units represent the starting times of all operations. Each ST-unit
represents one operation of job-shop scheduling problem with its activation representing the
startingtime of the operation. The second represents whether the sequence constraints are
violated. The third represents whether the resource constraints are violated.
The net input of an ST-unit (e.g., S¹ ) is calculated by
fi

N’Tfi (t)"Σ ×A (t))#Σ (= ×A (t))# (t!1), (5)

(= fi ’‘f fi’ ª ’T
fifi i ’ A fi
‘’
where the net input of unit S¹ is summed from three parts. The "rst part comes from the
weighted activations of SC-unitsfi connected with S¹ , which implements feedback adjustments
fi
because of sequence violations. The second part comes from the weighted activations of RC-units
connected
with S¹ , implementing feedback adjustments because of resource violations. The third part
fi the previous activation, with weight being #1, of unit S¹ itself.
comes from
fi
The activation function of ST-units is deterministic linear-segmented function of type B (as
shown in Fig. 2(b)) and is de"ned as follows:

’Tfi (t)(r ,
fi fi
(t), r )
A’Tf (t)" r ,’Tf

{
f ’T
(t)'d(t))d f ! ’Tf
!P , (6)
N f
N d! , N P ,
P
N
fi ’Tfi ’T fi
fi ’Tfi