Available online at www.sciencedirect.

com

Applied Mathematics and Computation 195 (2008) 6675

www.elsevier.com/locate/amc

Amodel
dynamic
BP neural
networks
andall
its parameters
applicationadaptive
on oil reservoir
prediction
Shiwei Yu *, Kejun Zhu, Fengqin Diao

School of Management, China University of Geosciences, Wuhan, Hubei 430074, PR China

Abstract
In this paper,
a dynamic
allerror
parameters
adaptive BPneural
neuralnetwork
networks
model is
fusing of
genetic
algorithms
(GAs),
simulated
annealing
(SA) and
back propagation
(BPNN)
toproposed
oset
the by
demerits
one paradigmweights
by
the
merits
of of
another.
Adopting
multi-encoding,
the model Under
can optimize
the
input nodes,
hidden
nodes, transfer
function,
and
bias
BP networks
dynamically
and
adaptively.
accurate
premise,
the simple
architecture
(less
input
and
hidden
nodes)
of
network
model
is
constructed
in
order
to
improve
networks
adaptation
and
generalization
ability,
and
to
greatly
reduce
the
subjective
choice
of
structural
parameters.
The
results
of
application
on
oil
reservoir
prediction
show
that
proposed model with comparatively simple structure can meet the precision request and enhance the generalization ability. the
2007 Elsevier Inc. All rights reserved.
Keywords: Dynamic; All parameters; Adaptive; Genetic algorithms; BP neural network; Structure identication

1. Introduction
Articial neural
networks (ANNs),
due proved
to theirtoexcellent
ability of non-linear
mapping, generalization,
selforganization
andnew
self-learning,
have
be
of widespread
utility
in engineering
and are
steadily
advancing
into
areas
[15].
Thebeen
feed-forward,
back
propagation
neural
network usually
trained
by
back-propagation
of
errors
(BPNN)
is
perhaps
the
most
popular
network
architecture
in
use
today
[6].
Although
encouraging
results have
been
reportedreports
in which
ANNs-based
systemthe
outperformed
widely-used
wellestablished
statistical
methods,
many
inconsistent
have
been
undermining
robustness
of
these

ndings.
The
reasons
of
these
discrepancies
are
well-known
problems
characterized
BPNN:
(1)
As
the
BP
algorithm
optimizesthe
a target
function
by using
the gradient
the calculation
may
overow
ornods
uctuate
between
optima;
BPNN
is sensitive
to thedescent
choice ismethod,
of topology
and
size.
ere
nt datahidden
transfer
function
may (2)
lead
to di
erent
results
and If
tested
on Di
thesets
same
base;
(3) and
The
convergence
of BPNN
is also
sensitive
to the when
initialBPNN
selection trained
of weights.
the initial
of
*

Corresponding author.
E-mail address: [email protected] (S. Yu).

0096-3003/$ - see front matter

doi:10.1016/j.amc.2007.04.088

2007 Elsevier Inc. All rights reserved.

S. Yu et al. / Applied Mathematics and Computation 195 (2008) 6675

67

weights
are not adjust
selected
properly,
optimization
could be
trapped
in a local optimum;
Gener- ally,
BP
algorithms
weights
andthe
bias
complying solution
with a certain
rule.
It is impossible
to adjust(4)
structure
selfadaptively
in a xed topology.
Despite
the
increasing
level
of
research
activity
the
problem
described
has
not
yet
been
solved
denitively.
Di

e
rent
approaches
can
be
categorized
as
follows:
(1)
empirical
or
statistical
methods,
that
are
used
to
study
the
eperformance
ect of ANNs
internal
parameters
andand
choose
appropriate
values for
themselves
based on
models
[710].
The[11].
most
systematic
general
of these
methods
utilize
the principles
fromtheTaguchis
design
of
experiments
The
best
combination
of
the
number
of
hidden
layers,
hidden
neurons,
choice
of
input
factors,
training(2)algorithm
parameters,
etc., [12].
can In
be the
identi
edof by
methods
evenanalysis
thoughis they
mostly
case-oriented.
Grey
correlation
analysis
case
thatthese
greyand
correlation
used are
to
determine
the
number
of
the
hidden
nods
in
the
BPNNs
optimal
network
to
improve
the
networks
performance.
(3) Network
pruning
algorithms
that,
respectively,
add and/or
remove neurons
an
initial
architecture
using agrowing/or
previously
specied
criterion
towhen
indicate
howis ANNs
performance
issquared
aefrom
ctederror
by
changes
The
rules
are
are
added
training
slow or
the mean
is
larger [1317].
than
aa change
specibasic
ed
value,
andthat
thatneurons
neurons
are
removed
when
a change
in
awhen
neurons
value
does
not
correspond
to
in
the
networks
response
or
when
the
weight
values
that
are
associated
with
this
neuron
remain
constant
for
a large
numbertheir
of training
epochs.
Since
bothminimum
constructive
andbepruning
algorithms
are
basically
gradient
descent
methods,
convergence
the
global
cannot
Therefore
they
can be
trapped
to that
a local
minimum
close to space
the to
point
of
the search
space
where
theguaranteed.
algorithm
starts.
(4)
Evolutionary
strategies
search
over
topology
by
varying
the
number
of
hidden
layers
and
hidden
neurons
through
application
of genetic
operators
and evaluation
of
the
dimade
erent in
architectures
according to
an
objective
function
[1721].
In
recent
years,
the
considerable
progress
has
been
last
category.
However,
most
of
previous
researchers
hybridized two of evolutionary techniques such as GABP, SABP, GASA. Little has
fused
GAs, SA,
and BP synchronously.
In this paper,
a simulated
dynamic all
parameters
BP back
neuralpropagation
networks model
is proposed
by fusingtogenetic
algorithms
annealing
(SA)adaptive
andTheerror
neural
network
(BPNN)
oset
demerits
of (GAs),
one
paradigm
by
merits
of
another.
algorithms
whose
every
chromosome
structures
consist
of
both
binary
and
real
parts
are
brie

y
described
as
following:
Firstly,
the
model
optimizes
input
nodes,
hidden
nodes,
transfer
function,
weights
and
bias
of
BP
networks
utilizing
GA.
Secondly,
in
order
to
get
proper
weights
and
bias
of
a

xed
network
structure,
the
real
part
of
chromosome
corresponding
weights
and
bias
value
go
to
SABP
hybrid algorithms
with in
a certain
probability.
The adjusted
better weights
and
bias canFinally,
be got set
by weights
using SA
global
search
avoiding
being
trapped
a local
minimum and
when
by BP
algorithm.
and
bias
obtained
by accuracy,
SABP
to the
corresponding
chromosome,
then go
to next
generation
operation.
Under
thenodes)
premise
of
ensuring
architecture
of
the
network
model
is
relatively
simple
(less
input
and
hidden
that to improve
adaptation and generalization ability of networks, and to greatly reduce subjective choice so
of
structural
parameters.
The paper is organized as follows. In Section 2, we describe design of the model in detail. The algorithm
approach of adaptive optimize parameters of BPNN is described in Section 3. In Section 4, applications
are presented and the performances of the various existing methods are compared. We conclude in Section 5.
2. Design of the model
GA is a global
search
procedure space,
that searches
from
population
of
another.
As the so
algorithm
continuously
the parameter
the search
is one
directed
toward
thepoints
area
oftothe
bestdisolution
far. This
algorithm
hassamples
beenapplication
shown
to perform
exceedingly
well
in obtaining
global
solutions
for

cuANN,
lt non-linear
functions
[4].
The
of
the
GA
to
one
particularly
complex
non-linear
function,
the
has disalso
been
shown
to
dominate
other
more
commonly
used
search
algorithms
[19,20,22].
It
has
some
main
advantages, for
example,
slowforconvergence
speed;
premature problems.
convergence
oras stalling;
theimplies,
larger exploits
number an
of
populations
and
individuals
some
complex
optimization
SA,
its
name
analogy
between
the
whichfora ametal
coolsinand
freezes
into system.
a minimum
energy crystalline
structure (the
annealing
process)
andway
thein
search
minimum
aasmore
general
Theequilibrium
algorithm
iscon
based
Kirkpatrick
et al. [23],
which
was originally
proposed
a means
of nding the
gu-upon that of

68

S. Yu et al. / Applied Mathematics and Computation 195 (2008) 6675

Fig. 1. A typical back-propagation ANNs model.

ration
of becoming
a collection
of atoms
at a given
temperature.
SAs employs
major advantage
otherwhich
methods
is thebetter
abilitybut
if
avoiding
trapped
at local
minima.
The probability.
algorithm
a powerful
randomover
search,
not
only
also
worse
neighboring
solutions
with
a certain
SA is aintensive,
optimization
technique
to
solve
the
combinatorial
optimization
problems,
but
it
is
computationally
especially
for
more
complicated
problems
[24].
BP algorithm
which is mostly
to
is a gradient-based
method; hence
some
problems
arestuck
frequently
encountered
inetc.
theused
useTo
oftrain
this networks
algorithm,
very slow convergence
speed
in inherent
training,
easily
toachieve
get
in ealocal
minimum,
[25].
overcome
these e.g.,
disadvantages,
combining
GA, SA
and
BP
could
more
ect
ive
results.
This
study
employs
genetic
algorithm
to
carry
out
near-optimal
neural
network structure, and SA, BP to realize optimal weights and bias of the network.
The
common
three-layer
feed-forward
type
of
ANNs,
as
shown
in
Fig.
1,
is
considered.
This
is
the
type
of
network
in which the units each perform a biased weighted sum of their inputs and pass this activation level
through
topology.a transfer function to produce their output, and the units are arranged in a layered feed-forward
and
parameters
of BP neural networks by a hybrid algorithm of GASA BP;
rstTo
ofdynamic
all, design
asadaptive
followingoptimize
must beall
done
in this section.
2.1. Chromosome multi-encoding
Inwith
order
to optimize BP
neural network
structure,
function,
weights
and
biasbinary
at theand
samereal
time,
we used
GA
multi-encoding.
structures
withtransfer
sixX
gene
both
parts,
it
1
2
4
2
is
demonstrated
inlayer,
Fig.as
2Chromosome
and as
presented
as chromsom

;neurons
Xsegments
; X 3 ;are
X include
; W 1 ; Wby
. Neurons
of of
net-chromosomes
work
inas
the
input
and
hidden
well
1the transfer
2
function
type
of
coded
the

rst
part
structure.
If
the
value
of
gene
X
and
X
,
coded
in
binary
system,
is
1,
the
neuron
is
selected,
on
the
contrary,
if the string
value is
0,
Forthe
example,
there are 10 inputting vari- ables, the
masking
binary
encoded
as inputs
[1the
1 1 neuron
1should
1 1 1 0is0beunselected.
0]removed.
presents that
rst
4 seven inputs of feature work piece parts
should
be
kept, and
the
lastand
three
X3 and
X
are the code
genesbinary
of determitransfer
function
of
hidden
output
layer
neurons
respectively.
For
example,
stringnation
as 0 0the
0, 0type
0 1,of0
1 1 representing the transfer function of hanrdlim, log sig and tan sig respectively. Those

binary part

real part
Weights and bias
of hidden layer

X1

X2

Input and hidden layer nods

Selection
0: unselected; 1: selected

X3

X4

Transfer function of hidden

and output layer nods
Fig. 2. GANN chromosome structure.

Weights and bias

of hidden layer

W
R: [-10 , 10]

S. Yu et al. / Applied Mathematics and Computation 195 (2008) 6675

69

genes
the real part, W1 and W2 are use to code the weights and bias parameters which may be in
valueslocated
of [ 10,in10].
2.2. Selection operator
Generally,
operator ofalgorithm
genetic algorithm
is implemented
using roulette-wheel
algorithm.while
The
main
defect
ofselection
the
roulette-wheel
is that emerging.
local
optimal
gene by
dominates
thewheel
whole
generation,
global
optimum
would
likely
be eliminated
before
Therefore,
the roulette
with
elitist
selection
method
is
considered
as
the
selection
mechanism
in
this
proposed
GASABP
algorithm.
This
method
including
two phases,
at new
rst, generation.
the elitist strategy
wasrest
adopted.
The generation
whose
tness is within
copied
to the
Then the
of chromosome
is selected
byroulette
wheel. top 10% will be directly
2.3. Crossover operator
Crossover is abinary
mechanism
of randomly
exchanging information
between two chromosomes. Dierent cross- overs
mechanisms
and real
partwhich
are implemented
in this
to the binaryfor
part of chromosome,
is demonstrated
instudy.
Fig. 3.The two-point crossover mechanism is applied
The
following
crossover
for
real
part:
Let
chrom1
and
chrom2
are
two chromosomes,
distributed random variables with range [0, 1]. We can get two new chromosome
by Eq.and
(1): are two indepen- dently
0
chrom1 chrom1 1 r1 chrom2 chrom1;
0
1
chrom2 chrom2 1 r2 chrom1 chrom2:
2.4. Mutation operator design
The purpose
of
mutation
operation
is to make
genetic algorithm
obtainforlocal
random
research
through
varying
certain
genesvalues.
of chromosome.
If a of
chromosome
is selected
mutation,
one
gene is capability
randomly
selected
for
changing
their
Initsthe
respect
representations
the
ispart
particularly
simple:
randomly
choose
a locus
and
switch
value,
eitheris,0binary
# 1p or
1#
0. While,
forprocess
the real
ofvariables
the chromosome,
let
p
is
a
parent,
then
the
child
chromosome
p0

where
is
a
distributed
random
with
range [0, 1].
2.5. Fitness function
A tness value is given by Eq. (2):
1
fitness
;
E a input no hide nod

where
E is the
squared
of neural network.
a (0 are
< a the
< 0.01)
is the of
aect
ion nods
coecand
ient hidden
of the number
of
input
nods
andmean
hidden
layererror
nods.
hide_nod
number
input
layer nods,
respectively.
input input_no,
nod
some, namely,Furthermore,
input_no P one
1, hide_nod
P and
1. one hidden layer nod must be selected at least in every chromoCrossover point
Parent 1

1011110000 111

Child 1

1011110101011

Parent 2

1111010101 001

Child 2

1111010000101

Fig. 3. Two-point crossover.

2.6. Adaptive probability of crossover and mutation

Traditional
crossovertheand
mutation
operators
areby
based
on a of
randomization
mechanism,
a cut
point,
and determining
position
of GA.
the
bit
shifted
mutation
theofsolution.
But
this
is noti.e.,
thegenerating
in natural
evaluation
which
isslightly
mimicked
byed
the
Actually
renewing
thecrossover
bits
theand
solution
is dynamic
orcase
adaptive,
but
not
random.
The
modi
adaptive
probabilities
of
mutation
given by
Srinivas
and
Patnaik
[26]
are
used
in
this
study
to
choose
the
probability
of
mutation
and
crossover
according
to
the

tness
value of the solutions. The modied expression of pc and pm are as follows:
fmax f 0 =fmax fave
if f 0 P favg ;
Pc
3
1:0
if f 0 < fave ;
0:5 fmax f =fmax fave if f P fave ;
4
P m
fave f =fave fmin
if f < fave :
Here,
f is and
the minimum
tness of an individual, fave the average tness value of
the population, and fmax and fmin the
0
maximum
solutions to
be crossed. tness value of the population respectively. f is the larger of the tness values of the
3. The algorithm approach of adaptive optimize parameters of BPNN
The algorithm
approach
of optimizing
the
network
structure,
transfer
and weights
value
using
GA
combining
SA and
include
two
phases:by
rstly,
theGA.
model
optimizes
thefunction
input
nodes,
hidden
nodes,
transfer
function,
weights
andBP
bias
ofreal
BPpart
networks
utilizing
Secondly,
inweights
order
to
get
proper
weights
and bias
of a

xed
network
structure,
the
of
chromosomes
corresponding
and
bias
value
go
to
SABP
hybrid
algorithms
withinaacertain
probability.
The
better by
weights
and bias The
can more
be gotdetails
by using
SA global assearch
avoiding
being trapped
local minimum
when
adjusted
BP algorithm.
are described
follows:
(1) Standardize the training and testing data.
(2) Initialize prior parameters of the model: the size of population = pop_size, the max generation max gen.
(3) Generate initial population chromsom X 1 ; X 2 ; X 3 ; X 4 ; W 1 ; W 2 , in which X 1 ; X 2 ; X 3 ; X 4 are binary
codes and W 1 ; W 2 are real number codes and the value of tness is calculated.
(4) The operation of relative genetic operators.
(4.1) Select and reproduce population according to value of tness.
(4.2) Calculate the self-adaptive crossover probability Pc, and crossover operation of dierent code genes.
(4.3) Calculate the self-adaptive crossover probability Pm, and mutation operation of dierent code gene.
(4.4) Estimate the number of 1s in the X1 gene code, if the number is 0, one of the genes set to 1 randomly.
The same principle is also used in the X2, else go to (5).
(5) Utilize SABP to get the optimal network weights basing on the xed input, structure, and transfer function, and let p = 1.
(5.1) If p 6 pop size, go to (5.2), else go to (6).
(5.2) chromsomp will be entered into SABP based on certain probability, which means if rand < P SABP , go to
(5.3), else p = p + 1, and go to (5.1).
(5.3) Set the initial annealing temperature t0, nal temperature ts, annealing ratio a and length of Markov L. If
t 6 ts the individual optimized by BP algorithm and can get chromsomp X 1 ; pX 2 ;pX 3 ;p X 4p; W 1p ; W 2p , else
let p = p + 1, go to step (5.2).
(5.4) Utilize SA to optimize the chromsomp X 1 ; pX 2 ;pX 3 ;p X 4p; W 1p ; W 2p globally, and get the individual optimized weights chromsomp X 1 ;pX 2p; X 3p; X 4p; W 1p ; W p2 . Let p = p + 1 go to (5.1).
(6) Calculate tness value of the population which optimized by GASABP from step (4) and (5),
chromsom X 1 ; X 2 ; X 3 ; X 4 ; W 1 ; W 2 .
(7) Let gen gen 1, if gen 6 max gne, go to next generation and implement a new GASABP operation
from the Step (4), else stop the iteration, get the best individual best chromsome,namely the best input
number, hidden nodes number, transfer function and the optimal network weights.

Standardize the sample data

Initialize population: pop_size, Maxgen, gen=0

Evaluate the fitness

Selection
Calculating P and crossover
c
Calculating P and mutation
m

Let X or X
to 1 randomly

The number of 1 in
2

1
X or X equal 0?

p=1
p pop _ size ?

p= p+1

rand < PSABP ?

N
Y
N

t < tS?
Y

BP algorithm :

SA algorithm:

1
2
3
4
chromsomp = [ Xp , Xp , Xp , X
,W
p
p

chromsom p = [ X p1 , Xp2 , Xp3 , Xp 4 ,Wp

,W
p

,Wp

chromsom = [ X 1 , X 2 , X 3 , X 4 ,W 1** ,W 2** ]

gen = gen + 1
N

gen max gne?

Y
Output the best _ chromsome
Fig. 4. The owchart of GASABP.

72

Seismic attribute

Oil well
G53

Average peak value

Average trough value
Max. amplitude
Mean amplitude
Min. amplitude
Sum of negative amplitude
Sum of positive amplitude
Arc length
Dominant frequency
Average energy
Energy half-time
Instantaneous frequency
Instantaneous phase
Average magnitude
Max. magnitude
Ratio of pos to neg sample
Sandstone thickness

0.07
0.28
0.13
0.1
0.31
20.11
3.55
1.15
116.21
0.03
0.02
91.25
0.04
1.24
2.85
46.15
2.2

G101
0.45
1.24
0.11
0.60
1.35
35.33
7.01
1.12
0.00
0.63
0.01
40.39
0.50
1.84
2.74
27.78
9.20

G104
0.33
1.07
0.33
0.26
1.07
24.04
12.17
1.05
36.13
0.31
0.02
37.32
0.12
1.45
2.42
66.67
6.80

G105
0.01
1.49
0.01
0.53
1.49
40.37
2.66
1.22
0.00
0.53
0.01
37.97
0.68
2.15
5.07
17.65
8.30

G108
0.01
1.59
0.01
0.60
1.59
29.81
2.85
1.08
0.00
0.70
0.01
37.54
0.23
1.48
3.11
37.50
6.80

G207
0.53
0.29
0.97
0.05
0.83
8.15
8.12
1.06
77.15
0.21
0.01
88.83
0.62
0.63
1.58
85.71
20.40

G406
0.11
0.80
0.00
0.40
1.05
33.17
2.10
1.21
0.00
0.25
0.01
45.39
0.90
1.68
3.93
10.53
2.20

G508
0.17
2.66
0.32
0.77
2.66
35.03
17.53
1.21
0.00
1.63
0.01
37.10
0.20
2.29
4.67
76.92
20.00

G43
0.03
0.48
0.00
0.32
0.89
24.57
0.78
1.08
0.00
0.20
0.01
35.19
0.44
1.27
2.45
11.11
11.20

G405
0.11
1.80
0.11
0.63
1.80
35.75
6.98
1.15
0.00
0.87
0.01
39.01
0.02
2.03
3.85
50.00
5.40

G34
0.12
0.89
0.12
0.32
0.89
21.12
7.41
1.08
0.00
0.23
0.02
40.15
0.57
1.24
2.70
53.33
12.40

G35
0.08
1.87
0.15
0.71
1.87
37.77
7.77
1.15
0.00
1.05
0.01
37.95
0.11
2.07
3.63
57.14
5.20

G36
0.03
0.98
0.03
0.45
1.64
26.90
5.47
1.16
0.00
0.50
0.02
42.88
0.57
1.35
3.13
26.32
24.00

G38
0.12
1.76
0.12
0.67
1.76
35.41
9.19
1.13
0.00
0.94
0.01
37.98
0.11
2.03
3.59
37.50
4.00

G23
0.02
1.51
0.02
0.58
1.51
38.24
2.82
1.10
0.00
0.63
0.01
38.67
0.19
2.05
3.73
17.65
11.60

S. Yu et al. / Applied Mathematics and Computation 195 (2008) 6675

Table 1
The samples and raw data

73

S. Yu et al. / Applied Mathematics and Computation 195 (2008) 6675

Thealgorithms
GASABP
model
is not a simple
combination
with the above
steps, but start
a mutual
integration.
Three
willNN
exert
their respective
merits.
Dynamic
allsolution
parameters
of GASABP
from many
initial
points
simultaneously
to
search
for random
a group
of nodes
in the
space,
construct
a constantly
evolving
population
serial,
and
combine
the
and
directional
searching
to
implement
winwin.
At
the
time
being,
individual
go
to
SAPB
based
on
certain
probability,
but
not
all
individuals
in
the
non-population.
This
could
both
assure precision
of the
network, in
andFig.
greatly
describing
approach
is presented
4. improve the operation speed of the model. A owchart of the
4. Application on oil reservoir prediction
In reservoir
the
application
of
lateral forecasting
technique
hasresults
become reservoir
a main
approach
in theengineering,
exploration
and
development
of reservoir
oil
and gas
It canand
obtain
believable
depth,
physical
properties,
utilizing
seismic
andeld.
log data,
tracking
thesignicant
horizontalon
distribution ofthickness,
reservoir. and
ANNs
approaches
commonly
provide
results
that
are statistically
more
than
conventional
methods
such
as multivariate
although
nature
of theArelationships
between
attributes
reservoirof
thickness
is more
dion
cultthe
toregression,
evaluateinformation
with
ANNsthe
results
simulation
study
to
predict
theand
thickness
oil reservoir
based
seismic
using
the[2729].
dynamic
all parameters
GA
SABP model.
4.1. Network input and sample
The major
tool
predict
the thicknessofofgeometric,
oil reservoir
is the seismic
information.
thanderived
50 seismic
attributes
which
areto
speci
c measurements
dynamic,
or statisticalMore
features
from
seismic
data
could
be
gotten.
all of
them have
kinematics,
nite
corresponding
relationship
with
thickness
of
reservoir;
obviously,
some
of
theNot
seismic
attributes
mayde
have
no
relationship
with
the
thickness
ofthe
reservoir
at
all,
which
results
in
repetition
and
redundancies
of
information.
The
in

nite
increase
of
the
number
of
attributes
brings
bad
e

ects
on
reservoir
prediction,
and
the
seismic
attributes
sensitive
to
targets
predicted
are
not
completely
identical in dierent work areas and dierent reservoirs. For that reason, seismic attributes have to be
optimized properly.
In this article,
we select an length
oil zone
which
to which
JiangHan
basin,
center
oforChina
extract
seismic
attributes
in
certain
thebelongs
line
167
15target,
wells
on
near. and
The
seismic15
attribute
values
will
used aswindow
inputs
sandattributes
thealong
thickness
of reservoir
willhave
be the
and
values
inputted
into
GA
SABP
to be
optimize
the
seismic
which
is wells
the appropriate
number
of
inputs.are
Table
1 wells
presents
samples
and
raw
data.
Randomly
select
12
of
the
15
as
the
sample
training
network,
and
three
as the
the
testing sample.
4.2. Comparative researches on network performance
According
model
design
in Section
2 andthethemax
procedures
algorithm
in Section
3, 20 random
iments
would to
be the
done,
when
the training
reaches
gen, andof
MSE
is 1.2323e
005. During
the 20 experTable 2
The parameters setting and the performance comparison of 20 randomly experiments with various existing methods
ANN algorithms

Geneticgeneration

pop size t0

Momentum with adaptive learning rate BP [29]a

GASA [30]a
10
50
50
GABP [30]a
10
50

SABP [31]a

50
GASABP with a xed structurea
10
50
Adaptive dynamic all parameters GASABP
10
50

20

20
50

Iterations of
BP training

Mean of training
MSE

Mean of testing
MSE

2500
1.7830e 004
0.1254

4.1201e 004
0.1167
100
8.7420e 005
0.1906
100
8.5731e 005
0.1308
20 100
7.4458e 005
0.1845
50 20
100
3.9709e 005
0.0315

a
Represents
designedselected
input node
number
is the largest
dimension
15,namely
hidden1581.
node number is 8, hidden layer and output layer
transfer
functiontherandomly
initially,
the topology
structure
is xed,

random
experimentswhich
with isthe631,
same namely
parameters
setting, there
will
be 18
timesthree
to get
the best
corresponding
the and
structure
arelayer
six
input
nodes,
hidden
nodsindividual
and one
output
node.
Thenetwork
transfer
functions
of hidden
layer
output
are tan
sig and
and the
log
sig respectively.
dotive
a
comparative
research
of all
dynamic
parameters
GASABP
network
model
Momentum
with And
adaplearning
rate
BP
model,
GASA
training
network
model,
GABP
training
network
model
and
SABP
training
network
model.
The
setting
of
parameters
and
the
results
of
20
random
experiments
are
listed
in
the
Table
2.
We can get the result that dynamic all parameters GASABP model has better precision and generation.
5. Conclusion
(1) Utilizing
multi-code
whichwinwin.
combineBecause
the GA,the
SAdynamic
and BPall
perfectly,
and each of them
canuse
exert
their
individual
parameter
make
ofSA
GA
ulationmerits
to search,
doand
therealize
SAthe
probability
combination;
randomGASABP
conjunction
of GA,
andpopBP
directional
itresearch
is easierand
to get
global optimization.
(2) Let
individual
enter SABP
certain
, this heuristic method could greatly improve the
speed
of calculation,
and alsobyassure
theprobability
convergencePSABP
of algorithm.
(3) The
relative
parameter
in the
the sample
BP network
design
be adapted
to and
optimize
the topology
structure,
transfer
function
based on
characters
in will
the learning
process
this could
greatly reduce
the
subjectivity
of
network.and get the simplest network structure with certain precision and also strengthen the gen- eration
Acknowledgement
This research was fully supported by National Natural Science Foundation Grant No. 70573101 of the
Peoples
Republic
of
China.
References
[1] T.
Statake, M.Journal
Katsumi,
N. Nakamura,
Neural
approach for minimizing the makespan of the general job-shop,
International
of Production
Economics
33network
(1994) 6774.
[2] J. Bode, Neural networks for cost estimation: simulations and pilot application, International Journal of Production Research 38 (6)
(2000) 12311254.
[3] I. Sabuncuoglu, B. Gurgun, A neural network model for scheduling problems, European Journal of Operational Research 93 (1996)
288299.
[4] Q.M. Zhu, A back propagation algorithm to estimate the parameters of non-linear dynamic rational models, Applied Mathematical
Modeling 27 (2003) 169187.
[5] A.G. Olabia, G. Casalino, An ANN and Taguchi algorithms integrated approach to the optimization of CO2 laser welding, Advances
in Engineering Software 37 (2006) 643648.
[6] Y.L. Loukas, Articial neural networks in liquid chromatography: Ecient and improved quantitative structureretention
relationship models, Journal of Chromatography A 904 (2000) 119129.
[7] J.F.C. Khaw, B.S. Lim, L.E.N. Lim, Optimal design of neural networks using the Taguchi method, Neurocomputing 7 (1995) 225
245.
[8] H.R. Maier, G.C. Dandy, The eect of internal parameters and geometry on the performance of back-propagation neural networks:
an empirical study, Environmental Modeling and Software 13 (1998) 193209.
[9] H.R. Maier, G.C. Dandy, Understanding the behavior and optimizing the performance of back-propagation neural networks: an
empirical study, Environmental Modeling and Software 13 (1998) 179191.
[10] P.G. Benardos, G.-C. Vosniakos, Prediction of surface roughness in CNC face milling using neural networks and Taguchis design of
experiments, Robotics and Computer Integrated Manufacturing 18 (2002) 343354.
[11] J.P. Ross, Taguchi Techniques for Quality Engineering, McGraw- Hill, New York, 1996.
[12] Wanmei Tang, The study of the optimal structure of BP neural network, Systems Engineering Theory and Practice 10 (2005) 95
100.
[13] S.E. Fahlman, C. Lebiere, The Cascade-Correlation Learning Architecture, Advances in Neural Information Systems, vol. 2,
Morgan- Kaufmann, Los Altos, CA, 1990.
[14] S.D. Balkin, J.K. Ord, Automatic neural network modeling for univariate time series, International Journal of Forecasting 16 (2000)
509515.

[15] M.M. Islam, K. Murase, A new algorithm to design compact two hidden-layer articial neural networks, Neural Networks 14 (2001)
12651278.
[16] X. Jiang, A.H.K.S. Wah, Constructing and training feed-forward neural networks for pattern classication, Pattern Recognition 36
(2003) 853867.
[17] L. Ma, K. Khorasani, A new strategy for adaptively constructing multilayer feed forward neural networks, Neurocomputing 51
(2003) 361385.
[18] P.A. Castillo, J.J. Merelo, A. Prieto, V. Rivas, G. Romero, GProp: global optimization of multilayer perceptrons using gas,
Neurocomputing 35 (2000) 149163.
[19] J. Arifovic, R. Gencay, Using genetic algorithms to select architecture of a feed forward articial neural network, Physica A 289
(2001) 574594.
[20] N. Harri, H. Teri, et al., Evolving the neural network model for forecasting air pollution time series, Engineering Applications of
Articial Intelligence 17 (2004) 159167.
[21] Guangzheng Zhang, Deshuang Huang, Prediction of inter-residue contacts map based on genetic algorithm optimized radial basis
function neural network and binary input encoding scheme, Journal of Computer-Aided Molecular Design 18 (2004) 797810.
[22] Shujuan Li, Yan Li, A GA-based NN approach for makespan estimation, Applied Mathematics and Computation 185 (2007) 1003
1014.
[23] S. Kirkpatrick, C.D. Gerlatt Jr., M.P. Vecchi, Optimization by simulated annealing, Science 220 (1983) 671680.
[24] Timo Poranen, A simulated annealing algorithm for determining the thickness of a graph, Information Sciences 172 (2005) 155172.
[25] Zhang Li, S. Ganesh, An evaluation of back-propagation neural networks for the optimal design of structural systems: Part I.
Training procedures, Computer Methods in Applied Mechanics and Engineering 191 (2002) 28732886.
[26] M. Srinivas, L.M. Patnaik, Adaptive probabilities of crossover and mutation in genetic algorithms, IEEE Transaction on System,
Man and Cybernetic 4 (1994) 656667.
[27] R.S. Balch et al., Using Articial Intelligence to Correlate Multiple Seismic Attributes to Reservoir Properties, SPE 56733, 1999.
[28] O. Tapias, Reservoir Engineer and Articial Intelligence Techniques for Data Analysis, SPE 68743, 2001.
[29] Yibao Li, Xueyong Zhang, et al., Study of improving algorithms based on the BP neural network [J], Journal of Hefei University of
Technology 6 (2005) 668671.
[30] Ling Wang, Intelligent Optimization Algorithms with Applications, Tsinghua Press, Beijing, 2001.
[31] Fujun Hou, Qizong Wu, Forecast on temporal sequence of railway freight transport volume based on BPSA mixing and optimizing
solution, Railway Transport and Economy 10 (2003) 5153.