Noname manuscript No.

(will be inserted by the editor)

Application of supervised machine learning methods on the

multidimensional knapsack problem
Abdellah Rezoug · Mohamed Bader-el-den · Dalila Boughaci

Received: date y / Accepted: date

Abstract Machine Learning (ML) has gained much that the tested methods were able to reach encouraging
importance in recent years as many of its effective ap- results. In addition, the study proposes a Genetic Algo-
plications are involved in different fields, healthcare, rithm (GA) that exploits ML outputs essentially in ini-
banking, trading, gaming, etc. Similarly, Combinato- tialisation operator and to repair unfeasible solutions.
rial Optimisation (CO) keeps challenging researchers The algorithm denoted GaPR explores the ML solu-
by new problems with more complex constraints. Merg- tion neighbourhood as a way of intensification to ap-
ing both fields opens new horizons for development in proach optimal solutions. The carried out experiments
many areas. This study investigates how effective is to indicated that the approach was effective and compet-
solve CO problems by ML methods. The work consid- itive.
ers the Multidimensional Knapsack Problem (MKP) as
a study case, which is an np-hard CO problem well- Keywords Machine learning · multidimensional
known for its multiple applications. The proposed ap- knapsack problem · genetic algorithm · combinatorial
proach suggests to use solutions of small-size MKP to optimisation · evolutionary computation
build models with different ML methods; then, to ap-
ply the obtained models on large-size MKP to predict
their solutions. The features consist of scores calculated 1 Introduction
based on information about items while the labels con-
sist of decision variables of optimal solutions calculated Machine learning (ML) is the field of computer sci-
from applying CPLEX Solver on small-size MKP. Su- ence that focuses on collecting, filtering, analysing data
pervised ML methods build models that help to predict to find patterns and structures to enable predicting,
structures of large-size MKP solutions and build them reasoning, and decision making independently of hu-
accordingly. A comparison of five ML methods is con- man interaction. During the last few years, there has
ducted on standard data set. The experiments showed been a remarkable interest in this field, and people are
more and more used to the ML applications. Image and
speech recognition, video surveillance, sentiment analy-
A. Rezoug
Department of Computer science sis, medical services (disease identification and risk sat-
University Mhamed Bougara, Boumerdes, Algeria isfaction), product recommendations, virtual personal
E-mail: [email protected] assistants (Siri, Alexa), email spam and malware filter-
M. Bader-el-den ing, etc. are the most common applications of ML.
School of computing, University of Portsmouth ML is also employed to solve some of well-known
Portsmouth, United Kingdom
E-mail: [email protected]
combinatorial optimisation problems. General surveys
about ML using for CO problems are given by [21,
D. Boughaci
Department of Computer science
6, 20]. Also, an interesting survey on learning-meta-
University of Science and Technology Houari Boumedienne heuristics is presented by [29]. The mathematical rep-
Algiers, Algeria resentation of Graph-nature CO problems facilitate to
E-mail: dalila [email protected] practice simple ML manipulation backed by simple or
2 Abdellah Rezoug et al.

sophisticated algorithms. The routing and networking involved: Bayesian ARD regression (BAR), Gaussian
problems such as the Travelling Salesman Problem (TSP), process regression (GPR), k-nearest neighbours regres-
the Vehicle Routing Problem (VRP), etc. have simple sion (NNR), random forest regression (RFR) and epsilon-
features easy to represent and their solution are of the support vector regression (SVR). A comparison revealed
same nature. However, data representation is a real is- that the GPR method performance was the best. Sec-
sue when trying to deal with many CO problems by ML ondly, we incorporate the best method (i.e. GPR) as
theories. Few works are done for other kind of CO prob- prior analysis to predict solution and then using it to
lems because representing data, solution and objective initialise the population of a Genetic Algorithm (GA).
function is not compatible with ML theories. The proposed algorithm named GA Combined with
Another problem concerns generalising models to GPR (GaPR) is examined across standard data set
more or larger data. In fact, when a model is built on and the obtained results are compared to the state-of-
data, it is likely hard, sometimes impossible to gener- the-art. The experiments showed encouraging results in
alise it for other than the same kind of data or even on terms of the fitness function.
larger data. However, some recent researches succeeded The remaining of this paper is organised as follows:
to develop techniques to tackle this issue but for one or Section 2 provides a review of some ML approaches
two CO problems. used to deal with CO problems. Section 3 describes the
Nowadays, researchers in different scientific fields multidimensional knapsack problem (MKP). The appli-
are interested in machine learning applications, how- cation procedure of some ML on MKP as well as the
ever, the use of ML concepts in the case of CO problem results are given in Sections 4 and 5 respectively. Sec-
has been poorly investigated. This paper intends to in- tion 6 and 7 present the GaPR variant and the obtained
vestigate whether ML methods are applicable or not results from its application on standard data set. The
to deal with CO problems. This investigation takes the conclusions are given in Section 8.
form of a case-study where the Multidimensional Knap-
sack Problem (MKP) is studied. Five ML algorithms
will be examined, all of which are a regression machine 2 Related works
learning. Consequently, two main contributions are pro-
posed: The first aims to create ML models from small- Several approaches have been proposed to deal with
size MKP instances. The models are then applied fol- CO problems, some focusing on exact methods, oth-
lowing a five-step procedure to build solutions to large- ers on approximate methods, however, few pay partic-
size MKP. A Cross-validation and hyper-parameters ular attention to ML except to tune algorithms param-
tuning processes are conducted on training and test eters. Although many authors have conducted studies,
data to avoid underfitting and overfitting problems. The this field is still insufficiently explored. This section at-
second integrates predictions of the best model in a Ge- tempts to survey the state-of-the-art of the aspect in-
netic Algorithm (GA) variant. Two of the genetic op- cluding the different classes of solutions. Three classes
erators are enhanced by ML finding 1) The initialisa- are distinguishable on how authors utilised ML meth-
tion operator is mainly based on the solutions predicted ods alongside with CO methods in many areas 1) solv-
by the ML procedure 2) The repair operator that fixes ing ML problems by CO methods 2) cooperation be-
unfeasible solutions also uses the predicted solution to tween ML and CO algorithms for CO problems 3) ML
complete the resulting feasible solution. methods directly dealing with CO. This section focuses
The approach uses results obtained from solving on the third class of approaches.
small-size MKP instances by CPLEX to build models. ML has limited capacity to directly solve discrete
As feature, a mathematical equation is used to calculate optimisation problems and needs further steps to for-
the score for each item which is based on the value (or mulate the knowledge they give to build feasible solu-
price) and the weight of the item. Also, the approach tion. However, it has been proven in many cases that
considers the decision variables given by optimal solu- this way of proceeding is very effective. The idea behind
tions as the labels. We use data created in the same is to train ML model to predict solutions directly from
way composed of simple and complex instances. Simple the input solutions.
data are used to build models which we use to predict if The application of ML to CO problems is not new.
the item is likely to be selected regarding its score and Neural networks have been widely studied in the past
then build the solution by a naive method. and even nowadays on standard CO problems such as
This paper is composed of two parts: firstly, we TSP, VRP, Knapsack Problem (KP), etc. Vinyals et
analyse and compare five ML methods effectiveness on al. [30] proposed a supervised Recurrent Neural Net-
MKP. The following supervised regressive methods are work (RNN)-based method as a solution to some CO
Application of supervised machine learning methods on the multidimensional knapsack problem 3

problems with special focus on TSP. A sequence 2 se- 3 Multidimensional Knapsack Problem (MKP)
quence model called pointer network uses RNN encoder
and decoder (Long Short Term Memory (LSTM) and The MKP is an important np-hard CO problem largely
attention Mechanism [2] respectively) to produce an en- studied to test optimisation approaches in operational
coding for each node of a TSP graph to predict future research and computer sciences. Moreover, it models
movements. The advantage of this method is that it can many real-life applications in a different field such as
be generalised to different sizes of graphs. Bello et al. resource allocation in distributed environment [12], re-
[5] suggested a platform based on Reinforcement Learn- source allocation of adaptive multimedia systems [1],
ing (RL) and the pointer network model. The pointer capital budgeting [22], project selection [4], etc. The
network model is driven by a RL based on the Actor- MKP is composed of n items and one knapsack with m
Critical algorithm [23] and negative turn length as re- different capacities ci where i ∈ {1, . . . , m}. Each item
ward. The RL is expensive in computational time but j where j ∈ {1, . . . , n} has a profit pj and can take wij
allowed to obtain solutions very close to optimal and of the capacity i of the knapsack. The goal is to pack
has surpassed supervised methods. the items in a way that maximises their profits with-
Although the RNN architecture is able to consider out exceeding the knapsack capacities. the MKP can be
some characteristics of graph-nature problems, it is not represented as the following integer program:
able to take into account all the characteristics espe-
Pn
cially the relationships between the different entities of

 max Pj=1 pj xj
a graph. The Graph Neural Network (GNN) is the tool n
M KP = sub to j=1 wij xj ≤ ci i ∈ {1 . . . m}
that offers this advantage and allows to represent more 
xj ∈ {0, 1} j ∈ {1 . . . n}
information about the problem. Therefore, many re-
searchers have recently focused on this approach. Kool In practice wij is represented with an integer matrix
et al. [17, 16, 18] utilised the Transformer’s architecture n x m where a line j corresponds to the weights of item
[24] to propose a GNN-based focus model to address j on each of the n dimensions. Whereas a column i is
several routing problems: TSP, VRP and Orienteering the weights of every item on dimension i. In addition,
Problem (OP) by introducing many prior knowledge to pj is represented with an n integer vector where every
the model. An attention mechanism is used to represent single number represents the value of the correspond-
the relationship between the graph nodes. The results ing item (value could be a priority or price). Solving
are interesting for small data only. A Graph Convolu- MKP consists to find the subset of items that max-
tional Network (GCN) model [15] was applied with a imises the overall pj without exceeding the knapsack
tree-guided search algorithm to address the problems capacities. Besides, ci is represented with an m integer
of maximum independent set and minimum summit vector which corresponds to the knapsack dimensions
coverage [19]. Nazari et al. [25] proposed a modified capacities.
pointer network architecture to address the VRP prob-
A feasible solution X for the MKP represents the
lem. Khalil et al. [14] proposed an end-to-end machine
selected items to be packed in the knapsack. The deci-
learning framework that combines deep graph embed-
sion variables xj are binary where xj = 1 means that
ding network (called structure2vec (S2V) [13]) with re-
the item j is packed in the knapsack, and xj = 0 means
inforcement learning (q-learning). By learning greedy
that item j is not packed in the knapsack. wij repre-
algorithms, the framework automatically designs heuris-
sents the weight of the item j on the dimension i.
tics for three graph-based combinatorial optimisation
problems (Travelling Salesman Problem, Minimum Ver-
tex Cover and Maximum Cut). The experiments showed
that the framework performed better than manually de- 4 Procedure of MKP solving with ML methods
signed greedy algorithms across different problems and
graph types and sizes. As an extension of the work by This section describes the procedure to build a solu-
[5], Emami et al. [10] have proposed deep reinforcement tion from the small-size MKP instances to its solution.
learning (DRL)-based methods for solving combinato- The procedure is divided into five steps. The small-size
rial problems. A data-driven algorithm for learning per- MKP instances and the CPLEX Solver are used to cre-
mutations based on a Sinkhorn Policy Gradient (SPG) ate labels and features for the ML methods to build
produces continuous relaxations of permutations ma- the models. Then, the produced models are used for
trices to train an actor-critic architecture. For further complex instances to predict information about their
details on ML usage for CO problems, the reader may solutions. The solutions are then built using a naive
refer to [21, 6, 20] where detailed surveys are provided. method.
4 Abdellah Rezoug et al.

Step 1 label

Small-size MKP instances (i.e. n = 100) are solved to

the optimum using CPLEX. The CPLEX Solver is a
mathematical optimisation software developed by IBM.
It implements many well-known methods such as: Sim-
plex, branch and bound, branch and cut, etc. capable
to deal with integer programming problems and very
large linear programming problems. Besides, CPLEX
provides interfaces for interacting in different program-
ming languages (Python, C++, Java, etc.). The Java
interface is the one used in this study to solve small-
size MKP instances. So, a model is defined composed
of two elements: the objective function in which the
decision variables are defined as binary and the list of
constraints. Then, the solver is invoked to start the op- Fig. 1: score values distribution
timisation process. Finally, the solutions represented by
the decision variables are recorded such that each item
corresponds to its decision variable. Each solution is this stage, each item has a feature and is labelled fol-
represented with a binary vector where 1 means a se- lowing the first step. Therefore, supervised ML methods
lected item and 0 a left item. This binary vector will be are applicable on this input and are able to construct
considered as a label for the input of ML methods in models. Figure 1 shows the distribution of items score
the rest of this work. of six MKP considered to make the feature in this study
(more detail is given in section 5.1).

Step 2 feature
Step 3 model
One feature is considered as MKP has a relatively sim-
ple mathematical model. An item is defined by a weight In the context of combinatorial optimisation, it is prob-
vector relative to the knapsack dimensions and a value ably less important to follow the known approach of
or a price (see Section 3). These two criteria act differ- machines learning, because the purpose of solving prob-
ently on the choice of items, an item that maximises lems is to optimise an objective function. Therefore, the
the objective function has a better probability of be- accuracy of an ML method is measured by how close
ing selected if it has high price (value) and low overall the achieved solution is to the optimum. Five regression
weight. In order to distinguish the best items from the methods of ML are examined in this study. The input
worst, the two criteria are translated by the price / of these methods is formed by data composed of fea-
weight scorej (score of item j) as shown by equations tures and their labels. A ML method produces a model
1-2 bellow : that will be used to predict how likely an item is to be
pj /mean(pj ) selected. The five ML methods involved in this study
scorej = j ∈ {1 . . . n} (1) are: Bayesian ARD regression (BAR), Gaussian pro-
ej /mean(ej )
cess regression (GPR), k-nearest neighbours regression
m n
X X (NNR), random forest regression (RFR) and epsilon-
ej = wij ( wil − ci ) i ∈ {1 . . . m} (2)
support vector regression (SVR).
j=1 l=1

To take into account the contribution of the con-

straints (weights) Senjo and Toyoda [28] calculated ej Step 4 prediction
(equation 2). This formula allows to avoid that a subset
of constraints dominates all others [26]. Furthermore, to After calculating the models, they are used to predict
calculate the score, one of ej and pj could dominate the outputs for large-size MKP instances. For each item a
other, dividing by the mean numbers makes all mea- real value is given which represents the model’s predic-
surements meaningful and close (i.e. values in [0, 2]). tion of its score. In different context, applying classifi-
Thus, makes the score data a correct ML feature. cation methods returns one of two classes, either item
In this study, a scorej vector is calculated for each selected or not selected (i.e. binary classification). But,
MKP instance and is considered as the only feature. At regression methods predict a continuous value for each
Application of supervised machine learning methods on the multidimensional knapsack problem 5

input that may help to take decision about whether with: n = 100, m = {5, 10, 30} alpha = 0.25 and
packing or not an item. Thus, for each MKP instance, instances {1, 2}, so the instances are : {5x100-0.25-
a vector containing the scores of items is entered for 1, 5x100-0.25-2, 10x100-0.25-1, 10x100-0.25-2, 30x100-
each model, and another vector containing the result- 0.25-1, 30x100-0.25-2}. Additionally, the data is divided
ing predictions is produced. Then, these predictions are into training set 70% and test set 30%. A K-fold cross-
used to calculate a feasible solution to the correspond- validation method is also considered on training set
ing MKP problem. with k = 4. The score of the obtained models on test set
was recorded and Table 1 presents the best parameters
and gives the score R2 on test sets of each method.
Step 5 solution Furthermore, the mean squared error regression loss
”mserr” and the maximum residual error ”merr” have
An MKP solution is built by a naive method as follow- been measured.
ing: According to the score R2, NNR model was able
to represent 83% of the test set, then GPR 70%, RFR
1. Sort items in descending order regarding their pre-
69%, SVR 61% and finally BAR 53%.
dictions.
2. Create S an empty solution.
3. Add the items one by one to the solution S, as long
as all constraints are satisfied to keep the solution 5.2 Comparison between the five models
feasible.
4. Calculate the ‘distance from the optimum’ or GAP Regression analysis was used to predict the outputs of
criteria of the solution S. the respective models to the items of each MKP prob-
5. Calculate the fitness function value. lem. Table 2 compares the results obtained from the
6. The result S is a feasible solution. preliminary analysis of the five methods. The average
GAP of each of the nine groups of data-set is examined.
The average GAP of each method, calculated using the
procedure explained in Section 4, were compared in or-
der to define which one achieves the best results. In
5 MKP solved with the five methods order to show the differences in terms of performing,
detailed comparison is given in Table 2 and Figure 3.
Empirical performance evaluations are of great bene- It can be seen from the data in the histograms of
fit, as they emphasise the strengths and weaknesses of Figure 2 that, considering the performing, the five meth-
the evaluated methods and thereby guide research ef- ods can be classified into three subsets. Firstly GPR
forts into promising directions. This section presents a and BAR, these two methods allowed to calculate the
comparison between the ML methods, and this is done best MKP solutions for all the nine groups. Secondly,
with respect to the GAP, a method is better if its GAP SVR was capable to predict high-quality solutions also
is low. The standard OR-Library data often used by and its results are not as good as the first two algo-
researchers in the field are used to do this study. It rithms. Finally, RFR and NNR results are of inferior
is composed of 270 instances of large-size MKP whose quality than the others on all the nine groups of data.
number of constraints m is in {5, 10, 30} and the num- Closer inspection of the data shows that all the
ber of variables n is in {100, 250, 500}. But to validate methods allowed to build high-quality solutions with
the models we need first to conduct a hyper-parameters low GAP. More specifically, GPR gave the lowest GAP,
tuning and a cross-validation processes. it is the best performing method with an overall GAP
less than 5% for almost all the nine groups except the
groups 4 and 7, for which the overall GAP was less
5.1 Hyper-parameters fitting and cross-validation than 5.5%. The results indicate that BAR obtained the
second-best performance and reached better solutions
When using ML methods, it is difficult to know ex- than the remaining methods with an overall GAP less
actly what are the best hyper-parameters of a given than 4% for groups 2, 3, 5, 6 and 9 and less than 6% for
algorithm on a given problem, therefore it is common the others. Besides, SVR was able to reach good results.
to apply random or grid search methods to fit them. RFR and NNR had a comparative performing with an
Thus, the grid search strategy is applied to perform the overall GAP between 3% and 7%. They achieved the
hyper-parameters optimisation. For that, six small-size worst results on all the data-set compared to the oth-
MKP instances have been chosen to create the data ers.
6 Abdellah Rezoug et al.

m = 5, n = 100 m = 5, n = 250 m = 5, n = 500

4 4

4
3 3
Gap (%)

2 2
2

1 1

0 0 0
SVR GPR BAR RFR NNR SVR GPR BAR RFR NNR SVR GPR BAR RFR NNR

m = 10, n = 100 m = 10, n = 250 m = 5, n = 500

6 4

3
4
Gap (%)

4
2

2
2
1

0 0 0
SVR GPR BAR RFR NNR SVR GPR BAR RFR NNR SVR GPR BAR RFR NNR

m = 30, n = 100 m = 30, n = 250 m = 30, n = 500

6

6
4

4
Gap (%)

2
2
2

0 0 0
SVR GPR BAR RFR NNR SVR GPR BAR RFR NNR SVR GPR BAR RFR NNR

Fig. 2: Comparison of the five methods performances for some MKP instances.

5x500
10x100

5x250

10x250

5x100

SVR

10x500 NNR

BAR
30x500
RFR

30x100 GPR
30x250

Fig. 3: Comparison of the five machine learning methods in terms of average GAP on the 9 groups of mkp dataset
Application of supervised machine learning methods on the multidimensional knapsack problem 7

method Best parameters R2 (%) merr mserr

SVR C = 1.0, cache size = 200, coef 0 = 0.0, degree = 3, epsilon = 0.001, gamma = 61 0.92 0.08
auto deprecated, kernel = rbf, max iter = −1, shrinking = T rue, tol = 0.001
BAR alpha 1 = 1e − 06, alpha 2 = 1e − 05, lambda 1 = 1e − 05, lambda 2 = 1e − 05, 53 1.62 0.09
n iter = 200, tol = 0.0001
NNR algorithm = brute, leaf size = 1, n neighbors = 15, p = 2, weights = distance 83 0.86 0.03
RFR n estimators = 500, min sampless plit = 20, min samples leaf = 2, 69 0.92 0.06
max f eatures = auto, max depth = 110, bootstrap = T rue
GPR alpha = 0.1, kernel = RBF (length scale = 10) 70 0.86 0.06

Table 1: Hyper-parameters fitting, cross-validation Score R2 (%), maximum residual error merr and mean squared
error regression loss mserr of the five ML.

m n SVR GPR BAR RFR NNR

5 100 4,46 4,34 4,26 4,45 4,69
250 3,31 3,14 3,20 3,55 3,87
500 2,69 2,54 2,58 3,04 3,11
Group average 3,49 3,34 3,35 3,68 3,89
10 100 5,11 5,01 5,03 5,66 6,14
250 4,16 3,98 3,96 4,71 5,02
500 3,05 2,82 3,05 3,87 3,77
Group average 4,10 3,94 4,01 4,75 4,98
30 100 5,56 5,37 5,33 5,79 6,15
250 4,54 4,45 4,38 5,13 5,43
500 3,85 3,71 3,76 4,18 4,77
Group average 4,65 4,51 4,49 5,03 5,45
Overall average 4,08 3,93 3,95 4,49 4,77

Table 2: The average GAP from the five ML methods application on the data set.

What stands out in the experiment is that: firstly, Python language has been utilised to draw the distri-
ML methods can successfully solve MKP with encour- butions.
aging results. Secondly, only one feature about MKP From the distributions of predictions, it can be no-
simple data set were useful to build models capable to ticed that SVR and GPR obtain almost similar repre-
predict solutions of complex data of the same prob- sentation even when complexity increases. In fact, by
lem. Thirdly, the execution time to calculate solutions calculating the feasible solution of the respective MKP,
of large-size MKP instances by using ML methods is in- both methods were able to construct comparable pre-
significant than when other methods are used. Finally, dictions. Prediction values for both methods vary in [0,
searching in the neighbourhood of the obtained solu- 1] with more density around 0. Also, BAR predictions
tions is likely to approach the optimal solutions. take a linear form as almost all predictions are on the
same line.
In addition, NNR has a model that act like a clas-
5.3 Behaviour of the five methods on data
sification method. The prediction gives three classes of
results: a first class of items with a prediction equal to
Figure 4 represents nine cases of MKP chosen semi-
0, a second class of items with prediction equal to 1 and
randomly, for each case, the distribution of ‘predicted
a third class between 0 and 1. Moreover, NNR classi-
regression’ is plotted as a function of score and this for
fies items into two main classes for case (a), class 1 and
all the five ML methods implied in this work. This ex-
class 0. Aside this case, NNR gets outputs close to 0
periment aims to inspect the way each model of the ML
for most of the scores of the instances included in this
methods represents the prediction and how the models
study. Also, the distribution of predictions when using
are built out of the data. The scikit-learn 1 library of
1
https://scikit-learn.org/stable/
8 Abdellah Rezoug et al.

RFR does not take any graphical form, in fact almost 1. X1 which is a feasible solution built with a simple
all points are placed randomly between 0 and 1. method from the items sorted decreasingly regard-
ing their ef f iciency as follows:
(a) The items are sorted based on their ef f iciency
(equation 3) [28]. The items at the top of this list
6 Genetic Algorithm combined with GPR are likely to be selected while the items at the
(GaPR) bottom are unlikely to appear in good solutions.
p
ef f iciencyj = Pm Pjn (3)
j=1 wij ( l=1 wil − ci )
ML method itself does not find optimal solutions, and
there is no way to introduce MKP constraints into its (b) X1 ← ∅
procedure. In addition to that, it is a statistics-based (c) Add the sorted items one by one to X1 while all
mechanism whose purpose is to support decision mak- constraints are satisfied.
ing. However, the information it offers about items can (d) Stop if at list one constraint is broken.
be very useful for supporting algorithms dedicated to 2. core which contains the few items that follow the
this operation, consequently, the heuristics are the most last item of X1. The core size depends on the MKP
suitable here. size.
Genetic Algorithm (GA) is a heuristic inspired by The pseudo-code of Algorithm 1 shows the algorithm
evolution and natural selection of living species. It is used to create the initial population:
based on a population of individuals that evolves in
iterative way by operations (or operators) of crossing
and mutation between the elites of individuals, at the Algorithm 1: Initialisation operator
same time bad individuals eventually disappear during input : GPR solution (gs), X1 core, MKP
the passage to the following generations. GA has a lot of instance (mkp), population size (ps),
GPR rate (gr), number of iteration (ni)
applications and has many variations since its inception
output: an initial population (pop)
aimed to improve its performance and making it faster.
pop ← ∅ ;
GA has been widely proposed in the form of MKP for i ← 1 to ps do
solution approaches, whether alone or hybridised with s ← ∅;
other heuristics or exact methods. In this part, the GA // create s a new solution
for j ← 1 to ni do
Combined with GPR (GaPR) is proposed. GaPR in-
r ← random real in [0, 1];
corporates GPR solutions in GA firstly, to create the // random rate: generate a random
initial population and secondly, to repair unfeasible so- probability
lutions. In GaPR solution, GA is combined with GPR if r < gr then
item ← a random item from gs;
method which represents a novelty and opens the way
else
to further research in this perspective. item ← a random item from X1 core;
In view of the results obtained previously, the com- if (s + item) verifies constraints then
parison showed that GPR method achieved the best s ← s + {item};
solutions. Knowing that these are close to optimal solu- // if adding item to s keeps it
tions, they are integrated into the creation of the initial feasible
population of a GA. The idea is that since the optimal pop ← pop + s
solutions are not far away it is enough to search around return pop
these solutions to find promising solution areas.

6.1 Initial population

6.2 Genetic operators
The individuals of initial population are created with
a special procedure that implies the GPR solution and The selection operator is based on Tournament algo-
X1 core items as follows: A probability r is generated rithm where a subset of individuals of known size is
randomly and if it is less than a pre-defined GPR rate chosen randomly and then the two best individuals (in
gr, then the procedure randomly chooses an item from terms of fitness) are selected. The crossing is performed
the GPR solution (We make sure that this probability on the two individuals selected according to the multi-
is high). Otherwise, the procedure chooses an item from point crossing algorithm. An exchange of multiple items
X1 core. X1 core is a vector of items made of: between the two individuals while checking the MKP
Application of supervised machine learning methods on the multidimensional knapsack problem 9

(a) 5x100-0.25-1 (b) 5x250-0.50-1 (c) 5x500-0.75-1

(d) 10x100-0.25-1 (e) 10x250-0.50-1 (f) 10x500-0.75-1

(g) 30x100-0.25-1 (h) 30x250-0.50-1 (i) 30x500-0.75-1

Fig. 4: The predicted regression through the score on some MKP instances for the five methods

constraints and the crossover rate determines the num- 1. Solution feasibility: as long as at least one con-
ber of items exchanged. Two new individuals are built straint is not verified the procedure removes several
to whom a repair is applied to make them feasible so- times an item according to a probability either: hav-
lutions. The repair algorithm is described in the next ing the lowest score or randomly, knowing that the
section. The mutation operator is simple where each first option is much more likely than the second.
new individual undergoes a simple mutation such that 2. Adding items: to complete a solution, the items,
a single item is removed from it, and a repair operation sorted according to their scores and the items of
is applied to it. the best individual in the population are utilised.
The procedure check them and each item that does
not cause conflict is added to the solution. Conse-
6.3 Repair operator quently, items that maximise the objective function
are favoured with this procedure.
The repair takes place in two stages, the first makes
the solution feasible by removing all eventual conflict
and the second tries to complete the solution by adding The pseudo-code of Algorithm 2 details the repair op-
items. erator.
10 Abdellah Rezoug et al.

Algorithm 2: Repair operator 1. For an MKP instance I calculate the score vector
input : MKP solution s, the currant best scoreI .
solution in pop bsts, sorted items si, ML 2. Calculate the model prediction of the scoreI by ap-
delete rate (mlr) plying the model.
output: s feasible and improved

Phase I: fixing s
while (s unfeasible) do
r ← random rate();
3. The items are then sorted according to their pre-
// random rate: generate a random diction prediction(scoreI ). After that, a feasible so-
probability lution P sol (for predicted solution) is created by
if r < mlr then adding the items respecting the order and of course
w item ← worst item(s);
s ← s − {w item};
the problem constraints.
// Remove the item having the worst 4. Initialisation operator: it is detailed by the pseudo-
score code of Algorithm 1.
else 5. Selection operator: it follows the tournament se-
s ← s − {random item}; lection algorithm where a tour of individuals are
// Remove a random item
selected randomly and from them two are chosen to
Phase II: improving s be the parents.
ir ← a random real in [0, 1]; 6. Crossover operator: the multi-point crossover method
if ir < bsts then is used to build two offspring offspring1, offspring2
foreach item it of the vector si do from the selected parents.
if s + {it} keeps s feasible then
7. Mutation operator: with a mutation probability,
s ← s + {it}
replace one item from each of f spring by another
else both chosen randomly.
foreach item it of the bsts do
8. repair (offspring1), repair (offspring2). The re-
if s + {it} keeps s feasible then
s ← s + {it} pair pseudo-code is given by Algorithm 2.
9. replace the two worst individuals in the population
return s by of f spring1 and of f spring2 if their fitness are
better.
10. Go to 9 if the stopping condition is false.
The GaPR algorithm is composed of two phases:
model creation and Genetic operators. It is explained
as follows :
7 Experimental results of GaPR on MKP
Phase 1. model creation
This section presents the results of applying the GaPR
1. Determine data : a subset of k MKP with n = 100 on standard data and the average fitness over 30 runs
and m = 5 and α = 0.25. is calculated. The data benchmark employed to under-
2. For each MKP in data, calculate the score vector take experimental tests are the standard OR-Library
according to equation 1. The feature of the GPR data set presented by Chu and Beasley [7] composed of
is composed of all these score vectors f eature = 270 instance divided into 9 classes of 30 instance each.
{score1 , score2 , ..., scorek }. Where the number of items n is in {100, 250, 500} and
3. Solve each of data MKP instances using CPLEX the number of constraints m is in {5, 10, 30 }. For each
solver. From this operation result k binary vectors combination (n, m), the constraint capacities are com-
Pn
that represent the solutions sol. Together they form puted as α j=1 wij , where α is a tightness ratio. For
the GPR target target = {sol1 , sol2 , ..., solk }. the first 10 instances, α = 0.25; for the next 10 in-
stances, α = 0.5; and for the remaining ones, α = 0.75.
The GaPR parameters were chosen in a way to con-
centrate the search at the neighbourhood of the GPR
solution. Table 3 summaries the parameters values utilised
in this work.
4. The model is then built from applying the GPR on
The results from applying GaPR on the 270 MKP
the input composed of (f eature, target).
instances are summarised in Table 4. The average dis-
Phase 2. genetic operators tance from optimum ’GAP’ values of each 10 instances
Application of supervised machine learning methods on the multidimensional knapsack problem 11

Parameter Description n= 100 250 500 are presented in such a way that allow to compare the
ps population size 60 60 60 GaPR performing according to the target parameters.
ni number of iteration 200 500 1000 From observing each box alone, we can understand the
gr GPR rate 0.9 0.9 0.9 GaPR behaviour when α changes in {0.25, 0.50, 0.75}.
cs core size 15 25 40
Whereas, observing the boxes vertically compares how
mlr ML delete rate 0.95 0.95 0.95
cr crossover rate 0.2 0.1 0.05 it behaves when the number of constrains m increases in
mr mutation rate 1 1 1 {5, 10, 30}. Similarly, observing them horizontally, ex-
tour tour of selection 5 5 5 plains how its performing is when the number of items
bsts rate best solution 0.15 0.15 0.15
n increases in {100, 250, 500}. The comparison for the
two former cases should be for the same α.
Table 3: The GaPR parameters
From the results, the first finding is that GaPR
keep giving high-quality performance even when the
is given knowing that each instance is executed 30 times MKP dimensions increase significantly. Concerning α,
to obtain the average GAP. What is interesting about the results show that GaPR performs better when α
the data in this table is that GAP is very close to increases, however, the gap is always close to zero. Re-
zero which means that the algorithms constantly reach garding m, the GaPR performing remains excellent even
close-to-optimal solutions even when n and m increased. when the number of constraints is 30, and it is better
Closer inspection of the table shows that the algorithm with less constraints. We can see that GaPR obtains
performed better on the instances where the correlation less good results when the number of items increase to
α between weight and value is less tight. 500. The worst performance can be seen in the seventh
box (m = 30, n= 100 and α = 0.25).
The results obtained from the application of GaPR
m α n = 100 n = 250 n = 500
on the MKP instances with n = 500 are summarised in
0,25 0,09 0,22 0,31
Table 5 (the first three instances of each class). Many
5 0,50 0,08 0,17 0,25
0,75 0,05 0,16 0,76 data are recorded from 30 runs on each instance where
: averagef the average fitness, f best, f worst the best
Group average 0,07 0,18 0,44
and the worst recorded fitness, P SG the gap from the
0,25 0,32 0,44 0,64
optimum of the GPR solution, gap the gap from the
10 0,50 0,17 0,32 0,46
0,75 0,12 0,28 0,89 optimum of the average fitness, and the gapbest gap of
the best obtained solution and the optimum.
Group average 0,2 0,34 0,66
The results indicate that in 16 of 27 cases the GPR
0,25 0,63 0,96 1,42
30 0,50 0,28 0,54 0,92
solution gap is less than 3%, in 10 cases between 3-5%
0,75 0,14 0,44 1,26 and only 1 case greater than 5%. They show that start-
Group average 0,35 0,65 1,2
ing from models created from small-size MKP instances
with n = 100, it is possible to predict high-quality so-
Overall average 0,21 0,39 0,77
lutions for other problems with greater variables and
constraints m = 30 and n = 500.
Table 4: Summary of the average GAP (%) from ap-
plying the proposed approach on the data set. The GaPR improved significantly the initial solu-
tion and reached high-quality solutions. For almost all
the cases, the gap is less than 1% and the worst fit-
ness is close to the best fitness. That means GaPR is
able to reach areas of the best solutions thanks to high-
In order to analyse the performance of GaPR when quality individuals in the initial populations. Further-
α, n and m change, we conducted a statistical experi- more, GaPR find better solution than the known best
ment. Randomly, we have picked the 5th MKP instance solutions for 3 cases. Also, GaPR even with the worst
out of each of the 27 classes (no rule is applied to make GPR solutions was able to improve their fitness.
this choice and all conclusions are valid no matter the Turning now to the comparison of GaPR with the
instance number). Then, GaPR was applied on each in- literature. Table 6 shows a comparison of GaPR to
stance thirteen (13) times and each time the gap from many other approaches. The proposed approach is com-
optimal solution was recorded. These records perme- pared to many other approaches: Shuffled Complex Evo-
ated to calculate the Best, the Worst, the Mean the first lution (SCE) [3]; general-purpose computation on graph-
and the third quartile solutions in order to draw box- ics processing units Simulated Annealing (GPGPU SA)
plots, the results are given by Figure 5. The boxplots
12 Abdellah Rezoug et al.

m instance optimum averagef f worst f best P SG gap gap best

0.25-1 120148 119940,3 119837 120024 1,8 0,17 0,10
0.25-2 117879 117594,6 117450 117785 2,1 0,24 0,08
0.25-3 121131 120797,3 120585 120905 3,7 0,27 0,18
0.50-1 218428 217846,7 217606 218077 3,0 0,27 0,16
5 0.50-2 221202 220465,3 220003 220671 4,0 0,33 0,24
0.50-3 217542 217208,2 217108 217380 0,8 0,15 0,07
0.75-1 295828 293471,3 291525 294204 3,1 0,79 0,55
0.75-2 308086 305770 304759 306732 3,9 0,75 0,43
0.75-3 299796 298010 297352 298648 2,6 0,59 0,38
0.25-1 117821 117452 117053 117945 4,6 0,31 -
0.25-2 119249 119161 118728 119525 2,6 0,07 -
0.25-3 119215 119106,6 118542 119367 3,9 0,09 -
0.50-1 217377 216778,6 216256 217013 1,9 0,27 0,16
10 0.50-2 219077 218364,6 217987 218907 2,8 0,32 0,08
0.50-3 217847 217261,2 217053 217420 2,8 0,26 0,19
0.75-1 304387 302250,3 301817 302855 2,5 0,71 0,50
0.75-2 302379 299757,7 298898 300257 2,8 0,86 0,70
0.75-3 302417 300335,8 299814 300949 2,8 0,69 0,48
0.25-1 116056 114478,9 114035 114760 5,6 1,36 1,11
0.25-2 114810 113968,4 113799 114263 4,7 0,73 0,47
0.25-3 116741 115394 115176 115796 3,9 1,15 0,81
0.50-1 218104 216225,4 215823 216539 2,6 0,86 0,72
30 0.50-2 214648 212852,9 212525 213051 2,9 0,83 0,74
0.50-3 215978 214861,5 214578 215173 2,2 0,51 0,37
0.75-1 301675 298312,9 297720 299136 2,7 1,11 0,84
0.75-2 300055 296720,5 295670 297209 3,3 1,11 0,95
0.75-3 305087 301510,5 300856 302695 2,6 1,17 0,78

Table 5: Detailed results of the GaPR application on some large-size instances with n = 500.

n m α ACO MCF SCE GPGPU SA Sequential SA GAGP GaPR

5 100 0.25 0.51 1.09 3.5 0.44 1.02 0.35 0.09
0.50 0.19 0.57 2.6 0.21 0.55 0.48 0.08
0.75 0.09 0.38 1.1 0.14 0.28 0.21 0.05
250 0.25 0.44 0.41 4.3 0.28 0.67 0.58 0.22
0.50 0.19 0.22 3.3 0.12 0.28 0.36 0.17
0.75 0.08 0.14 1.5 0.06 0.16 0.23 0.16
500 0.25 0.27 0.21 4.6 0.14 0.39 0.51 0.31
0.50 0.12 0.1 3.6 0.06 0.17 0.36 0.25
0.75 0.06 0.06 1.8 0.04 0.11 0.22 0.76
10 100 0.25 1.83 1.87 6.8 1.27 1.94 1.0 0.32
0.50 0.81 0.95 5.1 0.49 0.84 0.53 0.17
0.75 0.38 0.53 2.4 0.23 0.43 0.27 0.12
250 0.25 1.07 0.79 6.9 0.59 1.04 0.75 0.44
0.50 0.58 0.41 5.4 0.34 0.55 0.48 0.32
0.75 0.24 0.24 2.8 0.15 0.27 0.27 0.27
500 0.25 0.53 0.44 6.8 0.26 0.57 0.71 0.64
0.50 0.29 0.2 5.8 0.15 0.29 0.4 0.46
0.75 0.16 0.13 3.4 0.08 0.17 0.29 0.89
30 100 0.25 3.03 3.61 8.6 1.83 2.33 1.56 0.63
0.50 2.06 1.6 6.6 0.98 1.26 1.07 0.28
0.75 0.93 0.97 3.6 0.67 0.73 0.36 0.14
250 0.25 2.83 1.75 8.3 1.58 1.84 1.66 0.96
0.50 1.56 0.79 6.9 0.78 0.82 1.0 0.54
0.75 0.72 0.43 3.8 0.38 0.46 0.5 0.44
500 0.25 1.98 1.05 8.6 1.04 1.29 4.07 1.42
0.50 0.96 0.44 7.4 0.52 0.61 2.14 0.92
0.75 0.53 0.27 4 0.28 0.33 0.51 1.26

Table 6: The GaPR algorithm compared to other recent results of the literature.
Application of supervised machine learning methods on the multidimensional knapsack problem 13

(a) 5x100 mkp 5 (b) 5x250 mkp 5 (c) 5x500 mkp 5

(d) 10x100 mkp 5 (e) 10x250 mkp 5 (f) 10x500 mkp 5

(g) 30x100 mkp 5 (h) 30x250 mkp 5 (i) 30x500 mkp 5

Fig. 5: Analysis of the GaPR behaviour when α, n and m changes.

and Sequential SA [8]; Ant Colony Optimisation (ACO) superior than ours for n = 250 in some cases; but our
[11] algorithm for which the reported results are pro- approach performed better for n = 500.
duced for the parameters of 256 ants and 8 colonies; our The initialisation operator based on GPR solution
Genetic Algorithm Guided by Pre-treatment (GAGP) has lead to a population composed of individuals that
[27] and Modified Choice Function - Late Acceptance contains almost the optimal solution. The individuals
Strategy (MCF) [9]. Table 6 reports the average GAP of the initial population requires only deep search to
of all the algorithms and the best result is in bold. achieve optimal solutions. Focusing the search to the
GaPR performed better than the other approaches local neighbourhood, the process has more chance to
for 14 of the 27 groups. Moreover, the algorithm al- reach high-quality solutions. It is possible and recom-
most reached only optimal solutions for many groups. It mended to use small-size MKP data to build model
can be seen clearly that this algorithm has significantly that allows to predict helpful information to facilitate
improved our results produced by GAGP. In addition, solving large-size MKP if data have similar conception.
GaPR are competitive with ACO and GPGPU SA both The repair procedure uses selected information com-
very sophisticated algorithms. GaPR are comparative bined with a small rate of random permitted to remove
to GPGPU SA for n = 100; GPGPU SA results are the worst items and to replace them by the best once.
14 Abdellah Rezoug et al.

8 Conclusion learning to align and translate. arXiv preprint

arXiv:1409.0473, 2014.
ML and CO are two different research domains of math- 3. Marcos Daniel Valadão Baroni and Flávio Miguel
ematics and computer science. But many researches Varejão. A shuffled complex evolution algorithm
try to tackle CO problem using ML tools and obtain for the multidimensional knapsack problem using
encouraging findings. The purpose of this study is to core concept. In 2016 IEEE Congress on Evolution-
utilise ML powerful tools to deal with the multidimen- ary Computation (CEC), pages 2718–2723. IEEE,
sional knapsack problem (MKP). Therefore, a proce- 2016.
dure was given to explain how to build models and then 4. George J Beaujon, Samuel P Marin, and Gary C
to predict how likely items are to be selected. A com- McDonald. Balancing and optimizing a portfolio
parison implying five ML methods have been conducted of r&d projects. Naval Research Logistics (NRL),
on standard MKP data set including BAR, NNR, SVR, 48(1):18–40, 2001.
GPR and RFR. It revealed that firstly, ML methods 5. Irwan Bello, Hieu Pham, Quoc V Le, Mohammad
successfully help to produce high-quality solutions and Norouzi, and Samy Bengio. Neural combinatorial
secondly, GPR performed better than the other meth- optimization with reinforcement learning. arXiv
ods. The MKP solutions needed a close look at their preprint arXiv:1611.09940, 2016.
neighbourhood to approach optimal solution. There- 6. Yoshua Bengio, Andrea Lodi, and Antoine Prou-
fore, a Genetic Algorithm (GA) named GaPR was pro- vost. Machine learning for combinatorial opti-
posed that includes GPR solution to initialise its pop- mization: a methodological tour d’horizon. arXiv
ulation. An experimental test was conducted on well- preprint arXiv:1811.06128, 2018.
known complex data set and has given competitive re- 7. Paul C Chu and John E Beasley. A genetic algo-
sults. This work has proven that applying ML method rithm for the multidimensional knapsack problem.
for MKP gives encouraging results. Journal of heuristics, 4(1):63–86, 1998.
In optimisation situations where time is a critical 8. Bianca De Almeida Dantas and Edson Norberto
factor such as real-time optimisation, it is significantly Cáceres. A parallelization of a simulated annealing
practical to adopt our approach to quickly find good approach for 0-1 multidimensional knapsack prob-
configuration instead of executing time-consuming op- lem using gpgpu. In 2016 28th International Sym-
timisation process. The main challenge to overcome is posium on Computer Architecture and High Perfor-
to build a complete and coherent input (feature and mance Computing (SBAC-PAD), pages 134–140.
label). The more specific the input is, the more accu- IEEE, 2016.
rate the optimisation will be. In our study, with more 9. John H Drake, Ender Özcan, and Edmund K
features the procedure could give better outputs only Burke. Modified choice function heuristic selection
by model prediction without optimisation process. Fur- for the multidimensional knapsack problem. In Ge-
thermore, the approach makes finding optimal solutions netic and Evolutionary Computing, pages 225–234.
easier, in fact, it requires a quick and deeper look at Springer, 2015.
the neighbourhood of the predicted solutions. As per- 10. Patrick Emami and Sanjay Ranka. Learning per-
spective, it is necessary to develop tools that allow to mutations with sinkhorn policy gradient. arXiv
use ML method independently; reinforcement learning preprint arXiv:1805.07010, 2018.
and graph-nature CO problems are capable to be gen- 11. Henrique Fingler, Edson Norberto Cáceres, Hen-
eralised. rique Mongelli, and Siang W Song. A cuda based
solution to the multidimensional knapsack problem
using the ant colony optimization. In ICCS, pages
84–94, 2014.
12. Siddig Jihad, Xianqiao Chen, Bing Shi, and Soly-
References
man Aiman. Multidimensional knapsack problem
for resource allocation in a distributed competi-
1. Mohamed Abdel-Basset, Doaa El-Shahat, Hossam
tive environment based on genetic algorithm. In
Faris, and Seyedali Mirjalili. A binary multi-
2019 International Conference on Computer, Con-
verse optimizer for 0-1 multidimensional knapsack
trol, Electrical, and Electronics Engineering (ICC-
problems with application in interactive multime-
CEEE), pages 1–5. IEEE, 2019.
dia systems. Computers & Industrial Engineering,
13. Chaitanya K Joshi, Thomas Laurent, and Xavier
132:187–206, 2019.
Bresson. An efficient graph convolutional net-
2. Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua
work technique for the travelling salesman problem.
Bengio. Neural machine translation by jointly
Application of supervised machine learning methods on the multidimensional knapsack problem 15

arXiv preprint arXiv:1906.01227, 2019. 206, 2001.

14. Elias Khalil, Hanjun Dai, Yuyu Zhang, Bistra Dilk- 23. Volodymyr Mnih, Adria Puigdomenech Badia,
ina, and Le Song. Learning combinatorial optimiza- Mehdi Mirza, Alex Graves, Timothy Lillicrap, Tim
tion algorithms over graphs. In Advances in Neural Harley, David Silver, and Koray Kavukcuoglu.
Information Processing Systems, pages 6348–6358, Asynchronous methods for deep reinforcement
2017. learning. In International conference on machine
15. Thomas N Kipf and Max Welling. Semi-supervised learning, pages 1928–1937, 2016.
classification with graph convolutional networks. 24. Ofir Nachum, Shixiang Shane Gu, Honglak Lee,
arXiv preprint arXiv:1609.02907, 2016. and Sergey Levine. Data-efficient hierarchical re-
16. Wouter Kool, HV Hoof, and Max Welling. At- inforcement learning. In Advances in Neural Infor-
tention solves your tsp, approximately. Statistics, mation Processing Systems, pages 3303–3313, 2018.
1050:22, 2018. 25. Mohammadreza Nazari, Afshin Oroojlooy,
17. Wouter Kool, Herke Van Hoof, and Max Welling. Lawrence Snyder, and Martin Takác. Rein-
Attention, learn to solve routing problems! arXiv forcement learning for solving the vehicle routing
preprint arXiv:1803.08475, 2018. problem. In Advances in Neural Information
18. Wouter Kool, Herke van Hoof, and Max Welling. Processing Systems, pages 9839–9849, 2018.
Buy 4 reinforce samples, get a baseline for free! In 26. Jakob Puchinger, Günther R Raidl, and Ulrich
Seventh International Conference on Learning Rep- Pferschy. The multidimensional knapsack problem:
resentations ICLR 2019, 2019. Structure and algorithms. INFORMS Journal on
19. Zhuwen Li, Qifeng Chen, and Vladlen Koltun. Computing, 22(2):250–265, 2010.
Combinatorial optimization with graph convolu- 27. Abdellah Rezoug, Mohamed Bader-El-Den, and
tional networks and guided tree search. In Advances Dalila Boughaci. Knowledge-based genetic algo-
in Neural Information Processing Systems, pages rithm for the 0–1 multidimensional knapsack prob-
539–548, 2018. lem. In 2017 IEEE Congress on Evolutionary Com-
20. Michele Lombardi and Michela Milano. Boost- putation (CEC), pages 2030–2037. IEEE, 2017.
ing combinatorial problem modeling with machine 28. Shizuo Senju and Yoshiaki Toyoda. An approach
learning. arXiv preprint arXiv:1807.05517, 2018. to linear programming with 0-1 variables. Manage-
21. Nina Mazyavkina, Sergey Sviridov, Sergei Ivanov, ment Science, pages B196–B207, 1968.
and Evgeny Burnaev. Reinforcement learning 29. El-Ghazali Talbi. Machine learning into meta-
for combinatorial optimization: A survey. arXiv heuristics: A survey and taxonomy of data-driven
preprint arXiv:2003.03600, 2020. metaheuristics. 2020.
22. Helga Meier, Nicos Christofides, and Gerry Salkin. 30. Oriol Vinyals, Meire Fortunato, and Navdeep
Capital budgeting under uncertainty-an integrated Jaitly. Pointer networks. In Advances in neural
approach using contingent claims analysis and inte- information processing systems, pages 2692–2700,
ger programming. Operations Research, 49(2):196– 2015.