Expert Systems With Applications 232 (2023) 120762

Contents lists available at ScienceDirect

Expert Systems With Applications

journal homepage: www.elsevier.com/locate/eswa

Self-adaptive CMSA for solving the multidimensional multi-way number

partitioning problem
Marko Djukanović a ,∗, Aleksandar Kartelj b , Christian Blum c
a Faculty of Natural Sciences and Mathematics, University of Banja Luka, Bosnia and Herzegovina
b
Faculty of Mathematics, University of Belgrade, Serbia
c
Artificial Intelligence Research Institute (IIIA), Spanish National Research Council (CSIC), Spain

ARTICLE INFO ABSTRACT

MSC: The multidimensional multi-way number partitioning problem takes as input a set of 𝑛 vectors with a fixed
68W25 dimension 𝑚 ≥ 2, and aims to find a partitioning of this set into 𝑘 ≥ 2 non-empty subsets, such that the sums
68W20 per coordinate across all subsets are as similar as possible. The problem has applications in key encryption,
Keywords: multiprocessor scheduling, minimization of circuit size and delay, and clustering. This paper employs a hybrid
Number partitioning problem meta-heuristic, a self-adaptive version of the Construct, Merge, Solve, and Adapt algorithm equipped with an
Hybrid metaheuristics efficient local search procedure. Local search was able to accelerate the convergence towards promising regions
MILP models
of the search space. A comprehensive experimental evaluation shows that the proposed algorithm improves
Local search
over all four competing algorithms from the related literature, especially when it comes to instances with
higher 𝑘-values, i.e. 𝑘 ≥ 3. The observed average relative differences are for several instance groups larger than
25% in favor of the proposed algorithm compared to the second-best approach. In fact, a statistical evaluation
indicates that our algorithm performs significantly better than the other approaches on all instances with 𝑘 ≥ 3.

1. Introduction 𝑘 disjoint subsets—that is, a 𝑘-partitioning (𝑘 ≥ 2)—such that the sums

of the values at each coordinate for all subsets are balanced. Concrete
Number partitioning problems are highly relevant optimization examples of practical applications of the MDMWNPP are the following
problems with a wide range of applications in practice. The most well- ones:
known problem variant is the so-called number partitioning problem
(NPP) which requires to partition a given set of 𝑛 numbers into exactly • Partitioning an area containing cities into regions such that an
two subsets such that the sums of the numbers in the two subsets are as equilibrium among the region’s characteristics—including popu-
similar as possible. This problem is  -hard in case the maximal input lation, economic power, etc—is obtained (Nikolic, Djukanovic, &
number is exponential in 𝑛 (Mertens, 2006). In fact, the NPP is one of Matic, 2022).
Karp’s twenty-one basic  -hard problems (Karp, 1975). Moreover, • Forming commissions (teams) of experts (people) such that, in
its decision variant is  -complete. Practical applications of the NPP terms of certain competencies, the formed commissions are simi-
include multiprocessor scheduling (Coffman & Lueker, 1991), public
lar (Hacibeyoglu, Alaykiran, Acilar, Tongur, & Ulker, 2018).
key cryptography (Merkle & Hellman, 1978), and the assignment of
• Public-key cryptography (Pop & Matei, 2013a).
tasks in low-power application-specific integrated circuits (Ercegovac,
Kirovski, & Potkonjak, 1999). Other real-world applications of this problem are described in Faria, de
In this work, we deal with the so-called multidimensional multi-way Souza, and de Sá (2021) and Nikolic et al. (2022). In the following, we
number partitioning problem (MDMWNPP), which is notoriously hard provide a literature review for the main number partitioning problem
to solve. Given a set of 𝑛 𝑚-dimensional vectors, the MDMWNPP is variants.
concerned with finding a partitioning of the set of 𝑛 vectors into exactly

∗ Corresponding author.
E-mail addresses: [email protected] (M. Djukanović), [email protected] (A. Kartelj), [email protected] (C. Blum).

https://doi.org/10.1016/j.eswa.2023.120762
Received 17 September 2022; Received in revised form 22 March 2023; Accepted 6 June 2023
Available online 20 June 2023
0957-4174/© 2023 Elsevier Ltd. All rights reserved.
M. Djukanović et al. Expert Systems With Applications 232 (2023) 120762

1.1. Literature review remaining instance types cannot be drawn from the existing literature.
Moreover, the two algorithms were implemented in different pro-
The basic NPP (where 𝑚 = 1 and 𝑘 = 2) has been intensively studied gramming languages and they were executed on significantly different
within the last few decades. The efficient differencing method for solving computer hardware.
the NPP, proposed Karmarkar and Karp (1982), attracted a wide range
of researchers to work on solving the NPP. Note also that there exists 1.3. Contributions of this work
a tight relation between the NPP and quadratic programming since
the former can be expressed as an unconstrained quadratic binary pro- The main contributions of this work can be summarized as follows.
gram and solved efficiently by appropriate black-box solvers (Alidaee, Since the MDMWNPP is well studied in the context of mathematical
Glover, Kochenberger, & Rego, 2005). Apart from this, various efficient programming—remember the two existing MILP models, for example—
meta-heuristics were proposed, especially aimed at tackling large-sized we propose a MILP-based hybrid meta-heuristic called self-adaptive
instances. Examples include a tabu search (Alidaee et al., 2005), a ‘‘Construct, Merge, Solve and Adapt’’ (CMSA) algorithm enhanced by a
simulated annealing approach (Johnson, Aragon, McGeoch, & Schevon, local search procedure to solve the MDMWNPP. This algorithm features
1991), a quantum annealing (Graß, Raventós, Juliá-Díaz, Gogolin, & the following innovative aspects:
Lewenstein, 2016), and an artificial immune system approach (Corus,
Oliveto, & Yazdani, 2019). In addition, there are two incomplete tree • The black-box MILP solver is applied to efficiently solve a se-
search methods to solve the NPP (Pedroso & Kubo, 2010). An exact tree- ries of smaller sub-instances generated by merging a number of
search method was proposed (Korf, 1998). A number of randomized constructed solutions induced by the best-so-far found solution.
approaches to solve the NPP were also considered (Argüello, Feo, & • Parameters of the algorithm are carefully adjusted during a run,
Goldschmidt, 1996). Finally, a profound computational analysis on obtaining a more robust version of the basic CMSA algorithm.
solving the NPP was presented in Ducha and de Souza (2013). • Each generated solution is improved by means of a novel, efficient
Concerning problem generalizations, the NPP with arbitrary large local search procedure, which leads to a stronger search intensi-
values for 𝑘 was studied in various papers (Korf, 2009; Korf, Schreiber, fication behavior of the algorithm. The local search employs two
& Moffitt, 2014; Moffitt, 2013). Furthermore, the multi-dimensional two- different neighborhood structures, 1-move and 2-swap, explored
way number partitioning problem (MDTWNPP), is the first generalization within a two-phase procedure.
of the NPP w.r.t. parameter 𝑚, which considers vectors instead of
numbers as input. This problem was proposed in Kojić (2010) under The proposed algorithm, which is a hybrid of Adapt-Cmsa and LS, is
the assumption of a fixed value 𝑘 = 2. Concerning existing work, (Ko- henceforth labeled Adapt+Cmsa+Ls.
jić, 2010) proposed a mixed integer linear programming (MILP) for- A further contribution of our work is that we provide a comprehen-
mulation. In Ma and Ni (2013) a two-phase algorithm to solve the sive and fair comparison among all known exact and meta-heuristic
MDTWNPP was proposed. Further, two meta-heuristics, a Variable approaches from the literature concerning the MDMWNPP, including
Neighborhood Search (VNS) and an Electromagnetism-like metaheuris- our own Adapt-Cmsa+Ls approach. Before executing all algorithms on
tic were presented in Kratica, Kojić, and Savić (2014). GRASP with ex- the same computer and with the same resources, we actually re-
terior path relinking and restricted local search to solve the MDTWNPP implemented all algorithms in the same programming language (C++).
was introduced by Rodriguez, Glover, García-Martínez, Martí, and All source codes are made publicly available. Thus, this contributes to
Lozano (2017). A Genetic Algorithm (GA) and Simulated Annealing solving the literature issues, see Section 1.2.
(SA) were discussed in Hacibeyoglu et al. (2018). In Santucci, Baioletti, A thorough statistical evaluation is performed on the basis of the
Di Bari, and Milani (2019), the authors proposed a memetic algebraic obtained results, and appropriate conclusions are drawn. In short, the
differential evolution algorithm for binary search spaces (MADEB) to Adapt-Cmsa+Ls approach is able to significantly outperform all compet-
solve the MDTWNPP. They were able to show that their algorithm out- ing algorithms in terms of the obtained solution quality when applied
performed other state-of-the-art competitors in terms of the obtained to problem instances with moderate and large values of 𝑘, while GA
solution quality. In Santucci, Baioletti, and Di Bari (2021), the MADEB performs strongly in the context of instances with 𝑘 = 2.
algorithm was further improved by replacing it with a more robust
version—called iMADEB—utilizing a self-adaptive algebraic differen- 1.4. Organization of the Paper
tial mutation scheme on the basis of the Lévy flight concept (Jensi &
Jiji, 2016). This algorithm yielded a new state-of-the-art algorithm for This work is organized as follows. In Section 2, we present the model
the MDTWNPP. of the search space of the MDMWNPP. Section 3 presents the chosen
The MDMWNPP is the last generalization of the NPP that was MILP model from the literature whose role is significant for the quality
presented in Pop and Matei (2013b). In this problem version, arbitrary of our algorithm. Moreover, we describe our re-implementations of
values for 𝑚 and 𝑘 may be considered. Pop and Matei (2013b) proposed the two meta-heuristic approaches from the literature. In Section 4,
a genetic/memetic algorithm to tackle the problem. A VNS approach the proposed Adapt-Cmsa framework and its application to solve the
was introduced in Faria, de Souza, Souza, Silva, and Nazário Coelho MDMWNPP are described in detail. Section 5 reports on the exhaustive
(2018). Concerning exact approaches, recently, two MILP formulations numerical and statistical comparison on the basis of the obtained
were proposed: one was introduced in Faria et al. (2021) and the other results. Conclusions and an outline of future work are provided in
one in Nikolic et al. (2022). The former MILP model performs better Section 6.
than the latter one when 𝑘 is small, while the latter one is better for
larger values of 𝑘. 2. Problem definition and search space

1.2. Literature issues concerning the MDMWNPP Let 𝑆 = {𝑣1 , 𝑣2 , … , 𝑣𝑛 } be a set of 𝑛 vectors of a fixed length 𝑚 ∈ N,
that is, 𝑣𝑖 = (𝑣𝑖,1 , … , 𝑣𝑖,𝑚 ) ∈ R𝑚 , for 𝑖 = 1, … , 𝑛, and let 𝑘 ∈ {2, … , 𝑛} be
We emphasize that in Faria et al. (2018), a comparison between a fixed number.
the VNS and the GA approaches from Pop and Matei (2013b) was We say that  = (𝑆1 , … , 𝑆𝑘 ) is a 𝑘-partitioning of set 𝑆 if and only
only partially provided because the two approaches are compared only if the following three conditions are fulfilled:
on a subset of the standard benchmark set used in the literature. In
particular, type A instances are used, while the remaining instance • 𝑆𝑖 ≠ ∅, for 𝑖 ∈ {1, … , 𝑘}.
⋃
types (B, C, and D) were not considered. Therefore, conclusions for the • 𝑖∈{1,…,𝑘} 𝑆𝑖 = 𝑆.

2
M. Djukanović et al. Expert Systems With Applications 232 (2023) 120762

• 𝑆𝑝1 ∩ 𝑆𝑝2 = ∅, for all 𝑝1 , 𝑝2 ∈ {1, … , 𝑘} such that 𝑝1 ≠ 𝑝2 . • Binary variables 𝑥𝑖𝑗 , 𝑖 ∈ {1, … , 𝑛}, 𝑗 ∈ {1, … , 𝑘}, such that 𝑥𝑖𝑗 = 1
if vector 𝑣𝑖 is included in partition 𝑆𝑗 , otherwise 𝑥𝑖𝑗 = 0.
The MDMWNPP asks for finding a 𝑘-partitioning  = (𝑆1 , … , 𝑆𝑘 ) of
set 𝑆 that minimizes the following objective function: • Real variables 𝑦𝑙 , 𝑧𝑙 , 𝑙 ∈ {1, … , 𝑚} denote the maximum, respec-
tively minimum, sums of the numbers in the 𝑙th coordinate across
| |
| ∑ ∑ | all partitions.
| |
𝑓 () = max | 𝑣𝑖,𝑗 − 𝑣𝑖,𝑗 | (1) • A positive real variable 𝑟 that corresponds to the objective func-
𝑝1 ,𝑝2 ∈{1,…,𝑘}∣𝑝1 >𝑝2 ,𝑗∈{1,…,𝑚} | |
|𝑣𝑖 ∈𝑆𝑝1 𝑣𝑖 ∈𝑆𝑝 |
| 2 | tion:
In other words, the problem requires finding a 𝑘-partitioning of set
𝑟= max {𝑦𝑙 − 𝑧𝑙 }.
𝑆 such that the sums of the values at each coordinate for all subsets 𝑙∈{1,…,𝑚}
are as similar as possible. Any given MDMWNPP instance is henceforth The Coam model can then be stated as follows:
denoted by (𝑆, 𝑛, 𝑚, 𝑘).
Given this solution representation, the search space can be defined min 𝑟 (3)
as the set of all 𝑛-dimensional integer vectors 𝑠 = (𝑠1 , … , 𝑠𝑛 ), 1 ≤ 𝑠𝑖 ≤ 𝑘,
𝑠.𝑡.
𝑖 ∈ {1, … , 𝑛}. Hereby, 𝑠𝑖 = 𝑗 means that vector 𝑣𝑖 is assigned to partition
𝑆𝑗 . Moreover, such an 𝑠 represents a feasible solution if and only if for ∑
𝑘
each 𝑗 ∈ {1, … , 𝑘} there is at least one 𝑖 ∈ {1, … , 𝑛} such that 𝑠𝑖 = 𝑗, 𝑥𝑖𝑗 = 1, ∀𝑖 ∈ {1, … , 𝑛} (4)
i.e. there are no empty partitions. With 𝑗=1

max |𝑥 − 𝑦| = max 𝑥 − min 𝑥, ∑

𝑛
𝑥,𝑦∈𝑆 𝑥∈𝑆 𝑥∈𝑆 𝑥𝑖𝑗 ≥ 1, ∀𝑗 ∈ {1, … , 𝑘} (5)
𝑖=1
the objective function (1) can easily be rewritten in a more convenient
way: ∑
𝑛
𝑣𝑖𝑙 ⋅ 𝑥𝑖𝑗 ≤ 𝑦𝑙 , ∀𝑗 ∈ {1, … , 𝑘}, ∀𝑙 ∈ {1, … , 𝑚} (6)
⎛ ⎛∑ ⎞ ⎛∑ ⎞⎞ 𝑖=1
𝑓 () = max ⎜ max ⎜ 𝑣𝑖𝑙 ⎟ − min ⎜ 𝑣𝑖𝑙 ⎟⎟ (2) ∑𝑛
𝑙∈{1,…,𝑚} ⎜𝑗∈{1,…,𝑘} ⎜ ⎟ 𝑗∈{1,…,𝑘} ⎜ ⎟⎟ 𝑣𝑖𝑙 ⋅ 𝑥𝑖𝑗 ≥ 𝑧𝑙 , ∀𝑗 ∈ {1, … , 𝑘}, ∀𝑙 ∈ {1, … , 𝑚} (7)
⎝ ⎝𝑣𝑖 ∈𝑆𝑗 ⎠ ⎝𝑣𝑖 ∈𝑆𝑗 ⎠⎠
𝑖=1
Note that there is an obvious symmetry in this definition of the
search space of the MDMWNPP. For example, solutions (1, 1, 2) and 𝑦𝑙 − 𝑧𝑙 ≤ 𝑟, ∀𝑙 ∈ {1, … , 𝑚} (8)
(2, 2, 1) represent the same feasible solution for an example instance
𝑥𝑖𝑗 ∈ {0, 1}, ∀𝑖 ∈ {1, … , 𝑛}, ∀𝑗 ∈ {1, … , 𝑘} (9)
with 𝑛 = 3 and 𝑘 = 2. In order to eliminate redundant considerations
of symmetric solutions, the following symmetry breaking rule is applied. 𝑦𝑙 , 𝑧𝑙 ∈ R, ∀𝑙 ∈ {1, … , 𝑚} (10)
We only consider solutions  = (𝑆1 , … , 𝑆𝑘 ) such that, for all 𝑖 < 𝑗, the
vector with the lowest index in 𝑆𝑖 has a lower index than the vector 𝑟 ∈ [0, +∞) (11)
with the lowest index in 𝑆𝑗 .
Concerning the above example, for solution (1, 1, 2) the lowest index Constraints (4) ensure that each vector belongs to exactly one
of the vectors that belong to 𝑆1 is 1, while the lowest index of vectors in partition. Constraints (5) ensure that neither partition is empty. Con-
𝑆2 is 3. In other words, this solution is considered. On the other hand, straints (6)–(8) calculate the objective function whose value is stored
solution (2, 2, 1) is not considered since the lowest index of the vectors in variable 𝑟.
belonging to 𝑆1 is three, while the lowest index of the vectors belonging
to 𝑆2 is one. Note that solution (2, 2, 1) can easily be rearranged and 3.2. Re-implementation of the VNS algorithm for the MDMWNPP
transformed into solution (1, 1, 2). This also holds for any arbitrary
solution, and the computational cost of the rearrangement is 𝑂(𝑛). The VNS algorithm for the MDMWNPP, proposed in Faria et al.
Consequently, vector 𝑣1 always belongs to 𝑆1 . (2018), had to be re-implemented since neither the source code nor
binaries were available on the internet. In addition, we were not able
3. Approaches from the literature to obtain them from the authors. The re-implementation was done in
accordance with the VNS algorithm description, which consequently
In this section, we will describe three MDMWNPP approaches from produced results very similar to those obtained by the authors of the
the literature which are relevant for our Adapt-Cmsa+Ls approach as VNS approach.
well as for comparison purposes. In particular, we will describe the Algorithm 1 provides the pseudo-code of Vns for the MDMWNPP.
utilized MILP model and two meta-heuristic approaches, namely Vns The algorithm consists of the following steps:
and Ga, in the way in which they were re-implemented. This re-
implementation was necessary since source codes or binaries are not • First, the initial (and currently best) solution 𝑠𝑏𝑠𝑓 is constructed by
publicly available for these meta-heuristic approaches from the litera- using a simple greedy method, whose details can be found in Faria
ture. The basic principle that we followed in our re-implementations is et al. (2018).
as follows: in case of several ways to re-implement a certain aspect of a • After that, the algorithm iterates until the pre-defined CPU time
method (when the textual description from the original paper was not limit is reached. Each iteration consists of the following steps:
clear) we do it in a way that does not degrade the quality or efficiency
of the method. Details on our re-implementations are exhaustively – Shaking procedure in the neighborhood of size 𝑟, where
provided in the next sections, while source codes are made publicly parameter 𝑟𝑚𝑎𝑥 sets the maximal value of 𝑟. The authors of
available at https://github.com/kartelj/adapt-cmsa-mdmwnpp. VNS in Faria et al. (2018) used 𝑟𝑚𝑎𝑥 = 3 in their work. This
means that 𝑟 vectors are randomly selected from partitions
3.1. MILP model of 𝑠𝑏𝑠𝑓 . Each one of these vectors is subsequently moved to a
randomly chosen partition, thus producing a new candidate
In this section, we present the MILP model (Nikolic et al., 2022)— solution 𝑠𝑛𝑒𝑤 .
labeled Coam—that is used internally in Adapt-Cmsa+Ls. The second – Afterwards, a local search procedure named LS1best() is
available MILP model (Faria et al., 2021) is not described here, even applied to 𝑠𝑛𝑒𝑤 .
though later it is included in the experimental evaluation. – Depending on the value of 𝑟, additional local searches
The following variables are being used by Coam: LS2best() or LS3best() are applied.

3
M. Djukanović et al. Expert Systems With Applications 232 (2023) 120762

Algorithm 1 Vns for MDMWNPP respectively an executable, we finally decided—as in the case of Vns—
to re-implement this algorithm, which is henceforth simply labeled
1: Input: a problem instance  = (𝑆, 𝑛, 𝑚, 𝑘), 𝑟𝑚𝑎𝑥 – max. neighborhood
Ga. The re-implementation was done in accordance with the algorithm
size description given in Pop and Matei (2013b). However, due to several
2: Output: 𝑠𝑏𝑠𝑓 , the best solution found within the time limit ambiguities in the description provided in Pop and Matei (2013b), we
3: 𝑠𝑏𝑠𝑓 ←InitializeSolution() had to make a few implementation decisions that possibly deviate from
4: 𝑟 ← 1 the idea the original authors had in mind. They will be explained in the
further text.
5: while time limit is not reached do
6: 𝑠𝑛𝑒𝑤 ← Shake(𝑠𝑏𝑠𝑓 , 𝑟)
Algorithm 2 Ga for MDMWNPP
7: 𝑠𝑛𝑒𝑤 ← LS1best(𝑠𝑛𝑒𝑤 )
1: Input: a problem instance  = (𝑆, 𝑛, 𝑚, 𝑘), 𝑝𝑜𝑝𝑠𝑖𝑧𝑒 – population size,
8: if 𝑟 = 2 then
𝑐𝑟𝑜𝑠𝑠𝑝𝑟𝑜𝑏 – crossover probability, 𝑚𝑢𝑡𝑝𝑟𝑜𝑏 – mutation probability
9: 𝑠𝑛𝑒𝑤 ← LS2best(𝑠𝑛𝑒𝑤 )
2: Output: the best solution found within the time limit
10: else if 𝑟 = 3 then
3: 𝑝𝑜𝑝 ←InitializePopulation(, 𝑝𝑜𝑝𝑠𝑖𝑧𝑒 )
11: 𝑠𝑛𝑒𝑤 ← LS3best(𝑠𝑛𝑒𝑤 )
4: for 𝑖 = 1 to 𝑝𝑜𝑝𝑠𝑖𝑧𝑒 do
12: end if
5: 𝑝𝑜𝑝[𝑖] ← LS(𝑝𝑜𝑝[𝑖])
13: if 𝑓 (𝑠𝑛𝑒𝑤 ) < 𝑓 (𝑠𝑏𝑠𝑓 ) then
6: end for
14: 𝑠𝑏𝑠𝑓 ← 𝑠𝑛𝑒𝑤
7: Sort(𝑝𝑜𝑝)
15: 𝑟←1
8: while time limit is not reached do
16: else
9: 𝑜𝑣𝑒𝑟𝑝𝑜𝑝𝑠𝑖𝑧𝑒 ← 10 ⋅ 𝑝𝑜𝑝𝑠𝑖𝑧𝑒
17: 𝑟 ← 1 + (𝑟 mod 𝑟𝑚𝑎𝑥 )
10: 𝑜𝑣𝑒𝑟𝑝𝑜𝑝 ← [𝑝𝑜𝑝[1], 𝑝𝑜𝑝[2]]
18: end if
11: for 𝑖 = 2 to 𝑜𝑣𝑒𝑟𝑝𝑜𝑝𝑠𝑖𝑧𝑒 ∕2 do
19: end while
12: 𝑝𝑎𝑟𝑒𝑛𝑡1, 𝑝𝑎𝑟𝑒𝑛𝑡2 ←Select(𝑝𝑜𝑝),Select(𝑝𝑜𝑝)
20: return 𝑠𝑏𝑠𝑓
13: 𝑟 ← 𝑈 ([0, 1])
14: if 𝑟 ≤ 𝑐𝑟𝑜𝑠𝑠𝑝𝑟𝑜𝑏 then
15: 𝑐ℎ𝑖𝑙𝑑1, 𝑐ℎ𝑖𝑙𝑑2 ←Crossover(𝑝𝑎𝑟𝑒𝑛𝑡1, 𝑝𝑎𝑟𝑒𝑛𝑡2)
– If the new solution 𝑠𝑛𝑒𝑤 , after application of shaking and
16: else
the local search procedure(s), is better than 𝑠𝑏𝑠𝑓 , then 𝑠𝑛𝑒𝑤
becomes the current best solution and the shaking neighbor- 17: 𝑐ℎ𝑖𝑙𝑑1, 𝑐ℎ𝑖𝑙𝑑2 ← 𝑝𝑎𝑟𝑒𝑛𝑡1, 𝑝𝑎𝑟𝑒𝑛𝑡2
hood size is reset to 1. 18: end if
– Otherwise, the shaking neighborhood size is cyclically in- 19: 𝑐ℎ𝑖𝑙𝑑1, 𝑐ℎ𝑖𝑙𝑑2 ←Mutate(𝑐ℎ𝑖𝑙𝑑1, 𝑚𝑢𝑡𝑝𝑟𝑜𝑏 ),
creased w.r.t. 𝑟𝑚𝑎𝑥 . Mutate(𝑐ℎ𝑖𝑙𝑑2, 𝑚𝑢𝑡𝑝𝑟𝑜𝑏 )
The details of local search procedures LS1best(), LS2best() and 20: 𝑐ℎ𝑖𝑙𝑑1, 𝑐ℎ𝑖𝑙𝑑2 ←LS(𝑐ℎ𝑖𝑙𝑑1), LS(𝑐ℎ𝑖𝑙𝑑2)
LS3best() are described in Faria et al. (2018). Shortly, LS1best() 21: 𝑜𝑣𝑒𝑟𝑝𝑜𝑝 ← 𝑜𝑣𝑒𝑟𝑝𝑜𝑝 + [𝑐ℎ𝑖𝑙𝑑1, 𝑐ℎ𝑖𝑙𝑑2]
tries to find the best vector to move from its current partition to 22: end for
some other partition. Similarly, LS2best() finds the best simultaneous 23: Sort(𝑜𝑣𝑒𝑟𝑝𝑜𝑝)
movement of two vectors, while LS3best() finds the best simulta-
24: 𝑝𝑜𝑝 ← 𝑜𝑣𝑒𝑟𝑝𝑜𝑝[1 ∶ 𝑝𝑜𝑝𝑠𝑖𝑧𝑒 ]
neous movement of three vectors. All these procedures are executed
as long as there is an improvement. We also make use of the partial 25: end while
(incremental) calculation of the objective function values as described 26: return 𝑝𝑜𝑝[1]
in Faria et al. (2018).
We made a small change in the re-implementation compared to
Algorithm 2 presents the general scheme of Ga for the MDMWNPP.
the description of the original VNS. Namely, in the original work
The algorithm consists of the following steps.
LS1best() was applied only when 𝑟 = 1, while in our
re-implementation LS1best() is always applied. This change seemed • First, the initial population of chromosomes (𝑝𝑜𝑝) is generated.
to be beneficial for the overall VNS results, because LS1best() is This is done in the same way as the authors describe it in
relatively cheap in terms of computational complexity, and it often their paper, i.e. partially randomly and partially based on the
provides improvements. problem structure. The initial population is afterward improved
An additional difference is related to the application to large prob- by applying a local search procedure to each chromosome.
lem instances, which were not considered in Faria et al. (2018). While • After that, the algorithm iterates until the CPU time limit is
in Faria et al. (2018), the authors only considered instances with up to reached. Each iteration consists of the following steps.
100 input vectors, in this work we consider much larger instances with
up to 500 input vectors. In particular, using 𝑟𝑚𝑎𝑥 = 3 turned out to be – An intermediate population of offspring chromosomes
prohibitive in the case of many input vectors. This will be outlined in (𝑜𝑣𝑒𝑟𝑝𝑜𝑝) is generated. Hereby, the size of 𝑜𝑣𝑒𝑟𝑝𝑜𝑝 is 10
more detail in Section 5.1.3. times larger than the size of population 𝑝𝑜𝑝.
– The best two chromosomes from the population are directly
3.3. Re-implementation of the GA algorithm for MDMWNPP copied to the first two positions of the intermediate popu-
lation, in order to avoid losing the best solutions through
The genetic algorithm making use of local search for the MDMWNPP the iterations. In fact, this is the first difference w.r.t. Pop
was proposed in Pop and Matei (2013b). As the source code and/or and Matei (2013b). We believe this change is beneficial for
an executable of this algorithm was not publicly available, and as the the overall performance of the algorithm. The remaining
authors of the algorithm were not able to provide the source code, chromosomes of the intermediate population are generated

4
M. Djukanović et al. Expert Systems With Applications 232 (2023) 120762

by the iterative use of standard genetic operators (selection, of combinatorial optimization problems (COPs) and proved to be one
crossover, and mutation) and local search. of the most efficient MILP-based meta-heuristics. Example applica-
tions include the one to the unbalanced minimum common string
∗ The selection of parents is tournament-based with a
partition problem (Blum, 2016), the repetition-free longest common
tournament size of 2.
subsequence problem (Blum & Blesa, 2016), the taxi sharing prob-
∗ Child chromosomes are formed by combining two par-
lem (Ben-Smida et al., 2019), the weighted independent domination
ent chromosomes via 1-point crossover and a probabil-
problem (Davidson, Blum, & Lozano, 2018), and the maximum happy
ity of 𝑐𝑟𝑜𝑠𝑠𝑝𝑟𝑜𝑏 . In the remaining cases, no crossover is
vertices problem (Lewis, Thiruvady, & Morgan, 2019).
performed. As mentioned above, the main idea of CMSA consists of the itera-
∗ Mutation is performed on both children with a per- tive construction of sub-instances (samples) that are further solved by
vector probability of 𝑚𝑢𝑡𝑝𝑟𝑜𝑏 . This means that, with a means of state-of-the-art (exact) solvers. Each iteration of the algorithm
probability of 𝑚𝑢𝑡𝑝𝑟𝑜𝑏 , a vector is moved from its cur- basically consists of four steps that can roughly be described as follows.
rent partition to another randomly chosen partition.
∗ A local search procedure (see below) is applied to each • The construct phase: a number of solutions are probabilistically
child. generated (usually biased by an efficient greedy heuristic).
∗ Both children are added to the intermediate popula- • The merge phase: those solution components that are (1) present
tion. in at least one of the generated solutions and (2) do not already
form part of the sub-instance are added to the (initially empty)
– The intermediate population 𝑜𝑣𝑒𝑟𝑝𝑜𝑝 is sorted w.r.t. the sub-instance of the tackled problem instance. Moreover, their
objective function value and the best 𝑝𝑜𝑝𝑠𝑖𝑧𝑒 chromosomes so-called age values are set to zero.
become the members of the next population 𝑝𝑜𝑝. • The solve phase: the updated sub-instance is solved by a solver—
such as a MILP solver—under a given computation time limit.
The local search procedure from Pop and Matei (2013b) consists of
three parts: • The adapt phase: this phase depends on the outcome of the solver.
First, it is checked if the solution returned by the solver is a new
1. The cheapest part is called 1-change neighbor. According to the best solution. Furthermore, those solution components that form
authors, this procedure is based on the random selection of a part of the returned solution are regarded as promising by setting
vector (a locus in the chromosome) and its move to a random their age value to zero. The remaining solution components in
partition. Its reported complexity is 𝑂(𝑛), which leads us to the sub-instance are penalized by incrementing their age value.
believe that the actual implementation is not consistent with its Finally, those solution components in the sub-instance whose age
textual description—complexity 𝑂(𝑛) suggests that local search is value has reached a limit called 𝑎𝑔𝑒max are removed from the
exhaustive (rather than random). Therefore, instead of randomly sub-instance.
selecting vectors, we randomly selected just its starting position
As most other metaheuristics, CMSA is usually terminated after reach-
(to avoid positional bias), after which all vectors from that po-
ing a pre-defined computation time limit. As noted in Akbay, López
sition onward (cyclically) were probed. Probing, as described in
Serrano, and Blum (2022), the original CMSA may sometimes be sen-
the paper, consists of choosing a random partition to which the
sitive to the values of its control parameters. This holds especially in
vector is moved. If improvement occurs, the change is immedi-
those cases in which the set of problem instances to be tackled is rather
ately applied (first-improvement strategy) and 1-change neighbor
diverse. Therefore, as is the case for most other metaheuristics, control
LS is restarted.
parameter tuning usually needs to be performed separately for different
2. If there is no further 1-change neighbor improvement, the so-
instance types. In order to alleviate this computational burden, Akbay
called 2-change neighbor LS is executed. According to Pop and
et al. (2022) recently proposed a self-adaptive version of CMSA, denoted
Matei (2013b), it consists of the random selection of two vectors,
by Adapt-Cmsa. This algorithm variant can be seen as a more robust
after which their partitions are swapped. The complexity of
version of the original CMSA. There are mainly two differences between
this LS is 𝑂(𝑛2 ). Again, this leads us to believe that the textual
Adapt-Cmsa and the original CMSA:
description is not in accordance with the actual implementation.
Therefore, our re-implementation uses exhaustive probing over 1. The parameter that controls the number of solutions generated
pairs of vectors (in a random fashion to avoid positional bias). per iteration is dynamically adjusted over time, depending on
Again, in case of an improvement, the change is immediately the search history.
applied, and the 2-change neighbor LS is restarted. 2. Parameter 𝑎𝑔𝑒max and the mechanism of sub-instance updating
3. Last, there is also a 3-change neighbor local search, based on is replaced by a self-adaptive parameter 𝛼𝑏𝑠𝑓 where 0 ≤ 𝛼𝐿𝐵 ≤
the same idea as 2-change neighbor. The complexity of this LS 𝛼𝑏𝑠𝑓 ≤ 𝛼𝑈 𝐵 ≤ 1. This parameter controls the balance between
is 𝑂(𝑛3 ). diversification and intensification depending on the obtained
solution quality w.r.t. solution 𝑠𝑏𝑠𝑓 , the best solution found so
Since all local search procedures are making a high number of
far. Moreover, the lower bound 𝛼𝐿𝐵 and the upper bound 𝛼𝑈 𝐵
objective function calls, we have implemented a fast (incremental)
for parameter 𝛼𝑏𝑠𝑓 are fixed. Their value is subject to parameter
objective function, similar as the one used by Vns. For the experimental
tuning.
evaluation reported in Pop and Matei (2013b), the Ga parameters were
set to following values: 𝑐𝑟𝑜𝑠𝑠𝑝𝑟𝑜𝑏 = 0.85, 𝑚𝑢𝑡𝑝𝑟𝑜𝑏 = 0.1 and 𝑝𝑜𝑝𝑠𝑖𝑧𝑒 = 10𝑛. In this work we adopt the general mechanism of Adapt-Cmsa from
Instead of simply adopting these parameter value settings, Ga is subject Akbay et al. (2022) in order to solve the MDMWNPP. However, as
to parameter tuning as described in Section 5.1. we found it necessary in the case of the MDMWNPP to improve
generated solutions by applying a local search procedure, we denote
4. Adapt -Cmsa+Ls for the MDMWNPP our algorithm by Adapt-Cmsa+Ls. Its pseudo-code is given in Algorithm
3.
‘‘Construct, Merge, Solve & Adapt’’ (CMSA) is a hybrid meta- Before we describe the algorithm, remember that 𝑠 = (𝑠1 , … , 𝑠𝑛 )
heuristic originally proposed in Blum, Pinacho, López-Ibáñez, and where 1 ≤ 𝑠𝑖 ≤ 𝑘 for all 𝑖 = 1, … , 𝑛 denotes a solution to the
Lozano (2016). Most of the existing applications of CMSA make internal problem. Hereby, vector 𝑖 is assigned to partition 𝑠𝑖 in solution 𝑠,
use of MILP solvers for solving sub-instances of the tackled problem for all 𝑖 = 1, … , 𝑛. In the following, we call (𝑖, 𝑠𝑖 ) a solution com-
instance. Since it was introduced, CMSA has been applied to a variety ponent. In this way, a solution 𝑠 = (𝑠1 , … , 𝑠𝑛 ) consists of the set

5
M. Djukanović et al. Expert Systems With Applications 232 (2023) 120762

Algorithm 3 Adapt-Cmsa+Ls for the MDMWNPP components from these solutions are added to 𝐶 ′ (if not already there).
1: Input: a problem instance  = (𝑆, 𝑛, 𝑚, 𝑘) and Adapt-Cmsa+Ls Moreover, 𝑠𝑏𝑠𝑓 is updated in case a new best-so-far solution is found.

parameters 𝑡𝑝𝑟𝑜𝑝 , 𝑡𝐼𝐿𝑃 , 𝛼𝐿𝐵 , 𝛼𝑈 𝐵 , 𝛼𝑟𝑒𝑑 Next, the current sub-instance 𝐶 ′ is (approximately) solved in func-
tion 𝙰𝚙𝚙𝚕𝚢𝙴𝚡𝚊𝚌𝚝𝚂𝚘𝚕𝚟𝚎𝚛(𝐶 ′ , 𝑡𝐼𝐿𝑃 ) as follows. The MILP solver CPLEX is
2: Output: 𝑠𝑏𝑠𝑓 , the best solution found within the time limit
applied to a restricted version of the Coam model from Section 3. This
3: 𝑠𝑏𝑠𝑓 ← GenerateRandomSolution()
restricted MILP model is obtained by adding the following constraints
4: 𝑠𝑏𝑠𝑓 ← LS(𝑠𝑏𝑠𝑓 ) to Coam:
5: 𝑛𝑎 ← 1; 𝛼𝑏𝑠𝑓 ← 𝛼𝑈 𝐵
6: Initialize the sub-instance 𝐶 ′ with the solution components of 𝑠𝑏𝑠𝑓 𝑥𝑐 ′ ,𝑐 ′′ = 0, ∀𝑐 = (𝑐 ′ , 𝑐 ′′ ) ∉ 𝐶 ′ (12)
7: while CPU time limit is not reached do
This restricted model w.r.t. the set of solution components present in
8: for 𝑖 = 1 to 𝑛𝑎 do 𝐶 ′ with constraints (12) will be denoted by Coam(𝐶 ′ ) in the further text.
9: 𝑠← ProbabilisticSolutionGeneration(𝑠𝑏𝑠𝑓 , 𝛼𝑏𝑠𝑓 ) For example, if a sub-instance 𝐶 ′ , concerning some vector 𝑣𝑖 , contains
10: 𝑠 ← LS(𝑠) the following solution components, {(𝑖, 𝑗), (𝑖, 𝑘)}, it means that we only
11: if 𝑓 (𝑠) < 𝑓 (𝑠𝑏𝑠𝑓 ) then 𝑠𝑏𝑠𝑓 ← 𝑠 end if allow placing vector 𝑣𝑖 in partition 𝑗 or in partition 𝑘. Note that CPLEX
12: Add the solution components of 𝑠 to 𝐶 ′ (if not already in 𝐶 ′ ) is given a computation time limit of 𝑡𝐼𝐿𝑃 CPU seconds for solving
13: end for the restricted MILP at each iteration. Additionally, the time needed to
solve the sub-instance is monitored and stored in 𝑡𝑠𝑜𝑙𝑣𝑒 . Afterwards, the
14: (𝑠′𝑜𝑝𝑡 , 𝑡𝑠𝑜𝑙𝑣𝑒 ) ← ApplyExactSolver(𝐶 ′ , 𝑡𝐼𝐿𝑃 )
solution produced by the solver is possibly improved by applying the
15: 𝑠′𝑜𝑝𝑡 ← LS(𝑠′𝑜𝑝𝑡 )
LS() procedure.
16: if 𝑡𝑠𝑜𝑙𝑣𝑒 < 𝑡𝑝𝑟𝑜𝑝 ⋅ 𝑡𝐼𝐿𝑃 ∧ 𝛼𝑏𝑠𝑓 > 𝛼𝐿𝐵 then
Next, parameter 𝛼𝑏𝑠𝑓 value is adapted w.r.t. (1) the computation
17: 𝛼𝑏𝑠𝑓 ← 𝛼𝑏𝑠𝑓 − 𝛼𝑟𝑒𝑑 // make the subinstance harder
time 𝑡𝑠𝑜𝑙𝑣𝑒 required to solve the present sub-instance and (2) the quality
18: end if of the obtained solution 𝑠′𝑜𝑝𝑡 . Namely, if 𝑡𝑠𝑜𝑙𝑣𝑒 is low—that is, within a ra-
19: if 𝑓 (𝑠′𝑜𝑝𝑡 ) < 𝑓 (𝑠𝑏𝑠𝑓 ) then tio 𝑡𝑝𝑟𝑜𝑝 of the allowed solver time 𝑡𝐼𝐿𝑃 —the sub-instance is considered
20: 𝑠𝑏𝑠𝑓 ← 𝑠′𝑜𝑝𝑡 to be easy to solve. Therefore, a stronger diversification is needed for
21: 𝑛𝑎 ← 1 the probabilistic construction of solutions in function Probabilis-
22: 𝛼𝑏𝑠𝑓 ← 𝛼𝑈 𝐵 ticSolutionGeneration(𝑠𝑏𝑠𝑓 , 𝛼𝑏𝑠𝑓 ). This is achieved by decreas-
23: else if 𝑓 (𝑠′𝑜𝑝𝑡 ) > 𝑓 (𝑠𝑏𝑠𝑓 ) then ing 𝛼𝑏𝑠𝑓 by a value 𝛼𝑟𝑒𝑑 . Of course, in case 𝛼𝑏𝑠𝑓 ≤ 𝛼𝐿𝐵 , where 𝛼𝐿𝐵
24: if 𝑛𝑎 = 1 then is the lower bound for 𝛼𝑏𝑠𝑓 , 𝛼𝑏𝑠𝑓 remains unchanged. In those cases
𝛼𝑟𝑒𝑑 in which 𝑠′𝑜𝑝𝑡 is better than 𝑠𝑏𝑠𝑓 , 𝑠𝑏𝑠𝑓 is updated accordingly, 𝑛𝑎 is
25: 𝛼𝑏𝑠𝑓 ← min{𝛼𝑏𝑠𝑓 + 10
, 𝛼𝑈 𝐵 }
set to 1, while 𝛼𝑏𝑠𝑓 = 𝛼𝑈 𝐵 . This implies that exploration will be
26: else
performed in the close neighborhood of the new best-so-far solution
27: 𝑛𝑎 ← 1 within the next iteration (increased intensification). On the other hand,
28: 𝛼𝑏𝑠𝑓 ← 𝛼𝑈 𝐵 in case 𝑠′𝑜𝑝𝑡 and 𝑠𝑏𝑠𝑓 are equal, we increment 𝑛𝑎 , which is done for
29: end if the following reason. By increasing the number of probabilistically
30: else generated solutions that enter the sub-instance, the sub-instance will
31: 𝑛𝑎 ← 𝑛𝑎 + 1 be larger and the search will be given a higher chance to improve over
32: end if 𝑠𝑏𝑠𝑓 (increased diversification). Finally, in the last remaining case—that
is, if 𝑠′𝑜𝑝𝑡 is worse than 𝑠𝑏𝑠𝑓 , the intensification around 𝑠𝑏𝑠𝑓 should be
33: Re-initialize the sub-instance 𝐶 ′ with the solution components
increased. There are two ways for making intensification stronger under
of 𝑠𝑏𝑠𝑓
these circumstances. Firstly, when 𝑛𝑎 = 1, i.e. the number of random
34: end while
solutions around 𝑠𝑏𝑠𝑓 is already minimal, intensification is increased by
35: return 𝑠𝑏𝑠𝑓 slightly increasing 𝛼𝑏𝑠𝑓 , which has the effect that the probabilistically
generated solutions become more similar to 𝑠𝑏𝑠𝑓 . Otherwise, if 𝑛𝑎 > 1,
we reset 𝑛𝑎 to one and we set 𝛼𝑏𝑠𝑓 = 𝛼𝑈 𝐵 . In this way, intensification
{(1, 𝑠1 ), … , (𝑛, 𝑠𝑛 )} of solution components. Moreover, given an instance will be increased maximally in order to fully focus the search on the
(𝑆, 𝑛, 𝑚, 𝑘), the set of all possible solution components is as follows: close neighborhood around 𝑠𝑏𝑠𝑓 .
𝐶 = {(1, 1), … , (1, 𝑘), (2, 1), … , (2, 𝑘), … , (𝑛, 𝑘)}. Note that at the end of a major iteration of Adapt-Cmsa+Ls, 𝐶 ′ is re-
As shown in Algorithm 3, our Adapt-Cmsa+Ls algorithm starts by initialized by the solution components of the best-so-far solution 𝑠𝑏𝑠𝑓 .
generating a random solution which is subsequently improved by local This concludes the description of the main algorithm. In the remainder
search. This solution is stored as the first best-so-far solution 𝑠𝑏𝑠𝑓 . The of this section the functions for generating a random solution, for
procedure for producing a valid random solution is explained in the probabilistically generating a solution with a bias towards 𝑠𝑏𝑠𝑓 , and for
next section. Furthermore, the number of solutions (𝑛𝑎 ) is set to one, local search are described in detail.
the value of variable 𝛼𝑏𝑠𝑓 is set to the upper bound 𝛼𝑈 𝐵 , and the sub- Function GenerateRandomSolution(): First, 𝑘 vectors are cho-
instance 𝐶 ′ is initialized by adding all solution components of 𝑠𝑏𝑠𝑓 . sen uniformly at random from 𝑆. Subsequently, they assigned them
Thereafter, 𝑛𝑎 random solutions are generated by a utilizing procedure to different partitions. In this way, solution feasibility can be assured
ProbabilisticSolutionGeneration(𝑠𝑏𝑠𝑓 , 𝛼𝑏𝑠𝑓 ). The generation at the end of the solution generation. In a second step, the remain-
of solutions at each iteration is biased by 𝛼𝑏𝑠𝑓 : the larger the value ing vectors—that are not yet assigned to partitions—are assigned to
of 𝛼𝑏𝑠𝑓 , the larger will be the similarity between a (probabilistically) random partitions.
generated solution and 𝑠𝑏𝑠𝑓 , and vice-versa. Each generated solution Function ProbabilisticSolutionGeneration(𝑠𝑏𝑠𝑓 , 𝛼𝑏𝑠𝑓 ):
is improved by means of the LS() procedure. After that, all solution The core idea of this method is to construct random solutions whose

6
M. Djukanović et al. Expert Systems With Applications 232 (2023) 120762

Algorithm 4 Function ProbabilisticSolutionGenera- Algorithm 5 Function LS(𝑠)

tion(𝑠𝑏𝑠𝑓 , 𝛼𝑏𝑠𝑓 ) 1: Input: solution 𝑠
1: Input: 𝑠𝑏𝑠𝑓 , 𝛼𝑏𝑠𝑓 2: Output: possibly improved solution 𝑠𝐿𝑆
2: Output: a mutated solution 𝑠′𝑏𝑠𝑓 3: 𝑠𝐿𝑆 ← 𝑠
3: 𝑠′𝑏𝑠𝑓 ← 𝑠𝑏𝑠𝑓 4: 𝑏𝑒𝑠𝑡𝑓 𝑖𝑡 ← 𝑓 (𝑠)
4: for 𝑖 = 1 to 𝑛 do 5: 𝑖𝑚𝑝𝑟𝑜𝑣𝑒𝑑 ← 𝑇 𝑟𝑢𝑒
5: if the ‘‘𝑠′𝑏𝑠𝑓 [𝑖]"-th partition contains at least two vectors then 6: while 𝑖𝑚𝑝𝑟𝑜𝑣𝑒𝑑 do

6: 𝑟 ← 𝑈 ([0, 1]) 7: while 𝑖𝑚𝑝𝑟𝑜𝑣𝑒𝑑 do

8: 𝑖𝑚𝑝𝑟𝑜𝑣𝑒𝑑 ← 𝐹 𝑎𝑙𝑠𝑒
7: if 𝑟 ≤ 1 − 𝛼𝑏𝑠𝑓 then
9: for 𝑖 = 1 𝑡𝑜 𝑛 do// 1-move LS with the best-improvement
8: 𝑝𝑜𝑠 ← 𝑈 ({1, … , 𝑘})
strategy
9: 𝑠′𝑏𝑠𝑓 [𝑖] ← 𝑝𝑜𝑠
10: for 𝑝 = 1 𝑡𝑜 𝑘 do
10: end if 11: if 𝑝 allowed then // moving to 𝑝 produces feasible
11: end if solution
12: end for 12: 𝑠𝐿𝑆 [𝑖] ← 𝑝
13: if 𝑓 (𝑠𝐿𝑆 ) < 𝑏𝑒𝑠𝑡𝑓 𝑖𝑡 then
14: 𝑏𝑒𝑠𝑡𝑓 𝑖𝑡 ← 𝑓 (𝑠𝐿𝑆 )
structural closeness to 𝑠𝑏𝑠𝑓 is controlled by means of parameter 𝛼𝑏𝑠𝑓 . 15: 𝑏𝑒𝑠𝑡𝑖 ← 𝑖, 𝑏𝑒𝑠𝑡𝑝 ← 𝑝, 𝑖𝑚𝑝𝑟𝑜𝑣𝑒𝑑 ← 𝑇 𝑟𝑢𝑒
More precisely, as 𝛼𝑏𝑠𝑓 gets closer to 0, the similarity between solutions 16: end if
generated by the procedure and 𝑠𝑏𝑠𝑓 decreases. On the contrary, as 17: 𝑠𝐿𝑆 [𝑖] ← 𝑠[𝑖]
the value of 𝛼𝑏𝑠𝑓 gets closer to 1, the generated solutions become 18: end if
structurally more and more similar to 𝑠𝑏𝑠𝑓 . For example, if 𝛼𝑏𝑠𝑓 = 1, 19: end for
the method will simply return solution 𝑠𝑏𝑠𝑓 . However, this specific case 20: end for
is not of particular interest. The method that we designed is given in 21: if 𝑖𝑚𝑝𝑟𝑜𝑣𝑒𝑑 then 𝑠[𝑏𝑒𝑠𝑡𝑖 ] ← 𝑏𝑒𝑠𝑡𝑝 end if // continue 1-move
Algorithm 4. In particular, this method applies a mutation scheme as LS
known from the field of Evolutionary Computation (Back, Fogel, & 22: end while
Michalewicz, 1997). Each vector is assigned a probability of 1 − 𝛼𝑏𝑠𝑓 23: for 𝑖 = 1 𝑡𝑜 𝑛 − 1 do
to be relocated to a randomly chosen partition (including the one 24: for 𝑗 = 𝑖 + 1 𝑡𝑜 𝑛 do
it is currently assigned to). The obtained solution is then rearranged 25: 𝑝𝑖 ← 𝑠𝐿𝑆 [𝑖], 𝑠𝐿𝑆 [𝑖] ← 𝑠𝐿𝑆 [𝑗], 𝑠𝐿𝑆 [𝑗] ← 𝑝𝑖 // swapping
according to the rule of symmetry breaking outlined in Section 2. partitions
26: if 𝑓 (𝑠𝐿𝑆 ) < 𝑏𝑒𝑠𝑡𝑓 𝑖𝑡 then
Function LS(𝑠): during the development of our algorithm we re-
27: 𝑏𝑒𝑠𝑡𝑓 𝑖𝑡 ← 𝑓 (𝑠𝐿𝑆 )
alized that the application of local search (LS) for improving the
28: 𝑏𝑒𝑠𝑡𝑖 ← 𝑖, 𝑏𝑒𝑠𝑡𝑗 ← 𝑗
generated solutions significantly improved the performance of the al-
29: 𝑖𝑚𝑝𝑟𝑜𝑣𝑒𝑑 ← 𝑇 𝑟𝑢𝑒
gorithm. As already mentioned before, LS is called in two occasions.
30: end if
First, LS is applied both after the generation of the initial solution and
31: 𝑝𝑖 ← 𝑠𝐿𝑆 [𝑖], 𝑠𝐿𝑆 [𝑖] ← 𝑠𝐿𝑆 [𝑗], 𝑠𝐿𝑆 [𝑗] ← 𝑝𝑖 // swapping
after the probabilistic generation of solutions at each iteration. Second,
back
LS is applied to the solution returned by the exact solver (note that
32: end for
when the exact solver verifies optimality, local search is redundant).
33: end for
Calling LS after the generation of a solution reduces the diversity of
34: if 𝑖𝑚𝑝𝑟𝑜𝑣𝑒𝑑 then
solution components, which results in a stronger intensification than
35: 𝑝𝑖 ← 𝑠[𝑏𝑒𝑠𝑡𝑖 ], 𝑠[𝑏𝑒𝑠𝑡𝑖 ] ← 𝑠[𝑏𝑒𝑠𝑡𝑗 ], 𝑠[𝑏𝑒𝑠𝑡𝑗 ] ← 𝑝𝑖
when not using LS at all. On the other hand, applying LS to the output
36: end if
of the exact solver does not change the intensification/diversification
37: end while
balance of the algorithm. It rather enhances the solving phase in those
38: return 𝑠𝐿𝑆
scenarios in which the sub-instance is not solved to optimality. The LS
pseudo-code is given in Algorithm 5. It consists of two parts:
5. Experimental evaluation
• In the first phase—see lines 7–22—local search based on the 1-
move strategy is performed until no further improvements can be
The following approaches for solving the MDMWNPP are included
found in this way. Note that this neighborhood is explored in a
in the experimental comparison presented in this section:
best-improvement way.
• The second phase—see lines 23–36—consists in swapping those
• CPLEX applied to the MILP model (Faria et al., 2021), labeled
two vectors from different partitions that results in the best Faria;
improvement possible. In other words, exactly one move in the • CPLEX applied to the MILP model (Nikolic et al., 2022), labeled
2-swap neighborhood (best-improvement) is performed. In case of Coam;
success, the LS procedure switches again to the first phase. • Our re-implementation of the VNS approach (Faria et al., 2018),
as explained in Section 3.2 and labeled Vns;
It is important to note that—in all occasions—a fast (incremental) • Our re-implementation of the GA algorithm (Pop & Matei, 2013b)
calculation of the objective function is performed. After the application as explained in Section 3.3 and labeled Ga;
of LS, the obtained local optimum is then rearranged according to the • Our Adapt-Cmsa+Ls approach, as presented in Section 4.
rule of symmetry breaking (see Section 2).

7
M. Djukanović et al. Expert Systems With Applications 232 (2023) 120762

Table 1 Table 3
Domains of the Adapt-Cmsa+Ls parameters. Domains of Ga algorithm parameters.
𝑡𝐼𝐿𝑃 𝛼𝐿𝐵 𝛼𝑈 𝐵 𝛼𝑟𝑒𝑑 𝑡𝑝𝑟𝑜𝑝 𝑝𝑜𝑝𝑠𝑖𝑧𝑒 (×𝑛) 𝑐𝑟𝑜𝑠𝑠𝑟𝑎𝑡𝑒 𝑚𝑢𝑡𝑟𝑎𝑡𝑒
{1, 2, … , 99, 100} [0.2, 0.8] [0.85, 0.98] [0.03, 0.3] [0.05, 0.6] {0.5, 1.0, 1.5, 2.0, … , 4.5, 5.0} [0.05, 0.95] [0.05, 0.3]

Table 2 Table 4
Obtained parameter configurations for Adapt-Cmsa+Ls. Obtained parameter configurations for Ga.
𝑛 𝑡𝐼𝐿𝑃 𝛼𝐿𝐵 𝛼𝑈 𝐵 𝛼𝑟𝑒𝑑 𝑡𝑝𝑟𝑜𝑝 𝑛 𝑝𝑜𝑝𝑠𝑖𝑧𝑒 𝑐𝑟𝑜𝑠𝑠𝑟𝑎𝑡𝑒 𝑚𝑢𝑡𝑟𝑎𝑡𝑒
50 2 0.518 0.898 0.093 0.557 50 1.0 0.891 0.234
100 2 0.302 0.961 0.049 0.264 100 2.0 0.784 0.168
500 6 0.302 0.948 0.239 0.069 500 1.0 0.343 0.237

All methods were implemented in C++. Moreover, GCC 9.4 was 5.1.1. Tuning of the parameters of Adapt -Cmsa+Ls
used for compiling the software. Experiments were conducted on hard- For the purpose of tuning the parameters of Adapt-Cmsa+Ls, the
ware with an Intel Xeon E5-2640 CPU with 2.40 GHz and 8 GB of parameter domains as specified in Table 1 are used.
RAM. Note also that all experiments were conducted in single-threaded Three separate irace runs were executed, one for each value
mode (also the ones involving CPLEX). Standalone MILP models and of 𝑛 ∈ {50, 100, 500}. In this way, tuning is performed for small,
restricted MILP models within CMSA were solved by Cplex (Bliek1ú, medium, and large-sized instances. Each irace run was based on nine
Bonami, & Lodi, 2014) (version 12.1). Standalone MILP models were sample instances derived from file mdmwnpp_500_20a.txt, for all
solved exactly once, while the stochastic methods were applied 10 combinations of values 𝑚 ∈ {5, 10, 20} and 𝑘 ∈ {5, 10, 20}. A budget
times to each problem instance. of 1000 algorithm executions was given to each irace execution. The
Benchmark instances. All problem instances are generated from best configurations for Adapt-Cmsa+Ls obtained by irace are displayed
five basic data files named mdmwnpp_500_20x.txt, where 𝚡 ∈ in Table 2.
{𝚊, 𝚋, 𝚌, 𝚍, 𝚎} (Kojić, 2010). Each of these files contains 500 decimal
random vectors 𝑣𝑖 = (𝑣𝑖,1 , … , 𝑣𝑖,20 ), 𝑖 = 1, … , 500. For example, the 5.1.2. Tuning of the parameters of ga
specific problem instance with 𝑛 = 100, 𝑚 = 10, 𝑘 = 2 of type 𝙰 is In order to tune the parameters of our re-implementation of Ga, the
obtained from file mdmwnpp_500_20a.txt by extracting the first parameter domains as shown in Table 3 were used.
100 vectors 𝑣1 , … , 𝑣100 and the first 𝑚 = 10 coordinates of these vectors. Again, three separate irace runs were executed, one per each
From each of these five basic data files, 48 specific instances were value of 𝑛 ∈ {50, 100, 500}, referring to tuning the algorithm on small,
generated. This was done by combining the following values of 𝑛, 𝑚 medium, and large-sized instances. Each irace run was allowed a
and 𝑘 (3 × 4 × 4 = 48): budget of 1000 algorithm executions. The same set of test instances
was used for tuning as in the case of Adapt-Cmsa+Ls. The obtained
• 𝑛 ∈ {50, 100, 500} configurations for the Ga algorithm are presented in Table 4.
• 𝑚 ∈ {2, 5, 10, 20}
• 𝑘 ∈ {2, 5, 10, 20} 5.1.3. Tuning of the parameters of vns
That is, 48 × 5 = 240 specific problem instances were used in total for In the case of the Vns algorithm, just one parameter has to be tuned,
our experimental evaluation. that is 𝑟max . As already emphasized, due to the high complexity of
Moreover, both Ga and Vns were additionally evaluated on the the second and third neighborhood utilized by Vns, when applied to
set of instances obtained by combinations of 𝑛 ∈ {50, 100}, 𝑚 ∈ instances with a high number of vectors—that is, 𝑛 = 500—it turned
{2, 3, 4, 5, 10, 15, 20}, and 𝑘 ∈ {3, 4}. Thus, this set of 2 × 7 × 2 = 28 out to be very inefficient to use any but the first neighborhood. Thus,
in the case of instances with 𝑛 = 500, we set 𝑟max = 1. Moreover, during
additional instances (only for benchmark set A (Faria et al., 2018; Pop
preliminary experiments, we noticed that the value of 𝑘 had the highest
& Matei, 2013b) was also used in our experimental evaluation.
influence on the sensibility and efficiency of the algorithm. According
to that, all combinations of 𝑛 ∈ {50, 100}, and 𝑚 ∈ {5, 10, 20} were used
5.1. Parameter tuning
to derive six training instances from file mdmwnpp_500_20a.txt,
for each 𝑘 ∈ {5, 10, 20}. Subsequently, three separate irace runs
In this section, we give the details on the tuning process of the were executed, one for each value of 𝑘, and each using a budget of
following three algorithms: our two re-implementations, that is Vns and 1000 algorithm executions. Summarizing, the best configurations of Vns
Ga, and Adapt–Cmsa+Ls. Concerning Vns and Ga, tuning is performed obtained from irace are as follows:
for instances with 𝑘 ≥ 5, while for the remaining instances (𝑘 ∈
{2, 3, 4}) the algorithms’ configurations are taken from the literature, if • if 𝑛 = 500, then 𝑟max = 1;
not indicated differently. For the purpose of tuning, we used the well- • if 𝑘 = 5, and 𝑛 ∈ {50, 100} then 𝑟max = 1;
known irace package (López-Ibáñez, Dubois-Lacoste, Pérez Cáceres, • if 𝑘 = 10, and 𝑛 ∈ {50, 100}, then 𝑟max = 2;
Stützle, & Birattari, 2016), a tool capable of automatically config- • if 𝑘 = 20, and 𝑛 ∈ {50, 100} then 𝑟max = 1.
uring algorithms on the basis of in-advance given domains of the
parameters. This tool is implemented in programming language R and 5.2. Algorithm comparison
utilizes the iterated racing procedure, an extension of the iterated F-
race method (Birattari, Yuan, Balaprakash, & Stützle, 2010) to search In this section, we comprehensively compare the results of five
for the best algorithm configurations w.r.t. given parameter domains different approaches. First, we compare our algorithm and the re-
and tuning instances. As all five basic data files (see above) consist of implementations of Ga and Vns to the original results of the two latter
randomly generated real vectors, we decided to tune our algorithms on algorithms as reported in the literature, taking care of being as fair
the basis of type A instances. as possible with the comparisons. Thereafter, for the remaining set

8
M. Djukanović et al. Expert Systems With Applications 232 (2023) 120762

Table 5 • The best performing algorithm in terms of average solution qual-

Head-to-head comparison between the original and the re-implemented
ity is Ga-Lit (results reported in Pop and Matei (2013b)). Ga-Lit
versions of GA and VNS on the datasets from the literature.
is followed by the combined results of the specialized algorithms
avg. sol. quality # times better
for 𝑘 = 2 from Santucci et al. (2021), achieved by iMADEB (San-
Ga 28417.28 13/40
tucci et al., 2021) and the basic MADEB algorithm (Santucci
Ga-Lit 31466.58 27/40
et al., 2019). The third-best approach is Faria followed by our re-
Vns 36665.46 9/28
Vns-Lit 35087.72 19/28
implementation of Ga, Coam and Adapt–Cmsa+Ls, respectively. The
worst-performing approach in terms of delivered average solution
quality is clearly the re-implementation of Vns.
• On six (out of twelve) instances Ga-Lit was able to obtain the
of instances—for which no results of Ga and Vns are available in the best average solution quality, while the remaining five best results
literature—we evaluate our re-implementations of the two algorithms were obtained by the specialized algorithms for 𝑘 = 2 (Lit-Best).
with our Adapt-Cmsa+Ls approach and with the application of Cplex to Faria was able to achieve the best solution in just one instance,
the two available MILP models. Finally, we also provide a statistical while no other algorithms were able to achieve the best solution
evaluation of the results obtained by the five approaches (in addition for any instance.
to the know results from the literature, where applicable). • Similar conclusions can be drawn in the case of the remaining
However, before starting with the comparison across different val- benchmark sets (B, C, D and E); see Appendix A and the respective
ues of 𝑘, we briefly show an overall head-to-head comparison of the tables.
original and the re-implemented versions of GA and VNS. Note that the • Note that the results of Ga-Lit are better than those of our re-
original algorithm versions are henceforth labeled Ga-Lit and Vns-Lit. implemented Ga which indicates that either there are some am-
As can be seen from Table 5, the Ga from literature is better when it biguities in the description of Ga as given in Pop and Matei
comes to the number of best solutions (27/40 versus 13/40), while the (2013b), or that certain aspects were implemented with different
re-implemented Ga is better concerning the average solution quality data structures, for example. However, as pointed out before, we
(28417.28 versus 31466.58). Differences between the original and the did everything in order to achieve the best re-implementation
re-implemented VNS are less obvious. In fact, they are not significant of this algorithm as possible. Moreover, as we will see in the
as we will confirm later by means of statistical tests. next section, our re-implementation scales much better with an
increasing value of 𝑘 (number of partitions) in comparison to the
5.2.1. Comparison in case of instances with 𝑘 = 2 results reported in the literature.
Numerical results for the set of instances with 𝑘 = 2 concerning data
file (benchmark set) A are provided in Table 6. 5.2.2. Comparisons to the literature results for instances with 𝑘 ∈ {3, 4}
The results for the remaining benchmark sets—that is B, C, D and In this section, we compare our results in the context of instances
E—are provided in Appendix A. In addition to the results produced with 𝑘 ∈ {3, 4} from benchmark set A that were used in Faria et al.
by ourselves, the table(s) also contain the results of the original Ga (2018) and Pop and Matei (2013b). Table 7 presents numerical results
approach (Ga-Lit) as published in Pop and Matei (2013b). The first of the five approaches as well as the results of the original Vns and Ga
approaches from the literature. Table 7 has basically the same structure
three columns of the table(s) show the instance characteristics. The
as Table 6 from the previous section, with two exceptions. First, there
following columns provide the results of Faria, Coam, Vns, Ga, Adapt-
is no column Lit-best (as Lit-best referred to results from algorithms
Cmsa+Ls and Ga-Lit, respectively. The last column reports the best
specialized for 𝑘 = 2). Second, an additional column (with heading
results from Santucci et al. (2021) where three specialized algorithms
Vns-Lit) reports on the results of the original Vns from Faria et al.
for the case 𝑘 = 2 are compared in the context of average solutions
(2018).
quality: iMADEB, MADEB from Santucci et al. (2021) and Santucci et al.
Concerning the computation time limits used to execute the five
(2019), and GRASP with path re-linking from Rodriguez et al. (2017).
approaches that we implemented ourselves, they are chosen in order to
Each of the results in the last column is marked by a superscript letter
be fair with the original execution of Ga-Lit in Pop and Matei (2013b).
denoting which of the three algorithms achieved this result: iMADEB
Namely, the average computation time of Ga-Lit over all instances
in case of superscript 𝑎, MADEB in case of superscript 𝑏. Note that
reported in the aforementioned paper is around 1800 CPU seconds.
the GRASP algorithm with path relinking did not achieve any of these
Due to the aforementioned difference in the CPU processor speeds of
results. Finally, the table contains two (last) rows indicating the overall
the utilized machines (1.6 GHz vs. 2.4 GHz), we decided to use a
average results and the number of instances for which the best solution
computation time limit of 1200 CPU seconds for all five approaches. We
has been found by each of the considered algorithms.
believe that, in this way, none of the approaches enjoyed any significant
As the purpose of these experiments includes checking the efficiency
advantage over the others.
of our re-implementation of Ga w.r.t. Ga-Lit, we made sure of estab-
Concerning the parameter configurations of our re-implementation
lishing fair conditions for this comparison. The experiments reported
of Vns, we make use of all three neighborhoods—that is, 𝑟max = 3—
in Pop and Matei (2013b) were conducted on a machine equipped with
as in Faria et al. (2018). Furthermore, for Ga we used the same
a processor of 1.6 GHz and the average runtime over all experiments parameter configuration as reported for the original Ga in Pop and
was approx. 900 s per experiment. As our machine is equipped with Matei (2013b). This was done in order to achieve a fair comparison
a processor of 2.4 GHz, we decided to fix the runtime of all five of our re-implementations of Ga and Vns and their original versions.
competitors to 900 × 1.6∕2.4 = 600 CPU seconds. Note that in Santucci For Adapt-Cmsa+Ls approach we used the parameter configuration from
et al. (2021) the authors used the same computation time limit, tested Table 1.
under a very similar processor configuration as ours. Moreover, our Vns The following conclusions may be drawn from the obtained numer-
approach used the same parameter configurations as reported in Faria ical results:
et al. (2018). The configurations of our Ga are set according to Table 4,
while the configurations of Adapt-Cmsa+Ls are set according to Table 2. • In terms of average solution quality, Adapt-Cmsa+Ls is clearly the
The following conclusions may be drawn from the obtained ex- best algorithm. The second-best approach is our re-implementation
perimental results for instances with 𝑘 = 2 from benchmark set of Ga, being slightly better than Coam and Faria. The two worst-
A: performing approaches are Vns followed by Ga-Lit.

9
M. Djukanović et al. Expert Systems With Applications 232 (2023) 120762

Table 6
Comparison with the related literature for instances with 𝑘 = 2 (benchmark set A).
𝑛 𝑚 𝑘 Faria Coam Vns Ga Adapt-Cmsa+Ls Ga-Lit Lit-best
50 2 2 2.625 0.875 42.90 1.12 2.51 5.438 0.48𝑏
50 5 2 920.125 926.250 6763.29 1084.88 2456.40 925.226 920.32𝑎
50 10 2 16173.500 16173.500 36832.34 16365.51 18196.61 15722.290 16175.67𝑏
50 20 2 52826.375 52826.375 86204.41 54863.73 56692.47 50647.836 56919.47𝑏
100 2 2 13.750 19.250 38.48 19.19 24.00 19.513 0.62𝑎
100 5 2 2453.750 2447.000 6422.44 3267.70 3501.39 2347.346 2410.86𝑎
100 10 2 16010.500 15998.000 35671.53 17082.52 19599.97 15792.742 15990.02𝑎
100 20 2 48152.750 55236.500 86973.64 51772.25 55217.38 46782.412 52436.03𝑏
500 2 2 13.000 19.000 34.35 9.71 4.76 1.354 0.12𝑎
500 5 2 3526.000 3685.000 7251.19 634.58 2298.82 1922.135 2.44𝑎
500 10 2 21108.000 20071.000 40458.64 15653.14 18816.20 12938.304 12896.61𝑎
500 20 2 49877.000 54547.000 132146.10 51244.55 53518.80 32934.495 42461.60𝑏
average: 17589.781250 18495.812500 36569.942500 17666.573333 19194.109167 15003.257583 16684.52
#best: 1 0 0 0 0 6 5

Table 7
Comparison for instances with 𝑘 ∈ {3, 4} (those from benchmark set A used in Faria et al. (2018) and Pop and Matei (2013b)).
𝑛 𝑚 𝑘 Faria Coam Vns Ga Adapt-Cmsa+Ls Ga-Lit Vns-Lit
50 2 3 317.9375 311.1250 177.15 164.00 29.19 86.2 130.98
50 2 4 846.8125 294.0625 849.66 672.34 132.65 391.4 321.69
50 3 3 1405.3750 1387.8750 1768.05 1544.96 491.00 334.4 1045.94
50 3 4 4202.6250 5623.4375 4887.44 3776.72 1634.93 678.3 687.75
50 4 3 5492.6250 5569.4375 5393.63 4919.05 2178.63 3382.5 2044.01
50 4 4 9197.5000 9430.5625 11214.58 9833.64 5307.93 836.7 3185.66
50 5 3 10308.8125 10555.3750 11155.83 9386.20 5014.52 4125.8 14266.40
50 5 4 21140.6250 15535.0000 21044.33 16480.32 11268.64 1094.4 19979.40
50 10 3 28724.6875 40155.2500 44297.50 36934.34 29282.76 37521.6 38136.10
50 10 4 50224.6250 50849.7500 57326.62 49545.68 45498.90 42005.6 57025.20
50 15 3 63736.6875 61752.1875 73299.99 63701.58 58045.49 56015.2 68396.10
50 15 4 85889.3750 79702.4375 87108.10 77555.69 75205.67 56034.7 91035.60
50 20 3 85887.1875 81206.3750 95596.65 84929.06 81333.47 102652.0 92299.10
50 20 4 101370.6250 107893.3750 107167.42 103264.78 97517.29 123627.8 108404.00
100 2 3 193.2500 30.2500 134.92 141.39 38.20 178.1 50.70
100 2 4 601.8750 935.2500 545.64 570.57 51.84 687.2 99.28
100 3 3 1249.6250 1208.3750 1727.35 1302.33 683.28 531.3 2886.44
100 3 4 4938.8750 3709.1250 4096.42 2831.90 779.76 1213.7 5765.02
100 4 3 4529.1250 4356.8750 5220.42 4212.97 2257.03 867.5 8374.57
100 4 4 10313.8750 9903.5000 10400.96 7536.92 3023.79 1924.6 12219.90
100 5 3 10418.1250 9686.6250 11654.32 8929.78 4487.51 6224.5 11527.30
100 5 4 13800.3750 13307.7500 16714.95 14108.68 6514.77 8356.8 19056.70
100 10 3 33188.6250 39677.6250 41304.08 33495.31 28523.88 47004.8 37146.20
100 10 4 45014.0000 58699.3750 55965.44 45761.70 33450.13 75034.8 58992.60
100 15 3 59056.5000 60969.1250 69674.96 61119.63 54162.51 96827.3 61427.60
100 15 4 91919.3750 78554.0000 85095.32 75902.54 65026.44 122892.7 76239.60
100 20 3 95286.3750 86192.0000 91248.08 83530.70 75659.66 113112.5 84093.40
100 20 4 108824.2500 108505.3750 111563.15 100815.74 85404.72 174981.6 107619.00
average: 33859.991071 33785.767857 36665.462857 33024.730714 27607.306786 38522.285714 35087.722857
#best: 1 2 0 0 13 11 0

• The results of our implementation of Ga are significantly better 5.2.3. Comparison for instances with 𝑘 ≥ 5
than those of Ga-Lit as reported in Pop and Matei (2013b). It Numerical results for instances from all benchmark sets (from A
seems that our re-implemented Ga scales better with an increasing to E) are provided in Tables 8–12. In particular, the shown results
problem instance size, and especially with an increase of 𝑘 (number are averaged over 4 instances (with varying values of 𝑚) and 10
of partitions). algorithm runs per instance (apart from Coam and Faria, which were
• Concerning our re-implemented Vns and the results of Vns-Lit as applied exactly once to each problem instance). After the first two
reported in Faria et al. (2018), we tested the statistical significance table columns indicating 𝑛 and 𝑘, each table contains the following
of the results by means of the Wilcoxon ranked sum test. The result columns: the third and fourth columns provide the average solution
of the test was that there are no statistical differences between the quality (𝑜𝑏𝑗) of Faria and Coam, respectively, The next three blocks of
results of these two implementations, although a slight advantage columns report on the solution qualities achieved by Vns, Ga, and Adapt-
in the average solution quality in a favor of Vns-Lit can be noticed. Cmsa+Ls, respectively. Hereby, each block contains a column for the
• Concerning the number of instances for which best solutions were average solution quality 𝑜𝑏𝑗 (over 10 runs per instance) and a column
found, the best approach is Adapt-Cmsa+Ls, being successful in 13 for the qualities of the best-found solutions (𝑜𝑏𝑗 𝑏𝑒𝑠𝑡 ) averaged over four
(out of 28) cases. The second-best is Ga-Lit, which is successful instances. Best average results are shown in bold font. Averages over
in 11 cases. Note that Ga-Lit performs strongly in the case of the columns are shown in the second row from the bottom, while the
small-sized instances (with low values of 𝑛 and/or 𝑘), while Adapt- last row of each table provides the number of instances for which the
Cmsa+Ls performs significantly better when it comes to larger best solution has been found by the respective algorithm. The CPU time
instances (especially for 𝑛 = 100). This indicates that Adapt- limit for all algorithms and runs was set to 1800 s.
Cmsa+Ls scales better than the other approaches. In fact, this will The following conclusions may be drawn from the obtained exper-
be confirmed in the next section. imental results:

10
M. Djukanović et al. Expert Systems With Applications 232 (2023) 120762

Table 8
Numerical results for instances of set A.
𝑛 𝑘 Faria Coam Vns Ga Adapt-Cmsa+Ls
𝑜𝑏𝑗 𝑜𝑏𝑗 𝑜𝑏𝑗 𝑏𝑒𝑠𝑡 𝑜𝑏𝑗 𝑜𝑏𝑗 𝑏𝑒𝑠𝑡 𝑜𝑏𝑗 𝑜𝑏𝑗 𝑏𝑒𝑠𝑡 𝑜𝑏𝑗
50 5 52480.273438 50448.406250 43602.53825 48260.579500 46678.37900 48739.040450 37975.22225 42676.246175
50 10 74552.242188 74043.105469 59160.21125 63609.510825 60741.85950 63050.489150 54153.73200 56512.690300
50 20 97281.359375 88614.673828 83437.30425 91552.631150 76159.97625 77935.876125 73211.43750 74249.380700
100 5 52543.984375 52656.359375 43008.36400 45978.663050 45063.45075 46971.570925 33547.96075 35386.264600
100 10 75361.281250 72256.781250 61684.26375 67744.604400 58763.28150 66972.767650 50489.01175 52208.099000
100 20 114458.984375 94477.742188 77709.14550 84318.964625 73825.90700 81477.332575 61855.09200 63432.350175
500 5 49724.875000 52751.875000 40178.27150 47653.834675 38348.78450 42735.434425 32456.36175 33897.967300
500 10 82210.875000 67769.125000 56624.35675 63557.650575 50156.61650 54411.170225 44612.26800 45088.109400
500 20 126289.531250 95385.937500 102938.37425 120360.418075 58148.74075 66397.871175 55120.69800 56035.950725
average: 80544.822917 72044.889540 70337.428542 60965.728078 51054.117597
#best: 0 0 0 0 36

Table 9
Numerical results for instances of set B.
𝑛 𝑘 Faria Coam Vns Ga Adapt-Cmsa+Ls
𝑜𝑏𝑗 𝑜𝑏𝑗 𝑜𝑏𝑗 𝑏𝑒𝑠𝑡 𝑜𝑏𝑗 𝑜𝑏𝑗 𝑏𝑒𝑠𝑡 𝑜𝑏𝑗 𝑜𝑏𝑗 𝑏𝑒𝑠𝑡 𝑜𝑏𝑗
50 5 51177.765625 55740.101562 46453.833500 50329.964275 46423.910500 50084.186075 37975.222250 42676.246175
50 10 75443.601562 80852.105469 60120.504500 64585.905725 62380.194750 65193.646875 54153.732000 56512.690300
50 20 103236.794922 90294.685547 89501.946750 95170.036850 81286.982250 82822.785500 73211.437500 74249.380700
100 5 55555.859375 51131.734375 42619.043250 46259.218400 45430.129750 48064.432175 33547.960750 35386.264600
100 10 82270.023438 72310.289062 64275.776250 68331.355450 59935.975000 69128.336075 50489.011750 52208.099000
100 20 107080.718750 99085.972656 78474.268500 85616.907700 75304.677250 85937.622550 61855.092000 63432.350175
500 5 69325.000000 65162.000000 53732.823000 62440.645367 54341.332667 59785.343000 43234.032333 45119.943600
500 10 107032.083333 97416.750000 77988.785667 86855.548900 65238.388000 72466.533967 59298.372000 59887.724767
500 20 162537.875000 125619.875000 145754.264000 162031.269800 83559.443667 91409.661267 73054.540333 74262.867400
average: 88355.876657 80674.406960 78035.028661 68966.928839 55626.416094
#best: 0 0 0 0 36

Table 10
Numerical results for instances of set C.
𝑛 𝑘 Faria Coam Vns Ga Adapt-Cmsa+Ls
𝑜𝑏𝑗 𝑜𝑏𝑗 𝑜𝑏𝑗 𝑏𝑒𝑠𝑡 𝑜𝑏𝑗 𝑜𝑏𝑗 𝑏𝑒𝑠𝑡 𝑜𝑏𝑗 𝑜𝑏𝑗 𝑏𝑒𝑠𝑡 𝑜𝑏𝑗
50 5 57580.531250 55551.820312 46567.84750 51222.114300 48570.73700 51302.244975 41530.09525 43989.046275
50 10 80534.976562 79257.027344 62736.28875 67872.648600 62959.32325 65386.458425 59818.55725 61084.495775
50 20 106602.119141 90731.488281 90106.75925 96006.631275 79301.19900 81694.241525 75964.39500 78537.226600
100 5 55010.265625 51036.765625 40195.57850 45236.703450 43040.55875 46665.278950 36113.23650 38117.935750
100 10 78355.718750 77978.054688 62783.43050 69294.168450 63311.43850 69131.969725 52152.85575 53583.505175
100 20 107232.851562 98042.050781 80558.14600 86990.708650 72118.42800 83022.812475 62199.65175 63818.361900
500 5 51580.625000 51523.500000 41886.59000 47489.250900 38325.51925 44525.426000 34017.74200 34423.386750
500 10 78917.812500 76269.437500 57309.51375 63868.418175 52048.27600 55150.334600 44770.79700 45887.724200
500 20 133680.500000 100940.812500 104692.93050 118724.632250 58772.68400 71862.630650 57250.37100 57783.954375
average: 83277.266710 75703.439670 71856.141783 63193.488592 53025.070756
#best: 0 0 0 0 36

Table 11
Numerical results for instances of set D.
𝑛 𝑘 Faria Coam Vns Ga Adapt-Cmsa+Ls
𝑜𝑏𝑗 𝑜𝑏𝑗 𝑜𝑏𝑗 𝑏𝑒𝑠𝑡 𝑜𝑏𝑗 𝑜𝑏𝑗 𝑏𝑒𝑠𝑡 𝑜𝑏𝑗 𝑜𝑏𝑗 𝑏𝑒𝑠𝑡 𝑜𝑏𝑗
50 5 52722.523438 52511.054688 46438.21775 50119.441175 47721.01075 50317.955150 41723.54200 44115.056800
50 10 76588.000000 77985.046875 62007.21025 65902.012950 62168.48650 65633.364025 58831.10100 60954.307375
50 20 103887.498047 95599.542969 87363.97050 94905.408075 79230.26400 82028.862975 78114.77425 79720.241875
100 5 55021.484375 49089.796875 39157.51175 44677.900775 43746.70050 46612.200350 32487.27650 35013.209475
100 10 77907.398438 77904.054688 61937.34725 67131.737075 59536.78175 68816.347800 51174.78750 52657.089625
100 20 109512.062500 91041.835938 79754.95125 85740.222325 71585.53400 80702.630925 61058.89400 63027.354450
500 5 53325.875000 51141.375000 40982.49525 46176.690075 42167.07850 45964.860425 32317.96375 33496.323850
500 10 79939.375000 76752.062500 58264.81550 64047.749850 47837.12725 54195.767000 45894.50450 46399.119550
500 20 115068.125000 93760.843750 106092.04800 122768.206700 57828.50925 65759.571900 57551.79275 57762.288550
average: 80441.371311 73976.179253 71274.374333 62225.728950 52571.665728
#best: 0 0 0 0 36

11
M. Djukanović et al. Expert Systems With Applications 232 (2023) 120762

Table 12
Numerical results for instances of set E.
𝑛 𝑘 Faria Coam Vns Ga Adapt-Cmsa+Ls
𝑜𝑏𝑗 𝑜𝑏𝑗 𝑜𝑏𝑗 𝑏𝑒𝑠𝑡 𝑜𝑏𝑗 𝑜𝑏𝑗 𝑏𝑒𝑠𝑡 𝑜𝑏𝑗 𝑜𝑏𝑗 𝑏𝑒𝑠𝑡 𝑜𝑏𝑗
50 5 59180.671875 51808.585938 46535.47325 50524.878775 49364.62000 51800.529875 42139.70900 46207.731500
50 10 83591.144531 78656.062500 62542.98575 67378.651850 64306.78300 66917.611775 57730.75900 60711.596550
50 20 110378.949219 93354.359375 92320.40525 98133.122400 83569.62500 85775.873925 80057.10000 81883.902700
100 5 52455.734375 55958.453125 43136.01050 46203.926500 44086.84425 47736.317250 34147.98950 37063.122850
100 10 81878.367188 75980.765625 64232.72750 68700.418150 63148.40000 72326.877400 52257.74775 53902.971075
100 20 105608.503906 98227.203125 80086.90800 86557.766575 72441.85450 84939.003550 63179.15250 65171.232300
500 5 52203.625000 48451.625000 39112.68925 46918.158825 42141.17750 45971.129350 33670.37850 34001.313400
500 10 79189.437500 71592.562500 57052.13325 64458.622950 50035.10225 55385.909075 45888.24775 46413.464900
500 20 140641.812500 102363.562500 106337.21775 120260.400575 61653.69850 70682.052550 55549.67975 58505.280025
average: 85014.249566 75154.797743 72126.216289 64615.033861 53762.290589
#best: 0 0 0 0 36

• The best average results are produced—in all cases; thus in 36 competitor approaches), pairwise comparisons are further performed
(out of 36) cases—by Adapt-Cmsa+Ls. In our opinion, this is due, by using the Nemenyi post-hoc test (Pohlert, 2014). The outcome is
especially in the case of these rather large data sets, to the represented by means of critical difference (CD) plots. In a CD plot,
interplay between heuristic and exact algorithm elements that the compared approaches are placed on the horizontal axis according
guides the search towards more promising subinstances, for which to their average ranking. Thereafter, the CD score is computed for a
high-quality solutions can be obtained by the solver within rather significance level of 0.05. If the difference is small enough—that is, no
low computation times. statistical difference is detected—a horizontal bar linking statistically
• The Adapt-Cmsa+Ls approach delivers significantly better results equal approaches is drawn.
than the other four approaches. This is also indicated by the plots As already mentioned, the results are divided into five groups: the
shown in Fig. 1. They show the relative differences between the first group includes the results for all instances from all benchmark sets
results of Adapt-Cmsa+Ls and the combined best results of the with 𝑘 = 2; the second group includes the results for instances with
other (four) approaches, on benchmark set A. There are three 𝑘 ∈ {3, 4} (only concerning benchmark set A); the third group includes
plots, showing the results for instances grouped by 𝑘 ∈ {5, 10, 20}. the results for the instances with 𝑘 = 5 from all (five) benchmark sets,
Furthermore, each plot contains three box plots, one for each etc. The respective CD plots are shown in Figs. 2 and 3. The following
considered value of 𝑛 (𝑥 axis). They show the relative differences conclusions may be drawn from there:
for the results on respective instance sub-groups. The relative
average differences range from ≈ 5% in the case of the small-sized • For the instances with 𝑘 = 2, iMADEB from Santucci et al.
instances, to ≈ 30% in the case of the larger instances in favor of (2021) achieves the best average ranking, as shown in Fig. 2(a).
Adapt-Cmsa+Ls. The same plots are also provided in the context In fact, the ranking of iMADEB is slightly better than the one of
of the other four benchmark sets; see Appendix B. MADEB (Santucci et al., 2019). The third-best approach concern-
• While Vns and Ga perform equally good in the case of instances ing the average ranking is Ga-Lit. The differences between the
with small 𝑛 or 𝑘, Ga performs much better than Vns for the results of Ga-Lit and the ones obtained by iMADEB and MADEB
remaining cases. are not statistically significant. All other approaches perform
• Concerning the two exact approaches, Coam outperforms Faria statistically worse that the three aforementioned approaches. The
significantly for instances with more partitions (𝑘 ≥ 10). Note results obtained by Ga, Faria, Coam and Adapt-Cmsa+Ls are statis-
that the same conclusion is reported in Nikolic et al. (2022). tically equivalent. Clearly, the worst-performing algorithm on the
considered instances is Vns. We want to emphasize at this point
However, they generally cannot compete with the three heuristic
that an effective extension of the approaches specialized for 𝑘 = 2
approaches. This is with the exception of Vns in the case of the
(that is, iMADEB, MADEB, and GRASP) to the general MDMWNPP
largest instances, that is, for the instances with 𝑛 = 500 and 𝑘 = 20,
(𝑘 ≥ 3) is hard to achieve as these algorithms are based on
where Vns performs poorly.
a specific binary search space as well as on specific internal
procedures involved within the main core of the algorithms. In
5.3. Statistical evaluation contrast, despite of weaker results for the instances with 𝑘 = 2,
the application of Adapt-Cmsa+Ls is not limited to a specific value
In order to check the statistical significance of the obtained dif- of 𝑘.
ferences between the studied approaches, we employed the statis- • For the instances with 𝑘 ∈ {3, 4} on the limited subset of instances
tical methodology which is outlined below. This concerns the five from benchmark set A, the best performing algorithm in terms
approaches implemented by us, in addition to Vns from Faria et al. of average ranking as well as solution quality is Adapt-Cmsa+Ls.
(2018) and Ga from Pop and Matei (2013b) (where applicable) and, The second best w.r.t. this criterion is Ga-Lit, which performs
in the context of instances with 𝑘 = 2, also the specialized approaches statistically equivalent to our re-implementation of Vns. Note that
iMADEB, MADEB and GRASP as reported in Santucci et al. (2021). the differences between the best-performing Adapt-Cmsa+Ls and
Initially, Friedman’s test was separately executed for all competitor the other approaches are statistically significant. Note also that
approaches. The results for the considered benchmark sets are divided Vns-Lit and our re-implementation of Vns perform statistically
into four groups with respect to the value of 𝑘 (𝑘 ∈ {2, 5, 10, 20}) and equivalent which gives credibility to our re-implementations; see
separately evaluated. Note that the statistical evaluation is additionally Fig. 2(b).
performed in the context of instances with 𝑘 ∈ {3, 4}, where the • For the instances with 𝑘 = 5, Adapt-Cmsa+Ls obtains the perfect
complete results are only known for a subset of benchmark set A; see average ranking score, thus equal to one. The second best is Ga.
Table 7. The differences between these two are statistically significant in
In those cases in which the null hypothesis 𝐻0 was rejected (𝐻0 favor of Adapt-Cmsa+Ls. Ga and Vns deliver statistically equivalent
states that there are no statistical differences between the results of the results. The two approaches based on MILP-solving (Faria and

12
M. Djukanović et al. Expert Systems With Applications 232 (2023) 120762

Fig. 1. Plots showing the relative differences (in percent) between the results of Adapt-Cmsa+Ls and the results of the other (four) approaches for instances of benchmark set A.

Fig. 2. Statistical evaluation of the results for 𝑘 ∈ {2, 3, 4}.

Coam) are the worst-performing ones on this set of instances; see As a conclusion, the experimental comparison can be summarized
Fig. 3(a). as follows:
• Concerning the instances with 𝑘 = 10, Adapt-Cmsa+Ls again
obtains the perfect average ranking score, followed by Ga, see • For the instances with 𝑘 = 2, the best performing approach is
Fig. 3(b). Similarly as above, the difference between the results iMADEB (Santucci et al., 2021), followed by MADEB (Santucci
obtained by these two approaches are statistically significant et al., 2019) and the GA from the literature (Pop & Matei, 2013b).
in favor of Adapt-Cmsa+Ls. In this case, however, no statistical Even though the differences between the three approaches are
difference can be detected between the results of Ga and Vns. not statistically significant, they are significantly better than the
• Finally, for the instances with 𝑘 = 20, the same conclusions as remaining approaches.
above can be drawn concerning Adapt-Cmsa+Ls and Vns. Note • Concerning the instances with 𝑘 ∈ {3, 4}, the best performing
that, in this case, Ga outperforms Vns with statistical significance;
approach is Adapt-Cmsa+Ls, which outperforms—with statistical
see Fig. 3(c).
significance—all other approaches from the related literature.
Additionally, we have generated CD plots for a significance level of • For the instances with 𝑘 ≥ 5, Adapt-Cmsa+Ls delivers the best
0.01. However, as the conclusions reported in this paper remain valid solutions for all considered problem instances. Consequently, our
even for this lower significance level, we decided against presenting approach significantly outperforms the best competitor from the
these graphics. literature.

13
M. Djukanović et al. Expert Systems With Applications 232 (2023) 120762

Fig. 3. Statistical evaluation concerning the results for 𝑘 ∈ {5, 10, 20}.

6. Conclusions and future work Additionally, a deterministic annealing neural network approach (Wu,
Karimi, & Dang, 2019) could be applied to solve the MDMWNPP by
In this paper, we solved the multidimensional multi-way num- using two MILP models as the basis for devising an iterative numerical
ber partitioning problem (MDMWNPP), a generalization of the well- method that (theoretically) converges to an optimal solution of the
problem.
known number partitioning problem. We proposed a MILP-based meta-
Comparing the approaches in the context of real-world benchmark
heuristic algorithm, labeled Adapt-Cmsa+Ls, to solve this problem. The
sets seems also interesting as those instances could have some hidden
general framework of this algorithm is a more robust version of the ef-
patterns or there might exist internal dependencies between input
fective CMSA meta-heuristic from the literature, additionally equipped
vectors affecting the main conclusions derived in this paper.
with an effective local search procedure. This algorithm is compared
to all known competitors from the literature. For the purpose of a
CRediT authorship contribution statement
rigorous and complete comparison, and due to lack of publicly avail-
able source codes, we have carefully re-implemented the Vns and Ga
Marko Djukanović: Conceptualization, Writing – original draft,
approaches from the literature, which—in the original papers—were
Methodology, Software, Visualization. Aleksandar Kartelj: Writing,
only evaluated on a subset of the known benchmark sets. We have
Software. Christian Blum: Methodology, Writing – review & editing.
provided respective source codes which are made publicly available.
Additionally, the results of Vns and Ga from the original papers are
Declaration of competing interest
also extracted and included in the comparisons under fair conditions
whenever applicable. All five known benchmark sets from the literature
The authors declare no conflict of interest.
are used in our experimental evaluation, and the obtained results are
additionally statistically evaluated. While Ga, and the two versions of
Data availability
MADEB algorithms from literature significantly outperformed Adapt-
Cmsa+Ls in case of instances with a small number of partitions (𝑘 = 2), The link to GitHub repository with the material provided in the text.
the Adapt-Cmsa+Ls algorithm significantly outperformed all approaches
from the literature for 𝑘 ≥ 3. Thus, this algorithm may be consid- Acknowledgments
ered the new state-of-the-art algorithm for MDMWNPP in case of the
instances with 𝑘 ≥ 3. C. Blum was supported by grant PID2019-104156GB-I00 funded
Concerning future work, we plan to further hybridize the Adapt- by MCIN/ AEI/10.13039/501100011033. A. Kartelj was supported
Cmsa method by replacing the current LS procedure with some other by grant 451-03-47/2023-01/200104 funded by Ministry of Science
effective, time-efficient LS-based heuristics such as iterated LS, possibly Technological Development and Innovations of the Republic of Serbia.
restricted VNS, or even a tabu search which has already been success-
fully hybridized with CMSA for solving the maximum happy vertices Appendix A. Complete numerical results for the set of instances
problem (Thiruvady & Lewis, 2022). In that way, solution components with 𝒌 = 𝟐
found in the generated and subsequently improved solutions will result
in more promising sub-instances possibly containing new best solutions. See Tables A.13–A.16.

14
M. Djukanović et al. Expert Systems With Applications 232 (2023) 120762

Table A.13
Comparison for instances with 𝑘 = 2 (benchmark set B).
𝑛 𝑚 𝑘 Faria Coam Vns Ga Adapt-Cmsa+Ls Ga-Lit Lit-best
50 2 2 12.465 8.918 20.00 5.50 17.28 9.193 1.42𝑎
50 5 2 3212.029 3205.927 4150.72 2351.59 3781.12 3002.869 2609.12𝑎
50 10 2 19560.316 21039.450 31401.56 19178.38 24296.65 19560.318 19553.17𝑏
50 20 2 56089.632 51917.902 78909.39 53756.40 61815.75 50382.384 55173.79𝑏
100 2 2 3.913 14.691 11.97 7.78 21.88 19.513 0.77𝑎
100 5 2 2831.944 2086.786 4314.65 2893.53 3517.13 2539.387 2064.91𝑎
100 10 2 19374.434 19005.649 31067.38 17919.61 20993.65 18367.389 13589.79𝑎
100 20 2 49820.900 46685.387 72276.60 55033.86 59559.65 45673.837 53703.14𝑏
500 2 2 4.761 6.281 15.55 256.42 7.91 7.438 0.28𝑎
500 5 2 2730.849 3559.877 4719.40 10904.75 2042.73 1453.290 1755.66𝑎
500 10 2 20865.981 27029.589 28330.74 34012.78 13577.74 10918.141 12107.12𝑎
500 20 2 62739.316 61072.201 69171.82 73564.64 47943.11 38275.503 41464.94𝑎
average: 19770.545000 19636.054833 27032.481667 22490.436667 19797.883333 15850.771833 16835.2908
#best 0 0 0 2 0 5 5

Table A.14
Comparison for instances with 𝑘 = 2 (benchmark set C).
𝑛 𝑚 𝑘 Faria Coam Vns Ga Adapt-Cmsa+Ls Ga-Lit Lit-best
50 2 2 12.245 12.245 20.00 5.31 14.79 9.756 1.31𝑎
50 5 2 2705.863 2708.911 4150.72 1702.26 1807.43 2655.387 1394.91𝑎
50 10 2 17750.997 19619.579 31401.56 18032.39 22095.96 16373.839 17117.30𝑎
50 20 2 50560.864 50560.864 78909.39 57424.84 54939.17 51983.338 54141.94𝑏
100 2 2 6.443 5.231 11.97 8.82 11.11 6.302 0.62𝑎
100 5 2 2854.588 2851.534 4314.65 2166.07 2901.87 2837.741 2751.22𝑎
100 10 2 15392.672 15404.882 31067.38 15733.25 17351.70 15262.796 11985.23𝑎
100 20 2 47647.322 58128.360 72276.60 54956.14 55220.35 45211.776 51745.85𝑏
500 2 2 6.469 8.618 15.55 273.52 12.46 2.539 0.16𝑎
500 5 2 2496.388 1903.708 4719.40 10964.42 3010.90 4176.629 1720.18𝑎
500 10 2 23184.654 19180.032 28330.74 36030.08 17999.19 10324.870 12801.53𝑎
500 20 2 58277.355 63364.387 69171.82 74608.11 54373.45 39474.872 42599.51𝑎
average: 18407.988333 19479.029250 27032.481667 22658.767500 19144.865000 15693.320417 16354.98
#best 1 1 0 1 0 4 6

Table A.15
Comparison for instances with 𝑘 = 2 (benchmark set D).
𝑛 𝑚 𝑘 Faria Coam Vns Ga Adapt-Cmsa+Ls Ga-Lit Lit-best
50 2 2 1.703 10.722 13.46 6.39 19.56 5.0230 1.44𝑎
50 5 2 2281.898 2275.792 5399.42 2972.43 3297.21 2217.2880 2269.08𝑎
50 10 2 14928.077 20636.093 31237.07 19208.93 21261.10 15025.6370 14924.10𝑏
50 20 2 53955.965 53955.965 74014.20 59400.54 55862.45 52632.3070 56093.25𝑏
100 2 2 5.377 6.433 20.43 5.40 5.61 4.5230 0.00𝑎
100 5 2 2982.039 2978.993 5053.36 2767.32 2995.73 2717.6890 2941.45𝑎
100 10 2 18291.506 15568.264 30709.94 18513.72 19422.94 17066.3890 14962.60𝑏
100 20 2 54986.785 56641.234 74091.93 53309.92 59491.64 46182.3890 50172.85𝑏
500 2 2 3.187 16.045 19.62 309.73 14.42 1.0170 0.20𝑎
500 5 2 2424.010 1982.865 4659.75 9168.30 2166.34 2083.5850 2.69𝑎
500 10 2 23630.800 21291.872 28762.01 33761.08 16719.42 17283.4800 12800.82𝑎
500 20 2 63447.344 67054.022 71994.96 74194.41 49428.96 43763.0979 43664.83𝑎
average: 236938.6910 242418.3000 325976.1500 273618.1700 230685.3800 16581.8669 16486.1008
#best 0 0 0 0 0 4 8

Table A.16
Comparison for instances with 𝑘 = 2 (benchmark set E).
𝑛 𝑚 𝑘 Faria Coam Vns Ga Adapt-Cmsa+Ls Ga-Lit Lit-best
50 2 2 12.696 12.827 19.92 9.34 28.91 7.892 3.23𝑎
50 5 2 4682.901 4845.067 4823.40 2584.34 5301.83 4881.732 3959.34𝑎
50 10 2 20796.532 19303.636 29506.84 16638.17 20987.76 15263.833 15364.59𝑏
50 20 2 48281.499 48281.499 78638.57 59790.25 62510.22 48292.728 51239.31𝑏
100 2 2 4.729 1.067 13.99 5.66 6.57 4.241 1.04𝑎
100 5 2 3781.746 3778.692 5437.23 2758.43 4245.47 4112.556 2528.46𝑎
100 10 2 16516.277 16522.383 27530.33 18907.91 16819.20 15938.556 12180.65𝑎
100 20 2 49561.242 63320.091 73060.29 54804.84 53002.13 51822.651 51616.68𝑏
500 2 2 7.778 6.080 21.67 396.14 9.70 1.006 0.22𝑎
500 5 2 3248.500 4509.678 5425.47 9750.26 2820.35 2023.652 866.96𝑎
500 10 2 20221.348 23929.710 28746.59 33721.92 15563.92 15928.492 14854.88𝑎
500 20 2 63415.290 57691.895 71039.28 77832.69 57973.37 40654.702 41547.43𝑏
average: 19210.878167 20183.552083 27021.965000 23099.995833 19939.119167 16577.670083 16180.2325
#best 2 1 0 1 0 3 8

15
M. Djukanović et al. Expert Systems With Applications 232 (2023) 120762

Fig. B.4. Plots showing the relative difference (in percent) between the results of Adapt-Cmsa+Ls and the best results of the other approaches on benchmark set B.

Fig. B.5. Plots showing the relative difference (in percent) between the results of Adapt-Cmsa+Ls and the best results of the other approaches on benchmark set C.

16
M. Djukanović et al. Expert Systems With Applications 232 (2023) 120762

Fig. B.6. Plots showing the relative difference (in percent) between the results of Adapt-Cmsa+Ls and the best results of the other approaches on benchmark set D.

Fig. B.7. Plots showing the relative difference (in percent) between the results of Adapt-Cmsa+Ls and the best results of the other approaches on benchmark set E.

Appendix B. Relative differences between the results of adapt - Alidaee, B., Glover, F., Kochenberger, G. A., & Rego, C. (2005). A new modeling
cmsa+ls and the ones of the other approaches in the context of and solution approach for the number partitioning problem. Journal of Applied
Mathematics and Decision Sciences, 2005(2), 113–121.
instances with 𝒌 ∈ {𝟓, 𝟏𝟎, 𝟐𝟎}
Argüello, M. F., Feo, T. A., & Goldschmidt, O. (1996). Randomized methods for the
number partitioning problem. Computers & Operations Research, 23(2), 103–111.
See Figs. B.4–B.7. Back, T., Fogel, D. B., & Michalewicz, Z. (Eds.), (1997). Handbook of evolutionary
computation. IOP Publishing Ltd.
References Ben-Smida, H. E., Chicano, F., Krichen, S., et al. (2019). Construct, merge, solve and
adapt for taxi sharing.
Akbay, M. A., López Serrano, A., & Blum, C. (2022). A self-adaptive variant of Birattari, M., Yuan, Z., Balaprakash, P., & Stützle, T. (2010). F-race and iterated F-
CMSA: Application to the minimum positive influence dominating set problem. race: An overview. Experimental Methods for the Analysis of Optimization Algorithms,
International Journal of Computational Intelligence Systems, 15(44), 10. 311–336.

17
M. Djukanović et al. Expert Systems With Applications 232 (2023) 120762

Bliek1ú, C., Bonami, P., & Lodi, A. (2014). Solving mixed-integer quadratic program- Korf, R. E. (1998). A complete anytime algorithm for number partitioning. Artificial
ming problems with IBM-CPLEX: a progress report. In Proceedings of the twenty-sixth Intelligence, 106(2), 181–203.
RAMP symposium (pp. 16–17). Korf, R. E. (2009). Multi-way number partitioning. In IJCAI, vol. 9 (pp. 538–543).
Blum, C. (2016). Construct, merge, solve and adapt: application to unbalanced min- Korf, R. E., Schreiber, E. L., & Moffitt, M. D. (2014). Optimal sequential multi-way
imum common string partition. In International workshop on hybrid metaheuristics number partitioning. In ISAIM.
(pp. 17–31). Springer. Kratica, J., Kojić, J., & Savić, A. (2014). Two metaheuristic approaches for solving
Blum, C., & Blesa, M. J. (2016). Construct, merge, solve and adapt: application to the multidimensional two-way number partitioning problem. Computers & Operations
repetition-free longest common subsequence problem. In Evolutionary computation Research, 46, 59–68.
in combinatorial optimization (pp. 46–57). Springer. Lewis, R., Thiruvady, D., & Morgan, K. (2019). Finding happiness: an analysis of the
Blum, C., Pinacho, P., López-Ibáñez, M., & Lozano, J. A. (2016). Construct, merge, maximum happy vertices problem. Computers & Operations Research, 103, 265–276.
solve & adapt a new general algorithm for combinatorial optimization. Computers López-Ibáñez, M., Dubois-Lacoste, J., Pérez Cáceres, L., Stützle, T., & Birattari, M.
& Operations Research, 68, 75–88. (2016). The rpackageirace package: Iterated racing for automatic algorithm
Coffman, E. G., & Lueker, G. S. (1991). Probabilistic analysis of packing and partitioning configuration. Operations Research Perspectives, 3, 43–58.
Algorithms. Wiley-Interscience. Ma, F., & Ni, M. (2013). A two-phase method for multidimensional number partitioning
Corus, D., Oliveto, P. S., & Yazdani, D. (2019). Artificial immune systems can find problem. Numerical Algebra, Control and Optimization, 3(2), 203–206.
arbitrarily good approximations for the NP-hard number partitioning problem. Merkle, B., & Hellman, M. (1978). Hiding information and signatures in trapdoor
Artificial Intelligence, 274, 180–196. knapsacks. IEEE Transactions on Information Theory, 24(5), 525–530.
Davidson, P. P., Blum, C., & Lozano, J. A. (2018). The weighted independent domi- Mertens, S. (2006). Number partitioning. Computational Complexity and Statistical
nation problem: Integer linear programming models and metaheuristic approaches. Physics, 125.
European Journal of Operational Research, 265(3), 860–871. Moffitt, M. D. (2013). Search strategies for optimal multi-way number partitioning. In
Ducha, F. A., & de Souza, S. R. (2013). Algorithms analysis for the number partition Twenty-third international joint conference on artificial intelligence.
problem. In XXXIV CILAMCE. Nikolic, B., Djukanovic, M., & Matic, D. (2022). New mixed-integer linear programming
Ercegovac, M., Kirovski, D., & Potkonjak, M. (1999). Low-power behavioral synthesis model for solving the multidimensional multi-way number partitioning problem.
optimization using multiple precision arithmetic. In Proceedings of the 36th annual Computational & Applied Mathematics, 41(3), 1–72.
ACM/IEEE design automation conference (pp. 568–573). Association for Computing Pedroso, J. P., & Kubo, M. (2010). Heuristics and exact methods for number
Machinery. partitioning. European Journal of Operational Research, 202(1), 73–81.
Faria, A. F., de Souza, S. R., & de Sá, E. M. (2021). A mixed-integer linear programming Pohlert, T. (2014). The pairwise multiple comparison of mean ranks package (PMCMR).
model to solve the multidimensional multi-way number partitioning problem. R Package, 27(2020), 10.
Computers & Operations Research, 127, Article 105133. Pop, P. C., & Matei, O. (2013a). A genetic algorithm approach for the multidimensional
Faria, A. F., de Souza, S. R., Souza, M. J. F., Silva, C. A., & Nazário Coelho, V. (2018). A two-way number partitioning problem. In International conference on learning and
variable neighborhood search approach for solving the multidimensional multi-way intelligent optimization (pp. 81–86). Springer.
number partitioning problem. In International conference on variable neighborhood Pop, P. C., & Matei, O. (2013b). A memetic algorithm approach for solving the
search (pp. 243–258). Springer. multidimensional multi-way number partitioning problem. Applied Mathematical
Graß, T., Raventós, D., Juliá-Díaz, B., Gogolin, C., & Lewenstein, M. (2016). Quantum Modelling, 37(22), 9191–9202.
annealing for the number-partitioning problem using a tunable spin glass of ions. Rodriguez, F. J., Glover, F., García-Martínez, C., Martí, R., & Lozano, M. (2017).
Nature Communications, 7(1), 11524. GRASP with exterior path-relinking and restricted local search for the multidimen-
Hacibeyoglu, M., Alaykiran, K., Acilar, A. M., Tongur, V., & Ulker, E. (2018). A sional two-way number partitioning problem. Computers & Operations Research, 78,
comparative analysis of metaheuristic approaches for multidimensional two-way 243–254.
number partitioning problem. Arabian Journal for Science and Engineering, 43(12), Santucci, V., Baioletti, M., & Di Bari, G. (2021). An improved memetic algebraic
7499–7520. differential evolution for solving the multidimensional two-way number partitioning
Jensi, R., & Jiji, G. W. (2016). An enhanced particle swarm optimization with levy problem. Expert Systems with Applications, 178, Article 114938.
flight for global optimization. Applied Soft Computing, 43, 248–261. Santucci, V., Baioletti, M., Di Bari, G., & Milani, A. (2019). A binary algebraic differen-
Johnson, D. S., Aragon, C. R., McGeoch, L. A., & Schevon, C. (1991). Optimization tial evolution for the multidimensional two-way number partitioning problem. In
by simulated annealing: an experimental evaluation; part II, graph coloring and European conference on evolutionary computation in combinatorial optimization (Part
number partitioning. Operations Research, 39(3), 378–406. of EvoStar) (pp. 17–32). Springer.
Karmarkar, N., & Karp, R. M. (1982). The differencing method of set partitioning. Thiruvady, D., & Lewis, R. (2022). Recombinative approaches for the maximum happy
Computer Science Division (EECS), University of California Berkeley. vertices problem. Swarm and Evolutionary Computation, 75, Article 101188.
Karp, R. M. (1975). On the computational complexity of combinatorial problems. Wu, Z., Karimi, H. R., & Dang, C. (2019). An approximation algorithm for graph
Networks, 5(1), 45–68. partitioning via deterministic annealing neural network. Neural Networks, 117,
Kojić, J. (2010). Computers & Mathematics with Applications, 60(8), 2302–2308. 191–200.

18