See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/335954009

Solving Sudoku With Ant Colony Optimization

Article  in  IEEE Transactions on Games · September 2019

DOI: 10.1109/TG.2019.2942773

CITATIONS READS
5 412

2 authors:

Huw Lloyd Martyn Amos

Manchester Metropolitan University Northumbria University
54 PUBLICATIONS   576 CITATIONS    131 PUBLICATIONS   1,313 CITATIONS   

SEE PROFILE SEE PROFILE

Some of the authors of this publication are also working on these related projects:

MANTiCORE: The MMU Ant Colony Optimization Research Environment View project

Wireless Sensor Networks Applications to City Critical Infrastructure View project

All content following this page was uploaded by Martyn Amos on 11 November 2020.

The user has requested enhancement of the downloaded file.

This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TG.2019.2942773, IEEE
Transactions on Games
IEEE TRANSACTIONS ON GAMES 1

Solving Sudoku with Ant Colony Optimization

Huw Lloyd, Member, IEEE, and Martyn Amos

Abstract—In this paper we present a new algorithm for

the well-known and computationally-challenging Sudoku puzzle
game. Our Ant Colony Optimization-based method significantly
out-performs the state-of-the-art algorithm on the hardest, large
instances of Sudoku. We provide evidence that – compared to
traditional backtracking methods – our algorithm offers a much
more efficient search of the solution space, and demonstrate the
utility of a novel anti-stagnation operator. This work lays the
foundation for future work on a general-purpose puzzle solver,
and establishes Japanese pencil puzzles as a suitable platform
for benchmarking a wide range of algorithms.
Index Terms—Ant Colony Optimzation, Sudoku, Puzzle
Games. Fig. 1. The structure of a Sudoku puzzle instance (left), and its solution
(right).

I. I NTRODUCTION
(that is, 2×8=16 neighbours in the relevant row and column,
Sudoku is a well-known logic-based puzzle game that was plus 4 other cells occupying the same box; see Figure 2).
first published in 1979 under the name of “Number Place”. Sudoku is an NP-complete problem [3], as first shown in
It was popularised in Japan in 1984 by the puzzle company [4] via a reduction from the Latin Square Completion problem
Nikoli, and later named “Sudoku”, which roughly translates [5]. As such, the problem offers itself as a useful benchmark
to “single digits”. The puzzle gained attention in the West challenge, and a number of different types of algorithm have
in 2004, after The Times published its first Sudoku grid at been proposed for its solution (see the next Section for a
the instigation of Hong Kong-based judge Wayne Gould, who more detailed discussion of these). However, we also consider
first encountered the puzzle in 1997, and developed a computer the argument that “We should develop AI methods that work
program to automatically generate instances. Sudoku is now with not just one game, but with any game (within a given
a global phenomenon, and many newspapers now carry it range) that the method is applied to” [6]. That is, rather than
alongside their existing crosswords (see [1] for a general developing a multitude of algorithms to play one specific
history of the puzzle). game, we should seek methods that find broader applicability,
The simplest variant of Sudoku uses a 9×9 grid of cells across a range of games. Although the algorithm we present
divided into nine 3×3 subgrids (Figure 1 (left)). As we later here is demonstrated in the context of Sudoku, we later show
demonstrate, the problem scales to larger grids, but, for the how its lack of reliance on any heuristic information (that
moment, we focus on the most familiar variant. The aim of is, game-specific “hints”) means that it may be applied to a
the puzzle is to fill the grid with digits such that each row, number of different puzzle games.
each column, and each 3×3 subgrid contains all of the digits While such puzzle games may, superficially, appear to lack
1-9 (Figure 1 (right)). An instance of Sudoku provides, at the “real world” relevance, they in fact offer a significant challenge
outset, a partially-completed grid, but the difficulty of any grid for general-purpose AI methods; as argued in [6], “We need
derives more from the range of techniques required to solve it game playing benchmarks and competitions capable of ex-
than the number of cell values that are provided for the player. pressing any kind of game, including puzzle games, 2D arcade
Formally, a Sudoku problem of order n = 3 is made up of a games, text adventures, 3D action-adventures and so on; this
grid of cells (or squares), arranged into 3×3 subgrids known is the best way to test general AI capacities and reasoning
as boxes. A unit is a row, column or box, each containing skills.” While our algorithm could not be described as “general
exactly nine cells. A problem is solved when each unit (that purpose”, this does serve to underscore the importance of the
is, every row, column and box) contains a permutation of the puzzle game domain.
digits 1. . . 9 [2] The rest of the paper is structured as follows: in Sec-
Any given cell has exactly three units and 20 peers; the tion II we briefly review closely-related recent work on the
units are the row, column and box in which the cell resides, application of various algorithms to Sudoku. This motivates
and the set of peers is made up of the other cells in those units the description, in Section III of our own method, based on
Ant Colony Optimization (ACO), which introduces a novel
H. Lloyd is with the Department of Computing and Mathematics, operator which we call Best Value Evaporation. In Section IV
Manchester Metropolitan University, Manchester, UK (email: we present the results of experimental investigations, which
[email protected]).
M. Amos is with the Department of Computer and Information Sciences, confirm (1) that our algorithm out-performs existing methods,
Northumbria University, UK (email: [email protected]). and (2) that BVE is a necessary addition to the basic ACO

2475-1502 (c) 2019 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
Authorized licensed use limited to: Northumbria University Library. Downloaded on May 27,2020 at 10:42:37 UTC from IEEE Xplore. Restrictions apply.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TG.2019.2942773, IEEE
Transactions on Games
IEEE TRANSACTIONS ON GAMES 2

Fig. 2. Units and peers for a specific highlighted cell. The units (from right to left, column, row, and block) are highlighted in white. The union of the three
units, that is all the white cells, are the peers.

algorithm for solving large Sudoku instances. We conclude likely to be selected. After a single population iteration, the
in Section V with a discussion of our findings, and discuss best solution according to some objective function is selected
possible future work in this area. from the population, and the components it contains (e.g.,
edges in the graph) are given additional pheromone. In this
II. R ELATED WORK way, the population rapidly converges on high-quality solu-
We first consider a “traditional” backtracking approach to tions, although premature or sub-optimal convergence is dis-
solving Sudoku. The Exact Cover Problem [7] is a type couraged through the continuous “evaporation” of pheromone
of constraint satisfaction problem which may be phrased as concentrations. Some ACO variants include local pheromone
follows: given a binary matrix, find a subset of rows in which operators, which allow individual ants to record information
each column sums to 1 (that is, find a set of rows in which about their traversal during the solution construction process;
each column contains only a single 1). In [8], Knuth describes for example, ants may reduce the global pheromone value
the “dancing links” implementation of his Algorithm X (called associated with components as they are added to a solution,
DLX), a “brute force” backtracking algorithm for Exact Cover. to discourage following ants from taking the same path.
As any Sudoku puzzle may be transformed into an instance The archetypal ACO algorithm was named “Ant system”
of Exact Cover [9], DLX naturally offers an effective solution [25], and this was applied to the well-known Travelling Sales-
method for Sudoku [10]. man Problem as follows: each edge connecting two cities has a
In [2], Peter Norvig presents an alternative approach, based pheromone value, and the probability of an edge being selected
on constraint propagation followed by a search process (we by an ant is a function of both its pheromone concentration
discuss this in more detail shortly). Other notable approaches and its distance from the ant’s current location. This process
to solving Sudoku include formal logic [11], an artificial thus combines the autocatalytic power of the global pheromone
bee colony algorithm [12], constraint programming [13], [14], network with a greedy local search heuristic. Each ant also
evolutionary algorithms [15], [16], [17], [18], particle swarm maintains a “tabu” list of cities that it has visited, and an ant
optimisation [19], [20], simulated annealing [21], tabu search may not re-visit any city on its list. Once it has visited all cities,
[22], and entropy minimization [23]. As this diverse set of an ant then deposits an amount of global pheromone which is
solution methods demonstrates, Sudoku offers a challenging inversely proportional to the length of its tour; that is, shorter
yet conceptually simple test-bed for the comparative analysis tours deposit more pheromone. Once all ants have completed
of algorithms for problems involving complex reasoning. this process, the global pheromone matrix is evaporated, thus
In this paper, we focus on the application of ACO to the gradually removing the remnants of sub-optimal tours that
solution of Sudoku. ACO is a population-based search method persist over time. Dorigo et al. [25] demonstrate that positive
inspired by the foraging behaviour of ants [24], [25], and feedback, combined with local search, can offer a heuristic
it has been successfully applied to a wide range of compu- that is robust, versatile, broadly applicable, and amenable
tational problems (see [26], [27] for overviews of both the to parallelization, because of its inherent population-based
algorithm and its applications). The basic ACO algorithm uses structure. Since the publication of the original paper, ACO
a population of “ants” (agents), which individually explore a is now a well-established method [28].
given problem space and incrementally construct a solution, In [29], Mantere presents a hybrid ACO/genetic algorithm
combined with a global “pheromone” data structure, which is approach to Sudoku, which combines global (evolutionary)
used to inform decisions taken by the ants. Essentially, each search with greedy local (ACO-based) search. Schiff [30]
ant moves individually on some problem representation (for and Sabuncu [31] also present relatively recent work on
example, a graph), gradually building a solution and proba- applying ACO to Sudoku, but, in both cases, the performance
bilistically choosing its next move according to pheromone of the algorithm is relatively poor. Another nature-inspired
concentrations. Components with more pheromone are more approach was used by [12], who used a variant of the artificial

2475-1502 (c) 2019 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
Authorized licensed use limited to: Northumbria University Library. Downloaded on May 27,2020 at 10:42:37 UTC from IEEE Xplore. Restrictions apply.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TG.2019.2942773, IEEE
Transactions on Games
IEEE TRANSACTIONS ON GAMES 3

bee colony algorithm to solve 9 × 9 Sudoku puzzles. The 1) Eliminate from a cell’s value set all values that are fixed
algorithm was able to solve some difficult instances (such as in any of the cell’s peers.
the AIEscargot instance[32]) but the runtime performance is 2) If any values in a cell’s value set are in the only possible
relatively poor with an average solution time of over 6 minutes place in any of the cell’s units, then fix that value.
for difficult instances. Note that since this can lead to other cells having their values
For the purposes of comparison, in this paper we focus fixed, the procedure is recursive, and terminates when no
mainly on the work of Musliu, et al. [33], who present an further changes are possible.
iterated local search algorithm with constraint programming In Figure 3 we show the instance from Figure 1 after the
which represents the state-of-the-art in stochastic search algo- initial pass of our CP algorithm, which occurs when the board
rithms for the Sudoku problem, plus the algorithms of Knuth is set up, and before any search is performed. For easy cases,
[8] and Norvig [2]. the application of the CP algorithm is often sufficient to solve
the board, and no further search is required (see Section IV
for a discussion). However, in most cases, some search will
be required, and we now describe our ACO-based method for
this.

B. Our ACO algorithm

Our algorithm is based on Ant Colony System (ACS),
which is a variant of ACO introduced in [34]. We first give
an informal description of the algorithm, and then formally
specify its various components.
At each population-level iteration, every ant works inde-
Fig. 3. Instance from Figure 1 (left), and (right) cell value sets after initial pendently on its own copy of the board. However, the global
pass of constraint propagation algorithm. The value sets are represented as pheromone matrix persists across iterations, allowing for a
strings of allowable digits for the cell, for example ‘589’ represents the set combination of local search and global positive feedback to
of values {5, 8, 9}.
occur (i.e., when the best ant in each iteration updates the
global pheromone). The ants move round their boards in
III. O UR ALGORITHM parallel; the ant system iterates over the ants in turn, calling a
step function which moves each ant one step. This enables ants
In [2], Norvig describes a two-component approach to solv-
to discourage others from following the same path through the
ing Sudoku, using a combination of constraint propagation
local pheromone mechanism. The outer loop of the ant system
(CP) and search. CP ensures that the “rules” of Sudoku are
update therefore iterates c times, where c is the number of
observed, and repeatedly prunes the value set of each cell (that
cells, and at each iteration requests that each ant makes a
is, the set of possible values that cells might take). Importantly,
single step.
by using CP during search, we effectively “parallelise” the
As previously stated, once the initial pass of the CP has been
process, by eliminating large numbers of possible cell values
completed, then most cells will have a set of possible values.
every time we fix a cell’s value; selecting a specific value for
The aim of each ant, in a single population-level iteration,
a cell immediately rules out that value’s presence in a large
is to fix as many cell values as possible. Each ant starts on
number of other cells. In [2], Norvig combines CP with a
a different, randomly-selected cell, and then iterates over all
recursive depth-first search which, at each iteration, selects
cells on the board. We simply move from one cell to the next
the cell with the smallest value set and then chooses the first
because what is important is not the “next cell”, but the value
numeric ordered value for that cell. This essentially maximises
assigned to the next cell encountered. Whenever it leaves a
the probability of “guessing correctly”, and is referred to as
cell that does not have a fixed value (that is, a cell with a
the Minimum Remaining Values Heuristic.
number of possible values), an ant must make a decision on
Here, we present a variant of constraint propagation inspired
which element of that cell’s value set to choose, thus setting
by Norvig’s method, and use ACO (rather than depth-first
the cell to that value. Importantly, as soon as an ant sets the
search) to search the space of solutions. We now describe our
value of a cell, the constraints that it introduces are propagated
CP method in more detail. For clarity, this is written in terms
across the board.
of the 9×9 Sudoku puzzle, but the method generalises trivially
Decisions on which value to choose are based on relative
to larger sizes (e.g. 16 × 16, 25 × 25).
pheromone levels, which are assigned to each possible value.
These are stored in a pheromone matrix, which keeps track
A. Constraint propagation of a single pheromone amount for each possible value in
Throughout the constraint propagation (CP) process, each each cell. This is, for an order-3 (9 × 9) Sudoku puzzle,
cell maintains its value set – a list of possible values it might a matrix of 81 × 9 values, with each cell corresponding to
take; every cell starts with the same value set, [1 . . . 9]. Once a the pheromone level for each possible value (1 . . . 9) in a
set has been reduced to a single value, we call that value fixed cell (indexed 1 . . . 81). Depending on the “greediness” of the
for that cell. Our CP algorithm implements two basic rules, selection, either the value with the highest pheromone value
which are applied to a cell’s peers when it has its value fixed: is chosen, or a weighted (roulette) selection is made.

2475-1502 (c) 2019 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
Authorized licensed use limited to: Northumbria University Library. Downloaded on May 27,2020 at 10:42:37 UTC from IEEE Xplore. Restrictions apply.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TG.2019.2942773, IEEE
Transactions on Games
IEEE TRANSACTIONS ON GAMES 4

Algorithm 1: Our ACO algorithm for Sudoku We give a pseudo-code description of our approach in
1 read in puzzle; Algorithm 1, components of which we now formally specify.
2 for all cells with fixed values do
3 propagate constraints (according to Section III-A); Line 5: For a Sudoku puzzle of dimension d we define
4 end a two-dimensional global pheromone matrix, τ , in which
5 initialize global pheromone matrix; each element is denoted as τik , where i is the cell index
6 while puzzle is not solved do (1 ≤ i ≤ d2 ) and k is a possible value for the cell (k ∈ [1, d]).
7 give each ant a local copy of puzzle; τik represents the pheromone level associated with value k in
8 assign each ant to a different cell; cell i. Each element of the matrix is initialised to some fixed
9 for number of cells do value, τ0 (we use a value of 1/c, where c = d2 is the total
10 for each ant do number of cells on the board).
11 if current cell value not fixed then
12 choose value from current cell’s value set; Line 12: Where an ant has a choice of a number of values
13 fix cell value; in an “open” cell (i.e., one which does not yet have its value
14 propagate constraints; fixed), then we define the value set, vi of cell i as the set of all
15 update local pheromone; available values for that cell, from which we have to choose
16 end one. We have a choice of two methods to use when making
17 move to next cell; a selection; we might make a greedy selection, in which case
18 end the member of vi with the highest pheromone concentration
19 end is selected, or we might make a weighted (i.e., “roulette
20 find best ant; wheel”) selection, in which case the selection probabilities are
21 do global pheromone update; proportional to the pheromone associated with the available
22 do best value evaporation; choices. The relative probabilities of each type of selection
23 end are determined by the greediness parameter, q0 ∈ [0, 1]. A
value selection, s, is therefore made according to
(
argmaxk∈vi {τik } if q < q0
After the cell’s value is set, the standard ACS local s= (1)
R otherwise
pheromone operator is applied, which reduces the probability
of that value being selected by the following ant, thus prevent- where q ∈ [0, 1] is a uniform random deviate, and R is a se-
ing early convergence. lection from vi made according to the probability distribution
Once all ants have covered every square of the board, we
τk
then perform the global pheromone update, which rewards pki = Pi j , k ∈ vi (2)
only the best solution found so far (the global best, in line τi
j∈vi
with ACS principles). We characterise the “best” solution, at
each iteration, as the sequence of value selections that lead to where pki is the probability of selecting choice k from vi .
the greatest number of cells having their values fixed; the best If a cell has a value set of size zero (that is, it cannot have
solution is effectively the one found by the ant that “guesses” its value fixed due to other cells being fixed and the constraints
correctly the highest number of times. However, at this point, thus introduced), then we mark it as a “fail cell”; the number
we introduce a novel variation to the standard ACS algorithm, of fail cells is later subtracted from the number of cells to be
which we call best value evaporation (BVE). In what follows, fixed when we calculate the quality of a solution (see note
“best value” refers to an amount of pheromone that is added below, for Line 20).
to the global pheromone matrix whenever the best solution is
identified within a generation, and this value is itself subject Line 15: The local pheromone update operator is used to
to evaporation, along with the component pheromone values. make selected values less attractive in subsequent iterations,
In standard ACS, the global pheromone operator increases thus promoting exploration of the solution space. The local
the pheromone concentrations of all components of the global pheromone update is handled as follows; every time an ant
best solution with an amount of pheromone that is directly selects a value, s, at cell i, its pheromone value in the matrix
proportional to the absolute quality of that solution. However, is updated as follows:
this can gradually lead to stagnation, where all ants end up
selecting the same route. Instead, the amount of pheromone τis ← (1 − ξ)τis + ξτ0 (3)
that is added globally, which we call the best value, is
measured in terms of the proportionate quality of the best with ξ = 0.1 (the standard setting for ACS).
solution found so far (Equation 5). Importantly, the best value
itself is subject to evaporation over time, which prevents Line 20: In order to perform the global pheromone update,
“lock in”; taken together, these two components of BVE we must first find the best-performing ant. At each iteration,
prevent premature stagnation, which is confirmed by our later each ant n of the m ants keeps track of the number of cells,
experimental observations. fn , n ∈ {1 . . . m}, that it has managed to set to a specific

2475-1502 (c) 2019 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
Authorized licensed use limited to: Northumbria University Library. Downloaded on May 27,2020 at 10:42:37 UTC from IEEE Xplore. Restrictions apply.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TG.2019.2942773, IEEE
Transactions on Games
IEEE TRANSACTIONS ON GAMES 5

value. The value of fn corresponding to the iteration-best ant conducted by [33] of their algorithm against a number of
is fbest , given by competitors, and gives a measure of the practical applicability
of the algorithm in a time-constrained environment. In all
fbest = max fn . (4)
n∈{1...m} cases, we measured the statistical significance of results using
non-parametric tests, with a p value threshold for significance
We then calculate the amount of pheromone to add, ∆τ , as
of 0.05. In cases where multiple algorithms are compared
follows:
c together, this significance threshold was modified using the
∆τ = (5)
c − fbest Bonferroni correction. In comparing vectors of solution times,
where c is the total number of cells on the board. If the value of we use the Mann-Whitney U test in cases where the vectors
∆τ exceeds the current “best pheromone to add” value, ∆τbest have different lengths, which occurs when the success rates in
(a quantity initialized to 0 at the beginning of the run), then an experiment differ. This test is appropriate for determining
we set ∆τbest ← ∆τ , and replace the current best solution significance of differences in the means of differently-sized
with the solution found by the iteration-best ant. samples, when the distribution cannot be assumed to be nor-
Line 20: We then update all pheromone values corresponding mal. In cases where all algorithms solved all the instances, we
to values in the current best solution, where ρ ∈ [0, 1] is the use the Wilcoxon-signed rank test, which tests for significance
standard evaporation parameter: of difference in the means of paired observations, again with
no assumption on the distribution. The success rates are treated
as frequencies of a nominal variable (success/fail) for which
τis ← (1 − ρ)τis + ρ∆τbest . (6)
the Pearson χ2 test is appropriate.
Note that in ACS, there is no global evaporation of pheromone;
the global pheromone update (equation 6) is only applied
to pheromone values corresponding to fixed values in the A. Experimental environment
best solution; the evaporation parameter ρ represents the All of the codes were compiled using the same compiler and
“volatility” of the deposited pheromone, and is used to tune optimisation setting (g++ v5.4.0 with -O3). Experiments were
the convergence rate of the algorithm. run on a machine with an Intel Xeon E5-2460v4 processor
with a clock speed of 2.4GHz, running Ubuntu Linux. The
Line 22: In order to prevent “lock in”, we then additionally parameter settings for the iterated local search solver (ILS)
apply evaporation to the current best pheromone value, ∆τbest : were taken from the recommendations given in [33]. For the
ant colony code (ACS), we used the following settings: ρ =
∆τbest ← ∆τbest × (1 − ρBVE ) (7) 0.9, q0 = 0.9, ρBV E = 0.005, m = 10. Our code, and all the
instance files used for the experiments, may be downloaded
where ρBVE ∈ [0, 1] is a parameter which controls the rate of
from https://github.com/huwlloyd-mmu/sudoku acs.
evaporation of the best pheromone value.

IV. E XPERIMENTAL RESULTS B. Logic-solvable 9 × 9 instances

Our ant colony algorithm (ACS) was evaluated by com- We first selected instances based on known difficulty, or
paring it with (1) iterated local search code from Musliu et on previous use in the literature. We selected the ten instances
al. (ILS) [33], (2) a C++ implementation of the Dancing used in [31] (labelled here sabuncu1 to sabuncu10), five named
Links algorithm (DLX) [35], and (3) our own implementation instances identified by [36] as the most difficult (Platinum
of backtracking search, using the minimum remaining values Blond, Golden Nugget, Red Dwarf, coly013, tarx0134), and
heuristic, which uses the same problem representation and one instance (AI Escargot) [32], commonly regarded as an ex-
constraint propagation code as the ant colony algorithm (BS). tremely difficult puzzle. These instances are all logic solvable;
The code presented in [33] was itself compared against a in other words, they each have a unique solution which can
number of other stochastic algorithms, and was shown to be deduced from the given numbers. We ran the ACS, Iterated
be the best performing. We include the Dancing Links and Local Search (ILS), Dancing Links (DLX) and backtracking
backtracking algorithms for comparison with deterministic, ex- search (BS) algorithms 100 times on each instance, with a
haustive search. Furthermore, including a backtracking search timeout of 5 seconds. The puzzles were successfully solved
which uses the same underlying constraint propagation code in all cases by all four algorithms; there were no time-outs.
allows us to evaluate the effectiveness of the ant colony Table I shows the timing results for the four algorithms.
algorithm in searching the problem space, independent of the Since all the instances were solved in all cases, the vectors
details of the underlying implementation. of times per instance and algorithm are the same length; we
We conducted experiments using a number of logic-solvable therefore use the Wilcoxon Signed Rank test, to determine the
9 × 9 instances from the literature, as well as randomly significance of differences in the mean time. In all cases we
generated 9 × 9, 16 × 16 and 25 × 25 ‘general’ instances tested the fastest algorithm against the other three, using the
(which do not necessarily have a unique solution). In all the Bonferroni correction to lower the significance threshold on
experiments, we evaluated the algorithms for success rate over the p value of the tests by a factor equal to the number of
a number of trials or instances, subject to a timeout, and the tests. We also tested the times obtained by the two stochastic
mean time to solution. This is the same as the evaluation algorithms, ACS and ILS, against each other.

2475-1502 (c) 2019 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
Authorized licensed use limited to: Northumbria University Library. Downloaded on May 27,2020 at 10:42:37 UTC from IEEE Xplore. Restrictions apply.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TG.2019.2942773, IEEE
Transactions on Games
IEEE TRANSACTIONS ON GAMES 6

TABLE I
S OLUTION TIMES ( MEANAND STANDARD DEVIATION TIME OVER 100 RUNS ) FOR THE LOGIC - SOLVABLE INSTANCES . F IGURES IN BOLD INDICATE TIMES
WHICH ARE SIGNIFICANTLY LOWER FOR ONE ALGORITHM COMPARED THE OTHER THREE , BASED ON A W ILCOXON SIGNED RANK TEST; THE
B ONFERONNI CORRECTION IS APPLIED , SO THAT p VALUES LESS THAN 0.05/3 ARE TAKEN TO BE SIGNIFICANT. A STERISKS SHOW CASES IN WHICH THE
TIMES FOR ILS OR ACS ARE SIGNIFICANTLY LOWER THAN THE OTHER , USING THE W ILCOXON SIGNED RANK TEST WITH p < 0.05.

Solution Time/s
Instance ACS ILS DLX BS
sabuncu1 (4.8 ± 1.84) × 10−5∗ 0.00083 ± 0.00047 0.00105 ± 0.000362 (1.58 ± 0.651) × 10−6
sabuncu2 (4.82 ± 1.73) × 10−5∗ 0.00414 ± 0.00143 0.000937 ± 0.000308 (2.18 ± 6.45) × 10−6
sabuncu3 0.000993 ± 0.000457∗ 0.112 ± 0.0296 0.00104 ± 0.000366 0.000202 ± 0.0000775
sabuncu4 0.000625 ± 0.000708∗ 0.00859 ± 0.00229 0.00112 ± 0.000346 0.0001 ± 0.0000378
sabuncu5 (4.62 ± 1.5) × 10−5∗ 0.00097 ± 0.000556 0.00101 ± 0.000384 (1.68 ± 0.733) × 10−6
sabuncu6 0.0107 ± 0.00828∗ 0.105 ± 0.027 0.00153 ± 0.00045 0.000775 ± 0.000273
sabuncu7 0.00106 ± 0.000986∗ 0.0853 ± 0.0206 0.00102 ± 0.000318 (9.67 ± 3.75) × 10−5
sabuncu8 0.000728 ± 0.000343∗ 0.007 ± 0.00206 0.00107 ± 0.000374 (7.91 ± 2.74) × 10−5
sabuncu9 0.00163 ± 0.0014∗ 0.0153 ± 0.00437 0.00105 ± 0.000345 0.00016 ± 0.0000579
sabuncu10 (4.73 ± 1.85) × 10−5∗ 0.00136 ± 0.000641 0.00104 ± 0.000363 (1.6 ± 0.693) × 10−6
aiescargot 0.0204 ± 0.0152∗ 0.152 ± 0.0328 0.00208 ± 0.000648 0.000475 ± 0.000182
coly013 0.0488 ± 0.0518∗ 0.702 ± 0.0685 0.007 ± 0.00146 0.0278 ± 0.00517
goldennugget 0.0374 ± 0.0293∗ 0.442 ± 0.0918 0.00545 ± 0.00149 0.0152 ± 0.00304
platinumblond 0.113 ± 0.0859∗ 0.131 ± 0.0223 0.0059 ± 0.00152 0.00268 ± 0.000923
reddwarf 0.0404 ± 0.0354∗ 0.299 ± 0.0768 0.00514 ± 0.00132 0.00993 ± 0.00212
tarx0134 0.0259 ± 0.0193∗ 0.851 ± 0.0699 0.0185 ± 0.00303 0.038 ± 0.0074

The ten puzzles from [31] (sabuncu1–sabuncu10) are gen- each row, column and subgrid must contain all of the digits
erally solved in less time by all the algorithms than the six 1 . . . 16 and 1 . . . 25 respectively.
harder puzzles. In four cases (sabuncu1, sabuncu2, sabuncu5 These instances were generated by running the ACS code
and sabuncu10) the puzzle is solved by a single application of with an initially blank grid, to produce a set of Sudoku
our constraint propagation procedure, so that no searching is solutions. These are then converted into problem instances
required for either the ACS or BS algorithms. The difference by randomly blanking a number of the cells. The instances
in runtimes between the two algorithms for these instances generated in this way are not guaranteed to have a unique
may be explained by the difference in set-up times; in the solution. For each of the sizes 9 × 9, 16 × 16 and 25 × 25, we
case of ACS, the overhead of creating the ant colony and generated 100 instances for fixed cell fractions in steps of 0.05
initializing the pheromone matrix is clearly significant. On from 0 to 0.95, giving a total of 6000 individual instances. We
these four “trivial” instances, the BS algorithm is the fastest of ran the ACS, ILS, DLX and BS codes once on each instance,
all (running in times of order a microsecond). DLX requires at with timeouts set to 5 seconds for the 9 × 9 instances, 20
least of order a millisecond to solve all the puzzles; in all but seconds for 16 × 16 and 120 seconds for 25 × 25. These
the most difficult cases, this time is most likely dominated by timeouts are shorter than those used by [33]; however we
the calculations to convert the instance to and from an instance ran our experiments on a faster processor, and with compiler
of the exact cover problem. optimisations enabled. Taken together, these two differences
Overall, we find that the deterministic solvers perform best should amount to a factor of approximately 3 in time. We
on these instances. Either DLX or BS is significantly fastest designed the experiment so that each instance is used for one
for all of the instances. BS is the best performing overall, and run; this is preferable to carrying out multiple runs on each of
is fastest in twelve of the sixteen instances, with DLX fastest a smaller number of instances [37].
in the other four. ACS is significantly faster than ILS in all Figures 4, 5 and 6 show the results for average execution
cases, and faster than DLX in seven of the sixteen instances. time (for successful runs) and success rate for the four
Finally, we note that the times reported by Sabuncu[31] for algorithms. Summary results are given in Table II and the
their ACO algorithm to solve ten of the instances used here are raw data is given in Table III. In Table III, we indicate in bold
typically 1 to 3 seconds. This is several orders of magnitude quantities which are significantly best of all algorithms, and
slower than our times using ACS for the same instances which with asterisks significant differences between the stochastic
are of the order of milliseconds, or less; this is more than can algorithms ACS and ILS. Statistical significance is tested using
be accounted for by differences in hardware or efficiency of the χ2 contingency test for the success rates, and the Mann-
implementation and although we have not performed a direct Whitney U test for the solution times. We use the Mann-
comparison with their code, we can safely assume that our Whitney test here as the vectors of times will in general have
algorithm is the better performing of the two. differing lengths. In cases where we test all algorithms against
each other, we apply the Bonferroni correction to modify the
p-value threshold for signficance.
C. General instances As in [33] and [14], we observe a “phase transition” in
Following [14] and [33], we generated random instances for the difficulty of the instances as a function of the fixed cell
the 9 × 9, 16 × 16 and 25 × 25 Sudoku problem. In the latter fraction; the difficulty is markedly greater at fixed cell fractions
two cases, subgrids are of size 4×4 and 5×5 respectively, and of around 40 − 50%. For low values of the fixed cell fraction,

2475-1502 (c) 2019 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
Authorized licensed use limited to: Northumbria University Library. Downloaded on May 27,2020 at 10:42:37 UTC from IEEE Xplore. Restrictions apply.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TG.2019.2942773, IEEE
Transactions on Games
IEEE TRANSACTIONS ON GAMES 7

Fig. 4. Plots of solution time (left) and success rate (right) against fixed cell Fig. 5. Plots of solution time (left) and success rate (right) against fixed
percentage for runs of ACS, ILS, DLX and BS on the 9 × 9 general instances. cell percentage for runs of ACS, ILS, DLX and BS on the 16 × 16 general
instances.

the search space is large, but there also exist many possible
solutions. As the grid becomes denser, the size of search space
decreases as well as the number of possible solutions. At
around 45%, the combination of rarity of solutions and the
size of the search space leads to a sharp peak in difficulty.
The most difficult puzzles are the 25 × 25 instances with a
fixed cell fraction between 40% and 50%. For these fixed cell
fractions of 40% and 45%, ACS outperforms the other three
algorithms by a significant margin; ACS achieves success rates
of 98% and 85% (compared to 69% and 10% for ILS, 76%
and 49% for DLX, and 21% and 12% for BS). These are
Fig. 6. Plots of solution time (left) and success rate (right) against fixed
the only instances in all the experiments presented for which cell percentage for runs of ACS, ILS, DLX and BS on the 25 × 25 general
one algorithm achieved a significantly higher success rate than instances.
the other three. The mean times achieved by ACS on these
instances are lower than the other three algorithms, but the
difference is not statistically significant – this is most likely solvable instances, and the 25 × 25 general instances (since
due to the small samples of times for the three algorithms these are the most challenging). For the named 9 × 9 logic-
which recorded low numbers of successes. solvable instances, we find that ACS without BVE performs
It is interesting to note the difference in performance be- very poorly on the harder instances (aiescargot, coly013,
tween ACS and BS. These two codes use the same underlying goldennugget, platinumblond, reddwarf, tarx0134), failing to
problem representation and constraint propagation code; the solve these in most cases (see Table IV). Performance on the
only difference between them is the search strategy. This ten instances from [31] is similar to BVE, with the exception
comparison is compelling evidence that ACS is very efficient of sabuncu6, with a success rate of 95%. This suggests that
at searching the solution space, giving markedly improved these ten instances are not sufficiently difficult to provide a
performance on the hardest instances over an exhaustive search
strategy using the same underlying evaluation routines. For
the easier instances, BS outperforms ACS, perhaps due to TABLE II
the simplicity of the algorithm which requires very little S UMMARY OF RESULTS ON THE GENERAL INSTANCES (20 FILLED CELL
FRACTIONS FOR ORDERS 3, 4 AND 5). T HE TABLE SHOWS THE NUMBER
setup compared to ACS, or transformation to another problem OF INSTANCE CATEGORIES FOR WHICH THE ALGORITHM LISTED IN THE
representation, as in DLX. FIRST COLUMN (A LGORITHM 0) PRODUCES A SIGNIFICANTLY HIGHER
ACS returns significantly lower runtimes than ILS, the other SUCCESS RATE , OR LOWER MEAN SOLUTION TIME , THAN THE OTHER
ALGORITHMS (A LGORITHM 1, OR ALL ALGORITHMS ). S IGNIFICANCE IS
stochastic search algorithm, in 52 of the 60 instances, whereas TAKEN AT THE 0.05 LEVEL FOR PAIRWISE COMPARISONS , OR 0.05/3 FOR
ILS is significantly faster than ACS for only two instances. The ONE - AGAINST- ALL COMPARISONS . T HIS DATA IS SUMMARIZED FROM
performance of ACS on these general instances is significantly TABLE III; DETAILS OF THE STATISTICAL TESTS ARE GIVEN IN
S ECTION IV.
better than that of ILS both in terms of overall runtime, and
success rate on the hardest instances. Algorithm 1
Algorithm 0 ACS ILS DLX BS All
ACS - 3 3 3 2
D. Evaluation of Best Value Evaporation Success ILS 0 - 0 0 0
DLX 0 0 - 0 0
In order to evaluate the effectiveness of BVE as an anti- BS 0 0 0 - 0
stagnation mechanism, we ran experiments using the logic ACS - 52 37 11 2
solvable instances (section IV-B) and general instances (sec- Time ILS 3 - 21 3 0
DLX 21 38 - 9 5
tion IV-C) using the ACS algorithm with best-value evapora- BS 48 52 45 - 40
tion disabled by setting ρBV E = 0. We used all the logic-

2475-1502 (c) 2019 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
Authorized licensed use limited to: Northumbria University Library. Downloaded on May 27,2020 at 10:42:37 UTC from IEEE Xplore. Restrictions apply.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TG.2019.2942773, IEEE
Transactions on Games
IEEE TRANSACTIONS ON GAMES 8

TABLE III
S OLUTION RATES ( SOLVED INSTANCES OUT OF 100) AND TIMES ( MEAN AND STANDARD DEVIATION TIME OF SUCCESSFUL RUNS ) FOR THE GENERAL
INSTANCES . O IS THE ORDER OF THE PUZZLE (3 FOR 9 × 9, 4 FOR 16 × 16, 5 FOR 25 × 25) AND F IS THE PERCENTAGE OF GIVEN CELLS . F IGURES IN
BOLD DENOTE QUANTITIES FOR WHICH ONE ALGORITHM IS SIGNIFICANTLY SUPERIOR TO THE OTHER THREE . F OR THE SOLUTION TIMES , THE VECTORS
OF TIMES ARE COMPARED USING THE M ANN -W HITNEY U TEST. S UCCESS RATES ARE COMPARED USING A χ2 CONTINGENCY TEST. I N ALL CASES , THE
B ONFERONNI CORRECTION IS APPLIED , SO THAT p VALUES LESS THAN 0.05/3 ARE TAKEN TO BE SIGNIFICANT. A STERISKS SHOW QUANTITIES FOR
WHICH EITHER ILS OR ACS ARE SIGNIFICANTLY SUPERIOR TO THE OTHER , USING THE SAME TESTS WITH p < 0.05.

O F(%) Solution Rate Solution Time/s

ACS ILS DLX BS ACS ILS DLX BS
3 0 100 100 100 100 0.00187 ± 0.000653∗ 0.00778 ± 0.0022 0.00145 ± 0.000417 0.000433 ± 0.000152
3 5 100 100 100 100 0.00186 ± 0.000693∗ 0.00912 ± 0.00302 0.00132 ± 0.000391 0.000372 ± 0.000138
3 10 100 100 100 100 0.00156 ± 0.000563∗ 0.00974 ± 0.00443 0.00132 ± 0.000509 0.000332 ± 0.000109
3 15 100 100 100 100 0.00156 ± 0.000668∗ 0.0119 ± 0.00627 0.00117 ± 0.000373 0.000300 ± 000101
3 20 100 100 100 100 0.00142 ± 0.000569∗ 0.0152 ± 0.0101 0.00115 ± 0.00041 0.000297 ± 0.000293
3 25 100 100 100 100 0.00117 ± 0.000368∗ 0.0218 ± 0.0276 0.00107 ± 0.000338 0.000219 ± 0.000131
3 30 100 100 100 100 0.00091 ± 0.000354∗ 0.024 ± 0.0266 0.00107 ± 0.000332 0.000226 ± 0.000325
3 35 100 100 100 100 0.000572 ± 0.000256∗ 0.0202 ± 0.0243 0.0011 ± 0.000341 0.000115 ± 0.0000878
3 40 100 100 100 100 0.000266 ± 0.000207∗ 0.0102 ± 0.0178 0.00108 ± 0.000355 (4.52 ± 3.7) × 10−5
3 45 100 100 100 100 0.000129 ± 0.000111∗ 0.00625 ± 0.0176 0.00103 ± 0.000349 (2.04 ± 2.57) × 10−5
3 50 100 100 100 100 (7.21 ± 4.7) × 10−5∗ 0.00176 ± 0.000971 0.00113 ± 0.000412 (1.03 ± 1.06) × 10−5
3 55 100 100 100 100 (5.54 ± 4.98) × 10−5∗ 0.0011 ± 0.000671 0.00114 ± 0.000384 (6.06 ± 6.89) × 10−6
3 60 100 100 100 100 (3.87 ± 1.91) × 10−5∗ 0.00094 ± 0.000526 0.00115 ± 0.000382 (3.27 ± 3.68) × 10−6
3 65 100 100 100 100 (3.5 ± 1.78) × 10−5∗ 0.00076 ± 0.000512 0.00122 ± 0.000441 (2.31 ± 2.38) × 10−6
3 70 100 100 100 100 (3.14 ± 1.34) × 10−5∗ 0.0006 ± 0.00051 0.00127 ± 0.000421 (2.04 ± 1.72) × 10−6
3 75 100 100 100 100 (3.41 ± 1.72) × 10−5 0.00056 ± 0.000571 0.00121 ± 0.000388 (1.88 ± 1.95) × 10−6
3 80 100 100 100 100 (3.24 ± 1.19) × 10−5 0.00051 ± 0.0005 0.00126 ± 0.000453 (1.65 ± 0.876) × 10−6
3 85 100 100 100 100 (3.43 ± 1.31) × 10−5∗ 0.0004 ± 0.00049 0.00127 ± 0.000451 (1.57 ± 0.738) × 10−6
3 90 100 100 100 100 (3.21 ± 1.42) × 10−5 0.00055 ± 0.000497 0.00146 ± 0.00052 (1.58 ± 0.681) × 10−6
3 95 100 100 100 100 (3.43 ± 1.43) × 10−5 0.00046 ± 0.000498 0.00145 ± 0.000511 (1.49 ± 0.700) × 10−6
4 0 100 100 100 100 0.0327 ± 0.0129∗ 0.0802 ± 0.0205 0.0081 ± 0.00213 0.00363 ± 0.00127
4 5 100 100 100 99 0.0258 ± 0.0105∗ 0.0933 ± 0.0248 0.00739 ± 0.0042 0.00342 ± 0.000962
4 10 100 100 100 99 0.0261 ± 0.0103∗ 0.097 ± 0.0249 0.00608 ± 0.00167 0.0033 ± 0.00186
4 15 100 100 100 99 0.0231 ± 0.00846∗ 0.122 ± 0.0411 0.00583 ± 0.00158 0.00441 ± 0.00742
4 20 100 100 100 96 0.0212 ± 0.00958∗ 0.153 ± 0.0618 0.00575 ± 0.0012 0.331 ± 1.57
4 25 100 100 100 90 0.0202 ± 0.00885∗ 0.414 ± 0.945 0.00567 ± 0.00178 0.229 ± 1.87
4 30 100 100 100 89 0.0191 ± 0.0111∗ 0.698 ± 1.3 0.0346 ± 0.287 0.414 ± 2.38
4 35 100 100 100 91 0.0176 ± 0.0102∗ 1.84 ± 2.47 0.036 ± 0.254 0.525 ± 2.41
4 40 100 96 100 100 0.0127 ± 0.00945∗ 1.5 ± 1.98 0.0123 ± 0.0363 0.25 ± 1.27
4 45 100 100 100 100 0.00406 ± 0.00392∗ 0.246 ± 0.21 0.00652 ± 0.00165 0.00248 ± 0.0076
4 50 100 100 100 100 0.000588 ± 0.000564∗ 0.04 ± 0.0422 0.00596 ± 0.00192 0.000121 ± 0.000194
4 55 100 100 100 100 0.000205 ± 0.000162∗ 0.0125 ± 0.00548 0.00604 ± 0.00158 (2.55 ± 2.57) × 10−5
4 60 100 100 100 100 0.000156 ± 0.000106∗ 0.00736 ± 0.00314 0.00638 ± 0.00196 (1.47 ± 1.58) × 10−5
4 65 100 100 100 100 0.000121 ± 0.000059∗ 0.00476 ± 0.00224 0.00634 ± 0.00159 (8.72 ± 9.37) × 10−6
4 70 100 100 100 100 0.000108 ± 0.000050∗ 0.00314 ± 0.00144 0.00684 ± 0.00171 (6.35 ± 6.89) × 10−6
4 75 100 100 100 100 0.000109 ± 0.000050∗ 0.00241 ± 0.00102 0.0067 ± 0.00151 (4.73 ± 5.52) × 10−6
4 80 100 100 100 100 0.000107 ± 0.000041∗ 0.00183 ± 0.000749 0.00734 ± 0.00162 (3.41 ± 2.79) × 10−6
4 85 100 100 100 100 0.000105 ± 0.000044∗ 0.00141 ± 0.000634 0.0076 ± 0.0018 (2.94 ± 1.49) × 10−6
4 90 100 100 100 100 0.000101 ± 0.000041∗ 0.00128 ± 0.000618 0.00805 ± 0.00208 (2.94 ± 1.13) × 10−6
4 95 100 100 100 100 0.000101 ± 0.000033∗ 0.00111 ± 0.000488 0.00805 ± 0.0023 (2.96 ± 1.55) × 10−6
5 0 100 100 100 100 0.731 ± 0.724 0.474 ± 0.0598 0.0476 ± 0.00759 0.0178 ± 0.00351
5 5 100 100 100 98 0.682 ± 0.661 0.561 ± 0.114∗ 0.0582 ± 0.173 0.213 ± 0.672
5 10 100 100 100 95 0.749 ± 0.931 0.715 ± 0.185∗ 0.0347 ± 0.0392 0.173 ± 0.791
5 15 100 100 99 85 1.23 ± 1.41 0.943 ± 0.671 0.0494 ± 0.17 2.54 ± 12.1
5 20 100 100 98 78 1.33 ± 1.53∗ 2.27 ± 2.68 0.0382 ± 0.0281 3.27 ± 13.6
5 25 100 100 96 65 1.93 ± 1.69∗ 7.0 ± 6.54 1.16 ± 5.89 1.69 ± 7.03
5 30 100 100 91 50 2.85 ± 2.52∗ 17.2 ± 8.8 1.94 ± 11.1 5.39 ± 16.5
5 35 100 98 84 35 4.36 ± 3.4∗ 26.7 ± 10.5 3.57 ± 11.0 14.5 ± 29.3
5 40 98∗ 69 76 21 6.15 ± 5.61∗ 47.1 ± 25.7 10.5 ± 19.5 27.1 ± 37.3
5 45 85∗ 10 49 12 8.61 ± 10.1∗ 43.4 ± 26.8 28.3 ± 37.4 53.8 ± 42.1
5 50 93∗ 41 99 92 1.3 ± 4.82∗ 13.6 ± 21.9 1.41 ± 5.73 6.01 ± 14.9
5 55 100 100 100 100 0.00152 ± 0.0082∗ 0.243 ± 0.284 0.0278 ± 0.00469 0.000435 ± 0.00288
5 60 100 100 100 100 0.000341 ± 0.000153∗ 0.0857 ± 0.0275 0.027 ± 0.00411 (4.36 ± 3.19) × 10−5
5 65 100 100 100 100 0.000287 ± 0.000105∗ 0.039 ± 0.019 0.0269 ± 0.00405 (3.09 ± 2.19) × 10−5
5 70 100 100 100 100 0.000246 ± 0.000080∗ 0.019 ± 0.00721 0.0276 ± 0.00416 (2.12 ± 1.12) × 10−5
5 75 100 100 100 100 0.000227 ± 0.000063∗ 0.0109 ± 0.00381 0.0295 ± 0.00426 (1.55 ± 0.596) × 10−5
5 80 100 100 100 100 0.000219 ± 0.000050∗ 0.0077 ± 0.00257 0.0298 ± 0.00407 (1.41 ± 0.576) × 10−5
5 85 100 100 100 100 0.000228 ± 0.000061∗ 0.00584 ± 0.00194 0.0326 ± 0.00502 (1.42 ± 0.494) × 10−5
5 90 100 100 100 100 0.00021 ± 0.000047∗ 0.00399 ± 0.00121 0.0343 ± 0.00527 (1.27 ± 0.232) × 10−5
5 95 100 100 100 100 0.000212 ± 0.000051∗ 0.00289 ± 0.00103 0.0369 ± 0.00548 (1.28 ± 0.309) × 10−5

2475-1502 (c) 2019 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
Authorized licensed use limited to: Northumbria University Library. Downloaded on May 27,2020 at 10:42:37 UTC from IEEE Xplore. Restrictions apply.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TG.2019.2942773, IEEE
Transactions on Games
IEEE TRANSACTIONS ON GAMES 9

TABLE IV
P ERFORMANCE OF ACS WITH AND WITHOUT BVE ON THE SIXTEEN 9 × 9 LOGIC - SOLVABLE INSTANCES AND GENERAL 25 × 25 INSTANCES .
S UCCESS % IS THE NUMBER OF SUCCESSFUL SOLUTIONS FOUND IN 100 RUNS . T IMES ARE GIVEN IN SECONDS , WITH MEAN AND STANDARD DEVIATION
OVER 100 RUNS . N UMBERS IN BOLD INDICATE STATISTICALLY SIGNIFICANT DIFFERENCES BETWEEN THE ALGORITHMS , DETERMINED USING THE
M ANN -W HITNEY U TEST FOR THE TIMES , AND χ2 CONTINGENCY TEST FOR THE SUCCESS RATES .

Success % Solution Time/s

Instance BVE No BVE BVE No BVE
sabuncu1 100 100 (4.8e ± 1.84) × 10−5 (4.55 ± 1.85) × 10−5
sabuncu2 100 100 (4.82 ± 1.73) × 10−5 (4.87 ± 1.82) × 10−5
sabuncu3 100 100 0.000993 ± 0.000457 0.000993 ± 0.000454
sabuncu4 100 100 0.000625 ± 0.000708 0.000563 ± 0.000277
sabuncu5 100 100 (4.62 ± 1.5) × 10−5 (4.8 ± 1.7) × 10−5
sabuncu6 100 93 0.0107 ± 0.00828 0.561 ± 0.976
sabuncu7 100 100 0.00106 ± 0.000986 0.00126 ± 0.0021
sabuncu8 100 100 0.000728 ± 0.000343 0.000713 ± 0.000334
sabuncu9 100 100 0.00163 ± 0.0014 0.00225 ± 0.00324
sabuncu10 100 100 (4.73 ± 1.85) × 10−5 (4.77 ± 1.84) × 10−5
aiescargot 100 59 0.0204 ± 0.0152 0.949 ± 1.36
coly013 100 24 0.0488 ± 0.0518 0.633 ± 1.14
goldennugget 100 34 0.0374 ± 0.0293 0.92 ± 1.15
platinumblond 100 7 0.113 ± 0.0859 1.05 ± 0.905
reddwarf 100 35 0.0404 ± 0.0354 1.64 ± 1.61
tarx0134 100 47 0.0259 ± 0.0193 1.5 ± 1.54
25 × 25 0% 100 100 0.731 ± 0.724 0.503 ± 0.347
25 × 25 5% 100 100 0.682 ± 0.661 0.469 ± 0.297
25 × 25 10% 100 100 0.749 ± 0.931 0.521 ± 0.363
25 × 25 15% 100 100 1.23 ± 1.41 0.581 ± 0.535
25 × 25 20% 100 100 1.33 ± 1.53 0.792 ± 0.594
25 × 25 25% 100 100 1.93 ± 1.69 1.08 ± 0.783
25 × 25 30% 100 100 2.85 ± 2.52 1.77 ± 1.3
25 × 25 35% 100 100 4.36 ± 3.4 4.59 ± 8.25
25 × 25 40% 98 83 6.15 ± 5.61 8.77 ± 15.6
25 × 25 45% 85 49 8.61 ± 10.1 9.14 ± 18.2
25 × 25 50% 93 80 1.3 ± 4.82 1.52 ± 11.0
25 × 25 55% 100 100 0.00152 ± 0.0082 0.00101 ± 0.00273
25 × 25 60% 100 100 0.000341 ± 0.000153 0.000411 ± 0.000208
25 × 25 65% 100 100 0.000287 ± 0.000105 0.000321 ± 0.000105
25 × 25 70% 100 100 0.000246 ± 0.000081 0.000274 ± 0.000075
25 × 25 75% 100 100 0.000227 ± 0.000063 0.000253 ± 0.000063
25 × 25 80% 100 100 0.000219 ± 0.000050 0.000243 ± 0.000055
25 × 25 85% 100 100 0.000228 ± 0.000061 0.000246 ± 0.000057
25 × 25 90% 100 100 0.00021 ± 0.000048 0.000239 ± 0.000049
25 × 25 95% 100 100 0.000212 ± 0.000051 0.000251 ± 0.000054

good benchmark for solution algorithms: the search space after of solutions. Experiments show that our new algorithm signif-
applying constraints is either too small or, as is the case for icantly out-performs existing algorithms on the hardest, large
four of the instances, non-existent. instances of Sudoku, and we provide evidence that our method
We also evaluated BVE using the general 25 × 25 instances. provides a much more efficient search of the solution space
We see that the performance of ACS is significantly degraded than traditional backtracking algorithms for these problems.
without the BVE operator. Performance with respect to solu- For smaller or easier instances, we find that direct search
tion time is degraded to some extent, with significantly shorter algorithms such as Dancing Links or Backtracking Search
times without BVE in three fixed cell fractions, compared outperform stochastic algorithms, but these deterministic al-
to nine which are faster with BVE. The number of failures gorithms perform poorly on the hardest instances. Finally, we
is significantly higher; for the 45% fixed cell instances for find that our algorithm outperforms the state of the art Iterated
example, the success rate is 58%, compared to 92% with BVE Local Search algorithm [33] both in terms of runtime and
enabled. The average solution time for these instances is 9.1s, success rates on hard instances.
well within the timeout of 120s, suggesting that the failures The growing body of work on the automated solution of
are due to the search stagnating at a local minimum. pencil puzzles such as Sudoku and Nurikabe suggests that they
offer a ready-made algorithmic test-bed. As such, they may
provide an additional challenge for general-purpose algorithms
V. C ONCLUSIONS
(whether AI-based or not), and offer new insights into the
In this paper we presented a new algorithm for the Sudoku solution of constraint satisfaction problems (by, for example,
puzzle, based on Ant Colony Optimization. Our method in- suggesting new ways in which to search the solution space).
cludes a new operator, which we call Best Value Evaporation, Importantly, solvers such as ours can out-perform state-of-
and we show that this addition to the base algorithm is essen- the-art methods without any requirement for problem-specific
tial for the prevention of premature convergence or stagnation heuristics, which immediately offers two possibilities for fu-

2475-1502 (c) 2019 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
Authorized licensed use limited to: Northumbria University Library. Downloaded on May 27,2020 at 10:42:37 UTC from IEEE Xplore. Restrictions apply.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TG.2019.2942773, IEEE
Transactions on Games
IEEE TRANSACTIONS ON GAMES 10

ture work in this area. The first is a “problem agnostic” general [21] Z. Karimi-Dehkordi, K. Zamanifar, A. Baraani-Dastjerdi, and
Japanese pencil puzzle solver, which can solve large instances N. Ghasem-Aghaee, “Sudoku using parallel simulated annealing,”
in International Conference in Swarm Intelligence (ICSI). Springer,
of any problem in this class. By constructing this solver in a 2010, pp. 461–467.
modular fashion, we should easily be able to incorporate any [22] R. Soto, B. Crawford, C. Galleguillos, E. Monfroy, and F. Paredes, “A
suitable pencil puzzle, which will minimize the amount of hybrid ac3-tabu search algorithm for solving Sudoku puzzles,” Expert
Systems with Applications, vol. 40, no. 15, pp. 5817–5821, 2013.
effort required in future research. Importantly, this will allow [23] J. Gunther and T. Moon, “Entropy minimization for solving Sudoku,”
for the rapid (and experimentally consistent) solution of a wide IEEE Transactions on Signal Processing, vol. 60, no. 1, pp. 508–513,
range of pencil puzzles, which will (a) yield good solutions to 2012.
[24] M. Dorigo and G. Di Caro, “Ant colony optimization: a new meta-
these problems per se, (b) allow for easy comparison of the heuristic,” in Proceedings of the 1999 Congress on Evolutionary Com-
properties of those problems, and (c) provide a ready-made putation (CEC), vol. 2. IEEE, 1999, pp. 1470–1477.
platform for the subsequent investigation of problem-specific [25] M. Dorigo, V. Maniezzo, and A. Colorni, “Ant system: optimization by
a colony of cooperating agents,” IEEE Transactions on Systems, Man,
heuristics. and Cybernetics, Part B (Cybernetics), vol. 26, no. 1, pp. 29–41, 1996.
[26] M. Dorigo and M. Birattari, “Ant colony optimization,” in Encyclopedia
of Machine Learning. Springer, 2011, pp. 36–39.
[27] M. López-Ibáñez, T. Stützle, and M. Dorigo, “Ant colony optimization:
R EFERENCES A component-wise overview,” Handbook of Heuristics, pp. 1–37, 2016.
[28] M. Dorigo and T. Stützle, “Ant colony optimization: overview and recent
[1] J.-P. Delahaye, “The science behind Sudoku,” Scientific American, vol. advances,” in Handbook of Metaheuristics. Springer, 2019, pp. 311–
294, no. 6, pp. 80–87, 2006. 351.
[2] P. Norvig, “Solving every Sudoku puzzle,” available at [29] T. Mantere, “Improved ant colony genetic algorithm hybrid for Sudoku
http://norvig.com/sudoku.html, accessed March 13, 2018. solving,” in Third World Congress on Information and Communication
[3] M. R. Garey and D. S. Johnson, Computers and Intractability: A Guide Technologies (WICT). IEEE, 2013, pp. 274–279.
to the Theory of NP-Completness. WH Freeman: New York, 1979. [30] K. Schiff, “An ant algorithm for the Sudoku problem,” Journal of
[4] T. Yato and T. Seta, “Complexity and completeness of finding an- Automation, Mobile Robotics and Intelligent Systems, vol. 9, 2015.
other solution and its application to puzzles,” IEICE Transactions on [31] I. Sabuncu, “Work-in-progress: solving Sudoku puzzles using hybrid
Fundamentals of Electronics, Communications and Computer Sciences, ant colony optimization algorithm,” in 1st International Conference on
vol. 86, no. 5, pp. 1052–1060, 2003. Industrial Networks and Intelligent Systems (INISCom). IEEE, 2015,
[5] C. J. Colbourn, “The complexity of completing partial Latin squares,” pp. 181–184.
Discrete Applied Mathematics, vol. 8, no. 1, pp. 25–30, 1984. [32] A. Inkala, AI Escargot - The Most Difficult Sudoku Puzzle. Lulu.com,
[6] G. N. Yannakakis and J. Togelius, Artificial Intelligence and Games. Finland, 2007.
Springer, 2018. [33] N. Musliu and F. Winter, “A hybrid approach for the Sudoku problem:
[7] R. M. Karp, “Reducibility among combinatorial problems,” in Complex- using constraint programming in iterated local search,” IEEE Intelligent
ity of Computer Computations. Springer, 1972, pp. 85–103. Systems, vol. 32, no. 2, pp. 52–62, 2017.
[8] D. E. Knuth, “Dancing links,” arXiv preprint cs/0011047, 2000. [34] M. Dorigo and L. M. Gambardella, “Ant colony system: a cooperative
[9] M. Hunt, C. Pong, and G. Tucker, “Difficulty-driven Sudoku puzzle learning approach to the Traveling Salesman Problem,” IEEE Transac-
generation,” UMAP Journal, vol. 29, no. 3, pp. 343–361, 2007. tions on Evolutionary Computation, vol. 1, no. 1, pp. 53–66, 1997.
[10] S. Fletcher, F. Johnson, and D. R. Morrison, “Taking the mystery out of [35] J. Laire, “dlx-cpp,” available at https://github.com/jlaire/dlx-cpp, ac-
Sudoku difficulty: an Oracular model,” UMAP Journal, vol. 29, no. 3, cessed April 23, 2018.
pp. 327–341, 2007. [36] M. Ercsey-Ravasz and Z. Toroczkai, “The chaos within Sudoku,” Sci.
[11] T. Weber, “A SAT-based Sudoku solver,” in The 12th International Con- Rep., vol. 2, pp. 725–733, 2012.
ference on Logic for Programming, Artificial Intelligence, and Reasoning [37] M. Birattari, “On the estimation of the expected performance of a
(LPAR): Short Paper Proceedings, G. Sutcliffe and A. Voronkov, Eds., metaheuristic on a class of instances. how many instances, how many
2005, pp. 11–15. runs?” IRIDIA, Université Libre de Bruxelles, Brussels, Belgium, Tech.
Rep. TR/IRIDIA/2004-001, 2004.
[12] J. A. Pacurib, G. M. M. Seno, and J. P. T. Yusiong, “Solving Sudoku
puzzles using improved artificial bee colony algorithm,” in Fourth
International Conference on Innovative Computing, Information and
Control (ICICIC). IEEE, 2009, pp. 885–888.
[13] B. Crawford, M. Aranda, C. Castro, and E. Monfroy, “Using constraint Huw Lloyd is a Senior Lecturer at Manchester
programming to solve Sudoku puzzles,” in Third International Con- Metropolitan University. He was awarded his B.Sc.
ference on Convergence and Hybrid Information Technology (ICCIT), in Physics by Imperial College, London, and his
vol. 2. IEEE, 2008, pp. 926–931. Ph.D. in Astrophysics by the University of Manch-
[14] R. Lewis, “Metaheuristics can solve Sudoku puzzles,” Journal of Heuris- ester.
tics, vol. 13, no. 4, pp. 387–401, 2007.
[15] X. Q. Deng and Y. Da Li, “A novel hybrid genetic algorithm for solving
Sudoku puzzles,” Optimization Letters, vol. 7, no. 2, pp. 241–257, 2013.
[16] T. Mantere and J. Koljonen, “Solving, rating and generating Sudoku
puzzles with GA,” in IEEE Congress on Evolutionary Computation
(CEC). IEEE, 2007, pp. 1382–1389.
[17] C. Segura, S. I. V. Peña, S. B. Rionda, and A. H. Aguirre, “The
importance of diversity in the application of evolutionary algorithms to
the Sudoku problem,” in IEEE Congress on Evolutionary Computation
(CEC). IEEE, 2016, pp. 919–926. Martyn Amos is Professor of Computer and Infor-
[18] Z. Wang, T. Yasuda, and K. Ohkura, “An evolutionary approach to Su- mation Sciences at Northumbria University. He was
doku puzzles with filtered mutations,” in IEEE Congress on Evolutionary awarded his B.Sc. in Computer Science by Coventry
Computation (CEC). IEEE, 2015, pp. 1732–1737. University, and his Ph.D. in DNA computation by
[19] J. M. Hereford and H. Gerlach, “Integer-valued particle swarm optimiza- the University of Warwick. He is a Fellow of the
tion applied to Sudoku puzzles,” in IEEE Swarm Intelligence Symposium British Computer Society.
(SIS). IEEE, 2008, pp. 1–7.
[20] A. Moraglio and J. Togelius, “Geometric particle swarm optimization
for the Sudoku puzzle,” in Proceedings of the 9th Annual Conference
on Genetic and Evolutionary Computation (GECCO). ACM, 2007, pp.
118–125.

2475-1502 (c) 2019 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
Authorized licensed use limited to: Northumbria University Library. Downloaded on May 27,2020 at 10:42:37 UTC from IEEE Xplore. Restrictions apply.
View publication stats