Ranking of Tournament from pairwise comparison

Ranking from pairwise comparisons is a ubiquitous problem and has been studied in disciplines
ranging from statistics to operations research and from theoretical computer science to
machine learning. Here we are interested in the following general question: There are n items
and an unknown preference matrix P ∈ [0, 1]n×n , with Pij + Pji = 1 ∀i, j, such that whenever
items i and j are compared, item i beats item j with probability Pij and j beats i with probability
Pij = 1 − Pij . Given the ability to make comparisons among only O(n log n) pairs, which can be
compared more than once but must be chosen non-actively, i.e. before observing the outcomes
of any comparisons, under what conditions on P can we learn a good ranking of the n items?
This question is important in any pairwise comparison setting where the number of items n is
large and decisions on which pairs to compare cannot be made adaptively, as is often the case
for example in consumer surveys.

Traditional sorting algorithms in computer science, which apply when P corresponds to a full
deterministic ordering of the n items, i.e. when there exists an ordering or permutation σ ∈ Sn
such that Pij = 1 for all i, j with σ(i) < σ(j), require the O(n log n) pairs to be chosen in an active
manner (some pairs can be chosen only after observing the comparison outcomes of previous
pairs). Recent study showed that if P follows a noisy permutation (NP) model, i.e. if there is a
permutation σ ∈ Sn and a noise parameter p ∈ [0,1] such that Pij = 1 – p for all i, j with σ(i) <
σ(j), then one can use noisy sorting algorithms to learn a ranking close to σ by observing
comparisons of only O(n log n) pairs (using only one comparison per pair), but again, these
algorithms require active selection of pairs.

Problem statement: Let σ denote the ranking to be learned. The objective is to learn the
ranking by querying the reference for pairwise comparisons of the form qi,j := {θi ≺ θj}. (2) The
response or label of qi,j is binary and denoted as yi,j := 1{qi,j} where 1 is the indicator function;
ties are not allowed. The main results quantify the minimum number of queries or labels
required to determine the reference’s ranking, and they are based on two key assumptions. A1
Embedding: The set of n objects are embedded in R d (in general position) and we will also use
θ1, . . . , θn to refer to their (known) locations in R d . Every ranking σ can be specified by a
reference point rσ ∈ R d , as follows. The Euclidean distances between the reference and
objects are consistent with the ranking in the following sense: if the σ ranks θi ≺ θj , then kθi −
rσk < kθj − rσk. Let Σn,d denote the set of all possible rankings of the n objects that satisfy this
embedding condition

So we tried designing an algorithm for a specific case of pairwise ranking which is based on
marno carlo stimulation. This we tried for a specific case of the problem so as to conjecture a
statement that we r hoping to prove. We took a specific case of the problem, rank of the
corresponding probability matrix to be 7 and 8. We developed a graph isomorphism algorithm
which worked quite faster than the normal ones. With this graph isomorphism algorithm, we
stimulated tournaments of rank 7 and rank 8 using the embedding model of tournaments.
Throughout the code Numpy module has been a great advantage to us. We created random
7*4 vectors to finally stimulate random graphs. These vectors were multiplied by a rotational
matrix to decide the direction of the edges. Since a tournament is a Complete graph on n
nodes, it was easy for us to work on graphs. Most of the theoretical works were straightforward
to understand. Then we took a list of all isomorphic classes of graph of rank 7 and started to
remove the graphs which appeared in the stimulation. So that finally we get forbidden graphs
of rank 7. This would help us to establish ourselves in our basic assumptions. We required a
high end server for this as graph isomorphism is a NP Hard problem the time complexity is very
high to work in a local computer. We carried on the stimulation after the arrival of a high end
server from RBCDSCAI. We got the results as we expected on. There were no forbidden graphs
on rank 7. We moved on with rank 8 with some changes in the code. And as we thought we got
the exact same results. Concepts of flip graphs helped us to speed up our algorithm.
Throughout our experimentation BTL-models were our prime consideration. This had been
great use because it contains stochastic transitivity. So we were successful in conjecturing the
statement.

Traditional optimization losses targeting this objective are intuitive (e.g., count the number of
inversions between the training data and the scoring function,) but discontinuous and non-
convex. The substantial literature on learning to rank can be specialized to this setting by
learning scoring functions that only depend on the object identity. This approach suggests ways
to approximately solve the optimization problem by relaxing the intractable loss to convex
surrogates. Efficient Ranking from Pairwise Comparisons Although some of these methods (e.g.,
the SVM) can achieve an Ω(n) lower bound on a certain sample complexity, we feel that
optimization-based approaches may be unnecessarily complex in this situation. The question
arises whether simpler algorithms could be equally effective.

We focus on pairwise comparison queries for two reasons. First, pairwise comparisons admit a
half space representation in embedding spaces which allows for a geometrical approach to
learning in such structured ranking spaces. Second, pairwise comparisons are the most
common form of queries in many applications, especially those involving human subjects. For
example, consider the problem of finding the most highly ranked object, as illustrated by the
following familiar task. Suppose a patient needs a new pair of prescription eye lenses. Faced
with literally millions of possible prescriptions, the doctor will present candidate prescriptions
in a sequential fashion followed by the query: better or worse? Even if certain queries are
repeated to account for possible inaccurate answers, the doctor can locate an accurate
prescription with just a handful of queries. This is possible presumably because the doctor
understands (at least intuitively) the intrinsic space of prescriptions and can efficiently search
through it using only binary responses from the patient. We assume that the objects can be
embedded in R d and that the distances between objects and the reference are consistent with
the ranking (Assumption A1). The problem of learning a general function f : R d → R using just
pairwise comparisons that correctly ranks the objects embedded in R d has previously been
studied in the passive setting [12, 13, 14, 15]. The main contributions of this paper are
theoretical bounds for the specific case when f(x) = ||x − rσ|| where rσ ∈ R d is the reference
point. This is a standard model used in multidimensional unfolding and psychometrics [7, 18]
and one can show that this model also contains the familiar functions f(x) = r T σ x for all rσ ∈ R
d. We are unaware of any existing query-complexity bounds for this problem. We do not
assume a generative model is responsible for the relationship between rankings to
embedding’s, but one could. For example, the objects might have an embedding (in a feature
space) and the ranking is generated by distances in this space. Or alternatively, structural
constraints on the space of rankings could be used to generate a consistent embedding.
Assumption A1, while arguably quite natural/reasonable in many situations, significantly
constrains the set of possible rankings.

As we proceeded with random stimulations of graphs of rank 7 we did not get any forbidden
configuration. This went as we expected. Graphs of rank 8 gave also gave us 2 forbidden
configurations with a 7 cycle in it as we expected. This experimentally proven result has to
extended for Graphs of larger size using algebra and graph theory. Due to time constrain of my
intern period I was not able to work further and extend the results.

So the conclusion was extension of what professor Arun Rajkumar did for rank 2 Graphs worked
successfully for rank 4 graphs.