CAPTCHA:

Using Hard AI Problems For Security

Luis von Ahn1 , Manuel Blum1 , Nicholas J. Hopper1 , and John Langford2
1
Computer Science Dept., Carnegie Mellon University, Pittsburgh PA 15213, USA
2
IBM T.J. Watson Research Center, Yorktown Heights NY 10598, USA

Abstract. We introduce captcha, an automated test that humans can

pass, but current computer programs can’t pass: any program that has
high success over a captcha can be used to solve an unsolved Artifi-
cial Intelligence (AI) problem. We provide several novel constructions of
captchas. Since captchas have many applications in practical secu-
rity, our approach introduces a new class of hard problems that can be
exploited for security purposes. Much like research in cryptography has
had a positive impact on algorithms for factoring and discrete log, we
hope that the use of hard AI problems for security purposes allows us
to advance the field of Artificial Intelligence. We introduce two families
of AI problems that can be used to construct captchas and we show
that solutions to such problems can be used for steganographic commu-
nication. captchas based on these AI problem families, then, imply a
win-win situation: either the problems remain unsolved and there is a
way to differentiate humans from computers, or the problems are solved
and there is a way to communicate covertly on some channels.

1 Introduction

A captcha is a program that can generate and grade tests that: (A) most
humans can pass, but (B) current computer programs can’t pass. Such a program
can be used to differentiate humans from computers and has many applications
for practical security, including (but not limited to):

– Online Polls. In November 1999, slashdot.com released an online poll ask-

ing which was the best graduate school in computer science (a dangerous
question to ask over the web!). As is the case with most online polls, IP
addresses of voters were recorded in order to prevent single users from vot-
ing more than once. However, students at Carnegie Mellon found a way to
stuff the ballots by using programs that voted for CMU thousands of times.
CMU’s score started growing rapidly. The next day, students at MIT wrote
their own voting program and the poll became a contest between voting
“bots”. MIT finished with 21,156 votes, Carnegie Mellon with 21,032 and
every other school with less than 1,000. Can the result of any online poll be
trusted? Not unless the poll requires that only humans can vote.
 – Free Email Services. Several companies (Yahoo!, Microsoft, etc.) offer free
email services, most of which suffer from a specific type of attack: “bots”
that sign up for thousands of email accounts every minute. This situation
can be improved by requiring users to prove they are human before they can
get a free email account. Yahoo!, for instance, uses a captcha of our design
to prevent bots from registering for accounts. Their captcha asks users
to read a distorted word such as the one shown below (current computer
programs are not as good as humans at reading distorted text).

Fig. 1. The Yahoo! captcha.

– Search Engine Bots. Some web sites don’t want to be indexed by search
engines. There is an html tag to prevent search engine bots from reading
web pages, but the tag doesn’t guarantee that bots won’t read the pages; it
only serves to say “no bots, please”. Search engine bots, since they usually
belong to large companies, respect web pages that don’t want to allow them
in. However, in order to truly guarantee that bots won’t enter a web site,
captchas are needed.
– Worms and Spam. captchas also offer a plausible solution against email
worms and spam: only accept an email if you know there is a human be-
hind the other computer. A few companies, such as www.spamarrest.com are
already marketing this idea.
– Preventing Dictionary Attacks. Pinkas and Sander [11] have suggested
using captchas to prevent dictionary attacks in password systems. The idea
is simple: prevent a computer from being able to iterate through the entire
space of passwords by requiring a human to type the passwords.

The goals of this paper are to lay a solid theoretical foundation for captchas,
to introduce the concept to the cryptography community, and to present several
novel constructions.

Lazy Cryptographers Doing AI

Note that from a mechanistic point of view, there is no way to prove that a
program cannot pass a test which a human can pass, since there is a program —
the human brain — which passes the test. All we can do is to present evidence
that it’s hard to write a program that can pass the test. In this paper, we take
an approach familiar to cryptographers: investigate state-of-the-art algorithmic
developments having to do with some problem, assume that the adversary does
not have algorithms for that problem that are are much better than the state-
of-the-art algorithms, and then prove a reduction between passing a test and
exceeding the performance of state-of-the-art algorithms. In the case of ordi-
nary cryptography, it is assumed (for example) that the adversary cannot factor
1024-bit integers in any reasonable amount of time. In our case, we assume that
the adversary cannot solve an Artificial Intelligence problem with higher accu-
racy than what’s currently known to the AI community. This approach, if it
achieves widespread adoption, has the beneficial side effect of inducing security
researchers, as well as otherwise malicious programmers, to advance the field of
AI (much like computational number theory has been advanced since the advent
of modern cryptography).

A captcha is a cryptographic protocol whose underlying hardness as-

sumption is based on an AI problem.

An important component of the success of modern cryptography is the practice

of stating, very precisely and clearly, the assumptions under which cryptographic
protocols are secure. This allows the rest of the community to evaluate the as-
sumptions and to attempt to break them. In the case of Artificial Intelligence, it’s
rare for problems to be precisely stated, but using them for security purposes
forces protocol designers to do so. We believe that precisely stating unsolved
AI problems can accelerate the development of Artificial Intelligence: most AI
problems that have been precisely stated and publicized have eventually been
solved (take chess as an example). For this reason it makes practical sense for
AI problems that are used for security purposes to also be useful. If the under-
lying AI problem is useful, a captcha implies a win-win situation: either the
captcha is not broken and there is a way to differentiate humans from comput-
ers, or the captcha is broken and a useful AI problem is solved. Such is not the
case for most other cryptographic assumptions: the primary reason algorithms
for factoring large numbers are useful is because factoring has applications in
cryptanalysis.
In this paper we will present constructions of captchas based on certain
AI problems and we will show that solving the captchas implies solving the
AI problems. The AI problems we chose have several applications, and we will
show that solutions to them can be used, among other things, for steganographic
communication (see Section 5).

Related Work

The first mention of ideas related to “Automated Turing Tests” seems to appear
in an unpublished manuscript by Moni Naor [10]. This excellent manuscript
contains some of the crucial notions and intuitions, but gives no proposal for an
Automated Turing Test, nor a formal definition. The first practical example of
an Automated Turing Test was the system developed by Altavista [8] to prevent
“bots” from automatically registering web pages. Their system was based on
the difficulty of reading slightly distorted characters and worked well in practice,
but was only meant to defeat off-the-shelf Optical Character Recognition (OCR)
technology. (Coates et al [5], inspired by our work, and Xu et al [14] developed
similar systems and provided more concrete analyses.) In 2000 [1], we introduced
the notion of a captcha as well as several practical proposals for Automated
Turing Tests.
This paper is the first to conduct a rigorous investigation of Automated
Turing Tests and to address the issue of proving that it is difficult to write a
computer program that can pass the tests. This, in turn, leads to a discussion
of using AI problems for security purposes, which has never appeared in the
literature. We also introduce the first Automated Turing Tests not based on the
difficulty of Optical Character Recognition. A related general interest paper [2]
has been accepted by Communications of the ACM. That paper reports on our
work, without formalizing the notions or providing security guarantees.

2 Definitions and Notation

Let C be a probability distribution. We use [C] to denote the support of C. If

P (·) is a probabilistic program, we will denote by Pr (·) the deterministic program
that results when P uses random coins r.
Let (P, V ) be a pair of probabilistic interacting programs. We denote the
output of V after the interaction between P and V with random coins u1 and u2 ,
assuming this interaction terminates, by hPu1 , Vu2 i (the subscripts are omitted
in case the programs are deterministic). A program V is called a test if for all
P and u1 , u2 , the interaction between Pu1 and Vu2 terminates and hPu1 , Vu2 i ∈
{accept, reject}. We call V the verifier or tester and any P which interacts with
V the prover.

Definition 1. Define the success of an entity A over a test V by

SuccVA = Pr0 [hAr , Vr0 i = accept].

r,r

We assume that A can have precise knowledge of how V works; the only piece
of information that A can’t know is r 0 , the internal randomness of V .

CAPTCHA

Intuitively, a captcha is a test V over which most humans have success close
to 1, and for which it is hard to write a computer program that has high success
over V . We will say that it is hard to write a computer program that has high
success over V if any program that has high success over V can be used to solve
a hard AI problem.

Definition 2. A test V is said to be (α, β)-human executable if at least an α

portion of the human population has success greater than β over V .
Notice that a statement of the form “V is (α, β)-human executable” can only
be proven empirically. Also, the success of different groups of humans might
depend on their origin language or sensory disabilities: color-blind individuals,
for instance, might have low success on tests that require the differentiation of
colors.

Definition 3. An AI problem is a triple P = (S, D, f ), where S is a set of

problem instances, D is a probability distribution over the problem set S, and
f : S → {0, 1}∗ answers the instances. Let δ ∈ (0, 1]. We require that for an
α > 0 fraction of the humans H, Prx←D [H(x) = f (x)] > δ.

Definition 4. An AI problem P is said to be (δ, τ )-solved if there exists a pro-

gram A, running in time at most τ on any input from S, such that

Pr [Ar (x) = f (x)] ≥ δ .

x←D,r

(A is said to be a (δ, τ ) solution to P.) P is said to be a (δ, τ )-hard AI problem

if no current program is a (δ, τ ) solution to P, and the AI community agrees it
is hard to find such a solution.

Definition 5. A (α, β, η)-captcha is a test V that is (α, β)-human executable,

and which has the following property:
There exists a (δ, τ )-hard AI problem P and a program A, such that if
B has success greater than η over V then AB is a (δ, τ ) solution to P.
(Here AB is defined to take into account B’s running time too.)

We stress that V should be a program whose code is publicly available.

Remarks
1. The definition of an AI problem as a triple (S, D, f ) should not be inspected
with a philosophical eye. We are not trying to capture all the problems that
fall under the umbrella of Artificial Intelligence. We want the definition to
be easy to understand, we want some AI problems to be captured by it, and
we want the AI community to agree that these are indeed hard AI problems.
More complex definitions can be substituted for Definition 3 and the rest of
the paper remains unaffected.
2. A crucial characteristic of an AI problem is that a certain fraction of the
human population be able to solve it. Notice that we don’t impose a limit
on how long it would take humans to solve the problem. All that we require
is that some humans be able to solve it (even if we have to assume they will
live hundreds of years to do so). The case is not the same for captchas.
Although our definition says nothing about how long it should take a human
to solve a captcha, it is preferable for humans to be able to solve captchas
in a very short time. captchas which take a long time for humans to solve
are probably useless for all practical purposes.
AI Problems as Security Primitives

Notice that we define hard in terms of the consensus of a community: an AI

problem is said to be hard if the people working on it agree that it’s hard.
This notion should not be surprising to cryptographers: the security of most
modern cryptosystems is based on assumptions agreed upon by the community
(e.g., we assume that 1024-bit integers can’t be factored). The concept of a
hard AI problem as a foundational assumption, of course, is more questionable
than P 6= N P , since many people in the AI community agree that all hard AI
problems are eventually going to be solved. However, hard AI problems may be a
more reasonable assumption than the hardness of factoring, given the possibility
of constructing a quantum computer. Moreover, even if factoring is shown to be
hard in an asymptotic sense, picking a concrete value for the security parameter
usually means making an assumption about current factoring algorithms: we
only assume that current factoring algorithms that run in current computers
can’t factor 1024-bit integers. In the same way that AI researchers believe that
all AI problems will be solved eventually, we believe that at some point we will
have the computational power and algorithmic ability to factor 1024-bit integers.
(Shamir and Tromer [13], for instance, have proposed a machine that could factor
1024-bit integers; the machine would cost about ten million dollars in materials.)
An important difference between popular cryptographic primitives and AI
problems is the notion of a security parameter. If we believe that an adversary can
factor 1024-bit integers, we can use 2048-bit integers instead. No such concept
exists in hard AI problems. AI problems, as we have defined them, do not deal
with asymptotics. However, as long as there is a small gap between human and
computer ability with respect to some problem, this problem can potentially
be used as a primitive for security: rather than asking the prover to solve the
problem once, we can ask it to solve the problem twice. If the prover gets good
at solving the problem twice, we can ask it to solve the problem three times, etc.
There is an additional factor that simplifies the use of hard AI problems
as security primitives. Most applications of captchas require the tests to be
answered within a short time after they are presented. If a new program solves
the hard AI problems that are currently used, then a different set of problems
can be used, and the new program cannot affect the security of applications
that were run before it was developed. Compare this to encryption schemes: in
many applications the information that is encrypted must remain confidential
for years, and therefore the underlying problem must be hard against programs
that run for a long time, and against programs that will be developed in the
future.1
We also note that not all hard AI problems can be used to construct a
captcha. In order for an AI problem to be useful for security purposes, there
needs to be an automated way to generate problem instances along with their
solution. The case is similar for computational problems: not all hard computa-
tional problems yield cryptographic primitives.
1
We thank one of our anonymous Eurocrypt reviewers for pointing this out.
Who Knows What?

Our definitions imply that an adversary attempting to write a program that

has high success over a captcha knows exactly how the captcha works. The
only piece of information that is hidden from the adversary is a small amount
of randomness that the verifier uses in each interaction.
This choice greatly affects the nature of our definitions and makes the prob-
lem of creating captchas more challenging. Imagine an Automated Turing Test
that owns a large secret book written in English and to test an entity A it ei-
ther picks a paragraph from its secret book or generates a paragraph using the
best known text-generation algorithm, and then asks A whether the paragraph
makes sense (the best text-generation algorithms cannot produce an entire para-
graph that would make sense to a human being). Such an Automated Turing
Test might be able to distinguish humans from computers (it is usually the case
that the best text-generation algorithms and the best algorithms that try to de-
termine whether something makes sense are tightly related). However, this test
cannot be a captcha: an adversary with knowledge of the secret book could
achieve high success against this test without advancing the algorithmic state
of the art. We do not allow captchas to base their security in the secrecy of a
database or a piece of code.

Gap Amplification

We stress that any positive gap between the success of humans and current
computer programs against a captcha can be amplified to a gap arbitrarily
close to 1 by serial repetition. The case for parallel repetition is more complicated
and is addressed by Bellare, Impagliazzo and Naor in [3].
Let V be an (α, β, η)-captcha, and let Vkm be the test that results by repeat-
ing V m times in series (with fresh new randomness each time) and accepting
only if the prover passes V more than k times. Then for any  > 0 there exist
m and k with 0 ≤ k ≤ m such that Vkm is an (α, 1 − , )-captcha. In gen-
eral, we will have m = O(1/(β − η)2 ln(1/)) and sometimes much smaller. Since
captchas involve human use, it is desirable to find the smallest m possible.
This can be done by solving the following optimization problem:

m   k  
( )
X m i X m
min ∃k : β (1 − β)m−i ≥ 1 −  & η i (1 − η)m−i ≤ 
m i i=0
i
i=k+1

Notice that amplifying a gap can roughly be thought of as increasing the

security parameter of a captcha: if the best computer program now has success
0.10 against a given captcha (for example), then we can ask the prover to pass
the captcha twice (in series) to reduce the best computer program’s success
probability to 0.01.
3 Two AI Problem Families

In this section we introduce two families of AI problems that can be used to

construct captchas. The section can be viewed as a precise statement of the
kind of hardness assumptions that our cryptographic protocols are based on. We
stress that solutions to the problems will also be shown to be useful.
For the purposes of this paper, we define an image as an h × w matrix (h
for height and w for width) whose entries are pixels. A pixel is defined as a
triple of integers (R, G, B), where 0 ≤ R, G, B ≤ M for some constant M . An
image transformation is a function that takes as input an image and outputs
another image (not necessarily of the same width and height). Examples of
image transformations include: turning an image into its black-and-white version,
changing its size, etc.
Let I be a distribution on images and T be a distribution on image transfor-
mations. We assume for simplicity that if i, i0 ∈ [I] and i 6= i0 then T (i) 6= T 0 (i0 )
for any T, T 0 ∈ [T ]. (Recall that [I] denotes the support of I.)

Problem Family (P1). Consider the following experiment: choose an image

i ← I and choose a transformation t ← T ; output t(i). P1I,T consists of writing
a program that takes t(i) as input and outputs i (we assume that the program
has precise knowledge of T and I). More formally, let SI,T = {t(i) : t ∈ [T ]
and i ∈ [I]}, DI,T be the distribution on SI,T that results from performing
the above experiment and fI,T : SI,T → [I] be such that fI,T (t(i)) = i. Then
P1I,T = (SI,T , DI,T , fI,T ).

Problem Family (P2). In addition to the distributions I and T , let L be a

finite set of “labels”. Let λ : [I] → L compute the label of an image. The set
of problem instances is SI,T = {t(i) : t ∈ [T ] and i ∈ [I]}, and the distribution
on instances DI,T is the one induced by choosing i ← I and t ← T . Define
gI,T ,λ so that gI,T ,λ (t(i)) = λ(i). Then P2I,T ,λ = (SI,T , DI,T , gI,T ,λ ) consists
of writing a program that takes t(i) as input and outputs λ(i).

Remarks

1. Note that a (δ, τ ) solution A to an instance of P1 also yields a (δ 0 , τ + τ 0 )

solution to an instance of P2 (where δ 0 ≥ δ and τ 0 ≤ log |[I]| is the time that
it takes to compute λ), specifically, computing λ(A(x)). However, this may
be unsatisfactory for small δ, and we might hope to do better by restricting
to a smaller set of labels. Conversely, P1 can be seen as a special case of
P2 with λ the identity function and L = [I]. Formally, problem families P1
and P2 can be shown to be isomorphic. Nonetheless, it is useful to make a
distinction here because in some applications it appears unnatural to talk
about labels.
2. We stress that in all the instantiations of P1 and P2 that we consider, I, T
and, in the case of P2, λ, will be such that humans have no problem solving
P1I,T and P2I,T ,λ . That is, [T ] is a set of transformations that humans
 can easily undo in [I]. Additionally, it has to be possible to perform all the
transformations in [T ] using current computer programs.
3. There is nothing specific to images about these problem definitions; any other
space of objects which humans recognize under reasonable transformations
(e.g., organized sounds such as music or speech, animations, et cetera) could
be substituted without changing our results.
4. It is easy to build a (δI , τI )-solution to P1I,T , where δI = max{Prj←I [j =
i] : i ∈ [I]} and τI is the time that it takes to describe an element of
[I], by always guessing the image with the highest probability in I. Sim-
ilarly, it is easy to build a (δI,λ , τI,λ )-solution to P2I,T ,λ , where δI,λ =
max{Prj←I [λ(j) = λ(i)] : i ∈ [I]} and τI,λ is the time that it takes to de-
scribe a label in L. Therefore, we restrict our attention to finding solutions
to P1I,T where δ > δI and solutions to P2I,T ,λ where δ > δI,λ .

Hard Problems in P1 and P2

We believe that P1 and P2 contain several hard problems. For example, the
captcha shown in Section 1 (and other captchas based on the difficulty of
reading slightly distorted text) could be defeated using solutions to P2. To see
this, let W be a set of images of words in different fonts. All the images in W
should be undistorted and contain exactly one word each. Let IW be a distri-
bution on W , let TW be a distribution on image transformations, and let λW
map an image to the word that is contained in it. A solution to P2IW ,TW ,λW is a
program that can defeat a captcha such as the one that Yahoo! uses (assuming
TW is the same set of transformations they use). So the problem of determining
the word in a distorted image is an instantiation of P2 (it can be easily seen
to be an instantiation of P1 too). Reading slightly distorted text has been an
open problem in machine vision for quite some time. (For a good overview of
the difficulties of reading slightly distorted text, see [12].)
But P1 and P2 are much more general, and reading slightly distorted text
is a somewhat easy instance of these problems. In general it will not be the case
that the problem is reduced to matching 26 ∗ 2 + 10 different characters (upper
and lowercase letters plus the digits).
The hardness of problems in P1 and P2 mostly relies on T . In particular, it
should be computationally infeasible to enumerate all of the elements of [T ], since
I will normally be such that enumeration of [I] is feasible. Thus we are mainly
interested in (δ, τ ) solutions where τ  |[T ]|, while τ > |[I]| may sometimes be
acceptable. In addition to the size of the transformation set, the character of the
transformations is also important: it is necessary to defeat many simple checks
such as color histogram comparisons, frequency domain checks, etc.
Since instantiations of P1 and P2 have never been precisely stated and pub-
lished as challenges to the AI and security communities, there is no way to
tell if they will withstand the test of time. For now we refer the reader to
www.captcha.net for examples of I’s and T ’s which are believed to be good
candidates. Any instantiation of P1 and P2 for security purposes requires that
the precise I and T be published and thoroughly described.
4 Two Families of captchas
We now describe two families of captchas whose security is based on the hard-
ness of problems in P1 and P2. Notice that if P1I,T is (δ, τ )-hard then P1I,T
can be used to construct a captcha trivially: the verifier simply gives the prover
t(i) and asks the prover to output i. According to our definition, this would be
a perfectly valid captcha. However, it would also be a very impractical one:
if [I] is large, then humans would take a long time to answer. The captchas
we present in this section can be quickly answered by humans. The first family
of captchas, matcha, is somewhat impractical, but the second family, pix, is
very practical and in fact several instantiations of it are already in use.

4.1 MATCHA
A matcha instance is described by a triple M = (I, T , τ ), where I is a distri-
bution on images and T is a distribution on image transformations that can be
easily computed using current computer programs. matcha is a captcha with
the following property: any program that has high success over M = (I, T ) can
be used to solve P1I,T .
The matcha verifier starts by choosing a transformation t ← T . It then flips
a fair unbiased coin. If the result is heads, it picks k ← I and sets (i, j) = (k, k).
If the result is tails, it sets j ← I and i ← U ([I]−{j}) where U (S) is the uniform
distribution on the set S. The matcha verifier sends the prover (i, t(j)) and sets
a timer to expire in time τ ; the prover responds with res ∈ {0, 1}. Informally,
res = 1 means that i = j, while res = 0 means that i 6= j. If the verifier’s timer
expires before the prover responds, the verifier rejects. Otherwise, the verifier
makes a decision based on the prover’s response res and whether i is equal to j:
– If i=j and res = 1, then matcha accepts.
– If i=j and res = 0, then matcha rejects.
– If i 6= j and res = 1, then matcha rejects.
– If i 6= j and res = 0, then matcha plays another round.
In the last case, matcha starts over (with a fresh new set of random coins): it
flips another fair unbiased coin, picks another pair of images (i, j) depending on
the outcome of the coin, etc.

Remarks

1. It is quite easy to write a computer program that has success probability

1/2 over matcha by simply answering with res = 1. For most applications,
a distinguishing probability of 1/2 is unacceptable. In order for matcha to
be of practical use, the test has to be repeated several times.
2. Our description of matcha contains an obvious asymmetry: when i = j,
matcha presents the prover with (i, t(i)) for i ← I, and when i 6= j matcha
presents (i, t(j)), where i is chosen uniformly from the set [I] − {j}. This
gives the prover a simple strategy to gain advantage over M : if i seems to
 come from I, guess that t(j) is a transformation of i; otherwise guess that it
isn’t. The reason for the asymmetry is to make the proof of Lemma 1 easier
to follow. We note that a stronger captcha can be built by choosing i from
the distribution I restricted to the set [I] − {j}.
3. The intuition for why matcha plays another round when i 6= j and res = 0 is
that we are trying to convert high success against matcha into high success
in solving P1; a program that solves P1 by comparing t(j) to every image
in [I] will encounter that most of the images in [I] are different from t(j).
4. In the following Lemma we assume that a program with high success over
M always terminates with a response in time at most τ . Any program which
does not satisfy this requirement can be rewritten into one which does, by
stopping after τ time and sending the response 1, which never decreases the
success probability. We also assume that the unit of time that matcha uses
is the same as one computational step.

Lemma 1. Any program that has success greater than η over M = (I, T , τ ) can
be used to (δ, τ |[I]|)-solve P1I,T , where

η
δ≥ .
1 + 2|[I]|(1 − η)

Proof. Let B be a program that runs in time at most τ and has success σB ≥ η
over M . Using B we construct a program AB that is a (δ, τ |[I]|)-solution to
P1I,T .
The input to AB will be an image, and the output will be another image.
On input j, AB will loop over the entire database of images of M (i.e., the set
[I]), each time feeding B the pair of images (i, j), where i ∈ [I]. Afterwards,
AB collects all the images i ∈ [I] on which B returned 1 (i.e., all the images in
[I] that B thinks j is a transformation of). Call the set of these images S. If S
is empty, then AB returns an element chosen uniformly from [I]. Otherwise, it
picks an element ` of S uniformly at random.
We show that AB is a (δ, τ |[I]|)-solution to P1. Let p0 = PrT ,I,r [Br (i, t(i)) =
0] and let p1 = PrT ,j←I,i,r [Br (i, t(j)) = 1]. Note that

p0 p1 + (1 − p1 )(1 − σB ) p0 + p 1
Pr [hMr , Br0 i = reject] = 1 − σB = + = ,
r,r 0 2 2 1 + p1

which gives σB ≤ 1 − p0 and p1 ≤ 2(1 − σB ). Hence:

 n
X
Pr [AB
r (t(j)) = j] ≥ Pr [AB
r (t(j)) = j||S| = s] Pr [|S| = s] (1)
T ,I,r T ,I,r T ,I
s=1
n
X 1 − p0
= Pr [|S| = s] (2)
s=1
s T ,I
1 − p0
≥ (3)
ET ,I [|S|]
1 − p0
≥ (4)
1 + |[I]|p1
σB
≥ (5)
1 + 2|[I]|(1 − σB )

(2) follows by the definition of the procedure AB , (3) follows by Jensen’s in-
equality and the fact that f (x) = 1/x is concave, (4) follows because
X
ET ,I [|S|] ≤ 1 + Pr(B(i, t(j)) = 1)
I,r
i6=j

and (5) follows by the inequalities for p0 , p1 and σB given above. This completes
the proof.

Theorem 1. If P1I,T is (δ, τ |[I]|)-hard and M = (I, T , τ ) is (α, β)-human

executable, then M is a (α, β, (2−|[I]|)δ
2δ−|[I]| )-captcha.

4.2 PIX
An instance P2I,T ,λ can sometimes be used almost directly as a captcha. For
instance, if I is a distribution over images containing a single word and λ maps
an image to the word contained in it, then P2I,T ,λ can be used directly as a
captcha. Similarly, if all the images in [I] are pictures of simple concrete objects
and λ maps an image to the object that is contained in the image, then P2I,T ,λ
can be used as a captcha.
Formally, a pix instance is a tuple X = (I, T , L, λ, τ ). The pix verifier works
as follows. First, V draws i ← I, and t ← T . V then sends to P the message
(t(i), L), and sets a timer for τ . P responds with a label l ∈ L. V accepts if
l = λ(i) and its timer has not expired, and rejects otherwise.
Theorem 2. If P2I,T ,λ is (δ, τ )-hard and X = (I, T , L, λ, τ ) is (α, β)-human
executable, then X is a (α, β, δ)-captcha.
Various instantiations of pix are in use at major internet portals, like Yahoo!
and Hotmail. Other less conventional ones, like Animal-PIX, can be found at
www.captcha.net. Animal-PIX presents the prover with a distorted picture of a
common animal (like the one shown below) and asks it to choose between twenty
different possibilities (monkey, horse, cow, et cetera).
 Fig. 2. Animal-Pix.

5 An Application: Robust Image-Based Steganography

We detail a useful application of (δ, τ )-solutions to instantiations of P1 and

P2 (other than reading slightly distorted text, which was mentioned before).
We hope to convey by this application that our problems were not chosen just
because they can create captchas but because they in fact have applications
related to security. Our problems also serve to illustrate that there is a need
for better AI in security as well. Areas such a Digital Rights Management, for
instance, could benefit from better AI: a program that can find slightly distorted
versions of original songs or images on the world wide web would be a very useful
tool for copyright owners.
There are many applications of solutions to P1 and P2 that we don’t mention
here. P1, for instance, is interesting in its own right and a solution for the
instantiation when I is a distribution on images of works of art would benefit
museum curators, who often have to answer questions such as “what painting is
this a photograph of?”

Robust Image-Based Steganography.

Robust Steganography is concerned with the problem of covertly communicating

messages on a public channel which is subject to modification by a restricted
adversary. For example, Alice may have some distribution on images which she
is allowed to draw from and send to Bob; she may wish to communicate addi-
tional information with these pictures, in such a way that anyone observing her
communications can not detect this additional information. The situation may
be complicated by an adversary who transforms all transmitted images in an
effort to remove any hidden information. In this section we will show how to
use (δ, τ )-solutions to instantiations of P1I,T or P2I,T ,λ to implement a secure
robust steganographic protocol for image channels with distribution I, when
the adversary chooses transformations from T . Note that if we require security
for arbitrary I, T , we will require a (δ, τ )-solution to P1 for arbitrary I, T ; if
no solution works for arbitrary (I, T ) this implies the existence of specific I, T
for which P1 is still hard. Thus either our stegosystem can be implemented by
computers for arbitrary image channels or their is a (non-constructive) hard AI
problem that can be used to construct a captcha.
The results of this subsection can be seen as providing an implementation of
the “supraliminal channel” postulated by Craver [6]. Indeed, Craver’s observa-
tion that the adversary’s transformations should be restricted to those which do
not significantly impact human interpretation of the images (because the adver-
sary should not unduly burden “innocent” correspondents) is what leads to the
applicability of our hard AI problems.

Steganography Definitions
Fix a distribution over images I, and a set of keys K. A steganographic protocol
or stegosystem for I is a pair of efficient probabilistic algorithms (SE, SD) where
SE : K × {0, 1} → [I]` and SD : K × [I]` → {0, 1}, which have the additional
property that PrK,r,r0 [SDr (K, SEr0 (K, σ)) 6= σ] is negligible (in ` and |K|) for
any σ ∈ {0, 1}. We will describe a protocol for transmitting a single bit σ ∈
{0, 1}, but it is straightforward to extend our protocol and proofs by serial
composition to any message in {0, 1}∗ with at most linear decrease in security.
Definition 6. A stegosystem is steganographically secret for I if the distribu-
tions {SEr (K, σ) : K ← K, r ← {0, 1}∗} and I ` are computationally indistin-
guishable for any σ ∈ {0, 1}.
Steganographic secrecy ensures that an eavesdropper cannot distinguish traf-
fic produced by SE from I. Alice, however, is worried about a somewhat mali-
cious adversary who transforms the images she transmits to Bob. This adversary
is restricted by the fact that he must transform the images transmitted between
many pairs of correspondents, and may not transform them in ways so that they
are unrecognizable to humans, since he may not disrupt the communications
of legitimate correspondents. Thus the adversary’s actions, on seeing the image
i, are restricted to selecting some transformation t according to a distribution
T , and replacing i by t(i). Denote by t1...` ← T ` the action of independently
selecting ` transformations according to T , and denote by t1...` (i1...` ) the action
of element-wise applying ` transformations to ` images.
Definition 7. A stegosystem (SE, SD) is steganographically robust against T
if it is steganographically secret and
Pr [SDr (K, t1...` (SEr (K, σ))) 6= σ]
t1...` ←T ` ,r,r 0 ,K

is negligible (in |K| and `) for any σ ∈ {0, 1}.

Let F : K × {1, . . . , `} × L → {0, 1} be a pseudorandom function family.
We assume that λ : [I] → L is efficiently computable and A : SP2 → L is a
(δ, τ )-solution to P2I,T ,λ as defined above (recall that P1 is a special case of
P2), i.e. A operates in time τ and
Pr [Ar (t(i)) = λ(i)] ≥ δ .
t,i,r
Let c = Pri,j←I [λ(i) = λ(j)]. We require that c < 1 (that is, we require that there
is enough variability in the labels of the images to be useful for communication).
Notice that L can simply be equal to [I] and λ can be the identity function (in
case the images in [I] have no labels as in P1). We prove in the Appendix that
the following construction is an efficient, robust stegosystem for T .
Construction 1
Procedure SE: Procedure SD:
Input: K ∈ K, σ ∈ {0, 1} Input: K ∈ K, i01...` ∈ [I]`
for j = 1 . . . ` do for j = 1 . . . l do
draw d0 ← I, d1 ← I set σj = FK (j, A(i0j ))
if FK (j, λ(d0 )) = σ then Output: majority(σ1 , . . . , σ` )
set ij = d0
else
set ij = d1
Output: i1 , i2 , . . . , i`
Proposition 1. Construction 1 is steganographically secret and robust for I, T .
The proof of Proposition 1 is similar in flavor to those of Hopper, Langford
and von Ahn [7] and relies on the fact that when A returns the correct solution
on received image i0j , the recovered bit σj is equal to the intended bit σ with
probability approximately 12 + 41 (1 − c) and otherwise σj = σ with probability
1/2; therefore the probability that the majority of the σj are incorrect is negli-
gible in `. For details, see the Appendix.

Remarks
1. Better solutions to P2I,T ,λ imply more efficient stegosystems: if δ is larger,
then ` can be smaller and less images need to be transmitted to send a bit
secretively and robustly.
2. Since we assume that P2I,T ,λ (or, as it might be the case, P1I,T ) is easy for
humans, our protocol could be implemented as a cooperative effort between
the human recipient and the decoding procedure (without the need for a
solution to P1I,T or P2I,T ,λ ). However, decoding each bit of the secret
message will require classifying many images, so that a human would likely
fail to complete the decoding well before any sizeable hidden message could
be extracted (this is especially true in case we are dealing with P1I,T and
a large set [I]: a human would have to search the entire set [I] as many as
` times for each transmitted bit). Thus to be practical, a (δ, τ )-solution (for
small τ ) to P1I,T or P2I,T ,λ will be required.

6 Discussion and Closing Remarks

Interaction with the AI community
A primary goal of the captcha project is to serve as a challenge to the Artificial
Intelligence community. We believe that having a well-specified set of goals will
contribute greatly to the advancement of the field. A good example of this process
is the recent progress in reading distorted text images driven by the captcha in
use at Yahoo!. In response to the challenge provided by this test, Malik and Mori
[9] have developed a program which can pass the test with probability roughly
0.8. Despite the fact that this captcha has no formal proof that a program
which can pass it can read under other distributions of image transformations,
Malik and Mori claim that their algorithm represents significant progress in
the general area of text recognition; it is encouraging to see such progress. For
this reason, it is important that even Automated Turing Tests without formal
reductions attempt to test ability in general problem domains; and even though
these tests may have specific weaknesses it is also important that AI researchers
attempting to pass them strive for solutions that generalize.

Other AI problem domains

The problems defined in this paper are both of a similar character, and deal with
the advantage of humans in sensory processing. It is an open question whether
captchas in other areas can be constructed. The construction of a captcha
based on a text domain such as text understanding or generation is an important
goal for the project (as captchas based on sensory abilities can’t be used on
sensory-impaired human beings). As mentioned earlier, the main obstacle to
designing these tests seems to be the similar levels of program ability in text
generation and understanding.
Logic problems have also been suggested as a basis for captchas and these
present similar difficulties, as generation seems to be difficult. One possible source
of logic problems are those proposed by Bongard [4] in the 70s; indeed [1] presents
a test based on this problem set. However, recent progress in AI has also yielded
programs which solve these problems with very high success probability, exceed-
ing that of humans.

Conclusion
We believe that the fields of cryptography and artificial intelligence have much to
contribute to one another. captchas represent a small example of this possible
symbiosis. Reductions, as they are used in cryptography, can be extremely useful
for the progress of algorithmic development. We encourage security researchers
to create captchas based on different AI problems.

Acknowledgments
We are greatful to Udi Manber for his suggestions. We also thank Lenore Blum,
Roni Rosenfeld, Brighten Godfrey, Moni Naor, Henry Baird and the anonymous
Eurocrypt reviewers for helpful discussions and comments. This work was par-
tially supported by the National Science Foundation (NSF) grants CCR-0122581
and CCR-0085982 (The Aladdin Center). Nick Hopper is also partially supported
by an NSF graduate research fellowship.
References

1. Luis von Ahn, Manuel Blum, Nicholas J. Hopper and John Langford. The
CAPTCHA Web Page: http://www.captcha.net. 2000.
2. Luis von Ahn, Manuel Blum and John Langford. Telling Humans and Computers
Apart (Automatically) or How Lazy Cryptographers do AI. To appear in Commu-
nications of the ACM.
3. Mihir Bellare, Russell Impagliazzo and Moni Naor. Does Parallel Repetition Lower
the Error in Computationally Sound Protocols? In 38th IEEE Symposium on Foun-
dations of Computer Science (FOCS’ 97), pages 374-383. IEEE Computer Society,
1997.
4. Mikhail M. Bongard. Pattern Recognition. Spartan Books, Rochelle Park NJ, 1970.
5. A. L. Coates, H. S. Baird, and R. J. Fateman. Pessimal Print: A Reverse Turing
Test. In Proceedings of the International Conference on Document Analysis and
Recognition (ICDAR’ 01), pages 1154-1159. Seattle WA, 2001.
6. Scott Craver. On Public-key Steganography in the Presence of an Active Warden.
In Proceedings of the Second International Information Hiding Workshop, pages
355-368. Springer, 1998.
7. Nicholas J. Hopper, John Langford and Luis von Ahn. Provably Secure Steganog-
raphy. In Advances in Cryptology, CRYPTO’ 02, volume 2442 of Lecture Notes in
Computer Science, pages 77-92. Santa Barbara, CA, 2002.
8. M. D. Lillibridge, M. Abadi, K. Bharat, and A. Broder. Method for selectively
restricting access to computer systems. US Patent 6,195,698. Applied April 1998
and Approved February 2001.
9. Greg Mori and Jitendra Malik. Breaking a Visual CAPTCHA.
Unpublished Manuscript, 2002. Available electronically:
http://www.cs.berkeley.edu/~mori/gimpy/gimpy.pdf.
10. Moni Naor. Verification of a human in the loop or Identification via
the Turing Test. Unpublished Manuscript, 1997. Available electronically:
http://www.wisdom.weizmann.ac.il/~naor/PAPERS/human.ps.
11. Benny Pinkas and Tomas Sander. Securing Passwords Against Dictionary Attacks.
In Proceedings of the ACM Computer and Security Conference (CCS’ 02), pages
161-170. ACM Press, November 2002.
12. S. Rice, G. Nagy, and T. Nartker. Optical Character Recognition: An Illustrated
Guide to the Frontier. Kluwer Academic Publishers, Boston, 1999.
13. Adi Shamir and Eran Tromer. Factoring Large Numbers with the
TWIRL Device. Unpublished Manuscript, 2003. Available electronically:
http://www.cryptome.org/twirl.ps.gz.
14. J. Xu, R. Lipton and I. Essa. Hello, are you human. Technical Report GIT-CC-
00-28, Georgia Institute of Technology, November 2000.

A Proof of Proposition 1

Lemma 1. Construction 1 is steganographically secret.

Proof. Consider for any 1 ≤ j ≤ ` and x ∈ [I] the probability ρjx that ij = x,
i.e. ρjx = Pr[ij = x]. The image x is returned in the jth step only under one of
the following conditions:
 1. D0 : d0 = x and FK (j, λ(d0 )) = σ ; or
2. D1 : d1 = x and FK (j, λ(d0 )) = 1 − σ
Note that these events are mutually exclusive, so that ρjx = Pr[D0 ] + Pr[D1 ].
Suppose that we replace FK by a random function f : {1, . . . , `} × L → {0, 1}.
Then we have that Prf,d0 [D0 ] = 21 PrI [x] by independence of f and d0 , and
Prf,d1 [D1 ] = 12 PrI [x], by the same reasoning. Thus ρjx = PrI [x] when FK
is replaced by a random function. Further, for a random function the ρjx are
all independent. Thus for any σ ∈ {0, 1} we see that SE(K, σ) and I ` are
computationally indistinguishable, by the pseudorandomness of F .

Lemma 2. Construction 1 is steganographically robust for T .

Proof. Suppose we prove a constant bound ρ > 21 such that for all j, Pr[σj = σ] >
ρ. Then by a Chernoff bound we will have that Pr[majority(σ1 , . . . , σ` ) 6= σ] is
negligible, proving the lemma.
Consider replacing FK for a random K with a randomly chosen function
f : {1, . . . , `} × L → {0, 1} in SE, SD. We would like to assess the probability
ρj = Pr[σj = σ]. Let Aj be the event A(i0j ) = λ(ij ) and Aj be the event
A(i0j ) 6= λ(ij ), and write λ(djb ) as lbj . We have the following:

Pr [σj = σ] = Pr[σj = σ|Aj ] Pr[Aj ] + Pr[σj = σ|Aj ] Pr[Aj ] (6)

f,ij ,t
1
= δ Pr[σj = σ|Aj ] + (1 − δ) (7)
2
1−δ
= δ(Pr[f (j, l0j ) = σ or (f (j, l0j ) = 1 − σ and f (j, l1j )) = σ)] +
2
(8)
1 1−δ
= δ( + Pr[f (j, l0j ) = 1 − σ and f (j, l1j ) = σ and l0j =
6 l1j ]) +
2 2
(9)
1 1 1−δ
= δ( + (1 − c)) + (10)
2 4 2
1 δ
= + (1 − c) (11)
2 4
Here (7) follows because if A(i0j ) = l 6= λ(ij ) then Prf [f (j, l) = σ] = 21 , and
(8) follows because if A(i0j ) = λ(ij ), then f (j, A(i0j )) = σ iff f (j, λ(ij )) = 0; the
expression results from expanding the event f (j, λ(ij )) = 0 with the definition
of ij in the encoding routine.
For any constant δ > 0, a Chernoff bound implies that the probability of
decoding failure is negligible in ` when FK is a random function. Thus the
pseudorandomness of FK implies that the probability of decoding failure for
Construction 1 is also negligible, proving the lemma.