CHAPTER 14

A/B Testing on
PythonAnywhere
and MySQL
This is an ambitious chapter, so we’ll limit the scope in order to distill the essence of this
rich topic without going overboard. We’ll start by building a simple MySQL database
and table to track whether a visitor liked or didn’t like the art work on the landing page.
Because this is A/B Testing, we’re going to create two landing pages and switch them
randomly when users visit the site (Figure 14-1).

Figure 14-1. The final web application for this chapter

401
© Manuel Amunategui, Mehdi Roopaei 2018
M. Amunategui and M. Roopaei, Monetizing Machine Learning, https://doi.org/10.1007/978-1-4842-3873-8_14
Chapter 14 A/B Testing on PythonAnywhere and MySQL

Note Download the files for Chapter 14 by going to www.apress.com/

9781484238721 and clicking the source code button. You will need to install
MySQL on your local machine in order to follow along with the Jupyter notebook
“chapter14.ipynb.”

In analytics, A/B testing means using two different versions of something and
measuring how people react to each. This is commonly used in websites to try out
new designs, products, sales, etc. In our case, we’re going to expose our visitors to two
different versions of our landing page. The versions are going to be assigned randomly
and the visitor will be offered the opportunity to give the page a thumbs-up if they liked
it. In the background, we’re going to be tracking this traffic and whether or not a user
gives a thumbs-up. If the user doesn’t give the thumbs-up, we’ll assume that it was a
down vote.
This is an important topic and can yield valuable knowledge about your business
and your users. There is a famous anecdote where Marisa Meyer, while at Google, ran an
A/B test to determine which shade of blue, out of 40, the users preferred.1 Obviously, one
can go overboard with these types of tests.

A/B Testing
The goal of A/B testing is to expose different products to the public and measure their
reactions. In our web application, we’re going to show a web page with two different
images: an angry face and a friendly one. We will add a simple label to the page asking
the visitor to give the image a thumbs-up if they liked it. In the background we’re going
to count each visit and count each thumbs-up. To keep things simple, we’ll count an
initial visit as a thumbs-down and update it to a thumbs-up if the visitor clicks the voting
button. See Figures 14-2 and 14-3 for a version of each image.

1
https://iterativepath.wordpress.com/2012/10/29/testing-40-shades-of-blue-ab-testing/

402
 Chapter 14 A/B Testing on PythonAnywhere and MySQL

Figure 14-2. Image one

Figure 14-3. Image two

403
Chapter 14 A/B Testing on PythonAnywhere and MySQL

Tracking Users
There are various ways of tracking anonymous visitors. Popular methods include the
usage of cookies and databases. Each has its advantage and purpose. A cookie offers the
advantage of tracking a user over longer periods regardless of whether they closed their
browser or turned their computer off. A web page can easily check the visitor’s computer
for previous cookies and compare it with their database to determine if this is a repeat
visitor or not.
We won’t need to use cookies, as we will only consider single visits. Instead, we’ll
keep track of users using an HTML hidden tag and send that tag back using a post
request. When a visitor first visits the page, we’ll insert a row in the database with the
page background image, a timestamp, and a unique identifier (a very long string that is
unique to that user; the odds of creating two of the same are infinitesimal) referred to as a
UUID. As mentioned, we assume that a first page visit is a thumbs-down and write it to the
database. As we build the page, we insert a hidden HTML tag containing the UUID so that
if the user interacts with the page by clicking the thumbs-up button, we’ll pass the UUID
back to the web server, so we can update the row previously entered in the database. This
approach allows us to serve many visitors at the same time without worrying about who
has what page. There are many ways you can tweak and improve this process depending
on your needs. You can even pass that UUID from client to server and back as many times
as you want and always know which session and user you are dealing with.

UUID
The Universally Unique Identifier (UUID) is 128 bits long strong, and is guaranteed
to be unique. We’ll use the handy “uuid” Python library to automatically generate a
guaranteed unique identifier (Listing 14-1).

Listing 14-1. The “uuid” Kibrary

Input:

import uuid
str(uuid.uuid4())

Output:

'e7b1b80e-1eca-43a7-90a3-f01927ace7c9'

404
 Chapter 14 A/B Testing on PythonAnywhere and MySQL

In the “uuid” library, the “uuid4()” function generates a new random ID without
relying on your computer’s identifier, so it is unique and private. Check out the docs
for additional UUID details and options at https://docs.python.org/3/library/
uuid.html.

MySQL
We’re going to use the MySQL Community Server, which is a great and popular free
database. It is perfect to support our A/B testing needs. It is an open-source relational
database that can support a wide range of needs and is being used by big players
including WordPress, and large media companies like Google and Facebook.
Go ahead and download the version of MySQL Community Server for your OS at
https://dev.mysql.com/downloads. You will also find the installation instruction for
your OS if you have any questions or issues. We won’t use any front end, though there are
quite a few of them available in case you want to use one (Figure 14-4).

Figure 14-4. Find and download the correct version for your operating system

405
Chapter 14 A/B Testing on PythonAnywhere and MySQL

You will be prompted with a series of questions including setting up root password
and password encryption type (Figure 14-5).

Figure 14-5. Keeping it simple and using the legacy password system

You can also start and stop your database through the control center for your
operating system (this can also be done through the command line; Figure 14-6).

406
 Chapter 14 A/B Testing on PythonAnywhere and MySQL

Figure 14-6. Setting MySQL server to start automatically when the computer
starts

Command Line Controls

To start and stop MySQL (in most cases it should start automatically after your install
it and restart your machine). Check out the docs for other operating systems, changes
since this book was published, and additional commands at https://dev.mysql.com/doc/.
Let’s see how to start MySQL (Listings 14-2 and 14-3, and Figure 14-7).

Listing 14-2. Starting MySQL on the Mac

$ sudo /usr/local/mysql/support-files/mysql.server start

Listing 14-3. Starting MySQL on Windows

C:\> "C:\Program Files\MySQL\MySQL Server 8.0\bin\mysqld"

407
Chapter 14 A/B Testing on PythonAnywhere and MySQL

Figure 14-7. Starting MySQL using the command line

Let’s see how to stop MySQL (Listings 14-4 and 14-5).

Listing 14-4. Stopping MySQL on the Mac

$ sudo /usr/local/mysql/support-files/mysql.server stop

Listing 14-5. Stopping MySQL on Windows

C:\> "C:\Program Files\MySQL\MySQL Server 8.0\bin\mysqladmin" -u root

shutdown

MySQL Command Line Monitor

The command-line monitor is a handy tool that allows you to manage users and
permissions, create databases and tables, and much more (see the docs for other
operating systems and additional commands at https://dev.mysql.com/doc/).
To enter the monitor, change the drive to your MySQL directory or export a path,
then enter the “mysql -u root -p” command and you will be prompted for your
password that you created during the installation process (Listing 14-­6 and Figure 14-8).

Listing 14-6. Code Input

$ export PATH=$PATH:/usr/local/mysql/bin
$ mysql -u root -p

408
 Chapter 14 A/B Testing on PythonAnywhere and MySQL

Figure 14-8. Running the MySQL Command Line Monitor

You will know that you entered the monitor once your prompt changes to “mysql>.”
Let’s create a user, a database, and a table for our A/B testing.

Creating a Database
Let’s create a database named “ABTesting” (Listing 14-7).

Listing 14-7. Creating a Database

mysql> CREATE DATABASE ABTesting;

Creating a Table
Let’s create a new table using the “CREATE TABLE” statement. Whenever you are
creating a new table, it is a good idea to drop it first, otherwise you will get an error
(but make sure that you really do want to drop it as you will lose all data contained
therein). We will create a table called “tblFrontPageOptions” that will have a unique
identifier field called “uuid,” a Boolean flag called “liked” to hold whether or not the user
clicked the thumbs up, a page_id to mark whether this was an “A” or “B” page, and an
automated timestamp field (Listing 14-8).

409
Chapter 14 A/B Testing on PythonAnywhere and MySQL

Listing 14-8. Creating a Table

mysql> DROP TABLE ABTesting.tblFrontPageOptions;

mysql> CREATE TABLE ABTesting.tblFrontPageOptions (
      uuid VARCHAR(40) NOT NULL,
   liked BOOLEAN NOT NULL DEFAULT 0,
      pageid INT NOT NULL,
      time_stamp  TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP);

You can easily test that your table is working by inserting some data into it using an
“INSERT INTO” statement (Listing 14-9).

Listing 14-9. Inserting Data

mysql> INSERT INTO ABTesting.tblFrontPageOptions (uuid, liked, pageid)

VALUES(9999, 1, 2);

To check that the data did indeed make it into the table, we use a “SELECT *”
statement (Listing 14-10).

Listing 14-10. Querying Data

Input:

mysql> SELECT * FROM ABTesting.tblFrontPageOptions;

Output:

+------+-------+---------+---------------------+
| uuid | liked | page_id | time_stamp          |
+------+-------+---------+---------------------+
| 9999 |     1 |      2 | 2018-05-19 14:28:44 |
+------+-------+---------+---------------------+
1 row in set (0.00 sec)

We’re looking good; the table now has a new row in it. If you want to start with a
clean state, you can drop and re-create the table with the previous code. There are plenty
of great primers on SQL syntax on the Internet, but a great place to start is the w3schools
at https://www.w3schools.com/sql. Exit out of the “mysql>” prompt and open up the
Jupyter notebook for the chapter to practice inserting data into our table and reading it
out through the “mysql.connector” Python library.

410
 Chapter 14 A/B Testing on PythonAnywhere and MySQL

Creating A Database User

We are going to create a user dedicated to our A/B testing web application. It is a bad
idea to use your root password in your Flask code. Our user will be called “webuser” and
its password will be “thesecre” (Listing 14-11).

Listing 14-11. Creating a user

mysql> CREATE USER 'webuser'@'localhost' IDENTIFIED BY 'thesecret';

Next, we will grant this user all privileges, and once you are more comfortable with
MySQL (no, not my SQL, the MySQL product… you know they’re probably joking like
that all day long over at the MySQL headquarters…), you can tone this down to just read/
write permissions for specific tables). See Listing 14-12.

Listing 14-12. Granting Rights

mysql> GRANT ALL PRIVILEGES ON ABTesting.* TO 'webuser'@'localhost' WITH

GRANT OPTION;

Finally, you can check that the “webuser” user was successfully added with the
following handy command (Listing 14-13).

Listing 14-13. Checking Users

mysql> SELECT User FROM mysql.user;

+------------------+
| User             |
+------------------+
| mysql.infoschema |
| mysql.session    |
| mysql.sys        |
| root             |
| webuser          |
+------------------+
5 rows in set (0.00 sec)

You can exit out of the MySQL command line tool by simply entering the “exit”
command.

411
Chapter 14 A/B Testing on PythonAnywhere and MySQL

Python Library: mysql.connector

Most of the interaction with our database will be done through Python and Flask using
the handy “mysql.connector” library. Here you can refer to the corresponding Jupyter
notebook for Chapter 14 to follow along (please install the “requirements_jupyter.txt”
file to get the libraries needed). Keep in mind that in this chapter we will not create a
local Flask version, so get familiar with the commands using the notebook, then we’ll
jump directly to the cloud.
We will program three types of functions using the “SELECT,” “INSERT,” and
“UPDATE” SQL functions. If you are not familiar with these classic SQL functions,
check out the great primer from w3schools (I know, I keep pushing that site; it’s that
good and I swear that I have no relations with them whatsoever) at https://www.
w3schools.com/sql.

SELECT SQL Statement

“SELECT” is the most common SQL command and is used to read data from a table.
This is easily done using the “mysql.connector” library. You first create a connection to
the database by calling the “connect()” function and passing permissioned credentials
for a database. The connection returns a cursor to communicate and send orders to
the database. This is done using a query string holding the “SELECT” statement. We
will use this approach for all our SQL statements. The difference with this statement
versus “INSERT” and “UPDATE” is that we are expecting to receive data back from the
database. After executing our query string through the “execute()” function, we can
access the returned data through a loop. Each loop represents one row of data. Notice
that an open cursor and connection are both closed at the end of the call, as we don’t
want to hold onto resources longer than we need to (Listing 14-14).

Listing 14-14. “select” Statement with Cursor

Input:

cnx = mysql.connector.connect(user='webuser', password='thesecret',

database='ABTesting')
cursor = cnx.cursor()
query = "SELECT * FROM ABTesting.tblFrontPageOptions"
cursor.execute(query)

412
 Chapter 14 A/B Testing on PythonAnywhere and MySQL

for (uuid, liked, pageid, time_stamp) in cursor:

        
print("uuid: {} liked:{} pageid: {} on {:%m/%d/%Y %H:%M:%S}".format(
        uuid, liked, pageid, time_stamp))
cursor.close()
cnx.close()

Output (example, your output will only contain what's in the table so far):

uuid: 704a44d0-29f4-4a2d-bc6a-fe679017f7e9 liked:1 pageid: 2 on 05/19/2018

17:03:29

INSERT SQL Statement

The “INSERT” statement allows us to insert data into the ABTesting table of our
database. The first time a visitor hits the web page, we insert a row containing the
following fields, a “UUID,” a “liked” flag turned to false, the “pageid” representing which
of the two background images the user is viewing, and a “timestamp.” As done in the
“SELECT” statement, we first open a connection to the database, then create a cursor
and pass it our query statement. In this case, our query statement isn’t a “SELECT” but
an “INSERT.” We also use the handy “%s” statement as a variable placeholder to then be
filled by whatever value is held in the “args” tuple. Here we are inserting a new unique
ID, with “liked” set to false (or thumbs down), and the page ID viewed.
Also, whenever you are inserting or updating a table, don’t forget to call the
“commit()” function to commit your inserts before closing the connection. If you don’t
commit, your changes will get ignored (Listing 14-15).

Listing 14-15. SQL “INSERT” Statement with Cursor

cnx = mysql.connector.connect(user='webuser', password='thesecret',

database='ABTesting')
cursor = cnx.cursor()
query = "INSERT INTO ABTesting.tblFrontPageOptions (uuid, liked, pageid)
VALUES (%s, %s, %s);"
args = ("704a44d0-29f4-4a2d-bc6a-fe679017f7e9", 0, 1)
cursor.execute(query, args)
cursor.close()
cnx.commit()
cnx.close()
413
Chapter 14 A/B Testing on PythonAnywhere and MySQL

UPDATE SQL Statement

The “UPDATE” statement is similar to the “INSERT” statement, but instead of adding
a new row at the end of the table, you are updating an existing row. In order to update a
specific row, you have to be able to find the correct row before updating it. In this case,
as we have the handy “UUID” that is guaranteed unique, we can easily find that specific
row and not have to worry about updating another in error. In order to properly build
the “UPDATE” statement, we have to pass it two values, the “UUID” of “704a44d0-29f4-
4a2d-­bc6a-fe679017f7e9” and the “liked” flag set to true (Listing 14-16).

Listing 14-16. SQL “update” Statement with Cursor

cnx = mysql.connector.connect(user='webuser', password='thesecret',

database='ABTesting')
cursor = cnx.cursor()
query = "UPDATE ABTesting.tblFrontPageOptions SET liked = %s WHERE uuid = %s;"
args = (1, "704a44d0-29f4-4a2d-bc6a-fe679017f7e9")
cursor.execute(query, args)
cursor.close()
cnx.commit()
cnx.close()

Again, don’t forget to call the “commit()” function to commit your changes before
closing the connection (if you don’t, your changes will get ignored).

Abstracting the Code into Handy Functions

We need to abstract all our SQL code into simple to use functions. We start by creating
two global variables to hold the MySQL user account and password. This enables us to
only have to set it once and not worry about it during subsequent SQL calls. It also comes
in handy whenever you need to change user accounts (Listing 14-17).

Listing 14-17. Abstracting Account Data

mysql_account = 'webuser'
mysql_password = 'thesecret'
mysql_database = 'ABTesting'
mysql_host = 'localhost'

414
 Chapter 14 A/B Testing on PythonAnywhere and MySQL

We also can abstract the Uuid-generating code to keep things clean and simple
(Listing 14-18).

Listing 14-18. Abstracting “GetUUID()” function

def GetUUID():
    return (str(uuid.uuid4()))

Next, we create a function to insert new visits into the database. This function will
get a new UUID from “GetUUID(),” set the “liked” to false as we assume all new visits
don’t like or don’t want to interact with the site, the “pageid” representing the image that
was randomly selected for them, and the timestamp that is automatically generated by
MySQL (Listing 14-19).

Listing 14-19. Abstracting “InsertInitialVisit()” Function

def InsertInitialVisit(uuid_, pageid):

    try:
        cnx = mysql.connector.connect(user=mysql_account, password=mysql_
password, database=mysql_database, host=mysql_host)
        cursor = cnx.cursor()
        query = "INSERT INTO ABTesting.tblFrontPageOptions (uuid, liked,
pageid) VALUES (%s,%s,%s);"
        args = (uuid_, 0, pageid)
        cursor.execute(query, args)
        cursor.close()
        cnx.commit()
        cnx.close()
    except mysql.connector.Error as err:
        app.logger.error("Something went wrong: {}".format(err))

When a user interacts with the page and clicks the thumbs-up button, Flask uses the
“UpdateVisitWithLike()” function to update the row using the unique identifier for the
session and turns the “liked” flag to true (Listing 14-20).

415
Chapter 14 A/B Testing on PythonAnywhere and MySQL

Listing 14-20. Abstracting “UpdateVisitWithLike()” Function

def UpdateVisitWithLike(uuid_):
    try:
        cnx = mysql.connector.connect(user=mysql_account, password=mysql_
password, database=mysql_database, host=mysql_host)
        cursor = cnx.cursor()
        query = "UPDATE ABTesting.tblFrontPageOptions SET liked = %s WHERE
uuid = %s;"
        args = (1, uuid_)
        cursor.execute(query, args)
        cursor.close()
        cnx.commit()
        cnx.close()
    except mysql.connector.Error as err:
        app.logger.error("Something went wrong: {}".format(err))

Finally, we create the administrative dashboard to view how the A/B testing is going
by offering total visit counts, total thumbs up and down, and how many thumbs up for
each image. We also offer a log view where we dump all the content from the ABTesting
table (Listing 14-21).

Listing 14-21. Abstracting “GetVoteResults()” Function

def GetVoteResults():
    results = "
    total_votes = 0
    total_up_votes = 0
    total_up_votes_page_1 = 0
    total_up_votes_page_2 = 0
    try:
        cnx = mysql.connector.connect(user=mysql_account, password=mysql_
password, database=mysql_database, host=mysql_host)
        cursor = cnx.cursor()
        query = "SELECT * FROM  ABTesting.tblFrontPageOptions"
        cursor.execute(query)

        for (uuid_, liked, pageid, time_stamp) in cursor:

416
 Chapter 14 A/B Testing on PythonAnywhere and MySQL

            total_votes += 1
            if liked==1 and pageid==1:
                total_up_votes_page_1 += 1
            if liked==1 and pageid==2:
                total_up_votes_page_2 += 1
            if liked == 1:
                total_up_votes += 1
            results += ("uuid: {} liked:{} pageid: {} on {:%m/%d/%Y
%H:%M:%S}".format(uuid_, liked, pageid, time_stamp)) + "<br />"
        cursor.close()
        cnx.close()
    except mysql.connector.Error as err:
        app.logger.error("Something went wrong: {}".format(err))

    return (results, total_votes, total_up_votes, total_up_votes_page_1,

total_up_votes_page_2)

Designing a Web Application

Let’s download the files for Chapter 14 and unzip them on your local machine if you
haven’t already done so. Your “web-application” folder should contain the following
files as shown in Listing 14-22.

Listing 14-22. Web Application Files

web-application
        ├── main.py
        ├── static
                    └──images
                          ├── background1.jpg
                          ├── background2.jpg
        └── templates
                    ├── admin.html
                    └── index.html

417
Chapter 14 A/B Testing on PythonAnywhere and MySQL

Running a Local Version

Sorry folks, there’s no local version this time. Instead, we’ll use the PythonAnywhere
wizard to create a MySQL instance in the cloud and upload all the needed data, file by
file, as we’ve done previously.

Setting Up MySQL on PythonAnywhere

It is really easy to set up MySQL on PythonAnywhere using the built-in wizard. Click the
“Databases” link in the upper right hand of the dashboard and proceed through the
setup just like we did earlier on the local version (Figure 14-9).

Figure 14-9. Setting up MySQL on PythonAnywhere

418
 Chapter 14 A/B Testing on PythonAnywhere and MySQL

After you initialize MySQL, you will be able to create a database and get into
the MySQL console to create the “tblFrontPageOptions.” There are two caveats
you will have to contend with. First, PythonAnywhere appends your account
name in front of the database name. In my case, database “ABTesting” becomes
“amunateguioutloo$ABTesting.” This isn’t a big deal, but we will have to update any
code that talks to the database. The second issue is that the user it creates for you is the
one you will have to add to your script, as it won’t let you create additional users using
the “CREATE USER” command (Figure 14-10).

Figure 14-10. Creating the ABTesting database and clicking the console link to set
things up

419
Chapter 14 A/B Testing on PythonAnywhere and MySQL

Click the console link for the “…$ABTesting” database and create the
“tblFrontPageOptions” table and “webuser” account. Make sure to update the database
to reflect your database name (Listing 14-23).

Listing 14-23. Create Table Command and PythonAnywhere Confirming That

the Table “tblFrontPageOptions” was Successfully Created
Input:

CREATE TABLE amunateguioutloo$ABTesting.tblFrontPageOptions (

       uuid VARCHAR(40) NOT NULL,
   liked BOOLEAN NOT NULL DEFAULT 0,
       pageid INT NOT NULL,
       time_stamp  TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP);

Output:

mysql> CREATE TABLE amunateguioutloo$ABTesting.tblFrontPageOptions (

    -> uuid VARCHAR(40) NOT NULL,
    ->    liked BOOLEAN NOT NULL DEFAULT 0,
    -> pageid INT NOT NULL,
    -> time_stamp  TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP);
Query OK, 0 rows affected (0.03 sec)

That is all we need to do in the console; everything else will be done through Python.

A/B Testing on PythonAnywhere

Let’s upload and update all the code to work with our new database on
PythonAnywhere. Under “Files” create a new folder called “ABTesting” (Figure 14-11).

Figure 14-11. Creating the new folder to host our ABTesting site
420
 Chapter 14 A/B Testing on PythonAnywhere and MySQL

Next, we need to upload all the files, one-by-one, up to the site just like we did with
the other PythonAnywhere projects we already did (Figures 14-12 through 14-14).

Figure 14-12. Upload “main.py” under the “ABTesting” folder

Figure 14-13. Upload both images under “ABTesting/static/”

Figure 14-14. Upload both HTML files under “ABTesting/templates/”

Once you have uploaded all files, you need to go into “main.py” and update
the database account and all table references. You will need to update the following
variables with the ones assigned to you by PythonAnywhere. Click “Databases” in the
upper right corner of your PythonAnywhere dashboard to access the variables
(Listing 14-24).
421
Chapter 14 A/B Testing on PythonAnywhere and MySQL

Listing 14-24. Assigned account data

mysql_account='<<ENTER-YOUR-DATABASE-USERNAME>>'
mysql_password='thesecret'
mysql_database='<<ENTER-YOUR-DATABASE-USERNAME>>$ABTesting'
mysql_host="<<ENTER-YOUR-DATABASE-USERNAME>>.mysql.pythonanywhere-services.com"

Make sure to replace in the “main.py” code all the “<<ENTER-YOUR-DATABASE-­

USERNAME>>” with your database user name, otherwise it will not work.
Hit the big green button on the web tab and take the web application for a spin.
Go ahead and vote away (then check the administrative page; Figure 14-15).

Figure 14-15. The “Do You Like Me?” web application running on
PythonAnywhere

422
 Chapter 14 A/B Testing on PythonAnywhere and MySQL

A/B Testing Results Dashboard

In order to view the results of our A/B testing operation, we need to create a dashboard.
Though this isn’t essential, and you could just as well query the results directly through
MySQL using SQL statements, a dashboard will allow anybody to look at the results
throughout your testing without needing SQL knowledge or querying permissions to the
ABTesting table (Figure 14-16).

Figure 14-16. A simple dashboard with the latest results of our A/B test

We will keep things simple here and offer the total votes, the total up and down votes,
the up votes per image, and the full log of all participants.

423
Chapter 14 A/B Testing on PythonAnywhere and MySQL

Conclusion
A/B testing is one of the popular tools to better understand your users. It is also a loaded
science with many ways to approach it. Here, we made the assumption that any new visit
doesn’t like the site, thus defaults with a thumbs-down. This doesn’t necessarily mean
they thought the page was bad, as it could also mean they didn’t have time to read the
question. So, in this scenario, I would look closer at the number of up votes per image
rather than worry about the down votes; in either case you can extract which image was
favored by the majority.
Another tool we introduced here is MySQL; it is a great open-source and free
relational database that is widely used and supported.

424