17-Parallel Processing/01-Multithreading and Multiprocessing.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Multithreading and Multiprocessing\n",
    "\n",
    "Recall the phrase \"many hands make light work\". This is as true in programming as anywhere else.\n",
    "\n",
    "What if you could engineer your Python program to do four things at once? What would normally take an hour could (almost) take one fourth the time.<font color=green>\\*</font>\n",
    "\n",
    "This is the idea behind parallel processing, or the ability to set up and run multiple tasks concurrently.\n",
    "\n",
    "\n",
    "<br><font color=green>\\* *We say almost, because you do have to take time setting up four processors, and it may take time to pass information between them.*</font>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##  Threading vs. Processing\n",
    "\n",
    "A good illustration of threading vs. processing would be to download an image file and turn it into a thumbnail.\n",
    "\n",
    "The first part, communicating with an outside source to download a file, involves a thread. Once the file is obtained, the work of converting it involves a process. Essentially, two factors determine how long this will take; the input/output speed of the network communication, or I/O, and the available processor, or CPU.\n",
    "\n",
    "#### I/O-intensive processes improved with multithreading:\n",
    "* webscraping\n",
    "* reading and writing to files\n",
    "* sharing data between programs\n",
    "* network communications\n",
    "\n",
    "\n",
    "#### CPU-intensive processes improved with multiprocessing:\n",
    "* computations\n",
    "* text formatting\n",
    "* image rescaling\n",
    "* data analysis"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Multithreading Example: Webscraping\n",
    "\n",
    "Historically, the programming knowledge required to set up multithreading was beyond the scope of this course, as it involved a good understanding of Python's Global Interpreter Lock (the GIL prevents multiple threads from running the same Python code at once). Also, you had to set up special classes that behave like Producers to divvy up the work, Consumers (aka \"workers\") to perform the work, and a Queue to hold tasks and provide communcations. And that was just the beginning.\n",
    "\n",
    "Fortunately, we've already learned one of the most valuable tools we'll need – the `map()` function. When we apply it using two standard libraries, *multiprocessing* and *multiprocessing.dummy*, setting up parallel processes and threads becomes fairly straightforward.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here's a classic multithreading example provided by [IBM](http://www.ibm.com/developerworks/aix/library/au-threadingpython/) and adapted by [Chris Kiehl](http://chriskiehl.com/article/parallelism-in-one-line/) where you divide the task of retrieving web pages across multiple threads:\n",
    "\n",
    "\n",
    "    import time \n",
    "    import threading \n",
    "    import Queue \n",
    "    import urllib2 \n",
    "\n",
    "    class Consumer(threading.Thread): \n",
    "      def __init__(self, queue): \n",
    "        threading.Thread.__init__(self)\n",
    "        self._queue = queue \n",
    "\n",
    "      def run(self):\n",
    "        while True: \n",
    "          content = self._queue.get() \n",
    "          if isinstance(content, str) and content == 'quit':\n",
    "            break\n",
    "          response = urllib2.urlopen(content)\n",
    "        print 'Thanks!'\n",
    "\n",
    "\n",
    "    def Producer():\n",
    "      urls = [\n",
    "        'http://www.python.org', 'http://www.yahoo.com'\n",
    "        'http://www.scala.org', 'http://www.google.com'\n",
    "        # etc.. \n",
    "      ]\n",
    "      queue = Queue.Queue()\n",
    "      worker_threads = build_worker_pool(queue, 4)\n",
    "      start_time = time.time()\n",
    "\n",
    "      # Add the urls to process\n",
    "      for url in urls: \n",
    "        queue.put(url)  \n",
    "      # Add the poison pill\n",
    "      for worker in worker_threads:\n",
    "        queue.put('quit')\n",
    "      for worker in worker_threads:\n",
    "        worker.join()\n",
    "\n",
    "      print 'Done! Time taken: {}'.format(time.time() - start_time)\n",
    "\n",
    "    def build_worker_pool(queue, size):\n",
    "      workers = []\n",
    "      for _ in range(size):\n",
    "        worker = Consumer(queue)\n",
    "        worker.start() \n",
    "        workers.append(worker)\n",
    "      return workers\n",
    "\n",
    "    if __name__ == '__main__':\n",
    "      Producer()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Using the multithreading library provided by the *multiprocessing.dummy* module and `map()` all of this becomes:\n",
    "\n",
    "    import urllib2\n",
    "    from multiprocessing.dummy import Pool as ThreadPool\n",
    "    \n",
    "    pool = ThreadPool(4) # choose a number of workers\n",
    "    \n",
    "    urls = [\n",
    "    'http://www.python.org', 'http://www.yahoo.com'\n",
    "    'http://www.scala.org', 'http://www.google.com'\n",
    "    # etc.. \n",
    "    ]\n",
    "    \n",
    "    results = pool.map(urllib2.urlopen, urls)\n",
    "    pool.close() \n",
    "    pool.join()\n",
    "    \n",
    "In the above code, the *multiprocessing.dummy* module provides the parallel threads, and `map(urllib2.urlopen, urls)` assigns the labor!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Multiprocessing Example: Monte Carlo\n",
    "\n",
    "Let's code out an example to see how the parts fit together. We can time our results using the *timeit* module to measure any performance gains. Our task is to apply the Monte Carlo Method to estimate the value of Pi."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Monte Carle Method and Estimating Pi\n",
    "\n",
    "If you draw a circle of radius 1 (a unit circle) and enclose it in a square, the areas of the two shapes are given as\n",
    "\n",
    "<table>\n",
    "    <caption>Area Formulas</caption>\n",
    "    <tr><td>circle</td><td>$$πr^2$$</td></tr>\n",
    "    <tr><td>square</td><td>$$4 r^2$$</td></tr>\n",
    "</table>\n",
    "\n",
    "\n",
    "Therefore, the ratio of the volume of the circle to the volume of the square is $$\\frac{π}{4}$$\n",
    "\n",
    "The Monte Carlo Method plots a series of random points inside the square. By comparing the number that fall within the circle to those that fall outside, with a large enough sample we should have a good approximation of Pi. You can see a good demonstration of this [here](https://academo.org/demos/estimating-pi-monte-carlo/) (Hit the **Animate** button on the page).\n",
    "\n",
    "For a given number of points *n*, we have $$π = \\frac{4 \\cdot points\\ inside\\ circle}{total\\ points\\ n}$$\n",
    "\n",
    "To set up our multiprocessing program, we first derive a function for finding Pi that we can pass to `map()`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "from random import random  # perform this import outside the function\n",
    "\n",
    "def find_pi(n):\n",
    "    \"\"\"\n",
    "    Function to estimate the value of Pi\n",
    "    \"\"\"\n",
    "    inside=0\n",
    "\n",
    "    for i in range(0,n):\n",
    "        x=random()\n",
    "        y=random()\n",
    "        if (x*x+y*y)**(0.5)<=1:  # if i falls inside the circle\n",
    "            inside+=1\n",
    "\n",
    "    pi=4*inside/n\n",
    "    return pi"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's test `find_pi` on 5,000 points:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "3.1064"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "find_pi(5000)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This ran very quickly, but the results are not very accurate!\n",
    "\n",
    "Next we'll write a script that sets up a pool of workers, and lets us time the results against varying sized pools. We'll set up two arguments to represent *processes* and *total_iterations*. Inside the script, we'll break *total_iterations* down into the number of iterations passed to each process, by making a processes-sized list.<br>For example:\n",
    "\n",
    "    total_iterations = 1000\n",
    "    processes = 5\n",
    "    iterations = [total_iterations//processes]*processes\n",
    "    iterations\n",
    "    # Output: [200, 200, 200, 200, 200]\n",
    "    \n",
    "This list will be passed to our `map()` function along with `find_pi()`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Writing test.py\n"
     ]
    }
   ],
   "source": [
    "%%writefile test.py\n",
    "from random import random\n",
    "from multiprocessing import Pool\n",
    "import timeit\n",
    "\n",
    "def find_pi(n):\n",
    "    \"\"\"\n",
    "    Function to estimate the value of Pi\n",
    "    \"\"\"\n",
    "    inside=0\n",
    "\n",
    "    for i in range(0,n):\n",
    "        x=random()\n",
    "        y=random()\n",
    "        if (x*x+y*y)**(0.5)<=1:  # if i falls inside the circle\n",
    "            inside+=1\n",
    "\n",
    "    pi=4*inside/n\n",
    "    return pi\n",
    "\n",
    "if __name__ == '__main__':\n",
    "    N = 10**5  # total iterations\n",
    "    P = 5      # number of processes\n",
    "    \n",
    "    p = Pool(P)\n",
    "    print(timeit.timeit(lambda: print(f'{sum(p.map(find_pi, [N//P]*P))/P:0.7f}'), number=10))\n",
    "    p.close()\n",
    "    p.join()\n",
    "    print(f'{N} total iterations with {P} processes')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "3.1466800\n",
      "3.1364400\n",
      "3.1470400\n",
      "3.1370400\n",
      "3.1256400\n",
      "3.1398400\n",
      "3.1395200\n",
      "3.1363600\n",
      "3.1437200\n",
      "3.1334400\n",
      "0.2370227286270967\n",
      "100000 total iterations with 5 processes\n"
     ]
    }
   ],
   "source": [
    "! python test.py"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Great! The above test took under a second on our computer.\n",
    "\n",
    "Now that we know our script works, let's increase the number of iterations, and compare two different pools. Sit back, this may take awhile!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Overwriting test.py\n"
     ]
    }
   ],
   "source": [
    "%%writefile test.py\n",
    "from random import random\n",
    "from multiprocessing import Pool\n",
    "import timeit\n",
    "\n",
    "def find_pi(n):\n",
    "    \"\"\"\n",
    "    Function to estimate the value of Pi\n",
    "    \"\"\"\n",
    "    inside=0\n",
    "\n",
    "    for i in range(0,n):\n",
    "        x=random()\n",
    "        y=random()\n",
    "        if (x*x+y*y)**(0.5)<=1:  # if i falls inside the circle\n",
    "            inside+=1\n",
    "\n",
    "    pi=4*inside/n\n",
    "    return pi\n",
    "\n",
    "if __name__ == '__main__':\n",
    "    N = 10**7  # total iterations\n",
    "    \n",
    "    P = 1      # number of processes\n",
    "    p = Pool(P)\n",
    "    print(timeit.timeit(lambda: print(f'{sum(p.map(find_pi, [N//P]*P))/P:0.7f}'), number=10))\n",
    "    p.close()\n",
    "    p.join()\n",
    "    print(f'{N} total iterations with {P} processes')\n",
    "    \n",
    "    P = 5      # number of processes\n",
    "    p = Pool(P)\n",
    "    print(timeit.timeit(lambda: print(f'{sum(p.map(find_pi, [N//P]*P))/P:0.7f}'), number=10))\n",
    "    p.close()\n",
    "    p.join()\n",
    "    print(f'{N} total iterations with {P} processes')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "3.1420964\n",
      "3.1417412\n",
      "3.1411108\n",
      "3.1408184\n",
      "3.1414204\n",
      "3.1417656\n",
      "3.1408324\n",
      "3.1418828\n",
      "3.1420492\n",
      "3.1412804\n",
      "36.03526345242264\n",
      "10000000 total iterations with 1 processes\n",
      "3.1424524\n",
      "3.1418376\n",
      "3.1415292\n",
      "3.1410344\n",
      "3.1422376\n",
      "3.1418736\n",
      "3.1420540\n",
      "3.1411452\n",
      "3.1421652\n",
      "3.1410672\n",
      "17.300921846344366\n",
      "10000000 total iterations with 5 processes\n",
      "\n"
     ]
    }
   ],
   "source": [
    "! python test.py"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Hopefully you saw that with 5 processes our script ran faster!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## More is Better ...to a point.\n",
    "\n",
    "The gain in speed as you add more parallel processes tends to flatten out at some point. In any collection of tasks, there are going to be one or two that take longer than average, and no amount of added processing can speed them up. This is best described in [Amdahl's Law](https://en.wikipedia.org/wiki/Amdahl%27s_law)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Advanced Script\n",
    "\n",
    "In the example below, we'll add a context manager to shrink these three lines\n",
    "\n",
    "    p = Pool(P)\n",
    "    ...\n",
    "    p.close()\n",
    "    p.join()\n",
    "    \n",
    "to one line:\n",
    "\n",
    "    with Pool(P) as p:\n",
    "    \n",
    "And we'll accept command line arguments using the *sys* module.\n",
    "    "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Writing test2.py\n"
     ]
    }
   ],
   "source": [
    "%%writefile test2.py\n",
    "from random import random\n",
    "from multiprocessing import Pool\n",
    "import timeit\n",
    "import sys\n",
    "\n",
    "N = int(sys.argv[1])  # these arguments are passed in from the command line\n",
    "P = int(sys.argv[2])\n",
    "\n",
    "def find_pi(n):\n",
    "    \"\"\"\n",
    "    Function to estimate the value of Pi\n",
    "    \"\"\"\n",
    "    inside=0\n",
    "\n",
    "    for i in range(0,n):\n",
    "        x=random()\n",
    "        y=random()\n",
    "        if (x*x+y*y)**(0.5)<=1:  # if i falls inside the circle\n",
    "            inside+=1\n",
    "\n",
    "    pi=4*inside/n\n",
    "    return pi\n",
    "\n",
    "if __name__ == '__main__':\n",
    "    \n",
    "    with Pool(P) as p:\n",
    "        print(timeit.timeit(lambda: print(f'{sum(p.map(find_pi, [N//P]*P))/P:0.5f}'), number=10))\n",
    "    print(f'{N} total iterations with {P} processes')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "3.14121\n",
      "3.14145\n",
      "3.14178\n",
      "3.14194\n",
      "3.14109\n",
      "3.14201\n",
      "3.14243\n",
      "3.14150\n",
      "3.14203\n",
      "3.14116\n",
      "16.871822701405073\n",
      "10000000 total iterations with 500 processes\n"
     ]
    }
   ],
   "source": [
    "! python test2.py 10000000 500"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Great! Now you should have a good understanding of multithreading and multiprocessing!"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}