create_your_own_dataset.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# How do I create my own dataset?\n",
    "\n",
    "So Caffe2 uses a binary DB format to store the data that we would like to train models on. A Caffe2 DB is a glorified name of a key-value storage where the keys are usually randomized so that the batches are approximately i.i.d. The values are the real stuff here: they contain the serialized strings of the specific data formats that you would like your training algorithm to ingest. So, the stored DB would look (semantically) like this:\n",
    "\n",
    "key1 value1\n",
    "key2 value2\n",
    "key3 value3\n",
    "...\n",
    "\n",
    "To a DB, it treats the keys and values as strings, but you probably want structured contents. One way to do this is to use a TensorProtos protocol buffer: it essentially wraps Tensors, aka multi-dimensional arrays, together with the tensor data type and shape information. Then, one can use the TensorProtosDBInput operator to load the data into an SGD training fashion.\n",
    "\n",
    "Here, we will show you one example of how to create your own dataset. To this end, we will use the UCI Iris dataset - which was a very popular classical dataset for classifying Iris flowers. It contains 4 real-valued features representing the dimensions of the flower, and classifies things into 3 types of Iris flowers. The dataset can be downloaded [here](https://archive.ics.uci.edu/ml/datasets/Iris)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "WARNING:root:This caffe2 python run does not have GPU support. Will run in CPU only mode.\n",
      "WARNING:root:Debug message: No module named caffe2_pybind11_state_gpu\n"
     ]
    }
   ],
   "source": [
    "# First let's import some necessities\n",
    "from __future__ import absolute_import\n",
    "from __future__ import division\n",
    "from __future__ import print_function\n",
    "from __future__ import unicode_literals\n",
    "\n",
    "%matplotlib inline\n",
    "import urllib2 # for downloading the dataset from the web.\n",
    "import numpy as np\n",
    "from matplotlib import pyplot\n",
    "from StringIO import StringIO\n",
    "from caffe2.python import core, utils, workspace\n",
    "from caffe2.proto import caffe2_pb2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Raw data looks like this:\n",
      "5.1,3.5,1.4,0.2,Iris-setosa\n",
      "4.9,3.0,1.4,0.2,Iris-setosa\n",
      "4.7,3.2,1.3,0.2,Iris-setosa\n",
      "4.6,3.1,1.5,0.2,...\n"
     ]
    }
   ],
   "source": [
    "f = urllib2.urlopen('https://archive.ics.uci.edu/ml/machine-learning-databases/iris/iris.data')\n",
    "raw_data = f.read()\n",
    "print('Raw data looks like this:')\n",
    "print(raw_data[:100] + '...')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "# load the features to a feature matrix.\n",
    "features = np.loadtxt(StringIO(raw_data), dtype=np.float32, delimiter=',', usecols=(0, 1, 2, 3))\n",
    "# load the labels to a feature matrix\n",
    "label_converter = lambda s : {'Iris-setosa':0, 'Iris-versicolor':1, 'Iris-virginica':2}[s]\n",
    "labels = np.loadtxt(StringIO(raw_data), dtype=np.int, delimiter=',', usecols=(4,), converters={4: label_converter})"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Before we do training, one thing that is often beneficial is to separate the dataset into training and testing. In this case, let's randomly shuffle the data, use the first 100 data points to do training, and the remaining 50 to do testing. For more sophisticated approaches, you can use e.g. cross validation to separate your dataset into multiple training and testing splits. Read more about cross validation [here](http://scikit-learn.org/stable/modules/cross_validation.html)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "random_index = np.random.permutation(150)\n",
    "features = features[random_index]\n",
    "labels = labels[random_index]\n",
    "\n",
    "train_features = features[:100]\n",
    "train_labels = labels[:100]\n",
    "test_features = features[100:]\n",
    "test_labels = labels[100:]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXcAAAEICAYAAACktLTqAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAIABJREFUeJzt3XucXHWZ5/HPYxKBFkhc0iNIQrcMjs6oiNJhYQDNbRURo/uSZXFgFMdMxo7OgBGvLCQEowuvGBgvxI2wM46JSERlAHWXCR0W2FlDOlwCGNZFzIURQkNIALlI4Nk/fqeS6kpVn1Pdp+pc6vt+vepVVef8+pynTlU9fep3nt855u6IiEi5vCrrAEREJH1K7iIiJaTkLiJSQkruIiIlpOQuIlJCSu4iIiWk5N4mZjbOzJ41syPSbJtCXLPNbHOr19Ng3ePNzM2sN3p+lZl9OaVlH2lmz1Y9v8PMzklj2dHybjazs9Ja3gjr+TMzu9fMnjGz+a1eX1mY2VwzuzXrOLKk5N5AlFwrt1fM7Pmq501/qd39ZXc/0N23ptm2nVr9hXH3ue7+1QRxPGJm02OW9bC7H5hGXGb2FTP7x5rlv8fdV6Wx/BhfAG5294Pc/cqxLCjtf3AJ1/lOM7vLzJ4zs/VmdnQ719+ImV1tZr+OvttnZx1PKyi5NxAl1wOjBLEV+EDVtH2+1GY2vv1RSj0ley96gAeyDgKa365mth/wz8A/AK8FrgGuN7MJLQivWXcDnwTuzTqQlnF33WJuwGZgds20rwDXEj6wzwDnACcAvwR2Ao8C3wAmRO3HAw70Rs9XRvN/Ef39/wHe0GzbaP77gF8Du4BvAv8bOKfBa+kCvg88RUgaXwA2V83/L8DD0XoeAOZE098GvAC8DDwLPBFNnwPcE7XfClwYsy2/CDwG/BvwiTqvc1H0+I+An0fbcgdwWzT9GuAV4PkojgXAUdFyPh7FMFCZVrXeO4AlwGC0nX4KvDaaN7t6G0TTHgGmA6cBfwBeita3oWp550SPXwVcBGwBHgf+ETg4mleJ7aPRMoeALyb83N0Wbe8XonUfCewPLAO2AduBK4H9o/aHRNtsKHp/bwQOj+ZdWrOsK2q3UZ3XNTeK4RvRe7CoavqD0Tp+AUxtEP+pwNaq5xa977MbtJ8LbIo+S78B5lbNm034Hn4+en2/Az5aNb8buAl4mvAdXALcmmAb/xI4O+sc04qb9tzH5j8CPwAmEhL9buBcYDJwInAK8Dcj/P1fABcC/46QlC5ptq2Z/RGwGvhctN7fAseNsJzFwFRCojgV+FjN/F9HsU8kfEF+YGavc/f7gE8Dt3v49TI5av8scHbU/gPAuWZ2Wr0VR9PPBWYCfwK8d4Q4P0f4J9MNHBq9dtz9I4Qv9vuiOJZV/c27gDcD72+wzI9Gt9cTEs3lI6yfaH03AZcBq6L1HVun2VzCNpgO/DFhL/Xva9r8OSGZvhe42MzemGDd7yL8I/9ktO6HgaXAG4CjgTcCvcAF0Z+8CvgucARhj/+lShzu/oWaZZ0Xt/6quDcR3odLzex0wnvzwWjaOsJ3oJ63ULVn7CGb3hdNr2c74b07GPhr4Js13ThTgAMI798ngeVmdnA0bznhn8KhwDzgrxK+vtJSch+bO9z9Rnd/xd2fd/f17r7O3XdHX8QVwLtH+Pvr3H3Q3V8CVgHHjKLtacA97v7P0bzLgSdGWM4ZwFfc/Sl33wJ8q3qmu69290ej1/QDwt5SX6OFufuAu98ftb8X+OEIr/kM4Gp3/5W7/x5YNEKcLxG+xEe4+x/c/X+N0LZiobs/5+7PN5j/vap1XwScaWaWYLlxzgKWuvtv3f0Z4MvAX5hZ9fdrkbu/4O53EX4Rvb3ZlUTLmwucF71/TwNfA84EcPchd/9p9Fl8GvgqI3/+ktjq7ss9HAd6nrCz8lV3/7/uvpvwC/Y4Mzu8zt8eSPiVVG0XcFC9FUXfpYc9GABuAU6uavIC4bP7krvfALwI/EnUzfMhwq/G59x9I+HXaUdTch+bbdVPzOzNZvYzM3vMzJ4m7CVPrv+nQOieqHiO8GVotu3rq+OI9o4eGWE5h9XEvaV6ppmdE1Vn7DSznYQ94YavwcxOMLNbzWzIzHYRkk+j9sNirV13jf8azb/FzH5jZp8boW3FtibmbwH2I/wSGqvXM/y1bAFeTdizBcDdm3mvGzmUEHP1+3MToQsLM3tNVHG0Nfr8DTDy5y+J2m3aA3y7av1PELrJptT522cJe+HVDibsYe/DzE4zs3VmtiNa9nsYHv8T7v5y1fPKdnwdMI7kn62OoOQ+NrWn1PxvwP3AUe5+MGHvMI09w5E8StUXK9oTrbcXVfEYoVumYk+5pZkdSfh52w8c4u6TCH2rlddQ7xSiPwR+TOh3nQhcRePX/Gijdddy96fd/TPu3kvYK/uCmVX2QuueyjT6xzaS2nW/SOhL/j3hWASw58DhIdWLjlnu7whJr3rZfyD0Dadpe7TcN7n7pOg2MdruEPqj3wAcF33+Ztb8fe3r+D2AmXVVTTs05m+2AZ+oWv8kdz/A3dfViXfYL5Tos/k26hwgNrMDgOsIv0ReF332bibZ92c74R9Mos9Wp1ByT9dBhJ+dvzezP2Xk/va03AS808w+ECWlc6naY6xjNfBlM5sU1dF/umregYQv8xDhuziXsOdesR2YUlPtcBCww91fMLPjiboIRlj3X0W/cF4DLGzUMHo9fxwlhF2Eg4GVvbbthGMGzfpo1bovBlZH/xAeBA4ys/dGr20hUP0atwO9I3ThXAMsMLNeMzuIcKziGnd/JS6gaJzB7iTBR3utVwFXmFm3BVPM7D1Rk4MIe7NPmdkhhJ2LarXb7bHodraFsRXzGP5Pqp7vABdEn2+iz9HpDdoOAOPM7FNR5cy5hO62el1s+xF+7QwBL0fHZ2bFxAJA1B15PeFYxgFm9lbgL0f6GzN7tZntT/jnMcHM9k+piy43lNzT9VnCAcpnCHvx17Z6he6+HfjPhAqKJwkH9O4m7JXWs5CwB72ZUOnwT1XL2kiojLgzavNmwgGzin8B/h+w3cwq3Qz9wNfMrNLXvHqEWG8Evk34cv86Wl4jbyIkh2cJ1T9/7+53RPO+Svgi7zSzpAcGIfTDroxe2zjgvCiup4C/Bb5HqObYwfBusGsJiWeHmd1ZZ7nfjdrczt5Ko3MTxjSV8PqS+iyhy+FOwj+9mwkHViF8BiYSPgf/Snh/q10BfCTabsuif2x/TXjfniAc8K23B76Hu/8oWs+Poq6fjTQ4MO7uLxAOvM4lVD2dDXwwSsa1bXcCnyFUMe0ATifsuCTVTziQvR24mlB+OZIBQsXVccB/jx6f2MT6cs/if8lKkZjZOEI3wenufnvW8cjIosFR33f3W7KORcpFyb0EzOwUQpnbC8CXCHtjR7p7o713ESk5dcuUw0mE7oAnCLX1H1JiF+ls2nMXESmhxHvu0dH0u81sn4McUW30kJndE93mphumiIg0o5kTAZ1LGIZcOyih4lp3/3SDefuYPHmy9/b2NrF6ERHZsGHDE+4+UrkzkDC5m9kUwjkflhBO1DRmvb29DA4OprEoEZGOYWaJRt8m7Za5gjD6baRBGR82s41mdp2ZTa3XwMzmmdmgmQ0ODaU9eE9ERCpik3s0Uuxxd98wQrMbCadtPRpYQxgMsg93X+Hufe7e190d+6tCRERGKcme+4nAHAuXYvshMNPMVlY3cPcnq0rvvgvUOy2qiIi0SWxyd/cvufuU6AROZwID7j7sslRmdljV0zmEA68iIpKRUV+OzMwWA4PReZX/zszmEC5WsYNwVSIREclIZoOY+vr6XNUyAsBll8G0aTBjxt5pa9fC+vXw+c9nF5dIDpnZBndveAGdCp1+QLI3bRqccUZI6BDuzzgjTBeRUSnTVeKlqGbMgNWrQ0Lv74fly8Pz6j15EWmK9twlH2bMCIn9kkvCvRK7yJgouUs+rF0b9tgvvDDcV7poRGRUlNwle5U+9tWrYfHivV00SvAio6bkLtlbv354H3ulD379+mzjEikwlUKKiBSISiFFRDqYkruISAkpuYuIlJCSu4hICSm5i4iUkJK7iEgJKbmLiJSQkruISAkpuYuIlJCSu4hICSm5i4iUkJK7iEgJKbmLiJSQkruISAkpuYuIlJCSu4hICSm5i4iUkJJ7mV122b7XIV27NkwXkVJTci+zadOGX2i6ciHqadOyjUtEWm581gFIC1UuNH3GGdDfD8uXD78QtYiUlvbcy27GjJDYL7kk3Cuxi3QEJfeyW7s27LFfeGG4r+2DF5FSUnIvs0of++rVsHjx3i4aJXiR0lNyL7P164f3sVf64NevzzYuEWk5c/dMVtzX1+eDg4OZrFtEpKjMbIO798W1S7znbmbjzOxuM7upzrz9zOxaM3vIzNaZWW9z4YqISJqa6ZY5F9jUYN4ngKfc/SjgcuDSsQYmIiKjlyi5m9kU4P3AVQ2afBD4XvT4OmCWmdnYwxMRkdFIuud+BfB54JUG8w8HtgG4+25gF3BIbSMzm2dmg2Y2ODQ0NIpwRUQkidjkbmanAY+7+4aRmtWZts+RWndf4e597t7X3d3dRJgiItKMJHvuJwJzzGwz8ENgppmtrGnzCDAVwMzGAxOBHSnGKSIiTYhN7u7+JXef4u69wJnAgLufXdPsBuBj0ePTozbZ1FiKiMjoTxxmZouBQXe/Abga+L6ZPUTYYz8zpfhERGQUmhqh6u63uvtp0eOLosSOu7/g7v/J3Y9y9+Pc/eFWBCttduqpsGzZ8GnLloXpIpJrOv2ANDZ7Npx//t4Ev2xZeD57drZxiUgsnc9dGluwINyffz5cfz3ccQcsXbp3uojklvbcZWQLFsBJJ8Htt4d7JXaRQlByl5EtWxb22E8+OdzX9sGLSC4puUtjlT72pUvhttvCfXUfvIjklvrcpbE1a4b3sVfu16xR94xIzul87iIiBZL6+dxFRKQ4lNyL6s1vhvnzh0+bPz9ML5rLLtv3uq5r14bpIiWy6r5V9F7Ry6sufhW9V/Sy6r5VLVuXkntRzZwJy5fvTfDz54fnM2dmG9doTJs2/MLdlQt7T5uWbVwiKVp13yrm3TiPLbu24Dhbdm1h3o3zWpbg1edeZJWEPnUqbNsG/f1w5ZVZRzU6lYTe3x9eU/WFvUVKoPeKXrbs2rLP9J6JPWw+b3Pi5ajPvRNceeXexD51anETO4RE3t8Pl1wS7pXYpWS27tra1PSxUnIvsvnz9yb2bdv27YMvkrVrwx77hReG+9o+eJGCO2LiEU1NHysl96KqdMn098PWrXu7M4qY4CtdMqtXw+LF4b66D16kBJbMWkLXhK5h07omdLFk1pKWrE+DmIpqYGB4H3vlfmAgu5hGa/364X3sM2aE5+vXq3tGSuOst50FwAW3XMDWXVs5YuIRLJm1ZM/0tOmAqohIgeiAqsSLqy9X/blIYSm5d7K4+nLVn4sUlvrcO1mlb7tRfXncfBHJLe25d7q4+nLVn4sUkpJ7p4urL1f9uUghKbl3srj6ctWfixSWknsnG6m+PMl8Eckt1bmLiBSI6txHKy+13XmJQ6TF2nmO806i5F4rL7XdeYlDpIXafY7zTqJumXrycm7xvMQh0iJpneO8k6hbZizyUtudlzhEWqTd5zjvJEru9eSltjsvcYi0SLvPcd5JlNxr5aW2Oy9xiLRQu89x3kmU3GvlpbY7L3GItNBZbzuLFR9YQc/EHgyjZ2IPKz6womXnOO8kOqAqIlIgqR1QNbP9zexOM7vXzB4ws4vrtDnHzIbM7J7oNne0gUvk1FNh2bLh05YtC9NBdfAiMqIk3TIvAjPd/e3AMcApZnZ8nXbXuvsx0e2qVKPsRLNnw/nn703wy5aF57Nnh+eqgxeREcSez91Dv82z0dMJ0S2bvpxOsmBBuD//fLj+erjjDli6dO90nWtdREaQ6ICqmY0zs3uAx4F/cfd1dZp92Mw2mtl1Zja1wXLmmdmgmQ0ODQ2NIewOsWABnHQS3H57uK8k9grVwYtIA4mSu7u/7O7HAFOA48zsrTVNbgR63f1oYA3wvQbLWeHufe7e193dPZa4O8OyZWGP/eSTw31tH7zq4EWkgaZKId19J3ArcErN9Cfd/cXo6XeBY1OJrpNV+tiXLoXbbgv31X3wqoMXkREkqZbpNrNJ0eMDgNnAgzVtDqt6OgfYlGaQHWnNmuF97AsWhOdr1oTnqoMXkRHE1rmb2dGEbpZxhH8Gq919sZktBgbd/QYz+xohqe8GdgD97v5gw4WiOncRkdFIWueuQUwiIgWis0KOVhqDg+IGIKWxjHYMYirpQKlFi0b/t0kuLKGLT0geKLnXSmNwUNwApDSW0Y5BTCUdKHXxPmOsk0lyYQldfEJyw90zuR177LGeWwMD7pMnu194YbgfGGh+GV//uruZ+8knh/uvfz39ZaQRZ5x2rKPNwtC85vVc3uMsYp9bz+U9TbURGQvCsc7YHKs993rSGBwUNwApjWW0YxBTSQZKLVoEZuEGex8300WT5MISuviE5IWSez1pDA6KG4CUxjLaMYipJAOlFi2CsM8enlceN5Pck1xYQhefkNxIsnvfiltuu2Uq3RCV7ofa50lUulMq3Si1z9NYRhpxxmnHOjIw2m6ZlRtXeteSrmHdLV1LunzlxpVNtREZC9QtM0ppDA6KG4CUxjLaMYippAOlFi4c3d8lubCELj4heaE6dxGRAlGde9HF1ZiXtAa9DCp17rYouzr3+T+bz/jF47GLjfGLxzP/Z/PbHoNkS8k9r+JqzEtag1501XXuWDZ17vN/Np/lg8t52V8G4GV/meWDy5XgO4y6ZfKskrAbXYwjbr60Xe8VvSGx1+iZ2MPm8za3JYbxi8fvSezVxtk4dl+0uy0xSOuoW6YM4mrMS1KDXiZbdtavZ280vRXqJfaRpks5KbnnWVyNeUlq0MukZ1L9evZG01thnI1rarqUk5J7XsVdjEMX68ilJbOW0DWha9i0rgldLJm1pG0xzDt2XlPTpZyU3PMqrsa8pDXoRVdd545nU+d+5fuvpL+vf8+e+jgbR39fP1e+/8q2xSDZ0wFVEZEC6cwDqmnUfsctI41ztacRhxRW3Pnei3g++FafI1+aV67knkbtd9wy0jhXexpxSCHFne+9qOeDb+U58mV0ytctk0btd9wyKgn9pJPC2RqrzwGTJtWxl05cHXwe6uRHw2zvGTebUdTXm6XO7JaBdGq/45aRxrna04hDCifufO9FOh98u86RL6NTvuSeRu133DLSOFd7GnFI4cSd771I54Nv1znyZXTKldzTqP2OW0alS2bpUrjttnBf3Qefp9ciuRNXB5+HOvl26rTX207lSu5p1H7HLSONc7WnEYcUUtz53ot6PvhWniNfRqd8B1RFREqscw+ojlU7auVVwy4tlkbtuOrPi03JvVY7auVVwy4tlEbtuOrPi0/dMvW0o1ZeNezSImnUjqv+PL/ULTMW7aiVVw27tEgateOqPy8+Jfd62lErrxp2aZE0asdVf158Su612lErrxp2aaE0asdVf158Su612lErrxp2aaE0asdVf158OqAqIlIgqR1QNbP9zexOM7vXzB4ws31O7mlm+5nZtWb2kJmtM7Pe0YUtIiJpSNIt8yIw093fDhwDnGJmx9e0+QTwlLsfBVwOXJpumCQb+JOXwUFpDGLKy2uJjOViDM0so5XrSTIop2gDd+Jeqy3K9rWWaZsXJc493D3xDegC7gL+fc30/wmcED0eDzxB1OXT6Hbsscd6UwYG3CdPDvf1nidt0w5xcRTptUSgPcto1XpWblzpXUu6nEXsuXUt6fKVG1c21SZv8vxa8xJHGvIUJzDoCfJ1oj53MxsHbACOAr7t7l+omX8/cIq7PxI9/030D+CJRsscVZ97koE/eRkclMYgpry8FkZ/MYZml9Gq9SQZlFPEgTt5fq15iSMNeYoz1UFM7v6yux8DTAGOM7O31q6v3p/VCWqemQ2a2eDQ0FCSVQ+XZOBPXgYHpTGIKePXksbFGJIsox3rSTIopygDd+Je65ad9eOtnt6O11qmbV6UOKs1VQrp7juBW4FTamY9AkwFMLPxwERgR52/X+Hufe7e193d3Xy0SQb+5GVwUBqDmDJ+LWlcjCHJMtqxniSDcooycCfutfZMqh9v9fR2vNYybfOixFktSbVMt5lNih4fAMwGHqxpdgPwsejx6cCAJ+nvaUaSgT95GRyUxiCmvLyWkkgyKKcsA3fy8lrzEkcaihLnMHGd8sDRwN3ARuB+4KJo+mJgTvR4f+BHwEPAncCRcctt+oDqpZfuezBxYCBMb6ZNO8TFUaTXElm4sD3LaOV6Vm5c6T2X97gtMu+5vKfuwbAkbfIk7rWyMNvXWqZtnpc4SfOAaitoEJOISPM686yQOasNl+HSqGFPQxr19iPNb2c9dF62aSsVrr48J8q1517dTz1jxr7PJVNplDm2K464No3mVy5y8dxLz+2Z1jWhq2XnZcnLNm2Vdm/PIki6516u5A65qg2X4fKSiFqZ3NtdD52Xbdoqeaovz4vO7JaBzGvDZbg0atjbFUdcmyTLaEc9dF62aTsUsb48L7TnLm2Tl71M7bkXh/bc99WZe+6qDZeMFbIeOse0PUevXMldF8HItYULs44gSBJHXJtG89t9kYu8bNNW0UVDRq983TIiIiXWmd0ykkvN1CmP5VzsaZo+Pfs42nWAtIwHYluhaPX22nOXlmq2TrneAcIsap3zEEe7DpaW/aBsGvJUb9+5de6SK81WO4z2/ORpy0McSu75kaeqHXXLSC4kqVNO41zsaZg+vX4clS6aMtWwd1KtfBqKWG+v5C4tleQ82Gmciz0Nt95aP45bb21fHGmc2z5P6ymLUp7PXWQs0qhTzkutc17ikPYr4nuv5C4t1Wydcr267Sxqnd/97uzjaFcNe9lr5dNQxHp7HVAVESkQHVCVYdSXulfW26Jo9dJSTNpz7xAqd9sry22Rp3ppKSbtuYvk0AW3XDAssQM899JzXHDLBRlFJGWl5F5iqmXeKy/booj10lJM6pbpEOqW2SvLbZGnkY5STOqWEcmhItZLSzEpuXcI1TLvleW2KGK9tBSTumVERApE3TJSSq0+AKoadGmkaJ8N7blLobTyYKhq0KWRPH02tOcu0iTVoEsjRfxsKLlL7rWrRl016NJIET8bSu6Se+0693gRz9kt7VHEz4aSu0hENejSSBE/G0ruUiitrFFXDbo0UsTPhqplREQKRNUyIiIdLDa5m9lUM1trZpvM7AEzO7dOm+lmtsvM7oluF7Um3PIo2oCIJOIOcHbK2Sgr760tyu69LePnS5oT2y1jZocBh7n7XWZ2ELAB+JC7/6qqzXTgfHc/LemKO7lbJk8DItIUN8CoE85MmYf3Ng8xSOuk1i3j7o+6+13R42eATcDhYw+xcxVxQIQkk4f3Ng8xSPaa6nM3s17gHcC6OrNPMLN7zewXZvaWBn8/z8wGzWxwaGio6WDLoogDIhqJG2CUl4tktMuWnfXfw0bTW6FMny8ZvcTJ3cwOBH4MnOfuT9fMvgvocfe3A98Erq+3DHdf4e597t7X3d092pgLr4gDIhqJG2DUrgFIedEzqf572Gh6K5Tp8yWjlyi5m9kEQmJf5e4/qZ3v7k+7+7PR458DE8xscqqRlkgRB0RIMnl4b/MQg2QvSbWMAVcDm9x9WYM2h0btMLPjouU+mWagZVLEARFJxA0w6oQLhlS/t3g2721ZP1/SnCTVMicBtwP3Aa9Ek78MHAHg7t8xs08D/cBu4Hlggbv/60jL7eRqGRGR0UpaLTM+roG73wFYTJtvAd9KHp5ULFpU3v5nEcmORqhm7OKLs45ARMpIyV1EpISU3DPQabXfItJ+OitkxjphSL6IpEdnhRQR6WBK7hnrhNpvEWk/JfeMqZ9dRFpByb0A9A+gmPS+SZZ0QLUAdNC1mPS+SSvogKqISAdTcs8p1cIXk943yQt1yxSAft4Xk943aQV1y4iIdDAl9wJQLXwx6X2TLKlbRkSkQNQtI9JAGgc3dYBU8k577tJx0jjQqYOlkhXtuYuIdDAld+kIadSfq4ZdikTdMtJx1C0jRaZuGRGRDqbkLh0njfpz1bBL3qlbRkSkQNQtIyLSwZTcRURKSMldRKSElNxFREpIyV1EpISU3EVESkjJXUSkhJTcRURKSMldRKSEYpO7mU01s7VmtsnMHjCzc+u0MTP7hpk9ZGYbzeydrQm3M+msgyLSrCR77ruBz7r7nwLHA58ysz+rafM+4I3RbR6wPNUoO9zFF2cdgYgUTWxyd/dH3f2u6PEzwCbg8JpmHwT+yYNfApPM7LDUoxURkUSa6nM3s17gHcC6mlmHA9uqnj/Cvv8AMLN5ZjZoZoNDQ0PNRdphdGEIERmLxMndzA4Efgyc5+5P186u8yf7nG7S3Ve4e5+793V3dzcXaYdZtChcDKJy0s7KYyV3EUkiUXI3swmExL7K3X9Sp8kjwNSq51OA3409PBERGY0k1TIGXA1scvdlDZrdAHw0qpo5Htjl7o+mGGdH04UhRKRZ4xO0ORH4S+A+M7snmvZl4AgAd/8O8HPgVOAh4Dng4+mH2rnUFSMizYpN7u5+B/X71KvbOPCptIISEZGx0QhVEZESUnIXESkhJXcRkRJSchcRKSFz32esUXtWbDYEbMlk5cFk4IkM19+MosSqONNVlDihOLGWIc4ed48dBZpZcs+amQ26e1/WcSRRlFgVZ7qKEicUJ9ZOilPdMiIiJaTkLiJSQp2c3FdkHUATihKr4kxXUeKE4sTaMXF2bJ+7iEiZdfKeu4hIaSm5i4iUUEckdzMbZ2Z3m9lNdeadY2ZDZnZPdJubUYybzey+KIbBOvNzcxHyBLFON7NdVdv0oozinGRm15nZg9EF3k+omZ+LbZogzrxszzdVxXCPmT1tZufVtMl8myaMMy/b9DNm9oCZ3W9m15jZ/jXz9zOza6PtuS66Gl4y7l76G7AA+AFwU5155wDfykGMm4HJI8w/FfgF4QydxwPrchzr9HrbOoM4vwfMjR6/GpiUx22aIM5cbM+amMYBjxEG1OQpzA5PAAADHUlEQVRumyaIM/NtSrgU6W+BA6Lnq4FzatrMB74TPT4TuDbp8ku/525mU4D3A1dlHcsY6SLkTTCzg4F3ES40g7v/wd131jTLfJsmjDOPZgG/cffaUeaZb9MajeLMi/HAAWY2Huhi3yvYfZDwzx/gOmBWdAGlWKVP7sAVwOeBV0Zo8+HoJ+R1ZjZ1hHat5MDNZrbBzObVmZ/oIuRtEhcrwAlmdq+Z/cLM3tLO4CJHAkPAP0RdcleZ2Wtq2uRhmyaJE7LfnrXOBK6pMz0P27Raozgh423q7v8GLAW2Ao8SrmB3c02zPdvT3XcDu4BDkiy/1MndzE4DHnf3DSM0uxHodfejgTXs/S/Zbie6+zuB9wGfMrN31cxPdBHyNomL9S7Cz+C3A98Erm93gIQ9oncCy939HcDvgS/WtMnDNk0SZx625x5m9mpgDvCjerPrTMvkcxoTZ+bb1MxeS9gzfwPweuA1ZnZ2bbM6f5poe5Y6uRMuETjHzDYDPwRmmtnK6gbu/qS7vxg9/S5wbHtD3BPH76L7x4GfAsfVNMnNRcjjYnX3p9392ejxz4EJZja5zWE+Ajzi7uui59cRkmhtm6y3aWycOdme1d4H3OXu2+vMy8M2rWgYZ0626Wzgt+4+5O4vAT8B/rymzZ7tGXXdTAR2JFl4qZO7u3/J3ae4ey/h59mAuw/7z1jTHzgH2NTGECsxvMbMDqo8Bt4D3F/TLBcXIU8Sq5kdWukXNLPjCJ+zJ9sZp7s/BmwzszdFk2YBv6pplvk2TRJnHrZnjY/QuKsj821apWGcOdmmW4HjzawrimUW++afG4CPRY9PJ+SwRHvuSS6QXTpmthgYdPcbgL8zsznAbsJ/xHMyCOl1wE+jz9p44Afu/j/M7JOQu4uQJ4n1dKDfzHYDzwNnJv1ApuxvgVXRz/OHgY/ndJvGxZmX7YmZdQH/Afibqmm526YJ4sx8m7r7OjO7jtBFtBu4G1hRk5+uBr5vZg8R8tOZSZev0w+IiJRQqbtlREQ6lZK7iEgJKbmLiJSQkruISAkpuYuIlJCSu4hICSm5i4iU0P8HN+3W5cl0ewUAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x11268b290>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX4AAAEICAYAAABYoZ8gAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAIABJREFUeJzt3XucHFWd9/HPlyQCEQwoo7LkMiiuq3LVCboLK4REFkUC+9Jl44ZHWfUZDboLD7I+aiSQsHlUdDF7MxjBexQi3gKCK5fh4iqQAQPhprJAIAbJCCaIQSTJ7/njVJOeTvdMdaZnurr7+369+tXdp05V/bq6+9fVp+rUUURgZmadY5dmB2BmZmPLid/MrMM48ZuZdRgnfjOzDuPEb2bWYZz4zcw6jBN/C5C0q6SnJP3JGKzrOEn3j/Z6aqx7N0khaXL2/MuSPtygZf+ppI1lz2+WdEojlp0t7zpJf9uo5Q2xngMl3Zl9HnpHe33tQtL7JV3T7DiKwol/BLIvX+m2TdLTZc/njmC5g5JSRDwTEXtExPrGRN4Yo/1liohTI+L8HHH8WtKRwyzrFxGxVyPikvRJSRdVLP+YiLi0EcsfxkeBH2Sfh2UjWVCjf/xyrnO6pNWSNku6VdKBY7n+WrKdjF9m3+M5zY5ntDnxj0D25dsjIvYAHgZOKCtb3uz4LJE0vtkxNNA04O5mBwH1b1dJuwPfB5YBewPfAr5bkPfndqAXuKvZgYyJiPCtATfgIWBWRdk44GzgAeA3wHJgr2za84FLgCeAjcAtpC/DvwBbgT8AT2XPdwMCmJzNewmwBPgv4HfAfwPTytZ7PPDLbLlLgJuBU2rE/fwsro3AGtIe5f1l0xcAD2bruQs4Pis/LItxSxbnr7PyvwbuAJ4E1gIfG2a7zQceA9YB763yOj+ePX4p8MMszseB67LybwHbgM1ZHP8I/FkW1/8GHgF+VCorW+/NwHnAbcAm4NvApGzaceXbICv7NXAkcBLwR+DZbH23li3vlLL3fSFpZ+Ax4IvAntm0Umx/n73mAeCfcn7GflLx2ZgK7J69x49kMf47sGtWvwu4KlvHE6Sku282rdrnbNA2qvK63g9cB/wn8Nuy9+Z9wM+zdfwA2K9G/LOBB8qe75LFfHSN+u8D7iN99u4H3l027bis7GPZ6/sVMLds+ouBK0mfw58CnwCuybGN+4E5zc4no33zHv/o+ifgWFLCmExKFp/Npr0XGA/sB+wDfBD4Y0R8CFgFvDfSP4cP1Vj235GS9AuBR0mJBkkvBS4F/g/pi78eeN0QMS4mJdVu0hfz1IrpPwf+ApgEfAq4RNI+EfEz4Azg+izOl2b1n8xi24v0I3CWpOOqrVjSScBpwFGkpPPmIeL8v1ks+wD7AucCRMTfABuAY7M4/i2rPw54PfBK4MQay3wnMJf0HjyPlPyGFBHfAy4AvpKt7/Aq1d4HnAz8JfAKUhK6oGz6OKAHOAB4C7BY0styrPsvGPzZeJj0eZoMHER6rX8KfCSbZRfgQtIPxP5Z2WezZeX9nFV6I7Ca9D78S9YscgZwAvAS4GfA12vM+xrSTkHp9Wwj7Uy8pkb9R0mfiReQfnT+U1J53WmAgD8hfX8ulLRHNm0Z6YfoJcA84N05X19HcOIfXe8DPhIR6yPiD6Tk/LeSRPoR6AJeHhFbImJVRPy+jmWviIjbI+JZ4BvAoVn5bGBVRFyRTfsMae+slpOB8yJiY0Q8SNqbe05EXBoRj0bEtoj4GmnPquYPSURcGxF3Z/VvB1aQEnutdX8hIu6LiKfIfrxqeJb0BZ8aEX+MiBuHqFuyICI2R8TTNaZ/qWzd5wDvyLHMPOYCn46ItRHxJOlfzdzsfS85JyL+EBGrSHu1B9e7kqyJ5N3A6dn7twn4JDAHICIei4jvR8TT2bRPUPu9yOuBiPhCRGzNtuv7gH+OdAzlWdJ7eKSkl1SZdw/Sv6tym4A9q60oIlZGxIORXAPcQNqJKtkMfCIino2I75L+LR4gaTfS9+Dj2WtfTfpXaxkn/lGSfcmnAFdK2pidUfIz0jZ/EXAx6YN8maR1kv6fpHF1rOLXZY83k75UkJLjI6UJ2V7Vr4aI8SXl9UnNM+V13pOdRVJ6DQeQ9vaqknSEpBskDUjaRPoHUav+oFgr111hMenfS5+k+yWdOURdgG0x/MHwynVPlDRpmHny+BMGv5a1pCaZF2bPt0bEb8qml79/9a5nAnB32fvzPdI/DCTtKemLkh6W9CSpyavme5fTIxXPp5H2tEvrHyA1ZU2uMu9TpL33ci8gNeXsQNLs7ADwE9myj2Fw/APZ57uktB1fSvonkPez1XGc+EdJRAQp4R4TEXuV3XaLiN9EOlNnQUT8Genv89+Q7amR9lx21qOUfekk7UJqyqgV4wbSD1TJ1LJ5/5TUZtwLvDDSWTH3k75UteJcQWpqmhIRk4Avl9WvFmvVdVeJdVNEnB4R04C3AR+XdMQQceTZhpXr3pztGf8emFiaIGkC25N2nmWvJyXE8mU/TWp6aKRHSUn25WWfr0kR8aJs+kdIn4XpEfECUrNj+XtR+Tp+D4yTtGtZ2Usr6lTO8whwasVnfPeIuK1KvHcDh5SeZJ/NA6lysFrS80nHb84DXpx99q6j9mep3K+zOHN9tjqRE//ouhD4pKQpAJJeLOmE7PEsSa/OPvxPkr7AW7P5HgOGbfOtYSXweklvyZoCziQdNK5lBTBf0iRJ00ht7iV7kA6cDgC7SHo/aY+/5DFgSpYYS/8g9gAej4g/SPoL0g/aUOt+b3aO/R6kA8lVZXt/+2fr2ETaViPdXqeWrftc0g8WwL3ACyXNzF7bQgZ/Vx4DSrFU803SsY2pkvYE/hn4RvZDOySlfhR/yBN81rTyReBfJe2jZIqkN2VV9iTtBW+UtA/w8YpFVG639aT3eq6kcZJOo8ZOQ5kLST/Cr8zi31vS22rUvRrYPTsNeFfScajfAz+uUnd30r+ZDcA2SbOBo4eJBYCsWfVyYKGk3SUdTGp+q0nS87ImIgETlPqU5PmRaUlO/KPrfOAa4DpJvyOdlfHabNp+pLMsSmfLXElKhJAOwL1T0m8lDXsee7mIeJTUVv1vpDOJJpPO1nmmxiwfz+o9TDoj46tly7qd9MXuJ+1d7p89Lvkh6WymDZLWZYnt/cBnstf7YdJeW61Yv0s6CHcTqZ37v4Z4aa8CridtrxuBz0TEzdm0xaQDpBslfXCIZVT6GilJ/4r0A/ehLK7fAKeT2oXXkfYgy5tmLiH9I3hC0k+qLHcp8B3S+/0/pD394ZqmSqaQztLK6wxSwu4n/SD+kO0/zp8hNY08TkquV1bMO+hzFhFbSScdnEN6vVNIZz3VFBHfBP4D+E7WnLQaeFONuk+TDrS/n3R21hzgpIjYUqXub4CzSAn8cdLZVJXxD+V9pGbMx4DPA18apv6NpH9lryV9B54mnRzQlpRjJ8RaWLbX/2tSH4OfNjseG5qkr5MOeN/Q7FisfTnxtyFJbybtbT5DOqPkXcABEfHHpgZmZoXgpp729EZSp6sNwEzgr530zazEe/xmZh3Ge/xmZh2mCBdH2sE+++wT3d3dzQ7DzKxl3Hbbbb+JiK48dQuZ+Lu7u+nv7x++opmZASApd+/k3E09WYeOn0m6osq0MyXdk3XtvzbrCFSatlXp+turJa3Muz4zMxsd9ezxn07q0Vh5rQ1I16DpiYjNkuaROi6VRiN6OiIOrTKPmZk1Qa49fqWh8I4HLqo2PSL6ImJz9vRmql+gyczMCiBvU88SUvf7bcNVBN5DGvyhZDdJ/UrDvJ1UayZJvVm9/oGBgZxhmZlZvYZN/JLeCmyocbW9yrqnkAaY+HRZ8dSI6CENzrFE0surzRsRyyKiJyJ6urpyHZg2M7OdkGeP/whgtqSHSBenOia7nsggkmaRLg8wOyKeuyBY6ZroEfEA6SJbh408bCuM88+Hvr7BZX19qdzMCmnYxB8RH42IyRHRTbqa3nURcUp5HUmHka6ANzsiNpSV7126tnd2WdgjgHsaGL812/TpcPLJ25N/X196Pn16c+Mys5p2+jx+SYuA/ohYSWra2QP4VnYJ64cjYjbpUrqfl7SN9CPzyYhw4m8nM2bAihUp2c+bB0uXpuczZjQ7MjOroa7EHxHXk5priIgFZeWzatT/CWkQaGtnM2akpH/eeXD22U76ZgXna/XYyPX1pT39s89O95Vt/mZWKE78NjKlNv0VK2DRou3NPk7+ZoXlxG8js2rV4Db9Upv/qlXNjcvMairk9fh7enrCF2kzM8tP0m1Zn6lheY/fzKzDOPGbmXUYJ34zsw7jxG9m1mGc+M3MOowTv5lZh3HiNzPrME78ZmYdxonfzKzDOPGbmXUYJ34zsw7jxG9m1mFyJ35J4yT9TNIVVabtKulSSfdLukVSd9m0j2blP5f0V40J2zqWx/g1G7F69vhPB+6tMe09wG8j4gDgs8CnACS9mjRO72uA44DPSRq38+Fax/MYv2YjlivxS5oMHA9cVKPKicBXsseXATOVBt89EbgkIp6JiAeB+4HDRxaydbTyMX4XLNg+CIyHezTLLe8e/xLgw8C2GtP3Ax4BiIgtwCbgReXlmXVZ2Q4k9Urql9Q/MDCQMyzrSOVj/M6b56RvVqdhE7+ktwIbIuK2oapVKYshyncsjFgWET0R0dPV1TVcWNbJPMav2Yjk2eM/Apgt6SHgEuAYSV+vqLMOmAIgaTwwCXiivDwzGVg/wpitk3mMX7MRGzbxR8RHI2JyRHSTDtReFxGnVFRbCbwre/z2rE5k5XOys372B14B3Nqw6K3zeIxfsxEbv7MzSloE9EfESuBi4GuS7ift6c8BiIi7Ja0A7gG2AB+IiK0jD9s61oc/vGPZjBlu5zergwdbNzNrAx5s3czManLiNzPrME78ZmYdxonfzKzDOPGbmXUYJ34zsw7jxG9m1mGc+M3MOowTv5lZh3HiNzPrME78ZmYdxonfzKzJlq9ZTveSbnZZuAvdS7pZvmb5qK5vp6/OaWZmI7d8zXJ6L+9l87ObAVi7aS29l/cCMPeguaOyTu/xm5k10fxr5z+X9Es2P7uZ+dfOH7V1OvGbmTXRw5serqu8EZz4zcyaaOqkqXWVN0KewdZ3k3SrpDsk3S1pYZU6n5W0Orv9QtLGsmlby6atbPQLMDNrZYtnLmbihImDyiZOmMjimYtHbZ15Du4+AxwTEU9JmgD8WNJVEXFzqUJE/J/SY0n/ABxWNv/TEXFowyI2M2sjpQO486+dz8ObHmbqpKksnrl41A7sQo7Enw2a/lT2dEJ2G2q8xncA54w8NDOzzjD3oLmjmugr5WrjlzRO0mpgA3B1RNxSo940YH/gurLi3ST1S7pZ0klDrKM3q9c/MDBQx0swM7N65Er8EbE1a66ZDBwu6cAaVecAl0XE1rKyqdkAwH8HLJH08hrrWBYRPRHR09XVVcdLMDOzetR1Vk9EbASuB46rUWUO8M2KedZn9w9k8x6242w2rPPPh76+wWV9fanczKwOec7q6ZK0V/Z4d2AWcF+Veq8E9gZ+Wla2t6Rds8f7AEcA9zQm9A4zfTqcfPL25N/Xl55Pn97cuMys5eQ5q2df4CuSxpF+KFZExBWSFgH9EVE6RfMdwCXZweCSVwGfl7Qtm/eTEeHEvzNmzIAVK1KynzcPli5Nz2fMaHZkZtZiNDhPF0NPT0/09/c3O4xiWrAAzjsPzj4bFi1qdjRmVhCSbsuOpw7LPXdbSV9f2tM/++x0X9nmb2aWgxN/qyi16a9Ykfb0S80+Tv5mVicn/laxatXgNv1Sm/+qVc2Ny8xajtv4zczagNv4zcysJid+M7MO48RvZtZhnPjNzDqME7+ZWYdx4jcz6zBO/GZmHcaJ38yswzjxm5l1GCd+M7MO48RvZtZh8ozAtZukWyXdIeluSQur1DlV0oCk1dntvWXT3iXpl9ntXY1+AR0jz9CLzRie0UNCmrWcPHv8zwDHRMQhwKHAcZLeUKXepRFxaHa7CEDSC4FzgNcDhwPnSNq7QbF3ljxDLzZjeEYPCWnWcoZN/JE8lT2dkN3yXtLzr4CrI+KJiPgtcDW1B2q3oZQPvbhgwfZr85cPvZinTjPiMrNCydXGL2mcpNXABlIiv6VKtbdJulPSZZKmZGX7AY+U1VmXlVVbR6+kfkn9AwMDdbyEDjJjRhpv97zz0n215JqnTjPiMrPCyJX4I2JrRBwKTAYOl3RgRZXLge6IOBi4BvhKVq5qi6uxjmUR0RMRPV1dXfmi7zR5hl5sxvCMHhLSrKXUdVZPRGwErqeiuSYiHo+IZ7KnXwBelz1eB0wpqzoZWL9TkXa6PEMvNmN4Rg8JadZy8pzV0yVpr+zx7sAs4L6KOvuWPZ0N3Js9/i/gWEl7Zwd1j83KrF55hl5sxvCMHhLSrOUMO/SipINJTTfjSD8UKyJikaRFQH9ErJT0CVLC3wI8AcyLiPuy+d8NfCxb3OKI+NJwQXnoRTOz+tQz9KLH3DUzawMec9fMzGpy4m8njexF6x65Zm3Lib+dNLIXrXvkmrWt8c0OwBqovBftvHnpnPqd7UXbyGWZWaF4j7/dNLIXrXvkmrUlJ/5208hetO6Ra9aWnPjbSSN70bpHrlnbcuJvJ43sReseuWZtyx24zMzagDtwmZlZTU78ZmYdxonfzKzDOPGbmXUYJ34zsw7jxG9m1mGc+M3MOkyeoRd3k3SrpDsk3S1pYZU6Z0q6R9Kdkq6VNK1s2lZJq7Pbyka/ADMzq0+eq3M+AxwTEU9JmgD8WNJVEXFzWZ2fAT0RsVnSPOB84G+zaU9HxKGNDdvMzHbWsHv8kTyVPZ2Q3aKiTl9EbM6e3gxMbmiUZmbWMLna+CWNk7Qa2ABcHRG3DFH9PcBVZc93k9Qv6WZJJw2xjt6sXv/AwECu4M3MrH65En9EbM2aayYDh0s6sFo9SacAPcCny4qnZteP+DtgiaSX11jHsojoiYierq6uul6EmZnlV9dZPRGxEbgeOK5ymqRZwHxgdkQ8UzbP+uz+gWzew3Y+XLP6nXtusyMYmaLFv3zNcrqXdLPLwl3oXtLN8jXLd6qONc+wV+eU1AU8GxEbJe0O/Aj4VERcUVbnMOAy4LiI+GVZ+d7A5oh4RtI+wE+BEyPinqHW6atzWiNJUMCL0OZWpPiXr1lO7+W9bH5283NlEydMZNkJy5h70NzcdazxGn11zn2BPkl3AqtIbfxXSFokaXZW59PAHsC3Kk7bfBXQL+kOoA/45HBJ38yKa/618wcldIDNz25m/rXz66pjzZXnrJ47I+KwiDg4Ig6MiEVZ+YKIWJk9nhURL4mIQ7Pb7Kz8JxFxUEQckt1fPLovxyw599y0pyyl56XHRWs2qaWo8T+86eFhy/PUseZyz11rS+eem5pHSk0kpcfNTpx5FTX+qZOmDluep441lxO/meW2eOZiJk6YOKhs4oSJLJ65uK461lxO/Nb2zjmn2RGMTJHin3vQXJadsIxpk6YhxLRJ03Y4aJunjjWXx9w1M2sDHnPXzMxqcuI3M+swTvyj6S1vgQsuGFx2wQWp3GwYzer9evTRI19GKXad6567RZTnssy2s2bNgrPOSo/PPDMl/bPOgs98prlxWeFV9n5du2ktvZf3Aoz6QdIbbhjZ/INi19jGbvn44O5oKyX7I4+EH/84Jf0zz2x2VFZw3Uu6Wbtp7Q7l0yZN46EzHhrVdY/0EhHNjL2T+eBukZx5Zkr6N92U7p30LYex7v169NHVewrvTLPP2o3VY6xVbmPPiX+0XXBB2tP/y79M95Vt/mZVjHXv1+uvr95T+Prr61/WtL2qx1ir3MaeE/9oKm/Tv/HGdH/WWU7+NqxW7v3ayrF3Cif+0XTNNYPb9M88Mz2/5prmxmWF18zer0cdNbL5y2Mn3HO3iHxw18ysDfjgrpmZ1eTEb2bWYYZN/JJ2k3SrpDsk3S1pYZU6u0q6VNL9km6R1F027aNZ+c8l/VVjw2+S88+Hvr7BZX19qdyGNJbXkx+Nnq/Nvh5+pVlfnYUW6rnbrK/OanZIVqGI4w/n2eN/BjgmIg4BDgWOk/SGijrvAX4bEQcAnwU+BSDp1cAc4DWkAdo/J2lco4JvmunT4eSTtyf/vr70fPr05sbVAhbusNswOkq9R9duWksQz/UeHemXbqziz2PWV2dx7YPXDiq79sFrnfwLZLQ+hyOVZ+jFiIinsqcTslvlEeETga9kjy8DZkpSVn5JRDwTEQ8C9wOHNyTyZpoxA1asSMl+wYJ0v2JFKrdC6IRxXyuT/nDlNvaK+jnM1cYvaZyk1cAG0mDrt1RU2Q94BCAitgCbgBeVl2fWZWXV1tErqV9S/8DAQH2vohlmzIB58+C889K9k35NzRg/tpE9X4s6/q0VX1HHH86V+CNia0QcCkwGDpd0YEUVVZttiPJq61gWET0R0dPV1ZUnrObq64OlS+Hss9N9ZZu/PacZ48c2sudrUce/teIr6vjDdZ3VExEbgetJ7fXl1gFTACSNByYBT5SXZyYD63cy1uIotemvWAGLFm1v9nHyL4xO6D06c/+ZdZXb2Cvq5zDPWT1dkvbKHu8OzALuq6i2EnhX9vjtwHWReoatBOZkZ/3sD7wCuLVRwTfNqlWD2/RLbf6rVjU3rhYwVuPHjlbP1yKNf3vNO6/ZIcnP3H8m17zTPcOLoqjjDw/bc1fSwaQDt+NIPxQrImKRpEVAf0SslLQb8DXgMNKe/pyIeCCbfz7wbmALcEZEXDVcUO65a2ZWn3p67vqSDWZmbcCXbDAzs5qc+K2QfMbMdt4W1mhu6rFCGunwf+3E28LycFOPmZnV5MRvheEestt5W9hoclOPFZKbN7bztrA83NRjZmY1OfFbIRWph2yzeVtYo7mpx8ysDbipx8zManLiNzPrME78VkjtfNpi3jFYT/vBaYxfNB4tFOMXjee0H5w2xpF2riKOk9tIbuO3QmrXUxhLY7CWD8c3ccLEHS7Ve9oPTmNp/9Id5p/XM4/PHf+5MYm1U+V9j4rGV+e0lteuib97STdrN63doXzapGk8dMZDzz0fv2g8W2PrDvXGaRxbFmwZzRA7Xt73qGh8cNdaUif0Vs07Bmu1pD9UuTVOUcfJbaQ8I3BNkdQn6V5Jd0s6vUqdf5K0OrvdJWmrpBdm0x6StCab5t14q6kTxrbNOwbrOI2rWq9WuTVOUcfJbaQ8e/xbgA9FxKuANwAfkPTq8goR8emIODQbkP2jwA0R8URZlRnZ9Fx/Q8zaVd4xWHtf11t1/lrl1jhFHSe3kYZN/BHxaETcnj3+HXAvsN8Qs7wD+GZjwrNO1a69VfOOwfq54z/HvJ55z+3hj9M4H9gdI0UdJ7eR6jq4K6kbuBE4MCKerDJ9IrAOOKC0xy/pQeC3QACfj4hlw63HB3fNzOpTz8Hd8XUsdA/g26QB03dI+pkTgP+uaOY5IiLWS3oxcLWk+yLixirL7wV6AaZObZ+2NDOzosl1Vo+kCaSkvzwivjNE1TlUNPNExPrsfgPwXeDwajNGxLKI6ImInq6urjxhWQtqxIHa5zo2nduaHZvavXOQFV+es3oEXAzcGxEXDFFvEnAU8P2ysudL2rP0GDgWuGukQVvrWrhwZPOXOjZtja2gdHrj0v6lLZP8S52D1m5aSxCs3bSW3st7nfxtTA3bxi/pSOAmYA2wLSv+GDAVICIuzOqdChwXEXPK5n0ZaS8fUrPSNyJi2EPjbuNvXyPtmNXqHZtatXOQFV9D2/gj4seActT7MvDlirIHgEPyBGLt69xzB+/plzponXNO/U0/W7dtrfpp3LqtNTo2dULnICs+99y1UdfIjlnjdqnRsalGedF0QucgKz4nfmsprd6xqRM6B1nxOfHbmBppx6xBHZui9To2dULnICs+X53TzKwN+OqcZmZWkxO/mVmHceK3hmnk5ZPb6VLMZkXjNn5rmEaOmtWuI3CZjRa38ZuZWU1O/DYijRwusROGXjQrAjf1WMO4qcesedzUY2ZmNTnxW8M0crjEdh160awI3NRjZtYG3NRjZmY1OfGbmXWYPEMvTpHUJ+leSXdLOr1KnaMlbZK0OrstKJt2nKSfS7pf0kca/QI6UTuf3ujxaM1GX56hF/cF9o2I27Pxc28DToqIe8rqHA2cFRFvrZh3HPAL4E3AOmAV8I7yeatxG//Q2vVUx9J4tJuf3fxc2cQJE33ZYrMcGtrGHxGPRsTt2ePfAfcC++WM5XDg/oh4ICL+CFwCnJhzXusw86+dPyjpA2x+djPzr53fpIjM2lNdbfySuoHDgFuqTP5zSXdIukrSa7Ky/YBHyuqso8aPhqReSf2S+gcGBuoJqyN0Qq9Wj0drNjZyJ35JewDfBs6IiCcrJt8OTIuIQ4B/B75Xmq3Koqo2UkTEsojoiYierq6uvGF1jEaOW1tUHo/WbGzkSvySJpCS/vKI+E7l9Ih4MiKeyh5fCUyQtA9pD39KWdXJwPoRR21tyePRmo2NPGf1CLgYuDciLqhR56VZPSQdni33cdLB3FdI2l/S84A5wMpGBd+p2rVXq8ejNRsbec7qORK4CVgDbMuKPwZMBYiICyV9EJgHbAGeBs6MiJ9k878FWAKMA74YEcPuvvmsHjOz+tRzVo8v2WBm1gZ8yQYzM6vJid8a5uijmx2BmeXhxG8Nc8MNzY7AzPJw4jcz6zBO/DYiRx9dvUexm33Mimt8swOw1nb99dsft+vF48zajff4zcw6jBO/NcxRRzU7AjPLw4nfGqa82cfMisuJ38yswzjxm5l1mPZI/OefD319g8v6+lK5jZl2GhvArJ21R+KfPh1OPnl78u/rS8+nT29uXB1m4cJmR2BmebTHefwzZsCKFSnZz5sHS5em5zNmNDsyM7PCaY89fkhJft48OO+8dO+kPyY6YSxgs3bTPom/ry/t6Z99drqvbPO3UdEJYwGbtZs8Qy9OkdQn6V5Jd0s6vUqduZLuzG4/kXRI2bSHJK2RtFrS6IyuUmrTX7ECFi3a3uzj5G9mtoM8e/xbgA9FxKuANwAfkPTqijoPAkdFxMHAecCyiukzIuLQvKPD1G3VqsFt+qU2/1WrRmV1Vl1dAGx4AAAFxklEQVS7jgVs1m7qHnpR0veB/4iIq2tM3xu4KyL2y54/BPRExG/yrsNDL5qZ1WfUhl6U1A0cBtwyRLX3AFeVPQ/gR5Juk9Q7xLJ7JfVL6h8YGKgnLDMzq0Pu0zkl7QF8GzgjIp6sUWcGKfEfWVZ8RESsl/Ri4GpJ90XEjZXzRsQysiainp4eX9zXzGyU5NrjlzSBlPSXR8R3atQ5GLgIODEiHi+VR8T67H4D8F3g8JEGbdYoy9csp3tJN7ss3IXuJd0sX7O82SGZjbo8Z/UIuBi4NyIuqFFnKvAd4H9FxC/Kyp8vac/SY+BY4K5GBG42UsvXLKf38l7WblpLEKzdtJbey3ud/K3tDXtwV9KRwE3AGmBbVvwxYCpARFwo6SLgbcDabPqWiOiR9DLSXj6kZqVvRMTi4YLywV0bC91Lulm7ae0O5dMmTeOhMx4a+4DMRqCeg7vDtvFHxI8BDVPnvcB7q5Q/AByy4xxmzffwpofrKjdrF+3Tc9esTlMnTa2r3KxdOPFbx1o8czETJ0wcVDZxwkQWzxy2NdKspTnxW8eae9Bclp2wjGmTpiHEtEnTWHbCMuYeNLfZoZmNqrp77o4FH9w1M6vPqPXcNTOz1ufEb2bWYZz4zcw6jBO/mVmHceI3M+swhTyrR9IA2y//UK99gNzX/i+YVo4dWjv+Vo4dHH8zFSX2aRHRladiIRP/SEjqH7WRvkZZK8cOrR1/K8cOjr+ZWjF2N/WYmXUYJ34zsw7Tjom/cqD3VtLKsUNrx9/KsYPjb6aWi73t2vjNzGxo7bjHb2ZmQ3DiNzPrMC2b+CWNk/QzSVdUmXaqpAFJq7PbDqODNZOkhyStyWLb4TKkSv5N0v2S7pT02mbEWUuO+I+WtKls+y9oRpzVSNpL0mWS7pN0r6Q/r5he9G0/XPyF3PaSXlkW02pJT0o6o6JOYbd9zvgLue2rGXboxQI7HbgXeEGN6ZdGxAfHMJ56zYiIWp0+3gy8Iru9Hlia3RfJUPED3BQRbx2zaPL7V+CHEfF2Sc8DJlZML/q2Hy5+KOC2j4ifA4dC2mkDfsX28bhLCrvtc8YPBdz21bTkHr+kycDxwEXNjmWUnAh8NZKbgb0k7dvsoFqdpBcAbwQuBoiIP0bExopqhd32OeNvBTOB/4mIyt75hd32FWrF3zJaMvEDS4APA9uGqPO27O/iZZKmjFFceQXwI0m3SeqtMn0/4JGy5+uysqIYLn6AP5d0h6SrJL1mLIMbwsuAAeBLWTPhRZKeX1GnyNs+T/xQzG1fbg7wzSrlRd725WrFD8Xf9kALJn5JbwU2RMRtQ1S7HOiOiIOBa4CvjElw+R0REa8l/bX9gKQ3VkxXlXmKdN7tcPHfTrpuyCHAvwPfG+sAaxgPvBZYGhGHAb8HPlJRp8jbPk/8Rd32AGTNU7OBb1WbXKWsKNseGDb+Qm/7ci2X+IEjgNmSHgIuAY6R9PXyChHxeEQ8kz39AvC6sQ1xaBGxPrvfQGonPLyiyjqg/F/KZGD92EQ3vOHij4gnI+Kp7PGVwARJ+4x5oDtaB6yLiFuy55eREmllnaJu+2HjL/C2L3kzcHtEPFZlWpG3fUnN+Ftg2z+n5RJ/RHw0IiZHRDfpL9d1EXFKeZ2KdsHZpIPAhSDp+ZL2LD0GjgXuqqi2EnhndpbDG4BNEfHoGIdaVZ74Jb1UkrLHh5M+Z4+PdayVIuLXwCOSXpkVzQTuqahW2G2fJ/6ibvsy76B2M0lht32ZmvG3wLZ/Tiuf1TOIpEVAf0SsBP5R0mxgC/AEcGozY6vwEuC72edjPPCNiPihpPcDRMSFwJXAW4D7gc3A3zcp1mryxP92YJ6kLcDTwJwoThfxfwCWZ3/ZHwD+voW2PQwff2G3vaSJwJuA95WVtcy2zxF/Ybd9JV+ywcysw7RcU4+ZmY2ME7+ZWYdx4jcz6zBO/GZmHcaJ38yswzjxm5l1GCd+M7MO8/8BlFkDyPlKrkIAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x112858ad0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Let's plot the first two features together with the label.\n",
    "# Remember, while we are plotting the testing feature distribution\n",
    "# here too, you might not be supposed to do so in real research,\n",
    "# because one should not peek into the testing data.\n",
    "legend = ['rx', 'b+', 'go']\n",
    "pyplot.title(\"Training data distribution, feature 0 and 1\")\n",
    "for i in range(3):\n",
    "    pyplot.plot(train_features[train_labels==i, 0], train_features[train_labels==i, 1], legend[i])\n",
    "pyplot.figure()\n",
    "pyplot.title(\"Testing data distribution, feature 0 and 1\")\n",
    "for i in range(3):\n",
    "    pyplot.plot(test_features[test_labels==i, 0], test_features[test_labels==i, 1], legend[i])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now, as promised, let's put things into a Caffe2 DB. In this DB, what would happen is that we will use \"train_xxx\" as the key, and use a TensorProtos object to store two tensors for each data point: one as the feature and one as the label. We will use Caffe2's Python DB interface to do so."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "This is what the tensor proto looks like for a feature and its label:\n",
      "protos {\n",
      "  dims: 4\n",
      "  data_type: FLOAT\n",
      "  float_data: 6.69999980927\n",
      "  float_data: 3.0\n",
      "  float_data: 5.19999980927\n",
      "  float_data: 2.29999995232\n",
      "}\n",
      "protos {\n",
      "  data_type: INT32\n",
      "  int32_data: 2\n",
      "}\n",
      "\n",
      "This is the compact string that gets written into the db:\n",
      "\n",
      "\u0016\b\u0004\u0010\u0001\u001a\u0010ff�@\u0000\u0000@@ff�@33\u0013@\n",
      "\u0005\u0010\u0002\"\u0001\u0002\n"
     ]
    }
   ],
   "source": [
    "# First, let's see how one can construct a TensorProtos protocol buffer from numpy arrays.\n",
    "feature_and_label = caffe2_pb2.TensorProtos()\n",
    "feature_and_label.protos.extend([\n",
    "    utils.NumpyArrayToCaffe2Tensor(features[0]),\n",
    "    utils.NumpyArrayToCaffe2Tensor(labels[0])])\n",
    "print('This is what the tensor proto looks like for a feature and its label:')\n",
    "print(str(feature_and_label))\n",
    "print('This is the compact string that gets written into the db:')\n",
    "print(feature_and_label.SerializeToString())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Now, actually write the db.\n",
    "\n",
    "def write_db(db_type, db_name, features, labels):\n",
    "    db = core.C.create_db(db_type, db_name, core.C.Mode.write)\n",
    "    transaction = db.new_transaction()\n",
    "    for i in range(features.shape[0]):\n",
    "        feature_and_label = caffe2_pb2.TensorProtos()\n",
    "        feature_and_label.protos.extend([\n",
    "            utils.NumpyArrayToCaffe2Tensor(features[i]),\n",
    "            utils.NumpyArrayToCaffe2Tensor(labels[i])])\n",
    "        transaction.put(\n",
    "            'train_%03d'.format(i),\n",
    "            feature_and_label.SerializeToString())\n",
    "    # Close the transaction, and then close the db.\n",
    "    del transaction\n",
    "    del db\n",
    "\n",
    "write_db(\"minidb\", \"iris_train.minidb\", train_features, train_labels)\n",
    "write_db(\"minidb\", \"iris_test.minidb\", test_features, test_labels)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now, let's create a very simple network that only consists of one single TensorProtosDBInput operator, to showcase how we load data from the DB that we created. For training, you might want to do something more complex: creating a network, train it, get the model, and run the prediction service. To this end you can look at the MNIST tutorial for details."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The net looks like this:\n",
      "name: \"example_reader\"\n",
      "op {\n",
      "  output: \"dbreader\"\n",
      "  name: \"\"\n",
      "  type: \"CreateDB\"\n",
      "  arg {\n",
      "    name: \"db_type\"\n",
      "    s: \"minidb\"\n",
      "  }\n",
      "  arg {\n",
      "    name: \"db\"\n",
      "    s: \"iris_train.minidb\"\n",
      "  }\n",
      "}\n",
      "op {\n",
      "  input: \"dbreader\"\n",
      "  output: \"X\"\n",
      "  output: \"Y\"\n",
      "  name: \"\"\n",
      "  type: \"TensorProtosDBInput\"\n",
      "  arg {\n",
      "    name: \"batch_size\"\n",
      "    i: 16\n",
      "  }\n",
      "}\n",
      "\n"
     ]
    }
   ],
   "source": [
    "net_proto = core.Net(\"example_reader\")\n",
    "dbreader = net_proto.CreateDB([], \"dbreader\", db=\"iris_train.minidb\", db_type=\"minidb\")\n",
    "net_proto.TensorProtosDBInput([dbreader], [\"X\", \"Y\"], batch_size=16)\n",
    "\n",
    "print(\"The net looks like this:\")\n",
    "print(str(net_proto.Proto()))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "True"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "workspace.CreateNet(net_proto)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The first batch of feature is:\n",
      "[[5.5 2.5 4.  1.3]\n",
      " [5.2 4.1 1.5 0.1]\n",
      " [7.3 2.9 6.3 1.8]\n",
      " [6.5 3.2 5.1 2. ]\n",
      " [4.9 2.4 3.3 1. ]\n",
      " [5.7 2.6 3.5 1. ]\n",
      " [5.6 2.5 3.9 1.1]\n",
      " [5.2 3.5 1.5 0.2]\n",
      " [4.8 3.  1.4 0.3]\n",
      " [5.8 2.7 5.1 1.9]\n",
      " [7.2 3.2 6.  1.8]\n",
      " [6.8 2.8 4.8 1.4]\n",
      " [6.6 3.  4.4 1.4]\n",
      " [7.6 3.  6.6 2.1]\n",
      " [5.9 3.2 4.8 1.8]\n",
      " [5.5 3.5 1.3 0.2]]\n",
      "The first batch of label is:\n",
      "[1 0 2 2 1 1 1 0 0 2 2 1 1 2 1 0]\n",
      "The second batch of feature is:\n",
      "[[6.  3.4 4.5 1.6]\n",
      " [6.7 3.3 5.7 2.1]\n",
      " [5.8 2.7 4.1 1. ]\n",
      " [5.  3.5 1.3 0.3]\n",
      " [5.8 4.  1.2 0.2]\n",
      " [5.  3.4 1.5 0.2]\n",
      " [5.9 3.  4.2 1.5]\n",
      " [4.8 3.4 1.6 0.2]\n",
      " [6.7 3.  5.2 2.3]\n",
      " [5.5 2.3 4.  1.3]\n",
      " [4.8 3.1 1.6 0.2]\n",
      " [6.3 2.8 5.1 1.5]\n",
      " [5.  3.5 1.6 0.6]\n",
      " [5.8 2.7 3.9 1.2]\n",
      " [7.2 3.6 6.1 2.5]\n",
      " [4.3 3.  1.1 0.1]]\n",
      "The second batch of label is:\n",
      "[1 2 1 0 0 0 1 0 2 1 0 2 0 1 2 0]\n"
     ]
    }
   ],
   "source": [
    "# Let's run it to get batches of features.\n",
    "workspace.RunNet(net_proto.Proto().name)\n",
    "print(\"The first batch of feature is:\")\n",
    "print(workspace.FetchBlob(\"X\"))\n",
    "print(\"The first batch of label is:\")\n",
    "print(workspace.FetchBlob(\"Y\"))\n",
    "\n",
    "# Let's run again.\n",
    "workspace.RunNet(net_proto.Proto().name)\n",
    "print(\"The second batch of feature is:\")\n",
    "print(workspace.FetchBlob(\"X\"))\n",
    "print(\"The second batch of label is:\")\n",
    "print(workspace.FetchBlob(\"Y\"))"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 2",
   "language": "python",
   "name": "python2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.14"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}