Notebooks/Linear_Algebra/07_Eigendecomposition.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<h1 style=\"color:rgb(0,120,170)\">Neural Networks and Deep Learning</h1>\n",
    "<h2 style=\"color:rgb(0,120,170)\">Linear Algebra Basics</h2>\n",
    "\n",
    "[Source](https://hadrienj.github.io/posts/Deep-Learning-Book-Series-Introduction/)     \n",
    "[Github](https://github.com/hadrienj/deepLearningBook-Notes)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "import seaborn as sns"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Plot style\n",
    "sns.set()\n",
    "%matplotlib inline\n",
    "plt.rcParams['figure.figsize'] = (4, 4)\n",
    "# Avoid inaccurate floating values (for inverse matrices in dot product for instance)\n",
    "# See https://stackoverflow.com/questions/24537791/numpy-matrix-inversion-rounding-errors\n",
    "np.set_printoptions(suppress=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<style>\n",
       ".pquote {\n",
       "  text-align: left;\n",
       "  margin: 40px 0 40px auto;\n",
       "  width: 70%;\n",
       "  font-size: 1.5em;\n",
       "  font-style: italic;\n",
       "  display: block;\n",
       "  line-height: 1.3em;\n",
       "  color: #5a75a7;\n",
       "  font-weight: 600;\n",
       "  border-left: 5px solid rgba(90, 117, 167, .1);\n",
       "  padding-left: 6px;\n",
       "}\n",
       ".notes {\n",
       "  font-style: italic;\n",
       "  display: block;\n",
       "  margin: 40px 10%;\n",
       "}\n",
       "img + em {\n",
       "  text-align: center;\n",
       "  display: block;\n",
       "  color: gray;\n",
       "  font-size: 0.9em;\n",
       "  font-weight: 600;\n",
       "}\n",
       "</style>\n"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "%%html\n",
    "<style>\n",
    ".pquote {\n",
    "  text-align: left;\n",
    "  margin: 40px 0 40px auto;\n",
    "  width: 70%;\n",
    "  font-size: 1.5em;\n",
    "  font-style: italic;\n",
    "  display: block;\n",
    "  line-height: 1.3em;\n",
    "  color: #5a75a7;\n",
    "  font-weight: 600;\n",
    "  border-left: 5px solid rgba(90, 117, 167, .1);\n",
    "  padding-left: 6px;\n",
    "}\n",
    ".notes {\n",
    "  font-style: italic;\n",
    "  display: block;\n",
    "  margin: 40px 10%;\n",
    "}\n",
    "img + em {\n",
    "  text-align: center;\n",
    "  display: block;\n",
    "  color: gray;\n",
    "  font-size: 0.9em;\n",
    "  font-weight: 600;\n",
    "}\n",
    "</style>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "def plotVectors(vecs, cols, alpha=1):\n",
    "    \"\"\"\n",
    "    Plot set of vectors.\n",
    "\n",
    "    Parameters\n",
    "    ----------\n",
    "    vecs : array-like\n",
    "        Coordinates of the vectors to plot. Each vectors is in an array. For\n",
    "        instance: [[1, 3], [2, 2]] can be used to plot 2 vectors.\n",
    "    cols : array-like\n",
    "        Colors of the vectors. For instance: ['red', 'blue'] will display the\n",
    "        first vector in red and the second in blue.\n",
    "    alpha : float\n",
    "        Opacity of vectors\n",
    "\n",
    "    Returns:\n",
    "\n",
    "    fig : instance of matplotlib.figure.Figure\n",
    "        The figure of the vectors\n",
    "    \"\"\"\n",
    "    plt.axvline(x=0, color='#A9A9A9', zorder=0)\n",
    "    plt.axhline(y=0, color='#A9A9A9', zorder=0)\n",
    "\n",
    "    for i in range(len(vecs)):\n",
    "        if (isinstance(alpha, list)):\n",
    "            alpha_i = alpha[i]\n",
    "        else:\n",
    "            alpha_i = alpha\n",
    "        x = np.concatenate([[0,0],vecs[i]])\n",
    "        plt.quiver([x[0]],\n",
    "                   [x[1]],\n",
    "                   [x[2]],\n",
    "                   [x[3]],\n",
    "                   angles='xy', scale_units='xy', scale=1, color=cols[i],\n",
    "                  alpha=alpha_i)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "$$\n",
    "\\newcommand\\bs[1]{\\boldsymbol{#1}}\n",
    "\\newcommand\\norm[1]{\\left\\lVert#1\\right\\rVert}\n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<span class='notes'>\n",
    "    This content is part of a series following the chapter 2 on linear algebra from the [Deep Learning Book](http://www.deeplearningbook.org/) by Goodfellow, I., Bengio, Y., and Courville, A. (2016). It aims to provide intuitions/drawings/python code on mathematical theories and is constructed as my understanding of these concepts. You can check the syllabus in the [introduction post](https://hadrienj.github.io/posts/Deep-Learning-Book-Series-Introduction/).\n",
    "</span>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Introduction\n",
    "\n",
    "We will see some major concepts of linear algebra in this chapter. It is also quite heavy so hang on! We will start with getting some ideas on eigenvectors and eigenvalues. We will develop on the idea that a matrix can be seen as a linear transformation and that applying a matrix on its eigenvectors gives new vectors with the same direction. Then we will see how to express quadratic equations into the matrix form. We will see that the eigendecomposition of the matrix corresponding to a quadratic equation can be used to find the minimum and maximum of this function. As a bonus, we will also see how to visualize linear transformations in Python!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 2.7 Eigendecomposition"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The eigendecomposition is one form of matrix decomposition. Decomposing a matrix means that we want to find a product of matrices that is equal to the initial matrix. In the case of the eigendecomposition, we decompose the initial matrix into the product of its eigenvectors and eigenvalues. Before all, let's see what are eigenvectors and eigenvalues.\n",
    "\n",
    "# Matrices as linear transformations\n",
    "\n",
    "As we have seen in [2.3](https://hadrienj.github.io/posts/Deep-Learning-Book-Series-2.3-Identity-and-Inverse-Matrices/) with the example of the identity matrix, you can think of matrices as linear transformations. Some matrices will rotate your space, others will rescale it etc. So when we apply a matrix to a vector, we end up with a transformed version of the vector. When we say that we 'apply' the matrix to the vector it means that we calculate the dot product of the matrix with the vector. We will start with a basic example of this kind of transformation.\n",
    "\n",
    "### Example 1."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[-1,  3],\n",
       "       [ 2, -2]])"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "A = np.array([[-1, 3], [2, -2]])\n",
    "A"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[2],\n",
       "       [1]])"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "v = np.array([[2], [1]])\n",
    "v"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's plot this vector:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(-1.0, 4.0)"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAQoAAAD/CAYAAAAJ45jXAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8qNh9FAAAACXBIWXMAAAsTAAALEwEAmpwYAAAPOUlEQVR4nO3ca4xcZ32A8Wdmd8cOXhPcZRMSx8S58U+wA8WACFViILhbBARKIhWsQiLaVOBIXD6gCvqhUKTSCPKJkishkDolLS1VgopaooKoiNpYqLFVdhO/SXzJxUTJZu04tuPd8e5MP8w4mDTed2yf3TOzfn6SFe3Jmdd/RZNnz5nLW2k2m0jSbKplDyCp+xkKSVmGQlKWoZCUZSgkZRkKSVmFhyIivhwRzYhYXfTakspRaCgiYg1wCfB4ketKKldhoYiIRcCNwIai1pTUHfoLXOurwF0ppZ0RcSyPWwS8HXgamClwHkmz6wPOAH4JTM12YiGhiIh3Am8DvngcD3878Isi5pB0XC4D7p/thKKuKN4FXATsaF9NnAX8JCI+mVK6L/PYpwH27DlAo9Eb3zsZG9vC2rWXMjGxv+xROjY0NNhT84Izz7VqtcKyZUug/f/gbAoJRUrpeuD6wz9HxE7ggyml0Q4ePgPQaDR7JhSTk5MAPTPvYb02LzjzPMne8vs5CklZRb6Y+ZKU0sq5WFdSObyikJRlKCRlGQpJWYZCUpahkJRlKCRlGQpJWYZCUpahkJRlKCRlGQpJWYZCUpahkJRlKCRlGQpJWYZCUpahkJRlKCRlGQpJWYZCUpahkJRlKCRlGQpJWYZCUpahkJRlKCRlGQpJWYZCUpahkJRlKCRlGQpJWYZCUpahkJRlKCRlGQpJWf1FLRQR9wDnAA1gP/CZlNKWotaXVJ7CQgFck1LaCxARHwbuANYUuL6kkhR263E4Em2n0rqykLQAVJrNZmGLRcTtwAhQAd6XUhrr4GErgR2FDSHpWJ0D7JzthCJvPUgpXQsQEZ8AvgG8v9PHTkzsp9EoLlpzafPmTYyMrGN8fF/Zo3RseHhpT80LzjzXqtUKQ0ODnZ07FwOklDYC74mIoblYX9L8KiQUETEYESuO+PkKYHf7j6QeV9StxxLgnyJiCTBDKxBXpJR6415C0qwKCUVK6RngkiLWktR9/GSmpCxDISnLUEjKMhSSsgyFpCxDISnLUEjKMhSSsgyFpCxDISnLUEjKMhSSsgyFpCxDISnLUEjKMhSSsgyFpCxDISnLUEjKMhSSsgyFpCxDISnLUEjKMhSSsgyFpCxDISnLUEjKMhSSsgyFpCxDISnLUEjKMhSSsgyFpCxDISmrv4hFImII2AicB9SBR4FPpZTGi1hfUrmKuqJoAl9PKUVK6WJgG3B9QWtLKlkhVxQppd3Az4849ACwoYi1JZWv8NcoIqJKKxI/KnptSeWoNJvNQheMiBuB5cCVKaVGBw9ZCewodAhJx+IcYOdsJxRy63FYRNwAXABc0WEkXjIxsZ9Go9hozZXNmzcxMrKO8fF9ZY/SseHhpT01LzjzXKtWKwwNDXZ0bmGhiIivAW8FPpBSmipqXUnlK+rt0VXAl4BHgP+KCIAdKaWPFLG+pHIV9a7HGFApYi1J3cdPZkrKMhSSsgyFpCxDISnLUEjKMhSSsgyFpCxDISnLUEjKMhSSsgyFpCxDISnLUEjKMhSSsgyFpCxDISnLUEjKMhSSsgyFpCxDISnLUEjKMhSSsgyFpCxDISnLUEjKMhSSsgyFpCxDISnLUEjKMhSSsgyFpCxDISnLUEjKMhSSsgyFpKz+ohaKiBuAq4CVwMUppdGi1pZUriKvKO4B1gKPF7impC5Q2BVFSul+gIgoaklJXaKwUJyooaHBskfo2MjIOgCGh5eWPMmx6bV5wZm7RdeEYmJiP41Gs+wxOrJ58yZGRtYxPr6v7FE6Njy8tGvnbTRh14EK2/ZWeWxvlW0v9LHrQIUvX9bPebXunPlouvm/88tVq5WOf0F3TSh08vm3J/q5bazG9heqTM5UXjr+6oEmt7z7IJcs72d8vMQB9RLfHlVp3rt8msGB5m9FYnhxg43rXuStwzMlTqaXK/Lt0W8CVwKvA/4jIiZSSquKWl8LR30G/mX7ALc9VGPXgd/8rlox2OC7l7/IisHeuAU9mRT5rsdngc8WtZ4Wnslp+MG2AW5/qMYzB6u8ptbksxdPMbq7yq4DVW5/z0FOO8VIdCNfo9CcO3AI/uGxAe54uMZzk1WGFjf4wu9Osf6COoMDcPejA3zg7EleXSt7Uh2NodCc2VeHux6p8b00wPNTVYZPafClNZN89PxDnHLEM2/9BYfKG1IdMRQq3PNTcGeqsTHV2HeowpmvavC5t09y1bmHWNRX9nQ6HoZChZmYrPDdrQP8/SM1Xpyu8PrBBl9cM8mHVk5TMxA9zVDohD3zYoXvPFzjHx8bYHKmwrmvnuHTq+p84Oxp+n0DfkEwFDpuuw5U+PZDNf552wCHGhXiNTNsWF1n5Kxp+gzEgmIodMye2Ffh1odq3LN9gOlmhdW/0wrE5cunqVbyj1fvMRTq2La9VW4Zq/Gvj/fTaFZ4y2tnuG71FJedMUPFQCxohkJZW/e0AvHvT/TTpMI7Tp9mw6o6l5xuIE4WhkJH9auJKjeP1fjpUwMAXHbGNBtW1/0exknIUOj/eXC8yk2ji/jF062nx+XLD7FhdZ03DTVKnkxlMRQCoNmETc/2cfNojQee6adCk/e9/hAbVtW5cJmBONkZipNcswn3P93HTWM1Hhzvp1ppcsXKQ3x6VZ3zTzUQajEUJ6lmE362q4+bRxfxq9199FeaXHVunU+tqnP2Ur/Bqd9mKE4yjSbc92Q/N4/W2Pp8HwPVJh87v86fvbHOWe4DoaMwFCeJ6Qb8aEc/t4zV2PZCH4v6mlwdda69qM7przIQmp2hWOAONeDeHf1858ewfe8pvKq/ybUXTfHJCw/xWjeJUYcMxQJVn4Efbh/g2+3t5pbWYMOqKa65sM6yRWVPp15jKBaYg9Pwg8cGuP3hGs+2t5v73Jum+PzvLaL+Qr3s8dSjDMUCceBQa0u5O7bWmHiF7eZOXbQId77X8TIUPW5fHTY+UuPOrTWer1c47ZQGf7Fmkj962XZz0onwqdSj9kzBnVtr3PVIa7u55UsafP7NU1zpdnOaA4aixzx3sLXd3Pcf/e3t5j58zjQDbhajOWIoesQzL1a4/eEaP2hvN3dee7u597vdnOaBoehyuw5UuG2sxg+3t7abu/DwdnMr3E1K88dQdKnH91W4dazGvTt+s93cdaunuHy5m8Vo/hmKLrNtb2uzmB+3t5tbMzzNdavqXOp2cyqRoegSW/dUuWm0xn1P/ma7uetW13nHaQZC5TMUJfvfiSo3j9b42S63m1P3MhQl+Z/xPm4arXF/e7u5957V2izG7ebUjQzFPGo24YFn+rh5rMYmt5tTDzEU86DZhF883cdNo4vY/Fwf1UqTD7W3mzvP7ebUAwzFHGo24ae7WrtJjbrdnHqYoZgDMw2476lWIFJ7u7n1F7S2m1u+xECo9xQWioh4A3AnMARMAFenlB4tav1eMN2AHz/e2m5u+wt9LO5rck3U+VO3m1OPK/KK4hbgxpTSXRHxceBW4PIC1+9a9Rm4d+cAt43VeGJ/1e3mtOAUEoqIOA1YA/x++9DdwLciYjiltGD3S5k6vN3cWI1fv1hl6UCT61ZPcXW43ZwWlqKuKFYAu1JKMwAppZmI+HX7eEehGBvbwuTkZEHjzK19+17g70bhr/5zgD3Tixnsq/PR4ScYWfYUS2am2fkQ7Cx7yJcZGVnH5s2byh7jmDjz3Fq8eDFr117a0bld82JmpwN3i7/5b+gfWMxXLoE/ubjGYO184Pyyx5rVyMi6skc4Zs7cHYoKxZPA8ojoa19N9AFnto93ZGJiP41Gb9zPb968ic+8ex1/vHIfp/TDwb1wsOyhMoaHlzI+vq/sMY6JM8+tarXC0NBgZ+cW8RemlJ4FtgDr24fWA5sX8usTgzXck1InjSKf6p8G7oyIvwT2AFcXuLakEhUWipTSVuAdRa0nqXu426KkLEMhKctQSMoyFJKyDIWkLEMhKctQSMoyFJKyDIWkLEMhKctQSMoyFJKyDIWkLEMhKctQSMoyFJKyDIWkLEMhKctQSMoyFJKyDIWkLEMhKctQSMoyFJKyDIWkLEMhKctQSMoyFJKyDIWkLEMhKctQSMoyFJKyDIWkLEMhKctQSMrqP9EFIuLjwJ8DbwQ+n1L61glPJamrFHFFsQX4GPD9AtaS1IVO+IoipTQKEBGN41yiD6BarZzoKPNm8eLFQG/NDL03LzjzXDpizr7cuSccigKcAbBs2ZKy5+jY2rWXAjA0NFjyJMem1+YFZ54nZwDbZjshG4qIeBB4/VH+9ekppZnjGOxIvwQuA54GTnQtSZ3roxWJX+ZOzIYipbSmiIlmMQXcP8d/h6RXNuuVxGG+PSopq9JsNk9ogYhYD3wDWAbUgQPASErpoRMfT1I3OOFQSFr4vPWQlGUoJGUZCklZhkJSVjd8MhPonS+XRcQbgDuBIWACuDql9Gi5Ux1dRNwAXAWsBC4+/JH7bhURQ8BG4Dxa76I9CnwqpTRe6mAZEXEPcA7QAPYDn0kpbSlzpk5ExJeBr5B5bnTTFcUWeuPLZbcAN6aU3gDcCNxa8jw59wBrgcdLnqNTTeDrKaVIKV1M6wNB15c8UyeuSSm9OaX0FuAG4I6yB8qJiDXAJXTw3OiaUKSURtufvTjeL5fNuYg4DVgD3N0+dDewJiKGy5tqdiml+1NKT5Y9R6dSSrtTSj8/4tADwNkljdOxlNLeI348lS5+HgNExCJav+g2dHJ+14SiR6wAdh3+fkv7n79uH1fBIqJK64n8o7Jn6URE3B4RTwB/DVxT9jwZXwXuSint7OTkeXuNYh6+XKaF529p3e935etVL5dSuhYgIj5B69PK7y93olcWEe8E3gZ8sdPHzFso5uHLZfPhSWB5RPSllGYiog84s31cBWq/CHsBcEVKqasv418upbQxIm6LiKGU0kTZ87yCdwEXATsiAuAs4CcR8cmU0n2v9ABvPY5BSulZWi+6rm8fWg9s7vZX5HtNRHwNeCvwhymlqbLnyYmIwYhYccTPVwC723+6Tkrp+pTSmSmllSmllcBTwB8cLRLQRd/16JUvl0XEhbTeHl0G7KH19mgqd6qji4hvAlcCrwOeAyZSSqvKneroImIVMAo8AhxsH96RUvpIeVPNLiJOB+4FltDaU2U38IWU0oOlDtahiNgJfHC2t0e7JhSSupe3HpKyDIWkLEMhKctQSMoyFJKyDIWkLEMhKctQSMr6P1MPbOb7WMpuAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 288x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plotVectors([v.flatten()], cols=['#1190FF'])\n",
    "plt.ylim(-1, 4)\n",
    "plt.xlim(-1, 4)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now, we will apply the matrix $\\bs{A}$ to this vector and plot the old vector (light blue) and the new one (orange):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[1]\n",
      " [2]]\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "(-1.0, 4.0)"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAQoAAAD/CAYAAAAJ45jXAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8qNh9FAAAACXBIWXMAAAsTAAALEwEAmpwYAAAUmElEQVR4nO3de3Bc5X2H8efsTVfbGFk2vuELtl8Z2yExoZAEnASoknLJBdqCOwkMLQnYSbikmZR0Okknk2aYhEDLxdwcCDENDYUUaGkISWhanAwuLXYbCeu1Lcs2vsSW5ZuklXa1u6d/rAyGYr1r6+ye3dX3M+NhtD776jf28vjs2d1Xnu/7iIiMJBL2ACJS/hQKEXFSKETESaEQESeFQkScFAoRcQo8FMaYbxhjfGPM4qDXFpFwBBoKY8xS4Dxge5Driki4AguFMaYGuA9YEdSaIlIeYgGu9U3gcWvtNmPMidyvBjgH2ANkA5xHREYWBaYCrwKpkQ4MJBTGmA8A7wduO4m7nwO8HMQcInJSLgDWjnRAUGcUHwYWAl3DZxMzgJ8ZY66z1r7ouO8egIMH+8nlKuNzJ+3tG1i27Hx6evrCHqVgTU2NFTUvaOZii0Q8Jk5sgOH/B0cSSCistbcDtx/92hizDbjMWttWwN2zALmcXzGhGBwcBKiYeY+qtHlBM5eI8ym/3kchIk5BXsx8k7V2djHWFZFw6IxCRJwUChFxUihExEmhEBEnhUJEnBQKEXFSKETESaEQESeFQkScFAoRcVIoRMRJoRARJ4VCRJwUChFxUihExEmhEBEnhUJEnBQKEXFSKETESaEQESeFQkScFAoRcVIoRMRJoRARJ4VCRJwUChFxUihExEmhEBEnhUJEnBQKEXFSKETESaEQESeFQkScFAoRcVIoRMQpFtRCxphngDlADugDvmSt3RDU+iISnsBCAVxrrT0MYIz5JPAIsDTA9UUkJIE99TgaiWETyJ9ZiEgV8HzfD2wxY8xqoBXwgI9ba9sLuNtsoCuwIUTkRM0Bto10QJBPPbDWXg9gjPks8F3gkkLv29PTRy4XXLSKaf36dbS2Xkx3d2/YoxSsuXlcRc0LmrnYIhGPpqbGwo4txgDW2jXAR40xTcVYX0RKK5BQGGMajTEzj/n6cuDA8C8RqXBBPfVoAP7RGNMAZMkH4nJrbWU8lxCREQUSCmvtXuC8INYSkfKjd2aKiJNCISJOCoWIOCkUIuKkUIiIk0IhIk4KhYg4KRQi4qRQiIiTQiEiTgqFiDgpFCLipFCIiJNCISJOCoWIOCkUIuKkUIiIk0IhIk4KhYg4KRQi4qRQiIiTQiEiTgqFiDgpFCLipFCIiJNCISJOCoWIOCkUIuKkUIiIk0IhIk4KhYg4KRQi4qRQiIiTQiEiTrEgFjHGNAFrgDOANLAZuMFa2x3E+iISrqDOKHzgO9ZaY61dAnQCtwe0toiELJAzCmvtAeBXx9z0CrAiiLVFJHyBX6MwxkTIR+K5oNcWkXB4vu8HuqAx5j5gOnCFtTZXwF1mA12BDiH/X7oXIlGI1Yc9iZSfOcC2kQ4I5KnHUcaYO4D5wOUFRuJNPT195HLBRqtY1q9fR2vrxXR394Y9SmEySZo33ET32Q+BVyEzA83N4yrnz3hYJc0ciXg0NTUWdGxgoTDGfBs4G7jUWpsKal0ZpWyKCes+D14aPL0aLicnkEeOMWYR8DVgGvAbY8wGY8w/BbG2jEIuzfj/XEmi+9fQtCjsaaSCBfWqRzvgBbGWBCSXYfx/3UzN3pfyXzctDnceqWg6F61GfpZxr32Fmt0vvHXbqQqFnDyFogrVbf0hsYP/8+bXPh6cujDEiaTSKRRVaOCM6xhYkH+/W65mEtnG2RBvCHcoqWgKRTXKpanvuJtcfAIHP/IcqWmXhD2RVDiFogrVbn+S6MAukvM/T65uKsmFXw57JKlwCkW1yQ5Sb+8ll2hiYO61+dv0/gkZJT2Cqkxd14+IDu4lOf8GiOm6hARDoagmmST1m1eRrZ3MwJzPhD2NVBGFoorUdf2QSKqH5IKVEKsLexypIgpFlfCGeqnf9CDZumkMzro67HGkyigUVaKu81EiQ4dImi9CtCbscaTKKBRVwEsfpm7LarL1pzN4+h+GPY5UIYWiCtRteZhIppf+lpsgEg97HKlCCkWF81I91G39AZnGuaRmfirscaRKKRQVrn7zg0Qy/SRbbgEvGvY4UqUUigoWGdxHXdcaMuMNqemXhj2OVDGFooLVbVqFlx2kv+VWvU1bikqPrgoVSe6mbtsTDE1YTHpqa9jjSJVTKCpU/ab78HLp/CdDPe1CKMWlUFSgSP8Oarc/ydDE95Ge8pGwx5ExQKGoQA32Hjw/Q/+Zf66zCSkJhaLCRHs7qdnxE9JN5zI06YNhjyNjhEJRYert3XjkdG1CSkqhqCDRI5aanf9MevIFDE36vbDHkTFEoaggDRv/Fg+ffu2BKSWmUFSI2KE2ava8QOq0i8hMfG/Y48gYo1BUiPqNdwHQ36KzCSk9haICxA6sp2bvS6Sm/QHZU84MexwZgxSKCtCw8U58PPpbbgl7FBmjFIoyF9+/jkT3WlIzPkF2/IKwx5ExSqEoZ75P/cbv4XtRki03hT2NjGEKRRmLd68l0fMqgzOvINs4N+xxZAxTKMqV7+evTXgxkuZLYU8jY5xCUaYSe/+N+MENDM66ilzDzLDHkTFOoShHvk/9xjvxIwmS5gthTyNCLKiFjDF3AFcCs4El1tq2oNYeaxJ7fkb8cDvJudeRq5sa9jgigZ5RPAMsA7YHuObY42dp2HgXfrSW5IIVYU8jAgR4RmGtXQtgjAlqyTGpZtfzxHo3kZz3efza5rDHEQHA830/0AWNMduAy07gqcdsoCvQISpVLgNPfQiSv4OrX4PaprAnkrFhDrBtpAMCO6MYrZ6ePnK5YKNVLOvXr6O19WK6u3sDXbdm+1OMP9xJ/4IvkuxNQG9w6zc3jwt83qDkfNjV79F5OMKWwxE6j0TZ1e/xjQtinJEoz5mPp5z/nN8pEvFoamos6NiyCcWYl0vTYO8mFx/PwLzPhT1NSfx0R4yH2hNsPRJhMPvWbl3j4z4PfGSA86bH6O4OcUB5k14eLRO1258imnyDgXnX4yfGhz1OSVw0PUNj3H9bJJprc6y5OMnZzdkQJ5N3CvLl0buBK4DTgF8YY3qstYuCWr+qZVPUb7qXXGIiA3OvC3uaoktn4Sdb4zz0eoJd/W/9WzWzMcejFyaZ2VgZT0HHkiBf9bgJ0CeXTkLttieIDuyhb9Ft+PHCnjNWosEMPNkZZ/XrCfYORDgl4XPTkhRtByLs6o+w+qMDTK5TJMqRrlGELTNAw6b7yNVMYmDONWFPUxT9Q/APW+I8sjHB/sEITbU5vvLeFMvnp2mMwxOb41w6a5DxibAnleNRKEJW17WGSGo/fUu+DrG6sMcJVG8aHt+U4Ac2zqFUhOa6HF9bOshV84aoO+aRt3z+UHhDSkEUihB5Q33Ub36AbN1UBmb/SdjjBOZQCh6zCdbYBL1DHtPqc9x8ziBXzh2iJhr2dHIyFIoQ1W39AZH0QXrP+hZEa8IeZ9R6Bj0e7Yjz95sSJDMepzfmuG3pIJ+YnSGhQFQ0hSIkXvoIdVseJls/g8FZfxT2OKOyN+nx/Y0JfrwlzmDWY+74LDcuSnPprAwxvQBfFRSKkNR1riYydIQji/8KIpV5FW9Xv8fDryd4qjPOUM7DnJJlxeI0rTMyRBWIqqJQhMBLHaCu81EyDbNJzfx02OOcsB29Hg++nuCZrXEyvsfiU/OBuHB6hoh+HGpVUihCUL/lISKZPvrO+hZEKuevoPNwhAfaE/zL9hg53+N9k7KsXJzigqlZ/bzkKlc5j9Iq4Q12U7f1h2TGzSc147KwxylIx8F8IF7YEcPH49wpGVYsSnPeFAVirFAoSqx+8wN42QH6W24Fr7xfCvhtT4T72xP8cmccgAumZlixOK3PYYxBCkUJRQb2UNf1OEMTziQ97WNhj3Ncr3VHWNVWw8t78g+PC6cPsWJxmvc05UKeTMKiUJRQ/aZVeLk0yZZbwSuvlwV8H9bti3J/W4JX9sbw8Pn46UOsWJSmZaICMdYpFCUS6d9J7bYfMzTxLNKnXRT2OG/yfVi7J8qq9gSvdceIeD6Xzx7ixkVp5k1QICRPoSiRensPnj9E/8IvUw5XAH0fXtoV5f62Gn57IErM87lybpobFqWZNU6f4JS3UyhKINrXRe0bT5NuOoeh5gtCnSXnw4tvxLi/LUHHoSjxiM/V89J87sw0M7QPhByHQlEC9R1/h+dnSYZ4NpHJwXNdMR5oT9B5JEpN1Ocak+b6hWmm1CsQMjKFosiiRzZTs/M50s0fYmjSeSX//kM5eLYrxvefh62H66iP+Vy/MMV1LUNM0iYxUiCFosgaOu7Cw89fmyihdBae3hrn4eHt5sYlYMWiFNe2pJlY+R9UlRJTKIooeuh1anb/lNSUj5I5dWlJvudABp7cEmf1xgT7hrebu/k9KW75YA3pI+mSzCDVR6EoooaOuwBILry16N+rfyi/pdwjHQl63mW7uQk1NWjnezlZCkWRxA5uoOZ3vyA19WNkTllStO/Tm4Y1mxI81pHgUNpjcl2Ov1w6yB+/Y7s5kdHQQ6lIGjbehY9Hf8stRVn/YAoe60jw+Kb8dnPTG3LcclaKK7TdnBSBQlEEsZ5XSez7DwanX052Qkuga+8fyG8396PNb99u7pNzMsTL613hUkUUiiJo2HgnPhGSLTcHtubepMfqjQmeHN5u7ozh7eYu0XZzUgIKRcDi3b8hsf8VBmdeQXbcGaNeb1e/x0PtCZ7emt9uruXodnMztZuUlI5CESTfp2Hj9/C9GP0to/uhadt7PR5sT/Bs11vbza1cnOLC6dosRkpPoQhQYt+/Ez/wGgOzribXMOuk1ug8nN8s5vnh7eaWNmdYuSjN+dpuTkKkUATF96nfeCd+JEHSfPGE795xMMKqtgQvvvHWdnMrF6c5d7ICIeFTKAKS2PNz4od+y8Cca8jVTy/4fv/bE+H+tgQv7dJ2c1K+FIog+DkaOu7Ej9SQNF8o6C7/3R1lVVuCtcPbzV00I79ZjLabk3KkUASgZte/EjtiSc67nlzt5OMe5/vwyt4o97cnWKft5qSCKBSjlctQ33EXfrSe5Pwb3/UQ34eX90RZ1VbD+v1RIp7PJ4a3mztD281JBVAoRqlm57PE+rbSv2Alfk3T237P9+GXu/K7SbVpuzmpYArFaOSGaOi4m1xsHAPzPvfmzdkcvLgzHwg7vN3c8vn57eamNygQUnkCC4UxZgHwGNAE9ADXWGs3B7V+Oard8TTR5A76W27BT5xCJgfPb89vN7f1SJTaqM+1Js2fabs5qXBBnlE8ANxnrX3cGPMZ4EHgwgDXLy/ZFPX2HnLxUzg8+zqe6YzzUHuCHX0RbTcnVSeQUBhjJgNLgd8fvukJ4F5jTLO1tjr3S+l4nOjAbn49+TZueGEqu5MRxsV9Vi5OcY3RdnNSXYI6o5gJ7LLWZgGstVljzO7h2wsKRXv7BgYHBwMap7j6j+wnuftOkv4krtq2gkg0w1XNO2iduJOGbIZtr8O2sId8h9bWi1m/fl3YY5wQzVxctbW1LFt2fkHHls3FzEIHLgtD/bz8/HZ+vm8KXz2/gT9dAo2JecC8sCcbUWvrxWGPcMI0c3kIKhRvANONMdHhs4koMG349oL09PSRy1XG8/n169fxwUv+glk9vdTFehk4DANhD+XQ3DyO7u7esMc4IZq5uCIRj6amxsKODeIbWmv3ARuA5cM3LQfWV+31CaAxgfaklDEjyIf6jcBjxpivAweBawJcW0RCFFgorLUdwLlBrSci5UO7LYqIk0IhIk4KhYg4KRQi4qRQiIiTQiEiTgqFiDgpFCLipFCIiJNCISJOCoWIOCkUIuKkUIiIk0IhIk4KhYg4KRQi4qRQiIiTQiEiTgqFiDgpFCLipFCIiJNCISJOCoWIOCkUIuKkUIiIk0IhIk4KhYg4KRQi4qRQiIiTQiEiTgqFiDgpFCLipFCIiJNCISJOCoWIOMVGu4Ax5jPAV4EzgVustfeOeioRKStBnFFsAK4GfhTAWiJShkZ9RmGtbQMwxuROcokoQCTijXaUkqmtrQUqa2aovHlBMxfTMXNGXceOOhQBmAowcWJD2HMUbNmy8wFoamoMeZITU2nzgmYukalA50gHOENhjHkNOP04vz3FWps9icGO9SpwAbAHGO1aIlK4KPlIvOo60BkKa+3SICYaQQpYW+TvISLvbsQziaP08qiIOHm+749qAWPMcuC7wEQgDfQDrdba10c/noiUg1GHQkSqn556iIiTQiEiTgqFiDgpFCLiVA7vzAQq58NlxpgFwGNAE9ADXGOt3RzuVMdnjLkDuBKYDSw5+pb7cmWMaQLWAGeQfxVtM3CDtbY71MEcjDHPAHOAHNAHfMlauyHMmQphjPkG8Nc4HhvldEaxgcr4cNkDwH3W2gXAfcCDIc/j8gywDNge8hyF8oHvWGuNtXYJ+TcE3R7yTIW41lp7lrX2fcAdwCNhD+RijFkKnEcBj42yCYW1tm34vRcn++GyojPGTAaWAk8M3/QEsNQY0xzeVCOz1q611r4R9hyFstYesNb+6pibXgFmhTROway1h4/5cgJl/DgGMMbUkP+HbkUhx5dNKCrETGDX0c+3DP939/DtEjBjTIT8A/m5sGcphDFmtTFmB/A3wLVhz+PwTeBxa+22Qg4u2TWKEny4TKrPPeSf75fl9ap3stZeD2CM+Sz5dytfEu5E784Y8wHg/cBthd6nZKEowYfLSuENYLoxJmqtzRpjosC04dslQMMXYecDl1try/o0/p2stWuMMQ8ZY5qstT1hz/MuPgwsBLqMMQAzgJ8ZY66z1r74bnfQU48TYK3dR/6i6/Lhm5YD68v9inylMcZ8Gzgb+JS1NhX2PC7GmEZjzMxjvr4cODD8q+xYa2+31k6z1s621s4GdgIfO14koIw+61EpHy4zxrSQf3l0InCQ/MujNtypjs8YczdwBXAasB/osdYuCneq4zPGLALagE3AwPDNXdbaT4c31ciMMVOAZ4EG8nuqHAC+Yq19LdTBCmSM2QZcNtLLo2UTChEpX3rqISJOCoWIOCkUIuKkUIiIk0IhIk4KhYg4KRQi4qRQiIjT/wHQPE5x+TamOQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 288x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "Av = A.dot(v)\n",
    "print(Av)\n",
    "plotVectors([v.flatten(), Av.flatten()], cols=['#1190FF', '#FF9A13'])\n",
    "plt.ylim(-1, 4)\n",
    "plt.xlim(-1, 4)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can see that applying the matrix $\\bs{A}$ has the effect of modifying the vector.\n",
    "\n",
    "Now that you can think of matrices as linear transformation recipes, let's see the case of a very special type of vector: the eigenvector."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Eigenvectors and eigenvalues\n",
    "\n",
    "We have seen an example of a vector transformed by a matrix. Now imagine that the transformation of the initial vector gives us a new vector that has the exact same direction. The scale can be different but the direction is the same. Applying the matrix didn't change the direction of the vector. This special vector is called an eigenvector of the matrix. We will see that finding the eigenvectors of a matrix can be very useful.\n",
    "\n",
    "<span class='pquote'>\n",
    "    Imagine that the transformation of the initial vector by the matrix gives a new vector with the exact same direction. This vector is called an eigenvector of $\\bs{A}$.\n",
    "</span>\n",
    "\n",
    "This means that $\\bs{v}$ is a eigenvector of $\\bs{A}$ if $\\bs{v}$ and $\\bs{Av}$ are in the same direction or to rephrase it if the vectors $\\bs{Av}$ and $\\bs{v}$ are parallel. The output vector is just a scaled version of the input vector. This scalling factor is $\\lambda$ which is called the **eigenvalue** of $\\bs{A}$.\n",
    "\n",
    "$$\n",
    "\\bs{Av} = \\lambda\\bs{v}\n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Example 2.\n",
    "\n",
    "Let's $\\bs{A}$ be the following matrix:\n",
    "\n",
    "$$\n",
    "\\bs{A}=\n",
    "\\begin{bmatrix}\n",
    "    5 & 1\\\\\\\\\n",
    "    3 & 3\n",
    "\\end{bmatrix}\n",
    "$$\n",
    "\n",
    "We know that one eigenvector of A is:\n",
    "\n",
    "$$\n",
    "\\bs{v}=\n",
    "\\begin{bmatrix}\n",
    "    1\\\\\\\\\n",
    "    1\n",
    "\\end{bmatrix}\n",
    "$$\n",
    "\n",
    "We can check that $\\bs{Av} = \\lambda\\bs{v}$:\n",
    "\n",
    "$$\n",
    "\\begin{bmatrix}\n",
    "    5 & 1\\\\\\\\\n",
    "    3 & 3\n",
    "\\end{bmatrix}\n",
    "\\begin{bmatrix}\n",
    "    1\\\\\\\\\n",
    "    1\n",
    "\\end{bmatrix}=\\begin{bmatrix}\n",
    "    6\\\\\\\\\n",
    "    6\n",
    "\\end{bmatrix}\n",
    "$$\n",
    "\n",
    "We can see that:\n",
    "\n",
    "$$\n",
    "6\\times \\begin{bmatrix}\n",
    "    1\\\\\\\\\n",
    "    1\n",
    "\\end{bmatrix} = \\begin{bmatrix}\n",
    "    6\\\\\\\\\n",
    "    6\n",
    "\\end{bmatrix}\n",
    "$$\n",
    "\n",
    "which means that $\\bs{v}$ is well an eigenvector of $\\bs{A}$. Also, the corresponding eigenvalue is $\\lambda=6$."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can represent $\\bs{v}$ and $\\bs{Av}$ to check if their directions are the same:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[5, 1],\n",
       "       [3, 3]])"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "A = np.array([[5, 1], [3, 3]])\n",
    "A"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[1],\n",
       "       [1]])"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "v = np.array([[1], [1]])\n",
    "v"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(-1.0, 7.0)"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAQcAAAD/CAYAAAD8HRNnAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8qNh9FAAAACXBIWXMAAAsTAAALEwEAmpwYAAAZLUlEQVR4nO3dd5iU5b3/8ffM7LSlSBGQpiDlVhHFFRSkF8k5/nKixjQvj4IdYxJj/EWOsaZoPMQWlaixIkY9vxMTE0/yi4k0KQoqIDU3vQhKWZrLlN2Zec4fsxiCIyzsM/PMznxe1+XFtbPsPd+H3X37TL19juMgInIov9cDiEhxUhxEJCfFQURyUhxEJCfFQURyUhxEJKcKNxYxxnQDXj/oolZAS2ttGzfWF5HCcyUO1toNQL8DHxtjHnFrbRHxhuu/wMaYEHAZ8CW31xaRwsnH/92/Amyx1i5s4N8PAwOAj4F0HuYRkdwCQEfgPSB56CfzEYergOeO4u8PAGbnYQ4RaZihwJxDL3Q1DsaYzsBw4PKj+LKPAXbv3k8mk//XeSxfvphhw4ZQXV2T9+vyStu2zUv2+Er52KCwx+f3+2jduhnU/w4eyu0zh3HAn6y11UfxNWmATMYpSBwSiQQHrq+UlfLxlfKxgSfHl/PmvNvPcxjP0d2kEJEi5eqZg7W2t5vriYh39AxJEclJcRCRnBQHEclJcRCRnBQHEclJcRCRnBQHEclJcRCRnBQHEclJcRCRnBQHEclJcRCRnBQHEclJcRCRnBQHEclJcRCRnBQHEclJcRCRnFx7mzhjTAR4GBgDJIB3rLXXubW+iBSWm+8hOYlsFHpbax1jTAcX1xaRAnNrI93mwBVAF2utA2Ct3ebG2iLiDbfOHHoA1cDdxpiRQA1wh7X2c7voiEjT4HOcxm+gYYypAj4ALrPWvmyMORd4A+hprd13hC/vBqxv9BAiTVwqA4u2wYCOBb/q7sCGQy9068xhE5ACXgGw1s43xuwEegPvN2SB6uqaguz0s2jRfMaOHcOOHZ/m/bq80q5di5I9vlI8towDb26u4NElIe4dHijY8fn9Ptq2bf7Fn3fjSqy1O4EZwPkAxpjeQHtgjRvri5Qix4FZWwJc8pdKvj8nSruow9juXk/1D24+WjEBeM4Y8yBQB1xurd3j4voiJWPBtgAPLwmxcMc/fgUnnpXE58vHxvfHxrVJrLXrgBFurSdSqv62uYI7F4TZnfzHifuF3ero0ybj4VSfp2dIihTYmC4pvt6j7rOPwwGHm89MejhRbsVzDiNSBhwHJi0O89zKEKe3SbOv1scFJ9XRsVn+74w/WoqDSIE4Dtz7QZipq0L0a5vmmZExXlsX5GsHnUUUE8VBpAAyDvz4vTCvrgnRv12Kp0bEaR6EcaYOn8/r6XJTHETyLJ2BOxeEeW1diIEdUjwxPE5l/W9esYYBFAeRvEpl4LZ3I/xxQ5AhHVNMHhon0kR+65rImCJNT10Gbp0X4c+bgozslOKXQ+OEA15P1XCKg0ge1KbhB3Mj/O2jIOd3qeOhwQlCTSgMoDiIuC6ZhptmR5mxtYILTqxj0nkJgk3wGUWKg4iLEim48e0ocz6p4MJuddw3MEFFEwwDKA4iromlYMKsKPO3VXDJybX89JwkgSYaBlAcRFxRUwfXz4zy/o4KvtWzlrsHJPEX8cOUDaE4iDTSp7Vw7cxKFu0McHnvWm4/O1nUz19oKMVBpBH2JOHqGZUs2xXg6lNr+WG/0ggDKA4ix2xXwsdVM6Ks3B3ghj5JbjqjtmTCAIqDyDHZGfcxfnqU1XsDfK9vkhv71no9kusUB5GjtC2WDcO6fQFu6ZfkutNKLwygOIgclY/3+xg3rZKNNX5uq0ow/pTifLm1GxQHkQb6qCYbho/2+7mrf4LLepduGMDdvTI3kN0OL1F/0URr7ZturS/ipY2fZsPwSczHz85J8PWepR0GcP/M4WvW2mUuryniqXX7smHYmfBx/8AEF52c8nqkgtDNCpHDWL3Hz/jpUXYnffxiUIIvdyuPMIBL2+HBZzcr9gI+YA7wowbuW9ENbYcnRWjZDrj497A3Cc/8C3yll9cT5U1et8MDGGqt3WyMCQOPAI8D/97QL9Z2eO4pxS3jDijUsS3b5eeq6ZXEUvDokDiDWqXZsSPvV1vQ711BtsMDsNZurv8zCfwKGOzW2iKFtHinn/HTKomnYPKwOKO7pL0eyROunDkYY5oBFdbavcYYH/AtYLEba4sU0vvbA1w3M0ragadGxDnvhPIMA7h3s6ID8JoxJgAEgBXAt11aW6Qg5m8LMGFWFICnR8Q5p0P5hgFcikP9PplnubGWiBfmfhzg229HCfjg6ZFxzm5X3mEA7ZUpwqwt2TOGkB9eGBVTGOrpeQ5S1t7aXMH350ZoVgHPjYoV3U7XXtKZg5St/7+pgpvmRGgRdJgyWmE4lOIgZemNDRX8YG6E1mGHqWPinNJaYTiU4iBl53frKvjhvAjtIg5Tx8ToeZzCkIviIGXlv9YEue3dKB0rHV4aE6N7y/w/K7ep0h2SUjZeskF++kGELs0yvDgmRudmCsPhKA5SFp5fGeT+RRG6tcgwZXSMEyoVhiNRHKTkPbU8xEMfhunRMs0Lo+O0jyoMDaE4SMlyHJi8LMRjS8P0bpXmhVFx2kYUhoZSHKQkOQ48/GGIp1aEOa11mudGxWgd9nqqpkVxkJLjODBpUZjn/h6ib5s0z46KcVzI66maHsVBSorjwL0fhJm6KsRZx6d5ekSMFgrDMVEcpGRkHLjnvTD/tSbEgPYpnhwep3nQ66maLsVBSkI6A3csiPC7dUEGdUjxq+FxKvXT3Sj655MmL5WBie9E+J+NQYZ2TPH40DgR/WQ3mv4JpUmry8D/nRfhL5uCjOyc4tEhcUIBr6cqDYqDNFm1abh5boS3Pgrypa51PHBeQmFwkesvvDLG3G2McYwxp7u9tsgByTR8Z3aUtz4K8n9OquOhwQqD21w9czDGVAEDgY1uritysHgKbnw7ytxPKrioex33nZsgoNcXu861f9L6zWwmAze4tabIoWpq4bqZ2TB8vUctPx+oMOSLm2cOPwFestZuMMYc9RcfbucdN40dOwbI7ixUykrx+PYl4Rt/gAXbK7iqL0waGcLvK71nOBXL986tTW0GAf2B/zjWNbQdnntKcTu8fbVwzYxKPqwOMM7UcuvpSap3ej2V+0pxO7zhwKnA+voNdbsAbxpjxrq0vpSxPUkYPz0bhpvOhtuqkvh8Xk9V+tza1OZ+4P4DH9cH4svW2mVurC/la1fCx/jpUeyeADeenuSuwWF2luAZQzHS8xykaO2IZ8OwZm+Am85I8u3Ta/H59LrrQslLHKy13fKxrpSPbTEf46ZHWb8vwA/7JbjmtDqvRyo7OnOQorN1v49x0yrZVOPnR1UJxp2iMHhBcZCisrkmG4Yt+/3cMyDBpb0UBq8oDlI0NuzzMW56JdtiPu49N87XeqS8HqmsKQ5SFNbu9TNuepTqhI/7ByW4qLvC4DXFQTy3ao+f8dOj7En6ePC8BBecpDAUA8VBPLVydzYM++t8PDIkwdiuCkOxUBzEM0ur/Vw9o5JYCh4bGmdk57TXI8lBFAfxxKIdfq6ZWUldBp4YFmdoJ4Wh2CgOUnDvbw9w7cwoGQeeGh5n0AkKQzFSHKSg3vkkwA2zovh88MzIOAPaKwzFSnGQgpm9NcCNs6ME/fDMiBhntct4PZIcht5DRwpixpYAN7wdJeyH50cpDE2Bzhwk7/62uYKb50ZoFnR4fmSc09ooDE2Bzhwkr/68sYKb5kRoGXJ4cbTC0JQoDpI3f1hfwS3zIrSJOEwdHce0UhiaEsVB8uK3ayuY+E6E9lGHl0bH6HGcwtDUKA7iuldWB7l9fpROzRxeGhOjW8v8v3GwuE93SIqrXrRB7v0gQtfmGaaMjtG5mcLQVLkWB2PM60B3IAPUAN+11i52a30pfs+uDDJpUYRuLTK8ODpGh0qFoSlz88xhnLV2L4Ax5kLgOaDKxfWliD2xLMQjS8L0PC7NC6PitIsqDE2da3E4EIZ6x5E9g5AS5zjw2NIQk5eFMa2yYWgTURhKgdsb6T4DjAV8wL+4ubYUH8eBhz4M8esVYfq0TvPsqBit9c7xJcPnOO5X3hhzOXCptfaCBvz1bsB614eQvHIcuGM2PLEIqjrAaxfDcQpDU9Ud2HDohXmJA4AxJg50sdZWH+GvdgPWa69M9+R7v8WMAz97P8xvVoeoapfi6RFxmgfzdnX/pBT3AT2YR3tl5oyDK89zMMY0N8Z0PejjfwN21f8nJSTjwF0LsmE4p32KZwoYBikst+5zaAb8tzGmGZAmG4V/s9bqnqkSks7A7fMj/H59kMEnpJg8LE5Uz5QpWW5tpLsNGOjGWlKcUhmY+E6E/9kYZHinFI8NjRMOeD2V5JO6L0dUl4Fb5kZ4c3OQ0V3qeGRwgpDCUPIUBzms2jTcNCfC9C1BvtS1jgcHJwjqFTllQXGQL5RIwXdnR3n74wq+fFId/zkoQYXCUDYUB8kpnoJvvx1l3icVXNy9jnvPTRBQGMqK4iCfs78OJsyKsmB7Bd/oUcuPz0ni93k9lRSa4iD/pKYOrp0ZZeGOCi7rVcsd/RWGcqU4yGf21sI1MypZUh3gylNqmXhWEp/CULYUBwFgdxKuml7Jit0Brj8tyc1n1ioMZU5xEKoTPsZPj7JqT4DvnJ7kO30VBlEcyt72uI8rp0dZszfAzWcmmdCn1uuRpEgoDmVsW8zHFdMq2fCpn4lnJbjq1DqvR5IiojiUqS37fYybVsnmGj93nJ3gcqMwyD9THMrQ5hofV7xVydaYn5+ck+CbPRUG+TzFocxs2Je9KbE97uO+c+Nc0iPl9UhSpBSHMrJ2r59x06JUJ33856AEF3ZXGOSLKQ5lwu7xM35alL21Ph48L8EFJykMcniKQxlYscvPlTOi7K/z8eiQBGO6KgxyZIpDiVuy08/VMyqJp+HxoXFGdE57PZI0Ea7EwRjTFpgK9ABqgdXA9dbaHW6sL8dm4Q4/18yoJOXAk8PjDOmoMEjDufUKfQeYZK011tq+wFrgfpfWlmPw3vYAV8+oJOPAUwqDHAO33mB2FzDzoIveBW5wY205erM2wTUzovh98MzIOP3bKwxy9Fy/z8EY4ycbhj+6vbYc2dtbA3x3NgT98MzIGP2O15alcmxc3/HKGDMZ6Ax81VrbkJ/Mbmg7PFf8ZR2M/zNUVmS3pzurg9cTSRORc8crtzfSfQDoRXZDm6P6X5a2w2ucNzdV8IO5EVqEHF6/xM8JfMqOErw7WNvhueeg7fByf96tKzLG3AecDVxkrU26ta4c2Z82VHDz3Aitwg4vjo7Tt53XE0kpcOuhzD7AbcAqYJ4xBmC9tfZiN9aXL/b6ugpumx+hbcRhyqg4PY7TfQziDrcerVgO6L2DCuy/1wa5c36YEyodpoyOcVILbU0q7tFOBE3Uy6uD3DE/QqdmDlPHKAziPj19ugma8vcg9y2McGLzDFNGx+jUTGEQ9ykOTczTK0I8sDhM95ZppoyK06FSYZD8UByakMlLQzy6NEyv49K8MCrO8VGFQfJHcWgCHAd+uSTEE8vDnNIqzfOj4rSJKAySX4pDkXMceGBxiGdWhunTJs1zI2O0Cns9lZQDxaGIOQ7ctzDMizZEv7Zpnh4Zo2XI66mkXCgORSrjwE/eD/PK6hBnt0vx6xFxmge9nkrKieJQhNIZuOu9ML9dG+LcDimeHB6nUt8pKTD9yBWZVAZ+9G6EP2wIMuSEFI8PixPVd0k8oB+7IlKXgYnvRPjTxiAjOqV4dGiccMDrqaRcKQ5FojYNt8yL8NfNQcZ0qePhwQlCCoN4SHEoArVp+N6cKDO2VPCvJ9bxi/MSBPWqF/GY4uCxRApunB1lzscVfKVbHT8fmKBCYZAioDh4KJaCG2ZFeXdbBV89uY6fnZMgoDBIkVAcPFJTBxNmRXlvewXf7FnLPQOS+PWOGFJEFAcPfFoL186sZNHOAJf3ruX2s5P4FAYpMopDge2thaunV7J0V4CrTq3l1n4KgxQn1+JQ/87Tl5B9q/m+1tplbq1dKnYlfFw1I8rK3QEm9Eny/TNqFQYpWm7e/fU6MAzY6OKaJaM64WPctGwYvttXYZDi59qZg7V2DkD9O0/LQbbHfYyfFmXtvgC3nJnkuj61Xo8kckS6zyHPPon5GDetkg2f+rmtKsH4U+q8HkmkQYomDofbecdNY8eOAbI7C+Xbpn0w6Y0PSNR0YtKIjlxzZgSI5P16oTDH55VSPjYonuMrmjiU2nZ4O7b+nU8W/JLr0jUs6f8KF3Yq3PZ0pbxlXCkfGxTXdnhFE4dSEahZT2bpI5zyyRuc5nN4zfyZb/RKeT2WyFFz86HMR4GvAicAbxljqq21fdxav9j5Y1uptI8S2fRbfE4afLCm9SUMO/1Ur0cTOSZuPlrxPeB7bq3XlPjqPqXZyocIbvlTNgxAyhem9YAfoJ0rpanSy3xc4ARbsOz4K9mXjn52WbLX1WQqO3k4lUjjKA4uWL9+Cb0WXkaUGGvajyMTaku81wSvxxJpFN0h2Uhr1y6kz5JxVJDmg1OfpXfPKmq29sUJFsfDUSLHSnFohDWrFnDG8isBWNznBXr3PgeA5ImXeDmWiCt0s+IYrbbz6Ld8PBn8LD1jKj3rwyBSKhSHY2CXv03ViquoJcjKfi/Ro0eV1yOJuE5xOEorl01nwKpriVPJqqqX6d79TK9HEskLxeEorPjwTQatvp4aWrC+/8t0O6lsnuMlZUhxaKDli95g8Lob2UNrNp3zKl27nuL1SCJ5pTg0wLL3f8+wDTdRTTu2DnyVLp17ej2SSN7pocwjWLbg/zF8y3+wzdeJ6kG/oVOHk7weSaQgdOZwGEvf/Q0jt05kq68ruwa/SgeFQcqIzhy+wNJ5LzBq+4/ZyMnEhr5E+7YdvR5JpKAUhxyWzv41o6p/znp6UTviNxzfqp3XI4kUnOJwECfjsGz2rxi1+wHWcAqZEVNp0+p4r8cS8YTiUM/JOCyf9Qij9j7KKt/p+Ee+SOuWrb0eS8QzigPZMKyYOYmR+55kpa8foVHP07JFK6/HEvFU2cfByTisnH4vI2qeZbm/P5Wjn6V5s5ZejyXiOTffQ7I3MAVoC1QDV1hrV7u1fj5kMhnsW/cwPDaVJf6BtDz/aZpFC/MW+SLFzs3nOTwJTLbW9gYmA0+5uLbr0uk0q/92O8NiU1kcGEKrsc8qDCIHcSUOxpj2QBXwSv1FrwBVxpiifAwwlUqx7q+3MiT+KgsrRnL82KeJRiq9HkukqLh1s6IrsMVamwaw1qaNMVvrL2/QVi7Lly8mkUi4NM4X27t3DwtfvZ7zkn/gHUaw+4Sr2bniw7xfbyGNHTuGRYvmez1GXpTysUFhjy8SiTBs2JAv/HzR3CF5uCHdNuf1NbznfJmzv/lrQqFQwa63kA5s+1eKSvnYoHiOz604bAY6G2MC9WcNAaBT/eUNUsjt8M7/ykS2bd/L3r1JIJn36yy0Ut4yrpSPDYprOzxX7nOw1m4HFgOX1l90KbDIWlug3SGPjs/vIxAIeD2GSFFz82bFBGCKMeYuYDdwhYtri0iBubkd3t+Bc91aT0S8pfdzEJGcFAcRyUlxEJGcFAcRyUlxEJGcFAcRyUlxEJGcFAcRyUlxEJGcFAcRyUlxEJGcFAcRyUlxEJGcFAcRyUlxEJGcFAcRyUlxEJGcFAcRyUlxEJGcGv0eksaYfwduBU4Dvm+tfbzRU4mI59w4c1gMfAt42YW1RKRINPrMwVq7DMAYkznGJQKQ3WCjECKRCIW8Pq+U8vGV8rFB4Y7voOvJuYlLMWyH1xGgdetmBbmyA9vuHW6nn1JQysdXyscGnhxfR2DtoRceMQ7GmIXAiV/w6Q4HNs9thPeAocDHQGPXEpGGC5ANw3u5PnnEOFhrq9ye6BBJYE6er0NEcvvcGcMBeihTRHLyOU7jdrY2xlwK/AJoDdQC+4Gx1toVjR9PRLzS6DiISGnSzQoRyUlxEJGcFAcRyUlxEJGciuEZkgVjjOkNTAHaAtXAFdba1d5O5Q5jTFtgKtCD7KNGq4HrrbU7PB0sD4wxdwP3AH0PPH2/qTPGRICHgTFAAnjHWnudlzOV25nDk8Bka21vYDLwlMfzuMkBJllrjbW2L9knt9zv8UyuM8ZUAQOBjV7P4rJJZKPQu/77d6fH85RPHIwx7YEq4JX6i14Bqowx7bybyj3W2l3W2pkHXfQucJJH4+SFMSZMNuo3eD2Lm4wxzYErgDuttQ6AtXabt1OVURyArsCWA68Fqf9za/3lJcUY4yf7C/RHr2dx2U+Al6y1G7wexGU9yN7MvdsY874xZqYxZojXQ5VTHMrJY0ANUDJvvGOMGQT0B37l9Sx5EABOBhZZa/sDE4HfGWNaejlUOcVhM9DZGBMAqP+zU/3lJcMY8wDQC/imtfZY32OjGA0HTgXWG2M2AF2AN40xYz2dyh2bgBT1N3mttfOBnUBvL4cqmzhYa7eTfdeqS+svupRsqUvm3nxjzH3A2cBF1tqk1/O4yVp7v7W2k7W2m7W2G/AR8CVr7V89Hq3RrLU7gRnA+fDZo2rtgTVezlVWD2UCE4Apxpi7gN1k7wQqCcaYPsBtwCpgnjEGYL219mJPB5OGmgA8Z4x5EKgDLrfW7vFyIL3wSkRyKpubFSJydBQHEclJcRCRnBQHEclJcRCRnBQHEclJcRCRnBQHEcnpfwE8i/29SMpU2AAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 288x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "Av = A.dot(v)\n",
    "\n",
    "orange = '#FF9A13'\n",
    "blue = '#1190FF'\n",
    "\n",
    "plotVectors([Av.flatten(), v.flatten()], cols=[blue, orange])\n",
    "plt.ylim(-1, 7)\n",
    "plt.xlim(-1, 7)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can see that their directions are the same!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Another eigenvector of $\\bs{A}$ is\n",
    "\n",
    "$$\n",
    "\\bs{v}=\n",
    "\\begin{bmatrix}\n",
    "    1\\\\\\\\\n",
    "    -3\n",
    "\\end{bmatrix}\n",
    "$$\n",
    "\n",
    "because\n",
    "\n",
    "$$\n",
    "\\begin{bmatrix}\n",
    "    5 & 1\\\\\\\\\n",
    "    3 & 3\n",
    "\\end{bmatrix}\\begin{bmatrix}\n",
    "    1\\\\\\\\\n",
    "    -3\n",
    "\\end{bmatrix} = \\begin{bmatrix}\n",
    "    2\\\\\\\\\n",
    "    -6\n",
    "\\end{bmatrix}\n",
    "$$\n",
    "\n",
    "and\n",
    "\n",
    "$$\n",
    "2 \\times \\begin{bmatrix}\n",
    "    1\\\\\\\\\n",
    "    -3\n",
    "\\end{bmatrix} =\n",
    "\\begin{bmatrix}\n",
    "    2\\\\\\\\\n",
    "    -6\n",
    "\\end{bmatrix}\n",
    "$$\n",
    "\n",
    "So the corresponding eigenvalue is $\\lambda=2$."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[ 1],\n",
       "       [-3]])"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "v = np.array([[1], [-3]])\n",
    "v"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(-1.0, 3.0)"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAQoAAAD/CAYAAAAJ45jXAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8qNh9FAAAACXBIWXMAAAsTAAALEwEAmpwYAAAV10lEQVR4nO3deXhV9Z3H8ffNzUYIpBAwskSCLD9NQBRFpCxipWgVO62tVWdardqZ6lRt61JRi7YVKxXsPK3QaafV6rSKXR+t7WirdqwsAiKJssQfjLIECAjIIpD15s4fN8GomHPD/eWec08+r+fhecjl5JfvSczHs93vNxKPxxER6UiW3wWISPApKETEk4JCRDwpKETEk4JCRDwpKETEU7arhYwx84DPAWXAaGvtGldri4i/XB5RPAlMATY7XFNEAsDZEYW1djGAMcbVkiISEM6CIgV5wDigFoj5XItIdxIFBgCvAA0dbRiEoBgHLPK7CJFubDKwuKMNghAUtQB79x6ipSV87ztZu7aKKVMmsWfPQb9L6TLFxYWh3b8w71tWVoQ+fXpC6+9gR4IQFDGAlpZ4KIOivr4eIJT71l6Y9y/M+9bK85Tf2V0PY8yPjTFbgcHA88aYta7WFhF/ubzrcSNwo6v1RCQ49GSmiHhSUIiIJwWFiHhSUIiIJwWFiHhSUIiIJwWFiHhSUIiIJwWFiHhSUIiIJwWFiHhSUIiIJwWFiHhSUIiIJwWFiHhSUIiIJwWFiHhSUIiIJ5cjBUcCjwLFwB7gCmvtBlfri4h/XB5R/BRYYK0dCSwAfuZwbRHxkZOgMMYcB4wFFra+tBAYa4zp72J9EfGXq1OPUmCbtTYGYK2NGWO2t76+K5kF1q6tOjIDI0zeffcAAJWVy32upOtMnz4ttPsX5n3Lz89nypRJSW0bhAFAAEkXnGmamppY/OQcCiIRJv3TbX6X02WmT5/mdwldJsz7lixXQVEDDDLGRFuPJqLAwNbXk7Jnz8FQTmRa+epShu9+lhNb1rFixUSGDh3jd0nO9e/fi1273vW7jC4R5n3LyopQXFyY3LYuvqC19m2gCri89aXLgUprbVKnHWEWzYoSmbqAZrI57rWbqW+o87skkU5zedfjWuAGY8x64IbWjwUYPryCJf1voYw3eWvRA36XI9JpLkcKvgGMd7Ve2JRPuJo1f3mOsw8+xFI7jRHmLL9LEkmansxMk2g0Snz8XA7HCxhafQsHDx3wuySRpCko0uj4kiEsG3AnA9jG9sXf97sckaQpKNJs1JmXUxk9m4l1v6F6zQt+lyOSFAVFmkWyIuRPnMP+eG/KN8xk/4F3/C5JxJOCwgf9+h5P5ZB76BfZzZ7Fs4iH8PkRCRcFhU/KT72I5bkXML7xf1hX9bTf5Yh0SEHhk0hWhOJJ97A73o/TNs9i9zs7/C5J5CMpKHxU1Lsv64bPoShygLolt+sURAJLQeGzk0efy5IelzI29iJrXnnC73JEjkpBEQADJ93BdgZz1vbZ7Ni52e9yRD5EQREAhT17s+nkuRREDhNZfiuxWMzvkkTeR0ERECPMWfyj8GpGtbzCupcf9rsckfdRUATIiZNvYRPDmLhrHjVb1/tdjsgRCooAyc/rwdtjHiCbZgpfvYnGpka/SxIBFBSBM3ToGF762PWMjK9l/eIFfpcjAigoAmnkpK+xPjKKKfvms/GtKr/LEVFQBFFuTi4HT38g0T7v9VvUPk98p6AIqNLBI9u1z5vndznSzSkoAqx8wtWsyRrH2QcfZoN92e9ypBtLOSiMMV80xrxujGk2xlzvoihJeH/7vFvVPk984+KIogq4DHjcwVryAceXDGHZwG+rfZ74KuWgsNausdauA1oc1CNHMWrcZayKTk20z1ut9nmSfpF43M1bm40xjwArrbXzO/mpZcBGJ0WEWO3O7fR8ahKN5BH5/CKK+/bzuyQJj6HApo428JzrYYxZBZzwEf9c0jaYOFVhHSlYWbmc6dOnpTyWLjurF6uGzGbqlhtZ/vRNxM6fTyQr4qjK1IR57F6Y960zIwU9g8JaOzblisSJ8lNnsHzHXxnf+BderHqairGf9rsk6SZ0ezSDJNrnfU/t8yTtXNwevdwYsxW4BLjHGLPVGFOeemlyNEW9+7JuRKJ9Xv2SmWqfJ2mR8uxRa+1CYKGDWiRJJ486lyXbLmVi3W/4+4qFjD7rn/0uSUJOpx4Z6kj7vNp71T5PupyCIkMV9uzNpvJ5ap8naaGgyGAjRo7nH4XXqH2edDkFRYY7cfLNbGIYk3bNZUuN9bscCSkFRYZra58XJUavVTerfZ50CQVFCLy/fV5nn6AX8aagCIn32uctUPs8cU5BERK5ObkcOv2Hap8nXUJBESKDB49gSf9b1T5PnFNQhEz5hKtYk3Wm2ueJUwqKkEm0z7tf7fPEKQVFCCXa581KtM9bdK/f5UgIKChCatS4S1mVfQ4T63+r9nmSMgVFSEWyIhR8/D72x4uo+L+Z7N+/x++SJIMpKEKsuG8JlUPuoTiym3cWz1LvCjlmCoqQKz91BstzL+TMpmdYW/knv8uRDKWgCLm29nm74v0Zu+Uudu+p9bskyUAKim6gqHdf3mhrn7f0dp2CSKel3ArPGLMAOBdoAA4CX7fWrkx1XXHrpFGfYPG2y5hU94Ta50mnuTiieAYYba0dA9wH/MbBmtIFBk9OtM+bUDubHTs3+V2OZBAXIwX/bK1tav3wZWCwMUanNAHUs6AXm8rn0SNSR5ba50knOBspCGCMuRsYY629uBOfVoZGCqbVkt/dycR9P2PxgFlMmvF1v8sR/3mOFPQMimRHChpjLgO+C0yx1u7sRJFlwEaNFEyfhoZ6mp+5iIHxzaw+42lOKDUprRfmsXth3rd2IwVTnz2azEhBY8xngXuBczsZEuKDvLx8asc8wOCqi+m96iYaS/5Ibm6e32VJgLmYFDYD+CFwnrV2U8oVSVqUDT2FRX2uZ0R8HRuWqH2edMzFRcdfArnA740xVa1/ih2sK11s5MR/x0ZGM3nfT9j4VqXf5UiAuRgp2N9FIZJ+uTm5HD79AZpfmUHJ67dQN/BpeuQX+F2WBJBuY3Zzbe3zhvAWG9U+Tz6CgkIon3AVq7PGc/ahX6p9nhyVgkKIRqMw/n4OxQsYWn0Lh9Q+Tz5AQSEAHF9yAssHzmIA29m2aLbf5UjAKCjkiPfa5/2O6tXP+12OBIiCQo54f/u829mn9nnSSkEh71Pct4TKskT7vL1qnyetFBTyIeVjZrAsd4ba58kRCgr5kEhWhH6t7fNO3zJL7fNEQSFHV9S7D2+MmEPvyLvUL51JS0uL3yWJjxQU8pFOGvUJFve4jNNiL7F2xUK/yxEfKSikQ4Mn38E2SplQey+1Ozb5XY74REEhHepZ0IvN5XPpEakjuuJWYs3NfpckPlBQiKcRI8fzj8KvUNGykuqXH0q82Fznb1GSVgoKScqwyTezieFM3P0ANTXV9H71mxDXBc7uQkEhScnLy2fXmAfIooVhr/4LebV/JXvf636XJWmioJCkZB3eyqj9v6UhWkQRewHI2/5Xn6uSdFFQSFJaCgbT1HcsBRw88lq85llwOO5BgsvFSME7gUuBGBAB7rPWalpYCDWccDGxXsMpWHYteQ219KzfxP696+G4M/wuTbqYiyOK+dbaU6y1pwEXAD83xvRxsK4EUHOfU3j3nKeo6XEmAJVVL/hckaSDi5GC+9t9WAjEXawrwRXP70/Oub/iT9lXUrrvWV7c4ndF0tWc/EIbY641xrwBVAL/Zq1VI4OQi2bnMnjqd/l1y1V872+1HGj0uyLpSs5GCrZuOxp4DDinE2FRhmaPZqxHVsNNf49z2ckRfjLd72rkGKVnpGC7bVcbY7YDU4E/JPt5AJo9mpkuKIG/DOnFE9UwuV8d00rD9Yh3N5k96r1tql/MGFPe7u9DgdOAdamuK5khEoEfTYPeOXFmrchjT33E75KkC7i4RvEdY8xaY0wV8HvgRmtttYN1JUMMLIS7x9XzTkMWd6/I06MVIeRipOAXXBQime3CIc08t7WJZ7fk8NTGZj5zYrhOQbo73cYUJyIRuPuMBvrlt3DPq/nUHtIpSJgoKMSZvvlxZo+v52BThDuW5xPCa9PdloJCnDpnUIzPndjI0h3ZLNyQ43c54oiCQpy74/QGBvVs4f7KPDYd0ClIGCgoxLnCHLjvrHrqYxFmLutBs/rbZDwFhXSJ8SUxrjSNVO6O8lB1rt/lSIoUFNJlbhrTwIm9Yzy4Opc39uo/tUymn550mfxs+MGEelri8K2X82mMeX+OBJOCQrrUKcUtfLWiEbsvyvzVOgXJVAoK6XLXVTRS0SfGz6tzqdyl/+QykX5q0uVyo4lTkOwI3PZyDw7r6e6Mo6CQtBjxsRa+MaaBzQezmFeZ53c50kkKCkmbL5smzujfzGMbclm6I+p3OdIJCgpJm2gWzJlQT0F2nNuX5at9XgZRUEhalRbGmTm2gR2Hs7j31Xy/y5EkKSgk7b4wrIkpA5p5cmMOz9Wk3BJF0kBBIWkXicDs8fUU5ca5S+3zMoKCQnxRUhDnrjMS7fPuUvu8wHMWFMaYqcaYmDHmeldrSrhdOKSZ809o4vmtOTy1UacgQeZqAFAv4AfAMy7Wk+6hrX1ef7XPCzxXRxQ/BOYCux2tJ91E+/Z5ty9T+7ygcjHX41NAkbX29w7qkW5o6qAYnx/WyMs7s3l8vdrnBZHniaHHSEEDzAE+mWohyU4syjTTp08DEhOnwizV/Zv3SVj+Nsx7LZ+LKvIZ3sdRYQ6E/WeXDM/Zox0xxkwC/ggcbn2pH9AA/Mha+70klykDNmqkYOZyNXZv+c4oV7xQwKnFMR775GGyA3BPrpuMFEx99mhHrLWLgePaPjbGPAKstNbOT2Vd6Z7a2uc9anN5qDqXr1boGe+gCEBmi7xH7fOCyelPwlr7ZR1NSCrys+H+1vZ5t6p9XmAosiVwRhe3cG1FI+v3RXlQ7fMCQUEhgXTdqET7vF9U57JK7fN8p5+ABFJO1nvt82aqfZ7vFBQSWCM+1sI31T4vEBQUEmhXtmuft6RW7fP8oqCQQFP7vGBQUEjgtbXP21mXxeyVap/nBwWFZIS29nlPbVL7PD8oKCQjqH2evxQUkjFKCuLc3do+b5ba56WVgkIyyoVlzXzqhCZe2JrDk2qflzYKCsk4d4+rp39+C7NfzWe72uelhYJCMk6fPI60z7tD7fPSQkEhGWnqoBiXqH1e2igoJGPNHNvAoJ4tzK3KY+MBnYJ0JQWFZKzCHJhzVj0NscQbx5pb/K4ovBQUktHOLIlxpWmiak+UX1Srd0VXUVBIxvvmmAaG9Y4xX+3zuoyLuR6PGGO2GmOqWv/c6aIwkWTlZyd6V6h9Xtdx9cTKHPXKFD+NLm7huopG5q/J48HVudx8qt5m6pKO0yQ0rh3VSEVftc/rCq6+mzcZY1YbY540xpzsaE2RTsnJSnTwzo7AbWqf55TnpDCPkYIlwPFArbW2xRhzBXAPcKK1NtkzxTJgY5LbiniavwruWgTXnAJzz/G7mozgOSkspZGCR2OM2QOMtdZuTvJTytBIwYwWtLF7sRa44oUerNyVzcPnHGbigGO/uhm0fXOpMyMFXdz1GNTu7+cBMWBbquuKHCu1z3PPxTWKR1uvT7wGfBv4tLVWZ4fiq9LCOLerfZ4zKd8etdZOc1GIiGuXDGviuZpsntqUw7TSZqaX6v9fx0r3kCS0Ptg+b3ed3jh2rBQUEmpt7fP2NmRx1ytqn3esFBQSeheWNXOB2uelREEh3cJdap+XEgWFdAt98uBetc87ZgoK6TbObtc+7zG1z+sUBYV0K23t8+apfV6nKCikW2nfPu82tc9LmoJCup0zS2J8+aQmXlP7vKQpKKRbat8+r1rt8zzpOyTdUl70vfZ531qq9nleFBTSbbW1z1u/P8qPV+sUpCMKCunW2trnPaT2eR3Sd0a6NbXPS46CQrq94UUt3DSmgS0Hs5hbmQfAip1Rn6sKFr1DRgS48qQmXtiWzeMbcumdG+dvNdk8M+Ow32UFho4oRIAIcNtpDWRH4vx0bR5vHYjy5n79erTRd0IEWLU7yjX/W0Bz/L3Huv9WowPuNk6CwhhzgzHmjdbemVUu1hRJp9P7x/jD+YcYWfTeAxXPKSiOcNGF+2LgEmCctXY0cF7KVYn4oLQwzhPTD3NeaRMAa/dGqTngc1EB4eKI4mbgO9badwGstTsdrCnii5458KNJ9XzjlAYixPnzm35XFAwugqIcOMsYs9QYs9IY868O1hTxTSQC141q5D/PruOlGr+rCQYXIwX3Ao8BXwP6AUuAa6y1LyVZQxkaKSgB9eZeGFIE2eG+7O85Kczzao21dmxH/26M2QIstNa2AG8bY54DzgSSDQoANFIwc4V57N6wEO9bu5GC3ts6+HqPA+cDGGN6ApOB1xysKyIB4SIo/gMoNcasBVYAv7bWPudgXREJCBcjBeuALzmoRUQCKtyXaETECQWFiHhSUIiIJwWFiHhSUIiIJwWFiHhSUIiIJwWFiHhSUIiIJwWFiHhSUIiIJwWFiHhSUIiIJwWFiHhSUIiIJwWFiHhSUIiIJwWFiHhSUIiIp5R7Zhpjnicxz6NtvQpgjLX29VTXFpFgcNFcd1rb340xnwFmKyREwsX1uOargYc7+TlRSAwjCaP8/HwgvPvXJsz7F9Z9a7dfUa9tPUcKJssYczzwFjDEWrurE586CVjkpAgRORaTgcUdbeB5ROE1e9RaG2v9+xXAs50MCYBXSBRaC8Q8thURd6LAABK/gx1yeURRDdxqrf2zkwVFJDCc3B41xnwcKAKecbGeiASLq+corgL+u91piIiEiLNTDxEJLz2ZKSKeFBQi4klBISKeFBQi4sn1I9zHzBjzReBbQDnwDWvtfJ9LSpkxZiTwKFAM7AGusNZu8LcqN4wx84DPAWXAaGvtGn8rcssYUwz8ChgGNAIbgK8ewwOFgWWMeRIYCrQAB4EbrLVVR9s2SEcUVcBlwOM+1+HST4EF1tqRwALgZz7X49KTwBRgs891dJU4cL+11lhrRwNvAnN8rsm1K621Y6y1pwHz6OB9WoEJCmvtGmvtOhLplvGMMccBY4GFrS8tBMYaY/r7V5U71trF1toav+voKtbad6y1L7Z7aRkwxKdyuoS1dn+7D4vo4HcvMKceIVQKbGt7CM1aGzPGbG99PTSHr92BMSYLuA74k9+1uGaM+QUwHYgA53/UdmkLik68uUwkaB4kcQ6f8dfNPsha+xUAY8yXgLnABUfbLm1BYa0dm66vFRA1wCBjTLT1aCIKDGx9XTJE60XbEcBF1tpQnBYfjbX2V8aY/zLGFFtr93zw3wNzjSJsrLVvk7hAe3nrS5cDlWG6ah52xpjvA6cDn7HWNvhdj0vGmEJjTGm7jy8C3mn98yGBea+HMeZyEoc+fUjcjjoETG+9wJmRjDEnkbg92gfYS+L2qPW3KjeMMT8GLgaOB3YDe6y1Ff5W5Y4xpgJYA6wH6lpf3mit/ax/VbljjCkBngJ6kugD8w5wi7V21dG2D0xQiEhw6dRDRDwpKETEk4JCRDwpKETEk4JCRDwpKETEk4JCRDwpKETE0/8Dc+dWt+3Yv94AAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 288x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "Av = A.dot(v)\n",
    "\n",
    "plotVectors([Av.flatten(), v.flatten()], cols=[blue, orange])\n",
    "plt.ylim(-7, 1)\n",
    "plt.xlim(-1, 3)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This example shows that the eigenvectors $\\bs{v}$ are vectors that change only in scale when we apply the matrix $\\bs{A}$ to them. Here the scales were 6 for the first eigenvector and 2 to the second but $\\lambda$ can take any real or even complex value."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Find eigenvalues and eigenvectors in Python\n",
    "\n",
    "Numpy provides a function returning eigenvectors and eigenvalues (the first array corresponds to the eigenvalues and the second to the eigenvectors concatenated in columns):\n",
    "\n",
    "```python\n",
    "(array([ 6.,  2.]), array([[ 0.70710678, -0.31622777],\n",
    "       [ 0.70710678,  0.9486833 ]]))\n",
    "```\n",
    "\n",
    "Here a demonstration with the preceding example."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[5, 1],\n",
       "       [3, 3]])"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "A = np.array([[5, 1], [3, 3]])\n",
    "A"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(array([6., 2.]),\n",
       " array([[ 0.70710678, -0.31622777],\n",
       "        [ 0.70710678,  0.9486833 ]]))"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "np.linalg.eig(A)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can see that the eigenvalues are the same than the ones we used before: 6 and 2 (first array).\n",
    "\n",
    "The eigenvectors correspond to the columns of the second array. This means that the eigenvector corresponding to $\\lambda=6$ is:\n",
    "\n",
    "$$\n",
    "\\begin{bmatrix}\n",
    "    0.70710678\\\\\\\\\n",
    "    0.70710678\n",
    "\\end{bmatrix}\n",
    "$$\n",
    "\n",
    "The eigenvector corresponding to $\\lambda=2$ is:\n",
    "\n",
    "$$\n",
    "\\begin{bmatrix}\n",
    "    -0.31622777\\\\\\\\\n",
    "    0.9486833\n",
    "\\end{bmatrix}\n",
    "$$\n",
    "\n",
    "The eigenvectors look different because they have not necessarly the same scaling than the ones we gave in the example. We can easily see that the first corresponds to a scaled version of our $\\begin{bmatrix}\n",
    "    1\\\\\\\\\n",
    "    1\n",
    "\\end{bmatrix}$. But the same property stands. We have still $\\bs{Av} = \\lambda\\bs{v}$:\n",
    "\n",
    "$$\n",
    "\\begin{bmatrix}\n",
    "    5 & 1\\\\\\\\\n",
    "    3 & 3\n",
    "\\end{bmatrix}\n",
    "\\begin{bmatrix}\n",
    "    0.70710678\\\\\\\\\n",
    "    0.70710678\n",
    "\\end{bmatrix}=\n",
    "\\begin{bmatrix}\n",
    "    4.24264069\\\\\\\\\n",
    "    4.24264069\n",
    "\\end{bmatrix}\n",
    "$$\n",
    "\n",
    "With $0.70710678 \\times 6 = 4.24264069$. So there are an infinite number of eigenvectors corresponding to the eigenvalue $6$. They are equivalent because we are interested by their directions.\n",
    "\n",
    "For the second eigenvector we can check that it corresponds to a scaled version of $\\begin{bmatrix}\n",
    "    1\\\\\\\\\n",
    "    -3\n",
    "\\end{bmatrix}$. We can draw these vectors and see if they are parallel."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(-1.0, 3.0)"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAQoAAAD/CAYAAAAJ45jXAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8qNh9FAAAACXBIWXMAAAsTAAALEwEAmpwYAAAXqklEQVR4nO3deXxU5b3H8c/MZBkCJoWAKItswqMEREFEymqlURG7WK3aW7Uu91ava3G3glWgWoH2ZcXe9lqtXRS72Gq1V1u118qOQqJsPnAVkE0kyCKQdWbuH0kwKmYmzJM5Z06+79eLl8lw8uR3EvL1bPn9QolEAhGR5oS9LkBE/E9BISJJKShEJCkFhYgkpaAQkaQUFCKSVI6rhYwxs4BvAL2Bwdbala7WFhFvuTyieAYYC2x0uKaI+ICzIwpr7XwAY4yrJUXEJ5wFRRrygeHANiDmcS0ibUkEOBp4HahubkM/BMVwYJ7XRYi0YWOA+c1t4Ieg2Aawa9d+4nF//d5JbS2EQpCTxldp1apyxo4dzc6d+9wV5jPFxR0Cu39B3rdwOETHju2h4WewOX4IihhAPJ7wXVCEQnDNNVGmTaumc+fDq62qqgrAd/vmWpD3L8j71iDpKb+zux7GmJ8aYzYDPYCXjTGrXK3tlXAYiooSlJYWsGKFHjmRtsvlXY/rgetdrecXEyfW8dhjeUyaVMBPflLFuefWeV2SSMbpf5NJjBwZo2PHBJWVIa66qh333ptHTPdmpI1RUCSRmwtnnPHxUcScOfl861vt2L3bu5pEMk1BkYKJE2sPvp2fn+CrX61l166QhxWJZJaCIgXjxsUoKEgwbFiM6uoQGzeG6dMn8FfCRQ5SUKSgXTuYPLmGv/zlACUlMR58MI9ly/Slk7ZD/9pTdN11NUSjMGdOFTk5cO217ThwwOuqRDJDQZGiUMMliZKSOLfeWsM774SZMSPf26JEMkRBcRiuuaaG4cNjPPJIHvPmRbwuR6TVKSgOQyQCDz1USUFBghtuiLJ3r9cVibQuBcVh6ts3wdSp1WzeHOauu6JelyPSqhQUabjsslrGjavjqadyefFFnYJIcCko0hAKwYMPVlFYmGDy5CgVFXoIS4JJQZGmbt0S3HdfFRUVYW69NR+NcpUgUlA4cN55dZx9di3PP5/L00/7ocWHiFsKCgdCIZg5s5rOnePccUeUrVt1CiLBoqBwpHPnBLNnV7NnT4gbb4zqFEQCRUHh0Fln1XHBBbW8+moOjz+e63U5Is4oKBybMaOK7t3j3HNPPu++q1MQCQaXPTMHGGMWGWPWNvy3v6u1s0lhYf0t0wMHQlx/fVTdsCQQXB5R/Bx42Fo7AHgY+IXDtbPK2LExrryyhqVLc3j66V5elyOSNidBYYw5EhgKzG14aS4w1BjTxcX62eiuu6rp1y/Ob37TjxUrvK5GJD2ubvr3BLZYa2MA1tqYMWZrw+s7Ullg1arygzMwguK66wqZPHk4F18MM2YsJTc3mLdCSksnUFa2xOsyWkWQ9y0ajTJ27OiUtvXN00GpFpxNSkuhoiLGlnWLaF+1nPFn3+J1Sa2mtHSC1yW0miDvW6pcBcUmoLsxJtJwNBEBujW8npKdO/cFciLTF0ct5Nhe99F352qWLh1Jnz5DvC7JuS5djmDHjo+8LqNVBHnfwuEQxcUdUtvWxSe01n4AlAMXNbx0EVBmrU3ptCPIIuEIofEPU0cOR755E1XVlV6XJNJiLu96XAVcZ4xZC1zX8L4Axx5bwoIuN9Obd3h33myvyxFpMZcjBd8GRrhaL2gGjryclX97iXH7HmWhnUB/c6rXJYmkTE9mZkgkEiExYiYHEgX0WXMz+/arf55kDwVFBh3VtReLj/4+R7OFrfN/6HU5IilTUGTYoFMuoiwyjlGVv2fNyle8LkckJQqKDAuFQ0RH3c+eRCED193Onr0fel2SSFIKCg907nQUZb2m0TlUwc75U0gE8PkRCRYFhUcGnngOS/ImMqLmf1hd/pzX5Yg0S0HhkVA4RPHoaVQkOnPSxilUfPi+1yWJfC4FhYeKCjux+tj7KQrtpXLBHToFEd9SUHjs+MGns6DdBQyNvcrK15/yuhyRQ1JQ+EC30XeylR6cunU672/f6HU5Ip+hoPCBDu0L2XD8TApCBwgtuYWY+ueJzygofKK/OZV/dbicQfHXWb3oMa/LEfkEBYWP9B1zMxvox6gds9i0ea3X5YgcpKDwkWh+Oz4YMpsc6uiwbDI1tTVelyQCKCh8p0+fIbz2hWsZkFjF2vkPe12OCKCg8KUBo69hbWgQY3fPYf275V6XI6Kg8KO83Dz2DZtd3z7vrZvVPk88p6DwqZ49BjRpnzfL63KkjVNQ+NjAkZezMjyccfseY51d5HU50oalHRTGmG8bY94yxtQZY651UZTU+2T7vFvUPk884+KIohy4EHjSwVryKUd17cXibnepfZ54Ku2gsNautNauBuIO6pFDGDT8QpZHxte3z1uh9nmSeaFEws2vNhtjHgfesNbOaeGH9gbWOykiwLZt30r7Z0dTQz6h8+ZR3Kmz1yVJcPQBNjS3QdK5HsaY5cAxn/PXXRsHE6crqCMFy8qWUFo6Ie2xdDnhI1jeazrj37ueJc9NJnbmHELhkKMq0xPksXtB3reWjBRMGhTW2qFpVyRODDxxEkve/zsjav7Gq+XPUTL0K16XJG2Ebo9mkfr2efeqfZ5knIvboxcZYzYD5wPTjDGbjTED0y9NDqWosBOr+9e3z6tacLva50lGpD171Fo7F5jroBZJ0fGDTmfBlgsYVfl7/rl0LoNP/ZbXJUnA6dQjSx1sn7dthtrnSatTUGSpDu0L2TBwltrnSUYoKLJY/wEj+FeHK9Q+T1qdgiLL9R1zExvox+gdM3lvk/W6HAkoBUWWa2yfFyHGEctvUvs8aRUKigD4ZPu8lj5BL5KcgiIgPm6f97Da54lzCoqAyMvNY/+wH6t9nrQKBUWA9OjRnwVdblH7PHFOQREwA0dexsrwKWqfJ04pKAKmvn3eA2qfJ04pKAKovn3elPr2efNmeF2OBICCIqAGDb+A5TmnMarqD2qfJ2lTUARUKByi4Iv3sSdRRMn/3c6ePTu9LkmymIIiwIo7daWs1zSKQxV8OH+KelfIYVNQBNzAEyexJO9sTql9gVVlf/W6HMlSCoqAa2yftyPRhaHvTaVi5zavS5IspKBoA4oKO/F2Y/u8hXfoFERaLO1WeMaYh4HTgWpgH3CDtfaNdNcVt44b9CXmb7mQ0ZVPqX2etJiLI4oXgMHW2iHAfcDvHawpraDHmPr2eSO3Tef97Ru8LkeyiIuRgs9ba2sb3l0E9DDG6JTGh9oXHMGGgbNoF6okrPZ50gLORgoCGGPuBoZYa89twYf1RiMFM2rBH7/PqN2/YP7RUxg96QavyxHvJR0pmDQoUh0paIy5ELgHGGut3d6CInsD6zVSMHOqq6uoe+EcuiU2suLk5zimp0lrvSCP3QvyvjUZKZj+7NFURgoaY74OzABOb2FIiAfy86NsGzKbHuXnUrh8MjVd/0xeXr7XZYmPuZgUNgn4MXCGtXZD2hVJRvTucwLzOl5L/8Rq1i1Q+zxpnouLjr8C8oA/GWPKG/4UO1hXWtmAUf+JDQ1mzO6fsf7dMq/LER9zMVKwi4tCJPPycvM4MGw2da9PoutbN1PZ7TnaRQu8Lkt8SLcx27jG9nm9eJf1ap8nn0NBIQwceRkrwiMYt/9Xap8nh6SgECKRCIx4gP2JAvqsuZn9ap8nn6KgEACO6noMS7pN4Wi2smXedK/LEZ9RUMhBH7fP+yNrVrzsdTniIwoKOeiT7fPuYLfa50kDBYV8QnGnrpT1rm+ft0vt86SBgkI+Y+CQSSzOm6T2eXKQgkI+IxQO0bmhfd6w96aofZ4oKOTQigo78nb/+ykMfUTVwtuJx+NelyQeUlDI5zpu0JeY3+5CToq9xqqlc70uRzykoJBm9RhzJ1voychtM9j2/gavyxGPKCikWe0LjmDjwJm0C1USWXoLsbo6r0sSDygoJKn+A0bwrw5XUhJ/gzWLHq1/sa7S26IkoxQUkpJ+Y25iA8cyqmI2mzatoXDZ9yChC5xthYJCUpKfH2XHkNmEidNv2b+Rv+3v5Ox+y+uyJEMUFJKS8IHNDNrzB6ojRRSxC4D8rX/3uCrJFAWFpCRe0IPaTkMpYN/B1xKbXgSH4x7Ev1yMFPw+cAEQA0LAfdZaTQsLoOpjziV2xLEULL6K/OpttK/awJ5da+HIk70uTVqZiyOKOdbaE6y1JwETgUeMMR0drCs+VNfxBD467Vk2tTsFgLLyVzyuSDLBxUjBPU3e7QAkXKwr/pWIdiH39N/y15xL6bn7RV59z+uKpLU5+YE2xlxljHkbKAP+w1qrRgYBF8nJo8f4e/hd/DLu/cc29tZ4XZG0JmcjBRu2HQw8AZzWgrDojWaPZq3HV8Dkfya48PgQPyv1uho5TJkZKdhk2xXGmK3AeODpVD8OQLNHs9PErvC3Xkfw1BoY07mSCT2D9Yh3G5k9mnzbdD+ZMWZgk7f7ACcBq9NdV7JDKAQPToDC3ARTluazsyrkdUnSClxco/iBMWaVMaYc+BNwvbV2jYN1JUt06wB3D6/iw+owdy/N16MVAeRipOA3XRQi2e3sXnW8tLmWF9/L5dn1dXytb7BOQdo63cYUJ0IhuPvkajpH40xbFmXbfp2CBImCQpzpFE0wfUQV+2pD3LkkSgCvTbdZCgpx6rTuMb7Rt4aF7+cwd12u1+WIIwoKce7OYdV0bx/ngbJ8NuzVKUgQKCjEuQ65cN+pVVTFQty+uB116m+T9RQU0ipGdI1xqamhrCLCo2vyvC5H0qSgkFYzeUg1fQtjPLQij7d36Z9aNtN3T1pNNAd+NLKKeAJuXRSlJpb8Y8SfFBTSqk4ojvPdkhrs7ghzVugUJFspKKTVXV1SQ0nHGI+syaNsh/7JZSN916TV5UXqT0FyQnDbonYc0NPdWUdBIRnR/wtxbhxSzcZ9YWaV5XtdjrSQgkIy5jumlpO71PHEujwWvh/xuhxpAQWFZEwkDPePrKIgJ8Edi6Nqn5dFFBSSUT07JLh9aDXvHwgzY1nU63IkRQoKybhv9qtl7NF1PLM+l5c2pd0SRTJAQSEZFwrB9BFVFOUlmKr2eVlBQSGe6FqQYOrJ9e3zpqp9nu85CwpjzHhjTMwYc62rNSXYzu5Vx5nH1PLy5lyeXa9TED9zNQDoCOBHwAsu1pO2obF9Xhe1z/M9V0cUPwZmAhWO1pM2omn7vDsWq32eX7mY63EWUGSt/ZODeqQNGt89xnn9ali0PYcn16p9nh8lPTFMMlLQAPcDX063kFQnFmWb0tIJQP3EqSBLd/9mfRmWfACz3oxyTkmUYzs6KsyBoH/vUpF09mhzjDGjgT8DBxpe6gxUAw9aa+9NcZnewHqNFMxersbuLdke4ZJXCjixOMYTXz5Ajg/uybWRkYLpzx5tjrV2PnBk4/vGmMeBN6y1c9JZV9qmxvZ5v7Z5PLomj++W6Blvv/BBZot8TO3z/Mnpd8Ja+x0dTUg6ojnwQEP7vFvUPs83FNniO4OL41xVUsPa3REeUvs8X1BQiC9dPai+fd4v1+SxXO3zPKfvgPhSbvjj9nm3q32e5xQU4lv9vxDne2qf5wsKCvG1S5u0z1uwTe3zvKKgEF9T+zx/UFCI7zW2z9teGWb6G2qf5wUFhWSFxvZ5z25Q+zwvKCgkK6h9nrcUFJI1uhYkuLuhfd4Utc/LKAWFZJWze9dx1jG1vLI5l2fUPi9jFBSSde4eXkWXaJzpy6JsVfu8jFBQSNbpmM/B9nl3qn1eRigoJCuN7x7jfLXPyxgFhWSt24dW0719nJnl+azfq1OQ1qSgkKzVIRfuP7WK6lj9L47Vxb2uKLgUFJLVTuka41JTS/nOCL9co94VrUVBIVnve0Oq6VcYY47a57UaF3M9HjfGbDbGlDf8+b6LwkRSFc2p712h9nmtx9UTK/erV6Z4aXBxnKtLapizMp+HVuRx04n6NVOXdJwmgXHVoBpKOql9Xmtw9dWcbIxZYYx5xhhzvKM1RVokN1zfwTsnBLepfZ5TSSeFJRkp2BU4CthmrY0bYy4BpgF9rbWpnin2BtanuK1IUnOWw9R5cMUJMPM0r6vJCkknhaU1UvBQjDE7gaHW2o0pfkhvNFIwq/lt7F4sDpe80o43duTw2GkHGHX04V/d9Nu+udSSkYIu7np0b/L2GUAM2JLuuiKHS+3z3HNxjeLXDdcn3gTuAr5irdXZoXiqZ4cEd6h9njNp3x611k5wUYiIa+f3q+WlTTk8uyGXCT3rKO2p/38dLt1DksD6dPu8ikr94tjhUlBIoDW2z9tVHWbq62qfd7gUFBJ4Z/euY6La56VFQSFtwlS1z0uLgkLahI75MEPt8w6bgkLajHFN2uc9ofZ5LaKgkDalsX3eLLXPaxEFhbQpTdvn3ab2eSlTUEibc0rXGN85rpY31T4vZQoKaZOats9bo/Z5SekrJG1SfuTj9nm3LlT7vGQUFNJmNbbPW7snwk9X6BSkOQoKadMa2+c9qvZ5zdJXRto0tc9LjYJC2rxji+JMHlLNe/vCzCzLB2Dp9ojHVfmLfkNGBLj0uFpe2ZLDk+vyKMxL8I9NObww6YDXZfmGjihEgBBw20nV5IQS/HxVPu/ujfDOHv14NNJXQgRYXhHhiv8toC7x8WPd/9ikA+5GToLCGHOdMebtht6Z5S7WFMmkYV1iPH3mfgYUffxAxUsKioNcdOE+FzgfGG6tHQyckXZVIh7o2SHBU6UHOKNnLQCrdkXYtNfjonzCxRHFTcAPrLUfAVhrtztYU8QT7XPhwdFV3HhCNSESPP+O1xX5g4ugGAicaoxZaIx5wxjz7w7WFPFMKARXD6rhv8ZV8tomr6vxBxcjBXcBTwDXAJ2BBcAV1trXUqyhNxopKD71zi7oVQQ5wb7sn3RSWNKrNdbaoc39vTHmPWCutTYOfGCMeQk4BUg1KADQSMHsFeSxe/0CvG9NRgom39bB53sSOBPAGNMeGAO86WBdEfEJF0HxE6CnMWYVsBT4nbX2JQfriohPuBgpWAlc7KAWEfGpYF+iEREnFBQikpSCQkSSUlCISFIKChFJSkEhIkkpKEQkKQWFiCSloBCRpBQUIpKUgkJEklJQiEhSCgoRSUpBISJJKShEJCkFhYgkpaAQkaQUFCKSlIJCRJJKu2emMeZl6ud5NK5XAgyx1r6V7toi4g8umutOaHzbGPM1YLpCQiRYXI9rvhx4rIUfE4H6YSRBFI1GgeDuX6Mg719Q963JfkWSbZt0pGCqjDFHAe8Cvay1O1rwoaOBeU6KEJHDMQaY39wGSY8oks0etdbGGt6+BHixhSEB8Dr1hW4DYkm2FRF3IsDR1P8MNsvlEcUa4BZr7fNOFhQR33Bye9QY80WgCHjBxXoi4i+unqO4DPhNk9MQEQkQZ6ceIhJcejJTRJJSUIhIUgoKEUlKQSEiSbl+hPuwGWO+DdwKDARutNbO8biktBljBgC/BoqBncAl1tp13lblhjFmFvANoDcw2Fq70tuK3DLGFAO/BfoBNcA64LuH8UChbxljngH6AHFgH3Cdtbb8UNv66YiiHLgQeNLjOlz6OfCwtXYA8DDwC4/rcekZYCyw0eM6WksCeMBaa6y1g4F3gPs9rsm1S621Q6y1JwGzaOb3tHwTFNbaldba1dSnW9YzxhwJDAXmNrw0FxhqjOniXVXuWGvnW2s3eV1Ha7HWfmitfbXJS4uBXh6V0yqstXuavFtEMz97vjn1CKCewJbGh9CstTFjzNaG1wNz+NoWGGPCwNXAX72uxTVjzC+BUiAEnPl522UsKFrwy2UifvMQ9efwWX/d7NOstVcCGGMuBmYCEw+1XcaCwlo7NFOfyyc2Ad2NMZGGo4kI0K3hdckSDRdt+wPnWGsDcVp8KNba3xpj/tsYU2yt3fnpv/fNNYqgsdZ+QP0F2osaXroIKAvSVfOgM8b8EBgGfM1aW+11PS4ZYzoYY3o2ef8c4MOGP5/hm9/1MMZcRP2hT0fqb0ftB0obLnBmJWPMcdTfHu0I7KL+9qj1tio3jDE/Bc4FjgIqgJ3W2hJvq3LHGFMCrATWApUNL6+31n7du6rcMcZ0BZ4F2lPfB+ZD4GZr7fJDbe+boBAR/9Kph4gkpaAQkaQUFCKSlIJCRJJSUIhIUgoKEUlKQSEiSSkoRCSp/wdglPKhXVNd6gAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 288x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "v = np.array([[1], [-3]])\n",
    "Av = A.dot(v)\n",
    "v_np = [-0.31622777, 0.9486833]\n",
    "\n",
    "plotVectors([Av.flatten(), v.flatten(), v_np], cols=[blue, orange, 'blue'])\n",
    "plt.ylim(-7, 1)\n",
    "plt.xlim(-1, 3)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can see that the vector found with Numpy (in dark blue) is a scaled version of our preceding $\\begin{bmatrix}\n",
    "    1\\\\\\\\\n",
    "    -3\n",
    "\\end{bmatrix}$."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Rescaled vectors\n",
    "\n",
    "As we saw it with numpy, if $\\bs{v}$ is an eigenvector of $\\bs{A}$, then any rescaled vector $s\\bs{v}$ is also an eigenvector of $\\bs{A}$. The eigenvalue of the rescaled vector is the same.\n",
    "\n",
    "Let's try to rescale\n",
    "\n",
    "$$\n",
    "\\bs{v}=\n",
    "\\begin{bmatrix}\n",
    "    1\\\\\\\\\n",
    "    -3\n",
    "\\end{bmatrix}\n",
    "$$\n",
    "\n",
    "from our preceding example. \n",
    "\n",
    "For instance,\n",
    "\n",
    "$$\n",
    "\\bs{3v}=\n",
    "\\begin{bmatrix}\n",
    "    3\\\\\\\\\n",
    "    -9\n",
    "\\end{bmatrix}\n",
    "$$\n",
    "\n",
    "$$\n",
    "\\begin{bmatrix}\n",
    "    5 & 1\\\\\\\\\n",
    "    3 & 3\n",
    "\\end{bmatrix}\n",
    "\\begin{bmatrix}\n",
    "    3\\\\\\\\\n",
    "    -9\n",
    "\\end{bmatrix} =\n",
    "\\begin{bmatrix}\n",
    "    6\\\\\\\\\n",
    "    -18\n",
    "\\end{bmatrix} = 2 \\times\n",
    "\\begin{bmatrix}\n",
    "    3\\\\\\\\\n",
    "    -9\n",
    "\\end{bmatrix}\n",
    "$$\n",
    "\n",
    "We have well $\\bs{A}\\times 3\\bs{v} = \\lambda\\bs{v}$ and the eigenvalue is still $\\lambda=2$."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Concatenating eigenvalues and eigenvectors\n",
    "\n",
    "Now that we have an idea of what eigenvectors and eigenvalues are we can see how it can be used to decompose a matrix. All eigenvectors of a matrix $\\bs{A}$ can be concatenated in a matrix with each column corresponding to each eigenvector (like in the second array return by `np.linalg.eig(A)`):\n",
    "\n",
    "$$\n",
    "\\bs{V}=\n",
    "\\begin{bmatrix}\n",
    "    1 & 1\\\\\\\\\n",
    "    1 & -3\n",
    "\\end{bmatrix}\n",
    "$$\n",
    "\n",
    "The first column $\n",
    "\\begin{bmatrix}\n",
    "    1\\\\\\\\\n",
    "    1\n",
    "\\end{bmatrix}\n",
    "$ corresponds to $\\lambda=6$ and the second $\n",
    "\\begin{bmatrix}\n",
    "    1\\\\\\\\\n",
    "    -3\n",
    "\\end{bmatrix}\n",
    "$ to $\\lambda=2$.\n",
    "\n",
    "The vector $\\bs{\\lambda}$ can be created from all eigenvalues:\n",
    "\n",
    "$$\n",
    "\\bs{\\lambda}=\n",
    "\\begin{bmatrix}\n",
    "    6\\\\\\\\\n",
    "    2\n",
    "\\end{bmatrix}\n",
    "$$\n",
    "\n",
    "Then the eigendecomposition is given by\n",
    "\n",
    "$$\n",
    "\\bs{A}=\\bs{V}\\cdot diag(\\bs{\\lambda}) \\cdot \\bs{V}^{-1}\n",
    "$$\n",
    "\n",
    "<span class='pquote'>\n",
    "    We can decompose the matrix $\\bs{A}$ with eigenvectors and eigenvalues. It is done with: $\\bs{A}=\\bs{V}\\cdot diag(\\bs{\\lambda}) \\cdot \\bs{V}^{-1}$\n",
    "</span>\n",
    "\n",
    "$diag(\\bs{v})$ is a diagonal matrix (see [2.6](https://hadrienj.github.io/posts/Deep-Learning-Book-Series-2.6-Special-Kinds-of-Matrices-and-Vectors/)) containing all the eigenvalues. Continuing with our example we have\n",
    "\n",
    "$$\n",
    "\\bs{V}=\\begin{bmatrix}\n",
    "    1 & 1\\\\\\\\\n",
    "    1 & -3\n",
    "\\end{bmatrix}\n",
    "$$\n",
    "\n",
    "The diagonal matrix is all zeros except the diagonal that is our vector $\\bs{\\lambda}$.\n",
    "\n",
    "$$\n",
    "diag(\\bs{v})=\n",
    "\\begin{bmatrix}\n",
    "    6 & 0\\\\\\\\\n",
    "    0 & 2\n",
    "\\end{bmatrix}\n",
    "$$\n",
    "\n",
    "The inverse matrix of $\\bs{V}$ can be calculated with numpy:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[ 1,  1],\n",
       "       [ 1, -3]])"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "V = np.array([[1, 1], [1, -3]])\n",
    "V"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[ 0.75,  0.25],\n",
       "       [ 0.25, -0.25]])"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "V_inv = np.linalg.inv(V)\n",
    "V_inv"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "So let's plug\n",
    "\n",
    "$$\n",
    "\\bs{V}^{-1}=\\begin{bmatrix}\n",
    "    0.75 & 0.25\\\\\\\\\n",
    "    0.25 & -0.25\n",
    "\\end{bmatrix}\n",
    "$$\n",
    "\n",
    "into our equation:\n",
    "\n",
    "$$\n",
    "\\begin{align*}\n",
    "&\\bs{V}\\cdot diag(\\bs{\\lambda}) \\cdot \\bs{V}^{-1}\\\\\\\\\n",
    "&=\n",
    "\\begin{bmatrix}\n",
    "    1 & 1\\\\\\\\\n",
    "    1 & -3\n",
    "\\end{bmatrix}\n",
    "\\begin{bmatrix}\n",
    "    6 & 0\\\\\\\\\n",
    "    0 & 2\n",
    "\\end{bmatrix}\n",
    "\\begin{bmatrix}\n",
    "    0.75 & 0.25\\\\\\\\\n",
    "    0.25 & -0.25\n",
    "\\end{bmatrix}\n",
    "\\end{align*}\n",
    "$$\n",
    "\n",
    "If we do the dot product of the first two matrices we have:\n",
    "\n",
    "$$\n",
    "\\begin{bmatrix}\n",
    "    1 & 1\\\\\\\\\n",
    "    1 & -3\n",
    "\\end{bmatrix}\n",
    "\\begin{bmatrix}\n",
    "    6 & 0\\\\\\\\\n",
    "    0 & 2\n",
    "\\end{bmatrix} =\n",
    "\\begin{bmatrix}\n",
    "    6 & 2\\\\\\\\\n",
    "    6 & -6\n",
    "\\end{bmatrix}\n",
    "$$\n",
    "\n",
    "So with replacing into the equation:\n",
    "\n",
    "$$\n",
    "\\begin{align*}\n",
    "&\\begin{bmatrix}\n",
    "    6 & 2\\\\\\\\\n",
    "    6 & -6\n",
    "\\end{bmatrix}\n",
    "\\begin{bmatrix}\n",
    "    0.75 & 0.25\\\\\\\\\n",
    "    0.25 & -0.25\n",
    "\\end{bmatrix}\\\\\\\\\n",
    "&=\n",
    "\\begin{bmatrix}\n",
    "    6\\times0.75 + (2\\times0.25) & 6\\times0.25 + (2\\times-0.25)\\\\\\\\\n",
    "    6\\times0.75 + (-6\\times0.25) & 6\\times0.25 + (-6\\times-0.25)\n",
    "\\end{bmatrix}\\\\\\\\\n",
    "&=\n",
    "\\begin{bmatrix}\n",
    "    5 & 1\\\\\\\\\n",
    "    3 & 3\n",
    "\\end{bmatrix}=\n",
    "\\bs{A}\n",
    "\\end{align*}\n",
    "$$\n",
    "\n",
    "Let's check our result with Python:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[6, 0],\n",
       "       [0, 2]])"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "lambdas = np.diag([6,2])\n",
    "lambdas"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[5., 1.],\n",
       "       [3., 3.]])"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "V.dot(lambdas).dot(V_inv)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "That confirms our previous calculation. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Real symmetric matrix\n",
    "\n",
    "In the case of real symmetric matrices (more details about symmetric matrices in [2.6](https://hadrienj.github.io/posts/Deep-Learning-Book-Series-2.6-Special-Kinds-of-Matrices-and-Vectors/)), the eigendecomposition can be expressed as\n",
    "\n",
    "$$\n",
    "\\bs{A} = \\bs{Q}\\Lambda \\bs{Q}^\\text{T}\n",
    "$$\n",
    "\n",
    "where $\\bs{Q}$ is the matrix with eigenvectors as columns and $\\Lambda$ is $diag(\\lambda)$.\n",
    "\n",
    "### Example 3.\n",
    "\n",
    "$$\n",
    "\\bs{A}=\\begin{bmatrix}\n",
    "    6 & 2\\\\\\\\\n",
    "    2 & 3\n",
    "\\end{bmatrix}\n",
    "$$\n",
    "\n",
    "This matrix is symmetric because $\\bs{A}=\\bs{A}^\\text{T}$. Its eigenvectors are:\n",
    "\n",
    "$$\n",
    "\\bs{Q}=\n",
    "\\begin{bmatrix}\n",
    "    0.89442719 & -0.4472136\\\\\\\\\n",
    "    0.4472136 & 0.89442719\n",
    "\\end{bmatrix}\n",
    "$$\n",
    "\n",
    "and its eigenvalues put in a diagonal matrix gives:\n",
    "\n",
    "$$\n",
    "\\bs{\\Lambda}=\n",
    "\\begin{bmatrix}\n",
    "    7 & 0\\\\\\\\\n",
    "    0 & 2\n",
    "\\end{bmatrix}\n",
    "$$\n",
    "\n",
    "So let's begin to calculate $\\bs{Q\\Lambda}$:\n",
    "\n",
    "$$\n",
    "\\begin{align*}\n",
    "\\bs{Q\\Lambda}&=\n",
    "\\begin{bmatrix}\n",
    "    0.89442719 & -0.4472136\\\\\\\\\n",
    "    0.4472136 & 0.89442719\n",
    "\\end{bmatrix}\n",
    "\\begin{bmatrix}\n",
    "    7 & 0\\\\\\\\\n",
    "    0 & 2\n",
    "\\end{bmatrix}\\\\\\\\\n",
    "&=\n",
    "\\begin{bmatrix}\n",
    "    0.89442719 \\times 7 & -0.4472136\\times 2\\\\\\\\\n",
    "    0.4472136 \\times 7 & 0.89442719\\times 2\n",
    "\\end{bmatrix}\\\\\\\\\n",
    "&=\n",
    "\\begin{bmatrix}\n",
    "    6.26099033 & -0.8944272\\\\\\\\\n",
    "    3.1304952 & 1.78885438\n",
    "\\end{bmatrix}\n",
    "\\end{align*}\n",
    "$$\n",
    "\n",
    "with:\n",
    "\n",
    "$$\n",
    "\\bs{Q}^\\text{T}=\n",
    "\\begin{bmatrix}\n",
    "    0.89442719 & 0.4472136\\\\\\\\\n",
    "    -0.4472136 & 0.89442719\n",
    "\\end{bmatrix}\n",
    "$$\n",
    "\n",
    "So we have:\n",
    "\n",
    "$$\n",
    "\\begin{align*}\n",
    "\\bs{Q\\Lambda} \\bs{Q}^\\text{T}&=\n",
    "\\begin{bmatrix}\n",
    "    6.26099033 & -0.8944272\\\\\\\\\n",
    "    3.1304952 & 1.78885438\n",
    "\\end{bmatrix}\n",
    "\\begin{bmatrix}\n",
    "    0.89442719 & 0.4472136\\\\\\\\\n",
    "    -0.4472136 & 0.89442719\n",
    "\\end{bmatrix}\\\\\\\\\n",
    "&=\n",
    "\\begin{bmatrix}\n",
    "    6 & 2\\\\\\\\\n",
    "    2 & 3\n",
    "\\end{bmatrix}\n",
    "\\end{align*}\n",
    "$$\n",
    "\n",
    "It works! For that reason, it can useful to use symmetric matrices! Let's do the same things easily with `linalg` from numpy:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[6, 2],\n",
       "       [2, 3]])"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "A = np.array([[6, 2], [2, 3]])\n",
    "A"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[ 0.89442719, -0.4472136 ],\n",
       "       [ 0.4472136 ,  0.89442719]])"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "eigVals, eigVecs = np.linalg.eig(A)\n",
    "eigVecs"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[7., 0.],\n",
       "       [0., 2.]])"
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "eigVals = np.diag(eigVals)\n",
    "eigVals"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[6., 2.],\n",
       "       [2., 3.]])"
      ]
     },
     "execution_count": 24,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "eigVecs.dot(eigVals).dot(eigVecs.T)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can see that the result corresponds to our initial matrix."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Quadratic form to matrix form\n",
    "\n",
    "Eigendecomposition can be used to optimize quadratic functions. We will see that when $\\bs{x}$ takes the values of an eigenvector, $f(\\bs{x})$ takes the value of its corresponding eigenvalue.\n",
    "\n",
    "<span class='pquote'>\n",
    "    When $\\bs{x}$ takes the values of an eigenvector, $f(\\bs{x})$ takes the value of its corresponding eigenvalue.\n",
    "</span>\n",
    "\n",
    "We will see in the following points how we can show that with different methods.\n",
    "\n",
    "Let's have the following quadratic equation:\n",
    "\n",
    "$$\n",
    "f(\\bs{x}) = ax_1^2 +(b+c)x_1x_2 + dx_2^2\n",
    "$$\n",
    "\n",
    "These quadratic forms can be generated by matrices:\n",
    "\n",
    "$$\n",
    "f(\\bs{x})= \\begin{bmatrix}\n",
    "    x_1 & x_2\n",
    "\\end{bmatrix}\\begin{bmatrix}\n",
    "    a & b\\\\\\\\\n",
    "    c & d\n",
    "\\end{bmatrix}\\begin{bmatrix}\n",
    "    x_1\\\\\\\\\n",
    "    x_2\n",
    "\\end{bmatrix} = \\bs{x^\\text{T}Ax}\n",
    "$$\n",
    "\n",
    "with:\n",
    "\n",
    "$$\n",
    "\\bs{x} = \\begin{bmatrix}\n",
    "    x_1\\\\\\\\\n",
    "    x_2\n",
    "\\end{bmatrix}\n",
    "$$\n",
    "\n",
    "and\n",
    "\n",
    "$$\n",
    "\\bs{A}=\\begin{bmatrix}\n",
    "    a & b\\\\\\\\\n",
    "    c & d\n",
    "\\end{bmatrix}\n",
    "$$\n",
    "\n",
    "We call them matrix forms. This form is useful to do various things on the quadratic equation like constrained optimization (see bellow).\n",
    "\n",
    "<span class='pquote'>\n",
    "    Quadratic equations can be expressed under the matrix form\n",
    "</span>\n",
    "\n",
    "If you look at the relation between these forms you can see that $a$ gives you the number of $x_1^2$, $(b + c)$ the number of $x_1x_2$ and $d$ the number of $x_2^2$. This means that the same quadratic form can be obtained from infinite number of matrices $\\bs{A}$ by changing $b$ and $c$ while preserving their sum.\n",
    "\n",
    "### Example 4.\n",
    "\n",
    "$$\n",
    "\\bs{x} = \\begin{bmatrix}\n",
    "    x_1\\\\\\\\\n",
    "    x_2\n",
    "\\end{bmatrix}\n",
    "$$\n",
    "\n",
    "and\n",
    "\n",
    "$$\n",
    "\\bs{A}=\\begin{bmatrix}\n",
    "    2 & 4\\\\\\\\\n",
    "    2 & 5\n",
    "\\end{bmatrix}\n",
    "$$\n",
    "\n",
    "gives the following quadratic form:\n",
    "\n",
    "$$\n",
    "2x_1^2 + (4+2)x_1x_2 + 5x_2^2\\\\\\\\=2x_1^2 + 6x_1x_2 + 5x_2^2\n",
    "$$\n",
    "\n",
    "but if:\n",
    "\n",
    "$$\n",
    "\\bs{A}=\\begin{bmatrix}\n",
    "    2 & -3\\\\\\\\\n",
    "    9 & 5\n",
    "\\end{bmatrix}\n",
    "$$\n",
    "\n",
    "we still have the quadratic same form:\n",
    "\n",
    "$$\n",
    "2x_1^2 + (-3+9)x_1x_2 + 5x_2^2\\\\\\\\=2x_1^2 + 6x_1x_2 + 5x_2^2\n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Example 5\n",
    "\n",
    "For this example, we will go from the matrix form to the quadratic form using a symmetric matrix $\\bs{A}$. Let's use the matrix of the example 3.\n",
    "\n",
    "$$\n",
    "\\bs{x} = \\begin{bmatrix}\n",
    "    x_1\\\\\\\\\n",
    "    x_2\n",
    "\\end{bmatrix}\n",
    "$$\n",
    "\n",
    "and\n",
    "\n",
    "$$\\bs{A}=\\begin{bmatrix}\n",
    "    6 & 2\\\\\\\\\n",
    "    2 & 3\n",
    "\\end{bmatrix}\n",
    "$$\n",
    "\n",
    "$$\n",
    "\\begin{align*}\n",
    "\\bs{x^\\text{T}Ax}&=\n",
    "\\begin{bmatrix}\n",
    "    x_1 & x_2\n",
    "\\end{bmatrix}\n",
    "\\begin{bmatrix}\n",
    "    6 & 2\\\\\\\\\n",
    "    2 & 3\n",
    "\\end{bmatrix}\n",
    "\\begin{bmatrix}\n",
    "    x_1\\\\\\\\\n",
    "    x_2\n",
    "\\end{bmatrix}\\\\\\\\\n",
    "&=\n",
    "\\begin{bmatrix}\n",
    "    x_1 & x_2\n",
    "\\end{bmatrix}\n",
    "\\begin{bmatrix}\n",
    "    6 x_1 + 2 x_2\\\\\\\\\n",
    "    2 x_1 + 3 x_2\n",
    "\\end{bmatrix}\\\\\\\\\n",
    "&=\n",
    "x_1(6 x_1 + 2 x_2) + x_2(2 x_1 + 3 x_2)\\\\\\\\\n",
    "&=\n",
    "6 x_1^2 + 4 x_1x_2 + 3 x_2^2\n",
    "\\end{align*}\n",
    "$$\n",
    "\n",
    "Our quadratic equation is thus $6 x_1^2 + 4 x_1x_2 + 3 x_2^2$.\n",
    "\n",
    "### Note\n",
    "\n",
    "If $\\bs{A}$ is a diagonal matrix (all 0 except the diagonal), the quadratic form of $\\bs{x^\\text{T}Ax}$ will have no cross term. Take the following matrix form:\n",
    "\n",
    "$$\n",
    "\\bs{A}=\\begin{bmatrix}\n",
    "    a & b\\\\\\\\\n",
    "    c & d\n",
    "\\end{bmatrix}\n",
    "$$\n",
    "\n",
    "If $\\bs{A}$ is diagonal, then $b$ and $c$ are 0 and since $f(\\bs{x}) = ax_1^2 +(b+c)x_1x_2 + dx_2^2$ there is no cross term. A quadratic form without cross term is called diagonal form since it comes from a diagonal matrix."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Change of variable \n",
    "\n",
    "A change of variable (or linear substitution) simply means that we replace a variable by another one. We will see that it can be used to remove the cross terms in our quadratic equation. Without the cross term, it will then be easier to characterize the function and eventually optimize it (i.e finding its maximum or minimum).\n",
    "\n",
    "## With the quadratic form\n",
    "\n",
    "### Example 6.\n",
    "\n",
    "Let's take again our previous quadratic form:\n",
    "\n",
    "$$\n",
    "\\bs{x^\\text{T}Ax} = 6 x_1^2 + 4 x_1x_2 + 3 x_2^2\n",
    "$$\n",
    "\n",
    "The change of variable will concern $x_1$ and $x_2$. We can replace $x_1$ with any combination of $y_1$ and $y_2$ and $x_2$ with any combination $y_1$ and $y_2$. We will of course end up with a new equation. The nice thing is that we can find a specific substitution that will lead to a simplification of our statement. Specifically, it can be used to get rid of the cross term (in our example: $4 x_1x_2$). We will see later why it is interesting.\n",
    "\n",
    "Actually, the right substitution is given by the eigenvectors of the matrix used to generate the quadratic form. Let's recall that the matrix form of our equation is:\n",
    "\n",
    "$$\n",
    "\\bs{x} = \\begin{bmatrix}\n",
    "    x_1\\\\\\\\\n",
    "    x_2\n",
    "\\end{bmatrix}\n",
    "$$\n",
    "\n",
    "and\n",
    "\n",
    "$$\\bs{A}=\\begin{bmatrix}\n",
    "    6 & 2\\\\\\\\\n",
    "    2 & 3\n",
    "\\end{bmatrix}\n",
    "$$\n",
    "\n",
    "and that the eigenvectors of $\\bs{A}$ are:\n",
    "\n",
    "$$\n",
    "\\begin{bmatrix}\n",
    "    0.89442719 & -0.4472136\\\\\\\\\n",
    "    0.4472136 & 0.89442719\n",
    "\\end{bmatrix}\n",
    "$$\n",
    "\n",
    "With the purpose of simplification, we can replace these values with:\n",
    "\n",
    "$$\n",
    "\\begin{bmatrix}\n",
    "    \\frac{2}{\\sqrt{5}} & -\\frac{1}{\\sqrt{5}}\\\\\\\\\n",
    "    \\frac{1}{\\sqrt{5}} & \\frac{2}{\\sqrt{5}}\n",
    "\\end{bmatrix} =\n",
    "\\frac{1}{\\sqrt{5}}\n",
    "\\begin{bmatrix}\n",
    "    2 & -1\\\\\\\\\n",
    "    1 & 2\n",
    "\\end{bmatrix}\n",
    "$$\n",
    "\n",
    "So our first eigenvector is:\n",
    "\n",
    "$$\n",
    "\\frac{1}{\\sqrt{5}}\n",
    "\\begin{bmatrix}\n",
    "    2\\\\\\\\\n",
    "    1\n",
    "\\end{bmatrix}\n",
    "$$\n",
    "\n",
    "and our second eigenvector is:\n",
    "\n",
    "$$\n",
    "\\frac{1}{\\sqrt{5}}\n",
    "\\begin{bmatrix}\n",
    "    -1\\\\\\\\\n",
    "    2\n",
    "\\end{bmatrix}\n",
    "$$\n",
    "\n",
    "The change of variable will lead to:\n",
    "\n",
    "$$\n",
    "\\begin{bmatrix}\n",
    "    x_1\\\\\\\\\n",
    "    x_2\n",
    "\\end{bmatrix} =\n",
    "\\frac{1}{\\sqrt{5}}\n",
    "\\begin{bmatrix}\n",
    "    2 & -1\\\\\\\\\n",
    "    1 & 2\n",
    "\\end{bmatrix}\n",
    "\\begin{bmatrix}\n",
    "    y_1\\\\\\\\\n",
    "    y_2\n",
    "\\end{bmatrix} =\n",
    "\\frac{1}{\\sqrt{5}}\n",
    "\\begin{bmatrix}\n",
    "    2y_1 - y_2\\\\\\\\\n",
    "    y_1 + 2y_2\n",
    "\\end{bmatrix}\n",
    "$$\n",
    "\n",
    "so we have\n",
    "\n",
    "$$\n",
    "\\begin{cases}\n",
    "x_1 = \\frac{1}{\\sqrt{5}}(2y_1 - y_2)\\\\\\\\\n",
    "x_2 = \\frac{1}{\\sqrt{5}}(y_1 + 2y_2)\n",
    "\\end{cases}\n",
    "$$\n",
    "\n",
    "So far so good! Let's replace that in our example:\n",
    "\n",
    "$$\n",
    "\\begin{align*}\n",
    "\\bs{x^\\text{T}Ax}\n",
    "&=\n",
    "6 x_1^2 + 4 x_1x_2 + 3 x_2^2\\\\\\\\\n",
    "&=\n",
    "6 [\\frac{1}{\\sqrt{5}}(2y_1 - y_2)]^2 + 4 [\\frac{1}{\\sqrt{5}}(2y_1 - y_2)\\frac{1}{\\sqrt{5}}(y_1 + 2y_2)] + 3 [\\frac{1}{\\sqrt{5}}(y_1 + 2y_2)]^2\\\\\\\\\n",
    "&=\n",
    "\\frac{1}{5}[6 (2y_1 - y_2)^2 + 4 (2y_1 - y_2)(y_1 + 2y_2) + 3 (y_1 + 2y_2)^2]\\\\\\\\\n",
    "&=\n",
    "\\frac{1}{5}[6 (4y_1^2 - 4y_1y_2 + y_2^2) + 4 (2y_1^2 + 4y_1y_2 - y_1y_2 - 2y_2^2) + 3 (y_1^2 + 4y_1y_2 + 4y_2^2)]\\\\\\\\\n",
    "&=\n",
    "\\frac{1}{5}(24y_1^2 - 24y_1y_2 + 6y_2^2 + 8y_1^2 + 16y_1y_2 - 4y_1y_2 - 8y_2^2 + 3y_1^2 + 12y_1y_2 + 12y_2^2)\\\\\\\\\n",
    "&=\n",
    "\\frac{1}{5}(35y_1^2 + 10y_2^2)\\\\\\\\\n",
    "&=\n",
    "7y_1^2 + 2y_2^2\n",
    "\\end{align*}\n",
    "$$\n",
    "\n",
    "That's great! Our new equation doesn't have any cross terms!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## With the Principal Axes Theorem\n",
    "\n",
    "Actually there is a simpler way to do the change of variable. We can stay in the matrix form. Recall that we start with the form:\n",
    "\n",
    "<div>\n",
    "$$\n",
    "f(\\bs{x})=\\bs{x^\\text{T}Ax}\n",
    "$$\n",
    "</div>\n",
    "\n",
    "The linear substitution can be wrote in these terms. We want replace the variables $\\bs{x}$ by $\\bs{y}$ that relates by:\n",
    "\n",
    "<div>\n",
    "$$\n",
    "\\bs{x}=P\\bs{y}\n",
    "$$\n",
    "</div>\n",
    "\n",
    "We want to find $P$ such as our new equation (after the change of variable) doesn't contain the cross terms. The first step is to replace that in the first equation:\n",
    "\n",
    "<div>\n",
    "$$\n",
    "\\begin{align*}\n",
    "\\bs{x^\\text{T}Ax}\n",
    "&=\n",
    "(\\bs{Py})^\\text{T}\\bs{A}(\\bs{Py})\\\\\\\\\n",
    "&=\n",
    "\\bs{y}^\\text{T}(\\bs{P}^\\text{T}\\bs{AP})\\bs{y}\n",
    "\\end{align*}\n",
    "$$\n",
    "</div>\n",
    "\n",
    "Can you see the how to transform the left hand side ($\\bs{x}$) into the right hand side ($\\bs{y}$)? The substitution is done by replacing $\\bs{A}$ with $\\bs{P^\\text{T}AP}$. We also know that $\\bs{A}$ is symmetric and thus that there is a diagonal matrix $\\bs{D}$ containing the eigenvectors of $\\bs{A}$ and such as $\\bs{D}=\\bs{P}^\\text{T}\\bs{AP}$. We thus end up with:\n",
    "\n",
    "<div>\n",
    "$$\n",
    "\\bs{x^\\text{T}Ax}=\\bs{y^\\text{T}\\bs{D} y}\n",
    "$$\n",
    "</div>\n",
    "\n",
    "<span class='pquote'>\n",
    "    We can use $\\bs{D}$ to simplify our quadratic equation and remove the cross terms\n",
    "</span>\n",
    "\n",
    "All of this implies that we can use $\\bs{D}$ to simplify our quadratic equation and remove the cross terms. If you remember from example 2 we know that the eigenvalues of $\\bs{A}$ are:\n",
    "\n",
    "<div>\n",
    "$$\n",
    "\\bs{D}=\n",
    "\\begin{bmatrix}\n",
    "    7 & 0\\\\\\\\\n",
    "    0 & 2\n",
    "\\end{bmatrix}\n",
    "$$\n",
    "</div>\n",
    "\n",
    "<div>\n",
    "$$\n",
    "\\begin{align*}\n",
    "\\bs{x^\\text{T}Ax}\n",
    "&=\n",
    "\\bs{y^\\text{T}\\bs{D} y}\\\\\\\\\n",
    "&=\n",
    "\\bs{y}^\\text{T}\n",
    "\\begin{bmatrix}\n",
    "    7 & 0\\\\\\\\\n",
    "    0 & 2\n",
    "\\end{bmatrix}\n",
    "\\bs{y}\\\\\\\\\n",
    "&=\n",
    "\\begin{bmatrix}\n",
    "    y_1 & y_2\n",
    "\\end{bmatrix}\n",
    "\\begin{bmatrix}\n",
    "    7 & 0\\\\\\\\\n",
    "    0 & 2\n",
    "\\end{bmatrix}\n",
    "\\begin{bmatrix}\n",
    "    y_1\\\\\\\\\n",
    "    y_2\n",
    "\\end{bmatrix}\\\\\\\\\n",
    "&=\n",
    "\\begin{bmatrix}\n",
    "    7y_1 +0y_2 & 0y_1 + 2y_2\n",
    "\\end{bmatrix}\n",
    "\\begin{bmatrix}\n",
    "    y_1\\\\\\\\\n",
    "    y_2\n",
    "\\end{bmatrix}\\\\\\\\\n",
    "&=\n",
    "7y_1^2 + 2y_2^2\n",
    "\\end{align*}\n",
    "$$\n",
    "</div>\n",
    "\n",
    "That's nice! If you look back to the change of variable that we have done in the quadratic form, you will see that we have found the same values!\n",
    "\n",
    "This form (without cross-term) is called the **principal axes form**.\n",
    "\n",
    "### Summary\n",
    "\n",
    "To summarise, the principal axes form can be found with\n",
    "\n",
    "$$\n",
    "\\bs{x^\\text{T}Ax} = \\lambda_1y_1^2 + \\lambda_2y_2^2\n",
    "$$\n",
    "\n",
    "where $\\lambda_1$ is the eigenvalue corresponding to the first eigenvector and $\\lambda_2$ the eigenvalue corresponding to the second eigenvector (second column of $\\bs{x}$)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Finding f(x) with eigendecomposition\n",
    "\n",
    "We will see that there is a way to find $f(\\bs{x})$ with eigenvectors and eigenvalues when $\\bs{x}$ is a unit vector. \n",
    "\n",
    "Let's start from:\n",
    "\n",
    "$$\n",
    "f(\\bs{x}) =\\bs{x^\\text{T}Ax}\n",
    "$$\n",
    "\n",
    "We know that if $\\bs{x}$ is an eigenvector of $\\bs{A}$ and $\\lambda$ the corresponding eigenvalue, then $\n",
    "\\bs{Ax}=\\lambda \\bs{x}\n",
    "$. By replacing the term in the last equation we have:\n",
    "\n",
    "$$\n",
    "f(\\bs{x}) =\\bs{x^\\text{T}\\lambda x} = \\bs{x^\\text{T}x}\\lambda\n",
    "$$\n",
    "\n",
    "Since $\\bs{x}$ is a unit vector, $\\norm{\\bs{x}}_2=1$ and $\\bs{x^\\text{T}x}=1$ (cf. [2.5](https://hadrienj.github.io/posts/Deep-Learning-Book-Series-2.5-Norms/) Norms). We end up with\n",
    "\n",
    "$$\n",
    "f(\\bs{x}) = \\lambda\n",
    "$$\n",
    "\n",
    "This is a usefull property. If $\\bs{x}$ is an eigenvector of $\\bs{A}$, $\n",
    "f(\\bs{x}) =\\bs{x^\\text{T}Ax}$ will take the value of the corresponding eigenvalue. We can see that this is working only if the euclidean norm of $\\bs{x}$ is 1 (i.e $\\bs{x}$ is a unit vector).\n",
    "\n",
    "### Example 7\n",
    "\n",
    "This example will show that $f(\\bs{x}) = \\lambda$. Let's take again the last example, the eigenvectors of $\\bs{A}$ were\n",
    "\n",
    "$$\n",
    "\\bs{Q}=\n",
    "\\begin{bmatrix}\n",
    "    0.89442719 & -0.4472136\\\\\\\\\n",
    "    0.4472136 & 0.89442719\n",
    "\\end{bmatrix}\n",
    "$$\n",
    "\n",
    "and the eigenvalues\n",
    "\n",
    "$$\n",
    "\\bs{\\Lambda}=\n",
    "\\begin{bmatrix}\n",
    "    7 & 0\\\\\\\\\n",
    "    0 & 2\n",
    "\\end{bmatrix}\n",
    "$$\n",
    "\n",
    "So if:\n",
    "\n",
    "$$\n",
    "\\bs{x}=\\begin{bmatrix}\n",
    "    0.89442719 & 0.4472136\n",
    "\\end{bmatrix}\n",
    "$$\n",
    "\n",
    "$f(\\bs{x})$ should be equal to 7. Let's check that's true.\n",
    "\n",
    "$$\n",
    "\\begin{align*}\n",
    "f(\\bs{x}) &= 6 x_1^2 + 4 x_1x_2 + 3 x_2^2\\\\\\\\\n",
    "&= 6\\times 0.89442719^2 + 4\\times 0.89442719\\times 0.4472136 + 3 \\times 0.4472136^2\\\\\\\\\n",
    "&= 7\n",
    "\\end{align*}\n",
    "$$\n",
    "\n",
    "In the same way, if $\\bs{x}=\\begin{bmatrix}\n",
    "    -0.4472136 & 0.89442719\n",
    "\\end{bmatrix}$, $f(\\bs{x})$ should be equal to 2.\n",
    "\n",
    "$$\n",
    "\\begin{align*}\n",
    "f(\\bs{x}) &= 6 x_1^2 + 4 x_1x_2 + 3 x_2^2\\\\\\\\\n",
    "&= 6\\times -0.4472136^2 + 4\\times -0.4472136\\times 0.89442719 + 3 \\times 0.89442719^2\\\\\\\\\n",
    "&= 2\n",
    "\\end{align*}\n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Quadratic form optimization\n",
    "\n",
    "Depending to the context, optimizing a function means finding its maximum or its minimum. It is for instance widely used to minimize the error of cost functions in machine learning.\n",
    "\n",
    "Here we will see how eigendecomposition can be used to optimize quadratic functions and why this can be done easily without cross terms. The difficulty is that we want a constrained optimization, that is to find the minimum or the maximum of the function for $f(\\bs{x})$ being a unit vector.\n",
    "\n",
    "### Example 7.\n",
    "\n",
    "We want to optimize:\n",
    "\n",
    "$$\n",
    "f(\\bs{x}) =\\bs{x^\\text{T}Ax} \\textrm{ subject to }||\\bs{x}||_2= 1\n",
    "$$\n",
    "\n",
    "In our last example we ended up with:\n",
    "\n",
    "$$\n",
    "f(\\bs{x}) = 7y_1^2 + 2y_2^2\n",
    "$$\n",
    "\n",
    "And the constraint of $\\bs{x}$ being a unit vector imply:\n",
    "\n",
    "$$\n",
    "||\\bs{x}||_2 = 1 \\Leftrightarrow x_1^2 + x_2^2 = 1\n",
    "$$\n",
    "\n",
    "We can also show that $\\bs{y}$ has to be a unit vector if it is the case for $\\bs{x}$. Recall first that $\\bs{x}=\\bs{Py}$:\n",
    "\n",
    "$$\n",
    "\\begin{align*}\n",
    "||\\bs{x}||^2 &= \\bs{x^\\text{T}x}\\\\\\\\\n",
    "&= (\\bs{Py})^\\text{T}(\\bs{Py})\\\\\\\\\n",
    "&= \\bs{P^\\text{T}y^\\text{T}Py}\\\\\\\\\n",
    "&= \\bs{PP^\\text{T}y^\\text{T}y}\\\\\\\\\n",
    "&= \\bs{y^\\text{T}y} = ||\\bs{y}||^2\n",
    "\\end{align*}\n",
    "$$\n",
    "\n",
    "So $\\norm{\\bs{x}}^2 = \\norm{\\bs{y}}^2 = 1$ and thus $y_1^2 + y_2^2 = 1$\n",
    "\n",
    "Since $y_1^2$ and $y_2^2$ cannot be negative because they are squared values, we can be sure that $2y_2^2\\leq7y_2^2$. Hence:\n",
    "\n",
    "$$\n",
    "\\begin{align*}\n",
    "f(\\bs{x}) &= 7y_1^2 + 2y_2^2\\\\\\\\\n",
    "&\\leq\n",
    "7y_1^2 + 7y_2^2\\\\\\\\\n",
    "&=\n",
    "7(y_1^2+y_2^2)\\\\\\\\\n",
    "&=\n",
    "7\n",
    "\\end{align*}\n",
    "$$\n",
    "\n",
    "This means that the maximum value of $f(\\bs{x})$ is 7.\n",
    "\n",
    "The same way can lead to find the minimum of $f(\\bs{x})$. $7y_1^2\\geq2y_1^2$ and:\n",
    "\n",
    "$$\n",
    "\\begin{align*}\n",
    "f(\\bs{x}) &= 7y_1^2 + 2y_2^2\\\\\\\\\n",
    "&\\geq\n",
    "2y_1^2 + 2y_2^2\\\\\\\\\n",
    "&=\n",
    "2(y_1^2+y_2^2)\\\\\\\\\n",
    "&=\n",
    "2\n",
    "\\end{align*}\n",
    "$$\n",
    "\n",
    "And the minimum of $f(\\bs{x})$ is 2.\n",
    "\n",
    "### Summary\n",
    "\n",
    "We can note that the minimum of $f(\\bs{x})$ is the minimum eigenvalue of the corresponding matrix $\\bs{A}$. Another useful fact is that this value is obtained when $\\bs{x}$ takes the value of the corresponding eigenvector (check back the preceding paragraph). In that way, $f(\\bs{x})=7$ when $\\bs{x}=\\begin{bmatrix}0.89442719 & 0.4472136\\end{bmatrix}$. This shows how useful are the eigenvalues and eigenvector in this kind of constrained optimization."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Graphical views\n",
    "\n",
    "We saw that the quadratic functions $f(\\bs{x}) = ax_1^2 +2bx_1x_2 + cx_2^2$ can be represented by the symmetric matrix $\\bs{A}$:\n",
    "\n",
    "$$\n",
    "\\bs{A}=\\begin{bmatrix}\n",
    "    a & b\\\\\\\\\n",
    "    b & c\n",
    "\\end{bmatrix}\n",
    "$$\n",
    "\n",
    "Graphically, these functions can take one of three general shapes (click on the links to go to the Surface Plotter and move the shapes):\n",
    "\n",
    "1.[Positive-definite form](https://academo.org/demos/3d-surface-plotter/?expression=x*x%2By*y&xRange=-50%2C+50&yRange=-50%2C+50&resolution=49) | 2.[Negative-definite form](https://academo.org/demos/3d-surface-plotter/?expression=-x*x-y*y&xRange=-50%2C+50&yRange=-50%2C+50&resolution=25) | 3.[Indefinite form](https://academo.org/demos/3d-surface-plotter/?expression=x*x-y*y&xRange=-50%2C+50&yRange=-50%2C+50&resolution=49)\n",
    ":-------------------------:|:-------------------------:|:-------:\n",
    "<img src=\"images/quadratic-functions-positive-definite-form.png\" alt=\"Quadratic function with a positive definite form\" title=\"Quadratic function with a positive definite form\"> | <img src=\"images/quadratic-functions-negative-definite-form.png\" alt=\"Quadratic function with a negative definite form\" title=\"Quadratic function with a negative definite form\"> | <img src=\"images/quadratic-functions-indefinite-form.png\" alt=\"Quadratic function with a indefinite form\" title=\"Quadratic function with a indefinite form\">\n",
    "\n",
    "\n",
    "\n",
    "With the constraints that $\\bs{x}$ is a unit vector, the minimum of the function $f(\\bs{x})$ corresponds to the smallest eigenvalue and is obtained with its corresponding eigenvector. The maximum corresponds to the biggest eigenvalue and is obtained with its corresponding eigenvector."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Conclusion\n",
    "\n",
    "We have seen a lot of things in this chapter. We saw that linear algebra can be used to solve a variety of mathematical problems and more specifically that eigendecomposition is a powerful tool! However, it cannot be used for non square matrices. In the next chapter, we will see the Singular Value Decomposition (SVD) which is another way of decomposing matrices. The advantage of the SVD is that you can use it also with non-square matrices."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# BONUS: visualizing linear transformations\n",
    "\n",
    "We can see the effect of eigenvectors and eigenvalues in linear transformation. We will see first how linear transformation works. Linear transformation is a mapping between an input vector and an output vector. Different operations like projection or rotation are linear transformations. Every linear transformations can be though as applying a matrix on the input vector. We will see the meaning of this graphically. For that purpose, let's start by drawing the set of unit vectors (they are all vectors with a norm of 1)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAARoAAAD/CAYAAAAuTRk/AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8qNh9FAAAACXBIWXMAAAsTAAALEwEAmpwYAAAlUklEQVR4nO3deXhc9X3v8ffMaN+30WKtXuSfvO82xiyGgMFAqIkhYEIgaS8JISFpe1O6PE9Dkt72SW/Ic28JaUlIGyBQkxCIHcAG2wQwBhsbbNmWZf8sW9Zu7bs0kkaauX9o4CrGtrZz5hyNvq/n0ePR6Mycj46lj87+c/j9foQQwkxOqwMIIUKfFI0QwnRSNEII00nRCCFMJ0UjhDCdFI0QwnRhRr2RUupxYDNQACzSWpdcZJrvAw8DdYGn3tdaf9OoDEIIezKsaIBtwL8B740y3XNa6+8aOF8hhM0ZVjRa630ASimj3lIIESKMXKMZq3uUUhuAeuAxrfX+Mb4uElgFnAeGzAonhLgoF5AFHAL6x/viYBfNU8A/a629Sqkbge1KqXla65YxvHYVo2+WCSHMdTWwb7wvCmrRaK3rRzzerZSqBhYC747h5ecB2tp68PnsdX1WamocLS3dVsf4E5JpbCTT2DidDpKTYyHwezheQS0apVS21ro28Hgpw0eo9BhfPgTg8/ltVzSAZBojyTQ2dswUMKHdFkYe3n4C+AKQCexRSrVorRcopXYA39NafwT8i1JqRSDsAPDlkWs5QojQZORRp28D377I87eMePyAUfMTQkwdcmawEMJ0UjRCCNNJ0QghTCdFI4QwnRSNEMJ0UjRCCNNJ0QghTCdFI4QwnRSNEMJ0UjRCCNNJ0QghTCdFI4QwnRSNEMJ0UjRCCNNJ0QghTCdFI4QwnRSNEMJ0UjRCCNNJ0QghTCdFI4QwnRSNEMJ0UjRCCNNJ0QghTCdFI4QwnRSNEMJ0UjRCCNNJ0QghTGfY2NtKqceBzUABsEhrXXKRaVzAE8DNgB/4kdb6l0ZlEELYk5FrNNuAa4DKy0zzJWAOUAisBb6vlCowMIMQwoYMKxqt9T6tdfUok90NPK219mmtmxgup7uMyiCEsCfDNp3GKI8/XeOpAnKDnEEYrG9gkLauflq7+mnv6qer10u/d2j4Y2D4X4DYmAgGBgZxOh24nA7iosKJiwknLnr4IzE2AndSNBHhLou/I2G0YBfNpKWmxlkd4aLc7nirI3yG0Znau/qpON9BZX0Xlec7qarvoqaxi56+wYtOHxHuIjrSRWS4CxwOfEM+hnx+hnx+vIM+PP0Xf11aYhRZaXFkpsZQkJXAnNwkZmUnEhVhzo/rdPi/s1qwi6YKyAcOBT6/cA1nVC0t3fh8fqNzTYrbHU9TU5fVMf7EZDP5/X4a2jyUVbdzuqadspoOGts8n349Piac7LRYVs/PIDUhiuT4SFLiI0mOjyQuOoKoCBdOp+OymQaHfPR4vHQHPtq6+mls99DYNvxxoOQ8uw9WAeBwQHZaLAVZCczLT2Z+QQqJsRET/v4ulckO7JjJ6XRM6o98sIvmJeBBpdQrQCqwCbg6yBnEJXgHfZyqaqP4TDNHzzTT2tkPQFx0OIU5iaxfmk1+ZjzZabEkGPBLHuZykhgXSWJc5CWnaevqp6K+k4rzXVTUd3HkdBP7jp0HIMcdx4KZySyelcrcvCRcTjlbw66MPLz9BPAFIBPYo5Rq0VovUErtAL6ntf4I+DWwBigLvOyHWutzRmUQ4zc45OPY2RYOnKjn+LlW+geGiAh3sqAghdvWFjA3N4ms1BgcDsfob2aC5PhIkuPdLCt0A+Dz+6lq6OLEuVZKK9p46+Ma3jxYTUJMOCtUOquK0pmbm/SZtSlhLYffb6/NkMsoAM7JptPYXC6T3++nqqGbfcfP82FpA90eLwmxESwvTGNpYRpFecmm7JA1Yzn1e4c4fraFQ6caOXq2mQGvj8TYCK5anMW1S2aQlhQd9EyTZcdMIzadZgIV4339lNsZLCZucMjHR7qRXQerqajvIszlYGmhm3ULM1k4K2VKbnpEhrtYWZTOyqJ0+geGOHq2mf0l9ew4UMmO/ZUsmp3K+mXZLJ6VKms5FpKimQZ6+rzsLa5jz8c1tHX1k5kSw5dunMsVCzKIjQq3Op5hIiNcrJ6Xwep5GbR09LH3aB17j9XxxO+OkZEczS1r81m7IJMw19Qr1KlOiiaEefoH2f1RNW8erMLTP8S8/GTuv0mxaHYqTov2uQRLamIUd1wzi8+vK+BIWTOv76/gVztO8Yd959h4RT5XL84iPEzO1wkWKZoQNOAdYtu7Z/jN7tN0e7wsK0zjz66aSV5GaJ2bMRZhLieritJZqdwcL2/h1Q8qeH7XaV7fX8md187mtmvteV5WqJGiCSF+v5+PdBMvvlVGW1c/CwqSueOa2cyakWB1NMs5HA4Wz05j0axUTla28dI7Z3n6tVLePlrHndfMYm5uktURQ5ocdTKAHY4S1Lf28sIuzYmKNnLT43ho82KyEqMszXQhOyynT/j8fg6cqOf3752jpaOPlUXp3HtDIUmXOacnWOy0nD4hR52mOe+gj1c/qOCNDysJD3Ny7w2FXLc8m8yMRNv9sNqJ0+HgyoVZ3LRuFi/sKOW1DyopPdfK3Z+bw1WLsiw7byhUSdFMYTWN3fzi1VJqmrpZuyCDL14357Jn2YrPiooI4/Z1M1k9L4NndpzkVztO8WFpAw/cXIR7lHNwxNhJ0UxBPp+fXYeqeWXvWWIiw/j2nYtZOifN6lhTWmZKDI9+aTnvHqnlpXfO8th/HeSBm4tYMz/D6mghQYpmiunsGeCp7SWcqmpnWWEaD2wsIiFm8tcdieHNqeuW57Bodiq/+EMpP//DCU5WtrHlhsLhK9DFhEnRTCHldZ387PfH6fZ4+crGIq5eLPsSzJCWGM2j9y5j23vn2HGgkrN1HTy8aSFZqbFWR5uy5BTJKWLv0Tp+9MLHuJwO/uG+FVyzZIaUjInCXE7uXD+bv/7iEjp7Bvhfz33MiXOtVseasqRobM7n8/PC7tM8s/MUKi+Z731lFfmZ0+/EO6ssnJXKPz6wktSESP7Pb4/y9uEaqyNNSVI0NuYdHOKp7SW89XENG1bl8ld3LSEuOnSuTZoq0hKj+fv7VrBwVgq/3nWa/9592nbnctmd7KOxqd4+Lz99+Ti6up27r5/DTavzrI40rUVHhvHtzYv5zR/PsPujaro9Xv781nlygeYYSdHYUFfvAD/eWsz5lh6+dvt8rpifaXUkwfDZsVtuKCQhNpyX3y1nYNDH129fQHiYlM1oZAnZTLfHy+MvFtPQ1st37losJWNDt64tYMsNhRw+3cRPXznGQGCUB3FpUjQ20tvn5Se/KeZ8Sy+PbF7EwpmpVkcSl3Djyly+srGIE+WtPPn74wwO+ayOZGtSNDbh6R/kJ785Sk1jN9/6wkIpmSngmiUzuP9mRUl5K//5+kl8U+cC5aCTfTQ2MOTz8R/bS6is7+Kbdyxk8Wy5nGCquHZpNt0eLy+/W05cdDj33lAo5zddhBSNxfx+Py/sLqOkvJUHblYsm+u2OpIYp1uuyKer18uuQ9UkxkZw25UFVkeyHSkai715sJp3jtSy8Yo8rl2abXUcMQEOh4MvXj+Hzp4BXtlbTrY79tPhYcQw2UdjoaNnmnnp7TOsLEpn87WzrY4jJsHpcPCVjUXkZ8bz9Kul1DX3WB3JVqRoLNLc4eGXr5WSmx7H/7h1XsjfLHw6iAh38cgXFhER5uSnLx+jt89rdSTbkKKxwOCQj6e2n8Dn9/ONOxaaMlibsEZKQhQP37GI5o4+/vP1k0yhW+WaSorGAi+9fZbyuk6+unEeGckxVscRBpubm8Tma2dzpKyZ9wLjhE93UjRBVnKuhd0fVfO5FTmsLEq3Oo4wyYbVuczLT2brnjIaWnutjmM5KZog8vQP8uzOU2SlxvDF62TnbyhzOhz8xa3zcDkd/OLV0ml/5rBhh7eVUnOBZ4FUoAW4X2tddsE03wceBuoCT72vtf6mURns7qV3ztLa2c/ff3mFjJI4DaQkRHH/zYqntp9g96FqNl6Rb3Ukyxi5RvMU8DOt9VzgZ8DPLzHdc1rrpYGPaVMyJyvbeOdILTeuymVOdqLVcUSQrJ6XwdI5aWx//xzNHR6r41jGkKJRSqUDy4Gtgae2AsuVUnLWEsNHmX79piY9KZo7rplldRwRZPfeWAjA1j1lo0wZuozadMoFarXWQwBa6yGlVF3g+aYLpr1HKbUBqAce01rvH8+MAqPl2Y7bfenba/5h71nqW3v5x79YQ86MJFtkssp0zOR2x3PvhiKeeb2U8oZu1izMsjxTsAX7EoSngH/WWnuVUjcC25VS87TWLWN9g6k2JG63x8sLb5xiwcwUCtJigjZ6pB2HVZ3Oma6cn87uDyv5xbbj5KXFXPbOfHZcTiOGxJ3Y6w3KUQ1kK6VcAIF/ZwSe/5TWul5r7Q083h34+kKDMtjS9vfO4RkY5J7r58hVvdNYmMvJ5vWzaWzzsO/49Du3xpCi0Vo3AsXAlsBTW4AjWus/2WxSSmWPeLwUKAC0ERnsqKndwzvFtaxfmk22256bfCJ4lsxOZU52In/Yd47+aXZXPiOPOj0EPKKUOg08EvgcpdQOpdTKwDT/opQqUUodBZ4Gvqy1rjcwg63sOFCJw4HcNkAAw1d537l+Nu3dA/zx4+k1bIth+2i01qeANRd5/pYRjx8wan5219rZx75j57lmyQyS4yOtjiNsYm5uEotmpbLzwyquX5EzbYbalTODTbLzQBUAG6+QYVLEn7rlijy6PV4+KAnZlfnPkKIxQbfHy95jdaxdmElaYrTVcYTNzM1NoiAznl0Hq6bNfYalaEyw79h5vIM+NqzKtTqKsCGHw8FNq/NoaPNwtKzZ6jhBIUVjMJ/fzzvFtRTmJJIjR5rEJawscpOaEMnuj6pHnzgESNEYrLSilcY2D9ctk/v/iktzOZ1cvWQGp6raaW4P/WugpGgM9u6ROuJjwlmh5F4z4vKuXDg8Cul02CksRWMgT/8gR8+2sGZehozHLEaVlhhNUV4SH5TUh/wtP+W3wUDFZc0MDvlYPS/D6ihiili3KIvGdg9lNR1WRzGVFI2BDp1qJDk+klnZCVZHEVPECuUmzOXk8OkLb3IQWqRoDNLb56XkXAuritJl6BQxZlERYRTlJ1F8pjmkN5+kaAxyoqKNwSE/K+ReX2Kcls5Jo7HNQ30I38RcisYgpRWtREW4mDVDNpvE+CyZnQZA8ZnQPXlPisYgpRWtFOUl43LKIhXjk5oYRY47jpLyVqujmEZ+KwxQ39JDU3sf8wuSrY4ipiiVl0R5XSdDvtAclkWKxgDHAqu88wpSLE4ipqrCnET6vUNUN3ZbHcUUUjQGOFPdTnRkGDNSZXhbMTGfDMFTVh2a59NI0RjgTE07+Rlxck9gMWEpCVGkJkRRVitFIy5icMhHxflO8jNDa3gMEXwFmfHUNsmmk7iIuuYevIM+KRoxaVlpsTS0evAOht4OYSmaSapr6QEgV+49IyZpRloMPr+fuubQW6uRopmkpvY+ANKS5JadYnJmpMYCUFVvr8HjjCBFM0nN7R6S4yOnzd3shXkyU4aPWtaF4H4aKZpJamr3kJEih7XF5EWEu4iNCqOls8/qKIaTopmk5o4+0qVohEGS4iJpk6IRF+r2eEmKkwHihDGS4iJolaIRIw35fPQNDBEXHW51FBEiEuMiae3stzqG4aRoJqG3bxCA2BgpGmGMmKgwevu8VscwnGFjbyul5gLPAqlAC3C/1rrsgmlcwBPAzYAf+JHW+pdGZQi23v7hoomLjrA4iQgVEWEuBrxywt7lPAX8TGs9F/gZ8POLTPMlYA5QCKwFvq+UKjAwQ1D1DwwBEB0ph7aFMcLDnAwO+fD5Quu2noYUjVIqHVgObA08tRVYrtRn7mt5N/C01tqntW4CtgF3GZHBWnIxpTBGmGv4Z8k7FFprNUat0eQCtVrrIYDAv3WB50fKAypHfF51kWmEmLZ0VTtAyF3vZNg+mmBJTbXPNUVdA///h8Httt9FlZJpbOyUaalKp+RcK1kZCURFTrlfz0sy6jupBrKVUi6t9VBgp++MwPMjVQH5wKHA5xeu4YyqpaXbNtuvbW09gUd+mprsdX2K2x0vmcbAbpm6u4cPbbe399jq/tNOp2NSf+QN+U601o1AMbAl8NQW4EhgP8xILwEPKqWcgf03m4DfGZHBCp8Me/vJTmEhJss75MPldNiqZIxg5HfzEPCIUuo08Ejgc5RSO5RSKwPT/BooB8qAA8APtdbnDMwQVLFRw+fP9HhC77wHYQ3voI+I8NAqGTBwH43W+hSw5iLP3zLi8RDwDaPmabWYqOHF1y1FIwzS2z9IdAjtm/lE6FVnEIW5nESEO6VohGE6ugdIToiyOobhpGgmKTYqnM6eAatjiBDR3t1PihSNuFBqYhQNITxmsgiuDikacTHuxGgpGmGIwSEfXb1ekuOlaMQF3ElRtHSE5p3rRXA1tnnwAzPcsVZHMZwUzSS5k6Lx+wnJ2y+K4KprDoyokWGfM5WNIkUzSZmBYXBrm3pGmVKIy6tr6cEB5KTb5zIbo0jRTFKuOw6n00FlQ6fVUcQUd76ll9TEKKIi5DwacYGIcBd5GfFU1ofeEBkiuKoausgJ0YEIpWgMMDsnkcr6Tvx+e1zsKaaebo+X8y29zM5OsDqKKaRoDDAnJ4nOXi8tHbJDWEzMmZoOAApzkqwNYhIpGgMsmp0GwMnKNouTiKmqrKadMJeDmVmhd8QJpGgMkZcZT0JsBKVSNGKCTte0k58ZT3hYaN5/WorGAA6Hg/kFyZRWtOKT/TRinLo9XsrrOpmfn2J1FNNI0RhkQUEKXb1eahrl6JMYn2Nnm/H7YWlhmtVRTCNFY5CFM1NwAEfKmq2OIqaY4jMtJMZFkJ8ZmvtnQIrGMIlxkczNTeLgyQY5zC3GbHDIR0l5C0tmp+F0hO6wPVI0Blo1L53zLb3UNsvlCGJsjp9toW9giGUhvNkEUjSGWqHScTjg0MlGq6OIKeL9knoSYiNYMDN0dwSDFI2hEmMjKMpLZv+Jejn6JEbV1TvA0TPNXDE/gzBXaP8qhvZ3Z4Frl86guaOPkvJWq6MIm/uwtIEhn591i7KsjmI6KRqDLZ/rJiE2grcP11gdRdiY3+9n79E68jLiyA3B20JcSIrGYGEuJ9csyeLY2RaaOzxWxxE2VVrRRk1TD59bkWN1lKCQojHBtUuywQFvH661OoqwqTcPVpEYG8EV8zOtjhIUUjQmSE2MYlVROn88UitjPonPqGnqpuRcK9evyPl0WOVQNz2+SwvcdmUB/QND7D5UbXUUYTNvfFhFRJiT65ZlWx0laKRoTJLjjmPFXDd7Pq6ht2/Q6jjCJmqbutl/op7rlmcTFx1udZygmfTNSZVSMcCvgBXAIPBdrfVrF5luPbADOB14ql9r/ZmxukPJbVcW8PHpJnYdqmLT1bOsjiNs4JW95URFuLh1bYHVUYLKiDWa7wKdWus5wOeBXyqlLnW8rlRrvTTwEdIlA5CfGc/KonTeOFhFW1e/1XGExc7WdXCkrJmbV+dNq7UZMKZo7gZ+DqC1LgM+AjYa8L4h4a71s/H54HfvnLU6irCQ3+/npbfPkhATzo2rcq2OE3RGFE0eUDni8yrgUktyrlLqsFLqQ6XUAwbM2/bcSdHctDqX/SfqKa+TIVmmqwMnGjhd3c6ma2aF5HAqoxn1O1ZKHWa4TC4mYxzzOgzkaq07lFIzgT1KqVqt9Z5xvAepqfY8i9LtvvS9RO6/bQEflNTz23fO8L8fuQaXMzi3A7hcJqtMx0zdvQP89p0zqPxkNn9O4RzD/78dl9NkjFo0Wuvll/u6UqoKyAeaAk/lAW9f5H06Rzw+p5TaBqwDxlU0LS3d+Hz2umDR7Y6nqanrstPctX42v3i1lK07S7lp9aV6O7iZgm26ZnruTU1nzwB/ddcSWlpGvwOjHZeT0+mY1B95IzadXgK+DqCUKgRWAW9cOJFSKksp5Qg8TgE2AMUGzH9KWDM/g6Vz0nhlbzkNrb1WxxFBcrq6nXeP1HLDilzyQnBM7bEyomh+DCQppc4ArwFf01p3ASilfqiUeigw3WagRClVDOwFntNabzdg/lOCw+Hgyzcpwl1OfrXjpNxGYhro7Rvk6VdLSUuKYtPVM62OY6lJ75XSWvcAd13ia98b8fhJ4MnJzm8qS46P5J7PFfJfO06y62A1N68xfxNKWOeF3adp6+rn7+5bTnTk9NsBPJKcGRxk6xZlsmKum5ffPfvp6IQi9Bw82cD+E/XcdmU+c7ITrY5jOSmaIHM4HHz1liJSEiL5j+0ldPUOWB1JGKy+tZdn39DMmpHA59cVWB3HFqRoLBATFc7DmxbR1TvAL1+T/TWhxNM/yE9fPobL6eCh2xfgcsqvGEjRWCY/M54tnyvkeHkLL78rZw2HAp/fz9OvltLQ6uHhTQtJS4q2OpJtTO89VBZbvyyb6qYedh6oIj0pmmuXTp/bBoSibe+do/hMM/feUEhRfrLVcWxFisZCDoeDL91YSEtHH79+8zSpCVEsnJVqdSwxAW8fruG1Dyq4anHWtLk953jIppPFXE4nD/3ZAmakxfLv20qoqJfroaaagycbeH7XaZbMTuX+mxSOEB5xcqKkaGwgOjKMv7xrMbFR4fzkxWKqGux1+rm4tJJzLTz9ailzchL5xqaFIT8+00TJUrGJlIQo/ubeZUSEu3j8xWJqmka/JkZYq7SilSdfOU5WaizfuXMxEeEuqyPZlhSNjaQnRfPovctwuRw8vvWIjOFtY0fPNPN/XzqGOyma/3nPUmKipteNrMZLisZmMpJjeHTLMhwOBz96/mPO1MrZw3bz0alGnnzlONnuWP723uUkxkZYHcn2pGhsKCs1lr//8gpio8N5fOsRisuarY4kAt4pruU/tpcwMyuBv7ln2bS7JedESdHYVHpSNP9w3wqy3bE8+cpx9h6tszrStObz+XnxrTKee0OzYGYKf333EmKi5OyQsZKisbGE2Aj+Zssy5s9M5pmdp3hh12kGh3xWx5p2PP2DPPHyMXYdquaGFTl8587F0/J2nJMhS8vmoiLC+M6di/ndO2d582A1VY1dfGPTQpLiIq2ONi2cb+nh37eVcL65l/s2zOX65XIy3kTIGs0U4HI6ufv6Qr5++wIqG7r4wTOHOF3dbnWskPf+8fP84JlDdHQP8FdfXCIlMwmyRjOFrJmfQXba8D6bf33hMBuvyGfT1TPlJDGD9Q0M8vyu03xQUo/KTeJrty8gOV7WICdDimaKyUmP47GvruLFt8rYcaCSkvIWHvz8fLLd9hwdYqo5WdHKM2+corm9j9vXFXD7upljGrVAXJ78KZyCoiPD+Oot83hk8yLauvv5wTMf8eoHFXgHZUfxRPX2eXlm50l+/GIxDoeDR+9dxqarZ0nJGETWaKawZYVuZs9I5Pldmt/vLeeDknruu3EuC2amWB1tyvD7/Rw61cjWPafp6Blg45o8/uyqmXI5gcGkaKa4hNgIHr5jEcfLW3hh92l+8ptiVhal843NS5C/xZdXXtfJj18s5mRFK7npcTyyeTEzsxKsjhWSHP6pcxvJAuDcVB1ALhi8g0Ps/LCK1/dX4vf7uXZpNretzSfRJofC7bKcmts9vLK3nAOlDSTFR7LpqplctSjLNptJdllOI40YQG4mUDHe18saTQgJD3Nx+7rhX5rdh2vZ/WEV7x2r44YVudy8Jm/any7f0NrL6/sr2X+iHqfTwW1X5vPlWxfQ09VndbSQJ0UTglISovjWXUtZvziL7fvOsfNAJXs+rmbdoiw2rMwlIyXG6ohBVdPYzesHKjl4soEwl5P1y7LZuCaPlIQoYqLCpWiCQIomhGWkxPC12xdwy9p8dh2s5r2jdbxzuJYlc9K4cWUOKj8ZZ4jeDc476ONj3cjbR2opq+kgMsLFzavz2LA6T662toAUzTSQ447jz2+dx+ZrZ/HHw7W8faSW4jPNpCZEceXCTK5clElG8tRfy/H7/dQ09bD/RD37jp2n2+MlPSmaL143h6sWZ037TUcrSdFMI4lxkdxxzSxuXZvP4dNNvF9Sz2sfVPDqBxXMyUlkpUpnaWEa6VNsmJDapm4OnWrk0KlGzrf04nQ4WFqYxnXLsplXELprbVPJpItGKXUf8CgwH/jLwBjbl5r2QeBvAQewE/i21lrOMguyiHAXVyzI5IoFmbR29rH/RD0HTjTw4ltlvPhWGTPSYlk6J41Fs1KYNSOB8DB7nVPS2zfIqao2TlS0UnqulYY2Dw5A5SVxw8pcVsx1kyCbR7ZixBpNMXAP8HeXm0gpNRN4DFgGtDBcNPcBzxmQQUxQSkIUt64t4Na1BTS29VJ8poWjZ5p582AVOw5UEuZyUJCVQGFOInNzksjPjCcxNiJod/r3+f00tnmoON9JRX0XZ2s7OHe+C5/fT2S469NyWanctjmMLz5r0kWjtS4BUEqNtmZyJ7BNa90UmP5p4KtI0dhGenIMG1bFsGFVLr19XnR1O2U1HZRVt7PrYDU7D1QBEBsVRnZaLNnuOLJSY0hNiCI5IZLkuEjiYyPGvakyOOSjp2+Q9q5+Gts9NLb10tDmobG1l+qmHjz9gwCEhznJz4jnlrX5LChIZnZ2olxQOkUEcx9NHlA54vMqIDeI8xfjEBMVzrJCN8sK3QD0e4eoON9JTVMPtU3d1DT3cKC04dMS+ITL6SAuOpzICBdR4S4iI4Y/AMJcLvr6vQz5/MPl4hmky+P9zHsAJMZG4E6OZs38DAoy45mZlcCMtBgZy3qKGrVolFKHGS6Ji8nQWg8ZG+nyAmcn2o7bHW91hM8wOlPOjKQ/+dzv99Pe1U9zh4fm9j5aOjw0t3vo6vXS1z9I38AQfQODePoHcThgyAeuMBcRTgdhLie5meEkxEaQEBtJQkw4SQlRzEiLJTM1lujI4P0NnA7/d1Yb9X9Ta73coHlVAfkjPs8Dqsf7JnIJwtgEM1NSVBhJmXHMybz8H4GxZuru9BCsUa2m+//dWI24BGFCgrnp9DKwVyn1A4Z3Bj8I/HcQ5y+EsMikN3iVUluUUjXAXcA/KaVqlFLzA1/7oVLqIQCtdTnwT8ABoAwoB56f7PyFEPYnV28bwI6rupJpbCTT2Ez26m3ZhS+EMJ0UjRDCdFI0QgjTSdEIIUwnRSOEMJ0UjRDCdFI0QgjTSdEIIUwnRSOEMJ0UjRDCdFI0QgjTSdEIIUwnRSOEMJ0UjRDCdFI0QgjTSdEIIUwnRSOEMJ0UjRDCdFI0QgjTSdEIIUwnRSOEMJ0UjRDCdFI0QgjTSdEIIUwnRSOEMJ0UjRDCdFI0QgjTSdEIIUwXNtk3UErdBzwKzAf+Umv95CWmWw/sAE4HnurXWq+Z7PyFEPY36aIBioF7gL8bw7SlWuuVBsxTCDGFTLpotNYlAEop3+TjXJYLwOl0mDybibFjLsk0NpJpdCPyuCbyeiPWaMZjrlLqMOAF/l1r/ew4XpsFkJwca0qwyUpNjbM6wmdIprGRTOOSBZwd74tGLZpAMeRd4ssZWuuhMc7rMJCrte5QSs0E9iilarXWe8b4+kPA1cB5YKzzFEIYw8VwyRyayItHLRqt9fKJvPFF3qdzxONzSqltwDpgrEXTD+wzIosQYkLGvSbziaAd3lZKZSmlHIHHKcAGhnckCyFCnMPv90/qDZRSW4AfA8nAANADbNBalyqlfgjUaa2fUkp9C/gGw/tnwoBntdY/ntTMhRBTwqSLRgghRiNnBgshTCdFI4QwnRSNEMJ0UjRCCNMF+8zgcbHjBZtjzRSY9kHgbwEHsBP4ttbalEs1lFIxwK+AFcAg8F2t9WsXmW49Ji4rpdRc4FkgFWgB7tdal10wjQt4ArgZ8AM/0lr/0qgME8z0feBhoC7w1Pta62+amOlxYDNQACz65FKeC6YJ9nIaS6bvM4HlZOuiwZ4XbBYzhkyBs58fA5Yx/MO9E7gPeM6kXN8FOrXWc5RShcB7Sqk5Wuvui0xr5rJ6CviZ1vr5QCn/HLj+gmm+BMwBChn+5T+ilNqjta6wMBPAc1rr75qU4ULbgH8D3rvMNMFeTmPJBBNYTrbedNJal2itSwGzL9gcs3FkuhPYprVuCqzFPA3cbWK0uxn+BSLw1/ojYKOJ8/sMpVQ6sBzYGnhqK7BcKeW+YNK7gae11j6tdRPDP+B3WZwpqLTW+7TW1aNMFrTlNI5ME2LrohmnuUqpw0qpD5VSD1gdhuHrwypHfF4F5NpkfmYtq1yg9pPr3wL/1l0kRzCXzVgzAdyjlDqmlNqllFprUp7xCPbP0FiNezlZuulkows2zchkqNFyjeOtDFtWIeYp4J+11l6l1I3AdqXUPK11i9XBbGZCy8nSorHRBZuGZ2L4r0/+iM/zgAmvlo6WSyn1yfyaRszv7Yu8j2HL6iKqgWyllEtrPRTYmTmDz37fn2T95ErgC/9yG2lMmbTW9SMe71ZKVQMLgXdNyjUWwVxOYzLR5RQSm042vWDzZWCTUsqtlHICDwK/NXF+LwFfBwjsDF4FvHHhRGYuK611Y+C9tgSe2gIcCexfuDDrg0opZ2BfySbgd0ZkmGgmpVT2iMdLGT7yos3INA5BW05jNdHlZOtrnex4weZYMwWm/TrDh8IBdgHfMmvTSykVCzzD8FGuIeBRrfX2wNeCtqyUUkUMH0pOBtoYPpSslVI7gO9prT8KrFU8yXDJAfyr1voXRmWYYKZnGT41YIjh/9fHtNY7TMz0BPAFIBNoBlq01gssXk5jyTSh5WTrohFChIaQ2HQSQtibFI0QwnRSNEII00nRCCFMJ0UjhDCdFI0QwnRSNEII00nRCCFM9/8Aezl1+EyIZccAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 288x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "t = np.linspace(0, 2*np.pi, 100)\n",
    "x = np.cos(t)\n",
    "y = np.sin(t)\n",
    "\n",
    "plt.figure()\n",
    "plt.plot(x, y)\n",
    "plt.xlim(-1.5, 1.5)\n",
    "plt.ylim(-1.5, 1.5)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Then, we will transform each of these points by applying a matrix $\\bs{A}$. This is the goal of the function bellow that takes a matrix as input and will draw\n",
    "\n",
    "- the origin set of unit vectors\n",
    "- the transformed set of unit vectors\n",
    "- the eigenvectors\n",
    "- the eigenvectors scalled by their eigenvalues"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [],
   "source": [
    "def linearTransformation(transformMatrix):\n",
    "    orange = '#FF9A13'\n",
    "    blue = '#1190FF'\n",
    "    # Create original set of unit vectors\n",
    "    t = np.linspace(0, 2*np.pi, 100)\n",
    "    x = np.cos(t)\n",
    "    y = np.sin(t)\n",
    "\n",
    "    # Calculate eigenvectors and eigenvalues\n",
    "    eigVecs = np.linalg.eig(transformMatrix)[1]\n",
    "    eigVals = np.diag(np.linalg.eig(transformMatrix)[0])\n",
    "    \n",
    "    # Create vectors of 0 to store new transformed values\n",
    "    newX = np.zeros(len(x))\n",
    "    newY = np.zeros(len(x))\n",
    "    for i in range(len(x)):\n",
    "        unitVector_i = np.array([x[i], y[i]])\n",
    "        # Apply the matrix to the vector\n",
    "        newXY = transformMatrix.dot(unitVector_i)\n",
    "        newX[i] = newXY[0]\n",
    "        newY[i] = newXY[1]\n",
    "    \n",
    "    plotVectors([eigVecs[:,0], eigVecs[:,1]],\n",
    "                cols=[blue, blue])\n",
    "    plt.plot(x, y)\n",
    "\n",
    "    plotVectors([eigVals[0,0]*eigVecs[:,0], eigVals[1,1]*eigVecs[:,1]],\n",
    "                cols=[orange, orange])\n",
    "    plt.plot(newX, newY)\n",
    "    plt.xlim(-5, 5)\n",
    "    plt.ylim(-5, 5)\n",
    "    plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAQcAAAD7CAYAAABnjFFxAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8qNh9FAAAACXBIWXMAAAsTAAALEwEAmpwYAAAnNElEQVR4nO3deXAc53nn8e/b3TOD+wYIgCAIECCaJEhRvExKomhZvESt5Mhidm3LjjYub2IncezUbu3aqU3ilMtJbCfrymFt4mxSsSMn8h1JlqyDlEVRlERSvCQeUvMCAYIASFwk7jm6e/8YkhLFATED9MxggOdT5XKpp/t9Hxz8ofvtt99WrusihBAfpKW7ACHE9CThIISIScJBCBGThIMQIiYJByFETEa6CwACwBqgE7DTXIsQs4kOVAFvAsEPfjgdwmEN8Gq6ixBiFrsb2PPBjdMhHDoB+vuHcRxv51yUlubR2zvkaZvJlkk179+/h23b7suYeq/JpO/xNcmoWdMUxcW5cPXf4AdNh3CwARzH9TwcrrWbaTKl5pGRESBz6n0/qfkGMS/nZUBSCBGThIMQIiYJByFETBIOQoiYJByEEDFJOAghYpJwEELEJOEghIhJwkEIEZOEgxAiJgkHIURMEg5CiJgkHIQQMUk4CCFiknAQQsTk+XoOpml+FfhTYJllWce8bl+ML9JpET62A2e4H62oEl/T3RjVi9JdlshQnoaDaZorgXVAq5ftiomFT+9l7OXvorIK0EpqiJw7TOTkaxj1q8na8BlUIDfdJYoM41k4mKYZAB4DPgns8qpdMTFn5DJjr34PvaKR7Pv/B8qXhRsJEXr7eUKHnmL4yXZy7vsDtMLKdJcqMoiXYw5fA35gWdY5D9sUcQgffwnCQbLu+SzKlwWAMvwEVn6U7Ae+DMFhRp75Js7ApTRXKjKJ8uJFuqZp3gF8HdhkWZZrmuY54IE4xxzqgJYpFzFLua7L+cd+F1/ZXKo+8Ucx9wldaqPjB3+C5gtQ/ZlvYOQVp7hKMc3VA+c+uNGrcPgK8EUgdHVTDXAR+IxlWS9OcHgd0NLbO+T5Aprl5fl0dw962mayJVqzM3CJ4R/+LwJ3fRp/86Zx97N7zjHy9J+jlcwj54Evowz/lGvdvXsH27c/POO/x9NBMmrWNEVpaR6MEw6eXFZYlvUNy7KqLcuqsyyrDmgHtsYRDGKK7EtnAdArm265n15WR9ZHPodz6QzB1x5PRWkiw8k8hwznXO4EpeIabPTVr8K/4kHC1quET72egupEJkvKeyuunj2IFHAGu1G5JXFfJvhXPYTdaTH26vfR5zSiFVQkuUKRqeTMIcO5owOo7MK491eaTta9nwOlMbbrn3BdJ4nViUwm4ZDh3LFBVFZeQsdoeaVk3fkIdtdJwkd3JKkykekkHDJdJHR9bkMijKb16LXLCR74Gc5gTxIKE5lOwiHDuXYE9MSHjpRSZK1/FFCMvfY4XtzSFjOLhEOmc2yUpk/qUC2vlMDqj2G3vUXk3EGPCxOZTsIh0yk1pUFF39LNaCXzCL7xBG4kNPEBYtaQcMh0mgHO5MNBaTqBOx/BHeol9PbzHhYmMp2EQ4ZTmg52eEptGNWLMepXEzryDM5Qn0eViUwn4ZDpfFm4keCUmwms/Tg4DsED/+FBUWImkHDIcMqfhRsanXI7WkE5vuaNRE7uwe4770FlItNJOGQ45cuG0JgnbQVWPAj+LIL7fuJJeyKzSThkOBXIxQ0OedNWVh7+2x/EPv82kU7LkzZF5pJwyHAqOx93dNCzSUz+pRtR2YWE3vyZTIya5SQcMpzKKgDXhtCIN+0ZAfwrHsTuOol94bgnbYrMJOGQ4VR2PgDO6BXP2vQt/jAqt4TggZ/L2cMsJuGQ4VRuCQDu8GXv2tR9+Fd+FOfSWez2o561KzKLhEOG0/KuhYO3k5d8TetReaUEDz0tZw+zlIRDhlM5RQCez2xUuoF/+f04F09jd7zjadsiM0g4ZDhl+FFZ+bhDvZ637TPvRuUUETr0tOdti+lPwmEGUAXlSVmwRRl+/Mu3YXe+i33xtOfti+lNwmEG0PIrkvY2K9+ieyCQS+jIs0lpX0xfEg4zgJZfhjvUi+vYnretfAH8zZuItB7G7r/gefti+pJwmAG0ggpwnaSMOwD4lm4C3U/oreeS0r6YniQcZgBVFH2hjXO5Kynta1n5+BZtIHLqDZzh/qT0IaYfCYcZ4NrbrpwrnUnrw79sC+AQPr4zaX2I6UXCYQZQWfkQyE3amQNEL12MulWETryMG/bmEXExvUk4zADq6rsyncvJO3MA8N92H4RGCFt7ktqPmB4kHGYIvbga53JHcvuY04hW0UDo2Iu4U1jUVmQGCYcZQiuuwR0dwBkdSGo//mVbcQcuYZ9/O6n9iPSTcJghtJK5ADhJnotg1K9E5RYTOibv2JzpJBxmCK34ajj0JTcclGbgW/wR7AvHCQS9W0NCTD8SDjOEyimK3rFIwcrRvsX3gG5Q3i/rTM5kEg4zhFIKvbQWuzf54aBlF2A0rKXkSgtO0Jvl6cT0I+Ewg2iltTh955PyjMUH+ZdsRHcjDB59Jel9ifRI/N3tMZimWQo8DjQAIeAU8DnLsrq9aF/ERy+tJWyHca5cRC+uTm5fFQsYzirBd/B5ArV3oZRKan8i9bw6c3CBb1mWZVqWtQw4A3zDo7ZFnLTSWgCc3raU9NdT1ES4px2762RK+hOp5Uk4WJbVZ1nWrvdt2gvM96JtET+tuAp0A7vnXEr66y+Yj5aVS/j4SynpT6SW52MOpmlqwO8AsrZYiinNQCupxek+l5L+XM0g77aPEGk5iDMitzVnGk/GHD7g74Ah4DuJHFRampeEUqC8PD8p7SbTVGpWtU0MHn2FsrJclEruePP27Q8T6mlnYP8zBDoOUHTHQ0ntz0uz7fdiMjwNB9M0/wpYCDxoWVZCk+97e4dwHG+XQC8vz6e7e9DTNpNtqjWH8+bihka5dPo0WlGVh5XdbPfuHWzf/jB6ZRP9B14k1HBvRgxMzsbfi1g0Td3yj7Jnf1pM0/xzYBXwkGVZQa/aFYnRyusAsLtbUtanb/E9uAMXsTvfTVmfIvk8CQfTNJuBPwSqgddN0zximuZ/eNG2SIxWVA2GP6XhYNSvBn8O4Xd2paxPkXyeXFZYlnUcmP7nk7OA0nT08nrsS2dS16fhx9d0F+ETL+OODaGykjN+JFJLZkjOQHpFA05PG64dTlmfPvNucCKET7+Rsj5Fckk4zEBaRQM4EZye1pT1qZfWopXVEbZeTVmfIrkkHGYgvWIBAPalsynt12fejdPbhp3CUBLJI+EwA2m5xai80pS/ws7XuA50g7C1O6X9iuSQcJih9DmN2BdP4brezh25FRXIxahbTfj03pSOd4jkkHCYofTKhbjD/Ul7C9Z4fOZ6CA4TaT2S0n6F9yQcZih9zkIA7IunUttv9RJUThHhk6+ltF/hPQmHGUormQe+LOyu1IaD0jR8C+/EPn806Sthi+SScJihlKahVzSkPBwAjIV3gWsTOb035X0L70g4zGB6VRNOXztucDi1/ZbMjc55kEuLjCbhMIPpVYsAF7sz9Ss1+RbeidPbip3k92iI5JFwmMH08nrQDSJdqV9C3mhYC0ojckqmU2cqCYcZTBn+6LhDZ+rDQcspRJ+7hPCZvbiuvFczE0k4zHB6ZRNOzznc0GjK+/Y13oE72JPymZrCGxIOM5xetQhcNy0rRBv1q8Dwy6VFhpJwmOH0ykbQDCId76S8b+XLwpi/kvDZ/bhOJOX9i6lJxgKzYhpRRgB9TgP2FMLBdV26+kY41X6FtouDXOwboX8oxOUBjSePPo8CcrN8lBQEmFOcQ11lPmZtESUFWfga1xI5sxe7/QRG7W3efWEi6SQcZgG9ejGhg08lvEpTR88we452cuDdS/RcGQMgy69TVZpDZUkOudogCxZUMjQcZHg0Qt/AGO+29rMjEh2AnFuey9qmEjb4cgif2SfhkGEkHGYBvXoxHHySSKeFr37VhPufvnCFp/e0cKylD11TNNeXcP+6+Zi1RVSW5FxfYXr37i62b7/9hlWRbcfhQvcw77T2c/hkNz9/7TyB3GpWnX6T/iX/meo5Rcn6MoXHJBxmAb2iIbrobMeJW4ZD38AYP3zpFAesbgpyfDy8YQEblldTkOuPvy9No3ZOPrVz8tn6oVouXR7l6J4wvo7TPP5vT1Ky9A62b2ggJ0t+9aY7+QnNAko30CubsC+cGHefvSe6ePyFk9i2w0N317N1TS0Bvz7lviuKsrn3/k0MPf4MW3O6+MvDFzhyqofffnAJZm3xlNsXySN3K2YJo6YZ53InzlDfDdsdx+Xfd57kH58+wdyyXL722Q/x0bvqPQmGa5Sm42tYw9zQWf73I7fhNzS+9cRhdrx5PqWL0YjESDjMEvrcpQDYF45f3xaxHf7+qWPsPNDO5tXz+PKnVlBRnJOU/o0FayASYl64hT/5zTWsWFjOEy+d4ie7zkhATFMSDrOEVlKDyi4g0h4NB8dx+e7TxzlodfOJexv55KaF6Fryfh30SjPa/9n9ZAcMfvdjS/nIyrk8v6+Nn72S2oVwRXxkzGGWUEqhz23Gbj+G6zr8+OUz14Nhy4dqk9+/pmHUrSJ86jXcSBDNCPDpzU24jssv97ZSVpjFPSvmJr0OET85c5hFjJpm3LFB3j5whBffPM+mVTUpCYbr/V+9tIi0vQ1EA+vTW0yWLSjl33acpKVTVo6aTiQcZhG9JjrucHLfazRUF/Bf7m1Mbf9VJiorn8jZN69v0zTFb390CYV5fv7xFycIR+yU1iTGJ+Ewi2g5RfQbFTRq7Xz2gSUYemp//ErTMepWEjn/Nm4kdH17bpaPz2xbzMW+EZ7b25bSmsT4JBxmkdauQQ4MVrDA6GZOXnree2zUr4bw2E1zLprrS1htlvPcvjauDIfGOVqkkoTDLPLM6+doYR4aTlqe0oSrU7n92YRbDt702cc2LCAUttl54HwaKhMfJOEwS/QNjHHoVDd1t62ILll//u201KF0A6P2diKth256jLuqNJcVTeW8cqSDcERWj0o3CYdZYu+Ji7gu3L1iHsbcJUTa3k7b5COjfjUEh7E7bl6+bsPyaoZGwxw9m9o3dYmbSTjMEodOdlNflU9FUTZ67XLc4T6c/va01GLMWwq6n0jroZs+a64vJjfL4PDJ7jRUJt5PwmEWGBmL0NI5wNL6UgCMedF1Fa7NN0g1ZQQwapqJnDt809mLrmksqSvhRGt/WmoT75FwmAVaLw7iurCwphAALbcYrbQWu+2ttNVk1K2Mnr30tt70WWNNIf2DQfoHg2moTFzj2fRp0zSbgO8DpUAv8KhlWal/F5u4SUdP9I1XNRXvrQJl1C4ndORZ3OAwKpCb8pr02uWgFJFzh9HL6m74rKY8WmdHzzDF+YGU1yaivDxz+AfgMcuymoDHgO962LaYgt6BMQxdo/B9i7YYtcvBdYicP5qWmrTsAvQ5C2OOO5QXZgHROywifTwJB9M0K4CVwBNXNz0BrDRNs9yL9sXUDI+Gycs2ri/vBqCVL4hOZW49kra6jLoVOL3ncQZ7btienxMNsYERmQyVTl5dVswDLliWZQNYlmWbptlxdXtcw8779+9hZGTEo3Kitm9/mN27d3jaZrIlo+aOTkUkrG5qtzZQTmHLId585QVQif+dGBsboqevm927X51UXYHQCEuAd3/1Q3qKzevbHRdA56X9Z8gPe39lKr8XUTk5OWzbdt+4n0+bR7ZvVeRUbN/+cFLaTSavaz7/w0NcOdnN9u1bb9g+bM3l4k+/xf2rTbLrlt18YPAKvPM9uP1LQHQNiFOn3qbbepHS7h0sdg5x9MmfUNf8e6xaO7mf3/m/P0hTrs2H3/c1B8M2//rGM2y9y2T7tsWTanci8nsxMa/C4Tww1zRN/epZgw5UX90el97eIRzH20k55eX5N6yMnAmSUbNO9BT90qWBGy4t3PwG0A163nqdrNy6G47xd75I3lt/TCh/EW/2zUd1/Apz5CVMdRETGHTzeI0Pc0dkNwfeKaB7wV2TK27uMkaPv8Sljm6U78axhoCukvLzk9+LKE1TlJaO/6oCT8YcLMu6BBwBPnl10yeBw5ZlyUyWaaAkP0Ao7DA8duN0ZeULoFcvYeDMYXa1R38VVLAX/Y0vUrjvc+hjl8ju3s2GM/+Nu0f/nRGVx678z/H64icYeOAgXaWf4oSxlmWh3YwFJ/cuTmP+7eBEiLzvQay+gegtTLlTkV5e3q34PPD7pmmeBH7/6n+LaaCyNHqr8kL30A3bu0YUPwp+iOzRbq5cvMALe36J/txWSi7+4ob9DhT/Fm+ve5nch3bSvPErLDTX4fdFBw0vV20jW43ScvL1SdWmVy4Efzb2+wZGL/RE66wqS/0tVvEez8YcLMt6F1jrVXvCO3WV+QCc6RjArC3GdeHHZ3x863CAQHAdD/PP1B//Js35bxBBY0Avw685+JSLwmU5B7hS8gViXfTVLtsG7V9F79gByzYmXJvSDIyaZUTa3sJ1XZRSnLkwQF627/otTZEe02ZAUiRPQa6fueW5HG/pY2lzHX+0P4t9F6M/+iGtmKNaE7mREMfvfpOlpQ5BBfHOTaytWcAZTBaP7CAY+TqGkfivlFF7G5Gz+3F629BKaznW0otZW3TD+IhIPZk+PUvc3ljGu239fPlVxZWgwiyyaSqyaSq0OZa3hsZICy2dfWiT+PfYVriVEtXHuZYjk6pNr4neKYmcf5szFwa4PBTi9saySbUlvCNnDrPEuuZKnn2jlUdLW9i2dv4NnzlXljL8I9jGfmBLwm3n1W+CI39LqG0HLFyd8PFaTiFaWR32+aPs7mzE79NY2STz59JNzhxmiblluSyqLWLngfabFlLRCivRiucSOXfz6kzxmF/bTJdbxYLBF3AneTvamLeMyMXTvHW8jTuXVpEdkL9b6SbhMIvcf8d8+geD7Dpy4abPjPpV2F0ncUYTXx5eaRon87Ywj1YudExuRqNRuxzlOiw0Otj6oXmTakN4S8JhFmmuK2Hx/GKe3tNy0yKuRv1qcF0i525+ECoees1mAPrP7pzU8RfccoYdP/dU9DMnSa/kE4mRcJhFlFJ8anMTwbDN4y9YNyy0opXMQxXMueGdEomoa1zDgJtPdf+LCR8bjjj88y8tzrpzme/Ky3WnCwmHWaa6LJeHNzRw6GQ3Ow68t0ycUgrfgtXYHe/gjg3dooXY/D4/x7M2ssh9i57eTi6Nxn/b44mXTnH+0hBzlqyGkX6cyx0J9y+8J+EwC2350DxWLCzjR786dcNajdFLC4dI6+FJtRuqil5aHH37ZT6/KzuuY57f18auwxfYtq6W+pXROXR2+7FJ9S+8JeEwC2lK8dsPNlNXWcDfP3WMI6ej6yloZXWovFLCk7i0+M5RP3/dvYWQ66O8b0dcZw47D5znxy+fZvWiCrZvaEDLL0MrrLz+JnCRXhIOs1TAr/PfP76cmvI8vvOzo+w6fAGlFMaCNdjtxxO+tHjUDPHuUCGvOnezXttDLlfG3ddxXH6y6zT/vvMUK5vK+a0HlqBdnX2l1yzF7ngX1w5P6esTUyfhMIvlZvn4n59cQXN9Cf/6gsU/P3MCZ95qcO2E71oU+OHP1o7xrL0Nn4qwgdh3LfoHg3z7x0d4bm8b99xezed/rRmf8d6voVHTDHYI++LpKX1tYuokHGa57IDBl379Nj56Vx2vH+/ij5/sIpRVQvjMvoTburvaxlcTffhqE8/f8FnEdnjpYDt/9E/7ONV+hd/ctojf2Gre9DJfvWoRKO2md2mK1JNpaAJNUzx09wKWLSjl+89bvNJfzcax4xw5epbbmuuvn/LH4/Nrinn72ZXco3YyagcJOgb7Tlzkl3tbudQ/yuL5xTx6nznuXAblz0Yrrydy4QSBNdu9+hLFJEg4iOsa5hby1c+s5vA+De3oMQ7sfJF/fWUpa5fM4fbGMhZUF95wCRBLng8Gqjfz7skwrzz1OvtbdUaCEebPyeeL229jeWPphE9bGnOXRJfND42i/PHd9RDek3AQN9A1jVXrVjHcVsmDhd2M6AXsPNDOC/vPY+ga8ypyqSrNpTg/wMUOhbH7DFeujDI0GqZ3YIyOnhE6eppw3Cay9DC3L6rgw8uraZoX/yPY+twlcPgXRFoP41t4Z5K/YjEeCQdxE6UUvsZ15Bx8ii98ah5BYwnvtPZzqv0ybReHeKe1nytDIRxX40BrdE6CrilKCgLMKcnh9sb5rOz+M5pzLIa3vZrwytb6nEYAgvt+LOGQRhIOIiZfwzpCB58kcmYf2bfdx8qm8hseo3Zcl5d37eSBBx7gcv8Ifp92w5lB7rFGck6/TLD/LSIlKxLqW+k+9KpFuKFhz74ekTi5WyFi0ooq0crrCZ/eG/tzpfDpkJfjJ+DXb7pkCFZF14UIdE7uXQv63MU4ve2TmsotvCHhIMbla1iH03MO53JXwsdGSlbg+Evxd00yHKoXAy6RLmtSx4upk3AQ4zIaPgQowqffSPxgpROs2ogxeBp96GzCh+vl9aD7sTveTbxv4QkJBzEuLbcYfe5iwqffmNRj1KHK6INY/klcWijdh17ZKOGQRhIO4pZ8C+/EHbiEM4npzKGK9bh69uTHHSpNnL523KAMTKaDhIO4JaNuFeh+wqcm8dIaPYtQxQaMvkOoscRffqZXNQGuPGeRJhIO4paUPxujfiXhs/sn9aRksGozCpdA10sJH6tXLABNx+46mfCxYuokHMSEfI13QnCYSNtbCR8bqrwXV+mTG3cwAtEl6zslHNJBwkFMSK9pRmUXEjn5WsLHuv5iwqVr8HfvgUjiYwd6ZRN2dwtuJDTxzsJTEg5iQkrTMRbeSaTtLZyR8RdxGU+oagvKCeG/uDvhY43KJnAi2D3nEj5WTI2Eg4iLz1wfXV9ynBmTtxK8ekszMIkJUdqcBoBJ3S0RUyPhIOKiF89FK19A+OSrCc95cHJriBQsxt/1K3ASG9TUsgtQBXPkjkUaSDiIuPnM9Th97Tg9rQkfG6zajBa+gq/3QMLH6nMasC+elvdZpJiEg4ibr2Et6AZh69WEjw1dfRDL35n4S2/0OY24owO4g4nPlRCTJ+Eg4qYCuRj1qwmffj3huweRwiXY2dXR2ZIJngFcW99BLi1SS8JBJMS36MMQGiXSkuDlgVIEqzajj15Av/JOQodqxXPB8GN3tyTWp5iSKS/2YprmY8BGIAgMAV+yLCvxC0uREfQqE1VQQfjdV6BwdULHhqq2kHP2+wQ6X2SkaEncxylNRy+rk3BIMS/OHJ4DllmWtRz4C+BHHrQppimlNHzmBuxOi0BoIKFjw6VrcHwFk1rjQSuvx+k5h+tEEj5WTM6Uw8GyrGcsy7p2f+oNoMY0TblcmcF85npQGqWXExwD0HyE5tyL78oJtOH2ifd/H71iAdgRnL7EjhOT5/U/4i8Az1qW5XjcrphGtJwijPkrKLlyFjeS2LyFYNXkJkTp5QsAsC8lvnCMmJwJxxxM0zwE1I7z8RzLsuyr+30CeATYMJlCSkvzJnPYhMrL85PSbjJlQs0j67bRde4gw9Y+ypvXx39g4f1wKEBez6/IW/fFuA9zy/JozcrDP9ThyfcnE77HH5TqmicMB8uyVk60j2maHwP+DNhoWdbFyRTS2zuE43g7yaW8PJ/u7kFP20y2TKnZzV9A0JfHwKEXGa1YntCxBWV34e96hd4L53H9RXEfp0prGW4/DVP8/mTK9/j9klGzpqlb/lGe8mWFaZoPAN8GtlqWdW6q7YnMoJRGb1EjY23HsS93JHRsqGoTyrWj06kToJXOj64MZcugZCp4MebwL4Af+Klpmkeu/q/Ug3bFNNdbuAA0g/CJlxM6Lli5CReV+LhD2XxwIjj9FxI6TkzOlOc5WJZVPvFeYiaKGNnkLb6DIWsPgTXbUb6suI5zs8qJlKyIPsJtj4Ee33F6WR0ATm9bNChEUsktRzElBau3QXg04TUmg1WbUfYI/u74F5BRBRXRmZJyOzMlJBzElATmNqGVzSd8/KWEnpqczLL1StPQimtw+s4nXKdInISDmBKlFP7mTTj9F7A743/HhJ3fQCSvgUDXTnDtuI/TS2twes/L49spIOEgpsxoWIsK5BE+ltgAY6hqC1qwF6PvSNzHaCXzcMcGcUcTX65OJEbCQUyZMvz4Ft9D5NxhnIFLcR93fbZkAms8aCU1ADh9csci2SQchCd8zRtBaYSO7Yz7mEjxcuxAeXQBmDgvE7SiKgCcBOdWiMRJOAhPaLnFGA1rCFu7cUOj8R2kNEJVmzCGz6EPnYnvkOxCCOTi9Es4JJuEg/CMf+kWCI8Rfjf+JejfWz4uvvEKpRRaUZWcOaSAhIPwjF6xAL2yidCxF+NedyFUdgeOkZvQuINeVC1nDikg4SA85V++DXeol8jZN+M7QA8Qqvgwvv4jaKPxPbOnCiujdyxCI1OoVExEwkF4Sq9djlZUReit5+Kei3D90iLOl+1qhXMAcK7Ef2dEJE7CQXhKKQ3fbffh9LZhXzgR1zGhOR/BVUbclxbvhUPXpOsUE5NwEJ7zLbwTlVNE6Mgzce3v+gsIl63F1/MGKjzxmgVaQfRZP2dgUkuHiDhJOAjPKd2H/7at2B3vxP2uiWDV5ujLdi+9MnH7RgCVU4Q72DPVUsUtSDiIpPAt/ggEcgkdeTau/d97ECu+SVQqrwRnqHfS9YmJSTiIpFC+LPxLtxBpPYzdO/FTlE5ONeGiZVdftjvx27S0vDKcQQmHZJJwEEnjX7oJfNmEDj0V1/6hyk1okUF8Pfsm3FflleIO9eK6stB5skg4iKRRgVz8yzYTaTmAHccaDMGrtzQDccyW1PJKwIngjg1NuU4Rm4SDSCr/0i3Rs4eDE5892AUmds48/F07J3wQS+UUAsij20kk4SCSSmXlvXf20Ns2wc6KYNUW9NFOjCvHbr1rThEA7oiEQ7JIOIik8y/bCv4cgm/+bMJ9Q1Wbosd03HpClCbhkHQSDiLpVCAX/+33Y7e9RaTr1C33DZesxvEXT7hsvQrkAsiYQxJJOIiU8C/djMouJLT/J7d+5kIzCFVuxBiw0IZvcRnizwYUbmjY81pFlISDSAllBPCv+jXsrpPYrUduuW+w8tryceOfPSilQSAHd0zCIVkkHETK+BZtQCuqYmzfj2653kOo4m5cPSu6fNwtKH+2PLadRBIOImWUZhBY93HcK123foWekU2ofD2+3gOoYN/47el+sMNJqFSAhINIMX3ecvS5SwgefPKWg4nBqi0onFu/bNfw4Uo4JI2Eg0gppRSBOx6B0CjBN3867n6hyntx0W69xoNmgLxxO2kkHETK6SU1+JZuJvzOK9jdLTH3cQOlhEtX4e9+FSK3WM1aqSRVKSQcRFoEVj2Eyi5g7NXv4zqxX4cXqtqCsseiARGLvBIvqSQcRFoofzaBOz+F03OO8NHYlw7X3og17hoPjg2anqwSZz0JB5E2xoI1GPNXEDzwHzhXbl7yzcmdT6TAjL5sN8atTzc8ivJnp6LUWUnCQaSNUorA+kdB1xnd9f9iXl4EKzehhfrx9R28uYHQKMon4ZAsEg4irbTcYrLW/1eci6cJHf7FTZ+//41Y2kgH+lB0ANO1w7hjg9efzhTek3AQaedrXIex8E5Ch54i0vHO9e0q1E8kvxE7q5JAxy8p2vNxtNHocvTuUHRylJZfmpaaZwPPwsE0zXtM07RN0/yCV22K2SPrrt9AK6xibOf/vb5wrAoPUfr8WrRgD/poJ/pIO7jRSw+7vx0ArbAybTXPdJ6Eg2ma+cA3gee8aE/MPsqfTdaWL+DaYUZf+Gvc0AhO7jyGm7+Cct8/GBldM9K51AJKQyutTU/Bs4BXZw7fBv4SkBcJiEnTi6rJ3vR7OH0djL74d7iRIGN1jxAqv+v6PurqgrKRtrfQKxpQhj9d5c54Uw4H0zS3AYWWZY0/F1aIOBnzlpF1z2exO95l9Jf/Bzc0wuCKb+IYedEdXAf70lmcvvMYC9akt9gZzphoB9M0DwHjnbuZwDeAzVMtpLQ0b6pNxFRenp+UdpMpU2revv1hIAn1lm9lqCiPS0/9LWNPfY2Kj34R7Y6vw6t/QH6OxtDuJ9ByCqi66360wORuZWbK9/j9Ul2zivdNyLGYprke+Dlw7aH6MiAI/I1lWV+Ls5k6oKW3dwjH8XY6bHl5Pt3dE797cTrJpJp3797B9u0PJ63eSNcpxn71D7hDveiVJllhi7FgAfbQIFmbfhffJM8cMul7fE0yatY0de2Pcj1w7oOfT3jmcCuWZe0BKq79t2ma3wMOWJb1nam0KwSAUbmQ3F//OqETLxE5vZfhAT96np/s+/4Ao3Z5usub8aYUDkIkm/JnE7j9AQK3PxDd4LryJGaKeBoOlmX9ppftCXETCYaUkRmSQoiYJByEEDFJOAghYpJwEELEJOEghIhJwkEIEZOEgxAiJgkHIURMEg5CiJgkHIQQMUk4CCFiknAQQsQk4SCEiEnCQQgRk4SDECKm6bDYiw7RJauSIVntJlOm1JyTkwNkTr3vJzXf0F7MtxFPaQ1Jj6wHxnnHuhAiBe4G9nxw43QIhwCwBugEbn6TqhAiWXSgCniT6MLQN5gO4SCEmIZkQFIIEZOEgxAiJgkHIURMEg5CiJgkHIQQMUk4CCFiknAQQsQ0HaZPp4RpmvcALwFfmu4v+jVN8zFgI9GJKUNEaz6Q3qpuZppmE/B9oBToBR61LOtUequKzTTNUuBxoAEIAaeAz1mW1Z3WwuJkmuZXgT8FllmWdSwVfc6KMwfTNPOBbwLPpbuWOD1H9JdgOfAXwI/SXM94/gF4zLKsJuAx4LtprudWXOBblmWZlmUtA84A30hzTXExTXMlsA5oTWW/syIcgG8Dfwn0pLuQeFiW9YxlWeGr//kGUGOa5rT6WZmmWQGsBJ64uukJYKVpmuXpq2p8lmX1WZa1632b9gLz01RO3EzTDBAN3t9Jdd/T6hcuGUzT3AYUWpb103TXMklfAJ61LMtJdyEfMA+4YFmWDXD1/zuubp/Wrgbt7wBPp7uWOHwN+IFlWedS3XHGjzmYpnkIqB3vY6KnjptTV9HEJqh5zrV/cKZpfgJ4BNiQqtpmib8jOpYz3cee7gBWA19JR/8z+sEr0zTXAz8HRq5uKiM6yPc3lmV9LW2FxcE0zY8BfwVsTMdfjYlcvaw4CZRalmWbpqkTHZRcOJ0H+UzT/CvgNuBBy7JuehJxOjFN8yvAF4kOoALUABeBz1iW9WKy+5/R4fBBpml+DziQAXcrHiD6122zZVmn013PeEzT3AX8k2VZPzBN89PAZy3L+kiayxqXaZp/DtwB/CfLskYm2n+6MU3zHPBAqu5WZPxlxQz1L0T/WvzUNM1r2zZaltWbvpJi+jzwfdM0/wToBx5Ncz3jMk2zGfhDomc7r1/9vrZYlvWxtBY2jc2qMwchRPxm/N0KIcTkSDgIIWKScBBCxCThIISIScJBCBGThIMQIiYJByFETBIOQoiY/j90BCymJ26Y/gAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 288x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "A = np.array([[1,-1], [-1, 4]])\n",
    "linearTransformation(A)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can see the unit circle in dark blue, the non scaled eigenvectors in light blue, the transformed unit circle in green and the scaled eigenvectors in yellow.\n",
    "\n",
    "It is worth noting that the eigenvectors are orthogonal here because the matrix is symmetric. Let's try with a non-symmetric matrix:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAQcAAAD7CAYAAABnjFFxAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8qNh9FAAAACXBIWXMAAAsTAAALEwEAmpwYAAAnxUlEQVR4nO3daXAc533n8W93z4WbuAkQvAm2eB+iZFmWaN0yZdqSSFuSz7UTJ7Y33ji1VVvrVG05KVc2dhyXq7Kx4jixy1HsWId1y7JukYIOUhJFUuLZBEkQJAEQxH3P1f3siyEpQhwQA6BnGjP4f95InON5/gAGP3Q//fTzaEophBDio3SvCxBCTE8SDkKIpCQchBBJSTgIIZKScBBCJOXzugAgCFwFtAG2x7UIMZMYQA3wLhD56JPTIRyuAl73ugghZrDrgTc++uB0CIc2gJ6eIRzH3TkX5eWFdHUNutpmumVTze+88wabNn0qa+o9L5u+x+elo2Zd1ygtLYBzv4MfNR3CwQZwHOV6OJxvN9tkS83Dw8NA9tR7Mal5lKSn8zIgKYRISsJBCJGUhIMQIikJByFEUhIOQoikJByEEElJOAghkpJwEEIkJeEghEhKwkEIkZSEgxAiKQkHIURSEg5CiKQkHIQQSUk4CCGScn09B9M0/wb4W2CVZVn73W5fZDelHOzWw8RP7cPpPoWKDKEZfvSyOow5y/HNX4emG16XKXA5HEzTXA9cAzS72a7IfsqJEzvcQHTvs6jBLtB96KVz0PKKIB4l1vgWsYOvohVVEvzYPfgXXeV1yTOea+FgmmYQuB/4ArDdrXZF9rPbjxJ+7Vc4vW0Ys5fiv/rz+BasR/MFLrxGOTbxk+8Tfe9Jwi/fj738JoLXfkmOIjzk5pHDD4DfWpZ1wjRNF5sV2Uoph+jeZ4nuegKtsIy8276LMX8tmqZd8lpNN/AvWI9v3moi7zxK7IPnUbEwoRu+gabJ0JgXXAkH0zQ/DmwAvjfZNsrLC90o5RKVlUVpaTedsqXmrVu3AMnrVXaMjmfuJ3rgdQqWf4LKTd9EDxWk1vBn/oye0lJ6XnsQX2U1ZTd+2c2ygez5Hl8s0zW7deTwSWAZ0HTuqKEOeME0za9blvViKg10dQ26voBmZWURHR0DrraZbtlUc0PDS2zduuWSelUswsiL/4TdcpDAVZ9DW/tpugYcGEj961JLb8Pf3krvW08SKa3HN2e5a3Vn0/f4vHTUrOvaZf8ouxIOlmX9CPjR+X+bpnkC2CxXK2YeFY8y8uL/w249ROiGb+Bfet2k2tE0jeDHv4jdZhF+7VcU3PPDUWMUIv3kZE64RjkO4Vf/FbvlAKGNfzLpYDhP8wcJXv/fUINdRPendAAqXJSWfSssy1qQjnbF9BZ5+2HiJ3YTvPZL+M3rXWnTV7sMY95aonv+QGDZjWjBFMctxJTJkYNwRexwA7F9L+BfeSuBlbe62nZww90QCxM73OBqu+LyJBzElNkdTYTfeABjzgqC19znevtGxXyMmiuIHngZ5chey5ki4SCmxIkMM/LKz9HySsi7+dtpm7TkX3krarALu+VAWtoXl5JwEJOnFJ3P/ztqoIPQTd9EC6VnrgqAb95qCOQTa9yRtj7EaBIOYtJmDZxkcH8DgfV34atJ76xYzfDjX7SB+IndqHgkrX2JBAkHMSlOeIC69ncJzF5MYN3mjPTpW3Q1xCPYrYcz0t9MJ+EgJiXy1n9h2DGqPvMXGbs5ypi9FIwA8VP7MtLfTCfhICYsfno/8aM7aa9YQaBqfsb61XwBjNorJBwyRMJBTIiyY4Tf/C1aSTXtZSsy3r9vzgpUfzvOUE/G+55pJBzEhET3vYDqO0Po2i+hPFhrwaheDCTWiBDpJeEgUuYM9RDd/TS+BevxzV3tSQ16xXwwfNhnj3nS/0wi4SBSFn3vSXDstMyCTJVm+NErFuCcPe5ZDTOFhINIid3TQsxqwL/8JvTiKk9rMcrmYnefRil31/8Qo0k4iJRE3v49+EIE1n/W61LQy+ogOoySQcm0knAQ47Lbj2Kf3Etg7R3oIe+XV9PL6gBwek57XEluk3AQ44q89yRaqMj1W7EnSy+tBcDpafO4ktwm4SAuyz7TiH16P4E1m9D8Ia/LAUALFoI/hDPQ4XUpOU3CQVxWZPdTaHnF+Jff7HUpF2iahl5UKeGQZhIOYkz22eOJo4bVn0LzB70uZxS9uBI10Ol1GTlNwkGMKfr+HyGQj3/ZjV6XcgmtsBxnsMvrMnKahINIyuk7Q7zpPQLLb0IL5HldziW0/BKIhWVthzSScBBJRd9/HgwD/zS5QvFRel4JAGq4z+NKcpeEg7iEM9JPrPEN/PXXoeeXeF1OUlq+hEO6STiIS8QOvwZ2HP+q27wuZUxaXjEATrjf40pyl4SDGEU5cWIHX8WoW4lxbrLRdKQF8hP/Exn2tpAcJuEgRok37UYN9RBYeYvXpVzW+XBQ0RGPK8ldEg5ilNj+l9CKqzA8Wq8hZeeuoEg4pI+Eg7jA7j6N3d5IYPmNaNr0/mhougH+ECoqpxXpMr0/ASKjYodfA92Hb4q7Y2eK5gtAPOp1GTlLwkEAoOJRYkfexLdww7S4LTslhh9lx7yuImdJOAgA4sffhegw/mWf9LqU1Bl+kHBIGwkHAUDMakArqcaoucLrUlKmSTiklYSDwOk/i91m4V96HZqmeV1O6gyfnFakkYSDIHbkTUDDX3+t16VMkAayyGza+NxoxDTNcuA3wGIgCjQC37QsS1bjmOaUcog1voUxZxl6YbnX5UxMFh3kZCO3jhwU8GPLskzLslYBx4AfudS2SCP7TCNqoAN//Se8LmUSJB3SyZUjB8uyuoHtFz20E/i2G22L9Io37gBfEN/CDV6XMjlyWpE2ro85mKapkwiGp91uW7hL2XFiTe/iW7Bu2i0DJ7znypHDR/wzMAj8bCJvKi8vTEMpUFmZJRN6LpKpmoePvsdgZIjydTdQMIk+t27dAnj3PY7o4MsLTap/+VyMz9VwME3zJ0A98BnLspyJvLeraxDHcfcQsbKyiI6OAVfbTLdM1jzy3jYIFjBUvIThSfTZ0PASW7du8ex7HI9GcOJqwv3L5yJB17XL/lF27bTCNM2/B64E7rIsSxb2m+ZUPEq8eQ/+hVeiGek4gMwAOw56ltaeBdy6lLkC+GvgCPCWaZoATZZl3e1G+8J98dP7IBbGt+hjXpcyacqxQTe8LiNnuXW14gByXSmrxJveg2ABRq3pdSmT58Sz96gnC8gMyRlI2XHizXvwzV+HlsWH5SoWhmmyRV8uknCYgezWQxAdwZ+tcxsApRTEomg+uQSbLhIOM1C8aRf4QxhzlntdyuQ5cVA2yPyMtJFwmGGUchKnFHNXJ1ZSylaxxAWx6bLzdy6ScJhhnI4m1Eg/vvlrvS5lSlQssbCsnFakj4TDDBNv3guahm+6ry49DnV+v4pggbeF5DAJhxkmfnIvRnU9Wig909UzRUWGANCC+R5XkrskHGYQZ7ALp+tU1p9SwMXhkN0hN51JOMwg8ZPvA2DMW+ttIS74MBzktCJdJBxmEPvUPrTCcvRZNV6XMmUSDukn4TBDKCdOvPUQvrpV2bWI7BhUeACMAGTz5dhpTsJhhrDbj0EsjDF3pdeluEIN96HlFeVE0E1XEg4zhH16P2g6vmyeFXkRNdKPllfidRk5TcJhhoif3o9RtfjC1vXZLhEOxV6XkdMkHGYAFRnC6TiBUbfC61Jco0b60PMlHNJJwmEGiLcdBlR232h1EeXYqPCAnFakWfbezC9SZrccAl8Ao3LRpN6vlOJM9zCNp/s42T5Ae/cwPYNRevt1ntz3PBpQEPJTVhykujSfBbOLMOfNoqw4PTdFqeE+UAqtoCwt7YsECYcZwG49hDF76YRXTWrtHOKNfW3sOnyWzr4wAKGAQU15PrPL8inQB1i0aDaDQxGGRuJ094c53NzDS/HE2sJzKgu4+ooqrltdS2mRezdIqaFuAPTCUtfaFJeScMhxznAfTk8LgQnsg3m0pY+n32hif1M3hq6xYmEZd1wzH3PeLGaX5V+4fNjQcIatW9eOWhXZdhxaOoY41NzDniMdPPF6E0+9cYKrl1Wx+doF1FZMfdKSM9QDIEcOaSbhkOPstsMA+GqXjfva7v4wD73SyC6rg+J8P1s2LmLjmlqKC1KfaGToOvOqi5hXXcTtV8/jbO8I23afZvueVt45dJZPrqtl68bF5Icm/9FTg+eOHCQc0krCIcfZbUfAF0SvmH/Z1+08eIbfvHAE23a46/qF3H7VPIKBqa/sXDUrj3tvqueOa+bz1BtNbNvTwt7GTv78M8sx503utMAZ6gbDL7drp5mEQ46zzxzBqF6CNsYS7o6jeOjVRl7edZolc0r4xuZlVJW6PxeiKD/Al28zuXZlDf/+zAF+/OAe7rupnls21E14lqMa6EQvqpDZkWkmlzJzmIoM4XSfxpi9NOnzcdvh50/t5+Vdp7l1w1z+95fWpSUYLraotpjvf+0q1tVX8uArjfx++7HEYrET4Ax0ohVVpqlCcZ6EQw6z248CCmN2/SXPOY7iF08f4D2rg/tuWsIXbqnH0DPzccgL+vjvd6/kxvVzeP7tkzz22vEJvd8ZOIteVJGm6sR5clqRw+wzjaAZGFWLL3nukW1HLwTDbVfPy3htuqbx5VuXohzFH3c2U1ES4oZ1c8Z9n4oMQXQEvViOHNJNjhxymN3eiF4xD+0jy7e/c6idF989xS1X1nkSDOdpmsaXbzNZtaic/3rpCE1t/eO+xxnoSLxXTivSTsIhRynHxu5ouuSooXcwwn8+b7G4tph7blriUXUf0nWNP//sckoKA/zbMweJxe3Lvt7pT4SDLuGQdhIOOcrpaYF4FKN6dDg8su0o0bjDn25ejs+YHj/+gpCfr29aRnv3MM/tPHnZ1zp9ZwDQS6ozUdqMNj0+HcJ1dvsxgFFHDs1nBth5oJ3br57L7LLpdev2ioVlbDAree7tk/QNRcd8ndN3Bq2gVDazyQAJhxxlnz2OFioadW7+h7dOkB/0seljl58Q5ZW7Ny4iGrN5edepMV/j9J5BL5mdwapmLgmHHOWcPYZetejCRKHu/jC7Gzu4Yd2cKU1dTqea8gLWLa3ktb2txM7dvPVRTp+EQ6ZIOOQgFQvj9LZhVC688NjOg+0oBRvX1npY2fg2rqllcCTGvuNdlzynwoMQGZJwyBAJhxxkd50EFEbFgguP7T7SwcKaIqpm5XlWVypWLCylIORjz5GOS56ze1sB0GdJOGSChEMOcjqbAdArFwAwHI7T1NbPyoXlHlaVGkPXWb6gjIPNPZc853SfBkAvHX+ylJg6CYccZHecQMsrQc+fBUBz+wBKQX1ddiyrtqSuhJ6BCD0DkVGPOz0t4A+hFU7/kMsFro1Mmaa5FHgAKAe6gK9altXoVvsidU7niQtHDZBY0Qmgrio79pWsq0zU2do5NGoFKaenFb20Vu7GzBA3jxz+FbjfsqylwP3AL1xsW6RIxaM4va0YF63f0NUfxmfolExg0RYvVZYk5jB094dHPe70tGDIKUXGuBIOpmlWAeuBB8899CCw3jRNmeOaYU5PKyiFXjb3wmNDIzEK83xZ8xe3KD8RYv3DH06GcsIDqJF+GW/IILdOK+YCLZZl2QCWZdmmabaee/zSYeck3nnnDYaHh10qJ2Hr1i00NLzkapvpNtWay3qPMR/YffwkkdO9ALS2acRjmuvfi0h0hL6+XhoaXnW1XUcBGLzyzjGKYokz06KhNpYA+1s6GOyd2tcxEz8XyeTn57Np06fGfH7azIa5XJFTsXXrlrS0m05TqbnzpV8z0Bng0/d+5cLqT6ce2k3fkQ62br19yrXZcZsP9m4nevgh1g0/g/XYQ9zy+d9SUuTeYGckZvOfO/7A7Z8w2bopsfZl786n6D4Ft3zuyxgubGYz0z4Xk+FWOJwC5pimaZw7ajCA2nOPp6SraxDHmdiKQOOprCwatTJyNphqzcOnj6GVzqGz68OjMIPEIfrZs/2TPrVoaT1Kz+EnWNH3GOu0dgAGyac+vpszPcNEw+4NX50fawga2oXvxUhzI1pBGd1DGgxN7Wc6Ez8Xyei6Rnn52IPUrvxELcs6C+wFvnDuoS8AeyzLSumUQrhDKZVYFu6i8QaAsqIg0ZjDUDh+mfde+ljfQA/7dvyWzqfuZu07t3Jj/78Q0wJsm/VXvL74VxRqw+wp+hwFee5eBenuT1zCHHWloqsZvXzuWG8RaeDmacW3gAdM0/w+0AN81cW2RQpUeAAVHkAvGz1oN7s8sUpzS8dg0hWf93fr7Okw+IoZIxaPcfRwA8GTj7Mu8hJLtBhDKp838+5BW7SFxUuuYqWuc/ClHwBQsuYrrn8dLZ2DANSc2+MicQWmjcDCDa73JcbmWjhYlnUY+Jhb7YmJc3rOTy8eff/EgtlFABxr7R8VDkrB7xr9/HB3kL+o28PBU79n5cATbNQ6Adjr+wQ9tZ9j0fJbWZr34TLwkUiYtYOP06jqWXbFlXR2Dbr6dRxr6acwz3/hkqbTfTpxBabcu1WrZqJpMyApps45f+9B6ehwKC4IMKeigANN3dxxTWL+w2AMfrxjgLIzT/Fy4GFWde4H4KS2kPfLvk7VsjuZUzmHZBcOjx54kRu0Pl4wvkS97u7lUaUU+5u6MOfNujA+YnckFqC9+EYykX4SDjnE6W0DX/CSbeJ8PXu5qvwkTx0pp79/gMiZN+k8+AT/rF7BF7DpV0U8EP8KoSVbuHH1GlaN8wtfdvpBIiqAU+D+rt3HWvrpHYyydsmHq0vbHU1oecWy/V2GSTjkEKe3DX1WzYdXJJQidPwBCvf/PRtL7+FJdRO7Hvs+95Y+ia10tjk38oS6l52+2wnmh6jrd1gfCzPrMnvetp1pYrWzkx2hu/D73Z9x2fBBKwG/zvqlH86fczqa0CsXZs0krlwh4ZBDnJ5WjBoTAC3WT9Ge7xFsfQ6A+t7/YlV+LX/s/iRXr1+GWnAnK4qqWaMDOEBqE9C6Dj4KgFp0L7S7e2mtbzDCzgPtXLe6hrxg4qOpoiM4PW0EFl3tal9ifHJXZo5Q8ShqqBu9ZDa+3gOUbv/shWA4b/OVpXTHitkR20RRSTW+Cf70o7Eoq/oe4RTzWVzv/i/rM2+dQCnF7Vd/eMnS7mwGlIw3eEDCIUecX7I9GD3KrIYtGEPNKM2PEywnXriIWOk61vsaWD63kKffaLrsIq5jOXboNSq0ThrL70N3eXesk+0DbN/TysY1tVRftCWffTaxUK4u4ZBxclqRI5z+xKxFNftKute+huMvASMEHzlP/+LiIf721+/wmxcs/uLulRM6j88/+RBxZVC7wt1pvLG4w6+ePURBno+7Ny4a9ZzTfhStpBo9b+pTpsXEyJFDjlD9ZxP/U7MBJ282+PIuCQaA2ooCtmxczO4jHby063RKbT9y1M/ZzlbWx7axO3AL5WVVbpbOg680cursIF/ftIzCPP+Fx5VS2O1HMaov3etTpJ8cOeQIp/8sBAvQQuNPZb7t6rk0nu7l4VcbqSwJsW7p5e+s/7eDAQyeZIWmiMy/h6gNAcOdup9/+yTb97Sw6Zp5rK0fvTmu6j+LCg9gVHu/M9dMJEcOOcLpP4tenNpfdF3T+PPPrGDB7GJ+/tR+9h7tvOzre8IOm+O/o9WZzVcO3sEvD7lzCfPlXad4ZNtRNlxRxdaNl272m9glHDly8IiEQ45wBjrRJ7C2YjBg8D/vXUNdZSE/e2wf2/e0JH1dxIaPOa9Rp7fwO/uLXFmt8a0VEx/MHFWro/j99qP87uVG1i+t5M82L0dPMvHKPtMIgTz00pop9ScmR8IhByilUINdaEUV47/4IgUhP//rC+tYsbCM/3zB4ld/OMhIZPSdm91hja/4fgvAtsAX+PHHR5jKjOmegQg/fWQvz+08yQ1ra/nWnSvwj3FN1T5jYVTXo2nyMfWCjDnkABUeADs2oSOH8/KCPr77udU8/WYTz7x1goPNPdx3cz0bzEo0TWNgoJNNxnM0OJ/kr2+qojiQfCeq8cRth9f2tvJ4w3HitsPXNl3B9atrxrxa4gz34vS2ETSvn1R/YuokHHKAGkiMGUx2yXZd17jr+kWsWlTOA89b/PzJ/SysKeKOa+ZTM/AUfi2Os+Aels2aeDBEYjZvH2znjzubOdszwrL5pXz1U+aouQzJ2G1HADBqrpjU1ySmTsIhBziDia3jJnPkcLHFc0r4m69v4K39Z3jmzRPc/8R+Sn1VXFv8FVau3UAs7ox5CnCxSNTGOtXD7iOd7Dp8luFInPnVRfzl1tWsWVKe0twKu+0w+EPoFdNz09+ZQMIhB6ihbmDq4QCJHaeuX13LJ1bWsG/3W+zYuZfnezby7EP78Rk6c6sKqCkvoLQoSHurhq/hGH19IwyOxOjqD9PaOUxr5xCOUgQDBuvqK/jkmlqWzp01oQlXsYOvJm620l26ZiomTMIhB6jhPtB9ECwY/8Up0nWN67THuKXuCVque5793WU0nu7lZPsgh5p76BuM4iidXc2JdSAMXaOsOEh1WT5r68sx55ZSX1dCwD/xX25nuO/c/8ldmF6ScMgBzlAPWsHE/jKPR4v2EWz5I7GyKwlUmKyvYPRt1EqxbfvLbN68md6eYQJ+3bX+7dZDAISuc38JOpE6uUaUA9RwL9q5fTHdEjz9FJoTYWT+vUmf1zUNvwGF+QGCAcPVYLJbDkCwAL1cxhu8JOGQA9Rw34VNc91pUJF34iEcXyGROZ92r92UulbETx/AV7sMzeU7P8XEyHc/BzjDvWj57m0q4+vdh6//EJG6z4Lv8pcc3ab62lFD3Rhz3F+CTkyMhEOWU04cosNoLt7SHGp+GIDwGKcU6RRvSQxw+uasyHjfYjQJhyynwoll4bVQkTsNxocInn6aePEy4rNWudPmRLo/tQ+tuAotxZvIRPpIOGS5D8PBnV2ngi1/RI8PMrLgvqTrQaSTikexWw7hm7tKFpOdBiQcspwKJxZ5devIIa/5YZQeJFJ3pyvtTYTdZoEdxTd3Tcb7FpeScMhyboaD0X8Ef/d7RObcgQq4N8CZqvipD8DwY9TK/RTTgYRDllORxJLymguzI0PNjwDeDEQCxE9+gFG7DM3n/n4YYuIkHLJd9Fw4BPKm1o4dIXTqceKFC4mVZ36PCLu3FdXfjm/e6oz3LZKTcMhyKjqSGDj0XWabqhQE215Cj/Ykjho8GAyMn9gDgG/+uoz3LZKTcMhyKjoC/rwpj+6Hmh9CaT7Cc91ddj5V8RO70SsWuHJnqXCHhEOWU9GRKZ9S6EMnCXS8SXT2LajQ5VeiTgdnuBfn7HF8C+SoYTqRcMh2sTCaPzSlJs4PRI4suM+NiiYs3rwXUPjmr/ekf5GchEOWU3YUpjK678QJnXwUO6+WWNV17hU2AfET76EVVaKX1XnSv0hOwiHbxWNohn/8140h0L4dI9xOeN7nQcv8qksqPIh9+iC+hRtkVuQ0M+XFXkzTvB+4GYgAg8B3LcvaNdV2RWqUHUULTP7OyVDzwyg0wvM/72JVqYs37wFl41+c+cun4vLcOHJ4DlhlWdYa4IfAwy60KVJlT/7IQR9pJ9C+jVjVRpz8OS4XlprY8XfQiirQKxZ40r8Y25TDwbKsP1iWFTv3zx1AnWmacrqSKfEYTDIcQicfRVO2ZwORH55SXCWnFNOQ27/E3wGetSxrcjufiAlTSsFkdoRSDqHmR3CC5URn3+R+YSmInXhPTimmsXHHHEzT3A3MG+Ppasuy7HOvuw/4IrBxMoWUl7tzy/FHVVa6tM5BBk2k5hEdQnmBiX+dLQ0wfBJWf4fK6slNPNq6NTFharLf49YX3sVfVkP1sszfop3rnws3jBsOlmWNe/HZNM27gf8L3GxZVvtkCunqGsRx1GTeOqbKyiI6OgZcbTPdJlqzHY8TjtgT/jqL3v81IaC76m7sSX6PGhpeYuvWLZP6HjuDXYSbDxC48i46Owcn1f9kzYTPRSp0XbvsH2U3rlZsBn4K3GpZ1ompticmaBKnFVq0h2DbC0TLr8YuXJSmwi4vdvRtQOGv/7gn/YvxubFvxa+BKPCoaZrnH7vZsqwuF9oWaRA6+QSaE/Xs1myA+NEd6NVL0GU5uGlryuFgWVbmJ+OLD+k6qAmM/ypFqPlhHF8RkTl3pK+uy7A7m3G6TxH8xJc96V+kRi45ZjvNAGWn/HJfzx58A0eIzL0LjKndkzFZMasBDB/+xdd40r9IjYRDltN0HZzUwyF0IjFHbWS+R3Mb4lFiR3fiW3Cla4viivSQcMh2ugFOaqcVWmyAUMszxGatxp7lzaYx8RO7ITKE35zUFW+RQRIO2c7wo+zY+K8Dgi3PoNkjng5ExqzX0QrLMeYs86wGkRoJhyyn+YIQj6T02tCJh1FGHpG6z6S5quScvnbslgP4zY1ok5nVKTJKfkLZzh9ExcLjvszoO4i/9wPCczaj/N7MDowe2gaagf8KOaXIBhIOWU7zBSE2/pFD3gnv9r+EcwOR1uv4Fq5HLyj1pAYxMRIOWU7zh8Y/crDDBE8/SbyonniZN0uxxY+9nRiIXO7NTV5i4iQcsl2wABUZStydOdZLWp9Dj/V7tuy8UorogVfQZ9Vi1MhuVtlCwiHLaaFCsGMQj475mtCJh1F6gPDcuzNY2YfsM0dwOk/gX3mrrNuQRSQcstz5iUQqkvzORmPwOIGut4nU3IYKlmWytAtiHzyPFizEv/RaT/oXkyPhkOUuhMNI8tt5vd7/0uk7Q7x5L/4VNyUGT0XWkHDIcnpeYjdsNdx76ZNOLLHsfP5cYpXe/NWO7nsRdEMGIrOQhEOW085tH+cMdV/yXODMK+iRLsLz75ncUnJT5Az3ErMa8C+9Fj1/Vsb7F1Mj4ZDltLwS0HTU4KXhkFh2Xic873MeVAbRD54HxyawdrMn/YupkXDIcpquo+XPuuTIQR9uJdD+GtHqG3DyZme8Lic8QOzgNnyLr5EFXbKUhEMO0IsrUf0dox4Lnfw9GoqwR8vOx/a9CPEIgXVy1JCtJBxygF5SjdN3BgB/xw5QNqHmR7BDVUSrb8x4Pc5IP9F9L+JbdBVGqTeb5Yipk3DIAXpJDSo8gAoPknf0lxTv/AbGSGtirEF3Y5nQiYnufRbsKIEN3ky6Eu7I/CdHuE6flRhTcPrOoEe78Pe8D0DwzCvo0V4G1/xdxqZNO4NdxA6+gq/+OoxZtRnpU6SHHDnkgPNb19tdJ9EjHw5MatEehq74q4zeTxF970lQiuCVd2asT5EeEg45QCusgGABTmczWjQRDkrz0X/Vv6BCmVsc3O5sJma9gX/FLehFFRnrV6SHnFbkAE3TMCoWYHc2oetDAAyu+j/Ey6/MWA1KKSI7focWKiS4/rMZ61ekjxw55AijcgFOdwuO0gnX3Ul44Vcz2n+8aRd2m0Vgw91owYKM9i3SQ8IhRxizTXBshv0rGVj7w4yOM6joCJEdD6KX1eG/4pMZ61ekl4RDjjBm14Om0Vd2J/jyMtp3ZNcTqKEeQtd/DU03Mtq3SB8JhxyhBfLQKxYS62jNaL+RtmPEDryEf/mNGNVLMtq3SC8Jhxzim7cGp/0YznBfRvrTHJuzz/wMLa+E4NXe3Nwl0kfCIYf4FqwHFPHmPRnpr6bzfWIdJwlt/BO0QH5G+hSZI+GQQ/SyOrTiKuLH3017X/HWQ1R1H6J4/e345q1Oe38i8yQccoimafjrr8VuOYgz0DH+GybJGe4l/OoviASKKLs5s5dMReZIOOQYv3k9aBA73JCW9pUTJ/zKz1GRYZpqr0cPhNLSj/CehEOO0QvLMeauIXZoe0rb5E1EYhbkg9htFqGNXyMckp2rcpmEQw4KrtuMCg8QPfCqq+3G9j1P7MAr+Ffdjr9elpnPdRIOOcioXoJRt5Lo+8/ihJMvWT9Rsca3iOx8GN+iqwhe480y9yKzXAsH0zRvME3TNk3zO261KSYveM29EA0T2fnQlNuKWa8T3vbvGLXLCN3wZ2gerGQtMs+Vn7JpmkXAPwDPudGemDqjbC6BNZuIH3mT2PF3JtWGUorovhcIv/YrjDnLyfvUX6H5Ai5XKqYrt/4E/BT4R6DTpfaECwJX3olevYTwtl9inz0+ofeqeJTwa78isuNBfAvWk3f7d2XHqhlmyuFgmuYmoMSyrEddqEe4SDP85N36P9Dyixl+9sfET+9P6X3xNovhx75P/MgbBK68i9Ct35EjhhlIu9zW7QCmae4G5o31NPAqcKtlWWdN0/wPYJdlWT+bQA0LgKYJvF5MULy/izMP/x3Rs6coXn8bs677PL6i0ZchlVJETlv07nyS4SPv4ptVRcWmb5G/aI1HVYsMWgic+OiD44bD5ZimeR3wODB87qEKIAL8k2VZP0ixmQVAU1fXII4z+VqSqawsoqPDndH6TElXzSo6QmTX48QOvAyAMXsp+qwaMPyo4V7stiOokT4IFhBYcQuBNXeg+S9/GtHQ8BJbt26R73EGpKNmXdcoLy+EMcJhSsvEWZb1BnBhO6NJHjmIDNACeYSu/RKBFTcTa3yL+Kl9xI/vQjk2Wl4RRo2Jb+4qfIuuQvPLrEcha0jOOHrJbIIbthDcsMXrUsQ052o4WJb1NTfbE0J4R2azCCGSknAQQiQl4SCESErCQQiRlISDECIpCQchRFISDkKIpCQchBBJSTgIIZKScBBCJCXhIIRISsJBCJGUhIMQIikJByFEUhIOQoikpsNiLwYklqxKh3S1m07ZUnN+fj6QPfVeTGoe1Z6R7PkprSHpkuuA170uQogZ7HrgjY8+OB3CIQhcBbQBtse1CDGTGEAN8C6JhaFHmQ7hIISYhmRAUgiRlISDECIpCQchRFISDkKIpCQchBBJSTgIIZKScBBCJDUdpk9nhGmaNwCvAN+d7hv9mqZ5P3AziYkpgyRq3uVtVZcyTXMp8ABQDnQBX7Usq9HbqpIzTbMc+A2wGIgCjcA3Lcvq8LSwFJmm+TfA3wKrLMvan4k+Z8SRg2maRcA/AM95XUuKniPxIVgD/BB42ON6xvKvwP2WZS0F7gd+4XE9l6OAH1uWZVqWtQo4BvzI45pSYprmeuAaoDmT/c6IcAB+Cvwj0Ol1IamwLOsPlmXFzv1zB1Bnmua0+lmZplkFrAcePPfQg8B60zQrvatqbJZldVuWtf2ih3YC8z0qJ2WmaQZJBO+3M933tPrApYNpmpuAEsuyHvW6lkn6DvCsZVmO14V8xFygxbIsG+Dcf1vPPT6tnQvabwNPe11LCn4A/NayrBOZ7jjrxxxM09wNzBvraRKHjrdmrqLxjVNz9flfONM07wO+CGzMVG0zxD+TGMuZ7mNPHwc2AN/zov+cvvHKNM3rgMeB4XMPVZAY5Psny7J+4FlhKTBN827gJ8DNXvzVGM+504ojQLllWbZpmgaJQcn66TzIZ5rmT4DVwGcsy7rkTsTpxDTN7wF/SWIAFaAOaAe+blnWi+nuP6fD4aNM0/wPYFcWXK3YTOKv262WZR31up6xmKa5HfilZVm/NU3zy8CfWpZ1o8dljck0zb8HPg582rKs4fFeP92YpnkC2JypqxVZf1qRo35N4q/Fo6Zpnn/sZsuyurwrKalvAQ+Ypvl9oAf4qsf1jMk0zRXAX5M42nnr3Pe1ybKsuz0tbBqbUUcOQojU5fzVCiHE5Eg4CCGSknAQQiQl4SCESErCQQiRlISDECIpCQchRFISDkKIpP4/lFhZwQY1hBMAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 288x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "A = np.array([[1,1], [-1, 4]])\n",
    "linearTransformation(A)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this case, the eigenvectors are not orthogonal!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# References\n",
    "\n",
    "## Videos of Gilbert Strang\n",
    "\n",
    "- [Gilbert Strang, Lec21 MIT - Eigenvalues and eigenvectors](https://www.youtube.com/watch?v=lXNXrLcoerU)\n",
    "\n",
    "- [Gilbert Strang, Lec 21 MIT, Spring 2005](https://www.youtube.com/watch?v=lXNXrLcoerU)\n",
    "\n",
    "## Quadratic forms\n",
    "\n",
    "- [David Lay, University of Colorado, Denver](http://math.ucdenver.edu/~esulliva/LinearAlgebra/SlideShows/07_02.pdf)\n",
    "\n",
    "- [math.stackexchange QA](https://math.stackexchange.com/questions/2207111/eigendecomposition-optimization-of-quadratic-expressions)\n",
    "\n",
    "## Eigenvectors\n",
    "\n",
    "- [Victor Powell and Lewis Lehe - Interactive representation of eigenvectors](http://setosa.io/ev/eigenvectors-and-eigenvalues/)\n",
    "\n",
    "## Linear transformations\n",
    "\n",
    "- [Gilbert Strang - Linear transformation](http://ia802205.us.archive.org/18/items/MIT18.06S05_MP4/30.mp4)\n",
    "\n",
    "- [Linear transformation - demo video](https://www.youtube.com/watch?v=wXCRcnbCsJA)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.10"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}