{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "## Assignment 5 " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# MATH 7502 - Semsester 2, 2018\n", "## Mathematics for Data Science 2\n", "\n", "#### Created by Zhihao Qiao, Maria Kleshnina and Yoni Nazarathy" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Question 1" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Suppose $G_{k+2}$ is the average of two previous numbers $G_{k+1}$ and $G_k$\n", "\n", "$$ G_{k+2} = \\frac{1}{2}(G_{k+1}+G_k), \\ \\ \\ \\ G_{k+1} = G_{k+1}.$$\n", "\n", "(a) Find the eigenvalues and eigenvectors of matrix A such that\n", "$$ \\begin{bmatrix}\n", "G_{k+2}\\\\\n", "G_{k+1}\n", "\\end{bmatrix}\n", "=A \\begin{bmatrix}\n", "G_{k+1}\\\\\n", "G_{k}\n", "\\end{bmatrix}\n", "$$\n", "\n", "(b) Find the limit as $n \\rightarrow \\infty$ of the matrices $A^n = X \\Lambda X^{-1}.$\n", "\n", "(c) If $G_0=0, G_1=1$, find the limit. " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Question 2" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Let $ A = \\begin{bmatrix} \n", "6&3&-2\\\\\n", "-4&-1&2\\\\\n", "13&9&-3\n", "\\end{bmatrix}$. \n", "\n", "(a) Evaluate $e^{tA}.$\n", "\n", "(b) Find the general solutions of $\\frac{d\\vec{x}}{dt} = A\\vec{x}.$\n", "\n", "(c) Solve the initial value problem, if $\\vec{x(0)} = \\begin{bmatrix}\n", "-2\\\\\n", "1\\\\\n", "4\n", "\\end{bmatrix}.$\n", "\n", "(d) Give an example of matrix $M$ and $N$ such that \n", "$$ e^Me^N \\neq e^{M+N} $$\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Question 3" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Find the singular value decomposition $U\\Sigma V^T$ of $A$, where\n", "$$ A = \\begin{bmatrix}\n", "3&2&2\\\\\n", "2&3&-2\n", "\\end{bmatrix}.$$\n", "\n", "(a)What is the rank of $A$? \n", "\n", "(b) Suggest two rank 1 approximations of $A$ based on the SVD. Which one is better?\n", "\n", "(c) Write code that shows evidence of (b)." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Question 4" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Suppose we take a sample of 7 from the students' tests in Math, Science and History, the result is the following:\n", "\n", "| Math | Science | History |\n", "|------|----------|---------|\n", "| 7 | 5 | 4 |\n", "| 2 | 1.5 | 2.0 |\n", "| 3 | 9 | 8 |\n", "| 3 | 7 | 7 |\n", "| 5 | 3.5 | 5.0 |\n", "| 6 | 4.5 | 5.0 |\n", "| 7 | 3.5 | 4.5 |\n", "\n", "(a) Compute the sample covariance matrix $S$.\n", "\n", "(b) Find the eigenvalues of $S$. \n", "\n", "(c) Write code for (a) and (b).\n", "\n", "(d) Comment on the principal componenent analysis aspects of this and how they can be used.\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Question 5" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Suppose $Ax=\\lambda x$. If $\\lambda = 0$ then $x$ is in the nullspace. If $\\lambda \\neq 0$ then $x$ is in the column space. Those spaces have dimensions $(n-r)+r=n.$ Show that not every square matrix has $n$ linearly independent eigenvectors?" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Question 6" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Consider a data-set with $2M$ points, each a $2$-vector.\n", "\n", "The first $M$ points are given by, $i=1,\\ldots,M$ via the vectors $(i+1,i)$.\n", "\n", "The later $M$ points are given by, $i=1,\\ldots,M$ via the vectors $(i,i+1)$.\n", "\n", "For example, for $M=10$ a plot of the points is given below." ] }, { "cell_type": "code", "execution_count": 62, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAiIAAAGgCAYAAACXJAxkAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAAPYQAAD2EBqD+naQAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvIxREBQAAIABJREFUeJzt3X9slfX99/HX1dNRDl3PFUultLW1hbAbpRVBKrfg7WYkEkIaSRPFBQfTZCa7q1JJNilbdYvAEd38IsoNYhYHmSj+ASombiFMIUblV8VANkGhjKYWWINcB9tazTnX/cdJD3TggHpO3+3V5yM5OTmfXqfXmxPj9cx1XQcc3/d9AQAAGMiyHgAAAAxdhAgAADBDiAAAADOECAAAMEOIAAAAM4QIAAAwQ4gAAAAzhAgAADBDiAAAADOECAAAMEOIAAAAM9nWA/ynRCKhL774Qnl5eXIcx3ocAABwGXzf19mzZ1VcXKysrMs/zzHgQuSLL75QaWmp9RgAAKAPWlpadM0111z29gMuRPLy8iQl/yCRSMR4GgAA+pH3hfT//rekxHmLIen/fii5xVZTXZZYLKbS0tLUcfxyDbgQ6bkcE4lECBEAwNASiUh3r5K21kt+XHJCUs1KqXS89WSX7UpvqxhwIQIAwJA2eb409g7p9FEpf4zkllhPlFGECAAAA41bEvgA6cHXdwEAgBlCBAAAmCFEAACAGUIEAACYIUQAAIAZQgQAAJghRAAAgBlCBAAAmCFEAACAGUIEAACYIUQAAIAZQgQAAJghRAAAgBlCBAAAmCFEAACAGUIEAACYIUQAAIAZQgQAAJghRAAAgBlCBAAAmCFEAACAGUIEAACYIUQAAIAZQgQAAJghRAAAgJkrDpGdO3eqpqZGxcXFchxHb7zxRq+f+76vxx9/XEVFRQqHw5oxY4Y+++yztA0MAMAV8Vql5p3JZ2TMCa+rT++74hDp6OjQxIkTtXr16ov+/Omnn9aqVau0du1a7dq1S7m5uZo5c6a+/vrrPg0IAECfNW2QVlZK62uSz00brCcKpE17juvO/9nZp/dmX+kbZs2apVmzZl30Z77va+XKlfrtb3+ru+66S5K0YcMGFRYW6o033tC99957wXu6u7vV3d2deh2Lxa50JAAALuS1SlsXSn4i+dpPSFvrpbF3SG6J7WwB0uZ1qWHzASX8vr0/rfeINDc368SJE5oxY0ZqzXVdTZ06VR9++OFF3xONRuW6bupRWlqazpEAAEPV6SPnIqSHH5dOH7WZJ6Ca2zv6HCFSmkPkxIkTkqTCwsJe64WFhamf/aeGhgZ5npd6tLS0pHMkAMBQlT9Wcv7jMOeEpPwxNvMEVEVBrrKcvr/f/FszOTk5ikQivR4AAHxvbolU81wyPqTkc81KLsukWZEbVrS2SiGnbzVyxfeI/DejR4+WJJ08eVJFRUWp9ZMnT+rGG29M564AALi0yfOT94ScPpo8E0KEZMTc6jJNGp2j//U/V/7etJ4Rqaio0OjRo7V9+/bUWiwW065du3TLLbekc1cAAFwet0Sq+D9ESIaNdsN9et8VnxH56quv9Pnnn6deNzc3a//+/crPz1dZWZnq6+u1dOlSjRs3ThUVFWpsbFRxcbHmzJnTpwEBAEBwXXGI7N27V7fffnvq9aJFiyRJCxYs0J///Gf9+te/VkdHhx588EGdOXNGt956q/76179q+PDh6ZsaAAAEguP7/vf40k36xWIxua4rz/O4cRUAgEGir8dv82/NAACAoYsQAQAAZggRAABghhABAABmCBEAAGCGEAEAAGYIEQAAYIYQAQAAZggRAABghhABAABmCBEAAGCGEAEAAGYIEQAAYIYQAQAAZggRAABghhABAABmCBEAAGCGEAEAAGYIEQAAYIYQAQAAZggRAABghhABAABmCBEAAGCGEAEAXD6vVWremXxGxrR5XfrgSLvavC7rUTIu23oAAMAg0bRB2rpQ8hOSkyXVPCdNnm89VeBs2nNcDZsPKOFLWY4Ura3S3Ooy67EyhjMiAIBL81rPRYiUfN5az5mRNGvzulIRIkkJX1qy+WCgz4wQIgCASzt95FyE9PDj0umjNvMEVHN7RypCesR9X8faO20G6geECADg0vLHJi/HnM8JSfljbOYJqIqCXGU5vddCjqPyghE2A/UDQgQAcGluSfKeECeUfO2EpJqVyXWkTZEbVrS2SiEnWSMhx9Hy2koVuWHjyTLH8X3fv/Rm/ScWi8l1XXmep0gkYj0OAOB8Xmvyckz+GCIkg9q8Lh1r71R5wYhBEyF9PX7zrRkAwOVzSwiQflDkhgdNgHxfXJoBAABmCBEAAGCGEAEAAGYIEQAAYIYQAQAAZggRAABghhABAABmCBEAAGCGEAEAAGYIEQAAYIYQAQAAZggRAABghhABAABmCBEAAGCGEAEAAGYIEQAAYIYQAQAAZggRAABghhABAABmCBEAAGCGEAEAAGYIEQAAYIYQAQAAZggRAABghhABAABm0h4i8XhcjY2NqqioUDgc1tixY/Xkk0/K9/107woABjevVWremXxGxrR5XfrgSLvavC7rUXAR2en+hStWrNCaNWu0fv16TZgwQXv37tX9998v13X1yCOPpHt3ADA4NW2Qti6U/ITkZEk1z0mT51tPFTib9hxXw+YDSvhSliNFa6s0t7rMeiycJ+0h8sEHH+iuu+7S7NmzJUnl5eV69dVXtXv37otu393dre7u7tTrWCyW7pEAYGDxWs9FiJR83lovjb1DcktsZwuQNq8rFSGSlPClJZsP6rYfXa0iN2w7HFLSfmlm2rRp2r59uw4fPixJ+uSTT/T+++9r1qxZF90+Go3Kdd3Uo7S0NN0jAcDAcvrIuQjp4cel00dt5gmo5vaOVIT0iPu+jrV32gyEi0r7GZHFixcrFotp/PjxCoVCisfjWrZsmebNm3fR7RsaGrRo0aLU61gsRowACLb8scnLMefHiBOS8sfYzRRAFQW5ynLUK0ZCjqPyghF2Q+ECaT8j8vrrr+uVV17Rxo0b1dTUpPXr1+sPf/iD1q9ff9Htc3JyFIlEej0AINDckuQ9IU4o+doJSTUruSyTZkVuWNHaKoUcR1IyQpbXVnJZZoBx/DR/naW0tFSLFy9WXV1dam3p0qX6y1/+ok8//fSS74/FYnJdV57nESUAgs1rTV6OyR9DhGRQm9elY+2dKi8YQYRkUF+P32m/NNPZ2amsrN4nWkKhkBKJxHe8AwCGKLeEAOkHRW6YABnA0h4iNTU1WrZsmcrKyjRhwgR9/PHHevbZZ/XAAw+ke1cAAGCQS/ulmbNnz6qxsVFbtmzRqVOnVFxcrJ/+9Kd6/PHHNWzYsEu+n0szAAAMPn09fqc9RL4vQgQAgMGnr8dv/q0ZAABghhABAABmCBEAAGCGEAEAAGYIEQAAYIYQAQAAZggRAABghhABAABmCBEAAGCGEAEAAGYIEQAAYIYQAQAAZggRAABghhABAABmCBEAAGCGEAEAAGYIEQAAYIYQAQAAZggRAABghhABAABmCBEAAGCGEAEAAGYIEQAAYIYQARAMXqvUvDP5jIxp87r0wZF2tXld1qMgILKtBwCA761pg7R1oeQnJCdLqnlOmjzfeqrA2bTnuBo2H1DCl7IcKVpbpbnVZdZjYZDjjAiAwc1rPRchUvJ5az1nRtKszetKRYgkJXxpyeaDnBnB90aIABjcTh85FyE9/Lh0+qjNPAHV3N6RipAecd/XsfZOm4EQGIQIgMEtf2zycsz5nJCUP8ZmnoCqKMhVltN7LeQ4Ki8YYTMQAoMQATC4uSXJe0KcUPK1E5JqVibXkTZFbljR2iqFnGSNhBxHy2srVeSGjSfDYOf4vu9ferP+E4vF5LquPM9TJBKxHgfAYOG1Ji/H5I8hQjKozevSsfZOlReMIELQS1+P33xrBkAwuCUESD8ocsMECNKKSzMAAMAMIQIAAMwQIgAAwAwhAgAAzBAiAADADCECAADMECIAAMAMIQIAAMwQIgAAwAwhAgAAzBAiAADADCECAADMECIAAMAMIQIAAMwQIgAAwAwhAgAAzBAiAADADCECAADMECIAAMAMIQIAAMwQIgAAwAwhAgAAzBAiAADADCECAADMECIAAMBMRkKktbVV9913n0aOHKlwOKyqqirt3bs3E7sCAACDWHa6f+GXX36p6dOn6/bbb9c777yjq6++Wp999pmuuuqqdO8KQKZ4rdLpI1L+WMktsZ4msNq8LjW3d6iiIFdFbth6HMBE2kNkxYoVKi0t1csvv5xaq6ioSPduAGRK0wZp60LJT0hOllTznDR5vvVUgbNpz3E1bD6ghC9lOVK0tkpzq8usxwL6Xdovzbz11luaMmWK7r77bo0aNUqTJk3SSy+99J3bd3d3KxaL9XoAMOK1nosQKfm8tT65jrRp87pSESJJCV9asvmg2rwu28EAA2kPkaNHj2rNmjUaN26c/va3v+mXv/ylHnnkEa1fv/6i20ejUbmum3qUlpameyQAl+v0kXMR0sOPS6eP2swTUM3tHakI6RH3fR1r77QZCDCU9hBJJBKaPHmyli9frkmTJunBBx/UL37xC61du/ai2zc0NMjzvNSjpaUl3SMBuFz5Y5OXY87nhKT8MTbzBFRFQa6ynN5rIcdRecEIm4EAQ2kPkaKiIl1//fW91q677jodP378otvn5OQoEon0egAw4pYk7wlxQsnXTkiqWckNq2lW5IYVra1SyEnWSMhxtLy2khtWMSSl/WbV6dOn69ChQ73WDh8+rGuvvTbduwKQCZPnS2PvSF6OyR9DhGTI3Ooy3fajq3WsvVPlBSOIEAxZaQ+RRx99VNOmTdPy5ct1zz33aPfu3Vq3bp3WrVuX7l0ByBS3hADpB0VumADBkJf2SzPV1dXasmWLXn31VVVWVurJJ5/UypUrNW/evHTvCgAADHKO7/v+pTfrP7FYTK7ryvM87hcBAGCQ6Ovxm39rBgAAmCFEAACAGUIEAACYIUQAAIAZQgQAAJghRAAAgBlCBAAAmCFEAACAGUIEAACYIUQAAIAZQgQAAJghRAAAgBlCBAAAmCFEAACAGUIEAACYIUQAAIAZQgQAAJghRAAAgBlCBAAAmCFEAACAGUIEAACYIUQAAIAZQgQAAJghRIBM81ql5p3JZ2RMm9elD460q83rsh4FwBXIth4ACLSmDdLWhZKfkJwsqeY5afJ866kCZ9Oe42rYfEAJX8pypGhtleZWl1mPBeAycEYEyBSv9VyESMnnrfWcGUmzNq8rFSGSlPClJZsPcmYEGCQIESBTTh85FyE9/Lh0+qjNPAHV3N6RipAecd/XsfZOm4EAXBFCBMiU/LHJyzHnc0JS/hibeQKqoiBXWU7vtZDjqLxghM1AAK4IIQJkiluSvCfECSVfOyGpZmVyHWlT5IYVra1SyEnWSMhxtLy2UkVu2HgyAJfD8X3fv/Rm/ScWi8l1XXmep0gkYj0O8P15rcnLMfljiJAMavO6dKy9U+UFI4gQwEBfj998awbINLeEAOkHRW6YAAEGIS7NAAAAM4QIAAAwQ4gAAAAzhAgAADBDiAAAADOECAAAMEOIAAAAM4QIAAAwQ4gAAAAzhAgAADBDiAAAADOECAAAMEOIAAAAM4QIAAAwQ4gAAAAzhAgAADBDiAAAADOECAAAMEOIAAAAM4QIAAAwQ4gAAAAzhAgAADBDiAAAADOECAAAMEOIAAAAMxkPkaeeekqO46i+vj7TuwIAAINMRkNkz549evHFF3XDDTdkcjcYSrxWqXln8hkZ0+Z16YMj7WrzuqxHARBw2Zn6xV999ZXmzZunl156SUuXLs3UbjCUNG2Qti6U/ITkZEk1z0mT51tPFTib9hxXw+YDSvhSliNFa6s0t7rMeiwAAZWxMyJ1dXWaPXu2ZsyY8V+36+7uViwW6/UALuC1nosQKfm8tZ4zI2nW5nWlIkSSEr60ZPNBzowAyJiMhMhrr72mpqYmRaPRS24bjUblum7qUVpamomRMNidPnIuQnr4cen0UZt5Aqq5vSMVIT3ivq9j7Z02AwEIvLSHSEtLixYuXKhXXnlFw4cPv+T2DQ0N8jwv9WhpaUn3SAiC/LHJyzHnc0JS/hibeQKqoiBXWU7vtZDjqLxghM1AAAIv7SGyb98+nTp1SpMnT1Z2drays7O1Y8cOrVq1StnZ2YrH4722z8nJUSQS6fUALuCWJO8JcULJ105IqlmZXEfaFLlhRWurFHKSNRJyHC2vrVSRGzaeDEBQOb7v+5fe7PKdPXtW//rXv3qt3X///Ro/frwee+wxVVZW/tf3x2Ixua4rz/OIElzIa01ejskfQ4RkUJvXpWPtnSovGEGEALgsfT1+p/1bM3l5eRfERm5urkaOHHnJCAEuyS0hQPpBkRsmQAD0C/5mVQAAYCZjf4/I+d57773+2A0AABhkOCMCAADMECIAAMAMIQIAAMwQIgAAwAwhAgAAzBAiAADADCECAADMECIAAMAMIQIAAMwQIgAAwAwhAgAAzBAiAADADCECAADMECIAAMAMIQIAAMwQIgAAwAwhAgAAzBAiAADADCECAADMECIAAMAMIQIAAMwQIgAAwAwhAgAAzBAiQ5nXKjXvTD4jY9q8Ln1wpF1tXpf1KAAw4GRbDwAjTRukrQslPyE5WVLNc9Lk+dZTBc6mPcfVsPmAEr6U5UjR2irNrS6zHgsABgzOiAxFXuu5CJGSz1vrOTOSZm1eVypCJCnhS0s2H+TMCACchxAZik4fORchPfy4dPqozTwB1dzekYqQHnHf17H2TpuBAGAAIkSGovyxycsx53NCUv4Ym3kCqqIgV1lO77WQ46i8YITNQAAwABEiQ5FbkrwnxAklXzshqWZlch1pU+SGFa2tUshJ1kjIcbS8tlJFbth4MgAYOBzf9/1Lb9Z/YrGYXNeV53mKRCLW4wSb15q8HJM/hgjJoDavS8faO1VeMIIIARBYfT1+862ZocwtIUD6QZEbJkAA4DtwaQYAAJghRAAAgBlCBAAAmCFEAACAGUIEAACYIUQAAIAZQgQAAJghRAAAgBlCBAAAmCFEAACAGUIEAACYIUQAAIAZQgQAAJghRAAAgBlCBAAAmCFEAACAGUIEAACYIUQAAIAZQgQAAJghRAAAgBlCBAAAmCFEAACAGUIEAACYIUQAAIAZQgQAAJhJe4hEo1FVV1crLy9Po0aN0pw5c3To0KF07wYAAARA2kNkx44dqqur00cffaRt27bp22+/1Z133qmOjo5072rg8Fql5p3JZ2RUm9elD460q83rsh4FAJAGju/7fiZ38O9//1ujRo3Sjh07dNttt11y+1gsJtd15XmeIpFIJkdLj6YN0taFkp+QnCyp5jlp8nzrqQJp057jath8QAlfynKkaG2V5laXWY8FAFDfj98Zv0fE8zxJUn5+/kV/3t3drVgs1usxaHit5yJESj5vrefMSAa0eV2pCJGkhC8t2XyQMyMAMMhlNEQSiYTq6+s1ffp0VVZWXnSbaDQq13VTj9LS0kyOlF6nj5yLkB5+XDp91GaeAGtu70hFSI+47+tYe6fNQACAtMhoiNTV1engwYN67bXXvnObhoYGeZ6XerS0tGRypPTKH5u8HHM+JyTlj7GZJ8AqCnKV5fReCzmOygtG2AwEAEiLjIXIQw89pLffflvvvvuurrnmmu/cLicnR5FIpNdj0HBLkveEOKHkayck1axMriOtityworVVCjnJGgk5jpbXVqrIDRtPBgD4PtJ+s6rv+3r44Ye1ZcsWvffeexo3btwVvX/Q3awqJe8JOX00eSaECMmoNq9Lx9o7VV4wgggBgAGkr8fv7HQPUldXp40bN+rNN99UXl6eTpw4IUlyXVfhcEAPHG4JAdJPitwwAQIAAZL2MyKO41x0/eWXX9bPf/7zS75/UJ4RAQBgiBswZ0Qy/NeSAACAAOHfmgEAAGYIEQAAYIYQAQAAZggRAABghhABAABmCBEAAGCGEAEAAGYIEQAAYIYQAQAAZggRAABghhABAABmCBEAAGCGEAEAAGYIEQAAYIYQAQAAZggRAABghhABAABmCBEAAGCGEAEAAGYIEQAAYIYQAQAAZggRAABghhABAABmCBEAAGCGEBnC2rwufXCkXW1el/UoAIAhKtt6ANjYtOe4GjYfUMKXshwpWluludVl1mMBAIYYzogMQW1eVypCJCnhS0s2H+TMCACg3xEiQ1Bze0cqQnrEfV/H2jttBgIADFmEyBBUUZCrLKf3WshxVF4wwmYgAMCQRYgMQUVuWNHaKoWcZI2EHEfLaytV5IaNJwMADDXcrDpEza0u020/ulrH2jtVXjCCCAEAmCBEhrAiN0yAAABMcWkGAACYIUQAAIAZQgQAAJghRAAAgBlCBAAAmCFEAACAGUIEAACYIUQAAIAZQgQAAJghRAAAgBlCBAAAmCFEAACAGUIEAACYIUQAAIAZQgQAAJghRAAAgBlCBAAAmCFEAACAGUIEAACYIUQAAIAZQgQAAJghRAAAgBlCBAAAmCFEAACAGUIEAACYyViIrF69WuXl5Ro+fLimTp2q3bt3Z2pXAABgkMpIiGzatEmLFi3SE088oaamJk2cOFEzZ87UqVOnMrE7AAAwSDm+7/vp/qVTp05VdXW1XnjhBUlSIpFQaWmpHn74YS1evLjXtt3d3eru7k699jxPZWVlamlpUSQSSfdoAAAgA2KxmEpLS3XmzBm5rnvZ78tO9yDffPON9u3bp4aGhtRaVlaWZsyYoQ8//PCC7aPRqH7/+99fsF5aWpru0QAAQIadPXvWNkTa29sVj8dVWFjYa72wsFCffvrpBds3NDRo0aJFqddnzpzRtddeq+PHj1/RHwRXrqdeOfuUWXzO/YPPuX/wOfePwfg5+76vs2fPqri4+Irel/YQuVI5OTnKycm5YN113UHz4Q92kUiEz7of8Dn3Dz7n/sHn3D8G2+fclxMIab9ZtaCgQKFQSCdPnuy1fvLkSY0ePTrduwMAAINY2kNk2LBhuummm7R9+/bUWiKR0Pbt23XLLbeke3cAAGAQC/3ud7/7Xbp/aSQSUWNjo0pLS5WTk6PGxkbt379ff/rTn/TDH/7w0kOFQvrJT36i7GzzK0eBx2fdP/ic+wefc//gc+4fQ+VzzsjXdyXphRde0DPPPKMTJ07oxhtv1KpVqzR16tRM7AoAAAxSGQsRAACAS+HfmgEAAGYIEQAAYIYQAQAAZggRAABgZsCFyOrVq1VeXq7hw4dr6tSp2r17t/VIgRKNRlVdXa28vDyNGjVKc+bM0aFDh6zHCrynnnpKjuOovr7eepTAaW1t1X333aeRI0cqHA6rqqpKe/futR4rcOLxuBobG1VRUaFwOKyxY8fqySefFN93+H527typmpoaFRcXy3EcvfHGG71+7vu+Hn/8cRUVFSkcDmvGjBn67LPPjKbNjAEVIps2bdKiRYv0xBNPqKmpSRMnTtTMmTN16tQp69ECY8eOHaqrq9NHH32kbdu26dtvv9Wdd96pjo4O69ECa8+ePXrxxRd1ww03WI8SOF9++aWmT5+uH/zgB3rnnXf0j3/8Q3/84x911VVXWY8WOCtWrNCaNWv0wgsv6J///KdWrFihp59+Ws8//7z1aINaR0eHJk6cqNWrV1/0508//bRWrVqltWvXateuXcrNzdXMmTP19ddf9/OkGeQPIDfffLNfV1eXeh2Px/3i4mI/Go0aThVsp06d8iX5O3bssB4lkM6ePeuPGzfO37Ztm//jH//YX7hwofVIgfLYY4/5t956q/UYQ8Ls2bP9Bx54oNdabW2tP2/ePKOJgkeSv2XLltTrRCLhjx492n/mmWdSa2fOnPFzcnL8V1991WLEjBgwZ0S++eYb7du3TzNmzEitZWVlacaMGfrwww8NJws2z/MkSfn5+caTBFNdXZ1mz57d679rpM9bb72lKVOm6O6779aoUaM0adIkvfTSS9ZjBdK0adO0fft2HT58WJL0ySef6P3339esWbOMJwuu5uZmnThxotf/P1zX1dSpUwN1XBwwf29se3u74vG4CgsLe60XFhbq008/NZoq2BKJhOrr6zV9+nRVVlZajxM4r732mpqamrRnzx7rUQLr6NGjWrNmjRYtWqQlS5Zoz549euSRRzRs2DAtWLDAerxAWbx4sWKxmMaPH69QKKR4PK5ly5Zp3rx51qMF1okTJyTposfFnp8FwYAJEfS/uro6HTx4UO+//771KIHT0tKihQsXatu2bRo+fLj1OIGVSCQ0ZcoULV++XJI0adIkHTx4UGvXriVE0uz111/XK6+8oo0bN2rChAnav3+/6uvrVVxczGeN72XAXJopKChQKBTSyZMne62fPHlSo0ePNpoquB566CG9/fbbevfdd3XNNddYjxM4+/bt06lTpzR58mRlZ2crOztbO3bs0KpVq5Sdna14PG49YiAUFRXp+uuv77V23XXX6fjx40YTBdevfvUrLV68WPfee6+qqqr0s5/9TI8++qii0aj1aIHVc+wL+nFxwITIsGHDdNNNN2n79u2ptUQioe3bt+uWW24xnCxYfN/XQw89pC1btujvf/+7KioqrEcKpDvuuEMHDhzQ/v37U48pU6Zo3rx52r9/v0KhkPWIgTB9+vQLvn5++PBhXXvttUYTBVdnZ6eysnofMkKhkBKJhNFEwVdRUaHRo0f3Oi7GYjHt2rUrUMfFAXVpZtGiRVqwYIGmTJmim2++WStXrlRHR4fuv/9+69ECo66uThs3btSbb76pvLy81HVG13UVDoeNpwuOvLy8C+67yc3N1ciRI7kfJ40effRRTZs2TcuXL9c999yj3bt3a926dVq3bp31aIFTU1OjZcuWqaysTBMmTNDHH3+sZ599Vg888ID1aIPaV199pc8//zz1urm5Wfv371d+fr7KyspUX1+vpUuXaty4caqoqFBjY6OKi4s1Z84cw6nTzPprO//p+eef98vKyvxhw4b5N998s//RRx9ZjxQoki76ePnll61HCzy+vpsZW7du9SsrK/2cnBx//Pjx/rp166xHCqRYLOYvXLjQLysr84cPH+6PGTPG/81mgMfxAAAAgElEQVRvfuN3d3dbjzaovfvuuxf9f/KCBQt8309+hbexsdEvLCz0c3Jy/DvuuMM/dOiQ7dBp5vg+fy0eAACwMWDuEQEAAEMPIQIAAMwQIgAAwAwhAgAAzBAiAADADCECAADMECIAAMAMIQIAAMwQIgAAwAwhAgAAzBAiAADAzP8HHmzilAbn7UEAAAAASUVORK5CYII=", "text/plain": [ "PyPlot.Figure(PyObject
)" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "using PyPlot\n", "M = 10\n", "points1 = [[i+1,i] for i in 1:M]\n", "points2= [[i,i+1] for i in 1:M]\n", "plot(first.(points1),last.(points1),\".\")\n", "plot(first.(points2),last.(points2),\".\")\n", "xlim(0,M+1);ylim(0,M+1);" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "(a) Give an explicit expression for the normalized covariance matrix of these points. As an aid, below is a numerical computation of the covariance matrix." ] }, { "cell_type": "code", "execution_count": 63, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "2×2 Array{Float64,2}:\n", " 8.5 8.0\n", " 8.0 8.5" ] }, "execution_count": 63, "metadata": {}, "output_type": "execute_result" } ], "source": [ "X = [reshape(vcat(points1...),2,M) reshape(vcat(points2...),2,M)]\n", "muX = X*ones(2M,2M)/2M\n", "cvMat = (X-muX)*(X-muX)'/(2M)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "(b) Carry out PCA for this data-set, numerically." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "(c) Carry out PCA analytically and compare to your numerical results" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Question 7" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Each pair of singular vectors $v$ and $u$ has $Av=\\sigma u$ and $A^T u = \\sigma v$. \n", "\n", "(a) Show that the double vector \n", "$\\begin{bmatrix}\n", "u\\\\\n", "v\\end{bmatrix}$ is an eigenvector of the symmetric block matrix $M = \\begin{bmatrix}\n", "0&A^T\\\\\n", "A&0 \n", "\\end{bmatrix}$. \n", "\n", "(b) Show that the SVD of $A$ is equivalent to the diagonalization of $M$. \n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Question 8" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Suppose the same $X$ diagonalizes both $A$ and $B$. They have same eigenvectors in $A=X\\Lambda_1 X^{-1}$ and $B=X\\Lambda_2 X^{-1}$. Prove that $AB=BA$." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Question 9 \n", "\n", "Consider the code below.\n", "\n", "(a) Describe in words what the code does.\n", "\n", "(b) Replace line 5 of the code with a different implemination of pixel(). Have it create a square-looking donut. Carry out the computation with this image. Comment qualitatively on your results. " ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "using PyPlot,Distributions\n", "\n", "n,m = 100,100\n", "sig = 0.1\n", "pixel(i,j) = ( (i-50)^2+(j-50)^2 < 25^2 && (i-50)^2+(j-50)^2 > 10^2) + sig*rand(Normal())\n", "A = [pixel(i,j) for i in 1:n, j in 1:m]\n", "\n", "p = min(n,m)\n", "U,S,V = svd(A);\n", "svdApprox(k) = U[:,1:k]*diagm(S[1:k])*V[:,1:k]'\n", "err = [norm(A-svdApprox(k)) for k in 1:p]\n", "plot(1:p,err)" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "fig = figure(figsize=(10,10))\n", "d = 15\n", "for k in 1:d\n", " imm = fig[:add_subplot](d,1,k)\n", " imm[:imshow](svdApprox(k),cmap=\"Greys\")\n", "end" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Question 10 " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "This is a carry over from Question 10 of Assignment 4. If you didn't do it there, do it now: Carry out 16.5 from [VMLS], page 352.\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Question 11" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "This is a carry over from Question 11 of Assignment 4. If you didn't do it there, do it now" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Question 12" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "This is a carry over from Question 12 of Assignment 4. If you didn't do it there, do it now" ] } ], "metadata": { "kernelspec": { "display_name": "Julia 0.6.2", "language": "julia", "name": "julia-0.6" }, "language_info": { "file_extension": ".jl", "mimetype": "application/julia", "name": "julia", "version": "0.6.2" } }, "nbformat": 4, "nbformat_minor": 2 }