{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Assignment 4 "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# MATH 7502  - Semsester 2, 2018\n",
    "## Mathematics for Data Science 2\n",
    "\n",
    "#### Created by Zhihao Qiao,  Maria Kleshnina and Yoni Nazarathy"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Question 1 - Carry over of question 10 from previous assignmet:\n",
    "\n",
    "Solve Question 12.13 from [VMLS], page 241.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Question 2 - Carry over of question 11 from previous assignment:\n",
    "\n",
    "You are given a channel impulse response, the n-vector $c$. Your job is to find an equalizer impulse response, the n-vector $h$ that minimizes $\\|h*c-e_1\\|^2$. You can assume $c_1 \\neq 0 $. \n",
    "\n",
    "(a) Explain how to find $h$, Apply your method to find the equalizer $h$ for the channel $c = (1.0,0.7,-0.3,-0.1,0.05).$\n",
    "\n",
    "(b) Plot $c$, $h$, and $h*c$."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Question 3\n",
    "\n",
    "Say that you observe data points $(y_1,x_1,m_1),\\ldots,(y_n,x_n,m_1)$. Assume that $y$ and $x$ are real valued and that $m$ is binary valued. Say you wish to use least squares to fit a function,\n",
    "$$\n",
    "y(x) = \\beta_0 + \\beta_1 x + \\beta_2 m_1\n",
    "$$\n",
    "\n",
    "(i) Describe the $A$ matrix for the problem $\\min_{\\beta} ||A \\beta-y||$.\n",
    "\n",
    "(ii) Consider the data values below. Plot the data values on the x-y plane using different colors for m=0 and m=1\n",
    "\n",
    "(iii) Fit the model with for the data and plot the line(s) of best fit."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [],
   "source": [
    "xVals = [1, 6, 11, 16, 21, 26, 31, 36, 41, 46, 51, 56, 61, 66, 71]\n",
    "mVals = [1, 1, 0, 1, 1, 1, 1, 1, 0, 1, 0, 1, 1, 0, 1]\n",
    "yVals = [215.132, 259.396, 99.2338, 324.469, 253.776, 450.759,\n",
    "        305.793, 472.493, 258.894, 555.746, 335.42,\n",
    "        636.734, 624.769, 435.191, 638.885];"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Question 4\n",
    "\n",
    "Carry out 12.14 from [VMLS], page 242 dealing with recursive least squares."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Question 5\n",
    "\n",
    "Carry out 13.17 from [VMLS], pages 282-283."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Question 6\n",
    "\n",
    "Carry out 13.19 from [VMLS], page 283."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Question 7\n",
    "\n",
    "Carry out 14.17 from [VMLS], page 306."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Question 8\n",
    "\n",
    "Carry out 15.4 from [VMLS], page 334."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Question 9\n",
    "\n",
    "Carry out 15.11 from [VMLS], page 337."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Question 10\n",
    "\n",
    "Carry out 16.5 from [VMLS], page 352."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Question 11\n",
    "\n",
    "The code below considers a non-small least squares problem. With $A$ of dimension $40,000 \\times 1000$. It is constructed by selecting $A$ and $\\beta$ randomly with a fixed seed. Plot the running time of this code for increasing $n$ and $p$ (e.g. keep $p/n = 1/8$ and increase (or decrease $n$). ) Investigate the behaviour of the running time as the ratio of $p$ and $n$ changes."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "2.2697565816429908"
      ]
     },
     "execution_count": 24,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "using Distributions\n",
    "srand(1988)\n",
    "n = 40000\n",
    "p = 500\n",
    "A = round(50*rand(n,p))\n",
    "beta = 5*rand(p)\n",
    "y = A*beta + 2000*rand(Normal(),n);\n",
    "betaHat = A \\ y\n",
    "maximum(abs(betaHat - beta))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Question 12\n",
    "\n",
    "Implement a gradient descent algorithm for the the data of the previous question. Say that you wish to run it for a fixed number of iterations (with a specified fixed learning rate and fixed initial guess). Plot the maximum absolute value error rate as $n$ grows.\n"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Julia 0.6.2",
   "language": "julia",
   "name": "julia-0.6"
  },
  "language_info": {
   "file_extension": ".jl",
   "mimetype": "application/julia",
   "name": "julia",
   "version": "0.6.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}