{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Before you turn this problem in, make sure everything runs as expected. First, **restart the kernel** (in the menubar, select Kernel$\\rightarrow$Restart) and then **run all cells** (in the menubar, select Cell$\\rightarrow$Run All).\n",
    "\n",
    "Make sure you fill in any place that says `YOUR CODE HERE` or \"YOUR ANSWER HERE\", as well as your name below.\n",
    "\n",
    "Rename this problem sheet as follows:\n",
    "\n",
    "    ps{number of lab}_{your user name}_problem{number of problem sheet in this lab}\n",
    "    \n",
    "for example\n",
    "    \n",
    "    ps2_blja_problem1\n",
    "\n",
    "Submit your homework within one week until next Monday, 9 a.m."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "NAME = \"\"\n",
    "EMAIL = \"\"\n",
    "USERNAME = \"\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": false,
    "editable": false,
    "nbgrader": {
     "cell_type": "markdown",
     "checksum": "62d13ca5fdad3dfd2069fa8991c7d3d5",
     "grade": false,
     "grade_id": "cell-2f2cbd6f50d7ae76",
     "locked": true,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "source": [
    "# Introduction to Data Science\n",
    "## Lab 7: Logistic regression\n",
    "### Part A: Implementing the logistic function"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": false,
    "editable": false,
    "nbgrader": {
     "cell_type": "markdown",
     "checksum": "514133bedeea20e361714718bc18ff98",
     "grade": false,
     "grade_id": "cell-50dffcded16d74eb",
     "locked": true,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "source": [
    "**Task**:\n",
    "Implement the logistic function\n",
    "\n",
    "$$ \\sigma(x) = \\frac{e^x}{1+e^x} = \\frac{1}{1+e^{-x}}$$\n",
    "\n",
    "using methods from `numpy`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "deletable": false,
    "nbgrader": {
     "cell_type": "code",
     "checksum": "8f5184b62b56e5295d25bcabbc80bbe2",
     "grade": false,
     "grade_id": "cell-e34e975989fd0590",
     "locked": false,
     "schema_version": 3,
     "solution": true,
     "task": false
    }
   },
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "def sigma(x):\n",
    "    # YOUR CODE HERE\n",
    "    raise NotImplementedError()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "deletable": false,
    "editable": false,
    "nbgrader": {
     "cell_type": "code",
     "checksum": "f0d0e110d1eb1fe6df5967af9cde11ee",
     "grade": true,
     "grade_id": "cell-9d5048808a942daa",
     "locked": true,
     "points": 1,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "outputs": [],
   "source": [
    "assert sigma(0) == 1/2\n",
    "assert all(sigma(np.ones((4,))) == 1/(1+np.exp(-1)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": false,
    "editable": false,
    "nbgrader": {
     "cell_type": "markdown",
     "checksum": "6db07d233d0e4d04e2efe5d8d2fc5d1d",
     "grade": false,
     "grade_id": "cell-e1200e1e06b2bf32",
     "locked": true,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "source": [
    "Next, we want to investigate how the shape of the logistic function changes for an affine linear input, i.e.,\n",
    "\n",
    "$$ \\sigma(\\beta_0 + \\beta_1 x) $$\n",
    "\n",
    "for different values of $\\beta_0$ and $\\beta_1$.\n",
    "\n",
    "For this purpose, the following function `plot_sigma` (which uses your function `sigma`) has been defined.\n",
    "\n",
    "**Task**: Understand and execute the following code cell! There will be no plot here, since we are only defining a function."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "deletable": false,
    "editable": false,
    "nbgrader": {
     "cell_type": "code",
     "checksum": "ffe0aceb2149ffb4787421db8453761f",
     "grade": false,
     "grade_id": "cell-0fc2c91c190de57a",
     "locked": true,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "outputs": [],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "%matplotlib inline\n",
    "\n",
    "def plot_sigma(b0, b1):\n",
    "    plt.figure('Sigma interactive')\n",
    "    x = np.linspace(-10,10,1001)\n",
    "    plt.plot(x, sigma(b0 + b1*x), label = 'Logistic function')\n",
    "    plt.plot(x, 0.5*np.ones(x.shape), label = 'Const 0.5')\n",
    "    plt.ylim(-0.1, 1.1)\n",
    "    plt.xlabel('x')\n",
    "    plt.ylabel('p(x)')\n",
    "    plt.legend()\n",
    "    plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": false,
    "editable": false,
    "nbgrader": {
     "cell_type": "markdown",
     "checksum": "7d6b9de0d0468b76e28029ad716730d3",
     "grade": false,
     "grade_id": "cell-2f913c1c252996d6",
     "locked": true,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "source": [
    "**Task**: Call the function `plot_sigma` with arguments `b0 = 0.3` and `b1 = 0.4`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "deletable": false,
    "nbgrader": {
     "cell_type": "code",
     "checksum": "ad3ea89dc99a9712e46956595734724e",
     "grade": true,
     "grade_id": "cell-49a4ec4e6f79756e",
     "locked": false,
     "points": 1,
     "schema_version": 3,
     "solution": true,
     "task": false
    }
   },
   "outputs": [],
   "source": [
    "# YOUR CODE HERE\n",
    "raise NotImplementedError()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": false,
    "editable": false,
    "nbgrader": {
     "cell_type": "markdown",
     "checksum": "ec73960740548dacd58a5fbb8aa08b76",
     "grade": false,
     "grade_id": "cell-767e9d8d7dd764c4",
     "locked": true,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "source": [
    "**Task**: Execute the next code cell and take your time to investigate how the values of $\\beta_0$ and $\\beta_1$ affect the behaviour of the shape of the function.\n",
    "\n",
    "**You have nothing to implement here, only evaluate the cell below and play with the sliders.**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "deletable": false,
    "editable": false,
    "nbgrader": {
     "cell_type": "code",
     "checksum": "e34cc951084f7e3faf62c3a07178714c",
     "grade": false,
     "grade_id": "cell-8d72088ec5537ae5",
     "locked": true,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "outputs": [],
   "source": [
    "from ipywidgets import interactive\n",
    "interactive_plot = interactive(plot_sigma, b0=(-10.0, 10.0, 1.0), b1=(-3., 3., 0.2))\n",
    "output = interactive_plot.children[-1]\n",
    "output.layout.height = '350px'\n",
    "interactive_plot"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": false,
    "editable": false,
    "nbgrader": {
     "cell_type": "markdown",
     "checksum": "a08182aeb18c717f86745348ed89a5bd",
     "grade": false,
     "grade_id": "cell-2ae207c2b0664853",
     "locked": true,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "source": [
    "### Decide whether the statements are true or false!\n",
    "\n",
    "Set the answer-variable either to `True` or `False`."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": false,
    "editable": false,
    "nbgrader": {
     "cell_type": "markdown",
     "checksum": "e4cfbf13b3e21132b0e9c1f45e694fa3",
     "grade": false,
     "grade_id": "cell-bacc8702c42ad836",
     "locked": true,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "source": [
    "**Statement A**: The logistic function is monotonically increasing for $\\beta_0 > 0$.\n",
    "Store your answer in the variable `answer_A`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "deletable": false,
    "nbgrader": {
     "cell_type": "code",
     "checksum": "0008573c3e590aabd12aad82ebebc34c",
     "grade": false,
     "grade_id": "cell-ec99f53493c4e27e",
     "locked": false,
     "schema_version": 3,
     "solution": true,
     "task": false
    }
   },
   "outputs": [],
   "source": [
    "# YOUR CODE HERE\n",
    "raise NotImplementedError()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "deletable": false,
    "editable": false,
    "nbgrader": {
     "cell_type": "code",
     "checksum": "2000ba6c088becdcc204a9a481e6be6e",
     "grade": true,
     "grade_id": "cell-2c3481b9216171ec",
     "locked": true,
     "points": 1,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "outputs": [],
   "source": [
    "assert 'answer_A' in locals()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": false,
    "editable": false,
    "nbgrader": {
     "cell_type": "markdown",
     "checksum": "f2d2272b01f20fb3e5bd8095f04343fd",
     "grade": false,
     "grade_id": "cell-a0c64afc3ed6e04c",
     "locked": true,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "source": [
    "**Statement B**: The logistic function is monotonically increasing for $\\beta_1 > 0$.\n",
    "Store your answer in the variable `answer_B`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "deletable": false,
    "nbgrader": {
     "cell_type": "code",
     "checksum": "82f5fc9f24214dd0de7ccb4f708e57a3",
     "grade": false,
     "grade_id": "cell-cbe3bf01fbd82764",
     "locked": false,
     "schema_version": 3,
     "solution": true,
     "task": false
    }
   },
   "outputs": [],
   "source": [
    "# YOUR CODE HERE\n",
    "raise NotImplementedError()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "deletable": false,
    "editable": false,
    "nbgrader": {
     "cell_type": "code",
     "checksum": "9ce330f68bbed287ded383504f7664e6",
     "grade": true,
     "grade_id": "cell-38098558fe896986",
     "locked": true,
     "points": 1,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "outputs": [],
   "source": [
    "assert 'answer_B' in locals()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": false,
    "editable": false,
    "nbgrader": {
     "cell_type": "markdown",
     "checksum": "ef86b9e17a4feb370dc1ad866b440d2d",
     "grade": false,
     "grade_id": "cell-1adc1112648671b2",
     "locked": true,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "source": [
    "**Task**: Determine the derivative of $\\sigma$ with respect to $x$, i.e., determine $\\sigma'(x)$ and write it in the following `Markdown` cell.\n",
    "Furthermore, confirm that the equation\n",
    "\n",
    "$$\n",
    "\\sigma'(x) = \\sigma(x) \\, \\left( 1- \\sigma(x) \\right)\n",
    "$$\n",
    "\n",
    "holds.\n",
    "\n",
    "**Hint**: Use the chain rule!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": false,
    "nbgrader": {
     "cell_type": "markdown",
     "checksum": "a24aae68824da2618e948845e50c5950",
     "grade": true,
     "grade_id": "cell-fd5a2bd92861c285",
     "locked": false,
     "points": 2,
     "schema_version": 3,
     "solution": true,
     "task": false
    }
   },
   "source": [
    "YOUR ANSWER HERE"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.7"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}