{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Exercise 1 - Implementing the logistic function"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Task**:\n",
    "Implement the logistic function\n",
    "\n",
    "$$ \\sigma(x) = \\frac{e^x}{1+e^x} = \\frac{1}{1+e^{-x}}$$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "\n",
    "#def sigma(x):\n",
    "    # Put your definition here"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Next, we want to investigate how the shape of the logistic function changes for an affine linear input, i.e.,\n",
    "\n",
    "$$ \\sigma(\\beta_0 + \\beta_1 x) $$\n",
    "\n",
    "for different values of $\\beta_0$ and $\\beta_1$.\n",
    "\n",
    "**Task**: Take your time and try different values.\n",
    "What happens for negative/positive values of $\\beta_1$?\n",
    "What role does $\\beta_0$ play?\n",
    "\n",
    "**You have nothing to implement here, only evaluate the cells below.**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "def my_sigma(b0, b1) : return sigma(b0 + b1 * x)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "6fbac7ea43c144f9b598ee6e128d045d",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "interactive(children=(FloatSlider(value=0.0, description='b0', max=10.0, min=-10.0, step=1.0), FloatSlider(val…"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "%matplotlib inline\n",
    "from ipywidgets import interactive\n",
    "def f(b0, b1):\n",
    "    plt.figure(1)\n",
    "    x = np.linspace(-10,10,1001)\n",
    "    plt.plot(x, sigma(b0 + b1*x))\n",
    "    plt.plot(x,0.5*np.ones(x.shape))\n",
    "    plt.ylim(-0.1, 1.1)\n",
    "    plt.xlabel('x')\n",
    "    plt.ylabel('p(x)')\n",
    "    plt.show()\n",
    "\n",
    "interactive_plot = interactive(f, b0=(-10.0, 10.0, 1.0), b1=(-3., 3., 0.2))\n",
    "output = interactive_plot.children[-1]\n",
    "output.layout.height = '350px'\n",
    "interactive_plot"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Exercise 2 - Logistic regression in practice"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this lab, we want to investigate the `Default` data set known from the lecture.\n",
    "We first load the necessary modules.\n",
    "The command\n",
    "    \n",
    "    plt.rcParams['figure.figsize'] = [13, 5]\n",
    "    \n",
    "changes the size of the figure (in inches)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "plt.rcParams['figure.figsize'] = [13, 5]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Task**: Download the file `Default.csv` from the webpage and read it using the `pandas` function `read_csv`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Task**: Inspect the data using the methods you've learned so far, e.g., `describe`, `hist`, `head`, etc."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Observation**: If you try the `describe` function you should see that the predictors `default` and `student` are not part of the summary.\n",
    "This is due to the fact that these values were read in by the `read_csv` function as strings. We know from the lecture that these predictors are categorical (in particular binary)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In order to process these values we convert them to the data type `boolean`, i.e., we replace the `String` objects in the columns `default` and `student` by `Boolean`'s.\n",
    "There are a lot of ways to accomplish this task; the easiest might be\n",
    "\n",
    "    D.replace(to_replace='No',value=False,inplace=True)\n",
    "    \n",
    "**Task**: Replace every 'No' and 'Yes' in the `DataFrame` by the values `False` and `True`, resp."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Next, we want to plot both, the `income` and `balance` predictors as boxplots as a function of the `default` status.\n",
    "\n",
    "**Task**: Complete the plotting command in the following cell. What do you observe?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAyMAAAFYCAYAAABAqSPuAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4wLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvqOYd8AAAIABJREFUeJzt3XucXXV97//X2xCQYkQQSIGAoTZoMMdbU7BK20S8AJ6KPVVhqhU1JbUFjj2lhWh6xFsq4lFPrZaSNqlgyyD1QlNJQUoz1dRCAUUqRH7mIEokFTSIBBQIfH5/7BW6M8xkLsnMmsvr+XjMI3t/13et9dlr9kzmvb/ftVaqCkmSJEkab09quwBJkiRJ05NhRJIkSVIrDCOSJEmSWmEYkSRJktQKw4gkSZKkVhhGJEmSJLXCMCJJk0CSTyZ5f9t1tG1nxyHJm5OsH6P9vjvJ3wyz7+wkX0pyf5IP7+J+x+w1SdJEYBiRpBFIckeSnyTZmuTeJFckOazturolqSQ/33Yd09hS4AfAU6vqrN25Yb+3kqYaw4gkjdyvVdVTgIOB7wN/1nI9YyYd/l8xMs8Abi3vKixJQ/I/GEkapar6KfAZ4KjtbUn2TXJxknuSfCfJH2//Yz7JBUk+09X3g0muaf7gX5RkU5J3JvlBMwLzhsH2neS0JBuTbEmyJskhTfuXmi5fb0ZvTh5g3RlJPtzs59tJzmg+cd+jWd6XZEWSfwUeBH4uySHNfrY0+z2ta3s7TJ3a/lq6nt+R5B1Jbm1Gk/46yZO7lv/3JDcl+VGSryR5bteyFyT5ajPl6dPA4+sNfmjyZ0nuS/LNJMc1ja9LcmO/jmcluXyQjRyR5F+a/V4NHNBv+YuaWn+U5OtJFm0/FsCpwNnN8X9ZkqOT/FvTd3OSjyfZs+k/t/vYdx3/3x6gpiG/t5I02RhGJGmUkvwMcDJwbVfznwH7Aj8H/CrwJuAtzbKzgOc25wH8MrAEOLXrE/SfpfNH76F0/qBdmeRZA+z3pcAHgNfTGZ35DnApQFX9StPteVX1lKr69AClnwacADwfeCHwmgH6/Bad6Uazmu33ApuAQ4DXAn+y/Q/9YXoD8ErgmcCRwB83r+WFwGrgd4CnAxcCa5Ls1fzBfjnwKWB/4O+A3xhiP8cAt9M5jucCn0uyP7AGOCLJ/K6+b2y2PZBLgBub7byPzveDpuZDgSuA9zd1/SHw2SQHVtWbgb8Fzm+O/z8BjwL/q9nWLwHHAb83xOt4gmF+byVpUjGMSNLIXZ7kR8CPgZcDH4LOiAOdcPKOqrq/qu4APkznD3uq6kE6fwB/BPgb4Myq2tRv2/+7qh6qqn+h8wfv6wfY/xuA1VX11ap6CHgH8EtJ5g6z/tcDf1pVm6rqXuC8Afp8sqpuqaptdELSscA5VfXTqroJ+Kvtr2uYPl5Vd1bVFmAF0NO0nwZcWFXXVdWjVXUR8BDwouZrJvB/q+qRqvoMcP0Q+7m7q/+ngduAVzXH6dN0jj9JngPMBb7QfwNJDgd+kf/6XnwJ+IeuLm8E1lbV2qp6rKquBm4AThyooKq6saquraptzXviQjpBVZKmPcOIJI3ca6rqacBewBnAvyTZPqqxJ52RhO2+Q2ekA4Cq+nc6n9wHuKzfdu+tqgf6rXvIAPs/pHsfVbUV+GH3foZwCHBn1/M7B+jT3XYIsKWq7u9X23D313973a/rGcBZzRSmHzUh77Bm+SHA9/qde9F9bAcyUP/t+7oI+M0koROkLmtCSn+HMPD3YrtnAK/rV/OxdEapniDJkUm+kOQ/k/wY+BP6TfuSpOnKMCJJo9R8kv85OtNwjqVzBaVH6Pyxut3hwPe2P0lyOp0Qcxdwdr9N7pdkn37r3jXAru/q3kezztO79zOEzcCcrucDXQ2s+w/6u4D9k8zqV9v2/T0A/EzXsp8dYHvd++h+XXcCK6rqaV1fP1NVvU2dhzbhoXvdnRmo/10AVXUt8DDwy8BvMvgUrc0M/L3Y7k7gU/1q3qeqBhphArgA+CYwr6qeCryTThiFzrGDoY+fJE1JhhFJGqV0nATsB2yoqkfpjHasSDIryTOAP6AzJYskR9I5z+CNdD6ZPzvJ8/tt9j1J9mzOKfnvdM6T6O8S4C1Jnp9kLzqftF/XTAGCzhW+fm4npV8GvD3JoUmeBpyzs9dZVXcCXwE+kOTJzQnmS+icGwFwE3Bikv2bEaLfH2AzpyeZ05y/8U46U6YA/hJ4W5JjmuO5T5JXNcHn34BtwP9MskeS/wEcvbNagYOa/jOTvA6YD6ztWn4x8HFgW1UNeP+OqvoOnWlX278XxwK/1tXlb4BfS/LKdC4G8OR0TtqfM9D26Jx382Nga5JnA7/bta976IS6Nzbbeiud82oGM9T3VpImFcOIJI3cPyTZSucPzBV0TkK/pVl2Jp1Pu28H1tMJDqubqyX9DfDBqvp6VX2Lzh/ln2oCBcB/AvfS+ST/b4G3VdU3+++8qq4B/jfwWTqf4j8TOKWry7uBi5opRAOdc/KXwBeBm4Gv0fljfRudEZ7B9NA5x+Iu4PPAuc25EtAZYfg6cEez3YFOrL6kWXZ78/X+5rXcQOe8kY83r30j8OZm2cPA/2ie30vnfJzP7aRGgOuAeXRGqVYAr62qH3Yt/xSwgMFHRbb7TTonw2+hcyL8xdsXNOHsJDrfv3vojJT8EYP/n/qHzfbup3Ps+x+f05r1fwg8h07wG8y72fn3VpImlXgZdElqX3Np2L+pqsE+XR/LfZ8A/EVVPWPIzqPb/h3AbzdXlmpVkr3pnOT+wiYQSpJa5MiIJE0zSfZOcmIz9elQOp/8f77tusbJ7wLXG0QkaWLYY+gukqQpJsB76EwX+gmdSwi/q9WKxkEzQhMGvq+KJKkFTtPStDHaqSJJ+uhMn/mrsahLkiRpunKaliRJkqRWGEYkSZIktcIwounmF5PcmuTeJH/d3B9gv+buyPc07V8Y7H4BSZ6Z5J+T/DDJD5L8bXOfhu3L70jyh0luTnJfkk8neXLX8pOS3JTkx0n+X5Ljm/Z9k6xKsjnJ95K8P8mMsT8ckiRJ7TGMaLp5A/BKOvdlOBL4Yzo/B39N547Wh9M5offjg6wf4APAIXRupnYYnev+d3s9cDxwBPBcmnsmJDmazr0K/gh4GvArdO7LAHARnfs8/DzwAuAVwG+P+lVKkiRNAoYRTTcfr6o7q2oLnRui9VTVD6vqs1X1YFXd37T/6kArV9XGqrq6qh5q7pz8kQH6fqyq7mr28Q/A9jtsLwFWN+s/VlXfq6pvJpkNnAD8flU9UFV3Ax9lx5vYSZIkTTle2lfTzZ1dj78DHJLkZ+j88X88sF+zbFaSGVW1wx2pkxwEfAz4ZWAWnUB/b799/GfX4wfpjKJAZxRl7QA1PQOYCWxOsr3tSf1qlSRJmnIcGdF0c1jX48OBu4CzgGcBx1TVU+lMn4LOlKz+PgAU8Nym7xsH6TeQO+lMDxuo/SHggKp6WvP11Kp6zjC3K0mSNCkZRjTdnJ5kTpL9gXfSuenbLDrnifyoaT93J+vPArY2fQ+lc/7HcK0C3pLkuCRPSnJokmdX1Wbgi8CHkzy1WfbMJANOFZMkSZoqDCOabi6h84f/7c3X+4H/C+wN/AC4FrhyJ+u/B3ghcB+du1Z/brg7rqp/B95CZ0rYfcC/0JmiBfAmYE/gVjrTvj4DHDzcbUuSJE1G3oFdkiRJUiscGZEkSZLUCsOIJGlCS7I6yd1JvjHI8iT5WJKNzQ1HXzjeNUqSRscwIkma6D5J59LbgzkBmNd8LQUuGIeaJEm7gWFEkjShVdWXgC076XIScHF1XAs8LYkXgJCkScAwIkma7A5lx5uEbmraJEkT3IS+A/sBBxxQc+fObbuMKeeBBx5gn332absMaad8n+5+N9544w+q6sC26xgDA914dMBLRSZZSmcqF/vss88vPPvZzx7LuiRpWtiV/18mdBiZO3cuN9xwQ9tlTDl9fX0sWrSo7TKknfJ9uvsl+U7bNYyRTcBhXc/nAHcN1LGqVgIrARYuXFj+HyNJu25X/n9xmpYkabJbA7ypuarWi4D7qmpz20VJkoY2oUdGJElK0gssAg5Isgk4F5gJUFV/AawFTgQ2Ag8Cb2mnUknSSBlGJEkTWlX1DLG8gNPHqRxJ0m7kNC1JkiRJrTCMSJIkSWqFYUSSJElSKwwjkiRJklphGJEkSZLUCsOIJEmSpFYYRiRJkiS1wvuMSGpNklGt17mthCRJmuwcGZHUmqoa9OsZ53xh0GWSJGlqMIxIkiRJaoVhRJIkSVIrDCOSJEmSWmEYkSRJktQKw4gkSZKkVhhGJEmSJLXCMCJJkiSpFYYRSZIkSa0wjEiSJElqxZBhJMlhSdYl2ZDkliRvb9rfneR7SW5qvk7sWucdSTYmuS3JK7vaj2/aNiZZNjYvSZIkSdJksMcw+mwDzqqqryaZBdyY5Opm2Uer6v90d05yFHAK8BzgEOCfkhzZLP4E8HJgE3B9kjVVdevueCGSJEmSJpchw0hVbQY2N4/vT7IBOHQnq5wEXFpVDwHfTrIROLpZtrGqbgdIcmnT1zAiSZIkTUPDGRl5XJK5wAuA64CXAGckeRNwA53Rk3vpBJVru1bbxH+Flzv7tR8zwD6WAksBZs+eTV9f30hK1DBs3brV46pJwfepJElT27DDSJKnAJ8Ffr+qfpzkAuB9QDX/fhh4K5ABVi8GPj+lntBQtRJYCbBw4cJatGjRcEvUMPX19eFx1YR35RW+TyVJmuKGFUaSzKQTRP62qj4HUFXf71r+l8AXmqebgMO6Vp8D3NU8HqxdkiRJ0jQznKtpBVgFbKiqj3S1H9zV7deBbzSP1wCnJNkryRHAPODfgeuBeUmOSLInnZPc1+yelyFJkiRpshnOyMhLgN8C/iPJTU3bO4GeJM+nM9XqDuB3AKrqliSX0TkxfRtwelU9CpDkDOAqYAawuqpu2Y2vRZIkSdIkMpyraa1n4PNA1u5knRXAigHa1+5sPUmSJEnTh3dglyRJktQKw8g00tvby4IFCzjuuONYsGABvb29bZckSZKkacwwMk309vby9re/nQceeICq4oEHHuDtb3+7gUSSJEmtMYxME2effTYPP/zwDm0PP/wwZ599dksVSZIkabob0R3YNXlt2rSJ2bNns3r1ah599FFmzJhBT08PmzZtars0SZIkTVOOjEwjZ511FosXL2aPPfZg8eLFnHXWWW2XJEmSpGnMkZFp5CMf+QgLFy7k0UcfZd26dXzkIx8ZeiVJkiRpjBhGpok5c+awdetW3vrWt/Ld736Xww8/nJ/+9KfMmTOn7dIkSZI0TTlNa5o4//zzmTlz5g5tM2fO5Pzzz2+pIkmSJE13hpFpoqenh5NPPpnNmzfz2GOPsXnzZk4++WR6enraLk2SJEnTlNO0pone3l6uuOIK/vEf//Hxq2ktWbKEF7/4xQYSSZIktcKRkWlixYoVrFq1aoeraa1atYoVK1a0XZokSZKmKcPINLFhwwaOPfbYHdqOPfZYNmzY0FJFkiRJmu4MI9PE/PnzWb9+/Q5t69evZ/78+S1VJEmSpOnOMDJNLF++nCVLlrBu3Tq2bdvGunXrWLJkCcuXL2+7NEmSJE1TnsA+TWw/Sf3MM89kw4YNzJ8/nxUrVnjyuiRJklpjGJlGenp66Onpoa+vj0WLFrVdjiRJkqY5p2lJkiRJaoVhZBrp7e1lwYIFHHfccSxYsIDe3t62S5KkYUlyfJLbkmxMsmyA5YcnWZfka0luTnJiG3VKkkbGaVrTRG9vL8uXL2fVqlU73PQQ8LwRSRNakhnAJ4CXA5uA65Osqapbu7r9MXBZVV2Q5ChgLTB33IuVJI2IIyPThDc9lDSJHQ1srKrbq+ph4FLgpH59Cnhq83hf4K5xrE+SNEqOjEwT3vRQ0iR2KHBn1/NNwDH9+rwb+GKSM4F9gJeNT2mSpF3hyMg04U0PJU1iGaCt+j3vAT5ZVXOAE4FPJXnC/3FJlia5IckN99xzzxiUKkkaCcPINOFNDyVNYpuAw7qez+GJ07CWAJcBVNW/AU8GDui/oapaWVULq2rhgQceOEblSpKGy2la00RPTw9f+cpXOOGEE3jooYfYa6+9OO200zx5XdJkcD0wL8kRwPeAU4Df7Nfnu8BxwCeTzKcTRhz6kKQJzpGRaaK3t5eLL76Yxx57DIDHHnuMiy++2Mv7SprwqmobcAZwFbCBzlWzbkny3iSvbrqdBZyW5OtAL/Dmquo/lUuSNME4MjJNnHHGGdx///0cdNBB3H333ey///7cfffdnHHGGY6OSJrwqmotncv1dre9q+vxrcBLxrsuSdKuMYxME1u2bGG//fajt7f38fuM/MZv/AZbtmxpuzRJkiRNU07TmkbOPvvsHe4zcvbZZ7ddkiRJkqYxR0amkQ996EMcc8wxPProo6xbt44PfehDbZckSZKkacwwMkUlT7ws/5YtW3jpS1+6076e7ylJkqTx4jStKaqqdvi65JJLmDVrFjNnzgRg5syZzJo1i0suuWSHfpIkSdJ4MYxMEz09PVx44YUceeSRkCdx5JFHcuGFF3olLUmSJLXGaVrTSE9PDz09PcxddgXfOO9VbZcjSZKkac6REUmSJEmtMIxIkiRJaoVhRJIkSVIrhgwjSQ5Lsi7JhiS3JHl7075/kquTfKv5d7+mPUk+lmRjkpuTvLBrW6c2/b+V5NSxe1mSJEmSJrrhjIxsA86qqvnAi4DTkxwFLAOuqap5wDXNc4ATgHnN11LgAuiEF+Bc4BjgaODc7QFGkiRJ0vQzZBipqs1V9dXm8f3ABuBQ4CTgoqbbRcBrmscnARdXx7XA05IcDLwSuLqqtlTVvcDVwPG79dVIkiRJmjRGdM5IkrnAC4DrgNlVtRk6gQU4qOl2KHBn12qbmrbB2iVJkiRNQ8O+z0iSpwCfBX6/qn6cZNCuA7TVTtr772cpneldzJ49m76+vuGWqBHwuGoy8H0qSdLUNqwwkmQmnSDyt1X1uab5+0kOrqrNzTSsu5v2TcBhXavPAe5q2hf1a+/rv6+qWgmsBFi4cGEtWrSofxftqiuvwOOq8fS893yR+37yyIjXe/OVDwy77757z+Tr575ixPuQJEntGTKMpDMEsgrYUFUf6Vq0BjgVOK/59++72s9Icimdk9XvawLLVcCfdJ20/grgHbvnZUiayO77ySPccd6rRrROX1/fiELz3GVXjLAqSZLUtuGMjLwE+C3gP5Lc1LS9k04IuSzJEuC7wOuaZWuBE4GNwIPAWwCqakuS9wHXN/3eW1VbdsurkCRJkjTpDBlGqmo9A5/vAXDcAP0LOH2Qba0GVo+kQEmSJElTk3dglyRJktQKw4gkSZKkVhhGJEmSJLXCMCJJkiSpFYYRSZIkSa0wjEiSJElqhWFEkiRJUisMI5IkSZJaYRiRJEmS1ArDiCRJkqRWGEYkSZIktcIwIkmSJKkVhhFJkiRJrTCMSJIkSWrFHm0XIGnqmzV/Gf/tomUjX/GikewD4FUj34ckSWqNYUTSmLt/w3nccd7IgkJfXx+LFi0adv+5y64YYVWSJKltTtOSJEmS1ArDiCRJkqRWGEYkSZIktcIwIkmSJKkVhhFJkiRJrTCMSJIkSWqFYUSSJElSKwwjkiRJklphGJEkSZLUCsOIJEmSpFYYRiRJE16S45PclmRjkmWD9Hl9kluT3JLkkvGuUZI0cnu0XYAkSTuTZAbwCeDlwCbg+iRrqurWrj7zgHcAL6mqe5Mc1E61kqSRcGREkjTRHQ1srKrbq+ph4FLgpH59TgM+UVX3AlTV3eNcoyRpFAwjkqSJ7lDgzq7nm5q2bkcCRyb51yTXJjl+3KqTJI2a07QkSRNdBmirfs/3AOYBi4A5wJeTLKiqH+2woWQpsBTg8MMP3/2VSpJGxJERSdJEtwk4rOv5HOCuAfr8fVU9UlXfBm6jE052UFUrq2phVS088MADx6xgSdLwGEYkSRPd9cC8JEck2RM4BVjTr8/lwGKAJAfQmbZ1+7hWKUkaMcOIJGlCq6ptwBnAVcAG4LKquiXJe5O8uul2FfDDJLcC64A/qqoftlOxJGm4PGdEkjThVdVaYG2/tnd1PS7gD5ovSdIk4ciIJEmSpFYYRiRJkiS1YsgwkmR1kruTfKOr7d1JvpfkpubrxK5l70iyMcltSV7Z1X5807YxybLd/1IkSZIkTSbDGRn5JDDQzaM+WlXPb77WAiQ5is5VTp7TrPPnSWYkmQF8AjgBOAroafpKkiRJmqaGPIG9qr6UZO4wt3cScGlVPQR8O8lG4Ohm2caquh0gyaVN31tHXLEkSZKkKWFXrqZ1RpI3ATcAZ1XVvcChwLVdfTY1bQB39ms/Zhf2LWmSmbvsipGvdOXw19l375kj374kSWrVaMPIBcD7gGr+/TDwViAD9C0Gng5WA204yVJgKcDs2bPp6+sbZYnaGY+rxtMnj99nxOu8+coHRrye72tJkiaXUYWRqvr+9sdJ/hL4QvN0E3BYV9c5wF3N48Ha+297JbASYOHChbVo0aLRlKidufIKPK6a8HyfSpI05Y3q0r5JDu56+uvA9ittrQFOSbJXkiOAecC/A9cD85IckWRPOie5rxl92ZIkSZImuyFHRpL0AouAA5JsAs4FFiV5Pp2pVncAvwNQVbckuYzOienbgNOr6tFmO2cAVwEzgNVVdctufzWSJEmSJo3hXE2rZ4DmVTvpvwJYMUD7WmDtiKqTJEmSNGV5B3ZJkiRJrTCMSJIkSWqFYUSSJElSKwwjkiRJklphGJEkSZLUCsOIJEmSpFYYRiRJkiS1wjAiSZIkqRWGEUmSJEmtMIxIkiRJaoVhRJIkSVIrDCOSJEmSWmEYkSRJktQKw4gkSZKkVhhGJEmSJLXCMCJJkiSpFYYRSZIkSa0wjEiSJElqhWFEkiRJUisMI5IkSZJaYRiRJEmS1ArDiCRJkqRWGEYkSZIktcIwIkmSJKkVhhFJkiRJrTCMSJIkSWqFYUSSJElSKwwjkiRJklphGJEkSZLUij3aLkC75nnv+SL3/eSREa83d9kVw+67794z+fq5rxjxPiRJkqSdMYxMcvf95BHuOO9VI1qnr6+PRYsWDbv/SIKLJEmSNFxO05IkSZLUCsOIJEmSpFYYRiRJkiS1wjAiSZrwkhyf5LYkG5Ms20m/1yapJAvHsz5J0ugYRiRJE1qSGcAngBOAo4CeJEcN0G8W8D+B68a3QknSaBlGJEkT3dHAxqq6vaoeBi4FThqg3/uA84GfjmdxkqTRGzKMJFmd5O4k3+hq2z/J1Um+1fy7X9OeJB9rhtFvTvLCrnVObfp/K8mpY/NyJElT0KHAnV3PNzVtj0vyAuCwqvrCeBYmSdo1wxkZ+SRwfL+2ZcA1VTUPuKZ5Dp0h9HnN11LgAuiEF+Bc4Bg6n3Cduz3ASJI0hAzQVo8vTJ4EfBQ4a8gNJUuT3JDkhnvuuWc3lihJGo0hw0hVfQnY0q/5JOCi5vFFwGu62i+ujmuBpyU5GHglcHVVbamqe4GreWLAkSRpIJuAw7qezwHu6no+C1gA9CW5A3gRsGagk9iramVVLayqhQceeOAYlixJGo7RnjMyu6o2AzT/HtS0DzaUPuQQuyRJg7gemJfkiCR7AqcAa7YvrKr7quqAqppbVXOBa4FXV9UN7ZQrSRquPXbz9gYbSt/pEPsOG0iW0pnixezZs+nr69ttxU1VIz1GW7duHfE6fh/UBt93AqiqbUnOAK4CZgCrq+qWJO8FbqiqNTvfgiRpohptGPl+koOranMzDevupn2wofRNwKJ+7X0DbbiqVgIrARYuXFiLFi0aqJu2u/IKRnqM+vr6RrbOKPYh7TLfd+pSVWuBtf3a3jVI30XjUZMkadeNdprWGmD7FbFOBf6+q/1NzVW1XgTc10zjugp4RZL9mhPXX9G0SZIkSZqmhhwZSdJLZ1TjgCSb6FwV6zzgsiRLgO8Cr2u6rwVOBDYCDwJvAaiqLUneR2feL8B7q6r/SfGSJEmSppEhw0hV9Qyy6LgB+hZw+iDbWQ2sHlF1kqa0ZKDTybqWf3Dg9s6vGkmSNNl5B3ZJramqQb/WrVs36DJJkjQ1GEYkSZIktcIwIkmSJKkVhhFJkiRJrTCMSJIkSWqFYUSSJElSKwwjkiRJklphGJEkSZLUCsOIJEmSpFYYRiRJkiS1wjAiSZIkqRWGEUmSJEmtMIxImlB6e3tZsGABxx13HAsWLKC3t7ftkiRJ0hjZo+0CtGtmzV/Gf7to2chXvGgk+wB41cj3IY1Qb28vy5cvZ9WqVTz66KPMmDGDJUuWANDT09NydZIkaXczjExy9284jzvOG1lQ6OvrY9GiRcPuP3fZFSOsShqdFStWsGrVKhYvXvz4+3TVqlWceeaZhhFJkqYgp2lJmjA2bNjAscceu0Pbsccey4YNG1qqSJIkjSXDiKQJY/78+axfv36HtvXr1zN//vyWKpIkSWPJMCJpwli+fDlLlixh3bp1bNu2jXXr1rFkyRKWL1/edmmSJGkMeM6IpAlj+3khZ555Jhs2bGD+/PmsWLHC80UkSZqiDCOSJpSenh56enpGfKEFSZI0+ThNS9KE4n1GJEmaPhwZkTRheJ8RSZKmF0dGJE0Y3fcZ2WOPPVi8eDGrVq1ixYoVbZcmSZLGgGFE0oThfUYkSZpenKYlacKYP38+73nPe7j88ssfv5rWa17zGu8zIknSFGUYkTRhLF68mA9+8IN88IMf5KijjuLWW2/lnHPO4W1ve1vbpUmSpDFgGJE0Yaxbt45zzjmH1atXPz4ycs4553D55Ze3XZokSRoDhhFJE8aGDRv42te+xvvf//7H7zPyyCOP8IEPfKDt0iRJ0hjwBHZJE8b8+fNZv379Dm3r16/3nBFJkqYow4ikCWP58uUsWbKEdevWsW3bNtatW8eSJUtYvnx526VJkqQx4DQtSRPG9hsbnnnmmY+fM7JixQpveChJ0hRlGJE0ofT09NDT0/P4OSOSJGnqcpqWJEmSpFYYRiRJkiS1wjAiSZIkqRWGEUmSJEmtMIxIkiRJasUuhZEkdyT5jyQ3Jbmhads/ydVJvtX8u194PhTCAAAN4ElEQVTTniQfS7Ixyc1JXrg7XoAkSZKkyWl3jIwsrqrnV9XC5vky4Jqqmgdc0zwHOAGY13wtBS7YDfuWJEmSNEmNxTStk4CLmscXAa/par+4Oq4Fnpbk4DHYvyRJkqRJYFfDSAFfTHJjkqVN2+yq2gzQ/HtQ034ocGfXupuaNkmSJEnT0K7egf0lVXVXkoOAq5N8cyd9M0BbPaFTJ9QsBZg9ezZ9fX27WOLUN9JjtHXr1hGv4/dB420071NJkjS57FIYqaq7mn/vTvJ54Gjg+0kOrqrNzTSsu5vum4DDulafA9w1wDZXAisBFi5cWIsWLdqVEqe+K69gpMeor69vZOuMYh/Srhrx+1RTWpLjgT8FZgB/VVXn9Vv+B8BvA9uAe4C3VtV3xr1QSdKIjHqaVpJ9ksza/hh4BfANYA1watPtVODvm8drgDc1V9V6EXDf9ulckiQNJskM4BN0LoRyFNCT5Kh+3b4GLKyq5wKfAc4f3yolSaOxKyMjs4HPJ9m+nUuq6sok1wOXJVkCfBd4XdN/LXAisBF4EHjLLuxbXeYuu2LkK105/HX23XvmyLcvSbvP0cDGqrodIMmldC6Kcuv2DlW1rqv/tcAbx7VCSdKojDqMNP8pPG+A9h8Cxw3QXsDpo92fBnbHea8a8Tpzl10xqvUkqSUDXQDlmJ30XwL845hWJEnaLXb1BHZJksbasC6AApDkjcBC4FcHWf74RVIOP/zw3VWfJGmUxuI+I5Ik7U7DugBKkpcBy4FXV9VDA22oqlZW1cKqWnjggQeOSbGSpOEzjEiSJrrrgXlJjkiyJ3AKnYuiPC7JC4AL6QSRuwfYhiRpAjKMSJImtKraBpwBXAVsAC6rqluSvDfJq5tuHwKeAvxdkpuSrBlkc5KkCcRzRiRJE15VraVzVcbutnd1PX7ZuBclSdpljoxIkiRJaoVhRJIkSVIrDCOSJEmSWmEYkSRJktQKw4gkSZKkVhhGJEmSJLXCMCJJkiSpFYYRSZIkSa0wjEiSJElqhWFEkiRJUisMI5IkSZJaYRiRJEmS1ArDiCRJkqRWGEYkSZIktcIwIkmSJKkVhhFJkiRJrTCMSJIkSWqFYUSSJElSKwwjkiRJklphGJEkSZLUCsOIJEmSpFYYRiRJkiS1wjAiSZIkqRWGEUmSJEmtMIxIkiRJaoVhRJIkSVIrDCOSJEmSWmEYkSRJktQKw4gkSZKkVhhGJEmSJLXCMCJJkiSpFYYRSZIkSa0wjEiSJElqxbiHkSTHJ7ktycYky8Z7/5IkSZImhnENI0lmAJ8ATgCOAnqSHDWeNUiSJEmaGMZ7ZORoYGNV3V5VDwOXAieNcw2SJEmSJoDxDiOHAnd2Pd/UtEmSJEmaZvYY5/1lgLbaoUOyFFgKMHv2bPr6+sahrKln8eLFO12eDw7cvm7dujGoRhq5rVu3+vMvSdIUN95hZBNwWNfzOcBd3R2qaiWwEmDhwoW1aNGicStuKqmqQZf19fXhcdVE5/tUkqSpb7ynaV0PzEtyRJI9gVOANeNcgyRJkqQJYFxHRqpqW5IzgKuAGcDqqrplPGuQJEmSNDGM9zQtqmotsHa89ytJkiRpYvEO7JIkSZJaYRiRJEmS1ArDiCRJkqRWGEYkSRNekuOT3JZkY5JlAyzfK8mnm+XXJZk7/lVKkkbKMCJJmtCSzAA+AZwAHAX0JDmqX7clwL1V9fPAR4FBbu0qSZpIDCOSpInuaGBjVd1eVQ8DlwIn9etzEnBR8/gzwHFJMo41SpJGwTAiSZroDgXu7Hq+qWkbsE9VbQPuA54+LtVJkkZt3O8zMhI33njjD5J8p+06pqADgB+0XYQ0BN+nu98z2i5glAYa4ahR9CHJUmBp8/ShJN/YxdqmEn/mduTx2JHHY0cejx09a7QrTugwUlUHtl3DVJTkhqpa2HYd0s74PlWXTcBhXc/nAHcN0mdTkj2AfYEt/TdUVSuBleB7rD+Px448HjvyeOzI47GjJDeMdl2naUmSJrrrgXlJjkiyJ3AKsKZfnzXAqc3j1wL/XFVPGBmRJE0sE3pkRJKkqtqW5AzgKmAGsLqqbknyXuCGqloDrAI+lWQjnRGRU9qrWJI0XIaR6Wll2wVIw+D7VI+rqrXA2n5t7+p6/FPgdSPcrO+xHXk8duTx2JHHY0cejx2N+njEUWxJkiRJbfCcEUmSJEmtMIxMckkeTXJT19fcnfSd62Us1ZYkT+96n/5nku91Pd+z7fo0dSU5PsltSTYmWTbA8r2SfLpZft3Ofo9OBcM4Hn+Q5NYkNye5JslkvST0sAx1PLr6vTZJJZnSV1AazvFI8vrmPXJLkkvGu8bxNIyfl8OTrEvyteZn5sQ26hwPSVYnuXuwvyXT8bHmWN2c5IXD2q7TtCa3JFur6inD7DsX+EJVLRjToqQhJHk3sLWq/k+/9tD5vfRYK4VpykkyA/j/gJfTufzv9UBPVd3a1ef3gOdW1duSnAL8elWd3ErBY2yYx2MxcF1VPZjkd4FF0/l4NP1mAVcAewJnVNWoL2M6kQ3z/TEPuAx4aVXdm+Sgqrq7lYLH2DCPx0rga1V1QZKjgLVVNbeNesdakl8BtgIXD/S3ZBPEzgROBI4B/rSqjhlqu46MTEHNCMiXk3y1+XrxAH2ek+Tfm0+lb25+uZDkjV3tFzY/iNKYSfLzSb6R5C+ArwKHJflR1/JTkvxV83h2ks8luaF5n76orbo1aRwNbKyq26vqYeBS4KR+fU4CLmoefwY4rgnGU9GQx6Oq1lXVg83Ta+nc12WqGs77A+B9wPnAT8ezuBYM53icBnyiqu4FmKpBpDGc41HAU5vH+/LEeyBNGVX1JQa4f1OXk+gElaqqa4GnJTl4qO0aRia/vbumuny+absbeHlVvRA4GfjYAOu9jU5ifT6wkM6NwuY3/V/StD8KvGHsX4LEUcCqqnoB8L2d9PsYcH5zo6nXA381HsVpUjsUuLPr+aambcA+VbUNuA94+rhUN/6Gczy6LQH+cUwrateQxyPJC4DDquoL41lYS4bz/jgSODLJvya5Nsnx41bd+BvO8Xg38MYkm+hc8e/M8SltQhrp7xfAS/tOBT9pgkO3mcDHk2wPFEcOsN6/AcuTzAE+V1XfSnIc8AvA9c2HgnvTCTbSWPt/VXX9MPq9DHhW14fW+yXZu6p+MnalaZIbaISj//zk4fSZKob9WpO8kc6HVb86phW1a6fHI8mTgI8Cbx6vglo2nPfHHsA8YBGdUbMvJ1lQVT/qv+IUMJzj0QN8sqo+nOSX6NzvaME0nW48qt+lhpGp6X8B3weeR2f06wnDylV1SZLrgFcBVyX5bTpvoouq6h3jWawEPND1+DF2/IX25K7HAY5uhsul4dgEHNb1fA5PnEaxvc+mJHvQmWqxs6kIk9lwjgdJXgYsB361qh4ap9raMNTxmAUsAPqaD0F+FliT5NVT9LyR4f68XFtVjwDfTnIbnXAynA+UJpvhHI8lwPEAVfVvSZ4MHMD0/DB3WL9f+nOa1tS0L7C5SeW/ReeOxTtI8nPA7VX1MWAN8FzgGuC1SQ5q+uw/1a+ioomned/em2Re86nkr3ct/ifg9O1PmtE/aWeuB+YlOSKdq7adQud3Xrc1wKnN49cC/1xT9+ouQx6PZlrShcCrp/j5ADDE8aiq+6rqgKqa25yUfC2d4zIVgwgM7+flcmAxQJID6My+uH1cqxw/wzke3wWOA2imuz8ZuGdcq5w41gBvaq6q9SLgvqraPNRKhpGp6c+BU5NcS+eXxAMD9DkZ+EaSm4Bn0znh6Fbgj4EvJrkZuBoY8sQjaQycA1xJJyBv6mo/HXhJc9GFW+mcSCkNqjkH5AzgKmADcFlV3ZLkvUle3XRbBTw9yUbgD4BBL+862Q3zeHwIeArwd835iP3/+Joyhnk8po1hHo+rgB82v4PXAX9UVT9sp+KxNczjcRZwWpKvA73Am6fqhxlJeulM839Wkk1JliR5W5K3NV3W0gmmG4G/BH5vWNudosdLkiRJ0gTnyIgkSZKkVhhGJEmSJLXCMCJJkiSpFYYRSZIkSa0wjEiSJElqhWFE6pLk3Un+cCfLD0xyXZKvJfnlUWz/zUk+3jx+TZKjdqVeSZKkycwwIo3MccA3q+oFVfXlXdzWawDDiCRJmrYMI5r2kixPcluSfwKe1bQ9M8mVSW5M8uUkz27u9n0+cGJzI7C9k1yQ5IYktyR5T9c272juTEuShUn6+u3zxcCrgQ8123rmeL1eSZKkiWKPtguQ2pTkF4BTgBfQ+Xn4KnAjsBJ4W1V9K8kxwJ9X1UuTvAtYWFVnNOsvr6otSWYA1yR5blXdPNR+q+orzV2Nv1BVnxmjlydJkjShGUY03f0y8PmqehCgCQhPBl4M/F2S7f32GmT91ydZSudn6WA6066GDCOSJEkyjEgA1e/5k4AfVdXzd7ZSkiOAPwR+saruTfJJOkEGYBv/NQ3yyQOsLkmSNO15zoimuy8Bv96c/zEL+DXgQeDbSV4HkI7nDbDuU4EHgPuSzAZO6Fp2B/ALzePfGGTf9wOzdv0lSJIkTU6GEU1rVfVV4NPATcBnge1XyHoDsCTJ14FbgJMGWPfrwNea5auBf+1a/B7gT5N8GXh0kN1fCvxRc5lgT2CXJEnTTqr6z1CRJEmSpLHnyIgkSZKkVhhGJEmSJLXCMCJJkiSpFYYRSZIkSa0wjEiSJElqhWFEkiRJUisMI5IkSZJaYRiRJEmS1Ir/HxdQ4poAcJTfAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 936x360 with 2 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "fig, ax = plt.subplots(1,2)\n",
    "D.boxplot(column='balance',by='default', ax=ax[0]);"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Answer**:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Next, we want to fit a logistic regression model to our data.\n",
    "Use the `LogisticRegression` function in the module `sklearn.linear_model`.\n",
    "The behaviour is similar to a `LinearRegression` fit.\n",
    "\n",
    "You can find the documentation of this function [here](https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LogisticRegression.html).\n",
    "There are a lot of optional arguments, the most important might be the unimpressive looking parameter `C`, which determines the strength of regularization used in the algorithm that solves the maximum likelihood problem.\n",
    "\n",
    "We will discuss regularization later in the lecture as well as in the labs. For now, it suffices if you keep the following in mind:\n",
    "\n",
    "**The larger you choose `C`, the less the problem will be regularized.**\n",
    "\n",
    "**Task**: Fit a logistic regression model that predicts the probability of `default` using `balance` as predictor. You should obtain the following values: $\\beta_0: -10.6513$, $\\beta_\\text{balance}: 0.0055$.\n",
    "\n",
    "Choose the following optional parameters:\n",
    "* set the regularization parameter `C = 1e10` (which is the scientific notation of $C = 10^{10}$, and thus very large)\n",
    "* set the error tolerance to `tol=1e-10`\n",
    "* set the solver to `solver = 'liblinear'`\n",
    "\n",
    "in this and the upcoming problems."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Task**: Print the intercept as well as the coefficients in a nice way."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Task**:\n",
    "Predict the probability of `default` for a `balance` value of $\\$ 1.000 $ and $\\$ 2.000 $, resp.\n",
    "Use the method `predict_proba`.\n",
    "Interpret the results."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now, we want to incorporate the predictors `income` and `student` status as well. This can be done easily using the same methods."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [],
   "source": [
    "lr2= LogisticRegression(solver='liblinear',tol=1e-10,C=1e10)\n",
    "X = D.loc[:,['balance','income','student']]\n",
    "y = D.loc[:,'default']\n",
    "reg2 = lr2.fit(X,y)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Task**: Print the intercept as well as the coefficients in a nice way."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Task**:\n",
    "What is the default probability of a student and a non-student with a credit card balance of $\\$ 1500$, an income of $\\$40,000$?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Answer**:"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}