{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "0d471e53",
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import pandas as pd\n",
    "import matplotlib.pyplot as plt"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "215c0d1d",
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn import datasets\n",
    "from sklearn import model_selection\n",
    "from sklearn import preprocessing\n",
    "from sklearn import linear_model\n",
    "from sklearn import metrics"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "b69bd1bb",
   "metadata": {},
   "outputs": [],
   "source": [
    "np.random.seed(10)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1374fb45",
   "metadata": {},
   "source": [
    "# Klasifikacija"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4370dff0",
   "metadata": {},
   "source": [
    "Zadatak klasifikacije je odredjivanje modela kojim se predvidjaju vrednosti kategoricke ciljne promenljive. Kategorickim se smatraju promenljive koje uzimaju konacan broj vrednosti (diskretne promenljive) medju kojima nema uredjenja. Neki tipicni primeri zadataka klasifikacije su odredjivanje da li je poruka spam ili ne, da li je detektovani tumor benigni ili maligni, da li je obavljena transakcija regularna ili ne, i slicno. U navedenim primerima ciljna promenljiva ima samo dve moguce vrednosti, odnosno radi se o problemu **binarne klasifikacije**. Ukoliko ciljna promenljiva ima vise mogucih vrednosti (vise od dve), rec je o **viseklasnoj klasifikaciji**. Na primer, odredjivanje da li je novinski clanak iz domena politike, sporta ili kulture, identifikovanje jezika na kome je neki tekst napisan, prepoznavanje dekadne cifre koja se nalazi na slici su zadaci viseklasne klasifikacije. Bilo da je rec o binarnoj ili viseklasnoj klasifikaciji, ciljna promenljiva uzima tacno jednu vrednosti. U slucaju **viselabelarne klasifikacije** ciljna promenljiva moze uzimaati i vise vrednosti. Na primer, jedan clanak moze pripadati istovremeno kategorijama *turizam*, *Italija* i *hrana*."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "447fe414",
   "metadata": {},
   "source": [
    "<img src=\"assets/multiclass_multilabel_classification.png\" width=400>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "68cc56a8",
   "metadata": {},
   "source": [
    "# Model logisticke regresije"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "72de5207",
   "metadata": {},
   "source": [
    "**Logisticka regresija** je jedan od osnovnih modela binarne klasifikacije. Ciljna funkcija logisticke regresije je zadata formulom\n",
    "\n",
    "$$f_\\beta (X) = \\frac{1}{1 + e^{-(\\beta_0 + \\beta_1X_1 + \\cdot\\cdot\\cdot + \\beta_mX_m)}} = \\frac{1}{1 + e^{-X\\beta}}  = \\sigma (X\\beta)$$\n",
    "\n",
    "gde je $\\sigma(x)$ sigmoidna funkcija odredjena jednacinom $\\sigma(x) = \\frac{1}{1 + e^{-x}}$. Vrednost ciljne funkcije se interpretira kao verovatnoca pripadnosti odredjenoj klasi - ukoliko je njena vrednost veca od 0.5, model predvidja jednu, a u suprotnom drugu klasu. Kod binarne klasifikacije klasama se obicno pridruzuju numericka obelezja 0 i 1 ili -1 i 1. Klasu sa obelezjem 1 jos nazivamo **pozitivnom klasom**, a klasu sa obelezjem 0 (ili -1) **negativnom klasom**.\n",
    "\n",
    "**NAPOMENA**: Model logisticke regresije je linearni model. On pretpostavlja linearnu vezu izmedju atributa $X_1$, $X_2$, ..., $X_m$ i logaritma kolicnika verovatnoca pripadnosti pozitivnoj i negativnoj klasi (tzv. logiti) \n",
    "\n",
    "$$log\\frac{p(y=1 | X)}{p(y=0 | X)} = log\\frac{\\sigma(X\\beta)}{1 - \\sigma(X\\beta)} = log\\frac{\\frac{1}{1 + e^{-X\\beta}}}{\\frac{e^{-X\\beta}}{1 + e^{-X\\beta}}} = log\\frac{1}{e^{-X\\beta}} = X\\beta$$"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "118946b7",
   "metadata": {},
   "source": [
    "## Sigmoidna funkcija"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "81947848",
   "metadata": {},
   "source": [
    "Sigmoidna funkcija $\\sigma(x)$ je zapravo neprekidna aproksimacija indikatorske funkcije $I(x\\geq0)$. To znaci da je sigmoidnom funkcijom $\\sigma(x)$ definisan model koji vrsi klasifikaciju tacaka u odnosu na pravu $x = 0$. Analogno, model odredjen sa $\\sigma(\\beta_0 + \\beta_1X_1 + \\cdot\\cdot\\cdot + \\beta_mX_m)$ vrsi klasifikaciju tacaka u odnosu na hiperravan $\\beta_0 + \\beta_1X_1 + \\cdot\\cdot\\cdot + \\beta_mX_m = 0$.\n",
    "\n",
    "<img src=\"assets/sigmoid_class_bound.png\" width=700>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d69a0d53",
   "metadata": {},
   "source": [
    "Posmatrajmo model odredjen sa $f_{\\beta_0, \\beta_1}(x) = \\sigma(\\beta_0 + \\beta_1x)$. Podesavanjem parametra $\\beta_0$ kontrolise se gde ce biti granica izmedju klasa ($x = -\\frac{\\beta_0}{\\beta_1}$), dok se podesavanjem parametra $\\beta_1$ kontrolise strmost nagiba sigmoidne funkcije u okolini tacke $-\\frac{\\beta_0}{\\beta_1}$, odnosno sigurnost modela u svoja predvidjanja. Sto je vrednost parametra $\\beta_1$ veca, to je sigmoidna funkcija bliza indikatorskoj funkciji. U granicnom slucaju (kada $\\beta_1 \\rightarrow +\\infty$), to bi znacilo da je model sasvim siguran da su sve instance za koje je $x < -\\frac{\\beta_0}{\\beta_1}$ instance negativne klase, a sve instance za koje je $x > -\\frac{\\beta_0}{\\beta_1}$ instance pozitivne klase.\n",
    "\n",
    "<img src=\"assets/sigmoid_parameter_variation.png\" width=750>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d374b1c7",
   "metadata": {},
   "source": [
    "## Unakrsna entropija"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "537b0a78",
   "metadata": {},
   "source": [
    "Koeficijenti modela se odredjuju minimizacijom funkcije greske. Funkcija greske koja se minimizuje kod logisticke regresije je **unakrsna entropija** (eng. binary crossentropy loss)\n",
    "\n",
    "$$ -\\sum_{i=1}^{N} (y_i logf_\\beta(x_i) + (1-y_i)log(1-f_\\beta(x_i)))$$\n",
    "\n",
    "gde je $f_\\beta(x_i)$ vrednost ciljne funkcije, odnosno predvidjanje modela, $(x_i, y_i)$ instance skupa za treniranje, a $N$ ukupan broj instanci ovog skupa. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "48bcd261",
   "metadata": {},
   "outputs": [],
   "source": [
    "N = 50\n",
    "y_predicted = np.linspace(0, 1, N)[1:]\n",
    "\n",
    "loss_part_positive = -np.log(y_predicted)\n",
    "loss_part_negative = -np.log(1 - y_predicted)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "24e6e222",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 23724 characters]\n",
      "text/plain": [
       "<Figure size 720x360 with 2 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.figure(figsize=(10, 5))\n",
    "\n",
    "plt.subplot(121)\n",
    "plt.plot(y_predicted, loss_part_positive, color='blue')\n",
    "plt.title('Doprinos funkciji greske pozitivnih instanci')\n",
    "plt.xlabel(r'$f_\\beta(x)$', fontsize=12)\n",
    "plt.ylabel(r'$L(y, f_\\beta(x))$')\n",
    "\n",
    "plt.subplot(122)\n",
    "plt.plot(y_predicted, loss_part_negative, color='red')\n",
    "plt.title('Doprinos funkciji greske negativnih instanci')\n",
    "plt.xlabel(r'$f_\\beta(x)$', fontsize=12)\n",
    "plt.ylabel(r'$L(y, f_\\beta(x))$')\n",
    "\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5751a7d8",
   "metadata": {},
   "source": [
    "Ako je za $(x_i, y_i)$ instanca pozitivne klase, sto vise predikcija $f_\\beta(x_i)$ odstupa od 1, veci je doprinos funkciji greske (grafik levo). U granicnom slucaju za $f_\\beta(x_i) = 0$ dobija se beskonacna greska. Analogno, kod instanci negativne klase, sto vise model odstupa od 0, veci je doprinos funkciji greske (grafik desno). U granicnom slucaju za $f_\\beta(x_i) = 1$ dobija se beskonacna greska."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "595f4fe3",
   "metadata": {},
   "source": [
    "# Primer"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0ac99138",
   "metadata": {},
   "source": [
    "Potrebno je napraviti model kojim se vrsi klasifikacija tumora na maligne i benigne koristeci poznati Viskonsins skup podataka koji je dostupan u scikit-learn kolekciji podataka. Kolekcija podataka je dostupna kroz paket <code>datasets</code>, a podatke Viskonsins skupa mozemo ucitati koristeci <code>load_breast_cancer()</code> funkciju ovog paketa."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "00d425b8",
   "metadata": {},
   "outputs": [],
   "source": [
    "data = datasets.load_breast_cancer()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e6db41fc",
   "metadata": {},
   "source": [
    "## Analiza podataka"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fa2eed7a",
   "metadata": {},
   "source": [
    "Vise informacija o ucitanom skupu podataka:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "d7528f6f",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      ".. _breast_cancer_dataset:\n",
      "\n",
      "Breast cancer wisconsin (diagnostic) dataset\n",
      "--------------------------------------------\n",
      "\n",
      "**Data Set Characteristics:**\n",
      "\n",
      "    :Number of Instances: 569\n",
      "\n",
      "    :Number of Attributes: 30 numeric, predictive attributes and the class\n",
      "\n",
      "    :Attribute Information:\n",
      "        - radius (mean of distances from center to points on the perimeter)\n",
      "        - texture (standard deviation of gray-scale values)\n",
      "        - perimeter\n",
      "        - area\n",
      "        - smoothness (local variation in radius lengths)\n",
      "        - compactness (perimeter^2 / area - 1.0)\n",
      "        - concavity (severity of concave portions of the contour)\n",
      "        - concave points (number of concave portions of the contour)\n",
      "        - symmetry\n",
      "        - fractal dimension (\"coastline approximation\" - 1)\n",
      "\n",
      "        The mean, standard error, and \"worst\" or largest (mean of the three\n",
      "        worst/largest values) of these features were computed for each image,\n",
      "        resulting in 30 features.  For instance, field 0 is Mean Radius, field\n",
      "        10 is Radius SE, field 20 is Worst Radius.\n",
      "\n",
      "        - class:\n",
      "                - WDBC-Malignant\n",
      "                - WDBC-Benign\n",
      "\n",
      "    :Summary Statistics:\n",
      "\n",
      "    ===================================== ====== ======\n",
      "                                           Min    Max\n",
      "    ===================================== ====== ======\n",
      "    radius (mean):                        6.981  28.11\n",
      "    texture (mean):                       9.71   39.28\n",
      "    perimeter (mean):                     43.79  188.5\n",
      "    area (mean):                          143.5  2501.0\n",
      "    smoothness (mean):                    0.053  0.163\n",
      "    compactness (mean):                   0.019  0.345\n",
      "    concavity (mean):                     0.0    0.427\n",
      "    concave points (mean):                0.0    0.201\n",
      "    symmetry (mean):                      0.106  0.304\n",
      "    fractal dimension (mean):             0.05   0.097\n",
      "    radius (standard error):              0.112  2.873\n",
      "    texture (standard error):             0.36   4.885\n",
      "    perimeter (standard error):           0.757  21.98\n",
      "    area (standard error):                6.802  542.2\n",
      "    smoothness (standard error):          0.002  0.031\n",
      "    compactness (standard error):         0.002  0.135\n",
      "    concavity (standard error):           0.0    0.396\n",
      "    concave points (standard error):      0.0    0.053\n",
      "    symmetry (standard error):            0.008  0.079\n",
      "    fractal dimension (standard error):   0.001  0.03\n",
      "    radius (worst):                       7.93   36.04\n",
      "    texture (worst):                      12.02  49.54\n",
      "    perimeter (worst):                    50.41  251.2\n",
      "    area (worst):                         185.2  4254.0\n",
      "    smoothness (worst):                   0.071  0.223\n",
      "    compactness (worst):                  0.027  1.058\n",
      "    concavity (worst):                    0.0    1.252\n",
      "    concave points (worst):               0.0    0.291\n",
      "    symmetry (worst):                     0.156  0.664\n",
      "    fractal dimension (worst):            0.055  0.208\n",
      "    ===================================== ====== ======\n",
      "\n",
      "    :Missing Attribute Values: None\n",
      "\n",
      "    :Class Distribution: 212 - Malignant, 357 - Benign\n",
      "\n",
      "    :Creator:  Dr. William H. Wolberg, W. Nick Street, Olvi L. Mangasarian\n",
      "\n",
      "    :Donor: Nick Street\n",
      "\n",
      "    :Date: November, 1995\n",
      "\n",
      "This is a copy of UCI ML Breast Cancer Wisconsin (Diagnostic) datasets.\n",
      "https://goo.gl/U2Uwz2\n",
      "\n",
      "Features are computed from a digitized image of a fine needle\n",
      "aspirate (FNA) of a breast mass.  They describe\n",
      "characteristics of the cell nuclei present in the image.\n",
      "\n",
      "Separating plane described above was obtained using\n",
      "Multisurface Method-Tree (MSM-T) [K. P. Bennett, \"Decision Tree\n",
      "Construction Via Linear Programming.\" Proceedings of the 4th\n",
      "Midwest Artificial Intelligence and Cognitive Science Society,\n",
      "pp. 97-101, 1992], a classification method which uses linear\n",
      "programming to construct a decision tree.  Relevant features\n",
      "were selected using an exhaustive search in the space of 1-4\n",
      "features and 1-3 separating planes.\n",
      "\n",
      "The actual linear program used to obtain the separating plane\n",
      "in the 3-dimensional space is that described in:\n",
      "[K. P. Bennett and O. L. Mangasarian: \"Robust Linear\n",
      "Programming Discrimination of Two Linearly Inseparable Sets\",\n",
      "Optimization Methods and Software 1, 1992, 23-34].\n",
      "\n",
      "This database is also available through the UW CS ftp server:\n",
      "\n",
      "ftp ftp.cs.wisc.edu\n",
      "cd math-prog/cpo-dataset/machine-learn/WDBC/\n",
      "\n",
      ".. topic:: References\n",
      "\n",
      "   - W.N. Street, W.H. Wolberg and O.L. Mangasarian. Nuclear feature extraction \n",
      "     for breast tumor diagnosis. IS&T/SPIE 1993 International Symposium on \n",
      "     Electronic Imaging: Science and Technology, volume 1905, pages 861-870,\n",
      "     San Jose, CA, 1993.\n",
      "   - O.L. Mangasarian, W.N. Street and W.H. Wolberg. Breast cancer diagnosis and \n",
      "     prognosis via linear programming. Operations Research, 43(4), pages 570-577, \n",
      "     July-August 1995.\n",
      "   - W.H. Wolberg, W.N. Street, and O.L. Mangasarian. Machine learning techniques\n",
      "     to diagnose breast cancer from fine-needle aspirates. Cancer Letters 77 (1994) \n",
      "     163-171.\n"
     ]
    }
   ],
   "source": [
    "print(data.DESCR)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8509dbe7",
   "metadata": {},
   "source": [
    "Iz opisa vidimo da skup podataka raspolaze sa 30 razlicitih atributa koji su numerickog tipa. Njihove nazive mozemo izdvojiti pomocu atributa <code>feature_names</code>."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "2967f853",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array(['mean radius', 'mean texture', 'mean perimeter', 'mean area',\n",
       "       'mean smoothness', 'mean compactness', 'mean concavity',\n",
       "       'mean concave points', 'mean symmetry', 'mean fractal dimension',\n",
       "       'radius error', 'texture error', 'perimeter error', 'area error',\n",
       "       'smoothness error', 'compactness error', 'concavity error',\n",
       "       'concave points error', 'symmetry error',\n",
       "       'fractal dimension error', 'worst radius', 'worst texture',\n",
       "       'worst perimeter', 'worst area', 'worst smoothness',\n",
       "       'worst compactness', 'worst concavity', 'worst concave points',\n",
       "       'worst symmetry', 'worst fractal dimension'], dtype='<U23')"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "feature_names = data.feature_names\n",
    "feature_names"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9bb019d5",
   "metadata": {},
   "source": [
    "Vrednosti ciljne promenljive su 0 i 1. Nazive klasa cija su to numericka obelezja (redom) mozemo izdvojiti pomocu atributa <code>target_names</code>."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "d5c85c92",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array(['malignant', 'benign'], dtype='<U9')"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "target_names = data.target_names\n",
    "target_names"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "703bfd31",
   "metadata": {},
   "source": [
    "Zarad lakse manipulacije podacima, prelazimo na <code>DataFrame</code> strukture podataka."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "3419e26c",
   "metadata": {},
   "outputs": [],
   "source": [
    "x = pd.DataFrame(data.data, columns=data.feature_names)\n",
    "y = pd.DataFrame(data.target)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "90b582a1",
   "metadata": {},
   "source": [
    "Osnovne informacije o atributima, njihove tipove i uvid u prisustvo nedostajacih vrednosti:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "57bd11cf",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.core.frame.DataFrame'>\n",
      "RangeIndex: 569 entries, 0 to 568\n",
      "Data columns (total 30 columns):\n",
      " #   Column                   Non-Null Count  Dtype  \n",
      "---  ------                   --------------  -----  \n",
      " 0   mean radius              569 non-null    float64\n",
      " 1   mean texture             569 non-null    float64\n",
      " 2   mean perimeter           569 non-null    float64\n",
      " 3   mean area                569 non-null    float64\n",
      " 4   mean smoothness          569 non-null    float64\n",
      " 5   mean compactness         569 non-null    float64\n",
      " 6   mean concavity           569 non-null    float64\n",
      " 7   mean concave points      569 non-null    float64\n",
      " 8   mean symmetry            569 non-null    float64\n",
      " 9   mean fractal dimension   569 non-null    float64\n",
      " 10  radius error             569 non-null    float64\n",
      " 11  texture error            569 non-null    float64\n",
      " 12  perimeter error          569 non-null    float64\n",
      " 13  area error               569 non-null    float64\n",
      " 14  smoothness error         569 non-null    float64\n",
      " 15  compactness error        569 non-null    float64\n",
      " 16  concavity error          569 non-null    float64\n",
      " 17  concave points error     569 non-null    float64\n",
      " 18  symmetry error           569 non-null    float64\n",
      " 19  fractal dimension error  569 non-null    float64\n",
      " 20  worst radius             569 non-null    float64\n",
      " 21  worst texture            569 non-null    float64\n",
      " 22  worst perimeter          569 non-null    float64\n",
      " 23  worst area               569 non-null    float64\n",
      " 24  worst smoothness         569 non-null    float64\n",
      " 25  worst compactness        569 non-null    float64\n",
      " 26  worst concavity          569 non-null    float64\n",
      " 27  worst concave points     569 non-null    float64\n",
      " 28  worst symmetry           569 non-null    float64\n",
      " 29  worst fractal dimension  569 non-null    float64\n",
      "dtypes: float64(30)\n",
      "memory usage: 133.5 KB\n"
     ]
    }
   ],
   "source": [
    "x.info()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4b48877f",
   "metadata": {},
   "source": [
    "Elementarne statistike atributa:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "0ff6d67a",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>mean radius</th>\n",
       "      <th>mean texture</th>\n",
       "      <th>mean perimeter</th>\n",
       "      <th>mean area</th>\n",
       "      <th>mean smoothness</th>\n",
       "      <th>mean compactness</th>\n",
       "      <th>mean concavity</th>\n",
       "      <th>mean concave points</th>\n",
       "      <th>mean symmetry</th>\n",
       "      <th>mean fractal dimension</th>\n",
       "      <th>...</th>\n",
       "      <th>worst radius</th>\n",
       "      <th>worst texture</th>\n",
       "      <th>worst perimeter</th>\n",
       "      <th>worst area</th>\n",
       "      <th>worst smoothness</th>\n",
       "      <th>worst compactness</th>\n",
       "      <th>worst concavity</th>\n",
       "      <th>worst concave points</th>\n",
       "      <th>worst symmetry</th>\n",
       "      <th>worst fractal dimension</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>count</th>\n",
       "      <td>569.000000</td>\n",
       "      <td>569.000000</td>\n",
       "      <td>569.000000</td>\n",
       "      <td>569.000000</td>\n",
       "      <td>569.000000</td>\n",
       "      <td>569.000000</td>\n",
       "      <td>569.000000</td>\n",
       "      <td>569.000000</td>\n",
       "      <td>569.000000</td>\n",
       "      <td>569.000000</td>\n",
       "      <td>...</td>\n",
       "      <td>569.000000</td>\n",
       "      <td>569.000000</td>\n",
       "      <td>569.000000</td>\n",
       "      <td>569.000000</td>\n",
       "      <td>569.000000</td>\n",
       "      <td>569.000000</td>\n",
       "      <td>569.000000</td>\n",
       "      <td>569.000000</td>\n",
       "      <td>569.000000</td>\n",
       "      <td>569.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>14.127292</td>\n",
       "      <td>19.289649</td>\n",
       "      <td>91.969033</td>\n",
       "      <td>654.889104</td>\n",
       "      <td>0.096360</td>\n",
       "      <td>0.104341</td>\n",
       "      <td>0.088799</td>\n",
       "      <td>0.048919</td>\n",
       "      <td>0.181162</td>\n",
       "      <td>0.062798</td>\n",
       "      <td>...</td>\n",
       "      <td>16.269190</td>\n",
       "      <td>25.677223</td>\n",
       "      <td>107.261213</td>\n",
       "      <td>880.583128</td>\n",
       "      <td>0.132369</td>\n",
       "      <td>0.254265</td>\n",
       "      <td>0.272188</td>\n",
       "      <td>0.114606</td>\n",
       "      <td>0.290076</td>\n",
       "      <td>0.083946</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>3.524049</td>\n",
       "      <td>4.301036</td>\n",
       "      <td>24.298981</td>\n",
       "      <td>351.914129</td>\n",
       "      <td>0.014064</td>\n",
       "      <td>0.052813</td>\n",
       "      <td>0.079720</td>\n",
       "      <td>0.038803</td>\n",
       "      <td>0.027414</td>\n",
       "      <td>0.007060</td>\n",
       "      <td>...</td>\n",
       "      <td>4.833242</td>\n",
       "      <td>6.146258</td>\n",
       "      <td>33.602542</td>\n",
       "      <td>569.356993</td>\n",
       "      <td>0.022832</td>\n",
       "      <td>0.157336</td>\n",
       "      <td>0.208624</td>\n",
       "      <td>0.065732</td>\n",
       "      <td>0.061867</td>\n",
       "      <td>0.018061</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>6.981000</td>\n",
       "      <td>9.710000</td>\n",
       "      <td>43.790000</td>\n",
       "      <td>143.500000</td>\n",
       "      <td>0.052630</td>\n",
       "      <td>0.019380</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.106000</td>\n",
       "      <td>0.049960</td>\n",
       "      <td>...</td>\n",
       "      <td>7.930000</td>\n",
       "      <td>12.020000</td>\n",
       "      <td>50.410000</td>\n",
       "      <td>185.200000</td>\n",
       "      <td>0.071170</td>\n",
       "      <td>0.027290</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.156500</td>\n",
       "      <td>0.055040</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>11.700000</td>\n",
       "      <td>16.170000</td>\n",
       "      <td>75.170000</td>\n",
       "      <td>420.300000</td>\n",
       "      <td>0.086370</td>\n",
       "      <td>0.064920</td>\n",
       "      <td>0.029560</td>\n",
       "      <td>0.020310</td>\n",
       "      <td>0.161900</td>\n",
       "      <td>0.057700</td>\n",
       "      <td>...</td>\n",
       "      <td>13.010000</td>\n",
       "      <td>21.080000</td>\n",
       "      <td>84.110000</td>\n",
       "      <td>515.300000</td>\n",
       "      <td>0.116600</td>\n",
       "      <td>0.147200</td>\n",
       "      <td>0.114500</td>\n",
       "      <td>0.064930</td>\n",
       "      <td>0.250400</td>\n",
       "      <td>0.071460</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>13.370000</td>\n",
       "      <td>18.840000</td>\n",
       "      <td>86.240000</td>\n",
       "      <td>551.100000</td>\n",
       "      <td>0.095870</td>\n",
       "      <td>0.092630</td>\n",
       "      <td>0.061540</td>\n",
       "      <td>0.033500</td>\n",
       "      <td>0.179200</td>\n",
       "      <td>0.061540</td>\n",
       "      <td>...</td>\n",
       "      <td>14.970000</td>\n",
       "      <td>25.410000</td>\n",
       "      <td>97.660000</td>\n",
       "      <td>686.500000</td>\n",
       "      <td>0.131300</td>\n",
       "      <td>0.211900</td>\n",
       "      <td>0.226700</td>\n",
       "      <td>0.099930</td>\n",
       "      <td>0.282200</td>\n",
       "      <td>0.080040</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>15.780000</td>\n",
       "      <td>21.800000</td>\n",
       "      <td>104.100000</td>\n",
       "      <td>782.700000</td>\n",
       "      <td>0.105300</td>\n",
       "      <td>0.130400</td>\n",
       "      <td>0.130700</td>\n",
       "      <td>0.074000</td>\n",
       "      <td>0.195700</td>\n",
       "      <td>0.066120</td>\n",
       "      <td>...</td>\n",
       "      <td>18.790000</td>\n",
       "      <td>29.720000</td>\n",
       "      <td>125.400000</td>\n",
       "      <td>1084.000000</td>\n",
       "      <td>0.146000</td>\n",
       "      <td>0.339100</td>\n",
       "      <td>0.382900</td>\n",
       "      <td>0.161400</td>\n",
       "      <td>0.317900</td>\n",
       "      <td>0.092080</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>28.110000</td>\n",
       "      <td>39.280000</td>\n",
       "      <td>188.500000</td>\n",
       "      <td>2501.000000</td>\n",
       "      <td>0.163400</td>\n",
       "      <td>0.345400</td>\n",
       "      <td>0.426800</td>\n",
       "      <td>0.201200</td>\n",
       "      <td>0.304000</td>\n",
       "      <td>0.097440</td>\n",
       "      <td>...</td>\n",
       "      <td>36.040000</td>\n",
       "      <td>49.540000</td>\n",
       "      <td>251.200000</td>\n",
       "      <td>4254.000000</td>\n",
       "      <td>0.222600</td>\n",
       "      <td>1.058000</td>\n",
       "      <td>1.252000</td>\n",
       "      <td>0.291000</td>\n",
       "      <td>0.663800</td>\n",
       "      <td>0.207500</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>8 rows × 30 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "       mean radius  mean texture  mean perimeter    mean area  \\\n",
       "count   569.000000    569.000000      569.000000   569.000000   \n",
       "mean     14.127292     19.289649       91.969033   654.889104   \n",
       "std       3.524049      4.301036       24.298981   351.914129   \n",
       "min       6.981000      9.710000       43.790000   143.500000   \n",
       "25%      11.700000     16.170000       75.170000   420.300000   \n",
       "50%      13.370000     18.840000       86.240000   551.100000   \n",
       "75%      15.780000     21.800000      104.100000   782.700000   \n",
       "max      28.110000     39.280000      188.500000  2501.000000   \n",
       "\n",
       "       mean smoothness  mean compactness  mean concavity  mean concave points  \\\n",
       "count       569.000000        569.000000      569.000000           569.000000   \n",
       "mean          0.096360          0.104341        0.088799             0.048919   \n",
       "std           0.014064          0.052813        0.079720             0.038803   \n",
       "min           0.052630          0.019380        0.000000             0.000000   \n",
       "25%           0.086370          0.064920        0.029560             0.020310   \n",
       "50%           0.095870          0.092630        0.061540             0.033500   \n",
       "75%           0.105300          0.130400        0.130700             0.074000   \n",
       "max           0.163400          0.345400        0.426800             0.201200   \n",
       "\n",
       "       mean symmetry  mean fractal dimension  ...  worst radius  \\\n",
       "count     569.000000              569.000000  ...    569.000000   \n",
       "mean        0.181162                0.062798  ...     16.269190   \n",
       "std         0.027414                0.007060  ...      4.833242   \n",
       "min         0.106000                0.049960  ...      7.930000   \n",
       "25%         0.161900                0.057700  ...     13.010000   \n",
       "50%         0.179200                0.061540  ...     14.970000   \n",
       "75%         0.195700                0.066120  ...     18.790000   \n",
       "max         0.304000                0.097440  ...     36.040000   \n",
       "\n",
       "       worst texture  worst perimeter   worst area  worst smoothness  \\\n",
       "count     569.000000       569.000000   569.000000        569.000000   \n",
       "mean       25.677223       107.261213   880.583128          0.132369   \n",
       "std         6.146258        33.602542   569.356993          0.022832   \n",
       "min        12.020000        50.410000   185.200000          0.071170   \n",
       "25%        21.080000        84.110000   515.300000          0.116600   \n",
       "50%        25.410000        97.660000   686.500000          0.131300   \n",
       "75%        29.720000       125.400000  1084.000000          0.146000   \n",
       "max        49.540000       251.200000  4254.000000          0.222600   \n",
       "\n",
       "       worst compactness  worst concavity  worst concave points  \\\n",
       "count         569.000000       569.000000            569.000000   \n",
       "mean            0.254265         0.272188              0.114606   \n",
       "std             0.157336         0.208624              0.065732   \n",
       "min             0.027290         0.000000              0.000000   \n",
       "25%             0.147200         0.114500              0.064930   \n",
       "50%             0.211900         0.226700              0.099930   \n",
       "75%             0.339100         0.382900              0.161400   \n",
       "max             1.058000         1.252000              0.291000   \n",
       "\n",
       "       worst symmetry  worst fractal dimension  \n",
       "count      569.000000               569.000000  \n",
       "mean         0.290076                 0.083946  \n",
       "std          0.061867                 0.018061  \n",
       "min          0.156500                 0.055040  \n",
       "25%          0.250400                 0.071460  \n",
       "50%          0.282200                 0.080040  \n",
       "75%          0.317900                 0.092080  \n",
       "max          0.663800                 0.207500  \n",
       "\n",
       "[8 rows x 30 columns]"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x.describe()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "11cf9162",
   "metadata": {},
   "source": [
    "Graficki prikaz udela malignih i belingnih instanci u skupu podataka:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "471a56fd",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 5940 characters]\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.hist(y)\n",
    "plt.xticks([0, 1], labels=target_names)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "226f27eb",
   "metadata": {},
   "source": [
    "Na osnovu izvrsene analize podataka mozemo zakljuciti sledece:\n",
    "- klase nisu balansirane - treba voditi racuna o tome prilikom podele skupa podataka na skupove za trening i test\n",
    "- atributi uzimaju vrednosti na razlicitim sklalama - treba ih standardizovati"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a2f30ae6",
   "metadata": {},
   "source": [
    "## Podela skupa podataka na skupove za treniranje i testiranje"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d1ed4e56",
   "metadata": {},
   "source": [
    "Prilikom podele skupa podataka kod problema klasifikacije potrebno je voditi racuna da dobijeni podskupovi imaju priblizno isti udeo instanci svake od klasa kao polazni skup. Ovo je posebno vazno kada klase nisu balansirane. \n",
    "\n",
    "<img src=\"assets/stratified_train_test_split.png\" width=550 align=\"left\">"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "37f30843",
   "metadata": {},
   "source": [
    "**Stratifikacija** je tehnika za podelu podataka kojom se cuva distribucija klasa. Stratifikovanu podelu skupa podataka na skupove za treniranje i testiranje postizemo zadavanjem parametra <code>stratify</code> prilikom poziva funkcije <code>train_test_split()</code>. Parametar <code>stratify</code> ocekuje listu labela (obelezja klase) pridruzenih instancama koje ce koristiti prilikom primene stratifikacije."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2cfdf01a",
   "metadata": {},
   "source": [
    "Podatke delimo na skup za treniranje i skup za testiranje u razmeri 2:1. Vrednost parametra <code>stratify</code> postavljamo na niz vrednosti ciljne promenljive (sto su upravo labele klasa 0 i 1), a <code>random_state</code> parametar na vrednost 7."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "b443082e",
   "metadata": {},
   "outputs": [],
   "source": [
    "x_train, x_test, y_train, y_test = model_selection.train_test_split(x, y, test_size=0.33, stratify=y, random_state=7)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "id": "2980ebbd",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(381, 30)"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x_train.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "id": "13dbb918",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(188, 30)"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x_test.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8a7f91cd",
   "metadata": {},
   "source": [
    "Provera stratifikovanosti podele:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "id": "8735c0c9",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "1    239\n",
       "0    142\n",
       "dtype: int64"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y_train.value_counts()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "id": "81ea1ba0",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "1    118\n",
       "0     70\n",
       "dtype: int64"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y_test.value_counts()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "id": "3b3b4ae3",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 11008 characters]\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.hist([y_train.values.ravel(), y_test.values.ravel()], label=['trening skup', 'test skup'])\n",
    "plt.title('Vizuelna provera stratifikovanosti podele')\n",
    "plt.xticks([0, 1], labels=target_names)\n",
    "plt.legend(loc='best')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ea1d1cf1",
   "metadata": {},
   "source": [
    "## Standardizacija podataka"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "16ddabff",
   "metadata": {},
   "source": [
    "Vrsimo standardizaciju atributa."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "id": "da0091bd",
   "metadata": {},
   "outputs": [],
   "source": [
    "scaler = preprocessing.StandardScaler()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "id": "38bd9e35",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "StandardScaler()"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "scaler.fit(x_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "id": "09015da7",
   "metadata": {},
   "outputs": [],
   "source": [
    "x_train = scaler.transform(x_train)\n",
    "x_test = scaler.transform(x_test)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8ee00b3e",
   "metadata": {},
   "source": [
    "## Ucenje modela"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5658f843",
   "metadata": {},
   "source": [
    "Funkcija <code>LogisticRegression()</code> se koristi za konstruisanje modela logisticke regresije. Podrazumevano je i slobodni clan $\\beta_0$ pridruzen modelu. Ukoliko je potrebno izostaviti ga, to se postize postavljanjem parametra <code>fit_intercept</code> na <code>False</code> prilikom konstruisanja modela."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "id": "a1a71b51",
   "metadata": {},
   "outputs": [],
   "source": [
    "model = linear_model.LogisticRegression()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "599923a9",
   "metadata": {},
   "source": [
    "Metodom <code>fit()</code> se vrsi treniranje modela. **NAPOMENA**: kod modela logisticke regresije metod <code>fit()</code> ocekuje da vrednosti ciljne promenljive budu prosledjene iskljucivo u formi jednodimenzionog niza, dok je model linearne regresije prihvatao i dvodimenzione objekte dimenzija <code>(n, 1)</code>!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "id": "0490e047",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "LogisticRegression()"
      ]
     },
     "execution_count": 24,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model.fit(x_train, y_train.values.ravel())"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "52bab5d7",
   "metadata": {},
   "source": [
    "Nauceni koeficijenti modela mogu se ocitati preko atributa modela <code>intercept_</code> i <code>coef_</code>."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "id": "08fdaef4",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([0.33070776])"
      ]
     },
     "execution_count": 25,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model.intercept_"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "id": "89fed9d7",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[-0.28488259, -0.5036704 , -0.30845374, -0.43124456, -0.0885936 ,\n",
       "         0.46530026, -0.63431362, -0.5799997 , -0.19622748,  0.29418451,\n",
       "        -1.0734335 ,  0.17985058, -0.551678  , -0.88684956, -0.06873348,\n",
       "         0.79862352, -0.1564331 , -0.41965439,  0.27938466,  0.45749103,\n",
       "        -0.95434897, -1.14556574, -0.75325776, -0.975012  , -0.58396992,\n",
       "         0.08148169, -0.72493983, -0.89869581, -0.70564949, -0.64264675]])"
      ]
     },
     "execution_count": 26,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model.coef_"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e5468032",
   "metadata": {},
   "source": [
    "Zarad lakse analize i interpretacije vrednosti koeficijenta modela prikazacemo ih pomocu stubicastog grafikona."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "id": "e479d29a",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 36152 characters]\n",
      "text/plain": [
       "<Figure size 720x360 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.figure(figsize=(10, 5))\n",
    "plt.bar(feature_names, model.coef_[0])\n",
    "plt.xticks(rotation='vertical')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d7e291bf",
   "metadata": {},
   "source": [
    "Mozemo primetiti da su atributi kao sto su 'radius error', 'worst texture' i 'worst area' kljucni u predvidjanju negativne klase (maligni tumor), dok atribut 'compactness error' najvise doprinosi predvidjanju pozitivne klase (benigni tumor)."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "eb46e8eb",
   "metadata": {},
   "source": [
    "## Evaluacija"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f18c49da",
   "metadata": {},
   "source": [
    "Za evaluaciju klasifikacionih modela najcesce se koriste **tacnost** (eng. accuracy), **preciznost** (eng. precision), **odziv** (eng. recall) i **F1 mera**. Ove mere ce zbog jednostavnosti biti definisane u terminima binarne klasifikacije. Za slucaj viseklasne klasifikacije ove mere se izracunavaju za svaku od klasa pojedinacno po principu *one-vs-rest*. \n",
    "\n",
    "Sve cetiri mere kvaliteta modela se zasnivaju na **matrici konfuzije** (eng. confusion matrix ) u kojoj se sumiraju rezultati testiranja klasifikatora. To je matrica $C$ ciji element $c_{ij}$ predstavlja broj instanci klase $i$ koje su klasifikovane u klasu $j$. Elementi na glavnoj dijagonali predstavljaju broj instanci koje su ispravno klasifikovane, dok nedijagonalni elementi oznacavaju greske. To znaci da je klasifikacija najbolja kada je matrica konfuzije dijagonalna. U slucaju binarne klasifikacije matrica konfuzije ima karakteristican oblik:\n",
    "\n",
    "<img src=\"assets/confusion_matrix.png\" width=300>\n",
    "\n",
    "*Stvarno pozitivne* (eng. true positive) instance su instance iz pozitivne klase za koje je predvidJena klasa takode pozitivna. Broj takvih instanci oznacen je sa **TP**. *Stvarno negativne* (eng. true negative) instance su negativne instance za koje model predvidJa negativnu klasu. Broj takvih instanci oznacen je sa **TN**. *Lazno pozitivne* (eng. false positive) instance su instance negativne klase a koje je model prepoznao kao pozitivne. Broj takvih instanci oznacen je sa **FP**. *Lazno negativne* (eng. false negative) instance su pozitivne instance za koje model predvida negativnu klasu. Broj takvih instanci je oznacen sa **FN**.\n",
    "\n",
    "**Tacnost** predstavlja ocenu ukupnog broja uspesno klasifikovanih instanci i izracunava se po sledecoj formuli:\n",
    "\n",
    "$$Acc = \\frac{TP + TN}{TP + FP + TN + FN}$$\n",
    "\n",
    "Ova mera je obicno prvi izbor kod problema klasifikacije, medutim ona ne daje uvek dobru ocenu modela. U slucaju nebalansiranih klasa model koji uvek (ili skoro uvek) predvidja vecinsku klasu imace visoku tacnost na datom skupu podataka, a da pritom nije naucio nista iz podataka (osim koja je klasa vecinska). Takav model ima izuzetno losu sposobnost predvidjanja i iz tog razloga se pri radu sa nebalansiranim skupom podataka ova mera nikada ne koristi ili se koristi zajedno sa jos nekom merom koja nije osetljiva na nebalansiranost klasa.\n",
    "\n",
    "**Preciznost** ocenjuje koliko je pozitivnih instanci ispravno klasifikovano u odnosu na ukupan broj instanci koje model klasifikuje kao pozitivne. Sa druge strane, **odziv** predstavlja udeo ispravno klasifikovanih pozitivnih instanci u odnosu na ukupan broj pozitivih instanci u skupu. Ove dve mere definisane su sledecim formulama:\n",
    "\n",
    "$$Prec = \\frac{TP}{TP + FP} \\hspace{1cm} Rec = \\frac{TP}{TP + FN}$$\n",
    "\n",
    "Preciznost i odziv pojedinacno posmatrane nisu korisne mere:\n",
    "- visoka preciznost a nizak odziv - model retko klasifikuje instance kao pozitivne, i samim tim jos manje pravi gresku da neku instancu koja je negativna proglasi za pozitivnu\n",
    "- visok odziv a niska preciznost - model cesto klasifikuje instance kao pozitivne, i time retko za neku instancu koja je pozitivna predvida negativnu klasu\n",
    "\n",
    "Iz navedenih razloga se ove dve mere uvek posmatraju zajedno. Nacin na koji se to najcesce radi je tako sto se izracuna **F1 mera** - njihova harmonijska sredina:\n",
    "\n",
    "$$F1 = 2\\frac{Prec \\cdot Rec}{Prec + Rec}$$\n",
    "\n",
    "**NAPOMENA**: preciznost, odziv i F1 mera su definisane u odnosu na pozitivnu klasu, odnosno da nisu simetricne u odnosu na izbor neke klase kao pozitivne!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "id": "88790712",
   "metadata": {},
   "outputs": [],
   "source": [
    "y_test_predicted = model.predict(x_test)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2b0789f4",
   "metadata": {},
   "source": [
    "Matrica konfuzije se moze izracunati funkcijom <code>confusion_matrix()</code> paketa <code>metrics</code>."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "id": "b55e70bb",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[ 69,   1],\n",
       "       [  3, 115]])"
      ]
     },
     "execution_count": 29,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "metrics.confusion_matrix(y_test, y_test_predicted)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "dcd36d6a",
   "metadata": {},
   "source": [
    "Za detaljnu analizu prostora gresaka cesto je potrebno prouciti instance na kojima model vrsi pogresu klasifikaciju (FP i FN)."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "693458d4",
   "metadata": {},
   "source": [
    "Izdvajanje lazno pozitivnih i lazno negativnih instanci:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "id": "3c8e0e4f",
   "metadata": {},
   "outputs": [],
   "source": [
    "FP_mask = (y_test.values.ravel() == 0) & (y_test_predicted == 1)\n",
    "FN_mask = (y_test.values.ravel() == 1) & (y_test_predicted == 0)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cf812e6d",
   "metadata": {},
   "source": [
    "**NAPOMENA**: Logicki operator <code>and</code> se ne moze primeniti nad nizovima <code>bool</code> vrednosti, vec samo nad dve pojedinacne <code>bool</code> vrednosti. Iz tog razloga na ovom mestu koristimo bitovski operator <code>&</code> koji se moze primeniti nad nizovima 0 i 1 (<code>bool</code> vrednosti se podrazumevano konvertuju u 0 i 1 prilikom primene numerickih operacija)!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "id": "14dc45d3",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(array([38]),)"
      ]
     },
     "execution_count": 31,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "FP_indices = np.where(FP_mask == True)\n",
    "FP_indices"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "id": "d94cc0d5",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(array([32, 95, 97]),)"
      ]
     },
     "execution_count": 32,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "FN_indices = np.where(FN_mask == True)\n",
    "FN_indices"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8c32498a",
   "metadata": {},
   "source": [
    "Funkcija <code>np.where()</code> vraca indekse elemenata koji zadovoljavaju dati uslov. Povratna vrednost je torka koja sadrzi indekse odgovarajucih elemenata po svakoj od dimenzija niza nad kojim se ispituje uslov. U nasem slucaju to su jednodimenzioni nizovi (<code>FP_mask</code> i <code>FN_mask</code>), tako da je povratna vrednost 1-torka."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "id": "9d0c12d3",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[ 0.4871892 ,  0.01303733,  0.39338855,  0.3659976 , -1.30681649,\n",
       "        -0.95126999, -0.60343925, -0.54482173, -0.94354475, -1.16554729,\n",
       "        -0.65652153, -0.39907419, -0.70255831, -0.40017389, -1.38188071,\n",
       "        -0.93387363, -0.75781829, -1.10270943, -1.32385874, -0.96100738,\n",
       "         0.38108732,  0.91995101,  0.30136305,  0.22651728, -1.05909816,\n",
       "        -0.47730201, -0.21514306, -0.44845648, -0.34868429, -0.84855669]])"
      ]
     },
     "execution_count": 33,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "FP_instances = x_test[FP_indices]        \n",
    "FP_instances"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "id": "1f3956bc",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[ 3.00787228e-01,  6.27191395e-01,  2.85068601e-01,\n",
       "         1.66791649e-01, -8.26040721e-01, -4.08147858e-02,\n",
       "        -2.39296242e-01, -2.59789211e-01,  5.21053213e-01,\n",
       "        -5.27245251e-01, -3.06702066e-01,  2.36401902e-01,\n",
       "        -2.73586042e-01, -2.48496739e-01, -8.50670363e-01,\n",
       "         1.19740666e-01,  8.93328096e-02,  1.14354873e-02,\n",
       "        -1.76444532e-01,  7.29323545e-02,  1.35367992e-01,\n",
       "         9.01300544e-01,  1.23681584e-01,  7.27469626e-03,\n",
       "        -1.08945584e+00,  4.81254812e-01,  1.97050595e-01,\n",
       "         2.61877395e-01,  4.21965089e-01,  4.20941512e-01],\n",
       "       [-1.49137756e+00, -4.48140625e-01, -1.41384422e+00,\n",
       "        -1.19357933e+00,  7.01757825e-01,  7.21499884e-01,\n",
       "         3.11920158e+00, -1.21566444e-01,  1.13714354e+00,\n",
       "         2.36955951e+00, -2.72035273e-01, -3.12066536e-02,\n",
       "        -5.08525753e-01, -5.08240856e-01,  8.18567326e-01,\n",
       "         3.33916396e+00,  1.15489460e+01,  3.50196318e+00,\n",
       "         2.76725754e+00,  2.21980946e+00, -1.24279694e+00,\n",
       "        -4.81942427e-01, -1.26971518e+00, -9.71467485e-01,\n",
       "         6.66952567e-01,  1.11696870e+00,  4.73697275e+00,\n",
       "         9.57727509e-01,  2.13184340e+00,  1.73079948e+00],\n",
       "       [ 1.44450088e-01,  1.27509018e+00,  2.09639328e-01,\n",
       "         5.23019440e-02, -5.96692874e-01,  3.61954563e-01,\n",
       "         2.04683401e-01, -2.55911217e-01,  2.55433430e-01,\n",
       "         5.40170270e-02, -5.60399962e-01, -2.47931892e-01,\n",
       "        -1.04982249e-01, -3.77889565e-01,  2.94433907e-02,\n",
       "         1.14420358e+00,  3.08211929e-01, -4.43866865e-02,\n",
       "         3.17089975e-02,  8.84594405e-01,  1.99867423e-02,\n",
       "         9.29276245e-01,  2.26549801e-01, -9.77452060e-02,\n",
       "         5.11254227e-02,  1.01537912e+00,  6.43995940e-01,\n",
       "         9.97222774e-02,  4.60497558e-01,  9.34087151e-01]])"
      ]
     },
     "execution_count": 34,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "FN_instances = x_test[FN_indices]        \n",
    "FN_instances"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a9450ec0",
   "metadata": {},
   "source": [
    "Sve cetiri prethodno pomenute mere kvaliteta modela, i jos mnoge druge, dostupne su kroz modul <code>metrics</code>."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "id": "c22614aa",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.9787234042553191"
      ]
     },
     "execution_count": 35,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "metrics.accuracy_score(y_test, y_test_predicted)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "id": "ee4d12f9",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.9913793103448276"
      ]
     },
     "execution_count": 36,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "metrics.precision_score(y_test, y_test_predicted)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "id": "946d6d66",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.9745762711864406"
      ]
     },
     "execution_count": 37,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "metrics.recall_score(y_test, y_test_predicted)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "id": "7ea24fb9",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.982905982905983"
      ]
     },
     "execution_count": 38,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "metrics.f1_score(y_test, y_test_predicted)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c9ea8298",
   "metadata": {},
   "source": [
    "Sumarna evaluacija po klasama (uzimanjem ponaosob svake klase za pozitivnu klasu) moze se dobiti funkcijom <code>classification_report</code> modula <code>metrics</code>."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "id": "9552d30f",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "              precision    recall  f1-score   support\n",
      "\n",
      "           0       0.96      0.99      0.97        70\n",
      "           1       0.99      0.97      0.98       118\n",
      "\n",
      "    accuracy                           0.98       188\n",
      "   macro avg       0.97      0.98      0.98       188\n",
      "weighted avg       0.98      0.98      0.98       188\n",
      "\n"
     ]
    }
   ],
   "source": [
    "print(metrics.classification_report(y_test, y_test_predicted))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8e2a9983",
   "metadata": {},
   "source": [
    "**Makro prosek** (u klasifikacionom izvestaju obelezen kao 'macro avg') se racuna kao aritmeticka sredina dobijenih vrednosti mera po klasama. Npr. marko prosek za F1 meru se racuna kao $\\frac{0.97+0.98}{2} = 0.975$.\n",
    "\n",
    "**Tezinski prosek** (u klasifikacionom izvestaju obelezen kao 'weighted avg') uzima u obzir i broj instanci koje sluze kao potpora ocenjivanju. Na primer, tezinski prosek za preciznost se racuna kao $\\frac{70}{188}\\cdot 0.96 + \\frac{118}{188}\\cdot 0.99 = 0.978829$."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "af0c56c8",
   "metadata": {},
   "source": [
    "## Analiza sigurnosti klasifikatora"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d5c325be",
   "metadata": {},
   "source": [
    "Kod modela logisticke regresije, pored metoda <code>predict()</code> koji daje labele predikctovane klase, na raspolaganju je i metod <code>predict_proba()</code> kojim se mogu dobiti i ocene verovatnoca pripadnosti pozitivnoj, odnosno negativnoj klasi."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "357034bf",
   "metadata": {},
   "source": [
    "Verovatnoce pripadnosti negativnoj i pozitivnoj klasa za prvih 10 instanci test skupa:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "id": "6fac1a3a",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[3.44799955e-01, 6.55200045e-01],\n",
       "       [2.54526243e-02, 9.74547376e-01],\n",
       "       [2.54084458e-03, 9.97459155e-01],\n",
       "       [3.79091185e-05, 9.99962091e-01],\n",
       "       [9.99992937e-01, 7.06323413e-06],\n",
       "       [6.71895491e-03, 9.93281045e-01],\n",
       "       [3.34538229e-05, 9.99966546e-01],\n",
       "       [9.96629874e-01, 3.37012584e-03],\n",
       "       [1.55095948e-01, 8.44904052e-01],\n",
       "       [2.11101495e-04, 9.99788899e-01]])"
      ]
     },
     "execution_count": 45,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y_test_probabilities_predicted = model.predict_proba(x_test)\n",
    "y_test_probabilities_predicted[:10]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "id": "c507a640",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([1, 1, 1, 1, 0, 1, 1, 0, 1, 1])"
      ]
     },
     "execution_count": 41,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y_test_predicted[:10]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "789fb467",
   "metadata": {},
   "source": [
    "Sigurnost klasifikatora na test skupu:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "id": "bf6f8828",
   "metadata": {},
   "outputs": [],
   "source": [
    "model_confidence = []\n",
    "\n",
    "for p in y_test_probabilities_predicted:\n",
    "    model_confidence.append(np.abs(p[0] - p[1]))\n",
    "    \n",
    "model_confidence = np.array(model_confidence)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "id": "f324811c",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([0.31040009, 0.94909475, 0.99491831, 0.99992418, 0.99998587,\n",
       "       0.98656209, 0.99993309, 0.99325975, 0.6898081 , 0.9995778 ,\n",
       "       1.        , 0.99439018, 0.99999561, 0.94022299, 0.99999654,\n",
       "       0.95268582, 0.997779  , 0.97784376, 0.99997938, 0.96524211,\n",
       "       0.85450409, 0.99873232, 1.        , 0.99964659, 0.99991056,\n",
       "       0.99808181, 0.9896403 , 0.77364561, 0.9439107 , 0.94912212,\n",
       "       0.99833507, 0.99959998, 0.38079678, 0.99999997, 0.99875205,\n",
       "       0.9995952 , 0.99996776, 0.98676164, 0.32804962, 0.79466383,\n",
       "       1.        , 0.99725267, 1.        , 0.99835533, 0.88036114,\n",
       "       0.9701662 , 0.99533039, 0.98688448, 0.99918847, 0.99982539,\n",
       "       0.98277153, 0.99999355, 0.99987151, 0.99781027, 0.99648362,\n",
       "       0.99896155, 0.78959617, 0.9842207 , 1.        , 0.99592352,\n",
       "       0.99718164, 1.        , 0.93659014, 0.99680109, 0.95463716,\n",
       "       0.99937999, 0.98565494, 1.        , 0.99905698, 0.99445142,\n",
       "       0.99987738, 0.99612879, 0.95461312, 0.99621962, 0.99993558,\n",
       "       0.98194481, 1.        , 0.98596331, 0.99981551, 0.98479066,\n",
       "       0.9008494 , 0.99970061, 0.99999987, 0.88358167, 0.99999827,\n",
       "       0.99878012, 0.8660569 , 0.99935942, 0.99969689, 0.91063952,\n",
       "       0.99960924, 0.99968349, 0.92751609, 0.99931965, 0.99426276,\n",
       "       0.31668226, 0.99966112, 0.28043652, 0.14274813, 0.99277112,\n",
       "       0.99867795, 0.99435357, 0.98401367, 1.        , 0.99889926,\n",
       "       0.99991703, 0.8091638 , 0.99962379, 0.99927118, 0.97638142,\n",
       "       0.9999219 , 0.99882206, 0.99984236, 0.9999982 , 0.90272287,\n",
       "       0.99975052, 0.99999996, 0.99999999, 0.93659936, 0.98699657,\n",
       "       0.99996388, 0.89092313, 0.99733052, 0.80923357, 0.32477849,\n",
       "       0.99913841, 0.15186128, 0.97259303, 0.99999853, 0.85422331,\n",
       "       0.99988849, 0.9975237 , 0.99996156, 0.99817037, 0.99507621,\n",
       "       0.99628102, 0.998459  , 0.56721614, 0.99275224, 0.99998053,\n",
       "       0.98460181, 0.9973903 , 0.9999314 , 0.62382684, 0.99789212,\n",
       "       0.9998314 , 0.99413202, 0.99755877, 0.99999788, 0.99177274,\n",
       "       0.99630915, 0.90929555, 1.        , 1.        , 0.91971768,\n",
       "       0.94050029, 0.99999301, 1.        , 0.99981721, 0.80286107,\n",
       "       0.99999952, 0.99986874, 0.99999998, 0.9998575 , 0.99937285,\n",
       "       0.99977389, 0.99996276, 0.99984568, 0.99991933, 0.99861617,\n",
       "       0.64895509, 0.88437554, 0.99951566, 0.98311611, 0.99542814,\n",
       "       0.99552531, 0.99835814, 0.24332369, 0.99998743, 0.91109508,\n",
       "       0.98799914, 0.95745464, 0.99988357, 0.83450608, 0.97434319,\n",
       "       0.99999985, 0.59801182, 0.98841853])"
      ]
     },
     "execution_count": 44,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model_confidence"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "id": "ad930672",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([0.14274813, 0.15186128, 0.24332369, 0.28043652, 0.31040009,\n",
       "       0.31668226, 0.32477849, 0.32804962, 0.38079678, 0.56721614,\n",
       "       0.59801182, 0.62382684, 0.64895509, 0.6898081 , 0.77364561,\n",
       "       0.78959617, 0.79466383, 0.80286107, 0.8091638 , 0.80923357,\n",
       "       0.83450608, 0.85422331, 0.85450409, 0.8660569 , 0.88036114,\n",
       "       0.88358167, 0.88437554, 0.89092313, 0.9008494 , 0.90272287,\n",
       "       0.90929555, 0.91063952, 0.91109508, 0.91971768, 0.92751609,\n",
       "       0.93659014, 0.93659936, 0.94022299, 0.94050029, 0.9439107 ,\n",
       "       0.94909475, 0.94912212, 0.95268582, 0.95461312, 0.95463716,\n",
       "       0.95745464, 0.96524211, 0.9701662 , 0.97259303, 0.97434319,\n",
       "       0.97638142, 0.97784376, 0.98194481, 0.98277153, 0.98311611,\n",
       "       0.98401367, 0.9842207 , 0.98460181, 0.98479066, 0.98565494,\n",
       "       0.98596331, 0.98656209, 0.98676164, 0.98688448, 0.98699657,\n",
       "       0.98799914, 0.98841853, 0.9896403 , 0.99177274, 0.99275224,\n",
       "       0.99277112, 0.99325975, 0.99413202, 0.99426276, 0.99435357,\n",
       "       0.99439018, 0.99445142, 0.99491831, 0.99507621, 0.99533039,\n",
       "       0.99542814, 0.99552531, 0.99592352, 0.99612879, 0.99621962,\n",
       "       0.99628102, 0.99630915, 0.99648362, 0.99680109, 0.99718164,\n",
       "       0.99725267, 0.99733052, 0.9973903 , 0.9975237 , 0.99755877,\n",
       "       0.997779  , 0.99781027, 0.99789212, 0.99808181, 0.99817037,\n",
       "       0.99833507, 0.99835533, 0.99835814, 0.998459  , 0.99861617,\n",
       "       0.99867795, 0.99873232, 0.99875205, 0.99878012, 0.99882206,\n",
       "       0.99889926, 0.99896155, 0.99905698, 0.99913841, 0.99918847,\n",
       "       0.99927118, 0.99931965, 0.99935942, 0.99937285, 0.99937999,\n",
       "       0.99951566, 0.9995778 , 0.9995952 , 0.99959998, 0.99960924,\n",
       "       0.99962379, 0.99964659, 0.99966112, 0.99968349, 0.99969689,\n",
       "       0.99970061, 0.99975052, 0.99977389, 0.99981551, 0.99981721,\n",
       "       0.99982539, 0.9998314 , 0.99984236, 0.99984568, 0.9998575 ,\n",
       "       0.99986874, 0.99987151, 0.99987738, 0.99988357, 0.99988849,\n",
       "       0.99991056, 0.99991703, 0.99991933, 0.9999219 , 0.99992418,\n",
       "       0.9999314 , 0.99993309, 0.99993558, 0.99996156, 0.99996276,\n",
       "       0.99996388, 0.99996776, 0.99997938, 0.99998053, 0.99998587,\n",
       "       0.99998743, 0.99999301, 0.99999355, 0.99999561, 0.99999654,\n",
       "       0.99999788, 0.9999982 , 0.99999827, 0.99999853, 0.99999952,\n",
       "       0.99999985, 0.99999987, 0.99999996, 0.99999997, 0.99999998,\n",
       "       0.99999999, 1.        , 1.        , 1.        , 1.        ,\n",
       "       1.        , 1.        , 1.        , 1.        , 1.        ,\n",
       "       1.        , 1.        , 1.        ])"
      ]
     },
     "execution_count": 47,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model_confidence.sort()\n",
    "model_confidence"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bd676e9f",
   "metadata": {},
   "source": [
    "Na nekim instancama model moze biti vrlo siguran prilikom predikcije, kada je verovatnoca pripadnosti jednoj od klasa osetno veca od verovatnoce pripadnosti drugoj klasi. U slucajevima kada to nije tako, potrebno je ispitati o kojim se to instancama radi i analizirati sigurnosti modela na njima (razlika verovatnoca pripadnosti svakoj od klasa). Ako su to instance na kojima model gresi (FP i FN instance) i ako model na njima ima malu sigurnost, podesavanjem praga preko kojeg model predvidja pozitivnu klasu (podrazumevano 0.5) moze se izvrsiti prilagodjavanje modela i uticati na njegovi tacnost na ovim instancama."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.8"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}
