From c195dd8fc625aa3ae861150dbb3577f5f9117217 Mon Sep 17 00:00:00 2001
From: 202310715065 ANANDA DWI PRASETYO <202310715065@mhs.ubharajaya.ac.id>
Date: Wed, 19 Nov 2025 10:59:25 +0700
Subject: [PATCH] Upload files to "Tugas.Classification"
---
...310715065-Klasifikasi-Decision Trees.ipynb | 781 +++++++++++++++
...rasetyo-202310715065-Klasifikasi-KNN.ipynb | 926 ++++++++++++++++++
...5065-Klasifikasi-Logistic Regression.ipynb | 705 +++++++++++++
...rasetyo-202310715065-Klasifikasi-SVM.ipynb | 581 +++++++++++
4 files changed, 2993 insertions(+)
create mode 100644 Tugas.Classification/Ananda dwi prasetyo-202310715065-Klasifikasi-Decision Trees.ipynb
create mode 100644 Tugas.Classification/Ananda dwi prasetyo-202310715065-Klasifikasi-KNN.ipynb
create mode 100644 Tugas.Classification/Ananda dwi prasetyo-202310715065-Klasifikasi-Logistic Regression.ipynb
create mode 100644 Tugas.Classification/Ananda dwi prasetyo-202310715065-Klasifikasi-SVM.ipynb
diff --git a/Tugas.Classification/Ananda dwi prasetyo-202310715065-Klasifikasi-Decision Trees.ipynb b/Tugas.Classification/Ananda dwi prasetyo-202310715065-Klasifikasi-Decision Trees.ipynb
new file mode 100644
index 0000000..f52feee
--- /dev/null
+++ b/Tugas.Classification/Ananda dwi prasetyo-202310715065-Klasifikasi-Decision Trees.ipynb
@@ -0,0 +1,781 @@
+{
+ "cells": [
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "\n",
+ " \n",
+ " \n",
+ "
\n",
+ "\n",
+ "# Decision Trees\n",
+ "\n",
+ "Estimated time needed: **15** minutes\n",
+ "\n",
+ "## Objectives\n",
+ "\n",
+ "After completing this lab you will be able to:\n",
+ "\n",
+ "* Develop a classification model using Decision Tree Algorithm\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "In this lab exercise, you will learn a popular machine learning algorithm, Decision Trees. You will use this classification algorithm to build a model from the historical data of patients, and their response to different medications. Then you will use the trained decision tree to predict the class of an unknown patient, or to find a proper drug for a new patient.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Table of contents \n",
+ "\n",
+ "\n",
+ "
\n",
+ " About the dataset Downloading the Data Pre-processing Setting up the Decision Tree Modeling Prediction Evaluation Visualization \n",
+ "
\n",
+ " numpy (as np) \n",
+ " pandas \n",
+ " DecisionTreeClassifier from sklearn.tree \n",
+ " \n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "if you uisng you own version comment out\n"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 20,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "# Surpress warnings:\n",
+ "def warn(*args, **kwargs):\n",
+ " pass\n",
+ "import warnings\n",
+ "warnings.warn = warn"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 21,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "import sys\n",
+ "import numpy as np \n",
+ "import pandas as pd\n",
+ "from sklearn.tree import DecisionTreeClassifier\n",
+ "import sklearn.tree as tree"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "\n",
+ "
About the dataset \n",
+ " Imagine that you are a medical researcher compiling data for a study. You have collected data about a set of patients, all of whom suffered from the same illness. During their course of treatment, each patient responded to one of 5 medications, Drug A, Drug B, Drug c, Drug x and y. \n",
+ " \n",
+ "
Downloading the Data \n",
+ " To download the data, we will use pandas library to read itdirectly into a dataframe from IBM Object Storage.\n",
+ "\n"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 22,
+ "metadata": {},
+ "outputs": [
+ {
+ "data": {
+ "text/html": [
+ "\n",
+ "\n",
+ "
\n",
+ " \n",
+ " \n",
+ " Age \n",
+ " Sex \n",
+ " BP \n",
+ " Cholesterol \n",
+ " Na_to_K \n",
+ " Drug \n",
+ " \n",
+ " \n",
+ " \n",
+ " \n",
+ " 0 \n",
+ " 23 \n",
+ " F \n",
+ " HIGH \n",
+ " HIGH \n",
+ " 25.355 \n",
+ " drugY \n",
+ " \n",
+ " \n",
+ " 1 \n",
+ " 47 \n",
+ " M \n",
+ " LOW \n",
+ " HIGH \n",
+ " 13.093 \n",
+ " drugC \n",
+ " \n",
+ " \n",
+ " 2 \n",
+ " 47 \n",
+ " M \n",
+ " LOW \n",
+ " HIGH \n",
+ " 10.114 \n",
+ " drugC \n",
+ " \n",
+ " \n",
+ " 3 \n",
+ " 28 \n",
+ " F \n",
+ " NORMAL \n",
+ " HIGH \n",
+ " 7.798 \n",
+ " drugX \n",
+ " \n",
+ " \n",
+ " 4 \n",
+ " 61 \n",
+ " F \n",
+ " LOW \n",
+ " HIGH \n",
+ " 18.043 \n",
+ " drugY \n",
+ " \n",
+ " \n",
+ "
\n",
+ "
\n",
+ "
Practice \n",
+ " What is the size of data? \n",
+ "\n"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 23,
+ "metadata": {
+ "tags": []
+ },
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "Shape of data: (200, 6)\n",
+ "Number of rows: 200\n",
+ "Number of columns: 6\n"
+ ]
+ }
+ ],
+ "source": [
+ "# Menampilkan ukuran data\n",
+ "print(\"Shape of data:\", my_data.shape)\n",
+ "\n",
+ "# Menampilkan jumlah baris dan kolom secara terpisah\n",
+ "print(\"Number of rows:\", my_data.shape[0])\n",
+ "print(\"Number of columns:\", my_data.shape[1])\n",
+ "\n",
+ "\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Click here for the solution \n",
+ "\n",
+ "```python\n",
+ "my_data.shape\n",
+ "\n",
+ "```\n",
+ "\n",
+ "\n",
+ "
Pre-processing \n",
+ "\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Using my_data as the Drug.csv data read by pandas, declare the following variables: \n",
+ " X as the Feature Matrix (data of my_data) \n",
+ " y as the response vector (target) \n",
+ " \n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Remove the column containing the target name since it doesn't contain numeric values.\n"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 24,
+ "metadata": {},
+ "outputs": [
+ {
+ "data": {
+ "text/plain": [
+ "array([[23, 'F', 'HIGH', 'HIGH', 25.355],\n",
+ " [47, 'M', 'LOW', 'HIGH', 13.093],\n",
+ " [47, 'M', 'LOW', 'HIGH', 10.114],\n",
+ " [28, 'F', 'NORMAL', 'HIGH', 7.798],\n",
+ " [61, 'F', 'LOW', 'HIGH', 18.043]], dtype=object)"
+ ]
+ },
+ "execution_count": 24,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "X = my_data[['Age', 'Sex', 'BP', 'Cholesterol', 'Na_to_K']].values\n",
+ "X[0:5]\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "As you may figure out, some features in this dataset are categorical, such as **Sex** or **BP**. Unfortunately, Sklearn Decision Trees does not handle categorical variables. We can still convert these features to numerical values using the **LabelEncoder() method**\n",
+ "to convert the categorical variable into dummy/indicator variables.\n"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 25,
+ "metadata": {},
+ "outputs": [
+ {
+ "data": {
+ "text/plain": [
+ "array([[23, 0, 0, 0, 25.355],\n",
+ " [47, 1, 1, 0, 13.093],\n",
+ " [47, 1, 1, 0, 10.114],\n",
+ " [28, 0, 2, 0, 7.798],\n",
+ " [61, 0, 1, 0, 18.043]], dtype=object)"
+ ]
+ },
+ "execution_count": 25,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "from sklearn import preprocessing\n",
+ "le_sex = preprocessing.LabelEncoder()\n",
+ "le_sex.fit(['F','M'])\n",
+ "X[:,1] = le_sex.transform(X[:,1]) \n",
+ "\n",
+ "\n",
+ "le_BP = preprocessing.LabelEncoder()\n",
+ "le_BP.fit([ 'LOW', 'NORMAL', 'HIGH'])\n",
+ "X[:,2] = le_BP.transform(X[:,2])\n",
+ "\n",
+ "\n",
+ "le_Chol = preprocessing.LabelEncoder()\n",
+ "le_Chol.fit([ 'NORMAL', 'HIGH'])\n",
+ "X[:,3] = le_Chol.transform(X[:,3]) \n",
+ "\n",
+ "X[0:5]\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Now we can fill the target variable.\n"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 26,
+ "metadata": {},
+ "outputs": [
+ {
+ "data": {
+ "text/plain": [
+ "0 drugY\n",
+ "1 drugC\n",
+ "2 drugC\n",
+ "3 drugX\n",
+ "4 drugY\n",
+ "Name: Drug, dtype: object"
+ ]
+ },
+ "execution_count": 26,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "y = my_data[\"Drug\"]\n",
+ "y[0:5]"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "\n",
+ "
Setting up the Decision Tree \n",
+ " We will be using train/test split on our decision tree . Let's import train_test_split from sklearn.cross_validation .\n",
+ "\n"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 27,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "from sklearn.model_selection import train_test_split"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Now train_test_split will return 4 different parameters. We will name them: train_test_split will need the parameters: X and y are the arrays required before the split, the test_size represents the ratio of the testing dataset, and the random_state ensures that we obtain the same splits.\n"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 28,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "X_trainset, X_testset, y_trainset, y_testset = train_test_split(X, y, test_size=0.3, random_state=3)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Practice \n",
+ "Print the shape of X_trainset and y_trainset. Ensure that the dimensions match.\n"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 29,
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "Shape of X training set (140, 5) & Size of Y training set (140,)\n"
+ ]
+ }
+ ],
+ "source": [
+ "print('Shape of X training set {}'.format(X_trainset.shape),'&',' Size of Y training set {}'.format(y_trainset.shape))\n",
+ "\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Click here for the solution \n",
+ "\n",
+ "```python\n",
+ "print('Shape of X training set {}'.format(X_trainset.shape),'&',' Size of Y training set {}'.format(y_trainset.shape))\n",
+ "\n",
+ "```\n",
+ "\n",
+ "Click here for the solution \n",
+ "\n",
+ "```python\n",
+ "print('Shape of X test set {}'.format(X_testset.shape),'&','Size of y test set {}'.format(y_testset.shape))\n",
+ "\n",
+ "```\n",
+ "\n",
+ "\n",
+ "
Modeling \n",
+ " We will first create an instance of the DecisionTreeClassifier called drugTree . criterion=\"entropy\" so we can see the information gain of each node.\n",
+ "\n"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 31,
+ "metadata": {},
+ "outputs": [
+ {
+ "data": {
+ "text/plain": [
+ "DecisionTreeClassifier(criterion='entropy', max_depth=4)"
+ ]
+ },
+ "execution_count": 31,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "drugTree = DecisionTreeClassifier(criterion=\"entropy\", max_depth = 4)\n",
+ "drugTree # it shows the default parameters"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Next, we will fit the data with the training feature matrix X_trainset and training response vector y_trainset \n"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 32,
+ "metadata": {},
+ "outputs": [
+ {
+ "data": {
+ "text/plain": [
+ "DecisionTreeClassifier(criterion='entropy', max_depth=4)"
+ ]
+ },
+ "execution_count": 32,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "drugTree.fit(X_trainset,y_trainset)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "\n",
+ "
Prediction \n",
+ " Let's make some predictions on the testing dataset and store it into a variable called predTree .\n",
+ "\n"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 33,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "predTree = drugTree.predict(X_testset)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "You can print out predTree and y_testset if you want to visually compare the predictions to the actual values.\n"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 34,
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "['drugY' 'drugX' 'drugX' 'drugX' 'drugX']\n",
+ "40 drugY\n",
+ "51 drugX\n",
+ "139 drugX\n",
+ "197 drugX\n",
+ "170 drugX\n",
+ "Name: Drug, dtype: object\n"
+ ]
+ }
+ ],
+ "source": [
+ "print (predTree [0:5])\n",
+ "print (y_testset [0:5])\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "\n",
+ "
Evaluation \n",
+ " Next, let's import metrics from sklearn and check the accuracy of our model.\n",
+ "\n"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 35,
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "DecisionTrees's Accuracy: 0.9833333333333333\n"
+ ]
+ }
+ ],
+ "source": [
+ "from sklearn import metrics\n",
+ "import matplotlib.pyplot as plt\n",
+ "print(\"DecisionTrees's Accuracy: \", metrics.accuracy_score(y_testset, predTree))"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "**Accuracy classification score** computes subset accuracy: the set of labels predicted for a sample must exactly match the corresponding set of labels in y_true.\n",
+ "\n",
+ "In multilabel classification, the function returns the subset accuracy. If the entire set of predicted labels for a sample strictly matches with the true set of labels, then the subset accuracy is 1.0; otherwise it is 0.0.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "\n",
+ "
Visualization \n",
+ "\n",
+ "Let's visualize the tree\n",
+ "\n",
+ "\n"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 36,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "# Notice: You might need to uncomment and install the pydotplus and graphviz libraries if you have not installed these before\n",
+ "#!conda install -c conda-forge pydotplus -y\n",
+ "#!conda install -c conda-forge python-graphviz -y\n",
+ "\n",
+ "#After executing the code below, a file named 'tree.png' would be generated which contains the decision tree image."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 37,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "from sklearn.tree import export_graphviz\n",
+ "export_graphviz(drugTree, out_file='tree.dot', filled=True, feature_names=['Age', 'Sex', 'BP', 'Cholesterol', 'Na_to_K'])\n",
+ "!dot -Tpng tree.dot -o tree.png\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "### Thank you for completing this lab!\n",
+ "\n",
+ "## Author\n",
+ "\n",
+ "Saeed Aghabozorgi\n",
+ "\n",
+ "### Other Contributors\n",
+ "\n",
+ "Joseph Santarcangelo \n",
+ "\n",
+ "Richard Ye \n",
+ "\n",
+ "## © IBM Corporation 2020. All rights reserved. \n",
+ " \n",
+ " \n",
+ "
\n",
+ "\n",
+ "# K-Nearest Neighbors\n",
+ "\n",
+ "Estimated time needed: **25** minutes\n",
+ "\n",
+ "## Objectives\n",
+ "\n",
+ "After completing this lab you will be able to:\n",
+ "\n",
+ "* Use K Nearest neighbors to classify data\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "In this Lab you will load a customer dataset, fit the data, and use K-Nearest Neighbors to predict a data point. But what is **K-Nearest Neighbors**?\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "**K-Nearest Neighbors** is a supervised learning algorithm. Where the data is 'trained' with data points corresponding to their classification. To predict the class of a given data point, it takes into account the classes of the 'K' nearest data points and chooses the class in which the majority of the 'K' nearest data points belong to as the predicted class.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "### Here's an visualization of the K-Nearest Neighbors algorithm.\n",
+ "\n",
+ "Table of contents \n",
+ "\n",
+ "\n",
+ "\n",
+ "
About the dataset \n",
+ "\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Imagine a telecommunications provider has segmented its customer base by service usage patterns, categorizing the customers into four groups. If demographic data can be used to predict group membership, the company can customize offers for individual prospective customers. It is a classification problem. That is, given the dataset, with predefined labels, we need to build a model to be used to predict class of a new or unknown case.\n",
+ "\n",
+ "The example focuses on using demographic data, such as region, age, and marital, to predict usage patterns.\n",
+ "\n",
+ "The target field, called **custcat**, has four possible values that correspond to the four customer groups, as follows:\n",
+ "1- Basic Service\n",
+ "2- E-Service\n",
+ "3- Plus Service\n",
+ "4- Total Service\n",
+ "\n",
+ "Our objective is to build a classifier, to predict the class of unknown cases. We will use a specific type of classification called K nearest neighbour.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "### Load Data \n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Let's read the data using pandas library and print the first five rows.\n"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 3,
+ "metadata": {},
+ "outputs": [
+ {
+ "data": {
+ "text/html": [
+ "\n",
+ "\n",
+ "
\n",
+ " \n",
+ " \n",
+ " region \n",
+ " tenure \n",
+ " age \n",
+ " marital \n",
+ " address \n",
+ " income \n",
+ " ed \n",
+ " employ \n",
+ " retire \n",
+ " gender \n",
+ " reside \n",
+ " custcat \n",
+ " \n",
+ " \n",
+ " \n",
+ " \n",
+ " 0 \n",
+ " 2 \n",
+ " 13 \n",
+ " 44 \n",
+ " 1 \n",
+ " 9 \n",
+ " 64.0 \n",
+ " 4 \n",
+ " 5 \n",
+ " 0.0 \n",
+ " 0 \n",
+ " 2 \n",
+ " 1 \n",
+ " \n",
+ " \n",
+ " 1 \n",
+ " 3 \n",
+ " 11 \n",
+ " 33 \n",
+ " 1 \n",
+ " 7 \n",
+ " 136.0 \n",
+ " 5 \n",
+ " 5 \n",
+ " 0.0 \n",
+ " 0 \n",
+ " 6 \n",
+ " 4 \n",
+ " \n",
+ " \n",
+ " 2 \n",
+ " 3 \n",
+ " 68 \n",
+ " 52 \n",
+ " 1 \n",
+ " 24 \n",
+ " 116.0 \n",
+ " 1 \n",
+ " 29 \n",
+ " 0.0 \n",
+ " 1 \n",
+ " 2 \n",
+ " 3 \n",
+ " \n",
+ " \n",
+ " 3 \n",
+ " 2 \n",
+ " 33 \n",
+ " 33 \n",
+ " 0 \n",
+ " 12 \n",
+ " 33.0 \n",
+ " 2 \n",
+ " 0 \n",
+ " 0.0 \n",
+ " 1 \n",
+ " 1 \n",
+ " 1 \n",
+ " \n",
+ " \n",
+ " 4 \n",
+ " 2 \n",
+ " 23 \n",
+ " 30 \n",
+ " 1 \n",
+ " 9 \n",
+ " 30.0 \n",
+ " 1 \n",
+ " 2 \n",
+ " 0.0 \n",
+ " 0 \n",
+ " 4 \n",
+ " 3 \n",
+ " \n",
+ " \n",
+ "
\n",
+ "
\n",
+ "
Data Visualization and Analysis \n",
+ "\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "#### Let’s see how many of each class is in our data set\n"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 4,
+ "metadata": {},
+ "outputs": [
+ {
+ "data": {
+ "text/plain": [
+ "3 281\n",
+ "1 266\n",
+ "4 236\n",
+ "2 217\n",
+ "Name: custcat, dtype: int64"
+ ]
+ },
+ "execution_count": 4,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "df['custcat'].value_counts()"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "#### 281 Plus Service, 266 Basic-service, 236 Total Service, and 217 E-Service customers\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "You can easily explore your data using visualization techniques:\n"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 5,
+ "metadata": {},
+ "outputs": [
+ {
+ "data": {
+ "text/plain": [
+ "array([[]], dtype=object)"
+ ]
+ },
+ "execution_count": 5,
+ "metadata": {},
+ "output_type": "execute_result"
+ },
+ {
+ "data": {
+ "image/png": "iVBORw0KGgoAAAANSUhEUgAAAjkAAAGxCAYAAACA4KdFAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/NK7nSAAAACXBIWXMAAA9hAAAPYQGoP6dpAAAoVElEQVR4nO3df1DU94H/8ddGlxUIUIHCQkRDUm2vAW2C0WiTSFQwJMZ0bGtyGmvubM6cPy6MOome03HttGjNt8YbbExytWpqPZKbi17utAZslJQSU4LaqLnapCUaGxAPEVBwWeH9/cNjx4/gD3QFefN8zOzovve9nx+vvte+8mGXdRljjAAAACxzS3cfAAAAwI1AyQEAAFai5AAAACtRcgAAgJUoOQAAwEqUHAAAYCVKDgAAsBIlBwAAWImSAwAArETJAXBZGzZskMvl0meffdbdhwIAneLiax0AXM6JEyf05z//WXfffbc8Hk93Hw4AXDVKDgAAsBI/rgJwWRf/uCozM1NpaWkqKyvTAw88oIiICN1xxx1asWKFWltbHc89deqUFixYoDvuuEMej0cJCQl65JFH9Mc//jE45+TJk5o9e7Zuu+02hYWF6Y477tCSJUvk9/sd23K5XJo7d67Wr1+vr371qwoPD9fw4cO1Z88eGWP04osvKjU1VbfeeqvGjh2rTz/9tN257Ny5U+PGjVN0dLQiIiL0zW9+U7/5zW9CHxqAmwIlB0CnVVVVadq0aXrqqaf09ttvKycnR4sXL9amTZuCcxoaGnT//ffr1Vdf1d/93d/pv/7rv/TKK69oyJAhqqyslCSdPXtWDz30kF5//XXNnz9f27Zt01NPPaWVK1dq8uTJ7fb73//93/r5z3+uFStW6N/+7d/U0NCgRx99VAsWLNDvfvc7rVmzRq+99po+/vhjffvb39aFF6o3bdqk7OxsRUdHa+PGjXrzzTcVGxurCRMmUHQAWxkAuIz169cbSaaiosIYY8yYMWOMJPPBBx845n396183EyZMCN7/4Q9/aCSZoqKiS277lVdeMZLMm2++6Rj/yU9+YiSZwsLC4Jgk4/V6zenTp4NjW7duNZLMN77xDdPa2hocX716tZFkPvroI2OMMWfOnDGxsbHmsccec+ynpaXFDBs2zIwYMeIq0wDQk3AlB0Cneb1ejRgxwjE2dOhQHTlyJHj/17/+tYYMGaLx48dfcjvvvvuuIiMj9Z3vfMcx/vTTT0tSuyssDz30kCIjI4P3/+Zv/kaSlJOTI5fL1W687XhKS0t18uRJzZgxQ+fOnQveWltb9fDDD6usrExnzpy52tMH0EP07e4DANDzxMXFtRvzeDxqamoK3j9x4oQGDhx42e3U1NTI6/U6CookJSQkqG/fvqqpqXGMx8bGOu6HhYVddvzs2bOSpOPHj0tSuzJ1oZMnTzoKFICej5ID4Ib48pe/rGPHjl12TlxcnD744AMZYxxFp7q6WufOnVN8fHxIjqVtO/n5+brvvvs6nJOYmBiSfQG4efDjKgA3RE5Ojv70pz/p3XffveSccePG6fTp09q6datj/PXXXw8+Hgrf/OY39aUvfUkff/yxhg8f3uGt7eoPAHtwJQfADZGbm6s33nhDjz/+uBYtWqQRI0aoqalJxcXFmjhxoh566CF973vf089+9jPNmDFDn332mdLT01VSUqK8vDw98sgjl30/T2fceuutys/P14wZM3Ty5El95zvfUUJCgk6cOKE//OEPOnHihNauXRuSfQG4eVByANwQUVFRKikpkc/n02uvvaZly5apf//+uvfee/UP//APkqR+/fpp165dWrJkiV588UWdOHFCt912mxYuXKilS5eG9HieeuopDRw4UCtXrtSsWbPU0NCghIQEfeMb3wi+0RmAXfiNxwAAwEq8JwcAAFiJkgMAAKxEyQEAAFai5AAAACtRcgAAgJUoOQAAwEo98vfktLa26osvvlBUVFS777wBAAA3J2OMGhoalJycrFtuufHXWXpkyfniiy+UkpLS3YcBAACuweeff64BAwbc8P30yJITFRUl6XxI0dHR1729QCCgwsJCZWdny+12X/f2ejrycCIPJ/JwIg8n8nAiD6eTJ08qNTU1+P/jN1qPLDltP6KKjo4OWcmJiIhQdHQ0i1DkcTHycCIPJ/JwIg8n8nAKBAKS1GVvNeGNxwAAwEqUHAAAYCVKDgAAsBIlBwAAWImSAwAArETJAQAAVqLkAAAAK1FyAACAlSg5AADASpQcAABgJUoOAACwEiUHAABYiZIDAACsRMkBAABW6tvdB3Azun3RtivO+WzFo11wJAAA4FpxJQcAAFiJkgMAAKxEyQEAAFai5AAAACtRcgAAgJUoOQAAwEqUHAAAYCVKDgAAsBIlBwAAWImSAwAArETJAQAAVqLkAAAAK1FyAACAlSg5AADASpQcAABgJUoOAACwEiUHAABYiZIDAACsRMkBAABWouQAAAArUXIAAICVKDkAAMBKlBwAAGAlSg4AALASJQcAAFiJkgMAAKxEyQEAAFai5AAAACtRcgAAgJUoOQAAwEqUHAAAYKXrKjnLly+Xy+VSbm5ucMwYI5/Pp+TkZIWHhyszM1OHDh1yPM/v92vevHmKj49XZGSkJk2apGPHjl3PoQAAADhcc8kpKyvTa6+9pqFDhzrGV65cqVWrVmnNmjUqKyuT1+tVVlaWGhoagnNyc3O1ZcsWFRQUqKSkRKdPn9bEiRPV0tJy7WcCAABwgWsqOadPn9a0adP0r//6r+rfv39w3Bij1atXa8mSJZo8ebLS0tK0ceNGNTY2avPmzZKkuro6rVu3Tj/96U81fvx43X333dq0aZMOHDignTt3huasAABAr9f3Wp40Z84cPfrooxo/frx+9KMfBccrKipUVVWl7Ozs4JjH49GYMWNUWlqqWbNmqby8XIFAwDEnOTlZaWlpKi0t1YQJE9rtz+/3y+/3B+/X19dLkgKBgAKBwLWcgkPbNtr+9PQxV/0cG12cR29HHk7k4UQeTuThRB5OXZ1Dp0tOQUGB9u7dq7KysnaPVVVVSZISExMd44mJiTpy5EhwTlhYmOMKUNuctudfbPny5Vq2bFm78cLCQkVERHT2FC6pqKhIkrRyxJXnbt++PWT7vVm15YHzyMOJPJzIw4k8nMjjvMbGxi7dX6dKzueff67nnntOhYWF6tev3yXnuVwux31jTLuxi11uzuLFizV//vzg/fr6eqWkpCg7O1vR0dGdOIOOBQIBFRUVKSsrS263W2m+d674nIO+9lecbHFxHr0deTiRhxN5OJGHE3k41dTUdOn+OlVyysvLVV1drYyMjOBYS0uL3nvvPa1Zs0aHDx+WdP5qTVJSUnBOdXV18OqO1+tVc3OzamtrHVdzqqurNXr06A736/F45PF42o273e6QLpq27flbLl/I2ubaLtT59nTk4UQeTuThRB5O5HFeV2fQqTcejxs3TgcOHND+/fuDt+HDh2vatGnav3+/7rjjDnm9XsdluebmZhUXFwcLTEZGhtxut2NOZWWlDh48eMmSAwAA0FmdupITFRWltLQ0x1hkZKTi4uKC47m5ucrLy9PgwYM1ePBg5eXlKSIiQlOnTpUkxcTEaObMmVqwYIHi4uIUGxurhQsXKj09XePHjw/RaQEAgN7umj5ddTnPP/+8mpqaNHv2bNXW1mrkyJEqLCxUVFRUcM5LL72kvn37asqUKWpqatK4ceO0YcMG9enTJ9SHAwAAeqnrLjm7d+923He5XPL5fPL5fJd8Tr9+/ZSfn6/8/Pzr3T0AAECH+O4qAABgJUoOAACwEiUHAABYiZIDAACsRMkBAABWouQAAAArUXIAAICVKDkAAMBKlBwAAGAlSg4AALASJQcAAFiJkgMAAKxEyQEAAFai5AAAACtRcgAAgJUoOQAAwEqUHAAAYCVKDgAAsBIlBwAAWImSAwAArETJAQAAVqLkAAAAK1FyAACAlSg5AADASpQcAABgJUoOAACwEiUHAABYiZIDAACsRMkBAABWouQAAAArUXIAAICVKDkAAMBKlBwAAGAlSg4AALASJQcAAFiJkgMAAKxEyQEAAFai5AAAACtRcgAAgJUoOQAAwEqUHAAAYCVKDgAAsBIlBwAAWImSAwAArETJAQAAVqLkAAAAK1FyAACAlSg5AADASpQcAABgJUoOAACwEiUHAABYiZIDAACsRMkBAABWouQAAAArUXIAAICVKDkAAMBKlBwAAGAlSg4AALASJQcAAFiJkgMAAKxEyQEAAFai5AAAACtRcgAAgJUoOQAAwEqUHAAAYCVKDgAAsBIlBwAAWImSAwAArNSpkrN27VoNHTpU0dHRio6O1qhRo/TrX/86+LgxRj6fT8nJyQoPD1dmZqYOHTrk2Ibf79e8efMUHx+vyMhITZo0SceOHQvN2QAAAPyfTpWcAQMGaMWKFfrwww/14YcfauzYsXr88ceDRWblypVatWqV1qxZo7KyMnm9XmVlZamhoSG4jdzcXG3ZskUFBQUqKSnR6dOnNXHiRLW0tIT2zAAAQK/WqZLz2GOP6ZFHHtGQIUM0ZMgQ/fjHP9att96qPXv2yBij1atXa8mSJZo8ebLS0tK0ceNGNTY2avPmzZKkuro6rVu3Tj/96U81fvx43X333dq0aZMOHDignTt33pATBAAAvVPfa31iS0uL/v3f/11nzpzRqFGjVFFRoaqqKmVnZwfneDwejRkzRqWlpZo1a5bKy8sVCAQcc5KTk5WWlqbS0lJNmDChw335/X75/f7g/fr6eklSIBBQIBC41lMIattG25+ePuaqn2Oji/Po7cjDiTycyMOJPJzIw6mrc+h0yTlw4IBGjRqls2fP6tZbb9WWLVv09a9/XaWlpZKkxMREx/zExEQdOXJEklRVVaWwsDD179+/3ZyqqqpL7nP58uVatmxZu/HCwkJFRER09hQuqaioSJK0csSV527fvj1k+71ZteWB88jDiTycyMOJPJzI47zGxsYu3V+nS85Xv/pV7d+/X6dOndJ//Md/aMaMGSouLg4+7nK5HPONMe3GLnalOYsXL9b8+fOD9+vr65WSkqLs7GxFR0d39hTaCQQCKioqUlZWltxut9J871zxOQd9HV91ssHFefR25OFEHk7k4UQeTuThVFNT06X763TJCQsL01e+8hVJ0vDhw1VWVqZ/+Zd/0QsvvCDp/NWapKSk4Pzq6urg1R2v16vm5mbV1tY6ruZUV1dr9OjRl9ynx+ORx+NpN+52u0O6aNq252+5fClrm2u7UOfb05GHE3k4kYcTeTiRx3ldncF1/54cY4z8fr9SU1Pl9Xodl+Sam5tVXFwcLDAZGRlyu92OOZWVlTp48OBlSw4AAEBndepKzj//8z8rJydHKSkpamhoUEFBgXbv3q0dO3bI5XIpNzdXeXl5Gjx4sAYPHqy8vDxFRERo6tSpkqSYmBjNnDlTCxYsUFxcnGJjY7Vw4UKlp6dr/PjxN+QEAQBA79SpknP8+HFNnz5dlZWViomJ0dChQ7Vjxw5lZWVJkp5//nk1NTVp9uzZqq2t1ciRI1VYWKioqKjgNl566SX17dtXU6ZMUVNTk8aNG6cNGzaoT58+oT0zAADQq3Wq5Kxbt+6yj7tcLvl8Pvl8vkvO6devn/Lz85Wfn9+ZXQMAAHQK310FAACsRMkBAABWouQAAAArUXIAAICVKDkAAMBKlBwAAGAlSg4AALASJQcAAFiJkgMAAKxEyQEAAFai5AAAACtRcgAAgJUoOQAAwEqUHAAAYCVKDgAAsBIlBwAAWImSAwAArETJAQAAVqLkAAAAK/Xt7gPoqW5ftO2Kcz5b8WgXHAkAAOgIV3IAAICVKDkAAMBKlBwAAGAlSg4AALASJQcAAFiJkgMAAKxEyQEAAFai5AAAACtRcgAAgJUoOQAAwEqUHAAAYCVKDgAAsBIlBwAAWImSAwAArETJAQAAVqLkAAAAK1FyAACAlSg5AADASpQcAABgJUoOAACwEiUHAABYiZIDAACsRMkBAABWouQAAAArUXIAAICVKDkAAMBKlBwAAGAlSg4AALASJQcAAFiJkgMAAKxEyQEAAFai5AAAACtRcgAAgJUoOQAAwEqUHAAAYCVKDgAAsBIlBwAAWImSAwAArETJAQAAVqLkAAAAK1FyAACAlSg5AADASpQcAABgJUoOAACwEiUHAABYiZIDAACsRMkBAABWouQAAAArdarkLF++XPfee6+ioqKUkJCgb33rWzp8+LBjjjFGPp9PycnJCg8PV2Zmpg4dOuSY4/f7NW/ePMXHxysyMlKTJk3SsWPHrv9sAAAA/k+nSk5xcbHmzJmjPXv2qKioSOfOnVN2drbOnDkTnLNy5UqtWrVKa9asUVlZmbxer7KystTQ0BCck5ubqy1btqigoEAlJSU6ffq0Jk6cqJaWltCdGQAA6NX6dmbyjh07HPfXr1+vhIQElZeX68EHH5QxRqtXr9aSJUs0efJkSdLGjRuVmJiozZs3a9asWaqrq9O6dev0y1/+UuPHj5ckbdq0SSkpKdq5c6cmTJgQolMDAAC9WadKzsXq6uokSbGxsZKkiooKVVVVKTs7OzjH4/FozJgxKi0t1axZs1ReXq5AIOCYk5ycrLS0NJWWlnZYcvx+v/x+f/B+fX29JCkQCCgQCFzPKQS3c+Gfnj7murd54fZ6movz6O3Iw4k8nMjDiTycyMOpq3O45pJjjNH8+fN1//33Ky0tTZJUVVUlSUpMTHTMTUxM1JEjR4JzwsLC1L9//3Zz2p5/seXLl2vZsmXtxgsLCxUREXGtp9BOUVGRJGnliNBsb/v27aHZUDdpywPnkYcTeTiRhxN5OJHHeY2NjV26v2suOXPnztVHH32kkpKSdo+5XC7HfWNMu7GLXW7O4sWLNX/+/OD9+vp6paSkKDs7W9HR0ddw9E6BQEBFRUXKysqS2+1Wmu+d696mJB309cwfvV2cR29HHk7k4UQeTuThRB5ONTU1Xbq/ayo58+bN09tvv6333ntPAwYMCI57vV5J56/WJCUlBcerq6uDV3e8Xq+am5tVW1vruJpTXV2t0aNHd7g/j8cjj8fTbtztdod00bRtz99y+ULWme31ZKHOt6cjDyfycCIPJ/JwIo/zujqDTn26yhijuXPn6q233tK7776r1NRUx+Opqanyer2Oy3LNzc0qLi4OFpiMjAy53W7HnMrKSh08ePCSJQcAAKCzOnUlZ86cOdq8ebP+8z//U1FRUcH30MTExCg8PFwul0u5ubnKy8vT4MGDNXjwYOXl5SkiIkJTp04Nzp05c6YWLFiguLg4xcbGauHChUpPTw9+2goAAOB6darkrF27VpKUmZnpGF+/fr2efvppSdLzzz+vpqYmzZ49W7W1tRo5cqQKCwsVFRUVnP/SSy+pb9++mjJlipqamjRu3Dht2LBBffr0ub6zAQAA+D+dKjnGXPmj1S6XSz6fTz6f75Jz+vXrp/z8fOXn53dm9wAAAFeN764CAABWouQAAAArUXIAAICVKDkAAMBKlBwAAGAlSg4AALASJQcAAFiJkgMAAKxEyQEAAFai5AAAACtRcgAAgJUoOQAAwEqUHAAAYCVKDgAAsBIlBwAAWImSAwAArETJAQAAVqLkAAAAK1FyAACAlSg5AADASpQcAABgJUoOAACwEiUHAABYiZIDAACsRMkBAABWouQAAAArUXIAAICVKDkAAMBKlBwAAGAlSg4AALASJQcAAFiJkgMAAKxEyQEAAFai5AAAACtRcgAAgJUoOQAAwEqUHAAAYCVKDgAAsBIlBwAAWImSAwAArETJAQAAVqLkAAAAK1FyAACAlSg5AADASpQcAABgJUoOAACwEiUHAABYiZIDAACsRMkBAABWouQAAAArUXIAAICVKDkAAMBKlBwAAGAlSg4AALASJQcAAFiJkgMAAKxEyQEAAFai5AAAACtRcgAAgJUoOQAAwEp9u/sAbHb7om1XnPPZike74EgAAOh9uJIDAACsRMkBAABWouQAAAArUXIAAICVKDkAAMBKlBwAAGAlSg4AALBSp0vOe++9p8cee0zJyclyuVzaunWr43FjjHw+n5KTkxUeHq7MzEwdOnTIMcfv92vevHmKj49XZGSkJk2apGPHjl3XiQAAAFyo0yXnzJkzGjZsmNasWdPh4ytXrtSqVau0Zs0alZWVyev1KisrSw0NDcE5ubm52rJliwoKClRSUqLTp09r4sSJamlpufYzAQAAuECnf+NxTk6OcnJyOnzMGKPVq1dryZIlmjx5siRp48aNSkxM1ObNmzVr1izV1dVp3bp1+uUvf6nx48dLkjZt2qSUlBTt3LlTEyZMuI7TAQAAOC+kX+tQUVGhqqoqZWdnB8c8Ho/GjBmj0tJSzZo1S+Xl5QoEAo45ycnJSktLU2lpaYclx+/3y+/3B+/X19dLkgKBgAKBwHUfd9s22v709DHXvc3O7vtmcnEevR15OJGHE3k4kYcTeTh1dQ4hLTlVVVWSpMTERMd4YmKijhw5EpwTFham/v37t5vT9vyLLV++XMuWLWs3XlhYqIiIiFAcuiSpqKhIkrRyRMg2eUXbt2/vup11UlseOI88nMjDiTycyMOJPM5rbGzs0v3dkC/odLlcjvvGmHZjF7vcnMWLF2v+/PnB+/X19UpJSVF2draio6Ov+3gDgYCKioqUlZUlt9utNN87173Nq3XQd/P9eO7iPHo78nAiDyfycCIPJ/Jwqqmp6dL9hbTkeL1eSeev1iQlJQXHq6urg1d3vF6vmpubVVtb67iaU11drdGjR3e4XY/HI4/H027c7XaHdNG0bc/fcvlCFko386IPdb49HXk4kYcTeTiRhxN5nNfVGYT09+SkpqbK6/U6Lss1NzeruLg4WGAyMjLkdrsdcyorK3Xw4MFLlhwAAIDO6vSVnNOnT+vTTz8N3q+oqND+/fsVGxurgQMHKjc3V3l5eRo8eLAGDx6svLw8RUREaOrUqZKkmJgYzZw5UwsWLFBcXJxiY2O1cOFCpaenBz9tBQAAcL06XXI+/PBDPfTQQ8H7be+VmTFjhjZs2KDnn39eTU1Nmj17tmprazVy5EgVFhYqKioq+JyXXnpJffv21ZQpU9TU1KRx48Zpw4YN6tOnTwhOCQAA4BpKTmZmpoy59EesXS6XfD6ffD7fJef069dP+fn5ys/P7+zuAQAArgrfXQUAAKxEyQEAAFai5AAAACtRcgAAgJUoOQAAwEqUHAAAYCVKDgAAsBIlBwAAWImSAwAArETJAQAAVqLkAAAAK1FyAACAlSg5AADASpQcAABgJUoOAACwEiUHAABYiZIDAACsRMkBAABWouQAAAArUXIAAICVKDkAAMBKlBwAAGAlSg4AALASJQcAAFiJkgMAAKxEyQEAAFai5AAAACtRcgAAgJUoOQAAwEqUHAAAYCVKDgAAsFLf7j6A3u72RduuOOezFY92wZEAAGAXruQAAAArUXIAAICVKDkAAMBKlBwAAGAlSg4AALASJQcAAFiJkgMAAKxEyQEAAFai5AAAACvxG497AH4rMgAAnceVHAAAYCVKDgAAsBIlBwAAWImSAwAArETJAQAAVqLkAAAAK1FyAACAlSg5AADASpQcAABgJUoOAACwEiUHAABYiZIDAACsRMkBAABW4lvILcE3lQMA4MSVHAAAYCVKDgAAsBIlBwAAWImSAwAArMQbj+Fw+6Jt8vQxWjlCSvO9I3+Lq90c3sAMAOgJuJIDAACsRMkBAABW4sdVvcjV/C4dAABswZUcAABgJUoOAACwEiUHAABYiZIDAACs1K1vPH755Zf14osvqrKyUnfddZdWr16tBx54oDsPCVehK9/AzO/kAQBcq267kvPGG28oNzdXS5Ys0b59+/TAAw8oJydHR48e7a5DAgAAFum2KzmrVq3SzJkz9f3vf1+StHr1ar3zzjtau3atli9f3l2HhR7oaq4scUUIAHqfbik5zc3NKi8v16JFixzj2dnZKi0tbTff7/fL7/cH79fV1UmSTp48qUAgcN3HEwgE1NjYqJqaGrndbvU9d+a6t9mT9W01amxsVd/ALWppbf+1Dl3pKwvfvOKcq1nEV7OdDxaP63D8wvVx//9775q3c6GRy39zxTlXozv2dfHr5WbWFTl3Rx5Xe15Xsz5CrSetj65ws+ZxNWvoRqyfkydPSpKMMSHfdodMN/jrX/9qJJnf/e53jvEf//jHZsiQIe3mL1261Ejixo0bN27cuFlw+/Of/9wlfaNb33jscjmvEhhj2o1J0uLFizV//vzg/dbWVp08eVJxcXEdzu+s+vp6paSk6PPPP1d0dPR1b6+nIw8n8nAiDyfycCIPJ/Jwqqur08CBAxUbG9sl++uWkhMfH68+ffqoqqrKMV5dXa3ExMR28z0ejzwej2PsS1/6UsiPKzo6mkV4AfJwIg8n8nAiDyfycCIPp1tu6ZrPPXXLp6vCwsKUkZGhoqIix3hRUZFGjx7dHYcEAAAs020/rpo/f76mT5+u4cOHa9SoUXrttdd09OhRPfvss911SAAAwCLdVnKeeOIJ1dTU6Ic//KEqKyuVlpam7du3a9CgQV1+LB6PR0uXLm33I7HeijycyMOJPJzIw4k8nMjDqavzcBnTVZ/jAgAA6Dp8dxUAALASJQcAAFiJkgMAAKxEyQEAAFai5AAAACtRciS9/PLLSk1NVb9+/ZSRkaHf/va33X1IIbd8+XLde++9ioqKUkJCgr71rW/p8OHDjjlPP/20XC6X43bfffc55vj9fs2bN0/x8fGKjIzUpEmTdOzYsa48lZDw+XztztXr9QYfN8bI5/MpOTlZ4eHhyszM1KFDhxzbsCULSbr99tvb5eFyuTRnzhxJ9q+N9957T4899piSk5Plcrm0detWx+OhWg+1tbWaPn26YmJiFBMTo+nTp+vUqVM3+Ow673J5BAIBvfDCC0pPT1dkZKSSk5P1ve99T1988YVjG5mZme3WzJNPPumYY0MeUuheH7bk0dG/JS6XSy+++GJwTletj15fct544w3l5uZqyZIl2rdvnx544AHl5OTo6NGj3X1oIVVcXKw5c+Zoz549Kioq0rlz55Sdna0zZ5zfuP7www+rsrIyeNu+fbvj8dzcXG3ZskUFBQUqKSnR6dOnNXHiRLW0tHTl6YTEXXfd5TjXAwcOBB9buXKlVq1apTVr1qisrExer1dZWVlqaGgIzrEpi7KyMkcWbb+N/Lvf/W5wjs1r48yZMxo2bJjWrFnT4eOhWg9Tp07V/v37tWPHDu3YsUP79+/X9OnTb/j5ddbl8mhsbNTevXv1gx/8QHv37tVbb72lP/3pT5o0aVK7uc8884xjzbz66quOx23Io00oXh+25HFhDpWVlfrFL34hl8ulb3/72455XbI+uuRrQG9iI0aMMM8++6xj7Gtf+5pZtGhRNx1R16iurjaSTHFxcXBsxowZ5vHHH7/kc06dOmXcbrcpKCgIjv31r381t9xyi9mxY8eNPNyQW7p0qRk2bFiHj7W2thqv12tWrFgRHDt79qyJiYkxr7zyijHGriw68txzz5k777zTtLa2GmN619qQZLZs2RK8H6r18PHHHxtJZs+ePcE577//vpFk/vjHP97gs7p2F+fRkd///vdGkjly5EhwbMyYMea555675HNsyiMUrw+b8rjY448/bsaOHesY66r10auv5DQ3N6u8vFzZ2dmO8ezsbJWWlnbTUXWNuro6SWr3TbC7d+9WQkKChgwZomeeeUbV1dXBx8rLyxUIBBx5JScnKy0trUfm9cknnyg5OVmpqal68skn9Ze//EWSVFFRoaqqKsd5ejwejRkzJnietmVxoebmZm3atEl///d/L5fLFRzvTWvjQqFaD++//75iYmI0cuTI4Jz77rtPMTExPT6juro6uVyudl+c/Ktf/Urx8fG66667tHDhQseVL9vyuN7Xh215tDl+/Li2bdummTNntnusK9ZHt32tw83gf//3f9XS0tLum88TExPbfUO6TYwxmj9/vu6//36lpaUFx3NycvTd735XgwYNUkVFhX7wgx9o7NixKi8vl8fjUVVVlcLCwtS/f3/H9npiXiNHjtTrr7+uIUOG6Pjx4/rRj36k0aNH69ChQ8Fz6WhdHDlyRJKsyuJiW7du1alTp/T0008Hx3rT2rhYqNZDVVWVEhIS2m0/ISGhR2d09uxZLVq0SFOnTnV8y/a0adOUmpoqr9ergwcPavHixfrDH/4Q/FGoTXmE4vVhUx4X2rhxo6KiojR58mTHeFetj15dctpc+F+r0vkScPGYTebOnauPPvpIJSUljvEnnngi+Pe0tDQNHz5cgwYN0rZt29ot0Av1xLxycnKCf09PT9eoUaN05513auPGjcE3DF7LuuiJWVxs3bp1ysnJUXJycnCsN62NSwnFeuhofk/OKBAI6Mknn1Rra6tefvllx2PPPPNM8O9paWkaPHiwhg8frr179+qee+6RZE8eoXp92JLHhX7xi19o2rRp6tevn2O8q9ZHr/5xVXx8vPr06dOuFVZXV7f7rzZbzJs3T2+//bZ27dqlAQMGXHZuUlKSBg0apE8++USS5PV61dzcrNraWsc8G/KKjIxUenq6Pvnkk+CnrC63LmzN4siRI9q5c6e+//3vX3Zeb1oboVoPXq9Xx48fb7f9EydO9MiMAoGApkyZooqKChUVFTmu4nTknnvukdvtdqwZm/K40LW8PmzM47e//a0OHz58xX9PpBu3Pnp1yQkLC1NGRkbw8liboqIijR49upuO6sYwxmju3Ll666239O677yo1NfWKz6mpqdHnn3+upKQkSVJGRobcbrcjr8rKSh08eLDH5+X3+/U///M/SkpKCl5CvfA8m5ubVVxcHDxPW7NYv369EhIS9Oijj152Xm9aG6FaD6NGjVJdXZ1+//vfB+d88MEHqqur63EZtRWcTz75RDt37lRcXNwVn3Po0CEFAoHgmrEpj4tdy+vDxjzWrVunjIwMDRs27Ipzb9j6uOq3KFuqoKDAuN1us27dOvPxxx+b3NxcExkZaT777LPuPrSQ+sd//EcTExNjdu/ebSorK4O3xsZGY4wxDQ0NZsGCBaa0tNRUVFSYXbt2mVGjRpnbbrvN1NfXB7fz7LPPmgEDBpidO3eavXv3mrFjx5phw4aZc+fOddepXZMFCxaY3bt3m7/85S9mz549ZuLEiSYqKir4v/uKFStMTEyMeeutt8yBAwfM3/7t35qkpCQrs2jT0tJiBg4caF544QXHeG9YGw0NDWbfvn1m3759RpJZtWqV2bdvX/DTQqFaDw8//LAZOnSoef/99837779v0tPTzcSJE7v8fK/kcnkEAgEzadIkM2DAALN//37Hvyd+v98YY8ynn35qli1bZsrKykxFRYXZtm2b+drXvmbuvvtu6/II5evDhjza1NXVmYiICLN27dp2z+/K9dHrS44xxvzsZz8zgwYNMmFhYeaee+5xfKzaFpI6vK1fv94YY0xjY6PJzs42X/7yl43b7TYDBw40M2bMMEePHnVsp6mpycydO9fExsaa8PBwM3HixHZzeoInnnjCJCUlGbfbbZKTk83kyZPNoUOHgo+3traapUuXGq/Xazwej3nwwQfNgQMHHNuwJYs277zzjpFkDh8+7BjvDWtj165dHb4+ZsyYYYwJ3Xqoqakx06ZNM1FRUSYqKspMmzbN1NbWdtFZXr3L5VFRUXHJf0927dpljDHm6NGj5sEHHzSxsbEmLCzM3Hnnneaf/umfTE1NjWM/NuQRyteHDXm0efXVV014eLg5depUu+d35fpwGWPM1V/3AQAA6Bl69XtyAACAvSg5AADASpQcAABgJUoOAACwEiUHAABYiZIDAACsRMkBAABWouQAAAArUXIAAICVKDkAAMBKlBwAAGCl/w81p+JqPaH0LgAAAABJRU5ErkJggg==\n",
+ "text/plain": [
+ ""
+ ]
+ },
+ "metadata": {},
+ "output_type": "display_data"
+ }
+ ],
+ "source": [
+ "df.hist(column='income', bins=50)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "### Feature set\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Let's define feature sets, X:\n"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 6,
+ "metadata": {},
+ "outputs": [
+ {
+ "data": {
+ "text/plain": [
+ "Index(['region', 'tenure', 'age', 'marital', 'address', 'income', 'ed',\n",
+ " 'employ', 'retire', 'gender', 'reside', 'custcat'],\n",
+ " dtype='object')"
+ ]
+ },
+ "execution_count": 6,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "df.columns"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "To use scikit-learn library, we have to convert the Pandas data frame to a Numpy array:\n"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 7,
+ "metadata": {},
+ "outputs": [
+ {
+ "data": {
+ "text/plain": [
+ "array([[ 2., 13., 44., 1., 9., 64., 4., 5., 0., 0., 2.],\n",
+ " [ 3., 11., 33., 1., 7., 136., 5., 5., 0., 0., 6.],\n",
+ " [ 3., 68., 52., 1., 24., 116., 1., 29., 0., 1., 2.],\n",
+ " [ 2., 33., 33., 0., 12., 33., 2., 0., 0., 1., 1.],\n",
+ " [ 2., 23., 30., 1., 9., 30., 1., 2., 0., 0., 4.]])"
+ ]
+ },
+ "execution_count": 7,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "X = df[['region', 'tenure','age', 'marital', 'address', 'income', 'ed', 'employ','retire', 'gender', 'reside']] .values #.astype(float)\n",
+ "X[0:5]\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "What are our labels?\n"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 8,
+ "metadata": {},
+ "outputs": [
+ {
+ "data": {
+ "text/plain": [
+ "array([1, 4, 3, 1, 3])"
+ ]
+ },
+ "execution_count": 8,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "y = df['custcat'].values\n",
+ "y[0:5]"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "## Normalize Data\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Data Standardization gives the data zero mean and unit variance, it is good practice, especially for algorithms such as KNN which is based on the distance of data points:\n"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 9,
+ "metadata": {},
+ "outputs": [
+ {
+ "data": {
+ "text/plain": [
+ "array([[-0.02696767, -1.055125 , 0.18450456, 1.0100505 , -0.25303431,\n",
+ " -0.12650641, 1.0877526 , -0.5941226 , -0.22207644, -1.03459817,\n",
+ " -0.23065004],\n",
+ " [ 1.19883553, -1.14880563, -0.69181243, 1.0100505 , -0.4514148 ,\n",
+ " 0.54644972, 1.9062271 , -0.5941226 , -0.22207644, -1.03459817,\n",
+ " 2.55666158],\n",
+ " [ 1.19883553, 1.52109247, 0.82182601, 1.0100505 , 1.23481934,\n",
+ " 0.35951747, -1.36767088, 1.78752803, -0.22207644, 0.96655883,\n",
+ " -0.23065004],\n",
+ " [-0.02696767, -0.11831864, -0.69181243, -0.9900495 , 0.04453642,\n",
+ " -0.41625141, -0.54919639, -1.09029981, -0.22207644, 0.96655883,\n",
+ " -0.92747794],\n",
+ " [-0.02696767, -0.58672182, -0.93080797, 1.0100505 , -0.25303431,\n",
+ " -0.44429125, -1.36767088, -0.89182893, -0.22207644, -1.03459817,\n",
+ " 1.16300577]])"
+ ]
+ },
+ "execution_count": 9,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "X = preprocessing.StandardScaler().fit(X).transform(X.astype(float))\n",
+ "X[0:5]"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "### Train Test Split\n",
+ "\n",
+ "Out of Sample Accuracy is the percentage of correct predictions that the model makes on data that the model has NOT been trained on. Doing a train and test on the same dataset will most likely have low out-of-sample accuracy, due to the likelihood of our model overfitting.\n",
+ "\n",
+ "It is important that our models have a high, out-of-sample accuracy, because the purpose of any model, of course, is to make correct predictions on unknown data. So how can we improve out-of-sample accuracy? One way is to use an evaluation approach called Train/Test Split.\n",
+ "Train/Test Split involves splitting the dataset into training and testing sets respectively, which are mutually exclusive. After which, you train with the training set and test with the testing set.\n",
+ "\n",
+ "This will provide a more accurate evaluation on out-of-sample accuracy because the testing dataset is not part of the dataset that has been used to train the model. It is more realistic for the real world problems.\n"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 10,
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "Train set: (800, 11) (800,)\n",
+ "Test set: (200, 11) (200,)\n"
+ ]
+ }
+ ],
+ "source": [
+ "from sklearn.model_selection import train_test_split\n",
+ "X_train, X_test, y_train, y_test = train_test_split( X, y, test_size=0.2, random_state=4)\n",
+ "print ('Train set:', X_train.shape, y_train.shape)\n",
+ "print ('Test set:', X_test.shape, y_test.shape)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "\n",
+ "
Classification \n",
+ "\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "K nearest neighbor (KNN) \n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "#### Import library\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Classifier implementing the k-nearest neighbors vote.\n"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 11,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "from sklearn.neighbors import KNeighborsClassifier"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "### Training\n",
+ "\n",
+ "Let's start the algorithm with k=4 for now:\n"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 12,
+ "metadata": {},
+ "outputs": [
+ {
+ "data": {
+ "text/plain": [
+ "KNeighborsClassifier(n_neighbors=4)"
+ ]
+ },
+ "execution_count": 12,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "k = 4\n",
+ "#Train Model and Predict \n",
+ "neigh = KNeighborsClassifier(n_neighbors = k).fit(X_train,y_train)\n",
+ "neigh"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "### Predicting\n",
+ "\n",
+ "We can use the model to make predictions on the test set:\n"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 13,
+ "metadata": {},
+ "outputs": [
+ {
+ "data": {
+ "text/plain": [
+ "array([1, 1, 3, 2, 4])"
+ ]
+ },
+ "execution_count": 13,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "yhat = neigh.predict(X_test)\n",
+ "yhat[0:5]"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "### Accuracy evaluation\n",
+ "\n",
+ "In multilabel classification, **accuracy classification score** is a function that computes subset accuracy. This function is equal to the jaccard_score function. Essentially, it calculates how closely the actual labels and predicted labels are matched in the test set.\n"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 14,
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "Train set Accuracy: 0.5475\n",
+ "Test set Accuracy: 0.32\n"
+ ]
+ }
+ ],
+ "source": [
+ "from sklearn import metrics\n",
+ "print(\"Train set Accuracy: \", metrics.accuracy_score(y_train, neigh.predict(X_train)))\n",
+ "print(\"Test set Accuracy: \", metrics.accuracy_score(y_test, yhat))"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "## Practice\n",
+ "\n",
+ "Can you build the model again, but this time with k=6?\n"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 20,
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "Train set Accuracy: 0.52\n",
+ "Test set Accuracy: 0.31\n"
+ ]
+ }
+ ],
+ "source": [
+ "from sklearn.neighbors import KNeighborsClassifier\n",
+ "from sklearn import metrics\n",
+ "\n",
+ "# Set k = 6\n",
+ "k = 6\n",
+ "\n",
+ "# Build KNN model\n",
+ "neigh6 = KNeighborsClassifier(n_neighbors=k)\n",
+ "neigh6.fit(X_train, y_train)\n",
+ "\n",
+ "# Predict on test set\n",
+ "yhat6 = neigh6.predict(X_test)\n",
+ "\n",
+ "# Evaluate accuracy\n",
+ "train_acc = metrics.accuracy_score(y_train, neigh6.predict(X_train))\n",
+ "test_acc = metrics.accuracy_score(y_test, yhat6)\n",
+ "\n",
+ "print(\"Train set Accuracy: {:.2f}\".format(train_acc))\n",
+ "print(\"Test set Accuracy: {:.2f}\".format(test_acc))"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Click here for the solution \n",
+ "\n",
+ "```python\n",
+ "k = 6\n",
+ "neigh6 = KNeighborsClassifier(n_neighbors = k).fit(X_train,y_train)\n",
+ "yhat6 = neigh6.predict(X_test)\n",
+ "print(\"Train set Accuracy: \", metrics.accuracy_score(y_train, neigh6.predict(X_train)))\n",
+ "print(\"Test set Accuracy: \", metrics.accuracy_score(y_test, yhat6))\n",
+ "\n",
+ "```\n",
+ "\n",
+ ""
+ ]
+ },
+ "metadata": {},
+ "output_type": "display_data"
+ }
+ ],
+ "source": [
+ "plt.plot(range(1,Ks),mean_acc,'g')\n",
+ "plt.fill_between(range(1,Ks),mean_acc - 1 * std_acc,mean_acc + 1 * std_acc, alpha=0.10)\n",
+ "plt.fill_between(range(1,Ks),mean_acc - 3 * std_acc,mean_acc + 3 * std_acc, alpha=0.10,color=\"green\")\n",
+ "plt.legend(('Accuracy ', '+/- 1xstd','+/- 3xstd'))\n",
+ "plt.ylabel('Accuracy ')\n",
+ "plt.xlabel('Number of Neighbors (K)')\n",
+ "plt.tight_layout()\n",
+ "plt.show()"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 18,
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "The best accuracy was with 0.34 with k= 9\n"
+ ]
+ }
+ ],
+ "source": [
+ "print( \"The best accuracy was with\", mean_acc.max(), \"with k=\", mean_acc.argmax()+1) "
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "### Thank you for completing this lab!\n",
+ "\n",
+ "## Author\n",
+ "\n",
+ "Saeed Aghabozorgi\n",
+ "\n",
+ "### Other Contributors\n",
+ "\n",
+ "Joseph Santarcangelo \n",
+ "\n",
+ "## © IBM Corporation 2020. All rights reserved. \n",
+ " \n",
+ " \n",
+ "
\n",
+ "\n",
+ "\n",
+ "# Logistic Regression with Python\n",
+ "\n",
+ "\n",
+ "Estimated time needed: **25** minutes\n",
+ " \n",
+ "\n",
+ "## Objectives\n",
+ "\n",
+ "After completing this lab you will be able to:\n",
+ "\n",
+ "* Use scikit Logistic Regression to classify\n",
+ "* Understand confusion matrix\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "In this notebook, you will learn Logistic Regression, and then, you'll create a model for a telecommunication company, to predict when its customers will leave for a competitor, so that they can take some action to retain the customers.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Table of contents \n",
+ "\n",
+ "\n",
+ "most probable class for that data point. For this, we use Logistic Regression .\n",
+ "\n",
+ "\n",
+ "
Recall linear regression: \n",
+ "
\n",
+ "
\n",
+ " As you know,
Linear regression finds a function that relates a continuous dependent variable,
y , to some predictors (independent variables $x_1$, $x_2$, etc.). For example, simple linear regression assumes a function of the form:\n",
+ "
\n",
+ "$$\n",
+ "y = \\theta_0 + \\theta_1 x_1 + \\theta_2 x_2 + \\cdots\n",
+ "$$\n",
+ "
\n",
+ "and finds the values of parameters $\\theta_0, \\theta_1, \\theta_2$, etc, where the term $\\theta_0$ is the \"intercept\". It can be generally shown as:\n",
+ "
\n",
+ "$$\n",
+ "ℎ_\\theta(𝑥) = \\theta^TX\n",
+ "$$\n",
+ "
\n",
+ "\n",
+ "
y , is categorical. It produces a formula that predicts the probability of the class label as a function of the independent variables.\n",
+ "\n",
+ "Logistic regression fits a special s-shaped curve by taking the linear regression function and transforming the numeric estimate into a probability with the following function, which is called the sigmoid function 𝜎:\n",
+ "\n",
+ "$$\n",
+ "ℎ_\\theta(𝑥) = \\sigma({\\theta^TX}) = \\frac {e^{(\\theta_0 + \\theta_1 x_1 + \\theta_2 x_2 +...)}}{1 + e^{(\\theta_0 + \\theta_1 x_1 + \\theta_2 x_2 +\\cdots)}}\n",
+ "$$\n",
+ "Or:\n",
+ "$$\n",
+ "ProbabilityOfaClass_1 = P(Y=1|X) = \\sigma({\\theta^TX}) = \\frac{e^{\\theta^TX}}{1+e^{\\theta^TX}} \n",
+ "$$\n",
+ "\n",
+ "In this equation, ${\\theta^TX}$ is the regression result (the sum of the variables weighted by the coefficients), `exp` is the exponential function and $\\sigma(\\theta^TX)$ is the sigmoid or [logistic function](http://en.wikipedia.org/wiki/Logistic_function), also called logistic curve. It is a common \"S\" shape (sigmoid curve).\n",
+ "\n",
+ "So, briefly, Logistic Regression passes the input through the logistic/sigmoid but then treats the result as a probability:\n",
+ "\n",
+ "About the dataset \n",
+ "We will use a telecommunications dataset for predicting customer churn. This is a historical customer dataset where each row represents one customer. The data is relatively easy to understand, and you may uncover insights you can use immediately. Typically it is less expensive to keep customers than acquire new ones, so the focus of this analysis is to predict the customers who will stay with the company. \n",
+ "\n",
+ "\n",
+ "This data set provides information to help you predict what behavior will help you to retain customers. You can analyze all relevant customer data and develop focused customer retention programs.\n",
+ "\n",
+ "\n",
+ "\n",
+ "The dataset includes information about:\n",
+ "\n",
+ "- Customers who left within the last month – the column is called Churn\n",
+ "- Services that each customer has signed up for – phone, multiple lines, internet, online security, online backup, device protection, tech support, and streaming TV and movies\n",
+ "- Customer account information – how long they had been a customer, contract, payment method, paperless billing, monthly charges, and total charges\n",
+ "- Demographic info about customers – gender, age range, and if they have partners and dependents\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "### Load the Telco Churn data \n",
+ "Telco Churn is a hypothetical data file that concerns a telecommunications company's efforts to reduce turnover in its customer base. Each case corresponds to a separate customer and it records various demographic and service usage information. Before you can work with the data, you must use the URL to get the ChurnData.csv.\n",
+ "\n",
+ "To download the data, we will use `!wget` to download it from IBM Object Storage.\n"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "#Click here and press Shift+Enter\n",
+ "!wget -O ChurnData.csv https://cf-courses-data.s3.us.cloud-object-storage.appdomain.cloud/IBMDeveloperSkillsNetwork-ML0101EN-SkillsNetwork/labs/Module%203/data/ChurnData.csv"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "## Load Data From CSV File \n"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "churn_df = pd.read_csv(\"ChurnData.csv\")\n",
+ "churn_df.head()"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Data pre-processing and selection \n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Let's select some features for the modeling. Also, we change the target data type to be an integer, as it is a requirement by the skitlearn algorithm:\n"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "churn_df = churn_df[['tenure', 'age', 'address', 'income', 'ed', 'employ', 'equip', 'callcard', 'wireless','churn']]\n",
+ "churn_df['churn'] = churn_df['churn'].astype('int')\n",
+ "churn_df.head()"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "## Practice\n",
+ "How many rows and columns are in this dataset in total? What are the names of columns?\n"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "# write your code here\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Click here for the solution \n",
+ "\n",
+ "```python\n",
+ "churn_df.shape\n",
+ "\n",
+ "```\n",
+ "\n",
+ "Modeling (Logistic Regression with Scikit-learn) \n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Let's build our model using __LogisticRegression__ from the Scikit-learn package. This function implements logistic regression and can use different numerical optimizers to find parameters, including ‘newton-cg’, ‘lbfgs’, ‘liblinear’, ‘sag’, ‘saga’ solvers. You can find extensive information about the pros and cons of these optimizers if you search it in the internet.\n",
+ "\n",
+ "The version of Logistic Regression in Scikit-learn, support regularization. Regularization is a technique used to solve the overfitting problem of machine learning models.\n",
+ "__C__ parameter indicates __inverse of regularization strength__ which must be a positive float. Smaller values specify stronger regularization. \n",
+ "Now let's fit our model with train set:\n"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "from sklearn.linear_model import LogisticRegression\n",
+ "from sklearn.metrics import confusion_matrix\n",
+ "LR = LogisticRegression(C=0.01, solver='liblinear').fit(X_train,y_train)\n",
+ "LR"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Now we can predict using our test set:\n"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "yhat = LR.predict(X_test)\n",
+ "yhat"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "__predict_proba__ returns estimates for all classes, ordered by the label of classes. So, the first column is the probability of class 0, P(Y=0|X), and second column is probability of class 1, P(Y=1|X):\n"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "yhat_prob = LR.predict_proba(X_test)\n",
+ "yhat_prob"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Evaluation \n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "### jaccard index\n",
+ "Let's try the jaccard index for accuracy evaluation. we can define jaccard as the size of the intersection divided by the size of the union of the two label sets. If the entire set of predicted labels for a sample strictly matches with the true set of labels, then the subset accuracy is 1.0; otherwise it is 0.0.\n",
+ "\n"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "from sklearn.metrics import jaccard_score\n",
+ "jaccard_score(y_test, yhat,pos_label=0)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "### confusion matrix\n",
+ "Another way of looking at the accuracy of the classifier is to look at __confusion matrix__.\n"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "from sklearn.metrics import classification_report, confusion_matrix\n",
+ "import itertools\n",
+ "def plot_confusion_matrix(cm, classes,\n",
+ " normalize=False,\n",
+ " title='Confusion matrix',\n",
+ " cmap=plt.cm.Blues):\n",
+ " \"\"\"\n",
+ " This function prints and plots the confusion matrix.\n",
+ " Normalization can be applied by setting `normalize=True`.\n",
+ " \"\"\"\n",
+ " if normalize:\n",
+ " cm = cm.astype('float') / cm.sum(axis=1)[:, np.newaxis]\n",
+ " print(\"Normalized confusion matrix\")\n",
+ " else:\n",
+ " print('Confusion matrix, without normalization')\n",
+ "\n",
+ " print(cm)\n",
+ "\n",
+ " plt.imshow(cm, interpolation='nearest', cmap=cmap)\n",
+ " plt.title(title)\n",
+ " plt.colorbar()\n",
+ " tick_marks = np.arange(len(classes))\n",
+ " plt.xticks(tick_marks, classes, rotation=45)\n",
+ " plt.yticks(tick_marks, classes)\n",
+ "\n",
+ " fmt = '.2f' if normalize else 'd'\n",
+ " thresh = cm.max() / 2.\n",
+ " for i, j in itertools.product(range(cm.shape[0]), range(cm.shape[1])):\n",
+ " plt.text(j, i, format(cm[i, j], fmt),\n",
+ " horizontalalignment=\"center\",\n",
+ " color=\"white\" if cm[i, j] > thresh else \"black\")\n",
+ "\n",
+ " plt.tight_layout()\n",
+ " plt.ylabel('True label')\n",
+ " plt.xlabel('Predicted label')\n",
+ "print(confusion_matrix(y_test, yhat, labels=[1,0]))"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "# Compute confusion matrix\n",
+ "cnf_matrix = confusion_matrix(y_test, yhat, labels=[1,0])\n",
+ "np.set_printoptions(precision=2)\n",
+ "\n",
+ "\n",
+ "# Plot non-normalized confusion matrix\n",
+ "plt.figure()\n",
+ "plot_confusion_matrix(cnf_matrix, classes=['churn=1','churn=0'],normalize= False, title='Confusion matrix')"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Let's look at first row. The first row is for customers whose actual churn value in the test set is 1.\n",
+ "As you can calculate, out of 40 customers, the churn value of 15 of them is 1. \n",
+ "Out of these 15 cases, the classifier correctly predicted 6 of them as 1, and 9 of them as 0. \n",
+ "\n",
+ "This means, for 6 customers, the actual churn value was 1 in test set and classifier also correctly predicted those as 1. However, while the actual label of 9 customers was 1, the classifier predicted those as 0, which is not very good. We can consider it as the error of the model for first row.\n",
+ "\n",
+ "What about the customers with churn value 0? Lets look at the second row.\n",
+ "It looks like there were 25 customers whom their churn value were 0. \n",
+ "\n",
+ "\n",
+ "The classifier correctly predicted 24 of them as 0, and one of them wrongly as 1. So, it has done a good job in predicting the customers with churn value 0. A good thing about the confusion matrix is that it shows the model’s ability to correctly predict or separate the classes. In a specific case of the binary classifier, such as this example, we can interpret these numbers as the count of true positives, false positives, true negatives, and false negatives. \n"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "print (classification_report(y_test, yhat))\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Based on the count of each section, we can calculate precision and recall of each label:\n",
+ "\n",
+ "\n",
+ "- __Precision__ is a measure of the accuracy provided that a class label has been predicted. It is defined by: precision = TP / (TP + FP)\n",
+ "\n",
+ "- __Recall__ is the true positive rate. It is defined as: Recall = TP / (TP + FN)\n",
+ "\n",
+ " \n",
+ "So, we can calculate the precision and recall of each class.\n",
+ "\n",
+ "__F1 score:__\n",
+ "Now we are in the position to calculate the F1 scores for each label based on the precision and recall of that label. \n",
+ "\n",
+ "The F1 score is the harmonic average of the precision and recall, where an F1 score reaches its best value at 1 (perfect precision and recall) and worst at 0. It is a good way to show that a classifer has a good value for both recall and precision.\n",
+ "\n",
+ "\n",
+ "Finally, we can tell the average accuracy for this classifier is the average of the F1-score for both labels, which is 0.72 in our case.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "### log loss\n",
+ "Now, let's try __log loss__ for evaluation. In logistic regression, the output can be the probability of customer churn is yes (or equals to 1). This probability is a value between 0 and 1.\n",
+ "Log loss( Logarithmic loss) measures the performance of a classifier where the predicted output is a probability value between 0 and 1. \n"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "from sklearn.metrics import log_loss\n",
+ "log_loss(y_test, yhat_prob)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Practice \n",
+ "Try to build Logistic Regression model again for the same dataset, but this time, use different __solver__ and __regularization__ values? What is new __logLoss__ value?\n"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 7,
+ "metadata": {
+ "scrolled": true
+ },
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "Columns: Index(['tenure', 'age', 'address', 'income', 'ed', 'employ', 'equip',\n",
+ " 'callcard', 'wireless', 'longmon', 'tollmon', 'equipmon', 'cardmon',\n",
+ " 'wiremon', 'longten', 'tollten', 'cardten', 'voice', 'pager',\n",
+ " 'internet', 'callwait', 'confer', 'ebill', 'loglong', 'logtoll',\n",
+ " 'lninc', 'custcat', 'churn'],\n",
+ " dtype='object')\n",
+ "New LogLoss value: 0.5939226974155307\n"
+ ]
+ }
+ ],
+ "source": [
+ "# ======================================\n",
+ "# IMPORT LIBRARIES\n",
+ "# ======================================\n",
+ "import pandas as pd\n",
+ "import numpy as np\n",
+ "from sklearn.model_selection import train_test_split\n",
+ "from sklearn.linear_model import LogisticRegression\n",
+ "from sklearn.metrics import log_loss\n",
+ "\n",
+ "# ======================================\n",
+ "# LOAD DATASET\n",
+ "# ======================================\n",
+ "df = pd.read_csv(\"ChurnData.csv\")\n",
+ "\n",
+ "# Tampilkan kolom supaya yakin\n",
+ "print(\"Columns:\", df.columns)\n",
+ "\n",
+ "# ======================================\n",
+ "# PREPROCESSING\n",
+ "# ======================================\n",
+ "df['churn'] = df['churn'].astype(int)\n",
+ "\n",
+ "X = np.asarray(df[['tenure', 'age', 'address', 'income', 'ed', 'employ', 'equip']])\n",
+ "y = np.asarray(df['churn'])\n",
+ "\n",
+ "# Train/test split\n",
+ "X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=4)\n",
+ "\n",
+ "# ======================================\n",
+ "# LOGISTIC REGRESSION\n",
+ "# ======================================\n",
+ "model = LogisticRegression(solver='liblinear', C=0.5, max_iter=2000)\n",
+ "model.fit(X_train, y_train)\n",
+ "\n",
+ "yhat_prob = model.predict_proba(X_test)\n",
+ "new_logloss = log_loss(y_test, yhat_prob)\n",
+ "\n",
+ "print(\"New LogLoss value:\", new_logloss)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Click here for the solution \n",
+ "\n",
+ "```python\n",
+ "LR2 = LogisticRegression(C=0.01, solver='sag').fit(X_train,y_train)\n",
+ "yhat_prob2 = LR2.predict_proba(X_test)\n",
+ "print (\"LogLoss: : %.2f\" % log_loss(y_test, yhat_prob2))\n",
+ "\n",
+ "```\n",
+ "\n",
+ "Joseph Santarcangelo \n",
+ "\n",
+ "## © IBM Corporation 2020. All rights reserved. \n",
+ " \n",
+ " \n",
+ "
\n",
+ "\n",
+ "\n",
+ "# SVM (Support Vector Machines)\n",
+ "\n",
+ "\n",
+ "Estimated time needed: **15** minutes\n",
+ " \n",
+ "\n",
+ "## Objectives\n",
+ "\n",
+ "After completing this lab you will be able to:\n",
+ "\n",
+ "* Use scikit-learn to Support Vector Machine to classify\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "In this notebook, you will use SVM (Support Vector Machines) to build and train a model using human cell records, and classify cells to whether the samples are benign or malignant.\n",
+ "\n",
+ "SVM works by mapping data to a high-dimensional feature space so that data points can be categorized, even when the data are not otherwise linearly separable. A separator between the categories is found, then the data is transformed in such a way that the separator could be drawn as a hyperplane. Following this, characteristics of new data can be used to predict the group to which a new record should belong.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Table of contents \n",
+ "\n",
+ "\n",
+ "Load the Cancer data \n",
+ "The example is based on a dataset that is publicly available from the UCI Machine Learning Repository (Asuncion and Newman, 2007)[http://mlearn.ics.uci.edu/MLRepository.html]. The dataset consists of several hundred human cell sample records, each of which contains the values of a set of cell characteristics. The fields in each record are:\n",
+ "\n",
+ "|Field name|Description|\n",
+ "|--- |--- |\n",
+ "|ID|Clump thickness|\n",
+ "|Clump|Clump thickness|\n",
+ "|UnifSize|Uniformity of cell size|\n",
+ "|UnifShape|Uniformity of cell shape|\n",
+ "|MargAdh|Marginal adhesion|\n",
+ "|SingEpiSize|Single epithelial cell size|\n",
+ "|BareNuc|Bare nuclei|\n",
+ "|BlandChrom|Bland chromatin|\n",
+ "|NormNucl|Normal nucleoli|\n",
+ "|Mit|Mitoses|\n",
+ "|Class|Benign or malignant|\n",
+ "\n",
+ "Modeling (SVM with Scikit-learn) \n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "The SVM algorithm offers a choice of kernel functions for performing its processing. Basically, mapping data into a higher dimensional space is called kernelling. The mathematical function used for the transformation is known as the kernel function, and can be of different types, such as:\n",
+ "\n",
+ " 1.Linear\n",
+ " 2.Polynomial\n",
+ " 3.Radial basis function (RBF)\n",
+ " 4.Sigmoid\n",
+ "Each of these functions has its characteristics, its pros and cons, and its equation, but as there's no easy way of knowing which function performs best with any given dataset. We usually choose different functions in turn and compare the results. Let's just use the default, RBF (Radial Basis Function) for this lab.\n"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "from sklearn import svm\n",
+ "clf = svm.SVC(kernel='rbf')\n",
+ "clf.fit(X_train, y_train) "
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "After being fitted, the model can then be used to predict new values:\n"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "yhat = clf.predict(X_test)\n",
+ "yhat [0:5]"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Evaluation \n"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "from sklearn.metrics import classification_report, confusion_matrix\n",
+ "import itertools"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "def plot_confusion_matrix(cm, classes,\n",
+ " normalize=False,\n",
+ " title='Confusion matrix',\n",
+ " cmap=plt.cm.Blues):\n",
+ " \"\"\"\n",
+ " This function prints and plots the confusion matrix.\n",
+ " Normalization can be applied by setting `normalize=True`.\n",
+ " \"\"\"\n",
+ " if normalize:\n",
+ " cm = cm.astype('float') / cm.sum(axis=1)[:, np.newaxis]\n",
+ " print(\"Normalized confusion matrix\")\n",
+ " else:\n",
+ " print('Confusion matrix, without normalization')\n",
+ "\n",
+ " print(cm)\n",
+ "\n",
+ " plt.imshow(cm, interpolation='nearest', cmap=cmap)\n",
+ " plt.title(title)\n",
+ " plt.colorbar()\n",
+ " tick_marks = np.arange(len(classes))\n",
+ " plt.xticks(tick_marks, classes, rotation=45)\n",
+ " plt.yticks(tick_marks, classes)\n",
+ "\n",
+ " fmt = '.2f' if normalize else 'd'\n",
+ " thresh = cm.max() / 2.\n",
+ " for i, j in itertools.product(range(cm.shape[0]), range(cm.shape[1])):\n",
+ " plt.text(j, i, format(cm[i, j], fmt),\n",
+ " horizontalalignment=\"center\",\n",
+ " color=\"white\" if cm[i, j] > thresh else \"black\")\n",
+ "\n",
+ " plt.tight_layout()\n",
+ " plt.ylabel('True label')\n",
+ " plt.xlabel('Predicted label')"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "# Compute confusion matrix\n",
+ "cnf_matrix = confusion_matrix(y_test, yhat, labels=[2,4])\n",
+ "np.set_printoptions(precision=2)\n",
+ "\n",
+ "print (classification_report(y_test, yhat))\n",
+ "\n",
+ "# Plot non-normalized confusion matrix\n",
+ "plt.figure()\n",
+ "plot_confusion_matrix(cnf_matrix, classes=['Benign(2)','Malignant(4)'],normalize= False, title='Confusion matrix')"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "You can also easily use the __f1_score__ from sklearn library:\n"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "from sklearn.metrics import f1_score\n",
+ "f1_score(y_test, yhat, average='weighted') "
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Let's try the jaccard index for accuracy:\n"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "from sklearn.metrics import jaccard_score\n",
+ "jaccard_score(y_test, yhat,pos_label=2)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Practice \n",
+ "Can you rebuild the model, but this time with a __linear__ kernel? You can use __kernel='linear'__ option, when you define the svm. How the accuracy changes with the new kernel function?\n"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 9,
+ "metadata": {
+ "tags": []
+ },
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "Avg F1-score: 0.9639038982104676\n",
+ "Jaccard score: 0.9038461538461539\n",
+ "\n",
+ "Classification Report:\n",
+ "\n",
+ " precision recall f1-score support\n",
+ "\n",
+ " 0 1.00 0.94 0.97 90\n",
+ " 1 0.90 1.00 0.95 47\n",
+ "\n",
+ " micro avg 0.96 0.96 0.96 137\n",
+ " macro avg 0.95 0.97 0.96 137\n",
+ "weighted avg 0.97 0.96 0.96 137\n",
+ "\n"
+ ]
+ }
+ ],
+ "source": [
+ "import pandas as pd\n",
+ "import numpy as np\n",
+ "from sklearn import preprocessing\n",
+ "from sklearn.model_selection import train_test_split\n",
+ "from sklearn import svm\n",
+ "from sklearn.metrics import classification_report, f1_score\n",
+ "\n",
+ "# Fungsi hitung Jaccard manual (untuk sklearn lama)\n",
+ "def jaccard_manual(y_true, y_pred):\n",
+ " intersection = np.logical_and(y_true == 1, y_pred == 1).sum()\n",
+ " union = np.logical_or(y_true == 1, y_pred == 1).sum()\n",
+ " return intersection / union if union != 0 else 0\n",
+ "\n",
+ "# ================================\n",
+ "# 1. Load Data\n",
+ "# ================================\n",
+ "cell_df = pd.read_csv(\"cell_samples.csv\")\n",
+ "\n",
+ "# ================================\n",
+ "# 2. Bersihkan Kolom BareNuc\n",
+ "# ================================\n",
+ "cell_df = cell_df[pd.to_numeric(cell_df['BareNuc'], errors='coerce').notnull()]\n",
+ "cell_df['BareNuc'] = cell_df['BareNuc'].astype('int')\n",
+ "\n",
+ "# ================================\n",
+ "# 3. Buat Feature dan Label\n",
+ "# ================================\n",
+ "feature_df = cell_df[['Clump','UnifSize','UnifShape','MargAdh',\n",
+ " 'SingEpiSize','BareNuc','BlandChrom','NormNucl','Mit']].astype(float)\n",
+ "\n",
+ "X = np.asarray(feature_df)\n",
+ "y = np.where(cell_df['Class'] == 2, 0, 1) # 0 = Benign, 1 = Malignant\n",
+ "\n",
+ "# ================================\n",
+ "# 4. Split Train/Test\n",
+ "# ================================\n",
+ "X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=4)\n",
+ "\n",
+ "# ================================\n",
+ "# 5. Model SVM Kernel LINEAR\n",
+ "# ================================\n",
+ "model = svm.SVC(kernel='linear')\n",
+ "model.fit(X_train, y_train)\n",
+ "\n",
+ "# ================================\n",
+ "# 6. Prediksi\n",
+ "# ================================\n",
+ "y_pred = model.predict(X_test)\n",
+ "\n",
+ "# ================================\n",
+ "# 7. Evaluasi\n",
+ "# ================================\n",
+ "print(\"Avg F1-score:\", f1_score(y_test, y_pred, average='weighted'))\n",
+ "print(\"Jaccard score:\", jaccard_manual(y_test, y_pred))\n",
+ "print(\"\\nClassification Report:\\n\")\n",
+ "print(classification_report(y_test, y_pred))"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Click here for the solution \n",
+ "\n",
+ "```python\n",
+ "clf2 = svm.SVC(kernel='linear')\n",
+ "clf2.fit(X_train, y_train) \n",
+ "yhat2 = clf2.predict(X_test)\n",
+ "print(\"Avg F1-score: %.4f\" % f1_score(y_test, yhat2, average='weighted'))\n",
+ "print(\"Jaccard score: %.4f\" % jaccard_score(y_test, yhat2,pos_label=2))\n",
+ "\n",
+ "```\n",
+ "\n",
+ "Joseph Santarcangelo \n",
+ "\n",
+ "## © IBM Corporation 2020. All rights reserved. ![\"Skills](\"https://cf-courses-data.s3.us.cloud-object-storage.appdomain.cloud/assets/logos/SN_web_lightmode.png\")
\n",
+ " \n",
+ "![](\"https://cf-courses-data.s3.us.cloud-object-storage.appdomain.cloud/IBMDeveloperSkillsNetwork-ML0101EN-SkillsNetwork/labs/Module%203/images/KNN_Diagram.png\")
\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "In this case, we have data points of Class A and B. We want to predict what the star (test data point) is. If we consider a k value of 3 (3 nearest data points), we will obtain a prediction of Class B. Yet if we consider a k value of 6, we will obtain a prediction of Class A.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "In this sense, it is important to consider the value of k. Hopefully from this diagram, you should get a sense of what the K-Nearest Neighbors algorithm is. It considers the 'K' Nearest Neighbors (data points) when it predicts the classification of the test point.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "![](\"https://cf-courses-data.s3.us.cloud-object-storage.appdomain.cloud/IBMDeveloperSkillsNetwork-ML0101EN-SkillsNetwork/labs/Module%203/images/mod_ID_24_final.png\")
\n",
+ "\n",
+ "\n",
+ "The objective of the __Logistic Regression__ algorithm, is to find the best parameters θ, for $ℎ_\\theta(𝑥)$ = $\\sigma({\\theta^TX})$, in such a way that the model best predicts the class of each case.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "### Customer churn with Logistic Regression\n",
+ "A telecommunications company is concerned about the number of customers leaving their land-line business for cable competitors. They need to understand who is leaving. Imagine that you are an analyst at this company and you have to find out who is leaving and why.\n"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "#!pip install scikit-learn==0.23.1\n",
+ "!pip install scikit-learn\n",
+ "!pip install matplotlib\n",
+ "!pip install pandas \n",
+ "!pip install numpy \n",
+ "%matplotlib inline"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Let's first import required libraries:\n"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "import pandas as pd\n",
+ "import pylab as pl\n",
+ "import numpy as np\n",
+ "import scipy.optimize as opt\n",
+ "from sklearn import preprocessing\n",
+ "%matplotlib inline \n",
+ "import matplotlib.pyplot as plt"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "