Predict Client Subscription Using Logistic Regression

Introduction

This article aims to take you through the process of performing a Logistic Regression algorithm in python to classify whether clients will subscribe to a term deposit or not.

Logistic Regression is used for classification problems. It is used to carry out analysis when the dependent variable is binary (dichotomous). In other words, logistic regression is used when there are 2 classes. Logistic regression explains the relationship between one dependent binary variable and one or more nominal, ordinal, interval or ratio-level independent variables.

I assume you are familiar with the concept of logistic regression. In this blog, we shall see how to implement logistic regression using the sklearn library in python. This will enable you to have hands-on experience implementing the logistic regression algorithm and learning the sklearn library.

Let’s dive in!

Importing Required Libraries

Here we will import pandas, numpy, matplotlib, seaborn and scipy. These libraries are required to read the data, perform transformation and visualize the data, respectively.

#Importing the required libraries
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns
import scipy.stats as stats

Loading of the Dataset

Here we will import pandas, numpy, matplotlib, seaborn and scipy. These libraries are required to read the data, perform transformation and visualize the data, respectively.

#Loading the csv file containing the raw data
df = pd.read_csv("Bank Marketing project.csv")
df.head()

You need to load the dataset using panda’s library. Once the data is loaded, you can check the shape of the data i.e. you can find the number of columns and rows in the loaded dataset using the shape attribute of pandas which stores the number of columns and rows as a tuple.

# ives No. of rows and columns in data
df.shape

You can also use the described method of pandas to find the statistical data like minimum, mean, quartiles and maximum values of columns having numeric values.

df.describe()

Data Pre-processing

A. Remove Null Values

df.isnull().sum()

sns.boxplot(data=df,x='duration')

#Removing the outliers
q3 = df['duration'].quantile(.75)
q1 = df['duration'].quantile(.25)
iqr = q3-q1
iqr
upperrange = q3+1.5*iqr
bottomrange = q1-1.5*iqr
df1 = df[(df['duration']>bottomrange) & (df['duration']<upperrange)] 
sns.boxplot(data=df1,x='duration',color = 'purple')

#shows the no.of customers vs Client's Term Deposit Subscription Status
plt.subplots(figsize=(12,4))
sns.countplot(x ='job', hue = 'y', data = dfnew, palette = 'colorblind')
plt.ylabel('No. Of Customers')
plt.show

from sklearn.preprocessing import LabelEncoder
for column in bank.columns:
 if bank[column].dtype == np.number:
 continue
 bank[column] = LabelEncoder().fit_transform(bank[column])
bank.head()

Correlation Matrix

A correlation matrix is a table showing correlation coefficients between variables. Each variable is plotted along the row and column. The intersection between rows and columns gives the correlation coefficient between those variables. The value lies between -1 and 1. More the value towards -1, the variables are said to have a strong negative correlation, and more the value towards +1, the variables are said to have strong positive correlation.

#correlation visualization
plt.subplots(figsize=(8,8))
sns.heatmap(bank.corr(), annot = True, fmt = '0.0%')
plt.show

Splitting of Data into Train and Test

To estimate the performance of machine learning algorithms, you need to split data into train-test datasets. As a thumb rule, the training set is split into 70% of the actual data, and the Test set is split into 30 % of actual data. The train set statistics are known and used to fit the model. The  test data set is solely used for predictions.

#split data into 70% training and 30% testing.
from sklearn.model_selection import train_test_split
X = bank.iloc[:, 1:bank.shape[1]].values
Y = bank.iloc[:,0].values
X_train, X_test, Y_train, Y_test = train_test_split(X,Y, test_size = 0.3, random_state = 1)
X_train

Split Data into Independent ‘X’ and Dependent ‘Y’ Variables

This means to store the data of dependent variables into one variable, “Y”, and the independent variable into “X”. This is done to simplify the code and make it more user friendly.

#Split the data into independent 'X' and dependent 'Y' variables
X = bank.iloc[:, 1:bank.shape[1]].values
Y = bank.iloc[:, 0].values
bank.head()

Logistic Regression Model Building

#Logistic Regression model building
from sklearn.linear_model import LogisticRegression
logmodel=LogisticRegression(solver = 'sag',random_state=100,multi_class='ovr')
logmodel.fit(X_train,Y_train)

#Training Model Score
logmodel.score(X_train,Y_train)

0.9415242494226328

#Testing Model Score
logmodel.score(X_test,Y_test)

0.9398954703832753

#intercept 𝑏₀
logmodel.intercept_

array([-0.07177432])

#slope 𝑏₁
logmodel.coef_

#In the matrix below, each row corresponds to a single observation. 
#The first column is the probability of the predicted output being zero, that is 1 - 𝑝(𝑥). 
#The second column is the probability that the output is one, or 𝑝(𝑥).

logmodel.predict_proba(X)

#This function returns the predicted output values as a one-dimensional array.
logmodel.predict(X)

array([0, 0, 0, …, 0, 0, 0])

Confusion Matrix

A confusion matrix is a table which is  used to describe the performance of a classification model on a set of test data for which the true values are known.

#plotting confusion matrix
from sklearn.metrics import confusion_matrix
cm = confusion_matrix(Y_test,logmodel.predict(X_test))
TN = cm[0][0]
FN = cm[1][0]
FP = cm[0][1]
TP = cm[1][1]
print(cm)
print('The model testing accuracy = {}'.format((TP+TN)/(TP+TN+FN+FP)))

#Confusion matrix heat map

import seaborn as sns
sns.heatmap(cm/np.sum(cm), annot=True,
 fmt='.2%', cmap='Blues')
plt.show

Conclusion

To summarize the article, we learned why to use logistic regression algorithm and how to perform it using python. I also learned the flow and necessary steps to take before building the logistic regression model. This was a basic logistic regression model.

Further, to enhance the model for better results, you need to apply resampling techniques such as oversampling, undersampling or SMOTE when dealing with such imbalanced datasets. Imbalanced data means datasets where the target class has an uneven distribution of observations, i.e. in our case, the variable Term Deposit subscription status has two classes. The class “no”  has a very high number of observations, and the clas ‘yes” one class label has a very high number of observations, and the other has a very low number of observations.

The media shown in this article is not owned by Analytics Vidhya and is used at the Author’s discretion.

Login to continue reading and enjoy expert-curated content.

Free Courses