Gaussian Naive Bayes with Hyperparameter Tuning

This article was published as a part of the Data Science Blogathon.

Introduction

Naive Bayes is a classification technique based on the Bayes theorem. It is a simple but powerful algorithm for predictive modeling under supervised learning algorithms. The technique behind Naive Bayes is easy to understand.  Naive Bayes has higher accuracy and speed when we have large data points.

There are three types of Naive Bayes models: Gaussian, Multinomial, and Bernoulli.

1. Gaussian Naive Bayes – This is a variant of Naive Bayes which supports continuous values and has an assumption that each class is normally distributed. 
2. Multinomial Naive Bayes – This is another variant which is an event-based model that has features as vectors where sample(feature) represents frequencies with which certain events have occurred.
3. Bernoulli – This variant is also event-based where features are independent boolean which are in binary form.

Probability of Flight arriving in time:

P(Yes | Dusty) = P( Dusty | Yes) * P(Yes) / P(Dusty)

1. Calculating Prior probability

P(Dusty) = 6/16 = 0.375

P(Yes)= 9/16 = 0.563

2. Calculating Posterior probability

P (Dusty | Yes) = 4/9 = 0.444

Putting Prior and Posterior in our equation:

P (Yes | Dusty) = 0.444 * 0.563 / 0.375 = 0.666

P(No | Dusty) = P( Dusty | No) * P(No) / P(Dusty)

1. Calculating Prior probability

P(Dusty) = 6/16 = 0.375

P(No) = 7/16 = 0.438

2. Calculating Posterior probability

P(Dusty | No) = 2/7 = 0.285

Putting Prior and Posterior in our equation

P(No | Dusty) = 0.285*0.438 / 0.375 = 0.332

Zero Probability Phenomena

# Gaussian Naive Bayes Classification
import numpy as n
import pandas as p
from sklearn.model_selection import c
from sklearn.metrics import a, 
from sklearn.naive_bayes import G
from sklearn.model_selection import t,
import matplotlib.pyplot as p
import seaborn as S
.<a onclick="parent.postMessage({'referent':'.seaborn.set_style'}, '*')">set_style("whitegrid")
import W
.("ignore")
import scipy.stats as s
%matplotlib inline
 = .<a onclick="parent.postMessage({'referent':'.pandas.read_csv'}, '*')">read_csv('/kaggle/input/pima-indians-diabetes-database/diabetes.csv')
 = .(columns=['Outcome'],axis=1)
 = ['Outcome']

from sklearn.impute import S

 = (missing_values=0, strategy="mean")

 = .
 = .<a onclick="parent.postMessage({'referent':'.pandas.DataFrame'}, '*')">DataFrame(.())
 = .<a onclick="parent.postMessage({'referent':'.pandas.DataFrame'}, '*')">DataFrame(.())
.columns = 
.columns = 
.()

#Predicting train and test accuracy

 = .(, ).()

# Accuray Score on train dataset a

 = (,)
('accuracy_score on train dataset : ', )
# predict the target on the test dataset
 = .()
# Accuracy Score on test dataset
 = (,)
('accuracy_score on test dataset : ', )

#accuracy_score on train dataset : 0.7597765363128491
#accuracy_score on test dataset : 0.7575757575757576

Hyperparameter Tuning to improve Accuracy

.<a onclick="parent.postMessage({'referent':'.numpy.logspace'}, '*')">logspace(0,-9, num=10)

from sklearn.model_selection import R

 = (n_splits=5,  n_repeats=3, random_state=999)

from sklearn.preprocessing import P
 = {'var_smoothing': .<a onclick="parent.postMessage({'referent':'.numpy.logspace'}, '*')">logspace(0,-9, num=100)}

 = (estimator=, param_grid=p, cv=c,verbose=1,scoring='accuracy')
 = ().()
.(, );

 = .(.['params'])

['test_score'] = .['mean_test_score']

# predict the target on the test dataset
 = .()
# Accuracy Score on test dataset
 = (,)
('accuracy_score on test dataset : ', )

#accuracy_score on test dataset : 0.7922077922077922

The media shown in this article are not owned by Analytics Vidhya and is used at the Author’s discretion.

Login to continue reading and enjoy expert-curated content.

Free Courses