Gaussian Naive Bayes

Gaussian Naive Bayes is a type of Naive Bayes method working on continuous attributes and the data features that follows Gaussian distribution throughout the dataset. This “naive” assumption simplifies calculations and makes the model fast and efficient.

• It is widely used because it performs well even with small datasets and is easy to implement and interpret.
• Works well with continuous numerical features
• Fast, simple, and effective for classification problems
• Commonly used in spam detection, medical diagnosis, and pattern recognition

Mathematics

Gaussian Naive Bayes assumes that the likelihood (P()) follows the Gaussian Distribution for each within . Therefore,

Where:

• is the feature value,
• is the mean of the feature values for a given class ,
• is the standard deviation of the feature values for that class,
• is a constant (approximately 3.14159),
• is the base of the natural logarithm.

To classify each new data point x the algorithm finds out the maximum value of the posterior probability of each class and assigns the data point to that class.

Why Gaussian Naive Bayes Works Well for Continuous Data?

Gaussian Naive Bayes is effective for continuous data because it assumes each feature follows a Gaussian (normal) distribution. When this assumption holds true the algorithm performs well. For example, in tasks like medical diagnosis or predicting house prices, where features such as age, income or measurements follow a normal distribution, Gaussian Naive Bayes can make accurate predictions.

Practical Example

To understand how Gaussian Naive Bayes works here's a simple binary classification problem using one feature: petal length.

We want to classify a new sample with petal length = 1.6 cm.

1. Separate by Class

2. Calculate Mean and Variance

For class 0:

For class 1:

3. Gaussian Likelihood

The Gaussian PDF is:

For :

Class 0

Class 1

4. Multiply by Class Priors

Assume equal priors:

Then:

5. Prediction

Since ,

Implementation

Here we will be applying Gaussian Naive Bayes to the Iris Dataset, this dataset consists of four features namely Sepal Length in cm, Sepal Width in cm, Petal Length in cm, Petal Width in cm and from these features we have to identify which feature set belongs to which specie class. The iris flower dataset is available in Sklearn library of python.

Now we will be using Gaussian Naive Bayes in predicting the correct specie of Iris flower.

1. Importing Libraries

First we will be importing the required libraries:

• pandas: for data manipulation
• load_iris: to load dataset
• train_test_split: to split the data into training and testing sets
• GaussianNB: for the Gaussian Naive Bayes classifier
• accuracy_score: to evaluate the model
• LabelEncoder: to encode the categorical target variable.

2. Loading the Dataset and Preparing Features and Target Variable

After that we will load the Iris dataset from a CSV file named "Iris.csv" into a pandas DataFrame. Then we will separate the features (X) and the target variable (y) from the dataset. Features are obtained by dropping the "Species" column and the target variable is set to the "Species" column which we will be predicting.

3. Encoding and Splitting the Dataset

• Since the target variable "Species" is categorical we will be using Label Encoder to convert it into numerical form. This is necessary for the Gaussian Naive Bayes classifier as it requires numerical inputs.
• We will be splitting the dataset into training and testing sets using the train_test_split function. 70% of the data is used for training and 30% is used for testing. The random_state parameter ensures reproducibility of the same data.

4. Creating and Training the Model

We will be creating a Gaussian Naive Bayes Classifier (gnb) and then training it on the training data using the fit method.

Output:

5. Plotting 1D Gaussian Distributions for All Features

We visualize the Gaussian distributions for each feature in the Iris dataset across all classes. The distributions are modeled by the Gaussian Naive Bayes classifier where each class is represented by a normal (Gaussian) distribution with a mean and variance specific to each feature. Separate plots are created for each feature in the dataset showing how each class's feature values are distributed.

Output:

6. Making Predictions

At last we will be using the trained model to make predictions on the testing data.

Output:

The Accuracy of Prediction on Iris Flower is: 0.9777777777777777

High accuracy suggests that the model has effectively learned to distinguish between the three different species of Iris based on the given features (sepal length, sepal width, petal length and petal width). 

Related Articles

• Naive Bayes Classifiers
• Bernoulli Naive Bayes
• Gaussian Naive Bayes using Sklearn
• Multinomial Naive Bayes
• How to plot the decision boundary for a Gaussian Naive Bayes
• ML – Naive Bayes Scratch Implementation using Python