Naive Bayes Algorithm

Naive Bayes model is easy to build and particularly useful for very large data sets. Along with simplicity, Naive Bayes is known to outperform even highly sophisticated classification methods.

Bayes theorem provides a way of calculating posterior probability P(c|x) from P(c), P(x) and P(x|c). Look at the equation below:

Let’s understand it using an example. Below I have a training data set of weather and corresponding target variable ‘Play’ (suggesting possibilities of playing). Now, we need to classify whether players will play or not based on weather condition. Let’s follow the below steps to perform it.

Step 1: Convert the data set into a frequency table

Step 2: Create Likelihood table by finding the probabilities like Overcast probability = 0.29 and probability of playing is 0.64.

Step 3: Now, use Naive Bayesian equation to calculate the posterior probability for each class. The class with the highest posterior probability is the outcome of prediction.

Problem: Players will play if weather is sunny. Is this statement is correct?

We can solve it using above discussed method of posterior probability.

P(Yes | Sunny) = P( Sunny | Yes) * P(Yes) / P (Sunny)

Here we have P (Sunny |Yes) = 3/9 = 0.33, P(Sunny) = 5/14 = 0.36, P( Yes)= 9/14 = 0.64

Now, P (Yes | Sunny) = 0.33 * 0.64 / 0.36 = 0.60, which has higher probability.

Naive Bayes uses a similar method to predict the probability of different class based on various attributes. This algorithm is mostly used in text classification and with problems having multiple classes.

Pros:

Cons:

Again, scikit learn (python library) will help here to build a Naive Bayes model in Python. There are three types of Naive Bayes model under scikit learn library:

Based on your data set, you can choose any of above discussed model. Below is the example of Gaussian model.

Above, we looked at the basic Naive Bayes model, you can improve the power of this basic model by tuning parameters and handle assumption intelligently.

Here are some tips for improving power of Naive Bayes Model:

Related posts:

I told my friends on Labor Day weekend that I was moving to Taiwan to teach English. They laughed it off and didn’t believe me at the time. That was a bit over 10 years ago. 10 years that I spent…

We all know what it feels like to be stuck in a rut. You’re going through the motions day after day, but you can’t help feeling like there’s more to life than this. That’s where I was 2020, the year…

write my research paper 10 SIMPLE WAYS TO WRITE STRONGER INTRODUCTIONS 10 Simple Ways To Write Stronger Introductions A coauthor can help you write and publish a guide, however that book will be…