Naive Bayes-Machine Learning Algorithm

The term Naive Bayes denotes that there is an naive assumption such that every feature is independent of other features and Bayes refers as a name of statistician Thomas Bayes.

Naive Bayes is one of the most simple algorithms for the classification and for large datasets.

The Bayes equation is

Bayes Equation


P(A|B) = Probability of A is true given that B is true.

P(B|A) = Probability of B is true given that A is true.

P(A)= Probability of A to be true.

P(B)= Probability of B to be true.

There are four terms in this equation,

Posterior, Likelihood, Prior, Marginal/Evidence

Keywords in Bayes equation

Naïve Bayes Algorithm Working

Here we have our Data, which comprises of the Day, Outlook, Humidity, Wind Conditions and the final column being Play, which we have to predict.


Now we will create a frequency table

Frequency Tables

For each frequency table, we create likelihood table

For Outlook

Likelihood table- Outlook

For Humidity

Likelihood table- Humidity

For wind

Likelihood table- wind


Suppose we have a Day with the following values :
Outlook = Rain
Humidity = High
Wind = Weak
Play =?

Likelihood of ‘Yes’ on that Day = P(Outlook = Rain|Yes)*P(Humidity= High|Yes)* P(Wind= Weak|Yes)*P(Yes)
= 3/10 * 3/9 * 6/9 * 9/14 = 3/70 = 0.0428

Likelihood of ‘No’ on that Day = P(Outlook = Rain|No)*P(Humidity= High|No)* P(Wind= Weak|No)*P(No)
= 2/4 * 4/5 * 2/5 * 5/14 = 4/70 = 0.0571

Now we have to normalize the values

P(yes) = 0.0428/ (0.0428 + 0.0571) = 0.428

P(no) = 0.0571/ (0.0428 + 0.0571) = 0.571

Our model predicts that there is a 42.8% chance there will be a Game.

Advantages of Naive Bayes

· This method is better used from multi-class prediction problems.

· As the features are independent, it requires less training data. (Not practical)

Disadvantages of Naive Bayes

· If the frequency is zero, our prediction goes wrong. To overcome this, add some fix number of values as per situation demand.

· It doesn’t apply to real life as all features independent is very rare.

Application of Naive Bayes

· It is widely used for text classification.

· To mark email as spam or not.

· Classify news about technology, politics, sports etc.

Naïve Bayes Implementation in Python- colab

Click here





ML enthusiast