The term Naive Bayes denotes that there is an naive assumption such that every feature is independent of other features and Bayes refers as a name of statistician Thomas Bayes.
Naive Bayes is one of the most simple algorithms for the classification and for large datasets.
The Bayes equation is
P(A|B) = Probability of A is true given that B is true.
P(B|A) = Probability of B is true given that A is true.
P(A)= Probability of A to be true.
P(B)= Probability of B to be true.
There are four terms in this equation,
Posterior, Likelihood, Prior, Marginal/Evidence
Naïve Bayes Algorithm Working
Here we have our Data, which comprises of the Day, Outlook, Humidity, Wind Conditions and the final column being Play, which we have to predict.
Now we will create a frequency table
For each frequency table, we create likelihood table
Suppose we have a Day with the following values :
Outlook = Rain
Humidity = High
Wind = Weak
Likelihood of ‘Yes’ on that Day = P(Outlook = Rain|Yes)*P(Humidity= High|Yes)* P(Wind= Weak|Yes)*P(Yes)
= 3/10 * 3/9 * 6/9 * 9/14 = 3/70 = 0.0428
Likelihood of ‘No’ on that Day = P(Outlook = Rain|No)*P(Humidity= High|No)* P(Wind= Weak|No)*P(No)
= 2/4 * 4/5 * 2/5 * 5/14 = 4/70 = 0.0571
Now we have to normalize the values
P(yes) = 0.0428/ (0.0428 + 0.0571) = 0.428
P(no) = 0.0571/ (0.0428 + 0.0571) = 0.571
Our model predicts that there is a 42.8% chance there will be a Game.
Advantages of Naive Bayes
· This method is better used from multi-class prediction problems.
· As the features are independent, it requires less training data. (Not practical)
Disadvantages of Naive Bayes
· If the frequency is zero, our prediction goes wrong. To overcome this, add some fix number of values as per situation demand.
· It doesn’t apply to real life as all features independent is very rare.
Application of Naive Bayes
· It is widely used for text classification.
· To mark email as spam or not.
· Classify news about technology, politics, sports etc.