What is perplexity in NLP?

Perplexity in NLP

In terms of NLP or Machine Learning perplexity has three meanings that are:

• Perplexity is the measurement of the ease of prediction for the probability distribution.
• The measurement of the variability of the prediction model is known as perplexity.
• In most simple words, the prediction error measurement is known as perplexity.

All of these definitions are closely related to each other. Like for example, you are having a four-sided dice with different probabilities for all different sides like 0.10, 0.40, 0.20 and 0.30. Now this dice is having the perplexity of 3.5961. In the same manner, if you have similar dice with all sides equal that can be 0.25, 0.25, 0.25 and 0.25 then its perplexity will be 4.0. The prediction level for both of them is different. In the second case, it is easier to predict in comparison to the first one.