Likelihood is a confusing term. Likelihood is not a probability, but is proportional to a probability; the two terms can’t be used interchangeably. In this post, we will be dissecting likelihood as a concept and understand it’s importance in machine learning.
Intuition
Let us understand likelihood and how it is different from a probability distribution with an imaginary city, Databerg (a cringe name, but bear with me). Let’s also imagine we have access to the pricing data of all houses in this city. I don’t know exactly how this distribution looks since Databerg isn’t a real city, but intuitively I’d say we would notice many houses that are moderately priced and a few houses that are very expensive. If one were to plot a distribution of these prices, it might look something like this.