Probabilistic ML with Quantile Matching: an Example with Python

When we train regressive models, we obtain point predictions. However, in practice we are often interested in estimating the uncertainty associated to each prediction. To achieve that, we assume that the value we are trying to predict is a random variable, and the goal is to estimate its distribution. There are many methods available to estimate uncertainty from predictions, such as <a href="https://en.wikipedia.org/wiki/Prediction_interval" rel="noopener ugc nofollow" target="_blank">variance estimation</a>, <a href="https://www.probabilitycourse.com/chapter9/9_1_9_bayesian_interval_estimation.php" rel="noopener ugc nofollow" target="_blank">Bayesian methods</a>, <a href="https://arxiv.org/abs/2107.07511" rel="noopener ugc nofollow" target="_blank">conformal predictions</a>, etc. Quantile regression is one of these well-known methods. <h1>Quantile regression</h1> Quantile regression consists in estimating one model for each quantile you are interested in. This can be achieved by the use of an asymmetric loss function, known as <a href="https://www.lokad.com/pinball-loss-function-definition/" rel="noopener ugc nofollow" target="_blank">pinball loss</a>. Quantile regression is simple, easy to understand, and readily available in high performing libraries such as <a href="https://lightgbm.readthedocs.io/en/latest/Parameters.html" rel="noopener ugc nofollow" target="_blank">LightGBM</a>. However, quantile regression presents some issues: <ul> <li>There is no guarantee that the order of the quantiles will be correct. For example, your prediction for the 50% quantile could be greater than the one you get for the 60% quantile, which is absurd.</li> <li>To obtain an estimate of the entire distribution, you need to train many models. For instance, if you need an estimate for each point percent quantile, you have to train 99 models.</li> </ul> Here’s how <a href="https://stats.lse.ac.uk/q.yao/qyao.links/paper/mqe.pdf" rel="noopener ugc nofollow" target="_blank">quantile matching</a> can help. <a href="https://towardsdatascience.com/probabilistic-ml-with-quantile-matching-an-example-with-python-c367eee85f18">Learn More</a>