If you are interested in math and neural networks, it is impossible that you haven’t heard of Kullback-Leibler Divergence or KL-Divergence as it is popularly called in the deep learning community. It plays a pivotal role, especially in Generative Adversarial Networks (GANs). While the definiti...