Tutorial on Active Inference

Active inference is the <a href="https://www.nature.com/articles/nrn2787" rel="noopener ugc nofollow" target="_blank">Free Energy principle of the brain</a> applied to action. It is relatively supported by experimental neuroscience studies and is a popular model of ‘how the brain works’. In this tutorial, we will consider the latest version, which is formulated as planning in <a href="https://en.wikipedia.org/wiki/Markov_decision_process" rel="noopener ugc nofollow" target="_blank">discrete state-space and time</a>. The <a href="https://www.fil.ion.ucl.ac.uk/~karl/Action%20and%20behavior%20A%20free-energy%20formulation.pdf" rel="noopener ugc nofollow" target="_blank">initial motivation</a> of active inference is that the agent wants to remain alive, by maintaining its homeostasis. To this end, the agent must ensure that important parameters, like body temperature or blood oxygenation, don’t deviate too much from the norm, i.e. are not surprising. But since it’s only possible to infer these parameters from sensory measurements, the agent minimizes surprise of sensory observations instead. Interestingly, this is equivalent to continuously improving agent’s model of the world, as we will see shortly. So in short: remain alive -> maintain homeostasis -> avoid surprising states -> avoid surprising observations -> by minimizing approximation to surprise(free energy). While the <a href="https://medium.com/@solopchuk/intuitions-on-predictive-coding-and-the-free-energy-principle-3fc5bcedc754" rel="noopener">previous post</a> aimed to give a mere intuition on Free Energy, here we will get our hands dirty. No technical background is necessary except the <a href="https://www.mathsisfun.com/data/probability.html" rel="noopener ugc nofollow" target="_blank">probability theory</a> and <a href="https://www.mathsisfun.com/data/bayes-theorem.html" rel="noopener ugc nofollow" target="_blank">Bayes Theorem</a>. The idea goes as follows. <a href="https://medium.com/@solopchuk/tutorial-on-active-inference-30edcf50f5dc">Website</a>