A fAIry tale of the Inductive Bias

On the other hand, while we have plenty of work showing in practice that these models work well, the theoretical understanding of why has lagged behind. This is because these models are very broad and it comes difficult to experiment. The fact that <a href="https://en.wikipedia.org/wiki/Vision_transformer" rel="noopener ugc nofollow" target="_blank">Vision Transformers</a> outperform convolutional neural networks <a href="https://towardsdatascience.com/metas-hiera-reduce-complexity-to-increase-accuracy-30f7a147ad0b" rel="noopener" target="_blank">by having a theoretically less inductive bias for vision</a> shows that there is a theoretical gap to be filled. This article focuses on: <ul> <li>What exactly is inductive bias? Why this is important and what inductive bias do our favorite models have?</li> <li>The inductive bias of transformers and CNNs. What are the differences between these two models and why these discussions are important?</li> <li>How can we study inductive bias? How to be able to leverage the similarity between different models in order to capture their differences.</li> </ul> <a href="https://towardsdatascience.com/a-fairy-tale-of-the-inductive-bias-d418fc61726c">Visit Now</a>