BERT vs GPT: Comparing the NLP Giants

In 2018, NLP researchers were all amazed by the BERT paper [1]. The approach was simple, yet the result was impressive: it set new benchmarks for 11 NLP tasks. <blockquote> In a little over a year, BERT has become a ubiquitous baseline in Natural Language Processing (NLP) experiments counting over 150 research publications analysing and improving the model. [2] </blockquote> In 2022, ChatGPT [3] blew up the whole Internet with its ability to generate human-like responses. The model can comprehend a wide range of topics and carry the conversation naturally for an extended period, which sets it apart from all traditional chatbots. BERT and ChatGPT are significant breakthroughs in NLP, yet their approaches are different. How do their structures differ, and how do they impact the models’ ability? Let’s dive in! <h1>Attention</h1> We must first recall the commonly-used attention to understand the model structure fully. Attention mechanisms are designed to capture and model relationships between tokens in a sequence, which is one of the reasons why they have been so successful in NLP tasks. <a href="https://towardsdatascience.com/bert-vs-gpt-comparing-the-nlp-giants-329d105e34ec">Click Here</a>