Hello…there! Chatbots have become an integral part of our digital interactions, and they owe their prowess to Large Language Models (LLMs). One cutting-edge approach in chatbot development is the Retrieval Augmented Generation (RAG) architecture.
It combines the best of both worlds: knowledge bases and generative models, offering reduced hallucinations, up-to-date information, and domain-specific knowledge.
Graphics Credits: Pixababy
However, evaluating chatbot responses generated by these models has proven to be quite the puzzle. Human grading, while reliable, is labor-intensive and tough to scale.
But fear not!
Databricks, in collaboration with Quinn Leng, Senior Software Engineer, has embarked on a mission to shed light on LLM automated evaluation best practices.