AI Module Week 6 Topic 2: Naive keyword-based rag #63
Description
The goal here is to introduce the simplest type of RAG system that can be called RAG. It will be purposely brittle and short, to introduce key concepts and evaluation metrics using deepeval. The main point is to pair this with topic 3 to show how much better semantic retrieval is than keyword-based retrieval.
Here we introduce the basic idea: parse a set of PDFs into text chunks, and perform basic keyword matching on chunks, and inject the best-matching chunk into the prompt. Evaluate results. Highlights the limitations of brittle keyword search. This will be VERY simple, to illustrate the most basic concepts of RAG (like 100 lines of code).
Introduce concept of evaluation of RAG system using deepeval. This may need a different lesson/md file, will need to be handled carefully! Evaluation is a separate topic in itself, and we plan to run the same metrics through the remaining lessons.