15 different RAG implementation from Scratch

[adithya-s-k]

Content Type

This issue is to create 15 different Retrieval-Augmented Generation (RAG) implementations from scratch using base libraries like:

• litellm for LLM interaction
• lancedb for vector storage and retrieval
• Embedding models from HuggingFace (e.g., BAAI/bge-base-en-v1.5, sentence-transformers family)

The goal is to recreate each RAG variant found in repo using a minimal stack with readable, modifiable code, integrating AI Engineering Academy’s educational focus and data sources.

Topic Area

• Retrieval-Augmented Generation (RAG)
• LLMOps
• Embedding Techniques
• Knowledge Graphs
• RLHF for Retrieval

Target Knowledge Level

• Beginner
• Intermediate
• Advanced

Content Description

This initiative will rebuild 15 of the most educational and distinct RAG pipelines from the original “all-rag-techniques” repository into modular notebooks compatible with the [AI Engineering Academy format](https://github.com/adithya-s-k/AI-Engineering.academy/blob/main/docs/RAG/01_Basic_RAG/basic_rag_scratch.ipynb).

Each notebook should:

• Use litellm as the LLM client for generation.
• Use lancedb as the local vector database for embedding storage/retrieval.
• Use an open-source embedding model from HuggingFace.
• Leverage the shared dataset from [/archives/data](https://github.com/adithya-s-k/AI-Engineering.academy/tree/main/archives/data), replacing PDF test documents with unified AI-related content.

Notebooks to create:

1. Simple RAG
2. Semantic Chunking
3. Chunk Size Selector
4. Context Enriched RAG
5. Contextual Chunk Headers
6. Document Augmentation RAG
7. Query Transformation (includes subquery + step-back prompting)
8. Reranker (LLM-based)
9. RSE (Relevant Segment Extraction)
10. Contextual Compression
11. Feedback Loop RAG
12. Adaptive RAG
13. Fusion RAG (BM25 + vectors)
14. Graph RAG (concept traversal using networkx)
15. Self-RAG (retrieval + decision via LLM)

add more if possible

Each notebook will include:

• An explanation block (Markdown)
• Inline commentary for each code block
• Evaluation code (either LLM-assisted or metric-based)
• Visualizations for similarity scores and retrieval steps (matplotlib/seaborn)

Educational Rationale

Retrieval-Augmented Generation is a cornerstone of modern LLM applications. By rebuilding these techniques from scratch:

• Learners get a transparent view into retrieval pipelines.
• They learn to decouple from opaque frameworks like LangChain or LlamaIndex.
• It provides a foundation for deploying RAG in constrained environments.
• Each variant highlights different trade-offs (speed, interpretability, relevance, adaptability), which are crucial to master in AI product engineering.

By using open-source embedding models and a fast, local vector DB like lancedb, the content aligns with privacy-first and cost-effective AI engineering practices.

Resources

• Original RAG repo: https://github.com/FareedKhan-dev/all-rag-techniques
• Reference notebook format: [Basic RAG from Scratch](https://github.com/adithya-s-k/AI-Engineering.academy/blob/main/docs/RAG/01_Basic_RAG/basic_rag_scratch.ipynb)
• Data source: [AI Engineering Academy Data](https://github.com/adithya-s-k/AI-Engineering.academy/tree/main/archives/data)
• HuggingFace Embedding Models: https://huggingface.co/models (e.g., BAAI/bge-base-en-v1.5)
• Lancedb Docs: https://lancedb.github.io/
• litellm Docs: https://docs.litellm.ai/