Retrieval-Augmented Generation (RAG) has become foundational for grounding AI agents in factual information. But basic RAG implementations often fall short. Let's explore advanced techniques that define RAG 2.0.
## The Limitations of Basic RAG
Traditional RAG follows a simple pattern: embed a query, find similar chunks, stuff them into a prompt. This approach suffers from:
- **Chunk boundary issues**: Relevant information split across chunks
- **Semantic gap**: Query embeddings may not match document embeddings
- **Context window waste**: Retrieved chunks may contain redundant information
- **Ranking failures**: Most similar isn't always most relevant
## Advanced Retrieval Strategies
### Hybrid Search
Combine semantic (vector) search with keyword (BM25) search. This captures both conceptual similarity and exact term matches:
```python
results = alpha * vector_results + (1 - alpha) * keyword_results
```
Typical alpha values range from 0.5 to 0.7 depending on query types.
### Query Expansion
Transform user queries before retrieval:
- **HyDE (Hypothetical Document Embeddings)**: Generate a hypothetical answer, then search using its embedding
- **Multi-query**: Rephrase the query multiple ways and aggregate results
- **Query decomposition**: Break complex queries into sub-queries
### Contextual Compression
After retrieval, compress chunks to extract only relevant portions:
- Remove redundant sentences
- Highlight query-specific passages
- Merge overlapping information
### Reranking
Apply a cross-encoder model to rerank initial results:
```python
from sentence_transformers import CrossEncoder
reranker = CrossEncoder('cross-encoder/ms-marco-MiniLM-L-6-v2')
scores = reranker.predict([(query, doc) for doc in documents])
```
Cross-encoders are slower but significantly more accurate than bi-encoders.
## Agentic RAG
The most powerful advancement is agentic RAG, where the agent controls retrieval:
1. Agent analyzes query and plans retrieval strategy
2. Agent executes searches, evaluates results
3. Agent decides if more retrieval is needed
4. Agent synthesizes information into response
This loop enables iterative refinement that static pipelines cannot achieve.
## Chunking Strategies
Improved chunking also drives better retrieval:
- **Semantic chunking**: Split on topic boundaries, not character counts
- **Parent-child chunks**: Retrieve small chunks, return their parent context
- **Sliding windows**: Overlapping chunks prevent boundary issues
## Evaluation Metrics
Measure RAG performance with:
- **Retrieval precision/recall**: Are relevant documents retrieved?
- **Answer faithfulness**: Does the response match retrieved content?
- **Answer relevance**: Does the response address the query?
Tools like RAGAS provide automated evaluation frameworks.
## Implementation Tips
1. Start with hybrid search—it's low-effort, high-impact
2. Add reranking before complex compression
3. Monitor retrieval metrics continuously
4. Fine-tune embeddings on your domain data
RAG 2.0 isn't a single technique but a combination of improvements that compound to dramatically enhance AI agent capabilities.