Created this visual to break down 3 key techniques we can use to improve search relevance and retrieval quality:

1️⃣ Query Rewriting
Use an LLM to rewrite messy or ambiguous prompts into optimized ones before passing them to the retriever. This improves the likelihood of relevant results.

2️⃣ Named Entity Recognition (NER)
Extract specific entities (e.g. names, books, locations, dates) using tools like GLiNER. This enables more precise filtering and targeted retrieval.

3️⃣ Hypothetical Document Embeddings (HyDE)
Instead of matching prompt-to-document, we generate a “hypothetical answer document” from the query using an LLM. We then embed that document for retrieval—matching documents to documents.
✅ Improves relevance
⚠️ Comes with additional latency and cost

🔍 These methods are pushing the boundaries of how we think about semantic search and retrieval-augmented generation (RAG).

🧠 Curious to hear: Which of these have you used or are you most excited about?

#AI #LLM #RAG #SemanticSearch #QueryParsing #LLMEngineering #NLP #AIProduct