Search has been "good enough" for two decades because keyword inverted-index systems (Lucene, Elasticsearch, Postgres tsvector) are fast, predictable, and explainable. AI doesn't replace any of that — it adds two new capabilities: matching on meaning instead of words, and asking clarifying questions when the query is ambiguous.
Used together with classical search, these turn a search box from "exact-match retrieval" into "find what the user actually wanted." Used naively, they become a slower keyword search with worse explainability. The difference is design.
Three distinct upgrades, often deployed together:
These can be deployed independently. A team can add #1 alone (just better retrieval), or jump straight to #3 (a chatbot over their docs). Most products converge on all three over time.
┌──────────────────────────────┐
query ─▶│ Query understanding │
│ (rewrite, decompose, expand)│
└──────────┬───────────────────┘
│
┌──────────┴────────────┐
▼ ▼
┌──────────┐ ┌─────────────┐
│ BM25 │ │ Dense │
│ search │ │ retrieval │
│ (Lucene) │ │ (HNSW) │
└────┬─────┘ └──────┬──────┘
│ │
└────────────┬───────────┘
▼
┌──────────────┐
│ RRF fusion │
└──────┬───────┘
▼
┌──────────────┐
│ Rerank │
│ (cross-enc) │
└──────┬───────┘
▼
┌──────────────┐
│ Answer synth │
│ + citations │
└──────────────┘
Each stage is independently swappable. This pipeline is what most production "AI search" looks like under the hood; calling it AI search overstates the AI's role and understates how much classical IR is still doing the work.
See HybridRetrieval for this wiki's own implementation.
The query the user typed is rarely the right query for retrieval. Three useful transformations:
Don't apply all three. Each adds latency. Profile which transformation actually improves retrieval recall on your eval set; keep that one.
BM25 and dense retrieval miss different things. BM25 misses paraphrases ("car" vs "automobile"). Dense retrieval misses exact strings ("error code 451"). Run both, fuse with reciprocal rank fusion (RRF) or learned-to-rank.
RRF in five lines:
def rrf(*ranked_lists, k=60):
scores = {}
for hits in ranked_lists:
for rank, hit in enumerate(hits):
scores[hit.id] = scores.get(hit.id, 0) + 1 / (k + rank + 1)
return sorted(scores.items(), key=lambda x: -x[1])
This single change typically adds 5–15 points of retrieval recall. The most reliable upgrade in the entire stack.
Top 50 from retrieval is too noisy to feed directly to an LLM. A cross-encoder reranker (Cohere Rerank, BGE-reranker-large, or ms-marco-MiniLM-L-12-v2) reranks the candidates by query-document relevance and you keep the top 5–10.
The "AI search experience" — generative answer with citations — is RAG (RagImplementationPatterns). The search-specific concerns:
Did-you-mean / typo correction — classical, still useful. Pre-LLM or with a tiny model.
Faceted filtering — let users filter by metadata after search. AI search degrades without this; users feel like they lost control.
Query suggestions / autocomplete — show popular queries as user types. Reduces ambiguity by directing toward known-good queries.
Re-search with feedback — when no results match, ask the user to clarify. "Did you mean documentation, troubleshooting, or pricing?" beats an empty results page.
Recent searches and saved searches — personalisation that's easy to implement and hard to do badly.
Pure dense retrieval breaks on rare exact terms. Product SKUs, error codes, names with unusual spelling. BM25 catches these; pure-dense systems miss. Always include BM25 in the fusion.
Embedding drift. You silently swapped the embedding model. Old vectors are now noise relative to new queries. Pin embedding model versions in deployment config; reindex on planned upgrades.
Stale index. New content takes 24h to be searchable because indexing is batch. Users don't notice for days. Make indexing latency a tracked metric, alert above thresholds.
Filter blindness. User filtered by date range; search ignored the filter. Pre-filter when filters are selective; verify the filter is actually applied at every layer.
Ranking opaqueness. Users can't understand why result A came before B. AI search is harder to debug than keyword search. Build an internal "explain" tool — for a given query and result, show which retriever surfaced it, what its similarity score was, what the reranker did.
| Need | Buy | Build | Hybrid |
|---|---|---|---|
| Generic web search inside an app | Algolia, Vespa Cloud, Elastic | — | — |
| Internal knowledge search | Glean, Coveo, Pinecone Inference | Postgres + pgvector + BM25 | Use Elastic + your own LLM layer |
| Customer-facing AI assistant | Cohere, Voyage RAG | Hybrid retrieval + open LLM | Most production systems |
| Domain-specific (legal, medical) | Specialist vendors | Custom embedding fine-tune + curated retrieval | The interesting middle |
The "build" column is increasingly viable in 2026. Open-source embedding models (BGE-M3, Voyage open variants), Postgres-native vector search, and self-hostable rerankers cover most needs. Pick "buy" only if speed-to-market dominates control.
Your search eval set has 100–500 query-document relevance labels. Track:
Run weekly on your eval set; deploy changes only when metrics improve. See HybridRetrieval for the full eval pattern.