Hybrid Search Explained: Why Vector Search Still Needs Keywords

“Which air purifier models support the AF-240B replacement filter?”

That question needs evidence about AF-240B, not another filter with a similar name. A result can be about the right topic and still be about the wrong part.

Now consider: “How do I get my air purifier working properly again?” The answer might sit under “Replacing the filter,” without using the wording in the question.

These are different retrieval problems. Hybrid search combines semantic and keyword retrieval so the system has a better chance of finding both the intended meaning and the specific terms that matter. It does not make every result correct by default.

The short version

• Vector search helps with paraphrases and related meaning.
• Keyword search helps with terms, names, and identifiers present in the content.
• Hybrid search combines their candidate lists and ranks the results.
• Exact identifiers need explicit matching rules, not just relevance scores.
• Permissions, versions, chunk quality, and evaluation remain separate responsibilities.

What vector search does well

An embedding model turns text into a vector: a list of numbers representing learned features of the text. Vector search compares the question's representation with stored content representations using a chosen similarity measure.

Suppose a handbook says:

“Employees may work from their residence up to two days per week.”

A user asks:

“How many days can I work from home?”

A suitable embedding model may connect those phrases even though the wording differs. This is useful when people do not know the vocabulary used in a document.

“May” matters here. Language coverage, model quality, and the indexed content all affect the result.

Embedding generation is also separate from answer generation. Vector retrieval does not inherently ask a generative LLM to read the whole collection for every query. Document vectors are commonly prepared ahead of time, then compared with a query vector at search time.

Where similar meaning becomes a problem

Identifiers are not interchangeable

Compare AF-240B with AF-2408, or X100 with X100 Pro.

Their surrounding descriptions could be almost identical. Yet the products may have different compatibility requirements. An embedding model can learn patterns involving identifiers, but vector similarity is not an exact-match guarantee.

The same issue applies to invoice numbers, serial numbers, API names, error codes, contract clauses, dates, and software versions such as 2.10 versus 2.1.

If a value must match, I would make that requirement explicit rather than leave it to semantic ranking.

Related policies can have different answers

Two travel reimbursement policies can be semantically close while applying to different groups. One may cover employees, another contractors. One may be current, another expired.

Retrieval has to respect who is asking, when the policy applies, and which version is authoritative. Keyword retrieval can contribute useful terms, but metadata and filtering still need to enforce those boundaries.

The right passage may not reach the model

Systems usually pass only a limited set of results to the answering model. A relevant passage outside that cutoff is missing evidence, even if it exists in the collection.

Chunk boundaries, embedding quality, query wording, language, and candidate depth affect which passages survive. A stronger answer model cannot reliably compensate for evidence it never receives.

What keyword search adds

Keyword, or lexical, search works with terms found in the indexed text. Search engines commonly use an inverted index and ranking methods such as BM25. BM25 considers factors including term frequency, how uncommon a term is across the collection, and document length.

This gives a query term like AF-240B a direct role in retrieval. But lexical search is not automatically exact matching.

A text analyzer may split punctuation, lowercase text, or turn an identifier into several tokens. If a complete product code must match, use a field and query type that preserve and enforce the intended value. Define any normalization rules deliberately.

Keyword search has its own weakness: users may describe a need with words that never appear in the document. “Time off to look after my child” might need a passage labeled “Family care leave.” Semantic retrieval can help bridge that wording difference.

Neither method replaces the other in every collection.

A practical hybrid retrieval flow

At a conceptual level:

Question + user context + document constraints
                    |
           Authorized search scope
                    |
         +----------+----------+
         |                     |
    Vector search         Keyword / BM25
         |                     |
         +----------+----------+
                    |
        Fuse rankings and deduplicate IDs
                    |
           Optional reranking
                    |
  Validate access, versions, and evidence coverage
                    |
      Supply permitted evidence to the LLM

This is an explanatory design, not NEXT4I's internal architecture. Access control must cover every content-fetching path, including any external reranker. Hiding an answer afterward is not a substitute for keeping unauthorized text out of model context.

For the filter question, vector retrieval might find maintenance guidance while lexical retrieval finds passages containing the part code. The combined result still needs evidence of model compatibility. Merely mentioning the code is not enough.

Combining results without being misled by scores

BM25 and vector similarity scores do not share a universal scale. Adding them directly can let the numerically larger scale dominate, without proving that its results are more relevant.

Normalize and weight scores

One approach transforms each source's scores into a chosen scale, then combines them with weights. This offers control, but score distributions can vary across questions and collections.

Increasing the lexical weight for code-heavy queries may be useful. It still is not equivalent to enforcing a mandatory identifier constraint.

Fuse ranks with RRF

Reciprocal Rank Fusion uses positions in ranked lists rather than comparing their raw scores:

RRF(document) = sum of 1 / (c + rank(document))

Ranks start at 1. A document absent from a list receives no contribution from that list. The constant c controls how sharply rank positions affect the contribution; it is separate from the number of candidates retrieved.

A document ranked well by multiple retrievers can rise in the fused list. RRF does not check truth, policy applicability, or access rights. It is also different from PageRank, which uses a web link graph.

Reranking is another option: a model evaluates the relationship between the question and candidate passages. It can help reorder close results, at the cost of additional work and latency. It cannot recover a passage that never entered the candidate set.

A brief note on vector precision

Precision affects resource use, but it is not the main reason to use hybrid retrieval.

A float32 value uses four bytes and has roughly seven significant decimal digits of precision, not seven fixed places after the decimal point. At the same dimensionality, float16 and bfloat16 use two bytes per value. Quantizing values to int8 uses one byte per value.

Those ratios describe vector values, not the total database footprint. Indexes, metadata, quantization parameters, and retained original vectors also take space. The two 16-bit formats have different precision and range characteristics.

Quality loss and speed gains depend on the model, collection, hardware, and search configuration. I would not attach a universal accuracy-loss percentage to quantization. Check whether important evidence remains retrievable at the actual cutoff you use.

Evaluate the whole evidence path

Compare vector-only, keyword-only, and hybrid retrieval using the same labeled questions. Keep separate results for paraphrases, exact identifiers, policy conditions, and other query types.

• Recall@k: What fraction of known relevant evidence appears in the first k results?
• Hit@k: Does at least one relevant item appear? This is not the same as recall when multiple items matter.
• MRR or nDCG: Is useful evidence near the top, and does the ordering reflect relevance?
• Evidence completeness: Are the required conditions and exceptions included?
• Answer correctness and grounding: Does the answer follow from the retrieved text, with supporting citations?
• Latency and cost: Does the improvement justify the extra retrieval and ranking work?
• Access and version handling: Does forbidden or obsolete content stay out?

Include questions with no answer in the collection and questions whose supporting documents the user cannot access. Returning nothing can be the correct outcome.

Better retrieval metrics do not automatically establish better final answers. Test both.

The goal is evidence, not a smarter-looking stack

Vector search helps when users and documents use different words. Keyword search helps when specific terms matter. Hybrid search gives the system two ways to find candidates, then asks you to combine them responsibly.

I would start with questions the system currently gets wrong and trace where their evidence disappears: retrieval, fusion, or context assembly. That is a more useful starting point than changing the answer model without inspecting what it can read.

Further reading

• Microsoft Learn: Reciprocal Rank Fusion
• Elastic: Term queries and text analysis
• Qdrant: Quantization