Helping AI Search by Both Words and Meaning: An Introduction to Hybrid Search

What frustrates me about document search is not just when it finds nothing. Sometimes it finds something, and the answer is still unusable.

Two documents might both discuss travel reimbursement, but one applies to a different group of people, or an old policy has already expired. The system gives a fluent answer, but we cannot actually use it.

Relevant content is a starting point for an answer, not a guarantee that the answer is correct.

Everyone has their own way of asking

A user asks, “How many days can I work from home?” The document says, “Employees may work from their residence for no more than two days per week.”

If the system needs users to match the document's wording every time, the experience feels awkward. They almost need to know part of the answer before they can ask for it.

Vector search helps reduce that burden. An embedding model turns text into a set of numbers, and the system looks for text whose numerical representations are close to the question's representation.

That can connect a question with a passage written differently. How well it works depends on the model, the language, and the data.

Sometimes we need the exact code, not something similar

Searching by meaning is useful, but similarity alone is not enough when a code must match exactly.

Take AF-240B and AF-2408. They look very similar. If they are part numbers, though, the parts may not be interchangeable. The same goes for iPhone 6 and iPhone 16. Similar names do not make them the same phone.

Keyword search gives the actual words and codes a direct role in finding results. Exact identifier matching still needs the right field and query setup, because some search configurations split or normalize codes.

Other tasks call for a broader match. We might search for AF-240 and want both AF-240A and AF-240B. With a suitable prefix-search setup, that can be useful too.

I do not see vector search and keyword search as an either-or choice. I would rather have them work together. That is hybrid search.

Two search methods bring another problem: ranking the results

Once both methods return results, we need to decide how to combine and order them.

Their scores do not necessarily use the same scale or mean the same thing. We cannot simply add them and assume the larger total is better. We need a common way to rank the results, whether that means adjusting the scores or combining their positions in each result list.

There are several ways to do this before passing the evidence to the AI. But ranking is only part of the problem:

• Is the information current? If two versions disagree, which one should we use?
• Is the user allowed to access this information?
• If several results have similar scores, which should come first?
• What does the user actually need, and do these results answer the question yet?

Those are basic questions a good system still has to handle.

Sometimes an exact phrase points to the wrong subject entirely. Imagine a user discussing dessert recipes and asking about “nom maeo.” In Thai, that phrase literally reads as “cat milk,” but it is also used for a dessert fragrance.

Vector search might find a passage about the fragrance used in desserts. Keyword search might find a passage about actual milk from a cat. Even if both were displayed as “99% relevant,” those hypothetical labels would not prove that the scores are comparable or that both passages fit the conversation.

Give them equal importance and hand them to the AI without resolving the context, and things could get entertaining. Not in a good way. You may have seen a similar mix-up before.

Another example: a user asks about iPhone 16, but vector search also returns information about iPhone 6 or iPhone 16 Pro because the content is similar. If the system passes all of it along without checking the model names, the AI could mix details from different phones into one answer.

Hybrid search needs careful tuning and context checks, not just two lists of results.

The information can still be incomplete

Even when the system finds the right source, we need to check whether the text reaching the model is complete enough.

A passage might include a reimbursement limit but leave out the exceptions. It might include a heading without the conditions underneath it. Or it might refer to a table that is missing from the retrieved text.

All of that can be relevant, but it is not enough to support a reliable answer.

Chunking can cause this problem. I covered it in an earlier article on chunking in RAG.

How we split documents into chunks and assemble the final context matters just as much as how we search. Finding a passage may be the first step. We may still need to include related sections, while staying within the correct scope and the user's access rights.

Difficult questions reveal the system's limits

Average scores help us see the overall picture. Difficult questions show us where the system breaks down.

I would separate test questions into groups:

• Synonyms and different ways of asking the same thing.
• Codes that look almost identical.
• Different versions of the same policy.
• Questions involving several conditions.
• Cases where the information does not exist or the user cannot access it.

Then I would follow the evidence. Did the source that supports the answer appear in the search results? Was it near the top? Was it ranked high enough to actually reach the AI?

And once the AI answers, does it stay faithful to that source? Rephrasing is fine. Changing an important condition or mixing up a detail is not.

After that, I would look at the extra time and cost per request. Better retrieval still has to make sense for the way the system is used.

If we ask something the sources cannot answer, the system needs to say there is not enough information. It should not invent an answer to fill the gap.

The takeaway

Hybrid search does not guarantee accuracy. It gives the system more than one way to find evidence.

What I want to see is a traceable path for each question: what was retrieved, what was filtered out, which conditions were enforced, and which evidence actually reached the model.