What does a failed search report actually measure?
Most knowledge management systems include some form of gap reporting based on search. The details vary, but the inputs are usually the same:
- Zero-result searches: queries that returned nothing.
- No-click searches: queries that returned results, none of which were opened.
- Abandoned or reformulated searches: the agent searched, gave up, or tried three different phrasings.
- Low ratings: an article was opened and marked unhelpful.
Every one of these measures agent search behaviour. That is useful information, but it is a different thing from customer need. A search log is a record of what agents typed into one tool. It is silent about everything else that happened on the call.
For findability problems, search data is excellent. If agents keep searching "NCB letter" and the article is titled "Proof of no-claim bonus history", the log shows you the mismatch and the fix is a better title. The trouble starts when the same report is treated as the list of knowledge gaps.
Which knowledge gaps never appear in a search log?
For a gap to show up in a failed search report, an agent has to recognise they need help, choose the knowledge base over every other option, type a query, and get a poor result. Each of those steps filters out real gaps:
- The agent thinks they know. Experienced agents answer from memory. If their memory reflects last year's policy, the customer gets a wrong answer and nothing is logged.
- The agent asks a person. Team leaders, the desk next door and the team chat channel absorb a large share of hard questions. None of it reaches the search log.
- The agent has given up on the knowledge base. When people learn a topic isn't covered, they stop searching for it. The more obvious the gap, the less it is searched.
- The agent guesses. Under handle-time pressure, a plausible answer can feel safer than a long hold. The guess generates no search and often a repeat contact.
- The search "succeeds" on the wrong article. A query matches something nearby, the agent opens it, and the report records a success. This is how stale and contradictory articles hide.
- Nobody searches at all. Emails answered from templates, chats handled by bots, and customers who abandon self-service without contacting you never touch the agent search box.
Gaps in all six groups are in your conversations. Customers still asked the question, and the transcript or ticket records what happened next: the hold, the transfer, the callback promise, the wrong answer, the second contact.
Why is search-log gap data biased?
Search-based gap reports don't just miss gaps at random. They miss them in a consistent direction, which makes them misleading as well as incomplete.
New agents search more than experienced ones, so the report over-represents the questions that confuse new starters and under-represents the subtle, high-stakes questions that experienced agents handle on instinct. Topics with distinctive keywords ("refund", "password") are searched more than vague ones ("why did my bill change"). Findability problems produce lots of failed searches, while a genuinely missing article produces fewer over time as agents stop looking.
The result is a mismatch between search volume and conversation volume. In an illustrative example, a fictional insurer might see 9 failed searches about whether the excess is waived for storm claims in a month where 1,140 customers asked exactly that. Ranked by failed searches, the topic sits near the bottom of the list. Ranked by what customers asked, it is the highest-impact gap they have.
Is a successful search the same as a good answer?
No, and this is the gap that search reporting is least able to see. A search counts as successful when an article is returned and opened. Nothing in the log says whether the article was correct, current or consistent with the other three articles on the same subject.
Stale content is the clearest case. An article describing last year's claim form will be found, opened and followed, and every one of those searches looks like a success. The cost shows up later, as a callback when the form is rejected. Contradictions work the same way: two articles give different answers, both are "found", and the customer's answer depends on which one the agent opened.
Finding these needs two things a search log can't provide: the outcome of the conversation (did it resolve, or did the customer come back?) and a check of the content itself for validity, duplicates and contradictions, which is the work of knowledge base content health.
Does AI search fix the failed search problem?
AI search and semantic search are good at matching meaning instead of keywords, which genuinely helps findability. They also tend to return something for almost every query, and that changes the gap report in an unhelpful way.
When search always returns a best match, zero-result queries largely disappear. The failed search report gets quieter, which looks like progress. The underlying gaps are still there: the closest match to "is the excess waived for a storm claim" may be a general article about excesses that doesn't answer the question. AI summaries built on top of that article can make a partial answer sound complete, a problem we take apart in why AI search won't fix a legacy knowledge base. Better search over the same content makes the content easier to reach. It doesn't tell you what's missing from it.
How do you find knowledge gaps from conversations instead?
The method reverses the direction of the search report. Instead of starting from what agents typed, start from what customers said:
- Gather conversations from every channel: call transcripts, chats, emails and tickets for a representative period.
- Extract each customer's intent in plain language, including secondary questions raised during the same contact.
- Match every intent against the knowledge base and record whether an article fully answers it, partly answers it or doesn't answer it.
- Look at what happened on the contact: transfers, long holds, callbacks, unresolved outcomes and negative sentiment. These separate expensive gaps from harmless ones.
- Rank by impact, meaning volume weighted by how difficult the conversations were, and decide whether each gap needs a new article, an update, a better title, or nothing.
Done by hand, this means reading and coding hundreds of conversations, which is why most teams fall back on the search report. Done automatically, it covers every conversation and repeats daily. Our knowledge gap analysis guide walks through each step in more detail.
How should search data and conversation data work together?
Keep both, and give each the job it is good at. Put the two signals side by side for each topic and the pattern tells you what kind of problem you have:
| Customers ask it | Agents search it | What it usually means |
|---|---|---|
| Often | Rarely | Hidden gap. Agents answer from memory, ask colleagues or have given up. Usually the most expensive group. |
| Often | Often, with poor results | Visible gap. Missing or unfindable content. The search log already shows it. |
| Often | Often, with results opened | Check the content. Searches succeed, so look at outcomes. Repeat contacts here point to stale or contradictory articles. |
| Rarely | Often | Training or navigation. Agents are unsure about something customers seldom raise. Consider a quick reference or coaching. |
The first row is the one a search-only approach can never see, and it is where most of the cost sits.
What should you report instead of failed searches?
Move the headline measures from agent behaviour to customer demand. A simple funnel works well: conversations read, customer intents extracted, intents matched to an article, and intents left as gaps. Alongside it, track:
- Coverage rate: the share of customer intents an article answers, by topic.
- Size of open gaps: the conversations affected by everything not yet fixed, weighted by difficulty. Multiply by your own cost per contact for an estimated monthly figure, so the backlog has a number on it.
- Time to close: how long a gap stays open from first detection to a published fix.
- Outcome after the fix: transfers, callbacks and repeat contacts on the topic before and after the article changed. Measuring knowledge article impact shows how to make that comparison fair.
Keep the failed search report as a findability tool, reviewed alongside these measures. It will tell you which good answers need better titles. The conversation data tells you which answers don't exist yet.
This is what Knowledge Quest does on top of the platforms you already run. It reads every conversation, finds the gaps, ranks them by impact and drafts the fixes for your existing knowledge base, with every finding linked to the conversations behind it.
Frequently asked questions
Are zero-result searches useless?
No. They are a good signal for findability: missing synonyms, jargon, misspellings and article titles that don't match how people ask. They are a poor signal for coverage, because they only include questions someone searched for. Keep the report, and stop treating it as your gap list.
What is a knowledge coverage rate?
It is the share of customer intents, taken from conversations, that a knowledge article answers. Track it by topic and over time. There is no universal target, because it depends on how complex your products are, so the useful measure is the trend and which topics are dragging it down.
How many conversations do you need to find knowledge gaps?
Enough to cover a normal cycle of demand. A month of conversations across all channels is a sensible starting window, because it includes billing cycles and weekly patterns. Smaller operations may need longer to see lower-volume topics clearly.
Do we need to replace our knowledge management system to do this?
No. Gap detection from conversations works alongside the knowledge base you already have. It reads conversations from your contact centre platform or service desk and sends findings and drafts to your knowledge base, by live sync where supported or by import and export.