Why don't views and ratings measure article impact?
Most knowledge bases report activity: views, searches that led to the article, thumbs up and down, and sometimes time on page. These are easy to collect, and each one measures the article's popularity among the people who found it.
None of them show what happened to the customer. A heavily viewed article might be popular because it is the only result for a common search, while giving an answer that sends half its readers' customers back a week later. A low-rated article might be correct and simply deliver unwelcome news, such as a fee that can't be waived. Ratings come from the few agents who bother to click, under handle-time pressure, often before they know whether the answer worked.
Activity data also misses the conversations where nobody opened the article at all: the agent answered from memory, asked a colleague or guessed. Those are the contacts an article is meant to change, and they never appear in its view count.
What should a good knowledge article change?
An article exists to change the outcome of conversations on its topic. That gives you a short list of measures, all taken on the conversations about the topic, whether or not anyone opened the article:
| Measure | What a good article does to it | Watch out for |
|---|---|---|
| Re-contact | Falls, because customers get a complete, correct answer first time | Topics where a second contact is a normal step, such as sending documents |
| Resolution | Rises, because agents can resolve the issue without escalating | Resolution limited by policy or system access, not knowledge |
| Transfers and escalations | Fall, because front-line agents can answer | Routing changes made at the same time |
| Handle time | Falls, or stops varying so widely between agents | A better article can lengthen calls by adding a step that was being skipped |
| Unanswered share | Falls, because more of the topic's questions now have an answer | Questions on the topic that the article doesn't address |
| Sentiment | Improves when the answer is clear and consistent | Sentiment driven by the outcome, such as a declined claim |
Two notes on re-contact. It can be measured as the same customer returning within a set window, which needs a reliable customer identifier across channels, or by assessing each conversation for whether follow-up is still needed. Either works if you use it consistently; the knowledge management glossary defines both. Second, contact volume on the topic only drops if self-service also improves. For assisted contacts, a good article usually changes how conversations end, not how many arrive.
How do you set up a before-and-after comparison?
- Define the topic. Use your taxonomy category, or the specific intents the article was written for if the category is broader. Measuring a whole category for an article that answers one question in it will dilute the effect.
- Record the publish date and any later material edits. The comparison is anchored to these.
- Pick equal windows. Four weeks before against four weeks after is a sensible default. Equal lengths with the same weekdays avoid weekly patterns distorting the result.
- Exclude obvious distortions. Leave out public holidays, outage days and the week of a major release from both windows, or note them clearly.
- Leave a short gap after publishing. Agents need a few days to find and trust a new article. Starting the "after" window a week later gives a fairer picture.
- Compare rates, not counts. Report re-contact and resolution as percentages of the topic's conversations, with the count beside them so readers can judge the sample.
How do you separate the article's effect from everything else?
Plenty of things move contact centre numbers. A pricing change, a seasonal peak, a new cohort of agents or a fixed system bug can all shift a topic's outcomes in the same weeks as your article. Three habits make the comparison more honest:
- Use a comparison topic. Pick a similar topic whose knowledge didn't change and measure it over the same windows. If both improve by the same amount, something else drove it. If only the article's topic improves, the article is the likely cause.
- Check the mix. If the kinds of questions within the topic changed between windows, outcomes can move without the article doing anything. Look at the intents, not just the category total.
- Read a sample. Ten conversations from the "after" window will tell you whether agents are actually giving the new answer. Numbers that improve while agents still give the old answer point to another cause.
This is evidence, not proof. For most knowledge decisions, a consistent improvement on the article's topic that doesn't appear on a comparison topic is enough to act on.
How long should you wait before judging an article?
Long enough to have a meaningful number of conversations in each window. On a high-volume topic, two to four weeks is usually plenty. On a topic with a few dozen contacts a month, a single month is too small to read: a handful of re-contacts either way will swing the rate. Extend the windows, or combine several related articles into one measurement.
Re-contact also needs time to show up. If you measure customers returning within seven days, the last week of the "after" window isn't complete until a week after it ends.
What if the numbers don't move?
An article with no measurable effect is useful information. Work through the likely causes in order:
- Agents can't find it. Check whether its title and wording match how customers phrase the question. Our guide to failed searches vs knowledge gaps covers findability.
- Agents don't trust it. If another article says something different, agents fall back on memory. Look for duplicates and contradictions on the topic.
- It answers a different question. Read conversations from the "after" window. If customers still ask something the article doesn't cover, the gap was only partly closed.
- Knowledge wasn't the problem. Some outcomes are fixed by policy, system access or process. If the article is found, trusted and correct, and nothing changes, hand the topic to whoever owns that process.
What belongs on a knowledge impact scorecard?
Keep two levels. For individual articles written to close a gap, report the before-and-after comparison above for re-contact, resolution and the unanswered share on the topic. For the knowledge programme as a whole, report:
- Coverage rate: the share of customer intents that an article answers, by topic and over time.
- Gaps identified and gaps closed each period, so the backlog's direction is visible.
- Time to close: days from a gap being detected to a published fix.
- Articles updated, which shows whether maintenance keeps pace with change.
- Outcome trend on closed gaps, so the programme is judged on results. If you priced gaps with your own cost per contact, as in the cost of a knowledge gap, apply the same basis here.
A programme that closes many gaps without improving outcomes is busy without being effective. The knowledge gap analysis guide covers how to choose which gaps to close first.
How does Knowledge Quest help measure article impact?
Knowledge Quest reads every conversation, extracts each customer's intents and checks them against your knowledge base, recording a match to an article or a gap. Each conversation is assessed for resolution, sentiment and whether follow-up is needed, which Knowledge Quest reports as re-contact rate. Its briefs track coverage rate, gaps identified, gaps closed and articles updated for each period.
Kai, the built-in knowledge analyst, can compare resolution, re-contact, sentiment and volume by topic between two periods, show where open gaps sit by topic, chart a topic's trend over several weeks and link to the conversations behind its answers. That gives you the topic-level evidence for the method above without exporting data. Your knowledge base stays where it is; you find out whether what you put in it worked.
Frequently asked questions
What is the single best measure of article impact?
Re-contact on the article's topic. If customers stop needing a second contact about the same question, the answer worked. Pair it with the share of the topic's questions that are still unanswered, so you can tell whether the article covers the whole topic or only part of it.
Can we measure impact on self-service articles the same way?
Partly. Assisted contact volume on the topic should fall if the self-service article works, so the trend in contacts on that topic is the main signal. Watch for customers who read the article and contact you anyway: those conversations show what the article left out.
Should we measure every article?
No. Measure articles written or changed to close a specific gap, and articles on high-volume or high-risk topics. For the rest, coverage and content health checks are enough. Measurement effort should follow the cost of the topic.
How do we measure handle time impact if our platform only reports it by queue?
Use the queue that carries most of the topic as a proxy, and say so in the report. A queue mixes many topics, so handle time changes there are weak evidence on their own. Rely more on re-contact and resolution for the topic.
Is knowledge article impact the same as deflection?
No. Deflection counts contacts that self-service may have prevented, which is hard to observe directly. Impact on assisted contacts is easier to measure, because the conversations exist and show whether the answer worked.