Duplicate and contradicting knowledge articles: how to find and fix them

Why do duplicate articles appear?

Almost nobody sets out to write a duplicate. They are a side effect of normal work in a knowledge base with more than a few hundred articles:

• The original can't be found. An author searches, gets nothing useful because the title doesn't match how they think of the topic, and writes a new article. The first article was an unfindable answer; the fix created a second one.
• Several teams, one topic. Voice, chat, retail and complaints teams each keep their own version of "how to change a direct debit date".
• Migrations and imports. Content moves in from document libraries, intranets and older tools, often with the same procedure in a policy PDF, a quick reference guide and an FAQ.
• Variants. A regional or product variant starts as a copy and is then edited on its own.
• Changes applied in one place. A price or rule changes, and the update reaches the article someone remembered, not the two others that also mention it.

The last cause is how a harmless duplicate becomes a contradiction. Two copies that agreed on the day they were written are edited by different people at different times. Within a year they give different answers.

What is the difference between a duplicate and a contradiction?

A duplicate overlaps: two articles cover the same ground. A contradiction disagrees: two articles give different answers to the same question. They need different fixes, so it helps to keep them apart.

Two articles can be similar without contradicting, and they can contradict without being very similar. A short FAQ and a long procedure might share one sentence, and that sentence might be the one that disagrees. That is why the two problems need separate checks.

Why are duplicates worse with AI answers?

With a human searching, duplicates are an annoyance. The agent sees two results, opens both, and often notices that they differ. AI answers remove that moment of doubt.

Most AI answer and agent-assist tools retrieve a handful of passages that look most relevant to the question and generate an answer from them. When two articles cover the same topic, retrieval scores them as near-equals. Which one wins can depend on small differences in wording, so it can be the copy that wasn't updated. The generated answer then reads as fluent and certain. If both copies are retrieved and they disagree, the answer may pick one, blend them, or hedge, and the agent or customer has no way to see that a conflict was there at all.

Better models don't fix this, because the problem is in the content; it is one of the failures we separate out in can AI answers be hallucination-free? The answer can only be as consistent as the articles it draws on. Removing duplicates and resolving contradictions is one of the most effective things you can do before switching on AI answers, and one of the most important to keep doing after. It belongs with the rest of the work described in knowledge base content health.

How do you find duplicates?

Title matching catches only the obvious cases. Real duplicates usually have different titles, because the second author didn't find the first. A reliable approach compares content:

1. Compare within a knowledge area. Compare each article with every other live article in the same area: published, in review, proposed and draft. Leave out archived and superseded articles, which no longer reach agents.
2. Rank candidates by meaning. Use semantic similarity to put the most likely overlaps first, so the closest pairs are always examined.
3. Screen, then examine. A quick screen for overlapping content, then a detailed comparison of the pairs that pass, keeps the work affordable across a large collection.
4. Propose an action. For each real overlap, propose a merge with a primary article, or record why the overlap is intentional.

In Knowledge Quest this is the similarity check. Each run reports how many other articles each article was compared with and, where some weren't, why. A clean result therefore means "compared and clean", not "skipped".

How do you find contradictions?

Contradictions hide in specifics: a waiting period, a fee, an eligibility rule, a step order. Finding them means comparing claims, not topics.

The contradictions check in Knowledge Quest extracts the key claims from each article, screens pairs of articles for claims that might conflict, and runs a full comparison on the pairs that do. Each finding names what conflicts (for example, a 30-day rule in one article against a 14-day rule in another), quotes the statement from each article, and explains the reasoning so a reviewer can verify it.

Two related checks fill the remaining gaps. Validity compares an article with your authoritative source documents, which catches the case where every article agrees with each other and all of them are out of date. Gap detection from conversations catches the contradictions customers notice first: repeat contacts where the customer says "but the last person told me something different".

How do you decide what to merge, keep or retire?

Every finding should end in a decision, and the decision should be recorded. These patterns cover most cases:

• True duplicate, both useful. Merge into one article. Pick the clearer, more current one as the lead, and bring across anything the other covers that it doesn't.
• True duplicate, one redundant. Archive the redundant one rather than deleting it, so its history stays available.
• Deliberate overlap. Different region, product or audience. Keep both, make the difference visible in the title and first line, and dismiss the finding with the reason.
• One article is wrong. Correct it to match the right one and the source document, then check whether other articles repeat the error.
• Both are right in different situations. Rewrite both to state the boundary, for example "for purchases over $75" and "for purchases under $75".
• Nobody knows which is right. Escalate to the policy owner. Until it is settled, the contradiction is a risk to log, not a finding to dismiss.

Resolve both sides of a contradiction. Fixing only the article you happened to open leaves the other one waiting to cause the same problem.

How does combining articles work?

Merging is where most knowledge teams hesitate, because getting it wrong removes an answer agents rely on. A safe merge drafts first and switches over only when the result is approved. In Knowledge Quest it works like this:

1. Start from a similarity finding, or open the article you want to lead with and choose to combine it with up to four other articles from the same knowledge area.
2. Confirm the title for the combined article, which is kept exactly as you enter it, and optionally describe what it should cover.
3. The combined article is drafted for review. The originals stay live while it is reviewed, each marked as pending a merge, and the combined article lists which articles publishing it will supersede.
4. Publish the combined article and the originals are marked superseded. They are retired, not deleted. Archive the combined article instead and the originals are left exactly as they were.

Articles already in a pending merge can't join another one, and they aren't flagged as duplicates of each other while the merge is in review, because the merge already deals with that overlap. If a merge can't go ahead, for example because one source was archived in the meantime, nothing changes and you can try again.

How do you stop duplicates coming back?

• Make the existing answer findable. Retitle articles in the words customers and agents use. Many duplicates start as an unfindable answer.
• Link or update before you create. When a gap is found, check whether an existing article nearly covers it. In Knowledge Quest each gap recommendation shows how closely candidate articles match the customer's need, which steers people to update instead of writing another copy.
• Check drafts, not only published articles. Run similarity and contradictions before a new article is published.
• Fix the taxonomy. Overlapping categories invite overlapping articles. A taxonomy built from how customers ask helps.
• Re-run after migrations and policy changes. These are the two moments duplicates and contradictions arrive in bulk.

For the wider method, see knowledge gap analysis, which treats inconsistent answers as one of the four kinds of knowledge gap. Knowledge Quest finds duplicates and contradictions across the knowledge base you already run, drafts the merged article for review, and sends the approved version to your knowledge base by live sync or export. You keep your tools; they get one answer per question.