Knowledge base freshness· Quality and maintenance
Knowledge Base Freshness — Keeping a Chatbot's Sources Current
Quick answer: A chatbot that answers from your documents is only as current as the last time those documents were read in. Freshness goes wrong in three ways: a changed page is not re-read for days, a deleted page stays searchable, or a page is simply never reviewed. The bot then states an old price or policy fluently, with no sign that it is out of date.
Freshness is not the model's knowledge cutoff
Two separate things can be old. The model was trained on text up to some date and knows nothing newer unless it is given new text. The knowledge base is the content you supply at question time. A chatbot built on retrieval-augmented generation pulls passages from your content and asks the model to answer from them, so for questions about your business the knowledge base matters far more than the training date. If your return window changed last week and the bot quotes the old one, the model is not out of date; your index is.
The three ways content goes stale
| Failure | What happens | Typical cause |
|---|---|---|
| Late update | The page changed, but the bot still holds the old copy | Scheduled re-sync not yet run; manual upload forgotten |
| Missed deletion | The page was removed, but its text is still searchable | Sync adds and updates content but does not remove it |
| Unreviewed page | The page never changed and nobody checked it is still true | No owner, no review date |
Late update. Many platforms re-read website sources on a schedule rather than instantly. Intercom's help article on syncing public URLs says automatic re-syncs usually run weekly depending on source size, move to a 14-day schedule for very large or JavaScript-heavy sites, and pause once a source has not been used in a Fin answer for 90 days (while the old pages stay available). A change made the day after a sync can therefore wait close to a week.
Missed deletion. Removing a page from your website does not always remove it from the bot. Microsoft's documentation for Azure AI Search says an indexer on Azure Storage detects new and changed blobs through their last-modified timestamp but does not track deleted ones, so without a deletion strategy the search documents for deleted files remain in the index as orphans. Check how your own platform treats removed pages rather than assuming.
Unreviewed page. A page that never changed can still be wrong: a competitor comparison, a shipping carrier you no longer use. Only a review date and an owner catch this.
How freshness is measured
There is no standard score. Teams track a few plain numbers:
- Update lag: time from a content change to the bot using it. Worst case is roughly the sync interval.
- Age of oldest reviewed date: the share of pages not reviewed within a set period, such as 90 or 180 days.
- Orphan count: pages the bot can still cite that no longer exist on your site.
- Stale-answer rate: the share of a sampled set of answers that contradict current content, found by reading them or by re-running a saved golden dataset.
Different content deserves different limits. Prices, stock and opening hours may need same-day updates; a how-to article may be fine at six months.
Worked example (invented numbers)
A shop has 50 help articles indexed with a weekly re-sync. On a Tuesday the team changes the returns window from 30 days to 14 days, one day after the last sync, so the next re-sync is six days away. The bot gets about 40 returns questions a day: 6 × 40 = 240 answers quote the old 30-day window before the next sync. A manual re-sync after the edit cuts that window to minutes.
A review audit finds 12 of the 50 articles have no review in 180 days, which is 12 ÷ 50 = 24%. Neither number is a benchmark; the method is what carries over.
FAQ
What is knowledge base freshness? How closely the content a chatbot answers from matches what is currently true, including changed pages being re-read and removed pages being dropped.
Why does my chatbot give old prices or policies? Usually the changed page has not been re-read yet, the old page was never removed from the index, or the bot is answering from a page nobody has reviewed. Check these before blaming the model.
How often should a chatbot knowledge base be refreshed? It depends on the content. Fast-changing facts such as prices and hours need a re-sync right after each edit; slow-changing articles can follow the platform's schedule plus a periodic review.
Is this the same as the model's knowledge cutoff? No. The cutoff is when the model's training text ends. Freshness is about the documents you supply at question time.
Does fine-tuning fix stale answers? Not reliably. Facts that change are better kept in retrievable content you can update than baked into a trained model; see fine-tuning.
Related terms
- Chatbot knowledge base — the content the bot answers from.
- Retrieval-augmented generation — answering from retrieved passages.
- Vector database — where indexed passages are stored and where orphans can linger.
- AI hallucination — stale content produces confident wrong answers that look like one.
- Chatbot feedback loop — one way stale answers get noticed.
- Chunking — how pages are split before indexing.
Sources
- Intercom, Syncing and managing public URLs — intercom.com/help/en/articles/9357945-syncing-and-managing-public-urls, read 2 November 2026. Source of the weekly and 14-day re-sync schedule and the 90-day pause. The article does not say whether deleted pages are removed on re-sync.
- Microsoft Learn, Change and delete detection using indexers for Azure Storage in Azure AI Search — learn.microsoft.com/azure/search/search-how-to-index-azure-blob-changed-deleted, read 2 November 2026. Source of the last-modified change detection and the orphaned-document behavior without a deletion strategy.
- Chatbotscape evaluation methodology. /methodology (continuously updated).