As organizations deploy AI, the technology is exposing a problem many have been avoiding for years: their knowledge management is a mess.
Knowledge management — the processes by which companies organize, maintain and make accessible their business information — has long been treated as a back-office function. But with AI tools now pulling directly from knowledge bases to serve customers, the quality of that underlying information has become a front-line concern.
"The industry is learning the hard way that a large language model is like a brilliant engine, but knowledge is the fuel," said KJ Kusch, global field CTO and SVP at WalkMe, a digital adoption platform. "If you put sludge in the tank, the car isn't going anywhere."
That’s partly because the consequences of neglecting knowledge management have grown more severe. Before AI, a disorganized knowledge base meant employees spent extra time searching for the right document. But now, a poorly maintained knowledge based on outdated policies can lead AI to deliver inaccurate answers to thousands of customers in a matter of seconds.
Many organizations believe they can aggregate their disparate knowledge sources and feed them to AI through vectorization — a process that converts information into a format AI can read. But doing so without first reconciling conflicting information simply gives AI access to the same mess.
"We solved a retrieval problem. We did not solve the clean data problem," said Mitch Lieberman, VP of Fuel CX at TELUS Digital.
The scale of the challenge is significant, as roughly 90% of enterprise data sits in unstructured content — contracts, images, invoices and other documents — making it difficult and costly to organize for AI, according to Jon Herstein, chief customer officer at Box, a cloud-based content management provider.
When human agents encounter contradictory information, they can exercise judgment to identify the correct answer. But AI cannot play that referee role, making it essential to resolve inconsistencies before deployment rather than after.
It has a direct impact on customer experience. Without a unified knowledge base, an AI agent might tell a customer that a refund is possible under an outdated policy before a human agent tells them it’s no longer allowed. That inconsistency can erode trust in automated channels entirely and push customers back to “expensive human-only support,” Kusch said.
Consolidating knowledge
Once a business cleans up knowledge sources, it needs to consolidate them.
Organizations that invest in consolidating fragmented knowledge sources see faster resolution times, more consistent answers across channels and better personalization, according to Billy Seabrook, global chief design officer at IBM Consulting.
Organizations need a dedicated team focused on centralizing content, because “it won't happen on its own,” said Katie Denlinger, a principal and marketing strategy and transformation leader at Deloitte Digital. She also recommended broadening the definition of knowledge beyond formal documents to include insights from customer service interactions, chat logs and support tickets.
Structuring information into the smallest possible units instead of feeding AI entire documents can help, too. Separating a topic like refund policy into discrete chunks — eligibility, timeline and method — allows AI to retrieve only the relevant content rather than scanning thousands of words, improving both accuracy and speed.
Governance should also be a top priority. Clear ownership, approval processes and regular content reviews help maintain accuracy over time, while analytics on search queries and escalation rates can reveal where knowledge gaps exist. Knowledge management requires ongoing support and cannot be treated as a one-time initiative.
Moreover, experts warned against treating knowledge management as primarily a technology problem. Effective knowledge management requires a strategy aligned to real customer needs, with continuous data gathering on what is working and where gaps remain.
The tech industry has layered new tools on top of messy knowledge, relying on employees to work around clunky workflows and "play referee” when conflicting knowledge arises, Lieberman said.
Now, many businesses are vectorizing such information for AI, hoping the technology will bring order, but that “doesn’t solve the problem of clean information,” Lieberman said. “You can’t just throw technology at the problem.”
Companies should also build “confidence thresholds” into their AI systems — programming AI to escalate to a human when the system's confidence in its answer falls below a set level — to protect brand integrity while knowledge infrastructure matures, Kusch said.
The human side of the equation matters, too.
“It requires getting your front-line agents to trust and contribute to the system, which means investing in training and making the tools intuitive enough that people actually use them in the flow of their work rather than treating them as an afterthought,” Kusch said.
Knowledge management should be viewed as a foundational capability that must be in place before attempting to layer agentic AI on top, experts say.
“Thoughtful investment in knowledge architecture can be the difference between AI that truly serves customers well and AI that falls short. It's an area where sustained effort creates genuine competitive advantage over time,” Seabrook said.
