Dive Brief:
- Three-quarters of enterprises have already rolled back or shut down a customer-facing AI agent after deployment, according to a Sinch survey of more than 2,500 senior decision-makers released earlier this month.
- Among organizations with mature governance frameworks, that rate increases to 81%, according to the survey of large enterprises in the United States, UK, Australia, Brazil, Germany, France, India, Singapore, Mexico and Canada.
- A clear majority of rollbacks are due to governance failures, Sinch found. Nearly one-third of organizations cited customer data exposure as the leading cause of such a rollback, followed by 22% of organizations citing hallucination or brand risk. Another 16% cited the inability to diagnose what went wrong.
Dive Insight:
Retiring or scaling back AI agents doesn’t necessarily mean complete failure. Often, enterprises are learning from their mistakes and taking the opportunity to adjust accordingly.
“The most advanced organizations aren’t failing less; they’re seeing failures sooner,” Daniel Morris, CPO at Sinch, said in a prepared statement. “Higher rollback rates reflect better monitoring and control, not weaker performance.”
Greg Carlucci, senior director analyst at Gartner, says these rollbacks signify learning.
“The AI movement is moving so quickly that there inevitably will be a lot of tests and learning, so I don't necessarily feel that the high level of role black rollback is a negative,” Carlucci said. “In fact it's almost, in my view, a positive, because the organizations that are launching these tools want to make sure that they're getting it right.”
Brands can do thorough internal testing prior to launching, such as piloting questions consumers are most likely to ask. But it “isn't a perfect science,” Carlucci said. As much as an enterprise can prepare, there are certain issues it won’t discover until the tool is live.
“Until you actually see it in use with the customer, there are things that you'll need to adjust, whether it's the data that the AI agents trained on, the tools that it has access to,” Carlucci said.
The important thing is the ability to track issues and customer feedback to the tool.
Chuck Gahun, principal analyst at Forrester, pointed to several hiccups brands face in rolling out AI agents: cascading incorrect responses, data issues, and lack of tracing and logging at each step of the agent’s workflow.
“AI agents are only as good as the data they’re trained on in the access and permissions that they're granted,” Carlucci said.
The issue is most brands don’t have unified, clean data.
“An AI agent needs such a large amount of unified data that's organized and centralized for it to access,” Carlucci said. That takes time to build, as well as investment, meaning only a few companies are ready for fully functional autonomous AI agents.
But these rollbacks aren’t stifling AI agent development. More than 3 in 5 enterprises already have AI agents live in production, and nearly 9 in 10 say their AI agents will be live within a year, according to Sinch.
