Millions of Britons risk incurring financial losses because AI models are giving incorrect information, new research from Saturn has revealed.
The firm’s report ‘Artificial Authority: Should you trust AI to deliver financial advice?’ shows that popular AI engines such as ChatGPT, Claude, CoPilot, Grok and Gemini give wrong answers to financial inquiries around 57% of the time.
On harder questions, AI was found to make mistakes in 88% of cases, with some AI models giving wrong answers 99% of the time to more complex questions.
Saturn’s research tested 18 popular AI models against 121 different financial questions. Each question was repeated five times for consistency and in total, over 10,000 questions were put through the AI models. It found that answers contained errors in calculations, missed vital risk warnings, ignored upcoming tax changes or created rules that did not exist.
Free-to-use models were shown to give more inaccurate advice than paid-for models. While free AI models made mistakes in 63% of answers, paid-for models made mistakes in 49% of answers.
The worst performing model Claude Haiku 4.5 made mistakes in 82% of answers, while Gemini 3.1 Pro was the second worst performer, failing 73% of tests. The best performing model overall was Claude Opus 5, which made mistakes in 39% of answers.
Among the most serious and costly errors in the experiments, a mistake on pension tax rules by Claude Haiku 4.5 could have resulted in a pension saver facing a £17,500 charge from HMRC.
Meanwhile, flawed mortgage advice presented further potential harm for borrowers. Saturn said a Gemini model wrongly reassured a borrower that taking a mortgage payment holiday would not damage their credit score when in reality it could, making it harder to secure the most competitive rates in future.
The research follows the FCA’s Mills Review which showed 26% of consumers trust general-purpose AI tools like ChatGPT and Claude for financial advice.
Amal Jolly, chief executive at Saturn, said: “AI tools are fast becoming the first port-of-call for many consumers with financial questions. But our research reveals evidence that it is far too early to put so much trust in AI chatbots.
“The low quality of financial advice from mainstream AI models risks leading to widespread consumer harm. Millions of people are trusting the AI models for money advice, but they are getting wrong answers that can lose them money.”
Main image: nahrizul-kadri-OAsF0QMRWlA-unsplash
