AI models giving incorrect financial advice over half the time

Artificial intelligence (AI) models are giving out incorrect financial advice 57 per cent of the time, research from fintech Saturn has shown.

The firm warned that millions of people faced financial losses as the AI tools they rely on for advice were giving them wrong information for the majority of questions asked.

Its report, Artificial Authority: Should you trust AI to deliver financial advice?, found the most popular AI models, such as ChatGPT, Claude, CoPilot, Grok, and Gemini, only gave accurate answers to financial inquiries 43 per cent of the time.

For more complex questions, the AI models made mistakes in an average of 88 per cent of cases, while some gave wrong answers to 99 per cent of those harder questions.

For its report, Saturn tested 18 popular AI models against 121 different financial questions, with each question repeated five times.

It found their answers contained errors in calculations, missed risk warnings, ignored upcoming tax changes, or came up with rules that did not exist.

Free AI models were found to give more incorrect advice, making mistakes in 63 per cent of answers, while paid-for models made errors in 49 per cent of answers.

“The low quality of financial advice from mainstream AI models risks leading to widespread consumer harm,” said Saturn chief executive, Amal Jolly.

“Millions of people are trusting the AI models for money advice, but they are getting wrong answers that can lose them money.”

Among the most costly errors in the analysis was from one of the free models, which could have resulted in a pension saver facing a £17,500 charge from HMRC.

The report highlighted a wide range of issues affecting consumers in different age groups and income brackets, including debt, student loans, mortgages, pensions, tax and savings.

“AI tools are fast becoming the first port-of-call for many consumers with financial questions,” Jolly said.

“But our research reveals evidence that it is far too early to put so much trust in AI chatbots. We found some of the most widely used models gave the wrong answer to financial questions the majority of the time.

“In the worst examples, relying on AI’s answers to tax questions could cost families 10s or 100s of thousands of pounds.

“AI financial advice is currently unregulated, leaving consumers with none of the protections, including compensation, that they would get if they went to a human adviser.

“The FCA has started to think about this, but it needs to act fast to protect people. The FCA should regulate AI to ensure consumers are protected.”

---