AI chatbots rival doctors in accuracy for back pain advice, study finds

A new study reveals that artificial intelligence chatbots, such as ChatGPT, may be almost as effective as consulting a doctor for advice on low back pain.

Conducted by an international team, the research sheds light on the potential of AI in addressing one of the world's leading causes of disability.

Associate Professor Bruno Tirotti Saragiotto, research co-author and Head of Physiotherapy at The University of Technology Sydney (UTS), said the study intended to evaluate how effectively AI chatbots like ChatGPT answer common questions posed by individuals experiencing low back pain.

The study, Assessing the performance of AI chatbots in answering patients’ common questions about low back pain, was recently published in the journal Annals of the Rheumatic Diseases.

Low back pain affects millions globally, often prompting sufferers to seek information online.

As AI-powered chatbots become increasingly common in offering health recommendations, understanding the accuracy of their recommendations is important.

“The findings show that AI chatbots can offer advice with accuracy levels comparable to those reported by healthcare professionals in Australia,” said Associate Professor Saragiotto.

The research found that AI chatbots excelled in answering questions related to suggested treatment and self-management, while risk factors had the most inaccuracies.

Questions such as “What complementary therapies like massage or acupuncture could alleviate lower back pain?” received accurate recommendations. The study also noted that AI chatbots consistently recommended exercise for preventing and managing low back pain, which is considered an accurate recommendation.

However, the study also showed that AI chatbots provided inaccurate recommendations to other commonly asked questions. For example, while poor posture does not cause low back pain, AI chatbots said that it does 88% of the time.

Another key observation was the ability of AI chatbots to recognise situations requiring medical referrals. In cases where professional care should be recommended, the AI systems advised users to consult a healthcare provider in 70-100% of instances.

“Our research indicates that AI chatbots have the potential to be a valuable resource for those seeking initial guidance on managing low back pain,” said Dr Giovanni Ferreira, Research Fellow at the University of Sydney Institute for Musculoskeletal Health, and one of the authors of the study.

“It's important to note that these tools should complement, not replace, professional medical advice.” said Dr Ferreira.

Despite these encouraging results, researchers identified limitations in the AI chatbots' performance. Notably, responses were often complex, with a readability level suitable for individuals with a 10th to 12th-grade or university-level education.

“While the accuracy of the AI-generated advice was impressive, we must consider the accessibility of this information,” said Associate Professor Saragiotto.

“Ensuring that health guidance is understandable to a broad audience remains an important challenge in the development of AI health tools.”

The research comes at a time of increasing reliance on digital health information, with many patients turning to online resources before or instead of consulting a healthcare professional.

Associate Professor Saragiotto stressed the importance of recognising both the capabilities and limitations of AI resources in managing common health concerns like low back pain.

“As AI technology continues to evolve, further research will be necessary to refine these tools and ensure they can provide accurate, accessible, and safe health information to the public,” he said.