Risks posed by AI in healthcare

MEDTECH consultant Ivor Campbell shares insights into some of the issues around artificial intelligence (AI) in healthcare.

Got an opinion or experience to share? Let us know in up to 400 words via email to info@pharmacydaily.com.au.

The rise of AI-powered health apps that claim to diagnose conditions in real time is transforming how we approach healthcare.

From symptom checkers to wearable ECG monitors and AI stethoscope apps, these tools promise early diagnoses and personalised healthcare at our fingertips, empowering users with real-time insights into their health.

But as these technologies become more sophisticated, a critical question emerges: are they genuinely helpful, or do they introduce new dangers – and what happens when they go wrong?

For many people, these tools offer unprecedented access to medical insights, reducing the need for frequent GP visits and enabling earlier interventions.

The potential benefits are significant – AI can process vast amounts of data far more quickly than a human doctor, identifying patterns that might otherwise go unnoticed.

For patients in remote or underserved areas, AI diagnostics could be life changing: a smartphone app that detects atrial fibrillation or diabetic retinopathy can bridge gaps in healthcare access where few medical professionals are available.

Yet, for all their promise, AI health tools come with serious risks, and one of the most pressing concerns is misdiagnosis.

AI models are only as good as the data they’re trained on, and if that data is flawed or incomplete, the results can be dangerously inaccurate.

A study by Stanford Medicine found that some AI diagnostic tools performed well in controlled lab settings, but faltered in real-world scenarios, where patient diversity and environmental variables introduced unpredictability.

False positives and false negatives are another major issue – an AI app that incorrectly reassures a user that their chest pain is harmless could delay critical treatment, while one that falsely flags a benign mole as malignant might trigger unnecessary anxiety and even medical procedures.

Unlike human doctors, AI lacks the ability to contextualise symptoms – it does not know if a patient has a history of health anxiety or if their symptoms align with common, non-threatening conditions.

Regulation is another grey area.

Should AI diagnostic apps be classified as medical devices, subject to the same rigorous testing as traditional diagnostics?

In many jurisdictions, the answer is unclear.

Beyond accuracy, AI tools raise thorny ethical and legal questions.

If an AI app provides faulty advice that leads to harm, who is liable – the developer? The healthcare provider endorsing it? The user who trusted the results?

Legal frameworks have yet to catch up with these scenarios, leaving patients and providers in uncertain territory.

Data privacy is another major concern, with many AI health apps collecting sensitive personal information – if this data is mishandled or breached, it could be exploited by insurers, employers or malicious actors.

Then there is the psychological impact, where the ease of self-diagnosis can fuel ‘cyberchondria’ – a modern form of health anxiety where users obsessively research symptoms, often convincing themselves of worst-case scenarios.

Unlike a doctor who can offer reassurance, an AI tool may simply present probabilities, leaving users spiralling into unnecessary fear.

So, where does this leave us – will AI doctors replace general practitioners, or will they remain assistive tools?

The most likely scenario is a hybrid model – AI handling routine diagnostics and data analysis while human doctors focus on complex cases, patient communication and emotional support.

The challenge for regulators, developers, and healthcare providers is to strike a balance – harnessing AI’s potential while safeguarding against its pitfalls.

Robust validation, transparent algorithms, and clear accountability frameworks will be essential.

Patients, too, must approach AI diagnostics with caution, using them as supplements – not substitutes – for professional medical advice.

Ivor Campbell is Chief Executive of Snedden Campbell, a recruitment consultant for the global medical technology industry.