NHS Drop AI Health App due to Manipulation Fears

A planned trial of a chat-bot app has been abandoned, after patients admitted they would manipulate the system in order to see a GP quicker.

Babylon Health, a ‘chat-bot’ health app, has been officially dropped after concerns were raised about manipulation of the system.

The app uses Artificial Intelligence to ask and answer simple questions about patients’ symptoms. It had been hoped that NHS clinical commissioning groups in London could use the Babylon healthcare app for this function, freeing up GP’s time.

It’s estimated that over £2bn could be wasted in the UK alone each year just on unnecessary medical appointments within the NHS. The proposal was for Babylon Health to reassure patients when their symptoms don’t need medical intervention, directing them to a pharmacist or on how to self-care for their ailments.

The board of North West London Collaboration of CCGs produced a report, which said: “Patients who might use the app could mainly do so because they believed it would get them faster access to GP appointments.

The report went on to say: “…there is a risk of some people gaming the symptom checker to achieve a GP appointment.”

Further testing has not been continued.

Babylon told the Health Service Journal, which first published the story, that the original report was “factually incorrect and deeply misleading”, but their comments only seemed to relate to an issue with the wording of the original report, which had suggested that the app had already been piloted.

There has been concern about Babylon’s platform for some time, including amongst family doctor groups who have pointed to disclaimers about the app’s suitability for pregnant women, or frail older patients.