Healthcare organisations are becoming overly focused on finding artificial intelligence solutions before properly understanding the problems they are trying to solve, while expected capacity and cost savings from AI deployments are often overstated.

That’s according to the Director of AI Development at Singapore General Hospital, Professor Adam Chee, who told an audience at HealthTechX Asia in Singapore that many AI projects were predetermined before there was a clearly defined need.

In a presentation titled “There’s an AI for That. And That’s Exactly the Problem,” Prof Chee said he actively discouraged healthcare organisations from developing or adopting AI until they had addressed underlying workflow issues.

“Whenever I talk about AI, I actively discourage people from developing AI or adopting AI solutions until they have sorted out their underlying workflow problem,” Prof Chee said.

“In healthcare in particular – technology is never the problem. It’s almost always a financial or a workflow problem.”

Prof Chee said the rapid growth of AI had created a mindset where every challenge was immediately met with the suggestion that “there’s an AI app for that.”

“I have a patient engagement issue. Almost always someone says, ‘well, there’s an AI app for that.’ I have a queue management problem. ‘Oh, there’s an AI app for that,’” he said.

“And this, in my opinion, is sometimes where the problem starts. Because we always assume that there is a solution.”

Prof Chee said that for many AI projects “we identify the solution before we have understood the problem.”

He noted common requests from healthcare staff – who had seen a product at a conference, in another hospital or online and wanted it implemented at their own organisation.

In response, Prof Chee said he often asks ‘What is the problem you’re trying to solve? Because I’m not just going to develop a solution that you saw elsewhere.’

Prof Chee said that importing AI solutions from other organisations rarely translated automatically into success, because healthcare settings had different workflows, resources and patient populations.

“It doesn’t really work – because different hospitals have different workflow requirements. You have a different catchment area. You have different resources in-house.”

He also questioned the assumption that successful pilots automatically lead to successful deployments.

“Most AI, from my experience, fails shortly after deployment,” he said, noting that usually pilots operate in controlled environments with carefully defined use cases and measures of success – while real-world healthcare environments involve complex situations and cases that often don’t align to automated workflows.

“Pilot success doesn’t really mean real success,” he said, while adding that healthcare organisations were often measuring the wrong things when evaluating AI.

“A model can predict deterioration about maybe up to 95 per cent. But if it does not fit the clinician’s decision-making process, this will not work.”

He said that strong model performance did not necessarily translate into meaningful clinical impact, while deployment did not necessarily mean adoption.

“Just because an application is launched, is used, is deployed, doesn’t necessarily mean that it’s actually influencing part of your clinical decision support,” he said, also pointing out that usage metrics could create a misleading impression of success.

“If you try to measure metrics without understanding what is going on the ground, you will get very nice statistics – but you will not be able to tell whether it’s really making a clinical impact.”

Prof Chee was also unconvinced by claims that AI would significantly reduce healthcare costs, stating “We’re going to use AI to save cost? That’s not going to happen.”

“Healthcare is an interesting industry where the more capacity you free up, the busier you become – because we are under-serving to begin with.”

Prof Chee said that instead of asking whether an AI solution could be built, organisations should first determine whether it should be built.

“The real question is not whether we should use AI or not – AI is definitely powerful, but it doesn’t really belong everywhere.”

He said healthcare leaders should consider three questions: “Who is this for? What is the unmet need? What are the decisions, or behavior, or workflow that should change?”

“If they give you a vague answer, be very, very careful – they have not thought this through,” Prof Chee warned, adding that the real challenge of AI was not the technology itself.

“Building the wrong thing well is still the wrong thing to begin with. You don’t need AI. You need to fix your workflow first, then you need AI.”