AI in the hospital: physician or assistant?

The afternoon begins with a theatrical monologue, featuring Vera Versteeg in the year 2034. She is a fictional internist who has to appear before a disciplinary judge. Vera overlooked a tumor in one of her patients because her AI tool didn’t diagnose it either. “It’s me today, but it could be you tomorrow,” she tells the audience. “Because how can you develop a sense of direction when you never have to navigate?”

Caring for AI

Actress Margo Verhoeven shows the ethical dilemmas that artificial intelligence in healthcare could face, in the short performance ‘Caring for AI’. Her monologue at this 19th Zorg2025 meeting is about the collaboration between doctors and algorithms. The performance by theatre makers New Heroes is based on a large-scale 5-year research by the University of Amsterdam, on healthcare digitalisation. For her research, lead researcher and future anthropologist Roanne van Voorst reflects with doctors and nurses on ethical dilemmas associated with the development and adoption of AI in hospitals.

Ethical statements

After the monologue, Sebastiaan Berendsen and Corrette Ploem take the stage. Berendsen works at Deloitte on responsible AI and is pursuing his doctorate at the UvA on this topic. Ploem is associate professor of Law, Healthcare Technology and Medicine at Amsterdam UMC.

Led by moderator Niel Oedit Doebé, member of Young on Board, they discussed with each other and the audience the ethical dilemmas surrounding AI in healthcare. They kicked off discussions via a few statements.

AI in healthcare should never be a substitute for human contact between healthcare providers and patients

In the audience, about half disagree with the statement, but Ploem, on the contrary, is a strong proponent. “A good doctor decides together with the patient. That is a continuous process, in which precisely the human side of the profession is important. AI can never replace that.” But, it sounds from the audience: in real life, we cannot escape this, in order to continue to provide enough care at all, in times of staff shortages and aging.

“Dementia patients sit at home with their caregivers, case managers have gigantic workloads. That is not a sustainable situation. Here, social AI and conversational AI can indeed replace that human contact.” And, another thought from the room: “of course, there is also a generation growing up now that actually enjoys having contact not with a human being, but with a chatbot.”

Give AI the medical responsibility and not the physician. After all, AI is not influenced by human factors

Only a few people agree with this statement. Wouldn’t it save a lot of legal hassle if we could blame AI for everything, one of them wonders. Berendsen disagrees: “AI may not be influenced by human factors, but it has other flaws. Data quality plays a big role. We also know that AI does not always work so well in a new context.”

On the other hand, it cites the example of an oncologist who continued to check all AI analysis of scans himself for quite some time, and came to the conclusion that AI could recognize metastases much better than with the human eye, that is, than he could himself.

Should patients have the right to refuse AI-driven care regardless of the benefits?

“You always have the right, of course,” says Berendsen. “But whether it’s smart is a second thing. If you demand it, it can frustrate your relationship with the doctor.” Again, according to Ploem, it matters what form it takes. “If I have to bring a smartwatch for treatment, of course I can say: this is not for me. But for many screenings, AI is already being used by default. For example, whether someone can be discharged from the ICU. If you refuse AI there, it basically comes down to not wanting care at all. That’s a bad outcome.”

What does AI do for equity in health care? And if it works against it: should we use AI at all?

Berendsen thinks this is a complicated issue. “Some AI will be an improvement for a vast majority, but also a deterioration for a minority. What do you consider permissible then? You’ll have to examine that on a case-by-case basis. And you won’t know what happens until you deploy the technology.”

Someone in the audience actually interpreted the statement on these questions positively. “I hope AI creates inequality. So that the people who need care the most get better care.” That thought aligns with what the City of Amsterdam is doing, says moderator Oedit Doebé, who works there. “More money goes to some neighborhoods like Southeast with us. Unequal investment for equal opportunity, we call it.”

AI systems can make better ethical healthcare decisions than humans

Berendsen and Ploem differ quite a bit on this statement. “AI can make more objective decisions in some situations,” says Berendsen. “Beginning research shows that with a combination of multiple AI systems you can arrive at good ethical choices.” Ploem shouldn’t think about this happening. “Ethical decisions are so context-specific. A small subtle thing can make a big difference.”

Back to internist Vera Versteeg. Who said in her monologue, “I feel like we’ve been asleep for the last ten years. The question was not whether AI was going to change healthcare, but how. And we outsourced that how to developers, to big money. At the beginning of my career, I had a multidisciplinary meeting every day. Now I sit behind my screen all day.”

If it were up to today’s attendees, we wouldn’t let it come to that. And that’s already a great starting point.

Text: Mirjam Streefkerk