AI models hallucinate, and doctors are OK with that

HELLFIRE

https://www.theregister.com/2025/03/13/ai_models_hallucinate_and_doctors/

The tendency of AI models to hallucinate – aka confidently making stuff up – isn’t sufficient to disqualify them from use in healthcare settings. So, researchers have set out to enumerate the risks and formulate a plan to do no harm while still allowing medical professionals to consult with unreliable software assistants. …

Regards

Rook

…hallucinate – aka confidently making stuff up…

So the AI is acting more like a human. 🤖

I think that this shouldn’t be immediately disqualifying, as long as an actual person is using the outputs as an additional tool in the toolbox and are fully aware of the limitations.

“If an AI model outputs misleading diagnostic information, questions arise as to whether liability should fall on the AI developer for potential shortcomings in training data, the healthcare provider for over-reliance on opaque outputs, or the institution for inadequate oversight,” the authors say.

This seems like a question with a straightforward answer: the healthcare provider and institution should bear all liability for their mistakes. Blaming the AI developer should not be an option.

px_eliezer_

Rook

As with a GPS for your car—you can’t just blindly follow it, sometimes GPS has people drive off cliffs.

People are still needed.

Especially because NOT all knowledge is in the database.

“My dear Byerley, I see that you instinctively follow that great error – that the Machine
knows all. Let me cite you a case from my personal experience. The cotton industry engages
experienced buyers who purchase cotton. Their procedure is to pull a tuft of cotton out of a
random bale of a lot. They will look at that tuft and feel it, tease it out, listen to the crackling
perhaps as they do so, touch it with their tongue, and through this procedure they will determine
the class of cotton the bales represent. There are about a dozen such classes. As a result of their
decisions, purchases are made at certain prices; blends are made in certain proportions. – Now
these buyers cannot yet be replaced by the Machine.”
“Why not? Surely the data involved is not too complicated for it?”
“Probably not. But what data is this you refer to? No textile chemist knows exactly what
it is that the buyer tests when he feels a tuft of cotton. Presumably there’s the average length of
the threads, their feel, the extent and nature of their slickness, the way they hang together, and so on. – Several dozen items, subconsciously weighed, out of years of experience. But the
quantitative nature of these tests is not known; maybe even the very nature of some of them is not known. So we have nothing to feed the Machine. Nor can the buyers explain their own judgment.
They can only say, ‘Well, look at it. Can’t you tell it’s class-such-and-such?’ “
“I see.”
“There are innumerable cases like that. The Machine is only a tool after all, which can
help humanity progress faster by taking some of the burdens of calculations and interpretations
off its back. The task of the human brain remains what it has always been, that of discovering
new data to be analyzed, and of devising new concepts to be tested. A pity the Society for
Humanity won’t understand that.”
“They are against the Machine?”
“They would be against mathematics or against the art of writing if they had lived at the
appropriate time. These reactionaries of the Society claim the Machine robs man of his soul. I
notice that capable men are still at a premium in our society; we still need the man who is
intelligent enough to think of the proper questions to ask. Perhaps if we could find enough of
such, these dislocations you worry about, Coordinator, wouldn’t occur.”

Isaac Asimov, “The Evitable Conflict” from “I, Robot”.

vaxvms

How long will it be until lawyers who use AI when filing lawsuits go after doctors who used AI.
AI vs AI

px_eliezer_

vaxvms How long will it be until lawyers who use AI when filing lawsuits go after doctors who used AI.
AI vs AI

If only MAD Magazine were still around.