https://www.theregister.com/2025/07/24/ai_is_overconfident_does_not_learn_study/
https://link.springer.com/article/10.3758/s13421-025-01755-4
… The promise of an always-available expert who can chew the fat on a range of topics using conversational natural-language question-and-response has proven popular – but the reality has fallen short, thanks to issues with “hallucinations” in which the answer-shaped object it generates from a stream of statistically likely continuation tokens bears little resemblance to reality. … “We still don’t know exactly how AI estimates its confidence,” [co-author Danny] Oppenheimer adds, “but it appears not to engage in introspection, at least not skilfully.” …
Four popular commercial LLM products – OpenAI’s ChatGPT, Google’s Gemini, and Anthropic’s Claude Sonnet and Claude Haiku - were given various tasks, and the performances and confidence in each task were then compared to human participants.
As I’ve asked before : which would you rather have, Artificial Intelligence or Natural Stupidity? /cynical
I’m also reminded of this saying : “Knowledge is knowing a tomato is a fruit; wisdom is not putting it in a fruit salad.” -Miles Kington
Regards