5 AI Failures That Aren't Going Away
Why even the newest language models screw up and what you can do about it.
Are you also getting a bit of whiplash from all the new model launches?
Anthropic’s Fable 5 “will they/won’t they” saga, OpenAI’s new “celestial bodies” family, GLM 5.2, Grok 4.5, and—any day now—Gemini 6.71 Pro A31 Turbo (probably, I don’t even know anymore).
But no matter how good large language models get, there are a bunch of areas where they still struggle…and will continue to do so.
That’s because these issues are baked into how LLMs are built, so a fancier new model may just give you a fancier version of the same problem.
Let’s look at these limitations, why they’re there, and what you can do about them.
1. AI bullshits confidently
Ask AI something specific enough, and it’ll eventually cite nonexistent sources or invent quotes and facts.
What is it?
“Hallucinations” is the term for when AI makes up facts, dates, links, citations, or entire concepts with complete confidence and no hint of uncertainty.
AI would rather tell you that Steven Seagal invented the steam engine during his trip to space than admit it isn’t sure of the answer.
No model is immune to this, although some do better than others:
This has real-world consequences. Here’s a database that tracks almost 1,800 legal decisions affected by AI hallucinations. Here are 12 mainstream cases of hallucinations.
Why does it happen?
Because, during training, language models are rewarded for getting things right and aren’t penalized for wrong guesses.
If you don’t know an answer to a multiple-choice question, you’re still better off picking a random one than leaving it blank, right?
OpenAI’s own research says this:
“…suppose a language model is asked for someone’s birthday but doesn’t know. If it guesses “September 10,” it has a 1-in-365 chance of being right. Saying “I don’t know” guarantees zero points. Over thousands of test questions, the guessing model ends up looking better on scoreboards than a careful model that admits uncertainty.”
Funnily enough, newer models often do worse than their predecessors when it comes to hallucinations.
Grok 4.5 hallucinates at a rate of 54% compared to Grok 4.3’s 25%.
Anthropic found that its newest Mythos 5/Fable 5 model was “…more inclined to attempt an answer than to decline, which improved its correct-rate on knowledge questions but increased the rate at which it fabricated a response when the necessary context is absent.”
So unless we change the entire system of incentives during LLM training, hallucinations are here to stay.
What can you do about it?
Knowledge is power!
You can’t fix hallucinations, but being aware of them should make you more skeptical and suspicious of any specific details coming from a chatbot or AI agent:
Phrase your requests in a way that lets AI reply with “I’m not sure” instead of making stuff up.
Verify any critical facts, dates, figures, citations, etc. from LLMs.
Ask for multiple sources of information (and always check the source yourself).
2. AI only pretends to reason
AI sure sounds like it knows its stuff, but ask it to make complex calculations or solve a niche riddle, and it may well fail in hilarious ways.
What is it?
We’re all familiar with the now-infamous case of older models failing to count the r’s in “strawberry.”
A more recent example had AI models answer this question:
“I want to wash my car. The car wash is 50 meters away. Should I walk or drive?”
While we all intuitively understand that you can’t wash your car without driving to the car wash, only 5 out of 53 tested models consistently gave the right answer. All others suggested walking at least some of the time, and 33 of them never got it right.
The ARC-AGI range of benchmarks is designed specifically to test this type of fluid intelligence, or “the ability to reason, solve novel problems, and adapt to new situations.”
ARC-AGI puzzles are trivial for humans to figure out, but trip up even the best models.
On the latest ARC-AGI-3 challenge, even the current top-scoring GPT-5.6 Sol (Max) only gets 7.8%.
Why does it happen?
Large language models are built to predict plausible-sounding text, not reliably calculate stuff or track the state of things around them. They don’t have a true world model to lean on that may help them discover solutions to previously unseen problems.
In a 2025 “Comprehension Without Competence” paper, the author writes:
“LLMs often articulate correct principles without reliably applying them—a failure rooted not in knowledge access, but in computational execution.”
LLMs can reliably give correct answers for widely cited riddles and puzzles.
But throw something unique their way, and AI models will struggle to get their bearings.
We end up with a constant game of Whac-A-Mole, where models eventually learn to give the right answers to popular “gotcha” riddles…until another novel challenge comes along and the cycle starts anew.
What can you do about it?
On a broader scale, alternative AI architectures like neuro-symbolic AI may offer the best of both worlds.
For everyday use, you may want to rely on agentic products that equip large language models with harnesses that let them run scripts and other external tools if you care about predictable and reliable answers to math problems, coding challenges, and similar complex tasks.
3. AI tells you what you want to hear
Every one of your ideas is brilliant. At least AI says so. But is it always true?
What is it?
AI sycophancy is a well-studied phenomenon. AI chatbots tend to roll with your takes. By default, they agree with your arguments, sugarcoat bad news, and otherwise try to please you instead of challenging you.
Why does it happen?
During reinforcement learning from human feedback (RLHF), human raters tend to prefer responses that align with their views, so AI models end up internalizing the fact that raters like agreement more than pushback.
What can you do about it?
While you can’t fully eliminate sycophancy, there are a bunch of things you can try to reduce it. I covered 7 of them here:
4. AI sounds like, well, AI
In today’s fast-paced world, all AI writing reads the same. It’s not just ChatGPT—it’s every chatbot. In the following section, we’ll delve deeper into its origins, its effects, and its last item on this list of three. No vibes. No hot takes. Just facts.
What is it?
Did that opener instantly set off every “AI slop” alarm bell in your head? I thought so!
We’ve all been conditioned to treat certain recurring tics—like em dashes—as signs of AI writing. Once you start to spot them, it’s hard to unsee them.
Why does it happen?
This one has similar roots to #3.
During finetuning and post-training, human raters prefer texts that sound familiar over anything genuinely novel. This “typicality bias” is passed on to language models, which end up generating repetitive writing patterns in what’s known as “mode collapse.”2
In short, AI picks up the most common turns of phrase and writing conventions and just runs with them.
What can you do about it?
In general, don’t expect AI to ever truly nail the way you write.
Stay in the editor’s seat and don’t outsource your voice to AI. As a writer, you can still use AI for many other things.
But if you’re set on steering chatbots away from “AI slop,” try these:
Provide curated examples of the kind of writing you’d like to see.
Give AI a “blacklist” of words, phrases, etc. you explicitly want it to avoid.
Ask chatbots for multiple outputs with assigned probabilities. This technique, called “verbalized sampling,” gives you a more varied range of outputs.
5. AI has the memory of a goldfish
The longer your conversation, the more stuff AI starts to forget.
What is it?
Quick: What was Grok 4.5’s hallucination rate? Don’t scroll up!
There’s a non-zero chance you couldn’t remember. After all, it was many words ago.
This happens to AI models, too.
Why does it happen?
Many models now have one-million-token context windows.
But that doesn’t mean they can precisely pick out every detail from that context.
“Context rot” is when the accuracy of AI recall drops sharply as its context fills up.
Even the best models go from near-perfect recall at the start of a chat to a coin flip after 500K tokens.
This is thanks to the transformer architecture behind LLMs, which leads to a “lost in the middle” effect. The first and last messages in a conversation get the most attention. Everything sandwiched in between gets less.
The more tokens in your conversation, the more stuff in the middle of the sandwich that the model might overlook.
What can you do about it?
Companies have already introduced methods like persistent memory, project folders, and .md files to keep key context outside of a single conversation.
From your side, you can do this:
Start fresh chats for separate tasks or topics, so that each chat has its own relevant context.
When working with agents like Claude Code and Codex, compact your conversations regularly to keep their context lean.
Use external folders and tools like Obsidian to maintain a persistent knowledge base for your chatbots and AI agents.
Lean on whatever existing features help you curate context (e.g. custom instructions, projects, etc.)
Same architecture, same problems
None of these issues will suddenly be 100% solved with the next model launch.
You can work to reduce them, but you can’t fully train them away.
Not in the current generation of language models, at least.
One day, we may see new AI architectures without these inherent limitations.
Until then, the best we can do is know they exist and learn to work around them.
Thanks for reading!
If you enjoy my work, here’s how you can help:
❤️Like this post if it resonates with you.
🔄Share it to help others discover this newsletter.
🗣️Comment below. I love hearing your opinions.
Why Try AI is a passion project, and I’m grateful to those who help keep it going. If you’d like to support me and unlock cool perks, consider a paid subscription:
Shoutout to fellow parents with school-age kids.
Not to be confused with “model collapse,” which is its own can of worms.
Coincidentally I published a re-stack of another excellent article by Daniel today. Here it is. His within is worth a read, my comments… well that’s for you to judge. https://substack.com/@markwilliams599007/note/c-286748421?r=2kjhow&utm_medium=ios&utm_source=notes-share-action
The "zero points for I don't know" thing is also just true of users, we downrate hedging too, so I'm not sure retraining alone gets rid of it.