I’ve been thinking about a strange pattern in technological progress.
People can live with an old system that is obviously broken, inefficient, expensive, and sometimes even corrupt. Its failures become part of normal life. We complain about them, of course, but we rarely treat them as an emergency.
Then someone introduces a new technology that seems capable of producing much better results. Suddenly everyone becomes extremely careful. Every risk is examined. Every mistake becomes a scandal. People demand evidence, safeguards, accountability, and sometimes near perfection.
Why do we accept terrible results from what’s familiar while being so suspicious of something the evidence shows to be much better?
At first, I thought this was simply fear of technology. But the more I looked into it, the more I realized that six psychological and institutional forces are interacting at the same time.
1. Status quo bias
The first is status quo bias. We tend to prefer the current situation simply because it is already the current situation.
The choice doesn’t feel like a neutral comparison between two systems. It isn’t:
“System A works 40% of the time. System B works 80% of the time. Let’s choose System B.”
Psychologically, it feels more like:
“Keep living with what we already know, or actively decide to change everything.”
Those are very different choices. Keeping the old system feels passive and safe. Adopting the new one feels like a decision for which someone must take responsibility.
2. Omission bias
This led me to another idea that I found even more interesting: omission bias.
People often judge harm caused by an action more harshly than similar harm caused by inaction. We’re more uncomfortable with actively causing a bad result than with allowing a bad result to continue.
Take education.
Imagine the existing system leaves 100,000 children without basic math proficiency. There is no single moment when someone decides to do that. No headline says, “Government chooses to leave 100,000 children behind.”
It just happens. Year after year.
Now imagine that an AI tutoring system improves outcomes dramatically but fails to help 1,000 children. Suddenly we want to know who approved it, who is responsible, and why those children were placed at risk.
The old system failed 100,000 children, but those failures feel like an omission. The new system failed 1,000, and those failures feel like the consequence of a deliberate action.
We don’t treat the two equally.
This may be one of the most important parts of the whole idea: harm caused by the incumbent often feels like nobody’s decision. Harm caused by innovation feels like somebody’s decision.
3. Familiarity and risk perception
There’s also the way we perceive risk.
We don’t evaluate risk using statistics alone. Familiarity, control, uncertainty, and fear all change how dangerous something feels.
Cars are a good example. We tolerate a huge number of deaths and injuries from road traffic because cars are deeply embedded in normal life. The risk is familiar. We grew up with it.
But if a completely new transportation system, like self-driving cars, caused the same number of deaths during its first year, I can’t imagine society accepting it. There would be investigations, calls for bans, and a huge political reaction.
A familiar catastrophe can somehow feel safer than an unfamiliar improvement.
4. System justification theory
Then I came across system justification theory. The basic idea is that people don’t just tolerate existing arrangements. We often find ways to explain, defend, and rationalize them, even when they work against our own interests.
Once a system has existed for long enough, it stops feeling like one possible design among many. It starts feeling like reality itself.
This is how school works.
This is how healthcare works.
This is how government works.
This is how money works.
Education is probably the clearest example. A teacher in front of 25 or 30 children of the same age, teaching everyone the same material at roughly the same speed, with knowledge separated into subjects, classes, grades, and semesters, feels completely natural to us.
But none of that is a law of nature. It’s a technology. It’s one institutional design that emerged under particular historical conditions.
Researchers even have a name for this: the “grammar of schooling.” The basic structure of school has remained surprisingly stable through wave after wave of reform. New methods are often rejected, abandoned, or slowly changed until they fit the old structure.
5. Path dependence
And this isn’t only because people are stubborn.
Institutions create entire ecosystems around themselves. Teachers are trained for them. Buildings are designed for them. Laws assume them. Parents organize their work around them. Universities expect their credentials. Professional identities and careers depend on them.
The longer a system exists, the more other systems grow around it. This is called path dependence. Even if a better alternative appears, changing direction becomes difficult because you aren’t replacing one thing. You’re disturbing a whole network of connected things.
6. Algorithm aversion
I found another concept that fits especially well with AI: algorithm aversion.
Experiments have found that people can watch an algorithm make fewer mistakes than a human and still lose confidence in it more quickly once they see it make an error.
A person makes 30 mistakes, and we say, “Well, humans make mistakes.”
An algorithm makes 10, and we say, “Look, the algorithm failed.”
The unequal burden of proof
This helped me see the deeper pattern. Incumbents and challengers face completely different burdens of proof. The incumbent only needs to avoid giving us enough reason to replace it. The challenger has to prove that it is safe, effective, fair, reliable, scalable, and clearly better.
Education makes this asymmetry hard to ignore.
In the 2022 PISA international student assessment, 31% of students across OECD countries didn’t reach baseline proficiency in mathematics. The World Bank has also estimated that, after the pandemic, around 70% of 10-year-olds in low- and middle-income countries couldn’t read and understand a simple text. Even before COVID, the estimate was already 57%.
Now imagine someone proposing a completely new educational technology with those expected results.
Children will spend around 12 years using it. Governments will spend enormous amounts of money on it. At the end, roughly one-third of students across OECD countries will still lack baseline math proficiency.
Would anyone approve that experiment?
Probably not.
But the existing education system doesn’t have to reapply for permission to exist every year. Its failures have already been absorbed into normality.
A useful test
A useful way to expose this bias is to ask a simple question: if the current system didn’t already exist, and someone proposed it today with exactly the same costs and results, would we accept it?
This doesn’t tell us that the new system is automatically better. It just removes the old system’s greatest advantage: familiarity.
So I think the idea I was looking for is this:
Society doesn’t always compare innovation with the real incumbent. We compare innovation with perfection, while comparing the incumbent with the level of failure we’ve become used to.
This is probably a useful dynamic to keep in mind whenever we’re proposing a new idea, talking to policymakers, or dealing with incumbents. The resistance may not be only about the evidence. It may also come from the fact that the old system’s failures have become invisible, while every flaw in the new one feels like a decision someone has to defend.