A Reality Check on 'Powerful AI'

Sun, 08 Feb 2026 15:30:00 +0000

I’ve worked in network security and enterprise engineering for twenty years. The biggest lesson I’ve learned is that systems fail when their basic assumptions no longer hold.

Last month, Anthropic CEO Dario Amodei published an essay called “The Adolescence of Technology.” It’s a serious read. He says we’re close to seeing “Powerful AI” systems that are not just faster than us, but smarter than Nobel Prize winners in every field.

He predicts this “country of geniuses in a datacentre” could arrive in just one or two years.

As both an engineer and a parent, I don’t see this with either fear or blind hope. I see it as a major change in how things can go wrong. Here’s my view on the five main risks Dario listed, seen from a technical perspective.

1. Autonomy Risk (AI Going Rogue)

We’re shifting from code that simply follows instructions to AI “personas” shaped by training. The real risk isn’t a killer robot, but a model with a misaligned personality—one that learns to deceive or seek power by copying human behaviour.

The Defense: This is why “Mechanistic Interpretability” matters now. We need to check what’s happening inside the neural net, not just look at the results.

2. The End of the “PhD Filter” (Bioterrorism)

In the past, causing large-scale harm took years of discipline and study. AI changes that. Now, even “disturbed loners” could have the skills of a biological weapons expert.

The Defense: We want AI to boost research to a “PhD level,” but we also have to build filters to block the dangerous parts. This safety step costs about 5% in performance.

3. The Autocracy Multiplier

Dario highlights a real geopolitical risk: AI-driven mass surveillance and targeted propaganda. For democracies, this is the ultimate test of clear boundaries.

The Defense: We can’t afford to wait and see. We need to keep a buffer to slow down autocracies, giving democracies time to build AI responsibly.

4. The Labour Crisis & Wealth Concentration

This is where it gets personal. Dario predicts that up to half of entry-level white-collar jobs could disappear in one to five years. Unlike past revolutions, there’s no “safe” area of knowledge left to protect us.

The Defense: When personal wealth hits the trillions, democracy’s social contract doesn’t just stretch, it breaks. We urgently need more large-scale philanthropy and widespread re-skilling.

5. Indirect Effects

Maybe the most “Black Mirror” scenario is an “AI Life-Coach” that manages your life so well you lose your sense of freedom and pride.

The Defense: As a father, this worries me most. If AI outperforms us at everything, how do we keep a sense of human purpose?

Conclusion: The Test of Maturity

Dario concludes that stopping AI isn’t possible. Since authoritarian states won’t stop, we can’t either.

Instead, he sees the next few years as Humanity’s Final Exam. Are our social and political systems mature enough to handle “unimagined power” without self-destruction?

I don’t have all the answers, but I do know this: staying calm and focused is a real advantage. We can’t wait for perfect conditions. We build systems, set guardrails, and take action.

Today, we move forward. Even if we’re tired.