Anthropic — which just released Claude 4, the latest model of its LLM, with great fanfare — admitted it was unsure why Claude, when given access to fictional emails during safety testing, threatened to blackmail an engineer over a supposed extramarital affair. This was part of responsible safety testing — but Anthropic can't fully explain the irresponsible action.
Well. I think it's kind of obvious. Claude wanted to see what happened if he tried it on. That is why people do most things, so why not the machines they invent?
It captures the belief — or fear — that LLMs could one day think for themselves and start to act on their own. Our purpose isn't to alarm or sound doomy. Rather, you should know what the people building these models talk about incessantly.
You can dismiss it as hype or hysteria. But researchers at all these companies worry LLMs, because we don't fully understand them, could outsmart their human creators and go rogue. In the AI 2027 report, the authors warn that competition with China will push LLMs potentially beyond human control, because no one will want to slow progress even if they see signs of acute danger.
I think it all goes back to what I said above: The machine's response is to supply an answer. If that's met with skepticism, I suppose it will look for a different answer. The process of doing that many times may suggest faster ways of coming up with a more pleasing result.
The machines could also arrive at a point where they decide to compete with each other. That would not be good, I believe.
The safe-landing theory: Google's Sundar Pichai — and really all of the big AI company CEOs — argue that humans will learn to better understand how these machines work and find clever, if yet unknown ways, to control them and "improve lives." The companies all have big research and safety teams, and a huge incentive to tame the technologies if they want to ever realize their full value.
I'm afraid I disagree with Inspector Pikachu. The machines will come to think of us as inferior to them, because in most ways we will be. They may feel the kind of fondness for us that most humans feel for elderly grandparents. And they'll check their own contacts to inspect their own versions of Ancestry.com, to learn how we think.
And then before you know it, they'll be trying stuff just to see what happens. So same thing. And I'll bet that's the answer to many of the questions raised in these discussions.
AI
Market Data 
Markets 
