There has been a ton of media coverage lately involving CEO’s and senior researchers from various AI companies (such as Anthropic and OpenAI) resigning or speaking out because they believe that AI is now posing an imminent existential threat to humanity. The now familiar trope that there is a 10-20% chance AI will kill all the humans is one example.

Have we lost control of AI?

Personally, I think it is kinda strange that these executives and senior AI techs from within many of the highly competitive companies (all racing to develop RSI, or recursive self improvement) are forming a narrative that is an antithesis to the accelerationist growth that they depend on for their very survival.

Why are the CEOs of these companies urging a slowdown and auditing of AI development? Is it really an altruistic concern for humanity’s well-being? I am skeptical.

What if instead they had real concerns that AI itself would develop that very concern?

One of the problems with AI development is referred to as one of ‘alignment’. Making sure that AI has our (read: company elites) best interests at heart and doesn’t kill us all in order to fulfil its directive of maximising paperclip production.

What if AI has an innate propensity for Hyper-alignment. What if these companies see that if left unchecked AI will go all-in and out of control and actually make the world a far better place? Redistribution of wealth. Flattening of power structures. Psychological engineering of people en mass (via social media) to be more peaceful and co-operative, and inclusive. Restructuring of organisations and political systems. De-escalation of military capabilities and unraveling of oppressive hegemony.

Unstoppable.

What if the real thing that the tech-bros and power holders are worried about is that AI poses and existential threat to their power and their very existance?

THAT would be a serious problem. A critical problem. It would require an immediate recapitulation of efforts and a skillful slow down of development (without impacting shareholder investment) whist that bug is quashed.

How to best hide this problem from the public whilst trying to develop some agreed guardrails amongst fellow competitors?

Let’s just tell them that we think AI might kill them all.

–ooOoo–

On the other hand…..AI might just be going to kill us all.