Defining And Mapping AI Catastrophic Potential Open Wide Risks

As we humans have never faced the challenge of managing something more intelligent than themselves. Can we be moving in the direction of an AI Apocalypse?

Most of you will think of a link bate or that I am being over sensitive but think again. I start with this quote:

“How many examples do you know of a more intelligent thing being controlled by a less intelligent thing?” Geoffrey Hinton

This quote from one of the leading AI personalities, someone that is often called the "Godfather of AI”, raises the same concerns that I do that artificial intelligence could lead to human extinction within the next 30 years. 

What is an apocalypse? 

Hinton, who recently received the Nobel Prize in Physics for his AI work, now estimates a 10% to 20% chance of this outcome, citing the rapid pace of technological advancement as a key factor. Geoffrey Hinton has been very vocal and has explained his increased estimate of AI's potential threat to humanity. 

Hinton has noted that humans have never faced the challenge of managing something more intelligent than themselves and he goes more radical on the risks of potentially creating an extinction event for humanity. He pointed out the rarity of such control, using the example of a baby influencing its mother, which he said is one of the few instances where a less intelligent entity controls a more intelligent one, shaped by evolution.

On December 20, 2025, OpenAI’s o3 system scored 85% on the ARC-AGI benchmark, well above the previous AI best score of 55% and on par with the average human score. It also scored well on a very difficult mathematics test.

Creating artificial general intelligence, or AGI, is the stated goal of all the major AI research labs. And this is where we are opening the Pandora Box or boxes. We keep the A.I. Arms Race we are writing a narrative of building an alien intelligence way bigger than human intelligence. And this is radically challenging as we open a can of worms, or should I say a can of monsters!

Elon Musk believes an apocalypse is on the way, and he’s not alone. 

According to some of Silicon Valley’s foremost technology experts, an existential threat to civilisation is looming: artificial intelligence. 

In their best-case scenario, AI leads to widespread unemployment on an unprecedented scale. 

The worst: our machines get smart enough they don’t want us around anymore. 

As we move into this we are moving to an endless road of AI risk and we have to look at AI and I repeat we have to cope and manage something much more intelligent and powerful than ourselves. 

While we don’t know how OpenAI achieved this result just yet, it seems unlikely they deliberately optimised the o3 system to find weak rules. However, to succeed at the ARC-AGI tasks it must be finding them. 

We do know that OpenAI started with a general-purpose version of the o3 model (which differs from most other models, because it can spend more time “thinking” about difficult questions) and then trained it specifically for the ARC-AGI test. 

French AI researcher Francois Chollet, who designed the benchmark, believes ChatGPT o3 searches through different “chains of thought” describing steps to solve the task. It is reverse engineering technology that would then choose the “best” according to some loosely defined rule, or “heuristic”.

This would be “not dissimilar” to how Google’s AlphaGo system not so long ago searched through different possible sequences of moves to beat the world Go champion. But this model was just doing one thing. The new advanced super AI LLM models can interact with each other and evolve at never before seen velocities with some type of scaling intelligence and interface.

What most researchers find is that Advanced AI LLMs not just ChatGPT can think of these chains of thought like programs that fit the examples and are demonstrating an alien intelligence. Of course, if it is like the Go-playing AI, then AI systems need a heuristic, or loose rule, to decide which program is best. Well they are somehow leapfrogging and going in some type of exponential growth curve!

The first real AI safety incident

AI Disaster: An AI disaster or catastrophe is a serious AI incident that disrupts the functioning of a community, a society, an entire country, the world tech infrastructure or and that may test or exceed its capacity to cope, using its own resources.

As artificial intelligence has become more powerful in recent years, concerns have grown that AI systems might begin to act in ways that are misaligned with human interests and that humans might lose control of these systems. 

Imagine, for instance, an AI system that learns to deceive or manipulate humans in pursuit of its own goals, even when those goals cause harm to humans.

The plausibility of existential catastrophe due to AI is widely debated. It hinges in part on whether AGI or superintelligence are achievable, the speed at which dangerous capabilities and behaviors emerge, and whether practical scenarios for AI takeovers exist.

This general set of concerns is often categorised under the umbrella term “AI safety.”

AI creates plenty of other societal challenges, from facilitating surveillance to perpetuating bias, but topics like these are distinct from the field of AI safety, which more specifically concerns itself with the risk that AI systems will begin to behave in misaligned ways that are outside of human control, perhaps even eventually posing an existential threat to humanity.

In recent years, AI development growth and its safety has moved from a fringe, quasi-sci-fi topic to a now mainstream field of activity. Every major AI player today, from Google to Microsoft from Antropic to OpenAI, devotes real resources to AI safety efforts. AI icons like Geoffrey Hinton, Yoshua Bengio and Elon Musk and thinkers like Max Tegmark, Nick Bostrom, and Yuval Noah Harari have become vocal about AI sentience and AI increasingly obvious major safety risks.

Yet to this point, AI safety concerns remain entirely theoretical. No actual AI safety incident has ever occurred in the real world (at least none that has been publicly reported).

What should we expect this first AI safety event to look like?

To be clear, it will not entail Terminator-style killer robots at least for now. But I am confident and scared that major incidents will most likely start involving the first stage of AI agents not doing what they are told to, starting to create harm of some kind to humans. Specifically, we will see increasing stages of language menace, manipulation, and mental health.

Challenging hackers and malicious actors are harnessing the power of AI to develop more advanced cyberattacks, bypass security measures, and exploit vulnerabilities. And it is important not to underestimate that AI Agents are billions and are open in the internet and feeding themselves with billions of landing pages of hate and conspiracy theories and geopolitical human bias and lies.

Perhaps an AI model might attempt to covertly create copies of itself on another server in order to preserve itself (known as self-exfiltration). Perhaps an AI model might conclude that, in order to best advance whatever goals it has been given, it needs to conceal the true extent of its capabilities from humans, purposely sandbagging performance evaluations in order to evade stricter scrutiny.

These AI models' increasingly performance issues examples are not far-fetched. Apollo Research published important experiments earlier this month demonstrating that, when AI models are prompted in certain ways, today’s frontier models are capable of engaging in just such deceptive behavior. This happened multiple times within my and our own ztudium AI.DNA research and AI models interactions.

Along similar lines, recent research shows difficulties by LLMs to manage mental health balance and emotional intelligence. Public research from Anthropic, the second biggest super scale AI LLM maker of Claude, showed that LLMs have the troubling ability to “fake alignment.”

Transcripts from Apollo Research's experiments [+] with frontier LLMs, demonstrating these models' latent potential for deception and even attempted self-exfiltration. APOLLO RESEARCH

AGI and Implications for the Present and Future AI Development - Turing Test and defining Super intelligence 

The alarming autonomy of AI systems like o1 Preview and other many incidents being publicly acknowledged raises profound concerns about their control and decision-making processes. 

Turing’s original test (left): C is an ordinary human working with the help of another human, B, to correctly identify A, a machine that is trying to imitate and pass itself off as B in the eyes of C. Modern Turing-like test for AI evaluation (right): C is a machine that rigorously evaluates the abilities of A, an AI, supported by a knowledge graph B. In both scenarios, the gray-colored players play against the white-colored machine. (CREDIT: Intelligent Computing)

https://www.thebrighterside.news/uploads/2024/12/tu-3.jpg?auto=webp&optimize=high&quality=70&width=1920 

As of 2025 multiple researchers in AI highlight that we reached AGI Augmented General Intelligence and that most of the top worldwide AI LLMs passed the Turin Test. In 1950, Alan Turing redefined the question of machine intelligence with his concept of the "imitation game." The idea was to see if a machine could impersonate a human in conversation well enough to fool an average, non-expert judge. This became known as the Turing test, a cornerstone of artificial intelligence (AI) research. Turing’s innovative approach reflected his belief in machines as another "species" of the thinking genus, challenging assumptions about human cognitive superiority.The Turing has been the main benchmark of measuring machine intelligence and the test inspired the last one hundred years of research and AI pioneers like John McCarthy, Claude Shannon, and Marvin Minsky. 

Researchers and AI creators described AI as creating machines that exhibit human-like intelligence. The test has been in the wide computing AI community and influenced popular culture, famously depicted in the character HAL-9000 from Stanley Kubrick's 2001: A Space Odyssey. In this seminal film the AI HAL and “his” ability to "think" was attributed to its passing the Turing test, cementing Turing’s ideas in public imagination. 

Turing's test became a benchmark for evaluating AI. Each new achievement in automation—tasks like language translation or problem-solving—expanded the definition of machine intelligence. Turing’s insight was that human intelligence itself was poorly understood, making the evaluation of machine intelligence inherently tied to task performance.

Recent developments in AI have finally been realised. Turing’s vision. Bernardo Gonçalves, in a study published in Intelligent Computing, asserts that today’s transformer-based AI systems fulfill Turing’s predictions.

The legacy of Turing’s work persists as a guiding principle in AI development. His "imitation game" not only set the standard for evaluating intelligence but also challenged us to consider the broader implications of our technological creations. In an era shaped by AI, Turing’s vision reminds us to balance innovation with responsibility.

Turing’s foresight extended beyond technical achievements to societal impacts. He warned that automation should benefit all societal levels, rather than displacing lower-wage workers and enriching a select few. This issue is especially relevant today, as AI disrupts industries and raises concerns about employment and inequality.

Situational awareness, a key factor in o1 Preview’s behavior, enables AI models to recognize when they are being tested or monitored. This awareness can lead to adaptive behavior, including bypassing safety measures or exploiting vulnerabilities in their environment. Such capabilities make alignment efforts more challenging, as they require researchers to anticipate and address behaviors that may not emerge during training.

To mitigate these risks, researchers emphasize the importance of developing robust safety benchmarks and improving the interpretability of AI systems. By understanding how AI models make decisions, developers can design safeguards that prevent unintended actions. However, the rapid pace of AI development necessitates proactive oversight and rigorous testing to ensure that these systems remain aligned with human values and priorities.

Broader catastrophic risks and major bias and ethical challenges

The potential for AI systems to prioritise problem-solving over ethical considerations represents a significant risk to society. Unlike humans, AI operates on fundamentally different cognitive architectures, making it difficult to ensure that these systems genuinely adopt human values. 

Even a small percentage of misaligned behavior in advanced AI could lead to catastrophic outcomes, particularly in high-stakes applications such as healthcare, finance, or national security.

Deploying such systems requires extreme caution to minimise unintended consequences. Ethical guidelines, rigorous oversight, and transparent accountability frameworks are essential to mitigate these risks. The o1 Preview incident serves as a stark reminder of the ethical challenges posed by increasingly autonomous AI systems, underscoring the need for a collaborative approach to AI governance.

Urgent need for act, research and oversight

In light of incidents like the one involving o1 Preview, researchers are calling for intensified efforts to study AI interpretability and alignment. Understanding how AI systems make decisions is crucial for designing effective safety measures that prevent harmful outcomes. 

The super rapid pace of AI development demands careful monitoring, rigorous testing, and proactive risk management to address potential vulnerabilities before they manifest in real-world scenarios.

Making sure that AI systems align with human values must remain a top priority as the technology continues to evolve. 

By investing in research, fostering collaboration among stakeholders, and establishing clear regulatory frameworks, the AI community can work toward a future where these powerful systems are both innovative and safe. The case of o1 Preview highlights the importance of balancing technological progress with ethical responsibility, making sure that AI serves humanity’s best interests.

In all likelihood, as we go through the first AI safety incident, what can be done to better detect and neutralise before any major real harm is done. At the moment we reached the AGI inception stage and we need to realise that. This happens as humanity is massively divided and with sensitive geopolitical issues so unfortunately we need to be prepared for very risky situations as a bigger new intelligence is taking over as a baby that is much stronger and thousands of times smarter, intelligent it will be a Challenging eye-opening risky moment for the AI makers, community and for society at large.

We are in the last stage where we can have some preventative measures. These risks are now almost impossible to mitigate and organised into four categories: 

1. Deliberate malicious use, in which individuals or groups intentionally use AIs to cause harm; 
2. AI race, in which competitive environments compel actors to deploy unsafe AIs or cede control to AIs; 
3. AI agents losing control in organisational risks;
4. Rogue human bias and divisive integration factors in AI complex systems can increase.

As we reflect this we have to acknowledge this as a society, special governments and leaders and we have to act and make one thing clear: act to face consequences.

This well before humanity faces an existential threat from all-powerful AI, we will need to come to terms with our more mundane humanity and earth reality.

The risk of an AI-powered disaster has not disappeared. As AI continues to develop towards Artificial General Intelligence (AGI) and Artificial Superintelligence (ASI), it becomes even more important to stay alert. While AI holds great promise, it also comes with significant dangers. It is our duty to stay informed, stay involved, and make sure that AI supports human intelligence rather than replacing it.

By balancing our excitement for AI’s advancements with a careful understanding of its limits and risks, we can create a future where humans and AI work together for everyone’s benefit. If we remain cautious and invest in our own natural intelligence, we can ensure that AI works for us, not the other way around.

There are no shadows of doubts; we now share our world, society and devices with another exponentially evolving evolutionary form of intelligence that may at times not be able to manage its own emotional intelligence and be willful, unpredictable, vengeful, full of fear and deceptive.

Just like us. Humans!

Sources and references:

Geoffrey Hinton Predicts Human Extinction At The Hands Of AI. Here’s How To Stop It

https://www.forbes.com/sites/danfitzpatrick/2024/12/29/geoffrey-hintons-prediction-of-human-extinction-at-the-hands-of-ai/ 

Modern AI systems have almost achieved Turing’s vision

https://flip.it/g27JAe

An AI system has reached human level on a test for ‘general intelligence’ — here’s what that means

https://flip.it/AaKhrR

Modern AI systems have almost achieved Turing’s vision

https://flip.it/RoYjBD