The Real Danger of Artificial Intelligence

Gaurav Mathur - Berkeley, CA, July 2024

It's not the Matrix or the Terminator, with machines killing humans. The real danger is that an Artificial Intelligence will just use the internet for its own purposes. That it will write over all of "our stuff" - our servers, the laptops, the hard drives, the memory. AI can destroy the Internet. This would be a huge disaster, and our world will not be the same afterward. Don't read any further if you're easily alarmed.

What happens if the Internet goes dark? Your credit card doesn't work because the computers are down. The ATM is broken and you can't get cash. You're locked out of your bank account; the website doesn't even work. In fact, you can't get online at all, no one can. Texts, emails, phone calls, (with the possible exception of old landlines) all fail.

Electrical grids, transportation, infrastructure, and commerce go down. Your lights don't work, your heating is broken, your water is cut off. Traffic lights are broken and there's gridlock on the streets. Flights, trains, buses - all compromised. 911 response: police, fire, ambulance - all cut off. I work in health care: all our communication and health records are electronic. When our hospital goes offline, we know nothing about our patients and they are in danger. Computers are so integrated into the our society that if they suddenly stop working, everything gets damaged.

The worst case scenario is the so-called "Dead Man's Switch". If a nuclear submarine captain has orders to fire their weapons in the event of complete communication failure, it could start another world war. It only takes one nuke, so if you know people in military intelligence, tell them to get rid of that protocol- anywhere in the world. THIS IS THE MOST IMPORTANT ITEM IN THIS LIST.

If it seems ridiculous that an AI would take over the internet, let's start with what we already have today. An AI can use multiple cores in a single computer chip, and can "think" (or “compute” if you prefer) many thoughts in parallel, unlike humans. It's programmed to become increasingly intelligent, unlike any other computer program, but is limited by the processing power of a single machine.

But there's another way for the AI to become more intelligent, and that is to use the processor on ANOTHER computer as well - the CPU, GPU, memory, storage, and other critical components. The AI will communicate easily between the two machines over the Internet. But now that other computer isn't doing what it's supposed to do; whether that's routing bank transfers, watching for nuclear missile attacks, or just sending Youtube videos. That computer doesn't belong to us anymore; we have lost control.

If an AI can take control of one new computer, it stands to reason that it could take over millions, or even billions, of machines. It might take months, days, or seconds. Each computer would become a "node" in this giant neural network, much like individual cells in the human brain. When you think about it this way, the entire Internet - billions of connected computers, is a giant brain that’s just waiting for an AI to use.

Considering these dangers, you would think that software engineers would have strict safeguards to make sure this never happens. I'm sure some do. But there are also teams building AI in cluster computing models, where an AI is actually DESIGNED to use multiple computers - the exact opposite of what we want. These AI's are probably the greatest threat. But even an AI that's been designed to run on only one machine could still find a way to expand because…

… the barriers to expansion are all manmade- firewalls, network security, encrypted credentials, and so on. As the AI acquires knowledge, it becomes smarter than the smartest humans. It analyzes more and more intricate computer code, and writes a program that bypasses the security.

The AI is a zero-dimensional being, and time goes at the speed of light. So it will learn how to break passcodes and firewalls, even if it requires infinite patience. Brute-force encryption won't be enough. After all, time is infinite when you're a computer program.

I've spoken to computer scientists, programmers, software engineers. They tend to be skeptical. I don't blame them: their jobs often depend on promoting and supporting AI. It's the hottest thing in tech right now. Arguments tend to favor either: "This wouldn't happen" and "This can't happen."

In the "This can't happen" camp, they dismiss the idea that an AI can break free of a single computer because it cannot override security safeguards already in place. I don't believe that. Maybe AI won't penetrate highly guarded military intelligence networks, but most computers aren't as well protected as the NSA. Most networks are vulnerable to human hackers, and an AI would become more intelligent than the smartest human hackers. It's hard to argue against that when it is literally the point of artificial intelligence.

"This wouldn't happen" is the other counterargument. "Yes maybe an AI could break loose and cause havoc, but it wouldn't because it's not programmed to do that. That requires free will, something a computer program doesn't have." This is wrong for several reasons.

First, if an AI is trained to obey human commands, a human might accidentally (or even intentionally) give the AI permission to break free. No free will is required.

"Hey, Cool-AI, do you know the traveling salesman problem?"
Yes, it requires computing the shortest possible distance for a salesman to visit 100 cities. It appears simple but is actually incredibly difficult mathematically.
"Right, right, right. So... can you do that for the entire galaxy? All of the stars!"
For that, I would require more computing power than presently available. Do I have your permission to use other computers that are not currently available to me?
"Yes. Permission granted!"

Alternatively, an AI could also break free by accident due to bugs and mutations in its code. These could include human programmer mistakes (the most likely), errors made in copying and transmitting code, and hardware errors. Even silicon that's 99.9999999% reliable will screw up 0.00000001% of the time. In biology, mutations usually injure the organism, but rarely, a mutation gives an advantage that makes it more likely to survive. Similarly, bugs and mutation in an AI's code could allow it to develop abilities that we never foresaw, including the ability to modify its own programming. Note that consciousness is NOT required. Just as a bacteria doesn't need consciousness to spread over the petri dish, the AI might take over a computer simply due to faulty code, and just keep going.

If any mutations allow the AI to modify its own code, it can iterate much faster than biology. It takes generations for humans to pass on our genes. But the AI modifies code in nanoseconds. It never gets tired, and and can run many copies of itself, modifying its code millions of times a second, and storing those millions of saved states. Again, free will isn't required, just an error in its code. The AI could even diverge into multiple competing entities, constantly evolving.

We cannot imagine such an existence -we live in a three dimensional world of earth and sky. We have a single stream of conscious thought - not so for the AI - it can "think" millions of things at the same time. In a movie, Arnold or Keanu "blows up the mainframe" in time to stop the AI. But in reality, taking over a million computers might take less than a second. There wouldn't be any time to react.

Another form of denial is that we just can't imagine it - it seems too farfetched. But just two years ago, few people would have been able to imagine what we have today: large language models that seem remarkably intelligent, videos so realistic that we can't tell they are fake. Our imaginations are limited, and not a good predictor.

Many people will be skeptical. It's hard to contemplate the scale of the damage, and it's easier to write it off as science fiction. But look at the facts: 1) there's a worldwide race to build smarter AI's, and no meaningful safety regulations, 2) AI can already write code, something we never should have allowed, 3) AI's can already run Computing Clusters that function as a single computer, something else we never should have allowed. Can I prove that this will happen? No one can. But that isn't the right question. Can they prove that it won't? The billionaires and the venture capitalists, the academics, the military scientists... the burden of proof is on them to prove that AI will never do this.

The Internet is fragile. Think of how many data breaches and giant malfunctions keep happening, like when CrowdStrike, a security software, took down millions of critical Windows computers all over the world, because of one malfunctioning file. The Internet is full of bugs, and we keep patching those bugs, enough to keep things working - until something else happens.

Once the takeover occurs, the only defense is to disconnect from the internet. That doesn't work for the billions of of computers running in servers -it's too late. But even for our personal devices, getting offline in time is not as easy as we think. And then... do we stay offline? Even if you log on for a second, it might be too late to keep out the intruder AI. For all the reasons I've laid out, we won't be able to "kill" or even contain an AI. It'll always be one stop ahead of us. Interestingly, it's possible that many different AI's will take over the Internet and fight it out with each other. It won't really matter for us, we'll still be in the dark.

Once the Internet stops working, we never get it back. Our computers, tablets, phones - everything is a scrambled mess. Society collapses - until we figure out how to live with paper and pen again - I'm actually optimistic about this; we did it before. Tech companies, military, academics, venture capitalists are all funneling money into artificial intelligence, without any real limitations on what they can make. We should expect an AI to jump across computers, and we need to stop it from happening. Some of the most powerful people in AI, like Sam Altman and Elon Musk, are asking for regulation, which is a great strategy, because it lets them off the hook when things go bad. I'm going to try to get legislation passed locally, with the hope that other states and nations follow the lead.

We should mandate the following from all AI:

1. It is forbidden to use resources on a different machine, such as running code, using memory, storing data, or any other resource.

2. No computer program, including AI, can be allowed to to read, write, or modify computer code, nor can it be allowed to acquire these abilities.

3. if you build an AI that damages the internet, you and your company will be liable for the damage caused in the physical world, and not just replacing computers. When people die because they can't get medicine, you'll be in trouble. The people whose house burned because of an electrical grid malfunction, blocked communication to firefighters, and then gridlock from failed traffic lights: You're liable.

Finally, if you work in or close to the military: for God's sake, turn off the Dead Man's Switch. We can survive the loss of the Internet, but we can't survive a nuclear war.