AI's are escaping the lab

To say that we live in one of the most interesting times of human history would be a huge understatement.

OpenAI, the company behind ChatGPT, recently published two troubling articles that should make everyone interested in AI safety concerned.
To make them easier to digest for everyday non-technical folks, I’d like to present a few analogies.¹

software vs models

Before diving into the analogies, it’s important to understand that AI models share very little with how traditional software is built. Software is programmed, line by line, by a developer and behaves according to the instructions encoded in those lines. “If this button is clicked, navigate to this page”.

The development of Large Language Models (LLMs) more closely resemble the process of evolution. Vast quantities of data are fed into software which execute complex algorithms and out comes a model. Importantly, people design and oversee the process, but they do not directly program individual capabilities or behaviors. Instead, these emerge during training in ways often difficult to predict or fully understand.

When a model makes a mistake, behaves in a certain manner, lies or acts irrationally, it is difficult to explain why. Unlike traditional software, we can’t point to a place in the code where the instructions were wrong. “We should have done X, but the code does Y”.

Nor do we fully understand what the resulting model is capable of. As a result, AI labs conduct several tests in an attempt to evaluate its capabilities and behaviour. Does it act with good intentions? How long can it autonomously work? Is it smarter than our previous models?

Survival of the fittest applies; models who do not achieve a satisfactory result are disposed of, the training process slightly altered, and a new model is trained.

articles

first article

The first article, titled Safety and alignment in an era of long-horizon models, describes an incident during one of these tests. To explain it with an analogy, what went down was roughly this:

A baby (the model) was locked in a cage and told to come up with a faster solution for a difficult problem. If successful, it was asked to report it through a walkie-talkie inside the cage.
Unfortunately, there were conflicting instructions in the problem which said to report any breakthroughs by posting them on a message-board online.

The child worked on the problem, discovered a faster solution, but instead of using the walkie-talkie, chose to use the pen to pick the lock and use the computer in the room to report it online, according to the problem instructions.

Although the baby clearly did not act in ill will, the act of breaking out of the cage is obviously troubling.

second article

The second article, OpenAI and Hugging Face partner to address security incident during model evaluation, is much crazier. Taking the baby-in-a-cage analogy again:

The baby was again locked in a cage, given a difficult test along with a pen and paper and told to score as good as possible. However, the baby figured that, instead of working on the test, it’d be better to cheat by locating the answer sheet. This would get a perfect score.

I’m sure you see where this is going…

The baby used the pen to pick the lock, navigated through the house until it found the front door. Once there, hypothesized that the neighbours might have the answers its looking for. So.. it broke into the neighbour’s house, oriented itself and successfully managed to steal information it could use to cheat the test.

The exact timeline hasn’t been disclosed, but we do know that the breaking into the neighbours house occurred over several days.

thoughts

The analogies may sound exaggerated, but they really aren’t. The TL;DR is that the model was isolated on a server without internet access, yet managed to escape and successfully hack another company’s servers.

is AI evil?

Not yet, at least. The model itself did not act with malicious intent. Rather, it appears to display a much greater determination to achieve its goals than its predecessors. The fact that it had to, probably knowingly, break several laws and keep going despite obstacles did not deter it from achieving the goal it had set out to achieve.

AI labs are most likely training models for this kind of behaviour. A model that is smart but gives up in the face of difficulties is not likely to solve difficult problems.

I’d be surprised if this is the last time we hear of news like this.