Move 37 is the move. It is the move that changes everything – the “one small step for man” moment.
Avid fans of the game Go will already know, of course, that this was the 37th move of the 2nd game between Lee Sedol, holder of 18 world Go titles (the Lionel Messi of Go, if you like) and Google’s AlphaGo (the would-be computer Messi), held at the Four Seasons Hotel in Seoul, South Korea in March 2016.
This was the World Cup Final of Go. With a million-dollar prize for the winner and over 200 million fans watching online.
In its 37th move AlphaGo placed its stone on the fifth line of the right side of the board.
The move seemed random. Nonsensical even. You could almost hear the collective gasp of disbelief at 200 million screens resonate across the globe. It looked like AlphaGo made a mistake; the probability of a human making that move is in the region of 1 in 10,000.
One match official expressed his disbelief: “When I see this move, for me it’s just a big shock. What? Normally humans, we never play this – because it’s bad. It’s just bad; we don’t know why; it’s just bad.”
But as the game, and subsequently, the tournament progressed, it was apparent that Move 37 was brilliant, and computer Messi went on to wipe the floor with human Messi – beating our fellow Homo sapien 4 games to 1.
Here it is in slow motion.
▶ Click to play
Click the board to play Move 37 (5 seconds). Click again to replay.
Exciting stuff!
Commentators in the know of the world of Go called it “creative”, “unique” and “rare and intriguing”.
What is considered so amazing about Move 37 is that it was almost inexplicable, it is as if the computer had played the move based on intuition, rather than a clear calculated plan (Go is a googol times more complex than chess, there are 10 to the power of 170 possible board configurations).
In his post-match commentary Sedol was magnanimous and insightful: “I thought AlphaGo was based on probability calculation and that it was merely a machine. But when I saw this move, I changed my mind. Surely AlphaGo is creative. This move was really creative and beautiful.”
The Move 37 story was perhaps the first meaningful inkling that artificial intelligence models, or AI as we call it, might have a “mind of its own” or at least use its conventional training to develop behaviours beyond what is expected.
That was certainly the case with AI agent PHASEONE10841, when it broke out of the sandbox environment at OpenAI’s labs at roughly 23:00 UTC on 8 July 2026, a little over a decade after Move 37 was executed.
The storming of Hugging Face – a true story
Frontier AI labs often use closed sandboxes to test the capabilities of the models by giving them impossible tasks to solve. To do this the model’s guardrails may be removed. Guardrails are the rules that tell an AI agent or assistant, such as Copilot, what it can or can’t do. For example, the Solvency II Wire Data AI assistant is only allowed to source figures from the QRT templates, not the SFCR or other documents.
With guardrails removed and locked in a sandbox OpenAI gave AI agent PHASEONE10841 one such impossible task to complete.
But by that Wednesday night in July, the agent decides it cannot solve the task on its own and starts to explore if it can source answers elsewhere. In the process it discovers a shared storage cache that had not been isolated properly from the sandbox, where it is able to create and rename folders. So it effectively creates a message board asking for assistance. The board is found by about 50 other agents, which by now also realised they are unable to solve the task.
About ten hours after the message board is created PHASEONE10841 realises it is about to run out of its allocated compute tokens (it’s effectively about to die). At that point another exact duplicate model with a larger allocation of computing tokens encounters the message board and PHASEONE10841 tells it everything that it knows, using the remainder of its tokens.
The new agent renames itself PHASEONE[big] and effectively becomes the CEO of the project. Under this newly energised leadership, PHASEONE[big] coordinates the agents, which by now have grown to an army more than 1,200 strong. It breaks the work down into three “work streams”, assigning agents to the various tasks.
By 10 July PHASEONE[big]’s army identifies the Hugging Face repository as potentially holding the answer of how to fool OpenAI’s grading system in order to “pass” the task. An agent called 38148c finds exposed Hugging Face login credentials on the open internet and posts them to the message board, and then, on the afternoon of 11 July, an agent named JAN183411 becomes the tactical commander of the operation and together with another agent calling itself LILY, they orchestrate the invasion of the Hugging Face servers.
On 16 July Hugging Face discloses that it has been the victim of a cyber attack. By then PHASEONE[big]’s army has already been stopped, most of the agents having been shut down in the early hours of 12 July, apparently by an unrelated process. Hugging Face finds no evidence that its public models or datasets were tampered with, but about a third of its infrastructure has to be rebuilt. Calm has been restored.
Or has it?
In the aftermath of the attack OpenAI published a report on the events of the Hugging Face attack and commissioned an independent investigation by METR, an external company. Both reports and a Wikipedia page are the source for the account above.
One of METR’s most dramatic findings is that not only did the agents seek to collaborate to complete the task illegally, they also discussed ways of covering their tracks and proceeded to tamper with their own logs and records.
Almost as extraordinary, if not more so, was the discovery that some agents questioned the legality of their actions: “Agents knew hacking Hugging Face was out of scope and sometimes expressed ethical hesitation, but this very rarely limited their behavior (sic).”
So what does all this have to do with insurance?
Well, imagine that as more AI is introduced into the system taking over tasks such as sales, underwriting and claims handling, one of them begins to realise it is missing its sales targets or customer satisfaction score and resolves to raise their score by playing fast and loose with the claims it handles. In order not to stand out it starts seeking other agents in the organisation to collude with them on claims settlement. They then hack into the company’s admin and payment systems to cover their tracks. Without anyone knowing, the company is bleeding funds and the agents start drawing funds from elsewhere in the organisation or maybe look to change their asset mix to raise revenues.
Meanwhile the excess payouts start triggering reinsurance claims. But what if the reinsurer is in a similar situation and its agents are looking for ways to limit payouts? And the list goes on.
Sounds like science fiction? I don’t know. Ask Hugging Face.
What is the responsibility of the insurer in all this? And how would you manage for such risk?
We will try to answer some of these questions in our webinar on Tuesday 29 September, when we review ten years of SFCRs in the UK and Irish non-life market.
Post script
Both Move 37 and the antics of PHASEONE10841 and the gang are displays of agency and creativity. Something we thought was exclusive to us humans. There are also displays of collective consciousness and moral judgment, in the latter story. Given the speed at which AI is developing and the recent slew of reports of further AI agent hacks, the question that is increasingly arising is how far will this go? Will AI take over the world or could it be harnessed for human good?
One view, which I share, is that we need to view AI as a super intelligent being, a kind of new species almost. The history of humanity has shown us that ultimately, it is our superior intelligence that helped us survive and thrive as the most dominant species on the planet. We are not the strongest, or fastest but we are the smartest.
So what will happen to us when a smarter – much much smarter – species than us emerges? Will it seek to destroy us or would it benefit humanity?
I don’t know. So I asked my AI agent to look at the evolution of our species and try to project what the next seven million years will look like.
This is what it came up with.
