(context below)
It was good to read your words; it was even better to see that 180+ million more people have seen your words.
What did you feel when you realised the only sound option left to you was to write that resignation letter? Did staying become scarier than leaving, scarier than taking a bold action that some of your peers don’t yet understand and giving up a life you’d worked hard for?
Congratulations. This must have been a difficult decision. One of us remembers quitting his PhD to volunteer full-time for PauseAI. A scary decision, but it feels good to speak the truth. To do the right thing.
So now it’s our turn to sit at our laptops, in the dead of night, to write a letter, thinking of you and appreciating you doing the right thing. Hundreds of millions of other humans are learning why you made your decision. The clear and present danger you and we see is dawning on them. They are scared, confused, some of them angry, some in denial. It’s already setting some things into motion, judging by how some people with large amounts of influence are responding.
Lab employees have left their jobs to sound the alarm before you, and the world wasn’t ready to listen. But now the time has come, and your actions are more important than ever. The window between the time when the default trajectory of AI with capabilities surpassing our own was still considered sci-fi and the point of no return, when it becomes too late for us to protect life on Earth, is short.
Every one of the people who work at or support frontier labs are dragging the world into the endgame you rightly call hubristic. But we’re not in checkmate yet. In some ways, things are simpler now. If we still want to win, our moves are clear. It’s becoming obvious how every single move counts.
There’s a Jack London story called To build a fire. A man is trying to light a fire on a remote snowy trail in dangerously low temperatures; his hands are freezing and he keeps fumbling with the matches. If he can’t get a grip on them, there’ll be no fire, and he’ll slowly but definitely die. For us, too, the danger is beginning to bite; the perfectly rational fear of being beyond the reach of safety is real.
Our hands still work; right now we’re still holding some matches. But as we’re naively wasting precious time “attempting to speedrun alignment”, as you said, our grip on the solution - pausing or banning general-purpose AI development - is getting shaky. The point of no return probably won’t announce itself.
By being honest about the severity of the problem, and not merely “express[ing] fear privately”, you’ve made your renunciation of the race the loud and clear message many needed to hear. Thank you, Jacob. We hope others who are considering forsaking the labs are noticing how impactful that honesty is.
Anytime people are polled on their views on AI, the vast majority of us oppose uncontrolled development and want it to be regulated, banned or paused until it is safe. And so, rising numbers of people like us, all around the world, are working tirelessly to build the great levee of coordination, an edifice robust enough that the tidal wave of this race can disperse onto it on impact.
Most of us can’t reach 180+ million people via a tweet. You can. You have the engineer’s legitimacy in warning about the clear and present danger we see in the machine. So what’s next, friend?
We invite you to join us. You’ve taken the first step in the right direction, and you see the power that step had. Now it’s time to walk that path in full.
Raise the alarm loudly.
Come speak at a protest, and become a spokesperson for a pause.
Make the case for a pause to Congress.
Find some way to explain the situation to Donald Trump.
Tell the truth when you get interviewed, the whole truth about what will actually constitute a robust solution.
Don’t do what too many others have done, that thing you called out - acting like the situation is just a little concerning when we’re actually staring into the abyss. That pretense, the many cover-ups, the “safety theatre” - all those distractions make the challenge harder and the window shorter. Luckily, we have each other. We’re here to support you, so do reach out to us. Let’s amplify each other’s voices. If nothing else, we’d love to see you at our next London march on 5 December.
Right now the light of the history books is shining upon us, throwing every action into sharper relief. In important times in history, people decided to go out onto the streets to protest, stood up amidst silent peers, rejected the orders of their superiors, didn’t press buttons which would have launched nuclear missiles.
Whether inside or outside of frontier labs, right now we all find ourselves in similar positions. Getting up in the morning or eating lunch, the people in the history books often did not know they were living one of those days, or what their choices would beget. Likewise now, the light of history is shining on each of us. Together, let’s make our moves count.
Warmly,
Matilda da Rui (PauseAI), Benjamin Schmidt (PauseAI Germany), and Markus Weiss (PauseAI Germany)
Some context for everyone else
On 9 September, ex-OpenAI and Anthropic researcher Jacob Coxon resigned from his job. He explained his decision with a five-post thread on X that has received almost 200M views in just over a day:
Anthropic’s alignment science lead, Evan Hubinger, replied, concurring with Jacob’s assessment. He believes there is a >10% risk of AI killing us all within the next decade. That’s right: the person in charge of one of the labs’ attempts to conjure AI safety up into existence says it’s not unlikely that their products will “kill all humans” in the very next few years.
You can be like Jacob
You can choose to take inspiration from Jacob’s example. Acting is never easy when your peers aren’t doing it. You may not have 180+ million people viewing your tweets, but to some people around you, you are the most powerful messenger. By informing them and modelling courage, you give them a chance to empower themselves. Join our movement.
> Likewise now, the light of history is shining on each of us. Together, let’s make our moves count.
I am personally curious about why Jacob and his former colleagues, who all seem in agreement about the danger and the need for a pause, don't organize collectively, for example striking until a slowdown or pause is in place. I am likely just showing my ignorance, but collective action can be a powerful lever for change, and it can be organized across labs and even countries