BLUF: There’s a structural pull to overinvest in cosying up with frontier labs relative to publicly holding them to account. Now seems like a good time to rebalance.
When thinking about influencing an organisation (like, say, a frontier lab), it’s useful to distinguish between the inside game and the outside game. The inside game means trying to work with the insiders: building trust and influence with them, and looking for compromises. The outside game means applying external pressure: building broad political will and being willing to take a more adversarial posture.
In an ideal world, you do a mixture of the two: the good cop and the bad cop, the carrot and the stick, and so on. But AI safety does not have some single strategy. Instead, people self-sort based largely on local incentives and ambient pressures. And I think this disproportionately pulls towards the inside game.

Éminence Grise by Jean-Léon Gérôme
Clearly there’s a complex spectrum of inside-game-ness from literally working on post-training in a frontier lab to vocally calling for all labs to be shut down, and indeed it’s less like a single axis and more like a very high-dimensional space. Still, I think the rich tapestry of strategies is pretty well-approximated by a single sliding scale, and that things pull closer to the inside than is optimal, e.g. junior safety researchers erring on the side of caution when it comes to criticising frontier labs.
Before getting into my guesses for why, let me start by observing some ways that the inside game has been historically more powerful and higher status:
- Coefficient Giving, by far the largest funder, is also really inside gameHistorically one of their largest grants was buying an OpenAI board seat (possibly more than all TAIS spending at the time)Their governance spending is basically all inside game — they’ve never directly funded e.g. pause advocacyThis probably gets more extreme with the huge wave of additional AI safety funding from Anthropic employees and the OpenAI FoundationThe extremely influential CEO left to join Anthropic (as did the head of worldview investigations)
- Historically one of their largest grants was buying an OpenAI board seat (possibly more than all TAIS spending at the time)
- Their governance spending is basically all inside game — they’ve never directly funded e.g. pause advocacy
- This probably gets more extreme with the huge wave of additional AI safety funding from Anthropic employees and the OpenAI Foundation
- The extremely influential CEO left to join Anthropic (as did the head of worldview investigations)
- Constellation, the traditional seat of CG’s catastrophic risk team and broadly-respected orgs like Redwood and AIFP, is very inside gameIt has many offices for people who work in labsAs I understand it, at a certain point it revoked access to MIRI for being too outside gameI think it is currently explicitly committed to the inside game
- It has many offices for people who work in labs
- As I understand it, at a certain point it revoked access to MIRI for being too outside game
- I think it is currently explicitly committed to the inside game
- A lot of community building efforts skew hard towards inside gameMy impression is that there’s a huge premium on getting a mentor who works at a frontier labI have in fact heard reports of people applying to safety fellowships mainly because they expect it to be a fast track to joining a labI have also heard various people fret about not wanting to antagonise Anthropic despite feeling like Anthropic Fellows are mostly just not doing safety work
- My impression is that there’s a huge premium on getting a mentor who works at a frontier lab
- I have in fact heard reports of people applying to safety fellowships mainly because they expect it to be a fast track to joining a lab
- I have also heard various people fret about not wanting to antagonise Anthropic despite feeling like Anthropic Fellows are mostly just not doing safety work
- A pretty large share of the most talented and capable researchers work for labsEven outside of labs a lot of the most respected researchers and thinkers used to work for labs, or have closely collaborated with them
- Even outside of labs a lot of the most respected researchers and thinkers used to work for labs, or have closely collaborated with them
- Technical work generally skews towards things that are useful for frontier labsThis is a fiddlier point and many specific cases are disputed but I think that a lot of work on evals or alignment post-training has mostly just sped up progress with no real change to safety
- This is a fiddlier point and many specific cases are disputed but I think that a lot of work on evals or alignment post-training has mostly just sped up progress with no real change to safety
- I often hear people express some private desire for the outside game to get more support from somewhere elseLightly anonymised: funders thinking it would be good if someone else funded pause advocacy, going back several years; frontier lab researchers saying they want more people to be advocating externally for slowdowns but don’t personally feel like they can publicly back itFrontier labs often saying they want a more mature third party evaluations ecosystem or higher quality critiques of their public writing
- Lightly anonymised: funders thinking it would be good if someone else funded pause advocacy, going back several years; frontier lab researchers saying they want more people to be advocating externally for slowdowns but don’t personally feel like they can publicly back it
- Frontier labs often saying they want a more mature third party evaluations ecosystem or higher quality critiques of their public writing
There are also plenty of people and groups playing the outside game, but anecdotally I often hear such people talk about being randomly excluded in a way that seems overall pretty harmful and even counterproductive — from events, from funding, from office, and even parties.
All the specific cases I know about seem legitimately thorny, and I don’t really know what to make of that fact. But, for example, it seems bad that Lighthaven has struggled to get funding despite being such a clearly useful piece of community infrastructure, for reasons that at least rhyme with “not committed to being conspicuously inside game”.
And having said all this about labs, I get the sense this dynamic is also playing out in DC, except the people in DC keep their heads low. At least from what I’ve heard, this has largely backfired and provoked a lot of witch-hunting.
Now, a reasonable objection at this point would be: actually maybe this is the right allocation. Maybe the inside game is a much better strategy, and maybe outside gamers are actually kinda weird and counterproductive and at risk of burning the commons for the rest of us, so the optimal portfolio is to mostly go for the inside game and mostly ignore loud but counterproductive critics. But it sure feels like there is important outside game work to be done, and like there’s a bunch of latent preference cascades.
Pacing is mainstream, regulation is back on the table, labs are getting sued, and the same cosiness that lets third party auditors have access without government backing might make it wholly inappropriate for them to be the government-approved neutral body. And of course I’d be remiss not to observe that the inside game has also plausibly burned a pretty large swath of commons.
It’s really hard doing counterfactuals. Was buying an OpenAI board seat a good idea? Would the world be better if AI control had caught the huggingface incident? Would the world be worse if METR hadn’t been able to look at the logs? What would have happened if CG had funded PauseAI Global a year ago? Probably it was good that Jacob Coxon had a role at a lab from which he could resign, but maybe it is just too soon to say.
Still, my hunch is that we are leaning way too much on the inside game — that it’s the natural groove everyone follows until they decide to do something else, and that people privately expressing some desire for more outside game work that they themselves feel ill equipped to do is a sign of something amiss. At the very least, the landscape has clearly changed and it’s not obvious that we are adjusting fast enough to it.
So why might this be happening?
Some guesses about what’s going on
The first-order cynical approximation is that people are just self-interested, but I don’t think it’s the whole story, and I think that having a richer picture can help us figure out more constructive interventions.
Firstly, I expect a lot of this is less like corruption and powerseeking and more like unconscious economics: all things being equal, a safety organisation that earnestly believes the solution is prosaic alignment and lab boosterism is going to have an easier time getting grants, attracting talent, and writing spiffy conference papers than one that earnestly believes the critical path involves a lot of abstract conceptual breakthroughs.
Similarly, I think there is a real problem where Anthropic jedi mind tricks people into believing that we can kludge our way to superintelligence and racing is unfortunately the only option, but also lots of people just already believed this, and it happens to be that they mostly self-sorted into working in labs.
And the upshot of this dynamic is, even leaving aside any corruption or people twisting themselves up into pretzels, you should still expect the power and influence to mostly accumulate to the inside game. However, I do think the actual situation is a complex mix of unconscious economics (i.e. sincere differences in belief), legitimate corruption, and people twisting themselves up into a pretzel.
With all that said, let’s try to get a bit more granular.
The inside game is where a lot of power naturally lies
This is hard to overstate, and only becoming more true. Inside there’s money, externally legible prestige, connections, and political influence, alongside information, model access, and a lot of compute. People want that, and people who have that can get more stuff done. And all these resources emanate out along a fairly smooth gradient.[1]
And indeed, people committed to the outside game might have an actively hard time getting resources. There is a colossal amount of money in AI risk now but if you want your organisation to be visibly independent of labs then your options are much more limited. If you choose to socially distance yourselves from frontier lab employees then information will leak to you much more slowly.[2]
All this compounds: Smart people want to be around smart people, and the inside game sure has talent density. And of course, the inside game wants to have the best people. The whole point of a vesting cliff is to make you stick around and to make you invested in the success of the enterprise.
There’s also this very general problem with AI risk that the feedback loops are just awful, terrible, no-good. It’s so hard to tell if you are making any progress towards making superintelligence go well, and crucially, it’s extremely hard to tell if anyone else is. This makes it pretty hard at a glance to tell which people are competent or worth trusting with, say, power and influence.
As a result the community often ends up falling back on the halo of influence that emanates out from frontier labs, funders, and the realm of alignment engineering. Many obvious signs of competence like top conference papers, prior work at/collaboration with labs, or even just being a big org with lots of funding all kind of flow from the inside. Perhaps we in the know can have some separate credit assignment that depends more on lesswrong karma and personal recommendations, and perhaps we are more interested in people who take those things seriously, but it is unfortunately harder to scale and much less externally legible.
The outside game is personally risky
In a very general sense, the inside game generates optionality. People talk a lot about how they want their organisation to be respectable so that it can, at the right moment, credibly pull the fire alarm. Working in a lab lets you accumulate resources, including political capital, which you can at some point actually spend down. It also famously gives you the option of dramatically quitting.
But the path is a lot smoother from the inside out than vice versa. Daniel Kokotajlo, Richard Ngo, and Jacob Coxon get to be heterodox critics, but Eliezer Yudkowsky does not get to become head of strategy at a frontier lab. At best you’re allowed to dramatically leave OpenAI and then hastily join Anthropic. There also appears to be some kind of kafkaesque cosmic-stakes secretary problem where lab employees periodically get presented with some new really horrendous norm violation and have to ask themselves whether this is the moment to resign or whether they should bide their time for something worse.
Relatedly, I sometimes get the vibe that some junior researchers are afraid of antagonising Coefficient Giving or the frontier labs, out of a sense that it might compromise their future job prospects. Taking a political stance is individually costly. And for the most part, these people are way overrating the costs, but it’s totally reasonable for them to be conservative given that they know they have only a weak grasp of the power landscape.
A very general problem with criticism is that (1) it is hard to capture the positive externalities and (2) you personally incur a lot of downside risk. When you loudly announce that something bad is happening, or even provide a bunch of thoughtful critique, there is some chance that basically all you do is antagonise some powerful people. And even if things do go well — even if your criticism persuades some massive organisation to pivot its whole strategy — they are not going to split their winnings. So it is both underincentivised and really high variance.[3]
The upshot is that the risk averse move is to bide your time, gather resources and complain in private. The problem is, this strategy does not scale very well to whole movements.
The dynamics are more cursed at scale
Individual risk aversion around seeming too outside game becomes much more pernicious when people are making quick decisions about groups or on behalf of organisations.
For example, the culture of lab employees leaking information at parties and events rests on a kind of implicit promise that that information won’t be used to attack them. So if you want to run a chill social gathering where this very valuable activity takes place, you also have to curate your guest list, and somewhat err on the side of caution.
Relatedly, if you work at a grantmaking organisation, one single politically volatile grant could massively compromise your reputation and access to funding, which pushes you to be conservative under uncertainty. I think this is part of why there’s so much more money for fairly corporate community building and technical work than there is for actual political action or really high-decoupling capacity building.
These dynamics also play out in hiring, curating coworking spaces, and even professionally collaborating. I’ve heard outside game folks grumble about having their inside game counterparts ask them not to let other people know they communicate, out of some sense that the outside gamers are politically clueless about their own apparent toxicity.
All this sums up to a weird chilling effect where there’s a lot more private appetite for the outside game than public support. Seems bad!

From The School of Athens by Raffaello Sanzio da Urbino
The Well-Respected Outside Game
So what do we do about all of this? What’s the problem? The inside game has clearly been beneficial in a lot of ways, and I don’t want to indiscriminately endorse all outside game work. There’s also an unfortunately narrow bottleneck between the safety community and the labs — if everyone reading this post decided to go find one lab employee to chew out, it would look less like all capabilities researchers having a small chance of facing a frank conversation and more like a few safety researchers getting dogpiled.
But I do think a few things might help on the margin.
Rebalancing the resources?
We can try to intentionally correct for the resource skew. There’s some good discussion on the topic in Jan Kulveit’s comment on Richard Ngo’s post about Pragmatism and Pessimisation: roughly, it would be great if we could give people credit for having had important, dangerous insights which they then did not use, or for not having joined labs even though they could have.
We can also try to cut the strings off various inside game resources so that they can be spent on outside game work without either tainting its outside game-ness or coming back to bite the inside gamers. Maybe someone with a pile of inside money should hand it to someone with good outside taste? I think Lightcone Commons is partly a step in this direction, which I’m pretty excited about.
A related piece of social tech is tenure. Maybe we should try picking some respectable people with heterodox tendencies and straight up tenure them, which in this timeline might only be a five year commitment, and see if it makes them loosen up.
We can also try to have a bit more courage about calling people out for playing the inside game in a way that actually just made things worse. I think we do a decent job of this but somehow it often seems not to stick.
And we can try to push for things which uniformly level the playing field, like greater transparency from the inside forces so that you don’t have to be playing the inside game to know what’s going on.
Insider courage?
I’d feel a lot better about this whole asymmetry if I had more confidence that the people tying themselves up in pretzels to get inside and making devil’s bargains were actually using their end of the deal to good effect. Maybe they are and it’s a secret, I don’t know. It’s an epistemically fraught position to be on the outside of it all, and one which I am planning to publish more thoughts on quite soon.
I do think people playing the inside game ought to spend some time thinking about what implicit beliefs their strategy rests on, and what red lines would make them reevaluate. I am generally not pro tying yourself to the mast, but it seems good to make it very easy to notice if your values are drifting.
To be a bit more pointed, if one reason that we’ve skewed so far towards the inside game is that people are holding out for the moment they make the pivot, well, that hour is probably drawing near. And if you, dear reader, are someone who went in with that intention and now believe that such a pivot does not make sense, it would be good for the collective epistemics if you could loudly announce this fact.
Subsidising good criticism?
I specifically would like to improve the incentives around criticism, which mostly means making it perceptibly less risky for people to advance criticism, less costly to develop it, and more rewarding to do a good job. The history here is pretty messy — to the curious, I greatly recommend Zvi’s criticism of the EA criticism contest, and again I hope to write more in future. But if you are the face of an otherwise faceless entity and you think there’s a chance you could be making a horrific error, I’d be happy to chat about strategies.
Conclusion
Nuance is hard, so let me really emphasise: I don’t think the inside game is inherently bad, or that everyone doing it is twisting themself up like a pretzel. But I do think we are overinvested for structural reasons that it is in everyone’s interest that we try to rebalance.
Of course, the outside game has its own siren song — the blazing moral clarity, the media attention, the creep of audience capture — and I think that if we properly rebalanced we’d start to run into this a lot more, but right now it seems like much less of a problem.
This topic is an inherently thorny one because I’m talking a lot about social miasma; I’ve done my best to obliquely reference evidence I’ve received directly, but no doubt there is a lot more I’m lacking, so I’m pretty curious to hear other people’s perspectives. And as I say, I feel much more time could be spent hashing out the solutions, which is a bet that looks increasingly good on ever shorter timelines.
- ^I am obliged at this point to once again bring up Ellsberg’s warning, which everyone should read in full: it will have become very hard for you to learn from anybody who doesn’t have these clearances [...] after a while you give it up and just stop listening.
- ^Ironically, the most interesting leaks I’ve been privy to about frontier labs have come by way of capabilities folks.
- ^Even in this piece I’ve found myself shying away from claims that are, depending on how you look at it, some combination of “needlessly provocative”, “hard to justify beyond illegible vibes”, and “too antagonistic towards people with power”. Ironic!