Recently I've been hearing a lot about compute reallocation, and tbh, it seems like a bad way to go about slowing down AI progress. epistemic status: Cunningham’s Law
To be clear: I think that we need to slow or stop AI development and I'm all for an practical solution that gets us closer to that. What I'm against is compute reallocation: the idea that we should limit what percent of a companies available compute they can spend on R&D. Specifically Pacing AI call to limit labs to spending at most 15% of their available compute on "capabilities R&D".
Proposed of in:
- Pacing AI via Compute Reallocation by Thomas Larsen & romeo of AI 2027 fame
- How California can “pace the frontier” by Secure AI Project of SB 53 fame
Instead of Compute Reallocation, I think it's preferable to have a Compute Cap, where we limit the net amount of compute available to R&D research.
- The thing we actually care about is the how powerful models are. Regulating that is very difficult so instead we want to target AI inputs. Of those compute is the easiest to target. We can ~calculate how much an increase in compute corresponds to an increase in capabilities and right now I'm getting the impression that people backing this policy are using those number to calculate . Why not cut out the middle man and directly set the limit we want in the first place that we think would be safer?
- If there is a sudden break through in data center construction (ex: learning how to train on lower end GPUs), that won't translate into unforeseen capability gains.
- Doing net amount decreases the incentive to build new data centers, meaning it creates (hopefully) less of a compute over hang. I suspect the demand is so large regardless that the limit remains construction speed, but in theory this is still a demand decreasing move.
- One of the common concerns about AI regulation is that the big 5 will use it to make it impossible new labs to join the game. By setting a limit you make it easier for new people to join since they will only have to get enough computer to get up to the limit to be competitive rather than competing on equal footing with the major labs. Is it good to make it easier for their to be more frontier labs? Bad? I'll leave that as an exercise to the reader.
- When I asked Claude about reference class policies for input control, most of the examples it found across nuclear, arms & environmental controls use caps. None used this kind of percent allocation. That doesn't mean this is bad, but I would like to see more justification for taking a weird strategy as opposed to taking the more standard route.
Possible counter arguments in favor of Compute Reallocation:
- It's easier to implement -> No, it's not. Either way we need to measure how much compute their using, and doing a cap makes it in fact easier since you don't need to measure all other compute.
- Percentages let the number naturally rise over time -> Do we want that? If we do it makes more sense to me to set a compute inflation rate (say +25% a year?) where we can control based on actually safety research than based on however fast the market can build capacity.
Caveat: I think this makes sense when it comes to reallocating too safety work. ie: 15% of available compute must be used for safety work. I just don't think it makes sense as a limiting factor for the training. (caveat to the caveat: safety vs capabilities can be a soft distinction)