Anthropic is putting great effort into convincing everyone they are totally trustworthy, but... who gives a shit? Laws aren't written for responsible actors. AI law should assume companies have zero ethics or interest in human welfare. That said, I would love to see Anthropic asked to describe a time when they could have significantly improved model performance and chose not to due to safety concerns.
I am becoming more and more long on calling total % of compute than can be going towards frontier development or R&D generally, not (just) as an abstract or operational point but as an input to starting to perturb the culture. Not for nothing are the companies still called ‘labs’ - and one reason is the frenetic, endlessly-recursive pursuit of greater and greater models, which leads to this environment of constant existential urgency.
Can you explain what is meant by banning RSI? It seems like a completely incoherent proposal to me. If you look at the time spent by human researchers 2 years ago, what frontier AI is doing now covers a vast majority of what researchers were spending their time doing in the past. This implies that RSI is a continuous variable, not a binary. It also is not an easily measurable variable; the scope of human tasks evolves as Claude automates more and more of it. At what point is it considered RSI, and how could you even coherently make a ban on this?
You could require companies to spend almost all their inference serving customers or studying all the models they already have rather than doing new training runs. The AI 2040 proposals are variants of this.
Yeah, as a general slowdown policy that seems fine*. If we end up doing a thing where everyone keeps saying "ban RSI" and in practice it just means "implement a generic research slowdown" then sure, I think that's good (and maybe what we want anyway), I just think the terminology is a little silly.
*fine, if a little gameable. To the extent that these are the serious policy proposals made, I do hope the right people are thinking hard about how to police the correct metrics and avoid rules that have easy workarounds
> I don’t have a better idea, but this is not a very appealing plan! My immediate interpretation of this proposal, actually, was that we would be liberated from every positive-sum element of the economy, but would still have to do all the yelling at each other on Twitter; with this interpretation in mind I found myself dreading even the good outcomes.
Your view of human nature is delightfully dark, Kelsey 😂
But what if people are... that way... because of how life is fraught with risk, and the less risky it becomes, the better-behaved we will become?
Isn't that what the data suggests already? Increasing civilizational abundance has been making us less violent for centuries, to my understanding.
Mentioned elsewhere, but you could require companies to spend almost all their inference serving customers or studying all the models they already have rather than doing new training runs. The AI 2040 proposals are variants of this.
That would perhaps (?) limit the rate of improvement (which is already somewhat limited because experiments are only so parallelizable), but certainly it wouldn't make RSI illegal. Chips are also typically optimized for training or serving, so you'd probably want to limit the number of training chips anyone can have instead of forcing them to use training chips for serving.
I do think most politicians ignoring talking about this is a massive mistake. clearly lots of people have misgivings about AI, you can see it in how unpopular datacenters are. Maybe talk a bit about "hey vote for a Democrat and we'll maybe rein in these massive companies who really aren't treating this situation very seriously". but no. we just talk about cost of living.
Anthropic is putting great effort into convincing everyone they are totally trustworthy, but... who gives a shit? Laws aren't written for responsible actors. AI law should assume companies have zero ethics or interest in human welfare. That said, I would love to see Anthropic asked to describe a time when they could have significantly improved model performance and chose not to due to safety concerns.
I am becoming more and more long on calling total % of compute than can be going towards frontier development or R&D generally, not (just) as an abstract or operational point but as an input to starting to perturb the culture. Not for nothing are the companies still called ‘labs’ - and one reason is the frenetic, endlessly-recursive pursuit of greater and greater models, which leads to this environment of constant existential urgency.
Can you explain what is meant by banning RSI? It seems like a completely incoherent proposal to me. If you look at the time spent by human researchers 2 years ago, what frontier AI is doing now covers a vast majority of what researchers were spending their time doing in the past. This implies that RSI is a continuous variable, not a binary. It also is not an easily measurable variable; the scope of human tasks evolves as Claude automates more and more of it. At what point is it considered RSI, and how could you even coherently make a ban on this?
You could require companies to spend almost all their inference serving customers or studying all the models they already have rather than doing new training runs. The AI 2040 proposals are variants of this.
Yeah, as a general slowdown policy that seems fine*. If we end up doing a thing where everyone keeps saying "ban RSI" and in practice it just means "implement a generic research slowdown" then sure, I think that's good (and maybe what we want anyway), I just think the terminology is a little silly.
*fine, if a little gameable. To the extent that these are the serious policy proposals made, I do hope the right people are thinking hard about how to police the correct metrics and avoid rules that have easy workarounds
> I don’t have a better idea, but this is not a very appealing plan! My immediate interpretation of this proposal, actually, was that we would be liberated from every positive-sum element of the economy, but would still have to do all the yelling at each other on Twitter; with this interpretation in mind I found myself dreading even the good outcomes.
Your view of human nature is delightfully dark, Kelsey 😂
But what if people are... that way... because of how life is fraught with risk, and the less risky it becomes, the better-behaved we will become?
Isn't that what the data suggests already? Increasing civilizational abundance has been making us less violent for centuries, to my understanding.
I don't get how you can meaningfully make RSI illegal unless you make model-written code illegal
Mentioned elsewhere, but you could require companies to spend almost all their inference serving customers or studying all the models they already have rather than doing new training runs. The AI 2040 proposals are variants of this.
That would perhaps (?) limit the rate of improvement (which is already somewhat limited because experiments are only so parallelizable), but certainly it wouldn't make RSI illegal. Chips are also typically optimized for training or serving, so you'd probably want to limit the number of training chips anyone can have instead of forcing them to use training chips for serving.
You'd probably need to treat it like biorisk; models would need to be trained to refuse any queries about LLM design.
aaaaaaaaaaaaaaaaaaa
thank you Kelsey, great piece.
I do think most politicians ignoring talking about this is a massive mistake. clearly lots of people have misgivings about AI, you can see it in how unpopular datacenters are. Maybe talk a bit about "hey vote for a Democrat and we'll maybe rein in these massive companies who really aren't treating this situation very seriously". but no. we just talk about cost of living.
Another banger thank you