LW - SB 1047 Is Weakened by Zvi

The Nonlinear Library: LessWrong

Link to original article
Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: SB 1047 Is Weakened, published by Zvi on June 6, 2024 on LessWrong. It looks like Scott Weiner's SB 1047 is now severely weakened. Some of the changes are good clarifications. One is a big very welcome fix. The one I call The Big Flip is something very different. It is mind boggling that we can have a political system where a bill can overwhelmingly pass the California senate, and then a bunch of industry lobbyists and hyperbolic false claims can make Scott Weiner feel bullied into making these changes. I will skip the introduction, since those changes are clarifications, and get on with it. In the interest of a clean reference point and speed, this post will not cover reactions. The Big Flip Then there is the big change that severely weakens SB 1047. 1. 22602 (f)(1): Definition of covered model changed from trained with at least 10^26 flops OR a model expecting to have similar capabilities to what 10^26 flops would have gotten you in 2024 "was trained using a quantity of computing power greater than 10^26 integer or floating-point operations, AND the cost of that quantity of computing power would exceed one hundred million dollars ($100,000,000) if calculated using average market prices of cloud compute as reasonably assessed by the developer at the time of training." 2. On and after January 1, 2026, the dollar amount in this subdivision shall be adjusted annually for inflation to the nearest one hundred dollars ($100) based on the change in the annual California Consumer Price Index for All Urban Consumers published by the Department of Industrial Relations for the most recent annual period ending on December 31 preceding the adjustment. 3. Later: They will also publish the annual inflation adjustments. Bolded text is exact, except I capitalized AND for clarity. The AND, rather than an OR, makes my heart sink. Effectively, the 10^26 requirement is dead. Long live the $100 million. Where the law previously strengthened over time, now it weakens further. It starts weakening this year. The cost for buying one-time use of 10^26 flops of compute seems likely to fall below $100 million this year. Consider this from Jack Clark, where he got napkin math of $70 million a few months ago, or $110 million if you rented A100s. Jack clarified on Twitter that he expects B100s to offer a large further cost reduction. The compute minimum to be a covered model will begin to rise. The strength of non-covered models then rises both with the fall in compute costs, and also with gains in algorithmic efficiency. The previous version of the bill did an excellent job of handling the potential for Type I (false positive) errors via the limited duty exemption. If your model was behind the non-hazardous capabilities frontier, all you had to do was point that out. You were good to go. Alas, people willfully misrepresented that clause over and over. In terms of the practical impact of this law, the hope is that this change does not much matter. No doubt the biggest models will soon be trained on far more compute than $100 million can buy. So if you train on what $100 million can buy in 2026, someone else already trained a bigger model, and you had a limited duty exemption available anyway, so you not being covered only saved you a minimum amount of paperwork, and provides peace of mind against people spreading hyperbolic claims. What this does do is very explicitly and clearly show that the bill only applies to a handful of big companies. Others will not be covered, at all. If you are spending over $100 million in 2024 dollars on compute, but you then claim you cannot comply with ordinary regulations because you are the 'little guy' that is being stomped on? If you say that such requirements are 'regulatory capture' on behalf of 'big tech'? Yeah. Obvious Nonsense. I have no intention of pretend...