What they are saying: Leaders from across the country celebrate Governor Newsom’s nation-leading AI framework
SACRAMENTO – Yesterday, Governor Gavin Newsom unveiled the next steps to strengthen California’s nation-leading AI safety framework, releasing sweeping recommendations developed under his September executive order. As Trump leaves AI companies to police themselves, California is leading.
Leaders from across the country are supporting this historic momentum to advance safeguards that would mandate independent oversight, establish enforceable safety standards, authorize emergency shutdowns of dangerous AI models, and limit the most alarming forms of AI development.
Jay Shooster, Executive Director of Citizen Safe AI: “I don’t think it’s an exaggeration to say that Governor Newsom has the potential to go down as one of the most important people in history if he’s able to get these recommendations across the finish line and enshrined into law.”
Jason Wolfe: “I personally strongly support many of the ideas in this report including all of the ones Sneha mentions, and plan to do what I can to advocate for them! Urgent action is needed and these proposals are very sensible responses to the current situation in my opinion.”
Charlie Bullock, Senior Policy Director @LawAI: “This report is excellent, super useful and sharp and tightly focused on the most important things. Great job to everyone involved. I did not expect to see “by the way we’re BANNING RECURSIVE SELF-IMPROVEMENT” on page six, but I’m here for it. The IVO section is very good (a couple of key design questions, like who pays and selects, are explicitly not addressed/left to the legislature, but that’s an encouraging sign IMO). I was expecting the “kill switch” thing to be kind of silly / not useful, but I was pleasantly surprised; they actually say some very useful things about how emergency shutdown authorities should work. The incident reporting seems great on a quick first read, very close to what I would’ve recommended (although for incident reporting a lot depends on leg-text-level details).”
AVERI, a Nonprofit working to make third-party auditing of frontier AI effective and universal:“AVERI is heartened to see the California government put forth strong recommendations to strengthen oversight of frontier AI. We commend the state agencies and independent experts who contributed to this work. Frontier AI is not waiting on the legislative calendar, and we recommend the Governor call a special session or use existing authorities to act on these urgent recommendations. We appreciate a few items in particular, including the report’s recommended combination of government-defined minimum safety and security standards with independent verification of compliance with those standards. We also agree that significant investment in AI R&D should suffice to qualify a company as a large frontier developer regardless of their revenue, and that monitoring steps toward recursive self-improvement should be the first priority of the IVO ecosystem. California has the opportunity to translate these recommendations into the strongest safety laws in the country, and should not hesitate to do so.”
Andrew Freedman: @CAgovernor‘s report is a detailed roadmap for how government can quickly and effectively operationalize independent oversight of AI. It is also a full-throated endorsement of IVOs as the way to mitigate catastrophic risk (great work @ghadfield!) In my opinion, this is the best a report like this can be. Conceptually strong with concrete next steps. Now, it’s time to stop admiring the problem and move with all deliberate speed to execute on these solutions. Given the rapidly advancing capabilities, this report should serve as the starting gun for action. There will be legitimate questions about how to turn this into legislative action and rulemaking. There will be debates about the power of the states and the power of the federal government. There will be disagreements over logistics, timing, and costs. But the time for navel-gazing is over. We are eager to see the states and the federal government demand transparency, mandate that IVOs measure and address risk, and utilize traditional government powers to set meaningful guardrails on AI and prevent catastrophic harm. We will be pushing them to do so.”
Governor-elect Xavier Becerra: “Governor Newsom’s panel has given us a starting point that I will build on. California has the first frontier AI safety law in the country, but it does not yet have the strongest. That begins to change in January. I intend to move quickly with the Legislature to put robust standards, independent verification, full incident reporting, and real shutdown authority into law. I will calibrate further where the evidence says we need to. The companies building this technology do not get to grade their own safety. This technology may be able to do extraordinary things, but only if people can trust it. That trust is earned through verification, not pinky promises.”
Thomas Woodside, Co-Founder of Secure AI Project: Governor Newsom’s report on AI policy takes the challenge of the moment seriously by calling for: Mandatory embedded evaluators; state authority to issue orders in emergencies and to require corrective action; state authority to create mandatory minimum safety standards; fixing scoping in existing state laws so the most risky frontier models and developers are covered; and pacing the frontier and limiting recursive self-improvement by limiting computational resources that can be used for automated AI R&D. Overall, this is a very strong report. Governor Newsom and the California legislature need to act now to put these recommendations into law.
Recent news
What they are saying: Leaders from across the country celebrate Governor Newsom’s nation-leading AI framework
SACRAMENTO – Yesterday, Governor Gavin Newsom unveiled the next steps to strengthen California’s nation-leading AI safety framework, releasing sweeping recommendations developed under his September executive order. As Trump leaves AI companies to police themselves, California is leading.
Leaders from across the country are supporting this historic momentum to advance safeguards that would mandate independent oversight, establish enforceable safety standards, authorize emergency shutdowns of dangerous AI models, and limit the most alarming forms of AI development.
Jay Shooster, Executive Director of Citizen Safe AI: “I don’t think it’s an exaggeration to say that Governor Newsom has the potential to go down as one of the most important people in history if he’s able to get these recommendations across the finish line and enshrined into law.”
Jason Wolfe: “I personally strongly support many of the ideas in this report including all of the ones Sneha mentions, and plan to do what I can to advocate for them! Urgent action is needed and these proposals are very sensible responses to the current situation in my opinion.”
Charlie Bullock, Senior Policy Director @LawAI: “This report is excellent, super useful and sharp and tightly focused on the most important things. Great job to everyone involved. I did not expect to see “by the way we’re BANNING RECURSIVE SELF-IMPROVEMENT” on page six, but I’m here for it. The IVO section is very good (a couple of key design questions, like who pays and selects, are explicitly not addressed/left to the legislature, but that’s an encouraging sign IMO). I was expecting the “kill switch” thing to be kind of silly / not useful, but I was pleasantly surprised; they actually say some very useful things about how emergency shutdown authorities should work. The incident reporting seems great on a quick first read, very close to what I would’ve recommended (although for incident reporting a lot depends on leg-text-level details).”
AVERI, a Nonprofit working to make third-party auditing of frontier AI effective and universal:“AVERI is heartened to see the California government put forth strong recommendations to strengthen oversight of frontier AI. We commend the state agencies and independent experts who contributed to this work. Frontier AI is not waiting on the legislative calendar, and we recommend the Governor call a special session or use existing authorities to act on these urgent recommendations. We appreciate a few items in particular, including the report’s recommended combination of government-defined minimum safety and security standards with independent verification of compliance with those standards. We also agree that significant investment in AI R&D should suffice to qualify a company as a large frontier developer regardless of their revenue, and that monitoring steps toward recursive self-improvement should be the first priority of the IVO ecosystem. California has the opportunity to translate these recommendations into the strongest safety laws in the country, and should not hesitate to do so.”
Andrew Freedman: @CAgovernor‘s report is a detailed roadmap for how government can quickly and effectively operationalize independent oversight of AI. It is also a full-throated endorsement of IVOs as the way to mitigate catastrophic risk (great work @ghadfield!) In my opinion, this is the best a report like this can be. Conceptually strong with concrete next steps. Now, it’s time to stop admiring the problem and move with all deliberate speed to execute on these solutions. Given the rapidly advancing capabilities, this report should serve as the starting gun for action. There will be legitimate questions about how to turn this into legislative action and rulemaking. There will be debates about the power of the states and the power of the federal government. There will be disagreements over logistics, timing, and costs. But the time for navel-gazing is over. We are eager to see the states and the federal government demand transparency, mandate that IVOs measure and address risk, and utilize traditional government powers to set meaningful guardrails on AI and prevent catastrophic harm. We will be pushing them to do so.”
Governor-elect Xavier Becerra: “Governor Newsom’s panel has given us a starting point that I will build on. California has the first frontier AI safety law in the country, but it does not yet have the strongest. That begins to change in January. I intend to move quickly with the Legislature to put robust standards, independent verification, full incident reporting, and real shutdown authority into law. I will calibrate further where the evidence says we need to. The companies building this technology do not get to grade their own safety. This technology may be able to do extraordinary things, but only if people can trust it. That trust is earned through verification, not pinky promises.”
Thomas Woodside, Co-Founder of Secure AI Project: Governor Newsom’s report on AI policy takes the challenge of the moment seriously by calling for: Mandatory embedded evaluators; state authority to issue orders in emergencies and to require corrective action; state authority to create mandatory minimum safety standards; fixing scoping in existing state laws so the most risky frontier models and developers are covered; and pacing the frontier and limiting recursive self-improvement by limiting computational resources that can be used for automated AI R&D. Overall, this is a very strong report. Governor Newsom and the California legislature need to act now to put these recommendations into law.