Evaluating AI Providers' Frontier Safety Frameworks
Summary
A study assessing 12 frontier AI safety frameworks using 65 criteria, finding low median scores (18%) and limited effectiveness as accountability mechanisms.
Key quotes
current Frameworks are limited as accountability functions, with vague commitments that make it difficult to predict company decisions
Overall scores range from 34% (Anthropic) to 8% (Cohere), with a median of 18%
The paper analyzes the gap between existing frontier AI safety frameworks and established risk management principles from high-risk industries like aviation and nuclear power. It evaluates risk identification, analysis, treatment, and governance.