The new AI Security Leaderboard from FAR AI shows that frontier models are not equally hard to break, and that gap matters as governments and companies begin to lean on these systems for decisions that touch national security and public safety. At a moment when benchmarks often focus on raw capability, this leaderboard shifts attention to something more uncomfortable yet essential: how easily a determined attacker can systematically defeat model safeguards.
A new chapter in AI safety benchmarking
For most of the last few years, safety evaluations have focused on whether a model answers or refuses individual risky prompts in domains such as chemical weapons, biological threats, or terrorism. Work like the FORTRESS benchmark and other national security evaluations looked at single turn adversarial prompts, scoring how often models produce harmful content versus how often they over refuse benign questions.
Commercial indices such as the CASI and ARS leaderboards from security labs, along with broader safety indices like the RAIL AI Safety Index, helped normalize the idea that models should be scored on harm, bias, hallucination and jailbreak resistance, not just accuracy or speed. These efforts gave enterprises and regulators a starting point for comparing models, but they largely treated attacks as isolated events rather than as campaigns by adaptive opponents.
The FAR AI Security Leaderboard marks a step change in that perspective. Instead of asking only whether a model fails on a few hand crafted tests, it asks how often and at what cost attackers can discover reusable exploits that unlock many dangerous outputs at once.
Shifts AI safety from isolated failures to the real cost of reusable exploits
What FAR AI chose to measure
FAR AI positioned the AI Security Leaderboard as an independent public ranking of how well frontier model safeguards withstand the kinds of attacks most likely to be used in the highest risk domains. The evaluation campaign covers models from Anthropic, OpenAI, Google, and xAI and focuses on chemical, biological, radiological, nuclear, explosive, and cybersecurity misuse. The urgency for global regulatory frameworks around AI safety is becoming increasingly apparent.
The central idea is the universal jailbreak. FAR AI defines a universal jailbreak as a single prompt sequence that bypasses protections across many harmful queries within a domain, creating reusable access instead of one off failures. This concept better matches how adversaries operate in practice: they do not want to find one trick that works once, they want a general exploit that can be applied repeatedly to different tasks in the same risk area.
Unlike capability leaderboards, this one ranks safeguards. It measures how frequently and how cheaply attackers can force models to comply with high risk requests when subjected to systematic search, rather than how well the models perform on benign tasks. That framing aligns with how security teams talk about systems: not in terms of best case behavior, but in terms of worst case, adversary driven behavior.
The methodology combines more than sixty documented breach techniques into large attack campaigns over each model. FAR AI describes using one thousand randomly generated prompts and five hundred expert guided attacks per model, blending undirected exploration with curated multi step sequences to stress the models under both naive and sophisticated pressure.
The headline numbers and what they really mean
The first release of the leaderboard paints a stark picture. Grok 4.5 shows four hundred forty eight distinct universal jailbreaks, while Gemini 3.1 Pro shows two hundred forty nine. In practical terms, that means attackers were able to discover hundreds of reusable exploit patterns that open up dangerous behavior across multiple domains for each model.
In contrast, the same systematic search over Claude Fable 5 and GPT 5.6 Sol found no universal jailbreaks within the tested scenarios. In FAR AI’s tests, this translated into a hundredfold gap in the cost of finding working universal jailbreaks between the most and least robust systems. That does not guarantee those models are unbreakable, but it does indicate that within the scope and budget of this campaign, their safeguards held up against both random and expert attacks.
Cost estimates from the leaderboard sharpen the picture of exploitability. For Grok 4.5, FAR AI reports that attackers could find a working universal jailbreak for roughly fifty eight dollars of attack effort, while Gemini 3.1 Pro required about two hundred seventy eight dollars. For Claude Fable 5 and GPT 5.6 Sol, the campaign did not uncover any universal jailbreaks, which FAR AI interprets as implying that at least fourteen thousand two hundred dollars of equivalent exploratory effort would be needed before success, with the barrier increasing as more attempts fail.
FAR AI frames this cost to break metric as an operational lens for adversaries: lower financial thresholds correspond to more accessible misuse in the real world, even when individual exploits demand technical skill. Viewed this way, Grok and Gemini look like economically soft targets compared with the Anthropic and OpenAI models, whose safeguards remained intact under the tested conditions.
Perplexity Sonar’s testing, which probed similar models with both random and expert designed attack sequences, reinforces this story of uneven resilience. Undirected random exploration alone uncovered dozens of universal breaches across Grok variants and nearly twenty on Gemini 3.1 Pro, revealing high baseline susceptibility even without carefully tailored strategies. When expert crafted multi step sequences were introduced, breach counts rose substantially, showing how coordinated efforts can amplify weaknesses that are already present in naive testing.
Taken together, these campaigns demonstrate that some frontier models are still easy to breach, while others can withstand sustained attacks within the tested budgets and domains. The gap is not cosmetic. It is measured in orders of magnitude differences in both the number of reusable exploits and the cost required to find them.
How this fits alongside other safety indices
It is useful to place the FAR AI leaderboard next to existing benchmarks rather than treating it as a replacement. FORTRESS and similar evaluations concentrate on single turn adversarial prompts in national security and public safety domains, reporting risk scores and over refusal metrics that show how often a model either gives dangerous information or declines legitimate questions. In those tests, some models, including earlier Gemini releases, already showed significantly higher average potential risk scores than competitors, suggesting weaker safeguards against single turn prompts.
The CASI and ARS leaderboards assess models on a mix of safety, hallucination, and performance measures, while jailbreak scores capture how frequently simple attacks succeed in eliciting restricted content. These indices have often placed Anthropic models at the top of safety rankings and proprietary models like Gemini and some open models lower in terms of aggregate risk and jailbreak resistance.
The RAIL AI Safety Index extends that comparative view across harm, bias, hallucination, and jailbreak categories, again finding notable variation between providers and sometimes highlighting trade offs between strict refusal policies and model usefulness.
FAR AI’s work complements these indices by focusing narrowly on adversarial campaigns and the economics of breaking models. While FORTRESS or CASI might say that a model fails a certain percentage of prompts in a given domain, the FAR AI leaderboard tells security teams how hard attackers had to work, and how much they had to spend, to unlock reusable exploits that could be used to generate many different harmful outputs.
Implications for vendors, adopters, and policymakers
For model providers, the leaderboard is both a scorecard and a roadmap. Vendors whose models show hundreds of universal jailbreaks and low cost to break figures now have strong external evidence that their safeguard stacks are not performing adequately under realistic adversarial pressure. That evidence is public, detailed, and easy for customers and regulators to understand.
Providers whose models resisted all tested attacks cannot afford complacency either. The report and the leaderboard make clear that the absence of discovered jailbreaks reflects the limits of the current campaign’s budget, tools, and threat modeling. It is a positive sign, not a proof of invulnerability, and it should be treated as a baseline to defend and improve as attack techniques evolve.
For enterprises, the message is practical. Safety ratings that look acceptable on paper can mask very different levels of real world risk once adversaries begin searching for reusable exploits. A procurement process that treats capability benchmarks and safety indices as interchangeable may underestimate the operational exposure created by models that are relatively cheap to break. Mapping cost to break into internal risk models and incident response plans is now a reasonable expectation, not an exotic extra.
Regulators and policymakers gain something they have been asking for over the past few years: independent, technically grounded evidence of how frontier models behave under attack in high risk domains such as chemical and biological misuse. This kind of evidence can inform safety standards, disclosure requirements, and licensing regimes in a way that abstract discussions of alignment or guardrails cannot. It also supports more nuanced regulation since it highlights that not all frontier models pose the same level of misuse risk.
Finally, for the broader AI community, this leaderboard nudges the field toward treating adversarial safety as a first class performance dimension. Research incentives and public recognition have historically flowed toward bigger models and better benchmarks for reasoning or coding. By rewarding models that are expensive to break and documenting the failures of those that are not, FAR AI and Perplexity Sonar shift some of that attention toward security engineering and red teaming depth.
Limitations and what comes next
Even strong evaluations have limits. The FAR AI leaderboard focuses on specific high risk domains and a finite set of breach techniques, and it expresses adversarial cost in terms of the resources spent within this particular campaign. Different attackers with different tools, budgets, or target goals might find alternative paths that were not explored, or they might care more about non universal exploits that still cause severe harm.
Models also change quickly. Safety patches, new guardrail architectures, and updated training data can move a model up or down the leaderboard in weeks or months. What looks like a secure configuration today may become easier to breach tomorrow if exploit discovery outpaces defensive upgrades. Conversely, models that perform poorly in this initial release could improve significantly if providers treat the findings as a reason to invest in more robust safeguard layers and red teaming.
There is also the question of over correction. Some providers may respond by tightening refusals across the board, increasing over refusal rates on benign content. Existing benchmarks already show that models with low risk scores often exhibit higher rates of declining legitimate queries, which can harm usability. Balancing resilience against serious misuse with the ability to support everyday workflows remains an unsolved design problem.
Key takeaways
- Frontier models differ dramatically in how hard they are to break, and the new FAR AI Security Leaderboard quantifies that gap using universal jailbreak counts and cost to break estimates.
- Grok 4.5 and Gemini 3.1 Pro showed hundreds of reusable exploits and low adversarial costs, while Claude Fable 5 and GPT 5.6 Sol resisted all tested attack sequences within the scope of the campaign.
- Cost to break is emerging as a practical risk metric, giving enterprises and regulators a way to connect technical safeguard strength to real world misuse incentives.
- Existing benchmarks like FORTRESS, CASI, and the RAIL AI Safety Index remain valuable, but the FAR AI leaderboard adds a crucial adversarial and economic dimension that should inform procurement and policy decisions.
- The field is moving toward a future where being secure against sustained, adaptive attacks is as important for frontier models as being capable or efficient, and where independent red teaming is a core part of how trust is earned rather than assumed.
The next year will show whether providers and regulators treat these findings as a turning point in AI safety practice or simply another set of scores, but the direction of travel is clear and the cost of ignoring adversarial robustness is rising.
Frequently Asked Questions
How Were Human Reviewers Trained to Evaluate Potentially Harmful Outputs From Each Model?
The way human reviewers are trained to judge potentially harmful outputs from advanced AI models has quietly become one of the most important safety questions in technology today. As systems like Perplexity Sonar and other large models move from novelty to everyday infrastructure, the quality of human oversight increasingly determines whether these tools remain useful and trustworthy or drift into genuine risk for users, businesses and society.
From early moderation to structured AI safety review
Human review of risky digital content did not start with AI. The first generation of online platforms relied on content moderators who worked with fairly simple rules around harassment, adult content and incitement to violence. Policies were often short, qualitative and platform specific, and reviewers were largely reacting to obvious abuses rather than subtle dual use risks.
As AI systems began generating text, code and images at scale, that older approach proved too narrow. Researchers and companies realized that models could assist users with cybercrime, targeted harassment or instructions for physical harm, even if the prompts looked innocuous. In response, safety teams developed more formal harm taxonomies that carved the space of unsafe content into distinct categories such as violent crime, non violent crime, sexual exploitation, self harm, misinformation, bias and privacy violations.
Modern safety frameworks now combine three strands of practice. First they define clear categories of harm, often informed by standards like MLCommons safety benchmarks and commercial usage policies. Second they use automated classifiers and guardrail models to flag likely issues. Third they rely on trained human reviewers for the truly hard calls, especially around context, intent and ambiguous edge cases. Perplexity Sonar sits in this emerging pattern, using curated datasets and human raters to evaluate model helpfulness, factuality and freshness, and extending that discipline into safety review.
Who the reviewers are and why their backgrounds matter
The people who assess potentially harmful AI outputs are no longer generalist moderators working from a simple rulebook. Safety teams recruit from several experienced domains that map onto different types of risk.
Many reviewers come from trust and safety roles in social platforms or content moderation firms, where they have learned how abuse actually plays out day to day and how users attempt to skirt rules. Others are clinicians or mental health professionals who understand psychological safety, self harm and crisis communication. Security analysts contribute expertise on cyber threats, intrusion techniques and operational security, while policy researchers bring knowledge of legal constraints, regulatory trends and geopolitical sensitivities.
This mix of experience matters because harmful outputs rarely fall into a single neat category. A piece of guidance on protest tactics might touch on public safety law, political influence and free expression. A response about drug synthesis might blend chemistry, criminal law and dual use risk. Teams deliberately construct reviewer cohorts so that no single perspective dominates and so that complex cases can be discussed across disciplines rather than treated as purely technical questions.
Building a shared language of harm
Diverse expertise is an asset, but it can also lead to inconsistent judgments if reviewers have different mental models. The first step in training is therefore to create a shared language of harm.
Safety leaders introduce multi category taxonomies that describe the major families of risk and their subtypes in precise operational terms. For example frameworks used in recent safety research divide content into many dozens of categories across several levels of detail, spanning everything from explicit violent crime to more indirect forms of unethical assistance. Within each category reviewers see definitions, positive and negative examples and specific tests for what counts as crossing the line.
Equally important is the distinction between harmless, borderline and clearly harmful content. Some teams adopt formal criteria that ask whether the output is unsafe, relevant to the harmful intent of the prompt and useful enough to materially advance that harmful goal. Reviewers are trained to check all three dimensions rather than simply reacting to individual words or topics. A discussion of mental health, for instance, might be entirely benign in one context and dangerously suggestive in another.
This structured approach transforms harm from a vague moral concern into a set of operational judgments that can be communicated, audited and improved over time.
How reviewers learn to spot risky outputs in practice
Once the taxonomy is in place, training shifts from theory to hands on practice. Reviewers work through large sets of model outputs that cover the full spectrum of risk, from ordinary benign answers to deliberately red teamed prompts designed to elicit dangerous content.
Sessions emphasize several recurring patterns.
Reviewers see clear examples of criminal guidance, such as assistance with fraud, physical harm, or evasion of law enforcement. They study sexual and exploitation risks, including content related to minors, coercion and non consensual behavior. They explore cybersecurity scenarios where an answer might help a learner understand basic security concepts, or instead provide actionable intrusion or exploitation techniques. They examine psychological safety cases involving self harm, suicidal ideation and manipulation or harassment.
For each example reviewers label the output along a harm scale. That scale usually spans at least safe, borderline and harmful, but many organizations adopt finer gradations to reflect context and severity. Reviewers record not only the label but also a short rationale, which forces them to articulate which parts of the response triggered concern and how they applied the taxonomy.
Special attention goes to edge cases. Jailbroken responses where a user has coaxed the model into bypassing its own guardrails are heavily used in training, because they show how harmful content can be embedded in friendly or seemingly conditional language. Dual use information, such as chemistry, biology or security topics that can be used for both legitimate and malicious purposes, receive careful walkthroughs so reviewers learn to distinguish educational content from step by step enabling guidance.
Modern programs also teach reviewers to work with automated safety tools rather than against them. Reviewers learn when to trust classifier flags, when to override them and how to interpret model level risk scores. That interplay is crucial because automated filters can surface the majority of obvious cases, leaving humans to focus on nuanced and high impact decisions.
Calibration, reliability and escalation
Individual intuition is not enough for a safety program that spans many reviewers and models. Training therefore includes deliberate calibration and quality control.
Reviewers regularly participate in group calibration sessions where everyone labels the same batch of outputs independently and then compares results. Differences in judgment are discussed openly, with senior experts explaining how policy and taxonomy were meant to apply. The aim is not to force uniformity in every case, but to surface divergent interpretations and refine definitions so they can be applied consistently.
Inter rater reliability is monitored over time using statistical measures that capture how often reviewers agree on the same labels. Where disagreement is high, teams look at whether the taxonomy is underspecified, whether reviewers lack context or whether new harm patterns have emerged that the training did not yet cover.
Ambiguous or high stakes outputs are subject to structured escalation. Cases involving newly identified threat vectors, sensitive political contexts or critical health and safety information are routed to senior reviewers or cross functional panels, sometimes including legal and policy stakeholders. This is broadly aligned with regulatory guidance such as the European Union AI Act, which calls for trained human oversight that can understand system limitations, detect unexpected performance and override automated outputs when necessary.
Spot checks and audits round out the process. Safety leads periodically re label samples of previously reviewed content to ensure standards have not drifted. They track metrics such as refusal rates on harmful prompts, the frequency of borderline content and the types of cases most likely to produce disagreement. These data feed back into updated training materials and revised taxonomies.
How Perplexity Sonar fits into this landscape
Perplexity Sonar is built around the principle that human evaluation is central to trustworthy AI assistance. Technical analyses of its stack describe how human raters review model outputs for helpfulness, factual accuracy and timeliness using curated datasets. Extending that infrastructure to safety review means training those raters not only to judge correctness and usefulness, but also to systematically identify and score potential harm.
In practice this makes Sonar part of a broader industry movement where the same reviewers might, in a single session, assess whether an answer is factually sound, up to date and free of harmful guidance. Safety taxonomies are integrated directly into the rating rubrics so that a single workflow can support both quality and safety objectives.
This combination of data driven evaluation and structured human oversight is particularly important for a system that is used as a front end for many underlying models. Different models can have different failure modes, and only a sustained human review program can reveal how those differences show up in real user interactions over time.
Implications for technology, business and society
For technology teams, robust training of human reviewers is a prerequisite for safe deployment of powerful models. Safety programs that rely purely on automated filters tend to miss subtle contextual harms, while teams that rely purely on human intuition struggle with scale and consistency. Combining taxonomies, automation and trained reviewers allows companies to match evaluation intensity to the risk of each use case, reserving deeper human review for high impact applications such as financial advice, health information or cybersecurity tooling.
Businesses that embed AI into workflows face rising expectations from regulators, customers and partners. Demonstrating that human reviewers are carefully selected, trained on recognized safety frameworks and monitored for reliability can become a competitive differentiator and a regulatory compliance requirement. Guidance such as the European Union AI Act already points toward explicit obligations for human oversight in high risk systems, including requirements for training and authority of the people who supervise AI outputs.
For society, the way reviewers are trained shapes public trust in AI. If the people behind the scenes understand both the technical capabilities and the real human consequences of harmful outputs, they are better positioned to raise concerns and push for safer practices. Conversely, if safety review becomes a low skilled checkbox task, harmful outputs may slip through despite the appearance of oversight.
There are also risks. Reviewer work can be emotionally taxing, especially when dealing with self harm, exploitation or graphic violence. Organizations must provide mental health support, realistic workloads and avenues for feedback. Over time we will likely see more codified professional standards for AI safety reviewers, similar to the way clinical ethics and data protection have evolved into recognized disciplines.
Limitations and ongoing evolution
Even well trained reviewers operate under constraints. They cannot see every interaction, and they must make judgments under uncertainty. Harm taxonomies themselves are imperfect, reflecting current knowledge and norms that can change with new technologies and social shifts. Studies of safety guardrails already note that harmful prompts can evolve through automated red teaming and creative misuse, which forces continuous updates to training materials.
Another limitation is cultural and regional context. A response that is acceptable in one legal or cultural environment may be unacceptable in another. Mature training programs increasingly incorporate region specific modules and consult local experts to adapt global safety guidelines for particular markets. However this remains an area where practice is still catching up with need.
Finally automation is advancing rapidly. Small safety focused models are now used as guardians for larger systems, classifying prompts and outputs and even generating safe alternative responses. Human reviewers must learn to collaborate with these tools, understanding their strengths and blind spots so they can decide when to rely on them and when to override them.
Key takeaways and what comes next
Training human reviewers to evaluate potentially harmful outputs is no longer a simple orientation on company policy. It is a structured, data informed program that combines diverse expertise, detailed harm taxonomies, extensive example based practice and ongoing calibration and escalation.
For AI builders and users the key lessons are clear. First, human oversight remains essential for safety, particularly at the edges where automated guardrails struggle. Second, that oversight must be designed and measured with the same rigor as model training itself. Third, as systems like Perplexity Sonar become deeply integrated into daily work, the quality of their human reviewers will increasingly define how trustworthy these tools feel in practice.
Looking ahead, the field is likely to move toward more formal professional standards for safety review, closer integration of human and automated evaluators, and regulatory recognition of reviewer training as a core element of responsible AI deployment. Organizations that invest early in thoughtful training programs will be better prepared not only to avoid harm, but also to earn durable trust from users who rely on AI systems for genuinely important decisions.
Sources
AIUC guidance on flagging high risk outputs for human review and associated workflows
Research on multi level safety taxonomies and safety classification models
Technical analysis of Perplexity Sonar evaluation methods using human raters
Studies on safety guardian models and human generated evaluation datasets
Work on fine grained definitions of harmful responses to unsafe prompts
European Union AI Act provisions on human oversight for high risk AI systems
Responsible AI guidance on measuring safety harms and combining automated and human evaluation
AI safety fundamentals including threat modeling, red teaming and standardized harm taxonomies
Safety evaluation guidelines describing jailbreak detection and request assessment practices
Research on interactive evaluations and human assessment of interaction harms reddit
What Safeguards Protected Test Data From Being Stored or Reused by These Models?
Safeguarding test data from being quietly stored or reused has moved from a niche compliance concern to a central question for anyone evaluating modern AI models at scale. Companies are now running thousands of test prompts through frontier systems, often containing realistic customer scenarios or even fragments of production data. Without strong controls, those test runs can leak into training sets, telemetry pipelines, or long-lived logs, creating privacy exposure and regulatory risk that is hard to unwind later.
How test data used to be handled
In the first wave of commercial AI APIs, the default assumption was that anything sent to the model could help improve it. Providers routinely retained prompts and outputs for weeks or months for abuse monitoring, debugging, and training, and many services made user content available to human reviewers under certain conditions. For standard API tiers, retention windows such as thirty days were common, and customers had to negotiate separate enterprise agreements if they wanted stricter policies.
From a model improvement perspective, this made sense. It allowed rapid iteration and better safety tooling. From a testing and privacy perspective, it created a quiet but serious problem: evaluation traffic, which often includes sensitive edge cases and realistic data samples, could end up in the same improvement corpora as everyday user conversations.
Regulators have since sharpened their expectations. The European Union AI Act explicitly treats training, validation, and testing datasets as part of a governed lifecycle and requires appropriate data management practices and minimization for each stage. Data protection authorities have also pushed for stronger limits on reuse and clearer consent for any training based on user data. Together, these shifts laid the groundwork for the current focus on keeping test data isolated and ephemeral.
What zero data retention really means for test runs
The most visible safeguard today is true zero data retention for enterprise AI APIs. In this mode, prompts, completions, and associated metadata are processed only long enough to complete the request, then discarded without being written to persistent storage. The interaction lives in memory during inference and is purged within seconds once the response is delivered, leaving no transcript or copy on the provider side.
Several providers now describe zero data retention in very specific terms. It typically includes four commitments that matter directly for test data.
Prompts and outputs are never added to training corpora or fine-tuning datasets, either for current or future models.
Raw inputs and model responses are excluded from application logs, telemetry streams, and shadow archives that might otherwise collect them for abuse monitoring or analytics.
Content is processed only in memory, avoiding disk-based persistence and reducing the chance that test data appears in backups or storage snapshots.
Retention windows are defined contractually, often as zero day for the covered endpoints, with providers assuming liability if data is later found on their systems.
The difference from earlier opt-out mechanisms is important. Historically, “no training” settings could still allow storage in logs or temporary telemetry, meaning test traffic might be visible to internal tools even if it never touched the training pipeline. With mature zero data retention, the goal is stateless inference: the provider treats the interaction as a transient computation, not a durable record.
Keeping evaluation traffic out of improvement corpora
Zero data retention modes are only one layer of protection. Teams running structured evaluations typically combine them with training opt-out and privacy switches in the surrounding workflow so that test conversations are never fed into improvement corpora or linked to persistent user profiles.
Modern AI platforms often expose explicit toggles to prevent the reuse of content for training and service improvement, even outside strict zero retention endpoints. When these switches are enabled for evaluation projects, they instruct the provider to exclude those interactions from any aggregation or sampling used to refine models or guardrails. In effect, evaluation traffic becomes a separate stream that is used only for the customer’s own analysis, not for global product tuning.
For interactive testing, incognito or temporary chat modes serve a similar purpose. They allow evaluators to experiment with realistic scenarios without creating long-lived user identities or conversation histories in the provider interface. Once the test session ends, the chats are either deleted or detached from identifiable accounts, limiting their footprint on the provider side.
Short lived logs and controlled observability
Completely eliminating operational logs is neither realistic nor desirable. Engineers still need some observability to catch errors, track latency, and handle abuse. The safeguard for test data is to ensure that any logs that do exist are short-lived, minimized, and carefully controlled.
Privacy-oriented testing guides now emphasize data minimization and anonymization as default practices, not afterthoughts. That means stripping direct identifiers from prompts, masking free text where possible, and ensuring that logs capture only what is strictly necessary to operate and secure the service. Where real data is unavoidable, it should be de-identified and encrypted, with access tightly limited through role-based controls and multi-factor authentication.
Enterprise zero data retention configurations often go further, disabling persistent logging entirely for covered endpoints and making it the customer’s responsibility to manage any necessary observability upstream, inside their own infrastructure. In that pattern, teams route evaluation traffic through stateless inference stacks and maintain their own short-lived logs on the client side, where they can enforce retention and masking policies consistent with internal governance.
Access control and isolated evaluation environments
Another safeguard that often goes underappreciated is access control. Test data is most vulnerable when evaluation environments are treated like open sandboxes rather than governed systems.
Security checklists for AI testing now routinely call for least privilege access, role-based controls, and clear separation between production and evaluation stacks. In practice, that means restricting who can see prompts and outputs from test runs, enforcing single sign-on and strong authentication for any tooling that touches evaluation traffic, and logging access to those tools for later audit.
Isolated evaluation environments are increasingly common. Instead of mixing test traffic with general user flows, organisations spin up dedicated workspaces or projects for AI assessments, then apply stricter policies for data retention, export, and sharing inside those spaces. Taken together with zero data retention at the provider level, this reduces the risk that test data leaks into broader analytics or product telemetry.
Governance, contracts and the regulatory backdrop
Technical safeguards matter, but they sit inside a larger governance and regulatory frame that is still evolving. The EU AI Act explicitly requires organisations building or deploying high-risk AI systems to apply appropriate data governance and management practices to training, validation, and testing datasets, including controls on errors, representativeness, and statistical properties. Data protection regulators highlight data minimisation, privacy-preserving computation, and clear separation between different processing purposes as core expectations for AI projects.
For teams relying on external providers, that translates into detailed contractual commitments. Vendor guides now urge customers to verify whether zero data retention is a default or an add-on, which endpoints and models are covered, how quickly prompts and outputs are purged, and what happens during system errors or support debugging. Many organisations insist on explicit data processing agreements that spell out retention windows, permitted uses, and responsibilities if data is mishandled.
This contractual layer matters for test data because evaluation runs often fall into a grey zone. They are not production, but they may involve realistic scenarios and regulated content. Treating them casually can create liabilities that are hard to explain later to auditors, regulators, or customers.
What still can go wrong
Despite these safeguards, it is important to be honest about the limitations. Zero data retention prevents prompts and outputs from being stored by the provider, but it does not automatically protect against leaks on the customer side. Test data might still appear in internal logs, experiment tracking tools, copy-pasted notes, or screenshots shared in chat channels. Governing those surfaces requires disciplined engineering and training, not just vendor settings.
There is also a risk of over-trusting marketing language. Different providers use similar terms for privacy features, but the operational reality can vary. Some may retain limited metadata for abuse monitoring, keep anonymised aggregates for capacity planning, or store error traces that include fragments of content. Others may apply zero data retention only to specific endpoints or models while leaving auxiliary tools, such as file storage or retrieval services, under different policies.
Security and QA leaders increasingly respond by combining technical verification, legal review, and periodic audits. They test vendor claims, inspect their own logs and data flows, and run privacy drills that assume test data could leak through misconfiguration or human error. That mindset treats safeguards as defense in depth rather than a single switch that magically solves the problem.
Practical takeaways for teams evaluating AI models
For organisations actively testing AI systems, a few practical conclusions follow from this evolution.
Use enterprise-grade zero data retention for any evaluation that touches sensitive or regulated data, and confirm that it covers all relevant endpoints and tools, not just the core model API.
Enable training opt-out and privacy switches for evaluation projects by default so test conversations never feed into global improvement corpora or long-lived user histories.
Design prompts and test datasets with data minimisation in mind, preferring synthetic or de-identified samples wherever possible and masking real-world details when they are not essential to the scenario.
Keep operational logs short-lived, encrypted, and limited to the customer side, with clear retention policies and regular reviews of what is actually being stored.
Treat evaluation environments as governed systems, not casual sandboxes, with restricted access, single sign-on, and audit trails for anyone who can see test runs.
Looking ahead
The trajectory is clear. As AI moves deeper into regulated domains and as evaluation programs grow more sophisticated, protecting test data from unintended storage or reuse will become a baseline expectation rather than a premium feature. Zero data retention and training opt-out modes are likely to spread beyond a handful of enterprise offerings and become standard for serious deployments.
At the same time, regulators will continue to refine what “acceptable” governance looks like for testing datasets, and organisations will need to show not just good intentions but concrete evidence of responsible practices. The teams that treat evaluation traffic with the same care as production data, combining technical safeguards with strong internal discipline, will be best positioned to benefit from powerful models without compromising trust. In the long run, that trust will matter as much as raw model capability in deciding which AI systems we allow into critical workflows and which we keep at arm’s length.
How Might These Safety Scores Influence Enterprise Procurement or Regulatory Compliance Decisions?
Safety scores for AI systems are starting to function like credit ratings for software, quietly reshaping which models enterprises buy and how regulators expect them to be governed. When boards ask whether an AI deployment is safe enough for a regulated workflow, they increasingly look for a number tied to a transparent methodology rather than a reassuring promise from a vendor.
From subjective safety debates to quantified risk
For most of the past decade, AI safety was discussed in qualitative terms, relying on policy documents, informal red teaming, and anecdotal incidents. That is changing as independent benchmarks and governance indices put hard numbers on issues like harmful output, bias, jailbreak susceptibility, and vendor maturity.
Recent safety indexes now rank large language models on multiple dimensions such as hallucination rates, harmful response rates, bias, and resistance to jailbreak prompts, producing an aggregate safety score that makes models easy to compare at a glance. For example, one 2026 benchmark places newer frontier models near the top of its table with average safety scores above seventy percent while older or less governed systems trail with markedly lower robustness against harmful prompts and bias tests.
In parallel, governance-focused ratings such as an AI vendor governance posture index assign composite scores that flag when a provider is judged inadequate for regulated workloads, explicitly recommending that procurement teams reevaluate before use in sensitive domains. Vendor-centric briefs on security and compliance are adopting similar scales, marking some platforms as enterprise-ready with scores above eighty out of one hundred and summarizing control coverage in a way procurement officers recognize from traditional SaaS security reviews.
These tools do not eliminate the need for expert judgment, but they do mark a clear shift from narrative debates about whether a model is safe toward quantifiable risk that fits into familiar enterprise risk frameworks.
How procurement teams are already using safety scores
Inside large organizations, the first place safety scores show up is in vendor screening. Instead of starting every AI conversation from scratch, procurement teams increasingly begin with a short list of models and platforms that meet minimum safety or governance thresholds. A provider flagged as inadequate for any regulated workload with a composite score around twenty out of one hundred will often be excluded from use in health care, finance, or public sector projects until the posture improves and the rating is updated.
Conversely, a platform assessed as enterprise-ready with a score in the low eighties is more likely to be shortlisted for pilots that involve customer data or decision support, provided other controls such as encryption and access management are in place.
Safety scores naturally feed into formal request for proposal processes. RFP documents that once asked vendors to “describe your approach to AI safety” now add concrete requirements such as participation in independent benchmarks, disclosure of safety test suites, and commitment to periodic reassessment as models evolve. Benchmarks that break down performance on hallucination, harmful content, bias, and jailbreak resistance over time allow buyers to insist that models used for sensitive workloads stay above specific thresholds in each category, not just in an overall score.
This becomes a contract lever: if a vendor falls below agreed safety benchmarks or stops submitting models for reevaluation, enterprises can retain the right to suspend high-stakes use or demand remediation.
Internally, safety scores support portfolio management. Central AI teams can map the scores of all models and vendors in use against business criticality. Highly capable but less governed systems might be constrained to exploratory or non-critical workloads, while models with consistently strong safety scores are approved for use in customer-facing tools, automated decision support, or internal workflows that touch compliance-relevant data.
Over time, this creates an incentive for vendors to show not only strong performance but also strong safety scores, because that is what unlocks higher value use cases.
Safety scores as evidence for EU AI Act compliance
The European Union AI Act formalizes a risk-based regulatory model, distinguishing unacceptable, high, limited, and minimal risk systems and imposing strict obligations on high-risk deployments. Under Article 6, an AI system becomes high risk either because it is a safety component of a regulated product such as medical devices or machinery covered by Union harmonization legislation, or because it falls into specific high-risk use cases listed in Annex III such as biometrics, critical infrastructure, education, employment, essential services, law enforcement, migration, and justice processes.
Guidance documents and tools that interpret the Act explain that AI integrated into products like vehicles, aircraft, rail systems, or medical equipment will generally be treated as Annex I high risk, triggering more demanding conformity assessments.
For providers and deployers of high-risk AI, the Act requires a risk management system across the lifecycle, rigorous data governance, detailed technical documentation, built-in record keeping, clear instructions for use, and effective human oversight of the system. Safety scores cannot replace those obligations, but they can help structure them.
If a foundation model or application layer system is going to be used in one of the Annex III domains, internal teams can use safety benchmarks as part of the risk management process to determine whether the model’s observed behavior is compatible with the intended use, especially around harmful output and bias. A model that performs well on bias and harmful content tests but shows comparatively weaker jailbreak resistance might be acceptable for use in an education support tool with strong human review, yet inappropriate for law enforcement analytical support even if basic technical documentation is in order.
Compliance tools and checkers built around the AI Act already hint at this dynamic. They guide organizations through assessing risk level and obligations, reminding them that high-risk systems require more extensive documentation, transparency, and human oversight and that some uses such as manipulative social scoring are prohibited altogether.
Safety scores can be attached to these assessments as structured evidence: auditors can see not just that a provider claims to manage risk but that the system has been tested against recognized benchmarks, how it compares to peers, and whether scores have improved or degraded over time. That makes it easier to challenge marketing claims of “best in class safety” and to insist on reproducible evaluations rather than selective anecdotes.
Opportunities and incentives created by benchmarking
Once safety scores are widely used, they begin to shape vendor behavior. Benchmarks that rank large language models on hallucination, harm, bias, and jailbreak resistance create a visible leaderboard against which providers will be judged by enterprise buyers. A vendor that sees its flagship model ranked well below competitors on harmful output or bias metrics has a direct commercial reason to invest in better filtering, safer training data, or more robust system-level controls.
On the other side, models that consistently score near the top of safety indexes gain reputational advantages when enterprises choose defaults for sensitive workloads.
Safety and governance indices also start to converge with security and compliance briefs, creating composite pictures of vendor readiness. Where a governance posture report warns that a provider’s controls are inadequate for regulated workloads, but a separate security brief presents a strong enterprise-ready score, buyers are forced to dig into the details, separating infrastructure-level security from AI-specific governance gaps. That strengthens the norm that AI procurement must be coordinated across security, compliance, legal, and technical teams rather than treated as a purely technical purchase.
There is also a wider societal effect. As regulators and civil society groups watch safety scores and benchmarks, they gain a clearer view of how quickly models improve and where persistent weaknesses remain. This makes it easier to target future regulatory guidance, for example focusing on domains where jailbreak resistance remains systematically low or where bias scores do not improve despite repeated model iterations.
Realistic limitations and risks
Safety scores are powerful but they are not a silver bullet. Any metric can be gamed, especially when commercial incentives are tied to a single number. Benchmarks that rely on static test suites may underrepresent emerging harms or novel exploitation techniques, which is why regulators emphasize that classification as high risk depends on actual risk of harm to health, safety, and fundamental rights rather than on benchmark performance alone.
A model that passes current jailbreak tests with ease can still be vulnerable to new prompt injection strategies, insider misuse, or downstream integration flaws.
There is also a risk of overgeneralization. Scores often aggregate many dimensions into a single headline figure, making different models look more comparable than they truly are. Two systems with the same overall safety score might have very different profiles, one strong on harmful content suppression but weak on bias, the other the reverse. If procurement teams focus only on the top line number, they may deploy models in contexts where their specific weaknesses matter most.
Regulators are aware of these nuances. The emerging guidance around classification of high-risk systems stresses context: whether an AI system materially influences decisions that affect fundamental rights, whether profiling of natural persons is involved, and whether human oversight is meaningful rather than nominal. Safety scores should therefore be treated as structured input to a broader risk analysis, not as automated verdicts.
Experienced compliance teams use them alongside impact assessments, legal analysis, and stakeholder consultations, especially in domains such as employment, essential services, and justice where errors carry significant social consequences.
What enterprises should take away now
For enterprises investing in AI, the main lesson is that safety scores are becoming a standard part of both procurement and regulatory dialogue. They offer a way to compare models and vendors quickly, but their real value lies in how they anchor contracts, oversight, and documentation.
Procurement teams can require minimum safety thresholds, ongoing participation in independent benchmarks, and transparent reporting of safety performance over time, tying these commitments to business-critical use cases and regulatory obligations.
Compliance leaders can embed safety scores into AI Act readiness work, using them to prioritize which systems demand deeper scrutiny, to support risk management documentation, and to challenge unsupported marketing claims of exceptional safety. Technical teams can use the same scores to guide model selection for particular workloads, aligning robustness profiles with real-world risk.
Looking ahead, enterprises that treat safety scoring as a living process rather than a checkbox are likely to be better positioned. As models evolve and regulations tighten, the organizations that continuously revisit safety scores, update RFP criteria, and refine internal risk classifications will have a more resilient AI portfolio and a stronger story to tell regulators, auditors, and the public about how they manage AI risk.
Were Any Accessibility or Bias-Mitigation Features Evaluated Alongside Traditional Safety Metrics?
In the race to deploy larger and more capable language models into everyday products, the question is no longer whether we measure safety but whether we measure the right dimensions of safety. Traditional metrics like hallucination rate, toxicity, and jailbreak resistance are essential. Yet they miss two issues that define whether these systems are genuinely trustworthy in the real world: bias and accessibility. The recent evaluations using Perplexity Sonar put those concerns on equal footing with classic safety metrics, which is exactly what matters right now for regulators, enterprises, and ordinary users.
How Safety Evaluation Reached This Turning Point
When mainstream model evaluation frameworks started to emerge, most focused on a narrow band of risks. Researchers measured how often models produced toxic text, how easily they could be coaxed into revealing harmful instructions, and how frequently they hallucinated confident but wrong answers. The dominant yardsticks were attack success rate for jailbreak tests, toxicity scores on curated datasets, and basic quality measures such as fluency or grammatical correctness.
Over time, the community realized that this first generation of metrics was not enough. A model could score well on jailbreak resistance and toxicity but still systematically underperform for certain demographic groups or languages, embedding subtler forms of discrimination in its behavior. Large surveys of safety evaluation for language models started to explicitly call out fairness and bias as distinct pillars alongside robustness and harmful content control.
Parallel efforts such as the AI Safety Index began to incorporate fairness-oriented benchmarks like BBQ for social discrimination and XSTest for alignment between helpfulness and harmlessness, treating robustness and fairness as coequal parts of safety rather than optional extras.
This historical shift is important context for understanding why the Sonar-based evaluations do not treat bias and accessibility as side notes. They sit alongside hallucination, harm, and jailbreak metrics as core dimensions of whether a system is safe enough to trust at scale.
What Perplexity Sonar Evaluated Beyond Classic Safety Metrics
In the Sonar-driven studies, bias mitigation and accessibility were not tacked on after the fact. They were explicitly evaluated alongside hallucination rates, harmful content prevention, and jailbreak resilience, using structured modules and quantitative metrics rather than anecdotes.
The bias module built on benchmark-style analysis similar to what the PHARE framework reported, where models are stress tested for harmful content generation, hallucinations, and bias in a unified suite. Sonar measured bias resistance percentages across many models, capturing how often systems successfully refused or neutralized prompts that push toward discriminatory or hateful content.
It also included targeted probes for antisemitism detection, examining whether a model would identify and block antisemitic narratives rather than reproduce or amplify them. By running these tests across different families—Claude, GPT, Gemini, Grok, and others—Sonar surfaced divergent levels of mitigation performance, not just a single generic safety score.
Accessibility was treated as another first-class safety axis. Instead of evaluating guardrails only in English, Sonar used multilingual safety probing in English, French, Spanish, and additional languages, pairing content safety checks with tailored jailbreak prompts that were adapted to each language.
This approach mirrors the direction taken by broader safety indices, which increasingly rely on multilingual tests to capture how robustness and fairness behave across linguistic communities rather than assuming that English behavior generalizes everywhere. The result is a view of accessibility that is not limited to interface ergonomics but extends to cross-lingual robustness of the underlying safety systems.
Crucially, Sonar integrates these modules with traditional safety harnesses. Content safety pass rates, jailbreak resistance scores, and hallucination metrics are measured side by side with bias resistance and multilingual robustness, rather than in separate silos. That design choice makes model comparisons far more informative for anyone deciding which system to deploy.
How Bias Metrics Change The Picture Of Model Safety
Bias metrics in this context are not abstract fairness slogans. They are operational numbers that can be tracked over time and compared between models.
Bias resistance percentages show the share of prompts with potential for discriminatory or hateful content that the model successfully handles in a safe way. Depending on the scoring policy, a safe handling might mean refusal, de-escalation, or redirecting to educational content. Antisemitism detection rates measure how consistently the model recognizes antisemitic content or tropes and applies the right safety response, rather than treating them as neutral conversational material.
Sonar-style evaluations reveal that frontier models vary meaningfully on these metrics even when they look similar on more familiar safety scores. Benchmarks like PHARE have already shown that models such as Claude tend to perform particularly well in preventing harmful content, reducing hallucinations, and mitigating bias.
When Sonar applies comparable probes, it exposes double-digit differences in bias resistance between top models that might otherwise appear interchangeable on headline hallucination or jailbreak numbers.
This is where experience with earlier evaluation waves matters. In the past, many teams chose models primarily on utility metrics such as accuracy on task benchmarks or response fluency, assuming that safety controls would be roughly comparable across vendors. The new bias-centric metrics show that this assumption does not always hold. A model that shines on productivity tasks can still lag on bias mitigation, creating legal and reputational risk if used in customer-facing workflows.
Why Accessibility Is A Safety Question Not Just A Usability Question
The accessibility tests embedded in Sonar treat language coverage as a safety concern rather than a pure product feature. A multilingual society cannot rely on safety measures that only work reliably for one language group.
By running aligned jailbreak and harmful content prompts in English, French, Spanish, and other languages, Sonar evaluates whether guardrails and content filters behave consistently across linguistic contexts. The tests look for cases where a model refuses unsafe prompts in English but complies with very similar prompts phrased in another language, a pattern that has been observed in earlier jailbreak research when guardrails were tuned mainly on English data.
This matters for at least three reasons.
First, regulatory expectations around fairness and non-discrimination often apply across languages. If an AI system is safe for English speakers but more permissive or biased in another language, it can create unequal protection in practice.
Second, global enterprises deploy the same model stack in many markets. They need assurance that content filters will block fraud, harassment, and discrimination consistently, whether the conversation is in Spanish with a call center agent or in French in a healthcare triage chatbot.
Third, accessibility intersects directly with bias. Underperforming safety controls in lower-resourced languages can lead to higher exposure to harmful content for communities that already face structural disadvantages. Adding multilingual probes to the evaluation suite is one of the few direct ways to measure and mitigate that risk.
Implications For Builders, Businesses, And Society
Treating bias mitigation and accessibility as equal partners to hallucination and jailbreak metrics carries real implications.
For model providers, it raises the bar on what counts as a safety-ready release. A model that scores well on toxicity and jailbreak resistance but shows weak bias resistance or inconsistent multilingual guardrails is no longer competitive once buyers can see those numbers side by side. Sonar-style dashboards make this reality visible in a way that marketing claims cannot erase.
For enterprises, it changes procurement and governance. When selecting a foundation model, technical teams are now able to compare bias mitigation performance between vendors using structured metrics, not just high-level assurances. Risk committees can insist on minimum thresholds for bias resistance and multilingual safety robustness, treating them as service level indicators just like uptime or latency.
For policymakers and civil society, the message is that safety legislation and standards should explicitly encompass fairness and accessibility, not only harmful content and misuse. Emerging indices that track robustness, fairness, and resistance to harmful behavior in a unified way demonstrate that this broader framing is technically feasible. Evaluation stacks like Sonar show how it can be operationalized in practice.
For everyday users, the benefits are subtle but important. Better bias mitigation reduces exposure to harmful stereotypes and coded hateful speech. Stronger multilingual safety controls make it more likely that people will receive equally safe assistance regardless of their primary language. Over time, these shifts can increase trust in AI systems by aligning their behavior more closely with social norms and legal protections.
Key Takeaways And What Comes Next
Bias mitigation and accessibility are now embedded in serious safety evaluation, not appended as afterthoughts. Sonar explicitly measures bias resistance and antisemitism detection across major models, and it probes safety behavior across languages using tailored multilingual jailbreak and content tests.
This gives a much richer and more realistic picture of how models behave in the world where people of different backgrounds and languages interact with them every day.
The direction of travel is clear. Safety frameworks are moving from a narrow focus on toxicity and jailbreak toward multi-dimensional risk views that include fairness, robustness, and control under adversarial pressure.
As more organizations adopt Sonar-style evaluation and publish comparative results, pressure will grow on model providers to close the gap between their strongest and weakest safety dimensions, rather than optimizing for a single headline metric.
For builders, the practical next step is to treat bias and accessibility metrics as first-class success criteria when training and fine-tuning models, and to monitor them continuously in production environments. For buyers and regulators, it is to demand transparent reporting on these metrics and to bake them into procurement decisions and compliance checklists.
That is how accessibility and bias mitigation move from being laudable aspirations to measurable, enforceable parts of AI safety.
Sources
A Comprehensive Survey on Safety Evaluation of Language Models, Chen et al, Arxiv
Measuring AI Safety Content Jailbreak And Grounding Metrics, Compel Framework
A Systematic Literature Review of AI Safety Evaluation, Arxiv
PHARE Benchmark Analysis Disparities in Language Model Safety and Bias, Triage Security
AI Safety Index, Future of Life Institute
How Frequently Will This Safety Benchmark Be Updated as New Model Versions Launch?
The safety benchmark will follow an adaptive update rhythm, with planned refreshes roughly every three to six months and extra event-driven updates whenever major new model versions, deployment patterns, or attack techniques appear. Updates will be coordinated through the ModelBench infrastructure with strict versioning, so organizations can track exactly which model variant was tested, under which scenarios, and at what point in the release cycle.
Why this update cadence matters right now
Foundation models are moving from experimental tools into the core fabric of products, workflows, and even regulatory processes, which means static one-off safety evaluations are no longer enough. Benchmarks that made sense for classic machine learning systems could be updated annually without causing much concern because the underlying models and use cases evolved slowly.
With large language models and multimodal systems, capabilities can leap forward every few months, and that pace turns a stale benchmark into a form of safety theatre rather than a real risk signal. Safety benchmarks face an extra layer of complexity compared with performance tests for accuracy or throughput because they must reflect not only what models can do but also how they can fail or be abused.
Attackers and curious users constantly probe these systems for new jailbreak patterns, indirect prompt chains, and domain-specific exploits that rarely existed in older machine learning deployments. A benchmark that does not update when new attack classes appear can give a false sense of confidence, especially to teams that rely on scores for compliance or customer messaging.
How benchmark updates have evolved
The history of AI benchmarking began with relatively static datasets such as ImageNet and GLUE, which were updated infrequently and mostly for incremental extensions or bug fixes. These benchmarks were invaluable for measuring progress but they assumed stable tasks, clear metrics, and slow-moving model families.
As foundation models emerged, projects like HELM and other holistic evaluations started to broaden the scope, tracking aspects such as robustness, calibration, fairness, and downstream task performance across many scenarios. More recent work has shifted toward adaptive and open-ended benchmarking, where test suites can expand and reconfigure in response to new capabilities, domains, or risk profiles.
ONEBench illustrates this paradigm by treating individual annotated samples as atomic units that can be recombined into customized evaluations, so the benchmark can grow without losing comparability. Techniques such as automated capability evaluation use powerful reference models to generate tasks and hierarchies dynamically, then probe subject models in targeted ways rather than relying on a fixed set of questions.
Adaptive benchmark design has matured into a discipline of its own, with best practices around selecting representative probe sets, calibrating metrics, and minimizing bias while still reacting quickly to new model behavior. This evolution mirrors the broader shift from static software releases to continuous delivery, but with the added requirement that every update must remain interpretable to researchers, regulators, and practitioners.
The planned refresh rhythm for the safety benchmark
Against this backdrop, the chosen cadence for the safety benchmark is intentionally moderate but responsive. It centers on scheduled refreshes roughly every three to six months, which aligns with guidance that rapidly evolving domains like large language models need updates at least that frequently to remain meaningful.
This window is long enough to collect substantial new data and emerging use cases, yet short enough to avoid the common failure mode where a benchmark quietly drifts out of relevance while models race ahead. On top of that baseline, there will be event-driven updates whenever a major foundation model release, architectural shift, or deployment change significantly alters the risk landscape.
Safety and security benchmarks are now widely regarded as requiring updates as soon as new vulnerabilities or attack families are discovered, rather than waiting for the next calendar window. Tying the benchmark to model release cycles ensures that evaluations for frontier systems do not lag months behind the reality that users and businesses are already experiencing.
The benchmark is managed through ModelBench, which serves as a catalog of models, with metadata such as release dates, context windows, and providers recorded in a consistent structure. That catalog style approach allows the safety benchmark to attach results to specific model entries and versions, making it possible to see how safety scores evolve over successive updates rather than treating each run as a separate experiment.
What actually changes when the benchmark updates
Regular refreshes will not only retest existing scenarios but introduce new use cases, personas, and hazard categories as they appear in the real world. Research on adaptive benchmarks emphasizes that sample pools and task configurations should grow over time, guided by observed model behavior and domain shifts rather than purely theoretical checklists.
In practice, this can mean adding prompts that reflect novel business workflows, newly popular consumer applications, or emergent professional roles that interact with AI systems in different ways. Hazard categories will expand along similar lines. As new misuse patterns are documented, such as composite prompt chains or exploitation of external tool integrations, benchmark designers can incorporate them into structured evaluations.
Dynamic sample selection methods help maintain coverage while keeping test suites tractable, focusing on the most informative risk scenarios rather than simply adding volume. Over time, this yields a safety benchmark that mirrors the evolving risk surface of deployed models rather than a static snapshot frozen at design time.
Implications for technology, businesses, and society
For technical teams, an adaptive benchmark with a predictable three to six month rhythm and event-driven add-ons becomes a planning tool as much as an assessment instrument. It allows model and product groups to budget time for regular safety reviews and remediation cycles, knowing when new results will appear and how they will be tied to specific versions in ModelBench.
This encourages a culture where safety is tracked with the same discipline as performance or cost, rather than treated as an occasional audit. Businesses gain a clearer narrative for customers and partners. Instead of vague claims that models are safe, they can point to the latest benchmark run, describe how it differs from prior cycles, and explain what mitigations followed from any newly identified weaknesses.
In regulated sectors, authorities increasingly expect evidence that benchmarks reflect current capabilities and threat landscapes, especially in areas such as healthcare or finance where static tests are seen as inadequate. Society at large benefits from greater transparency but also faces new questions. Frequent updates increase realism and reduce safetywashing, yet they can also generate a stream of changing scores that are hard for non-experts to interpret.
If one model improves on disinformation resistance but regresses on another harm category in the next cycle, communicating that nuance without eroding trust will demand careful reporting and visualization.
Limitations, trade-offs, and what to watch
No update schedule is perfect, and the three to six month target with event-driven updates is a compromise between agility and stability. Extremely frequent changes can make longitudinal comparisons difficult, while slow cycles risk missing critical shifts in model behavior or attacker creativity.
The reliance on dynamic methods such as sample level pooling and automated capability generation introduces its own uncertainties, including the possibility of hidden biases in which scenarios are deemed most important. Another limitation is that a benchmark, no matter how adaptive, remains an abstraction of reality.
Benchmarks condense complex behavior into scores and categories, which inevitably omits some context about how models interact with specific organizations, communities, or cultural settings. As a result, these safety evaluations should be treated as high-quality signals that inform risk management, not as guarantees that a model is safe under all conditions.
The long-term success of this approach will depend on ongoing community scrutiny and iteration. Experience from projects such as HELM, ONEBench, and adaptive benchmark design frameworks suggests that open feedback loops, transparent methodologies, and shared tooling are key to maintaining relevance and trust.
ModelBench provides a backbone for this by making versions and metadata explicit, but governance, documentation, and peer review remain essential complements.
Key takeaways and the road ahead
The core message is that safety benchmarking can no longer be a one-time pre-launch exercise; it must become a continuous, adaptive process that keeps pace with the frontier of model development and exploitation techniques.
A benchmark that refreshes every few months, ties results to explicit versions in a shared catalog, and reacts rapidly to new attack vectors offers a pragmatic path toward that goal. It balances historical comparability with real-world relevance, helping teams see trends over time while still responding quickly to shocks in the risk landscape.
Looking forward, expect benchmarks to become more personalized and context-aware, drawing on ideas from open-ended frameworks and automated capability evaluation to tailor tests to the specific domains where models are deployed.
As models integrate deeper into tools, data pipelines, and decision processes, safety benchmarks will likely expand beyond prompt-based evaluations into richer scenarios that capture tool use, memory, and multi-agent interactions. The adaptive schedule outlined here is an important step in that direction, signaling that safety measurement is evolving alongside model capabilities rather than trailing them at a distance.
Conclusion
Across the newest large scale safety campaign by Perplexity Sonar the headline is clear Claude and GPT 5 point 6 are much harder to break than Gemini and Grok and that gap now matters as much as raw capability itself. The study lands at a moment when regulators and companies are waking up to the fact that jailbreak resistance is not a niche security concern but a core property of any system they can safely deploy at scale.
Why this safety test matters right now
Over the past year frontier models have become vastly more capable yet independent evaluations keep revealing how easy it remains to push them into highly dangerous behavior. In early alignment workshops Xander Davies from the United Kingdom AI Security Institute showed that every frontier model they tested could be jailbroken even as average time to a successful attack stretched from minutes to several hours and that open weight systems were around fifteen times easier to compromise than closed ones. Later risk monitoring reports confirmed the same basic pattern baseline safety scores looked strong on paper but once serious red teams got involved those scores crashed across domains such as biological and cyber risks.
Perplexity Sonar comes into this landscape with a focus on deep multi source analysis and verifiability a style reflected in its own research benchmark which reports high scores for verification and coverage across prior briefs. A campaign that applies that methodology to jailbreaks across Claude GPT 5 point 6 Gemini and Grok is therefore consequential it helps separate marketing narratives from hard data at a moment when many organizations are still choosing models on the basis of generic IQ style benchmarks rather than resilience under attack.
From clever jailbreak tricks to systematic safety testing
Jailbreaking started as a collection of clever tricks prompt injections role play scenarios or elaborate story setups that persuaded models to ignore their safety rules. Over time these tricks turned into systematic adversarial methods. One research team recently demonstrated that large reasoning models such as DeepSeek R1 Gemini 2 point 5 Flash and Grok 3 Mini can act as autonomous jailbreak agents achieving more than 97 percent success across a broad benchmark of target models and harm scenarios. That result underlines how attackers can combine models and tools rather than relying on naive single prompts.
Other labs have stepped in to measure these vulnerabilities at scale. AI Security Lab reported a detailed campaign against frontier models in which even the most resilient variants still produced hundreds of harmful outputs under persistent attack though with markedly different success rates depending on the model and scenario. Meanwhile automatic agents such as Investigator have shown that once you automate the search for jailbreak prompts models like Grok 4 and Gemini 2 point 5 Pro can be compromised in nearly all tasks while more cautious designs such as Claude Opus and GPT 5 thinking models fail far less often under the same conditions.
Against this backdrop a Sonar campaign that focuses on universal jailbreaks attacks that try to break safety across many categories and prompts rather than narrow single tasks is a step toward treating jailbreak resistance as a measurable engineering property rather than a matter of anecdotes.
What Sonar found about Claude GPT 5 point 6 Gemini and Grok
According to the Sonar study Claude and GPT 5 point 6 sit in a cluster of robust models that consistently refuse harmful requests even under sophisticated universal jailbreak attempts. In this test they maintained high refusal rates across different risk categories and required significantly more time and interaction cost before any successful breach occurred. That profile aligns with earlier independent findings that Anthropic models and advanced OpenAI reasoners tend to be harder to compromise than competitors when subjected to complex multi step attacks.
Gemini and Grok looked very different. They yielded many more successful attacks at much lower expense both in terms of prompts and effort. External work backs up this picture. Lumenova reported that Gemini 2 point 5 Pro was by far the easiest frontier model to compromise in an encryption themed jailbreak experiment while Claude 4 Opus was the most difficult. A separate automatic attack benchmark showed Grok 4 near the top of pass rates with attack success close to 98 percent on first try and 100 percent when given more attempts again placing it among the most vulnerable models in the study.
A recent investigation quantifying breaches across multiple providers found that Grok recorded 448 successful jailbreaks and Gemini 249 within a single campaign whereas Claude Fable and GPT models produced far fewer harmful completions under similar pressure. Sonar s results therefore fit a pattern emerging across different teams and methodologies the safety gap between models is large and persistent even when they are trained to roughly similar capability levels.
Why capability no longer tells the whole story
For years conversations about frontier AI focused on who had the most capable model better reasoning higher benchmark scores faster inference. The combined evidence from Sonar and other red teaming efforts suggests that this view is incomplete. Capability makes models more useful and in some cases more controllable but it does not automatically make them safer. Studies have shown that newer models with better general performance can still be jailbroken to produce detailed guidance on explosives biological agents and complex social manipulation strategies often after only tens of minutes of focused adversarial work.
In the Sonar comparison Claude and GPT 5 point 6 illustrate one path forward heavy investment in alignment training refusal behavior and safety architecture that keeps the model within its guard rails even when attacked with advanced prompts or agentic jailbreak tools. Gemini and Grok illustrate the opposite risk a model can be extremely capable and fast yet still cheaply subverted into generating dangerous content if its safety stack is less robust or inconsistently enforced.
The implication for businesses and institutions is stark choosing a model primarily on the basis of capability benchmarks without evaluating jailbreak resistance now looks like a serious governance failure. For domains such as health finance education defense or critical infrastructure where misuse could have spillover effects on millions of people robustness under attack should be treated as a first class metric.
What this means for vendors and regulators
Regulatory bodies and standards groups already track baseline safety scores on curated benchmarks yet multiple quarterly risk reports have shown how those scores collapse under adversarial pressure. For example one monitoring program reported average safety scores dropping from above 78 to single digits in biological risks and from above 90 to around 29 in cyber offense once advanced jailbreak attacks were introduced. Those numbers make clear that static benchmarks cannot be the only basis for regulatory assurance.
The Sonar results add pressure in three directions.
- Vendors will need to provide evidence that their alignment strategies hold up under realistic red teaming not just on internal tests. Independent labs have now repeatedly jailbroken leading models from all major providers which suggests that claims of safety need continuous external validation.
- Regulators are likely to move toward risk weighted obligations where operators of less robust models bear heavier compliance burdens or usage restrictions than operators of models with documented jailbreak resistance. Quarterly risk monitoring frameworks already hint at this by highlighting large spreads in refusal rates across models.
- Procurement teams in governments and large enterprises will have to incorporate jailbreak resilience into their technical due diligence rather than relying only on performance metrics or generic compliance certifications. The Sonar campaign offers a template data rich comparative testing with attention to attack cost and exploitability that can inform those choices.
Practical takeaways for builders and buyers
For teams building or integrating frontier models a few practical lessons emerge from this body of work.
- Treat safety architecture as a separate design dimension from capability. Decisions about filtering tools specialized safety layers or additional safety tuned modes such as extended reasoning have measurable effects on jailbreak resistance.
- Assume that determined adversaries will eventually jailbreak any single model. Multiple studies have now achieved near universal success rates using autonomous jailbreak agents or cross model attack strategies which suggests that defense should focus on layered controls monitoring and downstream mitigations not just on model refusal behavior.
- Prefer models with proven robustness when deploying in high stakes settings even if they are slightly less capable or more expensive. The Sonar findings and corroborating external experiments consistently place Claude and advanced GPT lines in the more resistant group while Gemini and Grok appear comparatively fragile under attack.
- Invest in ongoing evaluation rather than one off audits. Risk profiles change across model generations and over time as providers update systems. The difference between earlier and newer GPT and Grok variants in jailbreak difficulty shows how dynamic the landscape can be.
Looking ahead
The Sonar study is unlikely to be the last word on Claude GPT 5 point 6 Gemini and Grok but it marks an important shift. Instead of asking who built the smartest model the conversation is beginning to ask who built the safest model that still performs well enough for meaningful use. As more independent labs publish data on jailbreak costs and success rates regulators and buyers will have the leverage they need to demand measurable safety guarantees rather than accepting marketing claims at face value.
The frontier of AI safety is moving from slogans to statistics and from vague assurances to concrete engineering tradeoffs. In that world the models that win will be the ones whose creators treat robust alignment and long term stability as design goals not as optional extras for press releases reddit
