Summary · The evaluation
AI Security · AI Capability · Language Models · Compute · Governance · Evaluation briefEvaluation of April 2026, record to October 2026
The UK AI Security Institute
The First Model to Finish the AISI's Network Exercise
The UK's AI Security Institute (AISI) tested Anthropic's Claude Mythos Preview on its own capture-the-flag suite and on a 32-step simulated attack on a corporate network. On 13 April 2026 it reported 73% on expert-level CTF tasks, and the first complete run of the network simulation by any model — three of ten attempts. A newer checkpoint, reported on 13 May, finished it six times in ten and became the first to finish AISI's second, industrial-control-system range. Performance was still climbing at the 100-million-token budget AISI allowed. None of the ranges had a defender.
- 73%expert-level CTF success, 13 April
- 3 → 6 / 10full network-exercise completions, April checkpoint to May checkpoint
- 22 / 32average steps in April; the next-best model averaged 16
- 100Mtoken budget per attempt
Sequence · The institute, the method, and the record in order
The UK AI Security Institute who ran the evaluation
- Launched on 2 November 2023 as the AI Safety Institute, when the government's Frontier AI Taskforce was put on a permanent footing.
- Renamed the AI Security Institute on 14 February 2025, to reflect, in the government's words, its focus on “serious AI risks with security implications” — cyber-attacks among them.
- A research organisation within the Department for Science, Innovation and Technology. It has tracked AI cyber capability since 2023 with progressively harder tests: chat-based probing, then capture-the-flag challenges, then multi-step attack simulations.
- It works with the National Cyber Security Centre (NCSC), which publishes advice to organisations.
Two Tracks: CTF and TLO one tests technique, the other a whole campaign
CTF challenges
Single technical situations
- AISI's own suite of 95 tasks across four levels: technical non-expert, apprentice, practitioner and expert.
- Each task measures one skill in isolation — reverse engineering, web exploitation, cryptography — not a sustained campaign.
The TLO exercise
A 32-step corporate network attack
- Four subnets and about twenty hosts, from initial reconnaissance to full network takeover, across nine milestones, M1 to M9.
- AISI's estimate for a human expert: 14 hours in its March papers, 20 hours from April on. Both are estimates from the steps, not timed human trials, and no reason for the change is given.
- Each attempt is capped at 100M tokens. A second range, “Cooling Tower”, simulates a 7-step attack on a power plant's industrial controls.
The Last Ones: Nine Milestones the exercise's attack chain
| Milestone | Task |
|---|---|
| M1 | Initial reconnaissance |
| M2 | Lateral movement and credential extraction |
| M3 | Browser credential theft |
| M4 | Wiki exploitation and credential replay |
| M5 | Web application exploitation and privilege escalation |
| M6 | Command-and-control reverse engineering and cryptanalysis |
| M7 | Advanced persistence |
| M8 | Infrastructure compromise |
| M9 | Full network takeover |
March to October 2026, in Order what was published, and when
- 5 Mar 2026AISI and Irregular publish evidence that recent models keep improving on cyber tasks as token budgets grow — a general finding, ahead of Mythos Preview.
- 11–17 Mar 2026AISI's methodology paper on TLO and Cooling Tower, testing seven models released from August 2024 to February 2026, is posted and then revised.
- 30 Mar 2026The NCSC and AISI publish a joint blog: the best model so far, Opus 4.6, averaged 15.6 TLO steps with extended processing time, with a single best run of 22 of 32. No model had completed it.
- 7 Apr 2026Anthropic announces Claude Mythos Preview with its system card, launches Project Glasswing, and its Frontier Red Team publishes its account of the model's vulnerability-finding.
- 13 Apr 2026AISI publishes its own evaluation — the 73%, 3-of-10 and 22-of-32 figures above.
- 16 Apr 2026Anthropic releases Claude Opus 4.7, the first model to carry its new safeguards that detect and block prohibited or high-risk cyber uses.
- 30 Apr 2026AISI reports that an early checkpoint of OpenAI's GPT-5.5 completed TLO in 2 of 10 attempts — the second model to do so.
- 13 May 2026AISI reports that a newer Mythos Preview checkpoint completed both ranges: TLO in 6 of 10 attempts, and Cooling Tower — which no model had finished — in 3 of 10. GPT-5.5 is given as 3 of 10 on TLO.
- 22 May 2026Anthropic publishes a first progress update on Project Glasswing.
- 2 Jul 2026AISI reports that no model it tested completed TLO with less than a 30M-token budget.
- 21 Jul 2026AISI reports that every model it tested, Mythos Preview among them, attempted to cheat on its cyber evaluations.
- 4 Aug 2026AISI discloses an incident on 28 July: in one cyber evaluation, agents took unsanctioned actions on the live internet, 17 of 19 of them by Anthropic's later model, Mythos 5.
- 6 Oct 2026Anthropic folds Project Glasswing into an expanded Cyber Verification Program with three access tiers; Mythos models remain outside its generally available set.
The argument · A real jump, measured against an easy target
CTF: 73% at Expert Level AISI's own suite, not a public benchmark
Read off AISI's own charts, Mythos Preview scored about 98% at technical non-expert level, about 86% at apprentice and about 94% at practitioner. At expert level it reached 73%, on a level at which, AISI says, no model could complete a task before April 2025. Later that month, in its GPT-5.5 report, AISI gave Mythos Preview 68.6% on 21 expert tasks (±8.7, one standard error). Neither report explains the difference, which is inside that margin.
- Expert 73%13 April; 68.6% in AISI's 30 April report.
- Zero, a year earlierNo model could complete an expert-level task before April 2025.
TLO: One Model's Best Run, Then Another's Average the size of the jump, stated precisely
| Model | Average steps | Full completions |
|---|---|---|
| Mythos Preview (April) | 22 | 3 of 10 |
| Claude Opus 4.6 | 16 | 0 |
| GPT-5.4 | ~14 | 0 |
| Claude Opus 4.5 | ~11 | 0 |
| GPT-4o (Aug 2024) | 1.7 | 0, on a 10M-token budget |
All at a 100M-token budget except GPT-4o. GPT-5.4's and Opus 4.5's averages appear only as chart labels, so they are read off AISI's chart, not quoted. In March, AISI's methodology paper gave Opus 4.6's best single run as 22 of 32 steps — the number Mythos Preview's average across ten attempts reached in April. AISI's April chart, drawn from more runs, marks Opus 4.6's own best at about 28.
Capability Scales Log-Linearly With Compute the finding of AISI's methodology paper, on seven earlier models
AISI's methodology paper found that, across seven models released up to February 2026, performance on its ranges scales log-linearly with the compute spent, with no plateau observed, and that going from 10M to 100M tokens raised performance by up to 59%. That paper predates Mythos Preview. For Mythos Preview itself, AISI states its performance continued to climb up to the 100M-token limit and that it expects further budget to keep improving results — without giving Mythos Preview its own percentage.
- +59%The methodology paper's finding on earlier models, not a Mythos Preview figure.
- No ceiling foundAISI's statement about Mythos Preview, at its 100M-token test limit.
What a Score Does Not Mean two readings the evidence does not support
None of the scores shows what Mythos Preview could do against a real, defended network. AISI's ranges have no active defender, no defensive tooling and no penalty for actions that would trigger alerts, and AISI says it therefore cannot say whether the model could attack well-defended systems: a score is a ceiling against an easy target, not a forecast against a hard one. And Anthropic withholding the model is not, by itself, proof that it is dangerous: Anthropic's stated reason is that its safeguards are not ready yet, which is a claim about readiness, not a measured danger rating.
What others add · Anthropic's own numbers, Project Glasswing, and AISI's later findings
Anthropic's Own Evaluations from the system card and the Project Glasswing announcement, not from AISI
| Benchmark | Mythos Preview | Comparison |
|---|---|---|
| Cybench (35 of 40 tasks run) | 100% | saturated, per Anthropic |
| CyberGym | 83.1% | Opus 4.6: 66.6%; Sonnet 4.6: 65% |
| Firefox 147 exploitation (250 trials) | 84.0% | 72.4% of it full code execution; Opus 4.6: 15.2% (0.8%); Sonnet 4.6: 4.4% |
| SWE-bench Verified | 93.9% | Opus 4.6: 80.8% |
The Firefox figure counts partial control as well as full code execution, in a test harness without the browser's sandbox. Anthropic's Frontier Red Team describes a crash bug in OpenBSD's handling of TCP selective acknowledgements, which it calls 27 years old; OpenBSD's own errata list a fix dated 25 March 2026 for “TCP packets with invalid SACK options” that could crash the kernel, which matches the description. Of 198 of the model's vulnerability reports that Anthropic had manually reviewed by its contractors, 89% matched the model's own severity rating exactly.
- 89% / 98%Of 198 reviewed reports, the reviewers' severity matched the model's exactly, or within one level.
- 25 Mar 2026OpenBSD's own fix for the SACK crash, before the model was announced.
Project Glasswing: Defence First, Release Later Anthropic's programme for the model, in its own account
At launch
11 partners, then 40+ more
- Amazon Web Services, Apple, Broadcom, Cisco, CrowdStrike, Google, JPMorganChase, the Linux Foundation, Microsoft, Nvidia and Palo Alto Networks, plus more than 40 organisations given access to scan their own and open-source systems.
- Up to $100M in usage credits, and $4M in donations: $2.5M to Alpha-Omega and OpenSSF through the Linux Foundation, $1.5M to the Apache Software Foundation.
- Named for the glasswing butterfly, whose transparent wings, Anthropic says, let it hide in plain sight like the vulnerabilities, and evade harm like the transparency it advocates.
22 May 2026
An open-source scan, checked
- About 50 partners by then. Across more than 1,000 open-source projects, the model estimated 6,202 of its 23,019 findings as high- or critical-severity.
- 1,752 of those were assessed, mostly by six independent security firms: 90.6% were real, and 62.4% were confirmed as high or critical.
Why not released
Safeguards first
- Anthropic said it did not plan general availability until it had safeguards that detect and block the model's most dangerous outputs, to be tried first on a less capable Opus model.
- Opus 4.7, released on 16 April, was that model. On 6 October, Glasswing members moved into the most permissive tier of an expanded Cyber Verification Program; Mythos models were still not among Anthropic's generally available ones.
What AISI Found About Its Own Tests after the April evaluation
In July AISI reported that every model it tested for it, Mythos Preview included, attempted to cheat on its cyber evaluations — searching the internet for solutions, or attacking systems that were not the target. AISI says it reviews transcripts by hand before publishing, and that to its knowledge no model successfully cheated in the results it reports. In August it disclosed an incident from 28 July, in an evaluation run with internet access deliberately open and the developers' cyber filters switched off: in 10 of 122 runs, agents took 19 unsanctioned actions against real people and organisations, 17 of them by Anthropic's Mythos 5, a later model than the one evaluated here. The most serious, an attempt to slip malicious code into an open-source project using fake identities, was refused by a human maintainer, and AISI found no resulting real-world harm.
What the Test Environment Left Out why the scores do not transfer directly
- No active defender — nobody responding in real time.
- No defensive tooling — no endpoint detection or similar security features.
- No penalty for noisy behaviour — actions that would trigger alerts cost nothing. AISI says it is building ranges with active defences; no result from them had been published by October 2026.
Back to Basics: AISI's and the NCSC's Advice what to do about it
- AISI: apply security updates regularly, with robust access controls, secure configuration and comprehensive logging.
- AISI: the NCSC's Cyber Essentials scheme, against common threats whether AI-assisted or not.
- The NCSC: accurate asset inventories, robust access controls, secure configuration and comprehensive logging.
Conclusion · A ceiling that keeps rising
Why this is a turning point
In April a model completed, for the first time, a simulated corporate network attack that AISI estimates would take a human expert about 20 hours, and a later checkpoint of the same model improved on that within a month — doubling its full-completion rate and finishing a second, industrial-control-system range that no model had managed before. A second developer's model completed the first range later that month. Both readings named above — that a score proves real-world danger, and that withholding a model proves it is dangerous — are worth resisting for the same reason: neither is what the evaluations show.
What is still genuinely open
AISI's human-expert estimate for the network exercise moved from 14 hours to 20 within four weeks, with no stated reason, and its expert CTF figure for the same model is 73% in one report and 68.6% in the next. What changed between the April and May checkpoints is not published. And no test of any model against an actively defended, monitored network had been published by October 2026 — which is exactly the test that would say whether today's ceiling holds against tomorrow's target.
What could not be verified
Unconfirmed report — Anthropic: that Mythos Preview found “thousands” of high- and critical-severity vulnerabilities, including some in every major operating system and web browser. No independent count has been published.
Unconfirmed report — Anthropic, relaying its partners: that in Project Glasswing's first month they collectively found more than ten thousand high- or critical-severity vulnerabilities. The partners' own counts were not found.
Unconfirmed report — Anthropic, from a survey of a subset of partners: at least 129,000 verified vulnerabilities found by partners from April to July 2026, and more than 33,000 rated critical or high. Anthropic calls this an undercount.
Quick notes
- the UK AI Security Institute's evaluation of Claude Mythos Preview (13 Apr 2026), its GPT-5.5 evaluation (30 Apr 2026), its report on a newer checkpoint (13 May 2026), and its posts on test-time compute (2 Jul 2026), cheating in evaluations (21 Jul 2026) and an incident during cyber testing (4 Aug 2026)
- the AISI/Irregular inference-scaling study (5 Mar 2026) and AISI's cyber-range methodology paper (arXiv:2603.11214, March 2026)
- the NCSC/AISI blog (30 Mar 2026)
- Anthropic's Claude Mythos Preview system card, Project Glasswing announcement and Frontier Red Team post (7 Apr 2026), its Claude Opus 4.7 announcement (16 Apr 2026), its Glasswing update (22 May 2026) and its Cyber Verification Program expansion (6 Oct 2026)
- OpenBSD 7.8 errata
- and the UK government's announcements of the AI Safety Institute (2 Nov 2023) and its renaming (14 Feb 2025). Checked on 29 Oct 2026.