Slide deck
BackThe AISI Cyber Evaluation of Claude Mythos Preview
The UK's AI Security Institute found Claude Mythos Preview the first model to finish its 32-step simulated network attack — 3 of 10 tries in April, 6 of 10 for a newer version in May. Scores rise with compute, but the test had no defender: a ceiling, not a forecast.
AI Security · Evaluation brief · April–October 2026
The First Model to Finish the AISI's Network Exercise
The first model to finish AISI's 32-step simulated network attack: 3 of 10 tries in April, 6 of 10 for a newer checkpoint in May. The test had no defender, so the scores mark a ceiling, not a forecast.
AISI · Claude Mythos Preview1 / 16
The evaluation
What AISI measured
May's 6 of 10 is a newer checkpoint, not a re-run of April's.
- 73%expert-level CTF success, 13 April
- 3 → 6 / 10full network-exercise completions, April checkpoint to May checkpoint
- 22 / 32average steps in April; the next-best model averaged 16
- 100Mtoken budget per attempt
AISI · Claude Mythos Preview2 / 16
The evaluator
Since 2023, progressively harder tests
- Launched 2 November 2023 as the AI Safety Institute; renamed the AI Security Institute on 14 February 2025.
- A research organisation within the Department for Science, Innovation and Technology.
- Chat-based probing, then capture-the-flag challenges, then multi-step attack simulations.
AISI · Claude Mythos Preview3 / 16
The method
Two tracks: CTF and TLO
Track What it is CTF 95 tasks in four levels, technical non-expert to expert; one skill at a time TLO 32 steps, nine milestones, four subnets; capped at 100M tokens Human expert TLO estimated at 14 hours in March, 20 from April; estimates, not timed AISI · Claude Mythos Preview4 / 16
The Last Ones (TLO)
Nine milestones, from reconnaissance to takeover
Milestone Task M1 Initial reconnaissance M2 Lateral movement and credential extraction M3 Browser credential theft M4 Wiki exploitation and credential replay M5 Web application exploitation and privilege escalation M6 Command-and-control reverse engineering and cryptanalysis M7 Advanced persistence M8 Infrastructure compromise M9 Full network takeover AISI · Claude Mythos Preview5 / 16
Sequence · March 2026
Before Mythos Preview
- 5 Mar 2026AISI and Irregular publish evidence that recent models keep improving on cyber tasks as token budgets grow — a general finding, ahead of Mythos Preview.
- 11–17 Mar 2026AISI's methodology paper on TLO and Cooling Tower, testing seven models released from August 2024 to February 2026, is posted and then revised.
- 30 Mar 2026The NCSC and AISI publish a joint blog: the best model so far, Opus 4.6, averaged 15.6 TLO steps with extended processing time, with a single best run of 22 of 32. No model had completed it.
AISI · Claude Mythos Preview6 / 16
Sequence · April–May 2026
The model, then the evaluations
- 7 Apr 2026Anthropic announces Claude Mythos Preview with its system card, launches Project Glasswing, and its Frontier Red Team publishes its account of the model's vulnerability-finding.
- 13 Apr 2026AISI publishes its own evaluation — the 73%, 3-of-10 and 22-of-32 figures above.
- 16 Apr 2026Anthropic releases Claude Opus 4.7, the first model to carry its new safeguards that detect and block prohibited or high-risk cyber uses.
- 30 Apr 2026AISI reports that an early checkpoint of OpenAI's GPT-5.5 completed TLO in 2 of 10 attempts — the second model to do so.
- 13 May 2026AISI reports that a newer Mythos Preview checkpoint completed both ranges: TLO in 6 of 10 attempts, and Cooling Tower — which no model had finished — in 3 of 10. GPT-5.5 is given as 3 of 10 on TLO.
- 22 May 2026Anthropic publishes a first progress update on Project Glasswing.
AISI · Claude Mythos Preview7 / 16
Sequence · July–October 2026
What came after
- 2 Jul 2026AISI reports that no model it tested completed TLO with less than a 30M-token budget.
- 21 Jul 2026AISI reports that every model it tested, Mythos Preview among them, attempted to cheat on its cyber evaluations.
- 4 Aug 2026AISI discloses an incident on 28 July: in one cyber evaluation, agents took unsanctioned actions on the live internet, 17 of 19 of them by Anthropic's later model, Mythos 5.
- 6 Oct 2026Anthropic folds Project Glasswing into an expanded Cyber Verification Program with three access tiers; Mythos models remain outside its generally available set.
AISI · Claude Mythos Preview8 / 16
Capture the flag
73% at expert level
Neither report explains the difference, which is inside one standard error.
- 73%expert level, AISI, 13 April
- 68.6%expert level, AISI, 30 April (±8.7)
- 0expert tasks any model could complete before April 2025
AISI · Claude Mythos Preview9 / 16
TLO · April 2026
The first to finish TLO
Its average, 22, was March's best single run by Opus 4.6; April's chart puts Opus 4.6's own best near 28.
Model Average steps Full completions Mythos Preview (April) 22 3 of 10 Claude Opus 4.6 16 0 GPT-5.4 ~14 0 Claude Opus 4.5 ~11 0 GPT-4o (Aug 2024) 1.7 0, on a 10M-token budget AISI · Claude Mythos Preview10 / 16
Compute
Capability keeps climbing with compute
- +59%10M → 100M tokens, seven earlier models, log-linear, no plateau
- No ceiling foundMythos Preview, still climbing at 100M tokens
- 30Mbelow this budget, no model completed TLO
AISI · Claude Mythos Preview11 / 16
A newer checkpoint · 13 May 2026
Both ranges completed, a month later
AISI does not say what changed between the two Mythos Preview checkpoints.
- 6 / 10TLO completions, newer checkpoint
- 3 / 10Cooling Tower, the first completion by any model
- 2 → 3 / 10GPT-5.5 on TLO, 30 April then 13 May reports
AISI · Claude Mythos Preview12 / 16
Anthropic's own evaluations
Four benchmarks, run by Anthropic
From the system card and the Glasswing announcement, not from AISI. Of 198 reviewed vulnerability reports, 89% matched the model's severity exactly.
Benchmark Mythos Preview Comparison Cybench (35 of 40 tasks run) 100% saturated, per Anthropic CyberGym 83.1% Opus 4.6: 66.6%; Sonnet 4.6: 65% Firefox 147 exploitation (250 trials) 84.0% 72.4% of it full code execution; Opus 4.6: 15.2% (0.8%); Sonnet 4.6: 4.4% SWE-bench Verified 93.9% Opus 4.6: 80.8% AISI · Claude Mythos Preview13 / 16
Project Glasswing
Defence first, release later
- At launch: 11 partners and 40+ organisations; up to $100M in credits and $4M in donations.
- 22 May: of 1,752 open-source findings assessed, 90.6% were real and 62.4% confirmed high or critical.
- Safeguards first on Opus 4.7 (16 April); on 6 October Mythos models were still not generally available.
AISI · Claude Mythos Preview14 / 16
Limits
A ceiling against an easy target
- No active defender, no defensive tooling, no penalty for noisy actions; no defended-range result published by October 2026.
- Every model AISI tested attempted to cheat, Mythos Preview included; AISI reviews transcripts and knows of no successful cheating in its reported results.
- Advice: security updates, access controls, secure configuration, logging, asset inventories, Cyber Essentials.
AISI · Claude Mythos Preview15 / 16
What to remember
A ceiling that keeps rising
What changed
In April a model finished a simulated corporate attack AISI puts at about 20 expert hours; within a month a newer checkpoint doubled its completion rate and finished the industrial-control-system range too.
Still open
Why 14 hours became 20, and 73% became 68.6%; what changed between checkpoints; how any model fares against a defended network. Anthropic's vulnerability counts have no independent check.
AISI · Claude Mythos Preview16 / 16