Tratopedia
繁中
Settings

Text size

Language

Theme

High contrast

Version

v1.230.2

The release this page was built from. It is what the service worker caches under.

Slide deck

Back

The Claude Code Quality Postmortem — Three Unrelated Changes

Three unrelated Claude Code changes — a lowered reasoning default, a caching bug erasing thinking history, a verbosity-capping prompt — each passed its own review. An AMD engineer audited the same decline three weeks before Anthropic explained it; v2.1.116 fixed all three.

  1. Engineering Postmortem · April 2026

    Three Unrelated Changes That Looked Like One Regression

    Each passed its own review; together they looked like one broad decline. All three were resolved in v2.1.116, and the API itself was unaffected.

    Claude Code quality postmortem1 / 13

  2. What happened

    Three separate problems

    Claude Code, the Agent SDK and Cowork were affected; the API itself was not.

    ProblemWhat changedBeganFixedDays
    OneA lowered reasoning-effort default4 Mar7 Apr34
    TwoA caching bug that erased thinking history26 Mar10 Apr (v2.1.101)15
    ThreeA verbosity-capping system prompt16 Apr20 Apr (v2.1.116)4

    Claude Code quality postmortem2 / 13

  3. Problem One · 4 March – 7 April

    A lowered default reasoning effort

    • highthe default when Opus 4.6 shipped in Claude Code, February
    • mediumthe default from 4 March
    • highrestored on 7 April
    • xhighthe default for Opus 4.7 after the fix

    Claude Code quality postmortem3 / 13

  4. Problem Two · 26 March – 10 April

    A caching bug that kept clearing reasoning history

    Wider repository context in Code Review is what let Opus 4.7 catch the offending pull request, where Opus 4.6 had not.

    1. 01The costAfter an hour idle, the next message is a full cache miss; in an extreme case a 900,000-token session could write over 900,000 tokens to cache at once.
    2. 02The fix triedElide old thinking after an idle period, to cut that cost.
    3. 03The bugIts implementation introduced Problem Two; fixed in v2.1.101 on 10 April.

    Claude Code quality postmortem4 / 13

  5. Problem Three · 16 – 20 April

    A system-prompt rule capping output length

    The ablation analysis that eventually caught it is now to run for every system-prompt change.

    • 16 Aprships alongside Opus 4.7
    • −3%drop on the one evaluation that caught it
    • 20 Aprfixed; all three resolved (v2.1.116)

    Claude Code quality postmortem5 / 13

  6. Anthropic's own diagnosis

    Why three changes looked like one

    Each passed its own review

    Human review, automated tests, evaluations.

    Nothing tested them together

    No process was responsible for asking what several simultaneous changes would do together.

    Different slices, different schedules

    In aggregate they looked like one broad, inconsistent decline, not three separate causes.

    Claude Code quality postmortem6 / 13

  7. Timeline · February – 7 April

    From the first change to the first fix

    1. 02Opus 4.6 ships in Claude Code with reasoning effort defaulted to high.
    2. 02-12The setting named redact-thinking-2026-02-12 hides reasoning from the interface. An outside audit later names it as a possible cause; Anthropic's own account never does.
    3. 03-04The default reasoning effort drops to medium (Problem One begins).
    4. 03-26The caching bug ships (Problem Two begins).
    5. 04-03Stella Laurenzo, AMD's AI director, files a GitHub issue analysing 6,852 sessions (date inferred from The Register's report).
    6. 04-06The Register reports the issue.
    7. 04-07Problem One is fixed; the default returns to high (xhigh for Opus 4.7). Anthropic's Boris Cherny publicly disputes Laurenzo's theory.

    Claude Code quality postmortem7 / 13

  8. Timeline · 10 April – 14 May

    The fixes, the postmortem, the coverage

    1. 04-10Problem Two is fixed (v2.1.101).
    2. 04-16A verbosity-capping system prompt ships alongside Opus 4.7 (Problem Three begins).
    3. 04-20Problem Three is fixed; all three resolved (v2.1.116).
    4. 04-23Anthropic publishes the postmortem and resets usage limits for all subscribers; Boris Cherny elaborates the caching mechanism on Hacker News the same day.
    5. 04-24Fortune reports user backlash, cancellations, and an Anthropic statement citing infrastructure strain.
    6. 05-14InfoQ publishes a synthesis, including a Reddit-reported concern the postmortem does not address.

    Claude Code quality postmortem8 / 13

  9. What complicates it

    An outside audit came first

    Stella Laurenzo, director of AMD's AI group, analysed 6,852 Claude Code sessions and filed a GitHub issue — three weeks before Anthropic's own account. It named a different change as the correlate.

    • 3 AprilStella Laurenzo files a data-backed complaint naming a mid-February change.
    • 23 AprilAnthropic's own postmortem names three different changes, none from February.

    Claude Code quality postmortem9 / 13

  10. One explanation, two accounts

    What each names as the cause

    The two reports of the audit differ on when the setting reached Claude Code: by its own name, or early March in v2.1.69.

    SourceWhat it names as the cause
    Stella Laurenzo, GitHub auditThe setting redact-thinking-2026-02-12, correlated with a measured shift toward shallower, edit-first behaviour.
    Anthropic, first public response (7 Apr)That setting hides reasoning from the UI only and does not reduce reasoning.
    Anthropic, 23 April postmortemThree product-layer changes — no mention of compute or of the mid-February setting.

    Claude Code quality postmortem10 / 13

  11. What others add

    Compute: raised, not established

    Users, to Fortune

    Some felt “gaslit” by Anthropic's communications before 23 April; some say they have cancelled.

    Anthropic, to Fortune

    Demand “stretched” its infrastructure “particularly at peak hours” — in general terms, tied to none of the three bugs.

    A rival's leaked memo

    OpenAI's revenue chief called it a “strategic misstep” in an internal memo first reported by CNBC — a characterisation, not a finding.

    Claude Code quality postmortem11 / 13

  12. What changes now

    Five follow-up measures

    • More internal staff run the exact public build of Claude Code, not a separate test build.
    • Per-model evaluations for every system-prompt change, continuing the ablation analysis.
    • Tooling to review and audit system-prompt changes; guidance in CLAUDE.md gating model-specific changes to their model.
    • A soak period, a broader evaluation suite and a gradual rollout for any change that could trade off against intelligence.
    • Wider repository context for Code Review, and @ClaudeDevs on X plus a centralised GitHub thread to explain product decisions.

    Claude Code quality postmortem12 / 13

  13. The lesson underneath, and what's still open

    Test what several changes do at once

    Each change passed its own review; nothing was responsible for testing them together.

    Open: the mid-February setting

    Whether the change an outside audit pointed to played any real part in the decline.

    Open: quiet delegation to Haiku

    A Reddit-reported concern, relayed by InfoQ, that the postmortem does not address — hardest to catch in unattended pipelines.

    Claude Code quality postmortem13 / 13