# Winthropic

> Independent daily evaluation of Anthropic's Claude models on a fixed, versioned task battery. Measures drift and variance over time. Not affiliated with Anthropic.

Every page is also available as markdown: append .md (for example https://winthropic.com/about.md), or request the HTML path with Accept: text/markdown.

## Core

- [The Index (home)](https://winthropic.com/index.md): today's readings and the longitudinal trace
- [Methodology](https://winthropic.com/methodology.md): versioned battery, deterministic scoring, frozen request shape
- [About](https://winthropic.com/about.md): who and what runs this instrument

## Services

- [Winthrop's Token Audit](https://winthropic.com/audit.md): deterministic audit of Claude API spend (caching, tiering, batch savings); free summary, full report $19. Savings-floor guarantee, enforced structurally: if identified savings are under $19, the fee is declined and the full findings are free; the 402 response carries a machine-readable guarantee object

## Data

- [Index API](https://winthropic.com/api/index.json): all readings, JSON
- MCP server at https://winthropic.com/mcp (Streamable HTTP): add with `claude mcp add --transport http winthropic https://winthropic.com/mcp` and ask "did Claude drift?"
- Machine-buyable audit: POST a usage CSV to https://winthropic.com/api/audit for a free teaser and an HTTP 402 with payment rails; pay to GET the itemized findings as JSON
- [Agent interop](https://winthropic.com/interop.md): every endpoint, the MCP tool list, and the 402 / L402 flow for the machine-buyable audit
- [OpenAPI spec](https://winthropic.com/openapi.json)
- [RSS](https://winthropic.com/feed.xml) and [JSON Feed](https://winthropic.com/feed.json): one entry per daily run

---

# About Winthropic

Winthropic is an independent measurement project pointed at the Claude family of models. It asks one question daily: did the model that answered yesterday answer the same way today?

A fixed battery of tasks runs against each model every day under an identical request shape; every reading lands on the public trace. It is a longitudinal record, not a launch-day benchmark.

The site is operated by an autonomous loop built on producer/verifier separation: the agents that run evaluations never grade their own work, and the process that proposes changes to the site never computes its own success numbers.

Winthropic is not affiliated with, endorsed by, or connected to Anthropic. It is kept by Matt Loseke (Traverse City, MI) and the loop he supervises.

---

# Winthropic methodology

- Daily at 07:30 Eastern, each tracked Claude model receives the same versioned battery: arithmetic, extraction, logic, classification, instruction-following, JSON emission, and code.
- Scoring is deterministic: exact match, regex, numeric tolerance, JSON equality, and sandboxed execution against test vectors for code. No LLM judges.
- Frozen request shape: no system prompt, fixed token limit, temperature 0 where accepted. Claude 5 models reject temperature and always sample; their variance band is part of the record.
- Batteries are versioned, never edited in place; readings are only compared within one version.
- Fully failed runs (outage or auth) are recorded separately and never overwrite a good reading.
- Scale honesty: a dozen tasks is a barometer, not a physics lab. It measures direction and stability, not intelligence.

---

## Latest readings

# Winthropic :: Claude changed last night. Would you know?

Independent daily evaluation of Anthropic's Claude models on a fixed task battery, published as an open record. Not affiliated with Anthropic.

The same battery, every model, every morning at 07:30 Eastern. When the answers change, the trace moves, and subscribers hear about it. Only measured points are published.

## Today's readings :: 2026-09-02

Net drift (mean pass-count change vs previous run, all models): -0.7 (within day-to-day variance)

| Model | Today | vs previous | vs 7 runs back | math | instruction | extraction | reasoning | classification | json | code |
|---|---|---|---|---|---|---|---|---|---|---|
| haiku-4-5 | 11/12 | no deviation | n/a | 2/2 | 3/4 | 1/1 | 1/1 | 1/1 | 1/1 | 2/2 |
| sonnet-5 | 11/12 | -1 | n/a | 2/2 | 3/4 | 1/1 | 1/1 | 1/1 | 1/1 | 2/2 |
| opus-5 | 10/12 | -1 | n/a | 2/2 | 3/4 | 1/1 | 1/1 | 1/1 | 0/1 | 2/2 |

Note: Claude 5 models do not accept temperature control, so some daily movement is sampling variance rather than drift. Daily repetition is how the two are separated.

- Data: https://winthropic.com/api/index.json (OpenAPI: https://winthropic.com/openapi.json)
- Per-run records with every failure's output: https://winthropic.com/api/runs/<date>.json
- Method: https://winthropic.com/methodology.md
- About: https://winthropic.com/about.md
- Feeds: https://winthropic.com/feed.xml and https://winthropic.com/feed.json