OxAlpha benchmarks: how OxAlpha scores against frontier models

The OxAlpha aggregate scoreboard

The headline number comes from an aggregate community reasoning suite: a pooled set of logic, mathematics, science, and multi-step planning tasks that independent testers run against OxAlpha and the named frontier models under identical prompts and identical settings. On that suite, OxAlpha lands around 80%. The nearest competitor is roughly fifteen points behind.

Read the table the way the community does: as a consistent direction, not a decimal-precise ranking. No lab has published official scores for OxAlpha, so every figure here is a community measurement. What makes the OxAlpha lead credible is repetition — separate testers with separate task sets keep landing OxAlpha in the high seventies to low eighties while GPT-5, Fable 5, and Grok 4 cluster in the high fifties to mid sixties. For prompt-level head-to-heads rather than aggregates, see OxAlpha vs GPT-5 and OxAlpha vs Fable 5.

One more pattern is worth naming before the deep dives. The OxAlpha lead is not one lucky suite: when testers slice the aggregate into categories, OxAlpha finishes first in reasoning, first in long-context retrieval, and first in long-horizon coding, while the named models trade second place among themselves. A model that wins a single category can be tuned for it; a model like OxAlpha that wins across unrelated categories at once is usually just the stronger system.

OxAlpha reasoning benchmarks

Reasoning is where the gap is widest, and it is not hard to see why. OxAlpha is thinking-first: it produces an internal chain of reasoning before it writes a single word of its answer. On benchmark tasks, that design shows up as fewer skipped steps, fewer confidently wrong shortcuts, and a visible habit of checking intermediate work before committing.

Logic and mathematics

On multi-step logic puzzles and competition-style math problems, testers report that OxAlpha works the derivation in order instead of pattern-matching to a similar problem it has seen before. When OxAlpha makes an arithmetic slip mid-solution, the reasoning phase frequently catches it and repairs the chain — a behavior the comparison models show far less often. The result is a large accuracy edge on exactly the problems that punish guessing.

Science and planning tasks

The suite also includes scientific analysis questions and constraint-heavy planning tasks: scheduling under conflicting rules, resource allocation, adversarial word problems. OxAlpha holds its lead here too, and testers note that OxAlpha degrades gracefully — when a task is genuinely underspecified, OxAlpha tends to state the missing assumption rather than invent a fact. That honesty is itself a scored behavior in several community rubrics, and OxAlpha earns points on it consistently.

Across the reasoning suite as a whole, the OxAlpha pattern is stable rather than spiky. OxAlpha does not post perfect scores on the easy tiers and then collapse on the hard ones; instead, OxAlpha accuracy declines gently as difficulty rises, while the comparison models fall off sharply at the hardest tier. That shape — a slope instead of a cliff — is what a real reasoning advantage looks like when you plot it.

OxAlpha coding benchmarks: long-horizon repo tasks

Coding evaluations for OxAlpha go beyond single-function puzzles. The community’s most interesting results come from long-horizon repository tasks: give the model a real codebase, a failing test suite or a feature ticket, and let it plan, edit, run, and iterate over many steps. These tasks measure what working engineers actually care about — whether a model can carry a goal across hours of work without drifting.

Multi-file bug hunts

In multi-file debugging exercises, OxAlpha traces data flow across module boundaries before proposing a fix, and its patches tend to respect the surrounding architecture instead of fighting it. Testers report materially fewer regressions from OxAlpha patches than from the comparison models on the same tickets, largely because OxAlpha reads more of the repository before it writes.

Sustained agentic sessions

The 1M-token context window matters most here. Because OxAlpha can hold an entire large repository in view while it reasons, it avoids a whole class of mistakes that come from editing code it cannot see. In community agent runs, OxAlpha completes multi-hour plan-edit-test loops with the original objective still intact at the end — the failure mode where an agent quietly redefines its task midway through is rare with OxAlpha. The full capability picture, including tool calling and multimodal input, is on the OxAlpha capabilities page.

Coding results resist a single percentage, so the community reports task completion instead: on the shared long-horizon repo suite, OxAlpha finishes a clearly larger share of tickets end-to-end than GPT-5, Fable 5, or Grok 4 under the same harness. Just as telling, reviewers rate OxAlpha diffs as easier to merge, because OxAlpha explains what it changed and why inside the patch itself.

OxAlpha long-context retrieval at 1M tokens

A million-token window is only useful if the model can actually find things in it, so the community runs needle-in-a-haystack and multi-needle retrieval tests against OxAlpha at increasing depths: 128K, 256K, 512K, and the full million. The short version of the results: OxAlpha stays accurate deep into the window, including retrieval targets planted hundreds of pages apart that must be combined to answer correctly.

The harder variant is reasoning over retrieved material rather than merely quoting it — for example, planting two contradictory policy clauses half a million tokens apart and asking which one governs a scenario. OxAlpha handles these cross-document conflicts well, which testers attribute to the same thinking-first loop that drives its reasoning scores: OxAlpha appears to survey the context during its reasoning phase rather than answering from the most recent chunk. Long-context performance does come with latency — big prompts take longer to process — but OxAlpha accuracy holds, and that trade is one most long-document workflows will take. The mechanics behind this behavior are covered in how OxAlpha works.

Instruction following and structured output

The least glamorous benchmarks may be the most predictive of real-world value. Instruction-following suites score a model on compliance: respect the word limit, use the required format, obey the negative constraint, keep all seven rules in force at once. OxAlpha scores near the top of these suites, and its lead grows as the instruction count rises — the same long-attention behavior that shows up in retrieval keeps every constraint live for OxAlpha instead of letting early rules fade.

Structured output testing is blunter: request JSON against a schema, thousands of times, and count failures. Community harness runs report that OxAlpha emits valid, schema-conforming JSON with a failure rate low enough that many builders have removed their retry wrappers entirely. Tool-calling evaluations point the same way — function arguments from OxAlpha arrive well-typed, and OxAlpha recovers sensibly when a tool returns an error instead of looping on the failure. For anyone building agents or pipelines, these quiet numbers are a bigger endorsement of OxAlpha than the headline reasoning score.

There is a practical takeaway hiding here. If you deploy OxAlpha behind a schema, a template, or a strict prompt contract, the benchmark evidence says OxAlpha will hold that contract under load — and that reliability, multiplied across thousands of automated calls, is where a free model quietly saves real engineering time.

Methodology, caveats, and reproducing OxAlpha results

How the numbers are produced

Every OxAlpha score on this page is community-run. Independent testers call OxAlpha through OpenRouter under the model id stealth/ox-alpha, run the identical prompts against the comparison models, and publish prompts, transcripts, and grading rubrics so others can re-run them. There is no vendor in the loop: nobody who benefits from an OxAlpha win controls the task selection, and nobody has official numbers to defend.

What the numbers cannot tell you

Three caveats keep this page honest. First, these are directional measurements, not official scores — treat ~80% as “around eighty,” never as a precise figure. Second, community suites skew toward reasoning-heavy tasks, which is exactly where the thinking-first design of OxAlpha shines; a suite weighted toward speed or creative writing would narrow the gap. Third, OxAlpha is a stealth model that could be updated silently at any time, so a result from last month may not describe the OxAlpha you query today. Sample sizes also vary by category: the aggregate score is sturdy, but any single-category number is noisier.

Reproduce it yourself for free

The best property of the OxAlpha scoreboard is that verifying it costs nothing. Open the free OxAlpha chat — no sign-up, no API key — and paste in the hardest reasoning problem you know, ideally one where you can grade the answer yourself. Then run the same prompt through whichever frontier model you currently pay for and compare. Developers can go further and point an existing evaluation harness at the OxAlpha model id on OpenRouter, since the listing is free for both input and output tokens. Five prompts of your own will tell you more than any table on this page.

OxAlpha benchmarks FAQ

Are the OxAlpha benchmark scores official?

No. There are no official scores because no lab has claimed OxAlpha. Every figure on this page comes from community testing, which is why we write ~80% rather than a precise number. The strength of the evidence is its consistency: many independent testers, many task sets, and OxAlpha on top of essentially all of them.

How big is the OxAlpha lead over GPT-5?

On the aggregate reasoning suite, OxAlpha scores around 80% against roughly 65% for GPT-5 — about fifteen points. That is a generation-sized gap on reasoning-heavy work, though GPT-5 stays closer in some individual categories. The OxAlpha vs GPT-5 page breaks the comparison down prompt by prompt.

Does OxAlpha beat Fable 5 on coding?

On the community’s long-horizon repository tasks, yes — OxAlpha leads on aggregate, and Fable 5 posts roughly 62% on the reasoning suite against ~80% for OxAlpha. Fable 5 remains a strong coding model, and the honest per-category picture, including where the two are close, lives on the OxAlpha vs Fable 5 page.

Why does OxAlpha score so well on reasoning?

The consistent explanation is the thinking-first design: OxAlpha spends tokens on internal reasoning before answering, which lets it decompose problems, test intermediate steps, and catch its own mistakes. Benchmarks built from multi-step problems reward exactly that behavior, and OxAlpha pairs it with a 1M-token window that keeps every relevant detail in view.

Can I run these benchmarks against OxAlpha myself?

Yes, and you should. OxAlpha is free through the chat on this site and free via OpenRouter under stealth/ox-alpha, so reproducing any result costs only time. Most published community suites include their prompts and grading rubrics, and even an informal test — your own hardest problems, graded by you — is a legitimate check on everything reported here.

Where does OxAlpha lose?

OxAlpha is not the fastest model — the OxAlpha reasoning phase adds seconds of latency, and huge contexts add more. Suites that reward instant answers or subjective creative style show a much narrower OxAlpha advantage, and in some speed-weighted tests the named models win outright. If your workload is short, casual, and latency-sensitive, benchmark that specifically before switching to OxAlpha.

How often are the OxAlpha benchmark numbers updated?

This page tracks the community consensus, and it changes when the consensus moves. Because OxAlpha is a stealth listing, the model behind stealth/ox-alpha could be updated without notice, so testers periodically re-run their suites against OxAlpha to confirm the numbers still hold. If a silent update ever shifts OxAlpha results materially, the tables here change with them.

Which benchmark matters most when evaluating OxAlpha?

The one closest to your workload. Agent builders should weight the structured output and tool-calling results for OxAlpha; research teams should weight long-context retrieval; engineering teams should weight the long-horizon coding tasks. The aggregate reasoning score is the OxAlpha headline, but the category results above are the numbers worth acting on.