Shared run
Judge
Golden-diff judge · judge_verdict
Duration
1009h 38m
Policy
pass@1
Tokens
444M in / 91.6M out
Golden-diff judge · judge_verdict
Grok 4.6 — thinking levels
grok-4-6-mediumthinking medium
43 attempts
83.7%
36/43/43
21.3M
Powered by YourCodingBench — benchmark coding models on your own repo's commits. sign in
grok-4-6-highthinking high
43 attempts
88.4%
38/43/43
22M
grok-4-6thinking default
43 attempts
90.7%
39/43/43
22.1M
Gemini 3.1 Pro
gemini-3.1-pro
43 attempts
27.9%
12/43/43
119M
Claude Sonnet 5 — thinking levels
claude-sonnet-5-mediumthinking medium
43 attempts
61.9%
26/42/43
6.3M
1 judge failed
claude-sonnet-5thinking default (high)
43 attempts
73.8%
31/42/43
10.4M
1 judge failed
claude-sonnet-5-xhighthinking xhigh
43 attempts
83.7%
36/43/43
10.7M
GPT-5.6 Terra — thinking levels
gpt-5.6-terrathinking default (medium)
43 attempts
59.5%
25/42/43
10.8M
1 judge failed
gpt-5.6-terra-highthinking high
43 attempts
65.1%
28/43/43
13.5M
gpt-5.6-terra-xhighthinking xhigh
43 attempts
78%
32/41/43
17.4M
2 judge failed
GPT-5.6 Sol — thinking levels
gpt-5.6-solthinking default (medium)
43 attempts
80.5%
33/41/43
14.8M
2 judge failed
gpt-5.6-sol-highthinking high
43 attempts
83.7%
36/43/43
20.3M
gpt-5.6-sol-xhighthinking xhigh
43 attempts
88.1%
37/42/43
21.4M
1 judge failed
Claude Fable 5 — thinking levels
claude-fable-5-mediumthinking medium
43 attempts
76.7%
33/43/43
4.3M
claude-fable-5thinking default (high)
43 attempts
83.7%
36/43/43
4.2M
claude-fable-5-xhighthinking xhigh
43 attempts
88.4%
38/43/43
5.9M
Grok 4.7 — thinking levels
grok-4.7-lowthinking low
43 attempts
53.5%
23/43/43
22.8M
grok-4.7-mediumthinking medium
43 attempts
58.1%
25/43/43
48.7M
grok-4.7thinking default (high)
43 attempts
62.8%
27/43/43
62.7M
grok-4.7-xhighthinking xhigh
43 attempts
60.5%
26/43/43
66.1M
Claude Opus 5.5 — thinking levels
claude-opus-5-5-lowthinking low
43 attempts
46.5%
20/43/43
824k
claude-opus-5-5thinking default (medium)
43 attempts
67.4%
29/43/43
2M
claude-opus-5-5-highthinking high
43 attempts
69.8%
30/43/43
2.9M
claude-opus-5-5-xhighthinking xhigh
43 attempts
67.4%
29/43/43
6M
536M total — 444M in / 91.6M out
925 of 1032 terminal attempts reporting
pass rate = passed / (judged + agent failures) — a timeout, empty diff, agent error, or blown budget counts as a FAIL without a verdict (D48); infra failures and owner-cancelled attempts are excluded from the rate and annotated. Verdicts by grok-4.5 · effort=high · fast (pinned, D22) · verdict rubric v5 (D54). PASS = a correct solution to the same problem as the reference, equivalent or a valid alternative; FAIL = anything less. The only scorer since D26 (judge-only grading).