Shared run
Judge
Golden-diff judge · judge_verdict
Duration
17h
Policy
pass@1
Tokens
258M in / 66M out
Golden-diff judge · judge_verdict
Grok 4.6 — thinking levels
grok-4-6-mediumthinking medium
43 attempts
83.7%
36/43/43
21.3M
Powered by YourCodingBench — benchmark coding models on your own repo's commits. sign in
grok-4-6-highthinking high
43 attempts
88.4%
38/43/43
22M
grok-4-6thinking default
43 attempts
90.7%
39/43/43
22.1M
Gemini 3.1 Pro
gemini-3.1-pro
43 attempts
27.9%
12/43/43
119M
Claude Sonnet 5 — thinking levels
claude-sonnet-5-mediumthinking medium
43 attempts
61.9%
26/42/43
6.3M
1 judge failed
claude-sonnet-5thinking default (high)
43 attempts
73.8%
31/42/43
10.4M
1 judge failed
claude-sonnet-5-xhighthinking xhigh
43 attempts
83.7%
36/43/43
10.7M
GPT-5.6 Terra — thinking levels
gpt-5.6-terrathinking default (medium)
43 attempts
59.5%
25/42/43
10.8M
1 judge failed
gpt-5.6-terra-highthinking high
43 attempts
65.1%
28/43/43
13.5M
gpt-5.6-terra-xhighthinking xhigh
43 attempts
78%
32/41/43
17.4M
2 judge failed
GPT-5.6 Sol — thinking levels
gpt-5.6-solthinking default (medium)
43 attempts
80.5%
33/41/43
14.8M
2 judge failed
gpt-5.6-sol-highthinking high
43 attempts
83.7%
36/43/43
20.3M
gpt-5.6-sol-xhighthinking xhigh
43 attempts
88.1%
37/42/43
21.4M
1 judge failed
Claude Fable 5 — thinking levels
claude-fable-5-mediumthinking medium
43 attempts
76.7%
33/43/43
4.3M
claude-fable-5thinking default (high)
43 attempts
83.7%
36/43/43
4.2M
claude-fable-5-xhighthinking xhigh
43 attempts
88.4%
38/43/43
5.9M
324M total — 258M in / 66M out
681 of 688 terminal attempts reporting
pass rate = passed / (judged + agent failures) — a timeout, empty diff, agent error, or blown budget counts as a FAIL without a verdict (D48); infra failures and owner-cancelled attempts are excluded from the rate and annotated. Verdicts by grok-4.5 · effort=high · fast (pinned, D22) · verdict rubric v5 (D54). PASS = a correct solution to the same problem as the reference, equivalent or a valid alternative; FAIL = anything less. The only scorer since D26 (judge-only grading).