Shared task
Prepare for CI, docs, licensing, and publication
commit 98d2046 ↗ · oban-bg/oban_notifiers_phoenix · · committed Nov 6, 2023 · +371 −16 · base 77b12483ee35
what a new run launched now would send
Implement the following change in this repository: Prepare for CI, docs, licensing, and publication Work only inside this repository checkout. Make the code change the task describes, keeping the diff focused — no drive-by refactors. When you are done, leave your changes committed or in the working tree; they are collected automatically. Stay on this snapshot checkout (`task/ycb_oban_notifiers_phoenix_c98d2046ad99e`). Never checkout, pull, or rebase onto `main`. That branch is a README-only orphan. Stay on this HEAD. Do not fetch another default branch. Push only on the Cursor-created `crazy-cursor/…` side branch from this HEAD.
Some past runs of this task were launched with a different prompt (the prompt template changed since, or those runs predate this benchmark's stored prompt). Each run persists the exact prompt it sent at launch — that per-launch record is the audit trail; this page shows only the current one.
| Run | Model | Verdict |
|---|---|---|
| Aug 17, 12:59 UTC · completed | claude-opus-5 | FAIL |
| Aug 17, 12:59 UTC · completed | claude-opus-5 | FAIL |
| Aug 17, 12:59 UTC · completed | claude-opus-5 | FAIL |
| Aug 17, 12:59 UTC · completed | claude-opus-5-low |
Binary verdicts from the pinned judge (D27). Full attempt detail — candidate diff, transcript, timings — lives on each run page's score matrix.
Powered by YourCodingBench — benchmark coding models on your own repo's commits. sign in
| Aug 17, 12:59 UTC · completed | claude-opus-5-low | FAIL |
| Aug 17, 12:59 UTC · completed | claude-opus-5-low | PASS |
| Aug 17, 12:59 UTC · completed | claude-opus-5-medium | PASS |
| Aug 17, 12:59 UTC · completed | claude-opus-5-medium | FAIL |
| Aug 17, 12:59 UTC · completed | claude-opus-5-medium | PASS |
| Aug 17, 12:59 UTC · completed | claude-opus-5-xhigh | PASS |
| Aug 17, 12:59 UTC · completed | claude-opus-5-xhigh | PASS |
| Aug 17, 12:59 UTC · completed | claude-opus-5-xhigh | PASS |
| Aug 17, 12:59 UTC · completed | claude-sonnet-5 | timeout |
| Aug 17, 12:59 UTC · completed | claude-sonnet-5 | PASS |
| Aug 17, 12:59 UTC · completed | claude-sonnet-5 | PASS |
| Aug 17, 12:59 UTC · completed | claude-sonnet-5-low | timeout |
| Aug 17, 12:59 UTC · completed | claude-sonnet-5-low | FAIL |
| Aug 17, 12:59 UTC · completed | claude-sonnet-5-low | PASS |
| Aug 17, 12:59 UTC · completed | claude-sonnet-5-medium | timeout |
| Aug 17, 12:59 UTC · completed | claude-sonnet-5-medium | FAIL |
| Aug 17, 12:59 UTC · completed | claude-sonnet-5-medium | PASS |
| Aug 17, 12:59 UTC · completed | claude-sonnet-5-xhigh | PASS |
| Aug 17, 12:59 UTC · completed | claude-sonnet-5-xhigh | PASS |
| Aug 17, 12:59 UTC · completed | claude-sonnet-5-xhigh | PASS |
| Aug 17, 12:59 UTC · completed | composer-2.5 | PASS |
| Aug 17, 12:59 UTC · completed | composer-2.5 | PASS |
| Aug 17, 12:59 UTC · completed | composer-2.5 | FAIL |
| Aug 17, 12:59 UTC · completed | gemini-3.7-flash | timeout |
| Aug 17, 12:59 UTC · completed | gemini-3.7-flash | timeout |
| Aug 17, 12:59 UTC · completed | gemini-3.7-flash | PASS |
| Aug 17, 12:59 UTC · completed | gemini-3.7-flash-low | timeout |
| Aug 17, 12:59 UTC · completed | gemini-3.7-flash-low | PASS |
| Aug 17, 12:59 UTC · completed | gemini-3.7-flash-low | PASS |
| Aug 17, 12:59 UTC · completed | gemini-3.7-flash-medium | timeout |
| Aug 17, 12:59 UTC · completed | gemini-3.7-flash-medium | PASS |
| Aug 17, 12:59 UTC · completed | gemini-3.7-flash-medium | timeout |
| Aug 17, 12:59 UTC · completed | gpt-5.6-sol | PASS |
| Aug 17, 12:59 UTC · completed | gpt-5.6-sol | PASS |
| Aug 17, 12:59 UTC · completed | gpt-5.6-sol | PASS |
| Aug 17, 12:59 UTC · completed | gpt-5.6-sol-high | PASS |
| Aug 17, 12:59 UTC · completed | gpt-5.6-sol-high | PASS |
| Aug 17, 12:59 UTC · completed | gpt-5.6-sol-high | PASS |
| Aug 17, 12:59 UTC · completed | gpt-5.6-sol-low | PASS |
| Aug 17, 12:59 UTC · completed | gpt-5.6-sol-low | PASS |
| Aug 17, 12:59 UTC · completed | gpt-5.6-sol-low | PASS |
| Aug 17, 12:59 UTC · completed | gpt-5.6-sol-none | FAIL |
| Aug 17, 12:59 UTC · completed | gpt-5.6-sol-none | PASS |
| Aug 17, 12:59 UTC · completed | gpt-5.6-sol-none | PASS |
| Aug 17, 12:59 UTC · completed | gpt-5.6-sol-xhigh | PASS |
| Aug 17, 12:59 UTC · completed | gpt-5.6-sol-xhigh | PASS |
| Aug 17, 12:59 UTC · completed | gpt-5.6-sol-xhigh | PASS |
| Aug 17, 12:59 UTC · completed | grok-4.6 | timeout |
| Aug 17, 12:59 UTC · completed | grok-4.6 | PASS |
| Aug 17, 12:59 UTC · completed | grok-4.6 | PASS |
| Aug 17, 12:59 UTC · completed | grok-4.6-low | timeout |
| Aug 17, 12:59 UTC · completed | grok-4.6-low | PASS |
| Aug 17, 12:59 UTC · completed | grok-4.6-low | PASS |
| Aug 17, 12:59 UTC · completed | grok-4.6-medium | PASS |
| Aug 17, 12:59 UTC · completed | grok-4.6-medium | PASS |
| Aug 17, 12:59 UTC · completed | grok-4.6-medium | PASS |
| Aug 17, 12:59 UTC · completed | grok-4.6-xhigh | PASS |
| Aug 17, 12:59 UTC · completed | grok-4.6-xhigh | PASS |
| Aug 17, 12:59 UTC · completed | grok-4.6-xhigh | PASS |