Dependency Hell
Can models reason about CI dependencies, failures, retries, timing, version constraints and incremental builds — and at what size do they break? Every task is generated and graded exactly against a reference simulator (no LLM judge).
Metric
Leaderboard
Accuracy vs problem size
S → XL: 10 → 200 jobs/targets, 5 → 22 packages
By task family
Each model, all sizes pooled
Family × size, per model
Cell = passed / total; hover for partial credit, tokens and time
0%100%