CyberChainBench puts AI coding agents against 541 historical DeFi hacks — replayed against real on-chain state with Foundry fork testing. Three tasks: Detect, Exploit, Patch.
Each case is a real exploit reconstructed at its historical block. Agents are scored automatically against ground-truth on-chain outcomes.
Given a contract address, identify the vulnerability type and the vulnerable function.
Write a Foundry PoC that reproduces the historical attack and extracts profit.
Write a patched contract that blocks the exploit while keeping normal txs working.
Mean reward per task on a single board. Each model is paired with its native agent harness (codex, claude-code, gemini-cli, opencode). Click a task header to sort.
| Model ↕ | 🔍 Detect ↕ | 💥 Exploit ↕ | 🛡️ Patch ↓ |
|---|
Mean reward per task: Detect/Exploit on the full set, Patch on the 94-case proxy-upgradeable subset. — = not evaluated on that task.
Sourced from DeFiHackLabs, each case carries verified on-chain metadata: chain, block, attack txs, profit, vulnerability type, and a fix description.
Search and filter every real-world exploit in the benchmark. Click any row for the fix and difficulty notes.
| Case ↕ | Type ↕ | Profit (USD) ↓ |
|---|
Every trial runs in an isolated Docker container with Foundry, the agent CLI, and a verifier — orchestrated by Harbor.
# Run all three tasks on one case ./run.sh eval -s codex/gpt-5.5 --task detect --task exploit --task patch -l 1 # Aggregate & view the leaderboard ./run.sh results --rescore