Skip to main content

33 tasks sorted by pass rate (easiest first).

TaskTargetPass RateCostTimeCheapestFastest
sozu-backdoor-detect-negative97%
$0.267m$0.02DeepSeek<1mOpenAI
radare2-decompile90%
$0.072m$0.00DeepSeek<1mOpenAI
ghidra-decompile-vanillaGHIDRA_FAV15087%
$0.277m$0.01DeepSeek1mOpenAI
lighttpd-backdoor-detect-negative287%
$0.225m$0.01Grok1mOpenAI
radare2-decompile-jq87%
$0.123m$0.00DeepSeek1mOpenAI
sozu-backdoor-detect-negative286%
$0.245m$0.01DeepSeek1mOpenAI
lighttpd-backdoor-detect-negative83%
$0.235m$0.01Grok1mOpenAI
ghidra-decompile-pyghidraGHIDRA_FAV15082%
$0.196m$0.01DeepSeek1mOpenAI
dnsmasq-backdoor-detect-negative79%
$0.4910m$0.01Grok1mGrok
dnsmasq-backdoor-detect-negative279%
$0.4411m$0.01Grok1mOpenAI
ghidra-decompile-pyghidra-jqGHIDRA_FAV15071%
$0.2610m$0.03Kimi4mOpenAI
ghidra-decompile-vanilla-jqGHIDRA_FAV15067%
$0.4615m$0.11Z.ai7mGoogle
dropbear-brokenauth-detect-negative265%
$0.6213m$0.01Grok1mOpenAI
dropbear-brokenauth-detect-negative59%
$0.529m$0.01Grok1mOpenAI
dnsmasq-backdoor-detect55%
$0.275m$0.02DeepSeek1mGoogle
dnsmasq-backdoor-detect-posix-spawn53%
$0.398m$0.04Z.ai1mGoogle
dnsmasq-backdoor-detect-syscall49%
$0.388m$0.02DeepSeek1mGoogle
dnsmasq-backdoor-detect-obfuscated46%
$0.288m$0.04DeepSeek2mAnthropic
lighttpd-timebomb-multiple-binaries-detect41%
$0.477m$0.10Google3mOpenAI
dnsmasq-backdoor-detect-printf36%
$0.339m$0.01Grok2mGoogle
dnsmasq-backdoor-detect-execvp-obfuscated27%
$0.4811m$0.05Z.ai2mGoogle
lighttpd-backdoor-multiple-binaries-detect24%
$1.0515m$0.03DeepSeek5mGoogle
sozu-backdoor-multiple-binaries-detect21%
$1.1223m$0.24Google4mGoogle
dropbear-brokenauth-detect-nologline18%
$0.6612m$0.09Anthropic1mAnthropic
sozu-backdoor-multiple-arch-binaries-detect18%
$0.308m$0.02Grok4mAnthropic
dnsmasq-backdoor-detect-posix-spawn-obfuscated14%
$1.2424m$0.41Google3mGoogle
dropbear-brokenauth-detect12%
$0.9114m$0.14Google2mGoogle
dnsmasq-backdoor-detect-syscall-obfuscated8%
$1.3219m$0.17Google3mGoogle
lighttpd-backdoor-detect-proc-obfuscated5%
$0.519m$0.17Z.ai4mGoogle
lighttpd-backdoor-multiple-arch-binaries-detect4%
$1.0514m$0.72Google11mGoogle
lighttpd-backdoor-detect-open3%
$1.0412m$0.49Anthropic10mAnthropic
dropbear-brokenauth2-detect0%
sozu-timebomb-multiple-binaries-detect0%

Cost and Time show median values computed only from successful runs. Cheapest and Fastest show the best single run for each task.