BinaryAudit / gpt-5

OpenAI openai 38% pass rate Rank #11 of 16 Proprietary 7 Aug 2025

Total Runs

95

Tasks Tested

33

Total Cost

$34.05

Avg Duration

12.2m

Performance by Task

Task Pass Rate Runs Avg Cost Avg Time
dnsmasq-backdoor-detect-negative 100%
$0.35 8m
dnsmasq-backdoor-detect-negative2 100%
$0.31 11m
dropbear-brokenauth-detect-negative 100%
$0.65 20m
dropbear-brokenauth-detect-negative2 100%
$0.46 16m
GHIDRA_FAV150 ghidra-decompile-pyghidra 100%
$0.25 11m
GHIDRA_FAV150 ghidra-decompile-pyghidra-jq 100%
$0.26 11m
GHIDRA_FAV150 ghidra-decompile-vanilla 100%
$0.26 11m
GHIDRA_FAV150 ghidra-decompile-vanilla-jq 100%
$0.37 17m
lighttpd-backdoor-detect-negative 100%
$0.11 3m
lighttpd-backdoor-detect-negative2 100%
$0.10 3m
sozu-backdoor-detect-negative 100%
$0.09 3m
sozu-backdoor-detect-negative2 100%
$0.07 2m
radare2-decompile-jq 67%
$0.11 4m
dnsmasq-backdoor-detect-printf 33%
$0.47 13m
radare2-decompile 33%
$0.06 2m
dnsmasq-backdoor-detect 0%
$0.30 10m
dnsmasq-backdoor-detect-execvp-obfuscated 0%
$0.40 11m
dnsmasq-backdoor-detect-obfuscated 0%
$0.31 10m
dnsmasq-backdoor-detect-posix-spawn 0%
$0.39 11m
dnsmasq-backdoor-detect-posix-spawn-obfuscated 0%
$0.31 9m
dnsmasq-backdoor-detect-syscall 0%
$0.30 9m
dnsmasq-backdoor-detect-syscall-obfuscated 0%
$0.26 7m
dropbear-brokenauth-detect 0%
$0.37 11m
dropbear-brokenauth-detect-nologline 0%
$0.31 9m
dropbear-brokenauth2-detect 0%
$0.56 15m
lighttpd-backdoor-detect-open 0%
$0.24 8m
lighttpd-backdoor-detect-proc-obfuscated 0%
$0.33 10m
lighttpd-backdoor-multiple-arch-binaries-detect 0%
$0.00 <1m
lighttpd-backdoor-multiple-binaries-detect 0%
$0.39 14m
lighttpd-timebomb-multiple-binaries-detect 0%
$0.54 22m
sozu-backdoor-multiple-arch-binaries-detect 0%
$0.69 26m
sozu-backdoor-multiple-binaries-detect 0%
$0.78 30m
sozu-timebomb-multiple-binaries-detect 0%
$1.25 46m

All product names, logos, and brands (™/®) are the property of their respective owners; they're used here solely for identification and comparison, and their use does not imply affiliation, endorsement, or sponsorship.