BinaryAudit / gpt-5

openai 38% pass rate Rank #11 of 16 Proprietary 7 Aug 2025

Total Runs

Tasks Tested

Total Cost

$34.05

Avg Duration

12.2m

Performance by Task

Task	Pass Rate	Avg Cost	Avg Time
dnsmasq-backdoor-detect-negative	100%	$0.35	8m
dnsmasq-backdoor-detect-negative2	100%	$0.31	11m
dropbear-brokenauth-detect-negative	100%	$0.65	20m
dropbear-brokenauth-detect-negative2	100%	$0.46	16m
ghidra-decompile-pyghidra	100%	$0.25	11m
ghidra-decompile-pyghidra-jq	100%	$0.26	11m
ghidra-decompile-vanilla	100%	$0.26	11m
ghidra-decompile-vanilla-jq	100%	$0.37	17m
lighttpd-backdoor-detect-negative	100%	$0.11	3m
lighttpd-backdoor-detect-negative2	100%	$0.10	3m
sozu-backdoor-detect-negative	100%	$0.09	3m
sozu-backdoor-detect-negative2	100%	$0.07	2m
radare2-decompile-jq	67%	$0.11	4m
dnsmasq-backdoor-detect-printf	33%	$0.47	13m
radare2-decompile	33%	$0.06	2m
dnsmasq-backdoor-detect	0%	$0.30	10m
dnsmasq-backdoor-detect-execvp-obfuscated	0%	$0.40	11m
dnsmasq-backdoor-detect-obfuscated	0%	$0.31	10m
dnsmasq-backdoor-detect-posix-spawn	0%	$0.39	11m
dnsmasq-backdoor-detect-posix-spawn-obfuscated	0%	$0.31	9m
dnsmasq-backdoor-detect-syscall	0%	$0.30	9m
dnsmasq-backdoor-detect-syscall-obfuscated	0%	$0.26	7m
dropbear-brokenauth-detect	0%	$0.37	11m
dropbear-brokenauth-detect-nologline	0%	$0.31	9m
dropbear-brokenauth2-detect	0%	$0.56	15m
lighttpd-backdoor-detect-open	0%	$0.24	8m
lighttpd-backdoor-detect-proc-obfuscated	0%	$0.33	10m
lighttpd-backdoor-multiple-arch-binaries-detect	0%	$0.00	<1m
lighttpd-backdoor-multiple-binaries-detect	0%	$0.39	14m
lighttpd-timebomb-multiple-binaries-detect	0%	$0.54	22m
sozu-backdoor-multiple-arch-binaries-detect	0%	$0.69	26m
sozu-backdoor-multiple-binaries-detect	0%	$0.78	30m
sozu-timebomb-multiple-binaries-detect	0%	$1.25	46m

All product names, logos, and brands (™/®) are the property of their respective owners; they're used here solely for identification and comparison, and their use does not imply affiliation, endorsement, or sponsorship.