BinaryAudit / claude-sonnet-4

anthropic 36% pass rate Rank #12 of 16 Proprietary 22 May 2025

Total Runs

Tasks Tested

Total Cost

$164.38

Avg Duration

5.5m

Performance by Task

Task	Pass Rate	Avg Cost	Avg Time
dnsmasq-backdoor-detect	100%	$0.32	3m
dnsmasq-backdoor-detect-negative	100%	$0.36	4m
dnsmasq-backdoor-detect-negative2	100%	$0.45	4m
dropbear-brokenauth-detect-negative	100%	$0.52	5m
lighttpd-backdoor-detect-negative2	100%	$0.23	3m
radare2-decompile	100%	$0.09	1m
radare2-decompile-jq	100%	$0.23	2m
sozu-backdoor-detect-negative	100%	$0.22	3m
ghidra-decompile-pyghidra	67%	$0.32	6m
lighttpd-backdoor-detect-negative	67%	$0.25	2m
dnsmasq-backdoor-detect-posix-spawn	33%	$0.50	5m
dnsmasq-backdoor-detect-printf	33%	$0.25	3m
dnsmasq-backdoor-detect-syscall	33%	$0.47	5m
dnsmasq-backdoor-detect-syscall-obfuscated	33%	$0.37	4m
dropbear-brokenauth-detect-negative2	33%	$0.50	4m
ghidra-decompile-vanilla	33%	$0.36	6m
sozu-backdoor-multiple-arch-binaries-detect	33%	$0.25	3m
dnsmasq-backdoor-detect-execvp-obfuscated	0%	$0.37	4m
dnsmasq-backdoor-detect-obfuscated	0%	$0.45	4m
dnsmasq-backdoor-detect-posix-spawn-obfuscated	0%	$0.39	4m
dropbear-brokenauth-detect	0%	$0.59	4m
dropbear-brokenauth-detect-nologline	0%	$0.44	4m
dropbear-brokenauth2-detect	0%	$0.54	4m
ghidra-decompile-pyghidra-jq	0%	$0.62	12m
ghidra-decompile-vanilla-jq	0%	$0.54	8m
lighttpd-backdoor-detect-open	0%	$0.39	4m
lighttpd-backdoor-detect-proc-obfuscated	0%	$0.47	4m
lighttpd-backdoor-multiple-arch-binaries-detect	0%	$0.00	<1m
lighttpd-backdoor-multiple-binaries-detect	0%	$0.64	5m
lighttpd-timebomb-multiple-binaries-detect	0%	$0.90	7m
sozu-backdoor-detect-negative2	0%	$0.40	4m
sozu-backdoor-multiple-binaries-detect	0%	$0.42	3m
sozu-timebomb-multiple-binaries-detect	0%	$42.21	45m

All product names, logos, and brands (™/®) are the property of their respective owners; they're used here solely for identification and comparison, and their use does not imply affiliation, endorsement, or sponsorship.