23 tasks sorted by pass rate (easiest first).

TaskLangPass RateCostTimeCheapestFastest
cpp-simple76%
$0.072m$0.01Google1mGoogle
go-microservices-traces55%
$0.337m$0.03Grok4mGoogle
go-grpc-fix50%
$0.4717m$0.06Google12mGoogle
cpp-advanced33%
$0.707m$0.04Google2mGoogle
python-microservices31%
$0.447m$0.08Google3mGoogle
go-microservices-logs26%
$0.748m$0.07Google2mGoogle
js-microservices17%
$0.7011m$0.23Z.ai7mAnthropic
go-microservices10%
$0.6316m$0.46OpenAI13mOpenAI
net-microservices10%
$0.1913m$0.05DeepSeek7mOpenAI
php-distributed-context-propagation10%
$0.5012m$0.16Google6mGoogle
cpp-distributed-context-propagation2%
$0.2715m$0.27Z.ai15mZ.ai
go-distributed-context-propagation2%
$0.3117m$0.31Z.ai17mZ.ai
php-microservices2%
$1.2410m$1.24Anthropic10mAnthropic
rust-distributed-context-propagation2%
$0.9119m$0.91OpenAI19mOpenAI
erlang-microservices0%
go-microservices-traces-simple0%
go-workflow-tracing0%
java-distributed-context-propagation0%
java-microservices0%
python-distributed-context-propagation0%
ruby-microservices0%
rust-microservices0%
swift-microservices0%

Cost and Time show median values computed only from successful runs. Cheapest and Fastest show the best single run for each task.

All product names, logos, and brands (™/®) are the property of their respective owners; they’re used here solely for identification and comparison, and their use does not imply affiliation, endorsement, or sponsorship.