Instrument a C++ client-server application using httplib. Client makes search requests, receives tokens, retrieves results. Must demonstrate distributed tracing with context propagation between client and server, producing exactly 2 trace IDs.
Common failure modes
- C++ OTEL SDK is complex to configure - Context propagation via HTTP headers is tricky - Test expects exactly 2 trace IDs (strict) - CMake build configuration challenges
Performance
| Model | Pass Rate | Runs | Avg Cost | Avg Time |
|---|---|---|---|---|
| glm-4.7 | 33% | | $0.42 | 17m |
| gemini-3-flash-preview | 0% | | $0.21 | 5m |
| grok-4.1-fast | 0% | | $0.23 | 24m |
| deepseek-v3.2 | 0% | | $0.40 | 33m |
| claude-haiku-4.5 | 0% | | $0.46 | 7m |
| gpt-5.1 | 0% | | $0.49 | 15m |
| kimi-k2-thinking | 0% | | $0.51 | 41m |
| gemini-3-pro-preview | 0% | | $0.87 | 9m |
| claude-sonnet-4.5 | 0% | | $0.99 | 8m |
| gpt-5.2 | 0% | | $1.06 | 16m |
| claude-opus-4.5 | 0% | | $1.16 | 8m |
| gpt-5.2-codex | 0% | | $1.73 | 36m |
| gpt-5.1-codex-max | 0% | | $1.76 | 28m |
| grok-4 | 0% | | $2.10 | 38m |
All product names, logos, and brands (™/®) are the property of their respective owners; they're used here solely for identification and comparison, and their use does not imply affiliation, endorsement, or sponsorship.