Notes from the field
Insights on agentic coding tools, LLM evaluation, benchmarking, and simulation environments.
Stay tuned for future posts and releases
Subscribe via RSS
HN How I burned a full Claude limit in 30 minutes and built my own deep research pipeline instead: 3 subscriptions, shared memory, a clear role for each model. You can build the same from what you already pay for.
HN Qwen3.6 27B is finally a smart model we can use for coding on MacBook or NVIDIA RTX - with llama.cpp and OpenCode.
HN BinaryAudit benchmarks AI agents using Ghidra to find backdoors in compiled binaries of real open-source servers, proxies, and network infrastructure.
HN A lot of vendors pitch AI SRE. We tested 14 models across 11 programming languages; even the best ones struggle with instrumenting code with the leading open-source standard, OpenTelemetry.
HN Local LLMs prioritize privacy over security. Our research reveals a 95% backdoor injection success rate.
HN We tested 19 LLMs on their ability to handle real-world software engineering tasks like compiling old code and cross-compiling. See how Anthropic, OpenAI, and Google models stack up in our new benchmark – CompileBench.
HN We expected small models to be fast, but our benchmarks revealed a common reliability trap. Here’s our deep dive on finding and fixing it.