GPT-6 Astra is good at puzzles.
It gets 63% on ARC-AGI-3 with the standard agent harness, and 99% with a custom one. When using default harnesses, its predecessor GPT-5.6 Sol is just at 8% and Claude Opus 5 at 30%.
But is it benchmaxxing for ARC-AGI-3, or are these general puzzle-solving skills?
cozyblaze: “And… GPT-6 Astra has autonomously completed Portal! I didn’t expect this to happen so soon, but I’m glad we’ve made so much progress here.”
GPT-6 Astra works end-to-end, and without human help it beats the 3D puzzle game Portal, sharing both video and code.
The run took about 23 hours and 43 minutes. Token usage adds up to at least $570 at Astra’s list price, though cozyblaze used a $200 Codex subscription.
Przemysław “Psyho” Dębiak, who a year ago beat OpenAI’s model, as the only human to do so, now struggles to find puzzles too hard for Astra. As he writes:
It’s above an average human player and honestly the ~99% on ARC-AGI-3 might undersell it. At this point I tested Astra+codex against around 30 obscure puzzle games. […] Those were mostly puzzlescript games: some rule discovery, very tough levels, usually minimalistic level design, turn based and grid based.
Let’s see how it deals with Baba Is You.
Baba Is Astra
Previously, in our Baba Is Bench, we focused on solving the initial stages, The Intro and, for models that pass it, The Lake. We treated each level separately - easier to run, but also less natural than a typical walkthrough of learning from previous examples.
Since both GPT-6 Astra and Claude Fable 5.1 solved all levels, now there is time for a follow-up.
The world map of Baba Is You, from a human playthrough: 95 levels and 8 worlds after 15.8 hours.
We gave agents the 154 levels of the first ten worlds of Baba Is You, from The Intro to Volcanic Cavern. While we still used the text interface, agents unlocked levels, the same way as in the game.
Each agent had 6 hours, no internet, and an instruction to solve as many levels as possible within this limit. We used native harnesses - Codex for GPT-6 Astra and Claude Code for Claude Fable 5.1. They could take notes to save insights for the future.
I was surprised how far Astra gets!
While Fable 5.1’s results here do not look stunning, let’s put that in context - GPT-5.6 Sol used over 9h for The Lake alone, and was still missing one level.
Fable 5.1 solved 33 and completed just two worlds (The Lake and Solitary Island). After three hours it declared the nine levels it had open “infeasible” and stopped by itself, with half the time left. Astra solved six of those nine.
Fable's cost is billed per call by OpenRouter. Astra's is estimated from token counts and scaled to the account's usage, since Codex does not log OpenRouter generation ids.
There are some obvious caveats. It is a public game and some parts of it might have been used in training data.
We haven’t seen in agent trajectories any mentions of level names or recalled solutions. Astra’s reasoning is encrypted in the Codex logs, so we only see its commands. It solved 7 of 70 levels on the first attempt with no undo, one of them with a 79-move sequence 16 seconds after first seeing the level.
Is it its impeccable insight, or maybe it was trained on walkthroughs, or the game itself? Well, other parts of the trajectories show no signs of that, but the only way to distinguish memory from a generalizable skill is to test it on other challenges.
If you want to replicate this, see the repo and the tasks_campaign/main-00-09 Harbor task.
Not only Baba
The same pattern shows up on MazeBench, a 3D labyrinth of Sokoban-style puzzles: Astra scored 14% without Python after a run of over 60 hours, against 2% for Claude Fable 5.1.
MazeBench, July 2026: “a 3D open world environment that evaluates long term planning with visual spatial reasoning. Today’s best agents cannot progress beyond the initial levels.”
MazeBench vs GPT-6 Astra, September 2026: “Astra spent 60+ hours in this 3D open world spatial reasoning eval. Final score: 14%.”
But it is not only games and benchmarks: Claude Fable 5.1 solves the Cyphral Distich. I believe GPT-6 Astra would be able to do so as well.
What’s next
The original ARC-AGI-3 got saturated so surprisingly fast that the next iteration will be much harder. As Przemysław “Psyho” Dębiak speculates on X:
It seems that the original idea for ARC-AGI-4 was completely scrapped and they’re going directly to ARC-AGI-5
The progress also has a major implication that all puzzles (including security locks) that can be solved by experts are easily solved by frontier AI models.
Or maybe you know some puzzle games and logical challenges that are within reach of bright minds, yet beyond the scope of Astra and Fable?
