I had Ox Alpha working on coding tasks for a couple days non-stop, via OpenRouter and OpenCode Zen. Crush harness. It was able to complete tasks at a level that I'd put between Sonnet and Opus. It makes few mistakes, but is not that smart.
The main issue for me, is that it degraded into a doom loop several times. One of them was running the same bash command about a thousand times. The last model I've used that had this problem was Mimo 2.5, which is quite dated at this point. As a result of this, you cannot leave it unattended / not usable for agents.
I couldn't get past all the network errors on OpenCode. Seemed smart enough, and was useful when I was low on usage on Claude, but beyond that, really hard for me to say whether it was Good or Bad.
I asked it to create a design system in Paper, and it actually did a fairly decent job when it wasn't getting network errors. I'd say it's much, much closer to good than bad.
Doom loops are as much of a model problem as it is deficiency of the harness. I have not seen any other open source harness that deals with them except the one I started because of this obvious gap.
See my other comment with examples where 0x Alpha is working non-stop on various projects with zero problems.
Pi.dev has an extension that does it. None of the models I use seem to have that failure mode so I haven't bothered throwing it in.
The failure mode I run into commonly is agents just stop sometimes. Even sending a "." Or something they start back up, but I haven't worked out exactly how to fix that generally in harness, bit unclear how to tell if they're done or just derped to a stop.
I'm not 100% sure what I'd do to detect this for shells specifically, maybe prompt the user, but my toy harness scolds refuses to re-read files because the previous iteration of GLM was room-looping on that. Clearly something that they need to work on.
It feels like all these models are missing a long horizon orchestrator of some kind. Humans don't think in a single thread, they have thoughts that appear, pause and disappear.
The reason humans don't go into a doom loop is because of this ability of other thoughts to interject and say, no this is stupid.
I usually see doom loops when working with quants. Likely theyre trying to maximize the viability of a efficient model quant that can bw upgraded. Like cutting coke to get crack, quantiry over quality.
I know that AI is moving fast, but Mimo 2.5 literally came out four months ago. I literally had to double-check after I read this because it felt like just yesterday.
Also, I'm not sure what the industry standard is right now, but my (self-built) harness automatically exits with an error code whenever it detects similar tool calls being sent or when semantic repetition in the reasoning traces reaches a certain threshold. It's pretty easy to set that kind of thing up.
The main issue for me, is that it degraded into a doom loop several times. One of them was running the same bash command about a thousand times. The last model I've used that had this problem was Mimo 2.5, which is quite dated at this point. As a result of this, you cannot leave it unattended / not usable for agents.