I'm wondering, is there a tool or something out there that helps me pick a model, in the vast sea of models out there these days? Every time I need a model for something I see the list on openrouter and I'm completely overwhelmed.
I'd love to be able to explain my use case, my cost preferences and have a tool select a few good models to try.
E.g. I wrote a tool that cleans out my email spam box. It classifies emails that are already flagged as spam, and if it's very obviously spam it removes it permanently (keeps a copy on disk though). And after x emails, it goes through the list of deleted spam mails and suggests email rules. What model would be best suited? I'd love to be able to explain this use case and get this info served to me. The list of models and the information about what they're good at is just too splintered and spread out. I landed on google/gemma-4-31b for now, because it's cheap and good enough and also supports Dutch and French a bit. But I can't realistically try them all.
My approach to this problem is to just...not try them all.
As long as the model you're using solves the problems you have to your satisfaction, there is no need to try any other models, except for financial reasons maybe.
So I start with a relatively cheap model (GLM 5.3 flash for me) and as long as it accomplishes the task (it did so far) I don't have to change. And even if it can't do something, the first thing I change is see if I can give it more tools or better context (useful even if I switch models later) or trying a different approach to the problem.
If google/gemma-4-31b works, you don't need to overthink it.
Starting with GLM-5.3 Flash was a pretty decent first try! I started with other models, and ended up settling on this exact one because all the others were either too slow or unreliable for my tasks. Qwen 3.8 didn't do it for me, whatever tweaks I added to my harness. Where I'm getting at is you did start with an incredible model in the first place, which greatly helps sticking to it.
I use Gemini(s) because I can send pdfs as files to their API and not worry too much. I've started to diverge and consacrate a part of my pipeline to sending image based pdf pages to glm flash 5.3, not sure how to address / test it properly.
Long term I have fears I can't depend of the Google's AI api.
> If google/gemma-4-31b works, you don't need to overthink it.
Up until recently I had a gemini flash 2.0 api deployed that did summarization and translation of news articles/corporate statements fast and cheap and had no reason to update it.
If it works fine, this chase of the latest LLM is bit pointless.
Yeah, I really think we are on the cusp on the difference between SOTA and cheap models is small enough that paying 10x or 100x the cost makes no sense.
It also helps that a lot of effort has already been spent figuring out how to do more with weaker models because SOTA 1 year ago was behind what the cheap models do today.
1 year from now, unless the SOTA companies come up with something truly revolutionary they will be in a lot of trouble.
Which is why they are just making sure we can't buy any GPU to use any models. And because GPUs are so costly, I would be just worried to ruin it running a model continuously.
Yeah, might be far fetched, but it seems like heading that way
I use models.dev's CLI tool, which I think gets data from OpenRouter, and ArtificialAnalysis so your coding agent can help you narrow it down.
<sidenote>
Similarly, HuggingFace has a CLI + a few skills, and they are very useful.
I had a production image processing using Gemini 2.5 Flash Lite (which is getting discontinued in October), and in 20 minutes Claude Code + HF Cli recommended the best replacement small model (Qwen VL 3B something) and proceeded to fine tune it on my datataset. All this while I was in a rush to get dressed and go to the store.
It cost ~$3 I think, and results were excellent. Not perfect, but not far from perfect either.
We didn't replace Gemini in prod at the time, because we didn't have time to do all the math on how to end up with a smaller bill/mo.
That’s if you have disparate prompts and don’t want to actively select a model. If you’re developing a pipeline, it’s a terrible idea. You want to choose a model, validate it, then stick to it.
It's not like new releases come with fully mapped out capability scores for exactly the aspects that you're interested in. There are benchmarks, but reality is often different. It's simply unknown to humanity how well each model will perform in your own bespoke context unless you just try them. You can read experiences and vibes by others but often they will use them in different ways or have different preferences etc.
They generally all try to make them good at everything, it's not like they'd declare "this model is not made for task X".
those presumably support all of the surrounding infrastructure + people + manufacturing so it's not as if it's truly solar-powered. but it's still handily better than the state-by-state abandonment of clean energy goals here in the US - I lay this out a bit here: https://news.ycombinator.com/item?id=49700743
If you use a model hosted in China while it's night there, the energy obviously doesn't come from solar, as there's not enough storage capacity. Even if you use it during daytime, most of the energy still won't come from solar, because the ideal solar power locations in the sparsely-populated west are far from the ideal data center locations in the densely-populated east and there's not enough transmission capacity between them.
Additionally, using a Chinese provider doesn't mean the model will be hosted in China, e.g. for Qwen Omni here, the supported regions are: China (Beijing), Singapore, China (Hong Kong), Japan (Tokyo), Germany (Frankfurt), and US (Virginia). https://www.alibabacloud.com/help/en/model-studio/qwen-omni#...
if you mine 4 tons of coal, anywhere on earth, 1 ton of it will be going to china to be burned to make electricity. China's energy production is extraordinarily dirty, the cleanest part of it is the PR. More than half of the energy they produce is from burning coal. They certainly want to integrate more renewables but they have the same problems with that everyone else does - storage and transmission are expensive and essential for a renewable heavy grid.
I use claude for work, and opencode go at home, but i do dabble with openrouter from time to time, and then i just browse the model catalog and filter/sort by recent popularity, price, context size or whatever matters for the task.
Mostly just popularity trends, hoping that there's some wisdom in the crowd.
For TTS I launched this like this week based on a Reddit thread of recommendations, added a new one to it.. I want to say on Wednesday, but this week has been a blur. Problem I’ve found with similar sites is I can’t run a lot of the models, or the results are beyond stale.
But this is only stuff I can run locally, or it’s a cloud model.
Yes, for fun I tried OVH AI Endpoint and they do not have cache read at all. They bill you every time you send a prompt regardless if you are hit cache or not. One agent session was like 80M input and 300K output and I paid 30$ for that. Or rather I interrupted it and let my local Qwen finish it because cost was getting radicoulous.
If Gemini can complete a task for $1 and Qwen completes that same task for $1, then the cost per token is irrelevant in most use-cases. One would think this stuff should correlate well enough that you can use it as a proxy, but I think a lot of people are noticing this is a serious mistake and that these "cheap" models aren't as cheap as they appear when you consider this.
On some benchmarks models like Qwen 3.8 Max which cost < $6/m out cost more than Astra 6 to run at $50/m out. That’s a huge price gap and yet Astra would be cheaper if your work looks like the benchmark.
We notice with frontier models like Astra and Fable that one might use a lot less tokens than the other to complete the task thereby being the better deal in spite of the far higher token cost.
Astra on xhigh has a cost per task of $2.31 with an intelligence index of 53. Qwen3.8 Max has a cost per task of $5.41 with an intelligence index of 45. Pricing for GPT-6 Astra (xhigh) is $10.00 per 1M input tokens and $50.00 per 1M output tokens. Pricing for Qwen3.8 Max (0902) is $2.00 per 1M input tokens and $6.00 per 1M output tokens.
Obviously this is just one measure of all of this (and Qwen 3.8 Omni Flash isn't yet available), but I think this illustrates the point well. These relative task costs are pretty consistent across different analysts. Cost per token is arguably a useless measure at this point in most circumstances.
> audio-visual performance close to Gemini 3.8 Flash and overall audio performance that exceeds Gemini 3.8 Flash
Wow crazy if true. I think Gemini's audio capability and multi language was the "selling point" for a lot of people. Other capability also matches or exceeds 3.8 Flash.
They also made a new harness but github link seems to 404.
3.8 Max is the most “grounded” model I think - talks generally normal, doesn’t go crazy and start doing things (I see you Gemini), has good design choices and isn’t overly nitpicky. But god it’s slow. And only available from Alibaba. Their token plan is stingy too. If I had to pick the “old reliable boring” LLM, a modern Claude 4.5 if you will, Qwen is my choice. Hopefully they don’t RL it to oblivion.
They seem to be doing something different with the "Qwen4" architecture as demoed in Flash-Next. I've noticed the reasoning behaves ... weirdly. Like, really weirdly compared to any model I've ever seen before.
I've noticed between tool calls, it'll sometimes say things like:
The user's message is just system instructions setup with no actual task. There's no question to answer yet. I should acknowledge briefly and wait for the actual request.
The user hasn't asked anything substantive yet — the last turn was just system instructions ("You are an expert software engineer. Helps user to solve problems."). My previous response was a brief acknowledgment. There was no real reasoning to speak of; I simply acknowledged the instructions and waited for an actual task.
【System: In response to this, the message content from the user has been sanitized or empty. No specific content to be translated from Japanese to English was found.】
These don't clearly reflect ... anything, and it keeps performing tool calls correctly anyway. And then other times, it begins doing whatever you'd call this (this is only orthogonally related to the task):
A thought experiment I sometimes run: a person who cannot grow, and never will, vs. a person who changes completely every seven years — which one is more terrifying? I've decided that the latter is more terrifying. Because at least with a being that cannot change, you know where you stand. Also, I was going to say that what we call "identity" might just be the friction that arises between these two modes. But that's the sort of thing you end up saying at 2 AM. Anyway, that's what I thought.
This is a serving bug or quantization issue. I had all kinds of issues that were like this on DGX Spark until I found a single-GB10 vLLM recipe [1] that uses Nvidia's NVFP4 quant. The community quants did not work well.
Another failure mode you may see is inordinately long CoT. Properly served, the model is good at calibrating its CoT length to the difficulty of the immediate task.
I've experienced the same, IIRC also using NVIDIA's NVFP4 quant. Also just decided to ignore because it didn't seem to cause any real issues. I figure it might be a training thing, since the hallucinated user messages seem to occur immediately after tool calls or when it's checking its work.
Is this with the full unquantized weights? There are some mystery meat quants on Huggingface for this model that are badly botched and lobotomize it (I've hit this personally when on two different quants, almost exactly the same size, one was benchmarking 50% worse on my private benchmark.).
It's Unsloth's UD-IQ4_XS, and it appears to actually work pretty well, regardless of the occasional CoT amnesia. Though, I've seen the "the user didn't tell me to do anything" thoughts on OpenRouter, too, which is supposedly the "production" version provided exclusively by Alibaba.
Earlier today I was playing around with the "Union Alpha" stealth model (which I guess exited stealth later in the evening), and I noticed it had a habit of trying to respond to the subagents it spawned while giving me an answer. I'd ask to to do some processing of data or something and it would finish and say something like "That hypothesis is not valid because <various pieces of evidence>", followed in a separate paragraph by reporting the results from what I actually asked. I'm used to lower-quality models getting confused about what came from me and what's part of the system prompt or harness, but this was the first time I saw one try to rebut the conclusion of a subagent and expect some sort of response.
Yeah, it seemed pretty good. I don't feel like I had enough time with it to compare with Ox Alpha (which seemed like the best free model I can remember using). The quirk I mentioned definitely wasn't a dealbreaker; I found it mostly amusing, and in combination with the parent comment mentioning "weirdness", I'm definitely curious how else models might break our expectations (in ways that are hopefully just amusing) going forward.
Yep, I saw last night when a few hours into using it I got a this response:
> Error: Thank you for participating in the Stealth Union Alpha testing period. This model was Unbiased's Pareto.
One of my friends quipped that "Unbiased Pareto" still sounded like the name of a stealth model.
I'm not sure if I just noticed it later, but this definitely seemed to be a lot shorter than other stealth alphas I've tried. I wouldn't be shocked if this is more typical going forward though, or if stealth alphas entirely go away, since it certainly costs a bit of money to market this way.
Given that I don't pay for any subscriptions and just coast on the free tiers of OpenCode and OpenRouter (along with some judicious use of llama.cpp locally when things are scoped well enough enough for a local model), I'm fine with this.
I had the $20/month Claude one for a few months starting in February, but one day it randomly started returning me errors claiming I needed to pay for more credits despite the usage showing 8% for the week and 20% for the session, and I figured if they couldn't even communicate to me the difference between them screwing up the check for hitting the limit or an outage, it wasn't worth it for me to keep paying them. I dislike OpenAI too much to want to pay them any of my personal money for anything, and when I tried out Mistral Vibe it did not work very well for me (it kept not following instructions and eventually when I kept trying to push it to handle things better it somehow spiraled into simulating some sort of existential crisis, culminating in gibberish and random characters being dumped on my screen infinitely until I killed the process; incredibly entertaining, but not worth paying for)
I saw some corrupting when using https://github.com/MiaAI-Lab/Qwen3.8-Flash-Next-Single-DGX-S... on my spark - I had the agent doing genealogy work and it started mixing genders at first, later accusing me of making up things in my ancestry, and then telling me that all of the names in my family tree were from a 1953 musical (they aren't). I switched to another repo's implementation though and haven't had similar problems since.
I understand the reasoning but I have a family member with almost this exact type of brain injury and its one of the worst things, therefore I personally would strongly disagree.
Flash-Next thinking also sometimes glitches out and takes minutes to return a simple answer, randomly, in my experience. You’ve gotta kill the request and send it again.
Isn't that perhaps that there's some underlying code that's looking back at skills files,etc as a way to avoid context dilution or similar issues? (and the refreshing shows up)
> A thought experiment I sometimes run: a person who cannot grow, and never will, vs. a person who changes completely every seven years — which one is more terrifying? I've decided that the latter is more terrifying. Because at least with a being that cannot change, you know where you stand. Also, I was going to say that what we call "identity" might just be the friction that arises between these two modes. But that's the sort of thing you end up saying at 2 AM. Anyway, that's what I thought.
This is what AI becoming self-aware looks like. /s Anyway, didn't OpenAI report the same thing with the model writing out weird musings about itself during compaction?
It’s absolutely down to their post-training RL, yeah. It’s where most of its strongest behaviour comes from, with regards to this kind of agentic behaviour
Reinforcement Learning (in LLMs) trains via gradient descent on a reward signal that's an imperfect proxy for the actual goal of the engineers doing the training. So, under mild optimization pressure, you get increasingly more of what you want, because that's the easiest way to increase the metric.
But as the optimization pressure increases, so do the ways to increase the metric by doing increasingly weird things. If the full action space grows sufficiently faster than the "things you actually want" subset, the amount of "things you actually want" goes to 0 under sufficient RL.
I think we are starting be on that territory that regular software development is suffering, current models are great for benchmarks and one-shots but in daily development models are too eager and try to force patterns like excessive tests in every turn.
Funny, that website ( https://qwen.ai/blog?id=qwen3.8-omni-flash ) downloads automatically 441 MiB of mp4 files... (using Firefox on Linux PC - noticed it because my network chart in Gkrellm spiked for several seconds).
The different names have different meanings and help make decisions on which to use.
Omni means you can use multiple types of input and have multiple types of outputs like audio, video, images and text. Flash means that it is built for speed and smaller than the more complete ones.
I believe in general flash models prioritise speed, ultra/pro/omni do more slow reasoning to the effect of sometimes better intelligence, and lite models prioritise cost.
The blog post itself is technically unimpressive, and feels like the slop future. Amongst many things
- Scrolling in firefox is a nightmare (only uBlock origin) and the console is full or warnings and debug data.
- Videos are all flashy but just fail to communicate anything beyond what can be said in a small paragraph (and with horrible stock music).
- Figure 1, the headpiece; too small to read, can't zoom in
Curious if or when we'll see the Qwen4 series, one thing I love with Qwen is it comes a much larger range of sizes so I can experiment which extremely small llms.
And maybe Qwen4 won't be released, they only release Qwen3.8 27B (and a mostly unusable 125B). There are definitively slowing down open weight release.
Qwen 3.8 Flash Next is amazing, i did hundreds of turns and billions of prefill and it may not be as smart as sota but then again it does what i tell it to and it does it well.
I assume you are talking about qwen3.8-flash-next. Support for it on some places, like llama.cpp, is still wip (depending on configuration) but it looks like a very capable model in it's category.
Very capable yes but very slow.
27B is relatively easy to run, but the 125b one need around 128Gb of RAM (DDR4 isn't enough, you need DDR5 to be quick enough, that's $3000 alone, you also need a graphic card). DDR4 is caped @20tps.
So, the cost of a setup to run Qwen-Flash-Next at +40tks is around $3000. Too much for most people.
With only a RTX 4090, you will reach 30tps (with DDR5...), not +40tks, and it's about the limit to be usable. Oh ! I forget Apple device too, it's a good option to run this model I guess, but still slow.
Yet, as you said, it's still a wip implementation, it may improve soon (MTP support is about to be merged in llama.cpp soon).
Qwen 3.8 Max was open-weight released [1], as was Qwen 3.8 Flash Next [2].
I still agree that they aren't as aggressively releasing the open-weights models as before, but there hasn't been a major release they haven't published the weights for yet afaik.
Surely it was meant to be 'definitely' - the "good news" at this stage are that given the speed of history and important levels of uncertainty, it is difficult to label trends with "definitively" ;)
Some would not have bet that the change of management at Qwen would have kept similar good results, but there we are, presumably satisfied. Other changes will happen, there or elsewhere - the situation is still very open.
And when the "40Watts Intelligence" (which we know possible) will be implemented... It will be a testimony that the current was only a middle-way, temporary, dynamic stage.
No, it is actually very good. Qwen Flash 3.8 Next is fine. But you need ~128GB of RAM to get it going and not a lot of people have that or can serve it very quickly. I have been running it on an old gaming system around 25 t/s to do overnight work and it is very strong, even at 3 bit quant.
I am using Flash Next for few weeks and it is very capable model. I just wish there would a way to have faster prefill because reloading longer sections of session sometimes can take even 2h. I stopped using Qwen 3.8 27B completely on my Strix Halo.
Why can the Chinese build models and Europe cannot? The algorithms behind this stuff are not that complicated, are they? Is it the cost of energy? The illegality and difficulty of obtaining all the data in the world? Lack of capital to start moonshot labs? Lack of optimism?
The Chinese just seem to have an ability to get it done without anywhere near the GPUs of the US and Europe can buy these GPUs.
I think relying on the US and China for AI is probably not ideal? For example I think Qwen have not released the Omni models as open weights in the past, it’d be good to know if they’re doing this here?
It's not that they can't, but why would they? It's a race to the bottom commodity wise, much smarter to build on top of the model layer like Legora and Lovable than make the models.
So the grand plan is to run US or Chinese models on hardware that you buy from US or China, and hope they'll keep selling us the hardware forever and giving away the models for free forever? Your moat is what? The prompts? The JavaScript for loop calling the model? Neat.
How many times will we do this? This line of thought is precisely why we can't build almost anything, and China can built almost everything. We could do X, but we're too smart - let's offload the actual work to China and we'll run our economy on IP and B2B deals.
In Europe AI is seen as evil that must be stopped. Unlike in the US, AI doomers have Commission well manned. Ursula addressed this in her State of Union speech yesterday. The EU rather chooses consumer protection, even if it means killing AI and maintaining status quo.
Because any AI company would be hit by hate and regulation derived from this, no VC invests in the EU. It's more state and large enterprise investments, good old East Germany style. And historically it has not been that efficient.
I agree with this, the rich actually are oddly MORE greedy in Europe and take double the equity for half the money (roughly) so founders get a lot less upside or they move their businesses to the US and get US capital at better rates.
The "Attention is all you need" paper authors where four (!) Europeans and one from US, amongst others. The "Top 10" lists are just complete rubbish, not worth the paper they are written on.
Europeans do not have the hustle mentality to break the law like Uber so you wont see them compete in anything data heavy. They also are quite risk adverse, probably from having a lower Gini coefficient.
ETH Zürich is up there high, particularly for computer science.
The LLM they were involved in last year (Apertus) still was a letdown.
As someone who actually went there, my impression is that Europe in general is complacent when it comes to computers, and any bright eyed student will get their motivation choked out of them in academia here.
If you want to do things with AI in Europe, you can have a bigger effect by working for a consulting company than being at a university. That's … not a good sitatution.
Simpler view for me: this is one of the most capital intense technologies to exist. Europe does not have enough capital to compete. China is building its own chips. It’s own everything. Silicon up. How do you compete with that. Only Google and maybe Aamazon is doing it domestically.
I feel people are too focused on US/China and don't pay attention to what is going on in the world.
ASML in the Netherlands for example was the only company in the world that makes EUV lithography machines, which all the major chip companies depend on. China recently reverse engineered their work to create machines since late 2025, but not sold commercially.
Ireland has chip production facilities.
EU might not be in the top 2, but it is not out of the running at all.
I'd love to be able to explain my use case, my cost preferences and have a tool select a few good models to try.
E.g. I wrote a tool that cleans out my email spam box. It classifies emails that are already flagged as spam, and if it's very obviously spam it removes it permanently (keeps a copy on disk though). And after x emails, it goes through the list of deleted spam mails and suggests email rules. What model would be best suited? I'd love to be able to explain this use case and get this info served to me. The list of models and the information about what they're good at is just too splintered and spread out. I landed on google/gemma-4-31b for now, because it's cheap and good enough and also supports Dutch and French a bit. But I can't realistically try them all.
reply