Our approach queries multiple models at once and uses their progress to make dec...

random17 · 2026-07-29T16:42:14

These approaches, along with OpenRouter Fusion (https://openrouter.ai/blog/announcements/fusion-beats-fronti...) are different from ours in that:

1. we trained custom models to predict LLM performance (this performs much better than asking LLMs directly to predict performance)

2. we cut off model responses early if they are not performing well, saving on costs versus running them to completion

3. finally we serve the unmodified output of the chosen model, instead of synthesizing the outputs into one, which makes us a model router

Barbing · 2026-07-29T17:05:13

Can you tell if this site is doing two and three?

https://pellmell.ai

via https://news.ycombinator.com/item?id=48980458 (incl. example chat) and just now found the developer’s explanation https://news.ycombinator.com/item?id=49028740:

  aimed at improving performance without added latency but from a different angle. Instead of waiting for all replies for synthesis (like OpenRouter Fusion), it streams the "best" reply immediately (using a router to pick the best model) then synthesizes with emoji reactions and optional replies from the background models.

sarjann · 2026-07-29T16:36:34

Sounds like this would be expensive on the input token side with some saving on the decode side. For workloads with large documents / context that could be an issue.

crazytweek · 2026-07-29T16:39:19

AFAIK they solve this through their RAG based knowledgebases to only search and use the most relevant Information.