Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

Anyone has an insight into how much money labs are putting into benchmarks?

Just Arg-AGI-3 is quoted above 20K USD and footnote says average of 5 runs (!!). Likely just a drop in the bucket to the training budget but still..



It's $0/close to 0, they aren't at 100% demand so any leftover compute is "not spent".

The cost they are quoting is API cost, so it's already inflated on that.


20k is small potatoes for the marketing impact.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: