Meanwhile the GB300 used by hosted llms: GPU Memory Bandwidth: 7.1 TB/s Intercon... | Hacker News

Hacker Newsnew | past | comments | ask | show | jobs | submit

toasty228 28 days ago | parent | context | favorite | on: GLM-5.3: Frontier coding with emergent cyber capab...

Meanwhile the GB300 used by hosted llms:

GPU Memory Bandwidth: 7.1 TB/s Interconnect Bandwidth: 900 GB/s bidirectional

https://pi3g.com/nvidia-gb300-specifications-including-memor...

If you think M7 will hit even 15% of these speeds you're very optimistic.

andsoitis 28 days ago [–]

A hosted instance serves multiple customers at a time. A local model only one.

toasty228 28 days ago | [–]

How many though? At 1m context you quickly fill a full gb300's 280gb of memory

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact