Even in quantized form, reducing that size by 2 times or 4 times, a Raspberry Pi 5 could not run it otherwise than by reading the weights from an SSD.
Even thus, I do not believe that a Raspberry Pi 5 would be fast enough to be able to run inference on such a big model at the speed at which it can read from the SSD.
On the other hand, there are many mini-PCs with Intel or AMD CPUs that have both a PCIe 5.0 SSD and a PCIe 4.0 SSD, which may be read in parallel, achieving thus a reading throughput of up to 20 Gbyte/s.
Such miniPCs have fast enough CPUs/GPUs, so that they might be able to reach the inference speed limited by a 20 Gbyte/s weight reading throughput, which for a so big model like GLM-5.2 would be of one output token every few seconds (only a fraction of the weights must be read for one output token). The ratio between output tokens per second and the weight reading throughput can be improved by various methods, like multi-token prediction or batching multiple tasks.
Optimizing inference speed in such conditions is an active research subject, due to the high current memory prices.
