A chainsaw is a physical machine. Physical machines are technically non-deterministic if you look closely. They have Variance. The discipline to manage variance is called Tolerance.
Many physical machines and components come with a datasheet that will list their tolerances.
Failure to correctly document tolerances does in fact get you sued.
However, while this is truly a great idea, we're not going to be able to make it work for computational systems. Computers, software, and also LLMs are sensitive to initial conditions. Which is why tolerances are not so familiar to computer people. (but not entirely: eg your PSU might list 110-240Vac/300W as input tolerance)
Interestingly, LLMs actually have a somewhat lower sensitivity to initial conditions than traditional interpreters. See what happens if you misspell "What is One Plus nOe?". So they're actually a skosh off the edge and towards the middle, though I'd argue still very much at the computational end, just from the sheer scale of the valid inputs and outputs.
Mind you, if you have a pretrained LLM doing a measurable task on a line, possibly some sort of tolerances could be determined. Not so much when doing arbitrary chat.
Something unintuitive: I bet that often setting the temperature > 0 (aka introduce stochasticity deliberately, variously comparable to dithering or simulated annealing in other disciplines - doing the thing where you escape local minima) will tighten the output tolerance range and improve reliability, especially in iterated processes. This works for a lot of physical and digital processes actually, and LLMs simply stole the same trick.
(edit: I'm trying to compress a huge chunk of dynamics intuition in a few lines here. Hopefully still useful.
TL:DR; Everything real is continuous and noisy if you look close; and you're really trying to build attractors and bound variance, if you can. )
So today's the yearly presentation of the new budget, with protests expected in several places. Whether this sabotage campaign is (part of a) misguided protest action or something else is yet to be determined.
That does mean that I would not automatically jump to conclusions about Russia or war or whatever; since much of national public policy is presented today, it's also a popular day to express disagreement, and apparently some farmers had already announced they would be doing so today.
(I'd assume the BBC reporter to be aware that Prinsjesdag is today, and that they'd mention that fact in this article, but apparently not.)
The problemis that in the hugging face incident the only thing they had to defend themselves was with GLM, since the american frontier models all locked them out due to the security nature of the incident.
So banning open source models is potentially removing the last line of defense.
(ZFS might be described as the love child of lvm and btrfs, 'cept ZFS got there first)
( Very briefly: you hand ZFS a stack of disks (in mirrors or raidz groups, for redundancy) and it makes a zpool. Then you carve the pool into datasets as needed. Each dataset looks like an ordinary directory and takes only the space its files actually use, so you never guess partition sizes again. And each one can have its own compression, snapshot schedule, quota, encryption, and send/receive backups, etc etc etc. )
Not a btrfs user but OpenZFS u til recently didn’t even support O_DIRECT and may still duplicate page cache in arc. It’s not well suited as a general purpose FS where you can’t anticipate the workload, like on a desktop.
Anecdotal, but I've been happily using it under Linux desktops for years, and it works quite well. Workloads include: development, VM hosting, steam gaming, web browsing, multimedia playback. OSes include Debian, Proxmox, vanilla Arch, CachyOS, and others. Daily-driver hardware (ignoring servers and less general-purpose desktop stuff) included 2019 chromeboxes, 2015 (!) laptops, current-gen gaming laptops, and desktop towers with handfuls of spinning rust and solid state drives.
It seems to work well in a variety of situations: 4GB/single-slow-SSD ancient systems work just as well as spinning rust bulk storage pools with NVME ARC/ZIL caches for my gaming/server/database datasets, and all-SATA-SSD pools can get to near-NVME performance with bonus redundancy for boot volumes and latency-critical stuff. For personal desktop use, I haven't found dedup worth the squeeze in RAM costs and tuning (it works, but it's generally easier to solve most dedup-compatible problems at a layer closer to the cause).
ZFSBootMenu and the ability to roll back to snapshots and restore/maintenance disks from outside of the primary operating system, without having to think about fallback boot drives or physical backup volumes, is a godsend in the "try random sketchy commands that might trash my installation in order to get a low-level driver problem resolved" and "I could take the time to understand what this curl | bash invocation does, but I have better things to do; I want to be able to reverse it if it breaks stuff" departments.
In general, I strongly recommend ZFS for daily-driver use. Its core primitives are quite flexible, it makes redundancy/backups/drive addition/replacement easy, and it works fine on old and under-resourced systems; the mythos of "it requires ECC and enterprise-grade hardware and tons of RAM/CPU to work at all" was always bunk. The enterprise/SAN features are there if you want them, but are off by default, and the core FS capabilities are widely useful. Even casual desktop Linux users would do well to set it up, since there are a lot of rare-but-real ordinary user needs that, if they come up and you're not running something like ZFS, can't be done at all unless you connect purpose-specific hard drives or reinstall your OS.
Especially now that NVMEs are so expensive, ZFS should be considered for its ability to make RAIDing up a set of slower drives (or mostly slow drives with an NVME cache) very easy. That way, you can make your existing disks into something that performs well enough that you don't need to spend money on new hardware.
Just don't install it via DKMS; get a distro that ships it compiled into the kernel or as an installable kernel-paired module. Many such distros exist. The DKMS edition won't eat your data, but you'll get real tired of failed system updates because the kernel changed some source and the compile failed. That happens often; turns out that the volume of the kernel API surface used by something as massive as the ZFS codebase is quite large.
Edit: upon reading back through this, I'm a bit sheepish that I sound like such a breathless shill. I promise I'm not in the ~pocket~ zpool of big filesystem. I just like it.
ZFS is overrated and mostly a hobbyist/enthusiast thing.
Performance of zvol is worse than for LVM logical volumes. Last I've looked, zvol for swap is just asking for trouble, up to and including deadlocks. ZFS filesystems do have weird edge cases and incompatibilities, e.g. around free space reporting, quota and the like. And support with software vendors e.g. for enterprise distributions or backup software is spotty at best.
>> ZFS is overrated and mostly a hobbyist/enthusiast thing.
Not true at all.. but most large object storage redundancy has moved to the application layer. There are still workloads that are a great fit but I agree OpenZFS is currently not well suited as a general purpose FS.
And since you should probably be running a modern CoW filesystem then you probably shouldn't be using swapfiles ... is the conclusion I arrived at anyway. Either install multiple times more RAM than you need and enable zswap or else set aside a swap partition as has always been done.
Swapfiles on BTRFS seem to work great if you allocate them early (with the command linked in your article) and put them in a different subvolume than the rest of your system.
BTRFS in general seems more I/O constrained than ext4 or xfs so it's probably still worse to put a swapfile on a BTRFS partition, but so far my systems are doing fine.
> BTRFS in general seems more I/O constrained than ext4 or xfs so it's probably still worse to put a swapfile on a BTRFS partition,
AFAIK, on Linux swap bypasses the filesystem. It asks the filesystem for the extents which are used by the swapfile, and does the I/O directly on the block device. This explains the many restrictions swapfiles have on btrfs (the file can't be copy-on-write, can't be mirrored, etc), but it also means the speed for a swapfile will be the same on btrfs as on ext4 or xfs (assuming a similar layout, that is, the file is not too fragmented; IIRC, the kernel prints the number of extents when you do a swapon, so you can see when it's too fragmented).
zswap indeed is the real magic almost silver bullet nowadays. Device backed swap is often not needed anymore and it solves a lot of the issues with freezing under memory pressure, though it may not be ideal for servers.
And if I have a separate otherwise unused ssd then using a swap partition simply is easier than creating a file system for it and mounting it somewhere. Block level backups also are easier.
What would be nice about swap files is if they could dynamically grow and shrink without a special setup.
Way too many variables on Linux for a one size fits all.
That’s a good call out. I’ve never used ZFS on root. Been a ZFS user on Linux since the FUSE days but it has never seemed particular well suited (O_DIRECT support came late, ARC/page cache redundancy, etc). I can’t guarantee a consistent enough workload to optimize ZFS for and I don’t feel like it’s general purpose enough. Large object storage though? I’m sitting on a PB of ZFS.
Many physical machines and components come with a datasheet that will list their tolerances.
Failure to correctly document tolerances does in fact get you sued.
However, while this is truly a great idea, we're not going to be able to make it work for computational systems. Computers, software, and also LLMs are sensitive to initial conditions. Which is why tolerances are not so familiar to computer people. (but not entirely: eg your PSU might list 110-240Vac/300W as input tolerance)
Interestingly, LLMs actually have a somewhat lower sensitivity to initial conditions than traditional interpreters. See what happens if you misspell "What is One Plus nOe?". So they're actually a skosh off the edge and towards the middle, though I'd argue still very much at the computational end, just from the sheer scale of the valid inputs and outputs.
Mind you, if you have a pretrained LLM doing a measurable task on a line, possibly some sort of tolerances could be determined. Not so much when doing arbitrary chat.
Something unintuitive: I bet that often setting the temperature > 0 (aka introduce stochasticity deliberately, variously comparable to dithering or simulated annealing in other disciplines - doing the thing where you escape local minima) will tighten the output tolerance range and improve reliability, especially in iterated processes. This works for a lot of physical and digital processes actually, and LLMs simply stole the same trick.
(edit: I'm trying to compress a huge chunk of dynamics intuition in a few lines here. Hopefully still useful.
TL:DR; Everything real is continuous and noisy if you look close; and you're really trying to build attractors and bound variance, if you can. )
reply