llamafile: the USB-stick LLM
llamafile welds llama.cpp and a GGUF model into one executable that runs on macOS, Linux, Windows, and the BSDs: the same physical file, no installer, no dependencies. The trick underneath is Cosmopolitan libc, which produces a polyglot binary every OS accepts as its own. For about a year I had it filed under cute tricks.
Then mid-June happened.
the demo that almost died
Client workshop, forty-odd people, and the venue guest wifi died during the session before ours. The plan had been ordinary: pull a model onto the presentation machine, demo a small ticket-triage assistant against the client's sample data, take questions. The machine was a locked-down corporate laptop where I had no admin rights — and now no network to pull anything with.
What I did have was a llamafile on a USB stick, left over from a spring workshop. Copy it across, rename it to .exe because Windows, double-click. A chat UI comes up on localhost:8080 with the weights already inside. The demo ran fully offline on a machine I'd first touched that morning.
It nearly went differently. The stick was formatted FAT32, which refuses files over 4 GB, and the llamafile was 4.6 GB. So I spent ten nervous minutes in a hallway reformatting it to exFAT and re-copying while the previous speaker mercifully overran. Check your stick's filesystem before the day you need it.
The most reliable network at any venue is the USB stick already in your pocket.
cosmopolitan weirdness
The single-binary trick has edges. An older zsh on one of my machines refused to execute it until I launched it through sh. Gatekeeper and corporate antivirus both give a 4 GB executable off a USB stick exactly the reception you'd imagine, so budget time for that conversation. And Windows won't start executables over 4 GB at all. I learned that the dumb way months earlier, after baking an 8 GB model into a workshop build that ran beautifully on my Mac and refused to open on the machines it was actually for. The escape hatch is official: keep the engine llamafile small and pass the big weights alongside as a plain GGUF with the -m flag.
a distribution format, not a runtime
That's the frame that makes llamafile make sense. As a way to hand somebody a working LLM (an air-gapped demo, a training workshop, my mother-in-law's nine-year-old laptop, a model you archive now and can still run in 2031), it's the best thing going. As a daily runtime it's the wrong tool: no model management, no easy swapping, and every update means shipping gigabytes all over again. My own day-to-day setup looks like the local LLM guide; the llamafile lives in the bag for the days infrastructure betrays me.
One non-obvious caution before you hand these out like business cards: a llamafile with weights inside is model redistribution, and not every license is relaxed about that: the licenses piece covers which models travel well. Sort that out once, and the USB stick becomes the calmest deployment target you own.