AMD Instella-MoE: a fully open MoE trained end to end on Instinct, no CUDA involved

AMD released Instella-MoE-16B-A3B at the end of July, with coverage landing August 1 via the ROCm blog and MarkTechPost. It is a Mixture-of-Experts model with 16B total parameters and 2.8B active, built in the style of DeepSeek V3 with Gated Multi-head Latent Attention and a FarSkip-Collective communication scheme, and it was trained from scratch on AMD Instinct MI300X and MI325X GPUs using AMD's Primus framework. Not a single CUDA kernel in the training path. That sentence matters more than the benchmark line.

Fully open here means the whole recipe, not just the final weights: the data mix, every checkpoint from pre-training through mid-training, long-context extension, SFT, DPO and RL, plus the inference code. A base checkpoint average of 76.7 makes it the strongest among fully open models by AMD's reckoning. There is also a reasoning -Think variant. If you want to study how a modern MoE is actually cooked, this is the most complete public kitchen I know of.

Why no-CUDA matters

The narrative that frontier-class training cannot happen without Nvidia has been eroding for a while, but mostly on the inference side. An end-to-end pretrain of a MoE with latent attention and a custom collective on MI300X is a different class of evidence. For anyone pricing training capacity it means the second-source conversation is real: ROCm is not a port of CUDA anymore, it is a stack that trained a model people will cite. I wrote about the inference half of the AMD question in Nvidia vs AMD for local inference; the training half just got its data point.

The interesting number in Instella-MoE is not 76.7, it is zero: the count of CUDA kernels between the data and the checkpoint.

The A3B footprint for local inference

2.8B active parameters is the local-inference story. At decode time you stream roughly the active weights per token, not the full 16B, so on a bandwidth-bound machine the throughput profile looks like a 3B dense model while the quality profile sits closer to a mid-size dense model. The full 16B still has to fit in memory: at 4-bit that is somewhere around 8 to 9 GB plus KV cache, comfortable on any 24 GB Mac or a single consumer GPU. The Gated MLA is the part I want to profile. Latent attention compresses the KV cache, and on unified-memory machines that is the difference between a usable and an unusable long context. The August 2 brief flags exactly this as the profiling target and I agree with it.

The sparse activation also has a cost that dense-model intuition misses: expert routing adds memory traffic that does not show up in the active parameter count, and MoE decode on a quantised model depends on how each expert was quantised. Expect the measured tokens per second to land below the 3B-dense ceiling, and measure how far.

Licence: this is where it stops

ResearchRAIL. Research and academic use only, no commercial. That excludes it from every client deployment, every product and every internal tool that touches revenue. AMD knows what it is doing here: the release is a proof of the stack, not a competitor to Qwen or GLM in production. Treat it as a reference implementation you learn from, not a model you ship. The local models overview stays the place to pick an actual deployable.

What to do with it

• Read the recipe. The checkpoint sequence and data mix are the part with lasting value.
• Benchmark A3B decode on your own hardware against a 3B dense and an 8B dense to see where the MoE lands on tokens per second.
• Profile the KV cache footprint under Gated MLA at 32K and 128K context.
• Do not put it in a product. Put a Qwen or GLM with a permissive licence in the product and use Instella to understand why it behaves the way it does.

The honest limitation

The 76.7 average is AMD's own number over AMD's chosen suite, and strongest fully open depends on how narrowly you define fully open. The recipe being public is the verifiable part; the ranking is marketing until someone reproduces it. And a model you cannot legally ship is, for most readers here, an education rather than an option.