DeepGEMM on Ascend 950: what it takes to leave CUDA, and why the speedup claims need a grain of salt

There is a quiet class of release that matters more to the people who run inference than any new model, and it looks like this: a repository of matrix multiplication kernels, a license, and a README that does not try very hard to impress you. On September 30 DeepSeek published an Ascend port of DeepGEMM, released under the MIT license alongside TileLang-Ascend and DeepEP, aimed at Huawei's Ascend 950. It contains BF16, FP8 and FP4 GEMM kernels, MQA logits operators and MegaMoE operators, and it keeps the existing DeepGEMM API, so code written against the original should not need to be rewritten. That last detail is easy to skim past and is, I think, the most important one.

I have not run any of this, and I do not own an Ascend. What follows is an attempt to explain what the release is, why it is harder than it looks, and how I would read any performance number attached to it.

Why a GEMM library is the whole ballgame

Almost everything a large model does reduces to multiplying big matrices, and the speed of that operation on a given chip is the difference between a usable accelerator and a very expensive heater. CUDA's real moat is not the hardware. It is that two decades of tuned kernels, compilers and libraries sit between your model and the silicon. A competitor can build a fast chip and still lose, because the software that makes the chip fast has to be written by someone, tuned by someone, and trusted by someone.

A CUDA-free kernel library attacks exactly that moat. It says: here is the low-level code for the operations that dominate your workload, written natively for the other platform, with the interface your framework already expects. When the API is preserved, a team can switch the backend without touching the model code above it. That is a very different proposition from porting a model, and it is why I would call this a strategic release more than a technical one.

Low precision makes it harder, not easier

The FP8 and FP4 part deserves respect. Moving to narrow number formats saves memory and bandwidth, but it forces you to handle scaling factors, rounding and accumulation carefully, and the details depend on the instruction set. A kernel that is correct and fast on one architecture does not translate by renaming functions. Memory layout, tile sizes, how data moves between on-chip buffers and main memory, how many operations you can keep in flight: each of these is a separate design problem on a different chip. I looked at a version of that problem on another platform in the vLLM and DeepSeek V4.1 notes for AMD, where even a modest change to the cache format moved the picture.

So when a project says it supports FP4 on a new accelerator, I read it as: someone did a great deal of unglamorous work. It does not yet tell me the accuracy is acceptable for your model or that it is fast enough to matter.

How to read the speedup numbers

Here is the grain of salt, and it is a large one. The microbenchmarks that accompany a release like this are produced by the project, on hardware the project chose, with shapes the project picked. If somebody tells you a kernel is twice as fast, the honest question is: twice as fast as what, on which matrix shapes, at what precision, at what power draw?

A fair comparison between an Ascend and an Nvidia part needs the same model, the same precision, the same shapes and a comparable power budget. Anything less compares two stories, not two chips. I would also want to know whether the baseline was a heavily tuned library or a default build, because that single choice can account for the whole gap. DeepSeek's own framing is project-reported microbenchmarks, and I would carry that label with every figure.

A speedup without matched shapes, precision and power is a sentence, not a measurement.

For anyone who cares about throughput per watt or per dollar, the number to ask for is end-to-end: tokens per second on a real model, at a real batch size, with a real context length. That is also where the earlier work on long-context cost shows up, as in the HySparse2 prefill discussion, because the kernel is only one term in the bill.

Why this matters even if you never buy Ascend

You may think this is irrelevant unless you run Chinese hardware. I would argue otherwise, for two reasons. First, it is evidence about how portable the serious inference stack is becoming. Last month's story was kernels on Apple's Metal and on AMD's ROCm; this month there is a third accelerator family with an open kernel layer. The more platforms have credible open kernels, the weaker the lock-in argument becomes for everyone, including people who stay on Nvidia and simply gain negotiating leverage.

Second, there is a supply chain angle that clients do bring up. Export restrictions have made the question of what runs without Nvidia a practical one for some organizations, and an open, permissively licensed kernel stack changes the arithmetic of lock-in versus alternative. I would not make a procurement decision from a repository, but I would put it on the watch list.

What I would do with it

If you maintain an inference framework, read the API surface and see whether your backend abstraction could accommodate a non-CUDA path without a rewrite. If you advise on infrastructure, ask the questions above before repeating anybody's number. If you simply follow the field, watch for independent measurements, ideally from someone who owns both kinds of hardware and publishes shapes. Until those exist, the right posture is interested and unconvinced, which in my experience is where most good engineering opinions begin.