Porting NVIDIA's CUDA Megakernel (MPK) to China's T-Head PPU-ZW810E
A “theoretically impossible” experiment: take the Mirage Persistent Kernel (MPK) megakernel — hand-written for NVIDIA Hopper/Blackwell — and run it on Alibaba T-Head’s Zhenwu 810E (PPU-ZW810E, a non-NVIDIA architecture). The result: it not only ran, but single-card Qwen3-8B inference beat PPU’s own optimized stack.
This post records the full process of hitting walls, cracking the case, and tuning. All numbers come from real hardware.
TL;DR
- ✅ MPK built from source on the PPU (CUDA-compatible,
__CUDA_ARCH__=800). - ✅ The Qwen3-8B megakernel ran and produced correct output.
- ✅ After fixing one deadlock, 16-worker single-kernel path = 35.5 ms/token, beating native’s 39.4 ms (1.11×).
- 🔬 Using PPU’s full debugging toolchain (ppu-gdb / racecheck / synccheck / occupancy probes / instrumentation), I diagnosed a high-worker-count deadlock layer by layer as a scheduler-scale livelock.

1. The Cast: A CUDA Megakernel, and a Card That Isn’t NVIDIA
MPK (Mirage Persistent Kernel) takes a radical approach: fuse every operator and all communication of the entire LLM inference pass into a single “giant persistent kernel” launched once, with an in-kernel scheduler dispatching tasks to resident workers. It was hand-written for NVIDIA — full of NVIDIA-exclusive instructions like cp.async, wgmma, TMA, and nvshmem.
PPU-ZW810E (Zhenwu 810E) is Alibaba T-Head’s in-house AI chip. It is not an NVIDIA architecture — it uses T-Head’s own PPU parallel architecture plus an ICN inter-chip interconnect.
The first instinct, moving the former onto the latter, is “the architectures aren’t compatible.” But the truth flipped that on its head.
2. The First Surprise: the PPU Actually “Speaks CUDA”
The PPU’s SDK is CUDA source-compatible: ppu-clang (clang/llvm) can compile CUDA C/C++, ships with an nvcc wrapper, and even supports Triton. The single most important line, measured directly:
1 | __CUDA_ARCH__ == 800 |
The PPU reports itself as sm_80 (Ampere) compatible! That means every #if __CUDA_ARCH__ >= 800 sm80 code path in MPK is activated as-is — it doesn’t have the Hopper/Blackwell-exclusive TMA/wgmma, but the Ampere path doesn’t need them.

3. Build Hell: Domestic Mirror Sources All the Way
The PPU lives in an Alibaba Cloud data center, and almost nothing outside the firewall is reachable. Building MPK meant a chain of pitfalls, each requiring a switch to a domestic mirror:
- pip: the container defaults to T-Head’s internal source, which lacks public packages → add
-i https://mirrors.aliyun.com/pypi/simple/. - Rust: when
setup.pywas missing cargo, runningcurl https://sh.rustup.rs | shhung atinfo: downloading installer→ switch to the aliyun rustup mirror + the aliyun crates mirror. - cutlass/json submodules: shipping just the includes wasn’t enough; cmake needs
CUDA.cmake/CMakeLists.txt. - transformers: the container had 5.3, but the demo was written against the pinned 4.57.1 and threw
pad_token_id→ downgrade the version.
Once through all that, import mirage succeeded and the PersistentKernel API was in place.

4. First Run: the Native Baseline
Before bringing in the megakernel, I ran Qwen3-8B via the demo’s built-in torch path (which goes through PPU’s own optimized libraries):
1 | Prompt length 39, generate length 256, per-token latency 39.444 ms |
The output was a coherent, correct “introduction to large language models.” On a domestic card, Qwen3-8B inference is correct at 39.4 ms/token. Baseline established.

5. Bringing in the Megakernel: Compiles, and Produces Correct Results
With --use-mirage on, MPK generated and compiled the entire 14,000+ task megakernel on the PPU:
1 | [MPK INIT] Total tasks: 14140, Total events: 2246 |
⚠️ A big trap here: after
[MPK INIT]comes ~2 minutes of nvcc compilation, during which the GPU is also occupied. I initially thought it had “hung,” killed it early, and misjudged it as a deadlock — when it was just compiling.
The megakernel produced correct output. But the first version (limited to 8 workers) was 68.2 ms/token, slower than native — because it used only 8 of the 64 SMs, and it used MPK’s generic Ampere kernel rather than the PPU’s optimized libraries.

6. The Deadlock Mystery: It Hangs Once There Are Too Many Workers
When I tried to use more workers for speed, something strange happened: 8 workers ran fine, but ≥24 workers always deadlocked — the PPU spun at 100% utilization doing nothing, and finally threw unspecified launch failure.

7. Detective Work: Cracking the Case with PPU Debug Tools
This is the most interesting part. The PPU has a full debugging toolchain that mirrors NVIDIA’s, and I brought them out one by one:
1. ppu-gdb attached to the deadlocked process — found that every device warp was a worker, all stuck in execute_worker spin-waiting for tasks, with not a single scheduler frame present.

2. racecheck (the counterpart to compute-sanitizer) — 0 hazards, ruling out data races.
3. synccheck (checking for __syncthreads/barrier deadlocks, ran for 13 minutes) — 0 errors, ruling out barrier misuse.

4. cooperative occupancy probe — unexpectedly discovered the PPU actually has 256KB of shared memory/SM (MPK assumed 163KB under sm_80), and that all 64 megakernel blocks could be co-resident simultaneously, ruling out “co-residency exceeded.”

8. The Truth and the Fix: From “Two Kernels” to “One Kernel”
The key to cracking the case was in the launch code: MPK defaults to split_worker_scheduler=true, splitting workers and the scheduler into two independent kernels on two streams, counting on the PPU to execute the two persistent kernels concurrently.
But the PPU’s concurrent multi-kernel scheduling is more conservative than NVIDIA’s: once the worker count is high (≥~24), it stops scheduling the second scheduler kernel → the scheduler never starts → the workers wait forever. That’s exactly the source of “only workers, no scheduler.”
A one-line fix:
1 | // include/mirage/persistent_kernel/persistent_kernel.cuh:1617 |
Switching to the single-kernel path: workers and scheduler merge into one kernel, with roles assigned by blockIdx, naturally co-resident in the same kernel, no longer dependent on “two concurrent kernels.”
The result: single-kernel, 16 workers = 35.5 ms/token, beating native’s 39.4 ms!

9. The Unsolved Mystery: the “Chronic Deadlock” at 32 Workers
The fix raised the ceiling from 8w to 16w and beat native, but 32w still hangs — and in a different way. After ruling everything out with the full toolchain: it’s not occupancy, not a race, not barrier misuse, not a single-point spin, but a distributed livelock where workers “grab sporadic tasks and then wait, never converging as a whole.” This kind of high-level progress deadlock is invisible to standard sanitizers.
Breaking through it requires source-level changes to how MPK’s scheduler dispatches tasks at scale — a good candidate for an upstream issue, backed by this body of ruling-out evidence. For now, 16w (beating native) is the production-usable ceiling.

Closing
| Path | per-token latency |
|---|---|
| Native (torch + PPU optimized libs) | 39.4 ms |
| Megakernel, two kernels, 8w | 68.2 ms |
| Megakernel, single kernel, 16w (after fix) | 35.5 ms ✅ |
A CUDA megakernel hand-written for NVIDIA not only ran and computed correctly on a domestic PPU, but beat the vendor’s own inference stack — thanks to the PPU’s solid CUDA compatibility (down to __CUDA_ARCH__=800, cooperative launch, and the nvshmem-equivalent sailSHMEM), plus a one-line fix switching the scheduling mode.
The remaining 32w livelock is a deeper problem at the intersection of a domestic card’s conservative concurrent scheduling and MPK’s at-scale scheduler — saved for the next post.