Unlocking GPU-to-GPU P2P on the CMP 170HX — Nine Links in a Chain, and One Machine I Killed
The CMP 170HX is NVIDIA’s mining variant of the GA100: same silicon, same HBM2e, but the firmware reports 8GB of memory, pins PCIe at Gen1, and fuses BAR1 to 64MB. The community already has solid work unlocking memory geometry and Gen2. This post is about the last piece — GPU-to-GPU P2P.
The conclusion first: it works. All 14 intra-socket directed pairs pass, and latency drops from 15–37 µs to 2.0 µs — the same order of magnitude as a GPU touching its own framebuffer (2.04–2.59 µs).
Getting there took longer than I expected. This is not “apply a patch.” It is a chain of nine links in series: miss any one and P2P is refused at runtime, and each one fails with a different symptom. Along the way I got the machine into a state where only ping answered, and it needed a physical power cycle to come back.
Hardware: Supermicro SYS-4028GR-TR, dual Xeon E5 v4, 6× CMP 170HX (10de:20c2), Ubuntu 24.04 with nvidia-open 610.43.02.
1. What we’re actually unlocking
GA100 PCIe P2P normally runs over the write-mailbox protocol. On this card, that path goes like this:
The driver configures it completely — the connection type is selected, both GSP RPCs return NV_OK, the mailbox region inside BAR1 is allocated. And then it moves zero bytes. No error. No Xid. Nothing.

Everything configured successfully, data plane at zero. This is the nastiest class of failure, because every checkpoint tells you it’s fine.
The way around it is BAR1 P2P: rewrite the peer PTEs from GMMU_APERTURE_PEER to SYS_COH, pointing at the peer’s BAR1 bus address. Peer memory is then accessed as though it were system memory, and the dead peer aperture never participates.
The two paths are mutually exclusive — the driver only allows one PCIe P2P protocol at a time. So this isn’t “have both.” It’s “you must give up the mailbox.”
2. BAR1 P2P wants a 64GB BAR1; the card ships with 64MB
BAR1 P2P requires a 64GB BAR1 on every card (the whole framebuffer has to map into it). The CMP 170HX ships with BAR1 at 64MB.
The good news is the hardware supports it. Once the driver’s CYA unlock has run, lspci reports:
1 | BAR 1: current size: 64MB, supported: 64MB 128MB ... 32GB 64GB |
The bad news is that an entire PCIe resource-allocation problem sits between those two numbers.
I got this wrong once
I saw a 96MB bridge window and a 224MB root-port window against a requirement of 96GB/128GB, and blurted out “that’s three orders of magnitude short.”
That was wrong. 96MB is the window BIOS already assigned — not the address space available. The actual space was 512 GiB per socket:
1 | 20000000000-27fffffffff : PCI Bus 0000:00 ← 512 GiB |
Less dramatic, but it changes the diagnosis entirely: the space wasn’t missing, the windows were locked to the wrong size.
3. The address-space arithmetic, and a timing problem

Space first. Each card’s BAR1 is 64GB and must be 64GB-aligned; adding the 32MB BAR3, a PLX downstream port needs a 96GB window that has to start on a 64GB boundary — so it really occupies 128GB.
1 | per downstream port 96 GB window, 64 GB aligned |
Socket 1 on this box carries 4 cards, so the requirement lands exactly on 512GB, and the bottom of that window is already occupied by other bridges. It cannot fit. Changing MMIO High Granularity Size to 1024G in BIOS gives each socket 1 TiB, and the problem goes away.
One common misconception worth correcting: the setting is not Above 4G Decoding. That was already enabled on this machine — the proof being that GPU BARs sat at 0x27ff8000000, around 2.75 TB, which is unreachable without it.
The genuinely awkward part is timing
Space was sufficient and BAR1 was still 64MB. Because:
The card ships with BAR1 fused to 64MB, and the unlock happens at driver probe time — by which point PCIe enumeration is long finished, every window in the bridge chain has been sized for 64MB, and pci_resize_resource() has nowhere to put the larger BAR. It steps back down.
That is the reason the two kernel patches under kernel-patches/ exist:
- 0002 moves the ReBAR programming into
pci_fixup_early, which runs beforepci_read_bases(), so enumeration sees a 64GB BAR from the start - 0001 fixes one line in
pbus_size_mem()—size += max(r_size, align)becomesALIGN(r_size, align). The original doesn’t budget for the gap a child leaves when its size isn’t a multiple of its alignment, so parent windows come out undersized and some GPUs silently get no BAR at all
Upstream ships these against the 7.0 kernel. This box runs 6.8, and the way I rebased them was: make the equivalent edit in the real 6.8 source, then diff -u it back out — rather than hand-adjusting line numbers. A patch generated that way is guaranteed to apply, because it came from the tree it targets.
4. [The pitfall] One reboot took the whole machine offline
With the driver installed, I typed sudo reboot.
The machine came back. nvidia-smi hung. Then ssh stopped accepting connections. Then the only thing still answering was ping.

dmesg said:
1 | NVRM: GPU0 kflcnWaitForHalt_TU102: Timeout waiting for Falcon to halt |
My first root-cause call was wrong
Before I had the log, I asserted: “I made P2P default-on and wrote RMForceStaticBar1=1 unconditionally, and this machine only has a 64MiB BAR1 — forcing a 64GB static mapping into a 64MiB BAR1 hangs driver init.”
It sounded reasonable. The log matched none of it. The failure is in GSP/Booter bring-up, which belongs to the SEC2 unlock path and runs before any BAR1 or P2P code. RMForceStaticBar1 had nothing to do with it.
“I just changed X, so X broke it” is the easiest attribution error to make — especially when X really does look suspicious. Suspicious is not guilty.
The actual cause
This unlock uses the Booter as a write primitive for privileged registers. Many registers on a CMP simply cannot be written by the host at Level 0, so the code builds a SEC2 payload and calls kgspExecuteBooterLoad to have the Booter perform the write — one register at a time, dozens to hundreds of times.
And WPR2 and the PLM state live inside the GPU; a warm reboot does not clear them. Boot with WPR2 still armed from the previous session and the Booter can no longer run, which kills the write primitive — so every call burns a 4-second timeout.
Then the cascade: GSP boot fails 4/4 → RmInitAdapter failed → nvidia-smi and the nv_open_q / nv_queue kernel threads wedge in uninterruptible sleep, unkillable → every subsequent process touching the driver piles up behind them → the process count climbs until fork() fails → update-initramfs reports Cannot fork → sshd exits.

A driver failing to initialize should not be able to take a machine down to the point where ssh no longer works. That is a genuine robustness defect, and part of the responsibility sits with the unlock patch itself. Upstream turned NVIDIA’s hard refusal:
1 | NV_PRINTF("unexpected WPR2 already up, cannot proceed with booting GSP"); |
into a warning and carried on. What should have been a clean failure in seconds, with ssh still usable, became four rounds of timeouts that dragged the driver down with them.
The lesson
This unlock requires a cold boot; reboot does not count. Power actually has to leave the card: shutdown -h then power on, or chassis power cycle over IPMI. chassis power reset is not enough either — it doesn’t drop the 12V rails.
At the time, that machine had no working out-of-band management. Someone had to walk into the machine room.
So the correct order is: establish your recovery path first, then do the risky thing. I later brought up the BMC (it can be configured from inside the OS over KCS, without being able to reach it on the network first), and every cold boot after that was one command.
5. Kernel patches in, and only 4 of 6 cards came up
With the custom kernel booted, the pci_fixup_early quirk fired on all six cards — the log is unambiguous:
1 | pci 0000:04:00.0: CMP 170HX: BAR1 REBAR programmed to 64GB before enumeration |
64GB at enumeration time. And yet: four cards got a 64GB BAR1, and the two on socket 0 got nothing — not even the 64MB they used to have.
1 | pci 0000:04:00.0: BAR 1 [mem size 0x1000000000 64bit pref]: can't assign; no space |
That root-port window is still the 224MB value from firmware. The kernel never tried to reassign it — while socket 0 had a full, empty terabyte and needed 192GB.
The reason is Linux’s realloc heuristic: it only engages when it happens to notice a device with unassignable BARs. The SR-IOV device that tripped it sat on socket 1, so socket 1 got rescued and socket 0 was skipped.
The fix is a kernel parameter:
1 | GRUB_CMDLINE_LINUX_DEFAULT="... pci=realloc" |
which forces reallocation on every root bus instead of relying on the heuristic. After that:

6/6, all 64GB.
6. Everything in place, P2P still refused — and “no output” was the answer
BAR1 at 64GB on all six. Kernel patches in. Module parameters RMForceStaticBar1=1;RMPcieP2PType=1 live. Three of the four driver patches applied.
Ran the test. All 30 directed pairs: NO PEER MAPPING.
The driver-side gate function kbusIsPcieBar1P2PMappingSupported_GH100 has four checks, and I didn’t know which one was rejecting. This time I stopped guessing and ported a print-only, behaviour-free debug patch — an unconditional NV_PRINTF at the top of that function, dumping all four gate inputs.
Rebuilt, cold-booted, ran the test. And:
Not a single line of output.
That was the answer. The print is unconditional; it fires on entry. Zero output means the function was never called — the rejection happened further upstream, and none of the four gates was ever reached.
Following it up: GSP reports P2P as unsupported on a CMP, _gpuInitPcieP2PCapability stores that verdict, and p2pCapsGetStatus short-circuits on it — so control never reaches the BAR1 branch at all.
The fourth patch, p2p-force-caps, forces the advertised capability bits to OK. The upstream notes describe it as optional, something that “only changes what the driver claims.”
On this code path that is wrong — it is a hard prerequisite. Without it, the other three patches never get a chance to execute.
With it applied:
1 | CMPUNLOCK: PCIe P2P caps forced to OK |
Through.
There’s an inverse trap worth noting here: the
p2p-mailbox-skippatch’s log line never appears, even when the patch is applied — release builds stripNV_PRINTFformat strings in that translation unit. A missing log line does not mean missing code. Verify withnm/objdumpon the object file, not by grepping dmesg.
7. The nine required pieces

Miss any one and it’s refused at runtime, each with a different symptom:
| # | Piece | Layer |
|---|---|---|
| 1 | BIOS MMIO High Granularity Size = 1024G |
firmware |
| 2 | Kernel patch: size bridge windows for alignment padding | kernel |
| 3 | Kernel patch: program ReBAR before enumeration | kernel |
| 4 | pci=realloc |
kernel cmdline |
| 5 | p2p-bar1: BAR1 P2P HALs + peer PTE rewrite |
driver |
| 6 | p2p-mailbox-skip: skip mailbox peer pre-registration |
driver |
| 7 | p2p-readcap-override: restore the misjudged read cap |
driver |
| 8 | p2p-force-caps: force advertised caps (not optional) |
driver |
| 9 | RMForceStaticBar1=1;RMPcieP2PType=1 |
module params |
Number 6 deserves a note: it actively disables the mailbox peer pre-registration. _kbusInitP2P_GM107 unconditionally registers a mailbox peer id for every GPU pair at init, and the fourth BAR1 P2P gate explicitly refuses to coexist with a mailbox peer. Leave it in place and peerNumberMask stays non-zero, so BAR1 P2P is never selected.
So “the mailbox is still locked” isn’t a loose end — it’s a precondition of this approach.
One more trap in the module parameters: they must go on a single NVreg_RegistryDwords= line. modprobe does not merge duplicate assignments of the same parameter across files — a second options nvidia NVreg_RegistryDwords=... anywhere in /etc/modprobe.d silently replaces the whole set.
8. What it buys

Measured with NVIDIA’s own p2pBandwidthLatencyTest, the latency matrix is the most convincing number:
1 | P2P=Disabled Latency (us) P2P=Enabled Latency (us) |
Intra-socket pairs go from 15–37 µs to ~2.0 µs. The diagonal — a GPU touching its own framebuffer — is 2.04–2.59 µs. P2P latency is now at local-memory scale. Only a genuinely direct path does that; a staging protocol cannot.
Bandwidth had a bonus in it. The check tool flagged ACS redirection enabled on all six downstream ports (ReqRedir+ CmpltRedir+), which forces peer traffic within a switch up to the root complex and back. The symptom was obvious in hindsight — same-switch pairs were measuring slower than cross-switch ones:
| Bidirectional | ACS redirect on | ACS redirect off |
|---|---|---|
| Same switch (0↔1, 2↔3, 4↔5) | 9.89 GB/s | 12.98 GB/s |
| Cross switch, same socket | 10.99 | 10.99 |
| Cross socket | 8.53 | 8.53 |
Only those three same-switch pairs moved; everything else stayed put — exactly the distribution you’d expect when redirection stops pushing traffic out of the switch. +31%.
Making it permanent has a GRUB trap: pci=disable_acs_redir= is semicolon-separated, and ; is a statement separator in GRUB’s own script language, so the parameter gets truncated at the first one. Escape them:
1 | pci=disable_acs_redir=0000:03:08.0\;0000:03:10.0\;... |
The cost is that GPUs under one switch merge into a single IOMMU group. On this box (iommu=pt, no VMs) that’s free — but don’t do it if you need to pass individual GPUs through to guests.
9. The ceiling
14 of 30 directed pairs pass. All 16 failures are cross-socket.
That’s not a configuration problem, it’s topology. GPU 0/1 are on socket 0 and GPU 2–5 on socket 1 (SYS in nvidia-smi topo -m); peer traffic would have to cross QPI, and the driver refuses.
Put differently: of the 14 pairs that are physically capable of it, 14 work. None missing.

Worth noting that the advertised flag stays honest — 0 for cross-socket pairs, 1 for intra-socket. Even with forced capability bits, the pair-level check still correctly rejects cross-socket, so nothing claims to work and then times out.
On verification, one thing has to be said plainly: do not trust cudaDeviceCanAccessPeer(), nvidia-smi topo -p2p, or cudaMemcpyPeer(). The first two report what the driver claims, and this build forces that claim to OK. The third silently stages through host memory when P2P is unavailable, so it succeeds either way.
The test I wrote dereferences the peer pointer from inside a kernel — that can only succeed through a real peer mapping — and seeds both buffers with distinct per-GPU, per-slot patterns, so a mapping that quietly aliases back to local memory gets caught instead of reading back as a pass.
10. The reproducible checklist
The full walkthrough, including the pitfalls, is in COOKBOOK.md in the repo. Shortest path:
The order matters — build your recovery path before doing anything risky:
Get the BMC working (zero risk, pure upside). It can be configured from inside the OS over KCS, without reaching it on the network first:
1
2
3
4sudo ipmitool lan set 1 ipsrc static
sudo ipmitool lan set 1 ipaddr <ip>
sudo ipmitool user set password 2 '<pw>'
sudo ipmitool mc reset cold # resets only the BMC; the host keeps runningAfter that
chassis power cycleis a real cold boot, andsol activategives you a serial console.BIOS:
Advanced → PCIe/PCI/PnP Configuration → MMIO High Granularity Size = 1024G. Don’t just pick the maximum — the requirement is known and 1024G is already 2× headroom, whileMMIO High Baseset too high is the setting that stops the machine POSTing.Build the kernel with the two patches from
kernel-patches/. Boot it one-shot so a failure falls back on its own:1
sudo grub-reboot "Advanced options for Ubuntu>Ubuntu, with Linux <version>-cmp"
The entry name must match
grub.cfgexactly — a wrong name silently boots the default. Andgrub-rebootis consumed by one boot; set it again before each attempt.Add
pci=reallocto the kernel command line.Install the driver:
sudo ./install.sh --p2p(it checks the BAR1 prerequisite first and refuses if unmet).Cold boot, then verify:
1
sudo ./tools/p2p-check.sh
Closing
Looking back, the time didn’t go into writing code. It went into two bad calls:
Once, asserting a root cause before I had the log (blaming RMForceStaticBar1). Once, taking the upstream “optional” label at face value (p2p-force-caps is a hard prerequisite).
Both are the same pattern: I chose to infer when I could have verified.
The second time taught me something. When P2P was being refused and I didn’t know which of four gates was failing, I stopped enumerating possibilities and added one print statement. Three minutes to a definite answer, replacing several rounds of guesswork.
When you catch yourself saying “it could be A or it could be B,” ask first: is there an observation that separates them? Adding a log line or running a read-only command is almost always cheaper than another round of trial and error.
And the piece of evidence that settled it was nothing at all — not a single line printed.