Cnuas Validation Matrix¶
Scope: Every Cnuas component, its validating tests, and their status · Version: `5c3d075-dirty` · Updated: 2026-08-21
This page answers one question: for each implemented feature, which test proves it works, and did that test pass?
The Product Roadmap records what is planned. This page records what is proven. The two are deliberately separate: a roadmap entry can be marked available only when a row here backs it with a named, passing test.
Regeneratable evidence¶
The prose inventory below is not the release artifact. Regenerate the authoritative host evidence from a clean, tagged checkout with:
Each run records the commit and submodule revisions, dirty-tree state, host and
tool versions, exact commands, exit codes, durations, complete logs and JUnit
testcase records. It generates Markdown, CSV and LaTeX summaries from those
results and places SHA-256 hashes in SHA256SUMS. Verify an extracted artifact
before review with:
Counts are emitted only when JUnit or the native test binary provides a machine-readable count. Tests requiring guest devices, kernel modules, a two-node lab or an external reference checkout remain explicit gated records; they are never converted into host passes. See the evidence instructions for release and DOI packaging.
How to read this page¶
| Column | Meaning |
|---|---|
| Feature | The capability being validated |
| Test | The file or binary that validates it |
| Count | Number of test cases |
| Status | Result of the last recorded run |
| Gate | What must be present for the test to execute |
| Status | Meaning |
|---|---|
| Pass | Ran and passed on the reference host |
| Gated | Correctly skipped; needs hardware or a lab the reference host lacks |
| Manual | Exists, but is a standalone script that pytest does not collect |
| None | No automated test exists yet |
Summary¶
Recorded on the reference host (cnuas1). The pytest rows were recorded on
2026-08-03; the CnuasGPU rows were recounted after the compute backend
activation set (ABI 0.2), the host device backend, the vectorised
activations, and the ggml integration landed.
The host device backend (CNUAS_DEVICE_BACKEND=host) changed the shape of
this table. Tests that need a device but not a real one now run in the
ordinary suite instead of only inside a guest, which moved 4 binaries and
42 cases out of the gated row and added 2 new binaries. Only
multi_gpu_smoke remains gated, because it needs two devices and the host
backend presents one. Nothing was removed; the count rose from 510 to 609
at that point, and stands at 687 across those two rows today after the
additions described below. It rose then because more of the tree became
reachable without hardware, and because
compute_backend_smoke gained 9 cases when its softmax check was split
into three independent assertions instead of one that stopped at the
first failure.
The host binary count then rose from 567 to 636 with the addition of the
transposed GEMM. Sixty of those are in compute_backend_smoke, which
checks sgemm_nt twice over at four shapes, against a double precision
oracle and bit for bit against sgemm given an explicitly transposed
operand; nine are in cnuasblas_host_smoke, which now checks that
CNUASBLAS_OP_T produces the same product as the untransposed call
rather than checking that it is refused.
Those same nine cases appear a second time in the host device backend row,
which rose from 42 to 51 for that reason and no other. cnuasblas_smoke is
built twice from one source: once against a host stub, where it counts
towards the host binaries, and once against the real runtime on the host
device backend, where it counts towards that row. The duplication is
deliberate, since the two builds exercise different code beneath an
identical test body, but it does mean a case added to that file is counted
in both rows.
The count then rose from 636 to 649 with cnuasdev_arena_size_smoke, which
covers arena sizing and the 4 GiB offset boundary. Its thirteen cases exist
because the other arena test uses a 4 MiB arena and so never reaches an
offset that a 32-bit truncation would alias, and because a declared device
memory larger than installed RAM has to open at all. It declares 128 GiB but
writes only a few pages, so it costs no more than the small tests.
The C binaries are counted by this convention: a binary that reports its own
N check(s) line contributes N; one that only prints per case ok lines
contributes the number of those lines; one that prints only a final verdict
contributes 1. compute_backend_smoke now reports its own count, which is
why its contribution moved from 1 to 69 without any test being removed. New
binaries should report their own count so that this table can be rebuilt by
reading output rather than by inspection.
| Suite | Collected | Pass | Gated |
|---|---|---|---|
Superproject tests/ (switch data plane, RDMA end to end) |
61 | 36 | 25 |
cnuas/ control plane |
52 | 52 | 0 |
tools/ build, image, and VM lifecycle tooling |
122 | 122 | 0 |
facility/ data-centre twin |
66 | 66 | 0 |
timing/ calibrated virtual-time model |
46 | 46 | 0 |
src/cnuaslink/ fabric switch and client |
18 | 18 | 0 |
| Total pytest | 365 | 340 | 25 |
| CnuasGPU host C smoke binaries (13 binaries, cases counted) | 870 | 870 | 0 |
| CnuasGPU C smoke binaries on the node-less host backend (4 binaries) | 51 | 51 | 0 |
| CnuasGPU host character-device loaded-module configurations | 7 | 7 | 0 |
| CnuasGPU Python tool tests (CnuasIR tools, CnuasCC front end) | 356 | 356 | 0 |
| CnuasGPU device smoke binaries (1 binary, needs a guest with two devices) | 1 | 0 | 1 |
CnuasGPU ggml backend, test-backend-ops MUL_MAT against the CPU reference |
648 | 648 | 0 |
Two suites are run out of band because of their cost rather than their dependencies, and are not part of the totals above.
lib/test/activation_exhaustive.c sweeps all 2^32 binary32 bit
patterns for each of the six activations against a double-precision
oracle, which takes roughly seven minutes per function. It has been run
to completion for AVX2 and, on AVX-512 hardware, for AVX-512. Both pass
with identical error figures: relu bit-exact, sigmoid 3 ulp, tanh
1 ulp, silu worst case 1.5 percent of the conformance tolerance,
gelu within tolerance.
The ggml row is llama.cpp's own differential test, which runs each
operation on CnuasGPU and on the CPU reference and compares. It was run
against llama.cpp master and against the revision Ollama pins, with the
same result on both. Separately, greedy decoding of a GGUF model under
llama-cli produces byte-identical output whether the matrix multiplies
run on the CPU backend or on CnuasGPU.
An end to end run under Ollama is recorded here as a manual result rather
than as a counted case, because it depends on a released third party
binary and a downloaded model. Ollama 0.32.9, which ships ggml 0.19.0,
was started with GGML_BACKEND_PATH pointing at libggml-cnuas.so and
with no Cnuas variable set anywhere. qwen3:0.6b answers correctly, and
41 matrix multiplications were measured executing on CnuasGPU during
prompt processing, at 1024x510x1024 and 1024x510x3072. The first attempt
failed, and the reason is recorded in CnuasGPU_SoftGPU.md section
3.2.1: Ollama passes only GGML_BACKEND_PATH, PATH and
LD_LIBRARY_PATH to its runner, so CNUAS_DEVICE_BACKEND=host never
arrived. The device layer now selects the host arena automatically when
no character device node is present, which is what makes the run above
need no configuration.
Reproduce with:
pytest tests/ # superproject
(cd cnuas && PYTHONPATH=src pytest) # control plane
(cd tools && PYTHONPATH=src pytest) # tooling
(cd facility && PYTHONPATH=src pytest) # facility twin
(cd src/cnuaslink && make -C switch && pytest switch/tests cli/tests)
Build the switch first
The 35 switch data-plane cases spawn src/cnuasswitch/switch/build/cnuas-vswitchd.
If that binary is absent the fixture skips them with a build hint rather
than failing, so the counts above drop to 2 passed and 59 skipped. Run
make -C src/cnuasswitch/switch first to exercise the data plane.
1. CnuasSwitch, virtual top-of-rack switch¶
See CnuasSwitch datasheet for the part level specification.
| Feature | Test | Count | Status | Gate |
|---|---|---|---|---|
| Ethernet and InfiniBand frame classification, auto mode, runt drop | tests/test_classification.py |
3 | Pass | built cnuas-vswitchd |
| MAC learning, unicast forward, flood, ageing | tests/test_eth_forwarding.py |
4 | Pass | built cnuas-vswitchd |
| InfiniBand DLID routing through the linear forwarding table | tests/test_ib_forwarding.py |
2 | Pass | built cnuas-vswitchd |
| Data centre bridging: priority flow control, enhanced transmission selection, explicit congestion notification | tests/test_dcb.py |
5 | Pass | built cnuas-vswitchd |
| Management socket API: ports, forwarding database, linear forwarding table, priority flow control, telemetry, error handling | tests/test_mgmt_api.py |
21 | Pass | built cnuas-vswitchd |
| In-switch subnet manager, subnet administration path records | none | 0 | None | see roadmap Epic 1 |
2. CnuasNIC, RDMA network interface¶
Specified in the CnuasNIC datasheet, with the software split across the kernel modules and verbs provider sheets.
| Feature | Test | Count | Status | Gate |
|---|---|---|---|---|
| RoCEv2 end to end: reliable connection pingpong, transmit and receive paths, completion queue entries | tests/test_rocev2_e2e.py |
15 | Gated | CNUAS_E2E=1 plus vm-a and vm-b |
| Native InfiniBand end to end pingpong | tests/test_ib_e2e.py |
4 | Gated | CNUAS_E2E=1 plus vm-a and vm-b |
| NCCL build and link against the Cnuas verbs provider | tests/test_nccl_link.py |
7 | 1 Pass, 6 Gated | CUDA toolkit and an Nvidia GPU on the host |
Kernel modules cnuas_net.ko and cnuas_ib.ko |
covered indirectly by the two end-to-end gates | 0 | Gated | as above |
| DMA-BUF MR import and CnuasGPU-backed SEND/RECV, RDMA READ/WRITE and atomics | src/cnuasgpu/peermem device smoke |
2 | Gated | guest with CnuasGPU and CnuasNIC devices plus DMA-BUF verbs support |
CnuasNIC carries no tests inside its own repository. All of its validation lives in the superproject suite, because a meaningful test needs the switch, two guest virtual machines, and the verbs provider present at once.
3. CnuasGPU, virtual GPU and runtime¶
Specified in the CnuasGPU datasheet, with the host libraries covered by the CnuasRT and CnuasDev sheets.
| Feature | Test | Count | Status | Gate |
|---|---|---|---|---|
| Compute backend selection, scalar, AVX2, and AVX-512 dispatch | lib/test/compute_backend_smoke |
1 | Pass | none, runs on the host |
| Shared-object backend loading through the dlopen loader | lib/test/compute_so_smoke |
1 | Pass | none, runs on the host |
| CnuasIR object header parse and every rejection path | lib/test/cnuasir_object_smoke |
26 | Pass | none, runs on the host |
CnuasIR tools, object format, subset decoder cross checked against objdump, and the three commands |
tools/test/test_cnuasir_tools.py |
230 | Pass | none; the objdump group skips without a RISC-V toolchain |
| CnuasCC front end, lexer, precedence, the conversions the checker inserts, every rejection with its message, and the driver exit codes | tools/test/test_cnuascc.py |
113 | Pass | none, runs on the host |
| CnuasBLAS column major convention, level 1, level 2, level 3 and every refusal | lib/test/cnuasblas_host_smoke |
39 | Pass | none, runs on the host |
| CnuasLink wire format, golden byte layout, round trip, and every rejection | lib/test/cnuaslink_frame_smoke |
45 | Pass | none, runs on the host |
| CnuasCCL bootstrap over four threaded ranks, membership refusals, and the collectives | lib/test/cnuasccl_smoke |
71 | Pass | none, runs on the host |
| CnuasIR interpreter, every instruction class, the four shipped kernels, and the encoders cross checked against the assembler | lib/test/cnuasir_interp_smoke |
242 | Pass | none; the assembler group skips without a RISC-V toolchain |
| Kernel module load, launch argument checks, and a launch carried through the interpreter | lib/test/cnuasrt_module_smoke |
17 | Pass | none, runs on the host |
| CnuasBLAS against real device memory | lib/test/cnuasblas_smoke |
39 | Gated | a cnuasgpu device in a guest |
| Runtime API: allocate, copy, launch, synchronise | lib/test/cnuasrt_smoke |
1 | Gated | a cnuasgpu device in a guest |
| Single-precision general matrix multiply through the tensor path | lib/test/sgemm_smoke |
1 | Gated | a cnuasgpu device in a guest |
| Device discovery and initialisation | lib/test/compute_smoke |
1 | Gated | a cnuasgpu device in a guest |
| Multi-GPU visibility and peer copy | lib/test/multi_gpu_smoke |
1 | Gated | two or more cnuasgpu devices |
| Kernel module load, ioctl surface, doorbell | driver/test/cnuasgpu_smoke.c |
1 | Gated | a guest with cnuasgpu.ko |
| CnuasLink send and receive from the driver | driver/test/cnuasgpu_link_test.c, driver/test/cnuasgpu_link_recv.c |
2 | Gated | a guest with two endpoints |
| Peer-memory UAPI layout, ioctl numbering, export validation and userspace ownership/error paths | peermem/test/peermem_abi_smoke, peermem/test/peermem_api_smoke |
2 | Pass | none |
| Allocation export lifetime and GPU-memory RDMA transfer | peermem device smoke |
2 | Gated | guest with CnuasGPU and CnuasNIC devices |
| Host-character-device UAPI layout, flags, bounds and validators | lib/test/cnuasgpu_host_abi_smoke |
134 | Pass | none |
| Host-device selection, capability translation, pinned/export/async ownership and error paths | lib/test/cnuasdev_hostdev_fake_smoke |
87 | Pass | none |
Loaded cnuasgpu_host.ko: zero-copy coherence, pinned quota, DMA-BUF lifetime, async ordering/completions/errors, close and unload |
lib/test/cnuasgpu_host_device_smoke, scripts/run-hostdev-vm.sh |
7 configurations | Pass | QEMU test VM; no PCI device required |
The host tests above run as a set with make -C lib check in
src/cnuasgpu, which treats exit code 77 as a skip and fails on anything else.
The gated smoke binaries stop with a clear message rather than a crash, for
example FAIL cnuasInit(0) -> no CnuasGPU device found, which makes an
absent device easy to tell apart from a genuine regression.
4. CnuasLink, GPU fabric interconnect¶
See CnuasLink datasheet for the part level specification.
| Feature | Test | Count | Status | Gate |
|---|---|---|---|---|
| Management version handshake | switch/tests/test_cnuaslink_phase1.py |
1 | Pass | built cnuasgpu-link-switchd |
| Discovery learns the GPU forwarding database | switch/tests/test_cnuaslink_phase1.py |
1 | Pass | built cnuasgpu-link-switchd |
| Unicast frame forwarding between endpoints | switch/tests/test_cnuaslink_phase1.py |
1 | Pass | built cnuasgpu-link-switchd |
| Broadcast reaches peers and is not looped back to the sender | switch/tests/test_cnuaslink_phase1.py |
2 | Pass | built cnuasgpu-link-switchd |
| Port status, peer identifiers, and frame counters | switch/tests/test_cnuaslink_phase1.py |
1 | Pass | built cnuasgpu-link-switchd |
| A disabled port drops traffic and the drop is counted | switch/tests/test_cnuaslink_phase1.py |
2 | Pass | built cnuasgpu-link-switchd |
| Command line client: version, port status, forwarding database, telemetry, link up and down, bad socket handling | cli/tests/test_cli_e2e.py |
10 | Pass | built cnuasgpu-link-switchd |
Both files were standalone scripts that defined main() and no test
functions, so pytest collected zero cases from them and reported success. They
are now real pytest modules: 18 collected, 18 passing, and they skip cleanly
with a build hint when the daemon binary is absent rather than failing.
5. Control plane and platform software¶
Specified across the control plane, tools, and management tools sheets.
| Feature | Test | Count | Status | Gate |
|---|---|---|---|---|
| Component adapters: switch, NIC, GPU, link, system inventory | cnuas/tests/test_adapters.py |
23 | Pass | none |
| REST API surface | cnuas/tests/test_api.py |
17 | Pass | none |
cnuas command line |
cnuas/tests/test_cli.py |
7 | Pass | none |
| Service lifecycle | cnuas/tests/test_service.py |
5 | Pass | none |
| Build, package, image, virtual machine lifecycle, release, deploy tooling | tools/tests/ (16 files) |
122 | Pass | none |
6. Facility twin¶
See facility datasheet for the part level specification.
| Feature | Test | Count | Status | Gate |
|---|---|---|---|---|
| Campus and data-hall layout generation | facility/tests/test_layout.py |
19 | Pass | none |
| Live telemetry bridge | facility/tests/test_live.py |
19 | Pass | none |
| Rack and facility power model | facility/tests/test_power.py |
8 | Pass | none |
| OpenUSD stage authoring, including custom-configuration read-back | facility/tests/test_usd.py |
14 | Pass | none |
| Custom JSON template, strict loading, automatic plant sizing, layout rejection and CLI workflow | facility/tests/test_config.py |
6 | Pass | none |
7. Calibrated virtual-time and performance model¶
See timing model design for the evidence classes and calibration claim boundary.
| Feature | Test | Count | Status | Gate |
|---|---|---|---|---|
| Component costs, bandwidth, packet and operation rates, serial and overlapping resources | timing/tests/test_model.py |
26 | Pass | none |
| Deterministic integer-picosecond scheduling, parallel lanes, queue contention, replay, pipeline latency and throughput | timing/tests/test_model.py |
included above | Pass | none |
| Target-specific affine fitting and held-out MAE, RMSE, MAPE and maximum error | timing/tests/test_calibration.py |
9 | Pass | none |
Strict profile serialization, Cnuas reference profile, QEMU icount mapping and CLI workflows |
timing/tests/test_profiles_qemu_cli.py |
11 | Pass | none |
The supplied profile is analytic and uncalibrated. These tests prove the model's arithmetic, determinism, serialization, fitting, and error reporting. They do not prove prediction accuracy for physical hardware. That requires a named calibration target and separate held-out measurements.
7. Baseboard management and rack management¶
See CnuasBMC datasheet for the part level specification.
| Feature | Test | Count | Status | Gate |
|---|---|---|---|---|
| OpenBMC images for sled, rack management controller, and top of rack | build gate only, three images produced | 0 | Manual | Yocto build |
| ORV3 PSU and BBU firmware: Modbus RTU, register maps, faults and battery state | bmc/firmware/orv3/test/test_main.c, through make test |
177 | Pass | none |
| Rack management controller: RTU transport, shelf discovery and D-Bus publication | bmc/rmc/test/test_rmc.c, test_dbus.c, through make test |
45 | Pass | none |
| Sled OpenBMC QMP bridge: power lifecycle, GPIO and framed transport | tools/tests/sledbmc/ |
37 | Pass | none |
| Cnuas BMC control-plane adapters and REST resources | cnuas/tests/test_sled.py, test_api.py, test_service.py |
52 | Pass | test extra |
| UART front-panel MCU: framing, CRC, LEDs and button events | bmc/firmware/frontpanel/test/test_frontpanel.c, through make test |
98 | Pass | none |
| IPMI cipher-suite 17 and serial over LAN against a booted sled BMC | scripted integration procedure | 0 | Manual | OpenBMC and QEMU |
8. Accelerator software stack, state of each part¶
These appear in CnuasGPU Design as the intended software stack. Some are now implemented and appear with their tests in section 3 above; the rest have no code and therefore no tests. They are listed together so that the gap is explicit rather than implied. Each has a datasheet recording the specified design and what is built: cnuascc, CnuasIR, CnuasCCL, CnuasSHMEM, and the mathematics libraries.
| Component | Models | State |
|---|---|---|
cnuascc |
nvcc |
Front end implemented, no code generation |
| CnuasIR toolchain | PTX | Tools and interpreter implemented |
| CnuasCCL | NCCL | v0.1 implemented, host channel |
| CnuasSHMEM | NVSHMEM | Design only |
| CnuasBLAS | cuBLAS | v0.1 implemented over the shipped kernels, see section 3 |
| CnuasDNN | cuDNN | Design only |
| CnuasFFT | cuFFT | Design only |
| CnuasSPARSE | cuSPARSE | Design only |
| CnuasSOLVER | cuSOLVER | Design only |
cnuas-dcgmi, cnuas-dcgm |
dcgmi, dcgm | Design only |
cnuas-prof |
Nsight Compute | Design only |
9. Keeping this page current¶
This page is regenerated by hand after a change to any test suite. The rule is
that a roadmap entry may move to available only when a row here names a test and
records a pass. When adding a component, add its row here in the same change,
even if the only accurate entry is None.