Cnuas Virtual AI HPC Infrastructure
Cnuas (pronounced Knoo-us, the Irish word for a cluster) is a fully software-emulated AI and HPC infrastructure. Virtual GPUs, virtual GPU-peer fabric, virtual InfiniBand and RoCE switches and virtual RDMA NICs, all built from scratch in C around QEMU, so unmodified Linux guests run real distributed workloads on simulated silicon.
cnuas is the unified control plane. A single command surface drives the switch
fabric, the GPU fabric, GPUs, RDMA NICs and VM lifecycle, and the same service layer is
exposed as a REST API for automation. Every CLI capability is also an API endpoint.
$ cnuas system health ┏━━━━━━━━━━━┳━━━━━━━━━━━┓ ┃ Component ┃ Available ┃ ┡━━━━━━━━━━━╇━━━━━━━━━━━┩ │ hiswitch │ yes │ │ hilink │ yes │ │ higpu │ yes │ │ hinic │ yes │ │ vm │ yes │ └───────────┴───────────┘ $ cnuas switch set-mode 2 STRICT_IB { "status": "ok", "port": 2, "mode": "STRICT_IB" } $ cnuas vm up vm-a --gpus 1 vm-a running, HiNIC + HiGPU attached
Cnuas does not use host-based shortcuts like Linux bridges or Soft-RoCE. Instead it builds custom QEMU PCIe devices and their corresponding Linux kernel drivers, so the guest sees a real RDMA HCA and a real GPU on its PCIe bus. From the application point of view it is talking to hardware, even though every byte is moving through software.
hicain_net.ko, hicain_ib.ko) probe the device and register with ib_corelibibverbs works unmodified, so ibv_rc_pingpong, perftest and NCCL all runA student cannot bring home a $200,000 InfiniBand switch or a 4x H100 server. Existing Linux virt approaches (bridges, Soft-RoCE) hide the hardware semantics that matter, such as DCB, ECN marking, IB LIDs and GPU peer DMA. Cnuas keeps those semantics in software so what you learn here transfers directly to NVIDIA, AMD or Intel production hardware.
Cnuas is built from four hardware components, each its own open-source project under PacketFive, plus a software stack that mirrors the NVIDIA developer experience. Cnuas ties them together as the integration and orchestration layer.
PCIe device model exposing standard BARs, doorbells and DMA queues. The companion
kernel drivers register a netdev and an RDMA HCA. Guest applications use libibverbs
as if it were Mellanox or NVIDIA hardware.
Standalone C daemon acting as a top-of-rack switch. An epoll event loop listens on
per-port UNIX sockets, classifies frames (Ethernet, RoCEv2 or InfiniBand LRH), and
forwards using a per-pipeline forwarding table over a JSON-over-UDS control plane.
QEMU PCIe device with a SIMT execution model, including Streaming Multiprocessors, tensor units, shared memory and HBM-equivalent device memory. It ships a complete NVIDIA-equivalent userland, with HCC (a CUDA equivalent), hi-smi, an hcc compiler and HiCCL (an NCCL equivalent).
Independent GPU-to-GPU interconnect, an NVSwitch equivalent. A VM with a HiGPU but no
HiNIC can still do GPU peer DMA. It uses ivshmem as the fast path for
cross-VM shared memory.
HiGPU ships a complete NVIDIA-equivalent userland, so the same programming model and tooling students use here transfer directly to production GPUs. Each tool uses clean HiCAIN naming and mirrors a familiar NVIDIA counterpart.
HiCAIN Compute, the CUDA-equivalent stack. libhicart.so mirrors the CUDA
Runtime and libhi.so the driver API.
LLVM-backed device compiler that lowers GPU kernels to HiIR, the HiGPU intermediate representation.
nvcc + PTXMulti-GPU collective communications, including AllReduce and Broadcast, running over the HiLink peer fabric with a RoCE fallback.
NCCLOne-sided GPU-to-GPU shared memory for partitioned workloads, using HiLink put and get as the transport.
NVSHMEMDevice management and monitoring CLI. It lists GPUs, reports utilisation and memory, and exposes realistic telemetry.
nvidia-smiFleet health checks and diagnostics, reading per-SM and chip-wide counters for cluster management.
DCGMKernel profiler that surfaces per-SM occupancy and chip-wide performance counters for tuning workloads.
Nsight ComputeHiBLAS, HiDNN, HiFFT, HiSPARSE and HiSOLVER provide accelerated linear algebra, deep-learning and signal-processing primitives.
cuBLAScuDNNcuFFTExposes HiGPU devices to Docker and other container runtimes for reproducible GPU workloads.
nvidia-container-toolkitConfigure, set up and interact with the entire system from a scriptable CLI or a REST API. Both front ends call the same service layer, so nothing is CLI-only or API-only.
Interactive Rich tables by default, and --json on any
command for scripting. Command groups cover every domain of the rack.
cnuas switch, RoCE and InfiniBand fabric, 18 commandscnuas fabric, HiLink GPU peer fabriccnuas gpu, HiGPU device controlcnuas nic, HiNIC and RDMA device controlcnuas vm, virtual-machine lifecyclecnuas system, inventory, health and versionsA FastAPI service (cnuas api) with OpenAPI docs, so any
language or orchestrator can drive the platform over HTTP.
$ cnuas api --host 0.0.0.0 --port 8080 # OpenAPI docs at http://host:8080/docs $ curl host:8080/api/v1/switch/ports [{ "port": 0, "mode": "AUTO", "link": "up" }, { "port": 2, "mode": "STRICT_IB", "link": "up" }]
The reference deployment follows the Open Compute Project Open Rack v3, the rack design the modern AI buildout was created around. Two 1OU TOR switches (HiSwitch and HiLink) sit at the top of the rack, above a management host and eight 2OU VM blades acting as GPU compute nodes. The U-heights are visual in the emulator, but the layout maps 1:1 to real ORv3 deployments, making the same diagrams useful for teaching, design and real lab build-outs.
The HiSwitch fabric carries every Ethernet frame, including RoCEv2 and RDMA
on UDP 4791, the same path NCCL's IB transport, perftest and
ibv_rc_pingpong flow over. The HiLink fabric is independent
and carries only GPU-to-GPU peer traffic. A guest can use one, the other, or both,
exactly as in a real GPU rack.
Cnuas is an open platform for infrastructure research and hands-on learning, and it is the environment HiCAIN courses run against. Students and researchers run real distributed-training workloads, RDMA microbenchmarks and MPI collectives against the emulated fabric, building the skills modern AI cluster operations demand, on a laptop.
No physical GPUs, NICs or switches required. A multi-node GPU cluster fits on a single workstation.
IB LIDs, GRH and LRH headers, DCB pause frames and ECN marking, all the things that matter for real production debugging.
Open-source, modifiable and reproducible. Ideal for academic networking and HPC research where commercial silicon is a black box.
Cnuas stands on the open-source stack the industry runs on. Compute and devices are emulated with QEMU, the rack follows the Open Compute Project Open Rack v3, and the out-of-band management plane integrates OpenBMC and Renode for realistic Baseboard Management Controller behaviour.
Device and machine emulation
Open Rack v3 reference design
Firmware-accurate virtual BMC
Deterministic BMC simulation
QEMU, Open Compute Project, OpenBMC and Renode, and their logos, are trademarks of their respective owners. The logos are shown to indicate the open technologies Cnuas integrates. Cnuas is not affiliated with or endorsed by these projects. The "Built on OpenBMC" logo is by the OpenBMC Project, used under CC BY 4.0.
Cnuas is a product of PacketFive, the trading name of Packet Five Networks Limited, a research and development company based in Dublin, Ireland. We build open infrastructure for the AI/HPC datacenter, spanning simulation, training and facility security.
Cnuas is developed in the open and is the environment HiCAIN training courses run against. For commercial enquiries, research collaboration or support, get in touch with the team.
Packet Five Networks Limited, 51 Bracken Road, Sandyford, Dublin D18 CV48, Ireland. CRO 764119.