Distributed SystemsRDMALLM ServingNetworkingNSDI 2026

BURST Guide: High-Performance Soft-RDMA for LLM Clusters

The Network Interoperability Crisis in Hyperscale AI Clusters

The unprecedented expansion of large language model (LLM) training and inference has elevated high-performance networking from a supporting infrastructure detail to the primary architectural bottleneck. Modern frontier AI workloads—ranging from multi-thousand-GPU training runs with Megatron-LM and DeepSpeed to disaggregated prefill-decode serving engines (such as Mooncake, SGLang, and vLLM)—depend fundamentally on Remote Direct Memory Access (RDMA) over Converged Ethernet (RoCEv2) and InfiniBand.

RDMA provides two foundational capabilities that traditional TCP/IP networking cannot match:

  1. Kernel Bypass: Applications communicate directly with the Network Interface Card (NIC) from user space, eliminating operating system context switches, buffer copies, and interrupt handling overhead.
  2. Zero-Copy Remote Memory Access: NIC hardware reads and writes directly into remote host memory (or GPU High-Bandwidth Memory via GPUDirect RDMA) without involving the remote host CPU.

These properties make RDMA indispensable for transferring gigabytes of Key-Value (KV) cache tensors across disaggregated inference nodes and synchronizing multi-terabyte model weights during distributed training.

However, as hyperscale operators scale AI infrastructure across heterogeneous cloud regions and multi-tenant datacenters, the industry has encountered a severe hardware bottleneck: The Strict Hardware Dependency of RDMA NICs (RNICs).

+---------------------------------------------------------------------------------------------------------+ | THE RNIC FRAGILITY & INTEROPERABILITY CRISIS | +---------------------------------------------------------------------------------------------------------+ | | | HARDWARE RDMA NIC (RNIC) RIGIDITY: | | - Strict vendor protocol dependencies (Mellanox/NVIDIA ConnectX, Broadcom Thor, Intel) | | - Proprietary congestion control implementations (DCQCN, TIMELY, Swift) | | - Zero tolerance for non-compliant packets or malformed transport headers | | | | SYSTEM FAILURE MODES IN PRODUCTION: | | 1. Catastrophic TCP/IP Fallback: Corrupted or mismatched packets force RNICs to abort RDMA. | | Fallback to kernel TCP/IP causes a 4x to 8x throughput collapse and 300% latency inflation. | | 2. Cloud Heterogeneity Lock-In: Commodity Ethernet NICs (lacking hardware RoCEv2 engines) cannot | | participate in disaggregated KV-cache clusters or distributed training pools. | | 3. Legacy Soft-RDMA (Linux RXE) Failure: Kernel-based SoftRoCE burns up to 32 CPU cores on packet | | headers, capping out at under 40 Gbps on 400G physical links due to kernel lock contention. | +---------------------------------------------------------------------------------------------------------+

The Failure of Conventional Software RDMA (SoftRoCE / RXE)

To escape vendor lock-in and enable RDMA on standard Ethernet NICs, the open-source community created Soft-RDMA (most prominently the Linux kernel’s rdma_rxe driver). RXE emulates the RDMA transport engine inside the operating system kernel, parsing RoCEv2 headers and executing memory copies via software.

In theory, Soft-RDMA allows any server with a commodity Ethernet card to run standard RDMA software. In practice, when deployed on modern 100G, 200G, and 400G datacenter fabrics, Linux kernel Soft-RDMA collapses:

  • Prohibitive Host CPU Overhead: Kernel RXE relies on kernel worker threads, software interrupts (softirqs), and socket buffers (sk_buff). On a 400G network, processing packet headers in software exhausts dozens of CPU cores, stealing compute cycles from application threads.
  • Lock Contention and Cache Thrashing: Multi-core packet handling inside the kernel network stack triggers severe spinlock contention across queue pairs (QPs) and completion queues (CQs).
  • Throughput Saturation at Low Speeds: Benchmarks reveal that kernel RXE struggles to exceed 30 to 45 Gbps—utilizing less than 10% of a 400G physical link.

To eliminate this barrier, a research team from ByteDance and university collaborators presented BURST at USENIX NSDI 2026. BURST is a high-performance, user-space software RDMA stack designed specifically for 100G–?00G networks.

Operating entirely outside the Linux kernel, BURST combines a lock-free DPDK data plane, hardware-assisted memory copying via Intel Data Streaming Accelerator (Intel DSA), and a kernel-bypass connection manager. BURST reaches 98.7% line-rate bandwidth on 400G NICs, accelerates LLM inference KV-cache transfer latency to 25.2% of TCP’s latency (a near 4x improvement), and maintains 100% binary compatibility with standard libibverbs applications.

Architectural Foundations of BURST

BURST abandons in-kernel packet processing entirely, architecting Soft-RDMA as a dedicated, user-space high-throughput daemon that interfaces directly with physical NIC queues and hardware memory copy engines.

+---------------------------------------------------------------------------------------------------------+ | BURST SYSTEM ARCHITECTURE | +---------------------------------------------------------------------------------------------------------+ | | | +-------------------------------------------------------------------------------------------------+ | | | UNMODIFIED AI APPLICATIONS (PyTorch DDP, NCCL, vLLM / SGLang KV Transfer, Mooncake Cache Store) | | | +-------------------------------------------------------------------------------------------------+ | | | Standard libibverbs / RDMA Verbs API Calls | | v | | +-------------------------------------------------------------------------------------------------+ | | | BURST USER-SPACE PROVIDER PLUGIN (libburst.so) | | | | - Shared-Memory Zero-Copy Ring Buffers for Work Requests (WR) & Completion Queues (CQ) | | | +-------------------------------------------------------------------------------------------------+ | | | Inter-Process Shared Memory (IPC) | | v | | +-------------------------------------------------------------------------------------------------+ | | | BURST HIGH-PERFORMANCE USER-SPACE ENGINE (burst-core daemon) | | | | | | | | +------------------------------------+ +-----------------------------------------------+ | | | | | 1. Lock-Free DPDK Data Plane | | 2. Hardware Memory Copy Offload | | | | | | - Poll-Mode Driver (PMD) | | - Intel DSA (Data Streaming Accelerator) | | | | | | - Core-pinned packet dispatch ring | | - Asynchronous zero-copy memory movement | | | | | | - Zero kernel interrupt overhead | | - Bypasses host CPU core copy loops | | | | | +------------------------------------+ +-----------------------------------------------+ | | | | \ / | | | | v v | | | | +---------------------------------------------------------------------------------------+ | | | | | 3. Kernel-Bypass Connection Manager (B-CM) | | | | | | - User-space state machine for Queue Pair (QP) handshakes | | | | | | - 12x faster connection setup speed vs. native kernel rdma_cm | | | | | +---------------------------------------------------------------------------------------+ | | | +-------------------------------------------------------------------------------------------------+ | | | Direct Hardware Access | | v | | +-------------------------------------------------------------------------------------------------+ | | | HARDWARE LAYER: 400G Commodity Ethernet NICs (PCIe Gen5) + Intel DSA Hardware Accelerator | | | +-------------------------------------------------------------------------------------------------+ | +---------------------------------------------------------------------------------------------------------+

The Three Pillar Innovations

BURST is engineered around three core architectural breakthroughs:

  1. Lock-Free DPDK Data Plane: By integrating the Data Plane Development Kit (DPDK) Poll-Mode Driver (PMD), BURST reads raw Ethernet frames directly from NIC ring buffers into user-space hugepages. Packet parsing, RoCEv2 encapsulation/decapsulation, and sequence acknowledgment execute on dedicated, core-pinned polling threads without acquiring kernel mutexes or triggering OS context switches.
  2. Hardware Memory Copy Offloading via Intel DSA: In high-speed RDMA READ/WRITE operations, moving multi-megabyte payloads between network buffers and user registered memory (MR) typically consumes massive CPU memory bandwidth. BURST offloads these memory copy operations to on-die Intel Data Streaming Accelerator (Intel DSA) engines. DSA executes descriptor-based asynchronous DMAs in hardware, freeing host CPU cores to run application logic while saturating 400G line rate.
  3. Kernel-Bypass Connection Management (B-CM): Native RDMA connection management (rdma_cm) executes via the Linux kernel network stack, incurring multi-millisecond handshake delays and serializing socket setups. In dynamic LLM clusters—where thousands of transient connections are established between disaggregated prefill and decode instances—rdma_cm becomes a major bottleneck. BURST replaces this with a lightweight, user-space connection protocol that accelerates connection establishment by 12x.

DPDK Data Plane and Lock-Free Ring Queues

To achieve 400 Gbps line-rate throughput in software, every nanosecond of packet processing must be accounted for. At 400G with a standard MTU of 1,500 bytes, the network receives approximately 33 million packets per second—allowing a budget of only 30 nanoseconds per packet.

Kernel SoftRoCE fails because traversing Linux network layers (netif_receive_skb, socket queues, context switching) takes over 800 nanoseconds per packet.

Traditional Kernel SoftRoCE (RXE) Data Path: Physical NIC ---> [ Hardware Interrupt ] ---> [ Linux Kernel softirq ] ---> [ sk_buff Allocation ] ---> [ RXE Protocol Parser ] ---> [ Kernel Spinlock ] ---> [ Copy to User Space ] Total Overhead: ~850 ns per packet (Caps at ~35 Gbps, high CPU burn) BURST Lock-Free DPDK Data Path: Physical NIC ---> [ DPDK Poll-Mode Driver ] ---> [ Pre-allocated User Hugepage ] ---> [ Lock-Free Queue Pair Ring ] ---> [ Intel DSA Hardware DMA to App Memory ] Total Overhead: < 22 ns per packet (Reaches 394.8 Gbps on 400G link, < 2 CPU cores)

The Lock-Free Queue Pair State Machine

BURST decouples application threads from network I/O threads using lock-free Single-Producer Single-Consumer (SPSC) ring buffers:

  1. Work Request Dispatching: When an application calls ibv_post_send or ibv_post_recv, the user-space provider plugin writes a compact 64-byte descriptor into a shared-memory ring buffer. The application thread returns immediately without waiting for network completion.
  2. Polling Engine Batching: Dedicated BURST worker threads poll the ring buffer in batches (e.g., 32 or 64 descriptors at a time). The worker threads map descriptors directly to DPDK transmission descriptors, batching RoCEv2 header construction.
  3. Credit-Based Flow Control: To prevent buffer overruns without expensive lock synchronization, BURST implements a hardware-friendly credit-based sliding window algorithm across queue pairs. Send credits are piggybacked on incoming acknowledgment (ACK) packets, eliminating standalone flow control traffic.

Intel DSA Acceleration: Zero-Copy Memory Offloading

In standard RDMA NICs, the network card hardware contains an on-chip Direct Memory Access (DMA) engine that reads directly from host physical RAM. In software RDMA, however, because the commodity Ethernet card only transfers packets to network buffers, the CPU must copy payload bytes from packet buffers into the application’s Registered Memory (MR).

Copying a 4 GB KV cache across memory channels using standard CPU memcpy (even with AVX-512 vector instructions) consumes over 40 GB/s of CPU memory bus bandwidth and saturates multiple physical CPU cores.

+---------------------------------------------------------------------------------------------------------+ | INTEL DSA MEMORY COPY OFFLOADING | +---------------------------------------------------------------------------------------------------------+ | | | 1. APPLICATION REGISTERED MEMORY (vLLM KV Cache Paged Block in DRAM) | | [ Block Page 0 ][ Block Page 1 ][ Block Page 2 ][ Block Page 3 ] ... | | | | 2. BURST PACKET BUFFER (DPDK mbuf pool in User-Space Hugepages) | | [ RoCEv2 Packet 0 ][ RoCEv2 Packet 1 ][ RoCEv2 Packet 2 ][ RoCEv2 Packet 3 ] ... | | | | 3. OFFLOAD WORKFLOW VIA INTEL DSA (Data Streaming Accelerator): | | - CPU Worker prepares 64-byte DSA Batch Descriptor (Source Addr -> Dest Addr) | | - CPU submits descriptor via non-blocking `ENQCMD` instruction (takes < 15 ns) | | - Intel DSA hardware engine executes memory streaming in parallel on PCIe bus | | - CPU immediately proceeds to process the next network packet! | | | | BENEFIT: Reduces CPU utilization by 78%, freeing host cores for LLM inference scheduling! | +---------------------------------------------------------------------------------------------------------+

Exploiting Intel DSA in Software RDMA

Intel Data Streaming Accelerator (DSA) is an on-die hardware accelerator integrated into modern Intel Xeon processors (Sapphire Rapids, Emerald Rapids, Granite Rapids) and compatible architectures. DSA specializes in high-throughput data copy and transformation operations.

BURST interfaces directly with Intel DSA through the user-space idxd driver framework:

  1. When an incoming RDMA WRITE packet arrives, BURST extracts the target virtual address and memory key (R_Key).
  2. Instead of calling CPU copy routines, BURST formats an asynchronous DSA memory copy descriptor.
  3. The descriptor is enqueued into the DSA hardware work queue via the dedicated ENQCMD (Enqueue Command) CPU assembly instruction.
  4. The DSA hardware engine streams data directly between host physical addresses over the memory interconnect.
  5. Once complete, DSA signals completion through a shared memory flag, allowing BURST to post an IBV_WC_SUCCESS completion entry to the application’s completion queue.

By replacing software memory copy loops with hardware-driven DSA streams, BURST slashes CPU core requirements by 78%, allowing a single server to saturate 400 Gbps bandwidth using only 2 to 3 dedicated polling cores.

Step-by-Step Implementation: Building a High-Performance Soft-RDMA Bridge

To understand the mechanics of BURST, let us implement a functional Python architecture model demonstrating:

  1. User-space Queue Pair (QP) descriptor queues.
  2. Simulated lock-free packet processing and header encapsulation.
  3. Memory registration and hardware copy offload abstraction.

Step 1: Memory Registration & Queue Pair Structures

# burst_rdma_types.py import time from dataclasses import dataclass, field from enum import Enum from typing import Dict, List, Optional class WorkRequestOpcode(Enum): RDMA_WRITE = 1 RDMA_READ = 2 SEND = 3 RECV = 4 @dataclass class MemoryRegion: mr_id: int addr: int length: int rkey: int buffer_ref: bytearray @dataclass class WorkRequest: wr_id: int opcode: WorkRequestOpcode local_addr: int length: int remote_addr: Optional[int] = None rkey: Optional[int] = None status: str = "PENDING" completed_at: Optional[float] = None @dataclass class CompletionQueueEntry: wr_id: int status: str # "IBV_WC_SUCCESS" or error byte_len: int timestamp: float = field(default_factory=time.time)

Step 2: Lock-Free Shared-Memory Ring Buffer

# burst_ring_buffer.py from collections import deque from typing import Optional, List from burst_rdma_types import WorkRequest, CompletionQueueEntry class LockFreeQueuePair: """ Simulates a lock-free Single-Producer Single-Consumer (SPSC) ring buffer connecting application threads to the BURST DPDK engine. """ def __init__(self, qp_id: int, capacity: int = 1024): self.qp_id = qp_id self.capacity = capacity self.send_ring = deque(maxlen=capacity) self.recv_ring = deque(maxlen=capacity) self.completion_queue = deque(maxlen=capacity) def post_send(self, wr: WorkRequest) -> bool: if len(self.send_ring) >= self.capacity: return False # Queue full self.send_ring.append(wr) return True def post_recv(self, wr: WorkRequest) -> bool: if len(self.recv_ring) >= self.capacity: return False self.recv_ring.append(wr) return True def poll_cq(self, max_entries: int = 16) -> List[CompletionQueueEntry]: results = [] while self.completion_queue and len(results) < max_entries: results.append(self.completion_queue.popleft()) return results

Step 3: The BURST User-Space DPDK & DSA Engine

# burst_engine.py import time from typing import Dict, List from burst_rdma_types import WorkRequest, WorkRequestOpcode, CompletionQueueEntry, MemoryRegion from burst_ring_buffer import LockFreeQueuePair class MockIntelDSAAccelerator: """ Simulates on-die Intel Data Streaming Accelerator (DSA) executing asynchronous hardware memory copies without host CPU load. """ def async_copy(self, src_bytes: bytearray, dst_bytes: bytearray, length: int) -> float: # DSA copies memory in hardware; CPU merely incurs ~15ns enqueue overhead dst_bytes[:length] = src_bytes[:length] return 0.000015 # 15 microseconds simulated transfer for payload class BurstEngine: """ The BURST user-space engine that polls Queue Pairs, encapsulates RoCEv2 headers, and coordinates hardware DSA copies to achieve near-line-rate bandwidth. """ def __init__(self): self.qps: Dict[int, LockFreeQueuePair] = {} self.registered_mrs: Dict[int, MemoryRegion] = {} self.dsa = MockIntelDSAAccelerator() self.total_bytes_transferred = 0 def register_mr(self, mr_id: int, buffer_ref: bytearray) -> MemoryRegion: mr = MemoryRegion( mr_id=mr_id, addr=id(buffer_ref), length=len(buffer_ref), rkey=mr_id * 1000 + 42, buffer_ref=buffer_ref ) self.registered_mrs[mr_id] = mr return mr def create_qp(self, qp_id: int) -> LockFreeQueuePair: qp = LockFreeQueuePair(qp_id) self.qps[qp_id] = qp return qp def process_pending_requests(self, batch_size: int = 32) -> int: """ Poll-Mode Driver (PMD) processing loop. Batches Work Requests, performs zero-copy RoCEv2 transmission, and generates CQEs. """ processed_count = 0 for qp_id, qp in self.qps.items(): batch: List[WorkRequest] = [] while qp.send_ring and len(batch) < batch_size: batch.append(qp.send_ring.popleft()) for wr in batch: if wr.opcode == WorkRequestOpcode.RDMA_WRITE: # Execute RDMA WRITE via Intel DSA offload mr = self.registered_mrs.get(wr.wr_id) if mr: # Offload memory movement to DSA hardware dummy_dest = bytearray(wr.length) self.dsa.async_copy(mr.buffer_ref, dummy_dest, wr.length) self.total_bytes_transferred += wr.length # Post completion entry cqe = CompletionQueueEntry( wr_id=wr.wr_id, status="IBV_WC_SUCCESS", byte_len=wr.length ) qp.completion_queue.append(cqe) processed_count += 1 return processed_count

Empirical Benchmarks: Evaluating BURST

The authors of BURST evaluated the stack on extensive production testbeds featuring 400G Ethernet NICs (NVIDIA BlueField-3 in non-RoCE mode, Broadcom NetXtreme-E 400G, and Intel E810 100G) connected across high-speed enterprise switches.

The benchmarks compared BURST against:

  1. Hardware RDMA (Native RoCEv2 on ConnectX-7): Hardware gold-standard baseline.
  2. Kernel SoftRoCE (Linux RXE driver): Upstream in-kernel software RDMA implementation.
  3. Standard Linux Kernel TCP/IP: Default fallback networking stack.
Evaluation MetricKernel TCP/IPKernel SoftRoCE (RXE)BURST (NSDI 2026)Hardware RoCEv2 (RNIC)
Bandwidth (400G Link)185 Gbps (46.2% line)62 Gbps (15.5% line)394.8 Gbps (98.7% line)398.2 Gbps (99.5% line)
Throughput Speedup vs RXE3.0x1.0x (Baseline)6.3x Higher Throughput6.4x
CPU Core Utilization (400G)18 Cores (High)32 Cores (Saturated)2.5 Cores (DSA Offloaded)0 Cores (Hardware ASIC)
LLM KV-Cache Transfer Latency12.8 ms (Baseline)38.4 ms (Extreme jitter)3.2 ms (25.2% of TCP)2.8 ms
Connection Setup Latency4.8 ms (TCP Handshake)18.2 ms (Kernel rdma_cm)1.5 ms (12x faster B-CM)1.8 ms
App Code CompatibilityRequires SocketsRDMA Verbs100% RDMA Verbs Binary100% RDMA Verbs

Key Takeaways from Empirical Results

  1. Near-Hardware Line-Rate Performance on Commodity NICs: BURST achieves 394.8 Gbps on a 400G physical link (98.7% line-rate utilization), coming within 1% of native hardware RNICs. Compared to Linux kernel RXE, BURST provides a 3.2x to 6.3x throughput leap.
  2. Transforming Distributed LLM Inference Latency: In disaggregated prefill-decode architectures (such as Mooncake and Splitwise), transferring multi-gigabyte KV cache states across nodes over TCP incurs severe latency penalties. BURST slashes KV transfer latency down to 25.2% of TCP’s latency, reducing end-to-end Time-to-First-Token (TTFT) by nearly 4x without purchasing proprietary RNICs.
  3. 78% Reduction in Host CPU Burden: By offloading memory copies to Intel DSA engines and utilizing lock-free DPDK polling, BURST eliminates the CPU starvation that crippled earlier software RDMA attempts.

Integrating BURST Principles with AIFetchly Enterprise Stack

In distributed enterprise AI architectures, model inference, web extraction, and real-time data pipelines operate across diverse cloud regions and heterogeneous server fleets:

  1. Disaggregated Web Extraction & Vector Indexing: High-throughput automated web scraping pipelines continuously harvest unstructured text, converting pages into vector embeddings. With BURST, commodity cloud servers without expensive specialized RNICs can transfer streaming embeddings and raw document buffers across distributed cluster nodes at near-line rate.
  2. Resilient Cross-Datacenter API Integration: Modern enterprise systems running multi-tenant AI workloads rely on custom API integrations and real-time data fetching. In heterogeneous cloud environments where hardware RDMA is unavailable or fails due to network packet corruption, BURST prevents catastrophic TCP fallback, keeping data ingestion pipelines operating at high throughput.
  3. Drastic Infrastructure Capital Savings: Provisioning dedicated hardware RNICs across every node in a secondary or edge datacenter is prohibitively expensive. BURST enables organizations to deploy high-throughput LLM serving and distributed memory pooling on existing commodity Ethernet infrastructure, cutting networking hardware expenditure by 40% to 60%. To see how optimized infrastructure translates to operational efficiency, review our transparent pricing tiers and explore our comprehensive developer documentation. Discover how the AIFetchly platform unifies web intelligence, vector search, and scalable AI inference into a cost-effective enterprise stack.

Future Horizons: The Democratization of AI Networking

BURST marks a turning point in distributed systems engineering:

  • Universal Multi-Cloud RDMA Fabrics: As organizations deploy multi-cloud AI clusters spanning AWS, Google Cloud, Azure, and on-premises colocation facilities, BURST provides a uniform, hardware-agnostic communication abstraction.
  • SmartNIC and DPU Integration: Porting BURST’s lock-free pipeline to emerging Data Processing Units (DPUs) and ARM-based SmartNICs will allow line-rate Soft-RDMA execution with zero host CPU intervention.
  • Fault-Tolerant Hybrid Transport: Future evolutions of BURST will support seamless multi-path transport, automatically switching traffic between hardware RoCEv2, user-space Soft-RDMA, and optimized UDP without terminating active LLM generation sessions.

Key Technical Takeaways

  • Eliminating the RNIC Hardware Bottleneck: BURST proves that software-based RDMA can saturate 400G network links, freeing AI infrastructure from strict hardware vendor dependencies and fragile TCP fallback penalties.
  • DPDK Kernel-Bypass Architecture: By running entirely in user space with lock-free polling queues, BURST drops per-packet handling latency from 850 nanoseconds to under 22 nanoseconds.
  • Intel DSA Hardware Copy Acceleration: Offloading memory movements from CPU cores to dedicated on-die Intel DSA silicon engines slashes CPU overhead by 78%, delivering line-rate bandwidth with only 2.5 CPU cores.
  • 4x Acceleration for Disaggregated LLM Serving: In production KV-cache transfers across distributed prefill-decode nodes, BURST cuts network transfer latency to 25.2% of TCP, matching hardware RoCEv2 performance on commodity Ethernet.