priminghybrid-ssmmamba-2transformer-distillationkv-cachellm-inferencemodel-conversiontutorial

Priming Guide: Hybrid Mamba SSMs From Pre-Trained LLMs

October 13, 2026

Priming Architecture: Hybrid State-Space Models From Pre-Trained Transformers with Layer Transmutation and Weight Repurposing

The 36 Trillion Token Barrier: The Quadratic Scaling Crisis

In enterprise artificial intelligence, automated market intelligence, and continuous web monitoring systems powered by AI Fetchly, foundation models are increasingly deployed to ingest massive sequences:

Analyzing hundreds of dense enterprise financial filings, SEC 10-K forms, and legal contracts via real-time data fetching.
Processing complete multi-repository software codebases with thousands of inter-dependent functions.
Aggregating real-time unstructured web pages and data feeds gathered through automated web scraping.

While modern decoder-only Transformers (such as LLaMA-3, Qwen-2.5, and Mistral) demonstrate state-of-the-art general reasoning, serving them across extended context lengths (32k, 64k, 128k+ tokens) introduces a crippling hardware barrier: the quadratic attention bottleneck.

The Transformer Long-Context Serving Crisis:
┌────────────────────────────────────────────────────────────────────────┐
│ 1. Quadratic Prefill Latency Penalty: O(T^2) FLOPs                     │
│    • Pairwise dot-product attention across 128k prompt tokens          │
│    • Time-to-First-Token (TTFT) stretches into tens of seconds         │
├────────────────────────────────────────────────────────────────────────┤
│ 2. Linear Key-Value (KV) Cache Memory Bloat: O(T) Memory               │
│    • Retains every token's Key and Value activations in GPU HBM        │
│    • For an 8B model at 128k context: >16 GB VRAM PER SINGLE STREAM!   │
│    • Multi-tenant concurrency collapses due to Out-Of-Memory crashes   │
└────────────────────────────────────────────────────────────────────────┘

1. The Promise and Pitfalls of Hybrid State-Space Models

To eliminate the memory and latency penalty of attention, researchers pioneered State-Space Models (SSMs) like Mamba-1 and Mamba-2 (State Space Duality / SSD). By replacing quadratic attention matrices with linear-time continuous-to-discrete recurrences:

$$h_t = \mathbf{A} h_{t-1} + \mathbf{B} x_t, \quad y_t = \mathbf{C} h_t + \mathbf{D} x_t$$

State-space architectures achieve constant $O(1)$ decoding memory and linear $O(T)$ prefill scaling.

However, pure SSMs suffer from a fundamental information-theoretic limitation: compressing an unbounded sequence into a fixed-size hidden state $h_t$ degrades associative recall. On needle-in-a-haystack retrieval, exact variable binding, and multi-hop reasoning over long horizons, pure SSMs underperform dense Transformers.

To balance associative recall with linear efficiency, recent frontier architectures—such as Jamba (AI21 Labs), Bamba, and Samba—adopted a Hybrid Architecture: interleaving standard Attention layers with Mamba-2 SSM layers (e.g., a 3:1 or 7:1 SSM-to-Attention ratio). In a hybrid model:

Attention layers act as periodic associative memory anchors, preserving exact long-range retrieval.
SSM layers handle the bulk of sequence mixing in $O(T)$ linear time with zero Key-Value cache allocation.
The aggregate KV cache memory footprint is slashed by $75%$ to $87.5%$, and generation throughput accelerates by up to $5\times$!

2. The Multi-Million-Dollar Retraining Tax

While Hybrid SSM-Transformer architectures represent the ideal serving topology, pre-training them from scratch is economically insurmountable for all but the largest tech conglomerates:

Training Qwen-3-8B or LLaMA-3.1-8B consumed over 36 trillion tokens of compute—requiring tens of thousands of GPU hours and millions of dollars in electricity and infrastructure capital.
If enterprise teams must spend millions of dollars every time they want to train an efficient hybrid architecture from scratch, the benefits of linear state-space models remain inaccessible.

To eliminate this pre-training barrier, researchers from AWS Labs and the Allen Institute for AI (Ai2) introduced Priming (arXiv:2605.08301, open-sourced in awslabs/hybrid-model-factory): an algorithmic transformation framework that converts existing pre-trained Transformers into high-throughput Hybrid Mamba-2 models using less than 0.5% of the original pre-training compute budget.

The Priming Architecture: Layer Transmutation and Weight Repurposing

Rather than throwing away pre-trained foundation models, Priming uses both the learned weights and the structural representations of an existing Transformer as its starting point.

The Priming Conversion Pipeline:
Source Transformer (e.g., Pre-Trained LLaMA-3-8B / Qwen-2.5-8B)
  │
  ├──► [Selective Layer Transmutation (3:1 Ratio)]
  │     • Retain 25% of Attention Layers as Associative Anchors (e.g., layers 4, 8, 12, ...)
  │     • Selectively convert 75% of Attention Layers into Mamba-2 SSD Blocks
  │
  ├──► [Weight Repurposing & Subspace Fusion]
  │     • Map pre-trained W_q, W_k, W_v projections directly into Mamba-2 B, C, X matrices
  │     • Map Attention W_out directly into Mamba-2 Out_Proj
  │     • Preserve 100% of MLP Feed-Forward weights and LayerNorm parameters
  │
  ▼
[Continual Realignment & Distillation (<150B Tokens / <0.5% Compute)]
  └──► High-Throughput Primed Hybrid Model (75% Smaller KV Cache, 5x Generation Speed)

1. Interleaved Layer Transmutation Topology

In a standard Transformer with $L$ layers (e.g., $L=32$ in LLaMA-3-8B), every single layer contains a Multi-Head Attention block and a SwiGLU MLP block.

Priming replaces a fixed fraction of these attention blocks with Mamba-2 State-Space Duality (SSD) blocks while keeping the feed-forward MLP networks completely untouched:

3:1 Interleaving Ratio: 3 Mamba-2 blocks for every 1 Attention Anchor block.
In a 32-layer model: 24 layers become Mamba-2 blocks, and 8 layers remain Attention blocks.
Why keep 25% Attention? Research demonstrates that keeping 1 Attention anchor every 4 layers is sufficient to achieve $100%$ accuracy on 128k Needle-in-a-Haystack benchmarks, completely avoiding the retrieval degradation of pure SSMs while cutting total KV cache allocations by exactly $75%$!

2. Weight Repurposing Matrix Fusion

When initializing a new Mamba-2 layer, naive implementations initialize the state-space parameters randomly. Random initialization destroys the network’s latent geometry, requiring hundreds of billions of warmup tokens to recover basic language capability.

Priming introduces Weight Repurposing: In Mamba-2 (State Space Duality), the input projection maps incoming activations $x \in \mathbb{R}^{d_{\text{model}}}$ into four distinct sub-spaces:

$X$: The input signal to be modulated by state-space matrices ($d_{\text{inner}} = 2 \times d_{\text{model}}$).
$Z$: The multiplicative gate controlling output gating ($d_{\text{inner}}$).
$B$: The continuous input matrix ($d_{\text{state}}$).
$C$: The continuous output observation matrix ($d_{\text{state}}$).

Notice the mathematical similarity to Multi-Head Attention:

Attention projects input $x$ into Query $Q$, Key $K$, and Value $V$ matrices.
The inner product $K^T Q$ in attention functions identically to the state contraction matrix $B \cdot C^T$ in State Space Duality!

Priming exploits this exact mathematical correspondence by repurposing the pre-trained attention weights:

The Value projection $W_v$ and Query projection $W_q$ are concatenated and replicated to initialize the Mamba-2 $X$ and $Z$ projection weights: $$W_{\text{in}}^{\text{Mamba}}[X] \leftarrow W_q, \quad W_{\text{in}}^{\text{Mamba}}[Z] \leftarrow W_v$$
The Key projection $W_k$ is projected to initialize the state modulation matrix $B$.
The Attention Output projection $W_{\text{out}}$ directly initializes the Mamba-2 Output projection: $$W_{\text{out}}^{\text{Mamba}} \leftarrow W_{\text{out}}^{\text{Attn}}$$

Because the feed-forward MLP layers and layer normalizations retain their pre-trained weights verbatim, over $85%$ of the model’s total parameter mass is preserved from the original pre-trained checkpoint, providing an extraordinary inductive head start.

Continual Realignment: Distillation and Token Efficiency

With the hybrid architecture initialized, the model must realign its new Mamba-2 recurrent matrices to harmonize with the frozen MLP and Attention anchor layers.

1. The <0.5% Compute Budget

To understand the magnitude of Priming’s efficiency, consider pre-training budgets:

LLaMA-3-8B Pre-Training: 15 Trillion Tokens.
Qwen-3-8B Pre-Training: 36 Trillion Tokens.
Priming Realignment: Fewer than 150 Billion Tokens!

By requiring less than 150B tokens, Priming consumes less than 0.42% of the compute budget of the original source model! An engineering team can transform an existing 8B open-weights model into an enterprise hybrid engine in days on a modest cluster of 32x or 64x GPUs, saving millions of dollars in compute.

Pre-Training Token Budget Comparison:
┌──────────────────────────────────────────────────────────────┐
│ Qwen-3-8B Scratch Pre-Training: ████████████████████ 36.00 T │
│ LLaMA-3.1-8B Scratch Pre-Train: ████████████ 15.00 T         │
│ Priming Hybrid Realignment:     █ 0.15 T (<0.5% Compute!)    │
└──────────────────────────────────────────────────────────────┘

2. Multi-Stage Distillation Objective

During continual pre-training, Priming trains the transmuted model using a combination of next-token autoregressive cross-entropy loss $\mathcal{L}{\text{CE}}$ and layer-wise feature distillation $\mathcal{L}{\text{distill}}$ against the original frozen Transformer teacher:

$$\mathcal{L}{\text{total}} = \alpha \mathcal{L}{\text{CE}}(y, \hat{y}) + (1 - \alpha) \sum_{\ell \in \text{Anchors}} \text{MSE}\left( h_{\text{hybrid}}^{(\ell)}, ; h_{\text{teacher}}^{(\ell)} \right)$$

This dual objective ensures that:

Intermediate representations at the Attention anchor layers remain anchored to the teacher’s latent coordinates.
The recurrent Mamba-2 layers quickly learn to compress multi-turn context into state vectors without diverging from the pre-trained token vocabulary.

End-to-End Implementation: Building Priming in PyTorch

Let us construct a complete, self-contained, and production-grade implementation demonstrating Mamba-2 State-Space Duality, Attention anchors, the 3:1 Primed Hybrid Transformer, and the layer transmutation weight fusion algorithm in PyTorch.

# -*- coding: utf-8 -*-
"""
Priming: Hybrid State-Space Models From Pre-Trained Transformers
Demonstrates Layer Transmutation, Weight Repurposing, and Hybrid Interleaved Architecture in PyTorch.
"""

import math
import torch
import torch.nn as nn
from typing import Dict, List, Tuple, Optional


# =====================================================================
# 1. Mamba-2 State-Space Duality (SSD) Recurrent Block
# =====================================================================

class Mamba2Block(nn.Module):
    """
    Simplified Mamba-2 State-Space Duality (SSD) layer.
    Replaces quadratic self-attention with linear-time state-space recurrence.
    """
    def __init__(self, d_model: int = 2048, d_state: int = 64, d_conv: int = 4, expand: int = 2):
        super().__init__()
        self.d_model = d_model
        self.d_state = d_state
        self.d_inner = d_model * expand
        self.d_conv = d_conv

        # Input projections (X, B, C matrices)
        self.in_proj = nn.Linear(d_model, self.d_inner * 2 + self.d_state * 2, bias=False)

        # 1D Depthwise Convolution for local token mixing
        self.conv1d = nn.Conv1d(
            in_channels=self.d_inner,
            out_channels=self.d_inner,
            kernel_size=d_conv,
            groups=self.d_inner,
            padding=d_conv - 1
        )

        # Recurrent state parameters (A matrix log-representation & delta discretization)
        self.A_log = nn.Parameter(torch.log(torch.arange(1, d_state + 1, dtype=torch.float32)).unsqueeze(0))
        self.D = nn.Parameter(torch.ones(self.d_inner))

        # Output projection
        self.out_proj = nn.Linear(self.d_inner, d_model, bias=False)

    def forward(self, x: torch.Tensor) -> torch.Tensor:
        batch_size, seq_len, _ = x.shape

        # 1. Project input into combined representation
        projected = self.in_proj(x)
        # Split into branch X, gate Z, and SSM parameters B, C
        x_branch = projected[..., :self.d_inner]
        z_gate = projected[..., self.d_inner:self.d_inner * 2]
        b_mat = projected[..., self.d_inner * 2:self.d_inner * 2 + self.d_state]
        c_mat = projected[..., self.d_inner * 2 + self.d_state:]

        # 2. Local Convolutional Context Mixing
        x_conv = self.conv1d(x_branch.transpose(1, 2))[..., :seq_len].transpose(1, 2)
        x_act = torch.nn.functional.silu(x_conv)

        # 3. State-Space Duality Recurrent Scanning (Chunked Scan Simulation)
        A = -torch.exp(self.A_log)  # (1, d_state)
        # Recurrent state update: h_t = A * h_{t-1} + B * x_t
        h = torch.zeros(batch_size, self.d_state, device=x.device, dtype=x.dtype)
        y_ssm = []
        for t in range(seq_len):
            b_t = b_mat[:, t, :]
            c_t = c_mat[:, t, :]
            xt_scalar = x_act[:, t, :self.d_state]  # Aligned subspace
            h = h * torch.exp(A) + b_t * xt_scalar
            yt = torch.sum(h * c_t, dim=-1, keepdim=True)
            y_ssm.append(yt)

        # 4. Gated multiplicative combination & Output projection
        y_out = torch.cat(y_ssm, dim=-1).repeat(1, 1, self.d_inner // self.d_state)
        y_gated = y_out * torch.nn.functional.silu(z_gate)
        return self.out_proj(y_gated)


# =====================================================================
# 2. Standard Attention Anchor Block
# =====================================================================

class AttentionAnchorBlock(nn.Module):
    """
    Standard Multi-Head Attention layer retained as anchor to guarantee needle recall.
    """
    def __init__(self, d_model: int = 2048, n_heads: int = 16):
        super().__init__()
        self.d_model = d_model
        self.n_heads = n_heads
        self.head_dim = d_model // n_heads

        self.q_proj = nn.Linear(d_model, d_model, bias=False)
        self.k_proj = nn.Linear(d_model, d_model, bias=False)
        self.v_proj = nn.Linear(d_model, d_model, bias=False)
        self.out_proj = nn.Linear(d_model, d_model, bias=False)

    def forward(self, x: torch.Tensor) -> torch.Tensor:
        b, s, d = x.shape
        q = self.q_proj(x).view(b, s, self.n_heads, self.head_dim).transpose(1, 2)
        k = self.k_proj(x).view(b, s, self.n_heads, self.head_dim).transpose(1, 2)
        v = self.v_proj(x).view(b, s, self.n_heads, self.head_dim).transpose(1, 2)

        # Scaled dot-product attention with causal mask
        scores = torch.matmul(q, k.transpose(-2, -1)) / math.sqrt(self.head_dim)
        causal_mask = torch.triu(torch.full((s, s), float("-inf"), device=x.device), diagonal=1)
        attn = torch.softmax(scores + causal_mask, dim=-1)
        out = torch.matmul(attn, v).transpose(1, 2).contiguous().view(b, s, d)
        return self.out_proj(out)


# =====================================================================
# 3. Primed Hybrid Transformer-SSM Architecture (3:1 Ratio)
# =====================================================================

class PrimedHybridBlock(nn.Module):
    """
    A single block in the Primed model: either an Attention Anchor or Transmuted Mamba-2.
    """
    def __init__(self, d_model: int, is_attention: bool = False):
        super().__init__()
        self.is_attention = is_attention
        self.norm1 = nn.LayerNorm(d_model)
        if is_attention:
            self.mixer = AttentionAnchorBlock(d_model=d_model)
        else:
            self.mixer = Mamba2Block(d_model=d_model)

        self.norm2 = nn.LayerNorm(d_model)
        self.mlp = nn.Sequential(
            nn.Linear(d_model, d_model * 4, bias=False),
            nn.SiLU(),
            nn.Linear(d_model * 4, d_model, bias=False)
        )

    def forward(self, x: torch.Tensor) -> torch.Tensor:
        x = x + self.mixer(self.norm1(x))
        x = x + self.mlp(self.norm2(x))
        return x


class PrimedHybridModel(nn.Module):
    """
    Complete Primed Hybrid Foundation Model.
    Interleaves 3 Mamba-2 SSM layers for every 1 Attention Anchor layer (3:1 ratio).
    """
    def __init__(self, vocab_size: int = 32000, d_model: int = 2048, num_layers: int = 24):
        super().__init__()
        self.embed = nn.Embedding(vocab_size, d_model)
        self.layers = nn.ModuleList()

        for idx in range(num_layers):
            # Keep layer as attention anchor every 4th layer (e.g. layers 3, 7, 11, 15, 19, 23)
            is_attn = ((idx + 1) % 4 == 0)
            self.layers.append(PrimedHybridBlock(d_model=d_model, is_attention=is_attn))

        self.norm = nn.LayerNorm(d_model)
        self.lm_head = nn.Linear(d_model, vocab_size, bias=False)

    def forward(self, input_ids: torch.Tensor) -> torch.Tensor:
        h = self.embed(input_ids)
        for layer in self.layers:
            h = layer(h)
        return self.lm_head(self.norm(h))


# =====================================================================
# 4. Weight Transmutation & Distillation Pipeline
# =====================================================================

def transmute_attention_to_mamba2(
    attn_block: AttentionAnchorBlock,
    mamba_block: Mamba2Block
) -> None:
    """
    Initializes Mamba-2 projections from pre-trained Attention projection weights.
    Avoids random initialization and preserves the learned subspace geometry.
    """
    with torch.no_grad():
        d_model = attn_block.d_model
        d_inner = mamba_block.d_inner

        # 1. Repurpose W_q and W_k into Mamba-2 input projection slices
        # Extract pre-trained weights
        wq = attn_block.q_proj.weight  # Shape: (d_model, d_model)
        wk = attn_block.k_proj.weight  # Shape: (d_model, d_model)
        wv = attn_block.v_proj.weight  # Shape: (d_model, d_model)

        # Fill X branch and Gate branch from Q/V representations
        mamba_block.in_proj.weight[:d_inner, :] = wq.repeat(d_inner // d_model, 1)
        mamba_block.in_proj.weight[d_inner:d_inner*2, :] = wv.repeat(d_inner // d_model, 1)

        # 2. Output projection weight initialization from attention out_proj
        mamba_block.out_proj.weight[:, :d_model] = attn_block.out_proj.weight

        print("Successfully transmuted Attention weights into Mamba-2 SSD parameters!")

This complete PyTorch implementation illustrates how Priming preserves pre-trained parameter matrices, establishes the 3:1 Mamba-to-Attention ratio, and maps attention projections into State Space Duality parameters without random initialization penalties.

Empirical Benchmarks and Efficiency Evaluation

Researchers evaluated Primed Hybrid models against dense pre-trained Transformers (LLaMA-3-8B and Qwen-2.5-7B) across benchmark reasoning, long-context retrieval, and inference throughput.

1. General Reasoning and Language Benchmarks

Model Architecture	Training Token Budget	MMLU (5-shot)	GSM8K (8-shot)	HumanEval (Pass@1)	ARC-Challenge
Source LLaMA-3-8B	15.0 Trillion Tokens	68.4%	79.6%	62.2%	79.8%
Pure Mamba-2 (From Scratch)	3.5 Trillion Tokens	58.2%	61.4%	45.1%	69.2%
Jamba-Mini (From Scratch)	5.0 Trillion Tokens	65.1%	73.2%	54.8%	75.3%
Primed Hybrid LLaMA-8B	0.15 Trillion Tokens (<0.5%)	68.1%	78.9%	61.8%	79.5%

As demonstrated in the evaluation, the Primed Hybrid model retains $99.5%$ of the original Transformer’s benchmark reasoning capability, while requiring less than half a percent of the training compute budget.

2. Key-Value Cache Memory & Throughput Scaling

KV Cache Memory Footprint Across Context Lengths (8B Model):
┌──────────────────────────────────────────────────────────────┐
│ Dense LLaMA-3 (128k Tokens):  ████████████████████ 16.0 GB   │
│ Primed Hybrid (128k Tokens):  █████ 4.0 GB (75% Slashed!)    │
└──────────────────────────────────────────────────────────────┘

Autoregressive Generation Throughput (Tokens/Second on A100):
┌──────────────────────────────────────────────────────────────┐
│ Dense LLaMA-3 (64k Tokens):   ████ 18 tok/s                  │
│ Primed Hybrid (64k Tokens):   ████████████████████ 92 tok/s  │
└──────────────────────────────────────────────────────────────┘

By removing $75%$ of the KV cache, Priming delivers:

$75%$ Memory Reduction: Slashes KV cache consumption from 16 GB to 4 GB per request at 128k tokens, enabling $4\times$ higher concurrent batch density on enterprise servers.
Up to $5.1\times$ Generation Speedup: Because token generation is memory-bandwidth bound, reading $75%$ less KV cache data across the GPU memory bus accelerates decoding speeds from 18 tokens/sec to 92 tokens/sec.
$100%$ Needle-in-a-Haystack Retrieval: Preserves perfect multi-document retrieval across 128,000 tokens thanks to the interleaved Attention anchors.

Production Engineering Playbook: Converting Your Models with Priming

Converting proprietary fine-tuned Transformers into high-throughput Primed Hybrid models involves three production engineering steps:

1. Model Configuration & Layer Transmutation

Using the open-source hybrid-model-factory toolkit, define the transmutation configuration:

transmutation:
  source_model: 'meta-llama/Meta-Llama-3-8B'
  target_architecture: 'hybrid_mamba2_attention'
  interleaving_ratio: '3:1' # 3 Mamba-2 layers per 1 Attention layer
  weight_mapping:
    q_to_x: true
    v_to_z: true
    k_to_b: true
    out_to_out: true
    freeze_mlp: false

2. Distillation & Continual Realignment Run

Launch the continual pre-training distillation job using standard distributed training:

torchrun --nproc_per_node=8 run_priming_realignment.py \
    --config configs/llama3_primed_3to1.yaml \
    --dataset data/c4_tokenized_150b \
    --learning-rate 1.5e-4 \
    --warmup-steps 1000 \
    --max-tokens 150000000000 \
    --distill-loss-weight 0.4

3. Deploying on Hybrid Inference Runtimes

Once realigned, deploy the Primed Hybrid model on modern inference engines supporting hybrid Mamba-2 kernels (such as vLLM or TensorRT-LLM):

from vllm import LLM, SamplingParams

# Load the Primed Hybrid model with 75% smaller KV cache
llm = LLM(
    model="models/llama3-primed-hybrid-8b",
    max_model_len=131072,
    gpu_memory_utilization=0.90
)

prompts = ["Analyze this 100-page document: ..."]
sampling_params = SamplingParams(temperature=0.2, max_tokens=1024)
outputs = llm.generate(prompts, sampling_params)

In automated data harvesting, financial intelligence, and autonomous web agent pipelines, Priming unlocks the speed and efficiency of subquadratic models without the multi-million-dollar training tax.

For teams building high-concurrency automated agent platforms and enterprise knowledge pipelines, explore custom API integrations, examine our flexible pricing plans, or consult the developer documentation.

Frequently Asked Questions (FAQ)

What is the primary difference between Priming and pre-training a hybrid model from scratch?

Pre-training a hybrid model from scratch (like Jamba or Bamba) requires training across tens of trillions of tokens, costing millions of dollars and weeks of cluster time. Priming converts an existing, fully pre-trained Transformer (such as LLaMA-3 or Qwen-2.5) into a hybrid model by repurposing attention projection weights and realigning the architecture using fewer than 150 billion tokens—less than 0.5% of the original training compute.

Why does Priming use a 3:1 Mamba-to-Attention ratio instead of pure Mamba?

Pure state-space models compress unbounded sequences into a fixed-size hidden state, which causes severe degradation on long-range associative recall and needle-in-a-haystack tasks. By retaining 1 Attention layer for every 3 Mamba-2 layers (the 3:1 ratio), the attention anchors act as memory checkpoints that guarantee 100% retrieval accuracy while still eliminating 75% of the KV cache.

How does Priming initialize Mamba-2 weights from Attention weights?

Rather than initializing Mamba-2 layers randomly, Priming maps the pre-trained Query ($W_q$), Key ($W_k$), and Value ($W_v$) projection weights directly into Mamba-2’s input gating ($Z$), input signal ($X$), and state modulation ($B$) matrices, while mapping $W_{\text{out}}$ directly into Mamba-2’s output projection. This preserves learned subspace representations and speeds up realignment convergence.

What inference engines support serving Primed Hybrid models?

Primed Hybrid models are compatible with modern inference runtimes that support Mamba-2 State Space Duality kernels alongside standard PagedAttention, including vLLM, TensorRT-LLM, and Hugging Face transformers.

How much GPU memory does Priming save in production?

Because Mamba-2 layers require zero Key-Value cache storage, a 3:1 Primed Hybrid model slashes total KV cache memory consumption by exactly 75%. On a 128k context prompt, memory usage shrinks from 16 GB down to 4 GB per stream, enabling a 4x increase in concurrent serving throughput on the same hardware.