min-p-samplingllm-decodingtop-pnucleus-samplingllm-inferencevllmtutorial

Min-P Sampling Guide: Dynamic LLM Decoding Beyond Top-P

September 27, 2026

Min-P Sampling Architecture: Dynamic Confidence-Scaled LLM Decoding vs Top-P

The Problem: The Flawed Mechanics of Top-P and Top-K Decoding

In generative artificial intelligence, production data extraction pipelines, and automated reasoning systems like AiFetchly, converting raw model logits into fluent natural language requires a decoding strategy.

For over half a decade following its introduction by Holtzman et al. (ICLR 2020), Top-P (Nucleus Sampling) served as the default decoding algorithm across GPT-4, LLaMA, Mistral, and Claude.

Under Top-P, the model sorts vocabulary tokens by probability and samples from the smallest subset of tokens whose cumulative probability reaches a fixed threshold $P$ (typically $P = 0.90$ or $0.95$): $$\sum_{i \in V^{(p)}} P(x_i \mid x_{<t}) \ge P$$

While Top-P was designed to eliminate the long tail of low-probability noise found in pure temperature sampling, production deployment across millions of diverse generation tasks revealed three severe structural failure modes:

1. The Low-Entropy Dilemma (Forcing Tail Noise into Confident Predictions)

The fatal flaw of Top-P is that its cumulative threshold is completely static regardless of the model’s certainty:

Consider a scenario where an LLM is generating structured JSON, writing Python syntax, or reciting a factual entity. The top token has a probability of $p_1 = 0.85$, and the remaining candidate tokens have probabilities of $p_2 = 0.04, p_3 = 0.03, p_4 = 0.02$.
Because the top token ($0.85$) does not reach the required $P = 0.90$ quota on its own: $$0.85 < 0.90$$
Top-P is mathematically forced to include $p_2$ and $p_3$ into the candidate pool, even though the model is 85% confident that token 1 is the single correct answer!
Over a 1,000-token generation, this forced inclusion grants low-probability tail tokens a roughly 15% chance of being sampled at each confident step. This leads directly to syntax errors, hallucinated facts, and broken JSON schemas.

2. The High-Entropy Dilemma (Arbitrary Truncation in Creative Contexts)

Conversely, when the model is operating in an open-ended, creative, or brainstorming regime:

The probability distribution across tokens is broad and flat (e.g., top token $p_1 = 0.08$, followed by 20 synonymous words each with $p \approx 0.04$).
Here, Top-P accumulates mass across a fixed cutoff and abruptly chops off the distribution at an arbitrary boundary, arbitrarily pruning valid stylistic variations or plausible reasoning paths.

3. The Temperature Tuning Trap

Because Top-P cannot adjust to token confidence:

Engineers are forced to lower the temperature parameter ($T = 0.2$ to $0.5$) to prevent tail hallucinations on code and reasoning tasks.
However, lowering temperature squashes token diversity, causing models to fall into repetitive phrasing, bland robotic responses, and circular loops.
Conversely, raising temperature ($T \ge 1.0$) causes Top-P to sample bizarre, uncalibrated tokens from the swollen tail, producing gibberish.

The Solution: Min-P Sampling Formulation

To solve the low-entropy failure mode without sacrificing creative diversity, the open-source AI community (formalized by developer @kalomaze in 2024 and adopted across major inference engines in 2025/2026) introduced Min-P Sampling.

Min-P abandons static cumulative probability mass in favor of a dynamic probability threshold scaled directly by the model’s top-token confidence:

Min-P Sampling Confidence Dynamics: High-Certainty vs Creative Regimes

The Mathematical Definition of Min-P

Let $P(x_i)$ denote the post-softmax probability of token $i$ in vocabulary $V$, and let $p_{\max}$ denote the probability of the single most likely token: $$p_{\max} = \max_{i \in V} P(x_i)$$

Min-P establishes a dynamic cutoff threshold $p_{\text{threshold}}$ defined as: $$p_{\text{threshold}} = p_{\max} \cdot p_{\text{base}}$$

Where $p_{\text{base}} \in [0.0, 1.0]$ is a fixed hyperparameter, typically tuned between $0.05$ and $0.10$ (representing 5% to 10% of the top token’s probability).

Any token whose probability falls below $p_{\text{threshold}}$ is pruned: $$V_{\text{Min-P}} = \left{ i \in V ;\middle|; P(x_i) \ge p_{\text{threshold}} \right}$$

Finally, the surviving tokens in $V_{\text{Min-P}}$ are renormalized to sum to 1.0 before sampling.

Systems Mechanics: How Min-P Adapts Across Generation Regimes

The brilliance of Min-P lies in how the candidate pool size dynamically self-adjusts based on model certainty:

1. The High-Certainty Regime (Automatic Greedy Collapse)

Suppose an LLM is generating a closed Python closing bracket or a deterministic factual date:

The model outputs $p_{\max} = 0.90$.
With $p_{\text{base}} = 0.10$, the dynamic threshold is: $$p_{\text{threshold}} = 0.90 \times 0.10 = 0.09$$
Any runner-up candidate with $p < 0.09$ (e.g., tokens with probabilities of $0.03, 0.02, 0.005$) is instantly discarded!
The candidate pool automatically collapses to a single token.
Outcome: Min-P behaves exactly like deterministic greedy decoding when the model is confident, completely eliminating accidental hallucinations and syntax errors.

2. The Creative / Ambiguous Regime (Dynamic Expansion)

Now consider an open-ended narrative context where multiple adjectives or synonyms are equally valid:

The distribution is flat: the top token has only $p_{\max} = 0.10$.
With $p_{\text{base}} = 0.10$, the dynamic threshold scales down proportionally: $$p_{\text{threshold}} = 0.10 \times 0.10 = 0.01$$
Any token with probability $P(x_i) \ge 0.01$ is preserved in the candidate pool!
The candidate pool expands naturally to include 20 or 30 plausible synonyms.
Outcome: Min-P preserves rich semantic diversity without allowing garbage tokens (e.g., $p = 0.0001$) into the distribution.

3. Temperature Independence and High-$T$ Stability

Because Min-P scales relative to $p_{\max}$, it enables practitioners to raise generation temperature ($T = 1.0$ to $1.4$) without triggering hallucinations:

Raising temperature flattens the distribution and elevates creative choices.
Min-P ensures that no matter how flat the distribution becomes, candidates must still possess a probability proportional to the leading candidate.
You achieve rich, engaging prose and multi-perspective reasoning without ever generating gibberish.

Implementation: PyTorch Sampling Kernel & vLLM Configuration

Min-P can be implemented in a few lines of PyTorch tensor logic, and is natively supported across modern serving engines including vLLM, llama.cpp, and Ollama.

1. Pure PyTorch Implementation of Min-P Sampling

import torch
import torch.nn.functional as F

def sample_min_p(logits: torch.Tensor, p_base: float = 0.08, temperature: float = 1.0) -> torch.Tensor:
    '''
    Executes Min-P sampling on raw model logits.

    Args:
        logits: Tensor of shape [batch_size, vocab_size]
        p_base: Base threshold factor (typically 0.05 to 0.10)
        temperature: Logit scaling temperature (typically 1.0 to 1.2)
    Returns:
        Sampled token indices of shape [batch_size, 1]
    '''
    # 1. Apply temperature scaling
    scaled_logits = logits / max(temperature, 1e-5)

    # 2. Compute full softmax probabilities
    probs = F.softmax(scaled_logits, dim=-1)

    # 3. Find top-1 token probability per sequence: [batch_size, 1]
    p_max, _ = torch.max(probs, dim=-1, keepdim=True)

    # 4. Calculate dynamic confidence threshold
    p_threshold = p_max * p_base

    # 5. Mask out tokens below threshold
    mask = probs < p_threshold
    filtered_probs = probs.masked_fill(mask, 0.0)

    # 6. Renormalize surviving probabilities
    summed_probs = torch.sum(filtered_probs, dim=-1, keepdim=True)
    renormalized_probs = filtered_probs / torch.clamp(summed_probs, min=1e-9)

    # 7. Sample token index
    sampled_tokens = torch.multinomial(renormalized_probs, num_samples=1)
    return sampled_tokens

# Demonstration
if __name__ == "__main__":
    torch.manual_seed(42)
    # Simulate confident logits (Token 0 is 88% likely)
    confident_logits = torch.tensor([[10.0, 7.0, 6.0, 5.0, 4.0]])
    token_confident = sample_min_p(confident_logits, p_base=0.10, temperature=1.0)
    print(f"Sampled from Confident Distribution (Min-P): Token {token_confident.item()}")

    # Simulate flat creative logits (Top token is only slightly ahead)
    flat_logits = torch.tensor([[3.0, 2.9, 2.8, 2.7, 2.6]])
    token_flat = sample_min_p(flat_logits, p_base=0.05, temperature=1.2)
    print(f"Sampled from Flat Distribution (Min-P): Token {token_flat.item()}")

2. Configuring Min-P in Production Serving Runtimes

In production engines like vLLM or llama.cpp, disable Top-P and Top-K entirely to let Min-P control truncation:

# vLLM Production Configuration
from vllm import SamplingParams

production_sampling = SamplingParams(
    temperature=1.0,      # Natural temperature (no artificial chilling needed)
    min_p=0.08,           # Dynamic Min-P threshold (8% of top token)
    top_p=1.0,            # Disabled (set to 1.0)
    top_k=-1,             # Disabled (set to -1)
    max_tokens=512
)

Architectural Comparison: Decoding Strategies Compared

To see why the industry is rapidly standardizing on Min-P, compare it against legacy decoding algorithms:

Decoding Strategy	Cutoff Mechanism	Confident Tokens	Uncertain Tokens	High Temperature ($T \ge 1.2$)	Risk of Hallucination
Greedy Search	Top-1 only	Deterministic (Good)	Repetitive & robotic	N/A (Ignores temperature)	Zero (Bland output)
Top-K Sampling	Top $K$ tokens fixed	Forces $K$ candidates	Arbitrarily capped at $K$	High (Picks garbage tail)	Moderate to High
Top-P (Nucleus)	Fixed mass (0.90)	Forces noisy tail in	Abruptly truncates	High (Loops and gibberish)	High on tail tokens
Min-P Sampling	$p_{\max} \cdot p_{\text{base}}$	Collapses to 1 token	Expands dynamically	Stable & coherent	Minimal (Self-healing)

Production Engineering: Deploying Min-P in Autonomous Extraction & Agents

In structured data extraction, automated web scraping agents, and code generation pipelines like AiFetchly, switching from Top-P to Min-P provides immediate architectural benefits:

1. Robust Structured JSON Schema Adherence

When generating structured JSON payloads:

Keys, curly braces, and enum values have high conditional probability ($p_{\max} > 0.80$).
Under Top-P, low-probability tail tokens occasionally get sampled, corrupting closing quotes or injecting unexpected commas.
Under Min-P, the candidate pool instantly contracts to a single token during syntax emission, guaranteeing 100% valid JSON parse rates without grammar-constrained slowdowns.

2. Elimination of Repetition Loops in Long Context Summaries

In multi-document summarization:

Standard Top-P models frequently fall into repetitive degenerative loops when summarizing technical papers or long transcripts.
Min-P allows running models at higher temperatures ($T = 1.05$), ensuring expressive linguistic variety while dynamically filtering out nonsense tokens.

3. Simplified Hyperparameter Optimization

In agentic workflows:

Engineering teams previously spent weeks tuning different combinations of top_p, top_k, temperature, and repetition_penalty for different tasks.
Min-P simplifies the entire configuration: set min_p = 0.05 to 0.08 and temperature = 1.0, providing a universal, battle-tested default across code, analysis, and creative writing.

By replacing static probability mass with dynamic, confidence-scaled truncation, Min-P sampling establishes a more principled foundation for probabilistic language model inference across modern AI infrastructure.