diff --git a/README.md b/README.md
index 297d6445..aa5f18a7 100644
--- a/README.md
+++ b/README.md
@@ -12,7 +12,7 @@
 
 # GLiNER: Generalist and Lightweight Model for Named Entity Recognition
 
-**Zero-shot NER | Relation Extraction | PII Detection | Information Extraction | Token Classification**
+**Zero-shot NER | Streaming NER | Relation Extraction | PII Detection | Information Extraction | Token Classification**
 
 <div>
     <!-- Docs & Resources -->
@@ -39,7 +39,7 @@
   <img src="assets/banner.png" alt="GLiNER Banner" width="100%">
 </div>
 
-GLiNER is a framework for training and deploying small Named Entity Recognition (NER) models with zero-shot capabilities. In addition to traditional NER, it also supports joint entity and relation extraction, as well as multi-task token classification. GLiNER is fine-tunable, optimized to run on CPUs and consumer hardware, and has performance competitive with LLMs several times its size, like ChatGPT and UniNER.
+GLiNER is a framework for training and deploying small Named Entity Recognition (NER) models with zero-shot capabilities. In addition to traditional NER, it supports incremental streaming NER, joint entity and relation extraction, and multi-task token classification. GLiNER is fine-tunable, optimized to run on CPUs and consumer hardware, and has performance competitive with LLMs several times its size, like ChatGPT and UniNER.
 
 Other tasks such as text classification, entity linking, and schema extraction are supported through projects in the [Ecosystem](#ecosystem).
 
@@ -206,6 +206,7 @@ GLiNER supports multiple architectures tailored to different use cases:
 | **Bi-encoder** | Scalable to massive numbers of entity types via separate text and label encoding. | [gliner-bi-base-v2.0](https://huggingface.co/knowledgator/gliner-bi-base-v2.0) |
 | **RelEx** | Joint NER and relation extraction in a single model. | [gliner-relex-large-v1.0](https://huggingface.co/knowledgator/gliner-relex-large-v1.0) |
 | **GLiNER Decoder** | Hybrid architecture for open NER: entity types are generated with a small decoder for maximum flexibility. | [gliner-decoder-large-v1.0](https://huggingface.co/knowledgator/gliner-decoder-large-v1.0) |
+| **StreamingSpan** | Causal span model that reuses decoder, label, and word caches for incremental NER and rolling prediction updates. | [gliner-stream-pii-v1.0](https://huggingface.co/knowledgator/gliner-stream-pii-v1.0) |
 
 For more details, see the [documentation](https://urchade.github.io/GLiNER/architectures.html).
 
@@ -310,6 +311,31 @@ If you find **GLiNER** useful in your research, please consider citing the origi
 
 The GLiNER family has since been extended to additional information extraction and classification tasks:
 
+### GLiNER multi-task
+```bibtex
+@misc{stepanov2024glinermultitaskgeneralistlightweight,
+      title={GLiNER multi-task: Generalist Lightweight Model for Various Information Extraction Tasks}, 
+      author={Ihor Stepanov and Mykhailo Shtopko},
+      year={2024},
+      eprint={2406.12925},
+      archivePrefix={arXiv},
+      primaryClass={cs.LG},
+      url={https://arxiv.org/abs/2406.12925}, 
+}
+```
+### GLiNER bi-encoder
+```bibtex
+@misc{stepanov2026millionlabelnerbreakingscale,
+      title={The Million-Label NER: Breaking Scale Barriers with GLiNER bi-encoder}, 
+      author={Ihor Stepanov and Mykhailo Shtopko and Dmytro Vodianytskyi and Oleksandr Lukashov},
+      year={2026},
+      eprint={2602.18487},
+      archivePrefix={arXiv},
+      primaryClass={cs.CL},
+      url={https://arxiv.org/abs/2602.18487}, 
+}
+```
+
 ### GLiNER2
 
 ```bibtex
diff --git a/benchmarks/bench_streaming.py b/benchmarks/bench_streaming.py
new file mode 100644
index 00000000..8ff7bb2b
--- /dev/null
+++ b/benchmarks/bench_streaming.py
@@ -0,0 +1,246 @@
+#!/usr/bin/env python3
+"""Benchmark GLiNER StreamingSpan inference across sequence lengths and devices.
+
+Normal mode processes the complete text in one stateless call. Streaming mode
+feeds one model-split word per call while retaining the model's session KV cache.
+
+Example:
+    python benchmarks/bench_streaming.py models/checkpoint-15000 \
+        --devices cpu,cuda --lengths 16,32,64,128 --repeats 5
+"""
+
+from __future__ import annotations
+
+import argparse
+import json
+import statistics
+import time
+from dataclasses import asdict, dataclass
+from pathlib import Path
+from uuid import uuid4
+
+import torch
+
+from gliner import GLiNER
+
+
+DEFAULT_LABELS = ["person", "organization", "location", "date"]
+CORPUS_WORDS = (
+    "Alice Johnson joined Acme Corporation in London on Monday before meeting "
+    "Robert Smith from Global Research Institute in Paris to discuss a new "
+    "technology project supported by the European Commission and Stanford University"
+).split()
+
+
+@dataclass(frozen=True)
+class Result:
+    device: str
+    mode: str
+    requested_words: int
+    model_words: int
+    transformer_tokens: int
+    median_seconds: float
+    mean_seconds: float
+    stdev_seconds: float
+    words_per_second: float
+    tokens_per_second: float
+    repeats: int
+
+
+def parse_csv(value: str, cast=str) -> list:
+    items = [item.strip() for item in value.split(",") if item.strip()]
+    if not items:
+        raise argparse.ArgumentTypeError("expected a non-empty comma-separated list")
+    try:
+        return [cast(item) for item in items]
+    except ValueError as error:
+        raise argparse.ArgumentTypeError(str(error)) from error
+
+
+def make_text(word_count: int) -> str:
+    words = [CORPUS_WORDS[index % len(CORPUS_WORDS)] for index in range(word_count)]
+    return " ".join(words)
+
+
+def word_chunks(model, text: str) -> list[str]:
+    """Split text into appendable chunks using GLiNER's own word splitter."""
+    token_batches, _, end_batches = model.prepare_inputs([text])
+    tokens = token_batches[0]
+    ends = end_batches[0]
+    chunks: list[str] = []
+    previous_end = 0
+    for index, end in enumerate(ends):
+        chunk_end = len(text) if index == len(tokens) - 1 else end
+        chunks.append(text[previous_end:chunk_end])
+        previous_end = chunk_end
+    return chunks
+
+
+def transformer_token_count(model, text: str) -> int:
+    tokenizer = model.data_processor.transformer_tokenizer
+    encoded = tokenizer(text, add_special_tokens=False)
+    input_ids = encoded["input_ids"]
+    if input_ids and isinstance(input_ids[0], list):
+        input_ids = input_ids[0]
+    return len(input_ids)
+
+
+def synchronize(device: torch.device) -> None:
+    if device.type == "cuda":
+        torch.cuda.synchronize(device)
+    elif device.type == "mps":
+        torch.mps.synchronize()
+
+
+def run_once(model, mode: str, text: str, chunks: list[str], labels: list[str], threshold: float) -> None:
+    if mode == "normal":
+        model.inference([text], labels, threshold=threshold)
+        return
+
+    session_id = f"benchmark-{uuid4().hex}"
+    model.clear_session(session_id)
+    try:
+        for chunk in chunks:
+            model.inference([chunk], labels, session_id=[session_id], threshold=threshold)
+    finally:
+        model.clear_session(session_id)
+
+
+def measure(
+    model,
+    device: torch.device,
+    mode: str,
+    text: str,
+    chunks: list[str],
+    labels: list[str],
+    threshold: float,
+    warmups: int,
+    repeats: int,
+) -> list[float]:
+    for _ in range(warmups):
+        run_once(model, mode, text, chunks, labels, threshold)
+    synchronize(device)
+
+    samples = []
+    for _ in range(repeats):
+        synchronize(device)
+        started = time.perf_counter()
+        run_once(model, mode, text, chunks, labels, threshold)
+        synchronize(device)
+        samples.append(time.perf_counter() - started)
+    return samples
+
+
+def resolve_devices(requested: list[str]) -> list[torch.device]:
+    devices: list[torch.device] = []
+    for name in requested:
+        if name == "auto":
+            name = "cuda" if torch.cuda.is_available() else "cpu"
+        device = torch.device(name)
+        if device.type == "cuda" and not torch.cuda.is_available():
+            print(f"Skipping {name}: CUDA is not available")
+            continue
+        if device.type == "mps" and not torch.backends.mps.is_available():
+            print(f"Skipping {name}: MPS is not available")
+            continue
+        devices.append(device)
+    if not devices:
+        raise SystemExit("None of the requested devices is available")
+    return devices
+
+
+def print_results(results: list[Result]) -> None:
+    print()
+    print(
+        f"{'device':<9} {'mode':<10} {'words':>7} {'tokens':>7} "
+        f"{'median ms':>11} {'words/s':>12} {'tokens/s':>12}"
+    )
+    print("-" * 76)
+    for result in results:
+        print(
+            f"{result.device:<9} {result.mode:<10} {result.model_words:>7} "
+            f"{result.transformer_tokens:>7} {result.median_seconds * 1000:>11.2f} "
+            f"{result.words_per_second:>12.2f} {result.tokens_per_second:>12.2f}"
+        )
+
+
+def build_parser() -> argparse.ArgumentParser:
+    parser = argparse.ArgumentParser(description=__doc__)
+    parser.add_argument("model", help="Local path or Hugging Face ID for a StreamingSpan checkpoint")
+    parser.add_argument("--devices", default="cpu,cuda", help="Comma-separated devices (default: cpu,cuda)")
+    parser.add_argument("--lengths", default="16,32,64,128", help="Comma-separated input word counts")
+    parser.add_argument("--labels", default=",".join(DEFAULT_LABELS), help="Comma-separated entity labels")
+    parser.add_argument("--warmups", type=int, default=1, help="Warmup runs per mode and length")
+    parser.add_argument("--repeats", type=int, default=5, help="Measured runs per mode and length")
+    parser.add_argument("--threshold", type=float, default=0.5)
+    parser.add_argument("--local-files-only", action="store_true")
+    parser.add_argument("--json", type=Path, help="Optionally write detailed results as JSON")
+    return parser
+
+
+def main() -> None:
+    args = build_parser().parse_args()
+    lengths = parse_csv(args.lengths, int)
+    labels = parse_csv(args.labels)
+    requested_devices = parse_csv(args.devices)
+    if any(length < 1 for length in lengths):
+        raise SystemExit("All --lengths values must be positive")
+    if args.warmups < 0 or args.repeats < 1:
+        raise SystemExit("--warmups must be >= 0 and --repeats must be >= 1")
+    if not 0.0 <= args.threshold <= 1.0:
+        raise SystemExit("--threshold must be between 0 and 1")
+
+    results: list[Result] = []
+    for device in resolve_devices(requested_devices):
+        print(f"Loading {args.model!r} on {device} ...", flush=True)
+        model = GLiNER.from_pretrained(
+            args.model,
+            load_tokenizer=True,
+            local_files_only=args.local_files_only,
+            map_location=str(device),
+        ).to(device).eval()
+        if getattr(model.config, "model_type", None) != "gliner_streaming_span":
+            raise SystemExit("The checkpoint must use model_type='gliner_streaming_span'")
+
+        for requested_words in lengths:
+            text = make_text(requested_words)
+            chunks = word_chunks(model, text)
+            token_count = transformer_token_count(model, text)
+            print(
+                f"Benchmarking {device}: requested={requested_words}, "
+                f"model_words={len(chunks)}, transformer_tokens={token_count}",
+                flush=True,
+            )
+            for mode in ("normal", "streaming"):
+                samples = measure(
+                    model, device, mode, text, chunks, labels, args.threshold, args.warmups, args.repeats
+                )
+                median = statistics.median(samples)
+                results.append(
+                    Result(
+                        device=str(device),
+                        mode=mode,
+                        requested_words=requested_words,
+                        model_words=len(chunks),
+                        transformer_tokens=token_count,
+                        median_seconds=median,
+                        mean_seconds=statistics.mean(samples),
+                        stdev_seconds=statistics.stdev(samples) if len(samples) > 1 else 0.0,
+                        words_per_second=len(chunks) / median,
+                        tokens_per_second=token_count / median,
+                        repeats=len(samples),
+                    )
+                )
+
+        del model
+        if device.type == "cuda":
+            torch.cuda.empty_cache()
+
+    print_results(results)
+    if args.json:
+        args.json.write_text(json.dumps([asdict(result) for result in results], indent=2) + "\n")
+        print(f"\nWrote {args.json}")
+
+
+if __name__ == "__main__":
+    main()
diff --git a/configs/config_streaming_span.yaml b/configs/config_streaming_span.yaml
new file mode 100644
index 00000000..f3c5bd67
--- /dev/null
+++ b/configs/config_streaming_span.yaml
@@ -0,0 +1,82 @@
+model:
+  # StreamingSpanModel uses one causal decoder for label prompts and text.
+  model_type: gliner_streaming_span
+  model_name: HuggingFaceTB/SmolLM2-135M-Instruct
+  name: "streaming span gliner"
+
+  # Input layout: label1<<LABEL>>label2<<LABEL>><<SEP>>text
+  label_token: "<<LABEL>>"
+  sep_token: "<<SEP>>"
+
+  max_width: 12
+  # Rescore recent spans with future context; null defaults to max_width.
+  # Set to 0 for append-only streaming span classification.
+  right_context_width: null
+  hidden_size: 576
+  dropout: 0.3
+  fine_tune: true
+  subtoken_pooling: first
+  fuse_layers: false
+  post_fusion_schema: ""
+
+  # StreamingSpanLabelsEncoder processes only the label prompt through <<SEP>>.
+  labels_encoder_config:
+    model_type: deberta-v2
+    hidden_size: 576
+    num_hidden_layers: 2
+    num_attention_heads: 9
+    intermediate_size: 2304
+    hidden_dropout_prob: 0.1
+    attention_probs_dropout_prob: 0.1
+    relative_attention: true
+    pos_att_type: [p2c, c2p]
+    max_relative_positions: 512
+
+  # markerV2 combines the span start, span end, and final word embedding.
+  span_mode: markerV2
+  span_context_encoder: none
+  span_context_num_layers: 1
+  num_rnn_layers: 0
+
+  max_types: 100
+  max_len: 512
+  max_neg_type_ratio: 1
+  # null uses the decoder backbone's native context limit for session caches.
+  max_cache_length: null
+  span_loss_coef: 1.0
+
+data:
+  root_dir: gliner_logs/streaming_span
+  train_data: "data/data.json"
+  val_data_dir: "none"
+
+training:
+  prev_path: null
+
+  num_steps: 15000
+  train_batch_size: 4
+  eval_every: 500
+  warmup_steps: 0.05
+  scheduler_type: "cosine"
+
+  loss_alpha: 0.75
+  loss_gamma: 0
+  loss_prob_margin: 0
+  label_smoothing: 0
+  loss_reduction: "sum"
+  negatives: 1.0
+  masking: "none"
+
+  lr_encoder: 1e-5
+  lr_others: 3e-5
+  weight_decay_encoder: 0.01
+  weight_decay_other: 0.01
+  max_grad_norm: 10.0
+
+  save_total_limit: 3
+  size_sup: -1
+  shuffle_types: true
+  random_drop: true
+
+  # Available names include decoder_backbone and labels_encoder.
+  freeze_components: null
diff --git a/docs/architectures.md b/docs/architectures.md
index 4b57f182..7be1d74a 100644
--- a/docs/architectures.md
+++ b/docs/architectures.md
@@ -24,9 +24,17 @@ The GLiNER framework now supports multiple architecture variants, each optimized
 | **UniEncoderTokenDecoder** | Single encoder + generative decoder | Token-level with generation | Token-level detection + label generation | Long entities with open vocabulary |
 | **UniEncoderSpanRelex** | Single encoder + relation layers | Span-level + relations | Joint entity and relation extraction | Knowledge graph construction, IE |
 | **UniEncoderTokenRelex** | Single encoder + relation layers | Token-level + relations | Token-level entities with relation extraction | Long entities with relations |
+| **StreamingSpan** | Causal decoder with reusable prompt and text caches | Span-level | Incremental inference with rolling right-context rescoring | Live transcripts, logs, and chunked PII detection |
 
 The framework automatically selects the appropriate architecture based on your model configuration, providing a unified API across all variants.
 
+All stateless architectures apply the checkpoint's text-only `config.max_len`
+before encoding labels. Whether labels share a later backbone context with the
+text depends on the architecture: uni-encoders share it, while bi-encoders keep
+the two sequences separate. Cached StreamingSpan sessions use their causal
+decoder context instead. See [Input limits and truncation](input_limits.md) for
+the complete behavior and production preflight guidance.
+
 ## Vanilla GLiNER (UniEncoderSpan)
 
 ![alt text](images/image-2.png)
@@ -554,6 +562,109 @@ for relation in relations[0]:
 
 Same as UniEncoderSpanRelex - entities and relations are returned in identical formats, ensuring API consistency across architectures.
 
+## GLiNER StreamingSpan
+
+StreamingSpan (`model_type="gliner_streaming_span"`) is the incremental
+span-classification architecture. It replaces the usual bidirectional text
+encoder with a causal decoder backbone, allowing later calls to append text to
+the decoder's key-value (KV) cache instead of encoding the complete document
+again. This is useful when the input is revealed over time, such as a live
+transcript, chat, log stream, or progressively uploaded document.
+
+StreamingSpan should not be confused with `UniEncoderSpanDecoder`. The latter
+uses an auxiliary decoder to *generate* entity labels; StreamingSpan uses a
+decoder as its text backbone and still scores spans against the entity types
+provided by the caller.
+
+![GLiNER StreamingSpan architecture](images/gliner-streaming-architecture.svg)
+
+### Cold pass: prompt and first chunk
+
+The processor serializes the requested entity types before the text:
+
+```text
+person<<LABEL>>email address<<LABEL>><<SEP>>Alice can be reached at ...
+```
+
+Placing `<<LABEL>>` after each label is important for a causal model: the
+marker state can attend to every subtoken in the label that precedes it. The
+first call then performs the following work:
+
+1. The complete label prompt and first text chunk pass through the causal
+   decoder.
+2. The prompt slice, ending at `<<SEP>>`, passes through a compact label context
+   encoder. The states at `<<LABEL>>` become one representation per entity
+   type.
+3. Text subtoken states are pooled into word states.
+4. The model stores decoder KV states, label representations, word states, text
+   offsets, and span scores in the session cache.
+
+The label context encoder is independent of the causal backbone and may use a
+DeBERTa-v2, ModernBERT, or RNN context encoder. DeBERTa-v2, ModernBERT, and a
+bidirectional RNN give label names bidirectional context without making the
+text backbone non-causal.
+
+### Warm pass: append and reuse
+
+On later calls, only the new chunk is sent through the decoder. Its tokens
+attend to the cached history, use continuing position IDs, and extend the KV
+cache. The stored label representations are reused, while the new subtoken
+states are pooled and appended to the cached word states.
+
+This avoids recomputing the decoder states for the prompt and historical text.
+It does not make inference constant-cost: decoder attention, KV memory, word
+state storage, and the span-score history still grow with the session.
+
+### Rolling span representations and scores
+
+Candidate spans use absolute word indices and are limited by `max_width`. The
+default `markerV2` representation combines three projected vectors:
+
+- the candidate's first word;
+- the candidate's last word;
+- the last valid word currently visible to the session.
+
+An optional `span_encoder_config` can contextualize the cached word sequence
+before those vectors are gathered. The resulting span representation is scored
+against each projected label representation with a dot product, as in the
+standard span architecture.
+
+A causal model cannot know future context on its first attempt. StreamingSpan
+therefore maintains a rolling revision window. Every append scores all spans
+that end in the new chunk and re-scores spans whose end is within
+`right_context_width` words of the newest word. New scores replace the stored
+scores for the same `(start, end)` boundary. The default right-context width is
+`max_width`; setting it to `0` gives append-only scoring.
+
+When `span_encoder_config` is bidirectional, appending a word can change every
+contextualized word representation. In that case all historical span
+candidates are re-scored from the cached word states. Passing `recompute=True`
+goes further and rebuilds the complete decoder, prompt, word, and span state
+from the accumulated text.
+
+### Training
+
+Training uses the normal span-based GLiNER data format; examples do not need to
+be pre-split into streaming chunks. The label prompt and complete training text
+are passed through the causal backbone, all valid spans up to `max_width` are
+scored, and the causal backbone, label context encoder, span projections, and
+prompt projections are optimized with the configurable GLiNER span loss.
+Session caches are an inference optimization rather than a separate training
+objective.
+
+### Decoding and output semantics
+
+After merging new and revised scores into the session history, the normal flat
+or nested span decoder applies the threshold. A streaming call returns the
+complete current entity snapshot with document-relative character offsets. It
+is not a delta: callers that render live changes should compare consecutive
+snapshots to identify additions, updates, and removals.
+
+The [StreamingSpan guide](streaming.md) covers the session API, persistent
+batches, asynchronous microbatching, cache cleanup, configuration, and
+deployment trade-offs. A PII-oriented checkpoint is available at
+[knowledgator/gliner-stream-pii-v1.0](https://huggingface.co/knowledgator/gliner-stream-pii-v1.0).
+
 ## Choosing the Right Architecture
 
 Here's a quick guide to selecting the appropriate GLiNER architecture:
@@ -569,34 +680,37 @@ Here's a quick guide to selecting the appropriate GLiNER architecture:
 | Knowledge graph extraction | UniEncoderSpanRelex | Joint entity and relation extraction |
 | Long entities with relations | UniEncoderTokenRelex | Token-level entities with relation extraction |
 | Both long entities + many types | BiEncoderToken | Combines both advantages |
+| Incremental or live text | StreamingSpan | Reuses causal caches and revises recent predictions as context arrives |
 
 ### Decision Flowchart
 
 ```
 Start
   │
-  ├─ Need relation extraction?
-  │   ├─ Yes: Long entities expected?
-  │   │   ├─ Yes → UniEncoderTokenRelex
-  │   │   └─ No → UniEncoderSpanRelex
-  │   │
-  │   └─ No: Need open vocabulary labels?
-  │       ├─ Yes: Long entities expected?
-  │       │   ├─ Yes → UniEncoderTokenDecoder
-  │       │   └─ No → UniEncoderSpanDecoder
-  │       │
-  │       └─ No: Many entity types (>30)?
-  │           ├─ Yes: Long entities expected?
-  │           │   ├─ Yes → BiEncoderToken
-  │           │   └─ No → BiEncoderSpan
-  │           │
-  │           └─ No: Long entities expected?
-  │               ├─ Yes → UniEncoderToken
-  │               └─ No → UniEncoderSpan
+  ├─ Incremental or live input → StreamingSpan
+  │
+  └─ Complete input: Need relation extraction?
+      ├─ Yes: Long entities expected?
+      │   ├─ Yes → UniEncoderTokenRelex
+      │   └─ No → UniEncoderSpanRelex
+      │
+      └─ No: Need open vocabulary labels?
+          ├─ Yes: Long entities expected?
+          │   ├─ Yes → UniEncoderTokenDecoder
+          │   └─ No → UniEncoderSpanDecoder
+          │
+          └─ No: Many entity types (>30)?
+              ├─ Yes: Long entities expected?
+              │   ├─ Yes → BiEncoderToken
+              │   └─ No → BiEncoderSpan
+              │
+              └─ No: Long entities expected?
+                  ├─ Yes → UniEncoderToken
+                  └─ No → UniEncoderSpan
 ```
 
 ## References
 - The Intro section is based on the [Shahrukh Khan](https://www.linkedin.com/in/shahrukhx01/) article [Illustrated GLINER](https://medium.com/@shahrukhx01/illustrated-gliner-e6971e4c8c52) and placed into documentation with consent of the author.
 - [Urchade Zaratiana, Nadi Tomeh, Pierre Holat, and Thierry Charnois. 2023. Gliner: Generalist model for named entity recognition using bidirectional transformer](https://arxiv.org/abs/2311.08526)
 - [Ihor Stepanov, and Mykhailo Shtopko. 2024. GLINER MULTI-TASK: GENERALIST LIGHTWEIGHT MODEL FOR VARIOUS INFORMATION EXTRACTION TASKS](https://arxiv.org/abs/2311.08526)
-- [Meet the new zero-shot NER architecture | by Knowledgator Engineering | Aug, 2024 | Medium](https://blog.knowledgator.com/meet-the-new-zero-shot-ner-architecture-30ffc2cb1ee0)
\ No newline at end of file
+- [Meet the new zero-shot NER architecture | by Knowledgator Engineering | Aug, 2024 | Medium](https://blog.knowledgator.com/meet-the-new-zero-shot-ner-architecture-30ffc2cb1ee0)
diff --git a/docs/configs.md b/docs/configs.md
index d6828bb5..d128624d 100644
--- a/docs/configs.md
+++ b/docs/configs.md
@@ -64,7 +64,20 @@ Whether to fine-tune the encoder during training.
 #### `subtoken_pooling`
 `str`, *optional*, defaults to `"first"`
 
-Currently only first token pooling is supported. More approaches will be added in the future.
+Controls how contextualized subtoken representations are combined into one word
+representation.
+
+**Available options:**
+
+- `"first"` — Uses the first subtoken representation. This is the default and
+  remains compatible with existing checkpoints.
+- `"last"` — Uses the last subtoken representation.
+- `"mean"` — Averages all subtoken representations belonging to the word.
+- `"max"` — Takes the element-wise maximum across all subtoken representations
+  belonging to the word.
+
+Use the same pooling strategy for training and inference. Changing this setting
+for an already-trained checkpoint can reduce prediction quality.
 
 ---
 
@@ -141,7 +154,16 @@ Maximum number of entity types supported per batch.
 #### `max_len`
 `int`, *optional*, defaults to `384`
 
-Maximum sequence length accepted by the encoder.
+Maximum number of text tokens retained from GLiNER's configured word splitter.
+This limit is applied before the entity/relation prompt is added and before
+transformer subword tokenization. If an input is longer, stateless processing
+emits a `UserWarning` and keeps the first `max_len` splitter tokens. Checkpoint
+configurations may override the base default.
+
+This is not necessarily the backbone's total context length. Uni-encoder
+prompts and text share the later transformer subword/context limit, while
+bi-encoders encode labels separately. See [Input limits and
+truncation](input_limits.md) for all limits and the StreamingSpan exception.
 
 ---
 
@@ -149,7 +171,12 @@ Maximum sequence length accepted by the encoder.
 `str`, *optional*, defaults to `"whitespace"`
 
 Heuristic used for word-level splitting during inference.  
-**Choices:** `"whitespace"`, `"spacy"`, `"moses"`, `stanza`, `universal`
+The resulting token count is what `max_len` limits. The default `whitespace`
+splitter separates punctuation as well as whitespace, so `text.split()` is not
+an equivalent counter.
+
+**Choices:** `"whitespace"`, `"universal"`, `"spacy"`, `"mecab"`, `"jieba"`,
+`"hanlp"`, `"janome"`, `"camel"`, `"hindi"`, `"stanza"`
 
 ---
 
@@ -188,6 +215,26 @@ A nested config dictionary for the encoder model. If a dict is passed, its `mode
 
 ---
 
+#### `span_encoder_config`
+`dict` or `PretrainedConfig`, *optional*, defaults to `None`
+
+An optional context encoder applied to word representations immediately before
+the span representation layer. Supported `model_type` values are
+`"deberta-v2"`, `"modernbert"`, and `"rnn"`. Its internal `hidden_size` may
+differ from the model hidden size; projections are added automatically.
+
+```yaml
+span_encoder_config:
+  model_type: rnn
+  hidden_size: 512
+  num_hidden_layers: 1
+  bidirectional: true
+```
+
+Set this field to `null` to use the word representations unchanged.
+
+---
+
 #### `ent_token`
 `str`, *optional*, defaults to `"<<ENT>>"`
 
@@ -894,4 +941,4 @@ Ratio of negative to positive spans during training.
 #### `masking`  
 `str`, *optional*, defaults to `"none"`  
 Masking strategy for negative sampling.  
-**Choices:** `"none"`, `"global"`, `"label"`, `"span"`
\ No newline at end of file
+**Choices:** `"none"`, `"global"`, `"label"`, `"span"`
diff --git a/docs/images/gliner-streaming-architecture.svg b/docs/images/gliner-streaming-architecture.svg
new file mode 100644
index 00000000..882cf4d4
--- /dev/null
+++ b/docs/images/gliner-streaming-architecture.svg
@@ -0,0 +1,308 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<svg xmlns="http://www.w3.org/2000/svg" width="2100" height="1320" viewBox="0 0 2100 1320" role="img" aria-labelledby="title desc">
+  <title id="title">GLiNER StreamingSpan architecture</title>
+  <desc id="desc">Minimal architecture diagram showing a cached entity-type prompt on the left, an incrementally appended input stream on the right, a causal transformer with a reusable key-value cache, cached label and word states, rolling span representations, similarity scores, and a revised entity snapshot.</desc>
+
+  <defs>
+    <marker id="arrow" viewBox="0 0 10 10" refX="8.5" refY="5" markerWidth="7" markerHeight="7" orient="auto-start-reverse">
+      <path d="M0 0L10 5L0 10Z" fill="#31383f"/>
+    </marker>
+    <marker id="arrowRed" viewBox="0 0 10 10" refX="8.5" refY="5" markerWidth="7" markerHeight="7" orient="auto-start-reverse">
+      <path d="M0 0L10 5L0 10Z" fill="#df5961"/>
+    </marker>
+
+    <g id="vecPrompt">
+      <rect width="48" height="64" fill="#f7f5f5" stroke="#9b6870" stroke-width="1.4"/>
+      <rect x="8" y="7" width="32" height="8" fill="#c7a2a8"/>
+      <rect x="8" y="19" width="32" height="8" fill="#7f666b"/>
+      <rect x="8" y="31" width="32" height="8" fill="#d9c2c5"/>
+      <rect x="8" y="43" width="32" height="8" fill="#9d7b80"/>
+      <rect x="8" y="55" width="32" height="5" fill="#5e4f52"/>
+    </g>
+    <g id="vecCached">
+      <rect width="48" height="64" fill="#f4f6f7" stroke="#75828a" stroke-width="1.4"/>
+      <rect x="8" y="7" width="32" height="8" fill="#c0c8cc"/>
+      <rect x="8" y="19" width="32" height="8" fill="#7b878e"/>
+      <rect x="8" y="31" width="32" height="8" fill="#d3d9dc"/>
+      <rect x="8" y="43" width="32" height="8" fill="#929da3"/>
+      <rect x="8" y="55" width="32" height="5" fill="#626d73"/>
+    </g>
+    <g id="vecNew">
+      <rect width="48" height="64" fill="#edf7fb" stroke="#3885a8" stroke-width="2"/>
+      <rect x="8" y="7" width="32" height="8" fill="#a7d0e1"/>
+      <rect x="8" y="19" width="32" height="8" fill="#4f90ae"/>
+      <rect x="8" y="31" width="32" height="8" fill="#c7e2ed"/>
+      <rect x="8" y="43" width="32" height="8" fill="#72abc4"/>
+      <rect x="8" y="55" width="32" height="5" fill="#2f6f8d"/>
+    </g>
+    <g id="vecRevised">
+      <rect width="48" height="64" fill="#fff4f4" stroke="#df5961" stroke-width="2" stroke-dasharray="5 4"/>
+      <rect x="8" y="7" width="32" height="8" fill="#efb4b8"/>
+      <rect x="8" y="19" width="32" height="8" fill="#b36c72"/>
+      <rect x="8" y="31" width="32" height="8" fill="#f4d2d5"/>
+      <rect x="8" y="43" width="32" height="8" fill="#d78b91"/>
+      <rect x="8" y="55" width="32" height="5" fill="#8f555a"/>
+    </g>
+
+    <style>
+      text { font-family: Inter, ui-sans-serif, system-ui, -apple-system, BlinkMacSystemFont, "Segoe UI", sans-serif; fill: #111820; }
+      .title { font-size: 26.1px; font-weight: 720; letter-spacing: .2px; }
+      .subtitle { font-size: 11.3px; fill: #596570; }
+      .section { font-size: 18.3px; font-weight: 720; }
+      .heading { font-size: 14.8px; font-weight: 720; }
+      .body { font-size: 11.3px; }
+      .small { font-size: 9.6px; fill: #4f5b65; }
+      .tiny { font-size: 8.3px; fill: #59646d; }
+      .mono { font-family: "SFMono-Regular", Consolas, "Liberation Mono", monospace; }
+      .chip-text { font-size: 9.6px; font-family: "SFMono-Regular", Consolas, "Liberation Mono", monospace; text-anchor: middle; dominant-baseline: middle; }
+      .panel { stroke: #4f5962; stroke-width: 1.35; }
+      .dash { stroke-dasharray: 6 5; }
+      .flow { fill: none; stroke: #31383f; stroke-width: 1.8; marker-end: url(#arrow); }
+      .flow-red { fill: none; stroke: #df5961; stroke-width: 1.8; stroke-dasharray: 6 5; marker-end: url(#arrowRed); }
+      .prompt { fill: #eadff0; stroke: #725d82; stroke-width: 1.25; }
+      .marker { fill: #faeeee; stroke: #df6770; stroke-width: 1.25; }
+      .sep { fill: #d8e9ee; stroke: #537581; stroke-width: 1.25; }
+      .cached { fill: #e1e9ed; stroke: #70808a; stroke-width: 1.25; }
+      .current { fill: #cce5f0; stroke: #397c9b; stroke-width: 1.4; }
+      .person { fill: #eadff2; stroke: #7658a0; stroke-width: 1.2; }
+      .organization { fill: #f2e3a9; stroke: #a17a00; stroke-width: 1.2; }
+      .location { fill: #d6e8d5; stroke: #398148; stroke-width: 1.2; }
+    </style>
+  </defs>
+
+  <rect width="2100" height="1320" fill="#f1f2f2"/>
+  <g transform="translate(15 4.5) scale(1.15)">
+  <rect x="22" y="18" width="1756" height="1104" fill="none" stroke="#323a41" stroke-width="1.4" stroke-dasharray="6 5"/>
+
+  <!-- Title and legend -->
+  <text x="900" y="52" class="title" text-anchor="middle">GLiNER StreamingSpan Architecture</text>
+  <text x="900" y="75" class="subtitle" text-anchor="middle">Causal cache reuse · rolling span revision · complete entity snapshots</text>
+  <g transform="translate(1435 49)">
+    <rect x="0" y="-10" width="13" height="13" rx="2" class="cached"/>
+    <text x="20" y="1" class="tiny">cached</text>
+    <rect x="79" y="-10" width="13" height="13" rx="2" class="current"/>
+    <text x="99" y="1" class="tiny">new</text>
+    <line x1="145" y1="-3" x2="171" y2="-3" stroke="#df5961" stroke-width="2" stroke-dasharray="6 4"/>
+    <text x="179" y="1" class="tiny">rescored</text>
+  </g>
+
+  <!-- Prompt on the left -->
+  <text x="435" y="108" class="heading" text-anchor="middle" fill="#cf4e55">Target entity types prompt</text>
+  <line x1="322" y1="114" x2="548" y2="114" stroke="#cf4e55" stroke-width="1.5"/>
+  <rect x="55" y="122" width="760" height="153" fill="#f8e9e7" class="panel dash"/>
+  <text x="82" y="151" class="tiny" font-weight="700">SERIALIZED ON THE COLD PASS</text>
+  <g>
+    <rect x="80" y="172" width="100" height="43" rx="5" class="person"/>
+    <text x="130" y="193.5" class="chip-text">person</text>
+    <rect x="188" y="172" width="90" height="43" rx="5" class="marker"/>
+    <text x="233" y="193.5" class="chip-text">&lt;&lt;LABEL&gt;&gt;</text>
+    <rect x="286" y="172" width="140" height="43" rx="5" class="organization"/>
+    <text x="356" y="193.5" class="chip-text">organization</text>
+    <rect x="434" y="172" width="90" height="43" rx="5" class="marker"/>
+    <text x="479" y="193.5" class="chip-text">&lt;&lt;LABEL&gt;&gt;</text>
+    <rect x="532" y="172" width="110" height="43" rx="5" class="location"/>
+    <text x="587" y="193.5" class="chip-text">location</text>
+    <rect x="650" y="172" width="90" height="43" rx="5" class="marker"/>
+    <text x="695" y="193.5" class="chip-text">&lt;&lt;LABEL&gt;&gt;</text>
+  </g>
+  <text x="435" y="249" class="small" text-anchor="middle">label marker states → compact label encoder → cached label embeddings</text>
+
+  <!-- Separator between prompt and text -->
+  <text x="870" y="154" class="tiny" text-anchor="middle" font-weight="700">BOUNDARY</text>
+  <rect x="827" y="172" width="86" height="43" rx="5" class="sep"/>
+  <text x="870" y="193.5" class="chip-text">&lt;&lt;SEP&gt;&gt;</text>
+
+  <!-- Stream on the right -->
+  <text x="1345" y="108" class="heading" text-anchor="middle" fill="#3d8a31">Input text stream</text>
+  <line x1="1262" y1="114" x2="1428" y2="114" stroke="#3d8a31" stroke-width="1.5"/>
+  <rect x="925" y="122" width="820" height="153" fill="#e7f2e2" class="panel dash"/>
+  <text x="950" y="151" class="tiny" font-weight="700">CACHED TEXT · t−1</text>
+  <text x="1234" y="151" class="tiny" font-weight="700" fill="#28718f">APPEND · t</text>
+  <g>
+    <rect x="950" y="172" width="80" height="43" rx="5" class="cached"/>
+    <text x="990" y="193.5" class="chip-text">Alice</text>
+    <rect x="1038" y="172" width="100" height="43" rx="5" class="cached"/>
+    <text x="1088" y="193.5" class="chip-text">Johnson</text>
+    <rect x="1146" y="172" width="46" height="43" rx="5" class="cached"/>
+    <text x="1169" y="193.5" class="chip-text">…</text>
+    <line x1="1213" y1="158" x2="1213" y2="226" stroke="#3985a7" stroke-width="1.6" stroke-dasharray="5 4"/>
+    <rect x="1233" y="172" width="88" height="43" rx="5" class="current"/>
+    <text x="1277" y="193.5" class="chip-text">joined</text>
+    <rect x="1329" y="172" width="76" height="43" rx="5" class="current"/>
+    <text x="1367" y="193.5" class="chip-text">Acme</text>
+    <rect x="1413" y="172" width="46" height="43" rx="5" class="current"/>
+    <text x="1436" y="193.5" class="chip-text">in</text>
+    <rect x="1467" y="172" width="76" height="43" rx="5" class="current"/>
+    <text x="1505" y="193.5" class="chip-text">Paris</text>
+  </g>
+  <text x="1335" y="249" class="small" text-anchor="middle">the session may receive one word, sentence, or arbitrary text chunk per call</text>
+
+  <path d="M435 275L435 307" class="flow"/>
+  <path d="M1335 275L1335 307" class="flow"/>
+
+  <!-- Causal backbone -->
+  <rect x="55" y="309" width="1690" height="132" rx="7" fill="#cfe2f3" class="panel"/>
+  <text x="900" y="344" class="section" text-anchor="middle">1 · Causal Transformer Language Model</text>
+  <text x="900" y="368" class="body" text-anchor="middle">SmolLM2, Qwen, or another compatible decoder backbone</text>
+  <text x="900" y="388" class="small" text-anchor="middle">Cold pass: prompt + first chunk · warm pass: current tokens attend to cached history</text>
+
+  <rect x="430" y="398" width="207" height="31" rx="5" fill="#e9e5f0" stroke="#716485" stroke-width="1.15"/>
+  <text x="533.5" y="413.5" class="chip-text">KV cache · t−1</text>
+  <path d="M637 413.5L721 413.5" class="flow"/>
+  <rect x="723" y="398" width="354" height="31" rx="5" fill="#eef5f8" stroke="#628092" stroke-width="1.15"/>
+  <text x="900" y="413.5" class="chip-text">new token IDs + absolute positions</text>
+  <path d="M1077 413.5L1161 413.5" class="flow"/>
+  <rect x="1163" y="398" width="207" height="31" rx="5" fill="#e9e5f0" stroke="#716485" stroke-width="1.15"/>
+  <text x="1266.5" y="413.5" class="chip-text">updated KV cache · t</text>
+
+  <path d="M900 441L900 472" class="flow"/>
+
+  <!-- Decoder states -->
+  <text x="900" y="485" class="heading" text-anchor="middle">Reusable decoder token / word representations</text>
+  <text x="286" y="507" class="tiny" text-anchor="middle" font-weight="700">LABEL MARKER STATES · COLD ONLY</text>
+  <text x="842" y="507" class="tiny" text-anchor="middle" font-weight="700">CACHED WORD STATES</text>
+  <text x="1266" y="507" class="tiny" text-anchor="middle" font-weight="700" fill="#28718f">NEW WORD STATES</text>
+  <g>
+    <use href="#vecPrompt" x="145" y="518"/>
+    <use href="#vecPrompt" x="255" y="518"/>
+    <use href="#vecPrompt" x="365" y="518"/>
+    <rect x="483" y="518" width="48" height="64" fill="#eef6f7" stroke="#537581" stroke-width="1.4"/>
+    <rect x="491" y="527" width="32" height="8" fill="#aac5ce"/><rect x="491" y="539" width="32" height="8" fill="#62828e"/><rect x="491" y="551" width="32" height="8" fill="#d1e1e6"/><rect x="491" y="563" width="32" height="13" fill="#7898a3"/>
+
+    <use href="#vecCached" x="710" y="518"/>
+    <use href="#vecCached" x="785" y="518"/>
+    <use href="#vecCached" x="860" y="518"/>
+    <use href="#vecCached" x="935" y="518"/>
+    <line x1="1026" y1="507" x2="1026" y2="592" stroke="#3985a7" stroke-width="1.6" stroke-dasharray="5 4"/>
+    <use href="#vecNew" x="1095" y="518"/>
+    <use href="#vecNew" x="1170" y="518"/>
+    <use href="#vecNew" x="1245" y="518"/>
+    <use href="#vecNew" x="1320" y="518"/>
+  </g>
+  <g class="tiny" text-anchor="middle">
+    <text x="169" y="598">person</text><text x="279" y="598">organization</text><text x="389" y="598">location</text><text x="507" y="598">SEP</text>
+    <text x="734" y="598">Alice</text><text x="809" y="598">Johnson</text><text x="884" y="598">…</text><text x="959" y="598">past</text>
+    <text x="1119" y="598">joined</text><text x="1194" y="598">Acme</text><text x="1269" y="598">in</text><text x="1344" y="598">latest</text>
+  </g>
+  <path d="M1368 550L1448 550" class="flow"/>
+  <rect x="1450" y="520" width="235" height="59" rx="5" fill="#ece9f0" stroke="#7a7087" stroke-width="1.2"/>
+  <text x="1567.5" y="541" class="body" text-anchor="middle" font-weight="700">Session cache</text>
+  <text x="1567.5" y="561" class="small mono" text-anchor="middle">KV · words · labels · span scores</text>
+
+  <path d="M450 608L450 631" class="flow"/>
+  <path d="M1315 608L1315 631" class="flow"/>
+
+  <!-- Representation branches -->
+  <rect x="70" y="633" width="760" height="91" rx="5" fill="#f3b9bf" class="panel"/>
+  <text x="450" y="666" class="heading" text-anchor="middle">2A · Label representation layer</text>
+  <text x="450" y="691" class="body" text-anchor="middle">prompt slice through &lt;&lt;SEP&gt;&gt; → compact context encoder → one vector per &lt;&lt;LABEL&gt;&gt;</text>
+  <text x="450" y="710" class="tiny" text-anchor="middle">computed on the cold pass and reused for every warm append</text>
+
+  <rect x="900" y="633" width="830" height="91" rx="5" fill="#f3b9bf" class="panel"/>
+  <text x="1315" y="666" class="heading" text-anchor="middle">2B · Rolling span representation layer</text>
+  <text x="1315" y="691" class="body" text-anchor="middle">MarkerV2(start ⊕ end ⊕ latest) · width ≤ max_width</text>
+  <text x="1315" y="710" class="tiny" text-anchor="middle">all new endings + recent endings where latest − end ≤ right_context_width</text>
+
+  <path d="M450 724L450 751" class="flow"/>
+  <path d="M1315 724L1315 751" class="flow"/>
+
+  <!-- Label and span embeddings -->
+  <text x="450" y="760" class="small" text-anchor="middle" font-weight="700">CACHED LABEL EMBEDDINGS</text>
+  <g>
+    <use href="#vecPrompt" x="205" y="773"/>
+    <use href="#vecPrompt" x="425" y="773"/>
+    <use href="#vecPrompt" x="645" y="773"/>
+    <rect x="180" y="844" width="98" height="25" rx="3" class="person"/>
+    <text x="229" y="856.5" class="chip-text">ℓ_person</text>
+    <rect x="376" y="844" width="146" height="25" rx="3" class="organization"/>
+    <text x="449" y="856.5" class="chip-text">ℓ_organization</text>
+    <rect x="620" y="844" width="98" height="25" rx="3" class="location"/>
+    <text x="669" y="856.5" class="chip-text">ℓ_location</text>
+  </g>
+
+  <text x="1315" y="760" class="small" text-anchor="middle" font-weight="700">ROLLING SPAN EMBEDDINGS</text>
+  <g>
+    <use href="#vecRevised" x="978" y="773"/>
+    <use href="#vecNew" x="1110" y="773"/>
+    <use href="#vecNew" x="1242" y="773"/>
+    <use href="#vecNew" x="1374" y="773"/>
+    <use href="#vecNew" x="1506" y="773"/>
+    <use href="#vecNew" x="1638" y="773"/>
+  </g>
+  <g class="tiny" text-anchor="middle">
+    <text x="1002" y="850" fill="#c84f56">(0,1) revised</text>
+    <text x="1134" y="850">(1,3)</text>
+    <text x="1266" y="850">(3,4)</text>
+    <text x="1398" y="850">(4,4)</text>
+    <text x="1530" y="850">(4,6)</text>
+    <text x="1662" y="850">(6,6)</text>
+  </g>
+
+  <path d="M450 869L450 886" class="flow"/>
+  <path d="M1100 869L1100 886" class="flow"/>
+
+  <!-- Similarity matrix -->
+  <rect x="70" y="888" width="1055" height="165" fill="#ffffff" fill-opacity="0.42" class="panel"/>
+  <text x="610" y="913" class="heading" text-anchor="middle">3 · Entity type × span similarity</text>
+  <text x="1097" y="912" class="tiny" text-anchor="end">illustrative probabilities</text>
+
+  <g class="tiny" text-anchor="middle">
+    <text x="405" y="936">Alice Johnson</text>
+    <text x="555" y="936">Johnson … joined</text>
+    <text x="705" y="936">Acme</text>
+    <text x="855" y="936">Acme in Paris</text>
+    <text x="1005" y="936">Paris</text>
+  </g>
+
+  <rect x="115" y="945" width="185" height="26" class="person"/>
+  <text x="207.5" y="958" class="chip-text">person</text>
+  <rect x="115" y="974" width="185" height="26" class="organization"/>
+  <text x="207.5" y="987" class="chip-text">organization</text>
+  <rect x="115" y="1003" width="185" height="26" class="location"/>
+  <text x="207.5" y="1016" class="chip-text">location</text>
+
+  <g class="chip-text">
+    <!-- person row -->
+    <rect x="330" y="945" width="150" height="26" fill="#d9c9e9" stroke="#63717a"/><text x="405" y="958">0.93</text>
+    <rect x="480" y="945" width="150" height="26" fill="#eef2f3" stroke="#63717a"/><text x="555" y="958">0.21</text>
+    <rect x="630" y="945" width="150" height="26" fill="#f6f7f7" stroke="#63717a"/><text x="705" y="958">0.03</text>
+    <rect x="780" y="945" width="150" height="26" fill="#f5f6f6" stroke="#63717a"/><text x="855" y="958">0.05</text>
+    <rect x="930" y="945" width="150" height="26" fill="#f7f8f8" stroke="#63717a"/><text x="1005" y="958">0.02</text>
+    <!-- organization row -->
+    <rect x="330" y="974" width="150" height="26" fill="#f6f7f7" stroke="#63717a"/><text x="405" y="987">0.04</text>
+    <rect x="480" y="974" width="150" height="26" fill="#f2f3f3" stroke="#63717a"/><text x="555" y="987">0.10</text>
+    <rect x="630" y="974" width="150" height="26" fill="#eedb8d" stroke="#63717a"/><text x="705" y="987">0.88</text>
+    <rect x="780" y="974" width="150" height="26" fill="#f3e9c4" stroke="#63717a"/><text x="855" y="987">0.44</text>
+    <rect x="930" y="974" width="150" height="26" fill="#f7f8f8" stroke="#63717a"/><text x="1005" y="987">0.02</text>
+    <!-- location row -->
+    <rect x="330" y="1003" width="150" height="26" fill="#f6f7f7" stroke="#63717a"/><text x="405" y="1016">0.03</text>
+    <rect x="480" y="1003" width="150" height="26" fill="#f4f5f5" stroke="#63717a"/><text x="555" y="1016">0.08</text>
+    <rect x="630" y="1003" width="150" height="26" fill="#f5f6f6" stroke="#63717a"/><text x="705" y="1016">0.06</text>
+    <rect x="780" y="1003" width="150" height="26" fill="#e8f0e7" stroke="#63717a"/><text x="855" y="1016">0.33</text>
+    <rect x="930" y="1003" width="150" height="26" fill="#c9e2c8" stroke="#63717a"/><text x="1005" y="1016">0.91</text>
+  </g>
+
+  <path d="M1125 970L1148 970" class="flow"/>
+
+  <!-- Revision and output -->
+  <rect x="1150" y="888" width="580" height="165" fill="#dcebdc" class="panel"/>
+  <text x="1440" y="917" class="heading" text-anchor="middle">4 · Revise history and decode</text>
+  <text x="1180" y="946" class="body">• matching (start, end): replace the recent score</text>
+  <text x="1180" y="970" class="body">• new boundary: append → threshold → flat / nested NER</text>
+  <line x1="1180" y1="983" x2="1700" y2="983" stroke="#7d9680" stroke-width="1"/>
+  <text x="1180" y="1002" class="tiny" font-weight="700">COMPLETE SNAPSHOT · t</text>
+  <rect x="1180" y="1010" width="225" height="30" rx="4" fill="#f8f7fb" stroke="#7658a0" stroke-width="1.2"/>
+  <circle cx="1196" cy="1025" r="4" fill="#7658a0"/>
+  <text x="1300" y="1025" class="chip-text">Alice Johnson · person .93</text>
+  <rect x="1420" y="1010" width="145" height="30" rx="4" fill="#fffaf0" stroke="#a17a00" stroke-width="1.2"/>
+  <circle cx="1436" cy="1025" r="4" fill="#a17a00"/>
+  <text x="1498" y="1025" class="chip-text">Acme · org .88</text>
+  <rect x="1580" y="1010" width="130" height="30" rx="4" fill="#f5faf5" stroke="#398148" stroke-width="1.2"/>
+  <circle cx="1596" cy="1025" r="4" fill="#398148"/>
+  <text x="1649" y="1025" class="chip-text">Paris · loc .91</text>
+
+  <!-- Compact operational note -->
+  <text x="900" y="1082" class="body" text-anchor="middle" font-weight="650">Every append reuses KV, word, and label caches · only rolling candidates are rescored · the result is the full current snapshot</text>
+  <text x="900" y="1106" class="small mono" text-anchor="middle">StreamingBatch  ·  AsyncStreamingEngine  ·  inference(session_id=…)</text>
+  </g>
+</svg>
diff --git a/docs/index.md b/docs/index.md
index e2285b26..1c1dbfd1 100644
--- a/docs/index.md
+++ b/docs/index.md
@@ -1,6 +1,6 @@
 # GLiNER Documentation
 
-**GLiNER** is a framework for training and deploying Named Entity Recognition (NER) models that can identify any entity type using bidirectional transformer encoders (BERT-like). Beyond standard NER, GLiNER supports multiple tasks including joint entity and relation extraction through specialized architectures. It provides a practical alternative to both traditional NER models, which are limited to predefined entity types, and Large Language Models (LLMs), which offer flexibility but require significant computational resources.
+**GLiNER** is a framework for training and deploying Named Entity Recognition (NER) models that can identify any entity type. Its architectures include bidirectional transformer encoders, scalable bi-encoders, relation extraction models, and a causal StreamingSpan model for incremental text. It provides a practical alternative to both traditional NER models, which are limited to predefined entity types, and Large Language Models (LLMs), which offer flexibility but require significant computational resources.
 
 This documentation includes installation guides, tutorials, advanced topics, and full API reference.
 ```{toctree}
@@ -10,11 +10,13 @@ This documentation includes installation guides, tutorials, advanced topics, and
 intro
 instalation
 quickstart
+input_limits
 usage
+streaming
 configs
 training
 architectures
-add_custom_architectures
+add_custom_architecture
 convert_to_onnx
 serving
 ```
@@ -24,6 +26,7 @@ serving
 :caption: API Reference
 
 api/gliner.model
+api/gliner.streaming
 api/gliner.config
 api/gliner.training
 api/gliner.modeling
@@ -32,4 +35,4 @@ api/gliner.evaluation
 api/gliner.onnx
 api/gliner.decoding
 api/gliner.utils
-```
\ No newline at end of file
+```
diff --git a/docs/input_limits.md b/docs/input_limits.md
new file mode 100644
index 00000000..2ff23fa7
--- /dev/null
+++ b/docs/input_limits.md
@@ -0,0 +1,217 @@
+# Input limits and truncation
+
+GLiNER does not automatically split a long document into windows. For ordinary
+stateless inference, each input that exceeds the checkpoint's `config.max_len`
+is reduced to a prefix. In GLiNER 0.2.27 this produces a `UserWarning`, but the
+prediction call still succeeds and its return value does not say that part of
+the input was skipped.
+
+This behavior applies to span, token, bi-encoder, decoder, and relation
+extraction models. Cached StreamingSpan sessions are the exception described
+under **Architecture-specific limits** below.
+
+## What `config.max_len` counts
+
+`config.max_len` is the maximum number of **text tokens produced by GLiNER's
+word splitter**. It is not a character count, a whitespace-word count, a
+transformer subword count, or the total prompt-plus-text sequence length.
+
+For raw-text inference, the processing order is:
+
+```text
+raw text
+  -> WordsSplitter(config.words_splitter_type)
+  -> N text tokens
+  -> if N > config.max_len: warn and keep tokens[:config.max_len]
+  -> add the entity/relation-type prompt
+  -> transformer subword tokenization
+  -> model
+```
+
+The default `whitespace` splitter also separates punctuation. For example,
+`"Acme, Inc."` becomes four splitter tokens (`Acme`, `,`, `Inc`, `.`), even
+though a simple `text.split()` returns two strings. Other
+`words_splitter_type` values segment text differently. Always inspect the
+loaded checkpoint rather than assuming the configuration default:
+
+```python
+print(model.config.max_len)
+print(model.config.words_splitter_type)
+```
+
+The base configuration defaults to `max_len=384`, but checkpoints can save a
+different value. For example,
+`EmergentMethods/gliner_medium_news-v2.1` saves `max_len=296`.
+
+## Do labels reduce the text budget?
+
+At the `config.max_len` stage, **no**. GLiNER splits and truncates the text
+first, then prepends or separately encodes the entity-type prompt. Ten labels
+and one hundred labels therefore receive the same `config.max_len` allowance
+for text.
+
+There is a second, independent limit to consider. A transformer sees
+subtokens, and uni-encoder architectures put the label prompt and retained text
+in one transformer sequence. Prompt subtokens therefore consume part of any
+finite tokenizer or backbone context capacity. A finite tokenizer limit can
+truncate the text tail; a backbone limit can instead reject an oversized
+combined sequence. Either way, adding labels reduces the remaining combined
+headroom even though it does not change `config.max_len`.
+
+GLiNER calls the transformer tokenizer with `truncation=True` but without an
+explicit tokenizer `max_length`, so the tokenizer's registered limit and
+truncation side determine tokenizer-level behavior. Some tokenizers do not
+register a finite limit; Transformers may then warn that no maximum was
+provided and perform no tokenizer-level truncation. That does not prove that
+the backbone supports an unlimited sequence.
+
+## Behavior when text exceeds `max_len`
+
+For each input independently, when `num_tokens > model.config.max_len`, GLiNER
+0.2.27:
+
+1. emits a `UserWarning` such as
+   `Sentence of length 987 has been truncated to 296`;
+2. retains only the first `max_len` splitter tokens;
+3. runs inference normally on that prefix; and
+4. returns ordinary-looking predictions with no truncation metadata.
+
+Text after the retained prefix is never presented to the model. Entities there
+cannot be returned, and entities crossing the boundary are incomplete. The
+same prefix truncation occurs during training when a pre-tokenized example is
+too long.
+
+Python warning filters control whether the warning is visible. With the default
+filter, repeated warnings from the same location may be shown only once per
+process. Warnings may also bypass an application's structured logs. Do not use
+the presence or absence of the warning as a per-request completeness signal.
+
+As of 0.2.27, `predict_entities` and `inference` do not provide a
+`return_truncation_info` result and do not have a `truncation="error"` mode.
+[Issue #231](https://github.com/urchade/GLiNER/issues/231) tracks the request for
+fail-on-truncation behavior.
+
+## Production preflight without processor internals
+
+Use the public `prepare_batch` stage to count exactly the splitter tokens that
+stateless inference will receive. This avoids depending on
+`model.data_processor.words_splitter`:
+
+```python
+def truncation_info(model, text, labels):
+    prepared = model.prepare_batch(text, labels)
+    num_tokens = len(prepared["tokens"][0]) if prepared["tokens"] else 0
+    max_len = model.config.max_len
+    return {
+        "truncated": num_tokens > max_len,
+        "num_tokens": num_tokens,
+        "max_len": max_len,
+    }
+
+
+info = truncation_info(model, text, labels)
+if info["truncated"]:
+    raise ValueError(
+        f"GLiNER input has {info['num_tokens']} tokens; "
+        f"the model limit is {info['max_len']}"
+    )
+
+entities = model.predict_entities(text, labels)
+```
+
+For a batch, `prepared["tokens"]` contains one token list per non-empty input;
+`prepared["valid_to_orig_idx"]` maps those lists back to the original batch
+indices. Emit the resulting information in the service response or reject the
+request before inference, according to the service contract.
+
+This preflight covers `config.max_len`; it does not measure a combined
+uni-encoder subword sequence, an inference-packing limit, or a cached streaming
+session's remaining context.
+
+## Processing long documents
+
+For full-document coverage, split the text into overlapping windows no longer
+than `config.max_len`, predict each window, shift its character offsets back to
+document coordinates, and reconcile duplicate predictions from overlaps. The
+public preparation result includes the exact token-to-character maps needed to
+make splitter-aligned windows:
+
+```python
+def iter_gliner_windows(model, text, labels, overlap):
+    prepared = model.prepare_batch(text, labels)
+    if not prepared["tokens"]:
+        return
+
+    tokens = prepared["tokens"][0]
+    starts = prepared["start_token_map"][0]
+    ends = prepared["end_token_map"][0]
+    window_size = model.config.max_len
+
+    if not 0 <= overlap < window_size:
+        raise ValueError("overlap must be in [0, model.config.max_len)")
+
+    step = window_size - overlap
+    for first in range(0, len(tokens), step):
+        last = min(first + window_size, len(tokens))
+        char_start = starts[first]
+        char_end = ends[last - 1]
+        yield char_start, text[char_start:char_end]
+        if last == len(tokens):
+            break
+```
+
+For span models, an overlap of at least `max_width - 1` tokens ensures that an
+entity no wider than `max_width` is fully contained in some window. A larger
+overlap may be useful for contextual accuracy, and token-level models may need
+a task-specific overlap because their entity length is not bounded by
+`max_width`. Relation extraction also needs application-specific merging;
+relations whose endpoints never occur together in a window cannot be inferred.
+
+GLiNER does not merge window outputs for you. When shifting an entity from a
+window beginning at `char_start`, add `char_start` to both `entity["start"]` and
+`entity["end"]`. A common overlap policy is to group predictions by
+`(start, end, label)` and retain the highest score.
+
+## Changing `max_len`
+
+The loader's `max_length` argument overrides the saved `config.max_len`:
+
+```python
+model = GLiNER.from_pretrained(
+    "urchade/gliner_small-v2.1",
+    max_length=512,
+)
+```
+
+The name difference is intentional: `max_length=` at load time writes
+`model.config.max_len`. Passing `max_length` or `truncation` to
+`predict_entities` is not a supported way to change the word-level limit.
+
+Increasing this value does not resize the backbone context, change the
+transformer tokenizer's `model_max_length`, or guarantee quality beyond the
+lengths used to train the checkpoint. Subword expansion and the label prompt
+can make the combined sequence longer than the word-token count suggests, and
+longer inputs require more memory. Prefer windowing unless the checkpoint and
+backbone are known to support the larger value.
+
+## Architecture-specific limits
+
+| Architecture or path | How the label prompt is encoded | Effective-limit notes |
+|---|---|---|
+| UniEncoderSpan and UniEncoderToken | Prompt and text share one backbone sequence | `config.max_len` is text-only, but a finite backbone/tokenizer subword limit includes the prompt |
+| UniEncoder span/token decoders | Main encoder prompt and text share a sequence; generated labels use an auxiliary decoder | The main encoder has the same two-stage limits as other uni-encoders; the decoder has its own generation limits |
+| UniEncoder relation extraction | Entity labels, relation labels, and text share one sequence | Both prompt types can consume the combined backbone context, but neither changes the first-stage text-only `config.max_len` check |
+| BiEncoderSpan and BiEncoderToken | Text and labels are encoded separately | Label count does not consume the text encoder sequence; each encoder still has its own tokenizer/backbone limit |
+| StreamingSpan without `session_id` | Prompt and text share one causal sequence | Uses the normal stateless `config.max_len` prefix truncation, followed by the causal backbone limit |
+| Cached StreamingSpan session | The initial prompt and all appended text share the decoder context | Stateless `config.max_len` truncation is bypassed; exceeding the smaller of `max_cache_length` and the decoder's native limit raises `ValueError` instead of dropping old text |
+
+Inference packing adds another independent setting:
+`InferencePackingConfig.max_length` is measured in already-tokenized backbone
+token IDs, not splitter tokens. If one encoded request is longer than that
+value, the current packer keeps its first `max_length` token IDs without the
+`config.max_len` warning. Set the packing limit for the complete encoded
+request (including a uni-encoder prompt), or disable packing for requests that
+may exceed it.
+
+Finally, `max_width` is not an input-length limit. It controls the widest
+candidate entity span, in splitter tokens, for span-based architectures.
diff --git a/docs/serving.md b/docs/serving.md
index 8193a703..8855f1f6 100644
--- a/docs/serving.md
+++ b/docs/serving.md
@@ -69,6 +69,15 @@ outputs = client.predict(
 
 Network or server errors surface as `gliner.serve.client.GLiNERClientError`.
 
+:::{warning}
+The prediction response does not include a truncation flag. `--max-model-len`
+overrides the loaded model's text-only `config.max_len`; requests above it can
+still return HTTP success after GLiNER keeps only the prefix and emits a Python
+warning inside the replica. If the service contract requires full-input
+coverage, validate or window requests explicitly. See [Input limits and
+truncation](input_limits.md) for a preflight that uses the public model API.
+:::
+
 **HTTP request (no client library):**
 ```bash
 curl -X POST http://localhost:8000/gliner \
diff --git a/docs/streaming.md b/docs/streaming.md
new file mode 100644
index 00000000..5af8da12
--- /dev/null
+++ b/docs/streaming.md
@@ -0,0 +1,367 @@
+# StreamingSpan
+
+StreamingSpan is GLiNER's architecture for named entity recognition over text
+that arrives incrementally. It uses a causal decoder as the text backbone,
+retains reusable state between chunks, and revises recent span predictions when
+new right context becomes available.
+
+Use the
+[knowledgator/gliner-stream-pii-v1.0](https://huggingface.co/knowledgator/gliner-stream-pii-v1.0)
+checkpoint to get started with streaming PII detection. `GLiNER.from_pretrained`
+reads `model_type="gliner_streaming_span"` from the checkpoint and selects
+`StreamingSpanGLiNER` automatically.
+
+```python
+from gliner import GLiNER
+
+model = GLiNER.from_pretrained("knowledgator/gliner-stream-pii-v1.0")
+model.eval()
+```
+
+## Quick start
+
+Pass one chunk at a time with a stable session ID. The chunks are concatenated
+exactly as supplied, so retain spaces and punctuation at chunk boundaries.
+
+```python
+labels = ["person", "email address", "phone number"]
+session_id = "support-call-42"
+chunks = [
+    "Customer Alice Johnson ",
+    "can be reached at alice@example.com ",
+    "or +1 202-555-0147.",
+]
+
+try:
+    for chunk in chunks:
+        snapshot = model.inference(
+            [chunk],
+            labels,
+            session_id=[session_id],
+            threshold=0.5,
+        )[0]
+        print(snapshot)
+finally:
+    model.clear_session(session_id)
+```
+
+Each `snapshot` is the complete set of entities currently active for the
+accumulated session text. It is not a list of only the entities detected in the
+latest chunk. An entity has the same shape as an ordinary GLiNER prediction:
+
+```python
+{
+    "start": 9,           # document-relative, inclusive character offset
+    "end": 22,            # document-relative, exclusive character offset
+    "text": "Alice Johnson",
+    "label": "person",
+    "score": 0.93,
+}
+```
+
+Scores and even the active boundaries may change between snapshots as more
+context arrives. Consumers that need an event stream should diff consecutive
+snapshots by `(start, end, label)`.
+
+## Architecture
+
+![GLiNER StreamingSpan architecture](images/gliner-streaming-architecture.svg)
+
+StreamingSpan uses the following cold and warm paths:
+
+1. On the first append, labels and text are serialized as
+   `label<<LABEL>>...<<SEP>>text` and passed through the causal decoder.
+2. A compact label context encoder processes only the prompt through
+   `<<SEP>>`. Each `<<LABEL>>` state becomes an entity-type representation,
+   which is cached.
+3. Text subtokens are pooled into word representations. The model constructs
+   span representations and scores them against the label representations.
+4. On later appends, only the new decoder tokens are evaluated. Cached KV,
+   label, and word states are reused and extended.
+5. Scores for new and recently revisited span boundaries are merged into the
+   session's score history, then the complete history is decoded.
+
+The default `markerV2` span layer combines the candidate's start word, end
+word, and latest visible word. Candidate width is bounded by `max_width`.
+`right_context_width` determines how far behind the newest word an existing
+span may be revisited; it defaults to `max_width`. Setting it to `0` keeps old
+span scores fixed.
+
+An optional `span_encoder_config` adds a dense-input DeBERTa-v2, ModernBERT, or
+RNN encoder before span construction. A bidirectional span encoder can change
+every historical word representation, so the model re-scores all historical
+span candidates after each append. This still reuses causal decoder states;
+`recompute=True` is the option that rebuilds the entire accumulated sequence.
+
+For a component-level explanation, see
+[GLiNER StreamingSpan](architectures.md) in the architecture guide.
+
+## Choosing an inference mode
+
+StreamingSpan provides four inference surfaces. They share prediction
+semantics but differ in who owns the cache and how work is batched.
+
+| Surface | Cache ownership | Scheduling | Best fit |
+|---|---|---|---|
+| `predict_entities` or `inference` without `session_id` | None | Ordinary GLiNER batching | Complete, independent texts |
+| `inference(..., session_id=[...])` | One cache per ID on the model | Compatible sessions are batched per call | Flexible synchronous session sets |
+| `create_streaming_batch(...)` | One persistent batched cache on the handle | Fixed rows advance together | Stable groups with aligned arrival cadence |
+| `create_async_streaming_engine(...)` | One cache per ID on the model | Dynamic microbatching | Concurrent, independently arriving streams |
+
+### Stateless inference
+
+The architecture can process complete texts without retaining state. Omitting
+`session_id` delegates to the ordinary GLiNER inference pipeline.
+
+```python
+entities = model.predict_entities(
+    "Alice Johnson's email is alice@example.com.",
+    labels,
+    threshold=0.5,
+)
+
+batch = model.inference(
+    ["Alice called.", "Bob emailed."],
+    labels,
+    batch_size=2,
+)
+```
+
+Use stateless inference when the whole input is already available. It avoids
+session lifecycle and cache-memory concerns.
+
+### Flexible synchronous sessions
+
+Supplying `session_id` turns each input into an append operation. Use one
+stable, non-empty session ID per text; IDs must be unique within one call.
+
+```python
+session_ids = ["call-a", "call-b"]
+
+first = model.inference(
+    ["Alice Johnson ", "Bob Smith "],
+    labels,
+    session_id=session_ids,
+    batch_size=8,
+)
+second = model.inference(
+    ["shared her email.", "shared his number."],
+    labels,
+    session_id=session_ids,
+    batch_size=8,
+)
+
+model.clear_session(session_ids)
+```
+
+Cold sessions are batched together. Warm sessions with the same cached decoder
+length are also batched together; different lengths are processed in separate
+groups. `batch_size` limits how many append requests enter a group at once.
+Session order may change between calls because state is addressed by ID.
+
+A blank chunk returns `[]` and does not advance that session. Labels must stay
+in the same order for the lifetime of a session. To use a different label set,
+clear the session or append non-empty text with `recompute=True`.
+
+### Persistent fixed-order batches
+
+When the same streams advance together, a persistent batch avoids repeatedly
+stacking and splitting their historical KV caches. The session-to-row mapping
+and labels are immutable for the handle's lifetime.
+
+```python
+with model.create_streaming_batch(
+    session_ids=["call-a", "call-b"],
+    labels=labels,
+) as stream:
+    first = stream.append(["Alice Johnson ", "Bob Smith "])
+    second = stream.append(["shared her email.", "shared his number."])
+
+    # Keep call-a unchanged while call-b advances.
+    third = stream.append(["", " It is +1 202-555-0147."])
+```
+
+`append` returns one complete snapshot per row. An empty row retains its
+existing snapshot. If every row is empty, no model forward is performed.
+
+The handle offers two lifecycle operations:
+
+- `reset()` discards the complete batched cache but keeps the handle, row
+  mapping, and labels usable.
+- `close()` releases the cache and permanently closes the handle. The context
+  manager calls it automatically.
+
+Passing `recompute=True` to `append` rebuilds every row from its complete
+accumulated text. It is a batch-wide operation.
+
+### Asynchronous dynamic microbatching
+
+The asynchronous engine collects independently arriving appends for a short
+window and sends compatible sessions through batched forwards. Calls for the
+same session remain FIFO ordered, while different session IDs can be submitted
+concurrently.
+
+```python
+import asyncio
+
+
+async def consume(engine, session_id, chunks):
+    latest = []
+    async for latest in engine.stream(session_id, chunks, labels, threshold=0.5):
+        print(session_id, latest)
+    return latest
+
+
+async def main():
+    async with model.create_async_streaming_engine(
+        max_batch_size=32,
+        batch_wait_timeout_ms=2,
+        queue_capacity=4096,
+    ) as engine:
+        results = await asyncio.gather(
+            consume(engine, "call-a", ["Alice ", "shared her email."]),
+            consume(engine, "call-b", ["Bob ", "shared his number."]),
+        )
+        await engine.clear_session("call-a")
+        await engine.clear_session("call-b")
+        return results
+
+
+snapshots = asyncio.run(main())
+```
+
+The engine runs model work outside the event-loop thread and uses one worker per
+engine, avoiding competing cache mutations and CUDA launches.
+`max_batch_size` caps a microbatch, `batch_wait_timeout_ms` trades a small amount
+of latency for more batching opportunities, and `queue_capacity` applies
+backpressure to producers. Leaving the async context drains queued work and
+closes the scheduler.
+
+Use `await engine.clear_session(id)` while an engine is active; it waits for
+earlier work on that session before removing the cache. Blank appends return
+`[]` without entering the queue or changing state.
+
+## Session and cache lifecycle
+
+A session retains more than the decoder's KV tensors:
+
+| Cached state | Purpose |
+|---|---|
+| Decoder KV and attention state | Lets new tokens attend to the prompt and historical text without re-encoding it |
+| Pooled word states | Supports spans that cross chunk boundaries and recent span rescoring |
+| Label representations | Avoids re-encoding an unchanged entity-type prompt |
+| Text, tokens, and character offsets | Produces document-relative entity offsets and text |
+| Span-score history | Preserves old candidates and replaces scores for revised boundaries |
+
+KV, word, and label tensors stay with the model device. Historical span scores
+are kept on CPU so the prediction history does not consume progressively more
+accelerator memory.
+
+For sessions created through `inference(..., session_id=...)` or the async
+engine, use:
+
+```python
+model.clear_session("call-a")             # one session
+model.clear_session(["call-b", "call-c"])  # several sessions
+model.clear_sessions()                    # every model-owned session
+print(model.session_count)
+```
+
+Persistent `StreamingBatch` handles own their cache separately and are not
+included in `model.session_count`; use the handle's `reset()` or `close()`.
+Model-owned sessions have no automatic TTL or LRU eviction.
+
+### Context limits
+
+The effective session limit is the smaller of `max_cache_length`, when set, and
+the decoder backbone's native positional limit. The serialized label prompt
+also consumes decoder positions. Streaming sessions disable preprocessing
+truncation, and the implementation does not silently evict old context. An
+append that would exceed the limit raises `ValueError`; finish and clear the
+session, reset its batch, or start a new session.
+
+This differs from stateless StreamingSpan and other GLiNER inference, where
+`config.max_len` is a text-only splitter-token limit and an overlong input is
+reduced to a prefix. See [Input limits and truncation](input_limits.md).
+
+A persistent batch stores the padded physical width of every append. Group
+streams with similar chunk sizes to reduce padded cache positions and avoid
+reaching the physical context limit earlier than necessary.
+
+## Prediction and revision controls
+
+The streaming APIs accept the standard span-decoding controls:
+
+| Option | Effect |
+|---|---|
+| `threshold` | Minimum entity score; defaults to `0.5` |
+| `flat_ner` | If `True`, choose non-overlapping entities; set `False` for nested NER |
+| `multi_label` | Permit more than one label for a span |
+| `return_class_probs` | Add per-class probabilities to each returned entity |
+| `recompute` | Rebuild state from all accumulated text instead of incrementally appending |
+
+`packing_config`, `input_spans`, and external model-input tensors are not
+supported when `session_id` is supplied. They remain available to the stateless
+parent inference path where otherwise supported.
+
+Two model configuration fields control incremental revisions:
+
+- `max_width` is the maximum candidate span width in words.
+- `right_context_width` is the number of words behind the latest word whose
+  span endings are eligible for rescoring. `None` is normalized to `max_width`.
+
+A larger right-context window can improve revisions at the cost of scoring more
+candidates per append. `recompute=True` is useful as an occasional correctness
+check or when changing labels, but it forfeits incremental decoder savings for
+that call.
+
+## Chunking guidance
+
+- Prefer chunks that end at word, punctuation, or sentence boundaries. Splitting
+  one logical word across calls makes the word splitter treat the pieces as
+  separate streaming words.
+- Preserve boundary whitespace. `"Alice "` followed by `"joined"` reconstructs
+  `"Alice joined"`; `"Alice"` followed by `"joined"` reconstructs
+  `"Alicejoined"`.
+- Smaller chunks provide earlier updates but incur more Python, tokenization,
+  decoding, and scheduling overhead. Larger chunks improve throughput but delay
+  the first prediction.
+- Use stable, tenant-safe session IDs and always clear abandoned sessions.
+- Treat every response as replaceable state. Do not append snapshots directly
+  to a result list as though they were deltas.
+
+## Configuration and training
+
+StreamingSpan checkpoints use `model_type: gliner_streaming_span`. The main
+architecture-specific fields are:
+
+| Field | Description |
+|---|---|
+| `model_name` / `decoder_config` | Causal decoder backbone and its saved configuration |
+| `label_token` | Marker placed after each label; defaults to `<<LABEL>>` |
+| `sep_token` | Boundary between the entity prompt and text; defaults to `<<SEP>>` |
+| `labels_encoder_config` | DeBERTa-v2, ModernBERT, or RNN encoder for the compact prompt |
+| `span_mode` | Span representation; StreamingSpan defaults to `markerV2` |
+| `span_encoder_config` | Optional DeBERTa-v2, ModernBERT, or RNN word context encoder |
+| `subtoken_pooling` | `first`, `last`, `mean`, or `max` pooling into words |
+| `max_width` | Maximum entity span width in words |
+| `right_context_width` | Rolling span revision window |
+| `max_cache_length` | Optional upper bound on cached decoder positions |
+| `max_len` | Stateless preprocessing limit; cached sessions use the decoder context limit instead |
+
+Normally these values should come from the checkpoint rather than be overridden
+at inference time. To train or fine-tune the architecture, adapt
+`configs/config_streaming_span.yaml` and use the normal GLiNER training entry
+point:
+
+```bash
+python train.py --config configs/config_streaming_span.yaml
+```
+
+The data format and general trainer options are the same as for other
+span-based GLiNER models; see [Training](training.md) and
+[Configuration](configs.md). The executable
+[inference modes example](https://github.com/urchade/GLiNER/blob/main/examples/streaming_inference_modes.py)
+demonstrates every serving surface, while the
+[interactive streaming example](https://github.com/urchade/GLiNER/blob/main/examples/streaming_span.py)
+shows how to diff and render live snapshots.
diff --git a/docs/usage.md b/docs/usage.md
index ad5927d2..17103672 100644
--- a/docs/usage.md
+++ b/docs/usage.md
@@ -69,6 +69,16 @@ for entity in entities:
     print("---")
 ```
 
+:::{warning}
+Stateless GLiNER calls do not automatically window long documents. If the text
+contains more splitter tokens than the checkpoint's `model.config.max_len`,
+GLiNER 0.2.27 warns and predicts from only the retained prefix; the returned
+entities contain no truncation flag. Check the limit before inference or split
+the document into overlapping windows. See [Input limits and
+truncation](input_limits.md) for the exact token definition, a public preflight
+example, prompt-budget details, and architecture-specific behavior.
+:::
+
 ## Batch Processing
 
 For processing multiple texts efficiently, use the `inference` method:
@@ -1317,6 +1327,41 @@ Relevant knobs:
 Pseudo-labels are generated from the same `texts` used for compression, so one
 diverse in-domain corpus serves both roles.
 
+## StreamingSpan models
+
+StreamingSpan uses a causal text backbone and reuses its decoder, label, and
+word caches as text arrives. Load a StreamingSpan checkpoint normally; GLiNER
+selects the architecture from the saved configuration.
+
+```python
+from gliner import GLiNER
+
+model = GLiNER.from_pretrained("knowledgator/gliner-stream-pii-v1.0")
+labels = ["person", "email address", "phone number"]
+session_id = "document-a"
+
+try:
+    for chunk in ["Alice Johnson's email is ", "alice@example.com."]:
+        snapshot = model.inference(
+            [chunk],
+            labels,
+            session_id=[session_id],
+            threshold=0.5,
+        )[0]
+        print(snapshot)
+finally:
+    model.clear_session(session_id)
+```
+
+Every call returns the complete current snapshot, not only newly detected
+entities. Recent predictions may be added, updated, or removed as right context
+arrives. For aligned streams, use `model.create_streaming_batch(...)`; for
+independently arriving streams, use `model.create_async_streaming_engine(...)`.
+Ordinary `predict_entities` or `inference` calls without `session_id` remain
+stateless.
+
+See the [StreamingSpan guide](streaming.md) for all inference modes, cache
+lifecycle, architecture details, configuration, and operational guidance.
 
 ## Tips and Best Practices
 
@@ -1325,6 +1370,7 @@ diverse in-domain corpus serves both roles.
    - BiEncoder: Many entity types (50-200+)
    - Token-level: Long entity spans
    - Relation extraction: Knowledge graph construction
+   - StreamingSpan: Incremental text with reusable session state
 
 2. **Optimize threshold for your use case**:
    - High precision: threshold = 0.6-0.8
@@ -1399,4 +1445,4 @@ model = GLiNER.from_pretrained(
     "urchade/gliner_small-v2.1",
     map_location="cpu"
 )
-```
\ No newline at end of file
+```
diff --git a/examples/streaming_demo.py b/examples/streaming_demo.py
new file mode 100644
index 00000000..37c20d69
--- /dev/null
+++ b/examples/streaming_demo.py
@@ -0,0 +1,828 @@
+"""Beautiful word-by-word NER demo for a trained StreamingSpan checkpoint.
+
+Example:
+    python examples/streaming_span.py models/checkpoint-15000 --inbrowser
+"""
+
+from __future__ import annotations
+
+# This demo intentionally prints its startup URL and terminal fallback output.
+# Embedded CSS is intentionally kept one selector per line in several places.
+# ruff: noqa: E501, T201
+import time
+import argparse
+from html import escape
+from uuid import uuid4
+from typing import TYPE_CHECKING, Any
+from dataclasses import dataclass
+
+import torch
+
+from gliner import GLiNER
+
+if TYPE_CHECKING:
+    from collections.abc import Iterator
+
+DEFAULT_TEXT = "Alice Johnson joined Acme Corporation in London on Monday."
+DEFAULT_LABELS = "person, organization, location, date"
+
+
+def _new_session_id() -> str:
+    """Return a cache key for one browser session."""
+    return f"gradio-{uuid4().hex}"
+
+
+EXAMPLES = [
+    [
+        "Alice Johnson joined Acme Corporation in London on Monday.",
+        "person, organization, location, date",
+    ],
+    [
+        "Dr. Maya Chen presented the new malaria vaccine at the World Health Organization summit in Geneva.",
+        "person, job title, medicine, organization, event, location",
+    ],
+    [
+        "Apple CEO Tim Cook met Microsoft chairman Satya Nadella in Seattle on 14 March 2026.",
+        "company, person, job title, location, date",
+    ],
+    [
+        "Serena Williams won the Australian Open in Melbourne and later founded Serena Ventures.",
+        "person, competition, location, organization",
+    ],
+    [
+        "The European Space Agency launched the JUICE spacecraft from French Guiana toward Jupiter.",
+        "organization, spacecraft, location, planet",
+    ],
+    [
+        "PyTorch and TensorFlow are popular machine-learning frameworks used with Python on Linux.",
+        "software, programming language, operating system",
+    ],
+]
+
+# Accent, translucent background, dark foreground.
+LABEL_COLORS = [
+    ("#7c3aed", "#ede9fe", "#4c1d95"),
+    ("#0891b2", "#cffafe", "#164e63"),
+    ("#db2777", "#fce7f3", "#831843"),
+    ("#059669", "#d1fae5", "#064e3b"),
+    ("#d97706", "#fef3c7", "#78350f"),
+    ("#2563eb", "#dbeafe", "#1e3a8a"),
+    ("#dc2626", "#fee2e2", "#7f1d1d"),
+    ("#4f46e5", "#e0e7ff", "#312e81"),
+]
+
+
+@dataclass(frozen=True)
+class StreamStep:
+    """One completed word-level inference update."""
+
+    index: int
+    total: int
+    token: str
+    token_start: int
+    token_end: int
+    visible_text: str
+    entities: tuple[dict[str, Any], ...]
+    added: tuple[dict[str, Any], ...]
+    updated: tuple[dict[str, Any], ...]
+    removed: tuple[dict[str, Any], ...]
+
+
+def parse_labels(value: str) -> list[str]:
+    """Parse a comma-separated label list while preserving its order."""
+    labels = list(dict.fromkeys(label.strip() for label in value.split(",") if label.strip()))
+    if not labels:
+        raise ValueError("Provide at least one label, separated by commas.")
+    return labels
+
+
+def iter_word_chunks(model, text: str) -> Iterator[tuple[str, str, int, int]]:
+    """Yield one model-split word at a time without changing text offsets."""
+    token_batches, start_batches, end_batches = model.prepare_inputs([text])
+    tokens = token_batches[0]
+    token_starts = start_batches[0]
+    token_ends = end_batches[0]
+    previous_end = 0
+
+    for index, (token, token_start, token_end) in enumerate(zip(tokens, token_starts, token_ends)):
+        # Whitespace before a token belongs to that token's chunk. Include any
+        # trailing whitespace in the final chunk so concatenation recreates the
+        # original text exactly.
+        chunk_end = len(text) if index == len(tokens) - 1 else token_end
+        yield text[previous_end:chunk_end], token, token_start, token_end
+        previous_end = chunk_end
+
+
+def entity_key(entity: dict[str, Any]) -> tuple[int, int, str]:
+    """Return the stable identity of an entity prediction."""
+    return int(entity["start"]), int(entity["end"]), str(entity["label"])
+
+
+def _sorted_entities(entities) -> tuple[dict[str, Any], ...]:
+    return tuple(sorted(entities, key=lambda entity: (*entity_key(entity), -float(entity["score"]))))
+
+
+def stream_steps(
+    model,
+    text: str,
+    labels: list[str],
+    session_id: str,
+    threshold: float,
+    flat_ner: bool,
+    multi_label: bool,
+) -> Iterator[StreamStep]:
+    """Stream words through the compatibility ``inference(session_id=...)`` API."""
+    chunks = list(iter_word_chunks(model, text))
+    if not chunks:
+        raise ValueError("Text must contain at least one recognizable word.")
+
+    current_entities: dict[tuple[int, int, str], dict[str, Any]] = {}
+    visible_text = ""
+
+    model.clear_session(session_id)
+    try:
+        for index, (chunk, token, token_start, token_end) in enumerate(chunks, start=1):
+            visible_text += chunk
+            predictions = model.inference(
+                [chunk],
+                labels,
+                session_id=[session_id],
+                threshold=threshold,
+                flat_ner=flat_ner,
+                multi_label=multi_label,
+            )[0]
+
+            previous_entities = current_entities
+            step_entities = {entity_key(entity): entity for entity in predictions}
+            # Session inference returns the complete active prediction snapshot.
+            # Replacing it allows rolling reclassification to report removals
+            # and label changes as later context arrives.
+            current_entities = step_entities
+
+            added_keys = current_entities.keys() - previous_entities.keys()
+            removed_keys = previous_entities.keys() - current_entities.keys()
+            updated_keys = {
+                key
+                for key in current_entities.keys() & previous_entities.keys()
+                if float(current_entities[key]["score"]) != float(previous_entities[key]["score"])
+            }
+
+            yield StreamStep(
+                index=index,
+                total=len(chunks),
+                token=token,
+                token_start=token_start,
+                token_end=token_end,
+                visible_text=visible_text,
+                entities=_sorted_entities(current_entities.values()),
+                added=_sorted_entities(current_entities[key] for key in added_keys),
+                updated=_sorted_entities(current_entities[key] for key in updated_keys),
+                removed=_sorted_entities(previous_entities[key] for key in removed_keys),
+            )
+    finally:
+        model.clear_session(session_id)
+
+
+def _label_color(label: str, labels: list[str]) -> tuple[str, str, str]:
+    try:
+        index = labels.index(label)
+    except ValueError:
+        index = sum(label.encode("utf-8"))
+    return LABEL_COLORS[index % len(LABEL_COLORS)]
+
+
+def span_context_label(model) -> str:
+    """Return a user-facing name for the span-context strategy."""
+    return "bidirectional" if model.model.span_rep_layer.uses_bidirectional_context else "causal"
+
+
+def _label_legend(labels: list[str]) -> str:
+    chips = []
+    for label in labels:
+        accent, background, foreground = _label_color(label, labels)
+        chips.append(
+            '<span class="label-chip" '
+            f'style="--label-accent:{accent};--label-bg:{background};--label-fg:{foreground}">'
+            f"<i></i>{escape(label)}</span>"
+        )
+    return '<div class="label-legend">' + "".join(chips) + "</div>"
+
+
+def render_annotated_text(
+    text: str,
+    entities: tuple[dict[str, Any], ...],
+    labels: list[str],
+    current_word: tuple[int, int] | None = None,
+) -> str:
+    """Render streamed text with safe, overlap-aware entity highlighting."""
+    if not text:
+        body = (
+            '<div class="empty-stage"><span class="empty-orb">✦</span>'
+            "Press <strong>Start streaming</strong> to watch entities emerge.</div>"
+        )
+        return f'<section class="visual-card">{_label_legend(labels)}{body}</section>'
+
+    valid_entities = tuple(entity for entity in entities if 0 <= int(entity["start"]) < int(entity["end"]) <= len(text))
+    boundaries = {0, len(text)}
+    for entity in valid_entities:
+        boundaries.update((int(entity["start"]), int(entity["end"])))
+    if current_word is not None:
+        boundaries.update((max(0, current_word[0]), min(len(text), current_word[1])))
+    ordered_boundaries = sorted(boundary for boundary in boundaries if 0 <= boundary <= len(text))
+
+    fragments = []
+    for start, end in zip(ordered_boundaries, ordered_boundaries[1:]):
+        if start == end:
+            continue
+        active = [entity for entity in valid_entities if int(entity["start"]) < end and int(entity["end"]) > start]
+        is_current = current_word is not None and current_word[0] < end and current_word[1] > start
+        classes = ["text-fragment"]
+        styles = []
+        title = ""
+
+        if active:
+            active.sort(key=lambda entity: float(entity["score"]), reverse=True)
+            colors = [_label_color(str(entity["label"]), labels) for entity in active[:2]]
+            classes.append("entity-fragment")
+            styles.append(f"border-color:{colors[0][0]}")
+            if len(colors) == 1:
+                styles.append(f"background:{colors[0][1]}")
+                styles.append(f"color:{colors[0][2]}")
+            else:
+                styles.append(f"background:linear-gradient(135deg,{colors[0][1]} 0 50%,{colors[1][1]} 50% 100%)")
+                styles.append(f"color:{colors[0][2]}")
+            title = "; ".join(f"{entity['label']} · {float(entity['score']):.3f}" for entity in active)
+        if is_current:
+            classes.append("current-word")
+
+        style_attr = f' style="{";".join(styles)}"' if styles else ""
+        title_attr = f' title="{escape(title, quote=True)}"' if title else ""
+        fragments.append(f'<span class="{" ".join(classes)}"{style_attr}{title_attr}>{escape(text[start:end])}</span>')
+
+        ending_here = [entity for entity in valid_entities if int(entity["end"]) == end]
+        for entity in sorted(ending_here, key=lambda item: (int(item["start"]), str(item["label"]))):
+            accent, background, foreground = _label_color(str(entity["label"]), labels)
+            fragments.append(
+                '<span class="inline-label" '
+                f'style="--label-accent:{accent};--label-bg:{background};--label-fg:{foreground}">'
+                f"{escape(str(entity['label']))}<small>{float(entity['score']):.2f}</small></span>"
+            )
+
+    return (
+        '<section class="visual-card">'
+        '<div class="card-kicker"><span class="live-dot"></span>Live recognition</div>'
+        f"{_label_legend(labels)}"
+        f'<div class="stream-text">{"".join(fragments)}<span class="stream-caret"></span></div>'
+        "</section>"
+    )
+
+
+def render_word_rail(tokens: list[str], current_index: int = -1) -> str:
+    """Render all model-split words as a processed/current/pending timeline."""
+    word_chips = []
+    for index, token in enumerate(tokens):
+        if index < current_index:
+            state = "processed"
+        elif index == current_index:
+            state = "active"
+        else:
+            state = "pending"
+        word_chips.append(f'<span class="word-chip {state}"><small>{index + 1:02d}</small>{escape(token)}</span>')
+    return (
+        '<section class="word-rail-card"><div class="rail-heading">'
+        "<span>Model word stream</span><small>Every chip is one inference step</small></div>"
+        f'<div class="word-rail">{"".join(word_chips)}</div></section>'
+    )
+
+
+def render_entity_cards(entities: tuple[dict[str, Any], ...], labels: list[str]) -> str:
+    """Render the current entity set as confidence cards."""
+    if not entities:
+        cards = (
+            '<div class="no-entities"><span>⌁</span><div><strong>No entities yet</strong>'
+            "<small>The panel updates as soon as a span crosses the threshold.</small></div></div>"
+        )
+    else:
+        rendered = []
+        for entity in entities:
+            label = str(entity["label"])
+            accent, background, foreground = _label_color(label, labels)
+            score = max(0.0, min(1.0, float(entity["score"])))
+            rendered.append(
+                '<article class="entity-card" '
+                f'style="--label-accent:{accent};--label-bg:{background};--label-fg:{foreground}">'
+                '<div class="entity-card-top">'
+                f'<span class="entity-type">{escape(label)}</span><strong>{score:.1%}</strong></div>'
+                f'<div class="entity-value">{escape(str(entity["text"]))}</div>'
+                f'<div class="entity-offset">characters {int(entity["start"])}&ndash;{int(entity["end"])}</div>'
+                f'<div class="confidence-track"><i style="width:{score * 100:.1f}%"></i></div>'
+                "</article>"
+            )
+        cards = '<div class="entity-grid">' + "".join(rendered) + "</div>"
+
+    return (
+        '<section class="entities-section"><div class="section-heading">'
+        '<div><span class="eyebrow">Structured output</span><h3>Recognized entities</h3></div>'
+        f'<span class="entity-count">{len(entities)}</span></div>{cards}</section>'
+    )
+
+
+def render_status(
+    step: int,
+    total: int,
+    entity_count: int,
+    mode: str,
+    state: str = "ready",
+    message: str | None = None,
+) -> str:
+    """Render progress and cache state for the current stream."""
+    progress = 0 if total == 0 else min(100, round(step / total * 100))
+    state_label = {
+        "ready": "Ready to stream",
+        "streaming": "Streaming",
+        "complete": "Complete",
+        "stopped": "Stopped",
+        "error": "Could not continue",
+    }[state]
+    prompt_label = "Prompt pending" if step == 0 else "Prompt cached"
+    extra = f'<div class="status-message">{escape(message)}</div>' if message else ""
+    return (
+        f'<section class="status-card {state}">'
+        '<div class="status-top"><div class="status-title">'
+        f'<span class="status-icon"></span><strong>{state_label}</strong></div>'
+        f'<span class="status-percent">{progress}%</span></div>'
+        f'<div class="progress-track"><i style="width:{progress}%"></i></div>'
+        '<div class="metric-row">'
+        f"<span><strong>{step}</strong><small>of {total} words</small></span>"
+        f"<span><strong>{entity_count}</strong><small>entities</small></span>"
+        f"<span><strong>{escape(mode)}</strong><small>span context</small></span>"
+        f"<span><strong>{prompt_label}</strong><small>session KV cache</small></span>"
+        f"</div>{extra}</section>"
+    )
+
+
+def initial_view(model, text: str, labels_value: str):
+    """Build the four UI panels before inference starts."""
+    labels = parse_labels(labels_value)
+    chunks = list(iter_word_chunks(model, text))
+    if not chunks:
+        raise ValueError("Text must contain at least one recognizable word.")
+    tokens = [token for _, token, _, _ in chunks]
+    mode = span_context_label(model)
+    return (
+        render_status(0, len(tokens), 0, mode),
+        render_annotated_text("", (), labels),
+        render_word_rail(tokens),
+        render_entity_cards((), labels),
+    )
+
+
+def create_streaming_handler(model):
+    """Create a Gradio generator bound to one loaded model."""
+
+    def recognize(
+        text: str,
+        labels_value: str,
+        threshold: float,
+        delay: float,
+        nested_ner: bool,
+        multi_label: bool,
+        session_id: str,
+    ):
+        labels: list[str] = []
+        tokens: list[str] = []
+        mode = span_context_label(model)
+        last_step: StreamStep | None = None
+        try:
+            labels = parse_labels(labels_value)
+            chunks = list(iter_word_chunks(model, text))
+            if not chunks:
+                raise ValueError("Text must contain at least one recognizable word.")
+            tokens = [token for _, token, _, _ in chunks]
+
+            yield (
+                render_status(0, len(tokens), 0, mode),
+                render_annotated_text("", (), labels),
+                render_word_rail(tokens),
+                render_entity_cards((), labels),
+            )
+
+            for step in stream_steps(
+                model,
+                text,
+                labels,
+                session_id,
+                threshold,
+                flat_ner=not nested_ner,
+                multi_label=multi_label,
+            ):
+                last_step = step
+                state = "complete" if step.index == step.total else "streaming"
+                yield (
+                    render_status(step.index, step.total, len(step.entities), mode, state=state),
+                    render_annotated_text(
+                        step.visible_text,
+                        step.entities,
+                        labels,
+                        current_word=(step.token_start, step.token_end),
+                    ),
+                    render_word_rail(tokens, step.index - 1),
+                    render_entity_cards(step.entities, labels),
+                )
+                if delay and step.index != step.total:
+                    time.sleep(delay)
+        except Exception as error:
+            total = len(tokens)
+            completed = last_step.index if last_step is not None else 0
+            entities = last_step.entities if last_step is not None else ()
+            visible_text = last_step.visible_text if last_step is not None else ""
+            current_word = (last_step.token_start, last_step.token_end) if last_step is not None else None
+            safe_labels = labels or ["entity"]
+            yield (
+                render_status(completed, total, len(entities), mode, state="error", message=str(error)),
+                render_annotated_text(visible_text, entities, safe_labels, current_word=current_word),
+                render_word_rail(tokens, completed - 1),
+                render_entity_cards(entities, safe_labels),
+            )
+        finally:
+            model.clear_session(session_id)
+
+    return recognize
+
+
+def stream_words(
+    model,
+    text: str,
+    labels: list[str],
+    session_id: str,
+    threshold: float,
+    delay: float,
+    flat_ner: bool,
+    multi_label: bool,
+) -> list[dict[str, Any]]:
+    """Terminal wrapper retained for quick smoke tests and scripting."""
+    final_entities: tuple[dict[str, Any], ...] = ()
+    for step in stream_steps(model, text, labels, session_id, threshold, flat_ner, multi_label):
+        final_entities = step.entities
+        print(f"[{step.index:02d}/{step.total:02d}] {step.token!r}")
+        for entity in (*step.added, *step.updated):
+            print(
+                f"  {entity['text']!r} -> {entity['label']} "
+                f"[{entity['start']}:{entity['end']}] score={float(entity['score']):.3f}"
+            )
+        if not step.added and not step.updated and not step.removed:
+            print("  no entity change")
+        print(flush=True)
+        if delay and step.index != step.total:
+            time.sleep(delay)
+    return list(final_entities)
+
+
+HERO_HTML = """
+<section class="demo-hero">
+  <div class="hero-copy">
+    <div class="hero-badge"><span></span>StreamingSpan live session</div>
+    <h1>Watch entities <em>emerge</em><br>one word at a time.</h1>
+    <p>GLiNER reads a token, extends its KV cache, revisits provisional spans with
+       bounded future context, and updates recognized entities as they evolve.</p>
+    <div class="hero-pills">
+      <span>⚡ Cached decoding</span><span>✦ Open labels</span><span>↗ Exact offsets</span>
+    </div>
+  </div>
+  <div class="hero-visual" aria-hidden="true">
+    <div class="orbit orbit-one"></div><div class="orbit orbit-two"></div>
+    <div class="core-mark">GL<span>i</span>NER<small>STREAM</small></div>
+    <div class="floating-tag tag-one">PERSON</div>
+    <div class="floating-tag tag-two">LOCATION</div>
+    <div class="floating-tag tag-three">ORG</div>
+  </div>
+</section>
+"""
+
+
+DEMO_CSS = """
+.gradio-container {
+  max-width: 1480px !important;
+  background:
+    radial-gradient(circle at 12% -10%, color-mix(in srgb, var(--primary-400) 18%, transparent), transparent 34rem),
+    radial-gradient(circle at 92% 18%, color-mix(in srgb, #06b6d4 13%, transparent), transparent 30rem),
+    var(--body-background-fill);
+}
+.demo-hero {
+  position: relative; overflow: hidden; min-height: 330px; margin: 16px 0 24px; padding: 44px 52px;
+  display: grid; grid-template-columns: minmax(0, 1.4fr) minmax(280px, .6fr); align-items: center;
+  border: 1px solid color-mix(in srgb, var(--primary-400) 24%, var(--border-color-primary));
+  border-radius: 30px; color: #f8fafc;
+  background: linear-gradient(125deg, #111827 0%, #312e81 52%, #0e7490 118%);
+  box-shadow: 0 28px 80px rgba(15, 23, 42, .22);
+}
+.demo-hero:before { content:""; position:absolute; inset:0; opacity:.28; pointer-events:none;
+  background-image: linear-gradient(rgba(255,255,255,.08) 1px,transparent 1px),
+                    linear-gradient(90deg,rgba(255,255,255,.08) 1px,transparent 1px);
+  background-size: 38px 38px; mask-image:linear-gradient(to right,black,transparent 75%); }
+.hero-copy { position:relative; z-index:2; max-width:790px; }
+.hero-badge { display:inline-flex; align-items:center; gap:9px; padding:7px 12px; border:1px solid rgba(255,255,255,.2);
+  border-radius:999px; background:rgba(255,255,255,.09); font-size:12px; font-weight:750; letter-spacing:.08em; text-transform:uppercase; }
+.hero-badge span,.live-dot { width:8px; height:8px; border-radius:50%; background:#5eead4; box-shadow:0 0 0 5px rgba(94,234,212,.13); animation:live-pulse 1.7s infinite; }
+.demo-hero h1 { margin:18px 0 14px; color:white; font-size:clamp(38px,5vw,70px); line-height:.98; letter-spacing:-.055em; }
+.demo-hero h1 em { color:#a5f3fc; font-style:normal; }
+.demo-hero p { max-width:720px; margin:0; color:#cbd5e1; font-size:17px; line-height:1.65; }
+.hero-pills { display:flex; flex-wrap:wrap; gap:9px; margin-top:24px; }
+.hero-pills span { padding:8px 12px; border-radius:12px; background:rgba(255,255,255,.08); color:#e2e8f0; font-size:13px; }
+.hero-visual { position:relative; z-index:1; min-height:240px; display:grid; place-items:center; }
+.core-mark { width:154px; height:154px; display:grid; place-content:center; text-align:center; border-radius:42px;
+  background:linear-gradient(145deg,rgba(255,255,255,.22),rgba(255,255,255,.06)); border:1px solid rgba(255,255,255,.32);
+  box-shadow:inset 0 1px rgba(255,255,255,.35),0 24px 50px rgba(0,0,0,.24); backdrop-filter:blur(12px);
+  font-size:29px; font-weight:850; letter-spacing:-.06em; transform:rotate(-3deg); }
+.core-mark span { color:#67e8f9; }.core-mark small { display:block; margin-top:5px; font-size:9px; letter-spacing:.34em; color:#a5f3fc; }
+.orbit { position:absolute; border:1px solid rgba(255,255,255,.2); border-radius:50%; }
+.orbit-one { width:240px;height:240px;animation:spin 15s linear infinite; }.orbit-two { width:310px;height:150px;transform:rotate(-25deg); }
+.floating-tag { position:absolute; padding:7px 10px; border-radius:8px; background:white; color:#312e81; font-size:10px; font-weight:850;
+  box-shadow:0 12px 25px rgba(0,0,0,.2); letter-spacing:.08em; }
+.tag-one { top:23px;right:16px;transform:rotate(7deg); }.tag-two { bottom:24px;left:5px;transform:rotate(-8deg); }.tag-three { right:0;bottom:52px;transform:rotate(4deg); }
+.glass-panel { padding:20px !important; border:1px solid var(--border-color-primary) !important; border-radius:22px !important;
+  background:color-mix(in srgb,var(--block-background-fill) 92%,transparent) !important; box-shadow:0 15px 45px rgba(15,23,42,.07); }
+#control-panel textarea { font-size:15px; line-height:1.55; }
+#start-stream { min-height:48px; border:0; background:linear-gradient(110deg,#7c3aed,#2563eb,#0891b2); box-shadow:0 12px 24px rgba(79,70,229,.22); }
+#start-stream:hover { transform:translateY(-1px); filter:saturate(1.12); }
+.status-card,.visual-card,.word-rail-card,.entities-section { border:1px solid var(--border-color-primary); border-radius:22px;
+  background:color-mix(in srgb,var(--block-background-fill) 94%,transparent); box-shadow:0 14px 38px rgba(15,23,42,.065); }
+.status-card { padding:18px 20px; margin-bottom:14px; }
+.status-top,.status-title,.metric-row,.section-heading,.entity-card-top,.rail-heading { display:flex; align-items:center; }
+.status-top,.section-heading,.entity-card-top,.rail-heading { justify-content:space-between; }
+.status-title { gap:10px; }.status-icon { width:10px;height:10px;border-radius:50%;background:#94a3b8; }
+.status-card.streaming .status-icon { background:#22c55e;box-shadow:0 0 0 6px rgba(34,197,94,.12);animation:live-pulse 1.6s infinite; }
+.status-card.complete .status-icon { background:#06b6d4; }.status-card.error .status-icon { background:#ef4444; }
+.status-percent { font-size:13px;font-weight:800;color:var(--primary-500); }
+.progress-track,.confidence-track { overflow:hidden;background:color-mix(in srgb,var(--body-text-color) 9%,transparent);border-radius:999px; }
+.progress-track { height:7px;margin:13px 0 15px; }.progress-track i { display:block;height:100%;border-radius:inherit;background:linear-gradient(90deg,#7c3aed,#06b6d4);transition:width .35s ease; }
+.metric-row { gap:9px;display:grid;grid-template-columns:repeat(4,1fr); }.metric-row span { padding:9px 11px;border-radius:12px;background:var(--background-fill-secondary); }
+.metric-row strong,.metric-row small { display:block;overflow:hidden;text-overflow:ellipsis;white-space:nowrap; }.metric-row strong { font-size:12px; }.metric-row small { margin-top:2px;color:var(--body-text-color-subdued);font-size:10px; }
+.status-message { margin-top:12px;padding:10px 12px;border-radius:10px;color:#991b1b;background:#fee2e2;font-size:12px; }
+.visual-card { min-height:225px;padding:22px 24px; }.card-kicker { display:flex;align-items:center;gap:10px;margin-bottom:14px;color:var(--body-text-color-subdued);font-size:11px;font-weight:800;letter-spacing:.09em;text-transform:uppercase; }
+.label-legend { display:flex;flex-wrap:wrap;gap:7px;margin-bottom:18px; }.label-chip { display:inline-flex;align-items:center;gap:7px;padding:6px 9px;border-radius:999px;background:var(--label-bg);color:var(--label-fg);font-size:11px;font-weight:750; }
+.label-chip i { width:7px;height:7px;border-radius:50%;background:var(--label-accent); }
+.stream-text { min-height:95px;white-space:pre-wrap;overflow-wrap:anywhere;color:var(--body-text-color);font-size:clamp(20px,2.1vw,29px);font-weight:560;line-height:2.05;letter-spacing:-.018em; }
+.text-fragment { border-radius:5px;transition:background .25s ease; }.entity-fragment { padding:2px 1px;border-bottom:3px solid;box-decoration-break:clone;-webkit-box-decoration-break:clone; }
+.current-word { position:relative;outline:2px solid color-mix(in srgb,var(--primary-400) 70%,transparent);outline-offset:3px;animation:word-pop .45s ease both; }
+.inline-label { display:inline-flex;align-items:center;gap:4px;margin:0 4px 0 2px;padding:3px 6px;transform:translateY(-9px);border:1px solid color-mix(in srgb,var(--label-accent) 28%,transparent);
+  border-radius:7px;background:var(--label-bg);color:var(--label-fg);font-size:9px;font-weight:850;line-height:1;letter-spacing:.04em;text-transform:uppercase;white-space:nowrap; }
+.inline-label small { padding-left:4px;border-left:1px solid color-mix(in srgb,var(--label-accent) 25%,transparent);font-size:8px; }
+.stream-caret { display:inline-block;width:2px;height:1.15em;margin-left:3px;vertical-align:-.14em;background:var(--primary-500);animation:blink 1s step-end infinite; }
+.empty-stage { min-height:125px;display:grid;place-content:center;justify-items:center;gap:10px;color:var(--body-text-color-subdued);text-align:center; }.empty-orb { display:grid;place-items:center;width:42px;height:42px;border-radius:14px;color:white;background:linear-gradient(135deg,#7c3aed,#0891b2);box-shadow:0 9px 22px rgba(79,70,229,.2); }
+.word-rail-card { margin-top:14px;padding:17px 19px; }.rail-heading span { font-size:12px;font-weight:800; }.rail-heading small { color:var(--body-text-color-subdued);font-size:10px; }
+.word-rail { display:flex;gap:7px;margin-top:12px;padding:4px 2px 8px;overflow-x:auto;scrollbar-width:thin; }.word-chip { flex:none;display:inline-flex;align-items:center;gap:6px;padding:7px 10px;border:1px solid var(--border-color-primary);border-radius:10px;color:var(--body-text-color-subdued);background:var(--background-fill-secondary);font-size:12px;transition:.3s ease; }
+.word-chip small { font-size:8px;opacity:.55; }.word-chip.processed { color:var(--body-text-color);border-color:color-mix(in srgb,#10b981 28%,var(--border-color-primary));background:color-mix(in srgb,#10b981 8%,var(--block-background-fill)); }
+.word-chip.active { color:white;border-color:transparent;background:linear-gradient(115deg,#7c3aed,#2563eb);box-shadow:0 8px 18px rgba(79,70,229,.25);transform:translateY(-2px) scale(1.03); }
+.entities-section { margin-top:20px;padding:23px; }.section-heading { margin-bottom:16px; }.section-heading h3 { margin:2px 0 0;font-size:21px; }.eyebrow { color:var(--primary-500);font-size:9px;font-weight:850;letter-spacing:.12em;text-transform:uppercase; }
+.entity-count { display:grid;place-items:center;min-width:38px;height:38px;border-radius:12px;color:white;background:linear-gradient(135deg,#7c3aed,#0891b2);font-weight:850; }
+.entity-grid { display:grid;grid-template-columns:repeat(auto-fit,minmax(190px,1fr));gap:11px; }.entity-card { position:relative;overflow:hidden;padding:15px;border:1px solid color-mix(in srgb,var(--label-accent) 22%,var(--border-color-primary));border-radius:15px;background:linear-gradient(145deg,var(--label-bg),var(--block-background-fill) 72%); }
+.entity-card:before { content:"";position:absolute;inset:0 auto 0 0;width:4px;background:var(--label-accent); }.entity-type { padding:4px 7px;border-radius:7px;background:var(--label-bg);color:var(--label-fg);font-size:9px;font-weight:850;letter-spacing:.06em;text-transform:uppercase; }
+.entity-card-top strong { color:var(--label-accent);font-size:12px; }.entity-value { margin-top:13px;font-size:16px;font-weight:780;line-height:1.25; }.entity-offset { margin:5px 0 12px;color:var(--body-text-color-subdued);font-size:9px; }.confidence-track { height:5px; }.confidence-track i { display:block;height:100%;border-radius:inherit;background:var(--label-accent); }
+.no-entities { display:flex;align-items:center;justify-content:center;gap:14px;min-height:100px;border:1px dashed var(--border-color-primary);border-radius:15px;color:var(--body-text-color-subdued); }.no-entities>span { font-size:34px; }.no-entities strong,.no-entities small { display:block; }.no-entities small { margin-top:4px;font-size:11px; }
+.examples-heading { margin:30px 0 8px;text-align:center; }.examples-heading h2 { margin-bottom:4px;font-size:26px; }.examples-heading p { color:var(--body-text-color-subdued); }
+@keyframes live-pulse { 50% { opacity:.55;transform:scale(.82); } } @keyframes blink { 50% { opacity:0; } }
+@keyframes spin { to { transform:rotate(360deg); } } @keyframes word-pop { from { opacity:.45;transform:translateY(4px); } }
+@media (max-width:850px) { .demo-hero { grid-template-columns:1fr;padding:32px 25px; }.hero-visual { display:none; }.metric-row { grid-template-columns:repeat(2,1fr); }.glass-panel { padding:14px !important; } }
+"""
+
+
+def require_gradio():
+    """Import Gradio lazily so non-UI helpers remain usable without it."""
+    try:
+        import gradio as gr  # noqa: PLC0415
+    except ModuleNotFoundError as error:
+        raise ModuleNotFoundError("Install Gradio to run this demo: pip install 'gradio>=6'") from error
+    return gr
+
+
+def build_demo(
+    model,
+    model_source: str,
+    default_text: str = DEFAULT_TEXT,
+    default_labels: str = DEFAULT_LABELS,
+    default_threshold: float = 0.5,
+    default_delay: float = 0.3,
+):
+    """Construct the Gradio Blocks application."""
+    gr = require_gradio()
+
+    initial_status, initial_text, initial_words, initial_entities = initial_view(
+        model,
+        default_text,
+        default_labels,
+    )
+    handler = create_streaming_handler(model)
+    mode = span_context_label(model)
+    model_caption = escape(model_source)
+
+    with gr.Blocks(title="GLiNER Live — Streaming NER", fill_width=True) as demo:
+        session_id = gr.State(
+            value=_new_session_id(),
+            time_to_live=3600,
+            delete_callback=model.clear_session,
+        )
+
+        gr.HTML(HERO_HTML, container=False)
+
+        with gr.Row(equal_height=False):
+            with gr.Column(scale=5, elem_id="control-panel", elem_classes="glass-panel"):
+                gr.Markdown(
+                    f"### Compose the stream\nLoaded `{model_caption}` · `{mode}` span context",
+                    container=False,
+                )
+                text_input = gr.Textbox(
+                    value=default_text,
+                    label="Text",
+                    info="The model's own word splitter determines each streaming step.",
+                    lines=7,
+                    max_lines=12,
+                    placeholder="Paste a sentence or paragraph…",
+                )
+                labels_input = gr.Textbox(
+                    value=default_labels,
+                    label="Entity labels",
+                    info="Comma-separated; labels can be anything your model learned to recognize.",
+                    lines=2,
+                    placeholder="person, organization, location, date",
+                )
+                with gr.Row():
+                    threshold_input = gr.Slider(
+                        0.0,
+                        1.0,
+                        value=default_threshold,
+                        step=0.01,
+                        label="Confidence threshold",
+                    )
+                    delay_input = gr.Slider(
+                        0.0,
+                        1.0,
+                        value=default_delay,
+                        step=0.05,
+                        label="Seconds per word",
+                    )
+                with gr.Accordion("Recognition options", open=False):
+                    nested_input = gr.Checkbox(label="Allow nested entities", value=False)
+                    multi_label_input = gr.Checkbox(label="Allow multiple labels per span", value=False)
+                with gr.Row():
+                    start_button = gr.Button("▶  Start streaming", variant="primary", elem_id="start-stream", scale=3)
+                    stop_button = gr.Button("■  Stop", variant="stop", scale=1)
+                    reset_button = gr.Button("↺", variant="secondary", scale=0, min_width=52)
+
+            with gr.Column(scale=7):
+                status_output = gr.HTML(initial_status, container=False, elem_id="stream-status")
+                text_output = gr.HTML(initial_text, container=False, elem_id="stream-visual")
+                words_output = gr.HTML(initial_words, container=False, elem_id="word-timeline")
+
+        entities_output = gr.HTML(initial_entities, container=False, elem_id="entity-results")
+
+        gr.HTML(
+            '<div class="examples-heading"><h2>Start with a story</h2>'
+            "<p>Pick an example, adjust its open label vocabulary, then start the stream.</p></div>",
+            container=False,
+        )
+        gr.Examples(
+            examples=EXAMPLES,
+            inputs=[text_input, labels_input],
+            example_labels=["Work", "Medicine", "Technology", "Sports", "Space", "Software"],
+            label="Curated examples",
+            examples_per_page=6,
+        )
+
+        stream_inputs = [
+            text_input,
+            labels_input,
+            threshold_input,
+            delay_input,
+            nested_input,
+            multi_label_input,
+            session_id,
+        ]
+        stream_outputs = [status_output, text_output, words_output, entities_output]
+        run_event = start_button.click(
+            fn=handler,
+            inputs=stream_inputs,
+            outputs=stream_outputs,
+            show_progress="hidden",
+            concurrency_limit=1,
+            concurrency_id="streaming-span-inference",
+            stream_every=0.05,
+        )
+
+        def stop_stream(active_session_id: str):
+            model.clear_session(active_session_id)
+            return render_status(0, 0, 0, mode, state="stopped", message="The session cache was cleared.")
+
+        stop_button.click(
+            fn=stop_stream,
+            inputs=session_id,
+            outputs=status_output,
+            cancels=[run_event],
+            queue=False,
+        )
+
+        def reset_stream(active_session_id: str, text: str, labels_value: str):
+            model.clear_session(active_session_id)
+            try:
+                return initial_view(model, text, labels_value)
+            except ValueError as error:
+                return (
+                    render_status(0, 0, 0, mode, state="error", message=str(error)),
+                    render_annotated_text("", (), ["entity"]),
+                    render_word_rail([]),
+                    render_entity_cards((), ["entity"]),
+                )
+
+        reset_button.click(
+            fn=reset_stream,
+            inputs=[session_id, text_input, labels_input],
+            outputs=stream_outputs,
+            cancels=[run_event],
+            queue=False,
+        )
+
+        # Use an explicit load event rather than a callable State value. Gradio
+        # 6.20.0 leaves callable State values unevaluated, so callbacks otherwise
+        # receive the factory function instead of the generated cache key.
+        demo.load(fn=_new_session_id, outputs=session_id, queue=False)
+
+    return demo
+
+
+def resolve_device(requested: str) -> str:
+    """Resolve ``auto`` to the best available PyTorch device."""
+    if requested != "auto":
+        return requested
+    if torch.cuda.is_available():
+        return "cuda"
+    if torch.backends.mps.is_available():
+        return "mps"
+    return "cpu"
+
+
+def build_parser() -> argparse.ArgumentParser:
+    parser = argparse.ArgumentParser(description=__doc__)
+    parser.add_argument("model", help="Local path or Hugging Face ID of a trained StreamingSpan checkpoint")
+    parser.add_argument("--text", default=DEFAULT_TEXT, help="Initial text shown in the demo")
+    parser.add_argument("--labels", default=DEFAULT_LABELS, help="Initial comma-separated labels")
+    parser.add_argument("--threshold", type=float, default=0.5, help="Initial confidence threshold")
+    parser.add_argument("--delay", type=float, default=0.3, help="Initial seconds-per-word playback speed")
+    parser.add_argument("--device", default="auto", help="PyTorch device: auto, cpu, cuda, cuda:0, or mps")
+    parser.add_argument("--local-files-only", action="store_true", help="Do not download model files")
+    parser.add_argument("--server-name", default="127.0.0.1", help="Gradio server bind address")
+    parser.add_argument("--server-port", type=int, default=7860, help="Gradio server port")
+    parser.add_argument("--share", action="store_true", help="Create a public Gradio share link")
+    parser.add_argument("--inbrowser", action="store_true", help="Open the demo in a browser")
+    return parser
+
+
+def main() -> None:
+    args = build_parser().parse_args()
+    if not 0.0 <= args.threshold <= 1.0:
+        raise SystemExit("--threshold must be between 0 and 1")
+    if not 0.0 <= args.delay <= 1.0:
+        raise SystemExit("--delay must be between 0 and 1")
+    try:
+        parse_labels(args.labels)
+    except ValueError as error:
+        raise SystemExit(str(error)) from error
+
+    model = GLiNER.from_pretrained(
+        args.model,
+        load_tokenizer=True,
+        local_files_only=args.local_files_only,
+    )
+    if getattr(model.config, "model_type", None) != "gliner_streaming_span":
+        raise SystemExit("The checkpoint must use model_type='gliner_streaming_span'")
+
+    device = resolve_device(args.device)
+    model = model.to(device).eval()
+    print(f"Loaded {args.model!r} on {device}; starting the streaming Gradio demo…")
+
+    try:
+        demo = build_demo(
+            model,
+            args.model,
+            default_text=args.text,
+            default_labels=args.labels,
+            default_threshold=args.threshold,
+            default_delay=args.delay,
+        )
+    except ModuleNotFoundError as error:
+        raise SystemExit(str(error)) from error
+
+    gr = require_gradio()
+    theme = gr.themes.Soft(primary_hue="violet", secondary_hue="cyan", neutral_hue="slate")
+    demo.queue(max_size=32, default_concurrency_limit=1).launch(
+        server_name=args.server_name,
+        server_port=args.server_port,
+        share=args.share,
+        inbrowser=args.inbrowser,
+        show_error=True,
+        theme=theme,
+        css=DEMO_CSS,
+    )
+
+
+if __name__ == "__main__":
+    main()
diff --git a/gliner/__init__.py b/gliner/__init__.py
index e538c9fc..041f5e45 100644
--- a/gliner/__init__.py
+++ b/gliner/__init__.py
@@ -1,7 +1,8 @@
-__version__ = "0.2.27"
+__version__ = "0.2.28"
 
 from .model import GLiNER
-from .config import GLiNERConfig
+from .config import GLiNERConfig, StreamingSpanConfig
+from .streaming import StreamingBatch, AsyncStreamingEngine
 from .infer_packing import (
     PackedBatch,
     InferencePackingConfig,
@@ -14,10 +15,13 @@
 #                                     GLiNERDocREDEvaluator)
 
 __all__ = [
+    "AsyncStreamingEngine",
     "GLiNER",
     "GLiNERConfig",
     "InferencePackingConfig",
     "PackedBatch",
+    "StreamingBatch",
+    "StreamingSpanConfig",
     "pack_requests",
     "unpack_spans",
 ]
diff --git a/gliner/config.py b/gliner/config.py
index ed9e9730..880ad626 100644
--- a/gliner/config.py
+++ b/gliner/config.py
@@ -1,8 +1,124 @@
-from typing import Optional
+from typing import Union, Optional
 
-from transformers import PretrainedConfig
+from transformers import DebertaV2Config, ModernBertConfig, PretrainedConfig
 from transformers.models.auto import CONFIG_MAPPING
 
+SUBTOKEN_POOLING_MODES = ("first", "last", "mean", "max")
+CONTEXT_ENCODER_MODEL_TYPES = ("deberta-v2", "modernbert", "rnn")
+
+
+class RNNEncoderConfig(PretrainedConfig):
+    """Configuration for an RNN that contextualizes existing hidden states."""
+
+    model_type = "rnn"
+
+    def __init__(
+        self,
+        hidden_size: int = 512,
+        num_hidden_layers: int = 1,
+        dropout: float = 0.0,
+        bidirectional: bool = True,
+        rnn_type: str = "lstm",
+        **kwargs,
+    ):
+        super().__init__(**kwargs)
+        rnn_type = rnn_type.lower()
+        if rnn_type != "lstm":
+            raise ValueError("RNN context encoders currently support only rnn_type='lstm'")
+        if hidden_size < 1:
+            raise ValueError("RNN encoder hidden_size must be positive")
+        if bidirectional and hidden_size % 2:
+            raise ValueError("Bidirectional RNN encoder hidden_size must be even")
+        if num_hidden_layers < 1:
+            raise ValueError("RNN encoder num_hidden_layers must be positive")
+        if not 0.0 <= dropout < 1.0:
+            raise ValueError("RNN encoder dropout must be in the range [0, 1)")
+
+        self.hidden_size = hidden_size
+        self.num_hidden_layers = num_hidden_layers
+        self.dropout = dropout
+        self.bidirectional = bidirectional
+        self.rnn_type = rnn_type
+
+
+ContextEncoderConfig = Union[DebertaV2Config, ModernBertConfig, RNNEncoderConfig]
+
+
+def _default_num_attention_heads(hidden_size: int) -> int:
+    """Choose a practical attention-head count that divides ``hidden_size``."""
+    num_heads = max(1, hidden_size // 64)
+    while hidden_size % num_heads:
+        num_heads -= 1
+    return num_heads
+
+
+def normalize_context_encoder_config(
+    encoder_config: Optional[Union[dict, ContextEncoderConfig]],
+    *,
+    hidden_size: int,
+    dropout: float,
+    default_num_hidden_layers: int = 1,
+) -> Optional[ContextEncoderConfig]:
+    """Normalize configs shared by span and streaming-label context encoders."""
+    if encoder_config is None:
+        return None
+
+    if isinstance(encoder_config, dict):
+        values = encoder_config.copy()
+        model_type = values.pop("model_type", None)
+        if model_type not in CONTEXT_ENCODER_MODEL_TYPES:
+            supported = ", ".join(CONTEXT_ENCODER_MODEL_TYPES)
+            raise ValueError(f"Unknown context encoder model_type {model_type!r}. Expected one of: {supported}")
+
+        encoder_hidden_size = values.setdefault("hidden_size", hidden_size)
+        values.setdefault("num_hidden_layers", default_num_hidden_layers)
+
+        if model_type == "rnn":
+            values.setdefault("dropout", dropout)
+            encoder_config = RNNEncoderConfig(**values)
+        else:
+            values.setdefault("num_attention_heads", _default_num_attention_heads(encoder_hidden_size))
+            values.setdefault("intermediate_size", encoder_hidden_size * 4)
+            if model_type == "deberta-v2":
+                values.setdefault("hidden_dropout_prob", dropout)
+                values.setdefault("attention_probs_dropout_prob", dropout)
+                values.setdefault("relative_attention", True)
+                values.setdefault("pos_att_type", ["p2c", "c2p"])
+                values.setdefault("max_relative_positions", 512)
+                encoder_config = DebertaV2Config(**values)
+            else:
+                # ModernBERT is called with ``inputs_embeds``. Keep its unused
+                # token table tiny instead of allocating the default 50k rows.
+                values.setdefault("vocab_size", 1)
+                values.setdefault("pad_token_id", 0)
+                values.setdefault("bos_token_id", 0)
+                values.setdefault("eos_token_id", 0)
+                values.setdefault("cls_token_id", 0)
+                values.setdefault("sep_token_id", 0)
+                values.setdefault("attention_dropout", dropout)
+                values.setdefault("embedding_dropout", dropout)
+                values.setdefault("mlp_dropout", dropout)
+                encoder_config = ModernBertConfig(**values)
+    elif not isinstance(encoder_config, (DebertaV2Config, ModernBertConfig, RNNEncoderConfig)):
+        supported = ", ".join(config.__name__ for config in (DebertaV2Config, ModernBertConfig, RNNEncoderConfig))
+        raise TypeError(f"Context encoder config must be a dict or one of: {supported}")
+
+    if encoder_config.hidden_size < 1:
+        raise ValueError("Context encoder hidden_size must be positive")
+    if encoder_config.num_hidden_layers < 1:
+        raise ValueError("Context encoder num_hidden_layers must be positive")
+
+    if isinstance(encoder_config, (DebertaV2Config, ModernBertConfig)):
+        num_heads = encoder_config.num_attention_heads
+        if num_heads < 1:
+            raise ValueError("Context encoder num_attention_heads must be positive")
+        if encoder_config.hidden_size % num_heads:
+            raise ValueError("Context encoder hidden_size must be divisible by num_attention_heads")
+        if encoder_config.intermediate_size < 1:
+            raise ValueError("Context encoder intermediate_size must be positive")
+
+    return encoder_config
+
 
 class BaseGLiNERConfig(PretrainedConfig):
     """Base configuration class for all GLiNER models."""
@@ -32,6 +148,7 @@ def __init__(
         embed_ent_token: bool = True,
         class_token_index: int = -1,
         encoder_config: Optional[dict] = None,
+        span_encoder_config: Optional[dict] = None,
         ent_token: str = "<<ENT>>",
         sep_token: str = "<<SEP>>",
         _attn_implementation: Optional[str] = None,
@@ -53,7 +170,8 @@ def __init__(
             hidden_size (int, optional): Dimension of hidden representations. Defaults to 512.
             dropout (float, optional): Dropout probability. Defaults to 0.4.
             fine_tune (bool, optional): Whether to fine-tune the encoder. Defaults to True.
-            subtoken_pooling (str, optional): Subtoken pooling strategy. Defaults to "first".
+            subtoken_pooling (str, optional): Subtoken pooling strategy. One of
+                "first", "last", "mean", or "max". Defaults to "first".
             span_mode (str, optional): Span representation mode. Defaults to "markerV0".
             post_fusion_schema (str, optional): Post-fusion processing schema. Defaults to ''.
             num_post_fusion_layers (int, optional): Number of post-fusion layers. Defaults to 1.
@@ -67,6 +185,8 @@ def __init__(
             embed_ent_token (bool, optional): Whether to embed entity tokens. Defaults to True.
             class_token_index (int, optional): Index of class token. Defaults to -1.
             encoder_config (dict, optional): Encoder configuration dict. Defaults to None.
+            span_encoder_config (dict, optional): Optional hidden-state encoder
+                applied before span representations. Defaults to None.
             ent_token (str, optional): Entity marker token. Defaults to "<<ENT>>".
             sep_token (str, optional): Separator token. Defaults to "<<SEP>>".
             _attn_implementation (str, optional): Attention implementation. Defaults to None.
@@ -85,6 +205,11 @@ def __init__(
 
             encoder_config = CONFIG_MAPPING[encoder_config["model_type"]](**encoder_config)
         self.encoder_config = encoder_config
+        self.span_encoder_config = normalize_context_encoder_config(
+            span_encoder_config,
+            hidden_size=hidden_size,
+            dropout=dropout,
+        )
 
         self.model_name = model_name
         self.name = name
@@ -92,6 +217,9 @@ def __init__(
         self.hidden_size = hidden_size
         self.dropout = dropout
         self.fine_tune = fine_tune
+        if subtoken_pooling not in SUBTOKEN_POOLING_MODES:
+            supported = ", ".join(SUBTOKEN_POOLING_MODES)
+            raise ValueError(f"Unknown subtoken pooling strategy {subtoken_pooling!r}. Expected one of: {supported}")
         self.subtoken_pooling = subtoken_pooling
         self.span_mode = span_mode
         self.post_fusion_schema = post_fusion_schema
@@ -193,6 +321,83 @@ def __init__(self, **kwargs):
         self.represent_spans = True  # hardcoded to True for token decoder
 
 
+class StreamingSpanConfig(UniEncoderConfig):
+    """Configuration for span NER backed directly by a causal decoder.
+
+    Unlike :class:`UniEncoderSpanDecoderConfig`, the decoder is the text
+    backbone itself rather than an auxiliary label-generation head.  A
+    dedicated model type keeps the two architectures unambiguous when models
+    are saved and loaded through :class:`GLiNER`.  During cached inference,
+    ``right_context_width`` controls how many following words may revise a
+    span; when omitted it defaults to ``max_width``.  Set it to zero for the
+    previous append-only behavior.
+    """
+
+    model_type = "gliner_streaming_span"
+
+    def __init__(
+        self,
+        model_name: Optional[str] = None,
+        decoder_config: Optional[dict] = None,
+        labels_encoder_config: Optional[dict] = None,
+        span_encoder_config: Optional[dict] = None,
+        label_token: str = "<<LABEL>>",
+        sep_token_index: int = -1,
+        max_cache_length: Optional[int] = None,
+        right_context_width: Optional[int] = None,
+        labels_decoder: Optional[str] = None,
+        labels_decoder_config: Optional[dict] = None,
+        **kwargs,
+    ):
+        # ``labels_decoder`` used to identify this architecture's only
+        # backbone.  Consume the old fields when loading an existing checkpoint
+        # but do not retain or re-serialize them: ``model_name`` and
+        # ``decoder_config`` are the canonical fields for StreamingSpan models.
+        default_model_name = "microsoft/deberta-v3-small"
+        if labels_decoder is not None and (model_name is None or model_name == default_model_name):
+            model_name = labels_decoder
+        if model_name is None:
+            model_name = default_model_name
+        if decoder_config is None:
+            decoder_config = labels_decoder_config
+        if isinstance(decoder_config, dict):
+            decoder_config = decoder_config.copy()
+            decoder_config["model_type"] = decoder_config.get("model_type", "gpt2")
+            decoder_config = CONFIG_MAPPING[decoder_config["model_type"]](**decoder_config)
+
+        # Decoder streaming is causal by default. Optional word context lives
+        # in the span representation layer, so do not also create the legacy
+        # model-level BiLSTM unless a caller explicitly requests it.
+        kwargs.setdefault("num_rnn_layers", 0)
+        kwargs.setdefault("span_mode", "markerV2")
+        super().__init__(model_name=model_name, span_encoder_config=span_encoder_config, **kwargs)
+
+        # The default label scorer remains a compact, independent DeBERTa-v2
+        # encoder, but all supported context encoders use the same contract.
+        if labels_encoder_config is None:
+            labels_encoder_config = {"model_type": "deberta-v2", "num_hidden_layers": 2}
+        labels_encoder_config = normalize_context_encoder_config(
+            labels_encoder_config,
+            hidden_size=self.hidden_size,
+            dropout=self.dropout,
+            default_num_hidden_layers=2,
+        )
+        if max_cache_length is not None and max_cache_length < 1:
+            raise ValueError("max_cache_length must be positive when provided")
+        if right_context_width is None:
+            right_context_width = self.max_width
+        if right_context_width < 0:
+            raise ValueError("right_context_width must be non-negative")
+
+        self.decoder_config = decoder_config
+        self.labels_encoder_config = labels_encoder_config
+        self.label_token = label_token
+        self.sep_token_index = sep_token_index
+        self.max_cache_length = max_cache_length
+        self.right_context_width = right_context_width
+        self.model_type = "gliner_streaming_span"
+
+
 class UniEncoderRelexConfig(UniEncoderConfig):
     def __init__(
         self,
@@ -377,6 +582,7 @@ def _resolve_model_type(self):
         "gliner_uni_encoder_token": UniEncoderTokenConfig,
         "gliner_uni_encoder_span_decoder": UniEncoderSpanDecoderConfig,
         "gliner_uni_encoder_token_decoder": UniEncoderTokenDecoderConfig,
+        "gliner_streaming_span": StreamingSpanConfig,
         "gliner_uni_encoder_span_relex": UniEncoderSpanRelexConfig,
         "gliner_uni_encoder_token_relex": UniEncoderTokenRelexConfig,
         "gliner_bi_encoder": BiEncoderConfig,
diff --git a/gliner/data_processing/__init__.py b/gliner/data_processing/__init__.py
index 4cd98f4e..81c3ab03 100644
--- a/gliner/data_processing/__init__.py
+++ b/gliner/data_processing/__init__.py
@@ -1,5 +1,6 @@
 from .collator import (
     BiEncoderSpanDataCollator,
+    StreamingSpanDataCollator,
     BiEncoderTokenDataCollator,
     UniEncoderSpanDataCollator,
     UniEncoderTokenDataCollator,
@@ -12,6 +13,7 @@
     BaseProcessor,
     BaseBiEncoderProcessor,
     BiEncoderSpanProcessor,
+    StreamingSpanProcessor,
     BiEncoderTokenProcessor,
     UniEncoderSpanProcessor,
     UniEncoderTokenProcessor,
diff --git a/gliner/data_processing/collator.py b/gliner/data_processing/collator.py
index c33f46a4..d85cc268 100644
--- a/gliner/data_processing/collator.py
+++ b/gliner/data_processing/collator.py
@@ -6,6 +6,7 @@
 from .processor import (
     BaseProcessor,
     BiEncoderSpanProcessor,
+    StreamingSpanProcessor,
     BiEncoderTokenProcessor,
     UniEncoderSpanProcessor,
     UniEncoderTokenProcessor,
@@ -220,7 +221,12 @@ def __init__(
         self,
         config,
         data_processor: Optional[
-            Union[UniEncoderSpanProcessor, BiEncoderSpanProcessor, UniEncoderSpanDecoderProcessor]
+            Union[
+                UniEncoderSpanProcessor,
+                BiEncoderSpanProcessor,
+                UniEncoderSpanDecoderProcessor,
+                StreamingSpanProcessor,
+            ]
         ] = None,
         return_tokens: bool = False,
         return_id_to_classes: bool = False,
@@ -519,6 +525,12 @@ class UniEncoderSpanDataCollator(SpanDataCollator):
     pass
 
 
+class StreamingSpanDataCollator(SpanDataCollator):
+    """Data collator for :class:`StreamingSpanProcessor`."""
+
+    pass
+
+
 class BiEncoderSpanDataCollator(SpanDataCollator):
     """
     Backward compatibility alias for SpanDataCollator with BiEncoderSpanProcessor.
diff --git a/gliner/data_processing/processor.py b/gliner/data_processing/processor.py
index a29225a9..0e570d9b 100644
--- a/gliner/data_processing/processor.py
+++ b/gliner/data_processing/processor.py
@@ -270,16 +270,21 @@ def prepare_word_mask(self, texts, tokenized_inputs, skip_first_words=None, toke
             tokenized_inputs: Tokenized inputs from transformer tokenizer.
             skip_first_words: Optional list of word counts to skip per example
                 (e.g., prompt words).
-            token_level: If True, create token-level masks instead of word-level.
+            token_level: If True, assign word indices to every subtoken,
+                independently of the configured pooling strategy.
 
         Returns:
             Word mask array.
         """
+        subtoken_pooling = getattr(self.config, "subtoken_pooling", "first")
+        if not isinstance(subtoken_pooling, str):
+            subtoken_pooling = "first"
         return prepare_word_mask(
             texts,
             tokenized_inputs,
             skip_first_words=skip_first_words,
             token_level=token_level,
+            subtoken_pooling=subtoken_pooling,
         )
 
     def tokenize_inputs(self, texts, entities, blank=None, **kwargs):
@@ -621,6 +626,99 @@ def tokenize_and_prepare_labels(self, batch, prepare_labels, *args, **kwargs):
         return tokenized_input
 
 
+class StreamingSpanProcessor(UniEncoderSpanProcessor):
+    """Span processor for a causal-decoder text backbone.
+
+    Labels are terminated by a marker rather than introduced by one.  This is
+    important for causal backbones: the hidden state at ``<<LABEL>>`` can then
+    attend to all subtokens of the label that precedes it.
+    """
+
+    def __init__(self, config, tokenizer, words_splitter):
+        super().__init__(config, tokenizer, words_splitter)
+        self.label_token = getattr(config, "label_token", "<<LABEL>>")
+
+    def prepare_inputs(
+        self,
+        texts: Sequence[Sequence[str]],
+        entities: Union[Sequence[Sequence[str]], Dict[int, Sequence[str]], Sequence[str]],
+        blank: Optional[str] = None,
+        add_entities: Optional[bool] = True,
+        include_prompt: Union[bool, Sequence[bool]] = True,
+        **kwargs,
+    ) -> Tuple[List[List[str]], List[int]]:
+        """Build ``label<<LABEL>>...<<SEP>>text`` token sequences."""
+        del kwargs
+        include_flags = (
+            [include_prompt] * len(texts) if isinstance(include_prompt, bool) else list(include_prompt)
+        )
+        if len(include_flags) != len(texts):
+            raise ValueError("include_prompt must match the number of texts")
+
+        input_texts: List[List[str]] = []
+        prompt_lengths: List[int] = []
+        for i, text in enumerate(texts):
+            prompt: List[str] = []
+            if include_flags[i]:
+                ents = self._maybe_remap_entities(self._select_entities(i, entities, blank))
+                for ent in ents:
+                    if add_entities:
+                        prompt.append(str(ent))
+                    prompt.append(self.label_token)
+                prompt.append(self.sep_token)
+
+            prompt_lengths.append(len(prompt))
+            input_texts.append(prompt + list(text))
+        return input_texts, prompt_lengths
+
+    def tokenize_inputs(
+        self,
+        texts,
+        entities,
+        blank=None,
+        include_prompt=True,
+        truncation=True,
+        **kwargs,
+    ):
+        """Tokenize without an implicit EOS so cached sessions can be extended."""
+        input_texts, prompt_lengths = self.prepare_inputs(
+            texts,
+            entities,
+            blank=blank,
+            include_prompt=include_prompt,
+            **kwargs,
+        )
+        tokenized_inputs = self.transformer_tokenizer(
+            input_texts,
+            is_split_into_words=True,
+            return_tensors="pt",
+            truncation=truncation,
+            padding="longest",
+            add_special_tokens=False,
+        )
+        words_masks = self.prepare_word_mask(texts, tokenized_inputs, prompt_lengths)
+        tokenized_inputs["words_mask"] = torch.tensor(words_masks)
+        return tokenized_inputs
+
+    def tokenize_and_prepare_labels(
+        self,
+        batch,
+        prepare_labels,
+        *args,
+        include_prompt=True,
+        **kwargs,
+    ):
+        tokenized_input = self.tokenize_inputs(
+            batch["tokens"],
+            batch["classes_to_id"],
+            include_prompt=include_prompt,
+            **kwargs,
+        )
+        if prepare_labels:
+            tokenized_input["labels"] = self.create_labels(batch)
+        return tokenized_input
+
+
 class UniEncoderTokenProcessor(BaseProcessor):
     """Processor for token-based NER with uni-encoder architecture.
 
diff --git a/gliner/data_processing/utils.py b/gliner/data_processing/utils.py
index 2d93f872..d199ecf7 100644
--- a/gliner/data_processing/utils.py
+++ b/gliner/data_processing/utils.py
@@ -98,6 +98,7 @@ def prepare_word_mask(
     *,
     skip_first_words: Optional[Sequence[int]] = None,
     token_level: bool = False,
+    subtoken_pooling: str = "first",
 ) -> List[List[int]]:
     """Create word-level masks for subword tokenized sequences.
 
@@ -117,15 +118,16 @@ def prepare_word_mask(
         skip_first_words: Optional number of words to skip at the beginning of
             each sequence (e.g., prompt words). Must have the same length as texts
             if provided. Skipped words are masked as 0.
-        token_level: If True, assign a unique mask value to every token of a word
-            (enabling token-level granularity). If False, only the first subword
-            token of each word gets a mask value; continuation tokens are masked
-            as 0 (default: False).
+        token_level: If True, assign the word index to every subtoken, overriding
+            ``subtoken_pooling``. Defaults to False.
+        subtoken_pooling: Determines which subtokens receive a word index when
+            ``token_level`` is False: the first subtoken for ``first``, the last
+            for ``last``, or every subtoken for ``mean`` and ``max``.
 
     Returns:
         List of word mask lists, one per input sequence. Each mask list has the
         same length as the corresponding tokenized sequence. Values are:
-            - 0: Special tokens, skipped words, or continuation subwords
+            - 0: Special tokens, skipped words, or unselected subtokens
             - 1, 2, 3, ...: Word indices (1-indexed) after skipping
 
     Raises:
@@ -139,6 +141,11 @@ def prepare_word_mask(
         >>> # Result might be: [[0, 1, 0, 2, 0]]
         >>> #                   [CLS, Hel, ##lo, world, SEP]
     """
+    supported_pooling = {"first", "last", "mean", "max"}
+    if subtoken_pooling not in supported_pooling:
+        supported = ", ".join(sorted(supported_pooling))
+        raise ValueError(f"Unknown subtoken pooling strategy {subtoken_pooling!r}. Expected one of: {supported}")
+
     n = len(texts)
 
     if skip_first_words is None:
@@ -149,28 +156,33 @@ def prepare_word_mask(
     words_masks: List[List[int]] = []
 
     for i in range(n):
+        word_ids = tokenized_inputs.word_ids(i)
         mask: List[int] = []
         prev_word_id: Optional[int] = None
         seen_words = 0  # counts distinct word_ids we've traversed in this sequence
 
-        for wid in tokenized_inputs.word_ids(i):
+        for token_idx, wid in enumerate(word_ids):
             if wid is None:
                 # Special tokens (CLS, SEP, PAD, etc.)
                 mask.append(0)
-            elif wid != prev_word_id or token_level:
-                # If we just moved to a new word, update seen_words
-                if wid != prev_word_id:
+            else:
+                is_first_subtoken = wid != prev_word_id
+                if is_first_subtoken:
                     seen_words += 1
 
-                if seen_words <= skip_first_words[i]:
-                    # This word is in the skip range (e.g., prompt tokens)
+                next_word_id = word_ids[token_idx + 1] if token_idx + 1 < len(word_ids) else None
+                is_last_subtoken = next_word_id != wid
+                select_subtoken = (
+                    token_level
+                    or subtoken_pooling in {"mean", "max"}
+                    or (subtoken_pooling == "first" and is_first_subtoken)
+                    or (subtoken_pooling == "last" and is_last_subtoken)
+                )
+
+                if seen_words <= skip_first_words[i] or not select_subtoken:
                     mask.append(0)
                 else:
-                    # 1-based word index after skipping
                     mask.append(seen_words - skip_first_words[i])
-            else:
-                # same word continuation and token_level=False -> mask as 0
-                mask.append(0)
 
             prev_word_id = wid
 
@@ -253,3 +265,55 @@ def prepare_span_idx(num_tokens, max_width):
     starts = torch.arange(num_tokens, dtype=torch.long).unsqueeze(1).expand(-1, max_width).reshape(-1)
     offsets = torch.arange(max_width, dtype=torch.long).unsqueeze(0).expand(num_tokens, -1).reshape(-1)
     return torch.stack([starts, starts + offsets], dim=1)
+
+
+def prepare_streaming_span_idx(
+    past_tokens: int,
+    new_tokens: int,
+    max_width: int,
+    recompute_all: bool = False,
+    right_context_width: int = 0,
+):
+    """Generate absolute span candidates for the next streaming chunk.
+
+    In incremental mode, all spans ending in the new chunk are valid.  Spans
+    ending up to ``right_context_width`` words before the latest word are also
+    regenerated so future context can revise their scores.  A zero-width right
+    context preserves append-only scoring.  Full-recompute mode enumerates
+    every historical span again.
+
+    Returns:
+        A tuple ``(span_idx, span_mask)`` where ``span_idx`` has shape
+        ``(num_starts * max_width, 2)`` and uses absolute session word indices.
+    """
+    if past_tokens < 0 or new_tokens < 0:
+        raise ValueError("past_tokens and new_tokens must be non-negative")
+    if max_width < 1:
+        raise ValueError("max_width must be positive")
+    if right_context_width < 0:
+        raise ValueError("right_context_width must be non-negative")
+
+    total_tokens = past_tokens + new_tokens
+    if total_tokens == 0:
+        return torch.zeros((0, 2), dtype=torch.long), torch.zeros(0, dtype=torch.bool)
+
+    if recompute_all:
+        minimum_end = 0
+    else:
+        latest_word = total_tokens - 1
+        rolling_minimum_end = max(0, latest_word - right_context_width)
+        # Always score every span ending in the newly supplied chunk, even when
+        # a caller appends more words than the configured right-context window.
+        minimum_end = min(past_tokens, rolling_minimum_end)
+
+    first_start = max(0, minimum_end - (max_width - 1))
+    num_starts = total_tokens - first_start
+    span_idx = prepare_span_idx(num_starts, max_width) + first_start
+
+    span_mask = span_idx[:, 1] < total_tokens
+    if not recompute_all:
+        span_mask = span_mask & span_idx[:, 1].ge(minimum_end)
+        if new_tokens == 0:
+            span_mask.zero_()
+
+    return span_idx, span_mask
diff --git a/gliner/decoding/decoder.py b/gliner/decoding/decoder.py
index b81e42b4..c48fb0b7 100644
--- a/gliner/decoding/decoder.py
+++ b/gliner/decoding/decoder.py
@@ -562,6 +562,167 @@ def _build_span_tuple(
         ent_type = id_to_class[class_idx + 1]  # +1 because 0 is <pad>
         return Span(start=start, end=start + width, entity_type=ent_type, score=score, class_probs=class_probs)
 
+    def _decode_explicit_spans(
+        self,
+        id_to_classes,
+        model_output,
+        span_idx,
+        span_mask,
+        flat_ner=False,
+        threshold=0.5,
+        multi_label=False,
+        return_class_probs=False,
+        input_spans=None,
+    ):
+        """Decode logits paired with explicit absolute span boundaries."""
+        probabilities = torch.sigmoid(model_output)
+        if probabilities.dim() == 4:
+            probabilities = probabilities.flatten(1, 2)
+        span_idx = span_idx.view(span_idx.size(0), -1, 2)
+        span_mask = span_mask.view(span_mask.size(0), -1).bool()
+
+        batch_size, _, num_classes = probabilities.shape
+        thresholds = _expand_batch_param(threshold, batch_size, "threshold")
+        flat_values = _expand_batch_param(flat_ner, batch_size, "flat_ner")
+        multi_values = _expand_batch_param(multi_label, batch_size, "multi_label")
+
+        valid_spans = span_mask.clone()
+        if input_spans is not None:
+            if len(input_spans) != batch_size:
+                raise ValueError("input_spans must have one entry per batch item")
+            for batch_idx, allowed_spans in enumerate(input_spans):
+                if allowed_spans is None:
+                    continue
+                allowed_mask = torch.zeros_like(valid_spans[batch_idx])
+                for start, end in set(allowed_spans):
+                    allowed_mask |= (span_idx[batch_idx, :, 0] == start) & (
+                        span_idx[batch_idx, :, 1] == end
+                    )
+                valid_spans[batch_idx] &= allowed_mask
+
+        id_to_class_per_item = [
+            self._get_id_to_class_for_sample(id_to_classes, batch_idx)
+            for batch_idx in range(batch_size)
+        ]
+        valid_classes = torch.tensor(
+            [
+                [class_idx + 1 in id_to_class for class_idx in range(num_classes)]
+                for id_to_class in id_to_class_per_item
+            ],
+            dtype=torch.bool,
+            device=probabilities.device,
+        )
+
+        threshold_tensor = torch.as_tensor(
+            thresholds,
+            dtype=probabilities.dtype,
+            device=probabilities.device,
+        ).view(batch_size, 1, 1)
+        candidate_mask = (
+            valid_spans.unsqueeze(-1)
+            & valid_classes.unsqueeze(1)
+            & (probabilities > threshold_tensor)
+        )
+        batch_indices, span_positions, class_indices = torch.where(candidate_mask)
+        if batch_indices.numel() == 0:
+            return [[] for _ in range(batch_size)]
+
+        candidate_boundaries = span_idx[batch_indices, span_positions]
+        candidate_scores = probabilities[batch_indices, span_positions, class_indices]
+        index_rows = (
+            torch.column_stack((batch_indices, candidate_boundaries, class_indices))
+            .detach()
+            .cpu()
+            .tolist()
+        )
+        score_rows = candidate_scores.detach().cpu().tolist()
+
+        top_prob_rows = None
+        top_index_rows = None
+        if return_class_probs:
+            candidate_probabilities = probabilities[batch_indices, span_positions]
+            top_k = min(5, num_classes)
+            top_indices = torch.argsort(
+                candidate_probabilities,
+                dim=-1,
+                descending=True,
+                stable=True,
+            )[:, :top_k]
+            top_probabilities = torch.gather(candidate_probabilities, 1, top_indices)
+            top_prob_rows = top_probabilities.detach().cpu().tolist()
+            top_index_rows = top_indices.detach().cpu().tolist()
+
+        candidates_by_batch = [[] for _ in range(batch_size)]
+        for row_index, ((batch_idx, start, end, class_idx), score) in enumerate(
+            zip(index_rows, score_rows)
+        ):
+            id_to_class = id_to_class_per_item[batch_idx]
+            class_probs = None
+            if return_class_probs:
+                class_probs = {
+                    id_to_class.get(index + 1, f"class_{index}"): probability
+                    for index, probability in zip(
+                        top_index_rows[row_index],
+                        top_prob_rows[row_index],
+                    )
+                }
+            candidates_by_batch[batch_idx].append(
+                Span(
+                    start=start,
+                    end=end,
+                    entity_type=id_to_class[class_idx + 1],
+                    score=score,
+                    class_probs=class_probs,
+                )
+            )
+
+        return [
+            self.greedy_search(
+                candidates,
+                flat_values[batch_idx],
+                multi_label=multi_values[batch_idx],
+            )
+            for batch_idx, candidates in enumerate(candidates_by_batch)
+        ]
+
+    def decode(
+        self,
+        tokens,
+        id_to_classes,
+        model_output,
+        flat_ner=False,
+        threshold=0.5,
+        multi_label=False,
+        return_class_probs=False,
+        input_spans=None,
+        **kwargs,
+    ):
+        span_idx = kwargs.get("span_idx")
+        span_mask = kwargs.get("span_mask")
+        if span_idx is not None and span_mask is not None:
+            return self._decode_explicit_spans(
+                id_to_classes=id_to_classes,
+                model_output=model_output,
+                span_idx=span_idx,
+                span_mask=span_mask,
+                flat_ner=flat_ner,
+                threshold=threshold,
+                multi_label=multi_label,
+                return_class_probs=return_class_probs,
+                input_spans=input_spans,
+            )
+        return super().decode(
+            tokens=tokens,
+            id_to_classes=id_to_classes,
+            model_output=model_output,
+            flat_ner=flat_ner,
+            threshold=threshold,
+            multi_label=multi_label,
+            return_class_probs=return_class_probs,
+            input_spans=input_spans,
+            **kwargs,
+        )
+
 
 class SpanGenerativeDecoder(BaseSpanDecoder):
     """
diff --git a/gliner/model.py b/gliner/model.py
index 17a1b5b3..a7e8090d 100644
--- a/gliner/model.py
+++ b/gliner/model.py
@@ -6,6 +6,7 @@
 from abc import ABC, abstractmethod
 from typing import Any, Dict, List, Tuple, Union, Optional
 from pathlib import Path
+from threading import RLock
 
 import torch
 import onnxruntime as ort
@@ -23,6 +24,7 @@
 )
 from torch.utils.data import DataLoader
 from safetensors.torch import save_file
+from torch.nn.utils.rnn import pad_sequence
 from huggingface_hub.errors import EntryNotFoundError
 
 try:
@@ -35,6 +37,7 @@
     GLiNERConfig,
     BaseGLiNERConfig,
     BiEncoderSpanConfig,
+    StreamingSpanConfig,
     BiEncoderTokenConfig,
     UniEncoderSpanConfig,
     UniEncoderTokenConfig,
@@ -52,6 +55,7 @@
     TokenGenerativeDecoder,
 )
 from .training import Trainer, TrainingArguments
+from .streaming import StreamingBatch, AsyncStreamingEngine, _PersistentBatchState
 from .evaluation import BaseNEREvaluator, BaseRelexEvaluator
 from .onnx.model import (
     BaseORTModel,
@@ -67,6 +71,7 @@
 from .modeling.base import (
     BaseModel,
     BiEncoderSpanModel,
+    StreamingSpanModel,
     BiEncoderTokenModel,
     UniEncoderSpanModel,
     UniEncoderTokenModel,
@@ -75,10 +80,19 @@
     UniEncoderSpanDecoderModel,
     UniEncoderTokenDecoderModel,
 )
+from .modeling.cache import (
+    DEFAULT_CACHE_INITIAL_CAPACITY,
+    CacheState,
+    SessionCacheManager,
+    create_reusable_cache,
+    split_past_key_values,
+    stack_past_key_values,
+)
 from .modeling.utils import extract_prompt_features
 from .data_processing import (
     BaseProcessor,
     BiEncoderSpanProcessor,
+    StreamingSpanProcessor,
     BiEncoderTokenProcessor,
     UniEncoderSpanProcessor,
     UniEncoderTokenProcessor,
@@ -87,8 +101,10 @@
     UniEncoderTokenDecoderProcessor,
     RelationExtractionTokenProcessor,
 )
+from .data_processing.utils import prepare_streaming_span_idx
 from .data_processing.collator import (
     BiEncoderSpanDataCollator,
+    StreamingSpanDataCollator,
     BiEncoderTokenDataCollator,
     UniEncoderSpanDataCollator,
     UniEncoderTokenDataCollator,
@@ -804,6 +820,11 @@ def _set_tokenizer_spec_tokens(tokenizer):
             tokenizer.add_eos_token = True
         return tokenizer
 
+    @classmethod
+    def _get_tokenizer_source(cls, config: GLiNERConfig) -> str:
+        """Return the model identifier used to initialize the input tokenizer."""
+        return config.model_name
+
     @classmethod
     def _load_tokenizer(
         cls,
@@ -830,7 +851,7 @@ def _load_tokenizer(
             tokenizer = AutoTokenizer.from_pretrained(model_dir, cache_dir=cache_dir, local_files_only=local_files_only)
         else:
             tokenizer = AutoTokenizer.from_pretrained(
-                config.model_name,
+                cls._get_tokenizer_source(config),
                 cache_dir=cache_dir,
                 local_files_only=local_files_only,
             )
@@ -989,8 +1010,13 @@ def _download_model(
     @staticmethod
     def _resize_token_embeddings(instance, config_instance, tokenizer, resize_token_embeddings=True):
         add_tokens = instance._get_special_tokens()
+        missing_architecture_indices = getattr(config_instance, "sep_token_index", 0) == -1
         # Resize token embeddings if needed
-        if resize_token_embeddings and (config_instance.class_token_index == -1 or config_instance.vocab_size == -1):
+        if resize_token_embeddings and (
+            config_instance.class_token_index == -1
+            or config_instance.vocab_size == -1
+            or missing_architecture_indices
+        ):
             if tokenizer is not None:
                 tokenizer.add_tokens(add_tokens, special_tokens=True)
             instance.resize_embeddings()
@@ -1079,7 +1105,10 @@ def load_from_config(
         # Load tokenizer if requested
         tokenizer = None
         if load_tokenizer:
-            tokenizer = AutoTokenizer.from_pretrained(config_instance.model_name, cache_dir=cache_dir)
+            tokenizer = AutoTokenizer.from_pretrained(
+                cls._get_tokenizer_source(config_instance),
+                cache_dir=cache_dir,
+            )
             cls._set_tokenizer_spec_tokens(tokenizer)
         # Create model instance from scratch
         instance = cls(
@@ -1671,7 +1700,7 @@ def _get_freezable_components(self):
 
         # Labels encoder (optional)
         if (
-            self.config.labels_encoder is not None
+            getattr(self.config, "labels_encoder", None) is not None
             and hasattr(self.model, "token_rep_layer")
             and hasattr(self.model.token_rep_layer, "labels_encoder")
         ):
@@ -1679,7 +1708,7 @@ def _get_freezable_components(self):
 
         # Decoder (optional)
         if (
-            self.config.labels_decoder is not None
+            getattr(self.config, "labels_decoder", None) is not None
             and hasattr(self.model, "decoder")
             and hasattr(self.model.decoder, "decoder_layer")
         ):
@@ -3083,6 +3112,1267 @@ def forward(
         return UniEncoderSpanWrapper(core_model)
 
 
+class StreamingSpanGLiNER(BaseEncoderGLiNER):
+    """Causal-decoder span model with optional per-session streaming state."""
+
+    config_class = StreamingSpanConfig
+    model_class = StreamingSpanModel
+    ort_model_class = None
+    data_processor_class = StreamingSpanProcessor
+    data_collator_class = StreamingSpanDataCollator
+    decoder_class = SpanDecoder
+
+    def __init__(self, *args, **kwargs):
+        super().__init__(*args, **kwargs)
+        self._session_cache = SessionCacheManager()
+        self._stream_execution_lock = RLock()
+
+    @classmethod
+    def _get_tokenizer_source(cls, config):
+        return config.model_name
+
+    def _get_special_tokens(self):
+        return [self.config.label_token, self.config.sep_token]
+
+    def _create_data_processor(self, config, cache_dir, tokenizer=None, words_splitter=None, **kwargs):
+        if tokenizer is None:
+            tokenizer = AutoTokenizer.from_pretrained(config.model_name, cache_dir=cache_dir)
+        if tokenizer.pad_token is None:
+            if tokenizer.eos_token is None:
+                tokenizer.add_special_tokens({"pad_token": "<|pad|>"})
+            else:
+                tokenizer.pad_token = tokenizer.eos_token
+        tokenizer.padding_side = "right"
+        if words_splitter is None:
+            words_splitter = WordsSplitter(config.words_splitter_type)
+        self.data_processor = self.data_processor_class(config, tokenizer, words_splitter)
+        return self.data_processor
+
+    @staticmethod
+    def _resize_token_embeddings(instance, _config_instance, tokenizer, resize_token_embeddings=True):
+        """Match the real decoder embedding rows before checkpoint weights load.
+
+        StreamingSpan checkpoints can have valid saved special-token indices while
+        an older or absent nested decoder config still advertises the backbone's
+        pre-extension vocabulary.  The generic sentinel check cannot detect that
+        shape mismatch, so this architecture always compares the actual module to
+        its tokenizer when resizing is enabled.
+        """
+        if not resize_token_embeddings:
+            return
+        if tokenizer is not None:
+            tokenizer.add_tokens(instance._get_special_tokens(), special_tokens=True)
+        instance.resize_embeddings()
+
+    def set_class_indices(self):
+        tokenizer = self.data_processor.transformer_tokenizer
+        label_token_id = tokenizer.convert_tokens_to_ids(self.config.label_token)
+        sep_token_id = tokenizer.convert_tokens_to_ids(self.config.sep_token)
+        if label_token_id is None or label_token_id == tokenizer.unk_token_id:
+            raise ValueError(f"Label token {self.config.label_token!r} is not registered in the tokenizer")
+        if sep_token_id is None or sep_token_id == tokenizer.unk_token_id:
+            raise ValueError(f"Separator token {self.config.sep_token!r} is not registered in the tokenizer")
+        self.config.class_token_index = label_token_id
+        self.config.sep_token_index = sep_token_id
+
+    def resize_embeddings(self, set_class_token_index=True):
+        if set_class_token_index:
+            self.set_class_indices()
+        tokenizer_size = len(self.data_processor.transformer_tokenizer)
+        decoder_model = self.model.token_rep_layer.decoder_layer.model
+        if decoder_model.get_input_embeddings().num_embeddings != tokenizer_size:
+            embedding_weight = decoder_model.get_input_embeddings().weight
+            if embedding_weight.is_meta:
+                # Mean/covariance initialization calls Tensor.item(), which is
+                # unavailable on shape-only meta tensors. Checkpoint loading
+                # replaces every resized row immediately afterward.
+                decoder_model.resize_token_embeddings(tokenizer_size, mean_resizing=False)
+            else:
+                decoder_model.resize_token_embeddings(tokenizer_size)
+        embedding_size = decoder_model.get_input_embeddings().num_embeddings
+        self.config.vocab_size = embedding_size
+        self.config.decoder_config = decoder_model.config
+        self.config.decoder_config.vocab_size = embedding_size
+
+    def _get_freezable_components(self):
+        components = super()._get_freezable_components()
+        components["decoder_backbone"] = self.model.token_rep_layer.decoder_layer.model
+        components["labels_encoder"] = self.model.labels_encoder
+        return components
+
+    def clear_session(self, session_id: Union[str, List[str]]) -> None:
+        """Remove one or more cached inference sessions."""
+        self._session_cache.clear(session_id)
+
+    def clear_sessions(self) -> None:
+        """Remove every cached inference session."""
+        self._session_cache.clear()
+
+    def create_streaming_batch(self, session_ids: List[str], labels: List[str]):
+        """Create a persistent fixed-order streaming batch."""
+        return StreamingBatch(self, session_ids, labels)
+
+    def create_async_streaming_engine(
+        self,
+        *,
+        max_batch_size: int = 32,
+        batch_wait_timeout_ms: float = 2.0,
+        queue_capacity: int = 4096,
+    ):
+        """Create an asynchronous dynamic-microbatch streaming engine."""
+        return AsyncStreamingEngine(
+            self,
+            max_batch_size=max_batch_size,
+            batch_wait_timeout_ms=batch_wait_timeout_ms,
+            queue_capacity=queue_capacity,
+        )
+
+    @property
+    def session_count(self) -> int:
+        """Return the number of currently cached sessions."""
+        return len(self._session_cache)
+
+    def _collate_session_tokens(self, tokens, labels, include_prompt):
+        return self._collate_session_batch(
+            [list(tokens)],
+            [list(labels)],
+            include_prompt=include_prompt,
+        )
+
+    def _collate_session_batch(
+        self,
+        tokens_batch,
+        labels_batch,
+        *,
+        include_prompt,
+        span_candidates=None,
+    ):
+        """Collate one streaming step for multiple independent session rows."""
+        if len(tokens_batch) != len(labels_batch):
+            raise ValueError("tokens_batch and labels_batch must have the same length")
+        if not tokens_batch:
+            raise ValueError("At least one streaming row is required")
+
+        include_flags = (
+            [include_prompt] * len(tokens_batch)
+            if isinstance(include_prompt, bool)
+            else list(include_prompt)
+        )
+        if len(include_flags) != len(tokens_batch):
+            raise ValueError("include_prompt must have one value per streaming row")
+
+        class_to_ids = []
+        id_to_classes = []
+        for labels in labels_batch:
+            class_to_id = {label: index + 1 for index, label in enumerate(labels)}
+            class_to_ids.append(class_to_id)
+            id_to_classes.append({index: label for label, index in class_to_id.items()})
+
+        if span_candidates is None:
+            span_candidates = [
+                prepare_streaming_span_idx(
+                    0,
+                    len(tokens),
+                    self.config.max_width,
+                    recompute_all=True,
+                )
+                for tokens in tokens_batch
+            ]
+        if len(span_candidates) != len(tokens_batch):
+            raise ValueError("span_candidates must have one value per streaming row")
+
+        span_idx = pad_sequence(
+            [candidate[0] for candidate in span_candidates],
+            batch_first=True,
+            padding_value=0,
+        )
+        span_mask = pad_sequence(
+            [candidate[1].bool() for candidate in span_candidates],
+            batch_first=True,
+            padding_value=False,
+        )
+        raw_batch = {
+            "tokens": [list(tokens) for tokens in tokens_batch],
+            "classes_to_id": class_to_ids,
+            "id_to_classes": id_to_classes,
+            "span_idx": span_idx,
+            "span_mask": span_mask,
+            "seq_length": torch.tensor(
+                [[len(tokens)] for tokens in tokens_batch],
+                dtype=torch.long,
+            ),
+        }
+        model_input = self.data_processor.collate_fn(
+            raw_batch,
+            prepare_labels=False,
+            include_prompt=include_flags,
+            truncation=False,
+        )
+        model_input.update(
+            {
+                "span_idx": raw_batch["span_idx"],
+                "span_mask": raw_batch["span_mask"],
+                "text_lengths": raw_batch["seq_length"],
+                "tokens": raw_batch["tokens"],
+                "id_to_classes": raw_batch["id_to_classes"],
+            }
+        )
+        return model_input
+
+    def _decoder_context_limit(self) -> Optional[int]:
+        configured = getattr(self.config, "max_cache_length", None)
+        decoder_config = self.model.token_rep_layer.decoder_layer.model.config
+        backbone_limit = getattr(decoder_config, "max_position_embeddings", None)
+        if backbone_limit is None:
+            backbone_limit = getattr(decoder_config, "n_positions", None)
+        if configured is None:
+            return backbone_limit
+        return min(configured, backbone_limit) if backbone_limit is not None else configured
+
+    def _create_session_cache(self):
+        decoder_config = self.model.token_rep_layer.decoder_layer.model.config
+        return create_reusable_cache(
+            decoder_config,
+            max_cache_length=self._decoder_context_limit(),
+        )
+
+    def _validate_context_length(self, length: int, session_id: str) -> None:
+        limit = self._decoder_context_limit()
+        if limit is not None and length > limit:
+            raise ValueError(
+                f"Session {session_id!r} would contain {length} decoder tokens, exceeding "
+                f"the configured context limit of {limit}. Clear the session before continuing."
+            )
+
+    @staticmethod
+    def _compact_row(values: torch.Tensor, mask: torch.Tensor) -> tuple[torch.Tensor, torch.Tensor]:
+        valid = mask.bool()
+        compact = values[valid]
+        return compact, torch.ones(compact.size(0), dtype=torch.bool, device=compact.device)
+
+    @staticmethod
+    def _reserve_word_cache(
+        values: torch.Tensor,
+        mask: torch.Tensor,
+    ) -> tuple[torch.Tensor, torch.Tensor]:
+        valid = mask.bool()
+        compact = values[valid]
+        capacity = max(DEFAULT_CACHE_INITIAL_CAPACITY, compact.size(0))
+        storage = compact.new_empty(capacity, compact.size(-1))
+        storage_mask = torch.zeros(capacity, dtype=torch.bool, device=compact.device)
+        storage[: compact.size(0)].copy_(compact)
+        storage_mask[: compact.size(0)] = True
+        return storage, storage_mask
+
+    @staticmethod
+    def _merge_session_span_logits(
+        previous: Dict[Tuple[int, int], torch.Tensor],
+        model_output: Any,
+        *,
+        replace_all: bool = False,
+        batch_index: int = 0,
+        class_count: Optional[int] = None,
+    ) -> Dict[Tuple[int, int], torch.Tensor]:
+        """Update the CPU span-score history with candidates from one session call."""
+        logits = model_output.logits
+        if not isinstance(logits, torch.Tensor):
+            logits = torch.as_tensor(logits)
+        if logits.dim() == 4:
+            logits = logits.flatten(1, 2)
+        if logits.dim() != 3:
+            raise ValueError("Session span logits must have shape (batch, num_spans, num_classes)")
+        if not 0 <= batch_index < logits.size(0):
+            raise IndexError("batch_index is outside the model output batch")
+
+        span_idx = model_output.span_idx.view(logits.size(0), -1, 2)
+        span_mask = model_output.span_mask.view(logits.size(0), -1).bool()
+        if span_idx.size(1) != logits.size(1) or span_mask.size(1) != logits.size(1):
+            raise ValueError("Session span indices, mask, and logits must describe the same candidates")
+
+        row_mask = span_mask[batch_index]
+        valid_idx = span_idx[batch_index, row_mask].detach().cpu()
+        valid_logits = logits[batch_index, row_mask]
+        if class_count is not None:
+            valid_logits = valid_logits[..., :class_count]
+        valid_logits = valid_logits.detach().cpu()
+        merged = {} if replace_all else previous.copy()
+        for boundary, scores in zip(valid_idx.tolist(), valid_logits):
+            merged[(int(boundary[0]), int(boundary[1]))] = scores.clone()
+        return merged
+
+    def _state_from_output(
+        self,
+        session_id,
+        labels,
+        batch,
+        model_output,
+        text,
+        tokens,
+        char_starts,
+        char_ends,
+        *,
+        batch_index=0,
+        past_key_values=None,
+    ):
+        current_attention = (
+            batch["label_attention_mask"] if "label_attention_mask" in batch else batch["attention_mask"]
+        )
+        valid_tokens = current_attention[batch_index].bool()
+        input_ids = batch["input_ids"][batch_index][valid_tokens]
+        token_word_mask = batch["words_mask"][batch_index][valid_tokens]
+
+        raw_words, raw_word_mask = self._reserve_word_cache(
+            model_output.past_word_embeddings[batch_index],
+            model_output.past_word_mask[batch_index],
+        )
+        raw_prompts, prompt_mask = self._compact_row(
+            model_output.cached_prompts_embedding[batch_index],
+            model_output.cached_prompts_mask[batch_index],
+        )
+        cached_length = input_ids.size(0)
+        return CacheState(
+            past_key_values=(
+                model_output.past_key_values
+                if past_key_values is None
+                else past_key_values
+            ),
+            input_ids=input_ids.detach(),
+            attention_mask=torch.ones_like(input_ids),
+            token_word_mask=token_word_mask.detach(),
+            past_word_embeddings=raw_words.detach(),
+            past_word_mask=raw_word_mask.detach(),
+            prompts_embedding=raw_prompts.detach(),
+            prompts_mask=prompt_mask.detach(),
+            cached_length=cached_length,
+            next_position_id=cached_length,
+            session_id=session_id,
+            labels=tuple(labels),
+            text=text,
+            tokens=list(tokens),
+            char_starts=list(char_starts),
+            char_ends=list(char_ends),
+            span_logits=self._merge_session_span_logits(
+                {},
+                model_output,
+                replace_all=True,
+                batch_index=batch_index,
+                class_count=len(labels),
+            ),
+        )
+
+    def _decode_session_output(
+        self,
+        state,
+        threshold,
+        flat_ner,
+        multi_label,
+        return_class_probs,
+    ):
+        if not state.span_logits:
+            return []
+
+        id_to_classes = {index + 1: label for index, label in enumerate(state.labels)}
+        ordered_scores = sorted(state.span_logits.items())
+        span_idx = torch.tensor(
+            [[boundary for boundary, _ in ordered_scores]],
+            dtype=torch.long,
+        )
+        span_logits = torch.stack([scores for _, scores in ordered_scores]).float().unsqueeze(0)
+        span_mask = torch.ones(span_idx.shape[:2], dtype=torch.bool)
+        decoded = self.decoder.decode(
+            [state.tokens],
+            id_to_classes,
+            span_logits,
+            span_idx=span_idx,
+            span_mask=span_mask,
+            flat_ner=flat_ner,
+            threshold=threshold,
+            multi_label=multi_label,
+            return_class_probs=return_class_probs,
+        )[0]
+
+        entities = []
+        for span in decoded:
+            if span.start < 0 or span.end >= len(state.char_ends):
+                continue
+            start = state.char_starts[span.start]
+            end = state.char_ends[span.end]
+            entity = {
+                "start": start,
+                "end": end,
+                "text": state.text[start:end],
+                "label": span.entity_type,
+                "score": span.score,
+            }
+            if span.class_probs is not None:
+                entity["class_probs"] = span.class_probs
+            entities.append(entity)
+        return entities
+
+    def _prepare_session_item(self, text, labels, session_id, recompute):
+        """Prepare semantic and token metadata for one streaming append."""
+        state = self._session_cache.get(session_id)
+        if state is not None and state.labels != tuple(labels) and not recompute:
+            raise ValueError(
+                f"Labels for session {session_id!r} changed. Pass recompute=True or clear the session."
+            )
+
+        current_tokens, current_starts, current_ends = self.prepare_inputs([text])
+        current_tokens = current_tokens[0]
+        current_starts = current_starts[0]
+        current_ends = current_ends[0]
+        char_offset = len(state.text) if state is not None else 0
+        return {
+            "session_id": session_id,
+            "labels": tuple(labels),
+            "state": state,
+            "current_tokens": current_tokens,
+            "current_starts": current_starts,
+            "current_ends": current_ends,
+            "combined_text": (state.text if state is not None else "") + text,
+            "combined_tokens": (state.tokens if state is not None else []) + current_tokens,
+            "combined_starts": (state.char_starts if state is not None else [])
+            + [char_offset + value for value in current_starts],
+            "combined_ends": (state.char_ends if state is not None else [])
+            + [char_offset + value for value in current_ends],
+            "full_recompute": state is None or recompute,
+        }
+
+    def _run_full_session_batch(self, items):
+        """Run cold or explicitly recomputed sessions in one model forward."""
+        tokens_batch = [item["combined_tokens"] for item in items]
+        labels_batch = [list(item["labels"]) for item in items]
+        batch = self._collate_session_batch(
+            tokens_batch,
+            labels_batch,
+            include_prompt=True,
+        )
+        batch = {
+            key: value.to(self.device) if isinstance(value, torch.Tensor) else value
+            for key, value in batch.items()
+        }
+        token_counts = batch["attention_mask"].sum(dim=1).tolist()
+        for item, token_count in zip(items, token_counts):
+            self._validate_context_length(int(token_count), item["session_id"])
+
+        model_output = self.model(
+            **batch,
+            past_key_values=self._create_session_cache(),
+        )
+        row_caches = split_past_key_values(
+            model_output.past_key_values,
+            [int(length) for length in token_counts],
+            config=self.model.token_rep_layer.decoder_layer.model.config,
+            max_cache_length=self._decoder_context_limit(),
+        )
+        batch["label_attention_mask"] = batch["attention_mask"]
+
+        states = []
+        for row, (item, row_cache) in enumerate(zip(items, row_caches)):
+            states.append(
+                self._state_from_output(
+                    item["session_id"],
+                    item["labels"],
+                    batch,
+                    model_output,
+                    item["combined_text"],
+                    item["combined_tokens"],
+                    item["combined_starts"],
+                    item["combined_ends"],
+                    batch_index=row,
+                    past_key_values=row_cache,
+                )
+            )
+        for state in states:
+            self._session_cache.put(state)
+        return states
+
+    @staticmethod
+    def _stack_session_word_cache(states, capacity):
+        reference = states[0].past_word_embeddings
+        embeddings = reference.new_empty(
+            len(states),
+            capacity,
+            reference.size(-1),
+        )
+        mask = torch.zeros(
+            len(states),
+            capacity,
+            dtype=torch.bool,
+            device=reference.device,
+        )
+        for row, state in enumerate(states):
+            valid = state.past_word_mask.bool()
+            compact = state.past_word_embeddings[valid]
+            embeddings[row, : compact.size(0)].copy_(compact)
+            mask[row, : compact.size(0)] = True
+        return embeddings, mask
+
+    @staticmethod
+    def _stack_session_prompts(states):
+        max_prompts = max(state.prompts_embedding.size(0) for state in states)
+        reference = states[0].prompts_embedding
+        embeddings = reference.new_zeros(
+            len(states),
+            max_prompts,
+            reference.size(-1),
+        )
+        mask = torch.zeros(
+            len(states),
+            max_prompts,
+            dtype=torch.bool,
+            device=reference.device,
+        )
+        for row, state in enumerate(states):
+            valid = state.prompts_mask.bool()
+            compact = state.prompts_embedding[valid]
+            embeddings[row, : compact.size(0)].copy_(compact)
+            mask[row, : compact.size(0)] = True
+        return embeddings, mask
+
+    @staticmethod
+    def _reserve_batched_word_cache(values, mask):
+        capacity = max(DEFAULT_CACHE_INITIAL_CAPACITY, values.size(1))
+        if capacity == values.size(1):
+            return values.detach(), mask.bool().detach()
+        storage = values.new_empty(values.size(0), capacity, values.size(2))
+        storage_mask = torch.zeros(
+            values.size(0),
+            capacity,
+            dtype=torch.bool,
+            device=values.device,
+        )
+        storage[:, : values.size(1)].copy_(values)
+        storage_mask[:, : mask.size(1)] = mask.bool()
+        return storage.detach(), storage_mask.detach()
+
+    def _prepare_persistent_batch_items(
+        self,
+        state,
+        session_ids,
+        labels,
+        texts,
+    ):
+        items = []
+        for row, (session_id, text) in enumerate(zip(session_ids, texts)):
+            previous = None if state is None else state.sessions[row]
+            current_tokens, current_starts, current_ends = self.prepare_inputs([text])
+            current_tokens = current_tokens[0]
+            current_starts = current_starts[0]
+            current_ends = current_ends[0]
+            char_offset = len(previous.text) if previous is not None else 0
+            items.append(
+                {
+                    "session_id": session_id,
+                    "labels": tuple(labels),
+                    "previous": previous,
+                    "current_tokens": current_tokens,
+                    "combined_text": (previous.text if previous is not None else "") + text,
+                    "combined_tokens": (previous.tokens if previous is not None else []) + current_tokens,
+                    "combined_starts": (previous.char_starts if previous is not None else [])
+                    + [char_offset + value for value in current_starts],
+                    "combined_ends": (previous.char_ends if previous is not None else [])
+                    + [char_offset + value for value in current_ends],
+                }
+            )
+        return items
+
+    def _decode_persistent_batch_state(
+        self,
+        state,
+        threshold,
+        flat_ner,
+        multi_label,
+        return_class_probs,
+    ):
+        return [
+            self._decode_session_output(
+                session,
+                threshold,
+                flat_ner,
+                multi_label,
+                return_class_probs,
+            )
+            for session in state.sessions
+        ]
+
+    @torch.inference_mode()
+    def _run_persistent_stream_batch(
+        self,
+        state,
+        session_ids,
+        labels,
+        texts,
+        *,
+        threshold,
+        flat_ner,
+        multi_label,
+        return_class_probs,
+        recompute,
+    ):
+        """Advance a fixed-order batch while retaining one batched KV cache."""
+        with self._stream_execution_lock:
+            if state is not None and state.batch_size != len(session_ids):
+                raise ValueError("Persistent streaming batch size cannot change")
+            if state is not None and tuple(session.session_id for session in state.sessions) != tuple(session_ids):
+                raise ValueError("Persistent streaming session order cannot change")
+
+            if state is not None and not recompute and not any(text.strip() for text in texts):
+                return state, self._decode_persistent_batch_state(
+                    state,
+                    threshold,
+                    flat_ner,
+                    multi_label,
+                    return_class_probs,
+                )
+            if state is None and not any(text.strip() for text in texts):
+                return None, [[] for _ in texts]
+
+            items = self._prepare_persistent_batch_items(
+                state,
+                session_ids,
+                labels,
+                texts,
+            )
+            full_recompute = state is None or recompute
+
+            if full_recompute:
+                batch = self._collate_session_batch(
+                    [item["combined_tokens"] for item in items],
+                    [list(item["labels"]) for item in items],
+                    include_prompt=True,
+                )
+                batch = {
+                    key: value.to(self.device) if isinstance(value, torch.Tensor) else value
+                    for key, value in batch.items()
+                }
+                token_counts = batch["attention_mask"].sum(dim=1).long()
+                for item, token_count in zip(items, token_counts.tolist()):
+                    self._validate_context_length(int(token_count), item["session_id"])
+                model_output = self.model(
+                    **batch,
+                    past_key_values=self._create_session_cache(),
+                )
+                past_words, past_word_mask = self._reserve_batched_word_cache(
+                    model_output.past_word_embeddings,
+                    model_output.past_word_mask,
+                )
+                sessions = []
+                for row, (item, token_count) in enumerate(zip(items, token_counts.tolist())):
+                    sessions.append(
+                        CacheState(
+                            cached_length=int(token_count),
+                            next_position_id=int(token_count),
+                            session_id=item["session_id"],
+                            labels=item["labels"],
+                            text=item["combined_text"],
+                            tokens=item["combined_tokens"],
+                            char_starts=item["combined_starts"],
+                            char_ends=item["combined_ends"],
+                            span_logits=self._merge_session_span_logits(
+                                {},
+                                model_output,
+                                replace_all=True,
+                                batch_index=row,
+                                class_count=len(item["labels"]),
+                            ),
+                        )
+                    )
+                new_state = _PersistentBatchState(
+                    past_key_values=model_output.past_key_values,
+                    attention_mask=batch["attention_mask"].detach(),
+                    past_word_embeddings=past_words,
+                    past_word_mask=past_word_mask,
+                    prompts_embedding=model_output.cached_prompts_embedding.detach(),
+                    prompts_mask=model_output.cached_prompts_mask.bool().detach(),
+                    cached_lengths=token_counts.to(self.device),
+                    next_position_ids=token_counts.to(self.device),
+                    sessions=sessions,
+                )
+                return new_state, self._decode_persistent_batch_state(
+                    new_state,
+                    threshold,
+                    flat_ner,
+                    multi_label,
+                    return_class_probs,
+                )
+
+            span_candidates = []
+            for item in items:
+                previous = item["previous"]
+                if not item["current_tokens"]:
+                    span_candidates.append(
+                        (
+                            torch.zeros((0, 2), dtype=torch.long),
+                            torch.zeros(0, dtype=torch.bool),
+                        )
+                    )
+                    continue
+                right_context_width = getattr(self.config, "right_context_width", None)
+                if right_context_width is None:
+                    right_context_width = self.config.max_width
+                span_candidates.append(
+                    prepare_streaming_span_idx(
+                        previous.word_length,
+                        len(item["current_tokens"]),
+                        self.config.max_width,
+                        recompute_all=self.model.span_rep_layer.uses_bidirectional_context,
+                        right_context_width=right_context_width,
+                    )
+                )
+
+            batch = self._collate_session_batch(
+                [item["current_tokens"] for item in items],
+                [list(item["labels"]) for item in items],
+                include_prompt=False,
+                span_candidates=span_candidates,
+            )
+            batch = {
+                key: value.to(self.device) if isinstance(value, torch.Tensor) else value
+                for key, value in batch.items()
+            }
+            current_attention = batch["attention_mask"]
+            current_counts = current_attention.sum(dim=1).long()
+            for item, previous, current_count in zip(items, state.sessions, current_counts.tolist()):
+                self._validate_context_length(
+                    previous.cached_length + int(current_count),
+                    item["session_id"],
+                )
+
+            physical_length = state.physical_length + batch["input_ids"].size(1)
+            context_limit = self._decoder_context_limit()
+            if context_limit is not None and physical_length > context_limit:
+                raise ValueError(
+                    f"Persistent streaming batch would contain {physical_length} physical cache positions, "
+                    f"exceeding the configured context limit of {context_limit}. Reset the batch before continuing."
+                )
+
+            full_attention = torch.cat([state.attention_mask, current_attention], dim=1)
+            position_ids = state.next_position_ids.unsqueeze(1) + torch.arange(
+                batch["input_ids"].size(1),
+                device=self.device,
+            ).unsqueeze(0)
+            past_word_lengths = torch.tensor(
+                [session.word_length for session in state.sessions],
+                dtype=torch.long,
+                device=self.device,
+            )
+            model_output = self.model(
+                input_ids=batch["input_ids"],
+                attention_mask=full_attention,
+                label_attention_mask=current_attention,
+                past_word_length=past_word_lengths,
+                past_key_values=state.past_key_values,
+                past_word_embeddings=state.past_word_embeddings,
+                past_word_mask=state.past_word_mask,
+                prompts_embedding=state.prompts_embedding,
+                prompts_embedding_mask=state.prompts_mask,
+                words_mask=batch["words_mask"],
+                text_lengths=batch["text_lengths"],
+                span_idx=batch["span_idx"],
+                span_mask=batch["span_mask"],
+                position_ids=position_ids,
+            )
+
+            sessions = []
+            for row, (item, previous, current_count) in enumerate(
+                zip(items, state.sessions, current_counts.tolist())
+            ):
+                sessions.append(
+                    CacheState(
+                        cached_length=previous.cached_length + int(current_count),
+                        next_position_id=previous.next_position + int(current_count),
+                        session_id=item["session_id"],
+                        labels=item["labels"],
+                        text=item["combined_text"],
+                        tokens=item["combined_tokens"],
+                        char_starts=item["combined_starts"],
+                        char_ends=item["combined_ends"],
+                        span_logits=self._merge_session_span_logits(
+                            previous.span_logits,
+                            model_output,
+                            batch_index=row,
+                            class_count=len(item["labels"]),
+                        ),
+                    )
+                )
+
+            new_state = _PersistentBatchState(
+                past_key_values=model_output.past_key_values,
+                attention_mask=full_attention.detach(),
+                past_word_embeddings=model_output.past_word_embeddings.detach(),
+                past_word_mask=model_output.past_word_mask.bool().detach(),
+                prompts_embedding=model_output.cached_prompts_embedding.detach(),
+                prompts_mask=model_output.cached_prompts_mask.bool().detach(),
+                cached_lengths=state.cached_lengths + current_counts,
+                next_position_ids=state.next_position_ids + current_counts,
+                sessions=sessions,
+            )
+            return new_state, self._decode_persistent_batch_state(
+                new_state,
+                threshold,
+                flat_ner,
+                multi_label,
+                return_class_probs,
+            )
+
+    def _run_incremental_session_batch(self, items):
+        """Run equal-cache-length warm sessions in one model forward."""
+        cached_states = [item["state"].to(self.device) for item in items]
+        cached_lengths = {state.cached_length for state in cached_states}
+        if len(cached_lengths) != 1:
+            raise ValueError("Incremental session batches require equal cached token lengths")
+        cached_length = cached_states[0].cached_length
+
+        span_candidates = []
+        for item, state in zip(items, cached_states):
+            right_context_width = getattr(self.config, "right_context_width", None)
+            if right_context_width is None:
+                right_context_width = self.config.max_width
+            span_candidates.append(
+                prepare_streaming_span_idx(
+                    state.word_length,
+                    len(item["current_tokens"]),
+                    self.config.max_width,
+                    recompute_all=self.model.span_rep_layer.uses_bidirectional_context,
+                    right_context_width=right_context_width,
+                )
+            )
+
+        batch = self._collate_session_batch(
+            [item["current_tokens"] for item in items],
+            [list(item["labels"]) for item in items],
+            include_prompt=False,
+            span_candidates=span_candidates,
+        )
+        batch = {
+            key: value.to(self.device) if isinstance(value, torch.Tensor) else value
+            for key, value in batch.items()
+        }
+        current_attention = batch["attention_mask"]
+        current_token_counts = [int(value) for value in current_attention.sum(dim=1).tolist()]
+        for item, state, token_count in zip(items, cached_states, current_token_counts):
+            self._validate_context_length(
+                state.cached_length + token_count,
+                item["session_id"],
+            )
+
+        current_width = batch["input_ids"].size(1)
+        full_attention = torch.cat(
+            [torch.stack([state.attention_mask for state in cached_states]), current_attention],
+            dim=1,
+        )
+        position_starts = torch.tensor(
+            [state.next_position for state in cached_states],
+            dtype=torch.long,
+            device=self.device,
+        )
+        position_ids = position_starts.unsqueeze(1) + torch.arange(
+            current_width,
+            device=self.device,
+        ).unsqueeze(0)
+
+        past_word_lengths = torch.tensor(
+            [state.word_length for state in cached_states],
+            dtype=torch.long,
+            device=self.device,
+        )
+        required_word_capacity = max(
+            DEFAULT_CACHE_INITIAL_CAPACITY,
+            *(
+                state.word_length + len(item["current_tokens"])
+                for item, state in zip(items, cached_states)
+            ),
+        )
+        past_word_embeddings, past_word_mask = self._stack_session_word_cache(
+            cached_states,
+            required_word_capacity,
+        )
+        prompts_embedding, prompts_mask = self._stack_session_prompts(cached_states)
+        batched_cache = stack_past_key_values(
+            [state.past_key_values for state in cached_states],
+            config=self.model.token_rep_layer.decoder_layer.model.config,
+            max_cache_length=self._decoder_context_limit(),
+        )
+
+        model_output = self.model(
+            input_ids=batch["input_ids"],
+            attention_mask=full_attention,
+            label_attention_mask=current_attention,
+            past_word_length=past_word_lengths,
+            past_key_values=batched_cache,
+            past_word_embeddings=past_word_embeddings,
+            past_word_mask=past_word_mask,
+            prompts_embedding=prompts_embedding,
+            prompts_embedding_mask=prompts_mask,
+            words_mask=batch["words_mask"],
+            text_lengths=batch["text_lengths"],
+            span_idx=batch["span_idx"],
+            span_mask=batch["span_mask"],
+            position_ids=position_ids,
+        )
+        total_token_lengths = [cached_length + count for count in current_token_counts]
+        row_caches = split_past_key_values(
+            model_output.past_key_values,
+            total_token_lengths,
+            config=self.model.token_rep_layer.decoder_layer.model.config,
+            max_cache_length=self._decoder_context_limit(),
+        )
+
+        states = []
+        for row, (item, cached, row_cache, current_token_count) in enumerate(
+            zip(items, cached_states, row_caches, current_token_counts)
+        ):
+            valid_current = current_attention[row].bool()
+            current_ids = batch["input_ids"][row][valid_current]
+            current_word_mask = batch["words_mask"][row][valid_current].clone()
+            current_word_mask[current_word_mask > 0] += cached.word_length
+            raw_words, raw_word_mask = self._reserve_word_cache(
+                model_output.past_word_embeddings[row],
+                model_output.past_word_mask[row],
+            )
+            raw_prompts, prompt_mask = self._compact_row(
+                model_output.cached_prompts_embedding[row],
+                model_output.cached_prompts_mask[row],
+            )
+            state = CacheState(
+                past_key_values=row_cache,
+                input_ids=torch.cat([cached.input_ids, current_ids]).detach(),
+                attention_mask=torch.ones(
+                    cached.cached_length + current_token_count,
+                    dtype=torch.long,
+                    device=self.device,
+                ),
+                token_word_mask=torch.cat(
+                    [cached.token_word_mask, current_word_mask]
+                ).detach(),
+                past_word_embeddings=raw_words.detach(),
+                past_word_mask=raw_word_mask.detach(),
+                prompts_embedding=raw_prompts.detach(),
+                prompts_mask=prompt_mask.detach(),
+                cached_length=cached.cached_length + current_token_count,
+                next_position_id=cached.next_position + current_token_count,
+                session_id=item["session_id"],
+                labels=item["labels"],
+                text=item["combined_text"],
+                tokens=item["combined_tokens"],
+                char_starts=item["combined_starts"],
+                char_ends=item["combined_ends"],
+                span_logits=self._merge_session_span_logits(
+                    item["state"].span_logits,
+                    model_output,
+                    batch_index=row,
+                    class_count=len(item["labels"]),
+                ),
+            )
+            states.append(state)
+        for state in states:
+            self._session_cache.put(state)
+        return states
+
+    @torch.inference_mode()
+    def _run_session_items_batched(self, requests, *, return_exceptions=False):
+        """Batch compatible session appends and return results in request order."""
+        with self._stream_execution_lock:
+            if not requests:
+                return []
+            session_ids = [request["session_id"] for request in requests]
+            if len(set(session_ids)) != len(session_ids):
+                raise ValueError("Duplicate session IDs in one streaming batch are ambiguous")
+
+            prepared = [None] * len(requests)
+            results = [None] * len(requests)
+            for index, request in enumerate(requests):
+                try:
+                    prepared[index] = self._prepare_session_item(
+                        request["text"],
+                        request["labels"],
+                        request["session_id"],
+                        request.get("recompute", False),
+                    )
+                except Exception as error:
+                    results[index] = error
+
+            groups = {}
+            for index, item in enumerate(prepared):
+                if item is None:
+                    continue
+                key = (
+                    "full",
+                    0,
+                ) if item["full_recompute"] else (
+                    "incremental",
+                    item["state"].cached_length,
+                )
+                groups.setdefault(key, []).append(index)
+
+            for (kind, _), indices in groups.items():
+                group_items = [prepared[index] for index in indices]
+                try:
+                    if kind == "full":
+                        group_states = self._run_full_session_batch(group_items)
+                    else:
+                        group_states = self._run_incremental_session_batch(group_items)
+                except Exception as error:
+                    for index in indices:
+                        results[index] = error
+                    continue
+
+                for index, state in zip(indices, group_states):
+                    request = requests[index]
+                    try:
+                        results[index] = self._decode_session_output(
+                            state,
+                            request.get("threshold", 0.5),
+                            request.get("flat_ner", True),
+                            request.get("multi_label", False),
+                            request.get("return_class_probs", False),
+                        )
+                    except Exception as error:
+                        results[index] = error
+
+            if not return_exceptions:
+                for result in results:
+                    if isinstance(result, Exception):
+                        raise result
+            return results
+
+    def _run_session_item(
+        self,
+        text,
+        labels,
+        session_id,
+        recompute,
+        threshold,
+        flat_ner,
+        multi_label,
+        return_class_probs,
+    ):
+        item = self._prepare_session_item(text, labels, session_id, recompute)
+        state = item["state"]
+        current_tokens = item["current_tokens"]
+        combined_text = item["combined_text"]
+        combined_tokens = item["combined_tokens"]
+        combined_starts = item["combined_starts"]
+        combined_ends = item["combined_ends"]
+
+        full_recompute = item["full_recompute"]
+        if full_recompute:
+            batch = self._collate_session_tokens(combined_tokens, labels, include_prompt=True)
+            batch = {
+                key: value.to(self.device) if isinstance(value, torch.Tensor) else value
+                for key, value in batch.items()
+            }
+            token_count = int(batch["attention_mask"].sum().item())
+            self._validate_context_length(token_count, session_id)
+            model_output = self.model(**batch, past_key_values=self._create_session_cache())
+            batch["label_attention_mask"] = batch["attention_mask"]
+            new_state = self._state_from_output(
+                session_id,
+                labels,
+                batch,
+                model_output,
+                combined_text,
+                combined_tokens,
+                combined_starts,
+                combined_ends,
+            )
+        else:
+            batch = self._collate_session_tokens(current_tokens, labels, include_prompt=False)
+            batch = {
+                key: value.to(self.device) if isinstance(value, torch.Tensor) else value
+                for key, value in batch.items()
+            }
+            current_attention = batch["attention_mask"]
+            current_token_count = int(current_attention.sum().item())
+            self._validate_context_length(state.cached_length + current_token_count, session_id)
+
+            past_words = state.word_length
+            new_words = len(current_tokens)
+            recompute_spans = self.model.span_rep_layer.uses_bidirectional_context
+            right_context_width = getattr(self.config, "right_context_width", None)
+            if right_context_width is None:
+                right_context_width = self.config.max_width
+            span_idx, span_mask = prepare_streaming_span_idx(
+                past_words,
+                new_words,
+                self.config.max_width,
+                recompute_all=recompute_spans,
+                right_context_width=right_context_width,
+            )
+            batch["span_idx"] = span_idx.unsqueeze(0).to(self.device)
+            batch["span_mask"] = span_mask.unsqueeze(0).to(self.device)
+
+            cached = state.to(self.device)
+            full_attention = torch.cat(
+                [cached.attention_mask.unsqueeze(0), current_attention],
+                dim=1,
+            )
+            position_ids = torch.arange(
+                cached.next_position,
+                cached.next_position + batch["input_ids"].size(1),
+                device=self.device,
+            ).unsqueeze(0)
+            try:
+                model_output = self.model(
+                    input_ids=batch["input_ids"],
+                    attention_mask=full_attention,
+                    label_attention_mask=current_attention,
+                    # Same-device CacheState.to() returns this exact cache object.
+                    # ReusableDynamicLayer writes only the newly appended KV slice
+                    # and retains its backing storage until a capacity boundary.
+                    past_word_length=past_words,
+                    past_key_values=cached.past_key_values,
+                    past_word_embeddings=cached.past_word_embeddings.unsqueeze(0),
+                    past_word_mask=cached.past_word_mask.unsqueeze(0),
+                    prompts_embedding=cached.prompts_embedding.unsqueeze(0),
+                    prompts_embedding_mask=cached.prompts_mask.unsqueeze(0),
+                    words_mask=batch["words_mask"],
+                    text_lengths=batch["text_lengths"],
+                    span_idx=batch["span_idx"],
+                    span_mask=batch["span_mask"],
+                    position_ids=position_ids,
+                )
+            except Exception:
+                # KV layers append in place, so a partially failed decoder pass
+                # cannot safely be retried from the previous session metadata.
+                self.clear_session(session_id)
+                raise
+
+            valid_current = current_attention[0].bool()
+            current_ids = batch["input_ids"][0][valid_current]
+            current_word_mask = batch["words_mask"][0][valid_current].clone()
+            current_word_mask[current_word_mask > 0] += past_words
+            raw_words = model_output.past_word_embeddings[0]
+            raw_word_mask = model_output.past_word_mask[0]
+            raw_prompts, prompt_mask = self._compact_row(
+                model_output.cached_prompts_embedding[0],
+                model_output.cached_prompts_mask[0],
+            )
+            new_state = CacheState(
+                past_key_values=model_output.past_key_values,
+                input_ids=torch.cat([cached.input_ids, current_ids]).detach(),
+                attention_mask=torch.ones(
+                    cached.cached_length + current_token_count,
+                    dtype=torch.long,
+                    device=self.device,
+                ),
+                token_word_mask=torch.cat([cached.token_word_mask, current_word_mask]).detach(),
+                past_word_embeddings=raw_words.detach(),
+                past_word_mask=raw_word_mask.detach(),
+                prompts_embedding=raw_prompts.detach(),
+                prompts_mask=prompt_mask.detach(),
+                cached_length=cached.cached_length + current_token_count,
+                next_position_id=cached.next_position + current_token_count,
+                session_id=session_id,
+                labels=tuple(labels),
+                text=combined_text,
+                tokens=combined_tokens,
+                char_starts=combined_starts,
+                char_ends=combined_ends,
+                span_logits=self._merge_session_span_logits(state.span_logits, model_output),
+            )
+
+        self._session_cache.put(new_state)
+        return self._decode_session_output(
+            new_state,
+            threshold,
+            flat_ner,
+            multi_label,
+            return_class_probs,
+        )
+
+    @torch.no_grad()
+    def inference(
+        self,
+        texts: Union[str, List[str]],
+        labels: List[str],
+        flat_ner: bool = True,
+        threshold: float = 0.5,
+        multi_label: bool = False,
+        batch_size: int = 8,
+        packing_config: Optional[InferencePackingConfig] = None,
+        input_spans: Optional[List[List[Dict]]] = None,
+        return_class_probs: bool = False,
+        session_id: Optional[List[str]] = None,
+        recompute: bool = False,
+        **external_inputs,
+    ):
+        """Run stateless inference or append to cached streaming-span sessions.
+
+        Session calls reuse the causal decoder cache and rescore spans within
+        ``config.right_context_width`` of the newest word.  ``recompute=True``
+        takes precedence and refreshes every span over the combined session.
+        Returned entities are a complete session snapshot, including any
+        revisions or removals caused by the newly appended context.
+        """
+        if session_id is None:
+            return super().inference(
+                texts,
+                labels,
+                flat_ner=flat_ner,
+                threshold=threshold,
+                multi_label=multi_label,
+                batch_size=batch_size,
+                packing_config=packing_config,
+                input_spans=input_spans,
+                return_class_probs=return_class_probs,
+                **external_inputs,
+            )
+        if external_inputs:
+            raise ValueError("external model inputs are not supported with session inference")
+        if packing_config is not None:
+            raise ValueError("inference packing is not supported with session inference")
+        if input_spans is not None:
+            raise ValueError("input_spans are not supported with session inference")
+        if isinstance(texts, str):
+            texts = [texts]
+        if not isinstance(session_id, list) or not all(isinstance(value, str) for value in session_id):
+            raise TypeError("session_id must be a list of strings")
+        if len(session_id) != len(texts):
+            raise ValueError("session_id must have one value per input text")
+        if len(set(session_id)) != len(session_id):
+            raise ValueError("Duplicate session IDs in one inference call are ambiguous")
+        if not isinstance(recompute, bool):
+            raise TypeError("recompute must be a boolean")
+
+        normalized_labels = list(dict.fromkeys(labels))
+        if not normalized_labels:
+            raise ValueError("At least one label is required")
+        if not isinstance(batch_size, int) or batch_size < 1:
+            raise ValueError("batch_size must be a positive integer")
+        self.eval()
+        outputs = [[] for _ in texts]
+        requests = []
+        request_indices = []
+        for index, (text, current_session) in enumerate(zip(texts, session_id)):
+            if not isinstance(text, str) or not text.strip():
+                continue
+            request_indices.append(index)
+            requests.append(
+                {
+                    "text": text,
+                    "labels": normalized_labels,
+                    "session_id": current_session,
+                    "recompute": recompute,
+                    "threshold": threshold,
+                    "flat_ner": flat_ner,
+                    "multi_label": multi_label,
+                    "return_class_probs": return_class_probs,
+                }
+            )
+
+        for offset in range(0, len(requests), batch_size):
+            request_batch = requests[offset : offset + batch_size]
+            decoded_batch = self._run_session_items_batched(request_batch)
+            for index, decoded in zip(
+                request_indices[offset : offset + batch_size],
+                decoded_batch,
+            ):
+                outputs[index] = decoded
+        return outputs
+
+
 class UniEncoderTokenGLiNER(BaseEncoderGLiNER):
     config_class = UniEncoderTokenConfig
     model_class = UniEncoderTokenModel
@@ -4668,11 +5958,11 @@ def __init__(self, config: Union[str, Path, GLiNERConfig], **kwargs):
             if config_path.exists():
                 with open(config_path) as f:
                     config_dict = json.load(f)
-                config = GLiNERConfig(**config_dict)
+                config = self._config_from_dict(config_dict)
             else:
                 raise FileNotFoundError(f"Config file not found: {config}")
         elif isinstance(config, dict):
-            config = GLiNERConfig(**config)
+            config = self._config_from_dict(config)
 
         # Determine the appropriate GLiNER class based on config
         gliner_class = self._get_gliner_class(config)
@@ -4687,6 +5977,9 @@ def __init__(self, config: Union[str, Path, GLiNERConfig], **kwargs):
     @staticmethod
     def _get_gliner_class(config: GLiNERConfig):
         """Determine the appropriate GLiNER class based on configuration."""
+        if isinstance(config, StreamingSpanConfig) or getattr(config, "model_type", None) == "gliner_streaming_span":
+            return StreamingSpanGLiNER
+
         is_token_level = config.span_mode == "token_level"
         has_labels_encoder = config.labels_encoder is not None
         has_labels_decoder = config.labels_decoder is not None
@@ -4715,10 +6008,16 @@ def _get_gliner_class(config: GLiNERConfig):
             else:
                 return BiEncoderSpanGLiNER
 
-        if is_token_level:
-            return UniEncoderTokenGLiNER
-        else:
-            return UniEncoderSpanGLiNER
+        return UniEncoderTokenGLiNER if is_token_level else UniEncoderSpanGLiNER
+
+    @staticmethod
+    def _config_from_dict(config_dict: dict) -> BaseGLiNERConfig:
+        """Build the architecture-specific config encoded in a saved dictionary."""
+        config_dict = config_dict.copy()
+        model_type = config_dict.pop("model_type", None)
+        if model_type == "gliner_streaming_span":
+            return StreamingSpanConfig(**config_dict)
+        return GLiNERConfig(**config_dict)
 
     @classmethod
     def from_pretrained(
@@ -4864,9 +6163,7 @@ def from_pretrained(
         with open(config_file) as f:
             config_dict = json.load(f)
 
-        config_dict.pop("model_type", None)
-
-        config = GLiNERConfig(**config_dict)
+        config = cls._config_from_dict(config_dict)
 
         # Determine the appropriate class
         gliner_class = cls._get_gliner_class(config)
@@ -4952,11 +6249,15 @@ def from_config(
             if config_path.exists():
                 with open(config_path) as f:
                     config_dict = json.load(f)
-                config_ = GLiNERConfig(**config_dict)
+                config_ = cls._config_from_dict(config_dict)
             else:
                 raise FileNotFoundError(f"Config file not found: {config}")
         elif isinstance(config, dict):
-            config_ = GLiNERConfig(**config)
+            config_ = cls._config_from_dict(config)
+        elif isinstance(config, BaseGLiNERConfig):
+            config_ = config
+        else:
+            raise TypeError(f"config must be a GLiNERConfig object, path to config file, or dict. Got {type(config)}")
 
         # Determine the appropriate class
         gliner_class = cls._get_gliner_class(config_)
@@ -4986,6 +6287,11 @@ def model_map(self) -> dict[str, dict[str, Any]]:
             Dictionary mapping model types to their classes and descriptions.
         """
         return {
+            "gliner_streaming_span": {
+                "class": StreamingSpanGLiNER,
+                "description": "Streaming span NER with a causal decoder backbone",
+                "config": {"model_type": "gliner_streaming_span", "model_name": "required"},
+            },
             "gliner_uni_encoder_span": {
                 "class": UniEncoderSpanGLiNER,
                 "description": "Standard span-based NER with single encoder",
@@ -5066,6 +6372,7 @@ def get_model_type(self) -> str:
             "UniEncoderTokenDecoderGLiNER": "gliner_uni_encoder_token_decoder",
             "UniEncoderSpanRelexGLiNER": "gliner_uni_encoder_span_relex",
             "UniEncoderTokenRelexGLiNER": "gliner_uni_encoder_token_relex",
+            "StreamingSpanGLiNER": "gliner_streaming_span",
         }
 
         return type_mapping.get(class_name, "unknown")
diff --git a/gliner/modeling/__init__.py b/gliner/modeling/__init__.py
index 8b45611b..282beea9 100644
--- a/gliner/modeling/__init__.py
+++ b/gliner/modeling/__init__.py
@@ -2,6 +2,7 @@
     BaseModel,
     BaseBiEncoderModel,
     BiEncoderSpanModel,
+    StreamingSpanModel,
     BaseUniEncoderModel,
     BiEncoderTokenModel,
     UniEncoderSpanModel,
diff --git a/gliner/modeling/base.py b/gliner/modeling/base.py
index 71fd29bc..886cd10f 100644
--- a/gliner/modeling/base.py
+++ b/gliner/modeling/base.py
@@ -33,10 +33,16 @@
     extract_spans_from_tokens,
     extract_prompt_features_and_word_embeddings,
 )
-from .layers import CrossFuser, LstmSeq2SeqEncoder, create_projection_layer
+from .layers import CrossFuser, LstmSeq2SeqEncoder, StreamingSpanLabelsEncoder, create_projection_layer
 from .decoder import Decoder
 from .encoder import Encoder, BiEncoder
-from .outputs import GLiNERBaseOutput, GLiNERRelexOutput, GLiNERDecoderOutput
+from .outputs import (
+    GLiNERBaseOutput,
+    GLiNERRelexOutput,
+    GLiNERDecoderOutput,
+    GLiNERStreamingSpanOutput,
+    GLiNERRepresentationOutput,
+)
 from .scorers import Scorer
 from .span_rep import SpanRepLayer
 from .loss_functions import cross_entropy_loss, focal_loss_with_logits
@@ -121,11 +127,11 @@ def lookup_precomputed_prompts(
         return out, mask
 
     @abstractmethod
-    def get_representations(self) -> Tuple[torch.Tensor, ...]:
+    def get_representations(self) -> GLiNERRepresentationOutput:
         """Get intermediate representations from the model.
 
         Returns:
-            Tuple of tensors representing intermediate model outputs.
+            Structured intermediate model representations.
         """
         pass
 
@@ -160,6 +166,17 @@ def _fit_length(embedding: torch.Tensor, mask: torch.Tensor, target_len: int) ->
 
         return embedding, mask
 
+    @staticmethod
+    def _get_num_attention_heads(token_rep_layer: nn.Module) -> int:
+        """Read the attention-head count from encoder or decoder wrappers."""
+        if hasattr(token_rep_layer, "bert_layer"):
+            model_config = token_rep_layer.bert_layer.model.config
+        elif hasattr(token_rep_layer, "decoder_layer"):
+            model_config = token_rep_layer.decoder_layer.model.config
+        else:
+            raise TypeError("Unsupported token representation layer")
+        return model_config.num_attention_heads
+
     @abstractmethod
     def forward(self, x: Any) -> Any:
         """Forward pass through the model.
@@ -277,7 +294,7 @@ def __init__(
             cache_dir: Directory for caching pretrained models.
         """
         super().__init__(config, from_pretrained, cache_dir)
-        self.token_rep_layer = Encoder(config, from_pretrained, cache_dir=cache_dir)
+        self.token_rep_layer = self._init_token_rep_layer(config, from_pretrained, cache_dir=cache_dir)
 
         if self.config.num_rnn_layers > 0:
             self.rnn = LstmSeq2SeqEncoder(config, num_layers=self.config.num_rnn_layers)
@@ -286,12 +303,27 @@ def __init__(
             self.cross_fuser = CrossFuser(
                 self.config.hidden_size,
                 self.config.hidden_size,
-                num_heads=self.token_rep_layer.bert_layer.model.config.num_attention_heads,
+                num_heads=self._get_num_attention_heads(self.token_rep_layer),
                 num_layers=self.config.num_post_fusion_layers,
                 dropout=config.dropout,
                 schema=config.post_fusion_schema,
             )
 
+    def _init_token_rep_layer(
+        self,
+        config: Any,
+        from_pretrained: bool = False,
+        cache_dir: Optional[Union[str, Path]] = None,
+    ) -> nn.Module:
+        """Initialize the token representation layer.
+
+        Args:
+            config: Model configuration object.
+            from_pretrained: Whether to load from pretrained weights.
+            cache_dir: Directory for caching pretrained models.
+        """
+        return Encoder(config, from_pretrained, cache_dir=cache_dir)
+
     def _extract_prompt_features_and_word_embeddings(
         self,
         token_embeds: torch.Tensor,
@@ -324,6 +356,7 @@ def _extract_prompt_features_and_word_embeddings(
             text_lengths,
             words_mask,
             self.config.embed_ent_token,
+            subtoken_pooling=getattr(self.config, "subtoken_pooling", "first"),
         )
         return prompts_embedding, prompts_embedding_mask, words_embedding, mask
 
@@ -334,7 +367,7 @@ def get_representations(
         text_lengths: Optional[torch.Tensor] = None,
         words_mask: Optional[torch.LongTensor] = None,
         **kwargs: Any,
-    ) -> Tuple[torch.Tensor, torch.Tensor, torch.Tensor, torch.Tensor]:
+    ) -> GLiNERRepresentationOutput:
         """Get entity label and word representations from input.
 
         Args:
@@ -345,11 +378,7 @@ def get_representations(
             **kwargs: Additional arguments for the encoder.
 
         Returns:
-            Tuple containing:
-                - prompts_embedding: Entity label embeddings of shape (B, C, D).
-                - prompts_embedding_mask: Mask for prompts of shape (B, C).
-                - words_embedding: Word embeddings of shape (B, W, D).
-                - mask: Mask for words of shape (B, W).
+            Structured prompt and word representations.
         """
         word_lengths = kwargs.pop("word_lengths", None)
         token_embeds = self.token_rep_layer(input_ids, attention_mask, **kwargs)
@@ -362,7 +391,14 @@ def get_representations(
             batch_size, _, embed_dim = token_embeds.shape
             max_text_length = text_lengths.max()
             words_embedding, mask = extract_word_embeddings(
-                token_embeds, words_mask, attention_mask, batch_size, max_text_length, embed_dim, text_lengths
+                token_embeds,
+                words_mask,
+                attention_mask,
+                batch_size,
+                max_text_length,
+                embed_dim,
+                text_lengths,
+                getattr(self.config, "subtoken_pooling", "first"),
             )
             prompts_embedding, prompts_embedding_mask = self.lookup_precomputed_prompts(
                 batch_size, device=token_embeds.device, rel=False
@@ -377,7 +413,12 @@ def get_representations(
         if hasattr(self, "rnn"):
             words_embedding = self.rnn(words_embedding, mask, lengths=word_lengths)
 
-        return prompts_embedding, prompts_embedding_mask, words_embedding, mask
+        return GLiNERRepresentationOutput(
+            prompts_embedding=prompts_embedding,
+            prompts_embedding_mask=prompts_embedding_mask,
+            words_embedding=words_embedding,
+            mask=mask,
+        )
 
 
 class UniEncoderSpanModel(BaseUniEncoderModel):
@@ -407,6 +448,7 @@ def __init__(
             hidden_size=config.hidden_size,
             max_width=config.max_width,
             dropout=config.dropout,
+            context_encoder_config=getattr(config, "span_encoder_config", None),
         )
 
         self.prompt_rep_layer = create_projection_layer(config.hidden_size, config.dropout)
@@ -451,9 +493,13 @@ def forward(
             if key in kwargs
         }
 
-        prompts_embedding, prompts_embedding_mask, words_embedding, mask = self.get_representations(
+        representations = self.get_representations(
             input_ids, attention_mask, text_lengths, words_mask, **encoder_kwargs
         )
+        prompts_embedding = representations.prompts_embedding
+        prompts_embedding_mask = representations.prompts_embedding_mask
+        words_embedding = representations.words_embedding
+        mask = representations.mask
 
         target_W = span_idx.size(1) // self.config.max_width
         words_embedding, mask = self._fit_length(words_embedding, mask, target_W)
@@ -557,6 +603,445 @@ def loss(
         return loss
 
 
+class StreamingSpanModel(UniEncoderSpanModel):
+    def __init__(
+        self,
+        config: Any,
+        from_pretrained: bool = False,
+        cache_dir: Optional[Union[str, Path]] = None,
+    ) -> None:
+        """Initialize the streaming span model.
+
+        Args:
+            config: Model configuration object.
+            from_pretrained: Whether to load from pretrained weights.
+            cache_dir: Directory for caching pretrained models.
+        """
+        super().__init__(config, from_pretrained, cache_dir)
+        if self.token_rep_layer.decoder_hidden_size != config.hidden_size:
+            self.token_projection = nn.Linear(self.token_rep_layer.decoder_hidden_size, config.hidden_size)
+        self.labels_encoder = StreamingSpanLabelsEncoder(config)
+        self.span_rep_layer = SpanRepLayer(
+            span_mode=config.span_mode,
+            hidden_size=config.hidden_size,
+            max_width=config.max_width,
+            dropout=config.dropout,
+            context_encoder_config=getattr(config, "span_encoder_config", None),
+        )
+
+    def _init_token_rep_layer(
+        self,
+        config: Any,
+        from_pretrained: bool = False,
+        cache_dir: Optional[Union[str, Path]] = None,
+    ) -> Decoder:
+        """Initialize the token representation layer for the streaming span model.
+
+        Args:
+            config: Model configuration object.
+            from_pretrained: Whether to load from pretrained weights.
+            cache_dir: Directory for caching pretrained models.
+        """
+        return Decoder(config, from_pretrained, cache_dir=cache_dir, use_causal_lm=False)
+
+    @staticmethod
+    def _merge_cached_words(
+        past_word_embeddings: torch.Tensor,
+        past_word_mask: torch.Tensor,
+        current_word_embeddings: torch.Tensor,
+        current_word_mask: torch.Tensor,
+        past_word_length: Optional[Union[int, torch.Tensor]] = None,
+    ) -> Tuple[torch.Tensor, torch.Tensor]:
+        """Append each row's valid words into reusable capacity-backed storage.
+
+        ``past_word_length`` may be a scalar for the legacy batch-size-one path
+        or a vector for cached streaming batches.  When omitted, valid cached
+        words are compacted according to ``past_word_mask`` before appending.
+        """
+        if past_word_embeddings.dim() != 3 or current_word_embeddings.dim() != 3:
+            raise ValueError("Word embeddings must have shape (batch, words, hidden)")
+        if past_word_mask.shape != past_word_embeddings.shape[:2]:
+            raise ValueError("past_word_mask must match past_word_embeddings")
+        if current_word_mask.shape != current_word_embeddings.shape[:2]:
+            raise ValueError("current_word_mask must match current_word_embeddings")
+        if past_word_embeddings.size(0) != current_word_embeddings.size(0):
+            raise ValueError("Cached and current word batches must have the same size")
+
+        batch_size = past_word_embeddings.size(0)
+        inferred_lengths = past_word_length is None
+        if inferred_lengths:
+            past_lengths = past_word_mask.long().sum(dim=1)
+        elif isinstance(past_word_length, torch.Tensor):
+            past_lengths = past_word_length.to(device=past_word_mask.device, dtype=torch.long).reshape(-1)
+        else:
+            past_lengths = torch.full(
+                (batch_size,),
+                int(past_word_length),
+                dtype=torch.long,
+                device=past_word_mask.device,
+            )
+        if past_lengths.numel() == 1 and batch_size != 1:
+            past_lengths = past_lengths.expand(batch_size)
+        if past_lengths.numel() != batch_size:
+            raise ValueError("past_word_length must contain one value per batch row")
+
+        current_lengths = current_word_mask.long().sum(dim=1)
+        required_lengths = past_lengths + current_lengths
+        required = int(required_lengths.max().item()) if batch_size else 0
+        capacity = past_word_embeddings.size(1)
+
+        if inferred_lengths:
+            # Backward-compatible direct model calls return a compact tensor
+            # sized for the padded current word width. Capacity reservation is
+            # enabled only when callers supply authoritative cached lengths.
+            new_capacity = max(
+                required,
+                int(past_lengths.max().item()) + current_word_embeddings.size(1),
+            )
+            merged_embeddings = past_word_embeddings.new_empty(
+                past_word_embeddings.size(0),
+                new_capacity,
+                past_word_embeddings.size(2),
+            )
+            merged_mask = torch.zeros(
+                past_word_mask.size(0),
+                new_capacity,
+                dtype=torch.bool,
+                device=past_word_mask.device,
+            )
+        elif required > capacity:
+            new_capacity = max(required, max(1, capacity) * 2)
+            merged_embeddings = past_word_embeddings.new_empty(
+                past_word_embeddings.size(0),
+                new_capacity,
+                past_word_embeddings.size(2),
+            )
+            merged_mask = torch.zeros(
+                past_word_mask.size(0),
+                new_capacity,
+                dtype=torch.bool,
+                device=past_word_mask.device,
+            )
+            if capacity:
+                merged_embeddings[:, :capacity].copy_(past_word_embeddings)
+                merged_mask[:, :capacity] = past_word_mask.bool()
+        else:
+            merged_embeddings = past_word_embeddings
+            merged_mask = past_word_mask
+
+        # Older callers may supply a padded cache without authoritative lengths.
+        # Compact it once; explicit-length streaming paths preserve the prefix
+        # invariant and avoid copying historical words on every append.
+        if inferred_lengths:
+            past_valid = past_word_mask.bool()
+            past_rank = past_valid.long().cumsum(dim=1) - 1
+            past_rows = torch.arange(batch_size, device=past_word_mask.device).unsqueeze(1).expand_as(past_valid)
+            valid_values = past_word_embeddings[past_valid].clone()
+            valid_rows = past_rows[past_valid]
+            valid_columns = past_rank[past_valid]
+            merged_mask.zero_()
+            if valid_values.numel():
+                merged_embeddings[valid_rows, valid_columns] = valid_values
+                merged_mask[valid_rows, valid_columns] = True
+
+        current_valid = current_word_mask.bool()
+        current_rank = current_valid.long().cumsum(dim=1) - 1
+        destinations = past_lengths.unsqueeze(1) + current_rank
+        current_rows = torch.arange(batch_size, device=current_word_mask.device).unsqueeze(1).expand_as(current_valid)
+        if current_valid.any():
+            valid_rows = current_rows[current_valid]
+            valid_columns = destinations[current_valid]
+            merged_embeddings[valid_rows, valid_columns] = current_word_embeddings[current_valid]
+            merged_mask[valid_rows, valid_columns] = True
+        return merged_embeddings, merged_mask
+
+    def get_representations(
+        self,
+        input_ids: Optional[torch.FloatTensor] = None,
+        attention_mask: Optional[torch.LongTensor] = None,
+        label_attention_mask: Optional[torch.LongTensor] = None,
+        past_key_values: Optional[Any] = None,
+        past_word_embeddings: Optional[torch.FloatTensor] = None,
+        past_word_mask: Optional[torch.LongTensor] = None,
+        prompts_embedding: Optional[torch.FloatTensor] = None,
+        prompts_embedding_mask: Optional[torch.LongTensor] = None,
+        text_lengths: Optional[torch.Tensor] = None,
+        words_mask: Optional[torch.LongTensor] = None,
+        **kwargs: Any,
+    ) -> GLiNERRepresentationOutput:
+        """Get entity label and word representations from input.
+
+        Args:
+            input_ids: Input token IDs of shape (B, L).
+            attention_mask: Decoder attention mask. With cached keys this may include
+                both cached and current positions.
+            label_attention_mask: Mask for the current ``input_ids`` only. If omitted,
+                it is derived from the trailing part of ``attention_mask``.
+            past_key_values: Past key values for attention layers.
+            past_word_embeddings: Unprocessed word representations cached from prior calls.
+            past_word_mask: Valid-position mask for ``past_word_embeddings``.
+            prompts_embedding: Optional precomputed prompt representations.
+            prompts_embedding_mask: Mask for ``prompts_embedding``.
+            text_lengths: Length of each text in batch.
+            words_mask: Word boundary mask.
+            **kwargs: Additional arguments for the encoder.
+
+        Returns:
+            Structured prompt, word, and cache representations.
+        """
+        word_lengths = kwargs.pop("word_lengths", None)
+        past_word_length = kwargs.pop("past_word_length", None)
+        label_attention_mask = kwargs.pop("label_mask", label_attention_mask)
+        if input_ids is None or attention_mask is None:
+            raise ValueError("input_ids and attention_mask are required")
+
+        current_seq_length = input_ids.size(1)
+        if label_attention_mask is None:
+            if attention_mask.size(1) < current_seq_length:
+                raise ValueError("attention_mask cannot be shorter than input_ids")
+            label_attention_mask = attention_mask[:, -current_seq_length:]
+        elif label_attention_mask.shape != input_ids.shape:
+            raise ValueError("label_attention_mask must have the same shape as input_ids")
+
+        decoder_kwargs = dict(kwargs)
+        decoder_kwargs.setdefault("use_cache", True)
+        if past_key_values is not None:
+            decoder_kwargs["past_key_values"] = past_key_values
+        token_embeds, past_key_values = self.token_rep_layer(
+            input_ids,
+            attention_mask=attention_mask,
+            **decoder_kwargs,
+        )
+        if hasattr(self, "token_projection"):
+            # A pretrained causal backbone may honor its source config's BF16
+            # dtype while newly initialized GLiNER projections remain FP32.
+            # Cross the module boundary in the receiving layer's dtype.
+            token_embeds = token_embeds.to(dtype=self.token_projection.weight.dtype)
+            token_embeds = self.token_projection(token_embeds)
+
+        labels_encoder_dtype = next(self.labels_encoder.parameters()).dtype
+        if token_embeds.dtype != labels_encoder_dtype:
+            token_embeds = token_embeds.to(dtype=labels_encoder_dtype)
+
+        batch_size, _, embed_dim = token_embeds.shape
+        max_text_length = text_lengths.max()
+        current_words_embedding, current_word_mask = extract_word_embeddings(
+            token_embeds,
+            words_mask,
+            label_attention_mask,
+            batch_size,
+            max_text_length,
+            embed_dim,
+            text_lengths,
+            getattr(self.config, "subtoken_pooling", "first"),
+        )
+
+        use_precomputed = (
+            getattr(self.config, "precomputed_prompts_mode", False)
+            and getattr(self, "precomputed_prompts", None) is not None
+        )
+        if use_precomputed:
+            prompts_embedding, prompts_embedding_mask = self.lookup_precomputed_prompts(
+                batch_size, device=token_embeds.device, rel=False
+            )
+        elif prompts_embedding is not None:
+            if prompts_embedding_mask is None:
+                raise ValueError("prompts_embedding_mask is required with prompts_embedding")
+        else:
+            prompts_embedding, prompts_embedding_mask = self.labels_encoder(
+                token_embeds,
+                input_ids,
+                label_attention_mask,
+            )
+
+        if past_word_embeddings is not None:
+            if past_word_mask is None:
+                raise ValueError("past_word_mask is required with past_word_embeddings")
+            if past_word_length is None:
+                past_lengths = past_word_mask.long().sum(dim=1)
+            elif isinstance(past_word_length, torch.Tensor):
+                past_lengths = past_word_length.to(device=current_word_mask.device, dtype=torch.long).reshape(-1)
+                if past_lengths.numel() == 1 and batch_size != 1:
+                    past_lengths = past_lengths.expand(batch_size)
+            else:
+                past_lengths = torch.full(
+                    (batch_size,),
+                    int(past_word_length),
+                    dtype=torch.long,
+                    device=current_word_mask.device,
+                )
+            cached_words_embedding, cached_word_mask = self._merge_cached_words(
+                past_word_embeddings,
+                past_word_mask,
+                current_words_embedding,
+                current_word_mask,
+                past_word_length,
+            )
+            active_word_lengths = past_lengths + current_word_mask.long().sum(dim=1)
+            active_word_length = int(active_word_lengths.max().item())
+        else:
+            cached_words_embedding = current_words_embedding
+            cached_word_mask = current_word_mask
+            active_word_length = current_words_embedding.size(1)
+
+        active = slice(0, active_word_length)
+        active_words_embedding = cached_words_embedding[:, active]
+        active_word_mask = cached_word_mask[:, active]
+        words_embedding = active_words_embedding
+        if hasattr(self, "rnn"):
+            rnn_lengths = word_lengths if past_word_embeddings is None else None
+            words_embedding = self.rnn(words_embedding, active_word_mask, lengths=rnn_lengths)
+
+        return GLiNERRepresentationOutput(
+            prompts_embedding=prompts_embedding,
+            prompts_embedding_mask=prompts_embedding_mask,
+            words_embedding=words_embedding,
+            mask=active_word_mask,
+            past_key_values=past_key_values,
+            past_word_embeddings=cached_words_embedding,
+            past_word_mask=cached_word_mask,
+        )
+
+    def forward(
+        self,
+        input_ids: Optional[torch.FloatTensor] = None,
+        attention_mask: Optional[torch.LongTensor] = None,
+        label_attention_mask: Optional[torch.LongTensor] = None,
+        label_mask: Optional[torch.LongTensor] = None,
+        past_key_values: Optional[Any] = None,
+        past_word_embeddings: Optional[torch.FloatTensor] = None,
+        past_word_mask: Optional[torch.LongTensor] = None,
+        words_embedding: Optional[torch.FloatTensor] = None,
+        mask: Optional[torch.LongTensor] = None,
+        prompts_embedding: Optional[torch.FloatTensor] = None,
+        prompts_embedding_mask: Optional[torch.LongTensor] = None,
+        words_mask: Optional[torch.LongTensor] = None,
+        text_lengths: Optional[torch.Tensor] = None,
+        span_idx: Optional[torch.LongTensor] = None,
+        span_mask: Optional[torch.LongTensor] = None,
+        labels: Optional[torch.FloatTensor] = None,
+        **kwargs: Any,
+    ) -> GLiNERStreamingSpanOutput:
+        """Forward pass through the span-based model.
+
+        Args:
+            input_ids: Input token IDs of shape (B, L).
+            attention_mask: Attention mask of shape (B, L).
+            label_attention_mask: Mask for current input positions when the decoder
+                attention mask also contains cached positions.
+            label_mask: Alias for ``label_attention_mask`` used by cache batching helpers.
+            past_key_values: Optional reusable decoder KV cache.
+            past_word_embeddings: Optional reusable unprocessed word representations.
+            past_word_mask: Valid-position mask for cached word representations.
+            words_embedding: Backward-compatible alias for ``past_word_embeddings``.
+            mask: Backward-compatible alias for ``past_word_mask``.
+            prompts_embedding: Pre-computed entity label embeddings of shape (B, C, D).
+            prompts_embedding_mask: Mask for prompts of shape (B, C).
+            words_mask: Word boundary mask.
+            text_lengths: Length of each text sequence.
+            span_idx: Span indices of shape (B, L*K, 2).
+            span_mask: Mask for valid spans of shape (B, L, K).
+            labels: Ground truth labels of shape (B, L, K, C).
+            **kwargs: Additional arguments.
+
+        Returns:
+            Cache-aware streaming span output.
+        """
+        compact_cached_words = kwargs.get("past_word_length") is not None
+        representation_kwargs = {
+            key: kwargs[key]
+            for key in (
+                "position_ids",
+                "cache_position",
+                "use_cache",
+                "word_lengths",
+                "past_word_length",
+            )
+            if key in kwargs
+        }
+
+        if past_word_embeddings is None:
+            past_word_embeddings = words_embedding
+        if past_word_mask is None:
+            past_word_mask = mask
+        if label_attention_mask is None:
+            label_attention_mask = label_mask
+
+        representations = self.get_representations(
+            input_ids=input_ids,
+            attention_mask=attention_mask,
+            label_attention_mask=label_attention_mask,
+            past_key_values=past_key_values,
+            past_word_embeddings=past_word_embeddings,
+            past_word_mask=past_word_mask,
+            prompts_embedding=prompts_embedding,
+            prompts_embedding_mask=prompts_embedding_mask,
+            text_lengths=text_lengths,
+            words_mask=words_mask,
+            **representation_kwargs,
+        )
+        prompts_embedding = representations.prompts_embedding
+        prompts_embedding_mask = representations.prompts_embedding_mask
+        words_embedding = representations.words_embedding
+        mask = representations.mask
+
+        if span_idx is None or span_mask is None:
+            raise ValueError("span_idx and span_mask are required")
+
+        batch_size = words_embedding.size(0)
+        flat_span_idx = span_idx.view(batch_size, -1, 2)
+        flat_span_mask = span_mask.view(batch_size, -1).bool()
+        safe_span_idx = flat_span_idx * flat_span_mask.unsqueeze(-1).long()
+
+        span_rep, contextualized_words = self.span_rep_layer(
+            words_embedding,
+            safe_span_idx,
+            word_mask=mask,
+            return_words=True,
+            compact_words=compact_cached_words,
+        )
+
+        cached_prompts_embedding = prompts_embedding
+        cached_prompts_mask = prompts_embedding_mask
+        target_C = cached_prompts_embedding.size(1)
+        if labels is not None:
+            target_C = max(target_C, labels.size(-1))
+
+        cached_prompts_embedding, prompts_embedding_mask = self._fit_length(
+            cached_prompts_embedding, cached_prompts_mask, target_C
+        )
+
+        prompts_embedding = self.prompt_rep_layer(cached_prompts_embedding)
+        scores = torch.einsum("BLKD,BCD->BLKC", span_rep, prompts_embedding)
+
+        loss = None
+        if labels is not None:
+            loss = self.loss(
+                scores,
+                labels,
+                prompts_embedding_mask,
+                flat_span_mask,
+                **kwargs,
+            )
+
+        output = GLiNERStreamingSpanOutput(
+            logits=scores,
+            loss=loss,
+            prompts_embedding=prompts_embedding,
+            prompts_embedding_mask=prompts_embedding_mask,
+            words_embedding=contextualized_words,
+            mask=mask,
+            span_idx=flat_span_idx,
+            span_mask=flat_span_mask,
+            past_key_values=representations.past_key_values,
+            past_word_embeddings=representations.past_word_embeddings,
+            past_word_mask=representations.past_word_mask,
+            cached_prompts_embedding=cached_prompts_embedding,
+            cached_prompts_mask=prompts_embedding_mask,
+        )
+        return output
+
+
 class UniEncoderTokenModel(BaseUniEncoderModel):
     """Token-based NER model using uni-encoder architecture.
 
@@ -585,6 +1070,7 @@ def __init__(
                 hidden_size=config.hidden_size,
                 max_width=getattr(config, "max_width", 12),
                 dropout=config.dropout,
+                context_encoder_config=getattr(config, "span_encoder_config", None),
             )
 
     def get_span_representations(self, scores, span_idx, span_mask, words_embedding, labels, threshold):
@@ -652,9 +1138,13 @@ def forward(
             if key in kwargs
         }
 
-        prompts_embedding, prompts_embedding_mask, words_embedding, mask = self.get_representations(
+        representations = self.get_representations(
             input_ids, attention_mask, text_lengths, words_mask, **encoder_kwargs
         )
+        prompts_embedding = representations.prompts_embedding
+        prompts_embedding_mask = representations.prompts_embedding_mask
+        words_embedding = representations.words_embedding
+        mask = representations.mask
 
         if labels is not None:
             target_W = labels.shape[1]
@@ -795,7 +1285,7 @@ def __init__(
             self.cross_fuser = CrossFuser(
                 self.config.hidden_size,
                 self.config.hidden_size,
-                num_heads=self.token_rep_layer.bert_layer.model.config.num_attention_heads,
+                num_heads=self._get_num_attention_heads(self.token_rep_layer),
                 num_layers=self.config.num_post_fusion_layers,
                 dropout=config.dropout,
                 schema=config.post_fusion_schema,
@@ -834,7 +1324,7 @@ def get_representations(
         text_lengths: Optional[torch.Tensor] = None,
         words_mask: Optional[torch.LongTensor] = None,
         **kwargs: Any,
-    ) -> Tuple[torch.Tensor, torch.Tensor, torch.Tensor, torch.Tensor]:
+    ) -> GLiNERRepresentationOutput:
         """Get entity label and word representations using bi-encoder.
 
         Args:
@@ -848,11 +1338,7 @@ def get_representations(
             **kwargs: Additional arguments for the encoder.
 
         Returns:
-            Tuple containing:
-                - labels_embeds: Label embeddings of shape (B, C, D).
-                - labels_mask: Mask for labels of shape (B, C).
-                - words_embedding: Word embeddings of shape (B, W, D).
-                - mask: Mask for words of shape (B, W).
+            Structured label and word representations.
         """
         # word_lengths is only consumed by the RNN path; drop it here so it does
         # not leak into the label encoder's forward as an unexpected kwarg.
@@ -869,7 +1355,14 @@ def get_representations(
         max_text_length = text_lengths.max()
 
         words_embedding, mask = extract_word_embeddings(
-            token_embeds, words_mask, attention_mask, batch_size, max_text_length, embed_dim, text_lengths
+            token_embeds,
+            words_mask,
+            attention_mask,
+            batch_size,
+            max_text_length,
+            embed_dim,
+            text_lengths,
+            getattr(self.config, "subtoken_pooling", "first"),
         )
 
         labels_embeds = labels_embeds.unsqueeze(0)
@@ -883,7 +1376,12 @@ def get_representations(
                 words_embedding, labels_embeds, text_mask=mask, labels_mask=labels_mask
             )
 
-        return labels_embeds, labels_mask, words_embedding, mask
+        return GLiNERRepresentationOutput(
+            prompts_embedding=labels_embeds,
+            prompts_embedding_mask=labels_mask,
+            words_embedding=words_embedding,
+            mask=mask,
+        )
 
 
 class BiEncoderSpanModel(BaseBiEncoderModel):
@@ -910,6 +1408,7 @@ def __init__(
             hidden_size=config.hidden_size,
             max_width=config.max_width,
             dropout=config.dropout,
+            context_encoder_config=getattr(config, "span_encoder_config", None),
         )
 
         self.prompt_rep_layer = create_projection_layer(config.hidden_size, config.dropout)
@@ -960,7 +1459,7 @@ def forward(
             if key in kwargs
         }
 
-        prompts_embedding, prompts_embedding_mask, words_embedding, mask = self.get_representations(
+        representations = self.get_representations(
             input_ids,
             attention_mask,
             labels_embeds,
@@ -970,6 +1469,10 @@ def forward(
             words_mask,
             **encoder_kwargs,
         )
+        prompts_embedding = representations.prompts_embedding
+        prompts_embedding_mask = representations.prompts_embedding_mask
+        words_embedding = representations.words_embedding
+        mask = representations.mask
 
         target_W = span_idx.size(1) // self.config.max_width
         words_embedding, mask = self._fit_length(words_embedding, mask, target_W)
@@ -1141,7 +1644,7 @@ def forward(
             if key in kwargs
         }
 
-        prompts_embedding, prompts_embedding_mask, words_embedding, mask = self.get_representations(
+        representations = self.get_representations(
             input_ids,
             attention_mask,
             labels_embeds,
@@ -1151,6 +1654,10 @@ def forward(
             words_mask,
             **encoder_kwargs,
         )
+        prompts_embedding = representations.prompts_embedding
+        prompts_embedding_mask = representations.prompts_embedding_mask
+        words_embedding = representations.words_embedding
+        mask = representations.mask
 
         if labels is not None:
             target_W = labels.shape[1]
@@ -1219,16 +1726,17 @@ def __init__(
             cache_dir: Directory for caching pretrained models.
         """
         super().__init__(config, from_pretrained, cache_dir)
-        self.__init_decoder__(config, from_pretrained, cache_dir)
+        self.decoder = self._init_decoder(config, from_pretrained, cache_dir)
 
-    def __init_decoder__(self, config, from_pretrained, cache_dir):
-        self.decoder = Decoder(config, from_pretrained, cache_dir=cache_dir)
-        if self.config.hidden_size != self.decoder.decoder_hidden_size:
+    def _init_decoder(self, config, from_pretrained, cache_dir) -> Decoder:
+        decoder = Decoder(config, from_pretrained, cache_dir=cache_dir)
+        if self.config.hidden_size != decoder.decoder_hidden_size:
             self._enc2dec_proj = create_projection_layer(
                 self.config.hidden_size,
                 self.config.dropout,
-                self.decoder.decoder_hidden_size,
+                decoder.decoder_hidden_size,
             )
+        return decoder
 
     def select_decoder_embedding(
         self,
@@ -1340,7 +1848,7 @@ def decode_labels(
 
         attn_inputs = torch.cat([span_tokens_mask.to(decoder_labels_mask.dtype), decoder_labels_mask[:, :-1]], dim=1)
 
-        decoder_outputs = self.decoder(inputs_embeds=decoder_inputs, attention_mask=attn_inputs)
+        decoder_outputs, _ = self.decoder(inputs_embeds=decoder_inputs, attention_mask=attn_inputs)
 
         blank_for_spans = torch.full(
             (decoder_labels.size(0), span_tokens.size(1)),
@@ -1566,9 +2074,13 @@ def forward(
             if key in kwargs
         }
 
-        prompts_embedding, prompts_embedding_mask, words_embedding, mask = self.get_representations(
+        representations = self.get_representations(
             input_ids, attention_mask, text_lengths, words_mask, **encoder_kwargs
         )
+        prompts_embedding = representations.prompts_embedding
+        prompts_embedding_mask = representations.prompts_embedding_mask
+        words_embedding = representations.words_embedding
+        mask = representations.mask
 
         target_W = span_idx.size(1) // self.config.max_width
         words_embedding, mask = self._fit_length(words_embedding, mask, target_W)
@@ -1732,9 +2244,8 @@ def __init__(
             from_pretrained: Whether to load from pretrained weights.
             cache_dir: Directory for caching pretrained models.
         """
-        # Initialize through UniEncoderTokenModel's chain to get scorer
+        # Cooperative MRO initializes both the token scorer and decoder.
         super().__init__(config, from_pretrained, cache_dir)
-        self.__init_decoder__(config, from_pretrained, cache_dir)
 
     def select_token_decoder_embedding(
         self,
@@ -1918,9 +2429,13 @@ def forward(
             if key in kwargs
         }
 
-        prompts_embedding, prompts_embedding_mask, words_embedding, mask = self.get_representations(
+        representations = self.get_representations(
             input_ids, attention_mask, text_lengths, words_mask, **encoder_kwargs
         )
+        prompts_embedding = representations.prompts_embedding
+        prompts_embedding_mask = representations.prompts_embedding_mask
+        words_embedding = representations.words_embedding
+        mask = representations.mask
 
         if labels is not None:
             target_W = labels.shape[1]
@@ -2319,6 +2834,7 @@ def forward(
                 max_text_length,
                 embed_dim_e,
                 text_lengths,
+                getattr(self.config, "subtoken_pooling", "first"),
             )
             prompts_embedding, prompts_embedding_mask = self.lookup_precomputed_prompts(
                 batch_size_e, device=token_embeds.device, rel=False
diff --git a/gliner/modeling/cache.py b/gliner/modeling/cache.py
new file mode 100644
index 00000000..6b2ceaad
--- /dev/null
+++ b/gliner/modeling/cache.py
@@ -0,0 +1,548 @@
+"""Session cache support for decoder-backed span inference."""
+
+from __future__ import annotations
+
+from typing import Any, Iterable, Sequence
+from threading import RLock
+from dataclasses import field, replace, dataclass
+
+import torch
+
+try:
+    from transformers.cache_utils import DynamicCache, DynamicLayer
+except ImportError:  # transformers < 5 uses list-backed DynamicCache storage.
+    from transformers.cache_utils import DynamicCache
+
+    DynamicLayer = None
+
+
+DEFAULT_CACHE_INITIAL_CAPACITY = 256
+
+
+def _empty_long() -> torch.Tensor:
+    return torch.empty(0, dtype=torch.long)
+
+
+def _device_matches(actual: torch.device, requested: torch.device) -> bool:
+    return actual.type == requested.type and (
+        requested.index is None or actual.index == requested.index
+    )
+
+
+if DynamicLayer is not None:
+
+    class ReusableDynamicLayer(DynamicLayer):
+        """Append-only KV storage that grows geometrically instead of every turn."""
+
+        def __init__(
+            self,
+            initial_capacity: int = DEFAULT_CACHE_INITIAL_CAPACITY,
+            max_cache_length: int | None = None,
+        ) -> None:
+            super().__init__()
+            self.initial_capacity = initial_capacity
+            self.maximum_length = max_cache_length
+            self.current_length = 0
+            self.capacity = 0
+
+        def _next_capacity(self, required: int) -> int:
+            if self.maximum_length is not None and required > self.maximum_length:
+                raise ValueError(
+                    f"KV cache requires {required} positions, exceeding its "
+                    f"maximum length of {self.maximum_length}"
+                )
+            capacity = max(1, self.initial_capacity, self.capacity)
+            while capacity < required:
+                capacity *= 2
+            if self.maximum_length is not None:
+                capacity = min(capacity, self.maximum_length)
+            return capacity
+
+        @staticmethod
+        def _allocate(source: torch.Tensor, capacity: int) -> torch.Tensor:
+            shape = (*source.shape[:-2], capacity, source.shape[-1])
+            return source.new_empty(shape)
+
+        def lazy_initialization(
+            self,
+            key_states: torch.Tensor,
+            value_states: torch.Tensor,
+        ) -> None:
+            self.dtype, self.device = key_states.dtype, key_states.device
+            self.capacity = self._next_capacity(key_states.shape[-2])
+            self.keys = self._allocate(key_states, self.capacity)
+            self.values = self._allocate(value_states, self.capacity)
+            self.current_length = 0
+            self.is_initialized = True
+
+        def _grow(self, required: int) -> None:
+            if required <= self.capacity:
+                return
+            new_capacity = self._next_capacity(required)
+            new_keys = self._allocate(self.keys, new_capacity)
+            new_values = self._allocate(self.values, new_capacity)
+            if self.current_length:
+                active = slice(0, self.current_length)
+                new_keys[..., active, :].copy_(self.keys[..., active, :])
+                new_values[..., active, :].copy_(self.values[..., active, :])
+            self.keys = new_keys
+            self.values = new_values
+            self.capacity = new_capacity
+
+        def update(
+            self,
+            key_states: torch.Tensor,
+            value_states: torch.Tensor,
+            *args,
+            **kwargs,
+        ) -> tuple[torch.Tensor, torch.Tensor]:
+            del args, kwargs
+            if not self.is_initialized:
+                self.lazy_initialization(key_states, value_states)
+
+            new_length = key_states.shape[-2]
+            required = self.current_length + new_length
+            self._grow(required)
+            destination = slice(self.current_length, required)
+            self.keys[..., destination, :].copy_(key_states)
+            self.values[..., destination, :].copy_(value_states)
+            self.current_length = required
+            active = slice(0, required)
+            return self.keys[..., active, :], self.values[..., active, :]
+
+        def get_seq_length(self) -> int:
+            return self.current_length if self.is_initialized else 0
+
+        def get_max_cache_shape(self) -> int:
+            # The active attention length remains dynamic even though backing
+            # storage is reserved in larger blocks.
+            return -1
+
+        @property
+        def max_batch_size(self) -> int:
+            return self.keys.shape[0] if self.is_initialized else 0
+
+        @property
+        def max_cache_len(self) -> int:
+            return self.maximum_length if self.maximum_length is not None else -1
+
+        def reset(self) -> None:
+            self.current_length = 0
+
+        def crop(self, max_length: int) -> None:
+            if max_length < 0:
+                max_length = self.current_length - abs(max_length)
+            self.current_length = max(0, min(self.current_length, max_length))
+
+        def batch_repeat_interleave(self, repeats: int) -> None:
+            if self.is_initialized:
+                self.keys = self.keys.repeat_interleave(repeats, dim=0)
+                self.values = self.values.repeat_interleave(repeats, dim=0)
+
+        def batch_select_indices(self, indices: torch.Tensor) -> None:
+            if self.is_initialized:
+                self.keys = self.keys[indices, ...]
+                self.values = self.values[indices, ...]
+
+        def move_to(self, device: torch.device | str) -> None:
+            if not self.is_initialized:
+                return
+            self.keys = self.keys.to(device)
+            self.values = self.values.to(device)
+            self.device = self.keys.device
+
+else:
+    ReusableDynamicLayer = None
+
+
+class ReusableDynamicCache(DynamicCache):
+    """DynamicCache with stable, capacity-backed tensors when supported."""
+
+    def __init__(
+        self,
+        config=None,
+        initial_capacity: int = DEFAULT_CACHE_INITIAL_CAPACITY,
+        max_cache_length: int | None = None,
+    ) -> None:
+        if initial_capacity < 1:
+            raise ValueError("initial_capacity must be positive")
+        self.initial_capacity = initial_capacity
+        self.maximum_length = max_cache_length
+
+        if DynamicLayer is None:
+            # transformers 4.x has a list-backed cache API. Keep its compatible
+            # behavior; CacheState still avoids reconstructing it on every turn.
+            super().__init__()
+            self.uses_reusable_storage = False
+            return
+
+        super().__init__(config=config)
+        self.uses_reusable_storage = True
+
+        def new_layer():
+            return ReusableDynamicLayer(initial_capacity, max_cache_length)
+
+        for layer_index, layer in enumerate(self.layers):
+            # Preserve bounded sliding-window and recurrent/hybrid cache layers.
+            if type(layer) is DynamicLayer:
+                self.layers[layer_index] = new_layer()
+        if self.layer_class_to_replicate is DynamicLayer:
+            self.layer_class_to_replicate = new_layer
+
+    def move_to(self, device: torch.device | str):
+        if hasattr(self, "layers"):
+            for layer in self.layers:
+                if isinstance(layer, ReusableDynamicLayer):
+                    layer.move_to(device)
+                elif getattr(layer, "is_initialized", False):
+                    layer.keys = layer.keys.to(device)
+                    layer.values = layer.values.to(device)
+                    if hasattr(layer, "device"):
+                        layer.device = layer.keys.device
+        else:
+            self.key_cache = [tensor.to(device) for tensor in self.key_cache]
+            self.value_cache = [tensor.to(device) for tensor in self.value_cache]
+        return self
+
+
+def create_reusable_cache(
+    config,
+    *,
+    initial_capacity: int = DEFAULT_CACHE_INITIAL_CAPACITY,
+    max_cache_length: int | None = None,
+) -> ReusableDynamicCache:
+    """Create a session KV cache whose storage is reused between appends."""
+    if max_cache_length is not None:
+        initial_capacity = min(initial_capacity, max_cache_length)
+    return ReusableDynamicCache(
+        config=config,
+        initial_capacity=initial_capacity,
+        max_cache_length=max_cache_length,
+    )
+
+
+@dataclass
+class CacheState:
+    """All reusable state belonging to one streaming-span session."""
+
+    past_key_values: Any = None
+    input_ids: torch.Tensor = field(default_factory=_empty_long)
+    attention_mask: torch.Tensor = field(default_factory=_empty_long)
+    token_word_mask: torch.Tensor = field(default_factory=_empty_long)
+    past_word_embeddings: torch.Tensor | None = None
+    past_word_mask: torch.Tensor | None = None
+    prompts_embedding: torch.Tensor | None = None
+    prompts_mask: torch.Tensor | None = None
+    cached_length: int = 0
+    next_position_id: int | None = None
+    session_id: str | None = None
+    labels: tuple[str, ...] = ()
+    text: str = ""
+    tokens: list[str] = field(default_factory=list)
+    char_starts: list[int] = field(default_factory=list)
+    char_ends: list[int] = field(default_factory=list)
+    span_logits: dict[tuple[int, int], torch.Tensor] = field(default_factory=dict)
+    metadata: dict[str, Any] = field(default_factory=dict)
+
+    @property
+    def next_position(self) -> int:
+        """Return the absolute decoder position for the next token."""
+        return self.cached_length if self.next_position_id is None else self.next_position_id
+
+    @property
+    def word_length(self) -> int:
+        """Return the authoritative cached word count without a device sync."""
+        return len(self.tokens)
+
+    def to(self, device: torch.device | str) -> CacheState:
+        """Move reusable model tensors only when the target device changes."""
+        requested = torch.device(device)
+        model_tensors = (
+            self.input_ids,
+            self.attention_mask,
+            self.token_word_mask,
+            self.past_word_embeddings,
+            self.past_word_mask,
+            self.prompts_embedding,
+            self.prompts_mask,
+        )
+        tensors_match = all(
+            tensor is None or _device_matches(tensor.device, requested)
+            for tensor in model_tensors
+        )
+        if tensors_match and _past_kv_on_device(self.past_key_values, requested):
+            # The normal streaming path stays on one device. Returning the exact
+            # state preserves the cache object and its preallocated KV storage.
+            return self
+
+        return replace(
+            self,
+            past_key_values=_move_past_kv(self.past_key_values, device),
+            input_ids=self.input_ids.to(device),
+            attention_mask=self.attention_mask.to(device),
+            token_word_mask=self.token_word_mask.to(device),
+            past_word_embeddings=(
+                self.past_word_embeddings.to(device) if self.past_word_embeddings is not None else None
+            ),
+            past_word_mask=self.past_word_mask.to(device) if self.past_word_mask is not None else None,
+            prompts_embedding=self.prompts_embedding.to(device) if self.prompts_embedding is not None else None,
+            prompts_mask=self.prompts_mask.to(device) if self.prompts_mask is not None else None,
+            tokens=list(self.tokens),
+            char_starts=list(self.char_starts),
+            char_ends=list(self.char_ends),
+            # Span scores intentionally remain on CPU so long-running sessions
+            # do not retain an ever-growing GPU prediction history.
+            span_logits=self.span_logits.copy(),
+            metadata=self.metadata.copy(),
+        )
+
+
+class SessionCacheManager:
+    """Thread-safe ownership and lifecycle management for session states."""
+
+    def __init__(self) -> None:
+        self._states: dict[str, CacheState] = {}
+        self._lock = RLock()
+
+    def get(self, session_id: str) -> CacheState | None:
+        with self._lock:
+            return self._states.get(session_id)
+
+    def put(self, state: CacheState) -> None:
+        if state.session_id is None:
+            raise ValueError("A cached state must have a session_id")
+        with self._lock:
+            self._states[state.session_id] = state
+
+    def clear(self, session_ids: str | Iterable[str] | None = None) -> None:
+        with self._lock:
+            if session_ids is None:
+                self._states.clear()
+                return
+            if isinstance(session_ids, str):
+                session_ids = [session_ids]
+            for session_id in session_ids:
+                self._states.pop(session_id, None)
+
+    def __len__(self) -> int:
+        with self._lock:
+            return len(self._states)
+
+
+def _cache_layers(past_kv):
+    """Yield ``(layer_index, keys, values)`` across cache API generations."""
+    layers = getattr(past_kv, "layers", None)
+    if layers is not None:
+        for layer_idx, layer in enumerate(layers):
+            initialized = getattr(layer, "is_initialized", True)
+            if callable(initialized):
+                initialized = initialized()
+            keys = getattr(layer, "keys", None)
+            values = getattr(layer, "values", None)
+            if initialized and keys is not None and values is not None:
+                if ReusableDynamicLayer is not None and isinstance(
+                    layer, ReusableDynamicLayer
+                ):
+                    active = slice(0, layer.get_seq_length())
+                    keys = keys[..., active, :]
+                    values = values[..., active, :]
+                yield layer_idx, keys, values
+        return
+
+    # transformers 4.x DynamicCache stores tensors in parallel lists.  Keep
+    # support for it because GLiNER's dependency range spans both cache APIs.
+    key_cache = getattr(past_kv, "key_cache", None)
+    value_cache = getattr(past_kv, "value_cache", None)
+    if key_cache is not None and value_cache is not None:
+        for layer_idx, (keys, values) in enumerate(zip(key_cache, value_cache)):
+            if torch.is_tensor(keys) and torch.is_tensor(values):
+                yield layer_idx, keys, values
+
+
+def _dynamic_cache_from_layers(layer_values) -> DynamicCache:
+    cache = DynamicCache()
+    for layer_idx, keys, values in layer_values:
+        cache.update(keys, values, layer_idx)
+    return cache
+
+
+def _is_dynamic_cache_like(past_kv) -> bool:
+    return hasattr(past_kv, "layers") or (
+        hasattr(past_kv, "key_cache") and hasattr(past_kv, "value_cache")
+    )
+
+
+def _past_kv_on_device(past_kv, device: torch.device) -> bool:
+    if past_kv is None:
+        return True
+    if isinstance(past_kv, (tuple, list)):
+        return all(_past_kv_on_device(item, device) for item in past_kv)
+    cache_layers = list(_cache_layers(past_kv))
+    return all(
+        _device_matches(keys.device, device) and _device_matches(values.device, device)
+        for _, keys, values in cache_layers
+    )
+
+
+def _move_past_kv(past_kv, device):
+    if past_kv is None:
+        return None
+    if isinstance(past_kv, ReusableDynamicCache):
+        return past_kv.move_to(device)
+    if isinstance(past_kv, (tuple, list)):
+        return type(past_kv)(_move_past_kv(item, device) for item in past_kv)
+
+    cache_layers = list(_cache_layers(past_kv))
+    if cache_layers or _is_dynamic_cache_like(past_kv):
+        return _dynamic_cache_from_layers(
+            (layer_idx, keys.to(device), values.to(device))
+            for layer_idx, keys, values in cache_layers
+        )
+    if hasattr(past_kv, "to"):
+        return past_kv.to(device)
+    return past_kv
+
+
+def _cache_from_layers(
+    layer_values,
+    *,
+    config=None,
+    max_cache_length: int | None = None,
+) -> ReusableDynamicCache:
+    """Build a reusable cache from complete per-layer key/value tensors."""
+    cache = create_reusable_cache(config, max_cache_length=max_cache_length)
+    for layer_idx, keys, values in layer_values:
+        cache.update(keys, values, layer_idx)
+    return cache
+
+
+def copy_past_key_values(
+    past_kv,
+    *,
+    config=None,
+    max_cache_length: int | None = None,
+):
+    """Return an independent copy of a transformers-style KV cache.
+
+    Streaming cache layers append in place.  Tests, transactional helpers, and
+    dynamic batching therefore need an explicit copy operation rather than a
+    shallow ``copy.copy`` of the cache container.
+    """
+    if past_kv is None:
+        return None
+    if torch.is_tensor(past_kv):
+        return past_kv.detach().clone()
+    if isinstance(past_kv, (tuple, list)):
+        return type(past_kv)(
+            copy_past_key_values(
+                item,
+                config=config,
+                max_cache_length=max_cache_length,
+            )
+            for item in past_kv
+        )
+
+    cache_layers = list(_cache_layers(past_kv))
+    if cache_layers or _is_dynamic_cache_like(past_kv):
+        return _cache_from_layers(
+            (
+                (
+                    layer_idx,
+                    keys.detach().clone(),
+                    values.detach().clone(),
+                )
+                for layer_idx, keys, values in cache_layers
+            ),
+            config=config,
+            max_cache_length=max_cache_length,
+        )
+    raise TypeError(f"Unsupported past_key_values type: {type(past_kv).__name__}")
+
+
+def stack_past_key_values(
+    caches: Sequence[Any],
+    *,
+    config=None,
+    max_cache_length: int | None = None,
+) -> ReusableDynamicCache:
+    """Stack equal-length, batch-size-one caches along their batch dimension."""
+    if not caches:
+        raise ValueError("At least one cache is required")
+
+    cache_layers = [list(_cache_layers(cache)) for cache in caches]
+    if any(not layers for layers in cache_layers):
+        raise TypeError("Every cache must expose initialized key/value layers")
+
+    layer_indices = [tuple(layer_idx for layer_idx, _, _ in layers) for layers in cache_layers]
+    if any(indices != layer_indices[0] for indices in layer_indices[1:]):
+        raise ValueError("Caches must contain the same initialized layers")
+
+    stacked_layers = []
+    for layer_offset, layer_idx in enumerate(layer_indices[0]):
+        keys = [layers[layer_offset][1] for layers in cache_layers]
+        values = [layers[layer_offset][2] for layers in cache_layers]
+        reference_key_shape = keys[0].shape[1:]
+        reference_value_shape = values[0].shape[1:]
+        if any(key.size(0) != 1 or key.shape[1:] != reference_key_shape for key in keys):
+            raise ValueError("Only compatible batch-size-one caches can be stacked")
+        if any(value.size(0) != 1 or value.shape[1:] != reference_value_shape for value in values):
+            raise ValueError("Only compatible batch-size-one caches can be stacked")
+        stacked_layers.append(
+            (
+                layer_idx,
+                torch.cat(keys, dim=0),
+                torch.cat(values, dim=0),
+            )
+        )
+
+    return _cache_from_layers(
+        stacked_layers,
+        config=config,
+        max_cache_length=max_cache_length,
+    )
+
+
+def split_past_key_values(
+    past_kv,
+    lengths: Sequence[int],
+    *,
+    config=None,
+    max_cache_length: int | None = None,
+) -> list[ReusableDynamicCache]:
+    """Split a batched cache into compact batch-size-one session caches."""
+    normalized_lengths = [int(length) for length in lengths]
+    if not normalized_lengths:
+        return []
+    if any(length < 0 for length in normalized_lengths):
+        raise ValueError("Cache lengths must be non-negative")
+
+    layers = list(_cache_layers(past_kv))
+    if not layers:
+        raise TypeError("past_key_values must expose initialized key/value layers")
+    batch_size = layers[0][1].size(0)
+    if len(normalized_lengths) != batch_size:
+        raise ValueError("One cache length is required per batch row")
+
+    results = []
+    for row, length in enumerate(normalized_lengths):
+        row_layers = []
+        for layer_idx, keys, values in layers:
+            if keys.size(0) != batch_size or values.size(0) != batch_size:
+                raise ValueError("All cache layers must have the same batch size")
+            if length > keys.size(-2) or length > values.size(-2):
+                raise ValueError("Requested cache length exceeds the available cache")
+            row_layers.append(
+                (
+                    layer_idx,
+                    keys[row : row + 1, ..., :length, :].detach().clone(),
+                    values[row : row + 1, ..., :length, :].detach().clone(),
+                )
+            )
+        results.append(
+            _cache_from_layers(
+                row_layers,
+                config=config,
+                max_cache_length=max_cache_length,
+            )
+        )
+    return results
diff --git a/gliner/modeling/context_encoders.py b/gliner/modeling/context_encoders.py
new file mode 100644
index 00000000..7b03b93d
--- /dev/null
+++ b/gliner/modeling/context_encoders.py
@@ -0,0 +1,204 @@
+"""Context encoders for already embedded token or word sequences."""
+
+from abc import ABC, abstractmethod
+from copy import deepcopy
+from typing import Union, Optional
+
+import torch
+from torch import nn
+from transformers import DebertaV2Config, ModernBertModel, ModernBertConfig, PretrainedConfig
+from torch.nn.utils.rnn import pad_packed_sequence, pack_padded_sequence
+from transformers.models.deberta_v2.modeling_deberta_v2 import DebertaV2Encoder
+
+from ..config import RNNEncoderConfig, normalize_context_encoder_config
+
+
+class BaseContextEncoder(ABC, nn.Module):
+    """Common interface for contextualizing dense sequence representations."""
+
+    def __init__(self, input_size: int, encoder_size: int, output_size: int) -> None:
+        super().__init__()
+        self.input_size = input_size
+        self.encoder_size = encoder_size
+        self.output_size = output_size
+        self.input_projection = nn.Identity() if input_size == encoder_size else nn.Linear(input_size, encoder_size)
+        self.output_projection = nn.Identity() if encoder_size == output_size else nn.Linear(encoder_size, output_size)
+
+    @property
+    @abstractmethod
+    def requires_full_recompute(self) -> bool:
+        """Whether appended positions can change all earlier representations."""
+
+    @abstractmethod
+    def _encode(self, hidden_states: torch.Tensor, attention_mask: torch.Tensor) -> torch.Tensor:
+        """Run the architecture-specific encoder at ``encoder_size`` width."""
+
+    def _parameter_dtype(self, fallback: torch.dtype) -> torch.dtype:
+        parameter = next(self.parameters(), None)
+        return parameter.dtype if parameter is not None and parameter.is_floating_point() else fallback
+
+    def forward(self, hidden_states: torch.Tensor, attention_mask: torch.Tensor) -> torch.Tensor:
+        if hidden_states.ndim != 3:
+            raise ValueError("hidden_states must have shape [batch, sequence, hidden_size]")
+        if attention_mask.shape != hidden_states.shape[:2]:
+            raise ValueError("attention_mask must match the first two hidden_states dimensions")
+
+        hidden_states = hidden_states.to(dtype=self._parameter_dtype(hidden_states.dtype))
+        hidden_states = self.input_projection(hidden_states)
+        hidden_states = self._encode(hidden_states, attention_mask)
+        hidden_states = self.output_projection(hidden_states)
+        return hidden_states.masked_fill(~attention_mask.bool().unsqueeze(-1), 0)
+
+
+class IdentityContextEncoder(BaseContextEncoder):
+    """No-op encoder used when contextualization is disabled."""
+
+    def __init__(self, hidden_size: int) -> None:
+        super().__init__(hidden_size, hidden_size, hidden_size)
+
+    @property
+    def requires_full_recompute(self) -> bool:
+        return False
+
+    def _encode(self, hidden_states: torch.Tensor, attention_mask: torch.Tensor) -> torch.Tensor:
+        return hidden_states
+
+
+class DebertaV2ContextEncoder(BaseContextEncoder):
+    """DeBERTa-v2 encoder stack operating directly on dense representations."""
+
+    def __init__(self, config: DebertaV2Config, input_size: int, output_size: int) -> None:
+        super().__init__(input_size, config.hidden_size, output_size)
+        self.config = config
+        self.encoder = DebertaV2Encoder(config)
+
+    @property
+    def requires_full_recompute(self) -> bool:
+        return True
+
+    def _encode(self, hidden_states: torch.Tensor, attention_mask: torch.Tensor) -> torch.Tensor:
+        output = self.encoder(
+            hidden_states,
+            attention_mask=attention_mask,
+            return_dict=True,
+        )
+        return output.last_hidden_state
+
+
+class ModernBertContextEncoder(BaseContextEncoder):
+    """ModernBERT model using its public dense ``inputs_embeds`` path."""
+
+    def __init__(self, config: ModernBertConfig, input_size: int, output_size: int) -> None:
+        super().__init__(input_size, config.hidden_size, output_size)
+        self.config = config
+
+        # Be defensive when callers provide a pre-built config with the standard
+        # vocabulary: no token IDs are consumed by this encoder.
+        model_config = deepcopy(config)
+        model_config.vocab_size = 1
+        model_config.pad_token_id = 0
+        model_config.bos_token_id = 0
+        model_config.eos_token_id = 0
+        model_config.cls_token_id = 0
+        model_config.sep_token_id = 0
+        model_config.tie_word_embeddings = False
+        self.encoder = ModernBertModel(model_config)
+
+    @property
+    def requires_full_recompute(self) -> bool:
+        return True
+
+    def _encode(self, hidden_states: torch.Tensor, attention_mask: torch.Tensor) -> torch.Tensor:
+        output = self.encoder(
+            inputs_embeds=hidden_states,
+            attention_mask=attention_mask,
+            return_dict=True,
+        )
+        return output.last_hidden_state
+
+
+class RNNContextEncoder(BaseContextEncoder):
+    """Packed LSTM context encoder with optional bidirectionality."""
+
+    def __init__(self, config: RNNEncoderConfig, input_size: int, output_size: int) -> None:
+        super().__init__(input_size, config.hidden_size, output_size)
+        self.config = config
+        num_directions = 2 if config.bidirectional else 1
+        recurrent_hidden_size = config.hidden_size // num_directions
+        self.encoder = nn.LSTM(
+            input_size=config.hidden_size,
+            hidden_size=recurrent_hidden_size,
+            num_layers=config.num_hidden_layers,
+            dropout=config.dropout if config.num_hidden_layers > 1 else 0.0,
+            bidirectional=config.bidirectional,
+            batch_first=True,
+        )
+
+    @property
+    def requires_full_recompute(self) -> bool:
+        return self.config.bidirectional
+
+    def _encode(self, hidden_states: torch.Tensor, attention_mask: torch.Tensor) -> torch.Tensor:
+        batch_size, sequence_length, _ = hidden_states.shape
+        lengths = attention_mask.long().sum(dim=1)
+        nonempty = lengths.gt(0)
+        output = hidden_states.new_zeros(batch_size, sequence_length, self.encoder_size)
+        if not nonempty.any():
+            return output
+
+        packed = pack_padded_sequence(
+            hidden_states[nonempty],
+            lengths[nonempty].detach().cpu(),
+            batch_first=True,
+            enforce_sorted=False,
+        )
+        packed_output, _ = self.encoder(packed)
+        encoded, _ = pad_packed_sequence(
+            packed_output,
+            batch_first=True,
+            total_length=sequence_length,
+        )
+        output[nonempty] = encoded
+        return output
+
+
+ContextEncoderConfigLike = Optional[Union[dict, PretrainedConfig]]
+
+
+def build_context_encoder(
+    encoder_config: ContextEncoderConfigLike,
+    *,
+    input_size: int,
+    output_size: Optional[int] = None,
+    dropout: float = 0.0,
+) -> BaseContextEncoder:
+    """Build a context encoder with a stable dense-input/dense-output contract."""
+    if output_size is None:
+        output_size = input_size
+    if encoder_config is None:
+        if input_size != output_size:
+            raise ValueError("A disabled context encoder cannot change hidden size")
+        return IdentityContextEncoder(input_size)
+
+    config = normalize_context_encoder_config(
+        encoder_config,
+        hidden_size=input_size,
+        dropout=dropout,
+    )
+    if isinstance(config, DebertaV2Config):
+        return DebertaV2ContextEncoder(config, input_size, output_size)
+    if isinstance(config, ModernBertConfig):
+        return ModernBertContextEncoder(config, input_size, output_size)
+    if isinstance(config, RNNEncoderConfig):
+        return RNNContextEncoder(config, input_size, output_size)
+    raise TypeError(f"Unsupported context encoder config: {type(config).__name__}")
+
+
+__all__ = [
+    "BaseContextEncoder",
+    "DebertaV2ContextEncoder",
+    "IdentityContextEncoder",
+    "ModernBertContextEncoder",
+    "RNNContextEncoder",
+    "build_context_encoder",
+]
diff --git a/gliner/modeling/decoder.py b/gliner/modeling/decoder.py
index 1bd86b21..5d2daf4c 100644
--- a/gliner/modeling/decoder.py
+++ b/gliner/modeling/decoder.py
@@ -6,12 +6,12 @@
 """
 
 import warnings
-from typing import Any, List, Union, Optional
+from typing import Any, List, Tuple, Union, Optional
 from pathlib import Path
 
 import torch
 from torch import nn
-from transformers import AutoConfig, LogitsProcessor, LogitsProcessorList, AutoModelForCausalLM
+from transformers import AutoModel, AutoConfig, LogitsProcessor, LogitsProcessorList, AutoModelForCausalLM
 
 from ..utils import is_module_available
 from ..decoding.trie import LabelsTrie
@@ -75,36 +75,67 @@ class DecoderTransformer(nn.Module):
     """
 
     def __init__(
-        self, model_name: str, config: Any, from_pretrained: bool = False, cache_dir: Optional[Union[str, Path]] = None
+        self,
+        model_name: str,
+        config: Any,
+        from_pretrained: bool = False,
+        cache_dir: Optional[Union[str, Path]] = None,
+        use_causal_lm: bool = True,
     ) -> None:
         """Initializes the decoder transformer.
 
         Args:
             model_name: Name or path of the pretrained model to load.
-            config: Configuration object containing model hyperparameters. Must have
-                a `labels_decoder_config` attribute.
+            config: Configuration object containing model hyperparameters. StreamingSpan
+                models use ``decoder_config``; auxiliary generative decoders retain
+                the legacy ``labels_decoder_config`` field.
             from_pretrained: If True, loads pretrained weights. If False, initializes
                 from config only. Defaults to False.
             cache_dir: Optional directory for caching downloaded models. Defaults to None.
+            use_causal_lm: Use ``AutoModelForCausalLM`` and return vocabulary logits.
+                When False, use ``AutoModel`` and return backbone hidden states.
 
         Raises:
             Warning: If adapter config is found but PEFT package is not installed.
         """
         super().__init__()
-        decoder_config = config.labels_decoder_config
+        decoder_config = getattr(config, "decoder_config", None)
+        if decoder_config is None:
+            decoder_config = getattr(config, "labels_decoder_config", None)
         if decoder_config is None:
             decoder_config = AutoConfig.from_pretrained(model_name, cache_dir=cache_dir)
 
-        kwargs = {}
-        custom = False
-        ModelClass = AutoModelForCausalLM
+        # Decoder backbones are constructed separately from encoder backbones,
+        # so explicitly propagate GLiNER's top-level attention override.
+        attn_implementation = getattr(config, "_attn_implementation", None)
+        if attn_implementation is not None:
+            decoder_config._attn_implementation = attn_implementation
+
+        is_streaming_span = getattr(config, "model_type", None) == "gliner_streaming_span"
+        saved_vocab_size = getattr(config, "vocab_size", -1)
+        if is_streaming_span and not from_pretrained and saved_vocab_size > 0:
+            # StreamingSpan adds <<LABEL>> and <<SEP>> to the backbone tokenizer.
+            # The top-level size is saved with the trained embedding matrix and
+            # must win over a stale/base decoder config during reconstruction.
+            decoder_config.vocab_size = saved_vocab_size
+
+        model_class = AutoModelForCausalLM if use_causal_lm else AutoModel
 
         if from_pretrained:
-            self.model = ModelClass.from_pretrained(model_name, trust_remote_code=True)
-        elif not custom:
-            self.model = ModelClass.from_config(decoder_config, trust_remote_code=True)
+            model_kwargs = {
+                "cache_dir": cache_dir,
+                "trust_remote_code": True,
+            }
+            if attn_implementation is not None:
+                model_kwargs["attn_implementation"] = attn_implementation
+            self.model = model_class.from_pretrained(model_name, **model_kwargs)
         else:
-            self.model = ModelClass(decoder_config, **kwargs)
+            self.model = model_class.from_config(decoder_config, trust_remote_code=True)
+
+        if is_streaming_span:
+            # Keep the resolved Hugging Face config attached to GLiNER so later
+            # token resizing is serialized into gliner_config.json.
+            config.decoder_config = self.model.config
 
         adapter_config_file = Path(model_name) / "adapter_config.json"
 
@@ -119,8 +150,9 @@ def __init__(
                 self.model = get_peft_model(self.model, adapter_config)
 
         self.config = config
+        self.use_causal_lm = use_causal_lm
 
-    def forward(self, *args: Any, **kwargs: Any) -> torch.Tensor:
+    def forward(self, *args: Any, **kwargs: Any) -> Tuple[torch.Tensor, Optional[Any]]:
         """Forward pass through the decoder model.
 
         Args:
@@ -128,11 +160,14 @@ def forward(self, *args: Any, **kwargs: Any) -> torch.Tensor:
             **kwargs: Variable keyword arguments passed to the model.
 
         Returns:
-            Logits tensor of shape (batch_size, seq_len, vocab_size).
+            A tuple containing model representations and optional KV cache. The
+            representations are vocabulary logits in causal-LM mode and backbone
+            hidden states otherwise.
         """
         output = self.model(*args, **kwargs)
         encoder_layer = output[0]
-        return encoder_layer
+        past_key_values = output.past_key_values if hasattr(output, "past_key_values") else None
+        return encoder_layer, past_key_values
 
 
 class Decoder(nn.Module):
@@ -149,22 +184,37 @@ class Decoder(nn.Module):
     """
 
     def __init__(
-        self, config: Any, from_pretrained: bool = False, cache_dir: Optional[Union[str, Path]] = None
+        self,
+        config: Any,
+        from_pretrained: bool = False,
+        cache_dir: Optional[Union[str, Path]] = None,
+        use_causal_lm: bool = True,
     ) -> None:
         """Initializes the decoder.
 
         Args:
-            config: Configuration object containing model hyperparameters including
-                `labels_decoder` (model name) and decoder-specific settings.
+            config: Configuration object containing model hyperparameters. StreamingSpan
+                uses ``model_name`` for its backbone; legacy generative architectures
+                use ``labels_decoder`` for their separate decoder.
             from_pretrained: If True, loads pretrained weights for the decoder.
                 Defaults to False.
             cache_dir: Optional directory for caching downloaded models. Defaults to None.
+            use_causal_lm: Use a causal language-model head when True, or return
+                backbone hidden states through ``AutoModel`` when False.
         """
         super().__init__()
 
-        self.decoder_layer = DecoderTransformer(config.labels_decoder, config, from_pretrained, cache_dir=cache_dir)
+        model_name = getattr(config, "labels_decoder", None) or config.model_name
+        self.decoder_layer = DecoderTransformer(
+            model_name,
+            config,
+            from_pretrained,
+            cache_dir=cache_dir,
+            use_causal_lm=use_causal_lm,
+        )
 
         self.decoder_hidden_size = self.decoder_layer.model.config.hidden_size
+        self.use_causal_lm = use_causal_lm
 
     def ids_to_embeds(self, input_ids: torch.LongTensor) -> torch.FloatTensor:
         """Converts token IDs to their corresponding embeddings.
@@ -424,7 +474,7 @@ def generate(self, *args: Any, **kwargs: Any) -> torch.LongTensor:
         else:
             return self.decoder_layer.model.generate(*args, **kwargs)
 
-    def forward(self, *args: Any, **kwargs: Any) -> torch.Tensor:
+    def forward(self, *args: Any, **kwargs: Any) -> Tuple[torch.Tensor, Optional[Any]]:
         """Forward pass through the decoder.
 
         Computes logits for the input sequence without generation.
@@ -434,7 +484,9 @@ def forward(self, *args: Any, **kwargs: Any) -> torch.Tensor:
             **kwargs: Variable keyword arguments passed to the decoder layer.
 
         Returns:
-            Logits tensor of shape (batch_size, seq_len, vocab_size).
+            A tuple containing model representations and optional KV cache. The
+            representations are vocabulary logits in causal-LM mode and backbone
+            hidden states otherwise.
         """
-        decoded_embeddings = self.decoder_layer(*args, **kwargs)
-        return decoded_embeddings
+        decoded_embeddings, past_key_values = self.decoder_layer(*args, **kwargs)
+        return decoded_embeddings, past_key_values
diff --git a/gliner/modeling/layers.py b/gliner/modeling/layers.py
index 095607e3..6e44c505 100644
--- a/gliner/modeling/layers.py
+++ b/gliner/modeling/layers.py
@@ -5,6 +5,9 @@
 from torch import nn
 from torch.nn.utils.rnn import pad_packed_sequence, pack_padded_sequence
 
+from .utils import extract_prompt_features
+from .context_encoders import build_context_encoder
+
 
 class LstmSeq2SeqEncoder(nn.Module):
     """Bidirectional LSTM encoder for sequence-to-sequence models.
@@ -482,3 +485,93 @@ def forward(self, encoder_outputs: List[torch.Tensor]) -> torch.Tensor:
         output = self.output_projection(U_sum)  # [B, L, output_size]
 
         return output
+
+
+class StreamingSpanLabelsEncoder(nn.Module):
+    def __init__(self, config, **kwargs):
+        super().__init__()
+        self.config = config
+        self.encoder_config = config.labels_encoder_config
+        self.encoder = build_context_encoder(
+            self.encoder_config,
+            input_size=config.hidden_size,
+            output_size=config.hidden_size,
+            dropout=config.dropout,
+        )
+
+    def _slice_prompt(self, hidden_states, input_ids, attention_mask):
+        """Compact each prompt from its first valid token through its first separator."""
+        sep_token_id = getattr(self.config, "sep_token_index", -1)
+        if sep_token_id < 0:
+            raise ValueError("StreamingSpanLabelsEncoder requires config.sep_token_index to be set")
+
+        valid_mask = attention_mask.gt(0)
+        sep_mask = input_ids.eq(sep_token_id) & valid_mask
+        has_separator = sep_mask.any(dim=1)
+        if not has_separator.all():
+            missing = (~has_separator).nonzero(as_tuple=True)[0].tolist()
+            raise ValueError(f"No separator token was found for batch rows {missing}")
+
+        batch_size, seq_length, hidden_size = hidden_states.shape
+        positions = torch.arange(seq_length, device=input_ids.device).expand(batch_size, -1)
+        first_valid = positions.masked_fill(~valid_mask, seq_length).amin(dim=1)
+        first_separator = positions.masked_fill(~sep_mask, seq_length).amin(dim=1)
+        prompt_lengths = first_separator - first_valid + 1
+        if (prompt_lengths <= 0).any():
+            invalid = prompt_lengths.le(0).nonzero(as_tuple=True)[0].tolist()
+            raise ValueError(f"Separator token precedes the prompt for batch rows {invalid}")
+
+        max_prompt_length = int(prompt_lengths.max().item())
+        prompt_offsets = torch.arange(max_prompt_length, device=input_ids.device).unsqueeze(0)
+        source_positions = first_valid.unsqueeze(1) + prompt_offsets
+        within_prompt = prompt_offsets < prompt_lengths.unsqueeze(1)
+        safe_positions = source_positions.clamp_max(seq_length - 1)
+
+        prompt_hidden_states = torch.gather(
+            hidden_states,
+            1,
+            safe_positions.unsqueeze(-1).expand(-1, -1, hidden_size),
+        )
+        prompt_input_ids = torch.gather(input_ids, 1, safe_positions)
+        gathered_attention_mask = torch.gather(attention_mask, 1, safe_positions)
+        prompt_attention_mask = gathered_attention_mask.masked_fill(~within_prompt, 0)
+        valid_prompt_mask = within_prompt & prompt_attention_mask.gt(0)
+        prompt_hidden_states = prompt_hidden_states * valid_prompt_mask.unsqueeze(-1)
+        prompt_input_ids = prompt_input_ids.masked_fill(~valid_prompt_mask, 0)
+        return prompt_hidden_states, prompt_input_ids, prompt_attention_mask
+
+    def forward(self, hidden_states, input_ids, attention_mask, **kwargs):
+        """Forward pass.
+
+        Args:
+            hidden_states: (batch_size, seq_length, hidden_size) from decoder backbone
+            input_ids: (batch_size, seq_length)
+            attention_mask: (batch_size, seq_length)
+            **kwargs: Reserved for interface compatibility.
+
+        Returns:
+            Tuple containing:
+                - label representations of shape (batch_size, num_labels, hidden_size)
+                - label mask of shape (batch_size, num_labels)
+        """
+        prompt_hidden_states, prompt_input_ids, prompt_attention_mask = self._slice_prompt(
+            hidden_states,
+            input_ids,
+            attention_mask,
+        )
+        contextualized_hidden_states = self.encoder(
+            prompt_hidden_states,
+            prompt_attention_mask,
+        )
+        if self.config.class_token_index < 0:
+            raise ValueError("StreamingSpanLabelsEncoder requires config.class_token_index to be set")
+        batch_size, _, hidden_size = contextualized_hidden_states.shape
+        return extract_prompt_features(
+            self.config.class_token_index,
+            contextualized_hidden_states,
+            prompt_input_ids,
+            prompt_attention_mask,
+            batch_size,
+            hidden_size,
+            embed_ent_token=True,
+        )
diff --git a/gliner/modeling/outputs.py b/gliner/modeling/outputs.py
index 693b2181..35f20e5b 100644
--- a/gliner/modeling/outputs.py
+++ b/gliner/modeling/outputs.py
@@ -1,4 +1,4 @@
-from typing import Optional
+from typing import Any, Optional
 from dataclasses import dataclass
 
 import torch
@@ -40,6 +40,30 @@ class GLiNERBaseOutput(ModelOutput):
     span_logits: Optional[torch.FloatTensor] = None
 
 
+@dataclass
+class GLiNERRepresentationOutput(ModelOutput):
+    """Intermediate prompt and text representations produced by GLiNER backbones."""
+
+    prompts_embedding: Optional[torch.FloatTensor] = None
+    prompts_embedding_mask: Optional[torch.LongTensor] = None
+    words_embedding: Optional[torch.FloatTensor] = None
+    mask: Optional[torch.LongTensor] = None
+    past_key_values: Optional[Any] = None
+    past_word_embeddings: Optional[torch.FloatTensor] = None
+    past_word_mask: Optional[torch.LongTensor] = None
+
+
+@dataclass
+class GLiNERStreamingSpanOutput(GLiNERBaseOutput):
+    """Span-classification output with reusable decoder and word cache state."""
+
+    past_key_values: Optional[Any] = None
+    past_word_embeddings: Optional[torch.FloatTensor] = None
+    past_word_mask: Optional[torch.LongTensor] = None
+    cached_prompts_embedding: Optional[torch.FloatTensor] = None
+    cached_prompts_mask: Optional[torch.LongTensor] = None
+
+
 @dataclass
 class GLiNERDecoderOutput(GLiNERBaseOutput):
     """Output class for GLiNER models with decoder components.
diff --git a/gliner/modeling/span_rep.py b/gliner/modeling/span_rep.py
index 7d81cae1..cffd9c27 100644
--- a/gliner/modeling/span_rep.py
+++ b/gliner/modeling/span_rep.py
@@ -1,8 +1,11 @@
+from __future__ import annotations
+
 import torch
 import torch.nn.functional as F
 from torch import nn
 
 from .layers import create_projection_layer
+from .context_encoders import IdentityContextEncoder, build_context_encoder
 
 
 class SpanQuery(nn.Module):
@@ -442,7 +445,7 @@ def forward(self, h, span_idx):
         # h of shape [B, L, D]
         # query_seg of shape [D, max_width]
 
-        B, L, D = h.size()
+        B, _, D = h.size()
 
         # project start and end
         start_rep = self.project_start(h)
@@ -458,7 +461,8 @@ def forward(self, h, span_idx):
         cat = self.out_project(cat)
 
         # reshape
-        return cat.view(B, L, self.max_width, D)
+        num_starts = span_idx.size(1) // self.max_width
+        return cat.view(B, num_starts, self.max_width, D)
 
 
 class SpanMarkerV0(nn.Module):
@@ -498,7 +502,7 @@ def forward(self, h: torch.Tensor, span_idx: torch.Tensor) -> torch.Tensor:
         Returns:
             torch.Tensor: Span representations of shape [B, L, max_width, D].
         """
-        B, L, D = h.size()
+        B, _, D = h.size()
 
         start_rep = self.project_start(h)
         end_rep = self.project_end(h)
@@ -508,7 +512,8 @@ def forward(self, h: torch.Tensor, span_idx: torch.Tensor) -> torch.Tensor:
 
         cat = torch.cat([start_span_rep, end_span_rep], dim=-1).relu()
 
-        return self.out_project(cat).view(B, L, self.max_width, D)
+        num_starts = span_idx.size(1) // self.max_width
+        return self.out_project(cat).view(B, num_starts, self.max_width, D)
 
 
 class SpanMarkerV1(nn.Module):
@@ -559,7 +564,7 @@ def forward(self, h: torch.Tensor, span_idx: torch.Tensor) -> torch.Tensor:
         Returns:
             torch.Tensor: Span representations, shape [B, L, max_width, D].
         """
-        B, L, D = h.size()
+        B, _, D = h.size()
 
         # Pre-compute per-token projections
         start_rep = self.project_start(h)  # [B, L, D]
@@ -581,7 +586,55 @@ def forward(self, h: torch.Tensor, span_idx: torch.Tensor) -> torch.Tensor:
         out = self.out_project(span_feat)  # [B, S, D]
 
         # Reshape back to [B, L, max_width, D] (S = L x max_width)
-        return out.view(B, L, self.max_width, D)
+        num_starts = span_idx.size(1) // self.max_width
+        return out.view(B, num_starts, self.max_width, D)
+
+
+class SpanMarkerV2(nn.Module):
+    """Represent spans using their endpoints and the last valid word.
+
+    The final valid word supplies later sequence context to every candidate
+    scored in the current run.  Unlike older marker layers, the output size is
+    derived from ``span_idx`` and therefore supports rolling candidate windows
+    over a longer cached word sequence.
+    """
+
+    def __init__(self, hidden_size: int, max_width: int, dropout: float = 0.4):
+        super().__init__()
+        self.max_width = max_width
+        self.project_start = create_projection_layer(hidden_size, dropout)
+        self.project_end = create_projection_layer(hidden_size, dropout)
+        self.project_context = create_projection_layer(hidden_size, dropout)
+        self.out_project = create_projection_layer(hidden_size * 3, dropout, hidden_size)
+
+    def forward(
+        self,
+        h: torch.Tensor,
+        span_idx: torch.Tensor,
+        word_mask: torch.Tensor | None = None,
+    ) -> torch.Tensor:
+        batch_size, _, hidden_size = h.shape
+        if word_mask is None:
+            word_mask = torch.ones(h.shape[:2], dtype=torch.bool, device=h.device)
+        else:
+            word_mask = word_mask.bool()
+
+        start_span_rep = extract_elements(self.project_start(h), span_idx[..., 0])
+        end_span_rep = extract_elements(self.project_end(h), span_idx[..., 1])
+
+        last_positions = word_mask.long().sum(dim=1).sub(1).clamp_min(0)
+        last_context = extract_elements(self.project_context(h), last_positions.unsqueeze(1)).squeeze(1)
+        has_words = word_mask.any(dim=1)
+        last_context = last_context * has_words.unsqueeze(-1)
+        context_span_rep = last_context.unsqueeze(1).expand_as(start_span_rep)
+
+        span_features = torch.cat(
+            [start_span_rep, end_span_rep, context_span_rep],
+            dim=-1,
+        ).relu()
+        span_rep = self.out_project(span_features)
+        num_starts = span_idx.size(1) // self.max_width
+        return span_rep.view(batch_size, num_starts, self.max_width, hidden_size)
 
 
 class ConvShareV2(nn.Module):
@@ -693,7 +746,14 @@ class SpanRepLayer(nn.Module):
         span_rep_layer (nn.Module): The underlying span representation layer.
     """
 
-    def __init__(self, hidden_size, max_width, span_mode, **kwargs):
+    def __init__(
+        self,
+        hidden_size,
+        max_width,
+        span_mode,
+        context_encoder_config=None,
+        **kwargs,
+    ):
         """Initialize the SpanRepLayer with the specified mode.
 
         Args:
@@ -703,6 +763,7 @@ def __init__(self, hidden_size, max_width, span_mode, **kwargs):
                 - 'marker': SpanMarker
                 - 'markerV0': SpanMarkerV0
                 - 'markerV1': SpanMarkerV1
+                - 'markerV2': SpanMarkerV2
                 - 'query': SpanQuery
                 - 'mlp': SpanMLP
                 - 'cat': SpanCAT
@@ -711,6 +772,8 @@ def __init__(self, hidden_size, max_width, span_mode, **kwargs):
                 - 'conv_mean': SpanConv with mean pooling
                 - 'conv_sum': SpanConv with sum pooling
                 - 'conv_share': ConvShare
+            context_encoder_config: Optional DeBERTa-v2, ModernBERT, or RNN
+                context-encoder configuration.
             **kwargs: Additional arguments passed to the span representation layer.
 
         Raises:
@@ -718,12 +781,21 @@ def __init__(self, hidden_size, max_width, span_mode, **kwargs):
         """
         super().__init__()
 
+        self.context_encoder = build_context_encoder(
+            context_encoder_config,
+            input_size=hidden_size,
+            output_size=hidden_size,
+            dropout=kwargs.get("dropout", 0.0),
+        )
+
         if span_mode == "marker":
             self.span_rep_layer = SpanMarker(hidden_size, max_width, **kwargs)
         elif span_mode == "markerV0":
             self.span_rep_layer = SpanMarkerV0(hidden_size, max_width, **kwargs)
         elif span_mode == "markerV1":
             self.span_rep_layer = SpanMarkerV1(hidden_size, max_width, **kwargs)
+        elif span_mode == "markerV2":
+            self.span_rep_layer = SpanMarkerV2(hidden_size, max_width, **kwargs)
         elif span_mode == "query":
             self.span_rep_layer = SpanQuery(hidden_size, max_width, trainable=True)
         elif span_mode == "mlp":
@@ -745,15 +817,45 @@ def __init__(self, hidden_size, max_width, span_mode, **kwargs):
         else:
             raise ValueError(f"Unknown span mode {span_mode}")
 
-    def forward(self, x, *args):
+    @property
+    def uses_bidirectional_context(self) -> bool:
+        """Whether all historical words are re-contextualized on every call."""
+        return self.context_encoder.requires_full_recompute
+
+    def forward(
+        self,
+        x,
+        span_idx,
+        word_mask=None,
+        return_words=False,
+        compact_words=False,
+    ):
         """Forward pass through the selected span representation layer.
 
         Args:
             x (torch.Tensor): Input tensor, typically of shape [B, L, D].
-            *args: Additional arguments passed to the underlying layer.
+            span_idx (torch.Tensor): Flattened absolute start/end span indices.
+            word_mask (torch.Tensor, optional): Mask for valid word embeddings.
+            return_words (bool): Return contextualized words alongside spans.
+            compact_words (bool): The supplied words are a valid compact prefix,
+                allowing the identity context encoder to return its input view.
 
         Returns:
             torch.Tensor: Span representations, typically of shape
                 [B, L, max_width, D].
         """
-        return self.span_rep_layer(x, *args)
+        if word_mask is None:
+            word_mask = torch.ones(x.shape[:2], dtype=torch.bool, device=x.device)
+        if compact_words and isinstance(self.context_encoder, IdentityContextEncoder):
+            contextualized = x
+        else:
+            contextualized = self.context_encoder(x, word_mask)
+
+        if isinstance(self.span_rep_layer, SpanMarkerV2):
+            span_rep = self.span_rep_layer(contextualized, span_idx, word_mask)
+        else:
+            span_rep = self.span_rep_layer(contextualized, span_idx)
+
+        if return_words:
+            return span_rep, contextualized
+        return span_rep
diff --git a/gliner/modeling/utils.py b/gliner/modeling/utils.py
index ba21ffd2..5bbf5e0a 100644
--- a/gliner/modeling/utils.py
+++ b/gliner/modeling/utils.py
@@ -2,66 +2,173 @@
 
 import torch
 
+from ..config import SUBTOKEN_POOLING_MODES
 
-def extract_word_embeddings(
+
+def extract_first_word_embeddings(
+    token_embeds: torch.Tensor,
+    words_mask: torch.Tensor,
+    batch_size: int,
+    max_text_length: int,
+    embed_dim: int,
+) -> torch.Tensor:
+    """Select the first marked subtoken representation for each word."""
+    words_embedding = token_embeds.new_zeros((batch_size, max_text_length, embed_dim))
+    batch_indices, token_indices = torch.where(words_mask > 0)
+    target_word_indices = words_mask[batch_indices, token_indices] - 1
+    words_embedding[batch_indices, target_word_indices] = token_embeds[batch_indices, token_indices]
+    return words_embedding
+
+
+def extract_last_word_embeddings(
+    token_embeds: torch.Tensor,
+    words_mask: torch.Tensor,
+    batch_size: int,
+    max_text_length: int,
+    embed_dim: int,
+) -> torch.Tensor:
+    """Select the last marked subtoken representation for each word."""
+    words_embedding = token_embeds.new_zeros((batch_size, max_text_length, embed_dim))
+    batch_indices, token_indices = torch.where(words_mask > 0)
+    target_word_indices = words_mask[batch_indices, token_indices] - 1
+    words_embedding[batch_indices, target_word_indices] = token_embeds[batch_indices, token_indices]
+    return words_embedding
+
+
+def extract_mean_word_embeddings(
     token_embeds: torch.Tensor,
     words_mask: torch.Tensor,
     attention_mask: torch.Tensor,
     batch_size: int,
     max_text_length: int,
     embed_dim: int,
-    text_lengths: torch.Tensor,
-) -> Tuple[torch.Tensor, torch.Tensor]:
-    """Extract word-level embeddings from subword token embeddings.
+) -> torch.Tensor:
+    """Mean-pool all attended subtoken representations for each word."""
+    valid_subtokens = (words_mask > 0) & attention_mask.bool() & (words_mask <= max_text_length)
+    batch_indices, token_indices = torch.where(valid_subtokens)
+    flat_word_indices = batch_indices * max_text_length + words_mask[batch_indices, token_indices] - 1
+    selected_embeddings = token_embeds[batch_indices, token_indices]
+    expanded_indices = flat_word_indices.unsqueeze(-1).expand(-1, embed_dim)
+    flat_output_size = batch_size * max_text_length
+
+    words_embedding = token_embeds.new_zeros((flat_output_size, embed_dim))
+    if torch.onnx.is_in_onnx_export():
+        # ONNX does not support mean reduction or include_self=False for
+        # ScatterElements. Export an equivalent sum/count formulation.
+        words_embedding.scatter_reduce_(
+            0,
+            expanded_indices,
+            selected_embeddings,
+            reduce="sum",
+            include_self=True,
+        )
+        word_counts = token_embeds.new_zeros((flat_output_size, 1))
+        word_counts.scatter_reduce_(
+            0,
+            flat_word_indices.unsqueeze(-1),
+            token_embeds.new_ones((flat_word_indices.size(0), 1)),
+            reduce="sum",
+            include_self=True,
+        )
+        words_embedding = words_embedding / word_counts.clamp_min(1)
+    else:
+        words_embedding.scatter_reduce_(
+            0,
+            expanded_indices,
+            selected_embeddings,
+            reduce="mean",
+            include_self=False,
+        )
 
-    Maps subword token embeddings back to word-level embeddings using a word mask
-    that indicates which subword token corresponds to which word. Only the first
-    subword token of each word is typically used for the word representation.
+    return words_embedding.reshape(batch_size, max_text_length, embed_dim)
 
-    This is essential for span-based NER where predictions are made at the word
-    level but the transformer operates on subword tokens.
 
-    Args:
-        token_embeds: Subword token embeddings from transformer.
-            Shape: (batch_size, seq_len, embed_dim)
-        words_mask: Mask mapping subword positions to word indices. Non-zero values
-            indicate the word index (1-indexed). Zero values are special tokens or
-            continuation subwords to ignore.
-            Shape: (batch_size, seq_len)
-        attention_mask: Standard attention mask from tokenizer.
-            Shape: (batch_size, seq_len)
-        batch_size: Size of the batch.
-        max_text_length: Maximum number of words across all examples in batch.
-        embed_dim: Embedding dimension size.
-        text_lengths: Number of words in each example.
-            Shape: (batch_size, 1) or (batch_size,)
+def extract_max_word_embeddings(
+    token_embeds: torch.Tensor,
+    words_mask: torch.Tensor,
+    attention_mask: torch.Tensor,
+    batch_size: int,
+    max_text_length: int,
+    embed_dim: int,
+) -> torch.Tensor:
+    """Element-wise max-pool all attended subtoken representations for each word."""
+    valid_subtokens = (words_mask > 0) & attention_mask.bool() & (words_mask <= max_text_length)
+    batch_indices, token_indices = torch.where(valid_subtokens)
+    flat_word_indices = batch_indices * max_text_length + words_mask[batch_indices, token_indices] - 1
+    selected_embeddings = token_embeds[batch_indices, token_indices]
+    expanded_indices = flat_word_indices.unsqueeze(-1).expand(-1, embed_dim)
+    flat_output_size = batch_size * max_text_length
+
+    if torch.onnx.is_in_onnx_export():
+        # include_self=False is unsupported by the legacy ONNX exporter. Start
+        # from the dtype minimum and explicitly zero absent word positions.
+        words_embedding = token_embeds.new_full((flat_output_size, embed_dim), torch.finfo(token_embeds.dtype).min)
+        words_embedding.scatter_reduce_(
+            0,
+            expanded_indices,
+            selected_embeddings,
+            reduce="amax",
+            include_self=True,
+        )
+        word_counts = token_embeds.new_zeros((flat_output_size, 1))
+        word_counts.scatter_reduce_(
+            0,
+            flat_word_indices.unsqueeze(-1),
+            token_embeds.new_ones((flat_word_indices.size(0), 1)),
+            reduce="sum",
+            include_self=True,
+        )
+        words_embedding = words_embedding.masked_fill(word_counts == 0, 0)
+    else:
+        words_embedding = token_embeds.new_zeros((flat_output_size, embed_dim))
+        words_embedding.scatter_reduce_(
+            0,
+            expanded_indices,
+            selected_embeddings,
+            reduce="amax",
+            include_self=False,
+        )
 
-    Returns:
-        Tuple containing:
-            - words_embedding: Word-level embeddings extracted from token embeddings.
-              Shape: (batch_size, max_text_length, embed_dim)
-            - mask: Boolean mask indicating valid word positions (True) vs padding (False).
-              Shape: (batch_size, max_text_length)
-    """
-    words_embedding = torch.zeros(
-        batch_size, max_text_length, embed_dim, dtype=token_embeds.dtype, device=token_embeds.device
-    )
+    return words_embedding.reshape(batch_size, max_text_length, embed_dim)
 
-    # Find positions where words_mask > 0 (actual word positions)
-    batch_indices, word_idx = torch.where(words_mask > 0)
 
-    # Convert 1-indexed word mask to 0-indexed positions
-    target_word_idx = words_mask[batch_indices, word_idx] - 1
+def extract_word_embeddings(
+    token_embeds: torch.Tensor,
+    words_mask: torch.Tensor,
+    attention_mask: torch.Tensor,
+    batch_size: int,
+    max_text_length: int,
+    embed_dim: int,
+    text_lengths: torch.Tensor,
+    subtoken_pooling: str = "first",
+) -> Tuple[torch.Tensor, torch.Tensor]:
+    """Dispatch to the configured subtoken pooling implementation.
 
-    # Copy token embeddings to word positions
-    words_embedding[batch_indices, target_word_idx] = token_embeds[batch_indices, word_idx]
+    ``first`` and ``last`` expect one marked subtoken per word. ``mean`` and
+    ``max`` expect every subtoken to carry its 1-based word index.
+    """
+    if subtoken_pooling == "first":
+        words_embedding = extract_first_word_embeddings(
+            token_embeds, words_mask, batch_size, max_text_length, embed_dim
+        )
+    elif subtoken_pooling == "last":
+        words_embedding = extract_last_word_embeddings(token_embeds, words_mask, batch_size, max_text_length, embed_dim)
+    elif subtoken_pooling == "mean":
+        words_embedding = extract_mean_word_embeddings(
+            token_embeds, words_mask, attention_mask, batch_size, max_text_length, embed_dim
+        )
+    elif subtoken_pooling == "max":
+        words_embedding = extract_max_word_embeddings(
+            token_embeds, words_mask, attention_mask, batch_size, max_text_length, embed_dim
+        )
+    else:
+        supported = ", ".join(SUBTOKEN_POOLING_MODES)
+        raise ValueError(f"Unknown subtoken pooling strategy {subtoken_pooling!r}. Expected one of: {supported}")
 
-    # Create mask for valid word positions
     aranged_word_idx = torch.arange(max_text_length, dtype=attention_mask.dtype, device=token_embeds.device).expand(
         batch_size, -1
     )
-
-    mask = aranged_word_idx < text_lengths
+    mask = aranged_word_idx < text_lengths.reshape(batch_size, -1)[:, :1]
     return words_embedding, mask
 
 
@@ -112,7 +219,7 @@ def extract_prompt_features(
               Shape: (batch_size, max_num_types)
     """
     # Find all positions with the class token
-    class_token_mask = input_ids == class_token_index
+    class_token_mask = input_ids.eq(class_token_index) & attention_mask.gt(0)
     num_class_tokens = torch.sum(class_token_mask, dim=-1, keepdim=True)
 
     # Maximum number of class tokens across batch
@@ -151,6 +258,7 @@ def extract_prompt_features_and_word_embeddings(
     text_lengths: torch.Tensor,
     words_mask: torch.Tensor,
     embed_ent_token: bool = True,
+    subtoken_pooling: str = "first",
     **kwargs,
 ) -> Tuple[torch.Tensor, torch.Tensor, torch.Tensor, torch.Tensor]:
     """Extract both prompt embeddings and word embeddings in one call.
@@ -177,6 +285,8 @@ def extract_prompt_features_and_word_embeddings(
             Shape: (batch_size, seq_len)
         embed_ent_token: If True, use [ENT] token embedding. If False,
             use the token after [ENT] (the entity type name). Default: True.
+        subtoken_pooling: Reduction applied to subtokens belonging to the same
+            word. One of ``first``, ``last``, ``mean``, or ``max``.
         **kwargs: Additional keyword arguments passed to extract_prompt_features.
 
     Returns:
@@ -200,7 +310,14 @@ def extract_prompt_features_and_word_embeddings(
 
     # Extract word-level embeddings
     words_embedding, mask = extract_word_embeddings(
-        token_embeds, words_mask, attention_mask, batch_size, max_text_length, embed_dim, text_lengths
+        token_embeds,
+        words_mask,
+        attention_mask,
+        batch_size,
+        max_text_length,
+        embed_dim,
+        text_lengths,
+        subtoken_pooling,
     )
 
     return prompts_embedding, prompts_embedding_mask, words_embedding, mask
diff --git a/gliner/streaming.py b/gliner/streaming.py
new file mode 100644
index 00000000..5dd9d40e
--- /dev/null
+++ b/gliner/streaming.py
@@ -0,0 +1,389 @@
+"""Synchronous fixed-batch and asynchronous streaming inference surfaces."""
+
+from __future__ import annotations
+
+import asyncio
+from typing import TYPE_CHECKING, Any
+from functools import partial
+from threading import RLock
+from dataclasses import dataclass
+from collections.abc import Iterable, AsyncIterable
+from concurrent.futures import ThreadPoolExecutor
+
+if TYPE_CHECKING:
+    import torch
+
+    from .model import StreamingSpanGLiNER
+    from .modeling.cache import CacheState
+
+
+@dataclass
+class _PersistentBatchState:
+    """Tensor and semantic state owned by one fixed-order streaming batch."""
+
+    past_key_values: Any
+    attention_mask: torch.Tensor
+    past_word_embeddings: torch.Tensor
+    past_word_mask: torch.Tensor
+    prompts_embedding: torch.Tensor
+    prompts_mask: torch.Tensor
+    cached_lengths: torch.Tensor
+    next_position_ids: torch.Tensor
+    sessions: list[CacheState]
+
+    @property
+    def batch_size(self) -> int:
+        return len(self.sessions)
+
+    @property
+    def physical_length(self) -> int:
+        return self.attention_mask.size(1)
+
+
+class StreamingBatch:
+    """Persistent fixed-order batch for synchronous streaming inference.
+
+    The session-to-row mapping is immutable, allowing the decoder KV cache to
+    remain batched between calls without stacking or splitting historical keys.
+    """
+
+    def __init__(
+        self,
+        model: StreamingSpanGLiNER,
+        session_ids: list[str],
+        labels: list[str],
+    ) -> None:
+        normalized_ids = list(session_ids)
+        if not normalized_ids or not all(isinstance(value, str) and value for value in normalized_ids):
+            raise ValueError("session_ids must be a non-empty list of non-empty strings")
+        if len(set(normalized_ids)) != len(normalized_ids):
+            raise ValueError("session_ids must be unique within a streaming batch")
+        normalized_labels = list(dict.fromkeys(labels))
+        if not normalized_labels:
+            raise ValueError("At least one label is required")
+
+        self.model = model
+        self.session_ids = tuple(normalized_ids)
+        self.labels = tuple(normalized_labels)
+        self._state: _PersistentBatchState | None = None
+        self._closed = False
+        self._lock = RLock()
+
+    @property
+    def closed(self) -> bool:
+        return self._closed
+
+    @property
+    def initialized(self) -> bool:
+        return self._state is not None
+
+    def append(
+        self,
+        texts: list[str],
+        *,
+        threshold: float = 0.5,
+        flat_ner: bool = True,
+        multi_label: bool = False,
+        return_class_probs: bool = False,
+        recompute: bool = False,
+    ) -> list[list[dict[str, Any]]]:
+        """Append one text chunk per fixed session row and return snapshots."""
+        with self._lock:
+            if self._closed:
+                raise RuntimeError("StreamingBatch is closed")
+            chunks = list(texts)
+            if len(chunks) != len(self.session_ids):
+                raise ValueError("texts must contain one chunk per session row")
+            if not all(isinstance(text, str) for text in chunks):
+                raise TypeError("Every streaming chunk must be a string")
+            try:
+                self._state, outputs = self.model._run_persistent_stream_batch(
+                    self._state,
+                    self.session_ids,
+                    self.labels,
+                    chunks,
+                    threshold=threshold,
+                    flat_ner=flat_ner,
+                    multi_label=multi_label,
+                    return_class_probs=return_class_probs,
+                    recompute=recompute,
+                )
+            except BaseException:
+                # Persistent KV layers append in place. Conservatively discard
+                # the group if a forward or commit fails part-way through.
+                self._state = None
+                raise
+            return outputs
+
+    def reset(self) -> None:
+        """Discard the entire persistent cache while retaining row ownership."""
+        with self._lock:
+            if self._closed:
+                raise RuntimeError("StreamingBatch is closed")
+            self._state = None
+
+    def close(self) -> None:
+        """Release the persistent batch cache. Idempotent."""
+        with self._lock:
+            self._state = None
+            self._closed = True
+
+    def __enter__(self) -> StreamingBatch:
+        if self._closed:
+            raise RuntimeError("StreamingBatch is closed")
+        return self
+
+    def __exit__(self, _exc_type, _exc, _traceback) -> None:
+        self.close()
+
+
+@dataclass
+class _AsyncStreamRequest:
+    text: str
+    labels: list[str]
+    session_id: str
+    recompute: bool
+    threshold: float
+    flat_ner: bool
+    multi_label: bool
+    return_class_probs: bool
+    future: asyncio.Future
+
+    def as_model_request(self) -> dict[str, Any]:
+        return {
+            "text": self.text,
+            "labels": self.labels,
+            "session_id": self.session_id,
+            "recompute": self.recompute,
+            "threshold": self.threshold,
+            "flat_ner": self.flat_ner,
+            "multi_label": self.multi_label,
+            "return_class_probs": self.return_class_probs,
+        }
+
+
+class AsyncStreamingEngine:
+    """Dynamically microbatch independently arriving streaming sessions."""
+
+    _STOP = object()
+
+    def __init__(
+        self,
+        model: StreamingSpanGLiNER,
+        *,
+        max_batch_size: int = 32,
+        batch_wait_timeout_ms: float = 2.0,
+        queue_capacity: int = 4096,
+    ) -> None:
+        if max_batch_size < 1:
+            raise ValueError("max_batch_size must be positive")
+        if batch_wait_timeout_ms < 0:
+            raise ValueError("batch_wait_timeout_ms must be non-negative")
+        if queue_capacity < 1:
+            raise ValueError("queue_capacity must be positive")
+
+        self.model = model
+        self.max_batch_size = int(max_batch_size)
+        self.batch_wait_timeout_s = float(batch_wait_timeout_ms) / 1000.0
+        self._queue: asyncio.Queue = asyncio.Queue(maxsize=queue_capacity)
+        self._session_locks: dict[str, asyncio.Lock] = {}
+        self._worker_task: asyncio.Task | None = None
+        self._executor: ThreadPoolExecutor | None = None
+        self._loop: asyncio.AbstractEventLoop | None = None
+        self._closing = False
+        self._closed = False
+
+    async def start(self) -> AsyncStreamingEngine:
+        """Start the scheduler lazily on the current event loop."""
+        if self._closed:
+            raise RuntimeError("AsyncStreamingEngine is closed")
+        loop = asyncio.get_running_loop()
+        if self._loop is not None and self._loop is not loop:
+            raise RuntimeError("AsyncStreamingEngine cannot move between event loops")
+        self._loop = loop
+        if self._worker_task is None:
+            self._executor = ThreadPoolExecutor(
+                max_workers=1,
+                thread_name_prefix="gliner-streaming",
+            )
+            self._worker_task = loop.create_task(self._worker(), name="gliner-streaming-scheduler")
+        return self
+
+    async def append(
+        self,
+        session_id: str,
+        text: str,
+        labels: list[str],
+        *,
+        threshold: float = 0.5,
+        flat_ner: bool = True,
+        multi_label: bool = False,
+        return_class_probs: bool = False,
+        recompute: bool = False,
+    ) -> list[dict[str, Any]]:
+        """Queue one append; concurrent callers are dynamically microbatched."""
+        if not isinstance(session_id, str) or not session_id:
+            raise ValueError("session_id must be a non-empty string")
+        if not isinstance(text, str):
+            raise TypeError("text must be a string")
+        normalized_labels = list(dict.fromkeys(labels))
+        if not normalized_labels:
+            raise ValueError("At least one label is required")
+        if self._closing or self._closed:
+            raise RuntimeError("AsyncStreamingEngine is closing or closed")
+        await self.start()
+
+        # Match the synchronous compatibility API: blank appends do not mutate
+        # session state and do not consume a GPU batch slot.
+        if not text.strip():
+            return []
+
+        lock = self._session_locks.setdefault(session_id, asyncio.Lock())
+        await lock.acquire()
+        loop = asyncio.get_running_loop()
+        future = loop.create_future()
+        request = _AsyncStreamRequest(
+            text=text,
+            labels=normalized_labels,
+            session_id=session_id,
+            recompute=bool(recompute),
+            threshold=float(threshold),
+            flat_ner=bool(flat_ner),
+            multi_label=bool(multi_label),
+            return_class_probs=bool(return_class_probs),
+            future=future,
+        )
+        try:
+            await self._queue.put(request)
+        except BaseException:
+            lock.release()
+            raise
+
+        def release_session_lock(_future) -> None:
+            if lock.locked():
+                lock.release()
+
+        future.add_done_callback(release_session_lock)
+        return await asyncio.shield(future)
+
+    async def stream(
+        self,
+        session_id: str,
+        chunks: AsyncIterable[str] | Iterable[str],
+        labels: list[str],
+        **kwargs,
+    ):
+        """Yield a session snapshot for every chunk from sync or async input."""
+        if isinstance(chunks, AsyncIterable):
+            async for chunk in chunks:
+                yield await self.append(session_id, chunk, labels, **kwargs)
+            return
+        for chunk in chunks:
+            yield await self.append(session_id, chunk, labels, **kwargs)
+
+    async def clear_session(self, session_id: str) -> None:
+        """Wait for prior work on a session, then discard its cache."""
+        if not isinstance(session_id, str) or not session_id:
+            raise ValueError("session_id must be a non-empty string")
+        lock = self._session_locks.setdefault(session_id, asyncio.Lock())
+        async with lock:
+            self.model.clear_session(session_id)
+
+    async def _collect_batch(self, first: _AsyncStreamRequest) -> list[_AsyncStreamRequest]:
+        batch = [first]
+        if self.batch_wait_timeout_s == 0:
+            while len(batch) < self.max_batch_size:
+                try:
+                    item = self._queue.get_nowait()
+                except asyncio.QueueEmpty:
+                    break
+                if item is self._STOP:
+                    self._queue.task_done()
+                    self._closing = True
+                    break
+                batch.append(item)
+            return batch
+
+        loop = asyncio.get_running_loop()
+        deadline = loop.time() + self.batch_wait_timeout_s
+        while len(batch) < self.max_batch_size:
+            remaining = deadline - loop.time()
+            if remaining <= 0:
+                break
+            try:
+                item = await asyncio.wait_for(self._queue.get(), timeout=remaining)
+            except TimeoutError:
+                break
+            if item is self._STOP:
+                self._queue.task_done()
+                self._closing = True
+                break
+            batch.append(item)
+        return batch
+
+    async def _worker(self) -> None:
+        assert self._loop is not None
+        assert self._executor is not None
+        while True:
+            item = await self._queue.get()
+            if item is self._STOP:
+                self._queue.task_done()
+                return
+            batch = await self._collect_batch(item)
+            requests = [request.as_model_request() for request in batch]
+            try:
+                # One model worker owns all mutable session caches. The forward
+                # itself is batched (GPU-parallel); serial dispatch avoids cache
+                # races and competing CUDA launches on a single model replica.
+                job = self._loop.run_in_executor(
+                    self._executor,
+                    partial(
+                        self.model._run_session_items_batched,
+                        requests,
+                        return_exceptions=True,
+                    ),
+                )
+                # Polling keeps the event loop responsive on runtimes where a
+                # PyTorch worker's cross-thread wakeup can be delayed until the
+                # selector's next timer event.
+                while not job.done():
+                    await asyncio.sleep(0.001)
+                outputs = job.result()
+            except BaseException as error:
+                for request in batch:
+                    if not request.future.done():
+                        request.future.set_exception(error)
+            else:
+                for request, output in zip(batch, outputs):
+                    if not request.future.done():
+                        if isinstance(output, Exception):
+                            request.future.set_exception(output)
+                        else:
+                            request.future.set_result(output)
+            finally:
+                for _ in batch:
+                    self._queue.task_done()
+
+    async def close(self) -> None:
+        """Drain queued work and stop the scheduler. Idempotent."""
+        if self._closed:
+            return
+        self._closing = True
+        if self._worker_task is not None:
+            await self._queue.join()
+            await self._queue.put(self._STOP)
+            await self._worker_task
+            self._worker_task = None
+        if self._executor is not None:
+            self._executor.shutdown(wait=True, cancel_futures=False)
+            self._executor = None
+        self._closed = True
+
+    async def __aenter__(self) -> AsyncStreamingEngine:
+        return await self.start()
+
+    async def __aexit__(self, _exc_type, _exc, _traceback) -> None:
+        await self.close()
+
+
+__all__ = ["AsyncStreamingEngine", "StreamingBatch"]
diff --git a/pyproject.toml b/pyproject.toml
index e22ec6ff..0ec987e4 100644
--- a/pyproject.toml
+++ b/pyproject.toml
@@ -36,7 +36,7 @@ maintainers = [
 
 dependencies = [
     "torch>=2.0.0",
-    "transformers>=4.51.3,<5.7.0",
+    "transformers>=4.51.3,<5.14.0",
     "huggingface_hub>=0.21.4",
     "tqdm",
     "onnxruntime",