Chunking API

CDCParams

.. py:class:: CDCParams(window_words=48, min_tokens=180, max_tokens=640, primary_mask_bits=8, backup_mask_bits=6, snap_window_words=24, snap_to_boundaries=False, normalizer_version=’cairn-normalizer-v1’, tokenizer_id=’word-v1’, rolling_hash_id=’rabin64-word-v1’, boundary_rule_version=’tttd-v1’)

module:

steadlith

canonical:

steadlith.chunk.params.CDCParams

Validated settings for Rabin/TTTD content-defined chunking.

CDCChunker

.. py:class:: CDCChunker(params=None, token_counter=None, *, tokenizer_id=None)

module:

steadlith

canonical:

steadlith.chunk.cdc.CDCChunker

Framework-neutral, LangChain-compatible content-defined chunker.

Chunk

.. py:class:: Chunk(page_content, metadata=)

module:

steadlith

canonical:

steadlith.models.Chunk

A text chunk compatible with LangChain’s Document shape.

Steadlith intentionally depends only on the two attributes used by document consumers: page_content and metadata. Keeping this as a native model avoids importing a framework in the chunking core.

Result diagnostics

.. py:class:: ChunkingResult(chunks, stats)

module:

steadlith.models

Chunks plus diagnostics; behaves as a read-only chunk sequence.

.. py:class:: ChunkingStats(chunk_count=0, total_tokens=0, primary_boundaries=0, backup_boundaries=0, hard_cuts=0, snapped_boundaries=0, token_counts=())

module:

steadlith.models

Diagnostics from one deterministic chunking run.