inif.io

Path-based readers for INIF documents — unified across .inif and .inif.json.

The path-based unified read API in inif.io works on both formats — .inif indexed archives and plain .inif.json files. Pass any supported path; the dispatcher routes to the indexed-archive readers or to the full document loader as appropriate.

For symmetric save / load / to_dict / from_dict see the methods on InifDocument — those are the entry points for the common write / read cycle. The lower-level indexed-archive options (custom compression, partial loads) live in inif.indexed.

Unified read API

iter_samples

Stream :class:Sample objects from path (any supported format).

For .inif archives, samples are inflated lazily in manifest order. For .inif.json files, the full document is parsed once and its samples are then iterated. Either way the iterator yields the same :class:Sample objects.

Source

def iter_samples(path: str | Path) -> Iterator[Sample]

path str | Path: Path to a .inif indexed archive or a .inif.json file.

read_samples

Read a selection of samples from path by id.

Works on both .inif archives (random access) and .inif.json files (the full document is loaded then filtered). Passing a single id as a bare string / int is supported and is equivalent to passing [id]; the return type is always a list.

Source

def read_samples(
    path: str | Path,
    sample_ids: str | int | Iterable[str | int],
) -> list[Sample]

path str | Path: Path to a .inif or .inif.json file.
sample_ids str | int | Iterable[str | int]: A single sample id or an iterable of ids. Ids must be unique within the call.

read_info

Read a header view of path — metadata plus per-sample summaries.

Works on both .inif archives (only the manifest is parsed) and .inif.json files (the full document is loaded but only the summary dicts are materialised). The summary shape matches what the indexed archive stores in its manifest, so callers can rely on the same fields regardless of the file’s on-disk format.

Source

def read_info(path: str | Path) -> DocumentInfo

path str | Path: Path to a .inif or .inif.json file.

DocumentInfo

Header-only view of an INIF document — metadata plus per-sample summaries.

Returned by :func:read_info. Carries the document’s :class:Metadata and a compact summary dict per sample (id, n_tokens, n_texts, n_spans, scores, text_preview, plus optional target / error / input_tokens / output_tokens when present).

The full sequence list and full token streams are NOT included, so this is cheap to inspect even for very large documents.

Source

class DocumentInfo(BaseModel)