Quickstart

Build, save, view, and load an INIF document in five minutes.

A five-minute tour of the most common INIF flow: take some text, tokenize it, look at the result in the browser, then load it back into Python.

1. Build a document from text

from_texts tokenizes one or more strings (or chat message lists) with a HuggingFace tokenizer and returns an InifDocument:

from inif.converters.text import from_texts

doc = from_texts(
    texts=[
        "The capital of France is Paris.",
        "The capital of Italy is Rome.",
        "The capital of Spain is Madrid.",
    ],
    tokenizer="gpt2",
)
print(doc.total_samples)            # 3
print(len(doc.sequences))           # 1 — "The capital of" is shared

The shared "The capital of" prefix is automatically detected and stored once as a Sequence; each sample replaces that run with a single sequence-ref token.

2. Save to disk

INIF supports two on-disk formats. The suffix decides which one is used.

doc.save("capitals.inif.json")     # plain JSON, human-readable
doc.save("capitals.inif")          # indexed compressed archive
TipWhich format should I pick?

.inif.json is the right default while you’re exploring — diffs cleanly, opens in any editor, easy to inspect. Switch to .inif for large documents where you want random access, header-only reads, or streaming writes; see Saving and loading for the full rationale.

3. View in the browser

The HTML viewer is a single self-contained file — no server needed:

inif view capitals.inif.json

Or from Python (Jupyter shows it inline):

doc.show()                                    # Jupyter
doc.save_html("capitals.html")                # standalone HTML

Tokens are rendered with hover tooltips for every extra field, annotations are highlighted with a color legend, and the sidebar lists all samples.

4. Load it back

InifDocument.load is the symmetric counterpart of save:

from inif import InifDocument

doc = InifDocument.load("capitals.inif.json")
sample = doc.get_sample("sample_0")
print(len(sample.tokens))

For the indexed archive, you can also read just the header or pick a single sample without inflating the whole file — see Saving and loading.

Where to go next