inif.converters.inspect_ai

Convert Inspect AI EvalLog objects and .eval files into INIF documents.

Requires the inspect extra (pip install "inif[inspect]"). See the Inspect AI converter guide for what gets preserved (per-token logprobs, generated annotations, scores, generation config, eval / run ids) and how the tokenizer is resolved.

Entry points

from_eval_log

Convert an Inspect AI EvalLog to an InifDocument.

Source 
def from_eval_log(
    eval_log: Any,
    tokenizer: Any = "auto",
    include_messages: bool = True,
    deduplicate: bool = True,
    min_sequence_length: int = 5,
    tag_chat_roles: bool = True,
    tag_generated: bool = True,
    tag_reasoning: bool = True,
    extract_logprobs: bool = True,
) -> InifDocument
eval_log Any

An inspect_ai.log.EvalLog object.

tokenizer Any

One of:

  • "auto" (default) or None — auto-load AutoTokenizer.from_pretrained based on eval_log.eval.model, after stripping any routing prefix (together/, hf/, …). Raises ValueError if the model id is missing or the tokenizer cannot be loaded (closed-source ids like openai/gpt-4 will hit this).
  • a model id string — loaded via AutoTokenizer.from_pretrained; warns if the resolved id disagrees with the source eval’s model id.
  • a tokenizer instance — used as-is; warns if its name_or_path disagrees with the source eval’s model id.

Tokens are a load-bearing invariant of every INIF sample, so there is no way to opt out of tokenization — every option here yields a usable tokenizer or raises.

include_messages bool

Whether to include message-level text segments.

deduplicate bool

Whether to run sequence deduplication.

min_sequence_length int

Minimum length for common sequence detection.

tag_chat_roles bool

Whether to record per-message chat-template roles (system/user/assistant/template) on Sample.annotations with metadata={"source": "message_role"}.

tag_generated bool

Whether to tag the model’s response tokens with "generated". The response is the last assistant message.

tag_reasoning bool

Whether to annotate ContentReasoning blocks pulled from each assistant message. Tokens overlapping any reasoning span get reasoning + assistant; tokens within the span whose id appears in tokenizer.all_special_ids additionally get template. Best-effort: silently skipped on lossy tokenizers or when the reasoning text can’t be located in the rendered chat template.

extract_logprobs bool

Whether to attach per-token logprobs from the eval output to the response tokens (best-effort: requires the tokenizer and the eval-source tokenization to agree on token count).

from_eval_file

Load an Inspect AI eval log file and convert to InifDocument.

Source 
def from_eval_file(path: str, **kwargs: Any) -> InifDocument
path str
**kwargs Any