spacr.run_journal

Workflow inputs and outputs

Run History

Inspect saved run status, settings, outputs and logs; a completed run does not establish biological validity.

Open: the application’s Help/tools menus.

Inputs and outputs below include conditional alternatives. The guidance and handoff notes say which route applies.

Inputs

  • Run history and artifacts — Project run records, settings, output paths, artifact provenance, status and logs.

Outputs

  • Run history and artifacts — Project run records, settings, output paths, artifact provenance, status and logs.

API reference.

Module tutorial.

Run journal — reproducibility record for every pipeline invocation.

Every time a spaCR pipeline runs (mask / measure / classify / …), open_run() writes a timestamped folder under ~/.spacr/runs/ containing everything a reviewer needs to reproduce the result:

~/.spacr/runs/2026-07-23_143507_ab12cd34__mask/
    settings.csv          # exact settings dict, Key,Value CSV
    settings.json         # same, JSON (source of truth for machines)
    manifest.json         # spaCR version, git hash, python, packages,
                          # torch / cuda / cellpose, start time,
                          # end time, elapsed, exit status, model hashes
    log.txt               # tail of ~/.spacr/logs/spacr.log for the run
    outputs/              # optional — any pipeline-emitted artifacts
                          # (masks, DBs, CSVs, plots) copied in

Public API:

from spacr.run_journal import open_run

with open_run("mask", settings) as run:
    preprocess_generate_masks(settings)
    run.attach_output(Path("/path/to/mask.tif"))
    run.set_status("success")

The context manager records start / end timestamps and yields the Run object, whose dir attribute is the run folder. When something raises it stamps "status": "failed" (plus the traceback) into manifest.json and re-raises; no marker file is written.

Consumers of the journal:

  • spacr repro <run-folder> — replays the run (see spacr.cli_repro).

  • AI Console → “File as issue” — includes the last run’s manifest when present so bug reports are self-contained.

  • Home screen “Recent runs” list — enumerated from recent_runs() newest first.

Beside the runs, the journal keeps two records for blinded work. start_blinding() writes a blinding key (coded names and a shuffled order) to ~/.spacr/blinding and unblind() logs who opened it and when. lock_analysis() freezes an analysis plan, hashed and timestamped, in ~/.spacr/analysis_locks (settings of one or more pipelines, Gate Editor gating strategies, models and files); every later run of a locked pipeline on its src is checked against it by check_analysis_lock(), every model the run records is checked as it is recorded, and the verdict is written into that run’s manifest.json under analysis_lock, with any difference also listed in provenance_warnings.

Attributes

MANIFEST_SCHEMA_VERSION

Current on-disk reproducibility-manifest schema.

Classes

Run

A single pipeline invocation's on-disk record.

Functions

check_analysis_lock(→ Dict[str, Any])

Check a run's settings against its preregistered analysis lock.

current_run(→ Optional[Run])

Return the Run currently open on this thread, or None.

delete_runs(→ Tuple[int, List[str]])

Delete journalled run folders. Returns (deleted, refused).

diff_runs(→ Dict[str, Any])

Compare two journalled runs and report exactly what changed.

extract_seeds(→ Dict[str, Any])

Capture declared seeds plus already-loaded RNG state fingerprints.

format_run_diff(→ str)

Render diff_runs() output as a readable console report.

hash_file(→ Optional[str])

Return a file's SHA-256 — 16 hex chars, 64 if full — or None.

journal_totals(→ Dict[str, int])

Return aggregate counts across every stored run.

load_run_settings(→ Dict[str, Any])

Read a run's settings.json (falling back to settings.csv).

lock_analysis(, note, gates, models, pipelines, ...)

Freeze an analysis plan before its results are seen.

open_run(→ Iterator[Run])

Open a fresh run journal folder around a pipeline invocation.

recent_runs(→ List[Dict[str, Any]])

Return the limit most-recent runs newest-first.

resolve_run_dir(→ pathlib.Path)

Turn a run reference into a run-folder Path.

runs_root(→ pathlib.Path)

Return ~/.spacr/runs; created on first access.

search_runs(→ List[Dict[str, Any]])

Return searchable, dashboard-ready records for all journalled runs.

start_blinding(→ Dict[str, Any])

Shuffle items under coded names and keep the key away from them.

unblind(→ Dict[str, str])

Open a blinding key, and record who opened it and when.

values_equal(→ bool)

True when a and b mean the same thing.

Module Contents

class spacr.run_journal.Run[source]

A single pipeline invocation’s on-disk record.

Instances are produced by open_run(). Users don’t construct them directly.

Variables:

  • app_key – id of the pipeline app that opened the run.

  • settings – settings dict originally passed to the pipeline.

  • dir – run folder path (~/.spacr/runs/<ts>_<uuid>__<app>).

  • start_ts – unix epoch seconds when the run opened.

  • end_ts – unix epoch seconds when the run closed (set by open_run() on exit).

  • status – "running" / "success" / "failed" / "cancelled". The last is a run the user stopped, which is not a run that broke, and the two want different things done next.

  • model_hashes – dict of {human-name: "filename:sha256-16"}. Populated by callers via record_model().

  • model_files – full SHA-256, size, and path records for models.

  • input_hashes – per-file full SHA-256 input provenance.

  • output_hashes – per-file full SHA-256 output provenance.

  • seeds – declared seeds and runtime RNG-state identifiers.

  • provenance_warnings – non-fatal path/hash failures retained in the manifest instead of being silently discarded.

  • run_warnings – distinct warning lines emitted by the pipeline.

  • environment – host, spaCR, Git, and installed-package versions.

  • stages – consolidated FlowView lifecycle records in execution order.

  • stdout_path – captured standard-output log path, when one is attached.

  • error_traceback – formatted exception traceback for failed or cancelled runs; empty for successful runs.

attach_output(src_path: Any) → pathlib.Path | None[source]

Copy src_path into the run’s outputs/ folder.

Parameters:

src_path – path to a file (or folder) worth preserving for reproducibility.

Returns:

destination path in the run folder, or None on error.

hashing_enabled() → bool[source]

Whether to hash inputs and outputs for this run.

Off unless the settings say otherwise. Hashing every file under every path-valued setting is proportional to the DATA, not to the run: on a plate of raw images it is minutes of reading before the first mask is made, and it happens whether or not anybody will ever compare the digests.

Read from the settings dict, never from QSettings. A from PySide6 import in a pipeline module makes the package unimportable on a cluster (see the architecture notes), so the GUI reads the preference and passes it down as an ordinary setting, and a headless caller sets the same key.

record_input(path: Any, *, setting_key: str = '') → None[source]

Hash a file or directory as an explicit run input.

A no-op unless hashing_enabled() is true, i.e. unless the settings enable hash_inputs — a default run records nothing here. Directories are recorded as one full SHA-256 record per regular file. Unreadable files are reported in provenance_warnings and logs.

Parameters:

  • path – input file or directory.

  • setting_key – optional setting that referred to path.

record_model(name: str, checkpoint_path: Any) → None[source]

Fingerprint checkpoint_path and remember it under name.

Parameters:

  • name – human-readable key under which to record the model.

  • checkpoint_path – model checkpoint file to fingerprint.

Records "<filename>:<digest>" in model_hashes. An unreadable checkpoint is only logged and leaves no entry at all; any other failure is also appended to provenance_warnings, so it reaches manifest.json — model logging must never itself fail a run. When an analysis lock applies to the run, the model is also checked against it (see lock_analysis()), and the run’s verdict is updated.

record_output(path: Any, *, setting_key: str = '') → None[source]

Hash a file or directory as an explicit run output.

Like record_input(), a no-op unless hashing_enabled() is true, i.e. unless the settings enable hash_inputs.

Parameters:

  • path – output file or directory.

  • setting_key – optional setting that referred to path.

record_warning(message: Any) → None[source]

Retain a distinct warning for the run-history dashboard.

Parameters:

message – warning text captured from pipeline stdout/stderr or supplied directly by pipeline code.

set_status(status: str) → None[source]

Explicitly stamp status (success / failed / …).

Parameters:

status – lifecycle state to store on the run.

spacr.run_journal.check_analysis_lock(settings: Dict[str, Any], *, app_key: str, lock: Dict[str, Any] | None = None, gates: Any = None) → Dict[str, Any][source]

Check a run’s settings against its preregistered analysis lock.

Use the supplied analysis plan, or find the newest plan created by lock_analysis() for app_key and the settings’ src. Check the plan against its stored hash to detect changes to the file. Then compare each setting, each hashed file, and each recorded strategy for selecting measurement rows.

Each difference is stamped with when it was first seen (kept beside the lock), and it is post-hoc only when it was first seen after a blinding key on a folder of the plan was opened: an edit made while still blind stays a deviation even when the run that uses it comes later.

Parameters:

  • settings – the settings of the run being checked.

  • app_key – the pipeline running them.

  • lock – a lock record to check against instead of the newest one.

  • gates – the Gate Editor gating strategies in use, for gates the lock took as objects (see lock_analysis()).

Returns:

status – "unlocked" (no lock applies), "verified" (nothing changed), "deviation" (changed, every change first seen while still blind), "post_hoc" (a change first seen after a key on the plan’s folders was unblinded), "not_preregistered" (unchanged, but the lock was made after an unblinding) or "tampered" (the lock no longer matches its hash) – with lock_id, sha256, locked_utc, locked_by, pipelines, deviations (key, locked, now, first_seen_utc, post_hoc, and detail for gates), unblinded_utc and a one-sentence summary.

spacr.run_journal.current_run() → Run | None[source]

Return the Run currently open on this thread, or None.

Useful for pipeline internals (Cellpose model loaders, etc.) that want to record a model checkpoint hash without every caller needing to plumb the Run object through.

State is thread-local because the batch runner and database browser can execute independent workers concurrently.

spacr.run_journal.delete_runs(directories: Iterable[Any]) → Tuple[int, List[str]][source]

Delete journalled run folders. Returns (deleted, refused).

THIS REMOVES FILES, so every guard below is load-bearing rather than defensive habit:

  • A path is deleted only if, once resolved, it is strictly INSIDE runs_root(). A record’s dir is read from a manifest on disk, and a manifest is a file a user can edit; ../../.. in one must not reach anything. Resolving first is what makes the check real – comparing unresolved strings passes for a symlink that points anywhere.

  • runs_root() ITSELF is refused. “Delete everything” is a loop over children, never a removal of the root, so a caller that computes an empty selection cannot take the journal with it.

  • The run OPEN ON THIS THREAD is refused: deleting the folder a run is still writing into leaves it failing on its next write with an error that names nothing.

  • A path that is not a directory is refused rather than unlinked.

Refusals are RETURNED, not raised. Deleting fifty runs where one is live should delete forty-nine and say which one it kept – an exception at that point has already deleted an unknown number and tells the caller nothing about which.

Parameters:

directories – run folder paths, as in a record’s "dir".

Returns:

how many were removed, and a message per refusal.

spacr.run_journal.diff_runs(run_a: Any, run_b: Any) → Dict[str, Any][source]

Compare two journalled runs and report exactly what changed.

run_a / run_b may each be a run folder (str / Path), a run-id or unambiguous id prefix (resolved under runs_root()), or a Run object — see resolve_run_dir().

Results are bucketed by presence before value, because the settings schema drifts between releases and a flat diff of an old run against a new one is almost entirely schema noise:

{
  "changed":   [{"key": k, "a": av, "b": bv}, …],  # THE signal
  "only_in_a": ["key", …],      # schema drift / dropped options
  "only_in_b": ["key", …],      # schema drift / new options
  "same":      int,             # count only, to keep this small
  "env":       [{"key": k, "a": av, "b": bv}, …],  # from manifests
  "meta":      {"a": {...}, "b": {...}, "app_key_differs": bool},
}

changed holds only keys present in both runs whose values actually differ, sorted by key — those are the knobs someone turned. Values are compared structurally, not by repr (see _normalize_value()): [1, 2] == [1, 2], "[1, 2]" (as CSV round-trips it) == [1, 2], and "None" == None.

env diffs manifest.json’s env snapshot — spaCR version, git hash, python, platform, torch / cellpose / numpy versions — which is usually where an unexplained behaviour change actually lives.

Comparing two runs of different app_key (mask vs measure) is allowed — sometimes that is exactly the question — but flagged via meta["app_key_differs"] so the caller can warn that the two schemas were never meant to line up.

A missing or corrupt settings.json / manifest.json never raises: whatever could be read is diffed and the problem is listed in meta[side]["errors"].

Parameters:

  • run_a – baseline run reference.

  • run_b – comparison run reference.

Returns:

the diff dict described above (JSON-serialisable as long as the settings themselves are).

Raises:

FileNotFoundError – only when a run reference resolves to no run folder at all.

spacr.run_journal.extract_seeds(settings: Dict[str, Any]) → Dict[str, Any][source]

Capture declared seeds plus already-loaded RNG state fingerprints.

The state fingerprints do not import NumPy or Torch. When those libraries are already loaded, their state is captured; otherwise the manifest says so explicitly instead of changing application startup behavior.

Parameters:

settings – settings dict; every nested key whose name contains seed, random_state or random_seed is collected.

Returns:

a dict with declared (the collected seed settings), python_hash_seed (PYTHONHASHSEED, or None when unset), python_random_state_sha256, numpy_random_state_sha256 and torch_initial_seed — the last two None when the library is not already imported. A probe that raises contributes numpy_random_state_error / torch_seed_error instead of its value key.

spacr.run_journal.format_run_diff(diff: Dict[str, Any], max_drift_names: int = 6) → str[source]

Render diff_runs() output as a readable console report.

Ordering is deliberate: changed settings first (the signal), then the environment, then schema drift reduced to a one-line summary plus a handful of names. The drifted keys are never dumped in full — on a real cross-release pair that is ~200 lines of noise that buries the six settings the user actually changed.

Parameters:

  • diff – the dict returned by diff_runs().

  • max_drift_names – how many drifted key names to name before collapsing the rest into (+N more).

Returns:

a multi-line report (no trailing newline).

spacr.run_journal.hash_file(path: pathlib.Path, chunk_size: int = 1 << 20, *, full: bool = False) → str | None[source]

Return a file’s SHA-256 — 16 hex chars, 64 if full — or None.

Parameters:

  • path – regular file to hash.

  • chunk_size – bytes read per iteration.

  • full – return all 64 hexadecimal characters. The default preserves the historic 16-character public result; reproducibility manifests use the full digest.

spacr.run_journal.journal_totals() → Dict[str, int][source]

Return aggregate counts across every stored run.

Powers the Home-screen insights dashboard: total_runs (all manifests seen), mask_runs / measure_runs / classify_runs (per-app tallies), and models_recorded (distinct model hashes ever recorded, across runs of every app — not only mask runs). Returns zeros when no journal exists yet.

Counting is incremental: the totals and the names of the folders already counted are persisted in .journal_totals.json in the runs root, so each call parses only the manifests it has not seen. A folder that has since been deleted cannot be subtracted, so that case falls back to a full recount. A manifest that cannot be parsed is left out of the totals with a logged warning.

spacr.run_journal.load_run_settings(run_dir: pathlib.Path) → Dict[str, Any][source]

Read a run’s settings.json (falling back to settings.csv).

Parameters:

run_dir – journal run directory containing the settings files.

spacr.run_journal.lock_analysis(settings: Dict[str, Any], *, app_key: str, hypotheses: str = '', thresholds: Any = None, files: Iterable[Any] = (), note: str = '', gates: Any = None, models: Any = None, pipelines: Dict[str, Dict[str, Any]] | None = None) → Dict[str, Any][source]

Freeze an analysis plan before its results are seen.

The settings, the stated hypotheses and thresholds, the SHA-256 of every model, gate or threshold file the settings name (plus files), the gating strategies and the models are written with the time and the user to ~/.spacr/analysis_locks/<lock id>.json and hashed. From then on every journalled run of a locked pipeline on its src is checked against the newest lock by check_analysis_lock(), and every model a run records is checked as it is recorded; the verdict goes into the run’s manifest.json, and a difference is listed in the manifest’s warnings, the report and the methods text.

A lock made after a blinding key on a folder it covers was unblinded is marked as such, because it was not made blind.

Parameters:

  • settings – the settings the analysis will run with.

  • app_key – the pipeline it will run in, such as "classify".

  • hypotheses – the hypotheses, in words.

  • thresholds – the decision thresholds and gates, in any JSON-compatible form.

  • files – further files the plan depends on.

  • note – anything else worth keeping with the plan.

  • gates – Strategies for selecting measurement rows. Accept a gate set, its dictionary representation, a saved strategy file, a list of these values, or a mapping from labels to these values. Also include strategy files referenced by settings. Check strategy files on every run. To check a strategy supplied as an object, pass it to check_analysis_lock() using the same label.

  • models – {name: checkpoint path} for the models the analysis loads, under the names the pipeline records them by.

  • pipelines – further pipelines of the same plan, as {app_key: settings}; each is checked against its own settings on its own src, and one unblinding on any of their folders counts for the whole plan.

Returns:

the stored lock, including lock_id, locked_utc and sha256.

Raises:

  • ValueError – when pipelines repeats app_key, or gates holds something that is not a gating strategy.

  • FileNotFoundError – when a model in models is not a file.

spacr.run_journal.open_run(app_key: str, settings: Dict[str, Any]) → Iterator[Run][source]

Open a fresh run journal folder around a pipeline invocation.

Example:

from spacr.run_journal import open_run

with open_run("mask", settings) as run:
    run.record_model("cellpose_cyto", ckpt_path)
    preprocess_generate_masks(settings)
    run.set_status("success")
Parameters:

  • app_key – pipeline id ("mask", "measure", …).

  • settings – settings dict handed to the pipeline. Written to the run folder as both JSON and CSV.

Yields:

the Run object.

spacr.run_journal.recent_runs(limit: int = 10) → List[Dict[str, Any]][source]

Return the limit most-recent runs newest-first.

Ordered by the manifest’s start_utc timestamp (parsed as datetime.datetime), so runs opened in the same wall- clock second still sort correctly — folder names alone truncate to seconds and would produce ties. Only the newest max(limit * 4, limit + 64) folders by name are opened at all (for non-negative limit), so startup cost does not grow with the size of the journal.

PASS None FOR “EVERY RUN”, never a negative number. The truncation at the end is all_entries[:limit], so limit=-1 reads the whole journal and then hands back all but the OLDEST entry – observed 2026-09-03 on an 11,027-run journal, which returned 11,026. A folder with no manifest.json is skipped quietly; one whose manifest cannot be parsed is skipped with a logged warning.

Each entry is a dict with keys dir (Path), app_key (str), status (str), start_utc (ISO str), elapsed_s (float), and the raw manifest (dict, best-effort).

spacr.run_journal.resolve_run_dir(ref: Any) → pathlib.Path[source]

Turn a run reference into a run-folder Path.

Parameters:

ref – run object, directory path, run id, or unambiguous id prefix.

Accepts, in order of preference:

  • a Run object (uses its dir),

  • a path (str / Path) to an existing run folder,

  • a run-id — the folder basename, e.g. "2026-07-23_214737_b66bae6b__mask" — resolved under runs_root(),

  • an unambiguous prefix of a run-id ("2026-07-23_2147"), handy from a shell.

Raises:

FileNotFoundError – when nothing matches, or when a prefix matches more than one run.

spacr.run_journal.runs_root() → pathlib.Path[source]

Return ~/.spacr/runs; created on first access.

spacr.run_journal.search_runs(query: str = '', *, app_key: str = '', status: str = '', limit: int | None = None) → List[Dict[str, Any]][source]

Return searchable, dashboard-ready records for all journalled runs.

Search covers the run id, module, status, settings keys and values, input and output paths, warnings, failure traceback, and environment versions. Corrupt or interrupted run folders remain visible with an explicit "corrupt"/"running" status and diagnostic warnings.

Parameters:

  • query – whitespace-separated case-insensitive terms; every term must occur somewhere in the record.

  • app_key – optional exact module filter.

  • status – optional exact status filter.

  • limit – maximum returned records after newest-first sorting.

Returns:

JSON-friendly record dictionaries. dir remains a Path for convenient GUI use.

spacr.run_journal.start_blinding(items: Iterable[Any], *, scope: str, src: Any = '', seed: int | None = None) → Dict[str, Any][source]

Shuffle items under coded names and keep the key away from them.

Blind scoring: the person scoring sees each item only by its code (B0001, B0002 and so on, numbered in the shuffled order) and in that order, so neither a name nor its neighbours say which plate, well or condition it came from. The key that maps codes back to items is written to ~/.spacr/blinding/<key id>.json, outside the data folder, and every later event on it (unblinding, closing) is appended to <key id>.log.jsonl with who did it and when.

Parameters:

  • items – the things being scored, such as crop paths or image files; repeats are kept once, in first-seen order.

  • scope – what is being scored, such as "annotate"; stored with the key.

  • src – the experiment folder the items belong to. Analysis locks on the same folder read this key’s unblinding record.

  • seed – the shuffle’s seed; a random one is drawn and stored when omitted, so the order can be rebuilt from the key.

Returns:

key_id, order (the items, shuffled) and codes ({item: code}).

spacr.run_journal.unblind(key_id: str, *, reason: str = '') → Dict[str, str][source]

Open a blinding key, and record who opened it and when.

The record is appended to the key’s log before the key is returned, so the identities cannot be read through this call without leaving the record. An analysis lock on the same folder treats a difference found after this moment as post-hoc.

Parameters:

  • key_id – the key’s id, as start_blinding() returned it.

  • reason – why it was opened; stored with the record.

Returns:

{code: item}.

Raises:

FileNotFoundError – when there is no such key.

spacr.run_journal.values_equal(a: Any, b: Any) → bool[source]

True when a and b mean the same thing.

Parameters:

  • a – first settings value to compare after normalisation.

  • b – second settings value to compare after normalisation.

Compares _normalize_value() output structurally, falling back to a repr comparison for exotic values whose __eq__ refuses to produce a bool (numpy-style elementwise comparison, etc.) — and to “not equal” if even that blows up. A settings comparison must never be the thing that raises.

spacr.run_journal.MANIFEST_SCHEMA_VERSION = 3[source]

Current on-disk reproducibility-manifest schema.

Nested helpers

_desktop_os_notify.quoted(text: str) → str

text as an AppleScript string literal on one line.

spacr/run_journal.py:1787

_dispatch_notification.work() → None

Send, and keep what happened.

spacr/run_journal.py:1882

_iter_files.excluded(candidate: Path) → bool

Return whether candidate resolves to or below an excluded root.

A path that cannot be resolved is retained so one hostile entry does not abort or silently empty the rest of the file walk.

spacr/run_journal.py:652

_send_notification.secret(name: str) → str

The secret supplied to try, else the stored one.

spacr/run_journal.py:1831

recent_runs._sort_key(e)

Sort a recent-run entry by parsed start time, then folder mtime.

spacr/run_journal.py:2023

search_runs._history_sort_key(record: Dict[str, Any]) → Tuple[datetime, float]

Return a UTC-aware start and resilient directory-mtime tiebreaker.

spacr/run_journal.py:2175