spacr.chaining¶
Auto-chaining: a module’s inputs default to where the last run actually wrote.
Opening Measure has always meant typing the plate folder again, and getting it
wrong meant a twenty-minute run against the previous plate. The path was
never a mystery — Mask had just written it — but nothing carried the answer
across, so every module screen started from "path".
spacr.ports now declares what each module consumes and produces, and
spacr.artifacts records where every finished run put its outputs. This
module joins the two:
chained_inputs()asks the registry —latest(kind, project=…)— where the upstream module’s output is, and turns that into the value a settings key should hold. The answer comes from the row the producer wrote, never from re-deriving<root>/mergedand hoping;resolve_settings()applies those values to a settings dict without ever overwriting a path the user edited. A user edit is remembered in aPinStoreand wins forever after; when the upstream later moves, the new location is offered (HeldPin) rather than pushed;staleness_notes()turnsspacr.artifacts.Registry.is_stale()into sentences a user can act on, keyed by the cause codes so the reason is specific — “Mask ran again after this” is a different problem from “the settings changed”;next_steps()answers “this run finished — now what?”, usingspacr.ports.next_modules()for the candidates,chained_inputs()for their pre-filled settings andspacr.ports.check_ready()so a successor that cannot run is offered with its blocking reason rather than silently.
Nothing here imports Qt, numpy or torch: the Qt layer
(spacr.qt.chaining) is a thin skin over these functions, and the same
answers are available to the CLI and to a batch runner.
Public API¶
PinStore,pin_store,state_pathThe memory of which paths the user edited by hand.
Binding,BINDINGS,register_binding,binding_forWhich settings key an input port fills, and in what form.
ChainedInput,chained_inputs,resolve_settings,ResolutionAuto-chaining itself.
StaleNote,stale_inputs,stale_outputs,staleness_notesStaleness, said out loud.
NextStep,next_steps“Continue to the next step.”
Classes¶
How one consumed port becomes one settings value. |
|
One input a module can take from a run that already happened. |
|
A question a drop cannot answer on its own. |
|
What a dropped path means to one screen. |
|
One settings key a drop can fill, and what it resolved to. |
|
A settings key auto-chaining did not touch, because the user owns it. |
|
A module that can run on what the finished run just produced. |
|
Persist explicitly selected setting paths across application sessions. |
|
What auto-chaining did to a settings dict. |
|
One out-of-date artifact, with the reason and the fix. |
Functions¶
|
Return the binding for one consumed port, declared or derived. |
|
Return the project roots to search for |
|
Return where |
|
Return every SQLite database in a project, the declared one first. |
|
Render staleness cause codes as one readable clause. |
|
True when a settings value holds no real path. |
|
Return the folder names spaCR's project layout uses, sorted. |
|
True when |
|
Return what can run next, pre-filled with what this run just produced. |
|
Return the process-wide |
|
Return the values that mean "nothing chosen yet". |
|
Return the canonical declaration of where each kind lives. |
|
Return the project root a dropped path belongs to. |
|
Declare that one port fills one settings key. |
|
Work out what a dropped path means to |
|
Fill |
|
Return the result tables a project has written, sorted. |
|
Compare two settings values as paths, list-insensitively. |
|
True when |
|
Return the settings key naming |
|
Return the inputs |
|
Return the results |
|
Return every stale artifact around |
|
Return the file the user's pinned paths live in. |
Module Contents¶
- class spacr.chaining.Binding[source]¶
How one consumed port becomes one settings value.
- Parameters:
module – the consuming module key.
role – the port’s role within that module, e.g.
"merged".setting – the settings key it fills, e.g.
"src".form –
ROOT(the project the artifact belongs to) orPATH(the artifact itself).
- class spacr.chaining.ChainedInput[source]¶
One input a module can take from a run that already happened.
- Parameters:
module – the consuming module key.
setting – the settings key this fills.
role – the consumed port’s role.
kind – the
spacr.portskind, e.g."merged-arrays".value – what the settings key should become.
artifact – the registered upstream output it came from.
producer – the module that wrote it.
root – the project root the artifact belongs to.
staleness – the artifact’s own staleness, so a chained default can say “yes, and it is out of date” instead of quietly handing over a stale folder.
required – whether the consuming module needs this port.
- class spacr.chaining.DropChoice[source]¶
A question a drop cannot answer on its own.
Two databases in a folder, two projects under the folder that was dropped, two tables in the database — each has a right answer and none of them is “the first one”.
optionsis what to offer.- Parameters:
question – the sentence to put above the list.
kind – the vocabulary term the options are candidates for.
options – the candidates, in the order to offer them.
setting – the settings key the answer fills, when there is one.
- class spacr.chaining.DropResolution[source]¶
What a dropped path means to one screen.
- Parameters:
module – the screen key.
dropped – the path the user dropped, absolute.
root – the project root it resolved to.
targets – the inputs that were found.
choices – the questions that have to be asked first.
problems –
spacr.validate.Problemfor every input that is missing — the same sentencesspacr.ports.check_ready()writes, because they come from it.
- target_for(kind: str) DropTarget | None[source]¶
Return the resolved target of
kind, or None.- Parameters:
kind – port or artifact vocabulary kind to find.
- class spacr.chaining.DropTarget[source]¶
One settings key a drop can fill, and what it resolved to.
- Parameters:
module – the screen the drop landed on.
setting – the settings key it fills.
role – the port’s role.
kind – the
spacr.portskind.value – what the settings key should become — the project root for a
ROOTbinding, the artifact itself for aPATHone.location – the artifact’s own path, always. This is what an interface shows the user: “it resolved to this”.
source –
FROM_REGISTRYwhen the answer came from a recorded run — the same answer auto-chaining gives — orFROM_LAYOUTwhen it came from the declared folder layout.required – whether the screen needs this input.
paths – the individual files the port’s pattern matched, for a screen that wants one file rather than the folder holding them.
- class spacr.chaining.HeldPin[source]¶
A settings key auto-chaining did not touch, because the user owns it.
- Parameters:
setting – the settings key.
value – the value the user chose, which is what the key holds.
offered – the value auto-chaining would have used, or None when the registry has nothing to offer.
chained – the
ChainedInputbehindoffered.
- class spacr.chaining.NextStep[source]¶
A module that can run on what the finished run just produced.
- Parameters:
module – the successor’s module key.
source – the module that just finished.
root – the project it would run in.
kinds – the kinds it picks up from the finished run.
seed – settings to pre-fill its screen with — the artifact that was just produced, resolved through the registry like any other chained default. Paths are scalars even for a successor whose key holds a list: the receiving end normalises (the chip editor’s
set_valueandspacr.utils.normalize_src_path()both wrap a bare path), and guessing the container here would mean keeping a second copy of every module’s default shape in step with the first.readiness –
spacr.ports.check_ready()’s verdict.artifacts – the artifact ids the seed points at.
- class spacr.chaining.PinStore(path: str | None = None)[source]¶
Persist explicitly selected setting paths across application sessions.
A pin distinguishes a manually selected value from one populated by automatic chaining. Pinned values survive screen reopening, application restart, and subsequent upstream runs until they are cleared. Pins are stored separately from settings because a settings dictionary does not retain the origin of a value.
Read lazily and written through a temporary file plus
os.replace(), so a crash mid-write cannot leave a half-written JSON file that would lose every pin at once.- Parameters:
path – the JSON file. Defaults to
state_path().
Initialize a lazily loaded store at an expanded absolute path.
- clear(module: str = '') None[source]¶
Forget one module’s pins, or all of them.
- Parameters:
module – module key, or
""for every module.
- pin(module: str, setting: str, value: Any) None[source]¶
Store an explicitly selected value for a module setting.
An empty path removes the existing pin and re-enables automatic chaining for that setting. Empty values are not persisted.
- Parameters:
module – module key.
setting – settings key, e.g.
"src".value – the path (or list of paths) the user entered.
- pinned(module: str, setting: str) Any[source]¶
Return the pinned value, or None when the user never set one.
- Parameters:
module – module key.
setting – settings key.
- pins(module: str) Dict[str, Any][source]¶
Return every pin for one module, as a copy.
- Parameters:
module – module key.
- class spacr.chaining.Resolution[source]¶
What auto-chaining did to a settings dict.
- Parameters:
module – the module key.
settings – a new dict — the input is never mutated, because a caller that shows a diff needs both sides.
filled – settings key → the chained input that filled it.
held – settings key → the pin that stopped it being filled.
inputs – every chained input found, filling or not.
- class spacr.chaining.StaleNote[source]¶
One out-of-date artifact, with the reason and the fix.
- Parameters:
module – the module whose screen this is being shown on.
direction –
"input"(something this module reads is stale) or"output"(a result this module already produced is stale).kind – the
spacr.portskind.role – the port role.
path – where the artifact is.
producer – the module that wrote it.
artifact_id – the registry id.
causes – the machine cause codes, verbatim from
spacr.artifacts.Staleness.reasons – the registry’s own sentences, kept because they name the specific upstream path that moved.
missing – the artifact’s file is gone, which is an availability problem rather than a provenance one.
- to_problem()[source]¶
Return this note as a
spacr.validate.Problem.So a caller can print staleness through
spacr.validate.format_report()beside the settings pre-flight and the port readiness check, rather than inventing a third format.
- spacr.chaining.binding_for(module: str, port: spacr.ports.Port) Binding[source]¶
Return the binding for one consumed port, declared or derived.
Derived means: the port fills the module’s source folder, in
ROOTform. That is the shipped truth for every pipeline module — Measure, Classify, UMAP and the rest all take a plate folder and findmerged/ormeasurements/inside it — so the common case needs no declaration and a new module joins the chain the moment it declares ports.- Parameters:
module – the consuming module key or alias.
port – one of that module’s consumed ports.
- spacr.chaining.candidate_roots(module: str, settings: Mapping[str, Any] | None = None, *, root: str = '', roots: Sequence[str] = ()) Tuple[str, ...][source]¶
Return the project roots to search for
module’s inputs, in order.The module’s own source folder first — a user who has already named a plate means that plate — then whatever the caller offers. The Qt layer supplies the folders the upstream modules last ran in, which is how opening Measure on a blank screen finds the plate Mask just finished.
- Parameters:
module – module key or alias.
settings – the settings dict being edited.
root – an explicit root, tried first.
roots – further candidates, in preference order.
- Returns:
absolute, de-duplicated, existing-or-not (existence is the registry’s problem, not this function’s).
- spacr.chaining.chained_inputs(module: str, settings: Mapping[str, Any] | None = None, *, root: str = '', roots: Sequence[str] = (), registry: spacr.artifacts.Registry | None = None, check_staleness: bool = True) Tuple[ChainedInput, ...][source]¶
Return where
module’s inputs actually are, one entry per port.For every port
moduleconsumes, the registry is asked for the newest artifact of that kind in each candidate project, in order, and the first hit wins. The answer is the row the producer wrote — its path, its project, its settings hash — so a plate whose merged arrays ended up somewhere unusual chains correctly, which re-deriving<root>/mergednever could.Ports whose settings key the module does not have are skipped: inventing a key would put a value somewhere nothing reads.
- Parameters:
module – module key or alias.
settings – the settings dict being edited. Used for the current values (list-or-string, and “is it already filled?”) and for the module’s own project root.
root – an explicit project root, searched first.
roots – further project roots to search, in preference order.
registry – an open registry to ask instead of each project’s own.
check_staleness – also report whether each input is out of date. Costs one recursive query per input; off for a caller that only wants the paths.
- Returns:
one
ChainedInputper resolved port, in declaration order.- Raises:
spacr.ports.UnknownModule – when
moduledeclares no ports.
- spacr.chaining.db_candidates(root: str) Tuple[str, ...][source]¶
Return every SQLite database in a project, the declared one first.
- Parameters:
root – candidate project root to search.
The declared location comes from the
spacr.ports.MEASUREMENTS_DBport; the rest is a shallow listing of the root and of the folder that port names. Two databases in one project is not an error and not a thing to guess about — it is a question, and this is the list to ask it with.
- spacr.chaining.explain_causes(causes: Iterable[str]) str[source]¶
Render staleness cause codes as one readable clause.
- Parameters:
causes – cause codes from
spacr.artifacts.Staleness.causes.- Returns:
the sentences joined with “; “, de-duplicated in first-seen order. Unknown codes are passed through as themselves rather than dropped — a code this table has not caught up with is still a fact.
- spacr.chaining.is_empty_path(value: Any) bool[source]¶
True when a settings value holds no real path.
- Parameters:
value – the current value of a path-ish settings key.
- spacr.chaining.layout_directories() Tuple[str, ...][source]¶
Return the folder names spaCR’s project layout uses, sorted.
Read off
spacr.ports.PORTSrather than typed out —merged,measurements,masks,data,model,results,settings,orig,consolidatedall come from a declaration somebody already wrote — so a plugin that declares a port makes its own folder part of the layout without editing a list here.Cached against the size of the registry, so a late
spacr.ports.register_module_ports()is picked up.
- spacr.chaining.looks_laid_out(folder: str) bool[source]¶
True when
folderholds any of spaCR’s declared layout folders.- Parameters:
folder – candidate project directory to inspect.
The cheap structural answer to “is this a project?”, nine
statcalls againstlayout_directories().spacr.projects.looks_like_project()is the thorough one and reads the registry and every module’s outputs; a drop happens while the mouse button is still down, so this is the one that runs there.
- spacr.chaining.next_steps(module: str, settings: Mapping[str, Any] | None = None, *, root: str = '', roots: Sequence[str] = (), registry: spacr.artifacts.Registry | None = None, include_blocked: bool = True) Tuple[NextStep, ...][source]¶
Return what can run next, pre-filled with what this run just produced.
Candidates come from
spacr.ports.next_modules()— the modules that require one of the kinds this one produces — so the list is derived from the declared graph rather than a hand-written “after Mask, offer Measure”. Each is resolved against the registry for its settings, then run throughspacr.ports.check_ready(), so an offer either works or says why not.- Parameters:
module – the module that just finished; key or alias.
settings – the settings it ran with, for the project root.
root – explicit project root.
roots – further project roots for the successor’s inputs.
registry – an open registry instead of the project’s own.
include_blocked – keep successors that cannot run, carrying their blocking reason. False drops them entirely.
- Returns:
one
NextStepper successor, ready ones first, then in module order.- Raises:
spacr.ports.UnknownModule – when
moduledeclares no ports.
- spacr.chaining.pin_store(path: str | None = None, *, refresh: bool = False) PinStore[source]¶
Return the process-wide
PinStore.- Parameters:
path – use a specific file instead of
state_path(). Passing one always builds a fresh store rather than handing back a cached one pointed at a different file.refresh – rebuild the shared store, re-reading
state_path(). Needed after$SPACR_CHAINING_PINSchanges, which is exactly what a test that isolates the state does.
- spacr.chaining.placeholder_paths() Tuple[str, ...][source]¶
Return the values that mean “nothing chosen yet”.
- spacr.chaining.ports_for_kinds(kinds: Sequence[str]) Tuple[spacr.ports.Port, ...][source]¶
Return the canonical declaration of where each kind lives.
A screen that is not a pipeline module — the table explorers, the viewers — still says what it wants in the shared vocabulary, and this is what turns that word into a path. The declaration is looked up in
spacr.ports.PORTS: a produced port first, because the module that writes a kind is the one that knows where it goes, and a consumed port only when nothing produces it.- Parameters:
kinds – vocabulary terms such as
spacr.ports.MEASUREMENTS_DB.- Returns:
one
spacr.ports.Portper kind that is declared anywhere, in the order asked for. An undeclared kind is skipped rather than guessed at.
- spacr.chaining.project_root_of(path: Any, *, max_climb: int = MAX_CLIMB) str[source]¶
Return the project root a dropped path belongs to.
The layout is walked upwards:
<root>/measurements/measurements.db,<root>/merged,<root>/data/plate1/cell_pngand<root>itself all answer<root>, becausemeasurements,mergedanddataare declared folders (layout_directories()) and nothing else on the way up is.The highest declared folder within
max_climbwins, so a drop deep insidedata/still lands on the project rather than on a crop folder.- Parameters:
path – the dropped file or folder.
max_climb – how many levels above the drop to consider.
- Returns:
an absolute path. Never raises: a path that is nowhere near a project answers with its own folder, which is what a direct drop wants anyway.
- spacr.chaining.register_binding(binding: Binding, *, overwrite: bool = False) Binding[source]¶
Declare that one port fills one settings key.
The seam a module with an unusual input key uses, so this table never has to be edited by hand.
- Parameters:
binding – the declaration.
overwrite – allow replacing an existing one. Off by default, so two contributors claiming one port is an error rather than last-one-wins.
- Returns:
the stored binding.
- Raises:
ValueError – on an empty field, an unknown form, or a duplicate.
- spacr.chaining.resolve_drop(module: str, dropped: Any, *, kinds: Sequence[str] = (), form: str = PATH, settings: Mapping[str, Any] | None = None, registry: spacr.artifacts.Registry | None = None, max_climb: int = MAX_CLIMB) DropResolution[source]¶
Work out what a dropped path means to
module.The whole point of the function is that it is the same resolution auto-chaining performs. For every input the module declares, the registry is asked first —
spacr.artifacts.Registry.latest()for that kind in that project, exactly aschained_inputs()asks it — so a drop and an auto-chain fill the field with the same string. Only when no run was ever registered does the declared layout inspacr.ports.PORTSanswer instead, and then it answers with the folder the ports say it is in.Ambiguity is returned, never guessed:
the dropped folder holds several projects →
DropChoice;the project holds several databases →
DropChoice;nothing satisfies the module →
DropResolution.problems, which isspacr.ports.check_ready()’s own list of sentences.
- Parameters:
module – the screen key. When it declares ports those are used; otherwise
kindssays what it wants.dropped – the path the user dropped.
kinds – vocabulary terms, for a screen with no port declaration.
form –
ROOTorPATH— what akinds-driven screen wants in its field. A declared module’s own bindings always win.settings – the settings dict being edited, for its current values.
registry – an open registry to ask instead of each project’s own.
max_climb – how far above the drop the project root may sit.
- Returns:
- spacr.chaining.resolve_settings(module: str, settings: Mapping[str, Any], *, root: str = '', roots: Sequence[str] = (), registry: spacr.artifacts.Registry | None = None, pins: PinStore | None = None, check_staleness: bool = True) Resolution[source]¶
Fill
module’s input paths from the registry, respecting user edits.The precedence, which is the whole design:
a pinned value wins. If the user has ever typed a path for this key, it is restored and nothing overwrites it — not a newer upstream run, not a different plate, not a restart. When the upstream has moved since, the move is reported in
Resolution.heldfor an interface to offer;otherwise, a value already in
settingsthat is not a placeholder wins. Loading a settings CSV, dropping a folder or seeding from another screen all land here, and none of them should be second-guessed within the same session;otherwise the registry’s answer is used.
- Parameters:
module – module key or alias.
settings – the settings dict to resolve. Not mutated.
root – explicit project root, searched first.
roots – further project roots, in preference order.
registry – an open registry to ask instead of each project’s own.
pins – the user’s pinned paths. Defaults to
pin_store().check_staleness – also report whether each input is out of date.
- Returns:
a
Resolution.- Raises:
spacr.ports.UnknownModule – when
moduledeclares no ports.
- spacr.chaining.result_tables(root: str) Tuple[str, ...][source]¶
Return the result tables a project has written, sorted.
- Parameters:
root – candidate project root to search.
The folders searched are the ones the result-bearing ports declare —
results/andsettings/today — one level deep, so a drop on a screen that reads “a table or a CSV” can offer the CSVs beside the database tables instead of making the user go and find them.
- spacr.chaining.same_path(left: Any, right: Any) bool[source]¶
Compare two settings values as paths, list-insensitively.
- Parameters:
left – first scalar or nested list or tuple of path values.
right – second path value or collection to compare.
["/plate"]and"/plate"name the same folder; Classify keeps its source in a list and every other module keeps it as a string, so a comparison that called those different would record a pin every time a Classify screen was seeded with its own auto-chained value.
- spacr.chaining.satisfies(root: str, ports: Sequence[spacr.ports.Port]) bool[source]¶
True when
rootholds everythingportsrequires.- Parameters:
root – candidate project root whose artifacts are checked.
ports – input port declarations whose required artifacts must resolve beneath
root.
With no ports the question is “is this a project at all?”, which is what a screen that takes a whole project — the pipeline graph, the QC dashboard — is asking.
- spacr.chaining.source_key(module: str) str[source]¶
Return the settings key naming
module’s source folder.The same lookup
spacr.portsandspacr.validatemake:spacr.ports.ROOT_KEYSfirst (a module whose output names the project), thenspacr.validate.ALT_SRC_KEYS, thensrc.- Parameters:
module – module key or alias.
- spacr.chaining.stale_inputs(module: str, settings: Mapping[str, Any] | None = None, *, root: str = '', registry: spacr.artifacts.Registry | None = None) Tuple[StaleNote, ...][source]¶
Return the inputs
modulewould read that are already out of date.Running on a stale input produces a stale result, so this is the warning that saves the twenty minutes rather than explaining them afterwards. Settings are deliberately not compared here: this module’s settings say nothing about whether the previous module’s output is current.
- Parameters:
module – module key or alias.
settings – the settings currently on screen, for the project root.
root – explicit project root; otherwise derived from
settings.registry – an open registry instead of the project’s own.
- spacr.chaining.stale_outputs(module: str, settings: Mapping[str, Any] | None = None, *, root: str = '', registry: spacr.artifacts.Registry | None = None) Tuple[StaleNote, ...][source]¶
Return the results
modulealready produced that are out of date.The warning a user needs before they open a figure or hand a number to a collaborator: the measurements in this project were made from a Mask run that has since been redone, or with settings that are not the ones now on screen.
settingsis compared against the recorded settings hash, so editing a material knob marks the existing result stale immediately — before the run that would fix it.- Parameters:
module – module key or alias.
settings – the settings currently on screen. Supplying them adds the
spacr.artifacts.CAUSE_SETTINGS_CHANGEDcause.root – explicit project root; otherwise derived from
settings.registry – an open registry instead of the project’s own.
- Returns:
one note per stale output, in declaration order. Empty when there is no registry, which is the answer for a project that has never recorded a run.
- spacr.chaining.staleness_notes(module: str, settings: Mapping[str, Any] | None = None, *, root: str = '', registry: spacr.artifacts.Registry | None = None) Tuple[StaleNote, ...][source]¶
Return every stale artifact around
module: its inputs and its outputs.Inputs first — a stale input explains a stale output, and saying it the other way round asks the user to work backwards.
- Parameters:
module – module key or alias.
settings – the settings currently on screen.
root – explicit project root.
registry – an open registry instead of the project’s own.
- spacr.chaining.state_path() str[source]¶
Return the file the user’s pinned paths live in.
$SPACR_CHAINING_PINSwins; otherwise XDG state storage, matchingspacr.remote_execution.state_directory()rather than inventing a second convention for the same kind of data.- Returns:
an absolute path. The file need not exist.
Nested helpers¶
- same_path.flatten(value: Any) List[str]¶
Flatten one path-like value into normalised path strings.
- Parameters:
value – scalar path, nested list or tuple,
None, or another value whose stripped string representation names a path.- Returns:
depth-first path strings with empty values omitted and each retained value normalised with
os.path.normpath().
spacr/chaining.py:164