spacr.openmp_guard

Contain duplicate OpenMP runtimes during model fitting on macOS.

Torch, scikit-learn, and XGBoost wheels can load independent LLVM OpenMP runtimes into one process. Crossing their worker-thread state can terminate the process, particularly when KMP_DUPLICATE_LIB_OK suppresses the runtime’s duplicate-initialization guard.

single_threaded_openmp() detects mapped runtimes and, when more than one is resident, calls omp_set_num_threads(1) on the fitting thread. This serializes that thread’s parallel regions without imposing a process-wide OMP_NUM_THREADS=1 limit on segmentation and other workloads. Estimator n_jobs and nthread arguments are not substitutes because they do not prevent the OpenMP runtime from creating a worker team.

The module has no Qt dependency and is safe to use in command-line and cluster pipelines.

Classes

single_threaded_openmp

Serialize this thread's OpenMP regions while several runtimes are mapped.

Functions

guarded_n_jobs(requested[, label])	1 while the clamp is in force, otherwise requested unchanged.
openmp_runtime_is_duplicated(→ bool)	True when this process has more than one OpenMP runtime mapped.
resident_openmp_runtimes(→ List[str])	Distinct OpenMP runtime files currently mapped into this process.

Module Contents

class spacr.openmp_guard.single_threaded_openmp(label: str = 'this model')

Bases: contextlib.ContextDecorator

Serialize this thread’s OpenMP regions while several runtimes are mapped.

Usable as a decorator or a with block:

@single_threaded_openmp("classical ML")
def ml_analysis(...):
    ...

Does nothing at all when one runtime (or none) is mapped, when the platform is not one where the fault is documented, or when the user has set SPACR_OPENMP_GUARD=off. Restores each runtime’s previous value on the way out, so the clamp lasts exactly as long as the region it wraps.

Guard setup and restoration are best-effort and suppress their own failures. Exceptions from the protected body propagate unchanged.

Parameters:

label – user-facing name of the protected work, included in the one-time warning when duplicate runtimes require serialization.

Initialise a reusable OpenMP-clamping context.

Parameters:

label – user-facing protected-work name used in the warning.

__enter__()

Attempt to clamp every usable duplicate runtime and return self.

__exit__(exc_type, exc, tb)

Restore this entry’s limits and propagate any body exception.

Parameters:

• exc_type – protected-body exception type, or None.

• exc – protected-body exception instance, or None.

• tb – protected-body traceback, or None.

Returns:

False so protected-body exceptions propagate.

spacr.openmp_guard.guarded_n_jobs(requested, label: str = 'this step')

1 while the clamp is in force, otherwise requested unchanged.

Parameters:

• requested – caller-requested job count to constrain when guarded.

• label – name of the surrounding guarded call site; retained for API consistency and diagnostics but does not alter the returned count.

Returns:

1 when duplicate runtimes are being clamped, otherwise requested; probe failures also return requested.

For joblib call sites inside a single_threaded_openmp region — permutation_importance above all, which re-enters the fitted model.

The clamp is a per-thread ICV, so a joblib worker thread starts with the process default (10 here, not 1) and builds a team the region was supposed to prevent. Measured on this repository’s own ml_analysis: 19 threads parked in __kmp_launch_worker unguarded, still 10 with only the region clamp, and those 10 belong to Homebrew’s libomp — xgboost’s runtime, the one that crashed. Keeping joblib on the calling thread keeps the clamp meaningful.

This is NOT the earlier, wrong idea of clamping the estimator’s own thread argument: that was measured to change nothing at all.

spacr.openmp_guard.openmp_runtime_is_duplicated() → bool

True when this process has more than one OpenMP runtime mapped.

Reports the condition on every platform. Whether it is acted on is single_threaded_openmp()’s decision, not this one’s, unless SPACR_OPENMP_GUARD=off explicitly forces False.

spacr.openmp_guard.resident_openmp_runtimes() → List[str]

Distinct OpenMP runtime files currently mapped into this process.

Returns resolved paths, deduplicated and sorted. Two copies of a byte-identical build at two paths are still two runtimes with two sets of globals, so they are counted separately — but a symlink and its target are one file and are not.

Returns:

resolved, deduplicated runtime paths in sorted order, or [] on any platform or failure where the answer is unknown.

Unknown must read as “no evidence of trouble”, because the caller’s fallback is the behaviour spaCR has always had.