spacr.qt.resonance

What the Resonance backdrop is driven by, and the plate it draws.

Two halves, both Qt-free and both numpy-only, kept together because they are the two ends of one idea: a sound is measured here, and a vibrating plate is solved here, and the backdrop in spacr.qt.widgets.ambient.ResonanceEngine is only the painting.

The measurement. analyse() turns a rendered WAV into a few hundred kilobytes of envelope, four band energies, a spectral centroid and an onset strength, sixty times a second, and ensure_analysis() keeps that beside the WAV so it is computed once and read thereafter. spacr.qt.sound does the computing on its own audio thread when it loads the music bed, and records what is playing and when it started (set_now_playing()); the backdrop reads the small arrays and the clock. Nothing here listens to the machine. There is no system-audio capture anywhere in spaCR and there is not going to be: the only sounds this can see are the ones spaCR is playing and the file the user chose.

The plate. Chladni figures are what sand does on a plate driven at one of its resonances: it is thrown off the parts that move and comes to rest along the lines that do not, so the figure is the nodal set of a standing wave made visible. For a square plate the classical superposition is

\[w(x, y) = \cos(n \pi x)\cos(m \pi y) \pm \cos(m \pi x)\cos(n \pi y)\]

and the sand settles where \(w = 0\). Both signs are real families of figures and MODES alternates them; see there for what leaving the plus sign out looked like. lattice() evaluates that field and its gradient on a small grid, and settle() walks particles down it. Both are separable in x and y, so a whole lattice costs four outer products and a frame costs a handful of array lookups – which is the only reason a particle field can afford to be a backdrop at all.

WHY A LATTICE AND NOT AN EVALUATION PER PARTICLE. Eight relaxation rounds over 1 300 particles is 10 400 evaluations of eight trigonometric functions each if the field is evaluated where the particle is; on a 128x128 lattice it is four outer products, once, and then 10 400 integer lookups. Measured on this machine, eight rounds over 1 300 particles: 0.61 ms with the lattice against 0.76 ms without it, which is a modest 1.26x – the point is not the ratio today but that the FIELD’s share stops depending on the particle count at all, so density can be raised without the trigonometry following it.

AND WHY THE FIGURE IS RECOMPUTED FROM A FIXED START EVERY FRAME RATHER THAN CARRIED FORWARD. Every ambient engine promises that a frame is a pure function of (seed, clock, size) – test_the_clock_alone_decides_the _frame steps one engine twelve times and jumps another straight to the same clock and demands the same picture, and test_the_backdrop_survives_a_run shades the same clock on two threads and compares the bytes. A particle simulation carried forward from frame to frame satisfies neither. Relaxation from a seeded start does, costs 0.61 ms, and has the better behaviour besides: the figure always belongs to the sound that is playing now rather than to the last few seconds of it.

Classes

Moment	One instant of whatever is playing, all of it 0 to 1.
NowPlaying	What is coming out of the speakers, and when it started.
Resonance	Everything the visualiser knows about one piece of audio.

Functions

analyse(→ Resonance)	Measure what a visualiser needs, frame by frame.
analysis_path_for(→ pathlib.Path)	Where the analysis of wav_path is kept.
clear_now_playing(→ None)	Nothing is playing any more.
ensure_analysis(→ Optional[pathlib.Path])	Analyse wav_path unless a current analysis is already beside it.
lattice(→ Tuple[numpy.ndarray, numpy.ndarray, ...)	The plate's displacement and its gradient, on a grid x grid.
load_analysis(→ Optional[Resonance])	Read an analysis written by ensure_analysis(), once.
mode_blend(→ Tuple[Tuple[int, int, int], Tuple[int, ...)	Which two entries of MODES a figure is between.
now_playing(→ Optional[NowPlaying])	What is playing, as last recorded.
playing_moment(→ Moment)	The moment of whatever is playing, right now.
read_wav_mono(→ Tuple[numpy.ndarray, int])	Read a 16-bit PCM WAV down to one channel.
sample(→ numpy.ndarray)	field read at the particles, nearest lattice point.
set_now_playing(→ None)	Record what is playing, from whichever thread started it.
settle(→ Tuple[numpy.ndarray, numpy.ndarray])	Walk particles down |w| toward the plate's nodal lines.
silence(→ Moment)	The moment that stands for nothing playing.

Module Contents

class spacr.qt.resonance.Moment

Bases: NamedTuple

One instant of whatever is playing, all of it 0 to 1.

Parameters:

• level – overall loudness against the loop’s own loudest.

• bands – energy in each of BANDS, each against its own loudest, so a band that is always quiet still says something.

• centroid – where the spectrum’s centre of mass is, on a log scale between 60 Hz and 8 kHz.

• onset – spectral flux – how much of the sound is NEW. This is what a beat looks like when it is measured rather than guessed at from a tempo the visualiser was told.

class spacr.qt.resonance.NowPlaying

What is coming out of the speakers, and when it started.

THE CLOCK IS THE WHOLE MECHANISM, and it is worth saying why there is no other. QSoundEffect has no playback position to ask for, and capturing the machine’s audio output is not something a scientific tool should be doing to somebody’s computer. So the audio thread records the instant it called play() and the length of the loop, and the position is arithmetic. Both clocks are real time, so the two drift only by the audio device’s own rate error – tens of parts per million, a millisecond an hour – and the loop wraps it out anyway.

Parameters:

• analysis – path to the .resonance.npz for what is playing.

• started – time.monotonic() at the moment playback began.

• duration – length of the piece in seconds.

• loop – whether it repeats.

class spacr.qt.resonance.Resonance

Everything the visualiser knows about one piece of audio.

Parameters:

• fps – analysis frames per second.

• duration – length of the audio in seconds.

• level – per-frame loudness, 0 to 1.

• bands – (4, frames), each row 0 to 1.

• centroid – per-frame spectral centroid, 0 to 1.

• onset – per-frame spectral flux, 0 to 1.

• loop – whether the audio repeats seamlessly, which decides whether at() wraps or clamps.

at(seconds: float) → Moment

The moment at seconds into the audio.

Linear between neighbouring frames, so a 24 fps backdrop reading a 60 fps analysis moves smoothly instead of stepping. A looping piece wraps – including BETWEEN its last frame and its first, which is the one interpolation a naive clamp gets wrong and the one a listener would hear as a stall every time round.

Parameters:

seconds – position in the audio.

Returns:

the interpolated moment.

property frames: int

How many analysis frames there are.

spacr.qt.resonance.analyse(mono: numpy.ndarray, sr: int, fps: float = ANALYSIS_FPS, loop: bool = True) → Resonance

Measure what a visualiser needs, frame by frame.

Every row is normalised against its OWN 98th percentile rather than against the loudest thing in the file. A shaker that never rises above -40 dB is still the loudest thing in its own band, and a band scaled by the kick’s peak would be a row of zeros – which is how an audio visualiser ends up with three bars that move and one that does not.

Parameters:

• mono – one channel of samples.

• sr – sample rate.

• fps – analysis frames per second.

• loop – whether the audio repeats seamlessly.

Returns:

the measurement.

spacr.qt.resonance.analysis_path_for(wav_path, out_dir=None) → pathlib.Path

Where the analysis of wav_path is kept.

Beside the file by default, which is right for spaCR’s OWN rendered bed: it lives in the sound cache already and is swept away with the theme folder when the fingerprint changes.

IT IS WRONG FOR A FILE THE USER CHOSE, and that is what out_dir is for. Writing a sidecar into somebody’s music folder is a scientific tool leaving litter in a directory it was only asked to read from, and the folder may not even be writable. spacr.qt.sound passes the sound cache for a chosen file, and the name carries a hash of the source’s absolute path so two files called loop.wav in different folders are two analyses.

Parameters:

• wav_path – the audio file.

• out_dir – a folder to keep the analysis in instead of beside it.

Returns:

the .resonance.npz path.

spacr.qt.resonance.clear_now_playing() → None

Nothing is playing any more.

spacr.qt.resonance.ensure_analysis(wav_path, loop: bool = True, out_dir=None) → pathlib.Path | None

Analyse wav_path unless a current analysis is already beside it.

Never call this from the GUI thread. It reads a whole file and runs a few thousand FFTs; spacr.qt.sound calls it on the audio thread right after it has rendered or loaded the bed, which is where the rest of that work already is.

The stored analysis carries the source’s size and modification time, so a user who replaces their chosen WAV with a different one of the same name is analysed again rather than visualised as the old one.

Parameters:

• wav_path – the audio file.

• loop – whether the audio repeats seamlessly.

• out_dir – where to keep the analysis; see analysis_path_for().

Returns:

the analysis path, or None when the audio could not be read or the analysis could not be written.

spacr.qt.resonance.lattice(first: Tuple[int, int, int], second: Tuple[int, int, int], blend: float, grid: int = 128) → Tuple[numpy.ndarray, numpy.ndarray, numpy.ndarray]

The plate’s displacement and its gradient, on a grid x grid.

Both modes are evaluated and mixed rather than one mode being drawn at a time, because the nodal set of a mixture is a continuous deformation of the nodal sets of its parts: that is what makes one figure MORPH into the next instead of being replaced by it.

Separable throughout – every term is a product of one function of x and one of y – so the whole lattice is eight one-dimensional cosines and four outer products.

Parameters:

• first – (m, n, sign) of the drive being left.

• second – (m, n, sign) of the drive being arrived at.

• blend – 0 at first, 1 at second.

• grid – lattice edge.

Returns:

(w, dw/dx, dw/dy), each (grid, grid) and indexed [y, x], with w scaled to -1..1.

spacr.qt.resonance.load_analysis(path) → Resonance | None

Read an analysis written by ensure_analysis(), once.

Cached on the path and its modification time, and a file that cannot be read is remembered as unreadable so a missing analysis costs one failed stat and not one per frame.

Parameters:

path – the .resonance.npz.

Returns:

the analysis, or None.

spacr.qt.resonance.mode_blend(position: float) → Tuple[Tuple[int, int, int], Tuple[int, int, int], float]

Which two entries of MODES a figure is between.

Parameters:

position – where along the list the figure sits; wrapped, so a drive that walks off the end comes back to the simplest figure rather than sticking on the busiest.

Returns:

(first drive, second drive, 0..1 between them).

spacr.qt.resonance.now_playing() → NowPlaying | None

What is playing, as last recorded.

Returns:

the record, or None.

spacr.qt.resonance.playing_moment(at: float | None = None) → Moment

The moment of whatever is playing, right now.

Safe on any thread and cheap enough for a frame: a lock, an arithmetic position and two array reads, with NO file access at all – see set_now_playing(). Returns silence() whenever nothing is playing, the analysis is missing, or the piece has ended, so the caller has one code path and the backdrop idles rather than failing.

Parameters:

at – the monotonic clock to read it at; now, by default.

Returns:

the moment.

spacr.qt.resonance.read_wav_mono(path) → Tuple[numpy.ndarray, int]

Read a 16-bit PCM WAV down to one channel.

Deliberately NOT spacr.qt.sound_synth.read_wav(): that module imports scipy, and this one is reached from the backdrop’s shading thread by somebody who may have sound switched off entirely. Reading a WAV is the standard library and numpy, and that is all this needs.

Parameters:

path – the file.

Returns:

(mono samples in -1..1, sample rate).

Raises:

ValueError – for anything that is not 16-bit PCM.

spacr.qt.resonance.sample(field: numpy.ndarray, x: numpy.ndarray, y: numpy.ndarray) → numpy.ndarray

field read at the particles, nearest lattice point.

Parameters:

• field – any (grid, grid) from lattice().

• x – particle x in 0..1.

• y – particle y in 0..1.

Returns:

one value per particle.

spacr.qt.resonance.set_now_playing(record: NowPlaying | None) → None

Record what is playing, from whichever thread started it.

THE ANALYSIS IS READ HERE AND NOWHERE ELSE, which is the whole reason this is a setter rather than a variable. playing_moment() is called once a frame from the backdrop’s clock, which runs on the GUI thread; a stat there is a filesystem call on the GUI thread, and on a network home directory that is exactly the stall spaCR has spacr.qt.path_probe for. This runs on the audio thread, which has just rendered the file and is the right place to open it.

Parameters:

record – the piece, or None for silence.

spacr.qt.resonance.settle(x: numpy.ndarray, y: numpy.ndarray, field: numpy.ndarray, dx: numpy.ndarray, dy: numpy.ndarray, rounds: int, tightness) → Tuple[numpy.ndarray, numpy.ndarray]

Walk particles down |w| toward the plate’s nodal lines.

Each round is one Newton step onto the linearised zero set – p -= t * w * grad(w) / |grad(w)|^2 – which lands on the nodal line in one step where the field is locally straight and converges in a handful where it is not. That is what the sand does, and it is cheaper than the gradient descent it replaces, which needs a step size that depends on the mode numbers.

tightness below 1 leaves the particles short of the lines, which is what a plate driven gently looks like: the sand gathers but does not resolve. It is the control that makes silence read as a loose cloud and a loud passage as a figure. It may be one number for every particle or ONE PER PARTICLE, and per particle is what the backdrop uses: a plate where every grain converges equally draws a wire diagram, and a real one has a haze of sand that never quite arrives. The haze is what makes the picture look like sand rather than like a plot of an equation.

Parameters:

• x – particle x in 0..1. Copied, never modified in place: the caller’s array is the seeded start it needs again next frame.

• y – particle y in 0..1.

• field – w from lattice().

• dx – dw/dx.

• dy – dw/dy.

• rounds – how many Newton steps to take.

• tightness – fraction of each step to take, 0 to 1; a number or one value per particle.

Returns:

(x, y), new arrays in 0..1.

spacr.qt.resonance.silence() → Moment

The moment that stands for nothing playing.

Not zeros: a plate at rest still has a plate, and the backdrop has to idle beautifully rather than go out. The engine reads this as “breathe slowly on your own clock”, which is what spacr.qt.widgets.ambient.RESONANCE_IDLE is for.

Returns:

an all-zero moment.