spacr.sequencing¶

Workflow inputs and outputs¶

Map Barcodes¶

Supply the references that encode your experiment. Inspect mapped counts and barcode QC before pairing well-level guide counts with phenotype scores.

Open: Home → Map Barcodes.

Inputs and outputs below include conditional alternatives. The guidance and handoff notes say which route applies.

Inputs

Sequencing reads — Single-end or paired FASTQ files, with the matching barcode reference tables.

Outputs

Guide counts per well — Map Barcodes run folder: unique_combinations.csv and annotated_reads.h5. Well identity requires the corresponding barcode references. Relevant columns, depending on the route: count.

After this module

Regression: Pair guide counts with phenotype scores using consistent plate/well keys.
Barcode QC: Check mapping and coverage before guide filtering.

API reference.

Module tutorial.

Decode pooled-screen FASTQ reads into per-well guide counts.

WHAT IT IS FOR¶

The Map Barcodes module connects a sequencing run to an image-based screen. It finds the plate-column, guide, and plate-row barcode in each read, resolves those sequences against reference tables, and counts the resulting guides by well. generate_barecode_mapping() is the GUI and Python entry point (the historical barecode spelling remains part of the public API); graph_sequencing_stats() can then help choose a read-fraction cutoff.

WHAT IT NEEDS¶

src must contain gzip-compressed FASTQ files whose names let spaCR pair R1 and R2 reads. The run also needs a CSV reference table per barcode, with sequence and name columns. A run that decodes the three barcodes spaCR shipped names them as row_csv, column_csv and grna_csv, and a run that decodes any other number of barcodes lists them under barcode_set instead, one entry per barcode. target_sequence anchors the barcode window, while offset_start, expected_end, and a regex naming one group per barcode describe its layout. The shipped regex names columnID, grna and rowID. Use mode='paired' for a quality-weighted R1/R2 consensus or mode='single' with single_direction when only one mate should be read.

WHAT IT PRODUCES¶

Each sample gets its own output directory beneath src. The essential artifacts are unique_combinations.csv (guide counts by row and column) and qc.csv; annotated_reads.h5 is also written when save_h5 is enabled. Optional barcode QC adds a report and plots under barcode_qc. Run manifests and failure records identify samples that were skipped or could not be processed.

WHAT TO DO NEXT¶

Inspect the QC table and unmapped fraction before trusting the counts. Use test=True to process one chunk while checking the regex, read direction, and reference orientation, then run the full mapping and pass the resulting unique_combinations.csv files to spacr.ml.perform_regression() as count_data. barecodes_reverse_complement() can make an opposite-orientation copy of a reference CSV.

There are three easy ways to obtain plausible but incomplete output. The anchor match is exact and a wrong regex silently rejects reads; reference sequences are compared in their stored orientation; and a sequence within barcode_mismatches of two references remains unassigned instead of being guessed. Finally, read-level HDF5 output can be much larger than the count tables, and high compression may spend more time saving than decoding, so disable save_h5 unless those individual annotations are needed.

Functions¶

`barecodes_reverse_complement`(csv_file)	Write a copy of a barcode CSV with the `sequence` column reverse-complemented.
`create_consensus`(seq1, qual1, seq2, qual2)	Return a per-position consensus of two equal-length reads.
`display`(args, *kwargs)	Ignore notebook display requests while IPython is unavailable.
`extract_sequence_and_quality`(sequence, quality, start, end)	Return the `[start:end]` slice of a sequence and its paired quality string.
`generate_barecode_mapping`([settings])	Turn a folder of pooled-screen FASTQ files into per-well sgRNA count tables usable by `spacr.ml.perform_regression()`.
`get_consensus_base`(bases)	Return the higher-quality base from two `(base, quality)` pairs, preferring non-`N`.
`graph_sequencing_stats`(settings)	Pick the fraction cutoff that yields a target mean of unique gRNAs per well.
`map_sequences_to_names`(csv_file, sequences, rc[, ...])	Look up barcode / gRNA names for a list of DNA reads against a `sequence,name` mapping CSV.
`paired_read_chunked_processing`(r1_file, r2_file, ...)	Chunked paired-end FASTQ processing: extract, decode and stream barcodes to disk.
`process_chunk`(chunk_data)	Extract and map barcodes from a chunk of single- or paired-end FASTQ reads.
`reverse_complement`(seq)	Return the reverse complement of a DNA sequence via BioPython.
`save_df_to_hdf5`(df, hdf5_file[, key, comp_type, ...])	Append (or create) `df` to a `table`-format HDF5 dataset.
`save_qc_df_to_csv`(qc_df, qc_csv_file)	Accumulate a one-row QC frame into a CSV, summing it with what is there.
`save_unique_combinations_to_csv`(unique_combinations, ...)	Append per-barcode-combination counts to a CSV, summing duplicates.
`saver_process`(save_queue, hdf5_file, save_h5, ...)	Background writer that drains `save_queue` and persists each item.
`single_read_chunked_processing`(r1_file, r2_file, ...)	Chunked single-end FASTQ processing: extract, decode and stream barcodes to disk.

Module Contents¶

spacr.sequencing.barecodes_reverse_complement(csv_file)[source]¶

Write a copy of a barcode CSV with the sequence column reverse-complemented.

Output is saved in the same directory with the extension dropped and _RC.csv appended, so rows.csv becomes rows_RC.csv.

Parameters:: csv_file – input CSV path with a sequence column.
Returns:: None.

spacr.sequencing.create_consensus(seq1, qual1, seq2, qual2)[source]¶

Return a per-position consensus of two equal-length reads.

At each position the higher-quality base is kept; if one call is N the other is preferred regardless of quality.

Parameters:

seq1 – first DNA sequence.
qual1 – quality string for seq1.
seq2 – second DNA sequence.
qual2 – quality string for seq2.

Returns:

the consensus sequence as a string.

Raises:

ValueError – if either sequence and quality pair, or the two reads, have different lengths. A partial consensus could assign a barcode to the wrong well, so uneven reads are rejected rather than truncated.

spacr.sequencing.display(*args, **kwargs)[source]¶

Ignore notebook display requests while IPython is unavailable.

Parameters:

args – positional display values accepted for API compatibility.
kwargs – display options accepted for API compatibility.

Returns:

None.

spacr.sequencing.extract_sequence_and_quality(sequence, quality, start, end)[source]¶

Return the [start:end] slice of a sequence and its paired quality string.

Parameters:

sequence – DNA sequence.
quality – quality string of equal length.
start – inclusive start index.
end – exclusive end index.

Returns:

tuple (subsequence, subquality).

spacr.sequencing.generate_barecode_mapping(settings=None)[source]¶

Turn a folder of pooled-screen FASTQ files into per-well sgRNA count tables usable by spacr.ml.perform_regression().

Discovers the R1 and R2 files of each sample under src. Paired vs single-end and R1/R2 orientation are chosen from settings['mode'] and single_direction.

From every read it extracts each barcode the run decodes, using the configured regex and offset window. Each extracted sequence is then turned into a name through that barcode’s own lookup CSV (see map_sequences_to_names()).

Per sample it writes annotated_reads.h5 (optional), unique_combinations.csv (the per-well gRNA counts) and qc.csv.

Parameters:

settings –

Settings dict, canonicalized via spacr.settings.set_default_generate_barecode_mapping(). Key entries:

src — folder containing *.fastq.gz reads.
mode — 'paired' or 'single'.
single_direction — 'R1' or 'R2' (mode='single' only).
regex — regex extracting barcodes from a read.
target_sequence, offset_start, expected_end — anchor and slice window used to locate the barcode region.
column_csv / row_csv / grna_csv — barcode->name lookup CSVs for the three barcodes spaCR shipped.
barcode_set — the barcodes to decode when a run has other than those three, as a list with one entry per barcode. Each entry names the barcode, the reference CSV that names its sequences and the regex group that captures it. Absent, which is what every settings file written before sets existed is, the run decodes the three named above.
save_h5, comp_type, comp_level — HDF5 output knobs.
chunk_size, n_jobs, test, fill_na.
barcode_qc — when true, QC each finished sample with spacr.sequencing_qc.barcode_qc(), writing plots and a report into <dst>/barcode_qc (default False; not filled in by the settings defaults).
target_grnas_per_well — expected gRNAs per well, from which that QC step derives its abundance threshold (default 1).

Returns:

None. Writes per-sample outputs into <src>/<sample>_<mode>[_<direction>]/.

Example

from spacr.sequencing import generate_barecode_mapping
generate_barecode_mapping({
    'src': '/data/screen_v1/fastq',
    'mode': 'paired',
    'row_csv': '/data/barcodes/rows.csv',
    'column_csv': '/data/barcodes/cols.csv',
    'grna_csv': '/data/barcodes/grnas.csv',
})

Nested helpers¶

_map_within.resolve(sequence)¶

Return the unique reference name within the mismatch budget.

Exact matches bypass the scan. Inexact results, including an ambiguous pd.NA, are cached by their string representation so a repeated read does not rescan every same-length reference.

spacr/sequencing.py:186

barecodes_reverse_complement.reverse_complement(sequence)¶

Return the reverse complement of sequence (N stays N).

spacr/sequencing.py:1373

graph_sequencing_stats.find_and_visualize_fraction_threshold(df, target_unique_count=5, log_x=False, log_y=False, dst=None)¶

Return the fraction threshold whose per-well unique gRNA mean is closest to target_unique_count.

The sweep plot’s view is clamped to x in [0, 0.1] and y in [0, 20], so a returned threshold above 0.1 has its marker line drawn off the visible axes even though the value itself is correct.

Parameters:

df – one row per gRNA per well, with fraction plus plateID, rowID and columnID; any missing column raises KeyError.
target_unique_count – mean unique gRNAs per well to aim for. The answer is only the nearest point of a fixed 1000-step grid spanning 0.001 to 0.99, so an unreachable target silently saturates at one end of that grid instead of reporting that it was not met. None or a string raises from the subtraction.
log_x – log-scale the plot’s x axis. The hard-coded xlim(0, 0.1) applied afterwards is then rejected by matplotlib as a non-positive limit on a log axis, so the view autoscales.
log_y – the same for the y axis and ylim(0, 20).
dst – when set, writes <dst>/results/fraction_threshold.pdf, creating the directories; None only shows the figure. The sole caller always passes a path, so the default is unreachable there.

Returns:

the chosen threshold as a numpy.float64. A threshold that empties the table yields NaN rather than 0, so a table whose fractions are all below 0.001 makes every sweep point NaN and the fallback returns 0.99 – a value that then discards every read.

spacr/sequencing.py:1468

graph_sequencing_stats.find_and_visualize_fraction_threshold._line_plot(df, x, y, log_x, log_y)¶

Plot columns x and y using the shared figure style.

The returned figure and axes let the caller add its chosen threshold marker before either saving or displaying the plot.

spacr/sequencing.py:1496

map_sequences_to_names.rev_comp(dna_sequence)¶

Return the reverse complement of dna_sequence (N stays N).

spacr/sequencing.py:125

process_chunk._parse_record(record, label)¶

Validate and split one four-line FASTQ record.

Parameters:

record – the record as one string of four lines.
label – how to name this record in an error message.

Returns:

tuple (sequence, quality).

Raises:

ValueError – when the record is not four lines, is not a FASTQ record, or pairs a sequence with a quality string of another length.

spacr/sequencing.py:444

process_chunk._split_window(match)¶

Record every barcode the regex captured in this window.

Parameters:: match – the regex match over one consensus window.
Returns:: None. The captured text is appended to the lists in extracted, one list per barcode, so that every barcode of every matched read stays at the same position.

spacr/sequencing.py:470

process_chunk.paired_find_sequence_in_chunk_reads(r1_chunk, r2_chunk, target_sequence, offset_start, expected_end, regex)¶

Return consensus reads and their barcodes for paired-end chunks.

Parameters:

r1_chunk – four-line FASTQ record strings for R1.
r2_chunk – the matching R2 records, paired with r1_chunk by position only – headers are never compared.
target_sequence – anchor located with str.find (first occurrence). R2 is reverse-complemented before the search, so write the anchor in R1 orientation. A pair missing the anchor in either mate is dropped without a row, so the QC total_reads counts matched pairs rather than input pairs.
offset_start – bases from the anchor to the window start. A resulting start below zero is clamped to the read start, not rejected, so an over-negative offset quietly shifts the window.
expected_end – window length, not an end coordinate. A window cut short by the read end is right-padded with N (quality !) to exactly this length, so the length check always passes and a truncated read can still match with N inside a barcode – which then maps to NA and drops out of the per-well counts.
regex – pattern string applied with re.match: anchored at the window start, but bases past the last group are ignored, so an oversized expected_end only adds padding.

Returns:

the consensus sequence of every matched pair, one entry per pair, with that pair’s barcodes appended to extracted. A chunk with no matches prints a warning and retries the last window reverse-complemented as an orientation hint.

Raises:

ValueError – when the two chunks hold different read counts.

spacr/sequencing.py:481

process_chunk.single_find_sequence_in_chunk_reads(r1_chunk, target_sequence, offset_start, expected_end, regex)¶

Return R1 windows and their barcodes for single-end chunks.

No consensus is computed here: the R1 window is used as-is, so read quality never influences the base calls the way it does for pairs.

Parameters:

r1_chunk – four-line FASTQ record strings for R1.
target_sequence – anchor located with str.find (first occurrence). A read without it is dropped without a row, so the QC total_reads counts matched reads rather than input reads.
offset_start – bases from the anchor to the window start. A resulting start below zero is clamped to the read start, not rejected.
expected_end – window length, not an end coordinate. A window cut short by the read end is right-padded with N (quality !) to exactly this length, so a truncated read can still match with N inside a barcode – which then maps to NA and drops out of the per-well counts.
regex – pattern string applied with re.match: anchored at the window start, but bases past the last group are ignored.

Returns:

the window of every matched read, one entry per read, with that read’s barcodes appended to extracted. A chunk with no matches prints a warning and retries the last window reverse-complemented as an orientation hint.

spacr/sequencing.py:563