Mastering a reproducible scRNA-seq preprocessing Scanpy pipeline is the foundation of trustworthy single-cell transcriptomics. In this practical guide, you will learn how to take raw count matrices from droplet-based sequencing experiments, inspect distribution-based quality control metrics, remove low-quality cellular barcodes, normalize expression values, and isolate biologically informative variable genes. By the end of this tutorial, your data will be clean, structured, and ready for dimensionality reduction, integration, or downstream differential expression.

Single-cell RNA sequencing data is inherently noisy, sparse, and susceptible to technical artifacts such as ambient RNA contamination, barcode swapping, and broken cell membranes. Setting up a standardized preprocessing protocol ensures your biological conclusions reflect genuine cell states rather than computational artifacts or library-preparation bias.

---

Key Concepts in scRNA-seq Preprocessing Scanpy Pipelines

Before executing commands, it is essential to understand the underlying data infrastructure. The single-cell Python ecosystem centers on anndata, an open-source library that handles annotated data matrices.

Under the hood, AnnData relies on HDF5, a binary format designed for large hierarchical datasets featuring chunking and compression. This structure stores primary count matrices alongside observation metadata (.obs, such as cell barcodes, sample batches, and total counts) and variable metadata (.var, such as gene symbols and genomic coordinates). The format also accommodates multidimensional annotations (.obsm, .varm) and alternative matrix states (.layers). Saving annotated datasets in the native .h5ad format satisfies the FAIR data principles—Findable, Accessible, Interoperable, Reusable—which were published in 2016 to ensure computational scientific research remains transparent and exchangeable across institutions.

---

Step-by-Step Preprocessing Protocol

Here is a step-by-step workflow for preprocessing single-cell counts using scanpy.

Step 1: Ingest Raw Data and Initialize Quality Metrics

Begin by loading your unnormalized feature-barcode matrix into an AnnData object. Single-cell matrices are typically stored in sparse matrix formats to preserve memory. Immediately calculate basic quality metrics: total unique molecular identifier (UMI) counts per droplet, detected genes per droplet, and the proportion of reads mapping to mitochondrial genes.

import scanpy as sc
import numpy as np

# Configure Scanpy visualization and verbosity
sc.settings.verbosity = 2
sc.settings.set_figure_params(dpi=80, facecolor="white")

# Load raw feature-barcode matrix (10x Genomics HDF5 example)
adata = sc.read_10x_h5("raw_feature_bc_matrix.h5")
adata.var_names_make_unique()

# Identify mitochondrial, ribosomal, and hemoglobin genes
adata.var["mt"] = adata.var_names.str.startswith("MT-")
adata.var["ribo"] = adata.var_names.str.startswith(("RPS", "RPL"))

# Calculate standard QC metrics
sc.pp.calculate_qc_metrics(
    adata, 
    qc_vars=["mt", "ribo"], 
    percent_top=None, 
    log1p=False, 
    inplace=True
)

# Preserve pristine raw integer counts in a dedicated layer
adata.layers["counts"] = adata.X.copy()

Step 2: Filter Low-Quality Droplets and Uninformative Genes

Never apply arbitrary, hard-coded cutoffs across disparate tissue types without inspecting distributions. Broken cells release cytoplasmic RNA while retaining mitochondria inside damaged membranes, causing inflated mitochondrial percentages. Conversely, empty droplets contain ambient RNA with low overall UMI counts and few detected genes.

Inspect violin plots or scatter plots of total_counts versus n_genes_by_counts and pct_counts_mt. Identify outliers that deviate sharply from the primary distribution:

# Optional visualization of QC distributions prior to subsetting
# sc.pl.violin(adata, ['n_genes_by_counts', 'total_counts', 'pct_counts_mt'], jitter=0.4, multi_panel=True)

# Example filtering based on empirical distribution inspection
min_genes = 200
min_cells = 3
max_mt_percent = 20.0

sc.pp.filter_cells(adata, min_genes=min_genes)
sc.pp.filter_genes(adata, min_cells=min_cells)

# Retain only cells below the chosen mitochondrial threshold
adata = adata[adata.obs["pct_counts_mt"] < max_mt_percent, :].copy()

Step 3: Normalize Sequencing Depth and Transform

Variations in total sequencing depth per cell stem from stochastic capture efficiency and library construction rather than true biological divergence. To compare expression profiles, divide the counts within each cell by its total count, scale by a size factor (typically the median cell count or a standard target such as 10,000), and apply a logarithmic transformation:

# Normalize counts per cell to a target sum
sc.pp.normalize_total(adata, target_sum=1e4)

# Apply natural log transformation (log(x + 1))
sc.pp.log1p(adata)

# Store the normalized, unscaled state for differential expression testing
adata.raw = adata

Step 4: Select Highly Variable Genes (HVGs)

A standard single-cell experiment tracks tens of thousands of genes, but most either show uniform baseline expression or low non-informative variance. Filtering for highly variable genes focuses downstream analytical steps on informative biological signals.

# Identify highly variable genes using sample variance dispersion
sc.pp.highly_variable_genes(
    adata,
    min_mean=0.0125,
    max_mean=3,
    min_disp=0.5,
    batch_key=None  # Specify batch column name if multiple samples are pooled
)

# Subset or flag the AnnData object to variable features
adata_hvg = adata[:, adata.var["highly_variable"]].copy()

Step 5: Scale and Prepare for Latent Modeling

For traditional linear dimensionality reduction (Principal Component Analysis), shift gene expression values to zero mean and unit variance. If you plan to model batch effects or non-linear technical noise using deep generative tools like scvi-tools, do not feed z-score scaled data into their probabilistic models; instead, pass the unscaled integer count matrix stored in adata.layers["counts"].

To benchmark or annotate your cell types against large reference atlases, you can cross-reference curated reference profiles directly from cellxgene-census.

# Scale variable features for PCA and neighbor graph construction
sc.pp.scale(adata_hvg, max_value=10)

# Compute PCA coordinates
sc.tl.pca(adata_hvg, svd_solver="arpack")

# Write preprocessed object to disk in standard HDF5-backed format
adata_hvg.write("preprocessed_dataset.h5ad")

Step 6: Package into a Reproducible Pipeline

Interactive notebooks are helpful for exploratory data analysis, but reproducible workflows require formal pipeline automation. Workflow managers such as Snakemake and Nextflow define multi-step analyses as rules or processes with declared inputs and outputs. By wrapping your preprocessing Python script inside these managers, you guarantee deterministic reruns, parallel execution across high-performance computing clusters, and portable pipelines that other researchers can execute without manual intervention.

---

Checks Before You Trust the Result

Before proceeding to cluster annotation or trajectory inference, perform these validation checks on your preprocessed .h5ad file:

  • Verify matrix sparsity and layer contents: Confirm that adata.layers["counts"] retains integer values and that adata.raw contains log-normalized, unscaled expression.
  • Inspect mitochondrial vs. count correlations: Plot total_counts against pct_counts_mt. If a subset of cells forms a separate cluster defined solely by high mitochondrial reads and low gene counts, your initial filtering threshold was too permissive.
  • Evaluate variable gene distributions: Plot mean expression versus dispersion with sc.pl.highly_variable_genes(adata). Ensure that housekeeping genes (such as actin or ribosomal subunits) do not dominate the selected HVG list unless justified by biological perturbation.
  • Review batch contributions: Run an initial PCA and check whether the first few principal components correlate predominantly with sequencing batch, processing date, or donor ID rather than known biological cell markers.

---

A rigorous preprocessing protocol protects single-cell studies from technical noise, artifactual clusters, and irreproducible statistical findings. By combining AnnData container standards, clear filtering heuristics, and structured workflow automation, you establish a solid empirical foundation for downstream single-cell biology.

Before deploying these methods in production or designing autonomous agent workflows, visit the linked scanpy, anndata, scvi-tools, and cellxgene-census skill pages on ScienceSkills to review underlying documentation, parameter assumptions, and source dependencies.