Managing large and complex training datasets

Managing large and complex training datasets#

Large and complex datasets can be difficult to use for model training because of computational constraints; simply loading such datasets may require substantial amounts of RAM.

In addition, the time required to select genes for model training increases as dataset size grows.

However, using more training data does not always lead to a substantial improvement in model performance.

This tutorial presents a method for overcoming these computational limitations when training scAdam models on large, complex datasets with many samples.

[1]:

# Python packages
import warnings
warnings.simplefilter('ignore')

import scanpy as sc
import scparadise
import numpy as np
import pandas as pd
import os

sc.set_figure_params(dpi = 120)

[2]:

# Create folder to save files
# single nuclear RNA-seq of human retina
os.makedirs('snRNAseq_human_retina', exist_ok=True)

[3]:

# Download CELLxGENE dataset (snRNA-seq of human retina):
# https://cellxgene.cziscience.com/collections/4c6eaf5c-6d57-4c76-b1e9-60df8c655f1e
!wget https://datasets.cellxgene.cziscience.com/2e910e62-7eaf-4c06-80cb-8918e3eea16e.h5ad

--2025-02-08 13:49:05--  https://datasets.cellxgene.cziscience.com/2e910e62-7eaf-4c06-80cb-8918e3eea16e.h5ad
Resolving datasets.cellxgene.cziscience.com (datasets.cellxgene.cziscience.com)... 52.85.49.24, 52.85.49.28, 52.85.49.17, ...
Connecting to datasets.cellxgene.cziscience.com (datasets.cellxgene.cziscience.com)|52.85.49.24|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 37946797973 (35G) [binary/octet-stream]
Saving to: ‘2e910e62-7eaf-4c06-80cb-8918e3eea16e.h5ad’

2e910e62-7eaf-4c06- 100%[===================>]  35.34G  25.1MB/s    in 27m 20s

2025-02-08 14:13:58 (22.1 MB/s) - ‘2e910e62-7eaf-4c06-80cb-8918e3eea16e.h5ad’ saved [37946797973/37946797973]


[4]:

# Open the dataset in backed mode to inspect the cell-type annotations
# without loading the full expression matrix into memory.
adata = sc.read_h5ad('2e910e62-7eaf-4c06-80cb-8918e3eea16e.h5ad', backed = 'r')
adata

[4]:

AnnData object with n_obs × n_vars = 3177310 × 36406 backed at '2e910e62-7eaf-4c06-80cb-8918e3eea16e.h5ad'
    obs: 'reference_genome', 'gene_annotation_version', 'alignment_software', 'intronic_reads_counted', 'donor_id', 'donor_age', 'self_reported_ethnicity_ontology_term_id', 'donor_cause_of_death', 'donor_living_at_sample_collection', 'organism_ontology_term_id', 'sample_id', 'sample_preservation_method', 'tissue_ontology_term_id', 'development_stage_ontology_term_id', 'sample_collection_method', 'tissue_source', 'tissue_type', 'suspension_derivation_process', 'suspension_dissociation_reagent', 'suspension_enriched_cell_types', 'suspension_enrichment_factors', 'suspension_uuid', 'suspension_type', 'tissue_handling_interval', 'library_id', 'assay_ontology_term_id', 'sequenced_fragment', 'institute', 'library_id_repository', 'sequencing_platform', 'is_primary_data', 'cell_type_ontology_term_id', 'author_cell_type', 'disease_ontology_term_id', 'sex_ontology_term_id', 'majorclass', 'AC_subclass', 'AC_cluster', 'AC_celltype_number', 'BC_subclass', 'RGC_cluster', 'RGC_celltype_number', 'study_name', 'nCount_RNA', 'nFeature_RNA', 'percent.mt', 'sampleid', 'cell_type', 'assay', 'disease', 'organism', 'sex', 'tissue', 'self_reported_ethnicity', 'development_stage', 'observation_joinid'
    var: 'feature_is_filtered', 'feature_name', 'feature_reference', 'feature_biotype', 'feature_length', 'feature_type'
    uns: 'citation', 'default_embedding', 'schema_reference', 'schema_version', 'title'
    obsm: 'X_scVI', 'X_umap'

[5]:

# Count cells for each cell type in the dataset
adata.obs['cell_type'].value_counts()

[5]:

cell_type
retinal rod cell                    1066056
GABAergic amacrine cell              362866
Mueller cell                         221612
OFF midget ganglion cell             200402
ON midget ganglion cell              151685
flat midget bipolar cell             144086
glycinergic amacrine cell            131207
retinal cone cell                    118559
invaginating midget bipolar cell     106152
rod bipolar cell                     101199
H1 horizontal cell                    68583
diffuse bipolar 2 cell                68176
amacrine cell                         57236
retinal bipolar neuron                52358
diffuse bipolar 1 cell                50429
diffuse bipolar 4 cell                49157
diffuse bipolar 3b cell               35342
retinal ganglion cell                 31230
giant bipolar cell                    25269
diffuse bipolar 3a cell               21827
starburst amacrine cell               20270
diffuse bipolar 6 cell                18587
OFFx cell                             15495
astrocyte                             14085
H2 horizontal cell                    11965
OFF parasol ganglion cell             10104
S cone cell                            8501
ON parasol ganglion cell               6184
microglial cell                        4894
ON-blue cone bipolar cell              2931
retinal pigment epithelial cell         863
Name: count, dtype: int64

Train a scAdam model using a fraction of the dataset#

The entire dataset contains 3,177,310 cells and 36,406 genes and occupies 35.34 GB on disk, making it impractical to load into memory on a standard computer. Large single-cell datasets may require substantially more RAM than their file size suggests once loaded, depending on the matrix representation and associated metadata.

In addition, gene selection and model training on a dataset of this size require substantial computational time and resources.

Therefore, the scParadise team recommends extracting a smaller, representative subset of the dataset for scAdam model training.

[6]:

# Obtain 25000 cells from dataset
adata_fraction = scparadise.scnoah.get_frac(
    path = '2e910e62-7eaf-4c06-80cb-8918e3eea16e.h5ad',
    fraction = 25000,
    stratify = 'cell_type', # Use to get the same ratio as in the full dataset
    random_state = 0
)

[7]:

# Count cells for each cell type in the fraction of dataset
adata_fraction.obs['cell_type'].value_counts()

[7]:

cell_type
retinal rod cell                    8388
GABAergic amacrine cell             2855
Mueller cell                        1744
OFF midget ganglion cell            1577
ON midget ganglion cell             1193
flat midget bipolar cell            1134
glycinergic amacrine cell           1032
retinal cone cell                    933
invaginating midget bipolar cell     835
rod bipolar cell                     796
H1 horizontal cell                   540
diffuse bipolar 2 cell               536
amacrine cell                        450
retinal bipolar neuron               412
diffuse bipolar 1 cell               397
diffuse bipolar 4 cell               387
diffuse bipolar 3b cell              278
retinal ganglion cell                246
giant bipolar cell                   199
diffuse bipolar 3a cell              172
starburst amacrine cell              159
diffuse bipolar 6 cell               146
OFFx cell                            122
astrocyte                            111
H2 horizontal cell                    94
OFF parasol ganglion cell             79
S cone cell                           67
ON parasol ganglion cell              49
microglial cell                       39
ON-blue cone bipolar cell             23
retinal pigment epithelial cell        7
Name: count, dtype: int64

[8]:

# Get raw counts from adata_fraction.raw
adata_fraction = adata_fraction.raw.to_adata()
# Replace variable names with gene names
adata_fraction.var.set_index('feature_name', inplace = True)
adata_fraction.var_names = adata_fraction.var_names.astype('str')
adata_fraction.var_names_make_unique()
# Normalize data
sc.pp.normalize_total(adata_fraction, target_sum = None)
sc.pp.log1p(adata_fraction)

[9]:

# Select genes for model training (marker genes of cell types)
lst_genes = []
lst_annotations = ['majorclass', 'cell_type'] # annotation levels
for annotation in lst_annotations:
    sc.tl.rank_genes_groups(adata_fraction,
                            groupby = annotation,
                            method = 't-test_overestim_var', pts = True)
    # Filter marker genes of cell types
    sc.tl.filter_rank_genes_groups(adata_fraction,
                                   min_fold_change = 1.0,
                                   min_in_group_fraction = 0.4,
                                   key_added = 'filtered_rank_genes_groups')
    # Create list of genes for model training

    for i in adata_fraction.obs[annotation].unique():
        df = sc.get.rank_genes_groups_df(adata_fraction, group = i, key = 'filtered_rank_genes_groups', pval_cutoff = 0.05)
        df['pts_comparizon'] = df['pct_nz_group']/df['pct_nz_reference']
        lst_genes.extend(df.sort_values(by = 'logfoldchanges', ascending = False).head(20)['names'].tolist())
        lst_genes.extend(df.sort_values(by = 'pts_comparizon', ascending = False).head(20)['names'].tolist())
# Remove duplicates
lst_genes = np.unique(lst_genes).tolist()
print(f'Number of selected features: {len(lst_genes)}')

Number of selected features: 595

[10]:

# Subset genes for model training
adata_fraction = adata_fraction[:, lst_genes]

[11]:

# Alternative way to select genes for model training
# sc.pp.highly_variable_genes(adata_fraction,
#                             n_top_genes = 1000,
#                             subset = True)
# lst_genes = adata_fraction.var_names.tolist()

[12]:

adata_balanced = scparadise.scnoah.balance(
    adata_fraction,
    celltype_keys = lst_annotations
)

[13]:

# Train scadam model using adata_fraction dataset
scparadise.scadam.train(
    adata_balanced,
    path = 'snRNAseq_human_retina', # path to save model
    model_name = 'Human_Retina_scAdam', # folder name with model
    celltype_keys = lst_annotations,
    eval_metric = ['accuracy', 'balanced_accuracy']
)

Device: cuda
Number of features: 595
Label hierarchy: majorclass → cell_type
Annotation levels weights using strategy 'linear_offset':
  majorclass: 10 cell types, 0.4 relative weight
  cell_type: 31 cell types, 0.6 relative weight

Dataset split:
Train dataset contains: 20717 cells, it is 80.0 % of input dataset
Validation dataset contains: 5180 cells, it is 20.0 % of input dataset

Unsupervised pretraining: 100%|████████████████| 50/50 [04:09<00:00,  4.98s/it]
Training scAdam model:  10%|█▊                | 20/200 [01:31<13:39,  4.56s/it]

Early stopping triggered! Best score: 0.9943
Training completed!
Fitting unknown cells detector



UnknownCellDetector fitted successfully!
Model saved to snRNAseq_human_retina/Human_Retina_scAdam

Evaluation of model quality#

For model evaluation, we use another subset of 25,000 cells generated using a different random state.

[14]:

# Get test dataset for model quality evaluation
adata_test = scparadise.scnoah.get_frac(
    path = '2e910e62-7eaf-4c06-80cb-8918e3eea16e.h5ad',
    fraction = 25000,
    stratify = 'cell_type',
    random_state = 42
)

[15]:

# Check common cells between test and training datasets
lst_train = adata_fraction.obs_names.tolist()
lst_test = adata_test.obs_names.tolist()
lst_train.extend(lst_test)
lst_train = np.unique(lst_train)
percent = round((2 * len(lst_test) - len(lst_train))/len(lst_test)*100, 5)
print(f"There are {percent} % common cells ({2 * len(lst_test) - len(lst_train)} cells) between the test and training datasets")

There are 0.836 % common cells (209 cells) between the test and training datasets

Fewer than 1% of cells overlap between the training and test datasets. Although this overlap is minimal and is unlikely to materially affect the evaluation, it should be acknowledged when interpreting the model’s performance.

[16]:

# Apply the same preprocessing steps to the test dataset as used for training
# Get raw counts from adata_fraction.raw
adata_test = adata_test.raw.to_adata()

# Replace variable names with gene names
adata_test.var.set_index('feature_name', inplace = True)
adata_test.var_names = adata_test.var_names.astype('str')
adata_test.var_names_make_unique()

# Normalize data
sc.pp.normalize_total(adata_test, target_sum = None)
sc.pp.log1p(adata_test)
adata_test.raw = adata_test

[17]:

# Predict cell types using trained model
adata_test = scparadise.scadam.predict(
    adata_test,
    path_model = 'snRNAseq_human_retina/Human_Retina_scAdam'
)

Cell type annotation using the 2nd version of the scAdam model
scAdam model with unknown detector loaded from snRNAseq_human_retina/Human_Retina_scAdam
Gene alignment:
  Model features: 595
  Matched features: 595 (100.0%)

Predicting: 100%|██████████████████████████████| 98/98 [00:01<00:00, 69.80it/s]

Added cell type column: pred_celltype_l1
Added probabilities column: pred_celltype_l1_probability
Added cell type column: pred_celltype_l2
Added probabilities column: pred_celltype_l2_probability

[18]:

## Check model quality
df_l1 = scparadise.scnoah.report_classif_full(
    adata_test,
    celltype = 'majorclass',
    pred_celltype = 'pred_celltype_l1'
)
df_l1

[18]:
precision recall/sensitivity specificity f1-score geometric mean index balanced accuracy number of cells
AC 0.9991 0.9996 0.9998 0.9993 0.9997 0.9993 4496
Astrocyte 1.0000 0.982 1.0 0.9909 0.991 0.9802 111
BC 0.9998 0.9991 0.9999 0.9994 0.9995 0.9989 5437
Cone 1.0000 0.999 1.0 0.9995 0.9995 0.9989 1000
HC 0.9969 1.0 0.9999 0.9984 1.0 0.9999 634
MG 1.0000 0.9994 1.0 0.9997 0.9997 0.9994 1744
Microglia 1.0000 0.9487 1.0 0.9737 0.974 0.9439 39
RGC 0.9987 1.0 0.9998 0.9994 0.9999 0.9998 3144
RPE 1.0000 1.0 1.0 1.0 1.0 1.0 7
Rod 0.9995 0.9998 0.9998 0.9996 0.9998 0.9995 8388
macro avg 0.9994 0.9928 0.9999 0.996 0.9963 0.992
weighted avg 0.9994 0.9994 0.9998 0.9994 0.9996 0.9992
Accuracy 0.9994
Balanced accuracy 0.9928 
[19]:

## Check model quality
df_l2 = scparadise.scnoah.report_classif_full(
    adata_test,
    celltype = 'cell_type',
    pred_celltype = 'pred_celltype_l2'
)
df_l2

[19]:
precision recall/sensitivity specificity f1-score geometric mean index balanced accuracy number of cells
GABAergic amacrine cell 0.9916 0.9982 0.9989 0.9949 0.9986 0.9971 2855
H1 horizontal cell 0.9836 0.9981 0.9996 0.9908 0.9989 0.9976 540
H2 horizontal cell 0.9888 0.9362 1.0 0.9617 0.9675 0.9302 94
Mueller cell 1.0000 0.9994 1.0 0.9997 0.9997 0.9994 1744
OFF midget ganglion cell 0.9343 0.9112 0.9957 0.9226 0.9525 0.8996 1577
OFF parasol ganglion cell 0.9744 0.962 0.9999 0.9682 0.9808 0.9583 79
OFFx cell 1.0000 0.9754 1.0 0.9876 0.9876 0.973 122
ON midget ganglion cell 0.9411 0.9103 0.9971 0.9254 0.9527 0.8998 1193
ON parasol ganglion cell 0.9796 0.9796 1.0 0.9796 0.9897 0.9776 49
ON-blue cone bipolar cell 0.9524 0.8696 1.0 0.9091 0.9325 0.8582 23
S cone cell 0.9710 1.0 0.9999 0.9853 1.0 0.9999 67
amacrine cell 0.9907 0.9489 0.9998 0.9694 0.974 0.9439 450
astrocyte 1.0000 0.982 1.0 0.9909 0.991 0.9802 111
diffuse bipolar 1 cell 0.9875 0.9975 0.9998 0.9925 0.9986 0.997 397
diffuse bipolar 2 cell 0.9962 0.9869 0.9999 0.9916 0.9934 0.9856 536
diffuse bipolar 3a cell 1.0000 0.9942 1.0 0.9971 0.9971 0.9936 172
diffuse bipolar 3b cell 0.9964 0.9964 1.0 0.9964 0.9982 0.996 278
diffuse bipolar 4 cell 0.9948 0.9948 0.9999 0.9948 0.9974 0.9942 387
diffuse bipolar 6 cell 0.9932 0.9932 1.0 0.9932 0.9965 0.9924 146
flat midget bipolar cell 0.9912 0.9974 0.9996 0.9943 0.9985 0.9967 1134
giant bipolar cell 0.9899 0.9899 0.9999 0.9899 0.9949 0.9889 199
glycinergic amacrine cell 0.9836 0.9874 0.9993 0.9855 0.9933 0.9855 1032
invaginating midget bipolar cell 1.0000 0.9976 1.0 0.9988 0.9988 0.9974 835
microglial cell 1.0000 0.9231 1.0 0.96 0.9608 0.916 39
retinal bipolar neuron 0.9903 0.9951 0.9998 0.9927 0.9975 0.9945 412
retinal cone cell 1.0000 0.9957 1.0 0.9979 0.9979 0.9953 933
retinal ganglion cell 0.4695 0.626 0.993 0.5366 0.7884 0.5988 246
retinal pigment epithelial cell 1.0000 1.0 1.0 1.0 1.0 1.0 7
retinal rod cell 0.9995 0.9999 0.9998 0.9997 0.9998 0.9997 8388
rod bipolar cell 0.9987 0.9962 1.0 0.9975 0.9981 0.9958 796
starburst amacrine cell 1.0000 0.9811 1.0 0.9905 0.9905 0.9793 159
macro avg 0.9709 0.9653 0.9994 0.9676 0.9815 0.962
weighted avg 0.9841 0.9826 0.9992 0.9832 0.9906 0.9804
Accuracy 0.9826
Balanced accuracy 0.9653

The model performs well except for the retinal ganglion cell.

You could try using a different random state to generate another test dataset.

Iterative warm start training (optional)#

You may use another subset of the whole dataset to increase model generalization.

[20]:

# Do not set the lower bound of the range to 0, which was used for the primary training of the model
for i in range(1, 4):
    # Obtain 25000 cells randomly
    adata_fraction = scparadise.scnoah.get_frac(
        path = '2e910e62-7eaf-4c06-80cb-8918e3eea16e.h5ad',
        fraction = 25000,
        stratify = 'cell_type',
        random_state = i
    )

    # Get raw counts from adata_fraction.raw
    adata_fraction = adata_fraction.raw.to_adata()

    # Replace variable names with gene names
    adata_fraction.var.set_index('feature_name', inplace=True)
    adata_fraction.var_names = adata_fraction.var_names.astype('str')
    adata_fraction.var_names_make_unique()

    # Normalize data
    sc.pp.normalize_total(adata_fraction, target_sum = None)
    sc.pp.log1p(adata_fraction)
    adata_fraction.raw = adata_fraction

    # Subset genes for model training
    adata_fraction = adata_fraction[:, lst_genes]
    # Balance dataset
    adata_balanced = scparadise.scnoah.balance(
        adata_fraction,
        celltype_keys = lst_annotations
    )
    adata_balanced.raw = adata_balanced
    # Warm start requires second training dataset and path to pretrained model
    scparadise.scadam.warm_start(
        adata_balanced,
        path = 'snRNAseq_human_retina', # path to save model
        path_model = 'snRNAseq_human_retina/Human_Retina_scAdam', # folder name with pretrained model
        model_name = 'Human_Retina_scAdam',
        celltype_keys = lst_annotations,
        eval_metric = ['accuracy', 'balanced_accuracy']
    )

scAdam model with unknown detector loaded from snRNAseq_human_retina/Human_Retina_scAdam
Device: cuda
Gene alignment:
 Number of features: 595
 Matched features: 595 (100.0%)
Label hierarchy: majorclass → cell_type
Annotation levels weights using strategy 'linear_offset':
  majorclass: 10 cell types, 0.4 relative weight
  cell_type: 31 cell types, 0.6 relative weight

Dataset split:
Train dataset contains: 20717 cells, it is 80.0 % of input dataset
Validation dataset contains: 5180 cells, it is 20.0 % of input dataset

Warm-start model fine-tuning:  29%|███▏       | 29/100 [02:35<06:20,  5.35s/it]

Early stopping triggered! Best score: 0.9943
Fitting unknown cells detector



UnknownCellDetector fitted successfully!
Model saved to snRNAseq_human_retina/Human_Retina_scAdam
scAdam model with unknown detector loaded from snRNAseq_human_retina/Human_Retina_scAdam
Device: cuda
Gene alignment:
 Number of features: 595
 Matched features: 595 (100.0%)
Label hierarchy: majorclass → cell_type
Annotation levels weights using strategy 'linear_offset':
  majorclass: 10 cell types, 0.4 relative weight
  cell_type: 31 cell types, 0.6 relative weight

Dataset split:
Train dataset contains: 20717 cells, it is 80.0 % of input dataset
Validation dataset contains: 5180 cells, it is 20.0 % of input dataset

Warm-start model fine-tuning:  22%|██▍        | 22/100 [02:13<07:52,  6.05s/it]

Early stopping triggered! Best score: 0.9941
Fitting unknown cells detector



UnknownCellDetector fitted successfully!
Model saved to snRNAseq_human_retina/Human_Retina_scAdam
scAdam model with unknown detector loaded from snRNAseq_human_retina/Human_Retina_scAdam
Device: cuda
Gene alignment:
 Number of features: 595
 Matched features: 595 (100.0%)
Label hierarchy: majorclass → cell_type
Annotation levels weights using strategy 'linear_offset':
  majorclass: 10 cell types, 0.4 relative weight
  cell_type: 31 cell types, 0.6 relative weight

Dataset split:
Train dataset contains: 20717 cells, it is 80.0 % of input dataset
Validation dataset contains: 5180 cells, it is 20.0 % of input dataset

Warm-start model fine-tuning:  16%|█▊         | 16/100 [01:28<07:42,  5.50s/it]

Early stopping triggered! Best score: 0.9937
Fitting unknown cells detector



UnknownCellDetector fitted successfully!
Model saved to snRNAseq_human_retina/Human_Retina_scAdam

[21]:

# Predict cell types using trained model
adata_test = scparadise.scadam.predict(
    adata_test,
    path_model = 'snRNAseq_human_retina/Human_Retina_scAdam'
)

Cell type annotation using the 2nd version of the scAdam model
scAdam model with unknown detector loaded from snRNAseq_human_retina/Human_Retina_scAdam
Gene alignment:
  Model features: 595
  Matched features: 595 (100.0%)

Predicting: 100%|██████████████████████████████| 98/98 [00:01<00:00, 91.31it/s]

Added cell type column: pred_celltype_l1
Added probabilities column: pred_celltype_l1_probability
Added cell type column: pred_celltype_l2
Added probabilities column: pred_celltype_l2_probability

[22]:

## Check model quality
df_warm_start_l1 = scparadise.scnoah.report_classif_full(
    adata_test,
    celltype='majorclass',
    pred_celltype='pred_celltype_l1'
)
df_warm_start_l1

[22]:
precision recall/sensitivity specificity f1-score geometric mean index balanced accuracy number of cells
AC 0.9982 0.9996 0.9996 0.9989 0.9996 0.9992 4496
Astrocyte 1.0000 0.982 1.0 0.9909 0.991 0.9802 111
BC 0.9993 1.0 0.9998 0.9996 0.9999 0.9998 5437
Cone 1.0000 1.0 1.0 1.0 1.0 1.0 1000
HC 1.0000 1.0 1.0 1.0 1.0 1.0 634
MG 0.9989 0.9994 0.9999 0.9991 0.9997 0.9993 1744
Microglia 1.0000 1.0 1.0 1.0 1.0 1.0 39
RGC 1.0000 0.9987 1.0 0.9994 0.9994 0.9986 3144
RPE 1.0000 1.0 1.0 1.0 1.0 1.0 7
Rod 0.9999 0.9993 0.9999 0.9996 0.9996 0.9992 8388
macro avg 0.9996 0.9979 0.9999 0.9988 0.9989 0.9976
weighted avg 0.9994 0.9994 0.9999 0.9994 0.9996 0.9992
Accuracy 0.9994
Balanced accuracy 0.9979 
[23]:

## Check model quality
df_warm_start_l2 = scparadise.scnoah.report_classif_full(
    adata_test,
    celltype='cell_type',
    pred_celltype='pred_celltype_l2'
)
df_warm_start_l2

[23]:
precision recall/sensitivity specificity f1-score geometric mean index balanced accuracy number of cells
GABAergic amacrine cell 0.9951 0.9975 0.9994 0.9963 0.9985 0.9967 2855
H1 horizontal cell 0.9944 0.9944 0.9999 0.9944 0.9972 0.9938 540
H2 horizontal cell 0.9684 0.9787 0.9999 0.9735 0.9892 0.9765 94
Mueller cell 0.9994 0.9994 1.0 0.9994 0.9997 0.9993 1744
OFF midget ganglion cell 0.9263 0.9569 0.9949 0.9414 0.9757 0.9484 1577
OFF parasol ganglion cell 0.9870 0.962 1.0 0.9744 0.9808 0.9583 79
OFFx cell 0.9836 0.9836 0.9999 0.9836 0.9917 0.9819 122
ON midget ganglion cell 0.9412 0.9522 0.997 0.9467 0.9744 0.9451 1193
ON parasol ganglion cell 1.0000 0.9796 1.0 0.9897 0.9897 0.9776 49
ON-blue cone bipolar cell 0.9524 0.8696 1.0 0.9091 0.9325 0.8582 23
S cone cell 0.8933 1.0 0.9997 0.9437 0.9998 0.9997 67
amacrine cell 0.9844 0.9822 0.9997 0.9833 0.9909 0.9802 450
astrocyte 1.0000 0.982 1.0 0.9909 0.991 0.9802 111
diffuse bipolar 1 cell 0.9900 0.995 0.9998 0.9925 0.9974 0.9943 397
diffuse bipolar 2 cell 0.9944 0.9925 0.9999 0.9935 0.9962 0.9917 536
diffuse bipolar 3a cell 1.0000 1.0 1.0 1.0 1.0 1.0 172
diffuse bipolar 3b cell 0.9964 0.9928 1.0 0.9946 0.9964 0.9921 278
diffuse bipolar 4 cell 0.9949 1.0 0.9999 0.9974 1.0 0.9999 387
diffuse bipolar 6 cell 0.9931 0.9795 1.0 0.9862 0.9897 0.9774 146
flat midget bipolar cell 0.9947 0.9982 0.9997 0.9965 0.999 0.9978 1134
giant bipolar cell 0.9801 0.9899 0.9998 0.985 0.9949 0.9888 199
glycinergic amacrine cell 0.9874 0.9864 0.9995 0.9869 0.9929 0.9846 1032
invaginating midget bipolar cell 0.9988 1.0 1.0 0.9994 1.0 1.0 835
microglial cell 1.0000 1.0 1.0 1.0 1.0 1.0 39
retinal bipolar neuron 0.9976 0.9927 1.0 0.9951 0.9963 0.992 412
retinal cone cell 1.0000 0.9914 1.0 0.9957 0.9957 0.9906 933
retinal ganglion cell 0.6444 0.4715 0.9974 0.5446 0.6858 0.4456 246
retinal pigment epithelial cell 1.0000 1.0 1.0 1.0 1.0 1.0 7
retinal rod cell 0.9999 0.9994 0.9999 0.9996 0.9997 0.9993 8388
rod bipolar cell 0.9987 1.0 1.0 0.9994 1.0 1.0 796
starburst amacrine cell 0.9937 0.9874 1.0 0.9905 0.9937 0.9861 159
macro avg 0.9739 0.9682 0.9995 0.9704 0.9822 0.9657
weighted avg 0.9859 0.9866 0.9994 0.9861 0.9925 0.9851
Accuracy 0.9866
Balanced accuracy 0.9682 
[24]:

pd.set_option('display.max_rows', 100)
df_l2.compare(df_warm_start_l2, keep_equal=True, align_axis = 0, result_names=('default', 'warm start'))

[24]:
precision recall/sensitivity specificity f1-score geometric mean index balanced accuracy
GABAergic amacrine cell default 0.9916 0.9982 0.9989 0.9949 0.9986 0.9971
warm start 0.9951 0.9975 0.9994 0.9963 0.9985 0.9967
H1 horizontal cell default 0.9836 0.9981 0.9996 0.9908 0.9989 0.9976
warm start 0.9944 0.9944 0.9999 0.9944 0.9972 0.9938
H2 horizontal cell default 0.9888 0.9362 1.0 0.9617 0.9675 0.9302
warm start 0.9684 0.9787 0.9999 0.9735 0.9892 0.9765
Mueller cell default 1.0000 0.9994 1.0 0.9997 0.9997 0.9994
warm start 0.9994 0.9994 1.0 0.9994 0.9997 0.9993
OFF midget ganglion cell default 0.9343 0.9112 0.9957 0.9226 0.9525 0.8996
warm start 0.9263 0.9569 0.9949 0.9414 0.9757 0.9484
OFF parasol ganglion cell default 0.9744 0.962 0.9999 0.9682 0.9808 0.9583
warm start 0.9870 0.962 1.0 0.9744 0.9808 0.9583
OFFx cell default 1.0000 0.9754 1.0 0.9876 0.9876 0.973
warm start 0.9836 0.9836 0.9999 0.9836 0.9917 0.9819
ON midget ganglion cell default 0.9411 0.9103 0.9971 0.9254 0.9527 0.8998
warm start 0.9412 0.9522 0.997 0.9467 0.9744 0.9451
ON parasol ganglion cell default 0.9796 0.9796 1.0 0.9796 0.9897 0.9776
warm start 1.0000 0.9796 1.0 0.9897 0.9897 0.9776
S cone cell default 0.9710 1.0 0.9999 0.9853 1.0 0.9999
warm start 0.8933 1.0 0.9997 0.9437 0.9998 0.9997
amacrine cell default 0.9907 0.9489 0.9998 0.9694 0.974 0.9439
warm start 0.9844 0.9822 0.9997 0.9833 0.9909 0.9802
diffuse bipolar 1 cell default 0.9875 0.9975 0.9998 0.9925 0.9986 0.997
warm start 0.9900 0.995 0.9998 0.9925 0.9974 0.9943
diffuse bipolar 2 cell default 0.9962 0.9869 0.9999 0.9916 0.9934 0.9856
warm start 0.9944 0.9925 0.9999 0.9935 0.9962 0.9917
diffuse bipolar 3a cell default 1.0000 0.9942 1.0 0.9971 0.9971 0.9936
warm start 1.0000 1.0 1.0 1.0 1.0 1.0
diffuse bipolar 3b cell default 0.9964 0.9964 1.0 0.9964 0.9982 0.996
warm start 0.9964 0.9928 1.0 0.9946 0.9964 0.9921
diffuse bipolar 4 cell default 0.9948 0.9948 0.9999 0.9948 0.9974 0.9942
warm start 0.9949 1.0 0.9999 0.9974 1.0 0.9999
diffuse bipolar 6 cell default 0.9932 0.9932 1.0 0.9932 0.9965 0.9924
warm start 0.9931 0.9795 1.0 0.9862 0.9897 0.9774
flat midget bipolar cell default 0.9912 0.9974 0.9996 0.9943 0.9985 0.9967
warm start 0.9947 0.9982 0.9997 0.9965 0.999 0.9978
giant bipolar cell default 0.9899 0.9899 0.9999 0.9899 0.9949 0.9889
warm start 0.9801 0.9899 0.9998 0.985 0.9949 0.9888
glycinergic amacrine cell default 0.9836 0.9874 0.9993 0.9855 0.9933 0.9855
warm start 0.9874 0.9864 0.9995 0.9869 0.9929 0.9846
invaginating midget bipolar cell default 1.0000 0.9976 1.0 0.9988 0.9988 0.9974
warm start 0.9988 1.0 1.0 0.9994 1.0 1.0
microglial cell default 1.0000 0.9231 1.0 0.96 0.9608 0.916
warm start 1.0000 1.0 1.0 1.0 1.0 1.0
retinal bipolar neuron default 0.9903 0.9951 0.9998 0.9927 0.9975 0.9945
warm start 0.9976 0.9927 1.0 0.9951 0.9963 0.992
retinal cone cell default 1.0000 0.9957 1.0 0.9979 0.9979 0.9953
warm start 1.0000 0.9914 1.0 0.9957 0.9957 0.9906
retinal ganglion cell default 0.4695 0.626 0.993 0.5366 0.7884 0.5988
warm start 0.6444 0.4715 0.9974 0.5446 0.6858 0.4456
retinal rod cell default 0.9995 0.9999 0.9998 0.9997 0.9998 0.9997
warm start 0.9999 0.9994 0.9999 0.9996 0.9997 0.9993
rod bipolar cell default 0.9987 0.9962 1.0 0.9975 0.9981 0.9958
warm start 0.9987 1.0 1.0 0.9994 1.0 1.0
starburst amacrine cell default 1.0000 0.9811 1.0 0.9905 0.9905 0.9793
warm start 0.9937 0.9874 1.0 0.9905 0.9937 0.9861
macro avg default 0.9709 0.9653 0.9994 0.9676 0.9815 0.962
warm start 0.9739 0.9682 0.9995 0.9704 0.9822 0.9657
weighted avg default 0.9841 0.9826 0.9992 0.9832 0.9906 0.9804
warm start 0.9859 0.9866 0.9994 0.9861 0.9925 0.9851
Accuracy default 0.9826
warm start 0.9866
Balanced accuracy default 0.9653
warm start 0.9682

Iterative warm start training led to an increase in all model quality metrics (rows - macro average, weighted average, accuracy, and balanced accuracy).

[25]:

import session_info2
session_info2.session_info()

[25]:
PackageVersion
pandas2.3.3
scanpy1.11.5
scparadise1.2.0
numpy2.4.6
anndata0.12.16
ComponentInfo
Python3.11.15 (main, Jun 11 2026, 15:20:16) [GCC 14.3.0]
OSLinux-6.8.0-138-generic-x86_64-with-glibc2.39
Updated2026-09-04 00:04
Dependencies
DependencyVersion
stack-data0.6.3
tqdm4.68.2
packaging26.0
numcodecs0.16.5
prompt_toolkit3.0.52
ipykernel7.3.0
wcwidth0.8.1
fonttools4.63.0
h5py3.16.0
statsmodels0.14.6
pillow12.2.0
shap0.49.1
python-dateutil2.9.0.post0
typing_extensions4.15.0
jupyter_client8.9.1
comm0.2.3
pyzmq27.1.0
donfig0.8.1.post1
ipython9.14.1
scikit-learn1.9.0
mpmath1.3.0
cloudpickle3.1.2
sparse0.18.0
pure_eval0.2.3
rich15.0.0
triton3.7.0
typing3.7.4.3
executing2.2.1
imbalanced-learn0.14.2
debugpy1.8.21
parso0.8.7
Pygments2.20.0
cuda-bindings13.3.1
asttokens3.0.1
matplotlib3.11.0
natsort8.4.0
pytorch-tabnet4.1.0
leidenalg0.12.0
PyYAML6.0.3
scikit-image0.26.0
decorator5.3.1
charset-normalizer3.4.7
muon0.1.7
fsspec2026.4.0
kiwisolver1.5.0
optuna4.9.0
cffi2.0.0
cuda-pathfinder1.5.5
torch2.12.0 (2.12.0+cu130)
legacy-api-wrap1.5
dataclasses0.6
session-info20.4.1
sklearn-compat0.1.6
numba0.65.1
pynndescent0.6.0
llvmlite0.47.0
jedi0.20.0
six1.17.0
lazy-loader0.5
pyarrow24.0.0
pyparsing3.3.2
google-crc32c1.8.0
igraph1.0.0
threadpoolctl3.6.0
scverse-misc0.0.3
jupyter_core5.9.1
plottable0.1.5
cycler0.12.1
Cython3.3.0
zarr3.1.6
sympy1.14.0
colorlog6.10.1
patsy1.0.2
psutil7.2.2
opt_einsum3.4.0
pycparser3.0 (3.00)
joblib1.5.3
umap-learn0.5.12
xarray2026.4.0
pathlib1.0.1
tornado6.5.7
platformdirs4.10.0
defusedxml0.7.1
mudata0.3.8
matplotlib-inline0.2.2
argparse1.4.0 (1.1)
texttable1.7.0
pytz2026.2
seaborn0.13.2
narwhals2.22.1
setuptools81.0.0
traitlets5.15.1
scipy1.17.1
slicer0.0.8
Copyable Markdown 
| Package    | Version |
| ---------- | ------- |
| pandas     | 2.3.3   |
| scanpy     | 1.11.5  |
| scparadise | 1.2.0   |
| numpy      | 2.4.6   |
| anndata    | 0.12.16 |

| Dependency         | Version               |
| ------------------ | --------------------- |
| stack-data         | 0.6.3                 |
| tqdm               | 4.68.2                |
| packaging          | 26.0                  |
| numcodecs          | 0.16.5                |
| prompt_toolkit     | 3.0.52                |
| ipykernel          | 7.3.0                 |
| wcwidth            | 0.8.1                 |
| fonttools          | 4.63.0                |
| h5py               | 3.16.0                |
| statsmodels        | 0.14.6                |
| pillow             | 12.2.0                |
| shap               | 0.49.1                |
| python-dateutil    | 2.9.0.post0           |
| typing_extensions  | 4.15.0                |
| jupyter_client     | 8.9.1                 |
| comm               | 0.2.3                 |
| pyzmq              | 27.1.0                |
| donfig             | 0.8.1.post1           |
| ipython            | 9.14.1                |
| scikit-learn       | 1.9.0                 |
| mpmath             | 1.3.0                 |
| cloudpickle        | 3.1.2                 |
| sparse             | 0.18.0                |
| pure_eval          | 0.2.3                 |
| rich               | 15.0.0                |
| triton             | 3.7.0                 |
| typing             | 3.7.4.3               |
| executing          | 2.2.1                 |
| imbalanced-learn   | 0.14.2                |
| debugpy            | 1.8.21                |
| parso              | 0.8.7                 |
| Pygments           | 2.20.0                |
| cuda-bindings      | 13.3.1                |
| asttokens          | 3.0.1                 |
| matplotlib         | 3.11.0                |
| natsort            | 8.4.0                 |
| pytorch-tabnet     | 4.1.0                 |
| leidenalg          | 0.12.0                |
| PyYAML             | 6.0.3                 |
| scikit-image       | 0.26.0                |
| decorator          | 5.3.1                 |
| charset-normalizer | 3.4.7                 |
| muon               | 0.1.7                 |
| fsspec             | 2026.4.0              |
| kiwisolver         | 1.5.0                 |
| optuna             | 4.9.0                 |
| cffi               | 2.0.0                 |
| cuda-pathfinder    | 1.5.5                 |
| torch              | 2.12.0 (2.12.0+cu130) |
| legacy-api-wrap    | 1.5                   |
| dataclasses        | 0.6                   |
| session-info2      | 0.4.1                 |
| sklearn-compat     | 0.1.6                 |
| numba              | 0.65.1                |
| pynndescent        | 0.6.0                 |
| llvmlite           | 0.47.0                |
| jedi               | 0.20.0                |
| six                | 1.17.0                |
| lazy-loader        | 0.5                   |
| pyarrow            | 24.0.0                |
| pyparsing          | 3.3.2                 |
| google-crc32c      | 1.8.0                 |
| igraph             | 1.0.0                 |
| threadpoolctl      | 3.6.0                 |
| scverse-misc       | 0.0.3                 |
| jupyter_core       | 5.9.1                 |
| plottable          | 0.1.5                 |
| cycler             | 0.12.1                |
| Cython             | 3.3.0                 |
| zarr               | 3.1.6                 |
| sympy              | 1.14.0                |
| colorlog           | 6.10.1                |
| patsy              | 1.0.2                 |
| psutil             | 7.2.2                 |
| opt_einsum         | 3.4.0                 |
| pycparser          | 3.0 (3.00)            |
| joblib             | 1.5.3                 |
| umap-learn         | 0.5.12                |
| xarray             | 2026.4.0              |
| pathlib            | 1.0.1                 |
| tornado            | 6.5.7                 |
| platformdirs       | 4.10.0                |
| defusedxml         | 0.7.1                 |
| mudata             | 0.3.8                 |
| matplotlib-inline  | 0.2.2                 |
| argparse           | 1.4.0 (1.1)           |
| texttable          | 1.7.0                 |
| pytz               | 2026.2                |
| seaborn            | 0.13.2                |
| narwhals           | 2.22.1                |
| setuptools         | 81.0.0                |
| traitlets          | 5.15.1                |
| scipy              | 1.17.1                |
| slicer             | 0.0.8                 |

| Component | Info                                               |
| --------- | -------------------------------------------------- |
| Python    | 3.11.15 (main, Jun 11 2026, 15:20:16) [GCC 14.3.0] |
| OS        | Linux-6.8.0-138-generic-x86_64-with-glibc2.39      |
| Updated   | 2026-09-04 00:04                                   |