scEve model training

scEve model training#

[1]:

# Python packages
import warnings
warnings.simplefilter('ignore')

import scanpy as sc
import scparadise
import muon as mu
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt

sc.set_figure_params(dpi = 120)

Recommendations about training dataset#

We recommend the shifted logarithm data normalization method for gene expression data:

sc.pp.normalize_total(adata, target_sum=None)
sc.pp.log1p(adata)

However, you can use any other method of gene expression data normalization (use the same normalization method for the test dataset).

The training dataset should contain all genes that you want to use for model training in mdata_train.mod['rna'].X.

We recommend removing all non-marker genes from mdata_train.mod['rna'].X (removing such useless genes may improve performance and model quality metrics).

To demonstrate scEve model training and evaluation, we used eight samples from a CITE-seq 3’ PBMC dataset. The data was normalized using the shifted logarithm algorithm, and genes were pre-sorted for model training.

[2]:

# Download data from figshare
# You can use wget, or download the file in a browser by copying and pasting the link into a new tab.
link = 'https://figshare.com/ndownloader/files/51749906'

[3]:

# Load prepared for training anndata object
mdata = mu.read_h5mu('mdata_pbmc.h5mu')

[4]:

# Create mdata_train dataset (we used 4 samples for training model)
mdata_train = mdata[mdata.obs['orig.ident'].isin(['P2_0', 'P3_0', 'P4_0', 'P5_0', 'P6_0', 'P7_0', 'P8_0'])].copy()
# Create mdata_test dataset (we used 1 sample for test model quality)
mdata_test = mdata[mdata.obs['orig.ident'].isin(['P1_0'])].copy()

[5]:

# Extract RNA expression data from mdata_test for future predictions
adata_test_rna = mdata_test.mod['rna']

Default scEve model training#

[6]:

# Train default scEve model using mdata_train dataset
scparadise.sceve.train(
    mdata_train,
    path = '',
    first_modality_name = 'rna',
    second_modality_name = 'adt',
    detailed_annotation = 'celltype_l3',
    model_name = 'model_scEve',
    epochs = 200,
    eval_metric = ['rmse']
)

Device: cuda
Task: Predict 'adt' from 'rna'
Number of first modality features: 635
Number of second modality features: 224

Dataset split:
Train dataset contains: 37359 cells, it is 80.0 % of input dataset
Validation dataset contains: 9340 cells, it is 20.0 % of input dataset

Training scEve model:  22%|████               | 43/200 [03:35<13:07,  5.02s/it]

Early stopping triggered! Best score: 0.3039
Training completed!
Model saved to model_scEve

Check model quality#

[7]:

# Predict surface proteins using scEve model
mdata_pred = scparadise.sceve.predict(
    adata_test_rna,
    path_model = 'model_scEve',
    return_mdata = True # default
)

Feature prediction using the 2nd version of the scEve model
scEve model loaded from model_scEve
Feature alignment:
Model features: 635
New data features: 635
Matched features: 635 (100.0%)

Predicting: 100%|█████████████████████████████| 25/25 [00:00<00:00, 475.21it/s]

[8]:

# Add umap coordinates
mdata_pred.obsm['X_umap'] = adata_test_rna.obsm['X_umap']

[9]:

# List of proteins for visualization
proteins_observed = ['adt_CLEC12A', 'adt_CD4-1', 'adt_CD8', 'adt_IgM', 'adt_CD16']

# Create list of imputed proteins
proteins_imputed = []
for i in proteins_observed:
    i = i + '_pred'
    proteins_imputed.append(i)

[10]:

# Proteins visualization
proteins_dict = {
    "Observed": proteins_observed,
    "Imputed": proteins_imputed
}
# Choose a color map for protein abundance visualization
cmap = 'bwr'
# Make Axes
# Number of needed rows and columns (based on the row with the most columns)
nrow = len(proteins_dict)
ncol = max([len(vs) for vs in proteins_dict.values()])
fig, axs = plt.subplots(nrow, ncol, figsize=(3 * ncol, 3 * nrow))
# Plot expression for every marker on the corresponding Axes object
for row_idx, (prots, markers) in enumerate(proteins_dict.items()):
    col_idx = 0
    for marker in markers:
        ax = axs[row_idx, col_idx]
        if prots == 'Observed':
            mu.pl.umap(mdata_test, color=marker, cmap = cmap, ax=ax, vmax = 'p99', show=False, frameon=False)
        else:
            mu.pl.umap(mdata_pred, color=marker, cmap = cmap, ax=ax, vmax = 'p99', show=False, frameon=False)
        # Add cell type as row label - here we simply add it as ylabel of
        # the first Axes object in the row
        if col_idx == 0:
            # We disabled axis drawing in UMAP to have plots without background and border
            # so we need to re-enable axis to plot the ylabel
            ax.axis("on")
            ax.tick_params(
                top="off",
                bottom="off",
                left="off",
                right="off",
                labelleft="on",
                labelbottom="off",
            )
            ax.set_ylabel(prots + "\n", rotation=90, fontsize=14)
            ax.set(frame_on=False)
        col_idx += 1
    # Remove unused column Axes in the current row
    while col_idx < ncol:
        axs[row_idx, col_idx].remove()
        col_idx += 1
# Alignment within the Figure
fig.tight_layout()
../../../_images/tutorials_notebooks_scEve_scEve_train_14_0.png

The obtained model reliably predicts the main protein markers. However, there are proteins such as CD133, which the default model cannot predict.

[11]:

# Check prediction results
df = scparadise.scnoah.report_reg(adata_prot = mdata_test.mod['adt'],
                                  adata_pred_prot = mdata_pred.mod['adt'])
df

[11]:
EVS r2_score RMSE MedianAE MeanAE
score 0.281 0.235 0.328 0.201 0.248
EVS/r2_score higher value - better prediction
RMSE/MedianAE/MeanAE lower value - better prediction 
[13]:

# Add regression status using 5 metrics
for metric in ['RMSE', 'MeanAE', 'MedianAE', 'EVS', 'r2_score']:
    scparadise.scnoah.regres_status(adata_prot = mdata_test.mod['adt'],
                                    adata_pred_prot = mdata_pred.mod['adt'],
                                    metric=metric)
    mdata_pred.obs[f'regres_status_{metric}'] = mdata_pred.mod['adt'].obs[f'regres_status_{metric}']

[14]:

# Visualize metrics
mu.pl.umap(
    mdata_pred,
    color=[
        'regres_status_RMSE',
        'regres_status_MeanAE',
        'regres_status_MedianAE',
        'regres_status_EVS',
        'regres_status_r2_score'
    ],
    frameon = False,
    cmap = 'bwr',
    ncols = 5,
    wspace = 0.15,
    hspace = 0.1
)
../../../_images/tutorials_notebooks_scEve_scEve_train_18_0.png

Based on a per cell type regression metrics score, it is possible to draw conclusions about the uniformity of protein prediction across different cell types in the dataset.

[16]:

# Find correlations between imputed and observed proteins
df_corr = pd.DataFrame(columns=['Pearson coef', 'p-value'])
for i in mdata_test.mod['adt'].var_names.tolist():
    person_coef = scparadise.scnoah.pearson_coef(
        adata = mdata_test.mod['adt'],
        adata_pred = mdata_pred.mod['adt'],
        feature = i,
        feature_pred = i + '_pred',
        print_res = False
    )
    df_corr.loc[i] = [person_coef['Pearson coefficient'], person_coef['p-value']]
df_corr['p-value'] = df_corr['p-value'].astype('float64')
df_corr.sort_values('Pearson coef', ascending=False).head(5)
# p value = 0 means p-value < 0.0005

[16]:
Pearson coef p-value
adt_CLEC12A 0.963 0.0
adt_CD64 0.943 0.0
adt_CD11b-2 0.927 0.0
adt_CD8a 0.921 0.0
adt_CD4-1 0.919 0.0 
[17]:

# Find highly correlated protein 'Pearson coef' > 0.5
df_high_corr = df_corr[df_corr['Pearson coef'] >= 0.5].copy()
print('Number of proteins with Pearson coefficient >= 0.5:', len(df_high_corr))
df_high_corr.sort_values(by = 'Pearson coef', ascending=False).head(5)
# p value = 0 means p-value < 0.0005

Number of proteins with Pearson coefficient >= 0.5: 107

[17]:
Pearson coef p-value
adt_CLEC12A 0.963 0.0
adt_CD64 0.943 0.0
adt_CD11b-2 0.927 0.0
adt_CD8a 0.921 0.0
adt_CD4-1 0.919 0.0 
[18]:

# Save mdata with imputed proteins
mdata_pred.write_h5mu('mdata_pred_untuned.h5mu')

Hyperparameters tuning#

[19]:

scparadise.sceve.hyperparameter_tuning(
    mdata_train,
    path = '',
    first_modality_name = 'rna',
    second_modality_name = 'adt',
    detailed_annotation = 'celltype_l3',
    random_state = 42,
    model_name = 'model_scEve_tuning',
    eval_metric = 'rmse'
)

Device: cuda
Task: Predict 'adt' from 'rna'
Start model optimization using optuna...

[I 2026-09-03 10:37:57,873] A new study created in RDB with name: study

Fold 1 finished with rmse value = 0.305239
Fold 2 finished with rmse value = 0.305161
Fold 3 finished with rmse value = 0.302352
Fold 4 finished with rmse value = 0.303180
Fold 5 finished with rmse value = 0.302680

[I 2026-09-03 10:50:38,154] Trial 0 finished with value: 0.30372245270194725 and parameters: {'nc': 4, 'nb': 4, 'nh': 8, 'ed_nh_ratio': 32, 'ff_hd': 512, 'regressor_hd': 512, 'dropout': 0.1, 'lr': 0.0001, 'weight_decay': 0.0001, 'batch_size': 128, 'patience': 10, 'epochs': 200, 'use_augmentation': True, 'aug_probability': 0.5, 'prob': 0.15, 'noise_std': 0.1, 'dropout_aug': 0.1, 'alpha': 0.2}. Best is trial 0 with value: 0.30372245270194725.

Fold 1 finished with rmse value = 0.306941
Fold 2 finished with rmse value = 0.306451
Fold 3 finished with rmse value = 0.303622
Fold 4 finished with rmse value = 0.304643
Fold 5 finished with rmse value = 0.304347

[I 2026-09-03 10:53:16,116] Trial 1 finished with value: 0.3052006037701322 and parameters: {'nc': 6, 'nb': 5, 'nh': 6, 'ed_nh_ratio': 24, 'ff_hd': 1024, 'regressor_hd': 768, 'dropout': 0.16261504812838123, 'lr': 0.0003413272677329602, 'weight_decay': 0.00027511778112565614, 'batch_size': 1216, 'patience': 5, 'epochs': 150, 'use_augmentation': False, 'aug_probability': 0.38900714995749675, 'prob': 0.17539395895211118, 'noise_std': 0.019164036607161174, 'dropout_aug': 0.20444498288214497, 'alpha': 0.06383318214753718}. Best is trial 0 with value: 0.30372245270194725.

Fold 1 finished with rmse value = 0.345096
Fold 2 finished with rmse value = 0.344896
Fold 3 finished with rmse value = 0.346117
Fold 4 finished with rmse value = 0.344616
Fold 5 finished with rmse value = 0.344706

[I 2026-09-03 10:58:53,966] Trial 2 finished with value: 0.34508622190591975 and parameters: {'nc': 2, 'nb': 4, 'nh': 12, 'ed_nh_ratio': 16, 'ff_hd': 640, 'regressor_hd': 256, 'dropout': 0.19722932758539358, 'lr': 2.3851912291055947e-05, 'weight_decay': 0.00013314642372245125, 'batch_size': 1216, 'patience': 20, 'epochs': 75, 'use_augmentation': False, 'aug_probability': 0.6534602202853893, 'prob': 0.06120077486743636, 'noise_std': 0.2557582256745621, 'dropout_aug': 0.11852685845144416, 'alpha': 0.09464504609153623}. Best is trial 0 with value: 0.30372245270194725.

Fold 1 finished with rmse value = 0.306550
Fold 2 finished with rmse value = 0.310812
Fold 3 finished with rmse value = 0.304758
Fold 4 finished with rmse value = 0.308269
Fold 5 finished with rmse value = 0.305890

[I 2026-09-03 11:05:15,341] Trial 3 finished with value: 0.30725583727369 and parameters: {'nc': 2, 'nb': 5, 'nh': 8, 'ed_nh_ratio': 32, 'ff_hd': 384, 'regressor_hd': 384, 'dropout': 0.009248468571192948, 'lr': 0.009350872702638945, 'weight_decay': 0.00022181298243722608, 'batch_size': 1536, 'patience': 10, 'epochs': 95, 'use_augmentation': True, 'aug_probability': 0.9531995144700973, 'prob': 0.1907838238471088, 'noise_std': 0.14650124120104868, 'dropout_aug': 0.08930863499330144, 'alpha': 0.37548415875396984}. Best is trial 0 with value: 0.30372245270194725.

Fold 1 finished with rmse value = 0.339072

[I 2026-09-03 11:07:22,709] Trial 4 pruned.

Fold 1 finished with rmse value = 0.304088
Fold 2 finished with rmse value = 0.303920
Fold 3 finished with rmse value = 0.301742
Fold 4 finished with rmse value = 0.303156
Fold 5 finished with rmse value = 0.302213

[I 2026-09-03 11:14:45,087] Trial 5 finished with value: 0.3030238074092573 and parameters: {'nc': 8, 'nb': 6, 'nh': 4, 'ed_nh_ratio': 24, 'ff_hd': 128, 'regressor_hd': 640, 'dropout': 0.048886293190760856, 'lr': 0.004522802709008413, 'weight_decay': 0.0033345291249155017, 'batch_size': 960, 'patience': 20, 'epochs': 90, 'use_augmentation': True, 'aug_probability': 0.23600762203522965, 'prob': 0.3178312575625041, 'noise_std': 0.2128503266505688, 'dropout_aug': 0.26215196357868437, 'alpha': 0.3805954203306821}. Best is trial 5 with value: 0.3030238074092573.

Fold 1 finished with rmse value = 0.303914
Fold 2 finished with rmse value = 0.303244
Fold 3 finished with rmse value = 0.301805

[I 2026-09-03 11:17:44,561] Trial 6 pruned.

Fold 1 finished with rmse value = 0.316300

[I 2026-09-03 11:19:06,573] Trial 7 pruned.

Fold 1 finished with rmse value = 0.308581
Fold 2 finished with rmse value = 0.308702
Fold 3 finished with rmse value = 0.306235

[I 2026-09-03 11:21:56,205] Trial 8 pruned.

Fold 1 finished with rmse value = 0.311169

[I 2026-09-03 11:24:11,565] Trial 9 pruned.

Fold 1 finished with rmse value = 0.301786
Fold 2 finished with rmse value = 0.302252
Fold 3 finished with rmse value = 0.301285
Fold 4 finished with rmse value = 0.300736
Fold 5 finished with rmse value = 0.301339

[I 2026-09-03 12:48:23,838] Trial 10 finished with value: 0.3014799039404055 and parameters: {'nc': 16, 'nb': 8, 'nh': 16, 'ed_nh_ratio': 20, 'ff_hd': 128, 'regressor_hd': 640, 'dropout': 0.07464756430317242, 'lr': 0.0017562488910721954, 'weight_decay': 1.0582826098878695e-06, 'batch_size': 64, 'patience': 30, 'epochs': 115, 'use_augmentation': True, 'aug_probability': 0.13145570409998097, 'prob': 0.3945851261313719, 'noise_std': 0.36262539030994084, 'dropout_aug': 0.2781848960017646, 'alpha': 0.1820954250443647}. Best is trial 10 with value: 0.3014799039404055.

Fold 1 finished with rmse value = 0.302501
Fold 2 finished with rmse value = 0.303114
Fold 3 finished with rmse value = 0.300879
Fold 4 finished with rmse value = 0.300781
Fold 5 finished with rmse value = 0.300893

[I 2026-09-03 13:12:06,926] Trial 11 finished with value: 0.30163362870970806 and parameters: {'nc': 16, 'nb': 8, 'nh': 16, 'ed_nh_ratio': 20, 'ff_hd': 128, 'regressor_hd': 640, 'dropout': 0.07642291089376226, 'lr': 0.0019867524580957363, 'weight_decay': 2.7855521664791902e-06, 'batch_size': 256, 'patience': 30, 'epochs': 115, 'use_augmentation': True, 'aug_probability': 0.12781481849677886, 'prob': 0.3757608517599708, 'noise_std': 0.3858136105471351, 'dropout_aug': 0.2745932974085905, 'alpha': 0.25194943991820407}. Best is trial 10 with value: 0.3014799039404055.

Fold 1 finished with rmse value = 0.301904
Fold 2 finished with rmse value = 0.301933
Fold 3 finished with rmse value = 0.299808
Fold 4 finished with rmse value = 0.301285
Fold 5 finished with rmse value = 0.299771

[I 2026-09-03 14:48:56,149] Trial 12 finished with value: 0.300940450303323 and parameters: {'nc': 16, 'nb': 8, 'nh': 16, 'ed_nh_ratio': 20, 'ff_hd': 256, 'regressor_hd': 768, 'dropout': 0.07656298702590689, 'lr': 0.001657541174330011, 'weight_decay': 1.2763368710067887e-06, 'batch_size': 64, 'patience': 30, 'epochs': 125, 'use_augmentation': True, 'aug_probability': 0.10857778631741224, 'prob': 0.3728826382054171, 'noise_std': 0.39000574118175874, 'dropout_aug': 0.29411542704545374, 'alpha': 0.20505879700620355}. Best is trial 12 with value: 0.300940450303323.

Fold 1 finished with rmse value = 0.302976
Fold 2 finished with rmse value = 0.304298
Fold 3 finished with rmse value = 0.302625

[I 2026-09-03 14:59:41,368] Trial 13 pruned.

Fold 1 finished with rmse value = 0.301384
Fold 2 finished with rmse value = 0.302799
Fold 3 finished with rmse value = 0.300710
Fold 4 finished with rmse value = 0.301081
Fold 5 finished with rmse value = 0.300312

[I 2026-09-03 16:13:34,116] Trial 14 finished with value: 0.3012571776597419 and parameters: {'nc': 14, 'nb': 7, 'nh': 12, 'ed_nh_ratio': 20, 'ff_hd': 384, 'regressor_hd': 768, 'dropout': 0.10057257134589706, 'lr': 0.0014548098010617146, 'weight_decay': 1.2998497928831808e-05, 'batch_size': 64, 'patience': 25, 'epochs': 125, 'use_augmentation': True, 'aug_probability': 0.2832395437453594, 'prob': 0.3322680522123851, 'noise_std': 0.3287538718410844, 'dropout_aug': 0.21987441005755187, 'alpha': 0.1806193952522241}. Best is trial 12 with value: 0.300940450303323.

Fold 1 finished with rmse value = 0.305041
Fold 2 finished with rmse value = 0.304276
Fold 3 finished with rmse value = 0.303088

[I 2026-09-03 16:19:22,597] Trial 15 pruned.

Fold 1 finished with rmse value = 0.302454
Fold 2 finished with rmse value = 0.302762
Fold 3 finished with rmse value = 0.299947
Fold 4 finished with rmse value = 0.301122
Fold 5 finished with rmse value = 0.301587

[I 2026-09-03 16:41:26,911] Trial 16 finished with value: 0.30157417207726267 and parameters: {'nc': 14, 'nb': 7, 'nh': 12, 'ed_nh_ratio': 16, 'ff_hd': 512, 'regressor_hd': 768, 'dropout': 0.10602680892211101, 'lr': 0.0032919810536995273, 'weight_decay': 1.911594588426516e-05, 'batch_size': 384, 'patience': 25, 'epochs': 100, 'use_augmentation': True, 'aug_probability': 0.6984706603104913, 'prob': 0.27232438944235016, 'noise_std': 0.3206864607885958, 'dropout_aug': 0.18185574852800854, 'alpha': 0.28259455112702525}. Best is trial 12 with value: 0.300940450303323.

Fold 1 finished with rmse value = 0.305473

[I 2026-09-03 16:43:02,915] Trial 17 pruned.

Fold 1 finished with rmse value = 0.304766
Fold 2 finished with rmse value = 0.304575
Fold 3 finished with rmse value = 0.302899

[I 2026-09-03 17:20:57,627] Trial 18 pruned.

Fold 1 finished with rmse value = 0.304545

[I 2026-09-03 17:22:09,445] Trial 19 pruned.

Fold 1 finished with rmse value = 0.311632

[I 2026-09-03 17:27:38,273] Trial 20 pruned.

Fold 1 finished with rmse value = 0.302869
Fold 2 finished with rmse value = 0.302221
Fold 3 finished with rmse value = 0.301095

[I 2026-09-03 18:34:57,727] Trial 21 pruned.

Fold 1 finished with rmse value = 0.303217

[I 2026-09-03 18:41:11,665] Trial 22 pruned.

Fold 1 finished with rmse value = 0.304913

[I 2026-09-03 19:03:22,383] Trial 23 pruned.

Fold 1 finished with rmse value = 0.304225

[I 2026-09-03 19:06:21,712] Trial 24 pruned.

Fold 1 finished with rmse value = 0.305670
Fold 2 finished with rmse value = 0.305501
Fold 3 finished with rmse value = 0.303480

[I 2026-09-03 19:13:28,413] Trial 25 pruned.

Fold 1 finished with rmse value = 0.302114
Fold 2 finished with rmse value = 0.302534
Fold 3 finished with rmse value = 0.300211
Fold 4 finished with rmse value = 0.301764
Fold 5 finished with rmse value = 0.301168

[I 2026-09-03 19:43:14,044] Trial 26 finished with value: 0.3015582002754164 and parameters: {'nc': 16, 'nb': 6, 'nh': 16, 'ed_nh_ratio': 20, 'ff_hd': 384, 'regressor_hd': 768, 'dropout': 0.034486337244102196, 'lr': 0.003351782038514747, 'weight_decay': 7.0746965713933025e-06, 'batch_size': 192, 'patience': 25, 'epochs': 85, 'use_augmentation': False, 'aug_probability': 0.27127795950972045, 'prob': 0.38566098495662693, 'noise_std': 0.29043048906710406, 'dropout_aug': 0.29407925722149986, 'alpha': 0.17167015562211632}. Best is trial 12 with value: 0.300940450303323.

Fold 1 finished with rmse value = 0.303543
Fold 2 finished with rmse value = 0.304036
Fold 3 finished with rmse value = 0.302330
Fold 4 finished with rmse value = 0.303453
Fold 5 finished with rmse value = 0.303121

[I 2026-09-03 19:55:58,528] Trial 27 finished with value: 0.3032965958648434 and parameters: {'nc': 14, 'nb': 7, 'nh': 10, 'ed_nh_ratio': 28, 'ff_hd': 256, 'regressor_hd': 640, 'dropout': 0.11608485555065189, 'lr': 0.0017464746838049179, 'weight_decay': 0.0009534030653785205, 'batch_size': 832, 'patience': 30, 'epochs': 150, 'use_augmentation': True, 'aug_probability': 0.18864325412529412, 'prob': 0.35541186199717456, 'noise_std': 0.3458792379991403, 'dropout_aug': 0.22829618599484935, 'alpha': 0.22084116883658897}. Best is trial 12 with value: 0.300940450303323.

Fold 1 finished with rmse value = 0.304677

[I 2026-09-03 19:58:28,736] Trial 28 pruned.

Fold 1 finished with rmse value = 0.302390
Fold 2 finished with rmse value = 0.303243
Fold 3 finished with rmse value = 0.300684
Fold 4 finished with rmse value = 0.302895
Fold 5 finished with rmse value = 0.300961

[I 2026-09-03 20:18:48,709] Trial 29 finished with value: 0.3020347410981431 and parameters: {'nc': 16, 'nb': 2, 'nh': 16, 'ed_nh_ratio': 12, 'ff_hd': 512, 'regressor_hd': 512, 'dropout': 0.09266447029906963, 'lr': 0.0054396190408180145, 'weight_decay': 1.0635469741899445e-05, 'batch_size': 128, 'patience': 20, 'epochs': 185, 'use_augmentation': True, 'aug_probability': 0.7923320074778549, 'prob': 0.3259458567961114, 'noise_std': 0.36044154226673436, 'dropout_aug': 0.3262552288928661, 'alpha': 0.19874727184472507}. Best is trial 12 with value: 0.300940450303323.

Fold 1 finished with rmse value = 0.303615

[I 2026-09-03 20:24:34,366] Trial 30 pruned.

Fold 1 finished with rmse value = 0.303152
Fold 2 finished with rmse value = 0.302717
Fold 3 finished with rmse value = 0.299975
Fold 4 finished with rmse value = 0.302447
Fold 5 finished with rmse value = 0.301183

[I 2026-09-03 20:49:46,831] Trial 31 finished with value: 0.3018948443978674 and parameters: {'nc': 16, 'nb': 6, 'nh': 16, 'ed_nh_ratio': 20, 'ff_hd': 384, 'regressor_hd': 768, 'dropout': 0.030404561395365175, 'lr': 0.0036148466750685827, 'weight_decay': 8.674152577401145e-06, 'batch_size': 192, 'patience': 25, 'epochs': 85, 'use_augmentation': False, 'aug_probability': 0.2666603259161528, 'prob': 0.3790664714093666, 'noise_std': 0.29528478043968237, 'dropout_aug': 0.3036679602433802, 'alpha': 0.1623979524901912}. Best is trial 12 with value: 0.300940450303323.

Fold 1 finished with rmse value = 0.302634
Fold 2 finished with rmse value = 0.302486
Fold 3 finished with rmse value = 0.299303
Fold 4 finished with rmse value = 0.300523
Fold 5 finished with rmse value = 0.301124

[I 2026-09-03 21:49:40,465] Trial 32 finished with value: 0.3012139191329411 and parameters: {'nc': 16, 'nb': 6, 'nh': 16, 'ed_nh_ratio': 20, 'ff_hd': 384, 'regressor_hd': 768, 'dropout': 0.0621885113351765, 'lr': 0.001128073425453675, 'weight_decay': 1.4969497444723484e-06, 'batch_size': 64, 'patience': 25, 'epochs': 80, 'use_augmentation': False, 'aug_probability': 0.38825453480045, 'prob': 0.38911852996409224, 'noise_std': 0.3388717285393941, 'dropout_aug': 0.29451432916508896, 'alpha': 0.20413321689220748}. Best is trial 12 with value: 0.300940450303323.

Fold 1 finished with rmse value = 0.301317
Fold 2 finished with rmse value = 0.302070
Fold 3 finished with rmse value = 0.299734
Fold 4 finished with rmse value = 0.301272
Fold 5 finished with rmse value = 0.299421

[I 2026-09-03 22:47:00,463] Trial 33 finished with value: 0.3007628517181855 and parameters: {'nc': 14, 'nb': 7, 'nh': 14, 'ed_nh_ratio': 20, 'ff_hd': 256, 'regressor_hd': 640, 'dropout': 0.06828309471373599, 'lr': 0.0011039745179749379, 'weight_decay': 1.6203555613578371e-06, 'batch_size': 64, 'patience': 30, 'epochs': 55, 'use_augmentation': False, 'aug_probability': 0.4182094561013929, 'prob': 0.344076504108704, 'noise_std': 0.3763694706537752, 'dropout_aug': 0.21137611594064803, 'alpha': 0.2131686709295353}. Best is trial 33 with value: 0.3007628517181855.

Fold 1 finished with rmse value = 0.303664
Fold 2 finished with rmse value = 0.304343
Fold 3 finished with rmse value = 0.301960
Fold 4 finished with rmse value = 0.303402
Fold 5 finished with rmse value = 0.301723

[I 2026-09-03 22:58:16,442] Trial 34 finished with value: 0.3030184152065287 and parameters: {'nc': 12, 'nb': 6, 'nh': 14, 'ed_nh_ratio': 24, 'ff_hd': 384, 'regressor_hd': 768, 'dropout': 0.06059190033291512, 'lr': 0.0010319578463112307, 'weight_decay': 1.7198100229028122e-06, 'batch_size': 320, 'patience': 25, 'epochs': 55, 'use_augmentation': False, 'aug_probability': 0.4169640653848864, 'prob': 0.34026409709809513, 'noise_std': 0.33949278422311013, 'dropout_aug': 0.20918964788690184, 'alpha': 0.2178549173233587}. Best is trial 33 with value: 0.3007628517181855.

Fold 1 finished with rmse value = 0.303875
Fold 2 finished with rmse value = 0.303563
Fold 3 finished with rmse value = 0.302579

[I 2026-09-03 23:02:58,039] Trial 35 pruned.

Fold 1 finished with rmse value = 0.304633

[I 2026-09-03 23:12:45,190] Trial 36 pruned.

Fold 1 finished with rmse value = 0.304205

[I 2026-09-03 23:16:03,809] Trial 37 pruned.

Fold 1 finished with rmse value = 0.304275

[I 2026-09-03 23:17:09,057] Trial 38 pruned.

Fold 1 finished with rmse value = 0.311902

[I 2026-09-03 23:19:31,232] Trial 39 pruned.

Fold 1 finished with rmse value = 0.336983

[I 2026-09-03 23:20:23,313] Trial 40 pruned.

Fold 1 finished with rmse value = 0.302600
Fold 2 finished with rmse value = 0.302052
Fold 3 finished with rmse value = 0.300285
Fold 4 finished with rmse value = 0.301691
Fold 5 finished with rmse value = 0.300492

[I 2026-09-04 00:36:36,333] Trial 41 finished with value: 0.3014237692686413 and parameters: {'nc': 16, 'nb': 8, 'nh': 16, 'ed_nh_ratio': 20, 'ff_hd': 128, 'regressor_hd': 640, 'dropout': 0.06979924985056617, 'lr': 0.0011815521932134784, 'weight_decay': 1.0020220176238828e-06, 'batch_size': 64, 'patience': 30, 'epochs': 105, 'use_augmentation': False, 'aug_probability': 0.3303550236873096, 'prob': 0.3863478465984266, 'noise_std': 0.3729960757788814, 'dropout_aug': 0.2653920242191448, 'alpha': 0.18790416182606065}. Best is trial 33 with value: 0.3007628517181855.

Fold 1 finished with rmse value = 0.303491

[I 2026-09-04 00:44:56,196] Trial 42 pruned.

Fold 1 finished with rmse value = 0.302458
Fold 2 finished with rmse value = 0.302067
Fold 3 finished with rmse value = 0.300094
Fold 4 finished with rmse value = 0.302228
Fold 5 finished with rmse value = 0.300944

[I 2026-09-04 01:13:09,133] Trial 43 finished with value: 0.30155823610103555 and parameters: {'nc': 16, 'nb': 7, 'nh': 16, 'ed_nh_ratio': 24, 'ff_hd': 128, 'regressor_hd': 512, 'dropout': 0.06985113313739215, 'lr': 0.0020511124773831997, 'weight_decay': 1.4819675570983097e-06, 'batch_size': 192, 'patience': 30, 'epochs': 75, 'use_augmentation': False, 'aug_probability': 0.23622562146468223, 'prob': 0.3542717596996915, 'noise_std': 0.36616510711561523, 'dropout_aug': 0.24524697699198733, 'alpha': 0.12218370492711779}. Best is trial 33 with value: 0.3007628517181855.

Fold 1 finished with rmse value = 0.302699
Fold 2 finished with rmse value = 0.304564
Fold 3 finished with rmse value = 0.300872
Fold 4 finished with rmse value = 0.302281
Fold 5 finished with rmse value = 0.302083

[I 2026-09-04 02:13:50,095] Trial 44 finished with value: 0.302499766478193 and parameters: {'nc': 16, 'nb': 8, 'nh': 14, 'ed_nh_ratio': 16, 'ff_hd': 256, 'regressor_hd': 640, 'dropout': 0.024184127243732395, 'lr': 0.0007074534607081519, 'weight_decay': 2.745862672193678e-06, 'batch_size': 64, 'patience': 30, 'epochs': 90, 'use_augmentation': False, 'aug_probability': 0.46319728384545433, 'prob': 0.37865114686333506, 'noise_std': 0.39822909123043804, 'dropout_aug': 0.21801829336475412, 'alpha': 0.19352710378138827}. Best is trial 33 with value: 0.3007628517181855.

Fold 1 finished with rmse value = 0.305537

[I 2026-09-04 02:16:21,297] Trial 45 pruned.

Fold 1 finished with rmse value = 0.302578
Fold 2 finished with rmse value = 0.302947
Fold 3 finished with rmse value = 0.300814

[I 2026-09-04 02:39:06,025] Trial 46 pruned.

Fold 1 finished with rmse value = 0.304200

[I 2026-09-04 02:40:26,933] Trial 47 pruned.

Fold 1 finished with rmse value = 0.303629

[I 2026-09-04 02:44:01,163] Trial 48 pruned.

Fold 1 finished with rmse value = 0.306961
Fold 2 finished with rmse value = 0.307925
Fold 3 finished with rmse value = 0.303547

[I 2026-09-04 02:49:59,330] Trial 49 pruned.

Fold 1 finished with rmse value = 0.308789

[I 2026-09-04 02:51:41,323] Trial 50 pruned.

Fold 1 finished with rmse value = 0.302304
Fold 2 finished with rmse value = 0.301987
Fold 3 finished with rmse value = 0.300112

[I 2026-09-04 03:43:41,769] Trial 51 pruned.

Fold 1 finished with rmse value = 0.304205
Fold 2 finished with rmse value = 0.304017
Fold 3 finished with rmse value = 0.301556

[I 2026-09-04 04:03:52,002] Trial 52 pruned.

Fold 1 finished with rmse value = 0.303943
Fold 2 finished with rmse value = 0.303897
Fold 3 finished with rmse value = 0.302457

[I 2026-09-04 04:12:11,811] Trial 53 pruned.

Fold 1 finished with rmse value = 0.302578
Fold 2 finished with rmse value = 0.302803
Fold 3 finished with rmse value = 0.300463

[I 2026-09-04 04:29:34,442] Trial 54 pruned.

Fold 1 finished with rmse value = 0.302839
Fold 2 finished with rmse value = 0.302134
Fold 3 finished with rmse value = 0.299953
Fold 4 finished with rmse value = 0.301802
Fold 5 finished with rmse value = 0.300524

[I 2026-09-04 05:50:33,861] Trial 55 finished with value: 0.30145051742634615 and parameters: {'nc': 16, 'nb': 7, 'nh': 16, 'ed_nh_ratio': 24, 'ff_hd': 128, 'regressor_hd': 640, 'dropout': 0.08749701215118055, 'lr': 0.0021032097374586436, 'weight_decay': 7.647543889629015e-06, 'batch_size': 64, 'patience': 30, 'epochs': 105, 'use_augmentation': True, 'aug_probability': 0.19233435038891408, 'prob': 0.3358304091186426, 'noise_std': 0.32132473086444874, 'dropout_aug': 0.006898065513396778, 'alpha': 0.23766454154197308}. Best is trial 33 with value: 0.3007628517181855.

Fold 1 finished with rmse value = 0.302871
Fold 2 finished with rmse value = 0.302057
Fold 3 finished with rmse value = 0.300948
Fold 4 finished with rmse value = 0.301444
Fold 5 finished with rmse value = 0.300914

[I 2026-09-04 06:28:35,751] Trial 56 finished with value: 0.30164676796504886 and parameters: {'nc': 16, 'nb': 6, 'nh': 16, 'ed_nh_ratio': 24, 'ff_hd': 384, 'regressor_hd': 640, 'dropout': 0.08843636343522263, 'lr': 0.0020922762523728337, 'weight_decay': 1.0204983617399002e-05, 'batch_size': 128, 'patience': 30, 'epochs': 105, 'use_augmentation': True, 'aug_probability': 0.19864732021851564, 'prob': 0.3229746549695084, 'noise_std': 0.27740453743333704, 'dropout_aug': 0.019220135134808497, 'alpha': 0.24253981216753795}. Best is trial 33 with value: 0.3007628517181855.

Fold 1 finished with rmse value = 0.304199

[I 2026-09-04 06:39:34,999] Trial 57 pruned.

Fold 1 finished with rmse value = 0.303134
Fold 2 finished with rmse value = 0.303173
Fold 3 finished with rmse value = 0.301688

[I 2026-09-04 06:48:51,355] Trial 58 pruned.

Fold 1 finished with rmse value = 0.304892

[I 2026-09-04 06:51:12,238] Trial 59 pruned.

Fold 1 finished with rmse value = 0.303531

[I 2026-09-04 06:54:29,883] Trial 60 pruned.

Fold 1 finished with rmse value = 0.302813
Fold 2 finished with rmse value = 0.302186
Fold 3 finished with rmse value = 0.300798
Fold 4 finished with rmse value = 0.300934
Fold 5 finished with rmse value = 0.301080

[I 2026-09-04 08:13:43,277] Trial 61 finished with value: 0.30156225450368834 and parameters: {'nc': 16, 'nb': 8, 'nh': 16, 'ed_nh_ratio': 20, 'ff_hd': 128, 'regressor_hd': 640, 'dropout': 0.06780702601648333, 'lr': 0.0012579282508132386, 'weight_decay': 1.8886040597090936e-06, 'batch_size': 64, 'patience': 30, 'epochs': 120, 'use_augmentation': True, 'aug_probability': 0.13892797351744335, 'prob': 0.376051261377086, 'noise_std': 0.3591923214898349, 'dropout_aug': 0.07936222449011782, 'alpha': 0.19492910938501978}. Best is trial 33 with value: 0.3007628517181855.

Fold 1 finished with rmse value = 0.303416
Fold 2 finished with rmse value = 0.303958
Fold 3 finished with rmse value = 0.301812

[I 2026-09-04 08:35:57,069] Trial 62 pruned.

Fold 1 finished with rmse value = 0.302687
Fold 2 finished with rmse value = 0.303208
Fold 3 finished with rmse value = 0.301549

[I 2026-09-04 08:53:43,825] Trial 63 pruned.

Fold 1 finished with rmse value = 0.301610
Fold 2 finished with rmse value = 0.301668
Fold 3 finished with rmse value = 0.300306
Fold 4 finished with rmse value = 0.302091
Fold 5 finished with rmse value = 0.300400

[I 2026-09-04 10:19:29,998] Trial 64 finished with value: 0.3012149625597393 and parameters: {'nc': 14, 'nb': 7, 'nh': 14, 'ed_nh_ratio': 16, 'ff_hd': 256, 'regressor_hd': 640, 'dropout': 0.12213946086649591, 'lr': 0.0018689367173458313, 'weight_decay': 4.781187704833396e-06, 'batch_size': 64, 'patience': 30, 'epochs': 145, 'use_augmentation': True, 'aug_probability': 0.18433007814346367, 'prob': 0.3462252436331784, 'noise_std': 0.352826245894448, 'dropout_aug': 0.34086604548956134, 'alpha': 0.13522727856781996}. Best is trial 33 with value: 0.3007628517181855.

Fold 1 finished with rmse value = 0.301564
Fold 2 finished with rmse value = 0.301893
Fold 3 finished with rmse value = 0.301013
Fold 4 finished with rmse value = 0.301610
Fold 5 finished with rmse value = 0.300189

[I 2026-09-04 11:01:04,606] Trial 65 finished with value: 0.3012537507877987 and parameters: {'nc': 14, 'nb': 6, 'nh': 14, 'ed_nh_ratio': 12, 'ff_hd': 256, 'regressor_hd': 640, 'dropout': 0.1326897696121536, 'lr': 0.001988238931184998, 'weight_decay': 1.3839289433678638e-05, 'batch_size': 128, 'patience': 30, 'epochs': 150, 'use_augmentation': True, 'aug_probability': 0.1912552285150612, 'prob': 0.3348271159299272, 'noise_std': 0.35105019586978237, 'dropout_aug': 0.37883319495123763, 'alpha': 0.11236573972281655}. Best is trial 33 with value: 0.3007628517181855.

Fold 1 finished with rmse value = 0.307342

[I 2026-09-04 11:04:06,550] Trial 66 pruned.

Fold 1 finished with rmse value = 0.302307
Fold 2 finished with rmse value = 0.302860
Fold 3 finished with rmse value = 0.301338

[I 2026-09-04 11:36:16,071] Trial 67 pruned.

Fold 1 finished with rmse value = 0.302536
Fold 2 finished with rmse value = 0.302358
Fold 3 finished with rmse value = 0.300501

[I 2026-09-04 11:43:42,252] Trial 68 pruned.

Fold 1 finished with rmse value = 0.302898

[I 2026-09-04 11:48:09,894] Trial 69 pruned.

Fold 1 finished with rmse value = 0.304170
Fold 2 finished with rmse value = 0.304386
Fold 3 finished with rmse value = 0.302858

[I 2026-09-04 11:55:07,534] Trial 70 pruned.

Fold 1 finished with rmse value = 0.301297
Fold 2 finished with rmse value = 0.302167
Fold 3 finished with rmse value = 0.300217
Fold 4 finished with rmse value = 0.300972
Fold 5 finished with rmse value = 0.300108

[I 2026-09-04 13:08:35,865] Trial 71 finished with value: 0.3009522081345256 and parameters: {'nc': 14, 'nb': 7, 'nh': 14, 'ed_nh_ratio': 16, 'ff_hd': 256, 'regressor_hd': 640, 'dropout': 0.11963782170226178, 'lr': 0.0019980676538663773, 'weight_decay': 9.150768226820037e-06, 'batch_size': 64, 'patience': 30, 'epochs': 200, 'use_augmentation': True, 'aug_probability': 0.20799974511417166, 'prob': 0.32785094819563737, 'noise_std': 0.3207686328971764, 'dropout_aug': 0.3373855778366332, 'alpha': 0.14663276402899855}. Best is trial 33 with value: 0.3007628517181855.

Fold 1 finished with rmse value = 0.303221

[I 2026-09-04 13:15:37,849] Trial 72 pruned.

Fold 1 finished with rmse value = 0.301902
Fold 2 finished with rmse value = 0.302583
Fold 3 finished with rmse value = 0.300327

[I 2026-09-04 14:06:28,497] Trial 73 pruned.

Fold 1 finished with rmse value = 0.303335
Fold 2 finished with rmse value = 0.303851
Fold 3 finished with rmse value = 0.301749

[I 2026-09-04 14:19:28,099] Trial 74 pruned.

Fold 1 finished with rmse value = 0.302200
Fold 2 finished with rmse value = 0.301988
Fold 3 finished with rmse value = 0.300995

[I 2026-09-04 14:48:23,086] Trial 75 pruned.

Fold 1 finished with rmse value = 0.302541
Fold 2 finished with rmse value = 0.303394
Fold 3 finished with rmse value = 0.300768
Fold 4 finished with rmse value = 0.302998
Fold 5 finished with rmse value = 0.301895

[I 2026-09-04 15:27:12,830] Trial 76 finished with value: 0.30231936787546715 and parameters: {'nc': 14, 'nb': 7, 'nh': 14, 'ed_nh_ratio': 12, 'ff_hd': 512, 'regressor_hd': 512, 'dropout': 0.1338438702163067, 'lr': 0.0012170581655924335, 'weight_decay': 1.857914305153926e-06, 'batch_size': 192, 'patience': 30, 'epochs': 180, 'use_augmentation': True, 'aug_probability': 0.31803690773022, 'prob': 0.3069025387180504, 'noise_std': 0.2857824995366316, 'dropout_aug': 0.2304842662891602, 'alpha': 0.17528832095009592}. Best is trial 33 with value: 0.3007628517181855.

Fold 1 finished with rmse value = 0.302309
Fold 2 finished with rmse value = 0.302250
Fold 3 finished with rmse value = 0.300203
Fold 4 finished with rmse value = 0.301738
Fold 5 finished with rmse value = 0.299620

[I 2026-09-04 16:42:13,342] Trial 77 finished with value: 0.30122400904899543 and parameters: {'nc': 12, 'nb': 7, 'nh': 12, 'ed_nh_ratio': 16, 'ff_hd': 256, 'regressor_hd': 768, 'dropout': 0.10490694490802935, 'lr': 0.0016510188098712237, 'weight_decay': 4.804938566561205e-06, 'batch_size': 64, 'patience': 30, 'epochs': 200, 'use_augmentation': False, 'aug_probability': 0.16517147276503974, 'prob': 0.3876652703697823, 'noise_std': 0.2632218857151686, 'dropout_aug': 0.18884140924859807, 'alpha': 0.09565191642223808}. Best is trial 33 with value: 0.3007628517181855.

Fold 1 finished with rmse value = 0.302958

[I 2026-09-04 16:45:07,071] Trial 78 pruned.

Fold 1 finished with rmse value = 0.302525
Fold 2 finished with rmse value = 0.302461
Fold 3 finished with rmse value = 0.299739
Fold 4 finished with rmse value = 0.301062
Fold 5 finished with rmse value = 0.300525

[I 2026-09-04 18:46:51,008] Trial 79 finished with value: 0.3012622221591638 and parameters: {'nc': 12, 'nb': 7, 'nh': 10, 'ed_nh_ratio': 12, 'ff_hd': 384, 'regressor_hd': 896, 'dropout': 0.1498326549675716, 'lr': 0.002257210532566308, 'weight_decay': 1.1489278981767536e-05, 'batch_size': 64, 'patience': 25, 'epochs': 180, 'use_augmentation': False, 'aug_probability': 0.12096512272407171, 'prob': 0.32765631388583993, 'noise_std': 0.2212067512118525, 'dropout_aug': 0.18761443685355864, 'alpha': 0.08951609420965269}. Best is trial 33 with value: 0.3007628517181855.

Fold 1 finished with rmse value = 0.303494
Fold 2 finished with rmse value = 0.302879
Fold 3 finished with rmse value = 0.301895

[I 2026-09-04 18:53:21,465] Trial 80 pruned.

Fold 1 finished with rmse value = 0.302528
Fold 2 finished with rmse value = 0.301598
Fold 3 finished with rmse value = 0.300821
Fold 4 finished with rmse value = 0.301859
Fold 5 finished with rmse value = 0.300163

[I 2026-09-04 20:51:53,208] Trial 81 finished with value: 0.301393735607524 and parameters: {'nc': 12, 'nb': 7, 'nh': 8, 'ed_nh_ratio': 12, 'ff_hd': 384, 'regressor_hd': 896, 'dropout': 0.1751624772629689, 'lr': 0.002312594905031972, 'weight_decay': 9.878308037308075e-06, 'batch_size': 64, 'patience': 25, 'epochs': 200, 'use_augmentation': False, 'aug_probability': 0.1266325292982607, 'prob': 0.33085177947465044, 'noise_std': 0.2142119451659611, 'dropout_aug': 0.18943992402161267, 'alpha': 0.08885240271450812}. Best is trial 33 with value: 0.3007628517181855.

Fold 1 finished with rmse value = 0.302232
Fold 2 finished with rmse value = 0.302683
Fold 3 finished with rmse value = 0.301537

[I 2026-09-04 21:06:59,924] Trial 82 pruned.

Fold 1 finished with rmse value = 0.302343
Fold 2 finished with rmse value = 0.302456
Fold 3 finished with rmse value = 0.300571

[I 2026-09-04 21:34:45,785] Trial 83 pruned.

Fold 1 finished with rmse value = 0.302514
Fold 2 finished with rmse value = 0.302780
Fold 3 finished with rmse value = 0.299757
Fold 4 finished with rmse value = 0.300561
Fold 5 finished with rmse value = 0.300304

[I 2026-09-04 22:49:05,276] Trial 84 finished with value: 0.3011832391066326 and parameters: {'nc': 12, 'nb': 6, 'nh': 10, 'ed_nh_ratio': 8, 'ff_hd': 256, 'regressor_hd': 768, 'dropout': 0.139495113330932, 'lr': 0.0016369301935816615, 'weight_decay': 8.258492695445423e-06, 'batch_size': 64, 'patience': 25, 'epochs': 185, 'use_augmentation': False, 'aug_probability': 0.1523024283492808, 'prob': 0.299653882565446, 'noise_std': 0.24035729157639477, 'dropout_aug': 0.1384226904230565, 'alpha': 0.1155969029742371}. Best is trial 33 with value: 0.3007628517181855.

Fold 1 finished with rmse value = 0.304694
Fold 2 finished with rmse value = 0.303773
Fold 3 finished with rmse value = 0.301442

[I 2026-09-04 22:52:26,041] Trial 85 pruned.

Fold 1 finished with rmse value = 0.304015

[I 2026-09-04 22:56:47,179] Trial 86 pruned.

Fold 1 finished with rmse value = 0.303044

[I 2026-09-04 23:07:03,193] Trial 87 pruned.

Fold 1 finished with rmse value = 0.305877

[I 2026-09-04 23:08:16,617] Trial 88 pruned.

Fold 1 finished with rmse value = 0.303187

[I 2026-09-04 23:18:10,505] Trial 89 pruned.

Fold 1 finished with rmse value = 0.306136

[I 2026-09-04 23:21:51,069] Trial 90 pruned.

Fold 1 finished with rmse value = 0.301690
Fold 2 finished with rmse value = 0.302349
Fold 3 finished with rmse value = 0.300054
Fold 4 finished with rmse value = 0.301433
Fold 5 finished with rmse value = 0.299850

[I 2026-09-05 01:09:22,622] Trial 91 finished with value: 0.3010753380800998 and parameters: {'nc': 12, 'nb': 7, 'nh': 10, 'ed_nh_ratio': 12, 'ff_hd': 384, 'regressor_hd': 896, 'dropout': 0.16420656177132975, 'lr': 0.0023075292320534064, 'weight_decay': 1.1585287058969085e-05, 'batch_size': 64, 'patience': 25, 'epochs': 180, 'use_augmentation': False, 'aug_probability': 0.1470648389114809, 'prob': 0.326341753387518, 'noise_std': 0.1981325685604194, 'dropout_aug': 0.22310439473559004, 'alpha': 0.12916474478646892}. Best is trial 33 with value: 0.3007628517181855.

Fold 1 finished with rmse value = 0.302942

[I 2026-09-05 01:17:23,679] Trial 92 pruned.

Fold 1 finished with rmse value = 0.301733
Fold 2 finished with rmse value = 0.302628
Fold 3 finished with rmse value = 0.300310

[I 2026-09-05 02:50:28,823] Trial 93 pruned.

Fold 1 finished with rmse value = 0.302169
Fold 2 finished with rmse value = 0.303246
Fold 3 finished with rmse value = 0.300851
Fold 4 finished with rmse value = 0.301610
Fold 5 finished with rmse value = 0.300684

[I 2026-09-05 03:27:51,341] Trial 94 finished with value: 0.30171192942950203 and parameters: {'nc': 12, 'nb': 7, 'nh': 10, 'ed_nh_ratio': 20, 'ff_hd': 384, 'regressor_hd': 896, 'dropout': 0.12095672631262205, 'lr': 0.001959878443911861, 'weight_decay': 8.983899206689245e-06, 'batch_size': 128, 'patience': 25, 'epochs': 165, 'use_augmentation': False, 'aug_probability': 0.10094026027778266, 'prob': 0.36102729563657016, 'noise_std': 0.3441503054111788, 'dropout_aug': 0.1720605475839856, 'alpha': 0.15049686833794818}. Best is trial 33 with value: 0.3007628517181855.

Fold 1 finished with rmse value = 0.302762
Fold 2 finished with rmse value = 0.303543
Fold 3 finished with rmse value = 0.299804
Fold 4 finished with rmse value = 0.300701
Fold 5 finished with rmse value = 0.301252

[I 2026-09-05 03:58:33,383] Trial 95 finished with value: 0.30161232479432815 and parameters: {'nc': 14, 'nb': 7, 'nh': 12, 'ed_nh_ratio': 16, 'ff_hd': 256, 'regressor_hd': 896, 'dropout': 0.1533083764094914, 'lr': 0.00245994345650538, 'weight_decay': 1.6248049693656982e-06, 'batch_size': 192, 'patience': 30, 'epochs': 195, 'use_augmentation': False, 'aug_probability': 0.1381054978070014, 'prob': 0.37198562042930344, 'noise_std': 0.1983101641650336, 'dropout_aug': 0.21990665722686192, 'alpha': 0.2004413871951069}. Best is trial 33 with value: 0.3007628517181855.

Fold 1 finished with rmse value = 0.301698
Fold 2 finished with rmse value = 0.301039
Fold 3 finished with rmse value = 0.300366

[I 2026-09-05 04:40:43,984] Trial 96 pruned.

Fold 1 finished with rmse value = 0.302948

[I 2026-09-05 04:44:40,378] Trial 97 pruned.

Fold 1 finished with rmse value = 0.302579
Fold 2 finished with rmse value = 0.302396
Fold 3 finished with rmse value = 0.301052

[I 2026-09-05 04:59:40,883] Trial 98 pruned.

Fold 1 finished with rmse value = 0.304305

[I 2026-09-05 05:03:11,291] Trial 99 pruned.

Best value (rmse) = 0.3007628517181855
Best params saved to: model_scEve_tuning/best_params.txt

[19]:

{'nc': 14,
 'nb': 7,
 'nh': 14,
 'ed_nh_ratio': 20,
 'ff_hd': 256,
 'regressor_hd': 640,
 'dropout': 0.06828309471373599,
 'lr': 0.0011039745179749379,
 'weight_decay': 1.6203555613578371e-06,
 'batch_size': 64,
 'patience': 30,
 'epochs': 55,
 'use_augmentation': False,
 'aug_probability': 0.4182094561013929,
 'prob': 0.344076504108704,
 'noise_std': 0.3763694706537752,
 'dropout_aug': 0.21137611594064803,
 'alpha': 0.2131686709295353}

[20]:

# Train sceve model using mdata_train dataset
scparadise.sceve.train_tuned(
    mdata_train,
    path = '',
    path_tuned='model_scEve_tuning',
    model_name = 'model_scEve_hp_tuned',
    first_modality_name = 'rna',
    second_modality_name = 'adt',
    detailed_annotation = 'celltype_l3',
    eval_metric=['rmse']
)

Successfully loaded tuned hyperparameters!
Device: cuda
Task: Predict 'adt' from 'rna'
Number of first modality features: 635
Number of second modality features: 224

Dataset split:
Train dataset contains: 37359 cells, it is 80.0 % of input dataset
Validation dataset contains: 9340 cells, it is 20.0 % of input dataset

Training scEve model: 100%|████████████████████| 55/55 [11:50<00:00, 12.93s/it]

Training completed!
Model saved to model_scEve_hp_tuned

Check tuned model quality#

[21]:

# Predict surface proteins using tuned scEve model
mdata_pred = scparadise.sceve.predict(
    adata_test_rna,
    path_model = 'model_scEve_hp_tuned',
    return_mdata = True # default value
)

Feature prediction using the 2nd version of the scEve model
scEve model loaded from model_scEve_hp_tuned
Feature alignment:
Model features: 635
New data features: 635
Matched features: 635 (100.0%)

Predicting: 100%|██████████████████████████████| 25/25 [00:00<00:00, 62.19it/s]

[22]:

# Add umap coordinates
mdata_pred.obsm['X_umap'] = adata_test_rna.obsm['X_umap']

[23]:

# Proteins visualization
proteins_dict = {
    "Observed": proteins_observed,
    "Imputed": proteins_imputed
}
# Choose a color map for protein abundance visualization
cmap = 'bwr'
# Make Axes
# Number of needed rows and columns (based on the row with the most columns)
nrow = len(proteins_dict)
ncol = max([len(vs) for vs in proteins_dict.values()])
fig, axs = plt.subplots(nrow, ncol, figsize=(3 * ncol, 3 * nrow))
# Plot expression for every marker on the corresponding Axes object
for row_idx, (prots, markers) in enumerate(proteins_dict.items()):
    col_idx = 0
    for marker in markers:
        ax = axs[row_idx, col_idx]
        if prots == 'Observed':
            mu.pl.umap(mdata_test, color=marker, cmap = cmap, ax=ax, vmax = 'p99', show=False, frameon=False)
        else:
            mu.pl.umap(mdata_pred, color=marker, cmap = cmap, ax=ax, vmax = 'p99', show=False, frameon=False)
        # Add cell type as row label - here we simply add it as ylabel of
        # the first Axes object in the row
        if col_idx == 0:
            # We disabled axis drawing in UMAP to have plots without background and border
            # so we need to re-enable axis to plot the ylabel
            ax.axis("on")
            ax.tick_params(
                top="off",
                bottom="off",
                left="off",
                right="off",
                labelleft="on",
                labelbottom="off",
            )
            ax.set_ylabel(prots + "\n", rotation=90, fontsize=14)
            ax.set(frame_on=False)
        col_idx += 1
    # Remove unused column Axes in the current row
    while col_idx < ncol:
        axs[row_idx, col_idx].remove()
        col_idx += 1
# Alignment within the Figure
fig.tight_layout()
../../../_images/tutorials_notebooks_scEve_scEve_train_29_0.png 
[24]:

# Check prediction results
df_hp_tuned = scparadise.scnoah.report_reg(
    adata_prot = mdata_test.mod['adt'],
    adata_pred_prot = mdata_pred.mod['adt']
)
df_hp_tuned

[24]:
EVS r2_score RMSE MedianAE MeanAE
score 0.3 0.266 0.322 0.197 0.244
EVS/r2_score higher value - better prediction
RMSE/MedianAE/MeanAE lower value - better prediction 
[25]:

# Compare prediction results
# 'untuned' row represents untuned model
# 'hp tuned' row represents tuned model
df.compare(df_hp_tuned, keep_equal=True, align_axis = 0, result_names=('untuned', 'hp tuned'))
# EVS/r2_score:     higher value - better prediction
# RMSE/MedianAE/MeanAE:     lower value - better prediction

[25]:
EVS r2_score RMSE MedianAE MeanAE
score untuned 0.281 0.235 0.328 0.201 0.248
hp tuned 0.3 0.266 0.322 0.197 0.244

Hyperparameter tuning improved the performance of the scEve prediction model across all evaluated metrics. Compared with the untuned model, the tuned model achieved higher explained variance score (EVS: 0.300 vs. 0.281) and R2-score (0.266 vs. 0.235), indicating that it explains a larger fraction of variability in the target values. At the same time, prediction errors decreased: RMSE declined from 0.328 to 0.322, MedianAE from 0.201 to 0.197, and MeanAE from 0.248 to 0.244.

Overall, although the absolute improvements are modest, they are consistent across both goodness-of-fit and error-based metrics. Therefore, the hyperparameter-tuned scEve model should be preferred for subsequent analyses and predictions.

[27]:

# Add regression status using 5 metrics
for metric in ['RMSE', 'MeanAE', 'MedianAE', 'EVS', 'r2_score']:
    scparadise.scnoah.regres_status(
        adata_prot = mdata_test.mod['adt'],
        adata_pred_prot = mdata_pred.mod['adt'],
        metric=metric
    )
    mdata_pred.obs[f'regres_status_{metric}'] = mdata_pred.mod['adt'].obs[f'regres_status_{metric}']

[29]:

# Visualize observed proteins
mu.pl.umap(
    mdata_pred,
    color=[
        'regres_status_RMSE',
        'regres_status_MeanAE',
        'regres_status_MedianAE',
        'regres_status_EVS',
        'regres_status_r2_score'
    ],
    frameon = False,
    cmap = 'coolwarm',
    ncols = 5,
    wspace = 0.15,
    hspace = 0.1
)
../../../_images/tutorials_notebooks_scEve_scEve_train_34_0.png

Based on a per cell type regression metrics score, it is possible to draw conclusions about the uniformity of protein prediction across different cell types in the dataset.

[30]:

# Find correlations between imputed and observed proteins
df_corr_hp_tuned = pd.DataFrame(columns=['Pearson coef', 'p-value'])
for i in mdata_test.mod['adt'].var_names.tolist():
    person_coef = scparadise.scnoah.pearson_coef(
        adata = mdata_test.mod['adt'],
        adata_pred = mdata_pred.mod['adt'],
        feature = i,
        feature_pred = i + '_pred',
        print_res = False
    )
    df_corr_hp_tuned.loc[i] = [person_coef['Pearson coefficient'], person_coef['p-value']]
df_corr_hp_tuned['p-value'] = df_corr['p-value'].astype('float64')
df_corr_hp_tuned.sort_values('Pearson coef', ascending=False).head(5)

[30]:
Pearson coef p-value
adt_CLEC12A 0.965 0.0
adt_CD64 0.945 0.0
adt_CD11b-2 0.928 0.0
adt_CD8a 0.916 0.0
adt_CD19 0.915 0.0 
[31]:

# Find highly correlated protein 'Pearson coef' > 0.5
df_high_corr_hp_tuned = df_corr_hp_tuned[df_corr_hp_tuned['Pearson coef'] >= 0.5].copy()
print('Number of proteins with Pearson coefficient >= 0.5:', len(df_high_corr_hp_tuned))
df_high_corr_hp_tuned.sort_values(by = 'Pearson coef', ascending=False).head(5)

Number of proteins with Pearson coefficient >= 0.5: 113

[31]:
Pearson coef p-value
adt_CLEC12A 0.965 0.0
adt_CD64 0.945 0.0
adt_CD11b-2 0.928 0.0
adt_CD8a 0.916 0.0
adt_CD19 0.915 0.0 
[32]:

# Comparison of Pearson coefficients from the tuned and default models (tuned minus default).
df_compare = df_corr_hp_tuned.subtract(df_corr)
df_compare.rename(columns={'Pearson coef': 'Pearson coef comparison'}, inplace=True)
df_compare.sort_values(by = 'Pearson coef comparison', ascending=False).head(5)

[32]:
Pearson coef comparison p-value
adt_CD103 0.236 0.0
adt_CD34 0.082 0.0
adt_CD81 0.060 0.0
adt_CD45-2 0.057 0.0
adt_CD56-1 0.054 0.0

Hyperparameter tuning improved protein prediction performance in the scEve model. In particular, the Pearson correlation coefficient for adt_CD103 increased by 0.236 after tuning, indicating a substantial improvement in the model’s ability to predict this protein.

Moreover, the number of proteins predicted with a Pearson correlation coefficient of at least 0.5 increased from 107 to 113. Thus, hyperparameter optimization improved not only the performance of individual protein predictions but also the overall number of proteins predicted with moderate-to-strong agreement with the observed values. This is consistent with the broader improvement in predictive performance observed for the tuned model.

[34]:

# Save mdata with imputed proteins
mdata_pred.write_h5mu('mdata_pred_tuned.h5mu')

Recommendation#

We strongly recommend performing hyperparameter tuning during scEve model training whenever computational resources and time permit. As demonstrated by the improved global prediction metrics and the increased number of accurately predicted proteins, tuning can improve both overall model performance and predictions for individual proteins.

We also recommend evaluating the final tuned model on an independent test dataset that was not used for training or hyperparameter selection. This external evaluation is essential to assess model generalizability, detect potential overfitting, and obtain an unbiased estimate of predictive performance on new data.

[35]:

import session_info2
session_info2.session_info()

[35]:
PackageVersion
pandas2.3.3
scanpy1.11.5
scparadise1.2.0
muon0.1.7
numpy2.4.6
matplotlib3.11.0
mudata0.3.8
anndata0.12.16
ComponentInfo
Python3.11.15 (main, Jun 11 2026, 15:20:16) [GCC 14.3.0]
OSLinux-6.8.0-138-generic-x86_64-with-glibc2.39
Updated2026-09-05 15:22
Dependencies
DependencyVersion
ipykernel7.3.0
alembic1.18.4
igraph1.0.0
triton3.7.0
dataclasses0.6
stack-data0.6.3
matplotlib-inline0.2.2
cuda-bindings13.3.1
Cython3.3.0
texttable1.7.0
pyparsing3.3.2
sympy1.14.0
joblib1.5.3
xarray2026.4.0
defusedxml0.7.1
typing3.7.4.3
plottable0.1.5
pillow12.2.0
python-dateutil2.9.0.post0
jupyter_client8.9.1
pyzmq27.1.0
typing_extensions4.15.0
zarr3.1.6
comm0.2.3
rich15.0.0
leidenalg0.12.0
pycparser3.0 (3.00)
MarkupSafe3.0.3
numcodecs0.16.5
h5py3.16.0
six1.17.0
cffi2.0.0
decorator5.3.1
colorlog6.10.1
prompt_toolkit3.0.52
scverse-misc0.0.3
cuda-pathfinder1.5.5
kiwisolver1.5.0
pytorch-tabnet4.1.0
jedi0.20.0
narwhals2.22.1
asttokens3.0.1
wcwidth0.8.1
opt_einsum3.4.0
session-info20.4.1
lazy-loader0.5
legacy-api-wrap1.5
cycler0.12.1
fonttools4.63.0
natsort8.4.0
parso0.8.7
SQLAlchemy2.0.50
platformdirs4.10.0
debugpy1.8.21
optuna4.9.0
executing2.2.1
pathlib1.0.1
threadpoolctl3.6.0
greenlet3.5.1
scipy1.17.1
seaborn0.13.2
pure_eval0.2.3
google-crc32c1.8.0
sklearn-compat0.1.6
scikit-learn1.9.0
sparse0.18.0
argparse1.4.0 (1.1)
setuptools81.0.0
umap-learn0.5.12
packaging26.0
tornado6.5.7
psutil7.2.2
charset-normalizer3.4.7
cloudpickle3.1.2
tqdm4.68.2
Pygments2.20.0
ipython9.14.1
pyarrow24.0.0
pytz2026.2
fsspec2026.4.0
PyYAML6.0.3
traitlets5.15.1
shap0.49.1
jupyter_core5.9.1
scikit-image0.26.0
imbalanced-learn0.14.2
slicer0.0.8
pynndescent0.6.0
statsmodels0.14.6
Mako1.3.12
numba0.65.1
llvmlite0.47.0
patsy1.0.2
donfig0.8.1.post1
mpmath1.3.0
torch2.12.0 (2.12.0+cu130)
Copyable Markdown 
| Package    | Version |
| ---------- | ------- |
| pandas     | 2.3.3   |
| scanpy     | 1.11.5  |
| scparadise | 1.2.0   |
| muon       | 0.1.7   |
| numpy      | 2.4.6   |
| matplotlib | 3.11.0  |
| mudata     | 0.3.8   |
| anndata    | 0.12.16 |

| Dependency         | Version               |
| ------------------ | --------------------- |
| ipykernel          | 7.3.0                 |
| alembic            | 1.18.4                |
| igraph             | 1.0.0                 |
| triton             | 3.7.0                 |
| dataclasses        | 0.6                   |
| stack-data         | 0.6.3                 |
| matplotlib-inline  | 0.2.2                 |
| cuda-bindings      | 13.3.1                |
| Cython             | 3.3.0                 |
| texttable          | 1.7.0                 |
| pyparsing          | 3.3.2                 |
| sympy              | 1.14.0                |
| joblib             | 1.5.3                 |
| xarray             | 2026.4.0              |
| defusedxml         | 0.7.1                 |
| typing             | 3.7.4.3               |
| plottable          | 0.1.5                 |
| pillow             | 12.2.0                |
| python-dateutil    | 2.9.0.post0           |
| jupyter_client     | 8.9.1                 |
| pyzmq              | 27.1.0                |
| typing_extensions  | 4.15.0                |
| zarr               | 3.1.6                 |
| comm               | 0.2.3                 |
| rich               | 15.0.0                |
| leidenalg          | 0.12.0                |
| pycparser          | 3.0 (3.00)            |
| MarkupSafe         | 3.0.3                 |
| numcodecs          | 0.16.5                |
| h5py               | 3.16.0                |
| six                | 1.17.0                |
| cffi               | 2.0.0                 |
| decorator          | 5.3.1                 |
| colorlog           | 6.10.1                |
| prompt_toolkit     | 3.0.52                |
| scverse-misc       | 0.0.3                 |
| cuda-pathfinder    | 1.5.5                 |
| kiwisolver         | 1.5.0                 |
| pytorch-tabnet     | 4.1.0                 |
| jedi               | 0.20.0                |
| narwhals           | 2.22.1                |
| asttokens          | 3.0.1                 |
| wcwidth            | 0.8.1                 |
| opt_einsum         | 3.4.0                 |
| session-info2      | 0.4.1                 |
| lazy-loader        | 0.5                   |
| legacy-api-wrap    | 1.5                   |
| cycler             | 0.12.1                |
| fonttools          | 4.63.0                |
| natsort            | 8.4.0                 |
| parso              | 0.8.7                 |
| SQLAlchemy         | 2.0.50                |
| platformdirs       | 4.10.0                |
| debugpy            | 1.8.21                |
| optuna             | 4.9.0                 |
| executing          | 2.2.1                 |
| pathlib            | 1.0.1                 |
| threadpoolctl      | 3.6.0                 |
| greenlet           | 3.5.1                 |
| scipy              | 1.17.1                |
| seaborn            | 0.13.2                |
| pure_eval          | 0.2.3                 |
| google-crc32c      | 1.8.0                 |
| sklearn-compat     | 0.1.6                 |
| scikit-learn       | 1.9.0                 |
| sparse             | 0.18.0                |
| argparse           | 1.4.0 (1.1)           |
| setuptools         | 81.0.0                |
| umap-learn         | 0.5.12                |
| packaging          | 26.0                  |
| tornado            | 6.5.7                 |
| psutil             | 7.2.2                 |
| charset-normalizer | 3.4.7                 |
| cloudpickle        | 3.1.2                 |
| tqdm               | 4.68.2                |
| Pygments           | 2.20.0                |
| ipython            | 9.14.1                |
| pyarrow            | 24.0.0                |
| pytz               | 2026.2                |
| fsspec             | 2026.4.0              |
| PyYAML             | 6.0.3                 |
| traitlets          | 5.15.1                |
| shap               | 0.49.1                |
| jupyter_core       | 5.9.1                 |
| scikit-image       | 0.26.0                |
| imbalanced-learn   | 0.14.2                |
| slicer             | 0.0.8                 |
| pynndescent        | 0.6.0                 |
| statsmodels        | 0.14.6                |
| Mako               | 1.3.12                |
| numba              | 0.65.1                |
| llvmlite           | 0.47.0                |
| patsy              | 1.0.2                 |
| donfig             | 0.8.1.post1           |
| mpmath             | 1.3.0                 |
| torch              | 2.12.0 (2.12.0+cu130) |

| Component | Info                                               |
| --------- | -------------------------------------------------- |
| Python    | 3.11.15 (main, Jun 11 2026, 15:20:16) [GCC 14.3.0] |
| OS        | Linux-6.8.0-138-generic-x86_64-with-glibc2.39      |
| Updated   | 2026-09-05 15:22                                   |