Custom scEve model training and optimization

Custom scEve model training and optimization#

scEve models are designed to impute additional modalities from scRNA-seq data. The imputed modality provides a richer description of cellular functional states and cell identity, for example by helping distinguish cell types that are transcriptionally similar.

[1]:

# Python packages
import warnings
warnings.simplefilter('ignore')

import scanpy as sc
import scparadise
import muon as mu
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt

sc.set_figure_params(dpi = 120)

Recommendations about training dataset#

We recommend shifted logarithm data normalization method for gene expression data: sc.pp.normalize_total(adata, target_sum=None) sc.pp.log1p(adata)

But you can use any other method of gene expression data normalzation (Use the same normalization method for test dataset). Training dataset sould contain all genes that you want to use for model training in mdata_train.mod[‘rna’].X We recommend removing all non-marker genes from mdata_train.mod[‘rna’].X (removing such useless genes may improve performance and model quality metrics).

To demonstrate the scEve model training and evaluation, we used eight samples from the CITE-seq 3’ PBMC dataset. The genes were pre-sorted, and the data were normalized. Details of the dataset processing are available in the PBMC_3p_dataset script.

[2]:

# Download data from figshare
# You can use wget, or download the file in a browser by copying and pasting the link into a new tab.
link = 'https://figshare.com/ndownloader/files/51749906'

[3]:

# Load prepared for training anndata object
mdata = mu.read_h5mu('mdata_pbmc.h5mu')

[4]:

# Create mdata_train dataset (we used 4 samples for training model)
mdata_train = mdata[mdata.obs['orig.ident'].isin(['P2_0', 'P3_0', 'P4_0', 'P5_0', 'P6_0', 'P7_0', 'P8_0'])].copy()
# Create mdata_test dataset (we used 1 sample for test model quality)
mdata_test = mdata[mdata.obs['orig.ident'].isin(['P1_0'])].copy()

[5]:

# Extract RNA expression data from mdata_test for future predictions
adata_test_rna = mdata_test.mod['rna']

Default scEve model training#

[6]:

# Train default scEve model using mdata_train dataset
scparadise.sceve.train(
    mdata_train,
    path = '',
    first_modality_name = 'rna',
    second_modality_name = 'adt',
    detailed_annotation = 'celltype_l3',
    model_name = 'model_scEve',
    eval_metric = ['rmse']
)

Device: cuda
Task: Predict 'adt' from 'rna'
Number of first modality features: 635
Number of second modality features: 224

Dataset split:
Train dataset contains: 37359 cells, it is 80.0 % of input dataset
Validation dataset contains: 9340 cells, it is 20.0 % of input dataset

Training scEve model:  22%|████████████████▊                                                             | 43/200 [02:40<09:45,  3.73s/it]

Early stopping triggered! Best score: 0.3038
Training completed!
Model saved to model_scEve

Check model quality#

[7]:

# Predict surface proteins using scEve model
mdata_pred = scparadise.sceve.predict(
    adata_test_rna,
    path_model = 'model_scEve',
    return_mdata = True # default value
)

Feature prediction using the 2nd version of the scEve model
scEve model loaded from model_scEve
Feature alignment:
Model features: 635
New data features: 635
Matched features: 635 (100.0%)

Predicting: 100%|████████████████████████████████████████████████████████████████████████████████████████| 25/25 [00:00<00:00, 368.90it/s]

[8]:

# Add umap coordinates
mdata_pred.obsm['X_umap'] = adata_test_rna.obsm['X_umap']

[9]:

# List of proteins for visualization
proteins_observed = ['adt_CLEC12A', 'adt_CD4-1', 'adt_CD8', 'adt_IgM', 'adt_CD16']

# Create list of imputed proteins
proteins_imputed = []
for i in proteins_observed:
    i = i + '_pred'
    proteins_imputed.append(i)

[10]:

# Proteins visualization
proteins_dict = {
    "Observed": proteins_observed,
    "Predicted": proteins_imputed
}
# Choose a color map for protein abundance visualization
cmap = 'bwr'
# Make Axes
# Number of needed rows and columns (based on the row with the most columns)
nrow = len(proteins_dict)
ncol = max([len(vs) for vs in proteins_dict.values()])
fig, axs = plt.subplots(nrow, ncol, figsize=(3 * ncol, 3 * nrow))
# Plot expression for every marker on the corresponding Axes object
for row_idx, (prots, markers) in enumerate(proteins_dict.items()):
    col_idx = 0
    for marker in markers:
        ax = axs[row_idx, col_idx]
        if prots == 'Observed':
            mu.pl.umap(mdata_test, color=marker, cmap = cmap, ax=ax, vmax = 'p99', show=False, frameon=False)
        else:
            mu.pl.umap(mdata_pred, color=marker, cmap = cmap, ax=ax, vmax = 'p99', show=False, frameon=False)
        # Add cell type as row label - here we simply add it as ylabel of
        # the first Axes object in the row
        if col_idx == 0:
            # We disabled axis drawing in UMAP to have plots without background and border
            # so we need to re-enable axis to plot the ylabel
            ax.axis("on")
            ax.tick_params(
                top="off",
                bottom="off",
                left="off",
                right="off",
                labelleft="on",
                labelbottom="off",
            )
            ax.set_ylabel(prots + "\n", rotation=90, fontsize=14)
            ax.set(frame_on=False)
        col_idx += 1
    # Remove unused column Axes in the current row
    while col_idx < ncol:
        axs[row_idx, col_idx].remove()
        col_idx += 1
# Alignment within the Figure
fig.tight_layout()
../../../_images/tutorials_notebooks_scEve_scEve_train_14_0.png

The obtained model reliably predicts the main protein markers. However, there are proteins such as CD133, which the default model cannot predict.

[11]:

# Check prediction results
df = scparadise.scnoah.report_reg(
    adata_prot = mdata_test.mod['adt'],
    adata_pred_prot = mdata_pred.mod['adt']
)
df

[11]:
EVS r2_score RMSE MedianAE MeanAE
score 0.283 0.235 0.328 0.201 0.248
EVS/r2_score higher value - better prediction
RMSE/MedianAE/MeanAE lower value - better prediction 
[12]:

# Add regression status using 5 metrics
for metric in ['RMSE', 'MeanAE', 'MedianAE', 'EVS', 'r2_score']:
    scparadise.scnoah.regres_status(adata_prot = mdata_test.mod['adt'],
                                    adata_pred_prot = mdata_pred.mod['adt'],
                                    metric=metric)
    mdata_pred.obs[f'regres_status_{metric}'] = mdata_pred.mod['adt'].obs[f'regres_status_{metric}']

[13]:

# Visualize metrics
mu.pl.umap(
    mdata_pred,
    color=[
        'regres_status_RMSE',
        'regres_status_MeanAE',
        'regres_status_MedianAE',
        'regres_status_EVS',
        'regres_status_r2_score'
    ],
    frameon = False,
    cmap = 'bwr',
    ncols = 5,
    wspace = 0.15,
    hspace = 0.1
)
../../../_images/tutorials_notebooks_scEve_scEve_train_18_0.png

Based on a per cell type regression metrics score, it is possible to draw conclusions about the uniformity of protein prediction across different cell types in the dataset.

[14]:

# Find correlations between imputed and observed proteins
df_corr = pd.DataFrame(columns=['Pearson coef', 'p-value'])
for i in mdata_test.mod['adt'].var_names.tolist():
    person_coef = scparadise.scnoah.pearson_coef(
        adata = mdata_test.mod['adt'],
        adata_pred = mdata_pred.mod['adt'],
        feature = i,
        feature_pred = i + '_pred',
        print_res = False
    )
    df_corr.loc[i] = [person_coef['Pearson coefficient'], person_coef['p-value']]
df_corr['p-value'] = df_corr['p-value'].astype('float64')
df_corr.sort_values('Pearson coef', ascending=False).head(5)

[14]:
Pearson coef p-value
adt_CLEC12A 0.963 0.0
adt_CD64 0.943 0.0
adt_CD11b-2 0.927 0.0
adt_CD8a 0.922 0.0
adt_CD4-1 0.919 0.0 
[15]:

# Find highly correlated protein 'Pearson coef' > 0.5
df_high_corr = df_corr[df_corr['Pearson coef'] >= 0.5].copy()
print('Number of proteins with Pearson coefficient >= 0.5:', len(df_high_corr))
df_high_corr.sort_values(by = 'Pearson coef', ascending=False).head(5)

Number of proteins with Pearson coefficient >= 0.5: 107

[15]:
Pearson coef p-value
adt_CLEC12A 0.963 0.0
adt_CD64 0.943 0.0
adt_CD11b-2 0.927 0.0
adt_CD8a 0.922 0.0
adt_CD4-1 0.919 0.0 
[16]:

# Save mdata with imputed proteins
mdata_pred.write_h5mu('mdata_pred_untuned.h5mu')

Hyperparameters tuning#

[17]:

scparadise.sceve.hyperparameter_tuning(
    mdata_train,
    path = '',
    first_modality_name = 'rna',
    second_modality_name = 'adt',
    detailed_annotation = 'celltype_l3',
    random_state = 42,
    model_name = 'model_scEve_tuning',
    eval_metric = 'rmse'
)

Device: cuda
Task: Predict 'adt' from 'rna'
Start model optimization using optuna...

[I 2026-01-27 15:09:02,051] A new study created in RDB with name: study

Fold 1 finished with rmse value = 0.305253
Fold 2 finished with rmse value = 0.305124
Fold 3 finished with rmse value = 0.302512
Fold 4 finished with rmse value = 0.303252
Fold 5 finished with rmse value = 0.302669

[I 2026-01-27 15:15:16,961] Trial 0 finished with value: 0.3037621677516043 and parameters: {'nc': 4, 'nb': 4, 'nh': 8, 'ed_nh_ratio': 32, 'ff_hd': 512, 'regressor_hd': 512, 'dropout': 0.1, 'lr': 0.0001, 'weight_decay': 0.0001, 'batch_size': 128, 'patience': 10, 'epochs': 200, 'use_augmentation': True, 'aug_probability': 0.5, 'prob': 0.15, 'noise_std': 0.1, 'dropout_aug': 0.1, 'alpha': 0.2}. Best is trial 0 with value: 0.3037621677516043.

Fold 1 finished with rmse value = 0.305573
Fold 2 finished with rmse value = 0.305310
Fold 3 finished with rmse value = 0.302961
Fold 4 finished with rmse value = 0.305279
Fold 5 finished with rmse value = 0.303843

[I 2026-01-27 15:18:11,086] Trial 1 finished with value: 0.30459307118884127 and parameters: {'nc': 8, 'nb': 2, 'nh': 14, 'ed_nh_ratio': 24, 'ff_hd': 896, 'regressor_hd': 384, 'dropout': 0.03763045414850437, 'lr': 0.0005928224822180558, 'weight_decay': 1.5348951413904243e-05, 'batch_size': 1920, 'patience': 20, 'epochs': 95, 'use_augmentation': True, 'aug_probability': 0.9454515360844692, 'prob': 0.3803418496597643, 'noise_std': 0.28025129508819896, 'dropout_aug': 0.3702167920763811, 'alpha': 0.3929550245565865}. Best is trial 0 with value: 0.3037621677516043.

Fold 1 finished with rmse value = 0.303743
Fold 2 finished with rmse value = 0.303688
Fold 3 finished with rmse value = 0.302110
Fold 4 finished with rmse value = 0.302756
Fold 5 finished with rmse value = 0.302516

[I 2026-01-27 15:24:03,299] Trial 2 finished with value: 0.3029626403968464 and parameters: {'nc': 12, 'nb': 4, 'nh': 16, 'ed_nh_ratio': 12, 'ff_hd': 640, 'regressor_hd': 896, 'dropout': 0.07561808956396764, 'lr': 0.0006469817217609699, 'weight_decay': 3.955206943377081e-06, 'batch_size': 1920, 'patience': 25, 'epochs': 100, 'use_augmentation': False, 'aug_probability': 0.3850621064558062, 'prob': 0.21174366222409363, 'noise_std': 0.07877596280524585, 'dropout_aug': 0.11826032713600143, 'alpha': 0.2882706045361326}. Best is trial 2 with value: 0.3029626403968464.

Fold 1 finished with rmse value = 0.337210
Fold 2 finished with rmse value = 0.336620
Fold 3 finished with rmse value = 0.334800

[I 2026-01-27 15:25:32,302] Trial 3 pruned.

Fold 1 finished with rmse value = 0.305121
Fold 2 finished with rmse value = 0.305617
Fold 3 finished with rmse value = 0.302486
Fold 4 finished with rmse value = 0.303614
Fold 5 finished with rmse value = 0.303490

[I 2026-01-27 15:30:38,839] Trial 4 finished with value: 0.30406551616539473 and parameters: {'nc': 6, 'nb': 8, 'nh': 4, 'ed_nh_ratio': 24, 'ff_hd': 384, 'regressor_hd': 640, 'dropout': 0.0986994681091324, 'lr': 0.001979323316973329, 'weight_decay': 0.004073522663731959, 'batch_size': 1984, 'patience': 20, 'epochs': 130, 'use_augmentation': True, 'aug_probability': 0.20962012288865706, 'prob': 0.3625869413749469, 'noise_std': 0.3498370911485513, 'dropout_aug': 0.04207264457872171, 'alpha': 0.08851573354758427}. Best is trial 2 with value: 0.3029626403968464.

Fold 1 finished with rmse value = 0.331077

[I 2026-01-27 15:31:29,603] Trial 5 pruned.

Fold 1 finished with rmse value = 0.305870

[I 2026-01-27 15:36:12,589] Trial 6 pruned.

Fold 1 finished with rmse value = 0.303997
Fold 2 finished with rmse value = 0.304070
Fold 3 finished with rmse value = 0.302001
Fold 4 finished with rmse value = 0.303358
Fold 5 finished with rmse value = 0.303676

[I 2026-01-27 15:41:11,270] Trial 7 finished with value: 0.30342051888470856 and parameters: {'nc': 2, 'nb': 4, 'nh': 6, 'ed_nh_ratio': 12, 'ff_hd': 1024, 'regressor_hd': 896, 'dropout': 0.188098346643424, 'lr': 0.000546573273539246, 'weight_decay': 7.647442143296432e-06, 'batch_size': 1792, 'patience': 25, 'epochs': 155, 'use_augmentation': False, 'aug_probability': 0.47453169447113386, 'prob': 0.3818800136869987, 'noise_std': 0.0306456427727019, 'dropout_aug': 0.13408012428070784, 'alpha': 0.09226656321363347}. Best is trial 2 with value: 0.3029626403968464.

Fold 1 finished with rmse value = 0.304963
Fold 2 finished with rmse value = 0.304799
Fold 3 finished with rmse value = 0.303008

[I 2026-01-27 15:45:58,501] Trial 8 pruned.

Fold 1 finished with rmse value = 0.303850
Fold 2 finished with rmse value = 0.303664
Fold 3 finished with rmse value = 0.302650

[I 2026-01-27 15:48:17,396] Trial 9 pruned.

Fold 1 finished with rmse value = 0.305632
Fold 2 finished with rmse value = 0.304383
Fold 3 finished with rmse value = 0.304774

[I 2026-01-27 15:51:54,531] Trial 10 pruned.

Fold 1 finished with rmse value = 0.303184
Fold 2 finished with rmse value = 0.304385
Fold 3 finished with rmse value = 0.301891
Fold 4 finished with rmse value = 0.302960
Fold 5 finished with rmse value = 0.302195

[I 2026-01-27 15:59:15,272] Trial 11 finished with value: 0.3029231803973249 and parameters: {'nc': 12, 'nb': 4, 'nh': 12, 'ed_nh_ratio': 12, 'ff_hd': 1024, 'regressor_hd': 1024, 'dropout': 0.1979919189258571, 'lr': 0.001178507430650213, 'weight_decay': 1.284923993832759e-06, 'batch_size': 1600, 'patience': 30, 'epochs': 140, 'use_augmentation': False, 'aug_probability': 0.4759376384880973, 'prob': 0.25858059151630786, 'noise_std': 0.008058179685640331, 'dropout_aug': 0.20372262565563756, 'alpha': 0.1673293179241103}. Best is trial 11 with value: 0.3029231803973249.

Fold 1 finished with rmse value = 0.303183
Fold 2 finished with rmse value = 0.304343
Fold 3 finished with rmse value = 0.301797
Fold 4 finished with rmse value = 0.302180
Fold 5 finished with rmse value = 0.301829

[I 2026-01-27 16:06:58,484] Trial 12 finished with value: 0.30266661031236486 and parameters: {'nc': 12, 'nb': 4, 'nh': 12, 'ed_nh_ratio': 12, 'ff_hd': 768, 'regressor_hd': 1024, 'dropout': 0.19727084567207281, 'lr': 0.002023251214147288, 'weight_decay': 1.0884763255101343e-06, 'batch_size': 1536, 'patience': 30, 'epochs': 130, 'use_augmentation': False, 'aug_probability': 0.4016977814443879, 'prob': 0.2554802611578702, 'noise_std': 8.274994149432696e-05, 'dropout_aug': 0.23296475951508355, 'alpha': 0.2918068716241683}. Best is trial 12 with value: 0.30266661031236486.

Fold 1 finished with rmse value = 0.331892
Fold 2 finished with rmse value = 0.331240
Fold 3 finished with rmse value = 0.329188

[I 2026-01-27 16:12:51,775] Trial 13 pruned.

Fold 1 finished with rmse value = 0.302933
Fold 2 finished with rmse value = 0.303648
Fold 3 finished with rmse value = 0.301398
Fold 4 finished with rmse value = 0.302514
Fold 5 finished with rmse value = 0.301481

[I 2026-01-27 16:25:29,992] Trial 14 finished with value: 0.30239454624821277 and parameters: {'nc': 14, 'nb': 5, 'nh': 12, 'ed_nh_ratio': 16, 'ff_hd': 768, 'regressor_hd': 1024, 'dropout': 0.16355937184075683, 'lr': 0.0028341063229509076, 'weight_decay': 4.3152808526725466e-05, 'batch_size': 960, 'patience': 30, 'epochs': 150, 'use_augmentation': False, 'aug_probability': 0.1381896202659833, 'prob': 0.301916710185413, 'noise_std': 0.17703189731127722, 'dropout_aug': 0.3009574491873872, 'alpha': 0.2961564560371936}. Best is trial 14 with value: 0.30239454624821277.

Fold 1 finished with rmse value = 0.303964
Fold 2 finished with rmse value = 0.305583
Fold 3 finished with rmse value = 0.302601

[I 2026-01-27 16:35:13,919] Trial 15 pruned.

Fold 1 finished with rmse value = 0.303255
Fold 2 finished with rmse value = 0.303344
Fold 3 finished with rmse value = 0.302042

[I 2026-01-27 16:37:52,997] Trial 16 pruned.

Fold 1 finished with rmse value = 0.302724
Fold 2 finished with rmse value = 0.303129
Fold 3 finished with rmse value = 0.300565
Fold 4 finished with rmse value = 0.301767
Fold 5 finished with rmse value = 0.301012

[I 2026-01-27 16:49:11,894] Trial 17 finished with value: 0.30183928303626656 and parameters: {'nc': 10, 'nb': 5, 'nh': 14, 'ed_nh_ratio': 24, 'ff_hd': 768, 'regressor_hd': 1024, 'dropout': 0.13229250922909938, 'lr': 0.0030064462948243445, 'weight_decay': 0.0001035007677377549, 'batch_size': 1088, 'patience': 15, 'epochs': 85, 'use_augmentation': False, 'aug_probability': 0.6114583392977266, 'prob': 0.31285710166077113, 'noise_std': 0.15334364683542462, 'dropout_aug': 0.192281139163962, 'alpha': 0.24832145471282907}. Best is trial 17 with value: 0.30183928303626656.

Fold 1 finished with rmse value = 0.303566
Fold 2 finished with rmse value = 0.304152
Fold 3 finished with rmse value = 0.300583
Fold 4 finished with rmse value = 0.302795
Fold 5 finished with rmse value = 0.303277

[I 2026-01-27 17:01:00,329] Trial 18 finished with value: 0.30287440240851626 and parameters: {'nc': 10, 'nb': 6, 'nh': 14, 'ed_nh_ratio': 28, 'ff_hd': 384, 'regressor_hd': 896, 'dropout': 0.13007292220743427, 'lr': 0.003814119152243988, 'weight_decay': 0.00010857804161050793, 'batch_size': 1152, 'patience': 10, 'epochs': 70, 'use_augmentation': True, 'aug_probability': 0.6806557739508187, 'prob': 0.3140269794099323, 'noise_std': 0.14822882660817374, 'dropout_aug': 0.18487539482522589, 'alpha': 0.25259634077497195}. Best is trial 17 with value: 0.30183928303626656.

Fold 1 finished with rmse value = 0.304692

[I 2026-01-27 17:02:24,886] Trial 19 pruned.

Fold 1 finished with rmse value = 0.329613

[I 2026-01-27 17:06:26,779] Trial 20 pruned.

Fold 1 finished with rmse value = 0.304023

[I 2026-01-27 17:08:56,753] Trial 21 pruned.

Fold 1 finished with rmse value = 0.302591
Fold 2 finished with rmse value = 0.304081
Fold 3 finished with rmse value = 0.302026

[I 2026-01-27 17:16:12,262] Trial 22 pruned.

Fold 1 finished with rmse value = 0.302914
Fold 2 finished with rmse value = 0.302628
Fold 3 finished with rmse value = 0.301664
Fold 4 finished with rmse value = 0.301177
Fold 5 finished with rmse value = 0.300859

[I 2026-01-27 17:23:55,215] Trial 23 finished with value: 0.301848600027847 and parameters: {'nc': 16, 'nb': 3, 'nh': 12, 'ed_nh_ratio': 8, 'ff_hd': 896, 'regressor_hd': 896, 'dropout': 0.17741765681198113, 'lr': 0.0009791334995533276, 'weight_decay': 4.719478685642667e-05, 'batch_size': 640, 'patience': 30, 'epochs': 130, 'use_augmentation': False, 'aug_probability': 0.3801567091404545, 'prob': 0.24005145716801196, 'noise_std': 0.3678816327541079, 'dropout_aug': 0.2544144863514495, 'alpha': 0.2169800883219556}. Best is trial 17 with value: 0.30183928303626656.

Fold 1 finished with rmse value = 0.306396

[I 2026-01-27 17:24:16,602] Trial 24 pruned.

Fold 1 finished with rmse value = 0.311715
Fold 2 finished with rmse value = 0.308053
Fold 3 finished with rmse value = 0.305035

[I 2026-01-27 17:35:26,423] Trial 25 pruned.

Fold 1 finished with rmse value = 0.304357

[I 2026-01-27 17:37:12,077] Trial 26 pruned.

Fold 1 finished with rmse value = 0.303873
Fold 2 finished with rmse value = 0.303840
Fold 3 finished with rmse value = 0.302166

[I 2026-01-27 17:42:16,439] Trial 27 pruned.

Fold 1 finished with rmse value = 0.303944

[I 2026-01-27 17:42:50,952] Trial 28 pruned.

Fold 1 finished with rmse value = 0.304855

[I 2026-01-27 17:45:24,921] Trial 29 pruned.

Fold 1 finished with rmse value = 0.304903

[I 2026-01-27 17:46:14,509] Trial 30 pruned.

Fold 1 finished with rmse value = 0.303290
Fold 2 finished with rmse value = 0.303399
Fold 3 finished with rmse value = 0.301065
Fold 4 finished with rmse value = 0.301886
Fold 5 finished with rmse value = 0.301848

[I 2026-01-27 17:55:34,708] Trial 31 finished with value: 0.30229785813762605 and parameters: {'nc': 12, 'nb': 5, 'nh': 12, 'ed_nh_ratio': 12, 'ff_hd': 768, 'regressor_hd': 1024, 'dropout': 0.18337669959354821, 'lr': 0.002190479837052409, 'weight_decay': 8.720674411630758e-06, 'batch_size': 640, 'patience': 30, 'epochs': 130, 'use_augmentation': False, 'aug_probability': 0.3720095073767704, 'prob': 0.2531786112005286, 'noise_std': 0.06586203396540923, 'dropout_aug': 0.2143525949500101, 'alpha': 0.27598294959157554}. Best is trial 17 with value: 0.30183928303626656.

Fold 1 finished with rmse value = 0.303203
Fold 2 finished with rmse value = 0.303057
Fold 3 finished with rmse value = 0.301427
Fold 4 finished with rmse value = 0.302955
Fold 5 finished with rmse value = 0.301835

[I 2026-01-27 18:02:07,616] Trial 32 finished with value: 0.3024952970465537 and parameters: {'nc': 10, 'nb': 5, 'nh': 12, 'ed_nh_ratio': 12, 'ff_hd': 640, 'regressor_hd': 896, 'dropout': 0.18350619382216765, 'lr': 0.0008006982307977489, 'weight_decay': 1.4479315371924425e-05, 'batch_size': 640, 'patience': 30, 'epochs': 145, 'use_augmentation': False, 'aug_probability': 0.2665025164750561, 'prob': 0.1783048719839512, 'noise_std': 0.07195245258008759, 'dropout_aug': 0.2029370707974067, 'alpha': 0.3972981345272668}. Best is trial 17 with value: 0.30183928303626656.

Fold 1 finished with rmse value = 0.301851
Fold 2 finished with rmse value = 0.301972
Fold 3 finished with rmse value = 0.300852
Fold 4 finished with rmse value = 0.301662
Fold 5 finished with rmse value = 0.301317

[I 2026-01-27 18:13:13,081] Trial 33 finished with value: 0.3015305161970789 and parameters: {'nc': 14, 'nb': 5, 'nh': 14, 'ed_nh_ratio': 8, 'ff_hd': 768, 'regressor_hd': 1024, 'dropout': 0.14549178877131058, 'lr': 0.003336073832961678, 'weight_decay': 4.583217405749186e-06, 'batch_size': 512, 'patience': 30, 'epochs': 135, 'use_augmentation': False, 'aug_probability': 0.34965126361055054, 'prob': 0.24464235197790035, 'noise_std': 0.1785180610368859, 'dropout_aug': 0.1600614147562162, 'alpha': 0.2712587177421696}. Best is trial 33 with value: 0.3015305161970789.

Fold 1 finished with rmse value = 0.302151
Fold 2 finished with rmse value = 0.304993
Fold 3 finished with rmse value = 0.302402

[I 2026-01-27 18:20:27,273] Trial 34 pruned.

Fold 1 finished with rmse value = 0.303205
Fold 2 finished with rmse value = 0.303523
Fold 3 finished with rmse value = 0.301492
Fold 4 finished with rmse value = 0.301932
Fold 5 finished with rmse value = 0.301229

[I 2026-01-27 18:26:41,026] Trial 35 finished with value: 0.30227617888835784 and parameters: {'nc': 10, 'nb': 4, 'nh': 16, 'ed_nh_ratio': 8, 'ff_hd': 640, 'regressor_hd': 1024, 'dropout': 0.14540794549324818, 'lr': 0.0014222136195310418, 'weight_decay': 9.602433297343166e-06, 'batch_size': 256, 'patience': 20, 'epochs': 135, 'use_augmentation': True, 'aug_probability': 0.41749507398955266, 'prob': 0.20864964851341922, 'noise_std': 0.13087778110723486, 'dropout_aug': 0.14398753830271888, 'alpha': 0.27135512087936375}. Best is trial 33 with value: 0.3015305161970789.

Fold 1 finished with rmse value = 0.301683
Fold 2 finished with rmse value = 0.302127
Fold 3 finished with rmse value = 0.300457
Fold 4 finished with rmse value = 0.300606
Fold 5 finished with rmse value = 0.300477

[I 2026-01-27 18:40:41,482] Trial 36 finished with value: 0.30107012676884415 and parameters: {'nc': 8, 'nb': 2, 'nh': 16, 'ed_nh_ratio': 8, 'ff_hd': 640, 'regressor_hd': 896, 'dropout': 0.1450582342986098, 'lr': 0.0014032134642617382, 'weight_decay': 2.1012526174747154e-06, 'batch_size': 64, 'patience': 20, 'epochs': 115, 'use_augmentation': True, 'aug_probability': 0.4262016882932723, 'prob': 0.16812702190743048, 'noise_std': 0.12268347456014356, 'dropout_aug': 0.13264736381340325, 'alpha': 0.2274979018971504}. Best is trial 36 with value: 0.30107012676884415.

Fold 1 finished with rmse value = 0.303994

[I 2026-01-27 18:41:59,672] Trial 37 pruned.

Fold 1 finished with rmse value = 0.304460

[I 2026-01-27 18:42:28,883] Trial 38 pruned.

Fold 1 finished with rmse value = 0.304455

[I 2026-01-27 18:43:20,767] Trial 39 pruned.

Fold 1 finished with rmse value = 0.302104
Fold 2 finished with rmse value = 0.302396
Fold 3 finished with rmse value = 0.300226
Fold 4 finished with rmse value = 0.301247
Fold 5 finished with rmse value = 0.301611

[I 2026-01-27 18:59:30,824] Trial 40 finished with value: 0.3015168011407278 and parameters: {'nc': 6, 'nb': 3, 'nh': 16, 'ed_nh_ratio': 12, 'ff_hd': 256, 'regressor_hd': 256, 'dropout': 0.09698696773144716, 'lr': 0.004450512353373104, 'weight_decay': 0.0026240112889261385, 'batch_size': 64, 'patience': 25, 'epochs': 55, 'use_augmentation': True, 'aug_probability': 0.3389170274716813, 'prob': 0.15614393423841286, 'noise_std': 0.21678345909053826, 'dropout_aug': 0.13249443604259123, 'alpha': 0.24003417547975128}. Best is trial 36 with value: 0.30107012676884415.

Fold 1 finished with rmse value = 0.304430
Fold 2 finished with rmse value = 0.303527
Fold 3 finished with rmse value = 0.302753

[I 2026-01-27 19:10:12,745] Trial 41 pruned.

Fold 1 finished with rmse value = 0.305782

[I 2026-01-27 19:11:15,040] Trial 42 pruned.

Fold 1 finished with rmse value = 0.301283
Fold 2 finished with rmse value = 0.301086
Fold 3 finished with rmse value = 0.298322
Fold 4 finished with rmse value = 0.300311
Fold 5 finished with rmse value = 0.299521

[I 2026-01-27 19:32:50,492] Trial 43 finished with value: 0.30010433923270374 and parameters: {'nc': 8, 'nb': 3, 'nh': 14, 'ed_nh_ratio': 8, 'ff_hd': 256, 'regressor_hd': 512, 'dropout': 0.10651313633311207, 'lr': 0.0033348407355143434, 'weight_decay': 0.007996432493118261, 'batch_size': 64, 'patience': 20, 'epochs': 90, 'use_augmentation': True, 'aug_probability': 0.3241856434494987, 'prob': 0.10610993869597891, 'noise_std': 0.11615868747307673, 'dropout_aug': 0.1691310199130744, 'alpha': 0.21488667686479931}. Best is trial 43 with value: 0.30010433923270374.

Fold 1 finished with rmse value = 0.303589
Fold 2 finished with rmse value = 0.304303
Fold 3 finished with rmse value = 0.302350

[I 2026-01-27 19:42:22,888] Trial 44 pruned.

Fold 1 finished with rmse value = 0.313557

[I 2026-01-27 19:44:02,007] Trial 45 pruned.

Fold 1 finished with rmse value = 0.304233

[I 2026-01-27 19:45:11,339] Trial 46 pruned.

Fold 1 finished with rmse value = 0.303511

[I 2026-01-27 19:52:05,632] Trial 47 pruned.

Fold 1 finished with rmse value = 0.302257
Fold 2 finished with rmse value = 0.301736
Fold 3 finished with rmse value = 0.299315
Fold 4 finished with rmse value = 0.300518
Fold 5 finished with rmse value = 0.299450

[I 2026-01-27 19:56:37,666] Trial 48 finished with value: 0.3006552497374334 and parameters: {'nc': 8, 'nb': 2, 'nh': 6, 'ed_nh_ratio': 12, 'ff_hd': 128, 'regressor_hd': 768, 'dropout': 0.10896573246590859, 'lr': 0.004473259719716826, 'weight_decay': 0.006450838922550928, 'batch_size': 320, 'patience': 25, 'epochs': 55, 'use_augmentation': True, 'aug_probability': 0.4318576362186528, 'prob': 0.14186303183174742, 'noise_std': 0.26496046435824583, 'dropout_aug': 0.05501572790672232, 'alpha': 0.32001801431201493}. Best is trial 43 with value: 0.30010433923270374.

Fold 1 finished with rmse value = 0.302461
Fold 2 finished with rmse value = 0.302801
Fold 3 finished with rmse value = 0.301141
Fold 4 finished with rmse value = 0.300944
Fold 5 finished with rmse value = 0.301551

[I 2026-01-27 19:59:54,060] Trial 49 finished with value: 0.30177975725552003 and parameters: {'nc': 8, 'nb': 1, 'nh': 6, 'ed_nh_ratio': 12, 'ff_hd': 128, 'regressor_hd': 512, 'dropout': 0.05866431372783962, 'lr': 0.004446485589494753, 'weight_decay': 0.0022286661359174624, 'batch_size': 384, 'patience': 25, 'epochs': 50, 'use_augmentation': True, 'aug_probability': 0.9752513731624304, 'prob': 0.14533129480654394, 'noise_std': 0.2627316803061565, 'dropout_aug': 0.0017738762314724182, 'alpha': 0.3658623799914501}. Best is trial 43 with value: 0.30010433923270374.

Fold 1 finished with rmse value = 0.302696
Fold 2 finished with rmse value = 0.302828
Fold 3 finished with rmse value = 0.300440
Fold 4 finished with rmse value = 0.301693
Fold 5 finished with rmse value = 0.301524

[I 2026-01-27 20:06:23,529] Trial 50 finished with value: 0.3018361935538085 and parameters: {'nc': 6, 'nb': 2, 'nh': 4, 'ed_nh_ratio': 12, 'ff_hd': 128, 'regressor_hd': 256, 'dropout': 0.10798093810390895, 'lr': 0.004860518052026869, 'weight_decay': 0.005969115007587137, 'batch_size': 192, 'patience': 25, 'epochs': 60, 'use_augmentation': True, 'aug_probability': 0.43786319097448495, 'prob': 0.05371616502043705, 'noise_std': 0.3085635562658727, 'dropout_aug': 0.03220831126418499, 'alpha': 0.0007232867497535544}. Best is trial 43 with value: 0.30010433923270374.

Fold 1 finished with rmse value = 0.302114
Fold 2 finished with rmse value = 0.302760
Fold 3 finished with rmse value = 0.300430
Fold 4 finished with rmse value = 0.301507
Fold 5 finished with rmse value = 0.301552

[I 2026-01-27 20:09:40,614] Trial 51 finished with value: 0.3016723290257092 and parameters: {'nc': 8, 'nb': 1, 'nh': 6, 'ed_nh_ratio': 12, 'ff_hd': 128, 'regressor_hd': 512, 'dropout': 0.053429800627949095, 'lr': 0.0045043564021402275, 'weight_decay': 0.0022741525302574796, 'batch_size': 384, 'patience': 25, 'epochs': 50, 'use_augmentation': True, 'aug_probability': 0.33431513443994076, 'prob': 0.1441217456505338, 'noise_std': 0.25786639249727744, 'dropout_aug': 0.002900697900563059, 'alpha': 0.35321906246585216}. Best is trial 43 with value: 0.30010433923270374.

Fold 1 finished with rmse value = 0.303093
Fold 2 finished with rmse value = 0.302978
Fold 3 finished with rmse value = 0.301200
Fold 4 finished with rmse value = 0.301677
Fold 5 finished with rmse value = 0.302001

[I 2026-01-27 20:13:59,153] Trial 52 finished with value: 0.3021899034868451 and parameters: {'nc': 8, 'nb': 1, 'nh': 4, 'ed_nh_ratio': 12, 'ff_hd': 128, 'regressor_hd': 512, 'dropout': 0.06666824521195162, 'lr': 0.0025671367010626335, 'weight_decay': 0.005323255937889759, 'batch_size': 320, 'patience': 25, 'epochs': 60, 'use_augmentation': True, 'aug_probability': 0.17267144139835944, 'prob': 0.16849413797588148, 'noise_std': 0.2656464761540934, 'dropout_aug': 0.05125912854274348, 'alpha': 0.3190901662733259}. Best is trial 43 with value: 0.30010433923270374.

Fold 1 finished with rmse value = 0.305701
Fold 2 finished with rmse value = 0.304901
Fold 3 finished with rmse value = 0.302026

[I 2026-01-27 20:17:56,261] Trial 53 pruned.

Fold 1 finished with rmse value = 0.302850
Fold 2 finished with rmse value = 0.303198
Fold 3 finished with rmse value = 0.301127

[I 2026-01-27 20:20:07,014] Trial 54 pruned.

Fold 1 finished with rmse value = 0.303507

[I 2026-01-27 20:21:34,375] Trial 55 pruned.

Fold 1 finished with rmse value = 0.304319
Fold 2 finished with rmse value = 0.304243
Fold 3 finished with rmse value = 0.302085

[I 2026-01-27 20:23:34,250] Trial 56 pruned.

Fold 1 finished with rmse value = 0.302083
Fold 2 finished with rmse value = 0.301653
Fold 3 finished with rmse value = 0.300473
Fold 4 finished with rmse value = 0.300568
Fold 5 finished with rmse value = 0.301197

[I 2026-01-27 20:42:11,171] Trial 57 finished with value: 0.30119481836591866 and parameters: {'nc': 10, 'nb': 3, 'nh': 8, 'ed_nh_ratio': 16, 'ff_hd': 128, 'regressor_hd': 768, 'dropout': 0.0883689121599533, 'lr': 0.002564781319520247, 'weight_decay': 0.0067381534235856026, 'batch_size': 64, 'patience': 20, 'epochs': 65, 'use_augmentation': True, 'aug_probability': 0.24909788114419712, 'prob': 0.15736916669198453, 'noise_std': 0.20930244781402593, 'dropout_aug': 0.017528351889188967, 'alpha': 0.3794366792913043}. Best is trial 43 with value: 0.30010433923270374.

Fold 1 finished with rmse value = 0.301142
Fold 2 finished with rmse value = 0.301640
Fold 3 finished with rmse value = 0.300933
Fold 4 finished with rmse value = 0.301622
Fold 5 finished with rmse value = 0.299741

[I 2026-01-27 21:01:51,087] Trial 58 finished with value: 0.3010156937012313 and parameters: {'nc': 2, 'nb': 3, 'nh': 8, 'ed_nh_ratio': 16, 'ff_hd': 256, 'regressor_hd': 768, 'dropout': 0.09126608744083142, 'lr': 0.0024830322744238717, 'weight_decay': 0.0066523966780755285, 'batch_size': 64, 'patience': 20, 'epochs': 65, 'use_augmentation': True, 'aug_probability': 0.23253467572731462, 'prob': 0.1554167254345158, 'noise_std': 0.21853998982355016, 'dropout_aug': 0.11873823575253412, 'alpha': 0.3851131612075539}. Best is trial 43 with value: 0.30010433923270374.

Fold 1 finished with rmse value = 0.302433
Fold 2 finished with rmse value = 0.302673
Fold 3 finished with rmse value = 0.300937

[I 2026-01-27 21:12:24,144] Trial 59 pruned.

Fold 1 finished with rmse value = 0.302556
Fold 2 finished with rmse value = 0.302462
Fold 3 finished with rmse value = 0.300583

[I 2026-01-27 21:16:52,472] Trial 60 pruned.

Fold 1 finished with rmse value = 0.300921
Fold 2 finished with rmse value = 0.301007
Fold 3 finished with rmse value = 0.299573
Fold 4 finished with rmse value = 0.300622
Fold 5 finished with rmse value = 0.298831

[I 2026-01-27 21:39:21,585] Trial 61 finished with value: 0.3001908536596173 and parameters: {'nc': 4, 'nb': 3, 'nh': 8, 'ed_nh_ratio': 20, 'ff_hd': 384, 'regressor_hd': 768, 'dropout': 0.09784225528863587, 'lr': 0.0032440233597802633, 'weight_decay': 0.005210905212732526, 'batch_size': 64, 'patience': 20, 'epochs': 75, 'use_augmentation': True, 'aug_probability': 0.199263458114284, 'prob': 0.11466276349718905, 'noise_std': 0.1950555635054267, 'dropout_aug': 0.1453860500006737, 'alpha': 0.3316746811534767}. Best is trial 43 with value: 0.30010433923270374.

Fold 1 finished with rmse value = 0.300849
Fold 2 finished with rmse value = 0.302514
Fold 3 finished with rmse value = 0.299904
Fold 4 finished with rmse value = 0.300623
Fold 5 finished with rmse value = 0.300083

[I 2026-01-27 21:56:23,736] Trial 62 finished with value: 0.3007946815515693 and parameters: {'nc': 4, 'nb': 3, 'nh': 8, 'ed_nh_ratio': 20, 'ff_hd': 384, 'regressor_hd': 768, 'dropout': 0.09694697068584207, 'lr': 0.0019713003841161122, 'weight_decay': 0.006381647557973412, 'batch_size': 64, 'patience': 20, 'epochs': 75, 'use_augmentation': True, 'aug_probability': 0.19490676468169027, 'prob': 0.11179583083523345, 'noise_std': 0.19643040498675612, 'dropout_aug': 0.14581756033618765, 'alpha': 0.3362762327153398}. Best is trial 43 with value: 0.30010433923270374.

Fold 1 finished with rmse value = 0.304162
Fold 2 finished with rmse value = 0.304618
Fold 3 finished with rmse value = 0.302239

[I 2026-01-27 21:58:02,380] Trial 63 pruned.

Fold 1 finished with rmse value = 0.303159
Fold 2 finished with rmse value = 0.303425
Fold 3 finished with rmse value = 0.302042

[I 2026-01-27 22:01:33,577] Trial 64 pruned.

Fold 1 finished with rmse value = 0.302664
Fold 2 finished with rmse value = 0.303255
Fold 3 finished with rmse value = 0.301244
Fold 4 finished with rmse value = 0.302069
Fold 5 finished with rmse value = 0.301249

[I 2026-01-27 22:07:22,327] Trial 65 finished with value: 0.302096137691248 and parameters: {'nc': 4, 'nb': 3, 'nh': 8, 'ed_nh_ratio': 20, 'ff_hd': 512, 'regressor_hd': 768, 'dropout': 0.11269341036125713, 'lr': 0.002197610379474581, 'weight_decay': 0.0036970128911120123, 'batch_size': 256, 'patience': 20, 'epochs': 65, 'use_augmentation': True, 'aug_probability': 0.24044823764710466, 'prob': 0.13543673130461503, 'noise_std': 0.2833426258561196, 'dropout_aug': 0.17008614762201535, 'alpha': 0.36696393652454756}. Best is trial 43 with value: 0.30010433923270374.

Fold 1 finished with rmse value = 0.301781
Fold 2 finished with rmse value = 0.301934
Fold 3 finished with rmse value = 0.300377
Fold 4 finished with rmse value = 0.302035
Fold 5 finished with rmse value = 0.300446

[I 2026-01-27 22:15:38,865] Trial 66 finished with value: 0.30131462232738665 and parameters: {'nc': 2, 'nb': 2, 'nh': 6, 'ed_nh_ratio': 20, 'ff_hd': 128, 'regressor_hd': 640, 'dropout': 0.12461240874116263, 'lr': 0.002766608955553685, 'weight_decay': 0.0010975828917345973, 'batch_size': 192, 'patience': 20, 'epochs': 80, 'use_augmentation': True, 'aug_probability': 0.15879730754498872, 'prob': 0.0630040654430202, 'noise_std': 0.039207392859253676, 'dropout_aug': 0.17940052043779858, 'alpha': 0.3398605124066991}. Best is trial 43 with value: 0.30010433923270374.

Fold 1 finished with rmse value = 0.302452
Fold 2 finished with rmse value = 0.303187
Fold 3 finished with rmse value = 0.299668
Fold 4 finished with rmse value = 0.302903
Fold 5 finished with rmse value = 0.302346

[I 2026-01-27 22:35:43,430] Trial 67 finished with value: 0.30211123068072065 and parameters: {'nc': 10, 'nb': 3, 'nh': 8, 'ed_nh_ratio': 20, 'ff_hd': 512, 'regressor_hd': 768, 'dropout': 0.0993024773250441, 'lr': 0.0035144929380079787, 'weight_decay': 0.009792595582884582, 'batch_size': 64, 'patience': 20, 'epochs': 70, 'use_augmentation': True, 'aug_probability': 0.21270897265357602, 'prob': 0.11466539929972713, 'noise_std': 0.24577328692346906, 'dropout_aug': 0.14249363604128942, 'alpha': 0.355956447139964}. Best is trial 43 with value: 0.30010433923270374.

Fold 1 finished with rmse value = 0.302051
Fold 2 finished with rmse value = 0.302370
Fold 3 finished with rmse value = 0.299993
Fold 4 finished with rmse value = 0.301604
Fold 5 finished with rmse value = 0.301097

[I 2026-01-27 22:50:07,522] Trial 68 finished with value: 0.30142292719353936 and parameters: {'nc': 2, 'nb': 4, 'nh': 10, 'ed_nh_ratio': 16, 'ff_hd': 384, 'regressor_hd': 768, 'dropout': 0.11811664386485596, 'lr': 0.0022073405506831782, 'weight_decay': 0.0004165174090794889, 'batch_size': 128, 'patience': 15, 'epochs': 90, 'use_augmentation': True, 'aug_probability': 0.2931085338793493, 'prob': 0.09687708685056594, 'noise_std': 0.10648734399219903, 'dropout_aug': 0.09076123317253684, 'alpha': 0.38320164736478257}. Best is trial 43 with value: 0.30010433923270374.

Fold 1 finished with rmse value = 0.303435

[I 2026-01-27 22:51:25,017] Trial 69 pruned.

Fold 1 finished with rmse value = 0.303236
Fold 2 finished with rmse value = 0.303465
Fold 3 finished with rmse value = 0.301287

[I 2026-01-27 22:57:31,558] Trial 70 pruned.

Fold 1 finished with rmse value = 0.302111
Fold 2 finished with rmse value = 0.301766
Fold 3 finished with rmse value = 0.300279
Fold 4 finished with rmse value = 0.302196
Fold 5 finished with rmse value = 0.300169

[I 2026-01-27 23:13:51,118] Trial 71 finished with value: 0.3013043189145833 and parameters: {'nc': 2, 'nb': 2, 'nh': 6, 'ed_nh_ratio': 20, 'ff_hd': 128, 'regressor_hd': 640, 'dropout': 0.1266499639638743, 'lr': 0.002843991705377481, 'weight_decay': 0.005350099921626612, 'batch_size': 192, 'patience': 20, 'epochs': 80, 'use_augmentation': True, 'aug_probability': 0.18300766538010343, 'prob': 0.07149205411485267, 'noise_std': 0.03549229477543976, 'dropout_aug': 0.1809838740929569, 'alpha': 0.3986056411671911}. Best is trial 43 with value: 0.30010433923270374.

Fold 1 finished with rmse value = 0.302943

[I 2026-01-27 23:15:13,554] Trial 72 pruned.

Fold 1 finished with rmse value = 0.301240
Fold 2 finished with rmse value = 0.302448
Fold 3 finished with rmse value = 0.299778
Fold 4 finished with rmse value = 0.301629
Fold 5 finished with rmse value = 0.299779

[I 2026-01-27 23:22:31,306] Trial 73 finished with value: 0.30097450363851574 and parameters: {'nc': 4, 'nb': 3, 'nh': 8, 'ed_nh_ratio': 20, 'ff_hd': 128, 'regressor_hd': 640, 'dropout': 0.10960331195943292, 'lr': 0.0035867443556633063, 'weight_decay': 0.003163689826326449, 'batch_size': 320, 'patience': 15, 'epochs': 80, 'use_augmentation': True, 'aug_probability': 0.13919915402837502, 'prob': 0.10515649340190066, 'noise_std': 0.04933771963555282, 'dropout_aug': 0.21297897699692273, 'alpha': 0.3729750668648847}. Best is trial 43 with value: 0.30010433923270374.

Fold 1 finished with rmse value = 0.302404
Fold 2 finished with rmse value = 0.302448
Fold 3 finished with rmse value = 0.300523
Fold 4 finished with rmse value = 0.301696
Fold 5 finished with rmse value = 0.301535

[I 2026-01-27 23:29:55,302] Trial 74 finished with value: 0.3017213156070851 and parameters: {'nc': 4, 'nb': 4, 'nh': 10, 'ed_nh_ratio': 20, 'ff_hd': 256, 'regressor_hd': 768, 'dropout': 0.10725158291168253, 'lr': 0.005241566171232786, 'weight_decay': 0.0036310036529656874, 'batch_size': 320, 'patience': 15, 'epochs': 70, 'use_augmentation': True, 'aug_probability': 0.109282697315708, 'prob': 0.10643436962747631, 'noise_std': 0.2025269145414695, 'dropout_aug': 0.1552300305622873, 'alpha': 0.3766671662721056}. Best is trial 43 with value: 0.30010433923270374.

Fold 1 finished with rmse value = 0.303127

[I 2026-01-27 23:33:04,228] Trial 75 pruned.

Fold 1 finished with rmse value = 0.300961
Fold 2 finished with rmse value = 0.303058
Fold 3 finished with rmse value = 0.299473
Fold 4 finished with rmse value = 0.300604
Fold 5 finished with rmse value = 0.300080

[I 2026-01-27 23:42:16,591] Trial 76 finished with value: 0.30083530931894986 and parameters: {'nc': 4, 'nb': 3, 'nh': 8, 'ed_nh_ratio': 24, 'ff_hd': 128, 'regressor_hd': 896, 'dropout': 0.13644216488796748, 'lr': 0.0037449613330133255, 'weight_decay': 0.0018965910683843615, 'batch_size': 128, 'patience': 15, 'epochs': 55, 'use_augmentation': True, 'aug_probability': 0.12998687196641676, 'prob': 0.12403504036799273, 'noise_std': 0.1903311484461354, 'dropout_aug': 0.1249776108543412, 'alpha': 0.32609164885856234}. Best is trial 43 with value: 0.30010433923270374.

Fold 1 finished with rmse value = 0.304240
Fold 2 finished with rmse value = 0.303981
Fold 3 finished with rmse value = 0.301289

[I 2026-01-27 23:48:40,759] Trial 77 pruned.

Fold 1 finished with rmse value = 0.301284
Fold 2 finished with rmse value = 0.302047
Fold 3 finished with rmse value = 0.299890
Fold 4 finished with rmse value = 0.301505
Fold 5 finished with rmse value = 0.299817

[I 2026-01-27 23:54:35,364] Trial 78 finished with value: 0.3009085427706656 and parameters: {'nc': 6, 'nb': 3, 'nh': 8, 'ed_nh_ratio': 24, 'ff_hd': 512, 'regressor_hd': 896, 'dropout': 0.11202321330295112, 'lr': 0.0036048893696494135, 'weight_decay': 0.00024165465501944095, 'batch_size': 320, 'patience': 15, 'epochs': 65, 'use_augmentation': True, 'aug_probability': 0.21277740752107452, 'prob': 0.1185827530956741, 'noise_std': 0.1547936180484547, 'dropout_aug': 0.23331148269627222, 'alpha': 0.326392185488293}. Best is trial 43 with value: 0.30010433923270374.

Fold 1 finished with rmse value = 0.302148
Fold 2 finished with rmse value = 0.303014
Fold 3 finished with rmse value = 0.301570

[I 2026-01-27 23:57:23,510] Trial 79 pruned.

Fold 1 finished with rmse value = 0.301491
Fold 2 finished with rmse value = 0.304432
Fold 3 finished with rmse value = 0.301252

[I 2026-01-28 00:01:27,222] Trial 80 pruned.

Fold 1 finished with rmse value = 0.301592
Fold 2 finished with rmse value = 0.302402
Fold 3 finished with rmse value = 0.299687
Fold 4 finished with rmse value = 0.301141
Fold 5 finished with rmse value = 0.300517

[I 2026-01-28 00:08:25,710] Trial 81 finished with value: 0.3010678894260591 and parameters: {'nc': 4, 'nb': 3, 'nh': 8, 'ed_nh_ratio': 24, 'ff_hd': 512, 'regressor_hd': 896, 'dropout': 0.11971718863609059, 'lr': 0.003897780527365502, 'weight_decay': 0.0011681337893194503, 'batch_size': 256, 'patience': 15, 'epochs': 70, 'use_augmentation': True, 'aug_probability': 0.18356113360041876, 'prob': 0.13217242755862468, 'noise_std': 0.13091839753170587, 'dropout_aug': 0.1395852840938432, 'alpha': 0.3389960769711838}. Best is trial 43 with value: 0.30010433923270374.

Fold 1 finished with rmse value = 0.301387
Fold 2 finished with rmse value = 0.301723
Fold 3 finished with rmse value = 0.299684
Fold 4 finished with rmse value = 0.300881
Fold 5 finished with rmse value = 0.299838

[I 2026-01-28 00:15:57,596] Trial 82 finished with value: 0.30070269977720165 and parameters: {'nc': 4, 'nb': 3, 'nh': 8, 'ed_nh_ratio': 24, 'ff_hd': 512, 'regressor_hd': 896, 'dropout': 0.11870069241928681, 'lr': 0.003959043374939989, 'weight_decay': 0.0011123416443525357, 'batch_size': 256, 'patience': 15, 'epochs': 75, 'use_augmentation': True, 'aug_probability': 0.18889764327566824, 'prob': 0.10617019907117616, 'noise_std': 0.17365064405846445, 'dropout_aug': 0.22467649565339273, 'alpha': 0.3393218327080073}. Best is trial 43 with value: 0.30010433923270374.

Fold 1 finished with rmse value = 0.303509

[I 2026-01-28 00:17:07,327] Trial 83 pruned.

Fold 1 finished with rmse value = 0.301405
Fold 2 finished with rmse value = 0.301886
Fold 3 finished with rmse value = 0.299888
Fold 4 finished with rmse value = 0.300643
Fold 5 finished with rmse value = 0.300155

[I 2026-01-28 00:22:59,399] Trial 84 finished with value: 0.3007954203070439 and parameters: {'nc': 6, 'nb': 3, 'nh': 10, 'ed_nh_ratio': 24, 'ff_hd': 384, 'regressor_hd': 640, 'dropout': 0.09449711235918053, 'lr': 0.003337131719330051, 'weight_decay': 0.0027692975477210334, 'batch_size': 256, 'patience': 15, 'epochs': 55, 'use_augmentation': True, 'aug_probability': 0.15877741211294671, 'prob': 0.08210024873511737, 'noise_std': 0.17405424753765877, 'dropout_aug': 0.20567692870884996, 'alpha': 0.31287239545704676}. Best is trial 43 with value: 0.30010433923270374.

Fold 1 finished with rmse value = 0.302344
Fold 2 finished with rmse value = 0.302823
Fold 3 finished with rmse value = 0.302724

[I 2026-01-28 00:25:43,388] Trial 85 pruned.

Fold 1 finished with rmse value = 0.302540
Fold 2 finished with rmse value = 0.302269
Fold 3 finished with rmse value = 0.300271
Fold 4 finished with rmse value = 0.300851
Fold 5 finished with rmse value = 0.299541

[I 2026-01-28 00:34:48,117] Trial 86 finished with value: 0.30109445092267617 and parameters: {'nc': 6, 'nb': 4, 'nh': 10, 'ed_nh_ratio': 28, 'ff_hd': 384, 'regressor_hd': 512, 'dropout': 0.10982827727093913, 'lr': 0.003858330033762372, 'weight_decay': 0.002732599480844252, 'batch_size': 256, 'patience': 15, 'epochs': 80, 'use_augmentation': True, 'aug_probability': 0.19694153454764388, 'prob': 0.0821147122091319, 'noise_std': 0.14721182119720863, 'dropout_aug': 0.19629452399220923, 'alpha': 0.3200181087967745}. Best is trial 43 with value: 0.30010433923270374.

Fold 1 finished with rmse value = 0.303208

[I 2026-01-28 00:35:33,247] Trial 87 pruned.

Fold 1 finished with rmse value = 0.303441

[I 2026-01-28 00:37:28,804] Trial 88 pruned.

Fold 1 finished with rmse value = 0.312893

[I 2026-01-28 00:38:36,267] Trial 89 pruned.

Fold 1 finished with rmse value = 0.304342

[I 2026-01-28 00:40:01,342] Trial 90 pruned.

Fold 1 finished with rmse value = 0.300927
Fold 2 finished with rmse value = 0.300802
Fold 3 finished with rmse value = 0.299191
Fold 4 finished with rmse value = 0.300547
Fold 5 finished with rmse value = 0.299379

[I 2026-01-28 00:50:24,158] Trial 91 finished with value: 0.3001693842093885 and parameters: {'nc': 4, 'nb': 3, 'nh': 8, 'ed_nh_ratio': 24, 'ff_hd': 128, 'regressor_hd': 768, 'dropout': 0.12282332036930621, 'lr': 0.00218632226194418, 'weight_decay': 0.004415132881943295, 'batch_size': 128, 'patience': 15, 'epochs': 70, 'use_augmentation': True, 'aug_probability': 0.22878199147468545, 'prob': 0.12759518952368573, 'noise_std': 0.1840036102874873, 'dropout_aug': 0.1853879031963125, 'alpha': 0.3680145655113996}. Best is trial 43 with value: 0.30010433923270374.

Fold 1 finished with rmse value = 0.302060
Fold 2 finished with rmse value = 0.302616
Fold 3 finished with rmse value = 0.300605

[I 2026-01-28 00:54:05,631] Trial 92 pruned.

Fold 1 finished with rmse value = 0.301784
Fold 2 finished with rmse value = 0.301716
Fold 3 finished with rmse value = 0.299939
Fold 4 finished with rmse value = 0.299873
Fold 5 finished with rmse value = 0.300434

[I 2026-01-28 01:06:18,095] Trial 93 finished with value: 0.3007491506086501 and parameters: {'nc': 4, 'nb': 3, 'nh': 8, 'ed_nh_ratio': 24, 'ff_hd': 128, 'regressor_hd': 640, 'dropout': 0.12172913682129483, 'lr': 0.004246160143887892, 'weight_decay': 0.0002849309210617558, 'batch_size': 128, 'patience': 15, 'epochs': 75, 'use_augmentation': True, 'aug_probability': 0.15101257366445425, 'prob': 0.1269389361424827, 'noise_std': 0.1823890597930514, 'dropout_aug': 0.18877669731937607, 'alpha': 0.3370816159435451}. Best is trial 43 with value: 0.30010433923270374.

Fold 1 finished with rmse value = 0.300957
Fold 2 finished with rmse value = 0.301728
Fold 3 finished with rmse value = 0.299235
Fold 4 finished with rmse value = 0.300214
Fold 5 finished with rmse value = 0.300665

[I 2026-01-28 01:17:41,680] Trial 94 finished with value: 0.30055977314300264 and parameters: {'nc': 4, 'nb': 3, 'nh': 4, 'ed_nh_ratio': 24, 'ff_hd': 128, 'regressor_hd': 768, 'dropout': 0.12234848908333368, 'lr': 0.004261113698317009, 'weight_decay': 0.0003089193255938091, 'batch_size': 128, 'patience': 15, 'epochs': 75, 'use_augmentation': True, 'aug_probability': 0.1930738577162968, 'prob': 0.13764164958002242, 'noise_std': 0.18598659606680346, 'dropout_aug': 0.17438982466135242, 'alpha': 0.3218024508644942}. Best is trial 43 with value: 0.30010433923270374.

Fold 1 finished with rmse value = 0.302282
Fold 2 finished with rmse value = 0.305202
Fold 3 finished with rmse value = 0.300647
Fold 4 finished with rmse value = 0.300613
Fold 5 finished with rmse value = 0.301151

[I 2026-01-28 01:29:13,525] Trial 95 finished with value: 0.3019790194398707 and parameters: {'nc': 4, 'nb': 3, 'nh': 4, 'ed_nh_ratio': 28, 'ff_hd': 128, 'regressor_hd': 768, 'dropout': 0.13905629488692256, 'lr': 0.004284859887901999, 'weight_decay': 0.00014877125378956558, 'batch_size': 128, 'patience': 10, 'epochs': 75, 'use_augmentation': True, 'aug_probability': 0.18365398420624018, 'prob': 0.14026652132503076, 'noise_std': 0.18816503571894158, 'dropout_aug': 0.1898459037910403, 'alpha': 0.3430887794256124}. Best is trial 43 with value: 0.30010433923270374.

Fold 1 finished with rmse value = 0.302462
Fold 2 finished with rmse value = 0.302639
Fold 3 finished with rmse value = 0.300862

[I 2026-01-28 01:36:03,782] Trial 96 pruned.

Fold 1 finished with rmse value = 0.302291
Fold 2 finished with rmse value = 0.303467
Fold 3 finished with rmse value = 0.300386
Fold 4 finished with rmse value = 0.301295
Fold 5 finished with rmse value = 0.301016

[I 2026-01-28 01:43:46,286] Trial 97 finished with value: 0.30169083527239493 and parameters: {'nc': 4, 'nb': 4, 'nh': 2, 'ed_nh_ratio': 24, 'ff_hd': 128, 'regressor_hd': 768, 'dropout': 0.10437159999923565, 'lr': 0.002303933075642264, 'weight_decay': 0.0003028927720404818, 'batch_size': 192, 'patience': 15, 'epochs': 55, 'use_augmentation': True, 'aug_probability': 0.15290332580433064, 'prob': 0.09559127104701613, 'noise_std': 0.20840476805279715, 'dropout_aug': 0.1612323867832046, 'alpha': 0.28921999471764226}. Best is trial 43 with value: 0.30010433923270374.

Fold 1 finished with rmse value = 0.300654
Fold 2 finished with rmse value = 0.300790
Fold 3 finished with rmse value = 0.298919
Fold 4 finished with rmse value = 0.300012
Fold 5 finished with rmse value = 0.299666

[I 2026-01-28 01:52:07,317] Trial 98 finished with value: 0.30000812434354335 and parameters: {'nc': 4, 'nb': 2, 'nh': 6, 'ed_nh_ratio': 28, 'ff_hd': 128, 'regressor_hd': 768, 'dropout': 0.12171272445525481, 'lr': 0.003206982889606869, 'weight_decay': 0.000647236963309266, 'batch_size': 128, 'patience': 15, 'epochs': 60, 'use_augmentation': True, 'aug_probability': 0.39518904064529925, 'prob': 0.12616557080513133, 'noise_std': 0.19620457608189246, 'dropout_aug': 0.17289566010268698, 'alpha': 0.33593049660931495}. Best is trial 98 with value: 0.30000812434354335.

Fold 1 finished with rmse value = 0.301045
Fold 2 finished with rmse value = 0.301893
Fold 3 finished with rmse value = 0.300071
Fold 4 finished with rmse value = 0.300613
Fold 5 finished with rmse value = 0.300535

[I 2026-01-28 01:57:39,529] Trial 99 finished with value: 0.3008314583664033 and parameters: {'nc': 6, 'nb': 2, 'nh': 4, 'ed_nh_ratio': 28, 'ff_hd': 256, 'regressor_hd': 768, 'dropout': 0.12730315220256322, 'lr': 0.0031261697148412516, 'weight_decay': 0.0006418175362183453, 'batch_size': 256, 'patience': 15, 'epochs': 70, 'use_augmentation': True, 'aug_probability': 0.31033422073271766, 'prob': 0.14548655010365233, 'noise_std': 0.22706947383501055, 'dropout_aug': 0.1861715736726852, 'alpha': 0.3534253005788072}. Best is trial 98 with value: 0.30000812434354335.

Best value (rmse) = 0.30000812434354335
Best params saved to: model_scEve_tuning/best_params.txt

[17]:

{'nc': 4,
 'nb': 2,
 'nh': 6,
 'ed_nh_ratio': 28,
 'ff_hd': 128,
 'regressor_hd': 768,
 'dropout': 0.12171272445525481,
 'lr': 0.003206982889606869,
 'weight_decay': 0.000647236963309266,
 'batch_size': 128,
 'patience': 15,
 'epochs': 60,
 'use_augmentation': True,
 'aug_probability': 0.39518904064529925,
 'prob': 0.12616557080513133,
 'noise_std': 0.19620457608189246,
 'dropout_aug': 0.17289566010268698,
 'alpha': 0.33593049660931495}

[17]:

# Train sceve model using mdata_train dataset
scparadise.sceve.train_tuned(
    mdata_train,
    path = '',
    path_tuned='model_scEve_tuning',
    model_name = 'model_scEve_hp_tuned',
    first_modality_name = 'rna',
    second_modality_name = 'adt',
    detailed_annotation = 'celltype_l3',
    eval_metric = ['rmse']
)

Successfully loaded tuned hyperparameters!
Device: cuda
Task: Predict 'adt' from 'rna'
Number of first modality features: 635
Number of second modality features: 224

Dataset split:
Train dataset contains: 37359 cells, it is 80.0 % of input dataset
Validation dataset contains: 9340 cells, it is 20.0 % of input dataset

Training scEve model: 100%|███████████████████████████████████████████████████████████████████████████████| 60/60 [02:02<00:00,  2.05s/it]

Training completed!
Model saved to model_scEve_hp_tuned

Check tuned model quality#

[18]:

# Predict surface proteins using tuned scEve model
mdata_pred = scparadise.sceve.predict(
    adata_test_rna,
    path_model = 'model_scEve_hp_tuned',
    return_mdata = True
)

Feature prediction using the 2nd version of the scEve model
scEve model loaded from model_scEve_hp_tuned
Feature alignment:
Model features: 635
New data features: 635
Matched features: 635 (100.0%)

Predicting: 100%|████████████████████████████████████████████████████████████████████████████████████████| 25/25 [00:00<00:00, 460.72it/s]

[19]:

# Add umap coordinates
mdata_pred.obsm['X_umap'] = adata_test_rna.obsm['X_umap']

[20]:

# Proteins visualization
proteins_dict = {
    "Observed": proteins_observed,
    "Predicted": proteins_imputed
}
# Choose a color map for protein abundance visualization
cmap = 'bwr'
# Make Axes
# Number of needed rows and columns (based on the row with the most columns)
nrow = len(proteins_dict)
ncol = max([len(vs) for vs in proteins_dict.values()])
fig, axs = plt.subplots(nrow, ncol, figsize=(3 * ncol, 3 * nrow))
# Plot expression for every marker on the corresponding Axes object
for row_idx, (prots, markers) in enumerate(proteins_dict.items()):
    col_idx = 0
    for marker in markers:
        ax = axs[row_idx, col_idx]
        if prots == 'Observed':
            mu.pl.umap(mdata_test, color=marker, cmap = cmap, ax=ax, vmax = 'p99', show=False, frameon=False)
        else:
            mu.pl.umap(mdata_pred, color=marker, cmap = cmap, ax=ax, vmax = 'p99', show=False, frameon=False)
        # Add cell type as row label - here we simply add it as ylabel of
        # the first Axes object in the row
        if col_idx == 0:
            # We disabled axis drawing in UMAP to have plots without background and border
            # so we need to re-enable axis to plot the ylabel
            ax.axis("on")
            ax.tick_params(
                top="off",
                bottom="off",
                left="off",
                right="off",
                labelleft="on",
                labelbottom="off",
            )
            ax.set_ylabel(prots + "\n", rotation=90, fontsize=14)
            ax.set(frame_on=False)
        col_idx += 1
    # Remove unused column Axes in the current row
    while col_idx < ncol:
        axs[row_idx, col_idx].remove()
        col_idx += 1
# Alignment within the Figure
fig.tight_layout()
../../../_images/tutorials_notebooks_scEve_scEve_train_29_0.png

The tuned model outperforms the default model in predicting protein markers of small cell populations, such as CD133 of HSPCs (Hematopoietic Stem/Progenitor Cells ).

[21]:

# Check prediction results
df_hp_tuned = scparadise.scnoah.report_reg(
    adata_prot = mdata_test.mod['adt'],
    adata_pred_prot = mdata_pred.mod['adt']
)
df_hp_tuned

[21]:
EVS r2_score RMSE MedianAE MeanAE
score 0.304 0.267 0.321 0.197 0.244
EVS/r2_score higher value - better prediction
RMSE/MedianAE/MeanAE lower value - better prediction 
[22]:

# Compare prediction results
# 'untuned' row represents untuned model
# 'hp tuned' row represents tuned model
df.compare(df_hp_tuned, keep_equal=True, align_axis = 0, result_names=('untuned', 'hp tuned'))
# EVS/r2_score:     higher value - better prediction
# RMSE/MedianAE/MeanAE:     lower value - better prediction

[22]:
EVS r2_score RMSE MedianAE MeanAE
score untuned 0.283 0.235 0.328 0.201 0.248
hp tuned 0.304 0.267 0.321 0.197 0.244

Hyperparameter tuning of the scEve model led to increased performance across all metrics.

The error metrics (RMSE, MedianAE, MeanAE) decreased, indicating fewer errors in modality prediction.

[23]:

# Add regression status using 5 metrics
for metric in ['RMSE', 'MeanAE', 'MedianAE', 'EVS', 'r2_score']:
    scparadise.scnoah.regres_status(adata_prot = mdata_test.mod['adt'],
                                    adata_pred_prot = mdata_pred.mod['adt'],
                                    metric=metric)
    mdata_pred.obs[f'regres_status_{metric}'] = mdata_pred.mod['adt'].obs[f'regres_status_{metric}']

[24]:

# Visualize observed proteins
mu.pl.umap(
    mdata_pred,
    color=[
        'regres_status_RMSE',
        'regres_status_MeanAE',
        'regres_status_MedianAE',
        'regres_status_EVS',
        'regres_status_r2_score'
    ],
    frameon = False,
    cmap = 'bwr',
    ncols = 5,
    wspace = 0.15,
    hspace = 0.1
)
../../../_images/tutorials_notebooks_scEve_scEve_train_35_0.png

Based on a per cell type regression metrics score, it is possible to draw conclusions about the uniformity of protein prediction across different cell types in the dataset.

[25]:

# Find correlations between imputed and observed proteins
df_corr_hp_tuned = pd.DataFrame(columns=['Pearson coef', 'p-value'])
for i in mdata_test.mod['adt'].var_names.tolist():
    person_coef = scparadise.scnoah.pearson_coef(
        adata = mdata_test.mod['adt'],
        adata_pred = mdata_pred.mod['adt'],
        feature = i,
        feature_pred = i + '_pred',
        print_res = False
    )
    df_corr_hp_tuned.loc[i] = [person_coef['Pearson coefficient'], person_coef['p-value']]
df_corr_hp_tuned['p-value'] = df_corr['p-value'].astype('float64')
df_corr_hp_tuned.sort_values('Pearson coef', ascending=False).head(5)

[25]:
Pearson coef p-value
adt_CLEC12A 0.965 0.0
adt_CD64 0.944 0.0
adt_CD11b-2 0.929 0.0
adt_CD8a 0.919 0.0
adt_CD4-1 0.917 0.0 
[26]:

# Find highly correlated protein 'Pearson coef' > 0.5
df_high_corr_hp_tuned = df_corr_hp_tuned[df_corr_hp_tuned['Pearson coef'] >= 0.5].copy()
print('Number of proteins with Pearson coefficient >= 0.5:', len(df_high_corr_hp_tuned))
df_high_corr_hp_tuned.sort_values(by = 'Pearson coef', ascending=False).head(5)

Number of proteins with Pearson coefficient >= 0.5: 112

[26]:
Pearson coef p-value
adt_CLEC12A 0.965 0.0
adt_CD64 0.944 0.0
adt_CD11b-2 0.929 0.0
adt_CD8a 0.919 0.0
adt_CD4-1 0.917 0.0 
[27]:

# Comparison of Pearson coefficients from the tuned and default models (tuned minus default).
df_compare = df_corr_hp_tuned.subtract(df_corr)
df_compare.rename(columns={'Pearson coef': 'Pearson coef comparison'}, inplace=True)
df_compare.sort_values(by = 'Pearson coef comparison', ascending=False).head(5)

[27]:
Pearson coef comparison p-value
adt_CD103 0.190 0.0
adt_CD133-2 0.177 0.0
adt_CD34 0.168 0.0
adt_CD195 0.077 0.0
adt_CD133-1 0.071 0.0

The Pearson correlation coefficient for adt_CD103 increased by 0.190 as a result of model hyperparameters tuning.

Additionally, there is an increase in the number of proteins with a Pearson coefficient of 0.5 or greater from 107 to 112 as a result of model hyperparameter tuning.

[28]:

# Save mdata with imputed proteins
mdata_pred.write_h5mu('mdata_pred_tuned.h5mu')

Recommendation#

We strongly recommend hyperparameters tuning for scEve model training if possible. We also recommend testing the prediction results on a separate dataset (distinct from the training dataset).

[31]:

import session_info
session_info.show()

[31]:
Click to view session information 
-----
anndata             0.11.4
matplotlib          3.10.8
mudata              0.3.3
muon                0.1.7
numpy               2.2.6
pandas              2.3.3
scanpy              1.11.5
scparadise          1.1.0
session_info        v1.0.1
-----
Click to view modules imported as dependencies 
PIL                         12.1.1
anyio                       NA
arrow                       1.4.0
asttokens                   NA
attr                        25.4.0
attrs                       25.4.0
babel                       2.18.0
certifi                     2026.02.25
cffi                        2.0.0
charset_normalizer          3.4.4
cloudpickle                 3.1.2
colorlog                    NA
comm                        0.2.3
cuda                        12.9.4
cycler                      0.12.1
cython_runtime              NA
dateutil                    2.9.0.post0
debugpy                     1.8.20
decorator                   5.2.1
defusedxml                  0.7.1
exceptiongroup              1.3.1
executing                   2.2.1
fastjsonschema              NA
fqdn                        NA
fsspec                      2026.2.0
gseapy                      1.1.11
h5py                        3.15.1
idna                        3.11
imblearn                    0.14.1
ipykernel                   7.2.0
isoduration                 NA
jedi                        0.19.2
jinja2                      3.1.6
joblib                      1.5.3
json5                       0.13.0
jsonpointer                 3.0.0
jsonschema                  4.26.0
jsonschema_specifications   NA
jupyter_events              0.12.0
jupyter_server              2.17.0
jupyterlab_server           2.28.0
kiwisolver                  1.4.9
lark                        1.3.1
lazy_loader                 0.4
legacy_api_wrap             NA
llvmlite                    0.46.0
markupsafe                  3.0.3
matplotlib_inline           0.2.1
matplotlib_venn             1.1.2
mpl_toolkits                NA
mpmath                      1.3.0
msgpack                     1.1.2
natsort                     8.4.0
nbformat                    5.10.4
networkx                    3.4.2
numba                       0.64.0
opt_einsum                  3.4.0
optuna                      4.7.0
overrides                   NA
packaging                   25.0
parso                       0.8.6
patsy                       1.0.2
pexpect                     4.9.0
platformdirs                4.9.2
plottable                   0.1.5
prometheus_client           NA
prompt_toolkit              3.0.52
psutil                      7.2.2
ptyprocess                  0.7.0
pure_eval                   0.2.3
pyarrow                     23.0.1
pycparser                   3.00
pydev_ipython               NA
pydevconsole                NA
pydevd                      3.2.3
pydevd_file_utils           NA
pydevd_plugins              NA
pydevd_tracing              NA
pygments                    2.19.2
pynndescent                 0.6.0
pyparsing                   3.3.2
pythonjsonlogger            NA
pytorch_tabnet              NA
pytz                        2025.2
referencing                 NA
requests                    2.32.5
rfc3339_validator           0.1.4
rfc3986_validator           0.1.1
rfc3987_syntax              NA
rpds                        NA
scipy                       1.15.3
seaborn                     0.13.2
send2trash                  NA
shap                        0.49.1
simplejson                  3.20.2
six                         1.17.0
skimage                     0.25.2
sklearn                     1.7.2
sklearn_compat              0.1.5
slicer                      NA
sparse                      0.17.0
stack_data                  0.6.3
statsmodels                 0.14.6
sympy                       1.14.0
threadpoolctl               3.6.0
torch                       2.10.0+cu128
torchgen                    NA
tornado                     6.5.4
tqdm                        4.67.3
traitlets                   5.14.3
triton                      3.6.0
typing_extensions           NA
umap                        0.5.11
uri_template                NA
urllib3                     2.6.3
wcwidth                     0.6.0
webcolors                   NA
websocket                   1.9.0
yaml                        6.0.3
zmq                         27.1.0
zoneinfo                    NA

-----
IPython             8.38.0
jupyter_client      8.8.0
jupyter_core        5.9.1
jupyterlab          4.5.5
-----
Python 3.10.19 (main, Oct 21 2025, 16:43:05) [GCC 11.2.0]
Linux-6.6.87.2-microsoft-standard-WSL2-x86_64-with-glibc2.39
-----
Session information updated at 2026-03-23 17:47
Contents