Skip to content

Commit 0446902

Browse files
committed
Added full description of ABaCo computational cost estimation.
1 parent 3de2825 commit 0446902

1 file changed

Lines changed: 23 additions & 14 deletions

File tree

docs/tutorial/report-performance.ipynb

Lines changed: 23 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -930,26 +930,17 @@
930930
"id": "89bd923e",
931931
"metadata": {},
932932
"source": [
933-
"## ABaCo computational cost estimation"
933+
"## ABaCo Computational Requirements estimation"
934934
]
935935
},
936936
{
937937
"cell_type": "markdown",
938938
"id": "c8a76f48",
939939
"metadata": {},
940940
"source": [
941-
"For this analysis, the file `abaco_performance_interpolation_iter.csv` at `data` contains the runs done with the following configurations on number of samples and features:\n",
942-
"\n",
943-
"| Samples \\\\ Features | 50 | 100 | 250 | 500 | 750 | 1000 |\n",
944-
"|---------------------|----|-----|-----|-----|-----|------|\n",
945-
"| 50 | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ |\n",
946-
"| 100 | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ |\n",
947-
"| 250 | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ |\n",
948-
"| 500 | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ |\n",
949-
"| 750 | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ |\n",
950-
"| 1000 | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ |\n",
951-
"\n",
952-
"So a total of 36 runs were made, in which the wall-clock time, as well as the Peak GPU memory usage were recorded during the training and inference process of ABaCo. For each run configuration, 5 replicates were also recorded to also report variability of these two parameters. The computational time for each run configuration, as well as the GPU memory requirements can be displayed on the following plot:"
941+
"In this section, we provide a benchmark analysis of ABaCo's computational performance. While adversarial generative models can be computationally demanding, it is essential to assess the feasibility of the method for typical research environments. We report the training runtime and peak GPU memory usage across varying dataset sizes and dimensions.\n",
942+
"\n",
943+
"We evaluated the model's scalability by performing a grid search over dataset dimensions. Let $N$ denote the number of samples and $P$ the number of features. We varied $N \\in \\{50, \\dots, 1000\\}$ and $P \\in \\{50, \\dots, 1000\\}$. For each combination of $(N, P)$, we performed 5 independent training runs to capture variability in initialization and during training. All experiments were conducted on a a dual-socket Intel Xeon Gold 6226R server running in x86\\_64 mode with VT-x virtualization enabled (see [ABaCo's paper](https://www.biorxiv.org/content/10.1101/2025.09.22.677692v1), Section 4.5 for more details)."
953944
]
954945
},
955946
{
@@ -1030,7 +1021,25 @@
10301021
"id": "f803bf4f",
10311022
"metadata": {},
10321023
"source": [
1033-
"We can see that ABaCo's running time mainly depends on a linear relation on the number of sample of the dataset, in where the number of features has more relevance on the peak GPU memory usage. Furthermore, it can be visible that in the most computational exhaustive configuration (1000 samples, 1000 features) ABaCo uses around 0.2 GiB of GPU memory, demonstrating that the model can be run even with consumer-grade hardware and doesn't require high-performing GPUs."
1024+
"The figure above represents the computational resources assessment of ABaCo: \n",
1025+
"\n",
1026+
"- *Left*: Wall-clock training time (seconds) versus number of samples, stratified by number of features (colors). Solid lines represent the mean over 5 independent runs, while shaded regions denote the 95\\% confidence interval, illustrating training stability. \n",
1027+
"\n",
1028+
"- *Right*: Peak GPU memory usage (GiB) versus number of samples. The model exhibits a highly efficient memory footprint, peaking at approximately $0.20$ GiB for the largest dataset tested ($N=1000, P=1000$). \n",
1029+
"\n",
1030+
"\n",
1031+
"**Runtime Scalability:**\n",
1032+
"\n",
1033+
"The wall-clock time required for model training is primarily driven by the number of samples ($N$), with the number of features ($P$) having a comparatively minor impact on runtime. As shown in Figure \\ref{fig:abaco_performance} (left panel), the trajectories for varying feature dimensions ($P \\in [50, 1000]$) overlap significantly, indicating that the computational cost of the encoder and discriminator networks is dominated by batch processing rather than the input layer width. Overall, the training time $T(N)$ scales linearly with sample size:\n",
1034+
"\\begin{equation*}\n",
1035+
" T(N) \\propto \\mathcal{O}(N)\n",
1036+
"\\end{equation*}\n",
1037+
"\n",
1038+
"For the largest configuration tested ($N=1000, P=1000$), the mean training time was approximately $140$ seconds ($\\approx 2.3$ minutes). Across runs we can see variability in the training time, which is expected given the stochastic nature of adversarial optimization. Despite this, the computational cost remains predictable and linear, ensuring feasibility for larger datasets.\n",
1039+
"\n",
1040+
"**Memory Requirements:**\n",
1041+
"\n",
1042+
"We monitored the peak GPU memory (VRAM) usage during the training phase. The memory footprint of ABaCo is notably low. For the maximum complexity tested ($N=1000, P=1000$), the peak VRAM usage was approximately 0.2 GiB. This low memory requirement ($< 1$ GiB) confirms that ABaCo does not require high-performance enterprise GPUs (e.g., A100) and is fully compatible with standard consumer-grade hardware. The linear scaling of memory usage suggests that the model can process significantly larger batches (e.g., $N \\gg 1000$) on typical GPUs with 8-12 GiB of VRAM without encountering memory exhaustion."
10341043
]
10351044
}
10361045
],

0 commit comments

Comments
 (0)