fix: add frontmatter, show_doc, tests, See Also to all API notebooks

nathanhubens · nathanhubens · commit 8514336ca0fc · 2026-04-10T15:57:06.000+02:00
Structural alignment for all 10 API notebooks:
- Add raw YAML frontmatter (title, description, skip_showdoc)
- Add #| include: false setup cell with from nbdev.showdoc import *
- Add show_doc() for all public classes and functions
- Add #| hide test cells with fastcore.test assertions
- Add See Also cross-reference sections
- Remove stale nbdev_export() call from core.ipynb

Notebooks updated: core/core, core/utils, metrics/size, metrics/speed,
metrics/memory, metrics/energy, metrics/compute, analysis/profiling,
analysis/benchmark, visualization/plot
diff --git a/fasterbench/benchmark.py b/fasterbench/benchmark.py
@@ -1,5 +1,3 @@
-"""Unified benchmarking API for comprehensive model analysis"""
-
 # AUTOGENERATED! DO NOT EDIT! File to edit: ../nbs/analysis/benchmark.ipynb.
 
 # %% ../nbs/analysis/benchmark.ipynb #bbaee268
diff --git a/fasterbench/compute.py b/fasterbench/compute.py
@@ -1,5 +1,3 @@
-"""Compute modules for benchmarking"""
-
 # AUTOGENERATED! DO NOT EDIT! File to edit: ../nbs/metrics/compute.ipynb.
 
 # %% ../nbs/metrics/compute.ipynb #0091d170
diff --git a/fasterbench/core.py b/fasterbench/core.py
@@ -1,5 +1,3 @@
-"""Core modules for benchmarking"""
-
 # AUTOGENERATED! DO NOT EDIT! File to edit: ../nbs/core/core.ipynb.
 
 # %% ../nbs/core/core.ipynb #c59316b7
diff --git a/fasterbench/energy.py b/fasterbench/energy.py
@@ -1,5 +1,3 @@
-"""Energy modules for benchmarking"""
-
 # AUTOGENERATED! DO NOT EDIT! File to edit: ../nbs/metrics/energy.ipynb.
 
 # %% ../nbs/metrics/energy.ipynb #d27f26a4
diff --git a/fasterbench/memory.py b/fasterbench/memory.py
@@ -1,5 +1,3 @@
-"""Memory modules for benchmarking"""
-
 # AUTOGENERATED! DO NOT EDIT! File to edit: ../nbs/metrics/memory.ipynb.
 
 # %% ../nbs/metrics/memory.ipynb #16cd91b6
diff --git a/fasterbench/plot.py b/fasterbench/plot.py
@@ -1,5 +1,3 @@
-"""A module to plot the results of the benchmark"""
-
 # AUTOGENERATED! DO NOT EDIT! File to edit: ../nbs/visualization/plot.ipynb.
 
 # %% ../nbs/visualization/plot.ipynb #0c86ca8f-38d2-44d0-8204-3c977f1b1c19
diff --git a/fasterbench/profiling.py b/fasterbench/profiling.py
@@ -1,5 +1,3 @@
-"""Per-layer profiling for deep analysis of model performance"""
-
 # AUTOGENERATED! DO NOT EDIT! File to edit: ../nbs/analysis/profiling.ipynb.
 
 # %% ../nbs/analysis/profiling.ipynb #imports
diff --git a/fasterbench/size.py b/fasterbench/size.py
@@ -1,5 +1,3 @@
-"""Model size and parameter count measurement"""
-
 # AUTOGENERATED! DO NOT EDIT! File to edit: ../nbs/metrics/size.ipynb.
 
 # %% ../nbs/metrics/size.ipynb #4c37777e
diff --git a/fasterbench/speed.py b/fasterbench/speed.py
@@ -1,5 +1,3 @@
-"""Latency and throughput measurement for PyTorch models"""
-
 # AUTOGENERATED! DO NOT EDIT! File to edit: ../nbs/metrics/speed.ipynb.
 
 # %% ../nbs/metrics/speed.ipynb #e6b40d9e
diff --git a/fasterbench/utils.py b/fasterbench/utils.py
@@ -1,5 +1,3 @@
-"""Utility functions"""
-
 # AUTOGENERATED! DO NOT EDIT! File to edit: ../nbs/core/utils.ipynb.
 
 # %% ../nbs/core/utils.ipynb #436e19d0-bb2b-47fd-8499-f38d8ac96e56
diff --git a/nbs/analysis/benchmark.ipynb b/nbs/analysis/benchmark.ipynb
@@ -1,10 +1,12 @@
 {
  "cells": [
   {
-   "cell_type": "markdown",
-   "id": "f86c20a0",
+   "cell_type": "raw",
    "metadata": {},
-   "source": "# Benchmark\n\n> Unified benchmarking API for comprehensive model analysis\n\n## Overview\n\nThe `benchmark()` function is the main entry point for fasterbench. It returns a `BenchmarkResult` with typed access to all metrics.\n\n### Quick Example\n\n```python\nfrom fasterbench import benchmark\n\nresult = benchmark(model, sample, metrics=[\"size\", \"speed\", \"compute\"])\n\n# Typed access\nprint(result.size.size_mib)\nprint(result.speed[\"cpu\"].mean_ms)\n\n# Dict access (backward compatible)\nprint(result[\"size_size_mib\"])\n```\n\n### Available Metrics\n\n| Metric | Module | What It Measures |\n|--------|--------|------------------|\n| `\"size\"` | [size](../metrics/size.html) | Disk size, parameter count |\n| `\"speed\"` | [speed](../metrics/speed.html) | Latency, throughput |\n| `\"compute\"` | [compute](../metrics/compute.html) | MACs, operation count |\n| `\"memory\"` | [memory](../metrics/memory.html) | Peak/average memory |\n| `\"energy\"` | [energy](../metrics/energy.html) | Power, CO₂ emissions |"
+   "source": [
+    "---\ntitle: \"Benchmark\"\ndescription: \"Unified benchmarking API for PyTorch models\"\nskip_showdoc: true\n---"
+   ],
+   "id": "frontmatter"
   },
   {
    "cell_type": "code",
@@ -121,6 +123,35 @@
     "show_doc(benchmark)"
    ]
   },
+  {
+   "cell_type": "code",
+   "metadata": {},
+   "source": [
+    "show_doc(BenchmarkResult)"
+   ],
+   "id": "show_doc_0",
+   "outputs": [],
+   "execution_count": null
+  },
+  {
+   "cell_type": "code",
+   "metadata": {},
+   "source": [
+    "#| hide\n",
+    "from fastcore.test import *\n",
+    "\n",
+    "import torch, torch.nn as nn\n",
+    "_m = nn.Linear(10, 5)\n",
+    "_x = torch.randn(1, 10)\n",
+    "_r = benchmark(_m, _x, metrics=[\"size\"])\n",
+    "assert isinstance(_r, BenchmarkResult)\n",
+    "assert \"size_num_params\" in _r\n",
+    "assert _r[\"size_num_params\"] > 0"
+   ],
+   "id": "test_basic",
+   "outputs": [],
+   "execution_count": null
+  },
   {
    "cell_type": "markdown",
    "execution_count": null,
diff --git a/nbs/analysis/profiling.ipynb b/nbs/analysis/profiling.ipynb
@@ -1,14 +1,12 @@
 {
  "cells": [
   {
-   "cell_type": "markdown",
-   "id": "header",
+   "cell_type": "raw",
    "metadata": {},
    "source": [
-    "# Profiling\n",
-    "\n",
-    "> Per-layer profiling for deep analysis of model performance"
-   ]
+    "---\ntitle: \"Profiling\"\ndescription: \"Per-layer profiling for deep analysis of model performance\"\nskip_showdoc: true\n---"
+   ],
+   "id": "frontmatter"
   },
   {
    "cell_type": "code",
@@ -20,6 +18,16 @@
     "#| default_exp profiling"
    ]
   },
+  {
+   "cell_type": "code",
+   "metadata": {},
+   "source": [
+    "#| include: false\nfrom nbdev.showdoc import *"
+   ],
+   "id": "setup_showdoc",
+   "outputs": [],
+   "execution_count": null
+  },
   {
    "cell_type": "code",
    "execution_count": null,
@@ -427,6 +435,64 @@
     "                print(f\"  {r['name']:40} {r['type']:15} {val_str} ({pct:5.1f}%)\")\n",
     "            print()"
    ]
+  },
+  {
+   "cell_type": "code",
+   "metadata": {},
+   "source": [
+    "show_doc(LayerProfiler)"
+   ],
+   "id": "show_doc_0",
+   "outputs": [],
+   "execution_count": null
+  },
+  {
+   "cell_type": "code",
+   "metadata": {},
+   "source": [
+    "show_doc(LayerProfiler.profile)"
+   ],
+   "id": "show_doc_1",
+   "outputs": [],
+   "execution_count": null
+  },
+  {
+   "cell_type": "code",
+   "metadata": {},
+   "source": [
+    "show_doc(LayerProfiler.top)"
+   ],
+   "id": "show_doc_2",
+   "outputs": [],
+   "execution_count": null
+  },
+  {
+   "cell_type": "code",
+   "metadata": {},
+   "source": [
+    "show_doc(LayerProfiler.summary)"
+   ],
+   "id": "show_doc_3",
+   "outputs": [],
+   "execution_count": null
+  },
+  {
+   "cell_type": "code",
+   "metadata": {},
+   "source": [
+    "#| hide\nfrom fastcore.test import *\n\nimport torch, torch.nn as nn\n_m = nn.Sequential(nn.Linear(10, 20), nn.ReLU(), nn.Linear(20, 5))\n_x = torch.randn(1, 10)\n_lp = LayerProfiler(_m, _x)\n_results = _lp.profile(metrics=[\"size\"])\nassert len(_results) > 0\nassert all(\"params\" in r for r in _results)"
+   ],
+   "id": "test_basic",
+   "outputs": [],
+   "execution_count": null
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "---\n\n## See Also\n\n- [Benchmark](benchmark.html) — Unified benchmarking API\n- [Speed](../metrics/speed.html) — Latency measurement\n- [Profiling Tutorial](../tutorials/profiling.html) — Step-by-step profiling guide"
+   ],
+   "id": "see_also"
   }
  ],
  "metadata": {
diff --git a/nbs/core/core.ipynb b/nbs/core/core.ipynb
@@ -1,14 +1,12 @@
 {
  "cells": [
   {
-   "cell_type": "markdown",
-   "id": "0bbc6d9c",
+   "cell_type": "raw",
    "metadata": {},
    "source": [
-    "# core\n",
-    "\n",
-    "> Core modules for benchmarking"
-   ]
+    "---\ntitle: \"Core\"\ndescription: \"Internal utilities for benchmarking infrastructure\"\nskip_showdoc: true\n---"
+   ],
+   "id": "frontmatter"
   },
   {
    "cell_type": "code",
@@ -20,6 +18,16 @@
     "#| default_exp core"
    ]
   },
+  {
+   "cell_type": "code",
+   "metadata": {},
+   "source": [
+    "#| include: false\nfrom nbdev.showdoc import *"
+   ],
+   "id": "setup_showdoc",
+   "outputs": [],
+   "execution_count": null
+  },
   {
    "cell_type": "code",
    "execution_count": null,
@@ -45,20 +53,38 @@
   {
    "cell_type": "code",
    "execution_count": null,
-   "id": "eb1c9ec1",
+   "id": "1d0944ae",
    "metadata": {},
    "outputs": [],
+   "source": []
+  },
+  {
+   "cell_type": "code",
+   "metadata": {},
    "source": [
-    "import nbdev; nbdev.nbdev_export()"
-   ]
+    "show_doc(_validate_benchmark_params)"
+   ],
+   "id": "show_doc_0",
+   "outputs": [],
+   "execution_count": null
   },
   {
    "cell_type": "code",
-   "execution_count": null,
-   "id": "1d0944ae",
    "metadata": {},
+   "source": [
+    "show_doc(_run_on_devices)"
+   ],
+   "id": "show_doc_1",
    "outputs": [],
-   "source": []
+   "execution_count": null
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "---\n\n## See Also\n\n- [Benchmark](../analysis/benchmark.html) — Main benchmarking API\n- [Speed](../metrics/speed.html) — Latency and throughput measurement"
+   ],
+   "id": "see_also"
   }
  ],
  "metadata": {
@@ -70,4 +96,4 @@
  },
  "nbformat": 4,
  "nbformat_minor": 5
-}
+}
diff --git a/nbs/core/utils.ipynb b/nbs/core/utils.ipynb
@@ -1,14 +1,12 @@
 {
  "cells": [
   {
-   "cell_type": "markdown",
-   "id": "eb7b0e2e-f1e7-4b51-b790-d854ae073776",
+   "cell_type": "raw",
    "metadata": {},
    "source": [
-    "# utils\n",
-    "\n",
-    "> Utility functions"
-   ]
+    "---\ntitle: \"Utils\"\ndescription: \"Utility functions for parsing metric values\"\nskip_showdoc: true\n---"
+   ],
+   "id": "frontmatter"
   },
   {
    "cell_type": "code",
@@ -20,6 +18,16 @@
     "#| default_exp utils"
    ]
   },
+  {
+   "cell_type": "code",
+   "metadata": {},
+   "source": [
+    "#| include: false\nfrom nbdev.showdoc import *"
+   ],
+   "id": "setup_showdoc",
+   "outputs": [],
+   "execution_count": null
+  },
   {
    "cell_type": "code",
    "execution_count": null,
@@ -51,6 +59,40 @@
     "    multipliers = {'K': 0.001, 'M': 1.0, 'G': 1000.0}\n",
     "    return num * multipliers.get(suffix, 1.0)"
    ]
+  },
+  {
+   "cell_type": "code",
+   "metadata": {},
+   "source": [
+    "show_doc(parse_metric_value)"
+   ],
+   "id": "show_doc_0",
+   "outputs": [],
+   "execution_count": null
+  },
+  {
+   "cell_type": "code",
+   "metadata": {},
+   "source": [
+    "#| hide\n",
+    "from fastcore.test import *\n",
+    "\n",
+    "# parse_metric_value extracts numeric values from formatted strings\n",
+    "test_close(parse_metric_value(\"1.5M\"), 1.5, eps=0.01)\n",
+    "test_close(parse_metric_value(\"500K\"), 0.5, eps=0.01)\n",
+    "test_close(parse_metric_value(\"2.3\"), 2.3, eps=0.01)"
+   ],
+   "id": "test_basic",
+   "outputs": [],
+   "execution_count": null
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "---\n\n## See Also\n\n- [Benchmark](../analysis/benchmark.html) — Main benchmarking API\n- [Report](../analysis/report.html) — Formatted benchmark reports"
+   ],
+   "id": "see_also"
   }
  ],
  "metadata": {
@@ -62,4 +104,4 @@
  },
  "nbformat": 4,
  "nbformat_minor": 5
-}
+}
diff --git a/nbs/metrics/compute.ipynb b/nbs/metrics/compute.ipynb
diff --git a/nbs/metrics/energy.ipynb b/nbs/metrics/energy.ipynb
diff --git a/nbs/metrics/memory.ipynb b/nbs/metrics/memory.ipynb
diff --git a/nbs/metrics/size.ipynb b/nbs/metrics/size.ipynb
diff --git a/nbs/metrics/speed.ipynb b/nbs/metrics/speed.ipynb
diff --git a/nbs/visualization/plot.ipynb b/nbs/visualization/plot.ipynb

Original file line number	Diff line number	Diff line change
`@@ -1,5 +1,3 @@`
`1`		`-"""Unified benchmarking API for comprehensive model analysis"""`
`2`		`-`
`3`	`1`	`# AUTOGENERATED! DO NOT EDIT! File to edit: ../nbs/analysis/benchmark.ipynb.`
`4`	`2`
`5`	`3`	`# %% ../nbs/analysis/benchmark.ipynb #bbaee268`
Original file line number	Diff line number	Diff line change
`@@ -1,5 +1,3 @@`
`1`		`-"""Compute modules for benchmarking"""`
`2`		`-`
`3`	`1`	`# AUTOGENERATED! DO NOT EDIT! File to edit: ../nbs/metrics/compute.ipynb.`
`4`	`2`
`5`	`3`	`# %% ../nbs/metrics/compute.ipynb #0091d170`
Original file line number	Diff line number	Diff line change
`@@ -1,5 +1,3 @@`
`1`		`-"""Core modules for benchmarking"""`
`2`		`-`
`3`	`1`	`# AUTOGENERATED! DO NOT EDIT! File to edit: ../nbs/core/core.ipynb.`
`4`	`2`
`5`	`3`	`# %% ../nbs/core/core.ipynb #c59316b7`
Original file line number	Diff line number	Diff line change
`@@ -1,5 +1,3 @@`
`1`		`-"""Energy modules for benchmarking"""`
`2`		`-`
`3`	`1`	`# AUTOGENERATED! DO NOT EDIT! File to edit: ../nbs/metrics/energy.ipynb.`
`4`	`2`
`5`	`3`	`# %% ../nbs/metrics/energy.ipynb #d27f26a4`
Original file line number	Diff line number	Diff line change
`@@ -1,5 +1,3 @@`
`1`		`-"""Memory modules for benchmarking"""`
`2`		`-`
`3`	`1`	`# AUTOGENERATED! DO NOT EDIT! File to edit: ../nbs/metrics/memory.ipynb.`
`4`	`2`
`5`	`3`	`# %% ../nbs/metrics/memory.ipynb #16cd91b6`
Original file line number	Diff line number	Diff line change
`@@ -1,5 +1,3 @@`
`1`		`-"""A module to plot the results of the benchmark"""`
`2`		`-`
`3`	`1`	`# AUTOGENERATED! DO NOT EDIT! File to edit: ../nbs/visualization/plot.ipynb.`
`4`	`2`
`5`	`3`	`# %% ../nbs/visualization/plot.ipynb #0c86ca8f-38d2-44d0-8204-3c977f1b1c19`
Original file line number	Diff line number	Diff line change
`@@ -1,5 +1,3 @@`
`1`		`-"""Per-layer profiling for deep analysis of model performance"""`
`2`		`-`
`3`	`1`	`# AUTOGENERATED! DO NOT EDIT! File to edit: ../nbs/analysis/profiling.ipynb.`
`4`	`2`
`5`	`3`	`# %% ../nbs/analysis/profiling.ipynb #imports`
Original file line number	Diff line number	Diff line change
`@@ -1,5 +1,3 @@`
`1`		`-"""Model size and parameter count measurement"""`
`2`		`-`
`3`	`1`	`# AUTOGENERATED! DO NOT EDIT! File to edit: ../nbs/metrics/size.ipynb.`
`4`	`2`
`5`	`3`	`# %% ../nbs/metrics/size.ipynb #4c37777e`
Original file line number	Diff line number	Diff line change
`@@ -1,5 +1,3 @@`
`1`		`-"""Latency and throughput measurement for PyTorch models"""`
`2`		`-`
`3`	`1`	`# AUTOGENERATED! DO NOT EDIT! File to edit: ../nbs/metrics/speed.ipynb.`
`4`	`2`
`5`	`3`	`# %% ../nbs/metrics/speed.ipynb #e6b40d9e`
Original file line number	Diff line number	Diff line change
`@@ -1,5 +1,3 @@`
`1`		`-"""Utility functions"""`
`2`		`-`
`3`	`1`	`# AUTOGENERATED! DO NOT EDIT! File to edit: ../nbs/core/utils.ipynb.`
`4`	`2`
`5`	`3`	`# %% ../nbs/core/utils.ipynb #436e19d0-bb2b-47fd-8499-f38d8ac96e56`