Skip to content

Commit 1c24692

Browse files
feat: add import profiler tool with dynamic loaded lines code volume … (#17467)
### Overview This PR introduces `profiler.py`, a zero-dependency, process-isolated import profiling tool designed to benchmark Python SDK modules with extreme precision and zero dynamic cache interference. To evaluate authentic import initialization overhead without the "observer effect", the profiler leverages a Master/Worker architecture. The master orchestrator executes each benchmark iteration inside a completely isolated, dynamically spawned subprocess. By bypassing Python's `sys.modules` cache and sweeping disk-level `__pycache__` directories, the tool guarantees a 100% authentic "cold-start" evaluation every time. ### Key Telemetry Vectors & Features - **Initialization Latency (ms):** Measured at high resolution natively, securely omitting VM startup skew. - **Physical Memory Deltas (RAM):** Employs a dual-strategy approach tracking both internal block allocator shifts via `tracemalloc` (cross-platform), and pure OS-level physical memory footprints via `/proc/self/statm` deltas (Linux RSS) to catch heavy C/C++ backend extensions like `protobuf`. - **Deterministic Code Volume (LOC):** Tracks the physical delta of `sys.modules` across the import window, traversing disk paths to aggregate the exact number of physical Lines of Code (LOC) injected into the interpreter. - **CPU Pinning:** Binds worker processes to a single CPU core via `taskset` on Linux to eliminate OS-level context-switching jitter across iterations. - **Diagnostic Tooling:** Supports one-off diagnostic tracing for bottleneck discovery via `--trace` (`-X importtime`) and `--cprofile`. *A comprehensive breakdown of the architecture, data pipeline, and trace log analysis is documented in [`scripts/import_profiler/documentation.md`](./scripts/import_profiler/documentation.md).* ### Related Links - **Design Doc:** ` [go/sdk-performance-design](http://goto.google.com/sdk-performance-design)` - **Related Issue:** googleapis/python-aiplatform#4749 --------- Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
1 parent 8a1270c commit 1c24692

2 files changed

Lines changed: 408 additions & 0 deletions

File tree

Lines changed: 72 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,72 @@
1+
# Python SDK Import Profiler: Documentation & Breakdown
2+
3+
This document provides a comprehensive guide to the `import_profiler` scripts, directory files, and how to analyze the generated import trace logs to target optimization areas.
4+
5+
---
6+
7+
## 1. File Guide & Directory Structure
8+
The profiling tool is located in the [scripts/import_profiler/](./) directory:
9+
10+
* **[profiler.py](./profiler.py)**: The core executable script. It is designed as a single-file, self-spawning harness that performs process-isolated importing benchmarks and generates trace logs.
11+
12+
---
13+
14+
## 2. Profiler Mechanism (`profiler.py`)
15+
16+
**Objective**
17+
The Profiler functions as a process-isolated verification harness designed to capture before-and-after metrics across three distinct vectors: Initialization Latency (ms), Peak Memory Usage (MB), and Dynamic Code Volume (Loaded Modules & Lines of Code).
18+
19+
**Usage**
20+
21+
Run this command to collect the metrics:
22+
```bash
23+
python profiler.py --module <target_module> --iterations <N>
24+
```
25+
26+
**Expected Output**
27+
```text
28+
--- Results for <target_module> (<N> iterations) ---
29+
Code Volume (Deterministic):
30+
Loaded Modules: <count>
31+
Loaded Lines: <count>
32+
Time (ms):
33+
P50 (Median): <time>
34+
P90: <time>
35+
P99: <time>
36+
RAM (MB):
37+
P50 (Median): <memory>
38+
P90: <memory>
39+
P99: <memory>
40+
```
41+
42+
---
43+
44+
## 3. How to Interpret Python `-X importtime` Trace Logs
45+
When running with the `--trace` flag, the script captures the raw stderr trace produced by Python's `-X importtime` option. The trace looks like this:
46+
47+
```
48+
import time: self [us] | cumulative | imported package
49+
import time: 536 | 536 | _io
50+
import time: 1077 | 2385 | _frozen_importlib_external
51+
import time: 773659 | 793010 | google.cloud.compute_v1.types.compute
52+
```
53+
54+
### Explaining the Fields:
55+
1. **`self [us]` (Microseconds):** The time spent importing the module itself, *excluding* any time spent importing its child dependencies.
56+
2. **`cumulative` (Microseconds):** The total time spent loading the module *including* all nested imports. This represents the total wait time introduced by this line.
57+
3. **Hierarchy Indentation:** Indented packages are sub-imports triggered by the parent module. A package with higher indentation is loaded deeper in the call stack.
58+
59+
---
60+
61+
## 4. Execution Reference
62+
Ensure you are in the correct pyenv virtual environment where packages are installed in editable mode:
63+
64+
```bash
65+
# 1. Run the profiler to get baseline/optimized outcomes (e.g. 5 iterations)
66+
PYENV_VERSION=py312 python profiler.py --module=google.cloud.compute --iterations=5
67+
PYENV_VERSION=py312 python profiler.py --module=google.cloud.aiplatform --iterations=5
68+
69+
# 2. Run the profiler to generate trace logs
70+
PYENV_VERSION=py312 python profiler.py --module=google.cloud.compute --trace
71+
PYENV_VERSION=py312 python profiler.py --module=google.cloud.aiplatform --trace
72+
```

0 commit comments

Comments
 (0)