OracleTrace — Detect Python Performance Regressions with Execution Diff

Detect Python performance regressions before they merge.

Fail your CI when performance regresses. OracleTrace is a git diff for performance. Create a known-good baseline, trace the current run, and see exactly which functions got slower.

Documentation: https://kaykcaputo.github.io/oracletrace/

Featured in: PyCoder's Weekly #729 • La Experimental #30 • Python技术周刊 #15,#16 and #17 • Woudar's Blog #214 • awesome-debugger • awesome-profiling

---

Installation

pip install oracletrace

Quick Start

1. See where your program spends time instantly:

oracletrace app.py

2. Compare runs and detect regressions:

oracletrace app.py --json baseline.json
oracletrace app.py --json new.json --compare baseline.json

3. Manage baselines explicitly:

oracletrace baseline save app.py baseline.json
oracletrace app.py --json current.json
oracletrace baseline compare baseline.json current.json --fail-on-regression --threshold 10

---

Examples

Try these ready-to-run scripts to explore OracleTrace features:

# CPU hotspot — highlights the heaviest function
oracletrace examples/cpu_hotspot.py

# Nested call graph — see the tree visualization
oracletrace examples/nested_calls.py

# Regression demo — baseline vs slower path
oracletrace examples/regression_demo.py --json baseline.json
SLOW=1 oracletrace examples/regression_demo.py --json current.json --compare baseline.json

All examples are deterministic, finish in under a second, and live in examples/. They can also be reused in test suites as smoke tests.

---

See it in action

See exactly which functions got slower between runs:

---

Example Output

Starting application...

Iteration 1:
  > Processing data...
    > Calculating results...

Iteration 2:
  > Processing data...
    > Calculating results...

Application finished.

Summary:
                         Top functions by Total Time
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━┳━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━┓
┃ Function                     ┃ Total Time (s) ┃ Calls ┃ Avg. Time/Call (ms) ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━╇━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━┩
│ my_app.py:main               │         0.6025 │     1 │             602.510 │
│ my_app.py:process_data       │         0.6021 │     2 │             301.050 │
│ my_app.py:calculate_results  │         0.4015 │     2 │             200.750 │
└──────────────────────────────┴────────────────┴───────┴─────────────────────┘

Logic Flow:
<module>
└── my_app.py:main (1x, 0.6025s)
    └── my_app.py:process_data (2x, 0.6021s)
        └── my_app.py:calculate_results (2x, 0.4015s)

---

Why OracleTrace?

Problem

Performance regressions often reach production because correctness tests still pass.

Solution

OracleTrace gives CI a small, scriptable performance gate based on saved execution traces.

How it works

1. Run your script
2. Generate a trace
3. Compare results
4. Identify slowdowns

---

CI Integration

Fail your pipeline when performance degrades:

oracletrace run \
  --repeat 50 \
  --json current.json \
  --compare baseline.json \
  --fail-on-regression \
  --threshold 35 \
  -- pytest tests/

Add it to your CI to automatically fail on performance regressions.

Baseline-file workflow:

# on a stable branch or release job
oracletrace baseline save app.py baseline.json --repeat 50

# in a pull request job
oracletrace app.py --json current.json --repeat 50
oracletrace baseline compare baseline.json current.json --fail-on-regression --threshold 35

---

Key Features

• Detect slower and faster functions
• Identify new or removed functions
• Execution time and call count analysis
• Call graph visualization
• JSON and CSV export
• Regex-based filtering (--ignore)
• Top-N function focus (--top)
• CI regression gates
• Baseline save and compare commands

---

CLI Reference

Flag	Description
--json	Export trace to JSON
--csv	Export trace to CSV
--html	Export trace to html
--compare	Compare with another trace
--fail-on-regression	Exit with error if regression detected
--threshold	Regression percentage threshold
--ignore	Ignore functions/files via regex
--top	Show top N functions
--repeat	Repeat the tracing N times

Baseline commands:

oracletrace baseline save app.py baseline.json
oracletrace baseline compare baseline.json current.json --fail-on-regression --threshold 35

---

Use Cases

Primary

• Guard pull requests against Python performance regressions

Secondary

• CI performance validation
• Execution trace inspection
• Call graph visualization
• Release baseline tracking

---

How It Works

OracleTrace uses Python’s sys.setprofile() to intercept function calls and returns.

It measures execution time per function and records caller–callee relationships.

Filtering removes external/internal calls to focus on application code.

---

Requirements

• Python >= 3.11
• rich

When to use another tool

OracleTrace is for lightweight regression checks and function-level change detection. Use cProfile, py-spy, or a benchmark suite when you need deep profiler statistics, sampling of production processes, memory profiling, or statistically rigorous microbenchmarks.

---

Contributing

Contributions are welcome.

Please read the Contributing Guide for details on how to get started, coding standards, and the contribution process.

---

Contributors

---

⭐ Support the Project

If OracleTrace is useful, consider giving it a star:

GitHub Repository

---

Maintainers

• Kayk Caputo
• André Gustavo