Skip to content

Commit 785dbd7

Browse files
gpsheadclaude
andcommitted
Update PEP to match implementation and resolve open issues
- Remove LLM-isms: rewrite bold-numbered lists as plain bullets, fix "key insight" phrasing - Fix type description: int64_t in C struct, not generic "integer" - Fix exception name: AsyncStopIteration -> StopAsyncIteration - Remove tutorial from How to Teach This (not needed for advanced feature) - Expand Backwards Compatibility with explicit pickle format details - Add colorized output mention to Display Format - Document control flow check uses C type pointer identity - Resolve all 5 open issues into Rejected Ideas: runtime API, retroactive timestamps, custom formats, always-collect, configurable control flow exception set - Rewrite "Always Collecting" as collection vs display distinction Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
1 parent 39978a4 commit 785dbd7

File tree

1 file changed

+147
-129
lines changed

1 file changed

+147
-129
lines changed

peps/pep-08XX.rst

Lines changed: 147 additions & 129 deletions
Original file line numberDiff line numberDiff line change
@@ -30,26 +30,23 @@ patterns and exception groups introduced in :pep:`654`, understanding the
3030
temporal relationships between exceptions has become increasingly important.
3131
Consider the following scenarios:
3232

33-
1. **Async servers with exception groups**: When multiple exceptions are caught
34-
and grouped together, it's often difficult to understand which exception
35-
occurred first and how they relate temporally to each other and to external
36-
events in the system.
33+
- When multiple exceptions are caught and grouped together, it can be difficult
34+
to understand which exception occurred first and how they relate temporally
35+
to each other and to external events in the system.
3736

38-
2. **Distributed systems debugging**: When correlating exceptions across
39-
different services, having precise timestamps allows developers to match
40-
exceptions with logs from other systems, making root cause analysis more
41-
straightforward.
37+
- When correlating exceptions across different services, precise timestamps
38+
allow developers to match exceptions with logs from other systems, making
39+
root cause analysis more straightforward.
4240

43-
3. **Complex exception chains**: In applications with deep exception chains
44-
(exceptions raised during handling of other exceptions), timestamps help
45-
clarify the sequence of events that led to the final error state.
41+
- In applications with deep exception chains (exceptions raised during handling
42+
of other exceptions), timestamps help clarify the sequence of events that
43+
led to the final error state.
4644

47-
4. **Performance debugging**: Timestamps can reveal performance issues, such as
48-
delays between exception creation and handling, or help identify timeout-related
49-
problems.
45+
- Timestamps can reveal performance issues such as delays between exception
46+
creation and handling, or help identify timeout-related problems.
5047

5148
Currently, developers must manually add timing information to exception messages
52-
or use external logging frameworks, which can be cumbersome and inconsistent.
49+
or use external logging frameworks, which is cumbersome and inconsistent.
5350
This PEP proposes a built-in, standardized approach.
5451

5552

@@ -144,63 +141,60 @@ Why Exception Groups Need Timestamps
144141
While exception groups are conceptually "unrelated" exceptions that happen to be
145142
raised together, in practice they often have important temporal relationships:
146143

147-
1. **Causality isn't always explicit**: When multiple services fail in sequence,
148-
one failure might trigger cascading failures in seemingly unrelated services.
149-
Without timestamps, these cascade patterns are invisible. For example, a
150-
database connection pool exhaustion might cause multiple "unrelated" query
151-
failures across different services.
144+
- Causality isn't always explicit. When multiple services fail in sequence,
145+
one failure might trigger cascading failures in seemingly unrelated services.
146+
Without timestamps, these cascade patterns are invisible. For example, a
147+
database connection pool exhaustion might cause multiple "unrelated" query
148+
failures across different services.
152149

153-
2. **Concurrent doesn't mean simultaneous**: Tasks in an exception group may
154-
start concurrently but fail at very different times. A service that fails
155-
after 100ms versus one that fails after 5 seconds tells a different story
156-
about what went wrong - the first might be a validation error, the second
157-
a timeout.
150+
- Concurrent doesn't mean simultaneous. Tasks in an exception group may
151+
start concurrently but fail at very different times. A service that fails
152+
after 100ms versus one that fails after 5 seconds tells a different story
153+
about what went wrong -- the first might be a validation error, the second
154+
a timeout.
158155

159-
3. **Debugging distributed systems**: In microservice architectures, exception
160-
groups often collect failures from multiple remote services. Timestamps allow
161-
correlation with external observability tools (logs, metrics, traces) that
162-
are essential for understanding the full picture.
156+
- In microservice architectures, exception groups often collect failures from
157+
multiple remote services. Timestamps allow correlation with external
158+
observability tools (logs, metrics, traces) that are essential for
159+
understanding the full picture.
163160

164-
4. **Performance analysis**: Even for "unrelated" exceptions, knowing their
165-
temporal distribution helps identify performance bottlenecks and timeout
166-
configurations that need adjustment.
161+
- Even for "unrelated" exceptions, knowing their temporal distribution helps
162+
identify performance bottlenecks and timeout configurations that need
163+
adjustment.
167164

168165

169166
Why Not Use ``.add_note()`` When Catching?
170167
--------------------------------------------
171168

172-
A common question is why we don't simply use :pep:`678`'s ``.add_note()`` to add
173-
timestamps when exceptions are caught and grouped. This approach has several
174-
significant drawbacks:
169+
A common question is why not simply use :pep:`678`'s ``.add_note()`` to add
170+
timestamps when exceptions are caught and grouped. This approach has several
171+
drawbacks:
175172

176-
1. **Not all exceptions are caught**: Exceptions that propagate to the top level
177-
or are logged directly never get the opportunity to have notes added. The
178-
timestamp of when an error occurred is lost forever.
173+
- Not all exceptions are caught. Exceptions that propagate to the top level
174+
or are logged directly never get the opportunity to have notes added.
179175

180-
2. **Timing accuracy**: Adding a note when catching introduces variable delay.
181-
The timestamp would reflect when the exception was caught and processed, not
182-
when it actually occurred. In async code with complex exception handling,
183-
this delay can be significant and misleading.
176+
- Adding a note when catching introduces variable delay. The timestamp would
177+
reflect when the exception was caught and processed, not when it actually
178+
occurred. In async code with complex exception handling, this delay can be
179+
significant and misleading.
184180

185-
3. **Inconsistent application**: Relying on exception handlers to add timestamps
186-
means some exceptions get timestamps and others don't, depending on code
187-
paths. This inconsistency makes debugging harder, not easier.
181+
- Relying on exception handlers to add timestamps means some exceptions get
182+
timestamps and others don't, depending on code paths.
188183

189-
4. **Performance overhead**: Creating note strings for every caught exception
190-
adds overhead even when timestamps aren't being displayed. With the proposed
191-
approach, formatting only happens when tracebacks are rendered.
184+
- Creating note strings for every caught exception adds overhead even when
185+
timestamps aren't being displayed. With the proposed approach, formatting
186+
only happens when tracebacks are rendered.
192187

193-
5. **Complexity burden**: Every exception handler that wants timing information
194-
would need to remember to add notes. This is error-prone and adds boilerplate
195-
to exception handling code.
188+
- Every exception handler that wants timing information would need to remember
189+
to add notes. This is error-prone and adds boilerplate.
196190

197-
6. **Lost original timing**: By the time an exception is caught, the original
198-
failure moment is lost. In retry loops or complex error handling, the catch
199-
point might be seconds or minutes after the actual error.
191+
- By the time an exception is caught, the original failure moment is lost.
192+
In retry loops or complex error handling, the catch point might be seconds
193+
or minutes after the actual error.
200194

201-
The key insight is that **when** an exception is created is intrinsic, immutable
202-
information about that exception - just like its type and message. This information
203-
should be captured at the source, not added later by consumers.
195+
When an exception is created is intrinsic, immutable information about that
196+
exception, just like its type and message. It should be captured at the source,
197+
not added later by consumers.
204198

205199

206200
Rationale
@@ -210,19 +204,19 @@ The decision to add timestamps directly to exception objects rather than using
210204
alternative approaches (such as exception notes from :pep:`678`) was driven by
211205
several factors:
212206

213-
1. **Performance**: Adding timestamps as notes would require creating string
214-
and list objects for every exception, even when timestamps aren't being
215-
displayed. The proposed approach stores a single integer (nanoseconds since
216-
epoch) and only formats it when needed.
207+
- Adding timestamps as notes would require creating string and list objects for
208+
every exception, even when timestamps aren't being displayed. The proposed
209+
approach stores a single ``int64_t`` in the C struct (nanoseconds since epoch)
210+
and only formats it when needed.
217211

218-
2. **Consistency**: Having a standardized timestamp attribute ensures consistent
219-
formatting and behavior across the Python ecosystem.
212+
- A standardized timestamp attribute ensures consistent formatting and behavior
213+
across the Python ecosystem.
220214

221-
3. **Backwards compatibility**: The feature is entirely opt-in, with no impact
222-
on existing code unless explicitly enabled.
215+
- The feature is entirely opt-in, with no impact on existing code unless
216+
explicitly enabled.
223217

224-
4. **Simplicity**: The implementation is straightforward and doesn't require
225-
changes to exception handling semantics.
218+
- The implementation is straightforward and doesn't require changes to exception
219+
handling semantics.
226220

227221

228222
Specification
@@ -248,14 +242,17 @@ To avoid performance impacts on normal control flow, timestamps will **not** be
248242
collected for certain exception types even when the feature is enabled. By default:
249243

250244
- ``StopIteration``
251-
- ``AsyncStopIteration``
245+
- ``StopAsyncIteration``
252246

253247
These exceptions are frequently used for control flow in iterators and async
254-
iterators. However, other projects also use custom exceptions for control flow
255-
(e.g., Sphinx's ``Skip`` exception), making the need for configurability clear.
248+
iterators and are raised at extremely high frequency during normal operation.
249+
The check uses C type pointer identity (not ``isinstance``) so it adds
250+
negligible overhead to exception creation.
256251

257-
The current implementation hard-codes these two exceptions for performance
258-
reasons. See Open Issues for discussion about making this configurable.
252+
Some third-party projects also use custom exceptions for control flow (e.g.,
253+
Sphinx's ``Skip`` exception), but these are not performance-critical enough
254+
to warrant special handling. See Rejected Ideas for discussion of making
255+
this set configurable.
259256

260257
Configuration
261258
-------------
@@ -298,6 +295,9 @@ Example with ``PYTHON_TRACEBACK_TIMESTAMPS=iso``::
298295
raise OSError(2, "on a cloudy day")
299296
FileNotFoundError: [Errno 2] on a cloudy day <@2025-02-01T20:43:01.026176Z>
300297

298+
When colorized traceback output is enabled, the timestamp is rendered in a
299+
muted color to keep it visually distinct from the exception message.
300+
301301
Traceback Module Updates
302302
------------------------
303303

@@ -326,11 +326,20 @@ Backwards Compatibility
326326

327327
This proposal maintains full backwards compatibility:
328328

329-
1. The feature is disabled by default
330-
2. Existing exception handling code continues to work unchanged
331-
3. The new attribute is only set when the feature is explicitly enabled
332-
4. Pickle/unpickle of exceptions works correctly with the new attribute
333-
5. Third-party exception formatting libraries can ignore the attribute if desired
329+
1. The feature is disabled by default.
330+
2. Existing exception handling code continues to work unchanged.
331+
3. The ``__timestamp_ns__`` attribute is always readable as a C member
332+
descriptor on all ``BaseException`` instances, returning ``0`` when
333+
timestamps are not collected. It is only populated with a nonzero
334+
value when the feature is enabled.
335+
4. Pickle format is unchanged when timestamps are disabled: exceptions
336+
pickle as the traditional 2-tuple ``(type, args)``. When a nonzero
337+
timestamp is present, the exception pickles as a 3-tuple
338+
``(type, args, state_dict)`` with ``__timestamp_ns__`` included in
339+
the state dictionary. Older Python versions will unpickle these
340+
correctly, simply setting the extra attribute via ``__setstate__``.
341+
5. Third-party exception formatting libraries can ignore the attribute
342+
if desired.
334343

335344

336345
Security Implications
@@ -354,8 +363,7 @@ The feature should be documented in:
354363

355364
1. The Python documentation for the ``exceptions`` module
356365
2. The ``traceback`` module documentation
357-
3. The Python tutorial section on exception handling (as an advanced topic)
358-
4. The command-line interface documentation
366+
3. The command-line interface documentation
359367

360368
Example use cases should focus on:
361369

@@ -375,7 +383,7 @@ The implementation includes:
375383
- Core exception object changes to add the ``__timestamp_ns__`` attribute
376384
- Traceback formatting updates to display timestamps
377385
- Configuration through environment variables and command-line options
378-
- Special handling for ``StopIteration`` and ``AsyncStopIteration``
386+
- Special handling for ``StopIteration`` and ``StopAsyncIteration``
379387
- Comprehensive test coverage
380388
- Documentation updates
381389

@@ -393,15 +401,64 @@ An alternative approach would be to use the exception notes feature from
393401
2. The performance impact would be significant even when not displaying timestamps
394402
3. It would make the timestamp less structured and harder to process programmatically
395403

396-
Always Collecting Timestamps
397-
-----------------------------
398-
399-
Collecting timestamps for all exceptions unconditionally was considered but
400-
rejected due to:
401-
402-
1. Performance overhead for exceptions used in control flow
403-
2. Unnecessary memory usage for the vast majority of use cases
404-
3. Potential security concerns in production environments
404+
Always Collecting vs. Always Displaying
405+
----------------------------------------
406+
407+
It is important to distinguish between *collecting* timestamps (calling
408+
``clock_gettime`` during exception instantiation) and *displaying* them in
409+
formatted tracebacks.
410+
411+
Always displaying timestamps was rejected because it adds noise to traceback
412+
output that most users do not need.
413+
414+
Always collecting timestamps (even when display is disabled) was considered.
415+
The ``int64_t`` field exists in the ``BaseException`` C struct regardless,
416+
so there is no additional memory cost. The ``clock_gettime`` call itself is
417+
cheap for most exceptions. However, there was no compelling reason to collect
418+
timestamps when they would never be displayed, so collection is currently tied
419+
to the display configuration. This could be revisited as a separate future
420+
feature if programmatic access to exception timestamps proves useful
421+
independent of traceback display.
422+
423+
Runtime API for Enabling/Disabling Timestamps
424+
----------------------------------------------
425+
426+
A Python API to programmatically enable or disable timestamps at runtime was
427+
considered and rejected. There is no obvious use case, and global behavior
428+
changes at runtime are problematic for reasoning about program state. The
429+
feature is configured at startup via environment variable or command-line
430+
option and remains fixed for the lifetime of the process.
431+
432+
Retroactive Timestamps on Existing Exceptions
433+
-----------------------------------------------
434+
435+
Adding timestamps to already-created exception objects was considered and
436+
rejected. The timestamp represents when the exception was instantiated;
437+
adding one after the fact would be misleading. Users who need to annotate
438+
existing exceptions with timing information can use exception notes
439+
(:pep:`678`).
440+
441+
Custom Timestamp Formats
442+
-------------------------
443+
444+
Making the timestamp format string customizable beyond the predefined
445+
``us``/``ns``/``iso`` options was considered and rejected in favor of
446+
simplicity. The three built-in formats cover the common needs: human-readable
447+
with microsecond precision, raw nanoseconds for programmatic use, and ISO 8601
448+
for correlation with external systems.
449+
450+
Configurable Control Flow Exception Set
451+
-----------------------------------------
452+
453+
Making the set of timestamp-excluded control flow exceptions configurable was
454+
considered and rejected. The exclusion check is in the hot path of exception
455+
creation and must use C type pointer identity comparison to be fast enough.
456+
Supporting a user-configurable set would require either an isinstance check
457+
(too slow, as it walks the class hierarchy) or maintaining a hash set of type
458+
pointers (added complexity with unclear benefit). The hard-coded exclusion of
459+
``StopIteration`` and ``StopAsyncIteration`` is sufficient: these are the only
460+
exceptions used pervasively for control flow at frequencies where a
461+
``clock_gettime`` call would be measurable.
405462

406463
Millisecond Precision
407464
---------------------
@@ -417,46 +474,7 @@ nanosecond precision was chosen to:
417474
Open Issues
418475
===========
419476

420-
1. Should there be a Python API to programmatically enable/disable timestamps
421-
at runtime?
422-
423-
2. Should there be a way to retroactively add timestamps to existing exception
424-
objects?
425-
426-
3. Should the timestamp format be customizable beyond the predefined options?
427-
428-
4. **Always collecting timestamps vs. conditional collection**: Performance testing
429-
shows that collecting timestamps at exception instantiation time is cheap enough
430-
to do unconditionally. If we always collect them:
431-
432-
- The ``__timestamp_ns__`` attribute would always exist, simplifying the
433-
implementation and making the pickle code cleaner (though pickled exceptions
434-
would be slightly larger)
435-
- Exceptions will unpickle cleanly on older Python versions (they'll just have
436-
an extra attribute that older versions ignore)
437-
- However, we don't currently have extensive testing for cross-version pickle
438-
compatibility of exceptions with new attributes. Should we add such tests?
439-
Is this level of compatibility testing necessary?
440-
441-
5. **Control flow exception handling**: The current implementation does not collect
442-
timestamps for ``StopIteration`` and ``AsyncStopIteration`` to avoid performance
443-
impact on normal control flow. However, other projects use custom exceptions for
444-
control flow (e.g., Sphinx's ``Skip`` exception), requiring configurability.
445-
446-
Key challenges:
447-
448-
- **API Design**: What's the best way to allow projects to register their
449-
control-flow exceptions? Environment variable? Python API? Both?
450-
- **Performance**: The check is in the hot path of exception creation and must
451-
be extremely fast. How can we make this configurable without impacting
452-
performance for the common case?
453-
- **Subclass handling**: Should exclusions apply to subclasses? This adds
454-
complexity and performance overhead.
455-
- **Default set**: Should we expand the default exclusion list beyond
456-
``StopIteration`` and ``AsyncStopIteration``?
457-
458-
This needs careful API design and performance testing before committing to
459-
a specific approach.
477+
None at this time.
460478

461479

462480
Acknowledgements

0 commit comments

Comments
 (0)