Skip to content

Commit c9ce9f8

Browse files
gpsheadclaude
andcommitted
Merge branch 'main' into traceback-timestamps
Resolved conflicts in sysmodule.c (keep 3.13 in comment) and test_sys.py (adopt main's split of indexable vs name-only sys.flags attributes, add traceback_timestamps to the name-only section). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2 parents 3813d9e + 4a71946 commit c9ce9f8

File tree

14 files changed

+150
-31
lines changed

14 files changed

+150
-31
lines changed

Doc/library/stdtypes.rst

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -3514,6 +3514,11 @@ The representation of bytearray objects uses the bytes literal format
35143514
``bytearray([46, 46, 46])``. You can always convert a bytearray object into
35153515
a list of integers using ``list(b)``.
35163516

3517+
.. seealso::
3518+
3519+
For detailed information on thread-safety guarantees for :class:`bytearray`
3520+
objects, see :ref:`thread-safety-bytearray`.
3521+
35173522

35183523
.. _bytes-methods:
35193524

Doc/library/threadsafety.rst

Lines changed: 101 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -447,3 +447,104 @@ atomic:
447447
448448
Consider external synchronization when sharing :class:`set` instances
449449
across threads. See :ref:`freethreading-python-howto` for more information.
450+
451+
452+
.. _thread-safety-bytearray:
453+
454+
Thread safety for bytearray objects
455+
===================================
456+
457+
The :func:`len` function is lock-free and :term:`atomic <atomic operation>`.
458+
459+
Concatenation and comparisons use the buffer protocol, which prevents
460+
resizing but does not hold the per-object lock. These operations may
461+
observe intermediate states from concurrent modifications:
462+
463+
.. code-block::
464+
:class: maybe
465+
466+
ba + other # may observe concurrent writes
467+
ba == other # may observe concurrent writes
468+
ba < other # may observe concurrent writes
469+
470+
All other operations from here on hold the per-object lock.
471+
472+
Reading a single element or slice is safe to call from multiple threads:
473+
474+
.. code-block::
475+
:class: good
476+
477+
ba[i] # bytearray.__getitem__
478+
ba[i:j] # slice
479+
480+
The following operations are safe to call from multiple threads and will
481+
not corrupt the bytearray:
482+
483+
.. code-block::
484+
:class: good
485+
486+
ba[i] = x # write single byte
487+
ba[i:j] = values # write slice
488+
ba.append(x) # append single byte
489+
ba.extend(other) # extend with iterable
490+
ba.insert(i, x) # insert single byte
491+
ba.pop() # remove and return last byte
492+
ba.pop(i) # remove and return byte at index
493+
ba.remove(x) # remove first occurrence
494+
ba.reverse() # reverse in place
495+
ba.clear() # remove all bytes
496+
497+
Slice assignment locks both objects when *values* is a :class:`bytearray`:
498+
499+
.. code-block::
500+
:class: good
501+
502+
ba[i:j] = other_bytearray # both locked
503+
504+
The following operations return new objects and hold the per-object lock
505+
for the duration:
506+
507+
.. code-block::
508+
:class: good
509+
510+
ba.copy() # returns a shallow copy
511+
ba * n # repeat into new bytearray
512+
513+
The membership test holds the lock for its duration:
514+
515+
.. code-block::
516+
:class: good
517+
518+
x in ba # bytearray.__contains__
519+
520+
All other bytearray methods (such as :meth:`~bytearray.find`,
521+
:meth:`~bytearray.replace`, :meth:`~bytearray.split`,
522+
:meth:`~bytearray.decode`, etc.) hold the per-object lock for their
523+
duration.
524+
525+
Operations that involve multiple accesses, as well as iteration, are never
526+
atomic:
527+
528+
.. code-block::
529+
:class: bad
530+
531+
# NOT atomic: check-then-act
532+
if x in ba:
533+
ba.remove(x)
534+
535+
# NOT thread-safe: iteration while modifying
536+
for byte in ba:
537+
process(byte) # another thread may modify ba
538+
539+
To safely iterate over a bytearray that may be modified by another
540+
thread, iterate over a copy:
541+
542+
.. code-block::
543+
:class: good
544+
545+
# Make a copy to iterate safely
546+
for byte in ba.copy():
547+
process(byte)
548+
549+
Consider external synchronization when sharing :class:`bytearray` instances
550+
across threads. See :ref:`freethreading-python-howto` for more information.

Lib/base64.py

Lines changed: 7 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -100,7 +100,13 @@ def b64decode(s, altchars=None, validate=_NOT_SPECIFIED, *, ignorechars=_NOT_SPE
100100
break
101101
s = s.translate(bytes.maketrans(altchars, b'+/'))
102102
else:
103-
trans = bytes.maketrans(b'+/' + altchars, altchars + b'+/')
103+
trans_in = set(b'+/') - set(altchars)
104+
if len(trans_in) == 2:
105+
# we can't use the reqult of unordered sets here
106+
trans = bytes.maketrans(altchars + b'+/', b'+/' + altchars)
107+
else:
108+
trans = bytes.maketrans(altchars + bytes(trans_in),
109+
b'+/' + bytes(set(altchars) - set(b'+/')))
104110
s = s.translate(trans)
105111
ignorechars = ignorechars.translate(trans)
106112
if ignorechars is _NOT_SPECIFIED:

Lib/test/libregrtest/utils.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -150,7 +150,7 @@ def setup_unraisable_hook() -> None:
150150
sys.unraisablehook = regrtest_unraisable_hook
151151

152152

153-
orig_threading_excepthook: Callable[..., None] | None = None
153+
orig_threading_excepthook: Callable[..., object] | None = None
154154

155155

156156
def regrtest_threading_excepthook(args) -> None:

Lib/test/test_base64.py

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -293,6 +293,13 @@ def test_b64decode_altchars(self):
293293
eq(base64.b64decode(data_str, altchars=altchars_str), res)
294294
eq(base64.b64decode(data, altchars=altchars, ignorechars=b'\n'), res)
295295

296+
eq(base64.b64decode(b'/----', altchars=b'-+', ignorechars=b'/'), b'\xfb\xef\xbe')
297+
eq(base64.b64decode(b'/----', altchars=b'+-', ignorechars=b'/'), b'\xff\xff\xff')
298+
eq(base64.b64decode(b'+----', altchars=b'-/', ignorechars=b'+'), b'\xfb\xef\xbe')
299+
eq(base64.b64decode(b'+----', altchars=b'/-', ignorechars=b'+'), b'\xff\xff\xff')
300+
eq(base64.b64decode(b'+/+/', altchars=b'/+', ignorechars=b''), b'\xff\xef\xfe')
301+
eq(base64.b64decode(b'/+/+', altchars=b'+/', ignorechars=b''), b'\xff\xef\xfe')
302+
296303
self.assertRaises(ValueError, base64.b64decode, b'', altchars=b'+')
297304
self.assertRaises(ValueError, base64.b64decode, b'', altchars=b'+/-')
298305
self.assertRaises(ValueError, base64.b64decode, '', altchars='+')

Lib/test/test_sys.py

Lines changed: 18 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -860,30 +860,37 @@ def test_subinterp_intern_singleton(self):
860860
'''))
861861
self.assertTrue(sys._is_interned(s))
862862

863-
def test_sys_flags(self):
863+
def test_sys_flags_indexable_attributes(self):
864864
self.assertTrue(sys.flags)
865-
attrs = ("debug",
865+
# We've stopped assigning sequence indices to new sys.flags attributes:
866+
# https://github.com/python/cpython/issues/122575#issuecomment-2416497086
867+
indexable_attrs = ("debug",
866868
"inspect", "interactive", "optimize",
867869
"dont_write_bytecode", "no_user_site", "no_site",
868870
"ignore_environment", "verbose", "bytes_warning", "quiet",
869871
"hash_randomization", "isolated", "dev_mode", "utf8_mode",
870-
"warn_default_encoding", "safe_path", "int_max_str_digits",
871-
"lazy_imports", "traceback_timestamps")
872-
for attr in attrs:
872+
"warn_default_encoding", "safe_path", "int_max_str_digits")
873+
for attr_idx, attr in enumerate(indexable_attrs):
873874
self.assertHasAttr(sys.flags, attr)
874875
attr_type = bool if attr in ("dev_mode", "safe_path") else int
875876
attr_type = str if attr == "traceback_timestamps" else attr_type
876877
self.assertEqual(type(getattr(sys.flags, attr)), attr_type, attr)
878+
attr_value = getattr(sys.flags, attr)
879+
self.assertEqual(sys.flags[attr_idx], attr_value,
880+
msg=f"sys.flags .{attr} vs [{attr_idx}]")
877881
self.assertTrue(repr(sys.flags))
878-
self.assertEqual(len(sys.flags), 18) # Do not increase, see GH-122575
882+
self.assertEqual(len(sys.flags), 18, msg="Do not increase, see GH-122575")
879883

880884
self.assertIn(sys.flags.utf8_mode, {0, 1, 2})
881885

882-
# non-tuple sequence fields
886+
def test_sys_flags_name_only_attributes(self):
887+
# non-tuple sequence fields (name only sys.flags attributes)
883888
self.assertIsInstance(sys.flags.gil, int|type(None))
889+
self.assertIsInstance(sys.flags.thread_inherit_context, int|type(None))
890+
self.assertIsInstance(sys.flags.context_aware_warnings, int|type(None))
891+
self.assertIsInstance(sys.flags.lazy_imports, int|type(None))
884892
self.assertIsInstance(sys.flags.traceback_timestamps, str)
885893

886-
887894
def assert_raise_on_new_sys_type(self, sys_attr):
888895
# Users are intentionally prevented from creating new instances of
889896
# sys.flags, sys.version_info, and sys.getwindowsversion.
@@ -1923,7 +1930,9 @@ def test_pythontypes(self):
19231930
# - 'context_aware_warnings'
19241931
# - 'lazy_imports'
19251932
# - 'traceback_timestamps'
1926-
# It will not be necessary once GH-122575 is fixed.
1933+
# Not needing to increment this every time we add a new field
1934+
# per GH-122575 would be nice...
1935+
# Q: What is the actual point of this sys.flags C size derived from PyStructSequence_Field array assertion?
19271936
non_sequence_fields = 5
19281937
check(sys.flags, vsize('') + self.P + self.P * (non_sequence_fields + len(sys.flags)))
19291938

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
1+
Fix translation in :func:`base64.b64decode` when altchars overlaps with the
2+
standard ones.
Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
Remove :file:`Misc/vgrindefs` and :file:`Misc/Porting`.

Misc/Porting

Lines changed: 0 additions & 1 deletion
This file was deleted.

Misc/README

Lines changed: 0 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -11,7 +11,6 @@ ACKS Acknowledgements
1111
HISTORY News from previous releases -- oldest last
1212
indent.pro GNU indent profile approximating my C style
1313
NEWS News for this release (for some meaning of "this")
14-
Porting Mini-FAQ on porting to new platforms
1514
python-config.in Python script template for python-config
1615
python.man UNIX man page for the python interpreter
1716
python.pc.in Package configuration info template for pkg-config
@@ -22,4 +21,3 @@ SpecialBuilds.txt Describes extra symbols you can set for debug builds
2221
svnmap.txt Map of old SVN revs and branches to hg changeset ids,
2322
help history-digging
2423
valgrind-python.supp Valgrind suppression file, see README.valgrind
25-
vgrindefs Python configuration for vgrind (a generic pretty printer)

0 commit comments

Comments
 (0)