Skip to content

gh-130920: Fix data race in STORE_SUBSCR_LIST_INT#130923

Merged
colesbury merged 1 commit intopython:mainfrom
colesbury:gh-130920-list-subscr-int
Mar 6, 2025
Merged

gh-130920: Fix data race in STORE_SUBSCR_LIST_INT#130923
colesbury merged 1 commit intopython:mainfrom
colesbury:gh-130920-list-subscr-int

Conversation

@colesbury
Copy link
Copy Markdown
Contributor

@colesbury colesbury commented Mar 6, 2025 •
edited by bedevere-app bot
Loading

The write of the item to the list needs to use an atomic operation in the free threading build.

@colesbury @tom-pytel
pythongh-130920: Fix data race in STORE_SUBSCR_LIST_INT
3238136 
The write of the item to the list needs to use an atomic operation in
the free threading build.

Co-authored-by: Tomasz Pytel <tompytel@gmail.com>
@colesbury colesbury requested a review from mpage March 6, 2025 17:49
@colesbury colesbury requested a review from markshannon as a code owner March 6, 2025 17:50
@bedevere-app bedevere-app bot mentioned this pull request Mar 6, 2025
Closed
@colesbury
Copy link
Copy Markdown
Contributor Author

colesbury commented Mar 6, 2025

Test is adapted from @tom-pytel's code

mpage
mpage approved these changes Mar 6, 2025
View reviewed changes
Copy link
Copy Markdown
Contributor

@mpage mpage left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Why did you change the specialized instruction instead of PyList_SET_ITEM?

@colesbury
Copy link
Copy Markdown
Contributor Author

colesbury commented Mar 6, 2025 •
edited
Loading

We use PyList_SET_ITEM and PyList_GET_ITEM in contexts where we know the list is not exposed to other threads. I don't think we should try to make them thread-safe.

https://docs.python.org/3/c-api/list.html#c.PyList_SET_ITEM

@colesbury colesbury merged commit a025f27 into python:main Mar 6, 2025
72 checks passed
@colesbury colesbury deleted the gh-130920-list-subscr-int branch March 6, 2025 20:59
@bedevere-bot
Copy link
Copy Markdown

bedevere-bot commented Mar 6, 2025

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot aarch64 RHEL8 LTO 3.x (tier-2) has failed when building commit a025f27.

What do you need to do:

  1. Don't panic.
  2. Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
  3. Go to the page of the buildbot that failed (https://buildbot.python.org/#/builders/338/builds/8216) and take a look at the build logs.
  4. Check if the failure is related to this commit (a025f27) or if it is a false positive.
  5. If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/#/builders/338/builds/8216

Failed tests:

  • test.test_multiprocessing_spawn.test_manager

Summary of the results of the build (if available):

==

Click to see traceback logs 
Traceback (most recent call last):
  File "/home/buildbot/buildarea/3.x.cstratak-RHEL8-aarch64.lto/build/Lib/multiprocessing/managers.py", line 266, in serve_client
    raise ke
  File "/home/buildbot/buildarea/3.x.cstratak-RHEL8-aarch64.lto/build/Lib/multiprocessing/managers.py", line 260, in serve_client
    obj, exposed, gettypeid = id_to_obj[ident]
                              ~~~~~~~~~^^^^^^^
KeyError: 'ffff7d676450'
---------------------------------------------------------------------------
Timeout (0:20:00)!
Thread 0x0000ffff9d684590 (most recent call first):
  File "/home/buildbot/buildarea/3.x.cstratak-RHEL8-aarch64.lto/build/Lib/multiprocessing/popen_fork.py", line 28 in poll
  File "/home/buildbot/buildarea/3.x.cstratak-RHEL8-aarch64.lto/build/Lib/multiprocessing/popen_fork.py", line 44 in wait
  File "/home/buildbot/buildarea/3.x.cstratak-RHEL8-aarch64.lto/build/Lib/multiprocessing/process.py", line 149 in join
  File "/home/buildbot/buildarea/3.x.cstratak-RHEL8-aarch64.lto/build/Lib/unittest/case.py", line 623 in _callCleanup
  File "/home/buildbot/buildarea/3.x.cstratak-RHEL8-aarch64.lto/build/Lib/unittest/case.py", line 697 in doCleanups
  File "/home/buildbot/buildarea/3.x.cstratak-RHEL8-aarch64.lto/build/Lib/unittest/case.py", line 664 in run
  File "/home/buildbot/buildarea/3.x.cstratak-RHEL8-aarch64.lto/build/Lib/unittest/case.py", line 716 in __call__
  File "/home/buildbot/buildarea/3.x.cstratak-RHEL8-aarch64.lto/build/Lib/unittest/suite.py", line 122 in run
  File "/home/buildbot/buildarea/3.x.cstratak-RHEL8-aarch64.lto/build/Lib/unittest/suite.py", line 84 in __call__
  File "/home/buildbot/buildarea/3.x.cstratak-RHEL8-aarch64.lto/build/Lib/unittest/suite.py", line 122 in run
  File "/home/buildbot/buildarea/3.x.cstratak-RHEL8-aarch64.lto/build/Lib/unittest/suite.py", line 84 in __call__
  File "/home/buildbot/buildarea/3.x.cstratak-RHEL8-aarch64.lto/build/Lib/unittest/runner.py", line 259 in run
  File "/home/buildbot/buildarea/3.x.cstratak-RHEL8-aarch64.lto/build/Lib/test/libregrtest/single.py", line 84 in _run_suite
  File "/home/buildbot/buildarea/3.x.cstratak-RHEL8-aarch64.lto/build/Lib/test/libregrtest/single.py", line 42 in run_unittest
  File "/home/buildbot/buildarea/3.x.cstratak-RHEL8-aarch64.lto/build/Lib/test/libregrtest/single.py", line 162 in test_func
  File "/home/buildbot/buildarea/3.x.cstratak-RHEL8-aarch64.lto/build/Lib/test/libregrtest/single.py", line 118 in regrtest_runner
  File "/home/buildbot/buildarea/3.x.cstratak-RHEL8-aarch64.lto/build/Lib/test/libregrtest/single.py", line 165 in _load_run_test
  File "/home/buildbot/buildarea/3.x.cstratak-RHEL8-aarch64.lto/build/Lib/test/libregrtest/single.py", line 210 in _runtest_env_changed_exc
  File "/home/buildbot/buildarea/3.x.cstratak-RHEL8-aarch64.lto/build/Lib/test/libregrtest/single.py", line 319 in _runtest
  File "/home/buildbot/buildarea/3.x.cstratak-RHEL8-aarch64.lto/build/Lib/test/libregrtest/single.py", line 348 in run_single_test
  File "/home/buildbot/buildarea/3.x.cstratak-RHEL8-aarch64.lto/build/Lib/test/libregrtest/worker.py", line 92 in worker_process
  File "/home/buildbot/buildarea/3.x.cstratak-RHEL8-aarch64.lto/build/Lib/test/libregrtest/worker.py", line 127 in main
  File "/home/buildbot/buildarea/3.x.cstratak-RHEL8-aarch64.lto/build/Lib/test/libregrtest/worker.py", line 131 in <module>
  File "<frozen runpy>", line 88 in _run_code
  File "<frozen runpy>", line 198 in _run_module_as_main


Traceback (most recent call last):
  File "/home/buildbot/buildarea/3.x.cstratak-RHEL8-aarch64.lto/build/Lib/multiprocessing/process.py", line 313, in _bootstrap
    self.run()
    ~~~~~~~~^^
  File "/home/buildbot/buildarea/3.x.cstratak-RHEL8-aarch64.lto/build/Lib/multiprocessing/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
    ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/buildbot/buildarea/3.x.cstratak-RHEL8-aarch64.lto/build/Lib/test/_test_multiprocessing.py", line 1625, in f
    woken.release()
    ~~~~~~~~~~~~~^^
  File "/home/buildbot/buildarea/3.x.cstratak-RHEL8-aarch64.lto/build/Lib/multiprocessing/managers.py", line 1067, in release
    return self._callmethod('release')
           ~~~~~~~~~~~~~~~~^^^^^^^^^^^
  File "/home/buildbot/buildarea/3.x.cstratak-RHEL8-aarch64.lto/build/Lib/multiprocessing/managers.py", line 848, in _callmethod
    raise convert_to_error(kind, result)
multiprocessing.managers.RemoteError: 
---------------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/buildbot/buildarea/3.x.cstratak-RHEL8-aarch64.lto/build/Lib/multiprocessing/managers.py", line 264, in serve_client
    self.id_to_local_proxy_obj[ident]
    ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^
KeyError: 'ffff7d676450'


Traceback (most recent call last):
  File "/home/buildbot/buildarea/3.x.cstratak-RHEL8-aarch64.lto/build/Lib/threading.py", line 1054, in _bootstrap_inner
    self.run()
    ~~~~~~~~^^
  File "/home/buildbot/buildarea/3.x.cstratak-RHEL8-aarch64.lto/build/Lib/threading.py", line 996, in run
    self._target(*self._args, **self._kwargs)
    ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/buildbot/buildarea/3.x.cstratak-RHEL8-aarch64.lto/build/Lib/test/test_interpreters/test_stress.py", line 30, in task
    interp = interpreters.create()
  File "/home/buildbot/buildarea/3.x.cstratak-RHEL8-aarch64.lto/build/Lib/test/support/interpreters/__init__.py", line 76, in create
    id = _interpreters.create(reqrefs=True)
interpreters.InterpreterError: interpreter creation failed
k

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@mpage mpage mpage approved these changes

@markshannon markshannon Awaiting requested review from markshannon markshannon is a code owner

Assignees

No one assigned

Labels

skip news topic-free-threading

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@colesbury @bedevere-bot @mpage