Skip to content

Commit ef67c5d

Browse files
committed
pythongh-149202: Fix frame pointer unwinding on s390x and ARM
-fno-omit-frame-pointer is not enough to make every target walkable by the simple manual frame pointer unwinder. On s390x, GCC and Clang do not emit a usable backchain unless -mbackchain is also enabled. Without it, the unwinder stops at the current C frame and the test reports no Python frames. Once backchains are present, the helper must also stop at the current thread's known C stack bounds; otherwise it can follow the final backchain far enough to dereference an invalid frame and segfault. For Linux s390x backchain frames, the documented z/Architecture stack-frame layout saves r14, the return-address register, at byte offset 112 from the frame pointer, so read the return address from that named slot. The 112-byte offset comes from Linux's s390 debugging documentation: its Stack Frame Layout table shows z/Architecture backchain frames with the backchain at offset 0 and saved r14 of the caller function at offset 112: https://www.kernel.org/doc/html/v5.3/s390/debugging390.html#stack-frame-layout This helper remains scoped to Linux s390x backchain frames. GNU SFrame's s390x notes state that the s390x ELF ABI does not generally mandate where RA and FP are saved, or whether they are saved at all: https://sourceware.org/binutils/docs/sframe-spec.html#s390x On 32-bit ARM, GCC defaults to Thumb mode on common armhf toolchains. The Thumb prologue keeps the saved frame pointer and link register at offsets that depend on the generated frame, which breaks the fp[0]/fp[1] walk used by the helper. Use -marm when it is supported for frame-pointer builds, and teach the helper the GCC ARM-mode saved-fp and saved-lr slots.
1 parent 3efd2f4 commit ef67c5d

7 files changed

Lines changed: 250 additions & 24 deletions

File tree

Doc/howto/perf_profiling.rst

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -219,7 +219,9 @@ How to obtain the best results
219219

220220
For best results, keep frame pointers enabled. On supported GCC-compatible
221221
toolchains, CPython builds itself with ``-fno-omit-frame-pointer`` and, when
222-
available, ``-mno-omit-leaf-frame-pointer`` by default. These flags allow
222+
available, ``-mno-omit-leaf-frame-pointer`` by default. On 32-bit ARM,
223+
CPython also adds ``-marm`` when supported. On s390 platforms, CPython also
224+
adds ``-mbackchain`` when supported. These flags allow
223225
profilers to unwind using only the frame pointer and not on DWARF debug
224226
information. This is because as the code that is interposed to allow ``perf``
225227
support is dynamically generated it doesn't have any DWARF debugging information

Doc/using/configure.rst

Lines changed: 5 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -784,9 +784,11 @@ also be used to improve performance.
784784

785785
Disable frame pointers, which are enabled by default (see :pep:`831`).
786786

787-
By default, the build appends ``-fno-omit-frame-pointer`` (and
788-
``-mno-omit-leaf-frame-pointer`` when the compiler supports it) to
789-
``BASECFLAGS`` so profilers, debuggers, and system tracing tools
787+
By default, the build appends ``-fno-omit-frame-pointer``,
788+
``-mno-omit-leaf-frame-pointer`` when the compiler supports it,
789+
``-marm`` on 32-bit ARM when supported, and ``-mbackchain`` on s390
790+
platforms when supported, to ``BASECFLAGS`` so
791+
profilers, debuggers, and system tracing tools
790792
(``perf``, ``eBPF``, ``dtrace``, ``gdb``) can walk the C call stack
791793
without DWARF metadata. The flags propagate to third-party C
792794
extensions through :mod:`sysconfig`. On compilers that do not

Doc/whatsnew/3.15.rst

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -2305,8 +2305,9 @@ Build changes
23052305
(:pep:`831`). Pass :option:`--without-frame-pointers` to opt out.
23062306
Authors of C extensions and native libraries built with custom build
23072307
systems should add ``-fno-omit-frame-pointer`` and
2308-
``-mno-omit-leaf-frame-pointer`` to their own ``CFLAGS`` to keep the
2309-
unwind chain intact.
2308+
``-mno-omit-leaf-frame-pointer`` to their own ``CFLAGS``,
2309+
``-marm`` on 32-bit ARM, and ``-mbackchain`` on s390 platforms,
2310+
to keep the unwind chain intact.
23102311
(Contributed by Pablo Galindo Salgado and Savannah Ostrowski in :gh:`149201`.)
23112312

23122313
.. _whatsnew315-windows-tail-calling-interpreter:
Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,5 @@
11
Enable frame pointers by default for GCC-compatible CPython builds, including
2-
``-mno-omit-leaf-frame-pointer`` when the compiler supports it, so profilers
3-
and debuggers can unwind native interpreter frames more reliably. Users can pass
2+
``-mno-omit-leaf-frame-pointer``, ``-marm`` on 32-bit ARM, and ``-mbackchain``
3+
on s390 platforms when the compiler supports them, so profilers and debuggers
4+
can unwind native interpreter frames more reliably. Users can pass
45
``--without-frame-pointers`` to opt out.

Modules/_testinternalcapi.c

Lines changed: 126 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -59,6 +59,33 @@
5959

6060
static const uintptr_t min_frame_pointer_addr = 0x1000;
6161

62+
#ifdef __s390x__
63+
// Linux's s390 "Stack Frame Layout" table documents that z/Architecture
64+
// backchain frames start with the backchain at offset 0 and store "saved r14
65+
// of caller function" at offset 112. The same document's register table
66+
// identifies r14 as the return-address register, so this backchain unwinder
67+
// reads the return address from fp + 112.
68+
// https://www.kernel.org/doc/html/v5.3/s390/debugging390.html#stack-frame-layout
69+
//
70+
// This is only for Linux s390x backchain frames. The s390x ELF ABI does not
71+
// generally mandate where RA and FP are saved, or whether they are saved at all.
72+
// https://sourceware.org/binutils/docs/sframe-spec.html#s390x
73+
# define S390X_FRAME_RETURN_ADDRESS_OFFSET 112
74+
#endif
75+
76+
#if defined(__arm__) && !defined(__thumb__) && !defined(__clang__)
77+
// GCC ARM mode keeps the caller's fp below fp and the saved LR at fp[0].
78+
# define FRAME_POINTER_NEXT_OFFSET (-1)
79+
# define FRAME_POINTER_RETURN_OFFSET 0
80+
#elif defined(__s390x__)
81+
# define FRAME_POINTER_NEXT_OFFSET 0
82+
# define FRAME_POINTER_RETURN_OFFSET \
83+
(S390X_FRAME_RETURN_ADDRESS_OFFSET / (Py_ssize_t)sizeof(uintptr_t))
84+
#else
85+
# define FRAME_POINTER_NEXT_OFFSET 0
86+
# define FRAME_POINTER_RETURN_OFFSET 1
87+
#endif
88+
6289

6390
static PyObject *
6491
_get_current_module(void)
@@ -325,16 +352,97 @@ get_jit_backend(PyObject *self, PyObject *Py_UNUSED(args))
325352
#endif
326353
}
327354

355+
static int
356+
stack_address_is_valid(uintptr_t addr, uintptr_t stack_min, uintptr_t stack_max)
357+
{
358+
if (addr < min_frame_pointer_addr) {
359+
return 0;
360+
}
361+
if (stack_min != 0 && (addr < stack_min || addr >= stack_max)) {
362+
return 0;
363+
}
364+
return 1;
365+
}
366+
367+
static int
368+
frame_pointer_slot_is_valid(uintptr_t *frame_pointer, Py_ssize_t offset,
369+
uintptr_t stack_min, uintptr_t stack_max)
370+
{
371+
uintptr_t fp_addr = (uintptr_t)frame_pointer;
372+
uintptr_t slot_addr;
373+
uintptr_t delta = (uintptr_t)Py_ABS(offset) * sizeof(uintptr_t);
374+
if (offset < 0) {
375+
if (fp_addr < delta) {
376+
return 0;
377+
}
378+
slot_addr = fp_addr - delta;
379+
}
380+
else {
381+
if (fp_addr > UINTPTR_MAX - delta) {
382+
return 0;
383+
}
384+
slot_addr = fp_addr + delta;
385+
}
386+
if (!stack_address_is_valid(slot_addr, stack_min, stack_max)) {
387+
return 0;
388+
}
389+
if (stack_max != 0) {
390+
if (slot_addr > UINTPTR_MAX - sizeof(uintptr_t)) {
391+
return 0;
392+
}
393+
if (slot_addr + sizeof(uintptr_t) > stack_max) {
394+
return 0;
395+
}
396+
}
397+
return 1;
398+
}
399+
400+
static int
401+
next_frame_pointer_is_valid(uintptr_t *frame_pointer, uintptr_t *next_fp,
402+
uintptr_t stack_min, uintptr_t stack_max)
403+
{
404+
uintptr_t fp_addr = (uintptr_t)frame_pointer;
405+
uintptr_t next_addr = (uintptr_t)next_fp;
406+
if (!stack_address_is_valid(next_addr, stack_min, stack_max)) {
407+
return 0;
408+
}
409+
if ((next_addr % sizeof(uintptr_t)) != 0) {
410+
return 0;
411+
}
412+
#if _Py_STACK_GROWS_DOWN
413+
return next_addr > fp_addr;
414+
#else
415+
return next_addr < fp_addr;
416+
#endif
417+
}
418+
328419
static PyObject *
329420
manual_unwind_from_fp(uintptr_t *frame_pointer)
330421
{
331422
Py_ssize_t max_depth = 200;
332-
int stack_grows_down = _Py_STACK_GROWS_DOWN;
423+
uintptr_t stack_min = 0;
424+
uintptr_t stack_max = 0;
425+
426+
#ifdef __s390x__
427+
Py_BUILD_ASSERT(S390X_FRAME_RETURN_ADDRESS_OFFSET % sizeof(uintptr_t) == 0);
428+
#endif
333429

334430
if (frame_pointer == NULL) {
335431
return PyList_New(0);
336432
}
337433

434+
PyThreadState *tstate = _PyThreadState_GET();
435+
if (tstate != NULL) {
436+
_PyThreadStateImpl *tstate_impl = (_PyThreadStateImpl *)tstate;
437+
#if _Py_STACK_GROWS_DOWN
438+
stack_min = tstate_impl->c_stack_hard_limit;
439+
stack_max = tstate_impl->c_stack_top;
440+
#else
441+
stack_min = tstate_impl->c_stack_top;
442+
stack_max = tstate_impl->c_stack_hard_limit;
443+
#endif
444+
}
445+
338446
PyObject *result = PyList_New(0);
339447
if (result == NULL) {
340448
return NULL;
@@ -348,7 +456,21 @@ manual_unwind_from_fp(uintptr_t *frame_pointer)
348456
if ((fp_addr % sizeof(uintptr_t)) != 0) {
349457
break;
350458
}
351-
uintptr_t return_addr = frame_pointer[1];
459+
if (!stack_address_is_valid(fp_addr, stack_min, stack_max)) {
460+
break;
461+
}
462+
if (!frame_pointer_slot_is_valid(frame_pointer,
463+
FRAME_POINTER_NEXT_OFFSET,
464+
stack_min, stack_max)) {
465+
break;
466+
}
467+
if (!frame_pointer_slot_is_valid(frame_pointer,
468+
FRAME_POINTER_RETURN_OFFSET,
469+
stack_min, stack_max)) {
470+
break;
471+
}
472+
uintptr_t *next_fp = (uintptr_t *)frame_pointer[FRAME_POINTER_NEXT_OFFSET];
473+
uintptr_t return_addr = frame_pointer[FRAME_POINTER_RETURN_OFFSET];
352474

353475
PyObject *addr_obj = PyLong_FromUnsignedLongLong(return_addr);
354476
if (addr_obj == NULL) {
@@ -362,22 +484,10 @@ manual_unwind_from_fp(uintptr_t *frame_pointer)
362484
}
363485
Py_DECREF(addr_obj);
364486

365-
uintptr_t *next_fp = (uintptr_t *)frame_pointer[0];
366-
// Stop if the frame pointer is extremely low.
367-
if ((uintptr_t)next_fp < min_frame_pointer_addr) {
487+
if (!next_frame_pointer_is_valid(frame_pointer, next_fp,
488+
stack_min, stack_max)) {
368489
break;
369490
}
370-
uintptr_t next_addr = (uintptr_t)next_fp;
371-
if (stack_grows_down) {
372-
if (next_addr <= fp_addr) {
373-
break;
374-
}
375-
}
376-
else {
377-
if (next_addr >= fp_addr) {
378-
break;
379-
}
380-
}
381491
frame_pointer = next_fp;
382492
}
383493

configure

Lines changed: 100 additions & 0 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

configure.ac

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2548,6 +2548,16 @@ AS_VAR_IF([ac_cv_gcc_compat], [yes], [
25482548
AX_CHECK_COMPILE_FLAG([-mno-omit-leaf-frame-pointer], [
25492549
frame_pointer_cflags="$frame_pointer_cflags -mno-omit-leaf-frame-pointer"
25502550
], [], [-Werror])
2551+
AS_CASE([$host_cpu], [arm|armv*], [
2552+
AX_CHECK_COMPILE_FLAG([-marm], [
2553+
frame_pointer_cflags="$frame_pointer_cflags -marm"
2554+
], [], [-Werror])
2555+
])
2556+
AS_CASE([$host_cpu], [s390*], [
2557+
AX_CHECK_COMPILE_FLAG([-mbackchain], [
2558+
frame_pointer_cflags="$frame_pointer_cflags -mbackchain"
2559+
], [], [-Werror])
2560+
])
25512561
], [], [-Werror])
25522562
if test -n "$frame_pointer_cflags" && test "x$with_frame_pointers" != xno; then
25532563
BASECFLAGS="$frame_pointer_cflags $BASECFLAGS"

0 commit comments

Comments
 (0)