[Sampling profiler, 3.15+] sample all interpreters

[taegyunkim]

Feature or enhancement

Proposal:

Please correct me if I misunderstood the code.

RemoteUnwinder.get_stack_trace() only looks at the head interpreter

• it gets the address of interpreters.head from _PyRuntimeState from these lines of code

  cpython/Modules/_remote_debugging_module.c

  Lines 2089 to 2106 in c19db1d

  	uintptr_t address_of_interpreter_state;
  	int bytes_read = _Py_RemoteDebug_PagedReadRemoteMemory(
  	&unwinder->handle,
  	runtime_start_address + interpreter_state_list_head,
  	sizeof(void*),
  	&address_of_interpreter_state);
  	if (bytes_read < 0) {
  	set_exception_cause(unwinder, PyExc_RuntimeError, "Failed to read interpreter state address");
  	return -1;
  	}
  	if (address_of_interpreter_state == 0) {
  	PyErr_SetString(PyExc_RuntimeError, "No interpreter state found");
  	set_exception_cause(unwinder, PyExc_RuntimeError, "Interpreter state is NULL");
  	return -1;
  	}
  	*interpreter_state = address_of_interpreter_state;

• _PyInterpreterState for the head interpreter is copied in these lines

  cpython/Modules/_remote_debugging_module.c

  Lines 2691 to 2699 in c19db1d

  	char interp_state_buffer[INTERP_STATE_BUFFER_SIZE];
  	if (_Py_RemoteDebug_PagedReadRemoteMemory(
  	&self->handle,
  	self->interpreter_addr,
  	INTERP_STATE_BUFFER_SIZE,
  	interp_state_buffer) < 0) {
  	set_exception_cause(self, PyExc_RuntimeError, "Failed to read interpreter state buffer");
  	goto exit;
  	}

• Then, only iterate over PyThreadStates in that interpreter, and doesn't iterate over next interpreters:

  cpython/Modules/_remote_debugging_module.c

  Lines 2760 to 2785 in c19db1d

  	while (current_tstate != 0) {
  	PyObject* frame_info = unwind_stack_for_thread(self, &current_tstate);
  	if (!frame_info) {
  	Py_CLEAR(result);
  	set_exception_cause(self, PyExc_RuntimeError, "Failed to unwind stack for thread");
  	goto exit;
  	}
  	if (PyList_Append(result, frame_info) == -1) {
  	Py_DECREF(frame_info);
  	Py_CLEAR(result);
  	set_exception_cause(self, PyExc_RuntimeError, "Failed to append thread frame info");
  	goto exit;
  	}
  	Py_DECREF(frame_info);
  	// We are targeting a single tstate, break here
  	if (self->tstate_addr) {
  	break;
  	}
  	// If we're only processing the GIL holder, we're done after one iteration
  	if (self->only_active_thread && gil_holder != NULL) {
  	break;
  	}
  	}

Has this already been discussed elsewhere?

This is a minor feature, which does not need previous discussion elsewhere

Links to previous discussion of this feature:

No response

Linked PRs

• gh-138385: Sample all interpreters in the tachyon profiler #138398

[picnixz]

cc @pablogsal

[pablogsal]

This is indeed planned work before we release 3.15 :)

We are currently focused on the basics and on implementing PEP 799