fix(tracing): keep numpy loaded during module cleanup

[arpitjain099]

Description

When numpy 2.4+ is imported both before and after ddtrace.auto, the second import crashes with ImportError: cannot load module more than once per process. numpy 2.4 added a guard (numpy/numpy#29030) that refuses to re-initialize its C extension, and cleanup_loaded_modules() drops numpy from sys.modules because it is not in KEEP_MODULES, so re-importing it afterwards trips that guard. This adds numpy to KEEP_MODULES, which is the same fix already applied to google.protobuf in #6346 for the identical reload restriction. Fixes #18276.

Testing

Added test_numpy_not_unloaded in tests/internal/test_auto.py (guarded so it is a no-op when numpy is not installed) that asserts numpy stays in sys.modules after ddtrace.auto. I could not run the full native build locally, so I verified this by reading the cleanup logic and matching the existing google.protobuf precedent, and I am relying on CI to exercise the test.

Risks

Low. The change only keeps numpy and its submodules from being unloaded during startup cleanup, mirroring how google.protobuf is already handled.

Additional Notes

Thanks to the reporter for the precise diagnosis and for pointing at the google.protobuf fix.

[arpitjain099]

Thanks for the review, @KowalskiThomas. I've applied your suggested release-note wording so it no longer mentions the internal module-cleanup handling, and made the test noqa comments specific to F401 (unused import), which is the rule that fires on those subprocess imports. Both changes are in the latest commit.

[emmettbutler]

I think the risk is sufficiently low. Thanks for the contribution.

[KowalskiThomas]

Looks good to me!

Sorry I had missed your later push.

[KowalskiThomas]

Just pushed the latest version of your branch to run on the internal CI. ⌛

[KowalskiThomas]

@arpitjain099 for some reason, the internal CI isn't passing on your commit. Can you please merge main into your branch and let me know, so I can merge main into my replica of your branch in our dd-trace-py repository and re-run CI?

[arpitjain099]

Done, merged main into the branch (07f917b). Should be ready for you to sync into your replica and re-run CI. Thanks!

[P403n1x87]

@arpitjain099 may I ask a few more details about your current setup, please? In particular I'm interested to understand why numpy would be loaded before ddtrace.auto (I can only guess that gevent is one of the dependencies installed in your environment, since module cleanup is being triggered). In our documentation we ask that ddtrace.auto is imported as the very first thing to avoid this kind of issues. Otherwise we would end up with a never-ending list of modules that we would have to exclude from cloning, and this doesn't quite scale.

[arpitjain099]

Thanks @P403n1x87, fair question.

Importing ddtrace.auto first is the clean answer, and I agree a growing exclude list wouldn't scale, I don't want that either. My thinking was that numpy is a special case rather than the first of many: it carries C-extension state that doesn't survive being torn down and reimported (numpy doesn't support being reloaded), so if it gets cleaned up and later reimported the process is left with a half-initialized numpy instead of a clean one. Pure-Python modules make that round trip fine, which is why I scoped this to numpy alone rather than reaching for a general mechanism.

That said, I hear the maintenance concern. If the unconditional keep is the sticking point, I'm happy to narrow it, for example only skipping cleanup for numpy when it was already imported at instrumentation time, or gating it behind whatever check you'd want for native modules generally. Happy to go whichever way fits how you want to maintain this.