Add support for .pth files

[gdamjan]

A path configuration file is a file whose name has the form name.pth and exists in one of the four directories mentioned above; its contents are additional items (one per line) to be added to sys.path. Non-existing items are never added to sys.path, and no check is made that the item refers to a directory rather than a file. No item is added to sys.path more than once. Blank lines and lines beginning with # are skipped. Lines starting with import (followed by space or tab) are executed.

https://docs.python.org/3/library/site.html

.pth files are often used by python packaging systems (like uv, etc...) to make working on the current project more elegant, by pointing from the .venv/ path to the project path.

[melizalde-ds]

Hi, I'm interested in working on this, but I might need some clarification:

• What's the specific workflow that's currently broken? (e.g., "I did an editable install of my package, then ran componentize-py, and it couldn't find my local source")
• Where in componentize-py should this be implemented? Is it in the module discovery/bundling logic?
• Is there a minimal example I can use to reproduce the problem?

I want to make sure I understand the expected behavior. Thanks!

[dicej]

@melizalde-ds I don't have any experience with .pth files myself, but this seems like the relevant excerpt from https://docs.python.org/3/library/site.html:

A path configuration file is a file whose name has the form name.pth and exists in one of the four directories mentioned above; its contents are additional items (one per line) to be added to sys.path. Non-existing items are never added to sys.path, and no check is made that the item refers to a directory rather than a file. No item is added to sys.path more than once. Blank lines and lines beginning with # are skipped. Lines starting with import (followed by space or tab) are executed.

I'd guess this would go into the find_site_packages function in https://github.com/bytecodealliance/componentize-py/blob/main/src/command.rs. That function is already pretty ad-hoc and symptomatic of my ignorance of how Python packaging works, so apologies in advance if it looks weird :)

@brettcannon might be able to help with the first and third of your questions.

[brettcannon]

@brettcannon might be able to help with the first and third of your questions.

.pth files are handled by Python itself, so I don't understand how there's anything for componentize-py to do here?

[dicej]

.pth files are handled by Python itself, so I don't understand how there's anything for componentize-py to do here?

componentize-py needs to determine which host directories should be made available to the guest, so it needs to know ahead of time what Python will need access to. I'm not sure there's a way to do that without duplicating to some extent what Python does.

[gdamjan]

@melizalde-ds my bad,
I did report the issue on Discord, but should've repeat it here too

What's the specific workflow that's currently broken? (e.g., "I did an editable install of my package, then ran componentize-py, and it couldn't find my local source")

something like that. I'll add the exact steps in a bit, just need to test them again.

[brettcannon]

componentize-py needs to determine which host directories should be made available to the guest,

site-packages for this specific use case.

[posborne]

I have a working version of this in a separate tool which wraps componentize-py that has been working. I added it to that tool in order to support envs where I was using editable dependencies which are often handled with .pth files.

I can look to add this support, possibly along with my rewrite of the virtualenv parsing more generally back into upstream. If we can expose that from the componentize-py crate, I'd be happy to be able to use it directly as well from my tool.