python setup.py test is both deprecated and failing on python3.10

[jfhbrook]

I've been kicking the tires on COPR's pypi packaging feature, which uses pyp2rpm to generate spec files. One of the projects I'm trying to build is yq, a thin wrapper around jq. Implicit dependency on the jq package aside, I do get a build error and it's a wild one:

+ /usr/bin/python3 setup.py test
running test
WARNING: Testing via this command is deprecated and will be removed in a future version. Users looking for a generic test entry point independent of test runner are encouraged to use tox.
running egg_info
writing yq.egg-info/PKG-INFO
writing dependency_links to yq.egg-info/dependency_links.txt
writing entry points to yq.egg-info/entry_points.txt
writing requirements to yq.egg-info/requires.txt
writing top-level names to yq.egg-info/top_level.txt
reading manifest file 'yq.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
adding license file 'LICENSE'
writing manifest file 'yq.egg-info/SOURCES.txt'
running build_ext
Traceback (most recent call last):
  File "/builddir/build/BUILD/yq-2.13.0/setup.py", line 5, in <module>
    setup(
  File "/usr/lib/python3.10/site-packages/setuptools/__init__.py", line 153, in setup
    return distutils.core.setup(**attrs)
  File "/usr/lib64/python3.10/distutils/core.py", line 148, in setup
    dist.run_commands()
  File "/usr/lib64/python3.10/distutils/dist.py", line 966, in run_commands
    self.run_command(cmd)
  File "/usr/lib64/python3.10/distutils/dist.py", line 985, in run_command
    cmd_obj.run()
  File "/usr/lib/python3.10/site-packages/setuptools/command/test.py", line 232, in run
    self.run_tests()
  File "/usr/lib/python3.10/site-packages/setuptools/command/test.py", line 250, in run_tests
    test = unittest.main(
  File "/usr/lib64/python3.10/unittest/main.py", line 100, in __init__
    self.parseArgs(argv)
  File "/usr/lib64/python3.10/unittest/main.py", line 147, in parseArgs
    self.createTests()
  File "/usr/lib64/python3.10/unittest/main.py", line 158, in createTests
    self.test = self.testLoader.loadTestsFromNames(self.testNames,
  File "/usr/lib64/python3.10/unittest/loader.py", line 220, in loadTestsFromNames
    suites = [self.loadTestsFromName(name, module) for name in names]
  File "/usr/lib64/python3.10/unittest/loader.py", line 220, in <listcomp>
    suites = [self.loadTestsFromName(name, module) for name in names]
  File "/usr/lib64/python3.10/unittest/loader.py", line 191, in loadTestsFromName
    return self.loadTestsFromModule(obj)
  File "/usr/lib/python3.10/site-packages/setuptools/command/test.py", line 42, in loadTestsFromModule
    for file in resource_listdir(module.__name__, ''):
  File "/usr/lib/python3.10/site-packages/pkg_resources/__init__.py", line 1148, in resource_listdir
    return get_provider(package_or_requirement).resource_listdir(
  File "/usr/lib/python3.10/site-packages/pkg_resources/__init__.py", line 349, in get_provider
    return _find_adapter(_provider_factories, loader)(module)
  File "/usr/lib/python3.10/site-packages/pkg_resources/__init__.py", line 1379, in __init__
    self.module_path = os.path.dirname(getattr(module, '__file__', ''))
  File "/usr/lib64/python3.10/posixpath.py", line 152, in dirname
    p = os.fspath(p)
TypeError: expected str, bytes or os.PathLike object, not NoneType

I pulled on the threads a little, and from what I can gather the combination of python 3.10 and any reasonable version of pkg_resources will trigger this edge case any time python setup.py test is run. I'm still working on reproducing for a package I actually own - if I succeed I'll update the issue.

Of course, setuptools is dead set on ripping this command out, so this is going to be a problem even if there's a fix to setuptools, and it really sounds like their answer is tox.

I feel like there's a lot of unresolved tension between the needs of system packaging folks and the industry devs using virtualenvs inside/out and that makes solving this hard. Still, my best crack at a solution:

• if tox.ini is there, run tox instead
• if pytest is installed, run pytest . instead
• deprecate running python setup.py test as a third fallback

If tox is really the official pypa answer then it'll have good coverage, and pytest is pretty ubiquitous for newer test suites, so I think that would cover a lot of packages that would otherwise run into this issue.

This is just my $0.02 though - just wanted to make sure this was written down somewhere!

[gordonmessmer]

Is this actually related to the deprecation of the "test" command, though?

It is true that the fedora template needs to be updated, and we need to add some code to determine whether the %tox or %pytest macro is appropriate, but neither "tox" nor "pytest" actually do anything when run in the "yq" source dir. In this case, we'd need to run python3 -m unittest test/test.py or coverage run --source=yq ./test/test.py -v, and even then the tests fail when I run them (jq invocation errors).

I'd like to work on updating the fedora template soon to update to current standards, but I don't think the yq module will be usable as a test case.

[jfhbrook]

It sounds like at a minimum the default for yq won't ever work because of the unspecified dependency on jq. I'm happy to solve that one with a custom build.

I got distracted from an ansible-related yak shave but my intent is to build a bunch of python stuff on copr. Based on this feedback, my plan is to keep doing custom builds, take notes on what things I end up changing to get them working, and turn them into a more concrete proposal and/or better examples of things that 'should" work. Any advice on what to look for? If I'm chest deep in this maybe I can help.

[jfhbrook]

Between this and the other issues it also sounds like I should maybe ask the copr team about improved logs? 🤔