pip install after version 0.24.2 fails on raspian bullseye

[jschneider-bal]

Describe the bug

The AWS Greengrass DeviceDefender component has a dependency on the awsiotsdk which in turn depends on awscrt. It appears that a new version of the awsiotsdk was released on June 14th which depends on the latest awscrt version. The install fails. Below are logs which are the same (minus the Greengrass decorations) if I try to manually install awscrt. This is currently causing all our new Greengrass deployments to fail.

{"thread":"Copier","level":"WARN","eventType":"stderr","message":"running build_ext","contexts":{"scriptName":"services.aws.greengrass.DeviceDefender.lifecycle.install.script","serviceName":"aws.greengrass.DeviceDefender","currentState":"NEW"},"loggerName":"aws.greengrass.DeviceDefender","timestamp":1750783282281,"cause":null}
{"thread":"Copier","level":"WARN","eventType":"stderr","message":"Traceback (most recent call last):","contexts":{"scriptName":"services.aws.greengrass.DeviceDefender.lifecycle.install.script","serviceName":"aws.greengrass.DeviceDefender","currentState":"NEW"},"loggerName":"aws.greengrass.DeviceDefender","timestamp":1750783282281,"cause":null}
{"thread":"Copier","level":"WARN","eventType":"stderr","message":"File "/tmp/tmpgvc8anek_in_process.py", line 280, in ","contexts":{"scriptName":"services.aws.greengrass.DeviceDefender.lifecycle.install.script","serviceName":"aws.greengrass.DeviceDefender","currentState":"NEW"},"loggerName":"aws.greengrass.DeviceDefender","timestamp":1750783282282,"cause":null}
{"thread":"Copier","level":"WARN","eventType":"stderr","message":"main()","contexts":{"scriptName":"services.aws.greengrass.DeviceDefender.lifecycle.install.script","serviceName":"aws.greengrass.DeviceDefender","currentState":"NEW"},"loggerName":"aws.greengrass.DeviceDefender","timestamp":1750783282283,"cause":null}
{"thread":"Copier","level":"WARN","eventType":"stderr","message":"File "/tmp/tmpgvc8anek_in_process.py", line 263, in main","contexts":{"scriptName":"services.aws.greengrass.DeviceDefender.lifecycle.install.script","serviceName":"aws.greengrass.DeviceDefender","currentState":"NEW"},"loggerName":"aws.greengrass.DeviceDefender","timestamp":1750783282283,"cause":null}
{"thread":"Copier","level":"WARN","eventType":"stderr","message":"json_out['return_val'] = hook(**hook_input['kwargs'])","contexts":{"scriptName":"services.aws.greengrass.DeviceDefender.lifecycle.install.script","serviceName":"aws.greengrass.DeviceDefender","currentState":"NEW"},"loggerName":"aws.greengrass.DeviceDefender","timestamp":1750783282284,"cause":null}
{"thread":"Copier","level":"WARN","eventType":"stderr","message":"File "/tmp/tmpgvc8anek_in_process.py", line 204, in build_wheel","contexts":{"scriptName":"services.aws.greengrass.DeviceDefender.lifecycle.install.script","serviceName":"aws.greengrass.DeviceDefender","currentState":"NEW"},"loggerName":"aws.greengrass.DeviceDefender","timestamp":1750783282285,"cause":null}
{"thread":"Copier","level":"WARN","eventType":"stderr","message":"return _build_backend().build_wheel(wheel_directory, config_settings,","contexts":{"scriptName":"services.aws.greengrass.DeviceDefender.lifecycle.install.script","serviceName":"aws.greengrass.DeviceDefender","currentState":"NEW"},"loggerName":"aws.greengrass.DeviceDefender","timestamp":1750783282285,"cause":null}
{"thread":"Copier","level":"WARN","eventType":"stderr","message":"File "/tmp/pip-build-env-9t924nck/overlay/lib/python3.9/site-packages/setuptools/build_meta.py", line 435, in build_wheel","contexts":{"scriptName":"services.aws.greengrass.DeviceDefender.lifecycle.install.script","serviceName":"aws.greengrass.DeviceDefender","currentState":"NEW"},"loggerName":"aws.greengrass.DeviceDefender","timestamp":1750783282286,"cause":null}
{"thread":"Copier","level":"WARN","eventType":"stderr","message":"return _build(['bdist_wheel', '--dist-info-dir', str(metadata_directory)])","contexts":{"scriptName":"services.aws.greengrass.DeviceDefender.lifecycle.install.script","serviceName":"aws.greengrass.DeviceDefender","currentState":"NEW"},"loggerName":"aws.greengrass.DeviceDefender","timestamp":1750783282287,"cause":null}
{"thread":"Copier","level":"WARN","eventType":"stderr","message":"File "/tmp/pip-build-env-9t924nck/overlay/lib/python3.9/site-packages/setuptools/build_meta.py", line 423, in _build","contexts":{"scriptName":"services.aws.greengrass.DeviceDefender.lifecycle.install.script","serviceName":"aws.greengrass.DeviceDefender","currentState":"NEW"},"loggerName":"aws.greengrass.DeviceDefender","timestamp":1750783282288,"cause":null}
{"thread":"Copier","level":"WARN","eventType":"stderr","message":"return self._build_with_temp_dir(","contexts":{"scriptName":"services.aws.greengrass.DeviceDefender.lifecycle.install.script","serviceName":"aws.greengrass.DeviceDefender","currentState":"NEW"},"loggerName":"aws.greengrass.DeviceDefender","timestamp":1750783282288,"cause":null}
{"thread":"Copier","level":"WARN","eventType":"stderr","message":"File "/tmp/pip-build-env-9t924nck/overlay/lib/python3.9/site-packages/setuptools/build_meta.py", line 404, in _build_with_temp_dir","contexts":{"scriptName":"services.aws.greengrass.DeviceDefender.lifecycle.install.script","serviceName":"aws.greengrass.DeviceDefender","currentState":"NEW"},"loggerName":"aws.greengrass.DeviceDefender","timestamp":1750783282289,"cause":null}
{"thread":"Copier","level":"WARN","eventType":"stderr","message":"self.run_setup()","contexts":{"scriptName":"services.aws.greengrass.DeviceDefender.lifecycle.install.script","serviceName":"aws.greengrass.DeviceDefender","currentState":"NEW"},"loggerName":"aws.greengrass.DeviceDefender","timestamp":1750783282290,"cause":null}
{"thread":"Copier","level":"WARN","eventType":"stderr","message":"File "/tmp/pip-build-env-9t924nck/overlay/lib/python3.9/site-packages/setuptools/build_meta.py", line 317, in run_setup","contexts":{"scriptName":"services.aws.greengrass.DeviceDefender.lifecycle.install.script","serviceName":"aws.greengrass.DeviceDefender","currentState":"NEW"},"loggerName":"aws.greengrass.DeviceDefender","timestamp":1750783282291,"cause":null}
{"thread":"Copier","level":"WARN","eventType":"stderr","message":"exec(code, locals())","contexts":{"scriptName":"services.aws.greengrass.DeviceDefender.lifecycle.install.script","serviceName":"aws.greengrass.DeviceDefender","currentState":"NEW"},"loggerName":"aws.greengrass.DeviceDefender","timestamp":1750783282291,"cause":null}
{"thread":"Copier","level":"WARN","eventType":"stderr","message":"File "", line 524, in ","contexts":{"scriptName":"services.aws.greengrass.DeviceDefender.lifecycle.install.script","serviceName":"aws.greengrass.DeviceDefender","currentState":"NEW"},"loggerName":"aws.greengrass.DeviceDefender","timestamp":1750783282292,"cause":null}
{"thread":"Copier","level":"WARN","eventType":"stderr","message":"File "/tmp/pip-build-env-9t924nck/overlay/lib/python3.9/site-packages/setuptools/init.py", line 115, in setup","contexts":{"scriptName":"services.aws.greengrass.DeviceDefender.lifecycle.install.script","serviceName":"aws.greengrass.DeviceDefender","currentState":"NEW"},"loggerName":"aws.greengrass.DeviceDefender","timestamp":1750783282293,"cause":null}
{"thread":"Copier","level":"WARN","eventType":"stderr","message":"return distutils.core.setup(**attrs)","contexts":{"scriptName":"services.aws.greengrass.DeviceDefender.lifecycle.install.script","serviceName":"aws.greengrass.DeviceDefender","currentState":"NEW"},"loggerName":"aws.greengrass.DeviceDefender","timestamp":1750783282293,"cause":null}
{"thread":"Copier","level":"WARN","eventType":"stderr","message":"File "/tmp/pip-build-env-9t924nck/overlay/lib/python3.9/site-packages/setuptools/_distutils/core.py", line 186, in setup","contexts":{"scriptName":"services.aws.greengrass.DeviceDefender.lifecycle.install.script","serviceName":"aws.greengrass.DeviceDefender","currentState":"NEW"},"loggerName":"aws.greengrass.DeviceDefender","timestamp":1750783282294,"cause":null}
{"thread":"Copier","level":"WARN","eventType":"stderr","message":"return run_commands(dist)","contexts":{"scriptName":"services.aws.greengrass.DeviceDefender.lifecycle.install.script","serviceName":"aws.greengrass.DeviceDefender","currentState":"NEW"},"loggerName":"aws.greengrass.DeviceDefender","timestamp":1750783282295,"cause":null}
{"thread":"Copier","level":"WARN","eventType":"stderr","message":"File "/tmp/pip-build-env-9t924nck/overlay/lib/python3.9/site-packages/setuptools/_distutils/core.py", line 202, in run_commands","contexts":{"scriptName":"services.aws.greengrass.DeviceDefender.lifecycle.install.script","serviceName":"aws.greengrass.DeviceDefender","currentState":"NEW"},"loggerName":"aws.greengrass.DeviceDefender","timestamp":1750783282296,"cause":null}
{"thread":"Copier","level":"WARN","eventType":"stderr","message":"dist.run_commands()","contexts":{"scriptName":"services.aws.greengrass.DeviceDefender.lifecycle.install.script","serviceName":"aws.greengrass.DeviceDefender","currentState":"NEW"},"loggerName":"aws.greengrass.DeviceDefender","timestamp":1750783282296,"cause":null}
{"thread":"Copier","level":"WARN","eventType":"stderr","message":"File "/tmp/pip-build-env-9t924nck/overlay/lib/python3.9/site-packages/setuptools/_distutils/dist.py", line 1002, in run_commands","contexts":{"scriptName":"services.aws.greengrass.DeviceDefender.lifecycle.install.script","serviceName":"aws.greengrass.DeviceDefender","currentState":"NEW"},"loggerName":"aws.greengrass.DeviceDefender","timestamp":1750783282297,"cause":null}
{"thread":"Copier","level":"WARN","eventType":"stderr","message":"self.run_command(cmd)","contexts":{"scriptName":"services.aws.greengrass.DeviceDefender.lifecycle.install.script","serviceName":"aws.greengrass.DeviceDefender","currentState":"NEW"},"loggerName":"aws.greengrass.DeviceDefender","timestamp":1750783282298,"cause":null}
{"thread":"Copier","level":"WARN","eventType":"stderr","message":"File "/tmp/pip-build-env-9t924nck/overlay/lib/python3.9/site-packages/setuptools/dist.py", line 1102, in run_command","contexts":{"scriptName":"services.aws.greengrass.DeviceDefender.lifecycle.install.script","serviceName":"aws.greengrass.DeviceDefender","currentState":"NEW"},"loggerName":"aws.greengrass.DeviceDefender","timestamp":1750783282298,"cause":null}
{"thread":"Copier","level":"WARN","eventType":"stderr","message":"super().run_command(command)","contexts":{"scriptName":"services.aws.greengrass.DeviceDefender.lifecycle.install.script","serviceName":"aws.greengrass.DeviceDefender","currentState":"NEW"},"loggerName":"aws.greengrass.DeviceDefender","timestamp":1750783282299,"cause":null}
{"thread":"Copier","level":"WARN","eventType":"stderr","message":"File "/tmp/pip-build-env-9t924nck/overlay/lib/python3.9/site-packages/setuptools/_distutils/dist.py", line 1021, in run_command","contexts":{"scriptName":"services.aws.greengrass.DeviceDefender.lifecycle.install.script","serviceName":"aws.greengrass.DeviceDefender","currentState":"NEW"},"loggerName":"aws.greengrass.DeviceDefender","timestamp":1750783282300,"cause":null}
{"thread":"Copier","level":"WARN","eventType":"stderr","message":"cmd_obj.run()","contexts":{"scriptName":"services.aws.greengrass.DeviceDefender.lifecycle.install.script","serviceName":"aws.greengrass.DeviceDefender","currentState":"NEW"},"loggerName":"aws.greengrass.DeviceDefender","timestamp":1750783282301,"cause":null}
{"thread":"Copier","level":"WARN","eventType":"stderr","message":"File "/tmp/pip-build-env-9t924nck/overlay/lib/python3.9/site-packages/setuptools/command/bdist_wheel.py", line 370, in run","contexts":{"scriptName":"services.aws.greengrass.DeviceDefender.lifecycle.install.script","serviceName":"aws.greengrass.DeviceDefender","currentState":"NEW"},"loggerName":"aws.greengrass.DeviceDefender","timestamp":1750783282301,"cause":null}
{"thread":"Copier","level":"WARN","eventType":"stderr","message":"self.run_command("build")","contexts":{"scriptName":"services.aws.greengrass.DeviceDefender.lifecycle.install.script","serviceName":"aws.greengrass.DeviceDefender","currentState":"NEW"},"loggerName":"aws.greengrass.DeviceDefender","timestamp":1750783282302,"cause":null}
{"thread":"Copier","level":"WARN","eventType":"stderr","message":"File "/tmp/pip-build-env-9t924nck/overlay/lib/python3.9/site-packages/setuptools/_distutils/cmd.py", line 357, in run_command","contexts":{"scriptName":"services.aws.greengrass.DeviceDefender.lifecycle.install.script","serviceName":"aws.greengrass.DeviceDefender","currentState":"NEW"},"loggerName":"aws.greengrass.DeviceDefender","timestamp":1750783282303,"cause":null}
{"thread":"Copier","level":"WARN","eventType":"stderr","message":"self.distribution.run_command(command)","contexts":{"scriptName":"services.aws.greengrass.DeviceDefender.lifecycle.install.script","serviceName":"aws.greengrass.DeviceDefender","currentState":"NEW"},"loggerName":"aws.greengrass.DeviceDefender","timestamp":1750783282303,"cause":null}
{"thread":"Copier","level":"WARN","eventType":"stderr","message":"File "/tmp/pip-build-env-9t924nck/overlay/lib/python3.9/site-packages/setuptools/dist.py", line 1102, in run_command","contexts":{"scriptName":"services.aws.greengrass.DeviceDefender.lifecycle.install.script","serviceName":"aws.greengrass.DeviceDefender","currentState":"NEW"},"loggerName":"aws.greengrass.DeviceDefender","timestamp":1750783282304,"cause":null}
{"thread":"Copier","level":"WARN","eventType":"stderr","message":"super().run_command(command)","contexts":{"scriptName":"services.aws.greengrass.DeviceDefender.lifecycle.install.script","serviceName":"aws.greengrass.DeviceDefender","currentState":"NEW"},"loggerName":"aws.greengrass.DeviceDefender","timestamp":1750783282305,"cause":null}
{"thread":"Copier","level":"WARN","eventType":"stderr","message":"File "/tmp/pip-build-env-9t924nck/overlay/lib/python3.9/site-packages/setuptools/_distutils/dist.py", line 1021, in run_command","contexts":{"scriptName":"services.aws.greengrass.DeviceDefender.lifecycle.install.script","serviceName":"aws.greengrass.DeviceDefender","currentState":"NEW"},"loggerName":"aws.greengrass.DeviceDefender","timestamp":1750783282305,"cause":null}
{"thread":"Copier","level":"WARN","eventType":"stderr","message":"cmd_obj.run()","contexts":{"scriptName":"services.aws.greengrass.DeviceDefender.lifecycle.install.script","serviceName":"aws.greengrass.DeviceDefender","currentState":"NEW"},"loggerName":"aws.greengrass.DeviceDefender","timestamp":1750783282306,"cause":null}
{"thread":"Copier","level":"WARN","eventType":"stderr","message":"File "/tmp/pip-build-env-9t924nck/overlay/lib/python3.9/site-packages/setuptools/_distutils/command/build.py", line 135, in run","contexts":{"scriptName":"services.aws.greengrass.DeviceDefender.lifecycle.install.script","serviceName":"aws.greengrass.DeviceDefender","currentState":"NEW"},"loggerName":"aws.greengrass.DeviceDefender","timestamp":1750783282307,"cause":null}
{"thread":"Copier","level":"WARN","eventType":"stderr","message":"self.run_command(cmd_name)","contexts":{"scriptName":"services.aws.greengrass.DeviceDefender.lifecycle.install.script","serviceName":"aws.greengrass.DeviceDefender","currentState":"NEW"},"loggerName":"aws.greengrass.DeviceDefender","timestamp":1750783282308,"cause":null}
{"thread":"Copier","level":"WARN","eventType":"stderr","message":"File "/tmp/pip-build-env-9t924nck/overlay/lib/python3.9/site-packages/setuptools/_distutils/cmd.py", line 357, in run_command","contexts":{"scriptName":"services.aws.greengrass.DeviceDefender.lifecycle.install.script","serviceName":"aws.greengrass.DeviceDefender","currentState":"NEW"},"loggerName":"aws.greengrass.DeviceDefender","timestamp":1750783282308,"cause":null}
{"thread":"Copier","level":"WARN","eventType":"stderr","message":"self.distribution.run_command(command)","contexts":{"scriptName":"services.aws.greengrass.DeviceDefender.lifecycle.install.script","serviceName":"aws.greengrass.DeviceDefender","currentState":"NEW"},"loggerName":"aws.greengrass.DeviceDefender","timestamp":1750783282309,"cause":null}
{"thread":"Copier","level":"WARN","eventType":"stderr","message":"File "/tmp/pip-build-env-9t924nck/overlay/lib/python3.9/site-packages/setuptools/dist.py", line 1102, in run_command","contexts":{"scriptName":"services.aws.greengrass.DeviceDefender.lifecycle.install.script","serviceName":"aws.greengrass.DeviceDefender","currentState":"NEW"},"loggerName":"aws.greengrass.DeviceDefender","timestamp":1750783282310,"cause":null}
{"thread":"Copier","level":"WARN","eventType":"stderr","message":"super().run_command(command)","contexts":{"scriptName":"services.aws.greengrass.DeviceDefender.lifecycle.install.script","serviceName":"aws.greengrass.DeviceDefender","currentState":"NEW"},"loggerName":"aws.greengrass.DeviceDefender","timestamp":1750783282310,"cause":null}
{"thread":"Copier","level":"WARN","eventType":"stderr","message":"File "/tmp/pip-build-env-9t924nck/overlay/lib/python3.9/site-packages/setuptools/_distutils/dist.py", line 1021, in run_command","contexts":{"scriptName":"services.aws.greengrass.DeviceDefender.lifecycle.install.script","serviceName":"aws.greengrass.DeviceDefender","currentState":"NEW"},"loggerName":"aws.greengrass.DeviceDefender","timestamp":1750783282311,"cause":null}
{"thread":"Copier","level":"WARN","eventType":"stderr","message":"cmd_obj.run()","contexts":{"scriptName":"services.aws.greengrass.DeviceDefender.lifecycle.install.script","serviceName":"aws.greengrass.DeviceDefender","currentState":"NEW"},"loggerName":"aws.greengrass.DeviceDefender","timestamp":1750783282311,"cause":null}
{"thread":"Copier","level":"WARN","eventType":"stderr","message":"File "", line 382, in run","contexts":{"scriptName":"services.aws.greengrass.DeviceDefender.lifecycle.install.script","serviceName":"aws.greengrass.DeviceDefender","currentState":"NEW"},"loggerName":"aws.greengrass.DeviceDefender","timestamp":1750783282312,"cause":null}
{"thread":"Copier","level":"WARN","eventType":"stderr","message":"File "", line 361, in _build_dependencies","contexts":{"scriptName":"services.aws.greengrass.DeviceDefender.lifecycle.install.script","serviceName":"aws.greengrass.DeviceDefender","currentState":"NEW"},"loggerName":"aws.greengrass.DeviceDefender","timestamp":1750783282313,"cause":null}
{"thread":"Copier","level":"WARN","eventType":"stderr","message":"File "", line 239, in _build_dependencies_impl","contexts":{"scriptName":"services.aws.greengrass.DeviceDefender.lifecycle.install.script","serviceName":"aws.greengrass.DeviceDefender","currentState":"NEW"},"loggerName":"aws.greengrass.DeviceDefender","timestamp":1750783282314,"cause":null}
{"thread":"Copier","level":"WARN","eventType":"stderr","message":"File "", line 158, in get_cmake_path","contexts":{"scriptName":"services.aws.greengrass.DeviceDefender.lifecycle.install.script","serviceName":"aws.greengrass.DeviceDefender","currentState":"NEW"},"loggerName":"aws.greengrass.DeviceDefender","timestamp":1750783282314,"cause":null}
{"thread":"Copier","level":"WARN","eventType":"stderr","message":"Exception: CMake must be installed to build from source.","contexts":{"scriptName":"services.aws.greengrass.DeviceDefender.lifecycle.install.script","serviceName":"aws.greengrass.DeviceDefender","currentState":"NEW"},"loggerName":"aws.greengrass.DeviceDefender","timestamp":1750783282315,"cause":null}
{"thread":"Copier","level":"WARN","eventType":"stderr","message":"----------------------------------------","contexts":{"scriptName":"services.aws.greengrass.DeviceDefender.lifecycle.install.script","serviceName":"aws.greengrass.DeviceDefender","currentState":"NEW"},"loggerName":"aws.greengrass.DeviceDefender","timestamp":1750783282316,"cause":null}
{"thread":"Copier","level":"WARN","eventType":"stderr","message":"ERROR: Failed building wheel for awscrt","contexts":{"scriptName":"services.aws.greengrass.DeviceDefender.lifecycle.install.script","serviceName":"aws.greengrass.DeviceDefender","currentState":"NEW"},"loggerName":"aws.greengrass.DeviceDefender","timestamp":1750783282316,"cause":null}
{"thread":"Copier","level":"WARN","eventType":"stderr","message":"ERROR: Could not build wheels for awscrt which use PEP 517 and cannot be installed directly","contexts":{"scriptName":"services.aws.greengrass.DeviceDefender.lifecycle.install.script","serviceName":"aws.greengrass.DeviceDefender","currentState":"NEW"},"loggerName":"aws.greengrass.DeviceDefender","timestamp":1750783282317,"cause":null}
{"thread":"Copier","level":"

Regression Issue

• Select this option if this issue appears to be a regression.

Expected Behavior

The install should succeed.

Current Behavior

installation fails

Reproduction Steps

run pip install awscrt on a raspberry pi with python version 3.9.2 installed.

Possible Solution

No response

Additional Information/Context

No response

aws-crt-python version used

Happens on version 0.25.5 and above

Python version used

3.9.2

Operating System and version

Raspbian GNU/Linux 11 (bullseye)

[xiazhvera]

I dont think we provide a rapsberry pi distribution. You would need build the library from source.
According to the logs above, it seems you are missing CMake.

{"thread":"Copier","level":"WARN","eventType":"stderr","message":"Exception: CMake must be installed to build from source.","contexts":

You can find the prerequisites and install instructions here: https://github.com/aws/aws-iot-device-sdk-python-v2/blob/main/documents/PREREQUISITES.md

[TingDaoK]

piwheels/packages#518

The bullseye binary is distributed by piwheels, while they have been under maintenance, and not building any new packages for awscrt since v0.25.5

You'd need to build from source for now until piwheels gets back and see the requirement as @xiazhvera posted above.
You can track this issue, here

[jschneider-bal]

Thank you for your quick response.