Rework the catalyst CLI installation mechanism#2970
Conversation
The overhead of running the CLI through the shim in the previous version was about 100x. While only a couple of seconds total (on an empty file / pipeline), when used interactively this is clearly noticeable and off-putting. The new structure avoids importing Catalyst except the necessary pieces to locate the binary. The final result is about 10x slower than when invoked directly, but this is fast enough to not be a burden (~0.3s).
dime10
added
frontend
|
Hello. You may have forgotten to update the changelog!
|
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #2970 +/- ##
=======================================
Coverage 96.97% 96.97%
=======================================
Files 166 166
Lines 19185 19215 +30
Branches 1786 1788 +2
=======================================
+ Hits 18604 18634 +30
Misses 429 429
Partials 152 152 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
| spec = importlib.util.find_spec("catalyst") | ||
| if spec is None or spec.origin is None: | ||
| raise ImportError( | ||
| "Could not locate the 'catalyst' package, please make sure it is installed." |
There was a problem hiding this comment.
Should we provide an install hint here, ex. via "pip install pennylane-catalyst"?
There was a problem hiding this comment.
Hmm, I would consider that an installation instruction, which we usually want to keep in a centralized placed. I'm pretty sure any Python user also knows how to pip install a package 😅 But what we could do is link to the install guide, why not :)
|
|
||
| def get_cli_path() -> str: # pragma: nocover | ||
| """Method to obtain the Catalyst CLI path packaged via the data_files mechanism.""" | ||
| def get_cli_path() -> str: |
2 participants
In order to better support dynamically linked or loaded libraries from the Catalyst executable (the CLI tool), we need to guarantee a predictable location. Currently, the binary is installed via the
data_filesmechanism which is not the most predictable (see currentget_cli_pathimpl), and separates the CLI binary from other package libraries (typically undersite-packages/catalyst/lib).For dynamic linking of shared libs, we'd need the .so to be located on a fixed path relative to the executable, since this path needs to be set at build time (i.e.
RPATH). For dynamically loaded libraries (dlopen), the fixed relative path removes the need for heuristics to compute the real path of a shared lib.The new system leverages the console_scripts entry point mechanism to install a tiny shim in a location automatically available on PATH for most Python installs.
This is common practice among CLI tools distributed as a Python package (e.g. cmake) and comes with a few advantages:
catalystis available as a command for both wheel installations and editable installs (the current mechanism only works for wheel installs)get_cli_pathimplementation (fewer heuristics)[sc-122013]
Note on performance:
The overhead of running the CLI through the shim in the first version was about 100x. While only a couple of seconds total (on an empty file / pipeline), when used interactively this is clearly noticeable and off-putting. The new structure avoids importing Catalyst except the necessary pieces to locate the binary. The final result is about 10x slower than when invoked directly, but this is fast enough to not be a burden (~0.3s).