Include Functions and Cmdlets

[ThomasNieto]

I'd like to use the PSGalleryData.zip to a primer for my PS command not found cache. The problem is that the data currently doesn't store information about included functions and cmdlets. This data is stored in the AdditionalMetadata property under Functions and Cmdlets if a module doesn't include one or both then the key doesn't exist. If you do include this data currently the value is a string, for it to be easier to consume I'd recommend splitting the string on space to turn it into a collection for your json file.

I'm currently building a local cache in memory which takes time to build querying PSGallery. Using your already generated report would reduce the time greatly. Here is the example of what the cache is used for:

[jdhitsolutions]

I am actually trying to make this repo smaller! ;-) I would suggest you take the relevant lines from run.ps1 and adjust them to meet your needs. You can create a data file on your own and not rely on this repository. Or fork the repo and make adjustments as needed.

[ThomasNieto]

That's fine, I'll go ahead and close the issue.

[ThomasNieto]

@jdhitsolutions have you looked into LFS to reduce the size of the repo?

[jdhitsolutions]

I think I have it set up for the repo. This issue seems to be the commit history and blob storage for the PDFs that are created every time the GitHub action runs. I will freely admit to shortcomings in git and GitHub expertise,

[ThomasNieto]

I'm not that familiar with it either but it looks like you could add *.pdf to lfs and then migrate those to reduce space.

I also created psgallery-cache which stores the artifacts in GitHub actions on an hourly basis. I'm not entirely happy with it at the moment since it looks like you need a token to download the artifacts.

[jdhitsolutions]

I forgot that I removed PDFs, even though I did configure them for LFS.

[jdhitsolutions]

I have moved this from an issue to a discussion topic. There's a chance we might come up with something better between the two of use. In looking at your workflow, where are the artifacts getting uploaded? I assume it is defined actions/upload-artifact@v4

[ThomasNieto]

That's correct, that action uploads the clixml or json and uploads as a zip to the Action. On the main action page for the run it will list the artifacts at the bottom.

https://github.com/ThomasNieto/psgallery-cache/actions/runs/10479993100

[ThomasNieto]

The artifact REST API will return the artifacts for a public repo without authentication but to download the artifact you'll need to be authenticated.

[jdhitsolutions]

I see now. Instead of committing files to the repo they are cached "outside" of the repo. Have you tried using the gh client instead of the API? I think you might be able to run a command like gh run download.

[jdhitsolutions]

I was able to download the artifacts from your repo using gh.exe.

[ThomasNieto]

Yeah, I am able to make that work but the problem is that it requires the user to be authenticated either through gh or by creating a token. My use case would be a user downloads a module. The module then downloads the latest cache from github actions.

[jdhitsolutions]

I understand now. I'm not sure there is a way around the authentication requirement.

[ThomasNieto]

Found a way to save the artifact outside the repository and able to download without authentication. Instead of trying to download the GitHub Action Artifact, we instead need to create a release and attach those artifact as release assets.

Here is an example using PowerShell's latest release and one of the assets.

[ThomasNieto]

Got the release created and action updating the assets. Now there is a single URL to get the latest cache results. The cache files are not in the repository removing the growth issue. The asset download URLs are available without authentication.

# Clixml cache
Invoke-RestMethod https://github.com/ThomasNieto/psgallery-cache/releases/download/cache/psgallery-xml.zip -OutFile psgallery-xml.zip

# JSON cache
Invoke-RestMethod https://github.com/ThomasNieto/psgallery-cache/releases/download/cache/psgallery-json.zip -OutFile psgallery-json.zip

[jdhitsolutions]

Very good.