scraper: fix build and re-organize module#4170
Merged
joolswills merged 2 commits intoRetroPie:masterfrom Apr 7, 2026
Merged
Conversation
* Fixed the build errors caused by the `kjk/lzmadec` project being removed from Github.
The `vendor` folder contains a locally mirror of the module, but the `go` version we used ignores the vendored dependencies folder.
Bumping the `go` version fixes the build by using the local version of the package.
* Refactored the module to remove the `golang` module.
Since the `go` installation is needed _just_ for building (the result is a statically linked `scraper` binary), we don't need to have an entire installation kept outside the build step. Given the current lack of changes in the upstream project, I think it's better we provide a binary for installation instead of a source install + local `go` install. Thus:
- adopt the `golang` installation function into `scraper` as `_golang_scraper`, installing it in the build folder during a source installation.
- remove the `golang` module and add an update hook to remove its installation folder
- use `GOMODCACHE` and `GOPATH` so that `go` cache folders for packages download and build are not saved under `/root`.
In order to shorten the `go` commands, I used a subshell to be able to `export` the vars needed and keep them local.
Contributor
Author
|
A couple of other notes, not part of the commit msg:
|
Since the API keys have been revoked in Feb 2026 by TheGamesDB.Net site, replace the default key used by `scraper` with a new one. New keys have a limit of 1000 scrapes per month, per user (individual IP address).
Contributor
Author
|
Added a new commit with the API key replacement for TGDB, scraping would still work, though limited to 1000 requests/month. |
Member
|
Looks good! Cheers! |
Contributor
Author
|
@joolswills thanks for merging. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixed the build errors caused by the
kjk/lzmadecproject being removed from Github. Thevendorfolder contains a locally mirror of the module, but thegoversion we used ignores the vendored dependencies folder. Bumping thegoversion fixes the build by using the local version of the package.Refactored the module to remove the
golangmodule. Since thegoinstallation is needed just for building (the result is a statically linkedscraperbinary), we don't need to have an entire installation kept outside the build step. Given the current lack of changes in the upstream project, I think it's better we provide a binary for installation instead of a source install + localgoinstall. Thus:golanginstallation function intoscraperas_golang_scraper, installing it in the build folder during a source installation.golangmodule and add an update hook to remove its installation folderGOMODCACHEandGOPATHso thatgocache folders for packages download and build are not saved under/root. In order to shorten thegocommands, I used a subshell to be able toexportthe vars needed and keep them local.