- catalog is currently build by querying Github repos which contain keywords and have most recent changes
- this information is added to (or removed from, if repos are archived) the
data.json file which serves as the database to store workflow content
- however there are some disadvantages with this:
a) large and frequent git commits without much meaning, just containing changed repo data
b) more difficult to maintain branch protection rules when auto-commits and human code commits are mixed
c) size, indexing and speed limitations when high-jacking a JSON file as a data base
Ideally, we could separate the databases (data.json, skips.json) from the rest of the code, and avoid frequent git-controlled auto-commits. The ideal database is stored outside of the catalog repo, but can still integrate nicely with Github actions. This way, one workflow would regularly update the (external) database, and one would build the catalog files deploy the web page as it is done now.
@jonasfreimuth
data.jsonfile which serves as thedatabaseto store workflow contenta) large and frequent git commits without much meaning, just containing changed repo data
b) more difficult to maintain branch protection rules when auto-commits and human code commits are mixed
c) size, indexing and speed limitations when high-jacking a JSON file as a data base
Ideally, we could separate the databases (data.json, skips.json) from the rest of the code, and avoid frequent git-controlled auto-commits. The ideal database is stored outside of the catalog repo, but can still integrate nicely with Github actions. This way, one workflow would regularly update the (external) database, and one would build the catalog files deploy the web page as it is done now.
@jonasfreimuth