Publish new licenses to licensedb as soon as they are defined in develop

[DennisClark]

rather than wait for a new release of scancode-toolkit, we should publish new licenses to the licensedb as soon as we define and recognize them

[AyanSinhaMahapatra]

I've created a sample GitHub Action to run updates daily at 16 UTC or 8 AM PT to build and push to licenseDB repository from scancode-toolkit develop.

It works as follows in the GitHub action:

1. clones scancode-toolkit develop branch
2. creates a virtualenv and pip installs scancode-toolkit develop from the clone
3. in a script gets the latest commit from scancode-toolkit develop branch
4. using a python script and a JSON file with the last scancode-toolkit commit stored, checks if the develop branch has a new commit
5. if there is no new commit, the python script raises an Exception and fails (also fails if pip install scancode-toolkit fails), failing the script and the GitHub action, and does nothing
6. if there is new commits, we store the commit hash for the last commit at the JSON file
7. we build docs and dump the licensedb data using the new scancode-license-data console script
8. we delete the toolkit clone and commit only the doc changes i.e. the licensedb updates
9. we push this to scancode-licensedb main branch from the github action and the licensedb website is also refreshed soon after.

Example GitHub action on no new commits: failed action
Example GitHub action on new commits: successful action
Example automated commit pushed to develop: AyanSinhaMahapatra/scancode-licensedb@55e57e4e

Do you think this works?
Please suggest updates/improvements if any.

[AyanSinhaMahapatra]

@DennisClark this is completed!

See the following github action runs:

• a successful run when there are new commits in scancode, here we have updates commited and pushed automatically. See the commit, here nothing changes as there were no license updates, just the last seen commit hash was updated.
• a failed run when there aren't new commits in scancode. Nothing changes here.

Btw, @pombredanne added a workflow trigger so this can be triggered manually, in addition to the daily check.

Follow up issue to improve the commit message: aboutcode-org/scancode-licensedb#27

[AyanSinhaMahapatra]

This is completed, is running successfully for now and we have further improvements being tracked in seperate issues:

• Minimise churn and commits scancode-licensedb#31
• We need to improve the update commit message scancode-licensedb#27
  Closing this!