ci: add per-release Colab demo refresh workflow#746
Conversation
aditya-neuraco
added
the
version:patch
aditya-neuraco
force-pushed
the
feat/colab-demo-refresh-workflow
branch
from
ea0082a to
20de51b
Compare
aditya-neuraco
changed the title
aditya-neuraco
force-pushed
the
feat/colab-demo-refresh-workflow
branch
from
20de51b to
50adbed
Compare
aditya-neuraco
changed the title
|
@sdas-neuraco I have moved the Colab update to #762 to unblock it. |
aditya-neuraco
force-pushed
the
feat/colab-demo-refresh-workflow
branch
from
50adbed to
f617b1a
Compare
aditya-neuraco
force-pushed
the
feat/colab-demo-refresh-workflow
branch
from
f617b1a to
d5307b9
Compare
sdas-neuraco
left a comment
sdas-neuraco
left a comment
There was a problem hiding this comment.
LGTM w/c Also pls speek with Muneeb when he is back/ add it to his release document that this must be run before release, so platform team are also aware of this existance and workflow
| TERMINAL_STATES = {"COMPLETED", "FAILED", "CANCELLED", "ERROR"} | ||
|
|
||
| # Object path of the per-release model within the public bucket. | ||
| MODEL_OBJECT = "colab/bigym_pretrained_model.nc.zip" |
There was a problem hiding this comment.
maybe we should version this, because there is a gap between deployment and when this script is run, where there will be a missmatch. Also would aid in potential rollbacks incase of a bad deployment
There was a problem hiding this comment.
Good idea. It will catch the version mis-match early as well.
| if status != "COMPLETED": | ||
| raise RuntimeError(f"Training job {job_id} ended with status {status}") | ||
| model_path = _download_model(job_id, get_current_org()) | ||
| output.parent.mkdir(parents=True, exist_ok=True) |
There was a problem hiding this comment.
after the model is trained we should do some rollouts to check it is actually a trained and functinal model. to be extra safe since this is imediate client facing.
There was a problem hiding this comment.
Okay, will add a that after the model finishes training.
aditya-neuraco
added
version:none
2 participants
Features
admin/update_google_colab.py). Run once per release, it retrains the demo policy against the latest released neuracore version, downloads the trained model, uploads it to the public Google Cloud Storage bucket, bumps the version pinned in the notebook, and opens a pull request with the change.Bugfixes
Items
Related PRs