Add tectonic-on-arxiv github action#1360
Conversation
|
Interesting idea :) a bus factor >1 for tectonic-on-arXiv would be nice 😄 I'm not sure how the limits for GitHub Actions work exactly, but compute limits might end up being annoying here. It seems like GitHub comps 2000 compute-minutes per month. For reference: My somewhat aging system takes ~8 CPU-hours per run. If I understand correctly, the compute-minute limit is actually for the 4-core standard runners, so ~16 runs per month. |
|
That limit is specifically for private repos on the free plan (see this section) - as a public repository, tectonic can use more (though we do want to be a little cautious about not going overboard, so we'll see). I haven't gotten to a point where I'd try splitting it, but that's an interesting idea. Could be pretty simple as just a matrix job - then wait for all of them to complete for the final summary. At that point I maybe would want to split the dataset, maybe into bundles of ~1000. I'd have to give up the collective dataset to do that if we wanted to stay free, I'm using Cloudflare which has a free tier with limits that should be fine for us, but it caps at 10 GB storage before I start paying for it (access limits exist but are far beyond what we'd use, 10 million file GETs per month, and if it goes over that I'll just be out a couple cents, which, whatever). Alternatively I can try to slim it down - find the largest files and remove some of them, since I suspect some of the size comes from diagrams and images that, while it is nice to test, may be fine to skip at least some of. |
|
Checking rates - I'll just store more data if need be. The cost per month for like, 50 gigs is totally reasonable. |
|
1 hour 22 minutes for one runner to do it. Split into thousands, that should be reasonable speed. Alternatively, figuring out how to speed up document compilation even a bit should result in good wins :P Either way, just need to get it to upload the report as an artifact, turn on pulling from master, and it should be almost ready. I do want to look into requiring it to merge but only running on request somehow, given the compute requirements. |
This is a WIP idea to port tectonic-on-arxiv to use the actions workflow, allowing us to rely on github's artifacts and compute.
@Mrmaxmeier for thoughts here - as this is your brainchild primarily.
Some elaboration on specific ideas: