As we've seen twice in the last two months — both of which caused pages to go out to the on-call staff member — large numbers of crate publications can cause the Git index sync jobs to back up, since they each require an exclusive lock on the index repo and each sync performs a fetch, commit, and push.
The fix here may be to batch the pending syncs (so fetch once, commit many, then push once), but we need more data before we can answer that.
As we've seen twice in the last two months — both of which caused pages to go out to the on-call staff member — large numbers of crate publications can cause the Git index sync jobs to back up, since they each require an exclusive lock on the index repo and each sync performs a fetch, commit, and push.
The fix here may be to batch the pending syncs (so fetch once, commit many, then push once), but we need more data before we can answer that.