Enable the josh-sync cron job

[makai410]

No description provided.

[makai410]

Blocked on #104
@rustbot blocked

[rustbot]

Error: The feature shortcut is not enabled in this repository.
To enable it add its section in the triagebot.toml in the root of the repository.

Please file an issue on GitHub at triagebot if there's a problem with this bot, or reach out on #triagebot on Zulip.

[celinval]

Can you just add a comment to the top of the file saying what it is doing.

[makai410]

I'm running into some issues while testing the rustc push and I'm investigating, but rustc pull should be fine.

[makai410]

I think I should do an update here.

TL;DR: We're able to set up the josh sync, but should hold off on it to re-think our strategy based on the recent discussions on the MCP.

---

Okay so I finally found out what was blocking us from doing rustc push, and I fixed it in rust-lang/josh-sync#38 which just got merged. So now we should be able to sync both ways, but before that we have to do a force-push to this repo since the josh filter we're going to use is different from #104. We should be aware that once we determine the josh filter we can no longer change that unless we do a force-push again, so I'm going to propose the overall solution and want some discussions here.

First off, we're facing these problems:

1. The rust-lang/rust rustc_public is still in-tree, so we can't share the same Cargo.toml, and other stuff like tests/ and build.rs shouldn't be synced as well, that would cause some format errors and build-time errors since we have some MSRV checks in the build.rs.

2. We need to prepare a branch to force-push to the project-smir repo, which should enable josh to do pulling and pushing.

3. The previous round-trip check in the rustc-josh-sync wasn't considering the scenario of mirroring only one single directory of the subrepo, which was what blocked us from doing rustc push, but yeah as mentioned above that has been fixed in josh-sync#38 ^^.

To resolve the Problem 1, I'm going to propose the josh configuration below:

org = "rust-lang"
repo = "project-stable-mir"
filter = ":/compiler/rustc_public/src:prefix=rustc_public/src"
subtree-filter = ":/rustc_public/src:prefix=rustc_public/src"

I.e., we could mirror only one single rustc_public/src/, any other things outside src/ should be ignored. The only file outside src/ that was supposed to be synced as well is README.md, but I think that's not a big hassle to change it twice when needed.

Of course we could do something like this:

filter = ":/compiler/rustc_public:exclude[:[::build.rs,::tests/,::Cargo.toml]]:prefix=rustc_public"
subtree-filter = ":/rustc_public:exclude[:[::build.rs,::tests/,::Cargo.toml]]:prefix=rustc_public"

But that seems to be so much more fragile... It would be broken if we later try to add something to rustc_public/ which we don't want to sync.

To resolve the Problem 2, the solution is:

• Take the filtered upstream tree as the branch baseline using rustc-josh-sync pull, then cherry-pick all the existing commits in the project-stable-mir repo other than the commits from the previous josh clone PR. The commit history looks like: this.

This works well on my machine with rustc pull and push.

Nevertheless, we might have to re-think our sync strategy here, based on the recent discussions on the MCP, even though we've been ready to set up the josh sync here. I think it's possible to end up not needing a josh sync any more... I'm trying to come up with a doc that outlines the problems we're facing and some possible solutions.