Use internal crate feed in Azure Pipelines

[heaths]

Allows contributors to pull from crates.io without having to authenticate with the internal feed, but the registry can easily be switched. Azure Pipelines will use this internal feed to pull crates.

Resolves #2397

[Copilot]

Pull request overview

Adds support for switching Cargo to an internal Azure Artifacts registry and introduces a dedicated pipeline to pre-fetch/mirror dependencies into that feed.

Changes:

• Bump dependency versions used by several cargo -Zscript Rust helper scripts in eng/scripts/.
• Add eng/scripts/use-registry.rs to toggle .cargo/config.toml between crates.io and the internal registry.
• Introduce a manually-triggered eng/pipelines/fetch.yml pipeline and a fetch job template to install the credential provider and cargo fetch via the internal registry.

Reviewed changes

Copilot reviewed 8 out of 8 changed files in this pull request and generated 4 comments.

Show a summary per file

File	Description
eng/scripts/verify-keywords.rs	Updates inline script dependencies (serde/serde_json).
eng/scripts/verify-dependencies.rs	Updates inline script dependencies (serde/serde_json/toml).
eng/scripts/use-registry.rs	New cargo-script to toggle [source.crates-io].replace-with for internal registry usage.
eng/scripts/update-pathversions.rs	Updates inline script dependencies (regex/toml_edit) and applies formatting-only edits.
eng/scripts/update-cratenames.rs	Updates inline script dependencies (cargo-util-schemas/toml).
eng/pipelines/templates/jobs/fetch.yml	New job template to configure auth + registry and run cargo fetch.
eng/pipelines/fetch.yml	New manually-triggered pipeline wiring the fetch job into the platform matrix.
.cargo/config.toml	Adds internal registry definition and (commented) crates-io replacement setting.

[heaths]

With the upcoming changes, I ran the following tests:

#	Config before	Mode	Expected	Pass?
1	No ~/.cargo/config.toml	azure	File created with [registries.azure-sdk-for-rust] and [source.crates-io] replace-with	✅
2	Has azure-sdk-for-rust registry + comments, no replace-with	azure	Comment preserved, no duplicate registry, replace-with added	✅
3	Has a different registry + comments, no azure-sdk-for-rust	azure	Both registries kept, comment preserved, replace-with added	✅
4	Has two registries + comments + replace-with	crates.io	replace-with removed; both registries, comments, and [source.crates-io] preserved	✅
5	Has a registry, no [source] section	crates.io	No crash, file unchanged	✅

[heaths]

@benbp can you at least review the matrix code? I compared with several other pipelines but I'm not entirely sure it's correct. Basically, I want to make sure we fetch crates in the same matrix config as CI because some crates or their transitive dependencies can be platform- or even toolchain-specific.

[heaths]

Marking this as a draft while I talk to some other teams. The guidelines appear to be for binaries, which we don't ship. We have OSS-friendly (stored in GitHub, run by Azure Pipelines and GitHub Actions) workflows that already have all our platform build-out logic in place. In order to install msrustup we have to use a separate RustInstaller task in our pipeline, which means either putting public/internal conditions in our existing pipelines or having a completely separate pipeline, thereby duplicating all our platform build-out logic. And for what? We ship source, not binaries. 3P customers still have to pull from crates.io, while 1P customers will use their own artifact feeds which should upstream crates.io. We have Component Governance scans as well as run cargo audit and cargo deny in both PR and CI builds.