Skip to content

Commit 54c3a40

Browse files
dralleydralley
committed
Add documentation (first pass)
Assisted-By: claude-opus-4.6
1 parent 709e899 commit 54c3a40

5 files changed

Lines changed: 270 additions & 8 deletions

File tree

‎README.md‎

Lines changed: 13 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,18 @@
11
# pulp-rust
22

3-
A Pulp plugin to support hosting your own Rust package registry.
3+
A Pulp plugin to support hosting your own Rust/Cargo package registry.
4+
5+
> **Tech Preview**: This project is currently in tech preview. APIs, behaviors, and data models are subject to change, including breaking changes, without prior notice.
6+
7+
## Features
8+
9+
- Use Pulp as a pull-through cache for crates.io or any Cargo sparse registry
10+
- Host a private Cargo registry for internal crates
11+
- Implements the [Cargo sparse registry protocol](https://doc.rust-lang.org/cargo/reference/registry-index.html#sparse-index) for compatibility with standard Cargo tooling
12+
- Download crates on-demand to reduce disk usage
13+
- Every operation creates a restorable snapshot with Versioned Repositories
14+
- Host content either locally or on S3/Azure/GCP
15+
- De-duplication of all saved content
416

517
For more information, please see the [documentation](docs/index.md) or the [Pulp project page](https://pulpproject.org/).
618

‎docs/index.md‎

Lines changed: 15 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -1,12 +1,20 @@
1-
# Welcome to Pulp
1+
# Welcome to Pulp Rust
22

3-
The `` plugin extends pulpcore to support hosting packages.
4-
This plugin is a part of the Pulp Project, and assumes some familiarity with the
5-
[pulpcore documentation](site:pulpcore/).
3+
The `rust` plugin extends [pulpcore](site:pulpcore/) to support hosting Rust/Cargo package registries. This plugin is a part of the [Pulp Project](site:/), and assumes some familiarity with the [pulpcore documentation](site:pulpcore/).
64

7-
If you are just getting started, we recommend:
5+
!!! warning "Tech Preview"
6+
This plugin is currently in **tech preview**. APIs, behaviors, and data models are subject to change, including breaking changes, without prior notice.
87

9-
- [Getting Started with ](site:pulp_rust/docs/tutorials/getting-started.md),
10-
for a starting out with a common use case.
8+
If you are just getting started, we recommend getting to know the [basic workflows](site:pulp_rust/docs/user/guides/pull-through-cache/).
119

10+
See the [REST API documentation](site:pulp_rust/restapi/) for detailed endpoint reference.
1211

12+
## Features
13+
14+
- [Use Pulp as a pull-through cache](site:pulp_rust/docs/user/guides/pull-through-cache/) for crates.io or any Cargo sparse registry
15+
- [Host a private Cargo registry](site:pulp_rust/docs/user/guides/private-registry/) for internal crates
16+
- Implements the [Cargo sparse registry protocol](https://doc.rust-lang.org/cargo/reference/registry-index.html#sparse-index) for compatibility with standard Cargo tooling
17+
- Download crates on-demand to reduce disk usage
18+
- Every operation creates a restorable snapshot with Versioned Repositories
19+
- Host content either locally or on S3/Azure/GCP
20+
- De-duplication of all saved content

‎docs/user/guides/_SUMMARY.md‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
1+
* [Pull-Through Cache](pull-through-cache.md)
2+
* [Host a Private Registry](private-registry.md)

‎docs/user/guides/private-registry.md‎

Lines changed: 131 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,131 @@
1+
# Host a Private Cargo Registry
2+
3+
This guide walks you through setting up Pulp as a private Cargo registry for hosting internal
4+
crates. This is useful for organizations that need to distribute proprietary or internal-only
5+
Rust packages.
6+
7+
!!! note
8+
Package publishing support (`cargo publish`) is not yet available but is planned for an
9+
upcoming release. In the meantime, content can be uploaded through the Pulp REST API.
10+
11+
## Create a Repository
12+
13+
```bash
14+
pulp rust repository create --name my-crates
15+
```
16+
17+
## Create a Distribution
18+
19+
A distribution makes the repository's content available to Cargo over HTTP.
20+
21+
```bash
22+
pulp rust distribution create \
23+
--name my-crates \
24+
--base-path my-crates \
25+
--repository my-crates
26+
```
27+
28+
Your private registry is now served at `http://<pulp-host>/pulp/cargo/my-crates/`.
29+
30+
## Configure Cargo
31+
32+
Add the private registry to your Cargo configuration. Create or edit `~/.cargo/config.toml`:
33+
34+
```toml
35+
[registries.my-crates]
36+
index = "sparse+http://<pulp-host>/pulp/cargo/my-crates/"
37+
```
38+
39+
### Using the Private Registry as a Dependency Source
40+
41+
To depend on crates from your private registry, specify the registry in your `Cargo.toml`:
42+
43+
```toml
44+
[dependencies]
45+
my-internal-lib = { version = "1.0", registry = "my-crates" }
46+
```
47+
48+
### Setting the Default Registry
49+
50+
You can set your private registry as the default for `cargo publish` and other registry commands
51+
so you don't need to pass `--registry` every time:
52+
53+
```toml
54+
[registry]
55+
default = "my-crates"
56+
```
57+
58+
This affects commands like `cargo publish`, `cargo yank`, and `cargo owner`. It does **not**
59+
change where dependencies are resolved from — that is controlled by source replacement (below).
60+
61+
!!! tip
62+
Setting a default registry is recommended for organizations with private crates. Without it,
63+
running `cargo publish` without `--registry` will publish to crates.io by default, which could
64+
accidentally leak proprietary code to the public registry.
65+
66+
### Replacing crates.io Entirely
67+
68+
If you want all crate lookups to go through your private registry (for example, in an air-gapped
69+
environment), you can replace the default source:
70+
71+
```toml
72+
[source.crates-io]
73+
replace-with = "my-crates"
74+
75+
[source.my-crates]
76+
registry = "sparse+http://<pulp-host>/pulp/cargo/my-crates/"
77+
```
78+
79+
This redirects all dependency resolution — including transitive dependencies — through your
80+
private registry. Any crate not present in the registry will fail to resolve.
81+
82+
## Combining with Pull-Through Caching
83+
84+
If you need both private crates and public crates.io dependencies, we recommend keeping them as
85+
**separate registries** rather than mixing them into one. This avoids
86+
[dependency confusion](https://medium.com/@alex.birsan/dependency-confusion-4a5d60fec610) attacks,
87+
where a malicious package on a public registry could impersonate a private dependency.
88+
89+
```bash
90+
# Set up a separate pull-through cache for crates.io
91+
pulp rust remote create --name crates-io --url "sparse+https://index.crates.io/" --policy on_demand
92+
pulp rust repository create --name crates-io-cache --remote crates-io --retain-repo-versions 1
93+
pulp rust distribution create \
94+
--name crates-io-cache \
95+
--base-path crates-io-cache \
96+
--repository crates-io-cache \
97+
--remote crates-io
98+
```
99+
100+
Then configure Cargo to use both registries, with crates.io going through the cache and private
101+
crates resolved from your private registry:
102+
103+
```toml
104+
[registries.my-crates]
105+
index = "sparse+http://<pulp-host>/pulp/cargo/my-crates/"
106+
107+
[source.crates-io]
108+
replace-with = "crates-io-cache"
109+
110+
[source.crates-io-cache]
111+
registry = "sparse+http://<pulp-host>/pulp/cargo/crates-io-cache/"
112+
```
113+
114+
```toml
115+
[dependencies]
116+
serde = "1.0" # resolved from crates-io-cache
117+
my-internal-lib = { version = "1.0", registry = "my-crates" } # resolved from private registry
118+
```
119+
120+
!!! warning
121+
Avoid adding a public remote (such as crates.io) to a private registry's distribution. Mixing
122+
public and private packages in a single registry index creates a risk of dependency confusion
123+
attacks, where an attacker publishes a crate on the public registry with the same name as one
124+
of your private crates.
125+
126+
## Further Reading
127+
128+
- [Cargo registries configuration](https://doc.rust-lang.org/cargo/reference/registries.html) -- configuring alternate registries in Cargo
129+
- [Cargo source replacement](https://doc.rust-lang.org/cargo/reference/source-replacement.html) -- replacing crates.io with an alternate source
130+
- [Cargo config reference](https://doc.rust-lang.org/cargo/reference/config.html) -- full reference for `.cargo/config.toml`
131+
- [Specifying dependencies](https://doc.rust-lang.org/cargo/reference/specifying-dependencies.html#specifying-dependencies-from-other-registries) -- using dependencies from alternate registries

‎docs/user/guides/pull-through-cache.md‎

Lines changed: 109 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,109 @@
1+
# Use Pulp as a Pull-Through Cache
2+
3+
This guide walks you through setting up Pulp as a pull-through cache for an upstream Cargo registry
4+
such as [crates.io](https://crates.io). Cargo will fetch crates through Pulp, which transparently
5+
downloads and stores them on first access. Subsequent requests are served directly from Pulp's
6+
storage backend.
7+
8+
## Create a Remote
9+
10+
A remote tells Pulp where to fetch crates from. The URL should point to the sparse index of
11+
the upstream registry.
12+
13+
```bash
14+
pulp rust remote create --name crates-io --url "sparse+https://index.crates.io/" --policy on_demand
15+
```
16+
17+
The `--policy` flag controls how content is stored:
18+
19+
| Policy | Behavior |
20+
|-------------|---------------------------------------------------------------------------------------------|
21+
| `on_demand` | Downloads and caches crate files on first request. Metadata is saved to the Pulp database. |
22+
| `streamed` | Streams crate files to the client without saving them locally. |
23+
24+
For a pull-through cache, `on_demand` is recommended so that crates are retained for future
25+
requests. `streamed` is primarily useful in cases where you want a proxy in between clients and
26+
the public registry for e.g. access control - which is not so common in the Rust ecosystem.
27+
28+
## Create a Repository
29+
30+
Repositories store and organize content in Pulp. When using pull-through caching, crates are
31+
automatically added to the repository as they are downloaded.
32+
33+
```bash
34+
pulp rust repository create --name crates-io-cache --remote crates-io --retain-repo-versions 1
35+
```
36+
37+
!!! tip
38+
Setting `--retain-repo-versions 1` is recommended for pull-through caches. Each new crate
39+
download creates a new repository version, and without this setting the number of versions
40+
will grow unboundedly.
41+
42+
## Create a Distribution
43+
44+
A distribution makes the repository available to Cargo over HTTP. The `--base-path` determines the
45+
URL path where the registry is served.
46+
47+
```bash
48+
pulp rust distribution create \
49+
--name crates-io-cache \
50+
--base-path crates-io-cache \
51+
--repository crates-io-cache \
52+
--remote crates-io
53+
```
54+
55+
!!! note
56+
The remote must be set on both the repository and the distribution. The distribution's remote
57+
controls the proxy fallback for the sparse index, while the repository's remote is used for
58+
pull-through content storage.
59+
60+
Your registry is now available at `http://<pulp-host>/pulp/cargo/crates-io-cache/`.
61+
62+
## Configure Cargo
63+
64+
To use your Pulp instance as a registry, add it to your Cargo configuration. Create or edit
65+
`~/.cargo/config.toml`:
66+
67+
```toml
68+
[registries.pulp]
69+
index = "sparse+http://<pulp-host>/pulp/cargo/crates-io-cache/"
70+
71+
[source.crates-io]
72+
replace-with = "pulp"
73+
74+
[source.pulp]
75+
registry = "pulp"
76+
```
77+
78+
This tells Cargo to route all crate downloads through Pulp instead of directly to crates.io.
79+
80+
!!! tip
81+
You can also set this per-project by placing a `.cargo/config.toml` file in your project root.
82+
83+
Now any `cargo build`, `cargo add`, or `cargo install` command will fetch crates through Pulp:
84+
85+
```bash
86+
cargo add serde --features derive
87+
cargo build
88+
```
89+
90+
The first time a crate is requested, Pulp fetches it from the upstream registry and caches it
91+
locally. All subsequent requests for the same crate version are served from Pulp's storage backend.
92+
93+
## Verify Cached Content
94+
95+
You can inspect what content has been cached in Pulp:
96+
97+
```bash
98+
# List cached crates
99+
pulp rust content list --limit 10
100+
101+
# Check repository versions
102+
pulp rust repository version list --repository crates-io-cache
103+
```
104+
105+
## Further Reading
106+
107+
- [Cargo registries configuration](https://doc.rust-lang.org/cargo/reference/registries.html) -- configuring alternate registries in Cargo
108+
- [Cargo source replacement](https://doc.rust-lang.org/cargo/reference/source-replacement.html) -- replacing crates.io with an alternate source
109+
- [Cargo config reference](https://doc.rust-lang.org/cargo/reference/config.html) -- full reference for `.cargo/config.toml`

0 commit comments

Comments
 (0)