You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Addresses the remaining review items:
* xfer-manifest-build: drop the redundant --extra-lsjson-flags
"--fast-list" (fast_list=True is already the CLI default); note how
to disable it for POSIX sources instead.
* xfer-manifest-rebase: require explicit user confirmation before
rm -rf run/shards, and suggest `mv run/shards run/shards.pre-rebase`
as a safer alternative.
* xfer-rclone-config: replace the standalone "paths are per-system"
invariant with a pointer to CLAUDE.md plus a trimmed per-stage path
list. Reword the description so it no longer implies skills report
to each other.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
`<rclone-conf-path-on-this-cluster>` is the absolute path on the **build cluster's login node**, not the workstation path. If unsure, ask the user — or run `xfer-rclone-config` to resolve/deploy it for this cluster.
94
93
95
94
Notes:
96
95
- If the source is a POSIX path, the `--source` value is the filesystem path (e.g., `/mnt/data/dataset`), not an rclone remote. The destination remains an rclone remote.
97
-
-`--fast-list` is usually a win for S3 sources. Skip it for POSIX sources.
96
+
-`--fast-list` is already the default for S3 sources (see `cli.py:200`). Pass `--no-fast-list` when the source is a POSIX path.
98
97
- Stream output back so the user sees progress. Don't background it.
99
98
100
99
## Step 4 — Retrieve the manifest and note the vantage point
Copy file name to clipboardExpand all lines: .claude/skills/xfer-manifest-rebase/SKILL.md
+3-1Lines changed: 3 additions & 1 deletion
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -42,7 +42,9 @@ Always write to a new file (don't overwrite `manifest.jsonl`). Keeping the origi
42
42
43
43
Sharding is derived from the manifest, so **re-shard after rebasing**. The existing `run/shards/` directory contains pre-rebase paths and must be replaced.
44
44
45
-
Confirm with the user before removing the old shards (`run/shards` is small but removing it is irreversible locally):
45
+
**Before running the command below, ask the user explicitly:** "OK to delete `run/shards/` and re-generate from the rebased manifest?" Offer to move it aside (`mv run/shards run/shards.pre-rebase`) as a safer alternative if they want to keep the old shards for debugging.
Copy file name to clipboardExpand all lines: .claude/skills/xfer-rclone-config/SKILL.md
+7-7Lines changed: 7 additions & 7 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -1,6 +1,6 @@
1
1
---
2
2
name: xfer-rclone-config
3
-
description: Create or extend an rclone.conf for xfer — collect S3 endpoints and credentials for each remote (source, destination, optional staging), write the config with 0600 permissions, optionally test it, and guide the user through deploying it to each Slurm cluster that will run xfer jobs. Use whenever the user needs to set up rclone remotes, is bootstrapping a new transfer, or an existing skill reports a missing/incomplete rclone.conf.
3
+
description: Create or extend an rclone.conf for xfer — collect S3 endpoints and credentials for each remote (source, destination, optional staging), write the config with 0600 permissions, optionally test it, and guide the user through deploying it to each Slurm cluster that will run xfer jobs. Use whenever the user needs to set up rclone remotes, is bootstrapping a new transfer, or a previous step discovered a missing rclone.conf on a cluster.
4
4
---
5
5
6
6
# xfer-rclone-config
@@ -13,15 +13,15 @@ Authors an rclone.conf suitable for xfer. The config is consumed by **containeri
13
13
14
14
The authoritative template is `rclone.conf.example` at the repo root.
15
15
16
-
## Critical invariant — paths are per-system
16
+
## Per-stage rclone.conf paths
17
17
18
-
The `--rclone-config` flag always takes an **absolute path on whichever host the xfer command runs on**. That host differs between stages:
18
+
See CLAUDE.md's "Paths are per-system" cross-cutting invariant. Applied to `--rclone-config`, the host where each stage runs is:
19
19
20
-
-`xfer manifest build` — path on the **build cluster's login node**
21
-
-`xfer slurm render` — the path baked into `sbatch_array.sh`must be valid on the **transfer cluster's compute nodes**
22
-
- Local `uv run xfer analyze/shard/rebase` — path on the **workstation** (only if the user wants to sanity-check remotes locally via host rclone)
-`xfer slurm render` — transfer cluster's compute nodes (the path is embedded into `sbatch_array.sh`and resolved at job time)
22
+
- Local `uv run xfer analyze/shard/rebase` — workstation (only if the user wants to sanity-check remotes via host rclone)
23
23
24
-
A single workstation config does not automatically work on the cluster. **Always ask for and record the path per system.** Do not assume `~/.config/rclone/rclone.conf`on the workstation resolves the same way on the cluster — home directories, shared scratch, and site-standard locations (e.g., `/etc/rclone/rclone.conf`) all differ.
24
+
Collect the absolute path per system — don't assume `~/.config/rclone/rclone.conf`means the same thing on every host.
0 commit comments