Skip to content

Commit 3e01286

Browse files
Clean up low-priority skill wording
Addresses the remaining review items: * xfer-manifest-build: drop the redundant --extra-lsjson-flags "--fast-list" (fast_list=True is already the CLI default); note how to disable it for POSIX sources instead. * xfer-manifest-rebase: require explicit user confirmation before rm -rf run/shards, and suggest `mv run/shards run/shards.pre-rebase` as a safer alternative. * xfer-rclone-config: replace the standalone "paths are per-system" invariant with a pointer to CLAUDE.md plus a trimmed per-stage path list. Reword the description so it no longer implies skills report to each other. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
1 parent 2baaae6 commit 3e01286

3 files changed

Lines changed: 12 additions & 11 deletions

File tree

.claude/skills/xfer-manifest-build/SKILL.md

Lines changed: 2 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -85,16 +85,15 @@ ssh <user>@<login-node> '
8585
--dest <dest> \
8686
--out run/manifest.jsonl \
8787
--rclone-image rclone/rclone:latest \
88-
--rclone-config <rclone-conf-path-on-this-cluster> \
89-
--extra-lsjson-flags "--fast-list"
88+
--rclone-config <rclone-conf-path-on-this-cluster>
9089
'
9190
```
9291

9392
`<rclone-conf-path-on-this-cluster>` is the absolute path on the **build cluster's login node**, not the workstation path. If unsure, ask the user — or run `xfer-rclone-config` to resolve/deploy it for this cluster.
9493

9594
Notes:
9695
- If the source is a POSIX path, the `--source` value is the filesystem path (e.g., `/mnt/data/dataset`), not an rclone remote. The destination remains an rclone remote.
97-
- `--fast-list` is usually a win for S3 sources. Skip it for POSIX sources.
96+
- `--fast-list` is already the default for S3 sources (see `cli.py:200`). Pass `--no-fast-list` when the source is a POSIX path.
9897
- Stream output back so the user sees progress. Don't background it.
9998

10099
## Step 4 — Retrieve the manifest and note the vantage point

.claude/skills/xfer-manifest-rebase/SKILL.md

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -42,7 +42,9 @@ Always write to a new file (don't overwrite `manifest.jsonl`). Keeping the origi
4242

4343
Sharding is derived from the manifest, so **re-shard after rebasing**. The existing `run/shards/` directory contains pre-rebase paths and must be replaced.
4444

45-
Confirm with the user before removing the old shards (`run/shards` is small but removing it is irreversible locally):
45+
**Before running the command below, ask the user explicitly:** "OK to delete `run/shards/` and re-generate from the rebased manifest?" Offer to move it aside (`mv run/shards run/shards.pre-rebase`) as a safer alternative if they want to keep the old shards for debugging.
46+
47+
Only after explicit confirmation:
4648

4749
```bash
4850
rm -rf run/shards

.claude/skills/xfer-rclone-config/SKILL.md

Lines changed: 7 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
---
22
name: xfer-rclone-config
3-
description: Create or extend an rclone.conf for xfer — collect S3 endpoints and credentials for each remote (source, destination, optional staging), write the config with 0600 permissions, optionally test it, and guide the user through deploying it to each Slurm cluster that will run xfer jobs. Use whenever the user needs to set up rclone remotes, is bootstrapping a new transfer, or an existing skill reports a missing/incomplete rclone.conf.
3+
description: Create or extend an rclone.conf for xfer — collect S3 endpoints and credentials for each remote (source, destination, optional staging), write the config with 0600 permissions, optionally test it, and guide the user through deploying it to each Slurm cluster that will run xfer jobs. Use whenever the user needs to set up rclone remotes, is bootstrapping a new transfer, or a previous step discovered a missing rclone.conf on a cluster.
44
---
55

66
# xfer-rclone-config
@@ -13,15 +13,15 @@ Authors an rclone.conf suitable for xfer. The config is consumed by **containeri
1313

1414
The authoritative template is `rclone.conf.example` at the repo root.
1515

16-
## Critical invariant — paths are per-system
16+
## Per-stage rclone.conf paths
1717

18-
The `--rclone-config` flag always takes an **absolute path on whichever host the xfer command runs on**. That host differs between stages:
18+
See CLAUDE.md's "Paths are per-system" cross-cutting invariant. Applied to `--rclone-config`, the host where each stage runs is:
1919

20-
- `xfer manifest build`path on the **build cluster's login node**
21-
- `xfer slurm render` — the path baked into `sbatch_array.sh` must be valid on the **transfer cluster's compute nodes**
22-
- Local `uv run xfer analyze/shard/rebase`path on the **workstation** (only if the user wants to sanity-check remotes locally via host rclone)
20+
- `xfer manifest build` — build cluster's login node
21+
- `xfer slurm render`transfer cluster's compute nodes (the path is embedded into `sbatch_array.sh` and resolved at job time)
22+
- Local `uv run xfer analyze/shard/rebase` — workstation (only if the user wants to sanity-check remotes via host rclone)
2323

24-
A single workstation config does not automatically work on the cluster. **Always ask for and record the path per system.** Do not assume `~/.config/rclone/rclone.conf` on the workstation resolves the same way on the cluster — home directories, shared scratch, and site-standard locations (e.g., `/etc/rclone/rclone.conf`) all differ.
24+
Collect the absolute path per system — don't assume `~/.config/rclone/rclone.conf` means the same thing on every host.
2525

2626
## Step 1 — Inventory what's needed
2727

0 commit comments

Comments
 (0)