Skip to content

Commit 969918e

Browse files
wmaynerclaude
andcommitted
Condense the CHTC guide's manual sweep recipe to a pointer
Pattern A's hand-rolled run_cell.py and submit file duplicated what a campaign generates; the section now points to the campaigns guide as the supported workflow and keeps a short note (with CHTC's own guides and the runtime/file-transfer limits) for workloads campaigns don't express. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PEAxNzhDCaTrntX3o1JqMV
1 parent 6ec55d3 commit 969918e

1 file changed

Lines changed: 11 additions & 52 deletions

File tree

docs/howto/chtc.md

Lines changed: 11 additions & 52 deletions
Original file line numberDiff line numberDiff line change
@@ -62,60 +62,19 @@ saved results afterwards.
6262
{doc}`campaigns`. `pyphi.campaign.prepare` materializes a sweep into a
6363
ready-to-submit directory (task files, substrates, submit file, cost-balanced
6464
packing), and `status`/`collect` handle monitoring, resubmission, and
65-
reassembly into the exact local-sweep result. Everything below the campaign
65+
reassembly into the exact local-sweep result; `prepare_ces` does the same
66+
for one system's scoped cause-effect analysis. Everything below the campaign
6667
workflow is generated for you.
6768

68-
For workloads the sweep axes don't express, the manual recipe is: write
69-
results with `pyphi.provenance.save_json` (or `.save()` on result objects)
70-
so each output is self-describing, then collect the files afterwards.
71-
72-
`run_cell.py` — one cell per job, selected by the process number:
73-
74-
```python
75-
import sys
76-
77-
import pyphi
78-
79-
cell = int(sys.argv[1])
80-
81-
# Define your substrates/states/configs however you like; index them by cell.
82-
substrate = pyphi.examples.basic_substrate()
83-
states = list(pyphi.utils.all_states(substrate.size))
84-
state = states[cell % len(states)]
85-
86-
sia = pyphi.System(substrate, state).sia()
87-
sia.save(f"sia_state{cell}.json.gz")
88-
```
89-
90-
`sweep.sub`:
91-
92-
```
93-
universe = container
94-
container_image = pyphi.sif
95-
96-
executable = run_cell.py
97-
arguments = $(Process)
98-
transfer_executable = false
99-
100-
transfer_input_files = run_cell.py
101-
should_transfer_files = YES
102-
when_to_transfer_output = ON_EXIT
103-
104-
request_cpus = 1
105-
request_memory = 4GB
106-
request_disk = 4GB
107-
108-
log = sweep.log
109-
error = sweep.$(Process).err
110-
output = sweep.$(Process).out
111-
112-
queue 8
113-
```
114-
115-
Submit with `condor_submit sweep.sub`. Jobs have a 72-hour default runtime
116-
limit; keep per-job inputs/outputs under CHTC's file-transfer guidance
117-
(~100 MB per file) or arrange staging with CHTC. For dependent stages
118-
(compute → aggregate), see CHTC's DAGMan guides.
69+
For workloads campaigns don't express — heterogeneous per-job logic,
70+
non-PyPhi pipeline stages, DAGMan workflows — write your own submit file
71+
following the
72+
[CHTC HTC guides](https://chtc.cs.wisc.edu/uw-research-computing/htc-roadmap),
73+
run it in the PyPhi container, and write results with
74+
`pyphi.provenance.save_json` (or `.save()` on result objects) so each output
75+
is self-describing. Jobs have a 72-hour default runtime limit; keep per-job
76+
inputs and outputs under CHTC's file-transfer guidance (~100 MB per file) or
77+
arrange staging with CHTC.
11978

12079
## Pattern B — one big analysis on a fat node (fully supported)
12180

0 commit comments

Comments
 (0)