Skip to content

Commit e952bcd

Browse files
committed
update
Signed-off-by: Zhiyu Cheng <zhiyuc@nvidia.com>
1 parent 2e84f3b commit e952bcd

File tree

1 file changed

+6
-2
lines changed

1 file changed

+6
-2
lines changed

.claude/skills/evaluation/references/nel-ci-guide.md

Lines changed: 6 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -126,12 +126,16 @@ execution:
126126

127127
### Gated datasets
128128

129-
Tasks that download gated HuggingFace datasets (e.g., GPQA, HLE) need `HF_TOKEN` passed to the evaluation container. Set it at the evaluation level or per-task:
129+
Tasks that download gated HuggingFace datasets (e.g., GPQA, HLE) need `HF_TOKEN` passed to the evaluation container.
130+
131+
**NEL CI (JET)**: Handled automatically — the `COMPEVAL_HF_TOKEN` JET secret is pre-configured by the eval platform team. No user action needed; you don't even need personal access to the gated dataset.
132+
133+
**NEL SLURM executor**: You must provide your own HF token, AND your HuggingFace account must have been granted access to the gated dataset (e.g., request access at https://huggingface.co/datasets/Idavidrein/gpqa for GPQA).
130134

131135
```yaml
132136
evaluation:
133137
env_vars:
134-
HF_TOKEN: host:HF_TOKEN # SLURM executor
138+
HF_TOKEN: host:HF_TOKEN # SLURM executor — reads from your local env
135139
tasks:
136140
- name: simple_evals.gpqa_diamond
137141
env_vars:

0 commit comments

Comments
 (0)