Skip to content

Commit a8097d2

Browse files
author
Donglai Wei
committed
Migrate shared config profiles and fix regressions
1 parent a2fefcc commit a8097d2

43 files changed

Lines changed: 235 additions & 127 deletions

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

CLAUDE.md

Lines changed: 24 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -281,14 +281,17 @@ scripts/ # Entry points and utilities
281281
├── compare_config.py # Config comparison tool
282282
└── eval_curvilinear.py # Curvilinear structure evaluation
283283
284-
tutorials/ # Example configurations
285-
├── bases/ # Reusable profile bases (11 YAML files)
286-
├── all_profiles.yaml # Master profile index
284+
configs/ # Canonical shared YAML registries
285+
├── all_profiles.yaml # Master registry index used by tutorials
286+
├── profiles/ # Section-level profile registries
287287
│ ├── arch_profiles.yaml # Architecture presets
288-
│ ├── augmentation_profiles.yaml # Augmentation presets
289-
│ ├── loss_profiles.yaml # Loss function presets
290-
│ ├── optimizer_profiles.yaml # Optimizer presets
291-
│ └── ... # system, dataloader, decoding, etc.
288+
│ ├── loss_profiles.yaml # Loss presets
289+
│ ├── label_profiles.yaml # Label-transform presets
290+
│ └── ... # system, dataloader, augmentation, pipeline, tune
291+
└── templates/ # Explicit list-item templates
292+
└── decoding_templates.yaml # `inference.decoding` templates (`template: ...`)
293+
294+
tutorials/ # Example configurations
292295
├── misc/ # Miscellaneous experiments
293296
└── *.yaml # Dataset-specific configs (16 files)
294297
# mito_lucchi++, mito_mitoEM_*, mito_mitolab, mito_betaseg,
@@ -310,6 +313,20 @@ docker/ # Docker containerization
310313
### Hydra Configuration (Primary)
311314
The project uses **Hydra/OmegaConf** with dataclass-based configs for type safety and composability.
312315

316+
Canonical YAML layout:
317+
318+
- `configs/profiles/*.yaml`: section-level registries selected by `*.profile`
319+
- `configs/templates/*.yaml`: explicit list-item templates, currently for `inference.decoding`
320+
- `tutorials/*.yaml`: runnable experiments that `_base_` the shared registries
321+
322+
Canonical merge semantics:
323+
324+
- Profile payloads are merged into the target section as the base config.
325+
- Explicit keys override profile keys.
326+
- Explicit lists replace profile lists; list overrides are not additive.
327+
- Canonical decoding expansion is explicit `template:` inside `inference.decoding`.
328+
- Do not introduce `decoding_profile` or `- profile: decoding_*` usages.
329+
313330
**Config File Example** (`tutorials/lucchi.yaml`):
314331
```yaml
315332
system:

README.md

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -182,6 +182,19 @@ just test lucchi++ outputs/lucchi++/$EXPERIMENT_DATE/checkpoints/best.ckpt
182182
- 🔧 **[Troubleshooting](TROUBLESHOOTING.md)** - Common issues and solutions
183183
- 👨‍💻 **[Developer Guide](.claude/CLAUDE.md)** - Contributing and architecture
184184

185+
## Config Layout
186+
187+
- `tutorials/*.yaml`: runnable experiment configs
188+
- `configs/profiles/*.yaml`: section-level registries selected by `*.profile`
189+
- `configs/templates/*.yaml`: explicit list-item templates, currently used for `inference.decoding`
190+
191+
Merge rule:
192+
193+
- Profile payloads are merged into the target section as the base config.
194+
- Explicit keys in the tutorial/config override profile keys.
195+
- Explicit lists replace profile lists; they are not additive.
196+
- Canonical decoding syntax is explicit list expansion, for example `inference.decoding: [{template: decoding_waterz}]`.
197+
185198
---
186199

187200
## Example: Train a Model

configs/all_profiles.yaml

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,12 @@
1+
_base_:
2+
- profiles/arch_profiles.yaml
3+
- profiles/loss_profiles.yaml
4+
- profiles/label_profiles.yaml
5+
- profiles/system_profiles.yaml
6+
- templates/decoding_templates.yaml
7+
- profiles/activation_profiles.yaml
8+
- profiles/augmentation_profiles.yaml
9+
- profiles/dataloader_profiles.yaml
10+
- profiles/optimizer_profiles.yaml
11+
- profiles/tune_profiles.yaml
12+
- profiles/pipeline_profiles.yaml
File renamed without changes.
Lines changed: 2 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -18,7 +18,7 @@ label_profiles:
1818
kwargs:
1919
offsets: ["0-0-1", "0-1-0", "1-0-0", "0-0-3", "0-3-0", "3-0-0", "0-0-9", "0-9-0", "9-0-0"]
2020
deepem_crop: true
21-
label_affinity_9_sdt:
21+
label_aff9_sdt:
2222
erosion: 1
2323
targets:
2424
- name: affinity
@@ -29,9 +29,8 @@ label_profiles:
2929
kwargs:
3030
alpha: 0.8
3131
bg_value: -1.0
32-
relabel: true
3332

34-
label_affinity_12:
33+
label_aff12:
3534
erosion: 1
3635
targets:
3736
- name: affinity

0 commit comments

Comments
 (0)