|
1 | 1 | # Common Pitfalls |
2 | 2 |
|
3 | | -## Array Bounds & Ghost Cells |
4 | | -- Grid dimensions: `m`, `n`, `p` (cells in x, y, z). 1D: n=p=0. 2D: p=0. |
5 | | -- Interior domain: `0:m`, `0:n`, `0:p` |
6 | | -- Buffer/ghost region: `-buff_size:m+buff_size` (similar for n, p in multi-D) |
7 | | -- `buff_size` is **not** a single formula: it's set per reconstruction scheme (WENO/MUSCL/IGR) in |
8 | | - `s_configure_coordinate_bounds` (`m_helper_basic.fpp`) and floored higher for Lagrange bubbles and IB. |
9 | | - Read that routine for the current value rather than assuming one. |
10 | | -- Domain bounds: `idwint(1:3)` (interior `0:m`), `idwbuff(1:3)` (with ghost cells) |
11 | | -- Cell-center coords: `x_cc(-buff_size:m+buff_size)`, `y_cc(...)`, `z_cc(...)` |
12 | | -- Cell-boundary coords: `x_cb(-1-buff_size:m+buff_size)` |
13 | | -- Riemann solver indexing: left state at `j`, right state at `j+1` |
14 | | -- Off-by-one errors in ghost cell regions are a common source of bugs |
15 | | - |
16 | | -## Field Variable Indexing |
17 | | -- Conserved variables: `q_cons_vf(1:sys_size)`. Primitive: `q_prim_vf(1:sys_size)`. |
18 | | -- All equation indices live in the unified `eqn_idx` struct (`eqn_idx_info` type in `m_derived_types.fpp`). |
19 | | - Index ranges depend on `model_eqns` and enabled features (set in `m_global_parameters.fpp`): |
20 | | - - `eqn_idx%cont` — continuity range (partial densities, one per fluid) |
21 | | - - `eqn_idx%mom` — momentum range |
22 | | - - `eqn_idx%E` — total energy (scalar) |
23 | | - - `eqn_idx%adv` — volume fractions (advection equations) |
24 | | - - `eqn_idx%bub`, `eqn_idx%stress`, `eqn_idx%xi`, `eqn_idx%species`, `eqn_idx%B` — optional |
25 | | - - `eqn_idx%gamma`, `eqn_idx%pi_inf`, `eqn_idx%alf`, `eqn_idx%int_en` — additional scalars/ranges |
26 | | -- Use `eqn_idx%cont%beg`/`eqn_idx%cont%end`, `eqn_idx%mom%beg`/`eqn_idx%mom%end`, etc. (old `contxb`/`contxe`, `momxb`/`momxe` shorthands are gone) |
27 | | -- `sys_size` = total number of conserved variables (computed at startup) |
28 | | -- Changing `model_eqns` or enabling features changes ALL index positions |
29 | | - |
30 | | -## Blast Radius |
31 | | -- `src/common/` is shared by ALL three executables (pre_process, simulation, post_process) |
32 | | -- Any change to common/ requires testing all three targets |
33 | | -- Public subroutine signature changes affect all callers across all targets |
34 | | -- Parameter default changes affect all existing case files |
35 | | - |
36 | | -## Physics Consistency |
37 | | -- Pressure formula MUST match `model_eqns` setting |
38 | | -- Model-specific conservative ↔ primitive conversion paths exist |
39 | | -- Volume fractions must sum to 1.0 |
40 | | -- Boundary condition symmetry requirements must be maintained |
41 | | - |
42 | | -## Compiler-Specific Issues |
43 | | -- See the compiler-backend matrix in `.claude/rules/gpu-and-mpi.md` for which compilers |
44 | | - are CI-gated and which backends each supports. |
45 | | -- Each compiler has different strictness levels and warning behavior |
46 | | -- Fypp macros must expand correctly for both GPU and CPU builds |
47 | | - |
48 | | -## Test System |
49 | | -- Tests are generated **programmatically** in `toolchain/mfc/test/cases.py`, not standalone files |
50 | | -- Each test is a parameter modification on top of `BASE_CFG` defaults |
51 | | -- Test UUID = CRC32 hash of the test's trace string; `./mfc.sh test -l` lists all |
52 | | -- To add a test: modify `cases.py` using `CaseGeneratorStack` push/pop pattern |
53 | | -- Golden files: `tests/<UUID>/golden.txt` — tolerance-based comparison, not exact match |
54 | | -- If your change intentionally modifies output, regenerate golden files: |
55 | | - `./mfc.sh test --generate --only <affected_tests> -j 8` |
56 | | -- Do not regenerate ALL golden files unless you understand every output change |
57 | | - |
58 | | -## PR Checklist |
59 | | -The base loop (format → precheck → build → test → one logical commit) is the |
60 | | -Development Workflow Contract in `CLAUDE.md`. Beyond it, watch for: |
61 | | -- [ ] If adding parameters: definitions.py (_r + _nv) updated; cmake reconfigured; case_validator.py if constraints |
62 | | -- [ ] If modifying `src/common/`: all three targets tested |
63 | | -- [ ] If changing output: golden files regenerated for affected tests (`./mfc.sh test --generate --only <tests>`) |
| 3 | +Traps that live in no other doc, collected because their failure modes are silent — |
| 4 | +wrong answers and wrong indices, not error messages. General development pitfalls are |
| 5 | +covered in `docs/documentation/contributing.md`. |
| 6 | + |
| 7 | +## Indexing and Ghost Cells |
| 8 | + |
| 9 | +- Grid dimensions `m`, `n`, `p` (cells in x, y, z); 1D: n=p=0, 2D: p=0. Interior `0:m`; |
| 10 | + ghost region `-buff_size:m+buff_size`; bounds structs `idwint(1:3)` (interior) and |
| 11 | + `idwbuff(1:3)` (with ghosts); cell boundaries `x_cb(-1-buff_size:m+buff_size)`. |
| 12 | +- `buff_size` is **not** a single formula: it's set per reconstruction scheme in |
| 13 | + `s_configure_coordinate_bounds` (`m_helper_basic.fpp`) and floored higher for Lagrange |
| 14 | + bubbles and IB. Read that routine for the current value rather than assuming one. |
| 15 | +- Riemann solvers: left state at `j`, right state at `j+1`. |
| 16 | +- All equation indices live in the `eqn_idx` struct (`eqn_idx_info` in |
| 17 | + `m_derived_types.fpp`, populated in `m_global_parameters.fpp`): `%cont`, `%mom`, `%E`, |
| 18 | + `%adv`, plus optional ranges (`%bub`, `%stress`, `%species`, `%B`, ...). The old |
| 19 | + `contxb`/`momxb` shorthands are gone. Index positions depend on `model_eqns` and |
| 20 | + enabled features — changing either moves ALL indices; never hard-code one. |
| 21 | + |
| 22 | +## GPU |
| 23 | + |
| 24 | +- WARNING: do NOT wrap `GPU_LOOP` in `GPU_PARALLEL` for spatial loops — `GPU_LOOP` emits |
| 25 | + empty directives on Cray and AMD, causing silent serial execution. Spatial loops always |
| 26 | + use `GPU_PARALLEL_LOOP`/`END_GPU_PARALLEL_LOOP`. Macro API: |
| 27 | + `docs/documentation/gpuParallelization.md`; signatures: |
| 28 | + `src/common/include/parallel_macros.fpp`. Never call the `ACC_*`/`OMP_*` |
| 29 | + implementation layers directly. |
| 30 | +- Only `src/simulation/` is GPU-accelerated. Backends: OpenACC (nvfortran primary, Cray) |
| 31 | + and OpenMP offload (Cray primary, AMD flang, nvfortran). The CPU-only build must always |
| 32 | + work — every `#ifdef` needs a path for all configurations (CPU, ACC, OMP, with/without |
| 33 | + MPI). Gates: `MFC_GPU`, `MFC_OpenACC`, `MFC_OpenMP`, `MFC_MPI`, `MFC_DEBUG`, |
| 34 | + `MFC_SINGLE_PRECISION`/`MFC_MIXED_PRECISION`, `MFC_PRE_PROCESS`/`MFC_SIMULATION`/ |
| 35 | + `MFC_POST_PROCESS`, and compiler macros (`_CRAYFTN`, `__PGI`, ...). |
| 36 | +- `@:ACC_SETUP_VFs(...)`/`@:ACC_SETUP_SFs(...)` GPU pointer setup compiles only under |
| 37 | + Cray. Around MPI: `GPU_UPDATE(host=...)` before send, `GPU_UPDATE(device=...)` after |
| 38 | + receive. |
| 39 | + |
| 40 | +## Parameters |
| 41 | + |
| 42 | +- Adding one: `_r()` definition + `_nv()` `NAMELIST_VARS` registration in |
| 43 | + `toolchain/mfc/params/definitions.py`; `case_validator.py` only if physics-constrained |
| 44 | + (with a `PHYSICS_DOCS` entry). Fortran declarations and namelist bindings are |
| 45 | + auto-generated at CMake configure time — re-run cmake (or `./mfc.sh build`) after editing. |
| 46 | +- Still manual: array variables and derived-type members (declare in |
| 47 | + `src/*/m_global_parameters.fpp` / the relevant type), and case-optimization parameters |
| 48 | + (`CASE_OPT_PARAMS` + the `#:else` block in `src/simulation/m_global_parameters.fpp`). |
| 49 | + Gotcha: under `--case-optimization` those are baked into the binary and dropped from |
| 50 | + the namelist, so changing one needs a *rebuild*, not a case edit. |
| 51 | +- Runtime checks (`@:PROHIBIT`) go where they run: shared → |
| 52 | + `src/common/m_checker_common.fpp`; simulation-only → `src/simulation/m_checker.fpp`; |
| 53 | + pre/post-only → `src/{pre,post}_process/m_checker.fpp` (their `s_check_inputs` are |
| 54 | + currently empty — that IS the right place, not m_checker_common). |
| 55 | +- Analytic ICs are compiled into the binary (syntax error = build failure, not runtime). |
| 56 | + Each IC variable maps to an `eqn_idx%…` expression in `QPVF_IDX_VARS` |
| 57 | + (`toolchain/mfc/case.py`); a new patch-settable conserved variable means updating that |
| 58 | + map AND the Fortran `eqn_idx` builder to agree — a mismatch is a silent wrong index. |
| 59 | + Variables available in expressions: `docs/documentation/case.md`. |
| 60 | + |
| 61 | +## Tests |
| 62 | + |
| 63 | +- Tests are generated programmatically in `toolchain/mfc/test/cases.py` (parameter |
| 64 | + modifications on `BASE_CFG` via the `CaseGeneratorStack` push/pop pattern); test UUID = |
| 65 | + CRC32 of the trace string; `./mfc.sh test -l` lists all. |
| 66 | +- Golden files are tolerance-compared. Regenerate only the affected tests |
| 67 | + (`./mfc.sh test --generate --only <tests>`) — an unexplained golden-file diff is a bug |
| 68 | + report, not noise to be regenerated away. |
0 commit comments