Implement CMake build system for turbo-stack

.github/workflows/build-tests.yaml

@@ -59,6 +59,8 @@ jobs:
           which mpicc
           mpicc --version 2>/dev/null || true
 
+      # NOTE: ./build.sh here is the legacy mkmf-based build at the repo root,
+      # not scripts/build_with_spack.sh (the CMake-based build).
       - name: Build MOM6 with TIM
         run: |
           ./build.sh --compiler ${COMPILER_FAMILY} --machine container --infra TIM

CLAUDE.md

@@ -0,0 +1,143 @@
+# CLAUDE.md
+
+This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
+
+## Project Overview
+
+**TURBO Stack** is the central software hub for the TURBO (Towards Understanding the Representation of the Biological Ocean) Earth System Modeling project. It integrates MOM6 (ocean circulation), FMS (infrastructure), MARBL (marine biogeochemistry), and TIM (new AMReX-based infrastructure backend) into a unified build environment.
+
+## Build Commands
+
+Requires `TURBO_STACK_ROOT` (and, for the spack flavor, `SPACK_ROOT`) to be set in your shell profile. See [`scripts/README.md`](scripts/README.md) for the full reference.
+
+### 2-step pipeline
+
+```
+setup environment (toolchain + any from-source deps)  →  build turbo-stack
+```
+
+Different flavors fill in step 1 differently; step 2 is always the same. Per-machine env scripts under `scripts/setup_environment/` own the call list of `build_dep` invocations for that machine.
+
+### Local build (spack flavor, one command)
+
+```bash
+scripts/build_with_spack.sh                              # incremental build
+scripts/build_with_spack.sh --debug                      # full clean rebuild
+scripts/build_with_spack.sh --infra TIM                  # FMS2 default; --infra TIM also builds TIM from source
+scripts/build_with_spack.sh --recreate-spack-env --debug # nuke + recreate the spack env, then clean rebuild
+```
+
+### Explicit two-step (any flavor; faster iteration)
+
+```bash
+# spack flavor
+source scripts/setup_environment/spack_local_environment.sh
+scripts/build_turbo_stack.sh
+# (for --infra TIM, also `source scripts/build_dep.sh` + a `build_dep tim ...` call first)
+
+# module flavor (laptop emulation of Derecho — temp until derecho_cpu_gcc_openmpi.sh is fully verified)
+source scripts/setup_environment/emulate_derecho_modules_locally_with_spack.sh
+scripts/build_turbo_stack.sh
+```
+
+### Script structure
+
+| Script | Role | How invoked |
+|---|---|---|
+| `scripts/build_with_spack.sh` | Orchestrator: runs the full pipeline for the spack flavor | exec'd |
+| `scripts/setup_environment/<flavor>.sh` | Step 1 — toolchain load + per-machine `build_dep` calls | sourced |
+| `scripts/build_dep.sh` | Library — defines `build_dep <name> ... -- [cmake args]` | sourced |
+| `scripts/fetch_source.sh` | Library — defines `fetch_source` for source-only-consumed deps (MOM6, MARBL) | sourced |
+| `scripts/build_turbo_stack.sh` | Step 2 — cmake configure + build + ctest. No spack or infra knowledge. | exec'd |
+
+`build_turbo_stack.sh` options: `--debug`, `--ninja`, `--build_dir DIR`, `--infra FMS2|TIM`, `--parallel N`.
+
+### Where dep builds + installs land
+
+The orchestrators derive deps location from `--build_dir`: `<build_dir>/deps/{build,install}/`. With no `--build_dir`, deps land at `$TURBO_STACK_ROOT/deps/default/`. To override the deps location independently of the turbo-stack build dir, source the env script directly with `--deps-build-root DIR`.
+
+### Parallel build jobs
+
+`--parallel N` on the orchestrators exports `CMAKE_BUILD_PARALLEL_LEVEL=N` once; every downstream `cmake --build` (deps + turbo-stack) reads it natively. You can also set `CMAKE_BUILD_PARALLEL_LEVEL` in your shell profile to skip the CLI flag. When neither is set, cmake's own default applies (Make=1, Ninja=nproc).
+
+### Source-tree overrides
+
+`build_dep` resolves each dep's source via (first match wins): `--source PATH`, `--clone --url ... --ref ...`, `$<NAME>_ROOT` env var, or the corresponding submodule. So overriding any single dep with a local dev tree is:
+
+```bash
+export MOM6_ROOT=$HOME/projects/MOM6
+export FMS_ROOT=$HOME/projects/FMS
+source scripts/setup_environment/emulate_derecho_modules_locally_with_spack.sh
+scripts/build_turbo_stack.sh
+```
+
+### Spack environment
+
+Defined in `spack/spack.yaml`. Default env name: `turbo_stack`. Provides cmake, gmake, ninja, MPI (OpenMPI), NetCDF, pFUnit, AMReX. FMS and TIM are intentionally not in spack: `build_with_spack.sh` always calls `build_dep fms` (and `build_dep tim` when `--infra TIM`) so the build uses the local source tree (`$FMS_ROOT`/`$TIM_ROOT` or the submodule fallback). Turbo-stack tracks features ahead of the released FMS package, so linking against spack's FMS would risk quietly using a stale version.
+
+A second spack env defined in `spack/derecho_modules_emulation_with_spack.yaml` provides *just* the toolchain (cmake, MPI, NetCDF) — used by the temporary emulation driver to exercise the from-source path on a laptop until Derecho access is back.
+
+**AMReX mini-app tests** are built with CMake separately (see `src/amrex_mini_app/CMakeLists.txt`). They use GoogleTest (C++) and require HDF5.
+
+## Architecture
+
+### Component Relationships
+
+```
+build_with_spack.sh                                        ─── orchestrator (spack flavor)
+  └─→ setup_environment/spack_local_environment.sh           step 1: toolchain (sourced)
+  └─→ build_dep.sh + `build_dep tim ...`                     only when --infra TIM (sourced)
+  └─→ build_turbo_stack.sh                                   step 2: configure+build+test (exec'd)
+        ├─→ cmake configure (Unix Makefiles by default; --ninja for Ninja)
+        ├─→ cmake build
+        └─→ ctest
+
+build_on_derecho.sh                                        ─── orchestrator (Derecho module flavor)
+  └─→ setup_environment/derecho_cpu_gcc_openmpi.sh           step 1: modules + build_dep × 4 (sourced)
+  └─→ build_turbo_stack.sh                                   step 2 (exec'd)
+
+CMakeLists.txt (repo root)
+  ├─→ mom6_build/  (MOM6 framework + infra wrapper targets)
+  │     ├─→ MOM6::framework_base, MOM6::infra, MOM6::framework
+  │     ├─→ MOM6::grid, MOM6::io
+  │     └─→ MOM6::CVMix, MOM6::GSW  (pkg/ subdir)
+  └─→ tests/  (pFUnit unit tests, 40 tests, MPI-aware)
+
+src/amrex_mini_app/ (C++ / CMake — separate build)
+  └─→ AMReX tripolar grid development sandbox
+```
+
+### Key Directories
+
+| Path | Purpose |
+|------|---------|
+| `submodules/` | All external dependencies (MOM6, FMS, MARBL, amrex, TIM, pFUnit, CESM_share) |
+| `build-utils/makefile-templates/` | Legacy compiler × machine makefile fragments (mkmf era — being retired) |
+| `build-utils/mkmf/` | Legacy GFDL makefile generator (being retired in favour of CMake) |
+| `tests/` | Fortran unit tests (pFUnit, MPI-aware via `@test(npes=[1,2,4])`) |
+| `src/amrex_mini_app/` | C++ development sandbox for AMReX-based tripolar grid |
+| `examples/` | Reference simulation configurations (double_gyre, benchmark, cesm grids) |
+| `dev-utils/gcovlens/` | Code coverage aggregation tooling |
+
+### Infrastructure Backends
+
+- **FMS2** — traditional Flexible Modeling System; the stable default path
+- **TIM** — TURBO Infrastructure for MOM; new AMReX-backed layer under active development (CMake integration in progress)
+
+### Language Split
+
+- **Fortran** — MOM6, FMS, MARBL, unit tests under `tests/`
+- **C++20** — AMReX mini-app (`src/amrex_mini_app/`), GoogleTest-based tests there
+- **Bash** — build orchestration under `scripts/` (see [`scripts/README.md`](scripts/README.md))
+
+## CI/CD
+
+GitHub Actions workflows (`.github/workflows/`) test against a matrix of compilers (oneapi, gcc14, nvhpc, clang) and MPI libraries (MPICH, OpenMPI) on both `ubuntu-latest` and the custom `gha-runner-turbo` runner.
+
+Containers used: `ncarcisl/cisldev-x86_64-almalinux9-[compiler]-[mpi]`; activated via `/container/config_env.sh`.
+
+Clang-format (Google style, C++20, 120-char limit) is enforced on PRs and auto-applied on pushes to `main`.
+
+## Code Style
+
+C++ files must pass `clang-format` (config in `.clang-format`): Google style base, C++20, 120-char line limit, Allman braces.

CMakeLists.txt

@@ -0,0 +1,73 @@
+cmake_minimum_required(VERSION 3.24)
+project(TurboStack LANGUAGES Fortran C CXX)
+
+list(APPEND CMAKE_MODULE_PATH "${CMAKE_SOURCE_DIR}/cmake")
+
+# MOM6_SOURCE_DIR must be resolved before including MOM6Options (which sets
+# TURBO_INFRA and TURBO_MEMORY_MODE — the canonical definitions for both repos).
+if(NOT DEFINED MOM6_SOURCE_DIR)
+    set(MOM6_SOURCE_DIR "$ENV{MOM6_ROOT}" CACHE PATH "Path to MOM6 source directory")
+endif()
+if(NOT MOM6_SOURCE_DIR)
+    message(FATAL_ERROR
+        "TurboStack requires MOM6_ROOT to be set in the environment "
+        "(or pass -DMOM6_SOURCE_DIR=/path/to/MOM6)")
+endif()
+
+include(TurboOptions)
+include("${MOM6_SOURCE_DIR}/cmake/MOM6Options.cmake")
+include(TurboCompilerFlags)
+
+# Infrastructure backend — FMS2 or TIM.  Both come from CMAKE_PREFIX_PATH
+# (populated either by spack env activate or by build_dep.sh's per-call
+# CMAKE_PREFIX_PATH append).
+# Both paths define TURBO::infra_r8; nothing downstream needs to know which backend.
+#
+# fms-config.cmake / tim-config.cmake both call find_dependency(NetCDF). Most
+# NetCDF installs (incl. the Derecho spack module) ship no NetCDFConfig.cmake —
+# only nc-config — so a FindNetCDF.cmake must be on CMAKE_MODULE_PATH *before*
+# find_package(FMS|TIM). Each backend ships its own copy; pull from the
+# matching submodule so the two never drift through a shared assumption.
+if(TURBO_INFRA STREQUAL "FMS2")
+    list(APPEND CMAKE_MODULE_PATH "${CMAKE_SOURCE_DIR}/submodules/infra/FMS2/cmake")
+    find_package(FMS REQUIRED COMPONENTS R8)
+    add_library(turbo_infra_r8 INTERFACE)
+    add_library(TURBO::infra_r8 ALIAS turbo_infra_r8)
+    target_link_libraries(turbo_infra_r8 INTERFACE FMS::fms_r8)
+elseif(TURBO_INFRA STREQUAL "TIM")
+    # TIM is built+installed by scripts/build_dep.sh into
+    # deps/<tag>/install (or by spack, in principle, though spack does not
+    # currently package TIM). find_package(TIM) resolves via CMAKE_PREFIX_PATH.
+    list(APPEND CMAKE_MODULE_PATH "${CMAKE_SOURCE_DIR}/submodules/infra/TIM/cmake")
+    find_package(TIM REQUIRED COMPONENTS R8)
+    add_library(turbo_infra_r8 INTERFACE)
+    add_library(TURBO::infra_r8 ALIAS turbo_infra_r8)
+    target_link_libraries(turbo_infra_r8 INTERFACE TIM::tim_r8)
+else()
+    message(FATAL_ERROR "Unknown TURBO_INFRA='${TURBO_INFRA}'. Valid values: FMS2, TIM")
+endif()
+
+# MPI and NetCDF are found transitively by FMS/TIM but declared explicitly
+# so our own targets can link against them directly.
+find_package(MPI REQUIRED COMPONENTS Fortran)
+find_package(NetCDF REQUIRED COMPONENTS C Fortran)
+
+# Attach NetCDF::NetCDF_C to the infra wrapper so its absolute libnetcdf.so
+# path lands on every link line that uses TURBO::infra_r8. FMS::fms_r8 only
+# carries NetCDF::NetCDF_Fortran in its INTERFACE_LINK_LIBRARIES, plus bare
+# `-lnetcdf` flags from nf-config --flibs; on NetCDF installs with split
+# lib/lib64 layouts (e.g. NCAR's Derecho module) the bare flag fails to
+# resolve at link time. Explicit attachment makes the dependency truthful
+# regardless of upstream FMS / infra layout choices.
+target_link_libraries(turbo_infra_r8 INTERFACE NetCDF::NetCDF_C)
+
+# MARBL must be configured before MOM6 so that MARBL::marbl is already defined
+# when the MOM6 subdirectory processes its target_link_libraries calls.
+add_subdirectory(marbl_build)
+add_subdirectory("${MOM6_SOURCE_DIR}" mom6_build)
+
+if(TURBO_BUILD_UNIT_TESTS)
+    find_package(PFUNIT REQUIRED)
+    enable_testing()
+    add_subdirectory(tests)
+endif()

cmake/TurboCompilerFlags.cmake

@@ -0,0 +1,63 @@
+# Fortran compiler flags for all TurboStack targets.
+# Requires TurboOptions.cmake to be included first.
+#
+# Flags are applied via add_compile_options() scoped to Fortran only.
+# All targets in this project inherit them from the top-level scope.
+
+if(NOT CMAKE_Fortran_COMPILER_ID)
+    message(FATAL_ERROR "No Fortran compiler detected. Ensure FC is set before invoking CMake.")
+endif()
+
+# --- Compiler-specific flags ---
+if(CMAKE_Fortran_COMPILER_ID MATCHES "GNU")
+    set(_turbo_fortran_flags
+        -fdefault-real-8 -fdefault-double-8
+        -fconvert=big-endian
+        -fcray-pointer -ffree-line-length-none -fno-range-check -fallow-argument-mismatch
+    )
+
+elseif(CMAKE_Fortran_COMPILER_ID MATCHES "IntelLLVM|Intel")
+    set(_turbo_fortran_flags
+        -r8
+        -convert big_endian -assume byterecl
+        -ftz -traceback -fp-model source -no-fma
+        -fpp -Wp,-w
+    )
+
+elseif(CMAKE_Fortran_COMPILER_ID MATCHES "NVHPC|PGI")
+    set(_turbo_fortran_flags
+        -r8
+        -Mnofma -i4 -Mextend -byteswapio -Mflushz -Kieee
+    )
+
+elseif(CMAKE_Fortran_COMPILER_ID MATCHES "Flang|LLVMFlang")
+    set(_turbo_fortran_flags
+        -fdefault-real-8 -fdefault-double-8
+        -fconvert=big-endian
+    )
+
+else()
+    message(FATAL_ERROR
+        "Unrecognized Fortran compiler '${CMAKE_Fortran_COMPILER_ID}'. "
+        "No project-specific flags applied.")
+endif()
+
+# --- Code coverage (GNU only) ---
+if(TURBO_CODECOV)
+    if(NOT CMAKE_Fortran_COMPILER_ID MATCHES "GNU")
+        message(FATAL_ERROR
+            "TURBO_CODECOV requires a GNU Fortran compiler "
+            "(got ${CMAKE_Fortran_COMPILER_ID}).")
+    endif()
+    list(APPEND _turbo_fortran_flags -fprofile-arcs -ftest-coverage)
+    # Coverage requires unoptimized code
+    set(CMAKE_Fortran_FLAGS_RELEASE "-O0" CACHE STRING "Fortran release flags (overridden for coverage)" FORCE)
+endif()
+
+# --- Apply ---
+foreach(_flag IN LISTS _turbo_fortran_flags)
+    add_compile_options("$<$<COMPILE_LANGUAGE:Fortran>:${_flag}>")
+endforeach()
+
+message(STATUS "Fortran compiler: ${CMAKE_Fortran_COMPILER_ID} ${CMAKE_Fortran_COMPILER_VERSION}")
+message(STATUS "Fortran flags: ${_turbo_fortran_flags}")

cmake/TurboOptions.cmake

@@ -0,0 +1,5 @@
+# What to build
+option(TURBO_BUILD_UNIT_TESTS "Build pFUnit unit tests" ON)
+
+# Compiler-specific features (guarded in TurboCompilerFlags.cmake)
+option(TURBO_CODECOV  "Enable code coverage instrumentation (GNU only)" OFF)

cmake/add_mom_test.cmake

@@ -0,0 +1,51 @@
+# add_mom_tests(TEST_FILES file1.pf file2.pf ... [LINK_LIBRARIES lib1 lib2 ...])
+# TEST_FILES first, then optional LINK_LIBRARIES.
+# Convenience wrapper that calls add_mom_test() for each file in TEST_FILES.
+# LINK_LIBRARIES are passed through to each test (see add_mom_test below).
+function(add_mom_tests)
+    cmake_parse_arguments(SUITE "" "" "TEST_FILES;LINK_LIBRARIES" ${ARGN})
+    foreach(TEST_FILE IN LISTS SUITE_TEST_FILES)
+        if(SUITE_LINK_LIBRARIES)
+            add_mom_test("${TEST_FILE}" LINK_LIBRARIES ${SUITE_LINK_LIBRARIES})
+        else()
+            add_mom_test("${TEST_FILE}")
+        endif()
+    endforeach()
+endfunction()
+
+# add_mom_test(file.pf [LINK_LIBRARIES lib1 lib2 ...])
+# Creates a pFUnit CTest from a single .pf file.
+# Always links TURBO::infra_r8 (provides MPI and NetCDF transitively).
+# Pass additional libraries under test via LINK_LIBRARIES.
+function(add_mom_test TEST_FILE)
+    cmake_parse_arguments(TEST "" "" "LINK_LIBRARIES" ${ARGN})
+    get_filename_component(TEST_TARGET ${TEST_FILE} NAME_WE)
+
+    set(_pfunit_other_sources "")
+    if(BASE_MOM_PFUNIT_INFRA)
+        set(_pfunit_other_sources OTHER_SOURCES "${BASE_MOM_PFUNIT_INFRA}")
+    endif()
+    add_pfunit_ctest(${TEST_TARGET}
+        TEST_SOURCES "${TEST_FILE}"
+        LINK_LIBRARIES TURBO::infra_r8 ${TEST_LINK_LIBRARIES}
+        ${_pfunit_other_sources}
+        MAX_PES 4
+    )
+
+    # Drive the test exe link via g++ rather than gfortran. The test
+    # sources are Fortran, so CMake would otherwise pick LINKER_LANGUAGE
+    # Fortran -- but gfortran does not auto-link the full C++ runtime
+    # (libstdc++, libgcc_s, libgcc) that AMReX needs when TURBO_INFRA=TIM
+    # is pulled in transitively through TURBO::infra_r8. g++ drives the
+    # link in both FMS2 and TIM configurations; CMake still appends
+    # -lgfortran/-lquadmath because Fortran objects contribute to the
+    # target.
+    set_property(TARGET ${TEST_TARGET} PROPERTY LINKER_LANGUAGE CXX)
+endfunction()
+
+# copy_dummy_fms_input_nml()
+# Copies input.nml into the current build directory so FMS can find it at runtime.
+# Call once per test subdirectory that links against FMS if input.nml is not already present in the source directory.
+function(copy_dummy_fms_input_nml)
+    configure_file(${CONFIG_FILES}/input.nml input.nml COPYONLY)
+endfunction()