refactor(paths) + fix(symbols): BuildLayout single-source + PT_LOAD bloat filter (#455)

* refactor(paths): make BuildLayout the single source of truth for the build dir

Closes the FastLED-shaped path bug #432: a project staged
at `<repo>/.build/pio/<board>/` and built with `env == board` was
yielding `.build/pio/<board>/.fbuild/build/<board>/release/firmware.hex`
— the board name appearing twice and the path deep enough to threaten
Windows' 260-char MAX_PATH limit.

Two related issues fixed together:

1. **`BuildParams.build_dir` was dead code.** The daemon HTTP handlers
   carefully computed it via `get_project_build_root(&project_dir)` and
   passed it through, but `pipeline/context.rs` ignored that field and
   re-derived the build dir via `Cache::new(project_dir).get_build_dir
   (env, profile)`. Zero `params.build_dir` references in
   `crates/fbuild-build/src/**` confirmed the field was load-bearing
   only in name. HTTP callers had no way to actually override the
   build dir.

2. **No collapse rule for the `<env>` segment.** `compile_many.rs`
   separately hard-coded `.fbuild/build/<env>/<profile>` instead of
   going through `fbuild-paths`. Different code paths drifted.

Changes:

- `fbuild-paths::BuildLayout`: single resolver with precedence
  `override_root` > `FBUILD_BUILD_DIR` > `<project>/.fbuild/build`,
  plus the `<env>/<profile>` append. Drops the `<env>` segment when
  `flatten_env` is set or auto-detects when `project_dir.file_name()
  == env_name` — the FastLED `.build/pio/<board>/` case is now fixed
  with zero consumer-side change.

- `pipeline/context.rs:120` now reads `params.build_dir` directly
  instead of re-deriving via `Cache`. The build_dir field is finally
  load-bearing.

- `Cache::get_build_dir`, `compile_many::project_build_dir`, and
  `fbuild-paths::find_firmware` route through `BuildLayout` so layout
  decisions live in exactly one place.

- HTTP request models (`BuildRequest`, `DeployRequest`, `TestEmuRequest`)
  gained `build_dir_override` + `flatten_env`. Embedders (FastLED) can
  opt in over HTTP.

- Per-platform integration tests migrated from
  `tmp.path().join(".fbuild/build")` to the explicit
  `<env>/<profile>` shape that matches the new contract. Regression
  tests added: `find_firmware_in_collapsed_layout_when_basename_matches_env`
  in `fbuild-paths` and `project_build_dir_collapses_when_sketch_basename_matches_env`
  in `fbuild-build`.

Workspace verification:

- `soldr cargo clippy --workspace --all-targets -- -D warnings` clean.
- `soldr cargo test --workspace` — 52 test crates, zero failures.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(symbols): drop symbols outside PT_LOAD regions from bloat report

`fbuild bloat`/`fbuild symbols` was reporting nm's full symbol-table
view, including linker-script boundary markers whose nm-computed
"size" is the address gap to the next symbol — which can be multiple
gigabytes. On an nrf52840 firmware with a real `.text` of 32,740 B,
the bloat report claimed flash = 8,052,240,525 B (~8 GB), with
`__flash_arduino_end` (4 GB) and `__StackTop` (3.5 GB) topping the
list. Those are linker markers, not bytes in the final binary.

Fix:

- New `fbuild_core::symbol_analysis::LoadedRegion` representing a
  PT_LOAD program-header range. `FineGrainedSymbolMap::retain_loaded_symbols`
  filters the symbol list to those whose `[addr, addr + size)` fits
  strictly inside some loaded region, then recomputes totals.

- Strict-lower-bound containment (`addr < region.end`) so size-0
  boundary symbols at the end-of-segment (the `__StackTop` case)
  are excluded.

- `fbuild_build::symbol_analyzer::read_pt_load_regions` parses PT_LOAD
  segments from the ELF via the `object` crate (already a dev-dep;
  promoted to a regular dep). Supports ELF32 + ELF64. Failures
  degrade to a `tracing::warn!` plus unfiltered output rather than
  erroring — the bloat report should still surface useful data on a
  corrupt or non-ELF input.

- `analyze_elf` calls `retain_loaded_symbols` after building the map,
  so every consumer of `fbuild symbols` / `fbuild bloat` benefits
  automatically.

After the fix on the same nrf52840 ELF:
- 520 → 519 symbols
- total_flash 8 GB → 33,083 B (within 1 % of readelf's `.text` =
  32,740 B; small overshoot from weak symbols counted twice by nm).
- `__flash_arduino_end` and `__StackTop` no longer appear.
- `__HeapBase` correctly remains under RAM (its address is inside
  the RAM PT_LOAD range; the 227 KB number reflects the .heap
  region's NOBITS allocation).

Unit tests:

- `loaded_region_strict_containment`: covers strict lower bound,
  end-aligned upper bound, overflow guard.
- `retain_loaded_symbols_drops_boundary_markers`: pins the fix
  against synthetic `__StackTop` / `__flash_arduino_end` /
  oversized inputs.
- `retain_loaded_symbols_no_op_when_regions_empty`: defensive case
  for PT_LOAD probe failure (don't empty the map).

Workspace verification:

- `soldr cargo clippy --workspace --all-targets -- -D warnings` clean.
- `soldr cargo test --workspace` — 52 test crates, zero failures.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* chore(deploy): inline override-root build to keep deploy.rs under 1000 LOC

The BuildLayout migration in this branch pushed the file from 998 →
1002 LOC, tripping the workspace's `Reject .rs files over 1000 LOC`
gate. Compress the override-root resolution into a single chained
expression — same behavior, three fewer lines.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* refactor(daemon): extract resolve_build_dir helper so deploy.rs stays under 1000 LOC

The previous BuildLayout migration duplicated the
`override > FBUILD_BUILD_DIR > default + flatten_env` resolution in
deploy.rs and build.rs. rustfmt's preferred form pushed deploy.rs to
1002 LOC, tripping the workspace's per-file gate.

Lift the resolution into `common::resolve_build_dir` so:
- deploy.rs drops back to 998 LOC.
- Both HTTP handlers share one implementation of the precedence
  rules, so future changes can't drift between them.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

Author: zackees

crates/fbuild-build/Cargo.toml

@@ -24,9 +24,9 @@ walkdir = { workspace = true }
 sha2 = { workspace = true }
 blake3 = { workspace = true }
 tempfile = { workspace = true }
+object = { workspace = true }
 
 [dev-dependencies]
 filetime = { workspace = true }
 fbuild-test-support = { path = "../fbuild-test-support" }
 tar = { workspace = true }
-object = { workspace = true }

crates/fbuild-build/src/compile_many.rs

@@ -87,19 +87,14 @@ pub fn default_sketch_jobs() -> usize {
 
 /// Resolve the on-disk build root the orchestrator uses for a given sketch.
 ///
-/// Matches the convention used by every per-platform orchestrator:
-/// `<sketch>/.fbuild/build/<env>/<profile>/`. Centralized here so the
-/// stage-1→stage-2 cache-seeding code can address that path without
-/// re-deriving the same string in three places. See FastLED/fbuild#335.
+/// Routes through [`fbuild_paths::BuildLayout`] so layout decisions
+/// (env-segment auto-collapse, `FBUILD_BUILD_DIR` override) match
+/// what the daemon and the per-platform orchestrators use. Centralized
+/// here so the stage-1→stage-2 cache-seeding code can address that path
+/// without re-deriving the same string in three places. See
+/// FastLED/fbuild#335.
 pub fn project_build_dir(sketch: &Path, env: &str, profile: BuildProfile) -> PathBuf {
-    sketch
-        .join(".fbuild")
-        .join("build")
-        .join(env)
-        .join(match profile {
-            BuildProfile::Release => "release",
-            BuildProfile::Quick => "quick",
-        })
+    fbuild_paths::BuildLayout::new(sketch.to_path_buf(), env.to_string(), profile).resolve()
 }
 
 /// Seed a stage-2 project's framework `core/` from stage 1's, so the
@@ -833,6 +828,24 @@ mod tests {
         assert!(q.ends_with("sketch/.fbuild/build/esp32s3/quick"));
     }
 
+    /// FastLED stages each board's project at
+    /// `<repo>/.build/pio/<board>/` and asks fbuild to build with
+    /// `env == board`. `project_build_dir` must collapse the duplicate
+    /// `<board>` segment (via `BuildLayout`'s auto-detect rule) so the
+    /// resulting tree fits under Windows' 260-char `MAX_PATH` limit and
+    /// matches what `find_firmware` looks for. See FastLED/fbuild#432.
+    #[test]
+    fn project_build_dir_collapses_when_sketch_basename_matches_env() {
+        let sketch = Path::new("/repo/.build/pio/teensy40");
+        let p = project_build_dir(sketch, "teensy40", BuildProfile::Release);
+        let s = p.to_string_lossy().to_string();
+        assert!(
+            !s.contains("build/teensy40/release") && !s.contains("build\\teensy40\\release"),
+            "stage-2 build dir kept duplicated env segment: {s}"
+        );
+        assert!(p.ends_with("release"));
+    }
+
     #[test]
     fn resolve_env_picks_literal_env_name() {
         let tmp = tempfile::tempdir().unwrap();

crates/fbuild-build/src/pipeline/context.rs

@@ -110,16 +110,22 @@ impl BuildContext {
         }
 
         // 4. Setup build directories
+        //
+        // `params.build_dir` is the authoritative env-rooted build dir
+        // (caller resolves it via `fbuild_paths::BuildLayout`). We never
+        // re-derive it from `project_dir` here, so callers can override
+        // the layout (e.g. flatten the env segment when their project
+        // dir is already named after the env — the FastLED
+        // `.build/pio/<board>/` case). See FastLED/fbuild#432.
         let t0 = std::time::Instant::now();
-        let cache = fbuild_packages::Cache::new(project_dir);
-        if params.clean {
-            cache.clean_build(env_name, params.profile)?;
+        let build_dir = params.build_dir.clone();
+        if params.clean && build_dir.exists() {
+            std::fs::remove_dir_all(&build_dir)?;
         }
-        cache.ensure_build_directories(env_name, params.profile)?;
-
-        let build_dir = cache.get_build_dir(env_name, params.profile);
-        let core_build_dir = cache.get_core_build_dir(env_name, params.profile);
-        let src_build_dir = cache.get_src_build_dir(env_name, params.profile);
+        let core_build_dir = build_dir.join("core");
+        let src_build_dir = build_dir.join("src");
+        std::fs::create_dir_all(&core_build_dir)?;
+        std::fs::create_dir_all(&src_build_dir)?;
         if let Some(p) = perf.as_mut() {
             p.record("build-dirs", t0.elapsed());
         }

crates/fbuild-build/src/symbol_analyzer.rs

@@ -15,10 +15,93 @@ use std::collections::BTreeMap;
 use fbuild_core::subprocess::run_command_with_stdin;
 use fbuild_core::symbol_analysis::{
     build_fine_grained_map_with_synth, collect_map_derived_owners, parse_linker_map,
-    parse_nm_output, FineGrainedSymbolMap,
+    parse_nm_output, FineGrainedSymbolMap, LoadedRegion,
 };
 use fbuild_core::{FbuildError, Result};
 
+/// Read the `PT_LOAD` program-header ranges from an ELF. These are the
+/// regions that actually get programmed into the device's flash/RAM
+/// image at boot — every other byte in the ELF (debug info,
+/// `.ARM.attributes`, `.comment`, symbol tables) lives only in the
+/// host-side file and never reaches the binary.
+///
+/// Used by [`analyze_elf`] to drop linker-script boundary symbols
+/// (`__StackTop`, `__flash_arduino_end`, ...) that nm reports with
+/// nonsense sizes computed from the gap to the next symbol — these
+/// inflate the bloat report by gigabytes if not filtered.
+pub fn read_pt_load_regions(elf_path: &Path) -> Result<Vec<LoadedRegion>> {
+    use object::read::elf::{ElfFile32, ElfFile64, FileHeader, ProgramHeader};
+    use object::{Endianness, FileKind};
+
+    let bytes = std::fs::read(elf_path).map_err(|e| {
+        FbuildError::BuildFailed(format!(
+            "could not read ELF at {} for PT_LOAD probe: {e}",
+            elf_path.display()
+        ))
+    })?;
+    let kind = FileKind::parse(&bytes[..]).map_err(|e| {
+        FbuildError::BuildFailed(format!(
+            "could not identify file kind for {}: {e}",
+            elf_path.display()
+        ))
+    })?;
+
+    let mut regions = Vec::new();
+    match kind {
+        FileKind::Elf32 => {
+            let elf = ElfFile32::<Endianness>::parse(&bytes[..])
+                .map_err(|e| FbuildError::BuildFailed(format!("ELF32 parse failed: {e}")))?;
+            let endian = elf
+                .elf_header()
+                .endian()
+                .map_err(|e| FbuildError::BuildFailed(format!("ELF32 endian probe failed: {e}")))?;
+            for ph in elf.elf_program_headers() {
+                if ph.p_type(endian) != object::elf::PT_LOAD {
+                    continue;
+                }
+                let start = u64::from(ph.p_vaddr(endian));
+                let size = u64::from(ph.p_memsz(endian));
+                if size == 0 {
+                    continue;
+                }
+                regions.push(LoadedRegion {
+                    start,
+                    end: start.saturating_add(size),
+                });
+            }
+        }
+        FileKind::Elf64 => {
+            let elf = ElfFile64::<Endianness>::parse(&bytes[..])
+                .map_err(|e| FbuildError::BuildFailed(format!("ELF64 parse failed: {e}")))?;
+            let endian = elf
+                .elf_header()
+                .endian()
+                .map_err(|e| FbuildError::BuildFailed(format!("ELF64 endian probe failed: {e}")))?;
+            for ph in elf.elf_program_headers() {
+                if ph.p_type(endian) != object::elf::PT_LOAD {
+                    continue;
+                }
+                let start = ph.p_vaddr(endian);
+                let size = ph.p_memsz(endian);
+                if size == 0 {
+                    continue;
+                }
+                regions.push(LoadedRegion {
+                    start,
+                    end: start.saturating_add(size),
+                });
+            }
+        }
+        other => {
+            return Err(FbuildError::BuildFailed(format!(
+                "expected ELF, got {other:?} at {}",
+                elf_path.display()
+            )));
+        }
+    }
+    Ok(regions)
+}
+
 /// Auto-detect the cross-toolchain prefix from the directory containing
 /// an `nm` binary. e.g. `xtensa-esp32s3-elf-nm` → prefix
 /// `xtensa-esp32s3-elf-`, so `xtensa-esp32s3-elf-c++filt` can be derived.
@@ -181,14 +264,38 @@ pub fn analyze_elf(cfg: AnalyzeConfig<'_>) -> Result<FineGrainedSymbolMap> {
         synth_mangled.clone()
     };
 
-    Ok(build_fine_grained_map_with_synth(
+    let mut map = build_fine_grained_map_with_synth(
         elf_s,
         cfg.map_path.map(|p| p.to_string_lossy().to_string()),
         nm_rows,
         demangled,
         ranges,
         &synth_demangled,
-    ))
+    );
+
+    // Strip symbols that nm enumerated but that don't actually consume
+    // bytes in the final binary — most importantly linker-script
+    // boundary markers (`__StackTop`, `__flash_arduino_end`, ...)
+    // whose nm-reported "size" is the address gap to the next symbol
+    // and can be multiple gigabytes. The PT_LOAD probe is best-effort;
+    // when it fails (corrupt ELF, non-ELF input) we leave the map
+    // unfiltered rather than poisoning the report with an error.
+    match read_pt_load_regions(cfg.elf_path) {
+        Ok(regions) if !regions.is_empty() => map.retain_loaded_symbols(&regions),
+        Ok(_) => {
+            tracing::warn!(
+                "no PT_LOAD segments found in {}; emitting unfiltered symbol report",
+                cfg.elf_path.display()
+            );
+        }
+        Err(e) => {
+            tracing::warn!(
+                "PT_LOAD probe failed for {} ({e}); emitting unfiltered symbol report",
+                cfg.elf_path.display()
+            );
+        }
+    }
+    Ok(map)
 }
 
 /// Format a fine-grained per-symbol map as a human-readable text report

crates/fbuild-build/tests/avr_build.rs

@@ -75,9 +75,11 @@ fn build_uno_minimal() {
         return;
     }
 
-    // Use a temp build dir so we don't pollute the Python project
+    // Use a temp build dir so we don't pollute the Python project.
+    // `params.build_dir` is the resolved env-rooted dir per
+    // `BuildLayout::resolve()`.
     let tmp = tempfile::TempDir::new().unwrap();
-    let build_dir = tmp.path().join(".fbuild/build");
+    let build_dir = tmp.path().join(".fbuild/build/uno/release");
 
     let params = BuildParams {
         project_dir: project_dir.clone(),
@@ -172,7 +174,7 @@ fn compare_with_python_output() {
 
     // Build with Rust
     let tmp = tempfile::TempDir::new().unwrap();
-    let build_dir = tmp.path().join(".fbuild/build");
+    let build_dir = tmp.path().join(".fbuild/build/uno/release");
 
     let params = BuildParams {
         project_dir: project_dir.clone(),
@@ -258,7 +260,7 @@ void loop() {
     )
     .unwrap();
 
-    let build_dir = project_dir.join(".fbuild/build");
+    let build_dir = project_dir.join(".fbuild/build/uno/release");
     let params = BuildParams {
         project_dir: project_dir.to_path_buf(),
         env_name: "uno".to_string(),
@@ -435,7 +437,7 @@ fn cache_survives_tar_extract_uno() {
     let cold_result = orchestrator
         .build(&uno_build_params(
             &proj_a,
-            proj_a.join(".fbuild/build"),
+            proj_a.join(".fbuild/build/uno/release"),
             true,
         ))
         .expect("cold AVR build should succeed");
@@ -467,7 +469,7 @@ fn cache_survives_tar_extract_uno() {
     let same_project_warm = orchestrator
         .build(&uno_build_params(
             &proj_a,
-            proj_a.join(".fbuild/build"),
+            proj_a.join(".fbuild/build/uno/release"),
             false,
         ))
         .expect("same-project warm build should succeed");
@@ -512,7 +514,7 @@ fn cache_survives_tar_extract_uno() {
     let warm_result = orchestrator
         .build(&uno_build_params(
             &proj_b,
-            proj_b.join(".fbuild/build"),
+            proj_b.join(".fbuild/build/uno/release"),
             false,
         ))
         .expect("warm AVR build (post tar-extract) should succeed");

crates/fbuild-build/tests/eh_frame_strip_esp32.rs

@@ -19,7 +19,7 @@ use fbuild_build::{BuildOrchestrator, BuildParams};
 use fbuild_core::BuildProfile;
 
 fn make_params(project_dir: &Path) -> BuildParams {
-    let build_dir = project_dir.join(".fbuild/build");
+    let build_dir = project_dir.join(".fbuild/build/esp32dev/release");
     BuildParams {
         project_dir: project_dir.to_path_buf(),
         env_name: "esp32dev".to_string(),

crates/fbuild-build/tests/esp32_build.rs

@@ -60,7 +60,7 @@ void loop() {
     )
     .unwrap();
 
-    let build_dir = project_dir.join(".fbuild/build");
+    let build_dir = project_dir.join(".fbuild/build/esp32dev/release");
     let params = BuildParams {
         project_dir: project_dir.to_path_buf(),
         env_name: "esp32dev".to_string(),
@@ -149,7 +149,7 @@ void loop() {
     )
     .unwrap();
 
-    let build_dir = project_dir.join(".fbuild/build");
+    let build_dir = project_dir.join(".fbuild/build/esp32c6/release");
     let params = BuildParams {
         project_dir: project_dir.to_path_buf(),
         env_name: "esp32c6".to_string(),
@@ -231,7 +231,7 @@ void loop() {
     )
     .unwrap();
 
-    let build_dir = project_dir.join(".fbuild/build");
+    let build_dir = project_dir.join(".fbuild/build/esp32c3/release");
     let params = BuildParams {
         project_dir: project_dir.to_path_buf(),
         env_name: "esp32c3".to_string(),
@@ -314,7 +314,7 @@ void loop() {
     )
     .unwrap();
 
-    let build_dir = project_dir.join(".fbuild/build");
+    let build_dir = project_dir.join(".fbuild/build/esp32s3/release");
     let params = BuildParams {
         project_dir: project_dir.to_path_buf(),
         env_name: "esp32s3".to_string(),
@@ -387,7 +387,7 @@ fn build_esp32s3_fixture() {
         return;
     }
 
-    let build_dir = project_dir.join(".fbuild/build");
+    let build_dir = project_dir.join(".fbuild/build/esp32s3/release");
     let params = BuildParams {
         project_dir: project_dir.clone(),
         env_name: "esp32s3".to_string(),
@@ -451,7 +451,7 @@ fn build_nightdriverstrip_demo() {
     }
 
     let tmp = tempfile::TempDir::new().unwrap();
-    let build_dir = tmp.path().join(".fbuild/build");
+    let build_dir = tmp.path().join(".fbuild/build/demo/release");
 
     let params = BuildParams {
         project_dir: project_dir.clone(),
@@ -549,7 +549,12 @@ fn incremental_build_at(project_dir: &std::path::Path, env_name: &str) {
         env_name: env_name.to_string(),
         clean: false,
         profile: BuildProfile::Release,
-        build_dir: project_dir.join(".fbuild/build"),
+        build_dir: fbuild_paths::BuildLayout::new(
+            project_dir.to_path_buf(),
+            env_name.to_string(),
+            BuildProfile::Release,
+        )
+        .resolve(),
         verbose: true,
         jobs: None,
         generate_compiledb: false,
@@ -640,7 +645,12 @@ fn incremental_nightdriverstrip_one_file_changed() {
         env_name: env_name.to_string(),
         clean: false,
         profile: BuildProfile::Release,
-        build_dir: project_dir.join(".fbuild/build"),
+        build_dir: fbuild_paths::BuildLayout::new(
+            project_dir.clone(),
+            env_name.to_string(),
+            BuildProfile::Release,
+        )
+        .resolve(),
         verbose: true,
         jobs: None,
         generate_compiledb: false,

crates/fbuild-build/tests/stm32_acceptance.rs

@@ -60,7 +60,7 @@ fn stm32f103c8_blink_with_spi_auto_discovers_library_205_ac4() {
     )
     .unwrap();
 
-    let build_dir = project_dir.join(".fbuild/build");
+    let build_dir = project_dir.join(".fbuild/build/stm32f103c8/release");
     let params = BuildParams {
         project_dir: project_dir.to_path_buf(),
         env_name: "stm32f103c8".to_string(),

crates/fbuild-build/tests/teensy30_acceptance.rs

@@ -75,7 +75,7 @@ fn teensy30_analog_output_meets_205_ac2() {
     )
     .unwrap();
 
-    let build_dir = project_dir.join(".fbuild/build");
+    let build_dir = project_dir.join(".fbuild/build/teensy30/release");
     let params = BuildParams {
         project_dir: project_dir.to_path_buf(),
         // WHY env_name = "teensy30": must match the [env:teensy30] key