Skip to content

Commit 24b7de4

Browse files
zackeesclaude
andauthored
feat(build): migrate BuildInfo to NormalizedPath (Phase 2 of #437) (#454)
* feat(build): migrate BuildInfo path fields to NormalizedPath (Phase 2 of #437) Phase 2 of the NormalizedPath migration plan from #437. Phase 1 (#450) landed the `NormalizedPath` type in `fbuild-core::path`; this PR routes `BuildInfo`'s ten `*_path` fields plus the `aliases` block through it and proves the cross-platform JSON contract directly. Changes ------- - `NormalizedPath::serialize` now emits the slash-form, case-preserving rendering (`display_slash`). Cache keys (`key()`) keep their case-folded form for `Hash`/`Eq` stability on Windows + default macOS, but serialization stays human-readable while still being byte- identical across platforms. - Added `NormalizedPath::default()` (empty path) so `#[serde(default)]` preserves the existing "missing → empty-string" sentinel that downstream Python consumers (FastLED's `_create_board_info`, `_find_build_info`) rely on. - `BuildInfo`'s ten `*_path` fields (`prog_path`, `cc_path`, `cxx_path`, `ar_path`, `objcopy_path`, `size_path`, `nm_path`, `cppfilt_path`, `readelf_path`, `objdump_path`) and `aliases: BTreeMap<String, _>` values switched from `String` to `NormalizedPath`. JSON schema is unchanged for downstream consumers — `NormalizedPath` (de)serializes as a plain JSON string. - `fbuild-cli`'s `symbols_cmd` consumers updated to read/write `NormalizedPath` instead of `String`. Tests ----- - New `emit_json_uses_forward_slashes_regardless_of_input_separators` test pins the cross-platform JSON contract directly — constructs `BuildInfo` via `Path::join` (which leaks `\` on Windows) and asserts the emitted JSON contains zero backslashes. - Retired the `pj()` helper from build_info.rs: assertions now compare against literal slash-form expectations on every platform — the workaround the issue's acceptance criteria explicitly asked us to remove. - New `NormalizedPath` tests: `serialize_emits_slash_form` and (cfg=windows) `serialize_strips_extended_length_prefix`. - `json_round_trip_preserves_path_form` updated to assert `original == back` (Equal under `NormalizedPath`'s normalized comparison). Acceptance vs. #437 ------------------- - [x] `NormalizedPath` exists in `fbuild-core` and `Hash`/`Eq` produce identical bytes for `C:\Users\x\a` and `C:/Users/x/a` (#450). - [x] `BuildInfo`'s nine path fields are `NormalizedPath`, and `build_info.json` emits the canonical (slash-form) string on every platform. - [ ] `cargo dylint --all` errors on new raw-path slots — deferred to Phase 3 per the original plan. - [x] `fbuild symbols` regression test passes without a `pj()` helper. Closes #437 (Phase 2). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * test(core): split NormalizedPath serialize tests by platform CI on macOS + Ubuntu failed `serialize_emits_slash_form` because the test used `r"C:\Users\zach\bin\avr-nm"` as input and then asserted the serialized JSON contains no `\`. On Unix the backslash is a legal filename character — not a separator — so the implementation correctly preserves it. The assertion was wrong, not the code. Split into three: - `serialize_emits_slash_form`: forward-slash input survives unchanged everywhere (the core invariant). - `serialize_converts_backslash_input_to_forward_slashes_on_windows` (cfg=windows): Windows-only conversion of `\` → `/`. - `serialize_preserves_backslashes_as_content_on_unix` (cfg=not(windows)): proves we *don't* mangle real filenames containing `\` on Linux/macOS. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
1 parent c3be150 commit 24b7de4

3 files changed

Lines changed: 283 additions & 78 deletions

File tree

crates/fbuild-build/src/build_info.rs

Lines changed: 155 additions & 66 deletions
Original file line numberDiff line numberDiff line change
@@ -17,6 +17,7 @@
1717
use std::collections::BTreeMap;
1818
use std::path::{Path, PathBuf};
1919

20+
use fbuild_core::path::NormalizedPath;
2021
use fbuild_core::Result;
2122
use serde::{Deserialize, Serialize};
2223

@@ -25,39 +26,45 @@ use serde::{Deserialize, Serialize};
2526
/// All paths are emitted as strings (matching `pio project metadata`
2627
/// output); empty / missing toolchain entries are emitted as empty strings
2728
/// so consumers that do `Path(board_info["objcopy_path"])` never KeyError.
29+
///
30+
/// The ten `*_path` fields are [`NormalizedPath`] (Phase 2 of #437), so
31+
/// the emitted JSON is byte-identical across Linux/macOS/Windows even
32+
/// when the source paths were constructed via `PathBuf::join` (which
33+
/// would otherwise leak `\` separators on Windows — the original
34+
/// regression from PR #436).
2835
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq)]
2936
pub struct BuildInfo {
3037
/// Absolute path to the final firmware/program file (`.elf` if no
3138
/// `.hex`/`.bin` was produced, otherwise the converted firmware).
32-
pub prog_path: String,
39+
pub prog_path: NormalizedPath,
3340
/// Absolute path to the C compiler (`gcc`).
34-
pub cc_path: String,
41+
pub cc_path: NormalizedPath,
3542
/// Absolute path to the C++ compiler (`g++`).
36-
pub cxx_path: String,
43+
pub cxx_path: NormalizedPath,
3744
/// Absolute path to `ar` (empty when the linker doesn't expose it).
38-
pub ar_path: String,
45+
pub ar_path: NormalizedPath,
3946
/// Absolute path to `objcopy` (empty when the platform has no objcopy step,
4047
/// e.g. ESP8266 which produces ELF directly).
41-
pub objcopy_path: String,
48+
pub objcopy_path: NormalizedPath,
4249
/// Absolute path to `size`.
43-
pub size_path: String,
50+
pub size_path: NormalizedPath,
4451
/// Absolute path to `nm` (GCC convention: derived from `size_path` by
4552
/// replacing the `size` suffix with `nm`). Consumed by `fbuild symbols`
4653
/// (see #428) so users don't have to pass `--nm` on every invocation.
4754
#[serde(default)]
48-
pub nm_path: String,
55+
pub nm_path: NormalizedPath,
4956
/// Absolute path to `c++filt` (GCC convention). Used by the symbol
5057
/// analyzer to demangle the names `nm` emits.
5158
#[serde(default)]
52-
pub cppfilt_path: String,
59+
pub cppfilt_path: NormalizedPath,
5360
/// Absolute path to `readelf`. Useful for downstream section/segment
5461
/// inspection.
5562
#[serde(default)]
56-
pub readelf_path: String,
63+
pub readelf_path: NormalizedPath,
5764
/// Absolute path to `objdump`. Useful for downstream disassembly /
5865
/// section dumps.
5966
#[serde(default)]
60-
pub objdump_path: String,
67+
pub objdump_path: NormalizedPath,
6168
/// Effective C compile flags as seen by the compiler driver.
6269
pub cc_flags: Vec<String>,
6370
/// Effective C++ compile flags as seen by the compiler driver.
@@ -82,8 +89,13 @@ pub struct BuildInfo {
8289
/// `ci/util/symbol_analysis.py` and `ci/inspect_binary.py` read from
8390
/// it. Mirroring keeps existing PIO consumers working drop-in
8491
/// against fbuild-built artifacts. See #428.
92+
///
93+
/// Values are [`NormalizedPath`] (Phase 2 of #437) so on-disk JSON
94+
/// stays byte-identical across platforms; the `NormalizedPath`
95+
/// serde impls produce / accept plain JSON strings, so Python
96+
/// consumers see no schema change.
8597
#[serde(default)]
86-
pub aliases: BTreeMap<String, String>,
98+
pub aliases: BTreeMap<String, NormalizedPath>,
8799
}
88100

89101
impl BuildInfo {
@@ -131,16 +143,16 @@ impl BuildInfo {
131143
&objdump_path,
132144
);
133145
Self {
134-
prog_path: path_to_string(Some(prog_path)),
135-
cc_path: path_to_string(cc_path),
136-
cxx_path: path_to_string(cxx_path),
137-
ar_path: path_to_string(ar_path),
138-
objcopy_path: path_to_string(objcopy_path),
139-
size_path: path_to_string(Some(size_path)),
140-
nm_path: path_to_string(Some(nm_path.as_path())),
141-
cppfilt_path: path_to_string(Some(cppfilt_path.as_path())),
142-
readelf_path: path_to_string(Some(readelf_path.as_path())),
143-
objdump_path: path_to_string(Some(objdump_path.as_path())),
146+
prog_path: NormalizedPath::new(prog_path),
147+
cc_path: normalize_optional(cc_path),
148+
cxx_path: normalize_optional(cxx_path),
149+
ar_path: normalize_optional(ar_path),
150+
objcopy_path: normalize_optional(objcopy_path),
151+
size_path: NormalizedPath::new(size_path),
152+
nm_path: NormalizedPath::new(&nm_path),
153+
cppfilt_path: NormalizedPath::new(&cppfilt_path),
154+
readelf_path: NormalizedPath::new(&readelf_path),
155+
objdump_path: NormalizedPath::new(&objdump_path),
144156
cc_flags,
145157
cxx_flags,
146158
link_flags,
@@ -196,7 +208,7 @@ fn build_aliases(
196208
cppfilt_path: &Path,
197209
readelf_path: &Path,
198210
objdump_path: &Path,
199-
) -> BTreeMap<String, String> {
211+
) -> BTreeMap<String, NormalizedPath> {
200212
let mut aliases = BTreeMap::new();
201213
let entries: &[(&str, Option<&Path>)] = &[
202214
("gcc", cc_path),
@@ -210,17 +222,25 @@ fn build_aliases(
210222
("objdump", Some(objdump_path)),
211223
];
212224
for (key, path) in entries {
213-
let s = path_to_string(*path);
214-
if !s.is_empty() {
215-
aliases.insert((*key).to_string(), s);
225+
let Some(p) = *path else {
226+
continue;
227+
};
228+
// Skip the empty-path sentinel so `aliases["nm"]` is
229+
// guaranteed non-empty whenever the key is present.
230+
if p.as_os_str().is_empty() {
231+
continue;
216232
}
233+
aliases.insert((*key).to_string(), NormalizedPath::new(p));
217234
}
218235
aliases
219236
}
220237

221-
fn path_to_string(p: Option<&Path>) -> String {
222-
p.map(|p| p.to_string_lossy().to_string())
223-
.unwrap_or_default()
238+
/// Convert an optional `&Path` to [`NormalizedPath`], preserving the
239+
/// "missing → empty path" sentinel that `BuildInfo`'s schema requires
240+
/// for downstream Python consumers (`Path(board_info["objcopy_path"])`
241+
/// never throws KeyError).
242+
fn normalize_optional(p: Option<&Path>) -> NormalizedPath {
243+
p.map(NormalizedPath::new).unwrap_or_default()
224244
}
225245

226246
fn extract_prefixed(flags: &[String], prefix: &str) -> Vec<String> {
@@ -419,8 +439,13 @@ mod tests {
419439
"nodemcuv2".to_string(),
420440
"nodemcuv2".to_string(),
421441
);
422-
assert_eq!(info.ar_path, "");
423-
assert_eq!(info.objcopy_path, "");
442+
// Missing → `NormalizedPath::default()` (empty path). This is
443+
// the schema's "absent" sentinel that downstream Python
444+
// consumers (`Path(board_info["objcopy_path"])`) rely on.
445+
assert_eq!(info.ar_path, NormalizedPath::default());
446+
assert_eq!(info.objcopy_path, NormalizedPath::default());
447+
assert_eq!(info.ar_path.as_path().as_os_str(), "");
448+
assert_eq!(info.objcopy_path.as_path().as_os_str(), "");
424449
}
425450

426451
#[test]
@@ -432,6 +457,56 @@ mod tests {
432457
assert!(tmp.path().join("build_info.json").exists());
433458
}
434459

460+
/// The motivating test for #437 Phase 2: when `BuildInfo`'s ten
461+
/// `*_path` fields are constructed from `PathBuf::join` (which
462+
/// uses the platform separator), the emitted JSON must still
463+
/// contain only forward slashes — no `\` leakage on Windows.
464+
///
465+
/// PR #436 had to gate this exact assertion behind a platform
466+
/// `pj()` helper because the field type was `String` and just
467+
/// carried whatever separator the source used. The
468+
/// `NormalizedPath` migration removes that workaround entirely.
469+
#[test]
470+
fn emit_json_uses_forward_slashes_regardless_of_input_separators() {
471+
let tmp = tempfile::TempDir::new().unwrap();
472+
// Construct the source paths via `Path::join` so that on
473+
// Windows the in-memory strings contain `\`. The serialized
474+
// JSON must still come out slash-form.
475+
let bin = PathBuf::from("/bin");
476+
let info = BuildInfo::new(
477+
&bin.join("firmware.elf"),
478+
Some(&bin.join("avr-gcc")),
479+
Some(&bin.join("avr-g++")),
480+
Some(&bin.join("avr-ar")),
481+
Some(&bin.join("avr-objcopy")),
482+
&bin.join("avr-size"),
483+
vec![],
484+
vec![],
485+
vec![],
486+
vec![],
487+
"atmelavr".to_string(),
488+
"uno".to_string(),
489+
"uno".to_string(),
490+
);
491+
emit_build_info(tmp.path(), "uno", &info).unwrap();
492+
let bytes = std::fs::read(tmp.path().join("build_info_uno.json")).unwrap();
493+
let s = std::str::from_utf8(&bytes).unwrap();
494+
495+
// The literal Windows separator must not appear in any path
496+
// string. (A JSON escape sequence `\\` is two backslashes in
497+
// the raw file, so search for *any* backslash byte.)
498+
assert!(
499+
!s.contains('\\'),
500+
"build_info JSON must not contain backslashes; got:\n{s}"
501+
);
502+
// Every path field is present in slash-form.
503+
assert!(s.contains("\"/bin/avr-gcc\""));
504+
assert!(s.contains("\"/bin/avr-nm\""));
505+
assert!(s.contains("\"/bin/avr-c++filt\""));
506+
assert!(s.contains("\"/bin/avr-readelf\""));
507+
assert!(s.contains("\"/bin/avr-objdump\""));
508+
}
509+
435510
/// FastLED/fbuild#406: when FastLED drives fbuild per-example using a
436511
/// nested example name like `Fx/FxNoisePalette`, the env-specific file
437512
/// path expands to `build_info_Fx/FxNoisePalette.json` — i.e. a nested
@@ -576,15 +651,11 @@ mod tests {
576651

577652
// ---- #428: toolchain path derivation + aliases mirror ----
578653

579-
/// Encode a derived path the same way [`BuildInfo::new`] does
580-
/// (`Path::join` uses the platform separator, so test expectations
581-
/// must too — `/bin/avr-nm` on Unix, `/bin\avr-nm` on Windows).
582-
fn pj(parent: &str, name: &str) -> String {
583-
PathBuf::from(parent)
584-
.join(name)
585-
.to_string_lossy()
586-
.into_owned()
587-
}
654+
// Phase 2 of #437 retired the `pj()` Windows-vs-Unix helper: all
655+
// `BuildInfo` path fields are `NormalizedPath` now, so equality
656+
// assertions can compare against literal slash-form strings on
657+
// every platform. The `serialize_*` regression test below pins
658+
// that contract directly.
588659

589660
#[test]
590661
fn derive_gcc_tool_path_avr_prefix() {
@@ -633,42 +704,60 @@ mod tests {
633704
#[test]
634705
fn build_info_populates_new_toolchain_fields() {
635706
let info = sample_info();
636-
// GCC convention: /bin/avr-size → /bin/avr-{nm,c++filt,readelf,objdump}
637-
assert_eq!(info.nm_path, pj("/bin", "avr-nm"));
638-
assert_eq!(info.cppfilt_path, pj("/bin", "avr-c++filt"));
639-
assert_eq!(info.readelf_path, pj("/bin", "avr-readelf"));
640-
assert_eq!(info.objdump_path, pj("/bin", "avr-objdump"));
707+
// GCC convention: /bin/avr-size → /bin/avr-{nm,c++filt,readelf,objdump}.
708+
// Stored as `NormalizedPath` (#437 Phase 2), so the comparison
709+
// is platform-stable forward-slash form everywhere — no `pj()`
710+
// helper needed.
711+
assert_eq!(info.nm_path, NormalizedPath::new("/bin/avr-nm"));
712+
assert_eq!(info.cppfilt_path, NormalizedPath::new("/bin/avr-c++filt"));
713+
assert_eq!(info.readelf_path, NormalizedPath::new("/bin/avr-readelf"));
714+
assert_eq!(info.objdump_path, NormalizedPath::new("/bin/avr-objdump"));
641715
}
642716

643717
#[test]
644718
fn build_info_aliases_block_mirrors_paths() {
645719
let info = sample_info();
646720
// The aliases block carries short PIO-shape keys. Every present
647-
// long-form path must also appear under its alias key.
648-
// cc/cxx/ar/objcopy/size are passed in verbatim, so they keep the
649-
// literal "/" separator we constructed them with. The four derived
650-
// tools (nm/c++filt/readelf/objdump) go through Path::join so they
651-
// use the platform separator — match that via pj().
652-
assert_eq!(info.aliases.get("gcc"), Some(&"/bin/avr-gcc".to_string()));
653-
assert_eq!(info.aliases.get("g++"), Some(&"/bin/avr-g++".to_string()));
654-
assert_eq!(info.aliases.get("ar"), Some(&"/bin/avr-ar".to_string()));
721+
// long-form path must also appear under its alias key, in
722+
// `NormalizedPath` slash-form so the values are byte-identical
723+
// across Linux, macOS, and Windows. See #437 Phase 2 —
724+
// formerly this test had to use `pj()` to track the platform
725+
// separator that `Path::join` leaked into the value strings.
726+
assert_eq!(
727+
info.aliases.get("gcc"),
728+
Some(&NormalizedPath::new("/bin/avr-gcc")),
729+
);
730+
assert_eq!(
731+
info.aliases.get("g++"),
732+
Some(&NormalizedPath::new("/bin/avr-g++")),
733+
);
734+
assert_eq!(
735+
info.aliases.get("ar"),
736+
Some(&NormalizedPath::new("/bin/avr-ar")),
737+
);
655738
assert_eq!(
656739
info.aliases.get("objcopy"),
657-
Some(&"/bin/avr-objcopy".to_string())
740+
Some(&NormalizedPath::new("/bin/avr-objcopy")),
741+
);
742+
assert_eq!(
743+
info.aliases.get("size"),
744+
Some(&NormalizedPath::new("/bin/avr-size")),
745+
);
746+
assert_eq!(
747+
info.aliases.get("nm"),
748+
Some(&NormalizedPath::new("/bin/avr-nm")),
658749
);
659-
assert_eq!(info.aliases.get("size"), Some(&"/bin/avr-size".to_string()));
660-
assert_eq!(info.aliases.get("nm"), Some(&pj("/bin", "avr-nm")));
661750
assert_eq!(
662751
info.aliases.get("c++filt"),
663-
Some(&pj("/bin", "avr-c++filt"))
752+
Some(&NormalizedPath::new("/bin/avr-c++filt")),
664753
);
665754
assert_eq!(
666755
info.aliases.get("readelf"),
667-
Some(&pj("/bin", "avr-readelf"))
756+
Some(&NormalizedPath::new("/bin/avr-readelf")),
668757
);
669758
assert_eq!(
670759
info.aliases.get("objdump"),
671-
Some(&pj("/bin", "avr-objdump"))
760+
Some(&NormalizedPath::new("/bin/avr-objdump")),
672761
);
673762
}
674763

@@ -774,25 +863,25 @@ mod tests {
774863
.expect("aliases block must be emitted")
775864
.as_object()
776865
.expect("aliases must be an object");
777-
let expected_nm = pj("/bin", "avr-nm");
778-
let expected_cppfilt = pj("/bin", "avr-c++filt");
779-
let expected_readelf = pj("/bin", "avr-readelf");
780-
let expected_objdump = pj("/bin", "avr-objdump");
866+
// Phase 2 of #437: `NormalizedPath::Serialize` emits forward
867+
// slashes regardless of host platform, so these expected
868+
// values are literal slash-form strings — the old `pj()`
869+
// helper that papered over the cross-platform drift is gone.
781870
assert_eq!(
782871
aliases.get("nm").and_then(|v| v.as_str()),
783-
Some(expected_nm.as_str())
872+
Some("/bin/avr-nm"),
784873
);
785874
assert_eq!(
786875
aliases.get("c++filt").and_then(|v| v.as_str()),
787-
Some(expected_cppfilt.as_str())
876+
Some("/bin/avr-c++filt"),
788877
);
789878
assert_eq!(
790879
aliases.get("readelf").and_then(|v| v.as_str()),
791-
Some(expected_readelf.as_str())
880+
Some("/bin/avr-readelf"),
792881
);
793882
assert_eq!(
794883
aliases.get("objdump").and_then(|v| v.as_str()),
795-
Some(expected_objdump.as_str())
884+
Some("/bin/avr-objdump"),
796885
);
797886
}
798887
}

crates/fbuild-cli/src/cli/symbols_cmd.rs

Lines changed: 9 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -240,13 +240,15 @@ impl ToolPaths {
240240
}
241241
}
242242

243-
/// Treat an empty BuildInfo string field (the schema's "missing"
244-
/// sentinel) as `None`.
245-
fn option_path(s: &str) -> Option<PathBuf> {
246-
if s.is_empty() {
243+
/// Treat an empty BuildInfo path field (the schema's "missing"
244+
/// sentinel) as `None`. `BuildInfo`'s `*_path` fields became
245+
/// `NormalizedPath` in #437 Phase 2, so emptiness is checked on the
246+
/// underlying `OsStr` rather than on a `String`.
247+
fn option_path(p: &fbuild_core::path::NormalizedPath) -> Option<PathBuf> {
248+
if p.as_path().as_os_str().is_empty() {
247249
None
248250
} else {
249-
Some(PathBuf::from(s))
251+
Some(p.as_path().to_path_buf())
250252
}
251253
}
252254

@@ -274,8 +276,8 @@ mod tests {
274276
"test".to_string(),
275277
"test".to_string(),
276278
);
277-
info.nm_path = nm.to_string();
278-
info.cppfilt_path = cppfilt.to_string();
279+
info.nm_path = fbuild_core::path::NormalizedPath::new(nm);
280+
info.cppfilt_path = fbuild_core::path::NormalizedPath::new(cppfilt);
279281
info
280282
}
281283

0 commit comments

Comments
 (0)