ENH: Use NumberToString for lossless float metadata in NIfTI reader#6003
Conversation
|
| Filename | Overview |
|---|---|
| Modules/IO/NIFTI/src/itkNiftiImageIO.cxx | All floating-point NIfTI metadata fields (pixdim, intent_p1/2/3, scl_slope/inter, cal_max/min, slice_duration, toffset, quaternion params, srow matrix) now use ConvertNumberToString; srow loop refactor is correct. |
| Modules/IO/Meta/src/itkMetaImageIO.cxx | Fixes double/float scalar and vector metadata serialization; however, the joinElements static helper is now dead code after the vector path was replaced with a manual loop. |
| Modules/IO/Meta/include/itkMetaImageIO.h | WriteMatrixInMetaData template now uses ConvertNumberToString for each matrix element; change is correct and complete. |
| Modules/IO/NRRD/src/itkNrrdImageIO.cxx | Adds explicit double and float specializations of _dump_metadata_to_stream within the itk namespace; correct use of template specialization, no issues. |
| Modules/IO/MeshFreeSurfer/include/itkFreeSurferAsciiMeshIO.h | Removes outputFile.precision(6) and std::fixed, replaces with ConvertNumberToString; correct fix for precision loss in point coordinate serialization. |
| Modules/Nonunit/Review/src/itkVoxBoCUBImageIO.cxx | Spacing values now serialized with ConvertNumberToString; correct and minimal change. |
| Modules/IO/SpatialObjects/src/itkPolygonGroupSpatialObjectXMLFile.cxx | Polygon vertex coordinates now serialized with ConvertNumberToString; correct fix. |
| Modules/Nonunit/Review/test/itkVoxBoCUBImageIOTest.cxx | New precision sub-test verifies spacing round-trip, but the temporary precision file is never removed after the test completes. |
| Modules/IO/NIFTI/test/itkNiftiImageIOTest3.cxx | New TestNiftiFloatMetadataPrecision function correctly verifies pixdim round-trip; proper cleanup on pass path. |
| Modules/IO/Meta/test/itkMetaImageIOMetaDataTest.cxx | New double/float precision sub-tests verify bit-exact round-trip via stod/stof; well-structured and clear. |
| Modules/IO/NRRD/test/itkNrrdMetaDataTest.cxx | Existing test logic inversion fixed (EXIT_SUCCESS/EXIT_FAILURE swapped back to correct sense); new precision sub-tests added correctly. |
Flowchart
%%{init: {'theme': 'neutral'}}%%
flowchart TD
A[NIfTI / MetaImage / NRRD\nVoxBoCUB / FreeSurfer / PolygonXML\nIO Write Path] --> B{Numeric metadata\nfield}
B -->|float / double| C["itk::ConvertNumberToString()\n(Google double-conversion ToShortest)\nShortest exact round-trip string"]
B -->|int / string / other| D[Default ostream <<\nno change]
C --> E[Text header / MetaDataDictionary string]
D --> E
E --> F[Disk file]
F --> G[IO Read Path]
G --> H["std::stof / std::stod\nor direct float parse"]
H --> I{Bit-exact round-trip?}
I -->|With fix: YES| J[Correct metadata value]
I -->|Before fix: NO\n6-digit truncation| K[Different float bit-pattern]
Reviews (2): Last reviewed commit: "ENH: Add precision round-trip tests for ..." | Re-trigger Greptile
|
Added a second commit that applies the same lossless float fix to MetaImage IO ( Three call sites corrected in the metadata serialization loop:
All 18 MetaImage IO tests pass. |
|
Third commit: NRRD IO fix ( The generic Note: All 33 NRRD tests pass. |
28f52e6 to
1efb6dd
Compare
Adds test coverage to four IO modules verifying that float/double values survive disk round-trips without the 6-digit truncation that default stream precision introduces (ITK issue InsightSoftwareConsortium#3249): * itkNiftiImageIOTest3.cxx: TestNiftiFloatMetadataPrecision() writes an image with 0.12345679-mm spacing, reads back, and verifies the MetaDataDictionary["pixdim[1]"] string parses to the exact float32. * itkMetaImageIOMetaDataTest.cxx: encapsulates hpDouble=1.2345678901234568 and hpFloat=1.2345679f, writes/reads MetaImage, checks exact equality via stod/stof (bypassing the loose 1e-6 tolerance in Equal<double>). * itkNrrdMetaDataTest.cxx: same pattern for native double and float NRRD metadata keys. * itkVoxBoCUBImageIOTest.cxx: creates an image with 0.12345678901234 spacing, writes/reads CUB, asserts GetSpacing()[0] == original double. These tests FAIL without the ConvertNumberToString fixes in PR InsightSoftwareConsortium#6003 and PASS after them (Closes InsightSoftwareConsortium#3249). Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Adds test coverage to four IO modules verifying that float/double values survive disk round-trips without the 6-digit truncation that default stream precision introduces (ITK issue InsightSoftwareConsortium#3249): * itkNiftiImageIOTest3.cxx: TestNiftiFloatMetadataPrecision() writes an image with 0.12345679-mm spacing, reads back, and verifies the MetaDataDictionary["pixdim[1]"] string parses to the exact float32. * itkMetaImageIOMetaDataTest.cxx: encapsulates hpDouble=1.2345678901234568 and hpFloat=1.2345679f, writes/reads MetaImage, checks exact equality via stod/stof (bypassing the loose 1e-6 tolerance in Equal<double>). * itkNrrdMetaDataTest.cxx: same pattern for native double and float NRRD metadata keys. * itkVoxBoCUBImageIOTest.cxx: creates an image with 0.12345678901234 spacing, writes/reads CUB, asserts GetSpacing()[0] == original double. These tests FAIL without the ConvertNumberToString fixes in PR InsightSoftwareConsortium#6003 and PASS after them (Closes InsightSoftwareConsortium#3249). Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
0890665 to
203f858
Compare
This comment was marked as resolved.
This comment was marked as resolved.
Remove the now-unused joinElements helper from itkMetaImageIO.cxx, as its only call site was replaced with a ConvertNumberToString loop. Add temp file cleanup in the VoxBoCUB precision sub-test. Addresses review comments from @N-Dekker and @greptile-apps on PR InsightSoftwareConsortium#6003. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
My change request (removal of local joinElements function: #6003 (review)) is addressed, thanks Hans!
Adds test coverage to four IO modules verifying that float/double values survive disk round-trips without the 6-digit truncation that default stream precision introduces (ITK issue InsightSoftwareConsortium#3249): * itkNiftiImageIOTest3.cxx: TestNiftiFloatMetadataPrecision() writes an image with 0.12345679-mm spacing, reads back, and verifies the MetaDataDictionary["pixdim[1]"] string parses to the exact float32. * itkMetaImageIOMetaDataTest.cxx: encapsulates hpDouble=1.2345678901234568 and hpFloat=1.2345679f, writes/reads MetaImage, checks exact equality via stod/stof (bypassing the loose 1e-6 tolerance in Equal<double>). * itkNrrdMetaDataTest.cxx: same pattern for native double and float NRRD metadata keys. * itkVoxBoCUBImageIOTest.cxx: creates an image with 0.12345678901234 spacing, writes/reads CUB, asserts GetSpacing()[0] == original double. These tests FAIL without the ConvertNumberToString fixes in PR InsightSoftwareConsortium#6003 and PASS after them (Closes InsightSoftwareConsortium#3249). Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
05e1541 to
f7c9d52
Compare
Adds test coverage to four IO modules verifying that float/double values survive disk round-trips without the 6-digit truncation that default stream precision introduces (ITK issue InsightSoftwareConsortium#3249): * itkNiftiImageIOTest3.cxx: TestNiftiFloatMetadataPrecision() writes an image with 0.12345679-mm spacing, reads back, and verifies the MetaDataDictionary["pixdim[1]"] string parses to the exact float32. * itkMetaImageIOMetaDataTest.cxx: encapsulates hpDouble=1.2345678901234568 and hpFloat=1.2345679f, writes/reads MetaImage, checks exact equality via stod/stof (bypassing the loose 1e-6 tolerance in Equal<double>). * itkNrrdMetaDataTest.cxx: same pattern for native double and float NRRD metadata keys. * itkVoxBoCUBImageIOTest.cxx: creates an image with 0.12345678901234 spacing, writes/reads CUB, asserts GetSpacing()[0] == original double. These tests FAIL without the ConvertNumberToString fixes in PR InsightSoftwareConsortium#6003 and PASS after them (Closes InsightSoftwareConsortium#3249). Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
f7c9d52 to
4bc991c
Compare
This comment was marked as resolved.
This comment was marked as resolved.
Adds test coverage to four IO modules verifying that float/double values survive disk round-trips without the 6-digit truncation that default stream precision introduces (ITK issue InsightSoftwareConsortium#3249): * itkNiftiImageIOTest3.cxx: TestNiftiFloatMetadataPrecision() writes an image with 0.12345679-mm spacing, reads back, and verifies the MetaDataDictionary["pixdim[1]"] string parses to the exact float32. * itkMetaImageIOMetaDataTest.cxx: encapsulates hpDouble=1.2345678901234568 and hpFloat=1.2345679f, writes/reads MetaImage, checks exact equality via stod/stof (bypassing the loose 1e-6 tolerance in Equal<double>). * itkNrrdMetaDataTest.cxx: same pattern for native double and float NRRD metadata keys. * itkVoxBoCUBImageIOTest.cxx: creates an image with 0.12345678901234 spacing, writes/reads CUB, asserts GetSpacing()[0] == original double. These tests FAIL without the ConvertNumberToString fixes in PR InsightSoftwareConsortium#6003 and PASS after them (Closes InsightSoftwareConsortium#3249). Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
4bc991c to
e90b1bb
Compare
This comment was marked as resolved.
This comment was marked as resolved.
Please delay. Today is a holiday here, but I did determine at the end of yesterday that this breaks several of our tests. Likely just slight numerical differences, but I'd like to check early next week. |
N-Dekker
left a comment
There was a problem hiding this comment.
Thanks Hans! I think it's OK now, but you might of course consider my "nitpicks" 😺
WritePoints() is defined inside namespace itk, so the itk:: prefix on ConvertNumberToString is redundant. Suggested by N-Dekker in PR InsightSoftwareConsortium#6003.
This comment was marked as resolved.
This comment was marked as resolved.
seanm
left a comment
There was a problem hiding this comment.
Please delay. Today is a holiday here, but I did determine at the end of yesterday that this breaks several of our tests. Likely just slight numerical differences, but I'd like to check early next week.
AllocateInitialized() is an alias for Allocate(true) introduced in itkImageBase.h to make zero-initialization intent explicit in code. Replace all call sites in Modules/ and Examples/. The trailing "// initialize buffer to zero" comments are also removed since AllocateInitialized() is self-documenting. Suggested by N-Dekker in ITK PR InsightSoftwareConsortium#6003.
Replace the verbose two-line FixedArray initialization pattern: SomeType::SizeType size; size.Fill(N); with the modern single-line form using the static factory method: constexpr auto size = SomeType::SizeType::Filled(N); // test files auto var = VectorType::Filled(0.0); // template .hxx ::Filled(v) is a static method on itk::FixedArray (and all subclasses including SizeType, IndexType, SpacingType, PointType, VectorType, etc.) that returns an instance with all elements set to v. constexpr is used where the variable is never subsequently modified. auto (without constexpr) is used for accumulator variables in loop bodies. Skipped: cases where Fill() is followed by element-wise overrides (size[0] = w; size[1] = h;) or where typename disambiguation in nested template scopes makes the replacement non-trivial. Suggested by N-Dekker in ITK PR InsightSoftwareConsortium#6003.
|
@seanm Understood — I'll hold off on merging. Please take the time you need to investigate; early next week is fine. When you have a chance, it would help to know:
The change from 6 significant digits (default |
|
@hjmjohnson Do I understand correctly that some output files may become much larger, with this PR? Especially when writing data "for all points" (for example, their coordinates)...? |
SetImageIOMetadataFromNIfTI() was using std::ostringstream to convert float fields from the nifti_image struct to strings, which applies the default 6-digit stream precision and silently truncates values. Replace all float-field conversions with itk::ConvertNumberToString(), which uses Google's double-conversion ToShortest() algorithm (via ITKDoubleConversion) to produce the shortest decimal string that round-trips exactly through strtof. Integer and char* fields are unchanged. Fields corrected: intent_p1/p2/p3, pixdim[0-7], scl_slope, scl_inter, cal_max, cal_min, slice_duration, toffset, quatern_b/c/d, qoffset_x/y/z, srow_x/y/z. Fixes: InsightSoftwareConsortium#3249
Non-template MetaData fields of type double, float, vector<double>, and Matrix<double> were serialized via std::ostringstream with default 6-digit precision, silently truncating values with more than 6 significant digits. Replace the lossy stream insertions with itk::ConvertNumberToString(), which wraps Google double-conversion::ToShortest() and produces the shortest decimal string that round-trips exactly through strtod/strtof. Three call sites are affected: - ExposeMetaData<double> / ExposeMetaData<float> in WriteImageInformation() - ExposeMetaData<std::vector<double>> (manual loop replacing joinElements) - WriteMatrixInMetaData<N>() template in the header See: InsightSoftwareConsortium#3249 Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
_dump_metadata_to_stream() used default stream precision (6 significant digits) for float and double values, silently truncating user-defined metadata stored in NRRD key/value fields. Use if constexpr to dispatch float and double types to itk::ConvertNumberToString() for shortest-round-trip serialization, while leaving all other types on the default stream path. Array<double> and Array<float> were already handled losslessly via the specialized operator<< in itkArrayOutputSpecialization.cxx. See: InsightSoftwareConsortium#3249 Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Two persistence-layer paths used lossy default stream precision for floating-point coordinates: itkPolygonGroupSpatialObjectXMLFile.cxx: PolygonSpatialObject vertex coordinates (Point<double,3>) were streamed to the XML file with default 6-digit precision. Replace with itk::ConvertNumberToString() per component. itkFreeSurferAsciiMeshIO.h WritePoints<T>: hardcoded precision(6) with std::fixed limited output to 6 decimal places regardless of T. Replace with itk::ConvertNumberToString() for both float and double specializations, producing the shortest decimal string that round-trips exactly. See: InsightSoftwareConsortium#3249 Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
m_Spacing[] values (double) were written to the CUB text header with default stream precision (6 significant digits), silently truncating sub-millimeter or high-precision spacing values. Replace with itk::ConvertNumberToString() for shortest-round-trip serialization. See: InsightSoftwareConsortium#3249 Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Adds test coverage to four IO modules verifying that float/double values survive disk round-trips without the 6-digit truncation that default stream precision introduces (ITK issue InsightSoftwareConsortium#3249): * itkNiftiImageIOTest3.cxx: TestNiftiFloatMetadataPrecision() writes an image with 0.12345679-mm spacing, reads back, and verifies the MetaDataDictionary["pixdim[1]"] string parses to the exact float32. * itkMetaImageIOMetaDataTest.cxx: encapsulates hpDouble=1.2345678901234568 and hpFloat=1.2345679f, writes/reads MetaImage, checks exact equality via stod/stof (bypassing the loose 1e-6 tolerance in Equal<double>). * itkNrrdMetaDataTest.cxx: same pattern for native double and float NRRD metadata keys. * itkVoxBoCUBImageIOTest.cxx: creates an image with 0.12345678901234 spacing, writes/reads CUB, asserts GetSpacing()[0] == original double. These tests FAIL without the ConvertNumberToString fixes in PR InsightSoftwareConsortium#6003 and PASS after them (Closes InsightSoftwareConsortium#3249). Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…uide Several IO modules (NIfTI, MetaImage, NRRD, VoxBoCUB, SpatialObject, Mesh) now use itk::ConvertNumberToString() for float/double metadata, replacing the 6-significant-digit default. Tests comparing raw header text against golden baselines may need updating. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
998aaec to
cd9ec88
Compare
|
@seanm Do you need any assistance with this task? |
nifti. I don't use the others.
tolerance checks. I just need to confirm the results are not wildly different... should know today... |
Summary
Fixes float/double precision loss in ITK persistence-layer serialization (ITK issue #3249).
When ITK IO modules write numeric metadata to text-based format headers using default
std::ostreamprecision (6 significant digits), values like1.2345678901234568are truncated to"1.23457", which parses back to a different bit pattern. This PR appliesitk::ConvertNumberToString()(shortest exact round-trip string via Google double-conversion) consistently across 6 IO modules.Files Fixed
itkNiftiImageIO.cxxpixdim[],intent_p1/2/3,scl_slope/inter,cal_max/min,slice_duration→ MetaDataDictionary stringsitkMetaImageIO.cxx+.hvector<double>loop,WriteMatrixInMetaData<>templateitkNrrdImageIO.cxxfloat/doublespecializations of_dump_metadata_to_stream<T>itkPolygonGroupSpatialObjectXMLFile.cxxPoint<double,3>)itkFreeSurferAsciiMeshIO.hWritePoints<T>loop templateitkVoxBoCUBImageIO.cxxm_Spacing[0/1/2]in CUB text headerFiles Verified Correct (No Fix Needed)
itkNrrdImageIO.cxx—Array<double/float>already usesConvertNumberToStringviaitkArrayOutputSpecialization.cxxitkVoxBoCUBImageIO.cxxlines 682-684 — origin deliberately cast toint(CUB stores integer voxel coordinates)Test Coverage
Adds a new commit (
ENH: Add precision round-trip tests) with 4 precision regression tests:itkNiftiImageIOTest3.cxx):TestNiftiFloatMetadataPrecision()— writes image with0.12345679-mm spacing, reads back, verifiesMetaDataDictionary["pixdim[1]"]parses to the exact float32 bit patternitkMetaImageIOMetaDataTest.cxx): encapsulateshpDouble=1.2345678901234568andhpFloat=1.2345679f, writes/reads.mha, checks exact equality viastod/stof(bypasses the existing loose1e-6tolerance)itkNrrdMetaDataTest.cxx): same pattern for nativedoubleandfloatNRRD metadata keysitkVoxBoCUBImageIOTest.cxx): creates image with0.123456789012345spacing, writes/reads CUB, assertsGetSpacing()[0] == original doubleAll 4 tests FAIL against the unfixed IO code and PASS with this PR applied.
Root Cause
Default
std::ostreamprecision is 6 significant digits. Float needs ≥8 and double needs ≥17 for lossless serialization.itk::ConvertNumberToString()(wrappingdouble-conversion::ToShortest()) produces the shortest decimal string that round-trips exactly.Closes #3249
Test Plan
itkNiftiVectorImageTest— includesTestNiftiFloatMetadataPrecisionitkMetaImageIOMetaDataTestitkNrrdMetaDataTestitkVoxBoCUBImageIOTest1