Skip to content

[AI] Fix raw denoise output corrupted DNG / process crash on some libtiff builds#20898

Merged
TurboGit merged 2 commits intodarktable-org:masterfrom
andriiryzhkov:dng_tag_fix
Apr 29, 2026
Merged

[AI] Fix raw denoise output corrupted DNG / process crash on some libtiff builds#20898
TurboGit merged 2 commits intodarktable-org:masterfrom
andriiryzhkov:dng_tag_fix

Conversation

@andriiryzhkov
Copy link
Copy Markdown
Contributor

Fixes two symptoms reported in #20854 (comment):

  • Re-imported denoised DNG flagged as corrupt by rawspeed (Entry 0x828d not found)
  • Hard process crash (ThrowLoggedException assertion in ImageMagick) on some images, in both CPU and MIGraphX modes

Both trace to libtiff CFA / DNG extension tag info handling on the SubIFD in the canonical Adobe layout (IFD0 = JPEG preview, SubIFD0 = raw payload) introduced in #20854.

Fixes

1. Re-initialize SubIFD tag info. After TIFFWriteDirectory(IFD0), some libtiff packagings drop the registered field info for CFA / DNG extension tags. The next TIFFSetField(TIFFTAG_CFAREPEATPATTERNDIM, ...) then silently fails - producing a DNG without the CFA pattern that rawspeed can't decode. One-line fix: call TIFFCreateDirectory(tif) between IFD0 commit and SubIFD population. Applied in both dt_imageio_dng_write_cfa_bayer and dt_imageio_dng_write_linear. No-op on libtiff builds that already retained the tag info.

2. Install our own libtiff error/warning handlers. When the unknown-tag error fires on systems where ImageMagick has installed itself as the global libtiff error handler, ImageMagick's TIFFReadErrors calls ThrowLoggedException(NULL, ...) and aborts. Same pattern as imageio_tiff.c already uses: handlers log to stderr and never abort, overriding whatever else is in the process.

@TurboGit TurboGit added this to the 5.6 milestone Apr 29, 2026
@TurboGit TurboGit added bugfix pull request fixing a bug priority: high core features are broken and not usable at all, software crashes scope: image processing correcting pixels scope: software support wiring external libs and software: LittleCMS, colord, G'MIC, enfuse/enblend, etc. labels Apr 29, 2026
Copy link
Copy Markdown
Member

@TurboGit TurboGit left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me, thanks!

@TurboGit TurboGit merged commit 9f3e757 into darktable-org:master Apr 29, 2026
5 checks passed
@da-phil
Copy link
Copy Markdown
Contributor

da-phil commented Apr 29, 2026

@andriiryzhkov unfortunately this didn't fix the issue, this is the current output:

2026-04-30 00:15:49.328086631 [W:onnxruntime:DarktableAI, migraphx_execution_provider.cc:167 MIGraphXExecutionProvider] [MIGraphX EP] MIGraphX ENV Override Variables Set:
    70.7493 [neural_restore] raw preview: full=5240x3912 ori=0x0 patch_center=(0.500,0.500) -> sensor=(2244,1678 752x556) bayer
    72.5724 [neural_restore] raw preview: inference returned err=0 src=0x76c2b8a8d010 denoised=0x76c2b6373010 requested=752x556 actual=719x532
    73.7113 [neural_restore] job started: task=raw denoise, scale=1, images=1
    73.7114 [neural_restore] processing imgid 31758 -> /home/phil/Pictures/20170606_072239_raw-denoise.dng
    73.7114 [neural_restore] imgid 31758: flags=0x643 channels=1 filters=0x94949494 (bayer)
    73.7668 [restore_raw_bayer] 5240x3912 sensor (CFA origin 0,0), working 2620x1956 packed, tile T=2048, 2x1 grid (2 tiles)
    73.7704 [restore_raw_bayer] raw CFA range [258.0, 3545.0], black=[257,257,257,257] white=4003 wb_coeffs=[492.000,256.000,484.000,0.000] wb_norm=[1.000,1.000,1.000]
    73.8321 [restore_raw_bayer] tile0 model_input range R=[0.000,0.293] G1=[0.001,0.545] G2=[0.001,0.545] B=[0.000,0.273]
    74.8804 [restore_raw_bayer] tile0 model_output range R=[0.001,0.335] G=[0.002,0.647] B=[0.001,0.346] in_mean=0.074 out_mean=-60034606.923 gain=-1.237e-09
    75.9218 [restore_raw_bayer] cfa_out u16 range [260, 3882] mean=677 (DNG will advertise black~257 white=4003)
    75.9415 [neural_restore] embedded JPEG preview from source 3200x2400 (1000782 bytes)
    75.9578 [imageio_dng] error: TIFFSetField: /home/phil/Pictures/20170606_072239_raw-denoise.dng: Unknown tag 33421
    75.9578 [imageio_dng] error: TIFFSetField: /home/phil/Pictures/20170606_072239_raw-denoise.dng: Unknown tag 33422
    76.4479 [neural_restore] imported imgid=63602: /home/phil/Pictures/20170606_072239_raw-denoise.dng
   278.3986 [rawspeed] 20170606_072239_raw-denoise.dng corrupt: rawspeed::RawImage rawspeed::RawDecoder::decodeRaw(), line 334: rawspeed::TiffEntry* rawspeed::TiffIFD::getEntry(rawspeed::TiffTag) const, line 316: Entry 0x828d not found.

As I mentioned I am running libtiff-4 in version 4.5.1, which one are you using on your machine?

@andriiryzhkov
Copy link
Copy Markdown
Contributor Author

Do you have the same error on nightly build?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bugfix pull request fixing a bug priority: high core features are broken and not usable at all, software crashes scope: image processing correcting pixels scope: software support wiring external libs and software: LittleCMS, colord, G'MIC, enfuse/enblend, etc.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants