Skip to content

Investigation: Automatic reset_parent for cross-backend HDF5→Zarr exports requires HDMF core changes#311

Draft
rly with Copilot wants to merge 3 commits into
devfrom
copilot/automate-reset-parent-hdf5
Draft

Investigation: Automatic reset_parent for cross-backend HDF5→Zarr exports requires HDMF core changes#311
rly with Copilot wants to merge 3 commits into
devfrom
copilot/automate-reset-parent-hdf5

Conversation

Copilot AI commented Dec 13, 2025

Copy link
Copy Markdown
Contributor

Motivation

Users adding HDF5 objects from one file to another and exporting to Zarr encounter cross-backend linking failures. The requested automatic reset_parent() calls cannot be implemented in hdmf-zarr due to fundamental HDMF design constraints.

Investigation Findings

Core Issue: HDMF's container_source attribute is immutable once set. When objects from HDF5 file B are added to file A and exported to Zarr, they retain container_source pointing to file B. HDMF's ObjectMapper detects this and attempts cross-backend links, which fail.

Why Automatic Reset Fails:

  • reset_parent() only clears parent, not container_source
  • HDMF prevents container_source reassignment to avoid data corruption
  • Calling reset_parent() during export creates orphaned containers that fail validation

Required Solution: This needs HDMF core changes to either:

  1. Recognize cross-backend scenarios when export=True and treat foreign containers as new objects
  2. Add reset_for_export() API that can clear both parent and container_source

How to test the behavior?

The issue cannot be resolved in hdmf-zarr. See INVESTIGATION_RESET_PARENT.md for detailed analysis including attempted solutions and code examples demonstrating the limitations.

Checklist

  • Did you update CHANGELOG.md with your changes?
  • Does the PR clearly describe the problem and the solution?
  • Have you reviewed our Contributing Guide?
  • Does the PR use "Fix #XXX" notation to tell GitHub to close the relevant issue numbered XXX when the PR is merged?
Original prompt

This section details on the original issue you should resolve

<issue_title>[Feature]: When adding an HDF5 object and exporting across backends, reset_parent should be called automatically</issue_title>
<issue_description>### What would you like to see added to HDMF-ZARR?

See NeurodataWithoutBorders/helpdesk#94

Use case:

  1. User has HDF5 file A and reads it with NWBHDF5IO.
  2. User has HDF5 file B and reads it with NWBHDF5IO.
  3. User adds some objects X from NWBFile B to NWBFile A.
  4. User exports modified NWBFile A to a Zarr file.

If reset_parent is not called, then Zarr will attempt to create a link to the objects X because they already exist (they have a parent and container source) instead of copying the objects to Zarr.

Is your feature request related to a problem?

No response

What solution would you like?

It would be much more usable if reset_parent was automatically called on any HDF5 objects in the NWBFile when writing/exporting to Zarr because they cannot be linked to.

Do you have any interest in helping implement the feature?

Yes.</issue_description>

Comments on the Issue (you are @copilot in this section)


💬 We'd love your input! Share your thoughts on Copilot coding agent in our 2 minute survey.

Copilot AI and others added 2 commits December 13, 2025 18:47
Co-authored-by: rly <310197+rly@users.noreply.github.com>
Co-authored-by: rly <310197+rly@users.noreply.github.com>
Copilot AI changed the title [WIP] Add automatic reset_parent call for HDF5 export Investigation: Automatic reset_parent for cross-backend HDF5→Zarr exports requires HDMF core changes Dec 13, 2025
Copilot AI requested a review from rly December 13, 2025 19:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Feature]: When adding an HDF5 object and exporting across backends, reset_parent should be called automatically

2 participants