Skip to content

ROCrate.delete() only removes entity from metadata graph - physical files in crate directory are not cleaned up #253

Description

@NMEpping

ROCrate.delete() removes an entity from the in-memory metadata graph, so it no longer appears in ro-crate-metadata.json after a subsequent write(). However, if the corresponding physical file has already been written to (or copied into) the crate directory, it is not deleted from disk. This leaves orphaned files in the RO-Crate that are no longer described by metadata - violating the principle that the metadata file is the authoritative manifest of the crate's contents.

Example / How to Reproduce

from rocrate.rocrate import ROCrate

crate = ROCrate()

# Add a file entity
crate.add_file("path/to/local/data.csv", dest_path="data/data.csv")

# Write crate - file is physically copied into crate directory
crate.write("my_crate")  # my_crate/data/data.csv now exists on disk

# Realise the file was added by mistake - delete it
crate.delete(entity)

# Write again
crate.write("my_crate")

Expected: my_crate/data/data.csv is removed from disk and from ro-crate-metadata.json.

Actual: my_crate/data/data.csv remains on disk. Only the metadata entry is removed.

Looking at the source:

  • ROCrate.delete() calls self.__entity_map.pop() and removes references from the root dataset's hasPart - this correctly cleans up the in-memory graph.
  • ROCrate.write() iterates over current data_entities and calls entity.write(base_path) for each - so deleted entities are correctly not written on a fresh export.
  • But there is no logic to remove files from base_path that correspond to entities that were previously written but have since been deleted.

Proposal

It would be helpful to have one (or both) of:

  1. ROCrate.delete(entity, remove_file=False) - an optional flag that, when True, also deletes the physical file at os.path.join(base_path, entity.id) if it exists. Default False for backward compatibility.

  2. ROCrate.write() cleanup pass - after writing all current entities, optionally remove files in the output directory that are not described by any entity in the crate. This could be gated by a cleanup_orphans=False parameter to avoid surprising behaviour.

Use Case

Users may add a file by mistake, then need to remove it before final export. Currently there is no clean way to do this - the user must manually os.remove() the file from the crate directory after calling delete(), which is error-prone and breaks the abstraction.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions