Skip to content

Memory not released by generate_stitch.py #3

@amcmahon01

Description

@amcmahon01

During long runs, generate_stitch.py appears to be holding objects in memory after they've been processed.

Possible "band-aid" solution: close and recreate multiprocessing pool for every day of processing. This wouldn't solve the root cause but would at least limit the accumulated objects. For reference, 5 days of processing has left most processes in the pool with ~5GB of memory usage (each). With a pool size of 20, this is basically utilizing all available memory.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions