Skip to content

InvertedIndexVectorized._set_logger shares a process-wide logger across all dataset instances #89

Description

@heathersherry

Problem

InvertedIndexVectorized._set_logger (in muller/core/query/inverted_index_vectorized.py) does:

logger = logging.getLogger('my_logger')
...
if not logger.handlers:
    file_handler = logging.FileHandler(path, mode='a')
    ...

Because Python's logging.getLogger returns a process-wide singleton keyed by name, and the if not logger.handlers guard short-circuits on subsequent calls, the FileHandler bound during the FIRST InvertedIndexVectorized(...) construction is reused for the entire process. All later instances—even those for a different dataset path—log to that first file.

Why it matters

  • Cross-dataset log mixing in any workflow that opens more than one dataset (e.g. tests, batch jobs, notebooks that load several datasets).
  • The "remote path → console-only" branch added in Phase 1 only takes effect if the remote dataset is the FIRST one constructed in the process; otherwise it silently inherits a stale local FileHandler.
  • The logger name 'my_logger' is generic enough to collide with any other library that uses the same name.

Suggested fix

Use a per-instance child logger keyed by dataset path / column, e.g.:

name = f"muller.inverted_index.{_safe(self.dataset.path)}.{self.column_name}"
logger = logging.getLogger(name)

Or accept that this is best-effort logging and just use the standard module-level logger (logging.getLogger(__name__)) with NullHandler, letting application code configure handlers. The FileHandler at a dataset-derived path is the surprising part; consider dropping it entirely if the same information is available via the module logger.

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Fields

No fields configured for issues without a type.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions