Skip to content

CLI: Batch place scraping should return non-zero exit code on failures #54

Description

@MozzamShahid

File / lines: src/gmaps_scraper/cli.py, lines 342-350

Problem: When --kind place is used in batch mode, scrape_places() can return PlaceScrapeResult objects whose error field is set, but main() always prints the wrapped JSON and returns 0. The test test_place_kind_can_scrape_batch_from_input_file in tests/test_cli.py even encodes this behavior: it asserts exit_code == 0 even though a mocked result has "error": "blocked".

Why it matters: In CI/automation, the scraper reports success when every URL failed. Callers must parse JSON to detect failures.

Suggested fix:

  1. Add --fail-on-error flag.
  2. When set, after emitting the JSON payload, main() returns a non-zero exit code if any result has error is not None.
  3. Keep the default behavior unchanged to avoid breaking existing scripts, or change the default with a major version note.
  4. Add/update tests asserting stderr content and exit code.

Difficulty: good-first-issue / medium
Impact: High — pipeline-friendly observability improvement.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions