Skip to content

Add structured logging, config validation, thread-safe cache, and Selenium improvements#198

Open
natiixnt wants to merge 2 commits into
FujiwaraChoki:mainfrom
natiixnt:improve/logging-validation-cache-selenium
Open

Add structured logging, config validation, thread-safe cache, and Selenium improvements#198
natiixnt wants to merge 2 commits into
FujiwaraChoki:mainfrom
natiixnt:improve/logging-validation-cache-selenium

Conversation

@natiixnt
Copy link
Copy Markdown

Summary

  • Structured logging: Replace status.py print/termcolor output with Python's logging module. Logs rotate at 5MB with 3 backups in .mp/mpv2.log. Console output remains colored for backwards compatibility. Exposes get_logger() for modules that need direct logger access.
  • Config validation: New validate_config() function with schema-based checking -- verifies required keys exist, types match, email sub-structure is complete, imagemagick_path exists on disk, and stt_provider is a known value. Runs automatically at startup in main.py and prints all issues as warnings so users can fix their config before hitting cryptic errors later.
  • Thread-safe cache: All JSON cache reads now use fcntl.LOCK_SH (shared locks), all writes use atomic temp-file + os.replace() with fcntl.LOCK_EX (exclusive locks). This prevents cache corruption when multiple CRON jobs run concurrently against the same .mp/*.json files.
  • Selenium improvements: Replace hardcoded time.sleep() waits in YouTube upload_video() with WebDriverWait + explicit expected_conditions (presence/clickability). Adds automatic screenshot capture on upload failure for debugging. Adds helper methods _wait_and_find() and _wait_and_click().
  • Cleanup: Replace stray print(colored(...)) calls in YouTube.py and Outreach.py with proper info()/error()/success() functions.
  • Tests: 33 new tests covering config validation, atomic cache writes, file locking, and structured logging.

Test plan

  • All 33 tests pass locally (python -m pytest tests/ -v)
  • Manual smoke test of python src/main.py to verify config validation prints warnings for missing/invalid values
  • Verify .mp/mpv2.log is created after first run

…enium improvements

- Replace status.py print/termcolor with Python logging module backed by
  RotatingFileHandler (5MB, 3 backups) writing to .mp/mpv2.log, while
  keeping colored console output for backwards compatibility
- Add validate_config() with schema-based checking: required keys, types,
  email sub-structure, imagemagick_path existence, stt_provider values.
  Runs at startup in main.py and reports all issues before proceeding
- Make cache.py thread-safe: all reads use fcntl.LOCK_SH, all writes use
  atomic temp-file + os.replace() with fcntl.LOCK_EX to prevent corruption
  when multiple CRON jobs run concurrently
- Replace hardcoded time.sleep() waits in YouTube upload_video() with
  WebDriverWait + explicit expected_conditions for element presence/clickability
- Add screenshot capture on upload failure for debugging
- Replace stray print(colored()) calls in YouTube.py and Outreach.py
  with proper status functions (info/error/success)
- Add 33 tests covering config validation, atomic cache writes, file
  locking, and structured logging
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant