Skip to content

Enable local testing for all cachier backends with Docker#287

Merged
shaypal5 merged 17 commits intomasterfrom
local-all-cores-tests
Jul 14, 2025
Merged

Enable local testing for all cachier backends with Docker#287
shaypal5 merged 17 commits intomasterfrom
local-all-cores-tests

Conversation

@shaypal5
Copy link
Copy Markdown
Member

Description

This PR extends the MongoDB local testing solution (PR #285) to support all cachier backends (MongoDB, Redis, SQL, Memory, Pickle). It provides a unified interface for developers to test any combination of backends locally with Docker containers.

What's New

1. Unified Test Script (scripts/test-local.sh)

A comprehensive script that replaces the MongoDB-specific script with support for all backends:

  • Flexible core selection: test individual backends, groups, or all
  • Automatic dependency checking and installation
  • Smart Docker container lifecycle management
  • Support for unmarked general tests when running "all"

Usage examples:

# Test single backend
./scripts/test-local.sh mongo
./scripts/test-local.sh redis

# Test multiple backends
./scripts/test-local.sh mongo redis sql

# Test all backends (includes unmarked tests)
./scripts/test-local.sh all

# Test groups
./scripts/test-local.sh external  # mongo, redis, sql
./scripts/test-local.sh local     # memory, pickle

2. All-Cores Docker Compose (scripts/docker-compose.all-cores.yml)

Comprehensive docker-compose configuration with all external services:

  • MongoDB (port 27017)
  • Redis (port 6379)
  • PostgreSQL (port 5432)
  • Health checks for all services
  • Unified network configuration

3. Enhanced Makefile Targets

New make commands for convenience:

make test-local CORES="mongo redis"  # Test specific cores
make test-all-local                   # Test all backends
make test-external                    # Test external backends only
make test-redis-local                 # Test Redis only
make test-sql-local                   # Test SQL only
make services-start                   # Start all containers
make services-stop                    # Stop all containers

4. Backend-Specific Improvements

  • Separated MongoDB dependencies into tests/mongodb_requirements.txt
  • Fixed SQL test import isolation issue
  • Support for multiple pytest markers per core (e.g., pickle uses "pickle or maxage")
  • Automatic installation of backend-specific dependencies

5. Documentation Updates

  • Comprehensive guide in scripts/README-local-testing.md
  • Updated CLAUDE.md with new testing commands
  • Updated CI workflow to use mongodb_requirements.txt

Key Features

  1. Unified Interface: Single script handles all backends
  2. Smart Dependencies: Auto-installs only required dependencies
  3. Flexible Selection: Test any combination of backends
  4. Portable: Works with older bash versions (no associative arrays)
  5. Complete Coverage: Runs all 144 tests when using "all" option

Breaking Changes

  • Removed scripts/test-mongo-local.sh (replaced by unified script)
  • Removed scripts/docker-compose.mongodb.yml (replaced by all-cores version)

Testing

  • Tested locally on macOS with all backends
  • All 144 tests pass with ./scripts/test-local.sh all
  • Handles missing Docker gracefully
  • Proper cleanup even on test failure

Future Enhancements

  • Add support for additional backends (MySQL, Elasticsearch)
  • Parallel test execution for multiple backends
  • Performance benchmarking mode

@shaypal5 shaypal5 force-pushed the local-all-cores-tests branch from 1373907 to b6071c0 Compare July 14, 2025 14:34
@shaypal5 shaypal5 merged commit c891fc5 into master Jul 14, 2025
42 checks passed
@shaypal5 shaypal5 deleted the local-all-cores-tests branch July 14, 2025 18:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants