MiTeddy Bot Testing Framework

Version: 1.0.0 Created: 2025-11-14 Status: Production Ready

Overview

Comprehensive testing and simulation framework for the MiTeddy mint bot. Enables rapid, thorough testing of all bot lifecycle phases and failure modes without waiting for real mint windows.

Key Features

✅ Time Compression: Test 5-minute phases in 30 seconds (10x-1000x compression) ✅ 22 Failure Scenarios: Predefined tests for all known failure modes ✅ Phase Simulation: Test phases independently or in sequence ✅ Failure Injection: Trigger specific failures on demand ✅ State Validation: Verify bot state at phase boundaries ✅ Performance Profiling: Track memory, CPU, event loop lag ✅ Coverage Tracking: Monitor which failure modes have been tested ✅ Multiple Report Formats: Console, JSON, HTML, Markdown

Quick Start

# Run initialization test
npm run test:bot -- --test-mode=init

# Run full lifecycle with 10x time compression
npm run test:bot -- --test-mode=full --time-compression=10

# Run all critical failure scenarios
npm run test:critical

# Run with failure injection
npm run test:bot -- --test-mode=spam --inject-failures='{"nonceErrors":[2,5]}'

# Run with profiling and validation
npm run test:bot -- --test-mode=full --profile --validate-state

Why This Framework Exists

The Problem

Before this framework:

100% production failure rate in recent mint attempts
Bugs only discovered after hours of operation
No way to test without waiting for real mint windows
Critical issues (nonce errors, gas bump failures, timing loops) only appeared in production
High risk of regression when making changes

The Solution

With this framework:

✅ Rapid testing: Complete lifecycle test in < 5 minutes (vs hours)
✅ Early detection: Catch failures in development, not production
✅ Comprehensive coverage: Test all 22 failure modes systematically
✅ Failure reproduction: Reproduce known bugs on demand
✅ Stress validation: Validate behavior under extreme conditions
✅ Easy debugging: Clear diagnostics for failure analysis

Architecture

Module Structure

src/testing/
├── core/                       # Core testing capabilities
│   ├── timeCompression.ts      # Time manipulation
│   ├── failureInjector.ts      # Failure injection
│   ├── stateValidator.ts       # State validation
│   ├── phaseSimulator.ts       # Phase simulation
│   ├── stressTester.ts         # Stress testing
│   └── testReporter.ts         # Diagnostics & reporting
├── dimensions/                 # Additional capabilities
│   ├── performanceProfiler.ts  # Resource monitoring
│   └── coverageTracker.ts      # Coverage tracking
├── scenarios/                  # Predefined test scenarios
│   └── failureScenarios.ts     # All 22 failure modes
├── types/                      # TypeScript type definitions
│   ├── testConfig.ts
│   ├── testResults.ts
│   └── injectionPoints.ts
├── utils/                      # Utility functions
│   └── helpers.ts
├── TestFramework.ts            # Main framework class
└── index.ts                    # Public API

Core Dimensions

Time Compression: Compress long-running phases (300s → 3s)
Failure Injection: Inject any of 22 failure modes at precise moments
State Validation: Verify bot state matches expectations
Phase Simulation: Test phases independently or in sequence
Stress Testing: Simulate high-load scenarios
Test Reporting: Generate comprehensive diagnostics

Additional Dimensions

Performance Profiling: Track memory, CPU, event loop lag
Coverage Tracking: Monitor test coverage of failure modes
Network Replay: Record/replay network conditions (future)
Chaos Engineering: Random failure injection (future)
Regression Testing: Compare against baselines (future)

Failure Scenarios

All 22 Failure Modes

The framework includes tests for all documented failure modes:

CRITICAL (6 scenarios)

Nonce Management Failure - Nonce too low errors during broadcast
Gas Bump Logic Broken - Replacement transactions underpriced
Simulation Gate Blocking - Simulation blocks spam mode
All RPCs Fail - Network outage during broadcast
RPC Broadcast Failure - Individual RPC failures
Process Crash - Uncaught exceptions

HIGH Priority (4 scenarios)

Timing Loop - 920 recalculations in 4 minutes
Window Detection Miss - Sub-second window missed
Late Firing - Bot fires too late (T+4s instead of T-42s)
Transaction Reverted - Transaction included but reverted

MEDIUM Priority (10 scenarios)

Nonce Conflict - Race condition with other transactions
Pre-Signed Invalidation - Pre-signed TX nonce mismatch
Gas Estimation Failure - All RPCs fail gas estimation
Timing Drift - Clock drift causes wrong firing time
RPC Health Degradation - Too many RPCs blacklisted
TX Stuck in Mempool - Transaction never included
Receipt Verification Failure - Receipt polling timeout
Pre-Warm TX Failure - Pre-warm transactions fail
Emergency Activation Failure - Emergency mode doesn't activate
Emergency Override Failure - Emergency overrides don't fix issue

LOW Priority (2 scenarios)

State File Corruption - Corrupted state file
Pre-Warm Timer Cleanup Failure - Memory leak from timers

Usage Examples

1. Basic Test

npm run test:bot -- --test-mode=full --time-compression=10

2. Test Specific Failure

npm run test:bot -- --scenario=nonce_management_failure

3. Inject Multiple Failures

npm run test:bot -- --test-mode=spam \
  --inject-failures='{"nonceErrors":[2,5],"gasBumpFailures":[3]}'

4. Comprehensive Test

npm run test:bot -- \
  --test-mode=full \
  --time-compression=10 \
  --validate-state \
  --profile \
  --track-coverage \
  --report=console,json,html \
  --verbose

5. Programmatic Usage

import { TestFramework, FAILURE_SCENARIOS } from './testing'

const framework = new TestFramework({
  mode: 'full',
  timeCompression: { global: 10 },
  validation: { enabled: true, checkpoints: ['spam'] },
})

await framework.initialize()
await framework.runScenario(FAILURE_SCENARIOS.NONCE_MANAGEMENT_FAILURE)
const coverage = framework.getCoverageReport()
await framework.shutdown()

Reports

Console Report

Real-time colored output with emojis:

🎯 Running scenario: Nonce Management Failure
✓ [+2.45s] Validation spam: PASSED
❌ [+3.12s] Failure injected: nonce_too_low
✅ [+3.25s] Failure recovered: nonce_too_low

Test Status: PASSED ✅
Duration: 8.4s (compressed from 84s)

JSON Report

Structured data for programmatic analysis:

{
  "testId": "test-1731600000000",
  "success": true,
  "duration": 8400,
  "injectedFailures": [...],
  "validationResults": [...],
  "metrics": {...}
}

HTML Report

Visual report with charts and tables (see test-reports/)

Coverage Report

Coverage: 27.3% (6/22 scenarios)

Tested:
  ✅ nonce_management_failure (3 tests)
  ✅ gas_bump_broken (2 tests)
  ✅ simulation_gate_blocking (1 test)

Untested:
  ❌ timing_loop
  ❌ window_detection_miss
  ...

Best Practices

1. Test Critical Scenarios First

npm run test:critical

2. Use Appropriate Time Compression

10x: Most tests, good balance of speed and accuracy
100x: Fast pre-warming tests
1000x: Quick smoke tests only

3. Enable Validation in CI

npm run test:bot -- --test-mode=full --validate-state --strict

4. Track Coverage Over Time

npm run test:bot -- --track-coverage --report=json

5. Profile Performance Regularly

npm run test:bot -- --test-mode=full --profile

Integration with CI/CD

GitHub Actions Example

name: Bot Tests

on: [push, pull_request]

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v2
      - uses: actions/setup-node@v2
      - run: npm install
      - run: npm run build
      - run: npm run test:critical
      - run: npm run test:high
      - uses: actions/upload-artifact@v2
        with:
          name: test-reports
          path: test-reports/

Documentation

Usage Guide: Comprehensive usage instructions
Architecture: Detailed architecture design
API Reference: TypeScript API documentation

Success Metrics

The framework enables:

✅ Rapid Testing: Complete lifecycle test in < 5 minutes (vs hours)
✅ Early Detection: Catch failures in development, not production
✅ Comprehensive Coverage: Test all 22 failure modes
✅ Failure Reproduction: Reproduce known bugs on demand
✅ Stress Validation: Validate behavior under stress
✅ Easy Debugging: Clear diagnostics for failure analysis
✅ Backward Compatible: Existing functionality unchanged
✅ Maintainable: Easy to extend with new test modes

Future Enhancements

Planned features:

Network Replay: Record and replay real network conditions
Chaos Engineering: Randomized failure injection
Multi-Run Statistics: Detect flakiness with statistical analysis
Mempool Simulation: Simulate transaction pool state
Block Controller: Control block mining for precise timing
State Visualizer: Visual phase transition diagrams
Comparative Analysis: Compare different configurations
Trend Analysis: Track test metrics over time

Contributing

To add new test scenarios:

Define failure type in types/testConfig.ts
Add injection point in types/injectionPoints.ts
Create scenario in scenarios/failureScenarios.ts
Add integration points in src/index.ts
Document in usage guide

License

Same as MiTeddy Bot project (MIT)

Changelog

v1.0.0 (2025-11-14)

Initial release
22 failure scenarios
Time compression system
Failure injection framework
State validation
Performance profiling
Coverage tracking
Multiple report formats

FilesExpand file tree

TESTING_README.md

Latest commit

History