Skip to content

Latest commit

 

History

History
480 lines (344 loc) · 13.3 KB

File metadata and controls

480 lines (344 loc) · 13.3 KB

🚀 VideoAnnotator v1.5.0 Development Roadmap

Release Overview

VideoAnnotator v1.5.0 is the Feature Enhancement Release - bringing all the wishlist items that were intentionally deferred from v1.4.0 to keep the JOSS release focused. This release adds usability improvements, advanced features, and quality enhancements that make VideoAnnotator more powerful and easier to use.

Target Release: Q3 2026 (3-4 months after v1.4.0) Current Status: Planning Phase Main Goal: Enhanced usability and advanced capabilities Duration: 12-14 weeks

Prerequisites: v1.4.0 delivered JOSS publication foundation:

  • ✅ JOSS paper submitted/accepted
  • ✅ PyPI package published
  • ✅ 4 research workflow examples
  • ✅ Docker images (CPU/GPU)
  • ✅ Core documentation complete
  • ✅ Multi-platform testing

🎯 Core Principles

This release focuses on enhancing the user experience and adding advanced capabilities:

  • Usability First - Progress indicators, wizards, intuitive workflows
  • Quality & Performance - Assessment pipelines, batch optimization
  • Integration - FiftyOne, Label Studio, flexible exports
  • Developer Experience - Better logging, debugging, analysis tools
  • Advanced Features - Quality metrics, comparison tools, smart scheduling

Still OUT OF SCOPE (deferred to v1.6.0+):

  • ❌ Enterprise features (SSO, RBAC, multi-tenancy)
  • ❌ Plugin system architecture
  • ❌ Real-time streaming
  • ❌ GraphQL API
  • ❌ Cloud provider integration
  • ❌ Microservice architecture

📋 Feature Categories

All items in v1.5.0 were deferred from v1.4.0 to maintain JOSS focus. They are organized into logical feature groups:

Category A: Installation & Setup

  • Model auto-download with progress
  • Setup wizard for first-run
  • Enhanced health metrics

Category B: Progress & Feedback

  • Real-time progress indicators
  • Resource usage monitoring
  • Job notifications

Category C: Configuration

  • Interactive config wizard
  • Config templates library
  • YAML validation improvements

Category D: Logging & Debugging

  • Structured logging (JSON)
  • Log analysis tools
  • Enhanced version info

Category E: Export & Integration

  • FiftyOne integration
  • Label Studio integration
  • Custom CSV templates

Category F: Quality & Performance

  • Quality assessment pipeline
  • Batch processing optimization
  • Pipeline comparison tools

📋 v1.5.0 Deliverables

Phase 1: Installation & Setup Improvements (Weeks 1-2)

1.1 Model Auto-Download System

Problem: Users must manually download large model files before first use.

Solution:

  • Automatic model download on first use (not during pip install)
  • Progress bar with download speed and ETA
  • Configurable cache directory (VIDEOANNOTATOR_MODEL_CACHE)
  • Offline mode (use cached models only)
  • Manual download script for air-gapped systems

Deliverables:

  • src/videoannotator/models/downloader.py - Auto-download logic
  • Progress indicators using tqdm
  • Offline mode flag and detection
  • Documentation in docs/installation/models.md

Effort: 16 hours

1.2 First-Run Setup Wizard

Problem: New users don't know what to configure.

Solution:

  • videoannotator setup command for first-run configuration
  • Interactive prompts:
    • Detect GPU availability (CUDA/ROCm)
    • Set storage directory
    • Generate API token
    • Select common pipelines to cache
    • Test installation with sample video
  • Non-interactive mode with flags
  • Skip wizard option (for CI/automated deployments)

Deliverables:

  • src/videoannotator/cli/setup.py - Setup wizard
  • ASCII-safe prompts for Windows compatibility
  • Documentation in docs/usage/setup_wizard.md

Effort: 12 hours

1.3 Enhanced Health & Version Endpoints

Problem: Limited diagnostic information available via API.

Solution:

  • /api/v1/system/version endpoint
  • Enhanced health endpoint with GPU memory, storage, uptime
  • videoannotator version --detailed CLI command

Deliverables:

  • New version endpoint
  • Enhanced health metrics
  • CLI version command

Effort: 12 hours

Checkpoint: Installation experience improved (40 hours total)


Phase 2: Progress & Feedback (Weeks 3-4)

2.1 Real-Time Progress Indicators

  • Progress tracking (percentage, stage, ETA, frames processed)
  • CLI progress bar using rich library
  • API progress endpoint: GET /api/v1/jobs/{id}/progress
  • WebSocket support for real-time updates (optional)

Effort: 20 hours

2.2 Resource Usage Monitoring

  • Real-time CPU, GPU, RAM, disk I/O monitoring
  • Include in health endpoint
  • videoannotator monitor CLI command
  • Configurable warning thresholds

Effort: 16 hours

2.3 Job Completion Notifications

  • Pluggable notification system:
    • Email (SMTP)
    • Webhook (HTTP POST)
    • Desktop notification (CLI mode)
    • Slack integration
    • Discord integration
  • Per-job notification preferences
  • Notification templates

Effort: 20 hours

Checkpoint: Progress and feedback complete (56 hours total)


Phase 3: Configuration & Validation (Weeks 5-6)

3.1 Interactive Config Wizard

  • videoannotator config init --interactive command
  • Step-by-step prompts for common scenarios
  • Hardware detection (GPU, memory, storage)
  • Scenario selection (fast/balanced/high-quality)
  • Validation and preview before saving

Effort: 16 hours

3.2 Config Templates Library

  • Pre-built configuration templates:
    • templates/fast.yaml - Quick processing (lower accuracy)
    • templates/balanced.yaml - Default settings
    • templates/high-quality.yaml - Best accuracy (slower)
    • templates/cpu-only.yaml - No GPU required
    • templates/classroom.yaml - Multi-person + audio
    • templates/clinical.yaml - Face + emotion focus
  • videoannotator config list-templates command
  • videoannotator config use <template> command

Effort: 12 hours

3.3 Enhanced YAML Validation

  • Comprehensive edge case testing:
    • Malformed YAML handling
    • Missing required fields
    • Type mismatches
    • Circular references
    • Unknown pipeline names
  • Better error messages with line numbers
  • Schema validation using JSON Schema

Effort: 12 hours

Checkpoint: Configuration improvements complete (40 hours total)


Phase 4: Logging & Debugging (Week 7)

4.1 Structured Logging

  • JSON log format option (--log-format json)
  • Consistent log levels and categories
  • Request ID tracking across components
  • Performance timing instrumentation
  • Correlation IDs for distributed tracing

Effort: 12 hours

4.2 Log Analysis Tools

  • videoannotator logs analyze command
  • Parse and summarize log files
  • Extract and group errors
  • Identify performance bottlenecks
  • Detect common issues
  • Generate diagnostic reports

Effort: 12 hours

4.3 Legacy Example Cleanup

  • Mark deprecated examples with clear notices
  • Provide migration path to new examples
  • Standardize example format (README, deps, outputs)
  • Remove after one minor version (v1.6.0)

Effort: 8 hours

Checkpoint: Logging and debugging improved (32 hours total)


Phase 5: Export & Integration (Weeks 8-9)

5.1 FiftyOne Integration

  • Direct export to FiftyOne dataset format
  • Metadata preservation (video properties, pipeline info)
  • Sample-level and frame-level annotations
  • Visualization in FiftyOne App
  • videoannotator export fiftyone command

Effort: 20 hours

5.2 Label Studio Integration

  • Export annotations for review/correction
  • Import corrected annotations back
  • Active learning workflow support
  • Task creation from jobs
  • videoannotator export label-studio command
  • videoannotator import label-studio command

Effort: 24 hours

5.3 Custom CSV Templates

  • Flexible tabular export system
  • User-defined CSV column mapping
  • Template library for common formats
  • Direct Pandas DataFrame export (Python API)
  • Excel format support (optional)

Effort: 16 hours

Checkpoint: Integration features complete (60 hours total)


Phase 6: Quality & Performance (Weeks 10-12)

6.1 Quality Assessment Pipeline

  • Per-annotation quality metrics:
    • Model confidence scores
    • Bounding box quality assessment
    • Temporal consistency checks
    • Outlier detection
  • Quality report generation
  • Frame-level quality scores
  • Recommendations for re-processing
  • Quality visualization over time

Effort: 24 hours

6.2 Batch Processing Optimization

  • Smart job scheduling:
    • Priority queue (user-defined priorities)
    • Resource-aware scheduling (GPU vs CPU)
    • Parallel processing (multiple videos simultaneously)
    • Load balancing across workers
  • Progress aggregation for batches
  • Retry mechanism for failed jobs
  • Batch status API endpoints

Effort: 24 hours

6.3 Pipeline Comparison Tools

  • Side-by-side model comparison:
    • Run multiple pipelines on same video
    • Compare accuracy metrics
    • Compare processing time
    • Visual diff of annotations
  • Parameter optimization:
    • Grid search over parameters
    • Performance vs accuracy tradeoffs
    • Recommendation engine
  • Comparison visualization and reports

Effort: 24 hours

Checkpoint: Quality and performance enhancements complete (72 hours total)


Phase 7: Testing & Release (Weeks 13-14)

7.1 Comprehensive Testing

  • Unit tests for all new features
  • Integration tests for workflows
  • Performance regression tests
  • Multi-platform verification

Effort: 24 hours

7.2 Documentation Updates

  • Update all documentation for new features
  • Tutorial for progress indicators
  • Tutorial for FiftyOne/Label Studio integration
  • Configuration wizard guide
  • Quality assessment guide

Effort: 16 hours

7.3 Release Preparation

  • Release notes with complete feature list
  • Migration guide from v1.4.0
  • Update roadmap overview
  • Tag and publish v1.5.0

Effort: 8 hours

Checkpoint: v1.5.0 released (48 hours total)


✅ Success Criteria

Must-Have for Release

  • All 6 phase checkpoints completed
  • Test coverage maintained ≥ 80%
  • No breaking changes from v1.4.0 API
  • Documentation complete for all features
  • Multi-platform testing passed

Quality Targets

  • Installation wizard success rate ≥ 95%
  • Progress indicators working on all platforms
  • FiftyOne/Label Studio exports validated
  • Quality assessment provides useful insights
  • Performance: No regression from v1.4.0

User Experience

  • Setup wizard reduces initial configuration time by 50%
  • Progress indicators reduce support questions
  • Notifications improve workflow efficiency
  • Export integrations enable new workflows

📊 Effort Summary

Phase Duration Effort (hours) Key Deliverables
Phase 1: Installation & Setup Weeks 1-2 40 Auto-download, wizard, health metrics
Phase 2: Progress & Feedback Weeks 3-4 56 Progress bars, monitoring, notifications
Phase 3: Configuration Weeks 5-6 40 Config wizard, templates, validation
Phase 4: Logging & Debugging Week 7 32 Structured logs, analysis tools
Phase 5: Export & Integration Weeks 8-9 60 FiftyOne, Label Studio, CSV
Phase 6: Quality & Performance Weeks 10-12 72 Quality metrics, batch optimization
Phase 7: Testing & Release Weeks 13-14 48 Testing, docs, release
TOTAL 14 weeks 348 hours Feature-rich release

🚫 Still Out of Scope (v1.6.0+)

The following remain deferred to future releases:

Enterprise Features (v1.6.0+)

  • SSO integration (OAuth, SAML, LDAP)
  • Role-based access control (RBAC)
  • Multi-tenancy support
  • Audit logging for compliance
  • API rate limiting and quotas

Advanced Architecture (v1.6.0+)

  • Plugin system with sandboxing
  • Real-time streaming (WebRTC)
  • GraphQL API
  • Microservice decomposition
  • Cloud provider integration (AWS, Azure, GCP)

Advanced ML (v1.7.0+)

  • Active learning workflows
  • Multi-modal correlation analysis
  • Custom model training interface
  • Model ensemble optimization
  • Federated learning support

🎯 Summary

v1.5.0 delivers all the "nice-to-have" features that make VideoAnnotator easier to use and more powerful:

User Experience Wins:

  • ✅ No more manual model downloads
  • ✅ Guided setup for new users
  • ✅ Real-time progress feedback
  • ✅ Flexible notifications
  • ✅ Interactive configuration

Integration Wins:

  • ✅ FiftyOne for visualization
  • ✅ Label Studio for correction/review
  • ✅ Custom export formats

Quality Wins:

  • ✅ Quality assessment metrics
  • ✅ Batch optimization
  • ✅ Pipeline comparison tools

Developer Wins:

  • ✅ Structured logging
  • ✅ Log analysis tools
  • ✅ Better debugging experience

This positions VideoAnnotator as a mature, user-friendly platform ready for widespread adoption while maintaining the research focus from v1.4.0.


Last Updated: October 30, 2025 Target Release: Q3 2026 (3-4 months after v1.4.0) Duration: 12-14 weeks Status: Planning Phase - Feature Enhancement Focus