docs: update testing strategy and plugin development workflows (#53)

msureshkumar88 · Suresh Kumar Moharajan · web-flow · commit a028daee84c0 · 2026-05-05T09:05:45.000+01:00
* feat: add plugin scaffold generator Implements an interactive plugin scaffold generator that creates complete plugin structures from Jinja2 templates. Features: - Interactive CLI with prompts for plugin metadata - Non-interactive mode for automation - Generates all required files (Cargo.toml, pyproject.toml, Makefile, etc.) - Creates Python package with type stubs - Creates Rust source with PyO3 bindings - Includes test scaffolding and optional benchmarks - Automatically updates workspace Cargo.toml - Validates plugin name and configuration Usage: make plugin-scaffold # Interactive mode python3 tools/scaffold_plugin.py --help # See all options Components: - tools/scaffold_plugin.py: Main generator script - tools/templates/plugin/: Jinja2 templates for all plugin files - Makefile: Added plugin-scaffold and plugin-scaffold-help targets The generator follows patterns from existing plugins (url_reputation, pii_filter, etc.) and ensures consistency across the codebase. Tested with make plugins-validate - all checks pass. Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com> * feat: add all 12 hooks from mcp-context-forge to scaffold tool - Add agent hooks (agent_pre_invoke, agent_post_invoke) - Add HTTP hooks (http_pre_request, http_post_request, http_auth_resolve_user, http_auth_check_permission) - Organize hooks by category with comments - Total of 12 hooks across 5 categories now available for plugin scaffolding Signed-off-by: Suresh <suresh@example.com> Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com> * feat: enhance test template with comprehensive unit tests - Remove TODO placeholders for integration and performance tests - Add unit tests for config deserialization and immutability - Add unit tests for None value handling - Add edge case tests for empty strings and special characters - Add config roundtrip serialization test - Focus on unit-level testing within plugin directory scope Signed-off-by: Suresh <suresh@example.com> Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com> * feat: add basic Rust unit tests to engine template - Add test_engine_creation_with_defaults test - Add test_engine_creation_with_custom_config test - Remove TODO placeholder and provide concrete test implementations - Tests verify engine initialization and configuration handling Signed-off-by: Suresh <suresh@example.com> Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com> * docs: add plugin scaffold generator documentation to README - Add 'Creating a New Plugin' section with scaffold usage - Document all 12 available hooks across 5 categories - Include interactive and non-interactive mode examples - Update helper commands section with plugin-scaffold target - Provide clear guidance for new plugin development Signed-off-by: Suresh <suresh@example.com> Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com> * docs: add plugin scaffold generator to DEVELOPING guide - Add 'Using the Plugin Scaffold Generator' section - Document recommended workflow for creating new plugins - Include interactive and non-interactive mode examples - Clarify manual creation as alternative approach - Update plugin creation workflow with scaffold-first approach Signed-off-by: Suresh <suresh@example.com> Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com> * docs: update testing strategy and plugin development workflows - Add comprehensive testing strategy section to AGENTS.md - Document unit tests in cpex-plugins - Document integration/E2E tests in mcp-context-forge - Add cross-repository coordination guidelines - Expand DEVELOPING.md with detailed workflows - Current workflow: Rust + Python hybrid - Future workflow: Pure Rust (post-framework migration) - Add integration testing coordination - Document migration path - Rewrite TESTING.md with testing architecture - Add cross-repository testing workflow - Add testing coordination guidelines - Add future pure Rust testing approach - Add debugging and best practices sections - Update README.md with testing strategy summary - Add current/future architecture overview - Add proper cross-references to detailed docs Closes #20 Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com> * docs: clarify plugins are pure Python or pure Rust with no dual-path fallback Python entry points in Rust plugins are a packaging and distribution layer only, not a parallel implementation or Rust fallback. Each plugin uses one language for its core logic. Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com> * docs: add plugin-framework integration tests tier to testing strategy cpex-plugins/tests/ holds both unit tests and plugin-framework integration tests (make test-integration). Gateway integration and E2E tests live in mcp-context-forge. Update AGENTS.md, TESTING.md, and README.md to reflect this distinction. Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com> * docs: explicitly document plugin-framework integration tests as distinct layer Add a dedicated "Plugin-Framework Integration Tests" layer (layer 3) in the Testing Layers section. cpex-plugins/tests/ holds both unit tests (make test-all) and integration tests between plugin and framework (make test-integration). Gateway integration/E2E tests remain in mcp-context-forge. Update Running Tests and Debugging sections to match. Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com> * docs: move unit test location from cpex-plugins/tests to plugin directory Unit tests belong in the plugin folder, not the repo-level tests dir: - Python: plugins/rust/python-package/<slug>/tests/ - Rust: inline mod tests in source files Plugin-framework integration tests similarly live in the plugin's own tests/ directory alongside unit tests. Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com> --------- Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com> Signed-off-by: Suresh <suresh@example.com> Co-authored-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com>
diff --git a/AGENTS.md b/AGENTS.md
@@ -1,4 +1,4 @@
-# CLAUDE.md
+# AGENTS.md
 
 ## Git
 
@@ -8,11 +8,171 @@
 
 This is a monorepo of standalone plugin packages for the ContextForge Plugin Extensibility (CPEX) Framework. Each plugin lives in its own top-level directory with independent build configuration.
 
-- Plugins are Rust+Python (PyO3/maturin) or pure Python.
+- Plugins are implemented as **pure Python** or **pure Rust**. Each plugin uses one language for its core logic — there is no dual-path where a plugin ships both Rust and Python implementations with a Rust fallback. For Rust plugins, Python entry points (PyO3/maturin) are a packaging and distribution layer only, not a parallel implementation.
 - Each plugin has its own `pyproject.toml`, `Cargo.toml`, `Makefile`, and `tests/`.
 - Package names follow the pattern `cpex-<plugin-name>` (e.g., `cpex-rate-limiter`).
 - `mcpgateway` is a runtime dependency provided by the host gateway — never declare it in `pyproject.toml`.
 
+## Testing Strategy
+
+### Test Location by Type
+
+- **Unit tests**: Located within each plugin's own directory
+  - Python: `plugins/rust/python-package/<slug>/tests/` (current hybrid) or `plugins/python/<slug>/tests/` (pure Python)
+  - Rust: inline `mod tests` within source files (e.g., `src/lib.rs`)
+  - Test individual plugin functionality in isolation
+  - Fast, deterministic tests
+  - Run during plugin development and CI
+  - Scope: Plugin logic, Rust functions, Python bindings
+
+- **Plugin-framework integration tests**: Located in `plugins/rust/python-package/<slug>/tests/`
+  - Test plugin integration with the local plugin framework (PyO3 bindings, Python ↔ Rust interface)
+  - Run via `make test-integration` within the plugin directory
+  - Scope: PyO3 entry points, plugin loading by the Python framework, hook dispatch
+
+- **Gateway integration tests**: Located in `mcp-context-forge/tests/integration/`
+  - Test plugin integration with the full gateway
+  - Test cross-plugin interactions
+  - Test plugin lifecycle management
+  - Scope: Plugin loading in gateway context, hook execution, framework interaction
+
+- **E2E tests**: Located in `mcp-context-forge/tests/e2e/`
+  - Test complete workflows with plugins enabled
+  - Test plugin behavior in realistic scenarios
+  - Test multi-gateway plugin coordination
+  - Scope: Full request/response cycles, real-world usage patterns
+
+### Cross-Repository Testing Coordination
+
+When developing a plugin:
+
+1. Write unit tests in the plugin's own directory (Rust: inline `mod tests`; Python: `plugins/rust/python-package/<slug>/tests/`) and plugin-framework integration tests in `plugins/rust/python-package/<slug>/tests/`
+2. Run local tests: `make test-all` and `make test-integration` from plugin directory
+3. After plugin PR is merged, coordinate with `mcp-context-forge` team
+4. Write gateway integration/E2E tests in `mcp-context-forge/tests/`
+5. Ensure both repositories' CI passes before release
+
+See `mcp-context-forge/tests/AGENTS.md` for integration/E2E test conventions.
+
+## Plugin Development Workflows
+
+### Current Workflow: Rust + Python Hybrid
+
+**Architecture:**
+- Plugin logic implemented entirely in Rust — no Python fallback implementation
+- Python entry points (PyO3/maturin) are a packaging and distribution layer only
+- Published as Python packages to PyPI
+- Loaded by Python-based plugin framework in gateway
+
+**Why Python Entry Points?**
+The plugin framework is currently implemented in Python (`mcpgateway/plugins/framework/`). Python entry points allow the framework to discover and load plugins dynamically. This is a transitional packaging layer — all plugin logic remains in Rust. This is not a dual-path architecture.
+
+**Development Steps:**
+
+1. **Create Plugin** (in `cpex-plugins`):
+   ```bash
+   cd cpex-plugins
+   make plugin-scaffold  # Interactive plugin generator
+   ```
+
+2. **Implement Plugin** (in `cpex-plugins/plugins/rust/python-package/<slug>/`):
+   - Write Rust core logic in `src/`
+   - Implement Python bindings in `cpex_<slug>/plugin.py`
+   - Update `plugin-manifest.yaml`
+
+3. **Write Tests**:
+   ```bash
+   cd plugins/rust/python-package/<slug>
+   # Add Rust unit tests inline in src/ using mod tests
+   # Add Python unit tests in tests/
+   # Add plugin-framework integration tests in tests/ (run via make test-integration)
+   make test-all          # Run Rust + Python unit tests
+   make test-integration  # Run plugin-framework integration tests
+   ```
+
+4. **Build and Install**:
+   ```bash
+   uv sync --dev
+   make install  # Build Rust extension and install
+   ```
+
+5. **Create PR in cpex-plugins**:
+   - Include unit tests and plugin-framework integration tests
+   - Ensure `make ci` passes
+   - Tag release: `<slug>-v<version>`
+
+6. **Gateway Integration Testing** (in `mcp-context-forge`):
+   - Install plugin: `pip install cpex-<slug>`
+   - Configure in `plugins/config.yaml`
+   - Write integration tests in `tests/integration/`
+   - Write E2E tests in `tests/e2e/`
+
+7. **Release**:
+   - Tag in cpex-plugins triggers PyPI publish
+   - Update mcp-context-forge dependencies
+   - Deploy with new plugin version
+
+### Future Workflow: Pure Rust
+
+**Architecture (Post-Framework Migration):**
+- Plugins implemented in pure Rust
+- Plugin framework migrated to Rust
+- No Python entry points needed
+- Direct Rust-to-Rust plugin loading
+- Published to Cargo registry
+
+**What Changes:**
+- Remove `pyproject.toml` and maturin configuration
+- Remove Python entry points (`cpex_<slug>/plugin.py`)
+- Remove PyO3 bindings
+- Pure Rust crate structure: `plugins/rust/<slug>/`
+- Cargo-based dependency management
+
+**Development Steps (Future):**
+
+1. **Create Plugin** (in `cpex-plugins`):
+   ```bash
+   cd cpex-plugins
+   cargo new --lib plugins/rust/<slug>
+   ```
+
+2. **Implement Plugin** (in `cpex-plugins/plugins/rust/<slug>/`):
+   - Write Rust plugin in `src/lib.rs`
+   - Implement plugin traits from Rust framework
+   - Update `Cargo.toml`
+
+3. **Write Unit Tests** (inline `mod tests` in source files):
+   ```bash
+   cd plugins/rust/<slug>
+   cargo test  # Run Rust tests
+   ```
+
+4. **Build**:
+   ```bash
+   cargo build --release
+   ```
+
+5. **Create PR in cpex-plugins**:
+   - Include unit tests
+   - Ensure `cargo test` passes
+   - Version in `Cargo.toml`
+
+6. **Integration Testing** (in `mcp-context-forge`):
+   - Add plugin as Cargo dependency
+   - Configure in Rust plugin framework
+   - Write integration tests in `tests/integration/`
+   - Write E2E tests in `tests/e2e/`
+
+7. **Release**:
+   - Publish to Cargo registry
+   - Update mcp-context-forge `Cargo.toml`
+   - Deploy with new plugin version
+
+**Migration Timeline:**
+- Current: Hybrid Rust + Python (transitional)
+- Future: Pure Rust (after framework migration)
+- Python components will be removed in future releases
+
 ## Build & Test
 
 From within a plugin directory (e.g., `rate_limiter/`):
@@ -41,4 +201,4 @@ When bumping a plugin version, update all of these:
 2. `cpex_<plugin>/plugin-manifest.yaml` — the `version` field.
 3. `Cargo.lock` — updates automatically on the next build.
 
-Tag releases as `<plugin>-v<version>` (e.g., `rate-limiter-v0.0.2`) on `main` to trigger the PyPI publish workflow.
+Tag releases as `<plugin>-v<version>` (e.g., `rate-limiter-v0.0.2`) on `main` to trigger the PyPI publish workflow.
diff --git a/README.md b/README.md
@@ -24,6 +24,42 @@ Python integration tests live under `plugins/tests/<slug>/`; Rust unit tests liv
 
 Rust crates are owned by the top-level workspace in `Cargo.toml`. Python package names follow `cpex-<slug>`, Python modules follow `cpex_<slug>`, plugin manifests must declare a top-level `kind` in `module.object` form, and `pyproject.toml` must publish the matching `module:object` reference under `[project.entry-points."cpex.plugins"]`. Release tags use the hyphenated slug form `<slug-with-hyphens>-v<version>`, for example `rate-limiter-v0.0.2`.
 
+## Testing Strategy
+
+Testing spans two repositories:
+
+- **Unit tests**: within each plugin's own directory — Python in `plugins/rust/python-package/<slug>/tests/`, Rust inline via `mod tests` in source files
+- **Plugin-framework integration tests**: `plugins/rust/python-package/<slug>/tests/` — test PyO3 bindings and plugin loading by the Python framework (`make test-integration`)
+- **Gateway integration tests**: `mcp-context-forge/tests/integration/` — test plugin integration with the full gateway
+- **E2E tests**: `mcp-context-forge/tests/e2e/` — test complete workflows with plugins
+
+Unit tests and plugin-framework integration tests live in the plugin's own directory. Gateway integration and E2E tests live in `mcp-context-forge`.
+
+See [TESTING.md](TESTING.md) for detailed testing guidelines and cross-repository coordination.
+
+## Plugin Development
+
+### Current Architecture (Transitional)
+
+Plugins are implemented as **pure Python** or **pure Rust** — each plugin uses one language for its logic. There is no dual-path where a plugin ships both Rust and Python implementations with a Rust fallback.
+
+For Rust plugins, the current approach wraps the Rust implementation with PyO3/maturin bindings as a packaging layer:
+- Plugin logic implemented entirely in Rust
+- Python entry points (PyO3/maturin) are a packaging and distribution layer only, not a parallel implementation
+- Published as Python packages to PyPI
+- Loaded by Python-based plugin framework in `mcp-context-forge`
+
+### Future Architecture
+
+After the plugin framework is migrated to Rust:
+- Plugins will be **pure Rust** implementations
+- No Python entry points needed
+- Direct Rust-to-Rust plugin loading
+- Published to Cargo registry
+
+See [DEVELOPING.md](DEVELOPING.md) for detailed workflows for both current and future development.
+
+
 ## Creating a New Plugin
 
 Use the plugin scaffold generator to create a new plugin with all required files and structure:
@@ -68,3 +104,45 @@ make plugin-scaffold           # Create new plugin (interactive)
 ```
 
 The catalog and validator used by CI live in `tools/plugin_catalog.py`.
+
+## Quick Start
+
+### Develop a Plugin
+
+```bash
+cd plugins/rust/python-package/<slug>
+uv sync --dev              # Install dependencies
+make install               # Build Rust extension
+make test-all              # Run unit tests
+```
+
+### Plugin-Framework Integration Testing
+
+After unit tests pass, run plugin-framework integration tests within `cpex-plugins`:
+
+```bash
+cd plugins/rust/python-package/<slug>
+make test-integration  # Test PyO3 bindings and framework loading
+```
+
+### Gateway Integration Testing
+
+After the plugin PR is merged, coordinate with `mcp-context-forge`:
+
+```bash
+cd mcp-context-forge
+pip install /path/to/cpex-plugins/plugins/rust/python-package/<slug>
+# Configure plugin in plugins/config.yaml
+pytest tests/integration/  # Run gateway integration tests
+pytest tests/e2e/          # Run E2E tests
+```
+
+See [TESTING.md](TESTING.md) for cross-repository testing workflow.
+
+## Documentation
+
+- [AGENTS.md](AGENTS.md) - AI coding assistant guidelines
+- [DEVELOPING.md](DEVELOPING.md) - Plugin development workflows
+- [TESTING.md](TESTING.md) - Testing strategy and guidelines
+- [CONTRIBUTING.md](CONTRIBUTING.md) - Contribution guidelines
+- [SECURITY.md](SECURITY.md) - Security policy
diff --git a/TESTING.md b/TESTING.md
