Merge pull request #160 from helpfulengineering/phase-2

Phase 2

Author: touchthesun

.repo-map.md

@@ -3,7 +3,7 @@ REPOSITORY MAP (Aider Style)
 ================================================================================
 Repository: supply-graph-ai
 
-Total Python files: 280
+Total Python files: 287
 
 ├── demo/
 │   ├── __init__.py
@@ -158,6 +158,22 @@ Total Python files: 280
 │   │       def save_okh_manifest()
 │   │       def save_okw_facility()
 │   │       def main()
+│   ├── okh_generation_baseline_report.py
+│   │       def main()
+│   ├── okh_generation_batch.py
+│   │       def _parse_args()
+│   │       def _select_repos()
+│   │       def main()
+│   ├── okh_generation_chunked_evaluation.py
+│   │       def _load_report()
+│   │       def _row_map()
+│   │       def _field_presence_score()
+│   │       def _summarize()
+│   │       def _paired_ids()
+│   │       def _compare_paired()
+│   │       def main()
+│   ├── okh_generation_layer_compare.py
+│   │       def main()
 │   ├── populate_ome_from_synthetic_data.py
 │   ├── setup_storage.py
 │   ├── validate_docs.py
@@ -619,9 +635,19 @@ Total Python files: 280
 │       │   │       class BOMBuilder (__init__, build_bom, _validate_components)
 │       │   ├── built_directory.py
 │       │   │       class BuiltDirectoryExporter (__init__, _bom_to_markdown, _bom_to_csv...)
+│       │   ├── dataset_generation.py
+│       │   │       def _default_log()
 │       │   ├── engine.py
 │       │   │       class EngineMetrics (__post_init__)
 │       │   │       class GenerationEngine (__init__, _validate_config, _initialize_matchers...)
+│       │   ├── gitlab_instance.py
+│       │   │       def normalize_http_url()
+│       │   │       def parse_self_hosted_gitlab_hosts()
+│       │   │       def gitlab_path_has_namespace_and_project()
+│       │   │       def is_allowlisted_self_hosted_gitlab_url()
+│       │   │       def is_gitlab_http_clone_url()
+│       │   │       def extract_gitlab_owner_repo_from_path()
+│       │   │       def gitlab_api_v4_base_url()
 │       │   ├── layers/
 │       │   │   ├── __init__.py
 │       │   │   ├── base.py
@@ -632,7 +658,9 @@ Total Python files: 280
 │       │   │   ├── heuristic.py
 │       │   │   │       class FilePattern
 │       │   │   │       class HeuristicMatcher (__init__, _initialize_file_patterns, _initialize_content_patterns...)
+│       │   │   │       def _is_license_like_file_path()
 │       │   │   ├── llm.py
+│       │   │   │       class ChunkedLLMReduceSchema
 │       │   │   │       class LLMGenerationLayer (__init__, _create_llm_service, _load_okh_schema_prompt...)
 │       │   │   └── nlp.py
 │       │   │           class EntityPattern
@@ -704,6 +732,18 @@ Total Python files: 280
 │       │               class TextProcessor (__init__, _initialize_license_patterns, _initialize_version_patterns...)
 │       ├── llm/
 │       │   ├── __init__.py
+│       │   ├── chunking.py
+│       │   │       class TokenEstimator (__call__)
+│       │   │       class TokenBudget
+│       │   │       class TokenBudgetPolicy
+│       │   │       class ChunkingConfig (__post_init__)
+│       │   │       class TextChunk
+│       │   │       def default_token_estimator()
+│       │   │       def build_token_budget()
+│       │   │       def split_text_into_chunks()
+│       │   │       def _max_end_for_budget()
+│       │   │       def _apply_boundary_preference()
+│       │   │       def _suffix_start_for_overlap()
 │       │   ├── models/
 │       │   │   ├── __init__.py
 │       │   │   ├── metrics.py
@@ -714,6 +754,9 @@ Total Python files: 280
 │       │   │   │       class LLMRequestType
 │       │   │   │       class LLMRequestConfig
 │       │   │   │       class LLMRequest (__post_init__)
+│       │   │   │       class LLMPayloadSection (__post_init__)
+│       │   │   │       class LLMTraceContext
+│       │   │   │       class LLMStructuredRequest (__post_init__)
 │       │   │   └── responses.py
 │       │   │           class LLMResponseStatus
 │       │   │           class LLMResponseMetadata
@@ -771,7 +814,7 @@ Total Python files: 280
 │       │   │           class ConnectionError
 │       │   └── service.py
 │       │           class LLMServiceConfig (__init__)
-│       │           class LLMService (__init__, _get_providers_to_try, _update_metrics...)
+│       │           class LLMService (__init__, _build_structured_prompt, _validate_response_schema...)
 │       ├── main.py
 │       ├── matching/
 │       │   ├── __init__.py
@@ -1181,7 +1224,7 @@ Total Python files: 280
 
 ## Overview
 
-Total files analyzed: 280
+Total files analyzed: 287
 
 ## Entry Points
 
@@ -2159,7 +2202,7 @@ This module provides data models for LLM metrics ...
 
 This module provides data models for LLM requests...
 
-**Exports:** LLMRequestType, LLMRequestConfig, LLMRequest
+**Exports:** LLMRequestType, LLMRequestConfig, LLMRequest, LLMPayloadSection, LLMTraceContext
 
 **Classes:**
 - `LLMRequestType` (inherits: Enum)
@@ -2168,6 +2211,12 @@ This module provides data models for LLM requests...
   - Configuration for LLM requests....
 - `LLMRequest`
   - Base LLM request model....
+- `LLMPayloadSection`
+  - Named section of text for structured long-context workflows....
+- `LLMTraceContext`
+  - Trace metadata for chunked or multi-step request execution....
+- `LLMStructuredRequest`
+  - Structured request model for chunked map-reduce style workflows....
 
 ### `src/core/llm/models/responses.py`
 > LLM Response Models for the Open Hardware Manager.
@@ -3403,6 +3452,48 @@ This script generates OKH and OKW files specifically designed ...
 - `save_okh_manifest(manifest, output_path)`
   - Save OKH manifest to JSON file....
 
+### `scripts/okh_generation_baseline_report.py`
+> Generate OKH generation baseline report (Issue 1.3.1 Phase B5).
+
+Reads tests/data/okh_generation/rep...
+
+**Exports:** main
+
+**Functions:**
+- `main()`
+
+### `scripts/okh_generation_batch.py`
+> Batch OKH generation for every supported entry in tests/data/okh_generation/repositories.json.
+
+Writ...
+
+**Exports:** main
+
+**Functions:**
+- `main()`
+
+**Internal Dependencies:** 1 imports
+
+### `scripts/okh_generation_chunked_evaluation.py`
+> Evaluate chunked LLM canary quality gates against a baseline batch report.
+
+This compares two batch ...
+
+**Exports:** main
+
+**Functions:**
+- `main()`
+
+### `scripts/okh_generation_layer_compare.py`
+> Compare 3-layer vs 4-layer OKH manifests on disk (heuristic metrics).
+
+For each supported repo in te...
+
+**Exports:** main
+
+**Functions:**
+- `main()`
+
 ### `scripts/populate_ome_from_synthetic_data.py`
 > Populate the Azure Blob Storage container "ome" with OKH and OKW files from synthetic_data/.
 
@@ -4256,6 +4347,11 @@ This module provides functionality...
 - `BuiltDirectoryExporter`
   - Export generated manifests and BOMs to built directory structure...
 
+### `src/core/generation/dataset_generation.py`
+> Programmatic OKH generation from a repository URL or local clone path.
+
+Mirrors the CLI ``generate-f...
+
 ### `src/core/generation/engine.py`
 > Generation Engine for OKH manifest generation.
 
@@ -4272,6 +4368,24 @@ This module provides the main orchestration engine t...
 
 This class coordinates a...
 
+### `src/core/generation/gitlab_instance.py`
+> Self-hosted GitLab instance detection and API base URL helpers.
+
+Hosts are trusted as GitLab only wh...
+
+**Exports:** normalize_http_url, parse_self_hosted_gitlab_hosts, gitlab_path_has_namespace_and_project, is_allowlisted_self_hosted_gitlab_url, is_gitlab_http_clone_url
+
+**Functions:**
+- `normalize_http_url(url)`
+  - Match URLRouter-style normalization without importing URLRouter (avoid cycles)....
+- `parse_self_hosted_gitlab_hosts()`
+  - Read allowlist from env each time so tests can monkeypatch GITLAB_SELF_HOSTED_HO...
+- `gitlab_path_has_namespace_and_project(path)`
+- `is_allowlisted_self_hosted_gitlab_url(normalized_url)`
+  - True if hostname is in GITLAB_SELF_HOSTED_HOSTS and path looks like a project....
+- `is_gitlab_http_clone_url(normalized_url)`
+  - True for gitlab.com or allowlisted host with a valid project path (clone/API eli...
+
 ### `src/core/generation/layers/__init__.py`
 > Generation layers for OKH manifest extraction.
 
@@ -4326,9 +4440,11 @@ This layer applies rule-based pattern recogni...
 
 This layer uses Large Language Models for advance...
 
-**Exports:** LLMGenerationLayer
+**Exports:** ChunkedLLMReduceSchema, LLMGenerationLayer
 
 **Classes:**
+- `ChunkedLLMReduceSchema` (inherits: BaseModel)
+  - Minimal schema guardrail for chunked reduce output....
 - `LLMGenerationLayer` (inherits: BaseGenerationLayer)
   - LLM Generation Layer using Large Language Models for advanced content analysis.
 ...
@@ -4463,6 +4579,35 @@ This module provides URL validation, platform detect...
 
 This module provides a LLM provider abstraction ...
 
+### `src/core/llm/chunking.py`
+> Core chunking primitives for long-context LLM workflows....
+
+**Exports:** TokenEstimator, default_token_estimator, TokenBudget, TokenBudgetPolicy, ChunkingConfig
+
+**Classes:**
+- `TokenEstimator` (inherits: Protocol)
+  - Protocol for token estimation functions....
+- `TokenBudget`
+  - Resolved token budget for a specific request....
+- `TokenBudgetPolicy`
+  - Policy inputs used to derive a TokenBudget....
+- `ChunkingConfig`
+  - Chunking settings for splitting large input text....
+- `TextChunk`
+  - A chunk of source text and its position in the original string....
+
+**Functions:**
+- `default_token_estimator(text)`
+  - Conservative fallback estimator with no external dependencies.
+
+Uses a simple ch...
+- `build_token_budget(policy, request_type, provider)`
+  - Build a request-specific token budget and fail fast on impossible budgets....
+- `split_text_into_chunks(text, config, estimate_tokens_fn)`
+  - Split text into chunks by token budget with overlap and boundary preferences.
+
+T...
+
 ### `src/core/llm/provider_selection.py`
 > LLM Provider Selection Utility for the Open Hardware Manager.
 

docs/CLI/index.md

@@ -934,8 +934,15 @@ You can also explicitly specify the domain using the `--domain` option.
 - `--location TEXT` - Filter by location (city, country, or region)
 - `--capabilities TEXT` - Comma-separated list of required capabilities
 - `--materials TEXT` - Comma-separated list of required materials
-- `--min-confidence FLOAT` - Minimum confidence threshold (0.0-1.0, default: 0.7)
+- `--min-confidence FLOAT` - Minimum confidence threshold (0.0-1.0, default: 0.3)
 - `--max-results INTEGER` - Maximum number of results (default: 10)
+- `--max-depth INTEGER` - BOM matching depth (0 = single-level, >0 = nested component matching)
+- `--auto-detect-depth` - Auto-enable nested matching when manifest contains nested components
+- `--allow-facility-combinations` - Enable aggregated multi-facility matching for manufacturing
+- `--max-facilities-per-solution INTEGER` - Cap facilities used in one composite solution (default: 3)
+- `--combination-strategy [greedy]` - Composite solver strategy
+- `--no-alternative-solutions` - Return only the top composite solution
+- `--explain` - Include structured and human-readable per-solution explanations in output
 - `--output, -o TEXT` - Output file path
 - `--use-llm` - Enable LLM integration for enhanced matching
 - `--llm-provider [anthropic|openai|google|azure|local]` - LLM provider
@@ -946,6 +953,9 @@ You can also explicitly specify the domain using the `--domain` option.
 - `--table` - Output results in table format
 - `--verbose, -v` - Enable verbose output
 
+**Verbose behavior note:**
+- In `match requirements`, `--verbose` automatically enables explanation output (equivalent to `--explain`) so diagnostics are consistently shown during interactive debugging.
+
 **Examples:**
 ```bash
 # Match OKH requirements (manufacturing domain)
@@ -969,17 +979,26 @@ ohm match requirements my-design.okh.json --min-confidence 0.9
 # Match with LLM enhancement
 ohm match requirements my-design.okh.json --use-llm --quality-level professional
 
+# Composite matching across multiple facilities
+ohm match requirements my-design.okh.json --allow-facility-combinations --max-facilities-per-solution 3
+
+# Composite matching returning only best solution
+ohm match requirements my-design.okh.json --allow-facility-combinations --no-alternative-solutions
+
 # Save results to file
 ohm match requirements my-design.okh.json --output matches.json
 ```
 
 **Output:**
 The command displays matching facilities with:
+- Rank (`1` = best match)
 - Facility name
 - **Full facility ID (UUID)** - required for `ohm okw get` command
+- Composite detail bundle (`facility_details`) in JSON mode when a solution spans multiple facilities
 - Confidence score
 - Match type (manufacturing or cooking)
 - Location (if available)
+- Structured summary metadata (`match_summary`, `coverage_gaps`) in JSON mode
 
 **Note:** Facility IDs are displayed as full UUIDs. Use the full ID with `ohm okw get` to retrieve facility details.
 

docs/api/routes.md

@@ -1151,7 +1151,13 @@ The endpoint automatically detects the domain from the input data:
 
   // Optional Matching Parameters
   "min_confidence": 0.5,
-  "max_results": 10
+  "max_results": 10,
+
+  // Facility-combination controls (API-first)
+  "allow_facility_combinations": false,
+  "max_facilities_per_solution": 3,
+  "return_alternative_solutions": true,
+  "combination_strategy": "greedy"
 }
 ```
 
@@ -1212,12 +1218,20 @@ The endpoint returns a wrapped response with the following structure:
   - `facility`: Complete facility/kitchen information
   - `facility_id`: **Unique identifier** for the facility (required for API operations)
   - `facility_name`: Human-readable facility name
+  - `facility_ids`: List of facility IDs participating in the solution (**composite solutions only**)
+  - `facility_names`: List of facility names participating in the solution (**composite solutions only**)
+  - `facility_details`: List of `{facility_id, facility_name, facility}` objects (**composite solutions only**)
   - `match_type`: Domain type ("manufacturing" or "cooking")
   - `confidence`: Match confidence score (0.0-1.0)
-  - `rank`: Solution rank (1 = best match)
+  - `rank`: Solution rank (1 = best match, stable after confidence filtering and result limiting)
 - `total_solutions`: Total number of matching solutions found
 - `matching_metrics`: Breakdown of match types
 - `validation_results`: Validation results for each solution
+- `match_summary`: Structured summary object for API consumers (mode, counts, coverage, and combination flags)
+- `coverage_gaps`: Uncovered process requirements, if any
+- `match_summary_text`: Human-readable rendering of `match_summary` (for logs and quick inspection)
+  - `match_summary.coverage_gap_counts`: Multiplicity of each uncovered requirement
+  - `match_summary.warnings`: Non-fatal warnings (e.g. composite fallback to single-facility matching)
 
 **Status:**  **Fully Implemented** - **Complete matching engine with Azure Blob Storage integration**
 

docs/matching/match-explanation.md

@@ -11,6 +11,7 @@ When matching requirements to facilities, you can request **per-facility explana
   ```bash
   ohm match requirements my-design.okh.json --explain
   ```
+  In current CLI behavior, `--verbose` also enables explanation output automatically for `match requirements`.
   Explanations are printed under each facility in table output; in JSON output they appear as `explanation` and `explanation_human` on each solution.
 
 ### Manual testing with synthetic data (no server/storage)

pyproject.toml

@@ -53,6 +53,7 @@ addopts = "-v --tb=short"
 asyncio_mode = "strict"
 markers = [
     "e2e: end-to-end canonical workflow tests",
+    "integration: integration tests that may require services or seeded data",
     "slow: tests that take more than 10 seconds",
     "llm: tests requiring LLM API access",
     "benchmark: performance benchmark tests",