Merge branch 'feature/standard-classes' into 'develop'

rstrahan · rstrahan · commit 784f6e825e83 · 2026-03-09T22:22:02.000Z
Standard Class Catalog

See merge request genaiic-reusable-assets/engagement-artifacts/genaiic-idp-accelerator!573
diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -7,6 +7,8 @@ SPDX-License-Identifier: MIT-0
 
 ### Added
 
+- **Standard Class Catalog** — When adding a new document class in the Schema Builder, users can now choose between **Custom Class** (define from scratch) and **Standard Class** (import from a catalog of 35 pre-built document types). Standard classes are derived from AWS BDA standard blueprints and include common document types like Invoice, Receipt, W-2, Bank Statement, Payslip, US Driver License, US Passport, various tax forms (1040, 941, 940, W-9, 1098, 1099), insurance cards, birth/death/marriage certificates, and more. Each standard class comes with a complete extraction schema including attributes, descriptions, and nested types. Imported classes are fully editable. Run `make classes-from-bda` to refresh the catalog from the BDA API.
+
 - **Documentation Site** — Added a hosted documentation site built with [Astro Starlight](https://starlight.astro.build/), auto-deployed to GitHub Pages. Provides full-text search (Pagefind), sidebar navigation organized by topic, dark/light mode, and a professional landing page — all sourced directly from the existing `docs/` markdown files with zero content duplication. Browse at [aws-solutions-library-samples.github.io/accelerated-intelligent-document-processing-on-aws](https://aws-solutions-library-samples.github.io/accelerated-intelligent-document-processing-on-aws/).
 
 - **Discovery accessible from CLI and SDK** — Discovery can now be run programmatically via the IDP SDK (`client.discovery.run()`) and CLI (`idp-cli discover`), enabling users with many document classes to automate schema generation without the Web UI. Supports both modes: without ground truth (exploratory) and with ground truth (optimized). ([#228](https://github.com/aws-solutions-library-samples/accelerated-intelligent-document-processing-on-aws/issues/228))
diff --git a/Makefile b/Makefile
@@ -286,6 +286,13 @@ docs-deploy: docs-build
 	cd docs-site && npx gh-pages -d dist --dotfiles
 	@echo -e "$(GREEN)✅ Docs deployed to GitHub Pages!$(NC)"
 
+# Generate standard class catalog from BDA standard blueprints
+# Fetches all AWS standard blueprints and converts them to IDP class schemas
+classes-from-bda:
+	@echo "Generating standard class catalog from BDA standard blueprints..."
+	python3 scripts/generate_standard_classes.py --region us-east-1 --output src/ui/src/data/standard-classes.json
+	@echo -e "$(GREEN)✅ Standard class catalog updated! Review changes in src/ui/src/data/standard-classes.json$(NC)"
+
 # DSR (Deliverable Security Review) targets
 dsr-setup:
 	@echo "Setting up DSR tool..."
diff --git a/VERSION b/VERSION
@@ -1 +1 @@
-0.5.2-wip4
+0.5.2.4
diff --git a/docs/configuration.md b/docs/configuration.md
@@ -15,7 +15,7 @@ The GenAIIDP solution provides multiple configuration approaches to customize do
 
 The web interface allows real-time configuration updates without stack redeployment:
 
-- **Document Classes**: Define and modify document categories and their descriptions (using JSON Schema format)
+- **Document Classes**: Define and modify document categories and their descriptions (using JSON Schema format). Choose from **35+ pre-built standard classes** (Invoice, Receipt, W-2, Bank Statement, etc.) or create custom classes from scratch.
 - **Extraction Attributes**: Configure fields to extract for each document class (defined as JSON Schema properties)
 - **Few Shot Examples**: Upload and configure example documents to improve accuracy (supported in Pattern 2)
 - **Model Selection**: Choose between available Bedrock models for classification and extraction
diff --git a/docs/json-schema-migration.md b/docs/json-schema-migration.md
@@ -193,6 +193,9 @@ The web UI provides two ways to create/edit document schemas:
 1. **Schema Builder** - Visual editor with drag-and-drop interface
    - Navigate to Configuration → Document Schema tab
    - Click "Schema Builder" view
+   - Click "Add Class" to choose between:
+     - **Custom Class** — define your own class with custom fields
+     - **Standard Class** — import from 35+ pre-built document types (Invoice, Receipt, W-2, Bank Statement, Payslip, Driver License, Passport, tax forms, insurance cards, certificates, and more) derived from AWS BDA standard blueprints. Imported classes are fully editable.
    - Add/edit document types and properties visually
 
 2. **JSON View** - Direct JSON editing with validation
diff --git a/lib/idp_cli_pkg/idp_cli/cli.py b/lib/idp_cli_pkg/idp_cli/cli.py
@@ -209,7 +209,7 @@ def _display_deployment_failure(deployer, stack_name: str, result: dict):
 
 
 @click.group()
-@click.version_option(version="0.5.2-wip4")
+@click.version_option(version="0.5.2.4")
 def cli():
     """
     IDP CLI - Batch document processing for IDP Accelerator
diff --git a/lib/idp_cli_pkg/pyproject.toml b/lib/idp_cli_pkg/pyproject.toml
@@ -7,7 +7,7 @@ build-backend = "setuptools.build_meta"
 
 [project]
 name = "idp-cli"
-version = "0.5.2-wip4"
+version = "0.5.2.4"
 description = "Command-line interface for IDP Accelerator batch document processing"
 authors = [{name = "AWS"}]
 license = {text = "MIT-0"}
diff --git a/lib/idp_common_pkg/pyproject.toml b/lib/idp_common_pkg/pyproject.toml
@@ -20,7 +20,7 @@ exclude = [
 
 [project]
 name = "idp_common"
-version = "0.5.2-wip4"
+version = "0.5.2.4"
 description = "Common utilities for GenAI IDP Accelerator patterns"
 authors = [{ name = "AWS", email = "noreply@amazon.com" }]
 requires-python = ">=3.10,<3.14"
diff --git a/lib/idp_common_pkg/setup.py b/lib/idp_common_pkg/setup.py
@@ -117,7 +117,7 @@
 
 setup(
     name="idp_common",
-    version="0.5.2-wip4",
+    version="0.5.2.4",
     packages=find_packages(
         exclude=[
             "build",
diff --git a/lib/idp_sdk/idp_sdk/__init__.py b/lib/idp_sdk/idp_sdk/__init__.py
@@ -91,7 +91,7 @@
     StopWorkflowsResult,
 )
 
-__version__ = "0.5.2-wip4"
+__version__ = "0.5.2.4"
 
 __all__ = [
     # Client
diff --git a/lib/idp_sdk/pyproject.toml b/lib/idp_sdk/pyproject.toml
@@ -7,7 +7,7 @@ build-backend = "setuptools.build_meta"
 
 [project]
 name = "idp-sdk"
-version = "0.5.2-wip4"
+version = "0.5.2.4"
 description = "Python SDK for IDP Accelerator - programmatic access to document processing capabilities"
 authors = [{name = "AWS"}]
 license = {text = "MIT-0"}
diff --git a/scripts/generate_standard_classes.py b/scripts/generate_standard_classes.py
@@ -0,0 +1,295 @@
+#!/usr/bin/env python3
+"""
+Generate standard class catalog from BDA standard blueprints.
+
+This script fetches all AWS standard blueprints from the BDA API using
+resourceOwner=SERVICE, converts them to IDP JSON Schema class definitions,
+and saves the result as a static JSON catalog file for the UI.
+
+Only DOCUMENT modality blueprints are included.
+
+Usage:
+    python scripts/generate_standard_classes.py [--region us-east-1] [--output src/ui/src/data/standard-classes.json]
+    make classes-from-bda
+
+Requirements:
+    - AWS credentials configured with BDA API access
+    - boto3 >= 1.42 installed (pip install --upgrade boto3)
+"""
+
+import argparse
+import json
+import logging
+from copy import deepcopy
+from datetime import datetime, timezone
+from pathlib import Path
+
+import boto3
+
+logging.basicConfig(level=logging.INFO, format="%(levelname)s: %(message)s")
+logger = logging.getLogger(__name__)
+
+
+def fetch_standard_blueprints(region: str) -> list:
+    """Fetch all AWS standard DOCUMENT blueprints from the BDA API.
+
+    Uses resourceOwner=SERVICE to list AWS-managed standard blueprints,
+    then fetches full details for each one with DOCUMENT type.
+    """
+    client = boto3.client("bedrock-data-automation", region_name=region)
+    paginator = client.get_paginator("list_blueprints")
+
+    standard_blueprints = []
+    logger.info("Listing BDA standard blueprints (resourceOwner=SERVICE)...")
+
+    # Non-document blueprint names to skip (audio, video, image modalities)
+    non_document_names = {
+        "Keynote-Highlight",
+        "Media-Search",
+        "General-Audio",
+        "Conversational-Analytics",
+        "General-Image",
+    }
+
+    for page in paginator.paginate(
+        resourceOwner="SERVICE",
+    ):
+        for bp in page.get("blueprints", []):
+            arn = bp.get("blueprintArn", "")
+            name = bp.get("blueprintName", "")
+
+            # Skip known non-document blueprints
+            if name in non_document_names:
+                logger.info(f"  Skipping {name} (non-document modality)")
+                continue
+
+            try:
+                response = client.get_blueprint(
+                    blueprintArn=arn, blueprintStage="LIVE"
+                )
+                blueprint = response.get("blueprint", {})
+                bp_type = blueprint.get("type", "DOCUMENT")
+
+                # Double-check type from detail response
+                if bp_type and bp_type not in ("DOCUMENT", ""):
+                    logger.info(f"  Skipping {name} (type={bp_type})")
+                    continue
+
+                standard_blueprints.append(blueprint)
+                logger.info(f"  Fetched: {name}")
+            except Exception as e:
+                logger.warning(f"  Failed to fetch {name}: {e}")
+
+    return standard_blueprints
+
+
+# ============================================================================
+# Self-contained BDA blueprint → IDP JSON Schema conversion
+# (Mirrors the logic in idp_common.bda.bda_blueprint_service but without
+#  importing the full package and its transitive dependencies)
+# ============================================================================
+
+
+def _transform_bda_definition_to_idp(definition: dict) -> dict:
+    """Transform a BDA definition to IDP format."""
+    result = {}
+    if "type" in definition:
+        result["type"] = definition["type"]
+    if "description" in definition:
+        result["description"] = definition["description"]
+
+    if "properties" in definition:
+        result["properties"] = {}
+        for prop_name, prop_value in definition["properties"].items():
+            result["properties"][prop_name] = _transform_bda_property_to_idp(
+                prop_value
+            )
+
+    if "required" in definition:
+        result["required"] = definition["required"]
+
+    return result
+
+
+def _transform_bda_property_to_idp(prop: dict) -> dict:
+    """Transform a BDA property to IDP format."""
+    result = {}
+
+    if "type" in prop:
+        result["type"] = prop["type"]
+    if "description" in prop:
+        result["description"] = prop["description"]
+    if "inferenceType" in prop:
+        result["x-aws-idp-inference-type"] = prop["inferenceType"]
+
+    # Handle array items
+    if "items" in prop:
+        items = prop["items"]
+        if "$ref" in items:
+            result["items"] = {
+                "$ref": items["$ref"].replace("#/definitions/", "#/$defs/")
+            }
+        else:
+            result["items"] = _transform_bda_property_to_idp(items)
+
+    # Handle $ref
+    if "$ref" in prop:
+        result["$ref"] = prop["$ref"].replace("#/definitions/", "#/$defs/")
+
+    # Handle nested properties (inline objects)
+    if "properties" in prop:
+        result["properties"] = {}
+        for sub_name, sub_value in prop["properties"].items():
+            result["properties"][sub_name] = _transform_bda_property_to_idp(sub_value)
+
+    if "required" in prop:
+        result["required"] = prop["required"]
+
+    return result
+
+
+def transform_blueprint_to_idp_class(blueprint_schema: dict) -> dict:
+    """Convert a BDA blueprint schema to an IDP JSON Schema class definition."""
+    schema_copy = deepcopy(blueprint_schema)
+
+    idp_schema = {
+        "$schema": "https://json-schema.org/draft/2020-12/schema",
+        "$id": schema_copy.get("class", "Document"),
+        "x-aws-idp-document-type": schema_copy.get("class", "Document"),
+        "type": "object",
+    }
+
+    if "description" in schema_copy:
+        idp_schema["description"] = schema_copy["description"]
+
+    if "definitions" in schema_copy:
+        idp_schema["$defs"] = {}
+        for def_name, def_value in schema_copy["definitions"].items():
+            idp_schema["$defs"][def_name] = _transform_bda_definition_to_idp(def_value)
+
+    if "properties" in schema_copy:
+        idp_schema["properties"] = {}
+        for prop_name, prop_value in schema_copy["properties"].items():
+            idp_schema["properties"][prop_name] = _transform_bda_property_to_idp(
+                prop_value
+            )
+
+    if "required" in schema_copy:
+        idp_schema["required"] = schema_copy["required"]
+
+    return idp_schema
+
+
+def convert_to_catalog(blueprints: list, region: str) -> dict:
+    """Convert BDA standard blueprints to IDP class catalog format."""
+    catalog_classes = []
+    for bp in blueprints:
+        try:
+            # Get the blueprint schema
+            schema_str = bp.get("schema", "{}")
+            if isinstance(schema_str, str):
+                schema = json.loads(schema_str)
+            else:
+                schema = schema_str
+
+            # Convert to IDP class schema
+            idp_class = transform_blueprint_to_idp_class(schema)
+
+            # Count attributes
+            properties = idp_class.get("properties", {})
+            has_lists = any(
+                prop.get("type") == "array" for prop in properties.values()
+            )
+            has_nested = bool(idp_class.get("$defs", {}))
+
+            # Enrich with catalog metadata
+            catalog_entry = {
+                "schema": idp_class,
+                "metadata": {
+                    "source": "bda-standard-blueprint",
+                    "blueprintArn": bp.get("blueprintArn", ""),
+                    "blueprintName": bp.get("blueprintName", ""),
+                    "description": idp_class.get("description", ""),
+                    "attributeCount": len(properties),
+                    "hasListTypes": has_lists,
+                    "hasNestedTypes": has_nested,
+                },
+            }
+            catalog_classes.append(catalog_entry)
+            logger.info(
+                f"Converted: {idp_class.get('$id', 'Unknown')} "
+                f"({len(properties)} attributes)"
+            )
+
+        except Exception as e:
+            logger.warning(
+                f"Failed to convert blueprint {bp.get('blueprintName', '?')}: {e}"
+            )
+
+    return {
+        "version": "1.0",
+        "generatedAt": datetime.now(timezone.utc).isoformat(),
+        "region": region,
+        "description": "Standard document class definitions derived from AWS BDA standard blueprints. Regenerate with: make classes-from-bda",
+        "classes": catalog_classes,
+    }
+
+
+def main():
+    parser = argparse.ArgumentParser(
+        description="Generate standard class catalog from BDA standard blueprints"
+    )
+    parser.add_argument(
+        "--region", default="us-east-1", help="AWS region (default: us-east-1)"
+    )
+    parser.add_argument(
+        "--output",
+        default="src/ui/src/data/standard-classes.json",
+        help="Output file path",
+    )
+    args = parser.parse_args()
+
+    logger.info(f"Fetching BDA standard blueprints from {args.region}...")
+    blueprints = fetch_standard_blueprints(args.region)
+    logger.info(f"Found {len(blueprints)} standard DOCUMENT blueprints")
+
+    if not blueprints:
+        logger.warning(
+            "No standard blueprints found. "
+            "Ensure boto3 >= 1.42 is installed (pip install --upgrade boto3) "
+            "and AWS credentials have BDA access."
+        )
+        output_path = Path(args.output)
+        if output_path.exists():
+            logger.info(
+                f"Keeping existing catalog at {output_path} (not overwriting with empty)."
+            )
+        else:
+            logger.warning("No existing catalog found. Creating empty catalog.")
+            output_path.parent.mkdir(parents=True, exist_ok=True)
+            with open(output_path, "w") as f:
+                json.dump(
+                    {
+                        "version": "1.0",
+                        "generatedAt": datetime.now(timezone.utc).isoformat(),
+                        "region": args.region,
+                        "classes": [],
+                    },
+                    f,
+                    indent=2,
+                )
+        return
+
+    catalog = convert_to_catalog(blueprints, args.region)
+
+    output_path = Path(args.output)
+    output_path.parent.mkdir(parents=True, exist_ok=True)
+    with open(output_path, "w") as f:
+        json.dump(catalog, f, indent=2)
+
+    logger.info(f"Catalog written to {output_path}")
+    logger.info(f"Total classes: {len(catalog['classes'])}")
+
+
+if __name__ == "__main__":
+    main()
diff --git a/src/ui/src/components/json-schema-builder/SchemaBuilder.tsx b/src/ui/src/components/json-schema-builder/SchemaBuilder.tsx
diff --git a/src/ui/src/components/json-schema-builder/StandardClassCatalog.tsx b/src/ui/src/components/json-schema-builder/StandardClassCatalog.tsx
diff --git a/src/ui/src/data/standard-classes.json b/src/ui/src/data/standard-classes.json
diff --git a/src/ui/src/hooks/useSchemaDesigner.ts b/src/ui/src/hooks/useSchemaDesigner.ts

Original file line number	Diff line number	Diff line change
`@@ -91,7 +91,7 @@`
`91`	`91`	`StopWorkflowsResult,`
`92`	`92`	`)`
`93`	`93`
`94`		`-__version__ = "0.5.2-wip4"`
	`94`	`+__version__ = "0.5.2.4"`
`95`	`95`
`96`	`96`	`__all__ = [`
`97`	`97`	`# Client`