Auto-commit: Add saidata generation metadata tracking and schema enhancements

- Added saidata metadata section to schema 0.3 with model, generation_date, generation_time, test_date, and human_review_date fields
- Enhanced generation engine to automatically inject metadata during saidata generation
- Updated CLI commands (generate, update, batch) to pass model name for metadata tracking
- Added saidata/ directory to .gitignore to prevent accidental commits
- Updated devcontainer base image from Python 3.11-slim to Ubuntu 24.04
- Fixed git auto-commit hook to use --no-pager flag for git diff

Author: alvagante

.devcontainer/Dockerfile

@@ -1,4 +1,4 @@
-FROM python:3.11-slim
+FROM ubuntu:24.04
 
 # Set environment variables
 ENV PYTHONUNBUFFERED=1 \
@@ -21,6 +21,7 @@ RUN apt-get update && apt-get install -y \
     jq \
     tree \
     htop \
+    python3 \
     ca-certificates \
     gnupg \
     lsb-release \

.gitignore

@@ -163,6 +163,7 @@ cython_debug/
 .sai/
 *.cache
 cache/
+saidata/
 
 # Auto-generated version files (setuptools-scm)
 */_version.py
@@ -199,4 +200,4 @@ logs/
 htmlcov/
 .pytest_cache/
 test-results/
-llm-comparison*
+llm-comparison*

.kiro/hooks/git-auto-commit.kiro.hook

@@ -11,6 +11,6 @@
   },
   "then": {
     "type": "askAgent",
-    "prompt": "Files have been modified in the workspace. Please:\n1. Update CHANGELOG\n2. Run `git add .` to stage all changes\n3. Run `git commit -m \"Auto-commit: [describe the changes]\"` with an appropriate commit message describing what was changed\n"
+    "prompt": "Files have been modified in the workspace. Please:\n1. Update CHANGELOG. When runnging git diff use the --no-pager option\n2. Run `git add .` to stage all changes\n3. Run `git commit -m \"Auto-commit: [describe the changes]\"` with an appropriate commit message describing what was changed\n"
   }
 }

CHANGELOG.md

@@ -8,6 +8,13 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 ## [Unreleased]
 
 ### Added
+- **Saidata Generation Metadata**: Added comprehensive metadata tracking for generated saidata files
+  - New `saidata` metadata section in saidata-0.3-schema.json with model, generation_date, generation_time, test_date, and human_review_date fields
+  - Automatic metadata injection during saidata generation with LLM model name and ISO 8601 timestamps
+  - Generation time tracking in seconds for performance monitoring
+  - Support for test and human review date tracking for lifecycle management
+- **Saidata Directory Exclusion**: Added `saidata/` to .gitignore to prevent accidental commits of generated saidata files
+- **Development Container Update**: Updated devcontainer base image from Python 3.11-slim to Ubuntu 24.04 for better compatibility
 - **Repository Search Relevance Scoring**: Implemented intelligent search result ranking system
   - New `_calculate_relevance_score()` method in UniversalRepositoryDownloader for scoring search results
   - Exact name matches score 100, name prefix matches score 50, name contains query scores 25, description matches score 5
@@ -232,6 +239,9 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 - **Security Enhancements**: File size limits for provider YAML files to prevent DoS attacks
 
 ### Changed
+- **Generation Engine Metadata Handling**: Enhanced save_saidata method to accept optional model_name parameter for metadata injection
+- **CLI Commands Metadata Integration**: Updated generate, update, and batch commands to pass model name to save_saidata for proper metadata tracking
+- **Git Auto-Commit Hook**: Updated hook prompt to include --no-pager flag for git diff commands
 - **Repository Search Implementation**: Refactored search logic for better performance and accuracy
   - Search now applies limit at manager level instead of CLI level for better efficiency
   - Removed redundant limit application in CLI after manager already limits results

saigen/cli/commands/generate.py

@@ -259,7 +259,9 @@ def generate(
         async def run_generation():
             result = await engine.generate_saidata(request)
             if result.success:
-                await engine.save_saidata(result.saidata, output)
+                # Get model name from the result
+                model_name = engine._get_model_name(result.llm_provider_used)
+                await engine.save_saidata(result.saidata, output, model_name=model_name)
             return result
 
         result = asyncio.run(run_generation())

saigen/cli/commands/update.py

@@ -201,7 +201,9 @@ async def run_update():
         if result.success:
             # Save updated saidata
             async def save_result():
-                await generation_engine.save_saidata(result.saidata, output_path)
+                # Get model name from the result
+                model_name = generation_engine._get_model_name(result.llm_provider_used)
+                await generation_engine.save_saidata(result.saidata, output_path, model_name=model_name)
 
             asyncio.run(save_result())
 

saigen/core/batch_engine.py

@@ -300,7 +300,11 @@ async def process_software(software_name: str) -> GenerationResult:
                     # Save to file if successful and output directory specified
                     if result.success and result.saidata and request.output_directory:
                         output_path = self._get_output_path(software_name, request.output_directory)
-                        await self.generation_engine.save_saidata(result.saidata, output_path)
+                        # Get model name from the result
+                        model_name = self.generation_engine._get_model_name(result.llm_provider_used)
+                        await self.generation_engine.save_saidata(
+                            result.saidata, output_path, model_name=model_name
+                        )
 
                     # Update progress
                     progress_reporter.update(result.success, software_name)

saigen/core/generation_engine.py

@@ -231,6 +231,19 @@ async def generate_saidata(self, request: GenerationRequest) -> GenerationResult
                     llm_response.content, request.software_name, context, provider_name
                 )
 
+            # Add generation metadata
+            from datetime import datetime, timezone
+
+            from ..models.saidata import SaidataMetadata
+
+            if not saidata.saidata:
+                saidata.saidata = SaidataMetadata()
+
+            # Get model name from config (always returns a non-empty string)
+            model_name = self._get_model_name(provider_name)
+            saidata.saidata.model = model_name
+            saidata.saidata.generation_date = datetime.now(timezone.utc).isoformat()
+
             # Apply URL validation filter if enabled
             if self.enable_url_filter:
                 if self.logger:
@@ -244,6 +257,10 @@ async def generate_saidata(self, request: GenerationRequest) -> GenerationResult
 
             generation_time = time.time() - start_time
 
+            # Add generation time to metadata
+            if saidata.saidata:
+                saidata.saidata.generation_time = round(generation_time, 2)
+
             result = GenerationResult(
                 success=True,
                 saidata=saidata,
@@ -1228,12 +1245,44 @@ def _update_metrics(self, llm_response) -> None:
         if llm_response.cost_estimate:
             self._total_cost += llm_response.cost_estimate
 
-    async def save_saidata(self, saidata: SaiData, output_path: Path) -> None:
+    def _get_model_name(self, provider_name: str) -> str:
+        """Get the model name for a given provider.
+
+        Args:
+            provider_name: Name of the LLM provider
+
+        Returns:
+            Model name or provider name if model not found
+        """
+        # Try to get from config
+        if self.config and "llm_providers" in self.config:
+            llm_providers = self.config["llm_providers"]
+            if provider_name in llm_providers:
+                provider_config = llm_providers[provider_name]
+                if isinstance(provider_config, dict) and "model" in provider_config:
+                    model = provider_config["model"]
+                    if model:  # Only return if not None or empty
+                        return model
+
+        # Try from config object
+        if self.config_obj and hasattr(self.config_obj, "llm_providers"):
+            if provider_name in self.config_obj.llm_providers:
+                provider_config = self.config_obj.llm_providers[provider_name]
+                if hasattr(provider_config, "model") and provider_config.model:
+                    return provider_config.model
+
+        # Fallback to provider name (always return something)
+        return provider_name if provider_name else "unknown"
+
+    async def save_saidata(
+        self, saidata: SaiData, output_path: Path, model_name: Optional[str] = None
+    ) -> None:
         """Save saidata to file.
 
         Args:
             saidata: SaiData instance to save
             output_path: Path to save the file
+            model_name: Name of the LLM model used for generation (optional)
 
         Raises:
             GenerationEngineError: If saving fails
@@ -1246,6 +1295,17 @@ async def save_saidata(self, saidata: SaiData, output_path: Path) -> None:
                     # Ensure output directory exists
                     output_path.parent.mkdir(parents=True, exist_ok=True)
 
+                    # Add generation metadata if model_name is provided
+                    if model_name:
+                        from datetime import datetime, timezone
+
+                        from ..models.saidata import SaidataMetadata
+
+                        if not saidata.saidata:
+                            saidata.saidata = SaidataMetadata()
+                        saidata.saidata.model = model_name
+                        saidata.saidata.generation_date = datetime.now(timezone.utc).isoformat()
+
                     # Convert to dict and save as YAML
                     data = saidata.model_dump(exclude_none=True)
 
@@ -1259,12 +1319,24 @@ async def save_saidata(self, saidata: SaiData, output_path: Path) -> None:
                             "output_path": str(output_path),
                             "file_size_bytes": file_size,
                             "providers_count": len(saidata.providers) if saidata.providers else 0,
+                            "model_name": model_name,
                         }
                     )
             else:
                 # Ensure output directory exists
                 output_path.parent.mkdir(parents=True, exist_ok=True)
 
+                # Add generation metadata if model_name is provided
+                if model_name:
+                    from datetime import datetime, timezone
+
+                    from ..models.saidata import SaidataMetadata
+
+                    if not saidata.saidata:
+                        saidata.saidata = SaidataMetadata()
+                    saidata.saidata.model = model_name
+                    saidata.saidata.generation_date = datetime.now(timezone.utc).isoformat()
+
                 # Convert to dict and save as YAML
                 data = saidata.model_dump(exclude_none=True)
 

saigen/models/saidata.py

@@ -358,11 +358,22 @@ class Compatibility(BaseModel):
     versions: Optional[VersionCompatibility] = None
 
 
+class SaidataMetadata(BaseModel):
+    """Metadata about the saidata file generation and lifecycle."""
+
+    model: Optional[str] = None
+    generation_date: Optional[str] = None
+    generation_time: Optional[float] = None
+    test_date: Optional[str] = None
+    human_review_date: Optional[str] = None
+
+
 class SaiData(BaseModel):
     """Complete SaiData structure."""
 
     version: str = Field(default="0.3", pattern=r"^\d+\.\d+(\.\d+)?$")
     metadata: Metadata
+    saidata: Optional[SaidataMetadata] = None
     packages: Optional[List[Package]] = None
     services: Optional[List[Service]] = None
     files: Optional[List[File]] = None

schemas/saidata-0.3-schema.json

@@ -55,6 +55,36 @@
         "name"
       ]
     },
+    "saidata": {
+      "type": "object",
+      "description": "Metadata about the saidata file generation and lifecycle",
+      "properties": {
+        "model": {
+          "type": "string",
+          "description": "Name of the LLM model used to generate this saidata"
+        },
+        "generation_date": {
+          "type": "string",
+          "format": "date-time",
+          "description": "ISO 8601 timestamp when this saidata was generated"
+        },
+        "generation_time": {
+          "type": "number",
+          "description": "Time taken to generate this saidata in seconds",
+          "minimum": 0
+        },
+        "test_date": {
+          "type": "string",
+          "format": "date-time",
+          "description": "ISO 8601 timestamp when this saidata was last tested"
+        },
+        "human_review_date": {
+          "type": "string",
+          "format": "date-time",
+          "description": "ISO 8601 timestamp when this saidata was last reviewed by a human"
+        }
+      }
+    },
     "packages": {
       "type": "array",
       "description": "Default package definitions that apply across providers",