Merge pull request #9 from microsoft/py-reorg

DocGailZhou · web-flow · commit d5eccd2ae3aa · 2025-12-01T16:28:49.000-05:00
refactor: PY and doc reorganization
diff --git a/.devcontainer/README.md b/.devcontainer/README.md
@@ -93,7 +93,7 @@ The dev container includes:
 3. **Formatting**: Code is auto-formatted with Black on save
 4. **Security Scanning**: Use `bandit` to scan for security issues
 5. **Deployment**: Use `azd up` to deploy to Azure
-6. **Event Simulation**: Use `python infra/scripts/event_simulator.py` to generate real-time telemetry data
+6. **Event Simulation**: Use `python src/simulator/event_simulator.py` to generate real-time telemetry data
 7. **Data Ingestion**: Use `python infra/scripts/fabric/fabric_data_ingester.py` for batch data operations
 8. **Notebooks**: Use `jupyter lab` for interactive development
 
diff --git a/docs/DemonstratorGuide.md b/docs/DemonstratorGuide.md
@@ -45,7 +45,7 @@ When you open one of the emails, you will see something similar to below:
 
 ## Step 5. Demonstrate the Fabric Data Agent
 
-Please follow the [Fabric Data Agent Guide](./FabricDataAgentGuide.md) to understand how the Fabric Data Agent is created and set up. If it is not already set up, you can follow the instructions to create and configure a new Fabric Data Agent in minutes. After that, you can ask any questions you'd like the agent to answer. For starters, you can use the [sample questions and answers](../infra/data/fabric_data_agent/sample_test_questions.md). 
+Please follow the [Fabric Data Agent Guide](./FabricDataAgentGuide.md) to understand how the Fabric Data Agent is created and set up. If it is not already set up, you can follow the instructions to create and configure a new Fabric Data Agent in minutes. After that, you can ask any questions you'd like the agent to answer. For starters, you can use the [sample questions and answers](./fabric_data_agent/sample_test_questions.md). 
 
 In the Fabric workspace, find the Fabric Data Agent that is already set up. You will be able to chat with the agent right away. Below is the user interface you can expect: 
 
diff --git a/docs/EventSimulatorGuide.md b/docs/EventSimulatorGuide.md
@@ -33,7 +33,7 @@ export PRODUCTS_CSV_PATH="../data/products.csv" # Path to products file
 ### 3. Run the Simulator
 
 ```bash
-cd infra/scripts
+cd src/simulator
 ```
 
 ```bash
@@ -56,8 +56,8 @@ python event_simulator.py --assets-csv /path/to/assets.csv --products-csv /path/
 |--------|-------------|---------|
 | `--interval` | Seconds between events per asset | 5.0 |
 | `--max-runtime` | Maximum runtime in seconds | Unlimited |
-| `--assets-csv` | Path to assets.csv file | ../data/assets.csv |
-| `--products-csv` | Path to products.csv file | ../data/products.csv |
+| `--assets-csv` | Path to assets.csv file | infra/data/assets.csv |
+| `--products-csv` | Path to products.csv file | infra/data/products.csv |
 
 ## Interactive Runtime Controls
 
diff --git a/docs/FabricDataAgentGuide.md b/docs/FabricDataAgentGuide.md
@@ -12,18 +12,18 @@ This folder contains essential configuration files for setting up your Fabric Da
 
 ### Core Setup Files
 
-**[Agent Instructions - Master Prompt](../infra/data/fabric_data_agent/agent_instructions.md)**  
+**[Agent Instructions - Master Prompt](./fabric_data_agent/agent_instructions.md)**  
 Contains the primary instructions and behavior guidelines for your Fabric Data Agent. This file establishes the agent's role, scope, and response patterns.
 
-**[Data Source Instructions](../infra/data/fabric_data_agent/data_source_instructions.md)**  
+**[Data Source Instructions](./fabric_data_agent/data_source_instructions.md)**  
 Provides detailed information about your data structure, table relationships, and query patterns. This helps the agent understand your data and deliver more accurate, efficient responses.
 
-**[Example Queries](../infra/data/fabric_data_agent/example_queries.md)**  
+**[Example Queries](./fabric_data_agent/example_queries.md)**  
 Training pairs that map business questions to proven KQL queries. These examples help your data agent learn optimal response patterns for common manufacturing scenarios.
 
 ### Testing Files
 
-**[Test Sample Questions](../infra/data/fabric_data_agent/sample_test_questions.md)**  
+**[Test Sample Questions](./fabric_data_agent/sample_test_questions.md)**  
 A collection of sample questions you can use to test your agent's performance and evaluate response quality. Ideal for sharing with your testing team to jumpstart validation efforts.
 
 ## 💡 Customization Tips
diff --git a/docs/fabric_data_agent/agent_instructions.md b/docs/fabric_data_agent/agent_instructions.md
diff --git a/docs/fabric_data_agent/data_source_instructions.md b/docs/fabric_data_agent/data_source_instructions.md
diff --git a/docs/fabric_data_agent/example_queries.md b/docs/fabric_data_agent/example_queries.md
diff --git a/docs/fabric_data_agent/sample_test_questions.md b/docs/fabric_data_agent/sample_test_questions.md
diff --git a/infra/scripts/fabric/fabric_database.py b/infra/scripts/fabric/fabric_database.py
@@ -27,9 +27,6 @@
 from azure.identity import DefaultAzureCredential
 from azure.kusto.data import KustoClient, KustoConnectionStringBuilder, ClientRequestProperties
 from azure.kusto.data.exceptions import KustoServiceError
-from event import Event
-from asset import Asset
-
 
 def create_kusto_client(cluster_uri) -> KustoClient:
     """
@@ -189,8 +186,16 @@ def get_table_schemas():
             LocationId: int,
             PlantType: string
         )""",
-        
-        "assets": Asset.get_table_schema(),
+
+        # NOTE: schema here matches against entity Asset class in src/entities/asset.py
+        "assets": """(
+            Id: string,
+            Name: string,
+            SiteId: int,
+            Type: string,
+            SerialNumber: string,
+            MaintenanceStatus: string
+        )""",
         
         "products": """(
             Id: string,
@@ -207,7 +212,19 @@ def get_table_schemas():
             IsoCurrencyCode: string
         )""",
 
-        "events": Event.get_table_schema()
+        # NOTE: schema here matches against entity Event class in src/entities/event.py
+        "events": """(
+            Id: string,
+            AssetId: string,
+            ProductId: string,
+            Timestamp: datetime,
+            BatchId: string,
+            Vibration: real,
+            Temperature: real,
+            Humidity: real,
+            Speed: real,
+            DefectProbability: real
+        )"""
     }
 
 
diff --git a/src/azd_env_loader.py b/src/azd_env_loader.py
diff --git a/src/entities/asset.py b/src/entities/asset.py
@@ -2,7 +2,7 @@
 from datetime import datetime
 import random
 import uuid
-from event import Event
+from entities.event import Event
 
 @dataclass
 class Asset:
@@ -22,18 +22,7 @@ def to_dict(self) -> dict:
             "SerialNumber": self.SerialNumber,
             "MaintenanceStatus": self.MaintenanceStatus
         }
-
-    @staticmethod
-    def get_table_schema() -> str:
-        return """(
-            Id: string,
-            Name: string,
-            SiteId: int,
-            Type: string,
-            SerialNumber: string,
-            MaintenanceStatus: string
-        )"""
-    
+        
     @staticmethod
     def get_type_map():
         return {
diff --git a/src/entities/event.py b/src/entities/event.py
@@ -1,5 +1,5 @@
 from dataclasses import dataclass
-from datetime import datetime, timezone
+from datetime import datetime
 from decimal import Decimal
 
 @dataclass
diff --git a/src/sample_data.py b/src/sample_data.py
@@ -3,8 +3,8 @@
 from datetime import datetime, timedelta, timezone
 import pandas as pd
 import os
-from event import Event
-from asset import Asset, AssetType
+from entities.event import Event
+from entities.asset import Asset, AssetType
 
 def generate_locations():
     city_country_map = {
@@ -192,8 +192,8 @@ def generate_sample_data(
     random.seed(random_seed)
     
     try:
-        # Set path as a relative directory in the current script directory
-        path = "../data"
+        # Set path as a relative directory in the current src directory
+        path = "../infra/data"
         
         print(f"Generating sample data...")
         print(f"Output directory: {os.path.abspath(path)}")
diff --git a/src/simulator/event_hub_service.py b/src/simulator/event_hub_service.py
@@ -2,7 +2,6 @@
 
 try:
     from azure.eventhub import EventHubProducerClient, EventData
-    from azure.eventhub.exceptions import EventHubError
     from azure.identity import DefaultAzureCredential
 except ImportError:
     print("❌ Error: azure-eventhub and azure-identity packages are required.")
diff --git a/src/simulator/event_simulator.py b/src/simulator/event_simulator.py
@@ -30,8 +30,8 @@
 Environment Variables:
     AZURE_EVENT_HUB_NAMESPACE_HOSTNAME - Azure Event Hub namespace (e.g., myeventhub.servicebus.windows.net)
     AZURE_EVENT_HUB_NAME - Name of the Event Hub
-    ASSETS_CSV_PATH - Path to assets.csv file (default: ../data/assets.csv)
-    PRODUCTS_CSV_PATH - Path to products.csv file (default: ../data/products.csv)
+    ASSETS_CSV_PATH - Path to assets.csv file (default: infra/data/assets.csv)
+    PRODUCTS_CSV_PATH - Path to products.csv file (default: infra/data/products.csv)
     SIMULATION_INTERVAL - Seconds between events per asset (default: 5)
     MAX_RUNTIME_SECONDS - Maximum runtime in seconds (default: unlimited)
 
@@ -52,14 +52,19 @@
 import sys
 import threading
 import time
-from datetime import datetime, timezone
+import sys
 from pathlib import Path
+
+# Add parent directory to Python path to allow imports from sibling directories
+sys.path.insert(0, str(Path(__file__).parent.parent))
+
+from datetime import datetime, timezone
 from typing import List, Dict, Optional
 
 # Import our project classes
-from event import Event
-from asset import AssetType
-from event_hub_service import EventHubService
+from entities.event import Event
+from entities.asset import AssetType
+from simulator.event_hub_service import EventHubService
 from azd_env_loader import AZDEnvironmentLoader
 
 class AssetSimulator:
@@ -466,8 +471,8 @@ def main():
     env_loader = AZDEnvironmentLoader(required=False, log_events=True)
     env_loader.set_env_vars()
 
-    script_dir = Path(__file__).parent
-    data_dir = (script_dir / '..' / 'data').resolve()
+    src_dir = Path(__file__).parent.parent
+    data_dir = (src_dir / '..' / 'infra' / 'data').resolve()
     
     # Get configuration from environment variables or arguments
     event_hub_namespace_fqdn = os.getenv('AZURE_EVENT_HUB_NAMESPACE_HOSTNAME')
@@ -490,9 +495,9 @@ def main():
     
     # Convert relative paths to absolute
     if not Path(assets_csv_path).is_absolute():
-        assets_csv_path = (script_dir / assets_csv_path).resolve()
+        assets_csv_path = (src_dir / assets_csv_path).resolve()
     if not Path(products_csv_path).is_absolute():
-        products_csv_path = (script_dir / products_csv_path).resolve()
+        products_csv_path = (src_dir / products_csv_path).resolve()
     
     print("🏭 Manufacturing Event Simulator")
     print("="*60)
