migration docs

Author: MPCoreDeveloper

.github/workflows/ci.yml

@@ -15,6 +15,7 @@ env:
   DOTNET_CLI_TELEMETRY_OPTOUT: true
   DOTNET_NOLOGO: true
   CI_TEST_FILTER: 'Category!=Performance&Category!=Debug&Category!=Manual'
+  MIN_LINE_COVERAGE_PERCENT: '60'
 
 jobs:
   build:
@@ -97,7 +98,47 @@ jobs:
       env:
         CI: "true"
         GITHUB_ACTIONS: "true"
-      
+
+    - name: Validate coverage threshold
+      if: matrix.os == 'ubuntu-latest'
+      shell: python3 {0}
+      run: |
+        import glob
+        import os
+        import sys
+        import xml.etree.ElementTree as ET
+
+        threshold = float(os.environ.get("MIN_LINE_COVERAGE_PERCENT", "60"))
+        files = glob.glob("**/TestResults/**/coverage.cobertura.xml", recursive=True)
+
+        if not files:
+          print("No coverage.cobertura.xml files found.", file=sys.stderr)
+          sys.exit(1)
+
+        covered = 0
+        valid = 0
+        for file in files:
+          try:
+            root = ET.parse(file).getroot()
+            lines_valid = int(root.attrib.get("lines-valid", "0"))
+            lines_covered = int(root.attrib.get("lines-covered", "0"))
+            valid += lines_valid
+            covered += lines_covered
+            print(f"Coverage input: {file} -> {lines_covered}/{lines_valid}")
+          except Exception as ex:
+            print(f"Skipping unreadable coverage file {file}: {ex}")
+
+        if valid == 0:
+          print("Coverage reports found but lines-valid is 0.", file=sys.stderr)
+          sys.exit(1)
+
+        percent = (covered / valid) * 100.0
+        print(f"Merged line coverage: {percent:.2f}% (threshold: {threshold:.2f}%)")
+
+        if percent < threshold:
+          print("Coverage threshold not met.", file=sys.stderr)
+          sys.exit(1)
+
     - name: Upload test results
       if: always()
       uses: actions/upload-artifact@v4

.github/workflows/compatibility-smoke.yml

@@ -10,7 +10,7 @@ on:
       timeout:
         description: Seconds to wait for server ready
         required: false
-        default: '90'
+        default: '180'
 
 permissions:
   contents: read
@@ -26,13 +26,13 @@ env:
   SMOKE_PASSWORD: admin123
   SMOKE_DATABASE: smokedb
   SMOKE_CERT_PASS: smoketest
-  SERVER_READY_TIMEOUT: ${{ github.event.inputs.timeout || '90' }}
+  SERVER_READY_TIMEOUT: ${{ github.event.inputs.timeout || '180' }}
 
 jobs:
   compatibility-smoke:
     name: Server Compatibility Smoke
     runs-on: ubuntu-latest
-    timeout-minutes: 15
+    timeout-minutes: 20
 
     steps:
     - name: Checkout
@@ -109,26 +109,42 @@ jobs:
 
         port    = int(os.environ["SMOKE_HTTPS_PORT"])
         timeout = int(os.environ["SERVER_READY_TIMEOUT"])
-        url     = f"https://127.0.0.1:{port}/api/v1/health"
+        urls    = [
+          f"https://127.0.0.1:{port}/api/v1/health",
+          f"https://127.0.0.1:{port}/health"
+        ]
+
         ctx     = ssl.create_default_context()
         ctx.check_hostname = False
         ctx.verify_mode    = ssl.CERT_NONE
 
-        print(f"Polling {url} for up to {timeout}s …", flush=True)
+        print(f"Polling {urls[0]} / fallback {urls[1]} for up to {timeout}s …", flush=True)
         deadline = time.monotonic() + timeout
+        last_error = None
         while time.monotonic() < deadline:
-            try:
-                with urllib.request.urlopen(url, context=ctx, timeout=3) as r:
-                    if r.status == 200:
-                        print("Server is healthy.", flush=True)
-                        sys.exit(0)
-            except Exception as e:
-                pass
+            for url in urls:
+                try:
+                    with urllib.request.urlopen(url, context=ctx, timeout=3) as r:
+                        if r.status == 200:
+                            print(f"Server is healthy via {url}", flush=True)
+                            sys.exit(0)
+                except Exception as e:
+                    last_error = e
             time.sleep(2)
 
         print(f"Server not ready after {timeout}s — failing.", file=sys.stderr)
+        if last_error is not None:
+            print(f"Last readiness error: {last_error}", file=sys.stderr)
         sys.exit(1)
 
+    - name: Dump server logs on readiness failure
+      if: failure()
+      run: |
+        echo "=== Server stdout/stderr (tail -200) ==="
+        tail -n 200 tests/CompatibilitySmoke/smoke-logs/server-stdout.log || true
+        echo "=== Smoke log file (tail -200) ==="
+        tail -n 200 tests/CompatibilitySmoke/smoke-logs/smoke.log || true
+
     - name: Run compatibility smoke tests
       run: |
         python3 tests/CompatibilitySmoke/smoke_tests.py \

docs/OPEN_ITEMS_v1.7.0.md

@@ -21,6 +21,13 @@
   - `docs/storage/METADATA_IMPROVEMENTS_V1.5.0.md` → `docs/storage/METADATA_IMPROVEMENTS_v1.7.0.md`
   - `docs/storage/QUICK_REFERENCE_V1.5.0.md` → `docs/storage/QUICK_REFERENCE_v1.7.0.md`
 - Updated `docs/release/PHASE12_RELEASE_NOTES.md` version references to `v1.7.0`.
+- Fixed CI compatibility smoke readiness failures:
+  - Added server `GET /api/v1/health` endpoint to match smoke test expectations.
+  - Added TLS PFX password support in server startup configuration.
+  - Increased workflow readiness timeout and added startup log dump on readiness failures.
+- Added CI coverage threshold validation in `.github/workflows/ci.yml`:
+  - Aggregates Cobertura line metrics from generated test coverage files.
+  - Fails CI when merged line coverage drops below `MIN_LINE_COVERAGE_PERCENT`.
 
 ## Open items intentionally left for dedicated follow-up
 

docs/migration/FLUENTMIGRATOR_EMBEDDED_MODE_v1.7.0.md

@@ -5,6 +5,23 @@
 
 ---
 
+## 0. Short Answer for the Most Common Question
+
+**Question:** Does FluentMigrator only work with a remote SharpCoreDB server because the built-in `ISharpCoreDbMigrationSqlExecutor` implementation is the gRPC one?
+
+**Answer:** **No. Embedded mode is fully supported.**
+
+The gRPC executor is only the built-in implementation of the **optional custom executor extension point**. It is used for **remote** execution scenarios.
+
+For **embedded/in-process** execution, FluentMigrator does **not** require any `ISharpCoreDbMigrationSqlExecutor` registration. The migration pipeline runs directly against:
+
+1. `IDatabase` (preferred embedded path)
+2. `DbConnection` (fallback path)
+
+If your application registers SharpCoreDB locally and calls `AddSharpCoreDBFluentMigrator(...)`, migrations execute locally in-process without gRPC.
+
+---
+
 ## 1. What Embedded Mode Means
 
 Embedded mode runs migrations directly against a local SharpCoreDB engine instance in the same process.
@@ -37,6 +54,43 @@ builder.Services.AddSharpCoreDBFluentMigrator(runner =>
 });
 ```
 
+### What must be registered for embedded execution
+
+`AddSharpCoreDBFluentMigrator(...)` registers the FluentMigrator processor integration, but it does **not** force a remote client path.
+
+Embedded execution works when the DI container can resolve one of these local execution sources:
+
+- `IDatabase`
+- `DbConnection`
+
+That means this is valid embedded usage:
+
+```csharp
+using FluentMigrator.Runner;
+using SharpCoreDB.Extensions.Extensions;
+using SharpCoreDB.Interfaces;
+
+builder.Services.AddSingleton<IDatabase>(database);
+builder.Services.AddSharpCoreDBFluentMigrator(runner =>
+{
+    runner.ScanIn(typeof(Program).Assembly).For.Migrations();
+});
+```
+
+A custom `ISharpCoreDbMigrationSqlExecutor` is optional and should only be added when you intentionally want to override SQL execution behavior.
+
+### Why the API can look misleading at first
+
+Users often inspect the package and notice that the only built-in `ISharpCoreDbMigrationSqlExecutor` implementation is the gRPC one.
+
+That does **not** mean FluentMigrator is remote-only.
+
+It means:
+
+- the custom executor interface is an extension point
+- the current built-in custom executor implementation targets remote gRPC execution
+- embedded execution uses `IDatabase` or `DbConnection` directly instead of going through that custom interface
+
 ### Recommended startup execution
 
 ```csharp
@@ -119,11 +173,21 @@ public sealed class CreateUsersTableMigration : Migration
 
 ### Resolution order in `SharpCoreDbMigrationExecutor`
 
+`SharpCoreDbMigrationExecutor` resolves the execution target in this order:
+
 1. `ISharpCoreDbMigrationSqlExecutor` (custom override, if registered)
-2. `IDatabase` (embedded engine path)
+2. `IDatabase` (normal embedded engine path)
 3. `DbConnection` fallback
 
-If you only use `AddSharpCoreDBFluentMigrator(...)` and embedded database DI, path (2) is used.
+This design allows one integration pipeline to support both local and remote execution models.
+
+### What happens when no custom executor is registered
+
+If you only use `AddSharpCoreDBFluentMigrator(...)` and register an embedded SharpCoreDB `IDatabase`, the executor uses the `IDatabase` path directly.
+
+No gRPC client is created.
+No network hop is required.
+No remote server dependency is introduced.
 
 ### Transaction model
 
@@ -151,8 +215,19 @@ Cause:
 
 Fix:
 - Register SharpCoreDB database services before `AddSharpCoreDBFluentMigrator(...)`.
+- For embedded scenarios, prefer registering `IDatabase`.
+
+### B) "I do not see an embedded migration SQL executor implementation"
+
+Cause:
+- You are looking at the custom executor extension point and assuming every execution mode must implement it.
+
+Fix:
+- Use `AddSharpCoreDBFluentMigrator(...)` for embedded mode.
+- Ensure `IDatabase` or `DbConnection` is registered.
+- Only use `AddSharpCoreDBFluentMigratorGrpc(...)` when you explicitly want remote execution over gRPC.
 
-### B) Migration expression not supported
+### C) Migration expression not supported
 
 Cause:
 - Expression translates to SQL not supported by the current engine behavior.
@@ -161,7 +236,7 @@ Fix:
 - Rewrite migration to equivalent supported SQL operations.
 - For advanced custom behavior, use explicit SQL in migration steps.
 
-### C) Version table missing
+### D) Version table missing
 
 Cause:
 - Processor not initialized in the migration pipeline.
@@ -174,6 +249,8 @@ Fix:
 ## 8. Production Checklist (Embedded)
 
 - [ ] `AddSharpCoreDBFluentMigrator(...)` is registered.
+- [ ] Embedded `IDatabase` or `DbConnection` is registered in DI.
+- [ ] No remote gRPC executor is registered unless intentionally overriding execution behavior.
 - [ ] Migrations are scanned from correct assembly.
 - [ ] Startup path executes `MigrateUp()` exactly once.
 - [ ] Backup/restore strategy exists before schema upgrades.

docs/migration/FLUENTMIGRATOR_SERVER_MODE_v1.7.0.md

@@ -5,6 +5,21 @@
 
 ---
 
+## Quick Clarification
+
+A common point of confusion is seeing the built-in gRPC implementation of `ISharpCoreDbMigrationSqlExecutor` and assuming that FluentMigrator only works against a remote SharpCoreDB server.
+
+That assumption is incorrect.
+
+SharpCoreDB supports both:
+
+- **in-process server-host execution** via local `IDatabase` or `DbConnection`
+- **remote server execution** via `AddSharpCoreDBFluentMigratorGrpc(...)`
+
+The custom executor interface is an extension point. The built-in gRPC executor is only the implementation for the remote path.
+
+---
+
 ## Decision Summary (Package Placement)
 
 FluentMigrator support is intentionally delivered through `SharpCoreDB.Extensions` for v1.7.0 because it is currently an integration/glue feature (DI registration + migration processor + execution adapters), not a standalone provider domain.
@@ -52,9 +67,15 @@ builder.Services.AddSharpCoreDBFluentMigrator(runner =>
 
 This path resolves execution via:
 
-- custom executor (if registered)
-- then `IDatabase`
-- then `DbConnection`
+1. custom executor (only if you explicitly register one)
+2. `IDatabase`
+3. `DbConnection`
+
+### Important behavior note
+
+In-process server-host mode does **not** require `ISharpCoreDbMigrationSqlExecutor`.
+
+If the server host already has a local SharpCoreDB engine registered in DI, FluentMigrator runs locally in-process exactly like embedded mode. The fact that the process is a server does not automatically move it to the gRPC path.
 
 ---
 
@@ -84,6 +105,12 @@ This wires:
 - `ISharpCoreDbMigrationSqlExecutor` → `SharpCoreDbGrpcMigrationSqlExecutor`
 - SQL execution through `SharpCoreDB.Client`
 
+### When the custom executor is actually used
+
+The built-in `SharpCoreDbGrpcMigrationSqlExecutor` is used only when you register `AddSharpCoreDBFluentMigratorGrpc(...)` or explicitly register a custom `ISharpCoreDbMigrationSqlExecutor` yourself.
+
+If you do neither, SharpCoreDB falls back to local `IDatabase` / `DbConnection` resolution.
+
 ### Behavior notes
 
 - `MigrateUp` / target version / rollback are available.
@@ -108,6 +135,7 @@ This wires:
 
 - Preferred for single-service ownership deployments.
 - Run before endpoint binding/traffic readiness.
+- Uses the same local execution model as embedded mode when `IDatabase` or `DbConnection` is available.
 
 ### B) External migration job (remote)
 
@@ -131,13 +159,24 @@ This wires:
 | `Rollback(steps)` | ✅ | ✅ |
 | `PerformDBOperationExpression` with connection object | ✅ (if connection available) | ⚠️ Limited (`null` connection) |
 | Uses local `IDatabase` | ✅ | ❌ |
+| Uses local `DbConnection` | ✅ | ❌ |
+| Requires `ISharpCoreDbMigrationSqlExecutor` | ❌ | ✅ |
 | Uses `SharpCoreDB.Client` network transport | ❌ | ✅ |
 
 ---
 
 ## 7. Troubleshooting (Server Mode)
 
-### A) Authentication/session failures in remote mode
+### A) "Why do I only see a gRPC migration executor implementation?"
+
+Cause:
+- You are looking at the optional custom executor extension point rather than the full execution resolution model.
+
+Fix:
+- For in-process server-host execution, use `AddSharpCoreDBFluentMigrator(...)` and register local `IDatabase` or `DbConnection`.
+- For remote execution, use `AddSharpCoreDBFluentMigratorGrpc(...)`.
+
+### B) Authentication/session failures in remote mode
 
 Symptoms:
 - gRPC errors on command execution
@@ -148,7 +187,7 @@ Fix:
 - verify target database exists
 - verify TLS/connectivity settings
 
-### B) Migration works in embedded but fails in remote
+### C) Migration works in embedded but fails in remote
 
 Cause:
 - remote mode has no direct `IDbConnection` callback object for DB-operation expressions.
@@ -157,7 +196,7 @@ Fix:
 - replace DB-operation callback with explicit migration SQL steps
 - avoid connection-dependent callback logic in remote pipelines
 
-### C) Connection timeouts
+### D) Connection timeouts
 
 Fix:
 - increase `SharpCoreDbGrpcMigrationOptions.CommandTimeoutMs`
@@ -191,6 +230,8 @@ migrationRunner.MigrateUp();
 ## 9. Production Checklist (Server Mode)
 
 - [ ] Correct mode selected: in-process vs remote gRPC.
+- [ ] In-process mode uses local `IDatabase` or `DbConnection` registration.
+- [ ] Remote mode uses `AddSharpCoreDBFluentMigratorGrpc(...)` intentionally.
 - [ ] Migration credentials scoped and secret-managed.
 - [ ] TLS enforced in all non-local environments.
 - [ ] Migration step integrated in deployment pipeline.