Skip to content

Add automatic downstream EF Core project discovery and building for SQL projects with zero-config working builds and robust SQL file fingerprinting #65

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
Copilot wants to merge 13 commits into from
Closed

Add automatic downstream EF Core project discovery and building for SQL projects with zero-config working builds and robust SQL file fingerprinting #65

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
13 commits
Select commit Hold shift + click to select a range
666d501
Initial plan
Copilot Jan 8, 2026
1866dd4
Add core downstream project discovery and build triggering
Copilot Jan 8, 2026
c3970d7
Add simple-sql-integration sample demonstrating zero-config workflow
Copilot Jan 8, 2026
ac5d693
Add documentation for simple SQL integration feature
Copilot Jan 8, 2026
00b3815
Fix SQL project conditions to only require connection string for data…
Copilot Jan 8, 2026
06a77ce
Fix message count logic and address code review feedback
Copilot Jan 8, 2026
46f5b44
Address review feedback: add config, optimize search, improve orchest…
Copilot Jan 8, 2026
84c8ff6
Add build warning for EFCPT projects lacking ProjectReference
Copilot Jan 9, 2026
e1b8622
Add EfcptForceCompileDownstreamProjects and clarify warning behavior
Copilot Jan 9, 2026
fc18cbc
Implement zero-config working builds with optimization guidance
Copilot Jan 9, 2026
a00d61e
Fix efcpt-config.json to use correct schema format
Copilot Jan 9, 2026
68e868f
Enhance fingerprinting to include SQL source files for material chang…
Copilot Jan 9, 2026
142b3f3
Refactor SQL fingerprinting with robust normalization, configurabilit…
Copilot Jan 10, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
46 changes: 46 additions & 0 deletions docs/user-guide/api-reference.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -311,6 +311,52 @@ Applies MSBuild property overrides to the staged `efcpt-config.json` file. This
| `EfcptSolutionPath` | `$(SolutionPath)` | Solution file path |
| `EfcptProbeSolutionDir` | `true` | Whether to probe solution directory |

### Downstream Project Triggering Properties

These properties control automatic discovery and building of downstream EF Core projects when JD.Efcpt.Build is used in a SQL project.

| Property | Default | Description |
|----------|---------|-------------|
| `EfcptTriggerDownstream` | `true` | Enable/disable automatic downstream discovery and building. Only active when in a SQL project. |
| `EfcptDownstreamProjects` | *(empty)* | Explicit semicolon-separated list of downstream project paths. When set, overrides automatic discovery. Paths can be relative to the SQL project directory or absolute. |
| `EfcptDownstreamAutoDiscover` | `true` | Enable/disable automatic discovery of downstream projects. When `false`, only projects specified in `EfcptDownstreamProjects` are built. |
| `EfcptDownstreamSearchPaths` | *(empty)* | Additional semicolon-separated directories to search for downstream projects. Paths can be relative to the SQL project directory or absolute. |
| `EfcptDownstreamBuildProperties` | *(empty)* | Additional MSBuild properties to pass to downstream projects during build. Use semicolon-separated key=value pairs. |
| `EfcptForceCompileDownstreamProjects` | `false` | When enabled in an EF Core project (not SQL project), forces recompilation to ensure models are up to date. Note: This can reduce performance. Using `ProjectReference` is recommended instead. |

**Example Usage:**

```xml
<!-- In SQL Project: Disable automatic triggering -->
<PropertyGroup>
<EfcptTriggerDownstream>false</EfcptTriggerDownstream>
</PropertyGroup>

<!-- In SQL Project: Explicit downstream projects only -->
<PropertyGroup>
<EfcptDownstreamProjects>
..\DataAccessProject\DataAccessProject.csproj;
..\TestProject\TestProject.csproj
</EfcptDownstreamProjects>
</PropertyGroup>

<!-- In SQL Project: Disable auto-discovery but use explicit list -->
<PropertyGroup>
<EfcptDownstreamAutoDiscover>false</EfcptDownstreamAutoDiscover>
<EfcptDownstreamProjects>..\DataAccessProject\DataAccessProject.csproj</EfcptDownstreamProjects>
</PropertyGroup>

<!-- In SQL Project: Add custom search paths -->
<PropertyGroup>
<EfcptDownstreamSearchPaths>..\src\DataAccess;..\tests</EfcptDownstreamSearchPaths>
</PropertyGroup>

<!-- In EF Core Project: Force recompilation (not recommended - use ProjectReference instead) -->
<PropertyGroup>
<EfcptForceCompileDownstreamProjects>true</EfcptForceCompileDownstreamProjects>
</PropertyGroup>
```

### Advanced Properties

| Property | Default | Description |
Expand Down
48 changes: 48 additions & 0 deletions docs/user-guide/core-concepts.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -94,6 +94,7 @@ The pipeline consists of seven stages that run before C# compilation:
**What it does**:
- Computes an XxHash64 (fast, non-cryptographic) hash of:
- The DACPAC file contents (or schema fingerprint)
- All SQL source files from the SQL project directory with intelligent normalization
- The staged configuration file
- The staged renaming file
- All files in the staged template directory
Expand Down Expand Up @@ -139,6 +140,12 @@ The fingerprint includes multiple sources to ensure regeneration when any releva
- **Library version** - Version of JD.Efcpt.Build.Tasks assembly
- **Tool version** - EF Core Power Tools CLI version (`EfcptToolVersion`)
- **DACPAC content** (in .sqlproj mode) or **schema metadata** (in connection string mode)
- **SQL source files** - Configurable patterns (`EfcptSqlFileIncludePatterns` / `EfcptSqlFileExcludePatterns`)
- Default includes: `**/*.sql`
- Default excludes: `bin/**;obj/**;**/generated/**`
- Smart normalization: whitespace and comments ignored, string literals preserved
- Only material SQL changes (schema modifications) trigger regeneration
- Formatting changes (indentation, line breaks) are ignored
- **efcpt-config.json** - Generation options, namespaces, table selection
- **MSBuild property overrides** - All `EfcptConfig*` properties set in the .csproj
- **efcpt.renaming.json** - Custom naming rules
Expand All @@ -147,6 +154,47 @@ The fingerprint includes multiple sources to ensure regeneration when any releva

**Important**: The fingerprint is computed after MSBuild property overrides are applied, so changing any `EfcptConfig*` property (like `EfcptConfigRootNamespace`) will automatically trigger regeneration.

### SQL File Fingerprinting

SQL source files from the SQL project are fingerprinted using intelligent normalization to detect only material changes:

**What Triggers Regeneration (Material Changes)**:
- Adding or removing tables, views, stored procedures
- Adding, removing, or modifying columns
- Changing data types or constraints
- Modifying string literal content in SQL
- Any structural schema changes

**What Does NOT Trigger Regeneration (Formatting Changes)**:
- Adding or removing whitespace (spaces, tabs, newlines)
- Reformatting code (indentation, alignment)
- Adding or removing SQL comments (both `--` and `/* */`)
- Changing comment content
- Case changes in SQL keywords (optional normalization)

**Customizing SQL File Patterns**:

```xml
<PropertyGroup>
<!-- Include custom file extensions -->
<EfcptSqlFileIncludePatterns>**/*.sql;**/*.tsql</EfcptSqlFileIncludePatterns>

<!-- Exclude additional directories -->
<EfcptSqlFileExcludePatterns>bin/**;obj/**;**/generated/**;**/migrations/**</EfcptSqlFileExcludePatterns>
</PropertyGroup>
```

**Normalization Process**:
1. Extract string literals and preserve them exactly (e.g., `'Product Name'` kept as-is)
2. Remove all SQL comments (`--` line comments and `/* */` block comments)
3. Normalize all whitespace outside strings to single spaces
4. Restore string literals to their original positions
5. Compute hash of normalized content

This approach ensures that only substantive SQL changes trigger model regeneration, while developers remain free to format their SQL code however they prefer without causing unnecessary rebuilds.

**Enhanced Change Detection**: Starting with this release, the fingerprinting logic also includes all SQL source files from the SQL project directory. This means that when you modify a `.sql` file (such as adding a column, changing a constraint, or updating a stored procedure), the models will be regenerated automatically - even if the SQL project hasn't been rebuilt yet to create a new DACPAC. The system normalizes whitespace in SQL files, so only material changes (non-whitespace modifications) trigger regeneration, preventing unnecessary rebuilds due to formatting changes.

All hashing uses XxHash64, a fast non-cryptographic hash algorithm.

### How Fingerprinting Works
Expand Down
Loading