Skip to content

Commit de65ff0

Browse files
CopilotCopilot
authored
feat: Add automatic database-first SQL generation with two-project pattern (#52)
* Add core SqlProj generation tasks and MSBuild properties - Added RunSqlPackage task to extract database schema using sqlpackage - Added GenerateSqlProj task to create .sqlproj/.csproj/.fsproj files - Added MSBuild properties for SqlProj generation configuration - Registered new tasks in targets file - Added pipeline targets for SqlProj generation workflow * Refine SqlProj generation pipeline - Modified targets to properly pass extracted DACPAC to EF Core generation - Added EfcptGenerateSqlProjFile property for optional project file generation * Add database-first SqlProj generation sample - Created comprehensive sample demonstrating Database → DACPAC → EF Core Models workflow - Added detailed README explaining the new database-first approach - Sample includes EntityFrameworkCoreProject with EfcptGenerateSqlProj enabled - Documented configuration options and comparison with traditional workflow * Add schema fingerprinting for SqlProj generation - Added EfcptQueryDatabaseSchemaForSqlProj target to compute schema fingerprint before extraction - Schema fingerprint enables incremental builds - DACPAC only extracted when database schema changes - Set _EfcptUseConnectionString flag to integrate with existing fingerprinting logic - Ensures deterministic, fast builds for database-first workflow * Update README with database-first SqlProj generation feature - Added Database-First SqlProj Generation section highlighting the new capability - Updated Key Features list to include schema extraction - Added new sample to the samples list - Documented workflow: Live Database → DACPAC → EF Core Models * Implement proper database-first SqlProj generation with SQL scripts - Use sqlpackage /Action:Extract /p:ExtractTarget=Flat to extract individual SQL scripts - Generate properly structured SQL projects with organized folder hierarchy - Add auto-generation warning headers to all SQL files - Implement lifecycle hooks: BeforeSqlProjGeneration, AfterSqlProjGeneration, BeforeEfcptGeneration, AfterEfcptGeneration - Build generated SQL project to DACPAC for EF Core model generation - Workflow: Database → SQL Scripts → SqlProj → DACPAC → EF Core Models - SQL project serves as human-readable artifact that can be extended - .NET Framework 4.7.2 compatibility for relative path calculation
1 parent a8d9ae7 commit de65ff0

31 files changed

Lines changed: 3563 additions & 39 deletions

File tree

‎README.md‎

Lines changed: 42 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -53,6 +53,7 @@ dotnet build
5353

5454
- **Automatic generation** - DbContext and entities generated during `dotnet build`
5555
- **Incremental builds** - Only regenerates when schema or config changes
56+
- **Database-First SqlProj Generation** - Extract schema from live databases to DACPAC (NEW!)
5657
- **Dual input modes** - Works with SQL Projects (.sqlproj) or live database connections
5758
- **Smart discovery** - Auto-finds database projects and configuration files
5859
- **T4 template support** - Customize code generation with your own templates
@@ -89,12 +90,53 @@ dotnet build
8990
| [MSBuild.Sdk.SqlProj](https://github.com/rr-wfm/MSBuild.Sdk.SqlProj) | `.csproj` / `.fsproj` | Yes |
9091
| Traditional SQL Projects | `.sqlproj` | Windows only |
9192

93+
## New: Database-First SQL Generation
94+
95+
Automatically generate SQL scripts from your live database when JD.Efcpt.Build detects it's referenced in a SQL project:
96+
97+
**DatabaseProject** (SQL):
98+
```xml
99+
<Project Sdk="MSBuild.Sdk.SqlProj/3.3.0">
100+
<PropertyGroup>
101+
<EfcptConnectionString>Server=...;Database=MyDb;...</EfcptConnectionString>
102+
</PropertyGroup>
103+
<ItemGroup>
104+
<PackageReference Include="JD.Efcpt.Build" Version="*" />
105+
</ItemGroup>
106+
</Project>
107+
```
108+
109+
**DataAccessProject** (EF Core):
110+
```xml
111+
<ItemGroup>
112+
<ProjectReference Include="..\DatabaseProject\DatabaseProject.csproj" />
113+
<PackageReference Include="JD.Efcpt.Build" Version="*" />
114+
</ItemGroup>
115+
```
116+
117+
This enables the complete two-project workflow:
118+
119+
```
120+
Live Database → SQL Scripts (in SQL Project) → DACPAC → EF Core Models (in DataAccess Project)
121+
```
122+
123+
**Benefits:**
124+
- ✅ Automatic SQL project detection (no configuration needed)
125+
- ✅ Database as source of truth
126+
- ✅ Human-readable SQL scripts for review and version control
127+
- ✅ Clean separation: Database project (schema) + DataAccess project (models)
128+
- ✅ Incremental builds with schema fingerprinting
129+
- ✅ Works with .NET 10+ `dnx` (no sqlpackage installation required)
130+
131+
See the [Database-First SQL Generation sample](samples/database-first-sql-generation/) for a complete example.
132+
92133
## Samples
93134

94135
See the [samples directory](samples/) for complete working examples:
95136

96137
- [Simple Generation](samples/simple-generation/) - Basic DACPAC-based generation
97138
- [SDK Zero Config](samples/sdk-zero-config/) - Minimal SDK project setup
139+
- [Database-First SQL Generation](samples/database-first-sql-generation/) - Auto-generate SQL scripts from live database (NEW!)
98140
- [Connection String Mode](samples/connection-string-sqlite/) - Generate from live database
99141
- [Custom Renaming](samples/custom-renaming/) - Table and column renaming
100142
- [Schema Organization](samples/schema-organization/) - Multi-schema folder structure

‎docs/user-guide/error-codes.md‎

Lines changed: 292 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,292 @@
1+
# Error Codes Reference
2+
3+
This page documents all error codes that can be emitted by JD.Efcpt.Build tasks.
4+
5+
## Error Code Format
6+
7+
Error codes follow the format `JDxxxx` where `xxxx` is a four-digit number.
8+
9+
## Configuration and Connection String Errors (JD0001-JD0019)
10+
11+
### JD0001: Configuration File Type Warning
12+
**Severity**: Warning
13+
**Task**: ConfigurationFileTypeValidator
14+
15+
The configuration file type doesn't match the expected format.
16+
17+
**Example**:
18+
```
19+
warning JD0001: Configuration file 'appsettings.json' should use .json extension for JSON files
20+
```
21+
22+
**Resolution**: Rename the configuration file to use the correct extension, or verify you're using the correct configuration file type.
23+
24+
---
25+
26+
### JD0002: Connection String Missing
27+
**Severity**: Warning
28+
**Task**: AppConfigConnectionStringParser, AppSettingsConnectionStringParser
29+
30+
The specified connection string name was not found in the configuration file.
31+
32+
**Example**:
33+
```
34+
warning JD0002: Connection string 'MyDatabase' not found in appsettings.json
35+
```
36+
37+
**Resolution**: Add the connection string to your configuration file, or verify the connection string name is correct.
38+
39+
---
40+
41+
### JD0003: Connection String Resolution Warning
42+
**Severity**: Warning
43+
**Task**: ConnectionStringResolutionChain
44+
45+
General warning during connection string resolution process.
46+
47+
**Example**:
48+
```
49+
warning JD0003: Failed to resolve connection string from configuration
50+
```
51+
52+
**Resolution**: Check that your configuration file exists and contains valid connection string settings.
53+
54+
---
55+
56+
### JD0011: Configuration File Parse/Read Error
57+
**Severity**: Error
58+
**Task**: AppConfigConnectionStringParser, AppSettingsConnectionStringParser
59+
60+
Failed to parse or read the configuration file.
61+
62+
**Example**:
63+
```
64+
error JD0011: Failed to parse configuration file 'appsettings.json': Unexpected character encountered
65+
```
66+
67+
**Resolution**: Verify the configuration file is valid JSON/XML and not corrupted.
68+
69+
---
70+
71+
### JD0012: Connection String Null or Empty
72+
**Severity**: Error
73+
**Task**: AppSettingsConnectionStringParser
74+
75+
The connection string exists but is null or empty.
76+
77+
**Example**:
78+
```
79+
error JD0012: Connection string 'DefaultConnection' in appsettings.json is null or empty
80+
```
81+
82+
**Resolution**: Provide a valid connection string value in your configuration file.
83+
84+
---
85+
86+
### JD0013: Query Schema Metadata Error (Specific)
87+
**Severity**: Error
88+
**Task**: QuerySchemaMetadata
89+
90+
Specific error when querying database schema metadata.
91+
92+
**Resolution**: Check database connectivity and permissions. Verify the database exists and is accessible.
93+
94+
---
95+
96+
### JD0014: Query Schema Metadata Error (General)
97+
**Severity**: Error
98+
**Task**: QuerySchemaMetadata
99+
100+
General exception when querying database schema metadata.
101+
102+
**Example**:
103+
```
104+
error JD0014: Failed to query database schema metadata: Timeout expired
105+
```
106+
107+
**Resolution**: Check database connectivity, credentials, and network access.
108+
109+
---
110+
111+
### JD0015: SQL Project Warning
112+
**Severity**: Warning
113+
**Task**: ResolveSqlProjAndInputs
114+
115+
Warning related to SQL project resolution.
116+
117+
**Resolution**: Review the warning message for specific guidance.
118+
119+
---
120+
121+
### JD0016: Explicit Connection String Fallback Warning
122+
**Severity**: Warning
123+
**Task**: ResolveSqlProjAndInputs
124+
125+
Explicit connection string configuration provided but failed to resolve, falling back to .sqlproj detection.
126+
127+
**Resolution**: Verify your explicit connection string configuration is correct.
128+
129+
---
130+
131+
## SqlPackage and SQL Generation Errors (JD0020-JD0029)
132+
133+
### JD0020: Explicit Tool Path Does Not Exist
134+
**Severity**: Error
135+
**Task**: RunSqlPackage
136+
137+
The explicitly specified sqlpackage tool path does not exist.
138+
139+
**Example**:
140+
```
141+
error JD0020: Explicit tool path does not exist: C:\tools\sqlpackage.exe
142+
```
143+
144+
**Resolution**:
145+
- Verify the path to sqlpackage.exe is correct
146+
- Install sqlpackage using `dotnet tool install --global microsoft.sqlpackage`
147+
- Remove the `ToolPath` property to use automatic tool resolution
148+
149+
---
150+
151+
### JD0021: Failed to Start SqlPackage Process
152+
**Severity**: Error
153+
**Task**: RunSqlPackage
154+
155+
Failed to start the sqlpackage process.
156+
157+
**Example**:
158+
```
159+
error JD0021: Failed to start sqlpackage process
160+
```
161+
162+
**Resolution**:
163+
- Verify sqlpackage is installed: `dotnet tool list --global`
164+
- Install if missing: `dotnet tool install --global microsoft.sqlpackage`
165+
- Check system PATH includes the dotnet tools directory
166+
167+
---
168+
169+
### JD0022: SqlPackage Failed with Exit Code
170+
**Severity**: Error
171+
**Task**: RunSqlPackage
172+
173+
SqlPackage exited with a non-zero exit code, indicating an error during execution.
174+
175+
**Example**:
176+
```
177+
error JD0022: SqlPackage failed with exit code 1
178+
```
179+
180+
**Resolution**:
181+
- Check the detailed output for specific sqlpackage errors
182+
- Verify the connection string is correct and the database is accessible
183+
- Ensure you have sufficient permissions to read the database schema
184+
- Check network connectivity to the database server
185+
186+
**Common Causes**:
187+
- Invalid connection string
188+
- Database does not exist
189+
- Insufficient permissions
190+
- Network connectivity issues
191+
- Database server is offline
192+
193+
---
194+
195+
### JD0023: SqlPackage Execution Failed (Exception)
196+
**Severity**: Error
197+
**Task**: RunSqlPackage
198+
199+
An unexpected exception occurred while executing sqlpackage.
200+
201+
**Example**:
202+
```
203+
error JD0023: SqlPackage execution failed: Access to the path 'C:\output' is denied
204+
```
205+
206+
**Resolution**:
207+
- Check the exception details in the build output
208+
- Verify file system permissions
209+
- Ensure target directories are writable
210+
- Check for disk space issues
211+
212+
---
213+
214+
### JD0024: Failed to Create Target Directory
215+
**Severity**: Error
216+
**Task**: RunSqlPackage
217+
218+
Failed to create the target directory for SQL script extraction.
219+
220+
**Example**:
221+
```
222+
error JD0024: Failed to create target directory 'C:\output\scripts': Access denied
223+
```
224+
225+
**Resolution**:
226+
- Verify you have write permissions to the parent directory
227+
- Check if the path contains invalid characters
228+
- Ensure the disk has sufficient space
229+
- Check if the path length exceeds system limits (260 characters on Windows)
230+
231+
---
232+
233+
### JD0025: Failed to Add SQL File Warnings
234+
**Severity**: Error
235+
**Task**: AddSqlFileWarnings
236+
237+
An exception occurred while adding auto-generation warning headers to SQL files.
238+
239+
**Example**:
240+
```
241+
error JD0025: Failed to add SQL file warnings: Access to the path is denied
242+
```
243+
244+
**Resolution**:
245+
- Verify file system permissions on the SQL scripts directory
246+
- Ensure SQL files are not read-only or locked by another process
247+
- Check disk space
248+
- Verify the scripts directory path is valid
249+
250+
---
251+
252+
## Troubleshooting Tips
253+
254+
### General Troubleshooting Steps
255+
256+
1. **Check Build Verbosity**: Increase MSBuild verbosity to get more details
257+
```
258+
dotnet build -v:detailed
259+
```
260+
261+
2. **Enable Efcpt Logging**: Set `EfcptLogVerbosity` to `detailed` in your project file
262+
```xml
263+
<PropertyGroup>
264+
<EfcptLogVerbosity>detailed</EfcptLogVerbosity>
265+
</PropertyGroup>
266+
```
267+
268+
3. **Clean and Rebuild**: Sometimes cached state can cause issues
269+
```
270+
dotnet clean
271+
dotnet build
272+
```
273+
274+
4. **Check Permissions**: Ensure you have read/write permissions to all necessary directories
275+
276+
5. **Verify Tool Installation**: For SqlPackage errors, verify the tool is installed
277+
```
278+
dotnet tool list --global
279+
dotnet tool install --global microsoft.sqlpackage
280+
```
281+
282+
### Getting Help
283+
284+
If you encounter an error that isn't documented here or need additional help:
285+
286+
1. Check the [Troubleshooting Guide](troubleshooting.md)
287+
2. Search [existing issues](https://github.com/jerrettdavis/JD.Efcpt.Build/issues)
288+
3. Create a [new issue](https://github.com/jerrettdavis/JD.Efcpt.Build/issues/new) with:
289+
- The full error message including error code
290+
- Build output with verbosity set to `detailed`
291+
- Your project file configuration
292+
- Environment details (OS, .NET version, etc.)

‎samples/aspnet-core-appsettings/MyApp.Api/MyApp.Api.csproj‎

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -26,7 +26,9 @@
2626
<PackageReference Include="JD.Efcpt.Build" Version="*" />
2727

2828
<!-- EF Core SQL Server provider -->
29-
<PackageReference Include="Microsoft.EntityFrameworkCore.SqlServer" Version="9.0.0" />
29+
<PackageReference Include="Microsoft.EntityFrameworkCore" Version="10.0.1" />
30+
<PackageReference Include="Microsoft.EntityFrameworkCore.SqlServer" Version="10.0.1" />
31+
<PackageReference Include="Microsoft.EntityFrameworkCore.SqlServer" Version="10.0.1" />
3032
</ItemGroup>
3133

3234
<PropertyGroup>

‎samples/connection-string-mssql/EntityFrameworkCoreProject/EntityFrameworkCoreProject.csproj‎

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -26,7 +26,9 @@
2626
<PackageReference Include="JD.Efcpt.Build" Version="*" />
2727

2828
<!-- EF Core SQL Server provider -->
29-
<PackageReference Include="Microsoft.EntityFrameworkCore.SqlServer" Version="9.0.0" />
29+
<PackageReference Include="Microsoft.EntityFrameworkCore" Version="10.0.1" />
30+
<PackageReference Include="Microsoft.EntityFrameworkCore.SqlServer" Version="10.0.1" />
31+
<PackageReference Include="Microsoft.EntityFrameworkCore.SqlServer" Version="10.0.1" />
3032
</ItemGroup>
3133

3234
<PropertyGroup>

‎samples/custom-renaming/EntityFrameworkCoreProject/EntityFrameworkCoreProject.csproj‎

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -24,7 +24,9 @@
2424
<ItemGroup>
2525
<!-- Reference the local development package -->
2626
<PackageReference Include="JD.Efcpt.Build" Version="*" />
27-
<PackageReference Include="Microsoft.EntityFrameworkCore.SqlServer" Version="9.0.0" />
27+
<PackageReference Include="Microsoft.EntityFrameworkCore" Version="10.0.1" />
28+
<PackageReference Include="Microsoft.EntityFrameworkCore.SqlServer" Version="10.0.1" />
29+
<PackageReference Include="Microsoft.EntityFrameworkCore.SqlServer" Version="10.0.1" />
2830
</ItemGroup>
2931

3032
<PropertyGroup>

‎samples/dacpac-zero-config/EntityFrameworkCoreProject/EntityFrameworkCoreProject.csproj‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -10,7 +10,7 @@
1010

1111
<ItemGroup>
1212
<PackageReference Include="JD.Efcpt.Build" Version="*" />
13-
<PackageReference Include="Microsoft.EntityFrameworkCore" Version="8.0.11" />
14-
<PackageReference Include="Microsoft.EntityFrameworkCore.SqlServer" Version="8.0.11" />
13+
<PackageReference Include="Microsoft.EntityFrameworkCore" Version="10.0.1" />
14+
<PackageReference Include="Microsoft.EntityFrameworkCore.SqlServer" Version="10.0.1" />
1515
</ItemGroup>
1616
</Project>

0 commit comments

Comments
 (0)