Skip to content

Commit 2c917dc

Browse files
MelbourneDeveloperMelbourneDeveloper
committed
Add specs
1 parent 8cdabcc commit 2c917dc

2 files changed

Lines changed: 303 additions & 0 deletions

File tree

‎DATAPROVIDER_CURRENT_SPEC.md‎

Lines changed: 107 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,107 @@
1+
# DataProvider Current Functionality Specification
2+
3+
## Overview
4+
DataProvider is a source generator that creates compile-time safe database extension methods from SQL queries, providing zero-overhead data access with full type safety.
5+
6+
## Core Features
7+
8+
### 1. Source Generation
9+
- **Incremental Source Generators**: Uses Roslyn source generators for compile-time code generation
10+
- **SQL File Processing**: Processes `.sql` files in the project to generate C# extension methods
11+
- **LQL Integration**: Automatically processes `.lql` files through transpilation to SQL
12+
- **Schema Inspection**: Inspects database schema at build time to generate appropriate types
13+
14+
### 2. Database Support
15+
- **SQLite**: Full implementation with ANTLR parsing and schema inspection
16+
- **SQL Server**: Implementation with schema inspection and query parsing
17+
- **PostgreSQL**: Planned (in roadmap)
18+
19+
### 3. Type Safety
20+
- **Result Type Pattern**: All operations return `Result<T>` for explicit error handling (no exceptions)
21+
- **Nullable Reference Types**: Full support for nullable reference types
22+
- **Compile-Time Validation**: SQL queries validated during compilation against actual schema
23+
- **AOT Compatible**: Full ahead-of-time compilation support with no runtime reflection
24+
25+
### 4. Generated Code Architecture
26+
- **Extension Methods**: Generates static extension methods on `IDbConnection` and `IDbTransaction`
27+
- **Functional Style**: Pure static methods with no classes (only records)
28+
- **Expression-Based**: Prefers expressions over statements
29+
- **No Side Effects**: Pure functions with predictable behavior
30+
31+
### 5. Query Features
32+
- **Parameterized Queries**: Full support for SQL parameters with type safety
33+
- **CRUD Operations**: Auto-generated Insert/Update/Delete/Select methods
34+
- **Complex Joins**: Support for multi-table joins with result grouping
35+
- **Transactions**: Full transaction support through `IDbTransaction` extensions
36+
- **Async/Await**: All database operations support async patterns
37+
38+
### 6. LINQ Integration (Recent Addition)
39+
- **LINQ Select Expressions**: Support for LINQ-style select queries
40+
- **Expression Trees**: Converts LINQ expressions to SQL
41+
- **Type-Safe Queries**: Compile-time checking of LINQ queries
42+
- **Predicate Building**: Complex WHERE clause construction
43+
44+
### 7. Code Generation Templates
45+
- **ICodeTemplate Interface**: Basic template abstraction for code generation
46+
- **DefaultCodeTemplate**: Standard implementation for generating extension methods
47+
- **Database-Specific Templates**: SQLite and SQL Server specific code generation
48+
49+
### 8. Schema Features
50+
- **Table Discovery**: Automatic discovery of database tables
51+
- **Column Type Mapping**: Maps SQL types to C# types
52+
- **Primary Key Detection**: Identifies primary keys for CRUD operations
53+
- **Foreign Key Recognition**: Understands relationships for joins
54+
55+
### 9. Configuration
56+
- **DataProvider.json**: Configuration file for specifying queries and settings
57+
- **Grouping Configuration**: `.grouping.json` files for complex result set mapping
58+
- **Namespace Control**: Configurable namespace generation
59+
60+
### 10. Testing Support
61+
- **Integration Tests**: Comprehensive test suite for database operations
62+
- **Sample Data Seeding**: Built-in data seeding for testing
63+
- **Multiple Test Databases**: Support for testing against different database engines
64+
65+
## Implementation Details
66+
67+
### Parser Implementation
68+
- **ANTLR Grammar**: Uses ANTLR for SQL parsing (SQLite)
69+
- **Custom SQL Parsers**: Database-specific parsing implementations
70+
- **Parameter Extraction**: Automatic detection and typing of SQL parameters
71+
72+
### Code Generation Pipeline
73+
1. SQL file discovery during build
74+
2. Database schema inspection
75+
3. SQL parsing and validation
76+
4. C# code generation
77+
5. Compilation integration
78+
79+
### Error Handling
80+
- **Result<T,E> Pattern**: All operations return success/failure results
81+
- **No Exceptions**: Exception-free architecture for predictable behavior
82+
- **Detailed Error Messages**: Comprehensive error information for debugging
83+
84+
## File Structure
85+
```
86+
DataProvider/
87+
├── CodeGeneration/
88+
│ ├── ICodeTemplate.cs
89+
│ ├── DefaultCodeTemplate.cs
90+
│ └── DataAccessGenerator.cs
91+
├── DbConnectionExtensions.cs
92+
├── ICodeGenerator.cs
93+
└── Database-specific implementations
94+
```
95+
96+
## Usage Pattern
97+
```csharp
98+
// Generated extension method from SQL file
99+
var result = await connection.GetCustomersAsync(isActive: true);
100+
if (result.IsSuccess)
101+
{
102+
foreach (var customer in result.Value)
103+
{
104+
// Type-safe access to customer properties
105+
}
106+
}
107+
```

‎LQL_CURRENT_SPEC.md‎

Lines changed: 196 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,196 @@
1+
# LQL (Lambda Query Language) Current Functionality Specification
2+
3+
## Overview
4+
LQL is a functional pipeline-style DSL that transpiles to procedural SQL, providing an intuitive and composable way to write database queries using lambda expressions and pipeline operators. While it provides a cross platform way to write SQL queries, it really shines as a way of writing procedural SQL code in a more functional way.
5+
6+
## Core Features
7+
8+
### 1. Pipeline Syntax
9+
- **Pipeline Operator (`|>`)**: Chain operations in a functional style
10+
- **Composition**: Build complex queries from simple operations
11+
- **Readability**: Linear flow matches thought process
12+
- **Immutability**: Each operation returns a new query state
13+
14+
### 2. Database Support
15+
- **SQLite**: Full transpilation support with C# function generation
16+
- **SQL Server**: Complete dialect implementation
17+
- **PostgreSQL**: Full support with PostgreSQL-specific features
18+
- **Cross-Database**: Write once, transpile to multiple SQL dialects
19+
20+
### 3. Query Operations
21+
22+
#### Basic Operations
23+
- **select()**: Column selection with aliasing support
24+
- **filter()**: WHERE clause with lambda expressions
25+
- **distinct()**: Remove duplicate rows
26+
- **limit()**: Row limiting
27+
- **offset()**: Row skipping for pagination
28+
29+
#### Join Operations
30+
- **join()**: Inner joins with ON conditions
31+
- **left_join()**: Left outer joins
32+
- **right_join()**: Right outer joins (dialect-specific)
33+
- **cross_join()**: Cartesian product
34+
35+
#### Grouping & Aggregation
36+
- **group_by()**: Group rows by columns
37+
- **having()**: Filter grouped results
38+
- **Aggregate Functions**: COUNT, SUM, AVG, MIN, MAX
39+
40+
#### Set Operations
41+
- **union()**: Combine queries (distinct)
42+
- **union_all()**: Combine queries (with duplicates)
43+
- **intersect()**: Common rows between queries
44+
- **except()**: Rows in first query not in second
45+
46+
#### Ordering
47+
- **order_by()**: Sort results (ASC/DESC)
48+
- **Multiple columns**: Support for complex sorting
49+
50+
### 4. Lambda Expression Support
51+
- **Filter Predicates**: `fn(row) => row.column = value`
52+
- **Complex Conditions**: AND, OR, NOT operators
53+
- **Comparison Operators**: =, !=, <, >, <=, >=
54+
- **Pattern Matching**: LIKE, NOT LIKE
55+
- **IN Operator**: Check membership in lists
56+
- **NULL Handling**: IS NULL, IS NOT NULL
57+
58+
### 5. Function Support
59+
60+
#### String Functions
61+
- UPPER(), LOWER()
62+
- LENGTH(), CONCAT()
63+
- SUBSTRING(), TRIM()
64+
65+
#### Date Functions
66+
- NOW(), DATE()
67+
- YEAR(), MONTH(), DAY()
68+
- Date arithmetic
69+
70+
#### Mathematical Functions
71+
- Basic arithmetic: +, -, *, /
72+
- Modulo operator: %
73+
- Mathematical functions per dialect
74+
75+
#### Conditional Functions
76+
- CASE WHEN expressions
77+
- COALESCE()
78+
- NULLIF()
79+
80+
### 6. Advanced Features
81+
82+
#### Subqueries
83+
- **IN subqueries**: Filter based on subquery results
84+
- **Scalar subqueries**: Single value subqueries
85+
- **Correlated subqueries**: Reference outer query
86+
87+
#### Variables & Let Bindings
88+
- **let statements**: Define reusable query fragments
89+
- **Variable references**: Use defined queries in expressions
90+
91+
#### CTEs (Common Table Expressions)
92+
- **WITH clause**: Define temporary named result sets
93+
- **Recursive CTEs**: Planned feature
94+
95+
### 7. Tooling
96+
97+
#### CLI Tool (LqlCli)
98+
- **Transpilation**: Convert .lql to .sql files
99+
- **Validation**: Syntax checking without transpilation
100+
- **Console Output**: Quick testing of queries
101+
- **File Processing**: Batch processing of LQL files
102+
103+
#### VS Code Extension
104+
- **Syntax Highlighting**: LQL-specific syntax colors
105+
- **IntelliSense**: Auto-completion for keywords
106+
- **Error Diagnostics**: Real-time syntax validation
107+
- **Snippets**: Common query patterns
108+
- **Format on Save**: Code formatting
109+
110+
#### Browser Playground
111+
- **Interactive Editor**: Write and test LQL in browser
112+
- **Live Transpilation**: See SQL output in real-time
113+
- **Multiple Dialects**: Switch between SQL targets
114+
- **Schema Explorer**: Browse database structure
115+
- **Example Queries**: Learn from examples
116+
117+
### 8. Parser Implementation
118+
119+
#### ANTLR Grammar
120+
- **Lexer Rules**: Token definitions
121+
- **Parser Rules**: Grammar structure
122+
- **Visitor Pattern**: AST traversal and transformation
123+
124+
#### AST (Abstract Syntax Tree)
125+
- **Statement Types**: Query, expression, operation nodes
126+
- **Type System**: Basic type inference
127+
- **Validation**: Semantic analysis during parsing
128+
129+
### 9. Transpilation Pipeline
130+
1. **Lexical Analysis**: Tokenize LQL input
131+
2. **Parsing**: Build AST from tokens
132+
3. **Semantic Analysis**: Validate and type-check
133+
4. **Optimization**: Query optimization passes
134+
5. **SQL Generation**: Dialect-specific SQL output
135+
136+
### 10. Integration with DataProvider
137+
- **Automatic Processing**: .lql files processed during build
138+
- **Code Generation**: Extension methods from LQL queries
139+
- **Type Safety**: Compile-time validation
140+
- **Parameter Binding**: Automatic parameter detection
141+
142+
## Implementation Architecture
143+
144+
### Core Components
145+
```
146+
Lql/
147+
├── Parsing/
148+
│ ├── Lql.g4 (ANTLR grammar)
149+
│ ├── LqlLexer.cs
150+
│ ├── LqlParser.cs
151+
│ └── LqlToAstVisitor.cs
152+
├── AST/
153+
│ ├── Statement types
154+
│ └── Expression types
155+
├── PipelineProcessor.cs
156+
└── Dialect implementations
157+
```
158+
159+
### Dialect Structure
160+
Each dialect (SQLite, SQL Server, PostgreSQL) provides:
161+
- SQL generation context
162+
- Function mappings
163+
- Type mappings
164+
- Dialect-specific features
165+
166+
## Current Limitations
167+
- No window functions
168+
- Limited CTE support
169+
- No stored procedure generation
170+
- No trigger generation (planned)
171+
- No database-specific optimizations
172+
- Limited type inference
173+
174+
## Usage Examples
175+
176+
### Simple Query
177+
```lql
178+
users |> select(id, name, email)
179+
```
180+
181+
### Complex Query
182+
```lql
183+
let active_orders = Order
184+
|> filter(fn(row) => row.status = 'active')
185+
|> select(*)
186+
187+
Customer
188+
|> join(active_orders, on = Customer.Id = active_orders.CustomerId)
189+
|> group_by(Customer.Id, Customer.Name)
190+
|> having(fn(row) => COUNT(*) > 5)
191+
|> select(Customer.Name, COUNT(*) AS OrderCount)
192+
|> order_by(OrderCount DESC)
193+
```
194+
195+
## Transpilation Output
196+
LQL queries are transpiled to optimized, dialect-specific SQL maintaining semantic equivalence while leveraging database-specific features where appropriate.

0 commit comments

Comments
 (0)