feat: add semantic context and SQL intent planning to the generation pipeline#42
Merged
Merged
Conversation
…pipeline Introduce a first-class semantic business layer, structured SQL intent planning, semantic validation, CLI artifact loading, and local e-commerce validation for the DSL-first nlp2sql workflow. - add SemanticContext, SqlIntentPlan, SemanticValidationResult, and related entities - add semantic resolver and validator ports plus dict/file/noop adapters - upgrade QueryGenerationService with semantic resolution, intent planning, semantic validation, and repair-aware orchestration - expose semantic_hooks and semantic_context through connect() and ask() - add CLI support for semantic/example artifacts and runtime metadata display - expand local e-commerce integration coverage and sanitize public tests/docs - bump version to 0.2.0rc13 Closes #41 Closes #34
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This PR upgrades
nlp2sqlfrom a mostly schema-and-question driven pipeline to a business-aware generation pipeline:question -> semantic resolution -> schema/examples retrieval -> SQL intent plan -> prompt assembly -> SQL -> semantic/execution validation -> optional repairIt introduces a first-class semantic layer, structured intent planning before generation, semantic validation after generation, CLI support for semantic/example artifacts, and stronger public/local validation around the repository's e-commerce example domain.
What changed
Semantic domain model
Added first-class semantic entities:
SemanticContextSemanticEntityMappingMetricDefinitionDimensionDefinitionDomainRuleCanonicalQueryPatternSqlIntentPlanSemanticIssueSemanticValidationResultPorts and adapters
Added new extension points:
SemanticResolverPortSemanticValidatorPortAdded initial adapters:
NoOpSemanticResolverNoOpSemanticValidatorDictSemanticResolverFileSemanticResolverService orchestration
Upgraded
QueryGenerationServiceto orchestrate:Added supporting services:
SemanticResolutionServiceSqlIntentPlanningServiceSemanticValidationServicePromptAssemblyServiceExampleSelectionServiceQueryAnalysisServiceQueryRepairServicePublic API
Exposed semantic support through the DSL:
connect(..., semantic_hooks=..., semantic_context=...)nlp.ask(..., semantic_context=...)Kept execution hooks and semantic hooks separate so downstream services can control them independently.
CLI
Added support for:
--semantic-context-file--semantic-context-json--examples-file--examples-json--show-semantic-context--show-sql-intent-plan--show-selected-examples--validate--repairCentralized artifact loading in
utils/artifact_loader.py.Prompting and metadata
Improved provider prompt rendering with richer business context and structured SQL intent plan metadata.
Added richer result metadata including:
semantic_contextsql_intent_planselected_examplesrepair_attemptsexecution_validationLocal integration validation
Expanded the local PostgreSQL e-commerce schema to better validate semantic disambiguation.
Added regression-style integration coverage showing semantic context can move generation from a plausible transactional path to the intended aggregate fact table.
Sanitized automated tests and public docs to avoid private warehouse identifiers.
Docs and versioning
Refreshed public docs so they match the current DSL-first, semantic-aware architecture.
Bumped version to
0.2.0rc13.Why
Few-shot examples alone were not enough to capture business meaning such as:
This PR adds the missing semantic/business layer directly to the library while preserving the hexagonal architecture and keeping the public API ergonomic.
Validation
Ran:
uv lockuv syncuv run python -c "from nlp2sql import __version__; print(__version__)"uv run pytest -m "integration and not llm"uv run pytest -m "integration and llm"Results:
0.2.0rc13integration and not llm:21 passed, 6 skippedintegration and llm:3 passedNotes:
Follow-up
After this PR, the next step is to consume
nlp2sqlthrough a downstreamuvsource dependency and verify compatibility with an existing integration.Closes #41
Closes #34