Skip to content

Commit 315a4ae

Browse files
Ajit Pratap Singhclaude
authored andcommitted
feat: add stdin/stdout pipeline support (closes #65)
Implement comprehensive stdin/stdout pipeline support for all CLI commands (validate, format, analyze, parse) with Unix pipeline conventions and cross-platform compatibility. Features: - Auto-detection: Commands automatically detect piped input - Explicit stdin: Support "-" as stdin marker for all commands - Input redirection: Full support for "< file.sql" syntax - Broken pipe handling: Graceful handling of Unix EPIPE errors - Security: 10MB input limit to prevent DoS attacks - Cross-platform: Works on Unix/Linux/macOS and Windows PowerShell Implementation: - Created stdin_utils.go with pipeline utilities: - IsStdinPipe(): Detects piped input using golang.org/x/term - ReadFromStdin(): Reads from stdin with size limits - GetInputSource(): Unified input detection (stdin/file/direct SQL) - WriteOutput(): Handles stdout and file output with broken pipe detection - DetectInputMode(): Determines input mode based on args and stdin state - ValidateStdinInput(): Security validation for stdin content - Updated all commands with stdin support: - validate.go: Stdin validation with temp file approach - format.go: Stdin formatting (blocks -i flag appropriately) - analyze.go: Stdin analysis with direct content processing - parse.go: Stdin parsing with direct content processing - Dependencies: - Added golang.org/x/term for stdin detection - Testing: - Unit tests: stdin_utils_test.go with comprehensive coverage - Integration tests: pipeline_integration_test.go for real pipeline testing - Manual testing: Validated echo, cat, and redirect operations - Documentation: - Updated README.md with comprehensive pipeline examples - Unix/Linux/macOS and Windows PowerShell examples - Git hooks integration examples Usage Examples: echo "SELECT * FROM users" | gosqlx validate cat query.sql | gosqlx format gosqlx validate - gosqlx format < query.sql cat query.sql | gosqlx format | gosqlx validate Cross-platform: # Unix/Linux/macOS cat query.sql | gosqlx format | tee formatted.sql | gosqlx validate # Windows PowerShell Get-Content query.sql | gosqlx format | Set-Content formatted.sql "SELECT * FROM users" | gosqlx validate Security: - 10MB stdin size limit (MaxStdinSize constant) - Binary data detection (null byte check) - Input validation before processing - Temporary file cleanup in validate command 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
1 parent 7a26b14 commit 315a4ae

11 files changed

Lines changed: 1175 additions & 8 deletions

File tree

README.md

Lines changed: 42 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -105,6 +105,8 @@ go build -o gosqlx ./cmd/gosqlx
105105
## 🚀 Quick Start
106106

107107
### CLI Usage
108+
109+
**Standard Usage:**
108110
```bash
109111
# Validate SQL syntax
110112
gosqlx validate "SELECT * FROM users WHERE active = true"
@@ -119,6 +121,46 @@ gosqlx analyze "SELECT COUNT(*) FROM orders GROUP BY status"
119121
gosqlx parse -f json complex_query.sql
120122
```
121123

124+
**Pipeline/Stdin Support** (New in v1.6.0):
125+
```bash
126+
# Auto-detect piped input
127+
echo "SELECT * FROM users" | gosqlx validate
128+
cat query.sql | gosqlx format
129+
cat complex.sql | gosqlx analyze --security
130+
131+
# Explicit stdin marker
132+
gosqlx validate -
133+
gosqlx format - < query.sql
134+
135+
# Input redirection
136+
gosqlx validate < query.sql
137+
gosqlx parse < complex_query.sql
138+
139+
# Full pipeline chains
140+
cat query.sql | gosqlx format | gosqlx validate
141+
echo "select * from users" | gosqlx format > formatted.sql
142+
find . -name "*.sql" -exec cat {} \; | gosqlx validate
143+
144+
# Works on Windows PowerShell too!
145+
Get-Content query.sql | gosqlx format
146+
"SELECT * FROM users" | gosqlx validate
147+
```
148+
149+
**Cross-Platform Pipeline Examples:**
150+
```bash
151+
# Unix/Linux/macOS
152+
cat query.sql | gosqlx format | tee formatted.sql | gosqlx validate
153+
echo "SELECT 1" | gosqlx validate && echo "Valid!"
154+
155+
# Windows PowerShell
156+
Get-Content query.sql | gosqlx format | Set-Content formatted.sql
157+
"SELECT * FROM users" | gosqlx validate
158+
159+
# Git hooks (pre-commit)
160+
git diff --cached --name-only --diff-filter=ACM "*.sql" | \
161+
xargs cat | gosqlx validate --quiet
162+
```
163+
122164
### Library Usage - Simple API
123165

124166
GoSQLX provides a simple, high-level API that handles all complexity for you:

cmd/gosqlx/cmd/analyze.go

Lines changed: 71 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,8 @@
11
package cmd
22

33
import (
4+
"fmt"
5+
46
"github.com/spf13/cobra"
57
"github.com/spf13/pflag"
68

@@ -28,20 +30,34 @@ Examples:
2830
gosqlx analyze --all query.sql # Comprehensive analysis
2931
gosqlx analyze "SELECT * FROM users" # Analyze query directly
3032
33+
Pipeline/Stdin Examples:
34+
echo "SELECT * FROM users" | gosqlx analyze # Analyze from stdin (auto-detect)
35+
cat query.sql | gosqlx analyze # Pipe file contents
36+
gosqlx analyze - # Explicit stdin marker
37+
gosqlx analyze < query.sql # Input redirection
38+
3139
Analysis capabilities:
3240
• SQL injection pattern detection
3341
• Performance optimization suggestions
34-
• Query complexity scoring
42+
• Query complexity scoring
3543
• Best practices validation
3644
• Multi-dialect compatibility checks
3745
3846
Note: Advanced analysis features are implemented in Phase 4 of the roadmap.
3947
This is a basic implementation for CLI foundation.`,
40-
Args: cobra.ExactArgs(1),
48+
Args: cobra.MaximumNArgs(1), // Changed to allow stdin with no args
4149
RunE: analyzeRun,
4250
}
4351

4452
func analyzeRun(cmd *cobra.Command, args []string) error {
53+
// Handle stdin input
54+
if len(args) == 0 || (len(args) == 1 && args[0] == "-") {
55+
if ShouldReadFromStdin(args) {
56+
return analyzeFromStdin(cmd)
57+
}
58+
return fmt.Errorf("no input provided: specify file path, SQL query, or pipe via stdin")
59+
}
60+
4561
// Load configuration with CLI flag overrides
4662
cfg, err := config.LoadDefault()
4763
if err != nil {
@@ -83,6 +99,59 @@ func analyzeRun(cmd *cobra.Command, args []string) error {
8399
return analyzer.DisplayReport(result.Report)
84100
}
85101

102+
// analyzeFromStdin handles analysis from stdin input
103+
func analyzeFromStdin(cmd *cobra.Command) error {
104+
// Read from stdin
105+
content, err := ReadFromStdin()
106+
if err != nil {
107+
return fmt.Errorf("failed to read from stdin: %w", err)
108+
}
109+
110+
// Validate stdin content
111+
if err := ValidateStdinInput(content); err != nil {
112+
return fmt.Errorf("stdin validation failed: %w", err)
113+
}
114+
115+
// Load configuration
116+
cfg, err := config.LoadDefault()
117+
if err != nil {
118+
cfg = config.DefaultConfig()
119+
}
120+
121+
// Track which flags were explicitly set
122+
flagsChanged := make(map[string]bool)
123+
cmd.Flags().Visit(func(f *pflag.Flag) {
124+
flagsChanged[f.Name] = true
125+
})
126+
if cmd.Parent() != nil && cmd.Parent().PersistentFlags() != nil {
127+
cmd.Parent().PersistentFlags().Visit(func(f *pflag.Flag) {
128+
flagsChanged[f.Name] = true
129+
})
130+
}
131+
132+
// Create analyzer options
133+
opts := AnalyzerOptionsFromConfig(cfg, flagsChanged, AnalyzerFlags{
134+
Security: analyzeSecurity,
135+
Performance: analyzePerformance,
136+
Complexity: analyzeComplexity,
137+
All: analyzeAll,
138+
Format: format,
139+
Verbose: verbose,
140+
})
141+
142+
// Create analyzer
143+
analyzer := NewAnalyzer(cmd.OutOrStdout(), cmd.ErrOrStderr(), opts)
144+
145+
// Analyze the stdin content (Analyze accepts string input directly)
146+
result, err := analyzer.Analyze(string(content))
147+
if err != nil {
148+
return err
149+
}
150+
151+
// Display the report
152+
return analyzer.DisplayReport(result.Report)
153+
}
154+
86155
func init() {
87156
rootCmd.AddCommand(analyzeCmd)
88157

cmd/gosqlx/cmd/format.go

Lines changed: 96 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,7 @@
11
package cmd
22

33
import (
4+
"fmt"
45
"os"
56

67
"github.com/spf13/cobra"
@@ -34,12 +35,29 @@ Examples:
3435
gosqlx format "*.sql" # Format all SQL files
3536
gosqlx format -o formatted.sql query.sql # Save to specific file
3637
38+
Pipeline/Stdin Examples:
39+
echo "SELECT * FROM users" | gosqlx format # Format from stdin (auto-detect)
40+
cat query.sql | gosqlx format # Pipe file contents
41+
gosqlx format - # Explicit stdin marker
42+
gosqlx format < query.sql # Input redirection
43+
cat query.sql | gosqlx format > formatted.sql # Full pipeline
44+
3745
Performance: 100x faster than SQLFluff for equivalent operations`,
38-
Args: cobra.MinimumNArgs(1),
46+
Args: cobra.MinimumNArgs(0), // Changed to allow stdin with no args
3947
RunE: formatRun,
4048
}
4149

4250
func formatRun(cmd *cobra.Command, args []string) error {
51+
// Handle stdin input
52+
if ShouldReadFromStdin(args) {
53+
return formatFromStdin(cmd)
54+
}
55+
56+
// Validate that we have file arguments if not using stdin
57+
if len(args) == 0 {
58+
return fmt.Errorf("no input provided: specify file paths or pipe SQL via stdin")
59+
}
60+
4361
// Load configuration with CLI flag overrides
4462
cfg, err := config.LoadDefault()
4563
if err != nil {
@@ -87,6 +105,83 @@ func formatRun(cmd *cobra.Command, args []string) error {
87105
return nil
88106
}
89107

108+
// formatFromStdin handles formatting from stdin input
109+
func formatFromStdin(cmd *cobra.Command) error {
110+
// Read from stdin
111+
content, err := ReadFromStdin()
112+
if err != nil {
113+
return fmt.Errorf("failed to read from stdin: %w", err)
114+
}
115+
116+
// Validate stdin content
117+
if err := ValidateStdinInput(content); err != nil {
118+
return fmt.Errorf("stdin validation failed: %w", err)
119+
}
120+
121+
// Note: in-place mode is not supported for stdin (would be no-op)
122+
if formatInPlace {
123+
return fmt.Errorf("in-place mode (-i) is not supported with stdin input")
124+
}
125+
126+
// Load configuration
127+
cfg, err := config.LoadDefault()
128+
if err != nil {
129+
cfg = config.DefaultConfig()
130+
}
131+
132+
// Track which flags were explicitly set
133+
flagsChanged := make(map[string]bool)
134+
cmd.Flags().Visit(func(f *pflag.Flag) {
135+
flagsChanged[f.Name] = true
136+
})
137+
if cmd.Parent() != nil && cmd.Parent().PersistentFlags() != nil {
138+
cmd.Parent().PersistentFlags().Visit(func(f *pflag.Flag) {
139+
flagsChanged[f.Name] = true
140+
})
141+
}
142+
143+
// Create formatter options
144+
opts := FormatterOptionsFromConfig(cfg, flagsChanged, FormatterFlags{
145+
InPlace: false, // always false for stdin
146+
IndentSize: formatIndentSize,
147+
Uppercase: formatUppercase,
148+
Compact: formatCompact,
149+
Check: formatCheck,
150+
MaxLine: formatMaxLine,
151+
Verbose: verbose,
152+
Output: outputFile,
153+
})
154+
155+
// Create formatter
156+
formatter := NewFormatter(cmd.OutOrStdout(), cmd.ErrOrStderr(), opts)
157+
158+
// Format the SQL content using the internal formatSQL method
159+
formattedSQL, err := formatter.formatSQL(string(content))
160+
if err != nil {
161+
return fmt.Errorf("formatting failed: %w", err)
162+
}
163+
164+
// In check mode, compare original and formatted
165+
if formatCheck {
166+
if string(content) != formattedSQL {
167+
fmt.Fprintf(cmd.ErrOrStderr(), "stdin needs formatting\n")
168+
os.Exit(1)
169+
} else {
170+
if verbose {
171+
fmt.Fprintf(cmd.OutOrStdout(), "stdin is properly formatted\n")
172+
}
173+
}
174+
return nil
175+
}
176+
177+
// Write formatted output
178+
if err := WriteOutput([]byte(formattedSQL), outputFile, cmd.OutOrStdout()); err != nil {
179+
return err
180+
}
181+
182+
return nil
183+
}
184+
90185
func init() {
91186
rootCmd.AddCommand(formatCmd)
92187

cmd/gosqlx/cmd/parse.go

Lines changed: 74 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,8 @@
11
package cmd
22

33
import (
4+
"fmt"
5+
46
"github.com/spf13/cobra"
57
"github.com/spf13/pflag"
68

@@ -29,13 +31,27 @@ Examples:
2931
gosqlx parse -f yaml query.sql # YAML output format
3032
gosqlx parse "SELECT * FROM users WHERE id=1" # Parse query directly
3133
34+
Pipeline/Stdin Examples:
35+
echo "SELECT * FROM users" | gosqlx parse # Parse from stdin (auto-detect)
36+
cat query.sql | gosqlx parse # Pipe file contents
37+
gosqlx parse - # Explicit stdin marker
38+
gosqlx parse < query.sql # Input redirection
39+
3240
Output formats: json, yaml, table, tree
3341
Performance: Direct AST inspection without intermediate representations`,
34-
Args: cobra.ExactArgs(1),
42+
Args: cobra.MaximumNArgs(1), // Changed to allow stdin with no args
3543
RunE: parseRun,
3644
}
3745

3846
func parseRun(cmd *cobra.Command, args []string) error {
47+
// Handle stdin input
48+
if len(args) == 0 || (len(args) == 1 && args[0] == "-") {
49+
if ShouldReadFromStdin(args) {
50+
return parseFromStdin(cmd)
51+
}
52+
return fmt.Errorf("no input provided: specify file path, SQL query, or pipe via stdin")
53+
}
54+
3955
// Load configuration with CLI flag overrides
4056
cfg, err := config.LoadDefault()
4157
if err != nil {
@@ -81,6 +97,63 @@ func parseRun(cmd *cobra.Command, args []string) error {
8197
return parser.Display(result)
8298
}
8399

100+
// parseFromStdin handles parsing from stdin input
101+
func parseFromStdin(cmd *cobra.Command) error {
102+
// Read from stdin
103+
content, err := ReadFromStdin()
104+
if err != nil {
105+
return fmt.Errorf("failed to read from stdin: %w", err)
106+
}
107+
108+
// Validate stdin content
109+
if err := ValidateStdinInput(content); err != nil {
110+
return fmt.Errorf("stdin validation failed: %w", err)
111+
}
112+
113+
// Load configuration
114+
cfg, err := config.LoadDefault()
115+
if err != nil {
116+
cfg = config.DefaultConfig()
117+
}
118+
119+
// Track which flags were explicitly set
120+
flagsChanged := make(map[string]bool)
121+
cmd.Flags().Visit(func(f *pflag.Flag) {
122+
flagsChanged[f.Name] = true
123+
})
124+
if cmd.Parent() != nil && cmd.Parent().PersistentFlags() != nil {
125+
cmd.Parent().PersistentFlags().Visit(func(f *pflag.Flag) {
126+
flagsChanged[f.Name] = true
127+
})
128+
}
129+
130+
// Create parser options
131+
opts := ParserOptionsFromConfig(cfg, flagsChanged, ParserFlags{
132+
ShowAST: parseShowAST,
133+
ShowTokens: parseShowTokens,
134+
TreeView: parseTreeView,
135+
Format: format,
136+
Verbose: verbose,
137+
})
138+
139+
// Create parser
140+
parser := NewParser(cmd.OutOrStdout(), cmd.ErrOrStderr(), opts)
141+
142+
// Parse the stdin content (Parse accepts string input directly)
143+
result, err := parser.Parse(string(content))
144+
if err != nil {
145+
return err
146+
}
147+
148+
// CRITICAL: Always release AST if it was created
149+
if result.AST != nil {
150+
defer ast.ReleaseAST(result.AST)
151+
}
152+
153+
// Display the result
154+
return parser.Display(result)
155+
}
156+
84157
func init() {
85158
rootCmd.AddCommand(parseCmd)
86159

0 commit comments

Comments
 (0)