Skip to content

refactor: parser#322

Merged
psteinroe merged 21 commits intomainfrom
refactor/aprser
Apr 11, 2025
Merged

refactor: parser#322
psteinroe merged 21 commits intomainfrom
refactor/aprser

Conversation

@psteinroe
Copy link
Copy Markdown
Collaborator

@psteinroe psteinroe commented Apr 9, 2025

this rewires the workspace server for better DX and adds support for sql function bodies.

closes #178

hear me out:

  • the Document is responsible for extracting statements from a source file. right now, this is always SQL, but in the future we could make this generic.
  • the Parser is the new layer between the workspace server and the document. there is one parser per document, and the parser owns the document.
  • the parser is responsible for parsing the raw sql coming from the document, and is the main API layer for the workspace.
  • the consumer of the parser apis simply gets the statements and the related parse results. the consumer does not need to know anything about sub statements
  • I added a nice pattern where the consumer of the Parser can define a mapper and / or a filter and then simply calls parser.iter(ExecuteStatementMapper, CursorPositionFilter::new(pos)) and gets exactly the data it needs.
  • I also optimised the parsing in general: the only thing that actually cares about modified statements is tree sitter. for pg_query, we always just remove and add when we receive a modify statement. hence, we can simply lazily parse all but tree sitter and just use modified / remove notifications to clear the cache. this also opened up the way for a nicer api because if you want the pg_query results, you always get Result<Node, SyntaxError> and we do not need to wrap it in an Option<> anymore.
  • the statement id is now not just an usize anymore but an enum that can reflect sub statements. if we remove parse results for a parent, we also remove it for its sub statement.

@psteinroe psteinroe marked this pull request as ready for review April 11, 2025 10:53
Comment thread crates/pgt_workspace/src/workspace/server/parser.rs
Comment thread crates/pgt_workspace/src/workspace/server/parser.rs Outdated
Comment thread crates/pgt_workspace/src/workspace/server/parser.rs
Comment thread crates/pgt_workspace/src/workspace/server/parser.rs Outdated
Comment thread crates/pgt_workspace/src/workspace/server.rs
Comment thread crates/pgt_completions/src/complete.rs
Copy link
Copy Markdown
Collaborator

@juleswritescode juleswritescode left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great PR! :)
I was scared that accomodating nested statements would make things more complex, but this even makes it simpler!!

Comment thread crates/pgt_workspace/src/workspace/server/document.rs
Comment thread crates/pgt_workspace/src/workspace/server/pg_query.rs Outdated
Comment thread crates/pgt_workspace/src/workspace/server/parser.rs Outdated
Comment thread crates/pgt_workspace/src/workspace/server/parser.rs
Comment thread crates/pgt_workspace/src/workspace/server/parser.rs Outdated
Comment thread crates/pgt_workspace/src/workspace/server.rs
Comment thread crates/pgt_workspace/src/workspace/server/sql_function.rs
Comment thread crates/pgt_workspace/src/workspace/server/statement_identifier.rs
Comment thread crates/pgt_workspace/src/workspace/server/statement_identifier.rs Outdated
Comment thread crates/pgt_workspace/src/workspace/server/statement_identifier.rs Outdated
@psteinroe
Copy link
Copy Markdown
Collaborator Author

@juleswritescode for me ParsedDocument or DocumentParse makes more sense than ParsedStatements

@psteinroe psteinroe merged commit 4867189 into main Apr 11, 2025
7 checks passed
@psteinroe psteinroe mentioned this pull request Apr 15, 2025
@juleswritescode juleswritescode deleted the refactor/aprser branch October 28, 2025 15:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

support sql functions bodies

2 participants