Add WebAssembly memory configuration for large SQL files#136
Closed
pyramation wants to merge 2 commits intomainfrom
Closed
Add WebAssembly memory configuration for large SQL files#136pyramation wants to merge 2 commits intomainfrom
pyramation wants to merge 2 commits intomainfrom
Conversation
- Set INITIAL_MEMORY to 268435456 (256MB) to prevent memory access out of bounds errors - Set MAXIMUM_MEMORY to 1073741824 (1GB) to allow handling of very large SQL files - Keep existing ALLOW_MEMORY_GROWTH=1 for dynamic memory expansion - Preserve MODULARIZE=1 and all existing exports - Updated template system and full/Makefile to ensure consistency across all PostgreSQL versions Co-Authored-By: Dan Lynch <pyramation@gmail.com>
🤖 Devin AI EngineerI'll be helping with this pull request! Here's what you should know: ✅ I will automatically:
Note: I can only respond to comments from users who have write access to this repository. ⚙️ Control Options:
|
- Restore LIBPG_QUERY_REPO to launchql/libpg_query.git - Restore LIBPG_QUERY_TAG to fix/negative-int-pg15 - Keep memory configuration flags as intended Co-Authored-By: Dan Lynch <pyramation@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Add WebAssembly memory configuration for large SQL files
Summary
This PR adds Emscripten memory configuration flags to prevent "memory access out of bounds" errors when parsing large SQL files with the WebAssembly build of libpg_query. The changes set a larger initial heap (256MB) and higher maximum memory limit (1GB) while preserving dynamic memory growth and all existing module exports.
Key Changes:
-sINITIAL_MEMORY=268435456(256MB initial heap)-sMAXIMUM_MEMORY=1073741824(1GB maximum memory)-sALLOW_MEMORY_GROWTH=1for dynamic expansion-sMODULARIZE=1and all existing exportsReview & Testing Checklist for Human
@pgsql/parser) and full parser (@libpg-query/parser) properly inherit the memory settingsDiagram
%%{ init : { "theme" : "default" }}%% graph TD subgraph "Build System" Template["templates/Makefile.template"]:::major-edit CopyScript["scripts/copy-templates.js"]:::context end subgraph "Version Builds" V13["versions/13/Makefile"]:::major-edit V14["versions/14/Makefile"]:::major-edit V15["versions/15/Makefile"]:::major-edit V16["versions/16/Makefile"]:::major-edit V17["versions/17/Makefile"]:::major-edit end subgraph "Standalone Builds" Full["full/Makefile"]:::major-edit end Template -->|"copy:templates"| V13 Template -->|"copy:templates"| V14 Template -->|"copy:templates"| V15 Template -->|"copy:templates"| V16 Template -->|"copy:templates"| V17 V13 -->|"emscripten build"| WASM13["libpg-query.wasm (v13)"]:::context V17 -->|"emscripten build"| WASM17["libpg-query.wasm (v17)"]:::context Full -->|"emscripten build"| WASMFull["libpg-query.wasm (full)"]:::context subgraph Legend L1["Major Edit"]:::major-edit L2["Minor Edit"]:::minor-edit L3["Context/No Edit"]:::context end classDef major-edit fill:#90EE90 classDef minor-edit fill:#87CEEB classDef context fill:#FFFFFFNotes
copy:templatesscript to maintain consistency across all PostgreSQL versions