Skip to content

README.md

Latest commit

 

History

History
358 lines (244 loc) · 13.9 KB

README.md

File metadata and controls

358 lines (244 loc) · 13.9 KB
Raw

@libpg-query/parser

constructive.io


The Real PostgreSQL Parser for JavaScript

Bring the power of PostgreSQL’s native parser to your JavaScript projects — no native builds, no platform headaches.

This is the official PostgreSQL parser, compiled to WebAssembly (WASM) for seamless, cross-platform compatibility. Use it in Node.js or the browser, on Linux, Windows, or anywhere JavaScript runs.

Built to power pgsql-parser, this library delivers full fidelity with the Postgres C codebase — no rewrites, no shortcuts.

Features

  • 🔧 Powered by PostgreSQL – Uses the official Postgres C parser compiled to WebAssembly
  • 🖥️ Cross-Platform – Runs smoothly on macOS, Linux, and Windows
  • 🌐 Node.js & Browser Support – Consistent behavior in any JS environment
  • 📦 No Native Builds Required – No compilation, no system-specific dependencies
  • 🧠 Spec-Accurate Parsing – Produces faithful, standards-compliant ASTs
  • 🚀 Production-Grade – Millions of downloads and trusted by countless projects and top teams

🚀 For Round-trip Codegen

🎯 Want to parse + deparse (full round trip)?
We highly recommend using pgsql-parser which leverages a pure TypeScript deparser that has been battle-tested against 23,000+ SQL statements and is built on top of libpg-query.

Installation

npm install @libpg-query/parser

Usage

parse(query: string): Promise<ParseResult>

Parses the SQL and returns a Promise for the parse tree. May reject with a parse error.

import { parse } from '@libpg-query/parser';

const result = await parse('SELECT * FROM users WHERE active = true');
// Returns: ParseResult - parsed query object

parseSync(query: string): ParseResult

Synchronous version that returns the parse tree directly. May throw a parse error.

import { parseSync } from '@libpg-query/parser';

const result = parseSync('SELECT * FROM users WHERE active = true');
// Returns: ParseResult - parsed query object

parsePlPgSQL(funcsSql: string): Promise<ParseResult>

Parses the contents of a PL/pgSQL function from a CREATE FUNCTION declaration. Returns a Promise for the parse tree.

import { parsePlPgSQL } from '@libpg-query/parser';

const functionSql = `
CREATE FUNCTION get_user_count() RETURNS integer AS $$
BEGIN
    RETURN (SELECT COUNT(*) FROM users);
END;
$$ LANGUAGE plpgsql;
`;

const result = await parsePlPgSQL(functionSql);

parsePlPgSQLSync(funcsSql: string): ParseResult

Synchronous version of PL/pgSQL parsing.

import { parsePlPgSQLSync } from '@libpg-query/parser';

const result = parsePlPgSQLSync(functionSql);

fingerprint(sql: string): Promise<string>

Generates a unique fingerprint for a SQL query that can be used for query identification and caching. Returns a Promise for a 16-character fingerprint string.

import { fingerprint } from '@libpg-query/parser';

const fp = await fingerprint('SELECT * FROM users WHERE active = $1');
// Returns: string - unique 16-character fingerprint (e.g., "50fde20626009aba")

fingerprintSync(sql: string): string

Synchronous version that generates a unique fingerprint for a SQL query directly.

import { fingerprintSync } from '@libpg-query/parser';

const fp = fingerprintSync('SELECT * FROM users WHERE active = $1');
// Returns: string - unique 16-character fingerprint

normalize(sql: string): Promise<string>

Normalizes a SQL query by removing comments, standardizing whitespace, and converting to a canonical form. Returns a Promise for the normalized SQL string.

import { normalize } from '@libpg-query/parser';

const normalized = await normalize('SELECT * FROM users WHERE active = true');
// Returns: string - normalized SQL query

normalizeSync(sql: string): string

Synchronous version that normalizes a SQL query directly.

import { normalizeSync } from '@libpg-query/parser';

const normalized = normalizeSync('SELECT * FROM users WHERE active = true');
// Returns: string - normalized SQL query

scan(sql: string): Promise<ScanResult>

Scans (tokenizes) a SQL query and returns detailed information about each token. Returns a Promise for a ScanResult containing all tokens with their positions, types, and classifications.

import { scan } from '@libpg-query/parser';

const result = await scan('SELECT * FROM users WHERE id = $1');
// Returns: ScanResult - detailed tokenization information
console.log(result.tokens[0]); // { start: 0, end: 6, text: "SELECT", tokenType: 651, tokenName: "UNKNOWN", keywordKind: 4, keywordName: "RESERVED_KEYWORD" }

scanSync(sql: string): ScanResult

Synchronous version that scans (tokenizes) a SQL query directly.

import { scanSync } from '@libpg-query/parser';

const result = scanSync('SELECT * FROM users WHERE id = $1');
// Returns: ScanResult - detailed tokenization information

Initialization

The library provides both async and sync methods. Async methods handle initialization automatically, while sync methods require explicit initialization.

Async Methods (Recommended)

Async methods handle initialization automatically and are always safe to use:

import { parse, scan } from '@libpg-query/parser';

// These handle initialization automatically
const result = await parse('SELECT * FROM users');
const tokens = await scan('SELECT * FROM users');

Sync Methods

Sync methods require explicit initialization using loadModule():

import { loadModule, parseSync, scanSync } from '@libpg-query/parser';

// Initialize first
await loadModule();

// Now safe to use sync methods
const result = parseSync('SELECT * FROM users');
const tokens = scanSync('SELECT * FROM users');

loadModule(): Promise<void>

Explicitly initializes the WASM module. Required before using any sync methods.

import { loadModule, parseSync, scanSync } from '@libpg-query/parser';

// Initialize before using sync methods
await loadModule();
const result = parseSync('SELECT * FROM users');
const tokens = scanSync('SELECT * FROM users');

Note: We recommend using async methods as they handle initialization automatically. Use sync methods only when necessary, and always call loadModule() first.

Type Definitions

interface ParseResult {
  version: number;
  stmts: Statement[];
}

interface Statement {
  stmt_type: string;
  stmt_len: number;
  stmt_location: number;
  query: string;
}

interface ScanResult {
  version: number;
  tokens: ScanToken[];
}

interface ScanToken {
  start: number;          // Starting position in the SQL string
  end: number;            // Ending position in the SQL string
  text: string;           // The actual token text
  tokenType: number;      // Numeric token type identifier
  tokenName: string;      // Human-readable token type name
  keywordKind: number;    // Numeric keyword classification
  keywordName: string;    // Human-readable keyword classification
}

Note: The return value is an array, as multiple queries may be provided in a single string (semicolon-delimited, as PostgreSQL expects).

Build Instructions

This package uses a WASM-only build system for true cross-platform compatibility without native compilation dependencies.

Prerequisites

  • Node.js (version 16 or higher recommended)
  • pnpm (v8+ recommended)

Building WASM Artifacts

  1. Install dependencies:

    pnpm install

  2. Build WASM artifacts:

    pnpm run build

  3. Clean WASM build (if needed):

    pnpm run clean

  4. Rebuild WASM artifacts from scratch:

    pnpm run clean && pnpm run build

Build Process Details

The WASM build process:

  • Uses Emscripten SDK for compilation
  • Compiles C wrapper code to WebAssembly
  • Generates wasm/libpg-query.js and wasm/libpg-query.wasm files
  • No native compilation or node-gyp dependencies required

Testing

Running Tests

pnpm run test

Test Requirements

  • WASM artifacts must be built before running tests
  • If tests fail with "fetch failed" errors, rebuild WASM artifacts: 
    pnpm run clean && pnpm run build && pnpm run test

Versions

Built with the 18-latest branch from libpg_query (tag 18.0.0), based on PostgreSQL 18.

Troubleshooting

Common Issues

"fetch failed" errors during tests:

  • This indicates stale or missing WASM artifacts
  • Solution: pnpm run clean && pnpm run build

"WASM module not initialized" errors:

  • Ensure you call an async method first to initialize the WASM module
  • Or use the async versions of methods which handle initialization automatically

Build environment issues:

  • Ensure Emscripten SDK is properly installed and configured
  • Check that all required build dependencies are available

Build Artifacts

The build process generates these files:

  • wasm/libpg-query.js - Emscripten-generated JavaScript loader
  • wasm/libpg-query.wasm - WebAssembly binary
  • wasm/index.js - ES module exports
  • wasm/index.cjs - CommonJS exports with sync wrappers

Credits

Built on the excellent work of several contributors:

🛠 Built by the Constructive team — creators of modular Postgres tooling for secure, composable backends. If you like our work, contribute on GitHub.

Related

  • pgsql-parser: The real PostgreSQL parser for Node.js, providing symmetric parsing and deparsing of SQL statements with actual PostgreSQL parser integration.
  • pgsql-deparser: A streamlined tool designed for converting PostgreSQL ASTs back into SQL queries, focusing solely on deparser functionality to complement pgsql-parser.
  • @pgsql/parser: Multi-version PostgreSQL parser with dynamic version selection at runtime, supporting PostgreSQL 15, 16, and 17 in a single package.
  • @pgsql/types: Offers TypeScript type definitions for PostgreSQL AST nodes, facilitating type-safe construction, analysis, and manipulation of ASTs.
  • @pgsql/enums: Provides TypeScript enum definitions for PostgreSQL constants, enabling type-safe usage of PostgreSQL enums and constants in your applications.
  • @pgsql/utils: A comprehensive utility library for PostgreSQL, offering type-safe AST node creation and enum value conversions, simplifying the construction and manipulation of PostgreSQL ASTs.
  • pg-proto-parser: A TypeScript tool that parses PostgreSQL Protocol Buffers definitions to generate TypeScript interfaces, utility functions, and JSON mappings for enums.
  • libpg-query: The real PostgreSQL parser exposed for Node.js, used primarily in pgsql-parser for parsing and deparsing SQL queries.

Disclaimer

AS DESCRIBED IN THE LICENSES, THE SOFTWARE IS PROVIDED "AS IS", AT YOUR OWN RISK, AND WITHOUT WARRANTIES OF ANY KIND.

No developer or entity involved in creating Software will be liable for any claims or damages whatsoever associated with your use, inability to use, or your interaction with other users of the Software code or Software CLI, including any direct, indirect, incidental, special, exemplary, punitive or consequential damages, or loss of profits, cryptocurrencies, tokens, or anything else of value.