Skip to content

Commit 3eb6e0f

Browse files
authored
Support DuckDB main (PEG parser): port expression FFI + parser-swap regression test (#38)
* Add parser-agnostic regression test for DuckDB parser swap DuckDB is replacing its legacy PostgreSQL parser with a PEG parser (opt-in in 1.5, the sole parser on main, default in 2.0). Yardstick integrates via allow_parser_override_extension="fallback", so when the override declines a query, DuckDB's parser handles it -- including surfacing syntax errors, whose shape differs between the legacy and PEG parsers. Add test/sql/peg_parser.test guarding the two behaviors most exposed to the swap: the parser_override rewrite path still works, and the fallback path still raises a parser-agnostic "Parser Error". The test runs under whichever parser the build uses, so it becomes PEG coverage automatically once the suite runs against a PEG-default DuckDB. * Port parser FFI to DuckDB main's expression API (dual-API) DuckDB main refactored the parsed-expression API: subclass fields are now private (accessed via FunctionName()/GetArgumentsMutable()/LeftMutable()/...) and names use a dedicated Identifier type instead of std::string. This broke the duckdb-next-build CI lane (compile failure in yardstick_parser_ffi.cpp), which has been red for a week. Add a small compatibility shim, selected at compile time via __has_include("duckdb/common/identifier.hpp") (matching the project's existing __has_include pattern), and route all expression field access through it. The old field-based API (DuckDB 1.5 and the pinned submodule) and the new accessor-based API (main) are both supported, so the stable and next CI lanes both build. Behavior is unchanged: the shim is a pure access adaptation over the same tree walks. Verified with -fsyntax-only against both DuckDB main and v1.5-variegata headers (0 errors each). * Port ParserExtension parse_function to DuckDB main's token API Fixing the expression FFI let the next-build compile advance to the next DuckDB-main API change: parse_function_t now receives the post-PEG-failure token tail (const vector<SimpleToken>&) instead of the raw query string (yardstick_extension.hpp/.cpp). Yardstick does all of its rewriting in yardstick_parser_override, which still receives the full query string on both APIs, so on the new signature parse_function is a no-op fallback. The old string-based fallback is retained for DuckDB 1.5 via the same __has_include detection. Matching yardstick_parse to parse_function_t also fixes the function-pointer comparison in yardstick_bind. Verified with -fsyntax-only against both DuckDB main and v1.5-variegata (0 errors); all three extension translation units now compile against main.
1 parent 9d676ca commit 3eb6e0f

4 files changed

Lines changed: 423 additions & 115 deletions

File tree

src/include/yardstick_extension.hpp

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -22,8 +22,25 @@ class YardstickExtension : public Extension {
2222
BoundStatement yardstick_bind(ClientContext &context, Binder &binder,
2323
OperatorExtensionInfo *info, SQLStatement &statement);
2424

25+
// DuckDB main changed parse_function_t to receive the post-PEG-failure token
26+
// tail (vector<SimpleToken>) instead of the raw query string; DuckDB 1.5 and
27+
// earlier pass the query string. Detected via the header the refactor introduced.
28+
// Yardstick performs all of its rewriting in yardstick_parser_override (which
29+
// still receives the full query string on both APIs), so on the new signature
30+
// parse_function is a no-op fallback.
31+
#if __has_include("duckdb/common/identifier.hpp")
32+
#define YARDSTICK_TOKEN_PARSE_FN 1
33+
#else
34+
#define YARDSTICK_TOKEN_PARSE_FN 0
35+
#endif
36+
37+
#if YARDSTICK_TOKEN_PARSE_FN
38+
ParserExtensionParseResult yardstick_parse(ParserExtensionInfo *,
39+
const vector<SimpleToken> &tokens);
40+
#else
2541
ParserExtensionParseResult yardstick_parse(ParserExtensionInfo *,
2642
const std::string &query);
43+
#endif
2744

2845
ParserExtensionPlanResult yardstick_plan(ParserExtensionInfo *, ClientContext &,
2946
unique_ptr<ParserExtensionParseData>);

src/yardstick_extension.cpp

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -330,6 +330,16 @@ static bool StartsWithSemantic(const std::string &query, std::string &stripped_q
330330
return true;
331331
}
332332

333+
#if YARDSTICK_TOKEN_PARSE_FN
334+
// DuckDB main: parse_function receives the post-PEG-failure token tail rather
335+
// than the query string. Yardstick rewrites queries earlier, in
336+
// yardstick_parser_override (which sees the full query string), so there is
337+
// nothing to do here -- decline and let DuckDB proceed.
338+
ParserExtensionParseResult yardstick_parse(ParserExtensionInfo *,
339+
const vector<SimpleToken> &) {
340+
return ParserExtensionParseResult();
341+
}
342+
#else
333343
ParserExtensionParseResult yardstick_parse(ParserExtensionInfo *,
334344
const std::string &query) {
335345

@@ -422,6 +432,7 @@ ParserExtensionParseResult yardstick_parse(ParserExtensionInfo *,
422432
// Not a yardstick query, let DuckDB handle it
423433
return ParserExtensionParseResult();
424434
}
435+
#endif // YARDSTICK_TOKEN_PARSE_FN
425436

426437
//=============================================================================
427438
// PARSER OVERRIDE: intercepts ALL queries before DuckDB's native parser

0 commit comments

Comments
 (0)