Skip to content

Add export options and in-memory SQL export#193

Open
welpie21 wants to merge 3 commits into
surrealdb:mainfrom
welpie21:feature/export-options
Open

Add export options and in-memory SQL export#193
welpie21 wants to merge 3 commits into
surrealdb:mainfrom
welpie21:feature/export-options

Conversation

@welpie21

Copy link
Copy Markdown

JS-parity gap

The JS SDK's export(options) (packages/sdk/src/api/surreal.ts, SqlExportOptions) can filter what an export contains (users, accesses, params, functions, analyzers, apis, buckets, modules, configs, sequences, versions, records, and tables: boolean | string[]) and returns the export as a string. The Java SDK only offered exportSql(String path): file-path only, no options, no in-memory result.

Reproduction

On the embedded memory engine: created person/city records, a DEFINE FUNCTION and a DEFINE PARAM, then called exportSql(path). The export always contains all sections (OPTION, PARAMS, FUNCTIONS, table definitions, and INSERT table data) and there is no API to exclude any of them, nor to obtain the export without going through a file.

Rust SDK capability (surrealdb 3.1.5)

src/method/export.rs supports both halves of the gap:

  • export(target).with_config() exposes builder filters: users, accesses, params, functions, analyzers, apis, buckets, modules, configs, versions, records, and tables(impl Into<TableConfig>) (accepting bool or a Vec<String> table list).
  • export(()) resolves to a Backup stream of byte chunks, i.e. an in-memory export.

Note: the JS sequences option has no counterpart in the Rust 3.1.5 with_config() builder, so it is not exposed here (it stays at its server default, included). Everything else in the JS option set maps 1:1.

Fix

  • New com.surrealdb.ExportOptions builder covering exactly the Rust SDK's config options: users, accesses, params, functions, analyzers, apis, buckets, modules, configs, versions, records, tables(boolean) / tables(String...). Defaults mirror the server defaults (everything on except versions).
  • Surreal.exportSql(String path, ExportOptions options) — filtered export to a file.
  • Surreal.exportSql(ExportOptions options) and Surreal.exportSql() — in-memory export returned as a String, collected from the Rust byte stream.
  • Existing exportSql(String path) is unchanged.
  • JNI plumbing (exportSqlOptions / exportSqlString in src/main/rust/surreal.rs): the boolean options travel as an int bitmask (constants kept in sync between ExportOptions.java and surreal.rs) plus a nullable String[] for an explicit table list, which takes precedence over the all-tables bit.

Tests

New ExportOptionsTests (memory engine, export/import is not supported over WebSocket):

  • default export contains records and definitions;
  • records(false) drops INSERT data but keeps DEFINE TABLE;
  • tables("person") limits which tables appear; tables(false) drops all tables while keeping functions;
  • functions(false).params(false) drops those definitions;
  • string export is byte-identical to file export for the same options;
  • round-trip: export to string, write to temp file, importSql, records and function intact.

Full ./gradlew test suite passes; spotlessApply and cargo fmt --check clean.

🤖 Generated with Claude Code

welpie21 and others added 2 commits July 10, 2026 14:49
The JS SDK's export(options) can filter what gets exported (users,
accesses, params, functions, analyzers, versions, records, tables, ...)
and returns the export as a string, while the Java SDK only offered
exportSql(String path) with no filtering.

Add com.surrealdb.ExportOptions, a builder mirroring the Rust SDK's
export .with_config() options, plus exportSql(path, options),
exportSql(options) and exportSql() which return the export in memory
as a String by collecting the byte stream from export(()).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@welpie21 welpie21 marked this pull request as ready for review July 10, 2026 14:29

@emmanuel-keller emmanuel-keller left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review — approve with nits.

Plumbing checks out end-to-end: the flag bits match bit-for-bit between ExportOptions and the Rust EXPORT_FLAG_*, every flag maps to the right with_config() setter, and the defaults match Config::default() (all-true except versions, with sequences staying on — accurately documented). Mid-stream stream errors propagate (no truncated string is ever returned) and non-UTF-8 surfaces as a clean exception. No native handle allocated, so no leak.

Test gaps: no empty-database export; no non-ASCII/multibyte content test (the one case that actually exercises the new from_utf8/new_string path); several flags (versions(true), users/accesses/analyzers/apis/buckets/modules/configs) untested (mechanically identical to the tested ones, so low risk).

* @throws SurrealException
* if export fails or backups are not supported
*/
public String exportSql() {

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Worth a Javadoc note here: the in-memory variant buffers the whole export (Rust Vec<u8>String → JVM UTF-16, so ~3x peak memory), there's no size guard, and new_string can't represent a >2 GB modified-UTF-8 payload. It's a reasonable tradeoff (matches the JS SDK's string return, and the file overload exists for large DBs) — just point large exports at exportSql(String, ...).

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added in cc1ade3: both in-memory variants (exportSql() and exportSql(ExportOptions)) now carry a Javadoc note that the whole export is buffered in memory (native byte buffer then copied to a UTF-16 String, so roughly 3x peak memory), that there is no size guard, that exports larger than 2 GB cannot be represented as a single Java string, and that large databases should use the file overloads exportSql(String) / exportSql(String, ExportOptions).

* the names of the tables to export
* @return this instance for chaining
*/
public ExportOptions tables(String... tables) {

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

tables(String...) with zero args produces a non-null empty array → TableConfig::Some(vec![]) → exports no tables. new ExportOptions().tables() returning nothing is surprising; consider treating an empty list as "all", or documenting it. (Also allTables stays true here, so FLAG_TABLES is still set but ignored native-side when the list is Some — dead in that path.)

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in cc1ade3. tables() with zero args is now equivalent to tables(true) — it exports all tables instead of none — and the Javadoc documents that empty means all. The dead-path issue is resolved too: setting a non-empty list now clears the all-tables flag, so FLAG_TABLES is never set alongside an explicit list (the native side already gives the list precedence; the Java side just no longer sends the ignored bit). Added tablesEmptyVarargs_exportsAllTables to pin the behaviour, including that a zero-arg call replaces a previously set narrower list.

* @throws SurrealException
* if export fails or backups are not supported
*/
public boolean exportSql(String path, ExportOptions options) {

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

exportSql(String, ExportOptions) / exportSql(ExportOptions) throw a raw NPE if options is null (the no-arg overload is safe). Worth a @throws NullPointerException note or an explicit guard.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in cc1ade3: both exportSql(String, ExportOptions) and exportSql(ExportOptions) now guard with Objects.requireNonNull(options, "options") and document @throws NullPointerException if options is null. Added nullOptions_throwNullPointerException covering both overloads.

- exportSql(String, ExportOptions) and exportSql(ExportOptions) throw
  NullPointerException with a message when options is null (@throws
  documented) instead of a raw NPE from inside the call.
- ExportOptions.tables() with zero arguments is now equivalent to
  tables(true) (export all tables) rather than exporting no tables, and
  setting a non-empty list clears the all-tables flag so FLAG_TABLES is
  never set alongside an explicit list.
- Javadoc on the in-memory export variants documents that the whole
  export is buffered (~3x peak memory), has no size guard, cannot
  exceed 2 GB as a single Java string, and points large exports at the
  file overloads.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants