Skip to content

Commit 78a949d

Browse files
authored
feat: map geo type to string or binary. (#767)
* feat: map geo type to string or binary. * chore: add AGENTS.md * update README.md.
1 parent 3c11243 commit 78a949d

34 files changed

Lines changed: 407 additions & 245 deletions

AGENTS.md

Lines changed: 35 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,35 @@
1+
# AGENTS.md
2+
3+
Read this file first. Then open only the relevant file(s) under `agents/`.
4+
5+
This repository is a Rust workspace for BendSQL plus a separate frontend, language bindings, and Docker-based integration tests.
6+
7+
## Repo Map
8+
9+
- `cli/`: `bendsql` CLI, REPL, output formatting, and web UI host
10+
- `core/`: low-level Databend REST client
11+
- `driver/`: public Rust driver API and query execution layer
12+
- `sql/`: shared SQL/value decoding and encoding logic
13+
- `macros/`: proc macros used by the Rust driver stack
14+
- `frontend/`: editable browser UI source
15+
- `cli/frontend/`: generated frontend assets embedded by the CLI; do not edit by hand
16+
- `bindings/python/`: Python client bindings
17+
- `bindings/nodejs/`: Node.js client bindings
18+
- `tests/`: Docker-based integration test harness
19+
- `ttc/`: tcp test container utilities
20+
21+
## Global Rules
22+
23+
- Prefer source edits over generated output.
24+
- Do not manually edit `cli/frontend`; rebuild it from `frontend/`.
25+
- Keep changes inside the owning subsystem unless the task clearly crosses boundaries.
26+
- Check for uncommitted user changes before editing and do not revert unrelated work.
27+
- Report the exact verification commands you ran, plus anything you skipped and why.
28+
29+
## Task Routing
30+
31+
- Rust behavior, CLI flags, REPL, output, or public Rust API changes: read `agents/rust-workspace.md`
32+
- Browser UI, embedded assets, or frontend dev/proxy flow: read `agents/frontend.md`
33+
- Docker integration tests, Python bindings, or Node.js bindings: read `agents/integration-and-bindings.md`
34+
35+
Some tasks span multiple areas. In that case, read the relevant files before editing.

agents/frontend.md

Lines changed: 31 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,31 @@
1+
# Frontend
2+
3+
Use this guide for work in `frontend/` and for changes that affect how the CLI serves the web UI.
4+
5+
## Source of Truth
6+
7+
- Edit `frontend/` for browser UI code.
8+
- Do not manually edit `cli/frontend`; it is generated output copied from the frontend build.
9+
- Touch `cli/` only when changing how the CLI serves embedded assets or proxies the dev server.
10+
11+
## Workflow
12+
13+
- Interactive development: run `cd frontend && pnpm run dev` in one terminal and `make dev-run` in another.
14+
- Production asset rebuild: run `make build-frontend`.
15+
- `make build` and `make run` also rebuild frontend assets before building or running the CLI.
16+
17+
## Repo-Specific Behavior
18+
19+
- Development proxy mode depends on `BENDSQL_DEV_MODE=1`.
20+
- The CLI serves generated assets from `cli/frontend` for production-style runs.
21+
- If a task changes both browser UI and CLI wiring, validate both sides.
22+
23+
## Validation
24+
25+
- At minimum, run `cd frontend && pnpm run build` for frontend changes.
26+
- If UI serving or proxy behavior changed, also validate the relevant CLI flow with `make dev-run` or a normal CLI run.
27+
28+
## Response Expectations
29+
30+
- State whether you changed source UI code, CLI embedding logic, or both.
31+
- Report whether you rebuilt generated assets.

agents/integration-and-bindings.md

Lines changed: 36 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,36 @@
1+
# Integration And Bindings
2+
3+
Use this guide for Docker-based integration tests and for work in `bindings/python/` or `bindings/nodejs/`.
4+
5+
## Integration Harness
6+
7+
- Integration commands are slower and more side-effectful than unit tests.
8+
- `tests/Makefile` starts Docker Compose services.
9+
- The setup may append `127.0.0.1 minio` to `/etc/hosts` if it is missing.
10+
- Prefer the narrowest integration target that matches the change.
11+
12+
## Integration Commands
13+
14+
- `make integration-core`
15+
- `make integration-driver`
16+
- `make integration-bendsql`
17+
- `make integration-bindings-python`
18+
- `make integration-bindings-nodejs`
19+
- `make integration` only when multiple layers changed or broad final validation is needed
20+
21+
## Bindings Guidance
22+
23+
- Treat Python and Node.js bindings as separate deliverables.
24+
- Do not assume a Rust-layer fix is complete if the same behavior is exposed through bindings.
25+
- For externally visible API or type-mapping changes, inspect whether bindings need matching updates or verification.
26+
- After changes under `bindings/nodejs/`, run `cd bindings/nodejs && pnpm prettier --write .`.
27+
- After changes under `bindings/python/`, run `cd bindings/python && ruff format .`.
28+
- Python binding integration tests run with `behave`.
29+
- Node.js binding integration tests run with `pnpm run test`.
30+
- For local Python binding test runs, make sure the interpreter that owns `behave` is loading the current `databend_driver` build or install, not a stale site-packages copy.
31+
- When adjusting local binding test flow, prefer changes that keep `Makefile` targets and related helper scripts aligned with the CI workflow instead of introducing local-only behavior that can drift from CI.
32+
33+
## Response Expectations
34+
35+
- Call out any Docker or local-environment prerequisites for the commands you ran.
36+
- If integration or binding validation was skipped, say exactly what was skipped and why.

agents/rust-workspace.md

Lines changed: 43 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,43 @@
1+
# Rust Workspace
2+
3+
Use this guide for work in `cli/`, `core/`, `driver/`, `sql/`, `macros/`, and `ttc/`.
4+
5+
## Ownership
6+
7+
- `cli/`: CLI flags, REPL behavior, output formatting, local web server wiring
8+
- `core/`: low-level Databend REST client behavior
9+
- `driver/`: public Rust driver API, query execution, parameter binding
10+
- `sql/`: shared SQL parsing/value decoding/encoding logic
11+
- `macros/`: proc-macro behavior
12+
- `ttc/`: test-container utilities; touch only when the task is explicitly about TTC
13+
14+
## Change Routing
15+
16+
- CLI UX changes usually belong in `cli/`.
17+
- Protocol behavior, row decoding, type mapping, and parameter binding usually belong in `driver/`, `sql/`, or `core/`.
18+
- Do not move reusable logic into `cli/` if bindings or driver users should share it.
19+
- If a change affects a public API or observable query behavior, check whether bindings or docs also need updates.
20+
21+
## Validation
22+
23+
- Prefer targeted Rust validation first when the scope is narrow.
24+
- For general Rust changes, run `make test`.
25+
- For broad or user-facing changes, run `make check`.
26+
- This repo ignores `Cargo.lock`. If local Rust results differ from CI and the change may depend on dependency resolution, check whether a stale local `Cargo.lock` is masking the issue. Regenerate or remove it only as a targeted troubleshooting step, not as a default workflow.
27+
28+
`make check` currently runs:
29+
30+
- `cargo fmt --all -- --check`
31+
- `cargo clippy --all-targets --all-features -- -D warnings`
32+
- `cargo deny check`
33+
- `cargo machete`
34+
- `hawkeye check`
35+
- `typos`
36+
37+
If local tooling is missing, report that explicitly instead of silently skipping it.
38+
39+
## Response Expectations
40+
41+
- Summarize the behavior change, not just the files touched.
42+
- List the exact test or check commands that ran.
43+
- If you skipped validation, say what was skipped and why.

bindings/nodejs/Cargo.toml

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -15,6 +15,7 @@ doc = false
1515
[dependencies]
1616
chrono = { workspace = true }
1717
databend-driver = { workspace = true, features = ["rustls", "flight-sql"] }
18+
databend-driver-core = { workspace = true }
1819
env_logger = "0.11.8"
1920
tokio-stream = { workspace = true }
2021

bindings/nodejs/README.md

Lines changed: 13 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -98,8 +98,8 @@ await conn.close();
9898
| `MAP` | `Object` |
9999
| `VARIANT` | `String / Object` |
100100
| `BITMAP` | `String` |
101-
| `GEOMETRY` | `String` |
102-
| `GEOGRAPHY` | `String` |
101+
| `GEOMETRY` | `String / Buffer` |
102+
| `GEOGRAPHY` | `String / Buffer` |
103103

104104
Note: `VARIANT` is a json encoded string. Example:
105105

@@ -117,6 +117,17 @@ const value = JSON.parse(data);
117117
console.log(value);
118118
```
119119

120+
`GEOMETRY` and `GEOGRAPHY` follow the current `geometry_output_format` setting. Text formats such as `GeoJSON` or `WKT` return `String`; binary formats such as `WKB` or `EWKB` return `Buffer`.
121+
122+
For example:
123+
124+
```javascript
125+
const row = await conn.queryRow(
126+
"settings(geometry_output_format='WKB') SELECT st_point(60, 37)",
127+
);
128+
console.log(Buffer.isBuffer(row.values()[0]));
129+
```
130+
120131
We also provide a helper function to convert `VARIANT` to `Object`:
121132

122133
```javascript

bindings/nodejs/src/lib.rs

Lines changed: 16 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -371,9 +371,23 @@ impl ToNapiValue for Value<'_> {
371371
String::to_napi_value(env, s.to_string())
372372
}
373373
}
374-
databend_driver::Value::Geometry(s) => String::to_napi_value(env, s.to_string()),
374+
databend_driver::Value::Geometry(s) => match s {
375+
databend_driver_core::value::GeoValue::String(s) => {
376+
String::to_napi_value(env, s.to_string())
377+
}
378+
databend_driver_core::value::GeoValue::Binary(b) => {
379+
Buffer::to_napi_value(env, Buffer::from(b.as_slice()))
380+
}
381+
},
375382
databend_driver::Value::Interval(s) => String::to_napi_value(env, s.to_string()),
376-
databend_driver::Value::Geography(s) => String::to_napi_value(env, s.to_string()),
383+
databend_driver::Value::Geography(s) => match s {
384+
databend_driver_core::value::GeoValue::String(s) => {
385+
String::to_napi_value(env, s.to_string())
386+
}
387+
databend_driver_core::value::GeoValue::Binary(b) => {
388+
Buffer::to_napi_value(env, Buffer::from(b.as_slice()))
389+
}
390+
},
377391
databend_driver::Value::Vector(inner) => {
378392
let mut arr = ctx.create_array(inner.len() as u32)?;
379393
for (i, v) in inner.iter().enumerate() {

bindings/nodejs/tests/binding.js

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -45,6 +45,10 @@ const dsn = process.env.TEST_DATABEND_DSN
4545
? process.env.TEST_DATABEND_DSN
4646
: "databend://root:@localhost:8000/default?sslmode=disable";
4747

48+
const POINT_GEOJSON = '{"type": "Point", "coordinates": [60,37]}';
49+
const POINT_WKT = "POINT(60 37)";
50+
const POINT_WKB = Buffer.from("01010000000000000000004E400000000000804240", "hex");
51+
4852
Given("A new Databend Driver Client", async function () {
4953
this.client = new Client(dsn);
5054
const conn = await this.client.getConn();
@@ -226,6 +230,23 @@ Then("Select types should be expected native types", async function () {
226230
const row = await this.conn.queryRow(`SELECT to_datetime_tz('2024-04-16 12:34:56.789 +0800'))`);
227231
assert.deepEqual(row.values(), [new Date("2024-04-16 04:34:56.789Z")]);
228232
}
233+
234+
if (DRIVER_VERSION > [0, 31, 0] && DB_VERSION > [1, 2, 894]) {
235+
{
236+
const row = await this.conn.queryRow("SELECT st_point(60,37)");
237+
assert.deepEqual(row.values(), [POINT_GEOJSON]);
238+
}
239+
240+
{
241+
const row = await this.conn.queryRow("settings(geometry_output_format='WKT') SELECT st_point(60,37)");
242+
assert.deepEqual(row.values(), [POINT_WKT]);
243+
}
244+
245+
{
246+
const row = await this.conn.queryRow("settings(geometry_output_format='WKB') SELECT st_point(60,37)");
247+
assert.deepEqual(row.values(), [POINT_WKB]);
248+
}
249+
}
229250
});
230251

231252
Then("Select numbers should iterate all rows", async function () {

bindings/python/README.md

Lines changed: 18 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -151,15 +151,15 @@ except Exception as e:
151151

152152
### Semi-Structured Data Types
153153

154-
| Databend | Python |
155-
| ----------- | ------- |
156-
| `ARRAY` | `list` |
157-
| `TUPLE` | `tuple` |
158-
| `MAP` | `dict` |
159-
| `VARIANT` | `str` |
160-
| `BITMAP` | `str` |
161-
| `GEOMETRY` | `str` |
162-
| `GEOGRAPHY` | `str` |
154+
| Databend | Python |
155+
| ----------- | ------------- |
156+
| `ARRAY` | `list` |
157+
| `TUPLE` | `tuple` |
158+
| `MAP` | `dict` |
159+
| `VARIANT` | `str` |
160+
| `BITMAP` | `str` |
161+
| `GEOMETRY` | `str / bytes` |
162+
| `GEOGRAPHY` | `str / bytes` |
163163

164164
Note: `VARIANT` is a json encoded string. Example:
165165

@@ -177,6 +177,15 @@ value = json.loads(data)
177177
print(value)
178178
```
179179

180+
`GEOMETRY` and `GEOGRAPHY` follow the current `geometry_output_format` setting. Text formats such as `GeoJSON` or `WKT` return `str`; binary formats such as `WKB` or `EWKB` return `bytes`.
181+
182+
For example:
183+
184+
```python
185+
row = await conn.query_row("settings(geometry_output_format='WKB') SELECT st_point(60, 37)")
186+
assert isinstance(row.values()[0], bytes)
187+
```
188+
180189
## APIs
181190

182191
### Exception Classes (PEP 249 Compliant)

bindings/python/src/types.rs

Lines changed: 15 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -33,6 +33,7 @@ use crate::exceptions::map_error_to_exception;
3333
use crate::utils::wait_for_future;
3434
#[cfg(feature = "cp38")]
3535
use databend_driver::{self, zoned_to_chrono_datetime, zoned_to_chrono_fixed_offset};
36+
use databend_driver_core::value::GeoValue;
3637

3738
pub static VERSION: Lazy<String> = Lazy::new(|| {
3839
let version = option_env!("CARGO_PKG_VERSION").unwrap_or("unknown");
@@ -129,8 +130,20 @@ impl<'py> IntoPyObject<'py> for Value {
129130
}
130131
databend_driver::Value::Bitmap(s) => s.into_bound_py_any(py)?,
131132
databend_driver::Value::Variant(s) => s.into_bound_py_any(py)?,
132-
databend_driver::Value::Geometry(s) => s.into_bound_py_any(py)?,
133-
databend_driver::Value::Geography(s) => s.into_bound_py_any(py)?,
133+
databend_driver::Value::Geometry(s) => match s {
134+
GeoValue::String(s) => s.into_bound_py_any(py)?,
135+
GeoValue::Binary(b) => {
136+
let buf = PyBytes::new(py, &b);
137+
buf.into_bound_py_any(py)?
138+
}
139+
},
140+
databend_driver::Value::Geography(s) => match s {
141+
GeoValue::String(s) => s.into_bound_py_any(py)?,
142+
GeoValue::Binary(b) => {
143+
let buf = PyBytes::new(py, &b);
144+
buf.into_bound_py_any(py)?
145+
}
146+
},
134147
databend_driver::Value::Interval(s) => {
135148
let value = databend_driver::Interval::from_string(&s).unwrap();
136149
let total_micros = (value.months as i64) * 30 * 86400000000

0 commit comments

Comments
 (0)