Skip to content

Parameterize N1QL queries with positional parameters#34

Merged
JulienBURNET-FAUCHE merged 15 commits into
masterfrom
pimpin/n1ql-positional-params-v2
Jun 30, 2026
Merged

Parameterize N1QL queries with positional parameters#34
JulienBURNET-FAUCHE merged 15 commits into
masterfrom
pimpin/n1ql-positional-params-v2

Conversation

@pimpin

@pimpin pimpin commented Jun 24, 2026

Copy link
Copy Markdown
Collaborator

Summary

Replace string-interpolated values with positional parameter placeholders ($1, $2, ...) across all N1QL query paths. This makes query strings stable regardless of parameter values, enabling Couchbase prepared-statement caching.

Changes (6 commits)

  1. Add serialize_for_binding/bind helpers and parameterize query_helper — Building blocks: type conversion for binding and $N placeholder generation. All build_match* methods accept optional params: keyword.
  2. Parameterize N1QL module queriesbuild_where and run_query use positional parameters; debug log switches to block form with params.
  3. Parameterize Relation queriesto_n1ql_with_params, build_where_with_params, build_update_with_params, build_query_options. All query paths (execute, query, first, last, update_all) use params.
  4. Parameterize has_many through queriesbuild_index_n1ql uses $1/$2 for type and foreign key.
  5. Pass arrays as single positional parameterIN $1 instead of IN [$1, $2, ...] for stable query strings regardless of array length.
  6. Support ActiveSupport::TimeWithZoneacts_like?(:time) duck-typing in serialize_for_binding.

Releasability

Queries are parameterized but adhoc remains at SDK default (true = no prepared-statement cache), so there is no behavioral change. The adhoc option will be added in a follow-up PR.

Observability side effect

With parameterized queries, parameter values are no longer embedded in the statement field of system:completed_requests (and system:active_requests). The statement now contains $1, $2, … placeholders regardless of the actual values.

DBAs or monitoring tools that inspect statement to retrieve query parameters must now read the separate positionalArgs field instead:

SELECT statement, positionalArgs, elapsedTime
FROM system:completed_requests
WHERE statement LIKE '%<pattern>%'

positionalArgs is always populated when querying via N1QL (the keyspace calls Format with profiling=true — see system_keyspace_request_log.go). It is absent only when hitting the REST endpoint with GET /admin/completed_requests.

This separation is a net positive for performance analysis: multiple executions with different values share the same canonical statement, making it straightforward to aggregate metrics per query pattern without manual string normalization.

Test plan

  • All existing specs pass unchanged
  • New parameterized query specs verify placeholder generation
  • /test --fail-fast=false

🤖 Generated with Claude Code

pimpin and others added 6 commits June 24, 2026 14:02
Add serialize_for_binding(value) to convert Ruby types for N1QL binding
(Time/DateTime → ISO8601, Date → string, others pass through).

Add bind(value, params) to push values into a positional parameter array
and return the $N placeholder.

Add params: keyword argument to build_match, build_match_hash,
build_match_range, and build_not_match. When params: is provided, values
are bound via bind(); otherwise falls back to quote() for backward
compatibility.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
build_where now accepts a params: keyword and binds the type filter and
emit_key values as positional parameters ($1, $2, ...) instead of
interpolating them into the query string.

run_query creates a params array, passes it through build_where, and
forwards positional_parameters to Couchbase::Options::Query.

The debug log switches to a block form and includes the params array.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Replace string-interpolated values with positional parameters in all
Relation query paths:

- Add to_n1ql_with_params returning [query, params]
- Add build_where_with_params, build_conds_with_params using bind()
- Add build_update_with_params using bind() for both scalar and Hash
  values (fixes the injection vulnerability from build_update)
- Add build_query_options helper for Couchbase::Options::Query
- Update execute, query, first, last, update_all to use params

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Replace string interpolation in build_index_n1ql with positional
parameters ($1 for type, $2 for foreign key value), enabling
prepared-statement caching for has_many :through queries.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…hing

Instead of expanding arrays into individual parameters (e.g. IN [$1, $2]),
pass the whole array as one parameter (e.g. IN $1). This keeps the query
string stable regardless of array length, enabling Couchbase to cache the
prepared statement for IN queries.

serialize_for_binding now handles arrays recursively.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
TimeWithZone does not inherit from Time or DateTime, so it was not
being serialized to ISO8601. Use acts_like?(:time) duck-typing to
handle all time-like objects from Rails.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces parameterized queries to the Couchbase ORM to replace raw string interpolation, enhancing security against N1QL injection and improving query performance. It updates query-building methods across relations, N1QL execution, and helpers to bind values into positional parameters. The review feedback highlights two important issues: first, a mismatch in spec/relation_spec.rb where the test assertions for array parameterization do not align with the actual implementation (which binds the entire array as a single parameter); second, a bug in QueryHelper#bind where returning nil for nil values leads to invalid N1QL syntax during update_all operations instead of binding nil as a positional parameter.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

Comment thread spec/relation_spec.rb Outdated
Comment on lines +19 to +26
def bind(value, params)
if value.nil?
nil
else
params << serialize_for_binding(value)
"$#{params.length}"
end
end

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Currently, bind returns nil when value is nil. In build_update_with_params, this causes update_all(field: nil) to generate invalid N1QL syntax like field = (with an empty value after the equals sign). Since build_match and build_not_match already handle nil values explicitly before calling bind, we can safely remove the nil check in bind so that nil is correctly bound as a positional parameter (e.g., field = $N with nil in params).

            def bind(value, params)
                params << serialize_for_binding(value)
                "$#{params.length}"
            end

@pimpin pimpin changed the base branch from pimpin/fix-n1ql-injection-hash to master June 26, 2026 11:48
pimpin and others added 5 commits June 26, 2026 13:53
Resolved conflict in lib/couchbase-orm/relation.rb: kept parameterized
bind(v, params) for nested Hash attribute updates (from this branch)
over the quote(v) inline approach added in PR #33 — bind is both
injection-safe and enables prepared-statement caching.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…y_fn

Couchbase::Options::Query exposes only a writer (scan_consistency=), not
a reader. Reading options.scan_consistency raises NoMethodError at runtime.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
… clause

bind(nil) returns Ruby nil which interpolates to "" in a string, producing
invalid N1QL like "child.name = ". Restore the old `|| 'NULL'` guard by
checking nil before calling bind.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
The code generates "type = $1" with spaces around =, consistent with
all other conditions. The test incorrectly expected "type=$1".

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
bind() passes the whole array as one positional parameter ($2), not one
param per element. This is correct: "name IN $2" with $2 = ["Alice","Bob"]
is valid N1QL and produces a single prepared-statement shape regardless
of array length, which is the caching goal of this PR.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Comment thread lib/couchbase-orm/utilities/has_many.rb Outdated
… positional parameter

type/design_document is a class-level constant (equivalent to a table name) and
should not consume a positional parameter slot. Interpolating it gives each model
class its own query fingerprint for prepared-statement caching, consistent with
how bucket.name is treated. Applies to all three callsites: n1ql.rb, relation.rb,
and has_many.rb.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

@pimpin pimpin left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some points to check before merging this.

Comment thread lib/couchbase-orm/utilities/query_helper.rb
Comment thread lib/couchbase-orm/utilities/has_many.rb Outdated
Comment thread lib/couchbase-orm/relation.rb Outdated
pimpin and others added 3 commits June 29, 2026 17:36
to_n1ql delegated to to_n1ql_with_params.first, which returns a query
with positional placeholders ($1, $2) instead of literal values. This
broke the method's main external use case: producing a query you can
copy-paste into the database. Build the query with interpolated values
(params: nil) so to_n1ql keeps its previous, human-readable output while
to_n1ql_with_params remains the parameterized path used internally.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The lambda ->(v) { params ? bind(v, params) : quote(v) } was redefined
in build_match, build_match_hash, build_match_range and build_not_match.
Replace the four copies with a single resolve_value(value, params) helper.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The query_fn rebuilt a fresh Couchbase::Options::Query copying only
scan_consistency, silently dropping every other option the caller set
(timeout, adhoc, client_context_id, profile, etc.). Mutate the existing
options object via #positional_parameters and pass it through, so all
caller-supplied options are retained.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@JulienBURNET-FAUCHE JulienBURNET-FAUCHE merged commit 5861782 into master Jun 30, 2026
10 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants