Refactor jq/result paths for spaniter v0.2.0#46
Conversation
Add a resultset package for RowIterator materialization, drive lazy RowIter through spaniter, and adopt nil-return ResultSetStats semantics from v0.2.0. Co-authored-by: Cursor <cursoragent@cursor.com>
There was a problem hiding this comment.
Code Review
This pull request refactors the codebase to extract the logic for draining Cloud Spanner row iterators into a new resultset package, utilizing the updated spaniter v0.2.0 library. It also updates jqresult.RowIter to stream rows using Go's iter.Pull2 sequence mechanism. Feedback on the changes highlights a potential goroutine leak in RowIter.Stop(). When r.stopSeq is non-nil, the method returns early without calling r.rowIter.Stop(), which could leave the background goroutine blocked on an active Spanner network call. It is recommended to ensure r.rowIter.Stop() is always called to properly release resources.
Important
The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.
Propagate terminal iterator errors, stop both pull sequence and RowIterator, drop WithDrainOnEarlyStop from ordinary streaming, and use ResultSetStatsForDML on read-write DML paths so row_count_exact:0 is preserved. Co-authored-by: Cursor <cursoragent@cursor.com>
|
/gemini review |
There was a problem hiding this comment.
Code Review
This pull request refactors the codebase to extract result set materialization logic into a new resultset package, which leverages the updated github.com/apstndb/spaniter dependency (v0.2.0). The jqresult package has been updated to use spaniter sequences and pull-based iteration for row streaming, and to handle DML-specific statistics correctly via a new dml parameter. Additionally, obsolete helper functions in main.go have been removed, and comprehensive unit tests have been added for the new and refactored components. I have no feedback to provide as there are no review comments.
Important
The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.
Drain read-write lazy iterators on cleanup so DML runs when jq ignores input, use ResultSetStatsForDML only outside PLAN mode, and avoid double-stopping RowIterator after spaniter.RowIteratorSeq takes ownership. Co-authored-by: Cursor <cursoragent@cursor.com>
|
/gemini review |
There was a problem hiding this comment.
Code Review
This pull request refactors the jqresult package to leverage spaniter's iterator sequence and introduces a new resultset package to handle draining Cloud Spanner row iterators into protobuf ResultSets. It also adds support for preserving standard DML row counts and ensuring lazy JQ filters fully consume the RowIterator during read-write transactions. A review comment identified a potential bug in the test helper runLazyReadWriteDML where retrying a transaction could lead to duplicate elements in the output slice; resetting the slice inside the transaction block was suggested to resolve this.
Important
The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.
| var out []any | ||
| _, err := client.ReadWriteTransaction(ctx, func(ctx context.Context, tx *spanner.ReadWriteTransaction) error { | ||
| rowIter := tx.QueryWithOptions(ctx, spanner.Statement{SQL: sql}, spanner.QueryOptions{}) |
There was a problem hiding this comment.
In Cloud Spanner, the transaction function passed to ReadWriteTransaction can be executed multiple times if the transaction aborts and retries. Since out is declared outside the transaction scope and appended to inside the transaction, retrying will result in duplicate elements in out. Resetting out to nil at the start of the transaction function ensures that retries do not accumulate duplicate values.
| var out []any | |
| _, err := client.ReadWriteTransaction(ctx, func(ctx context.Context, tx *spanner.ReadWriteTransaction) error { | |
| rowIter := tx.QueryWithOptions(ctx, spanner.Statement{SQL: sql}, spanner.QueryOptions{}) | |
| var out []any | |
| _, err := client.ReadWriteTransaction(ctx, func(ctx context.Context, tx *spanner.ReadWriteTransaction) error { | |
| out = nil | |
| rowIter := tx.QueryWithOptions(ctx, spanner.Statement{SQL: sql}, spanner.QueryOptions{}) |
Read-write statements always materialize before jq so lazy mode applies only to read-only queries. Drop completeOnStop and use spaniter StatsEncoding, ResultSet, and PullRowIteratorSeq (local replace until release). Co-authored-by: Cursor <cursoragent@cursor.com>
Pass stats encoding through spaniter drain options, use StatsMapFromResult and ResultSet without bool flags, and keep lazy jq read-only only. Co-authored-by: Cursor <cursoragent@cursor.com>
Avoid accumulating jq output across Spanner transaction retries in runEagerReadWriteDML integration helper. Co-authored-by: Cursor <cursoragent@cursor.com>
Summary
replaceuntil apstndb/spaniter#6 ships as v0.3.0 (WithStatsEncoding,RowIteratorResult.StatsProto/ResultSet,PullRowIteratorSeq).resultsetpackage for RowIterator →*sppb.ResultSetmaterialization (Materialize,FromIteratorResult,CollectListValues,RowToListValue).jqresultto jq-specific concerns: protojson maps, lazy/eager gojq pipeline,RowToJSON.RowIterthroughPullRowIteratorSeqwithWithResult;Stop()stops the pull sequence when started, otherwise stops the raw iterator directly.true).WithStatsEncoding(DMLExact)for executed read-write DML viaspaniterStatsOpts; PLAN mode keeps default encoding sorow_count_exact: 0is not fabricated.PullRowIteratorSeqthrough lazyRowIter.Merge note: This PR depends on spaniter v0.3.0. Merge spaniter#6 first, then bump
go.modto the released tag and removereplace. GitHub CI build/lint will fail until then because the published v0.2.0 module does not include these APIs.Test plan
go test ./...(with localreplace github.com/apstndb/spaniter => ../spaniter)go test -run TestWithCloudSpannerEmulator .(emulator)trueand preservesrowCountExact: "0"for zero-row UPDATErowCountExact.rows, Stop without drain, terminal error propagation)