Skip to content

Commit 3b47112

Browse files
committed
feat: BigQuery Storage v1beta1 API migration guide
1 parent 94315ce commit 3b47112

1 file changed

Lines changed: 164 additions & 0 deletions

File tree

Lines changed: 164 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,164 @@
1+
# Migrating BigQuery Storage API from v1beta1 to v1: Java
2+
3+
This guide shows how to migrate Java code using the BigQuery Storage API from
4+
version `v1beta1` to `v1`.
5+
6+
## Key Changes
7+
8+
* **Service Client**: `BigQueryStorageClient` is replaced by
9+
`BigQueryReadClient`.
10+
* **Table Reference**: `TableReference` proto message is replaced by a simple
11+
string representation of the table path in `ReadSession`.
12+
* **Session Configuration**: Several fields from `CreateReadSessionRequest`
13+
have moved into `ReadSession` (which is now passed as a field in the
14+
request).
15+
* **Parallelism**: `requested_streams` is replaced by `max_stream_count`.
16+
* **Sharding Strategy**: `sharding_strategy` is removed. The server now
17+
automatically balances the streams.
18+
* **Read Rows Request**: `StreamPosition` is flattened. You now pass the
19+
stream name directly as `read_stream` and the `offset` as a top-level field
20+
in `ReadRowsRequest`.
21+
22+
## Code Comparison
23+
24+
### 1. Client Initialization
25+
26+
**v1beta1:**
27+
28+
```java
29+
import com.google.cloud.bigquery.storage.v1beta1.BigQueryStorageClient;
30+
31+
try (BigQueryStorageClient client = BigQueryStorageClient.create()) {
32+
// use client
33+
}
34+
```
35+
36+
**v1:**
37+
38+
```java
39+
import com.google.cloud.bigquery.storage.v1.BigQueryReadClient;
40+
41+
try (BigQueryReadClient client = BigQueryReadClient.create()) {
42+
// use client
43+
}
44+
```
45+
46+
### 2. Creating a Read Session
47+
48+
**v1beta1:**
49+
50+
```java
51+
import com.google.cloud.bigquery.storage.v1beta1.BigQueryStorageClient;
52+
import com.google.cloud.bigquery.storage.v1beta1.Storage.CreateReadSessionRequest;
53+
import com.google.cloud.bigquery.storage.v1beta1.Storage.DataFormat;
54+
import com.google.cloud.bigquery.storage.v1beta1.Storage.ReadSession;
55+
import com.google.cloud.bigquery.storage.v1beta1.Storage.ShardingStrategy;
56+
import com.google.cloud.bigquery.storage.v1beta1.ReadOptions.TableReadOptions;
57+
import com.google.cloud.bigquery.storage.v1beta1.TableReferenceProto.TableReference;
58+
59+
// ...
60+
61+
TableReference tableReference = TableReference.newBuilder()
62+
.setProjectId("bigquery-public-data")
63+
.setDatasetId("usa_names")
64+
.setTableId("usa_1910_current")
65+
.build();
66+
67+
TableReadOptions options = TableReadOptions.newBuilder()
68+
.addSelectedFields("name")
69+
.setRowRestriction("state = \"WA\"")
70+
.build();
71+
72+
CreateReadSessionRequest request = CreateReadSessionRequest.newBuilder()
73+
.setParent("projects/read-session-project")
74+
.setTableReference(tableReference)
75+
.setReadOptions(options)
76+
.setRequestedStreams(1)
77+
.setFormat(DataFormat.AVRO)
78+
.setShardingStrategy(ShardingStrategy.LIQUID)
79+
.build();
80+
81+
ReadSession session = client.createReadSession(request);
82+
```
83+
84+
**v1:**
85+
86+
```java
87+
import com.google.cloud.bigquery.storage.v1.BigQueryReadClient;
88+
import com.google.cloud.bigquery.storage.v1.CreateReadSessionRequest;
89+
import com.google.cloud.bigquery.storage.v1.DataFormat;
90+
import com.google.cloud.bigquery.storage.v1.ReadSession;
91+
import com.google.cloud.bigquery.storage.v1.ReadSession.TableReadOptions;
92+
93+
// ...
94+
95+
// Table path is now a string: projects/{project}/datasets/{dataset}/tables/{table}
96+
String tablePath = "projects/bigquery-public-data/datasets/usa_names/tables/usa_1910_current";
97+
98+
TableReadOptions options = TableReadOptions.newBuilder()
99+
.addSelectedFields("name")
100+
.setRowRestriction("state = \"WA\"")
101+
.build();
102+
103+
// ReadSession holds the session configuration (table, options, format)
104+
ReadSession readSession = ReadSession.newBuilder()
105+
.setTable(tablePath)
106+
.setDataFormat(DataFormat.AVRO) // format renamed to data_format
107+
.setReadOptions(options)
108+
.build();
109+
110+
CreateReadSessionRequest request = CreateReadSessionRequest.newBuilder()
111+
.setParent("projects/read-session-project")
112+
.setReadSession(readSession)
113+
.setMaxStreamCount(1) // requested_streams renamed to max_stream_count
114+
.build();
115+
116+
ReadSession session = client.createReadSession(request);
117+
```
118+
119+
### 3. Reading Rows
120+
121+
**v1beta1:**
122+
123+
```java
124+
import com.google.api.gax.rpc.ServerStream;
125+
import com.google.cloud.bigquery.storage.v1beta1.Storage.ReadRowsRequest;
126+
import com.google.cloud.bigquery.storage.v1beta1.Storage.ReadRowsResponse;
127+
import com.google.cloud.bigquery.storage.v1beta1.Storage.StreamPosition;
128+
129+
// ...
130+
StreamPosition readPosition = StreamPosition.newBuilder()
131+
.setStream(session.getStreams(0)) // Stream object
132+
.setOffset(0)
133+
.build();
134+
135+
ReadRowsRequest readRowsRequest = ReadRowsRequest.newBuilder()
136+
.setReadPosition(readPosition)
137+
.build();
138+
139+
ServerStream<ReadRowsResponse> stream = client.readRowsCallable().call(readRowsRequest);
140+
for (ReadRowsResponse response : stream) {
141+
// Process response.getAvroRows()
142+
}
143+
```
144+
145+
**v1:**
146+
147+
```java
148+
import com.google.api.gax.rpc.ServerStream;
149+
import com.google.cloud.bigquery.storage.v1.ReadRowsRequest;
150+
import com.google.cloud.bigquery.storage.v1.ReadRowsResponse;
151+
152+
// ...
153+
// ReadRowsRequest is flattened. We pass the stream name (string) and offset directly.
154+
ReadRowsRequest readRowsRequest = ReadRowsRequest.newBuilder()
155+
.setReadStream(session.getStreams(0).getName()) // Stream name string
156+
.setOffset(0)
157+
.build();
158+
159+
ServerStream<ReadRowsResponse> stream = client.readRowsCallable().call(readRowsRequest);
160+
for (ReadRowsResponse response : stream) {
161+
// Process response.getAvroRows()
162+
// Note: Prefer using response.getRowCount() over response.getAvroRows().getRowCount() (deprecated)
163+
}
164+
```

0 commit comments

Comments
 (0)