Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 7 additions & 2 deletions v1/README_Cloud_Bigtable_to_GCS_SequenceFile.md
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,7 @@ on [Metadata Annotations](https://github.com/GoogleCloudPlatform/DataflowTemplat
* **bigtableFilter**: Filter string. See: http://hbase.apache.org/book.html#thrift. Defaults to empty.
* **bigtableReadRpcTimeoutMs**: Bigtable read RPC timeout in milliseconds.
* **bigtableReadRpcAttemptTimeoutMs**: Bigtable read RPC attempt timeout in milliseconds.
* **bigtableMaxAttempts**: The maximum number of retry attempts for the Bigtable client.



Expand Down Expand Up @@ -138,6 +139,7 @@ export BIGTABLE_MAX_VERSIONS=2147483647
export BIGTABLE_FILTER=""
export BIGTABLE_READ_RPC_TIMEOUT_MS=<bigtableReadRpcTimeoutMs>
export BIGTABLE_READ_RPC_ATTEMPT_TIMEOUT_MS=<bigtableReadRpcAttemptTimeoutMs>
export BIGTABLE_MAX_ATTEMPTS=<bigtableMaxAttempts>

gcloud dataflow jobs run "cloud-bigtable-to-gcs-sequencefile-job" \
--project "$PROJECT" \
Expand All @@ -154,7 +156,8 @@ gcloud dataflow jobs run "cloud-bigtable-to-gcs-sequencefile-job" \
--parameters "destinationPath=$DESTINATION_PATH" \
--parameters "filenamePrefix=$FILENAME_PREFIX" \
--parameters "bigtableReadRpcTimeoutMs=$BIGTABLE_READ_RPC_TIMEOUT_MS" \
--parameters "bigtableReadRpcAttemptTimeoutMs=$BIGTABLE_READ_RPC_ATTEMPT_TIMEOUT_MS"
--parameters "bigtableReadRpcAttemptTimeoutMs=$BIGTABLE_READ_RPC_ATTEMPT_TIMEOUT_MS" \
--parameters "bigtableMaxAttempts=$BIGTABLE_MAX_ATTEMPTS"
```

For more information about the command, please check:
Expand Down Expand Up @@ -187,6 +190,7 @@ export BIGTABLE_MAX_VERSIONS=2147483647
export BIGTABLE_FILTER=""
export BIGTABLE_READ_RPC_TIMEOUT_MS=<bigtableReadRpcTimeoutMs>
export BIGTABLE_READ_RPC_ATTEMPT_TIMEOUT_MS=<bigtableReadRpcAttemptTimeoutMs>
export BIGTABLE_MAX_ATTEMPTS=<bigtableMaxAttempts>

mvn clean package -PtemplatesRun \
-DskipTests \
Expand All @@ -195,7 +199,7 @@ mvn clean package -PtemplatesRun \
-Dregion="$REGION" \
-DjobName="cloud-bigtable-to-gcs-sequencefile-job" \
-DtemplateName="Cloud_Bigtable_to_GCS_SequenceFile" \
-Dparameters="bigtableProject=$BIGTABLE_PROJECT,bigtableInstanceId=$BIGTABLE_INSTANCE_ID,bigtableTableId=$BIGTABLE_TABLE_ID,bigtableAppProfileId=$BIGTABLE_APP_PROFILE_ID,bigtableStartRow=$BIGTABLE_START_ROW,bigtableStopRow=$BIGTABLE_STOP_ROW,bigtableMaxVersions=$BIGTABLE_MAX_VERSIONS,bigtableFilter=$BIGTABLE_FILTER,destinationPath=$DESTINATION_PATH,filenamePrefix=$FILENAME_PREFIX,bigtableReadRpcTimeoutMs=$BIGTABLE_READ_RPC_TIMEOUT_MS,bigtableReadRpcAttemptTimeoutMs=$BIGTABLE_READ_RPC_ATTEMPT_TIMEOUT_MS" \
-Dparameters="bigtableProject=$BIGTABLE_PROJECT,bigtableInstanceId=$BIGTABLE_INSTANCE_ID,bigtableTableId=$BIGTABLE_TABLE_ID,bigtableAppProfileId=$BIGTABLE_APP_PROFILE_ID,bigtableStartRow=$BIGTABLE_START_ROW,bigtableStopRow=$BIGTABLE_STOP_ROW,bigtableMaxVersions=$BIGTABLE_MAX_VERSIONS,bigtableFilter=$BIGTABLE_FILTER,destinationPath=$DESTINATION_PATH,filenamePrefix=$FILENAME_PREFIX,bigtableReadRpcTimeoutMs=$BIGTABLE_READ_RPC_TIMEOUT_MS,bigtableReadRpcAttemptTimeoutMs=$BIGTABLE_READ_RPC_ATTEMPT_TIMEOUT_MS,bigtableMaxAttempts=$BIGTABLE_MAX_ATTEMPTS" \
-f v1
```

Expand Down Expand Up @@ -253,6 +257,7 @@ resource "google_dataflow_job" "cloud_bigtable_to_gcs_sequencefile" {
# bigtableFilter = ""
# bigtableReadRpcTimeoutMs = "<bigtableReadRpcTimeoutMs>"
# bigtableReadRpcAttemptTimeoutMs = "<bigtableReadRpcAttemptTimeoutMs>"
# bigtableMaxAttempts = "<bigtableMaxAttempts>"
}
}
```
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
#NOTE: This is a Maven Resolver internal implementation file, its format can be changed without prior notice.
#Fri Mar 27 14:08:05 EDT 2026
bigtable-beam-import-2.16.2-SNAPSHOT-shaded.jar>=
bigtable-beam-import-2.16.2-SNAPSHOT-tests.jar>=
bigtable-beam-import-2.16.2-SNAPSHOT.jar>=
bigtable-beam-import-2.16.2-SNAPSHOT.pom>=
Binary file not shown.
Binary file not shown.
Loading
Loading