Skip to content

Commit 9b089b4

Browse files
committed
Update README with full API coverage and modern examples
1 parent 7c30846 commit 9b089b4

1 file changed

Lines changed: 104 additions & 51 deletions

File tree

README.md

Lines changed: 104 additions & 51 deletions
Original file line numberDiff line numberDiff line change
@@ -1,85 +1,138 @@
1-
[![](https://travis-ci.org/durch/rust-bigtable.svg?branch=master)](https://travis-ci.org/durch/rust-bigtable)
2-
[![](http://meritbadge.herokuapp.com/bigtable)](https://crates.io/crates/bigtable)
31
[![MIT licensed](https://img.shields.io/badge/license-MIT-blue.svg)](https://github.com/durch/rust-bigtable/blob/master/LICENSE.md)
4-
[![Join the chat at https://gitter.im/durch/rust-bigtable](https://badges.gitter.im/durch/rust-bigtable.svg)](https://gitter.im/durch/rust-bigtable?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge&utm_content=badge)
52
[![Mentioned in Awesome Bigtable](https://awesome.re/mentioned-badge-flat.svg)](https://github.com/zrosenbauer/awesome-bigtable)
63

74
## rust-bigtable [[docs](https://durch.github.io/rust-bigtable/)]
85

96
Rust library for working with [Google Bigtable](https://cloud.google.com/bigtable/docs/) [Data API](https://cloud.google.com/bigtable/docs/reference/data/rpc/google.bigtable.v2)
107

11-
### Intro
12-
Interface towards Cloud Bigtable, supports all [Data API](https://cloud.google.com/bigtable/docs/reference/data/rpc/google.bigtable.v2) methods.
8+
### Supported API Methods
139

14-
+ CheckAndMutateRow
15-
+ MutateRow
16-
+ MutateRows
17-
+ ReadModifyWriteRow
18-
+ ReadRows
19-
+ SampleRowKeys
10+
Full coverage of the Bigtable Data API v2:
2011

21-
Includes support for [JWT auth](https://cloud.google.com/docs/authentication):
12+
| Method | Description |
13+
|--------|-------------|
14+
| `ReadRows` | Streams back the contents of all requested rows |
15+
| `SampleRowKeys` | Returns a sample of row keys in the table |
16+
| `MutateRow` | Mutates a row atomically |
17+
| `MutateRows` | Mutates multiple rows in a batch |
18+
| `CheckAndMutateRow` | Mutates a row atomically based on output of a predicate filter |
19+
| `ReadModifyWriteRow` | Modifies a row atomically on the server |
20+
| `PingAndWarm` | Warms up connection channels to the service |
21+
| `GenerateInitialChangeStreamPartitions` | Generates initial change stream partitions |
22+
| `ReadChangeStream` | Reads changes from a table's change stream |
23+
| `PrepareQuery` | Prepares a GoogleSQL query for execution |
24+
| `ExecuteQuery` | Executes a GoogleSQL query against a Bigtable table |
2225

23-
### How it works
26+
### How It Works
2427

25-
Initial plans was to go full `grpc` over `http/2`, unfortunately Rust support is not there yet, so a middle way was taken :).
28+
Requests are `protobuf` messages generated from [Google's proto definitions](https://github.com/googleapis/googleapis/blob/master/google/bigtable/v2/bigtable.proto). These messages are converted to JSON and sent to the predefined REST endpoints. Responses are returned as `serde_json::Value`.
2629

27-
Requests objects are `protobuf` messages, generated using `proto` definitions available from [Google](https://github.com/googleapis/googleapis/blob/master/google/bigtable/v2/bigtable.proto). And all configuration is done through very nice interfaces generated in this way. These messages are then transparently converted to `json`, and sent to predefined `google.api.http` endpoints, also defined [here](https://github.com/googleapis/googleapis/blob/master/google/bigtable/v2/bigtable.proto). Responses are returned as [serde_json::Value](https://docs.serde.rs/serde_json/value/index.html).
30+
Authentication is handled via [goauth](https://crates.io/crates/goauth) with JWT tokens.
2831

29-
In theory this should enable easy upgrade to full `grpc` over `http/2` as soon as it becomes viable, the only remaining work would be utilising proper return types, also available as `protobuf` messages.
32+
### Installation
3033

31-
### Configuration
34+
```toml
35+
[dependencies]
36+
bigtable = "0.6"
37+
```
3238

33-
You can provide the `json` service accounts credentials obtained from Google Cloud Console or the private key file in `pem` or (see [random_rsa_for_testing](https://github.com/durch/rust-bigtable/blob/master/random_rsa_for_tests) for proper format) format as well as Google Cloud service account with proper scopes (scopes are handled by [goauth](https://crates.io/crates/goauth), as part of authentication),
39+
### Configuration
3440

35-
### Usage
41+
Provide service account credentials from Google Cloud Console as a JSON key file:
3642

37-
*In your Cargo.toml*
43+
```rust
44+
use bigtable::utils::get_auth_token;
3845

39-
```
40-
[dependencies]
41-
bigtable = '0.3'
46+
let token = get_auth_token("service-account-key.json", true)?;
4247
```
4348

44-
#### Higher level wrappers (`wraps`)
49+
### Usage
50+
51+
#### High-Level Wrappers
4552

46-
There and higher wrappers available for reading and writing rows, so there is not need to craft `protobufs` manually. Write can also be used to update, but not very robustly yet, coming soon :).
53+
Simple wrappers for common operations:
4754

48-
##### Read and Write
55+
```rust
56+
use bigtable::utils::get_auth_token;
57+
use bigtable::wraps;
58+
use bigtable::support::Table;
59+
60+
// Read rows with limit
61+
let token = get_auth_token("credentials.json", true)?;
62+
let table = Table::default();
63+
let rows = wraps::read_rows(&table, &token, Some(100))?;
64+
65+
// Bulk write rows (uses MutateRows - higher throughput)
66+
let mut rows = vec![wraps::Row::default()];
67+
wraps::bulk_write_rows(&mut rows, &token, table.clone())?;
68+
69+
// Write rows one at a time (uses ReadModifyWriteRow)
70+
let mut rows = vec![wraps::Row::default()];
71+
wraps::write_rows(&mut rows, &token, &table)?;
72+
```
4973

50-
Read wrappers allows for simple limit on the number of rows, it uses the `ReadRows` underlying method.
74+
#### Direct API Access
5175

52-
There are two write strategies available, `bulk_write_rows` and `write_rows`. `bulk_write_rows` first collects all writes and fires only one request, underlying method is `MutateRows`, this results in a much higher write throughput. `write_rows` shoots one request per row to be written, underlying method is `ReadModifyWriteRow`.
76+
For full control, use the request builder directly:
5377

5478
```rust
79+
use bigtable::request::BTRequest;
80+
use bigtable::method::{BigTable, ReadRows, MutateRow};
81+
use bigtable::utils::{get_auth_token, encode_str};
82+
use bigtable::protos::data::{Mutation, mutation};
83+
84+
let token = get_auth_token("credentials.json", true)?;
85+
86+
// ReadRows
87+
let mut req = BTRequest {
88+
base: None,
89+
table: Default::default(),
90+
method: ReadRows::new(),
91+
};
92+
req.method.payload_mut().rows_limit = 10;
93+
let response = req.execute(&token)?;
94+
95+
// MutateRow with SetCell
96+
let mut req = BTRequest {
97+
base: None,
98+
table: Default::default(),
99+
method: MutateRow::new(),
100+
};
101+
102+
let mut set_cell = mutation::SetCell::new();
103+
set_cell.family_name = String::from("cf1");
104+
set_cell.column_qualifier = encode_str("col1");
105+
set_cell.timestamp_micros = -1;
106+
set_cell.value = encode_str("value1");
107+
108+
let mut m = Mutation::new();
109+
m.mutation = Some(mutation::Mutation::SetCell(set_cell));
110+
111+
req.method.payload_mut().row_key = encode_str("row1");
112+
req.method.payload_mut().mutations.push(m);
113+
114+
let response = req.execute(&token)?;
115+
```
55116

56-
extern crate bigtable as bt;
117+
### Testing
57118

58-
use bt::utils::*;
59-
use bt::wraps;
119+
Integration tests run against a live Bigtable instance:
60120

61-
const TOKEN_URL: &'static str = "https://www.googleapis.com/oauth2/v4/token";
62-
const ISS: &'static str = "some-service-account@developer.gserviceaccount.com";
63-
const PK: &'static str = "pk_for_the_acc_above.pem";
121+
```bash
122+
# Run integration tests (requires credentials)
123+
cargo test --test integration_tests -- --ignored --test-threads=1
64124

65-
fn read_rows(limit: i64) -> Result<(serde_json::Value), BTErr> {
125+
# Run doc tests
126+
cargo test
127+
```
66128

67-
let token = get_auth_token(TOKEN_URL, ISS, PK)?;
68-
let table = Default::default();
129+
### Dependencies
69130

70-
wraps::read_rows(table, &token, Some(limit))
131+
- `protobuf` / `protobuf-json-mapping` - Protocol buffer handling and JSON conversion
132+
- `goauth` / `smpl_jwt` - Google OAuth2 / JWT authentication
133+
- `curl` - HTTP client
134+
- `serde_json` - JSON serialization
71135

72-
}
136+
### License
73137

74-
fn write_rows(n: usize, bulk: bool) -> Result<(), BTErr> {
75-
let mut rows: Vec<wraps::Row> = vec!(wraps::Row::default()); // put some real data here
76-
let token = get_auth_token(TOKEN_URL, ISS, PK)?;
77-
let table = Default::default(); // Again use a real table here
78-
if bulk {
79-
let _ = wraps::bulk_write_rows(&mut rows, &token, table);
80-
} else {
81-
let _ = wraps::write_rows(&mut rows, &token, table);
82-
}
83-
Ok(())
84-
}
85-
```
138+
MIT

0 commit comments

Comments
 (0)