Skip to content

Commit 3e352d3

Browse files
committed
Add example of reading Glue Table into README
1 parent 17d1356 commit 3e352d3

1 file changed

Lines changed: 33 additions & 15 deletions

File tree

README.md

Lines changed: 33 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -40,7 +40,7 @@ This repository contains JS/TS BoilingData SDK. Please see the integration tests
4040

4141
### Callbacks
4242

43-
The SDK uses the BoilingData Websocket API in the background, meaning that events can arrive at any time. We use a range of global and query-specific callbacks to allow you to hook into the events that you care about.
43+
The SDK uses the BoilingData Websocket API in the background, meaning that events can arrive at any time. We use a range of global and query-specific callbacks to allow you to hook into the events that you care about.
4444

4545
All callbacks work in both the global scope and the query scope; i.e. global callbacks will always be executed when a message arrives, query callbacks will only be executed when messages relating to that query arrive.
4646

@@ -58,37 +58,45 @@ All callbacks work in both the global scope and the query scope; i.e. global cal
5858

5959
#### Setting Global Callbacks
6060

61-
Global callbacks can be set when creating the BoilingData instance.
61+
Global callbacks can be set when creating the BoilingData instance.
62+
6263
```typescript
63-
new BoilingData({
64-
username, password,
64+
new BoilingData({
65+
username,
66+
password,
6567
globalCallbacks: {
66-
onRequest: req => { console.log("A new request has been made with ID", req.requestId)},
67-
onQueryFinished: req => { console.log("Request complete!", req.requestId)},
68-
onLogError: message => { console.error("LogError",message)},
69-
onSocketOpen: (socketInstance) => {
70-
console.log("The socket has opened!")
68+
onRequest: req => {
69+
console.log("A new request has been made with ID", req.requestId);
70+
},
71+
onQueryFinished: req => {
72+
console.log("Request complete!", req.requestId);
73+
},
74+
onLogError: message => {
75+
console.error("LogError", message);
76+
},
77+
onSocketOpen: socketInstance => {
78+
console.log("The socket has opened!");
7179
},
7280
onLambdaEvent: message => {
73-
console.log("Change in status of dataset: ",message)
74-
}
75-
}
81+
console.log("Change in status of dataset: ", message);
82+
},
83+
},
7684
});
7785
```
7886

7987
#### Setting Query-level Callbacks
8088

8189
Query callbacks are set when creating the query
90+
8291
```typescript
8392
bdInstance.execQuery({
8493
sql: `SELECT COUNT(*) AS count FROM parquet_scan('s3://boilingdata-demo/demo2.parquet');`,
8594
callbacks: {
8695
onData: data => {
87-
console.log("Some data for this query arrived",data)
96+
console.log("Some data for this query arrived", data);
8897
},
8998
onQueryFinished: () => resolve(r),
9099
onLogError: (data: any) => reject(data),
91-
92100
},
93101
});
94102
```
@@ -106,6 +114,16 @@ bdInstance.execQuery(
106114
"s3://bucket/data/2022-01-03.parquet",
107115
])
108116
```
117+
109118
Results are streamed as soon as they are ready, so it is unlikely that you will recieve results in the same order that you specified the files.
110119

111-
If you do not need to query multiple files, then you do not need to specify the keys, for instance `SELECT COUNT(*) as rowCount FROM parquet_scan('s3://bucket/data/2022-01-01.parquet');`,
120+
If you do not need to query multiple files, then you do not need to specify the keys, for instance `SELECT COUNT(*) as rowCount FROM parquet_scan('s3://bucket/data/2022-01-01.parquet');`.
121+
122+
You can also now query Glue (Hive) Tables:
123+
124+
```typescript
125+
bdInstance.execQuery(
126+
sql: `SELECT 's3://KEY' as fileLocation, COUNT(*) as rowCount FROM parquet_scan('s3://KEY');`,
127+
keys: [ "glue.default.nyctaxis" ]
128+
)
129+
```

0 commit comments

Comments
 (0)