You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: README.md
+33-15Lines changed: 33 additions & 15 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -40,7 +40,7 @@ This repository contains JS/TS BoilingData SDK. Please see the integration tests
40
40
41
41
### Callbacks
42
42
43
-
The SDK uses the BoilingData Websocket API in the background, meaning that events can arrive at any time. We use a range of global and query-specific callbacks to allow you to hook into the events that you care about.
43
+
The SDK uses the BoilingData Websocket API in the background, meaning that events can arrive at any time. We use a range of global and query-specific callbacks to allow you to hook into the events that you care about.
44
44
45
45
All callbacks work in both the global scope and the query scope; i.e. global callbacks will always be executed when a message arrives, query callbacks will only be executed when messages relating to that query arrive.
46
46
@@ -58,37 +58,45 @@ All callbacks work in both the global scope and the query scope; i.e. global cal
58
58
59
59
#### Setting Global Callbacks
60
60
61
-
Global callbacks can be set when creating the BoilingData instance.
61
+
Global callbacks can be set when creating the BoilingData instance.
62
+
62
63
```typescript
63
-
newBoilingData({
64
-
username, password,
64
+
newBoilingData({
65
+
username,
66
+
password,
65
67
globalCallbacks: {
66
-
onRequest: req=> { console.log("A new request has been made with ID", req.requestId)},
console.log("A new request has been made with ID", req.requestId);
70
+
},
71
+
onQueryFinished: req=> {
72
+
console.log("Request complete!", req.requestId);
73
+
},
74
+
onLogError: message=> {
75
+
console.error("LogError", message);
76
+
},
77
+
onSocketOpen: socketInstance=> {
78
+
console.log("The socket has opened!");
71
79
},
72
80
onLambdaEvent: message=> {
73
-
console.log("Change in status of dataset: ",message)
74
-
}
75
-
}
81
+
console.log("Change in status of dataset: ",message);
82
+
},
83
+
},
76
84
});
77
85
```
78
86
79
87
#### Setting Query-level Callbacks
80
88
81
89
Query callbacks are set when creating the query
90
+
82
91
```typescript
83
92
bdInstance.execQuery({
84
93
sql: `SELECT COUNT(*) AS count FROM parquet_scan('s3://boilingdata-demo/demo2.parquet');`,
85
94
callbacks: {
86
95
onData: data=> {
87
-
console.log("Some data for this query arrived",data)
96
+
console.log("Some data for this query arrived",data);
88
97
},
89
98
onQueryFinished: () =>resolve(r),
90
99
onLogError: (data:any) =>reject(data),
91
-
92
100
},
93
101
});
94
102
```
@@ -106,6 +114,16 @@ bdInstance.execQuery(
106
114
"s3://bucket/data/2022-01-03.parquet",
107
115
])
108
116
```
117
+
109
118
Results are streamed as soon as they are ready, so it is unlikely that you will recieve results in the same order that you specified the files.
110
119
111
-
If you do not need to query multiple files, then you do not need to specify the keys, for instance `SELECT COUNT(*) as rowCount FROM parquet_scan('s3://bucket/data/2022-01-01.parquet');`,
120
+
If you do not need to query multiple files, then you do not need to specify the keys, for instance `SELECT COUNT(*) as rowCount FROM parquet_scan('s3://bucket/data/2022-01-01.parquet');`.
121
+
122
+
You can also now query Glue (Hive) Tables:
123
+
124
+
```typescript
125
+
bdInstance.execQuery(
126
+
sql: `SELECT 's3://KEY' as fileLocation, COUNT(*) as rowCount FROM parquet_scan('s3://KEY');`,
0 commit comments