Skip to content

Commit a815c3c

Browse files
Add thin wrapper to replicate tiledb-cloud-py API (#28)
* Add a wrapper as a facade to immitate cloud-py's api * Added option for user to select specific attribute to return * Fix downloadNotebookContents method and add UDF class * Add UDF methods * Fix downloadNotebookToFile method * Fix unshare array/udf * Tests for the client and udf classes * Added Sql methods * Add queries to the client and update README.md * Fix tests (dataToQuery.test.ts)
1 parent 123dcaa commit a815c3c

45 files changed

Lines changed: 1602 additions & 158 deletions

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

README.md

Lines changed: 83 additions & 131 deletions
Original file line numberDiff line numberDiff line change
@@ -19,24 +19,17 @@ npm install @tiledb-inc/tiledb-cloud
1919
### Basic usage:
2020

2121
```javascript
22-
const tiledb = require("@tiledb-inc/tiledb-cloud");
22+
import client = from "@tiledb-inc/tiledb-cloud";
2323

2424
// API tokens are the recommend way to access the cloud apis
25-
const config = new tiledb.v1.Configuration({
25+
const config = {
2626
apiKey: "<insert token from setting page here>"
27-
});
28-
29-
// Username and passwords are also supported, uncomment below and comment out above to use username/password auth instead
30-
// const config = {
31-
// username: "<tiledb-cloud-username>",
32-
// password: "<tiledb-cloud-password>"
33-
// };
27+
};
3428

35-
// First we must create API objects.
36-
// In the future we will improve and simplify this interface
37-
const arrayApi = new tiledb.v1.ArrayApi(config);
29+
// First we must create the client.
30+
const tiledbClient = new client(config);
3831

39-
arrayApi.getArraysInNamespace('<tiledb-cloud-username>').then((res) => {
32+
tiledbClient.userProfile().then((res) => {
4033
console.log(res);
4134
}).catch((ex) => {
4235
console.log(ex);
@@ -49,32 +42,21 @@ arrayApi.getArraysInNamespace('<tiledb-cloud-username>').then((res) => {
4942

5043
```javascript
5144

52-
const tiledb = require('@tiledb-inc/tiledb-cloud');
45+
import client = from "@tiledb-inc/tiledb-cloud";
5346

5447
// API tokens are the recommend way to access the cloud apis
55-
const config = new tiledb.v1.Configuration({
48+
const config = {
5649
apiKey: "<insert token from setting page here>"
57-
});
58-
59-
// Username and passwords are also supported, uncomment below and comment out above to use username/password auth instead
60-
// const config = {
61-
// username: "<tiledb-cloud-username>",
62-
// password: "<tiledb-cloud-password>"
63-
// };
50+
};
6451

65-
// First we must create API objects.
66-
// In the future we will improve and simplify this interface
67-
const sqlAPI = new tiledb.v1.SqlApi(config);
52+
// Create a client instance
53+
const tiledbClient = new client(config);
6854

6955
// SQL query
7056
const sql = "select `rows`, AVG(a) as avg_a from `tiledb://TileDB-Inc/quickstart_dense` GROUP BY `rows`";
7157

72-
const sqlDetails = {
73-
query: sql
74-
};
75-
7658
// Run SQL and print any returned data to console
77-
sqlAPI.runSQL("<tiledb-cloud-username>", sqlDetails).then((res) => {
59+
tiledbClient.sql.exec("<tiledb-cloud-username>", sql).then((res) => {
7860
console.log(res.data)
7961
/*
8062
Will print:
@@ -92,56 +74,6 @@ sqlAPI.runSQL("<tiledb-cloud-username>", sqlDetails).then((res) => {
9274

9375
```
9476

95-
96-
## Cap'n Proto serialization
97-
98-
Endpoints that support cap'n proto mime type user should set appropriate headers (Accept for GET requests and Content-Type for POST/PUT) to `application/capnp`.
99-
100-
For `POST` requests library will automatically serialize data to cap'n proto.
101-
102-
```javascript
103-
const tiledb = require("@tiledb-inc/tiledb-cloud");
104-
105-
const config = new tiledb.v1.Configuration({
106-
apiKey: "<insert token from setting page here>"
107-
});
108-
const arrayApi = new tiledb.v1.ArrayApi(config);
109-
110-
arrayApi
111-
.updateArrayMetadataCapnp(
112-
"ns",
113-
"array_name",
114-
data,
115-
{
116-
headers: {
117-
"Content-Type": "application/capnp",
118-
},
119-
})
120-
```
121-
122-
For `GET` requests library provides methods to deserialize data. If Accept header is set to "application/capnp", library will set `responseType` to `arraybuffer` since helpers accept [ArrayBuffer](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/ArrayBuffer) to deserialize the response (user can still override `responseType`).
123-
`deserializeCapnp` helper function receives an `ArrayBuffer` and the type of the data expected. Library provides a `DeserializableType` enum for all the supported types.
124-
125-
```javascript
126-
const tiledb = require("@tiledb-inc/tiledb-cloud");
127-
128-
const config = new tiledb.v1.Configuration({
129-
apiKey: "<insert token from setting page here>"
130-
});
131-
const arrayApi = new tiledb.v1.ArrayApi(config);
132-
133-
arrayApi
134-
.getArrayMetadataCapnp("ns", "array_name", {
135-
headers: {
136-
"Accept": "application/capnp",
137-
},
138-
})
139-
.then((data) => {
140-
// data.data is an ArrayBuffer
141-
console.log(tiledb.deserializeCapnp(data.data, tiledb.DeserializableType.arrayMetadata));
142-
})
143-
```
144-
14577
## Queries
14678

14779
TileDB-Cloud-JS supports TileDB queries, by serializing data to capnproto. `bufferSize` dictates the server the number of bytes that should allocated to make this query. In case the `bufferSize` is not enough, it will result to an incomplete query. For this reason `ReadQuery` is an async generator so a user could get results in batches.
@@ -151,23 +83,23 @@ TileDB-Cloud-JS supports TileDB queries, by serializing data to capnproto. `buff
15183
Dimensions should always be an array of 2 (start of the range and the end of the range).
15284

15385
```javascript
154-
import { TileDBQuery } from '@tiledb-inc/tiledb-cloud';
155-
import { Layout } from '@tiledb-inc/tiledb-cloud/lib/v2';
86+
import client = from "@tiledb-inc/tiledb-cloud";
15687

157-
const tileDBQuery = new TileDBQuery({
158-
apiKey: 'myApiKey'
159-
});
88+
// API tokens are the recommend way to access the cloud apis
89+
const config = {
90+
apiKey: "<insert token from setting page here>"
91+
};
16092

161-
const dimension1 = [636800,637800];
162-
const dimension2 = [851000,853000];
93+
// Create a client instance
94+
const tiledbClient = new client(config);
16395

16496
const query = {
165-
layout: Layout.RowMajor,
97+
layout: "row-major",
16698
ranges: [dimension1, dimension2],
16799
bufferSize: 1500000000,
168100
};
169101

170-
const generator = tileDBQuery.ReadQuery("namespace", "arrayName", query);
102+
const generator = tiledbClient.query.ReadQuery("namespace", "arrayName", query);
171103
generator.next().then(({value}) => {
172104
console.log(value)
173105
});
@@ -178,26 +110,29 @@ generator.next().then(({value}) => {
178110
Dimensions should always be an array of 2 (start of the range and the end of the range).
179111

180112
```javascript
181-
import { TileDBQuery } from '@tiledb-inc/tiledb-cloud';
182-
import { Layout } from '@tiledb-inc/tiledb-cloud/lib/v2';
113+
import client = from "@tiledb-inc/tiledb-cloud";
183114

184-
const tileDBQuery = new TileDBQuery({
185-
apiKey: 'myApiKey'
186-
});
115+
// API tokens are the recommend way to access the cloud apis
116+
const config = {
117+
apiKey: "<insert token from setting page here>"
118+
};
119+
120+
// Create a client instance
121+
const tiledbClient = new client(config);
187122

188123
const dimension1 = [636800,637800];
189124
const dimension2 = [851000,853000];
190125

191126
const query = {
192-
layout: Layout.RowMajor,
127+
layout: "row-major",
193128
ranges: [dimension1, dimension2],
194129
bufferSize: 1500,
195130
};
196131

197132

198133
// Iterate over all results
199134
(async function() {
200-
for await (let results of tileDBQuery.ReadQuery("namespace", "arrayName", query)) {
135+
for await (let results of tiledbClient.query.ReadQuery("namespace", "arrayName", query)) {
201136
console.log(results);
202137
}
203138
})()
@@ -219,25 +154,28 @@ const generator = tileDBQuery.ReadQuery("namespace", "arrayName", query);
219154
A dimension could be an array of ranges as well
220155

221156
```javascript
222-
import { TileDBQuery } from '@tiledb-inc/tiledb-cloud';
223-
import { Layout } from '@tiledb-inc/tiledb-cloud/lib/v2';
157+
import client = from "@tiledb-inc/tiledb-cloud";
224158

225-
const tileDBQuery = new TileDBQuery({
226-
apiKey: 'myApiKey'
227-
});
159+
// API tokens are the recommend way to access the cloud apis
160+
const config = {
161+
apiKey: "<insert token from setting page here>"
162+
};
163+
164+
// Create a client instance
165+
const tiledbClient = new client(config);
228166

229167
const dimension1 = [636800,637800];
230168
const dimension2 = [[1577836800, 1588878856], [1577836800, 1578878856]];
231169

232170
const query = {
233-
layout: Layout.RowMajor,
171+
layout: "row-major",
234172
ranges: [dimension1, dimension2],
235173
bufferSize: 15000000000000,
236174
};
237175

238176
// Iterate over all results
239177
(async function() {
240-
for await (let results of tileDBQuery.ReadQuery("my_namespace", "my_array", query)) {
178+
for await (let results of tiledbClient.query.ReadQuery("my_namespace", "my_array", query)) {
241179
// returns an object with keys the name of the attributes and values the result
242180
console.log(results);
243181
}
@@ -249,26 +187,29 @@ const query = {
249187
By setting a dimension as an empty array, query will select the whole dimension.
250188

251189
```javascript
252-
import { TileDBQuery } from '@tiledb-inc/tiledb-cloud';
253-
import { Layout } from '@tiledb-inc/tiledb-cloud/lib/v2';
190+
import client = from "@tiledb-inc/tiledb-cloud";
254191

255-
const tileDBQuery = new TileDBQuery({
256-
apiKey: 'myApiKey'
257-
});
192+
// API tokens are the recommend way to access the cloud apis
193+
const config = {
194+
apiKey: "<insert token from setting page here>"
195+
};
196+
197+
// Create a client instance
198+
const tiledbClient = new client(config);
258199

259200
const dimension1 = [636800,637800];
260201
// Setting empty array, query will select whole 2nd dimension
261202
const dimension2 = [];
262203

263204
const query = {
264-
layout: Layout.RowMajor,
205+
layout: "row-major",
265206
ranges: [dimension1, dimension2],
266207
bufferSize: 15000000000000,
267208
};
268209

269210
// Iterate over all results
270211
(async function() {
271-
for await (let results of tileDBQuery.ReadQuery("my_namespace", "my_array", query)) {
212+
for await (let results of tiledbClient.query.ReadQuery("my_namespace", "my_array", query)) {
272213
// returns an object with keys the name of the attributes and values the result
273214
console.log(results);
274215
}
@@ -281,15 +222,19 @@ const query = {
281222
For write queries user should provide an object with the attribute values and the coordinates of the cells (rows and cols in the object below). In this case we are writing to cells [1, 1] up to [1, 3].
282223

283224
```javascript
284-
import { TileDBQuery } from '@tiledb-inc/tiledb-cloud';
285-
import { Layout } from '@tiledb-inc/tiledb-cloud/lib/v2';
225+
import client = from "@tiledb-inc/tiledb-cloud";
226+
227+
// API tokens are the recommend way to access the cloud apis
228+
const config = {
229+
apiKey: "<insert token from setting page here>"
230+
};
231+
232+
// Create a client instance
233+
const tiledbClient = new client(config);
286234

287-
const tileDBQuery = new TileDBQuery({
288-
apiKey: 'myApiKey'
289-
});
290235

291236
const query = {
292-
layout: Layout.Unordered,
237+
layout: "unordered",
293238
values: {
294239
rows: {
295240
values: [1, 1, 1],
@@ -306,7 +251,7 @@ const query = {
306251
},
307252
};
308253

309-
tileDBQuery.WriteQuery("my_namespace", "my_array", query)
254+
tiledbClient.query.WriteQuery("my_namespace", "my_array", query)
310255
.then((result) => {
311256
// returns the query object
312257
console.log(result);
@@ -318,15 +263,18 @@ tileDBQuery.WriteQuery("my_namespace", "my_array", query)
318263
For Dense arrays we can provide a subarray instead of the coordinates and set the order (e.g. layout set to row-major).
319264

320265
```javascript
321-
import { TileDBQuery } from '@tiledb-inc/tiledb-cloud';
322-
import { Layout } from '@tiledb-inc/tiledb-cloud/lib/v2';
266+
import client = from "@tiledb-inc/tiledb-cloud";
323267

324-
const tileDBQuery = new TileDBQuery({
325-
apiKey: 'myApiKey'
326-
});
268+
// API tokens are the recommend way to access the cloud apis
269+
const config = {
270+
apiKey: "<insert token from setting page here>"
271+
};
272+
273+
// Create a client instance
274+
const tiledbClient = new client(config);
327275

328276
const query = {
329-
layout: Layout.RowMajor,
277+
layout: "row-major",
330278
subarray: [[1, 1], [1, 3]],
331279
values: {
332280
attr1: {
@@ -338,7 +286,7 @@ const query = {
338286
},
339287
};
340288

341-
tileDBQuery.WriteQuery("my_namespace", "my_array", query)
289+
tiledbClient.query.WriteQuery("my_namespace", "my_array", query)
342290
.then((result) => {
343291
// returns the query object
344292
console.log(result);
@@ -350,15 +298,19 @@ tileDBQuery.WriteQuery("my_namespace", "my_array", query)
350298
For nullables and var-length attributes user should provide `validity` attribute and/or the `offsets`.
351299

352300
```javascript
353-
import { TileDBQuery } from '@tiledb-inc/tiledb-cloud';
354-
import { Layout } from '@tiledb-inc/tiledb-cloud/lib/v2';
301+
import client = from "@tiledb-inc/tiledb-cloud";
302+
303+
// API tokens are the recommend way to access the cloud apis
304+
const config = {
305+
apiKey: "<insert token from setting page here>"
306+
};
307+
308+
// Create a client instance
309+
const tiledbClient = new client(config);
355310

356-
const tileDBQuery = new TileDBQuery({
357-
apiKey: 'myApiKey'
358-
});
359311

360312
const query = {
361-
layout: Layout.Unordered,
313+
layout: "unordered",
362314
values: {
363315
a1: {
364316
values: [100, 200],
@@ -383,7 +335,7 @@ const query = {
383335
}
384336
};
385337

386-
tileDBQuery.WriteQuery("my_namespace", "my_array", query)
338+
tiledbClient.query.WriteQuery("my_namespace", "my_array", query)
387339
.then((result) => {
388340
// returns the query object
389341
console.log(result);

lib/Sql/Sql.d.ts

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,8 @@
1+
import { Configuration, ConfigurationParameters, SqlApi } from "../v1";
2+
declare class Sql {
3+
config: Configuration;
4+
API: SqlApi;
5+
constructor(params: ConfigurationParameters);
6+
exec(namespace: string, query: string): Promise<import("axios").AxiosResponse<object[]>>;
7+
}
8+
export default Sql;

0 commit comments

Comments
 (0)