Skip to content

Commit b94a0b1

Browse files
author
Jeremy Khawaja
authored
copy paginated code into branch (#82)
1 parent eb034c1 commit b94a0b1

20 files changed

Lines changed: 3573 additions & 3 deletions

paginated/README.md

Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,20 @@
1+
# `paginated` tensors
2+
3+
This package is not ready for production usage or release. Use at your own risk. The API will try to be maintained, but the only API guarantees at this time are the Gorgonia [`tensor.Tensor`](https://github.com/gorgonia/tensor/blob/master/tensor.go#L27) API.
4+
5+
No behavior is currently guaranteed until a `1.0.0` architecture has been decided on, tested, benchmarked, profiled, and released.
6+
7+
## Overview
8+
9+
Allows for arbitrarily large tensors that live permanently on disk but are buffered in memory when used. The size of the in-memory buffer can be specified by a page size (in values) and the number of pages allowed in memory. Three default buffer size options are available: small (1/200 of system memory), medium (1/20 of system memory), and large (1/2 of system memory). But any buffer size less than system memory size can be specified.
10+
11+
The tensors are also easily shareable. Composed of only two files: a `main.json` file which specifies the tensors metadata, and an `index.gob.gz` file which specifies an index from which the tensor can be rebuilt.
12+
13+
## Contribute
14+
15+
If you would like to contribute then please contribute: tests, benchmarks, documentation, and concurrency-safe code suggestions.
16+
17+
## TODO
18+
19+
- tests, benchmarks, profiling
20+
- concurrency-safe code

0 commit comments

Comments
 (0)