Skip to content

Commit 8d38ea2

Browse files
committed
small restructuring of userguide
1 parent 85c75de commit 8d38ea2

2 files changed

Lines changed: 23 additions & 37 deletions

File tree

README.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -12,7 +12,7 @@ For comprehensive documentation, see the [**User Guide**](USERGUIDE.md), which i
1212
- Working with arrays and groups
1313
- Storage backends (Filesystem, HTTP, S3, ZIP, Memory)
1414
- Compression and codecs
15-
- Advanced topics and best practices
15+
- Best practices
1616
- Troubleshooting
1717

1818
## Quick Usage Example

USERGUIDE.md

Lines changed: 22 additions & 36 deletions
Original file line numberDiff line numberDiff line change
@@ -654,20 +654,31 @@ try {
654654
- `"No Zarr array found at the specified location"` - Check path and ensure `.zarray` (v2) or `zarr.json` (v3) exists
655655
- `"Requested data is outside of the array's domain"` - Verify that `offset + shape <= array.shape`
656656
- `"Failed to read from store"` - Check network connectivity, file permissions, or storage availability
657+
---
658+
657659
### Best Practices
658-
1. **Chunk sizes for Best Performance**:
659-
- refer to [Zarr Performance Guide](
660-
https://zarr.readthedocs.io/en/latest/user-guide/performance/) for recommendations
660+
1. **Chunk sizes for Best Performance**:
661+
- refer to [Zarr Performance Guide](
662+
https://zarr.readthedocs.io/en/latest/user-guide/performance/) for recommendations
661663
2. **Use compression**: Almost always beneficial for scientific data
662-
- Blosc is fast and effective for most use cases
663-
- Zstd for better compression ratios
664-
- Gzip for compatibility
664+
- Blosc is fast and effective for most use cases
665+
- Zstd for better compression ratios
666+
- Gzip for compatibility
665667
3. **Batch writes**: Write larger chunks at once rather than many small writes
666668
4. **Consider sharding**: For v3 arrays with many small chunks
667669
```java
668670
.withCodecs(c -> c.withSharding(new int[]{10, 10, 10}, inner -> inner.withBlosc()))
669671
```
670-
---
672+
5. **Access patterns**: Align chunk shape with your access pattern
673+
```java
674+
// For row-wise access
675+
.withChunkShape(1, 1000, 1000) // Read entire rows efficiently
676+
// For column-wise access
677+
.withChunkShape(1000, 1, 1000) // Read entire columns efficiently
678+
// For balanced 3D access
679+
.withChunkShape(100, 100, 100) // Balanced for all dimensions
680+
```
681+
671682
## API Reference
672683
### Array Methods
673684
#### Creation and Opening
@@ -915,6 +926,7 @@ public class ParallelIOExample {
915926
### Common Issues
916927
**Problem**: `ZarrException: No Zarr array found at the specified location`
917928
**Solution**: Check that the path is correct and contains `.zarray` (v2) or `zarr.json` (v3)
929+
918930
**Problem**: `OutOfMemoryError` when reading large arrays
919931
**Solution**: Read smaller subsets or increase JVM heap size with `-Xmx`
920932
```bash
@@ -927,6 +939,7 @@ java -Xmx8g -jar myapp.jar
927939
- Use appropriate compression (Blosc is fastest)
928940
- Check network bandwidth (for HTTP/S3)
929941
- For debugging, you can disable parallelism: `array.read(offset, shape, false)`
942+
930943
**Problem**: `IllegalArgumentException: 'offset' needs to have rank...`
931944
**Solution**: Ensure offset and shape arrays match the array's number of dimensions
932945
```java
@@ -939,7 +952,7 @@ array.read(new long[]{0, 0}, new long[]{10, 10}); // Wrong rank!
939952
**Solution**:
940953
- Verify data type matches between write and read
941954
- Check compression codec compatibility
942-
- Ensure proper store closing (especially ZIP stores)
955+
943956
**Problem**: `ZarrException: Requested data is outside of the array's domain`
944957
**Solution**: Check that `offset + shape <= array.shape` for all dimensions
945958
```java
@@ -972,34 +985,7 @@ try {
972985
store.close(); // Important!
973986
}
974987
```
975-
### Performance Tips
976-
1. **Chunk size optimization**:
977-
```java
978-
// Too small (many I/O operations)
979-
.withChunkShape(10, 10, 10) // ~1KB chunks
980-
// Good balance
981-
.withChunkShape(100, 100, 100) // ~1MB chunks (for UINT8)
982-
// May be too large (high memory usage)
983-
.withChunkShape(1000, 1000, 1000) // ~1GB chunks
984-
```
985-
2. **Access patterns**: Align chunk shape with your access pattern
986-
```java
987-
// For row-wise access
988-
.withChunkShape(1, 1000, 1000) // Read entire rows efficiently
989-
// For column-wise access
990-
.withChunkShape(1000, 1, 1000) // Read entire columns efficiently
991-
// For balanced 3D access
992-
.withChunkShape(100, 100, 100) // Balanced for all dimensions
993-
```
994-
3. **Compression trade-offs**:
995-
```java
996-
// Fastest (minimal compression)
997-
.withCodecs(c -> c.withBlosc("lz4", "noshuffle", 1))
998-
// Balanced (good speed and compression)
999-
.withCodecs(c -> c.withBlosc("zstd", "shuffle", 5))
1000-
// Best compression (slower)
1001-
.withCodecs(c -> c.withZstd(22))
1002-
```
988+
1003989
### Getting Help
1004990
- **GitHub Issues**: [github.com/zarr-developers/zarr-java/issues](https://github.com/zarr-developers/zarr-java/issues)
1005991
- **Zarr Community**: [zarr.dev](https://zarr.dev/)

0 commit comments

Comments
 (0)