@@ -19,6 +19,35 @@ Integrations with many systems and cloud vendors include (but not limited to):
1919- Microsoft Azure Storage
2020- Alibaba Cloud OSS etc.
2121
22+ ## S3 and S3-compatible object storage
23+
24+ The ` s3:// ` file system is registered when you ` import tensorflow_io ` . It reads
25+ the standard AWS credential and region environment variables
26+ (` AWS_ACCESS_KEY_ID ` , ` AWS_SECRET_ACCESS_KEY ` , ` AWS_REGION ` ), so it works with
27+ Amazon S3 out of the box:
28+
29+ ``` python
30+ import tensorflow as tf
31+ import tensorflow_io as tfio # registers the s3:// file system
32+
33+ dataset = tf.data.TFRecordDataset(" s3://my-bucket/train/shard-00000.tfrecord" )
34+ ```
35+
36+ It also works with any S3-compatible object store, such as Backblaze B2,
37+ Cloudflare R2, MinIO, and others, by pointing the client at the provider
38+ endpoint with the ` S3_ENDPOINT ` environment variable. The client uses path-style
39+ addressing by default, so no extra configuration is required:
40+
41+ ``` bash
42+ export AWS_ACCESS_KEY_ID=< access_key_id>
43+ export AWS_SECRET_ACCESS_KEY=< secret_access_key>
44+ export AWS_REGION=us-west-004
45+ export S3_ENDPOINT=https://s3.us-west-004.backblazeb2.com
46+ ```
47+
48+ The same ` s3:// ` paths work for reading and writing checkpoints and SavedModel
49+ artifacts through ` tf.io.gfile ` , ` tf.train.Checkpoint ` , and ` tf.saved_model ` .
50+
2251## Community
2352
2453* SIG IO [ Google Group] ( https://groups.google.com/a/tensorflow.org/forum/#!forum/io ) and mailing list: [ io@tensorflow.org ] ( io@tensorflow.org )
0 commit comments