TrueNAS Dataset best settings

[FPi-sh]

Hi everyone,
I am using RustFS on top of a TrueNAS server and I am wondering what the best settings for a dataset are.

These are my current settings.
Compression level: LZ4
Atime: OFF
ZFS Deduplication: OFF
Metadata (Special) small block size: 0
Record Size: 128KiB

What is the best value for Record Size?

Thank you

[For8stt]

I’d probably keep recordsize=128K unless you have a clearly measured reason to change it.

For object storage, there isn’t one universally “best” value. It depends mostly on object size and access pattern:

• many small objects → 128K is usually fine
• mostly large objects / sequential reads → 1M can be better
• DB-like random writes → smaller recordsize, but that does not sound like this case

Also, ZFS recordsize is a max block size, not a fixed size for every file. Small files won’t automatically consume a full 128K record. OpenZFS docs also mention that changing recordsize only affects newly written files, not existing ones.

So imo:

compression = lz4
atime = off
dedup = off
recordsize = 128K

is a good default starting point.

If most RustFS objects are large and mostly read sequentially, I’d test 1M on a separate dataset and compare. But I wouldn’t blindly switch production without a small benchmark first.

lmk what your typical object size is — that matters more than the storage backend itself 🙂

[FPi-sh]

Thank you! I already had a dataset with recordsize = 128K that gave me moderate IOPS and acceptable throughput, especially when working with large buckets.

Since most of the files are chunks between 5 and 10 MB, I tried setting recordsize = 1M in a new dataset, and the IOPS/throughput increased!

[For8stt]

Good to know, thanks for testing it.

For 5–10 MB chunks, recordsize=1M makes sense. If you already see better IOPS/throughput with it, I’d keep 1M for this RustFS dataset.

Just keep in mind it mainly helps newly written data, so old data would need to be rewritten/copied to benefit from the new recordsize