Use ShardBasedBuilder for HuggingFace datasets

This is a more efficient way and simpler way to generate HuggingFace datasets. This can save a lot of time and space, especially for large datasets.

This works on Beam and non-Beam.

PiperOrigin-RevId: 682225015

Author: tomvdw