For something like band, we know that there are only a few possible values, and parquet knows this too and does a dictionary-style encoding on those columns. Can we consider using this hint to convert those columns to Categorical Dtype and save space in-memory?
Is there an in-memory savings to be gained?
For something like
band, we know that there are only a few possible values, and parquet knows this too and does a dictionary-style encoding on those columns. Can we consider using this hint to convert those columns to Categorical Dtype and save space in-memory?Is there an in-memory savings to be gained?