Improve the dataset patching

[SarahAlidoost]

Currently there is a UserWarning in dataset about compatibility of patch size and image dimensions, as a results some part of data is uncovered. We want the dataset fixes this for users instead of warning.

[rogerkuou]

Hi @SarahAlidoost, I did a bit further investigation. This userwarning is cause by the fact that we are calling chunk reading to netcdf files here and here. Since the original nc files are not chunked. This chunking probably causes multiple file reading opt in chunks, hence the warning.

If I remove the chunking args but chunk afterwards, the warnings are gone. However on HPC this causes memeory issue.

based on this, I think it still worth to use Zarr since it's natively chunked files. I am running another tiny experiments to see if switching to Zarr helps with improving CPU efficiency as in #44

[SarahAlidoost]

@rogerkuou this issue is about this warning:

UserWarning: Patch size (120, 120) does not evenly divide image dimensions (H=720, W=640). Uncovered pixels: 0 in height, 40 in width. Consider adjusting patch_size or image dimensions for full coverage.
  warnings.warn(

I'll make another issue for zarr and move your comment there.

[rogerkuou]

@SarahAlidoost This warning is closely linked to #41 .

The direct cause of this is a wrong lat/lon subsetting, which is fixed by this commit. If issue #41 is fixed, no subsetting will be needed anymore, and the 120 patchsize will be deviable by the original size (720, 1440) again.

With a correct subset experiment this warning is gone, as you can see in this SLURM log, which log an experiment with a subset of (240, 240), and a patch size of 120.