Fix _get_variable_metadata silently dropping variables#1184
Conversation
| units=ds[name].units, | ||
| long_name=ds[name].long_name, | ||
| ) | ||
| else: |
There was a problem hiding this comment.
This code will still silently drop units or long name if the other one is missing, in other words it is all or nothing.
Previously we could disambiguate between units=“” (dimensionless) and missing units, now we can’t. Should we use None in some way to keep this ability, like making this a dict[str, VariableMetadata | None] or by making long name and units optional?
There was a problem hiding this comment.
Good point, but I think the and meant it was silently dropped if either units or long name were missing? In any case, good idea to keep what is available, will update.
There was a problem hiding this comment.
Yes, these are two separate issues - the first one is pre-existing but should probably get fixed given the scope of this PR. Let's say it's Optional.
The second issue is new in this PR, and is particularly worse if you were to fix the first issue (since long_name="" is kind of a sentinel to say the data is missing, while units="" is a valid value).
XarrayDataset._get_variable_metadatasilently omits variables that lackunitsorlong_nameattributes, causingvariable_metadatato under-report the loaded variable set.Changes:
fme.core.dataset.xarray.XarrayDataset._get_variable_metadata: fall back to empty-stringVariableMetadatafor variables withoutunits/long_nameattrsTests added
If dependencies changed, "deps only" image rebuilt and "latest_deps_only_image.txt" file updated
Resolves #1183