Negative scaledValueOfFirstFixedSurface and SecondFixedSurface fails

[ColemanTom]

What happened?

eccodes sets scaledValueOf*FixedSurface to be unsigned[4]. See https://github.com/ecmwf/eccodes/blob/develop/definitions/grib2/templates/template.4.horizontal.def#L12 . This is generally fine, but it does not cover all use cases. Potential vorticity levels in the southern hemisphere are negative. If you try to do grib_set -s typeOfFirstFixedSurface=109,scaledValueOfFirstFixedSurface=-2 file, it will fail because the level value is negative.

Could you please allow negative values for the scaledValueOf*FixedSurface metadata items as it is quite reasonable to have negative levels.

What are the steps to reproduce the bug?

grib_set -s typeOfFirstFixedSurface=109,scaledValueOfFirstFixedSurface=-2 file

Version

All

Platform (OS and architecture)

Linux

Relevant log output

$ grib_set -s typeOfFirstFixedSurface=109,scaledValueOfFirstFixedSurface=-2 z.54.grb2 z.54.grb22
ECCODES ERROR   :  Key "scaledValueOfFirstFixedSurface": Trying to encode a negative value of -2 for key of type unsigned
ECCODES ERROR   :  grib_set_values[1] scaledValueOfFirstFixedSurface (type=long) failed: Encoding invalid (message 1)

Accompanying data

No response

Organisation

Bureau of Meteorology

[joobog]

That’s an interesting point. I couldn’t find anything in the WMO documentation stating that it should be an unsigned integer. But does it make sense to have it signed?

Usually, the sign can be inferred from the surface description. For example, “depth below sea level” implies a negative value.

But for a “potential vorticity surface,” I can’t find anything similar. Since this is widely used, I’d like to check whether there is an established convention. I think we should double-check that before either of us makes a mistake.

[sebvi]

Hi @ColemanTom,

I am trying to understand your use case.
Since PV is positive in the North Hemisphere (NH) and negative in the South Hemisphere (SH), you are essentially trying to only encode half a grid , i.e only the SH?
My understanding of using TypeOfLevel=109 is that you specify the +/- value of PV and then encode the + values for the NH and the - value for the SH. Maybe the manual on codes is unclear and we should seek clarification from WMO?

[ColemanTom]

Hi @ColemanTom,

I am trying to understand your use case.
Since PV is positive in the North Hemisphere (NH) and negative in the South Hemisphere (SH), you are essentially trying to only encode half a grid , i.e only the SH?
My understanding of using TypeOfLevel=109 is that you specify the +/- value of PV and then encode the + values for the NH and the - value for the SH. Maybe the manual on codes is unclear and we should seek clarification from WMO?

Perhaps it is a bug/gap with the wmo definitions. Potential vorticity levels can be both positive and negative. For clockwise rotation thresholds in the northern hemisphere they would be positive, but negative for the southern hemisphere. But I can not see anything to indicate it should be unsigned. Most other similar items where negative is reasonable have two fixed surface type entries, eg height above and depth below. For example, looking at temperatures on PV levels, in the SH one would look around -2PVU for the tropopause.

Either a new level type for negative pv levels, or allowing signed values is needed. I'm happy to email the WMO, but I admit I'm unsure who/how.

[sebvi]

@ColemanTom : you need to open a "generic issue" here: https://github.com/wmo-im/GRIB2/issues and the WMO ET-Data Team will have a look at the request.

[sawom666]

Also consider what regressions would happen if it was decided to change unsigned to signed.
Consider the following example:
Imagine if an existing GRIB message has scaledValueOfFirstFixedSurface set to a large value (bigger than 2^32)

% grib_set -s scaledValueOfFirstFixedSurface=4294967294 samples/GRIB2.tmpl x
% grib_get -p scaledValueOfFirstFixedSurface x
4294967294
## Now the new version of ecCodes is released.
% grib_get -p scaledValueOfFirstFixedSurface x
-2147483646
# Ouch

Any value over 2 billion (2147483648) would suffer from this

[joobog]

WMO issue: wmo-im/GRIB2#400