@@ -133,6 +133,87 @@ Iris' optimisation all together, and will take its chunksizes from Dask's behavi
133133 (70, 37, 49)
134134
135135
136+ Split Attributes
137+ -----------------
138+
139+ Since Iris does not provide a "dataset" object, it is not possible to represent netcdf
140+ file-level, aka "global", attributes directly. Instead, the attributes of *cubes *,
141+ specifically, are "split" to discriminate between global (i.e. file) and local (i.e.
142+ variable) attributes.
143+
144+ Prior to Iris v3.8, this was done "automatically" according to a specific lists of
145+ known attribute names which would normally *only * appear as local (variable) attributes
146+ (e.g. "valid_min") or *only * as global (file) ones (e.g. "history").
147+ All other names were saved as "global if the same for all cubes, otherwise local".
148+
149+ Since Iris v3.8, but explicit handling has now been added, so that a ``cube.attributes ``
150+ is a :class: `~iris.cube.CubeAttrsDict ` object which combines global and local attributes
151+ within a single dictionary-like object.
152+ Finer-grained control of attribute saving is thus now possible, by setting the
153+ :data: `iris.Future.save_split_attrs ` control to ``True ``.
154+ Ideally, anyone who is saving cubes to netcdf files should now set this, and
155+ a warning is raised if it is not set.
156+
157+ .. note ::
158+ The default setting remains ``iris.FUTURE.save_split_attrs = False ``, purely for
159+ backwards compatibility, but it is recommended to override this for all new code.
160+
161+ The :class: `~iris.cube.CubeAttrsDict ` provides a regular dictionary access which mimics
162+ the older single dictionary, while also containing separate lists in its ``.global `` and
163+ ``.local `` properties which allow the user to specifically choose the nature of each
164+ attribute.
165+ On saving, each cube's ".attributes.local"s become variable-level attributes and its
166+ ".attributes.global"s become file-level attributes -- *except only *, if globals
167+ have different values in multiple cubes, then global attributes may be 'demoted' to
168+ local ones.
169+
170+ Summary
171+ =======
172+
173+ * You should now always set the ``iris.Future.save_split_attrs=True `` unless you have a
174+ specific need for backwards compatibility.
175+ * all Cube attributes are now stored as specifically "global" or "local"
176+ * you can continue to assign and fetch cube attributes as before, but when required you
177+ can also now access ``.global `` and ``.local `` properties specifically.
178+ * practically, all this is only relevant when **saving to netcdf files **
179+
180+
181+ Deferred Saving
182+ ----------------
183+
184+ In some cases it is useful for performance reasons to defer the actual writing of bulk
185+ data to a NetCDF file's variables, from after the "iris.save" call to a later time.
186+
187+ You can do this by adding a ``compute=False `` keyword to the ``iris.save `` call. The
188+ call then returns, instead of None, a :class: ``dask.delayed` ` object, which can be
189+ computed later to complete the writing of variable data to the file.
190+
191+ Where this can be useful is when you are saving multiple cubes whose lazy data is
192+ computed from shared data, for example multiple statistics over the same data.
193+
194+ To do this, you specify ``compute=False `` to multiple :func: ``iris.save` ` calls, and
195+ collect the resulting :class: ``dask.delayed` ` objects. You can then complete the saves
196+ in parallel, by supplying a list of the 'delayed' results to the :func: `Dask.compute `
197+ function.
198+
199+ This mechanism allows Dask to calculate the multiple "derived" results while only
200+ fetching the "common" data from which they derive **once **, whereas otherwise the source
201+ data may need to be fetched multiple times. In this respect, the benefit is the same as
202+ the :meth: ``iris.cube.CubeList.realise_cubes` ` method. However there is an additional
203+ potential benefit in this case, which is that the actual **writing ** of data to file can
204+ be performed in parallel. The scope of this, however, depends on the parallel
205+ capabilities of the Python ``netCDF4 `` package, the NetCDF C library, the file
206+ system and the operating system : *Typically * you can write data arrays in parallel
207+ to **separate files **, but not to different variables, or variable sections *within a
208+ single file *.
209+
210+
211+ Character and String data
212+ -------------------------
213+
214+ TBC
215+
216+
136217Variable-length datatypes
137218-------------------------
138219
@@ -189,17 +270,6 @@ loader so it can be make a more informed decision on lazy loading:
189270 False
190271
191272
192- Split Attributes
193- -----------------
194-
195- TBC
196-
197-
198- Deferred Saving
199- ----------------
200-
201- TBC
202-
203273.. _save_load_dataless :
204274
205275Dataless Cubes in NetCDF files
0 commit comments