@@ -131,6 +131,87 @@ Iris' optimisation all together, and will take its chunksizes from Dask's behavi
131131 (70, 37, 49)
132132
133133
134+ Split Attributes
135+ -----------------
136+
137+ Since Iris does not provide a "dataset" object, it is not possible to represent netcdf
138+ file-level, aka "global", attributes directly. Instead, the attributes of *cubes *,
139+ specifically, are "split" to discriminate between global (i.e. file) and local (i.e.
140+ variable) attributes.
141+
142+ Prior to Iris v3.8, this was done "automatically" according to a specific lists of
143+ known attribute names which would normally *only * appear as local (variable) attributes
144+ (e.g. "valid_min") or *only * as global (file) ones (e.g. "history").
145+ All other names were saved as "global if the same for all cubes, otherwise local".
146+
147+ Since Iris v3.8, but explicit handling has now been added, so that a ``cube.attributes ``
148+ is a :class: `~iris.cube.CubeAttrsDict ` object which combines global and local attributes
149+ within a single dictionary-like object.
150+ Finer-grained control of attribute saving is thus now possible, by setting the
151+ :data: `iris.Future.save_split_attrs ` control to ``True ``.
152+ Ideally, anyone who is saving cubes to netcdf files should now set this, and
153+ a warning is raised if it is not set.
154+
155+ .. note ::
156+ The default setting remains ``iris.FUTURE.save_split_attrs = False ``, purely for
157+ backwards compatibility, but it is recommended to override this for all new code.
158+
159+ The :class: `~iris.cube.CubeAttrsDict ` provides a regular dictionary access which mimics
160+ the older single dictionary, while also containing separate lists in its ``.global `` and
161+ ``.local `` properties which allow the user to specifically choose the nature of each
162+ attribute.
163+ On saving, each cube's ".attributes.local"s become variable-level attributes and its
164+ ".attributes.global"s become file-level attributes -- *except only *, if globals
165+ have different values in multiple cubes, then global attributes may be 'demoted' to
166+ local ones.
167+
168+ Summary
169+ =======
170+
171+ * You should now always set the ``iris.Future.save_split_attrs=True `` unless you have a
172+ specific need for backwards compatibility.
173+ * all Cube attributes are now stored as specifically "global" or "local"
174+ * you can continue to assign and fetch cube attributes as before, but when required you
175+ can also now access ``.global `` and ``.local `` properties specifically.
176+ * practically, all this is only relevant when **saving to netcdf files **
177+
178+
179+ Deferred Saving
180+ ----------------
181+
182+ In some cases it is useful for performance reasons to defer the actual writing of bulk
183+ data to a NetCDF file's variables, from after the "iris.save" call to a later time.
184+
185+ You can do this by adding a ``compute=False `` keyword to the ``iris.save `` call. The
186+ call then returns, instead of None, a :class: ``dask.delayed` ` object, which can be
187+ computed later to complete the writing of variable data to the file.
188+
189+ Where this can be useful is when you are saving multiple cubes whose lazy data is
190+ computed from shared data, for example multiple statistics over the same data.
191+
192+ To do this, you specify ``compute=False `` to multiple :func: ``iris.save` ` calls, and
193+ collect the resulting :class: ``dask.delayed` ` objects. You can then complete the saves
194+ in parallel, by supplying a list of the 'delayed' results to the :func: `Dask.compute `
195+ function.
196+
197+ This mechanism allows Dask to calculate the multiple "derived" results while only
198+ fetching the "common" data from which they derive **once **, whereas otherwise the source
199+ data may need to be fetched multiple times. In this respect, the benefit is the same as
200+ the :meth: ``iris.cube.CubeList.realise_cubes` ` method. However there is an additional
201+ potential benefit in this case, which is that the actual **writing ** of data to file can
202+ be performed in parallel. The scope of this, however, depends on the parallel
203+ capabilities of the Python ``netCDF4 `` package, the NetCDF C library, the file
204+ system and the operating system : *Typically * you can write data arrays in parallel
205+ to **separate files **, but not to different variables, or variable sections *within a
206+ single file *.
207+
208+
209+ Character and String data
210+ -------------------------
211+
212+ TBC
213+
214+
134215Variable-length datatypes
135216-------------------------
136217
@@ -187,17 +268,6 @@ loader so it can be make a more informed decision on lazy loading:
187268 False
188269
189270
190- Split Attributes
191- -----------------
192-
193- TBC
194-
195-
196- Deferred Saving
197- ----------------
198-
199- TBC
200-
201271.. _save_load_dataless :
202272
203273Dataless Cubes in NetCDF files
0 commit comments