Direct-access attributes (#134)

* Initial changes and fixes to provide simple-access attributes.

* Change naming scheme .attributes/_attributes -> .attrvals/attributes.

* Adjust AttrvalsDict printout.

* Various docs updates, adopting 'attrvals'.

* Report beyond 1st doctest error.

* Add 'html-keeplog' makefile target.

rename target.

* Use 'doctest' in place of 'code-block'.

* Various docs fixes + improvements.

* Rename attrvals as avals.

* Tests for AttrvalsDict.

* Provide an AttrvalsDict.copy().

* Improve docs for '.avals'.

* Added whatsnew.

* Fix ncdump usage, broken in merge resolution.

Author: pp-mo

.github/workflows/ci-tests.yml

@@ -84,10 +84,10 @@ jobs:
         if: matrix.session == 'doctests-docs'
         run: |
           cd docs
-          pytest --doctest-glob="*.rst"
+          pytest --doctest-glob="*.rst" --doctest-continue-on-failure
 
       - name: "Run doctests: API"
         if: matrix.session == 'doctests-api'
         run: |
           cd lib
-          pytest --doctest-modules
+          pytest --doctest-modules --doctest-continue-on-failure

docs/Makefile

@@ -21,6 +21,12 @@ allapi:
 towncrier:
 	towncrier build --keep
 
+
+# Tweaked "make html", which restores the changelog state after docs build.
+html-keeplog: html
+	git checkout HEAD change_log.rst changelog_fragments/
+
+
 # Catch-all target: route all unknown targets to Sphinx using the new
 # "make mode" option.  $(O) is meant as a shortcut for $(SPHINXOPTS).
 %: Makefile allapi towncrier

docs/changelog_fragments/117.feat.rst

@@ -0,0 +1,3 @@
+Added the ".avals" property as an easier way of managing attributes:
+This provides a simple "name: value" map, bypassing the NcAttribute objects and converting values to and from simple Python equivalents.
+This effectively replaces the older 'set_attrval' and 'get_attrval', which will eventually be removed.

docs/details/character_handling.rst

@@ -17,6 +17,8 @@ slash "/", since in some technical cases a component name needs to specified as
 "path-like" compound.
 
 
+.. _character-attributes:
+
 Characters in Attribute Values
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 Character data in string *attribute* values can likewise be read and written simply as

docs/userdocs/getting_started/installation.rst

@@ -17,7 +17,7 @@ Like this:
 
 .. code-block:: bash
 
-    pip install ncdata
+    $ pip install ncdata
 
 
 Check install

docs/userdocs/getting_started/introduction.rst

@@ -33,7 +33,7 @@ a dataset or group.  It contains :attr:`~ncdata.NcData.dimensions`,
 :attr:`~ncdata.NcData.variables`, :attr:`~ncdata.NcData.groups`,
 and :attr:`~ncdata.NcData.attributes`:
 
-.. code-block:: python
+.. doctest:: python
 
     >>> from ncdata import NcData, NcDimension, NcVariable
     >>> data = NcData("myname")
@@ -75,13 +75,12 @@ NetCDF files via the `netcdf4-python package <http://unidata.github.io/netcdf4-p
 
 Simple example:
 
-.. code-block:: python
+.. doctest:: python
 
     >>> from ncdata.netcdf4 import to_nc4, from_nc4
     >>> filepath = "./tmp.nc"
     >>> to_nc4(data, filepath)
-
-    >>> ncdump("tmp.nc")  # utility calling 'ncdump -h' (not shown)
+    >>> ncdump("tmp.nc")  # utility which calls 'ncdump' command (not shown)
     netcdf tmp {
     dimensions:
        x = 3 ;
@@ -107,7 +106,7 @@ Variables
 Variables live in a :attr:`ncdata.NcData.variables` attribute,
 which behaves like a dictionary:
 
-.. code-block:: python
+.. doctest:: python
 
     >>> data.variables
     {'vx': <ncdata._core.NcVariable object at ...>}
@@ -131,80 +130,54 @@ which behaves like a dictionary:
 
 Attributes
 ^^^^^^^^^^
-Attributes live in the ``attributes`` property of a :class:`~ncdata.NcData`
-or :class:`~ncdata.NcVariable`:
+Attributes are held in the ``.attributes`` property of a :class:`~ncdata.NcData`
+or :class:`~ncdata.NcVariable`.  However, they are more easily accessed via the ``.avals``
+property, which provides a simple name:value mapping :
 
-.. code-block:: python
+.. doctest:: python
 
     >>> var = data.variables["vx"]
-    >>> var.set_attrval('a', 1)
-    NcAttribute('a', 1)
-    >>> var.set_attrval('b', 'this')
-    NcAttribute('b', 'this')
+    >>> var.avals['a'] = 1
+    >>> var.avals['b'] = 'this'
 
     >>> print(var)
     <NcVariable(float64): vx(x)
         vx:a = 1
         vx:b = 'this'
     >
 
-    >>> print(var.attributes)
-    {'a': NcAttribute('a', 1), 'b': NcAttribute('b', 'this')}
-
-    >>> print(data)
-    <NcData: myname
-        dimensions:
-            x = 3
-    <BLANKLINE>
-        variables:
-            <NcVariable(float64): vx(x)
-                vx:a = 1
-                vx:b = 'this'
-            >
+    >>> var.avals['b'] = 7777
+    >>> print(var)
+    <NcVariable(float64): vx(x)
+        vx:a = 1
+        vx:b = 7777
     >
 
-For technical reasons, each attribute is represented as an independent python
-:class:`ncdata.NcAttribute` object, i.e. they are *not* simply stored as a
-values in a name/value map.
-
-Attribute values are actually :mod:`numpy.ndarray`, and hence have a ``dtype``.
-To make this easier, you can use regular python numbers and strings with
-:meth:`ncdata.NcAttribute.as_python_value` and the
-:meth:`~ncdata.NcVariable.set_attrval`
-and :meth:`~ncdata.NcVariable.get_attrval` of NcData/NcVariable.
+Attribute values are actually stored as :mod:`numpy.ndarray` arrays, and hence have a
+definite ``dtype``.  However, ``.avals`` allows you to treat them mostly as ordinary
+python values (numbers and strings).
 
 
 Deletion and Renaming
 ^^^^^^^^^^^^^^^^^^^^^
-Use python 'del' operation to remove:
+Use python 'del' operation to remove items:
 
-.. code-block:: python
+.. doctest:: python
 
     >>> del var.attributes['a']
     >>> print(var)
     <NcVariable(float64): vx(x)
-        vx:b = 'this'
+        vx:b = 7777
     >
 
 There is also a 'rename' method of variables/attributes/groups:
 
-.. code-block:: python
+.. doctest:: python
 
     >>> var.attributes.rename("b", "qq")
     >>> print(var)
     <NcVariable(float64): vx(x)
-        vx:qq = 'this'
-    >
-
-    >>> print(data)
-    <NcData: myname
-        dimensions:
-            x = 3
-    <BLANKLINE>
-        variables:
-            <NcVariable(float64): vx(x)
-                vx:qq = 'this'
-            >
+        vx:qq = 7777
     >
 
 .. warning::
@@ -246,18 +219,18 @@ at :ref:`interface_support`.
 
 Example code snippets :
 
-.. code-block:: python
+.. doctest:: python
 
     >>> # (make sure that Iris and Ncdata won't conflict over netcdf access)
     >>> from ncdata.threadlock_sharing import enable_lockshare
     >>> enable_lockshare(iris=True, xarray=True)
 
-.. code-block:: python
+.. doctest:: python
 
     >>> from ncdata.netcdf4 import from_nc4
     >>> data = from_nc4("tmp.nc")
 
-.. code-block:: python
+.. doctest:: python
 
     >>> from ncdata.iris import to_iris, from_iris
     >>> from iris import FUTURE
@@ -281,7 +254,7 @@ Example code snippets :
     >>> print(vv)
     v_mag / (m.s-1)                     (-- : 3)
 
-.. code-block:: python
+.. doctest:: python
 
     >>> from ncdata.xarray import to_xarray
     >>> xrds = to_xarray(from_iris([vx, vy, vv]))
@@ -296,7 +269,7 @@ Example code snippets :
     Attributes:
         Conventions:  CF-1.7
 
-.. code-block:: python
+.. doctest:: python
 
     >>> from ncdata.iris_xarray import cubes_from_xarray
     >>> readback = cubes_from_xarray(xrds)
@@ -318,7 +291,7 @@ Thread safety
     prevent possible errors when computing or saving lazy data.
     For example:
 
-    .. code-block:: python
+    .. doctest:: python
 
         >>> from ncdata.threadlock_sharing import enable_lockshare
         >>> enable_lockshare(iris=True, xarray=True)

docs/userdocs/user_guide/common_operations.rst

@@ -8,15 +8,31 @@ i.e. the operations of extract/remove/insert/rename/copy on the ``.dimensions``,
 
 Most of these are hopefully "obvious" Pythonic methods of the container objects.
 
+.. Note::
+
+    The special ``.avals`` property of :class:`NcData` and :class:`NcVariable` also
+    provides key common operations associated with ``.attributes``, notably ``rename`` and
+    the ``del`` operator.  But not those needing NcAttribute objects -- so ``add`` and
+    ``addall` are not available.
+
+
 Extract and Remove
 ------------------
 These are implemented as :meth:`~ncdata.NameMap.__delitem__` and :meth:`~ncdata.NameMap.pop`
 methods, which work in the usual way.
 
-Examples :
+For Example:
+
+.. testsetup::
+
+    >>> from ncdata import NcData, NcVariable
+    >>> dataset = NcData(variables=[NcVariable('x'), NcVariable('y')])
+    >>> data = dataset
+
+.. doctest:: python
 
-* ``var_x = dataset.variables.pop("x")``
-* ``del data.variables["x"]``
+    >>> var_x = dataset.variables.pop("x")
+    >>> del data.variables["y"]
 
 Insert / Add
 ------------
@@ -25,18 +41,31 @@ A new content (component) can be added under its own name with the
 
 Example : ``dataset.variables.add(NcVariable("x", dimensions=["x"], data=my_data))``
 
-An :meth:`~ncdata.NcAttribute` can also be added or set (if already present) with the special
-:meth:`~ncdata.NameMap.set_attrval` method.
+:meth:`~ncdata.NcAttribute`s can be treated in the same way, as a :class:`ncdata.NameMap`
+component of the parent object.  But it is more usual to add or set attributes
+using ``.avals`` rather than ``.attributes``.
 
-Example : ``dataset.variables["x"].set_attrval("units", "m s-1")``
+Example :
+
+.. testsetup::
+
+    >>> dataset = NcData(variables=[NcVariable("x")])
+
+.. doctest:: python
+
+    >>> dataset.variables["x"].avals["units"] = "m s-1"
 
 Rename
 ------
 A component can be renamed with the :meth:`~ncdata.NameMap.rename` method.  This changes
 both the name in the container **and** the component's own name -- it is not recommended
 ever to set ``component.name`` directly, as this obviously can become inconsistent.
 
-Example : ``dataset.variables.rename("x", "y")``
+Example :
+
+.. doctest:: python
+
+    >>> dataset.variables.rename("x", "y")
 
 .. warning::
     Renaming a dimension will not rename references to it (i.e. in variables), which
@@ -51,7 +80,7 @@ All core objects support a ``.copy()`` method.  See for instance
 These however do *not* copy variable data arrays (either real or lazy), but produce new
 (copied) variables referencing the same arrays.  So, for example:
 
-.. code-block::
+.. doctest:: python
 
     >>> # Construct a simple test dataset
     >>> import numpy as np
@@ -114,7 +143,7 @@ dataset which must "make sense".   see : :ref:`correctness-checks` .
 Hence, there is no great need to install things in the 'right' order (e.g. dimensions
 before variables which need them).  You can create objects in one go, like this :
 
-.. code-block::
+.. doctest:: python
 
     data = NcData(
         dimensions=[
@@ -131,7 +160,7 @@ before variables which need them).  You can create objects in one go, like this
 
 or iteratively, like this :
 
-.. code-block::
+.. doctest:: python
 
     data = NcData()
     dims = [("y", 2), ("x", 3)]