Updating missing data

Author: kasra-keshavarz

docs/scripts/ouranos-mrcc5-cmip6.rst

@@ -1,6 +1,8 @@
 Ouranos ``MRCC5-CMIP6`` dataset
 ===============================
 
+**Important:** Some driving datasets / model members have documented missing periods and/or missing variables; see the ``Known missing data (CRCM5-CMIP6 ensemble)`` section at the end for details.
+
 In this file, the details of the dataset is explained.
 
 Location of Dataset Files
@@ -31,7 +33,7 @@ and the structure of the dataset hourly files is as following:
    │   |       └── CRCM5
    |   |           └── v1-r1
    |   |               └── 1hr
-   |   |                   ├── hurs 
+   |   |                   ├── hurs
    |   |                   |   └── %version_number
    │   |                   |       ├── hurs_NAM-12_MPI-ESM1-2-LR_ssp126_r1i1p1f1_OURANOS_CRCM5_v1-r1_1hr_201501010030-201512312330.nc
    |   |                   |       ├── hurs_NAM-12_MPI-ESM1-2-LR_ssp126_r1i1p1f1_OURANOS_CRCM5_v1-r1_1hr_201601010030-201612312330.nc
@@ -42,11 +44,11 @@ and the structure of the dataset hourly files is as following:
    |   |                   |       └── hurs_NAM-12_MPI-ESM1-2-LR_ssp126_r1i1p1f1_OURANOS_CRCM5_v1-r1_1hr_210001010030-210012312330.nc
    │   |                   ├── %variable
    |   |                   |   └── %version_number
-   |   |                   |       ├── %variable_NAM-12_MPI-ESM1-2-LR_ssp126_r1i1p1f1_OURANOS_CRCM5_v1-r1_1hr_201501010030-201512312330.nc
-   |   |                   |       ├── . 
-   |   |                   |       ├── . 
-   |   |                   |       ├── . 
-   |   |                   |       └── %variable_NAM-12_MPI-ESM1-2-LR_ssp126_r1i1p1f1_OURANOS_CRCM5_v1-r1_1hr_210001010030-210012312330.nc
+   |   |                   |       ├── %variable_NAM-12_%model_%scenario_%ensemble_OURANOS_CRCM5_v1-r1_1hr_201501010030-201512312330.nc
+   |   |                   |       ├── .
+   |   |                   |       ├── .
+   |   |                   |       ├── .
+   |   |                   |       └── %variable_NAM-12_%model_%scenario_%ensemble_OURANOS_CRCM5_v1-r1_1hr_210001010030-210012312330.nc
    |   |                   └── vas
    |   |                       └── %version_number
    |   |                           ├── vas_NAM-12_MPI-ESM1-2-LR_ssp126_r1i1p1f1_OURANOS_CRCM5_v1-r1_1hr_201501010000-201512312300.nc
@@ -91,8 +93,12 @@ Coordinate Variables
 ~~~~~~~~~~~~~~~~~~~~
 
 The coordinate variables for this dataset are ``rlon`` and ``rlat``
-representing the longitude and latitude points, respectively. ###
-Time-stamps The time-stamps are included in the original files.
+representing the longitude and latitude points, respectively.
+
+Time-stamps
+~~~~~~~~~~~
+
+The time-stamps are included in the original files.
 
 ``MRCC5-CMIP6`` Climate Models
 ------------------------------
@@ -140,7 +146,7 @@ list of variables by browsing the dataset’s directory:
 
 .. code:: console
 
-   foo@bar:~$ ls /path/to/dataset/dir/CanESM5/ssp126/r1i1p2f1/CRCM5/v1-r1/1hr/
+   foo@bar:~$ ls /path/to/dataset/dir/CanESM5/ssp126/r1i1p1f1/CRCM5/v1-r1/1hr/
    hurs  huss  pr  ps  rlds  rsds  tas  uas  vas
 
 Spatial Extent
@@ -155,7 +161,7 @@ Temporal Extent
 
 The time-stamps are already included in the original files. The dataset
 offers **hourly** time-series of climate variables. The following table
-describes the temporal extent for senarios included in this dataset:
+describes the temporal extent for scenarios included in this dataset:
 
 .. list-table::
    :header-rows: 1
@@ -167,19 +173,18 @@ describes the temporal extent for senarios included in this dataset:
      - ``historical``
      - ``1950-01-01`` to ``2014-12-31``
    * - 2
-     - ``ssp245``
+     - ``ssp126``
      - ``2015-01-01`` to ``2100-12-31``
    * - 3
-     - ``ssp370``
+     - ``ssp245``
      - ``2015-01-01`` to ``2100-12-31``
    * - 4
-     - ``ssp585``
+     - ``ssp370``
      - ``2015-01-01`` to ``2100-12-31``
    * - 5
      - ``ssp585``
      - ``2015-01-01`` to ``2100-12-31``
 
-
 .. note::
   Values of the ``Temporal extent`` column are the limits for
   ``--start-date`` and ``--end-date`` options with the main
@@ -191,10 +196,9 @@ describes the temporal extent for senarios included in this dataset:
 
 .. note::
    ``MPI-ESM1-2-LR`` and ``CNRM-ESM2-1`` report ``pr``, ``rlds``,
-   ``rsds`` at the middle of the hour (YYYY-mm-ddTHH:30:00), while
+   ``rsds`` at the middle of the hour ``(YYYY-mm-ddTHH:30:00)``, while
    all other variables are reported at the top of the hour.
 
-
 Calendar
 --------
 +--------------------------+-------------------+
@@ -211,61 +215,182 @@ Calendar
 Short Description on ``MRCC5-CMIP6`` Variables
 ----------------------------------------------
 
-This dataset only offers seven climate variables: 1) hourly
-precipitation time-series (surface level), 2) hourly temperature
-time-series (@2m, near-surface level), 3) hourly specific humidity
-time-series (@2m, near-surface level), 4) hourly surface pressue
-time-series (surface level), 5) hourly wind speed (@10m, near-surface
-level), 6) hourly shortwave radiation (surface level), and 7) hourly
-longwave radiation (surface level).
+This dataset offers the following **hourly** climate variables:
+
+1) Relative humidity time-series (@2m, near-surface level),
+2) Specific humidity time-series (@2m, near-surface level),
+3) Precipitation time-series (surface level),
+4) Surface pressure time-series (surface level),
+5) Eastward wind time-series (@10m, near-surface level),
+6) Northward wind time-series (@10m, near-surface level),
+7) Shortwave radiation (surface level),
+8) Longwave radiation (surface level),
+9) Air temperature time-series (@2m, near-surface level).
 
-The table below, summarizes the variables offered by this dataset:
+The table below summarizes the variables offered by this dataset:
 
 .. list-table:: Variable Information
    :header-rows: 1
 
    * - Variable Name
      - Variable (keyword for ``--variable``)
      - Unit
-     - IPCC Abbreviation
      - Comments
+   * - relative humidity
+     - ``hurs``
+     - ``%``
+     - @2 meters above surface
    * - specific humidity
      - ``huss``
-     - 1
-     -
+     - ``1``
      - @2 meters above surface
    * - precipitation
      - ``pr``
      - kg m\ :sup:`-2` s\ :sup:`-1`
-     -
      - surface level
    * - surface air pressure
      - ``ps``
-     - Pa
-     -
+     - ``Pa``
      - surface level
    * - longwave radiation
      - ``rlds``
      - W m\ :sup:`-2`
-     -
      - surface level
    * - shortwave radiation
      - ``rsds``
      - W m\ :sup:`-2`
-     -
      - surface level
    * - air temperature
      - ``tas``
-     - K
-     -
+     - ``K``
      - @2 meters above surface
-   * - wind speed (eastward)
+   * - eastward wind
      - ``uas``
      - m s\ :sup:`-1`
-     -
-     - eastward wind speed @10 meters above surface
-   * - wind speed (northward)
+     - Eastward wind component @10 meters above surface
+   * - northward wind
      - ``vas``
      - m s\ :sup:`-1`
+     - Northward wind component @10 meters above surface
+
+
+Known missing data (CRCM5-CMIP6 ensemble)
+-----------------------------------------
+
+The CRCM5-CMIP6 ensemble has documented missing periods and/or missing
+variables for some driving datasets and members. If ``Missing Date End``
+is not provided, the data is missing for the entirety of the
+``Missing Date Start`` time unit (e.g., ``1979-01-01`` means the
+whole day; ``1979-01`` means the whole month). The missing-data table
+below is transcribed from the provided CRCM5-CMIP6 missing-data table PDF.
+Please refer to the dataset's Zenodo webpage for complete information at
+`https://zenodo.org/records/18624465 <https://zenodo.org/records/18624465>`_.
+
+.. list-table:: Missing periods / variables
+   :header-rows: 1
+
+   * - Driving data
+     - Period / scenario
+     - Ensemble member (if applicable)
+     - Missing date start
+     - Missing date end
+     - Variables
+     - Details
+   * - CNRM-ESM2-1
+     - historical
+     -
+     - 1950-01-01 00:00
+     - 1950-01-01 11:00
+     - All
+     - Simulation starts at 12:00, correction underway
+   * - CNRM-ESM2-1
+     - historical
+     -
+     - 1955-04-21 01:00
+     - 1955-04-30 00:00
+     - All
+     - Correction underway
+   * - CNRM-ESM2-1
+     - historical
+     -
+     - 2002-04-19 01:00
+     - 2002-04-30 00:00
+     - All
+     - Correction underway
+   * - CNRM-ESM2-1
+     - historical
+     -
+     - 2007-09
+     -
+     - hurs, huss
+     - Correction underway
+   * - CNRM-ESM2-1
+     - historical
+     -
+     - 2009-02-13 01:00
+     - 2009-02-28 00:00
+     - All
+     - Correction underway
+   * - CNRM-ESM2-1
+     - historical
+     -
+     - 2011-09
+     -
+     - rlds
+     - Correction underway
+   * - CanESM5
+     - ssp126, ssp245, ssp370
+     -
+     - 2015-01-01 00:00
+     -
+     - Instantaneous variables
+     - Missing from scenario start (see comprehensive output-variable list for which variables are instantaneous)
+   * - MPI-ESM1-2-LR
+     - ssp126, ssp245, ssp370
+     -
+     - 2015-01-01 00:00
+     -
+     - Instantaneous variables
+     - Missing from scenario start (see comprehensive output-variable list for which variables are instantaneous)
+   * - NorESM-MM
+     - historical
+     -
+     - 1950-01-01 00:00
+     - 1950-01-01 05:00
+     - All
+     - Simulation starts at 06:00
+   * - MPI-ESM1-2-LR
+     - historical
+     - r1i1p1f1
+     - 1950-01-10
+     -
+     - All
+     -
+   * - MPI-ESM1-2-LR
+     - historical
+     - r1i1p1f1
+     - 1950-01-20
+     -
+     - All
+     -
+   * - MPI-ESM1-2-LR
+     - historical
+     - r1i1p1f1
+     - 1950-01-30
+     -
+     - All
+     -
+   * - MPI-ESM1-2-LR
+     - historical
+     - r2i1p1f1
+     - 1951-01-01
+     - 1953-08-31
+     - All
+     -
+   * - MPI-ESM1-2-LR
+     - historical
+     - r3i1p1f1
+     - 1951-01-01
+     - 1953-06-30
+     - All
      -
-     - northward wind speed @10 meters above surface