Merge pull request #36 from jrleeman/siphon_cleanup

Siphon cleanup

Author: dopplershift

.gitignore

@@ -13,3 +13,4 @@ id_rsa
 id_rsa.pub
 
 *.mp4
+*.ar2v

pages/workshop/Siphon/Siphon Overview.ipynb

@@ -21,58 +21,55 @@
     "\n",
     "<div style=\"float:right; width:250 px\"><img src=\"https://unidata.github.io/siphon/latest/_static/siphon_150x150.png\" alt=\"TDS\" style=\"height: 200px;\"></div>\n",
     "\n",
-    "## Overview:\n",
-    "\n",
-    "* **Teaching:** 10 minutes\n",
-    "* **Exercises:** 10 minutes\n",
-    "\n",
     "### Questions\n",
     "1. What is a THREDDS Data Server (TDS)?\n",
     "1. How can I use Siphon to access a TDS?\n",
     "\n",
     "### Objectives\n",
-    "1. <a href=\"#threddsintro\">Use siphon to access a THREDDS catalog</a>\n",
-    "1. <a href=\"#filtering\">Find data within the catalog that we wish to access</a>\n",
-    "1. <a href=\"#dataaccess\">Use siphon to perform remote data access</a>"
+    "1. <a href=\"#threddsintro\">Use Siphon to Access a THREDDS Catalog</a>\n",
+    "1. <a href=\"#filtering\">Filtering Data</a>\n",
+    "1. <a href=\"#dataaccess\">Use Siphon to Perform Remote Data Access</a>"
    ]
   },
   {
    "cell_type": "markdown",
    "metadata": {},
    "source": [
     "<a name=\"threddsintro\"></a>\n",
-    "## 1. What is THREDDS?\n",
+    "## 1. Use Siphon to Access a THREDDS Catalog\n",
     "\n",
-    " * Server for providing remote access to datasets\n",
-    " * Variety of services for accesing data:\n",
+    "THREDDS is a server for providing remote access to datasets and a variety of server-side services. THREDDS make data access more uniform regardless of the on-disk format.\n",
+    " * Data Access Services:\n",
     "   - HTTP Download\n",
     "   - Web Mapping/Coverage Service (WMS/WCS)\n",
     "   - OPeNDAP\n",
     "   - NetCDF Subset Service\n",
     "   - CDMRemote\n",
-    " * Provides a more uniform way to access different types/formats of data"
+    "   \n",
+    "There is a server with real-time data setup at [http://thredds.ucar.edu](http://thredds.ucar.edu) that we'll use to explore the capability of THREDDS and learn how to access data. Let's open that link and explore in the browser what's available on THREDDS. Explore the NEXRAD level 3 data specifically.\n",
+    "\n",
+    "### THREDDS Catalogs\n",
+    "- XML descriptions of data and metadata\n",
+    "- Access methods\n",
+    "- Easily processed with `siphon.catalog.TDSCatalog`"
    ]
   },
   {
-   "cell_type": "markdown",
-   "metadata": {
-    "slideshow": {
-     "slide_type": "subslide"
-    }
-   },
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
    "source": [
-    "## THREDDS Demo\n",
-    "http://thredds.ucar.edu"
+    "from datetime import datetime, timedelta\n",
+    "\n",
+    "from siphon.catalog import TDSCatalog"
    ]
   },
   {
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "### THREDDS Catalogs\n",
-    "- XML descriptions of data and metadata\n",
-    "- Access methods\n",
-    "- Easily handled with `siphon.catalog.TDSCatalog`"
+    "Let's get data from yesterday at this time. We'll use the timedelta object to do this in an easy way."
    ]
   },
   {
@@ -81,13 +78,36 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "from datetime import datetime, timedelta\n",
-    "from siphon.catalog import TDSCatalog\n",
     "date = datetime.utcnow() - timedelta(days=1)\n",
+    "print(date)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "We'll then go find the URL for the level 3 radar data. Let's get the N0Q (tilt 1 base reflectivity) for the LRX radar. Notice that we change the `html` extension to `xml`. Siphon will do that for us, but issue a warning."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
     "cat = TDSCatalog('http://thredds.ucar.edu/thredds/catalog/nexrad/level3/'\n",
     "                 f'N0Q/LRX/{date:%Y%m%d}/catalog.xml')"
    ]
   },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "cat.datasets"
+   ]
+  },
   {
    "cell_type": "markdown",
    "metadata": {},
@@ -101,14 +121,14 @@
    "metadata": {},
    "source": [
     "<a name=\"filtering\"></a>\n",
-    "## 2. Filtering data"
+    "## 2. Filtering Data"
    ]
   },
   {
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "We *could* manually figure out what dataset we're looking for and generate that name (or index). Siphon provides some helpers to simplify this process, provided the names of the dataset follow a pattern with the timestamp in the name:"
+    "We *could* manually look through that list above and figure out what dataset we're looking for and generate that name (or index). Siphon provides some helpers to simplify this process, provided the names of the dataset follow a pattern with the timestamp in the name:"
    ]
   },
   {
@@ -143,10 +163,14 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "### Exercise\n",
-    "* Starting from http://thredds.ucar.edu/thredds/catalog/satellite/SFC-T/SUPER-NATIONAL_1km/catalog.html, find the composites for the previous day.\n",
-    "* Grab the URL and create a TDSCatalog instance.\n",
-    "* Using Siphon, find the data available in the catalog between 12Z and 18Z on the previous day."
+    "<div class=\"alert alert-success\">\n",
+    "    <b>EXERCISE</b>:\n",
+    "     <ul>\n",
+    "        <li>Starting from http://thredds.ucar.edu/, find the level 2 radar data for the Tulsa, OK radar (KINX) for the previous day.</li>\n",
+    "        <li>Grab the URL and create a TDSCatalog instance.</li>\n",
+    "        <li>Using Siphon, find the data available in the catalog between 12Z and 18Z on the previous day.</li>\n",
+    "    </ul>\n",
+    "</div>"
    ]
   },
   {
@@ -155,14 +179,16 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "# YOUR CODE GOES HERE\n"
+    "# YOUR CODE GOES HERE"
    ]
   },
   {
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "#### Solution"
+    "<div class=\"alert alert-info\">\n",
+    "    <b>SOLUTION</b>\n",
+    "</div>"
    ]
   },
   {
@@ -171,7 +197,7 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "# %load solutions/datasets.py\n"
+    "# %load solutions/datasets.py"
    ]
   },
   {
@@ -187,20 +213,24 @@
    "metadata": {},
    "source": [
     "<a name=\"dataaccess\"></a>\n",
-    "## 3. Accessing data"
+    "## 3. Use Siphon to Perform Remote Data Access\n",
+    "\n",
+    "Accessing catalogs is only part of the story; Siphon is much more useful if you're trying to access/download datasets.\n",
+    "\n",
+    "For instance, using our data that we just retrieved:"
    ]
   },
   {
-   "cell_type": "markdown",
-   "metadata": {
-    "slideshow": {
-     "slide_type": "subslide"
-    }
-   },
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
    "source": [
-    "Accessing catalogs is only part of the story; Siphon is much more useful if you're trying to access/download datasets.\n",
-    "\n",
-    "For instance, using our data that we just retrieved:"
+    "# Solution from above in case you had trouble\n",
+    "date = datetime.utcnow() - timedelta(days=1)\n",
+    "cat = TDSCatalog(f'https://thredds.ucar.edu/thredds/catalog/nexrad/level2/KINX/{date:%Y%m%d}/catalog.xml')\n",
+    "request_time = date.replace(hour=12, minute=0, second=0, microsecond=0)\n",
+    "datasets = cat.datasets.filter_time_range(request_time, request_time + timedelta(hours=6))"
    ]
   },
   {
@@ -232,6 +262,13 @@
     "ds.download()"
    ]
   },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Look in your file explorer panel or run the cell below to verify that we did actually download the file!"
+   ]
+  },
   {
    "cell_type": "code",
    "execution_count": null,
@@ -294,7 +331,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "By default this uses CDMRemote (if available), but it's also possible to ask for OPeNDAP (using netCDF4-python)."
+    "By default this uses CDMRemote (if available), but it's also possible to ask for OPeNDAP (using netCDF4-python). There is even XArray support which is great with the declarative plotting interface - more on that later."
    ]
   },
   {
@@ -331,9 +368,9 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.7.3"
+   "version": "3.7.6"
   }
  },
  "nbformat": 4,
- "nbformat_minor": 2
+ "nbformat_minor": 4
 }

pages/workshop/Siphon/solutions/datasets.py

@@ -1,5 +1,6 @@
+# Solution from above in case you had trouble
 date = datetime.utcnow() - timedelta(days=1)
-cat = TDSCatalog(f'http://thredds.ucar.edu/thredds/catalog/satellite/SFC-T/SUPER-NATIONAL_1km/{date:%Y%m%d}/catalog.xml')
+cat = TDSCatalog(f'https://thredds.ucar.edu/thredds/catalog/nexrad/level2/KINX/{date:%Y%m%d}/catalog.xml')
 request_time = date.replace(hour=12, minute=0, second=0, microsecond=0)
 datasets = cat.datasets.filter_time_range(request_time, request_time + timedelta(hours=6))
-print(datasets)
+print(datasets)