Merge pull request DOI-USGS#167 from aleaf/main

refactor(notebooks/part0_python_intro/03_useful-std-library-modules.i…

Author: mnfienen

notebooks/part0_python_intro/03_useful-std-library-modules.ipynb

@@ -7,10 +7,10 @@
     "# 03: Useful standard library modules\n",
     "(pathlib, shutil, sys, os, subprocess, zipfile, etc.)\n",
     "\n",
-    "These packages are part of the standard python library and provide very useful functionality for working with your operating system and files.  This notebook will provide explore these packages and demonstrate some of their functionality.  Online documentation is at https://docs.python.org/3/library/.\n",
+    "These packages are part of the standard python library and provide very useful functionality for working with your operating system and files.  This notebook will provide these packages and demonstrate some of their functionality.  Online documentation is at https://docs.python.org/3/library/.\n",
     "\n",
     "\n",
-    "#### Topics covered:\n",
+    "## Topics covered:\n",
     "* **pathlib**:\n",
     "    * listing files\n",
     "    * creating, moving and deleting files\n",
@@ -62,7 +62,9 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "#### Make a ``Path()`` object for the current folder"
+    "### Listing files\n",
+    "\n",
+    "#### Start by making a ``Path()`` object for the current folder"
    ]
   },
   {
@@ -75,13 +77,6 @@
     "cwd"
    ]
   },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "### Listing files"
-   ]
-  },
   {
    "cell_type": "code",
    "execution_count": null,
@@ -185,7 +180,9 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "#### Create a new path for the data subfolder"
+    "### Creating files and folders\n",
+    "\n",
+    "#### make a ``Path`` object for a new subdirectory"
    ]
   },
   {
@@ -194,8 +191,8 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "data_path = cwd / 'data'\n",
-    "data_path"
+    "new_folder = cwd / 'more_files'\n",
+    "new_folder"
    ]
   },
   {
@@ -235,28 +232,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "### Creating files and folders\n",
-    "\n",
-    "#### make a new subdirectory"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "new_folder = cwd / 'more_files'\n",
-    "new_folder"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "new_folder.exists()"
+    "#### make the actual folder"
    ]
   },
   {
@@ -793,7 +769,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "**Note**: Generally, importing code using ``sys.path`` is considered bad practice, because \n",
+    "**Note**: Generally, importing code using ``sys.path`` is often considered bad practice, because \n",
     "\n",
     "* it can hide dependencies.    \n",
     "\n",
@@ -802,7 +778,9 @@
     "\n",
     "* importing code using ``sys.path`` is also sensitive to the location of the script relative to the path. If the script is moved or used on someone else's computer with a different file structure, it'll break.\n",
     "\n",
-    "In general, [installing reusable code in a package is the best way to go](https://nsls-ii.github.io/scientific-python-cookiecutter/). Packages provide a framework for organizing, documenting, testing and sharing code in a way that is easily understood by others.\n",
+    "* this all said, sometimes using ``sys.path`` is expedient in reproducible workflows in that it can allow code to be consolidated and re-used across multiple scripts in various locations\n",
+    "\n",
+    "For code that is useful across multiple projects, [installing reusable code in a package can be the best way to go](https://learn.scientific-python.org/development/tutorials/). Packages provide a framework for organizing, documenting, testing and sharing code in a way that is easily understood by others.\n",
     "\n",
     "Whatever you do, avoid importing with an `*` (i.e. ``from mycode import *``) at all costs. This imports everything from the namespace of a module, which can lead to unintended consequences."
    ]
@@ -816,7 +794,7 @@
     "\n",
     "### Changing the current working directory\n",
     "``pathlib`` doesn't do this.   \n",
-    "Note: this can obviously lead to trouble in scripts, so should usually be avoided, but sometimes it is necessary."
+    "Note: this can obviously lead to trouble in scripts, so should usually be avoided, but sometimes it is necessary. In groundwater modeling workflows, for example, this can help keep flow and transport model files organized in separate folders."
    ]
   },
   {
@@ -912,6 +890,33 @@
     "os.environ['CONDA_PREFIX']"
    ]
   },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "### Running system commands\n",
+    "`os.system` provides a limited way to run system commands. For more flexibility, use `subprocess` (below)."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "os.system('ls -l')"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# on Windows\n",
+    "os.system('dir')"
+   ]
+  },
   {
    "cell_type": "markdown",
    "metadata": {},
@@ -943,7 +948,7 @@
    "outputs": [],
    "source": [
     "# if on mac/unix\n",
-    "print(subprocess.run(['ls', '-l'], shell=True))"
+    "print(subprocess.run(['ls', '-l']))"
    ]
   },
   {
@@ -959,7 +964,7 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "print(subprocess.run(['ls', '-l'], shell=True, cwd='..'))"
+    "print(subprocess.run(['ls', '-l'], cwd='..'))"
    ]
   },
   {
@@ -978,7 +983,7 @@
    "source": [
     "## ``zipfile`` — Work with ZIP archives\n",
     "\n",
-    "#### zip up one of the files in data/"
+    "### zip up one of the files in data/"
    ]
   },
   {
@@ -995,7 +1000,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "#### now extract it"
+    "### now extract it"
    ]
   },
   {
@@ -1014,7 +1019,7 @@
    "source": [
     "## Testing Your Skills with a truly awful example:\n",
     "\n",
-    "#### the problem:\n",
+    "### the problem:\n",
     "Pretend that the file `data/fileio/netcdf_data.zip` contains some climate data (in the NetCDF format with the ``*.nc`` extension) that we downloaded. If you open `data/fileio/netcdf_data.zip`, you'll see that within a subfolder `zipped` are a bunch of additional subfolders, each for a different year. Within each subfolder is another zipfile. Within each of these zipfiles is yet another subfolder, inside of which is the actual data file we want (`prcp.nc`). "
    ]
   },
@@ -1033,7 +1038,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "#### the goal:\n",
+    "### the goal:\n",
     "To extract all of these `prcp.nc` files into a single folder, after renaming them with their respective years (obtained from their enclosing folders or zip files). e.g.  \n",
     "```\n",
     "prcp_1980.nc\n",
@@ -1042,7 +1047,7 @@
     "```\n",
     "This will allow us to open them together as a dataset in `xarray` (more on that later). Does this sound awful? I'm not making this up. This is the kind of structure you get when downloading tiles of climate data with the [Daymet Tile Selection Tool](https://daymet.ornl.gov/gridded/)\n",
     "\n",
-    "#### hint:\n",
+    "### hint:\n",
     "you might find these functions helpful:\n",
     "```\n",
     "ZipFile.extractall\n",
@@ -1061,7 +1066,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "#### hint: start by using ``ZipFile.extractall()`` to extract all of the individual zip files from the main zip archive\n",
+    "### hint: start by using ``ZipFile.extractall()`` to extract all of the individual zip files from the main zip archive\n",
     "This extracts the entire contents of the zip file to a designated folder"
    ]
   },
@@ -1252,6 +1257,18 @@
    "display_name": "pyclass",
    "language": "python",
    "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.11.11"
   }
  },
  "nbformat": 4,