add scripting exercise and extend getting started

Author: fabridamicelli

book/01_1_getting_started.ipynb

@@ -16,10 +16,20 @@
    "id": "32684093-9150-48ce-8c92-74fa4277905d",
    "metadata": {},
    "source": [
-    "## Introduction\n",
-    "For historical and technical reasons installing Python and managing dependencies is more cumbersome than it should be and as of today (October 2024) there are way too many ways to deal with those problems.\n",
-    "My recommendation here is: Pick one system that works for you and stick to it.\n",
-    "\n",
+    "## Set up your code editor\n",
+    "When editing code more extensively we want to have a so called integrated development environment (IDE).\n",
+    "There are many options out there.\n",
+    "Do not spend too much time deciding which to pick, you can always change your choice later.  \n",
+    "[VS-Code](https://code.visualstudio.com/) is a reasonable general choice as of today, it is free and widely adopted.\n",
+    "For a user experience more similar to matlab or R-studio and oriented to scientific computing you can check out [Spyder](https://www.spyder-ide.org)."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "1100eea3-69e1-4833-abe5-bd852d84720e",
+   "metadata": {},
+   "source": [
+    "## Set up Python with `uv`\n",
     "We essentially need to carry out three tasks:\n",
     "\n",
     "- Install Python (the language itself)\n",
@@ -34,7 +44,7 @@
    "id": "fe4bbe76-976f-4782-9153-95937a54b880",
    "metadata": {},
    "source": [
-    "## Install `uv`\n",
+    "### Install `uv`\n",
     "\n",
     "Run only one of these commands in your terminal - pick according to your operating system:\n",
     "\n",
@@ -44,7 +54,11 @@
     "```\n",
     "\n",
     "**Windows**  \n",
-    "Make sure you are using **powershell**!\n",
+    "\n",
+    "::: {.callout-important}\n",
+    "Make sure to use **powershell**! (default if you execute this in the VSCode terminal).\n",
+    ":::\n",
+    "\n",
     "\n",
     "```bash\n",
     "powershell -ExecutionPolicy ByPass -c \"irm https://astral.sh/uv/install.ps1 | iex\"\n",
@@ -69,7 +83,7 @@
    "id": "96e086b1-d39b-4508-89ea-c258bdea61ed",
    "metadata": {},
    "source": [
-    "## Working on a project\n",
+    "### Working on a project\n",
     "To start a new project run in your terminal:\n",
     "\n",
     "```bash\n",
@@ -100,6 +114,18 @@
     "```"
    ]
   },
+  {
+   "cell_type": "markdown",
+   "id": "76d149f3-09ef-4621-a92b-3b8c96362294",
+   "metadata": {},
+   "source": [
+    "## Set up Git and GitHub\n",
+    "\n",
+    "If you don't yet have one, [set up a free GitHub account](https://github.com/signup).\n",
+    "\n",
+    "Also, make sure to [install Git in your machine](https://github.com/git-guides/install-git)."
+   ]
+  },
   {
    "cell_type": "markdown",
    "id": "e6ec7541-6fc9-41c4-9947-d965c96d6f5a",

book/01_2_executing_code.ipynb

@@ -162,19 +162,6 @@
     "7) Same but print it's execution time"
    ]
   },
-  {
-   "cell_type": "markdown",
-   "id": "5cea3d9c-127b-4074-87c5-8d13a88da514",
-   "metadata": {},
-   "source": [
-    "## Code Editor\n",
-    "When editing code more extensively we also want to have a so called integrated development environment (IDE).\n",
-    "There are many options out there.\n",
-    "Do not spend too much time deciding which to pick, you can always change your choice later.  \n",
-    "[VS-Code](https://code.visualstudio.com/) is a reasonable general choice as of today, it is free and widely adopted.\n",
-    "For a user experience more similar to matlab or R-studio and oriented to scientific computing you can check out [Spyder](https://www.spyder-ide.org)."
-   ]
-  },
   {
    "cell_type": "markdown",
    "id": "bd5b592f-4fca-4192-bb4f-d1f25063312d",

book/02_types_data_structures.ipynb

@@ -317,11 +317,21 @@
    "metadata": {},
    "source": [
     "::: {.callout-tip}\n",
-    "In case you want to learn more about floating point numbers you can watch this [great video](https://www.youtube.com/watch?v=PZRI1IfStY0), but for now: it is just a way that computers have to _approximately_ represent numbers.\n",
+    "For now: A floating point is just a way that computers have to _approximately_ represent numbers.\n",
     "For the most part, we will not have to think too much about them starting our Python journey.\n",
+    "\n",
+    "In case you want to learn more about floating point numbers you can watch this great video:\n",
+    "{{< video https://www.youtube.com/watch?v=PZRI1IfStY0 >}}\n",
+    "\n",
     ":::"
    ]
   },
+  {
+   "cell_type": "markdown",
+   "id": "c358262b-dcc7-4c0a-b80f-84d3ce89abb1",
+   "metadata": {},
+   "source": []
+  },
   {
    "cell_type": "markdown",
    "id": "59888694-ad70-4a7d-82f3-9d83bc537a5f",

book/05_comprehensions.ipynb

@@ -395,7 +395,7 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.10.13"
+   "version": "3.13.3"
   }
  },
  "nbformat": 4,

book/061_dependencies.ipynb

@@ -16,14 +16,18 @@
    "id": "c79acc5e-0543-4994-b09a-1af21a509cd2",
    "metadata": {},
    "source": [
-    "In contrast to some built-in modules such as `pathlib` and `json` that we can import directly after installing python, there are third-party libraries that we want to use but need to be first installed, such as `numpy` and `pandas`.\n",
+    "In contrast to the code that we ourselves write and can import or to some built-in modules such as `pathlib` and `json` that we can import directly after installing python, there are third-party libraries that we want to use but need to be first installed, such as `numpy` and `pandas`.  \n",
     "This libraries are also called \"packages\".\n",
+    "There are several repositories where people publish python packages and from which we can download them, but the official one is [PyPI](https://pypi.org) (Python Package Index).\n",
+    "\n",
     "\n",
     "These are open-source libraries maintained mostly by voluntary contributions of people of the community – and you can also contribute to them!\n",
     "\n",
     "They are being actively developed and thus in permanent evolution.\n",
     "In order to avoid compatibility problems and to ensure reproducibility, we might want to specify the exact versions of the packages we use.\n",
-    "It turns out that is not an easy problem and the reason why need a **package manager**."
+    "It turns out that is not an easy problem and the reason why need a **package manager**.\n",
+    "\n",
+    "The good news is that `uv` does that for us too."
    ]
   },
   {
@@ -39,7 +43,7 @@
    "id": "2dce4970-2fa6-43ea-958b-1ae6a379dc94",
    "metadata": {},
    "source": [
-    "The current convention to specify dependencies (and other things) in python projects is to use a file called [`pyproject.toml`](https://packaging.python.org/en/latest/guides/writing-pyproject-toml/) that looks like this:\n",
+    "The standard to specify dependencies (and other things) in python projects is to use a file called [`pyproject.toml`](https://packaging.python.org/en/latest/guides/writing-pyproject-toml/) that looks like this:\n",
     "\n",
     "```yaml\n",
     "[project]\n",
@@ -58,9 +62,14 @@
     "]\n",
     "```\n",
     "\n",
+    "::: {.callout-note}\n",
+    "Take a look at the `pyproject.toml` file in you project, we already have one that `uv` automatically created for us!\n",
+    ":::\n",
+    "\n",
     "The \"dependencies\" section is the one we care about here.\n",
     "There we list all the third-party libraries that our project requires.\n",
     "Notice that we can specify both exact versions (with \"==\") as well as constraints (with \"<,>,>=,<=\").\n",
+    "\n",
     "In general we don't want to deal manually with those aspects – that's the job of a package manager."
    ]
   },
@@ -136,7 +145,7 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.10.13"
+   "version": "3.13.3"
   }
  },
  "nbformat": 4,

book/06_imports.ipynb

@@ -16,7 +16,7 @@
    "id": "eb6b0843-d4d9-4f63-b155-11739ba90398",
    "metadata": {},
    "source": [
-    "Every Python installation has a number of modules that come by default.\n",
+    "Every Python installation includes a number of modules by default.\n",
     "They are called collectively the \"Python Standard Library\" or \"built-in\" modules.\n",
     "\n",
     "Even though these modules are available in our computer, we cannot directly use them, we need to import them into our current namespace to actually use them.\n",
@@ -51,7 +51,7 @@
    "id": "0ca12de9-9840-417c-b41f-fd3032093c12",
    "metadata": {},
    "source": [
-    "That means that the word pathlib is unknown in this context.\n",
+    "That means that the word pathlib is unknown in this context (namespace).  \n",
     "Let's bring it in:"
    ]
   },
@@ -99,11 +99,17 @@
    "id": "a12b0f16-b0d7-45fe-b3fe-1d443199a117",
    "metadata": {},
    "source": [
-    "We can also import only specific parts of the module:\n",
-    "\n",
-    "```python\n",
-    "from pathlib import Path\n",
-    "```"
+    "We can also import only specific parts of the module (submodules):"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "15e3f4c3-4509-46b8-8a86-380a2f4131df",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from pathlib import Path"
    ]
   },
   {
@@ -150,40 +156,73 @@
    "id": "3b452c0b-682f-4868-9fd6-4dc6f7a5ff69",
    "metadata": {},
    "source": [
-    "## Script vs. exportable code\n",
-    "A Python module (any `.py` file) might contain code that we want to run (for example as a one-off script) along code that we only want to use somewhere else.  \n",
-    "For example, we might have a bunch of code like this in a file:\n",
+    "## Script vs. Importable Code\n",
+    "We can also import code that we wrote ourselves with the same syntax.\n",
+    "For example, if we have a python file called `mymodule.py` we can import it from another file with:\n",
+    "\n",
+    "```python\n",
+    "import mymodule\n",
+    "```\n",
     "\n",
+    "A Python module (any `.py` file) might contain code that we want to run (for example as a one-off script) along code that we only want to use somewhere else.  \n",
     "We can isolate the part of the code that we want to run as a script with this trick.\n",
     "\n",
+    "That's easier to show than to explain, so let's do some exercises."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "ee8413b5-b03f-456f-8298-a82feda2b076",
+   "metadata": {},
+   "source": [
+    "## Exercises\n",
+    "Create 2 files like these in our working project \n",
+    "`/pycourse/src/pycourse/data.py`, `/pycourse/src/pycourse/preprocessing.py`:\n",
+    "\n",
+    "\n",
     "```python\n",
     "# data.py\n",
+    "RAWDATA = ...  # 3 dots are called \"Ellipsis\" and act as placeholder\n",
     "\n",
-    "def download(url):\n",
-    "    ...\n",
-    "def process(data):\n",
+    "def fetch_data(raw_data):\n",
     "    ... \n",
-    "def plot(data):\n",
-    "    ...\n",
-    "     \n",
+    "\n",
     "if __name__ == \"__main__\":\n",
-    "    print(\"I am running as a script\")    \n",
-    "    def script_func():\n",
-    "        ...\n",
+    "    print(\"Fetching data\")    \n",
+    "    data = fetch_data(RAWDATA)\n",
+    "    print(data)\n",
+    "```\n",
+    "\n",
+    "```python\n",
+    "# preprocessing.py\n",
+    "\n",
+    "def cleanup(data):\n",
+    "    ... \n",
     "    \n",
-    "```"
+    "def preprocess_pipeline(raw_data):\n",
+    "    data = fetch_data(raw_data)\n",
+    "    return cleanup(data)\n",
+    "     \n",
+    "if __name__ == \"__main__\":\n",
+    "    print(\"Running preprocessing\")    \n",
+    "    raw_data = ...\n",
+    "    preprocess_pipeline(raw_data)\n",
+    "```\n",
+    "\n",
+    "\n",
+    "1) Copy the `RAWDATA` from the previous lab exercises into a file `data.py`\n",
+    "2) Create another file called `preprocessing.py` in the same directory.\n",
+    "3) Import the `RAWDATA` string inside `preprocessing.py`\n",
+    "4) Run `data.py` as a script with `uv run src/pycourse/data.py`. Pay attention to the printed output\n",
+    "5) Run preprocessing as a script with `uv run src/pycourse/preprocessing.py`. Pay attention to the printed output. What difference do you see compared to the previous point?"
    ]
   },
   {
    "cell_type": "markdown",
-   "id": "ee8413b5-b03f-456f-8298-a82feda2b076",
+   "id": "1add3a63-5817-4a9c-bfcb-d041ab25eb79",
    "metadata": {},
    "source": [
-    "## Exercises\n",
-    "1) Copy the content of the code above into a file `data.py` and run it\n",
-    "2) Create another file called `analysis.py` right next to `data.py` (in the same directory)\n",
-    "3) Inside `analysis.py` import `download` from `data.py` and run `analysis.py`\n",
-    "4) Inside `analysis.py` import `script_func` from `data.py` and run `analysis.py`. Do you understand what happens?"
+    "Congratulations! You just ran your first python scripts 🚀"
    ]
   }
  ],
@@ -203,7 +242,7 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.10.13"
+   "version": "3.13.3"
   }
  },
  "nbformat": 4,

book/_quarto.yml

@@ -38,9 +38,9 @@ book:
     - 03_1_flow_control.ipynb
     - 03_2_error_handling.ipynb
     - 04_functions.ipynb
-    - 041_scope.ipynb
+    # - 041_scope.ipynb
     - 05_comprehensions.ipynb
-    - lab_syntax.ipynb
+    - lab_builtin_for_the_win.ipynb
     - 06_imports.ipynb
     - 061_dependencies.ipynb
     - 062_read_and_write.ipynb

book/lab_syntax.ipynb book/lab_builtin_for_the_win.ipynbbook/lab_syntax.ipynb renamed to book/lab_builtin_for_the_win.ipynb

@@ -27,7 +27,7 @@
    "outputs": [],
    "source": [
     "#| code-fold: true\n",
-    "data = \"\"\"\\\n",
+    "RAWDATA = \"\"\"\\\n",
     "onset\tduration\ttrial_type\trewards\n",
     "13\t0\toptions_partner\t n/a\n",
     "16\t0\tdecision_risky_partner\t  25\n",
@@ -247,11 +247,11 @@
    "id": "e5d96dc1-20b0-4a94-bdca-82be982261cd",
    "metadata": {},
    "source": [
-    "a) Copy the data into a python file\n",
-    "b) Traverse and print the lines. Hint: Look at strings methods\n",
-    "c) Make a list of all the lines\n",
-    "d) Split each line into a tuple of 4 elements (representing onset, duration, trial_type and rewards)\n",
-    "e) Do the same but with dictionaries, that is, for each line you should get a dictionary that looks like this\n",
+    "1) Copy the data into a python file\n",
+    "2) Traverse and print the lines. Hint: Look at strings methods\n",
+    "3) Make a list of all the lines\n",
+    "4) Split each line into a tuple of 4 elements (representing onset, duration, trial_type and rewards)\n",
+    "5) Do the same but with dictionaries, that is, for each line you should get a dictionary that looks like this\n",
     "\n",
     "```python\n",
     "{\n",
@@ -261,15 +261,16 @@
     "    \"rewards\": some-value,\n",
     "}\n",
     "```\n",
-    "f) Sort the list of dictionaries according to the rewards value. \n",
-    "g) Repeat the same but in descendant order (larger values first).\n",
-    "h) Make a list of the unique \"trial_type\" values that appear in the data.\n",
-    "i) Make a dictionary that counts the number of appearances of each \"trial_type\".\n",
-    "j) Make a list of only the entries where \"trial_type\" is of type \"decision_SOMETHING\"\n",
-    "k) Print for each line a string that follows this pattern: \n",
+    "6) Sort the list of dictionaries according to the rewards value. \n",
+    "7) Repeat the same but in descendant order (larger values first).\n",
+    "8) Make a list of the unique \"trial_type\" values that appear in the data.\n",
+    "9) Make a dictionary that counts the number of appearances of each \"trial_type\".\n",
+    "10) Make a list of only the entries where \"trial_type\" is of type \"decision_SOMETHING\"\n",
+    "11) Print for each line a string that follows this pattern: \n",
     "```python\n",
     "\"For the trial type: <TRIAL-TYPE> the reward is <REWARD-VALUE>\"\n",
-    "```"
+    "```\n",
+    "12) Write a function called `cleanup` that takes the raw input string and returns a list of dictionaries (like exercise item 5)"
    ]
   },
   {