diff --git a/notebooks/part0_python_intro/00_skills_test_on_basics.ipynb b/notebooks/part0_python_intro/00_skills_test_on_basics.ipynb index b23d2b1c..dd75155b 100644 --- a/notebooks/part0_python_intro/00_skills_test_on_basics.ipynb +++ b/notebooks/part0_python_intro/00_skills_test_on_basics.ipynb @@ -298,7 +298,7 @@ "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", - "version": "3.11.10" + "version": "3.11.13" } }, "nbformat": 4, diff --git a/notebooks/part0_python_intro/solutions/00_python_basics_review.ipynb b/notebooks/part0_python_intro/solutions/00_python_basics_review.ipynb new file mode 100644 index 00000000..3c7039c5 --- /dev/null +++ b/notebooks/part0_python_intro/solutions/00_python_basics_review.ipynb @@ -0,0 +1,1414 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# 00: Introduction to python for hydrologists\n", + "\n", + "In this background introduction, we will review concepts introduced in the \n", + "[University of Waterloo Python Background Tutorial](https://cscircles.cemc.uwaterloo.ca/)\n", + "\n", + "This notebook is a general overview of Python. We will cover many of the basic aspects of Python that are distinct (and sometimes confusing) relative to other programming and scripting languages. \n", + "\n", + "First let's consider three ways that we can interact with Python:\n", + "\n", + "1. Through a command-line session\n", + "2. Using iPython\n", + "3. Using a Jupyter notebook like this one\n", + "\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "print('hello world')" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Using Python as a Calculator\n", + "Python can be used to simple perform arithmetic operations.\n", + "(note: in an iPython notebook like this one, shift-ENTER evaluates the code)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Spaces are fine" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "6*9" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "6 + (99.98 * 5.)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Exponents use `**` rather than `^` in some other languages." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "print (5**9.1)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### What if you want to do more than basic arithmetic?\n", + "There are a couple ways to access more mathematical functions: " + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "import numpy as np\n", + "import math\n", + "np.sqrt(9)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "? np" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "math.sqrt(9.1)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Of these two options, we typically use `np` because, as we will cover in detail later, `numpy` provides many other features important to the kind of scientific work we perform." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Data Structures and a few definitions \n", + "**data structure** -- A way to store and organize data in a computer. \n", + "In Python, there are a variety of data structures -- some mutable, and others immutable\n", + "\n", + "**mutability** -- An _immutable_ object is one whose property or state cannot be changed, whereas, _mutable_ objects can change state \n", + "\n", + "The most basic data types are numbers, including: \n", + "`integer (int)` \n", + "`float/double precision (float)` \n", + "`long ingteger (long)` \n", + "`complex (complex)`\n", + "There are also groups of characters:\n", + "`string (str)`\n", + "\n", + "Variables can be of any type and the results can be assigned to symbols without predefining type (like `DIMENSION` in FORTRAN). `print` statements display values to standard output, and the `type` function tells you which number type a variable (or number) is.\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "a = 5.9\n", + "print (a)\n", + "aa = 5.91\n", + "print (a)\n", + "print (aa)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "a2 = 5.\n", + "print (a2)\n", + "type(a2)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Variables and types\n", + "\n", + "### Symbol names \n", + "\n", + "Variable names in Python can contain alphanumerical characters `a-z`, `A-Z`, `0-9` and some special characters such as `_`. Normal variable names must start with a letter. \n", + "\n", + "By convention, variable names start with a lower-case letter, and Class names start with a capital letter. \n", + "\n", + "In addition, there are a number of Python keywords that cannot be used as variable names. These keywords are:\n", + "\n", + "> `and, as, assert, break, class, continue, def, del, elif, else, except, exec, finally, for, from, global, if, import, in, is, lambda, not, or, pass, print, raise, return, try, while, with, yield`\n", + "\n", + "Note: Be aware of the keyword `lambda`, which could easily be a natural variable name in a scientific program. But being a keyword, it cannot be used as a variable name.\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "import keyword\n", + "keyword.kwlist" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "There are also built-in objects that should not be used as variable names." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "import builtins\n", + "dir(builtins)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Assignment\n", + "\n", + "The assignment operator in Python is `=`. Python is a dynamically typed language, so we do not need to specify the type of a variable when we create one.\n", + "\n", + "Assigning a value to a new variable creates the variable:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "b=99.1\n", + "# variable names can practically as long as you like\n", + "this_reallyDESCRIPTIVE_crazyvariableNAME_is_maybe_a_bit_long = 7\n", + "print(this_reallyDESCRIPTIVE_crazyvariableNAME_is_maybe_a_bit_long)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "In general, appropriate type conversion is possible:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "a = 8.9\n", + "b = int(a)\n", + "print (b)\n", + "c = str(a)\n", + "b" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Strings\n", + "Grouping characters together creates strings." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "avar = \"this is a string (Includes 'spaces' and punctuation)\"\n", + "print(avar)\n", + "len(avar)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Using square brackets (`[]`) we can dereference values as if the string were an array or a list (as we will discuss next)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "print(avar[0])\n", + "print(avar[1])\n", + "# negative indices count from the end of the string\n", + "print(avar[-1])\n", + "print(avar[-2])\n", + "# we can even print every nth character\n", + "print(avar[::3])" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Since strings are just sequences of characters, they can be operated on using addition and multiplication." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "d = 'a' + 'b' + ' c '\n", + "print (d)\n", + "print (d*5)\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "For comparisons, it can be important to match case, so we can cast to upper or lower or a couple other options" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "avar.upper()" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "avar.lower()" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "avar.capitalize()" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "avar.swapcase()" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "We can also replace text using `replace()`" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "avar.replace('and','&')" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Lists ###\n", + "There are a few ways to group values in Python. The most basic way is a list, defined by square brackets(`[ ]`) and comma-delimited \n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "a = [2.3, 3.4, 5.5, 12.4, 33]\n", + "print(type(a))\n", + "print(type(a[1]))\n", + "print(a)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "#### Important Note!!! -->\n", + "Python uses zero-referencing (like C, but unlike MATLAB and FORTRAN). Using square brackets, look at the various values of `a` (e.g. `a[0]`, `a[2]`). Also, see what you get with `a[-1]` and `a[-2]`" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "a[0]" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Lists of Lists\n", + "You can make lists made up of anything (e.g. numbers, other lists, etc.)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "b=5\n", + "c=[3,'4stuff',5.9]\n", + "d = [a,b,c]\n", + "print (d)\n", + "print (d[0])\n", + "print (d[1])\n", + "print (d[2])" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "aaaa = [1,2,3]\n", + "aaaa + [5]" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### `append` versus `extend`\n", + "You can add to a list in two ways -- using the `extend` or `append` methods. Let's look at `append` first:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "a=[1,2,3]\n", + "print (a)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "a.append(4)\n", + "a.append(5)\n", + "print (a)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "a.append([6,7,8])\n", + "print (a)\n", + "a[-1]" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "This did exactly what it should, but looks strange at first. We `appended` on the data we wanted to, but since we passed a list, we just took the list `[6,7,8]` on to the end of `a`. What if we wanted to continue our sequence? Then we could use `extend`." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "a=[1,2,3,4,5]\n", + "a.extend([6,7,8])\n", + "print (a)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "The same behavior will happen even if the list as it is is already made up of multiple types." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "print (d)\n", + "d.append([5,6,7,8,9])\n", + "print (d)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "We can create an empty list two ways" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "a = list()\n", + "print (type(a), a)\n", + "b = []\n", + "print (type(b), b)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "a.append('stuff')\n", + "a.append(99)\n", + "print (a)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "aaa = [0]\n", + "aaa.append(8)\n", + "aaa" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "We can also add and remove elements from lists in a couple ways:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "a" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "a = [ 2,99,2,99, 'stuff']\n", + "print (a.index(99))\n", + "a" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# index tells us where in a list a specific value resides\n", + "print (a.index(99))\n", + "print (a.index('stuff'))\n", + "# insert can add a value -- note that this shifts the list over\n", + "a.insert(0, 'newstuff')\n", + "print (a)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "a[0] = 99999\n", + "print (a)\n", + "b = a.pop(4) #remove item in position 4 and assign to b\n", + "print (b)\n", + "a" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Tuples\n", + "A similar construction to a `list` is a `tuple`. A `tuple` is defined using regular parentheses (`()`). The key differences is tuples are *immutable*.\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# we can make a list\n", + "a = [3,4,5]\n", + "print (type(a))\n", + "print (a)\n", + "# it's mutable meaning we can change it in place\n", + "a[0] = 999\n", + "print (a)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# a tuple is similar\n", + "a = (3,4,5)\n", + "print(type(a))\n", + "print (a)\n", + "# we can dereference values from it in the same way as we would a list\n", + "print (a[0])\n", + "# but we cannot change values in place\n", + "a[0] = 999" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "a=(1,'howdy')\n", + "b=list(a)\n", + "print (type(a))\n", + "print (type(b))" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "We won't talk very much about `tuples` except to say that they are sometimes required as input to functions, sometimes functions return them, and they can be useful for other reasons. Also, they can be constructed using `zip` to pull together a couple `lists`." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "a = zip([1,2],[11,22],[111,222])\n", + "print (a)\n", + "print(type(a))\n", + "for b in a:\n", + " print(b)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "We can also assign tuples without using parentheses, although it's kind of sloppy. We can unpack a tuple into multiple variables." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "a = 5,6,7\n", + "val1, val2, val3 = a\n", + "print (a)\n", + "print (val1)\n", + "print (val2)\n", + "print (val3)\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "This can be done all at once as well!" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "val1, val2, val3 = 5,6,7\n", + "print (val1)\n", + "print (val2)\n", + "print (val3)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Dictionaries\n", + "\n", + "Dictionaries are also like lists, except that each element is a key-value pair. The syntax for dictionaries is `{key1 : value1, ...}`:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "params = {\"parameter1\" : 1.0,\n", + " \"parameter2\" : 2.0,\n", + " \"parameter3\" : 3.0,}\n", + "\n", + "print(type(params))\n", + "print(params)\n", + "\n", + "parlist = [ 1.0, 2.0, 3.0]\n", + "print (params)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "params.items()" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "The keys can be obtained separate from their values as a list." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "for i in params.keys():\n", + " print (i)\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "ckeys = [i for i in params.keys()]" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "params[ckeys[1]]" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "print(\"parameter1 = \" + str(params[\"parameter1\"]))\n", + "print(\"parameter2 = \" + str(params[\"parameter2\"]))\n", + "print(\"parameter3 = \" + str(params[\"parameter3\"]))" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Can also get just the values" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "params.values()" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Or everyone's favorite....`tuples`! `.items()` returns tuples of the key, values pairs" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "params.items()" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "params[\"parameter1\"] = \"A\"\n", + "params[\"parameter2\"] = \"B\"\n", + "\n", + "# add a new entry\n", + "params[\"parameter4\"] = \"D\"\n", + "\n", + "print(\"parameter1 = \" + str(params[\"parameter1\"]))\n", + "print(\"parameter2 = \" + str(params[\"parameter2\"]))\n", + "print(\"parameter3 = \" + str(params[\"parameter3\"]))\n", + "print(\"parameter4 = \" + str(params[\"parameter4\"]))" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "#### A trick using `zip` and `dict` ####\n", + "Say we have two lists -- one of which is a set of keys and another is accompanying values (like we read them in from two columns in a spreadsheet)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "keys = ['HK','VKA','RCH','COND']\n", + "vals = [1.2e-6, 0.001, 1.9e-9, 4.0]\n", + "MFdict = dict(zip(keys,vals))\n", + "MFdict" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Process Flow\n", + "We will cover conditional statements and a variety of loop types.\n", + "### Conditional statements: if, elif, else\n", + "\n", + "The Python syntax for conditional execution of code use the keywords `if`, `elif` (else if), `else`:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "statement1 = True\n", + "statement2 = True\n", + "\n", + "if statement1==True:\n", + " print(\"statement1 is True\")\n", + " \n", + "elif statement2==True:\n", + " print(\"statement2 is True\")\n", + " \n", + "else:\n", + " print(\"statement1 and statement2 are False\")" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "There are a couple critical things to see here. \n", + "\n", + "1. There is no `end if ` or other closing of the statement. This is all controlled by indentation.\n", + "2. Each statement ends in a `':'`\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "statement1 or statement2" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "not statement2" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Rather than just `True` or `False`, we can check for equivalence to other values.\n", + "\n", + "1. '`==`' indicates test for equality\n", + "2. '`!=`' indicates test for nonequality" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "a = 5\n", + "print (a)\n", + "a!=5.01" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "statement1 = 5.2\n", + "\n", + "if statement1 == 'junk':\n", + " print ('junky')\n", + " \n", + "elif statement1 != 5:\n", + " print ('neither \"junky\" nor 5')\n", + "else:\n", + " print ('must be 5 eh?')\n", + "print ('wow - it worked?')" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### An aside on indentation\n", + "Technically, tabs or any *consistent* number of spaces can be used to control indentation. A casual standard is using 4 spaces. Many editors replace tabs with spaces to be safe. \n", + "\n", + "The level of indentation defines blocks without requiring a symbol or word to close a block or loop." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "#### few examples of indentation behavior:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "statement1 = True\n", + "statement2 = True\n", + "\n", + "if statement1==True:\n", + " if statement2==True:\n", + " print(\"both statement1 and statement2 are True\")\n", + "print( statement2)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# Bad indentation!\n", + "if statement1==True:\n", + " if statement2==True:\n", + " print(\"both statement1 and statement2 are True\") # this line is not properly indented" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "statement1 = True \n", + "\n", + "if statement1==True:\n", + " print(\"printed if statement1 is True\")\n", + " \n", + " print(\"Trying to say False but still inside the if block\")" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "if statement1:\n", + " print(\"printed if statement1 is True\")\n", + " \n", + "print(\"Statement 1 is false --- printed now outside the if block\")" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Loops\n", + "\n", + "In Python, loops can be programmed in a number of different ways. The most common is the `for` loop, which is used together with iterable objects, such as lists. The basic syntax is:\n", + "\n", + "\n", + "**`for` loops**:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "for i in [1,2,[9,8,0],4]:\n", + " print (i)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "The `for` loop iterates over the elements of the supplied list, and executes the containing block once for each element. Note the same indentation rules as for `if` statements. \n", + "\n", + "Any kind of list can be used in the `for` loop. For example:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "range(4)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "for x in range(4): # by default range start at 0\n", + " print(x)\n", + "print ('-'*12)\n", + "print ('4 is missing but the length is {0}'.format(len(range(4))))" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Uh oh! Where is 4? `range` constructions do not include the final value. This takes some getting used to! \n", + "\n", + "There is a long discussion here: http://stackoverflow.com/questions/4504662/why-does-rangestart-end-not-include-end\n", + "\n", + "Suffice it to say that the length of a range is the number stated and this is all related to zero indexing.\n", + "\n", + "Ranges can also start at values other than 0, but still never include the final value." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "for x in range(-3,3):\n", + " print(x)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Notice that we aren't using an index like `i` to dereference items from the list. Rather, we are just iterating over those items. They need not be numbers." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "a = ['This', 'is', 'different', 'than', 'F77']\n", + "for word in a:\n", + " print (word)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "We could do this like in other languages, but quickly it becomes clear that it would be contrived." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "for i in range(len(a)):\n", + " print (a[i])" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Sometimes we also need an index for some reason. We can get both using `enumerate`" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "for i, word in enumerate(a):\n", + " print ('word number {0} is \"{1}\"'.format(i, word))" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "We can also iterate over the keys of a dictionary in a couple ways. First, just iterating over the keys:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "for key in params.keys():\n", + " print (key + \" = \" + str(params[key]))" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# the following shorthand is now preferred\n", + "for key in params:\n", + " print (key + \" = \" + str(params[key]))" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Second we can iterate over the items and unpack them." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "for key, val in params.items():\n", + " print (key + \" = \" + str(val))\n", + " #print (poo)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### List Comprehension\n", + "\n", + "A very Pythonic and compact way to build a list using a `for` loop is using a `list comprehension`" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "a = [x**3 for x in range(10)]\n", + "print (a)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "You can get conditional statements into a list comprehension -- but, once it get too complicated, it gets to be \"clever\" and it's better to be more explicit" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "b = [x**3 if x>0 else x for x in range(-5,5)]\n", + "print (b)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "adict = dict(zip(range(10),a))" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "for key, value in adict.items():\n", + " print (str(key) + \"^3 = \" + str(value))" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "We can also do a \"dictionary comprehension\" \n", + "http://stackoverflow.com/questions/1747817/python-create-a-dictionary-with-list-comprehension" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "d = {str(a) + \"^3\": a**3 for a in range(10)}\n", + "print (d)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "**`while` loops**:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "i = 0\n", + "while i<4:\n", + " print(i)\n", + " i += 1 # shorthand for i = i + 1" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "i = 5\n", + "while i: # shorthand for while i != 0\n", + " print('howdy again')\n", + " i -= 1 # shorthand for i = i - 1" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "i = 0\n", + "while 3:\n", + " print(i)\n", + " i += 2\n", + " if i > 10:\n", + " break" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## HEEEEEELP!\n", + "\n", + "You will always need help with Python. There are so many packages, modules, functions, datatypes, and everything else it takes a while to keep it all straight, and you will always need to update \n", + "\n", + "1. Stackoverflow is amazing--it can be toxic at times, but the community often converges on useful answers. Here's an [example](http://stackoverflow.com/questions/5505380/most-efficient-way-to-pull-specified-rows-from-a-2-d-array).\n", + "2. Official documentation is good, but can overwhelm.\n", + " \n", + " * Python in general: https://docs.python.org/3/contents.html\n", + " * numpy and scipy: http://docs.scipy.org/doc/\n", + " * matplotlib: https://matplotlib.org/stable/users/index.html and especially https://matplotlib.org/stable/gallery/index.html\n", + "3. Of course, Google is good _Google your error message text!_\n", + "4. If you like books, consider [O'Reilly](http://shop.oreilly.com/product/0636920028154.do)\n", + "5. While in iPython (or a notebook) try `help(os)` or `? os` for example.\n", + "\n", + " * `help(os)` brings up docstrings in the output window\n", + " * `?os` opens a separate pane below (that you can kick out to another tab using a button next to the X) \n", + " \n", + "6. what about AI? For sure, you can try to get AI agents to write your code. It's pretty effective, but definitely you need to verify what it suggests!\n" + ] + }, + { + "cell_type": "code", + "execution_count": 1, + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "" + ], + "text/plain": [ + "" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "from IPython.display import Image, display, HTML\n", + "display(HTML(''))\n" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.11.13" + } + }, + "nbformat": 4, + "nbformat_minor": 4 +} diff --git a/notebooks/part0_python_intro/solutions/00_python_basics_review__solutions.ipynb b/notebooks/part0_python_intro/solutions/00_python_basics_review__solutions.ipynb deleted file mode 100644 index 931ba905..00000000 --- a/notebooks/part0_python_intro/solutions/00_python_basics_review__solutions.ipynb +++ /dev/null @@ -1,1253 +0,0 @@ -{ - "cells": [ - { - "cell_type": "markdown", - "id": "e8898c38", - "metadata": {}, - "source": [ - "# TEST YOR SKILLS #0" - ] - }, - { - "cell_type": "code", - "execution_count": 1, - "id": "941f8e32", - "metadata": {}, - "outputs": [], - "source": [ - "from pathlib import Path\n", - "from sklearn.feature_extraction.text import ENGLISH_STOP_WORDS\n", - "stopwords = list(ENGLISH_STOP_WORDS)\n" - ] - }, - { - "cell_type": "code", - "execution_count": 2, - "id": "a1a07ba5", - "metadata": {}, - "outputs": [], - "source": [ - "datapath = Path('../data')" - ] - }, - { - "cell_type": "code", - "execution_count": 3, - "id": "f9edd245", - "metadata": {}, - "outputs": [], - "source": [ - "speech_raw = open(datapath / 'dream.txt').readlines()" - ] - }, - { - "cell_type": "code", - "execution_count": 4, - "id": "bad99527", - "metadata": {}, - "outputs": [], - "source": [ - "speech = [i.strip() for i in speech_raw]" - ] - }, - { - "cell_type": "code", - "execution_count": 5, - "id": "ff208e23", - "metadata": {}, - "outputs": [], - "source": [ - "newdata = []\n", - "for i in speech:\n", - " newdata.append(i)\n", - "newdata = ' '.join(newdata)" - ] - }, - { - "cell_type": "code", - "execution_count": 6, - "id": "3f4842ad", - "metadata": {}, - "outputs": [ - { - "data": { - "text/plain": [ - "9149" - ] - }, - "execution_count": 6, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "len(newdata)" - ] - }, - { - "cell_type": "code", - "execution_count": 7, - "id": "f5bd8ab1", - "metadata": {}, - "outputs": [ - { - "data": { - "text/plain": [ - "\"I am happy to join with you today in what will go down in history as the greatest demonstration for freedom in the history of our nation. Five score years ago, a great American, in whose symbolic shadow we stand today, signed the Emancipation Proclamation. This momentous decree came as a great beacon light of hope to millions of Negro slaves who had been seared in the flames of withering injustice. It came as a joyous daybreak to end the long night of their captivity. But one hundred years later, the Negro still is not free. One hundred years later, the life of the Negro is still sadly crippled by the manacles of segregation and the chains of discrimination. One hundred years later, the Negro lives on a lonely island of poverty in the midst of a vast ocean of material prosperity. One hundred years later, the Negro is still languished in the corners of American society and finds himself an exile in his own land. And so we've come here today to dramatize a shameful condition. In a sense we've come to our nation's capital to cash a check. When the architects of our republic wrote the magnificent words of the Constitution and the Declaration of Independence, they were signing a promissory note to which every American was to fall heir. This note was a promise that all men, yes, black men as well as white men, would be guaranteed the unalienable Rights of Life, Liberty and the pursuit of Happiness. It is obvious today that America has defaulted on this promissory note, insofar as her citizens of color are concerned. Instead of honoring this sacred obligation, America has given the Negro people a bad check, a check which has come back marked insufficient funds. But we refuse to believe that the bank of justice is bankrupt. We refuse to believe that there are insufficient funds in the great vaults of opportunity of this nation. And so, we've come to cash this check, a check that will give us upon demand the riches of freedom and the security of justice. We have also come to this hallowed spot to remind America of the fierce urgency of Now. This is no time to engage in the luxury of cooling off or to take the tranquilizing drug of gradualism. Now is the time to make real the promises of democracy. Now is the time to rise from the dark and desolate valley of segregation to the sunlit path of racial justice. Now is the time to lift our nation from the quicksands of racial injustice to the solid rock of brotherhood. Now is the time to make justice a reality for all of God's children. It would be fatal for the nation to overlook the urgency of the moment. This sweltering summer of the Negro's legitimate discontent will not pass until there is an invigorating autumn of freedom and equality. Nineteen sixty-three is not an end, but a beginning. And those who hope that the Negro needed to blow off steam and will now be content will have a rude awakening if the nation returns to business as usual. And there will be neither rest nor tranquility in America until the Negro is granted his citizenship rights. The whirlwinds of revolt will continue to shake the foundations of our nation until the bright day of justice emerges. But there is something that I must say to my people, who stand on the warm threshold which leads into the palace of justice: In the process of gaining our rightful place, we must not be guilty of wrongful deeds. Let us not seek to satisfy our thirst for freedom by drinking from the cup of bitterness and hatred. We must forever conduct our struggle on the high plane of dignity and discipline. We must not allow our creative protest to degenerate into physical violence. Again and again, we must rise to the majestic heights of meeting physical force with soul force. The marvelous new militancy which has engulfed the Negro community must not lead us to a distrust of all white people, for many of our white brothers, as evidenced by their presence here today, have come to realize that their destiny is tied up with our destiny. And they have come to realize that their freedom is inextricably bound to our freedom. We cannot walk alone. And as we walk, we must make the pledge that we shall always march ahead. We cannot turn back. There are those who are asking the devotees of civil rights, When will you be satisfied? We can never be satisfied as long as the Negro is the victim of the unspeakable horrors of police brutality. We can never be satisfied as long as our bodies, heavy with the fatigue of travel, cannot gain lodging in the motels of the highways and the hotels of the cities. We cannot be satisfied as long as the negro's basic mobility is from a smaller ghetto to a larger one. We can never be satisfied as long as our children are stripped of their self-hood and robbed of their dignity by signs stating: For Whites Only. We cannot be satisfied as long as a Negro in Mississippi cannot vote and a Negro in New York believes he has nothing for which to vote. No, no, we are not satisfied, and we will not be satisfied until justice rolls down like waters, and righteousness like a mighty stream. I am not unmindful that some of you have come here out of great trials and tribulations. Some of you have come fresh from narrow jail cells. And some of you have come from areas where your quest -- quest for freedom left you battered by the storms of persecution and staggered by the winds of police brutality. You have been the veterans of creative suffering. Continue to work with the faith that unearned suffering is redemptive. Go back to Mississippi, go back to Alabama, go back to South Carolina, go back to Georgia, go back to Louisiana, go back to the slums and ghettos of our northern cities, knowing that somehow this situation can and will be changed. Let us not wallow in the valley of despair, I say to you today, my friends. And so even though we face the difficulties of today and tomorrow, I still have a dream. It is a dream deeply rooted in the American dream. I have a dream that one day this nation will rise up and live out the true meaning of its creed: We hold these truths to be self-evident, that all men are created equal. I have a dream that one day on the red hills of Georgia, the sons of former slaves and the sons of former slave owners will be able to sit down together at the table of brotherhood. I have a dream that one day even the state of Mississippi, a state sweltering with the heat of injustice, sweltering with the heat of oppression, will be transformed into an oasis of freedom and justice. I have a dream that my four little children will one day live in a nation where they will not be judged by the color of their skin but by the content of their character. I have a dream today! I have a dream that one day, down in Alabama, with its vicious racists, with its governor having his lips dripping with the words of interposition and nullification -- one day right there in Alabama little black boys and black girls will be able to join hands with little white boys and white girls as sisters and brothers. I have a dream today! I have a dream that one day every valley shall be exalted, and every hill and mountain shall be made low, the rough places will be made plain, and the crooked places will be made straight; and the glory of the Lord shall be revealed and all flesh shall see it together. This is our hope, and this is the faith that I go back to the South with. With this faith, we will be able to hew out of the mountain of despair a stone of hope. With this faith, we will be able to transform the jangling discords of our nation into a beautiful symphony of brotherhood. With this faith, we will be able to work together, to pray together, to struggle together, to go to jail together, to stand up for freedom together, knowing that we will be free one day. And this will be the day -- this will be the day when all of God's children will be able to sing with new meaning: My country 'tis of thee, sweet land of liberty, of thee I sing. Land where my fathers died, land of the Pilgrim's pride, From every mountainside, let freedom ring! And if America is to be a great nation, this must become true. And so let freedom ring from the prodigious hilltops of New Hampshire. Let freedom ring from the mighty mountains of New York. Let freedom ring from the heightening Alleghenies of Pennsylvania. Let freedom ring from the snow-capped Rockies of Colorado. Let freedom ring from the curvaceous slopes of California. But not only that: Let freedom ring from Stone Mountain of Georgia. Let freedom ring from Lookout Mountain of Tennessee. Let freedom ring from every hill and molehill of Mississippi. From every mountainside, let freedom ring. And when this happens, and when we allow freedom ring, when we let it ring from every village and every hamlet, from every state and every city, we will be able to speed up that day when all of God's children, black men and white men, Jews and Gentiles, Protestants and Catholics, will be able to join hands and sing in the words of the old Negro spiritual: Free at last! Free at last! Thank God Almighty, we are free at last!\"" - ] - }, - "execution_count": 7, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "newdata" - ] - }, - { - "cell_type": "code", - "execution_count": 8, - "id": "0051f87b", - "metadata": {}, - "outputs": [], - "source": [ - "newdata = newdata.lower()" - ] - }, - { - "cell_type": "code", - "execution_count": 9, - "id": "3371ec7d", - "metadata": {}, - "outputs": [ - { - "data": { - "text/plain": [ - "\"i am happy to join with you today in what will go down in history as the greatest demonstration for freedom in the history of our nation. five score years ago, a great american, in whose symbolic shadow we stand today, signed the emancipation proclamation. this momentous decree came as a great beacon light of hope to millions of negro slaves who had been seared in the flames of withering injustice. it came as a joyous daybreak to end the long night of their captivity. but one hundred years later, the negro still is not free. one hundred years later, the life of the negro is still sadly crippled by the manacles of segregation and the chains of discrimination. one hundred years later, the negro lives on a lonely island of poverty in the midst of a vast ocean of material prosperity. one hundred years later, the negro is still languished in the corners of american society and finds himself an exile in his own land. and so we've come here today to dramatize a shameful condition. in a sense we've come to our nation's capital to cash a check. when the architects of our republic wrote the magnificent words of the constitution and the declaration of independence, they were signing a promissory note to which every american was to fall heir. this note was a promise that all men, yes, black men as well as white men, would be guaranteed the unalienable rights of life, liberty and the pursuit of happiness. it is obvious today that america has defaulted on this promissory note, insofar as her citizens of color are concerned. instead of honoring this sacred obligation, america has given the negro people a bad check, a check which has come back marked insufficient funds. but we refuse to believe that the bank of justice is bankrupt. we refuse to believe that there are insufficient funds in the great vaults of opportunity of this nation. and so, we've come to cash this check, a check that will give us upon demand the riches of freedom and the security of justice. we have also come to this hallowed spot to remind america of the fierce urgency of now. this is no time to engage in the luxury of cooling off or to take the tranquilizing drug of gradualism. now is the time to make real the promises of democracy. now is the time to rise from the dark and desolate valley of segregation to the sunlit path of racial justice. now is the time to lift our nation from the quicksands of racial injustice to the solid rock of brotherhood. now is the time to make justice a reality for all of god's children. it would be fatal for the nation to overlook the urgency of the moment. this sweltering summer of the negro's legitimate discontent will not pass until there is an invigorating autumn of freedom and equality. nineteen sixty-three is not an end, but a beginning. and those who hope that the negro needed to blow off steam and will now be content will have a rude awakening if the nation returns to business as usual. and there will be neither rest nor tranquility in america until the negro is granted his citizenship rights. the whirlwinds of revolt will continue to shake the foundations of our nation until the bright day of justice emerges. but there is something that i must say to my people, who stand on the warm threshold which leads into the palace of justice: in the process of gaining our rightful place, we must not be guilty of wrongful deeds. let us not seek to satisfy our thirst for freedom by drinking from the cup of bitterness and hatred. we must forever conduct our struggle on the high plane of dignity and discipline. we must not allow our creative protest to degenerate into physical violence. again and again, we must rise to the majestic heights of meeting physical force with soul force. the marvelous new militancy which has engulfed the negro community must not lead us to a distrust of all white people, for many of our white brothers, as evidenced by their presence here today, have come to realize that their destiny is tied up with our destiny. and they have come to realize that their freedom is inextricably bound to our freedom. we cannot walk alone. and as we walk, we must make the pledge that we shall always march ahead. we cannot turn back. there are those who are asking the devotees of civil rights, when will you be satisfied? we can never be satisfied as long as the negro is the victim of the unspeakable horrors of police brutality. we can never be satisfied as long as our bodies, heavy with the fatigue of travel, cannot gain lodging in the motels of the highways and the hotels of the cities. we cannot be satisfied as long as the negro's basic mobility is from a smaller ghetto to a larger one. we can never be satisfied as long as our children are stripped of their self-hood and robbed of their dignity by signs stating: for whites only. we cannot be satisfied as long as a negro in mississippi cannot vote and a negro in new york believes he has nothing for which to vote. no, no, we are not satisfied, and we will not be satisfied until justice rolls down like waters, and righteousness like a mighty stream. i am not unmindful that some of you have come here out of great trials and tribulations. some of you have come fresh from narrow jail cells. and some of you have come from areas where your quest -- quest for freedom left you battered by the storms of persecution and staggered by the winds of police brutality. you have been the veterans of creative suffering. continue to work with the faith that unearned suffering is redemptive. go back to mississippi, go back to alabama, go back to south carolina, go back to georgia, go back to louisiana, go back to the slums and ghettos of our northern cities, knowing that somehow this situation can and will be changed. let us not wallow in the valley of despair, i say to you today, my friends. and so even though we face the difficulties of today and tomorrow, i still have a dream. it is a dream deeply rooted in the american dream. i have a dream that one day this nation will rise up and live out the true meaning of its creed: we hold these truths to be self-evident, that all men are created equal. i have a dream that one day on the red hills of georgia, the sons of former slaves and the sons of former slave owners will be able to sit down together at the table of brotherhood. i have a dream that one day even the state of mississippi, a state sweltering with the heat of injustice, sweltering with the heat of oppression, will be transformed into an oasis of freedom and justice. i have a dream that my four little children will one day live in a nation where they will not be judged by the color of their skin but by the content of their character. i have a dream today! i have a dream that one day, down in alabama, with its vicious racists, with its governor having his lips dripping with the words of interposition and nullification -- one day right there in alabama little black boys and black girls will be able to join hands with little white boys and white girls as sisters and brothers. i have a dream today! i have a dream that one day every valley shall be exalted, and every hill and mountain shall be made low, the rough places will be made plain, and the crooked places will be made straight; and the glory of the lord shall be revealed and all flesh shall see it together. this is our hope, and this is the faith that i go back to the south with. with this faith, we will be able to hew out of the mountain of despair a stone of hope. with this faith, we will be able to transform the jangling discords of our nation into a beautiful symphony of brotherhood. with this faith, we will be able to work together, to pray together, to struggle together, to go to jail together, to stand up for freedom together, knowing that we will be free one day. and this will be the day -- this will be the day when all of god's children will be able to sing with new meaning: my country 'tis of thee, sweet land of liberty, of thee i sing. land where my fathers died, land of the pilgrim's pride, from every mountainside, let freedom ring! and if america is to be a great nation, this must become true. and so let freedom ring from the prodigious hilltops of new hampshire. let freedom ring from the mighty mountains of new york. let freedom ring from the heightening alleghenies of pennsylvania. let freedom ring from the snow-capped rockies of colorado. let freedom ring from the curvaceous slopes of california. but not only that: let freedom ring from stone mountain of georgia. let freedom ring from lookout mountain of tennessee. let freedom ring from every hill and molehill of mississippi. from every mountainside, let freedom ring. and when this happens, and when we allow freedom ring, when we let it ring from every village and every hamlet, from every state and every city, we will be able to speed up that day when all of god's children, black men and white men, jews and gentiles, protestants and catholics, will be able to join hands and sing in the words of the old negro spiritual: free at last! free at last! thank god almighty, we are free at last!\"" - ] - }, - "execution_count": 9, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "newdata" - ] - }, - { - "cell_type": "code", - "execution_count": 10, - "id": "0b3c8d11", - "metadata": {}, - "outputs": [], - "source": [ - "punks = ['.',',',':',';','!','-']" - ] - }, - { - "cell_type": "code", - "execution_count": 11, - "id": "0ad2436e", - "metadata": {}, - "outputs": [], - "source": [ - "for punk in punks:\n", - " newdata = newdata.replace(punk, '')" - ] - }, - { - "cell_type": "code", - "execution_count": 12, - "id": "99e90459", - "metadata": {}, - "outputs": [ - { - "data": { - "text/plain": [ - "\"i am happy to join with you today in what will go down in history as the greatest demonstration for freedom in the history of our nation five score years ago a great american in whose symbolic shadow we stand today signed the emancipation proclamation this momentous decree came as a great beacon light of hope to millions of negro slaves who had been seared in the flames of withering injustice it came as a joyous daybreak to end the long night of their captivity but one hundred years later the negro still is not free one hundred years later the life of the negro is still sadly crippled by the manacles of segregation and the chains of discrimination one hundred years later the negro lives on a lonely island of poverty in the midst of a vast ocean of material prosperity one hundred years later the negro is still languished in the corners of american society and finds himself an exile in his own land and so we've come here today to dramatize a shameful condition in a sense we've come to our nation's capital to cash a check when the architects of our republic wrote the magnificent words of the constitution and the declaration of independence they were signing a promissory note to which every american was to fall heir this note was a promise that all men yes black men as well as white men would be guaranteed the unalienable rights of life liberty and the pursuit of happiness it is obvious today that america has defaulted on this promissory note insofar as her citizens of color are concerned instead of honoring this sacred obligation america has given the negro people a bad check a check which has come back marked insufficient funds but we refuse to believe that the bank of justice is bankrupt we refuse to believe that there are insufficient funds in the great vaults of opportunity of this nation and so we've come to cash this check a check that will give us upon demand the riches of freedom and the security of justice we have also come to this hallowed spot to remind america of the fierce urgency of now this is no time to engage in the luxury of cooling off or to take the tranquilizing drug of gradualism now is the time to make real the promises of democracy now is the time to rise from the dark and desolate valley of segregation to the sunlit path of racial justice now is the time to lift our nation from the quicksands of racial injustice to the solid rock of brotherhood now is the time to make justice a reality for all of god's children it would be fatal for the nation to overlook the urgency of the moment this sweltering summer of the negro's legitimate discontent will not pass until there is an invigorating autumn of freedom and equality nineteen sixtythree is not an end but a beginning and those who hope that the negro needed to blow off steam and will now be content will have a rude awakening if the nation returns to business as usual and there will be neither rest nor tranquility in america until the negro is granted his citizenship rights the whirlwinds of revolt will continue to shake the foundations of our nation until the bright day of justice emerges but there is something that i must say to my people who stand on the warm threshold which leads into the palace of justice in the process of gaining our rightful place we must not be guilty of wrongful deeds let us not seek to satisfy our thirst for freedom by drinking from the cup of bitterness and hatred we must forever conduct our struggle on the high plane of dignity and discipline we must not allow our creative protest to degenerate into physical violence again and again we must rise to the majestic heights of meeting physical force with soul force the marvelous new militancy which has engulfed the negro community must not lead us to a distrust of all white people for many of our white brothers as evidenced by their presence here today have come to realize that their destiny is tied up with our destiny and they have come to realize that their freedom is inextricably bound to our freedom we cannot walk alone and as we walk we must make the pledge that we shall always march ahead we cannot turn back there are those who are asking the devotees of civil rights when will you be satisfied? we can never be satisfied as long as the negro is the victim of the unspeakable horrors of police brutality we can never be satisfied as long as our bodies heavy with the fatigue of travel cannot gain lodging in the motels of the highways and the hotels of the cities we cannot be satisfied as long as the negro's basic mobility is from a smaller ghetto to a larger one we can never be satisfied as long as our children are stripped of their selfhood and robbed of their dignity by signs stating for whites only we cannot be satisfied as long as a negro in mississippi cannot vote and a negro in new york believes he has nothing for which to vote no no we are not satisfied and we will not be satisfied until justice rolls down like waters and righteousness like a mighty stream i am not unmindful that some of you have come here out of great trials and tribulations some of you have come fresh from narrow jail cells and some of you have come from areas where your quest quest for freedom left you battered by the storms of persecution and staggered by the winds of police brutality you have been the veterans of creative suffering continue to work with the faith that unearned suffering is redemptive go back to mississippi go back to alabama go back to south carolina go back to georgia go back to louisiana go back to the slums and ghettos of our northern cities knowing that somehow this situation can and will be changed let us not wallow in the valley of despair i say to you today my friends and so even though we face the difficulties of today and tomorrow i still have a dream it is a dream deeply rooted in the american dream i have a dream that one day this nation will rise up and live out the true meaning of its creed we hold these truths to be selfevident that all men are created equal i have a dream that one day on the red hills of georgia the sons of former slaves and the sons of former slave owners will be able to sit down together at the table of brotherhood i have a dream that one day even the state of mississippi a state sweltering with the heat of injustice sweltering with the heat of oppression will be transformed into an oasis of freedom and justice i have a dream that my four little children will one day live in a nation where they will not be judged by the color of their skin but by the content of their character i have a dream today i have a dream that one day down in alabama with its vicious racists with its governor having his lips dripping with the words of interposition and nullification one day right there in alabama little black boys and black girls will be able to join hands with little white boys and white girls as sisters and brothers i have a dream today i have a dream that one day every valley shall be exalted and every hill and mountain shall be made low the rough places will be made plain and the crooked places will be made straight and the glory of the lord shall be revealed and all flesh shall see it together this is our hope and this is the faith that i go back to the south with with this faith we will be able to hew out of the mountain of despair a stone of hope with this faith we will be able to transform the jangling discords of our nation into a beautiful symphony of brotherhood with this faith we will be able to work together to pray together to struggle together to go to jail together to stand up for freedom together knowing that we will be free one day and this will be the day this will be the day when all of god's children will be able to sing with new meaning my country 'tis of thee sweet land of liberty of thee i sing land where my fathers died land of the pilgrim's pride from every mountainside let freedom ring and if america is to be a great nation this must become true and so let freedom ring from the prodigious hilltops of new hampshire let freedom ring from the mighty mountains of new york let freedom ring from the heightening alleghenies of pennsylvania let freedom ring from the snowcapped rockies of colorado let freedom ring from the curvaceous slopes of california but not only that let freedom ring from stone mountain of georgia let freedom ring from lookout mountain of tennessee let freedom ring from every hill and molehill of mississippi from every mountainside let freedom ring and when this happens and when we allow freedom ring when we let it ring from every village and every hamlet from every state and every city we will be able to speed up that day when all of god's children black men and white men jews and gentiles protestants and catholics will be able to join hands and sing in the words of the old negro spiritual free at last free at last thank god almighty we are free at last\"" - ] - }, - "execution_count": 12, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "newdata" - ] - }, - { - "cell_type": "code", - "execution_count": 13, - "id": "10966edf", - "metadata": {}, - "outputs": [ - { - "data": { - "text/plain": [ - "1664" - ] - }, - "execution_count": 13, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "newdata = newdata.split()\n", - "len(newdata)" - ] - }, - { - "cell_type": "code", - "execution_count": 14, - "id": "58ff104d", - "metadata": {}, - "outputs": [], - "source": [ - "finaldata = []\n", - "for i in newdata:\n", - " if i not in stopwords:\n", - " finaldata.append(i)" - ] - }, - { - "cell_type": "code", - "execution_count": 15, - "id": "9f62bf76", - "metadata": {}, - "outputs": [ - { - "data": { - "text/plain": [ - "724" - ] - }, - "execution_count": 15, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "len(finaldata)" - ] - }, - { - "cell_type": "code", - "execution_count": 16, - "id": "3aa73eaa", - "metadata": {}, - "outputs": [ - { - "data": { - "text/plain": [ - "False" - ] - }, - "execution_count": 16, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "'i' in finaldata" - ] - }, - { - "cell_type": "code", - "execution_count": 17, - "id": "7a1321e4", - "metadata": {}, - "outputs": [], - "source": [ - "# make an empty dictionary that will have keys being the unique words in the speech, \n", - "# and values being their counts\n", - "counts = dict()" - ] - }, - { - "cell_type": "code", - "execution_count": 18, - "id": "f8aaf7f2", - "metadata": {}, - "outputs": [], - "source": [ - "for cw in finaldata:\n", - " if cw in counts.keys():\n", - " counts[cw] += 1\n", - " else:\n", - " counts[cw] = 1" - ] - }, - { - "cell_type": "code", - "execution_count": 19, - "id": "51d9dcd0", - "metadata": {}, - "outputs": [ - { - "data": { - "text/plain": [ - "{'happy': 1,\n", - " 'join': 3,\n", - " 'today': 9,\n", - " 'history': 2,\n", - " 'greatest': 1,\n", - " 'demonstration': 1,\n", - " 'freedom': 20,\n", - " 'nation': 10,\n", - " 'score': 1,\n", - " 'years': 5,\n", - " 'ago': 1,\n", - " 'great': 5,\n", - " 'american': 4,\n", - " 'symbolic': 1,\n", - " 'shadow': 1,\n", - " 'stand': 3,\n", - " 'signed': 1,\n", - " 'emancipation': 1,\n", - " 'proclamation': 1,\n", - " 'momentous': 1,\n", - " 'decree': 1,\n", - " 'came': 2,\n", - " 'beacon': 1,\n", - " 'light': 1,\n", - " 'hope': 4,\n", - " 'millions': 1,\n", - " 'negro': 13,\n", - " 'slaves': 2,\n", - " 'seared': 1,\n", - " 'flames': 1,\n", - " 'withering': 1,\n", - " 'injustice': 3,\n", - " 'joyous': 1,\n", - " 'daybreak': 1,\n", - " 'end': 2,\n", - " 'long': 6,\n", - " 'night': 1,\n", - " 'captivity': 1,\n", - " 'later': 4,\n", - " 'free': 5,\n", - " 'life': 2,\n", - " 'sadly': 1,\n", - " 'crippled': 1,\n", - " 'manacles': 1,\n", - " 'segregation': 2,\n", - " 'chains': 1,\n", - " 'discrimination': 1,\n", - " 'lives': 1,\n", - " 'lonely': 1,\n", - " 'island': 1,\n", - " 'poverty': 1,\n", - " 'midst': 1,\n", - " 'vast': 1,\n", - " 'ocean': 1,\n", - " 'material': 1,\n", - " 'prosperity': 1,\n", - " 'languished': 1,\n", - " 'corners': 1,\n", - " 'society': 1,\n", - " 'finds': 1,\n", - " 'exile': 1,\n", - " 'land': 4,\n", - " \"we've\": 3,\n", - " 'come': 10,\n", - " 'dramatize': 1,\n", - " 'shameful': 1,\n", - " 'condition': 1,\n", - " 'sense': 1,\n", - " \"nation's\": 1,\n", - " 'capital': 1,\n", - " 'cash': 2,\n", - " 'check': 5,\n", - " 'architects': 1,\n", - " 'republic': 1,\n", - " 'wrote': 1,\n", - " 'magnificent': 1,\n", - " 'words': 3,\n", - " 'constitution': 1,\n", - " 'declaration': 1,\n", - " 'independence': 1,\n", - " 'signing': 1,\n", - " 'promissory': 2,\n", - " 'note': 3,\n", - " 'fall': 1,\n", - " 'heir': 1,\n", - " 'promise': 1,\n", - " 'men': 6,\n", - " 'yes': 1,\n", - " 'black': 4,\n", - " 'white': 6,\n", - " 'guaranteed': 1,\n", - " 'unalienable': 1,\n", - " 'rights': 3,\n", - " 'liberty': 2,\n", - " 'pursuit': 1,\n", - " 'happiness': 1,\n", - " 'obvious': 1,\n", - " 'america': 5,\n", - " 'defaulted': 1,\n", - " 'insofar': 1,\n", - " 'citizens': 1,\n", - " 'color': 2,\n", - " 'concerned': 1,\n", - " 'instead': 1,\n", - " 'honoring': 1,\n", - " 'sacred': 1,\n", - " 'obligation': 1,\n", - " 'given': 1,\n", - " 'people': 3,\n", - " 'bad': 1,\n", - " 'marked': 1,\n", - " 'insufficient': 2,\n", - " 'funds': 2,\n", - " 'refuse': 2,\n", - " 'believe': 2,\n", - " 'bank': 1,\n", - " 'justice': 8,\n", - " 'bankrupt': 1,\n", - " 'vaults': 1,\n", - " 'opportunity': 1,\n", - " 'demand': 1,\n", - " 'riches': 1,\n", - " 'security': 1,\n", - " 'hallowed': 1,\n", - " 'spot': 1,\n", - " 'remind': 1,\n", - " 'fierce': 1,\n", - " 'urgency': 2,\n", - " 'time': 5,\n", - " 'engage': 1,\n", - " 'luxury': 1,\n", - " 'cooling': 1,\n", - " 'tranquilizing': 1,\n", - " 'drug': 1,\n", - " 'gradualism': 1,\n", - " 'make': 3,\n", - " 'real': 1,\n", - " 'promises': 1,\n", - " 'democracy': 1,\n", - " 'rise': 3,\n", - " 'dark': 1,\n", - " 'desolate': 1,\n", - " 'valley': 3,\n", - " 'sunlit': 1,\n", - " 'path': 1,\n", - " 'racial': 2,\n", - " 'lift': 1,\n", - " 'quicksands': 1,\n", - " 'solid': 1,\n", - " 'rock': 1,\n", - " 'brotherhood': 3,\n", - " 'reality': 1,\n", - " \"god's\": 3,\n", - " 'children': 5,\n", - " 'fatal': 1,\n", - " 'overlook': 1,\n", - " 'moment': 1,\n", - " 'sweltering': 3,\n", - " 'summer': 1,\n", - " \"negro's\": 2,\n", - " 'legitimate': 1,\n", - " 'discontent': 1,\n", - " 'pass': 1,\n", - " 'invigorating': 1,\n", - " 'autumn': 1,\n", - " 'equality': 1,\n", - " 'nineteen': 1,\n", - " 'sixtythree': 1,\n", - " 'beginning': 1,\n", - " 'needed': 1,\n", - " 'blow': 1,\n", - " 'steam': 1,\n", - " 'content': 2,\n", - " 'rude': 1,\n", - " 'awakening': 1,\n", - " 'returns': 1,\n", - " 'business': 1,\n", - " 'usual': 1,\n", - " 'rest': 1,\n", - " 'tranquility': 1,\n", - " 'granted': 1,\n", - " 'citizenship': 1,\n", - " 'whirlwinds': 1,\n", - " 'revolt': 1,\n", - " 'continue': 2,\n", - " 'shake': 1,\n", - " 'foundations': 1,\n", - " 'bright': 1,\n", - " 'day': 12,\n", - " 'emerges': 1,\n", - " 'say': 2,\n", - " 'warm': 1,\n", - " 'threshold': 1,\n", - " 'leads': 1,\n", - " 'palace': 1,\n", - " 'process': 1,\n", - " 'gaining': 1,\n", - " 'rightful': 1,\n", - " 'place': 1,\n", - " 'guilty': 1,\n", - " 'wrongful': 1,\n", - " 'deeds': 1,\n", - " 'let': 13,\n", - " 'seek': 1,\n", - " 'satisfy': 1,\n", - " 'thirst': 1,\n", - " 'drinking': 1,\n", - " 'cup': 1,\n", - " 'bitterness': 1,\n", - " 'hatred': 1,\n", - " 'forever': 1,\n", - " 'conduct': 1,\n", - " 'struggle': 2,\n", - " 'high': 1,\n", - " 'plane': 1,\n", - " 'dignity': 2,\n", - " 'discipline': 1,\n", - " 'allow': 2,\n", - " 'creative': 2,\n", - " 'protest': 1,\n", - " 'degenerate': 1,\n", - " 'physical': 2,\n", - " 'violence': 1,\n", - " 'majestic': 1,\n", - " 'heights': 1,\n", - " 'meeting': 1,\n", - " 'force': 2,\n", - " 'soul': 1,\n", - " 'marvelous': 1,\n", - " 'new': 5,\n", - " 'militancy': 1,\n", - " 'engulfed': 1,\n", - " 'community': 1,\n", - " 'lead': 1,\n", - " 'distrust': 1,\n", - " 'brothers': 2,\n", - " 'evidenced': 1,\n", - " 'presence': 1,\n", - " 'realize': 2,\n", - " 'destiny': 2,\n", - " 'tied': 1,\n", - " 'inextricably': 1,\n", - " 'bound': 1,\n", - " 'walk': 2,\n", - " 'pledge': 1,\n", - " 'shall': 5,\n", - " 'march': 1,\n", - " 'ahead': 1,\n", - " 'turn': 1,\n", - " 'asking': 1,\n", - " 'devotees': 1,\n", - " 'civil': 1,\n", - " 'satisfied?': 1,\n", - " 'satisfied': 7,\n", - " 'victim': 1,\n", - " 'unspeakable': 1,\n", - " 'horrors': 1,\n", - " 'police': 2,\n", - " 'brutality': 2,\n", - " 'bodies': 1,\n", - " 'heavy': 1,\n", - " 'fatigue': 1,\n", - " 'travel': 1,\n", - " 'gain': 1,\n", - " 'lodging': 1,\n", - " 'motels': 1,\n", - " 'highways': 1,\n", - " 'hotels': 1,\n", - " 'cities': 2,\n", - " 'basic': 1,\n", - " 'mobility': 1,\n", - " 'smaller': 1,\n", - " 'ghetto': 1,\n", - " 'larger': 1,\n", - " 'stripped': 1,\n", - " 'selfhood': 1,\n", - " 'robbed': 1,\n", - " 'signs': 1,\n", - " 'stating': 1,\n", - " 'whites': 1,\n", - " 'mississippi': 4,\n", - " 'vote': 2,\n", - " 'york': 2,\n", - " 'believes': 1,\n", - " 'rolls': 1,\n", - " 'like': 2,\n", - " 'waters': 1,\n", - " 'righteousness': 1,\n", - " 'mighty': 2,\n", - " 'stream': 1,\n", - " 'unmindful': 1,\n", - " 'trials': 1,\n", - " 'tribulations': 1,\n", - " 'fresh': 1,\n", - " 'narrow': 1,\n", - " 'jail': 2,\n", - " 'cells': 1,\n", - " 'areas': 1,\n", - " 'quest': 2,\n", - " 'left': 1,\n", - " 'battered': 1,\n", - " 'storms': 1,\n", - " 'persecution': 1,\n", - " 'staggered': 1,\n", - " 'winds': 1,\n", - " 'veterans': 1,\n", - " 'suffering': 2,\n", - " 'work': 2,\n", - " 'faith': 5,\n", - " 'unearned': 1,\n", - " 'redemptive': 1,\n", - " 'alabama': 3,\n", - " 'south': 2,\n", - " 'carolina': 1,\n", - " 'georgia': 3,\n", - " 'louisiana': 1,\n", - " 'slums': 1,\n", - " 'ghettos': 1,\n", - " 'northern': 1,\n", - " 'knowing': 2,\n", - " 'situation': 1,\n", - " 'changed': 1,\n", - " 'wallow': 1,\n", - " 'despair': 2,\n", - " 'friends': 1,\n", - " 'face': 1,\n", - " 'difficulties': 1,\n", - " 'tomorrow': 1,\n", - " 'dream': 11,\n", - " 'deeply': 1,\n", - " 'rooted': 1,\n", - " 'live': 2,\n", - " 'true': 2,\n", - " 'meaning': 2,\n", - " 'creed': 1,\n", - " 'hold': 1,\n", - " 'truths': 1,\n", - " 'selfevident': 1,\n", - " 'created': 1,\n", - " 'equal': 1,\n", - " 'red': 1,\n", - " 'hills': 1,\n", - " 'sons': 2,\n", - " 'slave': 1,\n", - " 'owners': 1,\n", - " 'able': 8,\n", - " 'sit': 1,\n", - " 'table': 1,\n", - " 'state': 3,\n", - " 'heat': 2,\n", - " 'oppression': 1,\n", - " 'transformed': 1,\n", - " 'oasis': 1,\n", - " 'little': 3,\n", - " 'judged': 1,\n", - " 'skin': 1,\n", - " 'character': 1,\n", - " 'vicious': 1,\n", - " 'racists': 1,\n", - " 'governor': 1,\n", - " 'having': 1,\n", - " 'lips': 1,\n", - " 'dripping': 1,\n", - " 'interposition': 1,\n", - " 'nullification': 1,\n", - " 'right': 1,\n", - " 'boys': 2,\n", - " 'girls': 2,\n", - " 'hands': 2,\n", - " 'sisters': 1,\n", - " 'exalted': 1,\n", - " 'hill': 2,\n", - " 'mountain': 4,\n", - " 'low': 1,\n", - " 'rough': 1,\n", - " 'places': 2,\n", - " 'plain': 1,\n", - " 'crooked': 1,\n", - " 'straight': 1,\n", - " 'glory': 1,\n", - " 'lord': 1,\n", - " 'revealed': 1,\n", - " 'flesh': 1,\n", - " 'hew': 1,\n", - " 'stone': 2,\n", - " 'transform': 1,\n", - " 'jangling': 1,\n", - " 'discords': 1,\n", - " 'beautiful': 1,\n", - " 'symphony': 1,\n", - " 'pray': 1,\n", - " 'sing': 3,\n", - " 'country': 1,\n", - " \"'tis\": 1,\n", - " 'thee': 2,\n", - " 'sweet': 1,\n", - " 'fathers': 1,\n", - " 'died': 1,\n", - " \"pilgrim's\": 1,\n", - " 'pride': 1,\n", - " 'mountainside': 2,\n", - " 'ring': 12,\n", - " 'prodigious': 1,\n", - " 'hilltops': 1,\n", - " 'hampshire': 1,\n", - " 'mountains': 1,\n", - " 'heightening': 1,\n", - " 'alleghenies': 1,\n", - " 'pennsylvania': 1,\n", - " 'snowcapped': 1,\n", - " 'rockies': 1,\n", - " 'colorado': 1,\n", - " 'curvaceous': 1,\n", - " 'slopes': 1,\n", - " 'california': 1,\n", - " 'lookout': 1,\n", - " 'tennessee': 1,\n", - " 'molehill': 1,\n", - " 'happens': 1,\n", - " 'village': 1,\n", - " 'hamlet': 1,\n", - " 'city': 1,\n", - " 'speed': 1,\n", - " 'jews': 1,\n", - " 'gentiles': 1,\n", - " 'protestants': 1,\n", - " 'catholics': 1,\n", - " 'old': 1,\n", - " 'spiritual': 1,\n", - " 'thank': 1,\n", - " 'god': 1,\n", - " 'almighty': 1}" - ] - }, - "execution_count": 19, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "counts" - ] - }, - { - "cell_type": "code", - "execution_count": 20, - "id": "57359e42", - "metadata": {}, - "outputs": [ - { - "data": { - "text/plain": [ - "14" - ] - }, - "execution_count": 20, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "top_counts = sorted(counts.values(), reverse=True)\n", - "\n", - "top_counts_unique = []\n", - "for i in top_counts:\n", - " if i not in top_counts_unique:\n", - " top_counts_unique.append(i)\n", - "len(top_counts_unique)" - ] - }, - { - "cell_type": "code", - "execution_count": 21, - "id": "ee69113b", - "metadata": {}, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "freedom 20\n", - "negro 13\n", - "let 13\n", - "day 12\n", - "ring 12\n", - "dream 11\n", - "nation 10\n", - "come 10\n", - "today 9\n", - "justice 8\n", - "able 8\n", - "satisfied 7\n", - "long 6\n", - "men 6\n", - "white 6\n", - "years 5\n", - "great 5\n", - "free 5\n", - "check 5\n", - "america 5\n", - "time 5\n", - "children 5\n", - "new 5\n", - "shall 5\n", - "faith 5\n", - "american 4\n", - "hope 4\n", - "later 4\n", - "land 4\n", - "black 4\n", - "mississippi 4\n", - "mountain 4\n", - "join 3\n", - "stand 3\n", - "injustice 3\n", - "we've 3\n", - "words 3\n", - "note 3\n", - "rights 3\n", - "people 3\n", - "make 3\n", - "rise 3\n", - "valley 3\n", - "brotherhood 3\n", - "god's 3\n", - "sweltering 3\n", - "alabama 3\n", - "georgia 3\n", - "state 3\n", - "little 3\n", - "sing 3\n", - "history 2\n", - "came 2\n", - "slaves 2\n", - "end 2\n", - "life 2\n", - "segregation 2\n", - "cash 2\n", - "promissory 2\n", - "liberty 2\n", - "color 2\n", - "insufficient 2\n", - "funds 2\n", - "refuse 2\n", - "believe 2\n", - "urgency 2\n", - "racial 2\n", - "negro's 2\n", - "content 2\n", - "continue 2\n", - "say 2\n", - "struggle 2\n", - "dignity 2\n", - "allow 2\n", - "creative 2\n", - "physical 2\n", - "force 2\n", - "brothers 2\n", - "realize 2\n", - "destiny 2\n", - "walk 2\n", - "police 2\n", - "brutality 2\n", - "cities 2\n", - "vote 2\n", - "york 2\n", - "like 2\n", - "mighty 2\n", - "jail 2\n", - "quest 2\n", - "suffering 2\n", - "work 2\n", - "south 2\n", - "knowing 2\n", - "despair 2\n", - "live 2\n", - "true 2\n", - "meaning 2\n", - "sons 2\n", - "heat 2\n", - "boys 2\n", - "girls 2\n", - "hands 2\n", - "hill 2\n", - "places 2\n", - "stone 2\n", - "thee 2\n", - "mountainside 2\n", - "happy 1\n", - "greatest 1\n", - "demonstration 1\n", - "score 1\n", - "ago 1\n", - "symbolic 1\n", - "shadow 1\n", - "signed 1\n", - "emancipation 1\n", - "proclamation 1\n", - "momentous 1\n", - "decree 1\n", - "beacon 1\n", - "light 1\n", - "millions 1\n", - "seared 1\n", - "flames 1\n", - "withering 1\n", - "joyous 1\n", - "daybreak 1\n", - "night 1\n", - "captivity 1\n", - "sadly 1\n", - "crippled 1\n", - "manacles 1\n", - "chains 1\n", - "discrimination 1\n", - "lives 1\n", - "lonely 1\n", - "island 1\n", - "poverty 1\n", - "midst 1\n", - "vast 1\n", - "ocean 1\n", - "material 1\n", - "prosperity 1\n", - "languished 1\n", - "corners 1\n", - "society 1\n", - "finds 1\n", - "exile 1\n", - "dramatize 1\n", - "shameful 1\n", - "condition 1\n", - "sense 1\n", - "nation's 1\n", - "capital 1\n", - "architects 1\n", - "republic 1\n", - "wrote 1\n", - "magnificent 1\n", - "constitution 1\n", - "declaration 1\n", - "independence 1\n", - "signing 1\n", - "fall 1\n", - "heir 1\n", - "promise 1\n", - "yes 1\n", - "guaranteed 1\n", - "unalienable 1\n", - "pursuit 1\n", - "happiness 1\n", - "obvious 1\n", - "defaulted 1\n", - "insofar 1\n", - "citizens 1\n", - "concerned 1\n", - "instead 1\n", - "honoring 1\n", - "sacred 1\n", - "obligation 1\n", - "given 1\n", - "bad 1\n", - "marked 1\n", - "bank 1\n", - "bankrupt 1\n", - "vaults 1\n", - "opportunity 1\n", - "demand 1\n", - "riches 1\n", - "security 1\n", - "hallowed 1\n", - "spot 1\n", - "remind 1\n", - "fierce 1\n", - "engage 1\n", - "luxury 1\n", - "cooling 1\n", - "tranquilizing 1\n", - "drug 1\n", - "gradualism 1\n", - "real 1\n", - "promises 1\n", - "democracy 1\n", - "dark 1\n", - "desolate 1\n", - "sunlit 1\n", - "path 1\n", - "lift 1\n", - "quicksands 1\n", - "solid 1\n", - "rock 1\n", - "reality 1\n", - "fatal 1\n", - "overlook 1\n", - "moment 1\n", - "summer 1\n", - "legitimate 1\n", - "discontent 1\n", - "pass 1\n", - "invigorating 1\n", - "autumn 1\n", - "equality 1\n", - "nineteen 1\n", - "sixtythree 1\n", - "beginning 1\n", - "needed 1\n", - "blow 1\n", - "steam 1\n", - "rude 1\n", - "awakening 1\n", - "returns 1\n", - "business 1\n", - "usual 1\n", - "rest 1\n", - "tranquility 1\n", - "granted 1\n", - "citizenship 1\n", - "whirlwinds 1\n", - "revolt 1\n", - "shake 1\n", - "foundations 1\n", - "bright 1\n", - "emerges 1\n", - "warm 1\n", - "threshold 1\n", - "leads 1\n", - "palace 1\n", - "process 1\n", - "gaining 1\n", - "rightful 1\n", - "place 1\n", - "guilty 1\n", - "wrongful 1\n", - "deeds 1\n", - "seek 1\n", - "satisfy 1\n", - "thirst 1\n", - "drinking 1\n", - "cup 1\n", - "bitterness 1\n", - "hatred 1\n", - "forever 1\n", - "conduct 1\n", - "high 1\n", - "plane 1\n", - "discipline 1\n", - "protest 1\n", - "degenerate 1\n", - "violence 1\n", - "majestic 1\n", - "heights 1\n", - "meeting 1\n", - "soul 1\n", - "marvelous 1\n", - "militancy 1\n", - "engulfed 1\n", - "community 1\n", - "lead 1\n", - "distrust 1\n", - "evidenced 1\n", - "presence 1\n", - "tied 1\n", - "inextricably 1\n", - "bound 1\n", - "pledge 1\n", - "march 1\n", - "ahead 1\n", - "turn 1\n", - "asking 1\n", - "devotees 1\n", - "civil 1\n", - "satisfied? 1\n", - "victim 1\n", - "unspeakable 1\n", - "horrors 1\n", - "bodies 1\n", - "heavy 1\n", - "fatigue 1\n", - "travel 1\n", - "gain 1\n", - "lodging 1\n", - "motels 1\n", - "highways 1\n", - "hotels 1\n", - "basic 1\n", - "mobility 1\n", - "smaller 1\n", - "ghetto 1\n", - "larger 1\n", - "stripped 1\n", - "selfhood 1\n", - "robbed 1\n", - "signs 1\n", - "stating 1\n", - "whites 1\n", - "believes 1\n", - "rolls 1\n", - "waters 1\n", - "righteousness 1\n", - "stream 1\n", - "unmindful 1\n", - "trials 1\n", - "tribulations 1\n", - "fresh 1\n", - "narrow 1\n", - "cells 1\n", - "areas 1\n", - "left 1\n", - "battered 1\n", - "storms 1\n", - "persecution 1\n", - "staggered 1\n", - "winds 1\n", - "veterans 1\n", - "unearned 1\n", - "redemptive 1\n", - "carolina 1\n", - "louisiana 1\n", - "slums 1\n", - "ghettos 1\n", - "northern 1\n", - "situation 1\n", - "changed 1\n", - "wallow 1\n", - "friends 1\n", - "face 1\n", - "difficulties 1\n", - "tomorrow 1\n", - "deeply 1\n", - "rooted 1\n", - "creed 1\n", - "hold 1\n", - "truths 1\n", - "selfevident 1\n", - "created 1\n", - "equal 1\n", - "red 1\n", - "hills 1\n", - "slave 1\n", - "owners 1\n", - "sit 1\n", - "table 1\n", - "oppression 1\n", - "transformed 1\n", - "oasis 1\n", - "judged 1\n", - "skin 1\n", - "character 1\n", - "vicious 1\n", - "racists 1\n", - "governor 1\n", - "having 1\n", - "lips 1\n", - "dripping 1\n", - "interposition 1\n", - "nullification 1\n", - "right 1\n", - "sisters 1\n", - "exalted 1\n", - "low 1\n", - "rough 1\n", - "plain 1\n", - "crooked 1\n", - "straight 1\n", - "glory 1\n", - "lord 1\n", - "revealed 1\n", - "flesh 1\n", - "hew 1\n", - "transform 1\n", - "jangling 1\n", - "discords 1\n", - "beautiful 1\n", - "symphony 1\n", - "pray 1\n", - "country 1\n", - "'tis 1\n", - "sweet 1\n", - "fathers 1\n", - "died 1\n", - "pilgrim's 1\n", - "pride 1\n", - "prodigious 1\n", - "hilltops 1\n", - "hampshire 1\n", - "mountains 1\n", - "heightening 1\n", - "alleghenies 1\n", - "pennsylvania 1\n", - "snowcapped 1\n", - "rockies 1\n", - "colorado 1\n", - "curvaceous 1\n", - "slopes 1\n", - "california 1\n", - "lookout 1\n", - "tennessee 1\n", - "molehill 1\n", - "happens 1\n", - "village 1\n", - "hamlet 1\n", - "city 1\n", - "speed 1\n", - "jews 1\n", - "gentiles 1\n", - "protestants 1\n", - "catholics 1\n", - "old 1\n", - "spiritual 1\n", - "thank 1\n", - "god 1\n", - "almighty 1\n" - ] - } - ], - "source": [ - "for i in top_counts_unique:\n", - " for ck, cv in counts.items():\n", - " if cv == i:\n", - " print(ck, cv)" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "eee3f73b", - "metadata": {}, - "outputs": [], - "source": [] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "a8ccf54e-d958-40e7-af3d-21e59c222e3b", - "metadata": {}, - "outputs": [], - "source": [] - } - ], - "metadata": { - "kernelspec": { - "display_name": "Python 3 (ipykernel)", - "language": "python", - "name": "python3" - }, - "language_info": { - "codemirror_mode": { - "name": "ipython", - "version": 3 - }, - "file_extension": ".py", - "mimetype": "text/x-python", - "name": "python", - "nbconvert_exporter": "python", - "pygments_lexer": "ipython3", - "version": "3.11.7" - } - }, - "nbformat": 4, - "nbformat_minor": 5 -}