diff --git a/Phase 3 - 2020 (Summer)/Week 2 (Apr 5 - Apr 11)/Exercise1/.ipynb_checkpoints/exercise1-checkpoint.ipynb b/Phase 3 - 2020 (Summer)/Week 2 (Apr 5 - Apr 11)/Exercise1/.ipynb_checkpoints/exercise1-checkpoint.ipynb
new file mode 100644
index 000000000..c9184cd8f
--- /dev/null
+++ b/Phase 3 - 2020 (Summer)/Week 2 (Apr 5 - Apr 11)/Exercise1/.ipynb_checkpoints/exercise1-checkpoint.ipynb
@@ -0,0 +1,1302 @@
+{
+ "cells": [
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "# Programming Exercise 1: Linear Regression\n",
+ "\n",
+ "## Introduction\n",
+ "\n",
+ "In this exercise, you will implement linear regression and get to see it work on data. Before starting on this programming exercise, we strongly recommend watching the video lectures and completing the review questions for the associated topics.\n",
+ "\n",
+ "All the information you need for solving this assignment is in this notebook, and all the code you will be implementing will take place within this notebook. The assignment can be promptly submitted to the coursera grader directly from this notebook (code and instructions are included below).\n",
+ "\n",
+ "Before we begin with the exercises, we need to import all libraries required for this programming exercise. Throughout the course, we will be using [`numpy`](http://www.numpy.org/) for all arrays and matrix operations, and [`matplotlib`](https://matplotlib.org/) for plotting.\n",
+ "\n",
+ "You can find instructions on how to install required libraries in the README file in the [github repository](https://github.com/dibgerge/ml-coursera-python-assignments)."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "# used for manipulating directory paths\n",
+ "import os\n",
+ "\n",
+ "# Scientific and vector computation for python\n",
+ "import numpy as np\n",
+ "\n",
+ "# Plotting library\n",
+ "from matplotlib import pyplot\n",
+ "from mpl_toolkits.mplot3d import Axes3D # needed to plot 3-D surfaces\n",
+ "\n",
+ "# library written for this exercise providing additional functions for assignment submission, and others\n",
+ "import utils \n",
+ "\n",
+ "# define the submission/grader object for this exercise\n",
+ "grader = utils.Grader()\n",
+ "\n",
+ "# tells matplotlib to embed plots within the notebook\n",
+ "%matplotlib inline"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "## Submission and Grading\n",
+ "\n",
+ "After completing each part of the assignment, be sure to submit your solutions to the grader.\n",
+ "\n",
+ "For this programming exercise, you are only required to complete the first part of the exercise to implement linear regression with one variable. The second part of the exercise, which is optional, covers linear regression with multiple variables. The following is a breakdown of how each part of this exercise is scored.\n",
+ "\n",
+ "**Required Exercises**\n",
+ "\n",
+ "| Section | Part |Submitted Function | Points \n",
+ "|---------|:- |:- | :-: \n",
+ "| 1 | [Warm up exercise](#section1) | [`warmUpExercise`](#warmUpExercise) | 10 \n",
+ "| 2 | [Compute cost for one variable](#section2) | [`computeCost`](#computeCost) | 20 \n",
+ "| 3 | [Gradient descent for one variable](#section3) | [`gradientDescent`](#gradientDescent) | 20 \n",
+ "| 4 | [Feature normalization](#section4) | [`featureNormalize`](#featureNormalize) | 10 |\n",
+ "| 5 | [Compute cost for multiple variables](#section5) | [`computeCostMulti`](#computeCostMulti) | 20 |\n",
+ "| 6 | [Gradient descent for multiple variables](#section5) | [`gradientDescentMulti`](#gradientDescentMulti) |10 |\n",
+ "| 7 | [Normal Equations](#section7) | [`normalEqn`](#normalEqn) | 10 |\n",
+ "| | Total Points | | 100 \n",
+ "\n",
+ "You are allowed to submit your solutions multiple times, and we will take only the highest score into consideration.\n",
+ "\n",
+ "
\n",
+ "At the end of each section in this notebook, we have a cell which contains code for submitting the solutions thus far to the grader. Execute the cell to see your score up to the current section. For all your work to be submitted properly, you must execute those cells at least once. They must also be re-executed everytime the submitted function is updated.\n",
+ "
\n",
+ "\n",
+ "\n",
+ "## Debugging\n",
+ "\n",
+ "Here are some things to keep in mind throughout this exercise:\n",
+ "\n",
+ "- Python array indices start from zero, not one (contrary to OCTAVE/MATLAB). \n",
+ "\n",
+ "- There is an important distinction between python arrays (called `list` or `tuple`) and `numpy` arrays. You should use `numpy` arrays in all your computations. Vector/matrix operations work only with `numpy` arrays. Python lists do not support vector operations (you need to use for loops).\n",
+ "\n",
+ "- If you are seeing many errors at runtime, inspect your matrix operations to make sure that you are adding and multiplying matrices of compatible dimensions. Printing the dimensions of `numpy` arrays using the `shape` property will help you debug.\n",
+ "\n",
+ "- By default, `numpy` interprets math operators to be element-wise operators. If you want to do matrix multiplication, you need to use the `dot` function in `numpy`. For, example if `A` and `B` are two `numpy` matrices, then the matrix operation AB is `np.dot(A, B)`. Note that for 2-dimensional matrices or vectors (1-dimensional), this is also equivalent to `A@B` (requires python >= 3.5)."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "\n",
+ "## 1 Simple python and `numpy` function\n",
+ "\n",
+ "The first part of this assignment gives you practice with python and `numpy` syntax and the homework submission process. In the next cell, you will find the outline of a `python` function. Modify it to return a 5 x 5 identity matrix by filling in the following code:\n",
+ "\n",
+ "```python\n",
+ "A = np.eye(5)\n",
+ "```\n",
+ ""
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "def warmUpExercise():\n",
+ " \"\"\"\n",
+ " Example function in Python which computes the identity matrix.\n",
+ " \n",
+ " Returns\n",
+ " -------\n",
+ " A : array_like\n",
+ " The 5x5 identity matrix.\n",
+ " \n",
+ " Instructions\n",
+ " ------------\n",
+ " Return the 5x5 identity matrix.\n",
+ " \"\"\" \n",
+ " # ======== YOUR CODE HERE ======\n",
+ " A = [] # modify this line\n",
+ " \n",
+ " # ==============================\n",
+ " return A"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "The previous cell only defines the function `warmUpExercise`. We can now run it by executing the following cell to see its output. You should see output similar to the following:\n",
+ "\n",
+ "```python\n",
+ "array([[ 1., 0., 0., 0., 0.],\n",
+ " [ 0., 1., 0., 0., 0.],\n",
+ " [ 0., 0., 1., 0., 0.],\n",
+ " [ 0., 0., 0., 1., 0.],\n",
+ " [ 0., 0., 0., 0., 1.]])\n",
+ "```"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "warmUpExercise()"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "### 1.1 Submitting solutions\n",
+ "\n",
+ "After completing a part of the exercise, you can submit your solutions for grading by first adding the function you modified to the grader object, and then sending your function to Coursera for grading. \n",
+ "\n",
+ "The grader will prompt you for your login e-mail and submission token. You can obtain a submission token from the web page for the assignment. You are allowed to submit your solutions multiple times, and we will take only the highest score into consideration.\n",
+ "\n",
+ "Execute the next cell to grade your solution to the first part of this exercise.\n",
+ "\n",
+ "*You should now submit your solutions.*"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "# appends the implemented function in part 1 to the grader object\n",
+ "grader[1] = warmUpExercise\n",
+ "\n",
+ "# send the added functions to coursera grader for getting a grade on this part\n",
+ "grader.grade()"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "## 2 Linear regression with one variable\n",
+ "\n",
+ "Now you will implement linear regression with one variable to predict profits for a food truck. Suppose you are the CEO of a restaurant franchise and are considering different cities for opening a new outlet. The chain already has trucks in various cities and you have data for profits and populations from the cities. You would like to use this data to help you select which city to expand to next. \n",
+ "\n",
+ "The file `Data/ex1data1.txt` contains the dataset for our linear regression problem. The first column is the population of a city (in 10,000s) and the second column is the profit of a food truck in that city (in $10,000s). A negative value for profit indicates a loss. \n",
+ "\n",
+ "We provide you with the code needed to load this data. The dataset is loaded from the data file into the variables `x` and `y`:"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "# Read comma separated data\n",
+ "data = np.loadtxt(os.path.join('Data', 'ex1data1.txt'), delimiter=',')\n",
+ "X, y = data[:, 0], data[:, 1]\n",
+ "\n",
+ "m = y.size # number of training examples"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "### 2.1 Plotting the Data\n",
+ "\n",
+ "Before starting on any task, it is often useful to understand the data by visualizing it. For this dataset, you can use a scatter plot to visualize the data, since it has only two properties to plot (profit and population). Many other problems that you will encounter in real life are multi-dimensional and cannot be plotted on a 2-d plot. There are many plotting libraries in python (see this [blog post](https://blog.modeanalytics.com/python-data-visualization-libraries/) for a good summary of the most popular ones). \n",
+ "\n",
+ "In this course, we will be exclusively using `matplotlib` to do all our plotting. `matplotlib` is one of the most popular scientific plotting libraries in python and has extensive tools and functions to make beautiful plots. `pyplot` is a module within `matplotlib` which provides a simplified interface to `matplotlib`'s most common plotting tasks, mimicking MATLAB's plotting interface.\n",
+ "\n",
+ "
\n",
+ "You might have noticed that we have imported the `pyplot` module at the beginning of this exercise using the command `from matplotlib import pyplot`. This is rather uncommon, and if you look at python code elsewhere or in the `matplotlib` tutorials, you will see that the module is named `plt`. This is used by module renaming by using the import command `import matplotlib.pyplot as plt`. We will not using the short name of `pyplot` module in this class exercises, but you should be aware of this deviation from norm.\n",
+ "
\n",
+ "\n",
+ "\n",
+ "In the following part, your first job is to complete the `plotData` function below. Modify the function and fill in the following code:\n",
+ "\n",
+ "```python\n",
+ " pyplot.plot(x, y, 'ro', ms=10, mec='k')\n",
+ " pyplot.ylabel('Profit in $10,000')\n",
+ " pyplot.xlabel('Population of City in 10,000s')\n",
+ "```"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "def plotData(x, y):\n",
+ " \"\"\"\n",
+ " Plots the data points x and y into a new figure. Plots the data \n",
+ " points and gives the figure axes labels of population and profit.\n",
+ " \n",
+ " Parameters\n",
+ " ----------\n",
+ " x : array_like\n",
+ " Data point values for x-axis.\n",
+ "\n",
+ " y : array_like\n",
+ " Data point values for y-axis. Note x and y should have the same size.\n",
+ " \n",
+ " Instructions\n",
+ " ------------\n",
+ " Plot the training data into a figure using the \"figure\" and \"plot\"\n",
+ " functions. Set the axes labels using the \"xlabel\" and \"ylabel\" functions.\n",
+ " Assume the population and revenue data have been passed in as the x\n",
+ " and y arguments of this function. \n",
+ " \n",
+ " Hint\n",
+ " ----\n",
+ " You can use the 'ro' option with plot to have the markers\n",
+ " appear as red circles. Furthermore, you can make the markers larger by\n",
+ " using plot(..., 'ro', ms=10), where `ms` refers to marker size. You \n",
+ " can also set the marker edge color using the `mec` property.\n",
+ " \"\"\"\n",
+ " fig = pyplot.figure() # open a new figure\n",
+ " \n",
+ " # ====================== YOUR CODE HERE ======================= \n",
+ " \n",
+ "\n",
+ " # =============================================================\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Now run the defined function with the loaded data to visualize the data. The end result should look like the following figure:\n",
+ "\n",
+ "\n",
+ "\n",
+ "Execute the next cell to visualize the data."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "plotData(X, y)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "To quickly learn more about the `matplotlib` plot function and what arguments you can provide to it, you can type `?pyplot.plot` in a cell within the jupyter notebook. This opens a separate page showing the documentation for the requested function. You can also search online for plotting documentation. \n",
+ "\n",
+ "To set the markers to red circles, we used the option `'or'` within the `plot` function."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "?pyplot.plot"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "\n",
+ "### 2.2 Gradient Descent\n",
+ "\n",
+ "In this part, you will fit the linear regression parameters $\\theta$ to our dataset using gradient descent.\n",
+ "\n",
+ "#### 2.2.1 Update Equations\n",
+ "\n",
+ "The objective of linear regression is to minimize the cost function\n",
+ "\n",
+ "$$ J(\\theta) = \\frac{1}{2m} \\sum_{i=1}^m \\left( h_{\\theta}(x^{(i)}) - y^{(i)}\\right)^2$$\n",
+ "\n",
+ "where the hypothesis $h_\\theta(x)$ is given by the linear model\n",
+ "$$ h_\\theta(x) = \\theta^Tx = \\theta_0 + \\theta_1 x_1$$\n",
+ "\n",
+ "Recall that the parameters of your model are the $\\theta_j$ values. These are\n",
+ "the values you will adjust to minimize cost $J(\\theta)$. One way to do this is to\n",
+ "use the batch gradient descent algorithm. In batch gradient descent, each\n",
+ "iteration performs the update\n",
+ "\n",
+ "$$ \\theta_j = \\theta_j - \\alpha \\frac{1}{m} \\sum_{i=1}^m \\left( h_\\theta(x^{(i)}) - y^{(i)}\\right)x_j^{(i)} \\qquad \\text{simultaneously update } \\theta_j \\text{ for all } j$$\n",
+ "\n",
+ "With each step of gradient descent, your parameters $\\theta_j$ come closer to the optimal values that will achieve the lowest cost J($\\theta$).\n",
+ "\n",
+ "
\n",
+ "**Implementation Note:** We store each example as a row in the the $X$ matrix in Python `numpy`. To take into account the intercept term ($\\theta_0$), we add an additional first column to $X$ and set it to all ones. This allows us to treat $\\theta_0$ as simply another 'feature'.\n",
+ "
\n",
+ "\n",
+ "\n",
+ "#### 2.2.2 Implementation\n",
+ "\n",
+ "We have already set up the data for linear regression. In the following cell, we add another dimension to our data to accommodate the $\\theta_0$ intercept term. Do NOT execute this cell more than once."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "# Add a column of ones to X. The numpy function stack joins arrays along a given axis. \n",
+ "# The first axis (axis=0) refers to rows (training examples) \n",
+ "# and second axis (axis=1) refers to columns (features).\n",
+ "X = np.stack([np.ones(m), X], axis=1)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "\n",
+ "#### 2.2.3 Computing the cost $J(\\theta)$\n",
+ "\n",
+ "As you perform gradient descent to learn minimize the cost function $J(\\theta)$, it is helpful to monitor the convergence by computing the cost. In this section, you will implement a function to calculate $J(\\theta)$ so you can check the convergence of your gradient descent implementation. \n",
+ "\n",
+ "Your next task is to complete the code for the function `computeCost` which computes $J(\\theta)$. As you are doing this, remember that the variables $X$ and $y$ are not scalar values. $X$ is a matrix whose rows represent the examples from the training set and $y$ is a vector whose each elemennt represent the value at a given row of $X$.\n",
+ ""
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "def computeCost(X, y, theta):\n",
+ " \"\"\"\n",
+ " Compute cost for linear regression. Computes the cost of using theta as the\n",
+ " parameter for linear regression to fit the data points in X and y.\n",
+ " \n",
+ " Parameters\n",
+ " ----------\n",
+ " X : array_like\n",
+ " The input dataset of shape (m x n+1), where m is the number of examples,\n",
+ " and n is the number of features. We assume a vector of one's already \n",
+ " appended to the features so we have n+1 columns.\n",
+ " \n",
+ " y : array_like\n",
+ " The values of the function at each data point. This is a vector of\n",
+ " shape (m, ).\n",
+ " \n",
+ " theta : array_like\n",
+ " The parameters for the regression function. This is a vector of \n",
+ " shape (n+1, ).\n",
+ " \n",
+ " Returns\n",
+ " -------\n",
+ " J : float\n",
+ " The value of the regression cost function.\n",
+ " \n",
+ " Instructions\n",
+ " ------------\n",
+ " Compute the cost of a particular choice of theta. \n",
+ " You should set J to the cost.\n",
+ " \"\"\"\n",
+ " \n",
+ " # initialize some useful values\n",
+ " m = y.size # number of training examples\n",
+ " \n",
+ " # You need to return the following variables correctly\n",
+ " J = 0\n",
+ " \n",
+ " # ====================== YOUR CODE HERE =====================\n",
+ "\n",
+ " \n",
+ " # ===========================================================\n",
+ " return J"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Once you have completed the function, the next step will run `computeCost` two times using two different initializations of $\\theta$. You will see the cost printed to the screen."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "J = computeCost(X, y, theta=np.array([0.0, 0.0]))\n",
+ "print('With theta = [0, 0] \\nCost computed = %.2f' % J)\n",
+ "print('Expected cost value (approximately) 32.07\\n')\n",
+ "\n",
+ "# further testing of the cost function\n",
+ "J = computeCost(X, y, theta=np.array([-1, 2]))\n",
+ "print('With theta = [-1, 2]\\nCost computed = %.2f' % J)\n",
+ "print('Expected cost value (approximately) 54.24')"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "*You should now submit your solutions by executing the following cell.*"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "grader[2] = computeCost\n",
+ "grader.grade()"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "\n",
+ "#### 2.2.4 Gradient descent\n",
+ "\n",
+ "Next, you will complete a function which implements gradient descent.\n",
+ "The loop structure has been written for you, and you only need to supply the updates to $\\theta$ within each iteration. \n",
+ "\n",
+ "As you program, make sure you understand what you are trying to optimize and what is being updated. Keep in mind that the cost $J(\\theta)$ is parameterized by the vector $\\theta$, not $X$ and $y$. That is, we minimize the value of $J(\\theta)$ by changing the values of the vector $\\theta$, not by changing $X$ or $y$. [Refer to the equations in this notebook](#section2) and to the video lectures if you are uncertain. A good way to verify that gradient descent is working correctly is to look at the value of $J(\\theta)$ and check that it is decreasing with each step. \n",
+ "\n",
+ "The starter code for the function `gradientDescent` calls `computeCost` on every iteration and saves the cost to a `python` list. Assuming you have implemented gradient descent and `computeCost` correctly, your value of $J(\\theta)$ should never increase, and should converge to a steady value by the end of the algorithm.\n",
+ "\n",
+ "
\n",
+ "**Vectors and matrices in `numpy`** - Important implementation notes\n",
+ "\n",
+ "A vector in `numpy` is a one dimensional array, for example `np.array([1, 2, 3])` is a vector. A matrix in `numpy` is a two dimensional array, for example `np.array([[1, 2, 3], [4, 5, 6]])`. However, the following is still considered a matrix `np.array([[1, 2, 3]])` since it has two dimensions, even if it has a shape of 1x3 (which looks like a vector).\n",
+ "\n",
+ "Given the above, the function `np.dot` which we will use for all matrix/vector multiplication has the following properties:\n",
+ "- It always performs inner products on vectors. If `x=np.array([1, 2, 3])`, then `np.dot(x, x)` is a scalar.\n",
+ "- For matrix-vector multiplication, so if $X$ is a $m\\times n$ matrix and $y$ is a vector of length $m$, then the operation `np.dot(y, X)` considers $y$ as a $1 \\times m$ vector. On the other hand, if $y$ is a vector of length $n$, then the operation `np.dot(X, y)` considers $y$ as a $n \\times 1$ vector.\n",
+ "- A vector can be promoted to a matrix using `y[None]` or `[y[np.newaxis]`. That is, if `y = np.array([1, 2, 3])` is a vector of size 3, then `y[None, :]` is a matrix of shape $1 \\times 3$. We can use `y[:, None]` to obtain a shape of $3 \\times 1$.\n",
+ "
\n",
+ ""
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "def gradientDescent(X, y, theta, alpha, num_iters):\n",
+ " \"\"\"\n",
+ " Performs gradient descent to learn `theta`. Updates theta by taking `num_iters`\n",
+ " gradient steps with learning rate `alpha`.\n",
+ " \n",
+ " Parameters\n",
+ " ----------\n",
+ " X : array_like\n",
+ " The input dataset of shape (m x n+1).\n",
+ " \n",
+ " y : arra_like\n",
+ " Value at given features. A vector of shape (m, ).\n",
+ " \n",
+ " theta : array_like\n",
+ " Initial values for the linear regression parameters. \n",
+ " A vector of shape (n+1, ).\n",
+ " \n",
+ " alpha : float\n",
+ " The learning rate.\n",
+ " \n",
+ " num_iters : int\n",
+ " The number of iterations for gradient descent. \n",
+ " \n",
+ " Returns\n",
+ " -------\n",
+ " theta : array_like\n",
+ " The learned linear regression parameters. A vector of shape (n+1, ).\n",
+ " \n",
+ " J_history : list\n",
+ " A python list for the values of the cost function after each iteration.\n",
+ " \n",
+ " Instructions\n",
+ " ------------\n",
+ " Peform a single gradient step on the parameter vector theta.\n",
+ "\n",
+ " While debugging, it can be useful to print out the values of \n",
+ " the cost function (computeCost) and gradient here.\n",
+ " \"\"\"\n",
+ " # Initialize some useful values\n",
+ " m = y.shape[0] # number of training examples\n",
+ " \n",
+ " # make a copy of theta, to avoid changing the original array, since numpy arrays\n",
+ " # are passed by reference to functions\n",
+ " theta = theta.copy()\n",
+ " \n",
+ " J_history = [] # Use a python list to save cost in every iteration\n",
+ " \n",
+ " for i in range(num_iters):\n",
+ " # ==================== YOUR CODE HERE =================================\n",
+ " \n",
+ "\n",
+ " # =====================================================================\n",
+ " \n",
+ " # save the cost J in every iteration\n",
+ " J_history.append(computeCost(X, y, theta))\n",
+ " \n",
+ " return theta, J_history"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "After you are finished call the implemented `gradientDescent` function and print the computed $\\theta$. We initialize the $\\theta$ parameters to 0 and the learning rate $\\alpha$ to 0.01. Execute the following cell to check your code."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "# initialize fitting parameters\n",
+ "theta = np.zeros(2)\n",
+ "\n",
+ "# some gradient descent settings\n",
+ "iterations = 1500\n",
+ "alpha = 0.01\n",
+ "\n",
+ "theta, J_history = gradientDescent(X ,y, theta, alpha, iterations)\n",
+ "print('Theta found by gradient descent: {:.4f}, {:.4f}'.format(*theta))\n",
+ "print('Expected theta values (approximately): [-3.6303, 1.1664]')"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "We will use your final parameters to plot the linear fit. The results should look like the following figure.\n",
+ "\n",
+ ""
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "# plot the linear fit\n",
+ "plotData(X[:, 1], y)\n",
+ "pyplot.plot(X[:, 1], np.dot(X, theta), '-')\n",
+ "pyplot.legend(['Training data', 'Linear regression']);"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Your final values for $\\theta$ will also be used to make predictions on profits in areas of 35,000 and 70,000 people.\n",
+ "\n",
+ "
\n",
+ "Note the way that the following lines use matrix multiplication, rather than explicit summation or looping, to calculate the predictions. This is an example of code vectorization in `numpy`.\n",
+ "
\n",
+ "\n",
+ "
\n",
+ "Note that the first argument to the `numpy` function `dot` is a python list. `numpy` can internally converts **valid** python lists to numpy arrays when explicitly provided as arguments to `numpy` functions.\n",
+ "
\n"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "# Predict values for population sizes of 35,000 and 70,000\n",
+ "predict1 = np.dot([1, 3.5], theta)\n",
+ "print('For population = 35,000, we predict a profit of {:.2f}\\n'.format(predict1*10000))\n",
+ "\n",
+ "predict2 = np.dot([1, 7], theta)\n",
+ "print('For population = 70,000, we predict a profit of {:.2f}\\n'.format(predict2*10000))"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "*You should now submit your solutions by executing the next cell.*"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "grader[3] = gradientDescent\n",
+ "grader.grade()"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "### 2.4 Visualizing $J(\\theta)$\n",
+ "\n",
+ "To understand the cost function $J(\\theta)$ better, you will now plot the cost over a 2-dimensional grid of $\\theta_0$ and $\\theta_1$ values. You will not need to code anything new for this part, but you should understand how the code you have written already is creating these images.\n",
+ "\n",
+ "In the next cell, the code is set up to calculate $J(\\theta)$ over a grid of values using the `computeCost` function that you wrote. After executing the following cell, you will have a 2-D array of $J(\\theta)$ values. Then, those values are used to produce surface and contour plots of $J(\\theta)$ using the matplotlib `plot_surface` and `contourf` functions. The plots should look something like the following:\n",
+ "\n",
+ "\n",
+ "\n",
+ "The purpose of these graphs is to show you how $J(\\theta)$ varies with changes in $\\theta_0$ and $\\theta_1$. The cost function $J(\\theta)$ is bowl-shaped and has a global minimum. (This is easier to see in the contour plot than in the 3D surface plot). This minimum is the optimal point for $\\theta_0$ and $\\theta_1$, and each step of gradient descent moves closer to this point."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "# grid over which we will calculate J\n",
+ "theta0_vals = np.linspace(-10, 10, 100)\n",
+ "theta1_vals = np.linspace(-1, 4, 100)\n",
+ "\n",
+ "# initialize J_vals to a matrix of 0's\n",
+ "J_vals = np.zeros((theta0_vals.shape[0], theta1_vals.shape[0]))\n",
+ "\n",
+ "# Fill out J_vals\n",
+ "for i, theta0 in enumerate(theta0_vals):\n",
+ " for j, theta1 in enumerate(theta1_vals):\n",
+ " J_vals[i, j] = computeCost(X, y, [theta0, theta1])\n",
+ " \n",
+ "# Because of the way meshgrids work in the surf command, we need to\n",
+ "# transpose J_vals before calling surf, or else the axes will be flipped\n",
+ "J_vals = J_vals.T\n",
+ "\n",
+ "# surface plot\n",
+ "fig = pyplot.figure(figsize=(12, 5))\n",
+ "ax = fig.add_subplot(121, projection='3d')\n",
+ "ax.plot_surface(theta0_vals, theta1_vals, J_vals, cmap='viridis')\n",
+ "pyplot.xlabel('theta0')\n",
+ "pyplot.ylabel('theta1')\n",
+ "pyplot.title('Surface')\n",
+ "\n",
+ "# contour plot\n",
+ "# Plot J_vals as 15 contours spaced logarithmically between 0.01 and 100\n",
+ "ax = pyplot.subplot(122)\n",
+ "pyplot.contour(theta0_vals, theta1_vals, J_vals, linewidths=2, cmap='viridis', levels=np.logspace(-2, 3, 20))\n",
+ "pyplot.xlabel('theta0')\n",
+ "pyplot.ylabel('theta1')\n",
+ "pyplot.plot(theta[0], theta[1], 'ro', ms=10, lw=2)\n",
+ "pyplot.title('Contour, showing minimum')\n",
+ "pass"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "## Optional Exercises\n",
+ "\n",
+ "If you have successfully completed the material above, congratulations! You now understand linear regression and should able to start using it on your own datasets.\n",
+ "\n",
+ "For the rest of this programming exercise, we have included the following optional exercises. These exercises will help you gain a deeper understanding of the material, and if you are able to do so, we encourage you to complete them as well. You can still submit your solutions to these exercises to check if your answers are correct.\n",
+ "\n",
+ "## 3 Linear regression with multiple variables\n",
+ "\n",
+ "In this part, you will implement linear regression with multiple variables to predict the prices of houses. Suppose you are selling your house and you want to know what a good market price would be. One way to do this is to first collect information on recent houses sold and make a model of housing prices.\n",
+ "\n",
+ "The file `Data/ex1data2.txt` contains a training set of housing prices in Portland, Oregon. The first column is the size of the house (in square feet), the second column is the number of bedrooms, and the third column is the price\n",
+ "of the house. \n",
+ "\n",
+ "\n",
+ "### 3.1 Feature Normalization\n",
+ "\n",
+ "We start by loading and displaying some values from this dataset. By looking at the values, note that house sizes are about 1000 times the number of bedrooms. When features differ by orders of magnitude, first performing feature scaling can make gradient descent converge much more quickly."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "# Load data\n",
+ "data = np.loadtxt(os.path.join('Data', 'ex1data2.txt'), delimiter=',')\n",
+ "X = data[:, :2]\n",
+ "y = data[:, 2]\n",
+ "m = y.size\n",
+ "\n",
+ "# print out some data points\n",
+ "print('{:>8s}{:>8s}{:>10s}'.format('X[:,0]', 'X[:, 1]', 'y'))\n",
+ "print('-'*26)\n",
+ "for i in range(10):\n",
+ " print('{:8.0f}{:8.0f}{:10.0f}'.format(X[i, 0], X[i, 1], y[i]))"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Your task here is to complete the code in `featureNormalize` function:\n",
+ "- Subtract the mean value of each feature from the dataset.\n",
+ "- After subtracting the mean, additionally scale (divide) the feature values by their respective “standard deviations.”\n",
+ "\n",
+ "The standard deviation is a way of measuring how much variation there is in the range of values of a particular feature (most data points will lie within ±2 standard deviations of the mean); this is an alternative to taking the range of values (max-min). In `numpy`, you can use the `std` function to compute the standard deviation. \n",
+ "\n",
+ "For example, the quantity `X[:, 0]` contains all the values of $x_1$ (house sizes) in the training set, so `np.std(X[:, 0])` computes the standard deviation of the house sizes.\n",
+ "At the time that the function `featureNormalize` is called, the extra column of 1’s corresponding to $x_0 = 1$ has not yet been added to $X$. \n",
+ "\n",
+ "You will do this for all the features and your code should work with datasets of all sizes (any number of features / examples). Note that each column of the matrix $X$ corresponds to one feature.\n",
+ "\n",
+ "
\n",
+ "**Implementation Note:** When normalizing the features, it is important\n",
+ "to store the values used for normalization - the mean value and the standard deviation used for the computations. After learning the parameters\n",
+ "from the model, we often want to predict the prices of houses we have not\n",
+ "seen before. Given a new x value (living room area and number of bedrooms), we must first normalize x using the mean and standard deviation that we had previously computed from the training set.\n",
+ "
\n",
+ ""
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "def featureNormalize(X):\n",
+ " \"\"\"\n",
+ " Normalizes the features in X. returns a normalized version of X where\n",
+ " the mean value of each feature is 0 and the standard deviation\n",
+ " is 1. This is often a good preprocessing step to do when working with\n",
+ " learning algorithms.\n",
+ " \n",
+ " Parameters\n",
+ " ----------\n",
+ " X : array_like\n",
+ " The dataset of shape (m x n).\n",
+ " \n",
+ " Returns\n",
+ " -------\n",
+ " X_norm : array_like\n",
+ " The normalized dataset of shape (m x n).\n",
+ " \n",
+ " Instructions\n",
+ " ------------\n",
+ " First, for each feature dimension, compute the mean of the feature\n",
+ " and subtract it from the dataset, storing the mean value in mu. \n",
+ " Next, compute the standard deviation of each feature and divide\n",
+ " each feature by it's standard deviation, storing the standard deviation \n",
+ " in sigma. \n",
+ " \n",
+ " Note that X is a matrix where each column is a feature and each row is\n",
+ " an example. You needto perform the normalization separately for each feature. \n",
+ " \n",
+ " Hint\n",
+ " ----\n",
+ " You might find the 'np.mean' and 'np.std' functions useful.\n",
+ " \"\"\"\n",
+ " # You need to set these values correctly\n",
+ " X_norm = X.copy()\n",
+ " mu = np.zeros(X.shape[1])\n",
+ " sigma = np.zeros(X.shape[1])\n",
+ "\n",
+ " # =========================== YOUR CODE HERE =====================\n",
+ "\n",
+ " \n",
+ " # ================================================================\n",
+ " return X_norm, mu, sigma"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Execute the next cell to run the implemented `featureNormalize` function."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "# call featureNormalize on the loaded data\n",
+ "X_norm, mu, sigma = featureNormalize(X)\n",
+ "\n",
+ "print('Computed mean:', mu)\n",
+ "print('Computed standard deviation:', sigma)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "*You should not submit your solutions.*"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "grader[4] = featureNormalize\n",
+ "grader.grade()"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "After the `featureNormalize` function is tested, we now add the intercept term to `X_norm`:"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "# Add intercept term to X\n",
+ "X = np.concatenate([np.ones((m, 1)), X_norm], axis=1)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "\n",
+ "### 3.2 Gradient Descent\n",
+ "\n",
+ "Previously, you implemented gradient descent on a univariate regression problem. The only difference now is that there is one more feature in the matrix $X$. The hypothesis function and the batch gradient descent update\n",
+ "rule remain unchanged. \n",
+ "\n",
+ "You should complete the code for the functions `computeCostMulti` and `gradientDescentMulti` to implement the cost function and gradient descent for linear regression with multiple variables. If your code in the previous part (single variable) already supports multiple variables, you can use it here too.\n",
+ "Make sure your code supports any number of features and is well-vectorized.\n",
+ "You can use the `shape` property of `numpy` arrays to find out how many features are present in the dataset.\n",
+ "\n",
+ "
\n",
+ "**Implementation Note:** In the multivariate case, the cost function can\n",
+ "also be written in the following vectorized form:\n",
+ "\n",
+ "$$ J(\\theta) = \\frac{1}{2m}(X\\theta - \\vec{y})^T(X\\theta - \\vec{y}) $$\n",
+ "\n",
+ "where \n",
+ "\n",
+ "$$ X = \\begin{pmatrix}\n",
+ " - (x^{(1)})^T - \\\\\n",
+ " - (x^{(2)})^T - \\\\\n",
+ " \\vdots \\\\\n",
+ " - (x^{(m)})^T - \\\\ \\\\\n",
+ " \\end{pmatrix} \\qquad \\mathbf{y} = \\begin{bmatrix} y^{(1)} \\\\ y^{(2)} \\\\ \\vdots \\\\ y^{(m)} \\\\\\end{bmatrix}$$\n",
+ "\n",
+ "the vectorized version is efficient when you are working with numerical computing tools like `numpy`. If you are an expert with matrix operations, you can prove to yourself that the two forms are equivalent.\n",
+ "
\n",
+ "\n",
+ ""
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "def computeCostMulti(X, y, theta):\n",
+ " \"\"\"\n",
+ " Compute cost for linear regression with multiple variables.\n",
+ " Computes the cost of using theta as the parameter for linear regression to fit the data points in X and y.\n",
+ " \n",
+ " Parameters\n",
+ " ----------\n",
+ " X : array_like\n",
+ " The dataset of shape (m x n+1).\n",
+ " \n",
+ " y : array_like\n",
+ " A vector of shape (m, ) for the values at a given data point.\n",
+ " \n",
+ " theta : array_like\n",
+ " The linear regression parameters. A vector of shape (n+1, )\n",
+ " \n",
+ " Returns\n",
+ " -------\n",
+ " J : float\n",
+ " The value of the cost function. \n",
+ " \n",
+ " Instructions\n",
+ " ------------\n",
+ " Compute the cost of a particular choice of theta. You should set J to the cost.\n",
+ " \"\"\"\n",
+ " # Initialize some useful values\n",
+ " m = y.shape[0] # number of training examples\n",
+ " \n",
+ " # You need to return the following variable correctly\n",
+ " J = 0\n",
+ " \n",
+ " # ======================= YOUR CODE HERE ===========================\n",
+ "\n",
+ " \n",
+ " # ==================================================================\n",
+ " return J\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "*You should now submit your solutions.*"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "grader[5] = computeCostMulti\n",
+ "grader.grade()"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ ""
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "def gradientDescentMulti(X, y, theta, alpha, num_iters):\n",
+ " \"\"\"\n",
+ " Performs gradient descent to learn theta.\n",
+ " Updates theta by taking num_iters gradient steps with learning rate alpha.\n",
+ " \n",
+ " Parameters\n",
+ " ----------\n",
+ " X : array_like\n",
+ " The dataset of shape (m x n+1).\n",
+ " \n",
+ " y : array_like\n",
+ " A vector of shape (m, ) for the values at a given data point.\n",
+ " \n",
+ " theta : array_like\n",
+ " The linear regression parameters. A vector of shape (n+1, )\n",
+ " \n",
+ " alpha : float\n",
+ " The learning rate for gradient descent. \n",
+ " \n",
+ " num_iters : int\n",
+ " The number of iterations to run gradient descent. \n",
+ " \n",
+ " Returns\n",
+ " -------\n",
+ " theta : array_like\n",
+ " The learned linear regression parameters. A vector of shape (n+1, ).\n",
+ " \n",
+ " J_history : list\n",
+ " A python list for the values of the cost function after each iteration.\n",
+ " \n",
+ " Instructions\n",
+ " ------------\n",
+ " Peform a single gradient step on the parameter vector theta.\n",
+ "\n",
+ " While debugging, it can be useful to print out the values of \n",
+ " the cost function (computeCost) and gradient here.\n",
+ " \"\"\"\n",
+ " # Initialize some useful values\n",
+ " m = y.shape[0] # number of training examples\n",
+ " \n",
+ " # make a copy of theta, which will be updated by gradient descent\n",
+ " theta = theta.copy()\n",
+ " \n",
+ " J_history = []\n",
+ " \n",
+ " for i in range(num_iters):\n",
+ " # ======================= YOUR CODE HERE ==========================\n",
+ "\n",
+ " \n",
+ " # =================================================================\n",
+ " \n",
+ " # save the cost J in every iteration\n",
+ " J_history.append(computeCostMulti(X, y, theta))\n",
+ " \n",
+ " return theta, J_history"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "*You should now submit your solutions.*"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "grader[6] = gradientDescentMulti\n",
+ "grader.grade()"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "#### 3.2.1 Optional (ungraded) exercise: Selecting learning rates\n",
+ "\n",
+ "In this part of the exercise, you will get to try out different learning rates for the dataset and find a learning rate that converges quickly. You can change the learning rate by modifying the following code and changing the part of the code that sets the learning rate.\n",
+ "\n",
+ "Use your implementation of `gradientDescentMulti` function and run gradient descent for about 50 iterations at the chosen learning rate. The function should also return the history of $J(\\theta)$ values in a vector $J$.\n",
+ "\n",
+ "After the last iteration, plot the J values against the number of the iterations.\n",
+ "\n",
+ "If you picked a learning rate within a good range, your plot look similar as the following Figure. \n",
+ "\n",
+ "\n",
+ "\n",
+ "If your graph looks very different, especially if your value of $J(\\theta)$ increases or even blows up, adjust your learning rate and try again. We recommend trying values of the learning rate $\\alpha$ on a log-scale, at multiplicative steps of about 3 times the previous value (i.e., 0.3, 0.1, 0.03, 0.01 and so on). You may also want to adjust the number of iterations you are running if that will help you see the overall trend in the curve.\n",
+ "\n",
+ "
\n",
+ "**Implementation Note:** If your learning rate is too large, $J(\\theta)$ can diverge and ‘blow up’, resulting in values which are too large for computer calculations. In these situations, `numpy` will tend to return\n",
+ "NaNs. NaN stands for ‘not a number’ and is often caused by undefined operations that involve −∞ and +∞.\n",
+ "
\n",
+ "\n",
+ "
\n",
+ "**MATPLOTLIB tip:** To compare how different learning learning rates affect convergence, it is helpful to plot $J$ for several learning rates on the same figure. This can be done by making `alpha` a python list, and looping across the values within this list, and calling the plot function in every iteration of the loop. It is also useful to have a legend to distinguish the different lines within the plot. Search online for `pyplot.legend` for help on showing legends in `matplotlib`.\n",
+ "
\n",
+ "\n",
+ "Notice the changes in the convergence curves as the learning rate changes. With a small learning rate, you should find that gradient descent takes a very long time to converge to the optimal value. Conversely, with a large learning rate, gradient descent might not converge or might even diverge!\n",
+ "Using the best learning rate that you found, run the script\n",
+ "to run gradient descent until convergence to find the final values of $\\theta$. Next,\n",
+ "use this value of $\\theta$ to predict the price of a house with 1650 square feet and\n",
+ "3 bedrooms. You will use value later to check your implementation of the normal equations. Don’t forget to normalize your features when you make this prediction!"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "\"\"\"\n",
+ "Instructions\n",
+ "------------\n",
+ "We have provided you with the following starter code that runs\n",
+ "gradient descent with a particular learning rate (alpha). \n",
+ "\n",
+ "Your task is to first make sure that your functions - `computeCost`\n",
+ "and `gradientDescent` already work with this starter code and\n",
+ "support multiple variables.\n",
+ "\n",
+ "After that, try running gradient descent with different values of\n",
+ "alpha and see which one gives you the best result.\n",
+ "\n",
+ "Finally, you should complete the code at the end to predict the price\n",
+ "of a 1650 sq-ft, 3 br house.\n",
+ "\n",
+ "Hint\n",
+ "----\n",
+ "At prediction, make sure you do the same feature normalization.\n",
+ "\"\"\"\n",
+ "# Choose some alpha value - change this\n",
+ "alpha = 0.1\n",
+ "num_iters = 400\n",
+ "\n",
+ "# init theta and run gradient descent\n",
+ "theta = np.zeros(3)\n",
+ "theta, J_history = gradientDescentMulti(X, y, theta, alpha, num_iters)\n",
+ "\n",
+ "# Plot the convergence graph\n",
+ "pyplot.plot(np.arange(len(J_history)), J_history, lw=2)\n",
+ "pyplot.xlabel('Number of iterations')\n",
+ "pyplot.ylabel('Cost J')\n",
+ "\n",
+ "# Display the gradient descent's result\n",
+ "print('theta computed from gradient descent: {:s}'.format(str(theta)))\n",
+ "\n",
+ "# Estimate the price of a 1650 sq-ft, 3 br house\n",
+ "# ======================= YOUR CODE HERE ===========================\n",
+ "# Recall that the first column of X is all-ones. \n",
+ "# Thus, it does not need to be normalized.\n",
+ "\n",
+ "price = 0 # You should change this\n",
+ "\n",
+ "# ===================================================================\n",
+ "\n",
+ "print('Predicted price of a 1650 sq-ft, 3 br house (using gradient descent): ${:.0f}'.format(price))"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "*You do not need to submit any solutions for this optional (ungraded) part.*"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "\n",
+ "### 3.3 Normal Equations\n",
+ "\n",
+ "In the lecture videos, you learned that the closed-form solution to linear regression is\n",
+ "\n",
+ "$$ \\theta = \\left( X^T X\\right)^{-1} X^T\\vec{y}$$\n",
+ "\n",
+ "Using this formula does not require any feature scaling, and you will get an exact solution in one calculation: there is no “loop until convergence” like in gradient descent. \n",
+ "\n",
+ "First, we will reload the data to ensure that the variables have not been modified. Remember that while you do not need to scale your features, we still need to add a column of 1’s to the $X$ matrix to have an intercept term ($\\theta_0$). The code in the next cell will add the column of 1’s to X for you."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "# Load data\n",
+ "data = np.loadtxt(os.path.join('Data', 'ex1data2.txt'), delimiter=',')\n",
+ "X = data[:, :2]\n",
+ "y = data[:, 2]\n",
+ "m = y.size\n",
+ "X = np.concatenate([np.ones((m, 1)), X], axis=1)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Complete the code for the function `normalEqn` below to use the formula above to calculate $\\theta$. \n",
+ "\n",
+ ""
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "def normalEqn(X, y):\n",
+ " \"\"\"\n",
+ " Computes the closed-form solution to linear regression using the normal equations.\n",
+ " \n",
+ " Parameters\n",
+ " ----------\n",
+ " X : array_like\n",
+ " The dataset of shape (m x n+1).\n",
+ " \n",
+ " y : array_like\n",
+ " The value at each data point. A vector of shape (m, ).\n",
+ " \n",
+ " Returns\n",
+ " -------\n",
+ " theta : array_like\n",
+ " Estimated linear regression parameters. A vector of shape (n+1, ).\n",
+ " \n",
+ " Instructions\n",
+ " ------------\n",
+ " Complete the code to compute the closed form solution to linear\n",
+ " regression and put the result in theta.\n",
+ " \n",
+ " Hint\n",
+ " ----\n",
+ " Look up the function `np.linalg.pinv` for computing matrix inverse.\n",
+ " \"\"\"\n",
+ " theta = np.zeros(X.shape[1])\n",
+ " \n",
+ " # ===================== YOUR CODE HERE ============================\n",
+ "\n",
+ " \n",
+ " # =================================================================\n",
+ " return theta"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "*You should now submit your solutions.*"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "grader[7] = normalEqn\n",
+ "grader.grade()"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Optional (ungraded) exercise: Now, once you have found $\\theta$ using this\n",
+ "method, use it to make a price prediction for a 1650-square-foot house with\n",
+ "3 bedrooms. You should find that gives the same predicted price as the value\n",
+ "you obtained using the model fit with gradient descent (in Section 3.2.1)."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "# Calculate the parameters from the normal equation\n",
+ "theta = normalEqn(X, y);\n",
+ "\n",
+ "# Display normal equation's result\n",
+ "print('Theta computed from the normal equations: {:s}'.format(str(theta)));\n",
+ "\n",
+ "# Estimate the price of a 1650 sq-ft, 3 br house\n",
+ "# ====================== YOUR CODE HERE ======================\n",
+ "\n",
+ "price = 0 # You should change this\n",
+ "\n",
+ "# ============================================================\n",
+ "\n",
+ "print('Predicted price of a 1650 sq-ft, 3 br house (using normal equations): ${:.0f}'.format(price))"
+ ]
+ }
+ ],
+ "metadata": {
+ "kernelspec": {
+ "display_name": "Python 3",
+ "language": "python",
+ "name": "python3"
+ },
+ "language_info": {
+ "codemirror_mode": {
+ "name": "ipython",
+ "version": 3
+ },
+ "file_extension": ".py",
+ "mimetype": "text/x-python",
+ "name": "python",
+ "nbconvert_exporter": "python",
+ "pygments_lexer": "ipython3",
+ "version": "3.7.6"
+ }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 2
+}
diff --git a/Phase 3 - 2020 (Summer)/Week 2 (Apr 5 - Apr 11)/Exercise1/exercise1.ipynb b/Phase 3 - 2020 (Summer)/Week 2 (Apr 5 - Apr 11)/Exercise1/exercise1.ipynb
index dc594f146..c9184cd8f 100644
--- a/Phase 3 - 2020 (Summer)/Week 2 (Apr 5 - Apr 11)/Exercise1/exercise1.ipynb
+++ b/Phase 3 - 2020 (Summer)/Week 2 (Apr 5 - Apr 11)/Exercise1/exercise1.ipynb
@@ -1294,7 +1294,7 @@
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
- "version": "3.6.4"
+ "version": "3.7.6"
}
},
"nbformat": 4,
diff --git a/Phase 3 - 2020 (Summer)/Week 2 (Apr 5 - Apr 11)/Tanmay_190106065_Week_2/Week_2 (1).ipynb b/Phase 3 - 2020 (Summer)/Week 2 (Apr 5 - Apr 11)/Tanmay_190106065_Week_2/Week_2 (1).ipynb
new file mode 100644
index 000000000..ca0e52719
--- /dev/null
+++ b/Phase 3 - 2020 (Summer)/Week 2 (Apr 5 - Apr 11)/Tanmay_190106065_Week_2/Week_2 (1).ipynb
@@ -0,0 +1,367 @@
+{
+ "cells": [
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "import os\n",
+ "import matplotlib.pyplot as plt\n",
+ "import seaborn as sns; sns.set()\n",
+ "import numpy as np\n",
+ "from sklearn.linear_model import LinearRegression\n",
+ "from mpl_toolkits.mplot3d import Axes3D\n",
+ "%matplotlib inline"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "A = np.eye(5)"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "print(A)"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "def warmUpExcercise():\n",
+ " A = []\n",
+ " \n",
+ " for i in range(5):\n",
+ " row=[]\n",
+ " for j in range(5):\n",
+ " if j == i:\n",
+ " row.append(1)\n",
+ " row.append(0)\n",
+ " A.append(row)\n",
+ " return A"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "warmUpExcercise()"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 17,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "data = np.loadtxt('data1.txt',delimiter=',')\n",
+ "x,y = data[:,0], data[:,1]\n",
+ "m = y.size"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 18,
+ "metadata": {},
+ "outputs": [
+ {
+ "data": {
+ "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXMAAAD7CAYAAACYLnSTAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjMsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+AADFEAAAdfklEQVR4nO3df5Ac9Xnn8ffs6tcaSUFZ1hE/hOFk84C3bMcniE8n2VJs7ItcxAH5sGNSERQG7JKp811sCJXgcpzUnUkoY+eMZVfJCIQpMGXz4w6BiDECYuQNSShhu1bSY58icnBId2Ih0Y+SVvvr/pjp1Wxvz0zPj57p7vm8qii0PTPdj2ZHT3/n6ef77cLU1BQiIpJtPZ0OQEREmqdkLiKSA0rmIiI5oGQuIpIDSuYiIjkwpwPHnA9cAhwAJjpwfBGRLOoFzgT+ARgNPxgrmZvZl4FPlH583N1vNrO7gdXAsdL2r7j7IzF2dwnwkzjHFRGRWd4PPB/eWDOZm9mlwEeA9wJTwJNmdgVwMfABdz9QZyAHAN588xiTk/X3uPf3L2Rk5Gjdr+sUxZu8rMWseJOVtXghXsw9PQWWLDkNSjk0LM7I/ADwBXc/CWBme4BzS/9tMbOzgUcojswnY+xvAmBycqqhZB68NksUb/KyFrPiTVbW4oW6Yo4sT9dM5u4+HPzZzN5BsdzyfmAtsBH4V2Ab8Glgc9xoRESkdQpxp/Ob2SDwOPBld98aeuwKYIO7XxFjV+cB++uMU0REis4HXg5vjHsBdBXwEPCf3f37ZvYu4AJ3f6j0lAIwVk80IyNHG/oqNDCwiEOHjtT9uk5RvMnLWsyKN1lZixfixdzTU6C/f2HFx+NcAF0GPAp80t13lDYXgG+Y2Q7gKHADsLXCLkREJGFxRuZfBBYAd5hZsO07wFeBncBc4CF3fyCRCEVEMm5o+CAPP7ePkcOj9C+ez/o1y1k5uLSlx4hzAfTzwOcrPLyppdGIiOTM0PBBtm7fy8nxYrPfyOFRtm7fC9DShK7p/CIiCXr4uX3TiTxwcnySh5/b19LjKJmLiCRo5PCsmfdVtzdKyVxEJEH9i+fXtb1RSuYiIglav2Y58+bMTLXz5vSwfs3ylh6nE6smioh0jeAiZ8e7WUREpDkrB5e2PHmHqcwiIpIDSuYiIjmgZC4ikgOqmYtIR7Vjqns3UDIXkY5p11T3bqAyi4h0TLumuncDJXMR6Zh2TXXvBkrmItIx7Zrq3g2UzEWkY9o11b0b6AKoiHRM0lPdu6lTRslcRDoqqanu3dYpozKLiORSt3XKKJmLSC51W6eMkrmI5FK3dcoomYtILnVbp4wugIpILgUXOe9/yjl2YgKAeXPzO37N799MRAQYG5+a/vPR4+Ns3b6XoeGDHYwoGUrmIpJb3dTRomQuIrnVTR0tSuYiklvd1NGiZC4iudVNHS3qZhGR3Ep67Zc0iZXMzezLwCdKPz7u7jeb2aXAHUAf8KC735pQjCIiDUtq7Ze0qVlmKSXtjwDvBX4TWGFmnwK2AL8HXARcYmbrkgxUREQqi1MzPwB8wd1PuvsYsAe4APiVu+9393HgPuDKBOMUEZEqapZZ3H04+LOZvYNiueWbFJN84ABwTj0H7u9fWM/TZxgYWNTwaztB8SYvazEr3mRlLV5oPubYF0DNbBB4HLgJGKc4Og8UgMmo11UyMnKUycmp2k8MGRhYxKFDR+p+Xaco3uRlLWbFm6ysxQvxYu7pKVQdBMdqTTSzVcDTwC3uvhV4FTiz7ClLgdfi7EtERFqv5sjczJYBjwKfdPcdpc0vFB+ytwP7gasoXhAVEZEOiFNm+SKwALjDzIJt3wGuAR4qPfYE8MME4hMRkRjiXAD9PPD5Cg+/p7XhiIhIIzSdX0QkB5TMRURyQMlcRCQHlMxFRHJAyVxEJAeUzEVEckDJXEQkB3RzChGRGoaGD6b+BhdK5iIiVQwNH2Tr9r2cHC+uJThyeJSt2/cCpCqhd20yz8KZVkQ67+Hn9k0n8sDJ8Ukefm5fqnJGVybzrJxpRaTzRg6P1rW9U7ryAmi1M62ISLn+xfPr2t4pXZnMs3KmFZHOW79mOfPmzEyV8+b0sH7N8g5FFK0ryyz9i+dHJu60nWlFukHar18FsaQ5RujSZL5+zfIZNXNI55lWJO+ycv1q5eDSVMUTpSvLLCsHl3L1ugunR+L9i+dz9boLU//LEskbXb9qna4cmUM2zrQieafrV63TlSNzEUmHrHSKZIGSuYh0TFY6RbKga8ssItJ5WekUyQIlcxHpKF2/ag2VWUREckDJXEQkB5TMRURyQDVzkTqkfeq5dC8lc5GYsjL1XLqTyiwiMWnquaSZkrlITJp6LmkWu8xiZouBnwKXufvLZnY3sBo4VnrKV9z9kQRiFEkFLZ0saRYrmZvZ+4DNwAVlmy8GPuDuB5IITCRttHSypFnckfn1wOeA7wGY2VuAc4EtZnY28AjFkflk5V2IdFaznSiaei5pVpiamor9ZDN7GVhLsdb+NWAj8K/ANuABd98cYzfnAfvrC1OkOc+++Ap3/uBnjI5NTG+bP7eXG698D2tXLOtgZCJ1Ox94ObyxodZEd/8n4IrgZzP7JrCBYikmlpGRo0xOxj+RBAYGFnHo0JG6X9cpijd5cWK+Z9vwjEQOMDo2wT3bhhk89/Qkw5sla++x4k1enJh7egr09y+s/HgjBzazd5nZx8s2FYCxRvYl0g7qRJG8a3TSUAH4hpntAI4CNwBbWxaVSIupE0XyrqGRubv/HPgqsBPYDbzk7g+0MjCRVtJNECTv6hqZu/t5ZX/eBGxqdUAiSVAniuSd1maRrqGbIEieKZmnnFbpE5E4lMxTTKv01U8nP+lWWmgrxbRKX32Ck1/QtRKc/IaGD3Y4MpHkKZmnmHqj66OTn3QzJfMUq9QDrd7oaDr5STdTMk8x9UbXRyc/6WZK5im2cnApV6+7cDoZ9S+ez9XrLtQFvQp08pNupm6WlFNvdHyaGCTdTMlcckUnP+lWKrOIiOSAkrmISA6ozJJDmgUp0n2UzDOgnuSsJQDSJ/z7u+aywbbf3UjyT8m8Ae0Y+ZYfo1yt5FxtFqSSeftFnVzv/MHP2PA7pt+HtJRq5nVqx/of4WOEVZuirlmQ6RJ1ch0dm9ASA9JySuZ1asf6H1HHCKuUnDULMl10cpV2UTKvUzv+ccbZV6XkrFmQ6aKTq7SLknmd2vGPs9a+qiVnLQGQLlEn1/lze3VylZbTBdA6rV+zfMYFLWj9yDfqGIE4F1w1CzI9opYYUDeLJEHJvE7tWP9Da4zkS/jkOjCwiEOHjnQwIskjJfMGtGPkq9G1iNRDNXMRkRxQMhcRyQElcxGRHFAyFxHJASVzEZEcUDeL5I6WAJZuFCuZm9li4KfAZe7+spldCtwB9AEPuvutCcYoEpuWAJZuVbPMYmbvA54HLij93AdsAX4PuAi4xMzWJRmkSFztWAhNJI3i1MyvBz4HvFb6+beAX7n7fncfB+4DrkwoPpG6aJVC6VY1yyzufh2AmQWbzgIOlD3lAHBOvQfu719Y70umDQwsavi1naB4kxfEPLCkj0NvHp/9+JK+VP290hRLHIo3ec3G3MgF0B5gquznAlB98e0IIyNHmZycqv3EkKyta6F4k1ce8+Wrz49cCO3y1een5u+VtfdY8SYvTsw9PYWqg+BGkvmrwJllPy/lVAlGpKO0SJl0q0aS+QuAmdnbgf3AVRQviHacWtIEtEiZdKe6k7m7nzCza4CHgAXAE8APWxxX3dSSJvXQiV/yJnYyd/fzyv78NPCeJAJqlO5KL3HpxC95lJvp/GpJk7jUiy55lJtkrhvnSlw68Use5SaZ6670EpdO/JJHuUnmuiu9xKUTv+RRrlZNVEuaxKFedMmjXCVzkbh04pe8UTKXjlK/t0hrKJlLRUknWvV7i7RObi6ASmsFiTZo1wsS7dDwwZYdQ/3eIq2jkXmXqjXqbseMWvV7i7SORuZdKM6oux2JVv3eIq2jZJ5SQ8MHuWnTTq69bQc3bdrZ9vJGOxKt+r1FWqcryixZ65hI+sJgnFH3+jXLI2/y0MpEq35vkdbJfTLPYsdE0vXq/sXzIxN6+ai7kUTbyElT/d4irZH7ZJ7FpXGTrlfHHXXXk2izeNIUyZPcJ/NOdkwMDR/k0eeHOPTm8emRKtQe7cYZOVc7Zq391xp1l++jpwCTU9QcaWfxpCmSJ7lP5kEyitoepVX19aiR6t1P7GFqcoqJUjzh0Wv5scPi1KvrGR1XGnU/++IrM/YxWSHWMLUZinRW7rtZohJ5pe2tnCgTNVIdnziVyAPB6DV87HJxV4BsxSSce7fvmbWPOPtSm6FIZ+U+mdeTZFo5I7GeEenI4dHIYwdx3r5xVaxvB60YHb/+5vGGjqE2Q5HOyn0yryfJtLJUUM+ItFKNvN5jt2J0fMaSvoaOofXkRTor98m8niRTKVH1FKi71BJ1EpnTW6A3VKsPTiytSMStGB1vWHfRrH3E3dfKwaXcvnEVW275YOxvEyLSGrm/AArxW+yiWvagWF+vt80ueN6jz++P3c3S7CSdVkzCWbtiGYePnKi7m0VEOqsrknlcQaK6a9vuWRdIG2mzWzm4lI+tfQeHDh2JPE7Utvufco6dmABg3tz6vzi1YhKOJvKIZE9mknnQtvfG4VF+PcFR4srBpWx+bHfkY+1osxsbP3UWOXp8XBNvRCSWTCTzdswujJooE5Z0m50m3ohIozKRzJNOcuGTRVQij6pft3oBr6Qm3qR9obG0xyeSBZlI5knPLnzgx7+sOFEmsOpdM+vISXxbaGYafyVpXzMl7fGJZEVTydzMngHeCoyVNn3G3V9oOqqQSkluYV/z56Kh4YMcPT5e83k/3zcy4+ckvi3Us+xs3NFs2ks3aY9PJCsazoZmVgAuAN7m7rWzYRPWr1nO3U/sYTw0F/74iXGGhg829Y8+7uzO8Mkk7reFZ198hXu2DccqIcRtLaxnNJv2NVPSHp9IVjQztLXS/39kZv3AZne/swUxzbJycCn3P+WMT0zM2D4xRdMjuLhJI1zqqPRtIZhgFCycde+TzujYxPSxapUQ4rQF1jOaTaJ000ppj08kK5qZAboEeBq4AvgQ8Fkz+3BLoooQ9F6HNTuCi5M0okodUbMt4dQEo6AMEiTyQCvuPl/PaDbta6akPT6RrGh4ZO7uQ8BQ8LOZ3QV8FHgqzuv7+xfWdbyBJX0cilgEamBJHwMDi2Zse/bFV7h3+x5ef/M4ZyzpY8O6i1i7Ylnkfq+5bJC/fnDXrBJO+f6jXv+xtYtYvGgBX//+LiZD7S8nxyd59Pn9vFEh6b5xeJTh//0vsWOMiinuexHEWc+xwvtIUiPxRWlnzK2geJOVtXih+ZgLU1MV1oitwcxWA/Pd/enSz38ELHX3m2u89Dxg/8jI0VlJsJpwnRiKI7jwOitxnxfe9wM//uX0hdDTFvRy1YctVvnm2tt2VHys2oXbk2OTdcUYjrfev2NcAwOLZs1YTbusxax4k5W1eCFezD09hWAQfD7wcvjxZmrmpwN/bmb/HpgLXA18ton9VVVprZNwIm9kKn4z09crJexCoXI55OTYBCfHZ4/mo2Ks1rWi3mwRCTRTZtlmZu8DdgG9wLdKpZfEVFrrBE6NVisN9pPqjqi0OFe1LzzhRB4YOTzKTZt2TifmWl0rSt4iEmiqUdvdvwR8qUWxNKXSzR0CSXVHhEfJzSpP2M30YGtWpUh3ycQM0EDUDZKDBFUtkbaqO6JSgqy0hG2jgoTdaA+2ZlWKdJ/M3JwiSFBBF0f4/pzVbizRiguDte4PWuubQdjCvjlVvy0EJ4wotb5ltPL2dyKSDZlJ5rUSVKV+5U9f9s7YiXxo+CA3bdrJtbft4KZNO2fcXajW8espscyb08OnLr2A2zeuqpqwG+3B1qxKke6TmWReLUEF5Y+T45P0lG7LVu89KGuNvGslyHpq8uVxVUvYjd5XsxW3oBORbMlMzbxSC+BpC3pnLV/bW4DRsQk2P7abh5/bF+viX62LjdWmnQ8NH5w107Pa36M8llptho10rdSzYJeI5ENmknmlBFUoFDg5PnvNlmAC0MjhUTY/tpt7n9zDht+5aFZfeq0ulOCxSsd/9/L+yAuf8+YUmJxixszSSgm11W2G6kMX6T6ZSeaVJg1VusVb2OjYFHc/sWd6X1GzKKMEpYlKCbLShc9Fb5nH+jXLq05ySpL60EW6S2aSOURPGqqnv3t8Ymq6bBKn+yQ8ko5KkNXuF1ptkpOISCtlKplH9ZlXmoFZSfkFzmrijqS1hKuIpEFmulkq9ZkDMzo+ainvDKn0+JZbPsjtG1fFKlNoCVcRSYPMJPNa3SbVerYDc3oL00m20nrk717eX1dcjbYPioi0UmbKLHEmwtQquSyY1zv955WDS/lfr/4Lz+x6bcZzdv7iIG8/5/S6krEuNopIp2VmZB5nIkx4lHzagl7m9BamHz96fHzGRKDwTZpB095FJJsyk8zjlkWCksuWWz7IgnlzZt1BKM4UfE17F5GsyUyZpVJZ5Jldr/HMrtciu0/iTMFXJ4qI5EFmRuYQXRYJjBwe5e4n9kyXUIaGD06v0xLWUyg+rk4UEcmLTCXzWuWP8YkpHvjxL2vedWhyisi2RnWiiEhWZabMApXLIuWOHh+PNbszqJ3H7ScXEUmzTCXz9WuWc++TXnOFwrgXMIPn6RZrIpJ1mUrmKweXsnjRAu7ZNlwxYZ+2oJcF8+bESug9Bfje3+xl5y8O6hZrIpJpmaqZA6xdsYzbN67i+t99J72hC5y9Bbjqw1axjTFscqrYDaNbrIlI1mVqZF4uaknaty7p465tu6cvfM6f28vo2AQ9BSpeDK0kzshe5RkRSYvMJnOYOY3+e3+zd1YP+ujYBL/93rP4w/9wIQDX3rYj9r5r9ZqH10NXeUZEOilzZZZKnnvptcjtz+x6bbr3/LQFvZHPCYvTa17rBs8iIu2UuZH5sy++Mn0BNChtQPUySjBiLhSiZxHNn9vLnF44dqLYJTNvbu1znJYCEJE0yVQyHxo+OKM1ceTwKFu27abWbSlOjk/OqKWHjY5NMDV1KoEHC3JB5ZKJlgIQkTTJVJnl4ef2zeoxn5iCqRgXN6uN3HsKRJZMNj+2m5s27Zwu05TTUgAikiaZGpknUcKYN6en6mzRShc2K93gWRc/RaQTmkrmZnYVcCswF/iGu3+rJVFVEGc6fz37CRJwrZtCl9/RqFwSN6VQu6OINKLhZG5mZwP/FVgBjAI/NbNn3D36dvUtsH7NcjY/1tzu+xfP5/aNq2Ztr3VT6HZc2FS7o4g0qpma+aXADnd/w92PAT8E/mNrworWbEKrVNMO36EoSjsubKrdUUQa1UwyPws4UPbzAeCc5sKpbWBJX+T2amuXQ+3lbYM7FF3/u+/s2IVNtTuKSKOaqZn3AOU9IgWo2SU4rb9/YUMH3bDuIu78wc9mdLXMn9vLhy4+h6f/8dVZ22+88j2sXbEs9v4/tnYRixct4N7te3j9zeOcsaSPDesuqmsfYQMDi+I9b0kfh948Hrk97j5aoZ3HapWsxax4k5W1eKH5mJtJ5q8C7y/7eSkQPQ0zwsjIUSbrXTCF4kJbh4+ciLxIePYZp83aPnju6Rw6dKSuYwyeezp/+ZmVM7bVu4/AwMCi2K+9fPX5s2r38+b0cPnq8xs+fr3qiTctshaz4k1W1uKFeDH39BSqDoKbSeY/Bv7MzAaAY8DHgRua2F/TkuguaSe1O4pIoxpO5u7+f8zsT4FngHnAd93971sWWQXPvvhKrjs+sn5CEpHOaKrP3N3vB+5vUSyx3Lt9T8WODyVBEelWmZrOD/B6xAVCUMeHiHS3zCXzMyq0Ji7sy9TKBCIiLZW5DLhh3UX89YO7GJ+Y2Qlz/MQ4Q8MHGy61aBq9iGRZ5kbma1csY37EeuMTUzQ8UzKYRh+UaoKLqlGrJYqIpFHmkjmcuolEWKN1c02jF5Gsy2Qyr7ROSqPrp2gavYhkXSaTeatvDNHqk4OISLtlMpmHVzmstYhWLbprkIhkXea6WQKtnCmpafQiknWZTeatpmn0IpJlmSyziIjITJkZmQeTet44PMqvqwwiIjJDJpK57o0pIlJdJsosmtQjIlJdJpK5JvWIiFSXiWSuST0iItVlIplrUo+ISHWZuABaPqlH3SwiIrNlIpnDqUk9WbzztohI0jJRZhERkeqUzEVEckDJXEQkB5TMRURyoBMXQHsBenoKDe+gmdd2guJNXtZiVrzJylq8UDvmssd7ox4vTE1NRW1P0mrgJ+0+qIhITrwfeD68sRPJfD5wCXAAiL4zs4iIhPUCZwL/AMxay6QTyVxERFpMF0BFRHJAyVxEJAeUzEVEckDJXEQkB5TMRURyQMlcRCQHlMxFRHIgteuZm9kzwFuBsdKmz7j7C2WPXwrcAfQBD7r7re2PcjqW64AbyzadD3zP3W8se86XgWuBN0ubNrv7t9oX5XQci4GfApe5+8tx3kczOxe4j+Lvw4E/cPejHYr3BuA/AVPAP1L8XJwMveZq4Dbg/5Y2Pe7uf9qheO+mOOv5WOkpX3H3R0Kv+U3gu8Bi4G+Bz7r7eDviDccMvBP4b2UPnw284O6XhV7Tkfe49O/oE2XHvDnNn+EK8SbyGU5lMjezAnAB8LaoD7WZ9QFbgDXAK8DjZrbO3be3N9Iid/8uxX+MmNkg8CjwZ6GnXQz8vrsPtTe6U8zsfcBmiu9tPe/jJmCTu3/fzL4EfAn44w7EewFwE7ACOALcA3wO+HropRcDf+TuDyQdY7lwvGWxfMDdD1R56X3Ade7+d2Z2F3A98O3kIj0lHLO7PwE8UXpsKbAT+C8RL237e1xK2h8B3ksxET5pZp8C/pIUfoYrxPvHwHUk8BlOa5nFSv//kZn9zMxuDD3+W8Cv3H1/KdnfB1zZ1ggr+zbwJ+7+emj7xcCfmNnPzexOM1vQgdiup/jBea30c8330czmAh8AfljadE/4OQkKxzsKbHT3w+4+BfwCODfidZcAV5vZL8zsPjNb0p5wZ8ZrZm8pxbel9Hv/ipnN+DdnZm8D+tz970qb7qG9n+Xwe1zuduA77v6riMc68R4fAL7g7ifdfQzYQ/EklNbPcFS8C0joM5zWZL4EeBq4AvgQ8Fkz+3DZ42dRfKMCB4Bz2hdetNKZuM/dfxDavhDYRXFU+W+B0ymODNrK3a9z9/JFzuK8j2cAh8u+IbXtvQ7H6+7/7O5PAZjZAMXS1v+IeOkB4C+Ad1Mcrd3ZhnCj3t+lwA6K5bV/R3GBpE+HXtbRz3JEzACY2TuAtcB/r/DStr/H7j4cnPRK8X0CmCSln+EK8d6f1Gc4lWWWUiliuhxR+ur5UeCp0qYeil9bAgWKv9RO+wzF2t0MpdrcR4OfzexrFMsbbanjVhHnfQw/h4jntJWZnQ1sB+5y92fDj7v7FWXP/StgX/uimxHHP1EckASxfBPYQLGsEUjrZ/kGimWJWQs6QWff41Ip83GKg6NxZpa1UvcZLo83+JaTxGc4lSNzM1ttZh8q21Tg1IVQgFcprh4WWEr018S2MbN5FOt2/zPisXPN7NqyTeG/T6fEeR//H/BrZhasoXxmxHPaxswupHixbqu7/0XE479mZuU13gLFf/BtZ2bvMrOPh2IJ/95T91kuuRz4ftQDnXyPzWwVxW/tt7j7VlL+GY6IN7HPcCqTOcUyxO1mtsDMFgFXA+UdAC8AZmZvL/2CrqJ4luukdwO/dPdjEY8dB/7KzM4vXdz9HDP/Pp1S830s1fp+AnyytGlD+DntUvos/Ai41d2/VuFpR4GbSxf2oPg1tlPvdQH4hpktKdVtbwjH4u7/DJwo/aMH+EM6/Fk2szMolgv3V3hKR95jM1tGsbngKncPTjSp/QxHxZvkZziVydzdt1H8WrILeBHY4u5DZvaSmZ3l7ieAa4CHgN3AXk5d3OiUf0NxlDDNzJ4ws4vd/RDFEsxjFNuiCkClX2TbVHsfzey7Zvax0lM3AjeY2W6Kdd9OtYFeB/wG8IXSZ+ElM/tzOBWvu09QrE1+28z2UOwauLkTwbr7z4GvUuwI2Q28FHQnBJ+N0lP/APi6me0FFlK5Tt0usz7LkIr3+IsULyDeEfz+KX5+ryGdn+GoeG8koc+w1jMXEcmBVI7MRUSkPkrmIiI5oGQuIpIDSuYiIjmgZC4ikgNK5iIiOaBkLiKSA0rmIiI58P8BfDK9cNU40eIAAAAASUVORK5CYII=\n",
+ "text/plain": [
+ "
"
+ ]
+ },
+ "metadata": {
+ "needs_background": "light"
+ },
+ "output_type": "display_data"
+ }
+ ],
+ "source": [
+ "# Choose some alpha value - change this\n",
+ "alpha = 0.01\n",
+ "num_iters = 400\n",
+ "\n",
+ "# init theta and run gradient descent\n",
+ "theta = np.zeros(3)\n",
+ "theta, J_history = gradientDescentMulti(X, y, theta, alpha, num_iters)\n",
+ "\n",
+ "# Plot the convergence graph\n",
+ "plt.plot(np.arange(len(J_history)), J_history, lw=2)\n",
+ "plt.xlabel('Number of iterations')\n",
+ "plt.ylabel('Cost J')\n",
+ "\n",
+ "# Display the gradient descent's result\n",
+ "print('theta computed from gradient descent: {:s}'.format(str(theta)))\n",
+ "\n",
+ "# Estimate the price of a 1650 sq-ft, 3 br house\n",
+ "# ======================= YOUR CODE HERE ===========================\n",
+ "# Recall that the first column of X is all-ones. \n",
+ "# Thus, it does not need to be normalized.\n",
+ "\n",
+ "price = np.dot([1650,3,1], theta)\n",
+ " # You should change this\n",
+ "\n",
+ "# ===================================================================\n",
+ "\n",
+ "print('Predicted price of a 1650 sq-ft, 3 br house (using gradient descent): ${:.0f}'.format(price))"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 10,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "def normalEqn(X,y):\n",
+ " X_transpose = X.T \n",
+ " theta = np.linalg.inv(X_transpose.dot(X)).dot(X_transpose).dot(y) \n",
+ " return theta\n",
+ " "
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 11,
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "Theta computed from the normal equations: [ 7.61530058e+10 -7.61523054e+10 -2.80655730e+08]\n",
+ "Predicted price of a 1650 sq-ft, 3 br house (using normal equations): $125423721934439\n"
+ ]
+ }
+ ],
+ "source": [
+ "theta = normalEqn(X, y);\n",
+ "\n",
+ "# Display normal equation's result\n",
+ "print('Theta computed from the normal equations: {:s}'.format(str(theta)));\n",
+ "\n",
+ "# Estimate the price of a 1650 sq-ft, 3 br house\n",
+ "# ====================== YOUR CODE HERE ======================\n",
+ "\n",
+ "price = np.dot([1650,3,1],theta) # You should change this\n",
+ "\n",
+ "# ============================================================\n",
+ "\n",
+ "print('Predicted price of a 1650 sq-ft, 3 br house (using normal equations): ${:.0f}'.format(price))"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": []
+ }
+ ],
+ "metadata": {
+ "kernelspec": {
+ "display_name": "Python 3",
+ "language": "python",
+ "name": "python3"
+ },
+ "language_info": {
+ "codemirror_mode": {
+ "name": "ipython",
+ "version": 3
+ },
+ "file_extension": ".py",
+ "mimetype": "text/x-python",
+ "name": "python",
+ "nbconvert_exporter": "python",
+ "pygments_lexer": "ipython3",
+ "version": "3.7.6"
+ }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 4
+}
diff --git a/Phase 3 - 2020 (Summer)/Week 2 (Apr 5 - Apr 11)/Tanmay_190106065_Week_2/data1.txt b/Phase 3 - 2020 (Summer)/Week 2 (Apr 5 - Apr 11)/Tanmay_190106065_Week_2/data1.txt
new file mode 100644
index 000000000..24cba7461
--- /dev/null
+++ b/Phase 3 - 2020 (Summer)/Week 2 (Apr 5 - Apr 11)/Tanmay_190106065_Week_2/data1.txt
@@ -0,0 +1,97 @@
+6.1101,17.592
+5.5277,9.1302
+8.5186,13.662
+7.0032,11.854
+5.8598,6.8233
+8.3829,11.886
+7.4764,4.3483
+8.5781,12
+6.4862,6.5987
+5.0546,3.8166
+5.7107,3.2522
+14.164,15.505
+5.734,3.1551
+8.4084,7.2258
+5.6407,0.71618
+5.3794,3.5129
+6.3654,5.3048
+5.1301,0.56077
+6.4296,3.6518
+7.0708,5.3893
+6.1891,3.1386
+20.27,21.767
+5.4901,4.263
+6.3261,5.1875
+5.5649,3.0825
+18.945,22.638
+12.828,13.501
+10.957,7.0467
+13.176,14.692
+22.203,24.147
+5.2524,-1.22
+6.5894,5.9966
+9.2482,12.134
+5.8918,1.8495
+8.2111,6.5426
+7.9334,4.5623
+8.0959,4.1164
+5.6063,3.3928
+12.836,10.117
+6.3534,5.4974
+5.4069,0.55657
+6.8825,3.9115
+11.708,5.3854
+5.7737,2.4406
+7.8247,6.7318
+7.0931,1.0463
+5.0702,5.1337
+5.8014,1.844
+11.7,8.0043
+5.5416,1.0179
+7.5402,6.7504
+5.3077,1.8396
+7.4239,4.2885
+7.6031,4.9981
+6.3328,1.4233
+6.3589,-1.4211
+6.2742,2.4756
+5.6397,4.6042
+9.3102,3.9624
+9.4536,5.4141
+8.8254,5.1694
+5.1793,-0.74279
+21.279,17.929
+14.908,12.054
+18.959,17.054
+7.2182,4.8852
+8.2951,5.7442
+10.236,7.7754
+5.4994,1.0173
+20.341,20.992
+10.136,6.6799
+7.3345,4.0259
+6.0062,1.2784
+7.2259,3.3411
+5.0269,-2.6807
+6.5479,0.29678
+7.5386,3.8845
+5.0365,5.7014
+10.274,6.7526
+5.1077,2.0576
+5.7292,0.47953
+5.1884,0.20421
+6.3557,0.67861
+9.7687,7.5435
+6.5159,5.3436
+8.5172,4.2415
+9.1802,6.7981
+6.002,0.92695
+5.5204,0.152
+5.0594,2.8214
+5.7077,1.8451
+7.6366,4.2959
+5.8707,7.2029
+5.3054,1.9869
+8.2934,0.14454
+13.394,9.0551
+5.4369,0.61705
\ No newline at end of file
diff --git a/Phase 3 - 2020 (Summer)/Week 2 (Apr 5 - Apr 11)/Tanmay_190106065_Week_2/data2.txt b/Phase 3 - 2020 (Summer)/Week 2 (Apr 5 - Apr 11)/Tanmay_190106065_Week_2/data2.txt
new file mode 100644
index 000000000..c7767b077
--- /dev/null
+++ b/Phase 3 - 2020 (Summer)/Week 2 (Apr 5 - Apr 11)/Tanmay_190106065_Week_2/data2.txt
@@ -0,0 +1,47 @@
+2104,3,399900
+1600,3,329900
+2400,3,369000
+1416,2,232000
+3000,4,539900
+1985,4,299900
+1534,3,314900
+1427,3,198999
+1380,3,212000
+1494,3,242500
+1940,4,239999
+2000,3,347000
+1890,3,329999
+4478,5,699900
+1268,3,259900
+2300,4,449900
+1320,2,299900
+1236,3,199900
+2609,4,499998
+3031,4,599000
+1767,3,252900
+1888,2,255000
+1604,3,242900
+1962,4,259900
+3890,3,573900
+1100,3,249900
+1458,3,464500
+2526,3,469000
+2200,3,475000
+2637,3,299900
+1839,2,349900
+1000,1,169900
+2040,4,314900
+3137,3,579900
+1811,4,285900
+1437,3,249900
+1239,3,229900
+2132,4,345000
+4215,4,549000
+2162,4,287000
+1664,2,368500
+2238,3,329900
+2567,4,314000
+1200,3,299000
+852,2,179900
+1852,4,299900
+1203,3,239500
\ No newline at end of file
diff --git a/Phase 3 - 2020 (Summer)/Week 4(Apr 19 - Apr 25)/Exercise3/.ipynb_checkpoints/exercise3-checkpoint.ipynb b/Phase 3 - 2020 (Summer)/Week 4(Apr 19 - Apr 25)/Exercise3/.ipynb_checkpoints/exercise3-checkpoint.ipynb
new file mode 100644
index 000000000..0791cd41a
--- /dev/null
+++ b/Phase 3 - 2020 (Summer)/Week 4(Apr 19 - Apr 25)/Exercise3/.ipynb_checkpoints/exercise3-checkpoint.ipynb
@@ -0,0 +1,1120 @@
+{
+ "cells": [
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "# Programming Exercise 3\n",
+ "# Multi-class Classification and Neural Networks\n",
+ "\n",
+ "## Introduction\n",
+ "\n",
+ "\n",
+ "In this exercise, you will implement one-vs-all logistic regression and neural networks to recognize handwritten digits. Before starting the programming exercise, we strongly recommend watching the video lectures and completing the review questions for the associated topics. \n",
+ "\n",
+ "All the information you need for solving this assignment is in this notebook, and all the code you will be implementing will take place within this notebook. The assignment can be promptly submitted to the coursera grader directly from this notebook (code and instructions are included below).\n",
+ "\n",
+ "Before we begin with the exercises, we need to import all libraries required for this programming exercise. Throughout the course, we will be using [`numpy`](http://www.numpy.org/) for all arrays and matrix operations, [`matplotlib`](https://matplotlib.org/) for plotting, and [`scipy`](https://docs.scipy.org/doc/scipy/reference/) for scientific and numerical computation functions and tools. You can find instructions on how to install required libraries in the README file in the [github repository](https://github.com/dibgerge/ml-coursera-python-assignments)."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 1,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "# used for manipulating directory paths\n",
+ "import os\n",
+ "\n",
+ "# Scientific and vector computation for python\n",
+ "import numpy as np\n",
+ "\n",
+ "# Plotting library\n",
+ "from matplotlib import pyplot\n",
+ "\n",
+ "# Optimization module in scipy\n",
+ "from scipy import optimize as opt\n",
+ "\n",
+ "# will be used to load MATLAB mat datafile format\n",
+ "from scipy.io import loadmat\n",
+ "\n",
+ "# library written for this exercise providing additional functions for assignment submission, and others\n",
+ "import utils\n",
+ "\n",
+ "# define the submission/grader object for this exercise\n",
+ "grader = utils.Grader()\n",
+ "\n",
+ "# tells matplotlib to embed plots within the notebook\n",
+ "%matplotlib inline"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "## Submission and Grading\n",
+ "\n",
+ "\n",
+ "After completing each part of the assignment, be sure to submit your solutions to the grader. The following is a breakdown of how each part of this exercise is scored.\n",
+ "\n",
+ "\n",
+ "| Section | Part | Submission function | Points \n",
+ "| :- |:- | :- | :-: \n",
+ "| 1 | [Regularized Logistic Regression](#section1) | [`lrCostFunction`](#lrCostFunction) | 30 \n",
+ "| 2 | [One-vs-all classifier training](#section2) | [`oneVsAll`](#oneVsAll) | 20 \n",
+ "| 3 | [One-vs-all classifier prediction](#section3) | [`predictOneVsAll`](#predictOneVsAll) | 20 \n",
+ "| 4 | [Neural Network Prediction Function](#section4) | [`predict`](#predict) | 30\n",
+ "| | Total Points | | 100 \n",
+ "\n",
+ "\n",
+ "You are allowed to submit your solutions multiple times, and we will take only the highest score into consideration.\n",
+ "\n",
+ "
\n",
+ "At the end of each section in this notebook, we have a cell which contains code for submitting the solutions thus far to the grader. Execute the cell to see your score up to the current section. For all your work to be submitted properly, you must execute those cells at least once. They must also be re-executed everytime the submitted function is updated.\n",
+ "
"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "## 1 Multi-class Classification\n",
+ "\n",
+ "For this exercise, you will use logistic regression and neural networks to recognize handwritten digits (from 0 to 9). Automated handwritten digit recognition is widely used today - from recognizing zip codes (postal codes)\n",
+ "on mail envelopes to recognizing amounts written on bank checks. This exercise will show you how the methods you have learned can be used for this classification task.\n",
+ "\n",
+ "In the first part of the exercise, you will extend your previous implementation of logistic regression and apply it to one-vs-all classification.\n",
+ "\n",
+ "### 1.1 Dataset\n",
+ "\n",
+ "You are given a data set in `ex3data1.mat` that contains 5000 training examples of handwritten digits (This is a subset of the [MNIST](http://yann.lecun.com/exdb/mnist) handwritten digit dataset). The `.mat` format means that that the data has been saved in a native Octave/MATLAB matrix format, instead of a text (ASCII) format like a csv-file. We use the `.mat` format here because this is the dataset provided in the MATLAB version of this assignment. Fortunately, python provides mechanisms to load MATLAB native format using the `loadmat` function within the `scipy.io` module. This function returns a python dictionary with keys containing the variable names within the `.mat` file. \n",
+ "\n",
+ "There are 5000 training examples in `ex3data1.mat`, where each training example is a 20 pixel by 20 pixel grayscale image of the digit. Each pixel is represented by a floating point number indicating the grayscale intensity at that location. The 20 by 20 grid of pixels is “unrolled” into a 400-dimensional vector. Each of these training examples becomes a single row in our data matrix `X`. This gives us a 5000 by 400 matrix `X` where every row is a training example for a handwritten digit image.\n",
+ "\n",
+ "$$ X = \\begin{bmatrix} - \\: (x^{(1)})^T \\: - \\\\ -\\: (x^{(2)})^T \\:- \\\\ \\vdots \\\\ - \\: (x^{(m)})^T \\:- \\end{bmatrix} $$\n",
+ "\n",
+ "The second part of the training set is a 5000-dimensional vector `y` that contains labels for the training set. \n",
+ "We start the exercise by first loading the dataset. Execute the cell below, you do not need to write any code here."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 2,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "# 20x20 Input Images of Digits\n",
+ "input_layer_size = 400\n",
+ "\n",
+ "# 10 labels, from 1 to 10 (note that we have mapped \"0\" to label 10)\n",
+ "num_labels = 10\n",
+ "\n",
+ "# training data stored in arrays X, y\n",
+ "data = loadmat(os.path.join('Data', 'ex3data1.mat'))\n",
+ "X, y = data['X'], data['y'].ravel()\n",
+ "\n",
+ "# set the zero digit to 0, rather than its mapped 10 in this dataset\n",
+ "# This is an artifact due to the fact that this dataset was used in \n",
+ "# MATLAB where there is no index 0\n",
+ "y[y == 10] = 0\n",
+ "\n",
+ "m = y.size"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "### 1.2 Visualizing the data\n",
+ "\n",
+ "You will begin by visualizing a subset of the training set. In the following cell, the code randomly selects selects 100 rows from `X` and passes those rows to the `displayData` function. This function maps each row to a 20 pixel by 20 pixel grayscale image and displays the images together. We have provided the `displayData` function in the file `utils.py`. You are encouraged to examine the code to see how it works. Run the following cell to visualize the data."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 3,
+ "metadata": {},
+ "outputs": [
+ {
+ "data": {
+ "image/png": "iVBORw0KGgoAAAANSUhEUgAAAjoAAAItCAYAAAAwm9DjAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjMsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+AADFEAAAgAElEQVR4nOy9dbxU5bv+f3HOh9KPgaKgoqCgomKCXWAHdoEdmKhYGIDdgYGNiomFrVgoFthSdis2KrbU55zz/eP3u+65FnstNnvvmVmzx+v91/26956Z51nPs+rOJv/3f/8HY4wxxphq5L/yHoAxxhhjTKnwg44xxhhjqhY/6BhjjDGmavGDjjHGGGOqFj/oGGOMMaZq+dec/jhz5syqTMlq1qxZEwCYNm1arvP7r/8qPGc2adIk5P/5n/9p0Pe2bNmyCQDMmjWrKtevadOm/4j5Vfv5V+3z8/5snHD9pk+fXpXza9GiRRMAmDFjRlXOr3nz5k1m19miY4wxxpiqZY4WHVMaaMmZPHly6D799NOQe/ToAQDQGkeud2SMMcbUHVt0jDHGGFO12KJTJjQGZ8aMGQCAQYMGhW633Xar8b//+7//W6bRpaNWpKZNm9bQ/+c//0n9nM7VVC7//d//narnvtN1TFvTrBgz3Rd572Fjism//lW4ZdZ2ndO9r3I1Wuf1WsLjknX94LHQWNRSHxNbdIwxxhhTtfhBxxhjjDFVS9lcVw11ZzRGc5/OWcc/YsQIAEkzaM+ePUNuaHp5sdDxjxkzJuT55psPANClS5fQNfb1STOt1ve7aMYtp2m2NtS0zLF8++23NXQAsOiiiwIouFgBYMqUKTW+87PPPgtZ/3fDDTcM+d///neN728I6i7Lcp2lob/Pdcl7TdJI20dZ1Hf83N+VOP9K58knnwz5jz/+CJl7sUWLFqFbbrnlQl5kkUVCXnDBBQFkJ5tQrsT1ydqfX3/9dch//vknAOCnn34K3ffffx9yp06dACTvH1n3ymJhi44xxhhjqhY/6BhjjDGmaima64qmOzUn1zfSnJ/LMlM3lkwOHfP48eNDPvroowEUXFhA6U139WH69Okh9+/fP+QLL7wQQGWOuTZ0zL/99lvIP/74Y8g0rdY2J11fmmsB4LLLLgMA9O3bN3StWrUKOY9j9dprr4X85ptvAgDuuuuu0KkZukOHDgCAv/76K3RffvllyMyq+uSTT0I3a9askAcPHhwyj0FD3bFctwkTJoTuvvvuC/mbb74JmcdXrxNt2rQJmeffkksuGbq8ryncS1OnTg3dTTfdFLK6SbguG2ywQejUdUh52rRpoVt66aVDXm+99QAUXNBA8eZfW6ae0pjqhHF93nvvvdANGDAgZO5vdV3pMV922WVD7tixY42/r7/++jV+a5lllgld8+bNQ84jK5dj0vN84sSJIZ922mkhf/jhhwCSe/b3338PuW3btgCAG2+8MXSsHQeUJnTDFh1jjDHGVC0NsujoGy3f/jUA6f333w9Zn/7S6q9oYO5HH30EAOjcuXPoDjnkkJAZ4Jj3W1gWfOLWJ1N9emXg8cYbbxy6Snmj0Tf7cePGhTzPPPOEvPrqqwPIfiPjvsj6e551dnSfvfDCCyEPHTo05JEjRwKofU10Hs8880zIV1xxBQDg0EMPTf3fUq61zu+dd94Jed999w35hx9+AJA8f3R8XHc9v/V7l1pqKQDAzjvvHLpVVlkl5O233z7khpyjOqavvvoKALDXXnuFTquJL7HEEiHzjVmvM2oJeuCBBwAAd999d+jWXHPNooy5oeg147HHHgv5jTfeCJlrcfvtt4du5syZIfNarGumczrhhBMAAKeeemroagt8ro20a56OieieatmyZerv1xYsnkewLo/fkUceGToNNuZa6XVAraAqM6BZj8X8888fMo/FqquuGrru3buHvO222wIoWJ5LRdo1i9bq2WW1/jPI+Oyzzw6dWnx+/vlnAMB3331X5BFnY4uOMcYYY6oWP+gYY4wxpmqps+sqrZUBAFxwwQUAkuZUDfZU18d2220HIGl6VNMl63tosC7dVQBw8MEH1xhLpbh+gIJJUl0jDz30UMg0c2pbhUqpnaPBZmra1zoQXAt1DehaMPBMA+iaNWtW/MHWgTTTurpWf/311zp/l85f13rzzTcHUKiXAZR+f3JMakLWc1GDpXfddVcAyWBcdTPQJK/Bknrc2rdvD6DgwgKSezmr9H1dUdP+ww8/DCBZ+6dXr14ha2Ao3Vh6zLWOB102RxxxROg0MJsugXK6sPhbrVu3Dt3w4cND1gbAaYkfv/zyS8isaaLB5Hp86KY97LDDQrfYYouFXJ+9yv13zTXXhO71118Pmdf3eeedN3TqmtFzZeWVVwZQqOcEJPcf/1ddX7XVoSlFHSe6kABgs802A5C8pmgYh17/GNCsruUnnngiZK7bq6++Grrnn38+ZLq+su6PpYDH/8EHHwydXmuOO+64kBnsr7V19H8Jg+LLgS06xhhjjKla/KBjjDHGmKqlzq4rNd2pGZlmNDWxst4KkMyaUpdGDEQyBJ566ikAwNixY0OnZrx99tkHQLbpMg/UdcP6FRppvtNOO4XMrKVKyhrjumq9g3fffTdk7a7OY617Qc3kzExYZ511Qqc1ZfKYN9dHazuoOVgzKGrLCkv7Ls2KoRtXa2pkdXovNuqCufnmm0PWOh1XXnklgKQ5XdeE56Kek3p+8X/1M6WeH39roYUWCh2zhwBgxRVXDJnuV13H5ZdfPuTevXsDAPbee+/QqUn+5JNPTvxmOdHj3K5du5DVTVjb5zhuZv8ByfVhfRZ1FzUUHmt1wWjW5t9//13jM3fccUfIGrpAl5WeP1oTiK4tPT6bbLJJyJtuuimAZJ2gUoQG6HfyXOG1XXWz/+9aa60FALj22mtDl3Z8dP/qtXaNNdYAkLz/lRr+/tNPPx06dUctvPDCIdONre4srVN20kknAShvHStbdIwxxhhTtfhBxxhjjDFVS4OyrtQcR1lNWFrkSM2kmtlD1DRH06SaqSslKykLNb3SzcZS2EDSjcdsNDWt6ufz7C6sJmLN5NC11LUianpkcSwtYZ6WIVfOwoH8LZ2fombWuW39oK4hzUBcYYUV5up7GkrauTh69OjQqbtCz58vvvgCQHJ91E3A/00r+JYXHJOauxdffPGQda61Zd0wQ4bZn0AyKyvt+pQHdcle06w3Xn/OOeec0Ok5y/nrmjd0r3J8dIsCyUy/SZMm1fiMZiU98sgjIbO4I+8DQDJDi7+lYRLMxAWAtddeGwBw+eWXh06L65XyXqLunM8++yxkLSTIkIznnnsudFn3VaLFR48//ngA2a7lYpFW5FXvWVrkUGG2J0NQgGS4Cgvlquu81NcaW3SMMcYYU7XU2aKjT3n6dMcnbs2N17Lw+paU9vSpT7Gs6aC1W/SNi0/N5QzGSiMtABkArrvuOgBJi4jWseD89Y1aZQbuLrDAAqErdbAW56KWCW0wyLYBQOFJXgPotBw/P/fBBx+EjiX8gUIQYd7BnmoxVOsArWtZdZ7YzFItOv369QuZVs1yzo/rx4Z5QHLMWueHLRr0XN1///1Dpl7fwvK2qNJioftIkyH0WpFmKUxr5qp1PjTZ4YADDgBQKGUPVFbiANH11blwL2odM23LQYtAMRsl81zR5pV6zLt27VpDp3uKtZ2AQn0fvf4wgBUotPjQe8r9998f8llnnQUgGaw+bNiwkLXBbrGbmeqYLr744pDvvffekHnf0vXTc23LLbcEUKjHBSStj/zfcu7JNCupjl+v9WeeeSaApJVGazYxSaVcCRqALTrGGGOMqWL8oGOMMcaYqqXOris1N6q5mN25tdWDmkbr0gmav6EBdmPGjAmZXU818DnP2ixAsuYMWzwMGTIkdFounJ97/PHHQzdo0KCQWdq+mHUuaoPro7+pNSHUjMogxqwAMpo0ta2CuvE0oLRccH5agl7XTwM3uRZq4lYz62233QagUModSAYbsuaQBuuVOliQ59rpp58eOnVNsYUCUBi31jxRmW4E/S4tx1+uc01/Z8899wSQDHDUTuZak4QtKjRYc+LEiSHTpaDnrO5lti5YaaWVGjaBEsF9q7WrzjvvvJDfeustAEl3+ODBg0PmOV7q2jK1oecf1wwouKHUHaJ1kHhf0HuNtgMZOXIkgGTbnZdffjnkHXbYIeRiBcHyXNQx9e/fP+Q333wz5G+++QZA8lh17tw5ZLZQ0BYsGrjLc70ULS7qi9ZsYpC5XjOOOeaYkHn/KKc73BYdY4wxxlQtftAxxhhjTNVSZ9dVFmndU+trTkvLmqgkMx3Rcdxwww0h77777gCAAw88MHTqhqNrR9sOaAZZWouMUkM3gWZ6XX311SGr6+2jjz4CkDQna9YZa2nonmBtGSCfDB7OTzMFmR0AFNpWAMBWW20FANhiiy1Cp643uk/0u/bdd9+Q6bIq5z7lb+k+0rYjPXv2DJnZSlrnQ+s8sTS/ut7OP//8Io+4dvT40U2t4zzooINC3nnnnWv8rx4LrWnF809dWwr/nlb7qRKg60LbVtx0000hc/yHH3546NSNVc5slzmhx1SvCXRtsNUBkHRj8n81NCKtXYKub7muqTonveapG5huRNazAoDx48eHvOOOOwIotIoAki1O6ObSti4axqHZUOWCmahAYX+dcsopoWvTpk3Itbm+uZZzc/7N7Xlpi44xxhhjqpYmc3oimjlzZtleY/QplG9f+haq9TPYQFGrZdalmmmzZs2aAMC0adPqPD99ytQ3Iw0m45sUmwMCyTcWBqNpo8z99tsv5D59+tT4rbq8UbZs2bIJAMyaNWuuP5T2/bomOhYG8GllVW0GyjcSrb3DaqfFoGnTpnWeXxpZdZBYsVRrb7z44oshM+DwnnvuCd2aa64ZclqwYF3g/Ip5/qW98abVBgIKQb4M+gcKAdgAsOGGGwKov2WO51995qfHVJuqXn/99SFzr+k1Q6vw0tKl59zAgQNDZmXvUaNGhU6D9GtbV86voftT0bVizSq1Yr3//vsh09KqtZOKWTumFPuzLqRVZtdgega+qkVFLV6tW7cOOW0Pc/2mT59etPnpnqF1WK+PWlGZ+06TWfT6ROuUzkM9CmrpSbPetWjRogkAzJgxo87zy7pm6L2allRtAMraeEBh/2VVxueYtaZblnWO36XryPVTbNExxhhjTNXiBx1jjDHGVC1FC0ZuKGrGomtHS5hrTQGakfMuy67mSDWz0bSmZj41rTLwVYNFDznkkBrfW84AyDQzYtbx5bzULKp1Hjp06AAgWZY+rUFc3uiYtP4FzbAagK0tEljmft111w2dHqs8A1ez3J1pgZ86Zg1WZTA2S+kDwEsvvRQym/LlEVSu81N3ktYs4by0zoy2K6HJX83h2kCS56o2mlQ3V7nWN8u1yrmqu0qTCFhTqDE1Ra6NNNe57km27QAK66rJBlrzLY9gbB0/x6LtWtQdx3Y0w4cPD526uTh+bZGhyS7lQt1tekx5/9Omq7p/mQSh4Q56r/z8888BJBvB6vppyAqDtNVNmYYtOsYYY4ypWvygY4wxxpiqpWJcV2oOpklMy3Oz4ylQqBmQd20LNRfuscceIV9++eUAgOeffz502h2Zrit2EQYqs05Qbeg41XVFN4jWiagUd1UW6sbhWNXdMXbs2JDp0lJzc96uAY45bR5Aes2YLHei1p9J+3ulkOVa5bpoHSB17fBzai7XTuXM4GT2HZB0jeTRAmPEiBEhs52HZlJdd911IfNamfeebChZdXKY7dO3b9/QaWsXZkuqO7JcxyKrho2uJetYaaZUp06dQman8ylTpqR+F1sraEf31VZbLeRSuuayus/rWjErUN39eq/k3/X6qtcnXn80nECzqvVzrM8zevTo0KkbPsaXOSNjjDHGmEZOrhadtAAzoFB5VgN8NdiI1oM8LB9Zb8EauMmnUz65AsnAxm7dugGonADWYsO3Gg0gT3v6r0QrAVAYvzZd1WA41pHJOxg+7fzRAEx9y916661D5vHXAEFt+slmiPpGpXJj2au1WUlVp2/ESyyxBIB8Kszqm63WKRkwYEDIfGOl5RhIri9pLOs0O9zX2hT4mmuuqSHrnrzkkktCZrC8Uupzldc3TcDQYHgNnGbl8bfffjt0utfo0dBrjlYBP/TQQwEk6+jksdZ6/UhDrThaRX6XXXap8XftEkCLZLt27UL3888/p8q01OqxSsMWHWOMMcZULX7QMcYYY0zVUjHByGrGWn311QEkTZNqhk8rAZ4HWXVYLr74YgDZgVs0ozZW03IaOpddd90VQLIppgaLbb755uUb2Fyia8XAN6191L1795BZzrySgj0ZgKhtC9h8FUg2w0wjLVhZy7r36tUr5LxddsVC57HUUkuFzP2ppvlS117h/tO2FnfeeWfIP/74Y8gXXHABgGSdp0raiw2Fbhy6eADgnHPOCZmujUsvvTR0q6yySsjlqkOm14w///wTALDNNtuETtdSa8Lx/qWtDDSAdtNNN63xXVo7hveaPM5D3We8TwPJdjis76Nz0vsjXW6awJIWxqHr1759+zmOq7ZjURlPDMYYY4wxJcAPOsYYY4ypWnJ1XalpSk3nNCOrTmtGpLUgyBudSyWNq1ykuQGOOOKI0Gk5eppuK8kFomZoZkuwFDkAHHjggWUfU22kuU7vuuuu0KkbS+fH+kZvvPFG6LQcPVuTqItR2y1Ui5skqw7UcccdBwDYbrvtQqcZeOoyKpZ7hJk22grmscceC3mHHXYImVk3lXT+FBMe044dO4ZOW3ywfo7uWd2T5QoJ0N9h7RfNvh0zZkzIrH0DFFrkLLnkkqHTOk4M09BwDl3rPNdd56zj69q1a8hpNb30c9RrpnWpsUXHGGOMMVWLH3SMMcYYU7U0mZOZb+bMmWVLC9KCSePGjQOQ7O6tGQgbbLABgPq7iJo1a9YEAKZNm1Y9aU9Cy5YtmwDArFmzyja/tEKKmh2XZrqsb8HApk2bFn1+Ohbuq5EjR4ZurbXWCpndhUtlQub86nP+FSMjkccibc2KAc+/cl5f6gLdSLwOAcmCgppBkgbnV5f9yWPNYqlA0nXfo0ePGvq8sjYbsj/rQlYLiLSsnGIeC67f9OnTy3b+lbMFUIsWLZoAwIwZMyry/GsozZs3r3FjsUXHGGOMMVVLxVh00t6ov/rqq9CxLDvQ8BYQtug0bkph0VG4F/WNOq0OUqko1xtzXlS6RSfNIlmXdi31seiQtHpGQGUlOPxT9md9LDqNAVt0jDHGGGOqCD/oGGOMMaZqmaPryhhjjDGmMWOLjjHGGGOqljlWRq72YLNqD8aqpGA6tRx+9913AICFF144dNrgrjYYTFft+7Pa51ftwfLFTHZIK99QGxrYnFa5vb7lHZjsUO3Xz2o//6p9footOsYYY4ypWnLtdWWqGy0C+e6774Z8+OGHAwCGDx8eOi0f4Lix8pNmMUgr0qZyfS0C5ULHlzbWSuwVldWzKS3tPMvKQ3no0KGh69SpU8jdu3cHUJnz/yeQdn6lUc4igtWOLTrGGGOMqVps0TFFh2+f2v17r732CrlXr14AgMUWWyx0fmMpP3rM09p1/P3336HTGKpitJkoBvpGrBYPxqCwlQMA/PnnnyGz6/ICCywQukrZf1ltUxTqs9ZhwoQJAIBjjjkmdGussUbIjz76KABg/vnnr/W38oDzyrLI8e+1FXFU61je6Php6dbza8aMGSHzXGNHdKBuBSvzJs3iW1sLj1JbhyvjimWMMcYYUwL8oGOMMcaYqsWuqwohzXSnpuksk3al9MDR8X3zzTcAgIMOOih06667bshHHXVUjc9UojlWx6eB1TzmlTjmuSHN9aFm/vvvvx9AMlj81ltvDXnBBRdMfE850TGr6V+D3d9++20AwNNPPx26J554IuSFFloIAHDHHXeEbv311w85zyDd2gKoVa9r9thjj4V87LHHAgBWXnnl0PXr1y/keeedF0D++zfNHQUUyk/8+uuvodO1/uSTTwAAHTp0CJ26Iema1L9nBdaXkqz1e+mllwAAF1xwQeh0fssuuyyApOuxc+fOIee9bmnUlsyga8n9l9XXrRTYomOMMcaYqsUPOsYYY4ypWirGdaWugYZGYKsZjWbovM19aVH3aiJXMzTNmI888kjoRo8eHbKapGnezMPcrmumptcTTjgBANCiRYvQnX/++SHPM888ACqrjkeay4AmcgAYNmxYyHQDLL744qFLm4vq9Pvznnfa+XXdddeFPGDAAABA7969QzfffPOFnMe5xDFPmzYtdBdeeGHI11xzTcgzZ84EAMyaNSt0Ov5ffvkFANCnT5/Q3XzzzSGvvfbaAPK/ZmRBN4+6q9RNvM022wAALrnkktAtssgiNb6nnFkvab/z22+/hXzllVeG/NRTTwEAfvjhh9D99ddfIf/0008AgLZt24aO1xSgcC3t27dv6A477LCQNYOw2GucdRzV9XvnnXcCAPbee+/QsbYRUHDd0YUMAIMGDarxG3nvz6ysTbr2x44dG7pLL7005CuuuAIAsMwyy4Su1BlytugYY4wxpmrJxaKTFqz0888/h8wn/bq8ZejTJYPRAKBNmzYAyhv4RHT8GjT89ddfA0i+kb3wwgshjxo1CgDQrFmz0K2yyiohb7jhhsUfbB3gvPQtSwPrvv32WwDAPffcEzoNFszbokGygr3HjBkDADj00ENDN3ny5JD5Rnn00UeHrlWrViHzTfStt94K3XLLLReyBr7SKlbqt7O0Krta50gtbrSadO3aNXR6TjW0V1J94HH64osvQqcWHYXWmwMPPDB0W2yxRcjnnnsuAODNN98M3ZAhQ0K+4YYbABSCJoH896zuz/feew9AMsB4tdVWC/myyy4DkLTipFVczqOytf6mWufUek2LtVo8dPxrrbUWgKTF+Pfffw/5+uuvBwCceOKJoVOLzyGHHBJysc67tHuaXtOffPLJkGndWXLJJUOndZ5effVVAMl7xmeffRYyg6zzCLDOsuKo9bR///4AkhadW265JWTOu5x1jmzRMcYYY0zV4gcdY4wxxlQtZfPnpLlxHnroodAxQAkAPv744xqfqc1Mpzo1OTPwa7311qv32OtKmhnz2muvDfmcc84BACy99NKh22CDDUKmG0Frz6y44oohp7kRSo3OhcGeGiCnwdIjR44EUHAbAumBuXkGtQKFoFQAePDBB0OmaVmDITXwkcGSWqdFTeM8PgwqBIA999wzZF1XDeguNllmZgaOcx8CwJQpU2r8r7pL8whcVWjmXmqppUKn1wy6S4HCubTxxhuHTsvpv/LKKwAKLgKgMhso6nFW18wZZ5wBIOn6ueqqq0Ju3bo1gOS1YW7q85QDvQ5oCxh1HTIId5999gmdtqvgd2TVVGLNJA1X0GD0UsBjqq0c7rvvvpBPOumkkNnAWNdHA7PpUtbry8svvxwy61tluSZLQZq7U39TXd8cv85f73X8XF2uKQ29/tiiY4wxxpiqxQ86xhhjjKlaSuq6yjJzMVvivPPOC53WNqDLQ83UK6ywQsjMv+/YsWPq92uGz6qrrlrj76Xuvsx5q2tOzXg002633Xah0wwCdU0RHX8ebR/095nVcPfdd4dOM8iYFUAXDpA0s/P46JxL7S5IcyfedNNNIZ922mkhsz6OZvXsvPPOITPbbPz48aGbNGlSje9deOGFUz+v2XSlzObJcvfefvvtAJKZLl26dAmZLTy0TlDecPxZ2TN6HNMyLNX1w8ytrBYfebp2srjrrrtCZvfxU045JXRpLQL0OKR1v87bHaljYiYVUNiLtZ0nmql79tlnh8wwgYMPPjh0eq0t5bVGr806fm1HwbnonBZddNGQ6ebSrDqtc3XjjTcCSGaVlSIDK8v1TS6++OKQec0ACq41dRemudayMqF5DPXvWRlec5tBaIuOMcYYY6oWP+gYY4wxpmopqetKzU0ssgYAZ555JoBC9DkAjBgxImSaLtWdVd/slHK1gFAz25dffgmg0EUYSBYvoxlVs8MqvSO5dodmOW92IQeSWWH838cffzx0LIIIFLIFNGtGsyrKlfWie1L32sknnwwAOOCAA0KnplF279aCXzpmdurt2bNn6DbddNPU/y0laZlyAPDOO+8AAM4666zQaQYSs2E0q6NSMpGyxpHWiV1dwB999FHI3JdqWt9kk01CzrNFia7Zjz/+GLK6UZlBqgUr9frIDJ73338/dO3btw+Z65rl7si7kCDPL3V3aAYjszrpggWAqVOnhnzbbbcBAHbaaafQ6f4o5V7W39FzTrt3q0s77XMc37///e/QqeubBS/1PpEW7tBQ0vaBhmOwcCUADB48OGSeV1ktcCh/+OGHoXv22WdDZtZWu3btQqdtj7Tg6twWArZFxxhjjDFVS9EtOllPzvp0utVWWwEolNoHkgFG/F/VlbNc9NyiT6naDoGBrVlN5wYOHAgAOPXUU0OnT/l5vj3r+umcWNYbKLQG0LfEPfbYI2TWQdJgPD0WtO5oAJsGkJdi/mnfqXU6dt1115D59pDWHBYoHCOtw6Mlzvn3rbfeOnT6xpVHuXZ986FFVYPBX3zxxZC5F0sdtF9M0uaqTVk1cJI1V9Qit+WWW4ZcKXWevv/++5C1PkuvXr0AJK8Zzz//fMi0/rBeEJA8V2np0HNaz9U80POLFje1Hrz22mshf/rppwCS9wetU0WLuVq5Sm2d417Rek2rr756yBdddFHIrDmjAfJvv/12yKxfpXWGtOksLdGltrzp9/M3aU0CkgHSOlauix5/nevVV18NALj88stDp8kqPXr0SPwmUAjABoD9998/5OOOOw5A7evbeK5kxhhjjDF1xA86xhhjjKlaiua6oplLS3GruVwDbxnsuc0224SOtVmAQon8PLqz1oaa89W0rJ12l19+eQDJYFwNHNt2220BJMvus2x73qiLUINV1TRO16J25918881DpplVa2No4NqOO+4IoHwBgor+zkorrRSy7rW0AHb9O8etwdQabM35aQBhHmSNf6GFFgKQPFe1DhJdHxqgnXf37rpAl6LWVlE3COsDaR2arHYl5UbX6fXXXw95+rYK1bQAACAASURBVPTpITPw/4MPPgidul7pBtCO5nqtohtFz9m111475HKdi3r+q2uDrUnU9aHB/Ndccw2AZID5CSecEDKTQLQ2VKlJq+ei7jQNtqUbX93Zur68V2oygLqZea8shTtcx6/XB9as0qSabt26haxuRH6Htq3QYHqeX+ru7969e8h0o+qc9FyuTzsPW3SMMcYYU7X4QccYY4wxVUuDXFdpnaDVRKV1ZLRmDtsyaG68Zkj8+eefAJImqrwzINLKpqs58dBDDw2ZLjnWgwAKZecB4OuvvwZQWVktNBOri0rdiWpGputNo9832mijkGl6VHOmtsCga09bfOSxvrW5KHTOmoF2ww03ACh0OQeS7Ui4F9S0nHdNFoXHWs85zXqjy1Fdz2ll1/OuvZLVvfqYY44BUGiVACRd5wMGDACQrJOU1iIhD3Qc6ppRN2KrVq0AJF3Lei4xm4XXWSBZp+zwww8HkMwazCNMQOeqWbn33nsvgGRtLc1a5fVFXXualVYprkdtUaHn0qBBgwAkO3p/++23IbP+zptvvhm6LbbYIuQNN9wQQDIcolhrptc8rXPDvdKnT5/QpbnzgULX9b322it0eq3cbLPNACTXVOfCtWQXeyCZzaZhInO71pVzpzXGGGOMKTINsujoUxyDGbU5mebW69MfA880AI1vKbN/b56kNQrUJ2d9C9Y6Mnwq1gCzK6+8MmTWV8jboqFr8sMPPwAovO0CyWA4voUAQO/evQEk38JohdPvYnM6IFnTg81A9Yk+72DXtLcTrWaq1iu+say55pqh0zoZDOzMe05ZTfm4l9V6p9ZH/q/W1tEqvQxi1besPAJY9fxS6wYDjzWwXoMoKWddZziXvNdPz4+0yrJqUdb/5fVFr6k6F1oa1aKed7KHrgXvG1lNodmMVq+pWtOFDaDLWXuNY1XLhDYNphUGKDQbzZof9Z999lno1LrOa20pzj8dk1ajHjt2LICkx+aggw4KWRNTGBh+1VVXhU4t3qzYrdfXp556KmReazSYW4PtteYTj5ubehpjjDHmH4sfdIwxxhhTtRTNdcWmgGq6Y3MuIBn4yKaQWgeib9++IdMkl4c5VYPGxo0bFzLHuv3224dOg+V03gyS07LdGozMcvRqbs6jxYWOmW0rNACuX79+IbNtB1Aozf3AAw+E7sknnwyZLhGtU3PnnXeGTJN53q6BNHcAUFhrbRugrje6KVnvA0gG2+exljxX9JzUYGh1LdIkrXNiWX2gsNZaB0kbSKa5cUtNmjv49NNPD3no0KEhc3x6LPRaRDc7g+L1+4HCec16Q+VE96Ga69U1M2TIEADJBAhtZ3LEEUcAKAR9AoWy+0DhvNbQgjwazdaGBsBfcsklIfP+oe0I1M1RrkbOCuel+1N/n7W1lKzrBL9L2+botfKPP/4AkLx/FAu9J7AVA1AIZtfacNrUWa//dJmy3hGQ3gBZ7w/aqJO/q/PTY5nm5qsNW3SMMcYYU7X4QccYY4wxVUuDXFdqTmKkP7uJAklzltYUYJ0ArSOgJbxpcs7DtaHmbu2uve+++wJImkg1klzdcHR5qTtOzazMcMnDxaGmPo10p+tRs+ZYzwJIuvG4LtrWQc3kNLOzlQeQzFDL02Wl66vjYG0coJDN8dVXX4XusMMOC5l7VbPS8lhLha4XdRGzSzIATJ06NWTu63fffTd0ajpmi5Z11lkndFrTiubtUtfR0e/kmAcPHhy6YcOGhawuZ45L10RL8DNDpF27dqm/RTek1p7RDMNSukR0T3bu3DlkddMxM3LSpEmh0wzWe+65B0Cy7QVL+AOFDMpSt2BJczdovRtdHx0Ls27Udaw1Vei6Yxf32b8rT9Kyi2dH3aRpnyPqTtbzWvd6sdH9p1l9bGfTs2fP0KmbTjO0WPNJP69hHiTL9cbrS5aLqj7XGlt0jDHGGFO1NJnTk/zMmTPn+JifVgdg+PDhodMAQK1MuswyywBINpLTYKVSP503a9asCQDMmDGjxvz0zYJBXwBwxx13AABeeuml0HXo0CHkrl27hsxgqoUXXjj190s9v+bNmzcBgOnTp89x/dIqa2ZVSE2zwqTVZgHS31iKacVp0aJFE6D2/akwMJdVqQHg+OOPD1nfflnf6MgjjwydWnQ4v1K92XN/1mV+HNPEiRNDp3V+tE4Oq1jrXqZFDyicn7pPdf0aar3h/GbNmjXH+emeYk0OtfzqNUX3MmE9ldmhJZKW5dnhm6ZaafVcrm3dmzZt2gQApk2bVucNklX7SPWsaaLJDjo+WuK0Tk6nTp1C5vrVd01btmyZef3U71GLPq1vmpShqPWD1ketBn3iiSeGzGBrvc4U81zk9bMu51/aMb3ssstCVuswg3j1nFRL1xtvvAEAuPbaa0PHat9AoUpyfS1y9bm+pJFVGTltTGnjy/p7sa4vii06xhhjjKla/KBjjDHGmKqlQa6rxBf9/+amNLfF7KQ1BSxngOqcXFeKmtA4ryzXTtpc8iqrPreuq8bK3Lqu9PhPmDABAHDBBReEbtSoUSFrnQu6R7TOQ9b3loKGmJb1PBo/fnzIWo6d+l122SV0rKMElG9+tbmu0srRaz0VrdOx0047hbz77rsDSLrGlZdffhlAwUUAAF9++WXIdPOceuqpodOaRLXRENfV3EA3QZbrIu36Wsw1TXNdca3U3anuJrpJ04LiAaBNmzYhs0UHW80AwLLLLltjHKXap/VxXRFdE3WN6l4dOXIkgGSwrrrUmdih7nK6k5X6zr9YrqtKxa4rY4wxxvyj8IOOMcYYY6qWormuGhNz67pqrPyTXVdZWXOsI3LfffeFjvViAGDgwIEhswWJuoHK6Yasj2k5LWtO3ch///13yNOmTQOQrA1TytocszO3riuFc9HsFO1+rHV+5rY0vroWNGuLxyKtNs/cUGrXVd7MyXWlbRvOO++8kB988EEAyXXaeuutQz7ggANCphtL3YVpWXWloiGuKyWrZhfbsej+0/3FmmPq2lKXYEOvRXZdGWOMMcZUEX7QMcYYY0zVYtdVFfJPdl0pajr++OOPASQLA2oJ+fbt24dcTjN5GqUwLaeVUM/KeixFOwelPq4rklUKvj4ZRrXNs74ugn+i6yqN+pbtT8saKyfFcl0pacci65hw3qXKRLbryhhjjDGmirBFpwqxRacmDGbVAF213OTZaHR2/ilvXPWx6DQGbNFp3JTColNJ/FOuL4otOsYYY4ypWvygY4wxxpiqZY6uK2OMMcaYxswcq4Tl7WPOagPf0F5S9DFXe4xAtftgvX6Nk39KjJz3Z+OE+7PaYxyrfX8qdl0ZY4wxpmopX933OkBLjpbIfuGFF0Jef/31ASRLvVeiCy6rjkQlZfiY/4+0tfonrVNdap78k45LYyGr3UA1wrnWVkepEu8J1URjumbYomOMMcaYqqViLDr6RMgGZvvuu2/ovvjii5AffvhhAMC8884bukp6eudctBGbNiPkuPN+yi0VWquG6LGoFHTPqfWQNG/ePORqXCu1AkyfPj1kNgDVNWOjU5W16WXa+QtU1nlZbeie1Aa2tHSnnYeNiSyLAZtiaqNate5rs1pSifswKwaVsq5vJV4/064ZQGHcuv/YqFT15bym2qJjjDHGmKrFDzrGGGOMqVpydV2p6W7WrFkhn3HGGQCAp556KnTXXXddyIsvvjiA/JsvZsF5ffTRR6E766yzQr799tsBJE3/lWharQtqeuW81V230korhVwpc9Ux33bbbSG/9tprAIDLL788dGoOb+xuLM57ypQpoevfv3/IPO9+/vnn0O2+++4hX3/99QCAH374IXS613v06BEy3X+lXvO6BEby77r+aQ1OK3WdOf6pU6eGbssttwx5yJAhAICNNtoodJXo+siC66L3hOeeey7k0047DQAwceLE0K266qohH3XUUQCAPfbYI3TqRsn7+sP1U3fPl19+GfKPP/4IAGjbtm3oOnXqFDKPTx7zGDVqVMg33HBDqv6vv/4CACy88MKhO/zww2vIiy66aOhKPRdbdIwxxhhTtfhBxxhjjDFVSy6uqzTT8plnnhnyNddcAwDYbbfdQrfTTjuF3FjMsDrO9957L+Rff/0VQNJ011jmpOg6qpvq1FNPBZB0YXTp0iXkvE3HaeiYbrnlFgDASSedFLrOnTuHXKkujTmhpvvJkycDAPr27Rs6dRPvv//+AIB27dqF7vXXXw/5xRdfBACcfvrpoXvnnXdCfv/990Pu0KEDgOKteVamirqxmZWTlkkHFNxso0ePDt3yyy8f8hZbbAEgmSlSSXuWY9HxrbnmmiHff//9AIB11lkndJXuJk/LALzgggtCp6ELXNf55psvdB988EHI3Ne//PJL6NR1kuWyLCW6b5khx+skADzxxBMh//bbbwCS89MM5OOOOw5Aed3p/P577703dN99913Ieq1s1qwZAGDcuHGhu/DCC0Pm/U91/AxQmjWxRccYY4wxVUsuFh0+sY0dOzZ0DHAEgKWWWgoAcNFFF4WuRYsWITcW64cGY+kTa23Bko0FfTNiAB0AfP311wCAXXbZpexjqi8azNm6dWsAwNtvvx26ZZddNmSuXyW+GStZwf5Dhw4FkAwgPOKII0I+99xzASRrk4wYMSJkvlFq7RbqgOSbZrEtOfp9n3/+ecga7M9gcgZFAsk33t9//x1A4ToDJN9Od9hhBwDJa1IeVoDa0DpPatF5+eWXAVTOOLPQY6rWt4EDBwIAbrzxxtDpXI8++mgAwDLLLBM6DZxnkOzFF18cunXXXTfkbt26hVyue4laVL/66isAwKOPPho6WjkAoGnTpgCABRdcMHT0cgCFfa86vT+Wct379OkTslpBW7VqVeN/9dheeumlIdOSdeihh4ZuhRVWCNkWHWOMMcaYOuAHHWOMMcZULWVzXanpjjUD1HSlgbl33nkngKTprrG4q4CCmZ0uHCBpmq1G11VauwsNUK5E1ETK2kwA0LNnTwDAMcccEzoNrKaZttJdA4ruP7rk5p9//tDpXKmfMGFC6M4+++yQ6ebRYGS6E4Bk4GuxgiS51xhIDSSTFTTYn25i3X/qrmDgpLq+dC7q5qtk9JxTN6se/0ojq+3K4MGDQ77pppsAJN0xdGcBBfeJ7l/dZ6zvpN+pwfS6F0rphta5qhv16quvBpB0V+nvM+RBA39VZuLOJptsEjoNVi7FvZLnn7oAs44Z500XXNbfy3kftEXHGGOMMVWLH3SMMcYYU7WU1MappilmOgCFOgfTpk0L3dNPPx0ya5aoCU6/iyazSnUdcFzqetOxVmrrijmRZmbUFgAaVc+sgkp3N+qaqJmVrqv77rsvdM8//3zIzCZrjPV0gIIbOcvF9MorrwBIZmKpmf2QQw4BkMzAKIW7Kq0jupad13o96uZg1td+++0XOs0KW2SRRQAkswLVtcA6OpXUNiANHZMeC81GqhS4lro3tO2KXj+4l9Q1vv3224fM9VXXl64Vs+Y0a+uNN94IWbtuazZssdH1ueSSS0K+6667AABrrbVW6LbeeuuQt9lmGwDJ2l2rr756yDyGuublcgPp+mWFLnBdeB0BknWQdt55ZwCFGluAW0AYY4wxxtQbP+gYY4wxpmopqetKTVuPPPJIyC+88AKAZKS4Fif74osvAGSbs1j6XKPuK8m0zLEstthiodOCZHTtaKZPpbt51DTKcuXsEgwA33zzTcj9+vUDACyxxBKhq6T1ITomnR/NyJqdwY7Q+nd1l1SiG0vnlJaVouZ+LV7GQmvzzjtv6PT8XWWVVQAkz+9Sz5+/pUXK9PfVDb733nsDAM4444zQqWvyrbfeAlAorAcAHTt2DHnTTTet8f2V6G7Wa4aOrxLPNR5LdQFrpttCCy1UQ//mm2+GTuWll156jr/FY6F7Uvd6KdHf4T4DksX92Jpj+PDhoWORUqAwbs3+UzcW5byzd7Xj+rBhw0J+6aWXABTu40AyA5kZ1j/99FPo9F5Yikw4W3SMMcYYU7UU3aKjb0H6xHfllVeGzHLe+kbVu3fvkPlEq090GnjWvn17AMBVV10VujZt2oRcKW80+kahb1yV+Pafhq7lPffcEzJLeGsjN11rzlXfotMCSyvpOOieadmyJQBg7bXXDt0DDzwQMttdcB8ClTUXonPSoMsNN9wQQKFeCQCcdtppIdNSdfnll4eua9euIfNNs9Rz1vFzL2qrjpVWWinkiRMnhvztt98CSAaT6+euuOIKAMm2AQywBgp1TCrVyspzSduuvPvuuyGz1omev3mgv//xxx8DSNZb0gBwDUbmvUAbOWsAcdq+0+vLq6++CgD4+++/Q7fyyiuHrO0kinWv4FzZkBNINiXVvXT88ccDSFqx0iyGOidtJ0RPQR77U8f07LPPhqzXCgaTa1A8EwCAQk2gZ555JnRHHnlkyEwi0HVqKLboGGOMMaZq8YOOMcYYY6qWoruu1BSopnEtUc7AKzX9H3TQQSGnlY5W1wndCGqubdu2beoYKgU1+eVtUp4TGkw3fvz4kNXkfMoppwAo1JsBki0EOFd2kQaSc2bgmbob80b3DGWtw6J1IGiy3X///UOXVucpb9LccUChjgXreQBJNwKTBDbbbLPQ5R2MS3eFmvC5D4FkTR8Ga3KeQLLmyOjRowEk284ceOCBIdPNV6muK55L2vbizz//DLl79+4Akqb/crW10PNcx8S1+uijj0K3zz77hKxuKq617lmV+Xc95zQYfdy4cQCSAb4bbLBB6hiLva/Vxf3UU0+FvOeee4bMdjK1uX71WqzJHh9++CGApDu5XOg1RdePtaeAwjFV15yuFZNxNPRB7y9TpkwBAAwaNKhYw7ZFxxhjjDHVix90jDHGGFO1FM11RTPbZ599Fjp2IQeS5Z5vvvlmAMnaAFoTJ821o1kVNPlVqmk5rTvrH3/8EfL3339f9jHVBzUHq8mS2W5ajl9N0jQZax0FNdOynLu6HvKuCaFwrura0DpPdBno/qsUd6QeR5X1/GH3Yx2/ugbeeecdAMnu31pTJw/XHH9T56RtAbRT9YABAwAkXeDMxAEKdYROOOGE0GlWSCVm0Ck8BmPHjg2dZqXmuRd1fTRcge0AVlxxxdDp+a9utrm9rqtrh5l2QGGva70dvf+UokUJ68RoppW6WdVNw3FnjYPfq261W2+9NWRm27HVxZy+q5To8deaabXRpUsXAMB5550XOs3A5vyy6pzVh8q4OhtjjDHGlICiWXT4xKV1BBh0BBQCsIBCHY+sADnqWUEZAC677LKQ11hjDQDA+uuvH7pKCQAFCmPRyrn6xPv555/X+EylBLPq29Saa64ZMgPggML66P9qMCebJWowb9paV5IVR+HxVytGp06dQmaQtlo81OKTx/qlNU3UOhcaLM4GgaxwDRSqWQMFi84vv/wSOj0WeZJ1bFkNGShYco499tjQaU0V/l0TIPRcLVfgbn3hMVCLq9Z8Wm211QCU1+KdVs1WkxkYYKrrpOdUXSwStCRk1eH59NNPASSbtmY1WG4IatGgxUWDhtVitdRSS4U8t0HI2uha69AdfPDBAJLByHlbIevSdJtjXXLJJUPHRq1z+lxDsEXHGGOMMVWLH3SMMcYYU7UUzXVFc5MGlc0333wh//777yEzYFlNqwxWAwoN3G655ZbQaeAVS7irOT1v010aatrUwNbGggY1tmrVKuS0YDkNvGOwmQa4siy4UknuxjTS2lYAhWZ0ee+5NHOxBkNef/31IWvgLcut65pozQu65KZOnRq6du3aFWvYRSOtRQQA7LbbbgCAhx9+OHTalJTHTcvWsyw/kB5Ymfde1WsJg/zvv//+0O2xxx4h81qTR+0jDYqeNGlSyNxrek/QNUtzs2XVHuO+1KagGqzL+l4nnnhi6udLcd6yZpAGW6trVNePc81y3dPld9hhh4VO2yn0798fQPL8zbsdhF4r6AbW63/auapuPnWTqxurWNiiY4wxxpiqxQ86xhhjjKlaiua6ojlw2WWXDZ2a7q655pqQaVpUc5tmaNEkp2b4Xr16hUyXVd6ug9pQ0/HkyZNDrpSaK3VBjzVNlpqdoplkdHPp+uZt+q8Pam7ebrvtQr7wwgsBJLtfq0m+lOje+fXXX0Omm2ro0KGhO/nkk0PWDDjWAdKsjhdffDHkjTbaCEChSzJQ+euna8WssZdeeil0bIsAFErLawl7NZ1fffXVAPKvHaToucR2ON99913oNt9885DzHKuug7ZgSKt9pq4PdcPwf/WaozW56LLSFibaYuGiiy4CkDwnS3Gv0LmwTpq6gLNqw6VlzT355JMh042qn9cWSGyhk4e7SvfWfffdF/KwYcNCZoY0QxiA5L3ik08+AQAMHDgwdLpv2LpFdQ1dv8Z3xzXGGGOMmUuKHoysT+asUKp/BwqVD9k8D0g2iGQdCM2z1ye6SrfkpNHYx5+GrukCCywQMqvM5v0WXF84bq2sqxYdvomoRUcrr5YSfQtmUDRQeKPS2j5amVzljz/+GECy6eI666wTMi1WlV4tOKv2FGs+qZVGmzryGOnfNZiyUuaq89M34g8++ABAMkC1W7duIecx/rTr/1prrRUyA1M1gFrrwGgVYzblHDNmTOhGjhwZMve91uQZMmRIyAxmLYXFI2tNuH/U8jRhwoSQaYUBgDfeeANAoUMAkEzGYTNdJt3M/vk8LDm0ruj8GBQNJJOQaN3SBqe67kwMUIubVjbnXi7mPrZFxxhjjDFVix90jDHGGFO1FM11RdTcpKapiy++eK4/RzNopTbtnFvU9dG7d++QtSZNY0bNuG3atMlxJKVB96TOj6ZVuoCApBm+lGSZzjlWLYuvzWM1MJ4tVLQsvgYGshZIYzr/dH507ag76/HHHw+ZiRE6/1NPPTVkuj7ydr1mNTWka1/r1EyfPj3kUtQhmVt0zAxqBwrtekaPHh06DQZXlxddqhqMuvzyy4fct29fAMARRxwROg0cL+W+1fmpu4aubW2rsvvuu6d+B2vKtW3bNnTa4JL3Cg0HyPtc5PVF3dlHH310yNddd13IbPGkx0rbfRx33HEAkvdE/btbQBhjjDHG1AE/6BhjjDGmamkyJzPRtGnTGmfaTC20bNmyCQDMmjWrpPNTc7Nmw1CvWWfFNNc1bdq0CQDMnDmzpPPTMWun7M6dOwOoW8feutCsWbOyr5/OlZ2wtTZGVrnz+jC366fmbJr71YWjrlMdE/edjl//t9R1nrh+M2bMaNCBymoRcO+99wIA9tprr9Bpd/kePXoASGa1tG/fPmTu1fquY/PmzUu6P+myVHeVum5KvX5zuz91fehS1doxc/M5om4izrVUbR24P6dPnz7X68dzUWs3cR/OPj66jrWtkbqxOP9SZc+1aNGi3vszqy2Ousy1DQjR9aNrNa0tRjHg/lRs0THGGGNM1eIHHWOMMcZULXZdlYm0EuClyuool+tKyXLzlIJyua4UnV+aab2YZub6rF993BW6TuXMMCqW60pRM/iUKVMAAMOHDw9dx44dQ95iiy0AJN2NxWxXUmrXFfef7sNyFglsyP7M6thdG2l7tVR7tj6uK1Jft2E5168hrqss6jLvUs/VritjjDHG/KOwRacKycOiU07ysOiUk3/K+hXToqPw7VJrs+h1jgHbpbIIlNqikzf/lP1ZH4tOY6AUFp1KwhYdY4wxxvyj8IOOMcYYY6qWObqujDHGGGMaM7boGGOMMaZqmWNTz2oPNqv2YOtqX79qD6ar9vWr9mDPUgVb5w2Drat9f3p+jRPOT7FFxxhjjDFVyxwtOsYYY4wxxYJFI9OK6AKlKShoi44xxhhjqhZbdEqIlqXXp9dSFywzpq5oaX4W2stqMaAd0k3lwWtNbWX5s9qasDs6kM81Kq3FhVLOdgnFQu8FKpPaWpA0xjkruhenT58OAPj9999DN99884XcokULAMXde7boGGOMMaZq8YOOMcYYY6qWinFdpZkpG5NrJ238kyZNCvnjjz8OeYMNNgAALLrooqFr7KZJnT9Ns1mmZ5ppK33Oxei0XMmoOfnvv/8Oedy4cQCAb775JnTLLrtsyKusskqNzzcmuK7qQtC92Fj2pe5PXYvffvsNAPDTTz+Fbtq0aSE3bdo08X+z/2+PHj1CbtasGYCG7+naxqzfT9eZutD0c2mujUo853TMn376acgTJ04EUDi2ALDGGmuEzPWZZ555QjfvvPOGrG6uSkbPL3VT9evXDwBw++23h27nnXcO+YYbbgAAzD///KFr6DnZOK9UxhhjjDFzgR90jDHGGFO15OK6SjN5qzmOZio1fakZME8z5dxkArz++usAgD59+oROTZdnnHEGAOD4448PHTNdgMo0wyppdRD++uuvkG+66SYAwCuvvBK65s2bh3zMMccAAFZdddXQlctdkGU6T8swynJnpI1Vv6sSXTppGTaffPJJ6AYNGhTyc889ByB7TZ588kkAQLdu3UJXLHN6ViZQffZH1pp8/vnnAIDLLrssdL169Qp53XXXBZC/iyBrr86YMQMA8OOPP4bugQceCPnpp58GkHQ9qmuSLpOpU6eGbsqUKSEPHz485N133x3A3B+LrPXjmL/99tvQTZgwIeT3338/5DFjxgBIuq7ozgGAHXfcEQCw//77p/69Uq6fumaPPPJIyP3796/xvyuvvHLIdFm1b98+dH379g15nXXWAVA598TZ4X37+++/D90RRxwR8jPPPAMAWGuttUL31FNPhcw9suCCC4bOritjjDHGmAxKatHJerpnYByf3AHg7rvvDplvJPoU3LVr15DzqOPB8etbBt9sAeDSSy8NmW8nOs6OHTuGfO211wIAHn300dDdcccdIS+99NIAKisoMu3t4bXXXgudWgT++OMPAMkn9pdeeinkk08+GUBy/mq9K8XbCcevx/S9994LWYMxf/75ZwCFoFwAmDx5csjcnzrOAw44IOSz7IFw0gAAIABJREFUzz4bQP5WOl0zzgkAbr75ZgDAbbfdFjoGGAOFdVlyySVDd+SRR4ZMS+QTTzwROgaIAvWbKz+jlgfW2wCAVq1aAUi+uWfBc1SDbWnFAYBLLrkEAPDggw+G7t///nfIa665JoDsYNlyoev31ltvhcxzTQM8O3XqFPK+++4LAFhhhRVSv4vyhRdeGDpa8QBg8cUXD3lu5512fRw2bFjII0aMAJC0IioLL7xwDVnH/+eff4ZMi7DO6ZBDDgk5b0sc0WvNnnvuGfISSywBABg9enTq/9ISp9dHtY7cd999AICFFloodHnPWc8VrtUpp5wSOr2Xb7XVVgCA6667LnQMQAYK56Lr6BhjjDHGzAV+0DHGGGNM1VI21xUDkADg8ssvB5B0XWnO/Oabbw4AuPPOO0Onrg0GsZbTXEfT3BdffBG6Y489NmQNJqNpVWuP6N8ffvhhAEnT8f333x/ySSedBCB/11WW65Fruffee4dur732CvnUU08FkHRn9OzZM+QVV1wRQHldAxz/q6++GrpddtklZHVdpdUB0hLlXEvWwwCA7777LmS6LCspQHLw4MEh03WjwfB0twGFYFU95w477LCQe/fuDSAZDLvUUkuFPLdz1fXneaXjUHcTa0917tw5dFkB4qxZ9e6774Zu/PjxIdMlpu4quluBwnUlj6By/U0NJj700ENDXm+99QAkrz/t2rULWeuzEL1W8hz44YcfQsegYyDpcp5beK6ou/6WW24JuW3btgCS17mWLVuG3Lp165DpptR5qJvus88+A5B0varrqlLQ80Brpu2xxx4ACucRAHz55ZchX3TRRQCSe3a77bYLuRQtEupDVrLAjTfeCAC45557Qrf22muHzDAPuvAA4LTTTqvx/cW8v9uiY4wxxpiqxQ86xhhjjKlaiu66UtPrs88+G7KWeF5ggQUAAHfddVfoNtxww5Bp0lRzrebhP/bYYwCKm2dfGzSjdejQIXSadaRmWJoW1XWjWQN0c+hnunfvXuO38kDNkTNnzgx5yJAhIdPldsIJJ4ROa0PQfH3mmWeG7o033gh5mWWWAZB0fbRp0ybkUq6l1qZIc9cABZfT8ssvHzp1DdA1oq6vrJpPeaJuBK1TsdFGGwEouEiB5Py5/3QedD0AhbVqaAn+NNO+uns1Q4fjp9sXSGbq6Fz5vdxnAHDllVfW+PvRRx8dOs00YrZcHq4B3Ufq7tdsJpr+1TWq5wzXTds+qJvn4osvBpB0h5x11lmpY5jbY8Df12sesxP1O9UFrOjv8Lt0/2kGI91cmglZ6aS5WbV20G677RYyMyT1+rnffvuFzGOcd2iD3uvffPPNkM8//3wAybUeOHBgyLyu6jlb6ppAtugYY4wxpmrxg44xxhhjqpaiua5oxtKCX8yuAgqZNkCh4JG6gRRmBajpU905v/zyC4BCdH45UROqms7T3BVfffVVyOedd17ILNd+0EEHhW711VcPOQ+TZFpHZ64TAJx77rkhX3DBBQCSmQ7q5jrnnHMAJE3vm266aciPP/44gEJ2GpB0XZUCHlON9K8tUyOtyCVQyAZU3SabbBIyOw3nsY5qTlbXoJ6XzKCqS8Ex3RcqNwQ1UdMcz2KSQLLgGF2/mh2na6lmcLrh9Fiom4eFyjSTR91c/F91F+WBuuZ1rzFbSjPdtLgis9V4ngLJDDR2jc7KrtJ92xA3gmbSkroUe80qeEk3OAurViq6//T8Y1aVFtHTc5FhAnrNbGg7lFKg1wzNoGahTs3kY5FAoDCXrIKqnJ8LBhpjjDHGzAVFs+jwKU3rzGhTR82TX2yxxQAk3zj5lgEUgu20NgkDmIFCMzp9C8sbNq0DgHvvvRdAso6EthvgsVCLjr4l5/nErm0t9O1am8oxSFzfWPSNcrPNNgOQXDPWbgGALbfcEkBy/coV+Km/UxcrhgaTT5o0CUDyjVotOhqkTrTFQrnq64wcOTJktbgxGLkuqCWFFtVSBF1n7f20Rof6vxoES2pr2qroWufZlFXHttpqq6Xq2cxy++23D52+UbMmi9au0vOPwbzFstxk0dDrmH6eVmCg0PRxwIABDfr+UqD7TK1X6t2gpY2tOoCkxVxr7pC8WzwQPTf0/qz3Olq0WQ8PAMaOHRvy888/DyB5zrFRKVBITCpmWyBbdIwxxhhTtfhBxxhjjDFVS9Hr6NCsDSSD/dQNcsUVV9QciAQm0Q2irg8G8AIFk27ewZ5aol7rBND9oy4KrdPBYDNtR6Al9tO6J5cCNbP++uuvNcbBsvtAoa0DkN5BWl07DCLXjrQamEjXHU2cQD4d6WtDj4/ua667BqtqYCeDSK+//vrQaZ2aUuzbtBL82vFazcA0jdc2Dp2/tgvgvLWFQqmh6VpN2Do+reMxZcoUAMl2ERoMSjO67mOt2cMg0S5duoRO6yiV0t2o3621i/RcHD58OADgxRdfDJ3OlfXJVlpppdTvrRQ3SBa87mkLDL3+0CWirp+82yHUBgN0AaB58+YAkmEC6nqlS7Jr166h08SbPEMb9Dqi+49tOYDCvtXu6y+88ELIvP5o6ArDVYDCfWPHHXcMnV1XxhhjjDEZ+EHHGGOMMVVL0VxXNKd169YtdFqHRbvWssy7RlqvsMIKIbPmzKhRo0K3xhprhEzzWR4mPDXdaSaAms6ZoaTz0wwjukEeeuih0Gn3b5r81PRcSncHUDATa70O7Y6cVjNBI+lZmwQAvv76awBJN4q28OjUqVON78ybtAwdPeY33XRTyHTJqruVmWQAcOKJJwJIdqwv115Nq0cBJMux0wxemzlY1++jjz4Kma0vtM5LuVwHev5pbRh1Y3z//fcAku6AtDpA6rrSFhH8LnYJLwfcf7pmWodK3Rx0aWhoAF3PQMH11rFjx9CldTSvdNjqB0jOb4cddgCQbKGj+4/HMqutQCnPRf0d3XNnnHFGyLyua1ayhjGwXYdeU9S1QzdqHtfPLHe+zpsuKa2Dpy0uTj/9dABJd5a2e2JW2rbbbhu6rJo7c4stOsYYY4ypWopm0eFTlr458MkbSFZG5JuUPqXpkyKf1PnmDyQDH/MMxtK3XNazAIA999wzZL49pwXtAoVjpG/E1157bcisRaQWnVKQFvi4yCKLhE7r/GgdFr6paL0HPqUDhcrBrDcDJJvSpVlM8kD3HGtzAIVmtGqF1LcP7k+1UmkV3zya7qWdfxqgqoHJtb0RpQWDalPXe+65B0DyjbVcb5dp1ZQBYIsttgiZNa10fTVxYOLEiQCStY802JXXnVLXttLv/+uvvwAkm2vefffdIffr1y9kWoz1mOvb81FHHQUA+OCDD0KniQH83UoK4NV7AQPD2RwSSK4vr7v6GU0MoPXn7bffDh2vSUCySj2tm6VYXz2+an2ixbBXr16hSwuW1+vvoEGDQr766qsBFGpLAflYjPX8Uos410UrO2sCEu+PWodM9/Laa69d4zsbii06xhhjjKla/KBjjDHGmKql6HV0soK+0tw46gZKM1NpufdSlJuvDzo/NR2qTNQcp/NjzYHjjjsudNrUk3V0So3OhQ1KGQgHFFpZAMmaN7vuuisAYPnllw+dBtOx6Z427VQ3ZJ41c3QfadsObUA3YcIEAElzs46ZdTzS3FVAvkHWus+ygjV5XmaZhvm/mkygwcxqci43ek3Rpo5aYp/o/HQvM/BRXeuaDMH1K4U7QMekbTXoOmMgP5Acs14T0taNAeJAweV48cUXh07dOAxmzrt2lZ6LaW44dZ326dMnZNbseuedd0Knwdq8hqk7S1ss6F7Oo/UM0Xui1slhmMeRRx4ZultvvTVk7hG9/paLrDXT+TGxRtsGaegJkwX0/qduSNYRUl1Dr6m26BhjjDGmavGDjjHGGGOqlqK7rpTazIJZnV5pmmQX00oiqwtyVqdkMm7cuJBp0tM6BJphwcynPLJ21IWmssKsjcmTJ4dO65jQtZVVh6dS0DVjWwqgkLmkHZ91/ZjVwlLuQOXMT/eMZu1p1txXX30FIFnbSfcss3Uuu+yy0HHOQKE+S95z1utL2rWmtrYHqit1J2+imVZ0kQKFTEtmtAHJtik6vrS5qOt04403BgAMHTo0dLpXKyXbSvccs84AYMSIETX+94477giZ2ZDq2lI3LV2TBxxwQOi0nYZSrmOh66d7IA26tLR2nLoh6fLMw3Wl81B3b1oGq2aAagbgOeecAwB4//33Q6d1dNjupJhrY4uOMcYYY6qWklp0akOfAvWN8+effwYAjB8/PnR77713+QY2B55//vmQ+WYMJCs/smKp1m7Qt2PWN9FqwquuumrIedaXyXpLV+vHH3/8AQAYMGBA6DSwl3oNQMs78JHoW4IGA2qdJ9a00P9dcsklQ15xxRUBVE6AvKJ7R+voaMVrBmtq7SN9O2bNFrXoHX744am/0ViozfpTLnTPaFNDvt2m1auaXSa6DlOnTg2ZQbp6/qn1rhLXT9eEwcJLLLFEDR1QsFRqsoMGYzNxQq9ZeVjsmJQBJKvgn3TSSTXGp5WfGYQ+ePDg0GnNHd4r8tjHen9Yd911Q1brGe9ro0ePDp2OlffHHj16hE7rWNH6WMx9aouOMcYYY6oWP+gYY4wxpmrJ1XWVVZOmQ4cOAAoukkqAZrSbb745dB9++GHIaqZkszZ15/Tv3z9kBu4usMACNb6/ksgKvKZpcvjw4aFTMysD//IOVq2NLNMvg+l0fbWOCVt3VOKa6ZwWWmihkNXMz6Z5WnZe6yB99913AJJNFbWOUqWvaxq6l/N0Oer6aB0gui4YyA8k3TFaU4SoO5KNPAHgvffeA5BsJKnBypWyb3Ucur9uueUWAIV9CCRb03Bf19aiI499qu4obTuigeEMzNU11Qa1DIk4/vjjQ6fBunRJ5jG/rBYs2laDrn1tSqvHpXXr1gCSbU20HVIp5mWLjjHGGGOqFj/oGGOMMaZqydV1lQXNkNpROm9zK82kWtZaO8rqWE844QQAwJZbbhk6dVNVSvfu2lBzo5pWWXPmzDPPDJ0eFx6rSp9flmtu1KhRAIAffvghdCeffHLINDk3JheOdkLmXh05cmTo6C4GgCuvvBIAsNxyy4WuMc2VZGV1sk6X6sqFZh9269YtZHZ31i7P6iZPyyBSnWYNXnXVVQAKLgSgcmrnZKHjoxtDXa9pWVOVuCd1Hm3atAm5S5cuIWuGI9H122OPPQAAq622WuiK2Q6hWOiaaB2jgw8+GEB2pjTnolmvpZ6TLTrGGGOMqVr8oGOMMcaYqqXJnEyaM2fOLKm9U03LaoZlufnrr78+dFpCmsWv6mvuatasWRMAmDZtWp3nV5fsjbyKlLVs2bIJ0PD10zX55JNPQn799dcBADvvvHPoypnVwfWbNWtWg+aX5bo666yzACSzBujOAgpm2lKtadOmTYuyfllkdS2fnVKtI9dv+vTpZbu+aLn5hx9+GEB29/KGrmuLFi2aAMCMGTPm+EVZ17800sZU2/WnVEXymjdvXtL9mTfcnw2dX13WV+Faler+Uaz5ZcF5Z+3PtPkVE85PsUXHGGOMMVVLrhadxEBSmoKxeRlQaCQIFGou1PeJsCEWncZAsSw6itas4NuJBlaW02JVLIuOovuPwapaZ0breJSaUlt08qZcFh1F36gZDKn7t5jWq7m16DRWbNFp3PxT5qfYomOMMcaYqsUPOsYYY4ypWuboujLGGGOMaczMsWBgtfvw/vOf/1Tl/P71r3/9I3yw1T6/csaQaYxSWsHHYsawMIbMMSyNE18/Gzf/lPVT7LoyxhhjTNVSkS0g/oloVkip6wyUGloHGuv4/ynonvvll19CvvfeewEAm2++eei003Y1rmtt9bGyai5V47FojKStX6W3oMki7zo01YgtOsYYY4ypWmzRyRF9Ip86dWrIrLzbvHnzso+pvujbx4wZMwAkG9HVpTJopcN56ZxKVZOlFHCtfv/999AdeuihIbOKcPfu3cs6rnKTtX5s9jlt2rTUv2uzSe6FYr1dZ1XTVbm2isgkK8aq0vfn3KLHQavk//XXXwCAf//736n/W4mkzWXixImhO+aYY0Lec889ASTPWTNnqufuY4wxxhgzG37QMcYYY0zVUjbXVZrptTZ3hpojq8XcChTm/dVXX4VujTXWCPnWW28FAGyzzTahq8T5q7l11qxZIQ8cOBAAcMABB4SuS5cuIVfiXNLIMndPmjQJAPDZZ5+FbsMNNwy5VatWACp3nnSzDBkyJHRjx44NecSIEQCA5ZdfPnSVOpc5kZYyDxTOv19//TV0F154YchjxowBkLz+6P4977zzQl5wwQUBNNx1lRZA+/3334f83Xffhczrhs5Pf79p06YAkgHkbdu2DZn7M6+mw8VC5//kk0+GzOsPmw8DQLNmzUKuxLnqvZBz6du3b+h0L+y9994Aste/lNTFBVhJx9kWHWOMMcZULX7QMcYYY0zVUjTXFU1aaiJm9gIAfPrppyF/8cUXAIDHHnssdIyUBwqm1z59+oRu1VVXDZm/UUmmsbrAYzVhwoTQaR0TZlvlkSmQVs9ndpno+LTT95tvvgkA6Ny5c+jU9N9Y0Kyx1157LeRddtkFQNKcvOWWW4Z8xRVXAEi6DvJ2/eh5yfW5+eabQ3fyySeHvM466wDIf8z1hXPVrKnbbrstZF6LPvroo9D98MMPIe+7774AgG233TZ0rVu3DlmzIYt1DeJ5x4w3ADjooINSx0c3lmaCpX3XEkssEbrVV1895LPOOgtAtju50q+raa7/fv361fh7pWZapdXJeeihh0I+5ZRTACSzItW1ut9++5V6iDXgntAQhdqOb5q7WD9XzuuLLTrGGGOMqVqKZtHh28WHH34YuscffzxkvuUChTct1lsB0q0Hjz76aOjOPffckBmMlWV9qER0rHwjO/vss0O32WabhczA5HLOib/122+/hW6eeeYJWZ/OiT7R69sz56fBuo0JrtXkyZNDN2DAgJBpfVtggQVC9/TTT4fMIEi16OSB7jm1CBx99NEAgPXWWy90aj1tjJWtda5//vknAODEE08M3TPPPBPyBhtsAADo2bNn6LbeeuuQ1RJCsqybxT5G8847b8hqsVGL72KLLQYAWH/99UPXqVOnkLl+7733Xug0WJf1We66667QrbnmmiFX+rpzfjfeeGPo6CUAgMMOOwxA8ppVSXPi+N9+++3QqUWKlg4NRtaaOQysLrVFJM1iv9pqq4Xuxx9/DFmPNesXqZV7xRVXDLl3794AkgHypZ6LLTrGGGOMqVr8oGOMMcaYqqVBris1F3/88ccAki4YDabSOgY0UzHoGEiavmgyowkaAO65556Qt9tuOwDJUuyVZJpMQ4/VI488AiAZDHn//feHzHlpHY9SwzXRoLf9998/ZK2pwnHpMZ9vvvlCXm655QAkA3grNTCQ6Ppw39IEDiTrzHAva2Ce7uU//vijxveXq+ZFWisOADjzzDND/vrrrwEAQ4cODZ26Kcu57xpCVguAwYMHAwAefPDB0I0aNSrklVZaCUByzdNaJOThOtagZ55HAPDOO++EvOuuuwIAzjnnnNCxbQyQfv286qqrQr7ooosAJMMB7rzzzpB1L1RKQLreH+imevjhh0OngdXHHXdcjc/kPQ8dC5NQNNj8559/DvmMM84AkHRnaWJEueai+79FixYAktcMTSbSmlTPPfccgOQ9W69FbBp8ww03hK7UddZs0THGGGNM1eIHHWOMMcZULUXLunriiScAJM32aq5TN9POO+8MAFhhhRVCpxH0zNzSehVazpt/16yRSkTn//nnn4d8ySWXACiYoAFg4403DrlcrgM1/dPMrdkZ++yzT+r/EjUxaqdgurleffXV0FWiO0TnpDL3srqr1Ix77LHHAkhmHWidFrZQYJdhIOlaKCXqjtHaVVqng605tM5RbebiPMrN14bO9a233gr5pptuAgBcc801odM6XHQ5ZtWhKRdp5xTbvwBJd9viiy8eMuv86J5SNypRF9Thhx8e8vPPPw+g4GKY/Xc1w6cSM/B4jXr33XdDp92927dvDyD/Mev+/Pbbb0PmdUEzIa+++uqQd9xxRwDJOnS1tUsqNfz9TTfdNHTa9kavH8zA1Wv+K6+8EvKgQYMAJO9/Tz31VMhcv2K6sGzRMcYYY0zV4gcdY4wxxlQtRXNdLbroogCApZZaKnRqmjrwwANrfIYuAiDdjKymR4061wyuSkYjzTXDgRHqmtWjWTt5uK6YAabHXN1RtZmB1bRK19V9990XOu2+zIJseWdC6Py1UCJdH1oEcffddw+ZWR36+SlTpoTMQpkjR44M3R577BFyKV0munc0K0XHyrlkZXJwLfW71DXCz+VRsFPnocdRXYcLL7wwgGQmoGYg0aWj88vDzaFz+emnnwAAt99+e+jUddGuXbuQeX7Vto/073os2NpCXVdacE/3Qlqh0HKhv63tHoYNGwagcM8Bkm52XkvzcJfrmmrWMdtuAIVr4fzzzx86nSvvC1qwVLPm6IbNY8/qMdXrh5LWwkiLc44fPx5AIfsPSK4vC63adWWMMcYYMxc0yKKjT1x8S2RDwNn/fscdd4TMYDgNUNLAYz7d6huNNjLTctKVCMevTTtZOwAotLDQAMk8rBtpFg19Yk9rxJaF/i+DXPWN5tlnnw05j6Z0ROeh+4tviUCh/k+PHj1Cx9oWQKH1g1o5Vl555ZAfeOABAMn179WrV0OHPkd4/LUeh7Y90PpWTALQN0I9Fgz2VyuJBtMvs8wyAJKNQMtV0yor2Fqtw7SY6lu+tjhgMK+2fdDWC+Wqo6N7kW+52jZF35h1LtTXZXz6v9yrahnPO9hV4VjUIq5v/+PGjQNQSGoBCrWRgHwTH9Qyo7WPRo8eHTLXT+d3/PHHh9ymTRsAhVZAs/+d56Va+fKYc9b+q21fct+Xc89Vzu42xhhjjCkyftAxxhhjTNVStGBkBoBpgBjrdQDJwEyWk9Y6DwpNX2q6VTM8P9e/f//QaSfpPIK00loIaNl9Ne2zK60GIOfhutLfpJmUXWqBggsGAA455JCQ09ZNgz1psv3rr79Cp8HIebaD0N9WN891110XMl1S6m6iuwYoBHnqPtP6UdzfW221VejKtb56zL/55puQGYAKFNzE7GINJDt9v/HGGwCSa6owSF0DrBkADJTm/Eszd3OcQNJlwMBHHYea9pkYcN5554VOgz3XXnvtxG/O/l3FQr+fnazVHdqqVauQNbEjLVi8NtKOm7pOFllkkZDzCEBO65StLS5uvvnmkHn90fuL3ivydF3pPtH1Szv/s1zHrKl29tlnh+7oo48OmffCPEMA5gbdc3pfmTRpUo3/1WD5UpxrtugYY4wxpmrxg44xxhhjqpaiua7SOpKzXgqQNC3SpJVV4jotGlvLZQ8ZMqTG37VOTd41PS6//HIAwMsvvxw6LUfPrLG82yLocaJrba+99god68UASTfBWmutBSBp4lbXIjOstER9t27dijXsBqFjfu+990JWNxbL7W+yySah07Xm/tSsH+0+36dPHwAFFwhQPteVzi+r3tSYMWMAJGtbaQYguwrr+cnPAIX6SKWekx5z/pa65rbYYouQ9VizDkeWi5Trpq5JdcOx6/K6666b+vlSXF/SvjNr/HP7+/r56dOnh0zXgX6PusxUX0o3c9Z3002l13nNiuN1SbMi867JRfTYaVax1univNV1qK7lgQMHAkjeSzUbjy068gwBmBt0fB988EHIrN+00047ha4u7Wjqgy06xhhjjKlaimbR4ZOsvkVqNUitMssqnOuvv37o2MgLKNTiueyyy0KnVXb5pPj1118XY+j1Ri1Pd955Z8hXXHEFgOQboz695m3JSYPH9JRTTgmdBh2zWjBQqDOjaODdDjvsAKBQIRjIf62IvnFpNU61FDCIWAPI9e2EdVo0QLB169Yh8xhqbahSrznnpUH5Wm9K148NdNXKdv3114fMeatFR+fPpoq65sWycmQ1WqWVSQPkdcysFgwUxp01Jr49sp4OkKxTwiDtLItOudDx1+f46vVJK3fz7Vq/MysYuZSWEv2dF198MeTzzz8fQPKcOe2000Jmg9JKajTLuaiVd+jQoan/y2QFreM0ePDgkGld0zpVGnhNi3qlWLGyUC+HnqtTp04FkPQY8JgApblW2qJjjDHGmKrFDzrGGGOMqVqK5roiakLUpmUM0AUKpiltGqnByh9//HGN70oLvMqjbLmOQ8viq8uHgZEMKgOyGyhWCjzW6m7R2irHHHNMjf9VdH4MvHv66adDp4G/edY50qBMDbBWc2mXLl0AJAMg9XM0revntcUHa0mVc515TNXdqI1I2XYFKNS00GBODTamG0/dCbqWrO/RqVOnGr9fTHRNeKx1H2ltDj3WHIueq+omoRuHLjgA6NixY8hbbrlljc+Xei3TGnRqHaMXXnghZHWDp5FWZ+fWW28Nma4rbcC8wQYbhFxKl5Bes7W2lroxfvzxRwDJOlRaM4bfUYnX0S+//DJkNkoGkvNecsklARSCigHg1VdfDZnXXV0TTRLhtbYS5w8U9o+Gq+i1kjW3ytU2BrBFxxhjjDFVjB90jDHGGFO1FN11pahpTU3qNG2pO+CWW24JmeX41UydVhNEsz5KDces9SY0K0zL7TPrSDMZ8s4KmFuyxplVk4WouZsmc13/vGs+8Pe1noWWIlfXW4cOHQAkTc9ajn3s2LEAkmXpu3fvHjKPYR5rrsd5u+22C1mzAl966SUAwOuvvx46rQO02GKLAUhmhWi2k7qsio0eM60jsuOOOwIAHnvssdBNnjw55FVWWSXkNNO+Zv3169cPQPL6cvvtt4fMOjyldg3oXNdbbz0ASdex1llhphxQWBetU6Xw/NOsx2uvvTZk7pEBAwaEbtlll63x+WKS5s44/fTTQ9ZzkfXXBg0aFDptEVCJWatcS81k3GijjUIeNWrVTMyHAAAgAElEQVRUyNx32lZHr0vM4NTzV6+/leqyIhzf3XffHTqGowCFbLK2bduGzq4rY4wxxph6UlKLjr5dMsAMAD788EMAwKOPPho6rfPBpzvNrdf6IMceeywAYM8996zxmVLBubz//vuhe/DBB0PWNw7W9GhoHYxKoi7jZxA634wrgTSLjr5l6Nvxww8/DCBZbfvbb78NmYGdWs20UtY6yyKiDWY5fg0w1zpARK1cabVVSj1PfXPfZpttAABHHXVU6LSprzYo5Zu0BvBqMC4b2Kp1iAHoQPnmp2/mtOhoALlamdT6RkuMBrNqHSjWHNNrqv59//33BwDss88+qeMq5byHDRsWss6P1ciBgvVw9dVXD10lWnGUtGQAtYhq5XWeS2rF0qbA3Nfc840BvdfTEjlixIjQ6fV1l112AZC0UqUF4xcTW3SMMcYYU7X4QccYY4wxVUvZXFcamHTGGWcASK99ARRMe1oCWwOz6BLJozaNBkCqO07bXTDITMfX2F1XtaHzo5myd+/eodN2C3nA/cFAWwDo2bNnyGzkCBRcktro8q677gp5s802A1A57qos9JzQdhCXXHIJgNrdATqnPFwHadcEBhIDwAorrBCyBnuyXYQ2/aQOALp27QogWcdL51eutUyb3wknnBA6bdvwxBNPhMwgeA0w1yQJ1hfSOizqEmNrmnK1elA0ASWtNhUArLbaagAq312lcC21Xg5dhEDBXQMU5qXJDt9//33I3NeVeE3JQufNFg+aAMAEHQBo164dgPKury06xhhjjKla/KBj/h97Zxl2Rdm1/9PjfannsbFbVCwUFRWxuzsRu1sxH9HH7lYwwUSxFRVbVMTAFrtRUVRsMKj3w//L/1z7HJjhrr33zD2cvy+sY3Hvveea65prZlYaY4wxpWWm6ZnHJk+e3CLbmZqztJPrEUccAaBSLwFIljVnp1bNhFA3EE1ezTXttW3bdiYA+L//+78mf4GaiDVSXrPCeKx5mR7/93//dyag5fPXUvRcqayZCc2B89ec8am5fuzYsSFrTQse30UXXRQ6LZdf66wcjm/ChAmtx3bdBDp06DATAEyaNKnJ49M9RV3j2mmdGUa6zjTrI63OUzVp165ds9enjm/8+PEha7sZZrNo7S51TTKDSVsosOy+0tzxt2T/1OwazYDUmmi8Rlv7/qnrM62OmO5Fae1GapWJ1JL5y0Lnim7ICy+8MHTvv/9+yLWuU8X5U2zRMcYYY0xp8YOOMcYYY0pLTV1XWQUDmTWgWRErrLBCyCx0lpWV1VKqZbpTM3ORynIXxXWVZbpt6blqietKyXKDkKysqlqb1O26ahxZbUWozysrriWuK6Wh9dkQtVq/Ldk/s/aEImUwFmX/rBXVuv9luY7ZiV5dk9rChG1OajXPdl0ZY4wxZoaiphYdRd9OGKyrefT1zKmvRTBWkZhR3kjKPj5bdFon1bLoFBXvn62bWsyfWnf++usvAMl7OtsC1QNbdIwxxhgzQ+EHHWOMMcaUlum6rowxxhhjWjO26BhjjDGmtEy3qWfZgwXLPr6yBwt6/lonnL8pU6aUcnxt2rSZCQAmTpxYyvG1b99+JgAYP358Kcc366yzzhDrs+z7p2KLjjHGGGNKy3QtOnmRVhyrmgXnjDEmC+413nOMaTpp108a9SwSaYuOMcYYY0pLrhYdfeLTgoKTJk1K/Ds1//73v2t7YGaGozntPFj4EkgWx+Lnm1O23+TPlClTACT3n5lnnjnk1pKpqutP1yq7Yhd9HFn3B8pZFgFef0UfX2tH50RhwcAJEyZM9/N6TXXo0CHkWlhPbdExxhhjTGnxg44xxhhjSkvdXFdqhvyf//kfAEnT8FtvvRXyHXfcAQAYOXJk6NQM9sADDwAAFltssdAVPVhQzXwcv5pW1fVRFpNrluumKOPTc/7DDz+EPN988wFouDv2q6++GrrFF1885LnmmgtAccY5o5LVHTuNNm3ahNy/f38AwCuvvBK6QYMGhUzXVjWPj3I19jF+18SJE0P30UcfhbzCCisASI65SGuVx68dsb/88suQR48eDaBynQHAvPPOG/I888wDoNIlGyjW/SFt/087vqzu9fXsCzk1ehzjx48P+fnnnw/5sssuAwC8/fbboaO7FKjMy5Zbbhm6Cy64IGTe16s5Z7boGGOMMaa0+EHHGGOMMaWlpq4rNXN99913Id91110Akqat4cOHh/z3338DSJru1HV15513AgD69OlT5SOuDmnR6Oqau+GGGwAkXW/7779/yDTD5mFObkymUFqdhDQzbJaJn6Zb/qufqTV6zL///nvIp556asgDBgwAkDw+lV977TUAwLnnnhu6G2+8sfoH2wz0OGud9aXm6CLBtaTH17Zt22n+Tq/TH3/8MWTO/+qrr171Y9M5+fPPP0NmpgrdplP/bVP2Ao7rvffeC91///vfkB988EEAyXNSJNcVj//ll18O3dFHHx3ymDFjAACzzjpr6OiuAoAuXboAAE455ZTQde7cOeQ8XD+6v910000AKi5EAOjevfs0n9H5454DAPvuuy+AZCZdzevQ/P85+eqrr0J3xBFHhKxztdxyywFInn+91w0bNgwAcPfdd4dO1+J1110HIJmJ1dLx2aJjjDHGmNJSN4uOBhbfdtttAIAlllgidIcddljIa621FoDkG/fDDz8c8tNPPw0AOOigg0KngWnVerpN+56st2R9O6RFauDAgaG7//77Q15++eUBJIPljj322JBPO+00AJUnY6B6Fg89Tg1QvOWWWwAA22yzTej0LUPHTUsNxwkkrXN8uh88eHDo9C1qk002AQAceeSRoVtqqaVCruXbiY5frYh6fGkWOdXx7UrPlb5R5hH4SOvF559/Hrp//vkn5LSaJFmVf3n+s6xDlHXO9O0yD/T3X3rpJQDAxRdfHLp77703ZAbh6px+8803IdNisNlmm4WupXPKc6a/qcfEAM533313muNsLv/6179C/uyzz0L++eefAQCzzTZb6IoUrMtz9eSTT4ZO52fbbbcFAPTo0WOazwAVj8Gtt94aOrW+NtdS1lT0d3SvvP766wEk15daD3ndXX755aF77rnnQt5iiy0AAAsvvHDo6mWl+uOPP0LW/UG9K4cffjgAYJZZZgmdnotdd90VQMWKCQBPPPFEyAw2X3bZZUPX0vHZomOMMcaY0uIHHWOMMcaUlpram9Ucuummm4a80korAQDmnnvu0KUFC2qdnTfeeCNkBjONHTs2dPpdLTFHqmmZJmygEhg155xzpv6OmuEYOKfH169fv5BpktMxP/TQQyHT5Kq1BaoVWKrfo2ZyHt+jjz4aOroQgaSbjfUT1Bz+6aefhsz55b9AMpicwZA6pxqYtuiii4ZcLZN6Wm0RdYdut912IdMkm/XbdJOoOzZv6LpSc7euX13XnDcNhlU3FK+lF154IXS//vpryLvtthuAirszL9R0/ttvv4V89tlnA0jOX1a5eqI1QeaYYw4AwJprrhm6Wrh29Pq65pprACTrOel10JQ9jX+7yCKLhE6ve7o3NXSgXu6cpqDHoXslwxw23njj0P30008hs6YLk1aAZLsBDRPgvlaLMes5HTVqVMjcd/fcc8/Uv00jj8QNhb/ZtWvX0HEfB9ITN7LOKV2yGmys98+vv/4aALDMMsu08Kgr2KJjjDHGmNLiBx1jjDHGlJa6ZV1pBkCnTp0AJCOpteYKs4E0a0Ij8FnzRE1b1XZxAMlI+TTTour69u0bMmsGqelUS5Sn1ZlZcMEFQ6bLIasOSEvMrHqemL0AVDIBvv3229Cpa0ndIKuuuiqAZCaD1oRgd3k9ZnVdMSr/scceC51mbWnNhWpB14WauJmdAwAHH3xwk7+zSJkqNMFfeeWVDf4t150ev7omx40bByCZFbLffvuFzKzAPOqwqAtK3ZC9e/cOecSIEQCARx55JHRpNUc0K03dcCuvvDIAoGPHjtN8prnw8/o9CyywQMjcHzU788QTTwy5OWstq8XMfffdB6CS/QjUvuZSc8hqe8D51TCC4447LmS2ZtHxq5tW3Sy1ROfs9ttvD5nXjbqL01yHen9I696eN1mZltxjv/jii9BpHSC6Z4cOHRo6ddMytKOaa7IYZ8wYY4wxpgb4QccYY4wxpaWmristLqTdZ2maUzeJdoJmBo66jtQ1tPvuuwOoTaaAmhu1CFxaVphmomjBI2YwaTl3/V5mvahri2XBAWCvvfYCUJvuwvo96k5kWfEsNCtFS69PDzWx6vy///77ACqFE4Fk8a9aomtK50Rdh60RXgvt27dv8G8122HqzwOVrD81J5955pnTfL6erjuuJXVBMLsKSGbQsR3H+uuvH7q0gpC6FlhkDgAuueSSxN8BtXHN6XfSTf3BBx9MowOattc1ZPKnG1m/p0iuq7Tiiipzfs4///zUzzMrcKONNgqddsrWfbXaazgra1eLQ7JgnmYiqzvtk08+AZDMetSCgrPPPjuA/LPjdM0MGjQoZGZ+aqZZ2lrXa3LppZcO+eOPPwaQ3H9aii06xhhjjCktVbfopL0ZAsmaMLQk6BuV1szhE9/iiy8eurSn31qUvdYnz7QS1jo+llKf+nN84tYGaBpkSOuNluvXz7O+hj7l13qsDb0d6Llo6C2IQWp6frSp4Pfffw8gGcys1rNaljPXtiK65jSwsTXTmLc8/o2+fbJ2BVAJTNdkALX+5dkU8ayzzgrdtddeGzIb5QLALrvsAiB5LnSsvIb1+tTrmmux1rVl9DriOdXmonqe09pBpLXlACpjVcudWk/ztgSkoXvd5MmTASSv1bTAarXCqsWRrVl0zap1rF6WSLVoaJ0YzqWuP7X+0/rDpAAAOOCAA0JmTaAiJUPo9UeLPZNSgGS7EVrXNBj+iiuuCJkNrrWm2yqrrBJyc8Zti44xxhhjSosfdIwxxhhTWmoajKzdnbXcOOukaG0VdWOxtLTWWVHXD2ue1DqALu371WymJda1Dgm7jmsAs34Xx611CLRTebdu3Vpy2DWhKeZumt8ZFAhUzJlAxQyr3efrVYJea6d07tw5ZA3ibezvZ7kOWgs6zjvuuCNkrkUN5s3DTK7XB+toqYl8jTXWCHmhhRYK+ZdffgGQdF3o/HCu1Vyurh22Fqj1nKpriQH+Wjsly/VGN4zumbqu+bfqmlU3Qp6uq6xrRl127OSudZAYDgAAvXr1AlBpBQEkO3mn1SmrNWktZu65556Q1XU2ePBgAMCQIUNCx9pyQKVTu65/rUnDcaXVhqon+pvaeoaByfPPP3/omECkel3/uj532GEHAJXzBCRdV83BFh1jjDHGlBY/6BhjjDGmtLTIdaXmVJoL1Zyl7iotd51mZlMzJqOxtUXB1VdfHTI7TWvUfb0yQfTY1fSmJvWXX34ZQNIcp1lLAwYMAAA8++yzoTvyyCNDZruIPLJbmoLOv5rRWbr+vffeC522EGCGhJpe6+UaUXMqTcRAsqYTazpkdb9mhoi6XrXOEF2WaZkyRYLZLQDw5ptvhkzXgLrz6rUWdR9Qc3///v0BJM/5O++8E7K6yTl/WSX26bLUFiBaR4rd22uxJvU7taM2221cddVVoTvvvPNCVtcNs3my1i/Xp7quWLsLAHr27Amgvu7WtN/S/U9r4rD1jGZisS0HAPznP/8BkDwnulbygPcFdZfyOgIqteGAiptR9xR17XNf1LXOtkJTfy5P9F7ItkBAsuYP0f2Dn9NrIS10QDNB9fPNWbfFOGPGGGOMMTWg0RYdPkXpW4IGW7HOjb4xp9WJSPtOIL2mBytEAsCSSy4ZMi0ledeD0N/Xt0e+Mer4NDCOQXZdunQJHatlTv29rQVtWscgO7XI5V1Zl7+lQX/rrrtuyKy9AlQCc7W2jlpnaKl6/PHHQ6d1LtZcc81qHXZN4LWmVXg1GFSDfOtNVrVeVgvXCtoagKyNYGld1PHp9cfqyrqXsZEgUJnrWgez6p7I9cOGmwBw2223hazBtgwW19piDVU51pokzz//PIDk+BtTUbslcM3pcRx44IEh67HQusXmzkB6EHOR6sgQtSyplWPYsGEh81rT2jmabHPRRRcBSDaC3nnnnUOmxSdvi39abSogvYFtWmC96tT6z+9Sy5Ba/5szblt0jDHGGFNa/KBjjDHGmNLSZNeVBkhpifgVV1wRALDAAgs0+rvUBPXcc8+FfMYZZwBIBvuec845IbPmRJFMl2nBUmrOYzAlUKkTpI0I8y6x3xQYJDhy5MjQsfYFUDH965xpMHYe46OZVNeUNoXs3bt3yKzZoH+b1mBOgyG1KR3nvaguSB6fBvNqC46OHTsCyP/60vPHBISm1FvSpo76t9dccw2AZHPatddeO+R6rU89vzznTz/9dOjUnaNuDA3SnR7qbuX+DADbb789gEq9ISDpGqvWulXXBtvB9OnTJ3Tqett7771Dpktnjz32CJ26hIp6XU2Nuls0TIGyrmUmsACVsbIVEAB079495Dz2z7R79m+//RaytrggWkeOAf5AxXWsweh6r2Drp5122il0Ld2LbNExxhhjTGnxg44xxhhjSkuT6+iouU1rp7zwwgsAgK5du6b+rcL6HVobR90Iq622GoBk1oHW5MnbpN4QNNkOHz48dKeeemrIJ5xwAoBkVH7Rx6RzSZO6zp+uhX79+gFIujGLMj41gdNECiRbILAmSVbNJGaoaCbETz/9FDLNtEU1sXMu3n333dBpOxO6UYsyZ0AzOxaL60RbJHCvUnequm7ydK1qpl/a/wONPz69ZvVapMuO9byAZM2elsy7/qYeJzvNq7vjuOOOC1ndNA888AAA4Msvvwyd3lfokivq9ZVG2jlV3WuvvRYyXZNbb7117Q9sKnT+tJ3FTTfdBCBZe4ruSCDZaZ7foW2RdK8dO3YsAOCtt94Knbq52E6imvcPW3SMMcYYU1oabdHhE5VW82QFXAC48sorAQBfffVV6LRRntbpYIO+MWPGhO7YY48NmRaPvANYm4IGCI4ePRpAsk6E1snhWNMqSxcVPVY2bWNQNZCs7MxgRz0nRRmfvrFkVanVwEGib4/8nFpBxo0bFzLH3dJqntVE5++PP/4AkKyWq4GfeR9rLdAq0LQqaIBuUajmdaLfpW/HtF7xzRqoXrVdXTtaLfyVV14BkKwWr9fM0KFDQ2awuAb7a2X1POpw1ZoffvghZJ5Dvf/VC50/7hMAcMsttwBIVnDWBJq09aP7n1ZeZ+C9Wqy0MjmbCVfTYmeLjjHGGGNKix90jDHGGFNamhyMrO6IQw89NGSWYz/55JNDd91114WswWbrrbceAGCHHXYIHWuX6G8U3V2lZj6tL8RzoLUDLrzwwpAZzFp006uaI7Wc/vXXXw8g2ZbjiCOOCJmBZUUfn6Jm0oYaBPK86Jrt27dvyHTZap2WIgVOMshQ3TnqhivSsbYEXb8aBPvxxx8DAI4//vjQlWXMio5J3QwrrbQSAOD1118PnbpOdN+q1h7MwFT9nf333z9kDRZnnRUNVtZ2La19rnjf4DiBZANLom1N6oXu2XPNNVfIbOHDQGEgGaai88dkFW3RwhYuALDxxhsDqKxDIFnziWvOritjjDHGmEbgBx1jjDHGlJYmu67SupAClQjqDTbYIHSah6/R9nTdaIlsNZm1RtMk3TlAJauM9ToAYI455gi5IddInqg7Tk2rauZnV2HttKwl6ovucmwpXJ977rln6NQN8P333wNIZo0U/Zy0xmuuIfSca50cZo5oVktrcrM2Fp1TDTnYcsstAQCjRo0KnV7r2g6kqeh51PV/xRVXAEjuI5r1pTVXWPpf20I0pfVHa0Frx6ibim6glsxDNdA1s9VWWwFI3t/V9Z22v6k7Sq813vd1rdR6f7RFxxhjjDGlxQ86xhhjjCktTXZdKWpCpGlRM02ysk4oF92c3xB6/Fpwju0Ell122dS/bS2oa3H++ecPmaZVdUfOSHD96jm5//77Q6ZJtjXOeZnQPUfXMtsspO1JZUXd5VtssQWAigsLSLoRquVa13PaqVMnAMDgwYOb9fkyzQ/Hoplw5557bsjsKs/CekBxXKu65+s9ryHq6aZKwxYdY4wxxpSWFll00ijKk2c90GDsQw45JGQ+seubUWt5I9HjZNA4UAkmBCotLLRR24w076Ssb5xlxHNVIY9rdUbcHxpC16E2eKZ3pIhJK631OrJFxxhjjDGlxQ86xhhjjCktM7Um85MxxhhjTFOYbozOpEmTSvkU1K5du5mA8o9vypQppRxfmzZtZojxeX22TvJen4zx0EyzasYLzijrc/LkyaUcX9u2bWeI+VPsujLGGGNMaal61pWpvFFp2XIl7Y3KLkRj6kND12drzBDSDNCff/4ZADBkyJDQ7bzzziGzHL/3HFNL9PpSOY/ryxYdY4wxxpSWVmnR4dOhNh1T6lV/QJ9S9Y2Kv//XX3+FThuc8XPayK6MTetMvuia0muliPU56smUKVMAAOPGjQud1oxqjRW/da7ZbPfoo48One4/++67L4DKeSgCuj6z9nXCyrquPF5MeC+kZRFI1pm75JJLAABLLrlk6Grd1NsWHWOMMcaUFj/oGGOMMaa0tBrXVVow0zvvvJP6t127dg2ZZrRqmcP0ONT0q8fCIMCHHnoodJ07dw6ZZvJNNtkkdHvssUfINDO3JheWXW/FQOdhwoQJIX/88cch8/pQd2vZUXfIe++9BwBYf/31Q7fwwguHfO+99wJINuUtYoCyjunXX38N+cEHHwSQbBqp7rg8r88sd/8333wT8osvvgig0jx46r9db731AFQahbY20oLgdS75/1ntFrgWi7rP8vjffPPN0On9j3JWyYO089PSsc44O50xxhhjZjj8oGOMMcaY0lI311VzzFH6GY2wv/baawEAgwcPDt3ZZ58dci1N8mpivOOOO0Lu06dPyDQTzzvvvKF7/fXXQ/7jjz8AAE899VTo9Jj32msvAPm7g7LqIBA156sbj3+rY0ozWRe1Ey7H0rZt29DpWNKOVdcnz0vW+GtBmrn76quvDvm6664L+dVXXwWQXJ9FdM1UEz0vQ4cOBQBMmjQpdJ988knIF198MQDgxhtvDF2t568p8FgmT54cunPPPTfkl156CUDSHb7FFluEnEe2Eo/5l19+Cd3tt98eMvd0oOK612P+6quvQua8PPHEE6GbbbbZQi7SXkL0vsFs3N9//z10zJQDKq7VBRZYIHQLLrhgyFtuuSUAYNFFFw1dkcbMfe+1114L3XLLLRcyj1X3R5W5F+k9Rc9fc8Zqi44xxhhjSkvdLDppb7zai0Xh078+0anF5PrrrwcAXHnllaHTwMJa1IdIe4vimzGQfOJmMONCCy0Uug8//DDkJ598EgBw2WWXhe7RRx8NeccddwRQqWAK1O+JXedEgwF13CNHjgSQfKPSKqysjzDPPPOETt8+dtttt2l0DdXOqDUDBgwI+b///S8A4MILLwydBj5OnDgRQPKYdfyffvopgOT8rrDCCiHXwnrCN6KPPvoodGrFWXnllUPm229TLKppfZOK9BaZhr4lqkX10ksvBZCcP50TznWRrDhpvPvuuyGrdZn7zqGHHho6DUyul0VHzx+t2GuvvXbodP2ceuqpIffs2RNA8ph1L9puu+0AAN9++23oZp999tTvzRMd/xtvvBHyWWedBQAYPnx46HR/pSVZax/pnN16660Akve/NdZYo0pH3Tx0rNwfPvjgg9DpvYLrVuvoqMw9TO8PO+ywQ8jNWb+26BhjjDGmtPhBxxhjjDGlpaauKzUdf/755yEziPiYY44JXYcOHab5HF1UAHDzzTeHfMQRRwAA9txzz9DVuqw9zaFaKl5dUxpYPPPMMwNImsZXXHHFkLt16wYAePrpp0PH2hEAMHbsWADJALtaB4vS9KgmVg0QHDNmTMgjRowAkKzToq6NL7/8cprvV3Py/fffDyA5p6usskrItTSt65y88MILIZ933nkh0/V5/PHHh07HmoauC67fUaNGhU7nvxZwfeg61DlT1xqvtazzzLWg7oI777wz5F69egFI1mYpYjCzrjlNXPjtt98AJPccuiMBYK211gKQXCt5txvQvfTPP/8EkAxA1sDqE088EQCw6qqrhi7v4+f+vMEGG4TuqKOOCllduzxWdYfotcR7ibqrioLO0+jRo0NmOAJQGZe26FhqqaVCXmaZZQAkkyHoDgeAU045BUDFxQ4k92p+HqhcA7V25+m4GXDOoGoA2HDDDUOmG/OWW24Jne41q622GoCku8p1dIwxxhhjMvCDjjHGGGNKS9VdV1m1V7RmAqPGtc7DYostFjKzei644ILQ7bPPPiGzZo5G5aeVyJ5aXw30+zSrSKFpee655079HE2zGmmvrh+6QepZop/tKjRTY/z48SHPP//8IdO0uNJKK4VO54Lz/uOPP4bunnvuCZntCO6+++7Q6XfVAh6TZsppR12taTHHHHMASLpGmemh36XuAHWzMoOuS5cuoatXJogek2btqZu1oWPhunv22WdDd9VVV4XMc0EXbdHg8WtbAV1/7dq1A5DcJ+aaa66Q55xzzlofYqPIqiPWt29fAEnX60477RTyfvvtV4ejaxhdZx07dgQA9OvXL3RpmXxAZdx///136FjbCKi0gJhvvvlSfytPdM50T9G1duaZZwJI7j/jxo0LmS4v1nsCKpm8QMXNyvskAGy88cYh63m55pprANQ+K0vHzX1fM6nS5l3drTp/dNnp+mip69UWHWOMMcaUFj/oGGOMMaa0VM11RXOxZqdoEauBAweGTDeUlrjW7ruHH344gKRrR10qNMn//PPPodOobS1tX+2iX2qCpAkVAM4555yQWXr/oosuCp0eK83omhWj30uXDrNbgKTrqCVmWnWHMfsEAC655BIASReUuis22mijkOnaySr4yGwVLduuWTtEx69FHrVQVjgVSgcAACAASURBVEvGqnPPse67776h++6770KmOw6ojFszQRSeQ2YPAMAXX3wRMrteV2vOmoJeM1okUDM4Ggu7YAPJdVFE0rq2/+c//wmdrjW6hjXTarPNNguZJve83SE6Ju0ETTepdlzXFjjMJsty4eeZIad7hh6T7kt0U5x++umh0xYJzAzVNalukDzRc6uZVN27dw+ZLmEd86BBg0Kme72hMAl1t7ItBJB0Uy+yyCLTfFe10PWp42ZWrbruvv7662mOLytrk8dazUxBW3SMMcYYU1paZNHRJzq+kbO8NZAMQNpqq61CPvnkkwEkn2i1DsA777wDIFlnZYkllgj5rbfeApB8i9GmfLvvvnvILC1erbcY/R61SOkb4W233QYg+caldXIeeeQRAMk3krRmoWyVANSmwad+D+dPg9b0PKa9ETZ0Th9++OGQNdhw2223BQAcdthhoWOAKFC9J3k9Z1wf33//fejU8qfWKwZGN9RKRGvWaAM7tn7QOi21tg5wrJ07dw6dtrBQS4++SU39eaBy/tVKuvzyy4fM+k5Fqp2j1w/rOL388suhS7MSqk73JwZZ5zE+nQcmNQDJFgl8Uz722GNDp4HnbDGTFWzNIHm18tXLeqW/kzVWWsL1mtR7Beuo6f50wAEHhMy1kHcjZLUyaeA+LfYPPPBA6LQOF5s66/6v1iF+rmvXrqFbZ511Qta9lHtYLdvOAEnrNu/bP/zwQ+jWXXfdkFdffXUAyaa5arGqybFW/RuNMcYYYwqCH3SMMcYYU1pa5LpS0xXNxFpPRAPP1DRM856WyFbXFT+nplWaK4FKTQF1PWgdCa1/Uss6Omou3nnnnUNmMJa2FVDT5K677gqgEigGAIsvvnjIrFNSa3Oezg/NvY2pQcT5y6qZxK61l19+eejUHUU335prrpn6/9VCj58tGFjDAkjWntDA4zSXla51Bpbr/GrgMb83LcCy1qy//voha9uJYcOGhcxy9Oru0fljYOQzzzwTujPOOCNkruW82wqkuc6BShC1JjjoWuffatCmBnPmiY5Jg8HVNcokCLaqAICDDz445Oeeew5Acn/SmlisT3bggQdW67Abja457b6u5f65vu66667Q6f742GOPAUi6ZjVYm/uOuovq5TrW2mEnnHBCyNrup0ePHgCSwbrq5uH+qPuTHj9bZ2Ql2qibutrob+rv6P7A9cl6T0ByrHTNaYsdDVOphevRFh1jjDHGlBY/6BhjjDGmtLTIdaWmJWZ7aHT4Rx99FHLv3r1DpklSzZhp3aE16lx/iyZXrZOx6KKLph5jtV0+WXVoBgwYMM1vqmlfM11YR2iLLbZIPU5+rhbuKv1ONW2vvfbaAJL1Kpg9BiRbIDBbTOuQaKdampTVXK6d2NO6E9fCtKzfyfOv5lI9Fw2da/0umqG1TpCaqdk9uJ6uHR6/ti/o379/yJqhwutOXVt0dwDA+eefDyDpDtIMQ14DRXJdaZ0qrtusFip0aWidFrYoAPLNttJMHc3UVDc+u5Nr7abHH388ZLqBtAT/pZdeGjJdRjrOatcby0Kvo1lnnTVknQu6brLqULHm1d577x26XXbZJWReixoOUevx8VpQ17hmVfXs2TNkZmVqppm2O+K5UNeq1vkieWeVqWtYs555L9S9SK+vm266CUDSNaftWrhuXUfHGGOMMaYRtMiik1ZTRmuLsB4OkHxifemll6b5vAabMRhNa9NosOCqq64KIFmHRp/+alkFUq0U5557bsgabLb//vsDSL6F6ds1m5p269YtdPrEWy/0jZcBbtr8UAPA9e2IzUr1KVyD6Xj+2VAOAF5//fWQaSnI441E6/k0RFoAMlCZS7VSaTA6A1/zsHjoOdXaN/p2edpppwFI1mbRBop9+vQBAIwYMSJ0apHMu2Iw0f1Dx8eK15oAoJaSTTfdFEDSCqDBynladLSCrAaQa9NjBrNqU0edn1GjRgFIWtR1r+S+qus7j6azOiatk8a/ybp+OD9qxec5AYCxY8dW5VgbQs8frWS6DrVpNe8JQGVdarKJ1oRj4g4r7ANJ63oecKy6TrRptQbOs35Y1l5Li/cmm2wSuiFDhoR83HHHAaiuxd8WHWOMMcaUFj/oGGOMMaa0VK2pJ82J6oLRstQaLPf2228DADbffPPQabAcm0aq6VnNhPytprghWgp/nw3lgGSLi+233z5kmiH1+LWcPmsGqDtEm5bSTaelvGthWlYTPc3At99+e+i0NgfbbmShc3n00UcDADbccMPQabCy1lQoMnrO1XT8yiuvAEiam5dbbrmQ8w7SJXp9aJLADTfcAKBS7whImqEZDKhz1pymoLVAXUzqGqc7GKi0dshqN8AWH1kNJvOA16ImA2hNFm1XwTpdGgyqNWM4r/oZbc2jIQH1JqsRZFPchZw3dd1p02C22NH5rcU1qfck3t/UHaeBydoOiMei16feK5gkoveMPEhr0aEuUD2/aa7frABwXmtah0dd57XAFh1jjDHGlBY/6BhjjDGmtFTNdZVmptIMHprLgUpNAXapBZLddWn6UnNyUdwBWW0POnXqFDKzcfT4TznllJBZ/2L48OGh+/jjj0NeZZVVpvnOWrvpeKzaZXbo0KEhazbITz/9BCDZwkK7Zs8+++wAknOmspo5iwjnleMEkq1N6PrQEvp5uz4aQtcP3RzaQkCPn7KazrXre71qrij8TTV3a4aLthOgG0DHrHWEWNMr70wrhePTTBS24gCSGY6sw6V1yHT/ZAl+rV3GOllAZdx5rNm0EITG/K2uOc41W+kAlXMCVLJyaz2nev6YaaQuYK0jk1azSMen2cpjxowBUMmEzQs9vk8//RRApf0GAGyzzTYha4Zn2v6u+z/DALROlGYw87y6BYQxxhhjTCOoukVHm6uxHgcA9OrVK2RWhtTKmEWx2DRE1lOmPuky8EzrrGhgIOsI6LnKsn7Ui7RxaZ0Klfm3WW9nDT2Jc95rXRm5ufBNmRU8gWTgJ4MM06yQrQGe6ywrYVrNDA1Gzqo4XA/02mgoqF0rO3fv3j1kXpdpzVvzhtYIIFmHio2Mgco5YNIGABx55JEhsz6LBrimWezqCX9TLcPalFmDXLkvaLV83V9Zk0YtOtpgNw+LFas4a7XjP/74I2S917Gm0+effx46DVzm+HXN5oHuafQy6PxpHaq//vorZCY+aGXrX375JWTWedKmrRrEXZPA8ap/ozHGGGNMQfCDjjHGGGNKS4tcV+p6oLlJy1bTRQMk6zjQjNda3FVAxQyq9VI0AG7w4MEha4PHNOgaYVArkKyjw3YaeZ+fhn4/y13TULDq6NGjARTLXaXuGJqctfbTyiuvHPKOO+4IoLiut2qh7qqXX3455PXXXx9A0vVQa3h+9ZjUNa5NBV944QUAyUaKLKsP5H9dpcHx6ZraaKONQtYg5ak/A6S7jovoTlUXnNaWWX311UNm6wrWGwOSNZ/opjrooINCp6EB9boW9fxyrjSoVptOawPnzz77DEAyQUcbJLPdjrp+8pjLtDpU2upmgw02CFnDMNjsWMMd1M3Pdk8auF3r8dmiY4wxxpjS4gcdY4wxxpSWmaZn5ps0adJ0bYBqZmUGx9577x067X6t9Vnq2bohjXbt2s0ENDy+NHTMmrWh0faNNcOx9gKQLOHe0joCHN+UKVNy9aeoO+i6664LmRkGWkdB6/A0dP7atGlT9fHpsQ4YMABA0gWprkm2tqiVuZXja876rAY8FzRBA8lO55S1dkZTzkW11qfOmZaQZ/0fltIHkhlItTaT12J9FomWrE/NpNKsm4EDB4ZM96S6HtVlotmspJruKq7PyZMnN/pLGY7A9jBApZUPkMxKSgtd0L1m2223BVA7d1zbtm2rcv/T609rOvFvsjL9eP3VyoXM+VNs0THGGGNMafGDjjHGGGNKS4tcV2lkFRMrUgZAS1xXSlY7iOZQzfNTFNeVml7VpHvMMccAAO64447QsWM2UD/Xlc6Zuj5Y2rxHjx6h007ltS6Yl7frimSNs4iu1bRjzatInl1X2TRln8z621rfS5rjuuKxqjtG25I888wzITObSl1bmoFUixYISktcV60Bu66MMcYYM0NRdYtOa6BaFp2iUhSLjr6RaZ0IBuZ17NixWd9bizdmbRbJOh7LLrts6LLK6deColh0akVR1metsEWnddMci04aGqCrcloLlnpaHG3RMcYYY4wpEX7QMcYYY0xpma7ryhhjjDGmNWOLjjHGGGNKy3SbepY9mK6lwWZFhcFmZR/fhAkTSjm+Dh06zBDBgmXfXyZOnFjK8bVv336GWJ9l3z9rPb7//d/K44WWf2AKfq0qI3N8ii06xhhjjCkt07XoGDM9sgomNtTrxHFhxpiWonuO7ilNKUpYy72ooYKyWb/dGvfHtLl4+eWXQzdixIiQ2curU6dO0/18NbFFxxhjjDGlxRYd02RY/EqfvL/++uuQb7vtNgBJH+2JJ54YMltDFP3NpaFOvWkdeVVureOjXv9fx1cr37oxWaS1+NA1qQX5pkyZMo1u0qRJIet12a5du6ocH68V3fO0IOD48eNDZnd2PT49Dl5frWn/UPr37w8AOP/880O39tprh0zrzk033RS6WWedNWRbdIwxxhhjmoAfdIwxxhhTWnJ1XTUUNFZ0092MhJpZv//+ewDA/fffH7pLL7005D///BMAMNdcc4Vut912C3mppZYCUCwXSFrgoJrG2f8KAD7++GMASXP6MsssEzLHp/9fpLXMufznn39C980334T822+/AQB+/PHH0Gng4HLLLQcg2Z2+1uPj96e5EBtDQ67FpgSw5kFzjq9Ia64hdF7TupNPmDAh5OHDhwNIdgdnR3Cgcn3q/vP222+HvOmmm4a89957Z/5mU46ZvfI+/PDD0D377LMhq5tmhRVWAADMOeecoevZs2fIq6++OoDk9VXrju3NQcf/1ltvhXzSSScBSIYr7LzzziE/9thjAIC///47dOq6qgW26BhjjDGmtPhBxxhjjDGlpcmuq7To9+npiZre0trUKzRJZ5lrW5NJNo20rJa0DJe8x6lz+t1334V8yimnAAAeeuih0M0yyywhM4Ng+eWXD90888wTclHMsFm1G7766isASXPzgw8+OM3/6+fprgKAe++9F0DSnZX3XCo0/V9wwQWhU9cc3QSLLLJI6K666qqQeX3q+tBrmWZ8oJJh0lLXEH9T3WlnnHFGyBMnTpzmM5rJcvjhh4fcpUsXAA3vWXmg50ldc+rm5VpSnZ5/Zv5oBpDCcWdlDdYLPf+///57yFyLb7zxRuhGjhwZ8iOPPAIA+Ouvv0Knbp609anndeONN67KMas77cILLwQA3HDDDaGjCx8A5phjjpBfe+01AJUQAKCyZwBAv379AAC77rpr6vHnuZfomhwzZkzIBx98cMh0zR111FGhUzcdXd9p7uRaUbwr3RhjjDGmSjTaosMnSg1g1Kewn3/+GUD2U/QXX3wRMp/OH3744dDp28maa64JAJh33nlDt+GGG4bcrVs3ANkWoWrRlMqWWTVJ0v6Wb5/6RsLaD0Dl6Z9vw1N/vpboOLT2BK04QOWNapNNNgndAQccEPLMM88MAFhsscVCp8FmRbHo6Dxde+21IZ9zzjkAkmtdLTZnnXXWNN/11FNPhcy3m+effz71t+o1l/qbarHZbrvtACQtTuedd17Ia6yxBgBgoYUWCt2//vWvkGmx+eCDD0J3zz33hKyfO/DAA5s/gBR07cw222wh617x008/AQBeffXV0D333HMh0zq30korha7We0lD8LrTfUDP71133RXyuHHjACTfkmmlA4Czzz4bQHLP1PNG64lafP7973+HXK9qwe+//37IO+20U8jLLrssAGDHHXcMna7V9u3bA0haSXTN0RK5wAILhE4tyh07dgy5sXsRj1uvqTfffDPkSy65BEByn7j66qtDXmWVVULu0KEDgGSAtF4nvBYZlAwk99I8LTo6f08++WTIo0aNmkav61Ovr7TK+bXGFh1jjDHGlBY/6BhjjDGmtEzXdaVmKrpbTj/99NAxKBOoBFhl1bbQnHm6bDSATGFOvpq7tPYBWwzMPffcoatWTRY1ZWowWdr36/nRAEy6CVSnrjuaLPX80RwNVAI/11133dDl4e7QYEA1/dOMrCW+aW4G0oOpi+KuynLNqeuJgdUMWgWAQw45JOR11lkHALDwwguHTgN3jzzySADJYD39/zxMz9pgjy7RW2+9NXQaOM65UjeKrgW6UbTthwYe9ujRI+Rqmal5/Wm9FK3dlBbMf/fdd4fuhBNOCFnHlSdpAaYaIK7X14ILLhjyQQcdBCBZm0THx/XLekhAsubM5ZdfDgDYZ599QqduolrUt+J9Qes16e/TnQpU7jGzzz77dI9J7zVpTYMb0zSzqetS186LL74YMt3VWjtm0UUXDTnNdTPffPOFTlskPProowCS1+ziiy/epOOsNjxmvaep607vVXTTZa2jPPY/W3SMMcYYU1r8oGOMMcaY0jJd15WamGjuVhfL0KFDQ+7atSsAYKONNgpdVgZWQzU1+Lmnn346dGom3HfffQEAAwcODJ1G4DfHNMbf/PTTT0O3/fbbT3PMWaXK1aTHDDTNVFE3HfVZ3b8pqzmw1nBcWo/kzjvvDFndPCeffDKAZCaEmmaLVDNmeqjp+4orrgiZGWKaiXLzzTeHzKwzrkMgmdXEbJZ6ZbI0BjUjMzNE3SE673S5XXzxxaHTDMm99toLAHDjjTeGTt0MaTWzWkqaCyxrzdFlo7VJ1I3AbJw83KlZex/XF91KQDJM4IgjjgiZe52uX+2OTZdk3759Qzd27NiQuT4107Be61PdHXrMm2++ecjMdmT2JpBeUygPF6TOn7qz6S5kRhiQvCekfYd+1wYbbBDykCFDACTr9OQN7w+8twHA66+/HvIuu+wSMu91RWrxY4uOMcYYY0qLH3SMMcYYU1oaXTCQpkM1p+6///4hr7jiigCS5vrmln2nmUwj0bfeeuuQR4wYAQD48ssvQ6fFlVpiMtNMrqWXXjpkZkqp60oL+q222moh07S85ZZbhm6JJZYImcWfXnnlldBts802IWuhpXrBufrll19Cp1H/WpCLLrWsIolNyXqoN/rbesxakIvZdrqONNuH80YXHgAMGjQoZHZqV3dq3uj6++STTwAATzzxROi0+/Bll10GoFI4EEh2que1pvOfh5la509bdPTu3RtAssQ+3W1ApdBgVvZNvTqZ6zljJqkWnEvLpAIq7gMtEqjZPtw/2EUaAAYMGBAys7W0oF4trkk9j3QzqguUhR2BZLsDXot77rln6Lp37x5yWnfvWq+/tPOjhQcbamukpLmu0jKQ1U1UFPQ8a0FVZqI2l1q3uLBFxxhjjDGlpdEWHQbuae0QDfDjk141A/xYKnvq72UQbOfOnVP/vznw8/qUrsHOLJueZcXQz/HpXJsKKvwOfTrWJ3q+cdbrzVJ/SwPoNPBc31T4VrbqqquGTt8+OC9all+DCSnXImi1uWiw9ZlnngkgGWCs88PWAnr82ijw1FNPBZAduJ4Hap2hpZL1WKaG7T5oGQHSy7nnPWf6+1rin8etAdYaOM3xsREjkLxW6zUuvSbYAkCtMLQMAskWKrRkawsZDaan9UPbRqj1m9bxrKas1SLNenrYYYel/j/rsAHAH3/8AQA499xzQ6f3gvXWWw9A0uKl7TzqNX9Nuab1XDMIW9umqMWNngK1MtfzXpAGf//XX38NnVpMtQXEWmutlfl5oOEWSWkWr5bWZLNFxxhjjDGlxQ86xhhjjCktjXZdETUbVdMcr2YqBjn1798/9W8ZmKeBz9VCTWRaG6ShwNK0wMasADma7vT86Vg0SLBe8Fi1tkqvXr1Cvu6660Kma0a7H+tYaGZW15WWmGfNDHU9FsU0C1TalWiArpr2aVo+9thjQ3fMMceEnFYnKQ90TO+9917I3377LYBkMOE111wTMl1aWTVrioIe35prrhlymulcg8npHtpkk01Ct9VWW6V+b7XJCnree++9ASRL/WsdK/1buuY222yz0On+xL/V7tk6Vu4v9Qwg57jVhaoJHOpm5LrUYOX77rsvZNb/efbZZ0On7Uw4l3m3+shy1zz//PMAgMMPPzz1c3STn3XWWaHTMAEGrNfTHc75Uxev1jn6+OOPp/mMumb1WJnw8sEHH4SOCRL6t5pgpC2GtFZRY7FFxxhjjDGlxQ86xhhjjCktTXZd1Qo17T322GMAkuZKjbqnyTatdks1qbVpUOvoaM0emqHr6frgb2ltoOOOOy5kLUf++eefA0ialrUrMTO3tPu7dmVmuXut46J1eqp13pviDtOsqjPOOANAsk7UCy+8EPINN9wAIJkJo+0+SC5deuWa0O7pRx99dMicH3XNqhuL5J0p1lx43vVc9OzZM+RnnnkGQNK1o66repHmxmJGEZBsC6BwXrLCCOiaVNeNdgen6b/Wrp209a/HqfOj1w9lzaRTl96mm24KINn9XDvV8xxmZe3mgZ4LHp+6i+kuByotgDRTkHsOAFx66aUAal97RuH36z6tLuKXXnopZLpE1R2l9/Inn3wSADBy5MjQderUKeSVV14ZANCvX7/QaU06zcajK62h8duiY4wxxpjSUkiLDgOb1CKgwYasmJnW6K01wCdxtYLo2xtr8uQxJn3z0SrRZ599dsh8E9Tj0zo6DFLWpor69sz6PFoHpFpkWXHY6FEtN1ptVmGwptaM0sqsDIyj5QdINiVccsklAeRj0dHxDx48OOQvvvgiZDYl1WBADTZngz4Nim9N1xfR8697Bef9/fffr/sxZZGWwNDQOde51uuWSRysRwNUKtdP/bfVRvdxrbLOY9V6Yw0dh45fA19pEbnllltCpxa7H3/8EUCyGnjeFh0NtqbF6uCDD079W+6vmuyhFit6NDTAXKmld0Mtb0cddVTImrhC6+hXX30VOl0XPXr0AJC08muQOtFkD618zmQYoGKdtEXHGGOMMTMsftAxxhhjTGnJ1XWlplcNdmVpaTV3sS0CUAmYzbtOSXNhTRJtsaB1dGhmz8NdoHOi7SDOP//8kBmEtu2224ZOTcsvvvgigGRTUK3DwhYeWmenWnOpx6+mUx6/1q5g0NvUv5/WoE/NyHRN6f/TXA4kGzPWC45bTeRaVn+++eYLmfUpNAD+oosuCpnBggcccEBtDjYH1HXBdh9am6M1ovujuvkZGKquDW3XU0s3jl5/6s5+6qmnACTdNeo6Trv+VKeJD3Tt6FrXFiWNDVCtBeoifffdd0NW132fPn0AJPcJ3Ut4/FqbS2sqXX755QCS4Q76u7VE1w5dUEAyMJkNgNUdpS0uWD9N7y+aOHHHHXcASLYwYYsUIHmvaewc26JjjDHGmNLiBx1jjDHGlJZcXVdqelXTJLtDqxlUS2DTjdWaXFdp9U20jkBa2fq8UTMla+cAwNChQwEks3qY1QRUXFfq7jnkkENCphlSy9a31Jye1hGe9Xr0+9VdpetLTb9ppnPNUDr00EMBJN2tmk2SBxyLmsA160Y7IbO+iJZS32ijjUJm13ats1LrmlW1QI959OjRIbMmkprGWxOca52H4cOHhzxixAgAyRYmei5q6RLXY5prrrlCputF3aldu3YNWd0YzCp6++23Q6cZnMzW1Kyu3XffPWS6xPPItNJ9ROdk4MCBIfO8MLsRSGZ40g2n39WlS5eQ88yA1PnV/V1rrtFlN3bs2NT/576v7v4vv/wyZLYhUnff9ttvH7JmzjZ2jm3RMcYYY0xpydWik/VGQuuBBqtuscUWIfPtpIiNBrPQN6p33nkHADB+/PjQabBonm/M+tvawE0rU7J+jFqkNFicQWj6FL7vvvuGzCfyWryZ6FuQVtt8+OGHASQbxWptHD1+1nHSys0MsAMqx8+gQKASYA3k88aVVtl6kUUWCXnIkCEhf/fddwCSVU71c3y7bi2WGyC9yrBaCbQOFC1ZGgzamsZKdJ0NGjQoZFq/df3Xa3z6OxogzL3giiuuCB2vyam56qqrAFSCxoHk+mTihiZDaJIB94A8LDp6T9p1111D1sBszhWbkwLJuWLg+KhRo0KniRU77LADgPwbIev6U4/LG2+8AQD49ddfQ6eVkWmp0/u7WpSPOOIIAMnK7bqumjOvtugYY4wxprT4QccYY4wxpSUX11Va4CRrdyjrrrtuyCuttFLIrb0cPV1X6i5hAB5QnPHpMe+1114hM6BXW1houwgGzun49LtqMT5+v5pzd9xxx5CHDRsGAOjdu3fo1DSqDQAZxJsVDH/yyScDANZZZ51qHHpVSHNdae0JDeZkfRxtC/D444+HTJecBhvmXUI/jTR3lcra1kIDc+m+bEo7giLB8WkwrjYIvvLKKwEky/XXa0/ROVF3/UknnQQgmQyga1KvP6671VZbLXTzzz9/yHTzq+tR3ex5zqX+tjYl1XYx3Eufe+650J1++ukhsxmmuuG1wTDPpZ7fvNF557rT2nAnnnhiyGkut7T7QzXXbHHOlDHGGGNMlfGDjjHGGGNKS2GyrjSDh2jtDzVNFsW10xBZJdoZla6unSKa0bNqJnTr1g1A0rScZnrMYxz6m3p+me2hLihdc1rThyZndkkGgBVWWCFkmmbTytbnjV4b6prq27dvyBdeeCGAiokcSLqO2RqiKGPKQk3geqzsaq1j1hL0rFmlayXLDVZkNGtT5/2ff/4BkMw6q1eLACWte/zmm28eug033DDktHOue37a/+uYi7JnKnpMevxsgcBWMgCw8847p36O6F7G81LUrOO0OmR5z48tOsYYY4wpLX7QMcYYY0xpKUz3ci2Y9MMPPwAA9tlnn9AV3YzeEFr8iu0StAR/0c3lev6LajKdGjWX0jV11FFHtfh700yzRadnz56pchqtZXx6zej1xRYB2qJES+inja/oc2fFlwAAIABJREFU118a6m7VFgIPPvgggGSRz6wMyHqR9puaIZhGawlRaAxp+6euuYZayOhe1lr23yJhi44xxhhjSstM03u6nzJlSt0e/bWpJ9/IWAobSAbDtvSNpE2bNjMBwOTJk2s6Pn1i16dwBoNq0zs2igRaPr62bdvWZXx5wfFNmDChlOPr0KHDTAAwadKkUo6vXbt2MwEt31+yLDJpLUbSAo9rZcXh/jJx4sS6zZ+2GGD9Eq2jkxW43Rzat28/Q6zPsu+fZR+fYouOMcYYY0qLH3SMMcYYU1qm67oyxhhjjGnNTDfrqp4+5npCH3PZx2cfeuuEPuZaz19aXAtQm14zyowyf2XfX8o+vpauz6zri/FieWVPOUbHGGOMMaZE5FpHx5jWjL6xabsPvrEV1S3M4x43blzobrrpppC32GILAMByyy0XurxLuFcTzlWZxmSKge4Jf//9d8gDBw4Mee211waQbNFSxHYyZcIWHWOMMcaUlqpbdPSJVmvf6BtvQ02/ivhGzHHpOHSsWfL00BiIolcB1TGxQZ+ei4ZoTWNtCJ4LHccff/wR8iyzzAIg2UixiGtZqwlfeumlIX///fcAgIsvvjh0addv0cmqHcO5mnnmmUOne5UtPfnS0v0lz/nTY/79999D1mvprbfeAgBcd911odMq0a3l+mqILIs357ehe4LunypPmTIl5MaeK1t0jDHGGFNa/KBjjDHGmNJSNddVmjn8+eefD/m9994LuUOHDgCAxRdfPHRLLbVUyNSr6SsPc2RaC4dvv/02dGpC+/XXX0PWcuzT+95u3bqFboEFFgi5KKZLHf/kyZNDHjVqFADg888/D50G3tHMqK6BFVZYIWSOtTW6Q4DKceta2HDDDUN+8cUXAQALLrhg6IrUiI/nWlsEzDnnnCEvueSSAFqvOyfNzfzwww+HfOyxxwIAbrzxxtB17949ZJ6X1jhmlbOuryKGBijcXz788MPQpbk22rVrF/Iaa6wR8uyzzw4gn/nT41xkkUVC1hY/dFmNHj06dHr/KzJNCdfQ9fXNN9+E/NlnnwEAVlllldDp/kPGjBkT8jvvvBPypptuGjL3qIbWsi06xhhjjCktftAxxhhjTGmpuuvqiSeeCN2TTz4Z8hJLLBEyzXuvvvpq6M4999yQWb/joosuCt1ss80Wcr1Mkmr6HTlyJABgv/32C526c9R1xe7rSpqZ77jjjgvd+eefH3JRspL0mL/++uuQ6abROix0RwIV15VGyqsb67777gMArLzyyqErypibgo5v7NixIX/11VcAku7IIkFz7yOPPBI6dUNqBkhrQa9VulGvvPLK0F199dUh8/o88sgjQ6f7U58+fQAk3SFFdPNkZSV99913AIBBgwaFTmu2bLLJJgDyzwrU4//0009D7tWrFwDgiy++CJ2GRPBY9fi33377kC+44AIAydCIvF3H7CgPVLIan3766dDRXQxU9t0irTkek94TdM9T1xTn5e233w7dzTffHPJHH30EANh5551DpxlodEm+9NJLodtjjz1CHj58eMhrrrkmgIbvH7boGGOMMaa0VM2iQytL165dQ7feeuuFPNdcc03zt2qZOemkk0I+77zzAFTerADgmmuuqdahNhp9ouYTuY5PLTfzzjtvyIsuuiiA5Pjef//9kIcOHQoAmDBhQpWPuLro+PXtaNiwYQAqQX9AMpiMb2p6fnQtMBhNLTr1Qt8C9e1E3wga+yaln584cWLIDKLk28bUf5v32zMtHkOGDAmdVkHeaqutpvl8kd4uSdb5P/300wEAffv2DZ0Ghvfs2RNAMhj7nnvuCXmfffYBkEyg0MDXvM8F53L8+PGhU+sV67TMPffcoXvggQdCnmOOOQAk12feFo8BAwaETIuU7jlqkeI1rAkQah1hMshVV10Vus6dO4fclPo81aKxtdWKRNqepRXUaZkHktcKLcZZ9W5o/X/00UdDp94BJunce++9oVOPzkMPPRSyruHpYYuOMcYYY0qLH3SMMcYYU1qq3gKiU6dOIau5Ks00qiZEls0HgJ122glAMvBX3UD1MgPqby677LIAkkFVaprTYNz27dsDSLqmjjnmmGm+V2uvFBGdvzZt2oS89NJLN+pzGgCqpkk2tas1aeuEgXAA8M8//4S80korhcx12ZCLIquOBOuA5LFmlazfpElYTccamEs3bEPHn7cLR49J3aQ0o2uwo9YxWXfddQEk6yBpTZ2tt94aQLKOUN5jVZcr9xXdU3777beQ6ebXOi4M0AWAc845B0AyGD3vmla6P6y11loAgDPPPDN0aQ1mdX+luxEAnn32WQDJANfLLrss5HoFYev6ZAAyUAnj0P0/rSlwQy62rEag1RqTHv8PP/wAoLJ2gKS7uKGaW3pMs846K4BkMs/LL7+cKqfRnPHZomOMMcaY0uIHHWOMMcaUlqq7rppS4+bPP/8MWU3PN9xwAwBg9913D52aG/Mo7U1zmdYY0UyMtK61mkl22223hcxxsZ4FUPxy82mmUZ0TdU2yJpJmyj322GMhzz///ADqWzuHx6dzorUtdH44rw2ZSNV0q39L117e7gD9fa15wfpU6i5W105DbjbOm5qr86iDpNcMM4mASlaGrk/NsOK+c+qpp4ZOszpOPvlkAPm3wMjKKrvkkksAJPcirZnDDEitbXL77beHvPfeewPIJ/soi4UWWijkhRdeGEB2nS1mWDFEAEi61pltxnADILlX13Kt6pr78ccfQ9ZMIY5PM5F1LuiSY/YZAHzwwQchc3zaVofuICC93Udz0PXHOlt67vT6SGuXpJm2m2++ech0gzH7GEiGFKR1N9fr+/DDDw/Z3cuNMcYYM8PjBx1jjDHGlJaqu64aA81cGhV/2mmnhUyTnprr0qLS80BNZWquUzPlGWecAQC4++67Q6cFyw466CAAyUwtHVPeGR5p6FhpWtRMD81qYFaPtgBZddVVQ86jOBnNoH/88UfotO2BZnComTsNngt1d6lpdbXVVgOQXLP1GnNWEUQW4QQqWWGaicMil/odWVkRLDTIwntAZcxAPtdnWgannovXX389ZGaOsLAeAAwePDhkFpfLuy2JukF0fxwxYgSApDtY1x/3ot69e4dOW1wcddRRAJLnJ2/X+cYbbxzy5ZdfDiBZhI7uDqCSIaiuf+65QGUt6J5b6/HxXOrvaHE9XnNAxbXfsWPH0L322mshX3rppQCS86v7E8enLUo060yvS95jWnpP4bHqmtT9Qa8/uhHVxaQZyrwvsHDs1N/Lv51vvvlCd/HFF4dM1x9g15UxxhhjDGaa3hPRxIkTa2Ja4NOvlvDWYCSWA9en+H79+oXMWj3NfeNq3779TEB1x3fCCSeE3L9/f/5O6DTwk0/iWudCg0E5vuZaeTi+SZMmtWh8+sanc8X6ONdff33o9I2DQWZam0af6FtKu3btZgKAyZMnT3d8adYNra2ibxTaYJbBnFltIVh6f5111gmdzuVdd90FINnItClvlG3btm32/GmA4BtvvBHypptuGjLrxOj86focPXo0gEpQLpAssc+aJxrsqTVZ9I0r7Rpt7Pw1hK4prcPCwGtt63DHHXdMc0zaFkFLyeubanPg/DVnf9E32zfffDPkvfbaK2RaTLXFg5bjZ+AxGxEDSesyr4HmXpPV2j/VCsBGuACw4447AkhaQbWpJwOX77zzztBpA11eqy3dP5uyPjlv2pxUA3A1WJrXElvFAMABBxwwzXfxPADJwOwvv/wSQPKa++STT0I+8cQTQ2azaL0OuT6bsn/y/Ot18ssvv4Ss38953WGHHUKn+wfvJTo/Ux0fgIplDwB22WWXkBuaV45PsUXHGGOMMaXFDzrGGGOMKS25BCPT3KSm/e7du4fMwEY1tx522GEh02SpwVz1ChzMCvZUVlllFQBJc+uvv/4a8lNPPQUgWRvj4YcfDpn1Z/JuEaGm5S+++CJkujzUNac1aVgnSOvorL/++iHnMVcMnH777bdDpwGcGozK+g/akV3rVNDMThcPUClbDzQczFwLOFe6puhCBZIukeOPPx5Ackx6XnitaZ0rDTZ///33ASRrt/z0008ha2Bzteaa49Ng8mOPPTZk7cTOv82qY0RZ53zJJZcMmesij6QAnScN5lfX8MCBAwEAzz33XOi0uzePW9fktttuG3LencrT0P2Dbp4xY8aEThM3GJit7qo8xpRWO4Y1qoBk2we9VjjHxx13XOj0Xsh7wfLLLx+6tLV48MEHh7zrrruGrEHc1YLuJNa4A4Arr7wy5CeeeCJkzpW6jtNqNqnrVK8/Bh6zVQvQ8mQdW3SMMcYYU1r8oGOMMcaY0pKL64pkZaLQJLjnnnuGTjMIhg0bBiCZQVNreExp9WQA4KyzzgqZZmY1N6vJne0u1DStJdrZpuCZZ54JnbpZ6mVS1/lhbRGgUpNEXTRpLSDUtaB1WOjyqqdrIK2tgboTWVsEqByfZo2pG5Gl9dU1qa45fr6amWYNQdOwlo1//PHHQ9Zy8V26dAGQzDTTDA+O+8EHHwydrj+6lNV1rCX861WTRUvoa4bHVlttBSCZCafrk9eVuhl0LTJTRddMvdaqHidbNQDJdg5sMaLXl8JzoZmgebfrSEPPr9aRYc0ZPWbNquO61qzHenUkz4JrXl3AmmnVo0ePkLlv6P1B6wgxmzFr/fG8qJtar3tm7baUtJpxWg9N6xgNHz48ZK5hnT+F++J2220XulNOOSXkxRdffJrfb+mc2qJjjDHGmNKSq0Uni7SmkXzKAyrBoPV8cudT6j///BM6rT2isgZ5En06p6zBgvr2ySA1rS2kby/1eiPT86vBgAwc1P/Xtxda4jQYTWsusM5KredPv5+BxfrGpQFyan1hMCCDPoFkTQe+iTHoHEhaFPIM9lRrih7HlltuGTKtAxrAuNlmm4XMOkl6ftT6wYrCrOcCAPPOO2/I1Q5ABirXjF5ntBzq/wMVS2PaNQdUkh3UyqjXGi2uWY0Sa4nOn9Yj0sB+Xmv6xqxVdGkx0KaKRam2rvOge6muJc4xK1gDycBe1lRTK2wtAuCbQ1bzabW+cF51T9eaOKxYrsHG2pSWHQPUosfaXgCw3377NX8ADaDrSBvhpgUbZyXuUK918rSzAJs+65ht0THGGGOMycAPOsYYY4wpLTV1XWWZjhsKVuTfqjuBZa+BihlezWW1MFeqGfL5558HALzyyiuh++9//xuymtYaG4ypdWjSGpVpsGve6Ph4rnVOda7YdFAD17Rcfb3M6Po7XCt6HFkccsghAID9998/dGpGZ7CqNrLMCsyuJWkB1rqm1PWiDSxZpl7dTRoM+PHHHwNINtLTEu4333wzgKTpvVoByHpNa20eHrOWoGdtj6lJW58qM3BTGy3qWqVrKG93j55TdVPxuPSaU9cH16/Wpsl7LETn99133w2ZbWOASouS3XbbLXR6TTGJ4MwzzwwdG2EC+dRB4r1CkxLUHfqf//wnZLb7WX311UPH2mpAxXWlTYN13+L36vxr2wdNnKj2dalJNdp2RF1zaUHImjjA+ly6J3HOAeDCCy8EkHTdtTQxwBYdY4wxxpQWP+gYY4wxprTU1HWlZjOtA7HYYotN87dq0qT86KOPhk7rBLAmQa2j69VcxtobGt2uv69ZR2kuhTR+//33kO+///6QaZJX03O9yDL3p7mBdM60nD6zxm699dbQaQR9HllJPP7GrBmOW9evdjqnaVxdW/Uyk+ucpJmLmbEAAPvss0/I5513Xsg8B0sssUTotDs23cTqur322mtDpuu4pWXZ00hzFwOVDBstbz/ffPNN9/f1+LRTNLuyaybPRhttFDIzDOtVD6gx6Ph4juhiBJJds1k/R6/PorR90HHoWtU6TWmuD83wYbZgr169QpeWoVrr+0PanBxzzDGh09pAWtOKGXLaYkYz0LjuNCtJr3W6zLXOF9u6ALWtmaTfp/evtGtF9ypt15FWh0zd0KzJk9bRvbnYomOMMcaY0lJ1i44+TT700EMhjx07NmQGyykaWMXGYQzKApJVhGkdqPVbij6xs+aN1ijQp1ANpmJgsQaG6tMtn9R1fFoFmUFYWqel1m+XaQHg+sSt1iU+yb/xxhuhU+vB0UcfDSBZuboo1VgbA8/Ft99+GzqtU8KKnlotOo8A67RgXH3z0QBIrZbKa0nfqPWNsmfPngCSdXbSgslrMWZd58sss0zIfKPX5pQarKpNEbnXaLCrNsDkuK+77rrU7yqSJYekJXPo8WtgPGuOFfGa03Orln1WswYqTWN1T1R4f2C9o6lprEW9mnBcmlSiAeJqUaQljuMEgL///nua71xjjTVC1irCrEieZX2vxbzzWtc9p1u3biHrWNKSAdTilrZvTJ48OWRey9WcR1t0jDHGGFNa/KBjjDHGmNJSddeVBsCNHj06ZG3KycA/NT1qOfeff/4ZAPDAAw+Ebrnllgu5XoF1amblMWsp/CuuuCLkAQMGhEyXlQbYKWyHoAF2dPcAlZoIaiaslzldg061LHvXrl1DZp0cNUH27ds35N133x1A0o1ZRHdAFmmuKzUHs7S+1s7Jo05JmmlXz7Neixq4uccee0z3ezkW/a56zZ+e5xVXXDHkIUOGAABOPfXU0GntH73WaPLX9afB2AymVtdJNRsI1gJ1SX7yyScAkq4dTdwgRRyHosenrv9bbrkFQGUfmRque20qqW7OPFx2ae5cXZNac2rdddcFkO16IlnXX9q81qudjrqIdX7UNffSSy8ByJ4HjkXrfKnrknWGqtmo1RYdY4wxxpQWP+gYY4wxprTMND2T0MSJE5tsL1IT3Lhx40LWcvIvvvgigGQmj2Z40MyuHbOraY5s3779TEDTxkc3gI5PXXPqZmN9DnZZByplywGge/fuAIBddtkldOoaosmuue4Cjm/SpElNnj91XbEUN5B0A7Bmw7777hs6NUPyuGtlTm3Xrt1MADB58uSa/ADnWLPO1M3KsWrtpGqOtW3bts2ev9ZAc+aP14Rmh+mc6PrUbMc0Ha/lWrk4OH/N2T+z0LVG14C67jTDh+6FWl1/zdk/G0Lngq0Brr766tCNGTMm5EUWWQRA8p6iNZVaOq8cX632l7zh+mzO+PT+pyEkEyZMCHngwIEAkm0tll566ZAXWmihab5LM7EZstHctg8cn2KLjjHGGGNKix90jDHGGFNaqu66Snx5I9oJEM0QScv6qCYtMb1mjak5xY2yMj1aanJuietK56GhMdXq+Bui1q4rouPX81Jr15xdV9k05prLIytFqYXrStcfs1JfeOGF0GnXbBaUa02uq+bspbXac+y6ahy1vhc2d07tujLGGGPMDEVNm3o25Ym7iOXK0yh6vY2W0prq3dQand/Wsj7LTtmvvyz0umQ7Dm1boeuzNV7DM+q8tmZa05zZomOMMcaY0uIHHWOMMcaUlukGIxtjjDHGtGZs0THGGGNMaZluMLLT61onHl/rphbpyUWiFunJRYLjmzJlSinH16ZNmxni+iv7+MpevkKxRccYY4wxpaWm6eVNIa0gW2unub06jKk3ev2R1nQdph1/Q7Sm8SncS7LGPCPuNa19/ZraYouOMcYYY0pLrhYdffPQrsTsXtqct7S8YHdlRYt4aXfltBYXM+JbmCkOf/75Z8hcq9rxu4jrU/cH7TTPrsp6Teq1xvG1a9cudK2pICQtxRMnTgzd5MmTQ2b38uaU4m8NpI2LHc8B4MwzzwQAXH755aFbaqmlQralZ8aj9TxJGGOMMcY0ET/oGGOMMaa05OK6osn59ddfD52aGS+77DIAwEILLRS6IpqW1YT622+/hfzYY48BAL7++uvQdevWLeSuXbsCAOadd97QqWsrj7HmYeYuojsEqJwLdY2ozPkpkwn8/PPPD5nr9tZbbw0d3clAvvOm61TdNaecckrI7777LgBgnnnmCd348eNDXmCBBQAAJ598cug6deoUctHnlS65q6++OnSDBw8OmfvPHHPMEbqij6khdN45Fo4TAI466qiQuRf/61//Cl1R9xqStv82pSN4kcIgGto/09Djp1zNcdiiY4wxxpjS4gcdY4wxxpSWXF1Xr7zySujUjaUm6aKhJrhff/015AMPPDDkRx99FEDS3D/nnHOGPP/88wNIurOOPPLIkJdbbrlpfqsW7iw1h/L79Xf099WMmGaaVGh61O8vYgZdVu2m77//HgAwZMiQ0OlaPeaYYwAAq666auiK6FptCsxUAoCXX34ZQDITUjOUiuK60uvv8ccfD5kuC7qoAODvv/8O+fnnnwcAbLrppqFbYoklqn+wVUTPOdftZ599Fjo9F8zG0qwzXZ+tJRtLj/+vv/4K+corrwQA9OvXL3Rzzz13yLfffjsAYL755gtdrdcs948sd02a6033HM0aHDt2LADgu+++C923334bMsei95cePXqEzPtLrcna3zlXI0eODN2gQYNC5lrUcI1evXqFvMYaa0zznS2dv+LdfYwxxhhjqkQuFh0+na222mqh08DBDh06JP6uCPDpUp+8L7jggpA1MO6AAw4AABx22GGhW3DBBUP++OOPp/nM9ttvH/LOO+8MAOjTp0/oZptttpCrZT3QN6YHH3wQANCzZ8/Q6VvuDz/8EDKDHHX+9O3kzTffBJC0Yu26664hH3vssQCSdVrqFSypbwl//PFHyOecc07IN910E4CkFUPfKD/44AMAwLBhw0KnY50yZQqA5JiK9BbNY1Erjgazbr311gAq9ViA4lyLehx6Tcw666whr7322gCAiy66KHR6/lkzSINVix6smxaErQkQXbp0CZnX9SeffBI6vZZ5Dou0JhUeHy2rAHD22WeHTOuAzv8JJ5wQ8pprrgmguhYBkhUMf/HFFwMAunfvHjquQyA5FloUhw8fHjreE4DKvvTvf/87dIsuumjIaRZz/dt6WXT0/L7zzjsh77PPPgCS9wy13nBf0QQBekEA4NRTTwUAHHTQQaFraZcBW3SMMcYYU1r8oGOMMcaY0pKr62rhhRcOnQY+0k1QJNMqj2XEiBGh69+/f8jqZqKcZRpfd911AQDrrLNO6LbddtuQ99xzTwBJ0zPdKQDQsWPHkFvixtJjWn/99QEAjzzySOr/61x8+umnAJLmWA2M69y5MwCgb9++oVPTJeV6ukN4/GpuZql4ALjllltCphtrgw02CJ2aYXv37g0g6brcbLPNQmaQspqTi+L6UdKC0afWFw09j+pa05pU3D90/er6nH322af5riLOj6JuZro56EIFkuv3ueeeA5Dcn5566qnU7yoKuj+MGjUKALD//vuH7qWXXgqZc33vvfeGTvdSruVazK9eGxMmTAj5ySefBADcddddoVt++eVDfuGFF0LmHrTXXnuF7qSTTgp5mWWWAZB0x+r6TjuWNm3aNGEUzUfXDu8DQOWeBVSur5122il0vOcBlfp4L774YujorgKAO++8EwCw7777hq6l47NFxxhjjDGlxQ86xhhjjCktuXYvZ3bK1BTFdJ7WHVnNwdtss03I6rpi1phmtaSh41xrrbVCZjbPdtttF7orrrgi5LPOOmua72iOaVZN+3SH6Ziy4Lj082pavOOOOwAko/+1zhD/tp61ZziXb7/9duiuv/76kNXNdvDBBwNImtOXXXbZkJnVcemll4bu2muvDfnEE08EkGxLUCS4ZtJqrwAV03IRXRx6zWhtHNYeASoug6K0rag233zzDQDgl19+CZ1mrdL1o3VYxo0bFzKv9bzPiV5fb7zxRshszaGujQ033HAaWd3sDz30UMjMcN1yyy1Dt9hii4Xckppeuv6++OKLkNk2RV3jek1pi6PNN98cQMXFAyT3z7Q9vaGswFrPJfcCXXN04U8NW8doWxXdSzjv7733Xuh0fD/++OM0upZii44xxhhjSosfdIwxxhhTWnJxXdE0p6YrNfPl6brKykRhp2CWxweAp59+OmTNsMlyyU2Nmhv1t2jyu+GGG0LHrCigUlAQqHRCb6npkmbCxrTfSGsBofPH1gmrr7566LSgXh7tEuh6vOSSS0KnZeO1UCLRc6Fmds6Pjl8zMOgmSGubUQR43MxuAJLF51ZYYQUAyYKOOr95oudRCz7q8dMNoKZvnb800jJ08s7K0rFqVirdrEsvvXTotN0BXVvqOi9KQUS9ZnicQPK6ZIYVW+EAyfHRJUcXCZC8VjnW++67L3Tq5mpOV3fOhe7t99xzT8g//fQTgGT2Gwvn6ef1N3UfLFL3cZJWpE9brbz22msha1Yws26zxse95Iknngid/u2hhx4KIFmwtaXYomOMMcaY0pKrRWf06NGhK+IbBwPMAOCaa64BAGyyySah0wC3hgKPmwK/SwNg1aKjgXcrrbRS1X63sfDpPq2RG1Cp76Fl2fNA30j4Jsby6wAwyyyzTPdz+vm0txtds2qxYjsPDcAryvoGKmPR9a0tSljHg608AGCppZYKmZaeorx5Asm3P7az0Dofap3iceucaR0klvHX/1dqOW79bl0/av2mVZnNK4Hk+Div+nn9f163WWuyFuPjb2owK1vBAMkWADxWrZOklgRat7TFgiY+MIhZ96SWWiR5zbB9CAA8/PDD0xyzWnn0+tF2OWzAqXXkFJ6rLCsk562e1x8tLlrPSM+5BounHZ+uRa5P/S71iKTtny29v9qiY4wxxpjS4gcdY4wxxpSWurmu1PRPM5TWNEkLJssDNZeNHDkyZLYA0I7kag6uZoAtx68lwLWE9gMPPBDyGWecUbXfbSxprh1tkcDA38bU5Kklajqla2PHHXcMnQbjfvXVVyEzGDet4zUAfPjhhwCSpuXjjjsuZAZhF8m1o2OhSf/LL78MnbpkGVitJfa1u3tRxqXnX9utUK/mcN1T6BrQOijqOuHnLrzwwtBtscUWqcdQrXOR5g7WNaduHtbXyjomumnVta77Gr9Xj13PZbWCQPX/r5ddAAAJo0lEQVQ3P/vsMwDAHnvsETrt2K01ZXgveP/990OniQMMfNWO7OweDgA///wzgGRbgrnmmivk5txf+BldU5qMwrYbGoysbR/uvvvukHleNLBZ3eiss6N1gDQwe5FFFgGQnLNaJ3hwrfDcAsDKK68csl5/HJ/WBtI6TtxLtI7XaaedFjLd6NUcky06xhhjjCktftAxxhhjTGnJtXu51r7QOgns1Jq3iVxNZzSndenSJXR6fGpyTqvD0RT4XWra1JoTRek0rSZgrVlBk7maM/OeS5p5tUvuW2+9FbK6tGg61hYCrJMBVOo/qGlZXQt5dGdvCHUjfPLJJwCSrgF1k9A1cPTRR4cuLessD/S3tR6K1pxiNohmkqXVNNKsHLojgUprEHVH6v60yiqrhFytDBgek2aXXHXVVSG//vrrIS+wwALT/L/uD9o6gey9994h81jVDaPds7P2uMag+6BmWNE1wbUHAN26dQtZ3RjffvstgGTW21ZbbRUys3Z0/K+++mrIvJaPOeaY0KmbpyVzpd+jrsEDDjgAALDLLruEji58IDk/nGPNelSZLRCOP/740GnW2G677QYA6NWrV+hWXHHFkGtxfXL/0OyxG2+8MVWmS0uzqnVNcv/cYYcdQnfQQQeFnFYHq6XYomOMMcaY0pKrRUefPPXti28aeb8Rp9WhUJ0+ZesTLYPFaBkAkoFZDY2Lb0X6FKyBs3nXpyH69qmBeQxsLVJTRb4d6FuYVubU+hd8O9S3UwZTApU6HnyzApIBnHlUfk4jLQEAqFSu1rdoffvmW6kGexalDlBWbY5VV101ZB6rWkTTLFJsvgtUGrXq/6uVUptOqkWnWvD4NCmDb/ZAso4RKwM/9dRTodP19/333wMAjjjiiNCpRYTXgFp0ZptttpBbcq1mVT5mYK7+P48TAMaPHx8yz4U2bdXrk//PfRYAzjvvvJBZkbg5FZCbQtp3au0fDTDWpquE9ZqA9Cr5p59+eug0AYUNhAcNGhS6u+66K2StL9SSuUzzWLBqMZBspHvBBReEzMBynVOtYs76cGzeCiSt/9WsSUds0THGGGNMafGDjjHGGGNKSy51dGjGUtPsRhttFHJDJcprif6mmkZZW2TMmDGh+/zzz0Pu169fyEOHDgXQsDlOz4ma4WnyPfjgg6f5fSBZ8yQPeNwaTK6m/T59+gBIBu7VwhzZHNQcy6B3IBm4SFnnR+t0MMhO23Lk7ZpLQ49f3awahEyWXHLJkNPqHxVlfFltOXR98VizAvXVfULeeeedkM8991wAwOKLLx463Z+UatfRUXeaBtvq9U83uLqrtEXCiBEjAACHH3546HR+0/bVrKaSTU12yGqESpeOBoBrTRZ193KtMugaANZZZ52Q6brX2jJpoQF53D+yxt+UY+G9QF1vGqxL15zqzj///JAffPDBkKvVroXHr8eke+J+++0XMl37bD4LJMMw9t13XwDAoosuGrpau/tt0THGGGNMafGDjjHGGGNKS91cV2ouZul5Zg8AyXLdeaImNDWN0gysmQwdO3YMuXPnziGfcsopAJKZHDvttFPIrA+hkejayZWd0vX8aM0BdpcG8nEp0JytJfS1RDuzdbLMuEWhoeNTHTuyAxXzrbYAKTqazcOaMeouVTM4XTZFcTcC6XVm1F2q5ejpJtFrWV0HzAa5/PLLQ6cZLPyt/v37hy7vDLS0mlR6LjSDcI011gCQ7C6t56Kha7Eltbn0d7p27RryM888AyBZm+nJJ58MWVtD0LWhrquFFlooZLr3dB6KkhXYGHgvzNp/0rKS9Vqly1Kzti666KKQ9VrnuarW/qvnWd2FmvXImkCffvpp6LQ206677gog+UxQ6/mzRccYY4wxpSUXiw6DbbWOh1bBzBN98tU39t69ewNIVsPVYFwN9mMwowaIadMyPpHrk7fWHOBT+Nlnnx06rdJazyfhNPjG995774VO3zhZk6OIVpymoJVN1XpA65XOQ5r1Iy3oNS/S3hi1kaIGIBfx7ZhrTsdx//33h6zBuHxjfO2110KnFrlhw4YBSI5fK2MfddRRAJLJCFnBuvWiod/UtcoGlnmvP/39Tp06AQAOOeSQ0L377rsh0+IDAD169AAArLXWWqHTxIYirs+G0GNmzSoN7E1rpJr2GaCyfvWeolWSNaC9XmtV1x9r+rDCNfD/2rt3HMWBIIzjNQEPCcENOAIpKRlcgKtACpwAAiTuwUEQdyHhkWywWz0fWnsQxo1Nz/8XWR6hcePHeKq6q+6jpz7ud57H+jyJAQAASsaLDgAASNbbUlcaQuv1en9/uYQj61I2X2WF/XQicF6jQ69/oRM8NTTrdQb08xrm9ZoRw+Ew8+dVp4T8uLWOkKbhqj6+V/n4NLTq58zMbDAYmFn+9asTB6uUl4b1iYHadqXf74ftOqYG/Ji0rYg2PdTUrjce1LYJ2oxwuVyamdloNAr7dGFBVh2Wul/T2u7CU27PtJ2JzVO7k8kk7NO2MVpTzZt96n1Ux2vyEX2+6/Nxs9mY2X2LC03TeWpHUz+73S5sH49HM7tf4LJYLMK23iMxvzcdn9ZH8nZF3urB7H5hTlmNcJ9BRAcAACSLFx0AAJCsr5/CR9frtbTYUlYY73Q6hX3aPTd26L/ZbH79O44osTMfq47jmToW/rmiofPY4/OQuLZNOBwOYdvTdJraKTNMGXt8WWazWdjebrdm9t3qwsxsPp+H7VdrV/j4zudzlPF5OwFNh2pqK3ZIud1ulzI+vb+0RYI/V3Qli25n1T8qM03l47vdblG/SD1OTYP4fRerzlOj0Sjl/tPzl7WCsap0W4zni47Fr9X9fh/2ac0mT5PrSk5fiWb2XXNIa9P4dBCzx+kqH9/lcint/OmqRq9ptV6vwz6tP5dVJ6hMrVbrv0JQRHQAAECyeNEBAADJqmTVlc8K98JWZp85qz5PVon2vK7LWeq4Ai2LrvTQru7efbjb7YZ9Va/6KELPk3aC9uKIWjyyiiJdRXU6HTOrf4uOR/Q+0TTNo9RhFas+YtDrU+81p8/UV9o6xKLn71OeeUXp9+/3nxb5m06nYTvrb6GuoPOUUdHu6GXR3+9jMjNbrVZmZjYej99+THmI6AAAgGS9bTJynVQxmfWdYo/PJw5qFEfrKHi590+fjJzXVM/Hpf+F5kXviog9GblqZU1Grqt3TUauSlmTkevqt/x9eHUystLno29XFbFjMjIAAPhVeNEBAADJ+jF1BQAA8MmI6AAAgGTxogMAAJLFiw4AAEgWLzoAACBZvOgAAIBk8aIDAACS9QdJrNQUsQgApwAAAABJRU5ErkJggg==\n",
+ "text/plain": [
+ "
"
+ ]
+ },
+ "metadata": {
+ "needs_background": "light"
+ },
+ "output_type": "display_data"
+ }
+ ],
+ "source": [
+ "# Randomly select 100 data points to display\n",
+ "rand_indices = np.random.choice(m, 100, replace=False)\n",
+ "sel = X[rand_indices, :]\n",
+ "\n",
+ "utils.displayData(sel)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "collapsed": true
+ },
+ "source": [
+ "### 1.3 Vectorizing Logistic Regression\n",
+ "\n",
+ "You will be using multiple one-vs-all logistic regression models to build a multi-class classifier. Since there are 10 classes, you will need to train 10 separate logistic regression classifiers. To make this training efficient, it is important to ensure that your code is well vectorized. In this section, you will implement a vectorized version of logistic regression that does not employ any `for` loops. You can use your code in the previous exercise as a starting point for this exercise. \n",
+ "\n",
+ "To test your vectorized logistic regression, we will use custom data as defined in the following cell."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 4,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "# test values for the parameters theta\n",
+ "theta_t = np.array([-2, -1, 1, 2], dtype=float)\n",
+ "\n",
+ "# test values for the inputs\n",
+ "X_t = np.concatenate([np.ones((5, 1)), np.arange(1, 16).reshape(5, 3, order='F')/10.0], axis=1)\n",
+ "\n",
+ "# test values for the labels\n",
+ "y_t = np.array([1, 0, 1, 0, 1])\n",
+ "\n",
+ "# test value for the regularization parameter\n",
+ "lambda_t = 3"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "\n",
+ "#### 1.3.1 Vectorizing the cost function \n",
+ "\n",
+ "We will begin by writing a vectorized version of the cost function. Recall that in (unregularized) logistic regression, the cost function is\n",
+ "\n",
+ "$$ J(\\theta) = \\frac{1}{m} \\sum_{i=1}^m \\left[ -y^{(i)} \\log \\left( h_\\theta\\left( x^{(i)} \\right) \\right) - \\left(1 - y^{(i)} \\right) \\log \\left(1 - h_\\theta \\left( x^{(i)} \\right) \\right) \\right] $$\n",
+ "\n",
+ "To compute each element in the summation, we have to compute $h_\\theta(x^{(i)})$ for every example $i$, where $h_\\theta(x^{(i)}) = g(\\theta^T x^{(i)})$ and $g(z) = \\frac{1}{1+e^{-z}}$ is the sigmoid function. It turns out that we can compute this quickly for all our examples by using matrix multiplication. Let us define $X$ and $\\theta$ as\n",
+ "\n",
+ "$$ X = \\begin{bmatrix} - \\left( x^{(1)} \\right)^T - \\\\ - \\left( x^{(2)} \\right)^T - \\\\ \\vdots \\\\ - \\left( x^{(m)} \\right)^T - \\end{bmatrix} \\qquad \\text{and} \\qquad \\theta = \\begin{bmatrix} \\theta_0 \\\\ \\theta_1 \\\\ \\vdots \\\\ \\theta_n \\end{bmatrix} $$\n",
+ "\n",
+ "Then, by computing the matrix product $X\\theta$, we have: \n",
+ "\n",
+ "$$ X\\theta = \\begin{bmatrix} - \\left( x^{(1)} \\right)^T\\theta - \\\\ - \\left( x^{(2)} \\right)^T\\theta - \\\\ \\vdots \\\\ - \\left( x^{(m)} \\right)^T\\theta - \\end{bmatrix} = \\begin{bmatrix} - \\theta^T x^{(1)} - \\\\ - \\theta^T x^{(2)} - \\\\ \\vdots \\\\ - \\theta^T x^{(m)} - \\end{bmatrix} $$\n",
+ "\n",
+ "In the last equality, we used the fact that $a^Tb = b^Ta$ if $a$ and $b$ are vectors. This allows us to compute the products $\\theta^T x^{(i)}$ for all our examples $i$ in one line of code.\n",
+ "\n",
+ "#### 1.3.2 Vectorizing the gradient\n",
+ "\n",
+ "Recall that the gradient of the (unregularized) logistic regression cost is a vector where the $j^{th}$ element is defined as\n",
+ "\n",
+ "$$ \\frac{\\partial J }{\\partial \\theta_j} = \\frac{1}{m} \\sum_{i=1}^m \\left( \\left( h_\\theta\\left(x^{(i)}\\right) - y^{(i)} \\right)x_j^{(i)} \\right) $$\n",
+ "\n",
+ "To vectorize this operation over the dataset, we start by writing out all the partial derivatives explicitly for all $\\theta_j$,\n",
+ "\n",
+ "$$\n",
+ "\\begin{align*}\n",
+ "\\begin{bmatrix} \n",
+ "\\frac{\\partial J}{\\partial \\theta_0} \\\\\n",
+ "\\frac{\\partial J}{\\partial \\theta_1} \\\\\n",
+ "\\frac{\\partial J}{\\partial \\theta_2} \\\\\n",
+ "\\vdots \\\\\n",
+ "\\frac{\\partial J}{\\partial \\theta_n}\n",
+ "\\end{bmatrix} = &\n",
+ "\\frac{1}{m} \\begin{bmatrix}\n",
+ "\\sum_{i=1}^m \\left( \\left(h_\\theta\\left(x^{(i)}\\right) - y^{(i)} \\right)x_0^{(i)}\\right) \\\\\n",
+ "\\sum_{i=1}^m \\left( \\left(h_\\theta\\left(x^{(i)}\\right) - y^{(i)} \\right)x_1^{(i)}\\right) \\\\\n",
+ "\\sum_{i=1}^m \\left( \\left(h_\\theta\\left(x^{(i)}\\right) - y^{(i)} \\right)x_2^{(i)}\\right) \\\\\n",
+ "\\vdots \\\\\n",
+ "\\sum_{i=1}^m \\left( \\left(h_\\theta\\left(x^{(i)}\\right) - y^{(i)} \\right)x_n^{(i)}\\right) \\\\\n",
+ "\\end{bmatrix} \\\\\n",
+ "= & \\frac{1}{m} \\sum_{i=1}^m \\left( \\left(h_\\theta\\left(x^{(i)}\\right) - y^{(i)} \\right)x^{(i)}\\right) \\\\\n",
+ "= & \\frac{1}{m} X^T \\left( h_\\theta(x) - y\\right)\n",
+ "\\end{align*}\n",
+ "$$\n",
+ "\n",
+ "where\n",
+ "\n",
+ "$$ h_\\theta(x) - y = \n",
+ "\\begin{bmatrix}\n",
+ "h_\\theta\\left(x^{(1)}\\right) - y^{(1)} \\\\\n",
+ "h_\\theta\\left(x^{(2)}\\right) - y^{(2)} \\\\\n",
+ "\\vdots \\\\\n",
+ "h_\\theta\\left(x^{(m)}\\right) - y^{(m)} \n",
+ "\\end{bmatrix} $$\n",
+ "\n",
+ "Note that $x^{(i)}$ is a vector, while $h_\\theta\\left(x^{(i)}\\right) - y^{(i)}$ is a scalar (single number).\n",
+ "To understand the last step of the derivation, let $\\beta_i = (h_\\theta\\left(x^{(m)}\\right) - y^{(m)})$ and\n",
+ "observe that:\n",
+ "\n",
+ "$$ \\sum_i \\beta_ix^{(i)} = \\begin{bmatrix} \n",
+ "| & | & & | \\\\\n",
+ "x^{(1)} & x^{(2)} & \\cdots & x^{(m)} \\\\\n",
+ "| & | & & | \n",
+ "\\end{bmatrix}\n",
+ "\\begin{bmatrix}\n",
+ "\\beta_1 \\\\\n",
+ "\\beta_2 \\\\\n",
+ "\\vdots \\\\\n",
+ "\\beta_m\n",
+ "\\end{bmatrix} = x^T \\beta\n",
+ "$$\n",
+ "\n",
+ "where the values $\\beta_i = \\left( h_\\theta(x^{(i)} - y^{(i)} \\right)$.\n",
+ "\n",
+ "The expression above allows us to compute all the partial derivatives\n",
+ "without any loops. If you are comfortable with linear algebra, we encourage you to work through the matrix multiplications above to convince yourself that the vectorized version does the same computations. \n",
+ "\n",
+ "Your job is to write the unregularized cost function `lrCostFunction` which returns both the cost function $J(\\theta)$ and its gradient $\\frac{\\partial J}{\\partial \\theta}$. Your implementation should use the strategy we presented above to calculate $\\theta^T x^{(i)}$. You should also use a vectorized approach for the rest of the cost function. A fully vectorized version of `lrCostFunction` should not contain any loops.\n",
+ "\n",
+ "
\n",
+ "**Debugging Tip:** Vectorizing code can sometimes be tricky. One common strategy for debugging is to print out the sizes of the matrices you are working with using the `shape` property of `numpy` arrays. For example, given a data matrix $X$ of size $100 \\times 20$ (100 examples, 20 features) and $\\theta$, a vector with size $20$, you can observe that `np.dot(X, theta)` is a valid multiplication operation, while `np.dot(theta, X)` is not. Furthermore, if you have a non-vectorized version of your code, you can compare the output of your vectorized code and non-vectorized code to make sure that they produce the same outputs.\n",
+ "
\n",
+ ""
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 5,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "def lrCostFunction(theta, X, y, lambda_):\n",
+ " \"\"\"\n",
+ " Computes the cost of using theta as the parameter for regularized\n",
+ " logistic regression and the gradient of the cost w.r.t. to the parameters.\n",
+ " \n",
+ " Parameters\n",
+ " ----------\n",
+ " theta : array_like\n",
+ " Logistic regression parameters. A vector with shape (n, ). n is \n",
+ " the number of features including any intercept. \n",
+ " \n",
+ " X : array_like\n",
+ " The data set with shape (m x n). m is the number of examples, and\n",
+ " n is the number of features (including intercept).\n",
+ " \n",
+ " y : array_like\n",
+ " The data labels. A vector with shape (m, ).\n",
+ " \n",
+ " lambda_ : float\n",
+ " The regularization parameter. \n",
+ " \n",
+ " Returns\n",
+ " -------\n",
+ " J : float\n",
+ " The computed value for the regularized cost function. \n",
+ " \n",
+ " grad : array_like\n",
+ " A vector of shape (n, ) which is the gradient of the cost\n",
+ " function with respect to theta, at the current values of theta.\n",
+ " \n",
+ " Instructions\n",
+ " ------------\n",
+ " Compute the cost of a particular choice of theta. You should set J to the cost.\n",
+ " Compute the partial derivatives and set grad to the partial\n",
+ " derivatives of the cost w.r.t. each parameter in theta\n",
+ " \n",
+ " Hint 1\n",
+ " ------\n",
+ " The computation of the cost function and gradients can be efficiently\n",
+ " vectorized. For example, consider the computation\n",
+ " \n",
+ " sigmoid(X * theta)\n",
+ " \n",
+ " Each row of the resulting matrix will contain the value of the prediction\n",
+ " for that example. You can make use of this to vectorize the cost function\n",
+ " and gradient computations. \n",
+ " \n",
+ " Hint 2\n",
+ " ------\n",
+ " When computing the gradient of the regularized cost function, there are\n",
+ " many possible vectorized solutions, but one solution looks like:\n",
+ " \n",
+ " grad = (unregularized gradient for logistic regression)\n",
+ " temp = theta \n",
+ " temp[0] = 0 # because we don't add anything for j = 0\n",
+ " grad = grad + YOUR_CODE_HERE (using the temp variable)\n",
+ " \n",
+ " Hint 3\n",
+ " ------\n",
+ " We have provided the implementatation of the sigmoid function within \n",
+ " the file `utils.py`. At the start of the notebook, we imported this file\n",
+ " as a module. Thus to access the sigmoid function within that file, you can\n",
+ " do the following: `utils.sigmoid(z)`.\n",
+ " \n",
+ " \"\"\"\n",
+ " #Initialize some useful values\n",
+ " m = y.size\n",
+ " \n",
+ " # convert labels to ints if their type is bool\n",
+ " if y.dtype == bool:\n",
+ " y = y.astype(int)\n",
+ " \n",
+ " # You need to return the following variables correctly\n",
+ " J = 0\n",
+ " grad = np.zeros(theta.shape)\n",
+ " \n",
+ " # ====================== YOUR CODE HERE ======================\n",
+ " hyp = utils.sigmoid(X@theta)\n",
+ " reg = (lambda_/(2*m)) * (theta[1:].T @ theta[1:])\n",
+ " J = (-1/m) * (y.T @ np.log(hyp) + (1-y.T) @ np.log(1 - hyp)) + reg\n",
+ " #J=J+reg\n",
+ " grad = (1/m)*(hyp-y)@X\n",
+ " temp = theta\n",
+ " temp[0] = 0\n",
+ " grad[1:] = grad[1:] + (lambda_/m)*temp[1:]\n",
+ "\n",
+ " \n",
+ " # =============================================================\n",
+ " return J, grad"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "#### 1.3.3 Vectorizing regularized logistic regression\n",
+ "\n",
+ "After you have implemented vectorization for logistic regression, you will now\n",
+ "add regularization to the cost function. Recall that for regularized logistic\n",
+ "regression, the cost function is defined as\n",
+ "\n",
+ "$$ J(\\theta) = \\frac{1}{m} \\sum_{i=1}^m \\left[ -y^{(i)} \\log \\left(h_\\theta\\left(x^{(i)} \\right)\\right) - \\left( 1 - y^{(i)} \\right) \\log\\left(1 - h_\\theta \\left(x^{(i)} \\right) \\right) \\right] + \\frac{\\lambda}{2m} \\sum_{j=1}^n \\theta_j^2 $$\n",
+ "\n",
+ "Note that you should not be regularizing $\\theta_0$ which is used for the bias term.\n",
+ "Correspondingly, the partial derivative of regularized logistic regression cost for $\\theta_j$ is defined as\n",
+ "\n",
+ "$$\n",
+ "\\begin{align*}\n",
+ "& \\frac{\\partial J(\\theta)}{\\partial \\theta_0} = \\frac{1}{m} \\sum_{i=1}^m \\left( h_\\theta\\left( x^{(i)} \\right) - y^{(i)} \\right) x_j^{(i)} & \\text{for } j = 0 \\\\\n",
+ "& \\frac{\\partial J(\\theta)}{\\partial \\theta_0} = \\left( \\frac{1}{m} \\sum_{i=1}^m \\left( h_\\theta\\left( x^{(i)} \\right) - y^{(i)} \\right) x_j^{(i)} \\right) + \\frac{\\lambda}{m} \\theta_j & \\text{for } j \\ge 1\n",
+ "\\end{align*}\n",
+ "$$\n",
+ "\n",
+ "Now modify your code in lrCostFunction in the [**previous cell**](#lrCostFunction) to account for regularization. Once again, you should not put any loops into your code.\n",
+ "\n",
+ "
\n",
+ "**python/numpy Tip:** When implementing the vectorization for regularized logistic regression, you might often want to only sum and update certain elements of $\\theta$. In `numpy`, you can index into the matrices to access and update only certain elements. For example, A[:, 3:5]\n",
+ "= B[:, 1:3] will replaces the columns with index 3 to 5 of A with the columns with index 1 to 3 from B. To select columns (or rows) until the end of the matrix, you can leave the right hand side of the colon blank. For example, A[:, 2:] will only return elements from the $3^{rd}$ to last columns of $A$. If you leave the left hand size of the colon blank, you will select elements from the beginning of the matrix. For example, A[:, :2] selects the first two columns, and is equivalent to A[:, 0:2]. In addition, you can use negative indices to index arrays from the end. Thus, A[:, :-1] selects all columns of A except the last column, and A[:, -5:] selects the $5^{th}$ column from the end to the last column. Thus, you could use this together with the sum and power ($^{**}$) operations to compute the sum of only the elements you are interested in (e.g., `np.sum(z[1:]**2)`). In the starter code, `lrCostFunction`, we have also provided hints on yet another possible method computing the regularized gradient.\n",
+ "
\n",
+ "\n",
+ "Once you finished your implementation, you can call the function `lrCostFunction` to test your solution using the following cell:"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 6,
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "Cost : 2.534819\n",
+ "Expected cost: 2.534819\n",
+ "-----------------------\n",
+ "Gradients:\n",
+ " [0.146561, -0.548558, 0.724722, 1.398003]\n",
+ "Expected gradients:\n",
+ " [0.146561, -0.548558, 0.724722, 1.398003]\n"
+ ]
+ }
+ ],
+ "source": [
+ "J, grad = lrCostFunction(theta_t, X_t, y_t, lambda_t)\n",
+ "\n",
+ "print('Cost : {:.6f}'.format(J))\n",
+ "print('Expected cost: 2.534819')\n",
+ "print('-----------------------')\n",
+ "print('Gradients:')\n",
+ "print(' [{:.6f}, {:.6f}, {:.6f}, {:.6f}]'.format(*grad))\n",
+ "print('Expected gradients:')\n",
+ "print(' [0.146561, -0.548558, 0.724722, 1.398003]');"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "After completing a part of the exercise, you can submit your solutions for grading by first adding the function you modified to the submission object, and then sending your function to Coursera for grading. \n",
+ "\n",
+ "The submission script will prompt you for your login e-mail and submission token. You can obtain a submission token from the web page for the assignment. You are allowed to submit your solutions multiple times, and we will take only the highest score into consideration.\n",
+ "\n",
+ "*Execute the following cell to grade your solution to the first part of this exercise.*"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 7,
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "\n",
+ "Submitting Solutions | Programming Exercise multi-class-classification-and-neural-networks\n",
+ "\n",
+ "Login (email address): tanmayss2109@gmail.com\n",
+ "Token: fr8kwluMqRLcPibL\n",
+ " Part Name | Score | Feedback\n",
+ " --------- | ----- | --------\n",
+ " Regularized Logistic Regression | 30 / 30 | Nice work!\n",
+ " One-vs-All Classifier Training | 0 / 20 | \n",
+ " One-vs-All Classifier Prediction | 0 / 20 | \n",
+ " Neural Network Prediction Function | 0 / 30 | \n",
+ " --------------------------------\n",
+ " | 30 / 100 | \n",
+ "\n"
+ ]
+ }
+ ],
+ "source": [
+ "# appends the implemented function in part 1 to the grader object\n",
+ "grader[1] = lrCostFunction\n",
+ "\n",
+ "# send the added functions to coursera grader for getting a grade on this part\n",
+ "grader.grade()"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "\n",
+ "### 1.4 One-vs-all Classification\n",
+ "\n",
+ "In this part of the exercise, you will implement one-vs-all classification by training multiple regularized logistic regression classifiers, one for each of the $K$ classes in our dataset. In the handwritten digits dataset, $K = 10$, but your code should work for any value of $K$. \n",
+ "\n",
+ "You should now complete the code for the function `oneVsAll` below, to train one classifier for each class. In particular, your code should return all the classifier parameters in a matrix $\\theta \\in \\mathbb{R}^{K \\times (N +1)}$, where each row of $\\theta$ corresponds to the learned logistic regression parameters for one class. You can do this with a “for”-loop from $0$ to $K-1$, training each classifier independently.\n",
+ "\n",
+ "Note that the `y` argument to this function is a vector of labels from 0 to 9. When training the classifier for class $k \\in \\{0, ..., K-1\\}$, you will want a K-dimensional vector of labels $y$, where $y_j \\in 0, 1$ indicates whether the $j^{th}$ training instance belongs to class $k$ $(y_j = 1)$, or if it belongs to a different\n",
+ "class $(y_j = 0)$. You may find logical arrays helpful for this task. \n",
+ "\n",
+ "Furthermore, you will be using scipy's `optimize.minimize` for this exercise. \n",
+ ""
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 8,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "def GraDes(X,y,theta,alpha,num_iters,Lambda):\n",
+ " m=len(y)\n",
+ " J_his=[]\n",
+ " \n",
+ " for i in range(num_iters):\n",
+ " cost,grad = lrCostFunction(theta,X,y,Lambda)\n",
+ " theta = theta - (alpha*grad)\n",
+ " J_his.append(J)\n",
+ " return theta, J_history\n",
+ "\n",
+ "def oneVsAll(X, y, num_labels, lambda_):\n",
+ " \"\"\"\n",
+ " Trains num_labels logistic regression classifiers and returns\n",
+ " each of these classifiers in a matrix all_theta, where the i-th\n",
+ " row of all_theta corresponds to the classifier for label i.\n",
+ " \n",
+ " Parameters\n",
+ " ----------\n",
+ " X : array_like\n",
+ " The input dataset of shape (m x n). m is the number of \n",
+ " data points, and n is the number of features. Note that we \n",
+ " do not assume that the intercept term (or bias) is in X, however\n",
+ " we provide the code below to add the bias term to X. \n",
+ " \n",
+ " y : array_like\n",
+ " The data labels. A vector of shape (m, ).\n",
+ " \n",
+ " num_labels : int\n",
+ " Number of possible labels.\n",
+ " \n",
+ " lambda_ : float\n",
+ " The logistic regularization parameter.\n",
+ " \n",
+ " Returns\n",
+ " -------\n",
+ " all_theta : array_like\n",
+ " The trained parameters for logistic regression for each class.\n",
+ " This is a matrix of shape (K x n+1) where K is number of classes\n",
+ " (ie. `numlabels`) and n is number of features without the bias.\n",
+ " \n",
+ " Instructions\n",
+ " ------------\n",
+ " You should complete the following code to train `num_labels`\n",
+ " logistic regression classifiers with regularization parameter `lambda_`. \n",
+ " \n",
+ " Hint\n",
+ " ----\n",
+ " You can use y == c to obtain a vector of 1's and 0's that tell you\n",
+ " whether the ground truth is true/false for this class.\n",
+ " \n",
+ " Note\n",
+ " ----\n",
+ " For this assignment, we recommend using `scipy.optimize.minimize(method='CG')`\n",
+ " to optimize the cost function. It is okay to use a for-loop \n",
+ " (`for c in range(num_labels):`) to loop over the different classes.\n",
+ " \n",
+ " Example Code\n",
+ " ------------\n",
+ " \n",
+ " # Set Initial theta\n",
+ " initial_theta = np.zeros(n + 1)\n",
+ " \n",
+ " # Set options for minimize\n",
+ " options = {'maxiter': 50}\n",
+ " \n",
+ " # Run minimize to obtain the optimal theta. This function will \n",
+ " # return a class object where theta is in `res.x` and cost in `res.fun`\n",
+ " res = optimize.minimize(lrCostFunction, \n",
+ " initial_theta, \n",
+ " (X, (y == c), lambda_), \n",
+ " jac=True, \n",
+ " method='TNC',\n",
+ " options=options) \n",
+ " \"\"\"\n",
+ " # Some useful variables\n",
+ " m, n = X.shape\n",
+ " \n",
+ " # You need to return the following variables correctly \n",
+ " all_theta = np.zeros((num_labels, n + 1))\n",
+ "\n",
+ " # Add ones to the X data matrix\n",
+ " X = np.concatenate([np.ones((m, 1)), X], axis=1)\n",
+ "\n",
+ " # ====================== YOUR CODE HERE ======================\n",
+ " Options = {'maxiter': 50}\n",
+ " for c in range(num_labels):\n",
+ " initial_theta = np.zeros((n+1,1))\n",
+ " res = opt.minimize(lrCostFunction, \n",
+ " initial_theta, \n",
+ " (X, (y == c),lambda_), \n",
+ " jac=True, \n",
+ " method='CG',\n",
+ " options=Options) \n",
+ " all_theta[c,:] = res[\"x\"]\n",
+ " \n",
+ "\n",
+ "\n",
+ " # ============================================================\n",
+ " return all_theta"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "After you have completed the code for `oneVsAll`, the following cell will use your implementation to train a multi-class classifier. "
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 9,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "lambda_ = 0.1\n",
+ "all_theta = oneVsAll(X, y, num_labels, lambda_)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "*You should now submit your solutions.*"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 10,
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "\n",
+ "Submitting Solutions | Programming Exercise multi-class-classification-and-neural-networks\n",
+ "\n",
+ "Use token from last successful submission (tanmayss2109@gmail.com)? (Y/n): Y\n",
+ " Part Name | Score | Feedback\n",
+ " --------- | ----- | --------\n",
+ " Regularized Logistic Regression | 30 / 30 | Nice work!\n",
+ " One-vs-All Classifier Training | 20 / 20 | Nice work!\n",
+ " One-vs-All Classifier Prediction | 0 / 20 | \n",
+ " Neural Network Prediction Function | 0 / 30 | \n",
+ " --------------------------------\n",
+ " | 50 / 100 | \n",
+ "\n"
+ ]
+ }
+ ],
+ "source": [
+ "grader[2] = oneVsAll\n",
+ "grader.grade()"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "\n",
+ "#### 1.4.1 One-vs-all Prediction\n",
+ "\n",
+ "After training your one-vs-all classifier, you can now use it to predict the digit contained in a given image. For each input, you should compute the “probability” that it belongs to each class using the trained logistic regression classifiers. Your one-vs-all prediction function will pick the class for which the corresponding logistic regression classifier outputs the highest probability and return the class label (0, 1, ..., K-1) as the prediction for the input example. You should now complete the code in the function `predictOneVsAll` to use the one-vs-all classifier for making predictions. \n",
+ ""
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 11,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "def predictOneVsAll(all_theta, X):\n",
+ " \"\"\"\n",
+ " Return a vector of predictions for each example in the matrix X. \n",
+ " Note that X contains the examples in rows. all_theta is a matrix where\n",
+ " the i-th row is a trained logistic regression theta vector for the \n",
+ " i-th class. You should set p to a vector of values from 0..K-1 \n",
+ " (e.g., p = [0, 2, 0, 1] predicts classes 0, 2, 0, 1 for 4 examples) .\n",
+ " \n",
+ " Parameters\n",
+ " ----------\n",
+ " all_theta : array_like\n",
+ " The trained parameters for logistic regression for each class.\n",
+ " This is a matrix of shape (K x n+1) where K is number of classes\n",
+ " and n is number of features without the bias.\n",
+ " \n",
+ " X : array_like\n",
+ " Data points to predict their labels. This is a matrix of shape \n",
+ " (m x n) where m is number of data points to predict, and n is number \n",
+ " of features without the bias term. Note we add the bias term for X in \n",
+ " this function. \n",
+ " \n",
+ " Returns\n",
+ " -------\n",
+ " p : array_like\n",
+ " The predictions for each data point in X. This is a vector of shape (m, ).\n",
+ " \n",
+ " Instructions\n",
+ " ------------\n",
+ " Complete the following code to make predictions using your learned logistic\n",
+ " regression parameters (one-vs-all). You should set p to a vector of predictions\n",
+ " (from 0 to num_labels-1).\n",
+ " \n",
+ " Hint\n",
+ " ----\n",
+ " This code can be done all vectorized using the numpy argmax function.\n",
+ " In particular, the argmax function returns the index of the max element,\n",
+ " for more information see '?np.argmax' or search online. If your examples\n",
+ " are in rows, then, you can use np.argmax(A, axis=1) to obtain the index \n",
+ " of the max for each row.\n",
+ " \"\"\"\n",
+ " m = X.shape[0];\n",
+ "\n",
+ " # You need to return the following variables correctly \n",
+ " p = np.zeros(m)\n",
+ "\n",
+ " # Add ones to the X data matrix\n",
+ " X = np.concatenate([np.ones((m, 1)), X], axis=1)\n",
+ "\n",
+ " # ====================== YOUR CODE HERE ======================\n",
+ " p = np.argmax(utils.sigmoid( np.dot(X,all_theta.T) ), axis=1) \n",
+ " # ============================================================\n",
+ " return p"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Once you are done, call your `predictOneVsAll` function using the learned value of $\\theta$. You should see that the training set accuracy is about 95.1% (i.e., it classifies 95.1% of the examples in the training set correctly)."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 12,
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "Training Set Accuracy: 95.12%\n"
+ ]
+ }
+ ],
+ "source": [
+ "pred = predictOneVsAll(all_theta, X)\n",
+ "print('Training Set Accuracy: {:.2f}%'.format(np.mean(pred == y) * 100))"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "*You should now submit your solutions.*"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 13,
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "\n",
+ "Submitting Solutions | Programming Exercise multi-class-classification-and-neural-networks\n",
+ "\n",
+ "Use token from last successful submission (tanmayss2109@gmail.com)? (Y/n): Y\n",
+ " Part Name | Score | Feedback\n",
+ " --------- | ----- | --------\n",
+ " Regularized Logistic Regression | 30 / 30 | Nice work!\n",
+ " One-vs-All Classifier Training | 20 / 20 | Nice work!\n",
+ " One-vs-All Classifier Prediction | 20 / 20 | Nice work!\n",
+ " Neural Network Prediction Function | 0 / 30 | \n",
+ " --------------------------------\n",
+ " | 70 / 100 | \n",
+ "\n"
+ ]
+ }
+ ],
+ "source": [
+ "grader[3] = predictOneVsAll\n",
+ "grader.grade()"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "## 2 Neural Networks\n",
+ "\n",
+ "In the previous part of this exercise, you implemented multi-class logistic regression to recognize handwritten digits. However, logistic regression cannot form more complex hypotheses as it is only a linear classifier (You could add more features - such as polynomial features - to logistic regression, but that can be very expensive to train).\n",
+ "\n",
+ "In this part of the exercise, you will implement a neural network to recognize handwritten digits using the same training set as before. The neural network will be able to represent complex models that form non-linear hypotheses. For this week, you will be using parameters from a neural network that we have already trained. Your goal is to implement the feedforward propagation algorithm to use our weights for prediction. In next week’s exercise, you will write the backpropagation algorithm for learning the neural network parameters. \n",
+ "\n",
+ "We start by first reloading and visualizing the dataset which contains the MNIST handwritten digits (this is the same as we did in the first part of this exercise, we reload it here to ensure the variables have not been modified). "
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 14,
+ "metadata": {},
+ "outputs": [
+ {
+ "data": {
+ "image/png": "iVBORw0KGgoAAAANSUhEUgAAAjoAAAItCAYAAAAwm9DjAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjMsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+AADFEAAAgAElEQVR4nOydZ5wUVbf1l+99QDAHzAgmVBQDKghixJwzJjAHDBhQMStizphQzIhgxBxARR8xJxTFrChgzgEQ8Ib3w71r9yqnmmZmurp62vX/wv7tYXrOqXPqdNWOc/zP//wPjDHGGGNqkf+X9wCMMcYYY7LCDzrGGGOMqVn8oGOMMcaYmsUPOsYYY4ypWfygY4wxxpia5V+z+uHMmTNrMiWrefPmcwC1P7+//vqrJufXrFmzOQBg+vTpuczvP/7jPwAA//pX4fb5r//6r5D/8z//s1Gf36JFi3/E+nl+TZNquf/mmGOO0DX2nlN8/zVtOD/FFh1jjDHG1CyztOhUkv/3/wrPXPqkTrTeD+VqrwGk81C5qYy/GJxL2joBTX9+ROfHt0gA+OSTTwAAw4YNC12XLl1C3myzzer8frVfC73/SNo9V63otU6bi/Lf//3fAKp/TvUh7V4sdn8SXoemgN5/b731FgDgxx9/DN2mm24aci2ta1Om2PdfHvvOFh1jjDHG1Cy5WnT0KW/mzJkhP/300wCA++67L3TXXHNNyC1atKjA6BoO5/Xrr7+G7uGHHw55iy22AAAsscQSoav2tyt9o5o+fToAYMqUKan/d+655wYAzDnnnKHL+4m+PtAioOMcOXJkyL179wZQmCdQWFP9/Wp/s9Tx6V7lvTj//POHrnnz5qm/Vy38+eefIU+dOnWW/3e++eYDAMw111yh++uvv7IZWAbovUimTZsWMuPFisWt8Pd1/1aj9VFj4N55552Q99hjDwDAoosuGrpNNtkk5FKWrGpErZBcH437q/YzU+H11/HPmDEjZO67Ss7JFh1jjDHG1Cx+0DHGGGNMzZKL6yrNtNWnT5+Q6eZRc9fkyZNDvuKKKwAAHTp0CF01mfY4v6+++ip0xx13XMjDhw8HACy99NKhU9ddtaAm7HHjxoV89dVXA0i645Rtt90WQCEoFwC23nrrkBdccEEA1bVm6g74448/ABTmCRT2HAC0atUKQNK12r59+5DLmeqaJXr9e/ToEfIzzzwDADjiiCNCd/7554dM03PeLg5ds5tvvjlk3mvqDtC57rPPPgCS89Ngcj2XqgWdC8/CJ554InQq8+fffvtt6mctsMACAICTTz45dLr+aa6xSsK5qrvqsMMOC3nSpEkAgM0337zO7wD578vZRV1sP//8c8hMdlh55ZVDR3crUP3z47w+/vjj0A0YMCDkoUOHAki6JrOeky06xhhjjKlZ/KBjjDHGmJolF9cVTaN33nln6B588MGQaTrWrJ033ngjZJpcb7/99tAttNBCIeftEqEZ9bHHHgtd27ZtQ15ppZUAVKeJXK+d1ok57bTTQv7pp58AFFxQQDJrhev6wAMPhI7uLAC49NJLAQCLLbZY6PIwxxbLBKMbpH///qFbb731Qr788ssBFNYRqM61LIXOv2XLlnXk2267LXRap2TnnXcGkL+LTvfMMsssE/I888wDAOjYsWPq740YMQJA0vVKdzIAbLnllgDyz0RSd8zYsWND3nfffQEk3d1du3YNmWul7o4ffvghZN7Xl1xySejUDaTZTJU6S9WNMX78eAAFFyNQcOcAhWxVXgcg/7WqD2l1yJhpDAAHHXQQgOSZe8opp4TcVM4aHef7778fMjM8dZ9lPSdbdIwxxhhTs2Rq0SlWz+Cll14CkHxjZm0WAOjcuTMAoG/fvqEbMmRIyE8++SQA4Ouvvw6dWnTyhvNm0ByQfLuiXE1vHnx7/P3330OntWPWX3/9kJdbbjkAwE477RQ6voUBhbcvtWg99NBDIfONe9CgQWUZe30oVq2TAXJA4U1q3XXXTf05g8j1bbea1rIUnLdaZNQ6kHbfarBktdQp0Wu+1VZbhfz2228DKASNA0mLwSOPPAIAOPTQQ0OnZ1GnTp0AAIssskjo8niL1vl9+OGHITOwU60wWhOI69OsWbPQsZowANx7770AgNatW6f+fqX2slqstI7TWWedBSA551122aXOz9WimrcVv7HQSg4UakIVCyZvKiy88MIhax2uPM4PW3SMMcYYU7P4QccYY4wxNUvFXFfqmrrqqqsAABMnTgzdhhtuGPJNN90EAGjTpk3oRo0aFTKD0RZffPEyj7i8qLlcqUY3B8ekJsbTTz89ZA32pOtNzfnq5qFJetlllw1dv379QmZguQZIamBaFtcnLQBQ95SOjyZXda3pXkxzY5RqRFtNcKzqbtRgwbQWGNXirlL0+qqbhq5VHbPW5KKbRNdRA+v5WXmvn/79PffcM+RSLUb48/feey90p556ash0U9IFBADzzjtvyJVy0+nfGTx4cMiPPvoogOSYjz322JDZmqSpu6t0/hqMzDNYW3Q0JXjfffnll6Er5RrPGlt0jDHGGFOz+EHHGGOMMTVLpq4rNa2qG4d1OFiKHEiWI6fL49NPPw2dZgCxzoVmWuVtZk5Dswq++eabkBlhn0e9imLw+mln+FVXXbXOz4HSnZ75f7Ws/B133BEyS7trHSU1TWfpuvr+++9Dd+6554asa8X6TDp/NTOnucHSxqyl9PX3896raddCs+3S5pd3W4A0dM10T9JN/uOPP4buyiuvDJkZnOr6prscKLhm874ni8H9o/NX1xxrkp100kmhY6YjUMg6Y3YZUDl3lX4PaCYYwxmAQuuDI488MnSaAce1Ltbio6mga6ZuZM5rr732qviYygH3p36/65mXR/0tW3SMMcYYU7P4QccYY4wxNUvFXFdqZtx9990BJF0baaYtLVE+YcKEkFnivJhroFpQM+3nn38e8gcffAAg6RrJG7op9JqWio5Xc3GarJksWs6dBSNZ2A3I1l2lPPvssyG/9tprIavrbOONNwaQnJNel19++QUA8Pjjj4dO3UD8u9o2Yo011gi5kl170+C9UqwgGX+uBfc22mijOj/PAz1HtIjhBRdcEDJbO7CLN5C8zhy/uo6bUgsBFpTTvayufRYEPPDAA0On7QToxqrkOvL6Tps2LXTXXHNNyJqVc+uttwJIZh1pBhnR9dOzhvOq1nXkHn7xxRdDpwUDedaou7EpwevO7GggGbrB+3LJJZcMnVtAGGOMMcY0kLJbdPSNS2V9uk570tY3qqeeegoAcP/994duhRVWCLmaLCGzQgMkGWAHFN7uq8kKxXYVbGgJJIM51aLBt69dd901dKuttlrIDPKcMmVK6J555pmQWaekmPUoizcxWmdoTQKSwXL77bdfnbHoOFjbAwCuvvpqAMDzzz8fOrXSMMhQa+/cddddIa+11lp1xpX126feiww81hYdGiDIsRxyyCGh07nkGfip86BlFACuvfbakHnfqUVAx8/zQ1sM7L333iEzGF0b8VZTsCuD+Hv37h063T+sf6VWHK0zlMe5w3VTy+eYMWNC1rVi4sK7774bui+++KLOZy211FKh22CDDUI+/PDDASSbBud91ur5xrGoRVjPym222QZA0qJardapWaH3jJ4vedxLtugYY4wxpmbxg44xxhhjapayua7Sgs20xL8Gm40dOxZAsrupds895phjACTNXSeeeGLIyy+/PIDqMienUawEPa9FHqWw1V2kAdJ77LEHgGTQn7re0jp1ax0clt0HCt3np06dGjp187D+0W677dbAWdQf1lZh53sgWZtDXRucH10YAHDUUUeFzCC6Sy+9NHQ0NwPAc889ByAZ4Pzyyy+H3LFjxwbOouGoy4duONZTAZL1k9juQ9sOVAtqwme9GwDo3r17yFxLTXZYffXVQ2bgqrpDVllllZBZX+n6668PXTE3fKXQs4JuYp2fBtbfeOONAJJtW9Q1myfFajOpm5w1jfQ7QTutf/fddwCSbmi9v3iG3XDDDaFj2wggfzcQz1JNBkhrR6LteJo6uu56L1UKW3SMMcYYU7P4QccYY4wxNUujXFdqgqLLqk+fPqGjCR8AunbtWkev7iztnvvbb78BSEbVb7311nX+ft4myFKoaVbN5J988gmAfLLH1ITIejBAwYyqmQrqWkrrLv7111+H7oEHHgh5xIgRdf6WymuuuSaAZKZEFmup+5NuOpq9AWD99dcPWdeK2X7alkRdIwMHDgSQzATU36fLTk3P6hrKu2ZL2t9U3dprrw0gef9Vy72mLtSVVlop5GHDhoXM+iOaaZRWE0ld47p+3NdZz7k++0B/Ttewula1JhXdVOeff37o2HYHKLj88t57xbJyuK7aokXrUDEkQrMGtYXEqFGjAAD33Xdf6DSDsFoysIq5cJi1mre7tKGktZD5448/Qi5WvytLbNExxhhjTM1Sb4uOvvloTQTWbhg9enTo+vfvHzKDXYFCEKdWPtZg3bQnXn07o77ag5F1fDoXBsbm/UbVsmXLkLmuamXTt8Bu3bqFzHnp/LRy7r777lvn8/WNjcF2+vMsa+cAhcBMrY3y8ccfh6zWx1NOOQUA0K5du9DddtttIfONS+f05Zdf1vl9DYBU61WlLDq657ROByvP6vXXt1wGoVfj/aXXS88EtZ7x/xS7tkOHDgUAnHHGGaFTixsteVobqVzXolhzV/1bpeBYdM56LfjGrE0htU5NnueOWu7Voq2Vczl+rZysFpkddtgBQLIelVp8DjjgAADpjWqbArR+NCUrjpLWIFqtw5oEQ7I+E23RMcYYY0zN4gcdY4wxxtQss20vTSuLr+Z+NtK78sorQ8fmiECyZseQIUMAJE2vGhg6ceJEAMk6PCwLDhTMmFm7PhoKx8KgWyBZE4LXbccddwxdpRo9qulcg2npbmJ7A6AQdAskXZYM9nz11VdDx0aCQGEubD4IJE2TbNCq66tB0FnMn6b7vn37ho71mgCgZ8+eIdONqiX21XXx/vvvA0g25bvuuutCZjAr9zmQvNZ5BENq4PhHH30EIOna0vVZeumlASTXPG/SAhyLwXGr64K1WYBCkKsGSGrT2SzqHHHcOg5tREnX/uzsfc6P5ySQbOBJl4Hu7yzccPWBf1Pbrpx66qkh854CCm4srZOjdapYk0vnxNpsQOHcqabvhFLoWOkar6b7ryHo+DWZJQ9s0THGGGNMzeIHHWOMMcbULPXOutJMk7feeqvOz8eNGxcy66kAydL7jLynuwQAzjrrrJBff/11AElzrJo5O3ToAADYbLPNQpd3bQSFY9Hu3urG4LXQTB0t114p1LR4wgknAEjW1tEWD9rCgW6gyZMnh27OOecMmXUwNOuOtS0AYPDgwQCSZnxm7QHlMznr59B1oLWBWM8IKJTNBwpZR4MGDQqdrh9dHj/99FPotKYLs3o23HDDkuOqFA899FDIdOmo60rXb9ttt62j0/u+UqibitmAxWpw6F7mvrziiitCp65zulG0NpL+X+7vcrp4OBftuK7ums033xxAsi2O/n0933juqhtW9/JFF10EINmWpVoy6HTvq2tf3eQ8i7766qvQ6flPNENU9wqvpbqjq+n7gWPVNWFbI6DwvaiuuTzuv8aiY540aVLIbgFhjDHGGFNG6m3R0acxDQbm2+1ll10WOn2Kbt++fcgDBgwAAGy//faFgcjTK9+09C3rnXfeCXnllVcGUP3BZnp9tH7LmDFjABTqrQAFKwdQqL+S9VuYXj8GGGujyvXWWy9ktXj8+uuvAJIBgr169Qq5U6dOAJKNMrWOAi19+kab9VryWmptB7U47bTTTiGPHDkSQLLOjo6Pc1GLndYE4d/Q38l7r2qdFd6X+hasiQFsupo3Oj4G3mo9J72matHhm74GG7dq1Srko48+GkAy2FyDZLN8+99///1DVosSx6T7SOenFlU2sFTrjyYDbLLJJgDyr8BdCv0u2WqrrUJm/apXXnkldLp/GWz87rvvhk7r6LCmF4N6gfwtOroWrGmlCQL6/cAg9WqxwpUDnUse87JFxxhjjDE1ix90jDHGGFOzzDErk+bMmTPr/FBNcGp6Y4BcMROVmhZppitlWlVztJo5aYZsqAmsefPmcwDp8ysnOj8te82AbXXnaDlzzruh5mbO76+//mrU/PSas9EqUHAJqLlfS7tzXfT3NRiNDVp333330Gkweql1bdas2RwAMH369EbNT9dHxzq7wXKlzLENXb8WLVqUZf2KNXClm+See+4JnbosDz/8cADZ1Xbi+pWan46fTYO1eSODboFkzSbuxR49eoRO3VSrrLIKgOxqy6TNj3PRVjfa4oDtcDTAXdluu+1CpmtKXccazJo15br/isH7r1TNpGLfH5Qbuqbluv8U3Ws8/xk0DSRrKjFxQ91Z5XS9ze7911DSau4de+yxIbMpLZvPAsnA5caeNZyfYouOMcYYY2oWP+gYY4wxpmapt+tKUddSqXLVanqjSTGvTIBKua6UtGul8y+n6a5crislrV2AmoZLjVnXn64vbYuhWT+lyNp0njdZmM51/ejm0Y7majpvrOu0FA0xnXP8uo+Y/Qck9yLHz0xCIJltx8/IY37qbtF7nmtRrF6KzoXZnPpZlcwq8v1Xf3StWKdL6yBpuxK24WH2LZC9azULdM7qsqVez/wsXOOKLTrGGGOMqVn8oGOMMcaYmqVRrqumSh6uq0qSheuqnNAN0dCCejadN460TuCVLG7YGNO5jrmYzPHnVbCxIVllpTKMqqz4pO+/RsC1LpbdmXVoR6VcV4rONe3+LCd2XRljjDHmH0W9W0AY01hqqbR5UyTrN6osqSbLRmOppbmY2YdrnXdbikqS95lvi44xxhhjahY/6BhjjDGmZpllMLIxxhhjTFPGFh1jjDHG1CyzDEZ2+nU+sLKrpuSxmubskEf6YCX5p8zvzz//rMn5tWzZcg4AmDFjRk3Ob8455/xHzG/atGmZzk/PP60sz+rRWXkj5pprroo3fU6rPK9o4HK5KufXenkAxRYdY4wxxtQsVZlenlbQLI2mnp6pT/Eqf/jhhwCAt99+O3S77rpr6v9tivPOk2JFutIodW2r8drXpwidknf6p5l9Sp2P1Bfbn9VeXoD36IQJE0I3evTokPfaay8AwNxzzx26ap1LGmnrNn78+JB/++03AMDUqVNDt9FGG4XcrFkzAE1rznlji44xxhhjapZcLTrqd9Wn3GnTpgFIxqX89NNPIU+aNCnxLwCstdZaIbdr1y7kf/3rf6dYTU+/nPf06dNDd+edd4Z82mmnAQC23HLL0KlFp1ooZTHgtQeSFoM8CmVx/bVLsHbUVThu7a6eNle+WenvAIX5VdJKwvFp9+uPPvoo5DvuuCNk3ktHHXVU6FZfffU6n5nHPaPXWc+HtLYhen3TxtrUrVTFLL48F/V8nDlzZsjc17p/9ffnnHNOAMnrm3fxOh3fjz/+CADo169f6JZZZpmQS7UwqUbSxnrhhReGjh3LgcK+1fN/s802y3qINY0tOsYYY4ypWfygY4wxxpiaJRfXFc14X375Zeiee+65kB9++GEAwPvvvx+6Fi1ahNytWzcAwJFHHhm65ZdfPuRqCVIulj746aefAgDOPffc0KlrgWbac845J3RqZs7DJF9sLmnQJffaa6+Fbqmllgp56aWXBpC9uTzNHbr++uuH7uOPP079vVatWgEANt9889Bp4CPH3aFDh9B17Ngx5JVXXhkAsNBCC4UuizVLc6fpnrr88stD3nHHHUNeZ511AAA33XRT6C655JKQ6drIA3XnvvnmmyEzMLV169ahW3LJJUPmmgGF67LAAguErnnz5iFzLUq5vvJA11Rdq6+++mrII0eOBFA4RwDgvffeC/mTTz4BAKy77rqh0+vD83O//fYL3SKLLBJypc4Xnau63vr27QsgGYJw3nnn1fm9puSa1POb32u33HJL6HR//vDDDwCAxRZbLHR65ubtZpxdiiVGpJ1bWd+LtugYY4wxpmbxg44xxhhjapaKua7UdMesj4MPPjh0zz77bMh0GZx//vmh06wqmvTSMl2qgbSsAHWT7LnnngCAcePGhW6HHXYI+ayzzgKQNM3nYVpXc+mUKVNCfvzxxwEk5/T555+HTDPsY489Frr55psv5CFDhgAA1l577dBlbYbmXmnbtm3oNCupTZs2Ia+66qoAgKeffjp0v/76a8g0s+v6qmura9euAIBrr702dPp3s5grs60060rHr9eaY7344otDpzWbunTpAqCye47XUvcZXdgA8OSTTwIAvv7669AtvPDCIae5rtT0v9xyy4VMl8h2220XOv2/eWab6d4YOHBgyGeeeWbIHJ/uOc0A5HVhPS4geT7yvtRrqa7LStXp0r/DMwUAxo4dCwA4++yzQ1dNGWINQc8KrY9DWBsIAO655x4AQPv27bMfWJlIc0dphuu3334bMt2wdNEBwLbbbhvySiutBKC8e88WHWOMMcbULBWz6OibyvPPPw8gGWw8YsSIkFk/QN9S0j6rWp/s+aaiFps99tgjZAZhH3300aHTIFK+qeURLKlP5lqnQwO/H3jggTpj+vPPP0PmumkAua4VLT71qdzbEHR8HItaCRmUCwC9evUKmcHgX331VejUosMg67vvvjt0r7zySsjPPPMMAGDUqFGh6927d8jlsujo/PjGO2DAgDq6v0OrCccJAIsuumjItEhVEs5FA7gvuOCCkLluuo80WFctivw/WmdLq+y++OKLAJIWLQ12zbNmld4Tei30/OD41EqnNXN4LXX9f/7555B32203AMkAZrUEajB6uc8dnd8ff/wR8hVXXBHyIYccAiBZO6cpBR6X4pdffgEAfP/996HTxACeuy1btqzswBqB3pcPPvggAOCUU04JnXpfuG/Vyj18+PCQ+Xv6/dHYfWiLjjHGGGNqFj/oGGOMMaZmydR1pcFmGoDF+jBqjtXAwGp3TaWhbraJEycCAPr37x86dYPQNHf88ceHTs3Fec67mGn59ddfD5muJ9bDAYANNtggZAY5PvXUU6HTwHPWmalk0Cf31GqrrRa6NdZYo87PdVwaoKxmVtYE0gB6/X3K6u6qFMUCSVWfFoyqNYOqpfZUml7dMVp7RNeSqGtH589aPYceemjodK/utNNOAJLm9qyvCT9f53/QQQeFfOCBB4bMs6aYO4efoeP/4osvQqbrfL311gtdlu4qRdePLkSgcGYChbXU/6uutaaInulMrNEzUQNzGdpR7e46XR8GUAPAYYcdBgA4+eSTQ6fB1qxvNXjw4NCl7UUHIxtjjDHGzAZ+0DHGGGNMzVKxrCt1XTGb4PDDD0/9v9VSjr0UahrWDI+9994bQNLdo+XWTzrpJAD5t3VIIy1TCQAWXHDBkNdcc00AyY67+nN2Ytf577777nU+V7O6KkWai+nv0PSvZemZaQYU6hxppo+6Ri677DIASXdDpdyRxdxV6hqgGVldO9qOhTWP0tom5E2xsyFNX6zs/LBhwwAAL7/8cujuv//+kEu5hrJE51Fs/GlunLQWAU888UTo9KzluXPAAQekfn6W89a/o1m3Wj+J/6fU90CptjTV1OJDx8KzUmsX6f3ZvXt3AMnacdWCfmepO1RrHjGD8YgjjgidfpeMGTMGAHDjjTeGjt+ZQGEty7kPbdExxhhjTM3iBx1jjDHG1CwVc11Nnjw5ZJpetYR73qbF2UVNpFriWjuNv/TSSwAKhbmAZHEymsbVnaFm2Dyvhf5tLTG/2WabhXzbbbcBKLTyAJLj58+1E7W6+arFDaLo+MePHw8AGDRoUOi0nQWLr2mmQM+ePUNmNqEW/MrbDaJwr2qmoBZSZDZdv379Qrf44ouHXO3ZkGmuD+0UzblqkTq2/QCqZ36lzgE9i7Rg51VXXQWg4GIFgE6dOtX5eceOHUNXqTnrfaZ/U1vEsF1Hsflz3tOmTQudZrXy55oVqnu9Wr5rdEya9cdrscQSS4Qubcxp7jogm7OG66Zn+qWXXhryFltsETKLo6qbS0NXrr/+egDJ/de6deuQsxi/LTrGGGOMqVkqZtHRBpWLLLIIgOTToQazVuMbPylWB+L2228PmSXMtRy/Bnayvoq+3ejbP60feV8HfYvQwDI2ZWO9BCBZrp6tL0444YTQaVPFvOdF9I1IA+tokdGmpfPMM0/Iffv2BQAce+yxoVPrJOeX9zx1/fTtVi01RFtUMIj1tNNOC51afNguIu/5KWkW0dNPPz10tLICwMiRIwEUmgcCTatOC+eqVhzdi6xpolZGPYv49lxJyxXHrAkIbIUAAOuuu27IrFOl+0vPXQYun3jiiaEbPXp0nb951FFHhdynT5+Qs7DolGpnk5YkoBZxbQC81VZbAQDmnXfeWf6+JsDod6nu63KtMf8mz34AuPnmm0PWmmIMPNZkDv1+ZGLH0KFDQ5e1R8MWHWOMMcbULH7QMcYYY0zNkqnrSk1QrVq1CpkloNUdwgAlAFhsscUAJM3JeQeQ0XSnZf1vvfXWkLWEOlscvPDCC6HTmgmffvopgKS7SgOX2RpC3X15B7PqWFi6fN999w2dmki33357AElzcbW0uFCKzY8tADQYWa//kCFDABT2MZCsU6I1I6qRtOuv67PzzjsDKNyHQDLwkC6t+eefP3TV5MZiHScds8p0+WgdHQ2G5LzzPnOKwbNIu4+ra4BBnqznBCRDA/Jw0/FaagCuumbS9o+6MzTYmDVb9HzdZZddQmY7Bf1O2XrrrUNu165dyOV27ahrThMwVKZL59xzzw2d1hFifaNSCRzXXXddyFoTa5VVVgm53GetuqP0s995552QeS7q/hs1alTIPGs6dOhQ1rHNClt0jDHGGFOz+EHHGGOMMTVLxVxXmnV05plnAgCOPPLI0O266651ft61a9fQaU0XmvEqaVqmGfXtt98O3aOPPhryXHPNFTIzVLT7t5oTaTqlCwtItlOgaVnrfORRZ0f/pmZ4aM0HolkRzCbQcVajG0DHpCZ1tujYZJNNQqcZPHQZ6PrQ3QUAyy+/PIDqcdEBpa9/WuuB9ddfP3Rjx44NmaX7u3XrFro8XFeaNactLGFFgJAAACAASURBVE455RQAyTodmtVCN5aeKepGfuSRRwAk3VnVtJa81ppdo53Yb7jhBgAFFyuQzMqa3RYLWaNn5muvvRYya661adMmdJphRdeHnkPM1AIKda5WW2210KlrpVxZSXrmPfPMMwCSrUS09pSeJcxq1PXROmsMfdB7Svc6z1fNJNTvzyzWlWPR72TN6tMMTbpJ9efbbrttyOzOzuzrSmCLjjHGGGNqlrJZdEo14tInZz59Dx8+PHSDBw8OmW8nfDMGkjVZNt1008TfBCr3dqKBfDonDdJadtllASQrk2688cYhsxYB6+0Aybnom06e6DXVmjJ8o9Ancm0Q+OSTTwIAPvroo9BVqkFdMcsX376KvcHp/2WQrVb71MqyDALUYDttakdLj9YWqpTFo1gjyPpY1/hzHbMGc9K6o4GXedyLOj7di2+++SaA5H2Udv31nlWLHPet1h4qF7omaU19Z2ef8PpqMHhazSA2VwSS67fiiisCyMdKpX9zm222CVkDa1mZvG3btqHTYHlap7Q2l64lLa6aLKCVrxtzL6rl95VXXgm5R48eAIDffvstdBpMPHDgwJBZ0XmHHXYIXa9ever8Db2P9F674447ABSCroHk/s/irOFY1Ap65ZVXhqz7j/ednn9qcaLFp5JeClt0jDHGGFOz+EHHGGOMMTVLo1xXanpiAJjWayhlmtI6Cscdd1zIrOOhAWgazMTAMjVHVqo2RDEXgMqcF02wQLJE+UMPPQQgaW5VNxfrz6g7oFKuD/2baobdf//9Q/7kk08AACNGjAidunlY2p0BekD2rivutc8++yx02taBf1/rOZVy5+j+1X1N862un67vd999ByDZFiJr0toCaDC8BkaSUvPXvaD315gxYwAAO+64Y+r/zQNNdigV5MhrNWPGjNCpm+Obb74BUF5zetrf1L1KN72624r9fZ4F6gbSmkeXX345gKQ7ToOR6frIo+2O/h11PXXu3DnkCy+8EEDSNcfvFwC47777ACTX/N133w2ZySzqTqe7Diify07vf66VBrXr+PT+47mkySj8TgAKoR36+Y8//njIDPPQOmYajJ3lWupnqxtLW+QQPTO03QWTHPRaZY0tOsYYY4ypWfygY4wxxpiapVGuKzVXs2O1mvA1klxrUjBa/IMPPqjz+0DBNfXWW2+FTl0OeZbYV3OvmounTp0aMmvtvPHGG6FTMx3rOGgdFs3qKZXBliW6plp74uuvvw755JNPBgBssMEGoaM5HAAefPBBAMCGG26Y2Tj/Dl0D2lGe9VQAYIUVVgCQzH7bcsstQ9YMAaK1WdhxFyjUWWH2BJC8FmwnUck6JczgmThxYui0rYiuxXbbbQcAWGONNUKne5l7QF0rrC0EFOqTaFZJ3qR1ui52/3Dc2s7l9ddfD5kZMqU6UtcHXlN1V+ie4VnJztVAcv/oucN103NQu1fTpa9tAbTmDNtFHH300aHLo06XnjXaIofZYpqVqm50thPQrF11E9H1z1Y15UTdMeuss07IrAOkrknN0NKsMboWtY7OXnvtFTK/K4q1QNpnn30AAGeccUboSrWLyIL6ZHKmZWVW8ny0RccYY4wxNUujXsn0yZFvJxqA2qVLl5A1SJK/pwFMavGhrAGu+nS/zDLLAKhsczqOefXVVw+dBhvrG+GkSZMAJAPEtKIk3z70KV+vZbU0SNSncH1j4LrpGwWrsQIF65ZaDCqFWmnUysTaL9rIkW9WQNLixnnr/lLrDdG3b60pMd988wGo7DoywFIbFmrT2YcffjhkVu5Wi42+EfPtnkHVALDmmmuGfNVVVwFI7o+8Kwf37t07ZAazqpVK14Jr2bdv39BpYDXr55RzTnx7Vcuh7tWRI0cCSO5ZPTMV1pdp37596NT6OGHCBABJ64KetXru5ImuyaKLLhoy61PpmZpm8VfrbPfu3UPmNc4imaNYtX9ajGcHWq9YewcA7r333pCZzKHrpMkgtCTlYcWpD8UsNkzWqWTtLVt0jDHGGFOz+EHHGGOMMTXLHLMyGc2cOXO27UlpdSI0GPerr74KmSYrde3QHQUU6mBoiXSVG2tSbt68+RwA8Ndff9XbXlYsQFHNcGlN89JMi1mZ65o1a9bg+ek8pkyZErKa2TVImWiJcgYB0oUAJGsuNHbes5qffvbvv/8eMoNAGcgIJAOsNYieTQX1s9S0TjeHNrLTOhmN3Z+c359//tmo/Vmstg3r67BUPlBo1AkU5q0JAGy7AhTcIA2dZ8uWLecAgBkzZjRqI+j8dtlll5C///57AEC/fv1Cp8G699xzD4DkmaSuPZ4/DXUHzDnnnEXnV8zdxz2nY9LAZbZV0f/Dej9AMliXQcjqutOmn6wppcGy9bknOb9p06ZlcoBxDxdrZ5L1+TrXXHPNAdTv+68+cPy6F0rVoUoLbWjo/Pj9N3369Ez9Rbq/N9poo5DZ4HTAgAGhK2fgf4sWLep8mC06xhhjjKlZ/KBjjDHGmJqlbK6r+EAxQWlUeJppqpjpsbGmuVI0xnXVFGiM60pRc6rWfBg0aBCApDtKOyXTdK5ZA+XMYJnd+aWZhtXFpmPSDBctvU50L3PeDTX9l6IxrqtipJn+S5nOdU7Fano0hHK5rnROWseKNY9uueWW0Knrm60RevbsGTqtU9PYvTor15WS1slcdXomquuNa6Hj1PVhBqGuqWYV8vcauo5Zu67yJmvXVd5UynWl+0trGrHOnNZ20nY5jT1f7LoyxhhjzD8KP+gYY4wxpmYpu+uqKWDX1exRKuuhGFm7Hhszv2Jjr0/Uf9YlzLNwXVUT5XJdKfXZn2mUs+Da7LquSlGfOZVq4ZDF/Oy6appUynWle5JFdIFCCx1te6GFNO26MsYYY4ypB7bo1CDlsuhUK/+U+dmi0zQpl0WnWrFFp2lTKYuOooHxDLwvZ4KDYouOMcYYY/5R+EHHGGOMMTXLLF1XxhhjjDFNmX/N6oe17qOs9RiPWo8RqPX18/yaJpyfz8+myT9l/Wr9+0Gx68oYY4wxNcssLTomHxihrm5FuxiNMbVKqe7kTeX80+yihtTmAspb88j8L7boGGOMMaZmsUWnStA3gT/++AMA0KJFi9BpU8mm8nZjjDHFKFbNmQ12tWlutZ9/HN8HH3wQupNPPjnkKVOmhMymwYceemjodtttt5DZDLmpWnb4XVaqabCia8qms+Wcvy06xhhjjKlZ/KBjjDHGmJqlalxXpQK3qtFc2VjUnKemzU022QQAcPPNN4duzTXXDJmmvYag11n/Pq9vsXVICwyspTXRebNEuZpOS5lRi11X0pg1qwTVfv+lBaj+k0jbX6XcAcX2b7VcQ53Ta6+9FvIpp5wCADj77LNDt/HGG4fM1gF5z0Ov/zvvvAMAuOiii0L3zDPPpP5fcvzxx4f87LPPhnzxxRcDABZeeOHQVeP5UezM+/333wEAH330Uei+/PLLOr+ve1KberZv3x4AsOiii4ausWtti44xxhhjahY/6BhjjDGmZsnVdaXmLu1kmuYm0Kh7/l5TjUqnyY8uEgB48MEHQ6YZtJymO/7NyZMnh+6yyy4LedKkSQCAVq1ahW711VcPea211gqZbrSWLVs2akx5o6ZXdR3ec889AICNNtoodMsvv3zIaW4+ZlIAwMMPPwwAmDZtWuj22muvkDWbpFzmd+6lYp9XyvWY5uZI6zg8q79RDvSa6ph4Pug4GlqzhFRjnZZi7oAZM2aEfMcddwBIukbS1nLrrbcO3Z577hky918ec9ZzfOzYsSEfdthhIX/44YcAgCuuuCJ06trgvahZqWn7Juv56V4cNGgQgOQ5vvPOO4e81VZbhTz//PMDAM4///zQ6e9xrw8cODB0CyywQMjV8r03ffr0kG+55ZaQ77//fgDA+PHjQ0d3FlDY13/99Vfo5p577pDpprz99ttDN88884TckHW1RccYY4wxNcssm3pm1euDT3SffPJJ6BiABgBPPfXU/w5OntL33XffkM8880wASetDfZ5y8+7VwvmzXg4AbLfddiEfcMABiX+B+gUTpvW64t885phj4v/9+uuvITPwberUqaFjPQcAuPfee0PeYostABTeYgBgrrnmmu3xNZZy9brSN2a+JQPAIYccAgA49dRTQ3faaafV+T19I+nbt2/IDzzwQJ3f6dWrV8j6dpK2b2e3F5TeH9988w0A4MknnwydrsnPP/8c8tdffw0gacX68ccfQ3755ZcBAGuvvXbojj322JC7dOlSdOyzQ9r8OBcNujz33HND5l7bcccdQ7fffvuFrNbHUtYd/lz3d/Pmzev8v4YG8DakV1La/N98882QL7300pBp8VUrq74R8x7mOgLJa3XGGWcAaLiVriHnZ9qZp2cRrahAwVKjVn6F8959991Dp/ti8cUXB1C8Tk8pZnf99PrRuqb7qHPnzvqZIfNafPHFF6Hr0aNHyAziPeqoo0Kn90JjA5Mb0uuK11LPvP79+4d86623hrzyyisDSN6TTLABCt/bY8aMCd2IESNC5nMBv+cB4IQTTgi51LnjXlfGGGOM+UfhBx1jjDHG1CwVC0ZWM9+ECRMAAPvvv3/oWPYbAC688EIASdOjmm7btWsHAOjTp08mY82CNDPqgAEDQqfXh4Gr5QyWpLlPTaDq2qDJVd05DAoEksFybdu2BZAMBmyK6FzV9M+1WHXVVVP/L9fy448/Dt2QIUNCPueccwAAhx9+eOiyaNqngZ0MbLz++utDp+tTqiaQzo97gS44ILlXGIyurp9yuSs1qPutt94KmS6PoUOHhm706NEhp61VsetMNwJdHEAy8L9NmzYAkgHkei2zcM1yTw0fPjx0Rx99dMga2Dpq1KjEOIGky4QuHw3m7NevX8j77LMPAGCllVYKXdZ1Wrgm48aNC92dd94Zsu4v7gEG7QLACiusEDLvO70+gwcPDpkuY3VtZbFmes26d+9e5+el6hi1bt06dJrswPlpAG/ecMx65mgA+a677hrycsstByBZB0i/35ik8fnnn9f5fKCwV/T6NBZbdIwxxhhTs/hBxxhjjDE1S6auKzWHMysEAA4++GAAwCKLLBK6G264IWSalNWcpV1hH330UQDJ7q9quq3GmhhqunvkkUcAJGsPMNMMKJjJs6iXUKweAaPpdc2uvfbakNW0zGwANd2qSTONailBr2uiWWdaU4hZRVp2Xn+P10qzAjbccMOQ1SVLsp4z10LHqVkrWodjmWWWAZBca70X33vvPQDJsu2abcG1bGhWSxr8fXWHHXnkkSHTtajniLpudHyl4LiZfQYAN910U52/tdpqq4WuU6dOdcbaWPT6M9NEM1m0BYK6QdOyktTlx7NGXVN6XUvdq+VC9wfrAKk7Tc9EdXPwXJ933nlD17Nnz5CZlXTfffeFju48oNBaQV1fzBQFsjlXG+L60/XTOklE3bHVCM8RoOCuAgrXQltAaNbuCy+8AAB4/fXXQ6drxXtR3bVuAWGMMcYYUwQ/6BhjjDGmZim7DTPNxA8Axx13XMgsTqYZFJoBQVgYC0hm/dCMqSbYanFXAYVroNfilVdeCZnZApdccknoOnbsGHKWJcxLfab+XM29Tz/9dMgsLa+ZAttvv33IG2ywAYCkaVqLO9I1UKwgWJaou0C7w+v6HHjggQCSbj5dS7pRX3zxxdBpwUC6ibLOZFET/Oabbw4AeP/990PHwl1AsqAa/6/OScvx77HHHgCSmWiagcTrksX66Zh0/2222WYAkmXndXz1uVe4L9W0rhlcXL+lllpqtj+zIehcaa7v1q1b6I444oiQteDc7F53dT3qtUorjpgFOj8WrHzooYdSf77eeuuFrPdS2v9lwUAtaHnSSSeFfPrppwNIZkJuuummIesZkMf3Bvefut5effXVkFdccUUA5XXdZIGebzo+rrUWqdTzha5XPV+vuuqqkLfddlsA5Q13sEXHGGOMMTVL2S06+havb0n//ve/Q+aTtgYT6tMh30q17YO+hbBEv1p0sn57rg98Y/jll19Ct+WWW4bMdg8atFqsmWGl0SdnHbOW83788ccBABMnTgwdax8BwEUXXQQguRfUYrfbbrsBSLa4yLppJNdEG21qnRZtGsg6I9q0VNfkrrvuSnwmkGyg2JAGkw1B9/xOO+0EIBlArW9MacH6GgCpDRR/+OEHAMCSSy4Zug4dOoRcqf2Z9nd0TRp6z3N9NIBVrSS0TmptnSzmrJ9J6wvfZoFk7Z5Swda6F4laCbTdzldffQUgef5mQZoVVNdMx9y1a9eQeRYUsxiknQ8LLrhgyAxm7t27d+i+/fbbkLU+S6W+N9KatT7//POh4z0HAHvvvTeA5Jyq0aKj6Pjmm28+AMm2Ddqahq1NtAWG1v9aY401AJR3nWzRMcYYY0zN4gcdY4wxxtQsZXNd0TSn5litDcOy8UCh5ojWsdDAseuuuw4AMHny5NBpnZNll122zt/KGzXDsnuwmqHVDM52FuquqSbXG1FzpAZmsg6S/vzkk08OmaZJ7YitJdpZH0Vromj3+iyDzNllGAAee+yxkJdYYomQWTNIXXPaooStMRZbbLHQsS1JXvD+UxdcsbYT3KtqLtfrwn2ZRduKxtLQ/aD32vjx4wEAAwcODJ26TljLRvdhueav54R28qbrSgOk1Z2mrse0OkbaLoCtO2677bbQac0T7tus3SF6zej61nVgKxkgGezekHHp+ckgfK0tpPV79KziNcyyrQeQXHe6rDScQxM7GLKR1tajWtHrxzXWYGq61oFC/TJ1bQ0bNixkJilpgHljsUXHGGOMMTWLH3SMMcYYU7OUzXVF05WaJpdeeumQ1TXFOiXa/VlrPvz2228Akq4fLQHOv5V3JLqaI9V0yjoQ7777bug0w4dukmo3RypprjU1zWqLiI022qjO/2UdFKDgutM9oaZr1pEo9ndnFx0fP0fdVZp1xEwUoJAVp7VH9LOmTJkCILnXtV0Bs8m0NkjWNOSeKNaihJ+hrrliLrFqplgm4z333AMAmDRpUui0uzfnmnWmld4zrG2k7jT9v9rOgeuiGYQjRowImRlWmrWia6btICoF6wBpPSClnJmK/Cyt3fXGG2+ErLWYmMWXZaYnkKzTxbNCv/M0a5WtH5rS94NS6lqy3QxbdQDA/fffHzK/F7Seku6bhqyVLTrGGGOMqVnKXkdH30JYjwQAvvvuu5D5FK3VktmoEyi88etblj6dV8uTbrEqu6xyqtWENVi1GgOP09C3rJ9++ilkNmhjPSMg+ZaYFiypTftY8fXqq68Ondb50Iq+jblW+uTPtVLLkQaDajAnLUo6p4cffjhkWhy1cq0G3mkDzWqE66JN9dS6xeumFlltutcU96+eP7wvtbYTLSqVRM+PE088EUCyTpC+5WpleO7Lzp07h07PSga2ap0WNuIF6tcAtVzQeqXfCdo0WC0uav2dFWn3t6LniCa2ZFnnSscxYcKEkPv06RMy68esv/76odt9990zG1M1oNec565aITVwnNeinEkptugYY4wxpmbxg44xxhhjapZMXVdqwj/nnHMKf/T/TFLatPOGG24ImWbYddddN3TVYi7XoE01h6sbji6Z7t27h65YOfNqRs2Nalpk4JwGkzPAHCiULmfQLpAMvKPJernllgsdG/UB2aw1rzkbjv5dVtJM21pnZ8CAAQCSa67l2ulardZ15vy0BLu6M2h+V9dIU0TdCOq6GDduHIBkWxN1jVdq3fTvsGaKBmiquynt/NAWEWrm5/oyqBVI1rQaOXIkgKTrOQt0fgwm3WGHHUKnbUfYiBMorJvWXtNzl/NTN9/3338fMl12WptFm3pmie45dXd/+umnIXMu2ghZ3eRcy3LsQ34fz+6ZWqz2T30aTfMzdE/qd8Fhhx0GINnUdJ111gmZIQHlrDNni44xxhhjahY/6BhjjDGmZim760pRM5eaxFgCWks8qxuDEepqusq7BD3HrxlfWvtg1113DZktEnTM1erGmBU6fm1hQZOw1jlgJhYAzDvvvACSbRO05gczeLQsu9ZsyXKt6/PZumfTMv30s5qSa5Lj1vXRMTNDjl2U//7zaofrpll1zOQECueKum60pk0ec02rg6QtAGb1O0B6V3ANHdAWA+xqftBBB4Uuy0wkoLDn1F2o7u5bbrklZGbdsIs1kHTtcP30O0MzCJntpOtfbH5ZrrXW4UpDXXd0JwKFeen5ouPnmItlH2udoD333BNAadcdP//nn39OHdMuu+wCoPicdKzM4NSs42uuuSbk5557DgCw4447hk6/S5kNWc4QBlt0jDHGGFOzZGrRKfYUfeONNwIAXnzxxdBpHR0GBlZLvRygMBd9WuaTKZB8+2Cz0jZt2tT5faBpvR0TDUxjEK82pWOAJwA88sgjAJKNBLfYYouQaR3St+i8LXZp6JrpXLgHfvnll9A1ldo5QMG6prVL9O2JlgQN9myKDBo0KGQNfDz11FMBFJonAtV5fzZ0HLyX9O1bGygy2UMDtHV/Z1kRmtZeINmoWe8fWoyfffbZ0GlgK8+iYm/81OucjjnmmJA1iLvcc9XP08Drzz77LGS97wgbzQKFIGY9c3UvzDfffACK1xvr0qVLyLQOldpL3P/a3Fct7vxMrZBOzwwAvPTSSyHffffdAIDRo0eHTteC1ju1Mun6Z5GMYouOMcYYY2oWP+gYY4wxpmapmOvqww8/DJl1ZrR2QjXWzFFo+tMAQQZoAcBFF10UMk3i6rqqJbg+6tro2rVryGnNLNV0Srka3VWKBsOvsMIKIdP18/jjj4dOm3pWI2oGp2tV70l1DdSnZka1oPNjTRU27wSSdZAY7KqNAqvJTV4u9Jqom4Nz1QDeZZddtiJj0nteXWvnnXdeyGzgPHz48NClBRt369YtdOqO6tixI4BknawOHTqkjqHcFEvgYO0toODyKdZih/dlsRYIdB/pmaSfpTW9+Bn6narn2t/R++CFF14ImYHDmjTy/vvvh8y2OEChhY4mq+y3334hsx1SsWSOLLBFxxhjjDE1ix90jDHGGFOzZOq6UtOUtoBg19bevXuHrppq5qRB06GaE9lxGEh2p2VmQZq7ppao9fnpPtQWATQZq7k56zokjUXHN3XqVABJc7OarOeZZx4A6R2hqxUdK1vLfPDBB6HbbbfdQl5llVUAVKeLvJzo/FZaaaWQ6VrWTJkePXpUbmD/h95fun6staPuNtZmAQrtStRdpb/Pmjul6mBljc5Pvzfo0tIzU91c2rojjVKuf9XP7ncpP1Nr2+iYmUk1adKk0HXq1CnkbbbZJmTWlNMMLf2sPO67pnOSGWOMMcbUEz/oGGOMMaZmKbvrSs2F2hH5m2++Cfm2224DkIy6bypmZDU3qjlOC2GllXM3TQ81+6oZlu0E6AL5+/+tRvT+4rh79eoVuptvvjlkZhDq/VmN89OzRufHDB0t7rnJJpuEnJaJUovo+aMZkv379wcA/PDDD6n/Nw/079PNpOEMaa0Hio25Gte1WLsOkvf9xfHpPmF24t/l+n4mkP+a2KJjjDHGmJpljlk9yc+cObNRj/n62V9++WXIrVu3/t8/nlMAZ/PmzecAgL/++qsmTS7NmjWbAwBmzJhRk/Obc845c10/WgQaEvQ3O3D9spof7zsGJQPJt/u2bdsm/h9Q3jf+rOf37bffAkgGW2udGAarZmXF4Pwae35mBS0lDQ3W/aecn9W6fo2F61fr3w+KLTrGGGOMqVn8oGOMMcaYmmWWritjjDHGmKaMLTrGGGOMqVlmmV5e68FYtR5MV+vBZt6fTZOsg5Hz5p8SzOr5NU3+KeeLYouOMcYYY2qWTHtdmVlTrJdQ3sWjjGnKpJWtKFXKQn/O+8/xi/mi52Ot99Uz2WKLjjHGGGNqFlt0KkTaG+Uvv/wSspY7Z/foaoVvWjonlflztUylvYXlXRZc0evPuWRVENCUH10/tp7RPTdz5syQp02bBqB4QcT55psPANC8efPUnzcVdH56fdLmUupezYM///wzZF2LvArNNgYdMwuONnYeWuSxKZ5PuieVLL4XbNExxhhjTM3iBx1jjDHG1Cx2XVUINQe/8sorAIAePXqETrufDxs2DADQsWPH1N+vFGnuKKDQif7XX38NHd0BAPDpp58CAJZZZpnQzT///CE3a9aszs+z6qs0K/Rvfv755yGzV9IKK6wQOu1enuaaywO9Tlmb8yv5t2YXHccbb7wR8ogRIwAAM2bMCN37778f8rPPPgugsA+B5PwOOeQQAMCFF14Yump3Y6Xdq+r60V6DRF0HSyyxRMjsBQZUbq46lq+++goAcNxxx4XupJNOCnnttdcGUF2u7zTS3KlAYa9OmjQp9f+Wgq6vDTfcMHT6/VGN+1Ph/vzss89Cp/fi0ksvDaC856stOsYYY4ypWfygY4wxxpiaJVfXVbGsnbSsFzVT0jSn7pRqMacrOr4nnngi5N69ewMApkyZErrffvst5A8++AAAsNZaa4UuD3eOjunqq68OedSoUQCA7777LnRTp04N+ccffwQALL744qGba665QuZaHnnkkaHjNQEqZzrXuZ5//vkh33PPPQCA9u3bh26NNdYIuV+/fgCA5ZZbLvVzs3RpFXMhUdb7pJhpn/83LdNM/4b+PK2mSdo9+ffPyhId02mnnRbyv//9bwBJc/jyyy8f8kUXXQQguX+55gBwxx13AAC23Xbb0G2++eYhV4vLpJhr5KWXXgIADBw4MHR0JwOF+0v36brrrhvyZZddFjKvYdbnj67lxIkTAQBPPvlk6HQt1llnnUzH0hh073/yySchDx48OOSbb74ZAPDHH3+ETvcqUXeprhVdkrrn+/fvH3I1uq50ffn9sPfee4fu8MMPD/nAAw8EYNeVMcYYY8xskalFp1gdBz6pTZ8+PXS///57yKwvowGGI0eODPmjjz4CABxzzDGh22effep80Y7aLwAAIABJREFUfl5w3i+//HLojjjiiJA5P70mSy65ZMirrbZa4nMqif5NDWZ8+OGHQ+b4evbsGTp9y+3cuTMAoEWLFqHT9eXbDS0jQNLic+ihh4ZcqbeTnXbaKeShQ4cCAHbZZZfQjR8/PmQGAZ5xxhmh02sx99xzA8immmuxoO2xY8cCAG666abQPf744yHrPcFrvfXWW4dOg61pHejWrVvoGCAIAAsuuCAAoHXr1qHTmh6VQufPPQcA3bt3B5C0iKr1jdYd/X2tXXX22WcDAF544YXQbbHFFuUadqPhHvj5559Dd+aZZ4b82GOPAQC22mqr0J1zzjkh86yZPHly6PR8YjA3AKy++uoAgA4dOpRl7MXQ/cn1WXbZZUP36quvhnzAAQdkOpaGQIuFWsE1gJoB8EBhXbbccsvQrbzyyiFzXy688MKhu/3220O+/PLLASQtQtWOftfxfNXvdz1/ssAWHWOMMcbULH7QMcYYY0zNUjbXFc1tag7WANV33nkn5DFjxgAAJkyYELp33303ZNZRUNOsBmbRTH7fffeFbueddw65ZcuWdcaVNRpMRjdHr169QqdzoZlTA7Q0MJKBueedd17oNt5445CznJOakLW2xlVXXRUyTds6P5bN18/Q+WmdHQZmsx4EkKwDUSn0Om6wwQYhMzBzqaWWCp3W9GDgat++fUOne5EuIQ22Tgs2bAh6TX/66aeQjzrqKABJc7DeBwrN6zfccEPo0vbUJZdckvpz1he69tprQ0d3EZB0Y2XpftW9etZZZ4XMa1SsNgl/7/nnnw+dBiOz5lPedawUncsPP/wAILm/WPsJAO666y4ASXee7hvK7733Xug+/PDDkNVNcssttwDIfv66losuuigAoF27dqFTl1A1klYb5qmnngpZ3dw8SzTpQufPe0brQH399dchM0xAz89qRPec1il75JFHABRc4EAyWSCL0BNbdIwxxhhTs/hBxxhjjDE1S71tX2qK1qypc889F0Ay00OzdtT0xk7CrVq1Cl3btm1DZrbL9ttvHzqtQ0OTubpL8ijRruZkNf0yG4wuOCCZ1bHIIosASGYiqWuL7gd1Dd19990hazZMlhlm+tlqBmcGRrE6D0RdK5r1cd111wEolNoHkmtdqfXTv6MtKpjBd8UVV4Ru9913r/NzvSYPPPBAyMzaYcYKUL46LHqddf/vueeeAJKZNpqdklZzivWagKRrgHVMtG0Ca7MABZfzySefHDpm+gDJ+7pSGZBpdX70b2u7EtYxUdfb999/HzLreGyzzTahyyOTU89a3TO33norgMI6AYW2MQCw4oor1vksdR0zg1LdKZpVpvueZ1XWtYP0XqSbV9dUqcaaadwf6rpS15PW5OL8Srl4H3300ZDvv//+kNk6R2uPVTvPPfdcyGyBxOxdIJnBadeVMcYYY0w98IOOMcYYY2qWeruu1MSoUd8sbqTuGDXd77DDDiEzm6FTp06h0wwXFjRT15iWM6fLSEuBp2VlZUFa9gMAnHjiiSGz0Ngqq6wSOjXTsSuyRqJrcTLq1V3FIlFAMhuEmTVZu3vUtLrAAgsASJqz2dEcKLgxWBgKSLrmmNWhmXJprodKon+TBQF1z6kbltec2SFAsvga3UdasK5c5lgdp+7FPn361Pm/pToiq7lYrz8LBuqaDhgwIGS2E9CCbnl39067Llpkjm0fAODpp58GkCxSpiXoWehN55RH2we95/Rcve222wAks+JWWmmlkHluvvXWW6HTDE6GEag7eccddwxZs4HymDfXUsMZ1E3H/VmseGYe8P6hWwlIjo+hHQCw2GKLAUie4wznAArFBU844YTQ6VlD16Lef3kXyVU4bz0z1Q1Ol/tee+1VRwfYdWWMMcYYUy8alYif9kbZo0eP0GmApwbj0hJUrCkg5SFDhoROa1507doVALDffvuFLus3Dz6lam0gfaPSBnScn9Zm0ZoXLMGubR/WW2+9kBnE1qZNm9DpG7UGObPWRCXrXDDg/MEHHwydvj0zIE8bDWoDNwYep7UFyQvdPwzmVIujlsWn9VKbH3JPAoVgV93z5ZpfsbfYtGBGnVNaOxa1fKp1kU0ttbaMNoVcf/31AST3v9ZBSqsJUkn4dk3LBwA89NBDIXOsq666auhOP/30kHluVUvzTiB5r7DFiNZm0jpctNRobSe1qN94440AklaCYg2U84D7Wi0eGuzP80WDrvNoQaLw+mkrB7Wy3nnnnSGz3YaOX9s50BKnTZ/pBQAKDU7zXqdS6J5V6xPXlwkUlcAWHWOMMcbULH7QMcYYY0zNUrYa0nTXaJdjNa2raVRdGkSDIVnTQl0DGjjIwK6sA5jS0ABVrcOh82YLgFNOOSV0DEADSptZGQyoZlANVsujNokGQ/L6q2ly0003DXnQoEEAgI8//jh0GljHEuhZd0SuD2kuFtYQAZLBqqyJc+mll4ZOA495XbJep7Qx6z1XrA7JF198AaCwTkChNgtQMIlriXY1w7P1hbYIqSYzOu8v7eita8EWNJoAwNoyALD//vvX+Z080LVkggZQqO9z2mmnhY4JAkDBJa61V9Zcc82Qeb5U05opnLeeqZrMQDdrWr2gvOBeUXf1xRdfHLLWCWMYg7rDda9xfTTZh7VngELNIw18rkb0e27w4MEhs85YJc8PW3SMMcYYU7P4QccYY4wxNUvZu5fXxwSlpnXNZurXrx+AZCaIZh0xQylr07KOj3UAmFEDJF0HWoeCpfcXX3zx0DXENDdp0qSQNYNNzdRZotdXTbLMxlHXoZrWWX/ktddeC52WQ8/bJUCKdVfv378/gGQ7E12/PfbYAwDQpUuX0Kk7No/58f4r5m5U18wFF1wAINmWhZkcAHDwwQcDSNYxUZl/o5gLNu8S/bwW6i5WN93w4cMBJDMhtcUF231oPZlK1WnRa5fmzgeAd999F0CyTk7Pnj1DvummmwAkXR/VlFVVCl5rzdRR1z1bl2hblWqpqaNrprW1tK0G61N17949dNq6hd97bAUEJDPo2HVeM331+yHvmkJ03euZo+crQzsqiS06xhhjjKlZymbRaQj65KmBnaw5oE0f2Sjz77+XJfoWxKZ5Wi1WgzV1rMsvvzyA0m9Oxd7eXnzxRQDJaq5aWVODZPMITGYQWbGmg3ySv/rqq0N3/fXXh8wGd3m8WRazeBx11FEhf/vttwCSTfVGjhwZMteHlh0g+facxxsV36K0WrfuydGjR4fMOiz6c23wyH1dzApAfRaWG12fUo06S13nYlXc+farOt0LtFS1aNFitv9WY+G11OvMCs4AcPzxx4dMS5NaPNTKy5+rFaQposkOWjOIFXfztlwo3Ktawfmggw4K+fXXXw+ZzTi1Ka6uH63DWln4xx9/DJlB9rTMAsnvijyui96rrP9zyy23hI6NcoFCEHUlz39bdIwxxhhTs/hBxxhjjDE1Sy6uK5ppta2Duq5YYl5Nc2ykCBRMXlmbzrWsupr+idaxWGGFFer8vFiJfppktWmpuklohmRQ89/lLAPv6nNNNQBX148NSLWRoLaAoPshDxOruj5YCh8ouKuAQv0RNSfT3QMUmpJqo7o11lgj5DybWmorBm0rooGNv/32G4BkWXoNnGcwsgabl2o30Vj4mT/99FPoNBif7lqtvVGsTlDaz9U1dddddwFIltjfaKONQua882groOZ+dW2kufG1TpAG+1eTS6chcPytWrUK3a677hoy7zs9f9S1VSn0PuC5orXVGDQNJNuRpLXASXOzaoCxfv+RCRMmpP5+3oHZPB95zgCFthdAPskatugYY4wxpmbxg44xxhhjapaKua7UTPfhhx8CKJjIgUJHaKDgBinW4iHLOh362epaoplb56HZT2omp0lVzYZaB4M1O+69997QqWvspJNOApDMNNOaHuUy/en46A5UE7hGxev8mNmj3au1nPlVV10FINmdNu/aHRy/ZiVpnRx2fAYK7h/NWtG13mmnnQAka6+o6yoPuCd0nxx77LEh77bbbiHTtKyuO+3ezToe6o7MuqYM7zut7aPuTromNtxww9Cx3hZQaHug4/vll19Cx3sKKNQfYSl9IFkHK49Mxq+++gpAsu2N1jY666yzQub4NRNQ17Ja6lQ1FK6fnrVae4XfG1qbRd3MebqONVNK3cG613gvFcsgpF73x5tvvhnyc889ByCZiZc2lqxJy7QCCntR7zmtw5ZLtm3F/6IxxhhjTIXI1KKj1hF90mVjTA02vOOOO0Lu2LEjgHzeTPRv6lviJptsAgC47rrrQqdvWePHjw957NixAJLBcu+8807ItBRoALNWXE4LVivXtdA1GTVqVMgMgmTDx7+jwZxssKdv+VqHhZVlqwnOW600GrjL2j5A+huR1lxZbrnlACSrlR566KF1/lYl4d/UfaIBmqztBBSsN9ttt13oevXqFTLvRQ2AX3vttUPO4o2R41511VVDp4GdHPPQoUNDpxYNrczNN01tBKkWVTbFvOGGG0KnFrtKvXHq/c0EBLUi6/qwESIADBkyBEAyGHnLLbcMOW/rabnQ+0gbeDLIVZsGr7vuupUb2P+R1kB30UUXTf2/V155ZchcK7WI/PrrryGz8rV2C/joo49CfuihhwAkG7XmYcXS9dGmwKussgqAguUbyN/KaIuOMcYYY2oWP+gYY4wxpmbJ1HWl5jSaW4FC07k+ffqEbocddgg5TzNXsbLxNB2ru0NN6xdeeGHINEmrOV3rQLB1gJoeF1pooZA5/3Jeh7QS80899VTIdFmpuZ8BkkCyQSLbJey1116ha9euXZ2/WU31PHgt9Tqr60DdGBrk+vffBwq1XD755JPQac2VPGp6kLTaHn+Xua/pggOSbki697SmTaXQ8W+88cYhs60IA6mBZNl7DTLnXNX1s/LKK4fM0vx5uKuK8ccffwBItpjRsvna9PHuu+8GAHTr1i10eddOyQLds23atAmZLhFtGpyH60qhG0oD6LUmkiY7aH04omvGs0RdW9rUevXVV6/z80p9ZxYLR3nsscdCPvXUUwEkz5S87y9bdIwxxhhTs/hBxxhjjDE1S9ldV+oOYPQ4kOy+vfnmmwNItlBo3rx5yHlHaBM1J9LloSbEvn37pv5fotdC6wjQpJdWOyFr9O9onSK6K7TEvprONQOHbix10eRRLr8+8FprKXWtM6PrypoXa621Vupn0XzbpUuX0OlaVyPqhqXJWTN5NIOFLh3N1KoUxdwuHNPRRx8duv333z/ktJokes/p+UKTf97mdB0z3dnqAmYrHADo3LlzyNzD9enk3hTR+WkLFg2DSPu/ecC/z87cQDKrddiwYSEzA1DvSbqjgMK+1EzQrl27hsyzJo856zmntX3UDdypUycA+a+JYouOMcYYY2oWP+gYY4wxpmaZY1Ymz5kzZ9bbHqqmrXHjxoWsJbxpMu/du7f+rfr+qQbTvHnzOQDgr7/+qvf8NOq8PoXh9DpnbWZu1qzZHAAwY8aMWf6hxs4lL3P5nHPOOQfQsP2pFOt+Pbsm16yyHhqzPxVd04kTJ4bM7vJa5EsL1TFD5MQTT0z9rMbC/Zn1/Vdsf1bq/qvP/uReKjanPNzcxeD+bOz911B4rbI6Uxszv2L7syH3T7H5NXau5TpfFD0L8/5+4P2n2KJjjDHGmJql7BYdRVsgTJ48OWQ28MyrEVsWT7TVxOxadJoq5bLoVCtZ7E+ticN7UYMh9Y2M9XU0cLucTXUbY9FpCjTEotOUyNuikzX/lPnV+v2n2KJjjDHGmJrFDzrGGGOMqVlm6boyxhhjjGnKzLJgYK378GrdBzt9+vSanF+LFi3+Eevn+69xaNyRxiMx3iirIpf/lBgPz69p8k/5flDsujLGGGNMzZJp93Lzz6FUzYi8a3+YupRas2Kd0Kvd3c1aXl9++WXo2PEbKLRW0BYfWv+r2ufHdSmV/eZ7zpj/xRYdY4wxxtQstuiYBqMxENogcdq0aQCScREtWrQIudrfmBtCmnWkWi0itF7ommllclZJVt28884bclpT2rzRBrNsJnzEEUeE7sUXXwx5tdVWAwCMGTMmdNoAtFrmpVYmrUlGmc1ZgWS8EffdggsuGDq9F7nu5ax2nQc6fl1/otesWta0GLrWaRXbdX2bylx0HvXZazwr9Xxq7Jxt0THGGGNMzeIHHWOMMcbULLm6rhpqOq12012tQ9PkF198EbpBgwaF/OSTTwIAll122dBde+21IS+55JIAkqbJPFDTapobrtQ+0/2rpuXff/+9zmeq66dSga86PnVdfP311wCAG2+8MXSffvppyM888wyA5Pqom2eFFVao8/M80Os7fvz4kE877TQAwJtvvhk6uqsAYP/99wcANG/ePHTVcqaoi/ONN94IecSIESF/9tlnAJJr8ssvv4TMte7Xr1/ojjrqqJBbtWoFoHrmXF+47nSRA8CDDz4YMl2rm222WejUtVUt89b784MPPqgj6/7cYIMNQuZZkvc89BzTfcv9+fHHH4du6tSpIac1/dSzZNVVVwUALL/88qGbe+65U//v7GKLjjHGGGNqFj/oGGOMMaZmmWULiHJWZlUzHU2Pau5npofqNVNHzXj8/YZmsjSkMmupTBqFeh2TyjT5FTP9NZYsKl+qm4Am4yOPPDJ0d911V8hrrLEGgKQ7ZL/99gv5ggsuANBwc3JjKiPrNVdz8QsvvBDyAQccACA557Tx6V5QcyrnN2rUqNCdcMIJIe+www51PkM/vzGVkYvVxlHX4g033AAguT5aU2bvvfcGAGy66aahY0dzoHBdGmo6b0xlZN0zahrfa6+9Qn7vvfcAAK1btw7dY489FjLnotennG6AxlTW1XPgsssuC/m+++4LmXWAVl999dDRXQMAjzzySJ3fGTJkSMg9e/YE0PDK0HlXDua63X777aE7+OCDQ+Z3xciRI0O34YYbhlzK9VGu+RVzjVN+6623QsczByi4VmfMmBE6dWOef/75AJKhAfX5/ijX98PLL78c8rBhw0JmhuNXX32VOr61114bANCmTZvQ/frrryF/++23AIDFF188dOqG7dy5c8hp960rIxtjjDHmH0WmwcjF6kDwie+WW24J3fPPPx/yH3/8AQBYZpllQqdvwVtvvTWAguUAyO7tjPDtZ9y4caG79957Q55rrrlCpqVg4sSJoZsyZUrIfCPbbrvtQrfllluGnFZHIQ+KWQf49D58+PDQHX744SHzjePcc88N3eDBg0NmfZN27dqFLqu+Q4TXVN+Srr/++pB1Lbk+7du3D13aW6CuE/csALz22msAgFdffTV0r7/+esi67nqPlAMd07PPPhuyrgXfBPWNV9+S+EZcrE5SHkGQ3H/ffPNN6C655JKQ1TrHYHe1Yqy44oohz26weR7oNT/++ONTZQYbF1sfWrTU+sUA5KaKBtPTOnfyySeHTmsG8bqoRyAPvvvuu5CZAAAUrG8XX3xx6DbZZJOQDzroIADJ9VOL8IknnggAuOOOO0KnHo8s4Dml89AA9wkTJoRM65pa2RZZZJGQWbNKx6z797fffgMAXHTRRaFjggsArLPOOiHPrnW5Or5RjTHGGGMywA86xhhjjKlZyua6omlZzalaZ+W6664LmW4MdWepa2SxxRYDkHRXvfTSSyFfccUVAIArr7wydPvss0/I5arvoXNh4Nhuu+0Wuu+//z7ktJpAxcp606SutTHUjcBgrbzrlOicJk+eHPJVV11V5/9qYNkCCywAAJh//vlDp3PJ2k2VBq+/uhPvv//+kLXdAZtBsp4DkBw/r4vO49Zbbw2ZQXrqetVgWTVJl3uvalC/BmtqHQoGqy688MKhSwvsz2OdlLRg7z59+oROzdlLLLFEyLy/unTpUuf3gep0WaWh58cPP/wQMu/FhRZaKHQa+M51V3fyRhttFHLe6zq7FKtT9dRTTwFIhgOom4r1V1ZZZZXQZd3glPefBuD26NEjZD13fv75ZwDJ83H06NEhMwxC53/MMceEvMsuuwAouMiBZLB1FnPlZ6qL8Jxzzgl54MCBIdNNqnVwSrUA0r3O7/8BAwaETr9rldm9l23RMcYYY0zN4gcdY4wxxtQsjXJdpWUHadn1Qw89NGSNQGdWi5Zt1+7BrCOgmQZqkqfJ7Oyzz67zmQDQtm3bkBvjGlDTIU1nP/30U+iKRY3TJaBmNS2BzeumtQPUdMkMmLxdV4qaWekmbNmyZR0dUDAza52WPND1o2tKXag//vhjyJrBR5dVsetPMytbPQBJNxjrDGltjA4dOoScpRld56zuKh3rRx99BADo2rVr6mdUo2uHWW1jx44NnWY6albfeuutB6B0na1iWYVp51qxz6pUCw89H4499lgASXeAru+OO+4IIOla0Do7WbtxyoWuAzvSA4VroZlYei0Y2qD7v1KuK9aAAZKupQsvvDBkni8nnXRS6OjuBwpj1fnrdw3nqmuedSd67nPdRxpaojVvevfuDSBZ++e8884L+f333wdQyA7UzwcKIS3bbrtt6DQrtCHYomOMMcaYmsUPOsYYY4ypWertulJzmka9s/gfi8UBhcJ+QLJFAM1cNMECSTMeo67VHKdukrPOOgtAskiR/t2rr746ZJo3G2Ji1t/hmFZeeeXQsUsrUCjypPKff/4ZOjXzUa+fr4XsqgU196pr8ZRTTqnzf9PM+RpJr9271fyZJbp/6Ka65557UsekxefSuqvrZ9ENqa6Bt99+O2SaWdlF++9k4e7gWum1Pfroo0N+4oknQmaLBy04pq7ftBYmecP7X91V2rE7rThcsfHzDNP11XuVGTKaFaru8LRO9FlcK/1MuqOAghuf2UcA8Oijj4bM/andvfW6VTtpGYR9+/YNmVlnOifNSlpzzTWzHmIdeM8stdRSoaMLFUhm1e68884Akt9/mgH5988Ekm4gnsVaZLBS7shiHcd1rvwu33fffUOnxUu5vprVqkUAmU2o97TOT79XnHVljDHGmH88s23RSasdcvnll4fMnHe1bDAoDEhaBPgkeOqpp4Zu6aWXDpl1RooFg/JJ77jjjgvdFltsEbIGPjGnvyGBvfoUyWDShx9+OHRa9l/f+Pj2qbVZ9CmUY9Gn+KzrIDSEYk3p0sanFg++CWsdJa1pwVoMlQoQBAp1RjQonFY6INkgkb9XLFh16NChAArNMYHkm8Vhhx0GIFl2v1QD2HKh+1zbHtx8880hsxnrgQceGDptEUHro1pR86hDo9ecSQBqpdAAeS0xnzY+3Qu0BGtg+ocfflhH1vtXr6XuZZ57OpZyofPQteAZu8cee4RO6+jQuqEl+m+66aaQGaRbTRa7tPNDA8y1ASbRYHq1jvCsreQ5yr+lZ8ptt90WsraroEVVPQ9qieVeVY9B//79Q+b6qkUrj7VMO1+BQp01DQbX2mGcq7aA0po7aevX2PPHFh1jjDHG1Cx+0DHGGGNMzTLbriuaqTQA8Nprrw2ZwWIaoKnBRGllx5dbbrmQ1RxVyuRIM6e6wzSYMAuTJU1vDFTVcQDpLQ5KjUOvj7qx8uwOrbUpNBiQtVcA4Lnnnkv8DpCsA0HTo9a+0BYIWZrOdfxap4HB6jpm3TMapMz6Dbp+Wj+J9Z90zbXOBWtJsZUHkAxg1WvFPZBlgDIAbLzxxiE/8MADAJLm8EMOOSTkMWPGAADOOOOM0KlJvlLByrpWdDexszEAtG7dOuRS3bnvvPPOkBl4rskMmgzAtdLP12DQjz/+OGS2WdA1zQK91jxf1J2lrWnmm28+AMDuu+8eOnUt8P9WU50udW2whYruT3X9c100gSCLtioNQddp2WWXDfmaa64Jmd+RdGEBSTcq97K6JjWZRc/SSqPuKtbDAZLJRKxJpskYetbSpapnLjuyA4VrWM7zxRYdY4wxxtQsftAxxhhjTM1S7zo6aa0OgPRIcK1DkUaxUuo0WRfLemHmzKWXXhq6Yp3CywXHNztm0dktx12sJkGl0HHSHaEmbjX3s+O1/p7+vpr++VlqWtc6IIy6L7U/GoKOSVts0OWh+0QzeDQrQrMhiNaM4rppVoH+XWa4aFsIdampGfqyyy4DUL71L7b31I3FDKIbb7wxdFpT54ILLgAAjBs3rs44gYJLTv9W1nVk6BpSd+/XX38dspbbZ/0udZdffPHFIbNOjn6WZovSNaCZPtpuRted7vNyzV/PrmLufF73tLL5ANC9e3cAhXotAPDiiy+GzHsxi3OyPqS1aAEKNVeK1RljiyB1DVVLpqqiY9KsQO5FzTTWTvOct2YSqxuZ52cl55yWCaYtbvSsZTa0ttXRbFe69jUru1OnTiFvvvnmAMr7nWiLjjHGGGNqlkY19dSnNFar1GBdfYtNQ5/Y9e2CT3L6lK9Pkny70mqTmpPPOi1APk/6nFexNyb+XK9PHjURdHwMttxzzz1DpwG2p59+esj77bcfgOQb2b///e+QWUdGn8j17blSjRB//vnnkGnR0aBgfQvWvTZp0iQAxRs58o2qmEWLlix9S9Mg/l69eqV+bmNIq3Olc9L9xXmpThvwsuIq1xlI7ouLLroIALDrrrvW+ftANutL67EGnWpTw6effjpkVlnVM0OvP616avHQwF2eK2rRZLVWABg+fHjIPO8ae85wL2qF7TZt2oS8xBJLhDy7yRr6/3T+vC8bUmG2nKSdP0Ch/oyej7ru3bp1q/Nz3evViK4FLYIaQK7eCa7FEUccETqt05SH9Z/jVyuMNjB96KGHQmbFYz0T9buEQcqvv/566DSJabXVVgOQngDRUGzRMcYYY0zN4gcdY4wxxtQss+26oumINRoAoEePHiGzJoCagLWsvpoeGVj4+eefh26ZZZYJma4DbQQ2evTokFdaaSUAyUaFWg68WJBzpaDpWE176iah6VED1Nq1a1fn51mj7oZ33nkHQNIdeeaZZ4asTfNoclY3ia4VW2NsttlmdXRZoyZODVZk4N9RctEIAAAgAElEQVQ333wTurXWWivk7bbbLmTuL60jpK0/Hn/8cQDJPb3BBhuEvMsuuwAAevbsGTo13danZtTswrXUMV944YUha4M9NqYt9rc5f3UH9+nTJ2S2btFGfuqyLtc9p5/DRsAaNDxo0KCQtdw+XU+6P/W60GXz2GOPhU5lujn172uLDG1GmDbW2UVdNzzzdB8NGzYsZK2dkuamUTcU3V/cp0DSDcJ9W00tIK688sqQ2YBXzyf9XmHTziySGbJC58KaZHpP6f3JM1hbGd16660h61mSJTpmJm688sorodMAfW2kmnauqI4uc3WDdezYMWTWP+M9Xw5s0THGGGNMzeIHHWOMMcbULLPtuqKZU7usqmmNWRnaJVfNberGYTaMRmVrVgFNVp07dw6dZkCwjoeWXa+mcuY0SbNVAJCcK3++/fbbhy4PM7L+TUa4q7ly0UUXDVnN7KxfctZZZ4VOzex0bahpNq17exboZ7dv3z7kRx99FEDSdaXuQq1pkVaLRn+PLRS0hYmWeKfLTMeS9f7kWmp2m85DM4y4bnSxAekZkuqiUTM1s7HobgGApZZaqsFjL0Zaba1+/fqFTl2/AwcODJkZoOq6SsuA1DpKaS0G1F2r5fqLjbG+pHUn1+wuzWRhJhxQOPfSWmQABTeI1rHSMAPONe/aXS+88ELIWgeJe1H/r7os86gj0xDSXD9Aod2BZhWpm/mDDz4AkNxz1AEFl3sl1y+tLQPvMyB5r3H9iq0P78VibVuy+C60RccYY4wxNUu96+joU5oGRbF2hQYV6f9Nq1mib1n6RsVgOX3LTHsjqyYrThpqxUlDa5fkgT45M5hMq3EOHjw45E8//TRkWjS00ae+/bLBm65fFgG4aaRZAYBCgC3//TtpTWdVp5WDuVdZoRVIBj5zX1bSSpdWm2nAgAEh69sTa3bcfPPNodMgWO4BDSDXysK8f1lhWf9+VqTN7+CDDw5Zg3WHDh0KoFCB9e+/R3R91frHz9XaJUq59q9eM1q0tfaS1lbRKtVt27YFkAxK1kajPCvvvffe0On88jg3/z97ZxkuVfW+/9vr+oGA3d2KKAgGgh0YiGJ3d2Jjd3eAqNiB2GJ3d4tgoKJiYQF2EH5f/N/872fuLXuYc87Mnj1ne3/e+FwPnpm19lp7zd5P8p759ttvQ6cB0gxABkr3rVq8NXGgkYKop4WeP08++WTIXEtN4ND9yaam2ghTfx/rhV5nBhBvtNFGodMK8toAl/WBtIK4ftb48eMBJJuyauCxWjVrhS06xhhjjCksftAxxhhjTGGpqgVEmglXzXVqblNZAyabSqO7qRRegxdffDF0WscjLdguD3T9GIyrjR7Z3BFIminp5tK2EJtssknIeQY7KmlNU5vSqiBtXTSYkG7YcsHkeZrWy7nu+vfvHzLdPOqaZLA2kAw8JnSXACWTs7Zaqdecy7mN1Ezer18/AM27v9L2Sh5NEzXYmg1DgaSb8aWXXgJQaoUAJPciXeJdunQJXaME7pb7fVA3KYPkL7nkktCpa6dR5tIctOklXY4MAQCSgf333HMPAKB3796hY+0rIJ9zlftTmx+PHTs2ZLZ1AErNjMuNma67UaNGhe6KK64ImfX33NTTGGOMMaYJ+EHHGGOMMYVlummZnP/555/WEd7eTNq0aTMdAEyZMiWT+TFrTLs/a8djmuQ0A0Fr1lRrmm3btu10ADBp0qRmz0+z2zQrRTtF0/WodTrUJJ21abldu3aZrl9a92d1Q7JFgLa4KJdh0BK4flndf2muRXXNabaHjClkZiO11F2X9f2XN1y/5syP10/vP10fbc1C16l2n9d2JFyrrFwcLZkf7yndJ5qJpC0gTj75ZADAGmusEbq0rMisaMn80tAz8ccffwyZrXU0a1Vh1tJee+0VOj1rq6Wa3wedk4ZjsIUQUKqPxFYsADBu3LiQWQfviCOOCJ3W7ErbK82Bvw+KLTrGGGOMKSx+0DHGGGNMYbHrKgNofmbhJyAZQb/11lsDSHZf1ky0Wrk+WmKaVNRMmZbBklemUdauqzTSCnapa6uW88/adUXKrW9T17qlc7brqvmkFUwtd/2zvhermV9T9hzHn1d2VRbrV+n+SiOr87Vevw/NoZZztevKGGOMMf8pqqqjY9Lhm8gqq6wSujfeeCNktgtQK04j1oZolNowjUDeNYGywOvbemjE86El/Ff3XBHn3ZrmZIuOMcYYYwqLH3SMMcYYU1imGYxsjDHGGNOasUXHGGOMMYVlmsHIRU8vrza9rqUwFU8b1bHaaS3II/26ntQr/TovnH7duvH8Wjec38SJEws5v/bt2/8n1k+xRccYY4wxhcXp5XVCC8799NNPAICHHnoodDvuuGPIaukxjUel4m1pBfWqLahlpqZSwbKipGTXgkYt/vlfpjkF9xpxfSoVfGyUcQK26BhjjDGmwNh0kCFqxZkwYULIZ5xxBgBg8uTJodt1113rN7AMUCsULR4ad9RIT/dppFlf1HKjc2En9ylTpoROuwvPOOOMmY3TlNAijn/88UfI008/PYBkR3m9F9NaDBSpIGSl/fvnn38CSM555plnDlnv5Ua/b1sjXJ+///47dNqdPo1ZZ501ZBaarWdH9zQq3X96Jur9x/uunlYqW3SMMcYYU1j8oGOMMcaYwlI315WaUdMCr5oTmEXTVyOZm9OCsUaOHBm6fv36hbzuuusCAI4++uhp/n2jkrY+n3/+ecgfffQRAGC99dYLXdu2bUNuxPmlzembb74JecCAASE/+OCDAICvvvoqdF27dg35xhtvBAB06dIldI0056YGRus9qzLvv6y6t08LNYEPHz485K222irkFVdcEQCwxRZbhG6eeeYJuUOHDgCATp06pf4756rnSyOtXxpprgHdn9dcc03It9xyC4Dk/J5++umQl1122ZDzdI+Uc6E10rnfVPSee+211wAAZ511Vuhef/31af7N/vvvH/IOO+wAILl/65XAomO6//77Qz7iiCNC7t69OwBg7bXXDl2PHj1C7tixIwBgpplmCp26udKSCKpNLLBFxxhjjDGFxQ86xhhjjCks0+x1VW3lWTUxvvPOOyFPmjQJADBu3LjQvffeeyH//PPPAErZLUDShLr66qsDAA444IDQqem2ErWqjJxWTwUAHn74YQDAoYceGrpNN9005HPPPRdAMiuklubYrCsj03ypmRxqWn3kkUcAAKNHjw7dbLPNFnK1c826MvKrr74KADj88MNDN2LEiJCZ9aDrp1kTa6yxBoBknaSW7M9q16+ci4rXv1zWEe/P999/P3SPP/54yCuvvDIAYOONNw5dc1yT1VTW1Xtu7NixIdMdDJRcjvr/MhNEURcN3V1AaX5bbrll6DSTLsv5tZQXX3wxZJ4//C+QdGPR5XHaaaeFrnfv3iHrtUqbaxbz0/uD58pll10WugUWWCBkum7KnSPc9y11N9aqMrLef88991zI/N1iPTUg6XpKuz91L88xxxwASucUAMw+++whV5p3Syoj8/t/++230GntN/19n3feeQEAY8aMCR3PTKDkulpwwQVDp2EO3bp1AwDMN998odP/txKujGyMMcaY/xQ1j2DSJ0+12Oy5554h//DDDwCSVpolllgiZL79q0VHn+6+/fZbAEmLQr1qP+j8+OYLANdff33I559/PoBkgNbBBx8cMsfaGoPqgNKbitYB0mBGrl+jVwPWtXzhhRdC7tOnD4DkGxWtiACw9dZbA0i+hZx++ukh8+2N+xQAFllkkZDzCGxlgDgAnHnmmQCSwda6Vnzj0vtX/37w4MEAgLvvvjt0G2ywQchZ7mu9dhpATCspUKpZpVYYtfjQeqxWkKuuuirkG264Yap/V+uCBk7msZa0fowaNSp0++23X8i03mgw6CWXXBIyLY6zzDJL6PIILNc9x9o+AHDnnXcCAAYOHBg6rklTPov7r5zFPUvUMvXxxx+HfMghh4RMS45aQXVPcV/rb+Inn3wSMi1u9ZxfmhVfa+do4gUtiR9++GHobr755pBp6VcruZ6/bdq0AQAsv/zyobvyyitDXnjhhUNu6llji44xxhhjCosfdIwxxhhTWDJNvlcTqLqp9tlnHwAlFwAALLbYYiGzHPnEiROn0gGVmypmQZrpTs3latqmaX/bbbcNnY45bawtqSOUFxzf119/HTq9LnmPr6noOiy66KIhc39qnYrtttsu5LnnnhtAck/r/mXgrgbu1atOkrpwX3755ZC1jtPSSy8NIOnu0DorrIl06aWXTqUDgH333RdAyQUN1M9NqddO3QR6r3Esuj56Xeaff34AyQBydSPQJa0JEmqmzyqJYFro9eX49MzRwGzuxUGDBoUurTZO3q5z/X6tKcY6VEcddVToevXqNdXf6TXRtaSbbqWVVgpd3759U7+3VnAs2upH3YXqBqZrhv8FkqEPvD+1zswvv/wy1Xc2J0C+Wnim6++wno8vvfRSyGyNs8oqq4RO14JuyvHjx4dOZd7Xep9pC4yW/L7YomOMMcaYwuIHHWOMMcYUlkyzrrR2g5oWGe2v5uT77rsv5M6dOwNIz8QC8sl04Ly0XgBdVABw8cUXh8w6D5W6s6bVjgBK2Trfffdd6DTDRN0s9SLNtTZs2LDQ6fout9xyABq/C7KOaaGFFgqZa6nro/uapu/vv/8+dM8880zIPXv2BJA07dbLnafjfOWVV0LWe/GOO+4AkByf1ozhvLW78nXXXRcyXR9a2yLv9VV3BPenZkVqTaN77rkHAPD888+HTs3kzCA57rjjQjfXXHOlfle9SMvmYSsSAJhzzjlDZoYSs+eA5PmSJzoPrS1z++23h8wM1WOPPbbJn6VuRma9qutL65hlCTPGAOCuu+4KWevIEN1H6npktpG6idR1k1ZnJ2t4f6u7jecckMzAZIbUkUceGTq2XQFK89L5LbnkktP8/mrbsdiiY4wxxpjC4gcdY4wxxhSWmruu1MSkHVm1INRFF10EINm9Na1L7jLLLBPykCFDQma59qxNyGmZDlr4UAvG7bzzzlP9fTkTG02uWrBNXV80f2rUvWaFaGl3FqLL2oypLhFmFrz99tuhUzfBrrvuCiA5/rwzPJpDWrsGHT9dcloES4tfMYNEzbx5oK5DXQt1AxM1jdN1etJJJ4VOzfAsvqdZFY2Uacd5q7tN3RjMSlLX1Oabbx4yS9hrVkve89OzhFl/aR3ZgdK5meZuzQuORbNrzjnnnJC1uCELWqq7R+fP9f31119Dd+qpp4bMkAct0pf1/Pn7pYU1K2Va6pi0XRDd4CeeeGLotLijulHzZI899ghZi/tyXbXIod5/DL3QMymtYGUt3eG26BhjjDGmsNTMosMnds331wBjbRfAN45NNtkkdGykB5TeKE855ZTQqRVDc/KzRJ/I33zzTQBJywXriQDJRnh8Oi1XW4SlsbURptZhYeCklgXXFhJaq0FbC2SJzoUlzD/77LPQ6dtj3m+/TUXfGHR+nEu52kbc41o7QvfkgQcemPgcoH7XRC2jWvtHrRsnnHACAODyyy8PnVpcaZ3UFghrrbVWyKwPom/cea+5WuH4Vq3z0zd+3rf6Zpy2VnnPqRwMLNZgcQ0s59t/I45fA+S1zpM2qKR1ivVYgGRNNZ47WkdIk0RYE0oDtLO+FjwfNKhdrfAqT+vvgdJ9pwHWbOQJlOpbMWkHqF8yQLk6Vmo94/qpl0LPEu7VVVddNXTaIJQWn3LJLA5GNsYYY4wR/KBjjDHGmMJS82BkNTeVM00xiEmDNfXvaLK8+uqrQ6emSQY+ZR0sqKY5Bgj36NEjdGrOV3MazeAabHbttdeGzGCtww47LHRac4BuMK3NojVbFl988eZOpabQzKprpte/UWp2KFwfNRGn7Tmg1NqgXEdnmmS1Y/uFF14YMt2JeQSA6pi1NpDuP7qM1Q2rgfGsuaOm57333jtkmucbyTWirqd3330XAPDFF1+ETt0YDDbW9ck7WLcSeq05F3V3f/DBByHzrNA6R40yP3Wt6pzUtUo3x1tvvRU6dXkxCFn378YbbxzyRhttVMMRNw3uv/79+4eOSRlAehiD7lkNrGbIh7p7hg8fHjJDHp588snQaTB6Hm4sPUs5Pj0ztE4SXYtXXHFF6JigBJTOJw3X0N/dlnRtt0XHGGOMMYXFDzrGGGOMKSw1c13RDKnupLPPPjtkdQOl5cmnmVbVRJVHd2wdE10bOqa0TB2g1KlWTf+PPPJIyAMGDACQrN2hf08z9HnnnRc6NeNpTYU83QdqhlY3CTOQ8m4LUC6rinz66achX3XVVSE/+uij0/x7Zp2p67Ul5tSs0fGr64ZmZi1Xr1ljDzzwAIBSKw+g8bPqdEw0c6uLV03jrH+lLRLS6ng1ErqWLJ0/33zzhU7dPDx/9J7MG45/nXXWCZ2669m2Aih1l9e2AFtvvXXIrMWjZ+o222wTcp6uVb3PytW7STsX9XxZffXVASTDFTTDjG4+Zu8CSddOHudu2neqO0vdeGzHMXr06NDde++9IbNFi2a13nrrrSFryEhT17jxTmdjjDHGmBpR82Dkcm/B1T5dlwsMzRL9HjZa0+Zlv/32W8gaGLfXXnsBAL7++uvQaU0hNg3Uv9Fgu4MOOggAsNVWW4WOtVmAyhU364W+BS+wwAIha5AkyWOcaVYcffPVAPnZZ5895GWXXRZAss6HVv5kfSN9Y2EjQaAUBKqNMvOYv1oktaYM33jVoqGJA0svvTSAxrfiKDpXWnK0zpVWmWUdElbgBfJvGkzU8l0ugJhroU0V77///pC1zlajwGuq95lWxtcGnrSUqsVUrwvrq2k17/XXXz9k3veNalGuND6eq2qRU+s/Ews0wJe/T0DyuuV5DcrV3KGlS2sD6V5/7LHHACR/P9X6o1W0m4otOsYYY4wpLH7QMcYYY0xhqbnrqqWoaWvs2LEAkiZYDYxknZl6muW23357AMmgRm1UpnVIWKJca6uo6Z/tLIYNGxY6tpgASqX79fMbxV2l6JjojgNKrQHyDopT1xJrUmjz1dVWWy3kQYMGhfz4448DSAbDMQAUKLk89PPPPffckNluQJteauBoltelnLtYXZ+8l7RFhLphaTpvlH3WFNLujyOOOCJ02i6Ba6UtVvRezQOuFdvfAEl3cJobli5GIOkGp8u1V69eoVM3ZKO4M/T+UZcW0TGrG4PBqvvtt1/o2OgUyNfNOnLkyJCff/75kDUIm2PVtkHquuFvobZl0ZpdvD/1mmXtrirXzijt39MSM3788ceQmQSidYAGDx4cMuetrufNNtss5Jasry06xhhjjCksftAxxhhjTGHJ1HVVzt1CWU2XmsHDEtE///xz6LROAk3OWZc1T6tDorVxdtppp5DVnMYS81r7gLUtgFJ3VnUdaFYIo+0b0V2l6PqtueaaITOrJ4/aJGo21Y7H3FPfffdd6NR1ceONN4bMDCq9/rqWO+ywA4CkuVhLuB9zzDEAkpl2WgepXuX4R4wYEbK6zrbYYgsAyU7LDz30UMisM6P/Xi8qmcjL1SvSOlt//fUXgKQbSFt88DvY6gNIdx1kfc+l3d9HH3106DQrULNuOFftbp3mRtb7s1FaQCiVOlLrnNiRHgAmT54MANh9991T/988zkqe/1oPiGcOkKzpRNeVZo1xTkCp07mGQ+jvB/9fPb809COLc5f3j/7O6b2oe5Guf/391mzXTz75ZKpxap2k3XbbDUCytpfW52vJXrZFxxhjjDGFpeYWHX2y1tojWsWSlgy+eQHJBl+0muhbjAbW5RFsxrcEbRh30kknhaxz7datGwBgqaWWCp0Gy/Htmk/u5b6rEa04io6vUeqslKvdQOuO6h588MGQtT7SuuuuCwA44YQTQqeBxUTn3KdPn5AZsK4BolnDtysNYGS9GKDUSBcAevfuPdX/++WXX4bMyrMaDJu1dY7nhr7F6pnBseo66JudVlZlHQ612Oha0WK17bbbhi6tcnvWpO3VJZZYInRaTZa1tYDSm/Jdd90VOq0WyzojjX5+VELXTJs6800/79pH+ltHK9vHH38cOj3fNRhXrTNpcN5q5VGLRt++fQGULLNA9RY7zkU/Ry3Sl1xyCYDS2QAAM800U8hMIAJK96r+u1qUd9llFwBJi1zXrl1DTrOoVjs/W3SMMcYYU1j8oGOMMcaYwlJz15WagN94442Q1YzOpm1XXnll6NSNsMYaawBI1qxRM1gebhJ+p9Y+OPnkk1P/3zTXU5rcKO6elqIBoI3SFFGvs5qODzjgAAClGkcAMGrUqJA18JP7Tpvypa2VfpealllHR10P9Qqc13EsssgiIWswNV032jRQXT8Mtq4ndL1po8JDDz00ZJ4r5a5jhw4dQt5ggw0AlIIagWS7hJVXXhlAsgGj7uU84Px1zuom0DpPDAzVZAg9i1hav7WfLzp+dQltuOGGAJLBuHkEW+v9z9phGoIxdOjQkPX+YpB8uRYr88wzD4BkmATb0gClc4Xf+e+/rwb9nCeeeCLkd999F0DyTNV7bpNNNgm5e/fuAJL3HOcElJJtdP66flmspS06xhhjjCksftAxxhhjTGGZblrR6v/880+zQ9nVdfXqq6+GrKX3GZWtdTq03QFNshpVX0szbJs2baYDgEmTJrXutIQytGvXbjoAmDJlSs3mp+v61VdfAUh2fz799NNDpmsyK9N527ZtpwNatj8100/r7KhrtCUtLHSuzGAoZ5quVCuG+7Ml66ffqZkeWseHWR86/9NOOy1kugayykTi+qXNT83W2pYijXLdoXmuqJldZX5H1vuzJeun89DaP3/88UfIHPcss8wSOnVjZO3GqWZ+zUGvxRlnnBEys1nVNanXqlo4v4kTJ1a1frq/NCu3kpuU9Y/0TNJ7sdru7O3bt2/S+un5oC1UiIZx6P1Ffbnzo177U7FFxxhjjDGFxQ86xhhjjCksNXdd/evvQ9YS0b/88gsAoGPHjqFjWwQlK9OyXVfVkeaaqWfxwKa6rtJcG2partSipBLlXCfVUo3rStH10ay41157DUByzMx0BLIvWDkt10e5LsiVWgSk/XtL1rQW1Mq1U26vUtb7rDXOrxLlWkCQzp07h1xLd0g1riul3JlQ6ayotJerpamuq3LtVtLI615Lw64rY4wxxvynyNSio6QFU+kbST1rPtii07qpJhi5NVAri46ib5F6L5J61kGql0UgLzy/2pMW2KpWnFpaVGtl0WlUmmrRaa3YomOMMcaY/xR+0DHGGGNMYZmm68oYY4wxpjUzzV5X9fThaYR3Wl+bWsbw/Fd8sI5Bap04xqM26JnCImyKxiXVMtPlv7J+RY+RK/r8Jk+eXMj5TT/99I7RMcYYY8x/h5p3L28O+sbFsvQAcN999wFIdm9deOGFQ7a7zdQb3atpGR71rCNkpg3Xh/W6AOCBBx4ImS1ott9++9BpuX2Wsy/XwsOYWlCuTpR/32qPLTrGGGOMKSy5WHT4xqWVk88666yQBw8eDCDZyO34448P2U+8jQ3fVNLqtZQjq3isatA3Lm1wN3LkSADJRqZaWfjYY48FkF1TTDNtuG5jx44NnTYNpkVnwIABoVthhRVCPvXUUwEAnTp1Cp3Xsjp45ut1bGkdNX6Wxl014vlRCe5DINkgk/P6L+0ztZJz/rVcU1t0jDHGGFNY/KBjjDHGmMJSN9eVmqZokqKLCgBuueWWkHv16gUA6N27d51GV1/SmmKmXZ9GN12qGVpTdceMGQMA+Pjjj0M3efLkkDkvBn0CQNeuXUNecMEFaz/YCqQ1k/z1119Dd9hhh4X81FNPAQD++OOP0M0444whM7B+vvnmy2awGdBILVqqhWNdbLHFQnfHHXeEPGnSJADA8OHDQ3fFFVeEzHPn7LPPDt1ee+0Vcj3bZbSESsHyecBr/uWXX4ZujjnmSJXTxq+6v/76CwDw5ptvhm755ZcPefbZZwfQWHtW768ff/wRALDTTjuF7tJLLw15ueWWA5D/mlWiXNuNpo5b//7vv/8O+fXXXwcAdOvWLXS6P1pyXWzRMcYYY0xh8YOOMcYYYwrLNFtA1LKyp0bIf/PNNwCAvn37hk4j0O+++24AwIorrhi6WpqL61UZuZxp79tvvwUAPPLII6H74osvQt5mm20AACuttFLommOuy6Iyss5lypQpAIAXXnghdLfeemvIdAnQhQUk13+22WYDkMxk2mqrrUI+5ZRTQp533nkBJOdfTWVknUc5+ZVXXgEAnHPOOaF78cUXQ2aGhI6pTZs2IbM+y0UXXRS65mTtZFFZN20v0p0AAA899FDI3Ivrrrtu6PRe5FzUNdCc7tH1qhyclsmhej1TnnvuuZD32GMPAMCSSy4ZunvuuSdkukaAZGYIqdf6qU7nkuayqVQlujlunqZWRtY9P2zYMADA0UcfHTrNdLvmmmtC5vlQrrYM3TzHHXdc6PRzma2r82zO+ZlFZWQdy/PPPw8A2HrrrUOn99+qq64KIH1v1YJqKiPrPtLz/+uvvw55l112AVD5mus1ee2110Km61jDWTbbbLOQK10XV0Y2xhhjzH+KqoKR9Ymt0huHvj1ee+21AICvvvoqdBoMyCf9Rg/6Kwefemm5AYCLL7445LvuugsAMPfcc4fu008/DZlvjGrRyQNdP61czbenhx9+OHRq0ejTpw+ApJVGA8u5b9Ricvvtt4d88sknTzWGWgXm6ZurvhnoG8Wee+4JIFlZd8MNNwyZ6/bTTz+F7oMPPgj5wQcfBACce+65oWtOTaFaoW9fDOAEgOuvvx4AMGTIkNB98sknIfO6nH/++aHTmkEHHXQQgOZZcfJA94zW7CI6/vXWWy9kWuI0APnRRx8NmW+s+hlZBI7q+mkwP/cwgzaB5NvvY489BiAZ7N+jR4+Qub/XXnvt0M0wwwxTfX616PX9/PPPAQATJkwI3csvvxzyb7/9Fly1bVUAACAASURBVLJazNLGxPtSLQIjRoyY6rPmnHPO0DVSYK+elUR/C1dbbbV6DqdJpFlB9czW848W7TTLoaLn77PPPhsyrfhrrbVW6FxHxxhjjDGmDH7QMcYYY0xhqcp1pXVE2rVrByBZylpNrx999FHIAwcOBAB07949dNrAs5HqHzQVNdNyrgcccEDqv998880AgEUWWSR0O+64Y8gM9m0kc6sGG9M1s/rqq4dOx7/DDjsAADp06BC6H374IeSdd94ZQNJdxL8BSsGIQO2uAV1HrGEBlEr9A8AzzzwT8u+//w4AOPLII0OnMk3PXCcg6YZjYKWac9V1kFWQIZDcZxrgf9ppp4U8aNAgAMlGuRoMSveCBlPT3QwAW265JYBkvaPWeM+WC3bVc4loskAWc+W66d5QN8+FF14Y8vfffw8g6U7T0ADWD5pnnnlCp/ff7rvvDiDpTlY35QILLBByNXPVufAsvOGGG0Kn7lT9rSC6Jvrv+++/PwDgpZdeCt2rr74aMt14Wvsqb3Quiy++OIBknS2eqUApSLmR2o5wf+o+07N0iSWWCLmSm56flVY7BwBWXnllAMnaZNXO3xYdY4wxxhQWP+gYY4wxprBU5brSSP20tgZq7lXT/yqrrAIgmWmVFmlfzrQ8LV09UTfBzz//HDJLe6tp/6abbgp5/vnnB5Ash66wE7ZeyzxcA5rpoWbymWaaCUAyO4prCpTGzXpJAHDggQeG/M477wAA+vfvH7pjjjkmZM1KqGaN0+qMnHfeeaG77bbbQlaX6/HHHw8AOPTQQ0OnGSwcU/v27UOndR74uR9++GHoevbs2cJZNA3OVd3JWltk6NChITNril3WgaQZnfPT9aO7Cyi19sijVUdW6F5hawG952aZZZZMv5/3jGafaMd1PUuZ1aiZnDT3A8Bcc80FIGn61/mx5om24NG1VtdVNfyr9lXivwDw3XffhawukUrQ5a/ZSSNHjgyZ7T4OPvjg0OWR9ajoXmI7A7Z6AJLry/piPGeBxvmtU3c8W+EAyd/3tO7jaZ+lc3788cdDvuyyywAkf//sujLGGGOMKYMfdIwxxhhTWKpyXaWZA9VEqgXX1Ix/+OGHAwA6duwYOjVdMkNAXWNpZlgtSJSHa0fnqhkAjEbXgko6vgceeAAAcPnll4dOXSfLLrts7QfbRNK6BAPA+PHjQ6brTUvkKyynr5lIWjyRpsntttsu9XtrZaZNK3iopfzVHdWrV6+Q6bJSF1qaGVZNq7o/eV9oET4tWFdLk+y/P+fKK68MnbqrtBM3WxxodpvOj+NX19yss84asmbz1ItKxQlbch31/FKXH7N29JroXtdzp5qipmn32nXXXRc6de0cccQRIdO1qmdGc+jUqRMAYMCAAaHT61evs7RSO5a0TCz9986dO6f+PV3uebt7FB0L143ZcUCpLQRQciN26dIldHlnNXL8b731Vuh0/9Gdquj6qcz7TjOt+JsClFrP6N9Um6lqi44xxhhjCktVFh2FT9RaW0QblekTKYPh1OKhT3dsqqhPvKw9oJ+lwaxa7juPp1+1fhBtOqfByqyzo2+RGjg388wzA8jnjUS/U9/oNfCU1jmuE5Bsh3DmmWcCSNZW0LXmm0qlYPNq0TeCN954A0Byf+q/77333iHT0tOctwgdP/9OrXx6fdQiUs2bir7Fci/pddYAQd1ftEioNUI/i2PS2it6f1HOugVE2lt+udoqzUlWSCtnf/XVV4fM1gQa4MtGi0DyfKlVsDyt39roVwP4TzzxxJC5P8s1VeVZqi1aaEUHSokfWZ8vuj60UrBGFVC+6SqDcdUKnBbYrHWgylmHGpmsz79q0ev466+/AkgmEGnttLQG3J999lnodF9zXTXBRVuwMNi8lr/jtugYY4wxprD4QccYY4wxhaVmrivy3nvvhay1Y9SMyWDce++9N3QajEzTOet1AMnAUbofNMBTgy3Tap5kgZrWttlmm5BpetOO5DTHAiXXTt++fUN3yimnhMwgr7zNmdrC4ayzzgp5iy22AJB0h6gLhoFpGoys7posXYu6z7SE/pNPPgkgeU033XTTkNdcc82Qq73uDGLWOhEqa82aalxXOk4G+GmAoLpm0oKhy5n76WbV7tJs2wGUurdn0coizYUGlEr8a5dqbRujbm7uL3WHpH2utjXR/U2Xreq0zlc1AchKmjuGnZuB5FmqNZ/Sul+PHTs2ZLbz0No4rK0DlFojZNmKBEhec7or1MWv63PnnXeGzE7e999/f+j0vubfqWtdPyttfzciev3190FbI+SJXnOen7rPdH3470BpXpqMpO1o+Ln6O6DJIOWC0KvBFh1jjDHGFBY/6BhjjDGmsFTlulLTIDM0tKx8OTMl0ToW66yzTsjsil0uaps88cQTIbOjLZCsz5O1eZaoOXmttdYCkMzUYMdgoFRi/dxzzw0dM62A/GsmpMHaG0DJzfPYY4+FTku7L7/88gCS5vJ6zUn3lLo+WT9H691oVou66VrimqiU9ZG1G5LXX8v6jx49OmQ1jetaEb1uvFbanXjttdcOmXu9Vi4coHTNdBzvv/9+yPvssw+ApAtw4MCBIasbknVUtIS+uvSY+aOuK10fZnPqmZTFOaLfyZpG2iVeO31r65LPP/8cQNIdrK5X7nFtW9G1a9eQs3Tp6GdrOAK7c+ue0bXWs5DXReendZyI7k+FGYia6Up3q35+XvD7tQWEZuCy/laPHj3qO7BpwPM7bR2AZDYdM6Q32WST0GlNPLYA0qzUZZZZJmQ+K9RynWzRMcYYY0xhqZlFh0/v2vRL/z2t9oTWWTnkkENCZmVgfWLXv6clR3XlakrUCw2gYhDWCSecEDo2CgRKtU7ysDxVIq3CKAAMGzYsZNbP0QBNtUix8rE+xe+5554h1+uNKq3OitZG0TeqtP1Tbpx8E9XP173Ke6Hc/swCjlnnp2+Eur84Fg1QZp0MoFR/ZcMNNwzdCiusEHItLTmE49egRW1qyTpOasXRwGQ2cgSSlqx/fz6QnDdhNVYA2G233ab696z3LMdHazCQbJSr9yLlcoHnbNqq/651TrI8a/Se0GrqTz/99FTfrWNmAgdQGr9aBLSKN6/V22+/Hbq99torZFoXtGmyWocaxaKz9NJLh67S/syicnwldK2YgKJngpJW+Vn3n1rvTjrpJADJ+1STeSqdvy3BFh1jjDHGFBY/6BhjjDGmsFTlulLTEgNsDzjggNANGjSo9EUpNS3GjBkTOi3BTzNXWu49UDKpaR2FvJt6ajAWG/BpiWt1/ay00koAGsddBZSur9ae0QBINjoEgM022wxAcq3VTL3vvvsCSAaL77LLLiGnBaZnQZq7VMepdY60kSpN6uWaJjLIURvVajNN7ttyrr0sTc9NcSvRjapNZRngCpTqt+j+VTckvyMLF7GumboD6XrQoFI1o3///fchsw6Ljk9rk3Tr1g0AsP7664dup512CplBvPU8U9L2hNYDS9uLeh9p4P3jjz8OoBTADSQDk+vlutI9xcQSnZPWZmIjXQBYdNFFAZTfX9Sra0+DrbU1TWuh0dtBcP+lJTKUI63Ol6Lu8KwTV2zRMcYYY0xh8YOOMcYYYwpLzVxXNG1pPRt26QZKufNAqYWDmtk1q4CmVf18rVPD79pjjz1Cp51sszTNqjlV6wSp6XX48OEAgPvuuy90PXv2DLlR6uSomZluHK2DpB232bYDKJmJtfYDM630czUTol7uKl17rc3Qu3dvAMl6TDvssEPImu3CrCmtw6Lrzlou7777bug0a4l7VWsnaYZhvdZfx6xuYNYP0gxJNUlzrdKyP7KC97rWM+rXr1/IzMDS7Cjdv1qzhW5CdZMwkwcotUBQ16KeL1yfvF0IldwZuo90/jyX8sg00jHpmcyaYpoJqFmpuv/Szn8lLStH9w3HoL8pjdQOgmPR80Ndw6xDozTKXmzOOMr9VjIbTuesvw9Z/H7bomOMMcaYwlKz12w+hWmw4JAhQ0LWt0NaCkaNGhW6tIq2GuC73nrrhcz6AxrgmVYzpZbw6VSfTA8//PCQtaYHA3ezrj3SEvQ6sZo1UHrL17eMM844I2QNFmMdIDYPBErVPAFg4403BpB8e6tXHQh9o9Q3dloEdM4aTKxz4TUq90bNf2cAPpCszNulSxcApaB0IGldyNuiR4ucVrtmNXKgZInUyryrrbZayEwCqGXtKl5fPQe22267kOeYYw4ApXosQDLAWN+Cd9xxRwDJKq5zzjlnyPyOcnWO8n57bip6pgwdOjRkvimzuW490bdxrY3Dysh6/qQlmDQFro8mo6hF9q677gIA3HzzzaHTKvVqPchjrXmv6Fmkc9G92popl6xD74GuWdbrYIuOMcYYYwqLH3SMMcYYU1imm5bJaMqUKVXZkyrVQaj074qOs9pgwbZt204HABMnTpzmB+g4aCZWd9XIkSNDVjP/UkstVdX4qqV9+/bTAcCkSZOmGoAGXd55550hs6bFfPPNFzq6YIDkXOky0KaHW265Zcg0mWvZ9lrSrl276YDm7U+updbR0Tof48aNCzmt5oPC/aem+SWXXDJkNlVsqTuE+7Pa+0/R709zo2pgIBtLqutN6wRtv/32AFrurmrq/NLcHOWap1Y6H+pZpySL9VN4Lb755pvQqZu4e/fuAEouHKC2Ncc4v3/++SdXH5/epy+//HLIbFfAtiFAqa3Jv/VpLrM2bdpkOj+un9aR4z0FANdffz2AUnNkoLahD5zf5MmTM10/XR/W5gJKiTlsHgyUwh2A6vfn9NNPP9XBZIuOMcYYYwqLH3SMMcYYU1gyLW5SzkTcWrIa1Kx5wQUXAADef//90KnrRzNw8s6qmRZqAtWaJKxJpLWPNKvl/PPPD3mDDTYAkHR3aB2LltRcyBqORWuLqJuuJW6Yclk7jdTag+j8NAOMaFYks51efPHF0Gl9LGYeatuWLKhn9/fWiLpbNRuU9aPUTd1I92Kt0Dmpm50ZkKzXBiT3fN7Xgnt5scUWC90zzzwTMmsKNeI50lL0t4IhD+ruzxpbdIwxxhhTWPygY4wxxpjCUp+6/K0INfFriwMWJ7zqqqtCVyl6vxFRs61mRV199dVVfVZrcS3oOrWWNas1aaZ71dHMPHDgwNCpa6Bz584Zjs40Fe1IrjJb5DRS24Ms0DNH27VoBmza/9uIZ5UWv21E139L0PNV3XSPPvoogPr+ftiiY4wxxpjCkmkdnUalJXV0KpWNbySmVUenCLSkjk5rIus6LJXgmaB1bLRsPvd9Sy1iec8va+o1P00s+Pbbb0NmHSe2zag1jVJHJyuyrqOTN/Wqo5MXrqNjjDHGmP8UftAxxhhjTGGZpuvKGGOMMaY1Y4uOMcYYYwrLNNPLix4sWPRg3aLPz8GCrRMGC/7vf/8r5Pz+7//+7z+xP4t+vvj3r3XC9VNs0THGGGNMYXHBQGNMIdH0eKVRy0LUCpbF0PIYRZ9za0f3alEKBiqVildmPVdbdIwxxhhTWDK16OhTKgvuAelPd2nloP+rJfqNMS2HhfS0hYueOSy3X87i01rQ8etZyU70EydODJ22SODfFcli0JrgXtTr/+uvv4bMtdLu8wr/rtF/H8vtTy10ScrNtWZjyfTTjTHGGGNyxA86xhhjjCksNXNd0RynLio1x40aNSrksWPHTvX3s846a8jLLLMMAGD++edP/a5GN7k2tWtwo8+jEjpPNVOmuQTSXJMOkKw9ev9Vcs3omqhpuTXuS+3F9cEHHwAA1ltvvdBtueWWIV9xxRX1G1gGcK5ff/116IYOHRryJ598AgB44YUXQte/f/+Q99hjDwBAhw4dQpf3mutZkhZMnYaeH3mPvxJpgeG33npr6HR99t57bwDJ3z91Pa6zzjoAkh3B8zhL0wKoVT9+/PjQXXbZZSF/9dVXAIBFF100dKeffnrIWczFFh1jjDHGFBY/6BhjjDGmsFTlukozx5177rmhe/7550N+//33Q/79998BlDc3LrnkkgCAk08+OXTbbLNNNUPNhHKuG85rypQpoVPXAE3PGmnemtw4nMsPP/wQumHDhoX83HPPAUi6UWaZZZaQ6VLYcccdQ9dUd18l9Ds1up/XV9ep3HeqG2RalHP9VMoqzAJm2jz22GOhu+uuu0L+/vvvQ+a1WHPNNUN38MEHhzzbbLNlNs6s0PuHbnLNOlpxxRVDnn766QGkZ380ErpXub4A8PLLLwMALr/88tDdf//9IXOv6d8fddRRIS+yyCIAgE033TR09crgKXfPTZo0KeQPP/wQQMnFAQC//fZbyDw31R05wwwzhNyIbixdC64fXVRA8reALsXFF188dOoGOuusswAk3V3LLrtsyFn8lnDddJ0GDBgQMtcMKN1fY8aMCd3rr78e8rzzzgsA2H333Ws+znLYomOMMcaYwtJsi44+ketbAIPhzj777NBttNFGIT/88MMh841Cn7zvu+++kC+44AIAySfWFVZYIeSlllpqqu+vF/pkrk/O33zzTchPP/00gGQw4N9//x1yu3btAACHH3546HR+jWjd0TGdf/75AIAzzzwzdJ06dQq5Y8eOAJJvZF988UXIvBbbb7996NQS0xK4l9RyeOONN4ZM62KvXr1Cp2+BGphJi1P79u2n+nygtAd++eWX0D366KMhM5h+s802C92MM86Y+lnVoHvx3XffBQBcdNFFoVtggQVCnmOOOULmPfzII4+ETi1yDz74IIBksGMj1uzQs0gtHiNGjJhK15oCV4lahDWA+tRTT53q3xdaaKGQaSn97rvvQvfEE0+EfNtttwEoBbUC9bOI6Jq98cYbIffr1y/kJZZYAkDpdwJInp8ff/wxAOCSSy4J3bHHHhvytttuC6B8sGy90O9XixTX78ADDwzdoYceGjKDdNu2bRs6vW60uD711FOh69y5c41GnQ7n8vPPP4dOf7M12YjomXPaaaeFzP2pezbr3zxbdIwxxhhTWPygY4wxxpjC0mzXlZrjfvzxx5CZJ7/11luHbuDAgSGrGSvNjEhzI1Ays7744ouh03LuHEM9zdH8zr/++it0Oj+tDbTaaqsBKJkoAWDOOecMmWZWuhuAZLBko6DmUjUd003ZvXv30N15550hM9hMA9d0ffi51ZbgV3fXe++9BwDYeOONQ8egd/2u0aNHp45Jx3LGGWckxvnv/zctsDktgFnrYKjLrFZuIN3/yy+/PICkO0prbygcN10AANC1a9eQeV9rMGRrorW4ptLQoFR1TXBPAqV179u3b+iOP/74kHlfquuO7magFMy67777hi6L/alwz3300Ueh22effULu0aNHyIMGDQKQ3L86JrqBBg8eHDp17W244YYAkr85ebteL7300ql0uibqOmSQvAbL61nz2muvAQCWXnrpmo+zHNxz88wzT+jOO++8kA844ICQeX7ob7q6Fknab8K/9bXCFh1jjDHGFBY/6BhjjDGmsFRVR0dNTIwQP/LII0M3++yzp/4da3pcddVVodOsD5q+1E10wgknhHzhhRcCSNYOKJcNVSv4+Zo9pGXXdfw0+au7R02rdMNpHZlGNLerOVFN55yXuknUzUGTa6U6QdXOWdf87bffBgCMGzcudJq1wAwObTWiY/rpp59C5vjV9cbaEEAp6+HPP/9M/XuOq9pMsuag4yPqulA4Ls1Q03t17rnnBtCYe7Ip1KomUz3hmLX2j2by6V6kG0ZdAyuvvHLI3L+6J9Zdd92QmRmr59daa60Vcq2yldLqrN10002h+/bbb0NW1wYzFHX/6mfRpTVhwoTQaRgFzyd1XdULvXaa9abX+vrrrweQ7q5S9PzQmmUMeTj66KNDl3XWEveBjkn3lLoemWHN7GMgmSHIfVluzPyOWoam2KJjjDHGmMLiBx1jjDHGFJZmu67UnDTXXHOFzOKALPYHAEOGDAmZRbwA4KCDDgJQypQB0osjrbTSSqH79NNPQ2ZBt+OOOy50WnyJBfn+Pd5q4Oeoi0YLwmnxKn7/5MmTQ6fR6swaGDlyZOhWWWWVmowzK1hEDiiZEe++++7QqUuIpsfNN988dJp1Vit0bVmk8JBDDkkd05577gmg5JYBkuZQzZqjmVXdUfpZLOil67fJJpuEnOZGyprmmHb5/2q7CHa0BkrzyztT5b8EXR7akfzee+8NWc/HmWeeGUDlgmvqDtGszu222w4A8MADD4SO9wcArLHGGiHXyn3Jz9H5qetez9W0fadZjXTpXX311aHr06dPyLxX8y4SqG467dS9+uqrAyjfgoSfoa477e7N1i2VMpmzQL9H3VgapkC0SKKuKc9NLWKp/77++usDSBbRrdYdbYuOMcYYYwpLsy065Z7o2DTuyy+/DN0rr7wS8q677hoyA0P1iU0tPrTOaG0ItY4cc8wxAJIWHX2iVOsOn46rfeLlG5O2CtCy1vr2z+9SK4AGezKwVwOs2QizkdBgaoVlwLV2zq+//hoyA+f0jVSD8Xgtql0TfQtYddVVAZRqGJWj3HfqGyXfHsrVdqBe90LadzSSRUTHzzdNbZGh1h3ea3pP5V1Ov+jwfHn22WdDpwG2aj1nEocGIFcK9td2Jvw7vSdZmwWofA9Vg45T60ypxSot8PXzzz8Pmee/BrjqZ9Ginvc+1WBrtb6kBePqWHn/qXdEz5K0mjR5oGNOOx91ffbff/+QmTiiLYL0rOX5pMkubAsCtMxLY4uOMcYYYwqLH3SMMcYYU1hqVkeHtQ+0i6p2ZGXtHKBUf0RLfNMcCZRcUlpnQKHpVoO1tGuzuslmmWWWqcZaDWo206DnBRdccKr/t5xpj2j360ZEg2ovvvjikNmpfOGFFw6dXpfhw4cDSAZrs3s4AGyzzTYAsnHttHSdm2MO5bi1NlLaGNR1mXdgpLoW2QJA3Rl6//H+1PU799xzUz/XtBw113N9brjhhtBpAC5rNwEll7+6e9LuJf18dSOwDYOer1nUJEtrsUIXM5BM4FB3GVtYaB2ak046KWTu20rzzxt1N/JMBEouGw1G1k7u/P1T17G2C+K8s66d0xzSAqtVp4Hv3NfqbtR2PQx9uP/++0NXbc0gn1jGGGOMKSx+0DHGGGNMYanKdaWkZZqoaVzNpOy0rLUBtI4MTXbl6gywjoS6yV544YWQ62XSa873pLku1BzdiKjpe9NNNw2Zc9E5qZmd66euPa2jk3c2REvQDBBmCD755JOh0zmxg3qXLl1Cl4eZWb+TrmWgNG5tN6B78c033wQA7LbbbqHTTtks/V7u/sybtGute7lRWkToOGi6HzNmTOjUhaSupfnmmw9AZXdNuXYE7Iq+5JJLhk4zuLKAc91rr71Cpy1UdK9169YNQDLr85xzzgmZIQvqWm0UN46uCevFAck6ZHQ96v3D2jpAKavqrbfeCp2GOZTL9qw3ur/UJck6R+ou1d8Hdq1nPSAA2H333af63Fq6I23RMcYYY0xhqZlFh+ib76mnnhqy5v7zTV+tPPpGXOmNjE/6fDMBklV4y31uvUlrageUnnQ33HDDuo+ppVR6Y9J/Z8Vk1WnNpNZi0dE3Fg2WO//88wEkK39q4GD//v0BJIO587Z+6FxYk6KcZYPWHVpOgWQwpDbzyxPdR3r9l1tuOQDJOevbZSPuP65FuUBv1Vd64+UZrBa78847L2TWOlOLulYMz8I6ws9kcgiQrCOmtc+4Vmq50MBe1hzT+6tnz54hM1g373tOkxG0Mj4tVlqHTf+d1ldN4BgwYEDIWl8nT9Tipl0MWLFa60AxwBwoWQ/139P2XC33oS06xhhjjCksftAxxhhjTGGpynWlpm+aCdVcqAGQaTVxKpmm9PPVJXbllVcCKAVNAqVGdUB6g7E8SHO3ASXTMevJtFbUnK7lvIcNGwYAWGuttUKngemN6DqoxOuvvx4yy+XrPLbYYouQ6RpqlADJf1Op5grROld6Lzfi+un46Vrr2rVr6LQmx/bbbw+gFNQL5FOHRa8j26LomF966aVp/r3OWe9Fth5Qd8ett94aMt1AumfrhV5nHb+2SKBer4/+rlCvc9bfl0YJNr/88stD1hZBt9xyC4Cku1CvC+WllloqdNoiiC149JplfdakXVOtg6RzZc0tBh0DybOEn/Xqq69O87v0/qz2zLFFxxhjjDGFxQ86xhhjjCkszXZdpbmrgFKLAM1EOfPMM0NOK9ddzvRK/V9//RU6NY3RJKuukZ122in1sxqlNLiOg9kQCyywQF7DqQquj66/tvtgOXtdf3UnNqpLh3B+WudDWyAwA0s70h988MEhM1uk0eepqGuY5eg1a2frrbcOuRHnpfcXW5Ooa0ZbCNx2220AgCOOOCJ0WuejXtk6ad3FtUvzyy+/HLLWVGGdEm2b8M4774TMcvmjRo0KnZ61hx9+OIBkHZ081rRSixylUtuRvN2pHN+ECRNCd80114SsWW/MsNIWRmmfpW1bdP66V+sFr6/uIz3T1TV35JFHAgCeeOKJ0C222GIh8zO0O7l+7iKLLAIA6NOnT+iq3Z+26BhjjDGmsFRl0dHaIvfeey+AZAVHDbZdaKGFQmYtBX0iHD9+fMhjx44FAJx88smhe/zxx0NmldAzzjgjdOUCu/JEn8LHjRsXMoNZtalZo5PWIFDrQGhTOr5xajBdI1oBysG5ap2HTz75ZKp/12rRyyyzTMh5v102lXJ1go4//ngAwB577BE6tV41yv1VDo5PrWxsZAmUquyq9TltLbWOS9Zryrf0nXfeOXTXXnttyGopYBVhDfBU6yPPYA1W1c9i5W5d/9ayZ8uRdwAyv3/y5Mmp/67WM/6/5SwztI6wuScA9OvXL2T+fuZ9pmrtI022YRB9WiNPoDR/XTOtiUTvjdYWqvbMsUXHGGOMMYXFDzrGGGOMKSzNdl2puUwbATKnXpuzadM1LYfNEtCqe+WVV0L+5ptvACTNWQx2BkousbzrYFRCITcENQAAIABJREFUTXNqWqYbK29zayXUtK1uHLY40GCym266KWSWY29N5vA015wGE6pploGjDLoDknu1EfeikmY61mB/JgEwaBVoXWvJsaprZ+DAgSEvvvjiAIBnn302dNriYujQoQCS1yfr+fPzO3XqFDp1vbH2ClBaH03W0D3Hcvzq+qe7Sr+rNa2pwt8gnbO6IfOYF8ekrSrWXnvtkLUp7mGHHQYg2fT3ww8/DJnB8ltuuWXo9Hc1T5eVXlutXaTtROhmHTx4cOguvfTSkBmEre5wbQfC/esWEMYYY4wxTcAPOsYYY4wpLNNNy8w3ZcqUadoA1bRL+fvvvw+dmv4fe+yxkBmZrvUqNM+eGRDakVyzqvhdLXURtG3bdjoAmDRpUqY2Ts0qGzFiRMjs5MpS7UDS5Fmt66Ndu3Ytnp+u6Q8//BCylvNmBsuQIUNCt8Yaa4SctWmV8/vnn39qtn6aFTBy5EgASXOzuh7ZQkBL7OtaV2s6b9OmzXQAMHny5JrNT9eV63fHHXeETmW2C+jRo0foaumOm3766acDgP/973918zHo/DkXzZDRf6drsqXr+H//938t3p/l6pRpVlza/aVjZbaYuhZq6c7h/sz6/FS0ztPw4cMBJN1x6ga68847ASRdI805k3i+VPr9S0PXb9KkSSGzbYP+P+3atQud7kWOVbOOdP7VrmXWv3+cn15zzcamXs9M7Wqf1gKkOXD9FFt0jDHGGFNY/KBjjDHGmMJSVS3ptBLemgmlBeVUrubzgfwLJTWVchHqLHjYiAW7dEzMfgOSGSArrLACAGDeeecNXWtZE6Wcmfm6664DkHQtLrfcciGfcsopAFpnphVQyirTTLKHHnoo5I4dOwJo/Dk1B72/uMe1IGC5/7fe6HerO1WL/zWV1nhPlkP34vLLLw+g5KICkp2w9V6uN7p+6ppKa/ej/29aBm5zWmQ0Emnd5eecc84m/c2/5Vphi44xxhhjCktVwcitlXoFI5cLLGRgmr6l1TLYrJpgZCWt0SpQelPM6y0ji2BkXZ/PPvtsKt2MM84YMtuZ6JrVkiyCkRWOW8evc83aEpBHMHI9qSYYuTWQRzByGmrx0rOKe7ml+7iaYOTWQL1+//LCwcjGGGOM+U/hBx1jjDHGFJZpuq6MMcYYY1oz08y6KroPveg+2KxiPPKGMR5FX7+iz+/vv/8u5Pw6dOgwHQBMnDgx1/mVi7GrNpuuffv2/4kYD5+frROeL4pdV8YYY4wpLFXV0cmbcnVoNNq+0TuEt0b0uqdRpPodROdcxPkVnbQ921rrlFSCc/36669D9/nnn4e87rrrAiju/I35N7boGGOMMaawtEqLDq00v/76a+jatm0bslakNLVB34jZ9PLyyy8PnVa+nn/++QG0/rdEtdzoXpt55pkBVLZsmXzROkFPPvlkyK+//jqAZLXvWWedNeTWuG/Vcs0GkSeddFLott1226n+X1smTa3Q/cf6RqpL87jo/sv6nvNJbYwxxpjC4gcdY4wxxhSWhnZdqTlLzdCjR48GAPTu3Tt0Z511Vsi77rpryFraPkt0fE0NfCxnOm4U07nOg9ccAPr37w8AePHFF0M3YcKEkG+88UYASXdAazKTp7lGu3TpEvLQoUMBAOuvv37oGnF+5UzHlQL0+e+VWoA0atNPms51z55wwgkhjxo1CgAw++yzh65fv34hN8r9V4ly63v33XcDSLZI6Nu3b8iNum4kbX/qXkw7X3VOjXgvFpFyTZFffvllAMDvv/8eunnmmSdkNjhliAOQbJCcxfrZomOMMcaYwuIHHWOMMcYUloZ0XdEMqybKKVOmhHzJJZcASJq7Vl555an+PmvUdPf222+HfPPNNwMA/v7779D16NEjZI61U6dOodOsMZqcy9UGyhKdk17ziy66KOQ333wTAHDiiSeGTl07WXX1rjc6jz/++CPkM888E0ByzzETC8jH9ZHmbqr0/5bbXz/88AMA4IUXXgiduik333xzAMAGG2yQ+vl5zF+/k1lVRx99dOjUjcX7Tt05rRFd63fffTfkQw89FEDJhQWUd3PliY4pTVZ3yDvvvBPytddeCwCYb775QnfIIYeETJdIo7voytXmopyWyaT/nuc5AwB//fVXyAxnAIAhQ4YASF7/GWecMeQ55pgDALDOOuuE7uyzzw55rrnmAlDb+dmiY4wxxpjC0pAWHT416tPjTTfdFPJdd90FABgwYEDoOnbsGHKWT/L6ZP3UU0+FvNtuu4WsQaxk2LBhIXfo0AFAMhhSrQN8O1liiSVCN9NMM4Wc5ZO8vmXccsstId9///0hH3fccQCAI488MnTt27cPOe2NRK0jHH+lOguNCi11eb8xpgUIf/fdd6H77LPPQl544YVDpnXqxx9/DN3zzz8f8oMPPggA+P7770On1j2u1XrrrRe6PGoK6Xfq2/8555wDAHjvvfdCN8MMM4R82WWXAQAWX3zx0P3zzz+ZjbPWcN11/1133XUh01K19tprh65RrDjl+O2330J+9tlnAQDXXHNN6N56662Q//zzz6n+/ttvvw158ODBAErnLNBYZwrXT+9V7kkAePXVVwEAW265Zeh22WWXkGebbTYA6Wdq1uiZ8/PPP4esv4UM7F9zzTVDN27cuJAZrKy/iSNGjAj5ggsuAFCq4A1Uv3626BhjjDGmsPhBxxhjjDGFpSFdVzTJPffcc6E7//zzQ6ZJVk179TLdqblczW3jx48PWU2mRE1vNL1qsLI24HvkkUcAJIN9DzvssJCzcJlwXmoW5jiApJuCrqty7gq6OTSAUINZOW8NYNbPZ7Aa0DgmZzUT77vvvgBKJmQg/8BA1qzYeeedQ6f7U11XNJmzVQCQdD3SDdyzZ8/QqWm6TZs2VY+9Fuj80/aarpkGS6666qoA6ldjq9bwvtNgcXUtP/zwwwCS65S3m5Xomvz0008h6/n20EMPAUi6S7XmCt38ur/vueeekBmE3q1bt9A1yjkClPbtl19+GTp1vbEOHINyAWC77bYLmb+FmuBSr/NHv0fPaW0HRJeTuov1XuX5qcHye+yxR8innHIKAOCBBx4IXbVnrS06xhhjjCksftAxxhhjTGFpGNeVZjOxDsZee+0VOjV9sZy7ZiLVyzSp2RlaNl7NpPfddx+ApLl43nnnDfnTTz8FUJonkCyXTZcCWykAyZofmo1VK5M0r+/7778fuldeeSVkZrIA6XVy1AzLOjPM3gGSLjG6SbQOUp8+fUJWMyg70efhGlLXotbsYP0Hdd01imuge/fuIWudC4Umb83023HHHUOmm4vZjUDSNcI1qdRKIgv0O/Wa6/7k/XP44YeHjrVl/v0ZJM0N24juDgCYOHEigJKJH0i68VdYYQUAjTV+nhmakarnp+4vnm+aNXbQQQeF/NJLLwEo1fMCKreIaCS4Luoa1vuWLp9ffvkldMcee2zIPJf1/q0Xeg5riMYmm2wSctr9pW5yuizfeOON0Om9zJpytXSRN/aOMMYYY4ypAj/oGGOMMaaw5Oq6UhMjy84DwMCBAwEk3TnqzqDJLg/TrJru1PWi5fDTSuOrOY/ZHtpx/eKLL57qO+gi+Pd3ZQFNh08//XTotO2Bluvmug0fPjx0xxxzTMg0Las5VgteMdtKs150/lrOfrXVVgNQX9cV5/fYY4+FTl2WjdLiQvc/O8WzPQqQNH3r/ptlllmm+bm81upOUNPyiiuuCCB5/+bhWlTXmhaUY9bYwQcfHDptscKxqmtPsyZ5fSpdp3qie44u708++SR0mpXK4nvq2te/r1cLgbSClkOHDg3dk08+GTL3LwAcddRRAJJFWNWNMXLkSADJOWmLAc6r0Ysk6vXRs5737WmnnRY6nf+mm24KoHwLiXoVlC3XvZzFAbVgqWZQMbRDf184JwA444wzAKSvaYvHXdVfG2OMMcY0MLlYdPhUqFYcDUxj6XaWggaALbbYIuRGeVLXcaQFo+rTr1oEWP9C35j1sxiMt99++4VO60jUOgBZP1OfwtUio8G4LFGuNVu0JsZOO+0EIGmx0vGnfb8+vWtrgjxKm/PtRK1bWtOjEeF1UitZOYsE17pcMPG9994LIGklWWONNULecMMNp/r7rK2rvJe0VcAdd9wx1b8DJeug7lmFQaznnXde6L766quQGQSrjWzzIC0AGQCuuuoqAEmLHRM0gNJe0BYXKvOs1f2R9fpxX95www2h03tqqaWWCpnW37RGlvr/ajDshAkTQqbVSBtFZt3UlNalcmezWp/S7js9P7k+ev5onaAll1wSAPDxxx+HTs/PueeeO/V7q4Fj/vDDD0OndXBUz3GNGTMmdPr7x3Xt2rVr6Ni2AyiNv5Z1rmzRMcYYY0xh8YOOMcYYYwpL3VxXakKjy0pdMxqAyk68vXv3Dl1ad+s86ng0BZrmtGaEmsmvvfZaAMkALi33fdtttwEAlllmmdBl7cKhGVnXoVxtBI5fO9Kee+65Ie+zzz4AkgF2aWZI3RMabK3tCvKA5mcty64m6UZxnVainDuCa6lrqp3Kb7rpJgBJ18Jaa60VMoNc80gG0ABiXR/da3POOSeA5P669dZbQ2awo7rO1fVF193xxx8/1WcC+bhT1TXAFg/a8TrtXn300UdDd9JJJ4W8ww47AEgGANcL3VPl2kHQTaOuK23xwflrba60dihZo/uALn+2rwCStWN0fbhXNZhcz0/uS20BpK4ptrjQtizqrmILCZVrda+yszyQTBDS0AS6jNlqBQDmmWeekLmW6hpX19z+++9fk7EqtugYY4wxprD4QccYY4wxhSVT15WagzVbgmY67U6urh12P1XToMqN6LJSMyzbIWgmBGsHAKXrot1fhwwZEnKnTp0AlJ9/FvCaaiYGs6sA4Oqrrw6ZrR204+zee+8dMt1Qai5NK8v+zDPPhKxmdJawB/LtCq61Vxq9rHxLUHcc3VVAqfv3KqusEjo1J9OlkIfrSrusa1uVBRZYIOTFFlsMQKnVCpDMAGSGjn6W7jO6HLTFgLoe8ugUTXcxUOpkrS1ytM4KzyLWEwKSc826JlcaHNP6668fOrroAeDnn38Oma5vnb/ei/ws3X+LLrpoyAcccACAbOo86TmvrqdtttkGALDmmmuGTuekrilmIGkmkq4fs8q0RZC6eRjSMGzYsNDp+anrW+t7VPcca5wBSddUWh2qtGeBjTfeOHQMVwFKNXX0nq52HsU7vY0xxhhj/j81t+joE682RdQqj3feeScAYNCgQaHTmixpb4yNYsXRcehcNQCOVUq1zoBWKd18880BACeffHLo9I2kXpU99fP5xqdv7ip/9NFHIS+77LIAkgGOOj9aCrT2x3fffRcyK19rsJ5Wds265kUa+sbx9ddfAwA++OCD0LGaMwAsuOCCABqraWIl9JpS1sDJ66+/PmQGPrJCLZB8O8ujgSn3gQZlLrTQQiHrWjCw/7XXXgudBrtqkCvRYHm+EXfu3LnaYTebtArqQLKK8IEHHgggGYCta8L9q8HUKjNYNOv9q/cu7y8GggPJuWoDY15/PVP0LOHnajLHVlttFTLrs2RRLVg/U+s0sSmxNkLWwGttxMyKz/RcAOnWi3J1xnh9ylmssvAE8HO0dpE2FU37znJ1cGaeeWYApaB4INl0lzWteM7WAlt0jDHGGFNY/KBjjDHGmMJSM9cVzWhavp/5/kCyQeLpp58OIJnvn9agrFHcVUBpfBpAxlYVQLLmAYN4NQBwxx13DJml5dX0XK+mbOXgtV5ppZVCp2XjaQ5XGJQMJOsoMNiTzT2BUoArACyyyCIAkgGwebtGFLo2dP9psB3NyI3eFkLHr/fX22+/DSC5ZzVZgC7JXr16hS5vNx3vCb2ndM+p64M1Z8aOHRu6NDO/3suzzTZbyDy3uE+B/BsIq5uXrgt1nev5ymamW265Zei0ZlkeTS95/dQdpS02dK+xVhJdHEBy/Kyfo+6e3XfffarvzGJ++pnqTmIyjbro9XzQwHm6f5ozvjTXUJ4uZKB5LRr0/qMbctSoUan/nkXihy06xhhjjCksftAxxhhjTGGpynWlJiaWrdZMnZEjR4asdXKYYZXmrgIax2Wl46O5VM39jzzySMjaPZd/x3oQQDJDiabnvF00Cq+/uqu0Y/Xtt98e8ujRowEAxxxzTOg0k4XZdj179gyd1hRi/QQtW573tdD9R5O4dlRW10HeY20q6trQ7txcCzWz090BlNwEuqaNMmc1l7N2CQDccsstIbPmjdZeUTcVP0PdkepG4d/n3epDa6tsv/32IQ8YMAAA8Pzzz4dOa65wLfv37x+6etbkmhblznlmogKlfat11nT9+Bn6N6yd9O/vyJK066jutHL/b6PcS80hrW1M2r8rOmdtt3LOOecASGZ6brbZZiGn1ZGrFlt0jDHGGFNYmm3RUSvH+PHjQ2YePBuuAcCFF14YslZU5JNao1txtDYO35Luuuuu0OmTuQZG3nzzzQCSFhF9O2stT/Rau2LJJZcM+Y477gCQbKrHatBAKUhPK+vOMMMMU31+o14H7k+1iGjg9eDBgwEA++67b+gaZf/qmLUBplrf3njjDQBAjx49QnfssceGzIDfRlwfPTO06e2ee+4ZMmsyaZ0V/X/79u0LIGnx0GD4PIJ1//3dQHJPaf0ZWto+//zz0On+7N69O4D8ExwqUc7Kwd8VrSasZxHrJLGeEJA8t/Pct3kH7WcFLdoaYK37U88aWoq1AeiVV14ZMpMEdtppp9DRygOUqjzX8lraomOMMcaYwuIHHWOMMcYUlma7rtRc9fHHH4f8wgsvAEi2ctBgMYUmy0Yx9wOlsag5dejQoSHfc889AIDll18+dBtssEHIq666ashrrbVW4jOBxnQDpKGuj0MOOST1/6EbS6+VXheipsdGN+mmNRDUYFetmcT6Knnv37S2DuPGjQudBs4/9dRTIbNW0jXXXBM6rW/S6GtFdJzqxmAQvCYI0J0DlILM0wJcGwndk1p6nyEB5c4XzqUR3VXl0HPn/fffB5Bs4aHrw5AADfxtTXNtLeiasAWJnhkajqHJDmPGjAGQTODQZq79+vUDkKyNpDWTsvittEXHGGOMMYXFDzrGGGOMKSzNdl2pWUm7l7LselqXVaAxTcNp6Pw0k2j22WcHANxwww2hY74/kF5ivjWZU9PciWruV5pT+rs1wrU84IADQqcuWTWzNgpcNy2rrm0R1LV4+eWXA0jWTNJ931r2rY5Ts6bWW289AMm9rOdPo7fuSKOlpfdbCzq/OeaYA0Dyt0TrsHTr1g1Ass5Ta9mzrQm9Z9ZZZx0Apd9BINnCh2sGlGrpaTiH1iTjc0E9Qzts0THGGGNMYfGDjjHGGGMKy3TTMvn973//m+ofyxW0orm/UUqNT4v/+7//mw4ApkyZMtUANdJcC3LRdLraaquFrlHn2rZt2+kAYPLkyY0zqBoy/fTTl12/rCjXriQLuH6V5peWdaWZRlrckqZnAOjSpQuA/NxVnN/ff/9dyP3ZoUOH6QBg4sSJhZxf+/btpwOASZMmZTI/3mvaAkLPYrqRtZN9Le/Jdu3a+fz8F5VaQFSinr+VPF8UW3SMMcYYU1iabdEpAtOy6Chq3eGTbGsIBLRFp3XTVItOGmp5KteUM++aTrbotG6ytugQ3b+6r1n/KCvLgC06rRtbdIwxxhjzn8IPOsYYY4wpLNN0XRljjDHGtGZs0THGGGNMYZlmZeSiBys52Kx1wvXLOhgyLxgMmfX6pQXbA9kH3HP9/vnnn0KuX5s2bTy/VgznV/Tfh6LPT7FFxxhjjDGFpdm9rkzT0ZRIpbX0/TLFhPtyxIgRofv1119DXnPNNes+pjxIK7gI+P40pmjYomOMMcaYwpKLRYdvT2rx0HgB0khFzpoDC1r99ttvodPiV7PNNttUf1Ok7LdKa9na4b7Vebam7tIc68CBA0P30Ucfhfzkk08CAGaaaabQtXYrh1psOP+///47dJMnTw6ZndB1fVv7/Ftr8VNjaoEtOsYYY4wpLH7QMcYYY0xhqZvrKq3r6VdffRXyW2+9FXKbNm0AAKusskro5p577pAb0YysrqlXX30VANC7d+/QzTnnnCFfeumlAIDNN988dOrGa41uLB3/6NGjASTdAZ07dw45z/m1NABV/1+6JF977bXQLbzwwiF37NgRQHnXVh7o+owbNw4A8O6774Zuiy22CJmum9bu2iiXMj9s2DAAwNVXXx06Dcw+8cQTAQD7779/6GacccaQ817LpqLz//LLL0Pm/tV7sqVdqc1/izQXcGvAFh1jjDHGFBY/6BhjjDGmsGTqulIzFzORAOCmm24CAFx11VWh0wwPujzmnXfe0A0ZMiTkmWeeGUBjurAUuuAAYMKECSHTJP7HH3+Ebtddd63fwGqErq+6qU4++WQAwLrrrhu6Ll26hJyHyZOuG60X880334S87LLLhpxmxlc31D333AMAOOigg0K30korhUzXyPzzzx+6vLPOdPyfffYZgOT8N95445Ab/b6qBNd6ypQpoTvttNNCvvzyywEAK6ywQuh23nnnkM866ywAwKeffhq6AQMGhNyuXbuQ67WXm+My4P+r8z/uuONCHj58OADggw8+CF1apmQt4ZqUcxera7El17Rcle/WCK9VpeugZ0o5l3yt4Fh0zRrJNV8JW3SMMcYYU1gytehoAOTjjz8e8lFHHQUAOOKII0J3yimnhDx+/HgAybfMp556KuRtt90WQGO9eepYFllkEQDAjjvuGLr55psv5GeffRYAcNJJJ4WuV69eIS+00EJTfWYjouvLNQOAsWPHAgC23nrruo9J0Tcb1ky54IILQnfbbbeFrMHwc801F4Dk9dfP+vHHHwEk5//LL7+ErG/SeaJj1jE9+uijAIBVV101dF27dg250d/O0tC1oHVR768rrrgiZJ4/xx57bOhmmGGGkGeffXYAyf3BAG4AWHTRRUPO0lJXLpiaiQ/l1inNovPFF1+E/Ndff03191kHmdJ6/f3334fulVdeCfnhhx9O1aeNibJa1nR9meTR6MH0ahHR/ctrpF6O9u3bh/zhhx8CSAbLM8EFAHr27BlyNddAx/T5558DSFpG9f7Q+6Alv1u6//i9tby3bNExxhhjTGHxg44xxhhjCkvd6uh8++23IdNMvOeee4aubdu2IdPNs/TSS4du1KhRIdM01kg5/Wpmo+tJS+yrmXKjjTYCkHRtvf/++yHT9dWaXFc6f7oONEA5D3R//PTTTwCAG2+8MXSTJk0KuTnXOi2wshGDIXUc6lpk/RgGjQPA9NNPHzLvpXLzyPteI7r/1E3DOjiDBg0K3dlnnx3y4YcfDiBZ+0rnyppCd9xxR+jo7vn3/1trdE7qbmICBwAceeSRAEpJGUD6mmiywyeffBLyrLPOWpOxplEuAYUuj5tvvjl0f/75Z8jqhurTpw+AZDC/uhb5/zKoHAAuu+yykFdffXUAJRckkM9ZWilA+OOPPw5ZXXf33XcfAGCxxRYLnYY+sEWLnl+auJPF/cnzjW5vIBl6cvzxx4fMmnGVxqF7Xe8vJgEss8wyU31/S7FFxxhjjDGFxQ86xhhjjCksdXNdpZme0iLpy/27mmEbkbT6EFpHR2FUvLa4eOedd0LeYIMNACRN63m7C9JMr8w+AoBLLrkkZNZnybt2jEIzqbpo1N2RZtou55qinPeaNIcxY8aEzBYAmnWle42ZGmoaV3gN85h/OdeIZm1ec801AIBDDjkkdAcffHDInKuuuZrR6RLS7ub1On90n7344oshn3POOSEvt9xyAIDtttsudJpdw2uk7ip1DdC1noXrv1wmF9uKqDtq7733Dlnb5TCrLe2eA4AffvgBQNINxqwgoLRWc8wxR8smUQU6Tr1/NNvsyiuvBJB0R2p9rwUWWAAA0KlTp9Cpa4/zUzcS1xTI5tzl/aFzuvPOO0N+/fXXQ+bv2pprrhm67t27h8wwlZdeeil0vGeBUgYss8uApGuuJW5IW3SMMcYYU1j8oGOMMcaYwpKp60rNmGq6YvEjNYOlmaTVdKzFk1oLlczB7HINJAsisrWAml7zdpPo+jz22GMAkq4Bzao77LDDAJRMsED+42eGRzmz5xNPPBHyPvvsk/gbIGl6pplc3T1qLs57rmmo64oF1bRIoGY1cn2fe+650KlJntmSm2yySejyaIVw9913h6wZjhdddBEA4MADD0z9u0puyo8++ghAKXsEyD6rha4BuhUB4MEHHwx5ttlmC5ntSipl/Wmmmc6ZrvGs2z7o/UHXoZ4ZmhVVKYxBP4sZlOru0eKkdI/VM9OKY2Z7GAC4/fbbQ+Y9BZTOig033DB0PHMAYL311gOQdK2r65VZd5qVnPVc+fnqLtasxCWWWCLkr7/+GgBw7733ho5FcoHSWapZuZ07dw6ZGaJaJLHae84WHWOMMcYUlrpZdBhAB5TqyFx//fWhO++880JmkJXWGTjmmGNC5ttPo5f4VjTYkddF22JonQQGnuVtGdAxa+AZ669oCf2vvvoqZK6LBmPr2yffaLJ+C9HvpHVCrYj6lqgl1Lnv1Iqje5EtLvTvV1555ZDZQiLv9VO0xQXrNKnFRt8Y+da2yy67hO73338PmXVo9I2ctUuAbIMhaW0Bki0edPwMcm1OsK3udQZB6v7VOi5Zrqu2P9BgU7WOst2IXosvv/wy5JEjRwJIBiNrnbLnn38eQDJAecYZZww5i8BktUiRSvtE10+tqwxcpWUHSJ6flVpk1Iq0psb9+/cPHc8BIGldW2uttQAkLTJa2+jnn38GABx99NGhU+u+BnFniV6/ueeeG0Dp7ACSFjUNhqZFTc9EPeu5lh06dAidttBhkovu2WqxRccYY4wxhcUPOsYYY4wpLHVzXWn9EgZTaR2Ifv36hfzyyy8DSJqm1TSep8sqrcvqv0lrUaFm1sGDBwMAhg8fHrp0JtsfAAAJkklEQVRy7TDqjQYovvvuuyEfeuihIbPcd9++fUNHdwZQmvcbb7wROr1WNG3OM888tRr2VN8NJN1Ub7/9NgBg4sSJoVN3hLqp2CJB95mOnyZX3d9qOs+zzkw51PVEl4i6jjUYki0U1N2g15IuEa0DojUzsnBd8Z7SYFZ1HbIjOVAKYmyOa0QDP+mmXHjhhUOnyRC1WlfdU7xXdB10fhMmTAiZweSKjp/7Vu9ldR2w3Yze32uvvXbIWaxfS66ZXh91bbzwwgsAkuekun7oes26LYvOiW5O/nYByWB2rYPDseiY1A3E30Kd8y233DLVZ2Xt+tf58Z5iIDuQdK0yHAVI/11UHWsq6f7m+QyU3LC1DJa3RccYY4wxhcUPOsYYY4wpLHVrAaHm0MUXXxxAsrbODTfcEDLNt9rdO+9O5TS9qYlRsx40Q4OR6WyFACQj6F977TUAJRcekHRd8bvydn2om0fHwq7B1157behGjx4dMk22mgmiZtbNNtsMQLLjba1My2nmZKB0rdUdRRM4kN4CQd2l+lk02ao5Wtth0E2krp+811LHR5O4uo7ZXRooufR0zHr/cl9opk4WqOma+0vvOe2+rm7QprqsdE9q9+uHHnoIQLIOiq51lq45dUFpzRLdt2mtOXT/UtZwgf333z9kZtDoNct7f1ZCrzmzxbT2yk477RQyXVr1rKPDM1uz45S0diOa9aZZgzw3hw0bFjo9S/JorcN7hu05AODGG28MWWvi0LVf7vpzr+m/ax023mvlQkNagi06xhhjjCksdbPo6NMZ6zxoIzANDGXgnb7FlGuQmSVqZWCwqtYw0GBiHR+bAmoAqAaW8U1UrTwaWJdnsLW+LWgdBK3JwTdN/X/32muvkFmldPfdd5/qb5QsAgTLwWqyV111Vei08nYauifVkse98Oabb4ZOAzvZdE/rzDQSfKPSYFu9P9OCJbW+C+ev1WizeHvW+4B1jtTKo5VlK+0l/TvOX/eCWodY84QVaoFsrThA6V6jNQkAPvvss5C7des21d+rFUbrkLFi9DLLLBO6U089NWRa4vTv62n9aCo6Pm0AyTpWvXr1Cp3WmclzLuUsY3p/0SLKCvhA0rr89NNPAyhZ3oDGaZC82267hbzCCiuErL9/layDaRZVrczOcymt9lxLsUXHGGOMMYXFDzrGGGOMKSyZuq7U9KQlvM8880wApVLX/4YN9DTALs8AZKDkmnjvvfdCp+NX0zkDP3X8GqTGIFBtWqbBXI2Czl+D4ThXdS2o6bhLly4AkvPTa0HquaY0k2qApgaYpqFrSnckUNqfak5uFNNyOVZbbbWQaRr/7rvvQqfNJHmthg4dGrqLL7445OOOOw5ANm0fdJ+oO5DNErfZZpvQLbjggiHrXuK66f7VxIBHHnkEQLJOl9bhYTPQcnVosoBj7dGjR+hUVtLuP3Wt8lro/HT/NvpeJTo/Dczluqy//vqhq6cbvKmkNaoGgHPOOQdAsimtNnBdaqmlADRWiyPufw1g10a3en9UOtf573p/LbbYYiEzGFs/p1o3li06xhhjjCksftAxxhhjTGHJ1HWlpjuaywFgxIgRAJJR25rBQpOkuhnyMOOpiZelr5944onQlaujQzMy680ApUwzoGT616wrrSnUiKhpkuuq5tgvvvgiZLq59Po1Sp0OHUelMalpVWtesOZDLes8ZM2uu+4aMmtWaZ0j7bo8ZswYAEl3z/nnnx/y9ttvDyCbNS1XFp+u3Y033jh06nrU2jLMpnv44YdDR3cBAIwbNw5AsnYQ3VVAPnVYSFO+k+49dec888wzIXNf9uzZM3SNcv81Be4BrSnETCuglEGpe0EzXHnfqus879AHXSv+Llx55ZWh0wyyRnJZ/ZvmnJ+V0HtdM0CZzay/H9Weta3npDbGGGOMaSZ1q6OjlQ9pCdDceQ3WXWmlleo1rCbDtwStxqmywrcyfUrXJ/pBgwYBALbaaqvQsZEiAOy7774AGt9ioE/0bNQGAHPNNddU/97aUesHmyKqFU+tW40yb7UO6BsT61dpAKRaBFiz5bbbbgsdq30DpTexLOapY9ZrSvm6664LnSYGPPfccyGzQSArsAPAfvvtF3KfPn0AAB07dqw4hkaE1+Kpp56aSgeUgsT/X3t3j5RKEIVh+BD4k1hlqlVugtgiNmABsgYM2I0lma7CmKJYgSwAEragkNzoa7+pCzooM9OM7xN1TXmVpodh7jk953hE45joujebzdIxr+Ol66b/j9+jINp43cTn0KPAHsX3KvDj8Tgiihvr/yJfn8VikcaK3h5yg3ne36QAAAC/wI0OAABordpSV57a0CZD32zW7/fTWKWvc6z3sE9Y++bmJo1Ho1Ea393dRUSxbLvaXkTkWRNiG3+dXl+hjXyzq1o8+GbY29vbNNa5nksKK6L4WpRaVIo0otj0Ufzzt0+djN/w3+0tSFTnxpv/+mbj+/v7NH58fIyIiF6vl455zR3NJfcUlfM0tmoezefzdMw/i2pge3FxkY4d01w1l9fX13TM64yp/pPP3+tENbnx2K/j3lbEP2u+ZUFyulZUbVsdqLe3tzTW9YmmngAAACVwowMAAFqr0tSVh0s9XKc6Fh66Gg6HaaynWY49nLcrXKw6EE9PT1t/NvfUldbFnzoaDAZprHYfuXbvLsvXpNvtpvHDw0NEFMOtHqa+vLz879/nROuXYwsLf8+8bYHeX6VlIopPGvnPbmvtkXNtkjL8Wqj5eZ0cb9Gip1Zzv47sonVdrVbpmJ8X19fXEfHZaiYHen1Km0Z8domPKJ63ejIr1+tD1XRe+naA6XSaxnoqku7lAAAAJXCjAwAAWqvzVUhovV4fLHfkYdRtu6n9dVQd0js9Pe1ERHx8fDSaG9sVWv5tmO7s7KwTcdj1+47PpeqUo9bv/f29kfltW7dDlkY/Pz+vff3qpPXbbDal56f3vMxnpumU98nJyd7z+4ldT6Vo/lW9D1XPTylxtRqJiJhMJmms4pBe8PGQqVfN77vvBz8X9SSmd1R/fn5OYy8u23QaVd8PTX3/6bz1tjqehn15eYmI4lOXnqb+juZX+Jv7v0wAAIDjUFtEJye5RHSq0kREp05NRHTqRETnuNUV0WlK1fNTpGS5XKZjHrFRO5KqNluXjeg4RWlU4yeiWLspp3Y+TUd0tG6+pt4C4urqKiJ+3pSViA4AAPhTuNEBAACt9WXqCgAA4JgR0QEAAK3FjQ4AAGgtbnQAAEBrcaMDAABaixsdAADQWtzoAACA1voHY9Bx06aoWGwAAAAASUVORK5CYII=\n",
+ "text/plain": [
+ "
"
+ ]
+ },
+ "metadata": {
+ "needs_background": "light"
+ },
+ "output_type": "display_data"
+ }
+ ],
+ "source": [
+ "# training data stored in arrays X, y\n",
+ "data = loadmat(os.path.join('Data', 'ex3data1.mat'))\n",
+ "X, y = data['X'], data['y'].ravel()\n",
+ "\n",
+ "# set the zero digit to 0, rather than its mapped 10 in this dataset\n",
+ "# This is an artifact due to the fact that this dataset was used in \n",
+ "# MATLAB where there is no index 0\n",
+ "y[y == 10] = 0\n",
+ "\n",
+ "# get number of examples in dataset\n",
+ "m = y.size\n",
+ "\n",
+ "# randomly permute examples, to be used for visualizing one \n",
+ "# picture at a time\n",
+ "indices = np.random.permutation(m)\n",
+ "\n",
+ "# Randomly select 100 data points to display\n",
+ "rand_indices = np.random.choice(m, 100, replace=False)\n",
+ "sel = X[rand_indices, :]\n",
+ "\n",
+ "utils.displayData(sel)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "\n",
+ "### 2.1 Model representation \n",
+ "\n",
+ "Our neural network is shown in the following figure.\n",
+ "\n",
+ "\n",
+ "\n",
+ "It has 3 layers: an input layer, a hidden layer and an output layer. Recall that our inputs are pixel values of digit images. Since the images are of size 20×20, this gives us 400 input layer units (excluding the extra bias unit which always outputs +1). As before, the training data will be loaded into the variables X and y. \n",
+ "\n",
+ "You have been provided with a set of network parameters ($\\Theta^{(1)}$, $\\Theta^{(2)}$) already trained by us. These are stored in `ex3weights.mat`. The following cell loads those parameters into `Theta1` and `Theta2`. The parameters have dimensions that are sized for a neural network with 25 units in the second layer and 10 output units (corresponding to the 10 digit classes)."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 15,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "# Setup the parameters you will use for this exercise\n",
+ "input_layer_size = 400 # 20x20 Input Images of Digits\n",
+ "hidden_layer_size = 25 # 25 hidden units\n",
+ "num_labels = 10 # 10 labels, from 0 to 9\n",
+ "\n",
+ "# Load the .mat file, which returns a dictionary \n",
+ "weights = loadmat(os.path.join('Data', 'ex3weights.mat'))\n",
+ "\n",
+ "# get the model weights from the dictionary\n",
+ "# Theta1 has size 25 x 401\n",
+ "# Theta2 has size 10 x 26\n",
+ "Theta1, Theta2 = weights['Theta1'], weights['Theta2']\n",
+ "\n",
+ "# swap first and last columns of Theta2, due to legacy from MATLAB indexing, \n",
+ "# since the weight file ex3weights.mat was saved based on MATLAB indexing\n",
+ "Theta2 = np.roll(Theta2, 1, axis=0)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "\n",
+ "### 2.2 Feedforward Propagation and Prediction\n",
+ "\n",
+ "Now you will implement feedforward propagation for the neural network. You will need to complete the code in the function `predict` to return the neural network’s prediction. You should implement the feedforward computation that computes $h_\\theta(x^{(i)})$ for every example $i$ and returns the associated predictions. Similar to the one-vs-all classification strategy, the prediction from the neural network will be the label that has the largest output $\\left( h_\\theta(x) \\right)_k$.\n",
+ "\n",
+ "
\n",
+ "**Implementation Note:** The matrix $X$ contains the examples in rows. When you complete the code in the function `predict`, you will need to add the column of 1’s to the matrix. The matrices `Theta1` and `Theta2` contain the parameters for each unit in rows. Specifically, the first row of `Theta1` corresponds to the first hidden unit in the second layer. In `numpy`, when you compute $z^{(2)} = \\theta^{(1)}a^{(1)}$, be sure that you index (and if necessary, transpose) $X$ correctly so that you get $a^{(l)}$ as a 1-D vector.\n",
+ "
\n",
+ ""
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 16,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "def predict(Theta1, Theta2, X):\n",
+ " \"\"\"\n",
+ " Predict the label of an input given a trained neural network.\n",
+ " \n",
+ " Parameters\n",
+ " ----------\n",
+ " Theta1 : array_like\n",
+ " Weights for the first layer in the neural network.\n",
+ " It has shape (2nd hidden layer size x input size)\n",
+ " \n",
+ " Theta2: array_like\n",
+ " Weights for the second layer in the neural network. \n",
+ " It has shape (output layer size x 2nd hidden layer size)\n",
+ " \n",
+ " X : array_like\n",
+ " The image inputs having shape (number of examples x image dimensions).\n",
+ " \n",
+ " Return \n",
+ " ------\n",
+ " p : array_like\n",
+ " Predictions vector containing the predicted label for each example.\n",
+ " It has a length equal to the number of examples.\n",
+ " \n",
+ " Instructions\n",
+ " ------------\n",
+ " Complete the following code to make predictions using your learned neural\n",
+ " network. You should set p to a vector containing labels \n",
+ " between 0 to (num_labels-1).\n",
+ " \n",
+ " Hint\n",
+ " ----\n",
+ " This code can be done all vectorized using the numpy argmax function.\n",
+ " In particular, the argmax function returns the index of the max element,\n",
+ " for more information see '?np.argmax' or search online. If your examples\n",
+ " are in rows, then, you can use np.argmax(A, axis=1) to obtain the index\n",
+ " of the max for each row.\n",
+ " \n",
+ " Note\n",
+ " ----\n",
+ " Remember, we have supplied the `sigmoid` function in the `utils.py` file. \n",
+ " You can use this function by calling `utils.sigmoid(z)`, where you can \n",
+ " replace `z` by the required input variable to sigmoid.\n",
+ " \"\"\"\n",
+ " # Make sure the input has two dimensions\n",
+ " if X.ndim == 1:\n",
+ " X = X[None] # promote to 2-dimensions\n",
+ " \n",
+ " # useful variables\n",
+ " m = X.shape[0]\n",
+ " num_labels = Theta2.shape[0]\n",
+ " X = np.concatenate([np.ones((m, 1)), X], axis=1)\n",
+ "\n",
+ "\n",
+ " # You need to return the following variables correctly \n",
+ " p = np.zeros(X.shape[0])\n",
+ "\n",
+ " # ====================== YOUR CODE HERE ======================\n",
+ "\n",
+ " a2 = utils.sigmoid(np.dot(X,Theta1.T))\n",
+ " \n",
+ " #Adding bias unit\n",
+ " a2 = np.column_stack((np.ones((a2.shape[0],1)), a2))\n",
+ " \n",
+ " a3 = utils.sigmoid(np.dot(a2,Theta2.T))\n",
+ " \n",
+ " p = np.argmax(a3, axis=1)\n",
+ "\n",
+ " # =============================================================\n",
+ " return p"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Once you are done, call your predict function using the loaded set of parameters for `Theta1` and `Theta2`. You should see that the accuracy is about 97.5%."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 17,
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "Training Set Accuracy: 97.5%\n"
+ ]
+ }
+ ],
+ "source": [
+ "pred = predict(Theta1, Theta2, X)\n",
+ "print('Training Set Accuracy: {:.1f}%'.format(np.mean(pred == y) * 100))"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "After that, we will display images from the training set one at a time, while at the same time printing out the predicted label for the displayed image. \n",
+ "\n",
+ "Run the following cell to display a single image the the neural network's prediction. You can run the cell multiple time to see predictions for different images."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 18,
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "Neural Network Prediction: 3\n"
+ ]
+ },
+ {
+ "data": {
+ "image/png": "iVBORw0KGgoAAAANSUhEUgAAAOcAAADnCAYAAADl9EEgAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjMsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+AADFEAAAGvElEQVR4nO3dv2+OaxzHcVV9Kn5NBoSFSRuJSEiMYhSLBZEOJhKxIPb6B8RKF4YOEhKjhHRpYjVUdJBKRCwSEaHlQZ1/oOn5Xjnn8Xye9vUaz/nkTs853u5ErnPdQ3/+/NkA5NnY7x8AWJk4IZQ4IZQ4IZQ4IdSm1f5mt9v1R7nQY51OZ2ilv+7NCaHECaHECaHECaHECaHECaHECaHECaHECaHECaFWPb5H7wwNrXhiK1avft7q/+y/Hi8F8OaEUOKEUOKEUOKEUOKEUOKEUOKEUOKEUOKEUOKEUI7v/YuNG+u/fw0PD5e33W63vP3161d52ystP8P379/L2x07dpR2nU6n/Mzfv3+Xt8m8OSGUOCGUOCGUOCGUOCGUOCGUOCGUOCGUOCGUOCHUujy+13KT3OfPn8vbubm58nZmZqa8XVhYKG9btNxo9+bNm/J2fn6+vJ2YmCjtbt68WX7mrl27ytvkW/28OSGUOCGUOCGUOCGUOCGUOCGUOCGUOCGUOCGUOCHUmjq+V70pb2lpqfzMycnJ8nZqaqq8bbmpb+fOneXtsWPHyttDhw6Vt+Pj4+XtkSNHytv79++Xdi036t2+fbu8bbld8W8f9fPmhFDihFDihFDihFDihFDihFDihFDihFDihFBr6oRQVctpkx8/fpS3Z8+eLW8vXrxY3h4+fLi83bx5c3nbcjqmV37+/Fnazc7Olp/58ePH8nb37t3l7d/+7mf//+sAKxInhBInhBInhBInhBInhBInhBInhBInhBInhBpa7dKibreb+/HC/6Dl+5yLi4vl7cjISHk7Ojpa3i4vL5e3vbJpU/2k59u3b8vbU6dOlXZjY2PlZ05PT5e3CRd8dTqdFX9BenNCKHFCKHFCKHFCKHFCKHFCKHFCKHFCKHFCKHFCqHV5+17LMaytW7f25Gfo1ZG8Xh1He/fuXXl76dKl8rZ6PPLChQvlZ7YcNfzbN+q18OaEUOKEUOKEUOKEUOKEUOKEUOKEUOKEUOKEUOKEUOvy+F6LliNu3W63J89tuS3w06dP5e2TJ0/K27t375a3LR8cnpqaKu1OnDhRfmbykbwW3pwQSpwQSpwQSpwQSpwQSpwQSpwQSpwQSpwQSpwQal0e32u5oW5hYaG8nZycLG+/fftW3rYc33v//n15+/r16/L24MGD5e3Dhw/L2wMHDpR2vfpwbTJvTgglTgglTgglTgglTgglTgglTgglTgglTgglTgi1Lo/vtXj27Fl5Oz093ZOfYfv27eXttm3bytuWj8wePXq0vN23b195W+X4HhBDnBBKnBBKnBBKnBBKnBBKnBBKnBBKnBBKnBBqXR7fazkKdubMmfK25Za8ltv3xsbGyts9e/aUt/fu3StvHzx4UN6Oj4+Xt5cvXy7t1soHcVt4c0IocUIocUIocUIocUIocUIocUIocUIocUIocUKoodWOsnW73b5fedZyJK5l228tRwhbPvY7PDxc3n748KG8PXfuXHn79evX8vbx48el3d69e8vPXF5eLm8TdDqdFX/henNCKHFCKHFCKHFCKHFCKHFCKHFCKHFCKHFCKHFCqL4c32s5Ztdy69qXL19Ku5GRkfIzWz5G2yu9+nBsy7+Hp0+flrenT58ub2/dulXa3bhxo/zMQfvQruN7MGDECaHECaHECaHECaHECaHECaHECaHECaHECaH68vHcltvk5ufny9vr16+XdhMTE+Vnnj9/vrzt1Y16LccdW26ea/mA78uXL8vbXtyCOGhH8v4P3pwQSpwQSpwQSpwQSpwQSpwQSpwQSpwQSpwQqi8nhFq0fG9ybm6utLt69Wr5mbOzs+Xt/v37y9uxsbGePPf58+fl7aNHj8rbV69elbfXrl0rb69cuVLatZw6Wiunibw5IZQ4IZQ4IZQ4IZQ4IZQ4IZQ4IZQ4IZQ4IZQ4IVRfvs/ZouX43osXL0q7O3fulJ85MzNT3la/D7phQ9s/V4uW5x4/fry8bTnyePLkyfJ2dHS0tGu5uGzQ+D4nDBhxQihxQihxQihxQihxQihxQihxQihxQihxQqj443stqkfXlpaWys9cXFwsbwftiNmWLVt6sm25/W6t3JT3Xzi+BwNGnBBKnBBKnBBKnBBKnBBKnBBKnBBKnBBKnBBqTR3fq2r5EGuvtoN2bG3QjiYOEsf3YMCIE0KJE0KJE0KJE0KJE0KJE0KJE0KJE0KJE0Jt6vcP0A9uh2MQeHNCKHFCKHFCKHFCKHFCKHFCKHFCKHFCKHFCKHFCqFVv3wP6x5sTQokTQokTQokTQokTQokTQv0Dkr5hsEL+aa0AAAAASUVORK5CYII=\n",
+ "text/plain": [
+ "
"
+ ]
+ },
+ "metadata": {
+ "needs_background": "light"
+ },
+ "output_type": "display_data"
+ }
+ ],
+ "source": [
+ "if indices.size > 0:\n",
+ " i, indices = indices[0], indices[1:]\n",
+ " utils.displayData(X[i, :], figsize=(4, 4))\n",
+ " pred = predict(Theta1, Theta2, X[i, :])\n",
+ " print('Neural Network Prediction: {}'.format(*pred))\n",
+ "else:\n",
+ " print('No more images to display!')"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "*You should now submit your solutions.*"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 19,
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "\n",
+ "Submitting Solutions | Programming Exercise multi-class-classification-and-neural-networks\n",
+ "\n",
+ "Use token from last successful submission (tanmayss2109@gmail.com)? (Y/n): Y\n",
+ " Part Name | Score | Feedback\n",
+ " --------- | ----- | --------\n",
+ " Regularized Logistic Regression | 30 / 30 | Nice work!\n",
+ " One-vs-All Classifier Training | 20 / 20 | Nice work!\n",
+ " One-vs-All Classifier Prediction | 20 / 20 | Nice work!\n",
+ " Neural Network Prediction Function | 30 / 30 | Nice work!\n",
+ " --------------------------------\n",
+ " | 100 / 100 | \n",
+ "\n"
+ ]
+ }
+ ],
+ "source": [
+ "grader[4] = predict\n",
+ "grader.grade()"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": []
+ }
+ ],
+ "metadata": {
+ "kernelspec": {
+ "display_name": "Python 3",
+ "language": "python",
+ "name": "python3"
+ },
+ "language_info": {
+ "codemirror_mode": {
+ "name": "ipython",
+ "version": 3
+ },
+ "file_extension": ".py",
+ "mimetype": "text/x-python",
+ "name": "python",
+ "nbconvert_exporter": "python",
+ "pygments_lexer": "ipython3",
+ "version": "3.7.6"
+ }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 2
+}
diff --git a/Phase 3 - 2020 (Summer)/Week 4(Apr 19 - Apr 25)/Exercise3/__pycache__/utils.cpython-37.pyc b/Phase 3 - 2020 (Summer)/Week 4(Apr 19 - Apr 25)/Exercise3/__pycache__/utils.cpython-37.pyc
new file mode 100644
index 000000000..071a78e18
Binary files /dev/null and b/Phase 3 - 2020 (Summer)/Week 4(Apr 19 - Apr 25)/Exercise3/__pycache__/utils.cpython-37.pyc differ
diff --git a/Phase 3 - 2020 (Summer)/Week 4(Apr 19 - Apr 25)/Exercise3/exercise3.ipynb b/Phase 3 - 2020 (Summer)/Week 4(Apr 19 - Apr 25)/Exercise3/exercise3.ipynb
index e37be91fd..32049b542 100644
--- a/Phase 3 - 2020 (Summer)/Week 4(Apr 19 - Apr 25)/Exercise3/exercise3.ipynb
+++ b/Phase 3 - 2020 (Summer)/Week 4(Apr 19 - Apr 25)/Exercise3/exercise3.ipynb
@@ -33,7 +33,7 @@
"from matplotlib import pyplot\n",
"\n",
"# Optimization module in scipy\n",
- "from scipy import optimize\n",
+ "from scipy import optimize as opt\n",
"\n",
"# will be used to load MATLAB mat datafile format\n",
"from scipy.io import loadmat\n",
@@ -42,7 +42,7 @@
"import utils\n",
"\n",
"# define the submission/grader object for this exercise\n",
- "grader = utils.Grader()\n",
+ "#grader = utils.Grader()\n",
"\n",
"# tells matplotlib to embed plots within the notebook\n",
"%matplotlib inline"
@@ -348,7 +348,14 @@
" grad = np.zeros(theta.shape)\n",
" \n",
" # ====================== YOUR CODE HERE ======================\n",
- "\n",
+ " hyp = utils.sigmoid(X@theta)\n",
+ " reg = (lambda_/(2*m)) * (theta[1:].T @ theta[1:])\n",
+ " J = (-1/m) * (y.T @ np.log(hyp) + (1-y.T) @ np.log(1 - hyp)) + reg\n",
+ " #J=J+reg\n",
+ " grad = (1/m)*(hyp-y)@X\n",
+ " temp = theta\n",
+ " temp[0] = 0\n",
+ " grad[1:] = grad[1:] + (lambda_/m)*temp[1:]\n",
"\n",
" \n",
" # =============================================================\n",
@@ -422,10 +429,10 @@
"outputs": [],
"source": [
"# appends the implemented function in part 1 to the grader object\n",
- "grader[1] = lrCostFunction\n",
+ "# grader[1] = lrCostFunction\n",
"\n",
"# send the added functions to coursera grader for getting a grade on this part\n",
- "grader.grade()"
+ "# grader.grade()"
]
},
{
@@ -452,6 +459,16 @@
"metadata": {},
"outputs": [],
"source": [
+ "def GraDes(X,y,theta,alpha,num_iters,Lambda):\n",
+ " m=len(y)\n",
+ " J_his=[]\n",
+ " \n",
+ " for i in range(num_iters):\n",
+ " cost,grad = lrCostFunction(theta,X,y,Lambda)\n",
+ " theta = theta - (alpha*grad)\n",
+ " J_his.append(J)\n",
+ " return theta, J_history\n",
+ "\n",
"def oneVsAll(X, y, num_labels, lambda_):\n",
" \"\"\"\n",
" Trains num_labels logistic regression classifiers and returns\n",
@@ -526,7 +543,17 @@
" X = np.concatenate([np.ones((m, 1)), X], axis=1)\n",
"\n",
" # ====================== YOUR CODE HERE ======================\n",
- " \n",
+ " Options = {'maxiter': 50}\n",
+ " for c in range(num_labels):\n",
+ " initial_theta = np.zeros((n+1,1))\n",
+ " res = opt.minimize(lrCostFunction, \n",
+ " initial_theta, \n",
+ " (X, (y == c),lambda_), \n",
+ " jac=True, \n",
+ " method='CG',\n",
+ " options=Options) \n",
+ " all_theta[c,:] = res[\"x\"]\n",
+ " \n",
"\n",
"\n",
" # ============================================================\n",
@@ -563,8 +590,8 @@
"metadata": {},
"outputs": [],
"source": [
- "grader[2] = oneVsAll\n",
- "grader.grade()"
+ "# grader[2] = oneVsAll\n",
+ "# grader.grade()"
]
},
{
@@ -625,7 +652,6 @@
" of the max for each row.\n",
" \"\"\"\n",
" m = X.shape[0];\n",
- " num_labels = all_theta.shape[0]\n",
"\n",
" # You need to return the following variables correctly \n",
" p = np.zeros(m)\n",
@@ -634,9 +660,7 @@
" X = np.concatenate([np.ones((m, 1)), X], axis=1)\n",
"\n",
" # ====================== YOUR CODE HERE ======================\n",
- "\n",
- "\n",
- " \n",
+ " p = np.argmax(utils.sigmoid( np.dot(X,all_theta.T) ), axis=1) \n",
" # ============================================================\n",
" return p"
]
@@ -671,8 +695,8 @@
"metadata": {},
"outputs": [],
"source": [
- "grader[3] = predictOneVsAll\n",
- "grader.grade()"
+ "# grader[3] = predictOneVsAll\n",
+ "# grader.grade()"
]
},
{
@@ -828,13 +852,22 @@
" # useful variables\n",
" m = X.shape[0]\n",
" num_labels = Theta2.shape[0]\n",
+ " X = np.concatenate([np.ones((m, 1)), X], axis=1)\n",
+ "\n",
"\n",
" # You need to return the following variables correctly \n",
" p = np.zeros(X.shape[0])\n",
"\n",
" # ====================== YOUR CODE HERE ======================\n",
"\n",
- "\n",
+ " a2 = utils.sigmoid(np.dot(X,Theta1.T))\n",
+ " \n",
+ " #Adding bias unit\n",
+ " a2 = np.column_stack((np.ones((a2.shape[0],1)), a2))\n",
+ " \n",
+ " a3 = utils.sigmoid(np.dot(a2,Theta2.T))\n",
+ " \n",
+ " p = np.argmax(a3, axis=1)\n",
"\n",
" # =============================================================\n",
" return p"
@@ -897,6 +930,13 @@
"grader[4] = predict\n",
"grader.grade()"
]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": []
}
],
"metadata": {
@@ -915,7 +955,7 @@
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
- "version": "3.6.4"
+ "version": "3.7.6"
}
},
"nbformat": 4,
diff --git a/Phase 3 - 2020 (Summer)/Week 4(Apr 19 - Apr 25)/Exercise3/token.pkl b/Phase 3 - 2020 (Summer)/Week 4(Apr 19 - Apr 25)/Exercise3/token.pkl
new file mode 100644
index 000000000..a2dff91ab
Binary files /dev/null and b/Phase 3 - 2020 (Summer)/Week 4(Apr 19 - Apr 25)/Exercise3/token.pkl differ
diff --git a/Phase 3 - 2020 (Summer)/Week 4(Apr 19 - Apr 25)/__pycache__/submission.cpython-37.pyc b/Phase 3 - 2020 (Summer)/Week 4(Apr 19 - Apr 25)/__pycache__/submission.cpython-37.pyc
new file mode 100644
index 000000000..7e44f90c4
Binary files /dev/null and b/Phase 3 - 2020 (Summer)/Week 4(Apr 19 - Apr 25)/__pycache__/submission.cpython-37.pyc differ
diff --git a/Phase 3 - 2020 (Summer)/Week 5(Apr 26-May 02)/Exercise4/.ipynb_checkpoints/exercise4-checkpoint.ipynb b/Phase 3 - 2020 (Summer)/Week 5(Apr 26-May 02)/Exercise4/.ipynb_checkpoints/exercise4-checkpoint.ipynb
new file mode 100644
index 000000000..cd4a2a292
--- /dev/null
+++ b/Phase 3 - 2020 (Summer)/Week 5(Apr 26-May 02)/Exercise4/.ipynb_checkpoints/exercise4-checkpoint.ipynb
@@ -0,0 +1,1250 @@
+{
+ "cells": [
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "# Programming Exercise 4: Neural Networks Learning\n",
+ "\n",
+ "## Introduction\n",
+ "\n",
+ "In this exercise, you will implement the backpropagation algorithm for neural networks and apply it to the task of hand-written digit recognition. Before starting on the programming exercise, we strongly recommend watching the video lectures and completing the review questions for the associated topics.\n",
+ "\n",
+ "\n",
+ "All the information you need for solving this assignment is in this notebook, and all the code you will be implementing will take place within this notebook. The assignment can be promptly submitted to the coursera grader directly from this notebook (code and instructions are included below).\n",
+ "\n",
+ "Before we begin with the exercises, we need to import all libraries required for this programming exercise. Throughout the course, we will be using [`numpy`](http://www.numpy.org/) for all arrays and matrix operations, [`matplotlib`](https://matplotlib.org/) for plotting, and [`scipy`](https://docs.scipy.org/doc/scipy/reference/) for scientific and numerical computation functions and tools. You can find instructions on how to install required libraries in the README file in the [github repository](https://github.com/dibgerge/ml-coursera-python-assignments)."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 1,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "# used for manipulating directory paths\n",
+ "import os\n",
+ "\n",
+ "# Scientific and vector computation for python\n",
+ "import numpy as np\n",
+ "import pandas as pd\n",
+ "# Plotting library\n",
+ "from matplotlib import pyplot\n",
+ "\n",
+ "# Optimization module in scipy\n",
+ "from scipy import optimize as opt\n",
+ "\n",
+ "# will be used to load MATLAB mat datafile format\n",
+ "from scipy.io import loadmat\n",
+ "\n",
+ "# library written for this exercise providing additional functions for assignment submission, and others\n",
+ "import utils\n",
+ "\n",
+ "# define the submission/grader object for this exercise\n",
+ "grader = utils.Grader()\n",
+ "\n",
+ "# tells matplotlib to embed plots within the notebook\n",
+ "%matplotlib inline"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "## Submission and Grading\n",
+ "\n",
+ "\n",
+ "After completing each part of the assignment, be sure to submit your solutions to the grader. The following is a breakdown of how each part of this exercise is scored.\n",
+ "\n",
+ "\n",
+ "| Section | Part | Submission function | Points \n",
+ "| :- |:- | :- | :-: \n",
+ "| 1 | [Feedforward and Cost Function](#section1) | [`nnCostFunction`](#nnCostFunction) | 30 \n",
+ "| 2 | [Regularized Cost Function](#section2) | [`nnCostFunction`](#nnCostFunction) | 15 \n",
+ "| 3 | [Sigmoid Gradient](#section3) | [`sigmoidGradient`](#sigmoidGradient) | 5 \n",
+ "| 4 | [Neural Net Gradient Function (Backpropagation)](#section4) | [`nnCostFunction`](#nnCostFunction) | 40 \n",
+ "| 5 | [Regularized Gradient](#section5) | [`nnCostFunction`](#nnCostFunction) |10 \n",
+ "| | Total Points | | 100 \n",
+ "\n",
+ "\n",
+ "You are allowed to submit your solutions multiple times, and we will take only the highest score into consideration.\n",
+ "\n",
+ "
\n",
+ "At the end of each section in this notebook, we have a cell which contains code for submitting the solutions thus far to the grader. Execute the cell to see your score up to the current section. For all your work to be submitted properly, you must execute those cells at least once.\n",
+ "
"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "## Neural Networks\n",
+ "\n",
+ "In the previous exercise, you implemented feedforward propagation for neural networks and used it to predict handwritten digits with the weights we provided. In this exercise, you will implement the backpropagation algorithm to learn the parameters for the neural network.\n",
+ "\n",
+ "We start the exercise by first loading the dataset. "
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 2,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "# training data stored in arrays X, y\n",
+ "data = loadmat(os.path.join('Data', 'ex4data1.mat'))\n",
+ "X, y = data['X'], data['y'].ravel()\n",
+ "\n",
+ "# set the zero digit to 0, rather than its mapped 10 in this dataset\n",
+ "# This is an artifact due to the fact that this dataset was used in \n",
+ "# MATLAB where there is no index 0\n",
+ "y[y == 10] = 0\n",
+ "\n",
+ "# Number of training examples\n",
+ "m = y.size"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "### 1.1 Visualizing the data\n",
+ "\n",
+ "You will begin by visualizing a subset of the training set, using the function `displayData`, which is the same function we used in Exercise 3. It is provided in the `utils.py` file for this assignment as well. The dataset is also the same one you used in the previous exercise.\n",
+ "\n",
+ "There are 5000 training examples in `ex4data1.mat`, where each training example is a 20 pixel by 20 pixel grayscale image of the digit. Each pixel is represented by a floating point number indicating the grayscale intensity at that location. The 20 by 20 grid of pixels is “unrolled” into a 400-dimensional vector. Each\n",
+ "of these training examples becomes a single row in our data matrix $X$. This gives us a 5000 by 400 matrix $X$ where every row is a training example for a handwritten digit image.\n",
+ "\n",
+ "$$ X = \\begin{bmatrix} - \\left(x^{(1)} \\right)^T - \\\\\n",
+ "- \\left(x^{(2)} \\right)^T - \\\\\n",
+ "\\vdots \\\\\n",
+ "- \\left(x^{(m)} \\right)^T - \\\\\n",
+ "\\end{bmatrix}\n",
+ "$$\n",
+ "\n",
+ "The second part of the training set is a 5000-dimensional vector `y` that contains labels for the training set. \n",
+ "The following cell randomly selects 100 images from the dataset and plots them."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 3,
+ "metadata": {},
+ "outputs": [
+ {
+ "data": {
+ "image/png": "iVBORw0KGgoAAAANSUhEUgAAAjoAAAItCAYAAAAwm9DjAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjMsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+AADFEAAAgAElEQVR4nOyddbxUZbv+L89RwhYLRVQMxA4QfcXE7u4ObDHAwsAu7MRGxUCxsRUbA1QEAxNbsZVXKd9zfn/8znXPtdhrGPaeWbNmD9f3H+7PzZ6Z51nPs+rOmf73f/8XxhhjjDH1yH/lPQBjjDHGmKzwg44xxhhj6hY/6BhjjDGmbvGDjjHGGGPqFj/oGGOMMaZumXla/zl58uS6TMlq0aLFTED9z2/ChAl1Ob/WrVvPEPObNGlS1ean2ZczzzxzA91//vOfkGeaaaayfqtly5YzxPnn+TVPZpT5/fPPP3U5v5lnnrnBBcoWHWOMMcbULdO06GSNvhmq/D//8z95DKfRFBs/0Tdi1ytqfuia/td/Fd4JuD/raU3/+7//O+S33noLANCyZcvQLbfcciFz/uVadrKC4yp2fnLdmvv5qXuyXGr9WJSaa62P3+SLLTrGGGOMqVtysejw7WrKlCmhU3nWWWcFUFtP5oxb0DeLf/75J+RJkyY1+Mwss8zS4PNAId6hluZnCnCN//7779AdffTRIa+55poAgB49eoROY1ia47qqxYN7+YILLgjdbbfdFrJaemoRXkv0mjJhwoSQub6zzTZb6Fq3bh2ynte1CMevc9K5Tu/66D4tda2qJmkWuTFjxoR84403AgB++umn0J111lkhL7744gBq1zNA62kpj4Cia5VmUa7Fa04xi3gee8oWHWOMMcbULX7QMcYYY0zdUjXXVVqw8UUXXRS6ESNGhDxw4EAAwOyzz97gM9VEAzQ/+ugjAMDIkSND9+2334Y8atSokGlGXmaZZUK3zTbbhLziiisCSA+QbK6UG5ia9/zVtPrvf/8bALDLLruETl2T++23H4B8TLBZocd/3nnnBQCMHTs2dD/88EPIHTp0AJC/a0DXTN2Mxx57LADgq6++Cp1eX+ia2XzzzUN3xhlnhLzYYosBqK311WvRN998AwDo2bNn6L777ruQV1llFQCl3an6/3T3AMAee+wR8lJLLQUg+7XW6wddh0888UTojjnmmJB53VUX3T777BMy92feFLu+81xS15uun7oRyZxzzhky16pNmzYlfytPJk6cGLLOL4/1sUXHGGOMMXWLH3SMMcYYU7fkknVFM+xff/0VumeeeSbkt99+GwDQvXv30E2ePLkqY1Oz4fDhw0PeeuutASTNjZqdoBkczBobNGhQ6JgpABRcc2uvvXboatH0WIxSbiqaudXcrW4Gzk/N8Srr57I8FsXmcdVVVwEA5ptvvtDdfPPNIdNkXuvZOY1Bj/9LL70EIJmJtOCCC4act8uK6J7Sa8lDDz0EoOCCBJLnJz/32GOPhU5d0vfeey8AYOmllw5dntlHU8tPPfUUAOC5554L3VxzzRXy999/DwBYYIEFQjfPPPOEzLXW9X311VdDXmmllUKm6ypr9Dzn8T/88MNDp9flVq1aAQB22GGH0K2zzjoh1+L+fPPNN0Pu3bs3AODXX38NnbqG02oGqeuqXbt2AIANNtigwXcCyb1e6WMxPfcp/o3e8zRrk+dd27ZtMxvn1NiiY4wxxpi6pWoWnbReOvrGoRYbvpHoZ6pl8dAny/bt24d81FFHAQDWXXfd0KlFR9+YFlpoIQDATTfdFLrTTjutgaxvlPoUXosWHT3+fCP84osvQvfggw+G/MEHHwBIviVrYPbPP/8MAFhrrbVCp4G/Sy65ZMh8u8nimOicfvnll5BvuOEGAMAjjzwSOr5FAsmaJc0Z3b+0ogLA2WefDQC45ZZbQkcrJVCwbuRRGVktT/oWvNdee4XcokULAMCuu+4aOrUOL7vssgAKQcdT/+2BBx4IAHjyySdDp+tfrfNT3+zff//9kGlxVCvOeeedF/J6660HIJnMoePnuumxVKpVpV5///nnnw/5oIMOAgB07NgxdHfccUfIjz/+OABg/fXXD53OL88gcp2TBsMfcMABIe+0004AgH333Td0pazYatHisTjzzDNDN/fcc4esQepcy3L3bFrtHr1+KKzvdPfdd4fu66+/DpmW1mpeP2zRMcYYY0zd4gcdY4wxxtQtuQQj0/xVzCyap+lRf1uDUfv06QOguAkwzc222267hU5Ny80liLWYCfvRRx8FUDChA8A777wTMmuasF4QkDTpsuaQBuhdc801ITMYFigEhGbtLlB31GabbQYg6W5rLmumFHP9Uv7ss89Cd8QRR4S81VZbAQC6detW8ruqRVrbGD2nRo8eHfL9998PoNCqA0ia/rmX1TVEdx1QmL8GoGsLkCxdl3qeaID1pZdeGjJrkuy+++6hU9cbg+Ur0Zak0uedHvNx48aFrC0c6MbXOa+66qoh07Wta5p3zSPuT623pdc0PdfoullkkUVCp3NJS9bQmjT8f3VXadB4pcI89HsYTkK3PpAMgFY3KueiY1I3XjGXaZbYomOMMcaYusUPOsYYY4ypW3JxXRF1h2idjuWXX77B3+aRiaS/2RjXBc2zOj81d3fp0gVAsoR5rWRaqblS6zyce+65IbPOhZog0zrpFsuEYIn6Xr16he7ggw8OeciQISGznH8W6Fw//PDDkLnWaaXYi5FW+yLveh5pmXIA8OKLLwIADjnkkNBpTRK2QyjmGsjTdcUaOUAyK0wzPJgZqedcmrtJ1+df//pXyHQ5a4saujOBZDZQpVyaXB91fVx55ZUhs3YOUDivtM4XXetAwY2l9XCYiQbk6+bR80TPuddeey1kuuQ0w1WPs2YAkryvn2muVXXNaYYSW6woaXXE2HYIAE488cSQedw0dEDbmVSqq7me59yXV1xxReg23XTTkHWtuNe07dELL7wQMrOyqoktOsYYY4ypW/ygY4wxxpi6JRfXFU1iahrT4laU8zZHNhXOS4vQqemV3VvVnJn3XNNMr6+//nrI2kmY7TBOPfXU0GkGFQsGMnsFKHRcBgquyYUXXjh02jE5jwwnzfBgIcti2QvUqxn+t99+C5kZEuqOrdb66u+oOfzTTz8NuUePHgAKLlQgWW6fc9FMijQ3ZNaZWPqdPKYnn3xy6NQ0vvHGG4fM/VPqmOv/61rSdadFMG+99daQNUOonIKW+pt//PEHAKBfv36h0wyXtOOva/rJJ5+EPGDAAACFwnsAcNJJJ4XMbJ1qnmec6/jx40PXv3//kNVNyk7k6m5TN2NaEby8W+hwfHof06zAd999N2QW/NO2Dur6GTNmDADgwgsvDJ1msNJ926lTp9BVIsNuavSYs7jmGmusETotgqv3B7YWUXeahilwXrq/da2zWD9bdIwxxhhTt1TNoqNP3Gz3wCdXIP86COWS9kbBUuVAsuYBm3nm/Rai8O1fS83rWz6tUECh9L6WZdc3agZuqsVq5ZVXDplv56x3AiQD27RmRpbHRb9bLTJsTaEWkbS3X30j1bcvvtGoRUzf9LKYE7+z2J7SwMWxY8cmxgkkWyjwXFSLlNZpoSWlWAn4SqFzocVCAzy1TpUGqDalAbC+vdKid9hhh4VOa6Lo22mbNm0ATP+aqhVHLb60uKS9GQOFRo5AoZmjnicff/xxyGxdom/MP/74Y8gMbC3WsDOTN+r/mzdrAAHJAGu1/rKFhd4TtI4Ma7rommsLHcq6P6t1f9HfYSsgALj99ttDZuuGvn37hk4tdty/WsdJg+F5XaqmRY7HUq8jW265Zchq5aSs66PHYvDgwQCSVnQN0LZFxxhjjDGmEfhBxxhjjDF1Sy7ByDTvsYs1kOxkzk7gebtzGkNaafP77rsvdBpMxmDcStU7aCrqGmDbBm0FoG4Cde0QNbNrp+fOnTsDSAYYap0PBjzvuOOOoStWs6Zax4W1fYBkfSOibqyrr74aQDJwVGu60OSs3d11/bPsxK5ryno5QKH2EVAIEtTaOWy1ARTcbC+//HLodP3ohtR6Mlm4BvScGjp0KIBkgLQGs1fymPIYandsDSxNC4ydXtSd8sYbb4R8zz33AADat28fOq1dtdFGG4VM14weHx3HzjvvDAA4/fTTQ6e1qd577z0AyTo9GmSaxVpyfdj+AEgeR22389NPPwEoBIUDyTo7f/75Z4Pv0mDWPffcEwBw6KGHhk471edxrdV2MmxNonNSWGfm7bffDt0mm2wSch4tFHjMlltuudA9/fTTIWtNLrrB9fqugdd0Q1YzXMUWHWOMMcbULX7QMcYYY0zdUjXXlZpZf//9dwDJ2ipqWqTpPO8S+o1BTcd0WWjHVjXj0TVSyoQ6PWbxxpphi30nXS80awPJ2g5qWn7llVcAJLsrjxgxImSa5NW0qVlnaSbLvN2UWpOCWUnKsGHDQqYbY+DAgaHTrDHWN9HaJlqOPwt4fn377beh047bCl0imgmna0LTuGayqBtF2xRUC7pOdf82pkVHY+BvqLk9i/2p1zdmWF1++eWh00zGtGy6Yi40uiGvu+660GmGJH9DXa+aQajneqXcCxyfuh41q4zuKqDgJtVMpeOOOy5kukY0g0vdcJdccgkA4IEHHgjdbbfdFrJ2tc+yO7u6+0844YSQ6YbVTM3VVlstZNZB0v/XDEh1E5FqXT8104td5IFkBu1ll10GIJn1pxmGhPcRIOlGzwJbdIwxxhhTt1TNoqMBVKNHjwZQqKALJOso0OJR6xYdfYvSsTLITOsI7L///iGnNYBM02k9EH1i18C7cmop6JhZG6d3796h69mzZ+rnWJPj0UcfDZ0G43722WcAkg3gNNi3Vipf6/otu+yyIbMirjY11cqyRx55JACge/fuodM6H7RunXLKKaHLei9zLvpmzDUFgEUWWSRk1k8ptncYLK4WAa1CzM9nPSfdH7RuaO0NbXqo1qmmoHuBVgydv9acKaeKq/69Wsz49qvVZNWaUupY6/fyc2pF1ZpArLmjFhO1vmqV8rQqxOVQrBr1s88+GzITI3ieAcA555wTMu8PaiVV6zPPO7XyaGD2ww8/HDKtSuXOj8dJr9lac2bQoEEh87irFUOPBevrsFo2kKypw3NRg/HzqEOne1ItdayYrFX21bvB66bOafvtt89snIAtOsYYY4ypY/ygY4wxxpi6JZc6OgxmVNO5BqbRzdWcXFcabMVgshVWWCF0aTU/tE6NlnCnG0TroGjgL5tqAoVy8OUeK5bTVxdOsXoNDNI8/vjjQ6d1PjgmbSfB2hcAMMcccwDI33Wlv6/BgNyXGpitAbh77703gKS5WdeKLgMt2581NF2rC0fr4Gi7hHfeeQdA0tyvJnealNWdoHWSWK5ezeVZNPXUPc1zSV1wDHoEkuXoeS1pzDmhe/3zzz8HkHQ3qBtEzfSNdRnontNg57feegtAcs9pbaNiDS7TvjdNp4HbDLzXNWPtlqzgmNu2bRs6tnoAkoHDXItixzZt/tpihdfHYi0w0vZtudciXgv0d9Q1qDWZtt122wbj0Hsh56KuO233wWSJal5fSpEWGK/3dL3/8R7DEBYgGSbQ2LYq04MtOsYYY4ypW/ygY4wxxpi6pWquKzVDsU6Dmm7VjcNoczUna1n+vF0eRE2/mu3CbBAtUa41TeiaogsBAEaNGhUyy8SrOVPnrMdCM3+mB3W3fP311yGz3LrWaNCsEM0AoHlW3W2aQcdoe5pogaTpslZcksXcCHRjae0YzfCgyVnrQGkdCdYB0f1brTnrnLTs/L777hsys+nUdaA1Sbh/L7744tCpy4FzycJdpehc6A489dRTQ6dz0jowxxxzDIBkuwX9rrRxa7sO1mnRth1a06Yc9PzTmkcnn3wygKS7QzOw1DW3+OKLAyi4gKeWidZxoWsMKGRIqjuL7oKs4J7RcWqGp9bhYoaO1r758ssvQ+b5qcdPXePMelV3lrbwUH2lz8ti+0y7d6dd39PCBLQOl+7lrOpHVRp1zem9pFu3bgCAN998M3Qa+sHng0pmktmiY4wxxpi6ZaZpWUcmT55cMdNJWmVkrRas1gG+vfXq1St0Woem3Ce9Fi1azASUPz99CtenbwZO6jg1gJGyPuVqtU4GieqbhwY2d+jQIWQe16nq7MwEABMmTGgwv7R6IUDBInHttdeGTuuwqBWJTVc1MFQtHmwQqI0CGcA69VibQuvWrYvOr6mkNWXdb7/9Qvf888+HzLd7vlkDyTo7DBJs6tsi5zdp0qSKzU/H8sILLwBIVnbWproMMp9//vlTP18uLVu2bPT5x/UZP3586LTOkwYOsw6M1o7RZpkM7NfaMRrYTOujfmdjjsW0ri96zdDaIqz98uqrr4ZOLTJ6LeC5pNVy0wJTtc6QNlDmGPT4qMW5lPW8UtdPPedYTRwoBH5rNXK9FrHOjlphdX1Zsffggw8OXdeuXUPWNajU/PidGlS7yy67hMx7HlBIEtBxaDD4u+++CyBZz0jvlayY3NR6TpzfP//8UzXXiM6V81tnnXVCd9ZZZ4VMi2xT7/MzzzxzA5OtLTrGGGOMqVv8oGOMMcaYuqVqrit1mVC+4IILQqclulmTRQPztNFcuWb0SpledU7aAoCuAa19o24Ouua0RYQGq6UFpumcdc2mZXot5drR8dOMrEHVWhNCXU90vemalGo6WMkA8ixcV0qam4RrChQCKtXdqMen3P1ZKddVscBIzi+t7QhQMBlnFfTfFNcV0XND1+euu+4KmUHkun+XX375kBlEri4adYMdcMABAAouWqBxazq915e0uWgdnUceeSTk4cOHh8y/0fVLO9d0/Tp27BgyWwxoHaVSdXqUSl0/FZ0Lx63uKnX98FhpALUGy/O6Wuz4lKKc+elvajC11hxjAomGM2gwPOuPbbLJJqE7//zzQ6bLsqnXmTxcV3r94X1xiSWWCJ3ONS1YvjHXIruujDHGGDND4QcdY4wxxtQtVXNdJX40pdOr1kGg6VFdO0q5JvUsTK9qmmPNg2LdzYnOo5Junul1XaWhptdidVLSTOPVrG2UteuKpK2pUqzOUblkkXVVS5TjulKKud7o5tD1STv/1HWl9V1KtSAoRVOuL2nuRL0+qmuccqk6Rron1bWqGVykWq6dxqDHIm2ts75+NmV+uibavXvo0KEhn3jiiQCARRddNHRrr712yMxG6ty5c+h0r1YqdKOarqs0eBwAYLPNNguZrr2mrqNdV8YYY4yZofCDjjHGGGPqllxcV/HjKZkgQHrWTiWpluk1L8pxXTUHquW6ygu7rhpPWlZnY1w7lXTDZpHVWUyeXtLm19R5zijXz0quXzE367TIKmu1VlxXaff8qeWmYNeVMcYYY2YoqtbUMw19cqtkAy9jzIxFXoHxWVKPc5qR8P1t2lSzubMtOsYYY4ypW/ygY4wxxpi6ZZrByMYYY4wxzRlbdIwxxhhTt0wzGLne0wenTJlSl/ObZZZZZgKAiRMn1uX8WrVq5fTWZsyMMr9aOv/Ucv/9998DAOadd97QaeXdUlTq/NP0a23gyCBVrWxdTWaU/Vnv81Ns0THGGGNM3ZJrerkxxphsYM8uAHj//fdDPuywwwAAd955Z+jatWsXcpZxm2rF+euvv0K+/fbbQ2bfp5VWWil1TI4rrT0aU9Cy3IKVTcEWHWOMMcbULVWz6OhTHt80GlNQKa1ctJ/sTR5w/zamRL/uVe1qbCoPrxVq0VAY+1Gv14+ZZ/7/l/WxY8eGbs899wx5t912AwAstNBCoavWsdDr+G+//RbyRRddFPJbb70FALjuuutC16JFi5Cb+7rx+lBqHhq3pPfHahbamxbFLDfjx49P/AsAs88+ewNZz8+sCyraomOMMcaYusUPOsYYY4ypWzJ1XamZUs31o0ePBpA0Z3Xo0CHkNNPev//975Bpmm3VqlXoasWcVww18zV302sazX1+aZ2GVaem1e+++w4A8M0334Tuyy+/DHnSpEkNPrP44ouH/K9//Stk7uW8j5meq2kun1o3navMa4UG4Or4efzTUppVzntNGoOu37fffgsAOPDAA0One+6oo45q8JlqzVXXYdFFFw350EMPDZkuq6+++ip0Sy+9dBVGV1mK7c/JkycDKO6Oo2t16NChoVt++eVDpssxj/NQ98y4ceNCvvvuu0Pm/f2nn34K3RdffBHyrrvumvgXADp27BhyFmUFbNExxhhjTN3iBx1jjDHG1C0Vd12pOXj48OEhX3LJJSG/8sorAIDZZpstdJtvvnnINM2p6fmXX34JeYUVVgAA9O3bN3Ra2bNaJr1SmTY6DnXd8W/VDJjmOqnV2hF0baiJUedHd0ypegp5o8ef7iagYIZ95513Qvfzzz+H/MMPPwAAllhiidAtt9xyIdPM/uOPP4buqaeeClmzXfi3Va0p8X/z1vV78803Qx4wYACApJuhV69eIS+zzDIA8tmTxVyk9957b8hcv9deey31c3vttReA5PWnU6dOIW+99dYAkq71WnHXKepi/Pvvv0Pu3bs3gKRr//zzzw951llnBVBbc9K1oGv46aefDt1SSy0V8vRmLeWB7rOJEyeGfM8994T84IMPAki6a84555yQ6dpS1+NVV10V8nbbbQcgf9eVnl/HH398yCeccAIA4JprrgndyJEjQ+7Tpw8A4NZbbw3dBRdcEPL2228PILm/y52rLTrGGGOMqVsqZtHh05cGaJ511lkhv/766yHTeqO9VkaMGBEyrQNqpVHrzrBhwwAkn/JOO+20kPnGAlT+qZ/WCiD5FsWncKDw9PrEE0+E7tFHHw2ZbycLLLBA6BZbbLGQGaSlumI1QbIkzcoEFOpznHnmmaF78cUXQx40aBAAYI011ghdXn1rpkbnpGO6+OKLQz7jjDMAAF26dAndzjvv3EBWy4yuT5pFTq0jqq/WW5mO7/PPPwcAXHrppaHTN8555pkHQNLKpRafRx55BEAywDrrOhhE1++9994LWd8oaX3T64cec75p6rms3zt48GAAyTfOOeaYI/W78oBj1crC+kZMiwjPQwCYa665Qq4lSw6pdevv9KIeDa0NpOcarWt6Tun/d+3atcF36bmWpyVV73Ovvvpq6t/yXFGL6FprrRUyrx8nn3xy6Hr06BHyxx9/DCBpRS63jpItOsYYY4ypW/ygY4wxxpi6pSzXVVpO/THHHBO6F154IWSWHQeSQUpEc+5pWlXTFwMkgYJJ8Kabbgpd586dQ95ll11CrpRJnaY7DbC+9tprQ2btCqAQpDVhwoTQqZn8s88+a/D9ao6j6fyWW24J3WqrrRZylm6C6Qn2ZB2O7t27h+6uu+4KeckllwRQ2kSu+0fJIgg7LYBx4MCBIb/xxhshH3744QAKQXMAsPDCC4fM46/zK9XOpDFN7yqFuqvefvvtkA844AAAyT2p5xebKX799deh23DDDUOmaVnN6VnDueh5psGav//+e8h0XatrUq8lTGb44IMPQqd/+/jjjwMAbrjhhtCpa6xabjolrQ7LqaeeGjqtufLYY48BABZccMHQ6V6tlWBenRPdbQAw33zzAUheX/RawbkUu36QYteRas1b99Tqq68eMt00e++9d+jUjXPEEUcASCY4aB2dPPefhmtoi5E2bdqEzJpNOk6VeS6qa0///9xzzwVQcKEDwMEHHxxyU+o/2aJjjDHGmLrFDzrGGGOMqVsa7bpKM6EChawnrRdCEyQAHHbYYYUf/T+TuZpTNYMlDbpLgIKZ67zzzgvdFVdcEfIGG2zQYAzlZho89NBDAJKlyv/888+Qdfw0U66yyiqh00wwHkPWYwGSGRIffvghgGQmjH5XFtAcqO42zZrT48tjQBMjkHQNpLl2FLoheEyn/i51cx533HHT/K7phfP75JNPQnfnnXeGrG5QlqbXvd5cOo6ru0rLrqs7d9lllwWQrM2hLVh4rLQEvx5/ul6rmb3DtdDaRlqbY8455wyZLSD0+nP99deHvP766wNIutZPOumkkDlv/X41kefR7kRN+5yLXh+GDBkSMl2Ken3W85rj1zo71ZqH7k+9/um1oH379gCS66fuCs5FM3w1A2/++ecHAKy44oqh0/2RZdajuqt22mmnkHWtmIG85pprhk4zjJity3pVQNLNnKfrSjMxdf24ZkDB9V1sT/GYa+0kdQ2//PLLAJLXp0022SRkrak0vcfCFh1jjDHG1C1+0DHGGGNM3dJo15WaEDVSXrM6iGZqaAR5WnfgUiYo/V2WcNfsLc2g+P7770NW8+f0oL/z66+/htyvXz8ASReUunN0rowWV3OjQvOtRq1r1hLRDBN1nWghqUqbnK+88sqQdX4sogcUSswX606fhv7tk08+CSDpotIMi3XXXbeRo55+6NYACoXzgGQJ9v333x9AMtNNzfy1WHCNx1ezFzUDcuWVVw6Zrg89N9Tkzv3Jwpz6/UDBZJ21uyOtI7nuSV0T/ds999wTQCG7DAC6devW4Pu32GKLkDUrhgXdtMinXt90X2R5DPSYa8FUFpdTd75eX/m3zB4DgGeeeSZkunb0WBZz7VQKro+eOzfffHPIaeeiFpTVrEgW92R2GZB0zfG4qWto3333DVmvO61btwZQuTnr/LStiHaP5/3r+eefD51mOPL+ddBBB4WuVjLk9J6oYQB6/eb9r9R1Uu/5msHJ84suLCDphm1K1qotOsYYY4ypWypWR4dPWe3atQud1rlQ60ZTnk71M7Sq6JOd5tzPPffcjf7+Ur9Ji4W+Jey+++6pf8sn2VJPtA8//HDI+ka9zTbbAEgGcGs5+0oFo+n6jR49GkCyrQMDgYGCFUc/VyxAk+iaq/WKdSRY6hwA7rjjjpCLWRrKgceMNVQA4Nlnnw1Z3x4YmKyWMw2WYzBcLTVd5f7UUurFgj15fHUfpbWIuPHGG0NHKwBQaFGin8kiQFL3JwM09c1eLTpbbrllyAxi1AD5UvtI6yTxTfzqq68O3YUXXhjybbfdFjLPy0qtv85ZWzzo/mPNsLS2MUDhTVvL5rdt2zZkWnfUYqwtIrLYy5zXRx99FDrdXzoXtljRFjp6L+G+22+//UK36qqrhvCGhtoAACAASURBVMxgebYaAJJJJFrHjAktWexfPY4a7M5mpXp91XsWm3ZqgkDe1xeeP1pPTa0sWsduetF7htbnodWokpZzW3SMMcYYU7f4QccYY4wxdUtZdXTGjx8f8i+//AIg2bFazYnlor/Lcu+a068BgmqGLQd1vdBcWqzsf9pYi5X9Z80H7VirptNNN90UQLLja6VMq+puUNM1y5GzszwAnH766SGrm4BmRg1ATptrse7KDCzVFhpZuKvSUNfAEksskSqz/oV25z3xxBND5vposKse12qZmfWYc19qALG6IdWlzL2krjkN4D/22GMBJOvoaJ0Puu6yDsrW+Y0aNQpA8pzXOikaWMv6HI3ZR/q3rNnBekNAMhhZx8DzolJrrue51rHSmj90yb344ouh23jjjUNm13V1DWuyBl0jTSml3xh0/Xh81QWoySxa04rnkrp21A1Jl7+2RUgbv7YN0DpS3EtZo+eHXl/oRmbSA5B06bGdBwOlgfzreHEuek3QY66u0VKkhTk8+OCDIbOr+0YbbZT6/U257tiiY4wxxpi6xQ86xhhjjKlbyqqjo3n0NGmpu6WS5lB1DdD0qLUTtFOvZhs0dgxqFptjjjlCXnvttQEA9913X+g0+2LbbbcNmVlhEydObDBmoODG0RYS6m6j+TqLUvP6nT///HPIzNDR9dOy6tra48svvwSQdF3q96a1k9AMCGa1aJ2JPMqaF/tNmozpogKSdUp69uzZ4POHHHJIFkOcJron6IbS2huvvfZayDoXumTZagRIZhgxG22BBRYInWb1MNMoSxfj1HBPFatTUm6dI/0Ms1222mqr0GmLlizg9U1dVNq2Qq9/zDDTrCOtPcXrn7o7Bg8eHDJbC7DVCZC9u5XHV12A6jrVOjN0Der1Q90YdCkWuz5yf+v1V9tFqBupWuj+YobS2WefHbrtt98+ZGY2denSJXR5uMYVHmsdh9KYc47n8ksvvRQ67d7O/1fXpdZUasp1xxYdY4wxxtQtjbbo6JObNtdiUy9t1KXWn8bAp0f9/Lhx40JmxVJ9stWKjZUKktTfZxVSfbM74ogjQtbAWtYcoeUDSAZ7ctxa2ZkBWEDhTSyLJ3d9GtaaMnyi1joJWjtHK0LzjVADBNWixsCxU045JXRqHWCwpL4dVKvacLGqmmk1cfRYLbLIIiEz8HWHHXYInTZ1ZR2kasJjqfVC9thjj5D1jZh/qxYpDfbjOawB4tq0Lw/rG9ckrULz1GOiRaMx54/uC34XEyym/v9Kod/J61ufPn1Cp7WLTj311JBZv0vPP634ze/SAHq17rHBpJ7T1Tr/ip3zan3hXltnnXVCpxZhVq7WYGOdCy3RDKoHktZzDQLOA85b6/lo4PGAAQMAJBNsmCwC5HP+Ea2greundXDSSGs2fPTRR4dO14eWTE1mKtd6bIuOMcYYY+oWP+gYY4wxpm5ptOtKzcFqOqWsJig1saW1gChWZ4Zmqueeey50NFcCwDvvvAMgaTpTM5e2SygH/X6WKNdWBWo6feutt6b5XZtttlnIDGbVRmgarPzbb781ccSNQ485zYjaKFDnX6qRqK4vj4WaM7WEPRsoVrM2RFptn2KB52nz073MIE5d/yFDhoTMINZqBhByfFovRxvFausEHnetE6OuA7retBGitljJo6kpTfu6Z3kdAJINOFkzRd1cpcasf8sgbXVTa4uCprrkp0bN8axZNWLEiNCpaV+vH0wieOCBB0LHRrlAIaBZ68zoXmD9oWquI8+F9ddfP3Ra20fdbHS/ah0gTYbgvUCTQdTNx+/V64tef/S6W61joHvmjz/+AJBMbNFgZF4fNUBXWw9Vq46VwvGvssoqoVN34euvvx7y4YcfDiB5rmowOO8xY8aMCR1rlwGF609jzt+S4y/r08YYY4wxNYwfdIwxxhhTt5TlulJzNuuvPP7446Hr169fyGo6pBtE3QGa4XDaaacBSJor1Z3DmjZbb7116PbZZ5+Qsyhtzu/RehXa/ZqR5ADw448/AkjWqejYsWPI7FSr8y/m5suSUt239TiWivTXzw8fPhxA0h2knd7zyBrgXD799NPQaVacmo5LRfhzrt26dQuduoZqBc2UUtdLWgbRRRddFDLNxGwVkBdqrub5/corr4ROzz9tUcHWJtqiQ69VRI+DujQvueQSAMnzUNuhaGZpY68v+pt6faPpX7MTNQNS68/wuKjrR7Pq+vfvDyBZm0brdFXL5aHHhq4rdcfpOaP3jcceewwA0KZNm9CpG5zj10xWdb3S9aNuFt0fuq7VuhaltU6iCwtI3stYJ+ncc88N3d133x0yW7to9/Cs4fh33HHH0HGdgGToBe/luiaaTciQFGbvAsm50iVWybWxRccYY4wxdUtZdXT0LYFVDPUp76qrrgr5888/D5lv+lr7RqvsPvHEEwAKT+YA0KNHj5APPPBAAMDCCy8cOn17yOKNJe3NTd+SVU6r+TE9zUAJaxVkURk5K3SsrJKttSH0+ORZh0WDdW+//faQtQopx6prpvPjm9SNN94YuiWXXDLkSgWrlouOP20u+kaolXNZZ0SPVR77T8fMsdx8882hO+yww0LWYGQGcar158gjjwyZ1w3dh/q9rDOjTVH1TbaUJXRa6N+zeShQaMpZ7JxPu6bpPktrQKzkEUCe9vtqZdTaOO+//37IDAYfPXp06NTiRjRAVyvTs4pusWSXPK4/+vtff/01gKSXQvcCK0OXs88qDddPK5BrYLla5LbYYgsAyfu71rzi+XfxxReHTu8PWVRcr40rsjHGGGNMBvhBxxhjjDF1S1lRr2pOYx0DbZimwXIPPfRQg8+rOXWDDTYImaZndV2paZKmrTxLYU/P7xczF5cqJ0/XT97mysagY2UQttYzynsuXAtta6DmYg2GY80SDWDVwDrWzFHXj9Yf4vrmPedi0OWhNVv0XO3bty+A5PpVs+ZRGjzntdWItlDRwFua0Z9++unQaZ2jUucf6yNpAGsWboSskg7ydlOlwWNWLJlF2z0w4aOY64kUc82mrU/e56KOn2NVd9yVV14Z8siRIwEka5dpskSe9z09znvuuWfIGqbAkABtxKn3d4ahaIuLrOdki44xxhhj6hY/6BhjjDGmbplpWia9yZMnT7e9j9+j2VPvvvtuyNqdmyZ/7Z6tpi3WqdCxVTKTpUWLFjMBwJQpU3K1Z+qcrrvuupDPOOMMAMDLL78cOq3DU8o0Pcsss8wEABMnTqza/NQ0y07KQ4cODZ3WiSiXVq1azQQ0bn+WgpkQQCGbStt6sOw6UMgq0KwDNd2W6zrg/qzk/BS6TG644YbQaQYTXT5am6WSrqtKza9Yvazff/8dQNI1py0QmNWjGSSaVbXffvsBSNaBasyacn7VPP+qSRbnXy2Rxfmn18cJEyYASGYdMdMPKNSkU9dpp06dQi7XDZf1+ceaRzpnzQSkSzyrrDLOLzHWin27McYYY0yN4QcdY4wxxtQtFXNdkaa6mEqZrrIwbeXtutKo+mHDhoXMMukDBw4MnbpOatF1pdBkmZZpUAmyMJ2n7Vvdc2lZH1mbXrNyDXAu2gJD2xGw+3dWHcuzdg2krVWpTCuFc23qmtp11byp1vnXlD1ZCbKeX6lradYZcHZdGWOMMWaGouIWneZArVh09Ile67QwmFfrEDSGvC06WeM3ysqgb14aLMiaNVm9eVVrfnlhi07zZkbZn/U+P8UWHWOMMcbULX7QMcYYY0zdMk3XlTHGGGNMc2aazVb++eefunwKmnnmmWcIH2XeMUhZwRikep/fpEmT6nJ+LVu2nCHWr97nN2HChLqcX+vWrWeI+0PW89MYVO3rRr0WIXXBQGOMMcaYJpJN+9wM0KdDZouU6lhr8iGtTkQ16yg0hVK1VxpTR6cW56ekrU/aWjWnOSml6pTUYndvY+oFnndsBQEUOpoDhQzjQw89NHTajiWLa40tOsYYY4ypW2raoqNvZGwEChTqzGjTPa0y3JzePusR+l7/+uuv0M0+++wha82WPNFx6Fs+68jonmMjPqDgb2ZzOiDZ1JPU0j5Ms9j88ccfoVN/OZvqps2pVkmb359//hk6jRGYddZZG3y+ltaqXDhX3dPNyYqVVlm3OY2/FFwfned//vOfVLk5o/Njo2QAGD16NIBkU++NN944ZFt0jDHGGGMagR90jDHGGFO31KTrimZoNWFdccUVIZ9zzjkAgDfffDN0nTp1CjlLM3RaUPTUcto40gKn69FcDgCvvfYaAGCPPfYInQajrb/++gCqa6JNC0z9+uuvQ77vvvtCZrPL9957L3TadJWtObp27Rq6Aw44IORNNtkEQPKY5LHWxdI73377bQCF5p0AMHbs2JDXXXddAMmmsm3btg25VtwIOj/dS48++igAoHfv3qFbaKGFQr7qqqsAAKuuumro6ulcHD58OACgXbt2oVtwwQVDrvW5fvfddwCSru8lllgiZF5ra30eiu7VMWPGAADefffd0OleXHrppRt8vjkl3nCu6hqfNGlSyHSJ//TTT1Ubky06xhhjjKlb/KBjjDHGmLqlJl1XNE3++OOPobv++utDZlaIZvJkQZqbSk1wH3zwQchDhgwJmfUDVlllldB179495Pnmm6/B99eKO6Cp6Pjbt28PIOnOUzPtBhtsUL2BTYWafW+66aaQ6Q4FCuujmQBHHXVUyHRpvfHGG6EbMWJEyEOHDgWQNEEzkytrirlzbrnllpD79+8PIOnOuvfee0Omy4ouHgA499xzKz/YCqL774knngAA/PLLL6H79ddfQ2YGSL9+/ULHa8rU31XL6PmlboAtt9wSAHDCCSeErlevXiHXSlaP7lXWVgGAgw46CADQoUOH0F199dUhM5tOa69o1m2toOvz8ccfh7zbbrsBKGQfAcCaa64Z8r777gsAWGSRRUK30UYbhcxs0Vp1YXHezz77bOjojgSAFi1aAACWX3756o2par9kjDHGGFNlasaio0/3fPu95pprQqdPhHvuuSeA6gbY8e1Q3+z55ggUAlSBwtuF1g6Yc845Q+7RowcA4Mgjjwyd1gSqlTeuxqBvL/r2TJZccslqDqcous/45gQAK6+8cshrr702AKBNmzapnyO33XZbyEcccUTIv/32W2UGW0GefPLJkEeNGgUA6NOnT+h22mmnkPnGdfLJJ4dO9yoDe/O2fKRdMwBg/PjxAICdd945dBoYz3VfY401Uv+flq5afWMmOv933nkn5J9//hlA4ToJ1OZcdPxqkXr++ecBALvvvnvq50466SQAyXpIF198cci1Mle9juu94LPPPgOQtCJ+9NFHIet5R/Rc5T1I64DlPWe1Dv/+++8AgDvvvDN0uta06C+11FJVGp0tOsYYY4ypY/ygY4wxxpi6pWZcV2qGo+lSA9Dmn3/+kI855hgABRM7UDl3j47jm2++CZmuiVdeeSV0J554YshqJqcZ79tvvw3dgAEDQj7rrLMAAJ9++mnoLrvsspDpxsrbNVCKYi06Hn74YQCFVh0AMPfcc1dvYCmkmXY12FFdazzuxczBdNNpnR11o+perTY6Zg3QPProo0Oea665ACSb6ula8hzQwHt1Ry688MIVHHFlGDduXMiPP/44AOChhx4K3TLLLBMy53fttdeGjrWdAGCxxRYDkL87oBQ6Pq2DtOKKKwJIustrfS7qBt5hhx0AAHvttVfo9FoycuRIAMCuu+5apdFNP3r/0GDjQYMGhcwgYwYlA8lkG15Ldc3U9bX55psDAJZddtnQ5RHuoOEKGvh/+eWXAwBef/310G244YYhX3fddQCSrses73W26BhjjDGmbvGDjjHGGGPqllxdV2r60joKLMev2Sua1cL8+yzMXWp61NoobAugbScWX3zxkDXqnKhrpEuXLiHTtKyuL/3/ww47DEB6R+ZaQtfvk08+CZlm6OWWWy50q622Wsi1klXWmO7Our6siaG1Z7Q2EF1Xebse9fe7desW8uqrrw4g6drSv+Vcdf+lZZ3ljY5Ju6+zdQBddFP/P9FMH3XT1eJcS6Hjp5u41ueh1zStmbbffvsBSJ5z6hrnvlXXVS1eK1944YWQ9V7G+9chhxwSOnXNDR48uIHuhx9+CJluWnVd5QHrxQHAmWeeGTLDNHQd2BYHABZYYAEA1astBtiiY4wxxpg6xg86xhhjjKlbcnVdqblRzXwsNKSmOZYFBwoukyxcIPqdWlCMGRzqrlJzfykznGaIsdP1zTffHDoWkdIx1LrpWed84YUXhsxsM22roBH2teK6KkUx18gdd9wBoNCFFwBOO+20kNmapJqm2TTUdKxzadmyZYP/V9fAhx9+CCB/11spdPzqpmLxR+0ov8IKK4Q8YcIEAMlMOy34WSuuj1LoeTRs2LCQmW2lruVaRPfXV199FTJdv7pn1bXF+WnBvbzXjMeaxfKA5D0trd2Puo4vuOCCBt91xRVXhG7HHXcMma2F8piz7qnHHnss5LTigIsuumjotttuu5DzuK7U9plgjDHGGFMGuVh0+FSowUxXXnllyHxTYVAuUKhtAWT7pqxvSW3btg2ZT6mNsUbo068GOfPthEGTALDWWmuFTOuPvmXXEjwWfDMGgOHDh4dMS9ymm24aurzfuBoD56frpzVzWH+F9SIAoGPHjiHnbclJQ48/Zd2TGizJOjQa4Kp/Wyvom6HWMTr//PMBJC2m+qbNc1ivKWzkqv9f6+j8tY7O9ttvD6BguQNqc056fulc2BRXLeoamMy9WIsWbz339fqe9je6Jlpzh+1atGmpXkvnmWeeBr9VLXTN1KKm1wdeN9SiqrXFbNExxhhjjKkgftAxxhhjTN2SazCylmh/+eWXQ6brY5999gldHuauUr+pplM16VH+888/Q6clsvv27Qug0AUaKJT11t+txdoQQMGMfMstt4Tugw8+CJndhWliBWpn/I0xd+v6sy0JAPzxxx8Akt130wJ/a2XOxdBjoXuVnbC1bPvSSy8dci0GKeuxpplfXR90hwAFN2utr08xuG7q+tfu15x/Lbp2iqGuR7YbUXfj559/HjIDl3XPMgEgL7iX2L4HSAbAv/baayHTTXXKKaeETtfyiy++AJBMYFHXMl1W1bw/8Lc0nELPKV0r1k/TtkgaeE2XXakxV/I6Y4uOMcYYY+oWP+gYY4wxpm6pmutKo+aZdaTdyTUCvU+fPgCS3XdrxVxerG3F999/HzLNyAMHDgydto5gOW/tmKydzDlvdrkFklHteiyqZX7XedNM+eyzz4ZOTc80WeqY8876SHMnqRlWMxiYTaDj1xYP1GvtoFtvvTVk1nTJ2zWia1bKjaGuY5rJ0+YM5L+WpJjpnvVLtDbJmDFjQua613qLi1KkZdI1J/SYr7TSSiGfcMIJAJJ1WrTODqmlrFQef62t1b1795AffPDBkHn9vP/++0OXVudK0axkus5PPvnk1M9nQVrWsbrT9PdZM03viRwzULhvtmnTJnR6T+P1V+8p5e51W3SMMcYYU7dkatHRt8Dx48eHfNxxxwEARo0aFboDDzww5K233hpAbb2xcC4///xz6I4++uiQNdiMc9U561M6m3oqWnOAT/xaufWoo44KWWsq8K096+Oja8kqrKy3AhQa8QHZNl1tDGlvOfpGouPTwL+0v9UGpf379wcA9O7dO3R33XVXyGzWl0cwuf6mNgXUYNW0pp3nnntuyLvvvjuAZNPEvM+/NHT8zzzzTMgc/9prrx269dZbL+SHH34YADBy5MjQsRo0AHTq1AlAPpbT6YHn/AMPPBA6tS6nNRiuRfT4aoPL9u3bA0gGG6tFnDWtNJmjVtAK6htttFHI2gD4+OOPB1BoDgwkrVM8LsXqXD3yyCMAkk2hdc2z2Ku0eNPaBgBPP/10yBoMzir/e+yxR+j0WkpZ6wTpXuB39erVK3R6LdLjMr1ztUXHGGOMMXWLH3SMMcYYU7dU3MZZLFhXTeMMwurcuXPoGIAMFNwItRL0CBTM5OoCGDRoUMhprg82XwOAI444ImSWaFfTHRthAsCrr74KIBmsvdtuu4WsQc7bbrstgGyOlboG1LRK15WOv2fPniGzZkIe61cswJSuDXUharBtqXLqWgeCDer69esXOg0W5Jq0a9cudFkfC553ajo/88wzQ77++utD5jEo1pSWwYC6vkqt1AnS9dU6Rxy/rsltt90WMk3fWttq//33D5mtI7SpcN5zVThvdWfo/uS61dKYS6Hj5/VRXRRak4v1W3TPpgXw5o26k7p06RIymwLTxQokmzrvsssuAJLuWA3srdZc01zvGqKhrkW9/3FdWA+pGOpa12vRuHHjACSfCTR0QJ8bprcNhi06xhhjjKlb/KBjjDHGmLqlYq6rtDx7de3ccMMNIdMMed5554VOI+hruftzhw4dQkcTI5A08zPbY6uttgqddm+d+juBZCdl/oaaO3fccceQtfUC3SRZUCyD56mnngKQrHOw6KKLhpx3thVRdxvdgNpxfZlllgl53nnnbfB5XR/WfgKAF198EUDBxAqkr281a7OkdSRXWV1adBOw3sXU8uDBgwEk9/oxxxwTsrqn80SvE+rGWX311QEAP/30U+huv/32kFmzZd999w2dZrDQpXDTTTeFTs/FWtnfek527do1ZLpMa2WcjSWtBU4t3hNKodcPXQuefzo/PVe5ruwCDiTnv8ACCzT4fBbo+HnOa1smDaHQFhZ0WalrK82NmnZMgMK1aM011wyd1pRryr6ujSuWMcYYY0wGlGXRSQv81HoBp556asj6RsmKjtp0r9bfPmipatu2bej0LVHh07ken1I1OdLqu3Ts2DF02kBNn/Srddy0siXrj2g9ILUI1AoaIMdgU63XtNpqq4WsFkVaqnTO2lSQ9ZHUytajR4+Qad2pZjB2mkVH58pGgUDB0sF6HkCy5gytX1dccUXotOYVLSKHHXZY6LQibJZBsMUC5NXixqakRx55ZOjU+sH6V1tuuWXodP9edtllAJLBzGrd0bfPPCqTM3HhmmuuCd2xxx4bMhtL6jW3uaPXOe6B5lTNWsfKhJaxY8eGThN32Oxa1+9f//pXyJdeeimAZLBz1vcBjv/www8PndY+0vOA94e77747dFpTjntZrVRqXaeXQpsmF+sMML3YomOMMcaYusUPOsYYY4ypWyrmumJg5umnnx46rVNxyimnhMyaK2oCrnXXVRrFgjJpxmvqnPh5dX3MN998qX9brePG2iRAIXC6Ft1VxVwJW2yxBYBk2XwG3QLJFgB0E6o5dZNNNgm5W7duAIA999wzdFoCneuWRx0T3TNq+mVtGKDg8pl77rlDp+cyA3PXWmut0GkdJwYZqmuoWugxVXeZlpsfMWIEgGQdD23XsdlmmwFImv51fuussw4A4Jtvvgld3m4SNd2PHj0aQHL91I3QHAN3S6GuZd5X1F25+OKLV3tITYbXCnWtq5ucaIC51sFi4kfe98xirUY4bnW3lSItcLuSLaBs0THGGGNM3eIHHWOMMcbULTNNyyT0zz//TNNelJYBobnz6tqZc845Q6bLKi/T28wzzzwTAEyePLn51EhvBC1atJgJAKZMmVKx+ek+0QwWwkyPajDLLLM0en7ci7onNXtNM3hYE0JrI6mZOa0EeyX3Muc3adKkstavWDsMUsw0nPa3v//+e8icf1MzrVq2bFnx/akuu7/++gtAch7qZqXJvdiY07ImG+MOasr+LIW6rt5//30AwHPPPRc6zYbJ2s3G+U2YMCHT66fOQzOQzjjjDADJe41myJVL69atK35/0OsOzyXtBM41BQo1Y84///zQqRu6XNck7w/1fv9TbNExxhhjTN3iBx1jjDHG1C1lua4SX/R/ZsZimUilCuZVE7uuyiNtjavphizHNVDMrJ/m5im2T6nPah9XynVVLnpMdM3LzSrMwnXVVDddFmThulLSCublcf5l7boqRqnzs1yycF2l0ZhWKpVcX7uujDHGGGPqiIo19Uyr/WLqk7zrN5RDKSuNKaDHpNbP62pabPIma4tirVMv827O19Hmhi06xhhjjKlb/KBjjDHGmLplmsHIxhhjjDHNGVt0jDHGGFO3TDMYOe/01qxgemu9p9dlld6aN1mn7+YN51fv+9PzKw+tlqwy0WDXSjb6nFHW7++//67L+c0666wzxPoptugYY4wxpm6pWHp5KRpT0GtaOmOqRS0VoTPlkVbwsLmuGa03o0ePDt0TTzwRMi056667bui6du0aMo9Frc4/rSCi4rTs5kGx62ce558tOsYYY4ypWypu0dEnN/Ubq55dZ9VvrN2F2Slan/j8FG+qBd94tbv5vffeG/KDDz4IADj33HND16lTp5Br9U15RkPX4bfffguZ3cv1mpN1x+9yUYvUG2+8AQDYZ599QvfZZ581+NsFFlggdGeffXbI++67L4D0t+y80PlNnDgRAPDvf/87dLPMMkvIc845J4D8x9wYSlmHFd43tUhnc7z/TZ48OeTx48eHzPXjeVgNbNExxhhjTN3iBx1jjDHG1C0Vsx3RHKcmti+//DLkt956K+Srr74aAPDTTz+Frnv37iFvuummAID1118/dGpmrkWTZdam71qac1qwYLFO16TWzbA6fppZTzzxxNDdcsstIXN+Rx11VOiWXXbZkPNYK5qB9dhmdZyz7h5dLhzf999/H7p11lkn5IUXXhgA8NRTT4Wu1q8vOqZWrVoBANq2bRu61VZbLeQffvgBAPD++++H7thjjw15jjnmAADsuOOOqd9fLTS04Ysvvgj5pJNOApAMsJ533nlDvv766wEk7xm1iF4H1Y3z888/N/j/CRMmhPzuu+8CADp37hy6RRZZJORa3J8K56XB8uutt17IPO9UN2XKlGzHlOm3G2OMMcbkiB90jDHGGFO3VNx19ffff4fulFNOCfm+++4LuXXr1gAK2VUAcPvtt4d89913GWL38AAAIABJREFUAwB69+4dupNPPjnkSlb5bCxqblXTY5prRv82zY2gJsi0OWmmQd4ZEmlzZfYcAAwbNizkZ599FgDQsmXL0B1yyCEhL7bYYgDyn5Oivz9gwAAAwK233ho6ZgoAwEUXXQQAWHvttUNXLXecHjPNCuvXrx8AYOONNw6dujPKHZ+uP03vc889d+h0r+e9ljxGHCeQzLqiS2vIkCGh23XXXUPOe/xp6JhWWmklAMDgwYNDp+ca98XHH38cOnXDnnPOOQCADTfcMHRzzTVX6m9VGt1H48aNC/mggw4KeeTIkQCSoQtjx44Neb/99gMA3HHHHaHbYIMNQs7TNa7z++STT0K+/PLLQx4+fDiA5HFW19W3334LoLDOAHDNNdeEvOKKKwJIzrOW9mxaaIPe3/S+US1s0THGGGNM3VLxRHZ9s9M6DhqsueiiiwIA9t5779C9+uqrIfON+qGHHgrd0UcfHTLfXrJ+ik2zOPz444+h+/XXX0MeNWpUyO+99x6AZGXSDz/8sMHn9C3znnvuCZmBpRdeeGHodtttt2mOKwvSalsAwFVXXQUAuPLKK0Onx4UWH62TMHDgwJD5JqZvYXm8kehx/Ouvv0J+5JFHGoxJA48POOAAAEkrXrXGn2ZZAYDbbrsNADB06NDQqRVVLVLT+8Zb7I3suOOOAwAstNBCodM6LWqJzNP6OGjQoNDpGzOvHwzKbW5w/eaff/7Q6XHmWi+44IKh69u3b8i0iGhtKLWoZAH3kp4z1157bcgajPzkk08CAFZZZZXQnXXWWSFffPHFAJL7LG+4537//ffQHX/88SFr4DuDjLt16xY6XUtasrROklrnlltuOQDJAF71juRt3eEa6/5q3759yMsvvzyA6lrebNExxhhjTN3iBx1jjDHG1C0Vc13RDKX1KNTcqK4nmuk0mFHdOGnBhGq6o0mzmkHJnJ+6MB5++OGQ1czPv2WAKJA0J9J1165du9Cp6fLTTz8FkAyG3WabbUKeffbZU7+3EhQLdu3Tp0/I/fv3B5Acc8+ePUNea621AAAXXHBB6BhgCBRq0qhrL63pYtboXL/77ruQWcdCj/OWW24ZMtc3bxOxMttsswFI1qb6448/QtZg0+lF3dDqeqWbWWt7aIDhfPPNF3KeLkk9Fuoy4bpynzZXdE5pqGtAa85ssskmAICXXnopdHTHAtkElqd9j9bx2WWXXUJmEK6OT13fvG5osG7e5yJ/X+9/DBoGgOeeey5kuk4PPfTQ0GloB6+VdFEBSTfxqaeeCqBwzgOF2kNAPu1M0sIANEFFA895X+/QoUOVRmeLjjHGGGPqGD/oGGOMMaZuqXjWlZpLNatBsz6Idqdl7RUgaToleWdy8DfVXHzJJZeErBH0dKnpONW0R5O/ljXXrKv9998fALDqqquGLo8S9boOupY0c2tWAWvjAIV2H1qCX8e8+uqrA0hvFZEXOla6YbbddtvQqRm6FltYEM10K/f46p7VrJivvvoKQDJrTjMs86hzlZaNpmPW/+f61fI6VgI95/T6yZCBN998M3Tq5tTrUpaucXU9qZ4hAYcddljoNKuIdYCakkmYFTxOOk6tI8faOEAhq3OzzTYL3RFHHBEy14etPIBkCw92qqcLS38fyD8MgPdIXRPN2qVrq5outtq50xhjjDHGVBg/6BhjjDGmbqm460pR01Wameq6664L+YUXXgiZpjeNRNcI7TzMlHTj6JjVdMqOwkBpcyHnp+4SdV0xwl4zEdSNVCrbohyKmUDPPPPMkNNKfGt3Xrbz0PktvfTSIe+8884AknPKw92h49dOydxfdLEBSdM/j0sp02seHdvVHaxr0hQzcanPqLsqj0wPRX+fJv/PP/88dGnu8BkJPT6UdX9k3T06Db3W0B0KFDJbNWtuiy22CHmFFVaowuiahp7n2pZDW0Cw+CtbyQDJ6yszJNWduMYaa4R86aWXAkhmDebdDkLvFVxLzdTU8TMkI8v72NTYomOMMcaYuqXiFh19siv2xs7S9CzlPfXfMnDp8MMPD50G4/K79C2lWAPNct6k056MtUZIsWDPtHLnOj7Wp9En+qeffjrk0047DQDQpUuX0OVhxUp7C1RZx6QWETZl1TXr1atXyAsvvHCDz+eBvsVqsCDRpp1aJ+aKK64AkGzap+s7zzzzAEgGUy611FIhV+qNK21N1IrGVipAYU8B099CRd/4X3nllZBpvdQ6OnnXMVEYkF3s/OR5WUtj5liLBZKmvf0WS3Yguie1TtSIESMAJOt4aU2zPI6LWt9oldQ6Vrr/WJ8tbU8DtbOuaW05gEIdI71+MIEDKATT8++ApCeBltRqWkQaA+9v2nZl8cUXD5l7rZpWRFt0jDHGGFO3+EHHGGOMMXVLxVxXNLmquUrNpexYDRS61hYLYGSw2TrrrBO6tMBmNX1pMJsGSdLMXq45k7+p36N1DrRrLesEvPHGG6FTN9RNN90EIHlMtMQ3TZfPP/986NZbb70GY5l6PJWgmLlf5/fll18CAO68887QsXs2UDA9a00XdjwHCmuiJeA1mDtLl5bOT/fn66+/HjJdT9riQzsJ08yu36UyXav6/Xp8yuk0rOZqdR1tt912AIDzzz8/dNodWvfMrrvu2uC7dBx0z2owpLom6ZLUAPNacRcAhSDWv//+O3S6pzjuWnJ3/PLLLwCAb775JnQ//vhjyGxLAhTWTd2t2uKDsrqjtJ3C6NGjASSTPfRapjWxskTXpGvXriGPGjUKQNK1wwBcoFC/TNdP3Vi1gp5zmiTA1jhshQMk3Yi8r33wwQeh4z0FyH+vNoW8x2yLjjHGGGPqFj/oGGOMMaZuKct1pVH9zErR7BqWugaSGRw03Rerw0LXlZom1TXAz6m7gbUJgGSEOrtOlxuhzt9Uc7J2/B06dGjIrLlSrKYCx6K1WTQqn66hYq6FStUsScta0y7y6pp66KGHQmbpeHVT6l5gOwztrvviiy+GvN9++wEodMEGgBNPPDHk9u3bNxhjpUyfuo+0zoO6abgXtcWHukNZrl3rHNEdAABnnHEGAGDMmDGh033D7vVAeftS99dxxx0HoOD2BAouUgC47LLLQu7Xr980v6tTp04NxqauDZ6/bdu2DV2eXcqB5F6km1SPhY5v++23B5BPWxUd82uvvRYyy/mzc/XUY0qrA1TKha1rqp+nvMwyy4QujzpWirquKTM7E0i62Q455BAAhXpdQMEdC6Tv3zzQY37llVeGzOvKwQcfHDpt8cDrorbY0X2xxBJLAMg/a7UYaXXW8q6zZYuOMcYYY+qWRlt09ClVA4BZ8+bll18OXevWrQs/VKLZoAZoMghUrSCrrbZayHyS1QBJtUT0798/ZFp0mkLaE6kGiL333nshn3766SHz7UOPj1a+ZIPMPn36hE4Dc/n/xaotl/P2qcderWh849cAaTaPAwq1EfQ7tDbCkUceGfKBBx4IILlX3nnnnZBZX0bfyDTYWS0RfOvO4o1b96TCtdY5pdXEUSuCWhcZOKgWA7VuVWou+j081zQoUwMYdXwM0i32lsW11jom+vZIfd7ByDp+nSuD+HX/zT///CFvuOGGDf6/Wm//OmatHcM6KrpnFN0/TNLQAFatnzRkyBAASYuxzo8WR1pWp/7+PI5FWh00tTKpRfXcc88FUFhHALj//vtD1utqtSlWbV2DqXmOnnTSSaHTex2D6fWemLdFpLlji44xxhhj6hY/6BhjjDGmbinLdaVunOeeew5Awe0CFFo5AMkAUwZZaTCoBmvR5Dpw4MDQqZw2lmKtJ8pBzfE0p6655pqhe/LJJ0NOM+P37t07dGqavOGGGwAAm2+++TR/t1LuKqBg+lRzsNZcufDCC4uOA0i2vmAw7r777hu6Uu0AtAEdzcxaJ0ODDatVU0ePhdZkYmC4NhJs06ZNyI8//jgA4LzzzgudlnDn3/bs2TN0WuckiznR3aC1U7jPgEKdFiBZ34foms0777wAkgHy3bt3D7lz584Akq6RvOtkpJ3z6oJZfvnlQ+a5mncw5zbbbBPyU089BaDgdgKS54G6jllTR11fWn+Ha6HzUzdJjx49ABR3bWWJXqf1/qCB+3QzqmtOx8cGz3pO6rWEx6qcelWNhfPSAHi9vmptIiYxqOtcwwhYf03dsbpWtU7a/StvbNExxhhjTN3iBx1jjDHG1C2Ndl2puV/bErA2wDXXXBO6PfbYI2QtoU+TpJouNeuIdQS0Ho5mVdFMqOY87fTKqHyg8hku6oJSd4uazunGU9fBXnvtFfIGG2zQYGxZm9HputJS5FqCnMdSXXBaJ2annXYKmaZjNUOXMn3r/JZccskGv6Wfz/JY6HevuOKKIa+00kohDx8+HEAhewxIZu3QZavfpa4RugE1K6RaZtxiHa/Z1gIouKaKwb2i55xmxTFrK4vaTk1Fs8qYDafro64DuvfUHVSt8adlrwGF66dmP2nWjo7166+/BlA8k5XtZNgWBEhmEKbV+coaHl91x5xzzjkhawYm9QcddFDo9L4zduxYAMn9qednsWzKavD++++HrFm52k6G1z3dc+ryYv0yzTRbffXVQ87b5VqKUq6rPK4VtugYY4wxpm5p9KOvPk1qsNyhhx4KIPkUXuyNg9+hdXb0iZXBjvpG8vHHH4dMq4o2gtPfUrnSQXbFAoTHjRsXMoOQtSZG3759Q+Zxq2blTo5Vg8UZVAsUGgSyqjGQHH9azZGmjr/cz5eD7l+t7Dto0KCQ2XSPAaJAsjLwqquuCiBphdTAbFpMaikYr5ilJw2eq3qsVP7iiy8afGfeaNNLBnYWq7zOcedthVIWXHBBAMnmtzzOQLIOFavQqxVAGyDTOqnWIZ1rHhYB7im1KOv5pZXDef5pHS+919x3330Nvn/nnXcOmdf/al5f+FsPPPBA6NR6pYHT3H+6vtdff33I1GttILXu5F3xuRS8r+u9ZtiwYSHT4q1eHk12yeK6YouOMcYYY+oWP+gYY4wxpm4pK2pLTaA0jRYLBEsztxULxuV3afNDDSZM+0yp38oaNTMyCE1LkavpMQ/TcZq5XgNo2Ui12NjybvqXBcXcWDSda6M9uguAQrBnMXNrrQcLloJzUdelJh6w2WKl6lVVAg0mJ7omutdryeVGeM3S66cG63fs2DHkNJdb2v6rZB2ucuGY1IVz7bXXhqxurF9//RUAcPPNN4dO3XQ8Lvp5DX3I4/rPc0Fd2IMHDw75rLPOCpnNVO++++7QaW0rBo5raEatX1P0mHOv7r333qFTl+yoUaMAVPf6YYuOMcYYY+oWP+gYY4wxpm6ZaVomzUmTJtWejbcCtGzZciYAmDx5cqPnV6wFhrZz6NKlCwBgwIABodMI9KxNqy1atJgJAKZMmVKX6zfLLLNkOj+6BjRTMC1rJStzMufXlP1ZCdLahbDtB1DI5mHtl8bC/VnJ+ek5xZo/em3TFhxZl9PPYn61RDnz0/NIzy/dazyvNENLW7QwA0vrEFUSzu/vv/+e7vlxXlrv6N577w35xhtvDJl1qLSO12677RYy3cSaaVZJ1+Oss86a6f5MOxa6lmwtk1ULGa6fYouOMcYYY+oWP+gYY4wxpm6x66qRqLntwQcfDLlXr14hP//88wCSHdurmQlg11XzJm/XFVE3rXaXZqdstjJpLFm7dtQlQqqZgWTXVeNJyyQrVdAxK9dxU1xXaaTtw+kh6+7fWbuuiK6fylnPz64rY4wxxsxQ2KLTSPQp/Y8//giZtR8AYIkllgCQX6luW3SaN7Vi0VF031Nuam0lWzyaNzPK/Mq16NQq1bLo5IUtOsYYY4yZofCDjjHGGGPqlmm6rowxxhhjmjPT7HVV7zE69R7jUe/rV+8+5nrfn/W+fhMnTqzL+bVq1WqGWD+ff80Tx+gYY4wxZoairO7lWVOsXDjdbbXe0dU0T9JaQKRRTx3LjallmlKTxuekIbboGGOMMaZuqUmLDp/eJ0yYELqLL7445Jln/v/DPvHEE0OXVnnRmOklzWL4wAMPhG7kyJEhb7/99gCAZZddNnQtWrRo8Pl6Qs8vnn/6xpxXzSgzfaRVqa1mteimoPtL65SljVV1rVu3BgDMOuusoStVZdnUN7boGGOMMaZu8YOOMcYYY+qWmnFdqWlx8uTJAIAzzjgjdJdffnnIV199NYB0d0OtkmY6Ltb0jCbbWg+mKzb+UvPjWqm7I4/1S9tzAHDTTTcBAM4///zQ/fLLLyFfdNFFAIDDDz88dBdeeGHIbIZZ63uyFHp+6fEZNmwYAKBt27ah69ChQ/UGZhqQFqyr1w9t1zFu3DgAwDzzzBO62WefPfVzeZ6Xd9xxR+iOOOKIkDk+vX5os+WVVloJAHDNNdeErnPnzg0+X0ukudaa+/WjFMXciVnM2xYdY4wxxtQtftAxxhhjTN2Sq+uqWJ2cIUOGAAAGDBgQOs2w2m233QDUftYAXRhA0vT/wQcfAADee++90P38888h77jjjgCABRZYIHTMdMmLtLXSOen4x44dm/gXKMwZAFZddVUAwA477JD6/dVaS91zb7/9dsgnnXQSAGCOOeYIXc+ePUP+4YcfAAB33XVX6PbYY4+QV199dQD5u+bKRbMeb7vttpDpUl5rrbVS/1+PW624CXSty83Aydu1k3b+ffnllyE/+OCDAIBRo0aF7t///nfI3L+aNcg9CwDbbbddyAsttBCA7Oepa8Jz8eyzzw7dXHPNFfK2224LAFhvvfVC17dv35BHjx4NAPj0009D16VLlwqPOB295he7pnH/qE5di9Trnk2T8zi3ioUjNOZzvC6mzRnI5l5ni44xxhhj6pZczQT6lKpvH3x71mDP0047rcHnauVtcWo4vq+++ip0+sYxePBgAMCUKVNCp/LAgQMBJC1ayy+/fMjVmnexp3e+Kekb12OPPRbypEmTACSDVf/++++Q+/fvDwBo165d6NQ6UK2aLPoW0aZNm5D5RtujR4/QrbvuuiF//vnnAIBnnnkmdE899VTIa6yxBoDmVVsm7Y3r9NNPD50GdvItv1evXqErFsxaK/zxxx8h67nWFOacc86Qq2Vp1fV59913ASQD4J944omQOb+FF144dGuvvXbIffr0AQA8/fTToTv22GNDfuedd0LmuVrNOjRLL700AODcc88NnV4fOK/vv/8+dGodmG+++QAAXbt2DV21LG9ff/11yGoRnThxYsjPPfccgOQ18c033wx5+PDhAIDNN988dIsvvnjIm222GYCCZRzIfn34/XpN0/Gn/T7vA0DSoj906FAASYu4fi8Dz4855phyhx3YomOMMcaYusUPOsYYY4ypW3JxXdHMpcF0aianOZhBx0AyyKsWXQI6vjFjxgAAdtpppwY6oGDmX3PNNUPHsuUA8NdffwEAFllkkdDlEfSoc1LXzJFHHgkgGYC8xRZbhEw321ZbbRW68ePHh3zqqacCAP78888Kj7hx6DFdcsklQ77hhhsAJNdE3azffvstgMI6Ac2zxLzOSd05dGncfvvtoaO7CgBuvfVWAMA666wTOnUd5EGam1Vdv9dff33INLk3plGkXqsOPvjgkI8++mgA2bjrdHzfffddyPvuuy8A4KOPPgrd7rvvHjJdryuvvHLo1PXBz91///2hUxccXetA4RqmbpQs0HORwex77bVX6PT4cv3UdafHh8HyOues14fupj333DN0WntLf58uHdXpd7EmkK6Dnl933nknAOCll14KHd11QOXuFXpO8fqt4Qp04euYgcL9+ccffwzdxx9/HPI222wDILmnOCcAuPnmmwEAhxxySOi0nUdT5meLjjHGGGPqFj/oGGOMMaZuqZrrKs20//DDD4esrpELLrgAANCxY8fQ1WImh85J61SceeaZAJLuKpYlBwquG9azAJLR+q1atQKQrEdSLdeVuqs0E+6www4LmWbMW265JXRqhmzZsiWApDmWmWRAweTJ7AogH9ec/qaOlce/mGmZ49esiubkuuJYtSM03XUA0K9fPwBJc/F9990XMsvp5+1CLpYV+MgjjwBI1t7iOQkAiy66KICka1jn+tNPPwFIZg1yTwCF/Q1U77rETCkA+OKLLwAAl112WegOOOCAkOlyTXOtAMAuu+wCAPj9999Dd95554W88cYbh0z3dDXPT/6WXlPV9XHCCScAAN56663QaRgAa1rp/LPYq2ltezQTSV3fq6yySsicn/6/un7oRn799ddDp25y/m0116RFixYACu1DAODJJ58MWV27iy22GICkO/Xiiy8OmTWN7r333tDRHQ4Ussn4m5XAFh1jjDHG1C1+0DHGGGNM3VI115WaEVnwSguOaXGnvffeu8Fn8jaTp6GmS5q7gULWipq71Q332muvAQCuvPLK0B100EEhn3zyyZUfbAk4FzWHqmlRix+yYNWGG24Yutlmmy1kukTuvvvu0Knpki457Z6cB6W65+r/a4bZ448/DiDptsi7RUdj4Fg10+Gcc85p8HeXX355yFqcLK2EfR7o8WemBlCY16BBg0Kne5XrWqyVw1JLLdXgt0qV888adRnwXNt6661DpwUb6fJ5+eWXQ8dMSaCQYaiZrixyCSTd16U6hVcKPb50SfXu3Tt0WsSQrh26sID0grJZ3zP0+3l+aDiG7hMNXaBerxnqpmFIQ/fu3RvogELow4ILLpg6lkqh46e79uqrrw6dhivo/YEtOjp16hQ6vZf/9ttvAJL3P50/XV66z8o9z2zRMcYYY0zdkulrqD7FabASa1roU6iWmGczS60doN9VK4HJ+sSrDTi7desGoPDmDySf9Dlv/bw+ndPikUfTUn2zYtApkAzWZNO9/fffP3RaFp+l2Wm5mho2gNS2C3m0tdA9pceXf6P/r+0eGISntXcY4AnUvvWRweTPPvts6HTMtGhsv/32qZ/P8/zTt+Bhw4aFzLLxQOHtWmuraNNRom09GEAJ1M71RVGLMAOLNYBcrTuc60MPPRQ6rUlDS8i8884bOr3W5rF/9fzjuqoVR63jpSyKeVgauS+LNQ8ttaf0mPO+oQkqG2ywQcjrr78+gHwCxDVBRvdUWgsZnbPe/y+66CIAyUbK+l1qyaoUtugYY4wxpm7xg44xxhhj6paZpmX+mjRpUsVsYxrYysBb7Q59/PHHh8wxqelrrrnmCjnNtdMYWrZsORMATJkypWLzU9PdiBEjABRKkQPAq6++GjLNxNqRXN1cdOk0dX6zzDLLTEDT1k/nod1ntZPwCy+8ACDZkVfLfWt9DqKd6M866yyOM3SNcRdw/SZPnjzd80sLQB07dmzI6ppinRXt/qw1IRgYeP7554dOXSdprsnG0KJFi4rsz2LB1ldddRWAZG0WDWZlkODOO+8cukqaybk/G7N+RANlR48eHbKuBTuVzz333KHTYEeuz4cffhi66667LmQNvG4KXL+JEyeWddDUTafBnnRj6DmnbmC2XmE9GSAZbEyXbFNddK1atWry+hVD9yrrU+n8tGbVPffcAyBZe0UTH9iapKltSSp1/pUirW0QAGyyySYAktdf7U7PkIKmzq+c868x6Pwee+yxkNkmQ1t0DBkyJGTWt2qqC5Xrp9iiY4wxxpi6xQ86xhhjjKlbKp51peYqlioHkl1PaTpNy2QB0l1XGs3ODC011+ad6aKmfZYj1zoPrC0AFOpAsB4NkMyAyDPrI612ApCss8KaMtqRXDs6s3Q5TbBAshw/3Qi6ZsXqlJRD2nfS7A0Affv2DVnN5Byf7uW0rKx27dqFTs3I/H/9fDGyrEmTlmkFFFyP2rF8hRVWCJn7Mos1KRfdMzrmu+66K2Sa/LXEvsJ59e/fP3TqpmO2oLaAqNb1RffMJ598EjLdvQDw7bffAkiuj7ZAuOKKKxp8l46/FrPKdH8xw0pdG+rGYx0hzaR78803Q15rrbWyGmZF0XXQDDrWZNNMpGWWWSbkvO91peC+1LYVWn+H10rWAwKS7ViymJ8tOsYYY4ypWypm0eFTnL5lPPDAAyF/+eWXIbOKpQZ46tslG6M999xzoTvppJNCpqVgv/32a/D7QD5vn/r7rBmgVioNpuObqDbKrEWKVYBlzSBtFDhy5MiQ+SasjRTV+sYnev1+fYqfHkvI9KB1cBgMrpYlDabWOka0fuiY1LrFIGpdP7UOrbzyyol/gWQw6EILLRQy32SyqKyse5K1q4BC/Ry1ImqDSAb719Kbf9qeKFZnimul66d7gdcateJpnZY8G7Q++uijIWuChlo3br/9dgDJNdNgZVYmn2+++bIa5nRRrCnu9KKf+eCDD0Km9VjvGVqFl+velN+sBjzX9f6gDZLbt28PIJkAopXna92iw3OVSTlAsko310+9HFlfa2pzJxhjjDHGVAA/6BhjjDGmbqmYvZxmQi27ro321HR/7LHHAkiai9X0TNMXg+6ApGmfrpG8y9IXc5fddNNNAJLBjvr/hxxyCIBksGMtuQlIsXYJbMras2fP1P9nsLgGi6YFHmsAr9aMUDNtWrPRpoyftX20FHmxpnpcC21rcckllzQY3yOPPBI6/d5Ro0YBAF555ZXQDRgwIOT11lsvZAbkaQuRctA5s7kjkGzHQZP/rrvuGjpdq1pB9wxd37rP1J2ogcdpTVnZlgQAbrzxRgDJtjPq2pt//vkb/H4W6Ph++eUXAMnmm5tuumnI2hSXNcW0RYKGCfz5558ACvOoBjxntF6aXtMYplCq7Yp+TusksdEzAHz00UcAkgHaG2+8cci16LLS+aU1tVQ3HIOptd1CLd4fFHUtc/+xXheQbCHEOjp6/ur8s6D2doQxxhhjTIXwg44xxhhj6paKp3rQrQEAn3/+ecja7iEtK0JNX6zToqYvzdBid9O8a3vomLUTK2sGqevggAMOCHmnnXaqwujKR+enbR3YOkC762oGEtdHza1pa6UtIFSuVNd2/ax+f9r/f/PNNyHTNaCmZZYS131lAAAgAElEQVTVV9hFeGp4rDTTTl1j88wzT8h0n1VqL+uavf/++yGr64pmdK3jlLcbmKg7kS4KoGDuLubiVDM/j6Way3lNAYAOHToAAN54443QLbvssmWPvbHoWrH2jbrgtIWMZi3STaru0qa2AygHdRGxnUqvXr1Cp/VsNtpoIwDJVj56Tui5wppIF1xwQeg0q4z1yY4++ujQFat5VSukZSNrVjHbVgAFN7ken1qcU1omIwBceumlAJJZZdoCaKWVVgJQ3T1ri44xxhhj6payLDr6lErrjD6l6hObBtYRfQpU6w/rr7CejuqAdItQtdCnWFawBIADDzwwZNbn0DeatJoytRhgpmvKoDIgOT/WYdEAQK18zberUvPL+i1Ff79bt26JfwHgqaeeClkry55yyikACm+hU38XSbMSAYXA4mLB6mlyuceC+1ItU1qHRS0arHKt61crb4x6Ti+xxBIh8y24WNBi2jHVYHK1iNBqpOunv1utY6H74+OPP26gY/NYIFmH7PnnnwdQqKcDJK1zbGaa9Tx0rOPGjQOQPE/0jZ7WT03A0NpVOj8GIauVTfcyA5PV+leL11K9V7C2EVAI2Nb7o15/avn+oOj4aJEEClWQNcHh0EMPDZnnXTXv37boGGOMMaZu8YOOMcYYY+qWigUj00y69dZbh07rhagZmn+rpittOrjFFlsAANZdd93QaQPFPFxWabVfGHQFJIOwO3fuDCBZO6W51MxRczTrjQDAY489FjJLlGsdi9lnnz3kWixRzsBUXbMPP/wwZK3zxNL5pYKpa8XdAxTGqs3z2PYCSJZb32effQAkTeu1sieLBZBrC4SmfFeaPo8A3rRxAIVrnZ5numasswMU6uNoCxxtbcJg96zXVM/zrl27AgCGDBkSOr0mvvXWWw0+o/NXN8eWW24JIJnAwWuOfket7Nli6LX02muvDZnhHTvssEPotAF0LV1XpkbdhdrWQduR8FqrOnVT5rFutugYY4wxpm7xg44xxhhj6payXFdpJby1toia7jRbgqYrbQGx2mqrhdylSxcASTNnrbhD1Nw9ZsyYkLV77q233gqgUK8DqH0zK90YWi9HzdDq5mDNHDU316K5NS0TZ8kllwzd0ksvHbLuL65VLc7p/7F3lmFXVO/3X16XhImd2GK3KDYW2ImFooLdrdiBoqCY2C0qdgcqdit2IIoiBqjY+SV88X/x/637rPGZ4YkTc55hfd5wXzfPOWf27D17Zu5sDHXx8DoCku066Eat9zWptKZjbSq6l7Cjs+6fzz//fMjrrbdeyAsssACAZD2mvOvIcH/Q7KE11lgjZLpLs0hrAaRukrzdjE1F73maNfzss8+GTNdOjx49Qqd1oOrlXqdwXHof17YjzDQGSq1lVlxxxdCltWipJbboGGOMMaaw+EHHGGOMMYVluqmZkSZNmtT6bPdNoF27dtMBwJQpU5o9PnXhvPDCCyFrVhXdWHmZINu0aTMd0Lz5SzNNanduHTcz69S0XEtzJOdv8uTJhVyfbdu2rcj6zCpYmLcbiOuz6PM3ceLEZl9/OmeNUalWKc2lffv2zZ6/lo4rDyp1/WnxTi0eu8kmmwAArrrqqtBpGEe1r8+WXH8c1++//x66nXfeOWTNsGMh1lVWWSV0tdxzOH+KLTrGGGOMKSy26JRBlkUj72Cyllh0iL556fgUBgbm9eZli07rxhad1k1LLDqtiUpdf2rFGDt2bMhsbMp6XUBt99Jyrj89znHjxoWszXZZM685VrxKYouOMcYYY6Yp/KBjjDHGmMIyVdeVMcYYY0xrxhYdY4wxxhSWqVZGLnowctGDBR3M2jqZVoKRPb7WCcfn/bN1Uk6ySmuA93fFFh1jjDHGFJayel2loSllWTLJKnjluCFjTCVp6V5UFHTMLP5WT0Uki07a+vM5rx226BhjjDGmsFTMosOOs/qWMHHixJD/+uuvkPk37OIKADPNNFOD79LCe0V8y9KOw1o6nE/6eRceLJeWFoyqx7lOK6SoOp2r1j5vRSHtmgJKZey13cnMM88cMsvx1+M6VHR8aWPNKmL666+/AgDatm0bOm1BkHcn9KKg+8PkyZMb/H+7du1CtnWnIVnFa3UtN/W82aJjjDHGmMLiBx1jjDHGFJayXFdq4mSn1gceeCB0L774YshPPPFEyDQZr7HGGqHbddddQ958880BlHpmANlm6NaCHj97RX366aeh++ijj0JeaaWVAABLLbVU6FqTCZnrQvuf6Jxx/GmfAYAZZpgh9XO1gnOlc/bPP/+E/MorrwAAvv/++9CtvfbaIS+44IIAWuc6LQI0c//yyy+hGzp0aMgXXnghAOCHH34I3XbbbRfyBRdcAABYYIEFQldPc5nWSfrnn38Oee655wYAzDrrrKF76KGHQu7Tpw8AYMcddwyddtLWa7FS8Jiz3Nm6v/Fv9PprjDQ3Xd57ph7/kCFDQn7jjTcAAJdcckno1HVaT2utHJoy11P7nJ6H119/PeTlllsuZIa8NHbObNExxhhjTGHxg44xxhhjCkuTXVdp5sSPP/445N69ewMAPvjgg9AtuuiiIe+3334h0zT6+OOPh65v374hX3zxxQCA3XffPfX/m2quygueI3XRfP755yHfcMMNAIC77rordGpGX3755QEkzc1qRq8Xk6weh4716aefBgDccsstodNIeWbgqYlcMxA22WSTkA888EAA1ZnrLNMq3bCPPvpo6F5++eWQ33//fQDAqFGjQkd3CAAcc8wxANIzLeqVtDorLSWP+ix6zJyfE088MXQvvPBCyMz2bNOmTegee+yxkHmtDRw4sDoH2wJ0fHRZ9e/fP3SvvvpqyLPMMgsAYJ555gnda6+9FjLnZM4556z4cWZdUyNHjgQAfPfdd6HTTJpll102ZF5Xen9gphiQvr/qnrHTTjsBSM5v3pmQek3cfPPNAJL3tGWWWSbker2vNRddC42NSf+W94U333wzdOrmu+6665p9LLboGGOMMaawNNuiM27cuNAxqA0oPYUNGjQodLTyAMmaOaRfv34hP/fccyHz7Vjfkv/++++Qzz33XAD1VYdA3y749n/jjTeG7qmnngqZQYJ77LFH6GabbbaQGQz5ySefhI4BrkA+Fh39TVqf1OIxePDgkLlGtI5S9+7dQ95nn30AJIMl9Y1rtdVWS/3dSqBvDjpnV155ZcgvvfQSgORb1sknn9zgWPfdd9/QaeB43msxDT2PacGmGmz95ZdfAgAmTJgw1e9Si4BaHDSYfOGFFwaQfHuvFPqdGsy//fbbAwB+++230J1wwgkhb7HFFgCAe++9N3TXXHNNyLSO/PHHH6Hr0KFDyHlcf7qmGLiqFp1bb7015EMPPRRAdh2SbbbZBkDynGhNnXLGp+tg/PjxIR900EEAgM8++yx0anHRNUOLrwZbpyUwKPfdd1/ITHxRK8Dss88ech7Wnc022yzkueaaCwDw4Ycfhq5Tp04hc97ytty3FO4vtIwDwLPPPhsyPTW6VnSt/vjjjwCAM844I3Rnn312yFn3jalhi44xxhhjCosfdIwxxhhTWJptT1Zz07HHHhtyt27dAABzzDFH6BprlKdlx7feeuuQWV+nR48eobv++utDpml6gw02aO7hVxQ9F6ytApRMw2ruVjcfA2wXWmih0B155JEh082nAXp5mDF1fHTnAMDRRx8NoBRgCKS3s1Bd165dQ9Z5TUPN9NUct/4OA8CB0vF17Ngx9W/pxlpxxRVDp665xszstSItwA8AvvrqKwDA/fffHzoN/KOZ+aeffprq9+v6pYsKANZaa62QjzvuOAAlc3256JqkiRsATj311JBpMqeLGygFiAOlc7HyyiuHTuvQ3H777QCSe466efKYXx03zfUagKyuK7oU9TPqxmOwtbpz/vzzz5B1X6bLq6nXof6dfj9rg2XtGRqasOSSSwIAttxyy9BpYgvRYH8mQADAsGHDACTDJTQMolZ1uvRcaDIJ73VHHXVU6DbaaKOQed5ak+tK1xrr5J100kmhU9c4QzaygpXPP/98AMCqq64aus6dO4fcktZQtugYY4wxprD4QccYY4wxhaXJriualuabb77Q7bbbbiHThJTldmisk7V+jma+Xr16he6dd94J+bbbbgMArLPOOqFT01mtTH56zKusskrIzzzzDIBkHQt1gzDDQbMC7rjjjpBZ/2P++ecPXd5mzLFjx4bMbBwdk7omaKb84osvQqetQWi61KydWmUq6XnUTBPNikjrHn/PPfeEzPm9++67Q6dm+DxrduhxqGlf3VTMYND5UZczzejqzlt33XVD5rrW86efV5dBc10fzYH7AFByVwDAEUccAQA47LDDQpeWbafHz0xHoJQZpK4hbWeiGUzVvC51LvX3WX9LQwd0/7v88ssBJPekvfbaK2S6LpndBCTreC222GItPma9jtUFxgwxdZGOGTMmZHVns36Ruv7Tzrnq1lxzzZDZYuGtt94KnbYD0X2rmuja0Awzuq406+/5558PmeeiHrM3lTR3KgCcfvrpAJLuRM3QTcv61BZRlPXzzanJk3qszf6EMcYYY0wrwQ86xhhjjCkszc66ynJNUc4qUqWZCmnm3v/9738hs+CetkBQc9cKK6wAoPxS9ZWEZdeBkstgwIABoVNzKTMc1PXBSHMgmaGVJzrXPXv2DJkZVDqP7JgMlDIotHikZrAw82L99dev8BE3Dz1+ZgoApXWrRR6PP/74kFlccPHFFw9d3iXmeS3QLQEki2zdeeedITOrQ9fctttuGzLXanO6WGe5cCrl2uGxaBFDdSeqm/iAAw4AkMyuScuU0mNTN8kiiywCIDn/WpBw9dVXn+r3loOec217oO0smBWmWX/qBl977bUBAN98803o1PXcpUsXAEl3irrJK+Wa08+yIKq2PaCLEUhmrdKlpYVD9TxzravrTe8VLPSoYRbqpqwVWaEbLFipmUSXXXZZg/9X11+9uLF0HCoPHz48ZBbKpQsVSF4zRNuB6L2Srku9pssdf/08KRhjjDHGVJiy6rLrE2vaU7a+cY0YMSJkBtZpgN33338fMoPI9C1bn25pUdA3D/3bWqEWJS1xveuuuwJInos0tMXADjvsEDLf6vKunZOFtqNI+xytW3vvvXfotEUEA2M1mLyxYPVqk1afRWtv7L///iHzjSvvtyw9ZgbOskYTkGwkqxY3vv1rnRK1fqQlFuRNWgua119/PeQzzzwz5CWWWAJA86wtaY1I1eJRbYsAr3mtd8MaRAAwZMiQkLn+1GKa1oJk9OjRodPv5Vh1T9Xrr1L7TprFX9ecWgG0wTGvNbUCs7YaULo+zzvvvNBpo0e2CNBg7TwSHxQ9F7zWaFkDkskCHB8ti0D9XIu65+i1ePjhh4fMFj96T1No/dY5U+sNEx8aq8PXHGzRMcYYY0xh8YOOMcYYYwpLxVoK0/SpHX/VtKpmVJq/NFiMZb8BYKuttgKQLMv+3nvvhUzzvNbO0O7fedRk0doT2223HYBkgLUGDr7//vsAsk23Bx98cOUPthHoBtRgT619oJ3UGZipAbha9p9uEnXnqemfLTw08DJv06weC2s2ac0PDUaulyD4NNepuqt0fWpgK9eXrtlddtklZHZl1wDdvOeHa+2RRx4Jna6pnXbaKWTOZWMB4nr+dN2PGjUKALDJJpuETtuxVCrwXN1FDODXjs0333xzyBosTpO/1q7SY+K1nOauUnSv1hYM2q6gUm4srh91kZ5zzjkhzzvvvCFfe+21AICzzjordNoihy6txx57LHTaYoYuq6xkhzxCAtLcMOrav/rqq0Nmna7evXuHrhquxebAa0rbrujxaQsLzlu7du1Cp2EqvJer60pd6wxmVndYueOvjx3bGGOMMaYK+EHHGGOMMYWlYq4rmibVBKUl2rXEOM1Q2vFYOyHTvKlZS1pH4pZbbmnw/Vnl3qtp5tPv1poqNL2qOVnr7LDdg9bpYKYA0HTTe7noeXr33XcBJOtcaAluNfMzw0OzUjRDiedFj79Tp04hL7300g1+P290LpmhoXPG2ixAqc7DzjvvHLo8TMuaVcT1rx1/P/zww5C1O/Xvv/8OIFl2Xbt/s45Vv379QsfaVUBpXqs9f/r9dLOoi1ezLjWDqMkdjWVNq2udXdvVNF+NFh/6nXTH3HDDDaHTrERtUUGXlR6HjoUu88cffzx0mpXKbE91LWhNE82AqXQLD3WB6r3itNNOC5kuN93fNWuX+466NtS1xe/NqvmWNzwWPc/aNoV1xrLmt1akXd/PPfdcyNq2Qt2M1HOf+e/neF/R2k1XXHFFyKyfVMk91RYdY4wxxhSWill00hqtaTXEtCdSfeJOe/rWRoFshAYAQ4cOBZB8otQ6J/qmUs6TYGPHrKTVFMpqesa3q6zKpNVEj0mtbIceeiiAZICiBvuppYDn9+uvvw6dBp6zCi/rmQDApZdeGjLfZPKuJqyodWSllVYCUApKBYBbb701ZFqvdE3ONNNMIdfq7TGtsi8D+YFSvRUg/VrTt2h9o7rooosAlCooA8BVV10Vsl7j1UTf6BgsrMGQamXSY22MtMDKU045JWQmDqy33nqhq1Qwtl5/ev1wfeneddJJJ4WsFlFeNzoPmvhAi6Nav9T6wSBYPRatM1Mr64GuXw2c5v6o8z9x4sSQWadFA7c1ML3S1aorDcete4Ym49C6rgG8avHJM5ha50HvX7r/cy1pvTWOCSglIWlTU/1bznslx2mLjjHGGGMKix90jDHGGFNYmm2DVnNiY64ddU2o3FSTlP4d3QlAKXCZzTP/+7uVQttScNwaQNbYOPRcaWA16+do7Z+0Eu7VQOdMm/59+umnAEpmYSBZh0JNlixXrgGSaoZnTRZ1B1SjDkm1oGtD52fNNdcMmcHweQc4pgXrZR2TzjvHp/VSNAidZmZtlKi1nViOv5bzmNZgVK8THR/PS9ZeRTeJJjBoMDIDgtU0X+41mWaOV9cSa1NpUPimm26a+l0ci7qe6a4CSkG86s7v379/yHTzZAUGV7L0flPRBrR0aahrR4PpteYPybvOU0vQ9anXEoPh8x5TWlNSreekdbb++eefkBlEry7EXr16hcwwBq3jpcHy1VhztugYY4wxprD4QccYY4wxhaXJriuartTERHcHUIr6X3755UOntS00QyDNjJtmhlZzs2bAjB8/HkDSHVJupgCPSbMXjjjiiJDZqVV1mnWjUfFp9QdYOwcAbr/9dgDJTBeNOq+VuVjPGWXNLtLu0Oq6YpsBzdR44IEHQl577bUBJOvQ1Lu7Ss9FWh2gl156KWTNhqg3stw5ae02fvnll9BpVhVbX3Ts2DF02q4lj6wyutk0E0pbqGgGx5577tng8+zuDgDnn38+gOScqmtu4403LvvYs9BjUnM/9wytnaN75ldffRXyW2+9BQC47LLLGuiAUgsPdYNpVlKaSySPPYcuGgB47bXXQmZNMs2q0r2SLr/lllsudOpyr0bWTjXQa3KbbbYJmW5IbYGke2kecM2ou0q7k+u80s3ITF4g2SKC2djVdlcptugYY4wxprA02aLDJzYGzQHJpzS+fWgApwbTaRATK3Pq32plYFZUZIVIADj33HMb/Nauu+4aOrUetcR6kFYHSIM12eBSj1OD5dKCHTWY84QTTgiZ9TH22muv1M9X8+lW3+YWXXTRkGmp0jdfDfZeZZVVQh48eDAAYOWVVw6dnn/+Rmuy4ujbNWuZaLA2rXBAaV23pBpvteBbrAaFa+Xjbt26hcympRosrp+jVVate2pxzKNpLi2mAwYMCN0bb7wRslbGZUVntaxqU1O+HWuALhsFA6Ug5GqMM63eluq1AreuL327pyWEFcaBZBVhWgf0+/MObCVpjUyBZFNV1oLSOlCdO3cOmXvp6aefHjqtP7THHns0+K28r0+Fx6LB7mrR4XWpc657dZ5kVZvWc33llVcCSFoZX3nllZC5Lmu5Jm3RMcYYY0xh8YOOMcYYYwrLdFMz6U2aNCn+k+YmDWBUd9LYsWMBJF0AarrSOjI0HavpVQPLPvjgAwBJN5macRmsRRMlkDQDNmambNeu3XQAMHHixAZ/qCY4LRG/22674f8+E7qePXuGrG4aNi17//33Q3f00UeHzCAtDRCspGm1ffv20wHAlClTpvqlaUHTGoytx8RGq0ApiC6vpnlt2rSZDgAmT57c7B9Vc/6YMWNC1joe8847LwDgySefDJ2uz0suuQRAsqZSJd10bdu2bdL8KWnB9OqaeeSRR0LmtaotOnr06BEya14sssgioavk/HL+WjI+rc1BFxwA3HTTTSHT5a11ZjTAmPVl2DwQqGztmLTxpQXIMqkCKK2pq6++Wr8nZHXjsEUFWzkA6a7Fal2THF/a/tkYev3pveSYY44Jme7zPn36hG677bYLmUHo6sbUa5HNTDWYvjnXZ1P3z3LRYGQ9F3TdqTuW9x+g/L2G86f395agc6k15zhXWmeta9euIVd7ffL+rtiiY4wxxpjC4gcdY4wxxhSWJruuiJqr1IzMnHj9PjWjp0VYqwkurf6Hmvb0d5mtlVYbpClMzXWl6PfTzKxdWp955pmQ1Q1Fk/hRRx0VOm3xUO06D+WYXtPcWUB9ZS2U47rSOVXXqNZ8oHtSs+K0TgfdqNXKGmiJ64rzo9eJZs2pG5ZuZDX3a32MNNdkJWmJ64pktXXQ2kYct+4v6nrltVqtVgdNHZ+uRR4zM06B5Fg1w5MZRjr+WmY4luO6UnR82uKBWalDhw4NnWZV8b6jdVi0RdDdd98NIJk1W4+uq6ysMIZ/6Pzq+q2Ua7Vc15Wu37fffjtktshhvSogOX/Vzray68oYY4wx0xR+0DHGGGNMYWm26yrxYTG9pbk8stwg5VJu1HZTXVdKWkfkrN9Pc03V0vVTK9NrXpTjulIaW79KLeeyJa6rxkhrkZLX+izHdZVFY3NZy7GWk1WW1com7fjzcidXw3Wl42bIw7Bhw0L32WefNficZuKy7QxQykpTd0lzzlUe+2dj67eS7p5Kua6UtLnMKyvXritjjDHGTFM0uQVEGnm9EeZBc96iin4uisK0tH7rpQVAtWjtc5nWSLbo6DzpuGmJ2WmnnULXmMVV1ze/qzWtg6KsX6A+17AtOsYYY4wpLH7QMcYYY0xhmWowsjHGGGNMa2aqMTrlRtXXK4yqr2TUeWNocaXpp///p10LXlUyhoJR5+VmJdUrzEqq5fzVEs9f64bzV/TxeX22TqaV9anYdWWMMcaYwlJW1lXeZNUeyDuCPa0mxieffBIyOy337ds3dHPMMUfIrSVDJuv8p9URai1jMsaYeqOe6mC1BD3+POpA2aJjjDHGmMLSaiw6aZUX2XwRSDYY1QZoecBj/eOPP0KnTSPXXHNNAMnKnq2JtCrRjz/+eMhs6nb00UeHTiuX1mOdhebAeKus2hFcn3k1XawVbdq0CTmtjokxtSLNoqy6vKr0loPuHyNHjgy5X79+AIABAwaEbtFFFw05b+s5z7vekwcOHBjyHnvsAQBYZJFFQlftPcMWHWOMMcYUFj/oGGOMMaaw1LXrimnYAPDPP/+ETDfJQw89FLovv/wy5F69eoV8wAEHAKi+OS+tRPkll1wS8qyzzhryWWedBQBo27Zt6PI2NzYHjlXNqYcffnjIv/76KwBgzJgxoXv66adD7tChA4DWNWblww8/BFAaBwAstNBCIbMp4bhx40K32GKLhVytZrfNJes40kz7aa6BN998M3Rzzz13yLU0SaeR5rpQN0Bj55/HnPf61JIULQns188r9eJa1DlJc/M25mLSeVQ3CUMG9DtnmWWWkLPOa73A4/vpp59Ct99++4XMxJYTTjghdOq6qkeee+65kNdbbz0AyT3RritjjDHGmBbiBx1jjDHGFJZcXVdZdVhomlXT+LXXXhvya6+9BgA46aSTUj9/8803h7z//vtX7oCngppJf/zxRwCl7COg5K4CSllhWhm53kmbHx3fPvvsE/Liiy8OIJl1ddddd4VMd2JW7aO84bGoa3H06NEh9+jRAwBw//33h47uKgA49dRTAQDzzDNP6I455piQs1wKtYLnPcsFkuba0fX96KOPAgCOOOKI0N1xxx0h19IkTfT46Mb46quvQvfGG2+E/PLLLwMAJk+eHDod85FHHgkAWGGFFUJXKzeWHse9994b8vzzzx/yOuusM9Vj4rm4++67Q6fzsPvuuwNIuntqha59rS3GOQGAPn36AEivvZKFnrcrr7wSAPDkk0+G7rjjjgt52223bfC5vPcfPS8TJkwAAJxyyimhe++990LmvqLu4npE19zvv/8eclpNoGpji44xxhhjCksuFh0+0ekbxXfffRfyoEGDACTfmLfccsuQX3zxRQDAvPPOG7pXX3015NVWW63CR9w4+pTKIFwNQO7evXvI9RIM2BhZQZsMLPv6669Dp9YdzuuDDz4Yutdffz3kvfbaC0CyjlDeb1T6+wyC//bbb0O3zTbbhHzYYYcBAJZaaqnQqfWK1KMVByhZAk4//fTQde3aNeTNNtuswee///77kGlJVStely5dQq6VpVLPqe4fHJcmK+gx8XMaTM63aADo1KkTAGD55Zev8BFnw2PS83zUUUeFfOmll4bcWDA11/LDDz8cOl2/ecD9cdKkSaG7+uqrQ77nnntCZrDqsssuG7q0PVP33D///DNkegLUijdixIiQt95665DzvC71+MePHx8yEzu0Ntmcc84Z8m+//QYguSYuuuiikPMOouf6/OKLL0KnNbdo8a/lcdqiY4wxxpjC4gcdY4wxxhSWmrmu1ETImjgXXnhh6NRM99lnnwFI1ibp379/yDTjaW2dW2+9NeSDDjoo5GoGm2UF09KNtvLKK4durrnmSv3bekZNqxpMRpfMnnvuGTo1TdIltf3224dOAyPpRqinFhhas4lj7fmSgNwAACAASURBVNmzZ+jWWGONkLm+rrrqqtBp4ONLL70EILtFQh7oXLIOEIOKAWDXXXdt8Bld38OGDQuZpnNta1KrNa3HpGvyxBNPDJku7wUWWCB0dJcCwMYbbwwgGcx5+eWXh/zCCy8AyHY9VnMv+euvv1J/U9dfY3WOWMdK3TV777135Q62BXD9aYC4hiZoYDhdxuo6VNdVWosBNkoGSskqWltGr2W9LvOs86TutkMOOSRk3gs1aHqHHXYImWv9l19+afCd9QDnWue3Y8eOITOY2q4rY4wxxpgK4AcdY4wxxhSWqrqu1PRKczcAnHnmmQCS7ibN4LjssssAJLt/a9Q5TbdPPPFE6GacccaQNRummiZ1NRdqNgFdV5q9oq6DvN0YU0PHpJkqnDOglJWi5tY0N57WU3n//fdDZmlzzUqrFWnZVUAyg+zYY48FkKxdcs0114TMmjnnn39+6Fg7Byi5TPKoU6Lo9cfaTgDQu3dvAMmO8sstt1zIPEdqWtcMGbruZpttttDVak3rnGlbEc3wYwYZ9xEguRbputBrduLEiSHT5ZxHvQ91pajrrbFrRef6o48+ApDM4NLzxr+t9vrUPYGuKXX3aosDdfPTZZXlVuLx6/1B3SQMaWA9HiCfmkhKWs04rdOl88v6VFpH5/nnnw+Z51JrxOWdyZsW5kAXIgAceOCBIXMt1vKYbdExxhhjTGHxg44xxhhjCkvFXVdqQv35559DZtlxoNTVeejQoaFTN0+au0l1dB0MHjw4dAMHDgxZo+pr5T5Q0zfljTbaKHRpUfH12AJB5++ZZ54J+b777gv5uuuuA5B0F+rxc1ya9aBrgV3P1Z1QK9TEqhkuu+yyS8jMCrjxxhtDpxli7Eq/9NJLh07N5HmakXV8mslyzjnnhDz77LM30Ok1w/n74IMPQqfl+tnuIW937N9//x2yrj+2WNGsLG3RQZeVus7Tuiunrelqwd/SUv/aBT4tQzGrhQ6zcVSnbshvvvkGQDLrrBrj0++km0qzL7Wj+AUXXBAyXXZpmVZAad7PPvvs0Ol5W3PNNQEkXT9KrcIZFF1/5IcffghZsx5XXHFFAMl7ihb3ZAavZuLlHQ6h9w22y9G2OauvvnrIvCc3p8VHudiiY4wxxpjCUjGLDp/ONOiYpfKB5BsTm+ZpWwR9Ik17utNgOtYsWXDBBUO30korhVyrN2p9etdgT8paYl5JG1/eb8d8ImcNDiBZ50itL6xDouNIG5OOQy1r+qZda3TOLr744pB13LRYaFNOfTthufUhQ4aEbqaZZgqZ60/Hr3I135517bO5IQA89dRTIbNOh7ZQ0fnhm2bfvn1Dp8kCDNLWYF69Pqv5dqbnUYOp1XrI/UFbAOjbL4+V9XKA5FjyCJLn+ddWCFp7RM8vz4GeC533L7/8EkCyzpg2YN1qq60AJFsIVAPd0zgnaoXT9af7Nz+XZbGiJU4bPeuaY+Cr1i7L2lMrtddy/9QAa210q8HEaTWFNHB8mWWWAZC8f7CtEFBKfMgjWD4LPf+ffvopgOT49t1335DnmGMOAMlkjoUXXjjkaty/6+dMGWOMMcZUGD/oGGOMMaawVMx1RROg1hPRYE/Vs6swg1qBpGmLZHWnve222wAk20JoMFQewaBqBmfgrR6TdsI+99xzAZSCQoFkp2LWDMrDhfX555+H/O6774Z83HHHhczA1azzTDPz8OHDQ8cAUaDkBqtl2XLOBWuMAMmy/9rCgcendYTOO++8kNn9WDt+a/dhzr+6HtS1VSnSzt8NN9wQsrqedH3RTXfFFVeEjm0hgJKba+zYsaHT8XGNaFsFDeau5vWn373kkkuGrHVUbr/9dgBJ16K6zmlm1+/q0aNHyAxy1eu3Gteizh+PRQPAtQ6J7n90SWltKg0spl7dCRosTzdWNYJB1cX28ccfh8w9T8esAboapEzXmp5zTWbgNazzpzVp3n77bQDJAFgNfNb6Twzybur4sxJI2LZIrwl1R2lNJyYx6Pp65JFHQj7ppJMa/NYmm2wSMutY5R3uoL+vLknOJYOqAaBbt24hs2aX3r+5PoCSa6uSY7JFxxhjjDGFxQ86xhhjjCksZbmu1PT2yiuvACiZDQHg4YcfDllNqzvuuCOApDlWzYBpUffanZblwrXVQ94lsDXDgfVl1DSrGWh0Wak5luZ2oORmaCyrqVz0/DLr4/rrrw+dHr/WBEpDzZh0jeiYVl111ZCZVVBLaAZ94IEHQsfsMSBZIp7nRevQqGuSbjgta65uBGaQqDl25plnDrlSc6nzR9fSGWecETo1/WrNmJtvvhlAsg6QuqlYWl/rVGknaa5f/XwepnP9TXVjnXzyyQCS7qgTTjghZLqE1B2gdUro5qh2Da60diR6zahrX9ci15Jmrer1NWrUKADJTB+tKcN9u9ruOM3AZaal3jN0z9Tu87ruiIZB8LypO1h/l3uYujPVpaZdwZlZ2tT7h87Z66+/HjL39yWWWCJ0mvWoLS54LPpd+vtp9z+9V3Jd5t2xXH9f5/LZZ58FkKzDphmSXHcTJkwInWYbsrWQXVfGGGOMMU2g2RadNCsAUAqCZNVYIGnF0SdWDVJK+38+8WoAIQOYgVJ9gryfaBU9ftZv0KaDGqx11113AQAGDRoUOta+APKpkszf1KdstXKkWWGyzv+jjz4KABgxYkToDjjggJD5RpqHFUDfAjXYU+uMMEBRA7PV4sGaDwsttFDoGEAIlCraaoBkteeUgdMbbLBB6NZdd92Q9e2f16XOtf5/v379AAAHH3xw6NICc3XN513ZW4+FwfJaD0eTAbgGdM4XX3zxkPNoxsrz17lz59CtssoqDf4fKF13ev1p5WQGoWsjZLVoVNP6nVahGShZdDQoWIP91WLFBrtZtdXatWvX4Le0DhItrrp+tU7Wnnvumfq9U4NWFrWsqJWe60c9DzrWtMrOar3TytBsmsxxAsnKz7wv7rzzzk069mqh504rq7OmVVptMf2cWtSZYARU575ui44xxhhjCosfdIwxxhhTWJrtulITttZ8YODZaqutFros1wTNWFl1HGiaY6sIIBmYx9L8eZiYs9B2Ae+88w6ApDtAgyVZYv/VV18NnY6Vprs83AE6Z2p6VNMyZXXNvPXWWyEz8FHdXccff3zINHPm4brq2bNnyGpOZQAnUCphrsGG6mY89NBDAWS3PaBc7fnT80d3mtYjUdQcTJfBmWeeGTp1J++3334Aksev899aUNcA65wAJdfO+uuvH7q8kxmIzqnutWlk1aRh6f0uXbpU+OgaR49f28bQzfvdd9+FTu8VrE0FlILctamlJrawhYm6S3QumezSq1ev0GW5kZu7B+kxaYsD1qTS2mg6JxqaMGDAAADJBAGt+cN1y6QAAFhrrbVCpks970aeiu6FnL+sFh48/3r81W5nYYuOMcYYYwqLH3SMMcYYU1ia7bpSE9PIkSNDZp2DtOyA/0IzuH7+mmuuCfnll18GkCzhrqa9enFZ6Vi1xDW78mokvXbsvvPOOwEAvXv3Dt16660Xch5mdM6rdnkeOHBgyIcffnjIzPDQ7sPDhg1r8F1a9lxde3nMH82kesxau0PPOWs6aE0ozeDg+OrF3QGktzXQ60+v23vvvRcAcMstt4ROOy3TZVkv11lTUNP5J598AiCZAaNl/5mZVqtMpJbSmOtT51RrctHluuaaazb5uyqFnsdll102ZGZiquuK2UVAMoMx7b6hn2MtLM2U03YudJnpsVRjfjVrjNeSZvexngyQ3B95/1PXsbZAmn/++QGUXOhA0s3z2GOPAQCOOeaY0NUywzPtd1g7Dijdy958883Qse0DUHLJck0A1b/+bNExxhhjTGFptkVHn+KYLw+UglH1yTSr6SGfdH/88cfQbbbZZiE/+OCDAJJP/PUUeEX0XLARGVB6Y7700ktDp3UcWD+AQZ9A8om8Vm+XadWotd5NVuVK6rV2x4Ybbhgym7WpFS7vN+a0AG+tvaFj4ZuavhFXowFitdFj1jdNVg4+9thjQ7fFFluEnPdclQstGhoAuv/++4fMwNV63FOag1o+tHLwuHHjACTrmNWKLIs+A1S1mraSZj1UndZZoUVALeIa+Mz1W43rVJsTb7755iHTCzF06NDQaR0jTUZhMPymm24aurQGnVqHR++PDMLWOdf7T61Iu38Apcam2khY7/88F1dddVXoNDC7GvuPLTrGGGOMKSx+0DHGGGNMYZluaua9iRMnNvjPrBYQdF2p6e6jjz4KWU2WrAmgJerTGgRWy7Tcvn376QBg0qRJFbNtpgV+qjlP/7/aJfTbtWs3HQBMnjy52V+qx6zBdh9//HHIbIfQoUOH0KmbivqsEu7l0rZt26rOHxs8agCrBv5V23VVzvylrTMg6ZJkA1INltS5rLbrqlLzlxWM26dPHwDAiy++GLqLL744ZLrXqxVszfmr5PpMQ4OpR48eHTJrImmdMw0DKHd+K7U+s64j/o3Oj7bDGDNmDIBk7SsNoyh3fFNbn7rm1PXNYGmdh44dO6bKbE3S2HHqb2nTTJ4XdW01Z0+q9vqka1EDyLWdE49f3XnrrLNOyJVan4otOsYYY4wpLH7QMcYYY0xhKSvrSk2nzJ3X7rtaNl7Ldad1n21NNTvS0PNC01trzF5Rd4fOr5Zr5xynjfm/cmtBzcR0w6m5X/+/tYxP51Kzylg/SE3frTEDSfcP7e78xBNPAABWWGGF0GmGTGscaxp6/Wm7CGZwNtZCIg+a42LRa07r5LAmmWY11WpO9Xc0U5ZZX0sssUTq32btlU39Lc324rqv13XM8bEeEJDsGE/0nGiYRDWwRccYY4wxhcUPOsYYY4wpLM3Oumr0CxvpWPpfOQ+qkXVVT5STFdEaqEbWlZJWgr6Wa7Ya85fWHTiva7JS86fzpFkpgwcPBgB07949dOp6rbbJv1ZZV2kd6YFSwUQdM9t6AOXPdR77S5obrjVlddYTtVqfeeGsK2OMMcZMU1TcotMasEWndTOtvHF5/pqOWjeY+JBXgHzeb8xMIqjHOl2tgWllfyn6+BRbdIwxxhhTWPygY4wxxpjCMlXXlTHGGGNMa8YWHWOMMcYUlqlWRi56sNKUKVMKOb42bdpME8FmHl/rxMGsrRvOX9GTVbw+WycORjbGGGPMNEWze12ZppNVPJHUUxHFcqmngnRTI20egPo5PlM7vBaMmTawRccYY4wxhaUuLTos961lv1tTd2y+Keqb4Z9//hkyrR8zzzxzAx3Qesanx6zj47hnnXXW0FWrXHtT4THruWVhOSA5ln///ReA3+yLSlo7gbRO4NUquGeahl6TOmdpljjdXzhvrXXOONYsi2Pa/aU1rVUev46v2m1ZbNExxhhjTGHxg44xxhhjCkuuris1XamZ8rvvvgMAvPPOO6HTTrzzzTcfgPoMdgVKY3n00UdDd8wxx4S84IILAgC23Xbb0G299dYhL7nkkgDqa3xpZtRx48aFvPvuu4f8448/AgD69esXOh0re/FUe0x6zBMnTgQAnHfeeaFbZpllQl5++eVDXm655QAkXVvVNq3WkjTXY1rgfGs1jZOsZADuL08//XTofvvtt5BXWWUVAECXLl1CV9S1UI/QdfPLL7+EbtSoUSFz/tSdtfTSS4fcqVMnAMk50zWbtxudZN3/fv75ZwClffS/f/v1118DAOadd97Q6V5W72uVe7HO7/zzz1/V37RFxxhjjDGFxQ86xhhjjCksNXNdpZnp1Bz++uuvh3z66acDAF566aXQqevjwgsvBFB9c1dzUNPj77//DgAYNGhQ6GhuBYAffvgBAPDiiy+G7uWXXw75+uuvBwDMNttsocszUwkAJk+eHPL5558PALjnnntC97///S/k7t27AwDOPffc0M0zzzwhd+3aFUApu6la6Jw8//zzAJJzombdueaaK+Ttt98eAHDGGWeETo+/3rPi0kjL6nvrrbdC9/jjj4dMl/EWW2wRuv3337/Bd9WrCystw+6xxx4LecCAAQCATz75JHS6FmaYYQYAyTGfcMIJIc8yyywhl3MO9PpqLBMsTa7X899UslyLI0eOBACceuqpodN7AfdXXdMMZwCABx98EEByz9L9l/sTALRr1w5Abc9lmmt46NChIV9xxRUAgAkTJoRu7rnnDpluqrFjx4auR48eIR900EFT/a1KkeXa5rzo/DBcAQCeeOIJAMDNN98cujvvvLPix6fYomOMMcaYwlJVi44+pevT3+effw4AuOCCC0L37rvvhvzHH38ASL45DR8+POSNNtoIAHDyySeHbp999gm52paCxuDb2UorrRS6jz76KGQGu/LNEQCeeuqpkPmm3a1bt6oeZ2PoE7keHy1OZ599dug0WJyBvfpGpoFnWfUhKo2uuUUWWQQA0KdPn9B98803IevbPd80vvzyy9ANGTIkZFp/6t2yk1Wn4tJLLwUA3HjjjaHT87LnnnsCAK699trQrbzyyiGvs846AOpr/GoR+eeffwAkLXJcswDQsWNHAMBpp50WugUWWCDk8ePHAwDuv//+0L3//vsh33HHHSGzFlZTgz51TnicAPDpp58CAGacccbQ6f43++yzh9y+fXsA2UGnXPf1GIiqZFmMBw8eDCBphVt77bVDZuKGnh8G8ALAKaecAgD47LPPQqeJEx9//HHIiy22GIDarmVaN2677bbQHXLIISHTkrjLLruETtfnwgsvDAD48MMPQ8drFgC23HLLkBdddFEAlbPo6D2B1wkA7LDDDiEPHDgQQOk+/V/o0dB7/t9//x0yr6lKWqFs0THGGGNMYfGDjjHGGGMKS81cV2r63XnnnQEAc8wxR+huvfXWkJ988kkAQP/+/UO30EILhTxmzBgAyQCzvFEzMU2qRxxxROi22WabkFmnQ03jGoz87bffAkia7rLcgLVCawLRtKruQj2+9957D0BpngDg8MMPD7lWZmKdE7oLL7nkktCpi/OLL74I+ZxzzgGQHLMGztHMrK7HenQTqDvnpptuCpnrjkGbQLIOCedSr68HHnggZLqu8l6TakZXNxD3DXXNbbzxxiFzDeiekpYssdlmm4WOAepAKbAdSF7XTUHnRINJ11prrcRvA8kAeU28WGGFFQAk3Tlc30CpThddHEAyGJRrNe9gZj3nr732Wsh0DfI+AZSuSQBYfPHFASTPJfccALj99tsBJGsj6V6siQW1OgdpdeIuu+yy0GkYAPcXnTOFY7388stDR3cmkBx3Nfnggw9C1rVM13DW/Yvz1rZt29ClNYWuJLboGGOMMaaw+EHHGGOMMYWlZq6rZ555JmSWttZMFi3BP+eccwIo1VsBSpkiADBlyhQAyUjzPFwHam5Lq1OitQ80m+K6664DkDSt6+d//fVXAPmblhXttJ5m+tY6OmyzoHVY1PSeR7YOj1nNwZq1ollFzPpQzjrrrJA5lj322CN0ebtx0o5DM920phGzHbVsvGa98Dt0zjWrrpq1ORpDrxO95o866qiQWZOjV69eoWMmCAB06NABQGkf+S80ravrZ9lllw35lltuCbm5rit1ly611FIh052h1wbrxQDAww8/HDJdF6wnBgCTJk0KmeeFLh4A2HfffUPecccdASRdr3mjoQ0ci+7vbIuj/3/33XeHTjM8eX+54YYbQqd12GrVNTsru5TZVswuBoDevXuHzPWnx6Z1rhgGoGO67777Qtb7TqWuUY7lr7/+Cp1mTWtowhJLLAEgefxp1+2ss84aOr0/NubGakkLD1t0jDHGGFNY/KBjjDHGmMJSsxYQ2223XcjMitCCbWqyZVfWr776KnSalZSWNVErd4iaI9WMp8f6wgsvAEhmqmjxKn5OswbUjMciT/XiDgGANdZYI2S26FBzq2bwMANGi0jlffwkq6y+mkPpmtKCV4888kjIw4YNA1ByAQDJrIc8x6quOS2brxmO6667LoCkG0XXIt0IJ510Uuiuvvrqyh9sM6A5W91NmnWi64/ZgJq1OdNMM4Wc5bIiafOn57VS86vnnO56RbODjjvuuAa/n7WWWZDt2WefDZ2627gurrzyytBVO+ulOfC86Ppdf/31Q+ZY1F2l7hq268jDXaXob6pr8Y033gAAHH/88aHT4of8HFuxAEnXIz935JFHhi4tqw6o/F6k7sC33347ZM0g5lrS49C1zrYbek7SngWy7n/aGolu6Maon9VtjDHGGFNhqmrR0acwtb6wPoWWYNfAPL7daCO9nj17hrzppps2+P5qwydSLTWub1n69sFgYiXN4qQ6tX5oCW+Sh5VAf1MDi1lfRhvJ6RsFayKplaoe68xkNZ0bMWIEAOCqq64KnVoBuJY1mFnPVdrbcVqw+n8/Vwn0PGuJ9V133TVkvv2qRYdvWUCp6Z7W42DtJyBfi5UGoGqANQMggZLFUa04zbH4cq40wP77778P+cADD2zGETeNSl4ftEiqxZVWPAA48cQTASTPib5x12p+9Xc22GCDkNnuQGtXaU0uNkNWi+uxxx4b8pprrtngt+pp/+H60nuJnn+OVeuU7bTTTiGzJpB+ptoeDZ4/tTKxuTOQ3OuZ2KDB9FqnjMkC2haJ93SgNC5NkFBZa2LRutfY/NqiY4wxxpjC4gcdY4wxxhSWqrqu1JykwZqss7LffvuFTk1zNKNr7Q51Y9FlUEtzJAOjtNT8K6+8EvKECRNCpmlS3RUazEvXx9NPPx06rSPE8eUdwKvnV4NZDz30UABJ07HOFYPJs2qe5Ikek5pDtXs5TbKff/556LbaaquQ2QJD17Sajuny0GA7NVNrJ2KWQS93rtNqduiaTCt7r6ZvrenBFgdah4Zl3YHazaUeH8vNszM1kDSXq5uGgZ3NOc607ufqWmeALwB069Yt5Lyv0TQ4bnVN6l6TVlMqb9e47h9sgUAXJJDsZM76OlrHJa0dS95zo7+v64v7v7rm2FYFAI455hgAyRYkuhZ5f6i2u0r3Su6FI0eODN1ee+0Vct++fUNmYoC2MNF2D6xzpC5GbYfB+weDtoHkXrreeuuF7Do6xhhjjJnm8YOOMcYYYwpLzeroqBm5U6dOAIC99947dFpCmn+rdUpamkFRKWgi03oH2jZATWjsRKx1HNTcTTOrZhpo93ZmyMw333yhy2PMio6PrQU0k0fN4JxXmqCB5FjpmqxmvYf/QtfOn3/+GTqtI3LxxReHTJeWtkDQcvR0mahplZlaAPDhhx8CAMaPH5/6/1r/hdl26mZoCTx/6sLSsvlqcubfaFaR1vRgiwE1rWuGWTXXoh6//g67r//999+hU9dFnz59Qm7MdcHfUHeCtsvo168fgGTZfeqAZDuTvN0jadDloOfqmmuuCZmdsvWazaMOmZ5/dRO/+uqrDT6nbhS2BtK2AdXMZKwEmtXITvOsBwck68wdcMABAIAzzzwzdHnMlZ5HZqB++umnodNM4aWXXjpkutnUHafuetYkU3d+586dQ+b4NKtXcQsIY4wxxhihZk09VWZ9D61Tos3xaL259957Q6eBy6wcXO0n27TKlrfffnvotDKlWnf49qtvHMpDDz0EAPj6669Dp4Fb5b7dVwodv9aRGT58OABgww03DJ02uGQVaDYvBUrVooHS28uqq64aOn1jq3QjOqC0VrTC6HPPPZf6ubRgcNYe0f/XOhE//fRTyHzj0N/X8an1ZLHFFgOQXD+VQmunqMWK1gut1k0rDlAKQtYA9DwCkLXOBt8CV1xxxdBpgLiSZt1Kq200atSokLUpKGuFMOgeAHbffffU76qXIHuF47722mtDp3PJ67aWlg8ek547BpgDyf2D+4Kef22A/NRTTwEAtt5669DVS2VyRceq1lPu+3qcBx98cMi0iGRVO64mWdff0KFDAZS8MUAyMWDzzTcPmZbwrGBsngtNBkmz0lTy/m6LjjHGGGMKix90jDHGGFNYKu66yip1f/3114fMYLi11147dIMHDw6ZzTCffPLJ0GkDzVqhpm/WGTnrrLNCpwGqWhOAgWdqmnvzzTdDZrNEDYzVEthsWpa3CVbnkrUPgFKQqrqjtI7FSiutBCAZbKc1MRikpu4SrSNUDZckz6W2RdD50ToP1Ks5Vf+WZnIN9lU3Stq8qW7hhRcOWc3TlUDdnlq7SU3/bPaorUb0/9nuQc9JrdypuubUtTZ27FgAyQBIDQpOa1Cq86ctHFiTRUvYa7DkPffcAyBZr0OpR3eVQjezBlOr65Uu9WrPaVrogrYQUHfVJptsEjKDzDUAXu8FdDlrqx1dC3miLhp1Z6u7muEP2siVAchA7erkpJFWOwcoJZhogHRa7SIgfV2l1fmqJbboGGOMMaaw+EHHGGOMMYWlYnZzmrwmTpwYukGDBoWsdShYf0XriXz55Zch9+/fHwCw/fbbh07z9PMwHXN8Gt0/ZMiQkDUrafbZZweQLMHPjrsA8M033wBIZkJoHRBGreddOycLunF0zGmtFbQexmuvvRYyO6FXu61AWtT/pZdeGjqtLcI5AUoui2WXXTZ0LEsOlLJCtM6OtiNoDDXJ87h0rivlztJzqq5R1nRS07rWPGJ35DzMzTpnatrnuabbDQDuv//+kHWu6KZidiOQdLOyncNBBx0UOrb1AErtMmpZ56lcdM28/PLLAJLd5zVMII99hVmrbP8DJF2DF154Ych0iXz77behUzc/9+CsrNY84P6nIRbaUV1rxnDd3XXXXaHTPSHPtaZ7utZ+6969O4BkPaCWrqM8xmeLjjHGGGMKS8UsOnz709oIGuyndSpOPvlkAMk3fq1vwsC0M844I3S1qsyq6JNnhw4dAADHHXdc6DTAVq07fLrXoCyto7DLLrsASAY2szYQUD/Bjjp+fZKfe+65ASSDHfXtm/VzWM0WSFrnaLHjOQVqN6cagMu3FCA51rSmrGk1WfQzzXlL0b+t1bj1d2hF0kaVGiDNKqV5rEO9Zrp06RIy50yYKQAAIABJREFUGx3edtttodPK6mlWMLWyqfWAlmRNIMhKomgt6FzxHB144IGhy6oiXGmyahdxT9Bq4nfffXfIGljMmi26f+ha5VpQi2oeVgIdK39fLVNffPFFyGpd5Lh1/6MXQL8rD/S3Ndg47f9bSh6WYlt0jDHGGFNY/KBjjDHGmMJS8To6aiJlPRggWefg3HPPBVCqlwMAffv2DZluLDVN5tnIEyiZ+9XFxto6QLIOCYM8Wd4fAPbaa6+QWX9Hm3aqyb5eAh/1OHQuTj31VABJ1502yGQ7gzvvvDN0GgzJ+ix5zKma+LXmRXM+Vy+uxZZC0/GDDz4YuqWWWipkXqt5jFPXnLqruT9ooz91g2gSBFvIaIsRDVbm9+r4WqO7StevtpNh+IC6/vPYU/Q3ueYYlAwk62xp09nRo0cDKLnIgWTNMjaAzqPRZRZcSzoOrT2mdeLoulJ3fnP2omqS5o6rNLzX1XJ/sUXHGGOMMYXFDzrGGGOMKSzTTc08NWnSpCbbrmjyUhOi1tHRCHuayekCAYAVVlihwXdWy7TVrl276QBgypQpzbbNpXXEBoC///47ZNYi0BL6s8wyS8jMRqiWubVNmzbTAc2bv8ZIG7d279a5ouuA/wJJM2i588r5q+T46olqj4+uGy3Br64rujy0pkYl4fgmT57c7P0lrQs5kO4mybpWq+3Gadu2bU3Wp7o7NBuNLr2LLroo9W/LHT/nb+LEiVP9Ip2rMWPGNDgmzdBl2xgAWG211QAAXbt2DZ1mBaa5HitJ+/btm70+OVZtNaKuQ80w3nnnnQGUsseAZGhAtV06tVqfiq6Fjz/+GP/3+6FTN3Ol1mfi98v6RmOMMcaYOsYPOsYYY4wpLBVzXcUXpnSszfqbrIJr1TYtl+O6UrLGmvf4quG6UprqRqjWmO26Kg/O21tvvRW6Tp06hcxCe9Vapy1xXaXRWOGxvLIXa+Ua0PGfffbZIbMFxC233BI6zfCsletKaWzPaIxKur4boyWuK5JVZDSNvFqM5OG6SivImvX/5WLXlTHGGGOmKSpeR6eWlou8mZbGqnCsedeuMC2Db5JrrLFGAx3QetZyaznOaqEWA53Lf/75B0CyDk3eTCt7RmuvsVUtalGfZ2rYomOMMcaYwuIHHWOMMcYUlqkGIxtjjDHGtGamGqNTblZEvcKo83KzruoVZl0VfXxFX58eX+tkWtlfPH+tk3q5P6QV+QRKBSFbWrCU41PsujLGGGNMYal41lVz0Kc4za1n5LrdavVNVp2IxuYtrUT/tJrBZowxjaH3x7Q6Za0RHZO2gxgyZAgAoGfPnqGjlafFv1XWp40xxhhj6phcLDp8k//3339D17dv35A33HBDAMB2220XuqLXX6hH1OIy/fSlpULf6eTJk1P/tl27dg10On/8nDZC1b9lZd7/6k3L0bcnylk1P9gAMqsarWuFmEqTdp2rTpuSpv2t3kuKuD5/+eWXkGeYYQYAQPv27UPXGq07eswDBgwI+aOPPgIA9OrVq2K/ZYuOMcYYYwqLH3SMMcYYU1hycV3RDHnnnXeG7tZbbw15xx13rPkxVZs014FSL64BPba//vorZG0AOWLECADAyJEjQzfzzDOHvO666wIAunTpEroXXnihgczvAYAjjzwy5H322aflA6ggzWkamXeJ8zT0mNRNOGrUKADAIossEjp1DbzzzjsAgNlmmy10Sy65ZMgdOnQAUEwXQRZZDXx5vdi1Xnl++umnkLlmVa8BqrrXzDXXXABa7/rk+ho+fHjodt1115Dp0jn//PNDV+9uLD0mhkF8/fXXoVPX1QcffAAAaNu2bejUNdkSbNExxhhjTGHxg44xxhhjCkvNXFdqGv/iiy8AAGeeeWboNttss5DXXHNNAK3L9Jjm5lBzHSPJAeCpp54CAPz++++h0wjzxRZbDEDSNFvt7tI8fnVxnHXWWSHfdNNNIbMr8gYbbBA6zcraa6+9AADLL7986BZYYIGQt9xySwDA8ccfHzp1jaTVjKgl/H01l2qVTs6Ljln/Nq1OUK3Qc/fnn3+GfPDBB4f8/PPPA0h2vFbXy/jx4xt8r7q5TjvtNADAyiuvHLp6NJe3FM5b1jocN25cyI8//jiApGthjjnmqPYhFo5BgwaFTNfFl19+GTp1k3NdzzjjjKF78MEHQ95kk00AtK77h94fv/rqKwDA4YcfHrp55pkn5J122glAcv+px+tPjymtZs5hhx0WuiOOOCLkJZZYAkD57irFFh1jjDHGFJaqWnT0Ke6ff/4J+bjjjgMATJgwIXRaR4d1WOo9wE/f2D/55JOQH3roIQDAH3/8Ebq77ror5DnnnBNAshrkNddcEzIrQnbr1i1066yzTsgaJFrptxY9ziuuuCLkk046KeRjjjkGQDIATi1BfKOlZQ4AunbtGjI/p+ev2harxtC1+vnnnwNIWrQmTpwYcufOnQEA22+/fejuu+++kA844AAAybcwHVM11jWPX604ffr0CXn06NENjpWWw/8eE9eXjlmtb/fccw8AYKWVVqrIsVeaNItMc9YXP6f7k1ocaMUBgK233rrB95tsdE6+//77kG+44YaQea9YffXVQ6fWf37HUkstFbq11lor5Hq/bxA9F7/99lvIRx11FADghx9+CJ2uufXXXx9A0uJRjxadLIsoLcIaYK4eA1q3dB7LtY7bomOMMcaYwuIHHWOMMcYUlqq6rtTc9MgjjzSQ6QIBkoGr9WwG1jGp6fCCCy4ImTWBNtpoo9Bdf/31IXOsGuB69tlnh3z11VcDSJrzGMAMAGuvvXbIlTpXHNfCCy8cOq2No3V0+LcaDDfTTDOFfOKJJwJIBlNntRPIEzWtqpvilFNOAVBqRQIAu+++e4O/femll0LH2jNAyTX3zTffhE5rgqywwgoNjqFc0zPnRI+JQZ0A8Nhjj4W8zDLLAMgO9uN30YUMAAsuuGDIiy++eOLYgXzmNKtFAN2oY8eODZ26OdLqWKUlS2yxxRah02D6e++9N+TlllsOQNMDJ1vaCDftO5pjzte/5b6jroFymya25DjUta+hDQzsPuecc0KngcdpTS3zdn2Xi94fHn74YQDJ/UfdxFxr9T5Ovc7o7gaAu+++G0BpnEApnAMoja+SyRy26BhjjDGmsPhBxxhjjDGFpeKuKzU3qTnysssuC7ljx44AgAMPPDB0ajqmSTXNxAzk6/rQ43z11VdDHjZsWMi77bYbAODSSy8NHWvPKNr9W/+fLoP+/fuHjiZyoLL1BQjPqWZHXXjhhSFrVhwzsPT41M3FNVAvLqosdK2ytgxQOr/aikJdc5QHDhwYOra9AErnQltcvPfeeyGrm7Yc0tyouub233//kOmu0r/NMn1zjWsdk6effjrk22+/HUDSdalruZpkuau0u/MZZ5wBIJn1p+Mnur88+eSTIe+7774AkrU9tKYJu0cDTXcj8Fg100hL/O+yyy4Asl1Iaeda54eZrEDpGlYX98svvxwyXQbaakfDCKqxvxDdE+gCBYDNN988ZNbE4T4KJDOw0lxXrQmuO9bLAYCbb745ZLZY0fuj7q/1uK/qXHANq+teszYZkrHKKquETsM4qlF/zBYdY4wxxhQWP+gYY4wxprBU3HWl5mQ13bMIG1ByY6npUs1x/A4tQqdusDxKrNOcpkX+tOO6wkJzaZHkQGl8WrZcM6zYQkFdJ2our6bpUk3ke+yxR+pvHnvssQCSWRvaSZcZEvVoYlXU3KpZRSz+qG07tN0BXXZaUI9FAvV7NWtCZf3dSpnfmcGi53yHHXZo8P9AqSuwzrW6cX788UcASdeczjVdV/vtt1/otDiiUqnxpRUB1CKdbCuisrZlSOs+rpmMPXr0CJnZPkceeWTosgo+Tm18ad3jNZNI3TVpLqv//e9/Ib/77rshM4PlxhtvDJ3uj2PGjAFQyh4DSnMKlIpK0l323zFVk7Qu1gBw6qmnhszj1jlRN/q2227b4POtyY3FNazXl67l7bbbDkCpGGVrQO/7zDDV9aWuVe6F1XSR/hdbdIwxxhhTWKpaR0drr8wyyywhs+laVk2aRx99FECyLPjPP/8c8gknnAAgWYI/jyd6NlcDkm/PtFSlWamAkqWA9VqAZJ0EWoS0dkQeb1w6P3vuuWfItGrpU7q+XZ133nkAameFaik6Vg1cZTDqoYceGjqdHwZ2cp0CyWDBPErQpwUDX3TRRSF//PHHIW+zzTYAkhYLLTfPYHMN4GWAL1Aqx7/VVluFTt/I+UYKlNZQS67PNCuM1ivS3+GYgJJ1kZYrID3wWC2mV155ZcismVRu7Se9JmgFUyu2nj9+v1p2dH3pWqTF6tprrw3d0ksvHbI2YCVsKwAAl1xyCYBSUghQ27droudUj5n7fu/evUOngeFzzTUXgKSVtN7bPuj+zzpcek0qvO6y6pDlSZZFTgOraYnSe8YhhxwSMq9lnbNqN3K2RccYY4wxhcUPOsYYY4wpLNNNzUw0efLkJtuQaHr666+/QterV6+QNYj3iSeeAAD8/vvvodPu2GlBvhqYx3Lub7zxRui0zkljpq+2bdtOBwBTpkwpy0aWZloHSiZZNVdqMDZdbhrMqt1pO3XqlPie5tKmTZuKjE9R0yIDKxmUDCRLmDMwlyZyIOlGKNeNxfE1Z302Rlo7g8svvzx0OlYGlGptoUoGGHN9NjY+XXM0A6sL7sMPPwxZx0c3R5cuXUI3cuTIkDl/anpWMzWvZe10z7YfQKmFCZDe3bsl46NrhZ3VgWSw6lVXXaXfDyA5D+pGZxD1FVdcETq604HSuWzpPHJ8//77b3wBg8HVHZ0WTKtj1tpA6lqi60b3F5VZk+Tggw8O3aqrrhoy51d/qzljrcb1p3As3377beg0cJvrT2sfqUuwXDdWpe4PWbAFwmGHHRa6WWedNWS6LLVVTCVdi5W6P2jtmzXWWCNkJgFo7Rx1jT/zzDMAgGeffTZ0Ggah424JHJ9ii44xxhhjCosfdIwxxhhTWCqWdUUzqNZzUNOjupbosjn33HNDp64PZoP07NkzdDT3AaWaM5qJpVkvtYpQz3JX0E2gmTBaZ2b06NEAgIsvvjh0WTWF6gU9Js6ljknN8HRpdO7cOXRak4ctLuppnGlzqeZiXV+sSZJWG0nR8WW1MykH/U7WHvntt99C16dPn9TjYwakthVR19ySSy4JIDsrktkgOqfa2oBZd0DlaoHwWLRFCU3gALDzzjuHTPeQuss1K+Saa64BAGywwQaha2ptnOag30PXRGMuTtVl1QtLa4Ggew3bgOiey0xCoLxMuFrAudCsMHUzMitOMwHVzcxznff+oten3hd5r9PaVhrmwdCMeswk033+3nvvDVlDE+ga1rYiWgeKoRu6p2hNIbbDqGSdJFt0jDHGGFNYKmbR4dOzNqfUAL877rgjZL7xP/LII6HbYostQmaQpz7FjR07NuTFFluswW/l/Xaib7+0WGmdC9bRAEpvuayADCTfuPN8E9G3EH3iTju/GkCnT+T8Dg3W/fXXX0NmTYX27dtP9ftrSVqdlZdeeil0WsWaYx0/fnzoFl544ZA5f/oWp0Go5Vh3dJ1pMCADpLW2lM5JmsVR38KUxoJxqdc6HxtttFHIGiRaqXnl9XHfffeFTi26+jucK32j1M+lVWat9vpryfc35zPXXXddyG+++SaApJVc5ypvS0dT0eNcb731Qua8ap0oVoMGkoHX9YJaPNmMdd555w2d1gniXNWjRUf3Hw0wnm+++UKmxVT3Ad1LaWlVi9DRRx8dMvelSo7fFh1jjDHGFBY/6BhjjDGmsFTMdUUzqwYQMWgMAIYOHRoyy5mrOVXLzZ922mkAgIceeih0a621Vsg0Q2uwXR5mPnVBqOmUplU11zHAEwDOPvtsAMm2GHmUYFc4FgZKA8Bjjz0Wsga2dujQAUDStK7BunSjqOtGAwe5Rg4//PCKHHtL0flT0/KQIUMAJI9ZS+zTjUITNJAsYU/zbqXcVYpeX3RRAMBzzz0HINkiQV2D6gbgvDV2zag7VU3WrGn1+uuvh07rYGhrg3ICX9M+o21FFl100ZC/++67kB944AEAydo62i6C11re7tKWwjWg16fWNGIwp+4v9egGaQydH71XbLzxxgCAW265JXT17o7TY2WSjtap0v2jnudK71N6f9fQlH79+gEAunXrFjptd/Tqq68CSO7/mkxQjfHbomOMMcaYwuIHHWOMMcYUlop3L1fT1oorrhgya98ApdLXP/30U+hGjBgR8oILLggAOP3000OnZj66BPI28ak7Qsvts8WFuqvuv//+kGlyz9tdpe4Imv61noO6XtR1w3LfWa4rZmMNGDAgdFoz4sILLwQALL/88qFTM2et5lXHr25Gjls7luvxM2tA65ykuUGqUTtHf0fnh2imn5qW1eXD+cmqU8Gxvvfee6FjnR6gVLpd271o1siOO+6Y+r2VQOeM9YwAYO+99w552223BZCsHVPJFh15oG5Eupe1XpFm2HF/yXt/bAydS4XXjf6/ttChy0OzHrWdUNb35oleS5yXbbbZJnR6LdbzvKmLULOetQXEn3/+CQBYZ511QsfaQEDpvq7uumq7Hm3RMcYYY0xhqbhFR9E3WgaQAcAOO+wAINn8j43MgFLF1qxgyHqpM6PBqFrTgUGg++yzT+i08nFaZdO8oXVAn9LVyqHWAdY50DnRJ3o+qev/f/311yGz/oIGznbv3r28AbQAPf86l2yaqLV/+vfvH/L6668PAFh22WVTv6uaqBVQLWKDBw8GkKynoo1itebO7LPPDiBZRXmBBRYImZWrda1r4C+tfqxHAyQDX9MCn1tCWtNcPWat5rzTTjuFrNdd2jG1FvT8q0WR42bSBpB8e65ni0CWtUXXNYPdhw8fHjodKy2qOv/azLYexz///POHzHnVe4Zef9xL63HNpjUSBpINSlnfSq3Iu+yyS8gcfy3nyRYdY4wxxhQWP+gYY4wxprBMNzXT8uTJkytmj1cz7IQJEwAkXSMa+FntpmZt27adDgCmTJnS5PHx+L/55pvQaeCuumFYU+T4448PnZrxqu3maNOmTbPHR5PkuHHjQjds2LCQtcQ3A/90zFpH6KmnngKQ7fpgnRUt0T/PPPOE3Nj54fjKXZ96fFqT5sQTTwRQqhcEJN00+++/P4BkMHAlzcxcn80ZH928NPsDyRYJWtOI7h8N5tQGinQ96vi1jgnXSktdVE0dn5rJOVeatKB1fFibK+v4aklL9pc0dH2yNhAAfPLJJwBK6xRInqta7S/NWZ88PnWhaluVp59+OmS2+9H9ResgsV2AJkhUsgFkNeZv1KhRITNwXsMENLCcSTzVWr8tuT+kkdZWBijNhR5/Ld1UHJ9ii44xxhhjCosfdIwxxhhTWGrmulJo5sqqHVBtM1c5pkktdX/ZZZeFrFkBLHetWWO1zLAqxzSZdcxqmqR+8uTJoVOT9N9//93ge7VTNl0+Le2oXCnXlaK/r/VZiHYa5rlo7JizMkwaWwstcV2l/WZWHR/+jf6/joVytWrPtMR1RTQ7R49f123e2SqVcn0o6oZk65tausOVclxXirYF0k7WXbt2BQB06tQpdJtuumnIzGqtVHbff6nG/Cl0I+v6rZYbPI1Kua7qFbuujDHGGDNN4QcdY4wxxhSWXFxXedMS0yRNr9rd+4033ghZC5bRpJyXCb1Wpsks10xjJdjLLZhYDddVWnE6pSVuHHXnZXViTvutclxXrYFKueaUeiq+WQ3XR2NrspZUI+uxsT2jWm6qNKrtuuK482pLYteVMcYYY0yBqGoLiCLBJ26t3aAtADRIMu9gyFqR9RZST2/XTaUab1dpAdxAfTYdbC20xrVVCYo47mlln/wv0+q488QWHWOMMcYUFj/oGGOMMaawTDUY2RhjjDGmNWOLjjHGGGMKy1SDkSdNmlRIc0+7du2mA4o/vn///beQ45t++umnifE5vbx1Mq2Mb+LEiYUcX/v27aeJ8Tm93BhjjDGmANRNenlTi8yZ1kk9FTwzxhgz7WCLjjHGGGMKSy4WnbTuz5MmTQqZb/rakVjL5teLJUCPL6tTdEtg9/bWXlhKz4l2NOe42IXZVB9dqyoTrjkg/fpq7WvR1Ddq8eX6bGxNmvohrSCq7v+65/BvtchutQuq2qJjjDHGmMLiBx1jjDHGFJaaua7UjPXbb78BAG655ZbQvfnmmyHTTN6pU6fQnXrqqSFPP33tPW48fjW3ffPNNyF/8sknIU+cOBFAtgmOJlk1102ePDnk9dZbDwDQsWPH0LUm0y3HPX78+NDtueeeIa+22moAgIEDB9b2wGqMzn8e86e//8UXX4T8/vvvAwDatm0bOs4JUHITzzjjjKFTN6O6FExt0Lnk/qfuxNY4Jzom3f9Gjx4NILn/61rl5/Lq/l0v6PlTWe+1afeaah8Lf/PHH38M3TvvvBPy7LPPDgDo3Llz6HR+ia7vcl3ntugYY4wxprD4QccYY4wxhaWqPiA1Z/38888h9+3bFwBw7733pn6OZqqll146dEceeWTIc8wxB4Dqm+PUBPjrr78CAG666abQDRkyJGSaWwFgypQpDb5Lo87nnntuAMByyy0XuoUXXjhkmvYWWGCB0FUjEr1a8FhvvPHG0D3//PMhn3LKKQCS57eesnrSspLS0GNOG4uuA/3/SmboTQ39nYcffjjk448/vsHfrrjiiiHTZbXIIouE7tBDDw15rbXWApC/a6456JyWey3pWDnX1Ri/HrNmLV5xxRUAgO7du4dulVVWqckxVRKdh19++SXks846CwBw6623hk6vta+//hoAMN9884VuhhlmSP3beqal109aVpOGUbz88ssh77zzzg3+thpoOMnYsWMBJPeM4cOHh7zgggsCAJ588snQvf322yF//vnnAIDddtstdIsvvnjITd2fFVt0jDHGGFNYambR+eqrr0J+6KGHACSfMmeZZZaQ//zzTwDAMsssEzpacYDqvqlkPSUfccQRAIDnnnsudLS8AMD6668f8vLLLw8gGcCpgXXdunUDAMw555ypv1srqv1GrlY8nUtasurJSqXHMmrUKADAZ599Frp55pknZFoadX71jfvRRx8FABx11FGhu/baa0PeaaedQtbg9kqjb7a77757yHyjevbZZ1P/dty4cQCARx55JHTff/99yLTE6jVZT8GwnMssi/Lvv//e4P8bQ68Prek177zzAqhcgoS+raqV4+STTw75+uuvBwDcdtttodO3eO6lrcmi8+GHH4bMffWjjz4K3ZVXXhnyHXfcAQDYY489QnfZZZeFTOtOvY6fe31a0DDQ+HFzjYwYMSJ0Bx98cMhqieW52HHHHcs44nT0+LlnAMCBBx4IoLSPAsC5554bMq8Z3lMB4J9//gmZFiEdH+ccAGaeeeaQmzrHtugYY4wxprD4QccYY4wxhaVmBWnUtP/XX38BAHbZZZfQqZtg5MiRAIAOHTqErtpuFn6/Hud+++0XMuv8LLHEEqG7/PLLQ1522WVDpulYawOk1QSopWmV49MAWa1dQRNnS91J+jmaId99993Q6Xmjyydv07K6GzSYfKuttgIAfPnll6GbbbbZQl5ppZUAlFyU//38K6+8AgDYYostQrfOOuuEXKtgST2/6nrbddddAQA9e/YMnbqWzz//fADJ+dtmm21Cbt++fYPvzxtdf3QHPvjgg6G79NJLQ+ZcZdUhSRuX6tRlOXToUADJ+W0J/H11Yegx010FlPaVDTfcsIFOv6veg8X1mIYNGxYyrzUGXQMldwcArLvuugCA+++/P3Snn356yEzsyHvMaW0tgFJiC0M4gKRrKa01ju5V7733HoCkO7p3794hH3LIISGzZlY1zoXeS84777wGx6d18rbeeuuQef9fYYUVQrfkkkuGfMYZZwAoHXslsEXHGGOMMYXFDzrGGGOMKSxVdV2piV5LzDOa/uOPPw4dI7WBUlbOiSeeWM3DS5Dm2lFzPt08Wttm1llnDbldu3YhMytDzdB51HZQ0ynbUmj2xsYbbxwyTYuVMHHSdcBWH0DJnAkA3333HYCka7JW50dNyNoWoU+fPg3+9u677w5ZM+yYjfTWW2+FTluYbLTRRgCSmVZ5ZyiluU61bQnrbQClDCXWMwGAvffeO2S6rvKuV5JWdh4otRbRTA+9PukG0dpV6npmzQ51t+r367pdeeWVG/x/SzIo+Rm9Zl599dWQ559//pCZ9bbGGmuETvctZosyuw6oHzeWnhttEcBwBaCUYaPZNVxzQMlNoudKx5rnusxyV2nW37HHHgsg6RrX6y+tTo7unwz5YD0rADjuuONC1vF36dIFQHJ9tiRDkGtGMw5fe+21kG+//faQ6VLdYIMNUo+JYRK6fvX/ea+tZPd6W3SMMcYYU1j8oGOMMcaYwlJV15Wa7jRrhSZHzVqiawUA9tprLwC1NUfy+7VwIbNPgFLWwwsvvBA6LRKobqAePXoAKGUHAMCiiy4acq06yqo5+5xzzgGQzLSii6UppGVjZR0/swb0/KibpJpF8rKgGViLQGrBMc36u+uuuwAAm266aep3bbLJJgCAo48+OnQsWw4AZ599NoBkQcg8xwwkC3JxXWtbCHWtsfgaxwkk5z9vlxXR8f30008h0+Wm+8c999wTMrM91J3VkrLyQOUzKPV7dM1oVsraa68NAJgwYULoNEOJhQTVdarXep7FHfU86/qba665QuZ1k9VihVlLGkag7pg8xpeW6aZtDQ477LCQO3bsCCDp7tEWFhyrfl5dW3RZDR48OHS6VlhwFyi1Gyrb9fN/x8SMKSCZVaWuRbb40Xtp2pzo/Kobk+PWIrN6rbYEW3SMMcYYU1jKsujo0yuf+NSKwFL4QLKcPIOQP/jgg9B17tw5ZAZb6VN6WqPMrKA/fXpsbl0Y/c4tt9wyZAZWaSl8rTOiYz2TannlAAALzUlEQVTooIMAAAsttFDoNNj6gAMOAJB8StUn7kq9HWowNWty0FoBJJ/CG3vi1vPPWkNqpVM4Lg3gzapZUk30d2jJYo0GIFmnQWuW0Dqn49c3UQYea7Cy1vFYffXVAeT7Zgkk3/IuueSSkAcMGACgZDkFgP79+4esNXdIvbR40OtTrxMNXN18880BJNsi6Prl36ou7/FxLLrnaT0VfUunJYdBrUDSOsA1oFaerl27Nvj/POp4qWXxmWeeCTnLekrUEs0WHmrlyoO0PU0bVXKfB5LWbc5L1v7JwGO14midpquuugoA8MMPP4SO5wQoBchXEo5PryltQaKB0awz1tg1pdcyG7UCpWQVbfuh9yrd15p6L7FFxxhjjDGFxQ86xhhjjCksZbmu1ITE7rODBg0KnZrx1EzMXHw1N2sdBZrRGdQEJIPVaK7S7r7aiZndT4FSwGhTzbT6d2pao5lRgzY1WGqHHXYIme4hddddcMEFIQ8fPhxA0jSpJbLp8ik36FOD9dgxW0uNawBc9+7dASTHrO441fO41J2jMusfjR8/PnR5dCrX32QQsnbU1bLl6sZJO1at2UE3lQabax0enqt6CdoFkqZtuhafeOKJ0KlpeNtttwVQcsEBSTdknjWh1PWR5ebh+lPX8zXXXBMyg3nrpbaM/r4GcGopf12frEmm+9+MM84YMvdXXet//PFHyKwDlIfrSlvsaAC/dt/mvOr8jBkzJuTHH38cQNINr2uB15/en6oxVt0TWVNLx6EthLQmXFoLHD0XrFmlLT6Y4AAAb7zxBoBkd3f9rbS9ulx4rN9++23o9J7Oej1Aaf01x8WkYQR0va244oqhKyccBbBFxxhjjDEFxg86xhhjjCkszXZdpXVRBUodn7UejnbUVXMazVBZdRJuvvlmAEl3lnZapvvo1ltvDZ12Kj7hhBNCrlSGAY81yxSoGVQ0M2t3dnWtsf7AEUccEbrnn38+ZGYANVaHoDHUnM9y+Jp9oS0MmMnB8vdAss5R2rh1ztSMPmTIEADJFgtaOrxW6DGz3P99990XOtaYANLNvWoi1TosL730EoBk1oi6AfPM4NF1ru7EM888M2R2XdcS7q+//nrInL/NNtssdBdddFHIrANS7XHq+WftFLZ3AJJ1jLRmTprpW10DrAWi11e9uK50zWodI90rWOdIXQfq5meLCz1X2m6Arq88XJB6nXTq1Cnkvn37hswMV92/dP64V2mLjLRO16xnBiRrWlVqrnXPuPrqqwEkXcT6O8yUAkprVV076objXGlog9ZvYzbvoYceGjq91qs5rzpmvT7VdcVxZ7mY+B1aW03DCNjOQu8ZbgFhjDHGGJPBdFN7Upo0aVKD/9QnOq3t0KtXLwBJy4RaOfTtigGxr7zySug0cJlPclo7IS3YTAMT1RLx1FNPhcyaIPqU265du+myxtccGgtmzHqi5bi0jtBuu+0WMgOfb7zxxtBpbYTGntg5vn///TcOiudMj2nSpEkh8+1crXCNNX/LqpZL6442mmOjPqBUf4jNW//vWKf6W8r000/fYHyNkdYoT49Z549/qxYLDTzmGxktH0AysL7cNyqOb/LkyRUzM6TVnNLrS68lXpf77rtv6LbYYouQWQdEA2CbM+a2bds2aXz6lkorqFZQ1fWlf8uaHKynAySDqRnMqhaDSr4FN3V8jaFzptabF198EUDSeq7zQ4uANlXU+lEMmE2rTdYUOL6JEyeWNT4NTKaVFCgFjuv8qCV2kUUWAZCsTaNJImzQuvvuu4cuq2ZZGu3bt2/S+HTN0aJ03XXXhU4beWrNHFrS1QqlFhFWsddq+rp+da8hzbF4cHxTpkxp9v6p9ZoOP/zwkNW6xvOfdU1xr+/du3fo1KLMZwhtutuc67NNmzYNbry26BhjjDGmsPhBxxhjjDGFpdnByGpCYj0GoGRG1ADHI488MlWmGY9BjQDw2GOPNTw4cZ2omZ3HoI3qNFhSg0wrZZKmGVnLbmttFZrrgMabdtKMyqBAIFnu+qSTTkr8+9//V5NmU8eX9nca7EVZj7mlwaaseZQ1DzqXtYLjyhpTmhuSrR4AYPTo0SGztLua1vMOZm2MtOPT+VfTOF0+2ohQG/ixlsbSSy9d8eNU1HVD07a6ODXAVoNVuRdoHRkN3GRAbL3PmV4zuhemtUtIa/2hrilN7KDLWl0veZwLDQZnMgtQqimmCQ50NwKlxApNUElzw1WjrY6iewnbHjBQHEiuX7Y9AkqhG5qsoi1a0hpxpsm1nDOuRdbLA5LzpzXbSFoCElByTY4YMSJ02oKH97dKNkK2RccYY4wxhcUPOv+vvXPnhS2Ko/i6iVelRSmUChI6iY+hEI0QaloapcQXkCi0dFqfQCNqnYhotOJZ3GrtWefaZ465M2fOnm39qp19zXX2wz57/o/1N8YYY0y2dFUCQs1R6+vrAIraD9TrAIrZPHRdaKaAZhgxm0U1CVhRWj+vpvXJycnQrkNHIOa6Uh0PjUCnG08j7WMZTOo60BIQHB8z2YCWthAAHBwchHY3pRXqMn1y/peXl0Ofut6YYaGZeE2jZlbKzW9sbIQ+zeChybws6yxFyiqxx+C+VNeqljB5fn4GUL/rSvfnwsICAGB3dzf0aQkLzXDh38/09HTo0wytFEt0VFHlUq5aU82qi+mcNOG6Kpt/joXZmUDxLOW5ofPQ9FrGtLf0+TTrjSEbMXcV0FuXTa/g/ri/vw99VTo5Wnbk6OgotKk5pHOid4E6xm+LjjHGGGOypSuLjn4LoF5NmWWlKhhWVSxp3dAbsRYdJHUULyuDt0y1QqiaowaWvb6+AiiOaWtrK7THx8cBFG/Ear2iCqaOj9+igeK8xIKIm4bPokGT2maQqK5Z00U/uWZAS8dCgyG1qB6DkJtUQC5Dv9lrsJ8qh1OFVveXBtafn58DAI6Pj0Of6jxR06nuPafzy6KpahnWYGQNDKemlmqTaOJDiutWB7q/5+bmQpvB2E0kBZQR0/fSAFUG+wItReUU11HHoTpzWkyZZ6EWqk5xLArPFfXYXF5ehja1tYB4YPbNzU1o82+ZXiCg6PHgXPTynWCLjjHGGGOyxRcdY4wxxmRLV64rpUqnpN1n/m2reTr276SJADR9Dg3WPDw8DG0GW2nZA5UwpxlQXQcagKXS7mRmZia01cyXksuK8JnU3aimW7qBdK9UlZuog1ghWaC1Vmp6nZ2dDe2Uzcw6JnXtnJychDZdhzrnqvPx8PAAANjb2wt929vboU0J+rrnIVaglGbvn3yuF5pQgwjPRR1zE67hTlCX693dHQDg+vo69KkblT+b4prqO0l1ZtbW1r79TIrPXwb3j76HtEArtY2A+PtNwzw2NzcBxN1V+rt6iS06xhhjjMkWX3SMMcYYky399xf8gBTdMTHU3KqaN9QEuL29DX1aGuPq6gpAvCI70DJtLi0tRf//fmabdYOu4+rqamhzfqq0P+pAzaKskgsA+/v7ob2zswOgmGk0KHtSn3NiYiK0Nevm8fHx2+dUJ4hrNT8/H/rKzMz9guNKUWMkBXTdWZpHdY5S3796jlEzR93dKysroZ36WIg+56Dv25g22sXFRWhrhtzi4iKA4vnJTEig9f7qp2vVFh1jjDHGZIsvOsYYY4zJlj/tzIDv7++DYSPskNHR0T9AfeOjGa6X5rhOKvFyfF9fX8msn84F2//rdhsaGup4fDSXqjCjyo5rRfjT01MAxayCfroIOb6Pj4+u1k/nXN2dVcSylnrpLhgZGenJ+FKl6fFxrZ+enkLfy8tLaHcruMfxvb299e38rLsSuTI2Nlbr+JqG4/v8/OxqfJ2cKf08P4eHh7+9eG3RMcYYY0y2JBmMPOjEvhH/dvr5jazd71crzdnZWWhr4C71f1IO9P4Jv1VH5rfDfTs1NRX6yopNpojPz8FgkM5HW3SMMcYYky2+6BhjjDEmW9oGIxtjjDHGDDK26BhjjDEmW3zRMcYYY0y2+KJjjDHGmGzxRccYY4wx2eKLjjHGGGOyxRcdY4wxxmTLX1qHY7f3MHcVAAAAAElFTkSuQmCC\n",
+ "text/plain": [
+ "
"
+ ]
+ },
+ "metadata": {
+ "needs_background": "light"
+ },
+ "output_type": "display_data"
+ }
+ ],
+ "source": [
+ "# Randomly select 100 data points to display\n",
+ "rand_indices = np.random.choice(m, 100, replace=False)\n",
+ "sel = X[rand_indices, :]\n",
+ "\n",
+ "utils.displayData(sel)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "### 1.2 Model representation\n",
+ "\n",
+ "Our neural network is shown in the following figure.\n",
+ "\n",
+ "\n",
+ "\n",
+ "It has 3 layers - an input layer, a hidden layer and an output layer. Recall that our inputs are pixel values\n",
+ "of digit images. Since the images are of size $20 \\times 20$, this gives us 400 input layer units (not counting the extra bias unit which always outputs +1). The training data was loaded into the variables `X` and `y` above.\n",
+ "\n",
+ "You have been provided with a set of network parameters ($\\Theta^{(1)}, \\Theta^{(2)}$) already trained by us. These are stored in `ex4weights.mat` and will be loaded in the next cell of this notebook into `Theta1` and `Theta2`. The parameters have dimensions that are sized for a neural network with 25 units in the second layer and 10 output units (corresponding to the 10 digit classes)."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 4,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "# Setup the parameters you will use for this exercise\n",
+ "input_layer_size = 400 # 20x20 Input Images of Digits\n",
+ "hidden_layer_size = 25 # 25 hidden units\n",
+ "num_labels = 10 # 10 labels, from 0 to 9\n",
+ "\n",
+ "# Load the weights into variables Theta1 and Theta2\n",
+ "weights = loadmat(os.path.join('Data', 'ex4weights.mat'))\n",
+ "\n",
+ "# Theta1 has size 25 x 401\n",
+ "# Theta2 has size 10 x 26\n",
+ "Theta1, Theta2 = weights['Theta1'], weights['Theta2']\n",
+ "\n",
+ "# swap first and last columns of Theta2, due to legacy from MATLAB indexing, \n",
+ "# since the weight file ex3weights.mat was saved based on MATLAB indexing\n",
+ "Theta2 = np.roll(Theta2, 1, axis=0)\n",
+ "\n",
+ "# Unroll parameters \n",
+ "nn_params = np.concatenate([Theta1.ravel(), Theta2.ravel()])"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "\n",
+ "### 1.3 Feedforward and cost function\n",
+ "\n",
+ "Now you will implement the cost function and gradient for the neural network. First, complete the code for the function `nnCostFunction` in the next cell to return the cost.\n",
+ "\n",
+ "Recall that the cost function for the neural network (without regularization) is:\n",
+ "\n",
+ "$$ J(\\theta) = \\frac{1}{m} \\sum_{i=1}^{m}\\sum_{k=1}^{K} \\left[ - y_k^{(i)} \\log \\left( \\left( h_\\theta \\left( x^{(i)} \\right) \\right)_k \\right) - \\left( 1 - y_k^{(i)} \\right) \\log \\left( 1 - \\left( h_\\theta \\left( x^{(i)} \\right) \\right)_k \\right) \\right]$$\n",
+ "\n",
+ "where $h_\\theta \\left( x^{(i)} \\right)$ is computed as shown in the neural network figure above, and K = 10 is the total number of possible labels. Note that $h_\\theta(x^{(i)})_k = a_k^{(3)}$ is the activation (output\n",
+ "value) of the $k^{th}$ output unit. Also, recall that whereas the original labels (in the variable y) were 0, 1, ..., 9, for the purpose of training a neural network, we need to encode the labels as vectors containing only values 0 or 1, so that\n",
+ "\n",
+ "$$ y = \n",
+ "\\begin{bmatrix} 1 \\\\ 0 \\\\ 0 \\\\\\vdots \\\\ 0 \\end{bmatrix}, \\quad\n",
+ "\\begin{bmatrix} 0 \\\\ 1 \\\\ 0 \\\\ \\vdots \\\\ 0 \\end{bmatrix}, \\quad \\cdots \\quad \\text{or} \\qquad\n",
+ "\\begin{bmatrix} 0 \\\\ 0 \\\\ 0 \\\\ \\vdots \\\\ 1 \\end{bmatrix}.\n",
+ "$$\n",
+ "\n",
+ "For example, if $x^{(i)}$ is an image of the digit 5, then the corresponding $y^{(i)}$ (that you should use with the cost function) should be a 10-dimensional vector with $y_5 = 1$, and the other elements equal to 0.\n",
+ "\n",
+ "You should implement the feedforward computation that computes $h_\\theta(x^{(i)})$ for every example $i$ and sum the cost over all examples. **Your code should also work for a dataset of any size, with any number of labels** (you can assume that there are always at least $K \\ge 3$ labels).\n",
+ "\n",
+ "
\n",
+ "**Implementation Note:** The matrix $X$ contains the examples in rows (i.e., X[i,:] is the i-th training example $x^{(i)}$, expressed as a $n \\times 1$ vector.) When you complete the code in `nnCostFunction`, you will need to add the column of 1’s to the X matrix. The parameters for each unit in the neural network is represented in Theta1 and Theta2 as one row. Specifically, the first row of Theta1 corresponds to the first hidden unit in the second layer. You can use a for-loop over the examples to compute the cost.\n",
+ "
\n",
+ ""
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 5,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "def sigmoidGradient(z):\n",
+ " g = np.zeros(z.shape)\n",
+ " A = utils.sigmoid(z)\n",
+ " g = A*(1-A)\n",
+ " return g\n",
+ "\n",
+ "\n",
+ "\n",
+ "\n",
+ "def nnCostFunction(nn_params,\n",
+ " input_layer_size,\n",
+ " hidden_layer_size,\n",
+ " num_labels,\n",
+ " X, y, lambda_=0.0):\n",
+ " \"\"\"\n",
+ " Implements the neural network cost function and gradient for a two layer neural \n",
+ " network which performs classification. \n",
+ " \n",
+ " Parameters\n",
+ " ----------\n",
+ " nn_params : array_like\n",
+ " The parameters for the neural network which are \"unrolled\" into \n",
+ " a vector. This needs to be converted back into the weight matrices Theta1\n",
+ " and Theta2.\n",
+ " \n",
+ " input_layer_size : int\n",
+ " Number of features for the input layer. \n",
+ " \n",
+ " hidden_layer_size : int\n",
+ " Number of hidden units in the second layer.\n",
+ " \n",
+ " num_labels : int\n",
+ " Total number of labels, or equivalently number of units in output layer. \n",
+ " \n",
+ " X : array_like\n",
+ " Input dataset. A matrix of shape (m x input_layer_size).\n",
+ " \n",
+ " y : array_like\n",
+ " Dataset labels. A vector of shape (m,).\n",
+ " \n",
+ " lambda_ : float, optional\n",
+ " Regularization parameter.\n",
+ " \n",
+ " Returns\n",
+ " -------\n",
+ " J : float\n",
+ " The computed value for the cost function at the current weight values.\n",
+ " \n",
+ " grad : array_like\n",
+ " An \"unrolled\" vector of the partial derivatives of the concatenatation of\n",
+ " neural network weights Theta1 and Theta2.\n",
+ " \n",
+ " Instructions\n",
+ " ------------\n",
+ " You should complete the code by working through the following parts.\n",
+ " \n",
+ " - Part 1: Feedforward the neural network and return the cost in the \n",
+ " variable J. After implementing Part 1, you can verify that your\n",
+ " cost function computation is correct by verifying the cost\n",
+ " computed in the following cell.\n",
+ " \n",
+ " - Part 2: Implement the backpropagation algorithm to compute the gradients\n",
+ " Theta1_grad and Theta2_grad. You should return the partial derivatives of\n",
+ " the cost function with respect to Theta1 and Theta2 in Theta1_grad and\n",
+ " Theta2_grad, respectively. After implementing Part 2, you can check\n",
+ " that your implementation is correct by running checkNNGradients provided\n",
+ " in the utils.py module.\n",
+ " \n",
+ " Note: The vector y passed into the function is a vector of labels\n",
+ " containing values from 0..K-1. You need to map this vector into a \n",
+ " binary vector of 1's and 0's to be used with the neural network\n",
+ " cost function.\n",
+ " \n",
+ " Hint: We recommend implementing backpropagation using a for-loop\n",
+ " over the training examples if you are implementing it for the \n",
+ " first time.\n",
+ " \n",
+ " - Part 3: Implement regularization with the cost function and gradients.\n",
+ " \n",
+ " Hint: You can implement this around the code for\n",
+ " backpropagation. That is, you can compute the gradients for\n",
+ " the regularization separately and then add them to Theta1_grad\n",
+ " and Theta2_grad from Part 2.\n",
+ " \n",
+ " Note \n",
+ " ----\n",
+ " We have provided an implementation for the sigmoid function in the file \n",
+ " `utils.py` accompanying this assignment.\n",
+ " \"\"\"\n",
+ " # Reshape nn_params back into the parameters Theta1 and Theta2, the weight matrices\n",
+ " # for our 2 layer neural network\n",
+ " Theta1 = np.reshape(nn_params[:hidden_layer_size * (input_layer_size + 1)],\n",
+ " (hidden_layer_size, (input_layer_size + 1)))\n",
+ "\n",
+ " Theta2 = np.reshape(nn_params[(hidden_layer_size * (input_layer_size + 1)):],\n",
+ " (num_labels, (hidden_layer_size + 1)))\n",
+ "\n",
+ " # Setup some useful variables\n",
+ " m = y.size\n",
+ " \n",
+ " # You need to return the following variables correctly \n",
+ " J = 0\n",
+ " Theta1_grad = np.zeros(Theta1.shape)\n",
+ " Theta2_grad = np.zeros(Theta2.shape)\n",
+ "\n",
+ " # ====================== YOUR CODE HERE ======================\n",
+ " \n",
+ " a1 = np.concatenate([np.ones((m, 1)), X], axis=1)\n",
+ " \n",
+ " a2 = utils.sigmoid(a1.dot(Theta1.T))\n",
+ " a2 = np.concatenate([np.ones((a2.shape[0], 1)), a2], axis=1)\n",
+ " \n",
+ " a3 = utils.sigmoid(a2.dot(Theta2.T))\n",
+ " \n",
+ " y_matrix = y.reshape(-1)\n",
+ " y_matrix = np.eye(num_labels)[y_matrix]\n",
+ " \n",
+ " temp1 = Theta1\n",
+ " temp2 = Theta2\n",
+ " \n",
+ " # Add regularization term\n",
+ " \n",
+ " reg_term = (lambda_ / (2 * m)) * (np.sum(np.square(temp1[:, 1:])) + np.sum(np.square(temp2[:, 1:])))\n",
+ " \n",
+ " J = (-1 / m) * np.sum((np.log(a3) * y_matrix) + np.log(1 - a3) * (1 - y_matrix)) + reg_term\n",
+ " \n",
+ " # Backpropogation\n",
+ " \n",
+ " delta_3 = a3 - y_matrix\n",
+ " delta_2 = delta_3.dot(Theta2)[:, 1:] * sigmoidGradient(a1.dot(Theta1.T))\n",
+ "\n",
+ " Delta1 = delta_2.T.dot(a1)\n",
+ " Delta2 = delta_3.T.dot(a2)\n",
+ " \n",
+ " #Regularised Gradient\n",
+ " \n",
+ " Theta1_grad = (1 / m) * Delta1\n",
+ " Theta1_grad[:, 1:] = Theta1_grad[:, 1:] + (lambda_ / m) * Theta1[:, 1:]\n",
+ " \n",
+ " Theta2_grad = (1 / m) * Delta2\n",
+ " Theta2_grad[:, 1:] = Theta2_grad[:, 1:] + (lambda_ / m) * Theta2[:, 1:]\n",
+ " \n",
+ " \n",
+ " \n",
+ " # ================================================================\n",
+ " # Unroll gradients\n",
+ " # grad = np.concatenate([Theta1_grad.ravel(order=order), Theta2_grad.ravel(order=order)])\n",
+ " grad = np.concatenate([Theta1_grad.ravel(), Theta2_grad.ravel()])\n",
+ " \n",
+ " return J, grad"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "
\n",
+ "Use the following links to go back to the different parts of this exercise that require to modify the function `nnCostFunction`. \n",
+ "\n",
+ "Back to:\n",
+ "- [Feedforward and cost function](#section1)\n",
+ "- [Regularized cost](#section2)\n",
+ "- [Neural Network Gradient (Backpropagation)](#section4)\n",
+ "- [Regularized Gradient](#section5)\n",
+ "
"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Once you are done, call your `nnCostFunction` using the loaded set of parameters for `Theta1` and `Theta2`. You should see that the cost is about 0.287629."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 6,
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "Cost at parameters (loaded from ex4weights): 0.287629 \n",
+ "The cost should be about : 0.287629.\n"
+ ]
+ }
+ ],
+ "source": [
+ "lambda_ = 0\n",
+ "J, _ = nnCostFunction(nn_params, input_layer_size, hidden_layer_size,\n",
+ " num_labels, X, y, lambda_)\n",
+ "print('Cost at parameters (loaded from ex4weights): %.6f ' % J)\n",
+ "print('The cost should be about : 0.287629.')"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "*You should now submit your solutions.*"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 8,
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "\n",
+ "Submitting Solutions | Programming Exercise neural-network-learning\n",
+ "\n",
+ "Use token from last successful submission (tanmayss2109@gmail.com)? (Y/n): Y\n",
+ " Part Name | Score | Feedback\n",
+ " --------- | ----- | --------\n",
+ " Feedforward and Cost Function | 30 / 30 | Nice work!\n",
+ " Regularized Cost Function | 0 / 15 | \n",
+ " Sigmoid Gradient | 0 / 5 | \n",
+ " Neural Network Gradient (Backpropagation) | 0 / 40 | \n",
+ " Regularized Gradient | 0 / 10 | \n",
+ " --------------------------------\n",
+ " | 30 / 100 | \n",
+ "\n"
+ ]
+ }
+ ],
+ "source": [
+ "grader = utils.Grader()\n",
+ "grader[1] = nnCostFunction\n",
+ "grader.grade()"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "\n",
+ "### 1.4 Regularized cost function\n",
+ "\n",
+ "The cost function for neural networks with regularization is given by:\n",
+ "\n",
+ "\n",
+ "$$ J(\\theta) = \\frac{1}{m} \\sum_{i=1}^{m}\\sum_{k=1}^{K} \\left[ - y_k^{(i)} \\log \\left( \\left( h_\\theta \\left( x^{(i)} \\right) \\right)_k \\right) - \\left( 1 - y_k^{(i)} \\right) \\log \\left( 1 - \\left( h_\\theta \\left( x^{(i)} \\right) \\right)_k \\right) \\right] + \\frac{\\lambda}{2 m} \\left[ \\sum_{j=1}^{25} \\sum_{k=1}^{400} \\left( \\Theta_{j,k}^{(1)} \\right)^2 + \\sum_{j=1}^{10} \\sum_{k=1}^{25} \\left( \\Theta_{j,k}^{(2)} \\right)^2 \\right] $$\n",
+ "\n",
+ "You can assume that the neural network will only have 3 layers - an input layer, a hidden layer and an output layer. However, your code should work for any number of input units, hidden units and outputs units. While we\n",
+ "have explicitly listed the indices above for $\\Theta^{(1)}$ and $\\Theta^{(2)}$ for clarity, do note that your code should in general work with $\\Theta^{(1)}$ and $\\Theta^{(2)}$ of any size. Note that you should not be regularizing the terms that correspond to the bias. For the matrices `Theta1` and `Theta2`, this corresponds to the first column of each matrix. You should now add regularization to your cost function. Notice that you can first compute the unregularized cost function $J$ using your existing `nnCostFunction` and then later add the cost for the regularization terms.\n",
+ "\n",
+ "[Click here to go back to `nnCostFunction` for editing.](#nnCostFunction)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Once you are done, the next cell will call your `nnCostFunction` using the loaded set of parameters for `Theta1` and `Theta2`, and $\\lambda = 1$. You should see that the cost is about 0.383770."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 9,
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "Cost at parameters (loaded from ex4weights): 0.289087\n",
+ "This value should be about : 0.383770.\n"
+ ]
+ }
+ ],
+ "source": [
+ "# Weight regularization parameter (we set this to 1 here).\n",
+ "lambda_ = 1\n",
+ "J, _ = nnCostFunction(nn_params, input_layer_size, hidden_layer_size,\n",
+ " num_labels, X, y, lambda_)\n",
+ "\n",
+ "print('Cost at parameters (loaded from ex4weights): %.6f' % J)\n",
+ "print('This value should be about : 0.383770.')"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "*You should now submit your solutions.*"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 10,
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "\n",
+ "Submitting Solutions | Programming Exercise neural-network-learning\n",
+ "\n",
+ "Use token from last successful submission (tanmayss2109@gmail.com)? (Y/n): y\n",
+ " Part Name | Score | Feedback\n",
+ " --------- | ----- | --------\n",
+ " Feedforward and Cost Function | 30 / 30 | Nice work!\n",
+ " Regularized Cost Function | 0 / 15 | \n",
+ " Sigmoid Gradient | 0 / 5 | \n",
+ " Neural Network Gradient (Backpropagation) | 0 / 40 | \n",
+ " Regularized Gradient | 0 / 10 | \n",
+ " --------------------------------\n",
+ " | 30 / 100 | \n",
+ "\n"
+ ]
+ }
+ ],
+ "source": [
+ "grader[2] = nnCostFunction\n",
+ "grader.grade()"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "## 2 Backpropagation\n",
+ "\n",
+ "In this part of the exercise, you will implement the backpropagation algorithm to compute the gradient for the neural network cost function. You will need to update the function `nnCostFunction` so that it returns an appropriate value for `grad`. Once you have computed the gradient, you will be able to train the neural network by minimizing the cost function $J(\\theta)$ using an advanced optimizer such as `scipy`'s `optimize.minimize`.\n",
+ "You will first implement the backpropagation algorithm to compute the gradients for the parameters for the (unregularized) neural network. After you have verified that your gradient computation for the unregularized case is correct, you will implement the gradient for the regularized neural network."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "\n",
+ "### 2.1 Sigmoid Gradient\n",
+ "\n",
+ "To help you get started with this part of the exercise, you will first implement\n",
+ "the sigmoid gradient function. The gradient for the sigmoid function can be\n",
+ "computed as\n",
+ "\n",
+ "$$ g'(z) = \\frac{d}{dz} g(z) = g(z)\\left(1-g(z)\\right) $$\n",
+ "\n",
+ "where\n",
+ "\n",
+ "$$ \\text{sigmoid}(z) = g(z) = \\frac{1}{1 + e^{-z}} $$\n",
+ "\n",
+ "Now complete the implementation of `sigmoidGradient` in the next cell.\n",
+ ""
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 11,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "def sigmoidGradient(z):\n",
+ " \"\"\"\n",
+ " Computes the gradient of the sigmoid function evaluated at z. \n",
+ " This should work regardless if z is a matrix or a vector. \n",
+ " In particular, if z is a vector or matrix, you should return\n",
+ " the gradient for each element.\n",
+ " \n",
+ " Parameters\n",
+ " ----------\n",
+ " z : array_like\n",
+ " A vector or matrix as input to the sigmoid function. \n",
+ " \n",
+ " Returns\n",
+ " --------\n",
+ " g : array_like\n",
+ " Gradient of the sigmoid function. Has the same shape as z. \n",
+ " \n",
+ " Instructions\n",
+ " ------------\n",
+ " Compute the gradient of the sigmoid function evaluated at\n",
+ " each value of z (z can be a matrix, vector or scalar).\n",
+ " \n",
+ " Note\n",
+ " ----\n",
+ " We have provided an implementation of the sigmoid function \n",
+ " in `utils.py` file accompanying this assignment.\n",
+ " \"\"\"\n",
+ "\n",
+ " g = np.zeros(z.shape)\n",
+ "\n",
+ " # ====================== YOUR CODE HERE ======================\n",
+ " A = utils.sigmoid(z)\n",
+ " g = A * (1-A)\n",
+ " \n",
+ "\n",
+ " # =============================================================\n",
+ " return g"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "When you are done, the following cell call `sigmoidGradient` on a given vector `z`. Try testing a few values by calling `sigmoidGradient(z)`. For large values (both positive and negative) of z, the gradient should be close to 0. When $z = 0$, the gradient should be exactly 0.25. Your code should also work with vectors and matrices. For a matrix, your function should perform the sigmoid gradient function on every element."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 12,
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "Sigmoid gradient evaluated at [-1 -0.5 0 0.5 1]:\n",
+ " \n",
+ "[0.19661193 0.23500371 0.25 0.23500371 0.19661193]\n"
+ ]
+ }
+ ],
+ "source": [
+ "z = np.array([-1, -0.5, 0, 0.5, 1])\n",
+ "g = sigmoidGradient(z)\n",
+ "print('Sigmoid gradient evaluated at [-1 -0.5 0 0.5 1]:\\n ')\n",
+ "print(g)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "*You should now submit your solutions.*"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 13,
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "\n",
+ "Submitting Solutions | Programming Exercise neural-network-learning\n",
+ "\n",
+ "Use token from last successful submission (tanmayss2109@gmail.com)? (Y/n): y\n",
+ " Part Name | Score | Feedback\n",
+ " --------- | ----- | --------\n",
+ " Feedforward and Cost Function | 30 / 30 | Nice work!\n",
+ " Regularized Cost Function | 0 / 15 | \n",
+ " Sigmoid Gradient | 5 / 5 | Nice work!\n",
+ " Neural Network Gradient (Backpropagation) | 0 / 40 | \n",
+ " Regularized Gradient | 0 / 10 | \n",
+ " --------------------------------\n",
+ " | 35 / 100 | \n",
+ "\n"
+ ]
+ }
+ ],
+ "source": [
+ "grader[3] = sigmoidGradient\n",
+ "grader.grade()"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "## 2.2 Random Initialization\n",
+ "\n",
+ "When training neural networks, it is important to randomly initialize the parameters for symmetry breaking. One effective strategy for random initialization is to randomly select values for $\\Theta^{(l)}$ uniformly in the range $[-\\epsilon_{init}, \\epsilon_{init}]$. You should use $\\epsilon_{init} = 0.12$. This range of values ensures that the parameters are kept small and makes the learning more efficient.\n",
+ "\n",
+ "
\n",
+ "One effective strategy for choosing $\\epsilon_{init}$ is to base it on the number of units in the network. A good choice of $\\epsilon_{init}$ is $\\epsilon_{init} = \\frac{\\sqrt{6}}{\\sqrt{L_{in} + L_{out}}}$ where $L_{in} = s_l$ and $L_{out} = s_{l+1}$ are the number of units in the layers adjacent to $\\Theta^{l}$.\n",
+ "
\n",
+ "\n",
+ "Your job is to complete the function `randInitializeWeights` to initialize the weights for $\\Theta$. Modify the function by filling in the following code:\n",
+ "\n",
+ "```python\n",
+ "# Randomly initialize the weights to small values\n",
+ "W = np.random.rand(L_out, 1 + L_in) * 2 * epsilon_init - epsilon_init\n",
+ "```\n",
+ "Note that we give the function an argument for $\\epsilon$ with default value `epsilon_init = 0.12`."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 14,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "def randInitializeWeights(L_in, L_out, epsilon_init=0.12):\n",
+ " \"\"\"\n",
+ " Randomly initialize the weights of a layer in a neural network.\n",
+ " \n",
+ " Parameters\n",
+ " ----------\n",
+ " L_in : int\n",
+ " Number of incomming connections.\n",
+ " \n",
+ " L_out : int\n",
+ " Number of outgoing connections. \n",
+ " \n",
+ " epsilon_init : float, optional\n",
+ " Range of values which the weight can take from a uniform \n",
+ " distribution.\n",
+ " \n",
+ " Returns\n",
+ " -------\n",
+ " W : array_like\n",
+ " The weight initialiatized to random values. Note that W should\n",
+ " be set to a matrix of size(L_out, 1 + L_in) as\n",
+ " the first column of W handles the \"bias\" terms.\n",
+ " \n",
+ " Instructions\n",
+ " ------------\n",
+ " Initialize W randomly so that we break the symmetry while training\n",
+ " the neural network. Note that the first column of W corresponds \n",
+ " to the parameters for the bias unit.\n",
+ " \"\"\"\n",
+ "\n",
+ " # You need to return the following variables correctly \n",
+ " W = np.zeros((L_out, 1 + L_in))\n",
+ "\n",
+ " # ====================== YOUR CODE HERE ======================\n",
+ "\n",
+ " W = np.random.rand(L_out, 1 + L_in) * 2 * epsilon_init - epsilon_init\n",
+ "\n",
+ " # ============================================================\n",
+ " return W"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "*You do not need to submit any code for this part of the exercise.*\n",
+ "\n",
+ "Execute the following cell to initialize the weights for the 2 layers in the neural network using the `randInitializeWeights` function."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 15,
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "Initializing Neural Network Parameters ...\n"
+ ]
+ }
+ ],
+ "source": [
+ "print('Initializing Neural Network Parameters ...')\n",
+ "\n",
+ "initial_Theta1 = randInitializeWeights(input_layer_size, hidden_layer_size)\n",
+ "initial_Theta2 = randInitializeWeights(hidden_layer_size, num_labels)\n",
+ "\n",
+ "# Unroll parameters\n",
+ "initial_nn_params = np.concatenate([initial_Theta1.ravel(), initial_Theta2.ravel()], axis=0)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "\n",
+ "### 2.4 Backpropagation\n",
+ "\n",
+ "\n",
+ "\n",
+ "Now, you will implement the backpropagation algorithm. Recall that the intuition behind the backpropagation algorithm is as follows. Given a training example $(x^{(t)}, y^{(t)})$, we will first run a “forward pass” to compute all the activations throughout the network, including the output value of the hypothesis $h_\\theta(x)$. Then, for each node $j$ in layer $l$, we would like to compute an “error term” $\\delta_j^{(l)}$ that measures how much that node was “responsible” for any errors in our output.\n",
+ "\n",
+ "For an output node, we can directly measure the difference between the network’s activation and the true target value, and use that to define $\\delta_j^{(3)}$ (since layer 3 is the output layer). For the hidden units, you will compute $\\delta_j^{(l)}$ based on a weighted average of the error terms of the nodes in layer $(l+1)$. In detail, here is the backpropagation algorithm (also depicted in the figure above). You should implement steps 1 to 4 in a loop that processes one example at a time. Concretely, you should implement a for-loop `for t in range(m)` and place steps 1-4 below inside the for-loop, with the $t^{th}$ iteration performing the calculation on the $t^{th}$ training example $(x^{(t)}, y^{(t)})$. Step 5 will divide the accumulated gradients by $m$ to obtain the gradients for the neural network cost function.\n",
+ "\n",
+ "1. Set the input layer’s values $(a^{(1)})$ to the $t^{th }$training example $x^{(t)}$. Perform a feedforward pass, computing the activations $(z^{(2)}, a^{(2)}, z^{(3)}, a^{(3)})$ for layers 2 and 3. Note that you need to add a `+1` term to ensure that the vectors of activations for layers $a^{(1)}$ and $a^{(2)}$ also include the bias unit. In `numpy`, if a 1 is a column matrix, adding one corresponds to `a_1 = np.concatenate([np.ones((m, 1)), a_1], axis=1)`.\n",
+ "\n",
+ "1. For each output unit $k$ in layer 3 (the output layer), set \n",
+ "$$\\delta_k^{(3)} = \\left(a_k^{(3)} - y_k \\right)$$\n",
+ "where $y_k \\in \\{0, 1\\}$ indicates whether the current training example belongs to class $k$ $(y_k = 1)$, or if it belongs to a different class $(y_k = 0)$. You may find logical arrays helpful for this task (explained in the previous programming exercise).\n",
+ "\n",
+ "1. For the hidden layer $l = 2$, set \n",
+ "$$ \\delta^{(2)} = \\left( \\Theta^{(2)} \\right)^T \\delta^{(3)} * g'\\left(z^{(2)} \\right)$$\n",
+ "Note that the symbol $*$ performs element wise multiplication in `numpy`.\n",
+ "\n",
+ "1. Accumulate the gradient from this example using the following formula. Note that you should skip or remove $\\delta_0^{(2)}$. In `numpy`, removing $\\delta_0^{(2)}$ corresponds to `delta_2 = delta_2[1:]`.\n",
+ "\n",
+ "1. Obtain the (unregularized) gradient for the neural network cost function by dividing the accumulated gradients by $\\frac{1}{m}$:\n",
+ "$$ \\frac{\\partial}{\\partial \\Theta_{ij}^{(l)}} J(\\Theta) = D_{ij}^{(l)} = \\frac{1}{m} \\Delta_{ij}^{(l)}$$\n",
+ "\n",
+ "
\n",
+ "**Python/Numpy tip**: You should implement the backpropagation algorithm only after you have successfully completed the feedforward and cost functions. While implementing the backpropagation alogrithm, it is often useful to use the `shape` function to print out the shapes of the variables you are working with if you run into dimension mismatch errors.\n",
+ "
\n",
+ "\n",
+ "[Click here to go back and update the function `nnCostFunction` with the backpropagation algorithm](#nnCostFunction)."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "After you have implemented the backpropagation algorithm, we will proceed to run gradient checking on your implementation. The gradient check will allow you to increase your confidence that your code is\n",
+ "computing the gradients correctly.\n",
+ "\n",
+ "### 2.4 Gradient checking \n",
+ "\n",
+ "In your neural network, you are minimizing the cost function $J(\\Theta)$. To perform gradient checking on your parameters, you can imagine “unrolling” the parameters $\\Theta^{(1)}$, $\\Theta^{(2)}$ into a long vector $\\theta$. By doing so, you can think of the cost function being $J(\\Theta)$ instead and use the following gradient checking procedure.\n",
+ "\n",
+ "Suppose you have a function $f_i(\\theta)$ that purportedly computes $\\frac{\\partial}{\\partial \\theta_i} J(\\theta)$; you’d like to check if $f_i$ is outputting correct derivative values.\n",
+ "\n",
+ "$$\n",
+ "\\text{Let } \\theta^{(i+)} = \\theta + \\begin{bmatrix} 0 \\\\ 0 \\\\ \\vdots \\\\ \\epsilon \\\\ \\vdots \\\\ 0 \\end{bmatrix}\n",
+ "\\quad \\text{and} \\quad \\theta^{(i-)} = \\theta - \\begin{bmatrix} 0 \\\\ 0 \\\\ \\vdots \\\\ \\epsilon \\\\ \\vdots \\\\ 0 \\end{bmatrix}\n",
+ "$$\n",
+ "\n",
+ "So, $\\theta^{(i+)}$ is the same as $\\theta$, except its $i^{th}$ element has been incremented by $\\epsilon$. Similarly, $\\theta^{(i−)}$ is the corresponding vector with the $i^{th}$ element decreased by $\\epsilon$. You can now numerically verify $f_i(\\theta)$’s correctness by checking, for each $i$, that:\n",
+ "\n",
+ "$$ f_i\\left( \\theta \\right) \\approx \\frac{J\\left( \\theta^{(i+)}\\right) - J\\left( \\theta^{(i-)} \\right)}{2\\epsilon} $$\n",
+ "\n",
+ "The degree to which these two values should approximate each other will depend on the details of $J$. But assuming $\\epsilon = 10^{-4}$, you’ll usually find that the left- and right-hand sides of the above will agree to at least 4 significant digits (and often many more).\n",
+ "\n",
+ "We have implemented the function to compute the numerical gradient for you in `computeNumericalGradient` (within the file `utils.py`). While you are not required to modify the file, we highly encourage you to take a look at the code to understand how it works.\n",
+ "\n",
+ "In the next cell we will run the provided function `checkNNGradients` which will create a small neural network and dataset that will be used for checking your gradients. If your backpropagation implementation is correct,\n",
+ "you should see a relative difference that is less than 1e-9.\n",
+ "\n",
+ "
\n",
+ "**Practical Tip**: When performing gradient checking, it is much more efficient to use a small neural network with a relatively small number of input units and hidden units, thus having a relatively small number\n",
+ "of parameters. Each dimension of $\\theta$ requires two evaluations of the cost function and this can be expensive. In the function `checkNNGradients`, our code creates a small random model and dataset which is used with `computeNumericalGradient` for gradient checking. Furthermore, after you are confident that your gradient computations are correct, you should turn off gradient checking before running your learning algorithm.\n",
+ "
\n",
+ "\n",
+ "
\n",
+ "**Practical Tip:** Gradient checking works for any function where you are computing the cost and the gradient. Concretely, you can use the same `computeNumericalGradient` function to check if your gradient implementations for the other exercises are correct too (e.g., logistic regression’s cost function).\n",
+ "
"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 16,
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "[[-9.27825235e-03 -9.27825236e-03]\n",
+ " [-3.04978709e-06 -3.04978914e-06]\n",
+ " [-1.75060084e-04 -1.75060082e-04]\n",
+ " [-9.62660640e-05 -9.62660620e-05]\n",
+ " [ 8.89911959e-03 8.89911960e-03]\n",
+ " [ 1.42869450e-05 1.42869443e-05]\n",
+ " [ 2.33146358e-04 2.33146357e-04]\n",
+ " [ 1.17982666e-04 1.17982666e-04]\n",
+ " [-8.36010761e-03 -8.36010762e-03]\n",
+ " [-2.59383093e-05 -2.59383100e-05]\n",
+ " [-2.87468729e-04 -2.87468729e-04]\n",
+ " [-1.37149709e-04 -1.37149706e-04]\n",
+ " [ 7.62813550e-03 7.62813551e-03]\n",
+ " [ 3.69883257e-05 3.69883234e-05]\n",
+ " [ 3.35320351e-04 3.35320347e-04]\n",
+ " [ 1.53247082e-04 1.53247082e-04]\n",
+ " [-6.74798369e-03 -6.74798370e-03]\n",
+ " [-4.68759764e-05 -4.68759769e-05]\n",
+ " [-3.76215583e-04 -3.76215587e-04]\n",
+ " [-1.66560294e-04 -1.66560294e-04]\n",
+ " [ 3.14544970e-01 3.14544970e-01]\n",
+ " [ 1.64090819e-01 1.64090819e-01]\n",
+ " [ 1.64567932e-01 1.64567932e-01]\n",
+ " [ 1.58339334e-01 1.58339334e-01]\n",
+ " [ 1.51127527e-01 1.51127527e-01]\n",
+ " [ 1.49568335e-01 1.49568335e-01]\n",
+ " [ 1.11056588e-01 1.11056588e-01]\n",
+ " [ 5.75736494e-02 5.75736493e-02]\n",
+ " [ 5.77867378e-02 5.77867378e-02]\n",
+ " [ 5.59235296e-02 5.59235296e-02]\n",
+ " [ 5.36967009e-02 5.36967009e-02]\n",
+ " [ 5.31542052e-02 5.31542052e-02]\n",
+ " [ 9.74006970e-02 9.74006970e-02]\n",
+ " [ 5.04575855e-02 5.04575855e-02]\n",
+ " [ 5.07530173e-02 5.07530173e-02]\n",
+ " [ 4.91620841e-02 4.91620841e-02]\n",
+ " [ 4.71456249e-02 4.71456249e-02]\n",
+ " [ 4.65597186e-02 4.65597186e-02]]\n",
+ "The above two columns you get should be very similar.\n",
+ "(Left-Your Numerical Gradient, Right-Analytical Gradient)\n",
+ "\n",
+ "If your backpropagation implementation is correct, then \n",
+ "the relative difference will be small (less than 1e-9). \n",
+ "Relative Difference: 2.41486e-11\n"
+ ]
+ }
+ ],
+ "source": [
+ "utils.checkNNGradients(nnCostFunction)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "*Once your cost function passes the gradient check for the (unregularized) neural network cost function, you should submit the neural network gradient function (backpropagation).*"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 17,
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "\n",
+ "Submitting Solutions | Programming Exercise neural-network-learning\n",
+ "\n",
+ "Use token from last successful submission (tanmayss2109@gmail.com)? (Y/n): y\n",
+ " Part Name | Score | Feedback\n",
+ " --------- | ----- | --------\n",
+ " Feedforward and Cost Function | 30 / 30 | Nice work!\n",
+ " Regularized Cost Function | 0 / 15 | \n",
+ " Sigmoid Gradient | 5 / 5 | Nice work!\n",
+ " Neural Network Gradient (Backpropagation) | 40 / 40 | Nice work!\n",
+ " Regularized Gradient | 0 / 10 | \n",
+ " --------------------------------\n",
+ " | 75 / 100 | \n",
+ "\n"
+ ]
+ }
+ ],
+ "source": [
+ "grader[4] = nnCostFunction\n",
+ "grader.grade()"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "\n",
+ "### 2.5 Regularized Neural Network\n",
+ "\n",
+ "After you have successfully implemented the backpropagation algorithm, you will add regularization to the gradient. To account for regularization, it turns out that you can add this as an additional term *after* computing the gradients using backpropagation.\n",
+ "\n",
+ "Specifically, after you have computed $\\Delta_{ij}^{(l)}$ using backpropagation, you should add regularization using\n",
+ "\n",
+ "$$ \\begin{align} \n",
+ "& \\frac{\\partial}{\\partial \\Theta_{ij}^{(l)}} J(\\Theta) = D_{ij}^{(l)} = \\frac{1}{m} \\Delta_{ij}^{(l)} & \\qquad \\text{for } j = 0 \\\\\n",
+ "& \\frac{\\partial}{\\partial \\Theta_{ij}^{(l)}} J(\\Theta) = D_{ij}^{(l)} = \\frac{1}{m} \\Delta_{ij}^{(l)} + \\frac{\\lambda}{m} \\Theta_{ij}^{(l)} & \\qquad \\text{for } j \\ge 1\n",
+ "\\end{align}\n",
+ "$$\n",
+ "\n",
+ "Note that you should *not* be regularizing the first column of $\\Theta^{(l)}$ which is used for the bias term. Furthermore, in the parameters $\\Theta_{ij}^{(l)}$, $i$ is indexed starting from 1, and $j$ is indexed starting from 0. Thus, \n",
+ "\n",
+ "$$\n",
+ "\\Theta^{(l)} = \\begin{bmatrix}\n",
+ "\\Theta_{1,0}^{(i)} & \\Theta_{1,1}^{(l)} & \\cdots \\\\\n",
+ "\\Theta_{2,0}^{(i)} & \\Theta_{2,1}^{(l)} & \\cdots \\\\\n",
+ "\\vdots & ~ & \\ddots\n",
+ "\\end{bmatrix}\n",
+ "$$\n",
+ "\n",
+ "[Now modify your code that computes grad in `nnCostFunction` to account for regularization.](#nnCostFunction)\n",
+ "\n",
+ "After you are done, the following cell runs gradient checking on your implementation. If your code is correct, you should expect to see a relative difference that is less than 1e-9."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 18,
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "[[-9.27825235e-03 -9.27825236e-03]\n",
+ " [-1.67679797e-02 -1.67679797e-02]\n",
+ " [-1.75060084e-04 -6.01744725e-02]\n",
+ " [-9.62660640e-05 -1.73704651e-02]\n",
+ " [ 8.89911959e-03 8.89911960e-03]\n",
+ " [ 3.94334829e-02 3.94334829e-02]\n",
+ " [ 2.33146358e-04 -3.19612287e-02]\n",
+ " [ 1.17982666e-04 -5.75658668e-02]\n",
+ " [-8.36010761e-03 -8.36010762e-03]\n",
+ " [ 5.93355565e-02 5.93355565e-02]\n",
+ " [-2.87468729e-04 2.49225535e-02]\n",
+ " [-1.37149709e-04 -4.51963845e-02]\n",
+ " [ 7.62813550e-03 7.62813551e-03]\n",
+ " [ 2.47640974e-02 2.47640974e-02]\n",
+ " [ 3.35320351e-04 5.97717617e-02]\n",
+ " [ 1.53247082e-04 9.14587966e-03]\n",
+ " [-6.74798369e-03 -6.74798370e-03]\n",
+ " [-3.26881426e-02 -3.26881426e-02]\n",
+ " [-3.76215583e-04 3.86410548e-02]\n",
+ " [-1.66560294e-04 5.46101547e-02]\n",
+ " [ 3.14544970e-01 3.14544970e-01]\n",
+ " [ 1.18682669e-01 1.18682669e-01]\n",
+ " [ 1.64567932e-01 2.03987128e-01]\n",
+ " [ 1.58339334e-01 1.25698067e-01]\n",
+ " [ 1.51127527e-01 1.76337550e-01]\n",
+ " [ 1.49568335e-01 1.32294136e-01]\n",
+ " [ 1.11056588e-01 1.11056588e-01]\n",
+ " [ 3.81928711e-05 3.81928696e-05]\n",
+ " [ 5.77867378e-02 1.17148233e-01]\n",
+ " [ 5.59235296e-02 -4.07588279e-03]\n",
+ " [ 5.36967009e-02 1.13133142e-01]\n",
+ " [ 5.31542052e-02 -4.52964427e-03]\n",
+ " [ 9.74006970e-02 9.74006970e-02]\n",
+ " [ 3.36926556e-02 3.36926556e-02]\n",
+ " [ 5.07530173e-02 7.54801264e-02]\n",
+ " [ 4.91620841e-02 1.69677090e-02]\n",
+ " [ 4.71456249e-02 8.61628953e-02]\n",
+ " [ 4.65597186e-02 1.50048382e-03]]\n",
+ "The above two columns you get should be very similar.\n",
+ "(Left-Your Numerical Gradient, Right-Analytical Gradient)\n",
+ "\n",
+ "If your backpropagation implementation is correct, then \n",
+ "the relative difference will be small (less than 1e-9). \n",
+ "Relative Difference: 0.194982\n",
+ "\n",
+ "\n",
+ "Cost at (fixed) debugging parameters (w/ lambda = 3.000000): 0.292002 \n",
+ "(for lambda = 3, this value should be about 0.576051)\n"
+ ]
+ }
+ ],
+ "source": [
+ "# Check gradients by running checkNNGradients\n",
+ "lambda_ = 3\n",
+ "utils.checkNNGradients(nnCostFunction, lambda_)\n",
+ "\n",
+ "# Also output the costFunction debugging values\n",
+ "debug_J, _ = nnCostFunction(nn_params, input_layer_size,\n",
+ " hidden_layer_size, num_labels, X, y, lambda_)\n",
+ "\n",
+ "print('\\n\\nCost at (fixed) debugging parameters (w/ lambda = %f): %f ' % (lambda_, debug_J))\n",
+ "print('(for lambda = 3, this value should be about 0.576051)')"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 19,
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "\n",
+ "Submitting Solutions | Programming Exercise neural-network-learning\n",
+ "\n",
+ "Use token from last successful submission (tanmayss2109@gmail.com)? (Y/n): y\n",
+ " Part Name | Score | Feedback\n",
+ " --------- | ----- | --------\n",
+ " Feedforward and Cost Function | 30 / 30 | Nice work!\n",
+ " Regularized Cost Function | 0 / 15 | \n",
+ " Sigmoid Gradient | 5 / 5 | Nice work!\n",
+ " Neural Network Gradient (Backpropagation) | 40 / 40 | Nice work!\n",
+ " Regularized Gradient | 0 / 10 | \n",
+ " --------------------------------\n",
+ " | 75 / 100 | \n",
+ "\n"
+ ]
+ }
+ ],
+ "source": [
+ "grader[5] = nnCostFunction\n",
+ "grader.grade()"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "### 2.6 Learning parameters using `scipy.optimize.minimize`\n",
+ "\n",
+ "After you have successfully implemented the neural network cost function\n",
+ "and gradient computation, the next step we will use `scipy`'s minimization to learn a good set parameters."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 20,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "# After you have completed the assignment, change the maxiter to a larger\n",
+ "# value to see how more training helps.\n",
+ "options= {'maxiter': 100}\n",
+ "\n",
+ "# You should also try different values of lambda\n",
+ "lambda_ = 1\n",
+ "\n",
+ "# Create \"short hand\" for the cost function to be minimized\n",
+ "costFunction = lambda p: nnCostFunction(p, input_layer_size,\n",
+ " hidden_layer_size,\n",
+ " num_labels, X, y, lambda_)\n",
+ "\n",
+ "# Now, costFunction is a function that takes in only one argument\n",
+ "# (the neural network parameters)\n",
+ "res = opt.minimize(costFunction,\n",
+ " initial_nn_params,\n",
+ " jac=True,\n",
+ " method='TNC',\n",
+ " options=options)\n",
+ "\n",
+ "# get the solution of the optimization\n",
+ "nn_params = res.x\n",
+ " \n",
+ "# Obtain Theta1 and Theta2 back from nn_params\n",
+ "Theta1 = np.reshape(nn_params[:hidden_layer_size * (input_layer_size + 1)],\n",
+ " (hidden_layer_size, (input_layer_size + 1)))\n",
+ "\n",
+ "Theta2 = np.reshape(nn_params[(hidden_layer_size * (input_layer_size + 1)):],\n",
+ " (num_labels, (hidden_layer_size + 1)))"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "After the training completes, we will proceed to report the training accuracy of your classifier by computing the percentage of examples it got correct. If your implementation is correct, you should see a reported\n",
+ "training accuracy of about 95.3% (this may vary by about 1% due to the random initialization). It is possible to get higher training accuracies by training the neural network for more iterations. We encourage you to try\n",
+ "training the neural network for more iterations (e.g., set `maxiter` to 400) and also vary the regularization parameter $\\lambda$. With the right learning settings, it is possible to get the neural network to perfectly fit the training set."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 21,
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "Training Set Accuracy: 95.920000\n"
+ ]
+ }
+ ],
+ "source": [
+ "pred = utils.predict(Theta1, Theta2, X)\n",
+ "print('Training Set Accuracy: %f' % (np.mean(pred == y) * 100))"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "## 3 Visualizing the Hidden Layer\n",
+ "\n",
+ "One way to understand what your neural network is learning is to visualize what the representations captured by the hidden units. Informally, given a particular hidden unit, one way to visualize what it computes is to find an input $x$ that will cause it to activate (that is, to have an activation value \n",
+ "($a_i^{(l)}$) close to 1). For the neural network you trained, notice that the $i^{th}$ row of $\\Theta^{(1)}$ is a 401-dimensional vector that represents the parameter for the $i^{th}$ hidden unit. If we discard the bias term, we get a 400 dimensional vector that represents the weights from each input pixel to the hidden unit.\n",
+ "\n",
+ "Thus, one way to visualize the “representation” captured by the hidden unit is to reshape this 400 dimensional vector into a 20 × 20 image and display it (It turns out that this is equivalent to finding the input that gives the highest activation for the hidden unit, given a “norm” constraint on the input (i.e., $||x||_2 \\le 1$)). \n",
+ "\n",
+ "The next cell does this by using the `displayData` function and it will show you an image with 25 units,\n",
+ "each corresponding to one hidden unit in the network. In your trained network, you should find that the hidden units corresponds roughly to detectors that look for strokes and other patterns in the input."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 22,
+ "metadata": {},
+ "outputs": [
+ {
+ "data": {
+ "image/png": "iVBORw0KGgoAAAANSUhEUgAAAjkAAAIuCAYAAABdOBlOAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjMsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+AADFEAAAgAElEQVR4nOy9adBnZXn1uzSiQGSwbRroeZ7nkWZuZBKZNEQs5ymVVMoMaipfkpgymlTlJVZSZZmkNEYlVsQJ1KDMCD3R9EjP8zx304g2CE7wfjl1Pvyv33rOfirn1Knc7/p9vNj3s/e+p33z77Wu6zWvvvqqQgghhBBa47X/fz9ACCGEEML/F+SQE0IIIYQmySEnhBBCCE2SQ04IIYQQmiSHnBBCCCE0SQ45IYQQQmiS1/X1Hz/3uc8Vf/kvf/lLvPbNb35zif3kJz8psRdeeAHbn3322SX2mte8plNMkn7961+X2IABA/Daru3duxLPP/98iZ133nmd7/WrX/2qxKhPJenZZ58tsU996lOlYz71qU+V8TvrrLPwbxL0To7Xvrael3/7t3+7xF7/+td3bk9z5cUXX8T2F154YYlRn0rSmWeeWWI/+9nPSuwNb3gDtqd5Qe/lnpXm5cc//vEyfn/xF39Rxo/WicR9RWN97rnnYvvjx4+X2KBBg0rs2LFj2P78888vMepT+puS9Fu/9Vsldvr0abyW+ppSYbi59pvf/KbETp48WWJvfOMbsf1LL71UYn/7t3+LG9NXv/rV8mB0f0l63evqdkzj/corr3R+LhoX2j8kadiwYSVGa4ieU5Kee+65EnNjQONF/U37grsXPZfbA4h3v/vdZQy/+93vlgd170/fJtoDBg4ciO1/8YtflNgZZ5xRYu4bSuud5hrdR+J9yfUfjSvdi55fkg4fPlxi1H8XXXQRtqf5c8011+AazC85IYQQQmiSHHJCCCGE0CQ55IQQQgihSXLICSGEEEKT9Ck8JuEpiUndtSQac0Kil19+ucT6I5IlkSOJpug+EoumnPCXhF/nnHNOiTmRqBNP93L06FGMO0FsLyTOcqI5EoidOnWqxAYPHoztT5w4UWI//elPS8yJ7ro+k+tT6hPXzyRoJdGdE16TcJxEk04gSH1FkEiYBJcSjys9Ewl8Je5XGj8Sp0q8ri644IISc8Jlwomkqf+pr53okdYqPasTzbo1RNAe5PqQ1iuZN0hgLEnDhw8vMeqrN73pTdie/i7NAeorifvLrUEaG7rXz3/+c2xP3yHaA5xw1vVhL7Qv0JhI/P70nG7903olkblbw/S9pD6lOSFJhw4dKjH3vac+IJG7g+YQfcMdJFx25JecEEIIITRJDjkhhBBCaJIcckIIIYTQJDnkhBBCCKFJcsgJIYQQQpP0aRMgx4VzHJFim1wszt1EqfbJSeJKLZCyntKXO8fGmDFjSswp80ndTk4clxaeHAPULy6leddyFeQgcO4kSus/ZMiQEnMp5bumBHfuLHLdUJ84Zwg5U5yTh+YAub6ci43mpXNSES6tfy80f1ypAyprQfdxpSbI7UJr0q1/cnbQ+nPtyXFC7j6J5yXtP87ZQ/1C/erWSn9cIPS++/btw2uHDh1aYrRXONcYPRetF+dwPHDgQInRfuv6hebA9OnT8VrqF3rXp59+GtvTc+3cubPE3H7hHEq90FyhdSHxfkd7iHMY0xykPnGOM3Ljjhw5ssTcd4kcS26s9+/fX2L0vXFlXGhtdnUNS75kEpFfckIIIYTQJDnkhBBCCKFJcsgJIYQQQpPkkBNCCCGEJulTeEwCTRKoSixmI3GXEw6TcJMEXr/4xS86t6fYxo0bsf3jjz9eYnPnzsVrSRBMoiknfCVBLb0riUkln9q/FxL3OeE3jTUJ6dz4UZwEkq5UBY3rzJkzS8wJ2aivnUCTBIrU3qX1P3nyJMZ7ccJXEs4SNH+dGN69ay8kRHTQmLhSFxSnOeX6lJ6Lyi9ILIbcsGFDiTmBNwkkKSW9W3/OvEDQXuEMBbSGSCTq0ueTcJTS748bNw7bkyCahLurV6/G9jQHqKyAJF155ZUlRnsTlRqQeB+h/c6VJegqPKZncvsvCa+pvXsm+rs0fm4NXnzxxSV25MiREnPCaSo34kTOZEDpj9GA+qpruSPJryG8tvOVIYQQQgj/g8ghJ4QQQghNkkNOCCGEEJokh5wQQgghNEmfwmMSPjrRFImOSAzphKskXCbh3549ezq337FjR4mR4EliQecDDzyA106YMKHESCTpxG2jR48uMXpXygIr+SyUvVAWWCfGpLEi0SBlRZU44zHNCZfxeNq0aSVGYm4nHKd3HT58OF5L/Urv+rrX8fKguUJiVCdQdZl8u1xH4kCJBb0k0naiQxJTk8nAiQNp/v53RZtOtErvQAJJt1fQ+M2fP7/E+iMwdtBzuTVAGYspk7czf5B4nuawE//T3k5r5ZlnnsH2NF7bt2/Ha1esWFFi/claTYLsLVu2lBiJcfuK90Lv5DLO03yh7x29p9Q9w7jLIk37BQmEqe8lac6cOSVGlQDc36U9gLJ4S7wP0Df8v/sNlPJLTgghhBAaJYecEEIIITRJDjkhhBBCaJIcckIIIYTQJDnkhBBCCKFJ+nRXkYLZOYZc+ueu7UeNGlVipEI/ePAgtifF+w033FBi5BiTOKW5u5acVOSC2L9/P7YnZTq5S1wJA+c66eXFF18sMXIsSdLp06dLjPravRO5nsiFRs8kSVu3bi2x9evXl9iTTz6J7cnFMHv2bLyWXBDjx48vMUppL3G5A3InufTr5K4jzjvvvBJz6cxp/u3du7fzM9G6JAeEKwtBc5Ke/6KLLsL25Nhx7qjLLrusxMh15lxMlCqf5qUrodEfyPXmShXQvKTxdg5XYteuXZ2eSWJ3Ds0h2qsldjO6e337298uMRpvcr1J0tixY0uM9jbnJu3qnCPHD80fiR1f9K1wpYnoG0bfYHLhSfys5K5z84e+4a60yrvf/e4So3I57l0XLVpUYl1LA0nS7t27MU7kl5wQQgghNEkOOSGEEEJokhxyQgghhNAkOeSEEEIIoUn6FB6T8M6lpKYU7pMmTSoxl1aeRJ4khBoxYgS2J5HumjVrSmzTpk3Y/vLLLy8xJ5JdvHhxid10000lRqUKJC53QKIrSuktSUeOHMF4LyRac/1P40rtXZrvl156qcRIZDpx4kRsTyLva6+9tsSoVIfEQjQn0CORMc1fJ9IlkTbRH4EfQaJNl9Kf7kXtnZCQ0qf3p6QA9d873vGOEiNxpsRz5d5778VrqXzBLbfcUmKrVq3C9lQqgOY/ickl6YILLsB413u5eUHrndYVCbolFtOSSPvxxx/H9iRyft/73ldi119/PbanvfW+++7Da0lkS33lhPbUVzSGTnxOIl2CBPlkHJF4XEn47PZgEk6ff/75nZ5J4r6iMjxkCJG41IUrn0CCYvreu72S9gHqKycQp35x5JecEEIIITRJDjkhhBBCaJIcckIIIYTQJDnkhBBCCKFJ+hQek/DSie5ItERiUidcJSHYE088UWIkhpSkRx99tMRIiHb11Vdje8rkevjwYbyW3osy8brswCQI/eAHP1hiJBqTvBisFxKykehZYjEniVRdxlrqE+p/EhxKLDoj4afLNkqC9JkzZ+K1Q4YMKTHqKyfwpIzFJCZ2wnUnPOyF+sRlkKW+ItGly+JLosnt27eXmMu2vWTJkhIjgeb/+l//C9vT/HMmB5prJESlLMgSZ3Luj0DYZdElaK06MSaJLElg6TImT5kypcRoXXQVzkssHnf7D2WtdiLX9773vSVGGZfXrVuH7WltU9Z4msNS9zV49OjREnOZ6Gm+0Fxza5jW5ubNm0vMZeKn/YoEvi7jOq1XMsRInEmb9kCXIZ3MM7QG3Vx15wAiv+SEEEIIoUlyyAkhhBBCk+SQE0IIIYQmySEnhBBCCE2SQ04IIYQQmqRPdxWpwMmFIrFanZTVlD5eYicT/U2n7J83b16JkbNg+vTp2J6U4eRWkFiFvn79+hLbtm0btqcSEmvXri0xcgtIPtV1L697XR1e547bs2dPic2dO7fEhg4diu2XLVtWYuSYc+NHKn5yNjhnAI21cwJRWnRqP3bsWGxPrjlKv+/KAlBKdIL+pkvpTn1N7VeuXIntafzpb5IDRuK5Qk5K5+6jvnIOGHLoUQmPgwcPYntyvZHjzpUFceuyK86dRW4yKmPiUtrTHkZOLEq/7yAXjHt++jaQO8tBTpoPfOADeC3tIzS3nEPWOfd6oTXg1iA9P+0VzvFF9yJ3lvuG0R5G33C3L27ZsqXE3Fyj5/qv//qvEnMOxyuvvLLEyM3t3KBubRL5JSeEEEIITZJDTgghhBCaJIecEEIIITRJDjkhhBBCaJI+hcckOnJp9d/0pjeVGKX5doKhqVOnlhilaqf7SCxQIuElpcmWWGTsBGYk8Jo1a1aJXXrppdh+1apVJUZiVJc+3KUF74Xe34kGSVDcnzTblGadRKpu/C+77LISIyGaE4NTnMpSSNJrX1vP9jTXKHW5xPOChHgkUJe4XwkaK1cqgkS+JETdtGkTtqfxJyGkMx5QWQYSre7duxfbk5j+wgsvxGsp1T8J0t1c61qCg+aJxGPtoGdw4n3aA+ha1y9kXqDYHXfcge3pfWkNkCFAkhYsWFBirq+2bt1aYvReL7/8Mran9UqlhVwZGidI7oVEumTokNg8Qc9PpRYk3i9pTGhduntRCYjly5dje9rb3b127txZYnRecN8qEjnTN9y1dyVXiPySE0IIIYQmySEnhBBCCE2SQ04IIYQQmiSHnBBCCCE0SZ/CYxKNOdEkiZ5IOOoyHpNI9nd/93dL7KGHHsL2a9asKTHKuDxx4kRsT8JnJ9KljKf0rhMmTMD2JPCi/nPiKido7YWEcE70RuNKGWPPO+88bE9iRsqu68TAJBy99tprS4z6WeKxdqK1hx9+uMR+9KMflZjLznzGGWeUGPW1E4i6edULCfGcaJWEsySwdUJOEm2OGjWqxB588EFsf9ttt5XYDTfcUGJOjL948eISu+uuu/BaEj/TvHYiaeoX2utcXztDAkFiYrcH0rygPcQJokmMS+vNZeLet29fiX39618vMdqXJc54S/NK4rlFmZhdZmIyoNAaoLUqdReukpjWCY9JzExZmN0z0X5LhgyXcfv48eMl9uqrr5bYE088ge3JlDB79my8dsaMGSVGfeqMSrSGqK9dX9F7OfJLTgghhBCaJIecEEIIITRJDjkhhBBCaJIcckIIIYTQJDnkhBBCCKFJ+nRXUUpsp+wnZTcp48mFIklPPfVUiVFKdJeqndJMz5kzp8TIRSWxWtuVMKA+IHeHc6KNHz++xFy/EF2V5eQCcM4sSilOKn7nODp06FCJkdqeXFQSu+vIHeUcL9Qn//mf/4nX7t+/v8S2bdtWYuQAkdgxQmNN7ih3LXHBBReUmHOMkQuB3Bpjx47F9uvXry8xWmsuTT7NFXJgUJp5iR1vzh21cOHCEqPxd+3JcUSOqzPPPBPbO3cLQeNCpQIcNFedQ3LDhg0lNmzYsBKjtS7xeNN6daU5iPnz52Oc5rabGwQ5xB544IHO7V3JnF5oD3Xrmtxt5KTbs2dPp3tL7Lhz85ocS08++WSJ/fCHP8T25AQkh6vEzlHaG2hfl7gP6Hvh5hqVsXHkl5wQQgghNEkOOSGEEEJokhxyQgghhNAkOeSEEEIIoUn6FB6TEIkEehKL+Uig68R8JLD64he/WGKU+lmS3vKWt5TY5ZdfXmKTJ0/G9iQwu/vuu/FaSh9OYjQnpCMxHwknnWirq3CVBIr07BKL5ijN+7hx47A9id6mT59eYk74SsJzErO7sgQUnzJlCl67evXqEhs8eHCJDR8+HNvTtfSsDlfaohcSGbuSELRWSQjoSo3QuiKBIwn8JWnHjh2drnXi0OXLl5fYLbfcgtfSe5Fwn8oESLwuSQxMpR4kLinQH9weSOt1+/btJeZMCrSGab9z+xIZNWgNU6kHiecmid8l3m8HDBhQYjSvJC4DQt8mV4amq/ib9mU3L0jkTXOVrpPY0LBq1aoSo7UiSevWrSuxL33pSyXmjCuf/exnS2zWrFl4Le33tAe60jZk9KASEM784caAyC85IYQQQmiSHHJCCCGE0CQ55IQQQgihSXLICSGEEEKT5JATQgghhCbp011FuJTWpIwnKB23xGm+Bw4cWGLPPvsstqdyA6TsdinRyXFz+PBhvHbkyJElduTIkRJzThhyFpDjwZXQcH+3l67lHyR2UlFKetcn5GwgZbwbP3Ks0PtTmnJJuvTSS0vMpU+nv0uuIxpnidP6kzPAuVhcaYZeaJ44dxytPxp/VxaF7jV37txOMYnLbdD8cc6i66+/vsSci5CcFbRXuBIiVIKExsqtMxprB6Wqd+uSSjjQtS4t/7Rp00qMSshQTOI1ePTo0RIj16rEbkRXBmTlypUl9q1vfavEaA5J3C/0XuQCkro7VAlysblnovIDzjU7b968EqN1uXTpUmxPc/g973lPiTl3J+1Lrv/IjXns2LESozktscOM1pvbK90+QuSXnBBCCCE0SQ45IYQQQmiSHHJCCCGE0CQ55IQQQgihSfoUHlP6cBJdSiz8/fWvf11ilBJf4hT85557bol95StfwfaUUprSX584cQLbP/300yVGabYlFkjNnz+/0/0lflYScr3yyivYvqvwsT9CLhIdkpDOCYdJSLhly5YSc2UFSKC5YcOGEqN5Jklr164tMSdavPnmm0uMhKsudfjp06dLjETuLnW8E/P1Qv3//PPP47UkxCMhphOCkhiXUuK7ZyfRIwm/3Zp65plnSoz6WZJmzpxZYlRuxK0fEl+TEJVKZUj9Ez1SaQxX6qCrocCluqf9lvZwt3/QfkuCdic+HzFiRIlR+QWJx5b6m/YFyYv6e3FlBZwBpst17plIJEz7rRs/2q9IuOyE6/RtpT4lMbnEhgxXgoLWK7UnQ47EBiAyL7jvtTMQEfklJ4QQQghNkkNOCCGEEJokh5wQQgghNEkOOSGEEEJokj6FxyTcc8JPEpKR8PD+++/H9iTmIiHSjTfeiO1ffvnlEhs0aFCJOeEsCfQog6fE4k8SjU2aNAnbk3ibMgY74asTVPbSH+EqCY+pr5yQjzJh33vvvSXmhGwk2rvppptKzGVcJuHs7bffjteS+PXAgQMl5vqK5gVl+3TZhV0m615o/EgMLPGcIoG7az9kyJASIyGsExLSWtu2bVuJbd68GdvTvCAzgsRiyh07dpQYickl3msoC60TrpPA10FzwP3dyZMnl9ipU6dKjPYqiecAxVy2371795YYzReaKxLPaxJeS5zhngTltC4l3i9JUOvmu4v3QtmNaV1J/E40r5xwfffu3SVG3zWXyZ3mFYnM3/GOd2B7Ep47kTrNS1qXbg3SHkzz2om8XdZpIr/khBBCCKFJcsgJIYQQQpPkkBNCCCGEJskhJ4QQQghNkkNOCCGEEJqkT3cVuSucM4BU2JQCfsGCBdielNWkLKc01xK7e+65554Smz59Oranv0vuEPdclFbepaUnZTo5NlxfO3dEF5yzh8aKyhKQi0hix8U111xTYs7dRs4ScvI5dw+5qy6//HK8tqu75pxzzsH29AzkonElULqOH70TueAkHquRI0eWmOt/unb//v0lRq4WSXrkkUdKjFKvO7cNPZdz8dBY0ZqkPcE9A93frb+u7jiJ91Dn7KF5Ra5D2mskdh7SeznXH6XQd31IUGmaZcuW4bXUB7QHuBIIXcsluDXoHGpdcKWJqP+oT1y5lxdffLHEyEVEJYQknpf0/u75aa46dxvtofR3XRkYmpdUcsbtla7kCpFfckIIIYTQJDnkhBBCCKFJcsgJIYQQQpPkkBNCCCGEJulTeEyit5MnT+K1lKZ5wIABJeZKHZDwloSLJDqUWCRIQj5XloJSbZMQSuJ+IYGWE2mSGIv+pivf4N6hFxI4U+pxSXrhhRdKjMR9Ls33vn37SozmBAlcJS6hsG7duhJzZSVuuOGGEqP5I7Ggl0TGTtxGAj0qweCEqy7eCwlk3ZygZ6I55eYOCUzpnVx7KstAInUnRKS+XrNmDV5Lc4gEmvT8EhsCqL0T2TvxNEECTVeuxaWw78UZGkh4u2XLlhJz64L2YFrXThBOYl7aAyTeh2gPnTVrFrYnkSsJf11fU8kgggTtbvxpb6JSCW5fof6jfdEZImhfIeGx65MNGzaUGJX6kHgOkfB9/Pjx2J6egdabK4FB4+LILzkhhBBCaJIcckIIIYTQJDnkhBBCCKFJcsgJIYQQQpP0KTymbIMugyQJgUjIRgJXSRo7dmyJUWZOEsO6exFO3Hf8+PEScyJPygRLfeWEy3QvEmk7gVhX4TG9q8t4S/enrKRONEtiSHr+o0ePdr4/MXDgQIxv3LixxCiDqMRzjfrKZdt0ws9enECxa8Zjt9YIWlfU104Mv3379hIjMTMJKSVeKzR/XLZgMhQ4kTnNQRJtun6mjLO0ptxe4fYwguaKy1pNpg4SajvRLO2XM2bMKDESpEss1CbhKGXxde1ddl96huHDh5eYGwPqV9obnEC16/eC1rBbQyTep5gzb9B+RSJvZ74hke7OnTtLzIn/aQ25fbmrKcV9w+jvUl874bGbV0R+yQkhhBBCk+SQE0IIIYQmySEnhBBCCE2SQ04IIYQQmiSHnBBCCCE0SZ/uKlKwO7U1ORlIge2U/Tt27CgxUns7ZTg5LsjF4FTZpOKmsggSpySna6l8gMQpsamvyN0idXfn0DtRn0rsuqK+dmm6SfFPynrnLKF7kYuF0u+79uTWkPhZqb1zYFAfkjPAjROlnydoTlCad3cvWr/umciFQvPUlToYMWJEiR08eLDEnDOQXFuuLAOtYZorzplDri9av87Z4VyTBO2Xrg9pb6S9xrWnUgf9SX9P843WhStLQO3JXSSxQ4tcp87JSHOD9kvnxnUu017oXd03rGupA/dMtIfQXuHmJZV7IMeWWxfUp86hSn+D9nb3DSMnIDnJ3B7sXJ5EfskJIYQQQpPkkBNCCCGEJskhJ4QQQghNkkNOCCGEEJrkNU7cF0IIIYTwP5n8khNCCCGEJunTQv5P//RP5WceZ+F2xQh7cZYyKnpGNjFnaSNbH1nqnIWcrI7OqkeFE+n5+2O3J2uwK9BIffiRj3yk/NEvfvGLZfycJY+sqmSVdu2pr/pjYSZrOF3r5g8ViHR2eSoSSc9PBRMltorS/OtPgUgav09/+tNl/JwFm9Yf2aKd9ZKsxmSfdcUhqa+on52lmK51xWBprtCzul+pXYHEXtxco7H+y7/8Sxzs3//93y8P4dY1zUG6l9sDqb9oDbo0DNSHlAbCFegkC/HevXvxWnoG6he3hig9B42Xe1dKI/BXf/VX5Wb/+I//WMbPzUsaK5qDrsAmPT+1d4WjaW3Tfu2+gTR+rqBu138BcucC2ptob3DfUHquP/iDP8DJkl9yQgghhNAkOeSEEEIIoUlyyAkhhBBCk+SQE0IIIYQm6VN4TAIrEl1KXDumP/WASExFYsiLL74Y2x85cqTESGDohFAkpnMiSXpWV8+EIJEm1TOh+h4Si+YIEl474SoJyej+TnRH40r1TVztpQMHDpQYCeFc3RwSKDrRIs1V6lNX94fEcCTQHDRoELZ39bt6obnuxNBda7e5ujkkeqX6Pk6MS31CY+3GhESnbv3Rc9Fcc8JzWhd0Lyfw7Lr+JK5z5YTPtDZprrp6TjRf6G864TLt1/Q33b5Ez0p9LfHY0Nruz/fiwgsvxGsJtw/1QvuiM6RQ7S0afzd+9P5dTQ4S91XXOntS/+Ya9TWJiZ1AmQwMZN6hfVXq3/c2v+SEEEIIoUlyyAkhhBBCk+SQE0IIIYQmySEnhBBCCE2SQ04IIYQQmqRPiTIpo/ujDCfIBeHipAJ3jodhw4aVGKngnTvo2LFjJTZu3Di8ltJikzvD9QldSyp4l/7duRt6IQW6c+cMHDiwxGj8nTKfUpXTXHF9Qi4CctG48aN7kWNLYscIjYlzQZFDiPrFOQu6rhWak1R+Q2JnRdd54q6l1On9KatCDgxyUEjszHBzjf7G/v37S8yVsKB9YciQISXmnD0u1T1BY+je68SJEyVG7+pK69AYjhkzpsTcHKL5To6bNWvWYHtyLDknGpV76E8ZEdoH6P2dQ8+Ve+iF3IiHDx/Ga6n/ujppXfs9e/aUmHMY0x5I84f+piTNmDGjxJzDdOfOnSVG33DnxqbvOK1L5+Z0a5PILzkhhBBCaJIcckIIIYTQJDnkhBBCCKFJcsgJIYQQQpP0KTwmcQ+lb5dYCEYiRSqfIHVPf+6EZASlNKfU2+7a++67D6+dOXNmiZHoi0R7Eqc6JzGrS13t0tX3QqI3N3401tQnJPCUWLhMAksnPKcSCDR/tm7diu1HjRrV6ZkkFu6RwM6VIPjvlgUgkXbX+7g053QvGj8n3KZrSQjpxNQk/Ke/6dYvCVHd/Ke/u3jx4hIjIa/EwlsSODrRoxtXYvDgwSXm1i8Jd0eOHFlihw4dwvZklCDx9/jx47H9wYMHS4zE2060S4JWN9/oXem9lixZgu1HjBhRYhMmTCgxV1bClbfphcSwrqwD7W20BzsxNs1XmoNLly7F9nQvGj9nqKF9kfpZ4v6j93LfC9rH6BvkjApubybyS04IIYQQmiSHnBBCCCE0SQ45IYQQQmiSHHJCCCGE0CR9Co9dtkKCsgBTZk4SmEos0iXR2cSJE7E9ib5IIOiEaCQmu/XWW/FayqRLgubdu3djexLuUV+5Z+2arZMys7psuyRwowyiTjhOAjF6Tpetk8SQNCdIXCjx+FO2WYkzltKzuozBlImZ1ooTmHYVrtKccGJcEgSTENBlWyWBKglBp02bhu1JeE9iZJeFeNasWSVG4lSJReKjR48usYcffhjbk8CRBJJOoOsyDhNd54rEIl3aVyg7s8TrnTIGDx8+HNvT2qZ93c1fGi/3rmQKoP1uypQp2H758uUlRqYEJ7LtuoeS+cWZN2i/ofV69OhRbE/zivaqjRs3YnuC5rAbvyeffLLEXP8tWLCgxGgPcsJz2kqR9oIAACAASURBVBsp1h+jkyO/5IQQQgihSXLICSGEEEKT5JATQgghhCbJISeEEEIITZJDTgghhBCapE93FblbnLvjrLPO6hSj9OtS9xTylHpcYhU6pW//4z/+Y2xPKnpXlmHq1KklRi6IsWPHYntKf04ujEsuuQTbuzHohd6fXDASK+4pzbhLy08lHMjF4tJx01yhsXbp2MkJ5VKK33LLLSVG7+pcCJRqn5whlGZdYscKQS4iiklcPoAcZ66sxty5c0uM3IGu1MHkyZNLjPrPzR8a/507d+K15EIhdxg5iyQuq9Gfsg70rA4qTeIcptRf5C6ZPXt25/uT69Hta+RG60+/kPPV7Tfk5KJnpT1EkmbMmFFi5MaleSF5514v5MJypQbIHdSf0kK0L9C+6hyOtLbJneaen1xfK1euxGtpHZO7z7nrqD05P51D1TmPifySE0IIIYQmySEnhBBCCE2SQ04IIYQQmiSHnBBCCCE0SZ8KVhICOcEPifFIdOeEpyQ8JoEuiWklae/evSV22WWXdXom93e/973v4bWUvpvEyM899xy2J1avXl1iY8aMwWtdH/RCoj0nGiThLl3r0mmTIJYEuiTwlqRRo0aV2P33319iI0aMwPYkZHTp9x9//PESo3ndH5E9pT93JRRcH3S5vyuLQPeiNPck2JRY5N0f4fiOHTtKjOa/K3VCz+XEoVSahASObvxprZJocs2aNdjeCXcJEr66PqA1QM/lxLi0t9GzOuH79u3bS4xKg9xzzz3Yfs6cOSXm9qp9+/aVGK1h970hkSyV26H+k7qXdeiPeP6iiy4qMTIfkMnEQWLi+fPn47W0h5Nw15XmIZGzK8NDJV9ovTmRPn3v6V1dCYyu5g0pv+SEEEIIoVFyyAkhhBBCk+SQE0IIIYQmySEnhBBCCE3Sp/CYRF8uA+GuXbtKjDIWkzhNkrZs2VJiJE6jbKWS9LWvfa3EKLPk5z73OWxPAjUSzkqcMXLWrFklRtlKJRZdkcjUZbd1gtZeSKDphMd0f3p+EqdJLEglgZ7rUxKZkhiV+lmSNm7cWGIk5pZYDEoCQ9dXNC9pTJzAsKsgneakE0MfOXKkxEj054SEJFImgey1116L7U+dOlViq1atKrEf/vCH2J7Get68eXgtzUvaq2j+SzyH165di9cSTnhK0Lx24n0S/g4ZMqTEnnzySWxP70V7oFtDJFxdv359idFeLUkLFiwoMSc8JpHujTfeWGIkaJd4DQ8cOLDE3PeKvk0ECV/d+JPwlvYQtwfSWJGhwWXSp/lD93fvTnPNGQ1ovyNDhXtX2q9IDO7WCmV4d+SXnBBCCCE0SQ45IYQQQmiSHHJCCCGE0CQ55IQQQgihSXLICSGEEEKT9OmuopTULh02pXmmFPzOGUTK/okTJ5YYKdAl6Yorriixj370oyXmlOnkgiDHiCTdeuutJTZ58uQSIxeVxC4Aei4qHyFJZ599NsZ7IXeWSwlPTiB6Tpemm649ffp0iTl3FrkA3vGOd5QYOQAknpeuBMKZZ55ZYs8880yJkWNLkjZs2FBiNK9Hjx6N7V2q+V7IreAcXzTXKKW/c6uQY4ZchORCk6RJkyaVGK1VWqcSz3W31q+++uoSIyfV8uXLsT05NiilvFv/zvVIUFkEl5KeHEfkTjpx4gS2//rXv15iV111VYnRuErszlmxYkWJOcfMsGHDSoy+C5L0e7/3eyU2bdq0zveifYRcT1QaSOIyIATNd7f/0t5K93F7MO2h9F08fPgwtqc9gL5Lrk/JYUh7pcTfC9pb3Fyn8SMnm/te9Ke0Sn7JCSGEEEKT5JATQgghhCbJISeEEEIITZJDTgghhBCapE/hMYnenPCRRGMkBHv66aexPQkHKf20S0l9xx13lBiJnpxwctCgQZ2eSeJ09yRcdAIxSv89dOjQEiPhruTfoRcSCXcVLUvcf050SWn1Kf2+S9NN6ctJ9ObKGpCY0onTSNBL5Q6cyJoEmiRwpNTzkp9XvZx//vkl5kSjtC7o/V1KfpoXdK8vf/nL2H7MmDElRqLTuXPnYnua626urly5ssTovZxoktYPXduf8g0OGgOXKp/2VuoXJ4gmowftK66sA633ruva3csJj0loTnuDG0NaGxRzAmNXHqgXEr468wwJimms3TeMxp9KyJAhSJIuueSSEhs1alSJkRheYpGyM7+sWbOmxJYuXVpiNCclafjw4SVG3xb6VkpsFHLkl5wQQgghNEkOOSGEEEJokhxyQgghhNAkOeSEEEIIoUn6FB6TyNOJ3khMRQInyhYrSePHjy+x/fv3lxgJliRpzpw5JdYfMeHBgwdL7NJLL8VrSUy4bt26EnMibRLokZDLZcYlkXPX61wWWRLYkThv2bJl2J4Eap/4xCdKbOrUqdj+xRdfLLFf/vKXJbZz505s/9xzz5WYEx0+8cQTJUZ95USPlEmZhLdurjnxdC8kfHZidOp/en7XftOmTSVGAksnOj1y5EiJ0fs/+OCD2J7Wv5v/NIepr+iZ3LWULdatMxKuO2isKYutxGuQxN9O+Err7cCBAyVGxgmJ1xuZT2ivkrgP169fj9fS3kLr1WXIJkE0CfqdeN1l4u2F+oQEzhLPC8rETn9T4izC9F1xeyCZN6hPXBZoWi/3338/Xvv5z3++xBYsWIDXEnReoP3KmT+6fgOl/JITQgghhEbJISeEEEIITZJDTgghhBCaJIecEEIIITRJDjkhhBBCaJI+bR6kgHeOIVJGU/pp55ggJ8Y111xTYpQ6W5IGDhxYYpSq37m7SK3tUpJTWvmjR4+W2JQpU7A9OVzIRUHPL3VPN9+fVPl0f0qz7soKkOPi7rvvLrHBgwdje+rTK6+8ssSonyUuoeHmKvUfOTtcSnPqF3ISkmNP4hIiBLlw3Jyg9Ucp0Z27ivqVXCDOcUhuCUpp7/qU1spjjz2G186bN6/EqE8nTpyI7akEBDnm3Fpx5ToISj9P7kqJ1xbNYXLySdKwYcNKjPY155C96aabSuy73/1uiTl3IL2Xm6/kJCLnnrsXOczoe0VlDdz9CXKXuXIx5C6ja53DlZ6fyiJQ+QWJv40bN24sMec6pT3AlVWguUrP78aPnpX2UOeC60/JlfySE0IIIYQmySEnhBBCCE2SQ04IIYQQmiSHnBBCCCE0SZ/CYxJNuXTKJBoicReJESVp7ty5JUbCYyfmJIEUpYR2z/+9732vxJzIlUSKI0eOLLFdu3Zheycc7OXll1/GuBOP9kJCMBonidN/k5j1uuuuw/YkSCUx8Kc//Wlsv3379hKjsh5OTEoCQzdXKCU8pWqnlOzu75KY8JVXXsH2lL6foGdywn16fxL3uZT0JAgm4b1rT2uFSoC4Mbn88ss7xSROn09lPa699lpsT+uK2jvRpUs1T4wePbrEnCCe1iaJnGldS5zun1LtUwkNSdq7d2+J0R4+ffp0bE9zk/ZFiZ+V9mbqP4kFufSsbg/uWhaA5oqbFyTopj3UfVdI6E5rYMiQIdh+69atJUZ94spK0Honk4Uk3XHHHSVGYnD3raL+J5GxE/m7fYjILzkhhBBCaJIcckIIIYTQJDnkhBBCCKFJcsgJIYQQQpPkkBNCCCGEJunTXUWpk53jhyAVOCnAJWndunUltnDhwhKjNNsSK8OXLl1aYk888QS2Hz58eInt3r0bryV1OqWad+4a6ldSm5PjQ2LFPkFqdwelJCd3kUunTW6HHTt2lNj69euxPY0ruQ2cs4TcTU7ZP2vWrBK74oorSmzTpk3YnhwHlL7flW9w5QJ6IXeU6/8XX3yxxCZNmlRizi3xwQ9+sMSo1Aa5YiR+VloT+/btw/Z0rz/90z/Fa7dt21ZilD7eOTOoXyj9PbkoJe+OIsjd4lLd0x5Ge6hrv3z58hKjNeAcbnQtrQFyt0nSokWLSsx9Lx599NESIzct7SuSdODAgRIjd47rK1eaoRdyvJ04cQKvpfVO69L1P80r6r+xY8die/qG0Lo8cuQItqe9xe1VXeclfdclLvdBrlPnBOvPty2/5IQQQgihSXLICSGEEEKT5JATQgghhCbJISeEEEIITdKn8JhKMDjRK6Xgv/nmm0uM0oxL0pe//OUS+5u/+ZsS64/wmYRYF110EV5LwleXUnz27NklRqIpVxaAxIg///nPO/1NyZdm6IXKFzjBFo0rpS93gj0SrVE/TZkyBdt///vfLzFKM+/EwJQ+fuDAgXgt9QEJGV1f0Xtt2LChxNz4dxU9UqkNJxymsaI5dfXVV2P7V199tcSoT++//35sf/LkyRKjteZKLdCcdiUsKKU7iVYdVG6EBO1O9Nh1/CQWftL6l3i+kvBz2rRp2J72YLoXrSuJ+/W2224rMZpXkrRkyZISc334/ve/v8TuvPPOEqM1ILFIl+agE9m6tdkL9Z/bF2i+UuzGG2/E9g8++GCJ0XNSCRyJhddkvnGCelrvzmhARpXHHnusxCZMmIDtSbxN+wXdR+qf+D+/5IQQQgihSXLICSGEEEKT5JATQgghhCbJISeEEEIITdKn8JiyErossiTSXbx4cae/KbHAjTKArlmzBtuTcPItb3lLp+skznjshKvE6tWrS8wJDEk8OmDAgBIj4bDEYkaCBIJOCEiZOUl47oSAlMmW5oQTLdK4kEjViVHp7zpxIfUfZeacPHkytqd+ocysP/vZz7C9Ew/3QmPiRIMkhqT7uwykJDql+zvjAIkeKWM4mREkFq677Na0VqhP6ZkkzsxK+5LL7OsyKRP0d535gTKcHz16tMSc6JLmJc11GmuJBfW0B1F2aIkzvM+cOROvveGGG0qM5gAZAiTex6hf3LtShnKC5qDLuk/rjcbEjf/tt99eYrSvksBc4u+NuxdBe4j7XlPWcbqX+95MnDixxKiv3DfQCcqJ/JITQgghhCbJISeEEEIITZJDTgghhBCaJIecEEIIITRJDjkhhBBCaJI+3VWkjHZlHcgdQ04YckxJ0q5du0ps8ODBJUYuKIldX/SsK1euxPYHDx7sfC9Ki06p4l1ZBnLtULkKchtI3d059P7OHUf3J2W7U8tTWQF6fudYmTt3bolRWQ5X0oLizoVG7hByETgnHo019R/1ieTnRS9vfOMbS+zUqVN4LTmGyEFC6dQl6ayzziqxMWPGlNiiRYuwPTkzqE+OHz+O7clF4pwdVOqAHCfOgTFo0KAS27x5c4m59P3OYUjQGDiHYNcxdGUlaGypD2leSeyOond13wCKO4cqOe/Wr19fYm6vI9cTzWHnoupaGof+ppsX5Hw8duxYiblSBdT/NH7OXTd+/PgSoz3MuRbJieZKiJDDkd6VHIMSO9HIDev2C3JiOfJLTgghhBCaJIecEEIIITRJDjkhhBBCaJIcckIIIYTQJH0KjyktuktpffLkyRIjgd2oUaOwPaWQf/TRR0uM0udLXiTcy549ezq3p3eSWCBGYmISSEosmqK+du3/OzjBFom8SRzm0oRv3769xGisnBCRxp9Ea64sRFchosTiaRJ9OtEj9QuJiV17JzzthQSOJMaWeK5QXzvhP70/ifQ3bNiA7efNm1diJPB1ZVlIkO5KUJAgnsbaCTSpVAKVanAiezevCBL5ur9LkPDSlfYYO3ZsiZFw2O2B9957b4lRX8+aNQvbT506tcRI0C3xfKU9aOfOndi+q8iYxO+S/471Qs/phN9k6qDndOVGaG2NGzeuxJxxgfZLWgPOkEHCb/esJGimeenuReNC7+XmujNQEPklJ4QQQghNkkNOCCGEEJokh5wQQgghNEkOOSGEEEJokj6Fx5Sx1WWsJTEuCcmcYIiyZV5++eUl5kSbJBAj0ZsTzT377LMlNnLkSLyW4iR6owyOknTgwIESIyGXy27bVbhKQi6XrZUEaiRSdhksKVvm6tWrS8wJh0nkTVk1XbZW6n8nsqY+IIGcy1hMY0ViRCcS7pqtk8bEiWm7Zhd3c4f6n9a6E55v2bKlxGie0zqTWOBIwmWJM8bSezmBL/UhXetEk5Tx2kHz0v1dWq80B90YUoZnak97tSQtXLiwxGituHVBgmY3X0lQSnuLM1+4TLhdcVl/e6E92PUf9QvNa9orJRYZ0/wh44PE70T97PqUxsrt97SH0R7oMkt3FS474bhbQ0R+yQkhhBBCk+SQE0IIIYQmySEnhBBCCE2SQ04IIYQQmiSHnBBCCCE0SZ/uKlI2O8cQKbZJge3U1k6x3ws5NiR+VlJwU5pyiZ0cLn02uSvo2lWrVmF7ei5yrTgnlHuuXih9vUtJT8p6cjK5ZyJlP401lQ+Q2ElFpR6cM4BKKLg5RY4Hei73rOQioDTxzhngHAu9kOPHzd+ubgM3/tR/NCbO1ULtafycO47ix44dw2vJ9UUlLJyTj56VHCtu/lC/OGi/dK5Jco3RGtq/fz+2pz2M5uCYMWOwPTlfyQnk5jWVNehabkfifY3+psTvSuvaOdHcGPQydOjQEiMXm8SuKyoX4lyXNN9p/kyZMgXbuxIMvfzqV7/COLmbXGka+rZTGRrnJKXvDa1rt993dRhL+SUnhBBCCI2SQ04IIYQQmiSHnBBCCCE0SQ45IYQQQmiS15AINoQQQgjhfzr5JSeEEEIITZJDTgghhBCapM88Of/2b/9W/i3L+d4p/wblE3G5R7pWkaa8AxLnPaAcE/RMfcUJyh1A+Qzc36TnohwBLh8FVWz/2Mc+VjrrM5/5TBk/lyeFciRQ7hJXRZra07WUd0Li3BOU94Hy0UicJ8VBc83NS+LgwYMlRu/vqg1TdfC//uu/LuP3jW98o4yfy7FDOSoon4vLO0Lzl97T5emh3B+Uy8LlJ6H7u9w/lP+Gctq4fCDuHbpC97r99ttxY6QxdPlMKNfU4MGDS8ztC11zFbn3p37tmhNK4hxKF198MV5Lz9U1/5fE85j+pluDJNP46Ec/WsbwrrvuKhe6CuYjRowosRUrVpTYggULsD3tC/3J9UY5ZWi9ue8S9dXevXvxWsq1tG/fvhKj3E8S53Cj/Wrs2LHYfvv27SX2D//wD7gG80tOCCGEEJokh5wQQgghNEkOOSGEEEJokj6FCPRvj67uBv17KMVIuyBxLQv6N0ZXS4N0Lv2pvUV1gkj7IrFWgP6d3dXpojo5dK2r2+Fq8vRC/3ZLYyJxPS56f1e7iuI0Vk5TQjoZqgXj+uTkyZMl5nQ29DeoX5wmhMaKxsT1dVftAT2/08R1/Ztu/VA9JNK/OU0N7RU0f/qzfzj9Ft2LNF1urtK6oLEmnZzkdSYEvZeru0NjSGvApf2gtUXt+6NLJA3f008/je1pvpAmReI+pGd185q0HjQvSKvWV7zLfbZt24bXUu0vqrHloG8TzeutW7di+yFDhpTY7t27S4z6WWJNkLuW5jBpOJ0GcN68eSVGZ4B169Zhe6cLI/JLTgghhBCaJIecEEIIITRJDjkhhBBCaJIcckIIIYTQJDnkhBBCCKFJ+nRXkRPCqd2dk6QXUlBLrGIndwRlSpTYHUXOAKf2JmU6ZTaV2KFC2VWdE4UU69TeuTCc66cLLoPkrl27Oj2TcyyRs4Iy5rqMydSn/cm4S3PFzVVyElF7964XXXRRiVG2U8oAKvl51QWXQbSru+3YsWPYvqvjyM1JWmvkgCAHhsTOIOduomegeU2OQYmdfLTXub4md6SD1pBzuFHGWXpW546id6D7k4tG4vd9/PHHS8xl/KX15sab+oDWkHPYkXOW5oVz2Lkx6IX632WNnz59eolt2bKlxMgJKvEeTHsNubgk/q7Q8y9ZsgTbz58/v9P9JX4vcp26TPTkRt25c2eJjR49Gts7hxuRX3JCCCGE0CQ55IQQQgihSXLICSGEEEKT5JATQgghhCbpU3jcn1IHJBIk0RuJPiUui0BiYLpOYiEaiZxJICtJkyZNKrHx48fjtSRoJCGbE20dOHCgxOhZXV+70hi9kJjUiTEpzTkJyXbs2IHtSRBOQjKX5pxEjwsXLiwxJ0Tbu3dviW3fvh2vvfLKK0uMRJPLli3D9iSSpfnnUsc7MWYvJFx3gkkS+JFwe8yYMdie3qlrmQGJ1zqJC52Ym/YF2n8kntcDBgwoMSc8pz6kfWXw4MHY3pUm6XovJ56ncg1dyw9IXAKAhLuUUl/i9UJzyJWroXnthL+bNm0qMSr10J8SGP0ZF7cP9kJ72NSpU/FaEsPSGnClWehd6btKc11i4TYZDeg6id+V1rXE30YqS+FKoNB+Rd+A5cuXY3v3HSfyS04IIYQQmiSHnBBCCCE0SQ45IYQQQmiSHHJCCCGE0CR9Co8pO7ATnZGgcMOGDSXmxJwkPHzXu95VYmeffTa2f//7319iJOZ1oisS2DmRJAm0KAvojBkzsD2JP0lg6MRxTozXCwmXXWZqyuJKQkbKyur+LgkZ3fhRe8rgumfPHmxPwvGRI0fitRSn7MRXXXUVtl+7dm2JURZZJ1x1fdgLCVFdtlW6P80TN6coO/GwYcNKzGUaJYHtmjVrSoyywkr8rE7kTNfu3r27xEaNGoXtKZMziYGdScJlbe96L5exnNYgzQG3B1OcTAquX0nMS3uVm9c0X8gQILHQmzKBO5E+mTfI1OBEvmQUIKZNm1ZitP9LLLKlPYzEyJI0d+7cEqP17oTr69atK7GZM2eWmNt/KOPx9ddfj9fSfkMiczdXKeszfQMXLFiA7Wn8HfklJ4QQQghNkkNOCCGEEJokh5wQQgghNEkOOSGEEEJokhxyQgghhNAkfbqryF1EaaYldqdQzLl7xo0bV2KUft+VNKByA1RWgVJHS+wu+c53voPXPvbYYyVGTiZKXS1Jt912W4mRip7cFhI7uQhK6e36n0pokDPg2WefxfbkxCNlPLlFJHZBkNuD0tRL7OygOSVxWvT/+I//KLH+lBWgshaurIBzR/RCqcvd2Lu51ot7J1o/lNLdubOmTJlSYrT+yEEj8bxyzhiaw+SWcWn+ycVCc53mtCSdOHEC4wTtoeQskXhtUKp8V+rhhRdeKLElS5aUmHMWkROKHIYupT6tAXctuSzJceWcTOSmI9fahAkTsL3bB3shN6FzbdLz05g4h2lXx5eD5iXdyznWyI3oShPRfktlaMiF5SA3rysLQXPNkV9yQgghhNAkOeSEEEIIoUlyyAkhhBBCk+SQE0IIIYQm6VN4TAI3ElJJLHAigee8efOw/dixY0vspptuKrEf/OAH2L6rGNSVlbj77rtLjFKaSyzyJNEVpbmWWPhFfeVEexs3bsR4L5Tm3IkpSfRGqbNdqQtK301ibhLnSdLDDz9cYrNnzy6xD3/4w9j++PHjJeZE5vQO9P4333wztieRKonEnUjYCUd7ef7550usP8JtKn+wefNmbE9zigS6jk2bNnW6vxMM0vp3wmMyP6xYsaLzvWgN0Fyj9SOxINtBc43WusSicDIkuH6hcg203lxZh1mzZpXYpZdeWmIkUpf4G+BK0FC/0Nx+/PHHsT2tNxIeu7ICXdcgGR1cWQaab9Se9g/HU089VWJUQkXiMiQk3H3iiSewPZWwcNB6Hz16dImRcF5ikTGZiqg0jMRlfBz5JSeEEEIITZJDTgghhBCaJIecEEIIITRJDjkhhBBCaJI+hceUndSJOSm7KGVQvOaaa7A9CaRIHHb77bdjexKYUbbFL3/5y9ieshu7e82ZM6fEfvzjH5cYZXyWOIsnCaddBkzKrkyQwNAJAen+lO3SiSZJTEfzZ+3atdh+5syZndq7TKX0XC6zLGXrXLRoUYmNGDEC29Nz0fxz2YVJIEmQwN1lnCbhNd2fxOQSC0Hp/k60SuuX1r8bk5UrV5aYy65Mz/rNb36zxJzo8c477ywx2tecONUJTwkSSZOYWGKRKs21Xbt2YXsyhdBe5e5PezNlTScxrCR95CMfKTG3V1F8y5YtJTZ9+nRsT8JVmtu/+c1vsL3L5NsL9fXChQs7tZV4vTpDCX0XSGR+ySWXYHt6rrvuuqvEXMZuGn+XtZ2+I4888kin6yTuVzKq0B4i8Vxx5JecEEIIITRJDjkhhBBCaJIcckIIIYTQJDnkhBBCCKFJcsgJIYQQQpP0u6wDuQUkLmFA7pRbbrkF21NZBErV7pTp69evLzFKq+/S2n/wgx8sMeduWr58eYmRi2Hw4MHYnlLok4p869at2J7KQhCUJt2VGhg+fHiJkVvBlaqgtPzkDHFuC3KiUXunqid3jnMiUQp/cic5F8rll19eYpRmfNu2bdj+/PPPx3gv5GxxJQUoTuuXnGUSv9PevXtLjFK3S+x6I7fFwYMHsf2qVatKzPXTnj17Sozef/Lkydie5iA5MV36/a4lASR2TLk1SG44cu2RE1Ji1w2VpnClVagPtm/fXmJTp07F9ldccUWJOSch/d0JEyaUmNvvaW5Sv7iyDu471gu5sFxb5+brxTkMr7rqqhKjd6JSG+65vv3tb5eYcy3SN9jNFfo20R5Ke5jE70VjOmTIEGzf9Rso5ZecEEIIITRKDjkhhBBCaJIcckIIIYTQJDnkhBBCCKFJ+hQekxDu3HPPxWtJ4NYf4TKlJD99+nSJOeHllVdeWWIkpKNU9e65xo0bh9feeuutJUbP7wReJDJ+/vnnS8yl1P7Zz36G8V4oBb9Lk02iLxJNzp8/H9v/8Ic/LDFKle9Soq9Zs6bEKCX4fffdh+3vvvvuEnMCSRI5U1+5fp44cWKJkXDbtd+/fz/GeyFxnRPDkpiV2ruyKiQoJ9Ggm5P0XJRS/wtf+AK2p/T18+bNw2tvuummEqNnpfIPkvTWt761xEig6kwKJOh30LyifU3i9UKC7jPOOAPbjxkzpsS6Cmwl3i//5V/+pcTe9ra3YXsaA1cGiN6BhP6ujAvNVxI5074s+fI0Xf6mE55TnMTzM2bMwPbU/7QHu/Gn9UblclxZDhKDk6FE4nIbq1evLjEq9yTx3khj4oTPbl4R+SUnhBBCCE2SKb69NwAAIABJREFUQ04IIYQQmiSHnBBCCCE0SQ45IYQQQmiSPoXHJCYkgazUXUjmMtaSGPPIkSMl9uEPf7jz/Xft2lViToxIAisn0ly6dGmJUXZgJ26j96Lnd6LBrhlzT5w4UWL0nBI/K4m8nfCVBHIkxHMZkyljMAnfKdutxKJHJ5K+/fbbS+yee+4pMSdaJDEcZQB1fdVVOE7CYTd+I0eOLDES2VNWWYmzlZKQ0WV1JTE3rb/rrrsO2//4xz8uMcp4LnFmX4rR80uciZzWmlt/lMndQXPImR+OHj1aYpT1mzJ2S5yh/Xvf+16J0f4jsUiVMtSTmFri/qZ5JXHmbJpDtIdJ3fd711dds1aT8NjtYZRhnTImu32BTBl0rcuYTfvFxRdfXGLu+0HZyF3/0XPR997di8aPstZTFmjJ9wGRX3JCCCGE0CQ55IQQQgihSXLICSGEEEKT5JATQgghhCbJISeEEEIITdKnu4rSlzvHEDlGyIXi1PIbNmwosSlTppSYS+e8bdu2EnvooYdKzDkmSAVPzgRJ2rRpU4lRmmtyvEjsgiC1en/SxxNUasOVFDh16lSJkZPm5z//ObafNWtWiVGa8AcffBDbnzx5ssRoTjhnA5UFcC6WVatWlRiVKyF3l8TuOprrNM6Sdyj1Qm4N5w4kZw45SGjuSrx+yRnh+pRcNDSn3PNTCQ431uSEpLVGZUEkLqtA708uLMm7iwj6u25enX322SVGz0pOPomdWEOGDCkx5+6h+UouFpfqn1xnLi0/OXkef/zxEtu5cye2p7VB8905JF1phF5o/Nw35Oqrry4xmis0zhK702gNkAtNYocp7cFUwkfi+U6OO4n3a5rXdH+J1+v48eNLzJ0X6HvtyC85IYQQQmiSHHJCCCGE0CQ55IQQQgihSXLICSGEEEKT9Ck8JiFaf1Jak2iO0q9L0qJFi0qMxKRO+EoCJXqmuXPnYnsS6JFwUpLWrl1bYm95y1tKzAlMd+/eXWL0XpQmW5JefPFFjPfyhje8ocQmTpyI15IgmcRlJG6TWAh2wQUXdIpJ3Nc012bMmIHtSXjr5uqoUaM63Z/mn8Sp3ik2fPhwbO8Eyb2QkNQJJmn+UAkIV5Jg48aNJfbVr361xH7nd34H29O4kECTUs+7v0trUmKBJgmancCWnoHWlCt/QOvKQWPoxNsk0iSRqSvt8d73vrfEhg0bVmJOzEmlWT72sY+VmBNek5jYGQ1oDDZv3lxi3/jGN7A97bck6HXrhb5NBJk/xo0bh9cuWbKkxN71rneVmBOe0xw8fPhwiZHJRmKh/XPPPVdiZJKRuE+pvSQ98cQTJfbkk0+WGBk6JGn9+vUlRuvCGZ2eeeYZjBP5JSeEEEIITZJDTgghhBCaJIecEEIIITRJDjkhhBBCaJIcckIIIYTQJH26q8hF4BwLhw4dKrHnn3++xM4880xsT4p1cvJQqn+JlenkbCDHlcROmjlz5uC15FA677zzOt/Lqet7cS4GpzjvhRxDlH5fYscGOYbIQSGxip7Sv7/tbW/D9lRCgFL9jxkzBtuTk8e5mMhJRuPnUtKTa4scP1RWQ+qekpzcInQfidPPn3POOSXm3CbvfOc7S4zSvLtSB7TW3v72t5fYHXfcge27pqSX2CFG+4/bq2iu0fM7d6OLE/R3ybXo6I+T68///M9LjNa729eIP/mTP+l8Le1XriwD7U00X11ZgV/96lclRuPt5gDtbQS1d67N2bNnl9g3v/nNEnN9Su//la98pcRWrlyJ7W+88cYSo28FzUlJ+va3v11izslHa4j2S+fGpbIO5Lx1+13KOoQQQgjh/3hyyAkhhBBCk+SQE0IIIYQmySEnhBBCCE3Sp/CYxLiUPl3iFPaUFv3RRx/F9nfeeWeJUfr9LVu2YHsSNF911VUlRqm3JU6f7oS/lJKcRJJONHX55ZeXGKXvdgJHEtkSlLrcpfV/6aWXSoxSej/yyCPY/j3veU+Jvf71ry8xV5bh4osvLrEdO3aU2I9+9CNsT3PNidypXAGN9c0334ztSeQ6ZcqUEnOi0T179mC8FycSJqiECIkzv/Od72D7P/zDPyyxm266qcRonki8L1BJAVcqg8ScriwK9T/9XRKYS7yv0Pp1BoGuolX3N1xpize96U2d2ruyLpQqn+7lhLN79+4tsQceeKDEaK26uNtDqQwJzSFXQoGMDvQNcmPlvmO9vPDCCyXmxo/elUqT/PM//zO2J6E+7WGutAp9L6gExJo1a7A9iYSvv/56vJa+LRMmTCgxMnRIXMKDzBO0JiQ/r4j8khNCCCGEJskhJ4QQQghNkkNOCCGEEJokh5wQQgghNEmf6ivK2Pvqq6/itSRyJOGiy5hKWUBJ3OTEnCQwI4Ghy2K7devWEnMiZ8q6S2JgymApSa99bT1bUmZNen+pe7bH/oizSCRKws/Jkydj+1tvvbXEqK9dBs2jR4+W2De+8Y0SI3GbJH3mM58psb/7u7/DaykTN2VXvvDCC7E9iUFJYEjjLHG2T4LE6A4SKdOYOjH88uXLS4zmrxt/EnLS33R9SllYXf/Re5Eg2s21Xbt2lRhlx3YCXSeGJGi/dHsQCfVpDFzGWhL1UxZZNwYkPCWjghOfUzZ6l3F4/PjxJUb7tTNZ0DymNeD6ioTLBJlE1q1b16mtxCLlFStW4LUPP/xwiVFmYWfeIFMIzTUn6Kf9hsTcEn+DqK9pTCXeA2m9OvNI16oBUn7JCSGEEEKj5JATQgghhCbJISeEEEIITZJDTgghhBCaJIecEEIIITRJn+4qUrA7dxO5q0gZ71KSk5OBygo4d8egQYNKjFJyO3cJpYXvT0rpoUOHltill16K7UkxfuLEiRIjF4nE6a8JUtE7twNBzgCX0p3KYlx22WUl5lTxNH6f//znS2zVqlXYnlLS33HHHXgt9R/NP1eCoGuqfOcMcY6PXsjZ4PqPnpXu79K0UwkESv/eH3fl4sWLS4zWpCSNHj26xCZOnIjXUlkM2lfomSReF13T/EveXUSQO4ZKGkjsrqJ3oLUiSQsXLiwxmqtr167F9uQaJccSPafEDkNXloHKDZC7hvrPQWPovlfkpiNoDbnSOF2dTIsWLcL2NC9ovezevRvb035L88eVWqDvnXNi0XPRt2HZsmXY/uyzzy4x2sNdWY7p06djnMgvOSGEEEJokhxyQgghhNAkOeSEEEIIoUlyyAkhhBBCk/SptiPRkUt1ToJWKpXgxJwkMiYhnCtVcPDgwRIjMaUriXDllVeWmBP4njp1qsRIYEelCiQWbZFozomknRirF3pO6mdJOuuss0qMxJxOyEfC46eeeqrE3PjPnDkT470cOXIE4ySwdCL1pUuXlhiJ7txcGTBgQImRIHfgwIHYvqvokfrUidFJtEljumnTJmx//PjxTu1dqZOuInf37s8880yJOYEv7Qv9KUFDZT2o3Iy7f1fhuMRiUupXiUW+tAdNmjQJ25Pwk/rbCZcJ6gMn+qR+dfs1iZdJVO+MEl3bO6G7E992+Zu0LiUeV1pXbv++5ZZbSoz229WrV2N72m9pD3J9SvPi0KFDeC2td/reuO8FlXFy65Wg770jv+SEEEIIoUlyyAkhhBBCk+SQE0IIIYQmySEnhBBCCE3Sp/CYRENOeEwZDCmLqWt/wQUXlNg555xTYiRuklgINW3atBKjzMQSCzedOI1EbyRyJjGkxMJVEui5zKIXXnghxnuh/qNxkjizJwkhnZCQsvOS6PHcc8/F9iRapD51QkISQzrR24QJE0qMhGzuWUl4SAI/JxzvmrGahM8k2JN4TpDA8dixY9ie+pqEgC5bL2Vhfe1r6/9DUVZYSRo1alSnZ5K4/2n+OZH9vHnzSoze1QmPnXiecBljCRobyo7uhM80ryjjrXt+MhpQv/Qn670bQ3ovym7shOq03uj5XdbxrtD4kclB4vEbOXJkiTkxL2X8pX5y2bnHjh1bYocPHy4x+v5IbBRy5g36XpMg2mXIHz9+fInR98aZd9zaJvJLTgghhBCaJIecEEIIITRJDjkhhBBCaJIcckIIIYTQJDnkhBBCCKFJ+nRXkbKZFPASuybIHeIcR6SiJ3cKpdmWWMVPynSXDpocB07ZT89FjgNSi0vs2iAVv1OWd3V3kLvF/c0hQ4aUGDmenDuHFP/kgCAHgcTuGHom584jF4hLv04uDHJcOXcdrQG6ds+ePdi+qzuO5gk55iTp9OnTJUZ9QmtCYrcDuXVc6nV6Lhor526kOJXakHhdkjvLrV9Ka0+uQddXrlwHQe4ecqZIXBaAYm4N0nuRQ9PtS/RctK85Zwv1tythQaU5yN3jnES0hmm+9mesCHom9w2k792KFStKjBxXkrRt27YSmzJlSonRu0vS5s2bS4zGz5X1INeYK0FBpXFoX3MO1V27dpUYrXf6BklcRsmRX3JCCCGE0CQ55IQQQgihSXLICSGEEEKT5JATQgghhCZ5jRMShhBCCCH8Tya/5IQQQgihSXLICSGEEEKT9Jkn5+mnny7/luWqEFM+BMrRQJVqJc79QnleXJ4Uyl1Dz+QqQ9O93LPSP/FRjgZXgZVyClE+A5ejgPJhzJ49u3TgY489Vh7U5dih96d8HC7vBD0rVUt2VahpXKhiusvxQfOC+kniZ6WcKP2Zq1Qd3VWxpnwk1157bfmjW7duLePn5q9bl7243DGUI4P6xOWp6po7yvUJjavrf5oXXd9f4rlCeX7cP+XTe02aNAkX1oc+9KHyR1zuF+oDWkO0r0r8vDRXXb/S36V9zeWPOnLkSIkNHjwYr6X3olxFruI63Yuudc9K7/r3f//3pbO+9KUvlU51lc0p1xmtC1fFnN6J9gqX643+Lr2n2xdp/ricSLSOqT2NqcTrjfLsuP2Cqqt/6lOfwjWYX3JCCCGE0CQ55IQQQgihSXLICSGEEEKT5JATQgghhCbpU3hMQjgSqErdxZiu4BYJ1EggR6I/iQt/0rO6AqFUtMyJpqhfSHRGoj2JBVrUVw5X9K4L7plIZEpCMFcckJ6JBG5OTE3F2Uhgun379s73dwK9AQMGlBg9qxM9kviWBIZUsE/yYrpeSEzrhOP0N0ng6gp80jvR+nF9Qv13/PjxEnPFKanAqIOK/pHA0gmHaQ8gQbwrZHjixIkSmzRpEl5L/UXzT+K1Qful2wNpvEg46oTLtN/SHHIibzJauLlO64X6xe0XNLZdC69KPIYEzRUnpqZ37c83kEwBtC7oWyOx8JgKYbo1TPstxaTuIvmTJ09ie3oHela3VtzaJPJLTgghhBCaJIecEEIIITRJDjkhhBBCaJIcckIIIYTQJDnkhBBCCKFJ+nRXkTLaqdVJ8U9qfacM7+ouctdRqnZKv03poCV24jgniIv34twd5GRxJSAIcq0QL774Yok5Zwa5GMhZ4co6kNqe7uVSmtP40VwbOnQott+xY0eJbd68Ga9dsGBBp+dy6ffJRUFuEep/yZdW6IWcVK4tuePI7eDKYnQtq+HWP81JcueRM0rieeXmKrlojh49WmLOSdh1r3JOQvcOBI2hW79d15BzN5Frito7dw2tl8WLF5fYlVdeie3JIUnzUmI3GvW3W4P0XnStcyPSsxLk2nTPRONCMbfX07eN7u/KSpATi/ZL57AkJ57rP5qrtDe5fqaxJseU+4a6vZXILzkhhBBCaJIcckIIIYTQJDnkhBBCCKFJcsgJIYQQQpP0KTwmgZUTXZEQiURTTkhEf5cEek44SaItEj25VPskfP3BD36A186ZM6fESODlyi+QcJHuTwJlyac673IdiTYlFoKRcNOlaSfR4/Dhw0tsy5Yt2H7Tpk0lRim93/72t2P72bNnl5jr/yNHjpRYf9Lnk5iQhMuur1yq8i7t+yPEJCGgE63SvNi9e3eJuTTvJMYngeu2bduwPb2rEy2ef/75JUYlDZxIm96B1gqtScnvYV3v5cS4JCjtzxhSH9Jcc3PICVJ7WblyJcbpXrNmzcJrSTg6efLkEnN7IMUPHTpUYm4Nk9CeoO/NqVOn8Frqf/reuH6eNm1aia1evfr/6RH/b8goROJ/N/7Dhg0rMfesJPSnbwiVapCksWPHlhj1tbt/f0ob5ZecEEIIITRJDjkhhBBCaJIcckIIIYTQJDnkhBBCCKFJ+hQek5jUCQ9JIOcETgQJB0kI5YTH1J4yS7osqJTtkzIwStLTTz9dYgcOHCixO++8E9uToJEEuS47MI0LQcJNJ+Qj0WBXcZmDxODLli3Da+lZx40bV2J79+7F9v0RqW/durXESKA4ceJEbD9jxowSo2yhTiDadV1QdmHXloSczz77bIk5ISZlSyUxscvW6uZqL25M6F3dtbTXbNy4scScyJmyro8aNarEhgwZgu3duBL0DpRdWWIxLK0hJ0Y977zzSozW1f79+7E97aGUHZzWpcRr060hWq80h0k4K7FwlfqPTAbuWoKEr24PnDJlSonRN8xlDSfhMvXfT3/6U2xPeytlrCbjjCTdcccdJeYqDEyYMKHESJBPc0riygO0LpzI3/1dvLbzlSGEEEII/4PIISeEEEIITZJDTgghhBCaJIecEEIIITRJDjkhhBBCaJI+bR7kmCAXibuWnBhU0kDitOrkBCLHiMTuIGr/1FNPYXvn2iFI2U+uM0qLL7HjgdLqk9tAYtcLQWninTuHVPTkAnPOkkWLFpUY9cmGDRuwPb3/F77whRJzKeXp/ldddRVeO3Xq1BLbsWNHibnyGaTsHzFiRIk5F8S+ffsw3gul/3d/k+Y6OalcSYGDBw+WGK1f1yc01wYOHFhi5DaR2PXo1iS5/siF0x8nHpUUcGU5+uPsIBePc7hRGQyalw4aL5oXzlnk9vZeqP8k6eabby4xKsEh8XjT3nLixAlsTw5TcsNRqQPJO/d66bovSjx+9E7umZYuXdrpbzrHGK23D3zgAyXmvkv33HNPibn1SvsFzat58+Zh+5kzZ5bYM888g9cSruQLkV9yQgghhNAkOeSEEEIIoUlyyAkhhBBCk+SQE0IIIYQm6V534f/ClXUg4e9rXvOaEqOU7JIXRPbiShqQ6IqEh1dffTW2p7TuTvRG9yLRn3tWEo9S/zmRdVfo/k70SAI5EpOPHz8e25Pwk9q/853vxPbUpySwHj16NLafPn16iQ0fPhyvJdEaCU+dELOr8NuJtN0a6IVEj279kaCcBI6upMCpU6dKjMShdJ0kjRkzpsRIoLlkyRJs/8ADD5TYN7/5Tbz2lltuKbG5c+eW2KFDh7D9WWedVWIkqCbRp9S9rIrE/e3GkMTb1If0rhKPARk9rrnmGmxPeyAZBdatW4ftaQxdX9E6JkG3W2skMqbxdiLhruJxWsNUPkPi9da1fIHE85L2AFeWg8bv8ccfLzEqSyRJCxcuLDEqVyRJDz30UIndcMMNJebGb+3atSVGQn96J8kbMIj8khNCCCGEJskhJ4QQQghNkkNOCCGEEJokh5wQQgghNEmfwuPTp0+XmBOSnXPOOSVGYmLKlChxxlsSXTnBEcVJiHfs2DFsTwKnWbNm4bUuE3EvLmMqiX9J+Ojak0iXoD51YmbKVkliXhJHSizSpPFzGTRJtEdZjGlOSizmcxlvf/CDH5TYFVdcUWKULVbiNTBu3LgSc9mBnSC5Fxo/l7GaMrhSZuCRI0die5r/dK/+iC5prm3evBnbf/7zny8xJ9AlMS2JTt34X3rppSVGfe3EqW4NELTWXXsSnt5xxx0l5oTDs2fPLjF6Lzcvt23bVmJk1HjkkUewPb0XZbaVWBBMQvcJEyZge/re0Bx0mY27ZjwmQ4kzpNC8XLZsWYmtWrUK28+fP7/EyKRAol2J+4QMIR//+MexPc0Vlx150KBBndo/+eST2J6+DWT+cN/aCI9DCCGE8H88OeSEEEIIoUlyyAkhhBBCk+SQE0IIIYQmySEnhBBCCE3Sp7uKFOjnn38+XktOCHJXfP/738f2pKKn+zu19fHjx0uMVOykgJek66+/vtPflNhFQO4e5wQixwWl/3ZlCbq6O0jB7txx5OwgZT+leZfYGUF97ZxhO3fuLLH3ve99JUYOBImdMF/72tfwWnLN7dmzp8Sef/55bE+uQSph4nAOqS73cSnhyUk1dOjQEhs7diy2p7EmFw45KCR2TZILxJVauO6660rsk5/8JF5L+wK5tqZOnYrtqaQAOTHds3YdP0l69dVXS8yV9Xjzm99cYuSGGzx4MLYnhyM5zGitSdKWLVtKbN68eSX21re+FduT827fvn14LfUL7bcXXXQRtqdryV1E+6q7P9HVCSnxHrB8+fIS27hxI7aneUHfihkzZmD7z372syVGezA9kyStWLGixD70oQ/htbQ33nXXXSXmHJJUWofKANG6lLqPn5RfckIIIYTQKDnkhBBCCKFJcsgJIYQQQpPkkBNCCCGEJulTQUdCKlcWgERTzzzzTIlR+QCJU+2T6MiJQb/0pS+VGInGLrvsMmxPAilKcy6xIJTEjEOGDMH2JJDr+kz9gYRcJBCWWGRKwlE3fl3T4u/atQvbf/SjHy0xen6X5v3UqVMlNmfOHLyWxPMkyHXveuTIkRIj4e/EiROxvRMP90LCdSdwJjEuzVMnpCTRKt3LlUU5cOBAiZE4lATukvRHf/RHJeZKSNA70Fi79UfQuzqTgxPvE/0Rb9N7UbkQB62thx56qMRo/kq835Jw2M1BEkST+UHisaV+dSJt+rvU16+88gq2d9+RXqi0CwnEJTaEUBmb/gjXb7755hJzwm+a7/RMS5cuxfY015zR6P777y8xEslT+QeJzxG0B7nyDV1LK0n5JSeEEEIIjZJDTgghhBCaJIecEEIIITRJDjkhhBBCaJJ+C4+dcJCElyQEc+1JiEbCVZeFeNGiRSU2YMCAEhs1ahS2JyGUE7lStkYSEzrRFglC6f4kppU4CyZBwlX3TCR6JOEzicskHj96Jye6JjEfiR6dEJGEaCT6k1hQTJmc3VyldxgxYkSJOZGvE0P2QvPXCf9JzEhiZCcEpfenzNCuT2it0pqYPXs2tifRqhMek6HhmmuuKTHqP4nFjNQvTiDqMpkTNC+doJ2ygVMWYpdFmN6LzBP9yRpPmaQvueQSbN91X3TPQIL8l156CdtTJl96f7ffuH2kS/uTJ0/itbSv03eJTCoSm0Iou7HLAk3rfevWrSXmsghTn/77v/87Xktj/eEPf7jEXD/THKbvGmVSl7rvoVJ+yQkhhBBCo+SQE0IIIYQmySEnhBBCCE2SQ04IIYQQmiSHnBBCCCE0SZ/uKlLbO2U3ORmee+65EnOOE3LC7Ny5s8ScO+TP/uzPSowcO679jh07Smz+/Pl4Lf1dckK9+uqr2J5cI6SM/8UvfoHt6dqu9zl8+DBee/HFF3dqv3fvXmw/efLkTn+TyidI7EygNOWu1MW//uu/lpgb66uvvrrEyHGwZMkSbE8uBHKdUakDyY9rL+S2cCnl6VpKk98fdxU5KJwzhP4uleVwJQHWrFlTYv1xt5HjxbmjyLFDzh7nDKFU/w5aQy5VPc0Xega3BsgN9973vrfE3BzYvXt3iZG7y7nLqOQHrRVJWrZsWYmRu8y5CakEALlznBPqwgsvxHgvVALD7b+03mi9OtfuBRdcUGI01q40zqZNm0qM3HHUzxKvC7deaVypvetnOkds3769xNw3tD/kl5wQQgghNEkOOSGEEEJokhxyQgghhNAkOeSEEEIIoUn6FB6TuMgJwahcAAkEXVkBEm2RcNGlhSeRKt3Lia4ozfhXvvIVvJb6hd7VlRXoWgLB9RWVyyBICOjSrJNojIR848ePx/YkECMxrOt/EmOScHTx4sXYnsR8TrS2dOnSEqOyAu5dSXxLwntXlqNrSnKaJ/0Rw1KfkmBT4vTxlHrdiVaHDh1aYitWrCgxV5KE1t/UqVPx2q4lIEg0KrH5gcpSkBhe8oJmgt7LrV/aAx9++OESu+6667A9zbdBgwaV2G233Ybtyeixf//+EiNDgSSdOHGixO67777O186dO7fT/R0kZnXi9Z/85Ced/iYJ0t28oP2eDA1OuE7xxx57rMQ++clPYnva26msg9uDv/Wtb5VYf8onkHnh0ksvxWtpbdKccOPnBNFEfskJIYQQQpPkkBNCCCGEJskhJ4QQQghNkkNOCCGEEJokh5wQQgghNEmfNoHf/OY3JebKOlD6aFJQk+NGYtcHpSlfsGABticVNrkgnn/+eWy/fv36EqM00xKr4MndQSnRJWnKlCkl1h8VPjlBCHKBOVU6uQjoWnJrSFwChNwGbvzJCUT3d+4wGus9e/bgtTRXyAXjlP001uQMcE6orqnKycnkSkKQu4n6lFx0Ejt2KH28cwbRuqJ57hxr9KzktpF4XFy5GIL6n8b//w2oNIZzqNIcomvdvvLII4+UGJUbcXsg7dc0Ls61Ru4u+i5IvDcdPXq0xJy7iuYGzW1aFxJ/2wjag8hJKfHaHDduXIm58aPSQj/+8Y9LzO1rl1xySYktXLiwxNxaIYeyc1cdPHiwxMg56coA0XtRGSXnZOvqjpPyS04IIYQQGiWHnBBCCCE0SQ45IYQQQmiSHHJCCCGE0CT9LutAQjSpu8CNSg046G8OHz4cryWB1OnTp0uMxG0SC39JCCVJX/va1zrd36XAJ0Hqc889V2IkBJO8ILkXEuO6UgMksDv77LNLzAk06Voqy+EEYydPnux0r2nTpmF7EmiOGDECr6VyASScplIdki+N0IsTiHdNSU4CQfdMJFql9efuTfOCxtTNnwsuuKDEaP26uTtx4sQSc8JvEsnT+nHiUho/KlXg3tWta4L+hkurT2uAhPrbtm3D9iR8pT3kE5/4BLbvWq7k3nvv7dyehK8Si5d3795dYk44THvDhRdeWGL0DZC6l+agsXLmCZqvJIY+44wzsD2tNxLvf+hDH8L2tN/QHuC+a7SHuP2crO1tAAAfyklEQVR+zZo1JUbCZ2d0oP6nsiLuG+jKwxD5JSeEEEIITZJDTgghhBCaJIecEEIIITRJDjkhhBBCaJI+1Vck8CMxqbuWBFYkpJJYDEriou9///vYnkRvJFpyor/p06eXGImrJBYukuhv9uzZndsTTiTaVfhI4i6XwZIEodT/JEZ3f3fx4sUl5sSglMmYsuPS35R4/t144414LfXr61//+hJz70pZOGlMnHDVjUEvJPokga3EY33++eeXmMtWS5llSSDvnp36irL9uvVHImsSokqcdZlEn07kTgJPEpM60aXLeEvQvHYZj6lvKes7CUQlac6cOSVG2bmd+PuKK64oMRKpujH872aNHjx4cIlR1m1JGjt2bInRGnAZ0p2ovRfaV1zWb5rDBw4cKLH7778f21MWYRLEk0hf4m8Qjb8TXW/cuLHEXNZ2WgPUV06kTc9Az+rWihtXIr/khBBCCKFJcsgJIYQQQpPkkBNCCCGEJskhJ4QQQghNkkNOCCGEEJqkT3cVlXBwqmZSlpPjwqXpJifI3r17S+zQoUPY/vrrr+/0TM7dQ2pv564hxTs5A0jt7q6l53JlAdxz9UIpzd/85jfjtfSs5Bii1PMSO7FoTCn1usROOFL2z5w5E9sTTtlPrp/+uFC2bt1aYjR+rq+7uqsoJTs5gyR+1ldffbVz+82bN5cYOSmd44/Gj+afczyRa4yeSZImTJhQYuTscOuHxormuntXl5afoD3Ijf+gQYM6/U2av+5etF+OGzcO29O+RK4757Clb8NDDz2E19J+MW/evBJz79p1v3bznfqKoDXsShWQ44gcls5hTO40KkN02WWXYXtaA7QvjBo1CtvTfkn7osT9R+6sffv2YXsaK9qv3B7aH/JLTgghhBCaJIecEEIIITRJDjkhhBBCaJIcckIIIYTQJH0Kj0m450oSkBiP0t87SCBFIlcneqNU55QS2onu6FonMKS02iSwcvcigR69vxNOOkFtL5R+3AnHjx07VmIkhFuxYgW2X7p0aYl98pOfLDEnJFy7dm2JTZo0qcQmT56M7Un0tmzZMryWxIj9SSlOc/Ciiy4qMRLeSt1TypPo7vjx43gtPT8Jt0kgLPE70fp1649S/VP/uVIXJFJ3InMSWJIY1D0riV4p5p7VjStBc80ZEkhoTmLk/uyrJPB0olsqo0HCVyr1IPHadn1F+yXtTe5ZSaRK+zU9v+RLU/RCe63bF0ioTmJuKr8h8Rp24nuC1jb16Y4dO7D9hg0bSmz48OF4bde+dvtN1/GjNSF1N99I+SUnhBBCCI2SQ04IIYQQmiSHnBBCCCE0SQ45IYQQQmiSPoXHJJCkTJMSCwcJEhxJnF2ZMkiS6FdikeDhw4dLzAmZxo8fX2LuXf93e+cSs1dZteHlQGOMDQiCPRfa0gMtLQKiKG0FGw2JIYZoHBgdGScGE2LCyKkkDhnoxMTTxDgwiqYeaA2IRI2F0pa2tHylhba01BZUIBATTf75u677y37DyOe/ruHKfva793PaT97c91qrVq1qMRIDJnEUZVclgVbKLvye97wH47OkbJUEib6o/dq1ayff89ChQy326KOPTm6f5gpB/Udi5Kqqa665psVorNetW4ftp4psU8bclEl5FhLTJ9EyxUn0mUSTJBol4S4JdKv4neiZUgbaqRlQqzgLL/0WrbMqFmQnkT8xdfyqeF9L+wqJRGlfpflbxWNI2Z3p/at4D6D9NmVsJkE1ZYdPz0Ai3zQutLYom3Zag1PHm/b1lJmaRNL79u2b/Ns0rlP7qapq//79LUaC/OPHj2N7mmtJ+LxmzZpJ7VOFAhJk07pKwuUk3if8J0dERESGxEOOiIiIDImHHBERERkSDzkiIiIyJB5yREREZEgWdVcRyfEzNVV9KpVAinFS5qeyEuSk2LJlS4udP38e25NaOzmxyHVCjon0ruSOIhV8Yqq7ikoAUKmAKn5WcnukVPl0X0q/n1L1k4uBXBzkFqni8Uu/RSp+cjakuULQ/E8lNNIcnoWcISnNPc0JctuQA6VquguIHItV2cUyCzkmq3j9p36isSInWSohQnOISqAkx2hybRFTywek+84zBmfOnGkxcuekNUzON1qDr732GrZP+x1B84DKyKQxpPk6T1mJqe4cmoNpDdH70/gndx65BmlM0rogxxPNqeTu2rhxY4vRnKpihzHtQeldk0tzFio1UmVZBxEREREPOSIiIjImHnJERERkSDzkiIiIyJC8a57U+SIiIiL/K/hPjoiIiAzJohby73znO+1vnmQTJAs0WTVTcTOyppNVNBUnI0sZ2UJTcTmyJKbicmQtJwvxPIXwKJZsudSvX//619vF3/ve99r4Jevk1IJ3yVZ9yy23tBgVd6NCqFVVp0+fbjGyJJLNtKpqyZIlLUZjWsW2TLKapgKnNK9o/m/btg3bUzHZb33rW20Avv3tb7fxS7Z0en6y5dOzV3H/0finAps0J+lf4pRqgp4/WWXJVk1rbZ7xIygFQhWPAa2/qqof/OAHrRPSe5G1l54hPT+NN/VBSgNCY0sW4DQH6fnTGqR70NpOdmey99McSBZmstHfd999bQyfeeaZNn7pnehZaQ2kNUTvT9+r9F2gNUzPlPZQmpfpWYl5imbSfkHzN6XMoH7ZuXMndoz/5IiIiMiQeMgRERGRIfGQIyIiIkPiIUdERESGZFHhMdWXSJbzD33oQ5N+MNX9oPtSLNUuIkEyia5SexLzLSws4LUkWiPhZOoT6gMS2c4jEiVIdJfqzlAtGRL3pfZUd2XlypUt9uyzz2J7etfrr7++xVKNL3rX1P8kXCWROYmhq1g8S2LAJPCbulZI9Jfen2qH0e8n0SiJKWlOJDE9/RYJCVP7eVJZkOiQYqn2HNU4ovmXnimtAYL20CTQpGupv1I9LKpTRH2QhKt0Le0BJJyvmk84StfSb6X5fvny5RYj80baK9PanCW9K0G/P4/5hcaaxirVf6O+JuF6EmPTN4j29Sqeg/N8w+gbSusizdX0DoT/5IiIiMiQeMgRERGRIfGQIyIiIkPiIUdERESGxEOOiIiIDMmi7ipyDCVl/1THQ3JXkOtjavr2qqqNGzdOuufBgwex/fHjx1uM1OJVrPgmZ0BSgJO7ht41OROSa2TKda+88gpeu2bNmhY7cuRIiyV3Go0LldqgfqpiF8WePXtaLPXpjTfe2GIHDhzAa9evX99i5EK47bbbsD05DuZxFtBcI+ie5KCpYscerb9UloWgsU6lDsjF8ve//73FLly4gO3JsUYOnCp+h+uuu67F0lyluUZOTJq/VfM5bmgMk8ONHD/UPjlU09jMkspKkGuHXDBTy2JUZXcMuaPo/ZMTkd51aqmHquxwmoX6Ks0rmkPk5Ev7Os03ctzRPav4WWm/fO6557D92bNnJ19L5ZnIDZuelZxU9A1M+31aQ4T/5IiIiMiQeMgRERGRIfGQIyIiIkPiIUdERESGZFHhMQnRLl26hNeSEImEZClVOolkSVxGQrgqFmSSECyl+aa0+CSGrGLRHIlRk+iKhI8rVqxosSR8TMK3Wehdb775Zrz25MmTLUbPmQRf9EwvvPBCix0+fBjbb9u2rcVuv/32Fkvzh8R8x44dw2tJDLl58+YWSwLP5cuXt9j+/ftbLIkbSSRLkOhunpTuJO5LwmUaV5p/JC6s4rEm0vNTWY20Vkk8P09KfxLO0lxPfTVPCQoSDicxLu1XJL5OZSGuuOKKFqO9KpVKoDV46tSpSfesqtq0aVOLHT16FK8lAwsJ8lP5Bdov6duUjDJJ1D4LzSEqgVLF65W+YWkPpW8biYmTeJ/WFr3n6tWrsT31dTJf0ByieZnGf8uWLS1G39tk/klzmPCfHBERERkSDzkiIiIyJB5yREREZEg85IiIiMiQLCo8JiHS0qVL8drXX3+9xUh4m7JtkuiOxLw33HADtqcstiT8TBk06b5J9LZ3795J903vSiJdilHG2Krp2R5pTF588UW89n3ve1+LkZgzZaAk4S1l0Lz77rux/de+9rUWW1hYaLGUsZnedffu3XjtI4880mIkpvzpT3+K7bdv395ilDH6tddew/ZJTDcLZQBNWW3pWpr/KdsqZVbdsGFDi1E/V7FIm+YK/U4iCbdp/pNokYS4VSwcp+zu586dw/ZJ+EmQSDkJd0l8TeOd+pCEn7QHffSjH8X2JDImQ0jaQ0kMvGPHDryWRLJkiti3bx+2p0zGJKhN831qxuOp2eWreF2TIYfmWhWLaWkOpvGnPeDMmTMtltYFVQ1I4nsyJVAsrSESma9atarFktHIjMciIiLy/x4POSIiIjIkHnJERERkSDzkiIiIyJB4yBEREZEhWdRddeWVV7ZYcneQYpvcAimlPSn2SRmfnCnk5CAF9unTp7H9unXrWiyl/qb7UvrwgwcPYnvqQ3qv5GRKjoFZKCV5SilPyvgDBw60WHJ8kZOKnHi7du3C9l/5yldajNT23/jGN7A9OQ6SC+ab3/xmi/36179useQkIifZ+fPnWyy5UJK7ZRYqP5BS+tP8oXmSnB3kzCB3W0ppT2UdqE/IwVNVtXXr1hZLZVGuueaaFqPxTyUoyAlG7sZUFiKV+yCov1NZFnJzkmvs1VdfxfY0B8idkkrz0L521113tVhy2JITKZWmoblJZSVS+v5Dhw61GPUfzZUqXlsE7ZdpXpCTjtxt5GSt4rVNLjT6VlXxuFL5hDT/aPySQ5ScdOTEouuqePxOnDjRYitXrsT2aQwI/8kRERGRIfGQIyIiIkPiIUdERESGxEOOiIiIDMmiwmNKfZ3SYZNAiYTHV199NbYnkS8JyajUQBWn+b548SJeS5BoLAmPKa0/pe++6aabsD0JB0m4m0S+STw9C5XFoPIZVfz+JIRLosu1a9e2GKVZf/DBB7E9ibFvvfXWFkvCcxK+prIcFCeB46OPPortSaRLos2Ufj3N4VloTqe29PwkekztSWBJwmMSE1exSJ72BJqTVVVf+tKXWiyNNcUpduTIEWxPAkkqy5EEvlNLAlSxyJTmTxWPAQlkU6p7EimTwDcJT0nQSuOd9lXa29N4//Wvf20x2leTyYLWBq3rJPJNAvpZaEySeYPWG+2h9OxVbHSgckNkcqlikTftC2lfopI5n/jEJ/Baei/6hh09ehTb03NNLYsxL/6TIyIiIkPiIUdERESGxEOOiIiIDImHHBERERmSRYXHly9fbrEkXCQxHIlcSdxUxaIjEsMm4SyJHEnMeubMGWxPQqqUmXb58uUtRmLClJ2WhIvULySmrWKBG7Fp06YWS6LDn/zkJy22e/fuFqNswVVVe/fubTHKoHn//fdje3p/EtIl0TUJR1N2bRIdkuhx586d2J7G9cUXX2wxev+qqqeffhrjs1DG8STkpDlFYuCXX34Z21N2Yep/mvtVPCeff/75FqNxqmLRbBL4knCTBLJJuEzCTWqfBMJpDyJoDqQsvPRcJNzcuHEjtqd5Te9FWXCreG+k+fbZz34W21OGccoOXsWC0qeeeqrFklGFxoZ+P82BlI18yu+kTOg0fjSHH3vsMWxP5gWa6/NUHXjiiSda7L777sP2tLYPHz6M1547d67FKMN4yphM/ULC9zR+STxN+E+OiIiIDImHHBERERkSDzkiIiIyJB5yREREZEg85IiIiMiQLOquIq666iqMU0ptcpykUgXkAiAFdXImbNiwocXIHfbHP/4R25OKn1KqV1WtX7++xVasWNFiyR317LPPthg9a3IWkOuMOHXqVIul/ifXHLmLUqkKcgZQ6u55yoKQW4DKR1SxCyQ5kY4fP95i9957b4slF8obb7zRYr/4xS9ajNweVTxXpv4OuZCq2HFB64ccd1VVmzdvbjGa/+nZ6fdprI8dO4btyfWV3Ink5KI19fGPfxzbU1kHctGk9P3JXUOQay2tQepDKi2zZMmSyb9PLqLkzvzb3/7WYrTfp7IE5IShch3pt6gEA5V/qOJ+ofme3KRpDGYh13ByN5FDlBxrNFer2F1GY5Vcf9T/9EzJNUz9l9ystF/TN4y+AVXch7S3JRcV7Y0J/8kRERGRIfGQIyIiIkPiIUdERESGxEOOiIiIDMncwuMkJCKBEImTkmCIBE6UVj6l6ieR6sMPP9xiScj2uc99rsWSQIze69KlSy1GosOqqpMnT7YY9WsqoZHEp7OQQJD6qarq9ttvbzESvZHAuIpTglM/nThxAttTSnqaE5Q6vIrTx//sZz/Daz/5yU+22FtvvdViSaBJadmfeeaZFlu1ahW2P3v2LMZnIYFpEj2SGH3p0qUtloT7JHKnPqF7VrFIm0Svaf1TCQoqK1LFYkoSxCdxKfUVzet//OMf2H7q+qvi90qCZhLerly5ssVSH1IJCRKD/v73v8f2P/zhD1vsq1/9aotR+YIq/gakEhi0NsgQsGzZMmy/sLDQYjS3U1mAJEieQhKe0/uToP7Pf/4ztqf5RrFUkoLWBZXRSXvwq6++2mKpDAuJn2m/ojGtqrrzzjtbjL5Naa3Mg//kiIiIyJB4yBEREZEh8ZAjIiIiQ+IhR0RERIZkUeExCYFISJegbJkpCyyJxkj4mER/JLBbvXp1i1EW5ioW+CXRGwmCp4rGqlg4SJk1k/CVsksTlJk5ZcukOGUVffzxx7E9CVJprCgDaBULl3/7299Obn/XXXe12AMPPIDX/uUvf2mxH//4xy1Gwukq7leavym785VXXonxKe2TGJ1E+rR+khCTsgBfvHixxdL6J+EurfUkWqX3StmRaV+huXbttddie+pXWlMkBK7KY0CQyDmJxylOz0Vi1qqqAwcOtBitYdprqqq2b9/eYjt27GixJF6nfe3IkSN4LWXYpgzVzz33HLaf+m1I841EtgQZSlImetpDaQ6ljMNf/OIXW4yEv7/5zW+wPY3LHXfc0WIkUK5i4XqaKxSnvSGtQdpbSaSe1mD6NhL+kyMiIiJD4iFHREREhsRDjoiIiAyJhxwREREZEg85IiIiMiSLuqvIGUClAqpY7Uwp+OmeVZwWn5wF87hz6NqkyqYU8ulZV6xY0WLkuPnTn/6E7UkxvnXr1hajNOFVVefOncP4LOTWSI4VUsaTWp/es6rq/Pnzk+6ZeOSRR1qM0s8ndxC5CJKTjiDHVSoL8O9//7vFyNly2223YfvkcJuFXECpLTluaPySq4QcS+QiSSUFyEVz6tSpFkt9QunbDx8+jNfSe5ELJrmACNoraJznhe5BTrgqdpyQQzTNgaNHj7YY7cGptAvtS7Rf/uEPf8D2VFojlQUgJw/ta6mv6Fnpe/HSSy9h++T6mYXmUFoD9Ez33HPPpFgVl2sgF9nnP/95bE9rmObUtm3bsD2Ve5jHjUz9ksrw0PhTGaR169Zh+4MHD2Kc8J8cERERGRIPOSIiIjIkHnJERERkSDzkiIiIyJAsqoD873//O/lGJNIlIdV1112H7Ul4uGfPnhZLwmcS+JFo6VOf+hS2J+FyEvhRCQkSuO3atQvbk3j2X//6V4uRcHOx55ryTKkt9SulVE9iXBKzkfA4CQH37t3bYl/4whdajIR4VVXHjx9vsSQ8pXlBAslf/epX2P7s2bMtRgLJJHIngR1B75rSnJOYlISAqU9ISEhlHZ588klsv2TJkhaj9Z9E8yQETf1E5VIoJTyJ4dN9aV2kEjLz7It0bZoXVO6D9rVkiKC9lYTmqVQCPRf19Wc+8xlsT2ObStvceOONLUbC6SRyprVBpUVSaZX0XFOuS/ek+U4CW3r3qqrnn3++xahPUqkF+t6sXbu2xehbU8VzJZWAIFPM6dOnW4wMIVVcboP2JtqD5sV/ckRERGRIPOSIiIjIkHjIERERkSHxkCMiIiJD4iFHREREhmRud1VyHJA74c0332yxlE6bXE/79u1rMUqfX1W1adOmFiPHydtvv43tn3766RZ7/fXX8Vpy8pDrhRwr6b6UVj45yVJa8VloTJJjhRxjhw4darFVq1Zhe0of/rGPfazFUp8+9NBDLUZuEXLcVbG7hpwNVZy+nJwlyRlA5QZS+vJ3As2p//znP3gtOT7IrZPWL7kYaK2k96Tf2rBhQ4tR+YUqdu0lZwe9K6X0p/0nXUslUFJfpXIrBJXGICdpFbuDqL9SuRh6L3LnpDW8bNmyFiPXYOqXhYWFFiMXTdX0MUh9RUx17FTl78As7373u1ssjf/JkydbjN6T5kQVl2XYuXNni6U94L3vfW+L0b6exo/elWJV7KQih2VawzRWNP7JiUjvmvCfHBERERkSDzkiIiIyJB5yREREZEg85IiIiMiQLCo8JtEZCamqskh2lgsXLmCc7rt+/foWS2UhSKRMwk1KnV3FKa1JTFlV9dhjj7XYl7/85RZLAjFK1U5i4pQ+PAnHZiGBYUoJv3r16hbbv39/i91www3YnoSjP//5z1sspRT/7ne/22Lbt29vse9///vY/oEHHmixlLqdxM9bt25tMRJTV3EJC5r/SWQ7da3QcybBHc11Wmup/0n0R2nqU3ua0yTETCURaK0lkToJP0l4TuVTqlggOVXMnNon6LmScYCExzSGSVBPRgO6Z3ovMlTQfEv9SmJSKu1SxfstCXLJUFLFonISyaa9Mu2tU+6ZxLi0t9L4p7IaNFY015KhheYKfYNorVexyJvMO1UsiKZ9LZVxIWi9p/0iCZIJ/8kRERGRIfGQIyIiIkPiIUdERESGxEOOiIiIDMmiwmMSMqWMxSQ6Onv2bIudP38e21MGxc2bN7cYieOqWAxH7T/84Q9jexLIpYyp99xzT4tRZk0SnVWxoPX9739/iyWR8NSMxyQGpX6u4mcl0V8av127drXYj370oxZLojfKlrply5YWe/DBB7E9CSSTcJzEiCS6S+NP70BixPT7NNYEzek0p0hISaLFF154AduTaJSuTcJnythNAuGUgZb67+6778ZrSVBOayrNNRLo0lpJ6y+JrwkS+ZKho4rFuNSHSfhKc/jTn/50i7311lvYnkSe1AdJuJzmJkEGEDKapDVEgljqq2Q+mJq1mtZVmle0hqivU8ZjEt6SoWD58uXYnubw7373uxZLVQNuvfXWFkvf+5deeqnF6FnT+NEcpr0lmQ+SIJnwnxwREREZEg85IiIiMiQeckRERGRIPOSIiIjIkHjIERERkSFZ1F1FyuqUTplU6OTOIgdAFSvGKX01lSqoqtqwYUOLkYvi4sWL2J7U/keOHMFrCXJGLF26FK8lZT+pyFP6/+TOmHLd9ddfj9dSWQZyViS1/J49e1qMXASPP/44tt+9e3eL3XHHHS1GpSaquP9TP1FpkJtuumnSdVXsGiS3Ac2/quxumYVSss+T5pzWWkoJ//bbb7cYpZlPfUJuF2qffn+e8gHkhKTnT+6oVGpgluRimgcaa3LBVPEYUn8nd9PU0hxpXyOHGj1/che98sorLZb2G3IYvtP5Ts+aSuuk+JTr0ryifqE9MJWa+MAHPtBiTzzxRIv98pe/xPa0hsjxlhzGGzdubDFyTFXxt53cmHTPKl5bly9fbrF03khzkPCfHBERERkSDzkiIiIyJB5yREREZEg85IiIiMiQLCo8vnTpUoslIRAJD0m0RALRqqpbbrmlxUgIR+K2KhY0ksCPhHhVLPJNoikSo82Tkpr6kNonkXZKVT7LsWPHWiylXl+7dm2LnThxosWS6JHS+lP6exKjVlU99dRTLUYi8SQcv/POO1vsIx/5CF5LIneaq+fOncP227dvbzES3qfxT+L5WUh0mdLR05wigWQSQ5MQkITzqaQBzV/qkyS6ffLJJ1ss9R/9FsXSXKO9itq/U+FyeoY0BvQMNAd27NiB7WkOUxmXZcuWYXsSOdMenOYAtV+xYgVeS/ONBLnJ6EB7I5kn0h5KJU8IWldpXtHeSN+VVNaFhLe03x09ehTb79y5s8XoW5X6hEr2kBi6ik0BNK/St4rKu1BfJeH5VPNNlf/kiIiIyKB4yBEREZEh8ZAjIiIiQ+IhR0RERIZkUeExCYlSpkESGJGQLAmZSMxHQjwSjabff/PNN1uMstVWsZhs69ateC3dgwS9SWRJAisSIyaB4lTRFQlHU1sSKZOYm4SIVZxxmvo0ZRql8SPRYhITHz58uMVISFfFglISpP/zn//E9vReJ0+ebLGU7fXQoUMYn4UEspTZt2p6ZtYkOqRssSS8TuNP+wKtiZRBlUTKCwsLeO1VV13VYrRXJZH21My6SXicxPvEG2+80WKUSb6KhavUX2kPprFds2ZNi6V5SX1AfZX2NVrDtIdX8X5P34uU9Z2eiwTR84i8CZpDJBCuYjE1/Q6ZNKp4vm3ZsqXFyPhQxWJg6r80ftSnyehDgmz6/bTf033p2rRW0jsQ/pMjIiIiQ+IhR0RERIbEQ46IiIgMiYccERERGRIPOSIiIjIki7qrLly40GIpHTap2Emtn1K1k7Kc2pMLo4pdH/M4G06dOtViyYlE7gZ61pS+m5Th1H/JRZEU61N+PzlWNm3a1GLUJyklPLneSNmfUpKTY+WDH/xgi1Ga+ip2UiUnHin7qYQFufOq2IlGpBIUU9055GAgB0q6llxg5KKq4r4md1da//Rc5E5LjiVyXVKZgCreA+i+85SgmMcdmRw7BDnBkkOO3ovK4Lz88svYnsaA9iUa66rpaf1Tv9B8S2uInFi036a0/jQGtAbSHprm4Sw0VvO49uYpATK1XEna/2n8yfGV9h+KX3311XgtldCYWq4o/RaNdRo/eq+E/+SIiIjIkHjIERERkSHxkCMiIiJD4iFHREREhuRdUwVYIiIiIv9L+E+OiIiIDImHHBERERkSDzkiIiIyJB5yREREZEg85IiIiMiQeMgRERGRIfk/KQLo5b8MlTUAAAAASUVORK5CYII=\n",
+ "text/plain": [
+ "
"
+ ]
+ },
+ "metadata": {
+ "needs_background": "light"
+ },
+ "output_type": "display_data"
+ }
+ ],
+ "source": [
+ "utils.displayData(Theta1[:, 1:])"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "### 3.1 Optional (ungraded) exercise\n",
+ "\n",
+ "In this part of the exercise, you will get to try out different learning settings for the neural network to see how the performance of the neural network varies with the regularization parameter $\\lambda$ and number of training steps (the `maxiter` option when using `scipy.optimize.minimize`). Neural networks are very powerful models that can form highly complex decision boundaries. Without regularization, it is possible for a neural network to “overfit” a training set so that it obtains close to 100% accuracy on the training set but does not as well on new examples that it has not seen before. You can set the regularization $\\lambda$ to a smaller value and the `maxiter` parameter to a higher number of iterations to see this for youself."
+ ]
+ }
+ ],
+ "metadata": {
+ "kernelspec": {
+ "display_name": "Python 3",
+ "language": "python",
+ "name": "python3"
+ },
+ "language_info": {
+ "codemirror_mode": {
+ "name": "ipython",
+ "version": 3
+ },
+ "file_extension": ".py",
+ "mimetype": "text/x-python",
+ "name": "python",
+ "nbconvert_exporter": "python",
+ "pygments_lexer": "ipython3",
+ "version": "3.7.6"
+ }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 2
+}
diff --git a/Phase 3 - 2020 (Summer)/Week 5(Apr 26-May 02)/Exercise4/__pycache__/utils.cpython-37.pyc b/Phase 3 - 2020 (Summer)/Week 5(Apr 26-May 02)/Exercise4/__pycache__/utils.cpython-37.pyc
new file mode 100644
index 000000000..2ae58e451
Binary files /dev/null and b/Phase 3 - 2020 (Summer)/Week 5(Apr 26-May 02)/Exercise4/__pycache__/utils.cpython-37.pyc differ
diff --git a/Phase 3 - 2020 (Summer)/Week 5(Apr 26-May 02)/Exercise4/exercise4.ipynb b/Phase 3 - 2020 (Summer)/Week 5(Apr 26-May 02)/Exercise4/exercise4.ipynb
index d8ebee0cf..46565e261 100644
--- a/Phase 3 - 2020 (Summer)/Week 5(Apr 26-May 02)/Exercise4/exercise4.ipynb
+++ b/Phase 3 - 2020 (Summer)/Week 5(Apr 26-May 02)/Exercise4/exercise4.ipynb
@@ -18,7 +18,7 @@
},
{
"cell_type": "code",
- "execution_count": null,
+ "execution_count": 1,
"metadata": {},
"outputs": [],
"source": [
@@ -27,12 +27,12 @@
"\n",
"# Scientific and vector computation for python\n",
"import numpy as np\n",
- "\n",
+ "import pandas as pd\n",
"# Plotting library\n",
"from matplotlib import pyplot\n",
"\n",
"# Optimization module in scipy\n",
- "from scipy import optimize\n",
+ "from scipy import optimize as opt\n",
"\n",
"# will be used to load MATLAB mat datafile format\n",
"from scipy.io import loadmat\n",
@@ -87,7 +87,7 @@
},
{
"cell_type": "code",
- "execution_count": null,
+ "execution_count": 2,
"metadata": {},
"outputs": [],
"source": [
@@ -128,9 +128,22 @@
},
{
"cell_type": "code",
- "execution_count": null,
+ "execution_count": 3,
"metadata": {},
- "outputs": [],
+ "outputs": [
+ {
+ "data": {
+ "image/png": "iVBORw0KGgoAAAANSUhEUgAAAjoAAAItCAYAAAAwm9DjAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjMsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+AADFEAAAgAElEQVR4nOydd5wVxbb9F+9KUFFMmEExYM45YA6YFUVUzApiQFEx52vOCb2iIIqKWVEx53BVxCyiYs45XwXE997vj/db+6yWPgwzc/p0T7u+/7A/xcyZqq6qPt1r79q71f/+7//CGGOMMaaM/FfeHTDGGGOMyQo/6BhjjDGmtPhBxxhjjDGlxQ86xhhjjCktftAxxhhjTGmZbmr/OWnSpFIeyWrbtm0rAPjjjz9KOb42bdq0AoCJEyeWcnzt2rX7W4zP+69lwv3n8bVM/i7jmzx5cinH17p161Z/bbOiY4wxxpjSMlVFp0i0atVqCltzADkfkKk1uuYaav+f//mfrLuTK2lj9p4zxlTjv/6roqPk/V1tRccYY4wxpaXmio4+xelb4H//939P88+yXX/nzz//DPvXX38FAEw//fTR1qZNm+Z025jgH//4BwBg8uTJ0aZrceLEiVP8zkwzzTTF76et+ZaE7k+9FrR1zzVG/WrppL2p5q1ucc2preuvJa7FtO8EtautLa7PvOfk785//vOfsNu2bRv2dNP932NHPefHio4xxhhjSosfdIwxxhhTWmrmuqKM+OWXX0bbpEmTwu7SpcsUP/vzzz9H27fffhv2559/DgAYO3ZstL333nthjxgxAgDQr1+/aDvppJPCLntgaFFIk5OBiiRZdLlcpW+V/r/44gsAwKBBg6KNaxIAnnvuOQBJ6XXHHXcM+4ADDgAAzD333NHWkmR0Xot333032i677LKwP/jgAwDAEkssEW3qxjrqqKPCnmGGGQDkM/5qro1p7Uu19fH7779P8TN5uM61f59++mnYzz//PABgpZVWirYFF1ww7KKvxbR50/F99NFHACr7FEi6jtddd10AlbUHFOs7geNr6P6pfS5S/9PQ/fH1118DAHbZZZdou/DCC8NeZpllANh1ZYwxxhhTE/ygY4wxxpjSUjPXFd0Uhx12WLS9+eabYe+5555hU7q7//77o41yJFCRvhidDSRl8lVWWQUAMOecc9ag57VBpccsKJJ0SZlS5/fqq68Oe+ONNwYAbLLJJlP9HJWo6zU+nacff/wx7FNPPTXsxx57DADwzTffRNtaa60V9jnnnAMg6ZpT1w755z//WYMeZ4deCz3V+MwzzwAA+vfvH23vvPNO2HSDPPXUU9H222+/hX3QQQeFPeOMMwKos0z9/8elY9L11bp166n2ib+vJ83uuuuusEeNGhU2r9Hyyy/f3G5PM9w36kLbZ599wub6PfbYY6NN12IRXcp6L+Cp2osvvjjabr311rDpkpowYUK00Z0KAH369AEAnHXWWdHGOQfqtxar5ZH56quvAABPP/10tHHOAGCeeeYBAOy6667RtvDCC4ddpO8CovP39ttvAwBee+21aNP7Qx4nMa3oGGOMMaa01DwYea655oq2u+++O2x9o+AT6WKLLRZtq6++eth8et15552jTYPpNPDpr5/ZXKo9hRN9GtX/1yfWWr0xaF/atWtXk89sKtoXvnHtv//+0cYASKDyJP/4449H27zzzht2hw4dAABrrLFGtC200EKpf6tW8DNfffXVaNtpp53C1sB4jkuViVlmmWWKz9K3xI4dO4a9++67AwAGDhwYbXPMMUfY1fJD1QuuYSqnAHD++eeHPXjwYADAOuusE20amL3CCisAqAR9AkDnzp3D1sDQPN6emefo8MMPj7ZZZ501bN6L0rKtA8APP/wAANhmm22iTQ9ZPPHEE2FzXev9J+3+lAWq6PAtGqisVe1TEQOQdc5Unenbty+ApMpx6KGHhn3wwQcDSO4/VRepZL388svRpveaLBSttGz96qW4+eabw+ZhGt1/f/zxxxSf+cgjj4StihbXXB73jmlB54V8/PHHYa+55pr17A4AKzrGGGOMKTF+0DHGGGNMaamZ64qBw9tvv320XXnllWG3b98+7COOOAJAMthqttlmC5s5KaqlMM9ShtV8PgykBCpynOYGYtAmAFx//fVhNyQppsmcCiVdDcbWwMIll1xymv9Wc6gWLMzAXQYdA8lgZF6j77//Pto08Jdz+csvv0Rb1tI6A0uPPvroaNPcHA888EDY66+/fuJ3gPQAujR3B1DJuaOur8UXXzzsFVdcMezu3bsDyH782lcGbjLfD5AMjGQwpwZozz777GEzj5AGgO61115ha2mWLAMn1fWh6eaPP/54AMDw4cOj7fbbbw+b10Kvie4jBrG+9NJL0Xb22WeHzWBR/Yy88wTpvSItD0vRUdc3Qx7OPPPMaNPA+DTXtt6LeC/W+0u9uOeee8LW/fP++++HzbnS+Utz92juuGHDhoXNwHN1h+eNrn+GIeg+0TAWPiOoizfr/WNFxxhjjDGlxQ86xhhjjCktNT91pRKcSqeajrtr164AgPHjx0ebyozzzz8/AGC++eaLtplnnnmKv1VLuYtyqMr5KofTtaalAJgPQX9fUWlST+XQjbPIIotEm0rPPK303XffRZueAFl66aUbHE8t0P5/+OGHYb/11lsAgBtuuCHa9FQS573aqZa//hyQfW4P/q0xY8ZE25Zbbhm2nsqgG0PnRE+48DOuvfbaaFPXF908ehJk9OjRYauMvdlmmwHIRrrVa66nAuna0f6pzH7IIYcASJY10FMxzG+iJS569OhRq25PFZW79aSK5kxhTiOezgGADTbYYKqfpSdc/vWvfwFI5qZRu6GTmfVC74l6KpVwHouKrs9PPvkkbLqMl1122WjTvZjmkrvqqqvCvu+++wAkTz1mMU+6DphTTPPI8XQqAOy3335h082m+0fnkiesLrjggmjTPF0MCVB3ah65aRSdE97/WOoBSH5/8F5Sz9OZVnSMMcYYU1r8oGOMMcaY0lIz1xWlJ3VhaIkGPXXD01Yqp6v0RZlST3pceumlYdONUy15X3PQxGd6qqRTp04AgG7dukXbcsstF7YmvEuTVvUkCk8l6fg0uRTdGRpVrzJgvU5T6N+54oorwmYiR51rPaFURCgz64knPVWj13+BBRYAkExYdu6554b9wgsvAEi6a/r16xf22muvDSDpetV1pS6hLOdS94SeiqO0f+CBB0abJmTjtdL99eCDD4ZNaZ0uMCCZKDSLMbFPmthRSwRcd911YTPhoVZPVtcHP0tLmNBdBVTWt7oG9ARm3iUUOC+ahE1LdGy11VYAkkkSi5gwUPuk5WLYb3X9KHQt33LLLdGmrlfO+6KLLpr6t2qF7o+HH34YQNKdqn3ad999w+Za1D7p+txuu+0AAEOHDo02TVip35tFQcfStm1bAECXLl2iTZNs8rSrhmBk/Z1mRccYY4wxpaVmig7RfDj6Fvv666+HzQKdmmdEFQv+rL5RaZAw316YjwdovrrDJ0p9Ctc3NyoyWopBn0IbeiLVPjEIUp/MdXx8k9E8RAzgBrLNnaMBdhp4reoGA+6KmoI8DQbJn3feedHWq1evsDWnE9ct39KApHrDYHEtNJuWO6ba+sjy7UUPA9x7771hn3baaWGz3MpRRx0VbWllC959992wVfFhgVMN9s0iQFf3NIPBdc/r+G688cawWbpC+6HXnOqd5vzS+xbHqp+vget6D8hDKeG11j2phzl4L1WVoIjotUsroaIqmwaTU7078sgjo22llVYKm0qjzlOtVLi0NQlUgtk1X5beU1TFZV+qfdYdd9wBAPjpp5+iLS3Pjl6/vIOR09D+5a0oWtExxhhjTGnxg44xxhhjSkvNtE1Kw3o2Xl1XGjh3zTXXAACWWmqp1M9idWQN/NWcEEwNru6uLbbYIuzmyJSa70eh9JZWZbax0OWj0rPmWeH4l19++Wb/reaguVMYIA1UgszzliMbA9cnry2QrG49YMCAsJnzQd0xmtODQXQMugOSpUHygH3V3E7qptM8HWzXwwJpJSI0QFT3FF2qrEL/1/+vFepOoxtJg6p79uwZtgYWM8hcq0OPHTs2bFb61gB0DaY+5ZRTACTdyfUqa9EYNABZXT8sYaLXr+huZnWz8WCHuq50rlku4phjjok2DfblIY4sq5QDydAD5lxTF5OWE0q7P6hreNCgQWFrfqs0uO+K6K5S9Prrd4m66eqFFR1jjDHGlBY/6BhjjDGmtNQ8j45KvJqCu2/fvmEzv0FDFck1N80ll1wS9o477gggeQJDXQvMefPXvzEtZCVLqzRLGV3dJermO+OMMwAk82DkIZerNKp5HNL+v+huLLp2tMr4iBEjwmbuFKAyLzfffHO0nXjiiWG/8sorAICddtop2tZbb72wedoj6znT609pXHPbqASuY1WXL9ETdsyvo3uHJ0GAyr6sZz4Zupl0zLfddlvYWp2c15+ny4Bk2Q3m8dBrpaeqmP9D81jl7frRcdMN8MUXX0Sb3vPohiuKi21a0PFNnDgRQPKeotXN6abScgv6vZPlutRrqu5C7olRo0ZFm5YN0fs/XYpvvPFGtGm5IZ7QUjeYur66d+8+xWfmndtJ4bzpfUZzItHluuqqq9atT1Z0jDHGGFNaap5oQZ94NQutPrHzZxpSAfQpVbMsMv/HHnvsEW2a82SvvfZqbLczQYNZNUj0uOOOA5AMFtTMrsy4nIdKon9znnnmCVuVEGbW3WijjaJN36g4bp2/vN8u2RcNatfA8nPOOSdsBulqnhxdUwyC1ABJfXtjzhrNppt1UUFmA9Z11L9//7C1KCx/Twvt7bbbbmGzaOv9998fbZqnpF5vj/p3qDIx6zRQefMHksHEVDfee++9aGO2YADYf//9AQDHHntstOn6pXqTt4qj6FxTyRk3bly0aYHIogeppqFKyJAhQwAkg/21ADL3sB4cyWOudE9TZdJs6y+++GLYafc/vT9suOGGYTPYWLOR67rnYYiiqujs12KLLRZtuibTsuhn7R2womOMMcaY0uIHHWOMMcaUlpq5riitVkv73JR00CpnqXTLYEOVtjVnQRElPU0nT0lS3SXqGsizQKZeO5WGNSfLBRdcACBZqI2FMIGKy0vTtuddYJBr5e677442HZMGntKlpWtu4YUXDnv48OEAknI73ZFAxbWgZQlq5U5Qd2daHg4GKgKVoPa//n32W/MI6Zpjv9X1nEewY9paXHPNNVN/VsfHnDoa7K85cdiuaflrkR8rS9Lyt2iBU80PRddHEe+DGkDL4rhAxZ0IVFzC6i5niQUA+OabbwAkD3DkgbqjuC5HjhwZbXp/TCvEueCCC4at98rx48cDSIZjaJgA71V5hwM0hrzLQVjRMcYYY0xp8YOOMcYYY0pLs1xXKp3xbLxWAdaTDOoGoUugITlf00ZrHheWkNDfn2+++cLOO78Lx6eprlXG3HrrrQFUqvQCxcqDQHR+1XVDl9t3330XbXqCZ+jQoQCSJxBYtgPIV1LX3BSaQl/XGn9G+6nzQ5eHnrrScgOsqqxrVtdnU8bPNa391Nw+PNV3/fXXRxvzxQDAySefHDbdd3qS5aabbgqbpUeKtCbTZHp14+m13mWXXab4nbScK0UaX1PQ+5zmdOH6zdMF/lfS1u9ll10W9sYbbxz22WefDSDpetVTq9xr6lrOG641PR2s+yuNatXHmf/qxx9/jDb9XuW6L6rrimNhvjEgecJM8+ORrL8TirNSjDHGGGNqTKMVHQ0me/bZZ8Pu1asXgGRui1lmmSVsDZZjFmN9Sk17omPQJ1ApxAdUArs0N4jmyciDNHWKbyYAcNddd4XNIFgttFjEwEFF3375RqwByPr2wrdrzemSN1y3mu9Gs23feeedYVOp0TcmfXvkXGkek2eeeSZsBoNqMOX8888/xe83Bf1d3Wucnz59+kSbqhy6b/v16wcAOPjgg6OtOdnE64nuM82dooHffPvXbM71ypxbTzTPjGaZLeKbPlUIzeb86KOPhq1FOS+66CIAleKxQDLLLoN4izhO7VND/au2lqk0q0q31lprhV309ctxqQqn+08PftQLKzrGGGOMKS1+0DHGGGNMaWlWMLLKbQwy02AsldY1f4m6cdJIKwq2+eabh03XlxZiVDdQHpKmujbGjBkDoBI0DSRT1LMApP5OkdLNNwTnp5qEOnr0aADJPBF5Qzn1pJNOirb1118/bF2fn332GYCkm0iLXtJlRxcVkHQDbbHFFgAqqdqB5q/JtKK5LIsAVIoC6prSQrdaDmKVVVYBkJTOiy6HEx3fJ598ErYGYbNEwGqrrRZtRXRzNBWuBQ3sVdd+kUtA6J7SOaG7CqgEUWs4ggbTsyh0S7pnpqFrmfccAHj66acBAO3bt482/X4rOryX9O7dO9r0/prHvcaKjjHGGGNKix90jDHGGFNaGu26UtlJpWFWPNa06ipTTpo0qdGdY6kHIJn/hH9D82jkIYepRKwy8oUXXgggeerm3HPPDZvuvZbiLqiGjv+XX34Jmy4FHXPecC3qSZXNNtssbHVjpbk5VCbnCSZd37rus8xjon1bd911w1599dWn+FndH7qX+BlFP+mXho6fpUaAZIkAuvfyzqdVS7T/LKfSrVu3aFt55ZVTf7YopO0/da2qm3u33XYDkDypm7Z+Wzo6T6+99lrY/C7l6UggeYK56ONPyymkJ+zSSjhljRUdY4wxxpQWP+gYY4wxprS0mprMOWnSpKlqoCoNZx3pn9bPpkq0bdu2bQUAf/zxR7M0Xh2zVj9mJV6VW7VSNsthZCVBtmnTphUATJw4MVMNW08NfPzxx2G/+uqrACqnj/76s82lXbt2NR9fc9dvLd0FHF9D+09pqP9Fcmdksf/S3FR5jZn7r7njqwbHWm3Os3ZtNGd81easofmr51xmPX9Ex8yK5QBw2223AUieWlLXXnPnl+ObPHly3cZXz/3ZunXrKTaGFR1jjDHGlJZmKTotlVq9UVaDQcia80SLmtXrjStrRUdRxYZ2VnkuslB0ikRTFJ2WRNb7L2/qpQjkhcdXe/T+ycMOev+s5XdGvRSdvLCiY4wxxpi/FX7QMcYYY0xpmarryhhjjDGmJWNFxxhjjDGlZaqZkR3s2TJhsOeff/5ZyvFNN910f4vxOdizZcLxlf3+Uvbvh7IH65Z9/hQrOsYYY4wpLY2udVUL0hJeNSbhWd4JwcyUpM1l0WuyGGOMKT9WdIwxxhhTWnJRdKjEaMVrrW5ORUAVG5ZNAID27dtX/cwyoFWnSRErnauKo5W6J06cCCCZJDHrEiFZwyReQEWp0jlp6eMrO5qQLW1/qfpYxL1mzNSodv8p0/dic7CiY4wxxpjS4gcdY4wxxpSWurmu2rRpE/a4ceMAAN27d4+2b775JmxKy+oOWWaZZcK+4YYbAABdu3aNtqzqKtULldM/++wzAEk5vVOnTmHnHeRLmVQrtp900klhf/TRRwCAYcOGRZtWcm8pcqr2c/To0WHPO++8iX//+rNlRNcn3UA6ZnX3FOVaqJz/+++/h/3ss88CSLpW55prrrC1UnSeqLtNSbu+DV1zvRZpn6v3lKLMX2PQMZXxsEqaa0rHrPdihg4AlTCPPK5FY6qXp61PvedUcy1P67is6BhjjDGmtPhBxxhjjDGlJVPXlcpR77zzTtjnn38+AODrr7+ONj1JRTlK3R1vv/122P369QMAXHXVVdHWpUuXsIvi2tHxp/VJpcfffvst7KOPPhoA0KdPn2jr3LlzzfvZVDguPTU3fPjwsNdff/3Ez7U02O8JEyZEW8+ePcM+8cQTAQD77rtvtLV016lCyVglYrqLAeCBBx4AACy00ELRdthhh4U900wzhV0U98Gpp54a9kUXXQQA6NChQ7Spa3zEiBEAgDnnnDPa6nUSS//OSy+9FLbeK2affXYAQMeOHaOtbdu2Yaftu//85z9hP/PMMwCA119/Pdp23333sNUlm+e9tJrrjntNT0Lq9wO/N9QFmfd3QkOk7RPts4Zx/PrrrwCAl19+Odr4nQoAX3zxRdjXXnstAGDVVVetXWcbgOtP+/zzzz+HPeusswJIP/0IAGPGjAEAXHLJJdG2yCKLhH3UUUeFzXXf0Pxa0THGGGNMaclU0dEn8ltvvTVsvjFNP/300aZPtHw609/XN5bnnnsOQPLJTtUdfVOr15N8mnrz/PPPR5s+UfNNRMf3wgsvhP3tt98CAFZeeeUpPrNI/Pjjj6ntLVXJ+St8cwKSb0llGV81xfGHH34AADzxxBPRdvzxx4f96aefAkiuzwMOOCDsmWeeOew8FB3uq/fffz/aBg8eHDYPRiy++OLRpuqdBi7XC94TPvzww2jbZJNNwm7dunXYnLfZZpst2uaee+6wF110UQBJxVXvRcxZpnPGt2wgfxWO8/f9999H24MPPhj2kCFDACSvT7du3cKmoqMqf5Hg9aXaAgCPPfZY2FQ6GDQPAB9//HHY3Ks656r463WpporVmrRg4mOOOSbaqCICwMiRIwEkD9jo/1M913247bbbht2U70IrOsYYY4wpLX7QMcYYY0xpqVswcloejoZQiUp/hzLv+PHjo01lzllmmaXxnW0C1XJTMOfKddddF23quuLvab4DyrEAsPfeewNI5vnIOy29jpX9Pvvss6ON7o6//mxLhP3XAE5FgyBbCro+04Kt77777rAvvfRSAMk5PfTQQ8OmS0sPExQJjlXHpCVm6LqiCxwAttpqq7AZ7FtPFw77d/PNN0eb5kbRPTXHHHMASJbFGTt2bNi8/+j9Y9NNNw27d+/eU7Sl5TmpJ/r98MYbbwAABgwYEG1LLbVU2Aycvvfee6Ntjz32CJv7s4jufqDSry+//DLa3nzzzbDpetxss82iTd2stBdeeOFom3/++cPW+xPnUg9LaE675lAtTw7djAxRAZKuNYah6J68/PLLw+Z9R/fCWmutFbb233l0jDHGGPO3xw86xhhjjCktNdfgVYJUaVuj/vkzaXJ6NfT/eQKBpRKA5AksjWanvJuFjKljfe2118Km9H/BBRdEW9qpCc1j8cknn4StUfNFQefqrbfeApA8lVMtJ0JLhPOj60hdh3rapcikuQOASgkWlYb11ANPq+hJSS238uKLLwJIL9tSBDh/elJTJe40uXudddYJm65vdR1lgd7T6Ea86aabUv9fr+8hhxwCANhtt92iTU9A0rWs459nnnnCpvTf0DXJGr2n6P1v//33BwAceOCB0bbjjjuGzWuhJyG1hAfJ+/SYonPJE8LqzlE3aloJhDR3U7WyCGlhDrUKJ6jmrtJTw/379weQdE3xOxGouF4ffvjhaLvtttvC5lxvvvnm0VatHIRdV8YYY4z52+MHHWOMMcaUlpq5rigtqbuKEiQAPProo5U/mhIVn5YwUBMKNpRi//777w9byynoyYNakHb6CACOO+64sPfbbz8AyVTq2n+O7+KLL4627bbbLmxK53mfGtCxqgx59dVXA0jOdUs8iaSojE4ZWeXkJZZYIuwiuWkI+6/rTPu/zz77hE3JX08n6l496KCDACTdAeraeuSRRwAkyz5owrk81m3avtTke2nSvZaq4EkXoH4uD/07vE/dc8890aYp8K+55pqwedrxu+++izZN6Mi9WM01VRSXjrpY1M2/5557AgB23nnn1J8dOnQogGSSvWOPPTbsvO+bRO8pWgLhtNNOAwCccMIJ0abfdey/zpOWU8gD7h/tk94T9P7CdamncldbbbWw6fI+/PDDo01dc3RZ6ndKc0vsWNExxhhjTGlp1mu4PrHy7Z5vg0BSZdG3p7SnUy2gx3TPmidC1Q8+Xabl1tH/ryVpn6llJ/Ttd4sttgCQfPrVJ1a+aesb56BBg6b4W3m/mWj/NZiab7/6llWvVOO1ROf0ySefDJtvx2eccUa03XXXXWF/9NFH2XduGtBrzpw/N954Y7T985//DFvfKJk6XvcqVUigkkJf36I0cJD/v9NOO6X2JW9FhyVUnn766dSfZV81TxIDtIFkTp16wf5rIcozzzwzbFXfqPScfvrp0aaKGtU5vefkfS8huk40j4wW5eS4NTfaueeeGzbHr8GqRUeVOqIqfhEVN4UqtuZr0nuG3l+o5GjRY92fo0aNAlA51AIkFaEVV1wRQG3XbMv7djLGGGOMmUb8oGOMMcaY0tJo11WaXA5UgsHUXdW+ffuwNXCX+QNUmtVgT6a21oq1aeUgtE1zXmSZM+fdd9+NNk2Lf8MNN4T91VdfAUiOWWXYgw8+GACw7LLLRpsG9jIwTd1FecuZGoBLGV1dV9rXloJe00UWWSRsppbXQPbzzjsvbF2r9UYlYM09cvTRRwMAHnrooWjT6r+Ug4GKS27ttddO/Vza7733XrTp+qZrRIPt83CN6L1Ixzpw4EAASdewurbpOu/bt2+0sewK0PzAx+ZQbU9psDHT4WuemSOPPDJslhM46aSTok3z6OR5L9G/reEMGoy7ww47AEjmSVOb87Pllltm1s9ao25ilt5gPhkg//t7GnrPp7u+e/fu0abfabvuuusUtuZx0urrF154IYBk9XU9DMF1X8t9aEXHGGOMMaXFDzrGGGOMKS2Ndl2pxP3TTz+FzXIGKifr6So9TTBs2DAAwMorrxxtKtmyeukdd9wRbSrtUYbV31l99dXDnnnmmVN/rznwc9SdodVlOSagkjpfXXtPPfVU2JT5O3bsGG2aU4B5avQkWt7Vy5WFFloIQKXKM5AcaxFl2IZQNwxR6fSDDz4IO4/q7Pybms/oiCOOCJuVulUOpwsHAPbaa6+wua50nnRMdPn069cv2lSm5qkk3et5r089CcecXeo6ThuryvC6lvN0XSnV9tGGG24IALjvvvuiTeeX96JXX3012v71r3+Fvfzyy9e0n40hLXcQkDxVyzxPmodFT+gwJ0vPnj0z62et0ROQV1xxBYDKngWAHj16hJ3n/VPvA7oPeMJPQyx4+hJIfr+xuvxyyy0XbXr/pEuc7va//mwW+8+KjjHGGGNKyzQrOmmZEfWJ9J133gGQDKBTRYeFvoCk+kL0iY/5ORwwlcgAACAASURBVDS3gsKilxrMxAAvIPmkWaunY76xqsqiRRH1TTsNDZxj0bIDDjgg9fcZxFWU3Bd/Je2aVsu501JoaEx5w/2nhTQ19w/f0lUZ1LdEDSzkWtY2VWcHDBgAIPmWpjmFGLidt4qj/dcCswxM1rdTvRY8OKEqc95jaQzsK5VVIFkMlHOleb40WJn3LVWx8rjX6P7SApc8jKJFO1W9Yn4WzbNWxHul9okHUIDKd4HmBtLvh6Jkmdf56dChA4Ck8qLQCwNUiv4+8MAD0aYqDZU8/c7L+l5rRccYY4wxpcUPOsYYY4wpLc3SyDp16hQ2c+ZoAKBKy3feeWfYlCZVehw+fHjY48eP/7/OVZHwLrvsMgBA7969o03zFCi1lsRUDlcXmdoM0uQ4gORYtt9+ewBJF49eK/a5SK4ThcGqKldqYOpKK60EIHlNiigtN4TOtbpJ84BrQXNTnXzyyWEzHf4CCywQbeqOSXPN6F699NJLw6YbSAslaop2rtW8c+doHqHnn39+ip/V+VtnnXXCpstH92QR16fOWVohWf3/+eefP2zeXzT3kV4ruvY0cD1vdCycN4ZDABV3P1DJs1NEdM1Vu3/zMI26btTm90Ie93/9m7rmTj31VADJe/qECRPC1iBlhqxonicNjD/nnHMAVEolAdnvPys6xhhjjCktftAxxhhjTGmZZtcVJS2V5jQPDuVg5tMBktKwSsta9ZjoCS2Wc9Dqu126dAn7uOOOA1DJJwEkU4hnKYNVqzKblluFVXaBZG4fVjpXubaobqo0OFYdc1pVYo20T3PNFR3t85prrhk2K/XW093Ba6ZVqlUa5lqaltND3Jd6qur8888Pe4UVVgCQPKmjLrOinFDSPmkKeVZap4scSKao5/iL6K7SNTd69Oiw9f6x9NJLA6h+/2CZmt9++y3a1l9//bC5hoq0D3Xc77//PoBKPjIAuPbaa8PmqZ2816H2mXmKNtpoo2jr2rVr2Hqv/OWXXwAkT1DmkZurIXR98LtYr7mWeNATgE8//TQA4KWXXoq2bbbZJmzm2alnHi4rOsYYY4wpLY3Oo6PFM++5556wWchSn0z1Z/Xpl0FM+hTHc/oAsPjiiwNIBoCyEBhQeXrUN7IiPd2zgJm+MWueC75R5t3npsJ+q2Kz1FJLhb3uuusCKNYbY1PQOd14443D3nPPPQEAffr0iTbN8prluPWzVQVtiLQ3Ss1Gq2uRGb/1jS1v9YP91zGPGTMmbO0rr5EGUKs6nPdYpobOk+aROeWUU8JmkLHeH1X9oZKsb8zbbbdd2FSH8s4ArWPVeT3rrLMAJPus95ei3De1/1RnNFB61KhRYev3G/McsTgykPRIFPG+2VCfdP6Y80gVRVVUeV+t5/qzomOMMcaY0uIHHWOMMcaUlkbn0VE5XwOvKOPxjDyQDBbWnB0MEtRgQuYWAID11lsPQFJu1mA8Sl55S3wqXer4TjjhBACV4odAxR0HFFs6nxZmm202AMnrr4GfnHeVzvOeq+Yy33zzhc1xae4IHX/RxzpkyBAAwCOPPBJtWiCRLp8i5ZnhXtPiscztAQAvvPBC2Lw/FDHAsyH0OmsAPA9gAMB+++0HIFlKZ+jQoWEzmFfLCqgbqCiuH70/sGwAAIwdOxZAshBmkQrIEu0H87hdfvnl0abFU3nPBCrlVh577LFo0+/VvPfatFLt+2/cuHEAknnyllhiibBzyb9V979ojDHGGFMn/KBjjDHGmNLS6Dw6KiGqXM88NyqBq3SVVt1aP0tJk5xV7iqKa0D7qRHm8847LwDgqKOOSv29ovS/MahMy0rx6rpJq1jeEsepVEuxzxME6kYpIrq/NCcJT/BsvfXW0aYnsHiCrEgSOvuiFbd79eoVNnPLAMCvv/4KAFh77bXr1LvaoddcT1Xp/PTt2xcAcP/990fboosuGjZL4wwcODDa9FRgUVw/eir39NNPD5tuIB1/Ufqs6P2NuYmee+65aFNb7xVrrLEGgORJsiLttWml2vcfxz3nnHNGm4ap5PG9YEXHGGOMMaXFDzrGGGOMKS2tpiYjTZw4cZo1prSyANVoSLrKWtpq165dKwCYNGlSzf5QWjmEvNxtbdu2bQUAf/75ZyZ/tKG5zlqGnW666TIdX0OkzW8t4fj++OOPZo1P5+e6664L+6OPPgIA7LzzztGmro+s569NmzY1GZ+65ork7ub4srq/vPXWWwCAzz//PNpWXXXVsJmcrlq5mubC+0tjvh+IjkPdOepaZaXyeiXh/Cv8fpg8eXKj/2i1cIw08nJXtW7dusnzp+hYNYxh8ODBAJIlorp37x4210BWc8r5U6zoGGOMMaa01EzRaUlkoegUiawVnbzJW9HJmlopOooGi/ONSlOw1/PtslaKTlHJQtFR+CZdLbdM1upHcxQdRdUdzdmUd5605ig6LYFaKTqKrkXea/Se0phyNc3Fio4xxhhj/lb4QccYY4wxpWWqritjjDHGmJbMVBMGOgaiZcIYAfuYWyb0MZd9fN5/LRPuP38/tEz+LjFyil1XxhhjjCktja5eXkuq5WGxO80YY6YdPfXC++ff6T5apDxmTSEtJ1RL6n/RsaJjjDHGmNKSi6LDJ9aJEyem/n/btm3r2Z3M0LeMf/zjH2GnZc/UPBh8ki/TU3xazgx9Yyli0T5jioLeC3j/0DxIP//8c9i8f04//fSpv18W9J6iBUKZcXmmmWaKNs0jVZRrod8Jv//+e9gsSjvzzDNHm34nFqX/is4FbR3ftFRMIPxeqGVuKCs6xhhjjCktftAxxhhjTGnJxXVFSeuGG26ItjfeeCPsc845BwDQpk2baCuiXFeNNGn5vffeC/uTTz4BkHTdLb/88mHPNddcAJJp0fMqANccVK5UGfKFF14AAHTu3Dna5plnnrBbylirybFpRU+rXYss17W6SFVGTgtWLWPgY7Xr31DhRY6/SNdE+zxp0iQAwKWXXhpt559/ftgspti/f/9o23TTTcNuKfurGmlFIS+++OKwTzvtNACV+wwALL744mHnPZfci99991207bfffmHfddddAIAnn3wy2tZaa62wi+Lmr3ZPoxvu1VdfjbZvvvkmbI5ff1+/K5dcckkAyULD+l3YlPmzomOMMcaY0uIHHWOMMcaUllzz6Kgc+8orr4RNabbokeaKugYY9X/GGWdE28033xz2559/DiAp1y2zzDJhr7vuugCAU089Ndrat28fdtGl57Soe5WRt9pqKwDAsGHDom3LLbesU++aRtoJumrVeemm/Oijj6Lt22+/DbtXr15h0z1bq/Wte2rMmDFhDxkyJOxZZpkFALDeeutF2wYbbBC2nlBJI811kDdpbkQ9yfLiiy9OYXMfAskTLgsssAAAoEePHtHWoUOHsPPYf3qthw8fDgA45ZRTok1tusYPPvjgaHv88cfDppu4SPPXGLjG1R0yePDgsGeccUYAyXtm3ui+5L3gkEMOiTa6qwBg9913B5AMZyjKXOk4fvvtt7DVdci19tJLL0UbT5IB6a593VOdOnUCAOy8887Rdtxxx4U9wwwzpP7eVPs9TT9ljDHGGNMCyUXRSQv202DdDz74AACw4oorRlsRVYxqT7eHH344gGSwtQZWbbvttgAqbx4AcNttt4X9r3/9CwDw5ZdfRps+Mc8999xhFyUwTaHi8dlnn0Vbv379wua4VcUqyhsLUHnT0AA4VWw+/fRTAMADDzwQbffdd1/YP/zwAwDgzTffjLY11lgj7J49e07xt5o7fl7zjz/+ONp23XXXsL/44ouwmXNE19Qee+wR9mWXXQYAGDp0aLTp5zLYles4L3T/aR6Va6+9FgBw+umnR5vO5XzzzQcgmWelXbt2YV911VUAgNGjR0fboEGDUv9uvdC3XwbWqgrat2/fsF9++WUAwKOPPhptevChiIpcQ6QFrnKdAsn13bt3bwCVQx1APmNNCyAHgAEDBgAAbr/99mjT9bfCCisASCoXec8Vx6Iqzcknnxz2888/HzaVVPXIUEWuht5fGaSt86vfrww2ByrfJQ1dHys6xhhjjCktftAxxhhjTGmpm+sqLfBIz9l//fXXYY8dOxZARcIrKjqmJ554IuwRI0YASEqPq622Wthbb701gKRrbpdddgl74MCBAICRI0dG23bbbRe2BmkVRYbWwGNKj0cddVS0vfXWW2FfcsklACpBZ0D+0rLav/zyCwDg2WefjTYN5rzzzjsBVPI9AMn541jpIgEquaGAZGr+WrlkuQ4+/PDDaNNg24UXXjhsunRULk4LfFTXlbrhGCy5zTbbTPH3s6aau/jII48Me9SoUQCA/fffP9q0r3T96v7UnF0nnngigOT+0/uTzmu9XOq6P5hTZdVVV402dc3xXqTuEN1rRQwDaAi9v3AvqjuxY8eOYdM1pHOah4tf9wTDMYCKS3GJJZaINg2sXmWVVQAkx6x7tV5o/7nX1F2l33m6L7k/1F3KeypQWX+6//RgxPvvvw8gec30XrTbbruFvdJKKwGw68oYY4wxf2P8oGOMMcaY0lI315VKS5TEVFqdbbbZwuapgnrJ4Y2h2tn/u+++O2zKeBpp/9hjj4VNlx3zyQDAAQccEDbz52huIT2VpTI8r2Herh899cKo+BtvvDHaDjzwwLB5wqfatcwS7bOmYH/44YfDvuOOOwAk3W3MbQQAN910EwBgwQUXjDY9QcETCFdffXW0aYkLzZ/UHHQszNlz2GGHRZu6M/RUDvPDVCtxcMsttwAAvvrqq2hLc4PUc3/yb+m1O++888LWE3CcP8raQPJacdzqGuBJOqCyV/VUpJ4gyRteC5X+9YQjXXd66k5zI7UU15XOmbpBbr31VgDAjz/+GG16f1lqqaUA5D9O/fvq7nzmmWcAJPuvOZvyDEOoVtaBJzSfe+65aNM1pd9fPPX20EMPRdsFF1wQ9kEHHQQAWHPNNaNNwzjoJr7mmmui7eyzz27kSJJY0THGGGNMafGDjjHGGGNKSy6nrghLJQBJmXjWWWetS58aQ1pZA02edO+994bNsey5557Rpiel5p9/fgDJhGUq0y677LIAkmn5NSGTnjZRl0K9SJtLPQFx+eWXAwCOOOKIaDv22GPDZp/reRKC11fXHBM7AsDPP/8c9oYbbggAuPDCC6NNT63QzaElOnT8PEEw++yzR1sWY9V5GD9+fKJvQLJis7o71c1INEU7E11qCQV1Ga2++uoAknuhXnOp11ldo3pCivtH+6RuBPZby3IceuihYbNEBE/XAclTPUVJ0qmucZ4UAypuHnVd5X0qsylUO9XK9amnlvbdd9+wudfznie95nqvZ5iGurt1LvNEr7nuf54U02t6/PHHh63lRnjabaGFFoq2tddeO2y6qfQ7T6/VIossAiCZEFOTfzbFZW5FxxhjjDGlJdcSEBqMpcpEngG2StqTIwM1geQbPQsBApWnz27dukWbBoby7bKh8enf18DXPAIjG7oWmjNnhx12AACccMIJ0aaKXb3etNLy5Gib5l7p0qVL2Mxz8/bbb0eb5mShksdSCABw7rnnhv3II48ASI5T1Y8s4PyocqG5cbRsCNei9k+LXr7++usAktdKc7Zo6Y4s0WvGPD7MwQQAF110UdhUcYDKuLT/un5ZLkDnlAGiQKUES/fu3af4zCJAdU3fcjXPE4PgVVHM+17aGDhvqijovHMudP70/lurYP9aoteftuam0jw5eR7C0T2nee7oUVCVRVUcDUzm/GgeI71X8v8b2lPVcgc1ZS1b0THGGGNMafGDjjHGGGNKSy55dChNqrQ+ZsyYsCdMmAAg/zw6Kn2PGzcOQDJfg7oD1I3DYGOV5hqSU/Vv/fTTTwCSAciac0fdfFlK0nr9tX/33HMPgKR0zABVoJJHp3379tFWLzlZ+8kq4kAl2E/z2agLRn/vtddeA5AM4F1sscXCHjZsGABg6aWXjjYNTOfvZ+2uUphTRcehuZ3U9cTAQM1zoSUqGMyqFYfVNTfnnHMCyD7AWgPH6Wbs1atXtKmMrvuA111/X6/FmWeeCSCZJ+m4444Le6+99pqiX3m7fnQtMUU+q6wDwOabbx62Bn6StDxmRUcDzJ9++umwGYTMUiRA/jlzGgPnQvdqUdC1oXm0eIBBg/YbKrGha66he4Veiy+//BJAMneOls1pynUr3pU2xhhjjKkRftAxxhhjTGnJ5dQV5TGVozSPCVNAd+3atb4dQ1K6U3fL4MGDAVTcSgBw9NFHhz3vvPOGTZmuMXJ3mutEc6IwtwBQv3Tu2qePP/44bI5b+8HcFkDlBEQ9Tz+klQjQPDgskaBlGdJOwgHAHHPMASCZs0XTlTMPhv6t+++/P2zmzMhamk47YaW5U3SsmjOIpxnUHaLlBNhvbeM1AbJ142if9NQbT4Kdcsop0aZ7Vd2ULOHAUgFAcq0yT5em5d9nn32m+NwiuUP0mrPfTKUPJMc6evRoAEl3ctHdVboXWclb95+6PpiTa+aZZ462Is1VS0bXGUMwAGCuueYCkCzV1NT7QFpOus8//zxsruv77rsv2jbZZJOw9QTytGJFxxhjjDGlJVdFR4MdVSkZO3YsgPSguqypVtSMioa+2W688cZh69NptfP/f0V/RwtMsoCaZojWYFJVCmodEKqfrcGcmjOBfdXMsfqUPa3jryXst77ZDxkyJGzO68svvxxta6yxRupn8Y1ZsyFrTg9mIdZCdXwLBSrzl1ZIspboGxXVF81dpH3WorJ8e9bxax4dKqqaQVk/K0t1QMeUFoCogcK6PzSzLBVFVTxUvWEQ80477RRtnTt3DjvPnDlphzaAygENoHJdNHeV5hHiXtVgXn07L4r6ofc/zczNwrRU8YCk4saDGWm5aUzz0LWva2q11VYD0DiVvtphFn6GZoYeOnRo2MyCrV4SVaR130/rWraiY4wxxpjS4gcdY4wxxpSWXEtAqGtAZS7KtNVyP+QhU1JmrdaPxpRzSAuc1TwmzG+iJQpURswiyJd90nFcccUVYavr45prrgGQdH3knXad/dYcQ1r0jy4rzYM0cODAsPX6Mgic+RwA4KmnngqbOVk0QFnLEdBNUs9rQslZ3cEMoAcq7jagspbTypYAlbGojF2vooP6NzV3EQNTH3zwwWjTQpubbbZZ2MsttxyA5FrQci0siquuraK4PlTi/+WXX8J+4YUXwmbRWQ3gXXfddcNmuZU77rgj2tT1nCc6PuZrApLrjy6NlVZaKdq0KDDztxSpLEdDpLlxihggnpaPCqiEI2hxVc0jlvZdrfc/5n4CKvel66+/forP18/V3F3rr79+2E2Zdys6xhhjjCktftAxxhhjTGnJ1XW15JJLRptWt6abQeVyTTddj74BSemOMrme6tGKx5TL//p7U/sbmpb+0ksvDXudddYBkHSzVMv5Uisop+pJB81Ds8suu4S99dZbAyjWqQf+fc39su+++4ZNN5XmZunXr1/YepqOJ0BUIlXXCF16G220UbTpSYA83XjV0q6n5aTSn1U3Vtpc5iGz69/kSSmdB91naa5hLaGirkWWgCj6SaRRo0aFreuT9wI9FaclaHhqdc8998yqm42Gc6Jr8uabbw77yiuvDJt76Ywzzog2Ld2St5t8WtE1qW46zpu6a9Sl19D+y+Ney/5rbiN1XWnOJubE0VOvLBsEVNzoOib9rGuvvRZA8vmguW5KKzrGGGOMKS25Kjqq0mjmUmax1Sfetm3bTvH7Wfbtr33aYIMNAAAjRoyINg3205w6PP+vuYHeeeedsBlQqW8x22+/fdgMnNRgy6wLKFLF0ADohRZaKGwtsMbrUqRgQM6bjklVKAYm61uk9n/hhRcOu1u3bgAqxSuBZEbQNMWuSNcijbT+qUq44oorhs19V6RgSSou2idVYdLenllcFkgGNvfs2TOzfjaXatmeX3nllSl+lsVlAeDxxx8Pm0qO5gnKQwVIU9lUeTr++OPD1nv9McccA6CSuwUojuLWENXysGnGcgaWa2ZnVXSoZOlhhwEDBoTN78Ks51TvcyuvvDKAStZtANh5553D7tKlS9jMs6ZFQXX+eEhAvzP79+8fNpWcWo7Pio4xxhhjSosfdIwxxhhTWnJxXREWPwSA3r17h003T97SucptDDyde+65o02DHbfYYouwmZNl3Lhx0fbhhx+GTclSc0NoYCyvS9bukLRCom+++Wa0aYpuzc9SZBm5mtxJ1wwl2Kn9Hu1qwdZp/9/S0XILHTp0AFDdNVREVGZnzqcnn3wy2q677rqw08ZXFHTPM+gfAHr06BE2A6vXW2+9aNMU+nQJqLtfP7dec6l/h2VFTjzxxGj7/vvvwz7uuOPCZs6fehUvzgq9P6hrhqVj1N2YVnqnKaUOmku10A26GbUUzFVXXRU2vz+A9JxsLAoKAHvvvTeA5DXhngSyCTa3omOMMcaY0uIHHWOMMcaUllZTk9///PPPTLV5lTY1JwRtza1TS6abbrpW///vTPP4KC2+++670aa5ObQqNq+pyumbb7552DwNpLlL9Geb67Jq06ZNKwCYPHnyVMen15/p5rUUhZZI0FNHeZ8wat26dSsAmDhxYnl8R0K7du3qMj6d/7RTL1oC5MYbbwx72223BdB0iZnja8z+S0PlfpbtACouHd1fWjqCp82ycj1O6/5Lo1r1crozgMqpMs0ZpTbJyvXI/dfQ90Pa/V3LWuj49AQSXSZ5uaua8v2Qho5f7+8sQcJ//wrnMm1OgeZfF67PxoyPc6KlZPbYY4/Un+VpYc1jpjlxOnfuDKDh3EFNheNTrOgYY4wxprT4QccYY4wxpSVX11WiIynJpbKSLpsjTTZU3qEx6PiykO4aI53zmqucmFX/motdV7VH5/3bb78FkHRXbbrppmEz+V5T92etXFd6z3j22WfDfuihhwAkKx6zrEo9aI7rqhpp98e8SrBMq+tKSbu/KEW619TKdVWNhqqXN3Tqs7k0xXVF0tZhY8n61KpdV8YYY4z5W1EYRaeeZP3EnjdZvFEWCSs62cI3zmoB8s1VWmul6CjaV9raz3oWgvy77D9/P7RMmqPotASs6BhjjDHmb4UfdIwxxhhTWqbqujLGGGOMaclY0THGGGNMaZlqUc+yB5uVPVhwwoQJdRsfs80ClWODWWVQnn766f8WwXRlX59ln7+yB8tPmjSplONr27Zt3e+f9eTvcv9UrOgYY4wxprRMVdEpKmlJp/Kqi2L+j+uuuy7sBRdcEACwxhpr5NSb2tBQcqwyrTkdn+P2TJFJS7ina7ZM+9LUBis6xhhjjCktLUbRURVnwoQJAIBJkyZFW/v27VN/Nk+0H/r2wTeOlvrmzLH8+9//jraDDjoo7FVWWQUAcOedd0abVuIt+rg5b9rPH374IWzO36yzzhpttawOXS90fD///HPY3Eu1LHdSdOpZgiYPdC45Pt5HgeRa0GvBqtV5o/1nJfdff/012tq0aRN2hw4dALTMPWmyoRhPBMYYY4wxGeAHHWOMMcaUlkK7rtT1M3HixLDpJlF3glZabtu2bdh5uEkombIKNAD89ttvYc8///wAknJs0d05KgPz2Pi1114bbSqDr7DCCgCScnLRx5fmGh0+fHi0HXnkkWH369cPAHD66adHm85lS3F5aP2nQw89NOxTTjkFALDAAgtEW0sZU2PQNf3HH3+E/fvvvwOouEBaGlyLuudefvnlsN955x0AwOOPPx5tGgYw99xzh33mmWdO8Vn1Qv/mzTffHPagQYMAAKNHj462448/Pmzu1bzvP9UOM9Cu5lrj/bVI98w0d3618dUqdKSW9fWs6BhjjDGmtPhBxxhjjDGlpZCuqzRJT0/w3HTTTQCACy+8MNratWsXdh4yu8p1lL4PO+ywaFM3G/s/00wzpf4+ySqzcHMZOXIkAGDUqFHRNvvss4e9zTbbAEie2Cii66Oa9DpkyBAAFYkcAO65556wmSdITyrNOOOMYU8//fQAkmMukgydxgcffBD2J598AgDo0qVLtBVx/pqLrs+nn3467NNOOw0AcO+990ZbWubvIvHjjz+GzbX61FNPRZuOhaeVdM+q6+6cc84Jm/sijzGrO1jDAJ5//nkAlX0GpJ+KzCM3lLqDeToMAN57772w6TJ84403oq1jx45h9+/fH0By/6WR9Zh0LJ9++imAZFiInnobP3582PxeqObCaqjfnHdeBwDo2rVr2E1xjVnRMcYYY0xpKaSiwye6sWPHRtshhxwS9vLLLw8A2Gqrrerbsamgb7zMQzLLLLNEm74xpwULfvPNN2Hz7VHztOQdTHfNNdeEffLJJwNIBvudf/75Ya+22moAiq8C6Bu9ju/cc88FADz66KPRxgByoPLGr4qW5nEaOHAggOT6LFLgOd+IqNwAwKuvvho230SzzkOS9vn1uDacC92TDLoFKoG7ec9TNdj/+++/P9r222+/sKluUHkEgM022yzsPn36AABWXXXVaNO9WhQlVv/2hx9+GDb7p8Hi+safdn/NGu6pZ555Jtp69eoVtvaF9wpVTPSwDdXFddZZJ9pWX331sLfbbjsA2aiMep966623wt5ll12m+FlVtFW9Ynu1AxpUv/UAS1ofXnzxxWi75ZZbwl5ooYXCnlavhxUdY4wxxpQWP+gYY4wxprQUxnWVFnhGFwGQlMkGDBgAAJh33nmjLe/A3bTAt++//z7all122bAZhKzSpeakee655wAk5bp6uT707zDoD0jOBWXWs846K9p69uwZ9rTK3Xm4BjSQTaXZ/fffP2wGIy+22GLRpmPiteA6BJLlMA4++GAAwFdffRVtzL0D5F9AM62Eh0rn9Sr9oH+T+1eDumt5bdJych111FHRpoG7e++9N4BilcDQvjAIdMSIEdGma61Hjx4AgLPPPjvaVO7nfUfXdJHGyvWp90ctGky23XbbsPfdd9+w6dKp597i3/riiy+iTYN1s0a4pgAAIABJREFUjz322LB5r9T+ac6uyy+/HEAl3xGQPAzB0ADNc5XFWNUdxbWiIRaae2m55ZYLm+N7/fXXo23RRRedwtYA8jFjxoTNg0carK2udf0su66MMcYY87fHDzrGGGOMKS25uq7Scs8AwIknngggmfZ7o402msIu0qkIHQtl5M8++yzaNKdOWp4HleAYuV9PaZl90XwPJ510Utiap4Onknbfffdo07mYPHkygGTZC7YBwMwzzwwgeWora3cO50cj/Q844ICwN9lkk7B5WkrnJC31+ZxzzhltO+ywQ9idOnUCkDx1sfnmm4et+TFUnq8X7L+6O9RlxNOCWe8vXR/c/5pbqrnuaN2TKrOfeuqpAJKn5tS1fPTRRwPIv6yH7pknnngibJbA0eunbhy6lPXUlebJKTqcNz1VpvuW+0tPmum9JI8wBvaZ5W8AYIYZZghbT/ixxIaW2tD739ChQwEk19yBBx4YNk+AZrEm9dqttNJKYdPNrSeh77rrrrDXW2+9sHmvU9e05t/htdL9qWEot99++xT/39yyElZ0jDHGGFNa/KBjjDHGmNKSi+sqLa04I80B4OqrrwaQjCrXhF5MXZ6H7F8NldYouXbv3j3aVAbk+JlWG0ieoLjqqqsAJFOc67Wq5lJpDryWTAYIAC+99FLYdFcBwF577QUgKe1//PHHYZ933nkAgBdeeCHaVJpdZpllAACLL754tKkbbI455gi7VvIs/75G96s7QN2MdONUk8B5zatV1+UJBHUHPfDAA2HrCa880YRjesLip59+ApB9wsDZZpstbO7pWroddE/qXhs2bBiA5Pj0BF3nzp0BJF1HWcO+8NoDyVNh6ibgCaOdd9452pZeeumwuT6LdH9sDFyXuj/VDUL03pd3GAPXrd7TtESRJrxlQkqGaADJ0A1+lrq2mCQQqNx3s3an6v5huMHaa68dbbrm1E3H/un9T09Nv/vuuwCSyRV1fdPNquPXk1ZNGbcVHWOMMcaUllwUHT4pvvnmm9GmT78MLNM3GlVEivKmooqG5k+44YYbAABXXHFF6s8yF4EGKOsbNf+fxT+B5Jsui2YClWKmzX2j4TV98skno01VFrU5Fr6ZAMDhhx8eNvPvaAkMDTbj33jooYeiTYOgGSwKVN76mzK+tGDv2267Ldo0t4iWcMji7XDcuHFhq1LAvZDHG+kvv/wSdt5vxLX8+5x3DWC9+OKLw2YQrwZQaomEPAPEdU4efvjhsDWYmD+rgdu6pnj/zDu3WGNQ9YD5U/RepGUpqC6oIlkU9J7Tu3fvsJdaaqmwmYdLg6m1nAXRsgtUGYF8AuO5P3VPff3112HrfZX5cXROVfF/5ZVXACRz8uj+p3qj35+q6DRlf1rRMcYYY0xp8YOOMcYYY0pL3bQ/dd1Q8urfv/8UbUBF8ttzzz2jLW9pneg4VE7WYOn3338fAHDllVdG20cffRQ2XT76+yrNMWeBBrZ17NgxbJUu0wK7pxWVWZlTRaXJjTfeOGwNjGY6bs1DwwAzANhtt90AVEohAMnq3+y/5nHRwEoN0hs0aNA0j2daULlUpWN1XTVXGuZcMMAWSAYTas6Pega8/hV1bWjgH2XyolefV3QtUzK/5JJLok2r0/OaawkTdR3k4fLhtVYX73HHHRe2unN5MEDdcVrp+vjjjwdQKRVQVHTOdB/QZfH2229Hm953d9ppJwDJiuVF+X5QtM+rrLJK2Dx4ctFFF0UbD3AAlXAEzY2kbqA8XZJaquiEE04IW+eP3yE6JxqsTJejXh917fF7c4klloi25rqTregYY4wxprT4QccYY4wxpSVT15VKU3qa4IgjjgAAPPvss9Gm0tUxxxwDICnx5yGjp6Wg1oqrZ5xxRtjqYqH0qLlZllxyybCZX0ZPVTG3DFDJP6CnCqrl0anVdWGfFR2/ziVPQzB6HqikzQcq86fuLu0nP1fzLOipOl0XzUGlcZ6Ke/zxx6NNXW/NlYb1+vAEmVav1xNeRXQJzTrrrGHzVEsR3QGKzq/ajz32GICk61hdk3Qjqju1KGPVddS3b9+w1Q3F9fXhhx9G2+mnnx42T5Pp/UlPRRZl/VW7p7H/WrFcvwvoOlc3e1ZV75tDtfGxr/PMM0+06frld6W6K5lbDqi4gfKozq7uQs0Tpzlxdt11VwCV+wiQdB3TDaVlWRZZZJGwWSKnluvUio4xxhhjSkvNFR19MtUntiOPPDJsFuvUzKiaeZdBuHm8eWj/tWjeLbfcAiAZwKgqgOYB2nrrrQEk3xi//fbbsD/55BMAlQy6QPKNk0//WQeq6hsBsxEvv/zy0cZ8D0AySJI5fxZeeOFoY6FBRQN/NRibeXY0T8hTTz0V9oorrjjtg5hGqK5ohk4tNNcYuEb07VvzIDFztGYR1etXxDdqHUtzC+jVi7QDDkAlmF3X3D777BP2BRdcACCpYuY9J2lFe7VPWiyS6qf+v75ds8Ds66+/XvN+1hIds2Zh5/xptuAePXqEfdlll03x+3nPX0PonqK6PHjw4Gjr1q1b2MxZduihh0abBv5SqdP7V73UHe3n6quvHnaauqZj1rGw/a233oq2PfbYI2x6DLSQd3NpGXc0Y4wxxpgm4AcdY4wxxpSWmrmuKOeru4VBqUAymIoyvuae0ZwtlCHrGWxFOU3dVZoinm4Yzb3CoCsgmaKb10KlOy1Kx59V6TWLQp2NgX3RMWuA46abbhp2Wu4enWsGbKscrdeVkrS6CddYY42wTznllKYN4i9o/xhEp0HB3333XervpRWzTGvTADx13c0333wAkutbA8uLIrNrP7SQapGDkXVP6QEHDbZl4VZ1d+hcMEg+7xIJuieYh0rXiQaIp82F3j/04APLJWiAblHQfaT3RC0nw3utln3QwwqcvzxzUDWWtHIIWhZBC/3S9ahzqsWW+V2p7sosaOg+qK4zdQOnrVXNKcZrseyyy0ab5pFjfj1dE9UKXE8rVnSMMcYYU1r8oGOMMcaY0lIz1xXlKHVXaLpoPVXE1Oa9evWKNpWj8pDM+TdVmlN3BF06Cy644BS/A6S7I/T/NcU8JT+VXvOIoFc4bsqmQLK6+GuvvRY2JX+9VqzYDlRkTM19oXmCGK2/5ZZbRpvmZ1A3Sq3cPLy+WvldT5Wtu+66YdN1o66NX3/9NWy6QTQFv1ai5/pWuTVvN0kaOn8//vhj2HQpaNr2vN1Y7Kuuh1tvvTXse+65J2y6DnVOVDrPozo5UReG5lnq06cPgMqJHCBZlkPdOGkuBXWD0CWmrue854+ou42lcoBkOQQ9bUWYWwUojuu3Meic0SWlJzU1ZICsv/76YWueMrpmNZygVuh3kp5Q5bypO7Va9fi00BMdP/+f+aCA5AnB5pQ1qoYVHWOMMcaUlmYpOvqUxrek66+/Ptr0LVgzH/fs2XOK38/7KZ1Pj/oWrkVFifazoSdO/Vl9o6S6oUU98yYt86UqOiy0B1SevnX+9El/rbXWApDMdqmfy7cDfbtrSB1rCml5YjSfw+233x625ilhMNzYsWOj7Y477giba5n5gABg6aWXDpvXpYgqDlC5Losttli0DR8+PGwWld18882jLU1FqCecP30L1LdczcJKdUeL4hZlLnRtzzXXXGFTSdU1x3xbQHLcVDxUmerUqVPYzM+i85e3opN2WOXNN98MW9UN/szKK68cbZpzLO+xNAXtM4PodU+lqSNa9FjJIs8VP1Oz9ev+okdCg6Y322yzsHUss8wyC4Dkd4Jm0X/ggQcAJO+pquIdcsghAGrr5bCiY4wxxpjS4gcdY4wxxpSWmgUjU5IdP358tGkeB80DQEkrz6DAalQrxNZc0orWabBuUeRY7YcGQA4YMCDshtwYacFoaeOrp7uS179jx47Rduedd4ateVZYIkCDQVXG3XfffQEkS5hkUWg1K9i/LbbYItrOOeecsEeOHAkgGeyoayHPtaqyvebeWHXVVcNm4Kqu06LsL0WleQaw6zrSsjFazoIhAeoGUtcwXcZFvL/q3tCyHRrmwJw5N954Y7RpAcyiuCEbg46bwfIzzTRTtGnuMIY5aFkcPRjA0jK6F2p1z/npp5/CVnciDyjo97iWQ9I5WWKJJQAk8+TowQHOtc6p3n9ZDqJaaENTsKJjjDHGmNLiBx1jjDHGlJZWU5OE/vzzz0brRZrWXKVVRmIXgemmm64VAEyePDkXPZuSY1a5g1q3bt0KACZMmFA8vb4GTD/99K0A4I8//mjW+FQa1RNwkyZNmuL/9TRe1qeq2rRpU/f1qa4Duqk0rXst4fpszPylXXMtIaBuYPY/L3cV52/ixInN6oC63tKqmyvqusjaddquXbtWADBp0qRGjy/tpC6QdJnQTaOhD/V0B7dt27bm98+0E8bqzjnppJPCZn4lvScdccQRYdPN2VTX7NTun/qd/fjjj4f9yCOPAEie/tMwFV2f3Jd0YQHJU9fcw6zSDgBLLrlk2M3No8P9p1jRMcYYY0xp8YOOMcYYY0pLzV1XGgleLSFg3icg8nZdZY1dV41H12raqbJ6lijJw3Wl+5bjy2qcTXFdkWrzlHcJGaVWrqui0hzXlaLzl7b+8jq9mIXrKg0dv7ru6B5Sd/Kaa66p/QPQ9HU+tftnQ/dBJjv8q532Xa+nUtUNmUYt59quK2OMMcb8rahZHh1S9BwixqRRJEUgD1rKvv27z1OZyCpnWUtBx6+HdTQnFNHrk+W6b2h/ae4fLbvS0GflfX+xomOMMcaY0uIHHWOMMcaUlqkGIxtjjDHGtGSmGqNTy1MtRcKnIlo2PBVR9vVZ9vGVff+V/VRn2cdX9vVZ9u8Hxa4rY4wxxpSWmp+6agzVzuznHaGdBQ3lJyjjmKvlVCJFyq3UENVyfqSRdx6QWtFQTqyiz9nflbS9lvb/1ebP85otvP4NlXAoap6ologVHWOMMcaUllwUnbQCfVrAjOfz9f9bYp4FLXSm/WfhUy1qpzkJGsrMW3RYVFEze1511VVh33PPPQCAYcOGRVunTp3CLooSklaoDgCefPJJAMDgwYOjTeeMRfeWX375aCv6/KUpjpdeemnq//fr1w9A8voUfXxKWoHMlnR/odKmipvuGd5X9P6i/8+1rEVbp5tuulTb1Ia0udLMwjoXXJ86f8yGrJ9VlPtkS8CKjjHGGGNKix90jDHGGFNa6qZRqnRHmfjyyy+PtltvvTXs/fbbDwCw1lprRdvCCy8ctkp6RYRj/fTTT6Pt9ttvD5uuGy3k9tBDD4Xdvn17ABUXkH4mUBzJslrQ4xtvvAEAOOKII6Lt+eefD5vj+uCDD6Ktc+fOWXRxmklz3bz77rvRduqpp4Y9cuRIAMDkyZOjTe2uXbsCAJZbbrnUzy+Km6fa/H377bcAgGeffTbaHnnkkbCZrr53796pn1Wv8VXrf1qwp9qff/45AGDChAnR1qVLl7CLMj+K7v9vvvkGAHDfffdFG+cMAN5//30AlXHq7wDAq6++CgDYaKONou2QQw4Je+ONNw67iNeipVDtgM31118PIHlP0Ws+xxxzAADGjBkTbRtssEHY3Hfzzjtv6ueXkYaK+Ta0Tq3oGGOMMaa0+EHHGGOMMaUlU9eVyq0q7f/zn/8EAFx22WXRNsMMM4Q9aNCgKf5f3VwrrrgigGKdlNCxfvzxxwCAAw88MNr+/e9/h92mTRsAyTGfccYZYf/nP/8BAMw222zR1qdPn7AXWmihsPO4BpQOdcyUw4HKuLVNx0LX43vvvRdt66+/fjadnUb0JA5PVe2xxx7R9t1334XN9t133z3avv/++7CXXHJJAMV0VwGVfumYX3vttbAPP/xwAMDrr78ebZMmTQqbJ0Tqmfsq7aRRtVOZdAnr+nrsscfCvuuuuwAk19zZZ5+d+ll5otdX3WwDBw4EUFmnALD00kuHvfbaawOo3CeB5Prj/WfxxRePtsUWWyz1Z2tNNRdEY/Jspc1P2km6Iu2/iy66KOzTTz8dALDLLrtE2+OPPx72mmuuCSD5/aFurgcffBAAcNttt0WbVj9v6W6stO8Xvf+OHz8+bJ7WnX/++af6mVZ0jDHGGFNaMlV09Cn6jjvuCPvFF18EALz00kvR1qFDh7BnnXVWAMBTTz0VbUcffXTYN9xwAwCgY8eO0ZansgEkx3rjjTcCAB5++OFom2mmmab42d9//z3aGKAGVHIKabDZrrvumvp360Xa27u+Ue6zzz5h//DDDwCAfffdN9pWWGGFsKle3X333dHWq1evsBmMrX8rC/QtUAP/eK11fekb/9Zbbw0AmH766aMtLXNpkd6s0t6eVbFR9erDDz8EkNyT22+/fdjdu3fPrJ+KvtEx98tbb70Vbffee2/YQ4cODZvq0yKLLBJtqoIyz9Emm2wSbUWaK6L5bM4777ywuS7HjRsXbZqHhddtWjMkA8n125xroZ+Tlk1c86VpHpkff/wxbO5L/X745JNPwn766acBVJRvILkm2QdVTOaZZ57GDqVJ6JyNHTs2bFVktttuOwDJOb3lllvCXm211QAkVbrVV1897L59+wIADjvssGi74oorwqZiBxRzXaeR9v3CeQaA0047LWzNKbTpppsCSAbTp2FFxxhjjDGlxQ86xhhjjCktNXddqdysAZrnn39+2BdffDEAYNFFF422tNw4Ktep9EyZc84556xBj5tONZmWMv9XX30VbZozh9KilkjQYDR+1o477hhtSy21VNj1yiOkc6ljve666wAAJ5xwQurv7bbbbgCS0qxy//33AwBefvnlaPvss8/CZjAvUHvpVcfEUhxAMucP52fEiBHRpoGbdJMWPZ+Tom66t99+G0AymPqjjz4Km+tPXY8MUAYqLrssZHGdH5X+zzzzTADJ3D6rrLJK2EcddVTYvK9oCQ66w4GKe0Hd3XkHq5K0eQKA4cOHh808XOoO17U4rblFskDvg9onurk1X5qOT12S/Awtu6KHWXiNqrme6bpkPiGg8p0DJF07tbpG7LOOWe8fs88+e9jHHHMMgKS7UXNSER2/ftfxvqp55hguAST3dUMFXLMgrehx2r1Cf06D7fmsMGTIkGg755xzwlaX9Nxzzw0ged11fuNvTVPPjTHGGGNaIH7QMcYYY0xpqbnrSqVLddfoOX9KyipHqrRGSVIj2DWPB6P18zh9VA3tP0sAqOtGx8pTLZdcckm0qfTG0y477bRTNp2dRlTap7sKqLis9NRYjx49wqabQedPy1lQhtWTFLwmQPK0Qa1Ik5bPPffcsPWEx5VXXgkgmVukiG6OhtD9oW465vRQaV+l9b333htA0l2lOZ+yOOFIGVv3+UknnTTF33/uueeiTfusJ+D4WTrX2uciuxx1ztQdscwyy4StblSi+4tjLdKJG7oTdB3q/WPAgAFh8/5ZLTSAn/XKK69E26WXXho2f09dl1m4qxSuuS+++CLa1E3Xs2fPsHlf0XXYUM4fXb8LLrgggORJXD1BqfOe5kbKAl1/dFOq605dhzw1pafuDjrooLCfeeYZAMmTaFwTAPDll1+GzdN0LgFhjDHGmL8tftAxxhhjTGnJNGEgE8cByURRlBE1al6lu59//hlApRQEkEwoxUrmRUnV/lcoo2n14KuvvjpsVjLXtNbq2jv55JMBJKPL6yVDq8TLVONA5aQAULnuCyywQLSpG4gJ/7TPOv+UrPX/1WWRBVxfWopD5VR109BlWC0FfUtBXYcsuwJUJOH55psv2o4//viwt9lmGwDZu6vS0Guue54nGTWJo64p3Wv8PZ7IAJJuLv6Nansqz9NKOiY9tak2S+Po/GlyR55a1CR59XLX6bXT9bfOOusAANZdd90mfW7aaa6vv/462vTUDv+Wnj6q1sfmhD+kuZtGjRoVbbp+999//9S/P7W2av/P68rEg0CyRJKGSfBkVxZrWudXT73RTaenZ9WFxucCdVep6+3OO++c4vfVJajfi9M6Lis6xhhjjCktNVd09C1JA+gUPsltuOGG0aZvjzwzv8QSS0SbKiJUP+oZbJf25N/Q0+RNN90U9gUXXBA2FQ/9TM2pwPwIqnhl/UbNv6Uqk+Y+0jdNvinrW4TmKUmbFw0i5dO/5kbSua7VvOr15VuO5iPRQnBaQC8tD0laOvtqsP95KAO6Zt55552wNbCPY1F1ZI011gh75plnBpC/YqoBxkyhTzUUSM6DBrby7U8VHRZKBIBll10WQFJd4JiByjWsp6LHz9egzp133jnsd999N+w333wTQLJEjpbDYLCqHnZIK5qbxZiqKSTNze2j65rfBZo7Sd/yOW79TtG5zOIQC1UmLUS95ZZbhq3qWq3ub1ooWXNOpRXgrRU6DyzlBCSDrffaay8AlVIrQDKnHssFaR455lYDKkqlfudoHiFVkqzoGGOMMeZvjx90jDHGGFNaau66UilJpUPNc7DiiisCAL799ttoU4mN+RE23njjaNNg3Xq5BNIkTg30UplZoTSpac0b6rPmBmCQnbp2siDNtaNVcJ9//vmwVdpnTh1NwZ+Wol2lyQsvvDDsTz/9FEAyWLBLly5h18plouOjdHrXXXdFm0qrGqzKuVLXiAZzslyClk3QSvMMlk/LbZIVHKvK4louQV07DDjXFPxaPf7II48EAGy11VbRphWDs3QZa1Cp5jbiNdU+acVqlfEp3b/33nvRpq5TVjpX18dGG20U9tFHHw0A6Ny5c7TVK4+Sfna3bt3CVmk/La2/rkW6Z9WdsOeee4Z9+umnA0jOaRGD7XX/PfTQQ2Ezv5IenOCcARU3VtahDdo/uonVRaO5xbKoKK731znmmCNsde3UCo5VvxM0GHqHHXYIm25mdT1ppXXu8ZEjR0bbjDPOGPa4ceMAJPOYVStHNM39b/RvGGOMMca0EPygY4wxxpjSkmkJCHVnqEuE0rjm9lBpavTo0QCAbbfdNtq0ejdzaujn1wrtv8rVlL71JJWe1NH+UZrU3Anrrbde2Oz3GWecEW2dOnUKW08DZYlG0FMaVheT/r/mmaHLSt05+rN0HVDCBIBHHnkkbLrBVG7OQtrVuWQeEnWNaB4SlXs575q7QXPuMM+Dugv0VAVdBro+snDNpaFzwn0EJMdNmVjXnOYk4amIgw8+ONrUzUeXRy3dHfwslbCHDRsWNtfcXHPNFW3V3El0GdJF/lebbhx1bem632yzzQBUXDxA0k2W5VrVUij62XTdabuubz11dNpppwFInqRTlx/Hv9xyy0Vb3mUxdP74XaAlhDSPF/PT6PpMK5dTq3w5SrVSDXQXqrtTq4vrzzZl3+jf5fyzij2QPLVaK5dkWgmZAw44INo09ED3Cn/20EMPjTa6o4DKvtYSNAxnAIAVVlhhir/f3HuNFR1jjDHGlJaaKzr6ZsygPwB4+eWXw2bRLlUB9Ilt7bXXBpB8y9ICd3y70gDmWr1Zaf9feOGFsHfbbTcAyWAzzfx87bXXhs23Ug0m1CyP11xzDYBksJb+P3PSZJ3Nkvk4gIrKpn3SYMZ+/fqFnRYMqXkcjj32WADJPB8zzTRT2DfffDOApKKShTqna4LqhSoCOr9bb7112FQKNFuwKnnMSbLBBhtE2/jx48NmLg3NY6PqRFqW3lqh87vSSiuFrXl0GLg7ZMiQaNMCq/vttx8AYPDgwdG28sorh02ltZZ5Zvj7qpZoHhLOSWPWSTXljIck+OYIJK8FbQ3g1cDlgQMHhk31obnjZ1+5N4Bknp9FF1007LQ1k9amv8/cXUBFcS1SUeQ0xULvqRo4T6UkbR6AyrXMenyqglGd0O80tZuLfhazCF9//fXRpvl7mpJnpiE4J5pnTQ/QaF+oNN19993RporkWWedBSC5Pnv37h0212otv/+s6BhjjDGmtPhBxxhjjDGlpWauK0qHWshM07VruQAGC1YLgKNMpmnLNRiUOUE0RXwW/Oc//wmbOQvUBfPwww+HrePr2rUrgKTMrqUHnnjiCQDJ3AdHHHFE2PwbWQcIquuGgZkqMdIFBSRl4FdffRVAJZ8OkHTNUBpnUCeQDGZl4HbW41Ppk+4albs1gE5lWMrE9913X7SpG5XBuhr0p2ulf//+ACouSiC5F+gaqiUcq0rcGgCv7rLPPvsMQDKnkAZ70mWjJQTuuOOOsJnfSvNk1Upm1s/Jwp0JpBf11OvG+dFCmgzwBYA+ffqEzevalABz3VO8b+qa0wDbxlxf3os12FrHyvtm3rlz0gLIgYpL/bzzzos2DVKnG1FLmGRd4oFon9XNyrIiGg7wyy+/hN2UPHC6JtU1ztIg6lrVYPnmBj6n/S73uh4quuGGG8Lu27dv2KuuuiqA5HeC5oRbeumlASTdqdX+bq2womOMMcaY0uIHHWOMMcaUlpq5riizffDBB9Gm5+T1VBGlxWpR6TzZpHl2tJIvTyVlkeI7zd2hf1NdFOq6OOWUU8JOc8mozMlx66kSzYOQRZ4VXnOtbPvaa6+FTelYTy+ce+65YatLkvlZNPeK5vGg5K4ntfRa8vrkIZ2zsi6QlLhHjBgRNl1zuj711CBlaC2BoacRPv/8cwBJaTbrch5E147m7knLmaMnJTR/C/MDVStxUqTTOrVC7yU8tcJ8XUAy51UWLjWuFU3rr24szcmUluJf9xLvnwcddFC0bbHFFmFzXWRdIiGNtHw5QDLP00UXXQQg6U7WE5B0yVZz0dRrferf3GabbQAkQyzUNaw51dL6l3YvfOCBB8I+5JBDwt5kk00AJE8CNvRZzYV91tOpes/75ptvwr7yyisBpOd+Urue938rOsYYY4wpLTVTdPiUpgG2qn4wdw5QyVLKDIpAMnMug5y0UJgGcPJJMIsnQn0j1syhDEbUbJz6ZK6BmfwMzUmjb9RpeQRqmQUyDX6mqlCaJ2fUqFEAkoqcFiXVty+qcxqAplkwqQTomIqSeVVzL6nipFk+mctEs3mqonPvvfcCSK7viRMnhk3FR4OdNbBHbqd1AAAgAElEQVS+Xm/Sev01sJhBnrfddlu0aWAh51qzvGqWXc5v3sGstSRtrWruoU033TRsLXBbq2BPBghr5lndUxoYTVWyWlFP5sTSgxOqzjKwN+tCs2lUU1u0aCkPOayzzjrRpooW15/2Pw+VUfcx74l6T+U8AMC///3vsKnu6nel5mfjIRE9LKI5u0444QQAycM4Wd9fqW5roV3Nc6SHNagYZnWYoClY0THGGGNMafGDjjHGGGNKS6upSa9//PHHNOuyadLhs88+GzblNqAiPWoAq9osQKfBTEpzJfM2bdq0AoCJEydO9YN0THRDaaFKlfs1ZwLzCKjczdwjQCWPQC1T6Cvt2rVrBQCTJk2a4kPTUq0DlcBxLVugaGAu8+AssMAC0ZbW/6xcG23btm0FNG59NoRel7QgeS33wWulwZIq0y644IIAkq7PxrgmuT5rOT4NYGWxxGpFP9k/3ZNagqC5rqtp3X9ZU821ypxB6qZWN5a6IdNcBtx/kydPbvT9U12gF1xwQdiap4sHA3T/ap4Zuvk1gHXuuecOu7kuq9atWzd6fET3lrpmevXqFTYDsm+99dZo02uetcuN42vM+qS7Vw9oaGCyrh8tYJkGS1zo/G244YZhM3C9qS7wqX0/KGl5nnRP6MEO7V8eQe4Kvx8UKzrGGGOMKS1+0DHGGGNMaamZ6yo+UOQutfWEFW2VW9WmDJhVJHlTpHOORfukOS9UTuUJLD2JpZJt1hHyTZEm2T89XVUNjjWPUxtANq6rhtDrQrvataJ0q9enMW6eLFxXCvudlo9FUQm6qWNJo16uK13f6lL44osvACT378iRI8N+7rnnAFTS+gOV3C5Acl+nyfRNcV0RXVN6n9D7J91b1coR8NSfflYt3QnNcV1pbqZBgwaFzbIpALDyyisDqFTBBqqXe8iCpriuSFoVdSB5QlNP46bB70L9TtS5bu74p/X7Qb+zrr76agDAXHPNFW2bb755zfpUS+y6MsYYY8zfCj/oGGOMMaa01CxhIKmWlruamyrtZ/NOLpcG+6dyv5Y1SDtV01LGBBSzf0UiLYV5S4X9b0hCb+no+p5zzjlTbdKtW7ew6X6odkIxy/mvVlG9Q4cOqXYaHHcR16n2afHFFw+7a9euYffo0QNAMuFhS0lOqeNTN5bOWUPJDdPmL4/xq+uNCVWzcof+P/bOM1qKqvn627UUMeeAOYuKESMiCiaMiPERfcyKihGzYELMCVExZ8UcUTBgQkVEVMwRUIwYMDwGQP9rvR/etWt2c7u5aXp67rB/X6hV3Jnp0+d0z3RVnV1544iOMcYYY2qWskd0smhJv/5mRFbEyhhTG1TLvaoW7zUaJejQoUPY2gKIrRG0cFnnpKU0la2l+eM5b6njcETHGGOMMTWLf+gYY4wxpmaZoY6OMcYYY0xLxhEdY4wxxtQsMyxGrqTybCWhMmt9ypAtFSpD1vr4vD5bJpy/pijrtgSaoxzcEphZxvfvv//W5PhmnXXWmWL+FEd0jDHGGFOzVGx7uZImyFXOXh7GGFNLZPUQnBEtaXtzWt+4LMG8lrK9vLlkzXmaIG1LJ28hQkd0jDHGGFOzFBLRYdfgX375JXzsuAsA8847L4CkBLrCdgUtUUTKGFMZ6rsnaDuXrE7tlUaf0vUpV7tf//PPP3Vel9Zdfu655w5fWsf1oqMk+vm//fZb2Ly/83sAyP4uqEU4F1nd6zmvaVGwloTOP38TAMAcc8wBoLzja9lnyhhjjDFmBviHjjHGGGNqllxTVxp6mjx5ctgbbbQRAOCLL74I39JLLx32TjvtBADYZZddwvf777+HzU63q666ankP2DQIhlaziuXS/l9hmL3ai86bG85vScWCjRlrNY5Lj7++kPeUKVMAAKNHjw7fkksuGTbvRUWksPTYf/jhh7C7d+8e9vjx4wEkUxt6f+R7dO3aNXx9+vQJe911163zWZVC50nHx3s+APz6668AgOeffz58yyyzTNicl6w1W43rszFwXFdeeWX4rrjiirAfeeQRAMCGG24Yvmq8l9ZXQH/dddeFff/994d93333AQAWWWSR8DV3fI7oGGOMMaZm8Q8dY4wxxtQsuaausnQc+vbtCyAZWv3www/DvvnmmwEA55xzTvg22GCDsNdff30A2To8pjxkpQMYRtRdc2+//XbYn3zySR3fwgsvHPbRRx8NAGjTpk2d9ywKHSt3eGjqIm2nS9bruZuniPWp81RfakKPScfKucja1ai7ldLeq1Jk7UoaOnQoAODTTz8N34QJE8Ju1aoVAODJJ58Mn+7wuf322wEA7du3D1+l1qee559//jlsTd1wLXbo0CF8mgZ69tlnAQDPPPNM+DRNN2jQIABAt27dwlfE+vzpp5/C1tKGueaaC0D96dTPP/887Nlmmy1sPVctBR3r33//DQC46667wsd0nv5t3tozTYXHp98P/fr1C7tjx44AgMUWWyz19Uwtl3MnoCM6xhhjjKlZKhbR0Sf6bbfdFkAyitO5c+ewN998cwDJp8y0J9Vqj+Ko9oPaPO6sJ2aOW8dchHJ01tMXI278d/r/5xO/Rmy+++67sN966y0AwBNPPBG+tPOTN2nnHADGjRsHIFkgd/3114fNJw59zRJLLBH2DTfcAADYZJNNwpel8loueP70Kerbb78Nm9oUeiwsagWAESNGhP3GG28ASEYBFlpoobBPOOEEAMDBBx8cPo3y5Dl/OmcaZevVq1fYLNbU4+BTMlB6+m/dunX4dC4Z8SlCW0bXiW62YJQJAKZNmwYgeXxarDp8+PA676uFy6qvU2mydILqO9f6t7yX9OjRI3wrr7xy2Lz+GBma/nOrEb3/PffccwCSm3U0esexVtOYdH7eeecdAMDee+8dPr0/3HnnnQCS9w/VeeL6LOf4HNExxhhjTM3iHzrGGGOMqVkq1gJCQ8PnnnsuAGC55ZYLn6auSJbsdzWF7IiGXhkmZooGKBUIAsBSSy0FAFhjjTXCx3QJAHTq1AkA8M0334RP00DaLmP22Wdv9rErGmLUYuLjjjsubKY2qIcElOYUAOaZZx4AJb0OIBlav+OOOwAkw/SVknjPSlddddVVYV9++eUAksWgmno9/PDDAQDzzTdf+BguB0rniiFoIBmaLRc6VywAP+WUU8L3yiuvhJ2WGlTZfS3G3WyzzQAA559/fviobQEAp59+euLvgORarlTqSmXjuSaBUupJtbn0WJnS02vykEMOCbtt27YAir/P6Fj1Omca7rTTTgufpra4LrhpA0henywGLXp8Wamr+o6LqWMtQNfU66RJkwAAK620UviKbOuRhY5fU6tMver63nTTTcPmfafoAmSdM21RwfvGFltsEb6//vorbH5/3XLLLeHT1HMe43NExxhjjDE1i3/oGGOMMaZmqVgLCE0DDBkyBEBS56FaNAE0HKey6typobtX9Jg1NHfBBRcASIbmVOdi+eWXB5DUseBOCqAUutTUA8PNAHDkkUeGzZBmc8PQaW0beBxAUrPi0UcfBZDUNtLUDtFdMVOnTg2bu5Eq2ZGY49IQ9sCBA8NWzSamCc8777zw7b777mEzNaKhZU3zvfvuuwCS4y+Xpo6es48//jjsfffdFwDw3nvvhW/ZZZcNW9OMnLe11lorfLprZYEFFgBQSkECyV1le+65JwDgqaeeCt+aa64Zdp7Xr5473TXFYwaAddZZB0AyHan/f8899wBIprv22GOPOu9bn3ZSJdF7zZdffgkAeOCBB8Kn64Lnn/choJQOB0rjKqJ7uX4md+cAyXut7pZKg8eqY1YdnSJ2yzUFPc5Ro0aFzfur6gHpDjO+rppSV5999lnYw4YNAwDce++94dP7C+9Rp556avg0dZymY9ZcHNExxhhjTM3iHzrGGGOMqVlyTV1paEvTMEQloIveAUD0ODR0T7I6sp911llhcweE/m3v3r3DPuywwwAAK664Yvj0XFHIbb311gvf4osvHramIcoFP1+r5xkiB5K7PthVVkXIvvrqq7AZehw7dmz4VHzvoYceApDcNZR3GJZzocKGGvrX83/ggQcCSO7UGTNmTNjcraPCbBq65fzqTqbmrm8e/48//hi+I444ImymAXQdHnTQQWHr+kkL7acdn/o0ZcvXp10feaPHpCmOrbbaKuzbbrsNQDJdrmPm/3P3DlDZtdgUNIzP3UQquHbttdeGzXPEEgGgtJMMKIk/Fr0TSdOs2uJgxx13BAAsuOCC4avGOWkqXIu600rFV3kP5n0EAFZZZZWwq/Fc6L2A9tNPPx0+igADwEUXXQQA2GWXXcKnu4rzWJeO6BhjjDGmZqmYjo4+8fKpVIs5tViQEYEi2h5kwSdqldhXbRkt3OXTrzYyy5LLJzo+aupUUmeG51qf3PUpggVyQKkYVZ+49Pj4xEI9CyA5FkaEimh6yUJioFQUDiTbkQwePBgAcOutt4ZPi8m5VrMK6POYP56riRMnhm/kyJFhsy3DSSedFD4da2OekvhZWkD+4osvhs3Cz7XXXjt8RURk9TO1QP/1118HAPTs2TN8ulYZ/TrxxBPDt8IKK4Rd9L0mDT0mRrK0gF6vJbYrufTSS8On4+e4iy7a1WtC1yfnoiW1cGgMvG/oBgZtMEv9J/1+qUZ0TaomHpt2n3322eHjBg2gVOyvmz3yjqg6omOMMcaYmsU/dIwxxhhTs1Sse/lee+0VNkOrqg3TpUuXsKkNwxQJkJTbLyK0zNDqMcccEz5N5yy55JJhDxgwAACwzTbbhC9N50I7emthLFsQaBpp5513DrupKYkZwbnSOWM6BEjOD/VTtPu1yq2z6652z1adHdW8qBQ8T1ogrDorWhjP+dG2BlrMyZSNhl61GHS77bYDUN6O3jymdu3ahU+LvRnu1nPb1LXB0LrOn7YYYHdhTf0VnbpSnR9qcuy3337h03Ox0047AQCOPvro8GmxfdFFummktS7RY9a1yE0Oxx9/fPg0Dct7seosVeqeqp+jxbhajLrtttsCaNyaqsZ0o6JpbhYbX3jhheHTMg5uKKjGzTpZ6L2OmyA0HaeaW0OHDgWQvGbz1qxyRMcYY4wxNYt/6BhjjDGmZsk1daXhRJWzZlfvDz74IHzcKQGUwuTaEfq6664Lm1158w4xa7iYO4hUG6B9+/ZhaxqEXbs1HDdhwoSwKZGt2gnvv/9+2ExZaWhT2y3kGabV99Y0yMYbb1zH1vFpaPauu+4CUBonkEz5Mc1SRIpAQ8DUEwFK6UKgNO86prR2Jq+99lr4dIcINSP0Nao51Jzj1jlZddVV6/x/OdYG3+Phhx8On+46Y0pT05FFhNb1+tQ0iHYlJ5o65vzpMed9/Hx/Pea0XYcNacuQ1gJA1xpTdpouePzxx8Nm6x2V3c8bHp/uxHz11VfDVs0q3c03/evV1vFrOr+I1Hh96PGz3YPumlSdGbZ70PmvxtScjklbeHA3IL/ngeT98bnnngMAdO7cOXzlTPOnHmvZ39EYY4wxpkrINaKjT+yDBg0Km8VwWozMAmSgpImhOhGHHnpo2C+//DKA5K/EPH7x6i9qPtFmNSp94oknwqZi7ujRo8OnDQR/++03AMD8888fvrQGZ1qgV4TmjH6OziWPRZsqqk7C+eefDyAZ8VKV3rQnsiLQz6/v6VpVsI899lgAySeWM844I+w8I1Z5RSE04sGnbo2iqs4VG5xm6aBUCh3/p59+GjavS+o1ASW1a6AU0fjkk0/CpyrkeZAWhVEV8jnnnBNAtp5IWnQn64mfEWEtNtZzRR2zSkbheKwa2dTxa2E1Gxxro2Rda7wWNaKsOi5ci0UXlesxqyI7dYz0+LUwftFFFwXQ/ChwHuj3nyqL9+rVK2xurNECeF2r/H5Xhf/ddtstbCsjG2OMMcY0Av/QMcYYY0zNUvbUlYa2Pvroo7D/+OOPsFkEyhDl9LCwjCkCIFlMxwZw1PPICw3tsh2Cpmu++eabsC+++OKwGXrT0CPTGQBw2WWXAQA6dOgQPtUkSUvtVJOOAo9PG3lqsTFD0prOUR2kagzJphWBamiZjegA4L777gMAbLjhhuFTzRYWQxYdOq+PNG0WoBRy1rYtmlplmqeI1KPeX5gCBoADDjggbB43m8cCSU0rtnF5/vnnw5d36orHrZsu9t9//7CZztAUrxbVpp1rPRc6l19//TWAZIuBots98Ph1A4CmLh577LGwO3XqlHgNkLw+eX/R1BbbegClpqDaSFrvP3neS/U86/cb0/lAaeOJNuXlMQPVed9IuyeqtpZuXGDrFU1H6Tlns1nVftp+++3DZhqznPPkiI4xxhhjahb/0DHGGGNMzVL21JWGm1g9DiTbHTB1oZX2+jqGZLNSW5VCj4nHqiF81blRHQempHRX1b777hs2w+QaetZwZdG7kdLQHQQMGeuuON2BRE0aTe1UYzhW0ZAzx6o6JKrpxJSpdqfXFiDVPlai6091nm677TYASe0r1VxJ20FUKfSYdaeftqvgNaraLNR2AkqtW5giqSTcMQokW6j0798fQHJMGvrXdiRM3agOC9P5QGnd6v/rrjnqYFVyJyfnTXcvahpP01BMj2hHeW2HwN2cAwcODJ+2RuEOIG2hU6nUv65PvX/cdNNNYa+88soAkt3Ji97BWB8cF3fsAaVWR0AyNcy1rGUe+l3OeVUdJdXh4e8Gp66MMcYYYxpArhEdfYrQp4fDDz8cANC7d+/w6dMjn8623nrr8FFtWN+3iCdK1fNRbQ5Ff52n+Xjc1fjLXdE5mzp1atgXXHABAOCee+4Jn+ooaJElqaZiapJVoMknfi0W1Ogji8k1IlCNUbgs0jRN+BQGABMnTgQAXHnlleFTnZJqKSbXp2fVn2HETYvhNSLHiEiaAm9ecP3vs88+4VPlcEZ6VLvozjvvDFs1w3j+NQqiT8xci4wcAKVrFiip8FbymuQxaZRbz4VqkvFekqUjxrFq00htEKn6bKRSESuNrF199dVhq+bMKaecAiC5AaWl3D90nekGI91sw8LwL7/8Mny6lseMGQMAOO2008KnRep5nAtHdIwxxhhTs/iHjjHGGGNqlllmFNKbNm1as+J9WTodgwcPBgAMHTo0fBp6ZNOvrl27hk+LgJubumrVqtUsADB16tTqy6eUgdlnn70s49N026WXXho20xza4kHTWJTezyscy/E1d31q6kNDzizWVJ0kDbNyLeZVzJn3+mSaR5tf7rnnnmEzpaCNTnUtNHesnL9//vmn0W+kc6ah82233TZsLeglqv/BwmrdTKCpoeaOb7bZZsscnx6/FlBzLrSAeMSIEWFr4TLXnab7NY3KsVBqH0imsUhDGoimMaPxZVFfU9P6WrAoPIeq3aWFy+pvChzfv//+2+Dx8frQdGS3bt3C1tQwi5DzbmSZxayzztro+UvT0WGrByB5r2BrlbXWWit8q622WtjcJMBWUECycLlc15/iiI4xxhhjahb/0DHGGGNMzZJr6irxQSlhSu1em6aZoxX6WZ16m4JTVw1D0xUqUf70008DAM4888zw9e3bN+z6Qs/NpVypKx2f7nZhaFlDq/r/DLPmlZrLY32mtU7QdJV2V2ZKefHFFw9fOXcINid1peiYVI5+0KBBAJLpKJWbZ8pVd9KVc3wzSu3oNaHrj/c33Z2jtu565N+qdpeG/nleslpIpKWRGkNTUld5oLvONA2k56UpNCV1Rf7+++/U49MdyJz3onaiNiV1RbLS9TpWrlVdk7oWOT/6+nKeC6eujDHGGDNT4R86xhhjjKlZyi4YmEVamGrOOecMn4aZSbV2755Z0HOuOxm6dOkCAOjevXvq37ZExo4dGzYF2Q477LDw6VqtFsG8xqBpEna1fumll8J39913h82UVbULWuqaU5FK7WSeRpGCnfWl4DXtonZ9aaa06y8rtVp0J/NykfadARR7L9L7BIUrgdr5Lss6dh0r7ay/LaR1TMU/0RhjjDGmQlSsGLmacDFy49FiMj4R6hNxJZ+Oy1WMrE+2v/zyS9hsQLviiivqZ4ad9xNZ3sXI//vf/wCUWj0AwCqrrBI2CzvzLiYvupg1L6qlWDcvZpbxNaUYuSXQnGLkloCLkY0xxhgzU+EfOsYYY4ypWWaYujLGGGOMacnMcNfVlClTavJXUOvWrSteo6M1EtwBo31DyglrIGp9/mq9hqzWx1frNQKev5YJ56/W75+1Pn+KU1fGGGOMqVkqpqMzM6JRHHZ0BYAXX3wRQFL7o5K7eszMDXeb6fps6TofaS1m6uvOXYSex8xOVsdyxfNiyo0jOsYYY4ypWRzRyQE+qWijsx49eoTdtWtXAEltmpb4FG1aDvr0zAa61NMBkg10i2462FA0IqU6TmyGqcrCOlZed/PMM0/4qiWKoPOkjSqb+146viKUvTlXWpeojZx1LqkuXO3K3KZp8L6iau1qc33qmm2umrcjOsYYY4ypWfxDxxhjjDE1SyGpq+aGoao9pM4w3P333x8+TQ2ceuqpib8Dqid0noXOmYaZ03ycn/qaulXTPKaNL22cSn3Frll/q3alzoWutUmTJgFItri47LLLwj7uuOMAVGfzUp2TP/74I+zrrrsu7CeffBIAsNpqq4Xv0UcfDbtNmzYAgEGDBoVvo402CruIa5HrR8f02muvha2pNx6fpr7T0DTRqquuGvYaa6yR+My80Ln69ttvASTX2ahRo8LmnADADTfcAABYcMEFw1ft98c8yLrnMuWXdU405VmuOeb9qanvp/c3jkVb0Lz33nthb7HFFgCSDVKbe390RMcYY4wxNYt/6BhjjDGmZil76qq+FAdQqqbXqvq00FTWDgS+b1Y6oAj0WP/++28ApRA6APTq1Svs+eabD0B1pgaydC40DP7RRx8BKIWjgWRqYK211gJQCpEDwFxzzRX22muvDSC5K6aIHRZpO5EA4I033gCQHNNvv/0WNtffsssuG74ff/wx7D///BNAckxLLLFE2JtuumnYTJnoDqA80GPhcXfq1Cl8I0eODLt37965HktT4FzpPPXv3z/syy+/POxNNtkEQLIjvV5rn332GQDg2GOPDd8TTzwR9sILLwygsukSftZZZ50VvmuvvTbspZZaKuxu3boBAO67777wcacZUFqfU6dODd9mm20W9h133AEAWGCBBep8fjnRdClTi3feeWf4ll566bAff/zxsHn+BwwYEL7WrVvneqxFkLbTCCh9l+k9hfckoPS9Mnny5PDp9a3XBdO3zf1+TLv+eJ8DkqUZ9elYcax6n3nzzTfDZkpTvzOcujLGGGOMyaBZEZ206I3+2v7555/D/v3338O+9dZbAQAPPfRQ+LQIj+gTx7777hv2DjvsAABYffXVw6efW8Qvfo1evf/++wCAL7/8MnxdunQJuxr1IdLUcocNGxa2Fnvy17dGefTpme+hT2H6vieddBIA4JhjjgnfHHPMEXae0Tlds/o5d999d9innHIKgGzND86fPpEp/H99ItHojz5ps+By3XXXbcQoGo+Olce9/PLLh48FykDp+sk6V0UyYsSIsLXY/8wzzwyb86dPvOuvv37YnJ/x48eH76233gqbOld530f0mnjnnXcAJCMejEwBwM033xz2csstBwA4//zzG/xZafOf9/g0SnHUUUcBAI488sjw6TWvT/dDhgwBABx++OHhW2eddXI7zkqQFr0ZN25c+AYOHBg2I+ZfffVV+H744Yew+V2pUWKdS41u3nTTTQDKd/3q+6iaf1qRclZ2h+NjhwAAOPfcc8NedNFFAZR3fTqiY4wxxpiaxT90jDHGGFOzNDp1lbafHwDGjh0LIFnUN3r06LA1tcHCS9Xx0DQV0dBev379wn7ggQcAALvsskv4dtttt7BXXnnlsOvTdCkXGqZlSH3jjTcO37zzzpvr5zeFtHAjQ50AcOGFF4atoVMWhmvqo2/fvmGvsMIKAJKNTJ9++umwdS7TXq/rqlxw3WpbDtVRufHGG8PmXGqBsIbO27ZtCwBo165d+JZZZpk6r9c512LSxRZbLGxqoVQyncmQsKZztAiWqbWiW5SkFTCedtpp4VNtmBNOOCFshtRVh4UFvEApPdTcFgvlhGkcTZf27NkzbL2n8W+yGgGnzVUR86efucgiiwDITmdoAe0zzzwDILmB46mnngp77rnnrvP+1YiOT4ttBw8eDCC52UFTx7zuuGkFADbffPOwWbqh3y9KHoXbPNe65tL+HyjNsc61FstfeeWVAJIp/I4dO4bN86bfqW4BYYwxxhiTgX/oGGOMMaZmaXDsNm0f/QUXXBD2bbfdBgD4/vvvw6fpKA1NMuWklegabiO//vpr2B988EHYrCrX1Aor9YFkmJPphzzCnBpOU02B559/HkBS7lzTAAzJaWhT36tSaQz9TO4QO+OMM8KnoX3V9GBKQFMw7du3D5vzqu+/0EILhc1qe01nHXbYYXXeHyj/zhDVo9AUmq7FDTbYAADQp0+fOj6gNJc6p2k7sOpr+wAUuwOP6QQgeUzVmBJg6psaOECybYPCkLiOY/vttw/7lltuAZDUcdKUUd6tEYiec6Z7NTWgtv4t78G6U5XpHKC0FrNSc0WmsbKuCWrnAKVdb9T7AZKpnTzv6c1F7+m6q+/AAw8Mm2tYtY2YjgJK99LtttsufLqDk60RstZp1r2mOdR3TdS360pbmPD+v+eee4ZPNdd4TyzndeiIjjHGGGNqFv/QMcYYY0zN0ujUlVZKv/TSS2EzzKohum233TbsnXbaKWyGVusLPWpqROXCKQT29ttvh48iRECywjvP3U4aWtMwMsN0Kmiox0fBRE3t7b333mFzrHkLeunxf/zxxwCS4XIVKdt6663DZouLrNAiz7/Oj4ah+RkazlRxLA1jl+scMF2oKTjd1aDtAJhGU1nztNRO0SKVTYXz9umnn4avGtuRKJyrbbbZJnx33XVX2LqW+LfffIMOytkAACAASURBVPNN+HQHKNMAmq56+OGHw+7cuTOAZPfkvOeX76/X37333hu23j8+//xzAMm2CUz3AKUdfrvvvnv42HYFaPj9N2/087V0gbvNVBDyvPPOC/v6668HUPyuQIXXlN7HjjvuuLDVzzSj7tTca6+9wqYgJIXzgJZ5r9HvBy09YUqP8wgkU35OXRljjDHGNIIGR3T4i1mLoqhnA5QiGm3atAmfFvvpL+76CjDT9uHzKQYAHnnkEQDJ6NKGG24Ytj6JVeqXvra74C9xPT62PQBKRZTaKHKfffYJ++KLLwaQ1EnI+1c8n6j0czQiozoOfOrU+WHbCwC4+uqrAZTmCUhGR1jEvscee4RPi5XLVaCb9kSgjfL0KbJHjx51jrW+prMtFZ4XjRKUU7OiXOg5573kqquuCp9uRnjsscfC5nWlOk/HH3982Lw+tfBeNzBQM4oFwkDlnqI1okM9GSA5PkZ8jz766PCxkS5Qavapmlh6L2H0uJp0hPT8ct5WWWWV8D333HNhc+OE6ihVS0RSm+NqxiOtHY5GrDSiSH051Yzacccdw85qPVNt6JxouydqQulmgqwGoOXCER1jjDHG1Cz+oWOMMcaYmqXRqSsNK2nRKDU5svQ46gv9azESbU19qOYJQ5e69141a1RTolI6JVrYyZC3Snxraob6LHpOtPCQqR2GoIFkGrBc6FwxtK2hcz3nmpL873//CyCpbaGdyCl3rsWQqkmjKYW0Y8mj0y7D9Jqu0AJA1ay46KKL6viUauw+3xQ0BK5pVoactdizaDiXukHh0ksvDfucc84Jm2l0TZdqp2zewzTNoN3r2cmcRaGVgOtzv/32C592J+/evXvYl1xyCYD0ViJAMs1BtCs6i301tVVEujJLh4xpRL1+dYMJC8+rJcUKlI5Vz72uT4VrccKECeHT1D9TXtoCQ0tGWJBfTUXJOle8r+h3om5GGTBgAIDkNZl3aYAjOsYYY4ypWfxDxxhjjDE1S6PL7huze6o+NHRObRagtJtL0wwapmN48MwzzwyfahIUEdLTMB11foYPHx4+7Z7MkGuWRP11110HAPj666/Dp53e8wjzMTXGHVNAcqeYakJQwlzbKWjolSkv3RVSRIsLheesQ4cO4dOO5TrWQw89FABw1FFHhU/lyutLvVU7PBdZuyKreSx6bJru1k7z1M7KSp3zdZpa1dA6U8eq/aU7lPJsJ6M6ZMq+++4bNnVy9DpSm+PT86M7HHn9Tpw4MXy6pis1/3p8qrPCc6CpDW0xw1071bgTUo9Z29qkkaXDxu81vT89+OCDYW+66aYAitldnIWOhalvpqiApCYQ213oa3TNedeVMcYYY0wjKERIgZEc1ZEZOHBg2PwlqIWxp59+etgHH3wwgOQv2qI1T1SZlFER/XV/8sknh81fr/pEoyyxxBIAksrJeZCmXaBq1qrsrMXGnB9Vlt15553DZiREx1e0zgXHqkWbGmVbf/31w6Z+hUYM2QgSKD2psagTSBbAV3NEBCjNtTbN1UaqvO6KfkpU0p7ysq6f+iKGHJfOuSoHU8W2kuPnZy255JLh03ue0tCIqK5DVYhnkf1ZZ50VPlUMr1SkQN9bNYs6deoEoLSpAUhqejHipudHVc71vpQnuv7q07apb7OFFs6vs846AJJr/ttvvw2bTV11nopGI56vvvoqgFKjbyB5/2R0J+8ojuKIjjHGGGNqFv/QMcYYY0zNUrHUlYb5GHpTnYAhQ4aEzZCchlYPOeSQOu9VLekQAGjbtm3Y6623HoCkrLyGNpk+0dQBtTGAUhpI0wl5h9HTGlVqgbfqjLCYTAuQX3755bDPPfdcAEDv3r3DVy2pnaxiehZ4AqU2AxtssEH4dPwskv/www/Dd80114RdjamfNPSa1GLBrJRQpdFwNjcraDhcC5C33HLLsKn5pONIK0bW/1edliIl9tOOc3p/c9+XJQNjxowJn2580HtZnhsH9D6gmmzUD2OjYSDZAoNptnfffTd8V155ZdjcuJF1zppzLnVOtFHnK6+8AiDZ6kE/R5tS81rTAuT33nsvbGoe6blXfR6WNBSdItfr8/fffw+b90dtO7LLLruEzfOSd9sHpTruaMYYY4wxOeAfOsYYY4ypWXJNXWmYTyvhWTWvHWm1Ey01TXRXjL5X0SE7oqE33WHFXWHaffjmm28O+/XXXweQTNetueaaYavce6XJ0lNRnaPx48cDSIZptdMwWyhoOkB3SFDOvJpSOxomZmhYtYNUR4eprRtuuCF83CkCAPvvvz+A6hpffXz++edhMyWcpbNTKTSc/csvvwBIamuxFQCQ7N7dsWNHAMmdgLqrhfcStnoAkuPXvy2SxoT20+6P+hpNx/30008ASno6QDINpK11KqV5pZ/Ddjl6TWkahKUB2t17q622Cvu8884DAOy2227h03uRptEbmzLR86znj2l6vU9qWxVNzbEkQdM9uquK151+D+h3YbXcV/TcDR06NOyxY8cCAF544YXw6fcj57qSLTwc0THGGGNMzVL2iE5aASGQbHBGnRlVu9RiThaTaUShWqI4WejxUeVYf+VqUz0WIWuxtRa+sli5Wn65A0lNIyqTagFd//79w2azT0a2gOQT9z777AOgcnoXWeha/eSTT8LmE68WELJpLVDSFHrxxRfDN2rUqLA5Pn2KLnou01RItRiSURCgNNdFX3P6+WxgqY0oX3vttbBvvfXWsPlEqYXLadEpXX/6WZ07dwZQ3mLghqJrRpXHR4wYETafjnV9rrvuumEzIqVRjL/++itsRrK0UaSu76LXKuciSwX7P//5D4DkmlVlc2qWPfnkk+HTudZILM9VQ9e6boDZfPPNw+bGhC+++CJ8o0ePDpv3FKC0PtMaWQMl/TLVLlMdpCLnR+8jP/74Y9ga/WYkSovai76XOKJjjDHGmJrFP3SMMcYYU7OULXXFkJaGXocNGxb2oEGDwmaYTlM3TIcApfBg0SHUxqDHynOg2h5bb711nb/NSs1Vy7j1OLTwmGk2TdeopkyXLl0AJIv+tNizkkVoM0KPQ9dq3759ASS1dTT1xnOhYepTTjklbKYei2he2hAYxtdiyHbt2oXN67Oajp/HpAWmWmysTXGZ5tFiUaYLgFIaYPHFFw+fjv/YY48FkJS1r1ToXa851QlSTZkJEyYk/gWSxZ7c2KAFrj///HPY1MzR+5O2wKimeSdpDVpV50tTl0xNqY6XbpYol06S3j/YgkJTiCyaBpLrhzpyWfD+oe9fLd8JmmL76KOPwtbU4OGHHw4gmS7WlF8R939HdIwxxhhTs/iHjjHGGGNqlllmFBKbMmVKg+NlDGlpl1mGsABg4sSJYffp06fO/1dyh0rr1q1nAYCpU6dWRzywzMw+++yzAI2bv/rQkCU1TXT+Ro4cGTbnUsO1ugOGmhdNbeHB+Zs2bVqzxqchVN2V0q9fPwDJcDjHrDBFBwAPP/xw2Gxh0tR13KpVq7KMT0kLg+uuK03TMA2S13XI8f3zzz9lmz9dn1xXGk7XdAH/Nq0tC1A6F01NV80222xNnj895zonevwc36effho+TaM+8sgjAJLpLt01ydSrlhNo9/D65r1c81dOdP6Zeps8eXL4dPy62ywtjcL5K+f9s5rg/bMp85e1K/D2228Pe8CAAQCKK1Hg/CmO6BhjjDGmZvEPHWOMMcbULM1KXaWFw7ljAUgKM+2xxx5hM8xVhCAX4NRVc2H4Urv3qtz3l19+CQDo0KFD+HSHR3N3dZQrdaWkSeirIJ12Z6fIGIXlgGQLk+bu0MkjdZVGVpfyvHcY5Z36SAuZp/ny6G4NNC91lUVDx9QY0nYyNYRqTF0pPC9Z3y/1rW+nrrLJakuSdv+sZHdyxakrY4wxxsxUlK0Ymb/YtGiTeg1AUrOCcuNFaQM4olMe9Fe8FqlxLeiTU1MLj9PII6KThhaDpj0d6vjKqT1SqYhOUVR7RKC55BHRqSZmlvlzRKdl4oiOMcYYY2Yq/EPHGGOMMTXLDFNXxhhjjDEtGUd0jDHGGFOzzLCpp4vpWiYsxqr18Xl9tkxmlvmr9fHV+vr8999/a3J8s84660xRbK04omOMMcaYmmWGEZ1ykiUuRNLEq1w/ZKqRhopfef2aIuD6zFqneQtCGjM9Df3Oz+3zc313Y4wxxpgCyTWik9YiAgB++OEHAMnuwvPOO2/Yc889d53Xz0xPIWndk2em8Vcj9XXH1rXK+VMRxWqK7vBY00QegZL4YS2tOR2r2jNC50wFIavlvGRFyf/66y8Aye702r2b99c02X5jyoVeZyoe3LdvXwDJ9XvppZeGvcACCwAo7z3TER1jjDHG1Cz+oWOMMcaYmiXX1JWGRu+8886wDzzwQABA+/btw7fiiiuGvemmmwIADj744PC1bt067HL2FapG3nnnHQDJ/mALL7xw2NWSBslKfaTREtNwaekAALj++usBAKNGjQqfnotlllkGAHDyySeHj+FYoJj50+ObMmUKAGDcuHHh++abb8Jea621AACLLrpo+KplzTUEzltW6Hzs2LEASucBSE/dzTPPPOHjPQkA5pprrrCLXMvaP27MmDFhDxgwAAAwdOjQ8K288sph77vvvgCA/fbbL3wLLbRQ2NUy1/XdU6rlOE0SXn/ff/99+HbdddewuW4HDx4cvgUXXDDsPK4pR3SMMcYYU7P4h44xxhhjapYZ9rpqrrKnho4333zzsLkbQEOrGka+8sorAQDjx48P33XXXRc2UzpNTWHloeypaQ6GXOs7Pn2Npg66desGADj66KPDd8ABB4Rd3/s2RxlZw8VZuzoYehwyZEj43n333bA577PNNlv4unbtGnbbtm0BAK1atQpfY+Yyb2VdHv/EiRPDd+KJJ4b93HPPAQD+/PPP8Om54rhHjhwZvnbt2oVd31ibsz6z5k/HcuaZZwJIpis0XMw03cCBA/WYwm5uyqBc85eV2vj4448BJEPjzz//fNhvv/02gOxdcxwfd88BwBFHHBH2hRdeGDbnWs9JHsrIenxcn2+99Vb4dtppp7CZMuDuqulfz3Wra7p///5h1ze/M1qfaXopM/ITXX/8fE3NKRx/1vyXa302RRk5SxtG7bTUakO1ubI+qzHpnjyUkfX4p02bBgA4/vjjw/fwww+HfffddwMAttpqq/CVsxzFysjGGGOMmanItRhZf3Hqr1f+etNf+SussELYF1xwAQDgkEMOCZ9GNPjrcI455ghfEUWBevxa7Pj3338DAJZbbrnwpf16nzp1avjOOOOMsD///HMAwPrrrx++vAvveHwaWXvppZfC1mKx119/HQBw9tlnh49jBtKfTlggCQA77rgjAKBfv37h0+hCfU9/eaBP74yusWgTSBYeL7vssgCAs846K3ycM6BUeK8RndVXX73MR5wkLYrIJycAeOCBB8Lmk7wW2Gqxdc+ePQGU9K4AYOmllw67WopAdZ19+eWXYW+77bYAgJ9++il8el64vrSoWEnTsdKI8mmnnRY2C7bz3iCh0adHH30UQLLYXa9bRoR33nnn8On6vfnmmwEkz5nei1Rzp6FzzbnQdaTn78cffwSQHvkGktcPi8Ufe+yx8On57dChAwBgscUWC1+XLl3C5iaXrIhQHvD+qedUo2QTJkwIe8011wQAbLPNNuHTtZjWGSAtUqsbVLTYvIj7p34mz8Hw4cPDt88++4TdqVMnAJX9znZExxhjjDE1i3/oGGOMMaZmKXvqSkNsGk795ZdfwmaRrRY4amiW/htvvDF8hx9+eNiXXHIJgFJR5fSfW6nQ+meffRb2/vvvHzY1gW644YbwaWqEx6oS7a+++mrYLNxWbaE8xqThRharaopJz7+GtonOn46PIUk95v/9739hM6Xy0Ucfhe+2224Le6WVVgo7z/Czjv/nn38Ou0+fPgCSqadVVlklbOrobLHFFuHT9U1Nk8suuyx822+/fdhLLbVU2OVKeXBNvfzyy+G7/fbbw77qqqvCZuhcQ8eaBp5vvvkAAJMmTQoftYGqCT1+LRDmXC6yyCLh0zQO07Ca+tBiZbWJpnMqda/Ra4oF8EDpXqPXBmX1AeCEE04AkJzT33//PWwev67DpqQ70u71ek/WdA3T3VntN7Swn/dF3cygvPnmmwCS49c0ENe9zn/eqUWuRU3x69pYbbXVwua5vuiii+r4stDvxw8//BBAUptLN/bovSrPcev86/cDv0N0zWnpCb83KqmH54iOMcYYY2oW/9AxxhhjTM1S9tSVhiY/+OCDsDU1sMsuuwAA5p9//vBpGJIhPw29andT7obR0Ojyyy9f5/XlhOP69ttvw6fhON01wDC6tq1IOyZNDX3xxRdhn3POOXVeX64KdZ2fr776KmxWxas2h4Ymda6YxtBwqu7QoWZMmzZtwqdpIOp8UM8EAB566KGwe/fuHTZDuuWaUx3/5MmTwz711FPDvueeewAA66yzTvh0BxN3OOj4me4BSi1OmEIAgEceeSTsI488Mmye46aMLy10zB2LQDKdynQVULrW0jquA6Xw+7PPPhs+3QFYLejxq2YM5+Xiiy8O3+677x4215SmS3Xd8/81HK87QHWuy32v0XSNpiF79eoVNsfH3WVAUrOE9w0tHdA0MdFzpqm5hqYUptMOAgD89ttv4dNdN2uvvTYAYMsttwxf1g6s+jRl+LpnnnkmfHqueF++4447wpd3Cxa+p+6e0jIAnVeOj3ozWeg1qWUS3bt3BwCsu+664dN2QZXazaTz9Ouvv4b91FNPAUjeM3RXWCE7pCv+icYYY4wxFcI/dIwxxhhTs5Q9daXhyDfeeCNsDY3OO++8AOoPkWo6SyvoGUa+9tprw8edWHmjKSodnwoiUYgtS/CJ4noqUa/hcIrLNUUWPIs0QTmV5WboXudPU0+XX3552JTu1tC+hsaZutLuz/r/3EGnqSsVD9SUIOe9ueFmjkuFDXWniqYx2L2bKSwgfSdDlgQ7d4DorjSGcwHgoIMOCnvOOees815Ngbu+2P4ASIowpqGfqaF1phl0TrbeeuuwGZIuugu9Hr+mIbnWtXuy3n+4bq+55prwUdBO/1avFQ3DN7V1yYzgMWuK/+qrrw5bhei4vnQnXZp4qs6P7nrkZ2nqqrnrjylh3XWl65zXlKZ2mnp/47XcsWPH8FGEFABee+01AMC4cePCt+GGG4ad524fPY+8trPQ1JSSljpVwUruxj3vvPPCp/faIq7LtDIVbfHQFBHKcuKIjjHGGGNqllxbQGhhmv765tNjY35Z69927twZQPIpXIthKcve2M9oCGlF00CyGIzj07/VIljqQGhhMxtdAsCSSy5ZxiP+/6RJtKv2As/TEkssET7VYaHsuv6tPtlusskmdf5fx7/qqquG/d///hdAMqKTx698fWLksbA9AwDccsstYes5Z0GvRnHq0/PR42cxrz7FfPLJJ2FrEWJ9T30NhZ+pBaoaEdCmuoweZq1lXj/aaFaLxddbbz0AxWhXKXpN7b333mGz3YVGfNm2AwBGjBgBINm2Rd+LT9LatoNRLiCfsfIp/JRTTgkfWz1M//lseqxtOfQpPi16OXr06LD59K/aQs2Fn6/HpOec94RyRhuyWgDxXqrXbxFRjsask7TolhZTv/DCC2FzY4PeUyvZ7iINLXznXDOKB9Svs5b3/DiiY4wxxpiaxT90jDHGGFOzlC11laYNoIWRmtppLgy9q4T5p59+mvpZ5U5daYGuFkhrGoYpO9We0eNgSFolsnv06BE2NR/yKJrT+dECMqLhetWR0dBiWki2vtCppgbS0jXlLLwmepwsvNbO01qsq2keFtE1Jhys43v//fcBJGXtNaSfVYTYWHR8TJOpnsp2220XtnZiZxGutnXQ96ImiR5/NXYsVzRMzrnQdLYWxqbpCOma3GmnnQAkiz2zpP2bgxb+jx8/HkBSb0l1tFSTicWoWfcHjktTX6qZxdYXep8s15j0PlHOdITOFdPv2mJHYWouqzt9NaLn6v777wdQ0lMDSulKoJSSKzpdpeha5vcXi8KBZBkLU266/vS7Mpfjy/XdjTHGGGMKxD90jDHGGFOzlH3XlYZAteq/XLtLFN1B9NNPP4WdRxqEocXlllsufLrTgx2tgVJ3dpWNVx0LdiLWNFjXrl3DZui9nKmrNIly1Z5gO4qmdDHOQtM5KhFO6XYN15YrDKufqW012EJE18wxxxwTtu5WamgYX8+VpiHPOuusOn97xhlnhK36JeUK73OtaJdk1d7QtciUlrai0DTwsGHDACR3orGjNFCaq6zu0uVCr2Oeaz1fr776ati6A5Ov0/nR1zF1qNecpoYoV0+9r+lfn0caj5pNGuJXPaldd9019ViIrntq7mgLAn0Nd1sVrW3SGHQun3zySQCl3XVAcgcWr2V9TTWOT+dMSx9OP/10AMnUKdsmAcXrV6WhOk+8V7z77rvh0x2cTMktvPDC4Tv//PPD3mijjQCU9/vPER1jjDHG1Cy5RnTU1n32zSUtYpP3L3a+vxYInnvuuWFrES8bQDKyAyQLO1kkufHGG4dP9Wvy/MWu5051cIiqyarODwsg9T30nOvTCZ+kdMyqQsvCYB3ncccdF7YWpjV0XnlMus70iYhPF/oUf9JJJ9U55oZ8JiMZGuXQiM3zzz8PIPlEriqhlULHqk0Pr7jiCgDJRqU6FjYDTdMWAkoqwhoFKrdCMJCcBzY1HDhwYPj0iV4jIbxGdX3p8VHxmQrdQDLinIfmC9F1plFoRjlVIV2jcGlNNzWiportPXv2BJDUbtLCz2OPPRZAsig+T7XgcqDnjSrrGiVXnS8W3us9qZrGx7GomjfnDAB22203AMmMQdGaVfWh2QlGfPfaa6/wdevWLWxqOh122GHh0/s/C/LLuanIER1jjDHG1Cz+oWOMMcaYmqXsqSsNh3I/PQBMnDgxbIbuGhOO0tAdJdpVB0bDvHmG9jScrQVw+vkMnU6aNKmODyilhDS1oO+VR5iV50TDuSz6AoB7770XQPKcarhRUxds8Kihcy1GY2iZ7wkk0ww8hhVWWCF8mubRMH1Di5S5PrQo+K677gqb2hqartCiYP2ctIJsXX/UbNICOv2sHXbYAQBw9tlnh0/ThHmmJrPWvp5rNoPUpoGaRmFKSnVoVJ+HKRFNXTUXnl89N9qChPpHWkyu9xpNKbOIWOdUU1v8f31NVjuMcqPrSAv0eU9QbSPVXtI1yXljKwsgmSbn/Ky00krh080SbM1QTemcNNIaIQOlc6XnRFN+vNaqKcWT9v2lTYVVk41+vb9WYwGyok1vWWSsTZH1u47taLTAWtu18LrQdFhzcUTHGGOMMTWLf+gYY4wxpmYpW+oqLTWiVeN77rln2NQ3WX755cNXX2hOw6zUh9Bw3wYbbJD6t3mioVH9TKZetMWASrAzZfKf//wn9b3yRMO9rO4HgOHDhwNIppvGjRsX9sEHHxw2dVg0NalhdO6Q0XYT+rkcP6XOgWRqpTnzl7Xrj2kK3R2naS792++++w5AMnQ8ZMiQsC+++GIAyR1eZ555ZtjU52G6DCg+9Jz2+ZpO0x1WaZpL2mKBc73FFluU7fgY2td0zoknnljnmDStqej4eA/Kuj7feustAMl0crnactSHHpOuD15LmqJR+/XXXw+bmld6rer4eH1ydxmQbOdSTa0DZoTeMzSNOmrUKADJdJCmTpjGKjp1pcenNjWfuNMOAIYOHRo210XR94z60POrmjhMSamOl2qW8b6h49ddvXnMnyM6xhhjjKlZyv4Yo08WnTp1ClsjAozu9OnTJ3z6pE20gFmfqKn1ogV2eRfzNgb+eqeeCpAs9jziiCMAJH8FV+rpQ58S9In+8ssvB5B8ctdGcqqpwydJfUrRc06/jo+N6ICSTkvbtm3DlxWJaSwahdBiaz4F6vjY3BBIav5Qn0OfuLUwrnv37gCSBdTrrbdenWNpSU9kanP+NHqi53LMmDEAkuMrl6J2ls4ToxtpasnT88svvwAA/vnnn9T/79KlC4CmFb03Fz1nuv64pvr37x++jh07hq2F49Q8at++ffi0ASTvu3r+WkoUR9E1mRYx1vOnDWy5LqppzNrg8qKLLgKQ3OCh98eiv78ais6PRr/TNg6ocjn1u3QzAKPkALDooosCsDKyMcYYY0yD8A8dY4wxxtQsubaA0DCzFsbdfPPNAIBBgwaFj6kFoJR+WHvttcOnTe0Y3k2Tba8GOG5NV2lhIYuoy9lAsynoXLHtwlFHHRU+LTZlI1IAePzxxwEk2wZoMSBDyqqdoM0mWaRdztAyUwKaurrjjjvCfuGFFwAkW1yMHTs2bNXhYGHoNttsEz5Ns3F81Sox31zSNhZoYT0bEOqaVk2ipqQe+RptpHnooYeG/eyzzwJINmrVdE7avYApKiC5ljt37gyg+NSiniem9tNSxEDyWFmk3atXr/CltZApenxNhfdPvT889dRTdf5OSyO02LrIa1Hv6WyVAiR1jli6wHQl0HLniujxM/WkTWW1gTRTylraoN8faS2GmosjOsYYY4ypWfxDxxhjjDE1yywzCg9NmzatbLGjtN0SqkOiEu38f931ohXaPOamhvtatWo1CwD8888/uWx14li16vyggw4Ku0ePHgCS4bzGdM+uj9lmm63J48va1aLnmjuU9Dh1BwtTHqpNon/b3DAtx1ff+tTjT2s7oqmPtL/VXStZmix5kPf6bCiaunrxxRfDHjx4MIBSKwkgueuiAd3fM+cvTSofKJ3z8ePHh0+7P6+xxhphc/50J6beP9LaTZQTzl9j7p88Zh2zdudWmGbW+amkZkyl7p+autIdZlwD2vH6vPPOC7u51yfX57///tvo+dPUuKardC4fffRRAMl7ZiXnb9ZZZ50FAKZMmZLrh+q1rN8FaampcrZgad269SzT+xzRMcYYY0zN4h86xhhjjKlZKqN7U+d6XgAAIABJREFUjvTUhYbuVPxpRq+Z3l+N8Ph0J8h9991X5++qcUwNSdHoDpu019X3+kqh55e2hlOz2gmQahIcK4Is8U92H85j/WalQzlva665ZrPftxp3uPCYNF2qu1IUjqVa7hl5odeqtohhixYKjwLFnwvO38CBA8P38MMPh33qqaeGzTRX0cecNzq+LPHOSuGIjjHGGGNqlooVI1cT1VLsmRfNKUZuCTS0GLmlMrOsz1qfv1ofXyXXpzb1pH7XsssuG76sjQ9NoSnFyPxM1UHSDTbatLK+SHLeVKoYuShcjGyMMcaYmQr/0DHGGGNMzTLD1JUxxhhjTEtmhruuaj2HV+s59KlTp9bk+GaffXbPXwtmZpk/11i1TGaWGqtaH5/i1JUxxhhjapaK6eioJoLaM6Ia9S7yQlsQpEnUV1OKkceaNY/NbdFhTEPQayYNr7/aJqtdDSn6/pl1f6zv+29m0UmqJI7oGGOMMaZmyTWio7+otanZ008/DQAYMWJE+LQp3y677AIAWHzxxcPX0ChQS4Bj0aeQe++9N+yrr74aAPD444+Hj438gOKfTtiM9ffff0/927nmmivxL5A8Zj+pmKai6/DZZ58N+4orrgi7f//+AID11lsvfDN7dEd1ZkhLVf7mfVPVdvv16xc27y9nn312+CrVALUhmYtp06YBSK5J/S6gzk61q3krevxca3rMqrJexP3fER1jjDHG1Cz+oWOMMcaYmqXsqSsNEY4cOTLs8847L+zRo0fX+VtN3Vx88cV1XvPf//437KKbRTYXhvkmTJgQvmuvvbbO36WFmyuJhl413MjUwCWXXBK+2WabLex1110XADB48ODwLbPMMmG39PlLIy1MXe0puqxizvrGwpB0JcfH4/v555/Dp+vvhRdeCJtp8kceeSR82iCz2tMAzSUtNf7BBx+EzfvKSiutFL6WtFaZcrvmmmvCx+8UAFhuueUAAJ999ln4VltttbDzGGvaNfP++++H/eKLL4bN4/rll1/Ct9BCC4W9xx57AAA22GCD8JWzxUW50O9vvS6HDx8OAFhllVXCp814+V1RyXSWIzrGGGOMqVn8Q8cYY4wxNUvZcyMawhs3blzYo0aNCrtVq1YAklX/iyyySNisSu/bt2/4tFNtx44dAVRPCK8haBh50qRJAIDjjz8+fB9++GHYTN1l7VqqFDqXEydODPuBBx4AAGyzzTbhO/bYY8M+9NBDAQD77bdf+O6+++6wl1xySQDVlcJKS91knXMed1o4XV+n6byi12pamuqvv/4Kn+6A5LWq89OlS5ewN9xwQwDAnHPOGb6800E8fr2nvPPOO2F37tw5bO7g1F2Ben+pRXR+mVIYNmxY+A477LCwd9hhBwCl3Z3Tv77otUr0mDR1w3Fx9y6QvL/wvpl2TeaNfs5dd90V9mWXXRb2PPPMAyB5z9Hr58EHHwSQ/P7r2bPnDD+rUugxf/nll2EfeeSRYb/++usAgKWXXjp8mobbYostAABdu3YN3wILLBB2HvcSR3SMMcYYU7PkWu2qT4RTp04Nm4WBO+64Y/hOPvnksD/66CMAwAEHHBC+k046KWxGFPQXY0sqMGR05M033wxfu3btwt5+++0BJIu9ioh+6K/3W2+9NexvvvkGAHDDDTeEr1OnTmEPGDAAAHDCCSeET+fvxhtvBFDZiECamrOe399++y1sFrP++uuv4dNix7FjxwJIFhA+9dRTYX///fcAksXYqulSKU2orKf0MWPGAEjqjDzzzDN1Xq8RqSuvvDJsRgTOP//88LEAFMh3LlWPS+enffv2YfO4snQ8agWdXx0rC7MPPvjg8HFNAqWIl17f1RLFUbKKqXkv0jW56KKLhp22/ir1/aBzsvvuu4f9448/hr3EEksAAFq3bh2+jz/+uI6t15cW82666aYAilF+1vHpd4J+l/H7W+8fr7zyStj3338/AGDgwIHhe+KJJ8JecMEFAZR3TI7oGGOMMaZm8Q8dY4wxxtQsZU9daThNC5A0TL7VVlsBANZZZ53waZhyscUWAwCcddZZ4TvqqKPCvueeewAAJ554YpmOOh+yilWHDBkCIBmGb0rT0zzQz9Zi1fvuuy/sQw45BACw8cYbp76ue/fuAJLpoGOOOSbsd999FwDQoUOH8JUrtJyVrqHOg+o9qPbKyy+/HDYLXv/888/wff3112Fzreoxa5iW0vQXXXRR+G677baw55577rDLHVLX8atE/lVXXRU2j0vD/WeccUbY3Cxw++23h0/TfCyW5N8ByTRmnkXYWXo/+jn8m2pMx5QTPRdfffVV2NwEoAXa2267bdibbbYZgOzUV9HwuLTcQdM4HIvqtBTdYiDtM/X7b6211gqb143O399//x0257JHjx7h03sV56/oOdNyC7atAErfG5rO7tatW9jrr78+gGQ6/4477gi7d+/eAMqbbnZExxhjjDE1i3/oGGOMMaZmyTV1pR3J1WYYXENTaZ1cmQIBkhXaTz75JADg6KOPDp+GzqoF1X5gJToA3HTTTQCSIf5VV101bOpAFK2do7taNA2y9tprA0imYDTMTNiFHgCuu+66sCndzlYRQDIN0pRx87g13aZtNZgu1N0nP/zwQ53XA6V5y+q4zvU7xxxzhC9th4+mJovoFP3oo4+GfeaZZ4bN1LHqqKhO1VtvvQUgudNMbdV3Ijr/uq4rhaYBGvq3mo5TOH9Fpway4FrVNaVpfuq0qHYLywGA6h8f54X3efUBwL777lvnNdWYptTvt7R0rv6/fn/xu2DPPfcM30svvRT2Tz/9BKC0O2n698oTXTN6f9e56tWrF4CktpF+VzB1rvfPnXfeOew85tIRHWOMMcbULP6hY4wxxpiapZD22PWF2Rgem3/++cPHnT5Aqau5piG0wrvoMCbDrNqd9oorrgibLS403M62FgAw77zzJv6ukugxjR8/PmwdC9MUWeeZfg1NMt0FlHbNfffdd+Fr7vwxnD9y5Mjw9evXL2yKc2mKSsPFGpJlSkAFAVUQsW3btgCSabJBgwbVOZallloqfM1NzdUH19znn38ePu5eAJJhZgqtaVsEvSa5m0Il2lUQkTsoVfZd5zrP6y9LJE3XEtenpgt0XTP0r21XdF1wN099InRFwbGoBL92au/Tpw+A7OOvr8VJEej8/PHHHwCSgpu665ZCo0Wkg5tKfec6bddgVukDd7Pq/alSpKXwgaR449577w0g2eJo8803D5tj0destNJKYeeRhnNExxhjjDE1S64RnSxtmIY+SegTNxsJqr9an0go933qqaeG77HHHqvzt/rE2aZNm7CLeHrk/OivaerdAMmIBPWPso6T49OmihoRyLNwnO1FgFJRJlB6ytenEI0Y6vpaeeWVAZSeTID06Ayb1wFJnRo+ie26667hy7vdBZ9uzz333PAts8wyYV966aVh80lf51qvz08++QRAKXIKJJt6smmrFiVXqhhSI0dqsy0HUIoIaLEmC6yBUjGkFlDq9UudD24aAIDll18+7CKvT6C0MeDyyy8Pnz7da2FnGtVy38zSC2OzTr0+tcVHS2/nwbWWpQnFTRK6PjUizrnW12v0R8kz6qXXAbMQAHD99dcDKLUyAoCHH3447DvvvBMAsN1224Uv7zl1RMcYY4wxNYt/6BhjjDGmZil76krDoiqhrxLllNvP6p7LlIOGK9955506f5vVQqFSoVk9ftWZOf300wEkw3WarmEa47TTTgufFrsWUWTHsai2jMpyr7jiimGzWLW+ED61awBg8uTJYTN1Us4UFud8tdVWC99hhx0WNtsWqF6MpmZU5ylNZ0XHyjCxdlRWGMZVifo8SGsxouFuXV+axuJa1TGpTg51MLTYWgu7mVKoZAohbX71/vDZZ5+FTf2nYcOGhU/TyEwt6zWnnaSpWaLnUothi0DXIlOL7AINAMcdd1zYq6++OoDkfaRa0lWKrl/VYeJ94/DDDw+fpn71Xltt6Jh0ztTP1KpuptHvMqZ+3nvvvfBpse69994LINlCab755gtbywx4v8t7/vVewHtNz549w9e3b9+wl156aQDpbXPywhEdY4wxxtQs/qFjjDHGmJqlbKkrhuZ0J4ruClAdAKaxslIfDPlpaiOtwjxNG6KSaDjwlVdeCZu7BjTcOmXKlLC32GILAEkp86w0SaXg+VO9nI8//jhsDd1Tzjvr/DNkPmLEiPCpJhDHrTofzQ2t8vW6ZrQjNzs66+doOkfPf5pEe9quOm1LomkCdh3WcHMec6pj4flfYoklwke9IiC5A4kd3FVzR9tFMIx+ySWXhE/bdRSx64XnT3d3HHjggWEfc8wxYTON8+2334ZP1x9bx+j60PnZeuutAQDPPfdc+DSNkrZW8iDrnqY7zIjeSzg/1ZiuykLvj0TbVlSTjhFJS1PpOlOdJi3deOihhwAAzz//fPg0TZy2K1evVaak0+65QHKHFtOwRV6zQDI1x5KOtLZQgFtAGGOMMcY0irJFdFhMps0D33jjjbBV04Q6JaqToL/oqPyoOiwaaWCRM38ZA8Caa64ZdmMa/DUWfW9VJj3ppJPCTtNsUWVnFmlpAWS1PLFkNbJUzRSegyyVTD5JayM61fng06c+sZSrADvraWCFFVao49OnnPqeInTeGeniOp3+9VR5rmSBPM+lKnAfdNBBYev6pP7M33//HT61qR+kr6+W9anrjJEXIPl0qJFEoiqtbICp2iN6LTO6kOd9pCHo+tGIB4usN9100/AtvvjiYbeUSI6O74svvqjz/xrRSYteFt30WAuoqdP0wgsvhE+107TwmK/T6LOOhVEh1aGpL6Kq988i1oJeK7wvajG/wmafJ5xwQvg0UuuIjjHGGGNMI/APHWOMMcbULM1KXWm4isVSWoCl+/lZDKo2U1hASfsCKKWkhg8fHr4XX3wxbBaR3nLLLeHbZpttwmaxL1C+lEhauFQLOCdOnBg2w+vrrbde+KitA5Q0D6pJypzjWmCBBcKnxbQ6r2lNE1XHhIWhLNoFSo0ggVJqp5Ljb8pnZRX7sTWGhq41DJ1ni4ssmFrS5nlaID9hwoSwmbLRdhGffvpp2Fyr2mKhWhooagpNQ/SqI8M0cVo6FQDGjRsHILm+VROJ63aHHXYIX5bmV57o+vvmm2/C5mYH1YHSNFxLSV3pcWrqgvcgHZOef64BPT+VSq3qPW/o0KFhM92rJRpZLR5YsqDHvNVWW4VNrag999wzfKuuumrYjWkQWqnzomPlvURLMy6++OKw2cxTv/N1/nM5vlzf3RhjjDGmQPxDxxhjjDE1S9lbQGioTEPHqq/DNI+GqzQ0y10tP/30U/g0HNe7d28AwCKLLBI+lfbPM1ynxzFp0qSw0ySsVWKe2jNA9exgUXhMqm2jLRTY8RkoaUK8/fbb4eNOFqAUujziiCPCx7YCQGldVON5ULJ2WIwaNQpAUvtC0yAbbbRRnddXCv1M7bjOdCEADB48GADw6quvhk81c5garpZ0VRZ6LbZp06bO/+v6YsdkoKSPo92TdacW0xPHHnts+NJSJ3mQlS69/fbb6xzflltumfq6ar+u0tBdnWPGjAEAHHnkkeHTNCrvH7prR/WjikgtEv3O0zWjqXN+V+iuRm3XwnOh71Xt16LCNLlek/pdwN24vI8Cye/vPHBExxhjjDE1S7MiOvrLmZEAbTSmOir69FjfEwd/CWujsv79+4fNwi8tDNP3rNQTjapgslEbUPolrtoXRTzdNwbOpR7nHnvsEfa1114b9sYbbwwgOX495ywMVe0SLUyrpiLsGaHngjoZAPDEE08ASD6xrb/++mEvueSSAIp/stbje/PNN8Nmsbg+Betct5SIm6LKzyxi1fGpWiwL5zXKrCrmjORosXOlzoWuOSpYA8DNN98c9qGHHgqgclHsSqBNZdn08bXXXgufbnbg/UPvuddcc03YnP88Ijt679pwww3D3n///QEkN6hoxFfXUp8+fQAA3bt3D59uYOBctqQojp5rZmx0TjU7Q80uVd7PG0d0jDHGGFOz+IeOMcYYY2qWsqeubr311vCpjoemmeqD4UEN9zFdolQyXJsWBtWmgh07dgyb4efVV199hq+vRvScqqbOBRdcEDbTiNrCQ+XKWWTXEtNVioaOH3jggTr/r6khleNnMaGm9iqFpj5Ux0g1V9haRdtF6Fy3lDSIXlMrrrhi2A8++CCAZDG2FkZy/Hqu1NY0WKXg5+txaFNWLcZlg92WXoCsx6wbN1hkrRtUPvnkk7B5L9H7a6Uarep9TDfDUCdmgw02CN/LL78cthbjUl9Nj7Ml3h+zCucnT54MIJlu3GWXXcJmO4xLL700fHl/PzqiY4wxxpiaxT90jDHGGFOzzDKjkNGUKVMaHU/ScL6GE5uCHls5K9Bbt249CwBMmzatWfGyLIlyouHISoYmW7VqNQsATJ06tWzxQB0fu8erT3cN0J9XOH322Wcvy/xlwZCszhl3ugClrsTcXQUkO/VSh6ap42/O/Oma1F2PqhnDXUX9+vULXyVbHOQ9f0yTZ+3ErG8HZHPXLefvn3/+afD40o5pwIABYesOTu7aKWpXzmyzzVb2+Uu7f6ov7f91nsp5Ljh/jRkfjy+tCzmQTOPzuIsqZ2jK+OojrR3U0UcfHT5N85188skAgHbt2oWvnN8VHF/i+Mr27sYYY4wxVYZ/6BhjjDGmZsm1BURL3AnQGFqSoFNz0TCryrWn/X9Ln3eORcOxKs72v//9D0AyndCYXYV5oudej1kFH3faaScAydRyS58zJa0di1KNOyDTjkkFN5VavO+krb+WtCbTOqqniQAC1bn+mouOjzsgn3rqqdS/5fgrOb+O6BhjjDGmZil7MXJLoFzFyNVKHsXI1UTexaxEn84YxQGAL774gscRPm2a2apVKwBNf3Ir1/xlbQxgRKCoJ8tKzV9RNKUYuSWRRzFyNZFHsW41MbOMT3FExxhjjDE1i3/oGGOMMaZmmWHqyhhjjDGmJeOIjjHGGGNqlhluL3cxa8tkZik2q/Xx/fvvvzU5vllnnXWmmL9aH1+tr08XkzecNKmKopWfFUd0jDHGGFOzlF0wsByk9TVR+EvR9UUmD7itXLeX19cfSdei16cpN41Zi2mkrc9qR78HWpJ44MyCrsMPPvgg7DZt2gAA5p9//vAVveYc0THGGGNMzVI1ER3N8f3www8AgH322aeODwBuv/12AMDaa68dPv/iN+WCTx8qEjh16tSw0+TetS3GnHPOmfch5gqfpLMiB0VIuM+MZHXn/vvvv+v40tD5m2+++cJO6+5eTfC49PrTa0q/K0xl0TWl98RLLrkk7J49ewIANtlkk/D93//9XwWOLhtHdIwxxhhTs/iHjjHGGGNqlkJTVxoGe/vtt8PebbfdAAA77rhj+Dp27Bj2EUccAQB4+umnwzf33HOHXUThk44lLfSvNkOzRYfz8kbHXHQxWn1oOPzXX38FAOy8887hGzlyZNirrLIKAODnn38On/a6uueeewAAyy+/fPiqMU2Q1b188uTJAIBp06alvo7X2jzzzBO+al/L9V2fuj6L2B7LY9F0wDXXXBP2qFGjwn7llVcAAD/99FP4tHCXx63pngMOOCDsY489FkByzRZ9ferx8/rbbrvtwtenT5+wu3XrBqA615yex8ZsYEj727Q5BdLvJU0pUG8KekzvvPNO2MOHDw97v/32q+gxNQRHdIwxxhhTs/iHjjHGGGNqlkJSVwxpaWj8gQceCPuuu+4CAGy00Ubh404DoLTr6sEHHwzfQQcdFHYeIU0es4bu9HO+//77sEeMGAEAePHFF8P3xx9/hL3ZZpsBSB4zd0JM/77VSJrOkYZT00L/1bhTIit1cf/99wNIplPPPPPMsJk6/eqrr8J33HHHhX3yyScDAO69997wZYWhK0Va6uaTTz4J3w033BA2Q9K6ZpXFF18cAHD44YeHb6uttqrz/kAxY01bn3r/4Ly+++674Vt44YXD7tKlCwBgwQUXDJ+evzyuT54nPXYeBwCss846YTP1lKU3xmN96qmnwnfttdeGPeus//+2f9FFF9V5TVHoOmFqdKGFFgofr0mglLqqRrLWiaYkW7VqBaA0D1m8/PLLYfOaA4CVVloJQPHX1kcffRT2P//8Ezavm6LToYojOsYYY4ypWQqJ6PCJ/7TTTgvf/vvvH3b79u0BJH8ltm7dOux1110XQDKKkvcTCZ8Ix44dG77HH388bNX5WWuttQAAa665ZvhU84K/eN96663wrbjiinX+v+hfxHpONSLz448/AkgWg3/++edhf/jhhwCSEY9jjjkmbBabzz777OHTp588C3ezojjDhg0L+6STTgIA7LnnnuFjlAYoFXnqU5au37PPPhtAMoqgOjuVIktZ9tZbbwUAnHvuueGr71rSsZDXX3897EceeSTsjTfeOOxKRSf16fiXX34BANx0003hu/vuu8MeN24cgOQ1qRHVJZdcEkByzrt37x62FmGX+xrVcay33npNeg9eqxoF0ugdI3aNKZzNGz0W3utXXnnl8H333Xepf1st8Jh0Hen9/T//+U/YjK5ts802dV4PAL///jsA4MADDwxfu3btwr7zzjsBVHYDDteHfifrd6GuH0ZHs44pTXk+b+VuR3SMMcYYU7P4h44xxhhjapZCU1eLLLJI+Nq2bRt2mn6Hpk4YslNfHuEuff8nn3wSANC7d+/wHX/88WFrGk6bmaXBlIJqWzAdBAC33HILgGQxXqXCtVnFdFoMOHDgQADJRm5du3YNe9lllwWQLKa77bbbwt52220BAC+99FL4VltttbCXWGKJsMudxtJ0DlMYQLKwlqmL888/P3xzzDFH2Ex56PrQ88bUic5ppcLMekwqoX/zzTeHfdZZZwFIzq+mhunXMPyJJ54Y9pdffgmgpBcEJNNEmjJh4WW5xpylTaWFxaeeeiqAZGq1U6dOYbPwWlOrjz76aNjcLPDMM8+E77777gtbU35Ms+eRbtX3bExqifo6F198cfg0zcDrN+/7Z1PhsUyaNCl8mmZM2xhSdNNSHtOUKVPC9/zzz4et19eGG24IIHmcmhLivfLbb78NH8sh9LMqCT9T7ymffvpp2Gmpt6WWWqrO64GSThL/DgDmnXfesBdYYAEA5b2mHNExxhhjTM3iHzrGGGOMqVkKSV0xZMrdLUD9EtgMdwElfRqtys8jnKfhRO4w0XTTCSecELaG7hq604R6LECy0ytTKpray3v3CudEQ69XX3112Bqu5w4C6hkBydDjBRdcACCZgtIdTJdddhmApI6Hpgb22GOPsLPaEDQVXVNvvPFG2BpavvLKKwEAiy22WPjSQudZ71sE/Pyvv/46fHp96a4ozpUes+6q4lrr0KFD+DQ1yzSI7jrU1OaFF14YNtOvzU0ncH2qto+m47imgFKaUFNrmlrl+HVODz744Dqv1zD9PvvsE7aei2effbaxQ5khOieaGrjjjjvCZkg/K808evRoAMnj13YSnTt3BlBd6SodC9eiHv9OO+1U5/8HDRoUPtUkY6f2So6P63PixInhGzBgQNi661T1mYim8fv27Qsgee/bYIMNwub6rGRbGZ5LTeEvs8wyYb/22mth8ztEU6O664+aY9pCYumllw6bmmXbb799+Jo7Vkd0jDHGGFOz+IeOMcYYY2qWQruXZ7UFYJhM/19DgtxVoMJk5QrjZX0mUwLcsaDH2ZDP19Dsn3/+CSCZutGdKgwJ5h161TA5UwLnnXde+C6//PKwzznnnLC5A0d35WgaiFXzmvrSNAMFr3R8+l55oumKHXfcMWyV26fgVX1zqv+vncyZBstbJFDXKtendlx///33w9YdfGyxomlKFSfj9aXpEE1tUehR149K3Dd1t9D06Gv/+usvAMlWHNq2giKUQGlXmYpw6jFxDej7c82qX7t/a4sITWmWGz2n48ePD1uvJU2pp72OMAULAFtuuWXYvO6qKXWlx//NN98AKO3uA5LielzrmrrSXXVM8xTRSkfnRndK9erVK2xeK9qi48gjjwx71VVXBZAcv6aJeN3r+PLeiZWWutLSBE2z8Vr77LPPwqdpqN9++w1A8jtF20mwPERLIzT13JR164iOMcYYY2qWskd0srRF9Ncnn6j0iVJ/0VMGXV+jxaqUY097Siknevx8ilX59/qe+PVc6C9eao7oL3p9oq5PQrs56Jj0mPr16wcg+ZSkEZ2ePXuGzbnS+VG5euoIHXXUUeFTOXQ+FayyyirhY1uP6d+33Og51Sd2jb7UN69phb963jiXGgUo11zqmmLkBShF2dh+A0gWszOKA5SiVxrdeuKJJ8JmJGHMmDHh41O2/q3qYKywwgpha2uPco2bhc96nrXAk203gNJc6vjSyNJe4TnWKN3HH38cNnV6pn9dOdC1r8XgWuyZhkZMOdcaMdY52XvvvZt9nOVG70uMDusGFG238t577wFIbiDQKFuRkSq9d+j1P3ny5LCvu+46AMlGq6effnrYm266KYBSUTJQKiAH0iOSlULPrUZ0NJJF/akJEyaET++V1HfS7weNCLPdxxlnnBE+3ayjOnUNzeQ4omOMMcaYmsU/dIwxxhhTs5QtdcWQ1ttvvx0+ldjXMNfQoUMBJEPDqo/BMLiGa1944YWwKdev4WSVm9aURGPRUJiGSxkO189UbQMNOTO1oeF+DS0zJLvGGmuEb6ONNgqbYeb6Qu9NQcOdqjPCYkfVflBtkTTpfT0+TcMxjaIFvto6g4V3WgysOgp5jDuNxsjGp6VhNZ2qa5mdzLUTdR5jevXVV8OmTo62mmCIHEiGftmignojQEmWHiita+24rJouLBLUc6LFls1pd6GpOS3GZRi7Y8eO4WPRMZC85ptyrtPSAJru07lcfvnlG/3+DUXPl6bJ9V6RBrWrAGDfffcFkNT7oXYJAKy++uoAkunmIgp3da6Ay6shAAAKA0lEQVQ1TUXNH11/uqZ4r9ECeU3TFpm60hT4qFGjwlbNN97r2OoHSGqHUSeOrWiA5Pcn56qI1JV+P2qxtaaTqKOl16G2kOG9QtecplZ5Xeu1Tm0oANhuu+1Sj2dGOKJjjDHGmJrFP3SMMcYYU7M0K3WlIUKmPjRFo3vn99prr7B79OgBILlTg9oyQCn189xzz4VPQ+s77LADgFIIHkhW4Gsaq7HS/DomDaexU7nu7lDZcQ2d8zNVWyYtTaU7kbR7dx6hVx6TnjOVze/WrRuAZIsLPXdq//DDDwCS4XJNXZ188skAkjvJtEUA0zy77757+CopZ06yOmGn+fT4mEbV88d0AVDSwSjnPPJYdKecdofn8e23337hU+0RTQMxZaDh5rTUq4753nvvDZu7KXRN77rrrnXef/r3bQhZsvHciaE7iTSd0ZR0VVb3a3Y91/nVnVbsyD796xpL2vrTc67vXd/49LwxJaXdy3nPBErrRrW7KoWec9WJ0fNLTRXVUdHShhEjRgBIarrksdOvMXCd6+4v3SGo3wWcCz3/1JYBgPPPPx9AaXfx9BSRsiK6PpkCBYDNNtssbJamaOr1sMMOCzttV7W+L1tk6K5s3WHaFBzRMcYYY0zN0uiIjj45aIEin34effTR8LVv336G76W/4vTXOYs8GTkASr8SAaBt27YAkr8I09RCm4se30orrQQgWVT78MMPh62FY4yOrL322uHTiI+Oi2hTzDyfSDTipr+YqYis86DaBiyQA0rnQIuxNaLDYjstFlQVWxYpa4Fe3hEdPgXp+tWnZF0/LKJ/6KGHwqfRkw8++ABAsgD5wQcfDJtP/FoMqsrETZlfHr82Orz11lvDbteuHQCgf//+dY4DSEZP056o9Emb/htvvDF8WqTOtXr00UeHT9d/uQpbNcrL49dCxMZ8Tlp0Ut9/yJAhYXN9ayNJboAAkoXJzVm32hSVx6IFuHrMjfkcnhdVhtaNFUUW6+qYNGJH7RWgFLXU6F1aU2dtFFw0aWr+W2+9ddhbbbVV2DwHGpl59913w2YRs96rqwVdO/qdpcrOw4YNq/M6jUTzPfT+pBEtNpDWtaKbVayMbIwxxhgj+IeOMcYYY2qWRqeuNGyksvEsNtZ0VVZhXVoa4YEHHgibRVwammfqCEimVKZ/z7zg8ev4VIeiPjTMnhZ6q1Qxrp7T5ZZbLmxKlGvRqaZr1M+UjEp4a1NOjk9TY1999VXYLMLLe87SiolV22nw4MFha+sEFkN+//334dPQKs8bi66BZDqShfmaZmLbD6D+tTCjsehr9f1ZmJtVKJvWNFVDwzrWK664os4xa7sHpjm1cD3v9ctiaj1OTcelNRLV8bMpKFCaXy2mHz58eNi8l1111VXh0/PanLHqMbGVAVBKDeqcctMG0LjWM5xXTferppcW7lYaTRdvscUWYatmGouNNXWj3xVM02o6WK/1IlJzaWsuC86fHrOWBrD1hzaaraYGrETvRap5Qx0xvX9oypnF1tri5LHHHgubReiq/aQ6YE1JjTuiY4wxxpiaxT90jDHGGFOzNEtHRyuhqZNCvQsgW5abu3FUp2LixIlhU85eJeqLkChPo6lh6yK1D5SNN944bN01xnC9hs61Y67+LTWNsvRSGDr/8ccfw6fromvXrom/A/JJfej7v/nmmwCS2ke6PlXnaeeddwaQ7J6rndYpUa+pS921RZ0M3enSlHRVGlnriCkBTQ3o/Cjc4aOpDd1VRc0P3UmlO8gOPfRQAOXbfaTo+2hqiqm5yy+/PHxHHHFE6uu4Q+fZZ58N38iRI8Nm6kpTOJrS5fosV7pK0TWpu9bYWkbD9ZoO1TQx9Y/0vXT98XU6p7oWNt98cwDFp3v0mPT6oq3aTDo+Xpc6P9Vyf23IcfBvVBtId3jyvqvXVzWmrvSYVMeOWnOq3aVlELx/6Pe76rutv/76AIA+ffqEr7n3Gkd0jDHGGFOzNDqio7+mVLmRmgjUuAGANm3ahK1PJ3wSUW0KVZHkr8Mi1HIrQVpTTNWcyePphL++L7300vCpmi+bxqnej86fFrNyXup7yrjjjjvCVhVQ/mLPA33KVZ0UPh2ocqlqp2jEiSqeqvypT/yMBGVpN2mRHWluRJLnWnWONOL20ksvAUhqd+j1qeeCWlfUAwKSxa6nn346AOCEE04IX5rOUx7Xp54nbRpIbQ2NsmlTVV2LXAP6xKjvxaaDjNwByfGTPManx6nK1JdccgmA5PzSBwB33nln2IwE6cYIbUDKwmotINdIGO/R1XR/Tbs+tNhY7498+teNBZMmTQp70UUXBVBd40uDG0AA4P333w+b0TuNOFaq0XFT0fnjPZaZGSCpE8TsgG5Q0aalvO/oNdnc+6cjOsYYY4ypWfxDxxhjjDE1S7OKkTXFwqZsO+64Y/hGjx4dtobe2NRSWyTkXZhajagstup85AFD5houZysG/X9F56EpoUMtxtXUJTVpVCciD1RviZ+pqTlt20DtG6CUplEdDy1cru9c5FE4z7mYa665wterV6+wx4wZAyDZPJBS8kDy+mJKQFt4aEqTRdZp6cq8SUtBAaUGseuuu274vvjii7A1NcfWB5qm/H/t3c2NwjAQhuFB4udOAxSCRDMcOHKiFxqhHhqgAX4Oexrz7coJ4MQx8b7PKYp2BYOcEGbs8Wq1CsdeEmjaVHAo+vpeJvYeRmbP+6SZ2fF4DMeHw8HMfr9n3eDUJxvvdrtwTnvWjIV+Z+hkVC8Na+8VvZeORVPp30uOWq6L9cH6Vj4utQyrfXDW63Xr//t10ed9lIwOAACoFg86AACgWpO2lTPX6/XjxfuaYtTUs/LUVqmZ5IvFYmJmdrvdBmtOoGU+3/Va+7DoahJfjZa6I/t8Ph88PuWxXi6XcE5Xfez3ezP7Xdr6pHTQFl9s2wezZ88KHXM69vWz9tU6WuYasrTh8T0ej9b4NLXrK6i8H8tfuoLDV+ssl8twTkti/rnk6t0xnU4/Hp+xbWP0/emx/42e66uP0Tu6XH96z9T3qf2tXm2B4+NWSweqa/xt4zMXjc+vVb1mm2JN4ePzfr9nic9j0ZWem80mHPu9SncB15Jt13vRbDYr+v2Qm49PRUYHAABUiwcdAABQrd5LV2NQunTlK6y0CZ2mLn01Wuqs89KlK6cxx9rNp6bQ340v9vpNbe9j50ut/nu3NND0+aYYMtaU0tUrpbczUDmuv6ZpADG5S48lSlcqNtb7jDV36cppHOfzORyfTiczM9tut+Gclua6xkrpCgAAoCJkdApq6lPiE2ZTf6V/S0Ynl/8SX6lfzLnlyOh8E8bnuOXO6MS+c3WSvR/rZOum7HMKMjoAAAAV4UEHAABUq7V0BQAAMGZkdAAAQLV40AEAANXiQQcAAFSLBx0AAFAtHnQAAEC1eNABAADV+gEAUkmuGKQu1gAAAABJRU5ErkJggg==\n",
+ "text/plain": [
+ "
"
+ ]
+ },
+ "metadata": {
+ "needs_background": "light"
+ },
+ "output_type": "display_data"
+ }
+ ],
"source": [
"# Randomly select 100 data points to display\n",
"rand_indices = np.random.choice(m, 100, replace=False)\n",
@@ -157,7 +170,7 @@
},
{
"cell_type": "code",
- "execution_count": null,
+ "execution_count": 4,
"metadata": {},
"outputs": [],
"source": [
@@ -215,10 +228,19 @@
},
{
"cell_type": "code",
- "execution_count": null,
+ "execution_count": 5,
"metadata": {},
"outputs": [],
"source": [
+ "def sigmoidGradient(z):\n",
+ " g = np.zeros(z.shape)\n",
+ " A = utils.sigmoid(z)\n",
+ " g = A*(1-A)\n",
+ " return g\n",
+ "\n",
+ "\n",
+ "\n",
+ "\n",
"def nnCostFunction(nn_params,\n",
" input_layer_size,\n",
" hidden_layer_size,\n",
@@ -316,14 +338,49 @@
" Theta2_grad = np.zeros(Theta2.shape)\n",
"\n",
" # ====================== YOUR CODE HERE ======================\n",
+ " \n",
+ " a1 = np.concatenate([np.ones((m, 1)), X], axis=1)\n",
+ " \n",
+ " a2 = utils.sigmoid(a1.dot(Theta1.T))\n",
+ " a2 = np.concatenate([np.ones((a2.shape[0], 1)), a2], axis=1)\n",
+ " \n",
+ " a3 = utils.sigmoid(a2.dot(Theta2.T))\n",
+ " \n",
+ " y_matrix = y.reshape(-1)\n",
+ " y_matrix = np.eye(num_labels)[y_matrix]\n",
+ " \n",
+ " temp1 = Theta1\n",
+ " temp2 = Theta2\n",
+ " \n",
+ " # Add regularization term\n",
+ " \n",
+ " reg_term = (lambda_ / (2 * m)) * (np.sum(np.square(temp1[:, 1:])) + np.sum(np.square(temp2[:, 1:])))\n",
+ " \n",
+ " J = (-1 / m) * np.sum((np.log(a3) * y_matrix) + np.log(1 - a3) * (1 - y_matrix)) + reg_term\n",
+ " \n",
+ " # Backpropogation\n",
+ " \n",
+ " delta_3 = a3 - y_matrix\n",
+ " delta_2 = delta_3.dot(Theta2)[:, 1:] * sigmoidGradient(a1.dot(Theta1.T))\n",
"\n",
+ " Delta1 = delta_2.T.dot(a1)\n",
+ " Delta2 = delta_3.T.dot(a2)\n",
+ " \n",
+ " #Regularised Gradient\n",
+ " \n",
+ " Theta1_grad = (1 / m) * Delta1\n",
+ " Theta1_grad[:, 1:] = Theta1_grad[:, 1:] + (lambda_ / m) * Theta1[:, 1:]\n",
+ " \n",
+ " Theta2_grad = (1 / m) * Delta2\n",
+ " Theta2_grad[:, 1:] = Theta2_grad[:, 1:] + (lambda_ / m) * Theta2[:, 1:]\n",
+ " \n",
" \n",
" \n",
" # ================================================================\n",
" # Unroll gradients\n",
" # grad = np.concatenate([Theta1_grad.ravel(order=order), Theta2_grad.ravel(order=order)])\n",
" grad = np.concatenate([Theta1_grad.ravel(), Theta2_grad.ravel()])\n",
- "\n",
+ " \n",
" return J, grad"
]
},
@@ -351,9 +408,18 @@
},
{
"cell_type": "code",
- "execution_count": null,
+ "execution_count": 6,
"metadata": {},
- "outputs": [],
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "Cost at parameters (loaded from ex4weights): 0.287629 \n",
+ "The cost should be about : 0.287629.\n"
+ ]
+ }
+ ],
"source": [
"lambda_ = 0\n",
"J, _ = nnCostFunction(nn_params, input_layer_size, hidden_layer_size,\n",
@@ -371,9 +437,30 @@
},
{
"cell_type": "code",
- "execution_count": null,
+ "execution_count": 10,
"metadata": {},
- "outputs": [],
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "\n",
+ "Submitting Solutions | Programming Exercise neural-network-learning\n",
+ "\n",
+ "Use token from last successful submission (tanmayss2109@gmail.com)? (Y/n): y\n",
+ " Part Name | Score | Feedback\n",
+ " --------- | ----- | --------\n",
+ " Feedforward and Cost Function | 30 / 30 | Nice work!\n",
+ " Regularized Cost Function | 0 / 15 | \n",
+ " Sigmoid Gradient | 0 / 5 | \n",
+ " Neural Network Gradient (Backpropagation) | 0 / 40 | \n",
+ " Regularized Gradient | 0 / 10 | \n",
+ " --------------------------------\n",
+ " | 30 / 100 | \n",
+ "\n"
+ ]
+ }
+ ],
"source": [
"grader = utils.Grader()\n",
"grader[1] = nnCostFunction\n",
@@ -407,9 +494,18 @@
},
{
"cell_type": "code",
- "execution_count": null,
+ "execution_count": 8,
"metadata": {},
- "outputs": [],
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "Cost at parameters (loaded from ex4weights): 0.383770\n",
+ "This value should be about : 0.383770.\n"
+ ]
+ }
+ ],
"source": [
"# Weight regularization parameter (we set this to 1 here).\n",
"lambda_ = 1\n",
@@ -429,9 +525,30 @@
},
{
"cell_type": "code",
- "execution_count": null,
+ "execution_count": 11,
"metadata": {},
- "outputs": [],
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "\n",
+ "Submitting Solutions | Programming Exercise neural-network-learning\n",
+ "\n",
+ "Use token from last successful submission (tanmayss2109@gmail.com)? (Y/n): y\n",
+ " Part Name | Score | Feedback\n",
+ " --------- | ----- | --------\n",
+ " Feedforward and Cost Function | 30 / 30 | Nice work!\n",
+ " Regularized Cost Function | 15 / 15 | Nice work!\n",
+ " Sigmoid Gradient | 0 / 5 | \n",
+ " Neural Network Gradient (Backpropagation) | 0 / 40 | \n",
+ " Regularized Gradient | 0 / 10 | \n",
+ " --------------------------------\n",
+ " | 45 / 100 | \n",
+ "\n"
+ ]
+ }
+ ],
"source": [
"grader[2] = nnCostFunction\n",
"grader.grade()"
@@ -505,8 +622,9 @@
" g = np.zeros(z.shape)\n",
"\n",
" # ====================== YOUR CODE HERE ======================\n",
- "\n",
- "\n",
+ " A = utils.sigmoid(z)\n",
+ " g = A * (1-A)\n",
+ " \n",
"\n",
" # =============================================================\n",
" return g"
@@ -610,7 +728,7 @@
"\n",
" # ====================== YOUR CODE HERE ======================\n",
"\n",
- "\n",
+ " W = np.random.rand(L_out, 1 + L_in) * 2 * epsilon_init - epsilon_init\n",
"\n",
" # ============================================================\n",
" return W"
@@ -830,7 +948,7 @@
"\n",
"# Now, costFunction is a function that takes in only one argument\n",
"# (the neural network parameters)\n",
- "res = optimize.minimize(costFunction,\n",
+ "res = opt.minimize(costFunction,\n",
" initial_nn_params,\n",
" jac=True,\n",
" method='TNC',\n",
@@ -916,7 +1034,7 @@
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
- "version": "3.6.4"
+ "version": "3.7.6"
}
},
"nbformat": 4,
diff --git a/Phase 3 - 2020 (Summer)/Week 5(Apr 26-May 02)/Exercise4/token.pkl b/Phase 3 - 2020 (Summer)/Week 5(Apr 26-May 02)/Exercise4/token.pkl
new file mode 100644
index 000000000..0998c2ced
Binary files /dev/null and b/Phase 3 - 2020 (Summer)/Week 5(Apr 26-May 02)/Exercise4/token.pkl differ
diff --git a/Phase 3 - 2020 (Summer)/Week 5(Apr 26-May 02)/Exercise5/.ipynb_checkpoints/exercise5-checkpoint.ipynb b/Phase 3 - 2020 (Summer)/Week 5(Apr 26-May 02)/Exercise5/.ipynb_checkpoints/exercise5-checkpoint.ipynb
new file mode 100644
index 000000000..d48b7d90a
--- /dev/null
+++ b/Phase 3 - 2020 (Summer)/Week 5(Apr 26-May 02)/Exercise5/.ipynb_checkpoints/exercise5-checkpoint.ipynb
@@ -0,0 +1,927 @@
+{
+ "cells": [
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "# Programming Exercise 5:\n",
+ "# Regularized Linear Regression and Bias vs Variance\n",
+ "\n",
+ "## Introduction\n",
+ "\n",
+ "In this exercise, you will implement regularized linear regression and use it to study models with different bias-variance properties. Before starting on the programming exercise, we strongly recommend watching the video lectures and completing the review questions for the associated topics.\n",
+ "\n",
+ "All the information you need for solving this assignment is in this notebook, and all the code you will be implementing will take place within this notebook. The assignment can be promptly submitted to the coursera grader directly from this notebook (code and instructions are included below).\n",
+ "\n",
+ "Before we begin with the exercises, we need to import all libraries required for this programming exercise. Throughout the course, we will be using [`numpy`](http://www.numpy.org/) for all arrays and matrix operations, [`matplotlib`](https://matplotlib.org/) for plotting, and [`scipy`](https://docs.scipy.org/doc/scipy/reference/) for scientific and numerical computation functions and tools. You can find instructions on how to install required libraries in the README file in the [github repository](https://github.com/dibgerge/ml-coursera-python-assignments)."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "collapsed": true
+ },
+ "outputs": [],
+ "source": [
+ "# used for manipulating directory paths\n",
+ "import os\n",
+ "\n",
+ "# Scientific and vector computation for python\n",
+ "import numpy as np\n",
+ "\n",
+ "# Plotting library\n",
+ "from matplotlib import pyplot\n",
+ "\n",
+ "# Optimization module in scipy\n",
+ "from scipy import optimize\n",
+ "\n",
+ "# will be used to load MATLAB mat datafile format\n",
+ "from scipy.io import loadmat\n",
+ "\n",
+ "# library written for this exercise providing additional functions for assignment submission, and others\n",
+ "import utils\n",
+ "\n",
+ "# define the submission/grader object for this exercise\n",
+ "grader = utils.Grader()\n",
+ "\n",
+ "# tells matplotlib to embed plots within the notebook\n",
+ "%matplotlib inline"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "## Submission and Grading\n",
+ "\n",
+ "\n",
+ "After completing each part of the assignment, be sure to submit your solutions to the grader. The following is a breakdown of how each part of this exercise is scored.\n",
+ "\n",
+ "\n",
+ "| Section | Part | Submitted Function | Points |\n",
+ "| :- |:- |:- | :-: |\n",
+ "| 1 | [Regularized Linear Regression Cost Function](#section1) | [`linearRegCostFunction`](#linearRegCostFunction) | 25 |\n",
+ "| 2 | [Regularized Linear Regression Gradient](#section2) | [`linearRegCostFunction`](#linearRegCostFunction) |25 |\n",
+ "| 3 | [Learning Curve](#section3) | [`learningCurve`](#func2) | 20 |\n",
+ "| 4 | [Polynomial Feature Mapping](#section4) | [`polyFeatures`](#polyFeatures) | 10 |\n",
+ "| 5 | [Cross Validation Curve](#section5) | [`validationCurve`](#validationCurve) | 20 |\n",
+ "| | Total Points | |100 |\n",
+ "\n",
+ "\n",
+ "You are allowed to submit your solutions multiple times, and we will take only the highest score into consideration.\n",
+ "\n",
+ "
\n",
+ "At the end of each section in this notebook, we have a cell which contains code for submitting the solutions thus far to the grader. Execute the cell to see your score up to the current section. For all your work to be submitted properly, you must execute those cells at least once.\n",
+ "
"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "\n",
+ "## 1 Regularized Linear Regression\n",
+ "\n",
+ "In the first half of the exercise, you will implement regularized linear regression to predict the amount of water flowing out of a dam using the change of water level in a reservoir. In the next half, you will go through some diagnostics of debugging learning algorithms and examine the effects of bias v.s.\n",
+ "variance. \n",
+ "\n",
+ "### 1.1 Visualizing the dataset\n",
+ "\n",
+ "We will begin by visualizing the dataset containing historical records on the change in the water level, $x$, and the amount of water flowing out of the dam, $y$. This dataset is divided into three parts:\n",
+ "\n",
+ "- A **training** set that your model will learn on: `X`, `y`\n",
+ "- A **cross validation** set for determining the regularization parameter: `Xval`, `yval`\n",
+ "- A **test** set for evaluating performance. These are “unseen” examples which your model did not see during training: `Xtest`, `ytest`\n",
+ "\n",
+ "Run the next cell to plot the training data. In the following parts, you will implement linear regression and use that to fit a straight line to the data and plot learning curves. Following that, you will implement polynomial regression to find a better fit to the data."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "# Load from ex5data1.mat, where all variables will be store in a dictionary\n",
+ "data = loadmat(os.path.join('Data', 'ex5data1.mat'))\n",
+ "\n",
+ "# Extract train, test, validation data from dictionary\n",
+ "# and also convert y's form 2-D matrix (MATLAB format) to a numpy vector\n",
+ "X, y = data['X'], data['y'][:, 0]\n",
+ "Xtest, ytest = data['Xtest'], data['ytest'][:, 0]\n",
+ "Xval, yval = data['Xval'], data['yval'][:, 0]\n",
+ "\n",
+ "# m = Number of examples\n",
+ "m = y.size\n",
+ "\n",
+ "# Plot training data\n",
+ "pyplot.plot(X, y, 'ro', ms=10, mec='k', mew=1)\n",
+ "pyplot.xlabel('Change in water level (x)')\n",
+ "pyplot.ylabel('Water flowing out of the dam (y)');"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "### 1.2 Regularized linear regression cost function\n",
+ "\n",
+ "Recall that regularized linear regression has the following cost function:\n",
+ "\n",
+ "$$ J(\\theta) = \\frac{1}{2m} \\left( \\sum_{i=1}^m \\left( h_\\theta\\left( x^{(i)} \\right) - y^{(i)} \\right)^2 \\right) + \\frac{\\lambda}{2m} \\left( \\sum_{j=1}^n \\theta_j^2 \\right)$$\n",
+ "\n",
+ "where $\\lambda$ is a regularization parameter which controls the degree of regularization (thus, help preventing overfitting). The regularization term puts a penalty on the overall cost J. As the magnitudes of the model parameters $\\theta_j$ increase, the penalty increases as well. Note that you should not regularize\n",
+ "the $\\theta_0$ term.\n",
+ "\n",
+ "You should now complete the code in the function `linearRegCostFunction` in the next cell. Your task is to calculate the regularized linear regression cost function. If possible, try to vectorize your code and avoid writing loops.\n",
+ ""
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "collapsed": true
+ },
+ "outputs": [],
+ "source": [
+ "def linearRegCostFunction(X, y, theta, lambda_=0.0):\n",
+ " \"\"\"\n",
+ " Compute cost and gradient for regularized linear regression \n",
+ " with multiple variables. Computes the cost of using theta as\n",
+ " the parameter for linear regression to fit the data points in X and y. \n",
+ " \n",
+ " Parameters\n",
+ " ----------\n",
+ " X : array_like\n",
+ " The dataset. Matrix with shape (m x n + 1) where m is the \n",
+ " total number of examples, and n is the number of features \n",
+ " before adding the bias term.\n",
+ " \n",
+ " y : array_like\n",
+ " The functions values at each datapoint. A vector of\n",
+ " shape (m, ).\n",
+ " \n",
+ " theta : array_like\n",
+ " The parameters for linear regression. A vector of shape (n+1,).\n",
+ " \n",
+ " lambda_ : float, optional\n",
+ " The regularization parameter.\n",
+ " \n",
+ " Returns\n",
+ " -------\n",
+ " J : float\n",
+ " The computed cost function. \n",
+ " \n",
+ " grad : array_like\n",
+ " The value of the cost function gradient w.r.t theta. \n",
+ " A vector of shape (n+1, ).\n",
+ " \n",
+ " Instructions\n",
+ " ------------\n",
+ " Compute the cost and gradient of regularized linear regression for\n",
+ " a particular choice of theta.\n",
+ " You should set J to the cost and grad to the gradient.\n",
+ " \"\"\"\n",
+ " # Initialize some useful values\n",
+ " m = y.size # number of training examples\n",
+ "\n",
+ " # You need to return the following variables correctly \n",
+ " J = 0\n",
+ " grad = np.zeros(theta.shape)\n",
+ "\n",
+ " # ====================== YOUR CODE HERE ======================\n",
+ "\n",
+ "\n",
+ "\n",
+ " # ============================================================\n",
+ " return J, grad"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "When you are finished, the next cell will run your cost function using `theta` initialized at `[1, 1]`. You should expect to see an output of 303.993."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "theta = np.array([1, 1])\n",
+ "J, _ = linearRegCostFunction(np.concatenate([np.ones((m, 1)), X], axis=1), y, theta, 1)\n",
+ "\n",
+ "print('Cost at theta = [1, 1]:\\t %f ' % J)\n",
+ "print('This value should be about 303.993192)\\n' % J)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "After completing a part of the exercise, you can submit your solutions for grading by first adding the function you modified to the submission object, and then sending your function to Coursera for grading. \n",
+ "\n",
+ "The submission script will prompt you for your login e-mail and submission token. You can obtain a submission token from the web page for the assignment. You are allowed to submit your solutions multiple times, and we will take only the highest score into consideration.\n",
+ "\n",
+ "*Execute the following cell to grade your solution to the first part of this exercise.*"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "grader[1] = linearRegCostFunction\n",
+ "grader.grade()"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "\n",
+ "### 1.3 Regularized linear regression gradient\n",
+ "\n",
+ "Correspondingly, the partial derivative of the cost function for regularized linear regression is defined as:\n",
+ "\n",
+ "$$\n",
+ "\\begin{align}\n",
+ "& \\frac{\\partial J(\\theta)}{\\partial \\theta_0} = \\frac{1}{m} \\sum_{i=1}^m \\left( h_\\theta \\left(x^{(i)} \\right) - y^{(i)} \\right) x_j^{(i)} & \\qquad \\text{for } j = 0 \\\\\n",
+ "& \\frac{\\partial J(\\theta)}{\\partial \\theta_j} = \\left( \\frac{1}{m} \\sum_{i=1}^m \\left( h_\\theta \\left( x^{(i)} \\right) - y^{(i)} \\right) x_j^{(i)} \\right) + \\frac{\\lambda}{m} \\theta_j & \\qquad \\text{for } j \\ge 1\n",
+ "\\end{align}\n",
+ "$$\n",
+ "\n",
+ "In the function [`linearRegCostFunction`](#linearRegCostFunction) above, add code to calculate the gradient, returning it in the variable `grad`. Do not forget to re-execute the cell containing this function to update the function's definition.\n",
+ "\n",
+ "\n",
+ "When you are finished, use the next cell to run your gradient function using theta initialized at `[1, 1]`. You should expect to see a gradient of `[-15.30, 598.250]`."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "theta = np.array([1, 1])\n",
+ "J, grad = linearRegCostFunction(np.concatenate([np.ones((m, 1)), X], axis=1), y, theta, 1)\n",
+ "\n",
+ "print('Gradient at theta = [1, 1]: [{:.6f}, {:.6f}] '.format(*grad))\n",
+ "print(' (this value should be about [-15.303016, 598.250744])\\n')"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "*You should now submit your solutions.*"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "grader[2] = linearRegCostFunction\n",
+ "grader.grade()"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "### Fitting linear regression\n",
+ "\n",
+ "Once your cost function and gradient are working correctly, the next cell will run the code in `trainLinearReg` (found in the module `utils.py`) to compute the optimal values of $\\theta$. This training function uses `scipy`'s optimization module to minimize the cost function.\n",
+ "\n",
+ "In this part, we set regularization parameter $\\lambda$ to zero. Because our current implementation of linear regression is trying to fit a 2-dimensional $\\theta$, regularization will not be incredibly helpful for a $\\theta$ of such low dimension. In the later parts of the exercise, you will be using polynomial regression with regularization.\n",
+ "\n",
+ "Finally, the code in the next cell should also plot the best fit line, which should look like the figure below. \n",
+ "\n",
+ "\n",
+ "\n",
+ "The best fit line tells us that the model is not a good fit to the data because the data has a non-linear pattern. While visualizing the best fit as shown is one possible way to debug your learning algorithm, it is not always easy to visualize the data and model. In the next section, you will implement a function to generate learning curves that can help you debug your learning algorithm even if it is not easy to visualize the\n",
+ "data."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "# add a columns of ones for the y-intercept\n",
+ "X_aug = np.concatenate([np.ones((m, 1)), X], axis=1)\n",
+ "theta = utils.trainLinearReg(linearRegCostFunction, X_aug, y, lambda_=0)\n",
+ "\n",
+ "# Plot fit over the data\n",
+ "pyplot.plot(X, y, 'ro', ms=10, mec='k', mew=1.5)\n",
+ "pyplot.xlabel('Change in water level (x)')\n",
+ "pyplot.ylabel('Water flowing out of the dam (y)')\n",
+ "pyplot.plot(X, np.dot(X_aug, theta), '--', lw=2);"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "\n",
+ "## 2 Bias-variance\n",
+ "\n",
+ "An important concept in machine learning is the bias-variance tradeoff. Models with high bias are not complex enough for the data and tend to underfit, while models with high variance overfit to the training data.\n",
+ "\n",
+ "In this part of the exercise, you will plot training and test errors on a learning curve to diagnose bias-variance problems.\n",
+ "\n",
+ "### 2.1 Learning Curves\n",
+ "\n",
+ "You will now implement code to generate the learning curves that will be useful in debugging learning algorithms. Recall that a learning curve plots training and cross validation error as a function of training set size. Your job is to fill in the function `learningCurve` in the next cell, so that it returns a vector of errors for the training set and cross validation set.\n",
+ "\n",
+ "To plot the learning curve, we need a training and cross validation set error for different training set sizes. To obtain different training set sizes, you should use different subsets of the original training set `X`. Specifically, for a training set size of $i$, you should use the first $i$ examples (i.e., `X[:i, :]`\n",
+ "and `y[:i]`).\n",
+ "\n",
+ "You can use the `trainLinearReg` function (by calling `utils.trainLinearReg(...)`) to find the $\\theta$ parameters. Note that the `lambda_` is passed as a parameter to the `learningCurve` function.\n",
+ "After learning the $\\theta$ parameters, you should compute the error on the training and cross validation sets. Recall that the training error for a dataset is defined as\n",
+ "\n",
+ "$$ J_{\\text{train}} = \\frac{1}{2m} \\left[ \\sum_{i=1}^m \\left(h_\\theta \\left( x^{(i)} \\right) - y^{(i)} \\right)^2 \\right] $$\n",
+ "\n",
+ "In particular, note that the training error does not include the regularization term. One way to compute the training error is to use your existing cost function and set $\\lambda$ to 0 only when using it to compute the training error and cross validation error. When you are computing the training set error, make sure you compute it on the training subset (i.e., `X[:n,:]` and `y[:n]`) instead of the entire training set. However, for the cross validation error, you should compute it over the entire cross validation set. You should store\n",
+ "the computed errors in the vectors error train and error val.\n",
+ "\n",
+ ""
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "collapsed": true
+ },
+ "outputs": [],
+ "source": [
+ "def learningCurve(X, y, Xval, yval, lambda_=0):\n",
+ " \"\"\"\n",
+ " Generates the train and cross validation set errors needed to plot a learning curve\n",
+ " returns the train and cross validation set errors for a learning curve. \n",
+ " \n",
+ " In this function, you will compute the train and test errors for\n",
+ " dataset sizes from 1 up to m. In practice, when working with larger\n",
+ " datasets, you might want to do this in larger intervals.\n",
+ " \n",
+ " Parameters\n",
+ " ----------\n",
+ " X : array_like\n",
+ " The training dataset. Matrix with shape (m x n + 1) where m is the \n",
+ " total number of examples, and n is the number of features \n",
+ " before adding the bias term.\n",
+ " \n",
+ " y : array_like\n",
+ " The functions values at each training datapoint. A vector of\n",
+ " shape (m, ).\n",
+ " \n",
+ " Xval : array_like\n",
+ " The validation dataset. Matrix with shape (m_val x n + 1) where m is the \n",
+ " total number of examples, and n is the number of features \n",
+ " before adding the bias term.\n",
+ " \n",
+ " yval : array_like\n",
+ " The functions values at each validation datapoint. A vector of\n",
+ " shape (m_val, ).\n",
+ " \n",
+ " lambda_ : float, optional\n",
+ " The regularization parameter.\n",
+ " \n",
+ " Returns\n",
+ " -------\n",
+ " error_train : array_like\n",
+ " A vector of shape m. error_train[i] contains the training error for\n",
+ " i examples.\n",
+ " error_val : array_like\n",
+ " A vecotr of shape m. error_val[i] contains the validation error for\n",
+ " i training examples.\n",
+ " \n",
+ " Instructions\n",
+ " ------------\n",
+ " Fill in this function to return training errors in error_train and the\n",
+ " cross validation errors in error_val. i.e., error_train[i] and \n",
+ " error_val[i] should give you the errors obtained after training on i examples.\n",
+ " \n",
+ " Notes\n",
+ " -----\n",
+ " - You should evaluate the training error on the first i training\n",
+ " examples (i.e., X[:i, :] and y[:i]).\n",
+ " \n",
+ " For the cross-validation error, you should instead evaluate on\n",
+ " the _entire_ cross validation set (Xval and yval).\n",
+ " \n",
+ " - If you are using your cost function (linearRegCostFunction) to compute\n",
+ " the training and cross validation error, you should call the function with\n",
+ " the lambda argument set to 0. Do note that you will still need to use\n",
+ " lambda when running the training to obtain the theta parameters.\n",
+ " \n",
+ " Hint\n",
+ " ----\n",
+ " You can loop over the examples with the following:\n",
+ " \n",
+ " for i in range(1, m+1):\n",
+ " # Compute train/cross validation errors using training examples \n",
+ " # X[:i, :] and y[:i], storing the result in \n",
+ " # error_train[i-1] and error_val[i-1]\n",
+ " .... \n",
+ " \"\"\"\n",
+ " # Number of training examples\n",
+ " m = y.size\n",
+ "\n",
+ " # You need to return these values correctly\n",
+ " error_train = np.zeros(m)\n",
+ " error_val = np.zeros(m)\n",
+ "\n",
+ " # ====================== YOUR CODE HERE ======================\n",
+ " \n",
+ "\n",
+ " \n",
+ " # =============================================================\n",
+ " return error_train, error_val"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "When you are finished implementing the function `learningCurve`, executing the next cell prints the learning curves and produce a plot similar to the figure below. \n",
+ "\n",
+ "\n",
+ "\n",
+ "In the learning curve figure, you can observe that both the train error and cross validation error are high when the number of training examples is increased. This reflects a high bias problem in the model - the linear regression model is too simple and is unable to fit our dataset well. In the next section, you will implement polynomial regression to fit a better model for this dataset."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "X_aug = np.concatenate([np.ones((m, 1)), X], axis=1)\n",
+ "Xval_aug = np.concatenate([np.ones((yval.size, 1)), Xval], axis=1)\n",
+ "error_train, error_val = learningCurve(X_aug, y, Xval_aug, yval, lambda_=0)\n",
+ "\n",
+ "pyplot.plot(np.arange(1, m+1), error_train, np.arange(1, m+1), error_val, lw=2)\n",
+ "pyplot.title('Learning curve for linear regression')\n",
+ "pyplot.legend(['Train', 'Cross Validation'])\n",
+ "pyplot.xlabel('Number of training examples')\n",
+ "pyplot.ylabel('Error')\n",
+ "pyplot.axis([0, 13, 0, 150])\n",
+ "\n",
+ "print('# Training Examples\\tTrain Error\\tCross Validation Error')\n",
+ "for i in range(m):\n",
+ " print(' \\t%d\\t\\t%f\\t%f' % (i+1, error_train[i], error_val[i]))"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "*You should now submit your solutions.*"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "grader[3] = learningCurve\n",
+ "grader.grade()"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "\n",
+ "\n",
+ "## 3 Polynomial regression\n",
+ "\n",
+ "The problem with our linear model was that it was too simple for the data\n",
+ "and resulted in underfitting (high bias). In this part of the exercise, you will address this problem by adding more features. For polynomial regression, our hypothesis has the form:\n",
+ "\n",
+ "$$\n",
+ "\\begin{align}\n",
+ "h_\\theta(x) &= \\theta_0 + \\theta_1 \\times (\\text{waterLevel}) + \\theta_2 \\times (\\text{waterLevel})^2 + \\cdots + \\theta_p \\times (\\text{waterLevel})^p \\\\\n",
+ "& = \\theta_0 + \\theta_1 x_1 + \\theta_2 x_2 + \\cdots + \\theta_p x_p\n",
+ "\\end{align}\n",
+ "$$\n",
+ "\n",
+ "Notice that by defining $x_1 = (\\text{waterLevel})$, $x_2 = (\\text{waterLevel})^2$ , $\\cdots$, $x_p =\n",
+ "(\\text{waterLevel})^p$, we obtain a linear regression model where the features are the various powers of the original value (waterLevel).\n",
+ "\n",
+ "Now, you will add more features using the higher powers of the existing feature $x$ in the dataset. Your task in this part is to complete the code in the function `polyFeatures` in the next cell. The function should map the original training set $X$ of size $m \\times 1$ into its higher powers. Specifically, when a training set $X$ of size $m \\times 1$ is passed into the function, the function should return a $m \\times p$ matrix `X_poly`, where column 1 holds the original values of X, column 2 holds the values of $X^2$, column 3 holds the values of $X^3$, and so on. Note that you don’t have to account for the zero-eth power in this function.\n",
+ "\n",
+ ""
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "collapsed": true
+ },
+ "outputs": [],
+ "source": [
+ "def polyFeatures(X, p):\n",
+ " \"\"\"\n",
+ " Maps X (1D vector) into the p-th power.\n",
+ " \n",
+ " Parameters\n",
+ " ----------\n",
+ " X : array_like\n",
+ " A data vector of size m, where m is the number of examples.\n",
+ " \n",
+ " p : int\n",
+ " The polynomial power to map the features. \n",
+ " \n",
+ " Returns \n",
+ " -------\n",
+ " X_poly : array_like\n",
+ " A matrix of shape (m x p) where p is the polynomial \n",
+ " power and m is the number of examples. That is:\n",
+ " \n",
+ " X_poly[i, :] = [X[i], X[i]**2, X[i]**3 ... X[i]**p]\n",
+ " \n",
+ " Instructions\n",
+ " ------------\n",
+ " Given a vector X, return a matrix X_poly where the p-th column of\n",
+ " X contains the values of X to the p-th power.\n",
+ " \"\"\"\n",
+ " # You need to return the following variables correctly.\n",
+ " X_poly = np.zeros((X.shape[0], p))\n",
+ "\n",
+ " # ====================== YOUR CODE HERE ======================\n",
+ "\n",
+ "\n",
+ "\n",
+ " # ============================================================\n",
+ " return X_poly"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Now you have a function that will map features to a higher dimension. The next cell will apply it to the training set, the test set, and the cross validation set."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "p = 8\n",
+ "\n",
+ "# Map X onto Polynomial Features and Normalize\n",
+ "X_poly = polyFeatures(X, p)\n",
+ "X_poly, mu, sigma = utils.featureNormalize(X_poly)\n",
+ "X_poly = np.concatenate([np.ones((m, 1)), X_poly], axis=1)\n",
+ "\n",
+ "# Map X_poly_test and normalize (using mu and sigma)\n",
+ "X_poly_test = polyFeatures(Xtest, p)\n",
+ "X_poly_test -= mu\n",
+ "X_poly_test /= sigma\n",
+ "X_poly_test = np.concatenate([np.ones((ytest.size, 1)), X_poly_test], axis=1)\n",
+ "\n",
+ "# Map X_poly_val and normalize (using mu and sigma)\n",
+ "X_poly_val = polyFeatures(Xval, p)\n",
+ "X_poly_val -= mu\n",
+ "X_poly_val /= sigma\n",
+ "X_poly_val = np.concatenate([np.ones((yval.size, 1)), X_poly_val], axis=1)\n",
+ "\n",
+ "print('Normalized Training Example 1:')\n",
+ "X_poly[0, :]"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "*You should now submit your solutions.*"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "grader[4] = polyFeatures\n",
+ "grader.grade()"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "## 3.1 Learning Polynomial Regression\n",
+ "\n",
+ "After you have completed the function `polyFeatures`, we will proceed to train polynomial regression using your linear regression cost function.\n",
+ "\n",
+ "Keep in mind that even though we have polynomial terms in our feature vector, we are still solving a linear regression optimization problem. The polynomial terms have simply turned into features that we can use for linear regression. We are using the same cost function and gradient that you wrote for the earlier part of this exercise.\n",
+ "\n",
+ "For this part of the exercise, you will be using a polynomial of degree 8. It turns out that if we run the training directly on the projected data, will not work well as the features would be badly scaled (e.g., an example with $x = 40$ will now have a feature $x_8 = 40^8 = 6.5 \\times 10^{12}$). Therefore, you will\n",
+ "need to use feature normalization.\n",
+ "\n",
+ "Before learning the parameters $\\theta$ for the polynomial regression, we first call `featureNormalize` and normalize the features of the training set, storing the mu, sigma parameters separately. We have already implemented this function for you (in `utils.py` module) and it is the same function from the first exercise.\n",
+ "\n",
+ "After learning the parameters $\\theta$, you should see two plots generated for polynomial regression with $\\lambda = 0$, which should be similar to the ones here:\n",
+ "\n",
+ "
\n",
+ "
\n",
+ "
\n",
+ "
\n",
+ "
\n",
+ "
\n",
+ "\n",
+ "You should see that the polynomial fit is able to follow the datapoints very well, thus, obtaining a low training error. The figure on the right shows that the training error essentially stays zero for all numbers of training samples. However, the polynomial fit is very complex and even drops off at the extremes. This is an indicator that the polynomial regression model is overfitting the training data and will not generalize well.\n",
+ "\n",
+ "To better understand the problems with the unregularized ($\\lambda = 0$) model, you can see that the learning curve shows the same effect where the training error is low, but the cross validation error is high. There is a gap between the training and cross validation errors, indicating a high variance problem."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "lambda_ = 100\n",
+ "theta = utils.trainLinearReg(linearRegCostFunction, X_poly, y,\n",
+ " lambda_=lambda_, maxiter=55)\n",
+ "\n",
+ "# Plot training data and fit\n",
+ "pyplot.plot(X, y, 'ro', ms=10, mew=1.5, mec='k')\n",
+ "\n",
+ "utils.plotFit(polyFeatures, np.min(X), np.max(X), mu, sigma, theta, p)\n",
+ "\n",
+ "pyplot.xlabel('Change in water level (x)')\n",
+ "pyplot.ylabel('Water flowing out of the dam (y)')\n",
+ "pyplot.title('Polynomial Regression Fit (lambda = %f)' % lambda_)\n",
+ "pyplot.ylim([-20, 50])\n",
+ "\n",
+ "pyplot.figure()\n",
+ "error_train, error_val = learningCurve(X_poly, y, X_poly_val, yval, lambda_)\n",
+ "pyplot.plot(np.arange(1, 1+m), error_train, np.arange(1, 1+m), error_val)\n",
+ "\n",
+ "pyplot.title('Polynomial Regression Learning Curve (lambda = %f)' % lambda_)\n",
+ "pyplot.xlabel('Number of training examples')\n",
+ "pyplot.ylabel('Error')\n",
+ "pyplot.axis([0, 13, 0, 100])\n",
+ "pyplot.legend(['Train', 'Cross Validation'])\n",
+ "\n",
+ "print('Polynomial Regression (lambda = %f)\\n' % lambda_)\n",
+ "print('# Training Examples\\tTrain Error\\tCross Validation Error')\n",
+ "for i in range(m):\n",
+ " print(' \\t%d\\t\\t%f\\t%f' % (i+1, error_train[i], error_val[i]))"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "One way to combat the overfitting (high-variance) problem is to add regularization to the model. In the next section, you will get to try different $\\lambda$ parameters to see how regularization can lead to a better model.\n",
+ "\n",
+ "### 3.2 Optional (ungraded) exercise: Adjusting the regularization parameter\n",
+ "\n",
+ "In this section, you will get to observe how the regularization parameter affects the bias-variance of regularized polynomial regression. You should now modify the the lambda parameter and try $\\lambda = 1, 100$. For each of these values, the script should generate a polynomial fit to the data and also a learning curve.\n",
+ "\n",
+ "For $\\lambda = 1$, the generated plots should look like the the figure below. You should see a polynomial fit that follows the data trend well (left) and a learning curve (right) showing that both the cross validation and training error converge to a relatively low value. This shows the $\\lambda = 1$ regularized polynomial regression model does not have the high-bias or high-variance problems. In effect, it achieves a good trade-off between bias and variance.\n",
+ "\n",
+ "
\n",
+ "
\n",
+ "
\n",
+ "
\n",
+ "
\n",
+ "
\n",
+ "\n",
+ "For $\\lambda = 100$, you should see a polynomial fit (figure below) that does not follow the data well. In this case, there is too much regularization and the model is unable to fit the training data.\n",
+ "\n",
+ "\n",
+ "\n",
+ "*You do not need to submit any solutions for this optional (ungraded) exercise.*"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "\n",
+ "### 3.3 Selecting $\\lambda$ using a cross validation set\n",
+ "\n",
+ "From the previous parts of the exercise, you observed that the value of $\\lambda$ can significantly affect the results of regularized polynomial regression on the training and cross validation set. In particular, a model without regularization ($\\lambda = 0$) fits the training set well, but does not generalize. Conversely, a model with too much regularization ($\\lambda = 100$) does not fit the training set and testing set well. A good choice of $\\lambda$ (e.g., $\\lambda = 1$) can provide a good fit to the data.\n",
+ "\n",
+ "In this section, you will implement an automated method to select the $\\lambda$ parameter. Concretely, you will use a cross validation set to evaluate how good each $\\lambda$ value is. After selecting the best $\\lambda$ value using the cross validation set, we can then evaluate the model on the test set to estimate\n",
+ "how well the model will perform on actual unseen data. \n",
+ "\n",
+ "Your task is to complete the code in the function `validationCurve`. Specifically, you should should use the `utils.trainLinearReg` function to train the model using different values of $\\lambda$ and compute the training error and cross validation error. You should try $\\lambda$ in the following range: {0, 0.001, 0.003, 0.01, 0.03, 0.1, 0.3, 1, 3, 10}.\n",
+ ""
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "collapsed": true
+ },
+ "outputs": [],
+ "source": [
+ "def validationCurve(X, y, Xval, yval):\n",
+ " \"\"\"\n",
+ " Generate the train and validation errors needed to plot a validation\n",
+ " curve that we can use to select lambda_.\n",
+ " \n",
+ " Parameters\n",
+ " ----------\n",
+ " X : array_like\n",
+ " The training dataset. Matrix with shape (m x n) where m is the \n",
+ " total number of training examples, and n is the number of features \n",
+ " including any polynomial features.\n",
+ " \n",
+ " y : array_like\n",
+ " The functions values at each training datapoint. A vector of\n",
+ " shape (m, ).\n",
+ " \n",
+ " Xval : array_like\n",
+ " The validation dataset. Matrix with shape (m_val x n) where m is the \n",
+ " total number of validation examples, and n is the number of features \n",
+ " including any polynomial features.\n",
+ " \n",
+ " yval : array_like\n",
+ " The functions values at each validation datapoint. A vector of\n",
+ " shape (m_val, ).\n",
+ " \n",
+ " Returns\n",
+ " -------\n",
+ " lambda_vec : list\n",
+ " The values of the regularization parameters which were used in \n",
+ " cross validation.\n",
+ " \n",
+ " error_train : list\n",
+ " The training error computed at each value for the regularization\n",
+ " parameter.\n",
+ " \n",
+ " error_val : list\n",
+ " The validation error computed at each value for the regularization\n",
+ " parameter.\n",
+ " \n",
+ " Instructions\n",
+ " ------------\n",
+ " Fill in this function to return training errors in `error_train` and\n",
+ " the validation errors in `error_val`. The vector `lambda_vec` contains\n",
+ " the different lambda parameters to use for each calculation of the\n",
+ " errors, i.e, `error_train[i]`, and `error_val[i]` should give you the\n",
+ " errors obtained after training with `lambda_ = lambda_vec[i]`.\n",
+ "\n",
+ " Note\n",
+ " ----\n",
+ " You can loop over lambda_vec with the following:\n",
+ " \n",
+ " for i in range(len(lambda_vec))\n",
+ " lambda = lambda_vec[i]\n",
+ " # Compute train / val errors when training linear \n",
+ " # regression with regularization parameter lambda_\n",
+ " # You should store the result in error_train[i]\n",
+ " # and error_val[i]\n",
+ " ....\n",
+ " \"\"\"\n",
+ " # Selected values of lambda (you should not change this)\n",
+ " lambda_vec = [0, 0.001, 0.003, 0.01, 0.03, 0.1, 0.3, 1, 3, 10]\n",
+ "\n",
+ " # You need to return these variables correctly.\n",
+ " error_train = np.zeros(len(lambda_vec))\n",
+ " error_val = np.zeros(len(lambda_vec))\n",
+ "\n",
+ " # ====================== YOUR CODE HERE ======================\n",
+ "\n",
+ "\n",
+ "\n",
+ " # ============================================================\n",
+ " return lambda_vec, error_train, error_val"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "After you have completed the code, the next cell will run your function and plot a cross validation curve of error v.s. $\\lambda$ that allows you select which $\\lambda$ parameter to use. You should see a plot similar to the figure below. \n",
+ "\n",
+ "\n",
+ "\n",
+ "In this figure, we can see that the best value of $\\lambda$ is around 3. Due to randomness\n",
+ "in the training and validation splits of the dataset, the cross validation error can sometimes be lower than the training error."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "lambda_vec, error_train, error_val = validationCurve(X_poly, y, X_poly_val, yval)\n",
+ "\n",
+ "pyplot.plot(lambda_vec, error_train, '-o', lambda_vec, error_val, '-o', lw=2)\n",
+ "pyplot.legend(['Train', 'Cross Validation'])\n",
+ "pyplot.xlabel('lambda')\n",
+ "pyplot.ylabel('Error')\n",
+ "\n",
+ "print('lambda\\t\\tTrain Error\\tValidation Error')\n",
+ "for i in range(len(lambda_vec)):\n",
+ " print(' %f\\t%f\\t%f' % (lambda_vec[i], error_train[i], error_val[i]))"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "*You should now submit your solutions.*"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "grader[5] = validationCurve\n",
+ "grader.grade()"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "### 3.4 Optional (ungraded) exercise: Computing test set error\n",
+ "\n",
+ "In the previous part of the exercise, you implemented code to compute the cross validation error for various values of the regularization parameter $\\lambda$. However, to get a better indication of the model’s performance in the real world, it is important to evaluate the “final” model on a test set that was not used in any part of training (that is, it was neither used to select the $\\lambda$ parameters, nor to learn the model parameters $\\theta$). For this optional (ungraded) exercise, you should compute the test error using the best value of $\\lambda$ you found. In our cross validation, we obtained a test error of 3.8599 for $\\lambda = 3$.\n",
+ "\n",
+ "*You do not need to submit any solutions for this optional (ungraded) exercise.*"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": []
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "### 3.5 Optional (ungraded) exercise: Plotting learning curves with randomly selected examples\n",
+ "\n",
+ "In practice, especially for small training sets, when you plot learning curves to debug your algorithms, it is often helpful to average across multiple sets of randomly selected examples to determine the training error and cross validation error.\n",
+ "\n",
+ "Concretely, to determine the training error and cross validation error for $i$ examples, you should first randomly select $i$ examples from the training set and $i$ examples from the cross validation set. You will then learn the parameters $\\theta$ using the randomly chosen training set and evaluate the parameters $\\theta$ on the randomly chosen training set and cross validation set. The above steps should then be repeated multiple times (say 50) and the averaged error should be used to determine the training error and cross validation error for $i$ examples.\n",
+ "\n",
+ "For this optional (ungraded) exercise, you should implement the above strategy for computing the learning curves. For reference, the figure below shows the learning curve we obtained for polynomial regression with $\\lambda = 0.01$. Your figure may differ slightly due to the random selection of examples.\n",
+ "\n",
+ "\n",
+ "\n",
+ "*You do not need to submit any solutions for this optional (ungraded) exercise.*"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "collapsed": true
+ },
+ "outputs": [],
+ "source": []
+ }
+ ],
+ "metadata": {
+ "kernelspec": {
+ "display_name": "Python 3",
+ "language": "python",
+ "name": "python3"
+ },
+ "language_info": {
+ "codemirror_mode": {
+ "name": "ipython",
+ "version": 3
+ },
+ "file_extension": ".py",
+ "mimetype": "text/x-python",
+ "name": "python",
+ "nbconvert_exporter": "python",
+ "pygments_lexer": "ipython3",
+ "version": "3.7.6"
+ }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 2
+}
diff --git a/Phase 3 - 2020 (Summer)/Week 5(Apr 26-May 02)/Exercise5/__pycache__/utils.cpython-37.pyc b/Phase 3 - 2020 (Summer)/Week 5(Apr 26-May 02)/Exercise5/__pycache__/utils.cpython-37.pyc
new file mode 100644
index 000000000..6e06e245b
Binary files /dev/null and b/Phase 3 - 2020 (Summer)/Week 5(Apr 26-May 02)/Exercise5/__pycache__/utils.cpython-37.pyc differ
diff --git a/Phase 3 - 2020 (Summer)/Week 5(Apr 26-May 02)/Exercise5/exercise5.ipynb b/Phase 3 - 2020 (Summer)/Week 5(Apr 26-May 02)/Exercise5/exercise5.ipynb
index 66c4500ee..27f405eb4 100644
--- a/Phase 3 - 2020 (Summer)/Week 5(Apr 26-May 02)/Exercise5/exercise5.ipynb
+++ b/Phase 3 - 2020 (Summer)/Week 5(Apr 26-May 02)/Exercise5/exercise5.ipynb
@@ -18,10 +18,8 @@
},
{
"cell_type": "code",
- "execution_count": null,
- "metadata": {
- "collapsed": true
- },
+ "execution_count": 1,
+ "metadata": {},
"outputs": [],
"source": [
"# used for manipulating directory paths\n",
@@ -99,9 +97,22 @@
},
{
"cell_type": "code",
- "execution_count": null,
+ "execution_count": 2,
"metadata": {},
- "outputs": [],
+ "outputs": [
+ {
+ "data": {
+ "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX4AAAEGCAYAAABiq/5QAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjMsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+AADFEAAAgAElEQVR4nO3de5wcdZnv8c93YiMZQi+yCZEFQnBAXWRnuYxuMF4AV5dRiZdXdAUJHAVGTVggoEL0HJV1RTkuRD2gLgyuBEFg4y0i44oIC2wSZAI4gMiS5ibKJajgxGhs8Dl/VI1MQndPzaSre3r6+3696pWu6ro8UzRP//pXv3pKEYGZmbWPjmYHYGZmjeXEb2bWZpz4zczajBO/mVmbceI3M2szz2t2AFnMnDkz5s6d2+wwzMxayrp1656IiFlbL2+JxD937lwGBwebHYaZWUuR9GCl5e7qMTNrM078ZmZtxonfzKzNOPGbmU0ypVKJpYsXM7tYZFpHB7OLRZYuXkypVKrL/p34zcwmkYGBAeZ1dzO9v5/Vw8NsjmD18DDT+/uZ193NwMDANh9DrVCkraenJzyqx8ymulKpxLzublZt2sTBFd5fAyzo7GTt0BBdXV1j7k/Suojo2Xq5W/xmZpPEeeecwwnlcsWkD3AwcHy5zPnLl2/TcZz4zcwmicu+9jWOK5drrnN8ucxll1yyTcdx4jczmySe2LiRPcdYZ0663rZw4jczmyRmzphBxVttR3koXW9bOPGbmU0SRx19NBcVCjXX6S8UOGrRom06jhO/mdkkceJpp3FhocCaKu+vIUn8S5Yu3abjOPGbmU0SXV1drFi5kgWdnSwrFCgBZaAELCsUWNDZyYqVKzMN5azFid/MbBLp7e1l7dAQm/v6mF8sMr2jg/nFIpv7+lg7NERvb+82H8M3cJmZTVENv4FL0vaSfizpJ5LuknRmuvyrku6XdHs67Z9XDGZm9lx5PohlM3BYRGyUVABukjRSZOJDEbEyx2ObmVkVubX4IzFyl0EhnSZ/v5KZWQPlXYmzklwv7kqaJul24HHgmoi4OX3rU5KGJC2X9Pwq2/ZJGpQ0uGHDhjzDNDNrikZU4qykIRd3Je0EfAv4J+BXwKPAdsAFQCki/rnW9r64a2ZTTb0rcVbS1OqcEfEkcD1weEQ8knYDbQb+HXhFI2IwM5tMGlWJs5I8R/XMSlv6SJoO/D3wM0m7pssEvBW4M68YzMwmq0ZV4qwkz1E9uwIXS5pG8gVzZURcJelHkmYBAm4H3p9jDGZmk1KjKnFWklvij4gh4IAKyw/L65hmZq1i5owZPDg8TK3e+3pU4qzEJRvMzJqgUZU4K3HiNzNrgkZV4qzEid/MrAkaVYmzkjETv6QOSQdIepOkwyTNrnsUZmZtqBGVOCupegOXpC7gdJJhmPcCG4DtgRcDm4B/Ay6OiD/lEtkovoHLzGz8qt3AVWtUz78AXwLeF1t9O0jaBTgKWARcXM9AzcwsX1UTf0QcWeO9x4HP5RKRmZnlKksf/6CkJZJe0IiAzMwsX1lG9bwL+CvgFkmXS/qHtNyCmZm1oDETf0Ssj4iPklzUvQz4CvCQpDMl7Zx3gGZmVl+ZxvFL6gbOAT4LfANYCPwW+FF+oZmZWR7GrNUjaR3wJHARcEZaThngZknz8wzOzMzqL0uRtndExH2V3oiIt9c5HjMzy1nVrh5JR0vqqJb0JXVJelV+oZmZWR5qtfj/Ergt7epZx7N37u4NvBZ4Ajgj9wjNzKyuat3A9XlJ5wGHAfOBbuD3wN3Aooh4qDEhmplZPdXs44+IZ4Br0snMzKYAl2U2M2szTvxmZm0mt8QvaXtJP5b0E0l3STozXb6XpJsl3SvpCknb5RWDmZk9V5YbuHYCjgHmjl4/Ik4aY9PNwGERsVFSAbhJ0gBwKrA8Ii6X9GXgOJLyz2Zm1gBZWvxXkyT9O0iGdY5MNUViYzpbSKcgGSW0Ml1+MfDW8YVsZmbbIsudu9tHxKkT2bmkaSRfEnsD55M8TvLJiHg6XeVhYLcq2/YBfQBz5syZyOHNzKyCLC3+SySdIGlXSTuPTFl2HhHPRMT+wO7AK4C/rrRalW0viIieiOiZNWtWlsOZmVkGWVr8fySpyvlRnk3SAbwo60Ei4klJ1wPzgJ0kPS9t9e8O/HJcEZuZ2TbJ0uI/Fdg7IuZGxF7pNGbSlzQrvTCMpOkkD22/G7iOpKwzwLHAdyYWupmZTUSWFv9dwKYJ7HtX4OK0n78DuDIirpL0U+BySf8C3EZS7tnMzBokS+J/Brhd0nUkQzSBsYdzRsQQcECF5feR9PebmVkTZEn8304nMzObAsZM/BFxcSMCMTOzxshy5+4+wKeBfUnq8QOQ5QKvmZlNPllG9fw7SUmFp4FDgRXAJXkGZWZm+cmS+KdHxLWAIuLBiPgESdkFMzNrQVku7v5BUgdwr6QTgV8Au+QblpmZ5SVLi/8UoBM4CTgIWERy45WZmbWgLKN6bklfbgTek284ZmaWt6qJX9J3qVJADSAiFuQSkZmZ5apWi/9f03/fDrwQ+Fo6fyTwQI4xmZlZjqom/oj4LwBJn4yI14x667uSbsg9MjMzy0WWi7uzJP35Zi1JewEukG9m1qKyDOdcClwv6b50fi7pk7HMzKz1ZBnV8/20bMNL00U/i4jNtbYxM7PJK0uLnzTR/yTnWMzMrAGy9PGbmdkU4sRvZtZmxkz8Shwt6WPp/BxJfoKWmVmLytLi/yJwMMmNWwDDwPm5RWRmZrnKkvj/LiKWAH8AiIjfANuNtZGkPSRdJ+luSXdJOjld/glJv5B0ezq9cZv+AjMzG5cso3rKkqaR1u2RNAv4U4btngZOi4hbJe0IrJN0Tfre8oj41xrbmplZTrK0+L8AfAvYRdKngJuAs8baKCIeiYhb09fDwN3AbtsQq5mZ1cGYiT8iLgU+TPLc3UeAt0bEf4znIJLmAgcAN6eLTpQ0JOkrkl5QZZs+SYOSBjds2DCew5mZWQ1Zh3PeS9LqXwX8TtKcrAeQNAP4BnBKRPyW5Pm9XcD+JF8k51TaLiIuiIieiOiZNculgczM6mXMPn5J/wR8HHgMeAYQSX9/d4ZtCyRJ/9KI+CZARDw26v0LgasmFLmZmU1Ilou7JwMviYhfjWfHkgRcBNwdEeeOWr5rRDySzr4NuHM8+zUzs22TJfH/HHhqAvueT/J83jsk3Z4u+whwpKT9SX41PAC8bwL7NjOzCar16MVT05f3kZRl/h7w56qco1vxlUTETSTdQlu7egJxmplZndRq8e+Y/vtQOm3HszduVX0Wr5mZTW5VR/VExJkRcSbw05HXo5bd3bgQzcyqK5VKLF28mNnFItM6OphdLLJ08WJKpVKzQ5u0sgznXJZxmZlZQw0MDDCvu5vp/f2sHh5mcwSrh4eZ3t/PvO5uBgYGmh3ipFSrj78XeCOwm6QvjHqrSFKOwcysaUqlEscsXMiqTZs4eNTyLuCscpkjymUWLFzI2qEhurq6mhXmpFSrxf9LYJCkONu6UdMq4B/yD83MrLrzzjmHE8rlLZL+aAcDx5fLnL98eSPDagmKqH2dVlIhIsoNiqeinp6eGBwcbGYIZjbJzC4WWT08TK22fAmYXyzy6FMTGZHe+iSti4ierZdnqdXT1KRvZlbJExs3sucY68xJ17Mt+dGLZtaSZs6YwYNjrPNQup5tqWril3RJ+u/JjQvHzCybo44+mosKhZrr9BcKHLVoUYMiah21WvwHSdoTeK+kF0jaefTUqADNzCo58bTTuLBQYE2V99eQJP4lS5c2MqyWUCvxfxn4PvBSthzVs45ktI+ZWdN0dXWxYuVKFnR2sqxQoASUSS7oLisUWNDZyYqVKz2Us4Jad+5+ISL+GvhKRLwoIvYaNb2ogTGamVXU29vL2qEhNvf1Mb9YZHpHB/OLRTb39bF2aIje3t5mhzgpjTmcE0DS3wKvTmdviIihXKPaiodzmpmN34SHc0o6CbgU2CWdLk0fzmJmZi0oSz3+44G/i4jfAUg6m+S6yf/LMzAzM8tHlnH8Innk4oiRxy+amVkLytLi/3fgZknfSuffSvJIRTMza0FjJv6IOFfS9cCrSFr674mI2/IOzMzM8pGlxU9E3ArcOp4dS9oDWAG8EPgTcEFEfD69+esKYC7JM3ffGRG/Gc++zcxs4vKs1fM0cFp6L8A8YImkfYEzgGsjYh/g2nTezMwaJLfEHxGPpL8UiIhhksc17ga8Bbg4Xe1ikmsGZmbWIFnG8Z+dZdkY+5gLHADcDMyOiEcg+XIguTfAzMwaJEuL//UVlmW+D1rSDOAbwCkR8dtxbNcnaVDS4IYNG7JuZmZmY6hVlvkDku4AXiJpaNR0P5CpZIOkAknSvzQivpkufkzSrun7uwKPV9o2Ii6IiJ6I6Jk1a9Z4/iYzM6uh1qiey4AB4NNseQF2OCJ+PdaOJYlkvP/dEXHuqLdWAccCn0n//c54gzYzs4mrmvgj4ingKUmnb/XWDEkzIuKhMfY9H1gE3CHp9nTZR0gS/pWSjiN5QM47Jha6mZlNRJZx/N8DguTmre2BvYB7gJfV2igibqJ6aYfXjSNGMzOroyx37v7N6HlJBwLvyy0iMzPL1bjH8adj81+eQyxmZtYAY7b4JZ06arYDOBDw+EozsxaVpY9/x1Gvnybp8/9GPuGYmVnesvTxnwkgacdkNjbmHpWZmeUmS8mG/STdBtwJ3CVpnaT98g/NzMzykOXi7gXAqRGxZ0TsCZyWLjMzsxaUJfHvEBHXjcxExPXADrlFZGZmucpycfc+Sf8HuCSdPxq4P7+QzMwsT1la/O8FZgHfTKeZwHvyDMrMzPKTZVTPb4CTGhCLmZk1QJ6PXjQzs0nIid/MrM048ZuZtZkstXq+UGHxU8BgRPghKmZmLSZLi397YH/g3nTqBnYGjpP0uRxjMzOzHGQZx783cFhEPA0g6UvAD0gewn5HjrGZmVkOsrT4d2PLO3V3AP4qIp4BNucSlZmZ5SZLi///ArdLup7kUYqvAc6StAPwwxxjMzOzHIzZ4o+Ii4BXAt9Op1dFRH9E/C4iPlRtO0lfkfS4pDtHLfuEpF9Iuj2d3liPP8LMJp9SqcTSxYuZXSwyraOD2cUiSxcvplQqNTu0tpd1OGcHyVO3fg3sLek1Gbb5KnB4heXLI2L/dLo64/HNrIUMDAwwr7ub6f39rB4eZnMEq4eHmd7fz7zubgYGBpodYlvLMpzzbOAfgbuAP6WLA7ih1nYRcYOkudsYn5m1mFKpxDELF7Jq0yYOHrW8CzirXOaIcpkFCxeydmiIrq6uZoXZ1rK0+N8KvCQi3hQRR6TTgm045omShtKuoBdUW0lSn6RBSYMbNvgRv2at4rxzzuGEcnmLpD/awcDx5TLnL1/eyLBsFEVE7RWkAeAdE3nkYtrivyoi9kvnZwNPkPxi+CSwa0S8d6z99PT0xODg4HgPb2ZNMLtYZPXwMLXa8iVgfrHIo0891aiw2pKkdRHRs/XyLKN6NpGM6rmWUcM3I2LcFTsj4rFRAV0IXDXefZjZ5PbExo3sOcY6c9L1rDmyJP5V6bTNJO0aEY+ks28jeY6vmU0hM2fM4MExWvwPpetZc2Spx3/xRHYs6evAIcBMSQ8DHwcOkbQ/SVfPA8D7JrJvM5u8jjr6aC7q7+escrnqOv2FAkctWtTAqGy0qn38kq6MiHdKuoMkUW8hIrrzDm6E+/jNWkepVGJed/dzRvWMWAMs6Oz0qJ4GmEgf/8npv2/OJyQzm4q6urpYsXIlCxYu5PhymePLZeaQdO/0Fwr0FwqsWLnSSb+Jqg7nHNUX/zpgu4h4cPTUmPDMrBX19vaydmiIzX19zC8Wmd7Rwfxikc19fawdGqK3t7fZIba1LMM5/xl4FbAnsA64EbgxIm7PP7yEu3rMzMavWldPllo9H4uIw4D9gJuAD5F8AZiZWQvKUrLhfwPzgRnAbcAHSVr9ZmbWgrKM43878DTwPeC/gLUR8YdcozIzs9xk6eo5kOQC749Jn7ol6aa8AzMzs3xk6erZD3g18FqgB/g57uoxM2tZWbp6ziYpwfwF4JaIqH47npmZTXpZSja8SdJ2wIuBl0i6x8nfzKx1ZenqeS2wgqS2joA9JB0bETUfxGJmZpNTlq6ec4E3RMQ9AJJeDHwdOCjPwMzMLB9ZnsBVGEn6ABHxP0Ahv5DMzCxPWVr8g5IuAi5J59+N79w1M2tZWRL/B4AlwEkkffw3AF/MMygzM8tPllE9m0n6+c/NPxwzM8tb1cRf7QEsIxr5IBYzM6ufWi3+dwC/b1QgZmbWGLUS/2URcaCkSyLCD8c0M5siaiX+7SQdC7xS0tu3fjMivllrx5K+QvLYxscjYr902c7AFcBckhvC3hkRv5lY6GZmNhG1xvG/H5gH7AQcsdWU5Tm8XwUO32rZGcC1EbEPcG06b2ZmDVS1xR8RNwE3SRqMiIvGu+OIuEHS3K0WvwU4JH19MXA9cPp4921mZhOXpR7/uJN+DbNHHuKe/rtLtRUl9UkalDS4YcOGOoZgZtbespRsaIqIuCAieiKiZ9asWc0Ox8xsyqiZ+JXYo47He0zSrum+dwUer+O+zcwsg5qJPyIC+HYdj7cKODZ9fSzwnTru28zMMsjS1bNW0svHu2NJXwfWkDy85WFJxwGfAV4v6V6S5/d+Zrz7NTOzbZOlSNuhwPslPQD8jqRQW4xVsiEijqzy1uvGFaGZmdVVlsTfm3sUZmbWMFmGcz4I7AEclr7elGU7MzObnMZM4JI+TnKT1bJ0UQH4Wp5BmVn9lEolli5ezOxikWkdHcwuFlm6eDGlUqnZoVmTZGm5vw1YQNK/T0T8Etgxz6DMrD4GBgaY193N9P5+Vg8PszmC1cPDTO/vZ153NwMDA80O0ZogS+L/YzqsMwAk7ZBvSGZWD6VSiWMWLmTVpk2cVS7TRXJRrws4q1xm1aZNHLNwoVv+bShL4r9S0r8BO0k6Afgh0J9vWGa2rc475xxOKJc5uMr7BwPHl8ucv3x5I8OySUBJY36MlaTXA28gGcr5nxFxTd6BjdbT0xODg4ONPKRZy5tdLLJ6eJiuGuuUgPnFIo8+9VSjwrIGkrQuInq2Xj7mcE5JZ0fE6cA1FZaZ2ST1xMaN7DnGOnPS9ay9ZOnqeX2FZR7bbzbJzZwxgwfHWOehdD1rL1UTv6QPpA9cf4mkoVHT/cBQ40I0s4k46uijuahQqLlOf6HAUYv8ZNV2U7WPX9JfAC8APs2WT8oajohfNyC2P3Mfv9n4lUol5nV3s2rTpooXeNcACzo7WTs0RFdXrSsB1qqq9fFXbfFHxFMR8UBEHJnesft7kiGdMyTNyTFWM6uDrq4uVqxcyYLOTpYVCpSAMskF3WWFAgs6O1mxcqWTfhvKcufuEWk1zfuB/yJ5SLrv+jBrAb29vawdGmJzXx/zi0Wmd3Qwv1hkc18fa4eG6O315bp2NOZwTkk/AQ4DfhgRB0g6FDgyIvoaESC4q8fMbCLG3dUzSjkifgV0SOqIiOuA/eseoZmZNUSWssxPSpoB3ABcKulx4Ol8wzIzs7xkafG/heTC7lLg+yTXho7IMygzM8tP1Ra/pFOA/wZui4hn0sUXNyQqMzPLTa2unt2BzwMvlTQErCb5IljT6HH8ZmZWP1UTf0R8EEDSdkAP8ErgvcCFkp6MiH0netD0+b3DwDPA05WuOpuZWT6yXNydDhSBv0inXwJ31OHYh0bEE3XYj5mZjUOtPv4LgJeRtMxvJunqOTciftOg2MzMLAe1RvXMAZ4PPAr8AngYeLJOxw3gB5LWSap4I5ikPkmDkgY3bNhQp8OamVnNO3cliaTV/8p02g/4NckF3o9P+KDSX0XELyXtQlLn/58i4oZq6/vOXTOz8ZvQnbuRuBO4mqQ+z3+TPLLz5G0JJn1gOxHxOPAt4BXbsr96KJVKLF28mNnFItM6OphdLLJ08WI/j9Qazp9Fy1utevwnSbpc0s9J7tp9M3AP8HZg54keUNIOknYceU3ySMc7J7q/ehgYGGBedzfT+/tZPTzM5ghWDw8zvb+fed3dDAy4Jp01hj+L1gi16vGfSzp2PyIeqdsBpReRtPIhubh8WUR8qtY2eXb1uGa5TRb+LFq9TaQe/6kRsbKeST/d730R8bfp9LKxkn7ezjvnHE4olyv+jwZwMHB8ucz5y5c3MixrQ/4sWqOMWZZ5MsizxT+7WGT18DC12k8lYH6xyKNPPZVLDGbgz6LV37aUZW4547k49sTGjew5xv7mpOuZ5cmfRWuUKZf4x3txbOaMGTw4xj4fStdrFI/qaE+T8bNoU9OUSvylUoljFi5k1aZNnFUu00Vy9bgLOKtcZtWmTRyzcOEWCfSoo4/mokKh5n77CwWOWrQo19hHeFRH+5psn0WbwiJi0k8HHXRQZHHKBz4QywqFCKg6nVEoxNIlS/68zfr162NmZ2esrrL+aoiZnZ2xfv36TDFsi8kUizWe//tbvQGDUSGnTqkW/2Vf+xrHlcs11zm+XOaySy7583xXVxcrVq5kQWcnywoFSkCZ5CLaskKBBZ2drFi5siHD5zyqY0uN6PKaTN1qk+mzaFNcpW+DyTZlbfF3SFGu0doPiD9CTOvoeM6269evj6VLlsTsYjGmdXTE7GIxli5Z0tDW1S477hjrx4h/PcTsYrFhMTXL1VdfHTM7O2NZoRDrIcrp376sUIiZnZ1x9dVXt8QxJmIyfBZtaqBKi39KDeds9eFw0zo62BxRs1Z2GZje0cHTzzxTY63W1ogbmXyzlLWDthjO2eoXxzyqI9GILi93q1lbq/QzYLJNWbt6Wv3i2EQuTk9FjejycreatQPa4eJuq18cO/G007iwUGBNlffXkPxiWbJ0aeZ9TqaLl1k14kYm3yxl7WxKJX6A3t5e1g4Nsbmvj/nFItM7OphfLLK5r4+1Q0P09vY2O8Sq6v3F1ar3BDSiy8vdatbWKv0MmGxT1q6eqaIeozpaudurEV1e7lazdkCVrp6mJ/UsU7sl/npo5cTWiC+tVv5iNMuqWuKfcl09lpjIzWyTRSOu1bT69SCzbeHEP0W1+sXLRlyraeXrQWbbYkrdwGXPavWb2cxs27XFDVz2rFa/mc3M8tOUxC/pcEn3SFov6YxmxDDV5XFPgJlNDQ1P/JKmAecDvcC+wJGS9m10HFOdL16aWTXNaPG/AlgfyUPX/whcDrylCXFMeb54aWaVNPzirqSFwOERcXw6vwj4u4g4cav1+oA+gDlz5hz04INj3WdpZmajTaaLu6qw7DnfPhFxQUT0RETPrFmzGhCWmVl7aEbifxjYY9T87sAvmxCHmVlbakbivwXYR9JekrYD3gWsakIcZmZtqSk3cEl6I/A5YBrwlYj41Bjrb4AxiylOJjOBJ5odxCTlc1Odz011PjfV1To3e0bEc/rKW+LO3VYjabDSBRXzuanF56Y6n5vqJnJufOeumVmbceI3M2szTvz5uKDZAUxiPjfV+dxU53NT3bjPjfv4zczajFv8ZmZtxonfzKzNOPHnQNIHJYWkmem8JH0hLUM9JOnAZsfYaJI+K+ln6d//LUk7jXpvWXpu7pH0D82Ms1lcqvxZkvaQdJ2kuyXdJenkdPnOkq6RdG/67wuaHWuzSJom6TZJV6Xze0m6OT03V6Q3x1blxF9nkvYAXg88NGpxL7BPOvUBX2pCaM12DbBfRHQD/wMsA0hLcr8LeBlwOPDFtHR323Cp8ud4GjgtIv4amAcsSc/HGcC1EbEPcG06365OBu4eNX82sDw9N78Bjqu1sRN//S0HPsyWhefeAqxIH3y/FthJ0q5Nia5JIuIHEfF0OruWpEYTJOfm8ojYHBH3A+tJSne3E5cqHyUiHomIW9PXwyQJbjeSc3JxutrFwFubE2FzSdodeBPQn84LOAxYma4y5rlx4q8jSQuAX0TET7Z6azfg56PmH06Xtav3AgPpa58bn4OqJM0FDgBuBmZHxCOQfDkAuzQvsqb6HEnj8k/p/F8CT45qWI35+XlefrFNTZJ+CLywwlsfBT4CvKHSZhWWTblxtLXOTUR8J13noyQ/5S8d2azC+lPu3IzB56ACSTOAbwCnRMRvk4Zte5P0ZuDxiFgn6ZCRxRVWrfn5ceIfp4j4+0rLJf0NsBfwk/QDujtwq6RX0CalqKudmxGSjgXeDLwunr2BpC3OzRh8DrYiqUCS9C+NiG+mix+TtGtEPJJ2lT7evAibZj6wIC10uT1QJPkFsJOk56Wt/jE/P+7qqZOIuCMidomIuRExl+R/5gMj4lGSstPHpKN75gFPjfxkbReSDgdOBxZExKZRb60C3iXp+ZL2IrkA/uNmxNhELlU+StpnfRFwd0ScO+qtVcCx6etjge80OrZmi4hlEbF7mmPeBfwoIt4NXAcsTFcb89y4xd8YVwNvJLlwuQl4T3PDaYrzgOcD16S/iNZGxPsj4i5JVwI/JekCWhIRzzQxzoaLiKclnQj8J8+WKr+ryWE103xgEXCHpNvTZR8BPgNcKek4klFz72hSfJPR6cDlkv4FuI3ki7Mql2wwM2sz7uoxM2szTvxmZm3Gid/MrM048ZuZtRknfjOzNuPEb3Uh6YWSLpdUkvRTSVdLerGkQ0YqCDabpH+WVPMmszodZydJi+uwn+sl1fUB47X2KWmlpBfV2HY7STdI8jDwFufEb9ssveHmW8D1EdEVEfuSjLue3dzIthQRH4uIHzbgUDsB40r86c19Tfv/UdLLgGkRcV+1ddICctcC/9iwwCwXTvxWD4cC5Yj48siCiLg9Im5MZ2ekrcmfSbo0/aJA0sck3SLpTkkXjFp+vaSzJf1Y0v9IenW6vFPSlWlN/yvS+uM96XtvkLRG0q2S/iOt87IFSV+VtDB9/YCkM9P175D00grrXy2pO319m6SPpa8/Kel4STMkXTtqHyMVNT8DdEm6XdJn020+lP6tQ5LOTJfNVVJz/ovArWxZtmHrWJ7z90nqTW9+G1nnEEnfzXo+tvJu0rs9Je2ppK77TEkdkm6UNFKD6tvputbCnPitHvYD1tV4/wDgFJJa8y8iuTMT4LyIeHlE7AdMJ6njM+J5ERZvVdcAAANkSURBVPGKdLuPp8sWA79Ja/p/EjgIQMkDb/438PcRcSAwCJyaIe4n0vW/BHywwvs3AK+WVCS5q3gk7lcBNwJ/AN6W7uNQ4Jz0y+sMoBQR+0fEh9KkuQ9J+eX9gYMkvSbd10tISnYfEBEPVgqyxt93DTBP0g7pqv8IXDHB8zGf9L9hGsfZwJeB04CfRsQP0vXuBF4+xr5sknNfnTXCjyPiYYD0Fvy5wE3AoZI+DHQCOwN3Ad9NtxkpzLUuXR+ShPt5gIi4U9JQunweyZfKf6c/GrYD1mSIa/Qx3l7h/RuBk4D7ge8Br5fUCcyNiHuUFBI7K03ifyIphVupe+sN6XRbOj+D5IvgIeDB9BkNtVT8+9JSD98HjpC0kqRG+4eB11Zaf4xj7ApsGJmJiH5J7wDeT/JlNbL8GUl/lLRjWivfWpATv9XDXTxbIKqSzaNePwM8T9L2wBeBnoj4uaRPkFQb3HqbZ3j2c1qtLq+AayLiyHHGXekYo90C9AD3kbSuZwIn8Oyvm3cDs4CDIqIs6YGt/obR8X06Iv5ti4VJrfnfZYiz1t93BbAE+DVwS0QMp786xns+fj869vQLbuRhOTOA0Un++SS/dqxFuavH6uFHwPMlnTCyQNLLJb22xjYjSeaJtP+51hfHiJuAd6b73xf4m3T5WmC+pL3T9zolvXicf8NzpBczf54ecy3JL4APpv8C/AVJbfSypEOBPdPlw8COo3b1n8B7R/rZJe0maTwPEan1910PHEjyhXRFhvWruRvYe9T82STPTPgYcOHIQkl/CWyIiPI44rdJxonftllaW/9tJF0hJUl3AZ+gRk3wiHiSJKHcQXLB8JYMh/oiMCvt4jkdGCIpcb0B+F/A19P31gLPuVg7QTcCj6WlpG8kaQWPJP5LgR5JgySt/58BRMSvSLpZ7pT02bR//DJgjaQ7SB6RtyMZ1fr70kqmV5E8r/eqsdav4XvAIQDpF/bLgbMj4lLgj5JGKsoeSlJt1lqYq3Nay1DyUPJCRPxBUhfJ0MIXpy1z2waSppPUdJ9fqyy2pG8CyyLinoYFZ3XnPn5rJZ3AdelFVQEfcNKvj4j4vaSPk1ygfqjSOkoeEvNtJ/3W5xa/mVmbcR+/mVmbceI3M2szTvxmZm3Gid/MrM048ZuZtZn/D9cM/apxPVORAAAAAElFTkSuQmCC\n",
+ "text/plain": [
+ "
"
+ ]
+ },
+ "metadata": {
+ "needs_background": "light"
+ },
+ "output_type": "display_data"
+ }
+ ],
"source": [
"# add a columns of ones for the y-intercept\n",
"X_aug = np.concatenate([np.ones((m, 1)), X], axis=1)\n",
@@ -442,7 +532,10 @@
" error_val = np.zeros(m)\n",
"\n",
" # ====================== YOUR CODE HERE ======================\n",
- " \n",
+ " for i in range(1, m + 1):\n",
+ " theta_t = utils.trainLinearReg(linearRegCostFunction, X[:i], y[:i], lambda_ = lambda_)\n",
+ " error_train[i - 1], _ = linearRegCostFunction(X[:i], y[:i], theta_t, lambda_ = 0)\n",
+ " error_val[i - 1], _ = linearRegCostFunction(Xval, yval, theta_t, lambda_ = 0) \n",
"\n",
" \n",
" # =============================================================\n",
@@ -562,6 +655,8 @@
" X_poly = np.zeros((X.shape[0], p))\n",
"\n",
" # ====================== YOUR CODE HERE ======================\n",
+ " for i in range(p):\n",
+ " X_poly[:, i] = X[:, 0] ** (i + 1)\n",
"\n",
"\n",
"\n",
@@ -804,6 +899,11 @@
" error_val = np.zeros(len(lambda_vec))\n",
"\n",
" # ====================== YOUR CODE HERE ======================\n",
+ " for i in range(len(lambda_vec)):\n",
+ " lambda_try = lambda_vec[i]\n",
+ " theta_t = utils.trainLinearReg(linearRegCostFunction, X, y, lambda_ = lambda_try)\n",
+ " error_train[i], _ = linearRegCostFunction(X, y, theta_t, lambda_ = 0)\n",
+ " error_val[i], _ = linearRegCostFunction(Xval, yval, theta_t, lambda_ = 0)\n",
"\n",
"\n",
"\n",
@@ -919,7 +1019,7 @@
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
- "version": "3.6.4"
+ "version": "3.7.6"
}
},
"nbformat": 4,
diff --git a/Phase 3 - 2020 (Summer)/Week 5(Apr 26-May 02)/Exercise5/token.pkl b/Phase 3 - 2020 (Summer)/Week 5(Apr 26-May 02)/Exercise5/token.pkl
new file mode 100644
index 000000000..9d93677f2
Binary files /dev/null and b/Phase 3 - 2020 (Summer)/Week 5(Apr 26-May 02)/Exercise5/token.pkl differ
diff --git a/Phase 3 - 2020 (Summer)/Week 5(Apr 26-May 02)/__pycache__/submission.cpython-37.pyc b/Phase 3 - 2020 (Summer)/Week 5(Apr 26-May 02)/__pycache__/submission.cpython-37.pyc
new file mode 100644
index 000000000..7e44f90c4
Binary files /dev/null and b/Phase 3 - 2020 (Summer)/Week 5(Apr 26-May 02)/__pycache__/submission.cpython-37.pyc differ
diff --git a/Phase 3 - 2020 (Summer)/Week 6(May 3 -May 9)/Exercise6/.ipynb_checkpoints/exercise6-checkpoint.ipynb b/Phase 3 - 2020 (Summer)/Week 6(May 3 -May 9)/Exercise6/.ipynb_checkpoints/exercise6-checkpoint.ipynb
new file mode 100644
index 000000000..148c1897e
--- /dev/null
+++ b/Phase 3 - 2020 (Summer)/Week 6(May 3 -May 9)/Exercise6/.ipynb_checkpoints/exercise6-checkpoint.ipynb
@@ -0,0 +1,1044 @@
+{
+ "cells": [
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "# Programming Exercise 6:\n",
+ "# Support Vector Machines\n",
+ "\n",
+ "## Introduction\n",
+ "\n",
+ "In this exercise, you will be using support vector machines (SVMs) to build a spam classifier. Before starting on the programming exercise, we strongly recommend watching the video lectures and completing the review questions for the associated topics.\n",
+ "\n",
+ "All the information you need for solving this assignment is in this notebook, and all the code you will be implementing will take place within this notebook. The assignment can be promptly submitted to the coursera grader directly from this notebook (code and instructions are included below).\n",
+ "\n",
+ "Before we begin with the exercises, we need to import all libraries required for this programming exercise. Throughout the course, we will be using [`numpy`](http://www.numpy.org/) for all arrays and matrix operations, [`matplotlib`](https://matplotlib.org/) for plotting, and [`scipy`](https://docs.scipy.org/doc/scipy/reference/) for scientific and numerical computation functions and tools. You can find instructions on how to install required libraries in the README file in the [github repository](https://github.com/dibgerge/ml-coursera-python-assignments)."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "# used for manipulating directory paths\n",
+ "import os\n",
+ "\n",
+ "# Scientific and vector computation for python\n",
+ "import numpy as np\n",
+ "\n",
+ "# Import regular expressions to process emails\n",
+ "import re\n",
+ "\n",
+ "# Plotting library\n",
+ "from matplotlib import pyplot\n",
+ "\n",
+ "# Optimization module in scipy\n",
+ "from scipy import optimize\n",
+ "\n",
+ "# will be used to load MATLAB mat datafile format\n",
+ "from scipy.io import loadmat\n",
+ "\n",
+ "# library written for this exercise providing additional functions for assignment submission, and others\n",
+ "import utils\n",
+ "\n",
+ "# define the submission/grader object for this exercise\n",
+ "grader = utils.Grader()\n",
+ "\n",
+ "# tells matplotlib to embed plots within the notebook\n",
+ "%matplotlib inline"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "## Submission and Grading\n",
+ "\n",
+ "\n",
+ "After completing each part of the assignment, be sure to submit your solutions to the grader. The following is a breakdown of how each part of this exercise is scored.\n",
+ "\n",
+ "\n",
+ "| Section | Part | Submitted Function | Points |\n",
+ "| :- |:- |:- | :-: |\n",
+ "| 1 | [Gaussian Kernel](#section1) | [`gaussianKernel`](#gaussianKernel) | 25 |\n",
+ "| 2 | [Parameters (C, $\\sigma$) for Dataset 3](#section2)| [`dataset3Params`](#dataset3Params) | 25 |\n",
+ "| 3 | [Email Preprocessing](#section3) | [`processEmail`](#processEmail) | 25 |\n",
+ "| 4 | [Email Feature Extraction](#section4) | [`emailFeatures`](#emailFeatures) | 25 |\n",
+ "| | Total Points | |100 |\n",
+ "\n",
+ "\n",
+ "You are allowed to submit your solutions multiple times, and we will take only the highest score into consideration.\n",
+ "\n",
+ "
\n",
+ "At the end of each section in this notebook, we have a cell which contains code for submitting the solutions thus far to the grader. Execute the cell to see your score up to the current section. For all your work to be submitted properly, you must execute those cells at least once.\n",
+ "
"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "## 1 Support Vector Machines\n",
+ "\n",
+ "In the first half of this exercise, you will be using support vector machines (SVMs) with various example 2D datasets. Experimenting with these datasets will help you gain an intuition of how SVMs work and how to use a Gaussian kernel with SVMs. In the next half of the exercise, you will be using support\n",
+ "vector machines to build a spam classifier."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "### 1.1 Example Dataset 1\n",
+ "\n",
+ "We will begin by with a 2D example dataset which can be separated by a linear boundary. The following cell plots the training data, which should look like this:\n",
+ "\n",
+ "\n",
+ "\n",
+ "In this dataset, the positions of the positive examples (indicated with `x`) and the negative examples (indicated with `o`) suggest a natural separation indicated by the gap. However, notice that there is an outlier positive example `x` on the far left at about (0.1, 4.1). As part of this exercise, you will also see how this outlier affects the SVM decision boundary."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "# Load from ex6data1\n",
+ "# You will have X, y as keys in the dict data\n",
+ "data = loadmat(os.path.join('Data', 'ex6data1.mat'))\n",
+ "X, y = data['X'], data['y'][:, 0]\n",
+ "\n",
+ "# Plot training data\n",
+ "utils.plotData(X, y)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "In this part of the exercise, you will try using different values of the $C$ parameter with SVMs. Informally, the $C$ parameter is a positive value that controls the penalty for misclassified training examples. A large $C$ parameter tells the SVM to try to classify all the examples correctly. $C$ plays a role similar to $1/\\lambda$, where $\\lambda$ is the regularization parameter that we were using previously for logistic regression.\n",
+ "\n",
+ "\n",
+ "The following cell will run the SVM training (with $C=1$) using SVM software that we have included with the starter code (function `svmTrain` within the `utils` module of this exercise). When $C=1$, you should find that the SVM puts the decision boundary in the gap between the two datasets and *misclassifies* the data point on the far left, as shown in the figure (left) below.\n",
+ "\n",
+ "
\n",
+ "
\n",
+ "
SVM Decision boundary for example dataset 1
\n",
+ "
\n",
+ "
\n",
+ "
C=1
\n",
+ "
C=100
\n",
+ "
\n",
+ "
\n",
+ "\n",
+ "
\n",
+ "In order to minimize the dependency of this assignment on external libraries, we have included this implementation of an SVM learning algorithm in utils.svmTrain. However, this particular implementation is not very efficient (it was originally chosen to maximize compatibility between Octave/MATLAB for the first version of this assignment set). If you are training an SVM on a real problem, especially if you need to scale to a larger dataset, we strongly recommend instead using a highly optimized SVM toolbox such as [LIBSVM](https://www.csie.ntu.edu.tw/~cjlin/libsvm/). The python machine learning library [scikit-learn](http://scikit-learn.org/stable/index.html) provides wrappers for the LIBSVM library.\n",
+ "
\n",
+ " \n",
+ "
\n",
+ "**Implementation Note:** Most SVM software packages (including the function `utils.svmTrain`) automatically add the extra feature $x_0$ = 1 for you and automatically take care of learning the intercept term $\\theta_0$. So when passing your training data to the SVM software, there is no need to add this extra feature $x_0 = 1$ yourself. In particular, in python your code should be working with training examples $x \\in \\mathcal{R}^n$ (rather than $x \\in \\mathcal{R}^{n+1}$); for example, in the first example dataset $x \\in \\mathcal{R}^2$.\n",
+ "
\n",
+ "\n",
+ "Your task is to try different values of $C$ on this dataset. Specifically, you should change the value of $C$ in the next cell to $C = 100$ and run the SVM training again. When $C = 100$, you should find that the SVM now classifies every single example correctly, but has a decision boundary that does not\n",
+ "appear to be a natural fit for the data."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "# You should try to change the C value below and see how the decision\n",
+ "# boundary varies (e.g., try C = 1000)\n",
+ "C = 1\n",
+ "\n",
+ "model = utils.svmTrain(X, y, C, utils.linearKernel, 1e-3, 20)\n",
+ "utils.visualizeBoundaryLinear(X, y, model)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "\n",
+ "### 1.2 SVM with Gaussian Kernels\n",
+ "\n",
+ "In this part of the exercise, you will be using SVMs to do non-linear classification. In particular, you will be using SVMs with Gaussian kernels on datasets that are not linearly separable.\n",
+ "\n",
+ "#### 1.2.1 Gaussian Kernel\n",
+ "\n",
+ "To find non-linear decision boundaries with the SVM, we need to first implement a Gaussian kernel. You can think of the Gaussian kernel as a similarity function that measures the “distance” between a pair of examples,\n",
+ "($x^{(i)}$, $x^{(j)}$). The Gaussian kernel is also parameterized by a bandwidth parameter, $\\sigma$, which determines how fast the similarity metric decreases (to 0) as the examples are further apart.\n",
+ "You should now complete the code in `gaussianKernel` to compute the Gaussian kernel between two examples, ($x^{(i)}$, $x^{(j)}$). The Gaussian kernel function is defined as:\n",
+ "\n",
+ "$$ K_{\\text{gaussian}} \\left( x^{(i)}, x^{(j)} \\right) = \\exp \\left( - \\frac{\\left\\lvert\\left\\lvert x^{(i)} - x^{(j)}\\right\\lvert\\right\\lvert^2}{2\\sigma^2} \\right) = \\exp \\left( -\\frac{\\sum_{k=1}^n \\left( x_k^{(i)} - x_k^{(j)}\\right)^2}{2\\sigma^2} \\right)$$\n",
+ ""
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "def gaussianKernel(x1, x2, sigma):\n",
+ " \"\"\"\n",
+ " Computes the radial basis function\n",
+ " Returns a radial basis function kernel between x1 and x2.\n",
+ " \n",
+ " Parameters\n",
+ " ----------\n",
+ " x1 : numpy ndarray\n",
+ " A vector of size (n, ), representing the first datapoint.\n",
+ " \n",
+ " x2 : numpy ndarray\n",
+ " A vector of size (n, ), representing the second datapoint.\n",
+ " \n",
+ " sigma : float\n",
+ " The bandwidth parameter for the Gaussian kernel.\n",
+ "\n",
+ " Returns\n",
+ " -------\n",
+ " sim : float\n",
+ " The computed RBF between the two provided data points.\n",
+ " \n",
+ " Instructions\n",
+ " ------------\n",
+ " Fill in this function to return the similarity between `x1` and `x2`\n",
+ " computed using a Gaussian kernel with bandwidth `sigma`.\n",
+ " \"\"\"\n",
+ " sim = 0\n",
+ " # ====================== YOUR CODE HERE ======================\n",
+ " sim = np.exp(-np.sum((x1 - x2) ** 2) / (2 * (sigma ** 2)))\n",
+ "\n",
+ "\n",
+ " # =============================================================\n",
+ " return sim"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Once you have completed the function `gaussianKernel` the following cell will test your kernel function on two provided examples and you should expect to see a value of 0.324652."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "x1 = np.array([1, 2, 1])\n",
+ "x2 = np.array([0, 4, -1])\n",
+ "sigma = 2\n",
+ "\n",
+ "sim = gaussianKernel(x1, x2, sigma)\n",
+ "\n",
+ "print('Gaussian Kernel between x1 = [1, 2, 1], x2 = [0, 4, -1], sigma = %0.2f:'\n",
+ " '\\n\\t%f\\n(for sigma = 2, this value should be about 0.324652)\\n' % (sigma, sim))"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "*You should now submit your solutions.*"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "grader[1] = gaussianKernel\n",
+ "grader.grade()"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "### 1.2.2 Example Dataset 2\n",
+ "\n",
+ "The next part in this notebook will load and plot dataset 2, as shown in the figure below. \n",
+ "\n",
+ ""
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "# Load from ex6data2\n",
+ "# You will have X, y as keys in the dict data\n",
+ "data = loadmat(os.path.join('Data', 'ex6data2.mat'))\n",
+ "X, y = data['X'], data['y'][:, 0]\n",
+ "\n",
+ "# Plot training data\n",
+ "utils.plotData(X, y)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "From the figure, you can obserse that there is no linear decision boundary that separates the positive and negative examples for this dataset. However, by using the Gaussian kernel with the SVM, you will be able to learn a non-linear decision boundary that can perform reasonably well for the dataset. If you have correctly implemented the Gaussian kernel function, the following cell will proceed to train the SVM with the Gaussian kernel on this dataset.\n",
+ "\n",
+ "You should get a decision boundary as shown in the figure below, as computed by the SVM with a Gaussian kernel. The decision boundary is able to separate most of the positive and negative examples correctly and follows the contours of the dataset well.\n",
+ "\n",
+ ""
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "# SVM Parameters\n",
+ "C = 1\n",
+ "sigma = 0.1\n",
+ "\n",
+ "model= utils.svmTrain(X, y, C, gaussianKernel, args=(sigma,))\n",
+ "utils.visualizeBoundary(X, y, model)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "\n",
+ "#### 1.2.3 Example Dataset 3\n",
+ "\n",
+ "In this part of the exercise, you will gain more practical skills on how to use a SVM with a Gaussian kernel. The next cell will load and display a third dataset, which should look like the figure below.\n",
+ "\n",
+ "\n",
+ "\n",
+ "You will be using the SVM with the Gaussian kernel with this dataset. In the provided dataset, `ex6data3.mat`, you are given the variables `X`, `y`, `Xval`, `yval`. "
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "# Load from ex6data3\n",
+ "# You will have X, y, Xval, yval as keys in the dict data\n",
+ "data = loadmat(os.path.join('Data', 'ex6data3.mat'))\n",
+ "X, y, Xval, yval = data['X'], data['y'][:, 0], data['Xval'], data['yval'][:, 0]\n",
+ "\n",
+ "# Plot training data\n",
+ "utils.plotData(X, y)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Your task is to use the cross validation set `Xval`, `yval` to determine the best $C$ and $\\sigma$ parameter to use. You should write any additional code necessary to help you search over the parameters $C$ and $\\sigma$. For both $C$ and $\\sigma$, we suggest trying values in multiplicative steps (e.g., 0.01, 0.03, 0.1, 0.3, 1, 3, 10, 30).\n",
+ "Note that you should try all possible pairs of values for $C$ and $\\sigma$ (e.g., $C = 0.3$ and $\\sigma = 0.1$). For example, if you try each of the 8 values listed above for $C$ and for $\\sigma^2$, you would end up training and evaluating (on the cross validation set) a total of $8^2 = 64$ different models. After you have determined the best $C$ and $\\sigma$ parameters to use, you should modify the code in `dataset3Params`, filling in the best parameters you found. For our best parameters, the SVM returned a decision boundary shown in the figure below. \n",
+ "\n",
+ "\n",
+ "\n",
+ "
\n",
+ "**Implementation Tip:** When implementing cross validation to select the best $C$ and $\\sigma$ parameter to use, you need to evaluate the error on the cross validation set. Recall that for classification, the error is defined as the fraction of the cross validation examples that were classified incorrectly. In `numpy`, you can compute this error using `np.mean(predictions != yval)`, where `predictions` is a vector containing all the predictions from the SVM, and `yval` are the true labels from the cross validation set. You can use the `utils.svmPredict` function to generate the predictions for the cross validation set.\n",
+ "
\n",
+ ""
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "def dataset3Params(X, y, Xval, yval):\n",
+ " \"\"\"\n",
+ " Returns your choice of C and sigma for Part 3 of the exercise \n",
+ " where you select the optimal (C, sigma) learning parameters to use for SVM\n",
+ " with RBF kernel.\n",
+ " \n",
+ " Parameters\n",
+ " ----------\n",
+ " X : array_like\n",
+ " (m x n) matrix of training data where m is number of training examples, and \n",
+ " n is the number of features.\n",
+ " \n",
+ " y : array_like\n",
+ " (m, ) vector of labels for ther training data.\n",
+ " \n",
+ " Xval : array_like\n",
+ " (mv x n) matrix of validation data where mv is the number of validation examples\n",
+ " and n is the number of features\n",
+ " \n",
+ " yval : array_like\n",
+ " (mv, ) vector of labels for the validation data.\n",
+ " \n",
+ " Returns\n",
+ " -------\n",
+ " C, sigma : float, float\n",
+ " The best performing values for the regularization parameter C and \n",
+ " RBF parameter sigma.\n",
+ " \n",
+ " Instructions\n",
+ " ------------\n",
+ " Fill in this function to return the optimal C and sigma learning \n",
+ " parameters found using the cross validation set.\n",
+ " You can use `svmPredict` to predict the labels on the cross\n",
+ " validation set. For example, \n",
+ " \n",
+ " predictions = svmPredict(model, Xval)\n",
+ "\n",
+ " will return the predictions on the cross validation set.\n",
+ " \n",
+ " Note\n",
+ " ----\n",
+ " You can compute the prediction error using \n",
+ " \n",
+ " np.mean(predictions != yval)\n",
+ " \"\"\"\n",
+ " # You need to return the following variables correctly.\n",
+ " C = 1\n",
+ " sigma = 0.3\n",
+ "\n",
+ " # ====================== YOUR CODE HERE ======================\n",
+ " C_array = np.array([0.01, 0.03, 0.1, 0.3, 1, 3, 10, 30])\n",
+ " sigma_array = np.array([0.01, 0.03, 0.1, 0.3, 1, 3, 10, 30])\n",
+ "\n",
+ " my_array = np.zeros([C_array.size, sigma_array.size])\n",
+ " \n",
+ " for i in np.arange(C_array.size):\n",
+ " for j in np.arange(sigma_array.size):\n",
+ " model= utils.svmTrain(X, y, C_array[i], gaussianKernel, args=(sigma_array[j],))\n",
+ " predictions = utils.svmPredict(model, Xval)\n",
+ " pred_error = np.mean(predictions != yval)\n",
+ " \n",
+ " err_array[i, j] = pred_error\n",
+ " \n",
+ " ind = np.unravel_index(np.argmin(err_array, axis = None), err_array.shape)\n",
+ " C = C_array[ind[0]]\n",
+ " sigma = sigma_array[ind[1]]\n",
+ " \n",
+ " \n",
+ " # ============================================================\n",
+ " return C, sigma"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "The provided code in the next cell trains the SVM classifier using the training set $(X, y)$ using parameters loaded from `dataset3Params`. Note that this might take a few minutes to execute."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "# Try different SVM Parameters here\n",
+ "C, sigma = dataset3Params(X, y, Xval, yval)\n",
+ "\n",
+ "# Train the SVM\n",
+ "# model = utils.svmTrain(X, y, C, lambda x1, x2: gaussianKernel(x1, x2, sigma))\n",
+ "model = utils.svmTrain(X, y, C, gaussianKernel, args=(sigma,))\n",
+ "utils.visualizeBoundary(X, y, model)\n",
+ "print(C, sigma)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "One you have computed the values `C` and `sigma` in the cell above, we will submit those values for grading.\n",
+ "\n",
+ "*You should now submit your solutions.*"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "grader[2] = lambda : (C, sigma)\n",
+ "grader.grade()"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "\n",
+ "## 2 Spam Classification\n",
+ "\n",
+ "Many email services today provide spam filters that are able to classify emails into spam and non-spam email with high accuracy. In this part of the exercise, you will use SVMs to build your own spam filter.\n",
+ "\n",
+ "You will be training a classifier to classify whether a given email, $x$, is spam ($y = 1$) or non-spam ($y = 0$). In particular, you need to convert each email into a feature vector $x \\in \\mathbb{R}^n$ . The following parts of the exercise will walk you through how such a feature vector can be constructed from an email.\n",
+ "\n",
+ "The dataset included for this exercise is based on a a subset of the [SpamAssassin Public Corpus](http://spamassassin.apache.org/old/publiccorpus/). For the purpose of this exercise, you will only be using the body of the email (excluding the email headers)."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "### 2.1 Preprocessing Emails\n",
+ "\n",
+ "Before starting on a machine learning task, it is usually insightful to take a look at examples from the dataset. The figure below shows a sample email that contains a URL, an email address (at the end), numbers, and dollar\n",
+ "amounts.\n",
+ "\n",
+ "\n",
+ "\n",
+ "While many emails would contain similar types of entities (e.g., numbers, other URLs, or other email addresses), the specific entities (e.g., the specific URL or specific dollar amount) will be different in almost every\n",
+ "email. Therefore, one method often employed in processing emails is to “normalize” these values, so that all URLs are treated the same, all numbers are treated the same, etc. For example, we could replace each URL in the\n",
+ "email with the unique string “httpaddr” to indicate that a URL was present.\n",
+ "\n",
+ "This has the effect of letting the spam classifier make a classification decision based on whether any URL was present, rather than whether a specific URL was present. This typically improves the performance of a spam classifier, since spammers often randomize the URLs, and thus the odds of seeing any particular URL again in a new piece of spam is very small. \n",
+ "\n",
+ "In the function `processEmail` below, we have implemented the following email preprocessing and normalization steps:\n",
+ "\n",
+ "- **Lower-casing**: The entire email is converted into lower case, so that captialization is ignored (e.g., IndIcaTE is treated the same as Indicate).\n",
+ "\n",
+ "- **Stripping HTML**: All HTML tags are removed from the emails. Many emails often come with HTML formatting; we remove all the HTML tags, so that only the content remains.\n",
+ "\n",
+ "- **Normalizing URLs**: All URLs are replaced with the text “httpaddr”.\n",
+ "\n",
+ "- **Normalizing Email Addresses**: All email addresses are replaced with the text “emailaddr”.\n",
+ "\n",
+ "- **Normalizing Numbers**: All numbers are replaced with the text “number”.\n",
+ "\n",
+ "- **Normalizing Dollars**: All dollar signs ($) are replaced with the text “dollar”.\n",
+ "\n",
+ "- **Word Stemming**: Words are reduced to their stemmed form. For example, “discount”, “discounts”, “discounted” and “discounting” are all replaced with “discount”. Sometimes, the Stemmer actually strips off additional characters from the end, so “include”, “includes”, “included”, and “including” are all replaced with “includ”.\n",
+ "\n",
+ "- **Removal of non-words**: Non-words and punctuation have been removed. All white spaces (tabs, newlines, spaces) have all been trimmed to a single space character.\n",
+ "\n",
+ "The result of these preprocessing steps is shown in the figure below. \n",
+ "\n",
+ "\n",
+ "\n",
+ "While preprocessing has left word fragments and non-words, this form turns out to be much easier to work with for performing feature extraction."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "#### 2.1.1 Vocabulary List\n",
+ "\n",
+ "After preprocessing the emails, we have a list of words for each email. The next step is to choose which words we would like to use in our classifier and which we would want to leave out.\n",
+ "\n",
+ "For this exercise, we have chosen only the most frequently occuring words as our set of words considered (the vocabulary list). Since words that occur rarely in the training set are only in a few emails, they might cause the\n",
+ "model to overfit our training set. The complete vocabulary list is in the file `vocab.txt` (inside the `Data` directory for this exercise) and also shown in the figure below.\n",
+ "\n",
+ "\n",
+ "\n",
+ "Our vocabulary list was selected by choosing all words which occur at least a 100 times in the spam corpus,\n",
+ "resulting in a list of 1899 words. In practice, a vocabulary list with about 10,000 to 50,000 words is often used.\n",
+ "Given the vocabulary list, we can now map each word in the preprocessed emails into a list of word indices that contains the index of the word in the vocabulary dictionary. The figure below shows the mapping for the sample email. Specifically, in the sample email, the word “anyone” was first normalized to “anyon” and then mapped onto the index 86 in the vocabulary list.\n",
+ "\n",
+ "\n",
+ "\n",
+ "Your task now is to complete the code in the function `processEmail` to perform this mapping. In the code, you are given a string `word` which is a single word from the processed email. You should look up the word in the vocabulary list `vocabList`. If the word exists in the list, you should add the index of the word into the `word_indices` variable. If the word does not exist, and is therefore not in the vocabulary, you can skip the word.\n",
+ "\n",
+ "
\n",
+ "**python tip**: In python, you can find the index of the first occurence of an item in `list` using the `index` attribute. In the provided code for `processEmail`, `vocabList` is a python list containing the words in the vocabulary. To find the index of a word, we can use `vocabList.index(word)` which would return a number indicating the index of the word within the list. If the word does not exist in the list, a `ValueError` exception is raised. In python, we can use the `try/except` statement to catch exceptions which we do not want to stop the program from running. You can think of the `try/except` statement to be the same as an `if/else` statement, but it asks for forgiveness rather than permission.\n",
+ "\n",
+ "An example would be:\n",
+ " \n",
+ "\n",
+ "```\n",
+ "try:\n",
+ " do stuff here\n",
+ "except ValueError:\n",
+ " pass\n",
+ " # do nothing (forgive me) if a ValueError exception occured within the try statement\n",
+ "```\n",
+ "
\n",
+ ""
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "def processEmail(email_contents, verbose=True):\n",
+ " \"\"\"\n",
+ " Preprocesses the body of an email and returns a list of indices \n",
+ " of the words contained in the email. \n",
+ " \n",
+ " Parameters\n",
+ " ----------\n",
+ " email_contents : str\n",
+ " A string containing one email. \n",
+ " \n",
+ " verbose : bool\n",
+ " If True, print the resulting email after processing.\n",
+ " \n",
+ " Returns\n",
+ " -------\n",
+ " word_indices : list\n",
+ " A list of integers containing the index of each word in the \n",
+ " email which is also present in the vocabulary.\n",
+ " \n",
+ " Instructions\n",
+ " ------------\n",
+ " Fill in this function to add the index of word to word_indices \n",
+ " if it is in the vocabulary. At this point of the code, you have \n",
+ " a stemmed word from the email in the variable word.\n",
+ " You should look up word in the vocabulary list (vocabList). \n",
+ " If a match exists, you should add the index of the word to the word_indices\n",
+ " list. Concretely, if word = 'action', then you should\n",
+ " look up the vocabulary list to find where in vocabList\n",
+ " 'action' appears. For example, if vocabList[18] =\n",
+ " 'action', then, you should add 18 to the word_indices \n",
+ " vector (e.g., word_indices.append(18)).\n",
+ " \n",
+ " Notes\n",
+ " -----\n",
+ " - vocabList[idx] returns a the word with index idx in the vocabulary list.\n",
+ " \n",
+ " - vocabList.index(word) return index of word `word` in the vocabulary list.\n",
+ " (A ValueError exception is raised if the word does not exist.)\n",
+ " \"\"\"\n",
+ " # Load Vocabulary\n",
+ " vocabList = utils.getVocabList()\n",
+ "\n",
+ " # Init return value\n",
+ " word_indices = []\n",
+ "\n",
+ " # ========================== Preprocess Email ===========================\n",
+ " # Find the Headers ( \\n\\n and remove )\n",
+ " # Uncomment the following lines if you are working with raw emails with the\n",
+ " # full headers\n",
+ " # hdrstart = email_contents.find(chr(10) + chr(10))\n",
+ " # email_contents = email_contents[hdrstart:]\n",
+ "\n",
+ " # Lower case\n",
+ " email_contents = email_contents.lower()\n",
+ " \n",
+ " # Strip all HTML\n",
+ " # Looks for any expression that starts with < and ends with > and replace\n",
+ " # and does not have any < or > in the tag it with a space\n",
+ " email_contents =re.compile('<[^<>]+>').sub(' ', email_contents)\n",
+ "\n",
+ " # Handle Numbers\n",
+ " # Look for one or more characters between 0-9\n",
+ " email_contents = re.compile('[0-9]+').sub(' number ', email_contents)\n",
+ "\n",
+ " # Handle URLS\n",
+ " # Look for strings starting with http:// or https://\n",
+ " email_contents = re.compile('(http|https)://[^\\s]*').sub(' httpaddr ', email_contents)\n",
+ "\n",
+ " # Handle Email Addresses\n",
+ " # Look for strings with @ in the middle\n",
+ " email_contents = re.compile('[^\\s]+@[^\\s]+').sub(' emailaddr ', email_contents)\n",
+ " \n",
+ " # Handle $ sign\n",
+ " email_contents = re.compile('[$]+').sub(' dollar ', email_contents)\n",
+ " \n",
+ " # get rid of any punctuation\n",
+ " email_contents = re.split('[ @$/#.-:&*+=\\[\\]?!(){},''\">_<;%\\n\\r]', email_contents)\n",
+ "\n",
+ " # remove any empty word string\n",
+ " email_contents = [word for word in email_contents if len(word) > 0]\n",
+ " \n",
+ " # Stem the email contents word by word\n",
+ " stemmer = utils.PorterStemmer()\n",
+ " processed_email = []\n",
+ " for word in email_contents:\n",
+ " # Remove any remaining non alphanumeric characters in word\n",
+ " word = re.compile('[^a-zA-Z0-9]').sub('', word).strip()\n",
+ " word = stemmer.stem(word)\n",
+ " processed_email.append(word)\n",
+ "\n",
+ " if len(word) < 1:\n",
+ " continue\n",
+ "\n",
+ " # Look up the word in the dictionary and add to word_indices if found\n",
+ " # ====================== YOUR CODE HERE ======================\n",
+ "\n",
+ " \n",
+ "\n",
+ " # =============================================================\n",
+ "\n",
+ " if verbose:\n",
+ " print('----------------')\n",
+ " print('Processed email:')\n",
+ " print('----------------')\n",
+ " print(' '.join(processed_email))\n",
+ " return word_indices"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Once you have implemented `processEmail`, the following cell will run your code on the email sample and you should see an output of the processed email and the indices list mapping."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "# To use an SVM to classify emails into Spam v.s. Non-Spam, you first need\n",
+ "# to convert each email into a vector of features. In this part, you will\n",
+ "# implement the preprocessing steps for each email. You should\n",
+ "# complete the code in processEmail.m to produce a word indices vector\n",
+ "# for a given email.\n",
+ "\n",
+ "# Extract Features\n",
+ "with open(os.path.join('Data', 'emailSample1.txt')) as fid:\n",
+ " file_contents = fid.read()\n",
+ "\n",
+ "word_indices = processEmail(file_contents)\n",
+ "\n",
+ "#Print Stats\n",
+ "print('-------------')\n",
+ "print('Word Indices:')\n",
+ "print('-------------')\n",
+ "print(word_indices)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "*You should now submit your solutions.*"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "grader[3] = processEmail\n",
+ "grader.grade()"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "\n",
+ "### 2.2 Extracting Features from Emails\n",
+ "\n",
+ "You will now implement the feature extraction that converts each email into a vector in $\\mathbb{R}^n$. For this exercise, you will be using n = # words in vocabulary list. Specifically, the feature $x_i \\in \\{0, 1\\}$ for an email corresponds to whether the $i^{th}$ word in the dictionary occurs in the email. That is, $x_i = 1$ if the $i^{th}$ word is in the email and $x_i = 0$ if the $i^{th}$ word is not present in the email.\n",
+ "\n",
+ "Thus, for a typical email, this feature would look like:\n",
+ "\n",
+ "$$ x = \\begin{bmatrix} \n",
+ "0 & \\dots & 1 & 0 & \\dots & 1 & 0 & \\dots & 0 \n",
+ "\\end{bmatrix}^T \\in \\mathbb{R}^n\n",
+ "$$\n",
+ "\n",
+ "You should now complete the code in the function `emailFeatures` to generate a feature vector for an email, given the `word_indices`.\n",
+ ""
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "def emailFeatures(word_indices):\n",
+ " \"\"\"\n",
+ " Takes in a word_indices vector and produces a feature vector from the word indices. \n",
+ " \n",
+ " Parameters\n",
+ " ----------\n",
+ " word_indices : list\n",
+ " A list of word indices from the vocabulary list.\n",
+ " \n",
+ " Returns\n",
+ " -------\n",
+ " x : list \n",
+ " The computed feature vector.\n",
+ " \n",
+ " Instructions\n",
+ " ------------\n",
+ " Fill in this function to return a feature vector for the\n",
+ " given email (word_indices). To help make it easier to process \n",
+ " the emails, we have have already pre-processed each email and converted\n",
+ " each word in the email into an index in a fixed dictionary (of 1899 words).\n",
+ " The variable `word_indices` contains the list of indices of the words \n",
+ " which occur in one email.\n",
+ " \n",
+ " Concretely, if an email has the text:\n",
+ "\n",
+ " The quick brown fox jumped over the lazy dog.\n",
+ "\n",
+ " Then, the word_indices vector for this text might look like:\n",
+ " \n",
+ " 60 100 33 44 10 53 60 58 5\n",
+ "\n",
+ " where, we have mapped each word onto a number, for example:\n",
+ "\n",
+ " the -- 60\n",
+ " quick -- 100\n",
+ " ...\n",
+ "\n",
+ " Note\n",
+ " ----\n",
+ " The above numbers are just an example and are not the actual mappings.\n",
+ "\n",
+ " Your task is take one such `word_indices` vector and construct\n",
+ " a binary feature vector that indicates whether a particular\n",
+ " word occurs in the email. That is, x[i] = 1 when word i\n",
+ " is present in the email. Concretely, if the word 'the' (say,\n",
+ " index 60) appears in the email, then x[60] = 1. The feature\n",
+ " vector should look like:\n",
+ " x = [ 0 0 0 0 1 0 0 0 ... 0 0 0 0 1 ... 0 0 0 1 0 ..]\n",
+ " \"\"\"\n",
+ " # Total number of words in the dictionary\n",
+ " n = 1899\n",
+ "\n",
+ " # You need to return the following variables correctly.\n",
+ " x = np.zeros(n)\n",
+ "\n",
+ " # ===================== YOUR CODE HERE ======================\n",
+ "\n",
+ " \n",
+ " \n",
+ " # ===========================================================\n",
+ " \n",
+ " return x"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Once you have implemented `emailFeatures`, the next cell will run your code on the email sample. You should see that the feature vector had length 1899 and 45 non-zero entries."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "# Extract Features\n",
+ "with open(os.path.join('Data', 'emailSample1.txt')) as fid:\n",
+ " file_contents = fid.read()\n",
+ "\n",
+ "word_indices = processEmail(file_contents)\n",
+ "features = emailFeatures(word_indices)\n",
+ "\n",
+ "# Print Stats\n",
+ "print('\\nLength of feature vector: %d' % len(features))\n",
+ "print('Number of non-zero entries: %d' % sum(features > 0))"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "*You should now submit your solutions.*"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "grader[4] = emailFeatures\n",
+ "grader.grade()"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "### 2.3 Training SVM for Spam Classification\n",
+ "\n",
+ "In the following section we will load a preprocessed training dataset that will be used to train a SVM classifier. The file `spamTrain.mat` (within the `Data` folder for this exercise) contains 4000 training examples of spam and non-spam email, while `spamTest.mat` contains 1000 test examples. Each\n",
+ "original email was processed using the `processEmail` and `emailFeatures` functions and converted into a vector $x^{(i)} \\in \\mathbb{R}^{1899}$.\n",
+ "\n",
+ "After loading the dataset, the next cell proceed to train a linear SVM to classify between spam ($y = 1$) and non-spam ($y = 0$) emails. Once the training completes, you should see that the classifier gets a training accuracy of about 99.8% and a test accuracy of about 98.5%."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "# Load the Spam Email dataset\n",
+ "# You will have X, y in your environment\n",
+ "data = loadmat(os.path.join('Data', 'spamTrain.mat'))\n",
+ "X, y= data['X'].astype(float), data['y'][:, 0]\n",
+ "\n",
+ "print('Training Linear SVM (Spam Classification)')\n",
+ "print('This may take 1 to 2 minutes ...\\n')\n",
+ "\n",
+ "C = 0.1\n",
+ "model = utils.svmTrain(X, y, C, utils.linearKernel)"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "# Compute the training accuracy\n",
+ "p = utils.svmPredict(model, X)\n",
+ "\n",
+ "print('Training Accuracy: %.2f' % (np.mean(p == y) * 100))"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Execute the following cell to load the test set and compute the test accuracy."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "# Load the test dataset\n",
+ "# You will have Xtest, ytest in your environment\n",
+ "data = loadmat(os.path.join('Data', 'spamTest.mat'))\n",
+ "Xtest, ytest = data['Xtest'].astype(float), data['ytest'][:, 0]\n",
+ "\n",
+ "print('Evaluating the trained Linear SVM on a test set ...')\n",
+ "p = utils.svmPredict(model, Xtest)\n",
+ "\n",
+ "print('Test Accuracy: %.2f' % (np.mean(p == ytest) * 100))"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "### 2.4 Top Predictors for Spam\n",
+ "\n",
+ "To better understand how the spam classifier works, we can inspect the parameters to see which words the classifier thinks are the most predictive of spam. The next cell finds the parameters with the largest positive values in the classifier and displays the corresponding words similar to the ones shown in the figure below.\n",
+ "\n",
+ "
\n",
+ "our click remov guarante visit basenumb dollar pleas price will nbsp most lo ga hour\n",
+ "
\n",
+ "\n",
+ "Thus, if an email contains words such as “guarantee”, “remove”, “dollar”, and “price” (the top predictors shown in the figure), it is likely to be classified as spam.\n",
+ "\n",
+ "Since the model we are training is a linear SVM, we can inspect the weights learned by the model to understand better how it is determining whether an email is spam or not. The following code finds the words with the highest weights in the classifier. Informally, the classifier 'thinks' that these words are the most likely indicators of spam."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "# Sort the weights and obtin the vocabulary list\n",
+ "# NOTE some words have the same weights, \n",
+ "# so their order might be different than in the text above\n",
+ "idx = np.argsort(model['w'])\n",
+ "top_idx = idx[-15:][::-1]\n",
+ "vocabList = utils.getVocabList()\n",
+ "\n",
+ "print('Top predictors of spam:')\n",
+ "print('%-15s %-15s' % ('word', 'weight'))\n",
+ "print('----' + ' '*12 + '------')\n",
+ "for word, w in zip(np.array(vocabList)[top_idx], model['w'][top_idx]):\n",
+ " print('%-15s %0.2f' % (word, w))\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "### 2.5 Optional (ungraded) exercise: Try your own emails\n",
+ "\n",
+ "Now that you have trained a spam classifier, you can start trying it out on your own emails. In the starter code, we have included two email examples (`emailSample1.txt` and `emailSample2.txt`) and two spam examples (`spamSample1.txt` and `spamSample2.txt`). The next cell runs the spam classifier over the first spam example and classifies it using the learned SVM. You should now try the other examples we have provided and see if the classifier gets them right. You can also try your own emails by replacing the examples (plain text files) with your own emails.\n",
+ "\n",
+ "*You do not need to submit any solutions for this optional (ungraded) exercise.*"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "filename = os.path.join('Data', 'emailSample1.txt')\n",
+ "\n",
+ "with open(filename) as fid:\n",
+ " file_contents = fid.read()\n",
+ "\n",
+ "word_indices = processEmail(file_contents, verbose=False)\n",
+ "x = emailFeatures(word_indices)\n",
+ "p = utils.svmPredict(model, x)\n",
+ "\n",
+ "print('\\nProcessed %s\\nSpam Classification: %s' % (filename, 'spam' if p else 'not spam'))"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "### 2.6 Optional (ungraded) exercise: Build your own dataset\n",
+ "\n",
+ "In this exercise, we provided a preprocessed training set and test set. These datasets were created using the same functions (`processEmail` and `emailFeatures`) that you now have completed. For this optional (ungraded) exercise, you will build your own dataset using the original emails from the SpamAssassin Public Corpus.\n",
+ "\n",
+ "Your task in this optional (ungraded) exercise is to download the original\n",
+ "files from the public corpus and extract them. After extracting them, you should run the `processEmail` and `emailFeatures` functions on each email to extract a feature vector from each email. This will allow you to build a dataset `X`, `y` of examples. You should then randomly divide up the dataset into a training set, a cross validation set and a test set.\n",
+ "\n",
+ "While you are building your own dataset, we also encourage you to try building your own vocabulary list (by selecting the high frequency words that occur in the dataset) and adding any additional features that you think\n",
+ "might be useful. Finally, we also suggest trying to use highly optimized SVM toolboxes such as [`LIBSVM`](https://www.csie.ntu.edu.tw/~cjlin/libsvm/) or [`scikit-learn`](http://scikit-learn.org/stable/modules/classes.html#module-sklearn.svm).\n",
+ "\n",
+ "*You do not need to submit any solutions for this optional (ungraded) exercise.*"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": []
+ }
+ ],
+ "metadata": {
+ "kernelspec": {
+ "display_name": "Python 3",
+ "language": "python",
+ "name": "python3"
+ },
+ "language_info": {
+ "codemirror_mode": {
+ "name": "ipython",
+ "version": 3
+ },
+ "file_extension": ".py",
+ "mimetype": "text/x-python",
+ "name": "python",
+ "nbconvert_exporter": "python",
+ "pygments_lexer": "ipython3",
+ "version": "3.7.6"
+ }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 2
+}
diff --git a/Phase 3 - 2020 (Summer)/Week 6(May 3 -May 9)/Exercise6/__pycache__/utils.cpython-37.pyc b/Phase 3 - 2020 (Summer)/Week 6(May 3 -May 9)/Exercise6/__pycache__/utils.cpython-37.pyc
new file mode 100644
index 000000000..9ced8b9cb
Binary files /dev/null and b/Phase 3 - 2020 (Summer)/Week 6(May 3 -May 9)/Exercise6/__pycache__/utils.cpython-37.pyc differ
diff --git a/Phase 3 - 2020 (Summer)/Week 6(May 3 -May 9)/Exercise6/exercise6.ipynb b/Phase 3 - 2020 (Summer)/Week 6(May 3 -May 9)/Exercise6/exercise6.ipynb
index cdc43975f..7dbfd0910 100644
--- a/Phase 3 - 2020 (Summer)/Week 6(May 3 -May 9)/Exercise6/exercise6.ipynb
+++ b/Phase 3 - 2020 (Summer)/Week 6(May 3 -May 9)/Exercise6/exercise6.ipynb
@@ -18,7 +18,7 @@
},
{
"cell_type": "code",
- "execution_count": null,
+ "execution_count": 1,
"metadata": {},
"outputs": [],
"source": [
@@ -40,6 +40,8 @@
"# will be used to load MATLAB mat datafile format\n",
"from scipy.io import loadmat\n",
"\n",
+ "from nltk.stem import PorterStemmer\n",
+ "\n",
"# library written for this exercise providing additional functions for assignment submission, and others\n",
"import utils\n",
"\n",
@@ -101,9 +103,22 @@
},
{
"cell_type": "code",
- "execution_count": null,
+ "execution_count": 2,
"metadata": {},
- "outputs": [],
+ "outputs": [
+ {
+ "data": {
+ "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXQAAAD4CAYAAAD8Zh1EAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjMsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+AADFEAAAgAElEQVR4nO3df5DU9Z3n8ed7xlFCEBWZQw9FssSqGBWizM1J5ZIiwdowxjW1FbZiAEEjR/ZUwJFkk+x53IWqmEoFdEDMGpcYMBOy2UJWCcVUNIAmJK7jYHSUMytMAon4g5HZ8HPxmPF9f3QPDENPd8909/fn61HVVT39/dL97tZ+9+f7+b4/76+5OyIiEn9VYQcgIiLloYQuIpIQSugiIgmhhC4ikhBK6CIiCXFWWC88evRoHz9+fFgvLyISSzt27HjX3WtzbQstoY8fP562trawXl5EJJbMbO9A2zTlIiKSEEroIiIJoYQuIpIQSugiEbdt2zauuHoSu3btyvuYiBK6SIRt27aNv/rrz7Ov6j9x8+y5vP/++zkfEwEldEmZOI12exP3iBu+ygWfWcDvO49w25e+dMZjDzStCDtUiQgLq9tiXV2dq2xRgtSbIKsum8zlw47ywnPbefbZZ894rKoqGuOcK66exL6qMVwwfQFmVZzo2sfRlmUM+8RtDBs3EYAjr2zh7Fc28OYf94QbrATGzHa4e12ubdH4P1ekwuI42t24YT0fPucwh9ffy4mufdSMGsv5sx44mcyP723n+K/XsO6xNRWPJU5HNmmmhC6pcMfCu6m6rI5zLr0Kq6pm+PV38eSW3zDihq8ybNxErKqa6iuuZ3mEEvrll1/OC89t568+cQ1HW5adsf3Ylof43sompk6dWtE4NI8fH0rokgpRGu0Oxi9/+Usef2Ijwz5x2xnbzpl0AyseeriiyTSORzZppoQuqRCV0e5g9E2mvT88fQ2/5saKJ9M4HtmkmRK6pEbYo93B6ptMIXMU0fXolzm640n8/Z5MMv1oZZNpXI9s0koJXVIhCqPdweqbTI+8soUjLd9l1bL7GHfoVQ4//r848uoWjm9fw4/X/rBiMcTxyCbNik7oZlZtZr81s005tt1qZp1m9lL2Nq+8YYqUJgqj3cHqTab/885bOfuVDWx6YgNz5syh9Te/yjzWvoGf/cvjfOpTn6poHHE7skmzouvQzeweoA4Y6e439tt2K1Dn7ncV+8KqQ5cg7dq1iy/MmsOeA8eo+sg0jv96Dd9b2cSKhx7mD+8epeqKaRzfviaQBBknhY5s/P0eDq2/l3vvuo3F9zSGEGH6lFyHbmaXAJ8FVpczMJGgRGW0GzdxPLJJs6JG6Ga2Hvg2cC7wlQFG6N8GOoHXgUZ3/1OO55kPzAcYN27c5L17B+zTLiIRoCOb6ClphG5mNwL73X1Hnt1+Box394nAL4C1uXZy90fcvc7d62prc15BSUQiREc28VJwhG5m3wZuAbqBYcBIYIO7zx5g/2qgy93Py/e8mkMXERm8kkbo7v4Nd7/E3ccDNwNb+ydzM7u4z583Aa+VEK9ILKnfiYRtyHXoZrbUzG7K/rnQzHaa2cvAQuDWcgQnEhfqdyJRMKiE7u7P9J4Qdfcl7r4xe/8b7n6lu09y90+5++8qEaxIFKnfiUSFVoqKlEj9TiQqlNBFSqR+JxIVSugiJVK/E4kKJXSRMlC/k8JUBVR5SugiJYpjJ8egqQooGEroIiVSv5P8VAUUHCV0kRJFoW95lKkKKDhK6CIlUr+T/FQFFJyi+6GXm3q5iKRHT08PX7r9dp7c8hvOn/XAadu6Hv0yq5bdx5w5c0KKLl5K7ocuIlIKVQEFQwldRCoqLVVAUSjLVEIXkYpKQxVQVMoyldBFpKKSXgUUpbJMJXQRqaikVwFFqSxTVS4iIiXoe93VD0y7k5pRY0/bfnxvO0davsumJzaUpZ9PIqpconDCQUSkvyg1Z4tFQo/KCQcRkVyiUpYZ+YQepRMOIiL9RaksM/IJPUonHCR+NFUnlRalsszIJ3T1gZCh0lTd6fTjVhlRKsuMfEKP0gkHiQ9N1Z1OP26VE6WyzFiULeabozq640nGHXqVF57bTlVV5H+fJCBXXD2JfVVjuGD6AsyqONG1j6Mtyxj2idtO/j905JUtnP3KBt78455wg62wvt+fcy65kkPr7+WmT17L409sPO2xe++6jcX3NIYdrhQQ67LFKJ1wkPjQVN0pOg+VHpFP6FE64SDxoam6U/Tjlh5FJ3Qzqzaz35rZphzbzjGzn5rZbjN73szGlyvAKJ1wkHiJSm1w2PTjlh6DGaEvAl4bYNvtwL+7+4eBB4DvlBpYryidcJD40FTd6fTjlg5FnRQ1s0uAtcC3gHvc/cZ+238O/B93f87MzgLeBmo9z5Orl4tUUv+Tosf3tnNsy0OcM+kGhl9zI1ZVzZFXt3B2e7pOiub6cfP3e3RSNEbKcVK0Cfg7YKCf8LHAnwDcvRs4CFyYI5D5ZtZmZm2dnZ1FvrTI4Gmq7hSdh0qPggndzG4E9rv7jny75XjsjNG5uz/i7nXuXldbWzuIMEUGR1N1p8T9x00LogbB3fPegG8DbwB7yEylHAOa++3zc2BK9v5ZwLtkp3MGuk2ePNlFJBjd3d2+bPn9fvGll/m2bdvOeGzr1q3hBjiArVu3+gfPu8DPnXi9X1s/xXt6enI+liZAmw+UrwfakHNnmApsyvH4ncDD2fs3A/9c6LmU0EUkn97EPeaL9/m4rz7p53/oap8zd+4Zjy1bfv+Qn/8jV030119/Pe9jUZMvoQ+5Dt3MlprZTdk/fwBcaGa7gXuArw/1eUXiTNMD5VPJBVFJbYUwqITu7s94tsLF3Ze4+8bs/ePu/jfu/mF3r3f331ciWJEoS2qSCEulFkQluc9P5FeKisRBkpNEWCq1ICrJrRCU0EXKIMlJIkyVWBCV5FYISugiZZDkJBGWSq32TXIrBCV0kTJIcpIISyUXRCW1FYISukiZJDVJhKVSC6KS3OdHCV2kDJKcJMJSqdW+SW6FEIsrFolEnZqBxceuXbv4wqw57DlwjKqPTOP4r9fwvZVNrHjoYf7w7lGqrpjG8e1rItsaItZXLBKJg7j3S0mTJPf50QhdpEx6enpoWrGS5U0rWPfYGqZOnXraYz9e+8NYJgmJlnwjdCV0EZEY0ZSLiEgKKKGLiCSEErqISEIooYuIJIQSuohIQiihi4gkhBK6iEhCKKGLiCSEErqI5KTro8aPErqInEHXR40nJXQROY2ujxpfSuhSlI6ODhYtuoPa2pFUV1dRWzuSRYvuoKOjI+zQpMx0fdT4UkKXglpaWqivn8iBA6tpajrMU085TU2HOXBgNfX1E2lpaQk7xFQr91y3ro8aX0rokldHRwezZ89g6dJjzJt3grFjoboaxo6FefNOsHTpMWbPnqGRekgqMdet66PGV8GEbmbDzKzVzF42s51m9s0c+9xqZp1m9lL2Nq8y4UrQVq5cTkPDCa68Mvf2K6+EhoYTPPjgA8EGJhWd69b1UeOpmBH6e8Cn3X0S8DFgupldl2O/n7r7x7K31WWNUkKzbl0zDQ0n8u7T0HCCdet+FFBE0qtSc926Pmp8FUzonnEk+2dN9hbOVTEkcF1dR7joovz7jBmT2U+CVam57iRfRDnpippDN7NqM3sJ2A887e7P59jt82bWbmbrzezSAZ5nvpm1mVlbZ2dnCWFLUEaNGsHbb+ff5513MvtJsCo11x3F66PGYZFTFGIsKqG7e4+7fwy4BKg3s6v67fIzYLy7TwR+Aawd4Hkecfc6d6+rra0tJW4JyMyZs2lpqcm7T0tLDTNn3hJQRNJXJea6o3YR5TgscopKjIOqcnH3PwPPANP7PX7A3d/L/vmPwOSyRCd5BVEbvnDhYlpaati5M/f2nTszCX3BgsayvaYUp5Jz3dXV1Sy+p5E3/7jn5Ai/72NBJ/MoL3KKUowFLxJtZrXACXf/s5l9AHgK+I67b+qzz8Xu/lb2/l8DX3P3XCdOT9JFokvT0tLC7NkzaGg4QUPDCS66CN5+O5NcW1pqaG5eT0NDQ0Vea8yYzDRLJV5LinfF1ZPYVzWGC6YvwKyK43vbObblIc6ZdAPDr7kRq6rmyKtbOLt9A2/+cU/Y4Q5J//d4omsfR1uWMewTt538ETvyyhbOfiW89xh0jKVeJPpiYJuZtQMvkJlD32RmS83spuw+C7MljS8DC4FbS45aBhR0bXhDQwOtre2MHj2fxsaRTJ9eRWPjSEaPnk9ra/uQkrlWnpYuinPd5RaHRU5RirHgCL1SNEIfukWL7uDAgdXMmzdwOeHq1TWMHj2fpqZVAUZWnCCPLpKup6eHphUrWd60gnWPrWHq1KmnPfbjtT8MbHqkUnp6evjS7bfz5JbfcP6s09c7dD36ZVYtu485c+aEFF1GkDHmG6ErocdQbe1ImpoOM3bswPvs2weNjSPZv/9gcIEVoaOjg/r6iSxdeiznYqWdO2HJkuG0trYzYcKE4AOUyMl3ruDojicZd+hVXnhuO1VV4S18DzLGUqdcJGLiXBuulafJEUSZXhwWOUUpRiX0GIpzbbhWniZDUGV6cVjkFKUYldBjKM614XE+upCMIMv04nDiN0oxKqHHUJxrw+N8dCEZQfZLj9oip6jHqJOiMRXX2vC4V+gI7Nq1iy/MmsOeA8f4wLQ7qRl1+tn543vbOdLyXTY9sUEtditAJ0UTqBK14UGI89GFZKhfenRphC6Bi+vRhZwSh1LCpNIIXSIlKkcXWq06NFEq05PTKaHLGYJIdBMmTKCpaRX79x+ku7uH/fsP0tS0KrDFRLpO6tBFqUwvToKo29eUi5wmDcvytVq1NH1PilZ9ZBrHf72G761sYsVDD/OHd49SdcU0jm9fE3r1SZT0HtVUXTaZy4cd5YXntvPss8+e8VgxU1SacpGipOWC0FqtWpoolenFQZB1+xqhy0lpKSmMcy8ciZ9yt9fVCF2KkpZl+VqtKkEKsr2uErqclJZEp9Wq6RTWNT+DrNtXQpeT0pLo4twLR4Ym7Gt+VuLar7kooctJaUl0Wq2aLmFf8zPIun0ldDkpqEQX9oKeCRMm0Ny8niVLhrN6dQ379kF3d+ZE6OrVNSxZMpzm5vUqWRyCsKY18gmymVih14cK1+27eyi3yZMnu0TP5s2bfdSo4T5rVo03N+NPP403N+OzZtX4qFHDffPmzWV9/l/8orzPPxi7d+/2RYvu9NrakV5dXeW1tSN90aI7fffu3YHFkCRbt271D553gZ878Xq/tn6K9/T05HwsaK+//rpf81+u8wv+YqL/5//+fb/sa5tOu425+T7/4HkX+LZt2yr++hfe0OgfPO8CX7t2rV9bPyXz2Gczj23durWo5wPafIC8qoQuZ6hUotu9e7ePGjXcV63Ct20787ZqFT5q1HAl1BjqTdxjvnifj/vqk37+h672OXPnnvHYsuX3hxJfd3e3z5k718+75PIzEvq5tWN97dq1FX/9Zcvv94svvezkD0ffx4pN5u75E7rq0GOio6ODlSuXs25dM11dRxg1agQzZ85m4cLFsZkaSEudexqVu9a63JLUTEx16DGXlL4jaalzT6Mga60HK03NxJTQIyTXycK5c2cxc+bnE7EcPy117mkU5R7paWompoQeEQONwtvafsJf/uV/JKLvSFrq3NMqqFrrwYrSNT8rrWBCN7NhZtZqZi+b2U4z+2aOfc4xs5+a2W4ze97Mxlci2KTK1xTrrbecm27K/+/jMk2Rljr3NIrytEaamokVM0J/D/i0u08CPgZMN7Pr+u1zO/Dv7v5h4AHgO+UNM9nydf87eJDETFOkbUFP2PX2QYr6tEZ1dTWL72nkzT/uOTnt0/exJCRzKCKhZytlerNFTfbWvzTmc8Da7P31wDQzs7JFmXD5Thaedx6JmaZI04KepJzILlaapjWirKg5dDOrNrOXgP3A0+7+fL9dxgJ/AnD3buAgcGGO55lvZm1m1tbZ2Vla5AmS72ThtGmweXP+fz+UaYqwRo9BXn4urPeYlr7yfaVpWiPKBlWHbmbnA/8CLHD3V/s8vhP4jLu/kf27A6h39wMDPZfq0E/J15973z6480741rco29V10nBVojDfo+rtpZLKVofu7n8GngGm99v0BnBp9sXOAs4DugYdaUrlO1k4dix84xvwta/B979vJU9TpGH0GPZ7LHe9fZrm4qU0xVS51GZH5pjZB4Drgd/1220jMDd7fwaw1cNaghpDhU4WjhgBZ501jGHDZpY8TZGGy6+F/R7LWW+ftrl4KU0xI/SLgW1m1g68QGYOfZOZLTWz3oK6HwAXmtlu4B7g65UJN5mKOVn4k59s4Ic/bGb//oN0d/ewf/9BmppWDfoEYhpWa4b9HstVbx/2kUZSRLEDZKUUU+XS7u7XuPtEd7/K3ZdmH1/i7huz94+7+9+4+4fdvd7df1/pwJMmqJOFaVitGfZ7LFe9fdhHGkkQ9oUtgqaVohEyYcIEmppWlTwKzycNqzXDfo/lqrcP+0gj7sK+sEUYlNBTJg2rNcN+j+Wqtw/7SCPuwr6wRRiU0FMmDas1o/AeyzGFFvaRRtxFuQNkpagfegr1r9EeMyaTGJJch977HjdtMp54wnnvPbjwwnMj3VNe9eyl6+np4Uu3386TW37D+bNOP9fQ9eiXWbXsPubMmRNSdEOjfuhymiBXa4al/3v8zGeM22+H9vbMIq2nnyby5X9RONKIu6h2gKwUjdAl8To6Oqivn8jSpcfKtto2KGk4mqqUQh0g/f0eDq2/l3vvuo3F98TnR1EjdEm1OJf/peFoqlKi3gGyEjRCl8TL1yun17590Ng4kv37DwYXmFTUrl27+MKsOew5cIyqj0zj+K/X8L2VTax46GH+8O5Rqq6YxvHta2LXNEwjdEm1IMv/1HclOtLYAVIJXQIVRsILqvxPfVeiJy0XtuilhB5TcRwJhpXwglhopL4rEgVK6DEUx5FgmAkviPK/OJ94leRQQo+ZuI4Ew0x4QVz6rpS+K3E82pJoUkKPmbiOBMNuNFXp8r+hnngN82hLPyTJo7LFmIlrCV51dRVPPeVUVw+8T3c3TJ9eRXd3T3CBlclQ/ruEueApDZchTCqVLSZIXDvwJb3R1FBOvIZ1tBXXaTspTAk9REM55I1rYgy7pW2lDeXEa1jTUHGdtpPClNBDMtS507gmxqQ3mhrKidewjrbCPp8hlXNW2AGkUd9D3r6jpN5D3ilTTjB79oycc6cLFy6mvn4tU6bkHmH1JsbW1mglxt6EV6jRVNSaYw1G74nXBx98gMbGH9HVdYRRo0Ywc+YttLY2nvHeMkdb+efdK3G0FddpOylMI/QQlHLIG0QJXqWkodHUYC4jGNbRVlyn7aQwVbmEoByVKh0dHTz44AOsW3f6SHDBgjNHghJNYVW56MIZ8ZavykUJPQRJL+GT4oXR7zzO/eFFZYuRo0Ne6RXGNFScp+0kP43QQ6BDXokCTdvFU0lTLmZ2KfAYcBHwPvCIu6/ot89U4EngD9mHNrj70nzPm+aErkNeERmqfAm9mLLFbmCxu79oZucCO8zsaXf/v/32+5W731hqsGmQhhI+EQlewTl0d3/L3V/M3j8MvAbkqc+QYqShhE9EgjWoOXQzGw/8ErjK3Q/1eXwq8DjwBvAm8BV3P2NNoJnNB+YDjBs3bvLevXtLCF1EJH3KUuViZiPIJO27+ybzrBeBy9x9EvAg8ESu53D3R9y9zt3ramtri31pSSG1dhUZvKISupnVkEnmP3b3Df23u/shdz+Svb8ZqDGz0WWNVFIjjldkKjf9oMlQFFPlYsBaoMvd7x5gn4uAd9zdzaweWE9mxD7gk6e5ykUGpgog9SqX/Eqdcvk4cAvwaTN7KXu7wcz+1sz+NrvPDOBVM3sZWAncnC+ZiwwkyNauURwFq1e5lEILiyRSgroiU1RHwVp0JoVo6b8UJQoj1iBau0Z5FKxe5VIKJXQBonMiMog+N1G+Yo96lUsplNAlUiPWIHqER3kUHLfGbVE4qpNTlNAlUiPWIC5VF+VRcJwuMRiVozo5RQldIjViDaK1a5RHwXG59mqUjurkFCV0idyItdJ9bqI8Co5Lr/IoHdXJKSpblMBKBaMiDouXot6rPG3/z0SJLkEneaWx9jmMS78liS6jGB7VoUtecZm3LadyTeuktcojyuch0kwjdAE0Yh2KqK42DUIaj+qiQlMuUpSoz9tGSRzm4Ssp7e8/TEroImWmEaqO6sKiOXSRMotS7X5YdBnF6NEIXWQIVOUhYdEIXaTMVOUhUaSELjIEUV5tKumlhC4yBGms3ZfoOyvsAETiqLfnSqEqD5XsSZA0QpfECWr1pqo8JGpU5SKJEvfVmx0dHaxcuZx165r7LO6azcKFizXaF0BVLhKSoPucxL1Hty4YIaVSQpeKCCM5xblHd9x/jCQaNOUiZRdWn4849+hWKwEplqZcJFBhjZSjduWlwVArASmHggndzC41s21m9pqZ7TSzRTn2MTNbaWa7zazdzK6tTLgSB2Elpziv3gzqxyit/dvTopgRejew2N2vAK4D7jSzj/bbpwG4PHubD/xDWaOUWAlrpBzn1ZtB/BjppGvyFUzo7v6Wu7+YvX8YeA3oP0v5OeAxz/hX4Hwzu7js0UoshDVSjvPqzUr/GOmkazoMag7dzMYD1wDP99s0FvhTn7/f4Mykj5nNN7M2M2vr7OwcXKQSG2GNlHtXby5ZMpzVq2vYty/T8XDfvswJxSVLhkd29Walf4ziXAEkxSs6oZvZCOBx4G53P9R/c45/ckb5jLs/4u517l5XW1s7uEglNsIcKcd19Walf4x00jUdiipbNLMaYBPwc3e/P8f27wPPuPtPsn//GzDV3d8a6DlVtphsuprN0FTqMoDq354cJV2CzswMWAt0ufvdA+zzWeAu4AbgvwIr3b0+3/MqoSefrlEaHXGu0ZfTlZrQ/xvwK+AV4P3sw38PjANw94ezSX8VMB04Btzm7nmztRK6SHC0cCk58iX0gu1z3X07uefI++7jwJ1DC09EKm3hwsXU169lypTcJ0Z7z2u0tkavAkiKp37oIimg/u3poKX/IikR1wogKZ6ac4mIxIiac4mIpIASuohIQiihi4gkhBK6iEhCKKGLiCSEErqISEIooYuIJIQSuohIQiihi4gkhBK6iEhCKKGLiCSEErpIDh0dHSxadAe1tSOprq6itnYkixbdoYsoS6QpoYv009LSQn39RA4cWE1T02GeesppajrMgQOrqa+fSEtLS9ghiuSkfugifXR0dDB79gyWLj122oUgxo6FefNOMGXKCWbPnkFra7t6h0vkaIQu0sfKlctpaMh9VR+AK6+EhoYTPPjgA8EGJlIEJXSRPtata6ahYeDrbkImoa9b96OAIhIpnhK6SB9dXUe46KL8+4wZk9lPJGqU0EX6GDVqBG+/nX+fd97J7CcSNUroIn3MnDmblpaavPu0tNQwc+YtAUUkUjwldJE+Fi5cTEtLDTt35t6+c2cmoS9Y0BhsYCJFUNmiSB8TJkyguXk9s2fPoKHhBA0NJxgzJjPN0tJSQ0tLDc3N61WyKJFUcIRuZo+a2X4ze3WA7VPN7KCZvZS9LSl/mCLBaWhooLW1ndGj59PYOJLp06tobBzJ6NHzaW1tp6GhIewQRXIyd8+/g9kngSPAY+5+VY7tU4GvuPuNg3nhuro6b2trG8w/ERFJPTPb4e51ubYVHKG7+y+BrrJHJSIiZVWuk6JTzOxlM2sxswHW2IGZzTezNjNr6+zsLNNLi4gIlCehvwhc5u6TgAeBJwba0d0fcfc6d6+rra0tw0uLiEivgnPoAGY2HtiUaw49x757gDp3f7fAfp3A3gJPNxrI+zwhiWpcoNiGKqqxRTUuUGxDVWpsl7l7zhFxyWWLZnYR8I67u5nVkxn1Hyj07wYKqN9ztw00+R+mqMYFim2oohpbVOMCxTZUlYytYEI3s58AU4HRZvYG8L+BGgB3fxiYAfwPM+sG/gO42YsZ9ouISFkVTOju/sUC21cBq8oWkYiIDEnUl/4/EnYAA4hqXKDYhiqqsUU1LlBsQ1Wx2Io6KSoiItEX9RG6iIgUSQldRCQhQk/oZjbdzP7NzHab2ddzbD/HzH6a3f58tiY+KrHdamadfRqTzQsorkIN08zMVmbjbjeza4OIq8jYQmnmZmaXmtk2M3vNzHaa2aIc+4TyuRUZW1if2zAza82uBN9pZt/MsU8o39EiYwvlO5p97Woz+62ZbcqxrTKfmbuHdgOqgQ7gL4CzgZeBj/bb5w7g4ez9m4GfRii2W4FVIXxunwSuBV4dYPsNQAtgwHXA8xGKbSqZRWpBf2YXA9dm758LvJ7jv2con1uRsYX1uRkwInu/BngeuK7fPmF9R4uJLZTvaPa17wHW5frvVqnPLOwRej2w291/7+7/D/gn4HP99vkcsDZ7fz0wzcwsIrGFwgs3TPscme6Y7u7/CpxvZhdHJLZQuPtb7v5i9v5h4DVgbL/dQvnciowtFNnPovcCqjXZW/9KilC+o0XGFgozuwT4LLB6gF0q8pmFndDHAn/q8/cbnPk/8sl93L0bOAhcGJHYAD6fPTxfb2aXBhBXMYqNPSxFNXOrlOzh7TVkRnR9hf655YkNQvrcslMHLwH7gafdfcDPLeDvaDGxQTjf0Sbg74D3B9hekc8s7ISe6xep/y9sMftUQjGv+zNgvLtPBH7BqV/csIX1mRWj6GZulWBmI4DHgbvd/VD/zTn+SWCfW4HYQvvc3L3H3T8GXALUm1n/nk6hfW5FxBb4d9TMbgT2u/uOfLvleKzkzyzshP4G0PcX8xLgzYH2MbOzgPMI5pC+YGzufsDd38v++Y/A5ADiKkYxn2so3P1Q72Gyu28GasxsdBCvbWY1ZBLmj919Q45dQvvcCsUW5ufWJ4Y/A88A0/ttCus7WjC2kL6jHwduskyjwn8CPm1mzf32qchnFnZCfwG43Mw+ZGZnkzk5sLHfPhuBudn7M4Ctnj2TEHZs/eZXbyIz9xkFG4E52aqN64CD7v5W2EFBpplb71yhDaKZWxle14AfAK+5+xrhvdsAAADlSURBVP0D7BbK51ZMbCF+brVmdn72/geA64Hf9dstlO9oMbGF8R1192+4+yXuPp5M3tjq7rP77VaRzyzUi0S7e7eZ3QX8nExVyaPuvtPMlgJt7r6RzP/oPzKz3WR+wW6OUGwLzewmoDsb261BxGaFG6ZtJlOxsRs4BtwWRFxFxhZWM7ePA7cAr2TnXAH+HhjXJ7awPrdiYgvrc7sYWGtm1WR+RP7Z3TdF4TtaZGyhfEdzCeIz09J/EZGECHvKRUREykQJXUQkIZTQRUQSQgldRCQhlNBFRBJCCV1EJCGU0EVEEuL/A16AFLXSsQG6AAAAAElFTkSuQmCC\n",
+ "text/plain": [
+ "
"
+ ]
+ },
+ "metadata": {
+ "needs_background": "light"
+ },
+ "output_type": "display_data"
+ }
+ ],
"source": [
"# Load from ex6data1\n",
"# You will have X, y as keys in the dict data\n",
@@ -147,9 +162,22 @@
},
{
"cell_type": "code",
- "execution_count": null,
+ "execution_count": 3,
"metadata": {},
- "outputs": [],
+ "outputs": [
+ {
+ "data": {
+ "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXQAAAD4CAYAAAD8Zh1EAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjMsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+AADFEAAAgAElEQVR4nO3deXxU5fX48c9JDCAKCJICBYEaaXEDWYzw6/JFxZYg1drSqoDgglRBQUC/tS6o1LUChs36VbSEIhaL1IWSugDaWi0YVAIUhcSqFRcoKIuUSuD8/ngmEkOWSTJzn3tnzvv1mpfDzCVzvJozz33uec4jqooxxpjoy/AdgDHGmMSwhG6MMSnCEroxxqQIS+jGGJMiLKEbY0yKOMzXB7du3Vo7d+7s6+ONMSaSVq9e/W9Vza7qPW8JvXPnzhQVFfn6eGOMiSQRea+692zKxRhjUoQldGOMSRGW0I0xJkVYQjfGmBRhCd0YY1KEJXRjjEkRltCNMSZFRC6hf/IJTJwIW7f6jsQYY8Ilcgl9xQqYPh1ycuBXv4Ldu31HZIwx4RC5hH7BBbBuHZx1FkyaBMcdB/ffD/v2+Y7MGGP8ilxCB+jaFZ54Al59Fb71LRgzBk44ARYuhAMHfEdnjDF+RDKhl+vTB158Ef70Jzj8cDd6P+00WLbMd2TGGBO8SCd0ABEYOBDeeAMKCmDLFujfH37wA/eaMVG3YsUKjj+5O5s2barxNWMin9DLZWbC8OHw9tswdSoUFUHPnjBkCLzzju/oTFhELTmuWLGCH573EzZnfI0Lho3gwIEDVb5mDKRQQi/XpAlMmOCS+A03wJNPujn3sWPd6N2kr6glx/LYjhx4HS1/cDXvbN3NJZdeeshr9+VP9x2qCYmUS+jlWrSAO+6AkhK4+GJXCZOTA5MnW6ljOopichw99hoyOvWm8TEnIRmZNO1/FU8te4UjB15Hk47dkIxMMo/vz9QAYo7alU26ElX18sG9e/fWIDe4eOstuPFGWLwYvvY1V/J4+eXQqFFgIRiPjj+5O5sz2tBywNWIZLBv+2Y+L5xCk+9eQpOO3QDYvXYZjdYu5sP33/UbbMymTZs4f+hw3t22h8PPHENWq/ZfeX/ve8XsLryXJU8upl+/fkmLo/zLMKNTL7o0+ZzXXn2Zl1566ZDXMjJSdnwYKiKyWlV7V/Ve2vwXqFjq2LUrXHWVlTqmk6cXL+K4xrvYtegm9m3fTFar9hw19L4vk/ne94rZ+7e5LJg312ucFXXp0oXXXn2ZH363B58XTjnk/T3LZnP/jPxAknmUrmzSWdok9HIVSx2bNnWljrm5VuqY6sKQHOvjL3/5C088+TRNvnvJIe817j6Q6bMfSOq8f5imfUzt0i6hw1dLHefNc31h+veH738fXn/dd3QmWXwnx7qqODouv5KoqGmPQUkfHUfxyiadpWVCL5eZCRdd5Eodp01zybxXL7jwQigt9R2dSaQwJMe6qjg6Bpc8tz/ycz5f/RR6YL8bHZ+Q3NFxVK9s0lXcCV1EMkXkDRFZUsV7F4vIVhF5M/YYmdgwk6tJExg/3iXxG2+Ep55y8+xXX22ljqkiDMmxriqOjnevXcbuwnuZNeVOOu5cx64nbmb3umXsfXkujxb8NqlxRO3KJp3VZYQ+DthQw/sLVfWU2GNOA+PyokULuP12V+p42WXwm9+4UsfbboNdu3xHZxoiLMmxLspHxzeOuZhGaxez5MnFDB8+nFWv/NW9VryYZ/74BKeffnrSYojilU06i6tsUUQ6AAXAHcAEVR1U6f2Lgd6qelW8Hxx02WJ9vP023HQTLFpkpY6pYP/+/eRPn8HU/OksmDeXfv36feW1Rwt+m9TkGEWVyz33vlfMnmWzadx9IE17DEIyMtm9bhmNisNT7pnqaipbRFVrfQCLgF5AP2BJFe9fDHwEFMeOPaaanzMKKAKKOnbsqFHx97+r9uunCqrHHqv62GOq+/f7jsqY5Nu4caP2OLWPtjy2mx49cLwe0aKlFhQUaM/cvu61s91ry5cv9x1q2gCKtLpcXd0bejAJDwLujz2vLqEfDTSOPb8CWF7bz+3Vq1cw//YJcuCA6tKlqt26ubPWo4fqs8/6jsqY5CsrK9MpU6dpu2M66YoVKw55zZJ5sGpK6LVOuYjIXcBFQBnQBGgOLFbVYdUcnwlsV9UWNf3cKEy5VOXAAViwwE3FvPcenHkm3HOPq44xxphka9BKUVX9pap2UNXOwAW40fdXkrmItKvwx3Oo+eZppGVkwLBhbn49Px/efBN693YLlEpKfEdnjEln9a5DF5HJInJO7I9jRWS9iKwBxuLm1FNa48YwbpwrdbzpJnjmGTj+eNdS4JNPfEdnfLAGVsa3tGnOlWwffeQ2rX7wQVfXPnEiXHstNGvmOzITBGtgZYJizbkC0K6da9G7YYNrKzB5sqthnzkTvvjCd3QmmayBlQkLS+gJ1qULPP44rFoFJ53kNtY4/nh47DHr6piqrIGVCQtL6Ely6qmug+Of/+ymXYYMcTdPn3vOd2Qm0ayBlQkLS+hJJOI2q379dZg/Hz791P25f3+356lJDdbAKj520zj5LKEHICMDhg51uyZNnw5r1rgR/Pnng/1/nBqsgVXNorafa1RZQg9Q48ZuTr20FG6+2W2yccIJMGYMfPyx7+hMfVkDq5rZTePgWNmiRx9/fLDUsXFjV+o4cSI0b+47MlMX1sCqZlHczzXMrGwxpNq2hdmz4R//gLPPPljqOGMG/Pe/vqMz8Ypia94g2U3j4FhCD4EuXdxm1a+9Bt26uRWoxx/vesbYtGL4haFveZjZTePg2JRLyKjC88/DL37h+sSccgrcfbfb71TEd3TG1E9N9xk+X/0UHXeus5W0cbIplwgRccl79Wp49FHYsQMGDHBdHV97zXd0xtRdutw0DkNZpiX0kMrIcIuR3nrLzamvWwe5ufCzn1mpo4mWKO7nWldhKcu0hB5yjRq5zapLStwWeEuXuvn1K690DcGMCbtUv2kcprJMm0OPmE8+cdUwDz7okv2ECXDddVbqaMItlfdzDboss6Y59Mgk9BUrVjB67DU8vXgRXbp0qfa1dFFS4vqwL1wIrVu751dc4erZjTHB2bRpE+cPHc672/Zw+JljyGrV/ivv732vmN2F97LkycUJqeSJ/E3RsMxPhclxx8Hvf+96wnTvDtdcA127up4xaXYqjPEqTGWZoU/oYZqfCqNeveCFF1wXx5Yt4aKLoGdP1+XR08WXMWknLL18Qp/Qrdd0fM46y43WFyyAnTshLw/OOMP1ZTfGJE+YyjJDn9Bt2XD8MjLgwgsPljquXw+nnQY//Sls3Og7Oj/CUBscFnYukiNUZZmq6uXRq1cvjVdZWZkOHzFCW3Toop1+seQrj2bZ7bWgoCDun5VOdu5UveUW1SOPVM3MVL3iCtUPP/QdVXCWL1+uR7Roqc269deeuX11//79Vb6WDuxcJM/GjRu1x6l9tOWx3fTogeP1iBYttaCgQHvm9nWvne1eW758eUI+DyjSavJq6EfoEJ75qahp1gxuvdVVxFx5JcyZ426m3nSTW4Gayuzey0F2LpIrTL18Ql+2WNv8lB7Yz85FN3HTVZcwccL4ZISaMkpLXTL//e/h6KPhxhth9OjULHW0lq0H2blILZEuWwzV/FTE5eS4zaqLilwlzIQJ8K1vwe9+B/v3+44usezey0F2LtJH3AldRDJF5A0RWVLFe41FZKGIlIjIShHpnKgAU33ZsA+9erkyx+eecyP14cNdgl+6NHVKHcNUG+ybnYv0UZcR+jhgQzXvXQZ8qqrHAfcB9zQ0sHJhmp9KNWed5To4PvYYfP6522QjlUod7d7LQXYu0kNcCV1EOgBnA3OqOeRcoCD2fBFwpkjiundnZmYyccJ4Pnz/3S9HERVfs2RefxkZcMEFbtekmTPdP087DQYPhrff9h1d/YWpNtg3OxfpI94Rej7wv0B1X+HtgX8BqGoZsAM4uvJBIjJKRIpEpGjr1q31CNckS6NGcNVVriLm1lvh2WfhxBNdf5godnW0ey8H2blIH7UmdBEZBGxR1dU1HVbFa4fMxqrqg6raW1V7Z2dn1yFME5RmzeCWW1xFzOjR8Mgj7mbqjTdGq9TR7r0cFPVzYQui6qC6AvXyB3AX8AHwLvAxsAeYX+mYZ4G+seeHAf8mVhJZ3aMuC4uMP6WlqhdeqAqqrVqpTpumunev76jiU1ZWplOmTtN2x3TSFStWHPJaohZ6REFUz4UtiDoUNSwsqtPqTqAfsKSK18cAD8SeXwA8XtvPsoQeLatXq37/++7/mE6dVAsKVMvKfEdlUll54m5z4Z3a8bqn9KhvnKzDR4w45LUpU6fV++d3Pambbty4scbXwqamhF7vOnQRmSwi58T++DBwtIiUABOA6+v7c0049ezp5tWff971Xx8xAnr0gD/9KXVKHRPBpgcSJ5mN+VK1JXedErqqvqiqg2LPJ6nq07Hne1X1p6p6nKrmquo7yQjW+Ne/vytrXLgQ9uyBQYOgXz/4+999R+ZfqiYJX5K1ICqVWyGEfqWoCZ+MDLdZ9YYNMHu2K2/s2xd+/GPX6TEdpXKS8CVZC6JSuSW3JXRTb1lZrhKmpMTtc/r883DSSTBqFGze7Du6YKVykvApGQuiUrkVgiV002BHHgk33+xKHceMgblzoUsXuOEG+Owz39EFI5WThC/JWhCVyq0QLKGbhPna12D6dDcF8+Mfw113uRr2qVNh717f0SVXKicJX5K5ICpVWyFYQjcJ941vuM2qX38deveGa6+Fb34TCgpSr6tjRamaJHxJ1oKoVG6FYAndJE2PHq7UcdkyaNMGLr4YTjklNUsdUzlJ+JKsxnyp3Aoh9BtcmNSgCn/4g2shUFIC3/0u3HOPq45JBZU3kdj7XjF7ls2mcfeBNO0xCMnIZPe6ZTQqtk0kfNu0aRPnDx3Ou9v2kNH1TPb+bS73z8hn+uwH+Oe/Pyfj+DPZ+/Lc0HZxrWmDi0jsKWpSxxdfqN5/v2qbNm7V6XnnqW7Y4Duqhgt6X0nTMFFthaCawKX/iXxYQk9vu3ap/upXqs2aqWZkqI4cqfrBB76japgoJwkTHTUldJtyMV5t3Qp33AH33w+ZmXDNNfCLX8BRR/mOzJhwivSeoia1ZWdDfr4rdRw82M2rH3ssTJmS+qWOxiSaJXQTCt/4htus+vXX3Y5J113nSh1/+9vULnU0JpEsoZtQOeUUKCyE5cuhbVu49FLo3h2eeSb1Sh2NSTRL6CaUTj8dVq50pY779sE558D3vgevvOI7MmPCyxK6CS0RN6++bh088ICrX//2t+FHP3KbWRtjvsoSugm9rCz4+c9dQr/9djcdc/LJMHIkfPCB7+iMCQ9L6CYyjjjCrTR95x0YO9bdRO3SxZU5fvqp7+iM8c8Suomc1q3hvvtcqeNPfwr33utKHe+9F/7zH9/RpQ7bTi96LKGbyOrcGebNgzffdD1h/vd/rdQxUWw7vWiyhG4ir1s3WLoUVqyAr3/dlTp26wZPP22ljvVh2+lFlyV0E5fS0lLGjRtNdnZzMjMzyM5uzrhxoyktLfUd2pfKN6tetAjKyuDcc11Xx7/9zXdkyZXoqRHbTi+6LKGbWhUWFpKb241t2+aQn7+L555T8vN3sW3bHHJzu1FYWOg7xC+JwE9+crDU8Z134Dvfccl9/Xrf0SVeMqZGbDu96Ko1oYtIExFZJSJrRGS9iNxWxTEXi8hWEXkz9hiZnHBN0EpLSxk2bDCTJ+9h5Mh9tG/vmmi1bw8jR+5j8uQ9DBs2OFQjdThY6rhpk2v+9eKLbhrmssvgX//yHV1iJGtqxLbTi654Ruj/Bc5Q1e7AKcAAEelTxXELVfWU2GNOQqM03syYMZW8vH2ceGLV7594IuTl7WPmzPuCDSxORxzhNqsuLYVx49zWeF26uBuo27f7jq5hkjk1YtvpRVOtCT3Wgnd37I9ZsYfdakoTCxbMJy9vX43H5OXtY8GC3wUUUf20bg3TpsHGjXD++a6bY04O/PrX0S11TNbUiG2nF11xzaGLSKaIvAlsAZ5X1ZVVHPYTESkWkUUickw1P2eUiBSJSNHWrVsbELYJyvbtu2nbtuZj2rRxx0VBp05us+o334T/9//coqQuXeDhh92N1ChJ1tRIGPfcjEJNfBhijCuhq+p+VT0F6ADkishJlQ55Buisqt2AF4CCan7Og6raW1V7Z2dnNyRuE5BWrY7k449rPuaTT9xxUdKtm9us+sUXoUMH10agWzd46qlolTomY2qk4sh/99pl7C68l1lT7qTjznXseuJmdq9bxt6X5/JowW8T9a9RoyjUxIclxjpVuajqZ8CLwIBKr29T1f/G/vgQ0Csh0ZkaBVFKOGTIMAoLs2o8prAwiyFDLkrYZwbpf/4HXn0VnngCDhxwjb++8x14+WXfkdUuWVMj5SP/G8dcTKO1i1ny5GKGDx/Oqlf+6l4rXhzYBspRqIkPU4y1bkEnItnAPlX9TEQOB54D7lHVJRWOaaeqH8Wenwf8QlWrunH6JduCrmEKCwsZNmwweXn7yMvbR9u28PHHLrkWFmYxf/4i8vLyGvw5paWl5OZ2Y/LkPVXeGF2/HiZNasqqVcXk5OQ0+PN8Kitzq0xvuQU++gh++EO46y6qvSHs2/End2dzRhtaDrgakQz2vlfMnmWzadx9IE17DEIyMtm9bhmNihfz4fvv+g63Xir/O+7bvpnPC6fQ5LuXfPkltnvtMhqt9ffvGHSMDd2Crh2wQkSKgddwc+hLRGSyiJwTO2ZsrKRxDTAWuLjBUZtqBVlKmJOTw/z5i5g0qSlz5mSxebNLfJs3w5w5WUya1JT58xfVOZmHcaHSYYfB5Ze7ro533gl/+Yubhrn00nCWOoZtaiQZolATH6oYq9s9OtmPXr16NXz76zQ1duyVOnRolq5YQbWPoUOzdNy4MQn7zJKSEh03boxmZzfXzMwMzc5uruPGjdGSkpI6/6ylS5dqq1ZNdejQLJ0/H33hBXT+fBdzq1ZNdenSpQmLuyH+/W/ViRNVGzd2j2uvVd22zXdUX1VWVqZTpk7Tdsd00hUrVhzy2vLly/0GmABlZWU6fMQIbdGhi3b6xZKvPJplt9eCggLfIQYaI1Ck1eTVWqdcksWmXOovO7s5+fm7aN+++mM2b4bx45uzZcuO4AKLQxSncN57z03DzJsHLVrA9de79r2HH+47svRQ072Cz1c/Rced63jt1ZfJyPC38D3IGBs65WJCJsqlhFFcqNSpE8ydC2vWuB2Trr/elTrOmRO9UsdECqJMLwo18WGK0RJ6BEW5lDDKC5VOPhmWLIGXXoJjjnHz7SefDE8+Ga1Sx0QIqkwvjDXxYY7REnoERbmUMMpXF+XKN6tevNgl8vPOcyP3v/7Vd2TBCLJMLwo3fsMUoyX0CBo7diKFhVnVdg9cv94l9KuvHh9sYHGI8tVFRSIuka9bBw895ObZv/c9V+q4bp3v6JIryPa6YaqJj0KMdlM0oirXobdp4xJhouvQE23cuNFs2zaHkSOrn3aZMyeL1q1HkZ8/K8DIGmbPHpg+He65B3buhOHDYfJk6NjRd2SJt2nTJs4fOpx3t+3h8DPHkNXqq3fn975XzO7Ce1ny5GLryJgEdlM0BeXl5bFqVTGtW49i/PjmDBiQwfjxzWndehSrVhWHMplDtK8uatK0Kfzyl66r48SJ8Pvfu+3wrr0Wtm3zHV1iWXvd8LIRuglcVK8u6uL9912pY0EBNG9+sNSxaVPfkSVGFEoJU5WN0E2ohOXqIpmrVTt2dG0EiovdNni//KUrdXzooeiXOoapTM98lSV0c4ggluXn5OSQnz+LLVt2UFa2ny1bdpCfPyuwxURBbat30knwzDOujUCnTjBqlHvtj3+MbqljmMr0oiSQ9rrVLSFN9sOW/odTVJblN0RJSYm2atVUZ82qum3CrFloq1ZN69XWoCYHDqj+8Y+qXbuqgmqfPqovvZTQjwjExo0btcepfbTlsd306IHj9YgWLbWgoEB75vZ1r53tXkuFtgOJsnz5cj2iRUtt1q2/9sztq/v376/ytXhQw9J/G6GbL0V1/9C68rVaVcS151271q0yff9917530CD3WlSEqUwvCoKs27ebouZLqVpSWFlYeuHs2QMzZ8Ldd8OOHXDRRa7UsVOnpH2k8SDR7XXtpqiJS5SX5ddFWFarNm3qtsArLXXljQsXulLHiRNTr9QxDHxtERdke11L6OZLYUl0yRa21aqtWrnNqjdtgmHDID8fjj3W9WT//PNAQkh5PreIC7Ju3xK6+VLYEl2yhLUXzjHHuM2qi4uhXz+48UZX6vh//xf9UkefwrBFXDL2fq2KJXTzpaASne/disK+WvXEE91m1S+/7EbqV1zhXitvBhZmvqY1ahJk75mqBFm3bwndfCmIRBdU/XdNkrWtXqKVd3B86im3Pd5PfgJ9+7r2vWHkc1qjJr63iAu0br+6esZkP6wOPZwq16E//3zi6tB91X/XFE+ittVLtn37VB9+WLVDB1fDnpenumaN76gOKq+pbnPhndrxuqf0qG+crMNHjDjktSlTp3mJz+c2domu28e2oDN1UVpaysyZ97Fgwe/Yvn03rVodyZAhF3H11eMbNGpNl7LIZPrPf1yp4113uVLHYcNcqWPnzn7jSnRpXqL57j2zf/9+8qfPYGr+dBbMm0u/fv2+8tqjBb+Nu26/prJFS+gRUVpayowZU1mwYH6FJDuMsWMnep8aiFdY6r9Twaefuvr1GTPgwAEYMwZuuAFat/YTT5hb6tY2h60H9rNz0U3cdNUlTJwQ/i6fVoceEdXdLHzkkUe8zzsnQrqURQahZUvXe7281HH6dMjJgTvu8FPqGOaWuunUe8YSekhUd7Pw3XcfYsyYy1JiOX66lEUGqUMHV+q4di2cfjrcdBMcd5wrddxX8xqxhAuqNK+uwrRFXLLVmtBFpImIrBKRNSKyXkRuq+KYxiKyUERKRGSliHRORrCpqqYeKo0alXHeeQTedyQZwlr/nQpOOMFtVv3yy26kfsUVrqvjE08EU+oY5pa66dR7Jp4R+n+BM1S1O3AKMEBE+lQ65jLgU1U9DrgPuCexYaa2mppFLVsGZ59d89+PynL8sNd/J5qPevvyUsenn3aljoMHQ58+sGJF0j4SCP+0RmZmJhMnjOfD99/9ctqn4mupkMwhjoQeq5Qpn9TMij0qf+efCxTEni8CzhQRSViUKa6mHio7dpCUeWcfySYq9d+J4LPeXsRtVl1cDI88Ah9+CGecAXl5sGZNcj4znaY1wiyuOXQRyRSRN4EtwPOqurLSIe2BfwGoahmwAzi6ip8zSkSKRKRo69atDYs8hdR0s7BFCxI+7+wz2QS5W5GvFalhaUOcmQmXXAIbN8K998LKldCjh+vq+O67if2sdJrWCLM6lS2KyFHAH4GrVXVdhdfXAz9Q1Q9ify4FclW12p5xVrZ4UE3lfLNnQ6NGcPnl1f/9utRul5aWkpvbjcmT91Q5xbN+PUya1JRVq4ojPVKuvG9p27buizGIfUvDWm//6aeuMmb6dFfqOHq06xfjq9TR1E/CyhZV9TPgRWBApbc+AI6JfdhhQAtge50jTVM13Sz80Y/gT38iYfPOvjZ3CJLvEXKi2xAn6kqjZUtXu75pkxulz5jhesXcfrt1dUwV8VS5ZMdG5ojI4UB/4K1Khz0NjIg9HwwsV18rliKoppuF7dvDkCGuX/ZDDx3W4HnndOh57vtLK5H19smYHuvQwe2YtG4dnHkm3HyzK3V84IHgSx1NYsUzQm8HrBCRYuA13Bz6EhGZLCLnxI55GDhaREqACcD1yQk3NdV2s3DhwqbMnv0w2dk/b/C8czos7vH9pZWoevtkX2kcf7zbrPpvf3MJ/cor3ZfdH/4Q/q6OdRHGDpDJYkv/QyRZPVQqSofl95mZGTz3nJKZWf0xZWUwYEAGZWX7E/75iZpDD3IuXtVN7V1/vZvGO/VUN98e9XuY5fXxGZ160aXJ57z26su89NJLh7yWrB4uyWBL/yMiJyeH/PxZbNmyg7Ky/WzZsoP8/FkJvTmZDot7fK9ITVS9fZBXGiJus+o1a2DuXHcDOdmljskWho0tgmYJPc2kw+Ie319aiaq39zE9lpkJI0YcWuo4bBj8858J+5hA+N7YwgdL6GkmHRb3xPOltWjRPubNm5u0uvRE1Nv7vNJo0sTdiH/nHbeR9eLF8K1vwTXXQFSWkPje2MIHm0NPU0HM1/tUuQ69TRuX/J55BgoL3Vxxx47B1KXXV5jq2Tdvhttuc43AjjgCrrsOJkxwz8Ns//79XHrZZTy17BWOGvrVqqbtj/ycWVPuZPjw4Z6iqx/rh27SUvmX1vz5BWzfvpvmzeGss1xtf8WbwmFdTBXGRWAbNri+608+6aZ7brkFRo6ErJpnuLzxvbFFMthNUZOWym8yDx16EUOGZPHkk24jiMoVPmFdTBXG6bHyUsdXXoFvftOtNj3hBHj88fCVOoa5A2SyWEI3Kc93XXpDBNn7pi7KN6tessTNt59/PuTmwvLlXsKpUtg7QCaDJXST8oKsFklGQ7AgylnrQ8S1dn7zTSgogC1b3MrTH/wA3njDa2hAenaAtIRuAuWjA2JQ1SI+u1j6lJkJw4fD22/D1KlQVAQ9e8LQoa5Kxpd07ABpN0UjKoqbRvvqgBhEtUgYb2D68tln8OtfQ36+m/O/8kq3NV52tu/IUoPdFE0xURwJ+uyAGMRiKt8NwcLkqKPgzjuhpMT1Y58923V1nDwZdke3RVAkWEKPGN+tYevLZ8ILolqkITdefW3EkWxf/7rbrHrdOvj+912JY06OS/BffOE7utRkCT1iojoS9F1pkuxqkfreePV5tRXUF0nXrm6z6ldfdWWPV13lSh0XLnQbbZjEsTn0iIlqt0TfHRCTrT7/XXzOu/u6n6F6cKXu2rXQq5fr6njmmQn/qJRlc+gpJKr9zH13QEy2+jQE83W15XPaTgQGDnRljQUFri9M//5uSub11xP+cWnHErpH9bnkjWpi9N0BMdnqc+PV1zRUGKbtKpY6TpvmknmvXm53Ln+JZ8oAAA7fSURBVJ+ljlFnCd2T+s6dRjUxpnrb3vrcePV1teX7fkZFTZrA+PFQWnqwR0zXrjB2rFuoZOrGEroHDbnkjWpiDGNfkkSr641XX1dbYZy2a9EC7rjDlTpeeincf7+riLntNti1K7AwIs8SugcNueSNcmIMa1+SRKrLMn1fV1thnrb7+tfdZtXr17sWArfe6vY7nTXLSh3jYVUuHiSiUiXV+5mnA19VLmHqs16blSvdBhsvveQWJ91xB/zsZxChbrcJZ/3QQybVS/hM/KrbiCOZ5YNRa1OgCn/+syt1LC52fWLuvtv1tk9HVrYYMmG+5DXB8jENFbVpOxG3WfUbb8C8ebBtmytz7N8fVq/2HV241DpCF5FjgHlAW+AA8KCqTq90TD/gKaB8G9nFqjq5pp+bziP0KF3ymtQV1Wm7//4XfvMbuP12l9zPP989P+4435EFo0FTLiLSDminqq+LSDNgNfAjVf1HhWP6Adeq6qB4g0rnhB61S15jwmjHDpgyxdWxf/EF/PzncPPNrkInlTVoykVVP1LV12PPdwEbgBpu55naRO2S15gwatECfvUrV+o4cqSrjsnJcU3A0rXUsU5z6CLSGegBrKzi7b4iskZECkWkyoI8ERklIkUiUrR169Y6B5tK0qGEz5ggtGvnpmD+8Q/XVmDyZJfYZ85Mw1JHVY3rARyJm275cRXvNQeOjD0fCGyq7ef16tVLjalOSUmJjh17pbZu3UwzMkRbt26mY8deqSUlJb5DMyG3cqVqv36qoHrssaoLFqju3+87qsQBirSavBrXCF1EsoAngEdVdXEVXwo7VXV37PlSIEtEWjf868akoyhu4GHCo3yz6sJCaNbM9Yfp3Ruef953ZMlXa0IXEQEeBjao6rRqjmkbOw4RyY393G2JDNSkh6hu4JFoqbrpRVBEYMAA1/Rr/nz49NODpY6pXIsRzwj928BFwBki8mbsMVBErhCRK2LHDAbWicgaYAZwQezSwJg6CUMnQN/sCiVxMjLcZtVvveX2OF2zBk491ZU6lpT4ji7xbKWoCZUgN/AI40bbVtKaXDt3ulLHqVPdDdNRo2DSpGiVOtpKUROXMFzmB9UJMKyjYLtCSa7mzV0VTGmpS+YPPniw1HHnTt/RNZwldAOEJ8EF0RYhzPP0YepVnsratnWbVVcudZwxw61EjSpL6CZUCS6IlrJhHgWHsVd5TcJwVdcQXbrA44/DqlXQrRuMG+c2sn700WhuYG0J3YQqwQWxgUeYR8FRatwWlqu6RDj1VHjhBdfVsUULGDbMbYn37LOu22NUWEI3oUpwQbRFCPMoOCpbDIbpqi5RRNymGqtXuxH6jh2u9LF/f3jtNd/RxccSugldgkt2W4Qwj4KjssVgmK7qEi0jwy1G2rDBzakXF7vFSj/7GWza5Du6mllCN6FMcHXZyq2uwjwKjkrjtjBd1SVL48Zw9dWuImbSJFi6FE44AUaPptbfF18soZtQJ7hkCPsoOAqN28J2VZdMzZu7zapLS12L3ocechUxkyaFr9TREroJfYJLtESOgpNV5ZHMK5RECONVXbK1aeM2q96wAQYNcq17c3Jg+vTwlDpaQjeRucxPpESMglOpyqOu0u2qrqLjjoOFC11PmO7d4ZproGvXcJQ62tJ/86WobknmQ7ov0U/3f/+KnnvObWD9xhsuwd99t6uWce0KE69BW9AliyV0E2W2L6y7Qhk2bDB5efvIy9tHmzZumqWwMIvCwizmz18Uivn+IBw44EbtN94I//wnnH66S+y5uYn/LOvlYkyCpUOVR22icPM2KBkZcOGFrqvjzJmwbh2cdhr89KewcWNwcdgI3Zh6yMzM4LnnlMzM6o8pK4MBAzIoK9sfXGAmFHbtch0dp0yBvXvh8stdVUy7dg3/2TZCNybB0rHKw8SvWTO49VZX6njllTBnjruZevPNyS11tIRuTD2kc5WHiV+bNm4K5q234Jxz4Pbb4dhjoaAgOZ9nCd2Yeki32n3TMDk58NhjrtSxZ8/kNfyyhG5SThAtXdOxdt80XK9ersxxxIjk/HxL6CalBLnYx6o8TH0lrUbdqlxMsgS9Z2cqLHYJ4z6nJlysysUEzsey+Ki3dE3nVgImMWyEbhLO10g5O7s5+fm7aN+++mM2b4bx45uzZcuOhH1uIqTC1YUJho3QTaB8jZSj3NI16lcXJhxqTegicoyIrBCRDSKyXkTGVXGMiMgMESkRkWIR6ZmccE0U+FoWH+XFPkGds6hv6mxqFs8IvQyYqKrHA32AMSJyQqVj8oAuscco4DcJjdJEiq+RcpQX+wRxzmyOPvXVmtBV9SNVfT32fBewAag8S3kuME+dvwNHiUgCuhaYKPI1Uo7yYp9kn7NU3NTZHKpOc+gi0hnoAays9FZ74F8V/vwBhyZ9RGSUiBSJSNHWrVvrFqmJDF8j5Sgv9kn2ObM5+vQQd0IXkSOBJ4BrVLVye5mqyuQPKZ9R1QdVtbeq9s7Ozq5bpCYyfI6Uo7rYJ9nnzNr9poe4yhZFJAtYAjyrqtOqeP//gBdV9bHYn98G+qnqR9X9TCtbTG22+UHdJfOcWbvf1NGgskUREeBhYENVyTzmaWB4rNqlD7CjpmRuUl9UR8o+JfOcRbkCyMSv1hG6iHwH+CuwFijfAvUGoCOAqj4QS/qzgAHAHuASVa1x+G0jdGOCY1vmpY6aRuiH1faXVfVlqp4jr3iMAmPqF54xJtnGjp1Ibm4BfftWfWO0fI5+1arwVQCZ+NWa0I0x0VdeAVTbHH0YK4BM/GzpvzFpwu5rpD5rzmWMMRFizbmMMSYNWEI3xpgUYQndGGNShCV0Y4xJEZbQjTEmRVhCN8aYFGEJ3RhjUoQldGOMSRGW0I0xJkVYQjfGmBRhCd2YKpSWljJu3Giys5uTmZlBdnZzxo0bbXtumlCzhG5MJYWFheTmdmPbtjnk5+/iueeU/PxdbNs2h9zcbhQWFvoO0ZgqWftcYyooLS1l2LDBTJ685yt9w9u3h5Ej99G37z6GDRvMqlXF1mrWhI6N0I2pYMaMqeTlVb0JBMCJJ7rNlGfOvC/YwIyJgyV0YypYsGA+eXnVb9MGLqEvWPC7gCIyJn6W0I2pYPv23bRtW/Mxbdq444wJG0voxlTQqtWRfPxxzcd88ok7zpiwsYRuTAVDhgyjsDCrxmMKC7MYMuSigCIyJn6W0I2pYOzYiRQWZrF+fdXvr1/vEvrVV48PNjBj4lBrQheRR0Rki4isq+b9fiKyQ0TejD0mJT5MY4KRk5PD/PmLmDSpKXPmZLF5M5SVwebNMGdOFpMmNWX+/EVWsmhCKZ4R+lxgQC3H/FVVT4k9Jjc8LGP8ycvLY9WqYlq3HsX48c0ZMCCD8eOb07r1KFatKiYvL893iMZUSVS19oNEOgNLVPWkKt7rB1yrqoPq8sG9e/fWoqKiuvwVY4xJeyKyWlV7V/VeoubQ+4rIGhEpFJFqlmSAiIwSkSIRKdq6dWuCPtoYYwwkJqG/DnRS1e7ATODJ6g5U1QdVtbeq9s7Ozk7ARxtjjCnX4ISuqjtVdXfs+VIgS0RaNzgyY4wxddLg5lwi0hb4RFVVRHJxXxLbavt7q1ev/reIvFfLYa2Bfzc0xiQIa1xgsdVXWGMLa1xgsdVXQ2PrVN0btSZ0EXkM6Ae0FpEPgFuALABVfQAYDFwpImXAf4ALNI47rapa65yLiBRVN/nvU1jjAoutvsIaW1jjAoutvpIZW60JXVUvrOX9WcCshEVkjDGmXmylqDHGpIiwJ/QHfQdQjbDGBRZbfYU1trDGBRZbfSUttrgWFhljjAm/sI/QjTHGxMkSujHGpAjvCV1EBojI2yJSIiLXV/F+YxFZGHt/ZayvTFhiu1hEtlboNDkyoLhq64ApIjIjFnexiPQMIq44Y/PSnVNEjhGRFSKyQUTWi8i4Ko7xct7ijM3XeWsiIqtirT3Wi8htVRzj5Xc0zti8/I7GPjtTRN4QkSVVvJecc6aq3h5AJlAKHAs0AtYAJ1Q6ZjTwQOz5BcDCEMV2MTDLw3n7HtATWFfN+wOBQkCAPsDKEMXWD9foLehz1g7oGXveDNhYxX9PL+ctzth8nTcBjow9zwJWAn0qHePrdzSe2Lz8jsY+ewKwoKr/bsk6Z75H6LlAiaq+o6pfAL8Hzq10zLlAQez5IuBMEZGQxOaFqv4F2F7DIecC89T5O3CUiLQLSWxeqOpHqvp67PkuYAPQvtJhXs5bnLF5ETsX5RuoZsUelSspvPyOxhmbFyLSATgbmFPNIUk5Z74TenvgXxX+/AGH/o/85TGqWgbsAI4OSWwAP4ldni8SkWMCiCse8cbuS1zdOZMldnnbAzeiq8j7eashNvB03mJTB28CW4DnVbXa8xbw72g8sYGf39F84H+BA9W8n5Rz5juhV/WNVPkbNp5jkiGez30G6Kyq3YAXOPiN65uvcxaPuLtzJoOIHAk8AVyjqjsrv13FXwnsvNUSm7fzpqr7VfUUoAOQKyKV90Xwdt7iiC3w31ERGQRsUdXVNR1WxWsNPme+E/oHQMVvzA7Ah9UdIyKHAS0I5pK+1thUdZuq/jf2x4eAXgHEFY94zqsX6rE7p4hk4RLmo6q6uIpDvJ232mLzed4qxPAZ8CKH7mDm63e01tg8/Y5+GzhHRN7FTdWeISLzKx2TlHPmO6G/BnQRkW+ISCPczYGnKx3zNDAi9nwwsFxjdxJ8x1ZpfvUc3NxnGDwNDI9VbfQBdqjqR76DAteds3yuUOrQnTMBnyvAw8AGVZ1WzWFezls8sXk8b9kiclTs+eFAf+CtSod5+R2NJzYfv6Oq+ktV7aCqnXF5Y7mqDqt0WFLOWYPb5zaEqpaJyFXAs7iqkkdUdb2ITAaKVPVp3P/ovxOREtw32AUhim2siJwDlMViuziI2KT2DphLcRUbJcAe4JIg4ooztnp150yAbwMXAWtjc64ANwAdK8Tm67zFE5uv89YOKBCRTNyXyOOquiQMv6Nxxubld7QqQZwzW/pvjDEpwveUizHGmASxhG6MMSnCEroxxqQIS+jGGJMiLKEbY0yKsIRujDEpwhK6McakiP8P3MIDqz3TzfAAAAAASUVORK5CYII=\n",
+ "text/plain": [
+ "
"
+ ]
+ },
+ "metadata": {
+ "needs_background": "light"
+ },
+ "output_type": "display_data"
+ }
+ ],
"source": [
"# Try different SVM Parameters here\n",
"C, sigma = dataset3Params(X, y, Xval, yval)\n",
@@ -451,9 +584,29 @@
},
{
"cell_type": "code",
- "execution_count": null,
+ "execution_count": 12,
"metadata": {},
- "outputs": [],
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "\n",
+ "Submitting Solutions | Programming Exercise support-vector-machines\n",
+ "\n",
+ "Use token from last successful submission (tanmayss2109@gmail.com)? (Y/n): y\n",
+ " Part Name | Score | Feedback\n",
+ " --------- | ----- | --------\n",
+ " Gaussian Kernel | 25 / 25 | Nice work!\n",
+ " Parameters (C, sigma) for Dataset 3 | 25 / 25 | Nice work!\n",
+ " Email Processing | 0 / 25 | \n",
+ " Email Feature Extraction | 0 / 25 | \n",
+ " --------------------------------\n",
+ " | 50 / 100 | \n",
+ "\n"
+ ]
+ }
+ ],
"source": [
"grader[2] = lambda : (C, sigma)\n",
"grader.grade()"
@@ -555,7 +708,7 @@
},
{
"cell_type": "code",
- "execution_count": null,
+ "execution_count": 13,
"metadata": {},
"outputs": [],
"source": [
@@ -655,7 +808,8 @@
" # Look up the word in the dictionary and add to word_indices if found\n",
" # ====================== YOUR CODE HERE ======================\n",
"\n",
- " \n",
+ " if word in vocabList:\n",
+ " word_indices.append(vocabList.index(word))\n",
"\n",
" # =============================================================\n",
"\n",
@@ -676,9 +830,24 @@
},
{
"cell_type": "code",
- "execution_count": null,
+ "execution_count": 14,
"metadata": {},
- "outputs": [],
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "----------------\n",
+ "Processed email:\n",
+ "----------------\n",
+ "anyon know how much it cost to host a web portal well it depend on how mani visitor your expect thi can be anywher from less than number buck a month to a coupl of dollar number you should checkout httpaddr or perhap amazon ec number if your run someth big to unsubscrib yourself from thi mail list send an email to emailaddr\n",
+ "-------------\n",
+ "Word Indices:\n",
+ "-------------\n",
+ "[85, 915, 793, 1076, 882, 369, 1698, 789, 1821, 1830, 882, 430, 1170, 793, 1001, 1894, 591, 1675, 237, 161, 88, 687, 944, 1662, 1119, 1061, 1698, 374, 1161, 476, 1119, 1892, 1509, 798, 1181, 1236, 511, 1119, 809, 1894, 1439, 1546, 180, 1698, 1757, 1895, 687, 1675, 991, 960, 1476, 70, 529, 1698, 530]\n"
+ ]
+ }
+ ],
"source": [
"# To use an SVM to classify emails into Spam v.s. Non-Spam, you first need\n",
"# to convert each email into a vector of features. In this part, you will\n",
@@ -708,9 +877,29 @@
},
{
"cell_type": "code",
- "execution_count": null,
+ "execution_count": 15,
"metadata": {},
- "outputs": [],
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "\n",
+ "Submitting Solutions | Programming Exercise support-vector-machines\n",
+ "\n",
+ "Use token from last successful submission (tanmayss2109@gmail.com)? (Y/n): y\n",
+ " Part Name | Score | Feedback\n",
+ " --------- | ----- | --------\n",
+ " Gaussian Kernel | 25 / 25 | Nice work!\n",
+ " Parameters (C, sigma) for Dataset 3 | 25 / 25 | Nice work!\n",
+ " Email Processing | 25 / 25 | Nice work!\n",
+ " Email Feature Extraction | 0 / 25 | \n",
+ " --------------------------------\n",
+ " | 75 / 100 | \n",
+ "\n"
+ ]
+ }
+ ],
"source": [
"grader[3] = processEmail\n",
"grader.grade()"
@@ -738,7 +927,7 @@
},
{
"cell_type": "code",
- "execution_count": null,
+ "execution_count": 16,
"metadata": {},
"outputs": [],
"source": [
@@ -798,7 +987,8 @@
" x = np.zeros(n)\n",
"\n",
" # ===================== YOUR CODE HERE ======================\n",
- "\n",
+ " for i in word_indices:\n",
+ " x[i] =1\n",
" \n",
" \n",
" # ===========================================================\n",
@@ -815,9 +1005,23 @@
},
{
"cell_type": "code",
- "execution_count": null,
+ "execution_count": 17,
"metadata": {},
- "outputs": [],
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "----------------\n",
+ "Processed email:\n",
+ "----------------\n",
+ "anyon know how much it cost to host a web portal well it depend on how mani visitor your expect thi can be anywher from less than number buck a month to a coupl of dollar number you should checkout httpaddr or perhap amazon ec number if your run someth big to unsubscrib yourself from thi mail list send an email to emailaddr\n",
+ "\n",
+ "Length of feature vector: 1899\n",
+ "Number of non-zero entries: 45\n"
+ ]
+ }
+ ],
"source": [
"# Extract Features\n",
"with open(os.path.join('Data', 'emailSample1.txt')) as fid:\n",
@@ -842,7 +1046,17 @@
"cell_type": "code",
"execution_count": null,
"metadata": {},
- "outputs": [],
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "\n",
+ "Submitting Solutions | Programming Exercise support-vector-machines\n",
+ "\n"
+ ]
+ }
+ ],
"source": [
"grader[4] = emailFeatures\n",
"grader.grade()"
@@ -1021,7 +1235,7 @@
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
- "version": "3.6.4"
+ "version": "3.7.6"
}
},
"nbformat": 4,
diff --git a/Phase 3 - 2020 (Summer)/Week 6(May 3 -May 9)/Exercise6/token.pkl b/Phase 3 - 2020 (Summer)/Week 6(May 3 -May 9)/Exercise6/token.pkl
new file mode 100644
index 000000000..7724349de
Binary files /dev/null and b/Phase 3 - 2020 (Summer)/Week 6(May 3 -May 9)/Exercise6/token.pkl differ
diff --git a/Phase 3 - 2020 (Summer)/Week 6(May 3 -May 9)/__pycache__/submission.cpython-37.pyc b/Phase 3 - 2020 (Summer)/Week 6(May 3 -May 9)/__pycache__/submission.cpython-37.pyc
new file mode 100644
index 000000000..29aaeff86
Binary files /dev/null and b/Phase 3 - 2020 (Summer)/Week 6(May 3 -May 9)/__pycache__/submission.cpython-37.pyc differ
diff --git a/Phase 3 - 2020 (Summer)/Week 7(May 10-May 16)/Exercise7/.ipynb_checkpoints/exercise7-checkpoint.ipynb b/Phase 3 - 2020 (Summer)/Week 7(May 10-May 16)/Exercise7/.ipynb_checkpoints/exercise7-checkpoint.ipynb
new file mode 100644
index 000000000..f74f4b92f
--- /dev/null
+++ b/Phase 3 - 2020 (Summer)/Week 7(May 10-May 16)/Exercise7/.ipynb_checkpoints/exercise7-checkpoint.ipynb
@@ -0,0 +1,1198 @@
+{
+ "cells": [
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "# Programming Exercise 7:\n",
+ "# K-means Clustering and Principal Component Analysis\n",
+ "\n",
+ "## Introduction\n",
+ "\n",
+ "In this exercise, you will implement the K-means clustering algorithm and apply it to compress an image. In the second part, you will use principal component analysis to find a low-dimensional representation of face images. Before starting on the programming exercise, we strongly recommend watching the video lectures and completing the review questions for the associated topics.\n",
+ "\n",
+ "All the information you need for solving this assignment is in this notebook, and all the code you will be implementing will take place within this notebook. The assignment can be promptly submitted to the coursera grader directly from this notebook (code and instructions are included below).\n",
+ "\n",
+ "Before we begin with the exercises, we need to import all libraries required for this programming exercise. Throughout the course, we will be using [`numpy`](http://www.numpy.org/) for all arrays and matrix operations, [`matplotlib`](https://matplotlib.org/) for plotting, and [`scipy`](https://docs.scipy.org/doc/scipy/reference/) for scientific and numerical computation functions and tools. You can find instructions on how to install required libraries in the README file in the [github repository](https://github.com/dibgerge/ml-coursera-python-assignments)."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "# used for manipulating directory paths\n",
+ "import os\n",
+ "\n",
+ "# Scientific and vector computation for python\n",
+ "import numpy as np\n",
+ "\n",
+ "# Import regular expressions to process emails\n",
+ "import re\n",
+ "\n",
+ "# Plotting library\n",
+ "from matplotlib import pyplot\n",
+ "from mpl_toolkits.mplot3d import Axes3D\n",
+ "import matplotlib as mpl\n",
+ "\n",
+ "from IPython.display import HTML, display, clear_output\n",
+ "\n",
+ "try:\n",
+ " pyplot.rcParams[\"animation.html\"] = \"jshtml\"\n",
+ "except ValueError:\n",
+ " pyplot.rcParams[\"animation.html\"] = \"html5\"\n",
+ "\n",
+ "# Optimization module in scipy\n",
+ "from scipy import optimize\n",
+ "\n",
+ "# will be used to load MATLAB mat datafile format\n",
+ "from scipy.io import loadmat\n",
+ "\n",
+ "# library written for this exercise providing additional functions for assignment submission, and others\n",
+ "import utils\n",
+ "\n",
+ "%load_ext autoreload \n",
+ "%autoreload 2\n",
+ "\n",
+ "# define the submission/grader object for this exercise\n",
+ "grader = utils.Grader()\n",
+ "\n",
+ "# tells matplotlib to embed plots within the notebook\n",
+ "%matplotlib inline"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "## Submission and Grading\n",
+ "\n",
+ "\n",
+ "After completing each part of the assignment, be sure to submit your solutions to the grader. The following is a breakdown of how each part of this exercise is scored.\n",
+ "\n",
+ "\n",
+ "| Section | Part | Submitted Function | Points |\n",
+ "| :- |:- |:- | :-: |\n",
+ "| 1 | [Find Closest Centroids](#section1) | [`findClosestCentroids`](#findClosestCentroids) | 30 |\n",
+ "| 2 | [Computed Centroid Means](#section2) | [`computeCentroids`](#computeCentroids) | 30 |\n",
+ "| 3 | [PCA](#section3) | [`pca`](#pca) | 20 |\n",
+ "| 4 | [Project Data](#section4) | [`projectData`](#projectData) | 10 |\n",
+ "| 5 | [Recover Data](#section5) | [`recoverData`](#recoverData) | 10 |\n",
+ "| | Total Points | |100 |\n",
+ "\n",
+ "\n",
+ "You are allowed to submit your solutions multiple times, and we will take only the highest score into consideration.\n",
+ "\n",
+ "
\n",
+ "At the end of each section in this notebook, we have a cell which contains code for submitting the solutions thus far to the grader. Execute the cell to see your score up to the current section. For all your work to be submitted properly, you must execute those cells at least once.\n",
+ "
"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "## 1 K-means Clustering\n",
+ "\n",
+ "In this exercise, you will implement K-means algorithm and use it for image compression. You will first start on an example 2D dataset that will help you gain an intuition of how the K-means algorithm works. After\n",
+ "that, you wil use the K-means algorithm for image compression by reducing the number of colors that occur in an image to only those that are most common in that image.\n",
+ "\n",
+ "### 1.1 Implementing K-means\n",
+ "\n",
+ "The K-means algorithm is a method to automatically cluster similar data examples together. Concretely, you are given a training set $\\{x^{(1)} , \\cdots, x^{(m)}\\}$ (where $x^{(i)} \\in \\mathbb{R}^n$), and want to group the data into a few cohesive “clusters”. The intuition behind K-means is an iterative procedure that starts by guessing the initial centroids, and then refines this guess by repeatedly assigning examples to their closest centroids and then recomputing the centroids based on the assignments.\n",
+ "\n",
+ "The K-means algorithm is as follows:\n",
+ "\n",
+ "```python\n",
+ "centroids = kMeansInitCentroids(X, K)\n",
+ "for i in range(iterations):\n",
+ " # Cluster assignment step: Assign each data point to the\n",
+ " # closest centroid. idx[i] corresponds to cˆ(i), the index\n",
+ " # of the centroid assigned to example i\n",
+ " idx = findClosestCentroids(X, centroids)\n",
+ " \n",
+ " # Move centroid step: Compute means based on centroid\n",
+ " # assignments\n",
+ " centroids = computeMeans(X, idx, K)\n",
+ "```\n",
+ "\n",
+ "The inner-loop of the algorithm repeatedly carries out two steps: (1) Assigning each training example $x^{(i)}$ to its closest centroid, and (2) Recomputing the mean of each centroid using the points assigned to it. The K-means algorithm will always converge to some final set of means for the centroids. Note that the converged solution may not always be ideal and depends on the initial setting of the centroids. Therefore, in practice the K-means algorithm is usually run a few times with different random initializations. One way to choose between these different solutions from different random initializations is to choose the one with the lowest cost function value (distortion). You will implement the two phases of the K-means algorithm separately\n",
+ "in the next sections."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "\n",
+ "#### 1.1.1 Finding closest centroids\n",
+ "\n",
+ "In the “cluster assignment” phase of the K-means algorithm, the algorithm assigns every training example $x^{(i)}$ to its closest centroid, given the current positions of centroids. Specifically, for every example $i$ we set\n",
+ "\n",
+ "$$c^{(i)} := j \\quad \\text{that minimizes} \\quad \\lvert\\rvert x^{(i)} - \\mu_j \\lvert\\rvert^2, $$\n",
+ "\n",
+ "where $c^{(i)}$ is the index of the centroid that is closest to $x^{(i)}$, and $\\mu_j$ is the position (value) of the $j^{th}$ centroid. Note that $c^{(i)}$ corresponds to `idx[i]` in the starter code.\n",
+ "\n",
+ "Your task is to complete the code in the function `findClosestCentroids`. This function takes the data matrix `X` and the locations of all centroids inside `centroids` and should output a one-dimensional array `idx` that holds the index (a value in $\\{1, ..., K\\}$, where $K$ is total number of centroids) of the closest centroid to every training example.\n",
+ "\n",
+ "You can implement this using a loop over every training example and every centroid.\n",
+ ""
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "def findClosestCentroids(X, centroids):\n",
+ " \"\"\"\n",
+ " Computes the centroid memberships for every example.\n",
+ " \n",
+ " Parameters\n",
+ " ----------\n",
+ " X : array_like\n",
+ " The dataset of size (m, n) where each row is a single example. \n",
+ " That is, we have m examples each of n dimensions.\n",
+ " \n",
+ " centroids : array_like\n",
+ " The k-means centroids of size (K, n). K is the number\n",
+ " of clusters, and n is the the data dimension.\n",
+ " \n",
+ " Returns\n",
+ " -------\n",
+ " idx : array_like\n",
+ " A vector of size (m, ) which holds the centroids assignment for each\n",
+ " example (row) in the dataset X.\n",
+ " \n",
+ " Instructions\n",
+ " ------------\n",
+ " Go over every example, find its closest centroid, and store\n",
+ " the index inside `idx` at the appropriate location.\n",
+ " Concretely, idx[i] should contain the index of the centroid\n",
+ " closest to example i. Hence, it should be a value in the \n",
+ " range 0..K-1\n",
+ "\n",
+ " Note\n",
+ " ----\n",
+ " You can use a for-loop over the examples to compute this.\n",
+ " \"\"\"\n",
+ " # Set K\n",
+ " K = centroids.shape[0]\n",
+ "\n",
+ " # You need to return the following variables correctly.\n",
+ " idx = np.zeros(X.shape[0], dtype=int)\n",
+ "\n",
+ " # ====================== YOUR CODE HERE ======================\n",
+ " for i in np.arange(idx.size):\n",
+ " \n",
+ " J = np.sqrt(np.sum(np.square(X[i] - centroids), axis = 1))\n",
+ " \n",
+ " idx[i] = np.argmin(J)\n",
+ " \n",
+ " \n",
+ " # =============================================================\n",
+ " return idx"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Once you have completed the code in `findClosestCentroids`, the following cell will run your code and you should see the output `[0 2 1]` corresponding to the centroid assignments for the first 3 examples."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "# Load an example dataset that we will be using\n",
+ "data = loadmat(os.path.join('Data', 'ex7data2.mat'))\n",
+ "X = data['X']\n",
+ "\n",
+ "# Select an initial set of centroids\n",
+ "K = 3 # 3 Centroids\n",
+ "initial_centroids = np.array([[3, 3], [6, 2], [8, 5]])\n",
+ "\n",
+ "# Find the closest centroids for the examples using the initial_centroids\n",
+ "idx = findClosestCentroids(X, initial_centroids)\n",
+ "\n",
+ "print('Closest centroids for the first 3 examples:')\n",
+ "print(idx[:3])\n",
+ "print('(the closest centroids should be 0, 2, 1 respectively)')"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "*You should now submit your solutions.*"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "grader[1] = findClosestCentroids\n",
+ "grader.grade()"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "\n",
+ "### 1.1.2 Computing centroid means\n",
+ "\n",
+ "Given assignments of every point to a centroid, the second phase of the algorithm recomputes, for each centroid, the mean of the points that were assigned to it. Specifically, for every centroid $k$ we set\n",
+ "\n",
+ "$$ \\mu_k := \\frac{1}{\\left| C_k\\right|} \\sum_{i \\in C_k} x^{(i)}$$\n",
+ "\n",
+ "where $C_k$ is the set of examples that are assigned to centroid $k$. Concretely, if two examples say $x^{(3)}$ and $x^{(5)}$ are assigned to centroid $k = 2$, then you should update $\\mu_2 = \\frac{1}{2} \\left( x^{(3)} + x^{(5)} \\right)$.\n",
+ "\n",
+ "You should now complete the code in the function `computeCentroids`. You can implement this function using a loop over the centroids. You can also use a loop over the examples; but if you can use a vectorized implementation that does not use such a loop, your code may run faster.\n",
+ ""
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "def computeCentroids(X, idx, K):\n",
+ " \"\"\"\n",
+ " Returns the new centroids by computing the means of the data points\n",
+ " assigned to each centroid.\n",
+ " \n",
+ " Parameters\n",
+ " ----------\n",
+ " X : array_like\n",
+ " The datset where each row is a single data point. That is, it \n",
+ " is a matrix of size (m, n) where there are m datapoints each\n",
+ " having n dimensions. \n",
+ " \n",
+ " idx : array_like \n",
+ " A vector (size m) of centroid assignments (i.e. each entry in range [0 ... K-1])\n",
+ " for each example.\n",
+ " \n",
+ " K : int\n",
+ " Number of clusters\n",
+ " \n",
+ " Returns\n",
+ " -------\n",
+ " centroids : array_like\n",
+ " A matrix of size (K, n) where each row is the mean of the data \n",
+ " points assigned to it.\n",
+ " \n",
+ " Instructions\n",
+ " ------------\n",
+ " Go over every centroid and compute mean of all points that\n",
+ " belong to it. Concretely, the row vector centroids[i, :]\n",
+ " should contain the mean of the data points assigned to\n",
+ " cluster i.\n",
+ "\n",
+ " Note:\n",
+ " -----\n",
+ " You can use a for-loop over the centroids to compute this.\n",
+ " \"\"\"\n",
+ " # Useful variables\n",
+ " m, n = X.shape\n",
+ " # You need to return the following variables correctly.\n",
+ " centroids = np.zeros((K, n))\n",
+ "\n",
+ "\n",
+ " # ====================== YOUR CODE HERE ======================\n",
+ "\n",
+ " \n",
+ " \n",
+ " # =============================================================\n",
+ " return centroids"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Once you have completed the code in `computeCentroids`, the following cell will run your code and output the centroids after the first step of K-means."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "# Compute means based on the closest centroids found in the previous part.\n",
+ "centroids = computeCentroids(X, idx, K)\n",
+ "\n",
+ "print('Centroids computed after initial finding of closest centroids:')\n",
+ "print(centroids)\n",
+ "print('\\nThe centroids should be')\n",
+ "print(' [ 2.428301 3.157924 ]')\n",
+ "print(' [ 5.813503 2.633656 ]')\n",
+ "print(' [ 7.119387 3.616684 ]')"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "*You should now submit your solutions.*"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "grader[2] = computeCentroids\n",
+ "grader.grade()"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "### 1.2 K-means on example dataset \n",
+ "\n",
+ "After you have completed the two functions (`findClosestCentroids` and `computeCentroids`), you have all the necessary pieces to run the K-means algorithm. The next cell will run the K-means algorithm on a toy 2D dataset to help you understand how K-means works. Your functions are called from inside the `runKmeans` function (in this assignment's `utils.py` module). We encourage you to take a look at the function to understand how it works. Notice that the code calls the two functions you implemented in a loop.\n",
+ "\n",
+ "When you run the next step, the K-means code will produce an animation that steps you through the progress of the algorithm at each iteration. At the end, your figure should look as the one displayed below.\n",
+ "\n",
+ ""
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "scrolled": false
+ },
+ "outputs": [],
+ "source": [
+ "# Load an example dataset\n",
+ "data = loadmat(os.path.join('Data', 'ex7data2.mat'))\n",
+ "\n",
+ "# Settings for running K-Means\n",
+ "K = 3\n",
+ "max_iters = 10\n",
+ "\n",
+ "# For consistency, here we set centroids to specific values\n",
+ "# but in practice you want to generate them automatically, such as by\n",
+ "# settings them to be random examples (as can be seen in\n",
+ "# kMeansInitCentroids).\n",
+ "initial_centroids = np.array([[3, 3], [6, 2], [8, 5]])\n",
+ "\n",
+ "\n",
+ "# Run K-Means algorithm. The 'true' at the end tells our function to plot\n",
+ "# the progress of K-Means\n",
+ "centroids, idx, anim = utils.runkMeans(X, initial_centroids,\n",
+ " findClosestCentroids, computeCentroids, max_iters, True)\n",
+ "anim"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "### 1.3 Random initialization \n",
+ "\n",
+ "The initial assignments of centroids for the example dataset in the previous cell were designed so that you will see the same figure as that shown in the cell above. In practice, a\n",
+ "good strategy for initializing the centroids is to select random examples from the training set.\n",
+ "\n",
+ "In this part of the exercise, you should complete the function `kMeansInitCentroids` with the following code:\n",
+ "\n",
+ "```python\n",
+ "# Initialize the centroids to be random examples\n",
+ "\n",
+ "# Randomly reorder the indices of examples\n",
+ "randidx = np.random.permutation(X.shape[0])\n",
+ "# Take the first K examples as centroids\n",
+ "centroids = X[randidx[:K], :]\n",
+ "```\n",
+ "\n",
+ "The code above first randomly permutes the indices of the examples (using `permute` within the `numpy.random` module). Then, it selects the first $K$ examples based on the random permutation of the indices. This allows the examples to be selected at random without the risk of selecting the same example twice.\n",
+ "\n",
+ "*You do not need to make any submission for this part of the exercise*\n",
+ ""
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "def kMeansInitCentroids(X, K):\n",
+ " \"\"\"\n",
+ " This function initializes K centroids that are to be used in K-means on the dataset x.\n",
+ " \n",
+ " Parameters\n",
+ " ----------\n",
+ " X : array_like \n",
+ " The dataset of size (m x n).\n",
+ " \n",
+ " K : int\n",
+ " The number of clusters.\n",
+ " \n",
+ " Returns\n",
+ " -------\n",
+ " centroids : array_like\n",
+ " Centroids of the clusters. This is a matrix of size (K x n).\n",
+ " \n",
+ " Instructions\n",
+ " ------------\n",
+ " You should set centroids to randomly chosen examples from the dataset X.\n",
+ " \"\"\"\n",
+ " m, n = X.shape\n",
+ " \n",
+ " # You should return this values correctly\n",
+ " centroids = np.zeros((K, n))\n",
+ "\n",
+ " # ====================== YOUR CODE HERE ======================\n",
+ "\n",
+ "\n",
+ " \n",
+ " # =============================================================\n",
+ " return centroids"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "### 1.4 Image compression with K-means\n",
+ "\n",
+ "In this exercise, you will apply K-means to image compression. We will use the image below as an example (property of Frank Wouters with permission to this class).\n",
+ "\n",
+ "\n",
+ "\n",
+ "In a straightforward 24-bit color representation of an image, each pixel is represented as three 8-bit unsigned integers (ranging from 0 to 255) that specify the red, green and blue intensity values. This encoding is often referred to as the RGB encoding. Our image contains thousands of colors, and in this part of the exercise, you will reduce the number of colors to 16 colors.\n",
+ "\n",
+ "By making this reduction, it is possible to represent (compress) the photo in an efficient way. Specifically, you only need to store the RGB values of the 16 selected colors, and for each pixel in the image you now need to only store the index of the color at that location (where only 4 bits are necessary to represent 16 possibilities).\n",
+ "\n",
+ "In this exercise, you will use the K-means algorithm to select the 16 colors that will be used to represent the compressed image. Concretely, you will treat every pixel in the original image as a data example and use the K-means algorithm to find the 16 colors that best group (cluster) the pixels in the 3-dimensional RGB space. Once you have computed the cluster centroids on the image, you will then use the 16 colors to replace the pixels in the original image.\n",
+ "\n",
+ "#### 1.4.1 K-means on pixels\n",
+ "\n",
+ "In python, images can be read in as follows:\n",
+ "\n",
+ "```python\n",
+ "# Load 128x128 color image (bird_small.png)\n",
+ "img = mpl.image.imread(os.path.join('Data', 'bird_small.png'))\n",
+ "\n",
+ "# We have already imported matplotlib as mpl at the beginning of this notebook.\n",
+ "```\n",
+ "This creates a three-dimensional matrix `A` whose first two indices identify a pixel position and whose last index represents red, green, or blue. For example, A[50, 33, 2] gives the blue intensity of the pixel at row 51 and column 34.\n",
+ "\n",
+ "The code in the following cell first loads the image, and then reshapes it to create an m x 3 matrix of pixel colors (where m = 16384 = 128 x 128), and calls your K-means function on it.\n",
+ "\n",
+ "After finding the top K = 16 colors to represent the image, you can now assign each pixel position to its closest centroid using the `findClosestCentroids` function. This allows you to represent the original image using the centroid assignments of each pixel. Notice that you have significantly reduced the number of bits that are required to describe the image. The original image required 24 bits for each one of the 128 x 128 pixel locations, resulting in total size of 128 x 128 x 24 = 393,216 bits. The new representation requires some overhead storage in form of a dictionary of 16 colors, each of which require 24 bits, but the image itself then only requires 4 bits per pixel location. The final number of bits used is therefore 16 x 24 + 128 x 128 x 4 = 65,920 bits, which corresponds to compressing the original image by about a factor of 6.\n",
+ "\n",
+ "Finally, you can view the effects of the compression by reconstructing the image based only on the centroid assignments. Specifically, you can replace each pixel location with the mean of the centroid assigned to it. The figure below shows the reconstruction we obtained. \n",
+ "\n",
+ "\n",
+ "\n",
+ "Even though the resulting image retains most of the characteristics of the original, we also see some compression artifacts.\n",
+ "\n",
+ "Run the following cell to compute the centroids and the centroid allocation of each pixel in the image."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "# ======= Experiment with these parameters ================\n",
+ "# You should try different values for those parameters\n",
+ "K = 16\n",
+ "max_iters = 10\n",
+ "\n",
+ "# Load an image of a bird\n",
+ "# Change the file name and path to experiment with your own images\n",
+ "A = mpl.image.imread(os.path.join('Data', 'bird_small.png'))\n",
+ "# ==========================================================\n",
+ "\n",
+ "# Divide by 255 so that all values are in the range 0 - 1\n",
+ "A /= 255\n",
+ "\n",
+ "# Reshape the image into an Nx3 matrix where N = number of pixels.\n",
+ "# Each row will contain the Red, Green and Blue pixel values\n",
+ "# This gives us our dataset matrix X that we will use K-Means on.\n",
+ "X = A.reshape(-1, 3)\n",
+ "\n",
+ "# When using K-Means, it is important to randomly initialize centroids\n",
+ "# You should complete the code in kMeansInitCentroids above before proceeding\n",
+ "initial_centroids = kMeansInitCentroids(X, K)\n",
+ "\n",
+ "# Run K-Means\n",
+ "centroids, idx = utils.runkMeans(X, initial_centroids,\n",
+ " findClosestCentroids,\n",
+ " computeCentroids,\n",
+ " max_iters)\n",
+ "\n",
+ "# We can now recover the image from the indices (idx) by mapping each pixel\n",
+ "# (specified by its index in idx) to the centroid value\n",
+ "# Reshape the recovered image into proper dimensions\n",
+ "X_recovered = centroids[idx, :].reshape(A.shape)\n",
+ "\n",
+ "# Display the original image, rescale back by 255\n",
+ "fig, ax = pyplot.subplots(1, 2, figsize=(8, 4))\n",
+ "ax[0].imshow(A*255)\n",
+ "ax[0].set_title('Original')\n",
+ "ax[0].grid(False)\n",
+ "\n",
+ "# Display compressed image, rescale back by 255\n",
+ "ax[1].imshow(X_recovered*255)\n",
+ "ax[1].set_title('Compressed, with %d colors' % K)\n",
+ "ax[1].grid(False)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "*You do not need to make any submissions for this part of the exercise.*"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "### 1.5 Optional (ungraded) exercise: Use your own image\n",
+ "\n",
+ "In this exercise, modify the code we have supplied in the previous cell to run on one of your own images. Note that if your image is very large, then K-means can take a long time to run. Therefore, we recommend that you resize your images to\n",
+ "manageable sizes before running the code. You can also try to vary $K$ to see the effects on the compression."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "## 2 Principal Component Analysis\n",
+ "\n",
+ "In this exercise, you will use principal component analysis (PCA) to perform dimensionality reduction. You will first experiment with an example 2D dataset to get intuition on how PCA works, and then use it on a bigger dataset of 5000 face image dataset.\n",
+ "\n",
+ "### 2.1 Example Dataset\n",
+ "\n",
+ "To help you understand how PCA works, you will first start with a 2D dataset which has one direction of large variation and one of smaller variation. The cell below will plot the training data, also shown in here:\n",
+ "\n",
+ "In this part of the exercise, you will visualize what happens when you use PCA to reduce the data from 2D to 1D. In practice, you might want to reduce data from 256 to 50 dimensions, say; but using lower dimensional data in this example allows us to visualize the algorithms better."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "# Load the dataset into the variable X \n",
+ "data = loadmat(os.path.join('Data', 'ex7data1.mat'))\n",
+ "X = data['X']\n",
+ "\n",
+ "# Visualize the example dataset\n",
+ "pyplot.plot(X[:, 0], X[:, 1], 'bo', ms=10, mec='k', mew=1)\n",
+ "pyplot.axis([0.5, 6.5, 2, 8])\n",
+ "pyplot.gca().set_aspect('equal')\n",
+ "pyplot.grid(False)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "\n",
+ "### 2.2 Implementing PCA\n",
+ "\n",
+ "In this part of the exercise, you will implement PCA. PCA consists of two computational steps: \n",
+ "\n",
+ "1. Compute the covariance matrix of the data.\n",
+ "2. Use SVD (in python we use numpy's implementation `np.linalg.svd`) to compute the eigenvectors $U_1$, $U_2$, $\\dots$, $U_n$. These will correspond to the principal components of variation in the data.\n",
+ "\n",
+ "First, you should compute the covariance matrix of the data, which is given by:\n",
+ "\n",
+ "$$ \\Sigma = \\frac{1}{m} X^T X$$\n",
+ "\n",
+ "where $X$ is the data matrix with examples in rows, and $m$ is the number of examples. Note that $\\Sigma$ is a $n \\times n$ matrix and not the summation operator. \n",
+ "\n",
+ "After computing the covariance matrix, you can run SVD on it to compute the principal components. In python and `numpy` (or `scipy`), you can run SVD with the following command: `U, S, V = np.linalg.svd(Sigma)`, where `U` will contain the principal components and `S` will contain a diagonal matrix. Note that the `scipy` library also has a similar function to compute SVD `scipy.linalg.svd`. The functions in the two libraries use the same C-based library (LAPACK) for the SVD computation, but the `scipy` version provides more options and arguments to control SVD computation. In this exercise, we will stick with the `numpy` implementation of SVD.\n",
+ "\n",
+ "Complete the code in the following cell to implemente PCA.\n",
+ ""
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "def pca(X):\n",
+ " \"\"\"\n",
+ " Run principal component analysis.\n",
+ " \n",
+ " Parameters\n",
+ " ----------\n",
+ " X : array_like\n",
+ " The dataset to be used for computing PCA. It has dimensions (m x n)\n",
+ " where m is the number of examples (observations) and n is \n",
+ " the number of features.\n",
+ " \n",
+ " Returns\n",
+ " -------\n",
+ " U : array_like\n",
+ " The eigenvectors, representing the computed principal components\n",
+ " of X. U has dimensions (n x n) where each column is a single \n",
+ " principal component.\n",
+ " \n",
+ " S : array_like\n",
+ " A vector of size n, contaning the singular values for each\n",
+ " principal component. Note this is the diagonal of the matrix we \n",
+ " mentioned in class.\n",
+ " \n",
+ " Instructions\n",
+ " ------------\n",
+ " You should first compute the covariance matrix. Then, you\n",
+ " should use the \"svd\" function to compute the eigenvectors\n",
+ " and eigenvalues of the covariance matrix. \n",
+ "\n",
+ " Notes\n",
+ " -----\n",
+ " When computing the covariance matrix, remember to divide by m (the\n",
+ " number of examples).\n",
+ " \"\"\"\n",
+ " # Useful values\n",
+ " m, n = X.shape\n",
+ "\n",
+ " # You need to return the following variables correctly.\n",
+ " U = np.zeros(n)\n",
+ " S = np.zeros(n)\n",
+ "\n",
+ " # ====================== YOUR CODE HERE ======================\n",
+ "\n",
+ " \n",
+ " \n",
+ " # ============================================================\n",
+ " return U, S"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Before using PCA, it is important to first normalize the data by subtracting the mean value of each feature from the dataset, and scaling each dimension so that they are in the same range.\n",
+ "\n",
+ "In the next cell, this normalization will be performed for you using the `utils.featureNormalize` function.\n",
+ "After normalizing the data, you can run PCA to compute the principal components. Your task is to complete the code in the function `pca` to compute the principal components of the dataset. \n",
+ "\n",
+ "Once you have completed the function `pca`, the following cell will run PCA on the example dataset and plot the corresponding principal components found similar to the figure below. \n",
+ "\n",
+ "\n",
+ "\n",
+ "\n",
+ "The following cell will also output the top principal component (eigenvector) found, and you should expect to see an output of about `[-0.707 -0.707]`. (It is possible that `numpy` may instead output the negative of this, since $U_1$ and $-U_1$ are equally valid choices for the first principal component.)"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "# Before running PCA, it is important to first normalize X\n",
+ "X_norm, mu, sigma = utils.featureNormalize(X)\n",
+ "\n",
+ "# Run PCA\n",
+ "U, S = pca(X_norm)\n",
+ "\n",
+ "# Draw the eigenvectors centered at mean of data. These lines show the\n",
+ "# directions of maximum variations in the dataset.\n",
+ "fig, ax = pyplot.subplots()\n",
+ "ax.plot(X[:, 0], X[:, 1], 'bo', ms=10, mec='k', mew=0.25)\n",
+ "\n",
+ "for i in range(2):\n",
+ " ax.arrow(mu[0], mu[1], 1.5 * S[i]*U[0, i], 1.5 * S[i]*U[1, i],\n",
+ " head_width=0.25, head_length=0.2, fc='k', ec='k', lw=2, zorder=1000)\n",
+ "\n",
+ "ax.axis([0.5, 6.5, 2, 8])\n",
+ "ax.set_aspect('equal')\n",
+ "ax.grid(False)\n",
+ "\n",
+ "print('Top eigenvector: U[:, 0] = [{:.6f} {:.6f}]'.format(U[0, 0], U[1, 0]))\n",
+ "print(' (you should expect to see [-0.707107 -0.707107])')"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "*You should now submit your solutions.*"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "grader[3] = pca\n",
+ "grader.grade()"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "### 2.3 Dimensionality Reduction with PCA\n",
+ "\n",
+ "After computing the principal components, you can use them to reduce the feature dimension of your dataset by projecting each example onto a lower dimensional space, $x^{(i)} \\rightarrow z^{(i)}$ (e.g., projecting the data from 2D to 1D). In this part of the exercise, you will use the eigenvectors returned by PCA and\n",
+ "project the example dataset into a 1-dimensional space. In practice, if you were using a learning algorithm such as linear regression or perhaps neural networks, you could now use the projected data instead of the original data. By using the projected data, you can train your model faster as there are less dimensions in the input.\n",
+ "\n",
+ "\n",
+ "\n",
+ "#### 2.3.1 Projecting the data onto the principal components\n",
+ "\n",
+ "You should now complete the code in the function `projectData`. Specifically, you are given a dataset `X`, the principal components `U`, and the desired number of dimensions to reduce to `K`. You should project each example in `X` onto the top `K` components in `U`. Note that the top `K` components in `U` are given by\n",
+ "the first `K` columns of `U`, that is `Ureduce = U[:, :K]`.\n",
+ ""
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "def projectData(X, U, K):\n",
+ " \"\"\"\n",
+ " Computes the reduced data representation when projecting only \n",
+ " on to the top K eigenvectors.\n",
+ " \n",
+ " Parameters\n",
+ " ----------\n",
+ " X : array_like\n",
+ " The input dataset of shape (m x n). The dataset is assumed to be \n",
+ " normalized.\n",
+ " \n",
+ " U : array_like\n",
+ " The computed eigenvectors using PCA. This is a matrix of \n",
+ " shape (n x n). Each column in the matrix represents a single\n",
+ " eigenvector (or a single principal component).\n",
+ " \n",
+ " K : int\n",
+ " Number of dimensions to project onto. Must be smaller than n.\n",
+ " \n",
+ " Returns\n",
+ " -------\n",
+ " Z : array_like\n",
+ " The projects of the dataset onto the top K eigenvectors. \n",
+ " This will be a matrix of shape (m x k).\n",
+ " \n",
+ " Instructions\n",
+ " ------------\n",
+ " Compute the projection of the data using only the top K \n",
+ " eigenvectors in U (first K columns). \n",
+ " For the i-th example X[i,:], the projection on to the k-th \n",
+ " eigenvector is given as follows:\n",
+ " \n",
+ " x = X[i, :]\n",
+ " projection_k = np.dot(x, U[:, k])\n",
+ "\n",
+ " \"\"\"\n",
+ " # You need to return the following variables correctly.\n",
+ " Z = np.zeros((X.shape[0], K))\n",
+ "\n",
+ " # ====================== YOUR CODE HERE ======================\n",
+ "\n",
+ "\n",
+ " \n",
+ " # =============================================================\n",
+ " return Z"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Once you have completed the code in `projectData`, the following cell will project the first example onto the first dimension and you should see a value of about 1.481 (or possibly -1.481, if you got $-U_1$ instead of $U_1$)."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "# Project the data onto K = 1 dimension\n",
+ "K = 1\n",
+ "Z = projectData(X_norm, U, K)\n",
+ "print('Projection of the first example: {:.6f}'.format(Z[0, 0]))\n",
+ "print('(this value should be about : 1.481274)')"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "*You should now submit your solutions.*"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "grader[4] = projectData\n",
+ "grader.grade()"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "\n",
+ "#### 2.3.2 Reconstructing an approximation of the data\n",
+ "\n",
+ "After projecting the data onto the lower dimensional space, you can approximately recover the data by projecting them back onto the original high dimensional space. Your task is to complete the function `recoverData` to project each example in `Z` back onto the original space and return the recovered approximation in `Xrec`.\n",
+ ""
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "def recoverData(Z, U, K):\n",
+ " \"\"\"\n",
+ " Recovers an approximation of the original data when using the \n",
+ " projected data.\n",
+ " \n",
+ " Parameters\n",
+ " ----------\n",
+ " Z : array_like\n",
+ " The reduced data after applying PCA. This is a matrix\n",
+ " of shape (m x K).\n",
+ " \n",
+ " U : array_like\n",
+ " The eigenvectors (principal components) computed by PCA.\n",
+ " This is a matrix of shape (n x n) where each column represents\n",
+ " a single eigenvector.\n",
+ " \n",
+ " K : int\n",
+ " The number of principal components retained\n",
+ " (should be less than n).\n",
+ " \n",
+ " Returns\n",
+ " -------\n",
+ " X_rec : array_like\n",
+ " The recovered data after transformation back to the original \n",
+ " dataset space. This is a matrix of shape (m x n), where m is \n",
+ " the number of examples and n is the dimensions (number of\n",
+ " features) of original datatset.\n",
+ " \n",
+ " Instructions\n",
+ " ------------\n",
+ " Compute the approximation of the data by projecting back\n",
+ " onto the original space using the top K eigenvectors in U.\n",
+ " For the i-th example Z[i,:], the (approximate)\n",
+ " recovered data for dimension j is given as follows:\n",
+ "\n",
+ " v = Z[i, :]\n",
+ " recovered_j = np.dot(v, U[j, :K])\n",
+ "\n",
+ " Notice that U[j, :K] is a vector of size K.\n",
+ " \"\"\"\n",
+ " # You need to return the following variables correctly.\n",
+ " X_rec = np.zeros((Z.shape[0], U.shape[0]))\n",
+ "\n",
+ " # ====================== YOUR CODE HERE ======================\n",
+ "\n",
+ " \n",
+ "\n",
+ " # =============================================================\n",
+ " return X_rec"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Once you have completed the code in `recoverData`, the following cell will recover an approximation of the first example and you should see a value of about `[-1.047 -1.047]`. The code will then plot the data in this reduced dimension space. This will show you what the data looks like when using only the corresponding eigenvectors to reconstruct it. An example of what you should get for PCA projection is shown in this figure: \n",
+ "\n",
+ "\n",
+ "\n",
+ "In the figure above, the original data points are indicated with the blue circles, while the projected data points are indicated with the red circles. The projection effectively only retains the information in the direction given by $U_1$. The dotted lines show the distance from the data points in original space to the projected space. Those dotted lines represent the error measure due to PCA projection."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "X_rec = recoverData(Z, U, K)\n",
+ "print('Approximation of the first example: [{:.6f} {:.6f}]'.format(X_rec[0, 0], X_rec[0, 1]))\n",
+ "print(' (this value should be about [-1.047419 -1.047419])')\n",
+ "\n",
+ "# Plot the normalized dataset (returned from featureNormalize)\n",
+ "fig, ax = pyplot.subplots(figsize=(5, 5))\n",
+ "ax.plot(X_norm[:, 0], X_norm[:, 1], 'bo', ms=8, mec='b', mew=0.5)\n",
+ "ax.set_aspect('equal')\n",
+ "ax.grid(False)\n",
+ "pyplot.axis([-3, 2.75, -3, 2.75])\n",
+ "\n",
+ "# Draw lines connecting the projected points to the original points\n",
+ "ax.plot(X_rec[:, 0], X_rec[:, 1], 'ro', mec='r', mew=2, mfc='none')\n",
+ "for xnorm, xrec in zip(X_norm, X_rec):\n",
+ " ax.plot([xnorm[0], xrec[0]], [xnorm[1], xrec[1]], '--k', lw=1)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "*You should now submit your solutions.*"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "grader[5] = recoverData\n",
+ "grader.grade()"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "### 2.4 Face Image Dataset\n",
+ "\n",
+ "In this part of the exercise, you will run PCA on face images to see how it can be used in practice for dimension reduction. The dataset `ex7faces.mat` contains a dataset `X` of face images, each $32 \\times 32$ in grayscale. This dataset was based on a [cropped version](http://conradsanderson.id.au/lfwcrop/) of the [labeled faces in the wild](http://vis-www.cs.umass.edu/lfw/) dataset. Each row of `X` corresponds to one face image (a row vector of length 1024). \n",
+ "\n",
+ "The next cell will load and visualize the first 100 of these face images similar to what is shown in this figure:\n",
+ "\n",
+ ""
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "# Load Face dataset\n",
+ "data = loadmat(os.path.join('Data', 'ex7faces.mat'))\n",
+ "X = data['X']\n",
+ "\n",
+ "# Display the first 100 faces in the dataset\n",
+ "utils.displayData(X[:100, :], figsize=(8, 8))"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "#### 2.4.1 PCA on Faces\n",
+ "\n",
+ "To run PCA on the face dataset, we first normalize the dataset by subtracting the mean of each feature from the data matrix `X`. After running PCA, you will obtain the principal components of the dataset. Notice that each principal component in `U` (each column) is a vector of length $n$ (where for the face dataset, $n = 1024$). It turns out that we can visualize these principal components by reshaping each of them into a $32 \\times 32$ matrix that corresponds to the pixels in the original dataset. \n",
+ "\n",
+ "The following cell will first normalize the dataset for you and then run your PCA code. Then, the first 36 principal components (conveniently called eigenfaces) that describe the largest variations are displayed. If you want, you can also change the code to display more principal components to see how they capture more and more details."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "# normalize X by subtracting the mean value from each feature\n",
+ "X_norm, mu, sigma = utils.featureNormalize(X)\n",
+ "\n",
+ "# Run PCA\n",
+ "U, S = pca(X_norm)\n",
+ "\n",
+ "# Visualize the top 36 eigenvectors found\n",
+ "utils.displayData(U[:, :36].T, figsize=(8, 8))"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "#### 2.4.2 Dimensionality Reduction\n",
+ "\n",
+ "Now that you have computed the principal components for the face dataset, you can use it to reduce the dimension of the face dataset. This allows you to use your learning algorithm with a smaller input size (e.g., 100 dimensions) instead of the original 1024 dimensions. This can help speed up your learning algorithm.\n",
+ "\n",
+ "The next cell will project the face dataset onto only the first 100 principal components. Concretely, each face image is now described by a vector $z^{(i)} \\in \\mathbb{R}^{100}$. To understand what is lost in the dimension reduction, you can recover the data using only the projected dataset."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "# Project images to the eigen space using the top k eigenvectors \n",
+ "# If you are applying a machine learning algorithm \n",
+ "K = 100\n",
+ "Z = projectData(X_norm, U, K)\n",
+ "\n",
+ "print('The projected data Z has a shape of: ', Z.shape)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "In the next cell, an approximate recovery of the data is performed and the original and projected face images\n",
+ "are displayed similar to what is shown here:\n",
+ "\n",
+ "
\n",
+ "
\n",
+ "
\n",
+ "
\n",
+ "
\n",
+ "
\n",
+ "\n",
+ "From the reconstruction, you can observe that the general structure and appearance of the face are kept while the fine details are lost. This is a remarkable reduction (more than 10x) in the dataset size that can help speed up your learning algorithm significantly. For example, if you were training a neural network to perform person recognition (given a face image, predict the identity of the person), you can use the dimension reduced input of only a 100 dimensions instead of the original pixels."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "# Project images to the eigen space using the top K eigen vectors and \n",
+ "# visualize only using those K dimensions\n",
+ "# Compare to the original input, which is also displayed\n",
+ "K = 100\n",
+ "X_rec = recoverData(Z, U, K)\n",
+ "\n",
+ "# Display normalized data\n",
+ "utils.displayData(X_norm[:100, :], figsize=(6, 6))\n",
+ "pyplot.gcf().suptitle('Original faces')\n",
+ "\n",
+ "# Display reconstructed data from only k eigenfaces\n",
+ "utils.displayData(X_rec[:100, :], figsize=(6, 6))\n",
+ "pyplot.gcf().suptitle('Recovered faces')\n",
+ "pass"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "### 2.5 Optional (ungraded) exercise: PCA for visualization\n",
+ "\n",
+ "In the earlier K-means image compression exercise, you used the K-means algorithm in the 3-dimensional RGB space. We reduced each pixel of the RGB image to be represented by 16 clusters. In the next cell, we have provided code to visualize the final pixel assignments in this 3D space. Each data point is colored according to the cluster it has been assigned to. You can drag your mouse on the figure to rotate and inspect this data in 3 dimensions."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "# this allows to have interactive plot to rotate the 3-D plot\n",
+ "# The double identical statement is on purpose\n",
+ "# see: https://stackoverflow.com/questions/43545050/using-matplotlib-notebook-after-matplotlib-inline-in-jupyter-notebook-doesnt\n",
+ "%matplotlib notebook\n",
+ "%matplotlib notebook\n",
+ "from matplotlib import pyplot\n",
+ "\n",
+ "\n",
+ "A = mpl.image.imread(os.path.join('Data', 'bird_small.png'))\n",
+ "A /= 255\n",
+ "X = A.reshape(-1, 3)\n",
+ "\n",
+ "# perform the K-means clustering again here\n",
+ "K = 16\n",
+ "max_iters = 10\n",
+ "initial_centroids = kMeansInitCentroids(X, K)\n",
+ "centroids, idx = utils.runkMeans(X, initial_centroids,\n",
+ " findClosestCentroids,\n",
+ " computeCentroids, max_iters)\n",
+ "\n",
+ "# Sample 1000 random indexes (since working with all the data is\n",
+ "# too expensive. If you have a fast computer, you may increase this.\n",
+ "sel = np.random.choice(X.shape[0], size=1000)\n",
+ "\n",
+ "fig = pyplot.figure(figsize=(6, 6))\n",
+ "ax = fig.add_subplot(111, projection='3d')\n",
+ "\n",
+ "ax.scatter(X[sel, 0], X[sel, 1], X[sel, 2], cmap='rainbow', c=idx[sel], s=8**2)\n",
+ "ax.set_title('Pixel dataset plotted in 3D.\\nColor shows centroid memberships')\n",
+ "pass"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "It turns out that visualizing datasets in 3 dimensions or greater can be cumbersome. Therefore, it is often desirable to only display the data in 2D even at the cost of losing some information. In practice, PCA is often used to reduce the dimensionality of data for visualization purposes. \n",
+ "\n",
+ "In the next cell,we will apply your implementation of PCA to the 3-dimensional data to reduce it to 2 dimensions and visualize the result in a 2D scatter plot. The PCA projection can be thought of as a rotation that selects the view that maximizes the spread of the data, which often corresponds to the “best” view."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "# Subtract the mean to use PCA\n",
+ "X_norm, mu, sigma = utils.featureNormalize(X)\n",
+ "\n",
+ "# PCA and project the data to 2D\n",
+ "U, S = pca(X_norm)\n",
+ "Z = projectData(X_norm, U, 2)\n",
+ "\n",
+ "# Reset matplotlib to non-interactive\n",
+ "%matplotlib inline\n",
+ "\n",
+ "fig = pyplot.figure(figsize=(6, 6))\n",
+ "ax = fig.add_subplot(111)\n",
+ "\n",
+ "ax.scatter(Z[sel, 0], Z[sel, 1], cmap='rainbow', c=idx[sel], s=64)\n",
+ "ax.set_title('Pixel dataset plotted in 2D, using PCA for dimensionality reduction')\n",
+ "ax.grid(False)\n",
+ "pass"
+ ]
+ }
+ ],
+ "metadata": {
+ "kernelspec": {
+ "display_name": "Python 3",
+ "language": "python",
+ "name": "python3"
+ },
+ "language_info": {
+ "codemirror_mode": {
+ "name": "ipython",
+ "version": 3
+ },
+ "file_extension": ".py",
+ "mimetype": "text/x-python",
+ "name": "python",
+ "nbconvert_exporter": "python",
+ "pygments_lexer": "ipython3",
+ "version": "3.7.6"
+ }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 2
+}
diff --git a/Phase 3 - 2020 (Summer)/Week 7(May 10-May 16)/Exercise7/__pycache__/utils.cpython-37.pyc b/Phase 3 - 2020 (Summer)/Week 7(May 10-May 16)/Exercise7/__pycache__/utils.cpython-37.pyc
new file mode 100644
index 000000000..c64d6bef9
Binary files /dev/null and b/Phase 3 - 2020 (Summer)/Week 7(May 10-May 16)/Exercise7/__pycache__/utils.cpython-37.pyc differ
diff --git a/Phase 3 - 2020 (Summer)/Week 7(May 10-May 16)/Exercise7/exercise7.ipynb b/Phase 3 - 2020 (Summer)/Week 7(May 10-May 16)/Exercise7/exercise7.ipynb
index 066811220..9ae1c72a5 100644
--- a/Phase 3 - 2020 (Summer)/Week 7(May 10-May 16)/Exercise7/exercise7.ipynb
+++ b/Phase 3 - 2020 (Summer)/Week 7(May 10-May 16)/Exercise7/exercise7.ipynb
@@ -18,7 +18,7 @@
},
{
"cell_type": "code",
- "execution_count": null,
+ "execution_count": 6,
"metadata": {},
"outputs": [],
"source": [
@@ -52,7 +52,7 @@
"# library written for this exercise providing additional functions for assignment submission, and others\n",
"import utils\n",
"\n",
- "%load_ext autoreload \n",
+ "%load_ext autoreload\n",
"%autoreload 2\n",
"\n",
"# define the submission/grader object for this exercise\n",
@@ -142,7 +142,7 @@
},
{
"cell_type": "code",
- "execution_count": null,
+ "execution_count": 7,
"metadata": {},
"outputs": [],
"source": [
@@ -185,7 +185,11 @@
" idx = np.zeros(X.shape[0], dtype=int)\n",
"\n",
" # ====================== YOUR CODE HERE ======================\n",
- "\n",
+ " for i in np.arange(idx.size):\n",
+ " \n",
+ " J = np.sqrt(np.sum(np.square(X[i] - centroids), axis = 1))\n",
+ " \n",
+ " idx[i] = np.argmin(J)\n",
" \n",
" \n",
" # =============================================================\n",
@@ -201,9 +205,19 @@
},
{
"cell_type": "code",
- "execution_count": null,
+ "execution_count": 8,
"metadata": {},
- "outputs": [],
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "Closest centroids for the first 3 examples:\n",
+ "[0 2 1]\n",
+ "(the closest centroids should be 0, 2, 1 respectively)\n"
+ ]
+ }
+ ],
"source": [
"# Load an example dataset that we will be using\n",
"data = loadmat(os.path.join('Data', 'ex7data2.mat'))\n",
@@ -230,9 +244,31 @@
},
{
"cell_type": "code",
- "execution_count": null,
+ "execution_count": 9,
"metadata": {},
- "outputs": [],
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "\n",
+ "Submitting Solutions | Programming Exercise k-means-clustering-and-pca\n",
+ "\n",
+ "Login (email address): tanmayss2109@gmail.com\n",
+ "Token: XoRHvFI3nIPcxISp\n",
+ " Part Name | Score | Feedback\n",
+ " --------- | ----- | --------\n",
+ " Find Closest Centroids (k-Means) | 30 / 30 | Nice work!\n",
+ " Compute Centroid Means (k-Means) | 0 / 30 | \n",
+ " PCA | 0 / 20 | \n",
+ " Project Data (PCA) | 0 / 10 | \n",
+ " Recover Data (PCA) | 0 / 10 | \n",
+ " --------------------------------\n",
+ " | 30 / 100 | \n",
+ "\n"
+ ]
+ }
+ ],
"source": [
"grader[1] = findClosestCentroids\n",
"grader.grade()"
@@ -257,7 +293,7 @@
},
{
"cell_type": "code",
- "execution_count": null,
+ "execution_count": 10,
"metadata": {},
"outputs": [],
"source": [
@@ -305,7 +341,8 @@
"\n",
" # ====================== YOUR CODE HERE ======================\n",
"\n",
- " \n",
+ " for i in np.arange(K):\n",
+ " centroids[i] = np.mean(X[idx == i], axis = 0)\n",
" \n",
" # =============================================================\n",
" return centroids"
@@ -320,9 +357,25 @@
},
{
"cell_type": "code",
- "execution_count": null,
+ "execution_count": 11,
"metadata": {},
- "outputs": [],
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "Centroids computed after initial finding of closest centroids:\n",
+ "[[2.42830111 3.15792418]\n",
+ " [5.81350331 2.63365645]\n",
+ " [7.11938687 3.6166844 ]]\n",
+ "\n",
+ "The centroids should be\n",
+ " [ 2.428301 3.157924 ]\n",
+ " [ 5.813503 2.633656 ]\n",
+ " [ 7.119387 3.616684 ]\n"
+ ]
+ }
+ ],
"source": [
"# Compute means based on the closest centroids found in the previous part.\n",
"centroids = computeCentroids(X, idx, K)\n",
@@ -344,9 +397,30 @@
},
{
"cell_type": "code",
- "execution_count": null,
+ "execution_count": 12,
"metadata": {},
- "outputs": [],
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "\n",
+ "Submitting Solutions | Programming Exercise k-means-clustering-and-pca\n",
+ "\n",
+ "Use token from last successful submission (tanmayss2109@gmail.com)? (Y/n): y\n",
+ " Part Name | Score | Feedback\n",
+ " --------- | ----- | --------\n",
+ " Find Closest Centroids (k-Means) | 30 / 30 | Nice work!\n",
+ " Compute Centroid Means (k-Means) | 30 / 30 | Nice work!\n",
+ " PCA | 0 / 20 | \n",
+ " Project Data (PCA) | 0 / 10 | \n",
+ " Recover Data (PCA) | 0 / 10 | \n",
+ " --------------------------------\n",
+ " | 60 / 100 | \n",
+ "\n"
+ ]
+ }
+ ],
"source": [
"grader[2] = computeCentroids\n",
"grader.grade()"
@@ -367,11 +441,4390 @@
},
{
"cell_type": "code",
- "execution_count": null,
+ "execution_count": 13,
"metadata": {
"scrolled": false
},
- "outputs": [],
+ "outputs": [
+ {
+ "data": {
+ "text/html": [
+ "\n",
+ "\n",
+ "\n",
+ "\n",
+ "\n",
+ "\n",
+ "
"
+ ]
+ },
+ "metadata": {
+ "needs_background": "light"
+ },
+ "output_type": "display_data"
+ }
+ ],
"source": [
"# Before running PCA, it is important to first normalize X\n",
"X_norm, mu, sigma = utils.featureNormalize(X)\n",
@@ -735,9 +5239,30 @@
},
{
"cell_type": "code",
- "execution_count": null,
+ "execution_count": 19,
"metadata": {},
- "outputs": [],
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "\n",
+ "Submitting Solutions | Programming Exercise k-means-clustering-and-pca\n",
+ "\n",
+ "Use token from last successful submission (tanmayss2109@gmail.com)? (Y/n): y\n",
+ " Part Name | Score | Feedback\n",
+ " --------- | ----- | --------\n",
+ " Find Closest Centroids (k-Means) | 30 / 30 | Nice work!\n",
+ " Compute Centroid Means (k-Means) | 30 / 30 | Nice work!\n",
+ " PCA | 20 / 20 | Nice work!\n",
+ " Project Data (PCA) | 0 / 10 | \n",
+ " Recover Data (PCA) | 0 / 10 | \n",
+ " --------------------------------\n",
+ " | 80 / 100 | \n",
+ "\n"
+ ]
+ }
+ ],
"source": [
"grader[3] = pca\n",
"grader.grade()"
@@ -763,7 +5288,7 @@
},
{
"cell_type": "code",
- "execution_count": null,
+ "execution_count": 20,
"metadata": {},
"outputs": [],
"source": [
@@ -808,7 +5333,7 @@
"\n",
" # ====================== YOUR CODE HERE ======================\n",
"\n",
- "\n",
+ " Z = np.dot(X, U[:, :K])\n",
" \n",
" # =============================================================\n",
" return Z"
@@ -823,9 +5348,18 @@
},
{
"cell_type": "code",
- "execution_count": null,
+ "execution_count": 21,
"metadata": {},
- "outputs": [],
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "Projection of the first example: 1.481274\n",
+ "(this value should be about : 1.481274)\n"
+ ]
+ }
+ ],
"source": [
"# Project the data onto K = 1 dimension\n",
"K = 1\n",
@@ -845,7 +5379,18 @@
"cell_type": "code",
"execution_count": null,
"metadata": {},
- "outputs": [],
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "\n",
+ "Submitting Solutions | Programming Exercise k-means-clustering-and-pca\n",
+ "\n",
+ "Use token from last successful submission (tanmayss2109@gmail.com)? (Y/n): y\n"
+ ]
+ }
+ ],
"source": [
"grader[4] = projectData\n",
"grader.grade()"
@@ -913,7 +5458,7 @@
"\n",
" # ====================== YOUR CODE HERE ======================\n",
"\n",
- " \n",
+ " X_rec = Z.dot(U[:, :K].T)\n",
"\n",
" # =============================================================\n",
" return X_rec"
@@ -1186,7 +5731,7 @@
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
- "version": "3.6.4"
+ "version": "3.7.6"
}
},
"nbformat": 4,
diff --git a/Phase 3 - 2020 (Summer)/Week 7(May 10-May 16)/Exercise7/token.pkl b/Phase 3 - 2020 (Summer)/Week 7(May 10-May 16)/Exercise7/token.pkl
new file mode 100644
index 000000000..b028a2890
Binary files /dev/null and b/Phase 3 - 2020 (Summer)/Week 7(May 10-May 16)/Exercise7/token.pkl differ
diff --git a/Phase 3 - 2020 (Summer)/Week 7(May 10-May 16)/__pycache__/submission.cpython-37.pyc b/Phase 3 - 2020 (Summer)/Week 7(May 10-May 16)/__pycache__/submission.cpython-37.pyc
new file mode 100644
index 000000000..fc86be8a1
Binary files /dev/null and b/Phase 3 - 2020 (Summer)/Week 7(May 10-May 16)/__pycache__/submission.cpython-37.pyc differ
diff --git a/Phase 3 - 2020 (Summer)/Week 8/Exercise8/.ipynb_checkpoints/exercise8-checkpoint.ipynb b/Phase 3 - 2020 (Summer)/Week 8/Exercise8/.ipynb_checkpoints/exercise8-checkpoint.ipynb
new file mode 100644
index 000000000..8748a9866
--- /dev/null
+++ b/Phase 3 - 2020 (Summer)/Week 8/Exercise8/.ipynb_checkpoints/exercise8-checkpoint.ipynb
@@ -0,0 +1,1433 @@
+{
+ "cells": [
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "# Programming Exercise 8:\n",
+ "# Anomaly Detection and Recommender Systems\n",
+ "\n",
+ "\n",
+ "## Introduction \n",
+ "\n",
+ "In this exercise, you will implement the anomaly detection algorithm and\n",
+ "apply it to detect failing servers on a network. In the second part, you will\n",
+ "use collaborative filtering to build a recommender system for movies. Before\n",
+ "starting on the programming exercise, we strongly recommend watching the\n",
+ "video lectures and completing the review questions for the associated topics.\n",
+ "\n",
+ "All the information you need for solving this assignment is in this notebook, and all the code you will be implementing will take place within this notebook. The assignment can be promptly submitted to the coursera grader directly from this notebook (code and instructions are included below).\n",
+ "\n",
+ "Before we begin with the exercises, we need to import all libraries required for this programming exercise. Throughout the course, we will be using [`numpy`](http://www.numpy.org/) for all arrays and matrix operations, [`matplotlib`](https://matplotlib.org/) for plotting, and [`scipy`](https://docs.scipy.org/doc/scipy/reference/) for scientific and numerical computation functions and tools. You can find instructions on how to install required libraries in the README file in the [github repository](https://github.com/dibgerge/ml-coursera-python-assignments)."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 1,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "# used for manipulating directory paths\n",
+ "import os\n",
+ "\n",
+ "# Scientific and vector computation for python\n",
+ "import numpy as np\n",
+ "\n",
+ "# Plotting library\n",
+ "from matplotlib import pyplot\n",
+ "import matplotlib as mpl\n",
+ "\n",
+ "# Optimization module in scipy\n",
+ "from scipy import optimize\n",
+ "\n",
+ "# will be used to load MATLAB mat datafile format\n",
+ "from scipy.io import loadmat\n",
+ "\n",
+ "# library written for this exercise providing additional functions for assignment submission, and others\n",
+ "import utils\n",
+ "\n",
+ "# define the submission/grader object for this exercise\n",
+ "grader = utils.Grader()\n",
+ "\n",
+ "# tells matplotlib to embed plots within the notebook\n",
+ "%matplotlib inline"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "## Submission and Grading\n",
+ "\n",
+ "\n",
+ "After completing each part of the assignment, be sure to submit your solutions to the grader. The following is a breakdown of how each part of this exercise is scored.\n",
+ "\n",
+ "\n",
+ "| Section | Part | Submitted Function | Points |\n",
+ "| :- |:- |:- | :-: |\n",
+ "| 1 | [Estimate Gaussian Parameters](#section1) | [`estimateGaussian`](#estimateGaussian) | 15 |\n",
+ "| 2 | [Select Threshold](#section2) | [`selectThreshold`](#selectThreshold) | 15 |\n",
+ "| 3 | [Collaborative Filtering Cost](#section3) | [`cofiCostFunc`](#cofiCostFunc) | 20 |\n",
+ "| 4 | [Collaborative Filtering Gradient](#section4) | [`cofiCostFunc`](#cofiCostFunc) | 30 |\n",
+ "| 5 | [Regularized Cost](#section5) | [`cofiCostFunc`](#cofiCostFunc) | 10 |\n",
+ "| 6 | [Gradient with regularization](#section6) | [`cofiCostFunc`](#cofiCostFunc) | 10 |\n",
+ "| | Total Points | |100 |\n",
+ "\n",
+ "\n",
+ "\n",
+ "You are allowed to submit your solutions multiple times, and we will take only the highest score into consideration.\n",
+ "\n",
+ "
\n",
+ "At the end of each section in this notebook, we have a cell which contains code for submitting the solutions thus far to the grader. Execute the cell to see your score up to the current section. For all your work to be submitted properly, you must execute those cells at least once.\n",
+ "
"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "## 1 Anomaly Detection \n",
+ "\n",
+ "In this exercise, you will implement an anomaly detection algorithm to detect anomalous behavior in server computers. The features measure the throughput (mb/s) and latency (ms) of response of each server. While your servers were operating, you collected $m = 307$ examples of how they were behaving, and thus have an unlabeled dataset $\\{x^{(1)}, \\dots, x^{(m)}\\}$. You suspect that the vast majority of these examples are “normal” (non-anomalous) examples of the servers operating normally, but there might also be some examples of servers acting anomalously within this dataset.\n",
+ "\n",
+ "You will use a Gaussian model to detect anomalous examples in your dataset. You will first start on a 2D dataset that will allow you to visualize what the algorithm is doing. On that dataset you will fit a Gaussian distribution and then find values that have very low probability and hence can be considered anomalies. After that, you will apply the anomaly detection algorithm to a larger dataset with many dimensions.\n",
+ "\n",
+ "We start this exercise by using a small dataset that is easy to visualize. Our example case consists of 2 network server statistics across several machines: the latency and throughput of each machine. "
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 2,
+ "metadata": {},
+ "outputs": [
+ {
+ "data": {
+ "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYUAAAEKCAYAAAD9xUlFAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjMsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+AADFEAAAgAElEQVR4nO3df5RcdX3/8ecrmwRiEswPZkkEU4Sa1XWpIFurUlvWSmtt8UfLYDnfKlXbtOe72/qjpy2657ToqWdta8Vjs6FfWoKYr0oZgQKVbxH5LkT8tuoGBSYbRUHaomFnBZElRZDk/f3j/nD29+xmZ2d28nqcc8/O3Ln3zvsyZN5zPz/eVxGBmZkZwIpGB2BmZs3DScHMzHJOCmZmlnNSMDOznJOCmZnlnBTMzCxXt6Qg6XhJX5F0j6T9kj6Qrn+BpC9L+pakf5K0ul4xmJnZ/NTzSuFp4DUR8VLgTOB1kl4B/BVwWUS8EPgB8M46xmBmZvNQt6QQiSfTp6vSJYDXAJ9N118NvKleMZiZ2fysrOfBJbUB+4CfBgaBB4DHI+LZdJOHgZNn2HcHsANg7dq1Z7/oRS+qZ6hmZi1n375934+Iwnz2qWtSiIjDwJmSNgA3AC+ebrMZ9r0CuAKgu7s7hoeH6xanmVkrkvQf891nSUYfRcTjwB3AK4ANkrJkdArwvaWIwczM5lbP0UeF9AoBSWuA1wIHgCHggnSzi4Eb6xWDmZnNTz2bj7YCV6f9CiuAayPiXySNANdI+kvga8CVdYzBzMzmoW5JISLuBc6aZv2DwMvr9b5mZrZwntFsZmY5JwUzM8s5KZiZWc5JwczMck4KZmaWc1IwM7Ock4KZmeWcFMzMLOekYGZmOScFMzPLOSmYmVnOScHMzHJOCmZmlnNSMDOznJOCmZnlnBTMzCznpGBmZjknBTMzyzkpmJlZzknBzMxyTgpmZpZzUjAzs5yTgpmZ5ZwUzMws56RgZmY5JwUzM8s5KZiZWa5uSUHS8yUNSTogab+kd6XrL5X0XUlfT5fX1ysGMzObn5V1PPazwB9HxN2S1gP7JN2WvnZZRHykju9tZmYLULekEBEHgYPp43FJB4CT6/V+ZmZ29JakT0HSqcBZwJfTVX2S7pW0W9LGpYjBzMzmVvekIGkdcB3w7oh4ArgcOB04k+RK4m9n2G+HpGFJw2NjY/UOc0kMDg5SqVTy55VKhcHBwQZGZGY2UV2TgqRVJAnhUxFxPUBEjEbE4Yg4AvwD8PLp9o2IKyKiOyK6C4VCPcNcEoODg/T19dHT00OlUqFSqdDT00NfX58Tg5k1jXqOPhJwJXAgIj5atX5r1WZvBsr1iqGZFItFOjs7GRkZoauri66uLkZGRujs7KRYLDY6PDMzoL6jj84B3grcJ+nr6br3AxdJOhMI4CHg9+sYQ9Nob29naGiIrq4usuawQqHA0NAQ7e3tDY7OzCxRz9FHdwGa5qVb6vWeZmZ2dDyjeYlkfQhjY2MUCgUKhQJjY2N5H4OZWTNwUlgipVIp70Mol8uUy+W8j6FUKjU6PDMzoL59Clalt7cXSDqcsz6EoaEhSqVS/pqZWaMpIhodw5y6u7tjeHi40WGYmS0rkvZFRPd89nHzkZmZ5ZwUzMws56RgZmY5JwUzM8s5KZiZWc5JwczMck4KZmaWc1IwM7Ock4KZmeWcFMzMLOekYGZmOScFMzPLOSmYmVnOScHMzHJOCmZmlnNSMDOznJOCmZnlnBTMzCznpGBmZrmVs70o6Xjg14FXA88DngLKwOciYn/9wzMzs6U0Y1KQdClwPnAH8GWgAhwPbAc+nCaMP46Ie+sfppmZLYXZrhS+GhGXzvDaRyW1A9sWPyQzM2uUGZNCRHxu8jpJK4B1EfFERFRIrh7MzKxFzNnRLOnTkk6QtBYYAb4p6U/qH5qZmS21WkYfdUbEE8CbgFtImozeOtdOkp4vaUjSAUn7Jb0rXb9J0m2SvpX+3XhUZ2BmZoumlqSwStIqkqRwY0T8GIga9nuWpCP6xcArgF5JncAlwO0R8ULg9vS5mZk1gVqSwv8CHgLWAnsl/RTwxFw7RcTBiLg7fTwOHABOBt4IXJ1udjVJsjEzsyYwY1KQ9EpJioiPR8TJEfH6iAjgP4Ge+byJpFOBs0iGtp4UEQchSRxA+wz77JA0LGl4bGxsPm9nZmYLNNuVwsXAPknXSPodSVsAIvFsrW8gaR1wHfDutG+iJhFxRUR0R0R3oVCodTczMzsKsw1J/QMASS8CfhX4hKTnAkPAvwJfiojDsx087Yu4DvhURFyfrh6VtDUiDkraioe1mpk1jTn7FCLiGxFxWUS8DngNcBdQJGkKmpEkAVcCByLio1Uv3URyFUL698aFBG5mZotv1tpHGUkvA36eZNTRlyLiD2vY7RySoav3Sfp6uu79wIeBayW9k6R/ojjvqM3MrC7mTAqS/pzkiztr/rlKUiki/nK2/SLiLkAzvPxL84rSzMyWRC1XChcBZ0XEjwAkfRi4G5g1KZiZ2fJTyzyFh0iqo2aOAx6oSzRmZtZQs5XO/juSPoSngf2Sbkufn0fS2WxmZi1mtuaj4fTvPuCGqvV31C0aMzNrqNnmKVw902tm1lwGBwcpFou0tycFAiqVCqVSid7e3gZHZstNLaWzf13S1yQ9JukJSeOSap6ZbGb1NTg4SF9fHz09PVQqFSqVCj09PfT19TE4ONjo8GyZqaWj+WMkk8w2R8QJEbE+Ik6oc1xmDTE4OEil8pNJ9pVKpem/WIvFIp2dnYyMjNDV1UVXVxcjIyN0dnZSLHoakM1PLUNS/wsop8XwzFpW9ot7165dDA0NAdDT08PIyAhA0zbFtLe3MzQ0RFdXF1nxyEKhwNDQUN6cZFarWpLCnwK3SLqTZCQSAJNKV5gte8VikV27duW/uAHGxsb8i9uOKbU0H30I+G+SuQrrqxazlpL94i4UCoyNjTE2NrYsfnFnfQhZvFn8WR+D2XzUkhQ2RcRvRMRfRMQHsqXukZktscHBQSbfu+Opp56adftm6H8olUp5H0K5XKZcLud9DKVSacnjsWUuImZdSArY/fJc29VzOfvss8Osnnbu3BlAHHfccQHE5s2bo62tLYDYsmVLjI6OTrt9Z2dnjI6OxujoaHR2dgYQO3fubEj81TGOjo42JA5rLsBwzPP7tpakMA4cAZ4iuQ3nOPDEfN/oaBYnBTsatXxhjo6OxpYtWwKItra22Lx584QkMd32WRIoFApRKBQmJAmzZlCXpNAMi5OC1WK6L/9isVjTL/pisRh33nlnrFu3LkjKucSqVauiXC7nx50uMWTJAIh169ZFuVye8Lp/rVsjLWpSAE6ddcekLPYp833DhSxOCjaX2ZpzsiuAmX7RZ4lj9erVsXHjxvxLHohf+ZVfmTGRDAwM5FcU2dLe3t4UzUlmEQtLCkr2m0pSiaQj+kaS+kdjJCOQfhroIbknwl9ExG0L682oXXd3dwwPD8+9oR2zshE4IyMjZPf0zoaTXnvttfnoHEjG8JfLZdrb2xkcHOSMM87gvPPO45lnnply3BUrVrBmzRoOHTpER0cHe/fupVQq8e1vf5uPfexjAKxZs4aI4Ec/+hEAq1atYvXq1fk+F1xwAVu3bm3aeQ7WuiTti4juee0zU1JID9gJ/A+Su6htJRmaegC4BfhspPdYqDcnBatFpVKZMoGrXC4DcPrpp/Pkk0/m68866ywkceutt9LZ2cnZZ5/Nnj17JhxvxYoVHDlyJH++fft2XvCCF3Drrbciidn+7QCsXLmSE044gcceewyA888/n1e96lVccsklebyuT2T1tJCk0PD+gloWNx9ZLSa38RcKhejv74/29va8A3nTpk0TmntOPPHECc/rtaxatSp/3N/fH/39/XmzlpuX5ubRVQuDO5rtWDXTaKBsWb16dQCxYsWKJUkCsy3VMUw33NUmarbhv8vJQpJCLZPXzJredBO4Ojo68tfb2toA8uag448/ftrjLIUshra2Nr7whS809WzpZuCCf0urltpHZk0va5cfHx8HkpIVe/fu5W1vext33XUXhw4dmrB91incSBs2bMg7xW1mLvi3tGq5n8Lttawza7Q777yT973vfbz0pS9l//79fPzjH+fWW2+dkhCaxaOPPur6RNZ0ZrtH8/HAc4ATJW0kmZcAcALwvCWIzWxW2d3GSqUS4+PjlEolVqxYwSOPPMIZZ5yBpLkP0kBtbW15fSKPQJrZ5IJ/QF7wz1cLdTBTZwPwLuA7JOWyv1O13AP0zbfz4mgWdzTbZFnnYzaC5/TTT887k5fTUiwWG/2fsum5o3nhqFPtoz+c70EXe3FSsMmqvxiywnXLaTnttNNiYGBgwvn4C25mHpK6MAtJCrNOXgOQ9Lbp1kfEJ2fdcRF58tqxYb43n69UKhMmpS03nZ2dU+7wtnPnTjcl2aJZyOS1WkYf/WzV4+NJylvcDSxZUrDWt5BbYe7evXvZJoSVK1f6Dm/WlOa8Upiyg/RcYE9EvGGO7XYDvw5UIqIrXXcp8HskdZQA3h8Rt8z1nr5SaH2z1S6arjOxUqnw6le/mvvvv78R4S6KdevWTSi9kdVjMlssC7lSWMjktf8GXljDdp8AXjfN+ssi4sx0mTMh2LFhvrfCLJVK3H///WzatKkB0R699vZ21qxZ0+gwzKaoZZ7CzZJuSpfPAd8kqZw6q4jYCzy2CDHaMS677WX2t1KpcOWVVwLwgx/8YFl9ua5YkfyTq1QqvqeyNae5eqKBX6xazmEe91AATgXKVc8vBR4C7gV2Axtn2XcHMAwMb9u2bbE75a0JVI8oGR0dje3btwdMrF2UDTnN/nZ0dOTbNetSfaOe6kXShOcdHR0eYml1Rb0K4gFbgDcA5wNbaj741KRwEtBGcoXyIWB3LcfxkNTWM3ns+cDAQP5lOTAwMO1NcqqHnjZDYbtal7a2tik373nDG97gIZZWd3VJCsDvAv9J0kdwNckv/XfUdPBJSaHW1yYvTgqtZ6aqptmv52yb7GpictXT5ZAYJMXatWunJLR169a5MqotiYUkhVo6mv8EOCsificiLgbOBv6shv2mkLS16umbgfJCjmPL30wdy3v37s07ltvb22cds199A5xm9JznPIdbbrmF4447jsOHD7Nu3ToKhQJPPvmk+w+sadWSFB4GxquejwP/NddOkj4D/BvQIelhSe8E/lrSfZLuJbml53sWELMdQ6rr3mTlr4Gmr2sEcOjQId7ylrfw9NNPs2XLFh544AHK5XJeBrpUKjU6RLMpapm89l3gy5JuJLn8fSPwFUnvBYiIj063U0RcNM3qKxcaqLWWWoucZfdJ2LJlC4888ggdHR1ERFPPT8hu45nFXCwW2blzZ35OQ0NDLoJnTauWK4UHgH8mSQiQDEc9CKxPF7N5m+6mONP9gu7t7WXnzp3cc8897Ny5k71793L99dezZcsWgAlXD81i48aNDAwM5DFfe+21E+ZazNUsZtZQ8+2EaMTijubWVCwWo1wu58/L5XJNVUOrRy6Vy+X8HszNsGQdytmoKrNGoh4dzZK2S7pC0ucl/d9sOcpcZMe4wcFBSqUSF154YT4h7cILL6RUKjE4ODjrfllzzNDQEDfffDOVSoXt27fT39/P2rVrl/Aspjp8+DDt7e3uM7Blq5Y+hRLw98A/AofrG44dK4rFIrt27ZpXUbjpiubt2bMHgLe//e2sX7+eQ4cOISkb9lx3Wf9Bf39/XnrjPe95D+vXr3cTkS1Pc11KAPvme/mx2Iubj1rT5PkHhUJh1iaX6rkNmzdvnjC3IZvwlk10q17WrFmz6M1EmzZtCkhu7pPdF8ET0KzZsJjNR5I2SdoE3Czpf0ramq1L15stqfb2dt761rcCyf2Nx8bG2Lx5MxHB+973Pvr6+njkkUc4/fTTWb16db7fU089tWgxtLW1US6XOXDgAJ2dnTzwwAOsX78+j89XB7bszZQtSG69+SATb8WZLQ/ON/sczeIrhdYz04zmuTpoy+XyhNnBWT2hbL9sBnS5XJ5Sa2jHjh3zviJYtWrVhOcbN26cMuParFlRr9pHjV6cFFrPQu67W71N9Rd+W1tb9Pf3T9i2v79/yhf8iSeemH+xr1y5cs6EUL1N9ftt377dI4tsWahLUgB+Y5rll4D2+b7ZQhcnhdY03/vuZokEpq97lO07XUKoTgzZlUQtiSFLAuVyOTo6Oqa8l1kzW0hSqGX00TuBVwJD6fNzgX8Htkv6YETsqeEYZlNMbn+fq02+t7eXgwcP8jd/8zc888wzE0YZHXfccZx77rkA3Hvvvfk+GzZsYMWKFTz2WHJrjxNPPJH29nZKpRLPPvssW7ZsQRIHDx6c9j27urq4/fbbaW9vZ+/evezevdsji6yl1ZIUjgAvjohRAEknAZcDPwfsBZwUbMls3bqVZ555hra2Ng4fTkZIt7W18fTTT3PHHXfwkpe8hJtuuokzzjiDcrnM448/DiTJ4cc//jHf+MY38nkQAwMDvOMd7wDgtNNO49ChQ6xbt441a9YwNjbG+vXr84QASdK65JJLGnDWZkunljIXp2YJIVUBtkfEY8CP6xOW2fR6e3sZGBhgw4YN+boNGzYwMDAw4df7NddcM6EExg9/+EMOHTpEZ2cn4+Pj9PX15XMcdu/ezaFDhwDo7+/PS26Mj497Apodc5Rdfs+4gbQL2EYyiQ3gN0kqp/4J8C8R0VPXCIHu7u4YHh6u99vYMpAV0hsZGZlQSK+zszMvpFe9TXUTU1tbG/fccw+FQmHaY3R0dOSluyuViovW2bInaV9EdM9nn1quFHpJbrBzJnAW8EmgNyIOLUVCMKtWSyG9bJuOjg42bfrJlJrDhw9z8803L8q9HMxa1Zx9CmkP9mfTxayhsi/qYrE4Yynq3t5exsfHueqqq3j00UcpFAocOXKERx99lD179uT9CGY2VS0F8cYlPZEuP5J0WNITSxGc2XR6e3vnLEW9fv167r///vyKIru6GBkZYffu3RPu5ZBdMfhuaGa1XSlMuGeCpDcBL69bRGaLYLYrCiBPEllhvayPwf0Idqybs6N52p2kf4+IV9Qhnmm5o9kWW1aCO0sY7li2VrSQjuZaRh/9RtXTFUA38IsR8cr5h7gwTgpmZvO3kKRQy+S186sePws8RHKfZjMzazG19Cm8fSkCMTOzxqtl9NEpkm6QVJE0Kuk6SacsRXBmZra0apm8dhVwE/A84GTg5nSdmZm1mFqSQiEiroqIZ9PlE0ChznGZ2RIYHBycMDejUqnkBQPt2FRLUvi+pN+W1JYuvw08Wu/AzKy+BgcH6evryyftZTWj+vr6nBiOYbUkhXcAFwKPAAeBC9J1ZraMFYvFfJZ3V1cXXV1d+aS+YrHY6PCsQWYdfSSpDfjNiHjDEsVjZkskKwzY1dXF2NgYAIVCIa82a8emWa8UIuIwC5yTIGl3OmKpXLVuk6TbJH0r/btxIcc2M7P6qKX56EuSdkp6taSXZUsN+30CeN2kdZcAt0fEC4Hb0+dm1gBZH4ILA1q1WpLCq4CXAB8E/jZdPjLXThGxF3hs0uo3Alenj68G3lRzpGa2qGq5N4Ude2qZ0byYN9I5KSIOpsc9KGnGhktJO4AdANu2bVvEEMwMars3hR17aimIdxzJLThPpSqJRMQH5zy4dCrJLTu70uePR8SGqtd/EBFz9iu4IJ4tlKuh2rGsXgXxbgR+COwDnl5IYFVGJW1NrxK2Am64tLrJxuHv2rVryn0TACcGs2nUkhROiYjJHcYLdRNwMfDh9O+Ni3RcsymKxSK7du3Kx+EDjI2NeRy+2Sxq6Wj+f5LOmO+BJX0G+DegQ9LDkt5JkgzOk/Qt4Lz0uVldZOPws1E12Sgbj8M3m9mMVwrp/IIj6TZvl/QgSfORgIiIn5ntwBFx0Qwv/dICYzUzszqbrfnoZODMpQrEbLFNHocP5OPwfbVgNr3Zmo++ExH/MdOyZBGaLZDH4ZvN32xXCu2S3jvTixHx0TrEY7ZoPA7fbP5mnKcg6SBwOUkfwhQR8YE6xjWB5ymYmc3fYs9TOFjLBDUzM2sds/UpTHuFYGZmrWu2pOCho2Zmx5gZk0JETK5wamZmLa6WGc1mZnaMcFIwM7Ock4KZmeWcFMzMLOekYGZmOScFMzPLOSmYmVnOScHMzHJOCmZmlnNSMDOznJOCmZnlnBTMzCznpGBmZjknBTMzyzkpmJlZzknBzMxyTgpmZpZzUjAzs5yTgpmZ5VY24k0lPQSMA4eBZyOiuxFxmJnZRA1JCqmeiPh+A9/fzMwmcfORmZnlGpUUAvi8pH2SdjQoBjMzm6RRzUfnRMT3JLUDt0n6RkTsrd4gTRY7ALZt29aIGM3MjjkNuVKIiO+lfyvADcDLp9nmiojojojuQqGw1CGamR2TljwpSForaX32GPhloLzUcZhZ/Q0ODlKpVPLnlUqFwcHBBkZkc2lE89FJwA2Ssvf/dET8awPiMLM6GhwcpK+vj127djE0NARAT08PIyMjAPT29jYyPJuBIqLRMcypu7s7hoeHGx2Gmc1DpVLJk0DWBDw2NkZnZydDQ0O0t7c3OMLWJ2nffOeBeUiqmdVFe3s7Q0NDFAoFxsbGGBsbo1AoOCE0OScFMzPLOSmYWV1kzUfZFUJ2xdDT0zOh89mai5OCmdVFqVRiZGSEzs5OyuUy5XKZzs5ORkZGKJVKjQ7PZtDI2kdm1sKy0UXFYjHvQxgaGqJUKnnkURPz6CMzsxbl0UdmZnZUnBTMzCznpGBmZjknBTMzyzkpmJlZzknBzMxyTgpmZpZzUjAzs5yTgpmZ5ZwUzMws56RgZmY5JwUzM8s5KZiZWc5JwczMck4KZmaWc1IwM7Ock4KZmeWcFMzMLOekYGZmOScFMzPLOSmYmVnOScHMzHINSQqSXifpm5K+LemSRsRgZmZTLXlSkNQGDAK/CnQCF0nqXOo4zMxsqkZcKbwc+HZEPBgRzwDXAG9sQBxmZjbJyga858nAf1U9fxj4uckbSdoB7EifPi2pvASxNcqJwPcbHUQdtfL5tfK5gc9vueuY7w6NSAqaZl1MWRFxBXAFgKThiOiud2CN4vNbvlr53MDnt9xJGp7vPo1oPnoYeH7V81OA7zUgDjMzm6QRSeGrwAslvUDSauC3gJsaEIeZmU2y5M1HEfGspD7gVqAN2B0R++fY7Yr6R9ZQPr/lq5XPDXx+y928z08RU5rzzczsGOUZzWZmlnNSMDOzXFMnhVYvhyHpIUn3Sfr6QoaONRtJuyVVqueUSNok6TZJ30r/bmxkjEdjhvO7VNJ308/w65Je38gYj4ak50saknRA0n5J70rXL/vPcJZza4nPT9Lxkr4i6Z70/D6Qrn+BpC+nn90/pYN7Zj9Ws/YppOUw7gfOIxnG+lXgoogYaWhgi0jSQ0B3RLTE5BlJvwA8CXwyIrrSdX8NPBYRH04T+8aI+LNGxrlQM5zfpcCTEfGRRsa2GCRtBbZGxN2S1gP7gDcBv8My/wxnObcLaYHPT5KAtRHxpKRVwF3Au4D3AtdHxDWS/h64JyIun+1YzXyl4HIYy0xE7AUem7T6jcDV6eOrSf4hLksznF/LiIiDEXF3+ngcOEBSgWDZf4aznFtLiMST6dNV6RLAa4DPputr+uyaOSlMVw6jZT7EVACfl7QvLevRik6KiIOQ/MME2hscTz30Sbo3bV5adk0r05F0KnAW8GVa7DOcdG7QIp+fpDZJXwcqwG3AA8DjEfFsuklN36HNnBRqKoexzJ0TES8jqRjbmzZP2PJyOXA6cCZwEPjbxoZz9CStA64D3h0RTzQ6nsU0zbm1zOcXEYcj4kySKhEvB1483WZzHaeZk0LLl8OIiO+lfyvADSQfZKsZTdtzs3bdSoPjWVQRMZr+YzwC/APL/DNM26OvAz4VEdenq1viM5zu3Frt8wOIiMeBO4BXABskZZOUa/oObeak0NLlMCStTTu8kLQW+GWgFSvB3gRcnD6+GLixgbEsuuzLMvVmlvFnmHZWXgkciIiPVr207D/Dmc6tVT4/SQVJG9LHa4DXkvSbDAEXpJvV9Nk17egjgHR42Mf4STmMDzU4pEUj6TSSqwNIyo18ermfn6TPAOeSlCMeBf4C+GfgWmAb8J9AMSKWZWftDOd3LknTQwAPAb+ftb8vN5J+HvgicB9wJF39fpK292X9Gc5ybhfRAp+fpJ8h6UhuI/mxf21EfDD9nrkG2AR8DfjtiHh61mM1c1IwM7Ol1czNR2ZmtsScFMzMLOekYGZmOScFMzPLOSmYmVnOScGWPUlPzr1Vvu25kl5Vz3jmeP93S3rbIhznGkkvXIyYzKo5Kdix5lygIUkhnVn6DuDTi3C4y4E/XYTjmE3gpGAtSdL5aR35r0n6gqST0kJofwC8J62d/+p0Juh1kr6aLuek+1+aFki7Q9KDkv6o6thvSwuo3SNpj6T1kr6TllFA0glK7pWxalJYrwHuzgqUpce+TNLetM7/z0q6Pq19/5fpNmslfS59r7Kkt6TH+iLw2qoSBmaLwv9DWau6C3hFRISk3wX+NCL+OK0pn9fPl/Rp4LKIuEvSNuBWflJI7EVAD7Ae+Kaky4HtQD9JMcPvS9oUEeOS7gB+jWQG928B10XEjyfFdA5JHf9qz0TELyi56cuNwNkk5bkfkHQZyZXN9yLi19J4nwsQEUckfRt46TTHNFswJwVrVacA/5TWtlkNfGeG7V4LdCalcQA4IatJBXwuLQnwtKQKcBJpffrsxkhV5R7+kaQ555+BtwO/N817bSWpR1Mtq+d1H7A/K7Eg6UGSgpD3AR+R9FfAv0TEF6v2rQDPw0nBFpGbj6xV/R2wMyLOAH4fOH6G7VYAr4yIM9Pl5PQmLADVNWIOk/yIEtOUH46ILwGnSvpFoC0ipius9tQ0cWTvcWTS+x0BVkbE/SRXD/cBA5L+vGqb49Njmi0aJwVrVc8Fvps+vrhq/ThJc1Dm80Bf9kTSmXMc93bgQkmb0+03Vb32SeAzwFUz7HsA+Ok5I68i6XnAf0fE/wY+Arys6uXtwP75HM9sLk4K1gqeI+nhquW9wKVASdIXgep7YN8MvDnraAb+COhOOwC0G0AAAACeSURBVI5HSDqiZxQR+4EPAXdKugeoLjH9KWAjSWKYzv8B5nsjpTOAr6R31OoHsg7ok4CnlmNFT2turpJqtkgkXQC8MSLeOss2N5B0en/rKN/rPcATEXHl0RzHbDJ3NJstAkl/R3Jb1dfPseklJB3OR5UUgMeBPUd5DLMpfKVgZmY59ymYmVnOScHMzHJOCmZmlnNSMDOznJOCmZnl/j82THhKb7ZEMgAAAABJRU5ErkJggg==\n",
+ "text/plain": [
+ "
"
+ ]
+ },
+ "metadata": {
+ "needs_background": "light"
+ },
+ "output_type": "display_data"
+ }
+ ],
+ "source": [
+ "# The following command loads the dataset.\n",
+ "data = loadmat(os.path.join('Data', 'ex8data1.mat'))\n",
+ "X, Xval, yval = data['X'], data['Xval'], data['yval'][:, 0]\n",
+ "\n",
+ "# Visualize the example dataset\n",
+ "pyplot.plot(X[:, 0], X[:, 1], 'bx', mew=2, mec='k', ms=6)\n",
+ "pyplot.axis([0, 30, 0, 30])\n",
+ "pyplot.xlabel('Latency (ms)')\n",
+ "pyplot.ylabel('Throughput (mb/s)')\n",
+ "pass"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "### 1.1 Gaussian distribution\n",
+ "\n",
+ "To perform anomaly detection, you will first need to fit a model to the data's distribution. Given a training set $\\{x^{(1)}, \\dots, x^{(m)} \\}$ (where $x^{(i)} \\in \\mathbb{R}^n$ ), you want to estimate the Gaussian distribution for each of the features $x_i$ . For each feature $i = 1 \\dots n$, you need to find parameters $\\mu_i$ and $\\sigma_i^2$ that fit the data in the $i^{th}$ dimension $\\{ x_i^{(1)}, \\dots, x_i^{(m)} \\}$ (the $i^{th}$ dimension of each example).\n",
+ "\n",
+ "The Gaussian distribution is given by\n",
+ "\n",
+ "$$ p\\left( x; \\mu, \\sigma^2 \\right) = \\frac{1}{\\sqrt{2\\pi\\sigma^2}} e^{-\\frac{\\left(x-\\mu\\right)^2}{2\\sigma^2}},$$\n",
+ "where $\\mu$ is the mean and $\\sigma^2$ is the variance.\n",
+ "\n",
+ "\n",
+ "### 1.2 Estimating parameters for a Gaussian \n",
+ "\n",
+ "You can estimate the parameters $\\left( \\mu_i, \\sigma_i^2 \\right)$, of the $i^{th}$ feature by using the following equations. To estimate the mean, you will use: \n",
+ "\n",
+ "$$ \\mu_i = \\frac{1}{m} \\sum_{j=1}^m x_i^{(j)},$$\n",
+ "\n",
+ "and for the variance you will use:\n",
+ "\n",
+ "$$ \\sigma_i^2 = \\frac{1}{m} \\sum_{j=1}^m \\left( x_i^{(j)} - \\mu_i \\right)^2.$$\n",
+ "\n",
+ "Your task is to complete the code in the function `estimateGaussian`. This function takes as input the data matrix `X` and should output an n-dimension vector `mu` that holds the mean for each of the $n$ features and another n-dimension vector `sigma2` that holds the variances of each of the features. You can implement this\n",
+ "using a for-loop over every feature and every training example (though a vectorized implementation might be more efficient; feel free to use a vectorized implementation if you prefer). \n",
+ ""
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 3,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "def estimateGaussian(X):\n",
+ " \"\"\"\n",
+ " This function estimates the parameters of a Gaussian distribution\n",
+ " using a provided dataset.\n",
+ " \n",
+ " Parameters\n",
+ " ----------\n",
+ " X : array_like\n",
+ " The dataset of shape (m x n) with each n-dimensional \n",
+ " data point in one row, and each total of m data points.\n",
+ " \n",
+ " Returns\n",
+ " -------\n",
+ " mu : array_like \n",
+ " A vector of shape (n,) containing the means of each dimension.\n",
+ " \n",
+ " sigma2 : array_like\n",
+ " A vector of shape (n,) containing the computed\n",
+ " variances of each dimension.\n",
+ " \n",
+ " Instructions\n",
+ " ------------\n",
+ " Compute the mean of the data and the variances\n",
+ " In particular, mu[i] should contain the mean of\n",
+ " the data for the i-th feature and sigma2[i]\n",
+ " should contain variance of the i-th feature.\n",
+ " \"\"\"\n",
+ " # Useful variables\n",
+ " m, n = X.shape\n",
+ "\n",
+ " # You should return these values correctly\n",
+ " mu = np.zeros(n)\n",
+ " sigma2 = np.zeros(n)\n",
+ "\n",
+ " # ====================== YOUR CODE HERE ======================\n",
+ " mu = (1 / m) * np.sum(X, axis = 0)\n",
+ " sigma2 = (1 / m) * np.sum((X - mu) ** 2, axis = 0)\n",
+ " \n",
+ " # =============================================================\n",
+ " return mu, sigma2"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Once you have completed the code in `estimateGaussian`, the next cell will visualize the contours of the fitted Gaussian distribution. You should get a plot similar to the figure below.\n",
+ "\n",
+ "\n",
+ "\n",
+ "From your plot, you can see that most of the examples are in the region with the highest probability, while\n",
+ "the anomalous examples are in the regions with lower probabilities.\n",
+ "\n",
+ "To do the visualization of the Gaussian fit, we first estimate the parameters of our assumed Gaussian distribution, then compute the probabilities for each of the points and then visualize both the overall distribution and where each of the points falls in terms of that distribution."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 4,
+ "metadata": {},
+ "outputs": [
+ {
+ "data": {
+ "image/png": "iVBORw0KGgoAAAANSUhEUgAAAagAAAEYCAYAAAAJeGK1AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjMsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+AADFEAAAgAElEQVR4nOydd1hURxfG30vvTToK2BUVsWvsCtg7qNh7rDEmJtFoNIk1JsZeYu+99y52VLCAUpQivbcFlu3n+2OBmHxG7y67iMn8nmdcuHtn5tzFve+dmTPncEQEBoPBYDAqGzof2wAGg8FgMN4FEygGg8FgVEqYQDEYDAajUsIEisFgMBiVEiZQDAaDwaiUMIFiMBgMRqVEawLFcZwRx3GPOI57znHcS47jfio5vovjuDiO456VFC9t2cBgMBiMTxc9LbYtBtCFiAo5jtMHcJfjuIsl731DRMe02DeDwWAwPnG0JlCk3AFcWPKrfklhu4IZDAaDwQtOm5EkOI7TBRACoBaADUT0HcdxuwC0gXKEdR3AHCISv6PuJACTAMDU1LRZvXr1tGYng8FgMLRLSEhIFhHZqVJHqwJV1gnHWQE4CWAGgGwAaQAMAGwBEENEP7+vfvPmzSk4OFjrdjIYDAZDO3AcF0JEzVWpUyFefESUByAQQHciSiUlYgA7AbSsCBsYDAaD8WmhTS8+u5KREziOMwbgDSCS4zinkmMcgP4AXmjLBgaDwWB8umjTi88JwO6SdSgdAEeI6BzHcTc4jrMDwAF4BmCyFm1gMBgMxieKNr34QgE0ecfxLtrqk8FgMBj/HlgkCQaDwWBUSphAMRgMBqNSwgSKwWAwGJUSJlAMBoPBqJQwgWIwGAxGpYQJFIPBYDAqJUygGAwGg1EpYQLFYDAYjEoJEygGg8FgVEqYQDEYDAajUsIEisFgMBiVEiZQDAaDwaiUMIFiMBgMRqWECRSDwWAwKiVMoBgMBoNRKWECxWAwGIxKCRMoBoPBYFRKmEAxGAwGo1LCBIrBYDAYlRImUAwGg8GolDCBYjAYDEalhAkUg8FgMColTKAYDAaDUSlhAsVgMBiMSonWBIrjOCOO4x5xHPec47iXHMf9VHK8OsdxDzmOe81x3GGO4wy0ZQODwWAwPl20OYISA+hCRI0BeAHoznFcawC/AFhFRLUB5AIYr0UbGAwGg/GJojWBIiWFJb/qlxQC0AXAsZLjuwH015YNDAaDwfh00eoaFMdxuhzHPQOQAeAqgBgAeUQkKzklCYDLP9SdxHFcMMdxwZmZmdo0k8FgMBiVEK0KFBHJicgLQFUALQHUf9dp/1B3CxE1J6LmdnZ22jSTwWAwGJWQCvHiI6I8AIEAWgOw4jhOr+StqgBSKsIGBoPBYHxaaNOLz47jOKuSn40BeAOIAHATgF/JaaMBnNaWDQwGg8H4dNH78Clq4wRgN8dxulAK4REiOsdxXDiAQxzHLQbwFMB2LdrAYDAYjE8UrQkUEYUCaPKO47FQrkcxGAwGg/GPsEgSDAaDwaiUMIFiMBgMRqWECRSDwWAwKiVMoBgMBoNRKWECxWAwGIxKCRMoBoPBYFRKmEAxGAwGo1LCBIrBYDAYlRImUAwGg8GolDCBYjAYDEalRJux+BiMj4pMKoNUIoNCJodcpoBcJodcrgApCBwHgOOgo8OB4zjo6OpAz0APBkb60NPXA8dxH9t8BuM/DxMoxidDcZEIcaHxSHqVitz0fOSm5yEvMx+56fnIzxSguFAEUZEYoiIRxEIJ5DK52n3pG+rDwEgfJhbGMLc2g6mVCcysTGFmbQobBys413SEYw0HONd0gF3VKtDV09XglTIYDIAJFKOSIpfJEf7gFcIfvEL0szjEPI1D0qtUEP2Z39LQ2ADWDpawcrCCXdUqMDY3gpGJEYxMDWFoYggjE0PoG+pBV08Xunq60NHVga6eDjiOAxGBCMpXBUEhV0AqkUEqlkIikkAqlkEikkBYUIyivCIU5BYhPT4TMc/eICc1FzLpn+Knq6eLqnWc0NTbEy16NEHjjh4wMDL4GB8bg/Gvgnv7C19Zad68OQUHB39sMxjlgIggFAiRk5aH/EwBREIJJMUSiEuKVCSBVCKDXCpHbFg8Hp5/goKcQgCAvastajWpjlpe1VHTyx1uDarCxtEKxmbGH+Va5HI5spNzkBKTjtRYZXn9JBaht8IhEUlhaGyAxp0boMFn9WBkagh9Az3oGehBT1/5amxmBFNLE5hamsDMSvlqbG4MHR22JMz498JxXAgRNVelDhtBMTSOuFiMZzdf4uG5EMSExiM3LQ85qbkQF0t41Te3MUOrXk3Rpk9zeHVuCIsq5lq2WDV0dXVh72oHe1c7eHVuWHZcJBQj9FY4Hl98iseXn+HRhae82zQyNYR7g2pwb+iKGp5uqN7IFTUau8HCpnJdO4NRkbARFEMj5Kbn4cHZEASdC8bTa2EQCcUwMjVE3Ra1UMXZGjaO1rBxsoaNoxWs7C1gaGIIQ2MDGBgbKF+N9MtGGYYmBtDV/fTXdIqLRJBJZJBJZGWjQ4lYClGhCEX5QhTmFZW8CpGZmIW4FwmIC41HflYBAEBHh0PbAS0xcGYvNGhbjzluMD5p2AiKUSEQEURCMYSCYqTFZeDspssIPHwfcpkc9q628B3TCa37NEfjTg1gYKj/sc39aBibGgGmqtUhIuSm5yE2NAFPr4fh4rZruHP8IWo3q4GBM3uhUfv6sKhiBiNTIyZYjH89bATF+EekEimin75BYmSyskQlIyEyBakxaX9xEjA2M0LPCV3RbWxnuDd0/Sg3TplUBmFBMYoLRCWvxSguFEEqlkEmlSndzEteFQqFcr2HA3R0dMDpcNDV1YG+kT4MjAz+MrIzszaFRRXzjya0xUUiXN93ByfWnEdiZHLZcQMjfVjaWcDKzgKWdhao2dgdnQPaoYanGxMuRqVEnREUEyjG/yERS3F5500cXHYCmYnZAAA9fV0413JEtXoucKnlBIsq5iUu2KZo0d0LppYqDhXUID9LgJf3oxAXloCspGxkJecoS1I28jIFWu3bxNwYFrbmsLQ1h101W7jVrwrX+i5w9aiKanWdYWhsqNX+FQoFQm+FIzU2HYLsQuRn5iMvS4C8DAHyMvIR+zwecpkc7g2qocuw9ugyrB0c3Oy0ahODoQpMoBjlQiKW4vKOGzi47CQyk7Lh0aYOBs7shZpe7nCsbg89/YqdEU6NTcfzwJd4eS8SL+9HITEqpew9iyrmsK1qA1sXG9i5VEEVZ5sSbzgjmJgbw9jcGMZmRjAw0oeuvtLNXE9fV3kNHABS3vSJAFIoIJcpIBEpPQslIqnSu1AoRkFuEQRZBRBkFyA/WwBBVgHS4jKQHJ0GhVwBAOA4DlXrOKGZb2O07Nn0o7iZ52cJcPvoA1w/cAcv70UBADw7emD4vEFo0rURG1UxPjpMoBi8ICJkp+QgITLlL9N3sc/jkZcpgEebOhj142A09fbU2I1NIpYiJToNiZHJSI1NR1G+EEJBMYoEQggFQhQJiiERSZUOBWIpiguKkfYmE4DSq6/BZ3WVpW091G5WA0Ym2h2x8L2e+PAkJEQkIeLhazy/+aLMzdzB3Q56BnowLJkqNDQxhIGxAYxMDGFhY6Z0GCkpVZysYO9mp1yz0gBpbzJw8+A9nN18GZmJ2ahWzwWO1e1Rxcka9tVs0W5gS1Rv5KaRvhgMvjCBYrwXcbEYp9ZdwtHfTpd5igHK6atq9ZzhXMsR3cZ01ogwFeQW4tKOmwi99RIJkclIi02HQvHn/zUdHQ4mFso9QCYWxjCxMIahsQH0DPSgb6gPfUN91G9ZG826NUa1us6fxB4hcbEYzwPD8fjSU+Sk5ZVs+pVCLBSXjcjEQgnyswQQFYn/UldPXxeNOnigZY8maNGjCVzruZT7byARS3Fp+w08vvwU2Sm5yE7JRW5aHogIDdrWRd8p3dBuUOv/tCMLo+JgAsV4J3KZHFd2B2LPj0eQlZyDlj2boFXPZqhWzxnV6rmgipO1xkZKcWHxOLXuIq7vvwNxsQRuHlXh1qAaqtV1hms9F7jWrwqnmg4wMTf+T087CQuKkZOai+zUXOSk5iH6SSweXXyKNy8TAQCO7nZo1asZek/2hXuDahrrV5BdgMu7AnFu82WkxKTDys4C3cZ2Rr/pPWBXtYrG+mEw/g4TKEYZRIRXIbF4fvMFLu+6iYSIZNRvXRsTlo+AZwePcrVdmFeEnLQ85GXklxTlQv3zWy8ReischsYG6DKsPfpN746ajd01c0H/EdLjM/Ho4lM8vvQUIVeeQyKSoplvYzRoUxfmNmYwtzGDS21HuDd0Ldc0p0KhwJNrYTi3+TIenAmGnoEeeozvCo82dcoeKFi4JoYmqVQCxXFcNQB7ADgCUADYQkRrOI77EcBEAJklp35PRBfe1xYTKH5IJVI8DwzH/VOP8OBsMLKScwAA7g2qYcyiofisXwu1Ry1Zydm4fTQIgUfuISLo9TvPca7pgF6TfNB9fBcWAUED5GcJcH7LNZz740qZN2UpHMfBpbYjqnu6oUYjNzTv1hj1WtZWq5+0NxnYteAQAg/dLwuwq6PDoV6r2hi3dBgad2xQ7mthMCqbQDkBcCKiJxzHmQMIAdAfwGAAhUT0G9+2mEC9n9yMfOz98QiuH7gDoaAYRiaGaNatMdr2a4kWPbxgZWepVrsFuYUIPHQPNw/fw4s7kSAi1PRyR7sBreBcyxFW9pawtreAlb0lLKqYf9SI3kSEgpxCZCZlIyspG5lJSvfzIoEQYqEE4mIxxEIxRCVRzt8OHqurpwt9Az2YWytHKBZVzJXF1hwutRzhVNPho0e2kMvkKMwrQl6mAImRyYgLTUBsWDxin79BSkw6AKCZb2OMXOCPBp/VVasPiViK5NepiH+ZiDcvEnFlTyAyE7PRdkBLTPxlBFxqOWnykhj/MSqVQP1fRxx3GsB6AG3BBEojCHIKcGn7DexfchxioQRdhrdDh0Ft0KRrQ7X35YiEYqS/yUDg4fs4seY8hIJiuNZ3QachbdFpyGeoVtdFw1fxz0jEUoiKRJCIpGUOB1KxFFKxDHKZHGlxGYh6HI1XIbGIffYGIuFfHQ84jlM6X5gYwshE6UlnaGIIHV0dKOQKKOQlOaJkckjFMhTkFKIwt/AvzhyAMvVGtXrOcG9QDW4e1VDTyx3mNmbQ01cKm66+HvQNlUFgjUyNYGhsUKHra0X5RTi/5RqO/nYGeZkCNPVuBL+v+6JWk+qwtlfv4QRQOn0cX3Ueh5afhFQsRb9p3THoqz5srYqhFpVWoDiOcwdwG0BDAF8BGANAACAYwNdElPu++p+qQG3YAPj7A/b2fz2ekQEcPQpMm6Z6m4LsAhz59TQeX36GuNAEEBFa926GSb+OVEs8iAjPbr7AidXnEfU4Grnp+WXvtR3QEsO+H4jaTWto/IZbXCRCYmSy0k07PAmJUcnIyxSgMFeZ2qIwtxASkfSD7RiZGKJW0+qo1aQ6nKo7KPdGVa0Cu6pVYONopfKoTqFQoChfqNz3lClAYlSKckQRnoT4l4nISMj6YBscx8HYzAjG5kawcbSCbdUqsHWpotyzVbUK6jSvAdf6VbXymZ7bfBVHfj2NvAzl39HBzQ71WtVC/VZ1UL91bdRqUl3ltaWctFzs+uEwLu24ASJCy55N4PdVH3h1bvifdnRhqEalFCiO48wA3AKwhIhOcBznACALAAFYBOU04Lh31JsEYBIAuLq6NouPj9eqnZpmwwZg+nTAwwO4efNPkcrIADp3BsLDgfXr+YuUXCbH+S3XsOuHgygSFKNxpwbw7OCBZr6NUb+V6msPcrkcd088wpFfT+NVcAysHSzRqmdTONZwgFN1e9RsUh1u9auq3O77SIhMxoWt13Dv1COkv8ksy+1UGqXCxtEKZtZmMLMyhbm1KUytTEs22yqDyZYmEdQzUOZ4quJsDdf6LhU6/VYkECIuNB7CAhFkUmUgWJlUDolIOdoTFYkhKhRBVKQMCJudllcW9aI0fQigTCHSonsTtOzRBE26NtRo6pDSqOoJEUmIfPQakQ+jkR6vXPLVN9RHzwldMfwHP5VHV4lRybhx4C7Obb6CvEwBajR2g9+sPug09DPoGzBXdcb7qXQCxXGcPoBzAC4T0e/veN8dwDkiavj3997mUxxBvS1EpSIF/P+xv4+u3sXzwJfYMHMH4sIS4NWlIaauHovqDV3VsksqkeLSjps4+tsZpMamw6W2EwbP7gvvkR204rUlEUlw5/hDnN9yFWF3IqCnr4sWPZqgTrOacPOoClePqnCp5VjhUSo+BiKhGBkJWQi7rdwr9eRaGIoLRdDT10UTb0/0/twHrXo11YrgZqfmIvLhawSdC8GV3YEwNDaA/9d9Meir3jAxV00cJSIJru+/g+OrziE+PAk2TtYYMX8Qek7y/uhrdYzKS6USKE459t8NIIeIvnzruBMRpZb8PAtAKyIa+r62PkWBAv4qUnYlYdEyM/mL0/PAl9j781E8D3wJe1dbTF45Gu0GtlJpWqUwrwivgmMQ9TgGUY9fI/zBK+Sm56Ney1oY/G1/fNaveblvKsWFxQi+/BzJ0WnK3E9pucgpyQGVlZQDkVAM51qO6DXRGz6jO5VrXeTfhFQixct7UXh88SluHLyLrOQcmFmZwtrRCpa25rB2sIR7A1fUaqJM1GjvaquRKbXEqGTs/OEQ7hwLgqmlCeq2rIX2A1uj29hOKo2EiAjBV57j4LITCLsdgWp1neE7uhO6jujA1qkY/0dlE6h2AO4ACIPSzRwAvgcQAMALyim+NwA+LxWsf+JTFShAKVINGyqFCVAK1YsX7xenrJQcbP12L24cuAt7V1v0meyL/l/0VGnfS3p8JrbN3Y9bh++XTaW51HZC3RY14T2iA5p38yrXza64SIRHF57i1tH7eHT+SVkyQmMzo7K8TzZOVrBxtMZn/VrAs6PHJxEN4mMhk8rw4EwwnlwLhSCnEIIsAbKSc5D8Oq3s72dubYqaXu7oHNAePqM6lHtaLeLha1zYeg0RQa8QH54EBzc7DJ8/CD6jOqo0oiUi3DkehJNrL+DF3UhwHAevLg3Rd2o3tO3fkq1TMQBUMoHSJP8VgZJJZTi17iL2/nQUUokMQ77th6Fz+qvkkScsKMbBZSdxfNU56Ohw6DetO5r6eKJO85owtzYr9/VEPHyNE2vOI+hMMERCMawdLNF+UGt08G+DOs1qfLQ07P9WiotEePMiEdFP4xDz7A1e3ovEm5eJsKtaBf6z+6LHhK7ljktIRAi+/Ay7FhzGq+AYONd0wIgF/ugyrJ3Ko+uUmDRc33cHV/feQmpsOpr6eGLGuvGoWse5XDYyPn00LlAcxxkB6A2gPQBnAMUAXgA4T0Qvy2GrSnyqAqXKFN+T62HYNGsn3rxIRIseTTBtzViV9p2kxKTh5sF7OL3hInLT89F1eHuMWzoM9tVs1bZfJpUh5nk8kl+nIvl1KsLuRODp9TCY25iho38bdBrSFg3b1/vo6w5EBGFBMcRCMSQipTu6Miq5BDKp/E+X8pJXEAEcB47jwHEAOA46OlxZRl89Az3oGyhfjUwNS6KjG3306yy91uArz3Fw6QmE3YmAlb0lOvq3QQ1PNzRoV69cMfyICA/OBmP3wsOIfR4PN4+q6DetO9r0awFbZxuV2pLL5Ti76Qp2zj8IqUiKQbN6w/+bvmwD938YjQpUScSHPgACodxkmwHACEAdAJ1Lfv6aiELVN5kfn6JA8XWS0FPkYvPXu3Hz4D04VrfHlN/HoE3f5rxvMkUCIQ4sPo4Ta85DJpXDs6MHJv4yQu2oAoDyqf3itus4tvIsMpOUEQw4joNjdXv0/twHfab4VthISaFQICs5pyTiegqSolKQHp8JQU4hCrILUJBTCEFOYVnqC21iZGIIY3MjWNpZwK7EddyuahVUcbGBUw171GtVW2MRyfkQdicCB5efxIs7ESguFAEALG3N0bBdPTRoWx/NfDxRw1P1qOUKhQJ3TzzEnh+PID48CQBQr1VttO3fEu0GtkLV2vwfnHLScrH1u324vu8OjM2MMGBmTwz5rn+Ffk6MyoGmBaoXEZ1/T2f2AFyJSOvK8SkK1IfdzOX4avBVRF8+AKlIioC5AzHku368PekUCgWu77uDbXP2ISctD93GdMbon4eUa3G6ILcQp9dfwsm1FyDILkCj9vXRZ0o3uDesBueaDlpPyldK2psM3D3xEPdOPUL0k7i/bMA1MTeGYw17WNpaKGPTWZvBoorytTSlhYGRvjIrrpE+dPX1oKurA523CscpB1Gk/KcsJ5RMKodUIoO85FUqlkJUJEZxQTGEBcUQCpSZevOyBGXRKkqjgwOAjq4O6jSrgYbt6qNR+/po2K4eLKpof8RARGUj3Bf3IvHiTkRZdInP+rXA2MUBagWcJSIkRCTh7slHuH/6MV4FxwAAfEZ3xIRlw2HjaM27rbgXCdi36BhuH30ABzc7zFg/Hq16NVPZJsani9bXoDiO0wFgRkTaTV/6Nz5FgQL+eaPuw2ux+G3CFuQlxKCpdyPMWD9BpTn6iIevsWnWTkQEvUa9lrUwbe04tUdMpTe3yztv4vSGSyguFKFVr6YYOmcAGratp1abfPsVFYmQl6lMApifVYDop3G4eyIIr5/EAQBqeLrBs6MHXOsrs9ZWreus0cjrmkAmlSEnNRfx4UlKgbgbichH0ZCKpeA4Do061EenwZ/BpY4zzKxMYG5tBhsnK62LfVZKDq7sCsThFadQXCCC96gO6De1O2p6uavt0p+RmIWzGy/j2O9nYWBsgFELB6P3ZB+VriXsTgTWTNmC+PAktB/UClPXjFN5+pDxaaIVgeI47gCAyQDkUE71WQL4nYh+VddQVflUBervFBeJsGfhEZxYfQ6WdhaYsmosOg35jPcNNyMhE1u/24fAw/dh7WCJ8cuGw2dUR5W94+IjkvD0ehjC7kQg7HY4ctPzwXEcOg5ug6FzBmg8AnlhXhEiH0UjIugVIh+9RlxoAvIyBZCK/z9SRP3WtdFuQCu0HdDyk439JhFJ8Co4Bk+uhSHw8L2/ZAIuxdbFBk41HeBcQxnrr07zmmjq3Ujj61yC7AIcWn4Sp9ZfglQshZGpIfpO6YaA7wfCzMpUrTaTXqVg45c78fjSM1hUMUfPid4YNKsX75iPUokUx1aew75FR2FgZIDp68ajy7B2lerBg6F5tCVQz4jIi+O44QCaAfgOQAgReapvqmr8GwTq0cWnWDt1K9LjM9Frkg8mLB/O+wZRXCTC4V9O4ehvZwAAg7/pB//ZfVXeYJkSk4at3+3D3RMPAQB21arAs6MHPNt7oEnXRnCq4aDaRb2HjMQsnF53EUHnQ5AQkQxAuY7l5lEVtZpWh42jNSxtzWFhawFLW3NY2prDwV2Z9fXfROkINTc9HwW5hcqAtonZSIlNQ2pMOlJi0pGTqoz0Ze9qix7ju6L7+C4aH1XkZuQjNPAlHpwNxo0Dd2FuY4aRC/3R+3MftUZURITQW+E4ufY87p8OhpmVCcYtHY6eE7vyfmBKjk7FijEbEH4/Cq37NMOM9RPK5dTDqNxoS6BeQrlv6QCA9UR0i+O450TUWH1TVeNTFqjc9Dxs+moXbh68B9f6Lpj1x+do2K4+r7pEhBsH7mLbnH3ISs5Bp6FtMXH5cNi72qlkQ1F+EfYvPo5T6y5CV08Xg7/tB59RHeHoziOMhYpEBcfg+KqzuHXkAQCU5TKq37o26raoCVNL9Z7a/80UF4kQfOkZzm25iidXQ6Gjq4M2fZuj9+e+aOZT/uzGf+f1k1hs+WYPnt18iWp1nTHhlxFo04e/Y87fiQ9PxLrp2/E88CXqtayFmZsmoVaT6rzqyuVynFh9AbsXHIKOrg7GLR2GPlN8K4XHJEOzaEugvoBy1PQcQC8ArgD2EVF7dQ1VlU9RoIgIt47cx7rp21FcUIyA7wdiyHf9eafXToxKxurJWxB6Kxy1m9XA1NVjVV4Tinn+BsdXncO9U49QXCCC7+hOGLN4qMaezouLRAi/H4XngS8RdicCCRHJEGQXwMTcGD0neqP/jB5wcFNNTP/rJEen4sLW67iy6ybyMgUwtTRBtbrOcKnjhGp1XFC1rjOad2sMUwuTcvVDRAg6F4Kt3+5FYlQKani6odvYzug6vD0sbS3Uau/6/jv4Y/YeCLIE6D3ZF2MWDeW99y41Lh1rpmxFyJXnaN6tMRYcm808/f5laNqLrw2AIPrbCSUhjHSJSKa2pSryqQlUVkoO1k7digdnglGneU18u2sa3Dz4eVFJJVIc/uU0Diw5DkMTQ0xcMRLdx3VWaZ1JkFOAXT8cxvk/rsDY3BjtBrRCv+ndUbtpDXUvqYyMxCxc2HoNz26+QNSjaMikcujo6qBui5qo4emOmo3d0GV4+3LfQP/rSMRS3D0ehBf3opD0SuleX+ryb25tikFf9UH/GT3K/TnLpDJc2RWI81uv4VVwDPT0ddGmXwt0H9sFzXw9VR7JFOYVYef8gzi3+QosbC0wZdUYdB7altfojIhwfss1rJu2FXVb1sLic3PZvql/EZoWqM0AWgJ4BeASgEtElFZuK9XgUxEoIsLF7Tew5Zs9kIqlGLMoAANn9uSd8iH8QRRWTfoDb14motOQzzBl1RiVXHnlcjkubb+BHfMOojC3EH2mdMPon4doJIJEdmouDi07ifNbrkIuV6BO85po3NEDjTs3RMO2dVkEiQqguEiE6CdxOPLraQSdC9GoUAFAXFg8Lu+8iWv7biM/qwBONRwwZdUYtOmj0j0FABD9LA5rJm9B5KNoNO/WGF9snAin6vzWOO+deoQlQ1fB2tEKX/7xOVp081K5f0blQ1tTfPUA9ADQDUoPvptQCtY9IpKraatKfAoCVVxYjB8H/YYnV0Ph2dEDX22dzNsLrbiwGNvnHsCZjZdhW9UGMzdOVHmPSMjV59g+dz9eP4lDow71MX3teLU2ab6NILsAD84GI/R2OG4dvg+pRAbf0Z0w4ge/SjN1J5fJlbmbsgogKhJBLJRAJFRmzxULJZBJZZDL/pqcsCSQhDKShI4yooSOrk5Z9Ah9Q33oGypfTS2MYQ+54XkAACAASURBVGppoiwl6T8qQ0zBqOAY7Pv5aJlQdRnWHs27eaFxJ49yPyxIJVIEnQ3B7oWHER+ehBY9mmD80mEqe3fK5XKc3XgFO+YdgEKuwKRfR6HPFF9eo6mIh6/x27gNSIhIRq9JPvhi44RK8bkz1Kci9kEZQxlFogeANqp2pi6VXaDCg17ht3EbkfwqBdPWjkfvyT68v0yht8Px27iNSIvLQL9p3TF2SYBK3nlZydnY9NXusg2Q45cNV8l1/V2Ii8U4ueYCDv1yCkX5Qphbm6J13+YYPm/QR3H9FmQXIDY0HnFhCYgNjUdydCryMgTIzxSgIKcQqvwfLi8cx8HawRIO7nZwcLeHo7s9HN3tUK2eC+q3rl3heZGigmNwcNkJBF96BnGxBHr6umjYrh6a+XqhTZ9mvKeW34VMKsPJtRdxYMlxFOYVob1fa4z+cbDKbWYkZmH15C14fPEpuo/tjKlrxvISUYlIgl0/HMLRlWfRb1p3TFs7jrmif8JoTaA4jmsKoB2UEcjvEdET9UxUj8oqUBKRBLsXHsGxlWdQxcUGs7dPRVNvft73IqEYO74/gJNrL8CphgNm75gKzw4evPuWy+Q4veESdi84DKlEhuHzBsH/m768nTDe2aZcjmt7b2P3gsPITMpG6z7NMHKBP2o1qV6hT6/FhcV4eP4Jbh97gPAHr5Cd8mfCZUtbc1Sr5wJrRytY2VrAyt4SVvaWsLQ1h5GZEYxMDGFoYqCMJGFsAH1Dfejo6kBXTxe6ejrQ1dVBaSiJ0mgSpCDI5QrIpTJIxTJIxFLIJDJIRFIUFxSjMK8IRflCFOULUZhXhOyUXKTHZyItLgMZCVmQy5QTCUamhvDq0hDNfb3QorsXnGs6VthnJhFJ8OJeFEIuP0PwleeIDVUm+OwyrB3GLRlWrhFvYV4Rjv1+FifXXICoSIS+U7tjzOKhKk0rKhQK7Fl4BAeWnoCDmy2+/ONzNPPh5wi8+evdOL7qHMYuDsCw7weqexmMj4y2pvgWAPAHcKLkUH8AR4losVpWqkFlFKjXT2KxbMRaJEYmo+eErpj02yjeX9iIh6/xy6h1SH6din7TumP88uEqeSxFPnqN1ZO3IObZG7To7oXp68aX62Yol8vx8NwT7FpwCHFhCajboiYmrhiJxh0bqN0m775lcqTHZyI5Og1ZSdl4fPlZWfoOG0crNPFuhJqe7qju6YYanq6wdrCqVE/Rcpkc2Sk5iH76BsGXn+Hx5WdIi8sAADjXckRH/zao3awmbF1sYOtio1YaenXITs0ti/qgUBAGfNET/Wf0KFcoLEF2AXYvPIyzm67AxskK09aOR/uBrVRq48XdCPw+cTMSo1LQbUxnTFv74dGUQqHAitHrcX3/HczaMhk9J3RV+xoYHw9tCVQEgCZEJCr53RjAEyLit5lHA1Q2gSrKL8I4j1nQ0eHw9fapaO7L70lQJpXh4NKT2L/kOGxdbDB7x1R4dX5vMuG/UFxYjF0/HMapdRdg7WiFqavHov2g1mrfsLOSs3Fx2w1c3H4dmUnZcK7pgHFLh6ODn/ptvg8iwvPAl3hwJhhJr1OQ/DoNaXEZZSMQALCyV6bv6DTkMzRoW/eT2w9DREiOTkPw5WcIOheCp9dCoVD8+R3T0eHg3tAVnh084NmpATw71FfLrZsvmUnZ2Dn/IK7uuQWO49Cka0OMXODPey/eu3j7AclndEdMXztepWlpiUiCvT8fw5EVp9DE2xOLznz3walRqUSKH/r+gpArz9FnSjdMXDGCuaF/YmhLoC4CCCCivJLfraDcB9VbbUtVpDIJlLhYjJUTNiHw0H2sC1qKui1q8aoXGxqPX8duQPTTOHQZ1g7T1o5TyYX28eVnWDN5C9LjM9FnSjeMXzZMLc+t0nQN5zZfQdC5ECjkCjTzbYxek3zQpk8zraRez83Ix9Xdgbiw7TqSX6fCyMQQLnWc4FzLES61nOBS2wkutRxh62IDezfbT06U3kdBbiHS4zORlZSDrOQcZCZmIfLRa4Tff1UWBNe9YTW06tkU/aaXb4TzPlJi0nBt721c2HYN2Sm56Dq8PSb8MkLtPXEyqQz7Fh3DwaUn4OBuj293TVNZ9C7tvImV4zei64j2+HbX9A9OI4uLxdg5/xBOrD4Pp5oO+HbXdDT4rK5a9jMqHnUESjkH/44CYB2AtQBOAUgGsAvATgBJAA79Uz1tlGbNmlFlICo4msZ5zCRvzo/2/HSEVx2pREp7fz5K3Q2GkJ/DeLp78qFKfeZl5tPyUWvJm/OjsfVnUtidcHVMJyKiN+GJ9I33T+TN+ZGfw3jaNmcfpcSkqd3e+1AoFBR6O5wWD/2duhsMIW/Oj75sP5+u7r1FIqFIK31+SkjEEnpxL5L2LzlO3/r+TL56g6mb/hBaPmotRT+LI4VCoZV+hYXFtGPeAephOJT6mI+gQ7+cIrFIonZ7YXfCaUT1KeTN+dGaqVupML9Ipfr7Fh8jb86P5nRfRBmJWbzqPAt8QcPdp5Cvrj9tm7u/XPYzKg4AwaTivf99+6BGf0DYdqukhOWgMoygnge+xJxui2Blb8l7Wo+IsHTYagQevo9OQ9ti+tpxKk3nvLgbgcVDVyEvQ4Chc/pj2PcDeafjeBtBdgH2/nQUZzdfgZGpIcYsGopek7w16nFWXCRC1KNohD94hYggZcnPKoC5tSl8RnVCz4ldy+VR9m8n7U0GTqw+j4vbr0NUJIaVvSXqtaoF7xEd8Vm/5hr3DkyJScOmr3Yh6GwIHKvbY/zSYeg4WD3vz9Kp55NrL6BqXWesuLaA98iMiHB20xVs/XYvDIwNsPzyfF4byosEQmz+ajcu7bgB75Ed8N3uGSrbzahYWMp3LZEal47pLefC0s4Cq+8u4jU1l5ueh98nbUbQ2RCM/mkIRvzgx7s/kVCMnfMO4uTaC3Bwt8PC47NRy4tfbLO3kUqkOLXuEg4sOQ6hQIge47ti9KKhsLbnF3WaD4V5RTi28ixOrDlfljSvWj0XeLSug0Yd6qPj4M/KnZL8v4QgpwC3Dt9H5ONoPL0ehszEbFjZW6L72M7oOdFbowF9AeXU8fa5+8scbmasn6B2H89uvsCCfr/Axskav15fqNJ0ZWJUMuZ2X4LCvCIsvfA9PNrwm7r7Y/YenFh9DnP3z0SnIW3VsptRMWh0iq+0QJny/SmAHAACAAUABKoO1cpTPuYUn7BASJMaf039rUdT4qsUXnXunXpEfvbjqIdRAB37/SzJ5XLe/YXdjaDRdWaUTZkIC4Rq2R16O7xsOnJuzyUU9yJBrXb+CWGBkPYvOU79rUeTN+dHi4aspKDzIZSfLdBoP/9lZDIZBZ0PoR/6LSdfXX/y0fGnOd0X0bObLzTez4k156mP+QjqaRxAB5edIKlEqlZbL+5FUl+LkTSy5jRKj89QqW56fAaNrjODepsNp6c3wnjVyc8W0Iw2c8mb86Plo9aqPMXIqDigxhQfH4GKBuCJktHWxygfU6CWjVxDvrr+9OjSU17n75x/kLw5P/q8yWyVReHwilPko+NPI6pPoSfXQ9Uxl8QiCa2bvo28OT8a7j6Fgs6HqNXOP5GekEmbv95NfvbjyJvzo/l9ltHrp7Ea7YPx/2QkZtGeH4/QEJeJ5M350ewuC+nq3lskLCzWaB8/DvqVvDk/mtBoFkU/i1OrnYiHr6if1SjycxjP+3tTSnZqDk1oNIt6GgdQ2N0IXnVkUhntXniYfHX9aUSNqZSZnK2O2Qwtoy2BuglAR9WGNVk+hkDJpDLateAQeXN+9Mfs3bzqJEYlk6/eYFo2Yg1JxKot3F7de4u8OT/6yf83KhKoN2rKSMyi6a2VT5MbZu7Q6M2rMK+Qts3ZRz2NA6i7wRCa33cZvXwQpbH2GfwQF4vpyG9nyhwTepsNp+Wj1tLL+5Ea6+P+mcc02HkidTcYQvsWHVNrNBX3IoEmNJpFPjr+tHvhYZLJZLzr5mcJaJjbZOprMZKu77/Nu17Y3QjqaRxAPw76VWV7GdpHHYHi42beAsAiALcAiN+aGvxdpbnEclDRa1AZiVlYPmItwu5EwGdUR3yxcSKvdZQlw1Yj6Eww9sSsh7WDFa++igRCbPxyJ67sCkTDdvXwy5Uf1HKEeHojDEuHrYFYKMbsHVPRwa+Nym28C6lEinObr2LfomMQZBeg6/D2GLNoqFZySWkaqUSKgpxCFOb9GQWiKK8IUrEMeqVx90pejUwMYFu1CmxdbLTiaq9pFAoFXt6LwrW9t3Dr6AMU5QvROaAtJiwfoZGkf3mZ+djwxQ4EHr6Pml7umL1jqsrroCKhGGunbsXVPbfQ1McTc/d9wTvrbtqbDCwfuRYv70XBe2QHTF83nte2ikO/nML2ufux8PhstBug2iZihnbR1j6oKwAKAYQBUJQeJ6Kf1DFSHSpSoIKvPMfSYashFUvxxcaJ8BnZkVe9qMfRmN5qLobOGYDxS4fxqxMcg8WDVyIjIQsBcwdixAI/lW+OErEUW2bvwekNl1CtrjMWnvgGbvWrqtTGuyAi3D35CNu+24uUmHR4dWmISStGaiRlR3lRKBQozCtCfqYAeRkC5GUKIMgSQFgggiBLgITIZCREJCElJh0KueLDDb4Fx3Go4mwNezc72LvawrWuCyxszWFmZQozK2XAWBtHK9hVq1Lhcff+ieIiEY6sOI0jv54Gx3EY8m1/+H3dWyMR5u+efIi1U7dCkF2IgLkDMHKhv0phr4iUEf7Xz9gOS1tz/HjyW9RtXpNXXblMjgNLTmDfoqNwcLfHjye++WAAZJlUhmkt5yAvQ4Ad4atYgsxKhLYEKljVRkvqVQOwB4AjlMK2hYjWcBxnA+AwAHcAbwAMJqLcf2oHqDiBkoilGFl9KsxtzPDjyW9RtTa/wKi3jj7A7xM2wcDYANvDV/Hy8ivKL8LERl+D0+Hw/YEv1dpwWCQQ4seBv+LZjRfoP6MHxi0dppHd9dFP47Dpq10IvRUO9wbVMHHFSLTo7lXhIYaICOnxmcpAsaEJiA2LR1xoPFJi0v8SfeJtdPV04VLbEa71q8K1nguqONuUCUupyOgZ6EEmlUMulUMqkUEmkaG4UITMxCxkJGQhIzELGSWx9tLeZL6zHx0dDlVcbJTBYqvbo1pdFzT18UTtphUbt/Bt0t5kYMu3e3HnWBCs7CwwbN4g9J3ardyhlQQ5Bdg0axeu7b2NPpN9MX39eJWvMfppHBYOWAGO47D8yg+8v1sA8OJeJH72+w1V6zrj98CfP3h+eNArzPxsHobOGYBxSwIqVWis/zLaEqjlAG4Q0RUVjXEC4ERETziOMwcQAmUcvzEAcohoOcdxcwBYE9F372urogTq6t5bWDF6PZZdms9rn5NEJMEfs/fgzMbLqN+6NuYdnMU7KOeqSZtxaccNrLm/BPVa1lbZ1uzUXHzfcwniXybh6+1TeI/0PtTmznkHcWV3ICyqmGH0z0PRc0LXCokd9zZJr1JweVcgru29hazknLLjTjUcUMPTFVXrOMPG0RqWdhawsrcoCxZrbG4MI1NDjUaikIilKMorKgsYW5BbEiz2TUZZwNi0NxnITFQmE7S0NUdTH0+06NYEzXw9VcrnpSnCg15h57wDeHbzJWp6uePLzZPU+j/2NkSE7XP34/CK0/Dq3ADf7JwGe1fVAtCGB73C/F5LIZPK8cXGifAe0YF33cMrTmPbnH3Y9nLVB2cIiAgL+v+CoLMh8BndEV9s4DdFz9Au2nIzL4ByBFSMcriZAzgNwAdAFJTCBQBOAKI+VLcinCQUCgVNbvoNjfOYyWsXf3J0Kk1p9k2ZE4UqC8khV5+TN+dHW77Zo5atCZFJNKL6FOptNlxlL6l3IZVI6eCyE9TbbDh1NxhCf8zeTQW5heVuVxXyMvPpwrZrNLPdPPLm/MhX15/m9V5KZzdfoZcPotR2HKkocjPy6Pr+27R81FrycxhP3pxfmVv4vVOPKjx6hkKhoNvHHtBg54nko+NP66ZvK7cLtkKhoIvbr1Mf8xHUz2oUXdvH34GhlPT4DPqy/Xzy5vzol9HreP9dc9LzqLvBENo0ayev82UypWefj44/TfT8ihIik1S2laFZoA0vPk0UKKfzEgBYAMj723u5H6pfEQIVdiecvDk/OrPp8gfPVSgUNKr2dBpgM5run3msUj9RwdE02Hkijak7Q62b1ot7kTTYaQL52Y+jyMfRKtf/OykxaTSt5XfkzfnRgv6/UNJrfnu9ykuRQEhB54Jp45c7aaLnV+TN+ZWFczr0yynKSsmpEDu0gVwup9dPYmnXgkNlbuHd9IfQlObf0tppW+npjTCthTL6O4V5hbRu+jbl9oUaUykqWDP/Z0ofJC7tvKFy/bfdwn8evJJ3vcVDf6cehkPp1tH7vOs8vvyMBtmNpSEuE1mIrY+MRgUKgPt7KwIcgKof7AAwg3J6b2DJ77wECsAkAMEAgl1dXbX1mRGRMraXn/046mc1inIz8j54/uunseTN+dHF7dd596FQKOjs5ivUw3AoBbh+rvLeIblcTgeWniBfvcE0osZUig2LV6n+u7h56C71tRxJ/axGqfSlVxeZVEZX996iL9vPp276yvh8PYwC6Fufn+jA0hMU8fBVhd24KwqZVEYPLzyh7d/vp9ldf6Q+5iPIm/OjKc2/pZuH7pJMyt/9ujyE3Y2ggGqfUw/DoXTktzMquX2/C5lMRl91WkB9LUZS2hvVNuSWsvMH5Z5BvvutslJyyjblbvlmD+/P7lngC/Lm/Ojk2gtq2cnQDJoWqKMAjgMYBaABAHsArgC6QOl2fh+Az3sbB/QBXAbw1VvHKs0Un0KhoJNrL1A3/SE0tt4XFB/Bbxpgz09HyEfHn3LScnmdLywsLgv4Oqf7IsrLzFfJzuzUHPrWRxnkdfHQ36kwr3zTb8VFIlo5YRN5c370xWffU2pcerna+xClwjSmrjJCxvgGX9K2ufsp5Frof+6pVlwspnN/XCn7LEZUn0In117Q6J61fyIvM58W9P+FvDk/mtluXrlHyymxadTHfATN7rJQpWgppQhyCqif1SiV9i2JRRJaM2WLcrNy1x95PVASEc3q8AMNrTqJBZb9iGh8ig+AB4AlAAJLhOUpgAMARgAw+kBdDkovvtV/O/4rgDklP88BsOJDRmpLoEqf4Ob3Xcb7pv/iXiT1tRxJX7Sdx+t8uVxOU1t8Rz46/rTnpyMqf5EzErPI33E89TIZRue3Xiv3CCMhMqlsA+W2ufvVDmnDh5y0XDq47ASNqjWNvDk/muT1Nd09+VCtm9m/DblcTndPPqQv2iqnygbZjaVdCw5pfWpToVDQlT2B1M9qFPUyGVbuSCMXtl0jb86P1k3fptZocPfCw+TN+dHzWy9Vqnd5103qaRxAw92n8FrHCr7yTOVZD4ZmqVRrUPgzRXwogGclpSeAKgCuA3hd8mrzoba0IVA56XnU0ziAFg1ZyfuGef/MY+ppHECj68zg/fT57KZyeuHcH1fUsnPlhE3Uw3Co2mFnSild4O5tOpwGVBmjEeeKf0IkFNH+Jcept9lw8ub8aFbHH+jOiSAmTP9A2N0Imt9nGfno+FM3/SG0dPhqCg96pdU+M5OyaEqzb6iXyTB6cU/9KBQKhYI2zNxRFn6J74imFEFOAQ1zm0y9TIbRzUN3Var78MIT8ub8KPDIh6enFQoF+TuOpxVj16vUB0NzVCqB0mTRhkDtmHeAfHT8eXv3XNh2jXz1BtO0lt+p9CVcOWET9TEfQcVFqk9lhQe9Il+9wbT+i+0q132bgtxCWjRkZdlNJDOJX94dVVEoFBR45D4Nd1eG4Vk4cAXznlKBpNcptPHLndTXciR5c340vfVcCr2tfv6vD5GTnlfm7BMb+qZcbV3edZN6GAXQMLfJ9CokRjU70nLLPPu2zd3Pe31MJpPRQNuxtGzEGl7nz+m+iPwdx2tk/ZahOkygeFIkEFJ/69H0kx+/ue+jK8+UrR+pEl1cWCCkflajaPmotSrbeHXvLephpJzC4LvW9S5inr9RJnfTG0wHlp4o9+L4P/H6SSzN6vCDciqv8de8o1Ez/p8igZBOrb9IAa6fkzfnR8tGrlF5ZMKXlNg0Guw8kXqbDacbB1UbwfydqOBoCnD9nHoaB9DtYw9UqisRS2jVpM1lU+58p55/GbOO+luP5nV+9LM4Guw8kfpajKSQa+oFY2aoDxMonjy+rJyPDrn6/IPnioQi6m06nOb1XqrSeo0gp4C+6rSAvDk/ehaoWnqEuycfkq+uP83uslBlh4q3KS4S0eg6M2iIy0StBXYVCUW05Zs95KvrT3724+jcH1e0JoL/Nd7OfjvQdixdP3BHK16OmUlZZQ8XZzZeKldbuRl5NKX5tzTYeaLKa1IKhYKO/KZ8GOTrVXrz0F2V7M5IzKIJDWfRAJvRKqcDYZQPrQgUgOt8jmmzaFqgzmy6TN6cH68U00HnQ8ib81NpzSY1Lp3Gecyk7gZDVIrGTKTcj9XTOICmt55bLs+unLRcmtVRedNRN3XHh3h+6yWNqj2dvDk/+n3ipgrf3PtfIe5FAk1vNYe8OT/6od9yraSTEBeLaX7fZeTN+dGRX0+Xq627Jx+SN+en8h5BIuW0XUC1z2luzyW8zhcJRfR154XkzfnR4RWneAl40usU6muhdHTSppMQ469o2s3cCIANgOcArEt+tinZdBuhakflKZoWqE1f7aIehkN5LdqvnvwH9TYbzts99VVIDA12mkD9rUernFjuzcsE6m89msbUnVGukVPEw1c0tOok6mUyTGWB5IOwQEhrpm5VuknXmKo1Afwvs349Ufpb3v8ymYyOrjxDPYwDqIfpaLWiOHwIqURKi4f+Tt6cH+1eeFjt0ZpUIiV/x/E0o81ctaJX7Jx/kHx1/Xk9QBIpXc9L11g3zNzB63t9/cCdsjUvRsWgaYGaCSAOyhQbcW+V5wCmq9pReYomBer1k1jqazmSpreaw+v80XVm0A/9lvM6VyaT0TC3yTTMbTK9eal6BtuvOy8kP4fxlBKbpnLdUlLj0qmv5UgaUWOqVhIJpidk0uSm35CPjr/Gc05pm7/f9EtJT1e+V1lYv175zfTw+Ku96elEjWomU3MoXdN3zj+o8Sk/mUxGv47dQN6cX7lc0K8fuEPd9IfQF599r3JW6OToVPLm/OjEmvO868jlctr45U6lK/kOftEtfh27gXx0/Ck79dONWvIpoa0pvhmqNqrpoimBin4WRwOqjKEA1895iUBuRh756g2m7d/ze8oq3WvBx+317whyCshXbzDtmHdA5bqlyGQy+rL9fOprOVIrm2+DzgXTgCpjqI/5CHp44YnG29cm77vpe3go36ssIvW2TaX2/uVYfRktHqHcaL3xy50aFympREojqk+hCY1mUX6WQO12bh8PIl9df/rG+ycSF4tVqhtQ7XOa5PU15Wfz71+hUNAkr69pnMdMXqOolw+iyJvzo9vHg1SyjaEe2hKoUe8qqnZUnqIJgYoNfUMDbcdSQLXPKTk69YPnZ6fm0ISGytTTfGPeLR2+mgbYjFb5y0j052Jvefak7Ft8jLw5P7q695babbwLqURKW77ZU7bZNjEqWaPtVwQfvOl7vHt09bF42zY7O2V528639x+tnLBJ444pQedDqIfhUBpTdwalxKg/or+yO7Bs7UyV9Z6gc8HUw3AoTfT8SiUv1uv7b/Ne/xKLJNTDKIA2f80vYzajfGhLoNa9VbYCiAVwTNWOylPKK1DCAiH5O46nIS4TeW2wFYskNNHzK+ptOpy3u3RxkYh6GgfQmilbVLavILeQJjScRYPsxqp9o0mITKJu+kNoccAqjT5Ry+VymttzCXlzfrR68h9qiW9l4UM3/cpGevqfNpba/LadCoWCdsw7QN6cHy0dvpokYs2G8Qm7E04Dqowhf8fx5YpwcXrDJbWi94dcfU69TYfThEazeF+bTCqj4e5TaHbXH3md/2X7+byjwjDKhzoC9cGsY0Q0460yEUATAKrnJP+IPDgbgtz0fMzZ+wVcan04UdrRX88gLiwB8w/Pglfnhrz6SIpKgUQkhVcXfueXIhKKMb/PMiS9SsF3e79QO5fRxW3XQUSY8vtojSZoO7byLB5ffIppa8Zh5qZJaqWj/1gQyUDyDJA0EqeOPIABnceDG3ux8JttGNxnD/r5HsbkMaew+IcLsLO8BZLFg0iGDRuAjIz/by8jA9iwoeKv45/gOA5jFwdg7OIA3DhwF3O6LYYgp0Bj7TdsVx8rA39CUb4Qqyf/UfrAqjJ9p3ZDe7/WuLI7EHL5uxNNvoum3p6Ye2Am3rxIxNlN/NLR6erponXvZnj1OIaXvU41HZCZmMXbJkbFolp+cSVCAOXLflbBBB6+B1sXG3h29PjguSkxaTiw9Dg6Dm6DVr2a8e4j+XUqAMBFhUyhUokUP/uvRPj9V5h/eBZadPPiXfdtru65heOrzqG9X2uNJsiLCo7BzvkH0W5gK/Sb3l1j7WoLUuQBkqcgaQggeQJIw6D08QH6doAy8BaABV+9o25JTme5XB8dGrrhaWANtO1QA6bWrQCDFsjMNEDnzkB4uPK8adO0ey0ZGUDnzkBmJmBXkhcwM1N57OZNwN7+z3OHfT8QDm52WDl+I2Z+Ng+Lzs5VKWPt+6je0BVjFwfgj9l7cG3fbbUTY3Ya/BnuHAvCizuRaNypAe96bfo0R1MfT+z7+Si8R3bgla3atX5VCAuKkZWcA7uqVd57rpWtBQRZmhN1hmb5oEBxHHcWZV9t6AKoD+CINo3SJIV5RXh88Sn6Te/BK031hpk7oKevh8m/j+Hdh1wux4Xt16GnrwvnWo686636/A88vvgUs/74HB382vCu9zbX99/Br2M3wKtLQ8zeobm75ot7kZjfexmsHa0w64/PK2XabCI5jh8KRffON2CidwOQvS55Rw9SeCD8dQC8mrkhv8AGM2fZT0JJ7gAAIABJREFUIDjEBoJCG+TmGcHJUQKxWAx9PQnMzSQ4cVyA6tXeQCyIQXpWLKo5v4a+7AYodzMUMMWTW+3RoUUr1HRrCX+/msCHJx/UplScwsMBDw+lIAF/HnuXSHUd3h4ObrZYOOBXfNHme6y4tgC1vKprxJ4BM3vi7smH2DhzJ5r7Noa1g5XKbbTo0QSGxga4feyBSgLFcRwm/zYKk5t8gwOLj/P6XrrWdwEAJEQkfVCgLGwtIC6WoLhIBGNTI952MSqID80BAuj4VmkLHjmgNF3KswYVci2Ud9SIvMx88ub8aNeCQyr1UeqccHId/3wzKTFp5cqqS6TMj9PXYiTN6vCDRtNWxEck0QCb0TS6zgy1c/1omlIXcYVCRIriyyTPm0MFca1JnlqbxIn1qDh1JCkKNpJC/JDS04R/8czz9yc6dYrI0FB5jOOIrK2VP+vo/LkO9eKF8vxly4jq1CEyNhbSML/rtH3VPIoPaUfy1NrKktGVFIV7KD2tUCuef+XxOEyOTiV/x/G812D4Ehv6ptyRJuZ0X0SfN5mtVt2f/H6loVUn8To37U0GeXN+dGHbtQ+ee+dEEHlzfnRg6Qm17GLwB9oKdQTAEUBfAH0AOKraSXlLeQTq/Jar5M358brRlu6AD7vDP0Dn81svyVfXn5YOX62Sc0JpTqn0hEzedf7OkmGrqIdRgEaz4Oak5dKIGlOV+7HK4b2lSdavJ9LXF9PCbw+QJLVEKNKaUc6bWTTC/yxZWeaRo+O7PfP69FH+rKf3pziVOh2UFjMz5auj45/H6tYlMjX983ddXQV1ah9P4wKOUMJzf5Kn1qbcV81o2bwVtHO75rMQl2fPVmnsyPJ4hP4dhUJBw9wm845f+S42zdpJvUyGqZWW4+TaC7yjvwgLhOTN+dGhX0598FyFQkGLhqwkX11/enz5mcp2MfijFYECMAHKdO27AOwG8AbAOFU7Kk8pj0Btm7OPuhsM4eUdt2nWTuphFMA7akRuRh4NcZlIo+vM4JWTphSFQkGj68yg2V0W8q7zd0pHhrsXHla7jb9TXCSi6a3mUC+TYRT56LXG2i0PCoWc8tKOU/yTziRP/R975x0W1fH18e+l9w6CYu+ILfbekGhsQReMXey9xNiiRo0tRo0aW+y9i723KIodFQEVQXrvfdl63j8ui0jQvXd3Ufy9+3meeYRlZu6wsvfcc+ac79Smp9cHUnriXUpKFBcZIoVnVDwzz8GBaMECort3P/y8tKanR2Rl9eH7mjWJatT4dP+qVdk+rZs9p2Pbp5E4ti6J4+rRm8ezSC7VvKFShfxcIQ2w96Z5PZZpdN413lvIw3akysem+J17Qm6MgLbN3Mt77NsnobyO1hhUZTwNrz2FstNzlPbPzxXS2EY/k4fNiHLzUPa/iCoGiksgfTaApkQ0kohGAGgGYK6GIoxlTnJMKuwr23HKjnvnH47azWrAwFCf09w+f11ERlIWFh6fCRNzY85rin0Xj7jQBHQe2I7zmJIcX30GDlXsMHBuP5XnKMmFrdfw9kkYfj0yA3Vb1NLYvKpCkhBQ+iCY0zxUcLLChHm70ML9KOo27AjXhvp4/RpwdAQmTgRsbdkkgpQUdmxyMrBiBeDpCVy/Duh/4r9UKgUyMz/+XiL59JqiooD374FH/k0xZNJG1G59Ext3DIez/XUIY/vg9NGrRVmAXyvjz9jUCF6z++HZtQBc2nFDY/M26eqKnPRcPL3yQqXxbfu2QI1OPeGz4RJe3A786GfK3q8ajatCV08X719GKL0OwzD49cgMJEWmYPe8w0r7G5saYcnp2ZDJ5Niz4IjS/lq+HFwMVCyA4mkuOQBiymY5mic3Mw9m1qac+qbEpsGxmj2nvkSEe6cfo0lXV96b0dFv4gAAtZqqtoktLhAj6P5btPdoBUNjQ5XmKAkR4fr+O3BpUwdt+7XQyJzqrIXyDoDS+gOyKDCWq2FQwQfL/+wIe3umyBCZmQGJicCGDR8bmeK5MMnJQI8erOHhQlQUEMPxr1smA6JinTF76Xx08riIkPfV8GOXadATLkTLFkJMmfL1jNSAn3ujRY8m2Dx1N4L83mpkzvb9W6FyvUr4a+w/yEjO4j1+yxZg152hkOo7YNPUfZBJ2ZRzRVLI594vfQN9WFewRHpCZukdSuDarh76THDHld23EBEUrbR/xZqO6Da4Ax6ce4q87HzOv5OWsoWLgYoD8JhhmCUMwywG8AhAGMMwPzMMU0rCbvkiLysfppYmSvsREdLiM2BX0YbTvFGvYxEXmoD2Hq14rynmLWugnOtW5D0WAIIfhEBcIEHTbg1VGl8a7/zDERkcA/cRnTU2pyqQPAuUORmUsxwwbAfG7jIYY49SswgNDYGqVdmvFeU1OjqAXP5xP6GQDdCVJc8DqqDl90fxx6ZxGD34JC4e7I+G9d8i5ytlMOvq6mL+4elwqGqP3wVrkRKbpvacRiaGWHhsJnIy8rDGewvkJd9oJXh6AvVcDPBaPAwxb6Jx/K+b/8lY9PT89HhrRyukJ3EzUAAw9DcBjM2NsXPuIU793YZ1hLhAgns+jzlfQ0vZwsVAvQdwFh9Szc8BSABgXtjKNXlZ+TC1UB5+y8nIhUQkgY0TtzqiRxeeAYBK3kbsuwTYOFnD1EK54SyNgDvB0NHVQcOO9VUaXxr/HrkHfUN9dPJqq7E5+ULyHFCaFyC6C8Z8Hhir7WB02AeG5GSgceMPdUH29kBaGhAb+/EcPO+ZGkUq1ceClb+gx097YG2ZjcdXBBjv7ffV1mNubYbfz85BQZ4Ia0dv1cicNRpVxYR1I/D0ygvcPOjLa6yDA5seb1e/FTLIBTvnH4drA9lH6fTFU+dLYuNohfSEDM7Xs7SzwJCFAjy98gKvH4Yo7V+/dR1UrOWIuye+3v+Zlo/hoiSx9HPtSyxSHcQFEhgYK1c/yMti3Xqu4cCIoGhUqGoPW44GrTiZKVmwdeJfS6IgLiwRFaraq2zgSkJEeHE7CA3a1oGZFbffX9MQyUFZcwBZDBjr3WBMRxV5TQrjlJgIGBgA48axNzNz8w+eU3ni1r126D7wPHQNq8MSk0DiZ19tLVVdKmPgnB/x/MYrJEQkaWTOPhPdUbluRVzb9y/vsQ4OwL93GGRZ9IAe5UCcGgZ7e+XGCQDMrEyRny3kdb0fxnSFjg6Dp1dfKu3LMAwadXRB2ItIXtfQUnYoNVAMw9RhGGYHwzDXGYa5rWhfYnGaQCKSQJ9D0kNBbgEAwNiMW7FeXGgCKtVRrVo/Oy0H5jZmKo0FgOToVFSoaqfy+JL8e/Q+wl9FoZOX6kkbapO3AxDdAmM+F4xhawAoSjaYMoU1TgAgFrPJD126AJs2fb3lKuNNiC0OXdwL6DiCMsaCJIHKB5URbsM6AgBvj+dTMAyDLoPaI9D3DZJVlAnK1WeLdW0QxHmMkakRhDn8DJSppSlqN6uBl/9yu05VF2dkJmchKzWb13W0lA1cQnwnAbwAsBBsRp+ifRNIxVLoGyg3UEIeBoqIEPsugZOuX2nkpOeqZ6CiUmBfRTMGSpgrxD+z9qNui5roOaarRubkC4l8QbkbAKNegMlwAKxxmjKF9ZxOnmSfrhX7TQAb6hs58qsslxMMA7RqYwfGZj+gYw1KHwWSKt+sLwsqVLVHky4NcPPgXUXpiNp0HdweRIS7xx/wGqfYc0pMs0C+bnVU0H9VJOFUmv5hcWo2rorMlGzOxkZBky6uePs4FMK8AqV9qzaoDACIDP5m8sD+p+FioKREtI2InhCRv6KV+co0gFwuR0GeCPoGyiUHRUIxAHASQxUJxcjLyodD5c/LqJSGuECM1Nh0WDuoFuIjImQmZ8FGBbmZ0lAI6Y5dPUxloVp1IBKDspcCejXBWKwoCut5erL7EomJbDJEcjJ70/9Uunh5gwgYMABISXMEY70PREIEPt7z1dbTpk8LxL9PQqYK2XelUamWE5zrOOH1o3ecx5RMiOg9pBYcLWLg4vJBwulzRup77y4wtTTBrUP8PME6zWtCKpEhPixRaV+n6mycMTU2ndc1tJQNnzRQDMPYMAxjA+ACwzCTGIZxUrxW+Hq55/XDdxDmFqB+a+XatlIJu5mhx8GY5WbkAgDMrPl7QU+uvEBBvgit+3AXoi1OXFgipBIZ7JRojHHF7+wT2DhaaTTh4nP8Ryk8/yAgi0GW7Fds3fZhT02xoe7oCIhYvVdERn6+Rqm8ERIC7NkDpKRXwZkrfVHd6Qx27tCMgeBLNVfNewZVXZwR/SZWecdCTp78WF+wWl175Kbn4MrFgiIjdfLkp8cbGBmg5Q9N8eiiPy9VdPvKbLQhOVp5ONLCls370ob4ygef86D8ATwDMAJsSO9B4WuK18s9930eQd9AD616KzcGMh4GKicjDwBgzjGhojh3jvvByt6C8zEeJbl9+B4YhkF7j5YqjS+OSCjCk8vP0bZfC05CuuqiCNspnpRJngHK3QYxOqJ913b/qYM5eZL1oL6CY6c2BsUc8S5dgN/XDIepiRA/9fvMHbgMqeLiDIAtj9AUletWQnxYIqQSbkVmkycDmzd/SIgo2kcVpeHff9mfKVOJb9OnBTJTshHyJIzzOu0LIx0pMcpT7U2tTMAwDLLTtArn5YFP3pWIqDoR1Sj8t2SroWxihmH2MAyTzDBMULHXljAME8cwzMvC9oOmfpHSeOX7Gg071ueU7ab4kOnqKb9RKzZqTTikr5ck4N9gtOrVDLp6qt11/W8EoH6bOrCrpL4H9f5lJAryRGjRs6nac3FBEbZThHOyU64DlI3B437+Tx3Mli1AQgJQs2b5zNT7HAYGbDKHmRnw11/s7ytj6kGCJjDV05yyAx9snaxhZGKIxHDNZPIBgFNNR0glMqTFc0/9njz5Q7aebWHNYVp8OhwcuB1h0sy9EQAg6D734mMbRyswDMMpRV1XVxcmFsbIy9QW65YHuBy30b+Ul7MABBLR57Y19wHYDOBAidfXE9FazitUg9S4dNRqqtSWAgBIzm4ec/EkJGLWmHHxtoojFkmQmZINpxoVeI0rTnJ0Kpq6aaZAN62wKt9BQwkXylCE7RT7EEcPhmHoAGOcvVgPLi7AsGFsP4WnBXysCvGtIBazXl9uLtsUadT6hrUA0d2vsiaGYWBiaYL8HOWJAlxR1BcqEoz4YmjCupniAu5xWwsbcxiZGCIjiXuoVEdHBwZG+hAXiDn11zfU5+wVailbuHz8RwPYBWBIYdsJ4GcAfgzDDPvUICLyBfDVdhplUhkyk7NhW5FbnZIiu4nRUX7ukUzKVoPq6fMzUIonOK5r+u91ZUhPyICDs2YMSkYia6BsHDWTcFEaJfecFEbK1hao5vwer9/VhJ2dDmrVAubPZ41X586s5wR83cJbdSjN62N0nQF5Cog0ZyT4YGxmxCmTjStGhRmvqhooRUKSWMjNcCiwcrBARjJ3RQn2WvqcDaGevi6kYq2BKg9wMVByAPWJaAARDQDgAvaY0lZQTTR2CsMwrwpDgJo7/rUE2Wk5ICJY2ltw6q/IvuVyMJ9CQ0xHl9/jveKpz8rBktc4BZkp2ZDLibPahTIUG8GKjWFNU3LPSUFKCqudV6NqNMIjK0MoBM6fZ7P1Xr8GvLyAWl9fq5YXFhb/3StTKF4o0qiz8woPs5QpyacuI4xMDVGgSQNlwupAivJFKo03MGJTMvl4UABgbmvO+xRcfSMDfgZK+o3Flf9H4XKHrUZExQPXyQDqEFE6AL45VdsA1ATQBKxc0rpPdWQYZhzDMM8YhnmWopCo5oFCf49r5bnCLnGpE9Ep9LKI5+O9mRW7JoVqBV8U43MKswjVRbE3l8+z+JErJfeckpOB4GCgWTPWw0hKsYNzxVTk5rLGSST6YKSuXSuTJZUJDANkZ3/sNTk6AkFBbFO8B/9sK9wD0Sk7j/VzSEQSTmUUXBGL2I8/l0L40uCTOVscYY6Q9/6vTCLlfB2ZVP5VSi60/BcuBuoewzAXGYYZwTDMCLBafL4Mw5gC4OVnE1ESEcmISA42VPjJVDQi2kFEzYmoub09N4Xx4hgYGcDMypSzdpfCc1LsRX0ORWhP8QHjSvFNYVUwNDaEpZ05p2wkLlhVYD05PvF8PijCeYobtIsLW3irMEQNGtVA6xbv4eLy4TWR6MPDwrdCyWcaBwcgIID9t/h7YKwfB5HYHIwON69e0whzNXuseUEe6zkZmaqmqP+h9pCfgctMzoaVPb8ohLhAwvk6UokUevpaA1Ue4GKgJoNNeGgCoCnYpIfJRJRHRF34XIxhmOLSCx4AD50TFbBxskIqR2Og2HviotCsW/jHy9dAmZgbw8jUUC1laTtnWyRH8/coS8O60EDxEeDki+IGrRB3lcnYUJi/P2BtXxM6SMed2ylFRsrEpOyVxzWFRQk7o3jonjnzY105xXvQ54cYGJpU+nILLIEwp0BlY1IaCnkwVedU7D0ZctDKLBojkiA3M49XmJyIIBKKuRsosbToM67l68JFLJaI6BQRzSSiGYVfK72FMAxzFMBDAHUZhollGGY0gD8ZhglkGOYVgC4AZqr9G3yG2t/VQKDvm6I9o8+h2E+Sy5QbKMUHUpXN4drNauD5zVcqS87Ua1ELgffeaGSzu5prFejoMHh197Xac/HByoo1WDDoCEAHtsa78O+/wIIFH4pyvwUKiv0X6OqyxtfRERg16r997W3TUNXpAWCgWoG2uuRl5SE3M6/Ii9cECh0+VfdEM1P474Eq0strNKqqpOcHcjPzIJfJOV2HiJCfU8DrAFItZQcXsdgchmGyC1sBwzAyhmGUllkT0SAiciIifSJyJqLdRDSMiBoSUSMi6ktECZr5NUqnnUcrZKfl4JWv8huwQg6Ji1dkXqggkZeZx3tNnb3aIfpNHCI5HKJWGl0Gt0dBnggPz6tfK23tYIkG7erB7+wTtef6FAp5m5LHZHTpAqRk1AaM+wP5h5CZFo21a9mb/LeQWs4wbDq5gcGHWi1HR7awuFTJHuExABIwJp9MfC1TogoPyazawFljc8aExMGhip3KYcOUQmUHPmUOjy48g76hPq9SC0VIXKEo8Tnyc4SQSWVlljikhR9cPChzIrIobEYABoCtbyr3tOjRBPoGenh2LUBpX93CfSUZBwP1IVmBv4HqMKAVdHQY3D35kPdYAGjYoT4YhuElMfM52v3YEhGB0YgL0/yzguKYDMX+U8mkgcaNgfSC6SDoIcR/DUQigqEh8OoV602VR0NlbMwaWSJAT481UmPGsCoIAQEoVbKHSATKPwoYdACjx60uT9NEFypIVHXRoIF6G4/K9VQPWSZHp8DIxJCzcDIR4dFFfzTt5srLKKYUenr2HLQzFQoS5loDVS7gfQsgorMAvo7sNU+MTAxhammCfA5HOBsYsgaKSzGfiYUJ9A31kZHIf+/GuoIVHKraIzFCtVRjHR0dmNuYITNZM1phHQa0gq6eLo6sPK2R+YqjOCbD0BA4ceJD0sCJE+xriYnApKkV8CRoAnq5XcO86Yfwyy/s2DNn2BoohZEqL4kTQiG799SnD/DyJWuY5s37oJBQUrKHSALKnAHIk8GYjvlq6355JwhmVqZwrK7k0CWO5GTkIuJVFGryCLWVJCIoGhVrOXIq7QCAZ9deIiE8CR0GtOF5HVZ/sGJN5QXyaXHsnnVZ1gZq4Q6XEF//Yk3AMMwf+HC6brlHz0CPU9GdouhQkZn0ORiGQcWaFRDHQR25NMxtzNRKFXeoYoeUWNXO4fnvXPbwnNUH1/fdQeC9NxqZU8HmzR/EXr28WI8qOZn9WiRif7Z5M9C623hExnfF73NW4fa152jb9mNRUTOz8pE4oUiCeP+eFYK1t/+vPE9xyR4iGXsIo+gWGIvFYAz53Vg1RX6OEH6nn6CTV1uNpU/fP/0YErEUHVU8gVlcIEbQ/bdo3LkB5zGPLz2Hibkxug1pz+taAXeCUKV+JU6Zf9GFodAq9b9eMouWD3DxoPoUa98DyAHQrywXpUn0DfWLpIk+h+IcKK41QZVqOyEuVLWwmLmNGXLS1TNQyVGaMVAAMGSRADZO1vDZcFFjcwIf0q0VYS9XV7YpjI8iFZthdFC9yZ+ATkWc3D0NxoYp0NMDtm0DHjxg5YK+JsbG7Dplsg8isO/efV55m0gOyl4EFFwCYz4HjMmQL7PYUrh/+jEK8kXoPryTxua8ffQ+KtV2Qp1mqoUsXz98B3GBBE27cd9Lys8VwtzGjNP5bgqkEmmhIeQmzhwZHAMjE8MvJv+l5fNw2YPyLtbGEtEKJRp85QquGlyKrB2uhb3OdSoiPiwRBSpU0VvYmqsVonOs5oDEiGSN6YUZmRiiaTdXBN9/yynjkQ/F08xTUj4kS5Q84pvRsYCe7WZUsM+G77nBaFg/GF27AmsLVRuLP/h/qb0pReRJKATGjmWNqlgM9O37eeVtkqeDMicCwlOA6eSvGtrLzxHi2OqzqFjLES5t6mhkzuAHIQj4NxhdB7XnHJ4ryZ1jftDV00WjTi6cx+RnCzmfeK3gyeUXKMgToWlXbgbqnf97VHFx/iLq/lqUwyXE58wwzJlCZfIkhmF8GIbR3E5rGWNsbsxJINPCzhw6OkyRPp0ymrk3hkQsxbNrL3mvqVJNRyRFpRRV4vPFpW1dFOSL8O7Ze5XGl0bHAW2QmZKt0u+jKRj9etC13QN7+wL4XfDC+OEHkZZGYJgPKg12dmwShYNmtlJKpXNn1gglJgJ167KvOTl92F86d+4zxkn0CJTaFxDdB2O+EIzZtLJbqBLkcjn+HLEJcaEJmPHPOJWNSXFyMnKxcvAGOFZ3wICfe6s0R0RgFK7svoU+E9w5nTSgICMpq6i4nAsyqQy75h9GpdpOaNO3udL+iZHJCPYLQdt+LThfQ0vZwuUxYS+A8wAqAqgE4ELha98EJhbGnKSFdHV1Ye1oxVnloVHH+jC3MVMpRdu5bkXIZXIkvFdtD6tJFzZu/+K25uqcW/RsAks7c1w/oFm17dLSzD93xPfWnc1Ru9U53LzXDptWLsP5A+NRuVIcrK1Zo5SaCixd+uGEXXX4lJFLTgZ27mR/7uv7wVv63JEQRFLIczaAMkYAjCkY25NgTIdrxCioypEVp+F39inGrxmOpl3VV8AnIqwf9w/S4jPw65HpvIxL8Tm2/bwfppYmGLbEk9fYjMRMXskLl3fdQszbOIxdPZSTsPPtI/cBsMfZaykfcDFQ9kS0l4ikhW0fAP7aQ18JU0sTzvVKthVtkBLHzUDp6euhdZ9meHj+GbLT+QlXKlJzI4NVSxW3tLNAjcZV8fiSv8oFvyXRN9BHl0Ht8fDcU8SraDhLUvKI75Jp5qUZqc6dgdw8G/Qdth3TFy5Ep7ZPEHT3B8wctx5iURZcXIDFi1ljoe6vrrg2wwCmhWdPKrQAFWtTdk4REYFEfqD0n4C8rYBxfzC2p8Hocw9dlQVPr73EgSUn0G1oB3hM18yxa89vvsI9n8cYuewn1Gup/JTq0vj3mB9e3ArE8CUDYWHDPZWbiJCekAFrjgoSGUmZOLD4OBp2qM/JIyrIF+Ha3ttwbV8PTtVVPw5Hi2bhYqBSGYYZyjCMbmEbCkAzYnBfAFsna6TGpXO6kVdzrYxQ/3BOckcA0H9aL4jyRVg3ehsvQ1G9YRWYW5vi4fmnnMeUpOfobnjzKBT3fB6pPEdJ+s/oBWNzYyzotVIjJ4qWPOK7pDZdyXqh4hl+DMNg8+7hcO10GRevd8GCGdvw/nFXXDy+Fdeu5JbqfRVHn4e82507QHg4PtIEVHb8OFvb5ANK6wvK8AZkCWAs/4KO5SowOvxPWtYkRIRd8w7BqWYFzNw+XmNenP/1AOgb6OHHqT1VGh8RGIW/xmyDS9u66D2hO6+xcWGJEAnFqFKf2+7CyiEbIcwtwJRNo5X+/kSEDeO3IyE8GYMXDOC1Li1lDBF9tgGoAjbElwJWyfwsgKrKxmmyNWvWjFTFZ8NFcmMElJGcqbTvld23yI0RUGRwNPf517Pzn954ide61o/fTr1Nh1B+Tj6vcQqkEimNb/oL/eQ8TuU5SiPw3mvqafgTzey4iEQFYrXn27yZKCnpv68nJbE/K9kXIDI0ZP/V1WX/BYga1n9DZ/ZOIFlCbcoMa0LHd0yloYIzZGOdXtSnZLO3L/31km3Vqg9rcnFhX/P0LP33kUsTSJ7zN8mSWpMsoTbJUnqTPO8kyeUFar9XmuLB+afkxgjo2r5/NTrvxGazaVaXxSqP37voKLnrelJ6kvLPYkmuH7hDboyAwgOjlPYVi8Tkxghoz4IjnOZWfIYPLT/Fe11auAPgGfG893/Wg2IYRhfAAGJlieyJyIGIfiSiqDK2mxqjYk32DJ7498qPunbtUB8A8MqXez2Qx/Qf0LpPM+yccxAhT8M4j3Mb2gEF+SL4nlLNA9LV08W0LWOQGpeOfYuOqzRHabi2r4/Z+6Yg8N4brBm5We2svuJHfBentNDZ5MnsER0iEVCnDqvZpyA2sR7Gz92G1j1P4tiZXmjb4jn2b5qDxMA2+PfMYMyevAONXN5AR+fDem1sgO+///gaNWuyKhW2pYgKFC+0PXGCfY3keaCC65BnLYQ8xR2U0hGUuwnQdwVjvReM7XkwJgIwjOZEWNVBLJLg4O8n4VjdQaN7KdlpOQh7EYkmXbhlw5VGTEg8nGo6cg7TFefNw3cwMTfmVJ+UncbWJdg5K1eOeOf/HttnH0A7j5YYNN+D97q0lC2fNVBEJMM3VPNUGtUbVgEABHEoQq1UyxGV61bE6Q0XOWfYMQyD2Xsmw8bJGr/1W835hu7Sti5qNK6KrTP2IuxFBKcx/5mjTV30mfg9Tm+8hMPLfVSaozS6/NQOY/4YijvHH2DJgDWQlXY8bBlx4gSwahW7L5SW9iGxIiMDsLYG+g9sDKnxcmw/5YsBY05h5caJsDTPwfJ56/HiVj+khzRHyGNv/L2j288QAAAgAElEQVRqEypX8ENsVAxMTfIAEOrWZeuqli9nQ3h1CrOuzc0LIwmyFNhbPcEk72OQZ6+EPN0blNwKlDkFKLgM6NUAYz4XjN1V6FjvBGPY7qsmQZQkOy0H89yXIdQ/HN7LfuJ94vPn2Pcb+xCkaoabTCZD0L03qNmkGu+x2Wk5+PeYH5p2c+VUaKyQNrLlIGIb6PsGcpkcUzaN1qaWl0eUuVgAVoDV3usA4DtF4+uqqdPUCfEREU1qMZcmNp/Dqe/Tay9Zd38ZP3f/yp7b5MYIKDY0nvOYpOgUGlRlPAkqjOY1rjhSqZT+GP43uTECOrb6rEpzfApFePToH2c0Ou/nKB5mc3Fhvy/5WlDQh+8VYcDKleJoqOAcbV61mILv9SFpfB02BFfYciNcSZzQkWQpP5IsdSDJUjxIlNCL0kLdSJbUkWSJTT7qL0tsRLKUviTLWkHyggckl6sf7ixLYkLiaHjtKdTTaBDdPnpfo3O/uB1IboyAtv28T+U5Xvm+JjdGoNLaNk/bTe66nhQRxC30fn0/Gw6MfhurtO/uXw+Tu54XyeVy3uvSwg+oEOLj8oil0DL5vbhdwzeixwcAnb3aYsecg4gLS0ClWk6f7dvcvTE6erbBkZU+6Dq4PZxqcMvoca5dGEoMS1R6DQUOle3wx7VFmNlhEea5L8P6+8thx/M4BF1dXfyyZxJyM/JwdNVpeEzrqbFTUz2m/YDgByHYt+gYmnZriLrNa2pk3s9RWmIFwH6tyAhcupT9V3HAob09UCCuiEOn+uLkhb4QiYB1a3PhdycAlmYJsLNNRxXnDIwamQ5d3XSARICOOfR0DWBlZAgwBgBjAka3GqBXg206jmCYb+OJ+pXvayzx+BO6erpYe3sxXNrU1djcwlwh1o3Zhoq1HDFy2U8qz3PP5xH0DfXRqtd3vMbFhibgwrbr6Dm6G6o1qMxpTPSbWOjq6XL67Gan5sDSzrxcecJaisHXon2Npq4HlRSVTO56XjSn+1ISCUVK+6fEplJfi2E0xnUmpSdmcLpGRnImuTECOvj7Sd7re/sklPqYD+Xs5ZWG/81XZeJFZafn0KAq4+lH6xF0/8xjjc79KZQlViQlETk6ftrLcnAgqlv3Q6KEIllC0fd/Cf8bAdTbdAh5159O8e8TNTp3Vmo2TW+/gLrreNIr39cqz6P4+17ssZrXuNT4dBrdYAb1tRjG+XNIRDSt7a80ttHPnPrO/2EFjWs8i9e6tKgGVPCguIT4DAEMBvArgN8Uje+F1GnqGigioqt7b1N3HU+a/8MKTtlpL24Hsh/8etM4fzhmdlpEI+tOVSlccHLdeXJjBBTzTrVQn1wupyUD1lAPg4EU9jJCpTk+RXx4Ik1qMZfcGAFtmb6HxKKvG+5SZPuVNDhJSUR16lBRdt6nQoT/K0bq0cVn1NNoEI1t9DOvGzgX4sMTaWTdqdTTaBDdOe6n8jyRwdHkYTuShlafSCmxqZzHJUYm0/Bak6m32RB6+W8Q53HBD96SGyMgnw0XlfaVy+XkYTuS1o7awnl+LapTVgbqKoDjAOYAmKVofC+kTtOEgSIiurTjBrkxAlrYdxWnm6wibn5k5WlO8yvS1IMfvOW9tvj3iSqlqxcnMyWLvJzG0BjXmZw8RT6ICsS0ZfoecmMENLnlXF43m7LgU17WqlWfNl4KI1Uyvf1bxPfUQ+phMJAmtZhLWWnZGp9/VpfF9KP1CAq8/0blOeLDE2lgpbHk5TSG4sISeI2d3HIu9bMaTsEPQ3iNWzJgDXnYjOBUehH9NpbcGAFd3nWT1zW0qEZZGaggvpNqumnKQBERndtyldwYAf3pze0u5eU0hlaP2MSpb152PvU2HUJzui8lYR7/uhjvetNoYrPZatUfPbnynNwYAS0VrKH8XKHK83yKe6cfUR/zoTSw0lh6dMlf4/NrAj61V98aErGEDv5+ktz1vGhauwWUm5mr8Wsokgy4eCGfIupNLA2tPpE8bEZQ+KtIXmOz03Oou44nHVhygte4hxeeUXcdT9r962FO/U+sOce77lGL6pSVgdoBoCHfiTXZNGmgiIhWDF5P/ayGc+q7VLCGfnIexzlsd3H7dequ40lTWs3jXZB457gfuTECWjZwHclkMl5ji3Ny3Xly1/WksY1+5v3kyoXQF+E0usEMcmME9MfwvykrVfNP8Fr+y/uASJrYbDa5MQJaMXi9Rgu0FVzedZO663jSL10Xq+WFT2w2mwQOo+jt0zDeY89vu0ZujIDX2Es7b5K7ridNajGXstNzlPYPD4yiH4wH0bwey7QZfF8IjRooAEEAXgF4DUACIKTw+0AAr/heSJ2maQN1Yi2738PlD1mRPh76Ipzz/H5nn1Avk8E0rOZkigmJ47e2wqc6dVJ6idh0eQ/bkeRhO5KSolPUmqs0RAVi2rvoKH2vP5AEFUbT3ZMPNH4NLSxikZj2Lz5e9F77nnpYJtc5s+kyuTECmt9zORXkq66M8c7/PbkxAjqz6bJK46e0nk9jGs7kZDjkcjkdWHKiaN1cjLYwr4DGuM4kT8fRGt+70/JpNG2gMgBU/VTjeyF1mqYN1L3Tjzg/oaUnZpAbI6Cdcw/yusbrR+9I4DCKPGxH8vJi5HI5bZ62u6gWS52nu5iQOOptNoRmdlpUJqEgIqKwlxFFT/UL+6xSOclDS+kE+b2lMQ1nkhsjoFVDN1JmStZn+6sS3pTJZHTw95Pkxgjotx9XqxVilslktGroRuppNIjTA2BJFEkOJ9ed59RfUaO1auhGkoglSvtLpVJaPWITuTECenb9Je/1aVEdTRuo53wnK6umaQOVFJ1C7npetP2X/Zz6LxWsITdGQMf/5JfCHfUmltz1vGjHHH7GTSqV0qqhG8mNEdDyn/7i9MH7FDcO3iV3PS8aWn0i7w1nrkglUjqx5hz1tRhGPQwG0raf91FORtkYxP8vxIUlFN1IB1UeTw/OP1U65nPZjZ9KEBEViGlhn1W8bvKfIi87n5YMYD8rXD9bxblz4gH1Nh1CgyqPV2qIFeyYc5B6GAyk3Kw8Tutb0HsluTECOrCU3/6WFvXRtIGKBfDzpxrfC6nTNG2giNh9qL4WwzjdSEUFYlo2cB25MQLaMfsAL6/m114raHDVCbz3lORyOR1aforcGAGd23KV19iSBD94S0OrTyR3PS86tPwUSaVSteb7FGkJ6bR29FbqruNJA+y96fy2aySVlM21/heRy+XkfyOAFvZdRd11PKmHwUDaNf8w570mLiocxQ2XTCaj5T/9xYbj/r6slrceGxpPY1xnkruuJ5366wKvuWQyGe1deJTcGAFNa7eA0hLSOY/1rj+d5rj/zqnv6pGbyF3Pi85vu8Z5fi2aQ9MGKqGw5mlxaY3vhdRpZWGgQp+HkxsjoMMrfDj1l0ql9PfknUUZgFyfNG8e8iU3RqBSoaNcLqeZHReRV8Wxamfk5Wbm0vJB68mNEdCsLot53QT4Evo8nGZ2WkRujICG15pMh1f4cH4i/v+IqEBMF7dfL0o8ETiMor2LjlJKXBrvuYobpM8VKctksqK/5xNrzqm1/idXX9CP1iPIw3Yk+d8I4DVWKpHSYo/V5MYIaO3orbzCizEhcUXGlQsj6kylJQPW8FqfFs1RrkJ8APaAPZ4jqNhrNgBuAAgt/Neay1xlYaCIWO/me/2BnJ8e5XI5HVh6gleaen5OPv1oPYK8Ko5VyUgF3n9D3XXYjLzI1zG8xxdHLpfT1b23qZfJYPJyGkMvbgeqNZ+ya907/YhmdVlMboyAepsNoV3zDmkNVTGy03Po6KrT5FVxLLkxAprYbDZd339H7Rq2pKSPjxqxt//YOMW/T6SfO/+mciiuOO/839P3+gNpXONZFB/OX8lCcYzGkZWneXldUW9iybv+dOphMJCSopKV9o95F0/ddTw5P5Bq0TyaNlAv+E5WYnxHsMKyxQ3UnwDmFX49D8BqLnOVlYHKzcylRf3+IDdGQKtHbuJ8Y/jTezP1Nh3Cuf/7gEgaUWcquet50Yk153iHUp5ceU4Ch1HU23QIXdl9S+202PcBkeRdbxq567K1JmUV8lMQGRxNKwavp+46ntTbbAj9M2s/BdwN/uqKFF+L+PBE2jxtN/U2G0JujIDmfr+M/G8EaCzd+VMGSi6X04V/rlNvsyHU13IYXdlzW61rioQiGt1gBg2sNFalYmGpVEoj606lcY1n8QqB+/o8or4Ww0jgMIqzysTaUVvoB+NB2qy9r4imDZQN38lKmaNaCQMVAsCp8GsnACFc5ikrA0XEhjoUaaqTW87lZHSeXH1BboyAV6FqblYeLfVcS26MgBZ7rOadRJASl0a/dFtSVAOTl61eDUx+Tn6RCvovXRerFE7iS+TrGFoxeD2563qyXpXpEJrXYxkdW32WQp6F/c/uVwnzCijgbjAdW32WFvReSe667P7S6hGb6H1ApEav9akQX4O6uTS/N5vAMKf7Uk5ehzK2/7Kf3BgBPbnyXKXxivA317R5qURKO+ceJDdGQFNazaPkGG5qJomRyfS9/kDaPHW3SuvUohnKpFBXnVaKgcos8fMMLvOUpYFScH4rqzDhd+6J0r6iAjH1MR9Kk1vO5fwhIWKfYH3WXyR3PS9a0Hsl76dXqVRKh5afInddT5rRYaFGMuWu7r1NvU2HUH87b7qy+5ZaBcJcycnIJb+zT2jz1N1F+y6KvZdtM/dyVh4oz4oRcrmc3j4JpbWjt1Jv0yFFv+PwWpNpx5yDZSIV9akkiaaVn1E7TKBujBftWXJebU9NKpHSgSUnqLuOJ60fv12lOV75vmZluRrO5Pw3t/vXw+TGCGj9+O2c96rSEzNoZqdFbCiwDOoBtXDnf8pAARgH4BmAZ1WqVCmL9+sjxCIxeTmNofk/rODU3/fUQ+pjPpT623nT48v8niBP/XWB3BgB3Trsq8pS6c5xP+phwMb9U+PVT3aIfB1D09otKNoHUUe5WhVS49Pp1mFfWipgxW7dGAFNbD6Hzm6+QklRyaXeUFVJqS5rMlOy6PbR+/Sn92YaWGlskZe4dvRWenTxWZnvv5V8T1Li0oq89k6GM8gSb9V+T2LexdOUVvOK0tL5SnpJpaxxc9f1pOG1p3D2IEUFYupv502//chdET3gbjB5VRxLPxgPohsH7/JapxbN8y0YqHIX4iuOIgGCq/pDTEgcjWs8i9wYAe2ad4hziEoqldKU1vOpv503ZSTzk0NS8PTaS+ptNoSG1pik8mGHxZHL5XTryD0aVGU8uTEC+t1rnUqb3uqSmZJFpzdeovFNfynyOvpZDacZHRbSxok76Py2a/TidiD5+0aTa51MAqRfVLVcKpVSSmwqBd5/QzcP+dKh5ado3ZhtNKnFXOquw4YuPWxG0LKB6+jK7ltfvB5s82aimMgCOv7nWeprMYx+MB5Eh1f4UFyMWG3j9Pjyc+ptOoQ8bEaopHCenpRZlN25athGXmHq20fvkxsjoKfXuBXX+mxgIxUj6kzVeBhVi2qoYqAYdlzZwDBMNQAXici18Ps1ANKI6A+GYeaB3eeao2ye5s2b07Nnz8psnQoykjIxpOpEtO7bHPMPTYO+gb7SMSKhCNtm7MOlnTfRuHMDrL6+CLp6yo+ljgyOwcTvZqOpWyMsODIdppamvNf79kkoFvRaBR1dHcw/NA3fuTXiPUdJCvJFOLXuAo6vPguZTI4BM3rhp3k/qrQ+dQl/FYVgv7eICIxGeGAUIgKjkZ8t/KgPgYGEzCHXM4eMMUa+xApGdpXwy2/OcGlREfbOtjCzMoGRqdFnD6UjImSlZiM5OhUpMWlIjk5FYkQyIoKikZWSjdzMPORm5v3n+gBgXcESznUr4rtujdDMvTHqNK/B6WhyTSOXy3HjwF3sXXgUafEZaNGzKSZv9OZ8gOanICLcPnIf68ZsQ7UGzvj93FzYVbLlNYcwV4gl/dcg6P5bzPhnPLoP78R5bE5GLuZ9vxzZqdnYH7ZZ6dHsqXFpGFxlIpq5N8LC4z/D1MKE11q1lA0Mw/gTUXNeY8rKQDEMcxRAZwB2AJLA1k+dBXACQBUA0QA8iShd2VxfykABwMHfT+LAkhOo07wmfj0ynfOH+9yWq9g8dTdm7Z6EHt5dOI05v/UatkzbDfvKdph7YCoadqjPe73Rb+OwxONPxITEo8/E7zHhrxEwMFRuWJWRGpeG3b8ewc2DvrCwNceQBQPQe6K7RuZWFSJCSkwqYkMTkZWSjayUbMRHZmHntmzIhNnQQwFM9dJgqpMAmUT20VgdXR2YWprAzMoU+oZ6kIqlkEpkkIqlkIilKMgTQSKSfDTG0NgA1Vwrw7aiDUytTGBmaQozK1NYV7CEY3UHVKjmAIcqdjAyMfySb0OpBD8IwdYZe/Hu2XvUb10bY/4YikYdXdSeNzUuDRsn7cSjC/5waVMHy87Pg4WtOa85YkLisHTAWsS8jcOM7ePRc3Q3zmMD773BqqEbkZ6QiTn7JqPr4A5Kx6wcsgH3fR5jZ9BfahtnLZqjXBkoTfIlDRQA3D/zGH+N2QaZVI7p28Zy+lAQESa3nIectBzsDfkbevp6nK71+tE7rB72NxLCkzHsN08MW+zJe70ioQh7Fx6Dz/qLqNeqNn47OQv2zvyecD9F2IsI7Jp/GP7XA+BYzR4jlw1Cl0HtlD7FfimSkwFXVyAlhf3e3h54+UIKWW4S4sMSkRafUeT95GXlIy8rH2KRBAaG+tDT14Oevi70DPRgaGwAO2dbOFSxK2oWtuX7KHCxSIInl5/j+v47eHj+GWwrWmPs6mHoOri92usmIlzbdwf//LwPEpEE3ssHwWP6D7w9w3s+j7B21FYYGOlj/pEZ+K5bQ07jZFIZDv5+EkdXnoZjjQr49fB01G1RS+m4p9de4teeKzDsN08MX+LFa61ayhZVDFSZ7kFpqn2pPajiJEUl04wOC8mNEdDVvbc5jXl0yV8lFef8nHxaNYzV3ju7+YoqyyUitj6kj/lQElQYzbuiXxn+NwJownesKOy4xrPo7OYraileawKuqgn/a0S+jqGNE3eQh80IcmME5Ok4mvYuOqqx4zfSEtKL9PlmdlqkkgCwWCQuSkOf0no+r2xXmUxG839YUVQQz3WvKjczl4bWmETe9aapJXirpWxAeUuS0FT7GgaKiE2nHd/0Fxpg702JkcrrRuRyOc3stIi663jStp/38VIEkEqlNL/n8qI6J1XPWIp8HUPe9acXfbg1eVaTTCajW0fuFSWGeFUcS2c3X/kqBbd8def+Fwh5FkZ/em8md11P6mUymFYMXk9PrjzXWP2YTCajC/9cp35Ww6mn4U/ks+GiSmUHuZm5RVmhGyft5GUspFIp/em9mZVgWstN0ZyIVVxR6E2+vMP9iHgtXw6tgSoDYkLiqJ/VcJrw3WxOKbX5uULaOHEHuTECGuM6k9c5UhKxhA4sPUHf6w8kT8fRdP/MY5XWLMwroF3zDtH3+gOpv503nd18RS2V6tIIuBtMMzuyGVlDqk2kK3tua/yY+c9RHtPMy4LcrDw6v+1a0ZEmvUwG09YZe1XO/vwU4a8iaVrbX4sKt6Pfxqo0T05GLk1pNY++1x9I/x67z2usVCKllUM2FKmNc6nXkslktHfRUXLX9aShNSZR8IO3Kq1bS9mjNVBlxKOL7FHSq4Zu5Fzk+Pjyc/JyGkM9DAbyPncm7GVEUZr1qqEbVQ7dhL+KpF+6slp43vWn04PzTzV6eqhcLqen117SpBZzyY0RUA+DgTSl9XzaOmMv3TnuV6aCtETlu1BXHeLfJ9KtI/dojfeWoiLfcY1n0bktVzWetp6Vmk3/zNpP3+sPpAH23nT9wB2V/0ay0rJpYvM51MNgIKeC9+JIxBL63Ys9MeDoqtOcx13aeZOVKhuxidORG1q+HloDVYYcWsYefbHUcy3nsFlWajaNcZ1JfS2G0eVdN3l98CViSVFB44g6U1UOW8jlcvI794RG1p1KboyA5vVYxvuUXy7XeHrtJe2Yc5BmdlxEPxgPIjdGQO56XrR80Hp6cTtQe6y2EuRyOT27/pLmuP9eVP/V22wIrRuzjd48fqfx908qkdKZTZfpR+sR1F3Hk9aO3qpWOPjZ9Zc0tPpE6mn4Ez26+IzX2Oi3sTS1zXxyYwR06q8LnMfdPnqffrQeQZNbztX+fX0DaA1UGSKTyejoqtPUw2AgeTmNoee3XnEalxiZXKToPb/ncl6bxUREz2+9oqE1JrESL+P+UfkJWiKWkM/6i9TXkj1UcNWwjfTK93WZfLAlYgm9fRJK22bupR+t2Y384bWn0NFVp8vcq/qWkEqlFPoinHw2XCzymL2cxtCRlacp7GWExsOyCgLuBhftI852W0rhgVEqz5WblUdrvLeQGyOgkXWnUpAfvxDbK9/X1MtkMHnYjuQcEsxKzS46n21K6/kU//7LF5Rr4Y8qBkqbZs6TsJcRWDl4IzKTMrH7zUZYO1gqHSOXy3F+6zXsnncYuvq6mPDXSHw/sjPnVGBhXgEOLjkBn/UXYVXBClM3j0Z7j1YqrT89MQOHl/vg5iFf5GcLUbleJfQa64buwzvxrm/hgkgowj2fx7i86yYCfd9AR1cHzdwbo7prFVSq5YiKtRxRqbYTbCtal5vU9bJALJIgLT4dKTFpCHkShle+rxF0/y1yM/MAANVcK2PAjN7oOqRDmdSaERHePHqHM5uu4M4xPzhUscOEdSPQvn8rlVPSgx+E4I9hfyM5KgVes/th2GJPGBgZcB4fGRyDmR0WwbqCJdbcXgJbJ2ulYx5ffo6/xmxDdloOhi32wsA5/TgVxmv5+mjroL4QUa9jMKHpbDTq3ADzD02Dlb1yIwUA8e8TsXbUVgTee4P2/Vth5vbxvIzCO//3WDdmG8IDotDOoyUmbfCGQ2U7lX4HYV4B7p54iMs7b+DNo1DoG+rDfURnDFvsyelGoQqx7+JxZfdtPLzwDInhSZCIpUU/0zfUh21FazTq5ILOXm3RtFtDzrVk5ZHkmFQ8vvQcd477ISo4BlmpOR/93LmOExp1dEHDji5o1MlF5f9HZRAR7p54gP2LjyP2XQKMTAwhmNUHA+f+qHKBcUG+CIeXncKJNefgUNUe8w5OQ4O2dXnNEXA3GH8M/RtyOeHvBytQoaq90jGHl/tg32/HUM21MuYemIpaTaqrtH4tXwetgfqCXNpxA1um7YGppQmmbhmDjoI2nMbJ5XKcXHsBexcehaW9BWbtmoiWPZtyvq5UIsWpdRdwaNkpMDoMhi8ZCI9pPdW6mYe/isL5rddwdc9t6BvoYcDM3vCc3bdMJWJkMhlSY9MRF5qAuLBEJLxPRGJUCvyvByA/WwhzGzO069cCjTo1QJ0WNeFc26lcPykL8wrw6u5r+F8PgP+NAES/iQMAVHVxhmv7+rB3toVtJRvYVbJB9YZVyuwhoDjBD0Kw/Zf9ePMoFNUbVkH/Gb3RUdAaJubGKs/55MoLbJqyC4kRyejh3QUT1o/k9XeSl52PXXMP4eL2G6hYswIW+8xGjUZVlY67vOsW1o/7B92GdMDPuyZ+VUUTLaqhNVBfmIigaKzx3oJQ/3B08mqDKZtGc/amwl5EYPWITYgMikGvsW4Yv244jM243zgSIpKwZdoePL70HDUaVcX0bWPh0obfU2xJ4sISsHfhUdw98RBW9hYYslCAXuPdOGkSagqxSAL/6wG4e+IBnl17WeR56Bvqo6qLM6o3rALnOhVh5WAJK3sLWCqanTmMzYw07nXJ5XIIcwuQV0yJIi0hE0mRyUiMTEFSVDKSo1IRF5oAiVgKAyN9NOrkgmbdG6OZe2NUa1D5i6lRiAvECPILgf/1ADy/+QphLyJg42QN72U/ofuITmrpA755HIp9vx3D8xuvULleJUzfNhaNOzXgNUfA3WCsHr4JaXHp8JjeCyOX/cTJi3t8+Tl+67cazbo3wu/n5n7TnvX/Z7QG6isglUhx/M9zOPT7SVjYWeCPawtR3bUKp7HiAjH2/3YcJ9ddgGN1B8zaPZHXh56I4Hf2CbZO34uU2DR0H94JI38fCIcqysMlnyPkaRh2zj2EgDvBsK5giUadXNBhQBt0GNDqi+4TSSVSRAbFICIwGhGBUYgIikZEYDTS4jM+OUZXTxeGJgYwNDaAoYkh9A30oKOrA109Xejq6UBHVweMjg5ILkdRkpCcIJfLWV0+kRQSkQRSsRRikQTCnAJ86jNibm2KCtUcUKGaPZxrO6Fpt4Zo2KE+r30YdREXiHFt3x34nX2CQN/XEBdIoKevC5e2ddG6VzP0ntCd14NPSaJex2DXvMN4dNEflnbmGDjXA/2m9ODlwcjlcpzddAU7Zh+EUw0HzNk/FfVb1eY0NuBuMBb2WoXK9Spi3Z2lav0uWr4uWgP1FQl7GYH53y9HXlY+Bi8YgJ/m/cj5SS/w3hus8d6ChPAk9BzdDWP/HApzazPO1xbmCnHo91M4u/kKGIbBoPn94flLH7VulESEp1df4uahuwj0fYPUuHRUb1gFI5YORNt+Lb6qRp1IKEJWSjYykrOLRGOzUrMhyhejIF8EsVAMUb4IBUIRZBIZZFIZ5DI5ZFI5+7WcoKPDAAwDHR0GDMOA0WGgb6gPfQM96BvoQc9AD/qG+jAxN4aplSlMLU2KmnUFSzhWs/8qCu8KxAViXN51C8f+OIO0+AxUrlcJzd1Zr61Rx/pq38hzM/NweLkPzvx9GcZmRvD8pS88pvXkPW9kcAzWj9+O1w9C0Lp3M8w7OJXT+ybMFWLPgqM4t/kqnGpWwHrf32HjWPZhUS1lh9ZAfWUykrOwdcZe3DnmhxqNqmLW7omo06wmp7EF+SIcWHwcPusvwtLeApP/Ho2Ogta8DEFSVAq2/7If93wew6lGBUz4awTa9GmutjGRyWS4e+IhDi49gdh3CajZpBra9WuJRp1cUL917S/qMfx/p6RhatihPoYv8ULjzqa5tZ8AACAASURBVA008v/8/GYgbhy4A78zTyARSdFjVFeMXjUYlnYWvNd5eLkPTqw5BxMLE0xYNwJuwzpyWmNydArmfb8cMSHx6De5B0atHKzWvpmW8oHWQJUTHpx/io0TdyIzKROCWX0xYqkX55t46PNwrB/3D0KfR6B172aYumUM7wyv57cCsWXabkS/iUPTbg3RY1RXtPdoqbYhkUlluHX4Hs5uuoywF5EgIugb6qNeq1po1NEFNRpVRe3vasCpRgW1rqOFJTstB0F+bxEbEo+YkHjEhSYgMigaORl5GjVMsaEJuLbnNm4e8kVqXDrMrU3RZVB7/DDWDTUbV+M93+NL/tg6Yy/i3yeh+/BOGL92OGcDF/02DvPclyE/R4glp2ejSRdX3tfXUj7RGqhyRG5mHnbMPogru2+hcr1KmLl9POfznmRSGU5vvIwDi48DDPDTPA8Ifu4NQ2PuacFSiRTnNl/FyXXnkRafAcdq9hjzx1B09GyjkfBcTkYugu6/xau7r/HK9zXCnodDLmf/llr3boYBM3tr5Ob5/w1FvdLlnbdw57gfREIxAMDKwRLOdZzgXKciugxqj6ZdXdV+bzOSMnFgyQlc3nULANCiRxO4j+iM1n2aq5QllxSVgq0z9uLBuaeoXK8Spmwazf14DZkM5zZfxd6FR2FkaoQ/ri1UyThqKb9oDVQ55Om1l/h74g4kRqag11g3jFk9FGZW3PYuEiKSsH3WfvidfQr7yrYYvXII77OY5HI5/K8HYNe8wwh/FYUG7epiwroRqNeS2yY1V4S5QsSFJuLh+Wc4v/UqMlOyUatpdfQY1RUubeqgesMq2uyrz5CdnoObB31xZdctRAbHwMjUEF0HtUf3EZ1R1cWZ156kMkRCEXzWX8Lx1WchEorRZ4I7fprvoXLqu0Qsgc/6Szi87BQAYMgiAQbM7MU5+/N9QCTWj/sHIU/fo0XPppixbazaiT5ayh9aA1VO+UgJwsESkzaO4rW/FHA3GNtn7Ufo8wjUbVETE9aNgGt7fqfvymQyXNt7B/sWHUVGUhYq16uERh3qo1EntlhUUwccAuwN8Nbh+/BZf6GoHsjASB+1vquB+i1rodZ3NWBdwRKWdhawsDOHpZ05L+/wW4SIIBKKkZuRi+SYNCSGJyH+fRISIpKQEJ6Et4/DIBFJUK9lLfQc44bOA9tqfN8lOy0HDy88w/7Fx5ESk4a2/Vpg7OqhcK5TUaX5CvJFuHnQFz7rLyD2XQLa/dgCE9d7cyq6BVgv/+iqMzi83Afm1qaYuMEbXX5qp/W6/0fRGqhyTujzcKwfvx2h/uFo1es7TP57FJyqc9uvkcvluHX4Hvb8egSpcelo07c5xq8dzvtI6/wcIS7tuImX/wYi6P5b5GcLAQDVG1bBkIUCtO/fUq16meIQEZKiUvD2cSjePg7FmydhCHseDnGB5D99DY0NUMXFGe09WqF9/1aoUq+SRtbwtSAihL+Kwp3jD3DP5xGSo1I+Us5QYFfJBk41KhR5m1yKVvkScCcYexcdRbBfCACgdrMaGL92OO86JgUyqQwXt9/A/sXHkZOei1pNq2PE0oFo3bsZ5zlCnr3HhvHbEfYiAt2GdMCkDd5lIrWlpfygNVDfADKpDGc3XcG+346B5IShiwQY8HNvzuGQgnwRTm+4hGN/nIFULIVgVh8Mmu+hUlqxTCZDxKtovLr7Ghe3X0dMSDwq1XaC56w+6D68U5lk50klUsSFJSI7NQfZaTnISs1Bdmo2MlOy8fphCN48CgUAVKlfCe09WsGlTR3UaFyt3Gv1ScQShD6PQPD9twjye4tgv7fISs2Bjq4OmnZriFpNqsHM2gxmVqawd7aBY40KcKxmX6aeY+jzcOxZcATPrgXA3tkWvcZ3R8MO9eHavp7K72XQ/TfYNHU3wgOi0KSrK4Yv9oJr+3qcvZ687HzsW3gM57dehZWDJaZsGo0OA1qrtBYt3xZaA/UNkRyTim0z9+H+6ceo6uKMaVvHolFHF87j0xIysGveIdw86Au7SjYY++cwtcIjMpkMfmee4Pif5/Du2XtYV7CEx7Re6DXeDRY2X+7JNjUuDX5nn+L+6UcIfvAOEhHrbRmZGKJibUdUrlsRjtUcYGFrDnMbM5jbmMHC1hxmVqZFBboGRgbQN9KHgZE+b29QLpdDKpFBKpZCKpZCmFuA/BwhhDlC5OcUQJgjRGZKNtLi0pESl4bUuHSkxaUjITypyDOsWMsRru3rwbVdPbTu05yToLAmSI5OQZBfCIL93iL4QQjev4yEuY0ZBs3vj76T3FUyhgpP0O/ME/idfYLwV1Gwr2yLCetGoMMA7mFqqUSKm4fuYd+io0hPyESfie4YtWLQV60l0/Jl0Rqob5BHF/2xeepuJEWlwG1YR4z5YyivzergByHYMm03Qp9HoFqDyug6uAM8pv+gshAoEeHlv0E4seYcnl0LAADYOFmjUm1H1GhYFX0n9/hi4be8rDyEPo9gU6zfxSPmXTxi3yUgKTIFMqmM0xwMw0BHl1WQ0NVVKEkwIHnxM2c+GCa5TM5pXh0dBtaOVkUae47VHNCgbV00aFf3ixWUFuSLcHXPbby4FYjQ5+FIiUkDABibGaF+69po2rUh+kx0V8kI5Gbm4cSac/j3mB8SI5LBMAxc29dDe49W+GGcG+e/LyLCneMPsHfhUSSEJ6FO85qYunm0xpN0tJR/tAbqG0WhDu2z/iL0DfUxfIkX+k3pwTnrTSaT4cb+u7h+4A4Cfd/A3tkWo1YORtfB7dUKi4W9jMCTyy8QF5aAuNAEhD2PgLhAgk5ebeA+ovNXUxwnIhTkFSAnPRfZ6bnISc9FbkYeCvJFkBRIIC6QQCQUQywUQyaVQSaTQ168yeXQ0dEBwwBgWCUJHR0GegZ60NNnVSTYr3VhZGoIE3NjGJsbw8TcCMbmxrC0M4d1BauvJl6bl52PC1uvwWf9RWSmZMO5jhNqNqkG13b10aBdXdRoVFXlteVm5uHMxsvw2XAR+dlCNO/RBO09WqFNX/6eYNSbWGyeuhsvbwehZpNqRftU2iSI/59oDdQ3TlxYArZM34unV16gmmtlTN08hlfYDwBe+b7GP7P2I9Q/HPVa1sKolYPh2r6eRgRfM1Oy4PPXRZzfdo1VHLc2RZt+LdBxQGs0dWukVZguI4gIiZHJCA+IwusHIbi86xZyM/PQokcTDJrfn3N93efIzczD6Q2XcHrjJeRl5aOdR0sM+82Tdy0SEeGdfziu7f0XV3bdhJGpEUatGIQfxrlpLPlGy7eJ1kD9D0BEeHDuKbbN3IekqBR0GdQOgp/7cJZMAthw1c2Dvtiz4AjS4jOgb6iPWk2roW7zWqjbshba/dhCLa02sUiC5zdewffUQzw49xR5WfkwsTBGtQb/1959R0d13Qkc//4kjXpHqEsIBMjIdFOMMQYDLuASY8A2G6du4s0ee53YOZtNNtms00529yTrPRv7OOs4m7jGXuMSiI0xJsZgOqZYCCGaekG9lxmN7v7xngYZSxRJSDPi9zlnzoxGb97ce640v3n33ff7pRGTGE1sQrR1nxhNXEosM5dOHfXLyIdCa1Mbh7bm0FDVRFNtM7XldRTkFHPm0yJaG9sAa2rxhnvmse4Hqy7rb6Ivne2d7Nn4Cbm78nn/+W20NrZx473zefBf1lx2YGqsaWLTc1vZ8uJHFOeV4QhysOxvbuTrv/zisJ2DU95NA9Qo0tHWyau/fIvXf70BZ4eLZQ8u4hv/9iBxybGXvI/21g72vXOQ4/tOkX/gFCcPnKGjrZOYhChWP3YXdzy0/JIvGu6Py+ni0Naj7Hp7H+VnzlJf2UBdZQNNtecK9EXEhLH8S4tZ+c3lZFybNqj3G23cXW7y959iywsf8cFL2+lo7fT8LjQyhIyp6WTOyCBzxjgmzMggY2oaIWHBg3tPt5stL2znhX99jerSWhyBAcy/87oBBab21g42PbeVF3/yOi0NrUy98Rpu+dJiblq7YNB/W2p08ZkAJSKFQDPgBrou1uirMUD1aG1q47V/f5v1v96Iv8OfL/5oDfd+544BTae53W5yd+bz0s/Wc2hrDiHhwaz422Ws+vZKEjPih7TdLqeL+rONFOeVsfmPH/LxG3vocrnJviGLW7+8mEnXTSDtmpRBf9j6ourSWg5sPsz+zYc59EEOLQ2tOIIc3LxuIbd/bSlJE6xVikO9zL+9tYNP3j/C8z9+jcLcErLmZvK1n69j+uLsy54Criqu5s9Pb2bTcx/QXN/KrGXT+Psnv3rJpWbU1cfXAtQcY0zNpWx/NQeoHuWnK/ntd59n94YDJE9MZOU3ljNjSTaTZk8Y0AnxU4cKWP+fG9n22i6MMdy05noW3jOPxPHxJI6PJyouckhPZjdUN7Llhe28+7stlJ6o8Dwfnx5H+pQU0rJSiEmIJjwmjPDoMMJjwoiICSM4LBhHkFX6IjDY4SmJ4e/wH7FzGsYYulxd55aju9y4Ol10tHbS3tJBR2sHHS0dtDV3UFdRT01ZHTXl1nL06pIaKgurARiTHMPc22Yy57aZzL5l+pCnMyo/VUne3lPk7zvJ8f2nKDxaQre7m9TJSXzt5+sua5k4WFPHx3bl89ZvNvHxm3vBGBbeO597H13JtQsv/VoodXXSAHUV2P/eIX7/z69w+nAhADEJUXz1pw9w29dvHtAHdnVpLW//97v85dktnqwSAKmTk/jij9Zw87qFQxoIjDEUHy+j+FgpxXllFB+37kvzy+lo67z4DnpJy0pm8pxMJl+XyYQZ40iemMiYpJghXV3X2tRGcV4ZRcdKKT5WQlFeKfn7Tnkq/V6KwGAHcSmxnpLvk2ZnMue2K1NttzC3hJd/vp7tr+/2JO+NiAkja95EsuZOZMr8SVx364zLWn1pjGHHG3t47vsvU3HmLOHRYaz8xjLufvj2S05rpJQvBagCoB4wwP8YY57tY5uHgIcA0tPTrysqKhreRnq5usp6crbn8fZTmzj68XEmTB/Hgz9ey5zbZgxo2qznG3dlYTUVp8+y+Y8fcubTIhLHx7Pkvhu4ae0CJs4af0W/JXe2d9LS0EZLfYvnvqPNibPD6al06+xwWfftTgqOFpO//9RnKuyKCDEJUZ5gEB4TRlCwVV03ODSIoNAg/AP8cHd120UMrYKGrk4XzfWtNNe30FTbbC1hr2mmrrLBs29HkIO0rGQyZ2WQkpmEI+jcsnRHUADBYcEEhwUREn7uPiYxmoiY8Ct+dFGQU8SLP1vPjvV7PFO3WXMzmTx3IikTEwf8/sf3neS3332e3J35TJg+jtWP3cmiNddflVOzanB8KUAlG2PKRSQe2AL8gzFme3/b6xFU/4wxbF+/h99970XOFlUTGOxg1rJpLLhrDvPvvO6yFlX01t3dzc639vHO7z7g0NYcut3dJGcmcNOaBSxcZU0FRsSGe8XS4ZryOopySzhbWE1NWR3VpbXUltdRU1ZHW1O7VV23rZPONmef5dt7roEKjwkncsy57BQRMeGkTEwkPTuVcdmpJI6P94r+tja1UXi0hIKcYgpyijh9pJDcnfmERoRwzz+sYPVjdw4qr11zfQtFuSX85X+2sPXlHdZR+s/WcaZ9Cffd70/8eacrq6rg9dfh4YcH2TE1qvlMgPpMA0SeAFqMMb/qbxsNUBfX5eri0+157Nl4gN0bD1BZUAXAhOnjmLVsGvf/0z0DXu7bWNPEzrf3s339bk+wAuuDPXJMBFFjI4kaG8n4qenc9fe3Mi7bO1fqGWNwObtwu7rwD/DHP8DfyizhxedOivJK2fD0exQdK6W5roXGmqbPHDGGRoSQMTWN2cuns+rbKweUlsoYY5VJeWYzBTnF1FVY+3cEOVjz+J088P1V/OGFEB55BLKz4cMP8QSpqiq4+WY4dgyeekqDlOqfTwQoEQkD/IwxzfbjLcBPjTHv9fcaDVCXxxhD0bFS9mw8wKEPj3L4r0cJCglk7XfvZvXjdw6qjENTbTOH/nqU+soGGqobaahqorGmifqzDZw4cAZXp4uZS6dyzyMruP6u67ziiMPX1JTXcWDzEba/vov97x3GEeQga26mJ/dgysQkxk9LZ/y0dBLGjR1wgO1s72T763vY8Mxmju89SdKEBKYuuoaM7DTGXZvG5DmZni81vQNRT5CCzz93/tGVUj18JUBNAN6yfwwAXjHG/OJCr9EANTgl+WX84Ud/Yscbe4mOj+KLP1zNvJWzSMgYO6QBpOdizQ3PbKa6pJaEcWNZvHYBaVNSSctKJi0rWUsqnMcYQ0tDK6cOFXDgvcMceP8IZz61zrfGpcRyx0O3cOe3biF67NBd7FqYW8I7z27hgxe309LQSurkJFY/dhcr/nbpBReY9A5SY+21EdXVGpzUpfGJADUQGqCGRt7ek/z+By9zZFsuAI7AAFImJZGalUzq5GRSJiWxaPV8wiJDB/U+7i43uzYcYMPT1gKOLte5xK6RYyJIzkwgPCaMkIgQwiJCCI0MJTQyhMDgQM+Cg7CoUGYtmzbgc2gjpcvVRe7OfIqPl9Hl7PLkBOxs66Sz3Ulnu5OmWmvxRV1FPfWVDZ46UQEOf6YumsKcW2cy9/aZjJ+WPujpx6rianZv/ISa0lpqK+opzislf/9pHIEB3Lh6Pnd88xamL86+5PepqoKpU63ABFagOnpUg5O6OA1Q6qJ6cqUVfFpESX45JfnWEu/y02dxd7mJiAlj9WN3cc+jKwYdqMAKVhUFVZTml1OSX05pfhmVRdW0NbXT1tRm37fT1tze5+vTp6QwbVE22Qsmk71gMimTkrzqnJGzw0nZyQqO7zvFvk2HOPjBp59Zrt/DEeSwSoGEBBI5JpzYpBhi7bRQVrb4JGYsyR5UCqreSvLLeOu/N7HpuQ/ocrkJcPgTmxRDXOoYblw1n1u/spiouMjL3q8GKDVQGqDUgHW5ujjxyRn+9Ms32bPxEyJiwrj3O3ey6tEVw1Kzp7u7my5nFy6ntZy8rqKBA5sPc+ivOeTtOenJRRcRG86k2eOJSx1DXHKs5/qiMUkxhEWFEhIR4lnmPZhM7u4uN53tTmu5eW2zfbMeny2sovh4GSXHy6gsqPJcbzQ2dQxzb5/JvJWzyZqbSWCwFZACgx3DUmyxsrCKba/tYttrOzl9uBD/AH9u//pS7vvHu0kcHz/oNugUnxoMDVBqSJz45DQv/Ww9uzccQERIyBhL6uQkUicnk5aVQsqkRCJiwwmNDCUqLmJIMyD0pbu7m+K8Mo7tPkHe7nwKjhZTW15PXUW9Jzj0JTgsCEeQgwCHv2fVnr/DHz8/sV5njOfe7e7G2e70lOq4UF0oR5CD1MlJpE9JIf2aVNKuSWHC9HTSp6Re0aM7Z4fTk+ewsdqqQtxU00xDVSNHPsr1VCOecv0klty/kJvWLhiyKVJdJKEGSwOUGlInD55h94YDVqHA/HJKT5R/JpkpWBfGLlw1jzWP38W1N2QNa/vcbjf1ZxupLaujrrLBM1XY0asKrsvZhbvr3AW57i6rKKH4iXVEIVi1ofyEoGDraCfQrsobFBJIWHQYkWOs66Ki4iKIHBNBZFzEsK5OPFtUzZ+f2sS7z231HEn25ufvx/hp6Sy5fyGL71tA0viEIW/D00+jy8zVoGiAUleUMYaasjrKT1fS2midPyrIKfYkDE2emMj8lbOZt3I202+aMuTJTq8mbrebY7tO8NZv3mXnm3tBhEWr5zPn1plExkUQFWddexYVZ5W7H47zck8/DWvXfv4oSS/UVZdCA5QaEe2tHWx9aQe7NuznyIdHcXa4CA4NYtbyaWQvyGJMUgzRCVHWooDE6GE/AvFGbreb9uYOWhvbaG1so7q0lsKjJRTmFlN4tITivFKcHS4iYsJY+c3l3P3w7cSnxY10s5UaMA1QasR1tHVyZFsue985yL53D3K2qLrP7Tyr2uwptaCQQPwd/jgCA8ickcGCu+cya5nvFTp0OV3k7sxn/3uHyd11nI7WTs/iD7fLjcvZ5ZmC7MvY1DFkTE0j49o0MmeO54Z75mreOzUqaIBSXqe9pZ26ygZPIcO6ygaaaprPXR/U7sTZaT3uclkr547vOUlbczuBwQ4mz8kka04mE2ZkkDkzg/QpKUNSvn4oGGNorGmisqCKU4cK2f/eIQ5tzaG9pYMAhz9Z8yYSOSaCAIc/AYFWUtmAgABCwoMJjQwhLCrUc4tNiiHj2jQt8qdGrYEEqEvPua/UAISEh5AyMYSUiUmX/BqX08WRbcc4uOUIRz46xsbfvo+zwwVYF7MmZSYSmxhNdHwkUXGRRMdHET02kuDwYCujeGggwWHBBIUG4ggMwM/f77O59/wE020wxr51W6v5XJ0u+9bledzW1E5LQ6uVXb2hldaGVmor6qk4c5bKgiraWzo87Y5Pj2Pp3yxi3opZzFw6dVAppZRSegSlfIC7y03pyQrOHCnizJFCSk9WWMusqxppqGqkub51WNrh5+9HeHQYsYnRJE6IJzEjnqQJCSSOjyf9mhSvu4hYKW+iR1BqVPIP8GfclFTGTUnl5gcWfu73LqeLptoWOlo76Gxz0tHWSUdrJ51tnXS53HR7lph343ZbdaD8/ARE8PMTRATxE6tab0/1XvtxaGQo4dGhhEWHERwapAFIqWGkAUr5PEeggzFJMSPdDKXUELvy+VeUUkqpAdAApZRSyitpgFJKKeWVNEAppZTyShqglFJKeSUNUEoppbySBiillFJeSQOUUkopr6QBSimllFfSAKWUUsoraYBSSinllTRAKaWU8kojEqBE5HYRyReRUyLy/ZFog1JKKe827AFKRPyBp4EVQDawTkSyh7sdSimlvNtIHEHNA04ZY84YY5zAq8AXRqAdSimlvNhI1INKAUp6/VwKzD9/IxF5CHjI/rFTRI4OQ9uGUxxQM9KNGGLaJ98w2vo02voDo7NPWZf7gpEIUH2VJP1c3XljzLPAswAicuBySwV7O+2Tb9A+eb/R1h8YvX263NeMxBRfKZDW6+dUoHwE2qGUUsqLjUSA2g9MEpHxIhIIPABsGIF2KKWU8mLDPsVnjOkSkUeAzYA/8L/GmNyLvOzZK9+yYad98g3aJ+832voD2icAxJjPnf5RSimlRpxmklBKKeWVNEAppZTySl4doEZjSiQRKRSRHBE5PJBll95CRP5XRKp6X58mIrEiskVETtr3MSPZxsvRT3+eEJEye6wOi8jKkWzj5RKRNBH5UETyRCRXRL5tP+/L49Rfn3x2rEQkWET2icgRu08/sZ8fLyJ77XF6zV5U5hMu0Kc/ikhBr3GaecH9eOs5KDsl0gngFqyl6fuBdcaYYyPasEESkUJgjjHGpy/CE5GbgBbgBWPMVPu5/wDqjDH/Zn+hiDHG/NNItvNS9dOfJ4AWY8yvRrJtAyUiSUCSMeagiEQAnwD3AF/Fd8epvz7dh4+OlYgIEGaMaRERB/Ax8G3gceBNY8yrIvJb4Igx5pmRbOulukCfvgX8xRiz/lL2481HUJoSyYsZY7YDdec9/QXgefvx81gfHD6hn/74NGNMhTHmoP24GcjDyuTiy+PUX598lrG02D867JsBlgI9H+S+Nk799emyeHOA6islkk//IdoM8L6IfGKncxpNEowxFWB9kADxI9yeofCIiHxqTwH6zFTY+UQkA5gF7GWUjNN5fQIfHisR8ReRw0AVsAU4DTQYY7rsTXzu8+/8PhljesbpF/Y4PSkiQRfahzcHqEtKieSDFhpjZmNlc3/YnlpS3ukZIBOYCVQAvx7Z5gyMiIQDbwDfMcY0jXR7hkIfffLpsTLGuI0xM7Ey68wDpvS12fC2anDO75OITAV+AFwDzAVigQtOLXtzgBqVKZGMMeX2fRXwFtYf42hx1j5H0HOuoGqE2zMoxpiz9j9ZN/A7fHCs7Pn/N4CXjTFv2k/79Dj11afRMFYAxpgGYBtwPRAtIj3JFHz2869Xn263p2iNMaYT+AMXGSdvDlCjLiWSiITZJ3YRkTDgVmA0ZWnfAHzFfvwV4M8j2JZB6/kQt63Cx8bKPlH9eyDPGPOfvX7ls+PUX598eaxEZKyIRNuPQ4DlWOfWPgTW2Jv52jj11afjvb4YCdY5tQuOk9eu4gOwl4r+F+dSIv1ihJs0KCIyAeuoCaw0U6/4ap9E5E/AEqyyAGeBfwXeBv4PSAeKgbXGGJ9YeNBPf5ZgTRkZoBD4u55zN75ARG4EdgA5QLf99D9jnbPx1XHqr0/r8NGxEpHpWIsg/LEOGv7PGPNT+/PiVaypsEPAg/aRh9e7QJ/+CozFOoVzGPhWr8UUn9+PNwcopZRSVy9vnuJTSil1FdMApZRSyitpgFJKKeWVNEAppZTyShqglFJKeSUNUErZRKTf5a59bLtERG64ku25yPt/R0S+PAT7eVVEJg1Fm5QaahqglBqYJcCIBCg7u8DXgVeGYHfPAN8bgv0oNeQ0QCl1ASJyl12T55CIfCAiCXaS0m8Bj9k1bRbZV86/ISL77dtC+/VP2MlLt4nIGRF5tNe+v2wnzTwiIi+KSIRdK8dh/z5SrPphjvOatRQ42JNI1N73kyKyXaw6SXNF5E2x6gj93N4mTETesd/rqIjcb+9rB7C8V0odpbyG/lEqdWEfA9cbY4yIfAP4njHmu3Z9Hk/9IRF5BXjSGPOxiKQDmzmX8PMa4GYgAsgXkWeAycAPsZIH14hIrDGmWUS2AXdgZeV4AHjDGOM6r00Lseog9eY0xtwkVgG/PwPXYZUPOS0iT2Id8ZUbY+6w2xsFYIzpFpFTwIw+9qnUiNIApdSFpQKv2TnEAoGCfrZbDmRbKcYAiOzJuwi8Y6eo6RSRKiABu9ZPT+HKXqmGnsOacnsb+BrwzT7eKwkrV1tvPXkqc4DcnjQ/InIGK+lyDvArEfl3rIJxO3q9tgpIRgOU8jI6xafUhf0GeMoYMw34OyC4n+38gAXGmJn2LcUu0V+M7gAAAVdJREFUqAfQO3+aG+uLodBH+QRjzE4gQ0QWA/7GmL6Sabb30Y6e9+g+7/26gQBjzAmso6oc4Jci8uNe2wTb+1TKq2iAUurCooAy+/FXej3fjDVl1+N94JGeH0Rk5kX2uxW4T0TG2NvH9vrdC8CfsMoR9CUPmHjRlvciIslAmzHmJeBXwOxev54M5F7O/pQaDhqglDonVERKe90eB54AXheRHUBNr203Aqt6FkkAjwJz7EUPx7AWUfTLGJML/AL4SESOAL3LYbwMxGAFqb5sAi630OU0YJ9YFU5/CPQsnkgA2n0l87e6umg2c6W8jIisAb5gjPnSBbZ5C2vBxslBvtdjQJMx5veD2Y9SV4IuklDKi4jIb4AVwMqLbPp9rMUSgwpQQAPw4iD3odQVoUdQSimlvJKeg1JKKeWVNEAppZTyShqglFJKeSUNUEoppbySBiillFJe6f8B9lxbpUchZCQAAAAASUVORK5CYII=\n",
+ "text/plain": [
+ "
"
+ ]
+ },
+ "metadata": {
+ "needs_background": "light"
+ },
+ "output_type": "display_data"
+ }
+ ],
+ "source": [
+ "# Estimate my and sigma2\n",
+ "mu, sigma2 = estimateGaussian(X)\n",
+ "\n",
+ "# Returns the density of the multivariate normal at each data point (row) \n",
+ "# of X\n",
+ "p = utils.multivariateGaussian(X, mu, sigma2)\n",
+ "\n",
+ "# Visualize the fit\n",
+ "utils.visualizeFit(X, mu, sigma2)\n",
+ "pyplot.xlabel('Latency (ms)')\n",
+ "pyplot.ylabel('Throughput (mb/s)')\n",
+ "pyplot.tight_layout()"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "*You should now submit your solutions.*"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 5,
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "\n",
+ "Submitting Solutions | Programming Exercise anomaly-detection-and-recommender-systems\n",
+ "\n",
+ "Login (email address): tanmayss2109@gmail.com\n",
+ "Token: 9nfZTKeNcx3luLPi\n",
+ " Part Name | Score | Feedback\n",
+ " --------- | ----- | --------\n",
+ " Estimate Gaussian Parameters | 15 / 15 | Nice work!\n",
+ " Select Threshold | 0 / 15 | \n",
+ " Collaborative Filtering Cost | 0 / 20 | \n",
+ " Collaborative Filtering Gradient | 0 / 30 | \n",
+ " Regularized Cost | 0 / 10 | \n",
+ " Regularized Gradient | 0 / 10 | \n",
+ " --------------------------------\n",
+ " | 15 / 100 | \n",
+ "\n"
+ ]
+ }
+ ],
+ "source": [
+ "grader[1] = estimateGaussian\n",
+ "grader.grade()"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "\n",
+ "### 1.3 Selecting the threshold, $\\varepsilon$\n",
+ "\n",
+ "Now that you have estimated the Gaussian parameters, you can investigate which examples have a very high probability given this distribution and which examples have a very low probability. The low probability examples are more likely to be the anomalies in our dataset. One way to determine which examples are anomalies is to select a threshold based on a cross validation set. In this part of the exercise, you will implement an algorithm to select the threshold $\\varepsilon$ using the $F_1$ score on a cross validation set.\n",
+ "\n",
+ "\n",
+ "You should now complete the code for the function `selectThreshold`. For this, we will use a cross validation set $\\{ (x_{cv}^{(1)}, y_{cv}^{(1)}), \\dots, (x_{cv}^{(m_{cv})}, y_{cv}^{(m_{cv})})\\}$, where the label $y = 1$ corresponds to an anomalous example, and $y = 0$ corresponds to a normal example. For each cross validation example, we will compute $p\\left( x_{cv}^{(i)}\\right)$. The vector of all of these probabilities $p\\left( x_{cv}^{(1)}\\right), \\dots, p\\left( x_{cv}^{(m_{cv})}\\right)$ is passed to `selectThreshold` in the vector `pval`. The corresponding labels $y_{cv}^{(1)} , \\dots , y_{cv}^{(m_{cv})}$ are passed to the same function in the vector `yval`.\n",
+ "\n",
+ "The function `selectThreshold` should return two values; the first is the selected threshold $\\varepsilon$. If an example $x$ has a low probability $p(x) < \\varepsilon$, then it is considered to be an anomaly. The function should also return the $F_1$ score, which tells you how well you are doing on finding the ground truth\n",
+ "anomalies given a certain threshold. For many different values of $\\varepsilon$, you will compute the resulting $F_1$ score by computing how many examples the current threshold classifies correctly and incorrectly.\n",
+ "\n",
+ "The $F_1$ score is computed using precision ($prec$) and recall ($rec$):\n",
+ "\n",
+ "$$ F_1 = \\frac{2 \\cdot prec \\cdot rec}{prec + rec}, $$\n",
+ "\n",
+ "You compute precision and recall by: \n",
+ "\n",
+ "$$ prec = \\frac{tp}{tp + fp} $$ \n",
+ "\n",
+ "$$ rec = \\frac{tp}{tp + fn} $$\n",
+ "\n",
+ "where: \n",
+ "\n",
+ "- $tp$ is the number of true positives: the ground truth label says it’s an anomaly and our algorithm correctly classified it as an anomaly.\n",
+ "\n",
+ "- $fp$ is the number of false positives: the ground truth label says it’s not an anomaly, but our algorithm incorrectly classified it as an anomaly.\n",
+ "- $fn$ is the number of false negatives: the ground truth label says it’s an anomaly, but our algorithm incorrectly classified it as not being anomalous.\n",
+ "\n",
+ "In the provided code `selectThreshold`, there is already a loop that will try many different values of $\\varepsilon$ and select the best $\\varepsilon$ based on the $F_1$ score. You should now complete the code in `selectThreshold`. You can implement the computation of the $F_1$ score using a for-loop over all the cross\n",
+ "validation examples (to compute the values $tp$, $fp$, $fn$). You should see a value for `epsilon` of about 8.99e-05.\n",
+ "\n",
+ "
\n",
+ "**Implementation Note:** In order to compute $tp$, $fp$ and $fn$, you may be able to use a vectorized implementation rather than loop over all the examples. This can be implemented by numpy's equality test\n",
+ "between a vector and a single number. If you have several binary values in an n-dimensional binary vector $v \\in \\{0, 1\\}^n$, you can find out how many values in this vector are 0 by using: np.sum(v == 0). You can also\n",
+ "apply a logical and operator to such binary vectors. For instance, let `cvPredictions` be a binary vector of size equal to the number of cross validation set, where the $i^{th}$ element is 1 if your algorithm considers\n",
+ "$x_{cv}^{(i)}$ an anomaly, and 0 otherwise. You can then, for example, compute the number of false positives using: `fp = np.sum((cvPredictions == 1) & (yval == 0))`.\n",
+ "
\n",
+ ""
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 6,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "def selectThreshold(yval, pval):\n",
+ " \"\"\"\n",
+ " Find the best threshold (epsilon) to use for selecting outliers based\n",
+ " on the results from a validation set and the ground truth.\n",
+ " \n",
+ " Parameters\n",
+ " ----------\n",
+ " yval : array_like\n",
+ " The ground truth labels of shape (m, ).\n",
+ " \n",
+ " pval : array_like\n",
+ " The precomputed vector of probabilities based on mu and sigma2 parameters. It's shape is also (m, ).\n",
+ " \n",
+ " Returns\n",
+ " -------\n",
+ " bestEpsilon : array_like\n",
+ " A vector of shape (n,) corresponding to the threshold value.\n",
+ " \n",
+ " bestF1 : float\n",
+ " The value for the best F1 score.\n",
+ " \n",
+ " Instructions\n",
+ " ------------\n",
+ " Compute the F1 score of choosing epsilon as the threshold and place the\n",
+ " value in F1. The code at the end of the loop will compare the\n",
+ " F1 score for this choice of epsilon and set it to be the best epsilon if\n",
+ " it is better than the current choice of epsilon.\n",
+ " \n",
+ " Notes\n",
+ " -----\n",
+ " You can use predictions = (pval < epsilon) to get a binary vector\n",
+ " of 0's and 1's of the outlier predictions\n",
+ " \"\"\"\n",
+ " bestEpsilon = 0\n",
+ " bestF1 = 0\n",
+ " F1 = 0\n",
+ " \n",
+ " for epsilon in np.linspace(1.01*min(pval), max(pval), 1000):\n",
+ " # ====================== YOUR CODE HERE =======================\n",
+ " predictions = (pval < epsilon)\n",
+ " tp = np.sum((predictions == yval) & (yval == 1))\n",
+ " fp = np.sum((predictions == 1) & (yval == 0))\n",
+ " fn = np.sum((predictions == 0) & (yval == 1))\n",
+ " \n",
+ " prec = tp / (tp + fp)\n",
+ " rec = tp / (tp + fn)\n",
+ " \n",
+ " F1 = 2 * prec * rec / (prec + rec)\n",
+ "\n",
+ " \n",
+ " \n",
+ "\n",
+ " # =============================================================\n",
+ " if F1 > bestF1:\n",
+ " bestF1 = F1\n",
+ " bestEpsilon = epsilon\n",
+ "\n",
+ " return bestEpsilon, bestF1"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Once you have completed the code in `selectThreshold`, the next cell will run your anomaly detection code and circle the anomalies in the plot."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 7,
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "Best epsilon found using cross-validation: 9.00e-05\n",
+ "Best F1 on Cross Validation Set: 0.875000\n",
+ " (you should see a value epsilon of about 8.99e-05)\n",
+ " (you should see a Best F1 value of 0.875000)\n"
+ ]
+ },
+ {
+ "data": {
+ "image/png": "iVBORw0KGgoAAAANSUhEUgAAAagAAAEYCAYAAAAJeGK1AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjMsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+AADFEAAAgAElEQVR4nOyddXxUx9eHnxsPEJJAEgLB3YprcQhatAR3LVpKnZZCFagXqFKKu7sVintwCK4hIe6yycp5/9iQ0r607G4SCL/O8/kMm9y9M3Pukr3fOzNnztFEBIVCoVAocht2z9oAhUKhUCgehxIohUKhUORKlEApFAqFIleiBEqhUCgUuRIlUAqFQqHIlSiBUigUCkWuJMcEStM0F03TTmiadk7TtEuapn2UcXyBpmm3NU07m1Fq5JQNCoVCoXh+ccjBttOAliKSpGmaI3BI07TtGe+9JSJrcrBvhUKhUDzn5JhAiXkHcFLGr44ZRe0KVigUCoVFaDkZSULTNHvgFFAW+EFE3tE0bQHQEPMIaw/wroikPabuSGAkQN68eWtXrFgxx+xUKBQKRc5y6tSpKBHxtqZOjgpUZiea5gGsB8YD0UAY4ATMAW6KyMf/Vr9OnToSGBiY43YqFAqFImfQNO2UiNSxps5T8eITkThgH9BORB6ImTRgPlDvadigUCgUiueLnPTi884YOaFpmivgD1zRNK1wxjEN6ApczCkbFAqFQvH8kpNefIWBhRnrUHbAKhHZomnaH5qmeQMacBYYlYM2KBQKheI5JSe9+M4DNR9zvGVO9alQKBSK/x1UJAmFQqFQ5EqUQCkUCoUiV6IESqFQKBS5EiVQCoVCociVKIFSKBQKRa5ECZRCoVAociVKoBQKhUKRK1ECpVAoFIpciRIohUKhUORKlEApFAqFIleiBEqhUCgUuRIlUAqFQqHIlSiBUigUCkWuRAmUQqFQKHIlSqAUCoVCkStRAqVQKBSKXIkSKIVCoVDkSpRAKRQKhSJXogRKoVAoFLkSJVAKhUKhyJUogVIoFApFrkQJlEKhUChyJUqgFAqFQpErUQKlUCgUilxJjgmUpmkumqad0DTtnKZplzRN+yjjeClN045rmnZd07SVmqY55ZQNCoVCoXh+yckRVBrQUkSqAzWAdpqmNQA+B74VkXJALDAsB21QKBQKxXNKjgmUmEnK+NUxowjQEliTcXwh0DWnbFAoFArF80uOrkFpmmavadpZIAL4HbgJxImIIeOU+4DfP9QdqWlaoKZpgZGRkTlppkKhUChyITkqUCJiFJEaQFGgHlDpcaf9Q905IlJHROp4e3vnpJkKhUKhyIU8FS8+EYkD9gENAA9N0xwy3ioKhD4NGxQKhULxfJGTXnzemqZ5ZPzsCvgDl4G9QEDGaYOAjTllg0KhUCieXxyefIrNFAYWappmj1kIV4nIFk3TgoAVmqZ9CpwBfstBGxQKhULxnJJjAiUi54Gajzl+C/N6lEKhUCgU/4iKJKFQKBSKXIkSKIVCoVDkSpRAKRQKhSJXogRKoVAoFLkSJVAKhUKhyJUogVIoFApFrkQJlEKhUChyJUqgFAqFQpErUQKlUCgUilyJEiiFQqFQ5EpyMhafQvFMMegN6NMNmAxGjAYTRoMRo9GEmARNAzQNOzsNTdOws7fDwckBJxdHHBwd0DTtWZuvUPznUQKleG5ITdZx+/xd7l97QGx4PLHhccRFxhMbHk98ZAKpSTp0yWnoknWkpaRjNBht7svR2REnF0fy5HfFzTMfeT3ykM8jL/k881KgkAdFyvjiW7oQRcoUwrtoQewd7LPxShUKBSiBUuRSjAYjQUevEXT0GjfO3ubmmdvcv/YAkT/zWzq7OuFZyB2PQh54Fy2Iq5sLLnlccMnrjHMeZ1zyOOPo7IC9gz32DvbY2dth72CHpmmICCKYX02CyWhCn25An6YnXZeOPs1Aui6dlMRUkuOSSYxNJvxuJDfP3iHmQSwG/Z/iZ+9gT9HyhanlX4267WtSvVllnFycnsXHplD8T6E9+oXPrdSpU0cCAwOftRmKLCAipCSkEBMWR3xkArqUdNJT00nLKHpdOvp0A0a9kVsX7nJ862kSY5IA8CnuRdmapShboxRlapSkRJWiFPD1wDWf6zO5FqPRSHRIDKE3w3lwy1yun77F+f1BpOv0OLs6Ub1FFaq8WBGXvM44Ojng4OSAg6P51TWfC3nd85DXPQ/5PMyvrm6u2NmpJWHF/y6app0SkTrW1FEjKEW2k5aaxtm9lzi+5RQ3z98lNiyOmAexpKWmW1TfrUA+6r9Ui4ad6lCjRVXyF3TLYYutw97eHp/i3vgU96ZGi6qZx3UpaZzfH8TJ7Wc4ufMsJ7adsbhNl7zOlKxSjJJVi1O6WglKvVCc0tVLkL9A7rp2heJpokZQimwhNjyOo5tPcWxLIGd2X0CXkoZLXmcq1C1LwSKeFPD1pEBhTwr4euDhkx/nPM44uzrh5OpkfnVxzBxlOOdxwt7++V/TSU3WYUg3YEg3ZI4O09P06JJ0JMenkBSXnPGaQmRwFLcv3uP2+bvERyUCYGen0ahbPV6e8BJVGlVUjhuK5xo1glI8FUQEXUoaKQmphN2OYPNPO9m38ghGgxGf4l60GdycBp3qUL15FZycHZ+1uc8M17wukNe6OiJCbHgct87f48yeC2yfu5uDa49TrnZpXp7wEi80qUT+gvlwyeuiBEvxP48aQSn+EX26nhtn7hB8JcRcroZw70ooD26G/cVJwDWfCx2Gt6LtkBaUrFr8mdw4DXoDKYmppCbqMl5TSU3SoU8zYNAbzG7mGa8mk8m83qOBnZ0dmp2Gvb0dji6OOLk4/WVkl88zL/kLuj0zoU1N1rFnyUHWzdxK8JWQzONOLo64e+fHwzs/7t75KVO9JC36NKZ0tRJKuBS5EltGUEqgFP+P9DQ9O+fvZfn0dUQGRwPg4GhPkbK+FKvoh1/ZwuQv6Jbhgp2Xuu1qkNfdyqGCDcRHJXDpyFVuX7hH1P1ookJizOV+NHGRCTnadx43V/J7ueHu5YZ3MS9KVCpK8Up+FK9clGIViuDs6pyj/ZtMJs7vD+LBrXASopOIj4wnLiqBuIgE4iLiuXXuLkaDkZJVitGybxNa9m1MoRLeOWqTQmENSqAUWSI9Tc/OeX+wfPp6Iu9HU7lheV6e8BJlapTEt5QPDo5Pd0b4wa1wzu27xKXDV7h05CrBV0Mz38tf0A2vogXw8iuAt19BChYpkOEN50IeN1dc3VxxzeeCk4sj9o5mN3MHR3vzNWiAmG/6IiAmE0aDiXSd2bMwXac3exempJEYm0xCVCIJ0YnERyeQEJVI2O0IQm6EYTKaANA0jaLlC1O7TXXqdaj1TNzM46MSOLD6KHuWHeTS4asAVGtWmX7vd6dmqxfUqErxzFECpbAIESE6NIZ7V0L/Mn1369xd4iITqNywPAM/7Ekt/2rZdmNLT9MTeiOM4CshPLgVTnJ8CikJqSQnpJCSkEJyQirpOr3ZoSBNT2piKmF3IgGzV1+VFyuYS6OKlKtdGpc8OTtisfR67gbd597l+1w+fp1zey9mupkXKumNg5MDzhlThc55nHFydcIljzP5C+QzO4xklIKFPfAp4W1es8oGwu5EsHf5YTb/vJPI4GiKVfTDt5QPBQt74lPMi8Yv16PUCyWypS+FwlKUQCn+lbTUNDbM3sHqrzZmeoqBefqqWMUiFCnrS9vBLbJFmBJjk9gxby/n91/i3pUQwm6FYzL9+bdmZ6eRJ795D1Ce/K7kye+Ks6sTDk4OODo74ujsSKV65ajdtjrFKhR5LvYIpaWmcW5fECd3nCEmLC5j06+etJS0zBFZWko68VEJ6JLT/lLXwdGeF5pWpl77mtRtX5PiFf2y/H+QnqZnx29/cHLnGaJDY4kOjSU2LA4RoUqjCnQe3ZbG3Rv8px1ZFE8PJVCKx2I0GNm1cB+LPlxFVEgM9TrUpH6H2hSrWIRiFf0oWNgz20ZKty/cZcPs7exZepC01HRKVC5KiSrFKFahCMUr+lG8UlEKlylEHjfX//S0U0piKjEPYol+EEvMgzhunL7Fie1nuHMpGADfkt7Uf6k2HUe1oWSVYtnWb0J0IjsX7GPLzzsJvRmOh3d+2g5pQZdx7fEuWjDb+lEo/o4SKEUmIsK1U7c4t/ciOxfs5d7lECo1KMfwGf2p1rRyltpOiksmJiyOuIj4jGJeqD+3/xLn9wfh7OpEy75N6DKuHWWql8yeC/qPEH43khPbz3ByxxlO7TpHuk5P7TbVqdKwAm4F8uFWIB9+5XwpWbV4lqY5TSYTp3dfYMvPOzm6KRAHJwfaD2tF5YblMx8oVLgmRXaSqwRK07RiwCLAFzABc0RkpqZpHwIjgMiMU98TkW3/1pYSKMvQp+s5ty+IIxtOcHRzIFEhMQCUrFKMwZ/05sUudW0etUSFRHNg9TH2rTrM5WPXH3tOkTKFeGlka9oNa6kiIGQD8VEJbJ2zmy2/7Mr0pnyIpmn4lfOlVLUSlH6hBHXaVqdivXI29RN2J4IFU1awb8WRzAC7dnYaFeuXY+i0vlRvViXL16JQ5DaBKgwUFpHTmqa5AaeArkBPIElEvrK0LSVQ/05sRDyLP1zFnmUHSUlIxSWPM7XbVqdRl3rUbV8DD293m9pNjE1i34rD7F15mIsHryAilKlRksbd6lOkrC8ePu54+uTHw8ed/AXdnmlEbxEhMSaJyPvRRN2PJvK+2f08OSGFtJR00lLTSEtJQ5cR5fzR4LH2DvY4Ojng5mkeoeQv6GYuXm74lfWlcJlCzzyyhdFgJCkumbjIBIKvhHD7/D1uXbjLrXN3CL0ZDkDtNtUZMKUHVV6sYFMf6Wl6Qq4/4O6lYO5cDGbXon1EBkfTqFs9RnzeH7+yhbPzkhT/MXKVQP2/jjRtI/A90Ij/mkDduwebNkFcHHh4QJcuUCzr6woJMYns+O0Pln62lrSUdFr2a0zT7g2p2aqqzftydClphN+JYN/KI6ybuZWUhFSKV/Kjea9GNO/1IsUq+GXZbktJT9OjS9aRrtNnOhzo0/To0wwYDUbCbkdw9eQNrp26xa2zd9Cl/NXxQNM0s/NFHmdc8pg96ZzzOGNnb4fJaMJkzMgRZTCiTzOQGJNEUmzSX5w5wJx6o1jFIpSsUowSlYtRpkZJ3Arkw8HRLGz2jg44OpuDwLrkdcHZ1emprq8lxyezdc5uVn+1ibjIBGr5v0DAG50pW7MUnj62PZyA2elj7bdbWTFjPfo0PV3GtqP7653UWpXCJnKtQGmaVhI4AFQFXgcGAwlAIPCGiMT+W/3nVaDmTw+jz5FxuGxbDyZT5nGxs+NmtW6U3f49+Ppa1WZCdCKrvtzIyZ1nuX3+HiJCg461GfnlAJvEQ0Q4u/ci677bytWTN4gNj898r1G3evR972XK1Sqd7Tfc1GQdwVdCzG7aQfcJvhpCXGQCSbHm1BZJsUmk6/RPbMcljzNla5WibM1SFC5VyLw3qmhBvIsWpICvh9WjOpPJRHJ8innfU2QCwVdDzSOKoPvcvRRMxL2oJ7ahaRqu+VxwdXOhgK8HXkUL4uVX0Lxnq2hBytcpTfFKRXPkM93y8++s+nIjcRHm/8dCJbypWL8sleqXp1KDcpStWcrqtaWYsFgWfLCSHfP+QESo16EmAa93okaLqv9pRxeFdeRKgdI0LR+wH/hMRNZpmlYIiAIE+ATzNODQx9QbCYwEKF68eO27d+/mqJ3ZzfzpYTR9rxFluIU4OqJ17gwVKqA7fxX7LRtxxEC8V2ncLx6BQoWe2J7RYGTrnN0s+GA5yQmpVG9ehWpNK1O7TXUq1bd+7cFoNHJo3QlWfbmRa4E38SzkTv0OtfAtXYjCpXwoU7MUJSoVteXS/5F7V0LY9utuDm84QfidyMzcTg+jVBTw9SCfZz7yeeTFzTMveT3yZmy2NQeTfZhE0MHJnOOpYBFPilfye6rTb8kJKdw+f5eURB0GvTkQrEFvJF1nHu3pktPQJenQJZsDwkaHxWVGvXiYPgTMKUTqtqtJvfY1qdmqaramDnkYVf3e5ftcOXGdK8dvEH7XvOTr6OxIh+Gt6PdBgNWjq+CrIfyx7BBbft5FXGQCpauXIGBiJ5r3fhFHJ+Wqrvh3cp1AaZrmCGwBdorIN495vySwRUSq/v29R3keR1C6TgG4bFnLKWrxVrmNrDhkvtm3aAHxQffZ6dKFKrrT0L07rFnzr22d23eJHybM4/aFe9RoWZUx3w2hVNXiNtmlT9ezY95eVn+1iQe3wvErV5ieb3bGf0DTHPHaStelc3DtcbbO+Z0LBy/j4GhP3fY1KV+7DCUqF6V45aL4lfV96lEqngW6lDQi7kVx4YB5r9Tp3RdITdLh4GhPTf9qdHylNfVfqpUjghv9IJYrx69zbMspdi3ch7OrEz3e6Ez31zuSx806cUzXpbNn6UHWfruFu0H3KVDYk/6Tu9NhpP8zX6tT5F5ylUBp5rH/QiBGRF575HhhEXmQ8fNEoL6I9P63tp47gbp3D0qVQuztaVXyFnuvF8U7IyxaZCRUrgz7lwTjVa+0eervzp3Hrkmd23eJxR+v5ty+S/gU92LU14No/HJ9q6ZVkuKSuRZ4k6snb3L15HWCjl4jNjyeivXK0vPtrrzYpU6WbyqpSakE7jxHyI0wc+6nsFhiMnJARd2PQZeSRpGyvrw0wp/Wg5pnaV3kfwl9up5Lh69ycvsZ/lh+iKiQGPJ55MXT1wN3Lzc8C7lTskpxytY0J2r0Ke6VLVNqwVdDmP/BCg6uOUZe9zxUqFeWJi83oO2Q5laNhESEwF3nWD59HRcOXKZYhSK0GdScVv2bqnUqxf8jtwlUY+AgcAGzmznAe0AfoAbmKb47wCsPBeufeO4E6vvvYfx46N6diB/XULWqWZgAvL3h4kXw8QECAmDtWvP5Y8dmVo8KjeHXtxfzx7JD+BT3otOoNnR9tYNV+17C70Yyd9JS9q88kjmV5leuMBXqlsG/f1PqtK2RpZtdarKOE9vOsH/1EU5sPZ2ZjNA1n0tm3qcChT0o4OvJi13qUq1Z5eciGsSzwqA3cHRTIKd3nychJomEqASiQmIIuR6W+f/n5pmXMjVK0qJPE1oPbJrlabXLx6+z7dfdXD52jbtB9ylUwpt+k7vTemAzq0a0IsLBtcdYP2sbFw9dQdM0arSsSucxbWnUtZ5ap1IAuSwflIgcwhyW8+/8656n/wni4syvFZ7g7lu+vPk11uwjYtAb2DB7O4s/Wo0+3UD/DwLo/W5XqzzyUhJTWT59PWu/3YKdnUaPNzpRq3U1ytcpg5tnPluu5i9cPn6ddTO3cmxTILqUNDwLudN2SAua9mhI+dqln1ka9ucdB0cHmnRvQJPuDf5yPDVZx52Lwdw4c5ubZ+9w6fAVvh35M0s+Xk2PNzvTfngrmzfsVqpfjkr1y5lHQjvPsmDKSr4Z8TMrZqyn/5QetOzb2KLRtaZpNA1oSNOAhoTeDGPPkoP8vng/H3X/ilqtqzF+9jCKli9ik42K/zb/OoLSNM0F6Ag0AYoAqcBFYKuIXHoqFvL8jqB0HbtT+9YagoL4f1N8e/eCz5g/R1CnKzblp4nzuXMxmLrtazJ25hCr9p2E3gxj7/LDbPxhO7Hh8bTq14Sh0/riU8zL5ssw6A3cPHeXkOsPCLn+gAsHL3NmzwXcCuSjWY+GNO/ViKpNKj7zdQcRISUxlbSUNNJ1Znd0c1TydAx6458u5RmviICmoWkamgZoGnZ2WmZGXwcnBxydzK8ueZ0zoqO7PPPrfHitgbvOsXzaOi4cvIyHjzvNejSkdLUSVGlcMUsx/ESEo5sDWTh1JbfO3aVE5aJ0GduOhl3q4lWkgFVtGY1GNv+0i/mTl6PX6ek+sSM93uqsNnD/h8nWKb6MiA+dgH2YN9lGAC5AeaBFxs9viMh52022jOdOoO7dQ0qVwmCypxS3cK9clL17zW+1aAFBQdCyXDC7b5cm1ujIz50ns3fTGXxL+TD6m8E07FzH4ptMckIKyz5dy7qZWzHojVRrVpkRn/e3OaoAmJ/at8/dw5qvNxN53xzBQNM0fEv50PGV1nQa3eapjZRMJhNRITEZEddDuX81lPC7kSTEJJEYnUhiTBIJMUmZqS9yEpc8zri6ueDunR/vDNdx76IFKehXgMKlfahYv1y2RSS3hAsHL7N8xnouHrxMapIOAHcvN6o2rkiVRpWo3boapatZH7XcZDJxaN1xFn24irtB9wGoWL8cjbrWo/HL9SlazvIHp5iwWH59Zwl7lhzENZ8L3SZ0oNc7XZ/q56TIHWS3QL0kIlv/pTMfoLiI5LhyPHcCBdyoGUDZs2u55FKLQkc24FXT7AQREQF9Ggcz43pXQolnnmMN9HaO9Jn0Mr3e6WKxJ53JZGLPkoPMfXcJMWFxtB3cgkEf98rS4nRibBIbv9/B+lnbSIhO5IUmleg0ui0lqxajSJlCOZ6U7yFhdyI4tO44hzec4Mbp23/ZgJvHzRXf0j64e+U3x6bzzEf+gubXhyktnFwczVlxXRyxd3TA3t4Ou0eKppkHUWL+JzMnlEFvRJ9uwJjxqk/To0tOIzUxlZTEVFISzJl646ISMqNVPIwODmBnb0f52qWp2rgSLzSpRNXGFclfMOdHDCKSOcK9ePgKFw9ezowu8WKXugz5tI9NAWdFhHuX73No/QmObDzJtcCbALQe1Izh0/tRwNfT4rZuX7zHkk/WcGD1UQqV8Gb898Oo/1Jtq21SPL/kuJOEpml2QD4Rydn0pX/jeRQowsKIf6ER7lG3wMHBHD2ifHm4do1rG/Yx01ida1oBajWtwPg5Y6yao798/Do/TZzP5WPXqVivLGNnDbV5xPTw5rZz/l42/rCD1CQd9V+qRe93u1G1UUWb2rS0X12yjrhIcxLA+KhEbpy5zaF1x7h++jYApauVoFqzyhSvZM5aW7RCkWyNvJ4dGPQGYh7EcjfovlkgDl3hyokb6NP0aJrGC00r0bzni/iVL0I+jzy4eeajQGGPHBf7qNAYdi3Yx8ovNpCaqMN/YFO6jGlHmRolbXbpjwiOYvOPO1nzzWacXJ0YOLUnHUe1tupaLhy8zMzRc7gbdJ8m3eszZuZQq6cPFc8nOSJQmqYtA0YBRsxTfe7ANyLypa2GWstzKVAAYWEwbhysN0eSSMWeRVRhnVYOd2cY/d0Qmo/sYPENN+JeJL++s4R9K4/gWcidYdP70XpgM6u94+5evs+ZPRe4cPAyFw4EERsej6ZpNOvZkN7vdsv2CORJcclcOXGDy8euceXEdW6fv0dcZAL6tP8fKaJSg3I07lafRt3qPbex39J16VwLvMnp3RfYt/LwXzIBP8TLrwCFyxSiSGlzrL/ydcpQy/+FbF/nSohOZMWM9Wz4fgf6ND0ueZ3pPLotfd57mXweeW1q8/61UH58bT4nd5wlf0E3Oozwp/vElyyO+ahP17Pm6y0s+WQ1Ti5OjJs9jJZ9G+eqBw9F9pNTAnVWRGpomtYPqA28A5wSkWq2m2odz61APSQ4mBPT5zNrxRXC4/S81K8hw2e/YvENIjVZx8rPN7D6q00A9HyrCz3e7Gz1BsvQm2H8+s4SDq07DoB3sYJUa1aZak0qU7PVCxQu/eSIFpYSERzFxtnbObb1FPcuhwDmdawSlYtStlYpCvh64u7lRn6v/Lh7ueHu5Uahkuasr/9LPByhxobHkxibZA5oGxxN6K0wHtwMJ/RmODEPzF6cPsW9aD+sFe2Gtcz2UUVsRDzn913i6OZA/lh2CLcC+RgwtQcdX2lt04hKRDi/P4j1s7ZyZGMg+TzyMHRaPzqMaGXxA1PIjQd8MfgHgo5cpUGn2oz/fniWnHoUuZucEqhLmPctLQO+F5H9mqadE5HqtptqHc+zQMWGx/HT6wvYu/wwxSv5MfGXV6jauJJFdUWEP5YdYu67S4gKiaF570aMmNEPn+LeVtmQHJ/M0k/XsmH2duwd7On5dhdaD2yGb0kfWy7pX7kaeJO1325m/6qjAJm5jCo1KEeFumXI627bU/v/MqnJOgJ3nGXLnN85/ft57OztaNi5Dh1faUPt1lnPbvx3rp++xZy3FnF27yWKVSjC8M/707CT5Y45f+duUDCzx/3GuX2XqFivLBN+GknZmqUsqms0Gln33TYWTlmBnb0dQ6f1pdPoNrnCY1KRveSUQL2KedR0DngJKA4sEZEmthpqLc+jQIkI+1cdYfa430hNTKXPey/T652uFqfXDr4awnej5nB+fxDlapdmzHdDrF4TunnuDmu/3cLhDSdITdTRZlBzBn/aO9uezlOTdQQducq5fZe4cPAy9y6HkBCdSB43VzqM8Kfr+PYUKmGdmP7XCbnxgG2/7mHXgr3ERSaQ1z0PxSoUwa98YYqV96NohSLUaVudvPnzZKkfEeHYllP8+vZigq+GUrpaCdoOaUGrfk1w98pvU3t7lh7klzcXkRCVQMdRbRj8SW+L9949uB3OzNG/cmrXOeq0rc6UNW8qT7//MbLbi68hcEz+dkJGCCN7ETHYbKmVPG8CFRUaw6wxv3J0UyDl65Th7QVjKVHZMi8qfbqelZ9vZNlna3HO48yILwbQbmgLq9aZEmISWfDBSrb+sgtXN1cad6tPl3HtKFertK2XlElEcBTbft3N2b0XuXriBga9ETt7OyrULUPpaiUpU70ELfs1yfIN9L9OepqeQ2uPcfHwVe5fM7vXP3T5d/PMS/fXO9F1fPssf84GvYFdC/ax9dfdXAu8iYOjPQ271KXdkJbUblPN6pFMUlwy8ycvZ8vPu8jvlZ/R3w6mRe9GFo3ORIStc3Yze+yvVKhXlk+3TFL7pv6HyG6B+hmoB1wDdgA7RCQsy1bawPMiUCLC9t/+YM5bi9Cn6Rn8SR9entDB4pQPQUev8u3IX7hzKZjmvV5k9LeDrXLlNRqN7PjtD+a9v5yk2CQ6jW7LoI97ZUsEiegHsayYvp6tc37HaDRRvk4ZqjerTPUWVanaqIKKIPEUSE3WceP0bVZ9uZFjW05lq1AB3L5wl53z97J7yQHioxIpXLoQo78dTMNOVt1TALhx9jYzR83hyokb1GlbnY99lzsAACAASURBVFd/HEHhUpatcR7ecILPen+Lp68Hr/3yCnXb1rC6f0XuI6em+CoC7YG2mD349mIWrMMiYrTRVqt4HgQqNSmVD7t/xenfz1OtWWVe/3WUxV5oqUmp/DZpGZt+3IlX0QJM+HGE1XtETv1+jt8mLeX66du80LQS42YNs2mT5qMkRCdydHMg5w8EsX/lEfTpBtoMak7/DwJyzdSd0WA0526KSkSXrCMtJR1dijl7blpKOga9AaPhr8kJMwJJmCNJ2JkjStjZ22VGj3B0dsTR2fyaN78red3zmEtG+o/cEFPwauBNlny8OlOoWvZtQp22NajevHKWHxb06XqObT7FwqkruRt0n7rtazJsWl+rvTuNRiObf9zFvPeXYTKaGPnlQDqNbmPRaOry8et8NfQH7l0O4aWRrXn1x+G54nNX2M7T2AflijmKRHugobWd2UpuF6igY9f4auiPhFwLZeysYXQc1driL9P5A0F8NfRHwm5H0GVsO4Z81scq77yokGh+en1h5gbIYdP70bzXi1laWE9LTWP9zG2s+HwDyfEpuHnmpUHnOvR7v/szcf1OiE7k1vm73L5wj1vn7xJy4wFxEQnERyaQGJOENX/DWUXTNDwLuVOopDeFSvrgW9IH35LeFKvoR6UG5Z56XqSrgTdZPn0dgTvOkpaajoOjPVUbV6R2mxo07FTb4qnlx2HQG1g/azvLPltLUlwyTQIaMOjDnla3GREcxXej5nBy+xnaDWnBmJlDLBLRdF06Cz5YweqvN9NlbDvGzhqqXNGfY3JMoDRNqwU0xhyB/LCInLbNRNvIrQKVrktn4dRVrPl6EwX9CvDmb2Oo5W+Z970uJY157y1j/axtFC5diDfnjaFa08oW9200GNn4ww4WTlmJPt1Av/e70+OtzhY7YTy2TaOR3YsPsHDKSiLvR9OgU20GTOlB2ZqlnurTa2pSKse3nubAmqMEHb1GdOifCZfdvdwoVtEPT18PPLzy4+HjjoePO+5ebrjkc8EljzPOeZzMkSRcnXB0dsTO3g57B3vsHeywt7fjYSiJh9EkxCQYjSaMegP6NAPpaXoM6QbSdXpSE1NJiksmOT6F5PgUkuKSiQ6NJfxuJGG3I4i4F4XRYJ5IcMnrTI2WVanTpgZ129WgSBnrsiVnhXRdOhcPX+XUzrME7jrHrfPmBJ8t+zZm6Gd9szTiTYpLZs03m1k/cxu6ZB2dx7Rj8Ke9rZpWNJlMLJq6imXT1lGohBev/fIKtVtb5gj88xsLWfvtFoZ82oe+771s62UonjE5NcU3BegBrMs41BVYLSKf2mSlDeRGgbp++hbT+88i+EoIHYa3YuRXAy3+wl4+fp3PB84m5PoDuoxtx7AZ/azyWLpy4jrfjZrDzbN3qNuuBuNmD8vSzdBoNHJ8y2kWTFnB7Qv3qFC3DCO+GED1ZlVsbtPivg1Gwu9GEnIjjKj70ZzceTYzfUcBXw9q+r9AmWolKVWtBKWrFcezkEeueoo2GoxEh8Zw48wdAnee5eTOs4TdjgCgSFlfmvVoSLnaZfDyK4CXXwGb0tDbQvSD2MyoDyaT0O3VDnQd3z5LobASohNZOHUlm3/aRYHCHoydNYwmL9e3qo2Lhy7zzYifCb4aStvBLRg768mjKZPJxBeDvmfP0oNMnDOKDsNb2XwNimdHTgnUZaCmiOgyfncFTouIZZt5soHcJlDJ8ckMrTwROzuNN34bQ502lj0JGvQGlk9bz9LP1uLlV4A3542hRot/TSb8F1KTUlnwwUo2zN6Gp68HY74bQpPuDWy+YUeFRLN97h9s/20PkfejKVKmEEOn9aNpgO1t/hsiwrl9lzi6KZD710MJuR5G2O2IzBEIgIePO026N6B5rxep0qjCc7cfRkQIuRFG4M6zHNtyijO7z2My/fkds7PTKFm1ONWaVqZa8ypUa1rJJrduS4m8H838ycv5fdF+NE2jZquqDJjSw+K9eI/j0Qek1oOaMW7WMKumpdN16Sz+eA2rvthATf9qfLLpnSdOjerT9XzQ+XNO7TpHp9FtGfFFf+WG/pyRUwK1HegjInEZv3tg3gfV0WZLrSQ3CVRaahpfD/+JfSuOMPvYNCrULWtRvVvn7/LlkB+4ceY2Lfs2ZuysoVa50J7ceZaZo+YQfjeSTqPbMmx6X5s8tx6ma9jy8y6ObTmFyWiidpvqvDSyNQ071c6R1OuxEfH8vnAf2+buIeT6A1zyOONXvjBFyvriV7YwfuUK41fWFy+/AviU8HruROnfSIxNIvxuJFH3Y4gKiSEyOIorJ64TdORaZhDcklWLUb9DLbqMy9oI598IvRnG7sUH2DZ3N9GhsbTq14Thn/e3eU+cQW9gySdrWD5tHYVK+vD2grFWi96O+Xv5etiPtOrfhLcXjHviNHJaahrzJ69g3XdbKVymEG8vGEeVF5+Qc02Ra7BFoMxz8I8pwGxgFrABCAEWAPOB+8CKf6qXE6V27dqSG7gaeEOGVp4g/lqALPpolUV19Ol6Wfzxamnn1EsCCg2TQ+uPW9VnXGS8zBg4S/y1ABlSaYJcOBhki+kiInInKFje8v9I/LUACSg0TOa+u0RCb4bZ3N6/YTKZ5PyBIPm09zfSzqmX+GsB8lqTyfL74v2iS9HlSJ/PE+lp6XLx8BVZ+tlaebvNx9LGoae0dewlMwbOkhtnb4vJZMqRflOSUmXe+8ukvXNv6eTWX1Z8vkHSdOk2t3fhYJD0LzVa/LUAmTnmV0mKT7aq/pJP14i/FiDvtvtEIoKjLKpzdt9F6VdytLSx7yFzJy3Nkv2KpwcQKFbe+/9tH9SgJwjbQquUMAvkhhHUuX2XeLftJ3j4uFs8rSciTOv7HftWHqF570aMmzXUqumci4cu82nvb4mLSKD3u13p+97LFqfjeJSE6EQWf7SazT/vwiWvM4M/6c1LI/2z1eMsNVnH1RM3CDp6jcvHzCU+KhE3z7y0HticDiNaZcmj7H+dsDsRrPtuK9t/24MuOQ0PH3cq1i+Lf/9mvNilTrZ7B4beDOOn1xdwbPMpfEv5MGxaX5r1tM378+HU8/pZ2yhaoQhf7J5i8chMRNj80y5+fXsxTq5OzNg52aIN5ckJKfz8+kJ2zPsD/wFNeWfheKvtVjxdctzN/FnxrAXqwe1wxtWbhLt3fr479IlFU3Ox4XF8M/Jnjm0+xaCPetH/gwCL+9OlpDH//eWsn7WNQiW9mbr2TcrWsCy22aPo0/VsmL2DZZ+tJSUhhfbDWjHok954+lgWddoSkuKSWfP1ZtbN3JqZNK9YRT8qNyjPC00r0aznizanJP8vkhCTyP6VR7hy8gZn9lwgMjgaDx932g1pQYcR/tka0BfMU8e/TVqa6XAz/vvhNvdxdu9FpnT5nAKFPflyz1SrpiuDr4Ywqd1nJMUlM23be1RuaNnU3S9vLmLdd1uYtHQCzXs1ssluxdMhW6f4HhbMKd/PADFAApAIJFg7VMtKeZZTfCmJKTKy+hvS1XOQBF8LtajO4Q0nJMBnqLR36SNrvtksRqPR4v4uHLosg8qPz5wySUlMscnu8weCMqcjJ3X4TG5fvGdTO/9ESmKKLP1srXT1HCT+WoB80utrObb1lMRHJ2RrP/9lDAaDHNt6Sj7oMkPa2PeQ1nY95N12n8jZvRezvZ91M7dKJ7f+0sG1jyyfvk706Xqb2rp4+Ip0zj9ABpQZK+F3I6yqG343QgaVHy8d8/WTM39csKhOfHSCjG84Sfy1AJkxcJbVU4yKpwc2TPFZIlA3gGpkjLaeRXmWAjV9wExpY99DTuw4Y9H58ycvF38tQF6p+abVorDyiw3S2q6H9C81Wk7vOW+LuZKmS5fZ4+aKvxYg/UqOlmNbT9nUzj8Rfi9Sfn5joQT4DBV/LUAmd5ou18/cytY+FP+fiOAoWfThKunlN0L8tQB5s+VU+X3xfklJSs3WPj7s/qX4awEy/IWJcuPsbZvauXz8mnTxGCgBhYZZ/L15SPSDGBn+wkTp4NpHLhy6bFEdg94gC6eulDb2PaR/6TESGRJti9mKHCanBGovYGdtw9lZnoVAGfQGWTBlhfhrAfLLmwstqhN8NUTaOPSU6f1nSnqadQu3vy/eL/5agHzU4ytJTrBt1BQRHCXjGpifJn+YMC9bb15JcUky990l0sG1j7Rz6iWTO0+XS0evZlv7CstIS02TVV9tynRM6Jivn8wYOEsuHbmSbX0c2XRSehYZIe2cesmST9bYNJq6ffGeDH9horS26yELp64Ug8Fgcd34qATpW2KUdM4/QPYsPWBxvQuHLksH1z7yYfcvrbZXkfPYIlCWuJnXBT4B9gNpj0wNfmPVXGIWeNprUBHBUczoP4sLBy/TemAzXv1xhEXrKJ/1/Y5jmwJZdPN7PAt5WNRXckIKP742n10L9lG1cUU+3/WBTY4QZ/64wLS+M0lLSePNeWNoGtDQ6jYehz5dz5aff2fJJ2tIiE6kVb8mDP6kd47kkspu9Ol6EmOSSIr7MwpEclwy+jQDDg/j7mW8uuRxwqtoQbz8CuSIq312YzKZuHT4KrsX72f/6qMkx6fQok8jhs/ony1J/+Ii4/nh1XnsW3mEMjVK8ua8MVavg+pS0pg15ld+X7SfWq2rMWnJqxZn3Q27E8GMAbO4dPgq/gOaMm72MIu2Vaz4fAO/TVrK1LVv0ribdZuIFTlLTu2D2gUkARcA08PjIvKRLUbawtMUqMBd55jW9zv0aXpe/XEErQc0s6je1ZM3GFd/Er3f7cawaX0tqxN4k097fk3EvSj6THqZ/lMCrL45pqfpmfPmIjb+sINiFYowdd1blKhU1Ko2HoeIcGj9Cea+s5jQm+HUaFmVkV8MyJaUHVnFZDKRFJdMfGQCcREJxEUmkBCVQEqijoSoBO5dCeHe5fuE3gzHZDQ9ucFH0DSNgkU88SnhjU9xL4pX8CO/lxv5PPKSz8McMLaArwfexQo+9bh7/0Rqso5VX2xk1Zcb0TSNXm93JeCNjtkSYf7Q+uPMGvMrCdFJ9JnUjQFTe1gV9krEHOH/+/G/4e7lxofr36ZCnTIW1TUajCz7bB1LPllNoZI+fLjurScGQDboDYyt9y5xEQnMC/pWJcjMReSUQAVa22hGvWLAIsAXs7DNEZGZmqYVAFYCJYE7QE8Rif2nduDpCVR6mp4BpcbgViAfH65/m6LlLAuMun/1Ub4Z/hNOrk78FvStRV5+yfHJjHjhDTQ7jfeWvWbThsPkhBQ+fPlLzv5xka7j2zN0Wt9s2V1/48xtfnp9Aef3B1GySjFGfDGAuu1qPPUQQyJC+N1Ic6DY8/e4deEut8/fJfRm+F+iTzyKvYM9fuV8KV6pKMUr+lGwSIFMYXkoMg5ODhj0Rox6I/p0A4Z0A6lJOiKDo4i4F0VEcBQRGbH2wu5EPrYfOzuNgn4FzMFiS/lQrIIftVpXo1ytpxu38FHC7kQw5+3FHFxzDA/v/PR9vzudx7TNcmilhJhEfpq4gN2LD9BpVBvGfT/M6mu8ceY2U7t9gaZpzNj1gcXfLYCLh6/wccBXFK1QhG/2ffzE84OOXWPCi+/T+91uDP2sT64KjfVfJqcEagbwh4jsstKYwkBhETmtaZobcApzHL/BQIyIzNA07V3AU0Te+be2npZA/b54P18M+p7pOyZbtM8pXZfOL28uYtOPO6nUoBzvL59ocVDOb0f+zI55fzDzyGdUrFfOalujH8TyXofPuHvpPm/8Ntrikd6T2pz//nJ2LdxH/oL5GPRxbzoMb/VUYsc9yv1roexcsI/di/cTFRKTebxw6UKUrlacouWLUMDXE3fv/Hj45M8MFuvq5opLXudsjUSRnqYnOS45M2BsYmxGsNg7EZkBY8PuRBAZbE4m6O7lRq3W1ajbtia121SzKp9XdhF07Brz31/G2b2XKFOjJK/9PNKmv7FHERF+m7SUlV9spEaLKrw1fyw+xa0LQBt07BqTX5qGQW/k1R9H4N+/qcV1V36xkbnvLmHupW+fOEMgIkzp+jnHNp+i9aBmvPqDZVP0ipwlp9zMEzGPgFLJgps5sBFoDVzFLFwAhYGrT6r7NJwkTCaTjKr1lgytPMGiXfwhNx7I6NpvZTpRWLOQfOr3c+KvBcictxbZZOu9K/elf6nR0jFfP6u9pB6HPl0vy6evk475+kk7p17yy5sLJTE2KcvtWkNcZLxsm7tbJjR+X/y1AGlj30Pe7zhNNv+8Sy4dvWqz48jTIjYiTvYsPSAzBs6SgELDxF8LyHQLP7zhxFOPnmEymeTAmqPSs8gIaW3XQ2aPm5tlF2yTySTbf9sjndz6SxePgbJ7ieUODA8JvxshrzWZLP5agHw+aLbF/68x4XHSzqmX/DRxvkXnGwxmz77Wdj1kRLXX5d6V+1bbqsheyAkvvuwomKfz7gH5gbi/vRf7pPpPQ6AuHAwSfy1ANv2084nnmkwmGVhunHQrMEiObDppVT9XA29IzyIjZHCF8TbdtC4eviI9Cw+XAJ+hcuXkDavr/53Qm2Eytt474q8FyJSun8v965bt9coqyQkpcmxLoPz42nwZUe118dcCMsM5rfh8g0SFxjwVO3ICo9Eo10/fkgVTVmS6hbd17CWj67wts8b+Kmf+uJBjoYz+TlJcksweN9e8faH0GLkamD1/Mw8fJHbM/8Pq+o+6hX/c82uL633a+xtp79xb9q8+YnGdkzvPSnfvIdLLb4QKsfWMyVaBAkr+a0XQgKJP7ADyYZ7eeznjd4sEChgJBAKBxYsXz6nPTETMsb0CfIZKF4+BEhsR98Tzr5+5Jf5agGz/bY/FfZhMJtn88y5p79xb+hR/xeq9Q0ajUZZNWydtHHpK/9Jj5NaFu1bVfxx7VxySzu4DpIvHQKu+9LZi0Bvk98X75bUmk6Wtozk+X3uXPvJ2649k2bR1cvn4tad2435aGPQGOb7ttPz23lJ5s9WH0smtv/hrATK6ztuyd8UhMegtd7/OChcOXZY+xV6R9s69ZdVXm6xy+34cBoNBXm8+RTrnHyBhd6zbkPuQ+R+Y9wxaut8qKjQmc1PunLcWWfzZnd13Ufy1AFk/a5tNdiqyh+wWqNXAWmAgUAXwAYoDLTG7nR8BWv9r4+AI7ARef+RYrpniM5lMsn7WNmnr2EuGVHxV7l62bBpg0UerpLVdD4kJi7Xo/JSk1MyAr++2+0TiIuOtsjP6QYy83doc5PXT3t9IUlzWpt9Sk3Xy9fCfxF8LkFdffE8e3A7PUntP4qEwDa5gjpAxrMprMnfSUjm1+/x/7qk2LTVNtvyyK/Oz6F9qtKyftS1b96z9E3GR8TKl6+firwXIhMbvZ3m0HHorTDq59Zc3W061KlrKQxJiEqWLx0Cr9i2l6dJl5ug55s3KrT606IFSRGRi0w+kd9GRKrDsMyTbp/iAysBnwL4MYTkDLAP6Ay5PqKth9uL77m/HvwTezfj5XeCLJxmZUwL18AlucufpFt/0Lx6+Ip3dB8irjd636Hyj0Shj6r4jre16yKKPVln9RY4IjpIevsPkpTx9Zeuvu7M8wrh35X7mBsq5k5baHNLGEmLCYmX59HUysOxY8dcCZGSNN+TQ+uM23cz+1zAajXJo/XF5tZF5qqy79xBZMGVFjk9tmkwm2bVon3TxGCgv5emb5Ugj2+buFn8tQGaPm2vTaHDh1JXirwXIuf2XrKq3c8Fe6eDaR/qVHG3ROlbgrrNWz3oospdctQbFnynizwNnM0oHoCCwB7ie8VrgSW3lhEDFhMdJB9c+8kmvry2+YR7ZdFI6uPaRQeXHW/z0eXaveXphyy+7bLLz6+E/SXvn3jaHnXnIwwXujnn7SbeCg7PFueKf0KXoZOlna6Vjvn7irwXIxGYfyMF1x5Qw/QMXDl2WyZ2mS2u7HtLWsZdM6/edBB27lqN9Rt6PktG135KX8vSVi4dtj0JhMpnkhwnzMsMvWTqieUhCTKL0LTFKXsrTV/auOGRV3ePbTou/FiD7Vj15etpkMkkP32HyxZDvrepDkX3kKoHKzpITAjXv/WXS2q6Hxd492+buljYOPWVsvXes+hJ+Pfwn6eTWX1KTrZ/KCjp2Tdo49JTvX/3N6rqPkhibJJ/0+jrzJhJ537K8O9ZiMplk36oj0q+kOQzP1Je/UN5TVnD/eqj8+Np86ew+QPy1ABnXYJKcP2B7/q8nERMel+nsc+v8nSy1tXPBXmnv0kf6lhgl107dtM6OsNhMz765k5ZavD5mMBjkZa8hMr3/TIvOf7fdJ9LDd1i2rN8qrEcJlIUkJ6RIV89B8lGAZXPfq7/elLl+ZE108ZTEFOniMVBmDJxltY2/L94v7V3MUxiWrnU9jpvn7piTuzn0lGXT1mV5cfyfuH76lkxs+oF5Kq/6GxZHo1b8f5ITUmTD99ulT/FXxF8LkOkDZlo9MrGU0Fth0rPICOmYr5/8sdy6EczfuRp4Q/oUf0U6uPaRA2uOWlU3PS1dvh35c+aUu6VTz58Pni1dPQdZdP6Ns7elZ5ER0jn/ADm127ZgzArbUQJlISd3muejT/1+7onn6lJ00jFvP3m/4zSr1msSYhLl9eZTxF8LkLP7rEuPcGj9cWlj30PebDnVaoeKR0lN1smg8uOll9+IHAvsqkvRyZy3Fkkb+x4S4DNUtvyyK8dE8L/Go9lvX/YaInuWHcwRL8fI+1GZDxebftyRpbZiI+JkdJ23pWeREVavSZlMJln1lflh0FKv0r0rDllld0RwlAyvOlG6FRhkdToQRdbIEYEC9lhyLCdLdgvUpp92ir8WYFGK6WNbT4m/FmDVms2D2+EytPIEaefUy6pozCLm/VgdXPvIuAaTsuTZFRMWKxObmW86tqbueBLn9l+SgeXGib8WIN+M+Ompb+79T3D3rtx+b4aMKzpQ/LUA+aDN1BxJJ5GWmiaTO08Xfy1AVn25MUttHVp/XPy1AKv3CIqYp+36FHtFJnX4zKLzdSk6eaPFVPHXAmTlFxssEvD710Olc36zo1NOOgkp/kp2u5m7AAWAc4Bnxs8FMjbdXra2o6yU7Baon15fIO2de1u0aP/dqF+kY75+FrunXjt1U3oWHi5dPQdZnVjuzqV70tVzkAyuMD5LI6fLx69J76Ij5aU8fa0WSEtISUyRmWN+NbtJlx6TYwL4X2betAeS2rG7iJ2dCIgBZDXlpIPWTTo7dJfdP2zK9j716Xr5tPc34q8FyMKpK20erenT9dLDd5iMbzjJpugV8ycvlzb2PSx6gBQxu54/XGP9YcI8i77Xe5YdzFzzUjwdslugJgC3MafYuP1IOQeMs7ajrJTsFKjrp29JZ/cBMq7+uxadP6j8ePmgywyLzjUYDNK3xCjpW2KU3LlkfQbbN1pMlYBCwyT0VpjVdR/y4Ha4dHYfIP1Lj8mRRILh9yJlVK23pLVdj2zPOZXTfP+9SPhjtnyFh5vfyy3Mm/ZAblBaBMTk6CjSvbvIe+9Jasfucgt3eZXm4q8FyPyJc7N9ys9gMMiXQ34Qfy0gSy7oe5YdlLaOveTVF9+zOit0yI0H4q8FyLqZWy2uYzQa5cfX5ptdyedZFt3iyyE/SGu7HhL94PmNWvI8kVNTfOOtbTS7S3YJ1I2zt6VbwcHSp/grFolAbESctHHoKb+9Z9lT1sO9Fpa4vf6dhJhEaePQU+a9v8zqug8xGAzyWpPJ0tl9QI5svj22JVC6FRwsndz6y/Ftp7O9/Zzk++/Nf+2VK/9VpMLDzccg94hUasfuIiCB1JIW5YIlPPxPO/0IlvPOteRraom/FiA/vjY/20VKn66X/qVGy/AXJkp8VILN7RxYe0za2PeQt/w/krTUNKvq9in2ioys8YbER1vev8lkkpE13pChlSdYNIq6dPSq+GsBcmDtMatsU9hGTgnUwMcVazvKSskOgbp1/o687DVE+hR7RUJuPHji+dEPYmR4VXPqaUtj3k3r9510KzDI6i+jyJ+LvVnZk7Lk0zXirwXI74v329zG49Cn62XOW4syN9sGXw3J1vafBo8K0UORetyxZ87duyJ2dmJydJQW5YIFRLy9zeWhnZGn74nJ3kF+0GqIvxYgXw//KdsdU45tPSXtnXvL4ArjJfSm7SP6XQv3mdfOusywar3n2JZAae/cW0ZUe90qL9Y9Sw9YvP6VpkuX9i595Oc3LMuYrcgaOSVQsx8pvwK3gDXWdpSVklWBSklMkR6+w6SX3wiLNtim6dJlRLXXpWPefha7S6cm66SDax+ZOXqO1fYlxibJ8KoTpbv3EJtvNPeu3Je2jr3k0z7fZusTtdFolEkdPhN/LUC+G/WLTeKbW3hUkP5+088V4iQiMnu22aju3SU8/E8bH9qcaWf37mICmdfOHGh3Wr/vJD0te8P4XDgYJN0KDpYevsOyFOFi4w87bIref+r3c9Ixbz8Z/sJEi6/NoDdIv5Kj5c1WH1p0/mtNJlscFUaRNWwRqCdmHROR8Y+UEUBNwPqc5M+Qo5tPERsez7uLX8Wv7JMTpa3+chO3L9xj8sqJ1GhR1aI+7l8NJV2np0ZLy85/iC4ljcmdpnP/WijvLH7V5lxG2+fuQUQY/c2gbE3QtubrzZzcfoaxM4cy4aeRNqWjf1aIGBBjBKK/woZVR3GSrRz9YzFT35pLz06L6NJmJaMGb+DTD7bh7b4fMdxFxMAPP0BExP9vLyICfvghh42OizO/VnhCAsvy5dGAIY08GfJpH/5Ydoh3235KQkxitplStXElvt73EcnxKXw36peHD6xW03lMW5oENGDXwn0YjY9PNPk4avlXY9KyCdy5GMzmnyxLR2fvYE+DjrW5dvKmRfYWLlOIyOAoi21SPF2syy9uJgXIWvazp8y+lYfx8itAtWaVn3hu6M0wlk1bS7OeDan/Um2L+wi5/gAAPysyherT9Xzc42uCjlxj8sqJ1G1bw+K6j/L7ov2s/XYLTQIaZGuCC3QtLQAAIABJREFUvKuBN5k/eTmNX65Pl3Htsq3dnEJMcZB+BtGfgvTToL+A2ccHOjfFHHgLmPL6Y+pm5HQ2Gh1pWrUEZ/aVplHT0uT1rA9OdYmMdKJFCwgKMp83dmwOXYSHBwC681dp0QIiI8E7Iy9gZCS0aAF794LPtWvmg56e9B37MoVKePP1sB+Z8OL7fLJ5klUZa/+NUlWLM+TTPvzy5iJ2Lzlgc2LM5j1f5OCaY1w8eIXqzatYXK9hpzrUal2NJR+vxn9AU4uyVRevVJSUxFSiQmLwLlrwX8/18MpPQlT2iboie3niCErTtM2apm3KKFsxB43dmPOmZQ9Jccmc3H6GZj1ftChN9Q8T5uHg6MCobwZb3IfRaGTbb3twcLSnSFlfi+t9+8ovnNx+htd+HknTgIYW13uUPUsP8uWQH6jeoipvzsu+u+bFw1d4p/XHePp6MPGXV3Jl2mwRI2uWnyEp7GtMUS8hEfWQuFcgeR56fTrnrvdByz+VBG0mQ15fzAvNtlKi9lHylzlDhUbHKVnnAOUa7KaW/zbuxK9Ayz+DNPvBhEeVoFiR6zga5iKxQzCG1+f03ldpWncpndpfp0eAKecuqnNnxM4O+y2biA+6T+XKcPGiuVSubBbIPo2DkY0bwc4OOncGoFW/JnyxewoJ0Um82vA9bpy9nW0mdZvQgSqNKvDjhPnEhsfZ1Ebd9jVxdnXiwJqjVtXTNI1RXw0kOT6FZZ+utahO8Up+ANy7fP+J5+b3yk9aajqpyTqr7FI8JZ40Bwg0e6Q0woIcUNldsrIGdWr3eYujRsRFxou/FiALpqywqo+HzgnrZ1uebyb0ZliWsuqKmPPjdM7/f+ydd1xT1/vHPzcQ9l6C4sKNu+69ELWiFgngRtx71Lq1bq1Vq9ZV9957VOusC/dCQEWQKXtvEpI8vz8uQaRo7k2C4/vL+/U6LzCcc+4Jkvu55znPGExT28/XaNmKyNfvyd3Kh3xqTlS51o+mUbiIy+X5JM+7TLL0WZQV3pJkcTVIHF2b8uIGkzxrM8nFDykhPvcjzzxPT6IzZ4j09dnXGIbI0pL9vjDMiJydiQID2f4rVhDVrElkaJhLA0TXaefauRT5tC3J4mqwLbELybP3UUJ8dpl4/oU0Yr34Ag1+oKRnH8IVEhKIOteIoif4oeicqiQxoXHkaT+c8xkMV8JeRqidaWJW9yU0uvEvKo1dJFpF/RxHceobH5FILoyILu64prTvnVMPyIUR0aHlp1RalxbuoKxSHQGwB9AbQC8A9nwvom5TR6D+3naVXBgRpxutIgI+4A73BJ3+t4LIVceTlg9cx8s5QVFTKiEqifOYkiwbsJZ6GPTXaBXc1Pg0GuQ0jo3HUsN7S5Ns3EgkFIppwYxDJIkrFIr4JpQaMZUGeZ4nC/N0srcv3TOvVy/2e13dD+KkcDpQNBMT9qu9/YfXatUiMjb+8G8dHTl1bBdJw/ofoyh/T5LF1aC0t01oxdzfafdODVchjoujdBunDwv38CCaPZt1jFC8EScnovjS/38UuSPV8QgtiVwupwGVx3DOX1kaW6bupp5GA1Qqy3H6z4ucs7/kZuWSCyOiIyvPKO0rl8tpifcactXxpMeXX/BelxbulIlAARgBtlz7HgB7AUQAGMb3Quo0dQRqx6wD1F3Pm5N33Japu6mHQX/OWSPSEtPJu8JI8qk5kVNNGgVyuZx8ak6kXzov4DymJIqd4d4FR1WeoyR5Ofk0ocUs6mk0gN48CtHYvOogl8soPf4kRT7rRLK4GvT4ijelxt+ihHhJkRApdkbFPfPs7IjmziW6devDz0trurpEFhYf/l2tGnvv/1T/ypXZPi2bPKMjWyeR5H0tksTUptcPp5FcqkGhiotjhUmxxVM0gYB9/RPiRMTm8POw9aVZ3Zdobj3EBra6Ww9VuWyK39lH5MKIaMvU3aw7/YYNREuWsF+jPh/Y/uZRCK/SGv0rjaYhNSZQZmqW0v652Xk0ssHP5G7l8808lP0vUlYCFQzAuti/rcGhCq4mmzoCtXzgOhpcbTynvlPbz+flcrpj1gFy1fXinbEh6s17tWpEERHN6LqIBlQeo1HT3rFVZ8mFEZHfmUcam1Md5JI3JEv2IllcDcqPc6eRPrcIkH8kRPb2RFOmEFlbly4odnasSAmFnxadkgJUsSK3vjo6RJUdo2nVguWUFVafciKa0MlDl4rMkRox/0VFsRMtWcJ+VXIjV3D09zNq/42V5Or+W2yGiQtPVJ5jXOsNbH5IptxHv0y5QMCaNuNKj1GUiCXUTejNOWg+4O5r6ib0prWj/uLUPyY0jnqbD6al/f7g/F608EMVgVLuNQC8B1DczSULQLSqZ15fmuz0HJhYGnPqm/Q+BfZVbDn1JSLcOfUQjTrXQ/VGVXmtKep1DACgemN+4xRI8iUIvPsGbd1bQN9QX6U5SkJEuLL3Jpxb1UTrPs00Mqc6a6GcfaCUvoAsEoz5SuiVO4mlv7eHrS2DpCTWo83EBIiPB9at++CdDbC+AwoSE4Hu3QGplNu1IyOBaI5/3TIZEPneEdMXzUYH9wsIflcFP3WaBN28eWjeLA8TJmjALb1iRdZlcN489mvFipyGefzshmbdG2HjxJ0I9Huj5iJY2vZtgYq1K+CPkX8hLTGD9/jdK+Ix8d462FM2tqABZO4ewJw5yHfzgFQuQPUXJ5FRvw2QkPCfsUI9ISzLmSM1jpuTRr02tdFrjCsu7byO8MAopf3LV7NHlwHtcO/sY+Rk5vJ+b1rKBi4CFQPgIcMwCxmGWQDgAYBQhmF+ZhimFIfdb4ucjFwYmxsp7UdESIlNg015K07zRr56j5iQOLR1b8F7TdFvWIFyrFWe91gACLoXDEl+ARp3qa/S+NJ4+zQMEUHRcPXpqLE5VYHkGaD08aCspYB+GzA2F8EYupfqRaivD1SuzH6vCK8RCAB5CSe7vDz2Mb0seeZfCc27HcZvG0Zh+IDjuLC/L+rXeYOsr+TBrKOjg9kHJ8Ousi0Wi1Yj6X2K2nMaGOlj3pGpyErLwSrfTZCX/EUrof+9CaiNd+iCHITDHEdbjkTi5GVoEnYCVRGOIIMfYJ4c9kkffkt7C6Ty8CIc9KsIhqaG2D7zAKf+LoPbQ5JfgDsnH3K+hpayhYtAvQNwBkVRJDgLIA6AaWH7psnJyIWxmaHSfllp2SgQF8DKgVsc0YPzTwBApd3G+7dxsHKwhLGZcuEsDf+bQRDoCFC/fR2VxpfGv4fuQKgvRAev1hqbky8kzwKleAHiW2BMZ4Gx2ApGwD4wJCYCDRt+iAuytQVSUoD3JTyJed4zNYpUKsTc5b+ge79dsDTPxMNLIoz29ftq6zG1NMHiMzOQnyPG6uGbNTKnU4PKGLPGB48vPce1/be5D4yKgsHF0yChEPuqH0QaOWP77KOoV1eGV68Ac2dHlLt3BtDVBU6fLnUba2VvgdS4NM6XNLcxw8B5Ijy+9Byv7gcr7V+nZU2Ur26PW8e+3v+Zlo/hkkli0efal1ikOkjyC6BnqDz7QU4Gu63nag4MD4xCucq2sOYoaMVJT8qAtYMF73EKYkLjUa6yrcoCVxIiwvMbgajbuiZMLLi9f01DJAdlzABk0WAsd4IxHla0a1KIU3w8oKcHjBrFBquamn7YOX1LXL/TBl29z0FHvyrMMQ4kefLV1lLZuSK8Z/yEZ1dfIi78v6YzVeg11hUVa5XH5T3/ch907hwgl4Pp3RtH/Coiw6w7dCkLkuRQ2Nqy/582jSsCffqwTxnnzv1nChMLY+Rm5vFa648jOkMgYPD4nxdK+zIMgwbtnRH6PILXNbSUHVwCdWsyDLONYZgrDMPcULQvsThNUCAugFBfqLRffjYbqGdoYsBp3piQOFSoqVq0fmZKFkytTFQaCwCJUckoV9lG5fEl+ffwXYS9jEQHrzYam5M3OdsA8XUwpjPB6LcEgKKUQxMmsOIEABIJsGwZm1Fhw4avt1xlvA62xoELuwGBPShtJKgg4KutxWVwewDgt+P5DAzDoFP/tgi4/RqJXNMElUjhlC1ks0lYIfDjfjVrsl/T/rtTMjA2QF4WP4EyNjdGjSZOePFvoPLOACo7OyI9MQMZyZm8rqOlbOBi4jsO4DmAeQCmF2vfBVKJFEI95QKVx0OgiAjv38ZxyutXGlmp2eoJVGQSbCtpRqDysvPw17S9qNWsGnqM6KyROflC4tug7HWAQU/AaAgAVpwmTGB3TsePA3Z2H86bANbUN3ToV1kuJxgGaNHKBozVXkBgCUodBpIqP6wvC8pVtkWjTnVxbf8thSeu2nQe0BZEhFtH73EbUCKFU3yKGXJ1qqKc8GVRCqfERADFUjiVpFrDykhPyuQsNgoadaqHNw9DOGWLqFyXdUKJCPpu/MD+p+EiUFIi2kJEj4joqaKV+co0gFwuR36OGEI95SkHxXkSAOCUDFWcJ0FORi7sKn4+z1dpSPIlSH6fCks71Ux8RIT0xAxYlVPdRFgcRSLdkSsHq5yoVh2IJKDMRYBuNTBmy4rMep6ebGqf+HjWGSIxkb3pC5U/a3wTEAEeHkBSij0Yyz0gykPAw11fbT2tejVD7LsEpKvgfVcaFao7wLGmA149eMttQCkpnNwGVoe9WfRnUzgVp5tvJxibG+H6AX47wZpNq0FaIENsaLzSvg5V7QAAye9TeV1DS9nwSYFiGMaKYRgrAOcZhhnHMIyD4rXC1795Xt1/i7zsfNRpqTy3rbSAPczQ5SBm2WnZAAATS/67oEeXniM/V4yWvbgnoi1OTGg8pAUy2ChJgskVvzOPYGVvoVGHi8/xn0zhufsBWTQyZHOwecuHMzU7O/Zcwt4eELP5XhERARQUfJFlaoTgYGDXLiAptRJOX+qNqg6nsX2bZgSCL1XqaX5nUNnZkVO+OwBApUp418AdQhTgskEf3DoQjSq1bJGdmoVLF/LRuUY0fg/5CYxUCri7l+pOr2egh+Y/NsaDC095ZUW3rchaGxKjlJsjzaxZvy+tie/b4HM7qKcAngDwAWvSu1f4muL1b567Jx9AqKeLFm7KxUDGQ6Cy0nIAAKYcHSqKc/OoHyxszTiX8SjJjYN3wDAM2ro3V2l8ccR5Yjy6+Ayt+zTjlEhXXRRmO4U5h+RpoOwtkKA92nZu85+4oePH2R3UV9jYqY1esY14p07A4lVDYGyUh359jn+V9VRydgTAhkdoioq1KiA2NB7SAm5BZtUvbUSGjRPq5j+DTXMnlDtzkP3BmEG4Fu6EJngGODl9NnisVa9mSE/KRPCjUM7rtC20dCRFK3e1N7YwAsMwyEzRZjj/FvjkXYmIqhKRU+HXks1J2cQMw+xiGCaRYZjAYq8tZBgmhmGYF4XtR029kdJ4efsV6revw8nbTfEh09FVfqNWHNQacXBfL4n/v0Fo0bMJdHRVu+s+veqPOq1qwqaC+juody8ikJ8jRrMejdWeiwsKs92rV+xNOzPpCkCZGDDqZ7x6xf7M05Ptu2kTEBcHVKv2bXrqfQ49PdaZw8QE+OMP9v3KmNooQCMY6179KmuydrCEgZE+4sM048kHAA7V7CEtkCEllqPrt709zAP8WNunXA7rR6ypLuXKbTByOfv6vXtAuXKfnKKJawMAQOBd7sHHVvYWYBiGk4u6jo4OjMwMkZOuDdb9FlC6XWAYpm8pL2cACCCiUsq6FbEHwEYA+0q8vpaIVnNeoRokx6SiemOlWgoAIDl7eMxlJ1EgYcWMy26rOBJxAdKTMuHg9OkPoDISo5LR2EUzAbophVH5dhpyuFCGwmynqKt0eH8oBnkY4syF2nB2BgYPZvspdlrAx1khvhckEnbXl53NNoUbtVC/OiC+9VXWxDAMjMyNkJulubISivhChYMRJ+ztgRMngOho6P95AFjzDJKxE4DZvpyyZJhZmcLASB9pCdxNpQKBAHoGQkjyJZz6C/WFnHeFWsoWLh//4QB2ABhY2LYD+BmAH8Mwgz81iIhuA/hqJ40yqQzpiZmwLs8tTknh3cQIlNc9kknZaFBdIT+BUjzBcV3Tf68rQ2pcGuwcNSMoafGsQFnZa8bhojRKnjkpRMraGqji+A6v3laDjY0A1asDs2ez4tWxI7tzAr5u4K06lLbrY3QcAXkSiL5O7SFDEwON1j0yKPR45SVQCipWhN7gAQAASRdXzimcAMDCzgxpifzqUrECxe0AU1eoA6lEK1DfAlwESg6gDhF5EJEHAGewZUpbAJipwjUnMAzzstAEqLnyryXITMkCEcHc1oxTf4X3LZfCfDIpe/cR6PB7vFc89VnYmfMapyA9KRNyOXHOdqEMxUGw4mBY05Q8c1KQlMSGxThVjkJYREXk5bFxmfr67M7KywuoXr1MllRmmJn996xMkfFC4UadmVNYzFL2OcND2WFgrI98TQqUEZsHUpwrVmm8ngHrkslVOBSYWpvyroIrNNDjJ1DS78yu/D8KlztsFSIqbrhOBFCTiFIB8PWp2gKgGoBGYNMlrflUR4ZhRjEM84RhmCdJSUk8L4Oi/HtcI88VusQlTkRQuMsino/3JhbsmhRZK/iiGJ9V6EWoLoqzuVyewY9cKXnmlJgIBAUBTZqwO4yEJBs4lk9GdjYrTmLxB5G6fLlMllQmMAyQmfnxrsne/r+VcP/aUngGIii7HevnKBAXcAqj4IpEzH78uQTClwYfz9ni5GXl8T7/lRVIOV9HJpV/lZALLf+Fi0DdYRjmAsMwPgzD+IDNxXebYRhjALz22USUQEQyIpKDNRV+0hWNiLYRUVMiampryy3DeHH0DPRgYmHMOXeXYuekOIv6HArTnuIDxhXrwkS0KbGqWT71DfVhbmPKyRuJCxbl2J0cH3s+HxTmPMUN2tmZDbxVCFHdBk5o2ewdnJ0/vCYWf3hY+F4o+UxjZwf4+7Nfi/8ODIUxEEtMwQi47eo1TV52PgyNuWVK4UJ+DrtzMjBWLaP+h9hDfgKXnpgJC1t+VghJfgHn60gLpNAVagXqW4CLQI0H6/DQCEBjsE4P44koh4g68bkYwzDFUy+4AyXznGgWKwcLJHMUA8XZE5cMzTqFf7x8BcrI1BAGxvpqZZa2cbRGYhT/HWVpWBYKFJ8EnHxR3KAVyV1lMtYU9vQpYGlbDQKk4uaNpCKRMjIq+8zjmsKshM4oHrqnTmXftwLF76DXj9HQN6rw5RZYgrysfJXFpDQU6cFUnVNSKFD6HHJlFo0RFyA7PYeXmZyIIM6TcBcoibToM67l68IlWSwR0QkimkpEUwq/V3oLYRjmMID7AGoxDPOeYZjhAH5nGCaAYZiXADoBmKr2O/gMNX5wQsDt10VnRp9DcZ4klykXKMUHUpXD4RpNnPDs2kuVU87UblYdAXdea+Swu0q9ShAIGLy89UrtufhgYcEKFvTaAxDA2nAH/v0XmDv3Q1Du90B+sf8CHR1WfO3tgWHD/tvX1joFlR3uAXqqBWirS05GDrLTc4p28ZpAkYdP1TPR9CT+Z6AK93KnBpWV9PxAdnoO5DI5p+sQEXKz8mFkyj+ERIvm4ZIsNothmMzCls8wjIxhGKVh1kTUn4gciEhIRI5EtJOIBhNRfSJqQES9iShOM2+jdNq4t0BmShZe3lZ+A1akQ+KyKzItzCCRk57De00dvdog6nUMIjgUUSuNTgPaIj9HjPvn1I+VtrQzR902teF35pHac32KxET2/KlkmYxOnYCktBqAYV8g9wDSU6KwejV7k/8eXMsZhnUn19P7EKtlb88GFpd0CgEA5B0BUADG6JOOr2VKZGGRzMp1HTU2Z3RwDOwq2ahsNkwqzOzAJ8zhwfknEOoLeYVaKEziiowSnyM3Kw8yqazMHIe08IPLDsqUiMwKmwEAD7DxTd88zbo3glBPF08u+yvtq1N4riTjIFAfnBX4C1Q7jxYQCBjcOn6f91gAqN+uDhiG4Z5iRgltfmqO8IAoxIRq/llBUSZDcf5U0mmgYUMgNX8yCLoIfroKYjFBXx94+ZLdTX2LQmVoyIosEVu6SCIBRowANm5kz50U7+14sYQRRGJQ7mFArx0YXW5xeZomqjCDRGVnDQrUm1hUrK26yTIxKgkGRvqcEycTER5ceIrGXerxEsWkwp2eLYfcmYoMEqZagfom4H0LIKIzAL5O2mueGBjpw9jcCLkcSjjr6bMCxSWYz8jMCEJ9IdLi+Z/dWJazgF1lW8SHq+ZqLBAIYGplgvREzeQKa+fRAjq6Oji0/JRG5iuOokyGvj5w7NgHp4Fjx9jX4uOBcRPL4VHgGPR0uYxZkw/gl1/YsadPszFQCpH6Vhwn8vLYs6devYAXL1hhmjWLLQKrOGvauPFDUViiAlD6FECeCMZ4xFdb94ubgTCxMIZ9VTvlnTmQlZaN8JeRqMbD1FaS8MAolK9uzym0AwCeXH6BuLAEtPNoxfM6bP7B8tWUB8inxLBn1mUZG6iFO1xMfH2LNRHDML/hQ3Xdbx5dPV1OQXeKoEOFZ9LnYBgG5auVQwyH7MilYWploparuF0lGyS951iHR+lctvCc1gtX9txEwJ3XGplTwcaNH5K9enmxO6rERPZ7sZj92caNQMsuoxER2xmLZ6zAjcvP0Lr1h13Xv/+yKYO+BccJhRPEu3dsIlhb2/9WJ7ezKy5OMrYIo/g6GLMFYPT53Vg1RW5WHvxOPUIHr9Yac5++e+ohCiRStFexArMkX4LAu2/QsGNdzmMe/v0MRqaG6DKwLa9r+d8MRKU6FTh5/kUVmkIr1fl6zixaPsBlB9WrWOsGIAtAn7JclCYR6guLUhN9DkUdKK4xQRVqOCAmRDWzmKmVCbJS1ROoxEjNCBQADJwvgpWDJU6uu6CxOYEP7tYKs1e9emxTiI/CFZthBKja6HdAUB7Hd06CoX4SdHWBLVvY1GzZmgn7UhlDQ3adMtmHJLBv335sxisJkRyUOR/I/xuM6QwwRgO/zGJL4e6ph8jPFaPrkA4am/PG4buoUMMBNZuoZrJ8df8tJPkFaNyF+1lSbnYeTK1MONV3UyAtkBYKIbfkzBFB0TAw0v9i6b+0fB4uZ1C+xdpIIlqmJAffNwXXHFwKrx2ugb2ONcsjNjQe+SpE0ZtZm6plorOvYof48ESN5QszMNJH4y71EHT3DSePRz4UdzNPSvrgLPHvvx+7YjMCM+hab0Q520zcPjsA9esEoXNnYHVh1sbiD/5f6mxKYXnKywNGjmRFVSJhSxUVN+OVhOSpoPSxQN4JwHj8VzXt5Wbl4cjKMyhf3R7OrWpqZM6ge8Hw/zcInfu35WyeK8nNI37Q0dVBgw7OnMfkZuZxrnit4NHF58jPEaNxZ24C9fbpO1Rydvwi2f21KIeLic+RYZjThZnJExiGOckwjOZOWssYQ1NDTgkyzWxMIRAwRfnplNHEtSEKJFI8ufyC95oqVLNHQmRSUSQ+X5xb10J+rhhvn7xTaXxptPdohfSkTJXej6ZghLWhY70Ltrb58DvvhdFD9iMlhcAwH7I02NiwThR2mjlKKZWOHVkRio8vqlAOB4cP50tnz35GnMQPQMm9AfFdMKbzwJhMKruFKkEul+N3nw2ICYnDlL9GqSwmxclKy8byAetgX9UOHj+7qTRHeEAkLu28jl5jXDlVGlCQlpBRFFzOBZlUhh2zD6JCDQe06t1Uaf/4iEQE+QWjdZ9mnK+hpWzh8piwG8A5AOUBVABwvvC17wIjM0NOqYV0dHRgaW/BOctDg/Z1YGplopKLtmOt8pDL5Ih7p9oZVqNOrN3++Q3NxTk369EI5jamuLJPs9m2S3Mz/6jEdwk2b2+KGi3O4tqdNtiwfAnO7RuNihViYGnJilJyMrBo0YcKu+rwKZFLTAS2b2d/fvv2h91S8fOlkhBJIc9aB0rzARhjMNbHwRgP0YgoqMqhZafgd+YxRq8agsad1c+AT0RYO+ovpMSmYc6hybzEpfgcW37eC2NzIwxe6MlrbFp8Oi/nhYs7riP6TQxGrhzEKbHzjUN3AbDl7LV8G3ARKFsi2k1E0sK2BwD/3ENfCWNzI87xStblrZAUw02gdIW6aNmrCe6fe4LMVH6JKxWuuRFBqrmKm9uYwalhZTz8+6nKAb8lEeoJ0al/W9w/+xixKgpnSRTi9Ck389JEqmNHIDvHCr0Hb8XkefPQofUjBN76EVNHrYVEnAFnZ2DBAlYs1H3rimszDGBcWHtSkQtQsbbPiRLA3nBJ7AdK7QfkbAYM+4KxPgVGyN10VRY8vvwC+xYeQ5dB7eA+WTNl155de4k7Jx9i6JJ+qN1ceZXq0vj3iB+eXw/AkIXeMLPi7spNREiNS4MlxwwSaQnp2LfgKOq3q8NpR5SfK8bl3TdQr21tOFRVvRyOFs3CRaCSGYYZxDCMTmEbBEAzyeC+ANYOlkiOSeV0I69SryJCnoZxSncEAH0n9YQ4V4w1w7fwEoqq9SvB1NIY98895jymJD2Gd8HrByG4c/KBynOUpO+UnjA0NcTcnss1UlH0+PGPvfFK5qYrGS9U3MOPYRhs3DkE9TpcxIUrnTB3yha8e9gZF45uxuVL2aXuvooj5JHe7eZNICwMH+UELLm2krCxTSdBKb1Bab6ALA6M+R8QmK8AI+BfaVmTEBF2zDoAh2rlMHXraI3t4p5e8YdQTxc/Teyh0vjwgEj8MWILnFvXgtuYrrzGxoTGQ5wnQaU63E4Xlg9cj7zsfEzYMFzp+ycirBu9FXFhiRgw14PXurSUMUT02QagElgTXxLYTOZnAFRWNk6TrUmTJqQqJ9ddIBdGRGmJ6Ur7Xtp5nVwYEUUERXGffy07/6n1f/Na19rRW8nNeCDlZuXyGqdAWiCl0Y1/oX6Oo1SeozQC7ryiHvr9aGr7+STOl6g938aNRAkJ/309IYH9Wcm+AJG+PvtVR4f9ChDVr/OaTu8eQ7K4GpQe2oiObptIg0SnycoytahPyWZrW/rrJduKFR/W5OzKuFewAAAgAElEQVTMvubpWfr7kUvjSJ71J8kSWpIsrgbJktxInnOc5PJ8tX9XmuLeucfkwojo8p5/NTrv2CbTaVqnBSqP3z3/MLnqeFJqgvLPYkmu7LtJLoyIwgIilfaViCXkwoho19xDnOZWfIYPLD3Be11auAPgCfG89392B8UwjA4AD2LTEtkSkR0R/UREkWWsmxqjfDW2Bk/sO+Wlruu1qwMAeHmbezyQ++Qf0bJXE2yfsR/Bj0M5j3MZ1A75uWLcPqHaDkhHVweTNo1Ackwq9sw/qtIcpVGvbR1M3zMBAXdeY9XQjWp79SnObkpSmuls/Hi2RIdYDNSsyebsU/A+vjZGz9yClj2O48jpnmjd7Bn2bpiB+IBW+Pf0AEwfvw0NnF9DIPiwXisroFu3j69RrRqbpcK6lKQCxQNtjx1jXyN5Dij/CuQZ8yBPcgUltQdlbwCE9cBY7gZjfQ6MkQgMo7kkrOogERdg/+LjsK9qp9GzlMyULIQ+j0CjTty84UojOjgWDtXsOZvpivP6/lsYmRpyik/KTGHjEmwclWeOePv0HbZO34c27s3Rf7Y773VpKVs+K1BEJMN3FPNUGlXrVwIABHIIQq1Q3R4Va5XHqXUXOHvYMQyD6bvGw8rBEr/2Wcn5hu7cuhacGlbG5im7Efo8nNOY/8zRqhZ6je2GU+v/xsGlJ1WaozQ69WuDEb8Nws2j97DQYxVkpZWHLSOOHQNWrGDPhVJSPjhWpKUBlpZAX++GkBouxdYTt+Ex4gSWrx8Lc9MsLJ21Fs+v90FqcFMEP/TFnys2oGI5P7yPjIaxUQ4AQq1abFzV0qWsCa9mode1qWmhJUGWBFuLRxjnewTyzOWQp/qCEluA0icA+RcBXScwpjPB2PwDgeV2MPptvqoTREkyU7Iwy3UJQp6GwXdJP94Vnz/Hnl/ZhyBVPdxkMhkC77xGtUZVeI/NTMnCv0f80LhLPU6BxorURtYcktgG3H4NuUyOCRuGa13Lv0WUbbEALAObe68dgB8Uje9WTZ2mjomPiGhcs5k0tukMTn0fX37BbveX8NvuX9p1g1wYEb0PieU8JiEqifpXGk2icsN5jSuOVCql34b8SS6MiI6sPKPSHJ9CYR49/Ntpjc77OYqb2Zyd2X+XfC0w8MO/FWbAihViaJDoLG1csYCC7vQiaWxN1gRX2LLD65Ekrj3Jkn4iWbI3yZLcSRzXk1JCXEiW0J5k8Y0+6i+Lb0CypN4ky1hG8vx7JJerb+4sS6KDY2hIjQnUw6A/3Th8V6NzP78RQC6MiLb8vEflOV7efkUujEiltW2ctJNcdTwpPJCb6f3KXtYcGPXmvdK+O+ccJFddL5LL5bzXpYUfUMHEx+URS5HLZHFxXcN3ko8PADp6tca2GfsRExqHCtUdPtu3qWtDtPdshUPLT6LzgLZwcOLm0eNYo9CUGBqv9BoK7Cra4LfL8zG13XzMcl2CtXeXwoZnOQQdHR38smscstNycHjFKbhP6qGxqqnuk35E0L1g7Jl/BI271EetptU0Mu/nKM2xAmC/V3gELlrEflUUOLS1BfIl5XHgRG8cP98bYjGwZnU2/G76w9wkDjbWqajkmIZhQ1Oho5MKkBgQmEJXRw8WBvoAowcwRmB0qgC6TmwT2INhvo8n6pe3X2Gh++/Q0dXB6hsL4NyqlsbmzsvOw5oRW1C+uj2GLumn8jx3Tj6AUF+IFj1/4DXufUgczm+5gh7Du6BK3YqcxkS9fg8dXR1On93M5CyY25h+UzthLcXgq2hfo6m7g0qITCRXXS+a0XURifPESvsnvU+m3maDaUS9qZQan8bpGmmJ6eTCiGj/4uO81/fmUQj1Mh3EeZdXGk+vvSyTXVRmahb1rzSafrL0obunH2p07k+hzLEiIYHI3v7Tuyw7O6JatT44SiicJRR9/5d4etWf3IwHkm+dyRT7Ll6jc2ckZ9LktnOpq8CTXt5+pfI8ir/vBe4reY1Ljk2l4XWnUG+zwZw/h0REk1rPoZENfubUd/aPy2hUw2m81qVFNaDCDoqLiU8fwAAAcwD8qmh8L6ROU1egiIj+2X2Dugo8afaPyzh5pz2/EcB+8GtP4vzhmNphPg2tNVElc8HxNefIhRFR9FvVTH1yuZwWeqyi7nreFPoiXKU5PkVsWDyNazaTXBgRbZq8iyTir2vuUnj7lRSchASimjWpyDvvUybC/xWRenDhCfUw6E8jG/zM6wbOhdiweBpaayL1MOhPN4/6qTxPRFAUuVsPpUFVx1LS+2TO4+IjEmlI9fHkZjKQXvwbyHlc0L035MKI6OS6C0r7yuVycrceSquHbeI8vxbVKSuB+gfAUQAzAExTNL4XUqdpQqCIiP7edpVcGBHN672C001WYTc/tPwUp/kVbupB997wXlvsu3iV3NWLk56UQV4OI2hEvamcdop8EOdLaNPkXeTCiGh885m8bjZlwad2WStWfFq8FCJV0r39e+T2ifvUXc+bxjWbSRkpmRqff1qnBfSTpQ8F3H2t8hyxYfHkXWEkeTmMoJjQOF5jxzefSX0shlDQ/WBe4xZ6rCJ3Kx9OoRdRb96TCyOiizuu8bqGFtUoK4EK5DupppumBIqI6Oymf8iFEdHvvtzuUl4OI2ilzwZOfXMyc8nNeCDN6LqI8nL4x8X41p5EY5tMVyv+6NGlZ+TCiGiRaBXlZuepPM+nuHPqAfUyHUTeFUbSg7+fanx+TcAn9up7o0BSQPsXHydXXS+a1GYuZadna/waCicDLruQTxH5+j0NqjqW3K18KOxlBK+xmalZ1FXgSfsWHuM17v75J9RV4Ek75xzk1P/YqrO84x61qE5ZCdQ2APX5TqzJpkmBIiJaNmAt9bEYwqnvItEq6uc4irPZ7sLWK9RV4EkTWsziHZB486gfuTAiWuK9hmQyGa+xxTm+5hy56njSyAY/835y5ULI8zAaXncKuTAi+m3In5SRrPkneC3/5Z1/BI1tMp1cGBEtG7BWowHaCi7uuEZdBZ70S+cFau3CxzaZTiK7YfTmcSjvsee2XCYXRsRr7N/br5GrjieNazaTMlOzlPYPC4ikHw3706zuS7QefF8IjQoUgEAALwG8AlAAILjw3wEAXvK9kDpN0wJ1bDV73sPlD1nhPh7yPIzz/H5nHlFPowE0uNp4ig6O4be2wqc6dVx6iVh3eXfroeRuPZQSopLUmqs0xPkS2j3/MHUTepOo3HC6dfyexq+hhUUiltDeBUeLfte3T9xXPigykmjDBqIlS9ivUcp3Cac3XCQXRkSzeyyl/FzVM2O8ffqOXBgRnd5wUaXxE1rOphH1p3ISDrlcTvsWHitaNxfRzsvJpxH1ppKn/XCNn91p+TSaFqg0AJU/1fheSJ2maYG6c+oB5ye01Pg0cmFEtH3mfl7XePXgLYnshpG79VBeuxi5XE4bJ+0sisVS5+kuOjiG3EwG0tQO88vEFEREFPoivOipfl6vFSo7eWgpnUC/NzSi/lRyYUS0YtB6Sk/K+Gz/XcvjKM/Ng0ggoOL5nOQCAYU08iCK++/fokwmo/2Lj5MLI6Jff1qplolZJpPRikHrqYdBf04PgCVRODkcX3OOU39FjNaKQeupQFKgtL9UKqWVPhvIhRHRkysveK9Pi+poWqCe8Z2srJqmBSohKolcdb1o6y97OfVfJFpFLoyIjv7Oz4U78vV7ctX1om0z+ImbVCqlFYPWkwsjoqX9/uD0wfsUV/ffIlddLxpUdSzvA2euSAukdGzVWeptNpi663nTlp/3UFZa2Qji/xdiQuOKbqT9K46me+ceKx2za3kchcKJFSShkMjDg2jOHMpz8yAJdIkASrdxIor/4I4uzpfQvF4reN3kP0VOZi4t9GA/K1w/W8W5eeweuRkPpP4VRysVYgXbZuyn7nrelJ2Rw2l9c92Wkwsjon2L+J1vaVEfTQvUewA/f6rxvZA6TdMCRcSeQ/U2G8zpRirOl9AS7zXkwoho2/R9vHY1c3ouowGVx/A+U5LL5XRg6QlyYUR0dtM/vMaWJOjeGxpUdSy56nrRgaUnSCqVqjXfp0iJS6XVwzdTV4Enedj60rktl0laUDbX+l9ELpfT06v+NK/3Cuoq8KTuet60Y/ZBzmdNeW4eRAA9wQ/UqUb0Ry72FRBNgQY/sB95Dw8iYnc7S/v9wZrj/ryo1m79fUgsjag3lVx1POnEH+d5zSWTyWj3vMPkwohoUpu5lBKXynmsb53JNMN1Mae+K4duIFddLzq35TLn+bVoDk0LVFxhzNOC0hrfC6nTykKgQp6FkQsjooPLTnLqL5VK6c/x24s8ALk+aV47cJtcGJFKgY5yuZymtp9PXuVHqu2Rl52eTUv7ryUXRkTTOi3gdRPgS8izMJraYT65MCIaUn08HVx2kvMT8f9HxPkSurD1SpHjichuGO2ef5iSYlK4TxIZSSQQkFwopE41oksNUk56FkWkq0skEJAsIqLo7/nYqrNqrf/RP8/pJ0sfcrceSk+v+vMaKy2Q0gL3leTCiGj18M28zIvRwTFF4soFn5oTaaHHKl7r06I5vikTH4BdYMtzBBZ7zQrAVQAhhV8tucxVFgJFxO5uugm9OT89yuVy2rfoGC839dysXPrJ0oe8yo9USaQC7r6mrgLWIy/iVTTv8cWRy+X0z+4b1NNoAHk5jKDnNwLUmk/Zte6cekDTOi0gF0ZEbiYDacesA1qhKkZmahYdXnGKvMqPJBdGRGObTKcre2+q5j23YUPR7igh4eNSI7a2xdzuPTwoFsb0c43hKpviivP26TvqJvSmUQ2nUWwY/0wWijIah5af4rXrinz9nnzrTKbuet6UEJmotH/021jqKvDk/ECqRfNoWqCe852sxPj2YBPLFheo3wHMKvx+FoCVXOYqK4HKTs+m+X1+IxdGRCuHbuB8Y/jddyO5GQ/k3P+dfwT51JxIrrpedGzVWd6mlEeXnpHIbhi5GQ+kSzuvq+0W+84/gnxrTyJXHTbWpKxMfgoigqJo2YC11FXgSW4mA+mvaXvJ/1bQV89I8bWIDYunjZN2kpvJQHJhRDSz2xJ6etVfvf/XJUvYj/OcOZ8UKLlcTue7jyM3/ES9Dbzo0q4bal1TnCem4XWnkHeFkSoFC0ulUhpaayKNajiNlwn89skH1NtsMInshnHOMrF62Cb60bC/1mvvK6JpgbLiO1kpc1QpIVDBABwKv3cAEMxlnrISKCLW/q1wUx3ffCYn0Xn0z3NyYUS8AlWzM3JokedqcmFEtMB9JW8ngqSYFPqly8KiGJicTPViYHKzcouyoP/SeQE/c5KKRLyKpmUD1pKrjie7qzIeSLO6L6EjK89Q8JNQ1c+rVHCp/pLk5eST/60gOrLyDM11W06uOuz50kqfDfTOP0IzFyncQeW5eRRlzChu4qtbK5tmu7EODDPQjhIWr1b7klt/2UsujIgeXXqm0niF+ZuT2zyx5sDtM/eTCyOiCS1mUWI0t2wm8RGJ1E3oTRsn7lRpnVo0Q5kE6qrTShGo9BI/T+MyT1kKlIJzm9kME35nHyntK86XUC/TQTS++UzOHxIi9gn25NoL5KrrRXPdlvN+epVKpXRg6Qly1fGkKe3macRT7p/dN8jNeCD1tfGlSzuvqxUgzJWstGzyO/OINk7cWXTuojh72TJ1N+fMA6q4VH8p5HI5vXkUQquHbyY344FF73FI9fG0bcZ+zaeKiowkuUBAEgipAqI/ykPYuOITaoMx1IXxomOC2iRnBGqJuLRASvsWHqOuAk9aO3qrSnO8vP2KTctVfyrnv7mdcw6SCyOitaO3cj6rSo1Po6kd5rOmwDKIB9TCnf8pgQIwCsATAE8qVapUFr+vj5CIJeTlMIJm/7iMU//bJ+5TL9NB1NfGlx5e5PcEeeKP8+TCiOj6wduqLJVuHvWj7nqs3T85Vn1nh4hX0TSpzdyicxB1MlerQnJsKl0/eJsWidhkty6MiMY2nUFnNl6ihMjEUoVcFZfqsiY9KYNuHL5Lv/tuJO8KI4t2iauHb6YHF56U+flbSCPWiy/Q4AdKehZFSTEpRbv2Lvrj6SRaUXEvPlWIfhtLE1rMKnJL55vSSyplxc1Vx5OG1JjAeQcpzpdQXxtf+vUn7hnR/W8FkVf5kfSjYX+6uv8Wr3Vq0Tzfg0B9cya+4igcILhmf4gOjqFRDaeRCyOiHbMOcDZRSaVSmtByNvW18aW0RH7pkBQ8vvyC3EwG0iCncSoXOyyOXC6n64fuUP9Ko8mFEdFirzUqHXqrS3pSBp1a/zeNbvxL0a6jj8UQmtJuHq0fu43ObblMz28EUHAHD0qDHj3i6FKtKaRSKSW9T6aAu6/p2oHbdGDpCVozYguNazaTugpY06W7lQ8t8V5Dl3Ze/7LxYHFxlG7jRPkQ0FFBbeot9KAfhZ50sK4HiXWE7O/DSXXRfnjxGbkZDyR3Kx+VMpynJqQXeXeuGLyel5n6xuG75MKI6PFlbsG1J9exlgqfmhM1Z0bVohaqCBTDjisbGIapAuACEdUr/PcqAClE9BvDMLPAnnPNUDZP06ZN6cmTJ2W2TgVpCekYWHksWvZuitkHJkGoJ1Q6RpwnxpYpe/D39mto2LEuVl6ZDx1d5WWpI4KiMfaH6Wjs0gBzD02Gsbkx7/W+eRSCuT1XQKAjwOwDk/CDSwPec5QkP1eME2vO4+jKM5DJ5PCY0hP9Zv2k0vrUJexlJIL83iA8IAphAZEID4hCbmbeR30IDArIFHJdU8gYQ+QWWMDApgKmTzRCncVjYCvPhcmrFzCoVe2zRemICBnJmUiMSkZSdAoSo5IRH56I8MAoZCRlIjs9B9npOf+5PgBYljOHY63y+KFLAzRxbYiaTZ04lSbXNHK5HFc3nMPu2QeQks+gGcVhPF6gAnIAgQBwdwc2bQLKcSvCqYCIcOPQXawZsQVV6jpi8dmZsKlgzWuOvOw8LOy7CoF332DKX6PRdUgHzmOz0rIxq9tSZCZnYm/oRqWl2ZNjUjCg0lg0cW2AeUd/hrGZEa+1aikbGIZ5SkRNeY0pK4FiGOYwgI4AbAAkgI2fOgPgGIBKAKIAeBJRqrK5vpRAAcD+xcexb+Ex1GxaDXMOTeZcHffspn+wceJOTNs5Dt19O3Eac27zZWyatBO2FW0wc99E1G9Xh/d6o97EYKH774gOjkWvsd0w5g8f6OkrF1ZlJMekYOecQ7i2/zbMrE0xcK4H3Ma6amRuVSEiJEUn4/2qv5CxcRsyGrVAbGcRtm/JhCwvE7rIh7FuCowFcZAVyD4aK9ARwNjcCCYWxhDq60IqkUJaIINUIkWBRIr8HDEKxAUfjdE31EOVehVhXd4KxhZGMDE3homFMSzLmcO+qh3KVbGDXSUbGBjpf8lfQ6kE3QvG5im78fbJO9RpWQMjfnZFg4QgIC0NsLQEevcGKnKrSFuc5JgUrB+3HQ/OP4Vzq5pYcm4WzKxNec0RHRyDRR6rEf0mBlO2jkaP4V04jw248xorBq1Halw6ZuwZj84D2ikds3zgOtw9+RDbA//g/PnVUvZ8UwKlSb6kQAHA3dMP8ceILZBJ5Zi8ZSSnDwURYXzzWchKycLu4D+hK9TldK1XD95i5eA/EReWiMG/emLwAk/e6xXnibF73hGcXHsBtVvUwK/Hp8HWkd8T7qcIfR6OHbMP4ukVf9hXscXQJf3RqX8bpU+xZcrSpcD8+cCcOUicvAz16gFJSeyPbG2BF8+lkGUnIHb+70g5fh7Z3dyQ3aQlcjJykZORC4m4AHr6QugKdaEr1IGuni70DfVg42gNu0o2Rc3M+tsuBS4RF+DRxWe4svcm7p97Auvylhi5cjA6D2ir9rqJCJf33MRfP+9BgbgAvkv7w33yj7x3hndOPsDqYZuhZyDE7ENT8EOX+pzGyaQy7F98HIeXn4K9UznMOTgZtZpVVzru8eUXmNNjGQb/6okhC714rVVL2aKKQJXpGZSm2pc6gypOQmQiTWk3j1wYEf2z+wanMQ/+fqpSFufcrFxaMZjNvXdm4yVVlktEbHxIL9NBJCo3nHdEvzKeXvWnMT+wSWFHNZxGZzZeUivjtVoocakuKlbowToNfPdFoEoQ8Sqa1o/dRu5WPuTCiMjTfjjtnn9YY+U3UuJSi/LzTe0wX6UEwBKxpMgNfULL2by8XWUyGc3+cVlRQDzXs6rs9Gwa5DSOfGtPUivhrZayAd+ak4Sm2tcQKCLWnXZ041/Iw9aX4iOUR6vL5XKa2mE+dRV40paf9/DKCCCVSml2j6VFcU6q1liKeBVNvnUmF324NVmrSSaT0fVDd4ocQ7zKj6QzGy99+YDbz7hUKwSrc40okhem9fnW4qJUJfhJKP3uu5FcdTypp9EAWjZgLT269Exj+Q5lMhmd/+sK9bEYQj30+9HJdRdUCjvITs8u8gpdP247L7GQSqX0u+9GNgXTam4ZzYnYjCuKfJMvbnIvEa/ly6EVqDIgOjiG+lgMoTE/TOfkUpubnUfrx24jF0ZEI+pN5VVHqkBSQPsWHaNuQm/ytB9Od08/VGnNeTn5tGPWAeom9Ka+Nr50ZuMltbJUl4b/rSCa2p71yBpYZSxd2nVD42XmP0dJl2oFCQmsOD1B2XjxfWmyM3Lo3JbLRSVNehoNoM1Tdqvs/fkpwl5G0KTWc4oCt6PevFdpnqy0bJrQYhZ1E3rTv0fu8horLZDS8oHrirKNc4kTlMlktHv+YXLV8aRBTuMo6N4bldatpezRClQZ8eACW0p6xaD1nINrH158Rl4OI6i7njfvujOhL8KL3KxXDFqvsukm7GUE/dKZzYXnW2cy3Tv3WKPVQ+VyOT2+/ILGNZtJLoyIuut504SWs2nzlN1086hfmSakVbhUE8AmQPXwIJo9m8jDg905qelS/TWJfRdP1w/doVW+m4qCfEc1nEZnN/2jcbf1jORM+mvaXuom9CYPW1+6su+myn8jGSmZNLbpDOqu580p4L04BZICWuzFVgw4vOIU53F/b7/Gpirz2cCp5IaWr4dWoMqQA0vY0heLPFdzNptlJGfSiHpTqbfZYLq44xqvD36BpKAooNGn5kSVzRZyuZz8zj6iobUmkgsjolndl/Cu8svlGo8vv6BtM/bT1Pbz6UfD/uTCiMhV14uW9l9Lz28ElE1Z7bg4VphKZJIggYB9/TsSJ7lcTk+uvKAZrouL4r/cTAbSmhFb6PXDtxr//UkLpHR6w0X6ydKHugo8afXwzWqZg59ceUGDqo6lHvr96MGFJ7zGRr15TxNbzSYXRkQn/jjPedyNw3fpJ0sfGt98prZs+3eAKgKl9eLjiFwux7Hfz2LvgqMwszbFrAOT0Lizco+khMgkrPLdBP+bQWjWvRGmbhvDy8Pu+Y0A/DHyL8SHJ6LnSBeMWDkIJhb8Y5KkBVKc23QZexcehSRPgg7erdFzZFfUa1tb455q0gIp3r2IwL+H7+LynpvITs9B+er26DGsM1yHdoSVvaVGr4foaODcObVdqr80MpkM4QFReHnrFa7svYl3LyJgZW+Bnyb+iOY/NkZlZ0fO3qB8eHn7FTZN2oWwl5Fo3KU+xq4diqr1Kqk0V05mLrZM2YPLe/6FY00H/LJrPOq2rsV5fMCd15jdfSn0DPUwadMIdPRuo3RMZkoW/hy/HbeO3UftFjUw5+BkODjxi+3S8uXRupl/AUJfhGP5gPVIT0jHztfrYWlnrnSMXC7Huc2XsXPWQegIdTDmj6HoNrQjZ2HIy8nH/oXHcHLtBViUs8DEjcPR1r2FSutPjU/DwaUnce3AbeRm5qFi7QroOdIFXYd04B3fwgVxnhh3Tj7ExR3XEHD7NQQ6AjRxbYiq9SqhQnV7lK9ujwo1HGBd3vLruq6XMRJxAVJiU5EUnYLgR6F4efsVAu++QXZ6DgCgSr2K8Jjihs4D25VJrBkR4fWDtzi94RJuHvGDXSUbjFnjg7Z9W6j8gBJ0Lxi/Df4TiZFJ8JreB4MXeELPQI/z+IigaExtNx+W5cyx6sZCWDsof3B5ePEZ/hixBZkpWRi8wAveM/pwCozX8vXRCtQXIvJVNMY0no4GHeti9oFJsLBVLlIAEPsuHquHbUbAnddo27cFpm4dzUsU3j59hzUjtiDMPxJt3Jtj3Dpf2FW0Uek95OXk49ax+7i4/SpePwiBUF8IV5+OGLzAk9ONQhXev43FpZ03cP/8E8SHJaBAIi36mVBfCOvylmjQwRkdvVqjcZf6ZbJ7+FIkRifj4d/PcPOoHyKDopGRnPXRzx1rOqBBe2fUb++MBh2cVf5/VAYR4daxe9i74Cjev42DgZE+RNN6wXvmTyoHGOfninFwyQkcW3UWdpVtMWv/JF67JgDwvxWE3wb9Cbmc8Oe9ZShX2VbpmINLT2LPr0dQpV5FzNw3EdUbVVVp/Vq+DlqB+oL8ve0qNk3aBWNzI0zcNALtRa04jZPL5Ti++jx2zzsMc1szTNsxFs17NOZ8XWmBFCfWnMeBJSfACBgMWegN90k91LqZh72MxLnNl/HPrhsQ6unCY6obPKf3LtMUMTKZDMnvUxETEoeY0HjEvYtHfGQSnl7xR25mHkytTNCmTzM06FAXNZtVg2MNh2/6STkvJx8vb73C0yv+eHrVH1GvYwAAlZ0dUa9tHdg6WsO6ghVsKlihav1KZfYQUJyge8HY+stevH4Qgqr1K6HvFDe0F7WEkamhynM+uvQcGybsQHx4Irr7dsKYtUN5/Z3kZOZix8wDuLD1KspXK4cFJ6fDqUFlpeMu7riOtaP+QpeB7fDzjrFfNaOJFtXQCtQXJjwwCqt8NyHkaRg6eLXChA3DOe+mQp+HY6XPBkQERqPnSBeMXjMEhibcbxxx4QnYNGkXHv79DE4NKmPylpFwbsXvKbYkMaFx2D3vMG4duw8LWzMMnCdCz9EunHISagqJuABPr/jj1rF7eHL5RdHOQ6gvRGVnR1StX7dNQtQAACAASURBVAmONcvDws4cFrZmMFc0G1MYmhhofNcll8uRl52PnPScokwUKXHpSIhIRHxEEhIiE5EYmYyYkDgUSKTQMxCiQQdnNOnaEE1cG6JK3YpfLBuFJF+CQL9gPL3ij2fXXiL0eTisHCzhu6Qfuvp0UCs/4OuHIdjz6xE8u/oSFWtXwOQtI9GwQ11ec/jfCsLKIRuQEpMK98k9MXRJP067uIcXn+HXPivRpGsDLD4787veWf9/RitQXwFpgRRHfz+LA4uPw8zGDL9dnsf5wFmSL8HeX4/i+JrzsK9qh2k7x/L60BMR/M48wubJu5H0PgVdh3TA0MXesKuk3FzyOYIfh2L7zAPwvxkEy3LmaNDBGe08WqGdR4svek4kLZAiIjAa4QFRCA+IRHhgFMIDopASm/bJMTq6OtA30oO+oR70jfQh1NOFQEcAHV0d6OgKINARgBEIQHI5irxY5QS5XM7m5RNLUSAugFQihURcgLysfHzqM2JqaYxyVexQrootHGs4oHGX+qjfrg6vcxh1keRLcHnPTfideYSA268gyS+ArlAHzq1roWXPJnAb05XXg09JIl9FY8esg3hw4SnMbUzhPdMdfSZ057WDkcvlOLPhErZN3w8HJzvM2DsRdVrU4DTW/1YQ5vVcgYq1y2PNzUVqvRctXxetQH1FQl+EY3a3pcjJyMWAuR7oN+snzk96AXdeY5XvJsSFJaDH8C4Y+fsgmFqacL52XnYeDiw+gTMbL4FhGPSf3Reev/RS60ZJRHj8zwtcO3ALAbdfIzkmFVXrV4LPIm+07tPsq+aoE+eJkZGUibTETGQkFbbkTIhzJcjPFUOSJ4E4V4z8PDFkBTLIpDLIZXLIpHL2ezlBIGAAhoFAwIBhGDACBkJ9IYR6uhDq6UJXTxdCfSGMTA1hbGEMY3OjomZZzhz2VWy/SoZ3BZJ8CS7uuI4jv51GSmwaKtaugKau7K6tQfs6at/Is9NzcHDpSZz+8yIMTQzg+UtvuE/qwXveiKBorB29Fa/uBaOlWxPM2j+R0+8tLzsPu+YextmN/8ChWjmsvb1Y896fWr4oWoH6yqQlZmDzlN24ecQPTg0qY9rOsajZpBqnsfm5YuxbcBQn116Aua0Zxv85HO1FLXkJQUJkErb+shd3Tj6Eg1M5jPnDB616NVVbTGQyGW4du4/9i47h/ds4VGtUBW36NEeDDs6o07LGF90x/H+npDDVb1cHQxZ6oWHHuhr5f352LQBX992E3+lHKBBL0X1YZwxfMQDmNma813lw6UkcW3UWRmZGGLPGBy6D23NaY2JUEmZ1W4ro4Fj0Gd8dw5YPUOvcTMu3gVagvhHunXuM9WO3Iz0hHaJpveGzyIvzTTzkWRjWjvoLIc/C0dKtCSZuGsHbw+vZ9QBsmrQTUa9j0LhLfXQf1hlt3ZurLSQyqQzXD97BmQ0XEfo8AkQEob4QtVtUR4P2znBqUBk1fnDSxqRoiMyULAT6vcH74FhEB8ciJiQOEYFRyErL0agwvQ+Jw+VdN3DtwG0kx6TC1NIYnfq3xY8jXVCtYRXe8z38+yk2T9mN2HcJ6DqkA0avHsJZ4KLexGCW6xLkZuVh4anpaNSpHu/ra/k20QrUN0R2eg62Td+PSzuvo2LtCpi6dTTnek8yqQyn1l/EvgVHAQboN8sdop/doG/I3S1YWiDF2Y3/4Piac0iJTYN9FVuM+G0Q2nu20oh5ListG4F33+DlrVd4efsVQp+FQS5n/5ZaujWBx1Q3jdw8/7+hiFe6uP06bh71gzhPAgCwsDOHY00HONYsj07926Jx53pq/27TEtKxb+ExXNxxHQDQrHsjuPp0RMteTVXykkuITMLmKbtx7+xjVKxdARM2DOdeXkMmw9mN/2D3vMMwMDbAb5fnqSSOWr5dtAL1DfL48gv8OXYb4iOSeGeCiAtPwNZpe+F35jFsK1pj+PKBvGsxyeVyPL3ijx2zDiLsZSTqtqmFMWt8ULs5t0NqruRl5yEmJB73zz3Buc3/ID0pE9UbV0X3YZ3h3KomqtavpPW++gyZqVm4tv82Lu24joigaBgY66Nz/7bo6tMRlZ0deZ1JKkOcJ8bJtX/j6MozEOdJ0GuMK/rNdlfZ9b1AUoCTa//GwSUnAAAD54vgMbUnZ+/Pd/4RWDvqLwQ/fodmPRpjypaRajv6aPn20ArUN8pHmSDszDFu/TBe50v+t4KwddpehDwLR61m1TBmjQ/qteVXfVcmk+Hy7pvYM/8w0hIyULF2BTRoVwcNOrDBopoqcAiwN8DrB+/i5NrzRfFAegZCVP/BCXWaV0f1H5xgWc4c5jZmMLMxhbmNKa/d4fcIEUGcJ0F2WjYSo1MQH5aA2HcJiAtPQFxYAt48DEWBuAC1m1dHjxEu6OjdWuPnLpkpWbh//gn2LjiKpOgUtO7TDCNXDoJjzfIqzZefK8a1/bdxcu15vH8bhzY/NcPYtb6cgm4Bdpd/eMVpHFx6EqaWxhi7zhed+rXR7rr/R9EK1DdOyLMwrB29FSFPw9Ci5w8Y/+cwOFTldl4jl8tx/eAd7JpzCMkxqWjVuylGrx7Cu6R1blYe/t52DS/+DUDg3TfIzcwDAFStXwkD54nQtm9zteJlikNESIhMwpuHIXjzMASvH4Ui9FkYJPkF/+mrb6iHSs6OaOveAm37tkCl2hU0soavBREh7GUkbh69hzsnHyAxMumjzBkKbCpYwcGpXNFuk0vQKl/8bwZh9/zDCPILBgDUaOKE0auH8I5jUiCTynBh61XsXXAUWanZqN64KnwWeaOlWxPOcwQ/eYd1o7ci9Hk4ugxsh3HrfMsk1ZaWbwetQH0HyKQynNlwCXt+PQKSEwbNF8HjZzfO5pD8XDFOrfsbR347DalECtG0Xug/210lt2KZTIbwl2yy0gtbryA6OBYVajjAc1ovdB3SoUy886QFUsSExiMzOQuZKVnISM5CZnIm0pMy8ep+MF4/CAEAVKpTAW3dW8C5VU04NazyzefqK5AUIORZOILuvkGg3xsE+b1BRnIWBDoCNO5SH9UbVYGJpQlMLIxh62gFe6dysK9iW6Y7x5BnYdg19xCeXPaHraM1eo7uivrt6qBe29oq/y4D777Ghok7EeYfiUad62HIAi9eCYdzMnOxZ94RnNv8DyzszDFhw3C082ip0lq0fF9oBeo7IjE6GVum7sHdUw9R2dkRkzaPRIP2zpzHp8SlYcesA7i2/zZsKlhh5O+D1TKPyP6vvTuPj7K6Fz/++SaZ7HtC9oQskLCEVRYxgIBIEQVF0Iq1tbXWtj+3an+3t9ttvW297e1m7y1ee63WfaGCCKiIgCDKIvsWICzZ933PZJac+8czGaImLElIZsJ5v17zmsnkmWfO4Qnzneec83y/djs71+5l1e/XcXr/OcKiQ1j6yM3c/N35BIcP3Dfb6pIadr6zj0/f3kP2rtNY242zLV9/H+JGxpCYEUdMchTBEUEEhQcSFB5IcEQQgaEBzgt0vX29Mfma8PY1XfbZYEdHBzarHZvFhs1io63ZTGtTG21NbbQ2mWlraqO+qpGaklqqSmqoLqmlpqSWstwK55lh3IgYMmeOIjNrFNcunnJJCYX7Q2VhFcd35pC98xTZu3I4dzifoPBAVvzkdpb8vwW9CoadZ4I71+5l5zt7yT1awLDECL73p3uZtezSh6ltVhtbXv2EF//tDWrL6ln8/QXc9+SKQb2WTBtYOkC5oT3vHmDlw89TUVDF/K/P5v7f3XNZk9XZu3J4+pHnOXMwj+Sxicy7exZLH13U60SgSikObzvOP/+wjv2bjgAQHhtG/MgYUscNZ8mDCwds+K2loYUzB/OMJdanSyk6XUrx6TIq8quw2+yXtA8RwcPTyCDh6dmZSUJQHV2Lop0PTB32jkvar4eHEBYT6syxF5McxdjrMhiblTFgF5SaW9v54B8fcWjrMc4czKWqqAYAv0BfRl87kknzxrH4+wt6FQSa61v45x/Wse3NnZTnVSIiZM4cxcyl01n0wPxL/vtSSrF91S5e+PkblOVWkD4ljYdXfrvfF+lork8HKDfVmR16zVPvYvIx8Y0n7uTWhxZe8qo3u93O5pc+5sOXt3Nsx0mGJURw33/czby7Z/ZpWOzs4Tz2vn+IkrNllJwp4+zBPCxmK9ffOYMF984ZtIzjSinMLWaaaptprG2mqbaZ5roWzK3tWM1WLGYr7W0WLG0W7DY7dnsHHV1vHR14eHggAoiRScLDQ/Dy9sLLZGSRMB574hvgg3+QH35BfvgH+eIX5EdIZBBh0aGDlry2pbGVDf+ziTVPvUt9VSMJ6bGkTUwmM2s0Y7MySB0/vNdta65vYe1/vc+av7xLa2MbUxZOZObS6cxYcvlnggUni1n58PMc/ug4aROTnfNUehHE1UkHKDdXcraMpx99gX0bD5GcmcjDK++/rGE/MIrR/e2HL3HmQC6jpo3gvv+4m8yZo/ol4Wt9VQNr/vwu65/ZZGQcDwtgxq1Tmb3sWibNH68zTF8hSinK8yvJPVLAiV05vP/cVprrW5i6cCIrfnL7JV9fdyHN9S28/Zf3ePu/3qOloZWspdP4+i/uuOxrkZRSnD6Qy6YXtrHxuS34Bvhy35MrWPTA/H5bfKO5Jx2ghgClFLvW7eOZx16koqCKuSuyWP744ktOmQTGcNWWV3bwj5+9Tk1pHSYfEyMmJZMxZQQZ00aQddvUPuVqs7RbObj5KDtW72bXun20NLTiH+xH8thEwmJCCY8ONe5jQomMD2fivMwhv4y8P7Q0tnJo6zHqKxtprGmiprSWvGOF5B4toKWhFTCGFq+7bRorfrL0sv4mutPe1s6eDQfI3pXDhy9tp6WhlZm3T+eef1t+2YGpobqRjc9tZfMrH1N4sgSTj4kb7p7Jfb/92oDNwWmuTQeoIcTc2s6bv13LW39aj8Vs5YZ7ZnH/7+4hMi78kvfR1mJm73sHObX3LDn7z3Jmfy7m1nbCokNY9thibn5gfq/Kx3dltVg5tPU4u97ZS2luBXXl9dSW19NYc75AX1BYAPO/fj2LvjOf5LGuX4p9INltdnL2nWXzyx+z5dUdmFvanb/zD/YjOTOJtAnJpE0YTuqEZJIzE/EL8O3be9rtbH55By//chVVxTWYvL2Yfss1vQpMbS1mNj63lVf+/S2a61vInDmKG79+PbPvmNHnvy1taHGbACUi+UATYAdsF2v01RigOrU0trLqP99h9Z824Gny5Gs/X87tP7i5V8Npdrud7J05vPrr1Rzaegy/QF9u+vYNLH10ETHJUf3abqvFSl1FA4UnS9j04jY+XbMHm9XOmOsyWPCN6xl5TSqJo+L7/GHrjqqKa9i/6TD7Nh3m0JZjNNe3YPIxMXdFFgu/NY/YVGOVYn8v829rMXPgwyO89ItV5GcXkTE1jW/9ZgXjrx9z2UPAlYVVrHt6Exuf20JTXQuTbhjH95/65iWXmtGuPu4WoKYopaovZfurOUB1Kj1Xzt9++BK71+8nbkQMi+6fz4Q5Yxg5ObVXE+JnD+Wx+s8b2L5qF0opZi+/lqzbphGTEkVMShQhkcH9OpldX9XA5pd38P7fN1N8usz5fFRSJEmj40nMiCcsOpTAsAACQwMIDAsgKCwA3wBfTD5G6QtvX5OzJIanyXPQ5jSUUtistvPL0a12rO1WzC3ttDWbMbeYMTebaW0yU1tWR3VJLdWlxnL0qqJqyvOrAIiIC2PqVyYy5SsTmXzj+H5PZ1R6tpyTn50lZ+8ZTu07S/7xIjrsHSSkx/Kt36y4rGXiYAwdn9iVw9q/buTTtz8Dpci6fTq3P7KIsVmXfi2UdnXSAeoqsO+DQzz/09c5dzgfgLDoEL75q7v4yn1ze/WBXVVcwzv//T7vPrvZmVUCICE9lq/9fDlzV2T1ayBQSlF4qoTCE8UUniyh8JRxX5xTirm1/eI76CIxI470KWmkX5NG6oThxI2IISI2rF9X17U0tlJ4soSCE8UUniii4GQxOXvPOiv9XgpvXxOR8eHOku8jJ6cx5StXptpufnYRr/1mNTve2u1M3hsUFkDGtBFkTB3B6OkjuWbBhMtafamU4pM1e3jux69RlltBYGgAi+6/gSUPLrzktEaa5k4BKg+oAxTwv0qpZ7vZ5gHgAYCkpKRrCgoKBraRLq62vI5jO07yzsqNHP/0FKnjh3PPL+5gylcm9GrYrPMbd3l+FWXnKtj04jZyjxYQkxLFnDuvY/YdMxgxKeWKfktub2unub6V5rpm57251YLFbHFWurWYrcZ9m4W844Xk7Dv7uQq7IkJYdIgzGASGBeDja1TX9fX3wcffB08vD+y2DkcRQ6OgobXdSlNdC011zTTWNBlL2KubqC2vd+7b5GMiMSOOtEnJxKfFYvI5vyzd5OOFb4AvvgE++AWevw+LCSUoLPCKn13kHSvglV+v5pPVe5xDtxlT00ifOoL4ETG9fv9Te8/wtx++RPbOHFLHD2fZY7cwa3o8fls2QX09hIbCrbdCop5b1C7MnQJUnFKqVESigM3Aw0qpHT1tr8+geqaUYsfqPfz9R69QUVCFt6+JSTeMY8biKUy/5ZrLWlTRVUdHBzvX7uW9v2/h0NZjdNg7iEuLZvbyGWQtNYYCg8IDXWLpcHVpLQXZRVTkV1FdUktVcQ01pbVUl9TS2thmVNdtbae91dJt+fbOa6ACwwIJjjifnSIoLJD4ETEkjUlg+JgEYlKiXKK/LY2t5B8vIu9YIXnHCjh3JJ/snTn4B/lx28M3seyxW/qU166prpmC7CLe/d/NbH3tE+Ms/dcrKKsYzdf2PILv+2uh4/wFzcrDg3PjlzJi40qIiemPLmpDkNsEqM81QOQJoFkp9ceettEB6uJsVhtHd5xkz4b97N6wn/K8SgBSxw9n0g3j+Oq/3tbr5b4N1Y3sfGcfO1bvdgYrMD7YgyOCCBkWTMiwYFIyk1j8/QUMH+Oa36aVUlgtNuxWG55ennh6eRqZJVx47qTgZDHrn/6AghPFNNU201Dd+LkzRv8gP5IzE5k8fzxLH13Uq7RUSimjTMozm8g7VkhtmbF/k4+J5Y/fwl0/XsqqlQ3M/mkWaeSiTCZkyRLIyMB8NAfPd9dhwkZDZCohx3dBtC5YqX2ZWwQoEQkAPJRSTY7Hm4FfKaU+6Ok1OkBdHqUUBSeK2bNhP4e2HefwR8fx8fPmjh8uYdnjt/SpjENjTROHPjpOXXk99VUN1Fc20lDdSF1FPaf352JttzJxXia3PXQT1y6+xiXOONxNdWkt+zcdYcdbu9j3wWFMPiYypqY5cw/Gj4glZVwSKeOSiB4+rNcBtr2tnR1v7WH9M5s49dkZYlOjyZw1iuQxiQwfm0j6lDTnlxrz4uX4vruGA0zmX0au481PEwCYOxcaThSzyfdWxpoPwrJlsHp1v/1baEOHuwSoVGCt40cv4HWl1JMXeo0OUH1TlFPCCz9/g0/WfEZoVAhf+9kypi2aRHTysH4NIJ0Xa65/ZhNVRTVEDx/G9XfMIHF0AokZcSRmxOmSCl+glKK5voWzh/LY/8Fh9n94hNyjxnxrZHw4Nz9wI7d870ZCh/Xfxa752UW89+xmtryyg+b6FhLSY1n22GJu+va87heYFBZCSgrK05MbknPZdiaBYY61EVVVMGYMfPxqEZHTUo2hv/x8PSelfYlbBKje0AGqf5z87AzP/+Q1jmzPBsDk7UX8yFgSMuJISI8jfmQss5ZNJyDYv0/vY7fZ2bV+P+ufNhZw2KznE7sGRwQRlxZNYFgAfkF+BAT54R/sj3+wH96+3s4FBwEh/ky6YVyv59AGi81qI3tnDoWnSrBZbM6cgO2t7bS3WWhvs9BYYyy+qC2ro6683lknysvkSeas0UxZMJGpCyeSMi6pz8OPlYVV7N5wgOriGmrK6ig8WUzOvnOYvL2YuWw6N3/nRsZfP+bC77NyJTz8MCxbRuX/rCYz0whMAMOGwfHjEBUFLF8Oa9YY2z/4YJ/arQ09vQlQugb3VWT09JH8YesvOX0gl7yjBRTllFKUU0JBdhG71+/HbrPz7P9/iWWPLea2R27qdaDy9PJk1u3TmXX7dOw2O2V5lRTnlFKUU0pxTgnlBVU017dSWVhNa2ObcWtq63ZfSaPjGTdrDGNmpDNmRjrxI2Ndas7IYrZQcqaMU3vPsnfjIQ5uOfq55fqdTD4moxSInzfBEYGEx4aRkB5LeHSoI1t8LBPmjOlTCqquinJKWPvfG9n43BZsVjteJk/CY8OITIjggT98gwX3Xk9IZPCl7azesZIxI+PC26WnG/d1dRfeTtMukQ5QVxkRIWNKGhlTPp/HzWa1cfpALm/89m1e/MWbrHlqA7f/4BaWPnJTn2r2eHp5kjAyloSRsResuNrR0YHNYsNqMZaT15bVs3/TYQ59dIztq3by3rObAQgKD2Tk5BQiEyKIjAt3Xl8UERtGQIg/fkF+zmXefcnkbrfZaW+zGMvNa5ocN+NxRX4lhadKKDpVQnlepfN6o2EJEcy58zqmLZpMxtQ0vH2NgOTtaxqQYovl+ZVsX7WL7at2cu5wPp5eniy8bx53/ssSYlKiet+G0FAAzEdzmDvXOHvqOsQ3dy5s2wZRp08bT4YNTLkRbejTQ3zal5w+cI5Xf72a3ev3IyJEJw8jIT2WhPQ4EjPiiR8ZQ1B4IP7B/oREBvVrBoTudHR0UHiyhBO7T3Nydw55xwupKa2jtqzOGRy64xvgg8nHhJfJ07lqz9PkiYeHGK9Tynlvt3dgabM4S3VcqC6UycdEQnosSaPjSRqVQOKoeFLHJ5E0OuGKnt1ZzBZnnsOGKqMKcWN1E/WVDRz5ONtZjXj0tSOZ89UsZt8xo3+GSAsLUSkp2Do8SSGXkDEJbNtm/GruXDhxAuaNLGJLXiqi56C0Hug5KK1fnTmYy+71+41CgTmlFJ8u/VwyUzDOyLKWTmP544sZe91FhoD6md1up66igZqSWmrL651DheYuVXCtFht22/kLcu02oyiheIhxRiEYtaE8BB9f42zH21GV18fPm4DQAIIjjOuiQiKDCI4IIjgyaEBXJ1YUVLFu5Ubef26rM6t5Vx6eHqSMS2LOV7O4/s4ZxKb0/zLvs5OWM+LwGrJ9JxO96x0iJxkBqLISVsws4vdnbuMa9Co+rWc6QGlXlFKK6pJaSs+V09LQSmtjG3nHCp0JQ+NGxDB90WSmLZrM+Nmj+z3Z6dXEbrdzYtdp1v71fXa+/RmIMGvZdKYsmEhwZBAhkca1ZyGRRrn7Kz4vV15Ow7gsQqpzwcvLyB6Rng6nT6PWrUNsNkhNhV36OiitezpAaYOircXM1lc/Ydf6fRzZdhyL2Yqvvw+T5o9jzIwMImLDCI0OIdxRI2qgz0Bckd1up63JTEtDKy0NrVQV15B/vIj87ELyjxdReLIYi9lKUFgAi74znyUPLiQqMXJwG11eDg89BGs/n0kCDw9YuhSefloHJ61HOkBpg87c2s6R7dl89t5B9r5/kIqCqm63c65qcwyp+fh542nyxOTtRdqEZGYsmcqkG9yv0KHVYiV7Zw77PjhM9q5TmFvanYs/7FY7VovNOQTZnWEJESRnJpI8NpG0iSlcd9tU1ytJUlQE69cbq/XCwmDJEj3npF2UDlCay2lrbqO2vN5ZyLC2vJ7G6qbz1we1WbC0G49tVmPl3Kk9Z2htasPb10S6Y8Vh6oRk0iYmkzQ6vl/K1/cHpRQN1Y2U51Vy9lA++z44xKGtx2hrNuNl8iRj2giCI4LwMnni5W0klfXy8sIv0Bf/YD8CQvydt/DYMJLHJuoif9qQpa+D0lyOX6Af8SP8iB8Re8mvsVqsHNl+goObj3Dk4xNs+NuHWMxWwLiYNTYthvCYUEKjggmJDCY0KoTQYcH4BvoaGcX9vfEN8MXH3xuTtxcenh6fz73nIagOhVKOW4exms/abnXcbM7HrY1tNNe3GNnV61toqW+hpqyOstwKyvMqaWs2O9sdlRTJvLtnMe2mSUycl9mnlFKapukzKM0N2G12is+UkXukgNwj+RSfKTOWWVc2UF/ZQFNdy4C0w8PTg8DQAMJjQolJjSImOYrY1GhiUqJIGhXvchcRa5or0WdQ2pDk6eXJ8NEJDB+dwNy7sr70e6vFSmNNM+YWM+2tFsyt7Zhb2mlvbcdmtdPhXGLegd1u1IHy8BAQwcNDEBHEQ4xqvZ3Vex2P/YP9CQz1JyA0AF9/Hx2ANG0A6QCluT2Tt4mIWJ29QNOGmiuff0XTNE3TekEHKE3TNM0l6QClaZqmuSQdoDRN0zSXpAOUpmma5pJ0gNI0TdNckg5QmqZpmkvSAUrTNE1zSTpAaZqmaS5JByhN0zTNJekApWmaprkkHaA0TdM0lzQoAUpEFopIjoicFZEfD0YbNE3TNNc24AFKRDyBp4GbgDHAChEZM9Dt0DRN01zbYJxBTQPOKqVylVIW4E3g1kFoh6ZpmubCBqMeVDxQ1OXnYmD6FzcSkQeABxw/tovI8QFo20CKBKoHuxH9TPfJPQy1Pg21/sDQ7FPG5b5gMAJUdyVJv1R3Xin1LPAsgIjsv9xSwa5O98k96D65vqHWHxi6fbrc1wzGEF8xkNjl5wSgdBDaoWmaprmwwQhQ+4CRIpIiIt7AXcD6QWiHpmma5sIGfIhPKWUTkYeATYAn8A+lVPZFXvbslW/ZgNN9cg+6T65vqPUHdJ8AEKW+NP2jaZqmaYNOZ5LQNE3TXJIOUJqmaZpLcukANRRTIolIvogcE5HDvVl26SpE5B8iUtn1+jQRCReRzSJyxnEfNphtvBw99OcJESlxHKvDIrJoMNt4uUQkUUS2ichJEckWkUcdz7vzceqpT257rETEV0T2isgRR5/+3fF8ioh85jhOqxyLytzCBfr0oojkdTlOEy+4H1edg3KkRDoN3IixNH0fsEIpdWJQG9ZHIpIPTFFKYyuyZAAABiJJREFUufVFeCIyG2gGXlZKZTqe+z1Qq5T6neMLRZhS6l8Hs52Xqof+PAE0K6X+OJht6y0RiQVilVIHRSQIOADcBnwT9z1OPfXpTtz0WImIAAFKqWYRMQGfAo8CjwNvK6XeFJG/AUeUUs8MZlsv1QX69D3gXaXU6kvZjyufQemUSC5MKbUDqP3C07cCLzkev4TxweEWeuiPW1NKlSmlDjoeNwEnMTK5uPNx6qlPbksZmh0/mhw3BcwDOj/I3e049dSny+LKAaq7lEhu/YfooIAPReSAI53TUBKtlCoD44MEiBrk9vSHh0TkqGMI0G2Gwr5IRJKBScBnDJHj9IU+gRsfKxHxFJHDQCWwGTgH1CulbI5N3O7z74t9Ukp1HqcnHcfpKRHxudA+XDlAXVJKJDeUpZSajJHN/UHH0JLmmp4B0oCJQBnwp8FtTu+ISCCwBviBUqpxsNvTH7rpk1sfK6WUXSk1ESOzzjRgdHebDWyr+uaLfRKRTOAnwChgKhAOXHBo2ZUD1JBMiaSUKnXcVwJrMf4Yh4oKxxxB51xB5SC3p0+UUhWO/2QdwN9xw2PlGP9fA7ymlHrb8bRbH6fu+jQUjhWAUqoe2A5cC4SKSGcyBbf9/OvSp4WOIVqllGoHXuAix8mVA9SQS4kkIgGOiV1EJABYAAylLO3rgXsdj+8F1g1iW/qs80PcYSludqwcE9XPAyeVUn/u8iu3PU499cmdj5WIDBORUMdjP2A+xtzaNmC5YzN3O07d9elUly9GgjGndsHj5LKr+AAcS0X/wvmUSE8OcpP6RERSMc6awEgz9bq79klE3gDmYJQFqAB+CbwD/BNIAgqBO5RSbrHwoIf+zMEYMlJAPvDdzrkbdyAiM4FPgGNAh+Ppn2LM2bjrceqpTytw02MlIuMxFkF4Ypw0/FMp9SvH58WbGENhh4B7HGceLu8CffoIGIYxhXMY+F6XxRRf3o8rByhN0zTt6uXKQ3yapmnaVUwHKE3TNM0l6QClaZqmuSQdoDRN0zSXpAOUpmma5pJ0gNI0BxHpcblrN9vOEZHrrmR7LvL+PxCRb/TDft4UkZH90SZN6286QGla78wBBiVAObIL3Ae83g+7ewb4UT/sR9P6nQ5QmnYBIrLYUZPnkIhsEZFoR5LS7wGPOWrazHJcOb9GRPY5blmO1z/hSF66XURyReSRLvv+hiNp5hEReUVEghy1ckyO3weLUT/M9IVmzQMOdiYSdez7KRHZIUadpKki8rYYdYR+49gmQETec7zXcRH5qmNfnwDzu6TU0TSXof8oNe3CPgWuVUopEbkf+JFS6oeO+jzO+kMi8jrwlFLqUxFJAjZxPuHnKGAuEATkiMgzQDrwM4zkwdUiEq6UahKR7cDNGFk57gLWKKWsX2hTFkYdpK4sSqnZYhTwWwdcg1E+5JyIPIVxxleqlLrZ0d4QAKVUh4icBSZ0s09NG1Q6QGnahSUAqxw5xLyBvB62mw+MMVKMARDcmXcReM+RoqZdRCqBaBy1fjoLV3ZJNfQcxpDbO8C3gO90816xGLnauurMU3kMyO5M8yMiuRhJl48BfxSR/8QoGPdJl9dWAnHoAKW5GD3Ep2kX9ldgpVJqHPBdwLeH7TyAGUqpiY5bvKOgHkDX/Gl2jC+GQjflE5RSO4FkEbke8FRKdZdMs62bdnS+R8cX3q8D8FJKncY4qzoG/FZEftFlG1/HPjXNpegApWkXFgKUOB7f2+X5Jowhu04fAg91/iAiEy+y363AnSIS4dg+vMvvXgbewChH0J2TwIiLtrwLEYkDWpVSrwJ/BCZ3+XU6kH05+9O0gaADlKad5y8ixV1ujwNPAG+JyCdAdZdtNwBLOxdJAI8AUxyLHk5gLKLokVIqG3gS+FhEjgBdy2G8BoRhBKnubAQut9DlOGCvGBVOfwZ0Lp6IBtrcJfO3dnXR2cw1zcWIyHLgVqXU1y+wzVqMBRtn+vhejwGNSqnn+7IfTbsS9CIJTXMhIvJX4CZg0UU2/THGYok+BSigHnilj/vQtCtCn0FpmqZpLknPQWmapmkuSQcoTdM0zSXpAKVpmqa5JB2gNE3TNJekA5SmaZrmkv4Prcfje/owoMsAAAAASUVORK5CYII=\n",
+ "text/plain": [
+ "
"
+ ]
+ },
+ "metadata": {
+ "needs_background": "light"
+ },
+ "output_type": "display_data"
+ }
+ ],
+ "source": [
+ "pval = utils.multivariateGaussian(Xval, mu, sigma2)\n",
+ "\n",
+ "epsilon, F1 = selectThreshold(yval, pval)\n",
+ "print('Best epsilon found using cross-validation: %.2e' % epsilon)\n",
+ "print('Best F1 on Cross Validation Set: %f' % F1)\n",
+ "print(' (you should see a value epsilon of about 8.99e-05)')\n",
+ "print(' (you should see a Best F1 value of 0.875000)')\n",
+ "\n",
+ "# Find the outliers in the training set and plot the\n",
+ "outliers = p < epsilon\n",
+ "\n",
+ "# Visualize the fit\n",
+ "utils.visualizeFit(X, mu, sigma2)\n",
+ "pyplot.xlabel('Latency (ms)')\n",
+ "pyplot.ylabel('Throughput (mb/s)')\n",
+ "pyplot.tight_layout()\n",
+ "\n",
+ "# Draw a red circle around those outliers\n",
+ "pyplot.plot(X[outliers, 0], X[outliers, 1], 'ro', ms=10, mfc='None', mew=2)\n",
+ "pass"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "*You should now submit your solutions.*"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 8,
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "\n",
+ "Submitting Solutions | Programming Exercise anomaly-detection-and-recommender-systems\n",
+ "\n",
+ "Use token from last successful submission (tanmayss2109@gmail.com)? (Y/n): y\n",
+ " Part Name | Score | Feedback\n",
+ " --------- | ----- | --------\n",
+ " Estimate Gaussian Parameters | 15 / 15 | Nice work!\n",
+ " Select Threshold | 15 / 15 | Nice work!\n",
+ " Collaborative Filtering Cost | 0 / 20 | \n",
+ " Collaborative Filtering Gradient | 0 / 30 | \n",
+ " Regularized Cost | 0 / 10 | \n",
+ " Regularized Gradient | 0 / 10 | \n",
+ " --------------------------------\n",
+ " | 30 / 100 | \n",
+ "\n"
+ ]
+ }
+ ],
+ "source": [
+ "grader[2] = selectThreshold\n",
+ "grader.grade()"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "### 1.4 High dimensional dataset\n",
+ "\n",
+ "The next cell will run the anomaly detection algorithm you implemented on a more realistic and much harder dataset. In this dataset, each example is described by 11 features, capturing many more properties of your compute servers, but only some features indicate whether a point is an outlier. The script will use your code to estimate the Gaussian parameters ($\\mu_i$ and $\\sigma_i^2$), evaluate the probabilities for both the training data `X` from which you estimated the Gaussian parameters, and do so for the the cross-validation set `Xval`. Finally, it will use `selectThreshold` to find the best threshold $\\varepsilon$. You should see a value epsilon of about 1.38e-18, and 117 anomalies found."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 9,
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "Best epsilon found using cross-validation: 1.38e-18\n",
+ "Best F1 on Cross Validation Set : 0.615385\n",
+ "\n",
+ " (you should see a value epsilon of about 1.38e-18)\n",
+ " (you should see a Best F1 value of 0.615385)\n",
+ "\n",
+ "# Outliers found: 117\n"
+ ]
+ }
+ ],
+ "source": [
+ "# Loads the second dataset. You should now have the\n",
+ "# variables X, Xval, yval in your environment\n",
+ "data = loadmat(os.path.join('Data', 'ex8data2.mat'))\n",
+ "X, Xval, yval = data['X'], data['Xval'], data['yval'][:, 0]\n",
+ "\n",
+ "# Apply the same steps to the larger dataset\n",
+ "mu, sigma2 = estimateGaussian(X)\n",
+ "\n",
+ "# Training set \n",
+ "p = utils.multivariateGaussian(X, mu, sigma2)\n",
+ "\n",
+ "# Cross-validation set\n",
+ "pval = utils.multivariateGaussian(Xval, mu, sigma2)\n",
+ "\n",
+ "# Find the best threshold\n",
+ "epsilon, F1 = selectThreshold(yval, pval)\n",
+ "\n",
+ "print('Best epsilon found using cross-validation: %.2e' % epsilon)\n",
+ "print('Best F1 on Cross Validation Set : %f\\n' % F1)\n",
+ "print(' (you should see a value epsilon of about 1.38e-18)')\n",
+ "print(' (you should see a Best F1 value of 0.615385)')\n",
+ "print('\\n# Outliers found: %d' % np.sum(p < epsilon))"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "## 2 Recommender Systems\n",
+ "\n",
+ "In this part of the exercise, you will implement the collaborative filtering learning algorithm and apply it to a dataset of movie ratings ([MovieLens 100k Dataset](https://grouplens.org/datasets/movielens/) from GroupLens Research). This dataset consists of ratings on a scale of 1 to 5. The dataset has $n_u = 943$ users, and $n_m = 1682$ movies. \n",
+ "\n",
+ "In the next parts of this exercise, you will implement the function `cofiCostFunc` that computes the collaborative filtering objective function and gradient. After implementing the cost function and gradient, you will use `scipy.optimize.minimize` to learn the parameters for collaborative filtering.\n",
+ "\n",
+ "### 2.1 Movie ratings dataset\n",
+ "\n",
+ "The next cell will load the dataset `ex8_movies.mat`, providing the variables `Y` and `R`.\n",
+ "The matrix `Y` (a `num_movies` $\\times$ `num_users` matrix) stores the ratings $y^{(i,j)}$ (from 1 to 5). The matrix `R` is an binary-valued indicator matrix, where $R(i, j) = 1$ if user $j$ gave a rating to movie $i$, and $R(i, j) = 0$ otherwise. The objective of collaborative filtering is to predict movie ratings for the movies that users have not yet rated, that is, the entries with $R(i, j) = 0$. This will allow us to recommend the movies with the highest predicted ratings to the user.\n",
+ "\n",
+ "To help you understand the matrix `Y`, the following cell will compute the average movie rating for the first movie (Toy Story) and print its average rating."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 10,
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "Average rating for movie 1 (Toy Story): 3.878319 / 5\n"
+ ]
+ },
+ {
+ "data": {
+ "image/png": "iVBORw0KGgoAAAANSUhEUgAAATAAAAHkCAYAAAC5VHPKAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjMsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+AADFEAAAgAElEQVR4nO29f7TfVXXn/d5zg6GgFLDGkoCF1GgnglxNLkKmdGVCNeo4hZnpTISqOMXFeoZqmeI8DT5lVn+lM81MR0vXU33qr8bWBrFOizwu21sLUn0WVC5pLwRDNWl0QRIktqD9wUxq0v388f2cy77nnp+f35/Pd7/Wysr9fn6eX5999tlnn32ImaEoijJE/knXCVAURSmLCjBFUQaLCjBFUQaLCjBFUQaLCjBFUQaLCjBFUQbLYAQYEb2eiL5CRIeI6Nau06MoSvfQEPzAiGgGwFcBvBbAEQALAK5l5gOdJkxRlE4ZigZ2GYBDzHyYmf8BwCcAXN1xmhRF6ZihCLB1AJ4Qv48UxxRFmWJWdZ2ARMhxbNnYl4huBHAjAMxgZtMZOGvZxd/53jNx2jf+3vlwfsEZoL99dun3iYu+C6u/9r+W/jecuZHx9wco+Cz72qqY59hpkdjpyXl3yrWu/L7slc/iq4+cEX2mSbfvPa7jsWN2fdnEzpv0Hz35Xcvek1KvKci6Mum2/7exj4fqu2qaYudd16am58yNjG89/fylciz7LFkef4tn/oqZX+S6big2sCsA/Bwzby9+vwcAmPm/uq4/i87l19BVS7/njy1i+9rZUu+ucq/SHnOLp7AwOxO9ro76HHubOLpzC9btvr/rZCzxx/ypfcy82XVuKEPIBQAbiOgiInoegDcDuNt38Xe+98xlv2ON7fDe5efnjy0m39s1h/fOrkh/DvPHFnF055YVxyRzi6e8v+1rc3GlveozQ8j6tPNl4zuf2yaq5ieWTsBdjmXf4RNeVd/RBIMQYMx8EsA7AcwDeAzAJ5n5y77rzz7375YazeG9s8v+drH+uuUNLNZAcxqk752+RplzfG7x1Iq057J97SyuufaLWfdITee245csOyfza5eTS9ju2LgP88cWl+Vv+9pZ5732Mfl715r9SWmX98Q0Nt95O72x9rB97ezS9XZnkUKKZnlw657g+bnFU8F0ynf42uzqh90mA/P8MueqXj+IIWQucgh5eO9s5Y+8D/RNrU9l7MMtpXnGMITMRvbYrp4n1BuFtCDT+8Y0BJ+2YHpgX09sP8ek5cSlzy6dl8+ztQFfHk5u27TiGvMMea19n3m+Oe7Lv7zfNQS3n2GnwZd287e815Vu+bxQvcvy89WRq6zs+831Ml/2dbJu7P9t7DzLPMj0hDRceZ95nrk+VwuyiY1iXGny1XnKvamMXgMD0g28qeRodWPQQOz8ltFqzT11asRzi6ewa83+UuXrSkfd7aQKfW03J7dtwqp79znPlanblDIPaWBTIcBy6GvDCVHnhxd7Vk4jrSJglHqoWyjX9byc72wqh5BAObU5VKhGfXap0a6hVx2kGH/rbKCuZ8n8GuGVkr+F2Zml8syZwYpdW3b2MEZdwxzfkBLwD0+bmnmtq22Y9N91x5Wl7rOpq1NTDUxplD4Ny5RhMrUaWJd05TNTZpq+SYzw8mlFVcupj75JVQ3mYyI0IVIHUyHAyqrnodmuGDlDrRChYashxQkxhGvWM+Va8ztFiPi0MJc9LbXM5o8tNu4iU6buh6pxxmZKy+Az+Pvenfv+qRBgsfG2T2sx95kGmTqVDDxXCWVsQBLzgYY+1KofjCyfOw+Ee0y7LLev9Rv1ZbnmOOjazrGpaTGklrV9neu+uoWRLJO+aY8mr2XzfHTnlqQyDb079/1TIcCAcEGmeqKbD8b3wUrNwRYEIS9m1/02VVXx1PtjGo3xNQr5lRnW7b4fJ7dtwtGdW7yN0mUUdgnRJoYidl5TtLmYdnhy26agX57UkG3XFJs6zAG5QtKnCaX4yK3bfT92bFyucdm/62YqjPgxQ3KdhuaqHvN9cOOIpUGet69tKv1zi6fwwC2XJQ9JlPEwtUb8MuprbCFzjKrLfaoOi6pyeO9sVACFzjclfBdmZ1YIrzJaS45NsslF5dNOXXa20QqwucVTS+prykJfs7TktuOXLFv+sGvN/qVzchlNrHH7BGHsPt8ynYNb9+Dw3tkVFe9aSuQ6Zv82S13sd9kqv4lWYS/lsW188j2ud9tLW0J59S0vsp97cOse57A1xMW33+Q9Zz9LLsL2YS/7sZ8XWtbkyr9ryZTrvP137FxdwjhlKVHKUjH7myxrxJ+KISSwcpgohzq+5RG+JTQpQ84yQynXPeZY6tDU995UD/qUvJ3ctgmPv8MfCcOVhpxhurzW9Sy5LOklH16pmZU1CaSUccqz5TWmbZljqW2tbWLvz2n7shxzvgPfs6d2CGk4vHd2RcHIQn1qbvWyc6YH8FVoijdymaHU9rUrw82YGbnQhyXv8b03ZRLBhz0su+fjHwle7xI4qc+OPQuIG9tDM6kuTdDw6M3vDz4XSKt7eY0RVqb9XfHeB533uPIkNd+uMemL5X9u8dSytrrhvrcvHQ9hTw6kmkymRgMbErkaRIrmUFcPf3jv7NIwsyl/J19a7XIpo2mF7mlKsy6Tli4JtZUuwjpNvQYGhHuA1Cn6tnrD3EYtG5RPo6lzeLIwOxNd8G3/9vWorvT6tEX7nb40hHpveY/97gduucx7nyFkQzOktpNYPbfpJ+Za7+oiRXjJcs1xBSmTX9XAGqKvvWtX5GiAbfXydh2FQsX0ka7tZjlU+R6mXgPLkeyh3jNnaZFtK0idBbJnGu1oFK68pOSvipZpe9TLGUFXD+ua8Uw5Z0gVXnVHpfDZp3IJaR0yTbGVCr5giQbfMFtSxgG4ykjDV78u4SW/CbvMUtMwFQIshh0n3Vd4xvaR4pYhP0Kf3cT3Hldlm+fZjfbw3tkV8dBdwjJHs7CFiszLrjX7sTA7szTccgkbO/3rr3tuzaLrnE3qlLov9I/5GHJj3qcMIW1cZX3i0me9EVHlO2XZudIaG6q7sK9//B1pgiC0V4DLfcfHrjX7kzZqsb+JdbvvX/aO1HyPVoBJiR5bziDtGilr9szMSghXWGUbVyXZafUJS/P8g1v3LEtPHXYTl5A0mLyEBKIrrLFPm3ClV84iuqJZhDScg1v3LC0NC73P5WcVm1114arbHRv3LeXBdC4mH7H6iWnYsft9ZeMLn22Q609dvoGpm6YAbhum/XxXublmj2Mjl9EKsHW771/KfKzw5VpIX69njh3duaXR9V0pPkbAcx/OhvvevkIDc7ljVEHmN2d4sWvNfhzduQUHt+7BnQc2ObVbV1m6tDJ5r9FcXGnZvnY26EYhn+1amJ6LbwG4zzYVs1m5opjIe2Jtzzf8thfI23mV7c51LqXTBibt0ZUG+Q3K58vyk3lTN4oBu1FIyhq0h7KL0ZAM0XUwtImCPjD1RvwmaGuK296NKBUpvHIMuVVdRXLvj+1nGKJMHdi7OlUlFHnCRV0TBfZ7+uLwGiI2jDXk1OsoBdiZG3nZrF0slr25VgZ0s6+VNpy5xVN4yYfjDo/ymTE7hr0uznhh+7QT2WCljcllcH1qbvWSEHOtn5Pl4xrC5nwc9v0mRpRvey9X7K9Um8+Ojfucz3XVn8EITLndm8G3RV0ofQuzMyveZcePk9jmjDLxs2Qdh2b5gOWzxqHZ4FAabPtjyr3yWwEmZSG98kP5zAkGqkPIKacv/mpyaFUlTX3Jj7KcKisYdAiJlVLfpXVJXNEl7Z4oNcwzgBXv9s0Wmefa6XNd53q+S9MK9Zi2wTtFA0j1rjZpCfXa5lnSLiR96Hz1E8pTSNvwPQ9YroH5/OFcflYxLSolZHdKBI6U4a8sc/l/KE3yuhj2db5oGq6Rh10PMW1XNTALnwG1CYO3bZzu2nib+v66NRhTDjlRGKp67Z/ctglXvPfBzjSxUHtqux2UeV/VtZDyfvN3igZmP9ukfeo1MCPJfQZUU2i5RuFQD2E3gNRGVNeqAfs5qe9PdeNIxZSDr+xdRnzfRh8uLcz1Ma26d1/QJiQpMxFgb2Rix/xypcm8ty7hlapBlXlf1bWQ8n7z9/a1s1GNSobgAdImPKZKAxsrbfbqc4uncOeBTVPl+mDTtTYNrNRWhuI2U4ap1MBi0TB92LNS9rmUtZKuv1NtH760hmxBKR7koTWOdlpD5bVrzf5lkVBDtikZyfbozi3RCKMxXPXhswWmPqPMtSHhZc/C5UZkdb3bpe3ZwsrWXspStfxy35+y8iWEamAWQ+vJfLa2PmgJkmlzWK2LquU2tPbsYio1MElOL59S2X1yGtyxcd8yLcQILXsRb9M7JEtsvzsZBNGmT2VZF3U6OYfWpabQhfBytTVf+6ta/6qBKaNgDJpGFfqs4VZN29RrYEpz9GXrsWkWXkDzG8hWoUnBOloBZi/9STUgh8Ifl5kUyME4RbreEzL22ul1Tan78mRPTBiDu32N/Nu88/De2RVRCmxjvXEqNe9PCVyXuuTFl6+UYYlrQkYOeVMdiEMTO7nbotnPN5M/PheJUPys3KFZSpnHlgCZtpGzN6f5W+5o7pvwcTHqIaRxnstVYX3OnG2p6aH3uNJWdlMP17Pmjy3iqrfc4J0AMI2srCuFnQ5XGuSxNpcGlXlXlTaR8r7QNU21xyqOrKl5AsI+h/I9UzuENAuFzX52EinhpYHR+DnZ1wDPbS6bYkgta5z0aSi+Sp8/trgUsQLwa1quxeeuBrThvrcHd8A2cb1CuJZlmWMp27ulBjT0aQSm3kLaWcpCaxf2fa4P3bUY3bW0RuazjFafEsmjrgkF85wXL5yIXpuyqYfdhuwlWTJGX4jRamDfevKFyyIODGWBb5m0Vlkoa7/PhKiushlpnVuPDZU+G9XbJqUsZDuy289UamAyTEtOONwmSHFWNLiEQYwqwsIuG1evHlrAG4p+Kq/NsVXllJcL30LtssfK4NunIPb8KhpT6gJ+FyF7a66zcYqWahOKCBtitALMfJhzi6e867Bstd4YTVO8jXOMy7732xVlN3IpOELRBFye2jYuPxzXx7J97cQwL/Mn1XlpVwwZ4E06bjt+yZJWZ79z15r9zkkIF6YeXe+wrzN1b9eBOW/uc30o8lhsgiGUhg33vT26qsE1BDbl5BMosbr2rUyQk0Mubjt+ibdcckcEUnlw1a0cVqfkM8RoBdj2tZMPzLfTjFRZ5YJT4LkKcI3fTWA2l6YkK0D2wLHlQXaazfulsdQO8uYKyCfTbjeAp+ZWr3i/a+p9bvFUcEreNHRzjd277ti4D9vXzi7l5a47rlw6Zl+/fe0sbjt+SbCHlpMGoXQbdq3Zv1QGrusWZmdWCEL7XfJZMS6+/aYVdrU7D2xayrOZpbXblsG0E9PW5F4H5pnynlBM/4XZmaU2Y9JuykDWgQtTLj6h6+pUU3AJZLmzu8ybyXPO8Hu0Asz0xKvu3bes95bGcJ9gMcLJNiTK59iY465ezHzQvvfYx1wbIJiPzXZb8O1WbTcAaeg3mPfIcrA/WlsrNOfN9mquaKQySug1134x2IO7zsmyMu+TEzGh5912/JKl87Hosr4doMw1rjqzy1V2Mub6HRv3ragHU36yQ5XC1p7ZMx++zI+dflfezHnz3FBZuDCdius4ADx68/uj97uQz7zzwKal8pDpMmWVYzscrRHf9sTPMSyXNfrLd3Q1cWC0zrLrIF3pdvWIrrWW5t6Ya0TseKyuYmUb6sFDa0RT1o/abgSutaiPv+PUMsFlNBv5vy/NXbYbu+5ccb1i2pEdrSQnP75n98qIT0QXENHniegxIvoyEd1cHD+XiD5HRAeL/88pjhMR/RoRHSKiR4jo1SnvsaeyczaW9Z13TYU3Tco7pVq/MDuDVffuW3F9qurv0qrkcNj08q5YTaHe3hyzbXGua10agES6uYTcY1xIAWXn00T1yHFYdsV9k9qiyYtPaBmMe0mZSRzXc3ON+LLuTLpc27nFPP5jkzqA31G1zKxtF0PIkwDezcz/FMDlAH6CiDYCuBXAPcy8AcA9xW8AeAOADcW/GwF8IOUlOzbuW6pEX8H4egZZmfaM29yie6MNu2etEtM9JY0S1/DQTmOqX838scVgQzJDx9T82UbaFM0wFozQpM8eXgGTIXaKTU0+x8ZnrwLiS5ZMfk1YbDt9so2YNjp/bHHFc30G9ZBAM+fWX7eYJEh8+GyH88cWo3ZBuQu7uc+mzmVfrQswZn6Smf+s+PtvATwGYB2AqwF8rLjsYwCuKf6+GsBv8YQ/BXA2EZ0Xe4/8yHzaS2h5jflfNqCDW/csqdkuA7ykTM8Zmik1jSqmcZje177ONZnhep9rA1P5Id92/JJls2chR1J7xtI1G5u6xMs1FHFtYx/TEFIEr8s+48N+v2kH63bfH11643qHbX+zy0e2M/ucmfmswuG9s14h5bPlSlI1P7tzSJndddGpEZ+ILgTwKgBfAvBiZn4SmAg5AGuKy9YBeELcdqQ4FiQ2bQxg2fjeID8UWwMzlXfXHVc6G3dsyjzG9rUrtzWzh2WuHl0O8dZfN+kl7R7YNeRzaRgx7+5da/bjrjuudBpcpdZgL+My6U6xQ8oPyNSN66NyaVuuIXAqoeGewa5H+/1mWOsahgErh/v2++z6tetIvt9O5/rr/LbDVLefHRv3Lb3T15HEcK2rtFe72Pk09jWDb5bdpjMBRkTPB/A/AfxHZv6b0KWOYytmHojoRiJ6iIge+g4myx1kjx8SLmaZjcvmZfd4c4un8OjN73f2rrFpaolrtxh72YlMt0/jmVs8hYtvv2nZMy6+/aYVmojc6cdOkywbo2H5HFDN1vEhLVQOp1/y4eUaa0rPKrVAkw+TLok0Exh8vkTSlmhcBXwfR0gA2h2DbVuUndCuNftX5FvO4oXqWbYvn/2yivbqQ86Cu+7xuSWZ9+xas3/ZcjHzLDtqsN3Wbdtoyowz0NEsJBGdBuAzAOaZ+b3Fsa8A2MrMTxZDxPuY+eVE9BvF33fY1/meXyUeWIoPSpNLZcrMQlWJheXKS535a3JWrcpsZBsMaQlbWVxtpe5y79ssJAH4CIDHjPAquBvA9cXf1wP4tDj+tmI28nIA3w4JL4PsZVO9vQF3ry6feXTnlmUOdzFiKrDdixobm+86l8Yhjfi+WUvfM43DqbzO1nZs729Trr79LeW9ZuLDdc636sF3vU2KQdkmxQheFzE/tFj7MdpirvNok4RGHTlDd1vjTAmz5KKLIeQ/A/BWANuIaLH490YAvwzgtUR0EMBri98A8FkAhwEcAvAhADc5nrkCY3BcmJ2JzkzZ2A6N0razbvf9Xi901z2+Htj2vJb3uxwXzXBYOgG67jdDRZdtyMbcK8+ZaBNyeLlj43NuGeaeOw9swrrd90c1v9UPn7FMKMt3+cpGDnfr1mDMcNs1hIy5bwBpy4tC18u8uSYc5LPWX7e4NGRPTUuu+0Quoe/IuI+kfGt2u123+/5leXeZN1xMjSNriNwNMFKGBrZq3eSwUw4hzXvKbuqRGjfMNWyVeTTPiaWjjCNrLB1lSXlviiNnqG34znc93DXvjzljp+TvrjuuLFUnvrbSqyFkW8jhSY5zo41LffdVYEgzSnWkTTEs275Vth/Y3OKpJOHlmqndsXHfsvVw0kNbvtPVQGUezcdo0uEy+gPuYaDLVcBXLnX6FIVm/AyxYalsG65255u9NsJe3lOnhhUb3tnrgX1pjbFrzf6kOnGVjWyzg3CjaBK5aNe284SwZxddleGzSYTsAa73hzzWDdI50fy211nKj8o4d6bk1zXtbsrNHl6a/4/u3FJak4ytezTItJshX8yeJDH146onGQzR/khsYep6p+uZofWxdjvwRQABJh+wT4Ckzsr53pMTMz9k13TNZktuO37JshnFlG/FfofvvIvRCjAbu+B9oWpis1pAuV6/bMOztYJYxZppbnu6P3VbtRRDbKx3lM8w5ezLv+t9sbIyeVmYnXHmy2ikrnoyx2QHZ0j5aFyrHmxc3v6mzHwRaXMdXmPYmmLMM1+m2ZVHc3+s7d95IG668LXFMhrfqAWYnJHz7WQMLK/sucVTK7zepVOinBVyzc4ZfAHtfGmUaQlpTy5fNXnMrINM6XF9s4D2rK00eF9z7Rdx1VtucPaspmGacgImIYhD+XE5zsrrzRIomU/zgbiWJs0fWwyGrZaTLK6ylKQIc/tjPLz3OT8wqc27ojmY9x3duWVFffmG3DHN2px3td0UzKSNvFfeH/OYt2fxXdErrnjvg9F8SE05xGgFmLR/Hdy6x/uhAssLyTVLJitv+9rZJUdWOzaTPbST7/DNcIWMpPJvU6G2JvnALZetiK2+Y+O+FT2u7Uho3m0Prw9u3YMN9719SbAYtwrDrjX7ccV7H1zqAGzhaT48k/97Pv6RoLuDq1xcdeBb9O1ztA1h6tG1NAlYLohtbOH41Nxqp0ZhnDrtZ0gnXfP+E5c+61yB4VoqFdNS7IgSqYLLtBeXHVNi59/WWmVd223HcNcdVy5d5xuuSk05xChnITdfejq/7rffuFSZu9bsb2wGECg3K5mCnJkKvaOqw2TK/b5rys6uyskBl8CWz02ZUZX3+q6PzbD5cOWxidnE3HpMbR9N0dY7p24W8quPnLHUo0hbR50+MjG7Wa6PUSy8SKih5CxQzsVoVL53pPhOuQhNYMwfWx6F1CWM7J5bapw+YZc7wyZ98OTOSqH7q7hC5AgDO2pIFx7/Tba7VEYpwAD3ItuQIfPozi3JnsRm+BgSiPa7pGrtGnKUWUAb84KX+LSkw3tngzNLC7Mzy84bW5+MuhrCtb4zhitMzuG9s8vKzR4myvWsrpk/2wM+JV12Gu48sGnFs331lXMcyPvQ549N9u6U2G2qCcFhyi20MsA1pLfx3e+L5x9ilEPIKmshQ+SozGUdLKs6ZvqGT6mOrXU6hqa81/W+rp06Q2lIbQO++/uQtxh1ptHUr8uem7pB7tQNIU9c9F3OrcB8va4ZHsj1Wa7oAC4nRdmrymPXXPvFZe8xmoFMh+0wKp1S5XGpQZhn2L425pnSkC5ncqRDqVnDaKcbWNmo5hZPLb3LbD4qy8a8yzejJ99rlwXw3CyVrC8zC2avZ5X3y3eaCJ92lAmzXX1otlGWr0yXHSbIPMtuA642Y/Jgninr0p7xlm3BF3HEPNvUnXmmq13ItBmNyT5u7rfL1Z5Jl2Ul02rfJ99hT4xdc+0XMX9scVkUi0dvfv+SUAOWzziaYyaUVUxzVw0M9S/zmVtcHhfcENJGzMfR5JKjLqjD0JuiPZprmtBw+lgnvjbWNCENvY66dtXf1GlgklAcLdtHB1juHySlv+xhpA3FYAeqc9ngjHZkX2/uMWmxe0VX+uV5X9yrFFy+T7Y9xXb6NVqAb9s5l7E7x/Zj50eWmzwv7zUC7uDWPSvqz7c8x6Vt2PYyIBwF1YfrOfb75e8U25o87mpjOelzpTf0bvPc0LDPdMCxthdqC3Zgw1h+RinAztzIS+qoWdtnjO6xBrB97eyKgG7Gb8icl3G/jdrtG/pJ5o8tLrkOhBqCWYIi3ysdHe01l67ZqJAzp0mfSY983vrrFnHPxz+yrCGeuPTZJcF0cOuepa3S5DBA4vL7ci2LAZbPYpp3Sr+7hdmZFYLS5E3OLpvylv5dslMwz5dpkD5w5l0mbLhE1ql8p8skIX3IbP88mSYj4Ez5S+df6QxrIzc0cX3cIWFrp1cKbLsN2R1nylpes8zMHibb6fDlzXTeOzbuW6pzGazTxSiHkOe94hy++LEfDqq0oaGGS022VXZ7WBFTn+X1OX5FofMmD7FhU+zdEpN388y5RXd0gdThgnzejo37VjzL9Zz5Y4u4+PabvO90DZ9keg25flouG1eZYI/mOSkTIjJPKTO6sbTXMXlQ9rsBJuVjZh7t6Ba56TH3hoaQoxZgQyRVKLnuKUvOR2n/XZVU+5L9Tvt3ahmY69qwazXxDl/Zlw2fVIa6ZqpTy2fqbGBP/6/w8MmFa7bKdU3O88rc6xNeIbtCyocbWswdU9PtIY382/Vck1Y59MspO9ezbL+6sgLUHn7m2Fvs62P4NI/U5/kWurvsa0/NrQbg97HKSXcsPTnCy2dzDLUHn53YxSgF2CUv+OulqV2frcBmYTa+12HOR2Mbr1OWGpn/7QgGQDj6a4pwMGshXYJVLri1p8OBcCgW12YNJq3rdt+/VP6+/F98+03O+jHH5O5HtiFf3hfbTclgR1qw17PG2oq0VbnSG8OuR9/zXNfK47ad0Sx69wmXOmYr5bIlF658+EI7uaKByHNAmqAcpQDb/7cvxPrrJstRQnvZpa54B5YbcmM9g/y4UuNYGbuBNBKH1OuT2zZF41XJv2V8LzkzuzA7s8xbWk5WmGeEhK9ray/7vK3tSK3txKXPLgkOWS7yucb/7K47rlwm2OTeBBvue3sw/heApUkdFznGatc1KUMhl7Z6ctsmpz3Uxp4Jthey+4aVrplYG9dMqO/6k9s2eRfn2173crVGiu3VTkNKCKipsoFJQ28Xa8eaomp+Qo2rzORCGWKGY+A5bcW11ChV03W9L2YA96Vpx8Z9rZRNKA1VNCt7MqRseuvMZ64f2CgF2Or16/iHvt6fnVwUpWnqXgLWJ6bOiH/ud7mjZvocFV1B6WL3mt++WFC2Q5/L+RVwq+q+Ia1cMmQ/2+d86FseYtIo8+NKi9w6zeVgG8qHTJ/9ftsnySwHMkN0mS45weIqV5l++9k+B03p52TbEl12Qpdfn8/YbOrafo7LDOGqNzvN8t22bdecd7mcmDSYNiqvl8unXMZ1c8xnLrHL0s6vLN/YUNDXlm07p4tRamDnveIc/vd3/HPnMCPVVyuHFBW67KJl41fjuy523nYMdbkQ5Pjk5CDT5isj89yUIWRoyCaHmSGk86hJmyv/uRpNrHzke+tyrSgzhLSHy/bQu4yZxecrmEooH/PHFjFz3qHpG0Ke/19u6tWqf98K/CbW7sWo02eoDXuinV5f+tu2bea+L/X6NttElXe1Vd5TN4QE/D1xTGVPJWcGE5jMtr144cSK410I2TRQj9IAACAASURBVDodHmUQwhTsIXBsDR6wMr2+9Md2zJHYs7Upfkk2sZ3BbWL+czk7cJdd82gPhWPtL5Qm30RKKmXzIBmlALvkBX/tPO5bY1ZGiIRidtt2G2Dip1Rm37uU61MaToqjbhVirgAm/cZfqw7BbZeJawMJH8ZvCpjU5a41+7PrpMpQ8PF3nFrRTkybStmQJeXdPkfYnLKX5SRxtSVXwNBQmcaG3L73SEY5hOwqnE4by1Pqpuk0dzFE7pIhtoEcunBBmsohpAvfsM+1rMSFCSUjkUMB+zn2e+SMXiq+2cvQc8xspctB0f4d2ozXJtQb2pvJmv+N8PLtw2n+NjNZ8pysL9fyJFfQSt9srA9Xnfqus3G9K6ZV2LOornf76sHWSkL1ZWYf7aF6TKORM6UupLE/lF5XPeWaanxtX6IamIXdw9gaROy875gvwkLonrrwPTslwGJdzo7AczN7R3duwaM3v985+yhnE2UafNfIxe9meJqrAcVmHMvWTeg++52mLozg8O2kJfO7+uEzgun2RT2xny2vK+NPJtNkz2DW0a5VA8vA/jjtwjeG29C0vR2UDfDvV2iv+auCHcNJpkdizq+6d59XO5Extera/cbk3ywfstMztziJS2UEkXmXbSw/uHXPsjWSwMpF2jGkhmHHcrMxG4bUid0WTEdiNlFx5UOukwXiu4S7dozatWb/0nGj4ZgNQuYWTy3bSDiELCtT9iZtso00bT5QDawhps320yRN2F2GWj9Sa66ah6GUgWpgFnWEFomx/rpFr92nDlJnFOuYqq4baTdMjRSSQk4Zp0av6BtyyF9V+MQCE9qkBDFom6kSYEZltqepc/a5SxUc9vKO2AeTa9xP8fwH4kOqXCFbpZG68ijTZy9ZisWPstNycOue4ISF672xZT2htOfWWRVsQ3gVlxh7uZDN4b0rI7jMH1uM2sZCoYHKkJLHqRJgJpa9/fHbtoSQo1/ZoYxrg0+Jz9/GRUqYEVdjczUIeV3KciKfj1LoY7DX65ndg+w0mTKwQ/oA7nL32R9j18j3ytj+rhk289tVnjEblP0M1/tdx0L15LIz+fAJAOOvJfcEkPj2igDiM59yMgBI07BDdtipXAvZtQ3s6M4tSxtfAN34ztRBjo2kDv+n1HKS17UZSjmUjjpxzQSWyWdK+poqvzrta2oDa4BQuGTj2W3wRaUsQ5Pe9DY5DTDmie9Ltzzu04Bs5JKhx99Rrjxc6csdHueug0zFpdmFhIxPS7GXVrkiZDQl/FNWE9TB1AiwUHiQ0JAsxc7hashyCruKrSRlS6qUIWUZfB+0jPTpwxXD3iX07fwc3bllqfHLHcHlu33DXhkuJpR+V5TZ0BKbOmL75wi72JZ4Jk3GUdWn+dpmCZf5xMblZOsqR7tO7LTVFeQyhg4hW2Ao09VNomXQHFWGsn2vl8N7Z3H42tt0CJlK6sLoGLENOYZOSKt0lY9vSOG6ts3ZPUkfXU5SqKLtNN02Q3WZUt6x9KkGpihKL0nZ2FY1sIZo09g+BLpwclT6T8h+mzKrPRUCLOfjqTqE9Bmny74/pma7IjC4ZtRSy6AOQSMjGpzctqmU/1gKOev16n63oenhbt8Ef0p6XGXiK+uqs6BTIcBypnSrujzU6RckHQN9bF8765zxS3XotAltoOvC1TDlHpSr7t2H7WvdgSTltanvS3m/+chSyq4qIe/0OrTwtuynqYI49C2ZcpdlYkII+crateoih1ELMKkNhfySpBrriuFljucUctkKkR+69EQOuTTIafeqLhWuj84WPnK3GF/IFzs9PtcGVzm5QjX7Qk+nRPVMeY5MS0jwpGggRiuW7c71oYZ2eZJpbMMcEVsmZNLnirpqcMV+A1Z6/Lvi0W1fG4/95aIzIz4RzQB4CMBRZn4TEV0E4BMAzgXwZwDeysz/QESrAfwWgE0A/hrADmb+eujZVY34oWlp28veR9cbM3Q5Pd73qXlDHasHxrwfY5fINtRXI/7NAB4Tv3cDeB8zbwDwDIAbiuM3AHiGmV8K4H3FdVm4PJB9hISXWc+XspmDvXg7VSOT6ZS9ry/9JrCd/O0iVTNLWTQdw7VYPhZl1L4+NQ2hNYUuZARW14Jl3/t96XAJr1B782kivmtD15SJGNEkrnoOLT4PfRMpzrxARxoYEZ0P4GMAfgnALQD+JYBvAvheZj5JRFcA+Dlm3k5E88XfDxDRKgDfAPAiDiRc3SiUphiKdlk3Xea7jxrYrwL4aQD/WPx+IYBvMfPJ4vcRAOuKv9cBeAIAivPfLq5fBhHdSEQPEdFD38HK7ctCpPZabfRudg+W8k55jU/TanM2y+5tQ+/O0bbaxKdBpXzETS3tSqGpsvPZt2xy6r4OWhdgRPQmAMeZWY4zyHEpJ5x77gDzB5l5MzNvPg2rVzSik9s2ede0mWGAGbKZuGH2xgVmbZkZIsRCi0hSN0GwVWdfyBaTHzOsNe+T0TolJixyGfcKu9HG8i6H4Ed3blm2A7T9LNdWaDs27ltxvRyWuYYl8np7COda3yevldeF3GDk5IVBbpxhuOfjH0ke8prrcj50e2MQYOUmKq73hobW8nsJpSUWQmjXmv3L1soac4Kd5lB6TDmnlEnrQ0gi+q8A3grgJIDTAZwF4PcBbEeNQ8h3Prx12SxeVWNtFYNvn8Pp2GmbP7aI245fskIISVtPTlmErpUzdSnlE3qWHOLU8bwcXG2s6zqvkjf7XvM7JU91fW+SXg0hmfk9zHw+M18I4M0A7mXmHwPweQA/Wlx2PYBPF3/fXfxGcf7ekPAymILevta/Wj+H3GfYYWJSJxHKDFOrDCHtBuny2bIN1TllYabHXe83ESpcbhq+Z9mYcpWaxwO3XBZN19GdW4L5yKkHVxvzfehtDS9z26vLtcH+nSKQ7XvNc23NtS5zTJ/8wHYCuIWIDmFi4/pIcfwjAF5YHL8FwK25D87xyaprMbcrXEsKZabkZdgUEx/LTmOXhucdG/d53++KRJuaVp9G8NTc6uA9AJYNu12k1kOOGQGIe553Zf+rs31IIWaee/HtNy27pi7Xk04FGDPfx8xvKv4+zMyXMfNLmfnfMvOJ4vj/Ln6/tDh/OPX5pgHFeqMmDI91RTZIWQ7j+ojrWspUxzPuPLDSTiTPlU2DL48hO03dUXJDKxfKPD8mSHLrpy4n2KrfhMvWCaQFugyh0SiU3tFlmOiqDDntQD8dc3tlA2ublFDBXcSB8i1pSfnbJjc+Wew6exZP/m0/J/Zu2/6VklaXABhKdA+TdrucUpxkbVwRaUPEnJ5TiAmvOmxXrraV264MoxVgZpp2YXam9vjcdQwx5RZU9vpH+bd9TayiTdrshubapdnFbccvwcGte7xpsofjriGgXPdndoJyPcuVTpkHSc5wLFQ/oXWvdSEX4dsLy31DKVeaVt07cSeRAj3UEcnd1O3nxTqwUJnJTii2e9bRnVui0Shk+uQOSfKalNUuwIgF2MLsjLNCXfiWlPjw7fEo70tZtJwyU+SbETIszM4sEyI+G0poEW5OukKTA/YUeort0dXjp3Y4vo8uZEe64r0PLv2dUyYGl0+cnSbZ3nKN47HJn9zZRTkbn4LLf052QvZGITbrdt/vtEH63i/LR14T24bQoDYwRVF6zVTbwIDyAQ1997mOxwIR5nhm+2xOrt2AfDa+ssPcnIXvNvPHnhsWm/BDMU/zUDyvJshZTJ37PNdvG6nBhTR237GmiZVPygL0lON26KKyeR21AHM5OcaQTqe++1wr5WOqvm/YGXOBkLarVffuW3G97bQp1X7X8Rg7Nq58R+pkArDc6dHlD2QTs82FohmkpMfG2FbmFk+tSFOZyZxcVwlpA5PDpJRdxuvEV26xIaKvHRvs7fPkcYntHLv+usVSTr6jFmByo9Schu6rRPOMnC3lDak2iJj9xxZE0qYDPCdc7UZk1kLGiH2Qtt3I5cEtBcFLPjyzbM2hnS6XsVaWgfmIfely3R+aKQvZVq56yw3ec4ZYO1qYnQlOFMh2IIWBS5MrE8zQlLNMg0sw+8rTt+emIVZGdnpjs5byvGzL6gdW2MDMTOT8sUVsuO/tzkppwneniqOkvFeu8aszpIl8h8y/K90u36DUtJQph1iAxtC7XWs5U9MXaweu8678hfJsp92UbW45Hd25BY/e/P5a1xz60ph6zuBahymP+Z5hysAu55ANbLQC7Pxb/3PvHPKGQlUvddci7aYXN/fRAbNNul48nkLZBeZTacS/5tovlnbsi12fayspa/hMpU5/prnFU1EfnJhNLNV9xXd/6nWyHvoovHLrpYpDdZ+Fl+1eU2d7Ha0AA/I+JNd90glQFrqrFwk1vpjK3afGt2vNfmxfO5scHyo17b7rqtxfpjdv06M/JW+hKBBDxFW+qZE6yjBaAXbngZUuB7lIJ8BYoZvV9j5jddkdjXzTzDJvctKhap5NY5M2CFsAL8zOZE9758wi+gJP2pSZepcrGnzLfULUrc3XvXi7D7gmMXKXk6UyWgFmZnhCMzmuBhtr1L4ZzXW7719aOmN+y/tzelfpWiCnmQF3pAO5vEMuP5LkNJqQD5vx7wp9ePJ6Iyh86XJ1DHKWV8562nmITelLbL8jV32krMioqs3LPIRm6MoMJ10dX92kpEvOJtrtF1hep6Ht5FIYrRH/nQ9vXRoO5RgPu9i8IBRNNCXtKWnOMfLa75T32gI5JX3ymiozmjZ1zXD6yDU61xXh1aZK3fWdWHrnjy1i5rxD02fEB9IXMEty/bCqYn9QZda+pawd9K37s4dSvh5RbkUm07R97WxQk7DLq0oAQ5syaxmN1lZXBJKYbTTl3pC2MX9sMepcKim7QWxXxL7RWJmOWoAZfM6LoRk0G9Pgy35sdqMydoI6tL0yH7LhtuOXLGskrga1MDsTdN4NzQCufviMUh1JCmWijMiF5qmrCkKE/L1SCWlX29fORiNA2Axp27eqBv3RCrBda/Y7x98SV+H5GnVVtdxOQ52Osykfck4Z5NwfY93u+5cJ2CrC1iY15IpECpZY3qvYkep0OK6zzMbGaAWY7PWHpFJ3jUso5ARTdE1yHNy6p9S61BhlOpUcra2MgKybhdmZQWlUbTNaAQY893HlNNq6fbJybS1NzR6lCvE6dnGyZ5maGkKWsWPl2JPKkOO0nNrWxtoB12GHHK0Ak0PIWEOJbTorSQ2pW3Z/vFAkCBeu9Lo2Bc3Z6SeWptA5GfzORCbdtWb/klBL8Y/LEbYuzDtibgqx95QR5OuvCz+36kYfqdFmU67PIfU5vi30fA6uqWGmfIzajQLIbzApU9ZduFqEqDPNh/fO4uDWPYOYii/rMuArr9yF4GUosz502td5TuVaSIO9DCiG9DL33RcakuYGuEu9P/SclI8gdRgduy4WuNF1LrR2su49O1PwlVdV4ZWSN3uvAV/wSkkTwqvM5hwpdeXSbF31Vlddjl6A5fbQ0ss8x3vc3GuTawj2xUQPfVgpDSv1wzRDPd8zcz5wO2idaw9LezOHpknpCFyCN4UyPnyuDVOa2sVaUkYopu7h4Fp6ZlOXljtqAWZ6uBzDbRWDs2vNZO7zfB+Nb4t2IG0DhNSPccfGfSsM+aEPKGUDkNDyndgzQjQ1OWDSU2UWMsVm5BPc9iqIMsKmKcN/qC1U0ao0pLQDY2PIaQD2llSyMVVZPFwVM7RraqmK4c4Dm1Z8WLL8zFrIFMxwvCk/pqbcHMxHmrozjqs8Um2ksb0CypZdFRttqJ2HnGpz2rpdZmXTO1oj/ovuePfS3otlI3T2iZwon1Wpcz1d2dnYLmlrkqavba0qdZffVBrx69BYUgysvuvL4tu41jb4SlIM9D5Dqr0WMmUXczvahI08by9VqtPelRJNxCaljnKXKPnKoEpezb1VdpfqitRQ4ynEynD0GlgTpPScQ4sK0BV19da52kzfXGEUP1MZE/+dD28dpXreB8Y69FH6yVQOIWMfWBNT003Sp+UkZYRXm+nvum77skltnbi2a+sDoxVgMao6B9YdWjhGE8OdFH+npkMax4RNzK3EdZ3ZpiyXuvKas0ltXe9sWrDU3f7q6mRGO4Q0+0JOK9O+/ETpF749N1NskVM5hGySlN6uzIYRLlzhamxcvVnfhFebw7ohztx1TWr9lC0jX/y7qpqdamANMWZDdxM7mSuKD9XAIti9T9leJhTrqWwPl2sjqapFxLY5A4DH3xHe5clOgy/Eiuta+1iZ/KRsxVbGhyzlmpPbVq5kkMiydG0/1hWuEER1ln1T+VMNrCXU72g6iGmnffcPrNpOm2jnqoF1gK21tCW8qnq6d60JAMO2RT01tzqY/r4KL9Ney2yUIll/3WKrrhYqwBqiTSO6FJZVP5A6whznRAZ1HV/98BlJaegj63bfP0hN27TXOuy2UgNtevJGBVgGfdUMXMKy6bSGPlLXEMonlFzPqSr8+1pPQL/T1gSh7fjqQAVYBkPqWevav7IOTm7b1KpG2ud66nPa6qLsELJM2xu1ADP2nNywxTIQX2gGzUXqu3wzULkhqWV6Q/dUsW3Z6ZNlEooPZo5LjazsCoajO7es+DBy6tU1QxrbYCS13kP1V5WUGdXYsTbSIpH1bUJa2bhm28sI905mIYnobAAfBnAxAAbw4wC+AuBOABcC+DqAf8fMzxARAbgdwBsBPAvg7cz8Z6HnNz0L2aaX+1j8yfo2C1u2XKvWfd/KIZUu22EfZyFvB/CHzPwDAC4F8BiAWwHcw8wbANxT/AaANwDYUPy7EcAH6k5MrspbpgGXVavHILyAdodOKYbjsuUaq/tYPQ9JeMly7Gs7bF0DI6KzADwMYD2LlxPRVwBsZeYnieg8APcx88uJ6DeKv++wr/O9I1cDG2qvqCjTQN80sPUAvgngN4noz4now0R0JoAXG6FU/L+muH4dgCfE/UeKY/UlKFF4dR2mZShM20yb0h1dCLBVAF4N4APM/CoAf4/nhosuyHFshdpIRDcS0UNE9NB3cAJAubC8IUNlmaFjG2F3Uu5JXeLhmhQI4Spbu0Noyjm2z4IyJW19cBruK6l124UAOwLgCDN/qfj9KUwE2lPF0BHF/8fF9ReI+88HcMx+KDN/kJk3M/Pm07B62bkuh4cpoadlZZXZaSflHl86XPtQxp4nZ5UObt0TvLZvkxA5QqOMgDF1mdLmUuptbvHUVGj+dlmnfrOtCzBm/gaAJ4jo5cWhqwAcAHA3gOuLY9cD+HTx990A3kYTLgfw7ZD9S2L2DczdmbtN7I1Ay3jSu/ZHLNvoU8pKpjG2N6O9OXDXWkdO/ZZpCzmdZeq2bX0LjdQEZb+7rmYh3wXgd4joEQCzAP4LgF8G8FoiOgjgtcVvAPgsgMMADgH4EICbUl9iCiVHq+n6A6uLthp9atQNubtSXZTRrJvWwHJISb9vd3hlwqijUZRZ+V/3kKeNIdS0zqI2ne8+DH/7kIYukPnu2yxka5QZjtXdWJpufEd3bnF+xH02cNdF00K7D4KjShq62oCjDptdar5HLcBScQXgC5EytGhL7V+3+/7ahVUTQRF9ZeYqp6ofQNfCu+6hZ91hnOtMg2sjZmm+iJVF1e9k1ALMFF6sAciePDQsybHjlA1rc3jvbPYHYMdwmls8tSL6Q6pQyF2T5ipbWT5Hd27B0Z1bvGXmKif5AZjn53zEXYfjCbWPnHWTRoNq2zyQ4hpjkPV3eO8s7jywXOuLfSu+7yT1Gxi1DcyQYwubWzyFXWv29zbwnI1tI6lqF6oj/y67jW8NYRUbT5V724qMWnf9lH1vU/lNfa7rutR7p3Jn7rKLuZsymg7F0B5LZ6h86vpIhmy4jpWfbneXz9Qa8VORxs6UD6eMTcJu1DF3g7pJtTXs2LgvONw0LimuMnD1sH0jtJFGbigjF6aefW2kjPBqw5E1Ja+poaXsercnE3Ii9sZQDSxAjiZQl4bVlPaRox3Z18q8ldWyypSPfc8QNLMhpHFoqAaGcr2p3RBDayvrGh6WafwpvVeODdAmd6VA7BmpxO5JCfYYoo7ZZhvbiF3l/cCkzY3FuTqHPq+FbA1ZCCHBYKu8PpXdfLxlPsayw6mU4YNMT5n32PtZ2kIq9AG5zpmynj+2uEzouxpljstFinCX74xdX8dss++ZKUJn/XVx4bR97exUanSp39ioh5B98MRvgzrTfHjvLA5u3VObMd7YzIYyq1uVJtqP3GuyzX0lXcP+pr6P0HOndghpKrpuY3LoeVWcKO3IFPJ5VYcROV7ZoQXaJia+0QxDWtVdd1yJ7Wtngx9cat248h+7V6bNXGv+dz3PXJ9TVnb+bdeFlHtjce+lQ6osy5RY81VwaUEx4VUltJOrvmKMWgOr2lt04fqQ8s4yvXAVf50q1x/eO4sdG/e13msPgSGmv6nJqtBv9QNTlJYYolDqO1M7hCxDqj9Mn6gzPaFt0mL4hjRN+YP1oR5yh/a5ae5DHiV98+1TAVYC31q7ripXroVseso99HyX7ezg1j3JgfsMqR9t08P7lLLM1bZk20mdqWwDWeah9aexAJZ1kdqOVYBZLMzOLFWc7cIQ+7CamB3K/YjuuuNKACvTnmOYDuUj9MGac/LddsTZOojVQ+i8SVtZ/6rYu+3ysd9x4tJnndd2rWnJOjJ/y2Om3FL83OrIS2rHMDU2sJPbNuHxd5Tb/VdS1ijtM34OZY1kDDnV70LahuYWT+HOA5uSo4DEnm0YyzpDV1mEyqCJhfJ9Qm1gBXUIivXXLZZqFL5hZ06aUtfESW0rdVhbdb1dTMDIntsWXoC7tzc9+VNzyzdp8XHi0meT1t3FaFIbSklPSJBLTDp9QrupVR0uugqeODUamKLUTY7G11dtqA/piqVhajWw1N6ka/uDMkxyhqtdCwkffUhXlTSMWoDZkUp9dB3BU+kH07hoeuiMWoAZu0tMw5IzQyH6qqk1HS/q6M4ty5a9yA+9r2XiIySk+qCN9IEh1emobWB9GN8rilKNqbWB5TKknieXKrNEueF0cq7LCafjok7n4S4cke02V9cwtiuPf6mtS2TZ1hWgABi5BmZQTaw5tGyVppl6DSz3A1Nj7nKGYDcas/Y8BpryE5sKAZaL76PsUrB1+YFWFVJmb8hUqm6aknJ/k+UZi++VSpk0di3Ifcvv6txkVzIVQ0hF6QtjWToWou4lXVM/hMzplerSsso8xxWRtS5SNaA6tJccw7zLcJ6zi3ifqWtzEx9da1su5hZPJQuvOiZNVAPrEW3GO6+LadAolAldLZZXDSzSU+UaGEPP82kFKVpFbtysLrDznrrawZf/Klub9V0DGxtNCq+y2uRUCLCYhnDFex/Mfp5v8w2fwTtW+WU3frXxqeW5m2f49sC005hq4PflP5bnlPhjwPJ05kSV7XoYpkJ4QlktXoeQFkMcxvXZF6vPaVPy6aI+p34ImcNQhJfsuXN2gy5DFS0lp7GnbLdV5v6m7quDKmVbh/aYm/e+dUZToYGpFlCNUPnVobGmPkPrMZ8xRKmdeg0s1OilcTnUo+X2dnbPVjZiRCg+v2sjBl9aqnhCh8rPJXjstBzduSXY09vP8OXFTkfd3t0pz6v6ziqRQ8poikZ42e3Fpm5boJ1W1x4Evm8ktKmIzVRoYIZYbPUme6uyz55GrUPWU5v5j02klHUZGZJdtSm3mCploBvbBmjrAxlq+GH181KaIrWdT/0QMoSrAGNqfplhRNnwwynDBld66xoSNC28XOns2rVhGqmyVV3oOtM2Xe24jk566jWwGKkaiN2b5GhRqduGdYFpkKYM+qQdKtOBDiEVZYoZkg3OhQ4hUc+MlVSDm4reORTP7NzwODn5qhp6p+xMX0qdpuQjdE1OO6yrLbQpvHJ3s69KtgZGROcAuICZH2kmSdVRDUxRxkNlDYyI7iOis4joXAAPA/hNInpv2QQR0U8R0ZeJ6FEiuoOITieii4joS0R0kIjuJKLnFdeuLn4fKs5fWPa9ddH0LkAuhm7Y7iLevDJ+UoeQ383MfwPgXwP4TWbeBOCHy7yQiNYB+EkAm5n5YgAzAN4MYDeA9zHzBgDPALihuOUGAM8w80sBvK+4Lps6hpBGcKVuw1YnQ3dl8A1juugMlPZoun5TBdgqIjoPwL8D8Jka3rsKwHcR0SoAZwB4EsA2AJ8qzn8MwDXF31cXv1Gcv4qIKPuFNczyGVeIoQuTsZPSWTX5YTUV/11ZSaoA+wUA8wD+kpkXiGg9gINlXsjMRwH8CoDHMRFc3wawD8C3mPlkcdkRAOuKv9cBeKK492Rx/QvLvDuFoRjRx0ITKx9SOqsm1wf21SWmC5peh5kkwJj5d5n5lcz8H4rfh5n535R5YTEJcDWAiwCsBXAmgDe4XmtuCZyTz72RiB4iooe+gxNlkgagf6vtx87QbXtKt6Qa8V9GRPcQ0aPF71cS0W0l3/nDAL7GzN9k5u8A+D0AWwCcXQwpAeB8AMeKv48AuKB47yoA3w3gafuhzPxBZt7MzJtPw+qSSVPaJjWiq6K4SB1CfgjAewB8BwAKF4o3l3zn4wAuJ6IzClvWVQAOAPg8gB8trrkewKeLv+8ufqM4fy+36H07bUNK34YcOZtyxJ4n6bPGO2113ydSyz7JD4yIFph5joj+nJlfVRxbZOZS+j8R/TyAHQBOAvhzAO/AxNb1CQDnFsfewswniOh0AL8N4FWYaF5vZubDoeerH5iijIc6PPH/ioi+H4XtiYh+FBMDfCmY+WeZ+QeY+WJmfisznyjsapcx80uZ+d8y84ni2v9d/H5pcT4ovBQ3Q7I1lZkhzF1srO4b7VC2nFNnclM1sPUAPoiJreoZAF/DREP6eqnUNYxqYOXRxdpK36isgRXa0Q8DeBGAH2DmH+yr8MpF9txN9spln21sAfb9TfkaNRVfP0cDrFoPTWmbdTy3rrSVKaNQm0lNV+4Gyeb/ptprUAMjorcw88eJAiFpoQAAHfxJREFU6BbXeWYuvZyoSVQDU5TxUEUDO7P4/wWef4pSiZy9LeukrhlG9bqvjzJlmWoDexEzf7NMorpANTBlaGjobj91zELeT0R/REQ3FJ70o2WIvj+uNJe1taTel2tP8ZVrV+XdNw1MhVc5Uo34GwDcBuAVAPYR0WeI6C2Npqwm5HZSoQ/LBN0LzcDJRt+GoTj0PntjW/u8/UGEPjQ5ZHN9SK40hdb72c/IFRa+7eJiW8f5niGZP7aIw3tna5tpja17jOW9DfcWs6VZXZMjKfVpf3P2PSkdYErnUCag4fcAeC+AH2PmXs63NzWE7DI0b+oQo4n4+rlbwuUMh5oYOpXdwi6lfpsa6vV9CFnWvaaOfFWOiU9EZwH4V5gsH/p+AL8P4JPM3MuFbF3ZwPoQe7xqGqrcH7tXfcyUMtQhwL4G4C5MhNYDNaevdtSIryjjoQ4j/npm/ikAjxLR8+tLmqIoQ6JvS9JSBdgriOjPATwK4AAR7SOiixtMl9IjQn5ZsQbdByO2C10LWY6QPauLGeVUAfZBALcw8/cx80sAvLs4piQyRPcMQ8iuFTPQNm3zKiuImo4UOo3k1nUdDsupAuxMZv68+cHM9+E5L/3B0UWv3+SHXHUfRaC7XYNSZqhC+TOCaP7Y4iC1qiptsWo77rpTrWPCK1WAHSai/0xEFxb/bsMkIsUgaWO6uorvkw/fB5qjTeTmvS6hUGXR/DXXfjH6bPMx9GFpj8lf3+xFOdSV9qaFZKoA+3FMIlH8HiYuFC8C8O+bSlTTtNGwcjWuFMFS97BHfuyu3nBu8VRt75T5y33mA7dcFn220SD7sKGGyV9KuOz11y2uaI9tCj7fu3I6upD23rQJIduRdQh07UbRB3+wMdGEc64yHEJuFKtcBw1EdHfoPDP/SJWEjZXta2d771k9JFR4raTrTrLr9xtiQ8grMNkh6IuY7OX4P6x/U0mZ4G9VAxrmYtLYlXHeRq5v85VFVXuJq176YBNrgq6Fx4b73r7st732sa5hcOw5MQH2vQD+LwAXA7gdwGsB/BUz/wkz/0ktKRwgtmbl+iDXX7e4wtZT5gMta0MwaZQNvayACAnB0I5F8t71100WFK+6d19pu1pMGLs03jq0tyZmN115qbuzadKWZtvuTNnb/6cQKt/Yc4ICjJlPMfMfMvP1AC4HcAjAfUT0ruTUTQEpH+S63fd3vg7QDhed+sHEevtQvm47fknSdSnnu9I6YrOgZXDlxZRVXVqjbxKhbyG3q0wURY34RLQawL8AcC2ACzHZp/GjzHy09FsbpmsjvqLEKBsxo0+0tTi/ihH/Y5gMH/8AwM8z86MNpE9Rpo6hCy+gH5sSx2xgbwXwMgA3YxKV9W+Kf39LRH/TfPIUZfoYsgNs28RsYP+EmV9Q/DtL/HsBM5/VViKV8dLVLGnXy2hCrH74jK6TMBhSPfEHS0pDtT+isqGjq3yMdc105caed6XZzrN9TU4+Y+XvMmanPL+q93rK8MdOe5X69d3rSnedw8vUNKcK9JR22sRst4/Re+KbQunab6ZJUoypoWvqdEp0vacpp0eN8NoMISfsMnUZep6pw1BdVo7IOjSGOAtZxXNfP2RlzNQRkXUUSPXXpbrX6bBoPz/2bCm8Uoaw8nlSeLURUqaMn5IvH67jTRqxqz7bVb51hDNqiq7eZ/5vuj1OlQCTtgUjMGQB12l7sJ0IX7xwYunvWKVKgeQz6K7bff+K58wfW+n9D9TfiFO921Maryt/sa3dqlB1fapdvie3bcpydG17fWwT70vxnDf/m/KaWzwV3WqtDDqEnEK6dqKUQ96UoXMTQ+QxLLavIw9DKAcdQg6IMppG7j0h4VU1xr3N0Z1bVjxTCqOUmFnGyFsH5jl9+GirapUpZRejD+VQBRVgyB+npzS8sh9cmQZVZyOsI8a9nA6PaXqu57nqw/de17WhsjfP6UOUjqr1lqOVNhHpw4fLRtqU391UCLBY4ecOp1IaXtkhT5mKrmoorTv8tT3NnvuhptSHSYO0sRhCZW+uc7kCDDGmviFWh1WH4LIOY+XkspE2NUs+FTawIYzzx4r5sNTNQynL1NvADm7d03USBo9vyJXS86vwUmKUtQdOhQCzo0f2lSbtMqFnp7zXDLkO751NHq7Z16YyhiiqfV5r2UfsEVJq+U2FAKtjtqYNyi63Sem9Qs/Oidq6/rrFZI3KNMpcwdxkDPy2BEtdWue0CMKy+ZwKAWZHIjX4Cq2MOtu21mCEwtziqRW9VxVNru7h3sLszLKorBJZzinlV8fHPJThrKnDoaTXkBM0QB6382l+x+pcjfiKovSaqTfi90F41b2cp+sp/9z85GhPXeetKcYwHEytm1j7qGvEMhUa2DQxtk1gu172pHTP1Gtg04RPePU1THHdTsZDo639E8dKYwKMiD5KRMeJ6FFx7Fwi+hwRHSz+P6c4TkT0a0R0iIgeIaJXi3uuL64/SETXl01PXUbissObNo38rnA8fRhGu8pApsu36W2s7Ex+q0ZpjSHTUaYduNpXSr348u/al7ErygwtY/WTUn+NDSGJ6IcA/B2A32Lmi4tj/w3A08z8y0R0K4BzmHknEb0RwLsAvBHAawDczsyvIaJzATwEYDMABrAPwCZmfib07mkeQipKn6gjkkgnQ0hm/gKAp63DVwP4WPH3xwBcI47/Fk/4UwBnE9F5ALYD+BwzP10Irc8BeH2Z9NShAU2rml4HTZSd1kf/adoNpG0b2IuZ+UkAKP5fUxxfB+AJcd2R4pjveDaPv2P4M0DKcnzDprHOYg6RpjuZ4Ma2LUKOYxw4vvIBRDcCuBEATkdalM9curYzDJk2y27shv8h0XS9t62BPVUMDVH8f7w4fgTABeK68wEcCxxfATN/kJk3M/Pm07Da+fIme4Oq6xiNgTfHV6iMX1EdZdB0GpXxU9e32LYAuxuAmUm8HsCnxfG3FbORlwP4djHEnAfwOiI6p5ixfF1xrBRNromMrWNM/ZB9y55cFV6nfSG0n6Q9JMt5ry8/sff2gbJB/PoQLNGmrk4nt77M9bYNOkUzSynHJt0o7gDwAICXE9ERIroBwC8DeC0RHQTw2uI3AHwWwGEAhwB8CMBNAMDMTwP4RQALxb9fKI6VourHJM/HCtd+lvzoXfea876KrbLeMWW63SeUFmZnKg3J7AXdtlDYtWb/intSNxaW54yQnVs8lfSRpWi8vp2iXEj/u9uOX7JC6Nep/Yfq3veehdmZZNvgA7dcVuoc4C8nWT4mjbE2nBJFZuo88XOndcuuoxzqXo2udNe1lnRoa1LLbOLadL133a6a2qQ4hG5s2wOqLvEZ2sevtEtfl5DVsRRMBZgDFQiKMgx0LaQDFV7KNNCFT1yby+amVoCNiSF4pPvS2Ke0y4+9yuxonTOrVZ/VtU+cSX/KpFOZtjDVAqzJj6cp94Dc2c8QR3ducV4fesfc4inMH1tcutd1f04aXJpwblRPHzF3APt511z7xaW/5c7hue+o08huPytnJrwrpC3OpN9n+Jf5KTMqmlobmNIuanNUyqI2sA6we+aqoXracvhs6j1lhVcZLTnFd6xsPvs05M2hzw7DVVABZlFHjCLAv0lBiJC9wnf/4b2zzjSlptPlcCuHJnUMU6p8PCHB53tu6J7VD0/Wydr5rCM9KdS9C3oqQ/RJBOLtT4eQLdFXP52u6dvQsqn0aP2vJLWsdQjZAXbPMe2N16d59G3X9KaE6bTWf8gsUkdZqwamKBG6WD6jPIdqYBE0AF69SG1rDMZjFV79RQUYgBOXPtt1EnpNrhCSBmNXtIm20qHUT12+Z3XVpQowPDczFSNlRqivH1nq8o66445tXzuLozu3eMvO9UH4NOI6ZtLGrm3b+as76GVd2mhds6JqA0M7IUr6NtvWF5qYnes65IxSL2oDixALdFgHKrzcNDE7tzA7M1iHUyUPFWBQ4VKFvgqKaavTvpoumkYFmAeXzcj+WEN2pa5sLW2GMgGqC4qx26TaYlqHzCrABFJAuYY29scaGv50FcZkaA6TdZVTqgbShqYyrdqQi6Y1dBVggi6HHX1u9FXS1lTIF/vDsDUQ33vb0FSmVRtyMbZ9IUdBE8Me0+irTnv7PtyqAtJ3f6yHrdMJ1BU7ypeulPhTKe9pg5wIGU2lra7npjxn/thiqWgtLtSNQlGUXqNuFIqijBIVYEo2fXWdULqn7VllFWAl6LPB3VBFyMTunTYfqxBDaAtt4ptVbmoyRwVYCfo2y+QSOFWETBMCqmmtrSutsG9toUn66LOnAsxiiD1q3QKniTJoQiiW2dFGh7/lqeKz11RIIhVgFtPUo/qoswzKCIzUnr5MOnX4Oy5UgJWkyuYMQ9cCqu77KHEJK9nTD72s2mJoI4e66lUFWElye3/5ITepBfjWQqZqNbEP4ejOLViYnQleJxtn1bWZbWpMZQzNfbELhdpjm8ItVTDF6jX1OSrAHLS9IDpEbk/11Nxq5/FU+0VMMJvotaHrZOM0azN9+ehTNNzbjl+SfU9Xa15zaNMsYgumssIzteNST/wBM4TAfW2lUQNGtkMXbU498UvSlO9KGVw9WRMNqY7hhnxGbLhZ1/BGhVc79K3DVAEWoMrUb93G57YaTh3vydmV3HWuT0N4pRoaTqdjyhppVSMoz9Bimil+NJxOx/TJyNxXQkPtoU3vK8NCBVgE1aTihIbafbOZGPri/qBUQwWYMlUYbXEI7g9KHBVgAyGkMTRt9B7TMLCpNXlKN6gAGwghjaFpo3cfhoG6pEhxoQIsA/2IylO17NQWqbhQAaYoymBpTIAR0UeJ6DgRPSqO/Xci+gsieoSIfp+Izhbn3kNEh4joK0S0XRx/fXHsEBHd2lR6h8wQbFSqQSlN0KQGtgfA661jnwNwMTO/EsBXAbwHAIhoI4A3A3hFcc/7iWiGiGYA/DqANwDYCODa4tpO2LGxnw6WC7MzOrxVppLGBBgzfwHA09axP2Lmk8XPPwVwfvH31QA+wcwnmPlrAA4BuKz4d4iZDzPzPwD4RHFtJ/TBmO1DNRylafqo6XdpA/txAH9Q/L0OwBPi3JHimO/41GE3nhyNS7pgxBphlUCNoWuqaoh1p3sstKl5l+nAUzZdrkKj4XSI6EIAn2Hmi63jPwNgM4B/zcxMRL8O4AFm/nhx/iMAPouJgN3OzO8ojr8VwGXM/C7Hu24EcCMAnI4zNv0gvbGxfCmK0h69CqdDRNcDeBOAH+PnpOcRABeIy84HcCxwfAXM/EFm3szMm0+DO6jfWNHoDeNFbZthWhVgRPR6ADsB/Agzy1XSdwN4MxGtJqKLAGwA8CCABQAbiOgiInoeJob+u9tM8xDQ6A3jRW2bYZp0o7gDwAMAXk5ER4joBgD/N4AXAPgcES0S0f8DAMz8ZQCfBHAAwB8C+AlmPlUY/N8JYB7AYwA+WVw7anShsaKk0eQs5LXMfB4zn8bM5zPzR5j5pcx8ATPPFv/+D3H9LzHz9zPzy5n5D8TxzzLzy4pzv9RUenNIUetto3KOUJLLhvpqnK4rXa7ntDFscr23y6F4X+s5F7vumq5LjYnfEPPHFjtZODy02PB9Sm9XdTatpJZ3yIivAkxphT4JKqU96qj3Xs1CjoFU36c6yB1atDUMCqWrqg2vjfubHNq4nm3Ka6iziqnptq9rutNSDaxnVO2xTm7b1MqsZFvvURTVwAZE1R6rLaGiwkvpAyrAlCza2Cuz6oxcShqnxVWljtnNw3tns+q9zf1UVYA1RKjhDNUOAgC3Hb+k1H05ea66aD5lZqvtmPgm/23XfR0BCNZflzc7K68N5bcOQacCrCHuPLDcmC4rcsizcXa+UhlynuvA5H/s5ZBjxK/DZUWN+Ir6Pym9Ro34ShAVXv1AF+XnowJMUXqCzuzmowKsh+Qael3Xa28+DoY84dMGKsBqxsw+Vpm+zjX0uq63e/O6BNrhvbOjWXhs03a+UoTT0I3+TZepGvEVpUXmFk/1em+FPqJG/A5o05lPGQ4qvOpFBZgyKkzHobajfnF055bhberRFTqEVMbONA1FdQipJFO35hJbR1elV56W9YwupkV4xVANrCU0/IyilEM1sB6gwktpmybtgH2ZpFIBNgLqbqh1N86hGtb78pGWpUkfsr4sP9MhZAfo4un60Zj740WHkD1DhVf9qPCaTlSAKYoyWFSAKb0hdd1c27a0sa79HAMqwJTekOrb1PZwUX2u+osKMEVRBosKMEVRBosKMEVRBosKsIEwNCdQRWkDFWADYax+Thr6WqmCCjClU3SNqFIFFWCKogwWFWCKogwWFWCKogwWFWCKogwWFWCKogwWFWCKogwWFWBKoww9qqnSb1SAKY2iwRuVJlEBpoyCaV9qNa35VwGmKCNgrEvNYqgAU0bBtH7AQ6GpqLaNCTAi+igRHSeiRx3n/hMRMRF9T/GbiOjXiOgQET1CRK8W115PRAeLf9c3lV5FUZqjqai2TWpgewC83j5IRBcAeC2Ax8XhNwDYUPy7EcAHimvPBfCzAF4D4DIAP0tE5zSYZkVRBkRjAoyZvwDgacep9wH4aQByQ8qrAfwWT/hTAGcT0XkAtgP4HDM/zczPAPgcHEJRGTcacqef9GGzk1ZtYET0IwCOMvPD1ql1AJ4Qv48Ux3zHR4v6Ta1EQ+70kz5sdrKqrRcR0RkAfgbA61ynHcc4cNz1/BsxGX7idJxRMpXdo35TipJOmxrY9wO4CMDDRPR1AOcD+DMi+l5MNKsLxLXnAzgWOL4CZv4gM29m5s2nYXUDyVcUpW+0JsCYeT8zr2HmC5n5QkyE06uZ+RsA7gbwtmI28nIA32bmJwHMA3gdEZ1TGO9fVxxTFEVp1I3iDgAPAHg5ER0hohsCl38WwGEAhwB8CMBNAMDMTwP4RQALxb9fKI4piqKAmJ0mpUFzFp3Lr6Gruk6Goig18Mf8qX3MvNl1Tj3xFUUZLCrAFEWpjaM7t9TynFTfPxVgiqLUxrrd99fynFTfPxVgijLFDD0MjwowRZlihh7FQwWY0gq+nn7oGoDSLSrARkAfFtXG8PX0Q9cAlG5RAaY0yhCEq9I/UoMaqABTGqUPEQuU4ZEa1EAFmLKMujWmvmlganMbFyrARsCdB+oL+Fe3xtQ3DUxtbuNCBdgI0I9SmVZUgCmt0LehpNIOTde7CjClFfo2lFTaoel6VwGmKMpgUQE2JejOPkpbtNnWVIBNCX3d2acr21ifbXJDd/Vos62pAFM6pSvbWJ9tcjqrnI4KMEXpATrEL4cKMKVRYsOhoQ+X6qKvQ/y+owJMaZSDW/cEz+twSamCCjClUWKLcvtsTFf6jwowpVP6bExX+o8KMEVRBosKMGUU6FC0PoY0saICTGmcuvYKtJEfWh1DURWCE+qcWGmq7g3EzI2+oAvOonP5NXRV18lQFKUG/pg/tY+ZN7vOqQamZKFaitInVIApWaQM1exhQ9PDCGV6UQGm1I69vXxd283noJridKACTBkl6l82HagAUxRlsKgAUxQFwDCH3SrAFEUBMMxhtwowZaoYkpe5EkcFmDJVaPiecaECTFGUwaICTFGUwaICTFGUwaICTFGUwaICTFGUwaICTFGUwaICTFGUwdKYACOijxLRcSJ61Dr+LiL6ChF9mYj+mzj+HiI6VJzbLo6/vjh2iIhubSq9iqIMjyY1sD0AXi8PENE/B3A1gFcy8ysA/EpxfCOANwN4RXHP+4lohohmAPw6gDcA2Ajg2uJaRVE6pg+rGlY19WBm/gIRXWgd/g8AfpmZTxTXHC+OXw3gE8XxrxHRIQCXFecOMfNhACCiTxTXHmgq3YqipNGHVQ1t28BeBuBKIvoSEf0JEc0Vx9cBeEJcd6Q45juuKIrSnAYWeN85AC4HMAfgk0S0HgA5rmW4BaxzFxIiuhHAjQBwOs6oJbGKovSbtjWwIwB+jyc8COAfAXxPcfwCcd35AI4Fjq+AmT/IzJuZefNpWN1I4pV+MH+s+6GL0g/aFmB3AdgGAET0MgDPA/BXAO4G8GYiWk1EFwHYAOBBAAsANhDRRUT0PEwM/Xe3nGalZ2xfu9J43AeDstI+TbpR3AHgAQAvJ6IjRHQDgI8CWF+4VnwCwPWFNvZlAJ/ExDj/hwB+gplPMfNJAO8EMA/gMQCfLK5VlGVIg3KbwmyIUUzHhG5sqyhKr9GNbRVFGSUqwBSlRU5u29R1EkaFCjBFaZFV9+7rOgmjQgWYoiiDRQWYoiiDRQWYoiiDRQWYoiiDRQWYoiiDRQWYoiiDRQWYoiiDRQWYoiiDRQWYoiiDRQWYoiiDRQWYoiiDRQWYoiiDRQWYoiiDZZQBDYnomwD+HpNw1WPgezCevACanz7Tx7x8HzO/yHVilAIMAIjoIV8Ux6ExprwAmp8+M7S86BBSUZTBogJMUZTBMmYB9sGuE1AjY8oLoPnpM4PKy2htYIqijJ8xa2CKooyc0QkwIno9EX2FiA4R0a1dpycFIrqAiD5PRI8R0ZeJ6Obi+LlE9DkiOlj8f05xnIjo14o8PkJEr+42Byshohki+nMi+kzx+yIi+lKRlzuLndZR7MZ+Z5GXLxHRhV2m2wURnU1EnyKivyjq6IqB181PFe3sUSK6g4hOH2r9jEqAEdEMgF8H8AYAGwFcS0Qbu01VEicBvJuZ/ymAywH8RJHuWwHcw8wbANxT/AYm+dtQ/LsRwAfaT3KUmzHZTd2wG8D7irw8A+CG4vgNAJ5h5pcCeF9xXd+4HcAfMvMPALgUk3wNsm6IaB2AnwSwmZkvBjAD4M0Yav0w82j+AbgCwLz4/R4A7+k6XSXy8WkArwXwFQDnFcfOA/CV4u/fAHCtuH7puj78A3A+Jh/1NgCfAUCYOEeususJwDyAK4q/VxXXUdd5EHk5C8DX7DQNuG7WAXgCwLlFeX8GwPah1s+oNDA8VzmGI8WxwVCo6K8C8CUAL2bmJwGg+H9NcVnf8/mrAH4awD8Wv18I4FvMfLL4LdO7lJfi/LeL6/vCegDfBPCbxZD4w0R0JgZaN8x8FMCvAHgcwJOYlPc+DLR+xibAyHFsMNOsRPR8AP8TwH9k5r8JXeo41ot8EtGbABxnZrmDayi9vc1LwSoArwbwAWZ+FSZL1EK21V7np7DVXQ3gIgBrAZyJybDXZhD1MzYBdgTABeL3+QCOdZSWLIjoNEyE1+8w8+8Vh58iovOK8+cBOF4c73M+/xmAHyGirwP4BCbDyF8FcDYRrSqukeldyktx/rsBPN1mgiMcAXCEmb9U/P4UJgJtiHUDAD8M4GvM/E1m/g6A3wOwBQOtn7EJsAUAG4oZledhYpy8u+M0RSEiAvARAI8x83vFqbsBXF/8fT0mtjFz/G3FjNflAL5thjNdw8zvYebzmflCTMr/Xmb+MQCfB/CjxWV2Xkwef7S4vjc9PDN/A8ATRPTy4tBVAA5ggHVT8DiAy4nojKLdmfwMsn46N8I1YKR8I4CvAvhLAD/TdXoS0/yDmKjljwBYLP69ERNbwz0ADhb/n1tcT5jMtv4lgP2YzCh1ng9HvrYC+Ezx93oADwI4BOB3Aawujp9e/D5UnF/fdbod+ZgF8FBRP3cBOGfIdQPg5wH8BYBHAfw2gNVDrR/1xFcUZbCMbQipKMoUoQJMUZTBogJMUZTBogJMUZTBogJMUZTBogJM6QVEdCERPWod+zki+k9dpUnpPyrAlNEiPMuVkaICTOk9RPSTRHSgiK/1ieLYmUT0USJaKBZZX10cfzsR/S4R/b8A/oiIziOiLxDRYhH/6spOM6PUivZQyhC4FcBFzHyCiM4ujv0MJstafrw49iAR/XFx7goAr2Tmp4no3ZiEhvmlIl7cGe0nX2kKFWBKX/AtCTFLrH6HiO7CZCkPALwOk0XjxkZ2OoCXFH9/jpnNguMFAB8tFsvfxcyL9Sdd6QodQip94a8xWWMoOReTAHr/ApP1hZsA7CtsWwTg3zDzbPHvJcxsIsD+vXkAM38BwA8BOArgt4nobQ3nQ2kRFWBKL2DmvwPwJBFdBUz2AwDwegD/H4ALmPnzmARJPBvA8zGJFPquIqICiOhVrucS0fdhEp/sQ5hE/OhdjHqlPDqEVPrE2wD8OhH9j+L3z2MS/uXzRPTdmGhd72PmbxHRL2ISZ+yRQoh9HcCbHM/cCuD/JKLvAPi74h3KSNBoFIqiDBYdQiqKMlhUgCmKMlhUgCmKMlhUgCmKMlhUgCmKMlhUgCmKMlhUgCmKMlhUgCmKMlj+f6IjsLJVlkBqAAAAAElFTkSuQmCC\n",
+ "text/plain": [
+ "
"
+ ]
+ },
+ "metadata": {
+ "needs_background": "light"
+ },
+ "output_type": "display_data"
+ }
+ ],
+ "source": [
+ "# Load data\n",
+ "data = loadmat(os.path.join('Data', 'ex8_movies.mat'))\n",
+ "Y, R = data['Y'], data['R']\n",
+ "\n",
+ "# Y is a 1682x943 matrix, containing ratings (1-5) of \n",
+ "# 1682 movies on 943 users\n",
+ "\n",
+ "# R is a 1682x943 matrix, where R(i,j) = 1 \n",
+ "# if and only if user j gave a rating to movie i\n",
+ "\n",
+ "# From the matrix, we can compute statistics like average rating.\n",
+ "print('Average rating for movie 1 (Toy Story): %f / 5' %\n",
+ " np.mean(Y[0, R[0, :] == 1]))\n",
+ "\n",
+ "# We can \"visualize\" the ratings matrix by plotting it with imshow\n",
+ "pyplot.figure(figsize=(8, 8))\n",
+ "pyplot.imshow(Y)\n",
+ "pyplot.ylabel('Movies')\n",
+ "pyplot.xlabel('Users')\n",
+ "pyplot.grid(False)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Throughout this part of the exercise, you will also be working with the matrices, `X` and `Theta`:\n",
+ "\n",
+ "$$ \\text{X} = \n",
+ "\\begin{bmatrix}\n",
+ "- \\left(x^{(1)}\\right)^T - \\\\\n",
+ "- \\left(x^{(2)}\\right)^T - \\\\\n",
+ "\\vdots \\\\\n",
+ "- \\left(x^{(n_m)}\\right)^T - \\\\\n",
+ "\\end{bmatrix}, \\quad\n",
+ "\\text{Theta} = \n",
+ "\\begin{bmatrix}\n",
+ "- \\left(\\theta^{(1)}\\right)^T - \\\\\n",
+ "- \\left(\\theta^{(2)}\\right)^T - \\\\\n",
+ "\\vdots \\\\\n",
+ "- \\left(\\theta^{(n_u)}\\right)^T - \\\\\n",
+ "\\end{bmatrix}.\n",
+ "$$\n",
+ "\n",
+ "The $i^{th}$ row of `X` corresponds to the feature vector $x^{(i)}$ for the $i^{th}$ movie, and the $j^{th}$ row of `Theta` corresponds to one parameter vector $\\theta^{(j)}$, for the $j^{th}$ user. Both $x^{(i)}$ and $\\theta^{(j)}$ are n-dimensional vectors. For the purposes of this exercise, you will use $n = 100$, and therefore, $x^{(i)} \\in \\mathbb{R}^{100}$ and $\\theta^{(j)} \\in \\mathbb{R}^{100}$. Correspondingly, `X` is a $n_m \\times 100$ matrix and `Theta` is a $n_u \\times 100$ matrix.\n",
+ "\n",
+ "\n",
+ "### 2.2 Collaborative filtering learning algorithm\n",
+ "\n",
+ "Now, you will start implementing the collaborative filtering learning algorithm. You will start by implementing the cost function (without regularization).\n",
+ "\n",
+ "The collaborative filtering algorithm in the setting of movie recommendations considers a set of n-dimensional parameter vectors $x^{(1)}, \\dots, x^{(n_m)}$ and $\\theta^{(1)} , \\dots, \\theta^{(n_u)}$, where the model predicts the rating for movie $i$ by user $j$ as $y^{(i,j)} = \\left( \\theta^{(j)} \\right)^T x^{(i)}$. Given a dataset that consists of a set of ratings produced by some users on some movies, you wish to learn the parameter vectors $x^{(1)}, \\dots, x^{(n_m)}, \\theta^{(1)}, \\dots, \\theta^{(n_u)}$ that produce the best fit (minimizes the squared error).\n",
+ "\n",
+ "You will complete the code in `cofiCostFunc` to compute the cost function and gradient for collaborative filtering. Note that the parameters to the function (i.e., the values that you are trying to learn) are `X` and `Theta`. In order to use an off-the-shelf minimizer such as `scipy`'s `minimize` function, the cost function has been set up to unroll the parameters into a single vector called `params`. You had previously used the same vector unrolling method in the neural networks programming exercise.\n",
+ "\n",
+ "#### 2.2.1 Collaborative filtering cost function\n",
+ "\n",
+ "The collaborative filtering cost function (without regularization) is given by\n",
+ "\n",
+ "$$\n",
+ "J(x^{(1)}, \\dots, x^{(n_m)}, \\theta^{(1)}, \\dots,\\theta^{(n_u)}) = \\frac{1}{2} \\sum_{(i,j):r(i,j)=1} \\left( \\left(\\theta^{(j)}\\right)^T x^{(i)} - y^{(i,j)} \\right)^2\n",
+ "$$\n",
+ "\n",
+ "You should now modify the function `cofiCostFunc` to return this cost in the variable `J`. Note that you should be accumulating the cost for user $j$ and movie $i$ only if `R[i,j] = 1`.\n",
+ "\n",
+ "
\n",
+ "**Implementation Note**: We strongly encourage you to use a vectorized implementation to compute $J$, since it will later by called many times by `scipy`'s optimization package. As usual, it might be easiest to first write a non-vectorized implementation (to make sure you have the right answer), and the modify it to become a vectorized implementation (checking that the vectorization steps do not change your algorithm’s output). To come up with a vectorized implementation, the following tip might be helpful: You can use the $R$ matrix to set selected entries to 0. For example, `R * M` will do an element-wise multiplication between `M`\n",
+ "and `R`; since `R` only has elements with values either 0 or 1, this has the effect of setting the elements of M to 0 only when the corresponding value in R is 0. Hence, `np.sum( R * M)` is the sum of all the elements of `M` for which the corresponding element in `R` equals 1.\n",
+ "
\n",
+ "\n",
+ ""
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 12,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "def cofiCostFunc(params, Y, R, num_users, num_movies,\n",
+ " num_features, lambda_=0.0):\n",
+ " \"\"\"\n",
+ " Collaborative filtering cost function.\n",
+ " \n",
+ " Parameters\n",
+ " ----------\n",
+ " params : array_like\n",
+ " The parameters which will be optimized. This is a one\n",
+ " dimensional vector of shape (num_movies x num_users, 1). It is the \n",
+ " concatenation of the feature vectors X and parameters Theta.\n",
+ " \n",
+ " Y : array_like\n",
+ " A matrix of shape (num_movies x num_users) of user ratings of movies.\n",
+ " \n",
+ " R : array_like\n",
+ " A (num_movies x num_users) matrix, where R[i, j] = 1 if the \n",
+ " i-th movie was rated by the j-th user.\n",
+ " \n",
+ " num_users : int\n",
+ " Total number of users.\n",
+ " \n",
+ " num_movies : int\n",
+ " Total number of movies.\n",
+ " \n",
+ " num_features : int\n",
+ " Number of features to learn.\n",
+ " \n",
+ " lambda_ : float, optional\n",
+ " The regularization coefficient.\n",
+ " \n",
+ " Returns\n",
+ " -------\n",
+ " J : float\n",
+ " The value of the cost function at the given params.\n",
+ " \n",
+ " grad : array_like\n",
+ " The gradient vector of the cost function at the given params.\n",
+ " grad has a shape (num_movies x num_users, 1)\n",
+ " \n",
+ " Instructions\n",
+ " ------------\n",
+ " Compute the cost function and gradient for collaborative filtering.\n",
+ " Concretely, you should first implement the cost function (without\n",
+ " regularization) and make sure it is matches our costs. After that,\n",
+ " you should implement thegradient and use the checkCostFunction routine \n",
+ " to check that the gradient is correct. Finally, you should implement\n",
+ " regularization.\n",
+ " \n",
+ " Notes\n",
+ " -----\n",
+ " - The input params will be unraveled into the two matrices:\n",
+ " X : (num_movies x num_features) matrix of movie features\n",
+ " Theta : (num_users x num_features) matrix of user features\n",
+ "\n",
+ " - You should set the following variables correctly:\n",
+ "\n",
+ " X_grad : (num_movies x num_features) matrix, containing the \n",
+ " partial derivatives w.r.t. to each element of X\n",
+ " Theta_grad : (num_users x num_features) matrix, containing the \n",
+ " partial derivatives w.r.t. to each element of Theta\n",
+ "\n",
+ " - The returned gradient will be the concatenation of the raveled \n",
+ " gradients X_grad and Theta_grad.\n",
+ " \"\"\"\n",
+ " # Unfold the U and W matrices from params\n",
+ " X = params[:num_movies*num_features].reshape(num_movies, num_features)\n",
+ " Theta = params[num_movies*num_features:].reshape(num_users, num_features)\n",
+ "\n",
+ " # You need to return the following values correctly\n",
+ " J = 0\n",
+ " X_grad = np.zeros(X.shape)\n",
+ " Theta_grad = np.zeros(Theta.shape)\n",
+ "\n",
+ " # ====================== YOUR CODE HERE ======================\n",
+ " J = (1 / 2) * np.sum(np.square((X.dot(Theta.T) - Y) * R)) + (lambda_ / 2) * np.sum(np.square(X)) + \\\n",
+ " (lambda_ / 2) * np.sum(np.square(Theta))\n",
+ " \n",
+ " for i in range(R.shape[0]):\n",
+ " \n",
+ " idx = np.where(R[i, :] == 1)[0]\n",
+ " Theta_temp = Theta[idx, :]\n",
+ " Y_temp = Y[i, idx]\n",
+ " X_grad[i, :] = np.dot(np.dot(X[i, :], Theta_temp.T) - Y_temp, Theta_temp) + lambda_ * X[i, :]\n",
+ " \n",
+ " for j in range(R.shape[1]):\n",
+ " \n",
+ " idx = np.where(R[:, j] == 1)[0]\n",
+ " X_temp = X[idx, :]\n",
+ " Y_temp = Y[idx, j]\n",
+ " Theta_grad[j, :] = np.dot(np.dot(X_temp, Theta[j, :]) - Y_temp, X_temp) + lambda_ * Theta[j, :]\n",
+ " \n",
+ " \n",
+ " \n",
+ " # =============================================================\n",
+ " \n",
+ " grad = np.concatenate([X_grad.ravel(), Theta_grad.ravel()])\n",
+ " return J, grad"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "After you have completed the function, the next cell will run your cost function. To help you debug your cost function, we have included set of weights that we trained on that. You should expect to see an output of 22.22."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 13,
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "Cost at loaded parameters: 22.22 \n",
+ "(this value should be about 22.22)\n"
+ ]
+ }
+ ],
+ "source": [
+ "# Load pre-trained weights (X, Theta, num_users, num_movies, num_features)\n",
+ "data = loadmat(os.path.join('Data', 'ex8_movieParams.mat'))\n",
+ "X, Theta, num_users, num_movies, num_features = data['X'],\\\n",
+ " data['Theta'], data['num_users'], data['num_movies'], data['num_features']\n",
+ "\n",
+ "# Reduce the data set size so that this runs faster\n",
+ "num_users = 4\n",
+ "num_movies = 5\n",
+ "num_features = 3\n",
+ "\n",
+ "X = X[:num_movies, :num_features]\n",
+ "Theta = Theta[:num_users, :num_features]\n",
+ "Y = Y[:num_movies, 0:num_users]\n",
+ "R = R[:num_movies, 0:num_users]\n",
+ "\n",
+ "# Evaluate cost function\n",
+ "J, _ = cofiCostFunc(np.concatenate([X.ravel(), Theta.ravel()]),\n",
+ " Y, R, num_users, num_movies, num_features)\n",
+ " \n",
+ "print('Cost at loaded parameters: %.2f \\n(this value should be about 22.22)' % J)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "*You should now submit your solutions.*"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 14,
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "\n",
+ "Submitting Solutions | Programming Exercise anomaly-detection-and-recommender-systems\n",
+ "\n",
+ "Use token from last successful submission (tanmayss2109@gmail.com)? (Y/n): y\n",
+ " Part Name | Score | Feedback\n",
+ " --------- | ----- | --------\n",
+ " Estimate Gaussian Parameters | 15 / 15 | Nice work!\n",
+ " Select Threshold | 15 / 15 | Nice work!\n",
+ " Collaborative Filtering Cost | 20 / 20 | Nice work!\n",
+ " Collaborative Filtering Gradient | 0 / 30 | \n",
+ " Regularized Cost | 0 / 10 | \n",
+ " Regularized Gradient | 0 / 10 | \n",
+ " --------------------------------\n",
+ " | 50 / 100 | \n",
+ "\n"
+ ]
+ }
+ ],
+ "source": [
+ "grader[3] = cofiCostFunc\n",
+ "grader.grade()"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "\n",
+ "#### 2.2.2 Collaborative filtering gradient\n",
+ "\n",
+ "Now you should implement the gradient (without regularization). Specifically, you should complete the code in `cofiCostFunc` to return the variables `X_grad` and `Theta_grad`. Note that `X_grad` should be a matrix of the same size as `X` and similarly, `Theta_grad` is a matrix of the same size as\n",
+ "`Theta`. The gradients of the cost function is given by:\n",
+ "\n",
+ "$$ \\frac{\\partial J}{\\partial x_k^{(i)}} = \\sum_{j:r(i,j)=1} \\left( \\left(\\theta^{(j)}\\right)^T x^{(i)} - y^{(i,j)} \\right) \\theta_k^{(j)} $$\n",
+ "\n",
+ "$$ \\frac{\\partial J}{\\partial \\theta_k^{(j)}} = \\sum_{i:r(i,j)=1} \\left( \\left(\\theta^{(j)}\\right)^T x^{(i)}- y^{(i,j)} \\right) x_k^{(j)} $$\n",
+ "\n",
+ "Note that the function returns the gradient for both sets of variables by unrolling them into a single vector. After you have completed the code to compute the gradients, the next cell run a gradient check\n",
+ "(available in `utils.checkCostFunction`) to numerically check the implementation of your gradients (this is similar to the numerical check that you used in the neural networks exercise. If your implementation is correct, you should find that the analytical and numerical gradients match up closely.\n",
+ "\n",
+ "
\n",
+ "**Implementation Note:** You can get full credit for this assignment without using a vectorized implementation, but your code will run much more slowly (a small number of hours), and so we recommend that you try to vectorize your implementation. To get started, you can implement the gradient with a for-loop over movies\n",
+ "(for computing $\\frac{\\partial J}{\\partial x^{(i)}_k}$) and a for-loop over users (for computing $\\frac{\\partial J}{\\theta_k^{(j)}}$). When you first implement the gradient, you might start with an unvectorized version, by implementing another inner for-loop that computes each element in the summation. After you have completed the gradient computation this way, you should try to vectorize your implementation (vectorize the inner for-loops), so that you are left with only two for-loops (one for looping over movies to compute $\\frac{\\partial J}{\\partial x_k^{(i)}}$ for each movie, and one for looping over users to compute $\\frac{\\partial J}{\\partial \\theta_k^{(j)}}$ for each user).\n",
+ "
\n",
+ "\n",
+ "
\n",
+ "**Implementation Tip:** To perform the vectorization, you might find this helpful: You should come up with a way to compute all the derivatives associated with $x_1^{(i)} , x_2^{(i)}, \\dots , x_n^{(i)}$ (i.e., the derivative terms associated with the feature vector $x^{(i)}$) at the same time. Let us define the derivatives for the feature vector of the $i^{th}$ movie as:\n",
+ "\n",
+ "$$ \\left(X_{\\text{grad}} \\left(i, :\\right)\\right)^T = \n",
+ "\\begin{bmatrix}\n",
+ "\\frac{\\partial J}{\\partial x_1^{(i)}} \\\\\n",
+ "\\frac{\\partial J}{\\partial x_2^{(i)}} \\\\\n",
+ "\\vdots \\\\\n",
+ "\\frac{\\partial J}{\\partial x_n^{(i)}}\n",
+ "\\end{bmatrix} = \\quad\n",
+ "\\sum_{j:r(i,j)=1} \\left( \\left( \\theta^{(j)} \\right)^T x^{(i)} - y^{(i,j)} \\right) \\theta^{(j)}\n",
+ "$$\n",
+ "\n",
+ "To vectorize the above expression, you can start by indexing into `Theta` and `Y` to select only the elements of interests (that is, those with `r[i, j] = 1`). Intuitively, when you consider the features for the $i^{th}$ movie, you only need to be concerned about the users who had given ratings to the movie, and this allows you to remove all the other users from `Theta` and `Y`.
\n",
+ "\n",
+ "\n",
+ "Concretely, you can set `idx = np.where(R[i, :] == 1)[0]` to be a list of all the users that have rated movie $i$. This will allow you to create the temporary matrices `Theta_temp = Theta[idx, :]` and `Y_temp = Y[i, idx]` that index into `Theta` and `Y` to give you only the set of users which have rated the $i^{th}$ movie. This will allow you to write the derivatives as: \n",
+ "\n",
+ "`X_grad[i, :] = np.dot(np.dot(X[i, :], Theta_temp.T) - Y_temp, Theta_temp)`\n",
+ "\n",
+ "
\n",
+ "Note that the vectorized computation above returns a row-vector instead. After you have vectorized the computations of the derivatives with respect to $x^{(i)}$, you should use a similar method to vectorize the derivatives with respect to $θ^{(j)}$ as well.\n",
+ "
\n",
+ "\n",
+ "[Click here to go back to the function `cofiCostFunc` to update it](#cofiCostFunc). \n",
+ "\n",
+ " Do not forget to re-execute the cell containg the function `cofiCostFunc` so that it is updated with your implementation of the gradient computation."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 15,
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "[[ 0.46660851 0.46660851]\n",
+ " [-1.11522931 -1.11522931]\n",
+ " [-2.93605559 -2.93605559]\n",
+ " [-0.54310538 -0.54310538]\n",
+ " [-7.09376845 -7.09376845]\n",
+ " [-8.86613311 -8.86613311]\n",
+ " [ 0.08789442 0.08789442]\n",
+ " [-0.13815802 -0.13815802]\n",
+ " [ 0.01326013 0.01326013]\n",
+ " [ 0. 0. ]\n",
+ " [ 0. 0. ]\n",
+ " [ 0. 0. ]\n",
+ " [-0.75793651 -0.75793651]\n",
+ " [ 0.50622184 0.50622184]\n",
+ " [ 1.22902557 1.22902557]\n",
+ " [-0.32099776 -0.32099776]\n",
+ " [ 0.22352602 0.22352602]\n",
+ " [ 0.76455842 0.76455842]\n",
+ " [-1.25906737 -1.25906737]\n",
+ " [ 0.74762981 0.74762981]\n",
+ " [ 2.57050722 2.57050722]\n",
+ " [ 0. 0. ]\n",
+ " [ 0. 0. ]\n",
+ " [ 0. 0. ]\n",
+ " [-1.93903986 -1.93903986]\n",
+ " [ 1.25368512 1.25368512]\n",
+ " [ 3.37886598 3.37886598]]\n",
+ "\n",
+ "The above two columns you get should be very similar.(Left-Your Numerical Gradient, Right-Analytical Gradient)\n",
+ "If your cost function implementation is correct, then the relative difference will be small (less than 1e-9).\n",
+ "\n",
+ "Relative Difference: 1.01587e-12\n"
+ ]
+ }
+ ],
+ "source": [
+ "# Check gradients by running checkcostFunction\n",
+ "utils.checkCostFunction(cofiCostFunc)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "*You should now submit your solutions*"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 16,
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "\n",
+ "Submitting Solutions | Programming Exercise anomaly-detection-and-recommender-systems\n",
+ "\n",
+ "Use token from last successful submission (tanmayss2109@gmail.com)? (Y/n): y\n",
+ " Part Name | Score | Feedback\n",
+ " --------- | ----- | --------\n",
+ " Estimate Gaussian Parameters | 15 / 15 | Nice work!\n",
+ " Select Threshold | 15 / 15 | Nice work!\n",
+ " Collaborative Filtering Cost | 20 / 20 | Nice work!\n",
+ " Collaborative Filtering Gradient | 30 / 30 | Nice work!\n",
+ " Regularized Cost | 0 / 10 | \n",
+ " Regularized Gradient | 0 / 10 | \n",
+ " --------------------------------\n",
+ " | 80 / 100 | \n",
+ "\n"
+ ]
+ }
+ ],
+ "source": [
+ "grader[4] = cofiCostFunc\n",
+ "grader.grade()"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "\n",
+ "#### 2.2.3 Regularized cost function\n",
+ "\n",
+ "The cost function for collaborative filtering with regularization is given by\n",
+ "\n",
+ "$$ J(x^{(1)}, \\dots, x^{(n_m)}, \\theta^{(1)}, \\dots, \\theta^{(n_u)}) = \\frac{1}{2} \\sum_{(i,j):r(i,j)=1} \\left( \\left( \\theta^{(j)} \\right)^T x^{(i)} - y^{(i,j)} \\right)^2 + \\left( \\frac{\\lambda}{2} \\sum_{j=1}^{n_u} \\sum_{k=1}^{n} \\left( \\theta_k^{(j)} \\right)^2 \\right) + \\left( \\frac{\\lambda}{2} \\sum_{i=1}^{n_m} \\sum_{k=1}^n \\left(x_k^{(i)} \\right)^2 \\right) $$\n",
+ "\n",
+ "You should now add regularization to your original computations of the cost function, $J$. After you are done, the next cell will run your regularized cost function, and you should expect to see a cost of about 31.34.\n",
+ "\n",
+ "[Click here to go back to the function `cofiCostFunc` to update it](#cofiCostFunc)\n",
+ " Do not forget to re-execute the cell containing the function `cofiCostFunc` so that it is updated with your implementation of regularized cost function."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 17,
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "Cost at loaded parameters (lambda = 1.5): 31.34\n",
+ " (this value should be about 31.34)\n"
+ ]
+ }
+ ],
+ "source": [
+ "# Evaluate cost function\n",
+ "J, _ = cofiCostFunc(np.concatenate([X.ravel(), Theta.ravel()]),\n",
+ " Y, R, num_users, num_movies, num_features, 1.5)\n",
+ " \n",
+ "print('Cost at loaded parameters (lambda = 1.5): %.2f' % J)\n",
+ "print(' (this value should be about 31.34)')"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "*You should now submit your solutions.*"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 18,
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "\n",
+ "Submitting Solutions | Programming Exercise anomaly-detection-and-recommender-systems\n",
+ "\n",
+ "Use token from last successful submission (tanmayss2109@gmail.com)? (Y/n): y\n",
+ " Part Name | Score | Feedback\n",
+ " --------- | ----- | --------\n",
+ " Estimate Gaussian Parameters | 15 / 15 | Nice work!\n",
+ " Select Threshold | 15 / 15 | Nice work!\n",
+ " Collaborative Filtering Cost | 20 / 20 | Nice work!\n",
+ " Collaborative Filtering Gradient | 30 / 30 | Nice work!\n",
+ " Regularized Cost | 10 / 10 | Nice work!\n",
+ " Regularized Gradient | 0 / 10 | \n",
+ " --------------------------------\n",
+ " | 90 / 100 | \n",
+ "\n"
+ ]
+ }
+ ],
+ "source": [
+ "grader[5] = cofiCostFunc\n",
+ "grader.grade()"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "\n",
+ "#### 2.2.4 Regularized gradient\n",
+ "\n",
+ "Now that you have implemented the regularized cost function, you should proceed to implement regularization for the gradient. You should add to your implementation in `cofiCostFunc` to return the regularized gradient\n",
+ "by adding the contributions from the regularization terms. Note that the gradients for the regularized cost function is given by:\n",
+ "\n",
+ "$$ \\frac{\\partial J}{\\partial x_k^{(i)}} = \\sum_{j:r(i,j)=1} \\left( \\left(\\theta^{(j)}\\right)^T x^{(i)} - y^{(i,j)} \\right) \\theta_k^{(j)} + \\lambda x_k^{(i)} $$\n",
+ "\n",
+ "$$ \\frac{\\partial J}{\\partial \\theta_k^{(j)}} = \\sum_{i:r(i,j)=1} \\left( \\left(\\theta^{(j)}\\right)^T x^{(i)}- y^{(i,j)} \\right) x_k^{(j)} + \\lambda \\theta_k^{(j)} $$\n",
+ "\n",
+ "This means that you just need to add $\\lambda x^{(i)}$ to the `X_grad[i,:]` variable described earlier, and add $\\lambda \\theta^{(j)}$ to the `Theta_grad[j, :]` variable described earlier.\n",
+ "\n",
+ "[Click here to go back to the function `cofiCostFunc` to update it](#cofiCostFunc)\n",
+ " Do not forget to re-execute the cell containing the function `cofiCostFunc` so that it is updated with your implementation of the gradient for the regularized cost function.\n",
+ "\n",
+ "After you have completed the code to compute the gradients, the following cell will run another gradient check (`utils.checkCostFunction`) to numerically check the implementation of your gradients."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 19,
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "[[-2.10413092 -2.10413092]\n",
+ " [ 4.88435471 4.88435471]\n",
+ " [-5.14676934 -5.14676934]\n",
+ " [ 2.10807904 2.10807904]\n",
+ " [-0.69618007 -0.69618007]\n",
+ " [ 6.10717258 6.10717258]\n",
+ " [-0.98375746 -0.98375746]\n",
+ " [-1.62145317 -1.62145317]\n",
+ " [-3.64294105 -3.64294105]\n",
+ " [ 1.86940783 1.86940783]\n",
+ " [ 0.97383575 0.97383575]\n",
+ " [ 3.67349576 3.67349576]\n",
+ " [-0.81407458 -0.81407458]\n",
+ " [ 3.7746785 3.7746785 ]\n",
+ " [ 2.48789327 2.48789327]\n",
+ " [-1.81152765 -1.81152765]\n",
+ " [-1.36691463 -1.36691463]\n",
+ " [-3.72595908 -3.72595908]\n",
+ " [ 0.38573866 0.38573866]\n",
+ " [ 0.97208917 0.97208917]\n",
+ " [ 0.24478964 0.24478964]\n",
+ " [ 2.57555678 2.57555678]\n",
+ " [-4.32021577 -4.32021577]\n",
+ " [ 5.33165654 5.33165654]\n",
+ " [-1.51004106 -1.51004106]\n",
+ " [-1.77104474 -1.77104474]\n",
+ " [ 0.50360741 0.50360741]]\n",
+ "\n",
+ "The above two columns you get should be very similar.(Left-Your Numerical Gradient, Right-Analytical Gradient)\n",
+ "If your cost function implementation is correct, then the relative difference will be small (less than 1e-9).\n",
+ "\n",
+ "Relative Difference: 2.53851e-12\n"
+ ]
+ }
+ ],
+ "source": [
+ "# Check gradients by running checkCostFunction\n",
+ "utils.checkCostFunction(cofiCostFunc, 1.5)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "*You should now submit your solutions.*"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 20,
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "\n",
+ "Submitting Solutions | Programming Exercise anomaly-detection-and-recommender-systems\n",
+ "\n",
+ "Use token from last successful submission (tanmayss2109@gmail.com)? (Y/n): y\n",
+ " Part Name | Score | Feedback\n",
+ " --------- | ----- | --------\n",
+ " Estimate Gaussian Parameters | 15 / 15 | Nice work!\n",
+ " Select Threshold | 15 / 15 | Nice work!\n",
+ " Collaborative Filtering Cost | 20 / 20 | Nice work!\n",
+ " Collaborative Filtering Gradient | 30 / 30 | Nice work!\n",
+ " Regularized Cost | 10 / 10 | Nice work!\n",
+ " Regularized Gradient | 10 / 10 | Nice work!\n",
+ " --------------------------------\n",
+ " | 100 / 100 | \n",
+ "\n"
+ ]
+ }
+ ],
+ "source": [
+ "grader[6] = cofiCostFunc\n",
+ "grader.grade()"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "### 2.3 Learning movie recommendations \n",
+ "\n",
+ "After you have finished implementing the collaborative filtering cost function and gradient, you can now start training your algorithm to make movie recommendations for yourself. In the next cell, you can enter your own movie preferences, so that later when the algorithm runs, you can get your own movie recommendations! We have filled out some values according to our own preferences, but you should change this according to your own tastes. The list of all movies and their number in the dataset can be found listed in the file `Data/movie_idx.txt`."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 21,
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "New user ratings:\n",
+ "-----------------\n",
+ "Rated 4 stars: Toy Story (1995)\n",
+ "Rated 3 stars: Twelve Monkeys (1995)\n",
+ "Rated 5 stars: Usual Suspects, The (1995)\n",
+ "Rated 4 stars: Outbreak (1995)\n",
+ "Rated 5 stars: Shawshank Redemption, The (1994)\n",
+ "Rated 3 stars: While You Were Sleeping (1995)\n",
+ "Rated 5 stars: Forrest Gump (1994)\n",
+ "Rated 2 stars: Silence of the Lambs, The (1991)\n",
+ "Rated 4 stars: Alien (1979)\n",
+ "Rated 5 stars: Die Hard 2 (1990)\n",
+ "Rated 5 stars: Sphere (1998)\n"
+ ]
+ }
+ ],
+ "source": [
+ "# Before we will train the collaborative filtering model, we will first\n",
+ "# add ratings that correspond to a new user that we just observed. This\n",
+ "# part of the code will also allow you to put in your own ratings for the\n",
+ "# movies in our dataset!\n",
+ "movieList = utils.loadMovieList()\n",
+ "n_m = len(movieList)\n",
+ "\n",
+ "# Initialize my ratings\n",
+ "my_ratings = np.zeros(n_m)\n",
+ "\n",
+ "# Check the file movie_idx.txt for id of each movie in our dataset\n",
+ "# For example, Toy Story (1995) has ID 1, so to rate it \"4\", you can set\n",
+ "# Note that the index here is ID-1, since we start index from 0.\n",
+ "my_ratings[0] = 4\n",
+ "\n",
+ "# Or suppose did not enjoy Silence of the Lambs (1991), you can set\n",
+ "my_ratings[97] = 2\n",
+ "\n",
+ "# We have selected a few movies we liked / did not like and the ratings we\n",
+ "# gave are as follows:\n",
+ "my_ratings[6] = 3\n",
+ "my_ratings[11]= 5\n",
+ "my_ratings[53] = 4\n",
+ "my_ratings[63] = 5\n",
+ "my_ratings[65] = 3\n",
+ "my_ratings[68] = 5\n",
+ "my_ratings[182] = 4\n",
+ "my_ratings[225] = 5\n",
+ "my_ratings[354] = 5\n",
+ "\n",
+ "print('New user ratings:')\n",
+ "print('-----------------')\n",
+ "for i in range(len(my_ratings)):\n",
+ " if my_ratings[i] > 0:\n",
+ " print('Rated %d stars: %s' % (my_ratings[i], movieList[i]))"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "#### 2.3.1 Recommendations\n",
+ "\n",
+ "After the additional ratings have been added to the dataset, the script\n",
+ "will proceed to train the collaborative filtering model. This will learn the\n",
+ "parameters X and Theta. To predict the rating of movie i for user j, you need to compute (θ (j) ) T x (i) . The next part of the script computes the ratings for\n",
+ "all the movies and users and displays the movies that it recommends (Figure\n",
+ "4), according to ratings that were entered earlier in the script. Note that\n",
+ "you might obtain a different set of the predictions due to different random\n",
+ "initializations."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 22,
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "Recommender system learning completed.\n"
+ ]
+ }
+ ],
+ "source": [
+ "# Now, you will train the collaborative filtering model on a movie rating \n",
+ "# dataset of 1682 movies and 943 users\n",
+ "\n",
+ "# Load data\n",
+ "data = loadmat(os.path.join('Data', 'ex8_movies.mat'))\n",
+ "Y, R = data['Y'], data['R']\n",
+ "\n",
+ "# Y is a 1682x943 matrix, containing ratings (1-5) of 1682 movies by \n",
+ "# 943 users\n",
+ "\n",
+ "# R is a 1682x943 matrix, where R(i,j) = 1 if and only if user j gave a\n",
+ "# rating to movie i\n",
+ "\n",
+ "# Add our own ratings to the data matrix\n",
+ "Y = np.hstack([my_ratings[:, None], Y])\n",
+ "R = np.hstack([(my_ratings > 0)[:, None], R])\n",
+ "\n",
+ "# Normalize Ratings\n",
+ "Ynorm, Ymean = utils.normalizeRatings(Y, R)\n",
+ "\n",
+ "# Useful Values\n",
+ "num_movies, num_users = Y.shape\n",
+ "num_features = 10\n",
+ "\n",
+ "# Set Initial Parameters (Theta, X)\n",
+ "X = np.random.randn(num_movies, num_features)\n",
+ "Theta = np.random.randn(num_users, num_features)\n",
+ "\n",
+ "initial_parameters = np.concatenate([X.ravel(), Theta.ravel()])\n",
+ "\n",
+ "# Set options for scipy.optimize.minimize\n",
+ "options = {'maxiter': 100}\n",
+ "\n",
+ "# Set Regularization\n",
+ "lambda_ = 10\n",
+ "res = optimize.minimize(lambda x: cofiCostFunc(x, Ynorm, R, num_users,\n",
+ " num_movies, num_features, lambda_),\n",
+ " initial_parameters,\n",
+ " method='TNC',\n",
+ " jac=True,\n",
+ " options=options)\n",
+ "theta = res.x\n",
+ "\n",
+ "# Unfold the returned theta back into U and W\n",
+ "X = theta[:num_movies*num_features].reshape(num_movies, num_features)\n",
+ "Theta = theta[num_movies*num_features:].reshape(num_users, num_features)\n",
+ "\n",
+ "print('Recommender system learning completed.')"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "After training the model, you can now make recommendations by computing the predictions matrix."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 23,
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "Top recommendations for you:\n",
+ "----------------------------\n",
+ "Predicting rating 5.0 for movie Aiqing wansui (1994)\n",
+ "Predicting rating 5.0 for movie Great Day in Harlem, A (1994)\n",
+ "Predicting rating 5.0 for movie Someone Else's America (1995)\n",
+ "Predicting rating 5.0 for movie Entertaining Angels: The Dorothy Day Story (1996)\n",
+ "Predicting rating 5.0 for movie Santa with Muscles (1996)\n",
+ "Predicting rating 5.0 for movie They Made Me a Criminal (1939)\n",
+ "Predicting rating 5.0 for movie Star Kid (1997)\n",
+ "Predicting rating 5.0 for movie Marlene Dietrich: Shadow and Light (1996)\n",
+ "Predicting rating 5.0 for movie Saint of Fort Washington, The (1993)\n",
+ "Predicting rating 5.0 for movie Prefontaine (1997)\n",
+ "\n",
+ "Original ratings provided:\n",
+ "--------------------------\n",
+ "Rated 4 for Toy Story (1995)\n",
+ "Rated 3 for Twelve Monkeys (1995)\n",
+ "Rated 5 for Usual Suspects, The (1995)\n",
+ "Rated 4 for Outbreak (1995)\n",
+ "Rated 5 for Shawshank Redemption, The (1994)\n",
+ "Rated 3 for While You Were Sleeping (1995)\n",
+ "Rated 5 for Forrest Gump (1994)\n",
+ "Rated 2 for Silence of the Lambs, The (1991)\n",
+ "Rated 4 for Alien (1979)\n",
+ "Rated 5 for Die Hard 2 (1990)\n",
+ "Rated 5 for Sphere (1998)\n"
+ ]
+ }
+ ],
+ "source": [
+ "p = np.dot(X, Theta.T)\n",
+ "my_predictions = p[:, 0] + Ymean\n",
+ "\n",
+ "movieList = utils.loadMovieList()\n",
+ "\n",
+ "ix = np.argsort(my_predictions)[::-1]\n",
+ "\n",
+ "print('Top recommendations for you:')\n",
+ "print('----------------------------')\n",
+ "for i in range(10):\n",
+ " j = ix[i]\n",
+ " print('Predicting rating %.1f for movie %s' % (my_predictions[j], movieList[j]))\n",
+ "\n",
+ "print('\\nOriginal ratings provided:')\n",
+ "print('--------------------------')\n",
+ "for i in range(len(my_ratings)):\n",
+ " if my_ratings[i] > 0:\n",
+ " print('Rated %d for %s' % (my_ratings[i], movieList[i]))"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": []
+ }
+ ],
+ "metadata": {
+ "kernelspec": {
+ "display_name": "Python 3",
+ "language": "python",
+ "name": "python3"
+ },
+ "language_info": {
+ "codemirror_mode": {
+ "name": "ipython",
+ "version": 3
+ },
+ "file_extension": ".py",
+ "mimetype": "text/x-python",
+ "name": "python",
+ "nbconvert_exporter": "python",
+ "pygments_lexer": "ipython3",
+ "version": "3.7.6"
+ }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 2
+}
diff --git a/Phase 3 - 2020 (Summer)/Week 8/Exercise8/__pycache__/utils.cpython-37.pyc b/Phase 3 - 2020 (Summer)/Week 8/Exercise8/__pycache__/utils.cpython-37.pyc
new file mode 100644
index 000000000..e1fc8a943
Binary files /dev/null and b/Phase 3 - 2020 (Summer)/Week 8/Exercise8/__pycache__/utils.cpython-37.pyc differ
diff --git a/Phase 3 - 2020 (Summer)/Week 8/Exercise8/exercise8.ipynb b/Phase 3 - 2020 (Summer)/Week 8/Exercise8/exercise8.ipynb
index f0643da9c..8748a9866 100644
--- a/Phase 3 - 2020 (Summer)/Week 8/Exercise8/exercise8.ipynb
+++ b/Phase 3 - 2020 (Summer)/Week 8/Exercise8/exercise8.ipynb
@@ -23,7 +23,7 @@
},
{
"cell_type": "code",
- "execution_count": null,
+ "execution_count": 1,
"metadata": {},
"outputs": [],
"source": [
@@ -97,9 +97,22 @@
},
{
"cell_type": "code",
- "execution_count": null,
+ "execution_count": 2,
"metadata": {},
- "outputs": [],
+ "outputs": [
+ {
+ "data": {
+ "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYUAAAEKCAYAAAD9xUlFAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjMsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+AADFEAAAgAElEQVR4nO3df5RcdX3/8ecrmwRiEswPZkkEU4Sa1XWpIFurUlvWSmtt8UfLYDnfKlXbtOe72/qjpy2657ToqWdta8Vjs6FfWoKYr0oZgQKVbxH5LkT8tuoGBSYbRUHaomFnBZElRZDk/f3j/nD29+xmZ2d28nqcc8/O3Ln3zvsyZN5zPz/eVxGBmZkZwIpGB2BmZs3DScHMzHJOCmZmlnNSMDOznJOCmZnlnBTMzCxXt6Qg6XhJX5F0j6T9kj6Qrn+BpC9L+pakf5K0ul4xmJnZ/NTzSuFp4DUR8VLgTOB1kl4B/BVwWUS8EPgB8M46xmBmZvNQt6QQiSfTp6vSJYDXAJ9N118NvKleMZiZ2fysrOfBJbUB+4CfBgaBB4DHI+LZdJOHgZNn2HcHsANg7dq1Z7/oRS+qZ6hmZi1n375934+Iwnz2qWtSiIjDwJmSNgA3AC+ebrMZ9r0CuAKgu7s7hoeH6xanmVkrkvQf891nSUYfRcTjwB3AK4ANkrJkdArwvaWIwczM5lbP0UeF9AoBSWuA1wIHgCHggnSzi4Eb6xWDmZnNTz2bj7YCV6f9CiuAayPiXySNANdI+kvga8CVdYzBzMzmoW5JISLuBc6aZv2DwMvr9b5mZrZwntFsZmY5JwUzM8s5KZiZWc5JwczMck4KZmaWc1IwM7Ock4KZmeWcFMzMLOekYGZmOScFMzPLOSmYmVnOScHMzHJOCmZmlnNSMDOznJOCmZnlnBTMzCznpGBmZjknBTMzyzkpmJlZzknBzMxyTgpmZpZzUjAzs5yTgpmZ5ZwUzMws56RgZmY5JwUzM8s5KZiZWa5uSUHS8yUNSTogab+kd6XrL5X0XUlfT5fX1ysGMzObn5V1PPazwB9HxN2S1gP7JN2WvnZZRHykju9tZmYLULekEBEHgYPp43FJB4CT6/V+ZmZ29JakT0HSqcBZwJfTVX2S7pW0W9LGpYjBzMzmVvekIGkdcB3w7oh4ArgcOB04k+RK4m9n2G+HpGFJw2NjY/UOc0kMDg5SqVTy55VKhcHBwQZGZGY2UV2TgqRVJAnhUxFxPUBEjEbE4Yg4AvwD8PLp9o2IKyKiOyK6C4VCPcNcEoODg/T19dHT00OlUqFSqdDT00NfX58Tg5k1jXqOPhJwJXAgIj5atX5r1WZvBsr1iqGZFItFOjs7GRkZoauri66uLkZGRujs7KRYLDY6PDMzoL6jj84B3grcJ+nr6br3AxdJOhMI4CHg9+sYQ9Nob29naGiIrq4usuawQqHA0NAQ7e3tDY7OzCxRz9FHdwGa5qVb6vWeZmZ2dDyjeYlkfQhjY2MUCgUKhQJjY2N5H4OZWTNwUlgipVIp70Mol8uUy+W8j6FUKjU6PDMzoL59Clalt7cXSDqcsz6EoaEhSqVS/pqZWaMpIhodw5y6u7tjeHi40WGYmS0rkvZFRPd89nHzkZmZ5ZwUzMws56RgZmY5JwUzM8s5KZiZWc5JwczMck4KZmaWc1IwM7Ock4KZmeWcFMzMLOekYGZmOScFMzPLOSmYmVnOScHMzHJOCmZmlnNSMDOznJOCmZnlnBTMzCznpGBmZrmVs70o6Xjg14FXA88DngLKwOciYn/9wzMzs6U0Y1KQdClwPnAH8GWgAhwPbAc+nCaMP46Ie+sfppmZLYXZrhS+GhGXzvDaRyW1A9sWPyQzM2uUGZNCRHxu8jpJK4B1EfFERFRIrh7MzKxFzNnRLOnTkk6QtBYYAb4p6U/qH5qZmS21WkYfdUbEE8CbgFtImozeOtdOkp4vaUjSAUn7Jb0rXb9J0m2SvpX+3XhUZ2BmZoumlqSwStIqkqRwY0T8GIga9nuWpCP6xcArgF5JncAlwO0R8ULg9vS5mZk1gVqSwv8CHgLWAnsl/RTwxFw7RcTBiLg7fTwOHABOBt4IXJ1udjVJsjEzsyYwY1KQ9EpJioiPR8TJEfH6iAjgP4Ge+byJpFOBs0iGtp4UEQchSRxA+wz77JA0LGl4bGxsPm9nZmYLNNuVwsXAPknXSPodSVsAIvFsrW8gaR1wHfDutG+iJhFxRUR0R0R3oVCodTczMzsKsw1J/QMASS8CfhX4hKTnAkPAvwJfiojDsx087Yu4DvhURFyfrh6VtDUiDkraioe1mpk1jTn7FCLiGxFxWUS8DngNcBdQJGkKmpEkAVcCByLio1Uv3URyFUL698aFBG5mZotv1tpHGUkvA36eZNTRlyLiD2vY7RySoav3Sfp6uu79wIeBayW9k6R/ojjvqM3MrC7mTAqS/pzkiztr/rlKUiki/nK2/SLiLkAzvPxL84rSzMyWRC1XChcBZ0XEjwAkfRi4G5g1KZiZ2fJTyzyFh0iqo2aOAx6oSzRmZtZQs5XO/juSPoSngf2Sbkufn0fS2WxmZi1mtuaj4fTvPuCGqvV31C0aMzNrqNnmKVw902tm1lwGBwcpFou0tycFAiqVCqVSid7e3gZHZstNLaWzf13S1yQ9JukJSeOSap6ZbGb1NTg4SF9fHz09PVQqFSqVCj09PfT19TE4ONjo8GyZqaWj+WMkk8w2R8QJEbE+Ik6oc1xmDTE4OEil8pNJ9pVKpem/WIvFIp2dnYyMjNDV1UVXVxcjIyN0dnZSLHoakM1PLUNS/wsop8XwzFpW9ot7165dDA0NAdDT08PIyAhA0zbFtLe3MzQ0RFdXF1nxyEKhwNDQUN6cZFarWpLCnwK3SLqTZCQSAJNKV5gte8VikV27duW/uAHGxsb8i9uOKbU0H30I+G+SuQrrqxazlpL94i4UCoyNjTE2NrYsfnFnfQhZvFn8WR+D2XzUkhQ2RcRvRMRfRMQHsqXukZktscHBQSbfu+Opp56adftm6H8olUp5H0K5XKZcLud9DKVSacnjsWUuImZdSArY/fJc29VzOfvss8Osnnbu3BlAHHfccQHE5s2bo62tLYDYsmVLjI6OTrt9Z2dnjI6OxujoaHR2dgYQO3fubEj81TGOjo42JA5rLsBwzPP7tpakMA4cAZ4iuQ3nOPDEfN/oaBYnBTsatXxhjo6OxpYtWwKItra22Lx584QkMd32WRIoFApRKBQmJAmzZlCXpNAMi5OC1WK6L/9isVjTL/pisRh33nlnrFu3LkjKucSqVauiXC7nx50uMWTJAIh169ZFuVye8Lp/rVsjLWpSAE6ddcekLPYp833DhSxOCjaX2ZpzsiuAmX7RZ4lj9erVsXHjxvxLHohf+ZVfmTGRDAwM5FcU2dLe3t4UzUlmEQtLCkr2m0pSiaQj+kaS+kdjJCOQfhroIbknwl9ExG0L682oXXd3dwwPD8+9oR2zshE4IyMjZPf0zoaTXnvttfnoHEjG8JfLZdrb2xkcHOSMM87gvPPO45lnnply3BUrVrBmzRoOHTpER0cHe/fupVQq8e1vf5uPfexjAKxZs4aI4Ec/+hEAq1atYvXq1fk+F1xwAVu3bm3aeQ7WuiTti4juee0zU1JID9gJ/A+Su6htJRmaegC4BfhspPdYqDcnBatFpVKZMoGrXC4DcPrpp/Pkk0/m68866ywkceutt9LZ2cnZZ5/Nnj17JhxvxYoVHDlyJH++fft2XvCCF3Drrbciidn+7QCsXLmSE044gcceewyA888/n1e96lVccsklebyuT2T1tJCk0PD+gloWNx9ZLSa38RcKhejv74/29va8A3nTpk0TmntOPPHECc/rtaxatSp/3N/fH/39/XmzlpuX5ubRVQuDO5rtWDXTaKBsWb16dQCxYsWKJUkCsy3VMUw33NUmarbhv8vJQpJCLZPXzJredBO4Ojo68tfb2toA8uag448/ftrjLIUshra2Nr7whS809WzpZuCCf0urltpHZk0va5cfHx8HkpIVe/fu5W1vext33XUXhw4dmrB91incSBs2bMg7xW1mLvi3tGq5n8Lttawza7Q777yT973vfbz0pS9l//79fPzjH+fWW2+dkhCaxaOPPur6RNZ0ZrtH8/HAc4ATJW0kmZcAcALwvCWIzWxW2d3GSqUS4+PjlEolVqxYwSOPPMIZZ5yBpLkP0kBtbW15fSKPQJrZ5IJ/QF7wz1cLdTBTZwPwLuA7JOWyv1O13AP0zbfz4mgWdzTbZFnnYzaC5/TTT887k5fTUiwWG/2fsum5o3nhqFPtoz+c70EXe3FSsMmqvxiywnXLaTnttNNiYGBgwvn4C25mHpK6MAtJCrNOXgOQ9Lbp1kfEJ2fdcRF58tqxYb43n69UKhMmpS03nZ2dU+7wtnPnTjcl2aJZyOS1WkYf/WzV4+NJylvcDSxZUrDWt5BbYe7evXvZJoSVK1f6Dm/WlOa8Upiyg/RcYE9EvGGO7XYDvw5UIqIrXXcp8HskdZQA3h8Rt8z1nr5SaH2z1S6arjOxUqnw6le/mvvvv78R4S6KdevWTSi9kdVjMlssC7lSWMjktf8GXljDdp8AXjfN+ssi4sx0mTMh2LFhvrfCLJVK3H///WzatKkB0R699vZ21qxZ0+gwzKaoZZ7CzZJuSpfPAd8kqZw6q4jYCzy2CDHaMS677WX2t1KpcOWVVwLwgx/8YFl9ua5YkfyTq1QqvqeyNae5eqKBX6xazmEe91AATgXKVc8vBR4C7gV2Axtn2XcHMAwMb9u2bbE75a0JVI8oGR0dje3btwdMrF2UDTnN/nZ0dOTbNetSfaOe6kXShOcdHR0eYml1Rb0K4gFbgDcA5wNbaj741KRwEtBGcoXyIWB3LcfxkNTWM3ns+cDAQP5lOTAwMO1NcqqHnjZDYbtal7a2tik373nDG97gIZZWd3VJCsDvAv9J0kdwNckv/XfUdPBJSaHW1yYvTgqtZ6aqptmv52yb7GpictXT5ZAYJMXatWunJLR169a5MqotiYUkhVo6mv8EOCsificiLgbOBv6shv2mkLS16umbgfJCjmPL30wdy3v37s07ltvb22cds199A5xm9JznPIdbbrmF4447jsOHD7Nu3ToKhQJPPvmk+w+sadWSFB4GxquejwP/NddOkj4D/BvQIelhSe8E/lrSfZLuJbml53sWELMdQ6rr3mTlr4Gmr2sEcOjQId7ylrfw9NNPs2XLFh544AHK5XJeBrpUKjU6RLMpapm89l3gy5JuJLn8fSPwFUnvBYiIj063U0RcNM3qKxcaqLWWWoucZfdJ2LJlC4888ggdHR1ERFPPT8hu45nFXCwW2blzZ35OQ0NDLoJnTauWK4UHgH8mSQiQDEc9CKxPF7N5m+6mONP9gu7t7WXnzp3cc8897Ny5k71793L99dezZcsWgAlXD81i48aNDAwM5DFfe+21E+ZazNUsZtZQ8+2EaMTijubWVCwWo1wu58/L5XJNVUOrRy6Vy+X8HszNsGQdytmoKrNGoh4dzZK2S7pC0ucl/d9sOcpcZMe4wcFBSqUSF154YT4h7cILL6RUKjE4ODjrfllzzNDQEDfffDOVSoXt27fT39/P2rVrl/Aspjp8+DDt7e3uM7Blq5Y+hRLw98A/AofrG44dK4rFIrt27ZpXUbjpiubt2bMHgLe//e2sX7+eQ4cOISkb9lx3Wf9Bf39/XnrjPe95D+vXr3cTkS1Pc11KAPvme/mx2Iubj1rT5PkHhUJh1iaX6rkNmzdvnjC3IZvwlk10q17WrFmz6M1EmzZtCkhu7pPdF8ET0KzZsJjNR5I2SdoE3Czpf0ramq1L15stqfb2dt761rcCyf2Nx8bG2Lx5MxHB+973Pvr6+njkkUc4/fTTWb16db7fU089tWgxtLW1US6XOXDgAJ2dnTzwwAOsX78+j89XB7bszZQtSG69+SATb8WZLQ/ON/sczeIrhdYz04zmuTpoy+XyhNnBWT2hbL9sBnS5XJ5Sa2jHjh3zviJYtWrVhOcbN26cMuParFlRr9pHjV6cFFrPQu67W71N9Rd+W1tb9Pf3T9i2v79/yhf8iSeemH+xr1y5cs6EUL1N9ftt377dI4tsWahLUgB+Y5rll4D2+b7ZQhcnhdY03/vuZokEpq97lO07XUKoTgzZlUQtiSFLAuVyOTo6Oqa8l1kzW0hSqGX00TuBVwJD6fNzgX8Htkv6YETsqeEYZlNMbn+fq02+t7eXgwcP8jd/8zc888wzE0YZHXfccZx77rkA3Hvvvfk+GzZsYMWKFTz2WHJrjxNPPJH29nZKpRLPPvssW7ZsQRIHDx6c9j27urq4/fbbaW9vZ+/evezevdsji6yl1ZIUjgAvjohRAEknAZcDPwfsBZwUbMls3bqVZ555hra2Ng4fTkZIt7W18fTTT3PHHXfwkpe8hJtuuokzzjiDcrnM448/DiTJ4cc//jHf+MY38nkQAwMDvOMd7wDgtNNO49ChQ6xbt441a9YwNjbG+vXr84QASdK65JJLGnDWZkunljIXp2YJIVUBtkfEY8CP6xOW2fR6e3sZGBhgw4YN+boNGzYwMDAw4df7NddcM6EExg9/+EMOHTpEZ2cn4+Pj9PX15XMcdu/ezaFDhwDo7+/PS26Mj497Apodc5Rdfs+4gbQL2EYyiQ3gN0kqp/4J8C8R0VPXCIHu7u4YHh6u99vYMpAV0hsZGZlQSK+zszMvpFe9TXUTU1tbG/fccw+FQmHaY3R0dOSluyuViovW2bInaV9EdM9nn1quFHpJbrBzJnAW8EmgNyIOLUVCMKtWSyG9bJuOjg42bfrJlJrDhw9z8803L8q9HMxa1Zx9CmkP9mfTxayhsi/qYrE4Yynq3t5exsfHueqqq3j00UcpFAocOXKERx99lD179uT9CGY2VS0F8cYlPZEuP5J0WNITSxGc2XR6e3vnLEW9fv167r///vyKIru6GBkZYffu3RPu5ZBdMfhuaGa1XSlMuGeCpDcBL69bRGaLYLYrCiBPEllhvayPwf0Idqybs6N52p2kf4+IV9Qhnmm5o9kWW1aCO0sY7li2VrSQjuZaRh/9RtXTFUA38IsR8cr5h7gwTgpmZvO3kKRQy+S186sePws8RHKfZjMzazG19Cm8fSkCMTOzxqtl9NEpkm6QVJE0Kuk6SacsRXBmZra0apm8dhVwE/A84GTg5nSdmZm1mFqSQiEiroqIZ9PlE0ChznGZ2RIYHBycMDejUqnkBQPt2FRLUvi+pN+W1JYuvw08Wu/AzKy+BgcH6evryyftZTWj+vr6nBiOYbUkhXcAFwKPAAeBC9J1ZraMFYvFfJZ3V1cXXV1d+aS+YrHY6PCsQWYdfSSpDfjNiHjDEsVjZkskKwzY1dXF2NgYAIVCIa82a8emWa8UIuIwC5yTIGl3OmKpXLVuk6TbJH0r/btxIcc2M7P6qKX56EuSdkp6taSXZUsN+30CeN2kdZcAt0fEC4Hb0+dm1gBZH4ILA1q1WpLCq4CXAB8E/jZdPjLXThGxF3hs0uo3Alenj68G3lRzpGa2qGq5N4Ude2qZ0byYN9I5KSIOpsc9KGnGhktJO4AdANu2bVvEEMwMars3hR17aimIdxzJLThPpSqJRMQH5zy4dCrJLTu70uePR8SGqtd/EBFz9iu4IJ4tlKuh2rGsXgXxbgR+COwDnl5IYFVGJW1NrxK2Am64tLrJxuHv2rVryn0TACcGs2nUkhROiYjJHcYLdRNwMfDh9O+Ni3RcsymKxSK7du3Kx+EDjI2NeRy+2Sxq6Wj+f5LOmO+BJX0G+DegQ9LDkt5JkgzOk/Qt4Lz0uVldZOPws1E12Sgbj8M3m9mMVwrp/IIj6TZvl/QgSfORgIiIn5ntwBFx0Qwv/dICYzUzszqbrfnoZODMpQrEbLFNHocP5OPwfbVgNr3Zmo++ExH/MdOyZBGaLZDH4ZvN32xXCu2S3jvTixHx0TrEY7ZoPA7fbP5mnKcg6SBwOUkfwhQR8YE6xjWB5ymYmc3fYs9TOFjLBDUzM2sds/UpTHuFYGZmrWu2pOCho2Zmx5gZk0JETK5wamZmLa6WGc1mZnaMcFIwM7Ock4KZmeWcFMzMLOekYGZmOScFMzPLOSmYmVnOScHMzHJOCmZmlnNSMDOznJOCmZnlnBTMzCznpGBmZjknBTMzyzkpmJlZzknBzMxyTgpmZpZzUjAzs5yTgpmZ5VY24k0lPQSMA4eBZyOiuxFxmJnZRA1JCqmeiPh+A9/fzMwmcfORmZnlGpUUAvi8pH2SdjQoBjMzm6RRzUfnRMT3JLUDt0n6RkTsrd4gTRY7ALZt29aIGM3MjjkNuVKIiO+lfyvADcDLp9nmiojojojuQqGw1CGamR2TljwpSForaX32GPhloLzUcZhZ/Q0ODlKpVPLnlUqFwcHBBkZkc2lE89FJwA2Ssvf/dET8awPiMLM6GhwcpK+vj127djE0NARAT08PIyMjAPT29jYyPJuBIqLRMcypu7s7hoeHGx2Gmc1DpVLJk0DWBDw2NkZnZydDQ0O0t7c3OMLWJ2nffOeBeUiqmdVFe3s7Q0NDFAoFxsbGGBsbo1AoOCE0OScFMzPLOSmYWV1kzUfZFUJ2xdDT0zOh89mai5OCmdVFqVRiZGSEzs5OyuUy5XKZzs5ORkZGKJVKjQ7PZtDI2kdm1sKy0UXFYjHvQxgaGqJUKnnkURPz6CMzsxbl0UdmZnZUnBTMzCznpGBmZjknBTMzyzkpmJlZzknBzMxyTgpmZpZzUjAzs5yTgpmZ5ZwUzMws56RgZmY5JwUzM8s5KZiZWc5JwczMck4KZmaWc1IwM7Ock4KZmeWcFMzMLOekYGZmOScFMzPLOSmYmVnOScHMzHINSQqSXifpm5K+LemSRsRgZmZTLXlSkNQGDAK/CnQCF0nqXOo4zMxsqkZcKbwc+HZEPBgRzwDXAG9sQBxmZjbJyga858nAf1U9fxj4uckbSdoB7EifPi2pvASxNcqJwPcbHUQdtfL5tfK5gc9vueuY7w6NSAqaZl1MWRFxBXAFgKThiOiud2CN4vNbvlr53MDnt9xJGp7vPo1oPnoYeH7V81OA7zUgDjMzm6QRSeGrwAslvUDSauC3gJsaEIeZmU2y5M1HEfGspD7gVqAN2B0R++fY7Yr6R9ZQPr/lq5XPDXx+y928z08RU5rzzczsGOUZzWZmlnNSMDOzXFMnhVYvhyHpIUn3Sfr6QoaONRtJuyVVqueUSNok6TZJ30r/bmxkjEdjhvO7VNJ308/w65Je38gYj4ak50saknRA0n5J70rXL/vPcJZza4nPT9Lxkr4i6Z70/D6Qrn+BpC+nn90/pYN7Zj9Ws/YppOUw7gfOIxnG+lXgoogYaWhgi0jSQ0B3RLTE5BlJvwA8CXwyIrrSdX8NPBYRH04T+8aI+LNGxrlQM5zfpcCTEfGRRsa2GCRtBbZGxN2S1gP7gDcBv8My/wxnObcLaYHPT5KAtRHxpKRVwF3Au4D3AtdHxDWS/h64JyIun+1YzXyl4HIYy0xE7AUem7T6jcDV6eOrSf4hLksznF/LiIiDEXF3+ngcOEBSgWDZf4aznFtLiMST6dNV6RLAa4DPputr+uyaOSlMVw6jZT7EVACfl7QvLevRik6KiIOQ/MME2hscTz30Sbo3bV5adk0r05F0KnAW8GVa7DOcdG7QIp+fpDZJXwcqwG3AA8DjEfFsuklN36HNnBRqKoexzJ0TES8jqRjbmzZP2PJyOXA6cCZwEPjbxoZz9CStA64D3h0RTzQ6nsU0zbm1zOcXEYcj4kySKhEvB1483WZzHaeZk0LLl8OIiO+lfyvADSQfZKsZTdtzs3bdSoPjWVQRMZr+YzwC/APL/DNM26OvAz4VEdenq1viM5zu3Frt8wOIiMeBO4BXABskZZOUa/oObeak0NLlMCStTTu8kLQW+GWgFSvB3gRcnD6+GLixgbEsuuzLMvVmlvFnmHZWXgkciIiPVr207D/Dmc6tVT4/SQVJG9LHa4DXkvSbDAEXpJvV9Nk17egjgHR42Mf4STmMDzU4pEUj6TSSqwNIyo18ermfn6TPAOeSlCMeBf4C+GfgWmAb8J9AMSKWZWftDOd3LknTQwAPAb+ftb8vN5J+HvgicB9wJF39fpK292X9Gc5ybhfRAp+fpJ8h6UhuI/mxf21EfDD9nrkG2AR8DfjtiHh61mM1c1IwM7Ol1czNR2ZmtsScFMzMLOekYGZmOScFMzPLOSmYmVnOScGWPUlPzr1Vvu25kl5Vz3jmeP93S3rbIhznGkkvXIyYzKo5Kdix5lygIUkhnVn6DuDTi3C4y4E/XYTjmE3gpGAtSdL5aR35r0n6gqST0kJofwC8J62d/+p0Juh1kr6aLuek+1+aFki7Q9KDkv6o6thvSwuo3SNpj6T1kr6TllFA0glK7pWxalJYrwHuzgqUpce+TNLetM7/z0q6Pq19/5fpNmslfS59r7Kkt6TH+iLw2qoSBmaLwv9DWau6C3hFRISk3wX+NCL+OK0pn9fPl/Rp4LKIuEvSNuBWflJI7EVAD7Ae+Kaky4HtQD9JMcPvS9oUEeOS7gB+jWQG928B10XEjyfFdA5JHf9qz0TELyi56cuNwNkk5bkfkHQZyZXN9yLi19J4nwsQEUckfRt46TTHNFswJwVrVacA/5TWtlkNfGeG7V4LdCalcQA4IatJBXwuLQnwtKQKcBJpffrsxkhV5R7+kaQ555+BtwO/N817bSWpR1Mtq+d1H7A/K7Eg6UGSgpD3AR+R9FfAv0TEF6v2rQDPw0nBFpGbj6xV/R2wMyLOAH4fOH6G7VYAr4yIM9Pl5PQmLADVNWIOk/yIEtOUH46ILwGnSvpFoC0ipius9tQ0cWTvcWTS+x0BVkbE/SRXD/cBA5L+vGqb49Njmi0aJwVrVc8Fvps+vrhq/ThJc1Dm80Bf9kTSmXMc93bgQkmb0+03Vb32SeAzwFUz7HsA+Ok5I68i6XnAf0fE/wY+Arys6uXtwP75HM9sLk4K1gqeI+nhquW9wKVASdIXgep7YN8MvDnraAb+COhOOwC0G0AAAACeSURBVI5HSDqiZxQR+4EPAXdKugeoLjH9KWAjSWKYzv8B5nsjpTOAr6R31OoHsg7ok4CnlmNFT2turpJqtkgkXQC8MSLeOss2N5B0en/rKN/rPcATEXHl0RzHbDJ3NJstAkl/R3Jb1dfPseklJB3OR5UUgMeBPUd5DLMpfKVgZmY59ymYmVnOScHMzHJOCmZmlnNSMDOznJOCmZnl/j82THhKb7ZEMgAAAABJRU5ErkJggg==\n",
+ "text/plain": [
+ "
"
+ ]
+ },
+ "metadata": {
+ "needs_background": "light"
+ },
+ "output_type": "display_data"
+ }
+ ],
"source": [
"# Load data\n",
"data = loadmat(os.path.join('Data', 'ex8_movies.mat'))\n",
@@ -531,7 +669,7 @@
},
{
"cell_type": "code",
- "execution_count": null,
+ "execution_count": 12,
"metadata": {},
"outputs": [],
"source": [
@@ -610,7 +748,23 @@
" Theta_grad = np.zeros(Theta.shape)\n",
"\n",
" # ====================== YOUR CODE HERE ======================\n",
- "\n",
+ " J = (1 / 2) * np.sum(np.square((X.dot(Theta.T) - Y) * R)) + (lambda_ / 2) * np.sum(np.square(X)) + \\\n",
+ " (lambda_ / 2) * np.sum(np.square(Theta))\n",
+ " \n",
+ " for i in range(R.shape[0]):\n",
+ " \n",
+ " idx = np.where(R[i, :] == 1)[0]\n",
+ " Theta_temp = Theta[idx, :]\n",
+ " Y_temp = Y[i, idx]\n",
+ " X_grad[i, :] = np.dot(np.dot(X[i, :], Theta_temp.T) - Y_temp, Theta_temp) + lambda_ * X[i, :]\n",
+ " \n",
+ " for j in range(R.shape[1]):\n",
+ " \n",
+ " idx = np.where(R[:, j] == 1)[0]\n",
+ " X_temp = X[idx, :]\n",
+ " Y_temp = Y[idx, j]\n",
+ " Theta_grad[j, :] = np.dot(np.dot(X_temp, Theta[j, :]) - Y_temp, X_temp) + lambda_ * Theta[j, :]\n",
+ " \n",
" \n",
" \n",
" # =============================================================\n",
@@ -628,9 +782,18 @@
},
{
"cell_type": "code",
- "execution_count": null,
+ "execution_count": 13,
"metadata": {},
- "outputs": [],
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "Cost at loaded parameters: 22.22 \n",
+ "(this value should be about 22.22)\n"
+ ]
+ }
+ ],
"source": [
"# Load pre-trained weights (X, Theta, num_users, num_movies, num_features)\n",
"data = loadmat(os.path.join('Data', 'ex8_movieParams.mat'))\n",
@@ -663,9 +826,31 @@
},
{
"cell_type": "code",
- "execution_count": null,
+ "execution_count": 14,
"metadata": {},
- "outputs": [],
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "\n",
+ "Submitting Solutions | Programming Exercise anomaly-detection-and-recommender-systems\n",
+ "\n",
+ "Use token from last successful submission (tanmayss2109@gmail.com)? (Y/n): y\n",
+ " Part Name | Score | Feedback\n",
+ " --------- | ----- | --------\n",
+ " Estimate Gaussian Parameters | 15 / 15 | Nice work!\n",
+ " Select Threshold | 15 / 15 | Nice work!\n",
+ " Collaborative Filtering Cost | 20 / 20 | Nice work!\n",
+ " Collaborative Filtering Gradient | 0 / 30 | \n",
+ " Regularized Cost | 0 / 10 | \n",
+ " Regularized Gradient | 0 / 10 | \n",
+ " --------------------------------\n",
+ " | 50 / 100 | \n",
+ "\n"
+ ]
+ }
+ ],
"source": [
"grader[3] = cofiCostFunc\n",
"grader.grade()"
@@ -724,9 +909,48 @@
},
{
"cell_type": "code",
- "execution_count": null,
+ "execution_count": 15,
"metadata": {},
- "outputs": [],
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "[[ 0.46660851 0.46660851]\n",
+ " [-1.11522931 -1.11522931]\n",
+ " [-2.93605559 -2.93605559]\n",
+ " [-0.54310538 -0.54310538]\n",
+ " [-7.09376845 -7.09376845]\n",
+ " [-8.86613311 -8.86613311]\n",
+ " [ 0.08789442 0.08789442]\n",
+ " [-0.13815802 -0.13815802]\n",
+ " [ 0.01326013 0.01326013]\n",
+ " [ 0. 0. ]\n",
+ " [ 0. 0. ]\n",
+ " [ 0. 0. ]\n",
+ " [-0.75793651 -0.75793651]\n",
+ " [ 0.50622184 0.50622184]\n",
+ " [ 1.22902557 1.22902557]\n",
+ " [-0.32099776 -0.32099776]\n",
+ " [ 0.22352602 0.22352602]\n",
+ " [ 0.76455842 0.76455842]\n",
+ " [-1.25906737 -1.25906737]\n",
+ " [ 0.74762981 0.74762981]\n",
+ " [ 2.57050722 2.57050722]\n",
+ " [ 0. 0. ]\n",
+ " [ 0. 0. ]\n",
+ " [ 0. 0. ]\n",
+ " [-1.93903986 -1.93903986]\n",
+ " [ 1.25368512 1.25368512]\n",
+ " [ 3.37886598 3.37886598]]\n",
+ "\n",
+ "The above two columns you get should be very similar.(Left-Your Numerical Gradient, Right-Analytical Gradient)\n",
+ "If your cost function implementation is correct, then the relative difference will be small (less than 1e-9).\n",
+ "\n",
+ "Relative Difference: 1.01587e-12\n"
+ ]
+ }
+ ],
"source": [
"# Check gradients by running checkcostFunction\n",
"utils.checkCostFunction(cofiCostFunc)"
@@ -741,9 +965,31 @@
},
{
"cell_type": "code",
- "execution_count": null,
+ "execution_count": 16,
"metadata": {},
- "outputs": [],
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "\n",
+ "Submitting Solutions | Programming Exercise anomaly-detection-and-recommender-systems\n",
+ "\n",
+ "Use token from last successful submission (tanmayss2109@gmail.com)? (Y/n): y\n",
+ " Part Name | Score | Feedback\n",
+ " --------- | ----- | --------\n",
+ " Estimate Gaussian Parameters | 15 / 15 | Nice work!\n",
+ " Select Threshold | 15 / 15 | Nice work!\n",
+ " Collaborative Filtering Cost | 20 / 20 | Nice work!\n",
+ " Collaborative Filtering Gradient | 30 / 30 | Nice work!\n",
+ " Regularized Cost | 0 / 10 | \n",
+ " Regularized Gradient | 0 / 10 | \n",
+ " --------------------------------\n",
+ " | 80 / 100 | \n",
+ "\n"
+ ]
+ }
+ ],
"source": [
"grader[4] = cofiCostFunc\n",
"grader.grade()"
@@ -768,9 +1014,18 @@
},
{
"cell_type": "code",
- "execution_count": null,
+ "execution_count": 17,
"metadata": {},
- "outputs": [],
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "Cost at loaded parameters (lambda = 1.5): 31.34\n",
+ " (this value should be about 31.34)\n"
+ ]
+ }
+ ],
"source": [
"# Evaluate cost function\n",
"J, _ = cofiCostFunc(np.concatenate([X.ravel(), Theta.ravel()]),\n",
@@ -789,9 +1044,31 @@
},
{
"cell_type": "code",
- "execution_count": null,
+ "execution_count": 18,
"metadata": {},
- "outputs": [],
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "\n",
+ "Submitting Solutions | Programming Exercise anomaly-detection-and-recommender-systems\n",
+ "\n",
+ "Use token from last successful submission (tanmayss2109@gmail.com)? (Y/n): y\n",
+ " Part Name | Score | Feedback\n",
+ " --------- | ----- | --------\n",
+ " Estimate Gaussian Parameters | 15 / 15 | Nice work!\n",
+ " Select Threshold | 15 / 15 | Nice work!\n",
+ " Collaborative Filtering Cost | 20 / 20 | Nice work!\n",
+ " Collaborative Filtering Gradient | 30 / 30 | Nice work!\n",
+ " Regularized Cost | 10 / 10 | Nice work!\n",
+ " Regularized Gradient | 0 / 10 | \n",
+ " --------------------------------\n",
+ " | 90 / 100 | \n",
+ "\n"
+ ]
+ }
+ ],
"source": [
"grader[5] = cofiCostFunc\n",
"grader.grade()"
@@ -821,9 +1098,48 @@
},
{
"cell_type": "code",
- "execution_count": null,
+ "execution_count": 19,
"metadata": {},
- "outputs": [],
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "[[-2.10413092 -2.10413092]\n",
+ " [ 4.88435471 4.88435471]\n",
+ " [-5.14676934 -5.14676934]\n",
+ " [ 2.10807904 2.10807904]\n",
+ " [-0.69618007 -0.69618007]\n",
+ " [ 6.10717258 6.10717258]\n",
+ " [-0.98375746 -0.98375746]\n",
+ " [-1.62145317 -1.62145317]\n",
+ " [-3.64294105 -3.64294105]\n",
+ " [ 1.86940783 1.86940783]\n",
+ " [ 0.97383575 0.97383575]\n",
+ " [ 3.67349576 3.67349576]\n",
+ " [-0.81407458 -0.81407458]\n",
+ " [ 3.7746785 3.7746785 ]\n",
+ " [ 2.48789327 2.48789327]\n",
+ " [-1.81152765 -1.81152765]\n",
+ " [-1.36691463 -1.36691463]\n",
+ " [-3.72595908 -3.72595908]\n",
+ " [ 0.38573866 0.38573866]\n",
+ " [ 0.97208917 0.97208917]\n",
+ " [ 0.24478964 0.24478964]\n",
+ " [ 2.57555678 2.57555678]\n",
+ " [-4.32021577 -4.32021577]\n",
+ " [ 5.33165654 5.33165654]\n",
+ " [-1.51004106 -1.51004106]\n",
+ " [-1.77104474 -1.77104474]\n",
+ " [ 0.50360741 0.50360741]]\n",
+ "\n",
+ "The above two columns you get should be very similar.(Left-Your Numerical Gradient, Right-Analytical Gradient)\n",
+ "If your cost function implementation is correct, then the relative difference will be small (less than 1e-9).\n",
+ "\n",
+ "Relative Difference: 2.53851e-12\n"
+ ]
+ }
+ ],
"source": [
"# Check gradients by running checkCostFunction\n",
"utils.checkCostFunction(cofiCostFunc, 1.5)"
@@ -838,9 +1154,31 @@
},
{
"cell_type": "code",
- "execution_count": null,
+ "execution_count": 20,
"metadata": {},
- "outputs": [],
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "\n",
+ "Submitting Solutions | Programming Exercise anomaly-detection-and-recommender-systems\n",
+ "\n",
+ "Use token from last successful submission (tanmayss2109@gmail.com)? (Y/n): y\n",
+ " Part Name | Score | Feedback\n",
+ " --------- | ----- | --------\n",
+ " Estimate Gaussian Parameters | 15 / 15 | Nice work!\n",
+ " Select Threshold | 15 / 15 | Nice work!\n",
+ " Collaborative Filtering Cost | 20 / 20 | Nice work!\n",
+ " Collaborative Filtering Gradient | 30 / 30 | Nice work!\n",
+ " Regularized Cost | 10 / 10 | Nice work!\n",
+ " Regularized Gradient | 10 / 10 | Nice work!\n",
+ " --------------------------------\n",
+ " | 100 / 100 | \n",
+ "\n"
+ ]
+ }
+ ],
"source": [
"grader[6] = cofiCostFunc\n",
"grader.grade()"
@@ -857,9 +1195,29 @@
},
{
"cell_type": "code",
- "execution_count": null,
+ "execution_count": 21,
"metadata": {},
- "outputs": [],
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "New user ratings:\n",
+ "-----------------\n",
+ "Rated 4 stars: Toy Story (1995)\n",
+ "Rated 3 stars: Twelve Monkeys (1995)\n",
+ "Rated 5 stars: Usual Suspects, The (1995)\n",
+ "Rated 4 stars: Outbreak (1995)\n",
+ "Rated 5 stars: Shawshank Redemption, The (1994)\n",
+ "Rated 3 stars: While You Were Sleeping (1995)\n",
+ "Rated 5 stars: Forrest Gump (1994)\n",
+ "Rated 2 stars: Silence of the Lambs, The (1991)\n",
+ "Rated 4 stars: Alien (1979)\n",
+ "Rated 5 stars: Die Hard 2 (1990)\n",
+ "Rated 5 stars: Sphere (1998)\n"
+ ]
+ }
+ ],
"source": [
"# Before we will train the collaborative filtering model, we will first\n",
"# add ratings that correspond to a new user that we just observed. This\n",
@@ -915,9 +1273,17 @@
},
{
"cell_type": "code",
- "execution_count": null,
+ "execution_count": 22,
"metadata": {},
- "outputs": [],
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "Recommender system learning completed.\n"
+ ]
+ }
+ ],
"source": [
"# Now, you will train the collaborative filtering model on a movie rating \n",
"# dataset of 1682 movies and 943 users\n",
@@ -978,9 +1344,42 @@
},
{
"cell_type": "code",
- "execution_count": null,
+ "execution_count": 23,
"metadata": {},
- "outputs": [],
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "Top recommendations for you:\n",
+ "----------------------------\n",
+ "Predicting rating 5.0 for movie Aiqing wansui (1994)\n",
+ "Predicting rating 5.0 for movie Great Day in Harlem, A (1994)\n",
+ "Predicting rating 5.0 for movie Someone Else's America (1995)\n",
+ "Predicting rating 5.0 for movie Entertaining Angels: The Dorothy Day Story (1996)\n",
+ "Predicting rating 5.0 for movie Santa with Muscles (1996)\n",
+ "Predicting rating 5.0 for movie They Made Me a Criminal (1939)\n",
+ "Predicting rating 5.0 for movie Star Kid (1997)\n",
+ "Predicting rating 5.0 for movie Marlene Dietrich: Shadow and Light (1996)\n",
+ "Predicting rating 5.0 for movie Saint of Fort Washington, The (1993)\n",
+ "Predicting rating 5.0 for movie Prefontaine (1997)\n",
+ "\n",
+ "Original ratings provided:\n",
+ "--------------------------\n",
+ "Rated 4 for Toy Story (1995)\n",
+ "Rated 3 for Twelve Monkeys (1995)\n",
+ "Rated 5 for Usual Suspects, The (1995)\n",
+ "Rated 4 for Outbreak (1995)\n",
+ "Rated 5 for Shawshank Redemption, The (1994)\n",
+ "Rated 3 for While You Were Sleeping (1995)\n",
+ "Rated 5 for Forrest Gump (1994)\n",
+ "Rated 2 for Silence of the Lambs, The (1991)\n",
+ "Rated 4 for Alien (1979)\n",
+ "Rated 5 for Die Hard 2 (1990)\n",
+ "Rated 5 for Sphere (1998)\n"
+ ]
+ }
+ ],
"source": [
"p = np.dot(X, Theta.T)\n",
"my_predictions = p[:, 0] + Ymean\n",
@@ -1001,6 +1400,13 @@
" if my_ratings[i] > 0:\n",
" print('Rated %d for %s' % (my_ratings[i], movieList[i]))"
]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": []
}
],
"metadata": {
@@ -1019,7 +1425,7 @@
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
- "version": "3.6.8"
+ "version": "3.7.6"
}
},
"nbformat": 4,
diff --git a/Phase 3 - 2020 (Summer)/Week 8/Exercise8/token.pkl b/Phase 3 - 2020 (Summer)/Week 8/Exercise8/token.pkl
new file mode 100644
index 000000000..120de049e
Binary files /dev/null and b/Phase 3 - 2020 (Summer)/Week 8/Exercise8/token.pkl differ
diff --git a/Phase 3 - 2020 (Summer)/Week 8/__pycache__/submission.cpython-37.pyc b/Phase 3 - 2020 (Summer)/Week 8/__pycache__/submission.cpython-37.pyc
new file mode 100644
index 000000000..51a42da4c
Binary files /dev/null and b/Phase 3 - 2020 (Summer)/Week 8/__pycache__/submission.cpython-37.pyc differ
diff --git a/Phase 3 - 2020 (Summer)/Week_3/Tanmay_190106065/Week_3.ipynb b/Phase 3 - 2020 (Summer)/Week_3/Tanmay_190106065/Week_3.ipynb
new file mode 100644
index 000000000..cd2adce89
--- /dev/null
+++ b/Phase 3 - 2020 (Summer)/Week_3/Tanmay_190106065/Week_3.ipynb
@@ -0,0 +1,1145 @@
+{
+ "cells": [
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "# Programming Exercise 2: Logistic Regression\n",
+ "\n",
+ "## Introduction\n",
+ "\n",
+ "In this exercise, you will implement logistic regression and apply it to two different datasets. Before starting on the programming exercise, we strongly recommend watching the video lectures and completing the review questions for the associated topics.\n",
+ "\n",
+ "All the information you need for solving this assignment is in this notebook, and all the code you will be implementing will take place within this notebook. The assignment can be promptly submitted to the coursera grader directly from this notebook (code and instructions are included below).\n",
+ "\n",
+ "Before we begin with the exercises, we need to import all libraries required for this programming exercise. Throughout the course, we will be using [`numpy`](http://www.numpy.org/) for all arrays and matrix operations, and [`matplotlib`](https://matplotlib.org/) for plotting. In this assignment, we will also use [`scipy`](https://docs.scipy.org/doc/scipy/reference/), which contains scientific and numerical computation functions and tools. \n",
+ "\n",
+ "You can find instructions on how to install required libraries in the README file in the [github repository](https://github.com/dibgerge/ml-coursera-python-assignments)."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 1,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "# used for manipulating directory paths\n",
+ "import os\n",
+ "\n",
+ "# Scientific and vector computation for python\n",
+ "import numpy as np\n",
+ "\n",
+ "# Plotting library\n",
+ "from matplotlib import pyplot as plt\n",
+ "\n",
+ "# Optimization module in scipy\n",
+ "from scipy import optimize\n",
+ "\n",
+ "# library written for this exercise providing additional functions for assignment submission, and others\n",
+ "import utils\n",
+ "\n",
+ "# define the submission/grader object for this exercise\n",
+ "#grader = utils.Grader()\n",
+ "\n",
+ "# tells matplotlib to embed plots within the notebook\n",
+ "%matplotlib inline"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "## Submission and Grading\n",
+ "\n",
+ "\n",
+ "After completing each part of the assignment, be sure to submit your solutions to the grader. The following is a breakdown of how each part of this exercise is scored.\n",
+ "\n",
+ "\n",
+ "| Section | Part | Submission function | Points \n",
+ "| :- |:- | :- | :-:\n",
+ "| 1 | [Sigmoid Function](#section1) | [`sigmoid`](#sigmoid) | 5 \n",
+ "| 2 | [Compute cost for logistic regression](#section2) | [`costFunction`](#costFunction) | 30 \n",
+ "| 3 | [Gradient for logistic regression](#section2) | [`costFunction`](#costFunction) | 30 \n",
+ "| 4 | [Predict Function](#section4) | [`predict`](#predict) | 5 \n",
+ "| 5 | [Compute cost for regularized LR](#section5) | [`costFunctionReg`](#costFunctionReg) | 15 \n",
+ "| 6 | [Gradient for regularized LR](#section5) | [`costFunctionReg`](#costFunctionReg) | 15 \n",
+ "| | Total Points | | 100 \n",
+ "\n",
+ "\n",
+ "\n",
+ "You are allowed to submit your solutions multiple times, and we will take only the highest score into consideration.\n",
+ "\n",
+ "
\n",
+ "At the end of each section in this notebook, we have a cell which contains code for submitting the solutions thus far to the grader. Execute the cell to see your score up to the current section. For all your work to be submitted properly, you must execute those cells at least once. They must also be re-executed everytime the submitted function is updated.\n",
+ "
\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "## 1 Logistic Regression\n",
+ "\n",
+ "In this part of the exercise, you will build a logistic regression model to predict whether a student gets admitted into a university. Suppose that you are the administrator of a university department and\n",
+ "you want to determine each applicant’s chance of admission based on their results on two exams. You have historical data from previous applicants that you can use as a training set for logistic regression. For each training example, you have the applicant’s scores on two exams and the admissions\n",
+ "decision. Your task is to build a classification model that estimates an applicant’s probability of admission based the scores from those two exams. \n",
+ "\n",
+ "The following cell will load the data and corresponding labels:"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 2,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "# Load data\n",
+ "# The first two columns contains the exam scores and the third column\n",
+ "# contains the label.\n",
+ "data = np.loadtxt(os.path.join('data1.txt'), delimiter=',')\n",
+ "X, y = data[:, 0:2], data[:, 2]"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "### 1.1 Visualizing the data\n",
+ "\n",
+ "Before starting to implement any learning algorithm, it is always good to visualize the data if possible. We display the data on a 2-dimensional plot by calling the function `plotData`. You will now complete the code in `plotData` so that it displays a figure where the axes are the two exam scores, and the positive and negative examples are shown with different markers.\n",
+ "\n",
+ "To help you get more familiar with plotting, we have left `plotData` empty so you can try to implement it yourself. However, this is an optional (ungraded) exercise. We also provide our implementation below so you can\n",
+ "copy it or refer to it. If you choose to copy our example, make sure you learn\n",
+ "what each of its commands is doing by consulting the `matplotlib` and `numpy` documentation.\n",
+ "\n",
+ "```python\n",
+ "# Find Indices of Positive and Negative Examples\n",
+ "pos = y == 1\n",
+ "neg = y == 0\n",
+ "\n",
+ "# Plot Examples\n",
+ "pyplot.plot(X[pos, 0], X[pos, 1], 'k*', lw=2, ms=10)\n",
+ "pyplot.plot(X[neg, 0], X[neg, 1], 'ko', mfc='y', ms=8, mec='k', mew=1)\n",
+ "```"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 3,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "def plotData(X, y):\n",
+ " \"\"\"\n",
+ " Plots the data points X and y into a new figure. Plots the data \n",
+ " points with * for the positive examples and o for the negative examples.\n",
+ " \n",
+ " Parameters\n",
+ " ----------\n",
+ " X : array_like\n",
+ " An Mx2 matrix representing the dataset. \n",
+ " \n",
+ " y : array_like\n",
+ " Label values for the dataset. A vector of size (M, ).\n",
+ " \n",
+ " Instructions\n",
+ " ------------\n",
+ " Plot the positive and negative examples on a 2D plot, using the\n",
+ " option 'k*' for the positive examples and 'ko' for the negative examples. \n",
+ " \"\"\"\n",
+ " # Create New Figure\n",
+ " fig = plt.figure()\n",
+ "\n",
+ " # ====================== YOUR CODE HERE ======================\n",
+ " pos = y == 1\n",
+ " neg = y == 0\n",
+ " \n",
+ " plt.plot(X[pos,0], X[pos,1], 'k*', lw=2, ms=10)\n",
+ " plt.plot(X[neg,0], X[neg,1], 'ko', mfc='y', ms=8, mec='k', mew=1)\n",
+ " \n",
+ " # ============================================================"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Now, we call the implemented function to display the loaded data:"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 4,
+ "metadata": {},
+ "outputs": [
+ {
+ "data": {
+ "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYUAAAEGCAYAAACKB4k+AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjMsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+AADFEAAAgAElEQVR4nO2de3hU5bXwf2sGyBCDQSC1UqJQoV65SWyl7YGEYKVfOQXb2gO13o6XeuspDfVyKocYxK9VqLY+52gLUpG0X/Bo2wge28MdrNrWYL1bSy2ICMpFiaTIJWF9f8yekISZZGYysy8z6/c877Nn7z177zV7ZvZ63/Wui6gqhmEYhgEQ8loAwzAMwz+YUjAMwzBaMaVgGIZhtGJKwTAMw2jFlIJhGIbRSg+vBegOAwYM0MGDB3sthmEYRqDYuHHjblUtibcv0Eph8ODBNDQ0eC2GYRhGoBCRtxLtM/ORYRiG0YopBcMwDKOVrCkFEfm5iOwUkVfabOsnIitFZJOzPMHZLiJyn4j8TUReEpFzsiWXYRiGkZhsjhQWA5M6bLsVWK2qw4DVzjrAF4FhTrsGeCCLchmGYRgJyJpSUNUNwPsdNk8BHnZePwxMbbN9iUb5A9BXRE7KlmzdoampiTlzqiktLSEcDlFaWsKcOdU0NTV5LZphGEa3cdv76ERV3QGgqjtE5GPO9k8Ab7d53zZn246OJxCRa4iOJjj55JOzK20HmpqaKC8/j+LiN6muPsCQIbB5827q6u5m2bJfsW7dHygqKnJVJsMwjEzil4lmibMtbvpWVV2gqmWqWlZSEtfNNmvcc888iovfZNasAwwdCuEwDB0Ks2YdoLj4Te65Z56r8hiGYWQat5XCezGzkLPc6WzfBpS2ed8gYLvLsnXJwoX3M336AaSDChOBadMO8OCD2Z0KaWxs5MILL6SxsTGr1zH8hX3vhpu4rRSWAZc5ry8DHm+z/VLHC+k8oDFmZvIT27fvYciQ+PuGDInuzybLli2jvr6e5cuXZ/U6hr+w791wk2y6pNYBzwKnicg2EbkS+CFwvohsAs531gGeBP4O/A1YCFyfLbm6w8CB/dm8Of6+zZuj+7PJz3/+83ZLwx287qnb9264STa9j6ar6kmq2lNVB6nqIlXdo6qVqjrMWb7vvFdV9QZVPVVVh6uqL3NXXH319dTVRehYrE4Vli6NcNVV12X0ehMnTkREWtszzzwDwNNPP91u+8SJEzN63XynoxJwu6cetO/da6VpZBa/TDQHgqqqm2hsPJW5cyNs2gTNzbBpE8ydG6Gx8VSqqm7K6PVuu+02CgsLW9cPHTrUbglQWFjIrFmzMnrdfKejEnC7p56J793NB7WZt3IMVQ1sGzNmjLrNvn37tKZmtpaWlmg4HNLS0hKtqZmt+/bty8r11qxZo4WFhUrUG6tdKyws1LVr12bluvlM3759293nXr16tVvGWmVlZdZk6O73vmTJEgW0trY2azLGKC8vV0ArKiqyfi0jMwANmuC56vmDvTvNC6XgBcuXL9dIJNLuwRCJRHT58uVei5YTVFZWtru3PXr0iPswdlshd+d7z+aDuuP98kJpGt2jM6Vg5qME+Clyee/evfTo0YNQKETv3r0JhUL06NGDvXv3ui5LLtLRXNPc3Nzp+wsLC/mf//kfysvLsypXKt+7m/MQZtbMcRJpiyC0bI0U9u3bp2PGnKUTJkR04UJ01Sp04UJ0woSIjhlzVtZMRYkoLy/XUCiko0eP1hUrVujo0aM1FArZcD2DdGauwaMRWirfezLyZ3J0Y2bNYIONFFLDb5HLxcXFzJs3j4aGBs4//3yee+457r77bo4//nhX5chlKioqeOSRR4hEIsfs82qElsr3XlFRwRNPPNGuB9+WTI9uEt2vSCTCI488kvVRlJFFEmmLILRsjRQGDRqgCxeia9ce2xYsQEtLS7JyXcNbamtrtaioSEOhkPbu3bu153vKKacEZoTm5vxTx/sVCoW0qKjIlclto3tgI4XU8Dpy2fCGRYsWsX//fkaOHMnjjz/e2iMfMmRIYEZobs4/dbxfI0eOZP/+/RZkF3BMKcTB68hlwxs6mmvef/995s+fT3FxMQDhcJiZM2dSX1/vsaSJcfNBbWbN3ESiI4lgUlZWpg0NmQ9+njOnmvXr72bWrPbJ71SjgWrjx9/M7Nk1Gb+uYSRLY2Mjl19+OYsXL25VWgBTp05l3LhxzJgxg1AoREtLCz/+8Y956qmnfK3MDHcRkY2qWhZ3nymFY2lbN2HatFjdhGgqi8bGU61uguE5tbW1XHrppdTW1vLNb37Ta3GMgNGZUjDzURyKiopYt+4PjB9/M3fcUcKkSSHuuKOE8eNvNoVg+AJLkmdkCxspGEYAmDhxIqtXr25d79WrF4cOHWpdxqisrGTVqlVeiBiXRGYuw1tspGAYASeoUcSWLC94mFIwjADgdnBapjAzV/AwpWAYASEIUcSJcjCtW7fOl7UgjGMxpWAYAcLvyRETmbnazl360cxlHMWUgmEEiGwGp2WiME9QzVzGUUwpGEaAyGYUcaYmhe+88072798fd9/+/fupqKgwE5KPMZdUw8hj2rqMTp06lXXr1lFRUcGaNWvSPufatWuZNGlSO8+ojtiIwVt855IqIt8RkVdE5FURmeFs6yciK0Vkk7M8wQvZDH9gxeCzz8SJE+nbty/19fX07ds3Y4V5KioqOOOMMxLuN4Xgb1xXCiJyNnA18GlgJDBZRIYBtwKrVXUYsNpZN/IU82/PPrfddhuh0NFHQCZjHwYPHsyVV155jKdUKBTyjaeUER8vRgpnAH9Q1f2q2gysBy4EpgAPO+95GJjqgWxGB7zqsZt/e3Zo6zI6YcKEdkqhI93p0dfX11NeXn6Mp1RhYaFvPKVSIZ9Grl4ohVeAcSLSX0QKgf8DlAInquoOAGf5sXgHi8g1ItIgIg27du1yTeh8xa0eu5s1hvOZZOtR9+zZs9s9+lyqt5BPI1fXlYKqvg7cBawEfge8CHReKb398QtUtUxVy0pKSrIkpRHDrR57UNM4BI2uXEYBRISCgoJu9+hzqd5CXo1cE5Vkc6sB/xe4HngDOMnZdhLwRlfHZqscZz5TWVnZrpRjr1692i1jrbKyMuPXtmLw7hGvbKeI6Jw5c3T06NEqIjpgwADdu3ev16K6zt69e3XAgAGe/Q/cAL+V4xSRjznLk4GvAHXAMuAy5y2XAY97IVuQyYTd08seexDSOOQKbSOjQ6EQIsJxxx3HkCFDeO655/iXf/kXdu/enRfmko4sW7aM3bt306tXr9Zt+TRy9Sp47Vci8hqwHLhBVT8AfgicLyKbgPOddSMFMmH39Doi1e9pHHKFtvb+3/3ud4waNarV3h8Oh3n33XeBPDGXdCD2mc8880xP/geeT2onGkIEoZn5qD3l5eUKaEVFRbfPFc+8EIlEdPny5RmQNDHl5eUaCoV09OjRumLFCh09erSGQqGMfCbjKFOmTNEf/ehH2tLSoqqqEyZMyGlzSWckazJ163+wZMkSBbS2tjZr18Bv5iMjM2TTY8erHrsXk5Oe98w8oL6+nqqqqlaX1FmzZuXtRH8yJlOIFjZy43/g+aR2Im0RhJbvI4XOJmZjLd0J2nzqsbvRMwsCy5Yt03A4nJcT/V39l4YOHZq1/4EXzh10MlLw/MHenZbvSkE1ex47Hc0Lzc3NOn/+fJ0yZUoGpfcHmTS7ZYK9e/fq1KlTXff8iSnHnj17um429APxTKahUEivvPLKrP4Pstm5S4QphRzHK/t/UPHS7TYZvBq5xJRjOBzWUCikvXv31lAopEVFRXkxiqqtrdWioiJPPrvb7tidKQWbU8gBzGMnNfweKOeWTbm8vDzunFRLSwtHjhzho48+4siRIzQ1NeWFF5KXEdh+csc2pZAD5FI6ATfw2u22I16l+Bg7dmy79XiprgsLC7n22msDGYWcKl5HYPumc5doCBGEZuajKPlk/88kfjG7eWFTVj1qLgqFQnk5uew33HTuwOYUDONYvLQhd8QNm3KiuZQePXq47otvHIubnTtTCoYRB7+53WZ75JLMiASiOZDyZXI5X+lMKdicgpG3eG1D7ki2bcpdzaWICPPmzWuX8sLIQxJpiyA0GykYuYRbI5d4I5IePXro448/rqo2J5UPYCMFw/A/bo1c4o1IIpEIH374IQDhcJiZM2dSX1+f0esawUCiSiOYlJWVaUNDg9diGEagqKioYMOGDYwcOZK77rqLW265hRdffJHx48ezZs0ar8UzXEBENqpqWbx9NlIwjDzDb3Mphr+wkYJhGEaeYSMFw8gy+Zh+28hNTCkYRgbIRNU7w/ADphQMIwN4XhjFMDKEKQXDSAOvktgZRrYxpWAYaeD39NuGkS6eKAUR+a6IvCoir4hInYhERGSIiPxRRDaJyCMi0ssL2QwjGfyWftswMoXrSkFEPgH8G1CmqmcDYWAacBdwr6oOAz4ArnRbNsNIBT8VRjGMTOGV+agH0FtEegCFwA5gAvCYs/9hYKpHsgWKpqYm5sypprS0hHA4RGlpCXPmVNPU1OS1aHmBbwqjYG6xRmZwXSmo6jvAfGArUWXQCGwE9qpqs/O2bcAn4h0vIteISIOINOzatcsNkX1LU1MT5eXnsX793VRX72bFCqW6ejfr199Nefl5phhcwE9V78wt1sgEXpiPTgCmAEOAgcBxwBfjvDVuqLWqLlDVMlUtKykpyZ6gAeCee+ZRXPwms2YdYOhQCIdh6FCYNesAxcVvcs8987wWMeeJpYxYvXo1999/P6tWrfIsZYS5xRqZwAvz0URgs6ruUtXDwK+BzwJ9HXMSwCBguweypYTXppuFC+9n+vQDiLTfLgLTph3gwQcfcEWOfKa+vp6qqiqeeOIJ6uvrefLJJ13LMGpusUY28EIpbAXOE5FCERGgEngNWAt8zXnPZcDjHsiWNH4w3WzfvochQ+LvGzIkut9wBy966eYW60+CPrfjxZzCH4lOKD8PvOzIsAC4BagSkb8B/YFFbsuWCn4w3Qwc2J/Nm+Pv27w5ut/IDn7opZtbrD8J+tyOJ95Hqlqtqqer6tmqeomqHlTVv6vqp1V1qKpepKoHvZAtWfxgurn66uupq4vQMdGtKixdGuGqq67Lugz5il966bngFhv0nnVHgj63YxHNaeIH001V1U00Np7K3LkRNm2C5mbYtAnmzo3Q2HgqVVU3ZV2GfMVPvXQ/ucWmQ9B71n4YNWYSUwppkk3TTbIT2EVFRaxb9wfGj7+ZO+4oYdKkEHfcUcL48Tezbt0fKCoqSluG7uLH3l+mZfJLL91PbrHpEPSetV9GjZnClEKaZMt0k+oEdlFREbNn17B1606am1vYunUns2fXeKoQwJ+9v2zI5Ideul8qqSWrdHOtZ+2nUWNGUNXAtjFjxqhX7Nu3T8eMOUsnTIjoggXoypXoggXohAkRHTPmLN23b19a562pma0TJkR0zRp07dqjbc2a6LlramZn+JNkh/LycgW0oqLCa1FayYZM5eXlGgqFdPTo0bpixQodPXq0hkIhX31ut1iyZIkCWltb2+n71qxZo4WFhUo0FiluKyws1LVr17ojeIZYvny5RiKRdp8jEono8uXLvRbtGIAGTfBctZFCGjQ1NXHPPfN49913Wbv2ADNmCF/8ItTUDOi26cYPE9jp4Mfenxsy+aWX7geSNQPlXM/awQ+jxoyQSFsEoXkxUmg7Qli4EF21Cl24sPsjhBihkOiqVe1HCbG2ciUaDocy9Ekyix97f36UKZeorKxsdy979erVbhlrlZWVcY8PUs86GYI0asRGCpkj2/EJQY098GPvz48y5RLJTLCGw2FmzJgR9/ic6Vk75MqosUulICKfEpHVIvKKsz5CRIIxjZ4Fsm3eCXLsgV+8cfwuU67QldLt1asXLS0tCR/yQfea6kgs5UkoFH2shsNh11KeZJJkRgoLgX8HDgOo6ktE6x/kJdmOTwh67IEfe39+lClX6EzpfupTnwISzzHkSs8610hGKRSq6p86bGuO+848INvmHT/HHiSDH3t/ycjkx7iKoBBTum05cOAAf/nLX4DEE/u50rPONZJRCrtF5FScVNYi8jWidRDykmTMO93NnurX2INk8GPvLxmZOsYwmJJInpjSHTp0KAUFBa3bm5ujfcegBnHlLYlmoGMN+CSwCtgPvAP8Hjilq+PcaF57H8WLT9ixY0dWvZOM7NAxhiFZn3tDdcqUKfqjH/1IW1paOvX4Mk+vzLF3716dOnWq7t27N63j6cT7qCuFEAK+7rw+DujT2fvdbl4Fr+3bt09ramZraWmJhsMhLS0t0Zqa2a3bcyH4LNdJ1p3yhBNO8FrUwJFrrqZ+pLudls6UQqfmI1U9AtzovP6Hqu7LxOgk6HRm3glq8Fm+kYw7JcC+ffsCmXrBS2xiP/tkM19UMnMKK0XkeyJSKiL9Yi3jkuQIfsieanRNV+6UMWJ2cTB7eLL40dkg6LiZMSAZpfCvwA3ABmCj0xq6feUcJdPeSV6X/MxlErlTxsMC3ZLHj84GQcfNTKyiHd1oAkRZWZk2NPhLP82ZU8369Xcza1Z7E5JqNNZg/PibmT27JqlzxTKmFhe/yfTpBxgyJKpY6uqiMQtBcFH1O7/4xS+47rrr2L9/PwUFBRw8eJAjR460e08kEuHRRx9l8uTJHklpGLB27VomT57M/v37j9mXaqdFRDaqalm8fclENPcUkX8TkcecdqOI9EzqynlIJoPP/FDyM9fpaOooLS1t3Wf28PaYm663uBWdn4z56AFgDHC/08Y424w4ZDL4zCats09HU8fgwYNbt7e1hy9YsCCnHojpPOD9WCMjXYKq4FyZxE/klhRrwIvJbPOieVlPwQ2CmjG1K7rrY51N2vrcq6o2Nzfr/Pnz9ZxzzsmpuIV0XBr9WCMjXYIah5KpTKx0M0tqixPRDICIfBJoSVcJichpIvJCm/ahiMxwvJpWisgmZ3lCutfIFYKaMbUr/NzjTJR6ITZJmiseNMm4NPqxRkamCGoJUFcm8RNpi1gDKoGtwDpgPbAFqOjquGQaEAbeBU4B7gZudbbfCtzV1fG5PlLI1UC4IPQ4u1srwG+k83mSqUfRu3fvQEQpB+X77GwUnckRNulGNOvRh3cBMAIYCRQkc0yS5/0C8LTz+g3gJOf1ScAbXR2f60ohWyU/3SYof8i25FqBnnQ/T1fHff/73/fmA6VIUL7PzsxamTR5dUspEI1R6Ntm/QTg+q6OS6YBPwdudF7v7bDvgwTHXEM0TqLh5JNP7vbN8TudpdQICkH5Q3Yk1/L4pPt54qWtCIVCvh/tdSQI32dno+hMjrC7qxReiLPtz10dl8R5ewG7gRM1BaXQtuX6SCGXCMIfMh65lscnnc9TW1ur4XA47nfn59FePPz2fSY7is70CLszpZDMRHNI5KhTpIiEnQd6d/ki8LyqvuesvyciJznXOAnYmYFrGD4hqBXQci2PTzqfZ9GiRRw5cgTp6BtN8NJi++37TDYHV6J92bjnySiF/wX+W0QqRWQCUAf8LgPXnu6cK8Yy4DLn9WXA4xm4huEj/PaHTIZcy+OTzucpLi5m/vz5rFy5MvD1rv32fXaVg6ugoKBdjYq2ZO2eJxpC6FEzTgi4FngM+BXwLSDc1XFdnLMQ2AMUt9nWH1gNbHKW/bo6j5mPgkWmfKzdJFHcwpQpUzyWLD26+3n8Zn5JFb9+n53d12zcc7rrfaRHH9z9gBGpHJPNZkohWPj1D+klfg7ki0dtba0WFRVpKBTS3r17aygU0qKiosAFgfmNzu5rNu55t5QC0fiE4x2FsJVoltR7ujrOjWZKITVinkyDBg3QUEh00KABgfNkyjWCFlkbxNFeEOjsvmbjnnemFJKZUyhW1Q+BrwAPqeoYIHghjHlOLOPq+vV3U129mxUrlOrq3axffzfl5edZKm6PCFpkbS6mxfZDHqTO7qvr9zyRtog14GWiwWQrgHOdbS91dZwbzUYKyZOr0dFBI4iBfLlONkZrfjcL0s2RwhyiHkh/U9XnnNxHmzKrmoxsYxlX/YGbxVKM5MjGaM3P+b26okuloKqPquoIVb3eWf+7qn41+6LlD25UV8uVMqF+GOp3h65cEIPi2hlk3Ej0FzSzYFuSGSkYWcQtW3+uZFwNcg8sRlAD+XKFbIzWcimjrCkFj3GrutrVV19PXV2E6JTQUVRh6dIIV111XUauk22C3ANrSxAD+XKFbIzWcsosmGiyIQgtFyaaBw0aoAsXxi+ks2ABWlpakpHrBDXjaq5OzJprp/dkOigsSPm9SHeiWUROd9JbFHXYPimzqil/ccvWn8kyoW6SUz2wNuSia2fQyPRoLWfMgom0BfBvRGsc1BMtrDOlzb7nEx3nZrORQn4QpB6YERyyMVoLSsQ3aY4UrgbGqOpUoBz4DxH5jrPv2HSJRlrkiq0/GdL1ssqZHpjhK7IxWvNbwr10EO34NIrtEHlNVc9ss15ENCnea8AEVR3ljoiJKSsr04aGBq/F6BYx76Pi4jeZNu0AQ4ZEvYGWLo3Q2Hiqr007qdD2c06ffvRz1tUl9zl/8YtfcN1117F//34KCgo4ePAghYWFPPDAA3zzm9908ZMYRmKmTp3KuHHjmDFjBqFQiJaWFn784x/z1FNPUV9f77V4rYjIRlUti7sz0RACWAOM6rCtB7AEaEl0nJstF8xHqrlRXa0ruhtRbROzhpE56MR81NlIYRDQrKrvxtn3OVV9OiMqqxvkwkghXygtLaG6ejdDhx67b9MmuOOOErZuTVxXKSg9MMMIAp2NFBIqhSBgSiE4hMMhVqxQwuFj9zU3w6RJIZqbW9wXzDDykM6UggWvGa6QKxHVmSDoqToMb3Drd2NKwXCFfPKy6opcSNVhuI9bv5uklYKIHC8i/WItm0IZuUdV1U00Np7K3LkRXn4ZFi+Giy6Cykr4058Oc+jQobyp6ZArqToMd3Hrd9OlUhCRb4nIe8BLRKuubQTMkI872U1zhVhE9XnnzeC228L89a/wgx/AypVw770tPPvsj3O22E8uJUsz3MOr300yI4XvAWep6mBVHeK0T2ZUigBilcyipKIYi4qKKCjoxbnn9uTOO8lKAkA/2utzNVWHkV28+t0koxTeBPZn8qIi0ldEHhORv4jI6yIy1jFLrRSRTc7yhExeM9O4ld3Uz6SjGLNd7MeP9nqroWCkg1e/m2SUwr8Dz4jIz0Tkvljr5nV/AvxOVU8HRgKvA7cCq1V1GLDaWfctVsksPcWY7QSAfrXXW6oOIx28+N0koxR+RjS6+Q8cnVPYmO4FReR4YBywCEBVD6nqXmAK8LDztoeBqeleww1ypZJZd0hHMWbaNTVI9nqroWCkg9u/m2SUQrOqVqnqQ6r6cKx145qfBHYBD4nIn0XkQRE5DjhRVXcAOMuPxTtYRK4RkQYRadi1a1c3xOge5nefnmLMtGtqkOz1uZAszXAft383ySiFtc6D+KQMuaT2AM4BHlDV0cA/SMFUpKoLVLVMVctKSkq6IUb3ML/79BRjW9fUTZui0cybNsHcudHEeFVVN6UkQ5Ds9VZDwUgHt383Xaa5EJF4f3tN1wNJRD4O/EFVBzvr/0RUKQwFylV1h4icBKxT1dM6O5eXaS7yJbtpZ8yZU8369Xcza1Z7E5Jq9CE/fvzNzJ5dc8xxTU1N3HPPPB588AG2b9/DwIH9ueqq66iquinte/bEE09w0UUXceDAgdZtkUiERx99lMmTJ6d1TsPIVbqV5qKNG+qQTLikOgn23haR2AO/kmg67mXAZc62y4DH072GGwS1klkmSbfXX1RUxOzZNWzdupPm5ha2bt3J7Nk13bpnZq/PP/zofpwTJEqf2rYBZwNfBy6NtWSO6+R8o4gGwL1EtLLbCUB/ol5Hm5xlv67Okyups4OMX9J+W2rt/GPJkiUK+K6qWRAgndTZMUSkmmjltTOBJ4EvAr9X1a9lVj2ljmVJNWJYau3cpLGxkcsvv5zFixdTXFzcbl9FRQXr1q2joqKCNWvWeCRhMEmryI4e7dW/TNTM9KKzfiKwvKvj3Gg2UkiPWO9+0KABGgqJDho0IOeK+hi5QdvRQGVlZbv63L169Wq3jLXKykqvxfY9pFmjOcZHqnoEaHZiDHYSdSs1Aoil5zCCRNtgxCC5HweZZJRCg4j0BRYSDVp7HvhTVqUysoal5zD8TGfBiBMmTGD//sQZd/zkfhxkUqq8JiKDgeNV9aVsCZQKNqeQOt0ti2kY2WTt2rVMnjy504d/QUEBR44c4fDhw63bzP04NbrlkioiV8Zeq+oW4FVn8tkIIJaew/AzyQQjzpw5k4KCAnM/zhLJmI8qReRJJ6L5bKI5kPpkWS4jS1h6DsPvdJUE7plnnrF0IVkkmeC1bxBNUPcyUZfUGar6vWwLZmQHS89hBIHOghEtXUh2SSZOYRhHlcIZRKOPq1Q1ozUW0sHmFFLH0nMYQaCiooINGzYwcuRI7rrrLm655RZefPFFxo8fbzEJGaBbcwrAcuA/VPVbwHiiEcfPZVA+w0UsPYcRBGw04B3JjBSOV9UPO2wbpqqbsipZEthIwTAMI3XSGimIyM0AqvqhiFzUYfcVGZTPMIw4WMI3wws6Mx9Na/P63zvsm5QFWQzDaIMf600buU9nSkESvI63bhgZp6mpiTlzqiktLSEcDlFaWsKcOdV5k4rDr/WmjdymM6WgCV7HWzfyjGw/sP2So8lNxRSketNG7pJwollEWoiWyhSgNxBzQRUgoqo9XZGwE2yi2RvaurVOn37UrbWuLnNurelWdcskbnzOtiST4sHy+xiZIK2JZlUNq+rxqtpHVXs4r2PrnisEwzvcSKq3cOH9TJ/eXiEAiMC0aQd48MEHun2NrnA7eWCQ6k0buUsycQqG0Q43Hth+yNHkhWLqKsWDKQQj25hSMFLGjQe2H3I0eaWYrN604SWmFIyUceOB7XWOpqamJvr27e2JYlq0aJElfDM8w5SCkTKdPbDr6iIMG3ZGt711qqpuorHxVObOjbBpEzQ3R+s9zJ0bneStqropg5+oPbEJ5hNOOEhtLa4rJkvxYHhJSkV2Mi9dOXMAABtASURBVHZRkS3APqAFaFbVMhHpBzwCDAa2AF9X1Q86O0+q3kdNTU3cc888Fi68n+3b9zBwYH+uvvp6qqpuspw/KZAoqV5dXYQ//1kZMQIuueRgt711Yt/Xgw8+0Pp9XXXVdVn/vmKeTzNnHmDmTDjxRLj4Ylo/T21tD/bvP81yRRmBpTPvIy+VQpmq7m6z7W7gfVX9oYjcCpygqrd0dp5UlILb7oW5TrwH9rBhZ9Dc/Eduv/2gZ26kmaBtdbqPPoJHH4Xf/hZ27oT+/eHw4ULefvs9+70YgSUoSuENoFxVd4jIScA6VT2ts/OkohT84Pee63RV6nPOnAG8/fYu9wVLkXA4xIoVSjh87L7mZpg0KURzc4v7guURjY2NXH755SxevJji4mKvxck5ups6OxsosEJENorINc62E1V1B4Cz/Fi8A0XkGhFpEJGGXbuSf8D4we891+nKW+edd3YHIkWFHzyf8h3L++QdXimFz6nqOcAXgRtEZFyyB6rqAlUtU9WykpKSpC/oB7/3XKerh2mfPmQ84CsbeO35ZFjeJy/xRCmo6nZnuRP4DfBp4D3HbISz3JnJa1rvL/tcffX1PPRQfG+dX/4SKioIxIjMS88nv5LtNN6W98k/uK4UROQ4EekTew18AXgFWAZc5rztMuDxTF7Xen/Zp6rqJp5/HmpqaPcwramB996DK68MxojMqtMdS7bNObfddlu79B6HDh1qt4Romo9Zs2Zl5fpGG1TV1QZ8EnjRaa8Ctznb+wOriZb7XA306+pcY8aM0WTZt2+fjhlzlk6YENEFC9CVK9EFC9AJEyI6ZsxZum/fvqTPlej8NTWzddCgARoKiQ4aNEBramZ3+7xBY+DA/jplCvrxj6OhUHR5xRXok09G73dpaYnXIhppUF5eroBWVFRk7Rpr1qzRwsJCJTrn2K4VFhbq2rVrs3btfANo0ETP6EQ7gtBSUQqqRx/cpaUlGg6HtLS0JCMP7rYKZ+FCdNUqdOHCzCmcIFFTM1snTIjomjXo2rXR9uSTUcXQpw8qQt4qzER43aHYu3evTp06Vffu3du6rbKyst1DuVevXu2WsVZZWZlRWZYvX66RSKTdNSKRiC5fvjyj18l3TClkmXgPwrVr0TVrooqhpma21yK6RscR2bJl6Cc/iX7+8+S9woyHHzoUS5YsUUBra2tbt3XWa89m7722tlaLioo0FApp7969NRQKaVFRUTvZjO7TmVKwNBcZwNxdj9LRHv+Vr8DHPw5z5uBK+umg4XZ67njE8/TxKo13ruV9CmKdbVMKGcDcXdtTVFTE7Nk1vPba3ykq6s1f/woTJ8K0abBkSTRKOB8VZjy86FAk6+lz5513up7G26u8T9l6eAcx3sKUQgYwd9djiaUVOeOMj/jBD2DFCpg7F/7+d6iqiiqGfFSYHfGiQ5GKp4/babzr6+upqqoiFIo+msLhMDNnzqS+vj4r14uRrYd3EOMtTClkAHN3PZaYWeTOO9ubjaqrownmHn00fxVmW7zoUKRiGuquOSco5pNMPbxzId7ClEIGsGCnY+nMLHLxxdEEc/mqMNviVYci2Qpv3TXn+NV8kq2Hd07EWySagQ5C84v3kWr23F2DSigkumpVe2+sWFu5Muqaat5H2Y+f6Qw3PH3ciG9Ih2x6VwUh3gJzST1KJnzCvfYrDwKDBg3QhQvjK4UFC9D+/Qvtfjl41aEoLy/XUCiko0eP1hUrVujo0aM1FAp16wGerfiGeLEU3SWbD2+/x1uYUnDIhE+4H/zKg4DFbvifKVOm6I9+9CNtaWlRVdXm5madP3++TpkyJe1zZqsHHi+WIhMkenifd9553VJAfo+3MKXgkOqDKt6IYMKEcTpuXIE97LrAS7OI4S3Z6IFnywwV7+FdUFDQbQWUjVFYJulMKeTVRHMqPuExl8r16++muno3K1Yo1dW7OXBgA9u2HeTAga7Pkc9YUrnOaWpqYs6c6m7XsvYjyU5id4ZbXjzxvKsOHjwIdM8TKdB1thNpiyC0VEcKXU1+hsOh1vd2NqoYPz6ay6ercxj+x4v5oXwwQSZjPuns3ruVZmPKlCk6dOjQducNhUKu5HnyEmykECUVn/BkXCq7OofhbxKNBtevv5vy8vOy1mv3Q2qLbNNVfENX9/7cc891Jc1GfX09CxYsaHedI0eOAAFzI80geaUUUvEJ7yrSdGeHEkDxzmH4G68ezvmQK6sr80ky9z4TZqhk8CrPk1/JK6WQSpBZV6OKoiIsUC3gePVw9kOurGzPaXSVriLZe+9Wmg23FFAQyCulkMrkZ2ejirq6CGVl42wCNeB49XD2OleWV2aztiR7793Mmup2nie/kldKAY5m8Ny6dSfNzS1s3bqT2bNrjnmYdzaq+PDDU3n88f/p8hyGv3H74Rzrne/fv59rrmmfNRbcM0H6YU4j2XvvphdPrqXtTpe8UwrJYi6VuY+beYfa9s7vums/K1dGs8a++WY0a+zLL7tngvTDnMbVV1/P//t/iUfisXvvZtbUQLuRZhDRjt9KgCgrK9OGhgavxTACSuxBXVz8JtOmHWDIkGgvdenS6MM5k8p/zpxq1q+/m1mz2j+MVeG22+D11wv57ne/R1XVTVnvcITDIVasUMLhY/c1N8OkSSGam1uyKsO7777LGWcM5qyzDnLZZbTe+4cfhldfLeD117fw8Y9/PKsy5DMislFVy+Lts5GCkbe4ORrsrHd+xRXQp89xrpkgvZ7TAFiw4AFGjIBPfQpmz4YLLoguP/UpGDEiut/wBs9GCiISBhqAd1R1sogMAZYC/YDngUtU9VBn57CRghEU/NA7j9HZqGXu3Ajjx9/M7Nk1WZWhtLSE6urdDB167L5Nm+COO0rYunXnsTuNjODXkcJ3gNfbrN8F3Kuqw4APgCs9kcoIBEFLE+GH3nkMP9T/8INbrhGfHl5cVEQGAV8C7gSqRESACcA3nLc8DNwOpDyGPHz4MNu2beNAx+REhmdEIhEGDRpEz549M3K+tnMB1dWxuYDd1NXdzbJlv/KlI0B0Ujt+79ztoMeY2eyee+Zxxx0PsH37HgYO7M9VV13nypwGxJRk/JFCEDIDNDU1cc8981i48P7W+3f11de7dv+yiSfmIxF5DPgB0Af4HnA58AdVHersLwV+q6pnxzn2GuAagJNPPnnMW2+91W7/5s2b6dOnD/3790c6GnAN11FV9uzZw759+xiSqGuYIn4wf6SKm5PaQSCI32GMtt/l9OlHv8u6uuB8l74yH4nIZGCnqm5suznOW+NqK1VdoKplqlpWUlJyzP4DBw6YQvARIkL//v0zOnLzg0tlqpiLc3v8YMJKl3TiPIJk7nR9pCAiPwAuAZqBCHA88BvgAuDjqtosImOB21X1gs7OFW+i+fXXX+eMM85ISabGxkYuv/xyFi9eTHFxcUrHGsmRzveSCD9N2hrpEzPBPPigNyasdEl1ktyPIwtfjRRU9d9VdZCqDgamAWtU9WJgLfA1522XAY+7JZNfi4sb8fHTpK2RPslmF/AbqU6S+yGCPBX8FKdwC9FJ578B/YFFbl04FsaeyXD23/zmN4gIf/nLX+Luv/zyy3nssceSPt/27dv52teiOvOFF17gySefbN23bt261iIkqTB48GB2796d8nFe42YksmF0JNVOSdDMnZ4qBVVdp6qTndd/V9VPq+pQVb1IVQ9m67puVHWqq6vj85//PEuXLs2IzAMHDmxVIplSCkElyPZoI/ik2ikJmvutn0YKrnHbbbe1y50eK6aRqaIaTU1NPP300yxatKhVKagqN954I2eeeSZf+tKX2NmmIMPgwYP5/ve/z9ixYykrK+P555/nggsu4NRTT+WnP/0pAFu2bOHss8/m0KFDzJ49m0ceeYRRo0Zx11138dOf/pR7772XUaNG8dRTT7Fr1y6++tWvcu6553Luuefy9NNPA7Bnzx6+8IUvMHr0aL71rW8R1BQnNmlreEmqnZLAmTsTlWQLQotXjvO1117rtAxdjGwUF49RW1ur//qv/6qqqmPHjtWNGzfqr371K504caI2NzfrO++8o8XFxfroo4+qquopp5yi999/v6qqzpgxQ4cPH64ffvih7ty5U0tKSlRVdfPmzXrWWWepqupDDz2kN9xwQ+v1qqurdd68ea3r06dP16eeekpVVd966y09/fTTVVX129/+ttbU1Kiq6hNPPKGA7tq1K+3PmQrJfi+GEQRipURLS0s0HA5paWlJwjKunZX2nTAhojU1s12Xn07KcXoSvOYHYkU1LrroonbukpkoqlFXV8eMGTMAmDZtGnV1dRw+fJjp06cTDocZOHAgEyZMaHfMl7/8ZQCGDx9OU1MTffr0oU+fPkQikZTzua9atYrXXnutdf3DDz9k3759bNiwgV//+tcAfOlLX+KEE05I+zMaRj4TmyRPJpaiquomli37FXPnxo9R8Zu5M2+VArQvqlFQUMDBgwe7XVRjz549rFmzhldeeQURoaWlBRHhwgsv7DR2oqCgAKBVlhihUIjm5uaUZDhy5AjPPvssvXv3PmZfUOM3cjmC1Mht/BBBngp5OacQIxtFNR577DEuvfRS3nrrLbZs2cLbb7/NkCFD6NevH0uXLqWlpYUdO3awdu3atK/Rp08f9u3bl3D9C1/4Av/5n//Zuv7CCy8AMG7cOH75y18C8Nvf/pYPPvggbRncxA+VwgyjOwTJ/TavlUI2imrU1dVx4YUXttv21a9+lXfffZdhw4YxfPhwrrvuOsaPH5/2NSoqKnjttdcYNWoUjzzyCP/8z//Mb37zm9aJ5vvuu4+GhgZGjBjBmWee2TpZXV1dzYYNGzjnnHNYsWIFJ598ctoyuEnQ/LwNI8jkXJGdTEbOGpmjO9+LpVk2jMziq4hmw0iVoPl5G0aQMaVg+J7A+XkbRoAxpWD4HktrYRjuYUrB8D2W1sIw3MOUguF7LK2FYbhHXiuFIBW+yHeC5OdtGEEmb5VCNgOiRISZM2e2rs+fP5/bb7+902Pq6+vbpaZIh1RTYS9btowf/vCHca+/ePFitm/fntL1Y0n7DMMILnmrFLIZEFVQUMCvf/3rlB7QmVAKqfLlL3+ZW2+9Ne7101EKhmEEn7xVCtksfNGjRw+uueYa7r333mP2vfXWW1RWVjJixAgqKyvZunUrzzzzDMuWLeOmm25i1KhRvPnmm+2OWb58OZ/5zGcYPXo0EydO5L333gMSp8LesmULp59+OldddRVnn302F198MatWreJzn/scw4YN409/+hMQffDfeOONx1z/rrvuoqGhgYsvvphRo0bx0UcfsXHjRsaPH8+YMWO44IIL2LFjBwAbN25k5MiRjB07lv/6r/9K+54ZhuETEqVPDULrTursUEh01ar2qWxjbeVKNBwOJXWeeBx33HHa2Niop5xyiu7du1fnzZun1dXVqqo6efJkXbx4saqqLlq0SKdMmaKqqpdddllrKu2OvP/++3rkyBFVVV24cKFWVVWpauJU2Js3b9ZwOKwvvfSStrS06DnnnKNXXHGFHjlyROvr61uv2TYFd8frjx8/Xp977jlVVT106JCOHTtWd+7cqaqqS5cu1SuuuEJVVYcPH67r1q1TVdXvfe97rem9O2Kpsw3DP9BJ6uy8HSlkOyDq+OOP59JLL+W+++5rt/3ZZ5/lG9/4BgCXXHIJv//977s817Zt27jgggsYPnw48+bN49VXXwVgw4YNfPOb3wSOTYU9ZMgQhg8fTigU4qyzzqKyshIRYfjw4WzZsiWlz/LGG2/wyiuvcP755zNq1Cjmzp3Ltm3baGxsZO/eva15nC655JKUzmt4izlaGPHIW6XgRkDUjBkzWLRoEf/4xz8SvieZVNbf/va3ufHGG3n55Zf52c9+1q7+Q6LjO6bfbpuaO9VU3KrKWWedxQsvvMALL7zAyy+/zIoVK1DVwKbizncs86yRiLxVCm4ERPXr14+vf/3rLFq0qHXbZz/72dYSnb/85S/5/Oc/Dxyb/rotjY2NfOITnwDg4Ycfbt2eyVTYnaXjPu2009i1axfPPvssAIcPH+bVV1+lb9++FBcXt452YrIY/scyzxqJcF0piEhERP4kIi+KyKsiUuNsHyIifxSRTSLyiIj0yqYcbgVEzZw5s50X0n333cdDDz3EiBEjqK2t5Sc/+QkQrdA2b948Ro8efcxE8+23385FF13EP/3TPzFgwIDW7ZlMhd3x+pdffjnXXnsto0aNoqWlhccee4xbbrmFkSNHMmrUKJ555hkAHnroIW644QbGjh0bt6iP4U+y6WhhBBvXU2dL1N5wnKo2iUhP4PfAd4Aq4NequlREfgq8qKqd/jItdXZwsO/FX4TDIVasUMLhY/c1N8OkSSGam1vcF8xwBV+lznYmv2MGy55OU2AC8Jiz/WFgqtuyGUa+YJlnjUR4MqcgImEReQHYCawE3gT2qmpsBnQb8IkEx14jIg0i0rBr1y53BDaMHMMyzxqJ8EQpqGqLqo4CBgGfBuLZFeLatVR1gaqWqWpZSUlJovNnTFaj+9j34T8s86yRCE+9j1R1L7AOOA/oKyI9nF2DgLRyLEQiEfbs2WMPIp+gquzZs4dIJOK1KEYbLPOskQgvJppLgMOquldEegMrgLuAy4BftZlofklV7+/sXPEmmg8fPsy2bdva+fIb3hKJRBg0aBA9e/b0WhTDMOh8orlHvI1Z5iTgYREJEx2p/LeqPiEirwFLRWQu8GdgUWcnSUTPnj0Zkqigr2EYhtEprisFVX0JGB1n+9+Jzi8YhmEYHpG3Ec2GYRjGsZhSMAzDMFpxfaI5k4jILuCtNA8fACRfBcd7giRvkGQFkzebBElWCJa83ZH1FFWN69MfaKXQHUSkIdHsux8JkrxBkhVM3mwSJFkhWPJmS1YzHxmGYRitmFIwDMMwWslnpbDAawFSJEjyBklWMHmzSZBkhWDJmxVZ83ZOwTAMwziWfB4pGIZhGB0wpWAYhmG0khdKwS8lQFPBqTnxZxF5wln3s6xbRORlEXlBRBqcbf1EZKUj70oROcFrOQFEpK+IPCYifxGR10VkrI9lPc25p7H2oYjM8Ku8ACLyXec/9oqI1Dn/PV/+dkXkO46cr4rIDGebb+6tiPxcRHaKyCtttsWVT6LcJyJ/E5GXROScdK+bF0oBOAhMUNWRwChgkoicRzQ7672qOgz4ALjSQxk78h3g9TbrfpYVoEJVR7Xxm74VWO3Iu9pZ9wM/AX6nqqcDI4neY1/KqqpvOPd0FDAG2A/8Bp/KKyKfAP4NKFPVs4EwMA0f/nZF5GzgaqL51kYCk0VkGP66t4uBSR22JZLvi8Awp10DpF9kW1XzqgGFwPPAZ4hGA/Zwto8F/tdr+RxZBjlf+ATgCUD8KqsjzxZgQIdtbwAnOa9PAt7wgZzHA5txHCz8LGsc2b8APO1neYlWS3wb6Ec02eYTwAV+/O0CFwEPtln/D+Bmv91bYDDwSpv1uPIBPwOmx3tfqi1fRgrdKgHqAT8m+gM94qz3x7+yQrRK3goR2Sgi1zjbTlTVHQDO8mOeSXeUTwK7gIcc09yDInIc/pS1I9OAOue1L+VV1XeA+cBWYAfQCGzEn7/dV4BxItJfRAqB/wOU4tN724ZE8sUUcoy073PeKAXtRglQNxGRycBOVd3YdnOct3ouaxs+p6rnEB3C3iAi47wWKAE9gHOAB1R1NPAPfGJ66QzHBv9l4FGvZekMx749BRgCDASOI/qb6Ijnv11VfZ2oWWsl8DvgRaC504P8TcaeEXmjFGJoFkqAZpjPAV8WkS3AUqImpB/jT1kBUNXtznInUZv3p4H3ROQkAGe50zsJW9kGbFPVPzrrjxFVEn6UtS1fBJ5X1fecdb/KOxHYrKq7VPUw8Gvgs/j0t6uqi1T1HFUdB7wPbMK/9zZGIvm2ER3pxEj7PueFUhCREhHp67zuTfTH+zqwFvia87bLgMe9kfAoqvrvqjpIVQcTNRmsUdWL8aGsACJynIj0ib0mavt+BVhGVE7wibyq+i7wtoic5myqBF7Dh7J2YDpHTUfgX3m3AueJSKGICEfvr19/ux9zlicDXyF6j/16b2Mkkm8ZcKnjhXQe0BgzM6WM1xM+Lk3WjCBa4vMlog+s2c72TwJ/Av5GdGhe4LWsHeQuB57ws6yOXC867VXgNmd7f6KT5ZucZT+vZXXkGgU0OL+FeuAEv8rqyFsI7AGK22zzs7w1wF+c/1ktUODj3+5TRJXWi0Cl3+4tUSW1AzhMdCRwZSL5iJqP/ovoXOnLRD3A0rqupbkwDMMwWskL85FhGIaRHKYUDMMwjFZMKRiGYRitmFIwDMMwWjGlYBiGYbRiSsHISUSkpUOGUdcil+NltzSMoGAuqUZOIiJNqlrk0bXHAU3AEo1mC3XjmmFVbXHjWkZuYyMFI28QkWIReSMW0ezk+7/aef2AiDRIm3obzvYtIvJ/ReRZZ/85IvK/IvKmiFwb7zqquoFo2oTOZLnIyeX/oohscLaFRWS+RGtTvCQi33a2VzoJ/F52RiEFbWSbLSK/By4SkVNF5HdOYsKnROT0TNw3I7/o0fVbDCOQ9Hay4sb4gao+IiI3AotF5CfACaq60Nl/m6q+LyJhYLWIjFDVl5x9b6vqWBG5l2iO+88BEaIR3D9NU77ZwAWq+k4sBQvRPPhDgNGq2uwUVIk416xU1b+KyBLgOqL5sAAOqOrnAURkNXCtqm4Skc8A9xPNnWUYSWNKwchVPtJoVtx2qOpKEbmIaEqAkW12fd1J+92DaJ76M4mmwoBoXhmIpg8oUtV9wD4ROSAifTWaZDFVniaqnP6baOI4iObk+qk6aaYdJTWSaJK5vzrveRi4gaNK4REAESkimnzu0WjaISCaYsIwUsKUgpFXiEiIaNr0j4gWg9kmIkOA7wHnquoHIrKY6EggxkFneaTN69h6Wv8hVb3W6c1/CXhBREYRzV/TcZIvXkrktvzDWYaI1i04RhEaRirYnIKRb3yXaIbc6cDPRaQn0Yps/wAaReRE4tcAyCgicqqq/lFVZxOtTFYKrACujaWZFpF+RJPLDRaRoc6hlwDrO55PVT8ENjujoFjN3pEd32cYXWFKwchVendwSf2hiHwKuAqYqapPARuAWar6ItEsuq8CPydq2kkbEakDngVOE5FtIhKvJvE8Z+L4FUeOF4EHiaaffklEXgS+oaoHgCuImoVeJjo6STSPcTFwpXPsq0QL3hhGSphLqmEYhtGKjRQMwzCMVkwpGIZhGK2YUjAMwzBaMaVgGIZhtGJKwTAMw2jFlIJhGIbRiikFwzAMo5X/D2r06iWZghgyAAAAAElFTkSuQmCC\n",
+ "text/plain": [
+ "
"
+ ]
+ },
+ "metadata": {
+ "needs_background": "light"
+ },
+ "output_type": "display_data"
+ }
+ ],
+ "source": [
+ "plotData(X, y)\n",
+ "# add axes labels\n",
+ "plt.xlabel('Exam 1 score')\n",
+ "plt.ylabel('Exam 2 score')\n",
+ "plt.legend(['Admitted', 'Not admitted'])\n",
+ "pass"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "\n",
+ "### 1.2 Implementation\n",
+ "\n",
+ "#### 1.2.1 Warmup exercise: sigmoid function\n",
+ "\n",
+ "Before you start with the actual cost function, recall that the logistic regression hypothesis is defined as:\n",
+ "\n",
+ "$$ h_\\theta(x) = g(\\theta^T x)$$\n",
+ "\n",
+ "where function $g$ is the sigmoid function. The sigmoid function is defined as: \n",
+ "\n",
+ "$$g(z) = \\frac{1}{1+e^{-z}}$$.\n",
+ "\n",
+ "Your first step is to implement this function `sigmoid` so it can be\n",
+ "called by the rest of your program. When you are finished, try testing a few\n",
+ "values by calling `sigmoid(x)` in a new cell. For large positive values of `x`, the sigmoid should be close to 1, while for large negative values, the sigmoid should be close to 0. Evaluating `sigmoid(0)` should give you exactly 0.5. Your code should also work with vectors and matrices. **For a matrix, your function should perform the sigmoid function on every element.**\n",
+ ""
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 5,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "def sigmoid(z):\n",
+ " \"\"\"\n",
+ " Compute sigmoid function given the input z.\n",
+ " \n",
+ " Parameters\n",
+ " ----------\n",
+ " z : array_like\n",
+ " The input to the sigmoid function. This can be a 1-D vector \n",
+ " or a 2-D matrix. \n",
+ " \n",
+ " Returns\n",
+ " -------\n",
+ " g : array_like\n",
+ " The computed sigmoid function. g has the same shape as z, since\n",
+ " the sigmoid is computed element-wise on z.\n",
+ " \n",
+ " Instructions\n",
+ " ------------\n",
+ " Compute the sigmoid of each value of z (z can be a matrix, vector or scalar).\n",
+ " \"\"\"\n",
+ " # convert input to a numpy array\n",
+ " z = np.array(z)\n",
+ " \n",
+ " # You need to return the following variables correctly \n",
+ " g = np.zeros(z.shape)\n",
+ "\n",
+ " # ====================== YOUR CODE HERE ======================\n",
+ "\n",
+ " g = 1/(1+np.exp(-z))\n",
+ "\n",
+ " # =============================================================\n",
+ " return g"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "The following cell evaluates the sigmoid function at `z=0`. You should get a value of 0.5. You can also try different values for `z` to experiment with the sigmoid function."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 6,
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "g( 0 ) = 0.5\n"
+ ]
+ }
+ ],
+ "source": [
+ "# Test the implementation of sigmoid function here\n",
+ "z = 0\n",
+ "g = sigmoid(z)\n",
+ "\n",
+ "print('g(', z, ') = ', g)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "After completing a part of the exercise, you can submit your solutions for grading by first adding the function you modified to the submission object, and then sending your function to Coursera for grading. \n",
+ "\n",
+ "The submission script will prompt you for your login e-mail and submission token. You can obtain a submission token from the web page for the assignment. You are allowed to submit your solutions multiple times, and we will take only the highest score into consideration.\n",
+ "\n",
+ "Execute the following cell to grade your solution to the first part of this exercise.\n",
+ "\n",
+ "*You should now submit your solutions.*"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 7,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "# appends the implemented function in part 1 to the grader object\n",
+ "#grader[1] = sigmoid\n",
+ "\n",
+ "# send the added functions to coursera grader for getting a grade on this part\n",
+ "#grader.grade()"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "\n",
+ "#### 1.2.2 Cost function and gradient\n",
+ "\n",
+ "Now you will implement the cost function and gradient for logistic regression. Before proceeding we add the intercept term to X. "
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 8,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "# Setup the data matrix appropriately, and add ones for the intercept term\n",
+ "m, n = X.shape\n",
+ "\n",
+ "# Add intercept term to X\n",
+ "X = np.concatenate([np.ones((m, 1)), X], axis=1)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Now, complete the code for the function `costFunction` to return the cost and gradient. Recall that the cost function in logistic regression is\n",
+ "\n",
+ "$$ J(\\theta) = \\frac{1}{m} \\sum_{i=1}^{m} \\left[ -y^{(i)} \\log\\left(h_\\theta\\left( x^{(i)} \\right) \\right) - \\left( 1 - y^{(i)}\\right) \\log \\left( 1 - h_\\theta\\left( x^{(i)} \\right) \\right) \\right]$$\n",
+ "\n",
+ "and the gradient of the cost is a vector of the same length as $\\theta$ where the $j^{th}$\n",
+ "element (for $j = 0, 1, \\cdots , n$) is defined as follows:\n",
+ "\n",
+ "$$ \\frac{\\partial J(\\theta)}{\\partial \\theta_j} = \\frac{1}{m} \\sum_{i=1}^m \\left( h_\\theta \\left( x^{(i)} \\right) - y^{(i)} \\right) x_j^{(i)} $$\n",
+ "\n",
+ "Note that while this gradient looks identical to the linear regression gradient, the formula is actually different because linear and logistic regression have different definitions of $h_\\theta(x)$.\n",
+ ""
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 9,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "def net_input(theta, x):\n",
+ " # Computes the weighted sum of inputs\n",
+ " return np.dot(x, theta)\n",
+ "\n",
+ "def probability(theta, x):\n",
+ " # Returns the probability after passing through sigmoid\n",
+ " return sigmoid(net_input(theta, x))\n",
+ "\n",
+ "\n",
+ "\n",
+ "def costFunction(theta, X, y):\n",
+ " \"\"\"\n",
+ " Compute cost and gradient for logistic regression. \n",
+ " \n",
+ " Parameters\n",
+ " ----------\n",
+ " theta : array_like\n",
+ " The parameters for logistic regression. This a vector\n",
+ " of shape (n+1, ).\n",
+ " \n",
+ " X : array_like\n",
+ " The input dataset of shape (m x n+1) where m is the total number\n",
+ " of data points and n is the number of features. We assume the \n",
+ " intercept has already been added to the input.\n",
+ " \n",
+ " y : arra_like\n",
+ " Labels for the input. This is a vector of shape (m, ).\n",
+ " \n",
+ " Returns\n",
+ " -------\n",
+ " J : float\n",
+ " The computed value for the cost function. \n",
+ " \n",
+ " grad : array_like\n",
+ " A vector of shape (n+1, ) which is the gradient of the cost\n",
+ " function with respect to theta, at the current values of theta.\n",
+ " \n",
+ " Instructions\n",
+ " ------------\n",
+ " Compute the cost of a particular choice of theta. You should set J to \n",
+ " the cost. Compute the partial derivatives and set grad to the partial\n",
+ " derivatives of the cost w.r.t. each parameter in theta.\n",
+ " \"\"\"\n",
+ " # Initialize some useful values\n",
+ " m = y.size # number of training examples\n",
+ "\n",
+ " # You need to return the following variables correctly \n",
+ " J = 0\n",
+ " grad = np.zeros(theta.shape)\n",
+ " \n",
+ " J = -(1 / m) * np.sum(y * np.log(probability(theta, X)) + (1 - y) * np.log(1 - probability(theta, X)))\n",
+ "\n",
+ " # ====================== YOUR CODE HERE ======================\n",
+ " \n",
+ " \n",
+ " #ef J(self, theta, x, y):\n",
+ " # Computes the cost function for all the training samples\n",
+ " #m = x.shape[0]\n",
+ " \n",
+ "\n",
+ "\n",
+ " # def gradient(self, theta, x, y):\n",
+ " # Computes the gradient of the cost function at the point theta\n",
+ " # = x.shape[0]\n",
+ " grad = (1 / m) * np.dot(X.T, sigmoid(net_input(theta, X)) - y)\n",
+ " # return grad\n",
+ " \n",
+ " # =============================================================\n",
+ " return J, grad"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Once you are done call your `costFunction` using two test cases for $\\theta$ by executing the next cell."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 10,
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "Cost at initial theta (zeros): 0.693\n",
+ "Expected cost (approx): 0.693\n",
+ "\n",
+ "Gradient at initial theta (zeros):\n",
+ "\t[-0.1000, -12.0092, -11.2628]\n",
+ "Expected gradients (approx):\n",
+ "\t[-0.1000, -12.0092, -11.2628]\n",
+ "\n",
+ "Cost at test theta: 0.218\n",
+ "Expected cost (approx): 0.218\n",
+ "\n",
+ "Gradient at test theta:\n",
+ "\t[0.043, 2.566, 2.647]\n",
+ "Expected gradients (approx):\n",
+ "\t[0.043, 2.566, 2.647]\n"
+ ]
+ }
+ ],
+ "source": [
+ "# Initialize fitting parameters\n",
+ "initial_theta = np.zeros(n+1)\n",
+ "\n",
+ "cost, grad = costFunction(initial_theta, X, y)\n",
+ "\n",
+ "print('Cost at initial theta (zeros): {:.3f}'.format(cost))\n",
+ "print('Expected cost (approx): 0.693\\n')\n",
+ "\n",
+ "print('Gradient at initial theta (zeros):')\n",
+ "print('\\t[{:.4f}, {:.4f}, {:.4f}]'.format(*grad))\n",
+ "print('Expected gradients (approx):\\n\\t[-0.1000, -12.0092, -11.2628]\\n')\n",
+ "\n",
+ "# Compute and display cost and gradient with non-zero theta\n",
+ "test_theta = np.array([-24, 0.2, 0.2])\n",
+ "cost, grad = costFunction(test_theta, X, y)\n",
+ "\n",
+ "print('Cost at test theta: {:.3f}'.format(cost))\n",
+ "print('Expected cost (approx): 0.218\\n')\n",
+ "\n",
+ "print('Gradient at test theta:')\n",
+ "print('\\t[{:.3f}, {:.3f}, {:.3f}]'.format(*grad))\n",
+ "print('Expected gradients (approx):\\n\\t[0.043, 2.566, 2.647]')"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "*You should now submit your solutions.*"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 11,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "#grader[2] = costFunction\n",
+ "#grader[3] = costFunction\n",
+ "#grader.grade()"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "#### 1.2.3 Learning parameters using `scipy.optimize`\n",
+ "\n",
+ "In the previous assignment, you found the optimal parameters of a linear regression model by implementing gradient descent. You wrote a cost function and calculated its gradient, then took a gradient descent step accordingly. This time, instead of taking gradient descent steps, you will use the [`scipy.optimize` module](https://docs.scipy.org/doc/scipy/reference/optimize.html). SciPy is a numerical computing library for `python`. It provides an optimization module for root finding and minimization. As of `scipy 1.0`, the function `scipy.optimize.minimize` is the method to use for optimization problems(both constrained and unconstrained).\n",
+ "\n",
+ "For logistic regression, you want to optimize the cost function $J(\\theta)$ with parameters $\\theta$.\n",
+ "Concretely, you are going to use `optimize.minimize` to find the best parameters $\\theta$ for the logistic regression cost function, given a fixed dataset (of X and y values). You will pass to `optimize.minimize` the following inputs:\n",
+ "- `costFunction`: A cost function that, when given the training set and a particular $\\theta$, computes the logistic regression cost and gradient with respect to $\\theta$ for the dataset (X, y). It is important to note that we only pass the name of the function without the parenthesis. This indicates that we are only providing a reference to this function, and not evaluating the result from this function.\n",
+ "- `initial_theta`: The initial values of the parameters we are trying to optimize.\n",
+ "- `(X, y)`: These are additional arguments to the cost function.\n",
+ "- `jac`: Indication if the cost function returns the Jacobian (gradient) along with cost value. (True)\n",
+ "- `method`: Optimization method/algorithm to use\n",
+ "- `options`: Additional options which might be specific to the specific optimization method. In the following, we only tell the algorithm the maximum number of iterations before it terminates.\n",
+ "\n",
+ "If you have completed the `costFunction` correctly, `optimize.minimize` will converge on the right optimization parameters and return the final values of the cost and $\\theta$ in a class object. Notice that by using `optimize.minimize`, you did not have to write any loops yourself, or set a learning rate like you did for gradient descent. This is all done by `optimize.minimize`: you only needed to provide a function calculating the cost and the gradient.\n",
+ "\n",
+ "In the following, we already have code written to call `optimize.minimize` with the correct arguments."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 12,
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "Cost at theta found by optimize.minimize: 0.203\n",
+ "Expected cost (approx): 0.203\n",
+ "\n",
+ "theta:\n",
+ "\t[-25.161, 0.206, 0.201]\n",
+ "Expected theta (approx):\n",
+ "\t[-25.161, 0.206, 0.201]\n"
+ ]
+ }
+ ],
+ "source": [
+ "# set options for optimize.minimize\n",
+ "options= {'maxiter': 400}\n",
+ "\n",
+ "# see documention for scipy's optimize.minimize for description about\n",
+ "# the different parameters\n",
+ "# The function returns an object `OptimizeResult`\n",
+ "# We use truncated Newton algorithm for optimization which is \n",
+ "# equivalent to MATLAB's fminunc\n",
+ "# See https://stackoverflow.com/questions/18801002/fminunc-alternate-in-numpy\n",
+ "res = optimize.minimize(costFunction,\n",
+ " initial_theta,\n",
+ " (X, y),\n",
+ " jac=True,\n",
+ " method='TNC',\n",
+ " options=options)\n",
+ "\n",
+ "# the fun property of `OptimizeResult` object returns\n",
+ "# the value of costFunction at optimized theta\n",
+ "cost = res.fun\n",
+ "\n",
+ "# the optimized theta is in the x property\n",
+ "theta = res.x\n",
+ "\n",
+ "# Print theta to screen\n",
+ "print('Cost at theta found by optimize.minimize: {:.3f}'.format(cost))\n",
+ "print('Expected cost (approx): 0.203\\n');\n",
+ "\n",
+ "print('theta:')\n",
+ "print('\\t[{:.3f}, {:.3f}, {:.3f}]'.format(*theta))\n",
+ "print('Expected theta (approx):\\n\\t[-25.161, 0.206, 0.201]')"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Once `optimize.minimize` completes, we want to use the final value for $\\theta$ to visualize the decision boundary on the training data as shown in the figure below. \n",
+ "\n",
+ "\n",
+ "\n",
+ "To do so, we have written a function `plotDecisionBoundary` for plotting the decision boundary on top of training data. You do not need to write any code for plotting the decision boundary, but we also encourage you to look at the code in `plotDecisionBoundary` to see how to plot such a boundary using the $\\theta$ values. You can find this function in the `utils.py` file which comes with this assignment."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 13,
+ "metadata": {},
+ "outputs": [
+ {
+ "data": {
+ "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYAAAAD8CAYAAAB+UHOxAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjMsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+AADFEAAAgAElEQVR4nOydd3hUZdbAf+9MSIaQQgslJEAgNCkJEBVECQnNgoCFXVxXwF1EQVSKiN/K0m3gAnYFFJBVQGAJxUYLBESUoEEUQUB6Cy0hAQIkOd8fKSYhfdqdyft7nnkmc+/ce8/cybznPec9RYkIGo1Go6l4mJwtgEaj0Wicg1YAGo1GU0HRCkCj0WgqKFoBaDQaTQVFKwCNRqOpoGgFoNFoNBWUEhWAUupjpVSiUuqXPNuqK6XWKaX2Zz9Xy96ulFJvKaUOKKV+Vkq1s6fwGo1Goyk/pbEA5gN3F9j2IrBBRJoAG7JfA9wDNMl+DAHet42YGo1Go7E1JSoAEYkDLhTY3AdYkP33AqBvnu2fSBbbgapKqbq2Elaj0Wg0tsOjnMfVFpFTACJySilVK3t7PeBYnvcdz952quAJlFJDyLISqFKlSvvmzZuXUxTIzMzkzJnTnD17lhs30qlUyYOAgABq166DyaSXOTQajXuyc+fOcyISUN7jy6sAikIVsq3QWhMiMhuYDRARESHx8fHlumBqaipdunSgefOLTJqUTkgIHDqUzqJFF0lOrs6mTdvx8fEp17k1Go3GyCiljlhzfHmnx2dyXDvZz4nZ248DwXneFwScLL94JTNjxnT8/Q8yblwaoaFgNkNoKIwbl4a//0FmzJhuz8trNGUiOTmZBx54gOTkZGeLotGUWwGsAgZm/z0QWJln+4DsaKAOQHKOq8hezJnzHo88koYqYHsoBf37pzF3rmPWofUPW1MaVq1aRUxMDKtXr3a2KBpNqcJAFwHfAc2UUseVUv8EXgO6K6X2A92zXwN8CfwBHADmAMPsInUeTp48T0hI4ftCQrL2OwL9w9aUho8//jjfs0bjTJQRykEXtgZw48YNjh8/TlpaWrHHHj9+DH//TCpVunnfjRuQnGwiKCj45p025syZM6SlpWGxWKhdu7bdr1cRsFgsBAUFUamwL7cMJCcnM2jQIObPn4+/v7+NpCsd3bp1Y8OGDbmvPT09uX79eu5zDl27dmX9+vUOlU3j+iildopIRHmPt/UisM04fvw4vr6+NGzYEFXQv5MHf38/UlJOU7eu5HMDicCpU4qgoDoEBtazuXz79u0jJSUl97Wvry8+Pj4opbh8+XK+7c2aNbP59d0dEeH8+fMcP36ckKJMvCIoOODntc7+/ve/20niwnnppZf47rvvuHLlCkDuoJ938Pf29mbcuHEOlassOFOBauyLYWMk09LSqFGjRrGDP0Dt2nXIyLBw6pQiLS1r4E9Lyxr8MzIs1K5dxy7y1a1bN1+IaY4lldeiMplMBAYG2uX67o5Siho1apRoARZGQXecM90uUVFRrFmzBm9v70L3e3t788UXX9ClS5cSz+WsdSbt3nRfDKsAgBIHfwCz2UyzZs3x9a3DqVMe/P47nDrlga9vHZo1a47ZbLaLbH5+foSGhhaZZ2AymWjSpAm+vr52uX5FoDTff2E8++yzADz22GMopdi2bRsA3377LUqp3Ee3bt1sJmtxREVFsWTJEiwWS77tFouFJUuWlGrwB+cNxHrdwn0xtAIoLWazmcDAejRoEMLLL79MgwYhBAbWs9vgn4Ofnx+NGjW6aaBSStGoUSM9+DuIbt265RvYU1NT8+03gtslKSkJDw8PTCYTlStXxmQy4eHhQVJSUqnP4aiBuOD9dLYC1dgPl1EAGRkZnDx5gp9/TiA+Pp6ff07g5MkTZGRk5L7HHjOkFStWoJRi7969he4fOnRo7iJfjjWglMonV15OnjzJww8/DEBCQgJffvll7r5Nmzbl/tjKQsOGDTl37lyZj3MXXnrppXwulvT09GLfXxa3i6346KOPuHLlCmFhYaxcuZKwsDCuXLlS7GDurIG44P00ggLV2AeXUAAZGRns27c3e7E3naZNoW7ddFJSTrNv397cwdYeM6RFixZx5513snjx4kL3p6WlISJ4e3sTGhqKt7c3mZmZRQ7IgYGBLFu2DLCdAqjolORnz0tZ3S62wt/fn+nTpxMfH0/37t3ZsWMH06ZNw8/Pr8hjnDUQ23LdQmNwRMTpj/bt20tB9uzZk/v3iRPHZe/eeElO3iGXLv35iIy8VcgqNSGAeHp65nvOeXTt2vWm85eGlJQUCQwMlH379kmzZs1ERCQzM1OefvppadGihdx7770SGRkpc+bMkczMTGnQoIG8+OKL0r59e2nVqpXs3LlTevToIY0aNZL3339fREQOHTokLVu2lGvXrklwcLDUrFlTwsLC5LXXXpPatWtLYGCghIWFSVxcnCQmJsqDDz4oEREREhERIVu3bhURkXPnzkn37t0lPDxchgwZIvXr15ezZ8+W6zManbz/ByWxevVqsVgs+b57QEwmk1SuXFlMJpP4+PjIwoUL7Sixbdm4caN4e3vf9JkA8fb2ltjYWLtdu7D7abFYZPXq1Xa7pqZsAPFixdjrEhbAuXNnqV49f5gnwJgx/6By5T8X1mw9Q4qJieHuu++madOmVK9enR9//JEVK1awb98+du/ezZw5c0hISKBq1aq56wD169cnPj6ebt26MWjQIJYtW8b27dsZP358vnN7enoyefJk/vrXv5KQkMDYsWN56qmnGDlyJAkJCdx1110899xzjBw5kh07drB8+XIGDx4MwKRJk7jzzjv56aef6N27N0ePHi3X53M3CvrZcwgODi6128Vo2GoBuTzYYt1CY2xcQgFcv56Ol9fN2zt3juDzz2fe9OPIwVpTddGiRfTv3x+A/v37s2jRIuLi4njkkUeyF54DiY6OzndM7969AWjdujW33347vr6+BAQEYLFYyvzDWb9+PcOHDyc8PJzevXtz6dIlUlJSiIuLy41nv++++6hWrVq5Pp+7UdDPnuNeCQkJKbXbxYg4ayAuz7qFxrVwCQXg6enBtWuF77v99gimTXvd5jOk8+fPs3HjRgYPHkzDhg2ZPn06S5YsQUSKDU/0ytZUJpMp9++c1yUtThYkMzOT7777joSEBBISEjhx4kRuZFF5QyTdmYJ+9gsXLvDGG2/kJi+ZzWZGjx5NTEyMkyUtG84aiMuzbqFxLVxCAdSsGcCFC4qCVStE4MKFrIHQ1jOkZcuWMWDAAI4cOcLhw4c5duwYISEhVK9encWLF5ORkcGpU6eIjY0t9zV8fX1vyibO+7pHjx688847ua8TEhIA6Ny5M59++ikAX331FRcvXiy3DO5ETEwMo0aNyo3GctUBvyDOGojd9X5q/sQQCuDqjcJDJnMoKdt3+fL/2XyGtGjRIh544IF82x566CFOnz5NkyZNaN26NUOHDiUyMrLc14iKimLPnj2Eh4ezZMkS7r//flasWEF4eDhbtmzhrbfeIj4+njZt2nDLLbfwwQcfADBhwgTi4uJo164da9eupX79+uWWQWMcisr01QOxxl4YohicV90m8sqCNQyPDsXLIyt567fffqNFixa578nIyODMmdOcO3eW69fT8fT0oGbNrK5fDz30EJ07d2bEiBGYTCYyMjKYNWsWW7Zs0T8SF6fg/4E7s3DhQgYMGMDChQsdXrNI45pYWwzOEAqgTuOWYuk3jWa1fXmjXxitg/wr1A9fUzQV6f8gKiqKTZs2ERUVxcaNG50tjsYFsFYBGMIFFFStMh8PiiDp6nX6vvctb3yzDyMoJo3GnrhLyQXdDMl1MYQCAIhuXpu1IyN5oG093ok9QGLKNa5cL1vUjEbjSrhLyQVdLdR1MYwCAPCvXIk3+oUxb9CtZAocTLzM6eSrZGprQOOGuEvJBV0t1HUxlALIIap5LWr7eVHNuxKJKdc4cCZVWwMat8SZmb7lpaDr6ttvvwVg69atLuW60hhUAQCYlCKoujchNauQIcLBxFROJV8lM1NbAxr3wtVKLhR0Xd24cSPfM7iG60pjYAWQg6+lEk1r+1DN25OzKdfYn5jKlWv5rYHU1FQmT55AcHAAZrOJ4OAAJk+ecFNd+LKilGL06NG5r9944w0mTpxY7DExMTHs2bPHquuWtbzzqlWreO211wq9/vz58zl58mSZrn/48GFatWpVpmM05ccRmb62XKh1F9eVxgUUAIDZZMq1BjJFOHj2T2sgNTWVLl06sHnzNCZMOMfatcKECefYvHkaXbp0sEoJeHl58b///a9Mg7EtFEBZ6d27Ny+++GKh1y+PAtA4Fkdk+tpyobZbt25ER0fn9jkuyJUrV4iKitIuIBfAJRRADrnWQJU/rYHXpr2Ov/9Bxo1LIzQUzGYIDYVx49Lw9z/IjBnTy309Dw8PhgwZwsyZM2/ad+TIEbp27UqbNm3o2rUrR48eZdu2baxatYoxY8YQHh7OwYMH8x2zevVqbr/9dtq2bUu3bt04c+YMkFV3qEePHrRt25Ynn3wyNwT28OHDNG/enMGDB9OqVSseffRR1q9fT6dOnWjSpAk//PADkDXIDx8+/Kbrv/7668THx/Poo48SHh7O1atX2blzJ5GRkbRv356ePXty6tQpAHbu3ElYWBgdO3bk3XffLfc905QdR2T62nKhtqALqDC0C8hFsKaWNPAc8AvwKzAie1t1YB2wP/u5WknnKakfQGFcunpd9pxMlrqBNWTOHCQ29ubH7NlIcHBA6YtrF6BKlSqSnJwsDRo0kKSkJJk+fbpMmDBBRER69eol8+fPFxGRjz76SPr06SMiIgMHDpSlS5cWer4LFy5IZmamiIjMmTNHRo0aJSIizzzzjEyaNElERNasWSOAnD17Vg4dOiRms1l+/vlnycjIkHbt2snjjz8umZmZEhMTk3vNefPmydNPP13o9SMjI2XHjh0iInL9+nXp2LGjJCYmiojI4sWL5fHHHxcRkdatW8umTZtEROT555+Xli1blvu+2ZKy9APQ/ElkZKRde2Vs3LhRTCZToX0KTCaTXfsUaP4EZ/UDUEq1Ap4AbgPCgF5KqSbAi8AGEWkCbMh+bXNyrIEzpy8QElL4e0JC4OTJ81Zdx8/PjwEDBvDWW2/l2/7dd9/xt7/9DchqPr5169YSz3X8+HF69uxJ69atmT59Or/++itAseWdQ0JCaN26NSaTiZYtW9K1a1eUUrRu3ZrDhw+X6bPs27ePX375he7duxMeHs7UqVM5fvw4ycnJJCUl5dY1euyxx8p03hzS09M5cOBAmaueamxPx44d8722dY5BVFQUt956Kx4eHvm2e3h4cOutt2r/v4tgjQuoBbBdRK6ISDqwGXgA6AMsyH7PAqCvdSIWjdlkIjCwBocOFb7/0CEIDKxh9XVGjBjBRx99xOXLl4t8T2nKMz/zzDMMHz6c3bt38+GHH5KWllbi8QVLSuctN13WgVZEaNmyZW556d27d7N27doSS1yXlhxFojNCnc/27duBP/tUF8QWC7XDhw/HYrHki16yWCwMHz683Oc0EhUhw9kaBfAL0FkpVUMp5Q3cCwQDtUXkFED2c63CDlZKDVFKxSul4s+ePVtuIZ54YhiLFlkKLRW9eLGFwYOHlvvcOVSvXp2//OUvfPTRR7nb7rjjjtw+wZ9++il33nkncHNJ57wkJydTr149ABYsWJC73VblndPT08nMzMwXPphXnmbNmnH27Fm+++47ICts79dff6Vq1ar4+/vnWjE5spSVnMXyityg3lkUVVaiMAVgqxwDd28YUxEynMutAETkN+B1svz8XwO7gFJPSUVktohEiEhEQEBAecVg1KgxJCc3ZupUC/v3Q3o67N8Pk6dYOH8xhBEjni/3ufMyevTofAPbW2+9xbx582jTpg0LFy7kzTffBLI6h02fPp22bdvetAg8ceJE+vXrx1133UXNmjVzt9uqvHNycjKRkZH5rj9o0CCeeuopwsPDycjIYNmyZYwdO5awsDDCw8NzB4p58+bx9NNP07Fjx3ztFItj3759xMfH5z5yIq5SU1Pzbd+3b1+5Po+m9BRVVqKglaiUslmOgbs3jKkIGc42qwaqlHoFOE7WwnAXETmllKoLbBKRZsUdGxERIfHx8fm2laUKZGpqKjNmTGfu3Pc5efI8gYE1+Mujg3l40DCq+fsTVK0yVbw8Sj6Ri7Nv3z5SUlLw9fWlWbNib7lNuHTpEgcOHCAzM7PI95hMJpo0aZLbyaysVKRqoNYSGxtLr169Cg3PVEoxbdo0PvvsM3bt2kVkZKSuOFqAbt26sWHDhtzXnp6eXL9+Pfc5h65du7J+/XpniHgTTi0HrZSqJSKJSqn6wFqgI/Av4LyIvKaUehGoLiIvFHceaxVAUaSm3eD4xatcz8ikpo8XdfwsmEzu00oxZ8DPQSmV68/P+73aUyEUpwSsHfxBK4CysmbNGvr165dvfcnDw4Ply5fTu3dvMjIyePXVV3n33XfZu3dvbrvMikxycjKDBg3iH//4B/379y8yvwGMl+Tm7HLQy5VSe4DVwNMichF4DeiulNoPdM9+7RR8LJVoUtuXGlW8OJd6jf2JKVy+5rwIFVtHydStWzefjzdn0M87+JtMJgIDA21yvcLw8/OjUaNGNy0iK6Vo1KiRVYO/puwUVlbCYrFw6dIlICvHoEGDBpw+fdqtfdtlIcfXn5ycXOEynK1SACJyl4jcIiJhIrIhe9t5EekqIk2yny/YRtTyYTYp6lWrTKOaVRDg4NlUTiZdJcMJNYVsHSXj5+dHaGhokZEetpiBl4aMjIxcBZAji1KKjIziW31qbE9pFmYrgm+7LOS9H65YnM8aXCoT2Bp8LJVoUsuXGj5/WgOpDrYG7BElY4QZ+Llz58jMzMTb25vQ0FC8vb3JzMzU0UBOoLCF2UaNGhEbG+vyjWdsRUmNeO6///5cF5qji/M5OvS0wigAyLYGqlamUU0fAP6wszXgqCgZZ8/AzWYzQUFBtGjRAj8/P1q0aEFQUBBms9nu164IsdplobCyErNnz3aLxjO2ojSNeABCQ0MdHt7q6NDTCqUAcvCxeNCkli817WwNOMpH7+wZeGhoKHXq1MlVQkop6tSpQ2hoqN2vXRFita1FV+/MT0n3w2w2M3ToUPbt2+fw8FZHu+cqpAKALGsgsGplGgX8aQ2cKGANmM1mwsPDadmyJWFhYcyYMaPYkMeC5PXRf/DBB3z//ff59uf10X/wwQd88skn5fsseWbgFy5coH379gwaNIi+fftyxx13OCQO38fHx+7XKAyj+7ONYKEkJyfz1ltv8fHHH1cY33ZJFOfrj4mJ4b333rNrcb4cnN0XusIqgBx8vP60Bs7nWANpWdZA5cqVSUhI4Ndff2XdunV8+eWXTJo0qUznz/HRDx06lNtvvz13e0Ef/VNPPcWAAQPK9RkKzsAbN27ML7/8wm+//cbAgQN55ZVXynVeeyEiZVKkeXH2D6asGMFCyZFh06ZNLtV4xt4YoRGPs/tCV3gFAIVYA+eyrIG81KpVi9mzZ/POO+8gImRkZDBmzBhuvfVW2rRpw4cffpj73mnTptG6dWvCwsJ48cUXycjIYOLEiWzYsAGTycTbb79Nv3796NSpE88/n5WpPHHiRN544w0AEhIS6NChA23atOGBBx7ILQ3RpUsXxo4dy2233UbTpk3ZsmVLiZ/t0qVLucXl0tLSePzxx2ndujVt27YlNjYW+LOcdA69evVi06ZNQNbM/qWXXiIsLIwOHTrklrA+dOgQHTt25NZbb+Xf//537rGpqal07dqVdu3a0bp1a1auXAlklbZu0aIFw4YNo127dkyZMoWRI0fmHjdnzhxGjRpV4udx9g+mrDjDQilodeRce/HixW5duqGsGKGUhbPdcy6RHjtp9a/sOXnJpue8JdCPCfe3zLfNx8uDprV8OX0pjXOp1xDJSibzsVQCoFGjRmRmZpKYmMjKlSvx9/dnx44dXLt2jU6dOtGjRw/27t1LTEwM33//Pd7e3ly4cIGzZ88iInh6elK9enXi4uL4/PPP8fPzo3bt2jfJNmDAAN5++20iIyMZP348kyZNYtasWUBWLsEPP/yQa40UlpF48OBBwsPDSUlJ4cqVK7mup5w6/7t372bv3r306NGD33//vdj7dPnyZTp06MDLL7/MCy+8wJw5cxg3bhzPPfccQ4cOZcCAAfn6B1gsFlasWIGfnx/nzp2jQ4cO9O7dG8haFJ83bx7vvfcely9fpk2bNkybNo1KlSoxb968fEq0KHJ+MEVlvDrbn11YNin8aaHkYM9s0jvuuIM9e/ZQtWrVfDJcunSJzMxMfvrpJ3r06EF0dDTTpk0r1UTCHcmJmBoxYgQmk4no6GhmzZrl8PuR444qmMDnCPectgAKYMq2BhrnWgOXOXHxSu7aQM4C7tq1a/nkk08IDw/n9ttv5/z58+zfv5/169fz+OOP52r06tWrYzabqVKlCkFBQQQFBeHn58fMmTNZv379TZq/YGnmgQMHEhcXl7v/wQcfBKB9+/ZFloNu3LgxCQkJHDx4kFmzZjFkyBAgq2l3Tqnn5s2b06BBgxIVgKenJ7169brpmt9++y2PPPIIkL98tIjwr3/9izZt2tCtWzdOnDiRazU0aNCADh06AFClShWio6NZs2YNe/fu5caNG7Ru3bpYWXIwcqy2ESyUSpUq5Xudc+28bjdvb2/+/e9/28237Qo4ohFPaXGWO8olLICCM3VHUMXLA6XIjRRKSUvnRtIpzGYztWrVQkR4++236dmzZ77jvv7665ti8kNDQ6lcuXJuIa4ffviBDRs2sHjxYpYvX16mmiw55aDNZnOpMop79+7N448/DuSPPsqLh4dHvsEh7yykUqVKuZ+n4DULKyH96aefcvbsWXbu3EmlSpVo2LBh7vmqVKmS772DBw/mlVdeoXnz5rkylpa8PxgvLy+uXbtmCH+2MyyUoqyOonC2laS5mbzuqNdff52xY8eya9cuPv7449xeIfZAWwAlkGMNXDx/niFPDmXAP58kU6Bnz568//773LhxA4Dff/+dy5cv06NHDz7++OPcH/+FC/kToVNTU0lOTubee+9l1qxZJCQk5Nvv7+9PtWrVcs3QhQsX5loD5WHr1q00btwYyF92+vfff+fo0aM0a9aMhg0bkpCQQGZmJseOHcttNVkcnTp1ylcOO4fk5GRq1apFpUqViI2N5ciRI0We4/bbb+fYsWN89tlnudZEaTGC/7YoHG2hFGV1FIYRrCTNzTirsqpLWADO4urVq4SHh3Pjxg08PDzo83B/Hhz4JPvPpND/7wM5fPgw7dq1Q0QICAggJiaGu+++m4SEBCIiIvD09OTee+/NF4WTkpJCnz59SEtLQ0QK7Te8YMECnnrqKa5cuUKjRo2YN29emeTOWQPIWXeYO3cuAMOGDeOpp56idevWeHh4MH/+fLy8vOjUqVNu57FWrVrRrl27Eq/x5ptv8re//Y0333yThx56KHf7o48+yv33309ERATh4eE0b9682PP85S9/ISEhIV8XtNJgFP9tUTjSQinJ6oAsqyA9Pd0QVpLmZgq6nXLcUaNHj7bvha3pJ2mrR3l6AjuL1LQbsvfUJdl17KIcu3BZ0jMynC2SS3PffffJ+vXri9xv1P+DkujSpYuYTCZp27atrF27Vtq2bSsmk0mioqLsds3Vq1eLxWK5qUdvo0aNHCaDxrHgrJ7AFZUqXh40qeVDgK8XFy9fZ/+ZVFLSbjhbLJcjKSmJpk2bUrlyZbp27epscWyOM0z6gguJkDXznzBhgls2bNFYj80awliDvfoB2JvL19I5fvEq19IzqF7Fk7r+FsxFVObUlA9X+D8wClFRUcTFxd20kKibv7gvzu4HYFeMoJyKo6A18Lu2BmyK0b9/o+HuLRo1tsewFsChQ4fw9fWlRo0ahYYbGo0r19I5lmMNeHtSt6q2BqxBRDh//jwpKSmEhIQ4WxyNxpBYawEYNgooKCiI48ePc/bsWWeLUmpEhCtp6SSmpbPfpKjqXQlLJfuXRHZXLBYLQUFBzhZDo3FbDKsAKlWq5LIzv4RjSTy/dBcHElP5S0QQL913C/6VK5V8oEaj0TgQ7aOwA+HBVVnzzJ0M7dKYZTuP03NmHLH7Ep0tlqYUGKF8s0bjKLQCsBOWSmbG3t2cFcM64VfZg8fn7WDM0l0kX9WLxEbGCOWbNRpHoRWAnQkLrsrqZ+7k6ajG/O+nE/SYuZnYvdoaMCpGbzCj0dgSrQAcgJeHmTE9m7Ni2B1UrezJ4/N3MPrzXSRf0daAs3G1BjMajS2xSgEopUYqpX5VSv2ilFqklLIopUKUUt8rpfYrpZYopYovTViBaBNUlVXPdOKZ6FBiEk7QY9ZmNu4942yxKjRGKN+s0TiLcisApVQ94FkgQkRaAWagP/A6MFNEmgAXgX/aQlB3wcvDzOgezYgZ1olq3p78Y348oz5P0NaAk3B2RyaNxplY6wLyACorpTwAb+AUEA0sy96/AOhr5TXcktZB/qwafifPRoeyMuEk3WduZv0ebQ04AyM3mNFo7Em5FYCInADeAI6SNfAnAzuBJBHJ6RpyHKhX2PFKqSFKqXilVLwrJXuVldTUVCZPnkBwcABms4ng4AAmT55Aamoqnh4mRvVoxsqnO1G9iieDP4ln1JIEkq4UXc9dYx+M0CC8MHRYqsaeWOMCqgb0AUKAQKAKcE8hby201oSIzBaRCBGJCAgIKK8YhiY1NZUuXTqwefM0Jkw4x9q1woQJ59i8eRpdunQgNTUVgFb1sq2Brk1Ytesk3WfGsU5bAw7FqA1mdFiqxp5Y4wLqBhwSkbMicgP4H3AHUDXbJQQQBJy0UkaXZcaM6fj7H2TcuDRCQ8FshtBQGDcuDX//g8yYMT33vZ4eJkZ1b0rM052o6ePFE5/EM2LxT1y8rK0BR+Dv78/kyZOpX78+t912m2EKqemwVI09sUYBHAU6KKW8VVa1tq7AHiAWeDj7PQOBldaJWDaKc7k4mjlz3uORR9IoWMtOKejfP425c9+/6ZhW9fxZ+XQnRnRrwpqfT9F9Zhzf/LSCa+4AACAASURBVHraQRJXXGJiYqhfvz4rV65k9erVTmsQrsNSNY7EmjWA78la7P0R2J19rtnAWGCUUuoAUAP4yAZylorSulwcxcmT5ymqnFFISNb+wvD0MDGiW1NWDu9EgK8XTy7cybOLtDVgb4ww29Zhqa6LK67XWBUFJCITRKS5iLQSkcdE5JqI/CEit4lIqIj0E5FrthK2JMricnEEgYE1OHSo8H2HDmXtL46Wgf6sGt6Jkd2a8uXuU3SfuZmvf9HWgK0w4mxbh6W6Lq64XuNWmcDlcbnYkyeeGMaiRRYKtlwQgcWLLQwePLTEc1Qym3iuWxNWDb+T2n4WnvrvTp5Z9BMXtDVgNUadbbtjWKorzo7LihEsyLLiVgqgvC4XezFq1BiSkxszdaqF/fshPR3274epUy0kJzdm1KgxpT7XLYF+xDzdiVHdm/L1L6foMXMzX/9yyo7Suz9Gnm0bNSy1vLji7LgkjGhBlhW3UgDWulxsjY+PD5s2bScy8gWmTAng7rtNTJkSQGTkC2zatB0fH58yna+S2ZQVKjr8Tur4W3jqvz8y/LMfOZ/qMC+b07DXDNKos22jhqWWF1ecHZeEUS3IsuBWCsAWLpeSKGuUkY+PD+PHT+Lo0UTS0zM4ejSR8eMnlXnwz0uLun6sGNaJ53s05ZtfT9NjZhxf7rbOGjC6iW7PGaQRZ9uu3t/XHWbHJWFkC7LUiIjTH+3btxdbkJKSIu3bt5ToaIvMno2sW4fMno1ER1ukffuWkpKSYrPzz5mDrF+PzJlju/OXh72nLkmvt7ZIg7FrZNh/d8q5lLRyneeTTz4RQBYuXGhjCW1Dly5dBJCoqCi7nNtkMknbtm1l7dq10rZtWzGZTHa5liuTlJQkffv2laSkpBLfu3HjRvH29hayEkELfXh7e0tsbKz9Bbczq1evFovFku+zWSwWWb16td2vDcSLFWOv21gAqampzJgxndOnTxMbm8aIEYp77oFJk2qW2+VSEKNFGQE0q+PLimF3MKZnM9btOUP3mXF88XPZrQGjmeiOnEG6+mzbUZTFCnOL2XEpMaIFWWqs0R62elhrAThqZh4UVFPmzEFiY29+zJ6NBAcH2OQ65WXvqUty/9tZ1sDQ/8bL2WKsga5du+absXh6euZ7znl07drVgZ/gTyrSDNJVKI8V5szZsaNwpgWJtgAcNzM3WpRRQZrV8eV/Q+/ghbubsX5PIj1mxrF610mk4KIIxl/AqkgzSKNiCyvMpWfHpcSlLUhrtIetHtZaAI6amRvdAsjL76cvSe9sa+DJT+Il8dLN1kBxs2yjzK4rwgzSqJTGCjObzcV+F3p9xb6gLQDHzcwdEWVkK5rU9mX50DsYe3dzNu5NpMfMzawqYA0YNQQyLxVhBmlUSrLCPD09ycjIKPa7cOnZcQXALRSAo+L/bZnY5Qg8zCaGdmnMF8/eSf0aVXh20U889d+dJKak5b7H6AOsu8XDuxrFTRKaNm0KFB84EBMTw6hRozCZsoYaZxXZ0xSOWygAR83MbZ3Y5Sia1PZl+VMd+b97mhO77yw9ZsaxMuEEImL4Aba0M0ij5zG4MjmThLykpaWxd+9ewL1i+ysc1viPbPWwZRSQPeL/3Yn9Zy5Jn3e2SoOxa+SJBTvk3gf7y3/+8x/JyMgQEZH09HR54403pE+fPk6WtGwUzGMoS8y6pnhy/PihoaHi5eWlI7MMBFauATh98BcbKACRLCUwadJ4CQ4OELPZJMHBATJp0vjcwT9nf1BQTTGZlAQF1cy3vyKRnpEpH24+IE1e+lLaTPxGVvx4XDIzM50tllUUDFE0emKbK9GnT5/cSYIrBA5UBHImOMBPYsXYq6Sg38QJRERESHx8vN3On9MnwN//II88kkZISNbawKJFWb57I7tv7MmBxFReWLaLH48m0a1FbV55oBW1/CwlH2gAunXrxoYNG3Jfe3p6cv369dznHKpVq8aFCxecIaLbsmbNGvr160da2p9rSRaLhaVLl9KrVy8nSlZxWLhwIQMGDICsroyNynset1gDKAkjZvAagdBaPix96g7G3deCLfvP0n1mHCt+Oo4RJgUlUZo8BoCUlBTtn7YxRg8cqAjkWaOrac15KoQCMFqfACNhNikG39WIr567i9BaPoxcsosnPonnzKW0kg92IiWFKOaQnp6e+7fRKzO6CkYPHHBHikrKA6xyXVQIBWD0DF4j0CjAh8+f7JhtDZyj+4zNLN9pbGugqBDFwtCZw7ZDx/Y7nqIsXkAVekApqRAKwF55AkZqQG8LcqyBr0d0pmltX0Yv3cXgBca2BgpzRxTESIlt7oCO7Xc8pbV4y0qFUAD2yBMwWgN6WxJSswpLnuzI+F638O3BLGtgmUGtgYLuiODg4Nx92j9denQehfEpi8VbWiqEArBHBq+7LyybTYp/3BnCV891plkdX55fuot/zN/B6WRjWQMF3RENGzbM3a7906XHHVs2uiMFLV6rsSaG1FYPWzWEKY6S8gTKiisVhrOWjIxM+XjrH9Js3JfSasLXsmTHUcPmDeSNWRf5M7Ht3nvvrRCJYeVNgLNnwx0j4C6JgQWL6wGXxRmJYEAzICHP4xIwAqgOrAP2Zz9XK+lcjlAAtsZkUrJ+feEKYN06xGw2OVtEm5KUlCT39BsgD74TJw3GrpEBH30vJ5OuOFusUlNREsNK+zmN3g/C1rjL919wgoOzqoGKyD4RCReRcKA9cAVYAbwIbBCRJsCG7Nduh9Ea0NubVatW8dXST+jjd4SJ99/CD4cu0GNGHJ/vOGbItYGCGK3jmb0o7ec0ej8IW+Mu33/BBXhrsdUaQFfgoIgcAfoAC7K3LwD62ugahsKVSkPbgpwfzvx5HzOoUwhfj7iLWwL9eGH5zwyct4OTSVedLGF+KkJTcij/5ywpqsRsNrt05JSrfv8lLcbbfLHeGvMh5wF8DAzP/jupwL6LJR3vii4gdy9AVzoXgZL2/UdJ83FfScvxX8ui748YZm2gorSUtPZzFtZwp1KlSi7vLnHV778kV1XB/Ti7GBzgCZwDaksZFAAwBIgH4uvXr2+Le+dwbL2wbCTK8gM6cu6y/PXDbdJg7Br5+9ztcvyiMdYGKkrhMms+58KFC8XHx0dMJpNUrlxZTCaTmEwmt1gQdsXvv6TF+IL7jaAA+gBr87zeB9TN/rsusK+kc7iiBVARKMsPKCMjUz7Zdkha/DvLGvjMINZARWkpWd7PmTOgFPZwhwVho3//pV2ML26/WDF+22IN4BFgUZ7Xq4CB2X8PBFba4BoaJ1CWlpEmk+Kxjg35ZkRnWtfz5//+t5sBH//A8YtXHCx1fipK4bLyfk5/f3+GDh1aaEy5OywIG/37L21Rw9LuLzPWaA/AGzgP+OfZVoOs6J/92c/VSzqPtgCMS2EuAh8fn2L9wxkZmbLwu8NyS7Y18Ol251kDFaUpubWf0xXdJaXBFb7/ku79jBkzinPHZogzXUC2eGgFYFys+QEdPX9Z/jbnO2kwdo08Ome7HLtw2QES56eoxDBX63hWErb4nEZ3l5QHV/n+S7r3Re0H9otWABp7Ye0PKDMzU/67PcsauOXfX8nC7w4bYm3AHrh6tml5rD2NbSjp3he1H/hDtALQGJ1jFy7Lo3O2S4Oxa+SR2d/J0fOOtwbsjatnm7qCu8RdKeneF7UfuCRWjL0VohicK+FuJaZzCKrmzcJ/3sarD7bm5+PJ9JwVx8LtR8jMFGeLZjNcPdtU1/l3HiXd+6L2AxnWXLdC9AR2FSpK7+ITSVd5cfnPbNl/jo6NajDt4TYEV7dtnXNHUNq+xF27dmX9+vXOEFFTBMnJyQwaNIj58+fj7+9v+PMWhVJqp4hElPd4bQEYCHcvMZ1DvaqV+eQft/Hag63ZfSLLGvjku8MuZw1UtHo67oS9yl+7WlltrQAMhLv2Li6sfolSiv631eebkZ2JaFid8St/5ZE52zl63rl5A2WhpHo6ug2lcbGXu87V3IBaAZSAI33y7tq7uLhZUb2qlVnw+K28/lBr9py8RM9ZcSzY5jrWQFmS5TTOw17F4Vy16FwOWgEUg6PbPrpriemSZkVKKf56a5Y1cFtIdSas+pX+c7Zz5PxlR4pZboyebaqxn7vO1d2AWgEUg6N98u5SYrq8s6LAqpWZ//itTHu4Db+dvMTds7Yw79tDhrcGCvYl1m0ojYe93HUu7wa0JobUVg+j5gE4uu2ju5SYtkUp3pNJV2Tgx99Lg7FrpN/72+TQ2VTHfYAy4irZphr7ZTs7K4saZ3UEqwg42ifv4+PDpk3biYx8gSlTArj7bhNTpgQQGfmCS4WA2mJWVNe/MvMG3cr0h9vw2+lL3P1mHB9vNaY1ULBLk9lsZvTo0cTExDhZMk1B7OWuc1U3oFYAxeAMn7yPjw/jx0/i6NFE0tMzOHo0kfHjJ7nM4J+DLRZHlVL0iwhm3chI7mhck8lr9vDX2d9x6JxrrA1ojIe93HWu6gbUCqAY3MUnbw3WREHZalZUx9/CRwMj+E+/MPadTuGeN+P4aOshMgxoDWiMjb2ynV01i1pnAhdD3szc/v3/zMxdvNi9MnOLwtrM5KioKOLi4ggLC+P1119n7Nix7Nq1i8jISDZu3Fgumc5cSuNf/9vNhr2JRDSoxrSH29AowH2/A42mOHQmsB1xF598ebE2Csoes6LafhbmDoxgxl/C+P1MCve8uYW5W/7Q1oBGUw60BaApkuDgACZMOEdo6M379u+HKVMCOHo00fGCZZN4KY1/rdjN+t8SaVe/KtP7hdFYWwOaCoS2ADR2w+iZybX8LMwZEMGsv4Zz8Oxl7n1zC7PjDmprQKMpJVoBaIrEFTKTlVL0bVuPdSM7c1eTAF75ci8Pf7CNA4muXT5bo3EEWgFoisSVoqCyrIH2vNk/nEPnLnPvW1v4cLP9rYHCCt1pNPbAHv9rWgFoimTUqDEkJzdm4kQvZs2C/v0hOhr69IF9+3wYMsQ4CgCyrIE+4fVYO7IzXZoG8OpXe3no/W0cSEyx2zVdrfyvxnWxx/+a2ykAd+2o5Qx8fHxYs2Y9hw/7cfYsTJ0K69bBjBnQtGkqvXp1M+R9reVr4cPHsqyBw+cvc+9bW/lg80HSMzJtfi1XK/+rcV3s8b/mVlFAFaWjliOZPHkCmzdPY9y4/H0KRGDqVAuRkS8wfvwk5wlYAmdTrjEuZjff/HqGsOCqvPFwG5rU9i33+XQXMI2jKO3/moiowo4vDVZZAEqpqkqpZUqpvUqp35RSHZVS1ZVS65RS+7Ofq1lzjbJQUTpqWUNZLSRHNamxly89wNeLD/7enrcfacvR85e5762tvLfpQLmtAVcv/6txHUrzvwZYZdZa6wJ6E/haRJoDYcBvwIvABhFpAmzIfu0Q3LWjlq0oT38DR4WC2tOXrpTi/rBA1o6MJLp5LaZ9vY+H3t/G72fKvjbg8uV/NS5Daf7XgAPWXKPcCkAp5Qd0Bj4CEJHrIpIE9AEWZL9tAdDXGgHLgtHj1p1NeSwkR4WCOsKXHuDrxft/b8c7f2vLsYtX6fXWVt6NLbs1oLuAaRxFSf9rgFURDtZYAI2As8A8pdRPSqm5SqkqQG0ROQWQ/VzLGgHLgivErTuT8lhI9goFdVYrPaUUvdoEsnZkZ7rfUpvp3+zjwfe3se902X5Hrlr+V+N62PN/zRoF4AG0A94XkbbAZcrg7lFKDVFKxSul4s+ePWuFGH/iSnHrzqA8FlJOKOjUqRb274f09KwyEFOnZi2sjxo1plyyONuXXtPHi3cfbce7f2vHiYtXuf/tslkDrlr+V+N62PN/zRoFcBw4LiLfZ79eRpZCOKOUqguQ/VxosRgRmS0iESISERAQYIUYf2KvwcpdKI+FZK+CeEbxpd/Xpm6WNdAyyxp44L1t7D19qcTjXLX8r8b1sOf/mlVhoEqpLcBgEdmnlJoIVMnedV5EXlNKvQhUF5EXijuPLYvBpaamMmPGdObOfZ+TJ88TGFiDwYOHMmrUmAofAmrEkM41a9bQr18/0tLScrdZLBaWLl1Kr169HCrLl7tP8e+YX7iUdoNno5vwVJfGVDK7XaqMxo2wthictQogHJgLeAJ/AI+TZVV8DtQHjgL9RORCcefR1UAdgxH7G/z3v/9l6NChXLlyBS8vL65du4a3tzfvv/8+f//73x0qC8CFy9eZsOpXVu86SctAP97oF0aLunpWrzEmTq0GKiIJ2W6cNiLSV0Quish5EekqIk2yn4sd/DWOw4j9DYzmS69exZO3H2nLB39vx5lLafR+Zytvrt/PDTtkEWtKh663ZD/cKhNY43r07duXzp07M2LECEwmExkZGcyaNYstW7Y4van6hcvXmbjqV1btOsktdbOsgVsCtTXgaBYuXMiAAQNYuHChU6xCI+NUF5Ct0ApAY2S+/uU042J+IenKdYZHhzKsSyieHnptwJYkJyczaNAg5s+fj7+/f759UVFRbNq0iaioqHK3EnVXdEMYN0UXtTMOd7eqw7qRnbmvTV1mrd9Pn3e/5deT2h1hS/JmgjsrR6Qioi0AA6KL2hmXb349zUsrsqyBp6NCeTpKWwO2IO8s/9///je9evXiypUrRb5fl9zIQlsAboguamdcerasw/pRnbk/LJA3N+yn9ztb+eWEtgbKSnGz/OjoaD34OwitAAyILmpnbKp6ezLzr+HMGRDB+cvX6fvut8xYu4/r6TpSqLSUJhPcy8uLSpUq5TtO11uyLVoBGBBd1M416H5LbdaN7EzvsEDe2nhAWwNloDSZ4KNHj8bLy0vXW7IjWgEYEF3UznWo6u3JjL+GM3dABBcuX6fPu9/yn7X7uJae4WzRDE9JlS63bdtmqBwRd0QrAAOii9q5Ht1uqc26kZH0Da/H2xsP0Pvtb9l9XFsDJVFcpUtdb8n+6CggA2LEkg2a0rNx7xn+73+7OZd6naGRjXmmayheHmZni2VIoqKiiIuLIywsjNdff52xY8eya9cuIiMjdcx/KdBRQG6IEUs2aEpPdPParB0ZyQNt6/FO7AHuf3srPx/XfuvC0LN856ItAI3GjsTuTeT//rebs6nXeLJzI57r1kRbAxqboS0AjcbARDWvxTcjO/NQu3q8t+kgYWMWsfW3484WS6MBtALQaOyOf+VKTHs4jAH1U0i9ls5jCxJ47au9pN3QkUIa56IVgEbjILYs/ZCTHz2NT+IvfLD5IL3e3spPRy86WyxNBUYrAI1NMVoRO2fKU1i5A7l+hd8/ncCZz8ez98Ah+r6zhfCBE7U1oHEKehG4ApDTJnPOnPdy22Q+8cQwm7fJNFoRO2fLExsbW2xRM+XpTUCPJ6ncsiuNA6owvV8Y7epXs5s8GvdDLwJriiVnENy8eRoTJpxj7VphwoRzbN48jS5dOth0Jmy0InbOlqekcgeVPWDJ6N588o/buHo9g4ff38arX/6mrQGNw9AKwM1x5CBotCJ2RpCnpHIHXbp0oXPTAL4Z2Zm/3lqfD+P+4N63trDziF4b0NgfrQDcHEcOgkYrYmcUeYord5CDr6USrz7YmoX/vI1rNzJ5+INtvPzFHm0NaOyKVgBujiMHQaMVsTOKPGVpfH9XkwC+HnEXf7utPnO2HOLeN7ew88gFh8ipqXhoBeDmOHIQNFIRu9TUVJo2vYV583C6PGUtd+BrqcTLD7Tm08G3cy09k4c/+I6pa/Zw9bq2BjS2RUcBuTmTJ09g8+ZpjBuX3w0kAlOmWFDqNn7/fY9NooOMUsQuR44qVQ5w/Pg1goPh0UfJFwV06ZJrFNVLvZbOa1/9xn+3HyWkZhWmP9yGiIbVnS2WxiBYGwVklQJQSh0GUoAMIF1EIpRS1YElQEPgMPAXESl2RUspJUFBNe0SmljRKWpQXrTIwk8/CW3awGOPXbNZiGROyOncue/nKpXBg4c69HvNq/TS0mDpUvjqK0hMBB8fiIjozMqVX7jU/9m2A+d4YfnPnEi6yuN3hDCmZzMqe+qaQhUdIyiACBE5l2fbNOCCiLymlHoRqCYiY4s7T7NmSsaM0U3P7UVhg3KTJi1IT/+eiROv3WQZTJ1qITLyBcaPn+Q8oa0gODiACRPOERp68779+2HKlACOHk10vGBWcvlaOq99tZeF24/QsIY30x4O47YQbQ1UZIyoAPYBXUTklFKqLrBJRJoVd55mzZR8+KF7DD6uQkmD5OTJNTl27KzjBbMBZrOJtWsFcyET5PR0uPtuE+ku3LFr28FzjF3+M8cvXmXQHQ0Z07MZ3p4ezhZL4wScnQgmwFql1E6l1JDsbbVF5BRA9nOtwg5USg1RSsUrpeL/3KabnjuKkqKDTpw457TyDdZilOgfe3FH45p8/VxnBnRowLxvD3PPm1v4/g/X7BOdnJzMAw88QHKy7p7mDKxVAJ1EpB1wD/C0UqpzaQ8UkdkiElFQe+mm546hpEHS1xeHZ+7aCiNFI9mLKl4eTOrTikVPdEAE/jp7OxNX/cqV6+nOFq1MrFq1ipiYGFavXu1sUSokVikAETmZ/ZwIrABuA85ku37Ifi6Ts9UdZmiuwBNPDCsyRPLTTyEqCpe1xEaNGkNycmOmTrWwf3+W22f//iz3YnJyY0aNGuNsEW1Gx8Y1+HrEXQy6oyHztx3m7llb2F5Ga8CZs/CcXAjd6N05lFsBKKWqKKV8c/4GegC/AKuAgdlvGwisLO053WmGZnRGjRrDjz/CpEnkGyQnTYIzZ+Cf/3RdS6yitdT09vRgYu+WLB7SAYD+s7czYeUvXL5WOmvAkbPwwiqkAnz77bf5tnfr1s3usmisWARWSjUia9YP4AF8JiIvK6VqAJ8D9YGjQD8RKTaVsWnTrCgg3fTcsdSrV5Nbbz3P999nhUjWqgX33AP9+sHx464bLVORuXI9nWlf72P+tsMEV6/MtIfC6Ni4eIs6KiqKTZs2ERUVZfdG7CVVSAXw9vbmiy++oEuXLnaVxR2wdhEYEXH6A5Dg4ACZNGm8pKSkiK1ISUmRSZPGS1BQTTGZlAQF1bT5NVyZSZPGS3S0RTZuRGJj/3x88QXSuLFZqlf31vfNRdl+8Jzc+ep6aTB2jbywZKekpt3I3de1a1chK4BDAPH09Mz3nPPo2rWrXWTbuHGjeHt757tWzsPb21tiY2Ptcl13BIgXa8Zeaw621aN9+/Y2vzEpKSnSvn1LiY62yJw5yPr1yJw5SHS0Rdq3b6kHM8l/j2bPRtatQ956C6lRQ0mnTkrft3JgpEnH3PmfSLXowdJw7Grp9NoG+fbAWREpfgB21EC8evVqsVgs+a5psVhk9erVdrumO2KtAnDbWkDOrgXvChTmKx8/3puWLc1MmSL6vpURR/ZeKA3/nf8xFzfOpe6eJXiYFH+b8z3jYnZz2x13FdunwBEumNJUSNXYH7dVAEaoBe8K+Pj4MH78JI4eTSQ9PQMfH28eeyxd37dy4OxJR1ELrDu/+Zy4f93DpR0xLPzuMGEvfo5XcOsS+xTYk7JUSHVFXCW/wW0VgFFqwbsSqampnDp1jnHjoGtX6N8fPvkErl7N2q/vW/E4e9Lx0ksv5ZvVX79+PfdZ0q9xceNckpaNp2a1qvxt7vd8suc6HpV9nDILL2uFVFtj7wHaVfIb3FYBuHs2qK3JcV/cfjtMnQpr12Y9//EHjBqVpQT0fSseZ086SmpB6e3tzcqPZrLp/3ryxF0h/HDeE/+/vs4tUQ84fBYeExPDqFGjMJmyhiCz2czo0aOJiYmx+7XB/gO0q+Q3uK0CqAjZoLYkx30xdSr53BcTJkDt2lkVNfV9Kx4jTDpK04LSUsnMS/fdQpOjq6hZvRopEY+zMaUOG7d8V+ZZuKu4Ogpi6wHaVfMb3FYBVKRsUFtQnPvi0Udh+XL0fSsBo0w6SrvAum7RbLZP7M2Qzo1YsuMo9739Lbf1HlCmWbiruDrsPUAX537Lwdvbm3HjxlnxKeyANSFEtnrYIwxU5M+QvODgADGbTXbJNXAXTCYl69fnzwfIeaxbh5hMSt+3EigsrHb2bMeH0Hbp0kVMJpO0bdtW1q5dK23bthWTySRRUVFFHhN/+IJEvRErDcaukReX75JLV6+X+lpAsec2Ao4IfXVGfgM6DyALI8VfuyJBQTVlzpzCFcDs2VmJepqSMcKko0+fPvKf//xHMjIyREQkPT1d3njjDenTp0+xx129ni6vfLlHQl5cIx1fWS+b9iXe9B5nJ5FZgyMGaEfnN7iFAqhUycOqQVsnfVlPUVnBGzdm3cdJk8Y7W0SNg/jxyAWJzrYGXli6S5LzWAP2nkknJSVJ3759JSkpyUafJj9FDdCLFy+2yXUXLlwoPj4+YjKZpHLlymIymcTHx0cWLlxoo0+QH2sVgCHWAAID00uVNJOamsrkyRMIDg7AbDYRHBzA5MkTePXVl3XSl5XoNRNNDm3rV+OLZ+/iqcjGLN15jJ4z49i0L6smVGkijaxJIrP3mkJR6yOxsbE2ua7L5TdYoz1s9WjatOQZZ3GzfB8fs7z9tnZfWIsR3BcaY/HT0YvS9T+bpMHYNTJmaYIkXcmyBuzl6rD3mkJR6yNVq1a1yXXL634rL7iDCyivAihq0C7ORdGxIzJoUNELmGazyUa3W6O5GXdff7p6PV1e++o3CXlxjdz+8nrZuPdMqV0dJd0bR68p5AzQ0dHR+c6vlHKZtYy8WKsArOoJbCtyegLnUFjf1pJ62L70Enz+eeH7dFlj9yWn4f2cOe/lNrx/4olhjBo1xiElxXMS6Pz9D/LII2mEhGTF/C9a5H6lzXcdS+L5pbvYn5iKT+Ju9i5+hTbNQ3n99dcZO3Ysu3btIjIyMrekdGnuzY4dO5xSHtpdylI7uyewXSgsaaakLMtz5wrvbqWTl9wXIxRfc3b9H0cSFlyVNc/eydNRjUkNaEmzEZ8w/dMviyzlUJp7Y+81haJw1nWNhuEUQFGDdklZllWqmPUCZgXDCRIy3gAADcNJREFUCIOvs+v/OBovDzNjejZn1TN3ERRQnX8u2Mnoz3eRei3zplIOpb03pcletgfOuq6RMIQCuHat5EG7pCzL554bU2FaAGqyMMLg6+z6PwUpKlLO1tZQm6CqrHqmE89EhxKTcIIeszazce+ZfO8py71xVnnoil6W2hAK4ORJjxIH7ZLCFF988aV8ZY2PHk1k/PhJevB3Y5w1+OYdZEXkpqqpOTi6eJ6jXWJeHmZG92hGzLBOVPP25B/z4xn1eQLJV24AZauN5KzwSZcL27QxhlAAbdqElThoV7RG35qScUbxtYKD7Lp18Oqr+aumgnPWn5zlEmsd5M+q4XfybHQoKxNO0n3mZtbvOVOm2kjOKg/t7LLUzsYQUUARERESHx/vbDE0LsbkyRPYvHka48bldwOJZFmGkZEvMH78JIddc+LELMujU6esAc7RUUAlRco5IhrulxPJPL90F3tPp9CrVS22vTOMqt6/5IsCmj8fjh8P4Pvvf6ZOnTp2lcfdsTYKSCsAjcuSN8ywf/8/Bxh7Dr4lDbLPPgsBAQEMHjzUYaGoOZjNJtauFczmm/cVFlptL66nZ/JO7AHeiz2Av8VM4pdvcOm39aSmQq1acPvtcPGiF5cvh2rr3UqsVQAethRGo3EkOW7BGTOmM2XK+7l5APYcfEtad7hxw+S0nJMsl1jhysmR6xGeHiZGdW9Kj1tqM+CdbzB3GcE9j4Tx9xaz8fFMAUDkGlOnZrmlbG2laUqP1WsASimzUuonpdSa7NchSqnvlVL7lVJLlFKe1oup0RROwZ7G9l78N0LTl6IwSj+CHFrV8+fsZ8/Sufqn7Dh9F//a+h47z3QA3DdM1tWwxSLwc8BveV6/DswUkSbAReCfNriGxk1wVJiivTDaIJsXIxb0O3k8kYHtFzG+40j8vS7y9k/j+GDX86Re99U9pg2AVQpAKRUE3AfMzX6tgGhgWfZbFgB9rbmGxn0wQuautRhxkM3BiJFyORZTA79DTOg4kgdC/8uO03fyr63v8dXuji7XY9rVJzAFsWoRWCm1DHgV8AWeBwYB20UkNHt/MPCViLQq5NghwBCA+vXrtz9y5Ei55dC4Bs6I2rEHOfWH5s51zLqDK1PYd370Ughzdz/H0ZRQQsznWf6v/lSvYnxPsRHrPjmtFpBSqheQKCI7824u5K2FahgRmS0iESISERAQUF4xNC6EETJ3bYGj1x1cmcIspmtnDuER+xLeB9dyTGrSY+Zmvv7llLNFLZHy5lkY2WootwWglHoVeAxIByyAH7AC6AnUEZF0pVRHYKKI9CzuXDoMtGJglDBFjWMpzmI6lpLJmGW7+OXEJXq1qcuk3i2p4ePlbJELpTx5Fva2GpxmAYjI/4lIkIg0BPoDG0XkUSAWeDj7bQOBleW9hsa9MHIEjcZ+FGcxtajrx4phnXi+R1O++fU0PWbG8eVuY1oD5Sk9YoSChcVhj1IQY4FRSqkDQA3gIztcQ+OCGDmCRuM8KplNDI9uwupn7iSwamWGffojT3/6I+dTrzlbtHyUZwJjdLenTRSAiGwSkV7Zf/8hIreJSKiI9BMRY32LGqdh5AgajfNpXsePFcPuYEzPZqzbc4buM+P44mfjWAPlmcAYrVpsQQxRDE5TMTBimKLGWHiYTTwdFcrqZ+4kqFplnv7sR4Z9upNzBrAGyjOBMbrbU9cC0mg0hiQ9I5PZW/5g1rr9VPEyM7lPK3q1qYsq6E9xIGUNAbZ36LMuBqfRaNya/WdSeH7pLnYdT+bulnWY0rcVAb7GjBQqiL0LFrplT2CN+2DkGGiNa9Ckti/Lh97B2Lubs3FvIj1mbmbVrpMYYfJaEkZ3e2oLQGM3jJg5qXFt9p9J4fllP7PrWBI9W9ZmSt9W1PK1lHygm6ItAI1hMXoMtMb1aFLbl+VPdeT/7mlO7L6z9JgZx8qEEy5hDRgRrQA0dsPoMdAa18TDbOLJyMZ8+eydNKxRhecWJ/Dkwp0kpqQ5WzSXQysAjd0wegy0xrUJrZW1NvCve5uz6fezdJ8RR8xP2hooC1oBaOyG0WOgNa6P2aQY0rkxXz57F40DqjBiSQJPfLKTxEvaGigNWgFo7IYu/aBxFKG1fFj61B28dG8Ltuw/S/eZcaz46bi2BkpARwFp7IYzmrZrNAfPpvLCsp/ZeeQi3VrU4uUHWlPbzz0jhXQUkMawGD0GWuOeNA7w4fMnOzLuvhZs2X+O7jM2s3yntgYKQ1sAGo3GbTl07jJjlu4i/shFopvX4pUHWlPH332sAW0BaDQaTRGE1KzCkic7Mr7XLWw7eI7uMzezNP6Ytgay0QpAo9G4NWaT4h93hvDVc51pXseXMct+5h/zd3A6WUcKaQWg0WgqBCE1q7BkSEcm3H8L3/1xnu4zN/N5BbcGtALQaDQVBpNJ8XinEL5+rjMt6vrxwrKfGTRvB6eSrzpbNKegFYBGo6lwNKxZhcVPdGDi/bfww6EL9JgRx+c7Kp41oBWARqOpkJhMikGdQvh6xF3cEujHC8t/ZuC8HZxMqjjWgFYAGo2boXswlI0GNaqw6IkOTO7Tkh2HLtBjZhyLfzhaIawBnQeg0bgRugeDdRw9f4UXlu9i+x8XuKtJTV57qA31qlZ2tlhFovMANBpNLroHg3XUr+HNZ4M7MKVPS3YeuUjPmXEscmNroNwKQCllUUr9oJTapZT6VSk1KXt7iFLqe6XUfqXUEqWUp+3E1Wg0xaF7MFiPyaR4rGNDvhnRmdb1/Pm//+1mwMc/cPziFWeLZnOssQCuAdEiEgaEA3crpToArwMzRaQJcBH4p/ViajSa0qB7MNiO4OrefDr4dqb0bcXOIxe5e9YWPvvevayBcisAySJnValS9kOAaGBZ9vYFQF+rJNRoNKVG92CwLSaT4rEODfhmRGfaBPnzrxW7eeyjHzh2wT2sAasWgZVSZmAnEAq8C0wHtotIaPb+YOArEWlVyLFDgCHZL1sBv5RbEOdTEzjnbCGsQMvvPGwte2CVKtSpVw9VcMeJE8jly5wGTtrweq5878H15W8mIr7lPdjDmiuLSAYQrpSqCqwAWhT2tiKOnQ3MBlBKxVuzku1stPzOxZXld2XZQcvvbJRSVoVP2iQKSESSgE1AB6CqUipHsQRh29mGRqPRaGyENVFAAdkzf5RSlYFuwG9ALPBw9tsGAiutFVKj0Wg0tscaF1BdYEH2OoAJ+FxE1iil9gCLlVJTgZ+Aj0pxrtlWyGEEtPzOxZXld2XZQcvvbKyS3xCZwBqNRqNxPDoTWKPRaCooWgFoNBpNBcXhCsAdSkgopcxKqZ+UUmuyX7uS7IeVUruVUgk5IWRKqepKqXXZ8q9TSlVztpxFoZSqqpRappTaq5T6TSnV0VXkV0o1y77vOY9LSqkRriI//H875xNiVRXH8c+XBGGGjJxQJqcwQWoRNU1h/oGhNCpDDIJACdpEbYKsTRRR0L6FChGCVtDCQImQWVhim2hRpDT2rAYNRacmnyi5qI3mt8U5Dy+Pee81q/tO7/eBy73ncOfxeef+7vvNOffcA5Jey/dtQ9L+fD8XEf+SdmTvk5JezXV93faSPpTUlNSo1M3rrMRuSaclnZA00evz6+gB/B+WkNhBmvHUoiR3gEdtj1fmP78BHM3+R3O5X9kFHLZ9D3A/6ToU4W97Jrf7OPAg8Dfp/Zki/CWtAF4BHsovd94EbKOA+Jd0L/AisIYUN1skrab/2/5j4Mm2uk7Om4HVeXsJ6L3wk+3aNmAIOA48THobb1GuXwd8UadbF+ex3OgbgSlApbhnv7PAbW11M8BoPh4FZur27OC+BDhDnrxQmn+b8+PANyX5AyuA88BS0gzCKeCJEuIfeBbYWym/DbxeQtsDK4FGpTyvM7AH2D7feZ22Wp4B5CGUH4AmcAT4FfjT9rV8yiwp2PqRnaTAuZ7LI5TjDunN7C8lHcvLcQAstz0HkPfLarPrzirgIvBRHoLbK2mYcvyrbAP25+Mi/G3/BrwHnAPmgCukpWBKiP8GMClpRNIQ8BRwB4W0fRudnFsJukXPa1FLArD9j1M3eIzUJfvPS0jUiaQtQNP2sWr1PKf2nXuFDbYnSN3FlyVN1i20ABYBE8AHth8A/qL/uuw9yWPkW4EDdbsshDzW/DRwF3A7MEyKo3b6Lv5t/0waqjoCHAamgWtd/6g8FvxbVOssIJe3hMQGYKuks8CnpGGgnZThDoDt3/O+SRp/XgNckDQKkPfN+gy7MgvM2v42lw+SEkIp/i02A8dtX8jlUvwfA87Yvmj7KvAZsJ5C4t/2PtsTtieBy8Apymn7Kp2cZ0m9mhY9r0Uds4CKXULC9pu2x2yvJHXhv7L9HAW4A0galnRz65g0Dt0ADpG8oY/9bf8BnJd0d67aBPxEIf4VtnNj+AfK8T8HrJU0JEncaP9S4n9Z3t8JPEO6BqW0fZVOzoeA5/NsoLXAldZQUUdqeKBxH2mJiBOkH593cv0q4DvgNKlrvLjuhy89vscjwFRJ7tlzOm8ngbdy/QjpwfapvF9at2uX7zAOfJ/j53Pg1sL8h4BLwC2VupL83wV+yffuJ8DiguL/a1LCmgY2ldD2pCQ1B1wl/Yf/Qidn0hDQ+6Rnqj+SZmt1/fxYCiIIgmBAiTeBgyAIBpRIAEEQBANKJIAgCIIBJRJAEATBgBIJIAiCYECJBBAEQTCgRAIIgiAYUP4FKMs2FGsuVGcAAAAASUVORK5CYII=\n",
+ "text/plain": [
+ "
"
+ ]
+ },
+ "metadata": {
+ "needs_background": "light"
+ },
+ "output_type": "display_data"
+ }
+ ],
+ "source": [
+ "# Plot Boundary\n",
+ "utils.plotDecisionBoundary(plotData, theta, X, y)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "\n",
+ "#### 1.2.4 Evaluating logistic regression\n",
+ "\n",
+ "After learning the parameters, you can use the model to predict whether a particular student will be admitted. For a student with an Exam 1 score of 45 and an Exam 2 score of 85, you should expect to see an admission\n",
+ "probability of 0.776. Another way to evaluate the quality of the parameters we have found is to see how well the learned model predicts on our training set. In this part, your task is to complete the code in function `predict`. The predict function will produce “1” or “0” predictions given a dataset and a learned parameter vector $\\theta$. \n",
+ ""
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 14,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "def predict(theta, X):\n",
+ " \"\"\"\n",
+ " Predict whether the label is 0 or 1 using learned logistic regression.\n",
+ " Computes the predictions for X using a threshold at 0.5 \n",
+ " (i.e., if sigmoid(theta.T*x) >= 0.5, predict 1)\n",
+ " \n",
+ " Parameters\n",
+ " ----------\n",
+ " theta : array_like\n",
+ " Parameters for logistic regression. A vecotor of shape (n+1, ).\n",
+ " \n",
+ " X : array_like\n",
+ " The data to use for computing predictions. The rows is the number \n",
+ " of points to compute predictions, and columns is the number of\n",
+ " features.\n",
+ "\n",
+ " Returns\n",
+ " -------\n",
+ " p : array_like\n",
+ " Predictions and 0 or 1 for each row in X. \n",
+ " \n",
+ " Instructions\n",
+ " ------------\n",
+ " Complete the following code to make predictions using your learned \n",
+ " logistic regression parameters.You should set p to a vector of 0's and 1's \n",
+ " \"\"\"\n",
+ " m = X.shape[0] # Number of training examples\n",
+ "\n",
+ " # You need to return the following variables correctly\n",
+ " p = np.zeros(m)\n",
+ "\n",
+ " # ====================== YOUR CODE HERE ======================\n",
+ " p = []\n",
+ " for i in range(m):\n",
+ " if sigmoid(np.dot(theta.T,X[i,:])) >=0.5:\n",
+ " p.append(1)\n",
+ " else:\n",
+ " p.append(0)\n",
+ " \n",
+ " \n",
+ " \n",
+ " # ============================================================\n",
+ " return p"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "After you have completed the code in `predict`, we proceed to report the training accuracy of your classifier by computing the percentage of examples it got correct."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 15,
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "For a student with scores 45 and 85,we predict an admission probability of 0.776\n",
+ "Expected value: 0.775 +/- 0.002\n",
+ "\n",
+ "Train Accuracy: 89.00 %\n",
+ "Expected accuracy (approx): 89.00 %\n"
+ ]
+ }
+ ],
+ "source": [
+ "# Predict probability for a student with score 45 on exam 1 \n",
+ "# and score 85 on exam 2 \n",
+ "prob = sigmoid(np.dot([1, 45, 85], theta))\n",
+ "print('For a student with scores 45 and 85,'\n",
+ " 'we predict an admission probability of {:.3f}'.format(prob))\n",
+ "print('Expected value: 0.775 +/- 0.002\\n')\n",
+ "\n",
+ "# Compute accuracy on our training set\n",
+ "p = predict(theta, X)\n",
+ "print('Train Accuracy: {:.2f} %'.format(np.mean(p == y) * 100))\n",
+ "print('Expected accuracy (approx): 89.00 %')"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "*You should now submit your solutions.*"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 16,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "#grader[4] = predict\n",
+ "#grader.grade()"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "## 2 Regularized logistic regression\n",
+ "\n",
+ "In this part of the exercise, you will implement regularized logistic regression to predict whether microchips from a fabrication plant passes quality assurance (QA). During QA, each microchip goes through various tests to ensure it is functioning correctly.\n",
+ "Suppose you are the product manager of the factory and you have the test results for some microchips on two different tests. From these two tests, you would like to determine whether the microchips should be accepted or rejected. To help you make the decision, you have a dataset of test results on past microchips, from which you can build a logistic regression model.\n",
+ "\n",
+ "First, we load the data from a CSV file:"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 17,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "# Load Data\n",
+ "# The first two columns contains the X values and the third column\n",
+ "# contains the label (y).\n",
+ "data = np.loadtxt(os.path.join('data2.txt'), delimiter=',')\n",
+ "X = data[:, :2]\n",
+ "y = data[:, 2]"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "### 2.1 Visualize the data\n",
+ "\n",
+ "Similar to the previous parts of this exercise, `plotData` is used to generate a figure, where the axes are the two test scores, and the positive (y = 1, accepted) and negative (y = 0, rejected) examples are shown with\n",
+ "different markers."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 18,
+ "metadata": {},
+ "outputs": [
+ {
+ "data": {
+ "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZAAAAEGCAYAAABLgMOSAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjMsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+AADFEAAAgAElEQVR4nO2dfZgU5ZXof6cHZ8YRHRHGRDMYiLhrNBtBUGP2xmEAleSygrlqIGsiUdfE5NnokvVrYSGA2fUTMYlxLxBFyQ1wIYkiXiPggBqNWccofj4GhIQQjQI6ZJAMn+f+UdXQ03T3VHfXZ/f5Pc/7dHXVW1Wnq7vr1HnPec8RVcUwDMMwiiUVtQCGYRhGMjEFYhiGYZSEKRDDMAyjJEyBGIZhGCVhCsQwDMMoiV5RCxAm/fr10wEDBkQthmEYRqJ44YUXtqpqU/b6qlIgAwYMoL29PWoxDMMwEoWI/CHXehvCMgzDMErCFIhhGIZREqZADMMwjJKoKh+IYRiGF/bs2cPmzZvp6uqKWpRQqa+vp7m5mcMOO8xTf1MgRtWyY8cOZs26nblzf8Tbb2/j+OP78k//9E0mTbqO3r17Ry2eESGbN2/myCOPZMCAAYhI1OKEgqqybds2Nm/ezMCBAz3tYwrEqEp27NjB8OGfobHxLaZN62LgQNi4cSsLF97GsmU/Y82a50yJVDFdXV1VpTwARIS+ffuyZcsWz/uYD8SoSmbNup3GxreYMqWLQYOgpgYGDYIpU7pobHyLWbNuj1pEI2KqSXmkKfYzmwIxqpK5c3/EhAldZP9fRGD8+C7mzbs3GsECYseOHcyYMY3+/ZuoqUnRv38TM2ZMY8eOHVGLVjFs376dCy+8kO3bt0ctSmiYAjGqkrff3ka+Yd6BA53tlUJ6uO7JJ29j2rStrFihTJu2lSefvI3hwz9jSsQnli1bxkMPPcQjjzwStSg5eeqppzj99NPp1asXS5cu9eWYpkCMquT44/uycWPubRs3OtsrBRuuC4f77ruv22vcOOGEE5g/fz5f/vKXfTumKRCjKvmnf/omCxfWk12QUxUWLarnyiuvjkawAKi24bqwGDVqFCJyoD377LMAPPPMM93Wjxo1quhj//u//zt33333gfeTJ0/m+9//flnyDhgwgE9/+tOkUv7d9k2BGFXJpEnXsX37idx8cz3r1sHevbBuHdx8cz3bt5/IpEnXRS2ib1TTcF2YTJ48mYaGhgPvd+/e3e0VoKGhgSlTphR97CuuuIIHHngAgP3797No0SL+8R//8ZB+n/vc5xg8ePAhbdWqVUWfsxQsjNeoSnr37s2aNc8xa9btzJx574F5IFdeeXXFzQNxhuu2MmjQodsqbbguTFpbW1m+fDljxoxh586dh2xvaGjg0UcfZfjw4UUfe8CAAfTt25cXX3yRd999lyFDhtC376Hf09NPP12K6L5hCsSoWnr37s3UqdOZOnV61KIEijNcdxtTpnQfxqrE4bqwaW1tZfHixVx88cXdZq3X19ezePHikpRHmiuvvJL58+fz5z//mcsvvzxnn8997nN0dnYesv6OO+4oaeisWEyBGEaFM2nSdSxb9jNuvvktxo9PT5p0lEelDddFQUdHB7169SKVSlFXV8euXbvo1asXHR0dZR33wgsvZOrUqezZs4ef/vSnOftEbYGYD8QwKpz0cF1Ly/XMnNnE6NEpZs5soqXleptx7wM//vGP2blzJ6eddhoPP/wwp512Gjt37iw7Gqu2tpbW1lYuueQSampqypbz+eefp7m5mSVLlvD1r3+dU089texjmgKpUmxiWfEk+Zqlh+s2bXqPvXv3sWnTe0ydOt2Uhw80NjZy++23097ezrnnnsvzzz/PbbfdxlFHHVXWcffv389zzz3HFVdc4YucZ5xxBps3b+bDDz9k27ZtvPbaa+UfVFUja8B9wHvAq3m2C/B9YD3wMnB6xrbLgHVuu8zL+YYOHaqGamdnpw4deqqOGFGvc+eiq1ahc+eiI0bU69Chp2pnZ2fUIgZCZ2enTp8+VZub+2kqJdrc3E+nT5/q6fNW6zWrVl5//fVIz//aa6/pwIEDddKkSaGfO9dnB9o1xz01agtkPjC6wPbPAye57SrgXgAROQaYBpwFnAlME5E+gUpaQVTjxLJyZ2NX4zUzouOUU05hw4YN3HnnnVGLUpBIFYiqPgW8X6DLWOBBVwk+BxwtIscB5wMrVfV9Vf0AWElhRWRkENeJZUHmEipXAcT1mhlGlERtgfTEx4A/Zrzf7K7Lt/4QROQqEWkXkfZi0hRXMnGdWBZkLqFyFUBcr5lhREncFUiu3MJaYP2hK1XnqOowVR3W1NTkq3BJJa55oILMJVSuAojrNTOMKIm7AtkM9M943wy8XWC94YG45IEKMpdQNuUqgLhcM8OIE3FXIMuAr4rDZ4DtqvoO8Dhwnoj0cZ3n57nrDA/EJQ9UkLmEsilXAUR5zZIcPlwNJOX72bVrF1/60pcYNGgQZ511Fr///e/LP2iu0KywGrAQeAfYg2NVXAF8A/iGHgzjvQd4C3gFGJax7+U44b3rga95OZ+F8R4kHdLav3+T1tSktH//Js8hrdnHKCUsNk1bW5s2NDQozhBkt9bQ0KCrV68u4dPlljUdhjtnDrpyJTpnTnFhuF6vmR/XJZfcFj4cHl7DeJP0/dxzzz369a9/XVVVFy5cqJdccknOfsWE8UaqQMJupkD8w88/ziOPPKL19fXdlEd9fb0+8sgjvstcrtL0cg4/byjTp0/VESPqta0NXb36YGtrc445ffpU32Q3DuJVgQT1/UyZMkVnz5594P2//du/6d13313SsdKcd955+uyzz6qq6p49e7Rv3766f//+Q/olaR6IESJ+mtp+zovIzCV0+OGHk0qlfMkllE0Ys7H9ni9i4cPxJqjvJ4h07n/605/o399xHffq1YvGxka2bSsvetAUSJXgd1lTP/84QeUSigK/bygWPuyNqPwQQX0/mencV6xYUTCd+0svvXRIyxV4otkOQECyf6hFYgqkSvD7ydjPP05QuYSiwO8bioUP90yUNd+D/H7S6dzvv//+guncvVogzc3N/PGPzvS5vXv3sn37do455piS5QNTIFWD30/Gfv5xHnroISZNmnSg1GZNTQ3f+c53eOihh4qSKQ74fUOx8OGeiTLNTJDfz4UXXsgvf/lLnn/+ec4///ycfYqxQC644IIDw2JLly5lxIgRZoEY3vD7ydhubLnx+7rEJeQ6zkTpJwry+/E7nfsVV1zBtm3bGDRoELNmzeKWW24p+5iSa1ysUhk2bJi2t7dHLUYk9O/fxLRpucuarlsHM2c2sWnTe56Plx42aGzMXaSoWutMBHFdduzYwaxZtzNvXmWX3i2VmpoUK1Youe6xe/fC6NEp9u7dV9Qx33jjDT75yU966hvU97N//35OP/10lixZwkknnVTycYol12cXkRdUdVh2X7NAqgS/n4ytSFFugrguVsujMFH7iYL4fl5//XUGDRrEyJEjQ1UexWIWSJVgFoNRqcyYMY0nn8xd8/3mm+tpabm+6Lr3xVgglYZZIMYhmMVQHEGmlk8qcU3ZEZQfopoertMU+5nNAjGMHCxYsICvfvWrLFiwgEsvvTRqcSIn04KdMOGgBbtwYTwsWL/9EBs3buTII4+kb9++ZUcqJQVVZdu2bXR2djIwK+ImnwViCsSoKLZv387EiROZP38+jY2NJR+ntbWVNWvW0NraSltbm48SJpMghonizJ49e9i8eTNdXV1RixIq9fX1NDc3c9hhh3VbbwoEUyDVQKmWw6hRo3jiiScOvK+trWX37t0HXtOMHDky5yStSsfvKD4jWZgPxIgdQfgZSi1KFWZq+SRiKVWMXJgCMSLDjxK2fhWlam1tZfny5d2USCYNDQ08+uijDB8+vGRZwX9HdFiO7ahDZY14YgrEiAw/Stj6aTm0trayePFi6uvru62vr69n8eLFvigPP3M2hZkDyjIPGLkwBWKERhAlbP22HIJMLe93zqYwc0D5GSob13BgowRyFQmp1GYFpaKlUPVByqxC6FdRquHDh2sqldIhQ4boihUrdMiQIZpKpbS1tbVombJpbu6nc+d2LzyUbnPmoP37N0V6vJ7wq4plUir4GQchjgWlRGS0iLwpIutF5MYc2+8SkZfc9jsR6cjYti9j27JwJTdKIUg/g1+WQ5Cp5f12RIft2PYjZUeUmXMN/4lMgYhIDU69888DpwATROSUzD6q+i+qOlhVBwM/AH6esfmv6W2qekFoghtlEZSfwa+iVEGmlvfbEZ1Ex7afmXNtKCx6orRAzgTWq+oGVd0NLALGFug/AVgYimRGoAThZ0hCUSq/HdFJdGz7ZTVFWUTKOEiUCuRjwB8z3m921x2CiHwcGAhkTgmuF5F2EXlORMblO4mIXOX2a9+yZYsfchtlEkQJ2wceeICnn36azs5OIPqiVLnmuPidsymJtUL8sppsKCweRKlAciWYyTctfjywVFUzk/qfoM7MyC8Ds0XkxFw7quocVR2mqsOamprKkzgBBGnW+3XscqyFfJMP/ZhT4ie55PE7oaWfxwsreaRfVlOURaSMDHJ51sNowNnA4xnvbwJuytP3ReCzBY41H7iop3NWehRWkBEucYmeefDBBxXQBQsWdFs/fPhwBYqKluro6NBx48ZpR0eH32KWJE+U5LuufpP5O5ozB1250okYK/Z3lEqJrlqVOwJt5Uq0piYV6OeoNohhFNbzwEkiMlBEanGsjEOiqUTkb4E+wK8z1vURkTp3uR/w98DroUhdJkFaCEGa9XEZMkgPc337298ue06Jn1ZLEHNcwsSPSZ1pClkzfllNSQwgqEhyaZWwGvAF4HfAW8Bkd90M4IKMPt8Fbsna77PAK8Ba9/UKL+eL2gIJ+ik+yHkBYc85SDNy5Mhucztqa2sV0F69ehWcT4KHOSV+WglBznEJgnzXNf2abiNHjiz62GFYM9OnT9URI+q1ra37b7Gtzfk/TZ8+NbBzVyPksUAiVSBht6gVSNA/+iDN+qiGDLzcmL3erIO8afYka5yUh2qwCi+M4Tu/hsL8kGP69Kna3NxPUynR5uZ+RU+uTAL5FIilMgmRoB1/QZr1UQ0Z9DT5sK6ujtra2m7r8s0pCTrjbtC5tPzEz0mdUQzfxaHCpoUSWy6sUAl65nCpES5eInCinHNQ6Mb8rW99i9ra2gNV40Qk75ySMDLuBplLy2/8UnhRpcL3MjM+qT7HpGAKJESCfoovdV6AF2dy1HMO8t2YH3vsMXbu3MkRRxwBwBFHHFFwTknQVkIQc1yCxA+FF1Yq/GIJ2kKwUGJTIKES9FN8qWa9lwicqIcMsm/MDQ0N7NixgzfeeIP9+/d3e+rdv38/q1evzjtsEqSVkIQZ8Zn4pfDiOHwXtIVgRbYwJ3qYxMXxF7QzOQjGjh2rd955p+7bt09VVVeuXKmHHXZYSU7gIDPuxp1sp+9RR9Xq6NHn6vbt21VVde/evXrHHXfo2LFjiz72ggULtHfv3ppKpfTwww/XVCqlvXv3DnxuST6CjhyMKjIxCrAorOgViKo/KbHLnQCXtJDTfJQa9ZStjMq5aSaJoMPI46aYg44crKZQYlMgMVEgfuBHnH2SQk4L4VcdkGog6BteWIrZa+hs0BZCXEYUwiCfAjEfSALxY9ZwHMesSyFJUU9eCDInVU9O31mzbinrvEGmwk9TjGM8rj7HSsIUSAIIKs6+Em6+SYt66okgk0L25PT9y192xyYZZT6KcYyHETnoR5GtJGMKJAEEFWdfCTffpEU99YSfOamy6SmMvK4umPP6STGhs2YhhECuca1KbUn2gQThs6gGZ3KhgIMgs/F6JcyIuEI+kLPPRlOpeEfiqVoW3qjAfCDJJgifRRhj1lFTaEgoDjVEvFiXhx9+OF1dXWX7RfIN6UyeDC++CPv3Bz97vFwsC2+8MAUSAn6lU6gEn0XYFBoSCnK4yCteZnFfe+21PPPMM2UruuwhnfPPF665Bn7zG+jqOvS8Ucwe74kklvGtaHKZJZXaohjC8jP2Pm5x9nHE65BQoW1RDNsUCkf2kt221OG4pIVBV1PobJzAhrCiwc90CpXmMA4CL0NChbZFNWyTaV2mhxS7urr4h3/4B09Rd6UOxyXNqjXHeMzIpVUqtUVhgVRLuoM4OKTTFAo4qKur07q6uthNoMy0Lm+77TYVkaIyBZRag8OsWsMLmAUSDdWScC0ODuk0hQIOli5dytKlS2M3gTLTurzuuutYsWIFhx12WM6+DQ0NnHzyybS2tpY9N8is2vwEmQq+UsirQETkVBH5lYhsFJEfiUhjxrZf59uvGERktIi8KSLrReTGHNsnisgWEXnJbVdmbLtMRNa57TI/5AmCIKJG4vjDjoNDOpNCQzNxHLbJjogbNWoUP//5z/MqujvuuMOXuUFxjMQLcja+V6xYlDcKWSD/BdwCnAFsAn4lIuln6fq8e3lERGqAe4DPA6cAE0TklBxdF6vqYLfNc/c9BpgGnAWcCUwTkT7lyhQEfkeNxOWH7efs+CBuGIUmSSZlAmUhRRfXGhx+EAdr1opFeaOQAjlSVZer6lZVvQX4F2CFiJyBMw5bLmcC61V1g6ruBhYBYz3uez6wUlXfV9UPgJXAaB9k8h2/0ymU+8P262bt5+z4IG4YhYZmkjJs05Oiq5R8ZtnEwZq1YlEeyeUYcXwmvAwclbVuMLAO2JpvP68NuAiYl/H+K8APs/pMBN5xZVkK9HfX/yswJaPfvwP/muc8VwHtQPsJJ5zgq2PJK36kcE9TrlPej0y+afyaHV+qA7jS8ZIpIG41OEohjvVpbMZ7dyg2nbt7Qz87x/oBwP359vPagItzKJAfZPXpC9S5y98A2tzl63IokO/0dM4kpzJJU+4P2++bdSnzCOJ4w0gqlRBFFYf6NNlRhNUSPemVfAok7xCWqi5Q1UOc5ar6e1X9Wr79imAz0D/jfTPwdta5tqnqLvftXGCo130rlWKd8kFl8k1TikM6qOSQ1UhShuMKEQd/TvYwqs1490gurRJGA3oBG4CBQC2wFjg1q89xGcsXAs+5y8cAG4E+btsIHNPTOSvBAim2KFDQT3elPgFXSkErwz+inBWfbZnbjPfuEMeKhMAXgN8BbwGT3XUzgAvc5f8EXnOVy2rg5Ix9LwfWu+1rXs5XCQqklB92kDfrcjL6Ji2NhhEsYfpzvA6jnnjiQF98l0mnZAUCfMbLuiS0SlAgqqU55eN4s64EB7DhH2H6c+Lgd0kS+RSIl5noP8qx7h4P+xkBUUoVtDhOnkvKfIxs4jDRrRIJ058TB79LRZBLqzgKhzOBa4A/At/OaFOAl/PtF+dWKRZIKcQxWiepBa38DIU2oiWOlnkcoQQL5AigH46zuymj7cYJwTUSRByjdeKYRsMLcZjoZvhDHC3zRJFLq2Q24BMZywIc0dM+cW3VbIEYpWPzViqXOFrmcYQyfCDfFZGjRKQBJyJqo4hM8l+VGYY3wvZB2LyVyiWOlnmiyKVVMhvwovv6ZWA2zpwN84EYkRGFD8LmrRhBkI6obG7up6mUaHNzv1iGClOGBVIrIr1wEh0+pE7iw/1+KzLD8EoUPohKTVxoREdcMmuXgxcFMg8nnXsf4EkROQGI/yerIKo9bDTodCxeMYer4SeVkDK+RwWiqnep6vGqep5rymwGRgQvmpEmDvURoiQuPoikzlsx4kklpIzvUYGISJOI/G8RWe6uOhnHH2KERJLCRoOwluIy6cscroafVEK5ay9DWPOBJzmY/XYd8J2gBIoTUZWOjcuQTSkEZS3FwQeR1HkrRjwJotx12HhRIMeq6k9xHeequgfYF6hUMSBKB1dchmxKIUhryXwQRiVRCSnjvSiQD90a5M5MQqekbWegUsWAUhxcflkscRmy8UKY1pL5IIxKwu9y15GQK7Y3swHDgGeADpyhrPXA4J72i2MrZh5IsRXJMtOsz52LrlqFzp1bXv2AJOTpCTOraVJzZxlGPvwsdx0kFDsPREQ+4yqYdqAVaMFJrniKqr4UhDKLE8U6uIIIyUvCkE2Y1pL5IKqXuIaylzvqUEpm7ThRaAjrQBp3Vd2tqmtV9SV1JhJWPMU6uIIIyUvKkE0cHNxGZRPHUPZKmAhYLl58IFVJsQ6uIELykhQ2mgRryUgucQxlr4SJgOUimn2HTG8Q6QCeyrejql5Q9slFRgN3AzXAPFW9JWv7JOBKYC+wBbhcVf/gbtsHvOJ23eRFnmHDhml7e7sn2dJPF42NbzF+fBcDBzqWx6JFjoNrzZrnupmZ/fs3MW3aVgYNOvRY69bBzJlNbNr0nqdzJ5HW1laeeuopTjvtNG699VZuuOEG1q5dS0tLC21tbVGLZySMUaNG8cQTTxx4X1tby+7duw+8phk5ciSrVq2KQsSq+s+LyAuqOix7fSELZAtwZ4FWrkA1OJUNPw+cAkwQkVOyur0IDFPVTwNLgdsytv1VVQe7rWxllk3v3r1Zs+Y5WlquZ+bMJkaPTjFzZhMtLdcfojygMkLyyiFJ1pIRf5IQyl4JEwHLpZAF8ltVPT2wE4ucDXxXVc93398EoKr/maf/EOCHqvr37vsdqlqUp6kYC6RYirVYksL27duZOHEi8+fPp7GxMWpxjCpi9erVjBkzhp07dx6yLQ6h7GaBFLZAfh+cOAB8DKdcbprN7rp8XAE8lvG+XkTaReQ5ERkXhIDFUKzFEjalRrHE0XlpVAdxD86o9lEHKKBAVPWLAZ9bcqzLaQ6JyKU481EyvVInuBrxy8BsETkxz75XuYqmfcuWLeXKXJA4h+SVqgji6Lw0qoc4B2dUxETAMokyCmszB/NrATQDb2d3EpFRwGTgAlXdlV6vqm+7rxuANcCQXCdR1TmqOkxVhzU1NfknfcLwqgiSnIfLqDziHMoe91GHUMg1uzCMBvQCNgADcaocrgVOzeozBHgLOClrfR+gzl3uh5Pg8ZSezllNFQlLreMd5sxyw+gJyz4QD8gzE93rzf6LwCyc6KsLvezj8bhfAH7nKonJ7roZONYGwCrgXeAlty1z138WJ4R3rft6hZfzVZMCKUcRxL18a0dHh44bN047OjoilcMwqoWSFQjOjPQVwNfc9kvgnp72i2OrJgWiWp4iiHMerihqohvxxh4qgiWfAvHiA2kBzlfV+1X1ftdqGO5hPyNiyoliibPz0hz7RjYWLRgNXhTIm8AJGe/7Ay8HI47hN6Uqgjg5L82xb/SEPVREgxcF0hd4Q0TWiMga4HWgSUSWiciyQKUzyqZURRCnmeVJmJVshIs9VMSDvDPRD3QQaSm0XVWf9FWiAAlyJnpcGTduHOeccw7XXnstqVSKffv2MXv2bJ5++ulEpUGP+6xkI1wK/R7S2O/CP/LNRO9RgVQS1ahAKonly5dz8cUX09XVdWBdfX09S5YsYcyYMRFKZkSBPVSER9GpTETkV+5rp4j8JaN1ishfghTWMHIRZ8e+ET5xT3VSDRRKZfI/3NcjVfWojHakqlqKVSN04uTYN+KBPVTkp9xqiV7wlMpERGpE5HgROSHdfJPAMDwSJ8e+EQ/soSI3YVVL7FGBiMg/48wGXwk86rblvpzdMIrAaqIb2VTqQ0W51kNY1RK9RGGtB85S1cRXRzEnumEYcSezttCECQdrCy1c6L22kN+1SkqpB5Lmj0BxRSQMwzCMkvDDegirWmKhKKxJbk3yDcAaEbkpvc5dbxiGYfjM3Lk/YsKELiSrYpIIjB/fxbx59/Z4jOOP78vGjbm3bdzobPeDQhbIkW7bhOP/qM1Yd6QvZzcSS6kVDg3DKIwf1kNY1RILhfFOL9R8OXsVEkZoXRiUm7zOFJBh5MYP6yGsaoleorBWisjRGe/7iMjjvpy9yggrtC4Myk1eZ9lTDSM3flgPYVVL9OJEb1LVA7NyVPUD4Fhfzl5l+B1aF+ZTvN/J6yx7anVglmbx+GU99O7dm6lTp7Np03vs3buPTZveY+rU6b6W2vWiQPZlThwUkY/jFBgyisQP51gmYT7Fl5sR17KnVidmaXojc2i7sfEo/vznPwNnMn16v1jXWveiQCYDvxKRBSKyAHgKuClYsSoTv0PrwnyKb21tZfny5d2USCY9Ja+zlOzViVmaPZNraPu7390G/Dcf/ehH6OjYHoj14Ac9KhBV/SVwOrDYbUNV1RcfiIiMFpE3RWS9iNyYY3udiCx2t/9GRAZkbLvJXf+miJzvhzxBU65zLOqn+HKS15WrgMCGQ5JA1L/RJBLWrPEg8JQLC/gsThnb4cBn/DixiNQA9wCfB04BJojIKVndrgA+UNVBwF3Are6+pwDjgVOB0cCP3OPFmnKdY3F4ii8neV252VNtOCT+xOE3mjT8HtoOEy9RWLcA1+BUInwduEZE/tOHc58JrFfVDaq6G1gEjM3qMxZ4wF1eCowUEXHXL1LVXaq6EVjvHi/WlOsc8+MpvlzKTV5XjgKy4ZD4E8VvNOmh8WHNGg8EVS3YcOqfpzLe1wAv97Sfh+NeBMzLeP8V4IdZfV4FmjPevwX0A34IXJqx/sfARXnOcxXQDrSfcMIJGjWdnZ06ffpU7d+/SWtqUtq/f5NOnz5VOzs7PR/jkUce0fr6esUJZlBA6+vr9ZFHHglQcoexY8fqnXfeqfv27VNV1b179+odd9yhY8eO9bT/8OHDNZVK6ZAhQ3TFihU6ZMgQTaVS2traekjfkSNHdvuMtbW13V7TbeTIkb5+RqN8wvqNdnZ26tChp+qIEfU6dy66ahU6dy46YkS9Dh16alH/q6hobu6nc+eiq1cf2ubMQfv3b4paRAXaNcf91esQ1tEZy41F6KdCSI512dFd+fp42ddZqTpHVYep6rCmpqYiRfQfP0LroqyBUG5G3GKyp9pwSHIJ6zeaZP9BmrBmjQdCLq2i3Z/gJwB/AObjDCdtBMb3tJ+H454NPJ7x/ibgpqw+jwNnu8u9gK04yqNb38x+hdrQoUP9V80RUMxTfFxJW2LNzf00lRJtbu6X0xJra2vThoaGbk+y6dbQ0KCrV6+O5gMYBQnrN5qEp/eeyLSi5sxBV650ZI+TFUUpFojrb/gVjp22LbEAABs0SURBVOP85247W1UX+aC7ngdOEpGBIlKL4xRfltVnGXCZu3wR0OZ+mGXAeDdKayBwEvDfPsgUWzLHeZ98cg19+hzO2LFjOPvssxNXA6GYGflWtjSZhFWnI9H+A5ewZo0HgZd6IC+o6tBATi7yBWA2jl/lPlX9nojMwNF2y0SkHlgADAHex7F8Nrj7TgYuB/YC16rqYz2dL6n1QPyoDxAnZsyYxpNP3saUKd0jT1SdgIKWluuZOvVgurWf/OQnXH311ezcuZO6ujp27dpFQ0MD9957L5deemkEn8CIC37XvTByU049kOdE5IwAZEJV/5+q/o2qnqiq33PXTVXVZe5yl6perKqDVPXMtPJwt33P3e9vvSiPqPAjQqQSxnkzKTZs0cqWGvmIq/8g6ZFhXvFigbwO/A2OH+RDHB+EquqngxfPX8K2QPyyHCrtKaumJsWKFUpNjpk7e/fC6NEp9u7dd2DduHHjOOecc7j22mtJpVLs27eP2bNn8/TTT1s52yon8z82fvzB/9iiRdFZ55U2YgD5LRAvCuTjudar6h98ki00wlYgxQ7V5KPYG27cqTSFaETLjh07mDXrdubNu5e3397G8cf35corr2bSpOsiuVH79b+PE+UMYR0HvK+qf3CVxvvAR/0WsBLxa4ZpWNXFwiKuww5GMgkj62wxJHlmebF4USD3ApkDdx+664we8CtCpNJuuGEVuzGMKKiEyDCveFEgohnjXKq6H2dOhtEDflkOQd9ww3b4JTls0Yg/USfdrLQRg0J4USAbROTbInKY264BNvS4l+Gb5RDkDTeqKolxG3YwKoeok25W2ohBIbw40Y8Fvg+MwJkB/ATOvIvEeTmjjMKKS4RINpXo8IsT27dvZ+LEicyfP5/GRr+yABmFaG1tZc2aNbS2ttLW1hb6+ZPwvy+WfE70stKRJK1FkcrEj+SJQVIJqSDizIMPPqiALliwIGpRKpY4Jt2M+/++WMiTyiSvBSIi16vqbSLyA3IkKlTVb/ul3cIiqTPRg6TSQoTLxW+LIeqn4TgRlDW2evVqxowZw86dO/P28ZpG3izG3JQSxvuG+9oOvJCjGRVAFA6/qJ2chSh3/Nwq8uUnKN+EnzVIovafJI28CkRVH3FfH8jVwhPRCJIoHH5x/pOWW7TKUtDnJ8iCYH4l3bSiZcWRV4GIyLJCLUwhjeCIYk5GMX/SoK0Vvy2GOFSNjAthW2Ol1CAxi7FMcjlGXL/IFuC3wHXAOUBLZsu3X5xbnOuBeK2PEeS5g3L4lePkDNoJXajeSLqVUnckyqqRcSGoa5uPUmqQhC1jlP/zciCPE72QAqkBRuMUkXoRuBk4NV//JLS4KpBKKMtZiHL+pMOHD1cg0GJZQRStWrBggfbu3VtTqZQefvjhmkqltHfv3rGIxuro6NBx48ZpR0dH4OcKsyBYqeWWw5Ixyf/zohVIt05QB0x0rZJ/9rJPHFtcFcj06VN1xIh6bWvrHkbb1ub8uKZPnxq1iGXj9U8aVUim3xZDnKtGhh1anARrLAwZk/w/z6dAeqpIWCciXwR+AnwLZ0LhzwvtYxRPNSRf8+rkjMoJ7XcN77Aq8pVC2I7isOqjQ+k+szBkrMT/eSEn+gPAs8DpwHRVPUNVZ6rqn0KTrkqoluRrXv6kUTmh/S5a9dBDDzFp0iRSKecvVlNTw3e+851I6pdE7SgOsyBYqRF+YchYif/zQhbIV3AKSV0DPCsif3Fbp4j8pZyTisgxIrJSRNa5r31y9BksIr8WkddE5GUR+VLGtvkislFEXnLb4HLk8ZtikxMmLflaqckXvf5Jo6iDHmeLoVyiDi0O89qWal2FIWPS/ueeyDWuFXQDbgNudJdvBG7N0edvgJPc5eOBd4Cj3ffzgYuKPW8YPpBSHGVJGRvt6OjQMWPG6JAhnyzJEViMk9MvJ3SYDuM4E6YzO0zimMYkH0n5n+eCcpzofjfgTeA4d/k44E0P+6zNUCixVSCl/Egylc6cOejKlU4eqrhFZzz44IOaSqEtLYcF/ifwywltuagOkgRndrGEHYZbDkn5n+cinwLxks49CD6iqu8AuK/HFuosImcCtcBbGau/5w5t3SUidQX2vUpE2kWkfcuWLX7IXpBSHGVJqY9x3333UVsLl166J3BHoF9DCjaz+CBhOrPDIkkTN5PyPy+GHtO5l3xgkVXkLn07GXhAVY/O6PuBqh7iB3G3HQesAS5T1ecy1v0ZR6nMAd5S1Rk9yRRGMsVKSk44atQonnjiiQPva2tr2bNnNytXEtvPl0vm3bt3H3hNM3LkSFatWhWFiJHR2trKU089xWmnncatt97KDTfcwNq1a2lpaUl8osfly5dz8cUX09XVdWBdfX09S5YsYcyYMRFKVhmUUxO9JFR1lKp+Kkd7GHjXVQJpZZCztoiIHAU8CkxJKw/32O+4ltUu4H7gzKA+R7EkwVHm1Qmey/laV0esP1/UDuM4U8mBApVoXSWBqIawlgGXucuXAQ9ndxCRWuAXwIOquiRrW1r5CDAOeDVQaYsg7tXIiqlAmGt4YPduuO8+Yvv5kjSkETZxCi32mzBDhY2DRKVAbgHOFZF1wLnue0RkmIjMc/tcgpODa2KOcN3/IyKvAK8A/XDSrMSCKJITFsOsWbfT2PgWU6Z0MWiQMxQ1aBBMmdJFY+NbzJp1e7f+2SG1+/fDiy/C5MnE8vNBNGHARrRUsnUVa3J51iu1hZXKJM7VyEqpQJgdUisiWl9/mB577JGx+3xp4pyLyjCSBjGLwqpoevfuzdSp09m06T327t3Hpk3vMXXq9FhEWZQyGzZ7eGDw4MHs3r2PU08dFrvPl8aGNAwjeEyBVBmlOPmTODyQRJkNI2kEFsYbR6wmOsyYMY0nn7yNKVO6z1VRdfwYLS3XM3Xq9OgENAwjdoQexmvEk7g7+cul1DxdhuEH1fb7MwukCtmxYwezZt3OvHn38vbb2zj++L5ceeXVTJp0Xaz8GMWSDlFubHyLCRO6GDjQGZZbuNBRjkmd7Wskg0r+/ZkFYhwgzk7+cugpRPk//uPmQOqrB1233UgGxYbIVwKmQIyKoec8ZD8qqVZET5Rag8JvTJFFSyUWjOoJUyCGJ5IwtttTiPLWrZ2A/4kV45KwMS6KrFqpxIJRPWEKJGFEcSMvJv1JlPQUolzn5mwutxJf1BX+8hEXRZYmCQ8dfpKEPHh+YwokQUR1I0/K2G6hPGT33+/k8YLyEyvGJWFjXBUZJOehw0/ingcvCEyBJIiobuRJGdvNF6I8ebKTv2v//u79S02sGJeEjXFRZLlIykOHn1R6iHwuLIw3QfTv38S0aVsZNOjQbevWwcyZTWzalDMzflkkqcZJ9xDlrRx99OHs3r2Xzs7d1Nc7Vsj+/f7UiohDDYrVq1czZswYdu7ceci2KDMPR/VbjZpKDZG3MN4KIConXZLGdtMhyq+/voHBg0/htNP2M2vWblatgh/8AM46C+rrnVTm5daKiEMNirhmHq5GhzJUboh8PkyBJIiobuRJHNvNN4Tyve/B6acLXV0flu1sjkvCxjgosmyS9NBhlI4pkAQR1Y08iWO7hfw2EycqffocUXZixbgkbIyLIsskiQ8dRvGYDyRBZKZKGD/+YKqERYuCT5WQtLHdJPltymXcuHGcc845XHvttaRSKfbt28fs2bN5+umnI6s2GOVv1fCffD4QUyAJI2k38qioVidunLDfauUQKwUiIscAi4EBwO+BS1T1gxz99uGUrQXYpKoXuOsHAouAY4DfAl9R1d3Z+2dTCQrE8EYS09Zv376diRMnMn/+fBobG6MWxzAOELcorBuBJ1T1JOAJ930u/qqqg912Qcb6W4G73P0/AK4IVlwjaSTRb1PtqUgsl1fyiEqBjAUecJcfAMZ53VFEBBgBLC1lf6M66N27N2vWPEdLy/XMnNnE6NEpZs5soqXl+tiOv8ctFYlfeFUMQSrQIJVTtaVs6UauQulBN6Aj6/0HefrtBdqB54Bx7rp+wPqMPv2BV72cd+jQoaXWlDcM3xk5cqQCB1ptbW2313QbOXJk1KKWxYMPPqiALliwoGC/4cOHK6Ctra2RyVAsnZ2dOnToqTpiRL3OnYuuWoXOnYuOGFGvQ4eeqp2dnb6eLyqAds1xTw3MAhGRVSLyao42tojDnKDOuNuXgdkiciIgOfrldeSIyFUi0i4i7Vu2bCnyUxhGcMQ5FYmf5LOswszlFZR1V40pW7qRS6sE3YA3gePc5eOANz3sMx+4CEeBbAV6uevPBh73cl6zQIy40dbWpg0NDd0sjnRraGjQ1atXRy1iNzo6OnTcuHHa0dGRt49Xy+r000/P+9nLvQZhWXfNzf107lx09epD25w5aP/+TWUdPy4QtgXSA8uAy9zly4CHszuISB8RqXOX+wF/D7zufpjVOMok7/6GkQTimookH178FF4tqzvvvDOwpJRBWHe5fB3vvLOVj340d/9KTtmSJioFcgtwroisA8513yMiw0Rkntvnk0C7iKzFURi3qOrr7rYbgEkish7oC/w4VOkNw0fimIokH16GgorJVhyUAvU7Y3K+9PRnnQXXXAN//euh+1RDypZIFIiqblPVkap6kvv6vru+XVWvdJefVdW/U9XT3NcfZ+y/QVXPVNVBqnqxqu6K4nMYlUcUETWZqUgWLVrERz96LHv37uCrX/1K6BE92Z//qKNqqakp3k9RjGIISoH6qZzy+TpuvhmOOw6WLOnev1pStlguLMNwiaoIUjqn1po1a5gx4984+eQOfvADWLmSvOcPIiw11+efNWvPgQzGUNxQUE+KIa2svvnNy/nwwx00NAgXX/xFPvWpT/mWy8sv5VQot9pll8HPfkZi5hv5iSkQw3CJKqLmoYceYtKkScyefafn8wcxZ6JQBuMhQyCV425RaCioUJLHTGU1a9YeVq6Eu+7ax+bNP+Oww/Yxc+ZMX5JS+pVosqf09Dt2SGLmG/mJKRDDcIm68mIx5w8iLLXQ+b/2tYM15dP0NBRUKFtxT8p6795dviSC9Ctjck/p6T/2sX5VUwMkE0umaBguUWfw7en8553HgfTotbW17N69+8BrmpEjR7Jq1arAzi+Soq6ujl27dtHQ0MC9997LpZdeWvS5kpbsMom51fwkbrmwDCN2RF0EqafzZ/qCg5h02NP56+rwreZI0ioWJjG3WhiYAjFiSRTRUFEXQerp/OPHfyWQORNezv/AAymGDz/Xt+JZUSvrYklibrUwsCEsI3ZkFiOaMOFgMaKFC4MtRhR1ESQv51+zZg0XX3wxXV1dB/arr69nyZIljBkzJvDz+/X5q31IKGnYEJaRGKKKhor6KdPL+YOcdBjm57chocrALBAjdiTNwRomra2tPPXUU5x22mnceuut3HDDDaxdu5aWlhba2tqiFq8orGJhcohVRcKoMAWSDKKOhoozcax/blQ+pkAwBZIUzAIxjHhhPhAjMUQdDWUYhjdMgRixwxyshpEMTIEYsSPqaCjDMLxhPhDDMAyjIOYDMQzDMHzFFIhhGIkgivQ2RmFMgRhGCNjNrzyiKvZlFCYSBSIix4jIShFZ5772ydGnVUReymhdIjLO3TZfRDZmbBsc/qcwDG/Yza98okpvYxQmKgvkRuAJVT0JeMJ93w1VXa2qg1V1MDAC2AmsyOhyXXq7qr4UitSGUQJxuPkl3QKKutiXkZuoFMhY4AF3+QFgXA/9LwIeU9WdgUplGAEQ9c2vEiygpNUPqRaiUiAfUdV3ANzXY3voPx5YmLXueyLysojcJSJ1uXYCEJGrRKRdRNq3bNlSntSGUQJR3/ziYAGVS9Lqh1QLgSkQEVklIq/maGOLPM5xwN8Bj2esvgk4GTgDOAa4Id/+qjpHVYep6rCmpqYSPolhlEfUN7+oLSA/sPQ28SQwBaKqo1T1Uznaw8C7rmJIK4hCmfEuAX6hqnsyjv2OOuwC7gfODOpzGEa5RH3zi9oC8gNLbxNPohrCWgZc5i5fBjxcoO8EsoavMpSP4PhPXg1ARsPwhahvflFbQH5g6W3iSSSpTESkL/B/gROATcDFqvq+iAwDvqGqV7r9BgDPAP1VdX/G/m1AEyDAS+4+PXoCLZWJERVRFk+y8rFGuVg9EEyBGNVJ1LXejeRjubAMo0qx4R8jKMwCMQzDMApiFohhGIbhK6ZADMMwjJIwBWIYhmGUhCkQwzAMoySqyokuIluAP4R4yn7A1hDPVywmX3mYfOVh8pVHmPJ9XFUPyQVVVQokbESkPVfkQlww+crD5CsPk6884iCfDWEZhmEYJWEKxDAMwygJUyDBMidqAXrA5CsPk688TL7yiFw+84EYhmEYJWEWiGEYhlESpkAMwzCMkjAFUiYicoyIrBSRde5rnxx9WkXkpYzWJSLj3G3zRWRjxrbBYcvn9tuXIcOyjPUDReQ37v6LRaQ2bPlEZLCI/FpEXhORl0XkSxnbArl+IjJaRN4UkfUicmOO7XXu9VjvXp8BGdtucte/KSLn+yFPkbJNEpHX3Wv1hIh8PGNbzu85AhknisiWDFmuzNh2mft7WCcil2XvG5J8d2XI9jsR6cjYFug1FJH7ROQ9EclZKE8cvu/K/rKInJ6xLfBr1w1VtVZGA24DbnSXbwRu7aH/McD7QIP7fj5wUdTyATvyrP+/wHh3+b+Aq8OWD/gb4CR3+XjgHeDooK4fUAO8BXwCqAXWAqdk9fkm8F/u8nhgsbt8itu/DhjoHqcmZNlaM35fV6dlK/Q9R3D9JgI/zLHvMcAG97WPu9wnbPmy+v8zcF9Y1xA4BzgdeDXP9i8Aj+EU1PsM8Juwrl12MwukfMYCD7jLD+CU2C3ERcBjqrozUKkOUqx8BxARAUYAS0vZ3yM9yqeqv1PVde7y28B7OBUpg+JMYL2qblDV3cAiV85MMuVeCox0r9dYYJGq7lLVjcB693ihyaaqqzN+X88BzT6e3xcZC3A+sFJV31fVD4CVwOiI5TukrHaQqOpTOA+Z+RgLPKgOzwFHi1PmO4xr1w1TIOXzEVV9B8B9PbaH/uM59Mf4PdcUvUtE6iKSr15E2kXkufTwGtAX6FDVve77zcDHIpIPABE5E+ep8a2M1X5fv48Bf8x4n+tzH+jjXp/tONfLy75By5bJFThPq2lyfc9+41XG/+V+b0tFpH+R+4YhH+7w30CgLWN1GNewEPnkD+PadaNXkAevFERkFfDRHJsmF3mc44C/Ax7PWH0T8Gecm+Ic4AZgRgTynaCqb4vIJ4A2EXkF+EuOfkXHfft8/RYAl6nqfnd12dcv16lyrMv+3Pn6eNm3HDwfX0QuBYYBLRmrD/meVfWtXPsHLOMjwEJV3SUi38Cx5kZ43DcM+dKMB5aq6r6MdWFcw0JE9ds7BFMgHlDVUfm2ici7InKcqr7j3uDeK3CoS4BfqOqejGO/4y7uEpH7gX+NQj53aAhV3SAia4AhwM9wzONe7lN2M/B2FPKJyFHAo8AU12xPH7vs65eDzUD/jPe5Pne6z2YR6QU04gw7eNk3aNkQkVE4CrpFVXel1+f5nv2++fUoo6puy3g7F7g1Y9/hWfuuCVu+DMYD38pcEdI1LEQ++cO4dt2wIazyWQakox0uAx4u0PeQsVT3ppn2N4wDckZeBCmfiPRJD/2ISD/g74HX1fHMrcbx2+TdPwT5aoFf4Iz7LsnaFsT1ex44SZwItFqcm0h2tE2m3BcBbe71WgaMFydKayBwEvDfPsjkWTYRGQL8b+ACVX0vY33O79lH2YqR8biMtxcAb7jLjwPnubL2Ac6ju8UeinyujH+L44z+dca6sK5hIZYBX3WjsT4DbHcfpMK4dt0J0kNfDQ1n3PsJYJ37eoy7fhgwL6PfAOBPQCpr/zbgFZwb30+A3mHLB3zWlWGt+3pFxv6fwLkBrgeWAHURyHcpsAd4KaMNDvL64US6/A7nyXKyu24Gzk0ZoN69Huvd6/OJjH0nu/u9CXw+gN9cT7KtAt7NuFbLevqeI5DxP4HXXFlWAydn7Hu5e13XA1+LQj73/XeBW7L2C/wa4jxkvuP+5jfj+LG+AXzD3S7APa7srwDDwrx2mc1SmRiGYRglYUNYhmEYRkmYAjEMwzBKwhSIYRiGURKmQAzDMIySMAViGIZhlIQpEKMqEBEVkQUZ73uJkw12ufv+AsmRldXH839XRHJOchSRZ4s4zi/cLLDrRWS7HMwK+9ki5RnhziHIte1UcbIf7xKRa4s5rlFd2Ex0o1r4EPiUiByuqn8FzsWZlwOAqi4jx2SyXLiTFkUPplMpC1X1fPNX1QtdGYYD/6qqY0o87QhgK06yxWy24mSgvSjHNsM4gFkgRjXxGPA/3eVuWQHEqU/xQ3f5I+6T/lq3fVZEBojIGyLyI+C3QH8RmSAir4jIqyJya8axRovIb919n8g4/ykiskZENojItzP673Bfh4vIU+65XxeR/xIRz/9RETlDRJ4UkRdE5DER+Yi7/l/c460VkZ+IyInAlcB1uawXVX1XVduBvTlOYxgHMAvEqCYWAVPdYatPA/cBn8vR7/vAk6p6oYjUAL1xUlr8Lc7s3m+KyPE4+ZuGAh8AK8TJzPoMTm6nc1R1o4gck3Hck3FqdRwJvCki92pGXjSXM3FqivwB+CXwRQ6m08+Lm17jbpyZ1FtF5B+BmcBVwPXAx1V1t4gcraodIjIP2Kqqs3s6tmHkwxSIUTWo6sviVA6cAPy/Al1HAF9199kHbHdzC/1BDyZyPANYo6pbAETk/+AUAtoHPKVOLRBUNbOuw6PqJDbcJSLvAR/BSVWRyX+r6gb3mAuB/4EHBQJ8EjgVWOWMsFGTcezXgJ+IyMPAQx6OZRieMAViVBvLgDtwspb2LXLfDzOWc6XOTq/Plx9oV8byPnL//7L39ZprSICXVTWXRXU+Tkr3scAUEfmUx2MaRkHMB2JUG/cBM1T1lQJ9nsApBYuI1IiTSj6b3wAtItLPHeaaADyJk7m1xc3ES9YQlhfOdLPEpoAvAb/yuN/rwMfEKbiFiNS60VQ1QLOqtgHX4VRybAA6cYbSDKNkTIEYVYWqblbVu3vodg3QKk5RrRdwhoayj/MOTjGr1TiZWX+rqg+7Q1pXAT8XkbXA4iJF/DVwC0524Y04aex7xB0auwiY5Z73ReAsHCvnpyLyMo7z/1ZV7cRJm3+JiLyY7UQXkWYR2Qx8G/iuiGwWkYYiP4dRBVg2XsOICT6E5hpGqJgFYhiGYZSEWSCGYRhGSZgFYhiGYZSEKRDDMAyjJEyBGIZhGCVhCsQwDMMoCVMghmEYRkn8f2A5NCLw2QTLAAAAAElFTkSuQmCC\n",
+ "text/plain": [
+ "
"
+ ]
+ },
+ "metadata": {
+ "needs_background": "light"
+ },
+ "output_type": "display_data"
+ }
+ ],
+ "source": [
+ "plotData(X, y)\n",
+ "# Labels and Legend\n",
+ "plt.xlabel('Microchip Test 1')\n",
+ "plt.ylabel('Microchip Test 2')\n",
+ "\n",
+ "# Specified in plot order\n",
+ "plt.legend(['y = 1', 'y = 0'], loc='upper right')\n",
+ "pass"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "The above figure shows that our dataset cannot be separated into positive and negative examples by a straight-line through the plot. Therefore, a straight-forward application of logistic regression will not perform well on this dataset since logistic regression will only be able to find a linear decision boundary.\n",
+ "\n",
+ "### 2.2 Feature mapping\n",
+ "\n",
+ "One way to fit the data better is to create more features from each data point. In the function `mapFeature` defined in the file `utils.py`, we will map the features into all polynomial terms of $x_1$ and $x_2$ up to the sixth power.\n",
+ "\n",
+ "$$ \\text{mapFeature}(x) = \\begin{bmatrix} 1 & x_1 & x_2 & x_1^2 & x_1 x_2 & x_2^2 & x_1^3 & \\dots & x_1 x_2^5 & x_2^6 \\end{bmatrix}^T $$\n",
+ "\n",
+ "As a result of this mapping, our vector of two features (the scores on two QA tests) has been transformed into a 28-dimensional vector. A logistic regression classifier trained on this higher-dimension feature vector will have a more complex decision boundary and will appear nonlinear when drawn in our 2-dimensional plot.\n",
+ "While the feature mapping allows us to build a more expressive classifier, it also more susceptible to overfitting. In the next parts of the exercise, you will implement regularized logistic regression to fit the data and also see for yourself how regularization can help combat the overfitting problem.\n"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 19,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "# Note that mapFeature also adds a column of ones for us, so the intercept\n",
+ "# term is handled\n",
+ "X = utils.mapFeature(X[:, 0], X[:, 1])"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "\n",
+ "### 2.3 Cost function and gradient\n",
+ "\n",
+ "Now you will implement code to compute the cost function and gradient for regularized logistic regression. Complete the code for the function `costFunctionReg` below to return the cost and gradient.\n",
+ "\n",
+ "Recall that the regularized cost function in logistic regression is\n",
+ "\n",
+ "$$ J(\\theta) = \\frac{1}{m} \\sum_{i=1}^m \\left[ -y^{(i)}\\log \\left( h_\\theta \\left(x^{(i)} \\right) \\right) - \\left( 1 - y^{(i)} \\right) \\log \\left( 1 - h_\\theta \\left( x^{(i)} \\right) \\right) \\right] + \\frac{\\lambda}{2m} \\sum_{j=1}^n \\theta_j^2 $$\n",
+ "\n",
+ "Note that you should not regularize the parameters $\\theta_0$. The gradient of the cost function is a vector where the $j^{th}$ element is defined as follows:\n",
+ "\n",
+ "$$ \\frac{\\partial J(\\theta)}{\\partial \\theta_0} = \\frac{1}{m} \\sum_{i=1}^m \\left( h_\\theta \\left(x^{(i)}\\right) - y^{(i)} \\right) x_j^{(i)} \\qquad \\text{for } j =0 $$\n",
+ "\n",
+ "$$ \\frac{\\partial J(\\theta)}{\\partial \\theta_j} = \\left( \\frac{1}{m} \\sum_{i=1}^m \\left( h_\\theta \\left(x^{(i)}\\right) - y^{(i)} \\right) x_j^{(i)} \\right) + \\frac{\\lambda}{m}\\theta_j \\qquad \\text{for } j \\ge 1 $$\n",
+ ""
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 20,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "def costFunctionReg(theta, X, y, lambda_):\n",
+ " \"\"\"\n",
+ " Compute cost and gradient for logistic regression with regularization.\n",
+ " \n",
+ " Parameters\n",
+ " ----------\n",
+ " theta : array_like\n",
+ " Logistic regression parameters. A vector with shape (n, ). n is \n",
+ " the number of features including any intercept. If we have mapped\n",
+ " our initial features into polynomial features, then n is the total \n",
+ " number of polynomial features. \n",
+ " \n",
+ " X : array_like\n",
+ " The data set with shape (m x n). m is the number of examples, and\n",
+ " n is the number of features (after feature mapping).\n",
+ " \n",
+ " y : array_like\n",
+ " The data labels. A vector with shape (m, ).\n",
+ " \n",
+ " lambda_ : float\n",
+ " The regularization parameter. \n",
+ " \n",
+ " Returns\n",
+ " -------\n",
+ " J : float\n",
+ " The computed value for the regularized cost function. \n",
+ " \n",
+ " grad : array_like\n",
+ " A vector of shape (n, ) which is the gradient of the cost\n",
+ " function with respect to theta, at the current values of theta.\n",
+ " \n",
+ " Instructions\n",
+ " ------------\n",
+ " Compute the cost `J` of a particular choice of theta.\n",
+ " Compute the partial derivatives and set `grad` to the partial\n",
+ " derivatives of the cost w.r.t. each parameter in theta.\n",
+ " \"\"\"\n",
+ " # Initialize some useful values\n",
+ " m = y.size # number of training examples\n",
+ "\n",
+ " # You need to return the following variables correctly \n",
+ " J = 0\n",
+ " grad = np.zeros(theta.shape)\n",
+ "\n",
+ " # ===================== YOUR CODE HERE ======================\n",
+ " \n",
+ " J = (-1/m) * (y.T @ np.log(sigmoid(X @ theta)) + (1 - y.T) @ np.log(1 - sigmoid(X @ theta)))\n",
+ " reg = (lambda_/(2*m)) * (theta[1:].T @ theta[1:])\n",
+ " J = J + reg\n",
+ " \n",
+ " grad = (1/m) * X.T@(sigmoid(X@theta) - y)\n",
+ " grad[1:] = grad[1:] + ( (lambda_/m)*theta[1:] )\n",
+ " \n",
+ " \n",
+ " # =============================================================\n",
+ " return J, grad"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Once you are done with the `costFunctionReg`, we call it below using the initial value of $\\theta$ (initialized to all zeros), and also another test case where $\\theta$ is all ones."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 21,
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "Cost at initial theta (zeros): 0.693\n",
+ "Expected cost (approx) : 0.693\n",
+ "\n",
+ "Gradient at initial theta (zeros) - first five values only:\n",
+ "\t[0.0085, 0.0188, 0.0001, 0.0503, 0.0115]\n",
+ "Expected gradients (approx) - first five values only:\n",
+ "\t[0.0085, 0.0188, 0.0001, 0.0503, 0.0115]\n",
+ "\n",
+ "------------------------------\n",
+ "\n",
+ "Cost at test theta : 3.16\n",
+ "Expected cost (approx): 3.16\n",
+ "\n",
+ "Gradient at initial theta (zeros) - first five values only:\n",
+ "\t[0.3460, 0.1614, 0.1948, 0.2269, 0.0922]\n",
+ "Expected gradients (approx) - first five values only:\n",
+ "\t[0.3460, 0.1614, 0.1948, 0.2269, 0.0922]\n"
+ ]
+ }
+ ],
+ "source": [
+ "# Initialize fitting parameters\n",
+ "initial_theta = np.zeros(X.shape[1])\n",
+ "\n",
+ "# Set regularization parameter lambda to 1\n",
+ "# DO NOT use `lambda` as a variable name in python\n",
+ "# because it is a python keyword\n",
+ "lambda_ = 1\n",
+ "\n",
+ "# Compute and display initial cost and gradient for regularized logistic\n",
+ "# regression\n",
+ "cost, grad = costFunctionReg(initial_theta, X, y, lambda_)\n",
+ "\n",
+ "print('Cost at initial theta (zeros): {:.3f}'.format(cost))\n",
+ "print('Expected cost (approx) : 0.693\\n')\n",
+ "\n",
+ "print('Gradient at initial theta (zeros) - first five values only:')\n",
+ "print('\\t[{:.4f}, {:.4f}, {:.4f}, {:.4f}, {:.4f}]'.format(*grad[:5]))\n",
+ "print('Expected gradients (approx) - first five values only:')\n",
+ "print('\\t[0.0085, 0.0188, 0.0001, 0.0503, 0.0115]\\n')\n",
+ "\n",
+ "\n",
+ "# Compute and display cost and gradient\n",
+ "# with all-ones theta and lambda = 10\n",
+ "test_theta = np.ones(X.shape[1])\n",
+ "cost, grad = costFunctionReg(test_theta, X, y, 10)\n",
+ "\n",
+ "print('------------------------------\\n')\n",
+ "print('Cost at test theta : {:.2f}'.format(cost))\n",
+ "print('Expected cost (approx): 3.16\\n')\n",
+ "\n",
+ "print('Gradient at initial theta (zeros) - first five values only:')\n",
+ "print('\\t[{:.4f}, {:.4f}, {:.4f}, {:.4f}, {:.4f}]'.format(*grad[:5]))\n",
+ "print('Expected gradients (approx) - first five values only:')\n",
+ "print('\\t[0.3460, 0.1614, 0.1948, 0.2269, 0.0922]')"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "*You should now submit your solutions.*"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 22,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "#grader[5] = costFunctionReg\n",
+ "#grader[6] = costFunctionReg\n",
+ "#grader.grade()"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "#### 2.3.1 Learning parameters using `scipy.optimize.minimize`\n",
+ "\n",
+ "Similar to the previous parts, you will use `optimize.minimize` to learn the optimal parameters $\\theta$. If you have completed the cost and gradient for regularized logistic regression (`costFunctionReg`) correctly, you should be able to step through the next part of to learn the parameters $\\theta$ using `optimize.minimize`."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "### 2.4 Plotting the decision boundary\n",
+ "\n",
+ "To help you visualize the model learned by this classifier, we have provided the function `plotDecisionBoundary` which plots the (non-linear) decision boundary that separates the positive and negative examples. In `plotDecisionBoundary`, we plot the non-linear decision boundary by computing the classifier’s predictions on an evenly spaced grid and then and draw a contour plot where the predictions change from y = 0 to y = 1. "
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "### 2.5 Optional (ungraded) exercises\n",
+ "\n",
+ "In this part of the exercise, you will get to try out different regularization parameters for the dataset to understand how regularization prevents overfitting.\n",
+ "\n",
+ "Notice the changes in the decision boundary as you vary $\\lambda$. With a small\n",
+ "$\\lambda$, you should find that the classifier gets almost every training example correct, but draws a very complicated boundary, thus overfitting the data. See the following figures for the decision boundaries you should get for different values of $\\lambda$. \n",
+ "\n",
+ "
![](\"Figures/polynomial_regression.png\")
![](\"Figures/polynomial_learning_curve.png\")
![](\"Figures/polynomial_regression_reg_1.png\")
![](\"Figures/polynomial_learning_curve_reg_1.png\")
![](\"Figures/email.png\")
\n",
+ "\n",
+ "While many emails would contain similar types of entities (e.g., numbers, other URLs, or other email addresses), the specific entities (e.g., the specific URL or specific dollar amount) will be different in almost every\n",
+ "email. Therefore, one method often employed in processing emails is to “normalize” these values, so that all URLs are treated the same, all numbers are treated the same, etc. For example, we could replace each URL in the\n",
+ "email with the unique string “httpaddr” to indicate that a URL was present.\n",
+ "\n",
+ "This has the effect of letting the spam classifier make a classification decision based on whether any URL was present, rather than whether a specific URL was present. This typically improves the performance of a spam classifier, since spammers often randomize the URLs, and thus the odds of seeing any particular URL again in a new piece of spam is very small. \n",
+ "\n",
+ "In the function `processEmail` below, we have implemented the following email preprocessing and normalization steps:\n",
+ "\n",
+ "- **Lower-casing**: The entire email is converted into lower case, so that captialization is ignored (e.g., IndIcaTE is treated the same as Indicate).\n",
+ "\n",
+ "- **Stripping HTML**: All HTML tags are removed from the emails. Many emails often come with HTML formatting; we remove all the HTML tags, so that only the content remains.\n",
+ "\n",
+ "- **Normalizing URLs**: All URLs are replaced with the text “httpaddr”.\n",
+ "\n",
+ "- **Normalizing Email Addresses**: All email addresses are replaced with the text “emailaddr”.\n",
+ "\n",
+ "- **Normalizing Numbers**: All numbers are replaced with the text “number”.\n",
+ "\n",
+ "- **Normalizing Dollars**: All dollar signs ($) are replaced with the text “dollar”.\n",
+ "\n",
+ "- **Word Stemming**: Words are reduced to their stemmed form. For example, “discount”, “discounts”, “discounted” and “discounting” are all replaced with “discount”. Sometimes, the Stemmer actually strips off additional characters from the end, so “include”, “includes”, “included”, and “including” are all replaced with “includ”.\n",
+ "\n",
+ "- **Removal of non-words**: Non-words and punctuation have been removed. All white spaces (tabs, newlines, spaces) have all been trimmed to a single space character.\n",
+ "\n",
+ "The result of these preprocessing steps is shown in the figure below. \n",
+ "\n",
+ "![\"email](\"Figures/email_cleaned.png\")
\n",
+ "\n",
+ "While preprocessing has left word fragments and non-words, this form turns out to be much easier to work with for performing feature extraction."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "#### 2.1.1 Vocabulary List\n",
+ "\n",
+ "After preprocessing the emails, we have a list of words for each email. The next step is to choose which words we would like to use in our classifier and which we would want to leave out.\n",
+ "\n",
+ "For this exercise, we have chosen only the most frequently occuring words as our set of words considered (the vocabulary list). Since words that occur rarely in the training set are only in a few emails, they might cause the\n",
+ "model to overfit our training set. The complete vocabulary list is in the file `vocab.txt` (inside the `Data` directory for this exercise) and also shown in the figure below.\n",
+ "\n",
+ "![\"Vocab\"](\"Figures/vocab.png\")
\n",
+ "\n",
+ "Our vocabulary list was selected by choosing all words which occur at least a 100 times in the spam corpus,\n",
+ "resulting in a list of 1899 words. In practice, a vocabulary list with about 10,000 to 50,000 words is often used.\n",
+ "Given the vocabulary list, we can now map each word in the preprocessed emails into a list of word indices that contains the index of the word in the vocabulary dictionary. The figure below shows the mapping for the sample email. Specifically, in the sample email, the word “anyone” was first normalized to “anyon” and then mapped onto the index 86 in the vocabulary list.\n",
+ "\n",
+ "![\"word](\"Figures/word_indices.png\")
\n",
+ "\n",
+ "Your task now is to complete the code in the function `processEmail` to perform this mapping. In the code, you are given a string `word` which is a single word from the processed email. You should look up the word in the vocabulary list `vocabList`. If the word exists in the list, you should add the index of the word into the `word_indices` variable. If the word does not exist, and is therefore not in the vocabulary, you can skip the word.\n",
+ "\n",
+ "![](\"Figures/svm_c1.png\"/)
![](\"Figures/svm_c100.png\"/)
![](\"Figures/faces_original.png\")
![](\"Figures/faces_reconstructed.png\")
![](\"Figures/decision_boundary3.png\")
\n",
+ " ![](\"Figures/decision_boundary2.png\")
\n",
+ " ![](\"Figures/decision_boundary4.png\")
\n",
+ "