Merge pull request #7 from FirstStreet/develop

v1.2.0 with lat/lng and address functionality, and bug fixes

Author: Lyetenth

README.md

@@ -14,6 +14,7 @@ The First Street Foundation API Access (Python) is a wrapper used to bulk extrac
   * [Running the Project - Method 2: Through the Client](#method2)
       - [Client Initialization Details](#client-init)
       - [Arguments](#client-arguments)
+      - [If you are using Jupyter Notebook](#jupyter_setup)
 - **[Products](#products)**
   - [Location](#location)
   - [Probability](#probability)
@@ -31,17 +32,12 @@ The First Street Foundation API Access (Python) is a wrapper used to bulk extrac
 <a name="installation"></a>
 # [Installation](#toc)
 **NOTE**: This project requires [Python](https://www.python.org/downloads/) 3.6+ to run.
+
 1. Go to the Python page (https://www.python.org/downloads/) and download then install Python version 3. **Make sure that the checkbox is checked for Python to be added to the PATH**
 
-    ![Screenshot](https://raw.githubusercontent.com/FirstStreet/fsf_api_access_python/master/doc/images//1.1.1.png)
-    
-    ![Screenshot](https://raw.githubusercontent.com/FirstStreet/fsf_api_access_python/master/doc/images//1.1.2.png)
-    
-    ![Screenshot](https://raw.githubusercontent.com/FirstStreet/fsf_api_access_python/master/doc/images//1.1.3.png)
+    A more detailed and in-depth guide on how to install Python can be found [here](https://realpython.com/installing-python/)
 
-    ![Screenshot](https://raw.githubusercontent.com/FirstStreet/fsf_api_access_python/master/doc/images//1.1.4.png)
-    
-2. [Optional] Open a new `powershell console` / `bash terminal` or close and re-open a console and create and activate a new virtual environment in the project directory:
+2. [Optional] Open a new `powershell console` / `bash terminal`and create and activate a new virtual environment in the project directory:
     ```sh
     python -m venv /path/to/new/virtual/environment
 
@@ -98,7 +94,7 @@ The First Street Foundation API Access (Python) is a wrapper used to bulk extrac
     ```sh
     cd /path/to/project
    
-    python -m firststreet -p <product>.<product_subtype> -i <fsids> -f <file_name> -l <lookup_type>
+    python -m firststreet -p <product>.<product_subtype> -i <search_item> -f <file_name> -l <lookup_type>
     ```
 
     ![Screenshot](https://raw.githubusercontent.com/FirstStreet/fsf_api_access_python/master/doc/images//3.2.1.png)
@@ -118,9 +114,11 @@ The First Street Foundation API Access (Python) is a wrapper used to bulk extrac
 
     Example: ```-v v1```
 
-- `[-i/--fsids FSIDS]`: [OPTIONAL] The FSIDs to search for with the product
+- `[-i/--search_items SEARCH_ITEM]`: [OPTIONAL] The Search Items to search for with the product. 
+
+    **NOTE** THE LIST MUST BE A SEPRATED WITH A SEMICOLON (;) INSTEAD OF A COMMA
 
-    Example: ```-i 541114211,541229206```
+    Example: ```-i 541114211;541229206```
 
 - `[-l/--location LOOKUP_TYPE]`: [OPTIONAL] The lookup location type (property, neighborhood, city, zcta, tract, county, cd, state)
 
@@ -134,7 +132,7 @@ The First Street Foundation API Access (Python) is a wrapper used to bulk extrac
 
     Example: ```-l False```
 
-- `[-f/--file FILE]`: [OPTIONAL] A file of FSIDs (one per line) to search for with the product
+- `[-f/--file FILE]`: [OPTIONAL] A file of Search Items (one per line) to search for with the product
 
     Example: ```-f sample.txt```
 
@@ -164,6 +162,7 @@ The First Street Foundation API Access (Python) is a wrapper used to bulk extrac
     import firststreet
     fs = firststreet.FirstStreet("api-key")
     ```
+    
     <a name="client-init"></a>
     #### [Client Initialization Details](#toc)
 
@@ -179,7 +178,7 @@ The First Street Foundation API Access (Python) is a wrapper used to bulk extrac
 
 2. Call one of the methods described below in the `Products` section with the required arguments. See the `Examples` section for more examples.
     ```python
-    fs.<product>.<product_subtype>(<fsids: list>, <lookup_type: string>, <csv: boolean>)
+    fs.<product>.<product_subtype>(<search_items: list>, <lookup_type: string>, <csv: boolean>)
     ```
 
     ![Screenshot](https://raw.githubusercontent.com/FirstStreet/fsf_api_access_python/master/doc/images//2.2.1.png)
@@ -194,15 +193,29 @@ The First Street Foundation API Access (Python) is a wrapper used to bulk extrac
 
     ![Screenshot](https://raw.githubusercontent.com/FirstStreet/fsf_api_access_python/master/doc/images//2.3.1.png)
 
+
+<a name="jupyter_setup"></a>
+### [If you are using Jupyter Notebook](#toc)
+1. Add the following to the top of the file before the [client initialization](client-init) to allow the jupyter notebook and download loops to work correctly
+
+    ```python
+    # Setup For Notebook only
+    import nest_asyncio
+    nest_asyncio.apply()
+    ```
+
 <a name="products"></a>
 # [Products](#toc)
 
 More information on each product can be found at the [First Street Foundation API Data Dictionary](https://docs.firststreet.dev/docs)
 
 <a name="Search Item"></a>
-#### [Search Item](#toc)
+#### [Search Item:](#toc) 
 
-For every product, a list or file of `search items` must be provided to the product call. There are 3 types of `search items` corresponding to the 3 types of lookups.
+For every product, a list or file of `search items` must be provided to the product call. 
+There are 3 types of `search items` corresponding to the 3 types of lookups. 
+(More information on the Lookup types can be found on the [Lookups Page on the First Street Foundation API Data Dictionary](https://docs.firststreet.dev/docs/lookups)
+)
 
 1. FSID (`int`): The simplest type of lookup is a fsid lookup. If you know the fsid of the specific location, you can navigate directly to the specific product's information using the location's fsid. Example FSID: `18`
 
@@ -226,8 +239,9 @@ Example file of `search items`:
 <a name="location"></a>
 #### [Location](#toc)
 
-The Location API provides `Detail` and `Summary` data for the given FSIDs.
-
+The Location API provides `Detail` and `Summary` data for the given SearchItems.
+(More information on the Location product can be found on the [Location Page on the First Street Foundation API Data Dictionary](https://docs.firststreet.dev/docs/location-introduction)
+)
 ```python
 location.<method>
 ```
@@ -238,7 +252,9 @@ location.<method>
 <a name="probability"></a>
 #### [Probability](#toc)
 
-The Probability API provides `Depth`, `Chance`, `Cumulative`, `Count` data for the given FSIDs.
+The Probability API provides `Depth`, `Chance`, `Cumulative`, `Count` data for the given SearchItems.
+(More information on the Probability product can be found on the [Probability Page on the First Street Foundation API Data Dictionary](https://docs.firststreet.dev/docs/probability-depth)
+)
 
 ```python
 probability.<method>
@@ -253,7 +269,9 @@ probability.<method>
 <a name="historic"></a>
 #### [Historic](#toc)
 
-The Historic API provides `Summary` and `Event` data for the given FSIDs.
+The Historic API provides `Summary` and `Event` data for the given SearchItems.
+(More information on the Historic product can be found on the [Historic Page on the First Street Foundation API Data Dictionary](https://docs.firststreet.dev/docs/historic-summary)
+)
 
 ```python
 historic.<method>
@@ -265,8 +283,10 @@ historic.<method>
 
 <a name="adaptation"></a>
 #### [Adaptation](#toc)
+(More information on the Adaptation product can be found on the [Adaptation Page on the First Street Foundation API Data Dictionary](https://docs.firststreet.dev/docs/adaptation-introduction)
+)
 
-The Adaptation API provides `Summary` and `Project` data for the given FSIDs.
+The Adaptation API provides `Summary` and `Project` data for the given SearchItems.
 
 ```python
 adaptation.<method>
@@ -279,7 +299,9 @@ adaptation.<method>
 <a name="fema"></a>
 #### [Fema](#toc)
 
-The Fema API provides `NFIP` data for the given FSIDs.
+The Fema API provides `NFIP` data for the given SearchItems.
+(More information on the Fema NFIP product can be found on the [Fema NFIP Page on the First Street Foundation API Data Dictionary](https://docs.firststreet.dev/docs/fema-nfip)
+)
 
 ```python
 fema.<method>
@@ -290,7 +312,9 @@ fema.<method>
 <a name="environmental"></a>
 #### [Environmental](#toc)
 
-The Environmental API provides `Precipitation` data for the given FSIDs.
+The Environmental API provides `Precipitation` data for the given SearchItems.
+(More information on the Environmental Precipitation product can be found on the [Environmental Precipitation Page on the First Street Foundation API Data Dictionary](https://docs.firststreet.dev/docs/environmental-precipitation)
+)
 
 ```python
 environmental.<method>
@@ -371,7 +395,7 @@ environmental.<method>
 
 5. Multiple FSIDs Extraction to CSV Through the Command Line:
     ```sh
-    python -m firststreet -p historic.get_summary -i 1912000,1979140 -l property
+    python -m firststreet -p historic.get_summary -i 1912000;1979140 -l property
     ```
 
 6. Bulk FSIDs Extraction From File to CSV Through the Command Line:

firststreet/__main__.py

@@ -3,6 +3,7 @@
 
 # Standard Imports
 import argparse
+import ast
 import os
 import logging
 from distutils.util import strtobool
@@ -18,28 +19,28 @@
     parser.add_argument("-p", "--product", help="Example: adaptation_detail", required=True)
     parser.add_argument("-api_key", "--api_key", required=False)
     parser.add_argument("-v", "--version", required=False)
-    parser.add_argument("-i", "--fsids", help="Example: 28,29", required=False,)
+    parser.add_argument("-i", "--search_items", help="Example: 28,29", required=False,)
     parser.add_argument("-l", "--location", help="Example: property", required=False)
     parser.add_argument("-limit", "--limit", help="Example: 100", required=False, default="100")
     parser.add_argument("-log", "--log", help="Example: False", required=False, default="True")
     parser.add_argument("-f", "--file", help="Example: ./sample.txt", required=False)
 
     argument = parser.parse_args()
 
-    # Merge FSIDs from file and list input
-    fsids = []
-    if argument.fsids:
-        try:
-            fsids += list(map(int, argument.fsids.strip().split(",")))
-        except ValueError:
-            logging.warning("An invalid fsid list was provided. Please check this input list is a comma-separated " 
-                            "list of integers: '{}'".format(argument.fsids))
+    # Merge search_item from file and list input
+    search_items = []
+    if argument.search_items:
+        for search_item in argument.search_items.strip().split(";"):
+            try:
+                search_items.append(ast.literal_eval(search_item))
+            except (SyntaxError, ValueError):
+                search_items.append(search_item)
 
     if argument.file:
-        fsids += read_search_items_from_file(argument.file)
+        search_items += read_search_items_from_file(argument.file)
 
-    # Ensure there is at least a product and FSID
-    if fsids:
+    # Ensure there is at least a product and search item
+    if search_items:
 
         # Try to get the API key either from env var or the parameter
         if not argument.api_key:
@@ -57,54 +58,54 @@
         limit = int(argument.limit)
 
         if argument.product == 'adaptation.get_detail':
-            fs.adaptation.get_detail(fsids, csv=True, limit=limit)
+            fs.adaptation.get_detail(search_items, csv=True, limit=limit)
 
         elif argument.product == 'adaptation.get_summary':
-            fs.adaptation.get_summary(fsids, argument.location, csv=True, limit=limit)
+            fs.adaptation.get_summary(search_items, argument.location, csv=True, limit=limit)
 
         elif argument.product == 'adaptation.get_details_by_location':
-            fs.adaptation.get_details_by_location(fsids, argument.location, csv=True, limit=limit)
+            fs.adaptation.get_details_by_location(search_items, argument.location, csv=True, limit=limit)
 
         elif argument.product == 'probability.get_depth':
-            fs.probability.get_depth(fsids, csv=True, limit=limit)
+            fs.probability.get_depth(search_items, csv=True, limit=limit)
 
         elif argument.product == 'probability.get_chance':
-            fs.probability.get_chance(fsids, csv=True, limit=limit)
+            fs.probability.get_chance(search_items, csv=True, limit=limit)
 
         elif argument.product == 'probability.get_count_summary':
-            fs.probability.get_count_summary(fsids, csv=True, limit=limit)
+            fs.probability.get_count_summary(search_items, csv=True, limit=limit)
 
         elif argument.product == 'probability.get_cumulative':
-            fs.probability.get_cumulative(fsids, csv=True, limit=limit)
+            fs.probability.get_cumulative(search_items, csv=True, limit=limit)
 
         elif argument.product == 'probability.get_count':
-            fs.probability.get_count(fsids, argument.location, csv=True, limit=limit)
+            fs.probability.get_count(search_items, argument.location, csv=True, limit=limit)
 
         elif argument.product == 'historic.get_event':
-            fs.historic.get_event(fsids, csv=True, limit=limit)
+            fs.historic.get_event(search_items, csv=True, limit=limit)
 
         elif argument.product == 'historic.get_summary':
-            fs.historic.get_summary(fsids, argument.location, csv=True, limit=limit)
+            fs.historic.get_summary(search_items, argument.location, csv=True, limit=limit)
 
         elif argument.product == 'historic.get_events_by_location':
-            fs.historic.get_events_by_location(fsids, argument.location, csv=True, limit=limit)
+            fs.historic.get_events_by_location(search_items, argument.location, csv=True, limit=limit)
 
         elif argument.product == 'location.get_detail':
-            fs.location.get_detail(fsids, argument.location, csv=True, limit=limit)
+            fs.location.get_detail(search_items, argument.location, csv=True, limit=limit)
 
         elif argument.product == 'location.get_summary':
-            fs.location.get_summary(fsids, argument.location, csv=True, limit=limit)
+            fs.location.get_summary(search_items, argument.location, csv=True, limit=limit)
 
         elif argument.product == 'fema.get_nfip':
-            fs.fema.get_nfip(fsids, argument.location, csv=True, limit=limit)
+            fs.fema.get_nfip(search_items, argument.location, csv=True, limit=limit)
 
         elif argument.product == 'environmental.get_precipitation':
-            fs.environmental.get_precipitation(fsids, csv=True, limit=limit)
+            fs.environmental.get_precipitation(search_items, csv=True, limit=limit)
 
         else:
             logging.error("Product not found. Please check that the argument"
                           " provided is correct: {}".format(argument.product))
 
     else:
-        raise InvalidArgument("No fsids were provided from either a fsid list or a file. "
-                              "List: '{}', File Name: '{}'".format(argument.fsids, argument.file))
+        raise InvalidArgument("No search items were provided from either a search item list or a file. "
+                              "List: '{}', File Name: '{}'".format(argument.search_items, argument.file))

firststreet/api/adaptation.py

@@ -12,16 +12,16 @@
 
 
 class Adaptation(Api):
-    """This class receives a list of fsids and handles the creation of a adaptation product from the request.
+    """This class receives a list of search_items and handles the creation of a adaptation product from the request.
 
         Methods:
             get_detail: Retrieves a list of Adaptation Details for the given list of IDs
             get_summary: Retrieves a list of Adaptation Summary for the given list of IDs
         """
 
     def get_detail(self, search_item, csv=False, limit=100, output_dir=None):
-        """Retrieves adaptation detail product data from the First Street Foundation API given a list of FSIDs and
-        returns a list of Adaptation Detail objects.
+        """Retrieves adaptation detail product data from the First Street Foundation API given a list of search_items
+         and returns a list of Adaptation Detail objects.
 
         Args:
             search_item (list/file): A First Street Foundation IDs, lat/lng pair, address, or a
@@ -45,7 +45,7 @@ def get_detail(self, search_item, csv=False, limit=100, output_dir=None):
 
     def get_details_by_location(self, search_item, location_type, csv=False, limit=100, output_dir=None):
         """Retrieves adaptation detail product data from the First Street Foundation API given a list of location
-        FSIDs and returns a list of Adaptation Detail objects.
+        search_items and returns a list of Adaptation Detail objects.
 
         Args:
             search_item (list/file): A First Street Foundation IDs, lat/lng pair, address, or a
@@ -70,11 +70,11 @@ def get_details_by_location(self, search_item, location_type, csv=False, limit=1
         api_datas_summary = self.call_api(search_item, "adaptation", "summary", location_type, limit=limit)
         summary = [AdaptationSummary(api_data) for api_data in api_datas_summary]
 
-        fsids = list(set([adaptation for sum_adap in summary if sum_adap.adaptation for
+        search_items = list(set([adaptation for sum_adap in summary if sum_adap.adaptation for
                           adaptation in sum_adap.adaptation]))
 
-        if fsids:
-            api_datas_detail = self.call_api(fsids, "adaptation", "detail", None, limit=limit)
+        if search_items:
+            api_datas_detail = self.call_api(search_items, "adaptation", "detail", None, limit=limit)
 
         else:
             api_datas_detail = [{"adaptationId": None}]
@@ -89,8 +89,8 @@ def get_details_by_location(self, search_item, location_type, csv=False, limit=1
         return [summary, detail]
 
     def get_summary(self, search_item, location_type, csv=False, limit=100, output_dir=None):
-        """Retrieves adaptation summary product data from the First Street Foundation API given a list of FSIDs and
-        returns a list of Adaptation Summary objects.
+        """Retrieves adaptation summary product data from the First Street Foundation API given a list of
+        search_items and returns a list of Adaptation Summary objects.
 
         Args:
             search_item (list/file): A First Street Foundation IDs, lat/lng pair, address, or a

firststreet/api/environmental.py

@@ -11,15 +11,15 @@
 
 
 class Environmental(Api):
-    """This class receives a list of fsids and handles the creation of a environmental product from the request.
+    """This class receives a list of search_items and handles the creation of a environmental product from the request.
 
         Methods:
             get_precipitation: Retrieves a list of Environmental Precipitation for the given list of IDs
         """
 
     def get_precipitation(self, search_item, csv=False, limit=100, output_dir=None):
-        """Retrieves environmental precipitation product data from the First Street Foundation API given a list of FSIDs
-        and returns a list of Environmental Precipitation objects.
+        """Retrieves environmental precipitation product data from the First Street Foundation API given a list of
+        search_items and returns a list of Environmental Precipitation objects.
 
         Args:
             search_item (list/file): A First Street Foundation IDs, lat/lng pair, address, or a

firststreet/api/fema.py

@@ -12,14 +12,14 @@
 
 
 class Fema(Api):
-    """This class receives a list of fsids and handles the creation of a fema product from the request.
+    """This class receives a list of search_items and handles the creation of a fema product from the request.
 
         Methods:
             get_nfip: Retrieves a list of Fema Nfip for the given list of IDs
         """
 
     def get_nfip(self, search_item, location_type, csv=False, limit=100, output_dir=None):
-        """Retrieves fema nfip product data from the First Street Foundation API given a list of FSIDs and
+        """Retrieves fema nfip product data from the First Street Foundation API given a list of search_items and
         returns a list of Fema Nfip objects.
 
         Args: