Skip to content
This repository was archived by the owner on Jan 18, 2023. It is now read-only.

Commit b303819

Browse files
authored
Merge pull request #19 from FirstStreet/develop
Merge develop for new release
2 parents 7f106bb + ec34e2c commit b303819

14 files changed

Lines changed: 513 additions & 60 deletions

File tree

.circleci/config.yml

Lines changed: 3 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -4,14 +4,13 @@ workflows:
44
version: 2
55
test:
66
jobs:
7-
- test-3.6
87
- test-3.7
98
- test-3.8
109

1110
jobs:
12-
test-3.6: &test-template
11+
test-3.7: &test-template
1312
docker:
14-
- image: circleci/python:3.6
13+
- image: circleci/python:3.7
1514

1615
working_directory: ~/fsf-api-access-python
1716

@@ -40,12 +39,8 @@ jobs:
4039
. venv/bin/activate
4140
pytest tests
4241
43-
test-3.7:
44-
<<: *test-template
45-
docker:
46-
- image: circleci/python:3.7
47-
4842
test-3.8:
4943
<<: *test-template
5044
docker:
5145
- image: circleci/python:3.8
46+

README.md

Lines changed: 31 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
# First Street Foundation API Access Documentation (Python 3.6+)
1+
# First Street Foundation API Access Documentation (Python 3.7+)
22
[![CircleCI](https://img.shields.io/circleci/build/gh/FirstStreet/fsf_api_access_python)](https://circleci.com/gh/FirstStreet/fsf_api_access_python)
33
![GitHub](https://img.shields.io/github/license/firststreet/fsf_api_access_python)
44

@@ -22,6 +22,7 @@ The First Street Foundation API Access (Python) is a wrapper used to bulk extrac
2222
- [Adaptation](#adaptation)
2323
- [Fema](#fema)
2424
- [Environmental](#environmental)
25+
- [Tile](#tiles)
2526
- **[Examples](#examples)**
2627
- **[CSV File Output:](#csv-output)**
2728
- [CSV File Name:](#csv-name)
@@ -31,7 +32,7 @@ The First Street Foundation API Access (Python) is a wrapper used to bulk extrac
3132

3233
<a name="installation"></a>
3334
# [Installation](#toc)
34-
**NOTE**: This project requires [Python](https://www.python.org/downloads/) 3.6+ to run.
35+
**NOTE**: This project requires [Python](https://www.python.org/downloads/) 3.7+ to run.
3536

3637
1. Go to the Python page (https://www.python.org/downloads/) and download then install Python version 3. **Make sure that the checkbox is checked for Python to be added to the PATH**
3738

@@ -136,6 +137,18 @@ The First Street Foundation API Access (Python) is a wrapper used to bulk extrac
136137

137138
Example: ```-e extra_param```
138139

140+
- `[-year/--year YEAR]`: [OPTIONAL] The year to use for the `Probability Depth Tile` product
141+
142+
Example: ```-year 2050```
143+
144+
- `[-return_period/--return_period RETURN_PERIOD]`: [OPTIONAL] The return period to use for the `Probability Depth Tile` product
145+
146+
Example: ```-return_period 500```
147+
148+
- `[-event_id/--event_id EVENT_ID]`: [OPTIONAL] The event id to use for the `Historic Event Tile` product
149+
150+
Example: ```-event_id 3```
151+
139152
- `[-f/--file FILE]`: [OPTIONAL] A file of Search Items (one per line) to search for with the product
140153

141154
Example: ```-f sample.txt```
@@ -326,6 +339,20 @@ environmental.<method>
326339

327340
* `get_precipitation`(search_items `list/file`, csv `bool`, [connection_limit `int=100`], [output_dir `str='cwd'`], [extra_param `str=None`]) - Returns an array of `Environmental Precipitation` product for the given county IDs, and optionally creates a csv file. Arguments provided to `extra_param` will be appened to the end of the endpoint call
328341

342+
<a name="tiles"></a>
343+
#### [Tiles](#toc)
344+
345+
The Flood Tiles product give the ability to customize maps by layering on flood tiles.
346+
(More information on the Tile product can be found on the [Tile Page on the First Street Foundation API Data Dictionary](https://docs.firststreet.dev/docs/tiles-introduction)
347+
)
348+
349+
```python
350+
tile.<method>
351+
```
352+
353+
* `get_probability_depth`(coordinate `tuple of int`, year `int`, return_period `int`, image `bool`, [connection_limit `int=100`], [output_dir `str='cwd'`], [extra_param `str=None`]) - Returns an array of `Probability Depth Tile` product for the given coordinates, and optionally creates an image file
354+
* `get_historic_event`(coordinate `tuple of int`, event_id `int`, image `bool`, [connection_limit `int=100`], [output_dir `str='cwd'`], [extra_param `str=None`]) - Returns an array of `Historic Event Tile` product for the given coordinates, and optionally creates an image file
355+
329356
<a name="examples"></a>
330357
# [Examples](#toc)
331358
**[Reminder] Keep your API key safe, and do not share it with others!**
@@ -495,6 +522,8 @@ Ex:
495522
#### [CSV File Content](#toc)
496523
The contents of the CSV file will follow similar formats as the `First Street Foundation API - V1.0 Overview and Dictionary`, but all lists will be expanded to a flat file. For any values that are null or not available, `<NA>` will be used.
497524

525+
If any of the input FSIDs are invalid, a `valid_id` column will appear with the invalid FSIDs marked with `False`
526+
498527
Ex:
499528
```csv
500529
fsid,year,returnPeriod,bin,low,mid,high

firststreet/__init__.py

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -12,6 +12,7 @@
1212
from firststreet.api.historic import Historic
1313
from firststreet.api.location import Location
1414
from firststreet.api.probability import Probability
15+
from firststreet.api.tile import Tile
1516
from firststreet.errors import MissingAPIKeyError
1617
from firststreet.http_util import Http
1718

@@ -52,3 +53,4 @@ def __init__(self, api_key=None, version=None, log=True):
5253
self.adaptation = Adaptation(self.http)
5354
self.environmental = Environmental(self.http)
5455
self.fema = Fema(self.http)
56+
self.tile = Tile(self.http)

firststreet/__main__.py

Lines changed: 17 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -25,6 +25,9 @@
2525
parser.add_argument("-log", "--log", help="Example: False", required=False, default="True")
2626
parser.add_argument("-f", "--file", help="Example: ./sample.txt", required=False)
2727
parser.add_argument("-e", "--extra_param", required=False)
28+
parser.add_argument("-year", "--year", required=False)
29+
parser.add_argument("-return_period", "--return_period", required=False)
30+
parser.add_argument("-event_id", "--event_id", required=False)
2831

2932
argument = parser.parse_args()
3033

@@ -56,7 +59,7 @@
5659

5760
fs = firststreet.FirstStreet(api_key, version=argument.version, log=bool(strtobool(argument.log)))
5861

59-
limit = int(argument.limit)
62+
limit = int(argument.connection_limit)
6063

6164
if argument.product == 'adaptation.get_detail':
6265
fs.adaptation.get_detail(search_items,
@@ -156,6 +159,19 @@
156159
connection_limit=limit,
157160
extra_param=argument.extra_param)
158161

162+
elif argument.product == 'tile.get_probability_depth':
163+
fs.tile.get_probability_depth(year=int(argument.year), return_period=int(argument.return_period),
164+
coordinate=search_items,
165+
image=True,
166+
connection_limit=limit)
167+
168+
elif argument.product == 'tile.get_historic_event':
169+
fs.tile.get_historic_event(event_id=int(argument.event_id), coordinate=search_items,
170+
image=True,
171+
connection_limit=limit)
172+
173+
# AND FILES
174+
159175
else:
160176
logging.error("Product not found. Please check that the argument"
161177
" provided is correct: {}".format(argument.product))

firststreet/api/api.py

Lines changed: 34 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -23,7 +23,8 @@ class Api:
2323
def __init__(self, http):
2424
self._http = http
2525

26-
def call_api(self, search_item, product, product_subtype, location, connection_limit=100, extra_param=None):
26+
def call_api(self, search_item, product, product_subtype, location=None, tile_product=None, year=None,
27+
return_period=None, event_id=None, connection_limit=100, extra_param=None):
2728
"""Receives an item, a product, a product subtype, and a location to create and call an endpoint to the First
2829
Street Foundation API.
2930
@@ -33,6 +34,10 @@ def call_api(self, search_item, product, product_subtype, location, connection_l
3334
product (str): The overall product to call
3435
product_subtype (str): The product subtype (if suitable)
3536
location (str/None): The location type (if suitable)
37+
tile_product (str/None): The tile product (if suitable)
38+
year (int/None): The year for probability depth tiles (if suitable)
39+
return_period (int/None): The return period for probability depth tiles (if suitable)
40+
event_id (int/None): The event_id for historic tiles (if suitable)
3641
connection_limit (int): max number of connections to make
3742
extra_param (str): Extra parameter to be added to the url
3843
Returns:
@@ -55,6 +60,16 @@ def call_api(self, search_item, product, product_subtype, location, connection_l
5560
raise InvalidArgument("File provided is not a list or a valid file. "
5661
"Please check the file name. '{}'".format(os.path.curdir + str(search_item)))
5762

63+
if tile_product:
64+
if not all(isinstance(t, tuple) for t in search_item):
65+
raise TypeError("Input must be a list of coordinates in a tuple of (z, x, y). Provided Arg: {}".format(search_item))
66+
67+
if not all(isinstance(coord, int) for t in search_item for coord in t):
68+
raise TypeError("Each coordinate in the tuple must be an integer. Provided Arg: {}".format(search_item))
69+
70+
if not all(0 < t[0] <= 18 for t in search_item):
71+
raise TypeError("Max zoom is 18. Provided Arg: {}".format(search_item))
72+
5873
# No items found
5974
if not search_item:
6075
raise InvalidArgument(search_item)
@@ -67,20 +82,29 @@ def call_api(self, search_item, product, product_subtype, location, connection_l
6782
for item in search_item:
6883
if location:
6984
endpoint = "/".join([base_url, version, product, product_subtype, location])
85+
elif tile_product:
86+
if event_id:
87+
endpoint = "/".join([base_url, version, product, product_subtype, tile_product,
88+
str(event_id), "/".join(map(str, item))])
89+
else:
90+
endpoint = "/".join([base_url, version, product, product_subtype, tile_product,
91+
str(year), str(return_period), "/".join(map(str, item))])
7092
else:
7193
endpoint = "/".join([base_url, version, product, product_subtype])
7294

73-
# fsid
74-
if isinstance(item, int):
75-
endpoint = endpoint + "/{}".format(item) + "?{}".format(extra_param)
95+
if not tile_product:
96+
97+
# fsid
98+
if isinstance(item, int):
99+
endpoint = endpoint + "/{}".format(item) + "?{}".format(extra_param)
76100

77-
# lat/lng
78-
elif isinstance(item, tuple):
79-
endpoint = endpoint + "?lat={}&lng={}&{}".format(item[0], item[1], extra_param)
101+
# lat/lng
102+
elif isinstance(item, tuple):
103+
endpoint = endpoint + "?lat={}&lng={}&{}".format(item[0], item[1], extra_param)
80104

81-
# address
82-
elif isinstance(item, str):
83-
endpoint = endpoint + "?address={}&{}".format(item, extra_param)
105+
# address
106+
elif isinstance(item, str):
107+
endpoint = endpoint + "?address={}&{}".format(item, extra_param)
84108

85109
endpoints.append((endpoint, item, product, product_subtype))
86110

firststreet/api/csv_format.py

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -147,10 +147,10 @@ def to_csv(data, product, product_subtype, location_type=None, output_dir=None):
147147
raise NotImplementedError
148148

149149
# Export CSVs
150-
if df['valid_id'].isna().values.all():
150+
if df['valid_id'].all():
151151
df = df.drop(columns=['valid_id'])
152152
else:
153-
df = df.fillna(True)
153+
df['valid_id'] = df['valid_id'].fillna(True)
154154

155155
df.fillna(pd.NA).astype(str).to_csv(output_dir + '/' + file_name, index=False)
156156
logging.info("CSV generated to '{}'.".format(output_dir + '/' + file_name))

firststreet/api/historic.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -74,7 +74,7 @@ def get_events_by_location(self, search_item, location_type, csv=False, connecti
7474
raise TypeError("location is not a string")
7575

7676
# Get data from api and create objects
77-
api_datas = self.call_api(search_item, "historic", "summary", location_type, connection_limit)
77+
api_datas = self.call_api(search_item, "historic", "summary", location_type, connection_limit=connection_limit)
7878
summary = [HistoricSummary(api_data) for api_data in api_datas]
7979

8080
search_item = list(set([event.get("eventId") for sum_hist in summary if sum_hist.historic for

firststreet/api/tile.py

Lines changed: 142 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,142 @@
1+
# Author: Kelvin Lai <kelvin@firststreet.org>
2+
# Copyright: This module is owned by First Street Foundation
3+
4+
# Standard Imports
5+
import datetime
6+
import logging
7+
8+
# Internal Imports
9+
import os
10+
11+
from firststreet.api.api import Api
12+
from firststreet.errors import InvalidArgument
13+
from firststreet.models.tile import ProbabilityDepthTile, HistoricEventTile
14+
15+
16+
class Tile(Api):
17+
"""This class receives a list of coordinate and parameters, and handles the return of a tile from the request.
18+
19+
Methods:
20+
get_probability_depth: Retrieves a list of Probability Depth for the given list of IDs
21+
get_historic_event: Retrieves a list of Probability Depth for the given list of IDs
22+
"""
23+
24+
def get_probability_depth(self, year, return_period, coordinate, image=False, connection_limit=100,
25+
output_dir=None, extra_param=None):
26+
"""Retrieves probability depth tile data from the First Street Foundation API given a list of search_items
27+
and returns a list of Probability Depth Tile objects.
28+
29+
Args:
30+
year (int): The year to get the tile
31+
return_period (int): The return period to get the tile
32+
coordinate (list of tuple): A list of coordinates in the form of [(x_1, y_1, z_1), (x_2, y_2, z_2), ...]
33+
image (bool): To output extracted image to a png or not
34+
connection_limit (int): max number of connections to make
35+
output_dir (str): The output directory to save the generated tile
36+
extra_param (str): Extra parameter to be added to the url
37+
38+
Returns:
39+
A list of Probability Depth tiles
40+
"""
41+
42+
if not year:
43+
raise InvalidArgument(year)
44+
elif not isinstance(year, int):
45+
raise TypeError("year is not an int")
46+
elif year not in [2020, 2035, 2050]:
47+
logging.error("Year provided is not one of: 2020, 2035, 2050")
48+
raise InvalidArgument(year)
49+
50+
if not return_period:
51+
raise InvalidArgument(return_period)
52+
elif not isinstance(return_period, int):
53+
raise TypeError("return period is not an int")
54+
elif return_period not in [500, 100, 20, 5, 2]:
55+
logging.error("Year provided is not one of: 500, 100, 20, 5, 2. "
56+
"(2 year return period is only available for coastal areas.)")
57+
raise InvalidArgument(year)
58+
59+
# Get data from api and create objects
60+
api_datas = self.call_api(coordinate, "tile", "probability", tile_product="depth", year=year,
61+
return_period=return_period, connection_limit=connection_limit,
62+
extra_param=extra_param)
63+
64+
if image:
65+
for data in api_datas:
66+
if data:
67+
date = datetime.datetime.today().strftime('%Y_%m_%d_%H_%M_%S')
68+
69+
# Set file name to the current date, time, and product
70+
file_name = "_".join([date, "probability_depth", str(year), str(return_period),
71+
str(data.get("coordinate"))]) + ".png"
72+
73+
if not output_dir:
74+
output_dir = os.getcwd() + "/data_csv"
75+
76+
if not os.path.exists(output_dir):
77+
os.makedirs(output_dir)
78+
79+
with open(output_dir + '/' + file_name, "wb") as f:
80+
f.write(data['image'])
81+
82+
logging.info("Images generated to '{}'.".format(output_dir))
83+
84+
product = [ProbabilityDepthTile(api_data, year, return_period) for api_data in api_datas]
85+
86+
logging.info("Probability Depth Tile Ready.")
87+
88+
return product
89+
90+
def get_historic_event(self, event_id, coordinate, image=False, connection_limit=100,
91+
output_dir=None, extra_param=None):
92+
"""Retrieves historic event tile data from the First Street Foundation API given a list of search_items
93+
and returns a list of Historic Event Tile objects.
94+
95+
Args:
96+
event_id (int): A First Street Foundation eventId
97+
coordinate (list of tuple): A list of coordinates in the form of [(x_1, y_1, z_1), (x_2, y_2, z_2), ...]
98+
image (bool): To output extracted image to a png or not
99+
connection_limit (int): max number of connections to make
100+
output_dir (str): The output directory to save the generated tile
101+
extra_param (str): Extra parameter to be added to the url
102+
103+
Returns:
104+
A list of Probability Count
105+
Raises:
106+
InvalidArgument: The location provided is empty
107+
TypeError: The location provided is not a string
108+
"""
109+
110+
if not event_id:
111+
raise InvalidArgument(event_id)
112+
elif not isinstance(event_id, int):
113+
raise TypeError("event id is not an int")
114+
115+
# Get data from api and create objects
116+
api_datas = self.call_api(coordinate, "tile", "historic", tile_product="event", event_id=event_id,
117+
connection_limit=connection_limit, extra_param=extra_param)
118+
119+
if image:
120+
for data in api_datas:
121+
if data:
122+
date = datetime.datetime.today().strftime('%Y_%m_%d_%H_%M_%S')
123+
124+
# Set file name to the current date, time, and product
125+
file_name = "_".join([date, "historic_event", str(event_id), str(data.get("coordinate"))]) + ".png"
126+
127+
if not output_dir:
128+
output_dir = os.getcwd() + "/data_csv"
129+
130+
if not os.path.exists(output_dir):
131+
os.makedirs(output_dir)
132+
133+
with open(output_dir + '/' + file_name, "wb") as f:
134+
f.write(data.get("image"))
135+
136+
logging.info("Images generated to '{}'.".format(output_dir))
137+
138+
product = [HistoricEventTile(api_data, event_id) for api_data in api_datas]
139+
140+
logging.info("Historic Event Tile Ready.")
141+
142+
return product

0 commit comments

Comments
 (0)