Skip to content

Era5/hourly #145

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
tommylees112 wants to merge 26 commits into
base: master
Choose a base branch
from
Open

Era5/hourly #145

Changes from all commits
Commits
Show all changes
26 commits
Select commit Hold shift + click to select a range
bd001c0
UPDATE the break_up to be yearly
tommylees112 Jan 16, 2020
9da52f7
blackify code
tommylees112 Jan 16, 2020
e1a138e
remove unnecessary import
tommylees112 Jan 16, 2020
543b0ea
update the preprocessor for hourly data too
tommylees112 Jan 16, 2020
df1aff0
update tests
tommylees112 Jan 16, 2020
d382dbc
blackify
tommylees112 Jan 16, 2020
4029a57
import pandas
tommylees112 Jan 16, 2020
8a8c52e
fix tests
tommylees112 Jan 16, 2020
9c5a5f5
blackify code
tommylees112 Jan 16, 2020
6dc8673
update script to include preprocessor for era5 hourly
tommylees112 Jan 17, 2020
7647050
test merge by variable
tommylees112 Jan 17, 2020
bfb4d88
test merge by variable
tommylees112 Jan 17, 2020
226557c
split merge by variable
tommylees112 Jan 17, 2020
e6e45d1
variable name
tommylees112 Jan 17, 2020
63cb74a
only update hourly preprocessor
tommylees112 Jan 17, 2020
a440a31
update preprocessor
tommylees112 Jan 17, 2020
339a423
extract to function to clear memory
tommylees112 Jan 20, 2020
dd4e3fc
extract to function to clear memory
tommylees112 Jan 20, 2020
85c2478
update for experiment
tommylees112 Jan 20, 2020
836bee9
update merge all files to index using numpy
tommylees112 Jan 20, 2020
c2b6e69
update preprocess to do BY variable
tommylees112 Jan 21, 2020
96ab6eb
better comments
tommylees112 Jan 21, 2020
b176008
ignore
tommylees112 Aug 11, 2020
3716806
Merge branch 'master' of https://github.com/esowc/ml_drought into era…
tommylees112 Aug 11, 2020
9facf47
black .
tommylees112 Aug 11, 2020
26c19c1
Merge branch 'master' into era5/export
tommylees112 Aug 11, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 13 additions & 0 deletions scripts/preprocess.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -270,6 +270,19 @@ def preprocess_boku_ndvi(subset_str: str = "kenya"):
)


def preprocess_era5_hourly():
data_path = get_data_path()

regrid_path = data_path / "interim/VCI_preprocessed/data_kenya.nc"
assert regrid_path.exists(), f"{regrid_path} not available"

processor = ERA5HourlyPreprocessor(data_path)

# W-MON is weekly each monday (the same as the NDVI data from Atzberger)
# processor.preprocess(subset_str="kenya", resample_time="W-MON")
processor.merge_files(subset_str="W-MON")


def preprocess_s5_ouce():
if Path(".").absolute().as_posix().split("/")[-1] == "ml_drought":
data_path = Path("data")
Expand Down