Merge branch 'develop'

# Conflicts:
#	CHANGELOG.md
#	climada/_version.py
#	setup.py

Resolved using 'develop'

Author: emanuel-schmid

.github/workflows/ci.yml

@@ -26,7 +26,7 @@ jobs:
     steps:
       -
         name: Checkout Repo
-        uses: actions/checkout@v3
+        uses: actions/checkout@v4
       -
         # Store the current date to use it as cache key for the environment
         name: Get current date
@@ -64,7 +64,7 @@ jobs:
       -
         name: Upload Coverage Reports
         if: always()
-        uses: actions/upload-artifact@v3
+        uses: actions/upload-artifact@v4
         with:
           name: coverage-report-unittests-py${{ matrix.python-version }}
           path: coverage/

AUTHORS.md

@@ -29,3 +29,6 @@
 * Raphael Portmann
 * Nicolas Colombi
 * Leonie Villiger
+* Kam Lam Yeung
+* Sarah Hülsen
+* Timo Schmid

CHANGELOG.md

@@ -1,5 +1,71 @@
 # Changelog
 
+## Unreleased
+
+Release date: YYYY-MM-DD
+
+Code freeze date: YYYY-MM-DD
+
+### Description
+
+### Dependency Changes
+
+Added:
+
+- `pyproj` >=3.5
+- `numexpr` >=2.9
+
+Updated:
+
+- `contextily` >=1.3 → >=1.5
+- `dask` >=2023 → >=2024
+- `numba` >=0.57 → >=0.59
+- `pandas` >=2.1 &rarr; >=2.1,<2.2
+- `pint` >=0.22 &rarr; >=0.23
+- `scikit-learn` >=1.3 &rarr; >=1.4
+- `scipy` >=1.11 &rarr; >=1.12
+- `sparse` >=0.14 &rarr; >=0.15
+- `xarray` >=2023.8 &rarr; >=2024.1
+- `overpy` =0.6 &rarr; =0.7
+- `peewee` =3.16.3 &rarr; =3.17.1
+
+Removed:
+
+- `proj` (in favor of `pyproj`)
+
+### Added
+
+- Convenience method `api_client.Client.get_dataset_file`, combining `get_dataset_info` and `download_dataset`, returning a single file objet. [#821](https://github.com/CLIMADA-project/climada_python/pull/821)
+- Read and Write methods to and from csv files for the `DiscRates` class. [#818](ttps://github.com/CLIMADA-project/climada_python/pull/818)
+- Add `CalcDeltaClimate` to unsequa module to allow uncertainty and sensitivity analysis of impact change calculations [#844](https://github.com/CLIMADA-project/climada_python/pull/844)
+- Add function `safe_divide` in util which handles division by zero and NaN values in the numerator or denominator [#844](https://github.com/CLIMADA-project/climada_python/pull/844)
+- Add reset_frequency option for the impact.select() function. [#847](https://github.com/CLIMADA-project/climada_python/pull/847)
+
+### Changed
+
+- Update Developer and Installation Guides for easier accessibility by new developers. [808](https://github.com/CLIMADA-project/climada_python/pull/808)
+- Add `shapes` argument to `geo_im_from_array` to allow flexible turning on/off of plotting coastline in `plot_intensity`. [#805](https://github.com/CLIMADA-project/climada_python/pull/805)
+- Update `CONTRIBUTING.md` to better explain types of contributions to this repository [#797](https://github.com/CLIMADA-project/climada_python/pull/797)
+- The default tile layer in Exposures maps is not Stamen Terrain anymore, but [CartoDB Positron](https://github.com/CartoDB/basemap-styles). Affected methods are `climada.engine.Impact.plot_basemap_eai_exposure`,`climada.engine.Impact.plot_basemap_impact_exposure` and `climada.entity.Exposures.plot_basemap`. [#798](https://github.com/CLIMADA-project/climada_python/pull/798)
+- Recommend using Mamba instead of Conda for installing CLIMADA [#809](https://github.com/CLIMADA-project/climada_python/pull/809)
+- `Hazard.from_xarray_raster` now allows arbitrary values as 'event' coordinates [#837](https://github.com/CLIMADA-project/climada_python/pull/837)
+- `climada.test.get_test_file` now compares the version of the requested test dataset with the version of climada itself and selects the most appropriate dataset. In this way a test file can be updated without the need of changing the code of the unittest. [#822](https://github.com/CLIMADA-project/climada_python/pull/822)
+- Explicitly require `pyproj` instead of `proj` (the latter is now implicitly required) [#845](https://github.com/CLIMADA-project/climada_python/pull/845)
+
+### Fixed
+
+- `Hazard.from_xarray_raster` now stores strings as default values for `Hazard.event_name` [#795](https://github.com/CLIMADA-project/climada_python/pull/795)
+- Fix the dist_approx util function when used with method="geosphere" and log=True and points that are very close. [#792](https://github.com/CLIMADA-project/climada_python/pull/792)
+- `climada.util.yearsets.sample_from_poisson`: fix a bug ([#819](https://github.com/CLIMADA-project/climada_python/issues/819)) and inconsistency that occurs when lambda events per year (`lam`) are set to 1. [[#823](https://github.com/CLIMADA-project/climada_python/pull/823)]
+- In the TropCyclone class in the Holland model 2008 and 2010 implementation, a doublecounting of translational velocity is removed [#833](https://github.com/CLIMADA-project/climada_python/pull/833)
+- `climada.util.test.test_finance` and `climada.test.test_engine` updated to recent input data from worldbank [#841](https://github.com/CLIMADA-project/climada_python/pull/841)
+- Set `nodefaults` in Conda environment specs because `defaults` are not compatible with conda-forge [#845](https://github.com/CLIMADA-project/climada_python/pull/845)
+- Avoid redundant calls to `np.unique` in `Impact.impact_at_reg` [#848](https://github.com/CLIMADA-project/climada_python/pull/848)
+
+### Deprecated
+
+### Removed
+
 ## 4.0.1
 
 Release date: 2023-09-27

CONTRIBUTING.md

@@ -1,8 +1,26 @@
 # CLIMADA Contribution Guide
 
-Thank you for contributing to CLIMADA!
+We welcome any contribution to CLIMADA and want to express our thanks to everybody who contributes.
 
-## Overview
+## What Warrants a Contribution?
+
+Anything!
+For orientation, these are some categories of possible contributions we can think of:
+
+* **Technical problems and bugs:** Did you encounter a problem when using CLIMADA? Raise an [issue](https://github.com/CLIMADA-project/climada_python/issues) in our repository, providing a description or ideally a code replicating the error. Did you already find a solution to the problem? Please raise a pull request to help us resolve the issue!
+* **Documentation and Tutorial Updates:** Found a typo in the documentation? Is a tutorial lacking some information you find important? Simply fix a line, or add a paragraph. We are happy to incorporate you additions! Please raise a pull request!
+* **New Modules and Utility Functions:** Did you create a function or an entire module you find useful for your work? Maybe you are not the only one! Feel free to simply raise a pull request for functions that improve, e.g., plotting or data handling. As an entire module has to be carefully integrated into the framework, it might help if you talk to us first so we can design the module and plan the next steps. You can do that by raising an issue or starting a [discussion](https://github.com/CLIMADA-project/climada_python/discussions) on GitHub.
+
+A good place to start a personal discussion is our monthly CLIMADA developers call.
+Please contact the [lead developers](https://wcr.ethz.ch/research/climada.html) if you want to join.
+
+## Why Should You Contribute?
+
+* You will be listed as author of the CLIMADA repository in the [AUTHORS](AUTHORS.md) file.
+* You will improve the quality of the CLIMADA software for you and for everybody else using it.
+* You will gain insights into scientific software development.
+
+## Minimal Steps to Contribute
 
 Before you start, please have a look at our [Developer Guide][devguide].
 

README.md

@@ -21,13 +21,14 @@ This is the Python (3.9+) version of CLIMADA - please see [here](https://github.
 ## Getting started
 
 CLIMADA runs on Windows, macOS and Linux.
-The released versions of the CLIMADA core can be installed directly through Anaconda:
+The released versions of CLIMADA are available from [conda-forge](https://anaconda.org/conda-forge/climada).
+Use the [Mamba](https://mamba.readthedocs.io/en/latest/) package manager to install it:
 
 ```shell
-conda install -c conda-forge climada
+mamba install -c conda-forge climada
 ```
 
-It is **highly recommended** to install CLIMADA into a **separate** Anaconda environment.
+It is **highly recommended** to install CLIMADA into a **separate** Conda environment.
 See the [installation guide](https://climada-python.readthedocs.io/en/latest/guide/install.html) for further information.
 
 Follow the [tutorials](https://climada-python.readthedocs.io/en/stable/tutorial/1_main_climada.html) in a Jupyter Notebook to see what can be done with CLIMADA and how.
@@ -61,7 +62,9 @@ Please use the following logo if you are presenting results obtained with or thr
 
 ## Contributing
 
-See the [Contribution Guide](CONTRIBUTING.md).
+We welcome any contribution to this repository, be it bugfixes and other code changes and additions, documentation improvements, or tutorial updates.
+
+If you would like to contribute, please refer to our [Contribution Guide](CONTRIBUTING.md).
 
 ## Versioning
 

climada/_version.py

@@ -1 +1 @@
-__version__ = '4.0.1'
+__version__ = '4.0.2-dev'

climada/engine/impact.py

@@ -451,12 +451,12 @@ def impact_at_reg(self, agg_regions=None):
         at_reg_event = np.hstack(
             [
                 self.imp_mat[:, np.where(agg_regions == reg)[0]].sum(1)
-                for reg in np.unique(agg_reg_unique)
+                for reg in agg_reg_unique
             ]
         )
 
         at_reg_event = pd.DataFrame(
-            at_reg_event, columns=np.unique(agg_reg_unique), index=self.event_id
+            at_reg_event, columns=agg_reg_unique, index=self.event_id
         )
 
         return at_reg_event
@@ -673,7 +673,7 @@ def plot_raster_eai_exposure(self, res=None, raster_res=None, save_tiff=None,
 
     def plot_basemap_eai_exposure(self, mask=None, ignore_zero=False, pop_name=True,
                                   buffer=0.0, extend='neither', zoom=10,
-                                  url=ctx.providers.Stamen.Terrain,
+                                  url=ctx.providers.CartoDB.Positron,
                                   axis=None, **kwargs):
         """Plot basemap expected impact of each exposure within a period of 1/frequency_unit.
 
@@ -694,7 +694,7 @@ def plot_basemap_eai_exposure(self, mask=None, ignore_zero=False, pop_name=True,
         zoom : int, optional
             zoom coefficient used in the satellite image
         url : str, optional
-            image source, e.g. ctx.providers.OpenStreetMap.Mapnik
+            image source, default: ctx.providers.CartoDB.Positron
         axis : matplotlib.axes.Axes, optional
             axis to use
         kwargs : dict, optional
@@ -764,7 +764,7 @@ def plot_hexbin_impact_exposure(self, event_id=1, mask=None, ignore_zero=False,
 
     def plot_basemap_impact_exposure(self, event_id=1, mask=None, ignore_zero=False,
                                      pop_name=True, buffer=0.0, extend='neither', zoom=10,
-                                     url=ctx.providers.Stamen.Terrain,
+                                     url=ctx.providers.CartoDB.Positron,
                                      axis=None, **kwargs):
         """Plot basemap impact of an event at each exposure.
         Requires attribute imp_mat.
@@ -789,7 +789,7 @@ def plot_basemap_impact_exposure(self, event_id=1, mask=None, ignore_zero=False,
         zoom : int, optional
             zoom coefficient used in the satellite image
         url : str, optional
-            image source, e.g. ctx.providers.OpenStreetMap.Mapnik
+            image source, default: ctx.providers.CartoDB.Positron
         axis : matplotlib.axes.Axes, optional
             axis to use
         kwargs : dict, optional
@@ -1475,9 +1475,14 @@ def _cen_return_imp(imp, freq, imp_th, return_periods):
 
         return imp_fit
 
-    def select(self,
-               event_ids=None, event_names=None, dates=None,
-               coord_exp=None):
+    def select(
+        self,
+        event_ids=None,
+        event_names=None,
+        dates=None,
+        coord_exp=None,
+        reset_frequency=False
+    ):
         """
         Select a subset of events and/or exposure points from the impact.
         If multiple input variables are not None, it returns all the impacts
@@ -1509,6 +1514,9 @@ def select(self,
         coord_exp : np.array, optional
             Selection of exposures coordinates [lat, lon] (in degrees)
             The default is None.
+        reset_frequency : bool, optional
+            Change frequency of events proportional to difference between first and last
+            year (old and new). Assumes annual frequency values. Default: False.
 
         Raises
         ------
@@ -1580,6 +1588,19 @@ def select(self,
             LOGGER.info("The total value cannot be re-computed for a "
                         "subset of exposures and is set to None.")
 
+        # reset frequency if date span has changed (optional):
+        if reset_frequency:
+            if self.frequency_unit not in ['1/year', 'annual', '1/y', '1/a']:
+                LOGGER.warning("Resetting the frequency is based on the calendar year of given"
+                    " dates but the frequency unit here is %s. Consider setting the frequency"
+                    " manually for the selection or changing the frequency unit to %s.",
+                    self.frequency_unit, DEF_FREQ_UNIT)
+            year_span_old = np.abs(dt.datetime.fromordinal(self.date.max()).year -
+                                   dt.datetime.fromordinal(self.date.min()).year) + 1
+            year_span_new = np.abs(dt.datetime.fromordinal(imp.date.max()).year -
+                                   dt.datetime.fromordinal(imp.date.min()).year) + 1
+            imp.frequency = imp.frequency * year_span_old / year_span_new
+
         # cast frequency vector into 2d array for sparse matrix multiplication
         freq_mat = imp.frequency.reshape(len(imp.frequency), 1)
         # .A1 reduce 1d matrix to 1d array

climada/engine/test/test_cost_benefit.py

@@ -35,7 +35,7 @@
 from climada.test import get_test_file
 
 
-HAZ_TEST_MAT = get_test_file('atl_prob_no_name')
+HAZ_TEST_MAT = get_test_file('atl_prob_no_name', file_format='matlab')
 ENT_TEST_MAT = get_test_file('demo_today', file_format='MAT-file')
 
 

climada/engine/test/test_impact.py

@@ -27,6 +27,7 @@
 import h5py
 from pyproj import CRS
 from rasterio.crs import CRS as rCRS
+import datetime as dt
 
 from climada.entity.entity_def import Entity
 from climada.hazard.base import Hazard
@@ -67,6 +68,24 @@ def dummy_impact():
         haz_type="TC",
     )
 
+def dummy_impact_yearly():
+    """Return an impact containing events in multiple years"""
+    imp = dummy_impact()
+
+    years = np.arange(2010,2010+len(imp.date))
+
+    # Edit the date and frequency
+    imp.date = np.array([dt.date(year,1,1).toordinal() for year in years])
+    imp.frequency_unit = "1/year"
+    imp.frequency = np.ones(len(years))/len(years)
+
+    # Calculate the correct expected annual impact
+    freq_mat = imp.frequency.reshape(len(imp.frequency), 1)
+    imp.eai_exp = imp.imp_mat.multiply(freq_mat).sum(axis=0).A1
+    imp.aai_agg = imp.eai_exp.sum()
+
+    return imp
+
 
 class TestImpact(unittest.TestCase):
     """"Test initialization and more"""
@@ -868,6 +887,22 @@ def test_select_imp_map_fail(self):
         with self.assertRaises(ValueError):
             imp.select(event_ids=[0], event_names=[1, 'two'], dates=(0, 2))
 
+    def test_select_reset_frequency(self):
+        """Test that reset_frequency option works correctly"""
+
+        imp = dummy_impact_yearly() # 6 events, 1 per year
+
+        # select first 4 events
+        n_yr = 4
+        sel_imp = imp.select(dates=(imp.date[0],imp.date[n_yr-1]), reset_frequency=True)
+
+        # check frequency-related attributes
+        np.testing.assert_array_equal(sel_imp.frequency, [1/n_yr]*n_yr)
+        self.assertEqual(sel_imp.aai_agg,imp.at_event[0:n_yr].sum()/n_yr)
+        np.testing.assert_array_equal(sel_imp.eai_exp,
+                                      imp.imp_mat[0:n_yr,:].todense().sum(axis=0).A1/n_yr)
+
+
 class TestConvertExp(unittest.TestCase):
     def test__build_exp(self):
         """Test that an impact set can be converted to an exposure"""

climada/engine/unsequa/__init__.py

@@ -22,3 +22,4 @@
 from .calc_base import *
 from .calc_impact import *
 from .calc_cost_benefit import *
+from .calc_delta_climate import *