You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
title={Predictive Analytics and Machine Learning – Practical Applications for Actuarial Modeling (Nested Stochastic)},
3
+
author={Larochelle, Jean-Philippe and Carlson, Peter and Carrier Cote, Vincent and Lu, Ying and Shapiro, Noah and Tam, Alex and Thusu, Viresh and Zhang, Ally},
title: 'Actuarial Open Source Benchmarks: Performance testing actuarial models on CPU and GPU'
3
+
tags:
4
+
- Python
5
+
- Julia
6
+
- actuarial science
7
+
- GPU
8
+
- performance testing
9
+
authors:
10
+
- name: Matthew Caseres
11
+
orcid: 0000-0001-5819-001X
12
+
affiliation: 1
13
+
affiliations:
14
+
- name: Georgia Institute of Technology, USA
15
+
index: 1
16
+
date: 27 May 2024
17
+
bibliography: paper.bib
18
+
---
19
+
20
+
# Summary
21
+
22
+
Actuaries are employed by insurance companies to manage risk related to uncertain future cashflows and play a large role in regulatory compliance for insurance companies.
23
+
Actuarial models are used to forecast future economic scenarios and cash flows related to insurance contracts. The benchmarks provided in the Actuarial Open Source GitHub organization test the consistency and performance of existing open source actuarial models and provides novel approaches to achieve greater performance and scale on certain tasks.
24
+
25
+
# Statement of need
26
+
27
+
The actuarial modeling software market is dominated by large vendors [@Halloran:2021; @Kinrade:2024]. Vendor software can restrict the sharing of actuarial software [@Larochelle:2023], demonstrating the value of open source solutions in enabling reproducible research.
28
+
29
+
A 2024 PwC publication [@Kinrade:2024] speculates that graphics processing units (GPUs) may become the norm for actuarial calculation engines. Despite discussion of actuarial models running on GPUs in several publications [@Kim:2018; @Hamamura:2022a; @Robidoux:2016], there are no reproducible benchmarks for GPU-based actuarial applications.
30
+
31
+
On the CPU the ecosystem is more mature and multiple open source packages might implement a particular calculation. In this case we can compare the execution time of the packages on a specific task, as well as validate that the packages can produce the same results.
32
+
33
+
# Benchmarking infrastructure
34
+
35
+
Calculations performed on CPU are run with GitHub actions on GitHub-hosted runners. We found that the execution times reported from Github hosted runners are generally consistent between runs and this has been independently verified by Milliman [@Milliman:2024].
36
+
37
+
Calculations performed on GPU are published to DockerHub using GitHub actions and tested on GPUs in the cloud.
38
+
39
+
# Benchmark categories
40
+
41
+
## LifeLib life insurance cash flow models
42
+
43
+
LifeLib [@Hamamura:2018] is an open source library that contains reference implementations for various life insurance product cash flow models. We provide implementations in Python and Julia to assess the performance and coding style of the following strategies:
44
+
45
+
* Recursion with memoization
46
+
* Recursion with memoization and cache eviction to reduce memory consumption [@Hamamura:2022b]
47
+
* Array based models using broadcasting and avoiding iteration
48
+
* Array based models using iteration that are optimized to reduce memory consumption [@Belmant:2022]
49
+
50
+
## Experience studies
51
+
52
+
Some actuarial techniques for calculating mortality rates involve partitioning date ranges [@Atkinson:2016]. These date partitioning algorithms are implemented by the actxps R package [@Heaphy:2024] and the ExperienceAnalysis.jl [@Loudenback:2020] Julia package . The benchmarking process identified a number of inconsistencies which were raised as issues on GitHub and quickly resolved by the maintainers of the packages.
53
+
54
+
## Mortality tables software
55
+
56
+
The Society of Actuaries provides a number of mortality tables in an XML format [@Strommen:2013]. These tables have been wrapped into packages for convenient access with the pymort [@Caseres:2021] Python package and the MortalityTables.jl [@Loudenback:2018] Julia package. There are many XML files and each XML file contains many numbers. It is not practical to verify that each package correctly parses every value from every XML file. A hash was computed using each value from 10 files verifying that the values are consistent between the two packages for these files.
57
+
58
+
# Conclusion
59
+
60
+
The benchmarks provided by the Actuarial Open Source organization on GitHub intend to assist in testing the performance and accuracy of open source actuarial software. This is accomplished by selecting a specific actuarial calculation and comparing the results and execution times of various approaches.
61
+
62
+
Our current benchmarks are chosen for simplicity so that the barrier to entry remains low. We have not implemented complex calculations like nested stochastic variable annuity models which could benefit from concrete benchmarks. More complex benchmarks that represent the computational challenges facing the insurance industry will require broader community engagement.
63
+
64
+
# Acknowledgements
65
+
66
+
I thank Alec Loudenback for contributing Julia benchmarks, providing feedback through GitHub issues, and creating many Julia packages through the JuliaActuary GitHub organization. I thank Fumito Hamamura for creating LifeLib, the Actuarial Open Source LinkedIn community page, modelx, and the insights found on the modelx blog. Many thanks Cédric Belmant for providing a Julia implementation to the universal life benchmark which is incredibly hard to beat and to Matt Heaphy for his implementations of experience studies software in R and Python.
0 commit comments