%%%% % % Nature Scientific Reports Template % %%%%%%%%%% \documentclass[fleqn,10pt]% %{article}% {wlscirep} \usepackage[utf8]{inputenc} \usepackage[T1]{fontenc} %\usepackage{graphicx} %\usepackage{url} %this package should fix any errors with URLs in refs. \usepackage{lineno} %\usepackage[inline]{trackchanges} %for better track changes. finalnew option will compile document with changes incorporated. %\usepackage{soul} \linenumbers %%%%%%% % As of 2018 we recommend use of the TrackChanges package to mark revisions. % The trackchanges package adds five new LaTeX commands: % % \note[editor]{The note} % \annote[editor]{Text to annotate}{The note} % \add[editor]{Text to add} % \remove[editor]{Text to remove} % \change[editor]{Text to remove}{Text to add} % % complete documentation is here: http://trackchanges.sourceforge.net/ %%%%%%% \newcommand{\fig}{Figure} \newcommand{\Fig}{Figure} \newcommand{\tab}{Table} \newcommand{\Tab}{Table} \newcommand{\sect}{section} \newcommand{\oneFigureWidth}{0.7\textwidth} %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % % TITLE % %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% \title{Neural network based estimates of the climate impact on mortality in Germany % - application to storyline climate simulations} %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % % AUTHORS AND AFFILIATIONS % %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% \author[1,*]{R. Schachtschneider} \author[1]{J. Saynisch-Wagner} \author[2]{A. S{\'a}nchez-Benítez} \author[1,3]{M. Thomas} %------------------------------------------------------------------------------- \affil[1]{Helmholtz Centre Potsdam GFZ German Research Centre for % Geosciences, Telegrafenberg, 14473 Potsdam, Germany} \affil[2]{Alfred Wegener Institute, Helmholtz Centre for Polar and Marine % Research, Am Handelshafen 12, 27570 Bremerhaven, Germany} \affil[3]{% %Freie Universit\"{a}t Berlin, % Free University Berlin, % Kaiserswerther Str. 16 - 18, % 14195 Berlin, Germany} % Corresponding author mailing address and e-mail address: \affil[*]{reyko.schachtschneider@gfz-potsdam.de} \keywords{Climate change, machine learning, mortality prediction, human health, % storyline simulations} %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % % ABSTRACT (and PLAIN LANGUAGE SUMMARY) % %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %\input{abstract} \begin{abstract} The aim of this work is the prediction of heat-related mortality for Germany under future, i.e. hotter, climate conditions. The prediction is made based on 2m temperature data from climate storyline simulations using machine learning techniques. We use an echo state network for linking the outputs of storyline climate simulations to the target data. The target data are all-cause mortality rates of Germany for all ages. The network is trained with present day climate model outputs. Model outputs of future, i.e. 2K and 4K warmer, storylines are used to predict mortality rates under such climatic conditions. We find that we can train an echo state network with recent temperature data and mortality and make plausible predictions about expected developments of mortality in Germany based on future climate storylines. The trained network can successfully predict mortality rates for future climate conditions. We find increased mortality during the summer months which is attributed to the presence of more severe heat waves. The mortality decrease found during winter can be explained milder winters leading to fewer deaths caused by respiratory diseases. However, mortality in winter is largely influenced by other factors such as influenza waves or vaccination rate and explainability due to temperature is limited. \end{abstract} %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % % Main Text % %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% \begin{document} %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % % BODY TEXT % %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% \flushbottom \maketitle \thispagestyle{empty} %%% Suggested section heads: % \section{Introduction} % %\input{intro} \section{Introduction} \label{sec:intro} %------------------------------------------------------------------------------- % State of the art: ML and application examples Machine learning (ML) has become a helpful tool in many areas of science and medicine \cite{Deo2015machine,Rajkomar2019machine}. It comprises algorithms for revealing non-linear relationships between input and output variables and can be helpful in many classification tasks or regression problems. In medicine, ML can teach researchers new ways of studying diseases, producing medicines or treating patients and can be applied for early disease detection where severe conditions are likely to manifest at a later time \cite{May2021eight,Rajkomar2019machine}. \cite{Than2019machine} used a gradient boosting model to predict the likelihood of acute myocardial infarction for patients. \cite{Esteva2017dermatologist} trained a deep neural network for classification of skin lesions into skin cancer risk categories. Mortality prediction has been conducted in various fields of medicine and for different diseases such as Chronic Obstructive Pulmonary Disease \cite{Moll2020machine}, COVID-19 \cite{Gao2020machine}, neonatal mortality \cite{Mangold2021machine}, or general mortality risk in populations with coronary artery disease \cite{Motwani2017machine} or general elderly populations \cite{Rose2013mortality}. \cite{Taylor2016prediction} predict mortality of hospitalized patients with sepsis in a data-driven ML-approach. The impact of climate change on mortality due to various diseases have been studied with the help of ML, e.g. for Malaria \cite{Nkiruka2021prediction}, myocardial infarctions\cite{Marien2022machine}, or chronic obstructive pulmonary disease \cite{Moll2020machine}. Impacts of climate change and increasing numbers of death from heat disorder have been studied, e.g. by \cite{McGeehin2001potential,Kim2016projection,Hajat2023current,Huang2011projecting}. \cite{Lee2023heat} investigated the influence of heat exposure on mental health and connected implications following climate change. This includes mortality due to suicides, mental disorders, and violence among others. \cite{Hirano2021machine} developed a ML-based model for mortality prediction for heat-related illnesses on the base of data from %2393 hospitalized patients in Japan. \cite{Kim2022explainable} used a random forest model for heat-related mortality prediction in a detailed area within a city with various climatic, demographic and socio-economic sectors. \cite{Winklmayr2022heat} employ a generalized additive model for the estimation of heat-related deaths in Germany for the period 1992--2021. They can characterize long-term trends and quantify the effect of heat on mortality over the years. \cite{Mistry2022comparison} compare results of mortality models based on temperature measurement data at a number of weather stations to models based on global reanalyses of weather observations and climate models by the European Centre for Medium-Range Weather Forecasts (ECMWF). They find that reanalysis data are a valid alternative source in temperature-related health risk studies. In this study we predict the mortality rate in Germany for two different climate storylines computed in the project SCENIC using an echo state network (ESN). We show that the network can be trained on five years of monthly maximum temperature input data and monthly mortality rates for Germany as target data. With the trained network we can reliably predict mortality rates in Germany for the two given future climate storylines. % Methods %\input{method} %=============================================================================== \section{Method} \label{sec:method} %------------------------------------------------------------------------------- \subsection{Background} \label{sec:method:bg} % General machine learning / artificial neural networks Artificial neural networks (ANN) are inspired by biological examples \cite{Lee2018machine}. Neurons in the brain are interconnected and capable of fulfilling enormous cognitive tasks with utmost precision. An ANN consists of a large number of artificial neurons that mimic the behavior of connected neurons in the brain. The artificial neurons have an activation threshold above which they pass on information and weights that encode the importance of the neuron connection. Several neurons are organized in a layer and many layers make up a neural network. The input layers receives information from the outside world, a number of hidden layers further processes the information from the input layer, and the output layer transforms the information from the network into a result. The shape of the result depends on the purpose the network is designed for. There are numerous kinds of layered networks for purposes of , e.g., classification, regression, clustering or object detection. % Training of ANNs In the training phase the weights of an ANN are adjusted. In supervised learning this is achieved by providing input samples with the corresponding known output. Through iterative back-propagation techniques the weights of the ANN nodes are adjusted to minimize the misfit between the network output and the known result. The challenge is to avoid over-fitting and obtain an ANN that performs well on unknown data. Unsupervised learning uses only input data to learn the distribution of the input variables. It is useful for clustering, compression, or feature extraction \cite{Lee2018machine}. % Recurrent neural networks A recurrent neural network (RNN) is a network with cyclic connections in which the neurons can send feedback signals to each other \cite{Grossberg2013recurrent}. They are more powerful than acyclic feed forward neural networks (FNNs) in that they can create and process memories of arbitrary sequences of input patterns \cite{Schmidhuber2015deep}. Therefore, while FNNs represent functions, RNNs represent dynamical systems which may develop a self-sustained temporal activation even in the absence of input\cite{Lukovsevivcius2009reservoir}. Supervised Long Short-Term Memory (LSTM) RNNs can discover and memorize events that happened thousands of time steps ago \cite{Schmidhuber2015deep}. Training of RNNs, however, is challenging. The well-known feed-forward back-propagation algorithm cannot be directly used because of the cyclic nature of the reservoir. One simple solution to overcome this difficulty is to neglect all indirect pathway (i.e. such containing cycles). However, this is a rather coarse approximation of the optimization problem \cite{Doya2003recurrent}. There exist two basic ways for calculating the exact gradient fo the output with respect to the weights: forward and backward methods \cite{Doya2003recurrent}. The forward method solves a linear dynamical equation system for the effect of a small change in a weight on the network state trajectory. While this can be calculated concurrently with the network dynamics the computational costs are quite high. In the backward method the causes of the output error are computed backward in time. In case of discrete time steps this can be achieved by setting up a multi-layer feed-forward network for unrolling the multi-step evolution of the network. More comprehensive overviews over RNNs and training algorithms are given by \cite{Tyagi2022recurrent, Doya2003recurrent, Jaeger2002tutorial, Lukovsevivcius2009reservoir}. % Echo state networks \subsection{The echo state network} \label{sec:esn} The ESN is a special form of a recurrent neural network that was introduced independently by \cite{Jaeger2001echo} and \cite{Maass2002real}. Originally, it incorporates an algorithm for supervised learning in which only the output layer is modified \cite{Doya2003recurrent}. A schematic of an ESN is shown in \fig~\ref{fig:esn_scheme}. %\textcolor{red}{ToDo: Elaborate!} \begin{figure} \centering \includegraphics[width=\oneFigureWidth]{Images/figure1} \caption{Schematic of an echo state network. The input layer is connected to a reservoir. In the reservoir there are connections between some of the nodes, including recurrent connections. The output layer is connected to the reservoir and to the input layer. After adaption of the neuron activations in the reservoir, the output weights are determined by a regression that minimizes the misfit, $E$, between prediction and target.} \label{fig:esn_scheme} \end{figure} In our network, the input weights are initialized randomly from a uniform distribution over the interval $[-0.5, 0.5]$. The reservoir weights are initialized by the same procedure and in addition a fraction $(1-c)$ of the weights, where $c \in [0, 1]$ is the connectivity of the reservoir, is set to zero. During the training process, the reservoir weights remain unchanged. Only the neuron activations are updated based on training input. The neuron activation decides whether a neuron is important and will be activated or not. The activation values are updated using % \begin{equation} \label{eq:neur_act} \vec{x}_{n} = (1-\alpha) \vec{x}_{n-1} + \alpha \tanh(\mat{W}_{\mathrm{in}} + \mat{W}\vec{x}_{n-1}) \end{equation} % where $\vec{x}_{n}$ are the neuron activations at step $n$, $\mat{W}_{\mathrm{in}}$ is the input weight matrix, $\mat{W}$ is the recurrent weight matrix of the reservoir, and $\alpha \in (0, 1]$ is the leakage parameter. The leakage parameter determines the proportion of the neuron activation update due to new input data at step $n$. Finally, the output weight matrix is determined by a ridge regression that minimizes the misfit between outputs and targets in a least squares sense. The ESN is described in more detail by \cite{Lukovsevicius2012practical}. The ESN parameters used in this study are given in Table~\ref{tab:esn_params}. \begin{table} \centering \caption{Parameters of the echo state network used in this study. $\alpha$ is the leakage parameter, $r_{\text{s}}$ is the spectral radius of the reservoir weights matrix, $c$ is the connectivity of the reservoir, and $\lambda$ is the regression coefficient in the ridge regression of the output weights matrix.} \begin{tabular}{l|lllll} Parameter & Reservoir size & $\alpha$ & $r_{\text{s}}$ & $c$ & $\lambda$\\ \hline Value & 9,000 & 0.5 & 1.25 & 0.5 & 1e-8 \end{tabular} \label{tab:esn_params} \end{table} The trained network can then be used to either forecast new values of the time series of input data from the known inputs in the past or to compute the response to new inputs. Here, we use the latter functionality since the goal is to predict socio-economic parameters for new given inputs (i.e. in warmer world scenarios). The output is then determined by % \begin{equation} \label{eq:esn_output} \vec{y}_{n} = \mat{W}_{\mathrm{out}}[1;\vec{u}_{n};\vec{x}_{n}] \end{equation} % where $\mat{W}_{\mathrm{out}}$ is the matrix of output weights, $[\cdot;\cdot;\cdot]$ denotes the vertical concatenation of vectors, $\vec{u}_{n}$ is the input vector, and $\vec{x}_{n}$ is the vector of neuron activations. Thus, the output at a certain time step is determined by both, the state of the reservoir and directly by the input data at that time. As indicated in \fig~\ref{fig:esn_scheme}, an error measure is calculated from the predictions and the target values. We choose the root-mean-square (RMS) error. As the network is initialized with random values, the network state after training can be different for each realization. Therefore, we compute an ensemble of trained ESNs and predictions and calculate the arithmetic mean from the ensemble of predictions. The ensemble size is 25. The statistics of the ensemble are used to estimate the robustness of the approach. % Data %\input{data} %=============================================================================== \section{Data} \label{sec:data} As input data for training the ESN we used outputs of global climate simulations. The used simulations are so-called climate storyline simulations. They recreate extreme weather events in climate models by keeping some boundary conditions (in this case winds in the higher troposphere) close to states of the atmosphere observed in the past during those events. This process is called nudging. When different climate conditions (e.g. elevated CO2 concentrations) are applied to the climate model with the same nudging, it is possible to study how the extreme event would enfold under those new conditions. The storyline simulations are computed using a model developed at the Alfred Wegener Institute, Helmholtz Centre for Polar and Marine Research (AWI). The model is called AWI CM1 and a description can be found in %the paper by \cite{Sidorenko2015towards}. For the current time scenario the tropospheric winds in the climate model are nudged to ECMWF weather reanalysis data (ERA5). The two additional future climate conditions considered are elevated concentrations of CO2 that result in increased global mean temperatures of +2K and +4K scenarios w.r.t. pre-industrial times, respectively. The method for computing climate storylines is described in detail in %by \cite{Sanchez-Benitez2022thejuly}. The climate storyline simulations that constitute our training data are available in a period from 2015 to 2021. The warmer world storylines that are used for the mortality rate predictions are available in a period from 2017 to 2021. In order to exclude excess mortality due to the COVID-19 pandemic we restrict the training and the prediction to before 2020. From the AWI CM1 model we used hourly 2m temperature fields from which we compute monthly maximum values. The data have a $1^{\circ}\times 1^{\circ}$ spatial resolution. We consider a region within the longitudinal range from $10^{\circ}$W to $40^{\circ}$E and the latitudinal range from $30^{\circ}$N to $65^{\circ}$N. This covers a region large enough to include all influences on people in Germany but reduces the computational effort compared to using the global data set. The temperatures are normalized such that the interval [-13$^{\circ}$C, 47$^{\circ}$C] maps to [0,1]. The temperature data used as inputs in this study have been published by \cite{Sanchez-Benitez2023global}. An example of a temperature field before normalization used as input is given in \fig~\ref{fig:temp_in}. \begin{figure} \centering \includegraphics[width=\oneFigureWidth]{Images/figure2.png} \caption{Example of temperature input data. Shown are the monthly maximal values at each grid point for July 2017. The input data consist of one 2D distribution of the 2m temperature for each input month. For the analysis, temperatures are normalized such that all values lie in the interval [0,1] over the whole investigation period. The red diamonds denote the locations of temperature time series shown in \fig~\ref{fig:temp_vs_time}.} \label{fig:temp_in} \end{figure} \Fig~\ref{fig:temp_vs_time} shows examples of temperature time series at four locations in Germany. Even though the global mean temperature is only 2K or 4K higher than in pre-industrial times, the local maximum temperature difference can be much higher. \begin{figure} \centering \includegraphics[width=\oneFigureWidth]{Images/figure3.png} \caption{Examples of monthly maximum temperature development over time for all considered scenarios. The locations of the profiles are shown in \fig~\ref{fig:temp_in}. The temperature differences between current and future scenarios can be much larger than the average difference.} \label{fig:temp_vs_time} \end{figure} The target data are monthly mortality values published by the German Federal Statistical Office. They are available under the code \emph{12613-03-01-4-B} at https://www.regionalstatistik.de \cite{regionalstatistik}. The data are all-cause mortality values over all ages. The mortality was scaled by the population of each federal state to yield the mortality per 100,000 inhabitants. The population statistics are available at the same website under the code \emph{12411-05-01-4-B}. The population data have annual values so the same value was used for each month of the year to scale the monthly mortality. % Results %\input{results} %=============================================================================== \section{Results} \label{sec:results} With the described ESN we are able to predict mortality rates in Germany from monthly maps of maximum 2m temperatures in Europe. \Fig~\ref{fig:esn_training} shows the time series of the target values used for training the network (left part) and the subsequent test predictions (right part, red circles). In the test months, the prediction of mortality is very accurate given the short training period. The RMS error is 1.7, i.e. $\approx$~1.9\% in the test period. \begin{figure} \centering \includegraphics[width=\oneFigureWidth]{Images/figure4.png} \caption{Training of the echo state network. This panel shows the five years of target data used for training and testing. The test period is seven months from June to December 2019 in the right part of the panel. The RMS error in the test period is 1.7, i.e. $\approx$~1.9\%.} \label{fig:esn_training} \end{figure} In \fig~\ref{fig:mean_pred}, we show the mortality rate prediction for the two climate storylines investigated in this study. We calculate ensembles of trained ESNs and predictions to test the robustness of the approach since the initialization of the ESN's input layer and reservoir are performed with random numbers. The ensemble size is 25. The blue and red shaded areas indicate the ensemble spread, i.e. the mean $\pm$ two standard deviation (SD), while the circles represent the mean values for each month. The SD for the +2K runs ranges between 0.42 and 1.0, for the +4K runs between 0.64 and 1.32, respectively. This is in the range of 0.5 to 1\% of the target values, showing that the results are robust. \begin{figure} \centering \includegraphics[width=\oneFigureWidth]% {Images/figure5.png} \caption{Mortality rate predictions for warmer climate storyline scenarios. Here, we show the prediction of the mortality rate for the warmer world scenarios: +2K in red and +4K in blue, along with the target values of the recent climate scenario on which the network was trained (black). The shaded areas represent one standard deviation from the ensemble mean values. The periods highlighted with orange background are the summer months (June--August). The warmer world scenarios are available only from 2017.} \label{fig:mean_pred} \end{figure} The difference between the predictions for the +2K and +4K scenarios and the target values representing the current mortality rates is shown in \fig~\ref{fig:diff_pred}. During the summer months, the values are elevated while in winter the values are lower than in the reference period. The increase in mortality rate in the summer months is negligible for +2K storyline and 4-5 per 100,000 people for the +4K storyline. The latter corresponds to about 3,200--4,000 additional fatalities in Germany per month. \begin{figure} \centering \includegraphics[width=\oneFigureWidth]% {Images/figure6.png} \caption{Differences between predictions and current mortality rates. Shown are the +2K scenario (red) and +4K scenario (blue). The intervals with the orange highlighted background are the summer months (June--August).} \label{fig:diff_pred} \end{figure} % Discussion %\input{disc} %------------------------------------------------------------------------------- \section{Discussion} \label{sec:disc} The predictions of mortality rates in warmer world scenarios show elevated values in summer months and decreased values in winter months. We interpret those results as higher mortality due to higher heat stress in summer. High day temperatures in combination with high temperatures at night are a major cause of fatality\cite{Watts2019the2019}. A decrease of mortality in winter time is plausible since warmer winters may lead to less severe influenza waves. However, this is hard to quantify since the impact of a influenza wave also depends on many other factors such as vaccination rate, effectiveness of the vaccination, etc. Therefore, this study's analysis is focused on the summer months and heat-related mortality. Even with heat related mortality there are factors other than the ambient temperature. After the 2003 heatwave in Europe with thousands of fatalities among elderly people as reported by \cite{DeBono2004Impacts} preventive measures have been taken in many European countries. This has lead to a reduced number of fatalities in the severe 2006 heatwave in comparison to the previous heatwaves without those measures\cite{Fouillet2008has}. However, \cite{Winklmayr2022heat} show that despite the evident adaption to heat, high temperature periods in summer still pose a significant thread to human health in Germany. % Justification of used data set In order to reduce computational costs we used only monthly maximum temperatures as input data. However, the results show that our network is able to model the connection between temperature inputs and mortality on a very accurate level. The spatial extend of input data is significantly larger than the investigated domain. Spatially extended high temperature distributions allow the conclusion that the temporal extend at one location in or near the center is also high since the weather conditions would not change very quickly if high temperatures are observed all over the domain. Therefore, the use of monthly maximum temperatures is a reasonable choice. Taking those facts into consideration, this analysis gives a hint as to what the impact of future climate change concerning mortality could be. Society and politics have to take the appropriate measures not only to prevent the warming as far as possible but also to reduce the risk of death under such climate conditions should they occur. One drawback of ESN is that techniques that attribute the outcome to specific (parts of) the input data such as layer-wise relevance propagation cannot be used due to the cyclic nature of the network. However, the advantage of yielding robust results with short input data time series outweighs that disadvantage of lack of interpretability. %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % % DATA SECTION and ACKNOWLEDGMENTS % %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% \section*{Availability of materials and data} The target mortality and population data used in this study are publicly available on the website of the German statistical service (see reference numbers in \sect~\ref{sec:data} of the main text). The climate model outputs used as input data for the network training and mortality prediction are available at https://zenodo.org/record/8014199. The R code used in this project is available from the authors upon request. Please contact R. Schachtschneider (reyko.schachtschneider@gfz-potsdam.de) for further information. \section*{Acknowledgments} This work was supported by funding from the Helmholtz Research Field Earth \& Environment for the Innovation Pool Project SCENIC. Climate storyline simulations were carried out at the supercomputing facilities of the German Climate Computing Center (DKRZ). %The storyline simulations %from which our input data were taken, were prepared by Antonio S{\'a}nchez-Benítez. \section*{Author contributions statement} R.S. conducted the research, model development, interpretation and wrote the first manuscript draft. J.S.-W. contributed to the methodology, financing of the project and project supervision. A.S.-B. conducted the climate simulations and provided the input data. M.T. contributed to project supervision. All authors were involved in revising the manuscript. \section*{Competing interests} The authors declare no competing interests. %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % % REFERENCES and BIBLIOGRAPHY % %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% \bibliography{scenic} \end{document}