We propose an objective framework for selecting rainfall hazard mapping models in a region starting from rain
gauge data. Our methodology is based on the evaluation of several
goodness-of-fit scores at regional scale in a cross-validation framework,
allowing us to assess the goodness-of-fit of the rainfall cumulative
distribution functions within the region but with a particular focus on their
tail. Cross-validation is applied both to select the most appropriate
statistical distribution at station locations and to validate the mapping of
these distributions. To illustrate the framework, we consider daily rainfall
in the Ardèche catchment in the south of France, a 2260 km

In recent years, Mediterranean storms involving various spatial and temporal
scales have hit many locations in southern Europe, causing casualties and
damages

A difficulty in producing rainfall return level maps is that knowing the CDF
at any grid point ideally requires observation of rainfall on a grid scale.
However, long-enough gridded data with good-enough quality are often lacking.
Radar and satellite estimations are usually available for about 10 years at
best, and only for selected regions. In addition, rainfall estimation in
complex topography is particularly tricky, e.g., due to the mountain ranges
shielding the radar beam

A second way of mapping rainfall hazard is, rather than interpolating the
point observations, to map the parameters of CDFs fitted on rain gauge
series. In addition to the choice of interpolation models comes now the
choice of the marginal model of rainfall amounts on wet days (referred to as
nonzero rainfalls). The most commonly used CDFs at daily scale include the
exponential, Gamma, lognormal, Pareto, Weibull and Kappa models

Our study aims at filling this gap by proposing an objective cross-validation
framework that is able to validate the full procedure of rainfall hazard
mapping starting from point observations. Our framework features three
characteristics: (i) it selects both the marginal and mapping models, (ii) it
validates the full spectrum of rainfall, from short-
to long-term extrapolated amounts, and (iii) it applies on a regional scale.
The framework is illustrated on the Ardèche catchment in the south of
France. Despite its relatively small size, this test case is particularly
challenging as it shows extraordinarily strong inhomogeneity in rainfall
statistics at a very short distance. Following previous studies in the region

We illustrate our framework on the Ardèche catchment (2260 km

Considered models for the marginal distributions of nonzero rainfall.

Region of analysis. The blue polygon is the Ardèche catchement. The red points show the locations of the stations. The upper triangle is station Antraigues and the lower triangle station Mayres (both lie at about 500 m a.s.l.). The background shows the altitude in gray scale (1 km raster cells). The top left insert shows a map of France with the studied region in red. The black lines are the 400 and 800 m a.s.l. isolines.

The Ardèche catchment is chosen for illustration purposes and because,
despite its relatively small size, it shows strong inhomogeneity in
rainfall distribution. To illustrate this, we show in Fig.

Panel

Let

In the models of Table

Summary of the considered scores for evaluating marginal and mapping models.

In this article, we will consider the supervised case (Eq.

Estimates of return levels are then obtained as follows. The

The goal of this evaluation is to assess which marginal model performs better
at the regional scale, i.e., for a set of

As shown in Table

The NRMSE (normalized root mean squared error) evaluates the reliability of
the fits in the whole observed range of nonzero rainfall, by comparing
observed and predicted return levels of daily rainfall. For a given station

Illustration of the

NRMSE assesses goodness-of fit of the whole distribution in the observed
range. Now let us have a closer look at the tail of the distribution, and in
particular at the maximum over

The

Last but not least, the SPAN criterion evaluates the stability of the return
level estimation, when using data for each of the two subsamples. More
precisely, for a given return period

For the sake of concision, in the rest of this article the scores
MEAN(NRMSE), AREA(

Let

The considered mapping models are listed in Table

Mapping models considered in this study, with involved coordinates. Kriging method provides exact interpolation, unlike the linear regression and thin plate spline.

For the kriging interpolation, cases with and without external drift are
tested

For the linear regression models, we start from regressions of the form

Last but not least, bivariate and trivariate thin plate splines are
considered for

Evaluation is performed in two ways. The first one is a leave-one-out cross-validation scheme aiming to test at regional scale how the interpolated distributions are able to fit the data of the stations when these data are left out for estimating the mapping model. The second step assesses spatial stability by comparing the interpolated distributions obtained at a given station whether the data of this station are used or not in the mapping estimation. In other words, it is a comparison between leave-one-out and leave-zero-out mappings. So the two evaluations differ in that the first one compares an interpolated distribution to data, while the second step compares two interpolated distributions.

First, let us consider a given parameter estimate

Second, we consider the set of all

Regional scores MEAN(TVD

We wish to evaluate and compare the performance of both marginal and mapping
models for estimating rainfall frequency across the region. We consider
models both with and without season/WPs. For the cases involving the use
of WPs, we use the WP classification described in

In cases where subsampling is also undertaken by season, we impose a
restriction of

The full cross-validation procedure for selecting both the marginal and
mapping models is summarized in Fig.

We divide the days of 1948-2013 into two subsamples of equal size,
denoted

For every station

We fit each distribution of Table

We compute the scores of Sect.

We repeat 50 times steps 1–4.

Schematic summary of the full cross-validation procedure for selecting both the marginal and mapping models.

Second we consider the mapping models of Sect.

We consider the estimates

We estimate the mapping models of Sect.

We compute the scores of Sect.

We estimate the mapping models of Sect.

We compute the spatial means of the TVD and KLD scores of Sect.

We repeat steps 1–5 for the estimates

We repeat steps 1–6 for each of the 50 subsamples.

At this step we have selected the best marginal model and the best mapping model (among those tested) for our data.

Finally, we consider the whole sample of data and apply the selected marginal
distribution and mapping model.

We estimate the selected marginal distribution

We estimate the mapping model associated with each marginal parameter, using
all

Scores of cross-validation when

We show in Fig.

Comparing the reliability scores NRMSE,

Obviously, there is a loss of stability when considering seasons and/or WPs
due to the increased number of parameters. However, the score of SPAN

Case of Antraigues when

Scores of cross-validation when

We illustrate the quality of the fit for station Antraigues, located in the
very foothills of the Massif Central slope (see Fig.

Case of Antraigues when

Due to its better fit for the Gamma model (Figs.

Stability score SPAN

Figure

The results of Figs.

Scores of mapping when

Cases of Antraigues (panels

Figure

Back to the kriging methods, the three tested alternatives give very similar
fits, with slightly less stability when considering a drift in station
altitude

Map of the probability of daily rainfall exceeding 1 mm and of the mean of nonzero rainfall in the three WPs of the season-at-risk. The points are colored with respect to the empirical estimates.

We illustrate the results in Fig.

We conclude following the results of Figs.

Figure

Last but not least, Fig.

Map of the probability of daily rainfall exceeding 100 mm. The points show the locations of the stations.

In this article we have presented an objective framework for selecting
rainfall hazard mapping models in a region starting from rain gauge data.
For this we have proposed an objective procedure involving split sampling
cross-validation and using several evaluation scores allowing us to validate
the accuracy of both the bulk and tail of the distribution and the stability
of these estimates when calibration data change. We have applied this procedure to daily rainfall in the
Ardèche catchment in southern France, a particularly challenging test
case subject to strong inhomogeneity of rainfall at a very short distance.
For illustration purposes, we chose to compare several classical marginal
distributions, which are possibly mixed over seasons and weather patterns to
account for the variety of climatological processes triggering precipitation,
and several classical mapping methods. Our results show that for this region,
the best marginal model (among those tested) is a mixture of Gamma
distributions over seasons and weather patterns, and that the best mapping
method (among those tested) is the bivariate thin plate spline. However, the
goal of this paper was neither to be exhaustive nor to find

A possible direction of improvement for the study region regards the choice
of the marginal distribution. Although the Gamma mixture was selected
according to the cross-validation scores, we noted a possible underestimation
of return levels at far extrapolation since the model is unable to produce
heavy tails in the sense of extreme value theory. It could be worth
considering hybrid models based on combining distributions for low and heavy
amounts

Despite the above reservations of prudence, some other results seem to us to
be generalizable, in particular regarding the mapping step. Among these is
the fact that the kriging method gives usually too patchy maps of rainfall
hazard by sticking the observations, unless nugget effects are considered
(which was not the case in this study). Finally, the linear model with
spatial covariates usually fails to map rainfall hazard because it is highly
unlikely to be ruled by simple-enough physics for the parameters to be well
represented as linear functions of the covariates, in particular in such
complex topography

Last for not least, we put this study in a framework of temporal stationarity
and we addressed the question of the spatial nonstationarity of the margins.
Yet several studies showed temporal trend in the rainfall distribution in the
region, particularly since the 1980s and particularly along the Massif
Central slope where daily rainfall is usually more intense

The dataset used in this study has been provided to the authors by EDF and Météo-France for this research. It could be made available to other researchers under a specific research agreement. Requests should be sent to dtg-demande-donnees-hydro@edf.fr.

JB developed the cross-validation framework, wrote the
code in R

The authors declare that they have no conflict of interest.

The authors thank Richard Katz, two anonymous referees and the editor for their valuable suggestions. Edited by: Carlo De Michele Reviewed by: Richard Katz and two anonymous referees