Quantifying epistemic uncertainty in volcanology using structured expert judgment methods

Abstract

Uncertainty plays a main role in quantitative volcanology and even more in hazard and risk assessments. From the point of view of the scientist, the volcano is a complex stochastic system that is investigated with incomplete and uncertain information. As a consequence the forecast of its behavior cannot be easily constrained by using simple probability models. A way for producing more robust estimates is adopting probability models which are doubly stochastic, in the sense that the physical distribution of a considered outcome (such as for instance the location of the next eruptive vent, or the volume of the next eruption) is represented using ill-constrained parameters: these parameters themselves can be treated as random variables. This allows potential uncertainties affecting the past record to be included in analysis and, most importantly, different conceptual models with various degrees of belief to be considered. The problem of estimating uncertain parameters or enumerating the evidential weight to be accorded to event probabilities, together with the uncertainties affecting them, arises commonly in volcanology. The concept of structured expert judgment (SEJ) is therefore useful because it involves the adoption of a formalized technique, or techniques, to be used for pooling judgments of a group of experts in order to inform decisions, forecasts or predictions. The understanding, improvement and publicizing of these techniques is the main goal of the EU-funded COST Action “Expert Judgement Network: bridging the gap between scientific uncertainty and evidence-based decision making”. In particular, a performance-based procedure includes an empirical step of expert ranking. Based on the answers given by the experts to test questions with known answers (called seed items), different weights are computed and attributed to individual experts concerning their ability to gauge uncertainties in terms of statistical accuracy and informativeness. These weights are then used to pool their judgments about target questions of specific interest. In practice, the goals of an elicitation can be twofold: to give reliable pointwise central value estimates of variables of interest and to assess the level of uncertainty of such estimates. Here, we mean an expert’s intrinsic statistical uncertainty, not the subjective precision of the estimate declared by a single expert who, by character, may be overconfident, proficient, or overly cautious in making judgments. It is possible to measure the performance of a weighting method by means of various indices that reflect different features: calibration, informativeness, and expected accuracy, the meaning of each of these scores will be explained. After a short presentation (about 20 min) for introducing some SEJ methods, a short hands-on session (about 30 min) will follow with realistic elicitation data for a volcanology decision problem. The application will be to Campi Flegrei volcano which is an example of an active, densely populated, caldera with very high risks associated with the occurrence of explosive eruptions. The available data from a published study aimed at developing vent opening maps and volcanic hazard assessments adopting SEJ methods represent a good case study for a training exercise (see Bevilacqua et al., 2015, Neri et al. 2015). The EXCALIBUR software and some simple Monte Carlo examples for elaborating data with R-Statistics software will be presented. Time will be made also for discussion and questions.