Triple Collocation to Assess Classification Accuracy Without a Ground Truth in Case of Earthquake Damage Assessment

Abstract

The assessment of satellite image classifications is usually carried out using a test sample assumed as the ground truth, from which a confusion matrix is derived. There are cases where the reference data, even those coming from a ground survey, are affected by errors and do not represent a reliable truth. In the field of geophysical parameter retrieval, the triple collocation (TC) technique is applied for validating remotely sensed products when the source of test data (e.g., ground data) does not represent a reliable reference. TC is able to retrieve the error variances of three systems observing the same target parameter, assuming that their errors are independent. In this paper, we exploit the same idea to test the classification accuracy in cases where the ground truth is not available. We extend the TC approach to the classification problem for a general number of classes, but we solve it numerically for a two-class problem (i.e., collapsed and noncollapsed buildings). The specific case refers to the detection of L'Aquila 2009 earthquake damage from very high-resolution optical data. The image classification, performed by exploiting an object-based analysis, is compared with those from two different ground surveys carried out after the earthquake by different teams and with different purposes. This paper demonstrates the power of the TC approach for assessing the classification accuracy with no reliable ground truth available, and provides an insight into the problem of assessing damage, from satellite and on ground, in a very critical and unsafe situation, like the one occurring after an earthquake. Moreover, it was found that the remotely sensed product can have an order of accuracy comparable to that of at least one of the ground surveys.