A posteriori analyses—advantages and pitfalls of pattern recognition techniques

Abstract

In this chapter, we deal with a posterior analysis of supervised and unsupervised learning techniques. Concerning supervised learning, we discuss methods of cross-validation and assessment of uncertainty of tests by means of the “Receiver Operation Curve” and the “Kappa-Statistics.” We show the importance of appropriate target information. Furthermore, features are critical; when they are not properly chosen, they fail to describe objects in a unique way. A critical attitude is mandatory to validate the success of an application. A high score of success does not automatically mean that a method is truly effective. At the same time, users should not despair when the desired success is not achieved. A posteriori analysis on the reasons for an apparent failure may provide useful insights into the problem. Targets may not be appropriately defined, features can be inadequate, etc. Problems can be often fixed by adjusting a few choices; sometimes a change of strategy may be necessary to improve results. In unsupervised learning, we ask whether the structures revealed in the data are meaningful. Cluster analysis offers rules giving formal answers to this question; however, such rules are not generally applicable. In some cases, a heuristic approach may be necessary.