Bayesian Aggregation for Hierarchical Classfication

Abstract:

Large numbers of overlapping classes are found to be organized in hierarchies in many domains. In multi-label classification over such a hierarchy, members of a class must also belong to all of its parents.

Training an independent classifier for each class is a common approach, but this may yield labels for a given example that collectively violate this constraint. We propose a principled method of resolving such inconsistencies to increase accuracy over all classes. Our approach is to view the hierarchy as a graphical model, and then to employ Bayesian inference to infer the most likely set of hierarchically consistent class labels from independent base classifier predictions. This method of Hierarchical Bayesian Aggregation (HBA) can work with any type of base classification algorithm. Experiments on synthetic data, as well as real data sets from bioinformatics, graphics, and music domains, illustrate the behavior of HBA under a range of conditions, and reliably demonstrates improvements in accuracy over all levels of a hierarchy.