Weighing Evidence without using a Gold Standard

Phil Long

Center for Computational Learning Systems,

Columbia University

We have developed a method for learning a scoring function to weigh evidence of different types. The algorithm does not need gold standard designations; it evaluates each source of evidence by the extent to which other sources tend to support it.  The details are guided by a probabilistic formulation of the problem, building on previous theoretical work.

While abstract, this work was driven by applications to biology.  It is often useful to collect together large numbers of biological propositions, together with the evidence supporting them, into databases to be used in other analyses.  Biological propositions can often be supported in a variety of different ways.  Sometimes, it is not clear how best to weigh the different kinds of evidence to make final judgments about which of the many candidate propositions are true.  If gold-standard designations of the truth or falsehood of a representative collection of propositions are available, then supervised learning methods can be used.  However, sometimes, usable gold-standard designations are not available.

Our method provides more accurate predictions when predicting protein-protein interactions in yeast than algorithms based on k-means and a spectral analysis.  It also performs well on synthetic data with similar characteristics as some biological problems.