Advances in decision-making under uncertainty: inference, finite-time analysis, and health applications
Report ID: TR-001-17Author: Wang, Yingfei
Date: 2017-04-24
Pages: 228
Download Formats: |PDF|
Abstract:
This thesis considers the problem of sequentially making decisions under uncertainty, exploring the ways where ecient information collection in uences and improves decision-making strategies. Most previous optimal learning approaches are restricted to fully sequential settings with Gaussian noise models where exact analytic solutions can be easily obtained. In this thesis, we bridge the gap between statistics, machine learning and optimal learning by providing a comprehensive set of techniques that span from designing appropriate stochastic models to describing the uncertain environment, to proposing novel statistical models and inferences, to nite-time and asymptotic guarantees, with an emphasis on how ecient information collection can expand access, decrease costs and improve quality in health care. Speci cally, we provide the rst nite-time bound for the knowledge gradient policy. Since there are many situations where the outcomes are dichotomous, we consider the problem of sequentially making decisions that are rewarded by \successes" and \failures". The binary outcome can be predicted through an unknown relationship that depends on partially controllable attributes of each instance. With the adaptation of an online Bayesian linear classi er, we design a knowledge gradient (KG) policy to guide the experiment. Motivated by personalized medicine where a treatment regime is a function that maps individual patient information to a recommended treatment, hence explicitly incorporating the heterogeneity in need for treatment across individuals, we further extend our knowledge gradient policy to a Bayesian contextual bandits setting. Since the sparsity and the relatively small number of patients make learning more dicult, we design an ensemble optimal learning method, in which multiple models are strategically generated and combined to minimize the incorrect selection of a particularly poorly performing statistical model. Driven by numerous needs among materials science society, we developed a KG policy for sequential experiments when experiments can be conducted in parallel and/or iii there are multiple tunable parameters which are decided at di erent stages in the process. Finally, we present a new Modular, Optimal Learning Testing Environment (MOLTE) as a public-domain test environment to facilitate the process of more comprehensive comparisons, on a broader set of test problems and a broader set of policies.