Quick links

Advances in decision-making under uncertainty: inference, finite-time analysis, and health applications

Report ID:
April 24, 2017
Download Formats:


This thesis considers the problem of sequentially making decisions under uncertainty,
exploring the ways where ecient information collection in
uences and improves
decision-making strategies. Most previous optimal learning approaches are restricted
to fully sequential settings with Gaussian noise models where exact analytic solutions
can be easily obtained. In this thesis, we bridge the gap between statistics, machine
learning and optimal learning by providing a comprehensive set of techniques that
span from designing appropriate stochastic models to describing the uncertain environment,
to proposing novel statistical models and inferences, to nite-time and
asymptotic guarantees, with an emphasis on how ecient information collection can
expand access, decrease costs and improve quality in health care.
Specically, we provide the rst nite-time bound for the knowledge gradient
policy. Since there are many situations where the outcomes are dichotomous, we
consider the problem of sequentially making decisions that are rewarded by \successes"
and \failures". The binary outcome can be predicted through an unknown
relationship that depends on partially controllable attributes of each instance. With
the adaptation of an online Bayesian linear classier, we design a knowledge gradient
(KG) policy to guide the experiment. Motivated by personalized medicine where a
treatment regime is a function that maps individual patient information to a recommended
treatment, hence explicitly incorporating the heterogeneity in need for
treatment across individuals, we further extend our knowledge gradient policy to a
Bayesian contextual bandits setting. Since the sparsity and the relatively small number
of patients make learning more dicult, we design an ensemble optimal learning
method, in which multiple models are strategically generated and combined to minimize
the incorrect selection of a particularly poorly performing statistical model.
Driven by numerous needs among materials science society, we developed a KG policy
for sequential experiments when experiments can be conducted in parallel and/or
there are multiple tunable parameters which are decided at dierent stages in the
process. Finally, we present a new Modular, Optimal Learning Testing Environment
(MOLTE) as a public-domain test environment to facilitate the process of more
comprehensive comparisons, on a broader set of test problems and a broader set of

Follow us: Facebook Twitter Linkedin