Simple Partial Models for Complex Dynamical Systems

An agent behaving in an unknown environment may wish to learn a model
that allows it to make predictions about future events and to
anticipate the consequences of its actions. Such a model can greatly
enhance the agent's ability to make good decisions. However, in
environments like the physical world in which we live, which is
stochastic, partially observable, and high dimensional, learning a
model is a challenge. One natural approach when faced with a difficult
model learning problem is not to model the entire system. Instead, one
might focus on capturing the most important aspects of the environment
and give up on modeling complicated, irrelevant phenomena. This
intuition can be formalized using partial models, which are models
that make only a restricted set of (abstract) predictions in only a
restricted set of circumstances. Because a partial model has limited
prediction responsibilities, it may be significantly simpler than a
complete model.

The goal of this thesis is to provide general results and methods for
learning partial models, specifically in partially observable systems.
Some of the main challenges posed by partial observability are
formalized and learning methods are developed to address some of these
issues. The learning methods presented are demonstrated empirically to
be able to learn partial models in systems that are too complex for
standard, complete model learning methods. Finally, many partial
models are learned and composed to form complete models that are used
for model-based planning in high dimensional arcade game examples.