Hierarchical Functional Category Learning for Efficient Value Function Approximation in Object-based Environments

Creating autonomous long-lived agent that can robustly function in a complex object-based environment has been a persistent goal in the field of artificial intelligence. Learning the functional categories of objects is one of the keys to achieve this goal, and is the theme of this thesis.

We formulate the research problem as finding efficient value function approximation algorithms, where the input to the value function is an object-based state representation, and output of the value function is the utility value of that input state. The challenges arise from the requirement of efficient learning, and incremental learning for complex non-linear value functions, whose input consists of diverse objects in high dimensional feature space. Inspired by observations from human category learning, we make the assumption of hierarchical distribution pattern in the functional space of objects, and make the general design of using hierarchical symbolic category representation to assist generalization, and achieve efficient learning of value functions. We provide two implementations based on this general design, with evaluations both based on functionality and on cognitive plausibility.

Traditionally, category learning and value function approximation are studied as separate problems. The thesis presents a unique synthesis of the two. On one hand, it provides efficient value function approximation algorithms that can take advantage of compact representational basis adaptively generated by hierarchical category learning. On the other hand, it provides a utility driven category learning model that offers new computational insights to human category learning.