Abstract/Details

Feature Selection for Value Function Approximation


2011 2011

Other formats: Order a copy

Abstract (summary)

The field of reinforcement learning concerns the question of automated action selection given past experiences. As an agent moves through the state space, it must recognize which state choices are best in terms of allowing it to reach its goal. This is quantified with value functions, which evaluate a state and return the sum of rewards the agent can expect to receive from that state. Given a good value function, the agent can choose the actions which maximize this sum of rewards. Value functions are often chosen from a linear space defined by a set of features; this method offers a concise structure, low computational effort, and resistance to overfitting. However, because the number of features is small, this method depends heavily on these few features being expressive and useful, making the selection of these features a core problem. This document discusses this selection.

Aside from a review of the field, contributions include a new understanding of the role approximate models play in value function approximation, leading to new methods for analyzing feature sets in an intuitive way, both using the linear and the related kernelized approximation architectures. Additionally, we present a new method for automatically choosing features during value function approximation which has a bounded approximation error and produces superior policies, even in extremely noisy domains.

Indexing (details)


Subject
Artificial intelligence;
Computer science
Classification
0800: Artificial intelligence
0984: Computer science
Identifier / keyword
Applied sciences; Approximation errors; Feature selection; Noisy domains; Reinforcement learning; Value function approximation
Title
Feature Selection for Value Function Approximation
Author
Taylor, Gavin
Number of pages
113
Publication year
2011
Degree date
2011
School code
0066
Source
DAI-B 72/07, Dissertation Abstracts International
Place of publication
Ann Arbor
Country of publication
United States
ISBN
9781124612850
Advisor
Parr, Ronald
Committee member
Conitzer, Vincent; Maggioni, Mauro; Sun, Peng
University/institution
Duke University
Department
Computer Science
University location
United States -- North Carolina
Degree
Ph.D.
Source type
Dissertations & Theses
Language
English
Document type
Dissertation/Thesis
Dissertation/thesis number
3453296
ProQuest document ID
867638250
Copyright
Database copyright ProQuest LLC; ProQuest does not claim copyright in the individual underlying works.
Document URL
http://search.proquest.com/docview/867638250
Access the complete full text

You can get the full text of this document if it is part of your institution's ProQuest subscription.

Try one of the following:

  • Connect to ProQuest through your library network and search for the document from there.
  • Request the document from your library.
  • Go to the ProQuest login page and enter a ProQuest or My Research username / password.