3. Neural computations underpinning the representation of cognitive maps
Lead Investigator: Zuzanna Balewski
Potential treatments: Depression, addiction, schizophrenia, Parkinson's
Reinforcement learning (RL) is a computational framework that formalizes how we learn to select actions to acquire reward and avoid punishment. RL includes two distinct systems. Model-free (MF) RL is associated with habits and skills, and relies on trial-and-error, storing the value of past actions and inflexibly repeating those that led to higher values. The second system is model-based (MB) RL, associated with goal-directed actions, and generates predictions via a computationally expensive process using a model of the environment. However, its output is flexible to environmental changes. Orbitofrontal cortex (OFC) is thought to play an important role in MB RL by constructing “cognitive maps” of the structure of behavioral tasks, but how this is accomplished at the single neuron level remains unclear.
MB RL is typically studied using reevaluation manipulations, which require subjects to use a task model to respond to some change in the task structure. The immediate update that is required prevents the use of trial-and-error based MF RL. A hindrance to neurophysiological investigations of these reevaluation manipulations is they depend on one-shot learning. As such, they cannot be studied using traditional neurophysiology methods, which only record from a small number of neurons at a time and require multiple sessions to build up an adequate sample size. Our solution to this problem has been to develop methods for recording from many OFC neurons simultaneously, thereby enabling us to collect a representative sample of OFC neural activity in just one or two recording sessions (see Innovation).