Published journal articles

The tortoise and the hare: interactions between reinforcement learning and working memory [pdf] [summary] Collins, AGE (2017) Submitted, bioRxiv preprint

Within and across-trial dynamics of human EEG reveal cooperative interplay between reinforcement learning and working memory [pdf] Collins, AGE and Frank, MJ (2017)
Submitted, bioRxiv preprint

Interactions Among Working Memory, Reinforcement Learning, and Effort in Value-Based Choice: A New Paradigm and Selective Deficits in Schizophrenia [pdf] Collins, AGE, Albrecht, MA, Waltz, JA, Gold, JM and Frank, MJ (2017)
Biological Psychiatry

Prefrontal Cortex in Control: Broadening the Scope to Identify Mechanisms [pdf] Alexander, WH, Brown, JW, Collins, AGE, Hayden, BY, and Vassena, E (2017)
Journal of Cognitive Neuroscience

Working memory load strengthens reward prediction errors [pdf] [SI]
Collins, AGE, Ciullo, B, Frank, MJ, and Badre, D. (2017)
Journal of Neuroscience

The cost of structure learning [pdf]
Collins, AGE (2017)
Journal of Cognitive Neuroscience

Stimulus discriminability may bias value-based probabilistic learning [pdf]
Schutte, I, Slagter, HA, Collins, AGE, Frank, MJ, and Kenemans, JL (2017)
PLoS One

Role of Prefrontal Cortex in Learning and Generalizing Hierarchical Rules in 8-Month-Old Infants [pdf]
Werchan, DM, Collins, AGE, Frank, MJ, and Amso, D (2016)
The Journal of Neuroscience

Probabilistic Reinforcement Learning in Patients with Schizophrenia: Relationships to Anhedonia and Avolition [pdf] Dowd, EC, Frank, MJ, Collins, AGE, Gold, JM, and Barch, DM (2016)
Biological Psychiatry

Neural signature of hierarchically structured expectations predicts clustering and transfer of rule sets in reinforcement learning [pdf]
Collins, AGE, Frank, MJ (2016)

Motor demands constrain cognitive rule structures [pdf]
Collins, AGE, and Frank, MJ (2016)
Plos Computational Biology

Surprise! dopamine signals mix action, value and error [pdf]
Collins, AGE, and Frank, MJ (2015)
Nature Neuroscience

8-Month-Old infants spontaneously learn and generalize hierarchical rules [pdf]
Werchan, DM, Collins, AGE, Frank, MJ, and Amso, D (2015)
Psychological Science

Working memory contributions to reinforcement learning impairments in schizophrenia [pdf]
Collins, AGE, Brown, J, Gold, J, Waltz, J, and Frank, MJ (2014)
Journal of Neuroscience

A reinforcement learning mechanism responsible for the valuation of free choice [pdf]
Cockburn, J, Collins, AGE, and Frank, MJ (2014)

Opponent Actor Learning (OpAL): modeling interactive effect of striatal dopamine on reinforcement learning and choice incentive [pdf] Collins, AGE and Frank, MJ. (2014)
Psychological Review

Foundations of human reasoning in the prefrontal cortex [pdf]
Donoso, M, Collins, AGE, Koechlin, E. (2014)

Human EEG uncovers latent generalizable rule structure during learning [pdf]
Collins, AGE, Cavanagh, JF, and Frank, MJ (2014)
Journal of Neuroscience

Cognitive control over learning: creating, clustering and generalizing task-set structure [pdf]
Collins, AGE and Frank, MJ. (2013)
Psychological Review

How much of reinforcement learning is working memory, not reinforcement learning? A behavioral, computational and neurogenetic analysis [pdf] Collins, AGE and Frank, MJ. (2012)
European Journal of Neuroscience

Negative symptoms and the failure to represent the expected reward value of actions: behavioral and computational modeling evidence [pdf]
Gold, JM; Waltz, JA; Matveeva, TM; Kasanova, Z; Strauss, G; Herbener, E; Collins, AGE; Frank, MJ (2012)
Arch Gen Psychiatry

Reasoning, learning, and creativity: frontal lobe function and human decision-making [pdf]
Collins A, Koechlin E (2012)
PLoS Biology

A computational theory of prefrontal executive control
Collins, A. and Koechlin, E. (2009)
Frontiers in Systems Neuroscience

Computational models and code

This is a link to the neural network model simulated in Collins & Frank, 2013, Psychological Review. It is a model of task-set structure learning in hierarchical corticostriatal circuits. It runs on the emergent neural simulator (Aisa et al., 2008). Details about emergent and previous neural networks on which this network builds can be found on Michael Frank's basal ganglia projects webpage, here.

This network includes a two-stage cascaded basal ganglia loop circuit enabling hierarchical control of action selection and learning by generating task-set structure, generalizable to novel situations. The model selects among four different motor actions, and at the higher level, three possible task-sets, and simultaneously learns to create (or re-use) abstract task-sets while also learning the particular response mappings given the selected task-set, using pure reinforcement learning.

This matlab script can be used for more detailed analysis of model output showing transfer, and here is an example mat file. Similarly, for more detailed analysis of a case in which there is incentive to clustering task-sets around context during initial learning, please use this matlab script.

The computations of this model were linked to those of a higher level "C-TS" (context task-set) model based on a non-parametric Bayesian approach to clustering task-sets using a Chinese Restaurant Process. Here is a single zip file including simulations from the C-TS model in matlab.