PSYCHOLOGY 720, Fall 2008-09

Module 3: Systems and Behavioural Neuroscience, Sue Becker (becker at mcmaster dot ca)

Content

This module focuses on the neural mechanisms underlying learning and memory, and in particular, on the neural mechanisms underlying reinforcement learning and hippocampal spatial memory. We will draw on articles covering a range of approaches including unit electrophysiology, computational modelling and neuroimaging.

Evaluation

Evaluation will be as per your Psych720 course outline, with the instructor's discretionary component for this module based on participation in discussions of required readings.

November 10: Lecture by Sue Becker (Theme: learning and memory systems in the brain)

Required readings:
  1. Eagleman, DM, Montague, PR (2002) Models of learning and memory. Encyclopedia of Cognitive Science, New York: MacMillan Publishers Ltd. pp. 806-812. link to PDF on PRM's web page
  2. Montague, PR, Eagelman, DM, McClure, SM, Berns, GS (2002) Reinforcement Learning, Encyclopedia of Cognitive Science, London, Eng: Macmillan Publishers Ltd. pp. 908-913. link to PDF on PRM's web page
  3. Burgess, N and Hitch, G. (2005), Computational models of working memory: putting long-term memory into context. Trends In Cognitive Sciences Volume: 9 Issue: 11 Pages: 535-541. link to article on NB's web page
  4. Becker, S. and Wojtowicz, J.M. (2007), A model of hippocampal neurogenesis in memory and mood disorders. Trends in Cognitive Sciences 11(2):70-76. link to article on journal's web site
  5. Bird, C.N. and Burgess, N. (2008), The hippocampus and memory: insights from spatial processing Nature Reviews Neuroscience 9(3):182-194. link to article on journal's web site
Supplemental readings:
  1. Becker, S. (2005), Modelling the mind: From circuits to systems. Chapter 2 in New Directions in Statistical Signal Processing: From systems to brain. Simon Haykin, Jose C. Principe, Terrence J. Sejnowski and John McWhirter (editors), MIT Press.
    2. link to PDF article on SB's web page, link to PDF references section of article on SB's web page
  2. Burgess N. (2002) The Hippocampus, space and viewpoints in episodic memory Quart. J. Exp. Psychol. (2002) 55A, 1057-1080. link to article on NB's web page
Questions to think about:
  1. Does each region of the neocortex have its own distinct form of learning?
  2. Are there specialized sub-systems, either within neocortex, or in other cortical and sub-cortical regions, for learning different types of information, e.g. reward contingencies, statistical structure of the world, memory for single events vs general knowledge, motor/ procedural memory, implicit versus explicit knowledge?
  3. Why would the brain be organized in this way?
  4. Do these systems interact or are they really separate? How do neuromodulators and attentional circuits influence learning and memory?

November 24 (Theme: Reinforcement learning)

Hour 1: Reward prediction errors, appetitive reinforcement learning and dopamine in the ventral striatum

Required reading: O'Doherty, J.P., Dayan, P., Friston, K., Critchley, H. and Dolan, R.J. (2003) Temporal difference models and reward-related learning in the human brain. Neuron 38:329-337. link to article on PD's web site
Presenter: Jonathan
Critiques: Adam, Scott, Nicole, Morgan
Supplemental readings:
  1. McClure SM, Berns GS, Montague PR (2003) Temporal prediction errors in a passive learning task activate human striatum. Neuron 38:339-346 link to article on PRM's web page
  2. O'Doherty, J, Dayan, P, Schultz, J, Deischmann, R, Friston, K & Dolan, RJ (2004), Dissociable roles of ventral and dorsal striatum in instrumental conditioning. Science 304 452-454. Link to article on PD's web page
  3. Chiu, PH, Lohrenz, TM, Montague, PR (2008) Smokers' brains compute, but ignore, a fictive error signal in a sequential investment task. Nature Neuroscience 11(4):514-520. link to article on PRM's web page
Question to think about: How much of human performance can be explained by reinforcement learning theory, and where does this theory fall short?

Hour 2: Separate systems for aversive and appetitive learning?

Required reading: Seymour, B., Daw, N., Dayan, P., Singer, T. and Dolan, R. (2007), Differential encoding of losses and gains in the human striatum. Journal of Neuroscience 27 (18): 4826-4831 link to article on PD's web site
Presenter: Blake
Critiques: Zachary, Sophia, Daniel
Supplemental readings:
  1. Kahnt, T., Park, S.Q., Cohen, M.X., Beck, A., Heinz, A. and Wrase, J. (2008, to appear), Dorsal Striatal-midbrain Connectivity in Humans Predicts How Reinforcements Are Used to Guide Decisions. Journal of Cognitive Neuroscience. link to article on journal's web site
  2. Pessiglione M, Seymour B, Flandin G, Dolan RJ, Frith CD (2006) Dopamine-dependent prediction errors underpin reward-seeking behaviour in humans. Nature 442:1042-1045 link to article on BS's web site
  3. Dayan P & Huys QJM (2008) Serotonin, inhibition and negative mood. Public Library of Science: Computational Biology 4 e4. link to article on PD's web site
Question to think about: Why might the brain have become organized in this way, what are the advantages and disadvantages?

Hour 3: Separate dopamine sub-systems in the basal ganglia for "go" and "no-go" responses

Required reading: Moustafa, A.A., Sherman, S.J. & Frank, M.J. (in press). A dopaminergic basis for working memory, learning and attentional shifting in Parkinsonism. Neuropsychologia. link to article on MF's web site
Presenter: Tiffany
Critiques: Laura, Ryan, Corrine, Mark
Supplemental readings:
  1. Pizzagalli, D.A., Evins, A.E., Schetter, E.C., Frank, M.J., Pajtas, P.E., Santesso, D.L. & Culhane, M. (2008). Single dose of a dopamine agonist impairs reinforcement learning in humans: Behavioral evidence from a laboratory-based measure of reward responsiveness. Psychopharmacology, 196, 221--232. link to article on MF's web site
  2. Frank, M.J., Samanta, J., Moustafa, A.A. & Sherman, S.J. (2007). Hold your horses: Impulsivity, deep brain stimulation and medication in Parkinsonism. Science, 318, 1309-1312. link to article on MF's web site
  3. Frank, M.J., Moustafa, A.A., Haughey, H., Curran, T. & Hutchison, K. (2007). Genetic triple dissociation reveals multiple roles for dopamine in reinforcement learning. Proceedings of the National Academy of Sciences, 104, 16311-16316. link to article on MF's web site
Question to think about: Why might the brain have become organized in this way, what are the advantages and disadvantages?

December 1: Spatial memory

Hour 1: Allocentric Coding In Human Memory

Required reading: Ekstrom, A.D., Kahana, M.J., Caplan, J.B., Fields, T.A., Isham, E.A., Newman, E.L. and Fried, I. (2003), Cellular networks underlying human spatial navigation. Nature. link to article on journal's web site
Presenter: Jillian
Critiques: Scott, Sophia, Daniel, Mark
Supplemental readings:
  1. TOLMAN EC (1948), Cognitive Maps In Rats And Men, Psychological Review 55(4):189-208. Available online from McMaster library e-journals
  2. King, J., Burgess, J., Hartley, T., Vargha-Khadem, F. and O'Keefe, J. (2002), THe human hippocampus and viewpoint dependence in spatial memory. Hippocampus. 2002;12(6):811-20. link to article on NB's web site
  3. Maguire EA, Gadian DG, Johnsrude IS, Good CD, Ashburner J, Frackowiak RSJ, Frith CD (2000), Navigation-related structural change in the hippocampi of taxi drivers, Proceedings Of The National Academy Of Sciences Of The United States Of America 97:4398-4403. link to article on journal's web site
Question to think about: Given that allocentric representations are difficult to compute, and are not directly available from the sensory input, why would we bother to create and use such representations? What limitations do they have?

Hour 2: Spatial Memory and Mental Imagery

Required reading: Hassabis, D., Kumaran, D. and Maguire, E.A. (2007), Using imagination to understand the neural basis of episodic memory JOURNAL OF NEUROSCIENCE 27(52):14365-14374 link to article on journal's web site
Presenter: Maria
Critiques: Adam, Zachary, Laura
Supplemental readings:
  1. Waller D, Hodgson E (2006), Transient and enduring spatial representations under disorientation and self-rotation, Journal of Experimental Psychology-Learning Memory and Cognition 32(4):867-882. link to article on journal's web site
  2. Wolbers T, Hegarty M, Buchel C, and Loomis, J.M. (2008), How the brain keeps track of changing object locations during observer motion Nature Neuroscience 11(10):1223-1230 link to article on journal's web site
  3. Burgess N (2006), Spatial memory: how egocentric and allocentric combine, Trends In Cognitive Sciences 10(12):551-557 link to article on NB's web site
Questions to think about: Is spatial memory dependent upon mental imagery? Can you have one without the other? Is the hippocampus just about memory, or is it also reconstructing the past and the future?

Hour 3: Neural Basis Of Spatial Strategy Differences

Required reading: Doeller, C.F. and Burgess, N. (2008), Distinct error-correcting and incidental learning of location relative to landmarks and boundaries, PROCEEDINGS OF THE NATIONAL ACADEMY OF SCIENCES OF THE UNITED STATES OF AMERICA 105(15):5909-5914. link to article on journal's web site
Presenter: Ellen
Critiques: Nicole, Ryan, Corrine, Morgan
Supplemental readings:
  1. Doeller, C.F., King, J.A. and Burgess, N. (2008), Parallel striatal and hippocampal systems for landmarks and boundaries in spatial memory, Proc Natl Acad Sci USA 105(15):5915-5920 link to article on journal's web site
  2. Packard MG, McGaugh JL (1996), Inactivation of hippocampus or caudate nucleus with lidocaine differentially affects expression of place and response learning, NEUROBIOLOGY OF LEARNING AND MEMORY 65(1):65-72. link to article on sciencedirect's web site
  3. Bohbot VD, Lerch J, Thorndycraft B, et al. (2007), Gray matter differences correlate with spontaneous strategies in a human virtual navigation task, Journal Of Neuroscience 27(38):10078-10083 link to article on VB's web site
Questions to think about: When does it make sense to rely on an allocentric navigation strategy versus a response learning strategy? Is one more primitive, and one more sophisticated than the other?