Computational models of Learning and memory (Psych734/CES734) Fall 2016



Instructor

Professor Sue Becker, becker (at mcmaster dot ca)
Office hours: by appointment.

TA and co-instructor

Kiret Dhindsa, dhindsj (at mcmaster dot ca)

Schedule

Thursdays in PC-311, Sept 15 - Dec 15 (no class Sept 29, Nov 3)

Overview

This course will cover some of the most influential computational models of learning and memory, and the application of these models to understanding how the brain learns and encodes information, the analysis of neuroscientific data and brain-computer interfaces. In the first 2 hours of each class, lectures will draw on classic papers in the literature, while in hour 3 students will present and discuss papers describing applications of the models.

Required background

Students must have some computer programming experience (in any programming language), and be comfortable writing programs that include loops, variables and procedures/functions.

If you've never programmed in Matlab, it would be useful to read the Matlab mini-tutorial in section 1.5 of Jay McClelland's book "Explorations in Parallel Distributed Processing: A Handbook of Models, Programs, and Exercises" (a draft of the 2nd edition is available online at this link).

Assessment

Participation 10%
3 programming assignments (10% each) 30%
One 20-30 minute oral presentation 20%
One final project 40%

Assignments
The 3 assignments will be programmed in Matlab, and will be due approximately every 2nd week in the earlier part of the course. Each assignment will involve simulating one of the models discussed in class and writing up your results in the format of a brief scientific report (intro, methods, results, discussion). Assignments must be turned in by 3:30pm on due dates posted (to be announced). 20% of the total possible mark per day will be deducted for late assignments.
Presentation
The presention of a research article will involve summarizing the key points of the article, suggesting some questions for discussion and leading the class discussion of the paper.
Participation
Participation marks will be earned for contributing to the class discussions of the 11 research articles. One mark for each paper in which a non-trivial contribution is made to the discussion, up to a maximum of 10 points.
Final Project
The project will involve the application of one of the learning models discussed in the course to the classification of EEG data for a brain-controlled interface application.
Due date: December 22, 4pm in hard-copy to Sue Becker's mailbox

Weekly outline and readings (more to be added beyond week 1)

Week 1, Sept 15. Overview and background.

Course overview, review of functional neuroanatomy, neurobiology of learning and memory, introduction to issues in computational modelling, Bottom-up models of learning: LTP, STDP.
Link to lecture slides (pdf)
Background Readings:
  • Yang D and Poo M (2004) Spike timing-dependent plasticity of neural circuits. Neuron 44.1:23-30.
  • McClelland JL, McNaughton BL & O'Reilly RC (1995). Why There Are Complementary Learning Systems in the Hippocampus and Neocortex: Insights From the Successes and Failures of Connectionist Models of Learning and Memory. Psychological Review, 102(3), 419-457. pdf
  • Rogers TT & McClelland JL (2014), Parallel Distributed Processing at 25: Further Explorations in the Microstructure of Cognition. Cognitive Science 38:1024-1077. pdf

Week 2, Sept 22: Hebbian associative memory models

Linear pattern associators, competitive learning, Kohonen networks (self-organizing maps), BCM learning rule.
Link to lecture slides (pdf)
Primary readings
  • McClelland J, Chapter 4, Learning in PDP Models: The Pattern Associator, in "Explorations in Parallel Distributed Processing: A Handbook of Models, Programs, and Exercises" (link to draft of the 2nd edition).
  • McClelland J, Chapter 6, Competitive Learning, in "Explorations in Parallel Distributed Processing: A Handbook of Models, Programs, and Exercises" (link to draft of the 2nd edition).
Supplementary readings
  • Bienenstock EL, Cooper LN & Munro PW (1982). "Theory for the development of neuron selectivity: orientation specificity and binocular interaction in visual cortex." The Journal of Neuroscience 2.1:32-48. pdf
  • Izhikevich EM & Desai NS (2003), Relating STDP to BCM. Neural Computation 15: 1511-1523. pdf
  • Marr D (1971), Simple Memory - Theory For Archicortex. Philosophical Transactions Of The Royal Society Of London Series B-Biological Sciences 262 (841): 23- pdf
  • McNaughton BL & Morris RGM (1987), Hippocampal Synaptic Enhancement and Information-Storage Within A Distributed Memory System. Trends in Neurosciences 10(10): 408-415 pdf
  • Becker S (in press), Marr's theory of the hippocampus as a simple memory: Decades of subsequent research suggest it is not that simple. To appear in Computational Theories and their Implementation in the Brain: The Legacy of David Marr. Edidted by L.M. Vaina and R.E. Passingham. Oxford University Press. pdf
  • Fusi S, Drew PJ & Abbott LF (2005), Cascade models of synaptically stored memories. Neuron 45(4): 599-611 pdf
Paper to be presented / discussed in hour 3
  • Presentation 1 Jensen O & Lisman JEE (1996) Theta/gamma networks with slow NMDA channels learn sequences and encode episodic memory: role of NMDA channels in recall. Learning and memory 3.2-3:264-278. pdf

Weeks 3-4, October 6, October 13. Hopfield Networks and Boltzmann Machines.

Link to lecture slides (pdf)
Primary readings
  • Hinton GE (1989) Connectionist learning procedures. Artificial Intelligence 40, Read pages 191-193 (Hopfield nets) and 209-214 (Boltzmann machines). pdf
  • Salakhutdinov RR & Hinton G E (2012), An Efficient Learning Procedure for Deep Boltzmann Machines, Neural Computation 24:1967-2006. pdf
Supplementary readings
  • Hopfield JJ (1982)"Neural networks and physical systems with emergent collective computational abilities" Proc. Natl. Acad. Sci. USA 79(2):554-2558. pdf
  • Smolensky P (1986), Information processing in dynamical systems: Foundations of harmony theory. In D. E. Rumelhart, J. L. McClelland & the PDP Research Group, Parallel Distributed Processing: Explorations in the Microstructure of Cognition. Volume 1: Foundations. Cambridge, MA: MIT Press/Bradford Books. 194.281. pdf
  • Hinton GE & Sejnowski TJ (1986), Learning and relearning in Boltzmann machines. In Rumelhart, D. E. and McClelland, J. L., editors, Parallel Distributed Processing: Explorations in the Microstructure of Cognition. Vol 1: Foundations, MIT Press pp 282-317 pdf
  • Sejnowski TJ & Hinton GE (1987), Separating figure from ground using a Boltzmann machine. In Arbib, M. and Hanson, A.R., editors, Vision, Brain and Cooperative Computation, MIT Press. pdf
  • Reichert, DP, Series, P and Storkey, AJ (2013), Charles Bonnet Syndrome: Evidence for a Generative Model in the Cortex? PLOS Computational Biology 9(7) DOI: 10.1371/journal.pcbi.1003134. pdf
  • Finnegan R & Becker S (2015), Neurogenesis paradoxically decreases both pattern separation and memory interference. Frontiers in Systems Neuroscience 9:136 doi: 10.3389/fnsys.2015.00136 pdf
October 6. Paper to be presented / discussed in hour 3
  • Presentation 2 Zhang Y, Li Y, Samonds JM, Lee TS (2016), Relating functional connectivity in V1 neural circuits and 3D natural scenes using Boltzmann machines. Vision Research 120:121-131. pdf
October 13. Paper to be presented / discussed in hour 3
  • Presentation 3 Hjelm RD, Calhoun VD, Salakhutdinov R, Allen EA, Adali T, Plis SM (2014), 96:245-260. Restricted Boltzmann machines for neuroimaging: An application in identifying intrinsic networks. Neuroimage 96:245-260. pdf

Weeks 5-6, Oct 20, Oct 27. Reinforcement learning.

Link to lecture slides (pdf)
Primary readings
  • Dayan P & Abbott L (2000), "Classical Conditioning and Reinforcement Learning", Chapter 9 in Theoretical Neuroscience, MIT Press. pdf
  • Shultz W, Dayan P & Montague PR (1997), A neural substrate of prediction and reward. Science March 14 1997, 275:1593-1599. pdf
  • Y Niv Y (2009) Reinforcement learning in the brain. The Journal of Mathematical Psychology 53(3):139-154 pdf
Supplementary readings
  • Sutton RS & Barto AG (1990), Time-derivative models of Pavlovian reinforcement. Chapter 12 in Learning and Computational Neuroscience: Foundations of Adaptive Networks. M. Gabriel and J. Moore (eds), p497-537, MIT Press. pdf
  • Montague PR, Dayan P & Sejnowski TJ (1996) A framework for mesencephalic dopamine systems babased on predictive Hebbian learning. Journal of Neuroscience 16:1936-1947.
  • pdf
  • Frank MJ (2005) Dynamic dopamine modulation in the basal ganglia: a neurocomputational account of cognitive deficits in medicated and nonmedicated parkinsonism. Journal of Cognitive Neuroscience 171:51-72. pdf
Paper to be presented / discussed in hour 3, October 20
  • Presentation 4 Moustafa AA, Cohen MX, Sherman SJ & Frank MJ (2008), A Role for Dopamine in Temporal Decision Making and Reward Maximization in Parkinsonism. Journal of Neuroscience 28(47):12294-12304. pdf
Paper to be presented / discussed in hour 3, October 27
  • Presentation 5 Mnih et al (2015) Human-level control through deep reinforcement learning. Nature vol 518, Feb 26. pdf

Weeks 7-9, November 10, 17, 24. Supervised learning models

Link to lecture slides (pdf)
Primary readings
  • Hinton GE (1989) Connectionist learning procedures. Artificial Intelligence 40. Read up to Page 209. pdf
  • Larochelle H et al. (2009) Exploring Strategies for Training Deep Neural Networks. Journal of machine learning research 10:1-40. pdf
Supplementary readings
  • Rumelhart DE, Hinton GE & Williams RJ (1986), Learning internal representations by error propagation. In Rumelhart, D. E. and McClelland, J. L., editors, Parallel Distributed Processing: Explorations in the Microstructure of Cognition. Volume 1: Foundations, MIT Press, Cambridge, MA. Chapter 8, pp 318-362. pdf
  • Waibel A, Hanazawa T, Hinton GE, Shikano K & Lang K (1989) Phoneme recognition using time-delay neural networks. IEEE Acoustics Speech and Signal Processing, 37(3):328-339. pdf
  • Elman JL (1990). Finding Structure in Time. Cognitive Science 14:179-211. pdf
  • Fels SS & Hinton GE (1993) Glove-Talk: A neural network interface between a data-glove and a speech synthesizer. IEEE Transactions on Neural Networks 4(1):2-8. pdf
  • Hochreiter S, Schmidhuber J (1997), Long Short-term Memory. Neural Computation 9:1735-1780. pdf
Nov 10 Paper to be presented / discussed in hour 3
  • Presentation 6 Cheyette SJ and Plaut DC (under review) Modeling the N400 ERP component as transient semantic over-activation within a neural network model of word comprehension. Note: This article is unpublished and was kindly shared by the author for educational purpose only. Please ask your instructor to send you a copy and do not cite or share without the author's permission.
Nov 17 Paper to be presented / discussed in hour 3
  • Presentation 7 Lee H & Kuhl BA (2016), Reconstructing Perceived and Retrieved Faces from Activity Patterns in Lateral Parietal Cortex. Journal of Neuroscience 36(22):6069-6082
Nov 24 Paper to be presented / discussed in hour 3
  • Presentation 8 Gregor K, Danihelka I, Graves A, Rezende DJ & Wierstra D (2015), DRAW: A Recurrent Neural Network For Image Generation http://arxiv.org/abs/1502.04623

Week 10, December 1. Neural networks and machine learning models for data analysis

Link to lecture slides (pdf)
Dec 1 Primary readings
  • Crisci C, Ghattas B & Perera G (2012). A review of supervised machine learning algorithms and their applications to ecological data. Ecological Modelling 240:113-122.
Dec 1 Supplementary readings
  • Schwenker F & Trentin E (2014). Pattern classification and clustering: a review of partially supervised learning approaches. Pattern Recognition Letters 37:4-14.
Dec 1 Paper to be presented in hour 3
  • Presentation 9 Zhang H, Gua C, Ang KK & Wang C (2012). BCI competition IV . data set I: learning discriminative patterns for self-paced EEG-based motor imagery detection. Frontiers in Neuroscience 6, doi: 10.3389/fnins.2012.00007
    pdf

Week 11, Dec 8.Decoding mental states and representing neural data

Link to lecture slides (pdf)
Dec 8 Primary reading
Dec 8 Supplementary reading
  • Nishimoto S, Vu TA, Naselaris T, Benjamini Y, Yu B & Gallant JL (2011). Reconstructing Visual Experiences from Brain Activity Evoked by Natural Movies. Current Biology 21:1641-1646
Dec 8 Paper to be presented in hour 3:
  • Presentation 10 Huth AG, Nishimoto S, Vu AT & Gallant JL (2012), A Continuous Semantic Space Describes the Representation of Thousands of Object and Action Categories across the Human Brain, Neuron 76(6)1210-1224. pdf

Week 12, Dec 15. Brain-computer Interfacing

Link to lecture slides (pdf)
Dec 15 Primary Reading
  • Wolpaw JR, Birbaumer N, McFarland DJ & Pfurtscheller G & Vaughan TM (2002), Brain-computer interfaces for communication and control. Clinical neurophysiology 113(6):767-91. Link to article in McMaster e-resources
Dec 15 Supplementary Readings
  • Nicolas-Alonso LF, Gil JG & Alonso LFN (2012) Brain computer interfaces, a review. Sensors 12(2):1211-79.
  • Hinterberger T, Veit R, Wilhelm B, Weiskopf N, Vatine JJ & Birbaumer N (2005). Neuronal mechanisms underlying control of a brain-computer interface. The European Journal of Neuroscience, 21(11), 3169.81. doi:10.1111/j.1460-9568.2005.04092.x
    Link to article in McMaster e-resources
Dec 15 Paper to be presented in hour 3
  • Presentation 11 Sturm I, Bach S, Samek W & Müller KR (2016). Interpretable Deep Neural Networks for Single-Trial EEG Classification. arXiv preprint https://arxiv.org/abs/1604.08201