Computational models of Learning and memory (Psych734/CES734) Winter 2019



Instructor

Professor Sue Becker, becker (at mcmaster dot ca)
Office hours: by appointment.

TA

Saurabh Shaw, shaws5 (at mcmaster dot ca)

Schedule

Thursdays 9-11:30am in PC-311 beginning Jan 10, 2019. Last class will be April 11 or 18 (no class during reading week and possibly in 1 or 2 weeks in March)

Overview

This course will cover some of the most influential computational models of learning and memory, and the application of these models to understanding how the brain learns and encodes information, the analysis of neuroscientific data and brain-computer interfaces. In the first 2 hours of each class, lectures will draw on classic papers in the literature, while in hour 3 students will present and discuss papers describing applications of the models.

Required background

Students must have some computer programming experience (in any programming language), and be comfortable writing programs that include loops, variables and procedures/functions.

If you've never programmed in Matlab, it would be useful to read the Matlab mini-tutorial in section 1.5 of Jay McClelland's book "Explorations in Parallel Distributed Processing: A Handbook of Models, Programs, and Exercises" (a draft of the 2nd edition is available online at this link).

Assessment

Participation 10%
3 programming assignments (10% each) 30%
One 20-30 minute oral presentation 20%
One final project 40%

Assignments
The 3 assignments will be programmed in Matlab, and will be due approximately every 2nd week in the earlier part of the course. Each assignment will involve simulating one of the models discussed in class and writing up your results in the format of a brief scientific report (intro, methods, results, discussion). Assignments must be turned in by 3:30pm on due dates posted (to be announced). 20% of the total possible mark per day will be deducted for late assignments.
Presentation
The presention of a research article will involve summarizing the objectives, background, and key aspects of the methods and results, critical appraisal and suggesting some questions for discussion and leading the class discussion of the paper.
Participation
Participation marks will be earned for contributing to the class discussions of 10 of the 11 research articles (not counting the one you are presenting). One mark for each paper in which a non-trivial contribution is made to the discussion, up to a maximum of 10 points.
Final Project
The project will involve the application of one of the learning models discussed in the course to the classification of EEG data for a brain-controlled interface application.
Due date: April 20, 4pm emailed to becker@mcmaster.ca and shaws5@mcmaster.ca

Weekly outline and readings (more to be added beyond week 1)

Week 1, Jan 10. Overview and background.

Course overview, review of functional neuroanatomy, neurobiology of learning and memory, introduction to issues in computational modelling, Bottom-up models of learning: LTP, STDP.
Link to lecture slides (pdf)
Background Readings:
  • Yang D and Poo M (2004) Spike timing-dependent plasticity of neural circuits. Neuron 44.1:23-30.
  • McClelland JL, McNaughton BL & O'Reilly RC (1995). Why There Are Complementary Learning Systems in the Hippocampus and Neocortex: Insights From the Successes and Failures of Connectionist Models of Learning and Memory. Psychological Review, 102(3), 419-457. pdf
  • Rogers TT & McClelland JL (2014), Parallel Distributed Processing at 25: Further Explorations in the Microstructure of Cognition. Cognitive Science 38:1024-1077. pdf

Week 2, Jan 17: Hebbian associative memory models

Linear pattern associators, competitive learning, Kohonen networks (self-organizing maps), BCM learning rule.
Link to lecture slides (pdf)
Primary readings
  • McClelland J, Chapter 4, Learning in PDP Models: The Pattern Associator, in "Explorations in Parallel Distributed Processing: A Handbook of Models, Programs, and Exercises" (link to draft of the 2nd edition).
  • McClelland J, Chapter 6, Competitive Learning, in "Explorations in Parallel Distributed Processing: A Handbook of Models, Programs, and Exercises" (link to draft of the 2nd edition).
Supplementary readings
  • Bienenstock EL, Cooper LN & Munro PW (1982). "Theory for the development of neuron selectivity: orientation specificity and binocular interaction in visual cortex." The Journal of Neuroscience 2.1:32-48. pdf
  • Izhikevich EM & Desai NS (2003), Relating STDP to BCM. Neural Computation 15: 1511-1523. pdf
  • Marr D (1971), Simple Memory - Theory For Archicortex. Philosophical Transactions Of The Royal Society Of London Series B-Biological Sciences 262 (841): 23- pdf
  • McNaughton BL & Morris RGM (1987), Hippocampal Synaptic Enhancement and Information-Storage Within A Distributed Memory System. Trends in Neurosciences 10(10): 408-415 pdf
  • Becker S (in press), Marr's theory of the hippocampus as a simple memory: Decades of subsequent research suggest it is not that simple. To appear in Computational Theories and their Implementation in the Brain: The Legacy of David Marr. Edidted by L.M. Vaina and R.E. Passingham. Oxford University Press. pdf
  • Fusi S, Drew PJ & Abbott LF (2005), Cascade models of synaptically stored memories. Neuron 45(4): 599-611 pdf
Paper to be presented / discussed in hour 3
  • Presentation 1
    • Presenters: Keon and Lauren
    • Gault, Richard, Thomas MMcginnity, and SonyaColeman. "A Computational Model of Thalamocortical Dysrhythmia in People With Tinnitus." IEEE Transactions on Neural Systems and Rehabilitation Engineering 26.9 (2018):1845-1857. link to article in McMaster library

Weeks 3-4, Jan 24, Jan 31. Hopfield Networks and Boltzmann Machines.

Link to lecture slides (pdf)
Link to lecture slides (pdf)
Primary readings
  • Hinton GE (1989) Connectionist learning procedures. Artificial Intelligence 40, Read pages 191-193 (Hopfield nets) and 209-214 (Boltzmann machines). pdf
  • Salakhutdinov RR & Hinton G E (2012), An Efficient Learning Procedure for Deep Boltzmann Machines, Neural Computation 24:1967-2006. pdf
Supplementary readings
  • Friston, KJ (2010), The free-energy principle: a unified brain theory? Nature Reviews Neuroscience 11(2):127-138, DOI: 10.1038/nrn2787
  • Hopfield JJ (1982)"Neural networks and physical systems with emergent collective computational abilities" Proc. Natl. Acad. Sci. USA 79(2):554-2558. pdf
  • Smolensky P (1986), Information processing in dynamical systems: Foundations of harmony theory. In D. E. Rumelhart, J. L. McClelland & the PDP Research Group, Parallel Distributed Processing: Explorations in the Microstructure of Cognition. Volume 1: Foundations. Cambridge, MA: MIT Press/Bradford Books. 194.281. pdf
  • Hinton GE & Sejnowski TJ (1986), Learning and relearning in Boltzmann machines. In Rumelhart, D. E. and McClelland, J. L., editors, Parallel Distributed Processing: Explorations in the Microstructure of Cognition. Vol 1: Foundations, MIT Press pp 282-317 pdf
  • Sejnowski TJ & Hinton GE (1987), Separating figure from ground using a Boltzmann machine. In Arbib, M. and Hanson, A.R., editors, Vision, Brain and Cooperative Computation, MIT Press. pdf
  • Reichert, DP, Series, P and Storkey, AJ (2013), Charles Bonnet Syndrome: Evidence for a Generative Model in the Cortex? PLOS Computational Biology 9(7) DOI: 10.1371/journal.pcbi.1003134. pdf
  • Finnegan R & Becker S (2015), Neurogenesis paradoxically decreases both pattern separation and memory interference. Frontiers in Systems Neuroscience 9:136 doi: 10.3389/fnsys.2015.00136 pdf
Week 3 -Jan 24. Paper to be presented / discussed in hour 3
  • Presentation 2
    • Presenters: Mehrnoosh and Mingjie
    • Niki, N, HNishitani, and RSammouda. "A comparison of Hopfield neural network and Boltzmann machine in segmenting MR images of the brain." IEEE Transactions on Nuclear Science 43.6 (1996):3361-3369. link to article in Mcmaster library system
Week 4 - Jan 31. Paper to be presented / discussed in hour 3
  • Presentation 3
    • Presenter: Isaac
    • Hjelm, R D, et al. "Restricted Boltzmann machines for neuroimaging: an application in identifying intrinsic networks." NeuroImage (Orlando, Fla. Print) 96(2014):245-260. link to article in McMaster library system

Weeks 5-6, Feb 7, 14. Reinforcement learning.

Link to lecture slides (pdf)
Primary readings
  • Dayan P & Abbott L (2000), "Classical Conditioning and Reinforcement Learning", Chapter 9 in Theoretical Neuroscience, MIT Press. pdf
  • Shultz W, Dayan P & Montague PR (1997), A neural substrate of prediction and reward. Science March 14 1997, 275:1593-1599. pdf
  • Y Niv Y (2009) Reinforcement learning in the brain. The Journal of Mathematical Psychology 53(3):139-154 pdf
Supplementary readings
  • Sutton RS & Barto AG (1990), Time-derivative models of Pavlovian reinforcement. Chapter 12 in Learning and Computational Neuroscience: Foundations of Adaptive Networks. M. Gabriel and J. Moore (eds), p497-537, MIT Press. pdf
  • Montague PR, Dayan P & Sejnowski TJ (1996) A framework for mesencephalic dopamine systems babased on predictive Hebbian learning. Journal of Neuroscience 16:1936-1947.
  • pdf
  • Frank MJ (2005) Dynamic dopamine modulation in the basal ganglia: a neurocomputational account of cognitive deficits in medicated and nonmedicated parkinsonism. Journal of Cognitive Neuroscience 171:51-72. pdf
Week 5 (February 7) Paper to be presented / discussed in hour 3
  • Presentation 4
Week 6 (February 14) Paper to be presented / discussed in hour 3
  • Presentation 5
    • Presenters: Kyle and Hanna
    • Doya, K, and JMorimoto. "Acquisition of stand-up behavior by a real robot using hierarchical reinforcement learning." Robotics and autonomous systems 36.1 (2001):37-51. Link to article in Mcmaster library system

Weeks 7-9. March 7, 14, 21. Supervised learning models

Link to lecture slides (pdf)
Primary readings
  • Hinton GE (1989) Connectionist learning procedures. Artificial Intelligence 40. Read up to Page 209. pdf
  • Larochelle H et al. (2009) Exploring Strategies for Training Deep Neural Networks. Journal of machine learning research 10:1-40. pdf
Supplementary readings
  • Rumelhart DE, Hinton GE & Williams RJ (1986), Learning internal representations by error propagation. In Rumelhart, D. E. and McClelland, J. L., editors, Parallel Distributed Processing: Explorations in the Microstructure of Cognition. Volume 1: Foundations, MIT Press, Cambridge, MA. Chapter 8, pp 318-362. pdf
  • Waibel A, Hanazawa T, Hinton GE, Shikano K & Lang K (1989) Phoneme recognition using time-delay neural networks. IEEE Acoustics Speech and Signal Processing, 37(3):328-339. pdf
  • Elman JL (1990). Finding Structure in Time. Cognitive Science 14:179-211. pdf
  • Fels SS & Hinton GE (1993) Glove-Talk: A neural network interface between a data-glove and a speech synthesizer. IEEE Transactions on Neural Networks 4(1):2-8. pdf
  • Hochreiter S, Schmidhuber J (1997), Long Short-term Memory. Neural Computation 9:1735-1780. pdf
Week 7 (March 7) Paper to be presented / discussed in hour 3
  • Presentation 6
    • Presenters: Shuo and Jimmy
    • Pereira, Sergio, et al. "Brain Tumor Segmentation Using Convolutional Neural Networks in MRI Images." IEEE Transactions on Medical Imaging 35.5 (2016):1240-1251. link to article in McMaster library system
Week 8 (March 14) Paper to be presented / discussed in hour 3
  • Presentation 7
    • Presenter: Aghigh
    • Cook and Hall (2017), Macroeconomic Indicator Forecasting with Deep Neural Networks pdf
Week 9 (March 21) Paper to be presented / discussed in hour 3
  • Presentation 8
    • Presenter: Jee Su
    • Pezoulas, Vasileios C, et al. "A Long Short-Term Memory deep learning network for the prediction of epileptic seizures using EEG signals." Computers in biology and medicine 99(2018):24-37. link to article in McMaster library system

Week 10. March 28. Neural networks and machine learning models for data analysis

Link to lecture slides (pdf)
Week 10 Primary readings
  • Crisci C, Ghattas B & Perera G (2012). A review of supervised machine learning algorithms and their applications to ecological data. Ecological Modelling 240:113-122.
Week 10 Supplementary readings
  • Schwenker F & Trentin E (2014). Pattern classification and clustering: a review of partially supervised learning approaches. Pattern Recognition Letters 37:4-14.
Week 10 (March 28) Paper to be presented in hour 3
  • Presentation 9
    • Presenters: Francis and David
    • Sturm I, Bach S, Samek W & Müller KR (2016). Interpretable Deep Neural Networks for Single-Trial EEG Classification. arXiv preprint https://arxiv.org/abs/1604.08201

Week 11. April 4. Decoding mental states and representing neural data

Link to lecture slides for week 11(pdf)
Week 11 Primary reading
Week 11 Supplementary reading
  • Nishimoto S, Vu TA, Naselaris T, Benjamini Y, Yu B & Gallant JL (2011). Reconstructing Visual Experiences from Brain Activity Evoked by Natural Movies. Current Biology 21:1641-1646
Week 11 (April 4) Paper to be presented in hour 3:
  • Presentation 10
    • Presenters: April and Karishini
    • Schirrmeister, R. T., Springenberg, J. T., Fiederer, L. D. J., Glasstetter, M., Eggensperger, K., Tangermann, M., ... & Ball, T. (2017). Deep learning with convolutional neural networks for EEG decoding and visualization. Human brain mapping, 38(11), 5391-5420.

Week 12. April 11. Brain-computer Interfacing

Link to lecture slides (pdf)
Week 12 Primary Reading
  • Wolpaw JR, Birbaumer N, McFarland DJ & Pfurtscheller G & Vaughan TM (2002), Brain-computer interfaces for communication and control. Clinical neurophysiology 113(6):767-91. Link to article in McMaster e-resources
Week 12 Supplementary Readings
  • Nicolas-Alonso LF, Gil JG & Alonso LFN (2012) Brain computer interfaces, a review. Sensors 12(2):1211-79.
  • Hinterberger T, Veit R, Wilhelm B, Weiskopf N, Vatine JJ & Birbaumer N (2005). Neuronal mechanisms underlying control of a brain-computer interface. The European Journal of Neuroscience, 21(11), 3169.81. doi:10.1111/j.1460-9568.2005.04092.x
    Link to article in McMaster e-resources
Week 12 (April 11) Paper to be presented in hour 3
  • Presentation 11
    • Presenters: Fatemeh and Sahand
    • Zhang H, Gua C, Ang KK & Wang C (2012). BCI competition IV . data set I: learning discriminative patterns for self-paced EEG-based motor imagery detection. Frontiers in Neuroscience 6, doi: 10.3389/fnins.2012.00007
      pdf