15-387/86-375/675 Computational Perception

Carnegie Mellon University

Fall 2023

Course Description

The perceptual capabilities of even the simplest biological organisms are far beyond what we can achieve with machines. Whether you look at sensitivity, robustness, adaptability and generalizability, perception in biology just works, and works in complex, ever changing environments, and can make inference on the most subtle sensory patterns. Is it the neural hardware? Does the brain use a fundamentally different algorithm? What can we learn from biological systems and human perception?

In this course, we will study the biological and psychological data of biological perceptual systems, mostly the visual system, in depth, and then apply computational thinking to investigate the principles and mechanisms underlying natural perception. You will learn how to reason scientifically and computationally about problems and issues in perception, how to extract the essential computational properties of those abstract ideas, and finally how to convert these into explicit mathematical models and computational algorithms. The course is targeted to both neuroscience and psychology students who are interested in learning computational thinking, as well as computer science and engineering students who are interested in learning more about the neural and computational basis of perception. Prerequisites: First year college calculus, differential equations, linear algebra, basic probability theory and statistical inference, and programming experience are desirable.

Course Information

Instructors Office Hours. Email (Phone)
Tai Sing Lee (Professor) Friday 9:00 am. Zoom Office Hour taislee@andrew.cmu.edu
Tianqin Li (TA) Monday/Tuesday 7:00-8:00 p.m. tianqinl@cs.cmu.edu
  • All Office Hours will be held on zoom, using course zoom link unless notified and arranged otherwise
  • Recommended Textbook

    Classroom Etiquette

    Grading Scheme 15-387/86-375

    EvaluationGrade Points
    Assignments 60
    Midterm 10
    Final Exam 20
    Class Participation 10

    Grading Scheme 86-675

    Assignments 60
    Midterm 10
    Final Exam * 20
    Journal Club * 3 Presentations
    Term Project * See below.
    Class participation 10


    Term Project

    Journal Club



    Date Lecture Topic Assignments
    M 8/28 1. Introduction    
    W 8/30 2. Perceputal Theories    
    M 9/4 Label Day (no class)    
    W 9/6 3. Sensors and Retina   Homework 1
    M 9/11 4. Frequency Analysis  
    W 9/13 5. Pyramid    
    M 9/18 6. Computation    
    W 9/20 7. Retinex   Homework 2
    M 9/25 8. Intrinsic Images    
    W 9/27 9. Shape and Motion    
    M 10/2 10. Visual Cortex   Mid-Course Evaluation
    W 10/4 11. Networks   Homework 3
    M 10/9 12. Contours    
    W 10/11 Midterm    
    F 10/13 Family Weekend    
    M 10/16 Fall break    
    W 10/18 Fall break    
    F 10/20 Fall break    
    M 10/23 13. Junctions   Mid-term Grade. Project Proposal due
    W 10/25 14. Organization    
    F 10/27 Community Day - No Class ?    
    M 10/39 15. Texture    
    W 11/2 16. Metamers   Homework 4 (Nov 5)
    M 11/6 17. Surfaces    
    W 11/8 18. Objects    
    M 11/13 19. Scenes    
    W 11/15 20. Synthesis   Proposal due. Homework 5 (out Nov 18)
    M 11/20 21. Integration    
    W 11/22 Thanksgiving break    
    M 11/27 22. Composition    
    W 11/29 22. Attention    
    M 12/4 23 Cognition   HW 5 due
    W 12/6 24. Review    
    S 12/X Final Exam TBA    

    Reading (relevant, but optional reading)

    Week 1 (Lectures 1 and 2) Observations, Theories and Computational Philosophy

    Week 2,3 (Lectures 3, 4, 5, 6) Retina, Frequency, Pyramid and Computation

    Week 4 (Lecture 7,8) Lightness perception and Intrinsic Images

    Week 5 (Lecture 9,10). Surfaces, Shapes and Visual Cortex

    Week 6, 7 (Lecture 11, 12, 13). Perceptual Learning and Inference

    Week 8,9 (Lectures 14, 15, 16, 17) Perceptual Organization and Segmentation

    Week 10. (Lecture 18, 19) Objects, Scenes and Inverse Graphics

    Week 11 (Lecture 21, 22) Integration and Composition

    Week 12 (Lecture 23, 24) Attention, Cognition and Consciouness

    Additional Exploration: Art and Beauty

    Questions or comments: contact Tai Sing Lee
    Last modified: August 2023, Tai Sing Lee