AS.050.375, AS.050.675

Probabilistic Models of the Visual Cortex:
 
Tues/Thurs: 9:00-10:15am Fall 2020.
 
 

---

Course Description

The course gives an introduction to computational models of the mammalian visual cortex. It covers topics in low-, mid-, and high-level vision. It briefly discusses the relevant evidence from anatomy, electrophysiology, imaging (e.g., fMRI), and psychophysics. It concentrates on mathematical modelling of these phenomena taking into account recent progress in probabilistic models of computer vision and developments in machine learning.

Important information

• Instructor: Prof. Alan Yuille, ayuille1@jhu.edu, office hours: by email appointment. TA: Yingda Xia, yxia25@jhu.edu, office hours Wed 4-5pm (starts on the second week). CA: Yihong Sun, ysun86@jhu.edu, office hours Sat 8-9pm. The zoom link will be posted online.
• We are using Zoom (Microsoft Team as backup) for lectures this semester. The link will be sent by email from the instructor and also posted on the Blackboard and Piazza. For willing students that are not yet registered, you can require access by email to Prof. Yuille.
  
• Piazza discussion site [Piazza Course Site] We encourage the students to post questions about the course and homeworks on Piazza.
  
• Grading Plan: 5 homework assignments (to be posted here) + participation points (counted as a sixth homework). Late homework will not be accepted unless permission is obtained well in advance in documented extenuating circumstances. Please email Prof. Yuille (ayuille1@jhu.edu) for approval and cc TA(yxia25@jhu.edu) and CA(ysun86@jhu.edu). Participation points are given based on in-class / Piazza questions.
  
• To encourage participation, we will split you into several groups based on the timezone. Each group can meet offline to discuss about the course material and ask questions during each lecture. Details can be found in the following Online Plan.
• Please read [The Online Plan] on details about lecture strategies, participation, homework and assistance.

Reading and Background Material

• Course notes and course readings (posted online).
  
• A.L. Yuille and D.K. Kersten. EarlyVision. Invited Book Chapter in From Neuron to Cognition. 2016. This covers much of the first part of the course.
  
• Understanding Vision: Theories, Models, and Data. Zhaoping Li. Oxford University Press. 2014. This gives more detailed coverage of some topics in the first part of the course.
  
• Note: the second part of the course is based on recent work and so there is no book reference (just class notes and readings).
• Seeing: The Computational Approach to Biological Vision.  J.P. Frisby and J.V. Stone. (2nd Ed) MIT Press. 2010. Introduces the computational approach to vision but with limited mathematics.
• Stanford tutorials, the engineering website, http://deeplearning.stanford.edu/wiki/index.php/UFLDL_Tutorial, is a good resource for students with limited statistics background, as it introduces very basic regression, dimension reduction (PCA, ICA)  and vectors.
• Stanford tutorials,  http://vision.stanford.edu/teaching/cs231n/2017/ is a good supplement for the later part of the course, it describes how convolutional network are done and has a lot of codes and examples.
• Syllabus.
  

Homeworks (Submit to Gradescope. Please check the Blackboard for the entry code.)

Homework 1 due on Sep 24 before class.
Homework 2 due on Oct 20 before class.
Homework 3 due on Nov 10 before class.
Homework 4 due on Dec 3 before class.
Homework 5 due on Dec 20 11:59 pm.

Preliminary Schedule (based on the last year's schedule and will be updated in red as the semester goes)

Lecture	Date	Topics	Handouts	Required Reading	Optional Reading
1	Sept-1	Introduction	Slides	YuilleKersten (Section 1.1, 1.2)	An extended version of the lecture:Slides
2	Sept-3	Introduction to Retina and Primary Visual Cortex (V1)	Retina V1	YuilleKersten (Section 1.3)	ReadMe Lecture by Clay Reid V1_Mike_May BialekPhotoReceptors GollischRetina VisualCrowding
3	Sept-8	Linear Filtering	Slides	YuilleKersten (Section 2.1)	KokkinosLinearFiltering CarandiniEarlyVisualSystem
4	Sept-10	Sparsity	Sparsity	YuilleKersten (Section 2.2)	SparsityPowerpoints SparsityYuilleKersten Mini-Epitomes Background redeading:BarlowSparsity1972
5	Sept-15	Filters for Binocular Stereo and Motion	Stereo Motion Figures	YuilleKersten (Section 2.4)
6	Sept-17	Hebbian Learning and Regression	Hebbian Regression	YuilleKersten (Section 2.3)	TalebiV1RF GallantNaturalStimulus ZhangCNN ZhangCNNV1Patterns
7	Sept-22	Vision as Bayesian Inference: Edges	VisionAsBayesianInference VisionAsProbabilisticInference	YuilleKersten (Section 3.1, 3.2)	chater2006probabilistic yuille2006vision
8	Sept-24	Bayes Decision Theory	BayesDecisionTheoryYuilleKersten	YuilleKersten (Section 3.1, 3.2)	YuilleLecture2UCLA
9	Sept-29	Cue Coupling Weak	Slides		Yuille&Buelthoff(1993)
10	Oct-1	Cue Coupling II	CueCouplingStrong DivisiveNormalization		ProbModelsOnGraphs
11	Oct-6	Context and Markov Random Fields	Slides	YuilleKersten (Section 4)	BeliefPropagationMFT
12	Oct-8	Context Examples	Slides	YuilleKersten (Section 4)	Same as Lecture 11
13	Oct-13	Boltzmann Machines & More Context Examples	Slides		TS Lee (2014)
14	Oct-15	Motion and Kalman Filter	Slides BarlowTripathy Burgi et al.		A198
15	Oct-20	Intro to Deep Nets	Slides FerusVittal MathDetails HintonAlexNet		YaminsNature2016 Yuille2020
	Oct-22	Fall Break
16	Oct-27	Adversarial Machine Learning	Slides PatchAttack	Jason_Yosinski ZhouFirestone
17	Oct-29	Interpretable Deep Networks	Jason_Yosinski Bolei Zhou UnsupervisedDeepNets		SmirnakisYuille1994 QiaoFewshot UnsupervisedFlow UnsupervisedNAS
18	Nov-3	The 3D world and unsupervised learning	GeometryAndMotion Motion_Geometry2020 LambertianLighting BootstrappingDeepNetCueCoupling		EveryPixelCountsChenxuLuo2018
19	Nov-5	Human/Animal Parsing	HumanAnimalParsing ParsingHumansCourse
20	Nov-10	Learning by Immagination	YouOnlyAnnotateOnce PhysicalSceneUnderstanding SimulationEngine
21	Nov-12	Compositional Models	CompositionalTheory ComplexityFundamentalProblem CompositionalModelsLearning		Generative Vision Model
22	Nov-17	Compositional Networks	AdversarialPatches CompositionalModelsOcclusion CompNetsAdamK		GeorgeCAPCHAS Detection_with_CompositionalNets Occluder_Localization_with_CompNets HongruZhuCogSci2019
23	Nov-19	Analysis by Synthesis	AnalysisBySynthesisIntro AnalysisBySynthesisDDMCMC		Image Parsing
	Nov-24	Thanksgiving
	Nov-26	Thanksgiving
24	Dec-1	Vision, Language, and Turing Tests	JunhuaMaoTextCaptioning CLEVR-Ref+
25	Dec-3	Model Robustness and Generalizability	AdversarialExaminerIntro CompositionalOpenSetActivity
26	Dec-8	Human Parsing from Static Images and Sequences	Key-Pose-Motifs