Course Description: How can we use computers to recognize objects, people, actions, animals, places, etc from images? This seemingly trivial task that people perform without much effort has remained one of the core problems in Computer Vision. Recent advances in representation learning using multiple layers of abstraction (deep learning) have demonstrated to be an important aspect for designing artificial systems for visual recognition. In this class we will study, conceive, and implement deep learning models and learning algorithms for computational visual recognition. After this class you will be able to understand, design, implement, and assess the impact of deep learning techniques for a diverse range of visual recognition tasks.
Learning Objectives: (a) Develop intuitions between aspects in human vision and computer vision, (b) Understanding foundational concepts for representation learning using neural networks, (c) Become familiar with state-of-the-art models for tasks such as image classification, object detection, image segmentation, scene recognition, etc, and (d) Obtain practical experience in the implementation of visual recognition models using deep learning.
Prerrequisites: This course requires no previous background in computer vision or machine learning but knowledge in either of those will be helpful. You need to know about matrices, calculating derivatives, and probabilities (bayes rule). You will also need to be at least a moderately proficient programmer in python. There will be several lab assignments. These assignments will show you the basics of modern general visual recognition algorithms and models, and will give you the tools for implementing more advanced ones. Finally, we will have a class project where you will be able to work on something beyond your assignments and where you will have more freedom to pursue a focused problem that is of your interest and better matches your background. Finally we will be using python/pytorch in the lecture notes, so being proficient in Python by completing a few projects in this language before the class starts is helpful. You should install python, jupyter, and pytorch, and complete the following notebook [pytorch_tensors].
|Mon, January 13th||Introduction to Visual Recognition [pptx] [pdf] + Primer on Image Processing [link]|
|Assignment on Image Processing and Manipulation [Colab]. Due January 26th 5pm EST.|
|Wed, January 15th||Image Processing and Image Manipulations [pptx] [pdf]|
|Mon, January 20th||MLK Holiday -- no class this day|
|Assignment on Image Classification [Colab]. Due February 3rd 11:59pm EST.|
|Wed, January 22nd||Softmax Classifier + Stochastic Gradient Descent [pptx] [pdf]|
|Mon, January 27th||Shallow Image Features and the Bag of Features model [pptx] [pdf]|
|Assignment on Deep Learning Basics [Colab]. Due February 10th 11:59pm EST.|
|Wed, January 29th||Neural Networks and the Multi-layer Perceptron Model [pptx] [pdf]|
|Mon, February 3rd||Convolutional Neural Networks (CNNs) [pptx] [pdf]|
|Assignment on Convolutional Neural Networks [Colab]. Due February 24th 11:59pm EST.|
|Wed, February 5th|
|Mon, February 10th||Convolutional Neural Network Architectures: LeNet, AlexNet, VGG, GoogLeNet, ResNet [pptx] [pdf]|
|Wed, February 12th||Deep Learning-based Object Detection [pptx] [pdf]|
|Mon, February 17th||Deep Learning-based Semantic Image Segmentation [pptx] [pdf]|
|Wed, February 19th||Generative Adversarial Networks (GANs) [pptx] [pdf]|
|Mon, February 24th||
Paper Review: CNNs as Features for Transfer Learning
|Wed, February 26th||Recurrent Neural Networks (RNNs) [pptx] [pdf]|
|Mon, March 2nd||Paper Review: Face Recognition and Pose Estimation|
|Wed, March 4th||Paper Review: Recent Methods for Object Detection and Instance Segmentation.|
|Mon, March 9th||Spring recess -- no class this day|
|Wed, March 11th||Spring recess -- no class this day|
|Mon, March 16th||Extended Spring recess due to COVID-19 -- no class this day -- stay safe!|
|Wed, March 18th||Extended Spring recess due to COVID-19 -- no class this day -- stay safe!|
|Mon, March 23rd||
Paper Review: Interpreting and Explaining Deep Neural Networks
|Wed, March 25th|| Paper Review: Image to Text: Image Captioning
|Mon, March 30th|| Paper Review: Structured Prediction with Partial Labels + Efficient NNs I
|Wed, April 1st||
Paper Review: Conditional Generative Adversarial Networks (GANs)
|Mon, April 6th||
Paper Review: Avoiding Visual Bias in Computer Vision
|Wed, April 8th||
Paper Review: Video Recognition + Efficient NNs II
|Mon, April 13th|| Paper Review: Transformer Networks
|Wed, April 15th|| Paper Review: Self-supervised Learning
|Mon, April 20th|| Paper Review: Colorization and Super-resolution
|Wed, April 22nd|| Paper Review: Neural Architecture Design and Search
|Mon, April 27th||Course Re-cap and Good Bye! [slides]|
Disclaimer: The professor reserves to right to make changes to the syllabus, including assignment due dates. These changes will be announced as early as possible.
Grading: ** Due to the COVID-19 state of emergency we have exceptionally changed the grading scheme** to use by default the following distribution: Assignments: 600pts (4 assignments: 150pts + 150pts + 150pts + 150pts), Class Project: 200pts, Reading Summaries: 100pts, Class Presentation: 100pts. Letter grades to be decided as follows: A+ (950pts), A (850pts), A- (800pts), B+ (770pts), B (750pts), B- (730pts), C+ (700pts), C (670pts), C- (650pts), D+ (630pts), D (600pts), D- (570pts).
Note: The old grading scheme will be applied if this leads to a more favorable grade for the student: Assignments: 400pts (4 assignments: 100pts + 100pts + 100pts + 100pts), Class Project: 400pts, Reading Summaries: 100pts, Class Presentation: 100pts. Letter grades to be decided as follows: A+ (1000pts), A (930pts), A- (900pts), B+ (870pts), B (830pts), B- (800pts), C+ (770pts), C (730pts), C- (700pts), D+ (670pts), D (630pts), D- (600pts).
Late Submission Policy: No late assignments will be accepted in this class. Unless the student has procured special accommodations for this class.
Academic Integrity Statement: "The School of Engineering and Applied Science relies upon and cherishes its community of trust. We firmly endorse, uphold, and embrace the University’s Honor principle that students will not lie, cheat, or steal, nor shall they tolerate those who do. We recognize that even one honor infraction can destroy an exemplary reputation that has taken years to build. Acting in a manner consistent with the principles of honor will benefit every member of the community both while enrolled in the Engineering School and in the future. Students are expected to be familiar with the university honor code, including the section on academic fraud."
Accessibility Statement: "The University of Virginia strives to provide accessibility to all students. If you require an accommodation to fully access this course, please contact the Student Disability Access Center (SDAC) at (434) 243-5180 or firstname.lastname@example.org. If you are unsure if you require an accommodation, or to learn more about their services, you may contact the SDAC at the number above or by visiting their website at https://www.studenthealth.virginia.edu/student-disability-access-center/about-sdac."