CSE/ISE 364: Advanced Multimedia
Lectures: Tues/Thurs 1:00-2:20pm, Rm 2205 Computer Science
Course Webpage: http://tamaraberg.com/teaching/Spring_13/mm/

Instructor: Tamara Berg  (tlberg -at- cs.sunysb.edu)
Office: 1411 Computer Science
Office Hours: Tuesdays/Thursdays 3:00-4:00pm and by appointment

Topics will include:
  • Text, Sound, Images, and Video
  • Retrieval
  • Morphing
  • Tagging & Annotation
  • Social Media
  • Location Information
  • Interaction with the XBox Kinect
  • Recommendation systems
This course will cover a broad range of topics related to current research in multimedia, especially with a focus on retrieval and access via the web. We will also include a module on multimodal interaction using the XBox Kinect. For each type of digital media studied we will discuss fundamentals as well as algorithms for organizing, retrieving, and manipulating media.


Tentative Schedule:

DateTopicReadings (S=Szeliski book, C=Chapman book, others -> follow link)Assignments
Jan 29Introduction - Slides--
Jan 31Text basics & the Web - Slides"The Anatomy of a Search Engine"-
Feb 5Text basics (cont)"The Anatomy of a Search Engine"-
Feb 7Text modeling and Classification - Slides
The Vector space model & Vector Space Classification (kNN)-
Feb 12Matlab Basics Lab - hereSee tutorials below under "Useful links"-
Feb 14String Processing Lab - here-HW1 out
Feb 19String Processing Lab (cont) --
Feb 21Intro to Sound & Digitization - SlidesC8 (first half)-
Feb 26Sound Analysis - SlidesC8 (second half)-
Feb 28Sound Lab - sound.pdf-HW2 out
March 5Sound Applications - Slides"Content-Based Music Information Retrieval"-
March 7Intro to Images - SlidesC4-
March 12Image Content Analysis - SlidesS4-
March 14Images Lab - Image lab-HW3 out
March 19Spring Break--
March 21Spring Break--
March 26Guest Lecture - Yejin Choi--
March 28Image Retrieval - Slides"QBIC", "PageRank for Product Image Search"-
April 2Project Proposals -Prepare a 5 minute presentation and submit a 2 paragraph summary
April 4Cameras - SlidesC5-
April 9Guest Lecture - Timothy Vallier--
April 11Blending & Compositing - Slides S3.5,S10.4-10.5-
April 16Warping & Morphing - SlidesS2.1, S6.1-
April 18Social Media & Location Data - Slides"HT06, Tagging Paper, Taxonomy, Flickr, Academic Article, ToRead", "IM2GPS: estimating geographic information from a single image"-
April 23Project Updates-Prepare a 10 minute presentation and submit a 4 paragraph summary
April 25Project Updates-Prepare a 10 minute presentation and submit a 4 paragraph summary
April 30The Kinect"Real-Time Human Pose Recognition in Parts from a Single Depth Image"-
May 2Interaction with Kinect Lab - here--
May 7Final Project Presentations 1) Edward & Richard, 2) Radhika, 3) Mindy, 4) Josh, 5) Jiahe, 6) Jung & Jaewan, 7) Craig, 8) Greg, 9) Joe & VictorPrepare a 10 minute presentation
May 9Final Project Presentations 1) Neeti & Sara, 2) David & Allen, 3) Mike & Danielle, 4) David, 5) Joseph & Andrew, 6) Mark, 7) Joe Kim, 8) Michael & JamesPrepare a 10 minute presentation
May 20Final Project Write-Up-Write-up of your project due - 8 pages include intro, description of motivation, implementation, results, figures, tables, and other visuals.

This course will focus on developing a hands on understanding of various types of multi-media. There will be 3 programming and/or written assignments related to the course topics. There will also be short in class quizzes approximately every other week. Students will also be responsible for defining and developing a project related to multi-media over the course of the semester, including a project proposal, status update, and final project presentation. A project write-up will serve as your final exam. HWs and Projects may be completed in pairs.

Students will be allowed 3 free homework late days of their choice over the semester. After those are used late homeworks will be accepted with a 10% reduction in value per day late. To accomodate for missed quizzes, students will be allowed to drop their lowest quiz grade

Grading will consist of 40% assignments, 30% project, 20% quizzes, 10% participation.

You do not need to have taken CSE/ISE 334 (on the first day of class I will take down names and student ID numbers to waive the pre-requisite requirement). However, students are expected to be proficient in programming and the basics of digital media. Come talk to me if you have any questions!

Useful links:
Matlab tutorial by Hany Farid and Eero Simoncelli - Link
A more comprehensive Matlab tutorial by David Griffiths - Link
Matlab Answers from MIT - Link
Online Mathworks Matlab documentationLink
Lots of Matlab tutorialsLink

Digital Multimedia by Nigel Chapman and Jenny Chapman (The library should have a copy. I also have 2 copies that I am happy to loan out for a few days at a time).
Computer Vision: Algorithms and Applications by Rick Szeliski (free online draft)

Other useful reference books
Artificial Intelligence: A Modern Approach, Russel and Norvig.
Computer Vision: A Modern Approach, Forsyth and Ponce.
Foundations of Statistical Natural Language Processing, Christopher D. Manning, and Hinrich Schutze.

Americans with Disabilities Act: If you have a physical, psychological, medical or learning disability that may impact your course work, please contact Disability Support Services, ECC (Educational Communications Center) Building, room 128, (631) 632-6748. They will determine with you what accommodations, if any, are necessary and appropriate. All information and documentation is confidential.

Academic Integrity: Each student must pursue his or her academic goals honestly and be personally accountable for all submitted work. Representing another person's work as your own is always wrong. Faculty are required to report any suspected instances of academic dishonesty to the Academic Judiciary. Faculty in the Health Sciences Center (School of Health Technology & Management, Nursing, Social Welfare, Dental Medicine) and School of Medicine are required to follow their school-specific procedures. For more comprehensive information on academic integrity, including categories of academic dishonesty, please refer to the academic judiciary website at http://www.stonybrook.edu/uaa/academicjudiciary/

Critical Incident Management: Stony Brook University expects students to respect the rights, privileges, and property of other people. Faculty are required to report to the Office of Judicial Affairs any disruptive behavior that interrupts their ability to teach, compromises the safety of the learning environment, or inhibits students' ability to learn. Faculty in the HSC Schools and the School of Medicine are required to follow their school-specific procedures.