CSE/ISE 364: Advanced Multimedia
Lectures: T/Th 3:50-5:10pm, Rm 2205 Computer Science
Course Webpage: http://tamaraberg.com/teaching/Spring_10/

Instructor: Tamara Berg  (tlberg -at- cs.sunysb.edu)
Office: 1411 Computer Science
Office Hours: Tuesdays/Thursdays 5:10-6:10pm and by appointment



Topics will include:
  • Text
  • Sound
  • Images
  • Video
  • Tagging & Annotation
  • Social Media
  • Location Information
  • Speech Processing
  • Recommendation systems
The focus of this course will be on multimedia as accessed via the web. For each type of digital media studied we will discuss fundamentals including storage, compression, and use. We will also study various algorithms for organizing, retrieving, and manipulating digital media.

Assignments
Assignment 1 Due Feb 18
Assignment 2 Due March 2
Assignment 3 Due March 11


Schedule:
DateTopicReadings
Jan 26Introduction - Slides-
Jan 28Text basics & Web document retrieval- SlidesThe Anatomy of a Search Engine
Feb 2Text modeling and classification - Slides, Demo Code-
Feb 4Using Matlab for multimedia - Tutorial (from Clarkson University), String Manipulation Exercise (from David Griffiths) An Extended Matlab Reference
Assignment 1 out
Feb 9Useful Matlab functionality - Some string code: text.m-
Feb 11Document clustering - Slides
Feb 16Intro to sound & digitization - Slides, Demo1, Demo2Chapter 6 of Textbook
Feb 18Sound analysis - SlidesAssignment 2 out
Feb 23Sound analysis & applications - See slides for Feb 18 - Filtering Demo"Content-Based Music Information Retrieval"
Feb 25Guest Lecture - Margaret Schedel - Digital Music-
March 2Intro to Computer Vision - Slides-
March 4Images, cameras, & color - SlidesAssignment 3 out
Chapter 4 of textbook
March 9Image Content Analysis - Slides, Demos-
March 11Image Blending & CompositingA Multiresolution Spline With Application to Image Mosaics, Peter J. Burt and Edward H. Adelson
March 16Image Warping & MorphingAssignment 4 out
March 18Guest Lecture - Klaus Mueller - Visualization-
March 23Image retrieval-
March 25Videos and TrackingChapter 5 of textbook
March 30Spring Break-
April 2Spring Break-


Grading:
 This course will focus on developing a hands on understanding of various types of multi-media. There will be 4 programming assignments related to the course topics. Students will also be responsible for defining and developing a project related to multi-media over the course of the semester, including a project proposal, status update, and final project presentation. A project write-up will serve as your final exam.

Late homeworks will be accepted with a 10% reduction in value per day late.

Grading will consist of 55% assignments, 35% project, 10% participation.

Prerequisites:
 You do not need to have taken CSE/ISE 334. Students are expected to be proficient in programming and the basics of digital media, but I will provide most necessary background for the course as we go. Come talk to me if you have any questions!


Useful links:
Matlab
 Matlab tutorial by Hany Farid and Eero Simoncelli - Link
A more comprehensive Matlab tutorial by David Griffiths - Link
Matlab Answers from MIT - Link
Online Mathworks Matlab documentationLink
Lots of Matlab tutorialsLink

Book
 Fundamentals of Multimedia, Ze-Nian Li, and Mark S. Drew.

Other useful reference books
 Artificial Intelligence: A Modern Approach, Russel and Norvig.
Computer Vision: A Modern Approach, Forsyth and Ponce.
Foundations of Statistical Natural Language Processing, Christopher D. Manning, and Hinrich Schutze.