CSE/ISE 364: Advanced Multimedia
Lectures: T/Th 2:20-3:40pm, Rm 2205 Computer Science
Course Webpage: http://tamaraberg.com/teaching/Spring_11/CSE364

Instructor: Tamara Berg  (tlberg -at- cs.sunysb.edu)
Office: 1411 Computer Science
Office Hours: Tuesdays/Thursdays 3:40-5:10pm

Topics will include:
  • Text, Sound, Images, and Video
  • Retrieval
  • Morphing
  • Multi-modal Interaction
  • Tagging & Annotation
  • Social Media
  • Location Information
  • Recommendation systems
This course will cover a broad range of topics related to current research in multimedia, especially with a focus on retrieval and access via the web. This year we will also include a new module on multimodal interaction using the XBox Kinect. For each type of digital media studied we will discuss fundamentals as well as algorithms for organizing, retrieving, and manipulating media.

Homework 1 (due Feb 24 - deadline extended to Feb 27) - updated classify.m on Feb 22. Submit by email to cseise364@gmail.com
Homework 2 (due March 10). Submit by email to cseise364@gmail.com
Homework 3 (due March 22). Submit by email to cseise364@gmail.com
Homework 4 (due April 5). Submit by email to cseise364@gmail.com

Tentative Schedule:

Feb 1Introduction - Slides-
Feb 3Text basics & Web document retrieval - SlidesThe Anatomy of a Search Engine
Feb 8Text modeling and classification - Slides-
Feb 10Text modeling and classification (cont)-
Feb 15Getting started with Matlab - Tutorial from Clarkson University, String ProcessingHW1 out
Feb 17Matlab for text processing (Text Demo), Document clustering - Slides-
Feb 22Intro to sound & digitization - SlidesChapter 6 of Textbook
Feb 24Sound analysis - Slides -
March 1Using Matlab for sound - DemosHW2 out
March 3Sound applications - Slides"Content-Based Music Information Retrieval"
March 8Guest Lecture - Margaret Schedel - Music-
March 10Intro to Images - SlidesChapter 4 of textbook
March 15Cameras, & color - Slides
HW3 out
March 17Image Content Analysis - Slides-
March 22Matlab for Images - Demos (code in demo.m), imageRep.tar.gz-
March 24Image Warping & Morphing - SlidesHW4 out
March 29Image Blending & Compositing - SlidesA Multiresolution Spline With Application to Image Mosaics, Peter J. Burt and Edward H. Adelson
March 31Image retrieval - SlidesChapter 18 of textbook. Please come to office hours to discuss project ideas.
April 5Words & Pictures - SlidesPlease come to office hours to discuss project ideas
April 7Project Proposal PresentationsPlease prepare a 5 minute presentation describing your project proposal.
April 12Guest Lecture - Klaus Mueller (Visualization)-
April 14Meta data, Tagging, and Location Information - Slides -
April 19Spring Break-
April 21Spring Break-
April 26Project Update PresentationsPlease prepare a 5 minute presentation describing project progress so far
April 28Project Help dayPlease bring project questions for discussion
May 3Recommendation Systems - Slides-
May 5Interaction using the XBox Kinect - SlidesReal-Time Human Pose Recognition in Parts from a Single Depth Image
May 10Final Project PresentationsKyle+Rene
May 12Final Project PresentationsDerrick+Eric
May 20Final Project Write-Up due8 page document including abstract, introduction (motivation), method, results, & figures

This course will focus on developing a hands on understanding of various types of multi-media. There will be 4 programming or written assignments related to the course topics. Students will also be responsible for defining and developing a project related to multi-media over the course of the semester, including a project proposal, status update, and final project presentation. A project write-up will serve as your final exam.

Students will be allowed 5 free homework late days of their choice over the semester. After those are used late homeworks will be accepted with a 10% reduction in value per day late.

Grading will consist of 55% assignments, 35% project, 10% participation.

You do not need to have taken CSE/ISE 334 (on the first day of class I will take down names and student ID numbers to waive the pre-requisite requirement). Students are expected to be proficient in programming and the basics of digital media, but I will provide most necessary background for the course as we go. Come talk to me if you have any questions!

Useful links:
Matlab tutorial by Hany Farid and Eero Simoncelli - Link
A more comprehensive Matlab tutorial by David Griffiths - Link
Matlab Answers from MIT - Link
Online Mathworks Matlab documentationLink
Lots of Matlab tutorialsLink

Fundamentals of Multimedia, Ze-Nian Li, and Mark S. Drew.

Other useful reference books
Artificial Intelligence: A Modern Approach, Russel and Norvig.
Computer Vision: A Modern Approach, Forsyth and Ponce.
Foundations of Statistical Natural Language Processing, Christopher D. Manning, and Hinrich Schutze.

Americans with Disabilities Act: If you have a physical, psychological, medical or learning disability that may impact your course work, please contact Disability Support Services, ECC (Educational Communications Center) Building, room 128, (631) 632-6748. They will determine with you what accommodations, if any, are necessary and appropriate. All information and documentation is confidential.

Academic Integrity: Each student must pursue his or her academic goals honestly and be personally accountable for all submitted work. Representing another person's work as your own is always wrong. Faculty are required to report any suspected instances of academic dishonesty to the Academic Judiciary. Faculty in the Health Sciences Center (School of Health Technology & Management, Nursing, Social Welfare, Dental Medicine) and School of Medicine are required to follow their school-specific procedures. For more comprehensive information on academic integrity, including categories of academic dishonesty, please refer to the academic judiciary website at http://www.stonybrook.edu/uaa/academicjudiciary/

Critical Incident Management: Stony Brook University expects students to respect the rights, privileges, and property of other people. Faculty are required to report to the Office of Judicial Affairs any disruptive behavior that interrupts their ability to teach, compromises the safety of the learning environment, or inhibits students' ability to learn. Faculty in the HSC Schools and the School of Medicine are required to follow their school-specific procedures.