CSE/ISE 364: Advanced Multimedia
Lectures: T/Th 3:50-5:10pm, Rm 2205 Computer Science
Course Webpage: http://tamaraberg.com/teaching/Spring_12/CSE364

Instructor: Tamara Berg  (tlberg -at- cs.sunysb.edu)
Office: 1411 Computer Science
Office Hours: Tuesdays/Thursdays 1:00-2:00pm and by appointment

Topics will include:
  • Text, Sound, Images, and Video
  • Retrieval
  • Morphing
  • Tagging & Annotation
  • Social Media
  • Location Information
  • Interaction with the XBox Kinect
  • Recommendation systems
This course will cover a broad range of topics related to current research in multimedia, especially with a focus on retrieval and access via the web. We will also include a module on multimodal interaction using the XBox Kinect. For each type of digital media studied we will discuss fundamentals as well as algorithms for organizing, retrieving, and manipulating media.

Tentative Schedule:

Jan 24Introduction - Slides-
Jan 26Text basics & Web document retrieval - SlidesThe Anatomy of a Search Engine
Jan 31Text modeling and Classification - Slides-
Feb 2Matlab Basics - Matlab Intro Tutorial-
Feb 7Matlab String Processing - String LabHW1 out
Feb 9Guest Lecture - Nicolas Maigret *Special Location* - Wang Center Lecture Hall 1
Feb 14Intro to Sound & Digitization - SlidesChapter 8 (first half)
Feb 16Sound Analysis - SlidesChapter 8 (second half)
Feb 21Sound Applications - Slides-
Feb 23Matlab for Sound - sound.pdfHW2 out
Feb 28Intro to Images - SlidesChapter 4 - Resolution
March 1Cameras & Color - SlidesChapter 5
March 6Image Content Analysis - Slides HW3 out
March 8Image Content Analysis and Matlab demos - imagedemos.tar.gz (demo.m contains code)-
March 13Matlab for Images - image.pdfHW4 out
March 15Image Retrieval - (see demo package from March 8, imagmageretrieval.m contains code) - Slides-
March 20Image Retrieval (cont. see slides from March 15)-
March 22Image Warping & Morphing - Slides-
March 27Project Proposals - Instructions HerePrepare 5 minute presentation
March 29Image Blending & Compositing - Slides-
April 3Spring Break-
April 5Spring Break-
April 10Words & Pictures-
April 12Pictures & Places-
April 17No Class - Traveling-
April 19Project UpdatesPrepare 5 minute presentation
April 24Interaction with Kinect-
April 26Interaction with Kinect - Kinect Demo-
May 1Final Project Presentations
Kyle, George, Eric, Aaron, John K, Tristan
Prepare 10 minute presentation
May 3Final Project Presentations
Edward&Daniel, John G, Nohoon, Louis, Andrew, Spencer, Taojie&Jin
Prepare 10 minute presentation
May 11Final Project Write-UpWrite-up of your project due

This course will focus on developing a hands on understanding of various types of multi-media. There will be 4 programming or written assignments related to the course topics. Students will also be responsible for defining and developing a project related to multi-media over the course of the semester, including a project proposal, status update, and final project presentation. A project write-up will serve as your final exam.

Students will be allowed 5 free homework late days of their choice over the semester. After those are used late homeworks will be accepted with a 10% reduction in value per day late.

Grading will consist of 55% assignments, 35% project, 10% participation.

You do not need to have taken CSE/ISE 334 (on the first day of class I will take down names and student ID numbers to waive the pre-requisite requirement). However, students are expected to be proficient in programming and the basics of digital media. Come talk to me if you have any questions!

Useful links:
Matlab tutorial by Hany Farid and Eero Simoncelli - Link
A more comprehensive Matlab tutorial by David Griffiths - Link
Matlab Answers from MIT - Link
Online Mathworks Matlab documentationLink
Lots of Matlab tutorialsLink

Digital Multimedia by Nigel Chapman and Jenny Chapman

Other useful reference books
Artificial Intelligence: A Modern Approach, Russel and Norvig.
Computer Vision: A Modern Approach, Forsyth and Ponce.
Foundations of Statistical Natural Language Processing, Christopher D. Manning, and Hinrich Schutze.

Americans with Disabilities Act: If you have a physical, psychological, medical or learning disability that may impact your course work, please contact Disability Support Services, ECC (Educational Communications Center) Building, room 128, (631) 632-6748. They will determine with you what accommodations, if any, are necessary and appropriate. All information and documentation is confidential.

Academic Integrity: Each student must pursue his or her academic goals honestly and be personally accountable for all submitted work. Representing another person's work as your own is always wrong. Faculty are required to report any suspected instances of academic dishonesty to the Academic Judiciary. Faculty in the Health Sciences Center (School of Health Technology & Management, Nursing, Social Welfare, Dental Medicine) and School of Medicine are required to follow their school-specific procedures. For more comprehensive information on academic integrity, including categories of academic dishonesty, please refer to the academic judiciary website at http://www.stonybrook.edu/uaa/academicjudiciary/

Critical Incident Management: Stony Brook University expects students to respect the rights, privileges, and property of other people. Faculty are required to report to the Office of Judicial Affairs any disruptive behavior that interrupts their ability to teach, compromises the safety of the learning environment, or inhibits students' ability to learn. Faculty in the HSC Schools and the School of Medicine are required to follow their school-specific procedures.