What do Images Sound Like? (due date extended to March 30, 11:59pm)

In this homework you will be sonifying image data -- essentially translating between image colors and sounds. In the first part you will implement a simple translation between hue values and sine waves. In the second part you will extend this to use sound samples.

Part 1 - Image to Sine Wave Translation

Image Processing Steps:

• Read in the image specified by filename
• Resize the image to be size 32x32 pixels
• Convert the image to HSV color space and double precision.

Sonification Steps:

• Take the H part of the HSV converted image.
• Create a mapping between hue (which will have values between 0 and 1) and frequency (you should play around with different frequency ranges and mappings until you find something that sounds reasonable). As a concrete example of what a mapping means, hue values between 0 and 0.05 could map to frequency 200, values between 0.05 and 0.1 could map to frequency 250, and so on. Try to create a mapping that sounds reasonable for a bunch of images. Include a description in your write-up of the mapping you use.
• Create an empty resultwav variable which will become your final sonified sound (resultwav= []).
• For each pixel in the 32x32 hue image, create a note with the appropriate mapped frequency and concatenate it to the end of your resultwav sound wave (your resultwav variable will essentially play each pixel's sound in sequence to create the "sound" of your image). Notes can be created as in HW2, but you might want to make their duration rather short so it doesn't take too long to play your image.
• Play the resulting wave, resultwav, and save it to a wave file using the wavwrite command.

Data:

• To test your function, download this image: colorful.jpg
• Collect at least 5 other images from the web and create their sonifications

Part 2 - Image to Samples Translation

Image Processing Steps (same as Part 1):

• Read in the image specified by filename
• Resize the image to be size 32x32 pixels
• Convert the image to HSV color space and double precision.

Modified Sonification Steps:

• Take the H (hue) part of the HSV converted image.
• Collect some sound samples (.wav files as used in HW2) from the web that you think represent different colors well. Include a description in your write-up of the samples you select (points will be given here for thought and creativity). Note, depending on the compression scheme, matlab may have trouble reading in compressed wave files so you might need to find a package to uncompress .wav files you find on the web.
• Create a mapping between hue (ranges between 0 and 1) and your samples, for example one range of hue values might map to sample1, another range to sample2, etc. Include a description in your write-up of the mapping you use.
• Create an empty resultwav variable which will become your final sonified sound (resultwav= []).
• For each pixel in the 32x32 hue image, concatenate the mapped sample to the end of your resultwav sound wave (your resultwav variable will essentially play each pixel's sound in sequence to create the "sound" of your image).
• Play the resulting wave, resultwav, and save it to a wave file using the wavwrite command.

Data:

• To test your function, download this image: colorful.jpg
• Collect at least 5 other images from the web and create their sonifications

Write-Up

• Description of how you mapped between hue values and frequencies in Part 1.
• Description of the samples you selected for Part 2 and why.
• Description of how you mapped between hue values and samples in Part 2.
• Results (wavefiles written out using the wavwrite command) on the test image colorful.jpg and on the additional images collected from the web for Parts 1 and 2.
• Description of the results. What did the images sound like? How well did it work. What might you add if you wanted to extend this idea.