What do Images Sound Like? (due date extended to March 30, 11:59pm)
In this homework you will be sonifying image data -- essentially translating
between image colors and sounds. In the first part you will implement a simple
translation between hue values and sine waves. In the second part you will extend this
to use sound samples.
Part 1 - Image to Sine Wave Translation
Implement this function, sonifySine.m, which takes
as input an image file name (filename), and returns a sine wave sonified
version of the input image (resultwav). The function should also write out a
resulting wave file using the wavwrite command.
Image Processing Steps:
- Read in the image specified by filename
- Resize the image to be size 32x32 pixels
- Convert the image to HSV color space and double precision.
Sonification Steps:
- Take the H part of the HSV converted image.
- Create a mapping between hue (which will have values between 0 and 1) and
frequency (you should play around with different frequency ranges and mappings
until you find something that sounds reasonable). As a concrete example of what
a mapping means, hue values between 0 and 0.05 could map to frequency 200,
values between 0.05 and 0.1 could map to frequency 250, and so on. Try to
create a mapping that sounds reasonable for a bunch of images. Include a
description in your write-up of the mapping you use.
- Create an empty resultwav variable which will become your final sonified
sound (resultwav= []).
- For each pixel in the 32x32 hue image, create a note with the appropriate
mapped frequency and concatenate it to the end of your resultwav sound wave
(your resultwav variable will essentially play each pixel's sound in sequence
to create the "sound" of your image). Notes can be created as in HW2, but you might
want to make their duration rather short so it doesn't take too long to play your image.
- Play the resulting wave, resultwav, and save it to a wave file using the
wavwrite command.
Data:
- To test your function, download this image: colorful.jpg
- Collect at least 5 other images from the web and create their sonifications
Part 2 - Image to Samples Translation
Implement this function, sonifySamples.m, which takes
as input an image file name (filename), and returns a sample sonified version of the
input image (resultwav). The function should also write out a resulting wave file using the
wavwrite command. This part will essentially be the same algorithm as above, except
instead of sonifying using sine waves, you will translate between hue values and
imported sound samples.
Image Processing Steps (same as Part 1):
- Read in the image specified by filename
- Resize the image to be size 32x32 pixels
- Convert the image to HSV color space and double precision.
Modified Sonification Steps:
- Take the H (hue) part of the HSV converted image.
- Collect some sound samples (.wav files as used in HW2) from the web that
you think represent different colors well. Include a description in your
write-up of the samples you select (points will be given here for thought and
creativity). Note, depending on the compression scheme, matlab may have trouble
reading in compressed wave files so you might need to find a package to
uncompress .wav files you find on the web.
- Create a mapping between hue (ranges between 0 and 1) and your samples, for
example one range of hue values might map to sample1, another range to sample2, etc.
Include a description in your write-up of the mapping you use.
- Create an empty resultwav variable which will become your final sonified
sound (resultwav= []).
- For each pixel in the 32x32 hue image, concatenate the mapped sample to the
end of your resultwav sound wave (your resultwav variable will essentially play
each pixel's sound in sequence to create the "sound" of your image).
- Play the resulting wave, resultwav, and save it to a wave file using the
wavwrite command.
Data:
- To test your function, download this image: colorful.jpg
- Collect at least 5 other images from the web and create their sonifications
Write-Up
Provide a homework web page or pdf document, including:
- Description of how you mapped between hue values and frequencies in Part 1.
- Description of the samples you selected for Part 2 and why.
- Description of how you mapped between hue values and samples in Part 2.
- Results (wavefiles written out using the wavwrite command) on the test image colorful.jpg and on the additional images collected from the web for Parts 1 and 2.
- Description of the results. What did the images sound like? How well did it work. What might you add if you wanted to extend this idea.
Email your webpage/document and commented code to cseise364@gmail.com.