Code can be found here:
Lab Directory
Tarred and GZipped
PKZipped

Section 1. MIDI

3.1.1)
Hook up the MIDI cables from a keyboard synthesizer to/from the connectors on the computer. Run the program MIDITest.exe, located in \CourseDir...\Lab3. This program accepts MIDI input and displays the raw bytes and message types in a console window. It can also accept numbers from your typing and send them to the MIDI out jack of the computer. We'll use that later. Press a key on the music keyboard and verify that things are connected correctly and you're receiving MIDI. Figure out the keyboard mapping (which byte carries the note information, which notes are what, etc.). Hit one key softly, then hit the same key harder.
• Which byte carries the note number information? What's the note range of the keyboard you're using. Which byte carries the information related to how hard you hit a key?

Move the pitch and modulation wheels (on the left of the keyboard).

• What happens in the MIDI stream?

Plug pedals into the jacks on the back of the keyboard and see what MIDI messages are mapped to these controllers.

• List the message types you see and the ranges of values.

Press a key on the keyboard, holding it down. Exert pressure on the key while it's down.

• What happens?

3.1.2)
Now we'll use the MIDITest program to accept unsigned decimal bytes (0-255) from your typing into a console window and send them them to the MIDI output. You enter decimal values separated by spaces on a single line, and when you hit carriage return, the program sends them to the MIDI output. See if you can get the keyboard to make sound by sending a NoteOn message. Turn the note off by sending an appropriate NoteOff message. Note: It's very possible to confuse the MIDITest program and/or the receiver of the messages. Confine your messages to one, two, and three byte messages, and try to make sure they're correct before sending. If things get confused, you might have to rerun MIDITest or power cycle the MIDI synthesizer. Turn on a note, then send a pitch bend message (you'll have to look this up somewhere, Input Devices Web Lecture Notes by Cook and the links from there might be a good place to start. You could also repeat section 3.1.1 noting the message format when you wiggle the pitch bend wheel).
• What happens? Determine which number(s) do what in the pitch bend message.

3.1.3)
Look at the output of at least one other MIDI device in your lab (if there are any more) using the MIDITest.exe program. See what's different about the capabilities of this device from the first keyboard you used.
• List different messages here.

Stop the program MIDITest by entering q with a carriage return.

Section 2. Sound Synthesis

To do some of these exercises, you should open a DOS shell and type commands into it (you can also run some of the programs from the 'Run' command in the Startup Menu). Once you have a DOS shell, cd to the Lab3 directory. Princeton folks remember that you must Map Network Drive to a drive letter in order to use cd commands.

3.2.1)

Run the example script TestSynt.bat and listen to the sound examples. These are all being synthesized in real time on the Pentium. Move a window around while the synthesis is taking place. Launch and terminate a program like MATLAB while the synthesis is taking place. If you have a program open with scrollbars (like Netscape if you're reading this lab from it), do some scrolling while the synthesis is taking place.
• What happens to the sound when you move a window or launch a program? Why is this? How might you fix this?

Look at one or two scorefiles,

• Note the similaries and differences from MIDI.

Copy and edit one of the scorefiles, by copying and pasting and/or deleting some lines. Change some of the numbers, but not by a lot. The first numerical argument on each line is the deltaTime, and other arguments vary according to the message type. After you've changed a scorefile, call MonoScor.exe with an instrument of your choice and your newly edited scorefile. The calling convention for this is

MonoScor [Instrument] [Score]

So the first call used in the TestSynt script is:

MonoScor Plucked scores\duelingb.ski

Note: the programs work best if you cd to the directory where the executable resides, so there are two options for making and saving changes.

either: Save your edited score in a directory somewhere other than the Lab3 directory (c:\temp is a good place). Then you must type a complete path to your edited score.

or: Copy the whole Lab3 directory to your home directory. Then you can hack and destroy at will.

3.2.2)
Run MONOMIDI.exe, giving it an instrument of your choice when it prompts you. If MIDI is still hooked up correctly, you should be able to play the instrument from the MIDI keyboard (make sure cables are connected correctly. If necessary, use MIDITest.exe to "scope" the MIDI input from the keyboard and make sure MIDI is being received). Based on what you observed in 3.2.1 where host-based synthesis competed with other applications, you might want to stop any other major programs you have running on the PC. Try to determine the latency. Devise your own means of measuring this.

(Some suggestions: you could use another computer's input to simultaneously record the sound of your pressing a key and of the synthesized sound, using the Mic input to record the sound of you whacking the key, and using the Line Input to record the computer synthesized tone. Or, you could strike the key repeatedly, trying to synchronize your strikes with the delayed sounds, and then you could count the number of strikes per second. Or, devise another test of your own).
• Note your qualitative (how bad does it feel?, etc) and quantitative results regarding latency.

Stop the MONOMIDI program by entering q with a carriage return.

3.2.3)
Connect the Radio Baton to the MIDI input and output of the computer (or use a computer already connected to a Radio Baton). Run the program BatVForm.exe. This program uses one baton to control pitch, volume, and voiced/unvoiced mixing of the voice source in the synthesis model. The other baton is used to control the center frequencies of the first three formants (vocal tract resonances). Try to find the vowels /i/ (feet), /a/ (father), and /U/ (boot).
• Note the coordinates of the three vowels (/a/ /i/ /U/) in x,y,z (in inches or centimeters, assume the upper left corner is x=0, y=0, and z=0 is the lowest stick height).
See if you can learn to "sing" a melody using la la la, or some other word. Happy Birthday is a good melody (extra credit^3 for anyone learning to sing the actual words and music to happy birthday using the radio baton).
• How successful were you at learning to sing with the Radio Baton? What might be some of the factors that make this so hard? How might you change or modify the interface to make it easier?

Stop the BatVForm program by entering q with a carriage return.

Section 3. Sound Analysis

3.3.1)
Using the program GoldWave (In the top-level directory of the CS436 partition) , record and save two short soundfiles of your own voice, one of an "ahh" vowel (as in father), and one of an "eee" vowel (as in beet). Use 8KHz, Mono, 16 bit as the format. Once you've recorded the sound, you can drag the cursor over sections you wish to cut out, and use the standard cut command to edit to the file. The left mouse button controls the left selection cursor, and the right button controls the right cursor. Cut out everything you don't want until you have only the piece you want to save left. Make sure that the files you eventually save are pretty short (between .1 and .3 seconds), and that the sound is relatively stable throughout the file. Save the files as .mat (matlab) files in the c:\temp directory.

Run MATLAB, cd to c:\temp, and type

load('ahh.mat')

ahh = wavedata

load('eee.mat')

eee = wavedata

Inspect the spectra of these vowels by typing:

ahhspect = abs(fft(ahh))

eeespect = abs(fft(eee))

plot(ahhspect(1:length(ahhspect)/2))

plot(eeespect(1:length(eeespect)/2))

• Note here the differences between your two vowel spectra. Describe how you might construct a system capable of differentiating between these two vowels.

3.3.1)
Now record two more files, one with an "sss" sound (as in sister), and the other with a "shh" sound (as in ship). Watch out for pops, taking care to not blow any air directly into the microphone. Edit your new consonant files to be pretty short, save them, use save them as matlab files as before (I think there is no need to use raw2text) , and use MATLAB to inspect the spectra of these files.
• How do the two consonant spectra differ from each other? What's different about these spectra when compared to the vowels? How might you extend your system above to differentiate between vowels and consonants?

• Based on your experience in looking at the two vowels, and also based on your pattern classification experience, what might be the problems in constructing a system which can differentiate between all common English vowels (about 15 sounds)? To differentiate between all common English phoneme sounds (about 50 sounds including vowels and consonants).

Section 4. 3D Sound Processing

3.4.1)
Use the program GoldWave to record your voice saying two short words. Record them at 22KHz, mono, 16 bits. This is a record preference in GoldWave. Short words which contain some plosive consonants are recommended, like "test", "click", "pumpernickel", etc. Save the two words as one raw data file with the .raw extension (this is an option under the Save dialog box in GoldWave). The file can be reasonably long, but 2-4 seconds is recommended. We'll be using this two-word file in a loop to investigate spatial processing of sound.

3.4.2)
Run the program PanLoop.exe. It will launch a DOS shell (unless you ran it from a shell to begin with). At the prompt for the source file, enter the complete path to your saved two-words file. You'll then hear those two words played back to you. As you enter numbers from 0.0 to 1.0 with a carriage return, the program will pan the sound of the loop using linear volume only. Listen using headphones first. Wait to put in the next number until the file has played and prompted you for the next number. Otherwise the program can get confused. If it does get confused, just control-c out of the program and rerun it.
• Make a map of the apparant location of the sound, as a function of the panning number you enter.
  

  
  (For all of these spatial experiments, it is best if one person enters numbers, and another does the listening without knowing what numbers have been entered. They can then change places and repeat the experiment.)

3.4.3)
Repeat 3.4.2 using speakers.
• What are the differences between using headphones and speakers?

Stop PanLoop by entering q with a carriage return.

3.4.4)
Run the program DelyLoop.exe. At the prompt for the source file, enter the complete path to your saved two-words file. You'll then hear your two words played back to you. As you enter numbers from -1.0 to 1.0 with a carriage return, the program will impose different delays on each sound. The numbers you are entering are calibrated in seconds, equal to left delay minus right delay, so -1.0 means that the right sound lags the left by one second, and 1.0 means the left sound lags the right sound by 1 second. 0.0 means both sounds come out simultaneously. Verify things are working like you think they should by entering delays of 0.9 and -0.9.

3.4.5)
Using very small delay increments (like increments of 0.0001 seconds or less to total no more than a few milliseconds), first using headphones, vary the delay and listen.
• Make a map of the apparant sound location, as a function of the delay difference numbers you enter. To find out more about this model, consult the ITD model in Professor Duda's 3D sound lecture notes.
• Using longer delays like 0.001 second steps, try to determine a minimum perceivable delay. As with all perceptual topics, this might have multiple meanings. There might be a couple of perceptual thresholds you can detect, like the point of very short delay at which you can tell there's something different between the two sounds, and the point when you begin to hear a distinct echo-like sound. Make note of any such thresholds you can find and measure.

3.4.6)
Repeat 3.4.5 using speakers.
• What are the differences between using headphones and speakers?

Stop DelyLoop by entering q with a carriage return.

3.4.7)
Run the program FiltLoop.exe. At the prompt for the source file, enter the complete path to your saved two-words file. You'll then hear your two words played back to you. As you enter numbers with a carriage return, the program will impose different filtering operations on each sound. The numbers can be anything, but they are used as cosine arguments, so +/- PI is the only thing that makes sense. To find out more about the model being used here, consult the IID model in Professor Duda's 3D sound lecture notes.

Verify things are working by typing in a few numbers.

3.4.8)
Using small filter increments, first using headphones, vary the filter settings and listen. Vary by about 0.05 or so per step (remember these are in radians).
• Make a map of the apparant sound location, as a function of the filter control parameters you enter.

3.4.9)
Repeat 3.4.8 using speakers.
• What are the differences between using headphones and speakers?

Stop FiltLoop by entering q with a carriage return.