Archive for the 'Thesis' Category

Final Lap Thesis

Thursday, May 1st, 2008

It’s week Last Lap before I present my thesis to the world. After meeting with Agnieszka Roginska from NYU’s Steinhardt Music & Technology division, who specializes in 3D sound & localization, it was clear that I had to switch gears very late in the game, and right before my last class presentation before the critics. The crits were…
- Red Burns, Chair, ITP
- Bob Greenberg, CEO, R/GA
- Allegra Burnette, Director of Digital Media, MoMA
- Lauren Cornell, Executive Director, Rhizome
- Jake Barton, Principal, www.localprojects.net and my class pairs.

Some of the crits I received was expected because my visuals were still in development, and encouraging because they saw value in aspects I took for granted. One of the comments I received was the meaning of the colors in my visuals with respect to sound information. I am now mapping the data derived from frequency and amplitude only. Other comments I received was interest in my process of collecting the sounds at Union Square where I sat there for 17th hours. They here interested in listening to what I saw, felt, what was different verses the norm. I goes I tell a good story. Lauren even referred to me as an artists. But the meat of my process was the sound and what I can get from it. Before I was trying to extract localized information ( X, Y, Z coordinates based on recorded source ) from a binaural source recording and then visualize its position in the 3D plane. This cannot be achieved at this point in time due to the complexity of sound itself and our perception vs a computer algorithm. Here’s a break down of what I mean.

The sound we hear is influenced my many factors. First the location of sound relative to our ears is calculated in our brains after it’s received by each of our Auricles (Pinna) or the outer ear. This sound ( vibration) is trapped & funneled to the inner ear. This sound vibration is then converted into information filtered by the inner ear ( incus, Malleus, Semi-Circular Canals, Stapes, Cochlea ) and then transfered electrically to the brain via the auditory and vestibular nerves. <- Hope those were simple enough description.

Here's an image of the ear

But other factors help determine the localization (direction) of sound.
The distance between each ear affects the perceived arrival of the sound. Does it get to the left ear first or right ear? Does it sounds far away or very close?
Also, familiarity to sounds and visual cues help determine if the sound lives in ones relative space or a projection. What I mean here is that if we here a dog barking and it sounds really close, but your visual scope reveals there’s nothing within the space to verify that there is a dog in the immediate space, then it’s probably manufactured by an external source. Maybe a recording of some sort.
In terms of direction if we here a plane going by and we are in the middle of the city we know due to conditioning, familiarity, and process of deduction that the sound is coming from above - somewhere. It’s very difficult to perceive height distance.
Sound also changes tone over time reflected in distance. For instance we are all familiar with the Doppler effect named after Christian Doppler. This where a sound changes it’s frequency and wavelength due to changing distance relative to the listener. Think of a car horn which indicates that the car is coming towards you, it’s then at your location, then it away in the opposite direction. The original sound source, the horn, did not change. Only the conditions in which the sound has to travel relative to the listener at an inconstant distance compounded by the reflection of sound off the surface of it’s surroundings.

Car horn sound:

As you can hear and imagine there’s alot that needs to be calculated, memorized and recalled in order for us to go about our day with certainty. At least with respect to sound for those who aren’t hearing impaired. But let’s add to this, the complexity of many sounds happening at once and changing rapidly. Imagine yourself in a crowded room with a friend and everyone’s is talking. Think the bar. But you are able to hear your friend with some clarity. This process is called selective and adaptive hearing. Selective in that we focus our hearing on what’s meaningful at the time - the conversation, thus filtering out, or just turning down the volume on the other sounds around us. Adaptive meaning the sounds that does not add any meaning at all is ignored or masked out. For instance, the low hum of a generator barely audible but annoying the least if focused upon. We do this in order to secure our safety and quality of our everyday conduct. Of course we may not have control over very loud sounds or frequency irritating sounds. Think of the screeching of brakes from cars of the trains - that really urkes me. In that case we either change our environment, cover our ears or find some other distraction if applicable.

Screeching tires sound:

Inside Bar sound:

Free Sound Effects Download thousands of free sound effects from PartnersInRhyme.com

How would one create a computer algorithm to compensate for all these factors? Forget algorithm how would one construct this process for a machine? Then there’s the nostalgia of a place which provokes a certain feeling which can be collectively agreed upon or just indiviualized based on personal reflection.

As you can see those were some of the challenges I am facing. How can I re-represent those facets of sound visually if I cannot effectively extract some of the basic components? And I am also employing the use of a Binaural recording source - two microphones place on opposite sides of the head. Improbable but not impossible. I do believe that research & technology would continue to take us into new directions and deliver new findings both imaginable and inconceivable. Who ever thought that man would fly clone an animal?

My thesis will be an ongoing addition to those who are interested and researching the the interest of sound localization.

Generated Visuals to come…

Sound and the City:
Binaural Recordings at Union Square Park

Thursday, March 27th, 2008

In pursuit of gathering sounds for my thesis one suggested task was to record sounds from one location in NYC. I chose Union Square because it is a familiar site for me.
Side note: Soon after 911 when there was still a solemn mood in the air I was out with my friend in this part of the city. We were off from work, drinking, and we did not want to call it a night. We then decided to sit and spend the night sitting in Union Square Park. It was the Summer and everyone was out that night; the skaters, couples, dog owners, the man with the Hawk, kids playing, everyone… It was really interesting to view the transition of people, all types of people, different animals that grace the night, and the awakening of the next day. We were there from 11-ish pm till 6:00 am the next day. What was particularly interesting at that time for me was the Rats that dominated the park in the wee hours of the morning. That’s one nocturnal part of the of the life of the city. Then in the morning seeing all the television news vans gather around the south side of the square. I was thinking that they were sharing news and plans for the day. No wonder all the news is the same.

Anyway, almost seven years later here I am this time recording sounds for my thesis. The plan, to record 10 minutes of audio every hour on the hour for 24 hours. It was cold that evening, approximately 34 degrees F. I was dressed the the night ready for this challenge. I started at 6:00 pm, a few hours later than I had planed. I was equipped with my binaural headphones, a stealth way of recording, my zoom recorder in my pocket, and my camera in hand. I chose my spot on a bench , a strategic place to get an optimal stereo recording maximizing differences between the fields. Here is a photo and below is the Time-stamp of my session.

Wednesday March 26 2008.

[ 5:55 - 6:11 PM ]

Its light out
Church bell rings from a distance. Orientation hard to decipher
Girl passes by on phone - walk by right to left
Rush hour traffic to my right at a distance - approx 13 meters
Street brass band playing in a distance to my front right - approx 20 meters away
Skateboarders to my right about - approx 6 meters
Dogs running on gravel in doggy park behind me - right close to left far
Dog chain rattling behind me
Helicopter in sky - height undetermined
People chatting all around walking by

[ 6:55 - 7:11 PM ]

Still light out
Church bell rings…
Skateboarder still practicing
Friends talking bench to my left telling recipes into phone - approx 1 meter
Echo of ambulance siren. Moving so orientation is hard to decipher
Group of guys practice fighting
Traffic is getting quieter

[ 7:55 - 8:11 PM ]

Starting to get a bit dark
Church bell rings…
Street brass band playing in a distance to my front right…
Less people walking by, less activity

[ 8:55 - 9:11 PM ]

Church bell rings…
Men talking on bench to my right - approx 1.5 meters
Plane overhead…
Guy to my left talking on phone - approx 1 meter
Plane overhead…
Higher frequencies more prevalent

[ 9:55 - 10:11 PM ]

No church bell!
Two people on bench having business meeting on phone to my left on bench - 1 meter
Lots of horns honking
More lively than pas hour with people walking by
Plane overhead

[ 10:55 - 11:11 PM ]

Sounds of garbage trucks
Helicopter high up
Park ranger is putting chains in position for park closure
Police are walking/patroling the park
Homeless guy sits to my right - 1.5 meters
Same traffic from two hours away

[ 11:55 PM - 12:11 AM ]

Mid Traffic at midnight - 13 meters away
Less people walking by due to park closure
Getting colder sitting on bench

[ 12:55 - 1:11 AM ]

Getting Quieter
Dark barking in park behind me - apparently its owner left it alone in the doggy park
Saw one rat - vast difference from last time I spent in the park

[ 11:55 - 2:11 AM ]

Changed location - was asked by park anger to leave park bench area
MTA cleaning crew on scene - loud generator sound to my right but sound is traveling omnidirectional
Dog still barking - now attracting several police officers and park ranger
Police pulls over minivan at a distance in from of me - 13 meters away
Police shoots dog with tranquilizer gun - very tragic but I guess necessary.

[ 2:55 - 3:11 AM ]

Much quieter and colder now.
Very little activity with people.
Still few cars on street - city is resting but not sleeping
MTA cleaning crew still on the job

[ 3:55 - 4:11 AM ]
[audio4am.mp3]

Much quieter and colder now.
Very little activity with people.
Still few cars on street - city is resting but not sleeping
MTA cleaning crew still on the job

[ 4:55 - 5:11 AM ]

Birds chirping indicating dawn
Light traffic flowing freely
Morning Busses in the background

[ 5:55 - 6:11 AM ]

Getting lighter out
Birds still chirping…
Light traffic flowing freely
Morning Busses in the background

[ 6:55 - 7:11 AM ]

Getting brighter
Birds still chirping…
Traffic is slightly lighter than before
Morning Busses in the background

[ 7:55 - 8:11 AM ]

Much lighter now
More birds chirping
Traffic still light
Busses in the background
Man performing Tai-Chi
Park cleaning crew picking up garbage
Doggy park open and visitors arriving
Dogs barking
Sounds of traffic getting a little louder

[ 8:55 - 9:11 AM ]

Church bell rings
Birds chirping
Squirrel running around
Construction behind me begins - approx 10 meters
Park garbage van in area

[ 9:55 - 10:11 AM ]

Church bell rings
Traffic slightly milder
More construction emanating form far
Over cast

[ 10:55 - 11:11 AM ]

Church bell rings
Traffic slightly milder
More people activity
Gloomy and grey day
Starting to drizzle

As you may have guessed I did not complete the 24 hours. It started to drizzle and I needed to get the equipment away from potential damage. Not to mention I was really tired in need of sleep. I do think it was enough to graph a moments of the park over a period…

Here are my accounts from that night minus the list.
Being there by myself, not speaking to anyone made for a very lonely night. I reflected on how important it is to have familiar people around you. It provides some sort of psychological warmth.
It was excessively noisy at night, with the cleaning crew doing there work. At times it was louder than the day, at least where I was sitting.
They closed the park, something I was not aware of. Duh!
I was expecting to see the rats frolic in the absence of people. I saw only one. I later learned that during the Winter months many rodents do not survive during periods of cold and snow.

See - Bureau of Community Hygiene:
http://hrla.doh.dc.gov/hrla/cwp/view,a,1384,q,572855.asp

All and all it was a success and my binaural recordings totally present a stereo and timed picture.
My next step is to extract the stereoscopic data, i.e. Frequency, Amplitude an Phase and illustrate it in 3D (Three Dimensions, i.e [ X, Y, Z axis ]).

Check out 3D Audio Spatialization Method:

Also see 2D v.s. 3D