Analyzing music the digital way

Ge Wang was every bit the image of an orchestra conductor. Clad all in black, with a dramatic mane of shoulder-length hair parted in the middle, he used fluid gestures to summon forth an auditorium full of sound.

Yet there were some curious differences from what you might see at the Kimmel Center. His musicians were sitting cross-legged on the stage floor, for one thing. And their instruments, linked by a wireless network, were laptop computers.

Electronic wizardry has been an instrumental part of music for decades, at least since Wendy Carlos released Switched-On Bach in 1968 and played a Brandenburg Concerto on a Moog synthesizer.

But the international crowd of engineers, musicians and computer types who gathered at Drexel University last week are pushing their trade in directions your grade-school piano teacher never imagined.

Computers are used not only to play music, as Wang and the Princeton Laptop Orchestra did on Wednesday night, but now they also can "listen" to it as well - breezing through complex tasks that would defeat their slower human masters.

The five-day event was the International Conference on Music Information Retrieval, an annual gathering that represents the flowering of a new academic discipline: harnessing the power of computers to analyze and manage the world of sound.

Need a friendly guide for a live performance of Mahler's Eighth Symphony? A computer program developed at Swarthmore College and Drexel can meet your needs by monitoring the progress of the orchestra and supplying written lyrics and commentary at just the right time.

Can't get the hang of improvising a jazz tune on the piano? Looking for music that exactly matches the tempo of your morning jog? Unable to remember the name of that catchy tune stuck in your head? Sophisticated statistical software can guide the way in each case, to hear the various exhibitors tell it.

The event was first held in 2000, in Plymouth, Mass., with music theorists and librarians heavily represented among the few dozen attendees. Now it's a high-tech extravaganza, with curious industry representatives among the 270 people on hand this year at Drexel's airy Bossone Center.

Some of the technologies had the whiff of things that could end up in iPods in the next 18 months, perhaps to help listeners sort through an unruly music collection. Others seemed more like the outgrowth of unbridled intellectual curiosity - the sorts of what-if academic exercises scholars tackled because no one told them not to.

"There are no preconceived notions of what it is exactly they should be doing," marveled Swarthmore's Jon Kochavi, a visiting assistant professor of music who came to the event with 10 students in tow. "It is fascinating."

It's no surprise, perhaps, that a new academic discipline should arise at the confluence of music and computers. It is often said that aptitude for music and math go hand in hand; Albert Einstein was a violinist, after all.

And the celebrated mathematician and philosopher Gottfried Leibniz once said: "Music is nothing but unconscious arithmetic." That quotation was projected onto a screen before another musical performance at the conference last week, by the University of Southern California's Elaine Chew.

But here's betting that Einstein and Leibniz never imagined anything like the software that accompanied Chew on her Yamaha grand piano.

As she launched into Fêtes (Variations on Happy Birthday) by Ivan Tcherepnin, a visual representation of the tonal structures in the piece was projected on a giant screen as she played. The image was a whirling array of spirals, displaying the musical keys, triads and pitches as they came.

Beforehand, members of the laptop orchestra produced a range of electronic sounds, tapping vigorously on their keyboards and "bowing" their track-pads. One piece evoked the rushing wind and was described as "a sonic rumination of crystal caves in the clouds."

A key part of the conference each year is the announcement of results from a sort of software shoot-out - a competition in which various universities pit their music-analysis algorithms against one another.

Entrants from more than a dozen countries competed in 18 tasks, using their computers to "listen" to selections of music, then identify such things as the genre, mood, composer or title. The eventual goal: to help people search for music they might like by combing through millions of audio files in a database.

Competition organizer J. Stephen Downie, an associate professor of library and information science at the University of Illinois at Urbana-Champaign, was especially taken with the entrants' success at identifying cover songs by different artists.

For example, given one rendition of Led Zeppelin's "Stairway to Heaven," the electronic listeners had to sift through 1,000 songs and pick out 10 performances of "Stairway" by other artists, one of them on the banjo. A team from Barcelona, Spain, won that challenge, with a 75 percent success rate.

In another task, the computer had to identify tunes that someone hummed. "The idea is, you go into the karaoke bar and start humming, and the computer retrieves your song," Downie said.

The five-day conference was not just about getting the computer to listen. Engineers from Drexel programmed a robot to dance in response.

The team cooked up software to extract the drumbeat from music, then analyze it to pinpoint the beat. That information is then sent from a laptop to a 12-inch humanoid robot, which begins moving its arms and legs in time with the piece.

The robot drew an admiring crowd Thursday as it gyrated to a pulsing techno beat, while Drexel engineering student David Grunberg watched. He said larger versions of the robot might be useful to choreographers, in case they need to try out a dance step but cannot immediately hire a troop of dancers.

Or such robots might prove useful in studying the difference between random movement and creativity - the psychology of what makes us human, said Drexel's Youngmoo Kim, an assistant professor of electrical and computer engineering.

"When does an action become art?" Kim said.

One spectator tried to tinker with the robot's software so it would perform a salsa step. Another, presumably speaking for more than one person in attendance, said simply:

"It's pretty cool."