A little light music.

A Little Light Music

Lots of musicians dance while they perform. But they dance with their feet. A new computerized instrument lets your fingers do the dancing -- while they're also doing the playing.

The videoharp, still being refined, uses light and shadows to make music. It creates sounds based on the size of shadows cast by objects such as fingers, rather than the exact locations where fingers or other objects touch the instrument. The apparatus is reminiscent of some Asian musical performances emphasizing the gestures used to play instruments, says music professor Reza Vali of Carnegie Mellon University in Pittsburgh. Vali, who experimented with the videoharp at the request of SCIENCE NEWS, says, "Playing it is like choreographing a dance."

It's also like conducting an orchestra. Connected to programmable synthesizer channels, the videoharp can mimic keyboard, string, percussion and wind instruments -- successively or simultaneously.

The videharp's inventors, Carnegie Mellon computer scientists Paul McAvinney (pictured below) and Dean H. Rubine, say that by using gestures to initiate sound, videoharp musicians can "play" traditional instruments in ways otherwise impossible. Motions normally used to play one instrument can simulate another on the videoharp.

For example, sliding the side of a hand across an area programmed to mimic the strings of a cello yields cello-like sounds. Using that same bowing gesture on a section programmed to imitate a clarinet results in a succession of tonal changes that sound clarinet-like but could not be produced by applying a cello bow to a clarinet. Vali, who plays piano and trombone, says the videoharp could give musicians innovative ideas about the conventional tools of their trade. "It makes you much more aware of your gestures when you play a traditional instrument," he says.

Only extensive practice will produce a skilled videoharp player, but all it takes to play the instrument is something that casts a shadow. When placed against the videoharp, opaque objects prevent some of the light generated by a 28-inch neon tube from reaching a sensor (black dot in photo) encased in the 13-pound instrument. Unobstructed light from the tube skims across the videoharp's two plastic sides, illuminating mirrors that focus the light on the sensor, a silicon chip sensitive to light itensity.

"The sensing device doesn't know exactly where on the videoharp your fingers are. It only records how fat your fingers look to the neon tube," McAvinney says, adding that one finger close to the mirrors can produce the same-size shadow as three farther away. Similarly, any opaque object can create sounds. This means a musician can play the videoharp in unconventional ways; by pouring sand on it, for instance, or by rubbing knees against it. The music thus produced, like that created with hand gestures, depends on how individual regions of the instrument's surface are programmed to sound.

For example, a musician can use the same hand first to tap piano sounds and then to strum guitar sounds just by moving the hand several inches across the plastic surface. At the same time, the musician can play tuba, marimba, conga or other sounds on the opposite side. Software specifies the lowest note possible and the register, or range of possible tones, for each region.

Running a finger through a "piano" region produces glissando, the sweep of adjacent keyboard keys in succession. But the same motion in a "trumpet" region yields portamento, the smooth progression from one pitch or tone to another -- what musicians call "bending" a note. When a finger is held in place, a "piano" sound will fade as if a key were held down, but a "trumpet" note will persist as if a musician were supplying a steady breath.

McAvinney and Rubine say they plan to design interchangeable videoharp sides with fret, key and string locations and other guides for musicians, once the creators determine exactly where to place the features. "We're only really been learning how to play the thing for a few weeks," Rubine says. "The surfaces don't have to be plastic," McAvinney adds. "They could be made of fabric or some other material."

Not every motion on the videoharp's sides triggers a sound. Some movements occur faster than the sensor's successive recordings of light, although equally rapid gestures on traditional instruments commonly generate sound. McAvinney and Rubine are working to shorten the sensor's 30-millisecond record interval to detect quicker, subtler motions and abbreviate the split-second time delays between gestures and corresponding sounds. "We would like to achieve 10-millisecond resolution," McAvinney says, adding that audiences can perceive tempo changes within that time span.

Alternatively, the scientists may replace the sensor with a faster one, such as a charge-coupled device (CCD) designed for use in video cameras. McAvinney says compared to the sensor now used, "CCDs are more expensive and more difficult to interface to computer equipment." But he thinks a faster sensor might be worth the cost and effort because it could detect differences in how swiftly and forcefully fingers contact the instrument, making the videoharp's simulations more realistic. "A lot of the difference between musicians is these minute variations in timing," Rubine says.

McAvinney plans to reconfigure the hardware inside the instrument so a musician can use a personal computer to program a synthesize producing videoharp sounds. He and Rubine already have received inquiries from instrument manufacturers and a variety of musicians interested in obtaining videoharps, although only one model now exists.

Ultimately, the videoharp's appearance may be what propels it into the limelight, McAvinney says. "Can you imagine a band coming onto a dark stage and plugging in four videoharps, filling the room with sound and neon light?"