Understanding piano playing through MIDI: students' perspectives on performance analysis and learning.

In language, if inflection and nuance enhance the effect of the spoken word--in music they create the meaning of notes. (1) The notated score cannot conceivably provide all the indications that must be observed for the performance of a piano piece to be musically satisfactory. These elements can be best understood through the detailed study of the performance itself.

Much of the research on expressive performance has been concerned with the analysis of skilled performances. Several researchers have demonstrated that music performances are characterized by somewhat systematic variations in timing and intensity from the strict mechanical regularity of the musical scores that are related to music structure. (2, 3, 4, 5, 6)

Comparative performance analysis can aid students in their perception and understanding of the many simultaneous dimensions of musical experience one must attend to in performance. One of the biggest stumbling blocks for students is to successfully attend to all the subtleties of interpretation at the same time. Often times, a student grasps an understanding of an artist's interpretation of a certain phrase, but when asked to technically articulate these specifics on the instrument, is unable to do so. At the expert level, all these skills and the attention needed for developing them have become automatic.

Comparison of different performances recorded on the Yamaha Disklavier offers students the opportunity to analyze, while slowing the tempo in playback and listening repeatedly. Using the Disklavier in conjunction with a sequencing program allows MIDI data to be displayed as a piano roll providing visual feedback for the student. Augmented visual feedback can have striking effects on the acquisition and improvement of technical skills. (7) Cognitive feedback can improve expressive skills of music performance. (8, 9) Pianists control only two factors--timing and intensity. However, many musicians, teachers and students often fail to recognize the significance of these factors. Author C. Palmer examined three aspects of timing in piano performance that are not explicitly notated in the score: chord asynchronies, rubato patterns and legato/staccato patterns. (10) Results suggest that pianists may distort timing more than other instrumentalists who also are able to manipulate pitch and timber.

The Technology

MIDI is a data interface designed to communicate musical messages. The Yamaha Disklavier piano, used in the study discussed later, is an acoustic instrument equipped with optical laser sensors to measure the mechanical velocity of each key's hammer by registering the speed of its down-stroke. The pedals also are equipped with sensors.

The parameters are recorded digitally on a floppy disc for possible data analysis or playback. The playback feature ensures exact mechanical replication of the sounds produced in the original performance.

By utilizing music sequencing software to read the floppy disc, MIDI data can be displayed on a video screen as a scrolling piano roll, as shown in Figure 1. With respect to a display of the keyboard aligned vertically to the left, horizontal bars of various lengths indicate which keys are depressed, in what sequence and for how long. Some software, such as Emagic's Logic, displays the note bars in colors that vary from blue (soft) to red (loud) to represent dynamic levels.

[FIGURE 1 OMITTED]

Feedback

How does the feedback process work? The Disklavier piano is connected to a computer via MIDI cables. Playing is recorded on the Disklavier or through the sequencing software program. Feedback on the performance is provided immediately to the student through playback on the Disklavier, while simultaneously viewing the piano roll on the computer screen. The role of the piano roll visual feedback in improving less-advanced students' understanding of rhythmic notation was examined more closely in two imitation approaches: 1) Disklavier/piano roll presentations of expressive models followed similarly by Disklavier/piano roll feedback of how well students' imitation attempts matched the models; 2) Disklavier-only presentation of models and aural feedback. (11)

It is through subtle timing and intensity variations that interpretive nuance in music intensifies the richness of the creative experience on the part of the performer and the listener. This technology permits one to compare performances in terms of technical skills and personal interpretation, and to examine difficult-to-notate elements of timing and dynamics in detail, such as specifically where and to what extent to ritard or accelerate the tempo.

METHOD

Participants

Eight college-level piano students volunteered to participate in a study of their assessments of a performance analysis method incorporating aural/visual feedback via MIDI data. (12) All students were majoring in piano performance and had no formal training in MIDI technology. None of the students had previously studied the selected piece.

Preliminary Procedure

Three concert artists performed the Chopin Nocturne in F Major, Opus 15, No. 1 on the Yamaha Disklavier piano. The MIDI data from the performances were analyzed in terms of expressive timing and dynamics using Logic sequencing software. This data was displayed as a piano roll score on the computer screen to be viewed simultaneously with hearing the Disklavier playback. The piano roll graphs showed deviations from the regularity of the score information in terms of note duration, intensities and articulation.

The findings of the comparative performance analysis confirmed Palmer and Todd's observations that large amounts of slowing of tempo often occur at phrase endings; the slower the tempo, the softer the notes are played; there are louder dynamics at melodic contour peaks and softer dynamics at contour valleys. (13, 14)

Individual interpretive differences were noted as well from the analysis. Pianist One favored the use of delaying the onset of the melody note for beginnings of phrases and melodic peaks. Pianist Two built a crescendo to the penultimate note of a phrase, particularly peaks, accenting the arrival by a drop in intensity. Pianist Three made use of asynchronies to create rubato.

Procedure

Each student participated in an initial fifteen-minute interview. They were asked about their musical background and experiences with computers and MIDI technology. Students participated individually in three two-hour sessions of performance analysis with playback on the Disklavier, coupled with visual representation of the piano roll score scrolling across the computer screen. Performances were analyzed both in entirety and in sections. The comparative performance analysis was presented to each student, and replication of artists' select phrases was used to investigate students' understanding and technical mastery of interpretive elements in performance. Sessions were tape-recorded for comments, transcribed and summarized for further analysis. The eight students wrote several paragraphs in response to questions about the usefulness of the technology.

RESULTS

Performance Analysis Aided by Disklavier and Piano Roll

All students performed a preliminary sight reading of section A of the Nocturne. An open discussion followed their initial playing of the piece and first listening of the three artists' performances. Students heard artist performances in entirety and phrase by phrase while viewing the piano roll. Piano roll graphs of each phrase were printed for handouts to be used with the analysis. The following are examples of questions asked:

* Do all the performances follow the dynamic markings on the score?

* Which performer had the greatest range in dynamics?

* Where does Pianist One relax the tempo in the first phrase? Pianist Two? Pianist Three?

The graphs were confusing to some students at first. Comparing the three pianists' performance graphs and discussing the differences in colors of the note bars--dark blue indicates pianissimo, and a gradation of colors from blue to green indicates a range from piano to mezzo forte--helped students listen more intently for the sound variations. One student commented that comparing the different color representations of dynamics and length of the bars on the piano roll helped her understand each pianist's use of dynamics.

After listening to two of the pianists' performances of the first phrase while viewing the piano roll, student A was asked to identify differences in dynamics between the two pianists. The student asked to hear Pianist Three's performance again. After listening, he replied: "Well, I'm not really sure. The bass is louder, I think. I'm not sure." Alter listening once more, he replied: "I don't know." He was instructed to study the piano roll graph more closely. He noticed that the artist increased dynamics through the accompaniment in bar four and did not pause before the downbeat of bar five.

Student B became more aware of the dynamics of the sounds produced after seeing their colors on the graph. The student noticed the change in colors of the note bars on the graph and asked to listen again to the dynamics. He was able to pinpoint that the pianist had decreased dynamics in bar two, and the tempo slowed down before the beginning of the next phrase.

Student A summarized: "The graphs give you more than just listening because you can't take in everything in one listening. You look at the graph and you can see it as a reference. It helped a lot--you can analyze more objectively what each performer is doing and you get a clear picture of what they had in mind."

Asynchrony in Phrasing

After a general listening to all three pianists, student C indicated a preference for Pianist Three. After listening to the playback and analyzing the piano roll score for Pianist Three, student C noticed the use of asynchronies to create rubato in the phrases. Studying the piano roll graph at the high point, bar twenty, student D immediately understood the slowing down of the triplets, the space before the high D and the asynchrony between the grace notes and the accompaniment after the high D, as highlighted in Figure 2. The arrows indicate the beginning of the grace note figure and the placement of each note in the triplet figure for the left hand. By looking at the graph and seeing the colors and shapes, these students were able to listen again and hear the shape of the sounds with more definition.

[FIGURE 2 OMITTED]

Student C did not observe the differences in timing and dynamics in her initial listening to the performance; she was able to hear and describe specific details of the three pianists' phrasing. She found the performance graphs very helpful, especially with the red bar lines added. Her concentration improved and, as the analysis continued, less repeated listening was required for each phrase.

Imitation Aided by Disklavier and Piano Roll

Students were asked to replicate select phrases of the performances while following the piano roll graph, either melody or accompaniment alone or both together. The students listened to the select phrase on the Disklavier while following it on the piano roll score, "shadowing" it an octave lower on the Disklavier. Immediately after practicing the artist's performance, each student recorded it. During these imitative attempts on the Disklavier, IOI (inter-onset interval) data were collected to compare the imitation to its model. Feedback consisted of playback on the Disklavier of students' imitative attempts, while following a cursor line sweep over the piano roll score of the imitative model.

When the students played along with the artist's melody while following the piano roll score, they were able to replicate timing and dynamics. Imitation was not as successful following just the visual representation of the piano roll without the sound. When used alone, the piano roll was seen and perceived as just a timing device. In melody to accompaniment replication, the piano roll of the artist's accompaniment was shown during playback while the student played the melody. This process allowed the students to concentrate on technically replicating the melody while listening to the integration of voices.

When students attempted to play the accompaniment to the artist's melody, they were hampered by not knowing the notes well enough.

Self-Direction Aided by Disklavier and Piano Roll Piano Technique

Disklavier playback and the piano roll graphs offered insight into the pianists' technical approaches, as discussed in the analysis. During Disklavier playback of the performance, the keys move on the Disklavier. Student F noticed Pianist Two's use of overlap legato, particularly in the melody and when hearing Pianist Two's performance again, he realized the overlap legato created the dynamic of the tone produced. He listened and watched the keys move on the Disklavier. By comparing Pianist Two's playing to Pianist One's, he saw and heard a different legato approach. "I see and hear that Pianist One did not overlap the legato. The sound is different than that of Pianist Two with a heavy overlap." After seeing the graph, another student commented: "While Pianist Two's accompaniment was softer than the melody, it sounded heavy. His legato approach seems to add tension to the hand and produce a heavier sound."

Student H became more aware of her own overlap legato. In the second session, she recorded the Nocturne and listened to the playback. Looking at the piano roll score, she noticed the note bars indicated she had a tremendous overlap in her melody notes. She played again, trying to correct this and found that it was not so simple. She said: "I'm not thinking. Instead of listening carefully, I'm just shaping the line with my fingers." She was able to improve her legato technique as she worked with the feedback from the Disklavier and piano roll.

Interpretation

Comparative performance analysis can be helpful to students for consideration of different approaches to a phrase in terms of timing and dynamics. Sessions included asking students to work on phrases where they had specific questions regarding the use of rubato or increase in dynamic scale. The piano roll graphs of the artists' interpretations of a specific phrase were analyzed and the differences in approach were discussed while listening to the Disklavier. Students were able to hear and see specifically the beat on which the artist took a rubato in the phrase or where a crescendo began and how the musical lines were woven together. The students' own performances were recorded, graphed and then compared to the artists' performances, providing immediate aural and visual feedback to track their understanding.

Student C was unsure how she wanted to interpret the second phrase. To help clarify the artists' choices, she listened to their performances again while following the piano roll. The student commented: "Both pianists' approaches work. You could even choose to begin it in the middle of the phrase." She was beginning to consider other ways to shape her interpretation. She recorded the phrase several times and compared her performance aurally and visually to the artists'. Several student comments follow:

Student E: My recordings of the Nocturne were compared, both aurally and visually, to the artists' recordings as well as to my other recordings each week. I was able to see and hear what I was doing incorrectly in terms of balance and interpretation. This enabled me to make corrections quickly. Seeing the analyses on the screen helped me a great deal. This was very beneficial in teaching me to listen to myself as I play. It gave me goals to strive for in my playing.

Student G: Working with the Disklavier and piano roll score on the screen have been excellent tools in helping me move to another stage of development in my own playing. Seeing the piano roll along with hearing the playback helped me distinguish differences in interpretation. This experience has not only helped me to develop a more critical ear for sound, but it has inspired me to listen more intensely as I create my own interpretation.

Student F: Working with the Disklavier and performance (piano roll) score enabled me to critically listen to and compare different pianists' interpretations. It made it easier to try and imitate them. I was able to see their legato technique and pedaling in playback on the piano, as well as the piano roll graph of their playing on the computer screen. I learned to listen for each voice, to achieve the proper balance among the voices in replication as well as in my own interpretation.

Conclusions

Students' abilities to perceive expression and make sense of it can be investigated by asking them to imitate others' performances. The main drawback to this approach is that it combines a performance task with a perceptual one, making it difficult to separate the perceptual from the skill components in the analysis of the results. (15) To ensure as much as possible that failure to imitate is not a motor skill failure, the material presented must be technically undemanding, as well as be within the performance capability of the participants.

MIDI is an educationally powerful tool for providing listening and analytical access to music. Several students were skeptical at the beginning of the study. However, after working with the Disklavier and piano roll, students agreed it provided a new way of listening.

The goal of the study was not to prove this teaching tool is infallible or negate traditional methods. The effectiveness of the Disklavier with piano roll, combined with the effectiveness of a good piano teacher, seems to be worthwhile and is an area for future research. This teaching technique helps students discover with their ears and eyes the many rich variations in interpretations among performances. Being able to sit and listen repeatedly to artists' performances "live" on the Disklavier with graphic displays, allows for the analysis and teaching of the normative aspects of performance. Students can hear and analyze the same performance over and over, hands separately and even at a slower or faster tempo. A fast passage is sometimes difficult to hear and understand when a student is first learning a composition. The ability to change tempo within a piece and play it on the Disklavier, dividing the hands to play separately or together, assists in practicing and developing better technical skills. The skills learned in performing a particular piece are applicable to other pieces.

The ultimate goal of music teaching is to enable students to improve their own performances. Regardless of how they are developed, a willingness to explore and the achievement of technical mastery are two essential components of all artistic experiences. While the study was not representative of a large group of piano students, the students' responses clearly indicate that the Disklavier and piano roll technology enhanced their understanding and learning of a piece.

The acoustic Disklavier piano provides for the pianist an immensely important form of access to MIDI. While many electronic MIDI keyboards are weighted and touch-sensitive, they do not adequately replicate the touch and sound of the acoustic instrument. Furthermore, many classical pianists are not familiar with these instruments, and teachers consider them to be inadequate substitutes. There is limited utilization in the private teaching studio. The application of this technology with individual teaching seems worthwhile.

NOTES

(1.) Lewers, J. M. "Rehearsal as the Search for Expressiveness: Implications for Music Reading in the High School Mixed Chorus." (Unpublished doctoral dissertation, Teachers College, Columbia University, New York, 1980).

(2.) Bengtsson, I. and A. Gabrielsson. "Analysis and Synthesis of Musical Rhythm." Studies of Music Performance, edited by J. Sundberg. (Stockholm: Royal Swedish Academy of Music, 1983): 27-60.

(3.) Clarke, Eric F. "Imitating and Evaluating Real and Transformed Musical Performances." Music Perception 10(3), 1993: 317-41.

(4.) Gabrielsson, A. "Performance of Rhythm Patterns." Scandinavian Journal of Psychology, 15, 1974: 63-72.

(5.) Seashore, H. G. "An Objective Analysis of Artistic Singing." Objective Analysis of Musical Performance: University of Iowa Studies in the Psychology of Music, edited by C. E. Seashore. (Iowa City, IA: University of Iowa, 1937): 12-157.

(6.) Shaffer, L. H., E. F. Clarke and N. P. Todd. "Metre and Rhythm in Piano Playing." Cognition, 20: 61-77.

(7.) Tucker, W. H., R. H. T. Bates, S. D. Frykberg, R. J. Howarth, W. K. Kennedy, M. R. Lamb and R. G. Vaughan. "An Interactive Aid for Musicians." International Journal of Man-Machine Studies 9, 1977:635-51.

(8.) Juslin, P. N. and P. Laukka. "Improving Emotional Communication in Performance Through Cognitive Feedback." Musicae Scientiae, 4(2), 2000:151-81.

(9.) Juslin, P. N. "Communicating Emotion in Music Performance." Music & Emotion, edited by 12. N. Juslin and J. A. Sloboda. (New York: Oxford University Press, 2001): 309-37.

(10.) Palmer, C. "Timing in Skilled Music Performance." (Doctoral dissertation, Cornell University, 1988).

(11.) Riley, K. and E. E. Coons. "Improving Pianists' Rhythmic Perception in Score Reading Through Imitation and Feedback." Journal of Technology in Music Learning, 3(1) 2005.

(12.) Riley-Butler, Kathleen. "Understanding Interpretive Nuance in Piano Performance Through Aural/Visual Feedback." (Unpublished doctoral dissertation, New York University [UMI Number: 9992360], 2001).

(13.) Palmer, C. "Anatomy of a Performance: Sources of Musical Expression." Music Perception 13(3) 1996: 433-53.

(14.) Todd, N. P. "A Model of Expressive Timing in Tonal Music." Music Perception, 3, 1992: 33-59.

(15.) Clarke. "Imitating and Evaluating Real and Transformed Musical Performances," 1993.

Kathleen Riley, a Yamaha artist and research consultant to Yamaha Corporation of America, holds bachelor and master of music degrees and a Ph.D. from New York University, where she is an adjunct professor of music.