![error while trying to synchronize audio and midi. error while trying to synchronize audio and midi.](https://appletoolbox.com/wp-content/uploads/2021/04/create-new-user-account-mac.png)
Visual communication is only rarely a matter of one performer giving directions to another rather, even if there is a designated leader, collaborating musicians’ body movements interrelate (Chang, Livingstone, Bosnyak, & Trainor, 2017 Moran, Hadley, Bader, & Keller, 2015) and can be mutually influential (Badino, D’Ausilio, Glowinski, Camurri, & Fadiga, 2014). Usually, perception of jointly-produced sound gives sufficient information for performers to coordinate, but visual communication can be important too (Bishop & Goebl, 2015). Such an application requires a detailed understanding of gesture kinematics and how they relate to performers’ intentions.ĭuring skilled ensemble performance, most communication through audio and visual channels is nonverbal. In recent years, researchers have been applying their knowledge of the communication processes involved in group music-making to computer systems that replicate or react to performer movements (Dahl, 2014 Hoffman & Weinberg, 2011). Moreover, the possible means of communication are constrained by the task of performing an instrument, which can limit freedom of movement for much of the body, as well as conventions of public performance, which may prohibit, for example, counting out loud or using a metronome. Group music-making presents an intriguing context in which to study communication and coordination, since precise coordination must be achieved under inherently ambiguous temporal conditions (even for notated music, timing is only loosely defined by the score). Many researchers have used group music-making paradigms to investigate the communication processes underlying interpersonal coordination. Given these complexities, it is often difficult to determine from a research standpoint exactly what is being communicated and how group members are assimilating incoming information. The signals exchanged are typically multimodal (e.g., auditory and visual) and multilayered (e.g., involving facial expressions and body movements simultaneously), and can be subtle, comprising only a raised eyebrow or a brief moment of eye contact (Davidson, 2012). Communication during these tasks is continuous and interactive, with collaborators constantly adapting their own intentions and actions in response to the signals they receive from each other (Schiavio & Høffding, 2015).
![error while trying to synchronize audio and midi. error while trying to synchronize audio and midi.](https://d154hy2b5aa41t.cloudfront.net/user_avatars/2a9b93af-2d1c-44d6-a22d-3c256bf579b7/c930671b9c0cd7fb952d52bfe15bd187.jpg)
Interpersonal communication is critical for joint action tasks like playing piano duets, playing team sports, or dancing, which require collaborators to align their intentions and coordinate their actions in time. These results might be applied to interactive systems using gesture recognition or reproduction for music-making tasks (e.g., intelligent accompaniment systems). Synchronization was also more successful with more experienced leaders’ gestures. Musicians’ synchronization with leaders’ first onsets improved as cueing gesture smoothness and magnitude increased and prototypicality decreased.
![error while trying to synchronize audio and midi. error while trying to synchronize audio and midi.](https://i.stack.imgur.com/hpga7.png)
Musicians were found to align their first taps with the period of deceleration following acceleration peaks in leaders’ head gestures, suggesting that acceleration patterns communicate beat position. A subset of motion capture recordings were then presented as point-light videos with corresponding audio to a sample of musicians who tapped in synchrony with the beat.
#ERROR WHILE TRYING TO SYNCHRONIZE AUDIO AND MIDI. TRIAL#
The leader of each trial listened to a particular tempo over headphones, then cued their partner in at the given tempo, without speaking. Audio/MIDI and motion capture recordings were made of piano duos performing short musical passages under assigned leader/follower conditions. It was expected that increased precision of the alignment between leaders’ head gestures and first note onsets, increased gesture smoothness, magnitude, and prototypicality, and increased leader ensemble/conducting experience would improve gesture synchronizability. This study aimed to (1) replicate prior findings suggesting that points of peak acceleration in head gestures communicate beat position and (2) identify the kinematic features of head gestures that encourage successful synchronization. These cues must communicate beats clearly, especially if the piece requires interperformer synchronization of the first chord. Ensemble musicians often exchange visual cues in the form of body gestures (e.g., rhythmic head nods) to help coordinate piece entrances.