Hybrid Percussion: Extending Physical Instruments Using Sampled Acoustics by Roberto Mario Aimi S.B., Biology, Massachusetts Institute of Technology (1997) S.M., Media Arts and Sciences, Massachusetts Institute of Technology (2002) Submitted to the Program in Media Arts and Sciences, School of Architecture and Planning in partial fulfillment of the requirements for the degree of Doctor of Philosophy in Media Arts and Sciences at the MASSACHUSETTS INSTITUTE OF TECHNOLOGY February 2007 (cid:13)c Massachusetts Institute of Technology 2007. All rights reserved. Author............................................................................ Program in Media Arts and Sciences October 31, 2006 Certified by........................................................................ Tod Machover Professor of Music and Media Thesis Supervisor Accepted by....................................................................... Andrew Lippman Chairman, Department Committee on Graduate Students 2 Hybrid Percussion: Extending Physical Instruments Using Sampled Acoustics by Roberto Mario Aimi Submitted to the Program in Media Arts and Sciences, School of Architecture and Planning on October 31, 2006, in partial fulfillment of the requirements for the degree of Doctor of Philosophy in Media Arts and Sciences Abstract This thesis presents a system architecture for creating hybrid digital-acoustic percussion instruments by combining extensions of existing signal processing techniques with specially- designed semi-acoustic physical controllers. This work aims to provide greater realism to digital percussion, gaining much of the richness and understandability of acoustic instru- ments while preserving the flexibility of digital systems. For this thesis, I have collaborated with percussionists to develop a range of instruments, to refine and extend the algorithmic and physical designs, and to determine successful models of interaction. Conventional percussion controllers measure and discretize the intensity of strikes into discrete trigger messages, but they also ignore the timbre of the hits and fail to track more ambiguous input. In this work, the continuous acoustic output of a struck physical object is processed to add the resonance of a sampled instrument. This is achieved by employing existing low-latency convolution algorithms which have been extended to give the player control over features such as damping, spectral flattening, nonlinear effects, and pitch. One of the advantages of this approach is that light taps, scrapes, rubs, or stirring with brushesalltakeonahybridtimbreoftherealandsampledsoundthatissurprisinglyrealistic and controllable. Since part of its behavior is inherently acoustic, a player’s intuition about interacting with physical objects can be applied to controlling it. The ability to transform the apparent acoustic properties of objects also suggests applications to HCI and product design contexts. Thesis Supervisor: Tod Machover Title: Professor of Music and Media 3 4 Hybrid Percussion: Extending Physical Instruments Using Sampled Acoustics by Roberto Mario Aimi Thesis Readers: Thesis Reader ..................................................................... Joseph A. Paradiso Associate Professor MIT Media Laboratory Thesis Reader ..................................................................... Hiroshi Ishii Associate Professor of Media Arts and Sciences MIT Media Laboratory 5 6 Acknowledgments Thanks to: my advisor, Tod Machover; my readers, Joe Paradiso and Hiroshi Ishii percussionists: Rakalam Bob Moses, Jamey Haddad, Curt Newton, Dave Flaherty Bill Gardner, Gili Weinberg, my friends, family, the Opera of the Future group everyone who had input on the document, the instruments, and the talk. the Media Lab staff, students, faculty for their help, feedback, and daily inspiration. 7 8 Contents 1 Introduction 17 1.1 Contributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19 1.2 How this approach differs from existing trigger-based electronic percussion systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19 1.3 Musical vision: realistic, physically grounded timbral behavior for digital percussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21 1.4 Structure of this document . . . . . . . . . . . . . . . . . . . . . . . . . . . 22 2 Musical mappings, spectral continuity, and percussion. 23 2.1 The mapping problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23 2.1.1 Simpler isn’t always better. . . . . . . . . . . . . . . . . . . . . . . . 25 2.1.2 An instrument as an extension of the body . . . . . . . . . . . . . . 25 2.2 How is percussion special? . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26 2.3 Spectral continuity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28 3 Background 31 3.1 Historical precedents for electronic/acoustic instruments . . . . . . . . . . . 31 3.1.1 The Stroh violin . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32 3.1.2 The National Resonator Guitar . . . . . . . . . . . . . . . . . . . . . 38 3.1.3 The Rickenbacker electric guitar . . . . . . . . . . . . . . . . . . . . 41 3.1.4 Ondes Martenot . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42 3.2 Electronic percussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46 3.2.1 A brief history of electronic percussion . . . . . . . . . . . . . . . . . 46 3.2.2 Contemporary commercial approaches . . . . . . . . . . . . . . . . . 47 3.3 Convolution in computer music . . . . . . . . . . . . . . . . . . . . . . . . . 53 9 3.3.1 A graphical example of convolution . . . . . . . . . . . . . . . . . . . 56 3.4 Other approaches: physical modeling and modal synthesis . . . . . . . . . . 58 4 System design and software implementation 61 4.1 How the system works . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61 4.2 Initial proof of concept . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62 4.3 Further implementations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62 4.3.1 Pd patch architecture . . . . . . . . . . . . . . . . . . . . . . . . . . 63 4.4 Nonlinear responses . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65 5 Other expressive controls, extensions to realtime convolution 67 5.1 Damping . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67 5.1.1 Simple damping model . . . . . . . . . . . . . . . . . . . . . . . . . . 68 5.1.2 Frequency-dependent damping . . . . . . . . . . . . . . . . . . . . . 78 5.1.3 Pitch shifting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 80 5.1.4 Cross fading. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82 5.1.5 Inverse filtering . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83 5.1.6 Crashing a ride cymbal: pseudo- nonlinear processing . . . . . . . . 85 5.2 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 86 6 Physical controllers 89 6.1 Cymbal . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 89 6.1.1 Assembly . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90 6.1.2 Function . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91 6.1.3 Brushes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92 6.1.4 Wireless brush . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93 6.1.5 Wired brush . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93 6.2 Pad . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 94 6.3 Frame drum . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95 6.3.1 Assembly . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 96 6.3.2 Function . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97 6.4 Bass drum with speaker . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 98 6.4.1 Assembly . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 98 10