For your convenience Apress has placed some of the front matter material after the index. Please use the Bookmarks and Contents at a Glance links to access them. contents at a Glance Foreword �������������������������������������������������������������������������������������������������������������������������xiii About the Authors ����������������������������������������������������������������������������������������������������������xv About the Technical reviewer �������������������������������������������������������������������������������������xvii About the cover Image Artist ���������������������������������������������������������������������������������������xix Acknowledgments ���������������������������������������������������������������������������������������������������������xxi Introduction ������������������������������������������������������������������������������������������������������������������xxiii chapter 1: Diving into the shallow end ��������������������������������������������������������������������������1 chapter 2: Art by numbers ��������������������������������������������������������������������������������������������33 chapter 3: Processing Boot camp �������������������������������������������������������������������������������65 chapter 4: creating Across Time and curved space �����������������������������������������������107 chapter 5: expressive Power of Data �������������������������������������������������������������������������149 chapter 6: organizing chaos ��������������������������������������������������������������������������������������187 chapter 7: creative Abstraction ����������������������������������������������������������������������������������233 chapter 8: Drawing with recursion ����������������������������������������������������������������������������277 chapter 9: Painting with Bits ���������������������������������������������������������������������������������������311 chapter 10: expressive Imaging ���������������������������������������������������������������������������������369 chapter 11: spreading Your creative coding Wings ������������������������������������������������413 Index �������������������������������������������������������������������������������������������������������������������������������437 v Introduction Creative Coding grew primarily out of the digital art and design community, as an approach to programming based on intuitive, expressive, and organic algorithmic development, with an iterative leap-before-you-look style. Related, but broader, is the idea of Creative Computation, which explores computation as a universal, generative, and primary creative medium. We find these paradigms well suited for introducing computing to a new generation of students, who respond well to creative tasks and visual feedback. This book attempts to introduce programming and the fundamentals of computing and computer science by engaging you, the reader, in Creative Coding and Creative Computation contexts. This book is designed for independent learning as well as a primary text for an introductory computing class (also known as CS1, or CS Principles, in the computing education community). A lot of the material grew out of our very successful NSF TUES funded project to develop a complete CS1 curriculum using Processing and Creative Coding principles. The central goal of the project was to strengthen formative/introductory computer science education by catalyzing excitement, creativity, and innovation. The digital representation of data, access to authentic sources of big data, and creative visualization techniques are revolutionizing intellectual inquiry in many disciplines, including the arts, humanities, and social sciences. We strongly believe that the introductory computing curriculum should be updated with contemporary, diverse examples of computing in a modern context. We developed our introductory computing curriculum based on the philosophy that it should cover the same set of core CS1 topics as any conventional Java-based CS1, but show applications in the visual arts and interactive media, as well as through clean, concise, intuitive examples of advanced areas not typically accessible to CS1 students, including physics-based simulations, fractals and L-systems, image processing, emergent systems, cellular automata, aspects of data science and data visualization. While it is entirely possible to learn to program on your own with this book, teaching with this book will require additional organization and initiative on the part of the instructor. This is an unconventional CS1 text in that our priority was to demonstrate the Creative Computation way of thinking (or way of teaching) and how it connects to the introductory computing curriculum, rather than to provide systematic/detailed lesson plans. Many standard, but basic programming constructs have not received the typical amount of attention because we trust a certain level of instructor experience and comfort to fill the gaps in the classrooms. It is our hope and belief that once shown the creative possibilities, following the same philosophy and adapting the material to your own classrooms will be an enjoyable and motivating task. resources The home of the Processing project is the website: http://processing.org First and foremost, install Processing from their Download section. At the time of this writing, Processing 2.0 is still in later stages of Beta (the most current version is 2.0b8). The stable release remains to be 1.5.1. The Processing installation comes with an extensive collection of examples directly accessible from the IDE through the File➤Examples pull-down menu that should not be overlooked. This book includes coverage of most of the key features included in Processing 2.0. Those using Processing 1.5.1 release should bear this in mind. xxiii Introduction Besides the Download section, the Processing website maintains a complete API Reference. We refer to it as the Processing Reference in this book and would encourage the reader to become familiar with navigating their browsers to it as well as to learn to read the rich documentation and examples it provides. Much learning can and will take place with increased usage of the Processing Reference. Equally important for the beginner (as well as the instructor) is the Learning section of the Processing.org website. The set of tutorials provided there will be valuable for anyone starting to learn Processing from this book. These comprehensive tutorials are good starting points for the basics, as well as additional information. In fact, these should be treated as ancillary materials for the earlier chapters. Both reference and tutorials are also conveniently accessible from the Processing IDE from the Help pull-down menu (just access the Help➤Reference and Help➤Getting Started options). We have taught using the approach presented here well over a dozen times in the past few years. Consequently, we have accumulated a wealth of curricular material, much more than there is room for in this book. Instructors interested in gaining access to complete syllabi, lecture notes, class examples, assignments, and problem sets, please write to us. The book is sprinkled with a number of “Try This” suggestion boxes, which we have chosen to place wherever they are relevant. Some merely test a grasp of the current discussion; others require more substantial thought and even full-fledged programming work. The complete code examples in the book can be downloaded from the book publisher’s resource website: http://apress.com/9781430244646 While it would be trivial to copy the code and run it, we strongly urge everyone to actually type the programs yourselves. This is a book about learning after all, and there is no substitute for experiencing the coding process first hand. Entering your own code from scratch, even if you are copying from these examples, is essential to learning the concepts that are being communicated. Question every word in every line of the code as you type, and don’t be afraid to experiment. You will learn more! We also recommend visiting the online community OpenProcessing: http://www.openprocessing.org This website is devoted to sharing open-source Processing sketches. There is much to learn from and get inspired by from the sketches shared there by creative coders, students, and teachers alike from across the globe. We hope that, as you learn and evolve your own creative style, you will contribute your own sketches. Finally, the Processing.org website itself has its own discussion forum where you can post queries and get involved in discussions involving all things Processing. We urge you to support these efforts in any way you can. Buy a Processing t-shirt! In the Book In line with the philosophy of leap-before-you-look we begin by literally diving into the shallow end (Chapter 1) where we provide background and introduction to creative coding, Processing, its origins, and its relationship with Java. We offer a 30,000 feet overview of creative coding, the distinction between programming and the discipline of computer science. We also provide a detailed tour of the Processing IDE, which could be skimmed in the initial encounter but returned to later as the need arises. We follow this, in Chapter 2 (Art by Numbers) with a whirlwind tour of pseudocode, algorithms, and basic programming constructs: variables, simple types, expressions, and comments. Drawing with these coding primitives is presented by introducing Processing’s xxiv Introduction coordinate system and commands for drawing 2D shapes. This sets the stage for a deep immersion in Processing and creative coding. By taking a boot camp approach (Chapter 3), we introduce functions, all the basic control structures (conditionals and loops), and additional drawing concepts like transformations, and drawing contexts. Instructors of introductory computing courses may want to take a more deliberate approach here especially with the key programming constructs. These are further developed in Chapter 4 to enable a deeper understanding of structuring programs. At the same time, we introduce the dynamic mode in Processing, including handling mouse and keyboard events, as well as a deeper treatment of drawing curves. By this time, most of the basics of programming as well as Processing have been introduced. We would encourage the readers to take some time to reflect, take stock, and review before moving on into the book. Beyond the basics, we next delve into some of the core CS1 topics: arrays, object-oriented programming, and recursion. We use the backdrop of learning about arrays to introduce concepts in data science and visualization. Arrays are introduced as mechanisms for storing, processing, and visualizing datasets (Chapter 5). Also, one of the core computer science ideas: algorithms and space-time complexity are given an informal treatment. Object-oriented programming and its manifestation in Processing is introduced in Chapter 6 as a mechanism for organizing the chaos of large programs and also as a way of modeling “live” or animated entities in a sketch by way of particles and physical motion models. We introduce Processing’s PVector class and go beyond basic physics by providing a detailed example of Verlet Integration for natural looking motion and behaviors in sketches, thereby truly enhancing a creative coder’s conceptual toolbox. We go deeper into creative data visualization (Chapter 7) by deconstructing (and reconstructing) a popular visualization application: word clouds. In the process, strings, ArrayLists, sorting algorithms, and font metrics are introduced. In the context of a fairly non-trivial example, we have attempted to illustrate program design and redesign principles and how object-oriented programming facilitates well-designed, maintainable, clean code. More creative fun follows with recursion (Chapter 8) as a computational medium for creating and experimenting with models of biological systems, fractal geometry, and advanced data visualization. Our journey continues deeper both into computing and creative computing where in Chapter 9 we explore painting with bits in the context of image processing and manipulation. Two dimensional arrays and pixel buffers as underlying computing concepts come together in manipulating images to create stunning visual effects. We go beyond, by using bitwise operations and Processing’s color component functions to see examples of steganography. Further, iterative solutions are presented as solutions for modeling emergent systems, cellular automata, and glorious tiling patterns. We build further on image manipulation in Chapter 10 where we try to impress upon the idea that image processing is a creative medium with wider range of applications that go beyond traditional “photo manipulation.” This is also where the fact that Processing is designed by artists and creative coders really pays off: We introduce several built-in image manipulation functions that are typically not accessible to students in an introductory computing course. If you consider this book a (big) meal, you can think of the last chapter as dessert–a hard earned and guilt free dessert! It is truly a buffet, with a glimpse into the three-dimensional world, developing sketches for Android devices, and going beyond Processing into Java, OpenGL, C++, and other programming environments. In today’s ubiquitous world of computation, the boundaries between languages, environments, and devices are blurring. Processing gives you a handle on the myriad of techniques and tools that serve as building blocks at first and then takes you to the very cusp of exciting new frontiers of creative computing. We hope you will enjoy this gradual, yet deliberate computational, intellectual, and visual feast. If we have not been successful at any aspect of the book, please write to us, we want to know. Bon appétit and happy creative coding! xxv Chapter 1 Diving into the Shallow End Imagine enrolling in a class to learn a new sport, perhaps Irish Hurling. (This type of hurling involves a ball and bat.) You arrive at class on the first day excited to dive into this exotic new hands-on activity, only to be confronted by a long lecture on the theoretical foundation of hurling. Over the next fourteen weeks, the course proceeds in a similar fashion, with maybe a couple of contrived on-the-field exercises to reinforce the lectures. We don’t know about you, but we’re not so sure we’d make it through this class long enough to get to the on- the-field part (the part that got us excited about learning hurling to begin with.) This is what many students experience when they take their first computer science class. Of course, learning the theory behind Irish Hurling might provide you with pretty interesting and ultimately valuable information, especially if your goal is to become a world-class Hurler. However, for most of us, diving directly into the theoretical aspects of an activity such as hurling, or computer science, before getting a handle on why the theory might actually be useful can be intimidating and off-putting (and very few of us are destined for the Hurling Hall of Fame.) Worst of all, this approach can lead to wider societal misconceptions, such as: computer science is obscure, difficult, and even boring. These misconceptions can also become self-fulfilling prophecies, ultimately attracting only the types of students who buy into the misconception, leading to a popu- lation of students and practitioners lacking in diversity and varied perspective. In the last few years, some computer scientists and other computing professionals, the authors of this book included, have begun challenging the entrenched and narrow approach to teaching computer science. This book champions a new way–a creative coding approach, in which you’ll learn by doing. Building creative code sketches, you’ll learn the principles behind computer science, but in the context of creating and discovery. Returning to the hurling analogy, first you’ll learn how to whack the hurling ball (the silotar) with the hurling bat (the hurley) wicked hard; then you’ll learn the physics behind it. Or, to use some computing lingo, first 1 Chapter 1 you’ll learn to code a cool app, then you’ll learn about the fundamental principles behind it. Not only will this make coding easier and more fun to learn, but it will make the theory part much more relevant and hopefully even fascinating. This chapter provides just a little context and background for the rest of the book. You’ll learn about the history of Processing, including its origins at the famous MIT Media Lab. We’ll discuss the creative coding approach in a bit more detail, including some relevant research into its effectiveness. Finally, you’ll have a detailed tour of the Processing language and development environment. Programming vs. Computer Science If you want to tick off a computer scientist, tell him that you know computer science because you can write some code. (Of course, many computer scientists don‘t show a lot of emotion so you may not even be sure you’ve succeeded.) Kidding aside, programming is not computer science. BUT, from our perspective, it is often the most fun part of it. Yet, there are computer scientists who don’t actually program. These are theoreticians who see computation more as applied mathematics than as hands-on implementation. Such a theoretician might be interested in proving something about computing, using mathematical proofs. However, to the average end-user, programming is often equated with computer science. According to Dictionary.com, computer science is defined as: the science that deals with the theory and methods of processing information in digital computers, the design of computer hardware and software, and the applications of computers. The first part of the definition, the theory and methods of processing information is concerned with more fun- damental mathematical principles behind computing. This is perhaps the most pure scientific part of computer science. Research in this area affects things like the speed, efficiency, and reliability of computers. Arguably, this area of research provides the bedrock for all other aspects and applications of computing. Though pro- gramming is a part of this branch of computer science, its role is primarily for testing and verifying theory. A company like Apple spends a great deal of time and resources researching how its hardware and software should look, feel, and function in the hands of users. This area of computer science research, the second part of the dictionary.com definition, the design of computer hardware and software, is where science gives way to engineering and design–where theory is applied, creating tangible systems. Another way of describing this area might be: the interface between the mathematical and theoretical aspects of computing and the incred- ible things we can do with it. Programming is a huge part of this area and is commonly referred to as software engineering. The last part of the definition, applications of computers (not to be confused with computer apps) is about how computers (really computation) can be applied in the world. This part of the definition may be too general, as computers impact nearly all aspects of life, and it’s extremely likely this impact will only increase in the future. It's not such a leap to imagine our cars driving themselves, our walls and countertops acting like smart touch screens, and our communication devices shrinking and getting even further integrated into perhaps even our 2 Diving into the Shallow End physical bodies. Programming is very relevant to this part of computer science as well, mostly in the develop- ment of specialized software and hardware targeting specific application domains. Google developed and released the Android Software Development Kit, which includes libraries of code and application software for creating custom Android apps. Apple has its own similar development platform, as do many other companies. These development environments enable people to efficiently program applications, without the need of years of formal computer science training. Clearly, this evolution in software development is challenging long held notions of required technical expertise. There are high school students writing highly suc- cessful mobile applications, artists programming interactive artworks, and many other “non-experts” creating small software businesses overnight. So no, programming is not computer science, but apparently computer science is not necessarily required for programming either. Art + Science = Creative Coding At Southern Methodist University in Dallas there is a Center of Creative Computation (C3) that explores computation as a fundamental creative medium. C3 considers computer code (as well as other aspects of computation) the same way a painter thinks about paint, or a musician sound or even how a dancer thinks about gesture. C3 is less concerned with why computation solves a specific problem and more interested in how it is solved, and most importantly, how it can be solved in a more interesting and novel way. Yet in spite of this creative approach, C3 requires students to take very challenging courses in computer science, math, and physics. It also requires an equal amount of rigorous creative courses. This integration of quantitative material with creative practice can be a daunting challenge for some students, especially those who were labeled at an early age: “the artist” or “the geek,” but probably not both. C3 has been successful (as has a similar interdisciplinary approach at Bryn Mawr College) integrating dif- ficult quantitative material with creative practice in the classroom, and research lab, utilizing a “Creative Coding” approach. This approach was originally developed at the Massachusetts Institute of Technology (MIT) Media Lab, by past lab director John Maeda, who you’ll hear more about shortly. Creative coding combines approaches from the arts classroom, such as critiques, portfolio development and emphasis on aesthetics and personal expression, with fundamental principles from computer science. Creative coding uses computer code as the creative medium by which students develop a body of art, while developing core competency in programming. In 2010, researchers from Bryn Mawr College and C3 at Southern Methodist University received a National Science Foundation grant to explore the use of creative coding in the introductory computer science classroom. Based on early research results, it is very promising that students learning the creative coding approach develop significantly greater personal interest in programming as compared to students in a more traditional computer science class. To help facilitate this integration in the classroom, the creative coding approach relies on some innovative programming languages and development environments, especially Processing, which grew directly out of work done at the MIT Media Lab. 3 Chapter 1 MIT Media Lab The MIT Media Lab was founded by MIT professor Nicholas Negroponte and then-MIT President Jerome Wiesner in 1985. Its mission, as stated on the Media Lab site (http://www.media.mit.edu/about), is to: envision the impact of emerging technologies on everyday life—technologies that promise to fundamentally transform our most basic notions of human capabilities. Though an academic lab at MIT within the School of Architecture and Planning, the Media Lab has always radi- cally crossed disciplines and blurred distinctions between theory and implementation, academia and industry, and science and art. It has been involved in fundamental breakthroughs of the digital age since its founding, including the World Wide Web and wireless networks. The lab has also pioneered innovative research and devel- opment in radically new areas, such as smart toys, ubiquitous computing, and aesthetics and computation. The Aesthetics + Computation Group (ACG) at MIT was created in 1996 by John Maeda, a formally trained computer scientist and graphic designer. Maeda and ACG explored novel approaches to software tools and language development, as well as computational artistic practice. One of the projects developed at the Media Lab was a new programming language and programming environment named “Design By Numbers” (DBN). DBN is a very simplified programming language built on top of the Java programming language (explained a bit later in this chapter). DBN greatly simplified the process of graphics programming using Java by creating a simplified language syntax (the commands and rules used to program) and a development environment that enabled fast prototyping of simple graphics patterns, code art, and designs. DBN was never intended as a full-featured programming language, but rather a proof of concept for a radically new approach to language design; it was tested primarily in the design arts classroom to teach programming to beginners. DBN as a proof of concept was a big success, though as a usable language, it wasn’t much more than an aca- demic exercise. Two of Maeda’s students in the Media Lab, Ben Fry and Casey Reas, worked on DBN. After finishing their studies at the Media Lab, Fry and Reas decided to take the lessons learned developing DBN and build a more full-featured language. They named their new project Processing, which they kicked off in 2001. What Is Processing? In the very simplest sense, Processing is a software application that allows you to write, edit, compile (which will be explained shortly), and run Java code. However, before discussing Processing further, it will help you to understand a little bit about Java, but even before we talk about Java, we need to talk briefly about computing in general. (Please note, this will be one of the only places in the book where we throw some theory at you without a fun, hands-on activity.) Bits and Bytes You probably have some sense that computers and 1’s and 0’s go together. But it may not be so clear to you how a bunch of 1’s and 0’s can lead to Shrek running through a field of blowing grass, your computer’s operat- ing system, or Facebook. It really is truly remarkable what has been done with a bunch of 1’s and 0’s. Though really, it’s not about 1’s or 0’s, but instead a state of being true or false, or more accurately, something being 4