A s f J o a un va d in 8 BOOKS FOR PROFESSIONALS BY PROFESSIONALS® Fischer Java Closures and Lambda RELATED Java Closures and Lambda introduces you to significant new changes to the Java language coming out of what is termed Project Lambda. These new changes make their debut in Java 8, and their highlight is the long-awaited support for lambda expressions in the Java language. You’ll learn to write lambda expressions and use them to create functional interfaces and default methods for evolving APIs, among many other uses. The changes in Java 8 are significant. Syntax and usage of the language are changed considerably with the introduction of closures and lambda expressions. This book takes you through these important changes from introduction to mastery. Through a set of clear examples, you’ll learn to refactor existing code to take advantage of the new language features. You’ll learn what those features can do for you, and when they are best applied. You’ll learn to design and write new code having these important new features in mind from the very beginning. Over the course of this book, you will learn to: • E ncapsulate key bits of logic into anonymous functions without the need for anonymous classes • Define closures to be used as generators of comparison functions • Define map, filter, and reduce functions that are useful in working with lists and other collections • Trap and handle exceptions involving lambdas and closures, including the passing of exception-type parameters • Anticipate and provide for concurrency so that your code successfully scales across multiple CPU cores • Refactor existing code to take advantage of newly-possible closures and lambda expressions US $49.99 Shelve in ISBN 978-1-4302-5998-5 Programming Languages/Java 54999 User level: Beginning–Advanced SOURCE CODE ONLINE 9781430259985 www.apress.com For your convenience Apress has placed some of the front matter material after the index. Please use the Bookmarks and Contents at a Glance links to access them. Contents at a Glance About the Author �����������������������������������������������������������������������������������������������������xi About the Technical Reviewer �������������������������������������������������������������������������������xiii Acknowledgments ���������������������������������������������������������������������������������������������������xv Introduction �����������������������������������������������������������������������������������������������������������xvii ■ Chapter 1: Java 8: It’s a Whole New Java �������������������������������������������������������������1 ■ Chapter 2: Understanding Lambdas in Java 8 �����������������������������������������������������11 ■ Chapter 3: Lambda’s Domain: Collections, Maps, and Streams ��������������������������33 ■ Chapter 4: I/O with Lambdas �������������������������������������������������������������������������������51 ■ Chapter 5: Data Access with Lambdas ����������������������������������������������������������������71 ■ Chapter 6: Lambda Concurrency �������������������������������������������������������������������������95 ■ Chapter 7: Lambdas and Legacy Code ���������������������������������������������������������������111 ■ Chapter 8: Lambdas in Java Bytecode ��������������������������������������������������������������139 ■ Appendix A: A Tour of Paradigms ����������������������������������������������������������������������153 Index ���������������������������������������������������������������������������������������������������������������������195 iii Introduction This book is the culmination of many brash years and hard lessons. The story starts all the way back when I migrated from C++ into perl. The perl programming language was amazingly powerful compared to the low-level manipulations and bookkeeping of C++. (The fact that it was “slow” never bothered me – I would rather write powerful, effective slow code than weak, buggy fast code.) In the world of perl, there was the idea of an “anonymous subroutine” that could be passed around and manipulated. You could also directly manipulate the symbol table. The symbol table is the collection of function names available to the program. Between these two things, I realized that I could code at a higher level: I could write subroutines that returned subroutines and store those into the symbol table, effectively having my code write code at runtime. In perl, these subroutine factories are called “template functions.” I proceeded to write some truly unreadable – but truly powerful – perl. I shared this revelation with my friend and mentor, Brian Hurt. He was the grizzled veteran developer who seemed to have seen it all. Brian told me that what I was doing was this thing called “functional programming,” and encouraged me to look into proper functional languages, specifically OCaml, and its derivative, JoCaml. I was immediately hooked. By 2008, I presented “Why Rubyists Should Learn OCaml” to the Ruby Users Group of Minnesota (Ruby.MN).1 There was a power in functional programming that was truly incredible compared to the then-standard way of writing code. Moreover, my mathematical background played very nicely with functional programming: the fact that state did not change meant that I could employ the same kind of reasoning to my programs that I employed with mathematical equations. I presumed at the time that a functional programming language would rise and fundamentally transform what it means to program, much as Java ascended and made Object Oriented Programming ubiquitous. So far, this hasn’t happened. The next chapter came with the rise of the Groovy programming language. Groovy’s MetaClass functionality provided a way for me to perform the same tricks as in perl, but in a way that leveraged all that existing Java code. This was having my cake and eating it, too. All that open source software that existed for Java, and all that technical expertise that had been built up on the JVM could finally mix with these “functional stunts.” It wasn’t functional programming – and I got in quite a bit of trouble on the Internet for saying as much – but it was certainly borrowing some powerful tricks from functional programming. We get into what was happening in chapter 1. When Java 7 rolled out, it introduced the invokedynamic keyword (which we cover in chapter 8). The keyword was touted as being support for dynamic languages, but I recognized it for what it was: JVM support for those same functional stunts. There was no longer any technical reason why you could not perform the same functional stunts in Java itself. The Java language’s syntax, however, simply could not keep up with its underlying implementation. We had to wait until Java 8. With Java 8, we finally got lambdas into the Java language itself, along with basic support for some of the most powerful functional stunts. This is a truly exciting language change. I think it is as transformative to the Java language as the introduction of object oriented programming in the first place, and it shows that the functional stunts, which were esoteric yet impressive back in perl, are truly primed to become industry standard practices. That is why this book is so critical: it is no exaggeration to say that learning the content in this book will help you program throughout the remainder of your career. Sooner or later, you are going to have to learn to harness the power of lambdas. Might as well start now. 1http://www.youtube.com/watch?v=2T5syww4Nn4. xvii Chapter 1 Java 8: It’s a Whole New Java This book is about lambdas (closures) in Java 8. More than that, though, it’s about the new language that Java has become. The revolution was not televised, and it was met with little fanfare, but it has happened. It is possible for you to continue to write the same old Java code in our new Java 8 world, but unless you get onboard with the revolution, you will increasingly discover code that is called “Java,” but which has language, syntax, and customs that are foreign to you. This book is here to help you enter into this exciting new world. You should be eager to join this brave new Java 8 world, because it will actually enable you to write code that is more succinct, more readable, and more performant: it’s rare that you get all three of these wins together, but lambdas in Java 8 bring all this to the table. To understand exactly where the new Java is coming from, let me go back into history. A long time ago in an old version of Java, a member of my team wrote a piece of code that would clone a list, but without including any of the null elements that may have been in the original list. That team member’s implementation looked something like Listing 1-1. The tech lead saw that code and was not impressed, and so he rewrote it into Listing 1-2, which was longer, but both more readable and more efficient. Listing 1-1. Original Implementation of cloneWithoutNulls(List) public static <A> List<A> cloneWithoutNulls(final List<A> list) { List<A> out = new ArrayList<A>(list); while(out.remove(null)) {} return out; } Listing 1-2. Refined Implementation of cloneWithoutNulls(List) public static <A> List<A> cloneWithoutNulls(final List<A> list) { List<A> out = new ArrayList<A>(list.size()); for(A elt : list) { if(elt != null) out.add(e); } return out; } This is when I became responsible for that code. I reimplemented the method as something more readable to my eyes. I did so by leveraging Apache’s Commons-Collections library. This creates the implementation given in Listing 1-3. The additional readability is because Commons-Collections has a way 1 Chapter 1 ■ Java 8: It’s a Whole NeW Java of saying, “Give me all of the collection’s elements where some predicate is true.”1 Because they had this predicate infrastructure, you can express that idea succinctly. I thought that I had really made some progress in clarifying and simplifying the code. Plus, if we ever needed to add another criterion to filter on, it’s easy to extend the Predicate to make that happen. Listing 1-3. Apache Commons-Collections Implementation of cloneWithoutNulls(List) public static <A> List<A> cloneWithoutNulls(final List<A> list) { Collection<A> nonNulls = CollectionUtils.select(list, PredicateUtils. notNullPredicate()); return new ArrayList<>(nonNulls); } Unfortunately, the time came when the new hot library, Google’s Guava, replaced Apache Commons-Collections on this project. Guava also contains a Predicate infrastructure, but it has an entirely different package name, inheritance tree, and API than the Commons-Collections Predicate infrastructure. Because of that, we had to change the method to the new code in Listing 1-4. Listing 1-4. Google Guava Implementation of cloneWithoutNulls(List) public static <A> List<A> cloneWithoutNulls(final List<A> list) { Collection<A> nonNulls = Collections2.filter(list, Predicates.notNull()); return new ArrayList<>(nonNulls); } The problem is that the Predicate infrastructure was useful, but Java didn’t have it in its core, and so extensions to the standard library such as Guava and Apache Commons each ended up building their own implementations. Of course, none of these implementations are compatible with the others. The alternative JVM languages also started shipping their own (universally mutually incompatible) implementations. With Java 8, we finally have a canonical implementation of that infrastructure and a lot more with it. The directly converted Java 8 version of this code looks like Listing 1-5. Listing 1-5. Java 8 Predicate Implementation of cloneWithoutNulls(List) public static <A> List<A> cloneWithoutNulls(final List<A> list) { List<A> toReturn = new ArrayList<>(list); toReturn.removeIf(Predicate.isEqual(null)); return toReturn; } But these predicates are not the extent of what Java 8 introduces. There’s a whole new syntax to succinctly express this idea: see Listing 1-6. That right arrow? That’s an example of the most exciting new feature of Java 8: it’s a lambda. But it goes even further. You will increasingly see Java code that looks like Listing 1-7. In this example, we have an interface (Comparator) with static and instance methods, and some real strangeness happening involving two colons. What is going on? That’s the Java 8 revolution, and that’s what this book is all about. 1Unfortunately, they don’t have the same capability with Lists specifically, so I had to do some work to get to the right type. 2 Chapter 1 ■ Java 8: It’s a Whole NeW Java Listing 1-6. Java 8 Lambda Implementation of cloneWithoutNulls(List) public static <A> List<A> cloneWithoutNulls(final List<A> list) { List<A> toReturn = new ArrayList<>(list); toReturn.removeIf(it -> it == null); return toReturn; } Listing 1-7. Example of Defining a Comparator of Java 8 Comparator c = Comparator .comparing(User::getLastName) .thenComparing(User::getFirstName); Java 8 Returns Java to the Forefront Software development is exciting because its core problems have not been solved yet: the practices, processes, and tools that go into software development are in a state of constant evolution and development. We can see this in the way that the dominant programming paradigm has changed: in the late ‘80s and through the ‘90s, the answer to every question was objects; now, programmers expect a wider palette with which to express code. Over the course of the last decade, programmers have begun borrowing more from a paradigm called “functional programming.” Whereas an object-oriented paradigm thinks about objects, and objects have behaviors, functional programming thinks in terms of verbs (functions), which act on nouns (arguments). Whereas object-oriented programming builds up a mechanism of interacting objects, functional programming builds up a function that is resolved through composite functions. An object-oriented program can be thought of as a corporate environment. You have a CEO (the “Main Class”), who issues high-level orders. Subordinates take up these orders. Although these subordinates each have some of their own data (their own files on their computer and their own stickie notes on their desk), these subordinates mostly delegate to their own subordinates. As you get farther down this chain, the subordinates become more technical (implementation-specific), with the end of the chain performing the last precise bit of work. Then the result of the action (either an exception or a return value) is communicated back up through the chain of subordinates to the CEO, who may then issue another command. Main Class Query Data Print Result Map DB Get DB Print to Query DB Results to Format Result Connection Standard Out Data Objects 3 Chapter 1 ■ Java 8: It’s a Whole NeW Java A functional program, however, is modeled on mathematics. The idea is that the inputs to the application are the arguments to a function, and the outputs of the application are the value that the function returns. Whereas the object-oriented subordinates may contain their own state, mathematical functions are stateless: they will always return the same value and have no side effects. A mathematical function is not so much a behavior as a statement of eternal truth: "f(2)" does not mean “apply 2 to f”, but rather “the value when f’s argument is bound to 2”. In functional programming, the goal is to have all the functions in the application behave like these mathematical functions. This function may be defined in terms of other functions, but the program is ultimately a single function application, not a series of executive commands. f(x,y) = g(x,y) + h(y) g(x,y) = k(x) + m(y) h(y) = y k(x) m(y) y Although functional programming and object-oriented programming are theoretically equivalent, some solutions are more naturally expressed in one paradigm versus the other: for a given problem, the intuitive solution within each paradigm can be quite different. In the example I gave at the start of this chapter, object oriented required us to create a “Predicate” object with the behavior of returning true if the argument is not null. In functional programming, we simply provide a function that returns true if the argument is not null: there’s no need for an object to hold onto that behavior. In this case, functional programming was the more intuitive solution to the problem because there was no state, nothing to encapsulate, and inheritance was nonsensical. Throughout the early 21st century, many forward-looking developers saw the value in the functional programming style. That style melds especially well with many development techniques that were then becoming popularized, such as test-driven development, and it was especially useful in highly distributed or highly concurrent environments. Many of the problems people were encountering—such as how to send not just data but also commands over a line—have natural solutions within functional programming style. Some technology pundits, including your humble author, predicted that a new functional programming language would arise that would render Java obsolete in much the way that Java has rendered C++ obsolete. Obviously, this did not happen on the Java Virtual Machine.2 What happened was actually stranger than we expected. Instead of a pure functional programming language arising, new object-oriented languages populated the development environment. These languages, although undoubtedly object oriented, also integrated techniques and capabilities from the functional programming styles. In these hybrid languages, the developer can work with functions like in the functional languages, but those functions can have state, making them more like objects. Ruby, Groovy, Scala, and JavaScript are all examples of this kind of language. After many years, Java has now joined the ranks with Java 8. Starting with Java 8, Java integrates some of these popular techniques into the core Java language itself. 2It has happened on Apple’s platform, where Swift is replacing Objective-C. It’s also worth noting that Clojure has made a valiant effort on the JVM front. Some of my peers will defend themselves by pointing to the relative success of Scala, but Scala is not actually a functional programming language: http://blog.enfranchisedmind.com/2009/05/ scala-not-functional/. 4 Chapter 1 ■ Java 8: It’s a Whole NeW Java Within the functional programming paradigm, one of the most popular techniques is lambdas, which are more informally called closures.3 A lambda is a set of instructions that can be saved as a variable, passed about through the program, and executed at a later time (possibly multiple times). To understand what this means, consider a simple for loop, such as in Listing 1-8: when the program execution encounters this loop, it executes the loop immediately and then moves on. Imagine, instead, that you could store the loop into a variable, and then pass it around, executing it later at some faraway location, such as in the made-up code in Listing 1-9. That is what lambdas allow you to do; Java 8’s version of this code is in Listing 1-10. Listing 1-8. Simple Loop for(Object it : list) { System.out.println(it); } Listing 1-9. Storing a Simple Loop (Not Real Code) // This is not Java code Loop printObjects = for(Object it : list) { System.out.println(it); } Listing 1-10. Storing a Simple Loop in Java 8 Consumer<Iterable> printObjects = list -> { for(Object it : list) { System.out.println(it); } }; // For the record, this is the same result in more idiomatic Java 8 code: Consumer<Iterable> printObjects = list -> list.forEach(System.out::println); This may seem like a relatively minor language feature, but the impact of introducing lambdas was so significant that the lambdas were too big to be a part of Java 7’s Project Coin,4 and instead became the driving feature of Java 8. This language feature is so significant because it enables a new way of interacting with the objects that make up a Java program: now, we can have behaviors without needing to reference an object; and we can define verbs without first having to define a noun. The rest of this book will explore the implications of this new language feature, and how we can change the way that we write Java code to take advantage of this change. The result will be more functional code in both respects: both more function-oriented code, and also code that works better. The reason is because we can now express more directly and succinctly our intent in certain very common cases, and we have a powerful new tool for decomposing our code into more maintainable pieces. To understand how this works, let’s turn to a very specific and likely familiar situation in Java. 3We will get into the language of “closure” vs. “lambda” later on. In general, a developer can get along just fine treating them as synonyms. Java prefers “lambda,” whereas most other languages on the Java Virtual Machine prefer “closure.” Ruby, notably, has seven variations on “lambda” and “closure,” leading to this quite entertaining exploration of the programming language: http://innig.net/software/ruby/closures-in-ruby. 4Project Coin was a project for small change. (Get it?) Introducing lambdas was initially proposed as a bit of syntactic sugar, but it quickly became clear that the implications of having lambdas would have wide-reaching ramifications for the Java SDK’s API. 5 Chapter 1 ■ Java 8: It’s a Whole NeW Java Java Has Had Functional Programming All Along If you ask around about Java 8 and lambdas, you may hear someone say that Java 8 makes it possible to “do functional programming” in Java. Yet the reality is that Java has always been capable of functional programming: it’s just been disguised and awkward. What lambdas do is remove the disguise and make it easier to work with functions directly. Although I could (and many do) try to explain this fact through abstract arguments, this is best demonstrated through an example of practical Java code and how it changes with the introduction of lambdas. We will draw our example from the world of database access, since it is a place near to my heart. Before the rise and fall of Object-Relational Mapping (ORM) frameworks5 and the innovation of other data access tools such as JOOQ,6 there was the venerable JdbcTemplate from Spring’s JDBC support library.7 This class—with help from a broad family of interfaces and supporting classes—allows developers to avoid the coding overhead of interacting with a database, and keeping both developers and the code focused on the relevant and interesting bits of code. JdbcTemplate does this through a pattern called “Inversion of Control.” The basic idea is that you can get rid of boilerplate by having some library execute the boilerplate, and then calling back to client code with the relevant pieces of context. The “control” is “inverted” because calling into a library normally gives control to that library, but in this case the library is giving control back to the caller.8 A standard example of using JdbcTemplate this way (before Java 8) is given in Listing 1-11. The boilerplate creation of the statement, execution of the statement, iteration over the result set, closing of objects, and handling of errors is all managed by JdbcTemplate. The arguments to the JdbcTemplate call fill in the gaps: the SQL given as the first argument is used to prepare a statement; the interface implementation passed as the second argument is used to assign the parameters; and the interface implementation passed as the third argument is used to consume each row of the result set. The lambdas introduced in Java 8 make this kind of code much easier to write, much more succinct, and much more expressive. The Java 8 example is given in Listing 1-12. Using lambdas drops the code from 16 lines to 4. How? Why did Java once require this overhead, and how does the introduction of lambdas enable shorter code? Listing 1-11. Example of JdbcTemplate usage before lambdas jdbcTemplate.query("SELECT bar FROM Foo WHERE baz = ?", new PreparedStatementSetter() { @Override public void setValues(PreparedStatement ps) throws SQLException { ps.setString(1, "some value for baz"); } }, new RowMapper<SomeModel>() { @Override public SomeModel mapRow(ResultSet rs, int rowNum) 5E.g., Hibernate. 6http://www.jooq.org/. 7API at http://bit.ly/jdbcTemplate. 8The more generalized version of the “Inversion of Control” pattern is the “Hole in the Middle” pattern. http://blog.enfranchisedmind.com/2007/07/the-hole-in-the-middle-pattern/. 6