Ant: The Definitive Guide Jesse Tilly Eric Burke Publisher: O'Reilly First Edition May 2002 ISBN: 0-596-00184-3, 288 pages Ant is the premier build-management tool for Java environments. Ant is part of Jakarta, the Apache Software Foundation's open source Java project repository. Ant is written entirely in Java, and is platform independent. Using XML, a Java developer describes the modules involved in a build, and the dependencies between those modules. Ant then does the rest, compiling components as necessary in order to build the application. Table of Contents Dedication.............................................................................................................................1 Foreword...............................................................................................................................2 Preface...................................................................................................................................5 Structure of This Book.......................................................................................................5 Audience.............................................................................................................................7 What You Should Know....................................................................................................7 Which Platform and Version..............................................................................................7 Conventions Used in This book.........................................................................................7 Comments and Questions...................................................................................................8 Acknowledgments..............................................................................................................9 Chapter 1. Ant Jumpstart.................................................................................................10 1.1 Files and Directories...................................................................................................10 1.2 The Ant Buildfile.......................................................................................................11 1.3 Running Ant...............................................................................................................13 1.4 Ant Command-Line Reference..................................................................................15 1.5 Buildfile Outline.........................................................................................................17 1.6 Learning More............................................................................................................17 Chapter 2. Installation and Configuration......................................................................18 2.1 The Distribution.........................................................................................................18 2.2 Installation..................................................................................................................18 2.3 Configuration.............................................................................................................25 Chapter 3. The Buildfile....................................................................................................28 3.1 Why XML?................................................................................................................28 3.2 Ant Building Blocks...................................................................................................29 3.3 An Example Project and Buildfile.............................................................................33 3.4 The Buildfile Execution Process................................................................................45 3.5 AINASL: Ant Is Not a Scripting Language...............................................................54 3.6 Buildfile Authoring Issues.........................................................................................56 Chapter 4. Ant DataTypes.................................................................................................58 4.1 DataTypes Defined.....................................................................................................59 4.2 XML Attribute Conventions......................................................................................59 4.3 Argument DataType...................................................................................................61 4.4 Environment DataType..............................................................................................64 4.5 FileList DataType.......................................................................................................66 4.6 FileSet DataType........................................................................................................67 4.7 PatternSet DataType...................................................................................................70 4.8 FilterSet DataType.....................................................................................................72 4.9 Path DataType............................................................................................................74 4.10 Mapper DataType.....................................................................................................76 Chapter 5. User-Written Tasks.........................................................................................80 5.1 The Need for Custom Tasks.......................................................................................80 5.2 Ant's Task Model.......................................................................................................81 5.3 The Task Life Cycle...................................................................................................89 5.4 An Example Through Analysis: The jar Task............................................................92 5.5 Miscellaneous Task Topics......................................................................................106 Chapter 6. User-Written Listeners.................................................................................109 6.1 The BuildEvent Class...............................................................................................109 6.2 The BuildListener Interface.....................................................................................111 6.3 An Example: XmlLogger.........................................................................................113 6.4 The Parallel Problem................................................................................................115 Chapter 7. Core Tasks.....................................................................................................117 7.1 Task Summary..........................................................................................................117 7.2 Common Types and Attributes................................................................................119 7.3 Project and Target....................................................................................................121 7.4 Core Task Reference................................................................................................124 Chapter 8. Optional Tasks...............................................................................................212 8.1 Task Summary..........................................................................................................212 8.2 Optional Task Reference..........................................................................................213 Appendix A. The Future of Ant......................................................................................300 A.1 Ant2.........................................................................................................................300 A.2 Ant1 RIP 2002?.......................................................................................................301 Appendix B. Ant Solutions..............................................................................................302 B.1 Testing Library Availability....................................................................................302 B.2 Cleaning Up Does More Than Keep Things Neat...................................................303 B.3 Using Ant to Consolidate Libraries.........................................................................303 B.4 Documenting the Buildfile's Targets.......................................................................305 B.5 Setting Properties Outside of the Buildfile..............................................................305 B.6 Using pathconvert....................................................................................................306 B.7 Usage Statements.....................................................................................................307 B.8 Forking Processes....................................................................................................308 B.9 Using Cascading Projects and Buildfiles.................................................................309 Colophon...........................................................................................................................312 Ant: The Definitive Guide Dedication For my grandmother, Dorothy Tilly. —Jesse Tilly For Mom. —Eric M. Burke 1 Ant: The Definitive Guide Foreword I have to confess that I had absolutely no idea that Ant, the little build tool that could, would go as far as it did and make such a mark on the Java developer community. When I wrote the first version of Ant, it was a simple hack to help me solve a cross-platform build problem that I was having. Now it's grown up and being used by thousands of developers all over the planet. What's the magic behind this? How did this little program end up being used by so many people? Maybe the story of how Ant came to be holds some clues. Ant was first written quite some time before it was checked into Apache's CVS servers. In mid-1998, I was given the responsibility at Sun Microsystems to create the Java Servlet 2.1 specification and a reference implementation to go with it. This reference implementation, which I named Tomcat, was to be a brand new codebase, since the previous reference implementation was based somewhat on code from the Java Web Server, a commercial product that was migrated from JavaSoft to iPlanet. Also, the new implementation had to be 100% Pure Java. In order to get the 100% Pure Java certification, even for those of us working on the Java Platform at Sun, you had to show Key Labs (an independent certification company) that you could run on three different platforms. To ensure that the servlet reference implementation would run anywhere, I picked Solaris, Windows, and the Mac OS. And not only did I want Tomcat to run on these three platforms, but I wanted to be able to build and develop on all three platforms as well as on Linux. I tried using GNU Make. And shell scripts. And batch files. And God knows what else. Each approach had its own unique problem. The problems stemmed from the fact that all of the existing tools had a worldview rooted in building C programs. When these practices were applied to Java, they worked, but slowly. Even though Java programs themselves can perform well, the startup overhead associated with the Java Virtual Machine is lengthy. And when Make creates a new instance of the VM with every file that needs to be compiled, compile times grow linearly with the number of source files in a project. I tried many approaches to write a make file that would cause all of the source files in a project that needed to be recompiled to be passed to javac in one go. But, no matter how hard I tried, and how many Make wizards I consulted with, I couldn't get an approach that would work the same way on multiple platforms. I got very, very tired of the !&#$%#ing tab formatting of make files. As much as I've been a proponent of Emacs in my life, any tool that requires Emacs to properly write its files so that you can make sure that no unintended spaces creep in should not be tolerated.1 It was on a flight back from a conference in Europe that I finally got fed up once and for all of trying to create some make file that would work the same way everywhere. I decided to "make" my own tool: one that would examine all the Java source files in a project, compare them with any compiled classes, and pass the list of sources that needed to be compiled directly to javac. In addition, it would do a couple of other things like stuff all the classes into a JAR file and copy some other files around to make a distributable version of the software. In order to ensure that things would work the same way on every supported platform, I decided to write the tool in Java. 1 I've been told that the original designer of the make file format knew after the first week that the tab thing would be a problem. But he already had dozens of users and didn't want to break compatibility. 2 Ant: The Definitive Guide A few hours later, I had a working tool. It was simple, crude, and consisted of just a few classes. It used the functionality of java.util.Properties to serve as its data layer. And it worked. Beautifully. My compile times dropped by an order of magnitude. When I got back to the states and tested it out on Solaris, Linux, and Mac OS, it worked just fine on all of them. Its biggest problem at that time was that the number of things it could do was limited to compiling files and copying files — and that this functionality was hardcoded. A few weeks later I showed the tool, which I had named Ant because it was a little thing that could build big things,2 to my friend Jason Hunter (author of Java Servlet Programming, published by O'Reilly). Jason thought that it was a decent enough tool, but didn't really think it was a big deal. That is, until I mentioned that I was thinking of using Java's reflection abilities to provide a clean way to extend Ant's abilities so that a programmer could write their own tasks to extend it. Then the light bulb went off over his head and I had my first Ant user as well as evangelist. Jason also has an uncanny ability to find a bug in any piece of software within moments and helped me stomp out quite a few problems. Once the reflection layer was in place, I wrote a few more tasks and Ant became useful to other groups at Sun. However, the build file format was getting a bit bulky. Properties files don't really lend themselves to hierarchical grouping well, and with the introduction of tasks came the idea of targets (collections of tasks). I played around with a few different ways of solving the problem, but hit on the solution when I was on another flight back from Europe. This solution structured the project-target-task hierarchy to follow an XML document hierarchy. It also leveraged the reflection work I had done earlier to associate XML tag names with task implementations. Evidently I do my best coding while flying over the ocean. I wonder if there's something about the increased radiation at high altitude that helps. Or maybe trips to Europe bring out something creative in me. Only more experimentation will tell. Ant, as we know it, had come into being. Everything that you see in the version of Ant that you use today (the good and the bad) is a result of the decisions made up to that point. To be sure, a lot has changed since then, but the basics were there. It was essentially this source code that was checked into Apache's CVS repository alongside Tomcat in late 2000. I moved on to other things, principally being Sun's representative to the Apache Software Foundation as well as working on XML specifications such as JAXP from Sun and DOM from the W3C. Amazingly enough, people all over the world started talking about Ant. The first people to find it were those that worked on Tomcat at Apache. Then they told their friends about it. And those friends told their friends, and so on. At some point more people knew about and were using Ant than Tomcat. A strong developer and user community grew up around Ant at Apache, and many changes have been made to the tool along the way. People now use it to build all manner of projects, from very small ones to incredibly huge J2EE applications. The moment I knew that Ant had gone into the history books was during JavaOne in 2001. I was at a keynote presentation in which a new development tool from a major database software company was being demoed. The presenter showed how easy it was to draw lines between boxes to design software, and then hit the build button. Flashing by in the console 2 Also, the letters ANT could stand for "Another Neato Tool." Silly, I know. But true. 3 Ant: The Definitive Guide window were those familiar square brackets that every user of Ant sees on a regular basis. I was stunned. Floored. The number of Ant users continues to increase. Evidently the little itch that I scratched is shared by Java developers world wide. And not just Java developers. I recently stumbled across NAnt, an implementation of Ant's ideas for .NET development.3 If I had known that Ant was going to be such a runaway success, I would have spent a bit more time on it in the first place polishing it up and making it something more than the simple hack it started out as. Yet that might have defeated exactly the characteristic that made it take off in the first place. Ant might have become over-engineered. If I had spent too much time trying to make it work for more than just my needs, it might have become too big a tool and too cumbersome to use. We see this all the time in software, especially in many of the Java APIs currently being proposed. It might be that the secret to Ant's success is that it didn't try to be successful. It was a simple solution to an obvious problem that many people were having. I just feel honored to be the lucky guy who stumbled across it. The book you now hold in your hands will guide you in using Ant as it exists today. Jesse and Eric will teach you how to use Ant effectively, extend it, and tell you how all the various tasks, both the built-in ones as well as widely used optional ones, can be used. In addition, they will give you tips to avoid the pitfalls created by some of Ant's design decisions. Before placing you in their capable hands, I want to leave you with just one last thought: always scratch your own itch where possible. If a tool out there doesnt do what you need it to do, then look around for one that will. If it doesnt exist, then create it. And be sure to share it with the world. Thousands of other people might have just the same itch that you do. —James Duncan Davidson San Francisco, CA, April 2002 3 You can find NAnt at http://nant.sourceforge.net/. 4 Ant: The Definitive Guide Preface Compilation of all Java™ source files is no longer the only step necessary to build many Java-based projects. For the typical HelloWorld program, book examples, and simple applets, source file compilation is sufficient. Complex Java-based projects, like web applications or Swing-based programs (such as JBuilder), require much more. Up-to-date sources must be retrieved from source control. Dependencies not automatically handled by the Java compiler need to be managed. Various classes must be bundled and delivered to multiple locations, sometimes as JAR or WAR files. Some Java technologies, such as Enterprise Java Beans (EJB) and Remote Method Invocation (RMI) classes, need separate compilation and code generation steps not performed by the Java compiler. While shell scripts and GNU Make are often the first choice tools for performing these alternative tasks — in terms of "getting the job done," these tools perform adequately — they turn out to be poor choices in the long run. As functional as it may be, GNU Make leaves a lot to be desired in terms of ease-of-use. Makefiles have their own language syntax, requiring a separate knowledge set for their authors. GNU Make lacks platform-independence, requiring multiple versions of the same makefile (one for each target platform) to be maintained and distributed. The nature of shell scripts and GNU Make (remembering that GNU Make is simply a language extension on top of an existing shell) makes moving from operating system to operating system, and even from shell to shell, difficult or impossible for anyone but an expert user. While it is not unusual to use GNU Make, the time and maintenance required to follow this path is too high for modern Java-based projects. Sun provides Java versions of all their SDK tools. Executables such as javac are simply wrappers executing the Java code. Other vendors' tools, like BEA's EJB compiler for WebLogic, JUnit, and the Jakarta tools and libraries are all written in Java. GNU Make can only call executables from the command line. For example, to invoke a Java class, GNU Make must use the java command to invoke the JVM, and pass the class name as a command- line argument. Make is incapable of programmatically using any of the Java tools' libraries, such as exception and error objects. These libraries allow for a more flexible build process. A tool written in Java (such as WebLogic's ejbc compiler) can share information from exceptions and errors with other objects (such as Ant task objects) available inside the same JVM. This serves to enhance the build process beyond command-line return codes and after- the-fact error-message string parsing. The problems with GNU Make and the possibilities of a build tool written in Java influenced James Duncan Davidson to write Ant. Ant runs the Java compiler as a class, not as a call from the command line. Remaining inside the JVM allows for specialized code to handle errors, and for action on results Sun provides through its compiler. Ant uses XML as its buildfile syntax, therefore enhancing, rather than straining, developers' and project managers' skill sets. Ant extends the build process beyond just running programs, and is more properly termed a build environment than a build tool. Structure of This Book Ant: The Definitive Guide contains all of the knowledge a newcomer to Ant needs. For the Ant expert, Ant: The Definitive Guide is a reference, providing detailed definitions of 5 Ant: The Definitive Guide Ant's core tasks, discussing the main features of Ant, providing some best practices for managing projects with Ant, and explaining workarounds for some of Ant's problems. Chapter 1, walks through a very basic Ant buildfile example, with the intent of getting you up and running quickly. We show how to create directories, compile code, and generate a JAR file, but do not delve into the details of how everything works. The chapter also includes detailed information on Ant's command-line usage. We conclude with a rough outline of a buildfile for use as a starter template. Chapter 2, shows how to get Ant, install it, and configure it on Windows and Unix platforms. We list some of the pitfalls found with these development platforms, as well as provide workarounds and solutions. Chapter 3, shows an example Ant buildfile in the context of a sample project. We dissect and describe the major parts and structures of the buildfile, explaining such things as the general flow of the Ant engine and the benefits of Ant using XML, highlighting the major parts of a buildfile. Chapter 4, describes each of the Ant DataTypes in detail. While DataTypes are used in earlier chapters, this is where we really dig into them. We show how to use environment variables and pass command-line arguments to processes, as well as how to work with lists of files and patterns. Chapter 5, covers one of Ant's best features: the ability to extend Ant. With the capability to write extensions, you are able to handle anything a particular project may require. As a bonus, you can reuse your tasks in future projects, reaping benefits from your effort well beyond the initial implementation. Your tasks can even be shared and distributed publicly so that people you don't even know can benefit from your work. Chapter 6, covers how to design and develop your own build-event listeners. With these, you can write classes that perform operations based on the flow related to buildfile processing. These operations range from sending emails when certain complex tasks complete, to redirecting the same events to a centralized "build-listening framework." The possibilities, just like with user-written tasks, are endless. The chapter also covers a further extension to listeners: user-written loggers. With these, you can augment or even replace Ant's default logging system. Chapter 7, is a comprehensive reference to the entire set of core Ant tasks. For each task, you'll find a description, a list of Ant versions supporting the task, and definitions for all task attributes. You'll also find helpful samples of task usage. Chapter 8, provides a reference, similar in form to Chapter 7, for Ant's rich library of optional tasks. Appendix A, discusses just that. We cover future directions and expected new features, as well as suggest steps you can take to avoid using soon-to-be-obsolete features. Appendix B, delves into some of the more common ways Ant is used to solve various build problems. Additionally, we talk about using buildfiles with cascading project structures. These are project structures with a primary project directory and many subproject 6