Early Praise for Explore It! Reading this book taught me new skills and heuristics; but even better, it helped me channel my tester “spidey sense” more creatively and usefully. I keep this book handy at all times and occasionally do one of the practice sessions to keep my awareness keen. Explore It! helps me make sure our customers and our com- pany get real value from our software. It’ll help you too. ➤ Lisa Crispin Coauthor with Janet Gregory, Agile Testing: A Practical Guide for Testers and Agile Teams Explore It! starts with a bang. Elisabeth catches your imagination and has filled the book with practical ideas for exploring everything from your typical GUI sce- narios to testing ideas (requirements), and she even includes suggestions for programmers on how to explore low-level code. This book should be on every de- velopment team member’s desk, not only testers. It is the book I carry with me whenever I introduce exploratory testing to development teams. ➤ Janet Gregory Coauthor with Lisa Crispin, Agile Testing: A Practical Guide for Testers and Agile Teams At Moolya, we were wondering how to get newbie testers started with exploratory testing. Explore It! appears to be timely and useful to the problem we have in hand. ➤ Pradeep Soundararajan Founder, Moolya Software Testing Private Limited There’s a hidden secret to Explore It! While attempting to capture the nuance and depth of exploratory testing, Elisabeth Hendrickson has managed to write one of the best books on test design I’ve ever read. Good test design comes from good test ideas, and this book is overflowing with both great test ideas and pragmatic stories to back them up. ➤ Alan Page Microsoft Elisabeth Hendrickson has been working on a book ever since I met her many years ago. This book was worth waiting for! Explore It! explores exploratory testing, a topic that desperately needed a well-written book. Not only is this book clear, good, and useful, but Elisabeth’s stories also make it a fun read. I recommend exploring it. ➤ Bas Vodde Odd-e Explore It! is packed with a ton of practical advice on planning and executing ex- ploratory testing, a topic that is becoming increasingly popular with Agile teams. As an introduction to the topic, the book should be interesting to anyone serious about quality on Agile projects, but it will be particularly useful to testers working with Agile teams as a reference for things to investigate. I found the chapters on various heuristics for finding interesting things to explore particularly interesting, including spotting variables, varying sequences and interactions, and exploring state transitions. ➤ Gojko Adzic Elisabeth has written a book that should be on every team’s bookshelf, well thumbed and frequently referenced! She explains what exploratory testing is—and very importantly, what it isn’t—in an accessible and easy-to-understand style. She tells interesting stories and offers examples that explain and clarify the advice she gives. I heartily recommend this book to anyone who is involved in testing real-world software systems. ➤ Shane Hastie When I saw this book announced, I knew I had to read it as soon as possible. I was amazed, delighted, and overwhelmed by the practical guidance that Elisabeth provides. This book closes the gap on exploratory testing on the recommended reading list for my ET classes. ➤ Markus Gärtner Author of ATDD by Example: A Practical Guide to Acceptance Test-Driven Devel- opment, it-agile GmbH Elisabeth Hendrickson’s terrific new book is an essential guide not only for testers but also for the whole team. New testers will learn how to generate innumerable useful test ideas quickly. Experienced testers will gain nuance and fresh insight into techniques they’ve used for years, and they will learn how to apply their existing skills to help their teams in new ways. Managers and developers will learn how exploratory testing can help them stay productive and keep out of trouble before, during, and after development. ➤ Dale Emery Consultant Explore It! Reduce Risk and Increase Confidence with Exploratory Testing Elisabeth Hendrickson The Pragmatic Bookshelf Dallas, Texas • Raleigh, North Carolina Many of the designations used by manufacturers and sellers to distinguish their products are claimed as trademarks. Where those designations appear in this book, and The Pragmatic Programmers, LLC was aware of a trademark claim, the designations have been printed in initial capital letters or in all capitals. The Pragmatic Starter Kit, The Pragmatic Programmer, Pragmatic Programming, Pragmatic Bookshelf, PragProg and the linking g device are trade- marks of The Pragmatic Programmers, LLC. Every precaution was taken in the preparation of this book. However, the publisher assumes no responsibility for errors or omissions, or for damages that may result from the use of information (including program listings) contained herein. Our Pragmatic courses, workshops, and other products can help you and your team create better software and have more fun. For more information, as well as the latest Pragmatic titles, please visit us at http://pragprog.com. The team that produced this book includes: Jackie Carter (editor) Potomac Indexing, LLC (indexer) Molly McBeath (copyeditor) David J Kelly (typesetter) Janet Furlow (producer) Juliet Benda (rights) Ellie Callahan (support) Copyright © 2013 The Pragmatic Programmers, LLC. All rights reserved. No part of this publication may be reproduced, stored in a retrieval system, or transmitted, in any form, or by any means, electronic, mechanical, photocopying, recording, or otherwise, without the prior consent of the publisher. Printed in the United States of America. ISBN-13: 978-1-937785-02-4 Encoded using the finest acid-free high-entropy binary digits. Book version: P2.0—September 2013 Contents Foreword . . . . . . . . . . . . . . xi Acknowledgments . . . . . . . . . . . xiii Preface . . . . . . . . . . . . . . xv Part I — Establishing Foundations 1. On Testing and Exploration . . . . . . . . . 3 1.1 Two Sides of Testing 4 1.2 Essential Elements of Exploratory Testing 6 1.3 Working in Time-Boxed Sessions 7 1.4 Practice Sessions 8 2. Charter Your Explorations . . . . . . . . . 11 2.1 Chartering the Expedition 11 2.2 A Simple Charter Template 13 2.3 Good Charters 15 2.4 Generating Charters 16 2.5 The Nightmare Headline Game 21 2.6 Planning Charters 24 2.7 Practice Sessions 24 3. Observe the Details . . . . . . . . . . . 25 3.1 But Did You See the Moonwalking Bear? 25 3.2 Digging Deeper 27 3.3 Testability and Making the Invisible Visible 30 3.4 Consoles and Logs 31 3.5 Practice Sessions 32 Contents • viii 4. Find Interesting Variations . . . . . . . . . 33 4.1 Variables Are Things That Vary 34 4.2 Subtle Variables, Big Disasters 36 4.3 Identifying Variables 38 4.4 Variables! They’re Everywhere! 44 4.5 Practice Sessions 44 5. Evaluate Results. . . . . . . . . . . . 47 5.1 Never and Always 48 5.2 Alternative Resources 50 5.3 Approximations 52 5.4 Practice Sessions 55 Part II — Adding Dimensions 6. Vary Sequences and Interactions . . . . . . . 59 6.1 Nouns and Verbs 60 6.2 Random Navigation 62 6.3 Personas 63 6.4 Practice Sessions 64 7. Explore Entities and Their Relationships . . . . . 67 7.1 Recognizing Entities, Attributes, and Dependencies 67 7.2 CRUD: Create, Read, Update, Delete 70 7.3 Following the Data 73 7.4 Practice Sessions 73 8. Discover States and Transitions . . . . . . . . 75 8.1 Recognizing States and Events 76 8.2 Diagramming a State Model 78 8.3 Exploring with State Models 80 8.4 Changing the Representation: State Tables 82 8.5 Practice Sessions 85 9. Explore the Ecosystem . . . . . . . . . . 87 9.1 Diagramming the Ecosystem 87 9.2 Trust Boundaries 92 9.3 What If? 93 9.4 Follow the Data 95 9.5 Practice Sessions 96 Contents • ix Part III — Putting It in Context 10. Explore When There Is No User Interface . . . . . . 99 10.1 Exploring an API 100 10.2 Exploring a Programming Language 102 10.3 Exploring a Web Service 104 10.4 Characterizing Bugs 106 10.5 Practice Sessions 107 11. Explore an Existing System . . . . . . . . 109 11.1 Starting with a Recon Session 109 11.2 Sharing Observations 112 11.3 Interviewing Stakeholders to Gather Questions 114 11.4 Selecting Techniques 116 11.5 Recording Your Findings 116 11.6 What About the Dreaded Unreproducible Bugs? 117 11.7 Practice Sessions 119 12. Explore Requirements . . . . . . . . . . 121 12.1 Getting Into the Requirements Meeting 122 12.2 In the Requirements Meeting 125 12.3 Charter During Requirements Discussions 128 12.4 Active Reading 130 12.5 Practice Sessions 132 13. Integrate Exploration Throughout . . . . . . . 133 13.1 Exploring as Part of Your Test Strategy 133 13.2 Paired Exploration 137 13.3 Discovering Systemic Sources of Problems 138 13.4 Estimating Exploration 140 13.5 How to Tell When You Have Explored Enough 141 13.6 Debriefing Stakeholders 142 13.7 Capturing Useful Nuggets of Wisdom 144 13.8 Practice Sessions 145 A1. Interviewing for Exploratory Testing Skills . . . . . 147 A1.1 Paired Exploratory Testing as Part of an Interview 148 A1.2 Assessing the Candidates 150