Explaining Testing to Anybody James Bach, Satisfice, Inc. [email protected] www.satisfice.com Uhoh… Howdoyourespondwhensomeone youworkwithdoesn’tunderstand howtestingworks? 1 “In the testing process, test plans are made, test cases specified, tests databases and files set up, test output created, and so on. It is important to regard this testware not as a disposable product, but as one that has to be filed, managed, and handed over for are you listening?” reuse upon— head… heavy… strength… fading… zzzzzzzz… Basics of Explaining (cid:131) Be quick, then stop and check in. (cid:131) Be humble. Exceptions and alternatives abound. (cid:131) You can be wrong, as long as you’re thoughtful. (cid:131) Be real. Own the explanation. (cid:131) Be respectful. Honor objections and questions. (cid:131) Be patient. Often, experience must come first. (cid:131) Be relevant. How does it relate to them? What’s the bottom-line? What do you want them to do? 2 Basics of Explaining (cid:131) Be prepared − Quickpoint (afewsentences at most.) − Simile &Metaphor(relatetosomethingfamiliar) − Vocabularycheck (are youusingwords thesame way?) − 5-minutewhiteboardtalk(diagrams arepowerful) − Concreteexamples or demonstration(in-housedata) − Anticipatecommonquestions andobjections − References &supportingmaterial (haveit handy) Explanations (cid:131) Confusions − “What is testing?” − “What’s sohardabout testing?” − “Howdo youknow when you’redone?” − “Whydidn’t youfindthat bug?” (cid:131) “Let’s {do my crazy plan},okay?” − “…changetheproduct at thelast minute…” − “…specifythosetests inadvance[andautomatethem]… 3 “What is Testing?” (cid:131) Quick points (choose one): − Testingis organizedskepticism.It’s thebeliefthat things maynot beas theyseem; that things couldbedifferent. − Testingis comparingthe ambiguous totheinvisible,soas toavoidtheunthinkablehappeningtotheunknown. − Testingbrings vital informationtolight,sowecansee where weareandmakebetterdecisions. − Testingis asupport functionthat helps developers look goodbyfindingtheirmistakes before anyoneelse does. “What is Testing?” (cid:131) Similes and Metaphors − Policepatrol (scanningfortroubleinacomplexworld) − Secret Service (makingriskyactivities safer) − Radar (threat detectionwhileavoidingfalsealarms) − Copyediting (savingauthors from embarrassment) − Soccer/hockeydefense(wedefendthegoal) − Science(conjectureandrefutation) 4 “What is testing?” Project Environment Test Techniques Quality Product Criteria Elements Perceived Quality “What is testing?” Logistics Problem Project Environment Coverage Oracle Problem Test Problem Techniques Quality Product Criteria Elements Reporting Perceived Problem Quality 5 “What is testing?” (cid:131) Supporting Material − http://www.kaner.com/imposs.htm Explainstheimpossibilityofcompletetesting. − http://www.satisfice.com/articles/test_automation_snake_oil.pdf − http://www.kaner.com/lawst1.htm Explainthechallengesofregressiontestautomation. − http://www.satisfice.com/articles/good_enough_testing.pdf Explainstestinginthecontextofhowmuchisenough. − http://www.rstcorp.com/marick/classic/mistakes.html Agoldmineofmarvelouslittleexplanations. Explanation Hint Try highlighting one key dynamic, instead of explaining the whole issue. 6 “What’s so hard about testing?” Data smoke testing reliabilitytesting Functions function testing risktesting Explanation Hint Try beginning by explaining things in terms of the mission, then get more concrete from there. 7 “How do you know when you’re done?” 1. When management has enough information to make an informed decision about the benefits and risks associated with the product. AND 2. Management decides to release or cancel the project. Bothmustbetrueor testingisn’tdone. Explanation Hint Use a diagram that helps you examine different scenarios and issues as part of the dialog. 8 “How do you know when you’re done?” Test Cycle Convergence GoodEnoughQuality Testsfailed:knownbad y CycleStart CycleEnd t ali } Testsnotrun: u Unknown Q Build UntestableQuality Testspassed:knowngood Time “How do you know when you’re done?” Partial Convergence: duetoincompletetestsuite y t i l a u Q Time 9 “How do you know when you’re done?” Slow Convergence: duetoinefficienttestingorlowtestability y t i l a u Q Time “How do you know when you’re done?” No Convergence: duetointerruptedtestcycle y t i l a u Q Time 10