Development testing of Object-Oriented software Erkki P(cid:246)yh(cid:246)nen, [email protected] Nokia, Technology Platforms Product Creation Services and Tools Few slides by Roland M(cid:252)ller& TeroLaine, Nokia 1 'NOKIA Development testing of OO SW.ppt/ 2004-05-04 / EAP Introduction (cid:149) Background (cid:149)OO Testing takes mostly place in lower abstraction levels (cid:149)Development testing (Unit, Component, Module, Program testing) has a big effect but it is badly managed activity (cid:149)Overlooked by traditional testing approaches, testing professionals and project management (cid:149) Objectives (cid:149)To highlight how different practices support each other (cid:149)Bring awareness for both development and testing roles 2 'NOKIA Development testing of OO SW.ppt/ 2004-05-04 / EAP Contents: Development testing of OO SW (cid:147)If there are two or more ways to do something, and one of thoseways can result in a catastrophic results, then someone will do it.(cid:148) Edward A. Murphy, Jr. 1. Objects are different - in good and bad (cid:149) What has changed in OO SW organizations 2. It is still software (cid:149) Still no silver bullet 3. Models are helping (cid:149) Being early has its benefits in many ways 4. Focus, focus, focus (cid:149) Strategy for winning the fight 3 'NOKIA Development testing of OO SW.ppt/ 2004-05-04 / EAP 1/4 Objects are different - in good and bad (cid:149) Testability (visibility and controllability) in low level will suffer (cid:149) Because of the information hiding (cid:149) Because of the active objects (cid:149) Improved support for managed unit testing in class or component levels (cid:149) Test harnesses (drivers and stubs) flexible to implement using available programming tools and skills (cid:149) Low level testing parallel with development means fast feedback (cid:149) Models provide an effective way for preventing software faults by testing the abstractions = inspecting SW models at early stage 4 'NOKIA Development testing of OO SW.ppt/ 2004-05-04 / EAP Object benefits (cid:149) Object-Oriented methods have a great potential in normal case over older methods (cid:149) It takes a lot more than using an object-oriented language to really achieve the potential benefits (cid:149) Agreed analysis and design practices help to achieve SW asset (models/documents & code) transferability across organization (cid:149) Proper OO competence development with agreed modeling methods (UML with style guide) help to achieve better quality models andcode (cid:149) Well-managed system and application architecture models is the basis for SW asset reuse and testing (cid:149) Remember: large and complex problems arelarge and complex -no matter which software engineering approach is used (cid:149) Software is now bigger and more complex, but quality has not improved (Les Hatton) 5 'NOKIA Development testing of OO SW.ppt/ 2004-05-04 / EAP Different concerns, different models Development process Use case Object Sequence Collaboration Class Statechart Activity Deployment Component Structure Behavior (cid:149)Different model types are used in different project phases and for different modeling targets (static/structural to dynamic/behavior) (cid:149)But: different abstraction levels can share model types (cid:149) Example: class models used in levels from whole system to few classes 6 'NOKIA Development testing of OO SW.ppt/ 2004-05-04 / EAP One good source (cid:149) (cid:147)Testing Object-Oriented Systems(cid:148) by Robert V. Binder (Addison-Wesley 2000, ISBN 0-201-80938-9), rbsc.com (cid:149) Tells how to test Object-Oriented Software (surprise ☺) (cid:149) What is special about OO SW for testing? (cid:149) Collection of tens of OO testing and integration patterns (cid:149) Binder speaks about: (cid:149)(cid:149)MMooddeellssthat are necessary for test design (cid:149)(cid:149)PPaatttteerrnnssexpressing best practices for test design (cid:149)(cid:149)TToooollssassisting in the implementation of the test design 7 'NOKIA Development testing of OO SW.ppt/ 2004-05-04 / EAP Deriving Tests from UML Diagrams (cid:149) Use Case (cid:149) Information from UML use cases is typically not specific enough for testing: (cid:149) Therefore testing should be done with scenarios or extended use cases (cid:149) State Machines (cid:149) Sequence of transitions (cid:149) Class Diagram (cid:149) Relationships between objects (1:1, 1:n, life cycle) (cid:149) Sequence Diagram (cid:149) Sequence of messages and triggered responses (cid:149) Activity Diagram (cid:149) Action 1 is followed by Action 2 (cid:149) OCL= Object Constraint Language (cid:149) Straightforward usage for invariants, preconditions, postconditions 8 'NOKIA Development testing of OO SW.ppt/ 2004-05-04 / EAP Model-based testing Meta-Model Consistency Checking CompletenessChecking Required Component Behavior Representation Validation RespoTnessitb miliotRyde-esblpaosnesdi bility-based Testing Implementation-based Test model Observed Component Behavior Implementation Implementation-based Testing 9 'NOKIA Development testing of OO SW.ppt/ 2004-05-04 / EAP An OO Testing manifesto [BIND00, ch. 4.5] Technical and process issues (cid:149) Reuse does notreduce the amount of OO testing (cid:149) Negative impact of OO features on testing (cid:149)Test case design, testability and coverage analysis all suffer from Inheritance, polymorphism, late binding and encapsulation (cid:149) OO development is iterativeand incremental: (cid:149)Thus, also test planning, design and execution must be iterative (cid:149) Regression testing must be automated and is an essential part of OO tester toolbox 10 'NOKIA Development testing of OO SW.ppt/ 2004-05-04 / EAP An OO Testing manifesto [BIND00, ch. 4.5] Effective testing of OO software (cid:149) Unique bug hazards in OO SW means also using established test design practices (cid:149) Application and domain specific test tools (cid:149)must be object-oriented (cid:149)and work against the testability problems of OO SW (cid:149) Testing process must adapt to iterative and incremental development and small-scale modularity. (cid:149) Test design must consider method, class and component levels simultaneously 11 'NOKIA Development testing of OO SW.ppt/ 2004-05-04 / EAP Specific OO Hazards (cid:149) The use of OO languages may reduce some kinds of errors, but increase the chance of others (cid:149) Methods often are small "control-flow bugs are less likely (cid:149) OO developers are not more immune to errors "Coding errors (misspelling, misnaming, wrong syntax) are as likely as ever (cid:149) Dynamic binding and complex inheritance structures "faults due to unanticipated bindings or misinterpretation of correct usage (cid:149) Many small components "more interfaces to have programming and usage errors (cid:149) Object state control is distributed over the whole program "state control errors are likely 12 'NOKIA Development testing of OO SW.ppt/ 2004-05-04 / EAP Especially dangerous OO constructs (cid:149) Classes that send relatively more messages to instance variable and message parameter objects are more likely to be buggy (cid:149) Classes with greater depth (number of superclasses) and higher specialization (number of new and overridden methods defined in a class) are more likely to be buggy [BASILI96] u t B (cid:149) Most literature agrees that biggest threat to programs is not dangerous coding but ambiguous requirements and sloppy design (cid:149)Normally bad programming constructs are a sign of bad design or delayed code restructuring (compare to refactoring rules in XP practice) 13 'NOKIA Development testing of OO SW.ppt/ 2004-05-04 / EAP OO Testing at different levels 1/2 (cid:149)Use Cases & Scenarios (cid:149)Abstraction level similar to functional specification (cid:149)Possible to create tests on subsystems and components (cid:149)Useful as functional acceptance tests, complementing module tests (cid:149) On SW team level each iteration could end at use case (cid:150) based tests of team-developed subsystem (cid:149) Development team could test the subsystem / component, produced and already module tested by a partner (cid:149)Extended Use Case pattern 14 'NOKIA Development testing of OO SW.ppt/ 2004-05-04 / EAP OO Testing at different levels 2/2 (cid:149) Class clusters (cid:149)PACT (Parallel Architecture for Class Testing) makes test suites maintainable and modular (cid:149) Class (cid:149)State-based (modal) test methods (FREE pattern) (cid:149)N-switch coverage (cid:149) Method (cid:149)Classical Black box techniques (cid:149)Code-based coverage measured (cid:149)Category partition pattern 15 'NOKIA Development testing of OO SW.ppt/ 2004-05-04 / EAP OO Testing above component level (cid:149) Integration (cid:149) Component interfaces (structural aspect) (cid:149) Subsystem use-cases (functional aspect) (cid:149) Architecture (cid:149) Testing frameworks by developing test applications that exercisethe framework on a determined way (cid:149) Many different kind of architecture validation techniques to assess success in achieving different architecture objectives (cid:149) OO Middleware (cid:149) Typically CORBA in Nokia (cid:149) Also DCOM & .NET elsewhere (cid:149) Standard interface description languages (cid:149) Support for 3rdparty tools (cid:149) Visibility on system-level use-case operation 16 'NOKIA Development testing of OO SW.ppt/ 2004-05-04 / EAP System Testing and Use Cases (cid:149) System tests are based on use cases (cid:149) Use cases are normally connected with functional requirements (cid:149) Use cases, as defined by UML, are not sufficient for system testing test cases (cid:149) requires more describing information (cid:149) You have to also collect information from other sources (cid:149) Several attributes are needed for prioritization between use cases: (cid:149) Risk (cid:149) Frequency = density of enquiries (measurable) (cid:149) Performance = answering time to an enquiry (measurable) (cid:149) Load = allowed load generated to system and is the load constantor varying (measurable) (cid:149) General technique is to establish three levels: (cid:149) use case (can be in several layers) (cid:149) scenarios (instances of the use case) (cid:149) test cases (based on scenarios with additional details) 17 'NOKIA Development testing of OO SW.ppt/ 2004-05-04 / EAP Lessons learned from current OO design & testing practice (cid:149) Caper Jones analyzed 600 projects from 150 organizations depending on OO methods and found out that 1. OO learning curve is very steep and causes many first-use errors 2. OO analysis and design seem to have higher defect potential thanolder design methods 3. Defect removal efficiency against OO design problemsseems lower than against older design methods (cid:149) OO projects typically produce programs of increased volume and complexity against more complex requirements "OO model inspection is even more important than with older design methods 4. OO programming languages seem to have lower defect potentials than procedural programming languages 5. Defect removal efficiency against programming errorsis roughly equal to or somewhat better than removal efficiency against older procedural language errors [JONES97] 18 'NOKIA Development testing of OO SW.ppt/ 2004-05-04 / EAP 2/4 It is still software (cid:149) The software must still be tested, regardless of the programming paradigm used (cid:149) Testing just the software and not the models is an expensive way to identify quality problems (cid:150)(cid:147)Quality retrofit(cid:148) (cid:149) Development testing provides the most important guidance for lower fault density and also to SW reliability (cid:149) Low fault density is the essential basis for good reliability (cid:149) On very buggy software the advanced reliability methods provide only a marginal effect : user/usage profiling, fault tolerance, parallel programming, (cid:133) (cid:149) The promising statistical reliability methods will fail because of too much noise (cid:149) Even development testing of OO SW is still first based on black box methods and focused in higher level risk planning (cid:149) Product risks and specific component level risks 19 'NOKIA Development testing of OO SW.ppt/ 2004-05-04 / EAP Development Testing (DT): Process vs. Development Approach Two different aspects overlap when speaking about development testing: 1.Organizational aspect / 2. Module Testing as part of Formal Process developer(cid:146)s daily work, SW (cid:149)DT as entry criteria for Development Culture next phase (cid:149) How work is conducted (cid:149)Reporting, approving of in development group or planning artifacts by developer (cid:149)Organizational roles, collaboration (cid:149) Agile method: MT done all the time, after each (cid:149)DT final run as exit implementation step criteria is one step, but prepared over time (TDD, XP) 20 'NOKIA Development testing of OO SW.ppt/ 2004-05-04 / EAP