Acknowledgments xiii Introduction xv Part I Goals, Strategies, and Tactics 1 Chapter 1 What Does It Mean to Be Pragmatic? 3 What Do Effective and Efficient Mean? 3 What Effects Do You Want? 4 What Is the Right Level of Efficiency? 4 Avoiding Redundancy 4 Reducing Cost 5 What Software Testing Isn’t…But Is Often Thought to Be 6 Five Phases of a Tester’s Mental Life 7 Other Founding Views on Testing 8 Testing as a Form of Quality Risk Management 9 So What Is the Test Team All About? 9 What Does “Quality” Mean to You? 10 Chapter 2 Triangle Test Exercise 13 Exercise: The Triangle Test 13 Your Solution for the Triangle Test 14 Your Solution for the Triangle Test (Continued) 15 Author’s Triangle Test Solution 15 Chapter 3 Aligning Testing with the Project 19 Why Do Organizations Test? 19 Perspectives on Testing 20 Testing in Context 20 Common Test Phases and Objectives 21 Testing Throughout the Organization 21 The V Model 23 Evolutionary and Incremental Models 25 The Spiral Model 28 Regrettably Common Model: Code and Fix 28 Testing Maintenance Releases 29 System Integration 30 Hardware/Software Development 32 The Test Process 32 Chapter 4 Understanding Test Strategies, Tactics, and Design 35 Aligning Mission, Strategies, and Tactics 35 Analytical Test Strategies 36 Model-Based Test Strategies 37 Methodical Test Strategies 38 Process-Oriented Test Strategies 38 Dynamic Test Strategies 39 Philosophical Test Strategies 40 Regression 41 Regression Strategy 1: Repeat All Tests 42 Regression Strategy 2: Repeat Some Tests 42 Three Other Regression Strategies 43 Tactics: Categories of Testing Techniques 44 Strategic and Tactical Considerations 46 The Test System 47 Classic Principles for Test Design 50 Phases of Test Development 51 Synopsis of Test Strategies, Tactics, and Design 52 Part II Risk-Based Testing 53 Chapter 5 Understanding Risks to System Quality 55 Categories of Quality Risks 56 Functionality 56 Performance and Reliability 57 Stress, Capacity, and Volume 59 States 59 Transactions 60 Installation and Deinstallation 61 Operations 62 Maintenance and Maintainability 63 Regression 63 Usability and User Interface 64 Data Quality 65 Error and Disaster Handling and Recovery 66 Date and Time Handling 67 Localization 68 Configuration and Compatibility 69 Networked, Internetworked, and Distributed 69 Standards and Regulatory Compliance 70 Security 70 Timing and Coordination 71 Documentation 71 Can You Think of Other Quality Risks? 72 Chapter 6 Aligning Testing with Quality Risks 73 Prioritize Risks to System Quality 73 Testing, Customer Usage, and System Configurations 74 Approaches for Quality Risks Analysis 76 Informal Quality Risk Analysis 78 Tips for Risk Analysis 81 Challenges of Risk Analysis 82 Chapter 7 Quality Risk Analysis Exercise 85 My Solution 86 Comments on My Solution 91 A Funny Thing Happened to Me on the Way to the Quality Risks… 92 Bonus Exercise 95 Template 96 My Solution 100 Comments on My Solution 101 Bonus Example: Grays and Blues Quality Risk Analysis 102 Part III Static Testing 105 Chapter 8 Reviews and Other Static Tests 107 Testing Requirements and Designs 107 Reviews Costs and Benefits 108 Types of Reviews 109 Reviews for Verification and Validation 109 Reviews as a Way to Improve the Process, Improve the System, and Reduce Costs 110 Answers to the Three Questions about Static Testing 114 Reviews as a Way to Achieve Consensus and Understanding 115 The Review Process, Roles, and Responsibilities 116 Deliverables and Ground Rules from Reviews 117 Common Requirements and Design Bugs 118 Reviewing (and Testing) Documentation 119 Other Static Tests 120 Chapter 9 Review Exercise 123 Reviews 123 Your Solution 124 My Solution 128 Comments on My Solution 132 Bonus Exercise: Reviewing Triangle Requirements Title Page 132 Triangle Requirements Review Using Wiegers’s List 132 The Requirements Bugs You Found 134 My Solution: Requirements Bugs 137 Comments on My Solution 139 Part IV Behavioral Testing 141 Chapter 10 Equivalence Classes and Boundary Values 143 Equivalence Partitioning 144 Boundary Value Analysis 144 Integer 145 Real Numbers 147 Character and String 149 Date 152 Time 154 Currency 155 Beyond Functional Classes and Boundaries 157 Chapter 11 Equivalence Classes and Boundary Values Exercise 159 Functional Boundaries and Classes 159 My Solution and Comments 161 Nonfunctional Boundaries and Classes 166 Omninet Marketing Requirements Document (v0.3) 166 Omninet System Requirements Document (v0.2) 168 Chapter 12 Use Cases, Live Data, and Decision Tables 169 Use Cases and Scenario Tests 169 Nouns and Verbs, Adjectives and Adverbs 172 Live Data and Customer Tests 173 Decision Tables 174 Chapter 13 Decision Table Exercise 177 Decision Table Tests 177 My Solution and Comments 178 Decision Tables and Boundary Values 181 My Solution and Comments 183 Building a Decision Table for Testing 188 My Solution and Comments 190 Chapter 14 State Transition Diagrams 197 Describing Systems with States Using State Transition Diagrams 197 State Tables 199 Printer Server State Transition Diagram 200 Chapter 15 State Transition Diagram Exercise 203 Kiosk States 203 My Solution and Comments 204 ATM State Models 211 My Solution and Comments 212 Grays and Blues and State Transition Diagrams 213 My Solution and Comments 214 Chapter 16 Domain Testing 217 Combinatorial Explosions 217 A Domain Example Using Frequent-Flyer Programs 218 Possible Domain Test Values 220 An Aerospace Example 221 When Domain Rules Change 223 Domain Analysis Summary 224 Complex Domain Testing Example 225 A General Rule for Complex Domains 228 Chapter 17 Domain Testing Exercise 233 My Solution and Comments 234 Chapter 18 Orthogonal Arrays and All Pairs 239 Combinatorial Explosions 239 Orthogonal Arrays and All-Pairs Tables 241 Two Orthogonal Arrays 241 Selecting an Orthogonal Array 243 Applying Orthogonal Arrays to a Complicated Real-World Example 244 All-Pairs Tables 258 Other Thoughts on Configuration Testing 259 Chapter 19 Orthogonal Arrays Exercise 261 My Solution and Comments 263 Chapter 20 Reactive Testing 267 General Facts about Reactive Tests 267 Error Guessing, Attacks, and Bug Taxonomies 268 Bug Hunting 269 Exploratory Tests 270 Checklists 271 Other Sources of Inspiration for Reactive Testing 273 Advantages and Disadvantages 274 A Case Study of Exploratory Testing 276 Part V Structural Testing 279 Chapter 21 Control-Flow Testing 281 Code Coverage 282 McCabe Cyclomatic Complexity 284 Chapter 22 Control-Flow Testing Exercise 287 My Solution and Comments 288 Chapter 23 Data-Flow Testing 293 Chapter 24 Data-Flow Testing Exercise 297 My Solutions and Comments 298 Chapter 25 Integration Testing 307 Drivers and Stubs 308 Integration Techniques 309 Backbone Integration 311 McCabe Basis Paths for Integration 313 Enhanced Hex Converter Program 316 Call Flow 318 Chapter 26 Integration Basis Test Exercise 321 My Solution and Comments 323 Part VI Appendices 325 Appendix A Omninet: The Internet Everywhere Marketing Requirements Document 327 1 Scope 329 1.1 Terms, Acronyms, and Abbreviations 329 1.2 Applicable Documents 330 2 Required release date 330 3 Description of requirements 330 3.1 General technical requirements 330 3.1.1 Welcome 330 3.1.2 Payment 330 3.1.3 Internet Browser 331 3.1.4 Performance 331 3.1.5 Localization 331 3.1.6 Content Control 331 3.1.7 Session Termination 331 3.1.8 Confidentiality 332 3.2 Administration 332 3.2.1 Software Updates 332 3.2.2 View Kiosks 332 3.2.3 View Users 333 3.2.4 Modify User 333 3.2.5 Terminate User 333 Appendix B Omninet: The Internet Everywhere System Requirements Document 335 Functionality System Requirements 336 Reliability System Requirements 340 Usability System Requirements 341 Efficiency System Requirements 342 Maintainability System Requirements 343 Portability System Requirements 344 Design Models 345 Omninet System Architecture 345 Payment Processing Decision Table 346 Kiosk Module Flow 347 Kiosk State-Transition Diagram 348 Kiosk State-Transition Table 348 Kiosk OS/Browser/Connection Speed Configuration Orthogonal Array 348 Appendix C Bibliography and Other Recommended Readings 351 RBCS Company Profile 353 Index 355

A hands-on guide to testing techniques that deliver reliable software and systems Testing even a simple system can quickly turn into a potentially infinite task. Faced with tight costs and schedules, testers need to have a toolkit of practical techniques combined with hands-on experience and the right strategies in order to complete a successful project. World-renowned testing expert Rex Black provides you with the proven methods and concepts that test professionals must know. He presents you with the fundamental techniques for testing and clearly shows you how to select and apply successful strategies to test a system with budget and time constraints. Black begins by discussing the goals and tactics of effective and efficient testing. Next, he lays the foundation of his technique for risk-based testing, explaining how to analyze, prioritize, and document risks to the quality of the system using both informal and formal techniques. He then clearly describes how to design, develop, and, ultimately, document various kinds of tests. Because this is a hands-on activity, Black includes realistic, life-sized exercises that illustrate all of the major test techniques with detailed solutions. By the end of this book, you'll know more about the nuts and bolts of testing than most testers learn in an entire career, and you'll be ready to put those ideas into action on your next test project. With the help of real-world examples integrated throughout the chapters, you'll discover how to: Analyze the risks to system qualityAllocate your testing effort appropriately based on the level of riskChoose the right testing strategies every timeDesign tests based on a system's expected behavior (black box) or internal structure (white box)Plan and perform integration testingExplore and attack the systemFocus your hard work to serve the needs of the project The author's companion Web site provides exercises, tips, and techniques that can be used to gain valuable experience and effectively test software and systems. Wiley Technology Publishing Timely. Practical. Reliable. Visit the author's Web site at http://www.rexblackconsulting.com/