xvi ◾ Contents
19.4 Remaining Questions .......................................................................................... 388
19.4.1 Specification or Code Based? .................................................................. 388
19.4.2 Configuration Management? .................................................................. 388
19.4.3 Granularity? ........................................................................................... 388
19.5 Pros, Cons, and Open Questions of TDD ........................................................... 390
19.6 Retrospective on MDD versus TDD ................................................................... 390
20 A Closer Look at All Pairs Testing ...........................................................................395
20.1 e All Pairs Technique ........................................................................................395
20.1.1 Program Inputs ....................................................................................... 396
20.1.2 Independent Variables ............................................................................ 398
20.1.3 Input Order ............................................................................................ 399
20.1.4 Failures Due Only to Pairs of Inputs ...................................................... 403
20.2 A Closer Look at the NIST Study........................................................................ 404
20.3 Appropriate Applications for All Pairs Testing ..................................................... 404
20.4 Recommendations for All Pairs Testing ............................................................... 405
References ...................................................................................................................... 406
21 Evaluating Test Cases .............................................................................................. 407
21.1 Mutation Testing ................................................................................................. 407
21.1.1 Formalizing Program Mutation .............................................................. 408
21.1.2 Mutation Operators ................................................................................ 409
21.1.2.1 isLeap Mutation Testing .......................................................... 410
21.1.2.2 isTriangle Mutation Testing .....................................................411
21.1.2.3 Commission Mutation Testing ................................................412
21.2 Fuzzing ................................................................................................................. 414
21.3 Fishing Creel Counts and Fault Insertion ............................................................. 414
References .......................................................................................................................415
22 Software Technical Reviews .....................................................................................417
22.1 Economics of Software Reviews ...........................................................................417
22.2 Roles in a Review..................................................................................................419
22.2.1 Producer ..................................................................................................419
22.2.2 Review Leader .........................................................................................419
22.2.3 Recorder ................................................................................................. 420
22.2.4 Reviewer ................................................................................................. 420
22.2.5 Role Duplication .................................................................................... 420
22.3 Types of Reviews ................................................................................................. 420
22.3.1 Walkthroughs ..........................................................................................421
22.3.2 Technical Inspections ..............................................................................421
22.3.3 Audits ......................................................................................................421
22.3.4 Comparison of Review Types ................................................................. 422
22.4 Contents of an Inspection Packet ........................................................................ 422
22.4.1 Work Product Requirements .................................................................. 422
22.4.2 Frozen Work Product ............................................................................. 422
22.4.3 Standards and Checklists ........................................................................ 423
22.4.4 Review Issues Spreadsheet ...................................................................... 423
22.4.5 Review Reporting Forms ........................................................................ 424