Software Testing - White Box Methods

No one method can find all types of defects, so be sure to use all methods when designing tests. Some methods are better at finding certain types of errors than other methods. Using all methods will ensure the best possible test coverage.

Path Analysis, Code Based

Review the code and test each decision option (each If, While, etc.). Path analysis is well suited to finding logic errors, but is poor at finding mathematical errors (like mis-coding an equation). Advantages of path testing are:

• Powerful: a relatively small number of tests finds most of the defects.
• Objective: different people using the technique will agree on the number of tests required.
• Defines equivalence tests fairly easily.

Limitations of path analysis are:

• Requires a good understanding of the code.
• Not all programming languages lend themselves to this approach (such as Spreadsheets).
• Will not detect errors that require multiple passes through a loop (such as counter overflow).
• Dynamically changing items (e.g. pointers) make path identification difficult.
• Does not check validity of the input.

Statement Coverage

Analyze the code and ensure that each line of code is exercised at least once. This was once useful, but with event driven systems and multi-tasking operating systems, statement coverage is less useful. Branch Coverage is preferable to Statement Coverage.

Branch Coverage

This is similar to path analysis. Review the code to ensure that each branch is exercised at least once. The primary difference between Branch Coverage and Path Analysis is demonstrated in this example:
Consider this pseudo-code:
  IF (var1 < 3) OR (var1 > 10) THEN
     DoThis()
  ELSE
     DoThat()
  ENDIF
By Path Analysis, there are two possible paths: DoThis() and DoThat(). However, by Branch Coverage, there are three conditions:
   var1<3 : DoThis()
   var1>10 : DoThis()
   var1 between 3 and 10 : DoThat()

NOTE: Branch coverage will generate the same number of tests as the McCabe's Cyclomatic Complexity number. This makes a McCabe's value useful in estimating the minimum number of required test cases. There are some code coverage tools that will calculate the McCabe's Complexity. To manually calculate the number for a given function or subroutine:

Start with "1".
Add +1 for each keyword that is equivalent to:
  And
  Or
  If ("Then" does not count, since the number of branches is the same)
  While
  Do
  For
  Each case in a Case (or Switch) statement
  try... (in VB, it's "OnErrorGoto")
  each additional catch beyond the first in a try...catch block

Define Equivalence Classes

An "equivalence class" is defined as the class of all inputs that are expected to produce the same output. For each input, identify the range of values over which system behavior will be the same. Test one representative value within each range and assume that it’s result correctly predicts the results for all other values in that same range. This method is used primarily to eliminate redundant tests.

Boundary Value

Test at the boundaries of a range as well as somewhere in the center. This method is well suited to find off-by-one errors. Integer boundaries are easy to test, but with floating point numbers it becomes less precise. Because of round off errors, the best way to do a boundary test on a floating point value is to allow a fractional error and test for a "greater than" or "less than" condition. NEVER test for an Equal condition when using floating point numbers!