Adequacy of Test Suites with Respect to Coverage Criteria

Test suite adequacy refers to how well a set of test cases satisfies a coverage criterion. A test suite is considered adequate if:

1. All test cases pass (i.e., the system behaves correctly under them), and
2. All “test obligations” defined by the coverage criterion are met.

What is a coverage criterion

A coverage criterion defines what it means for the test suite to sufficiently “cover” the code. Examples:

• Function coverage: Every function must be called at least once.
• Statement coverage: Every line of code must be executed.
• Branch coverage: Every decision point (e.g., if/else) must take both true and false paths.

Example

import math
 
def determinant(a, b, c):
    return (b**2 - 4*a*c)
 
def positive_root(a, b, c, d):
    return (-b + math.sqrt(d)) / (2*a)
 
def negative_root(a, b, c, d):
    return (-b - math.sqrt(d)) / (2*a)
 
def qroots(a, b, c):
    d = determinant(a, b, c)
    if d > 0:
        return positive_root(a, b, c, d), negative_root(a, b, c, d)
    else:
        return "No real roots"
 

Example test case: qroot(1, 0, 0)

• Calls:

  • qroots()
  • determinant()

• Does NOT call: positive_root() or negative_root()

• Function coverage = 2 / 4 = 50%

Example test case: qroot(1, 10, 11)

• Calls:

  • qroots()
  • determinant()
  • positive_root()
  • negative_root()

• Function coverage = 4 / 4 = 100%

Test suite adequacy

If our criterion is:

“Achieve 100% function coverage”

Then:

• {qroots(1, 0, 0)} is not adequate
• {qroots(1, 10, 11)} is adequate

Subsumption

Test case A subsumes test case B with respect to a coverage criterion if A covers all the program elements that B covers, and possibly more.

From the above case:

• qroots(1,10,11) subsumes qroots(1,0,0) for function coverage.
• So if we already have the test case qroots(1,10,11) in our test suite, there’s no need to include qroots(1,0,0) just for the sake of function coverage.

---

Back to parent page: Software Testing

Web_and_App_Development Software_Testing SOFT3202 Test_Coverage Statement_Coverage Branch_Coverage Whitebox_Testing