Unit 5: Structural Testing Using Automated Tool

I. Orientation: Testing the Internal Structure

Structural testing, also called white-box testing or glass-box testing, derives tests from the internal structure of software. Testers examine source code, control flow, decisions, loops, and data usage to determine which program elements have been exercised.

• Governing principle: Adequacy is measured by how much of a chosen structural model is executed, such as statements, branches, paths, or data-flow relationships.
• Required knowledge: The tester normally needs access to source code, bytecode, design details, or an instrumented executable.
• Test basis: Tests are derived from implementation elements such as if conditions, loops, methods, exception handlers, and variable definitions.
• Measurement convention: Coverage is usually expressed as a percentage:

Coverage (%) = (Number of covered elements / Total number of elements) × 100

• Covered elements: Structural items executed by at least one test.
• Total elements: All measurable items under the selected criterion.
  • Automation: A coverage tool instruments or monitors the program, executes the test suite, and reports which elements were covered.
  • Core limitation: High structural coverage shows that code was executed; it does not prove that outputs were correct or that all requirements were tested.

II. Structural Testing: Basis and Procedure

A. Structural Testing

Structural testing evaluates software by designing tests around its internal logic and measuring the structures those tests exercise.

• Purpose: It identifies untested implementation logic, including missed conditions, exceptional flows, and loop behavior.
• Typical levels: It is most common in unit and integration testing, where individual methods, classes, and component interactions are visible.
• Control-flow model: A control-flow graph represents execution structure.
  • Nodes: Statements or basic blocks containing sequential statements.
  • Edges: Possible transfers of control between nodes.
  • Decision nodes: Points such as if, while, or switch that produce alternative edges.
• Basic procedure:
  1. Select a code unit and coverage criterion.
  2. Examine its statements, decisions, loops, and data dependencies.
  3. Design tests for uncovered structural elements.
  4. Run the tests through a coverage tool.
  5. inspect uncovered code and improve tests where useful.
• Example structure:

int fee(int age) {
    if (age < 18) {
        return 5;
    }
    return 10;
}

• age < 18 creates two outcomes.
• Tests using age = 15 and age = 25 execute both outcomes.
  • Strength: Structural feedback is objective and traceable to exact code locations.
  • Limitation: Tests derived only from code may miss omitted requirements because nonexistent code cannot appear as uncovered.

III. Coverage Criteria: Measuring Test Thoroughness

A. Data and Code Coverage

Data coverage follows how values are defined and used, whereas code coverage measures execution of selected program structures.

1. Data coverage:

   • Definition-use pair: A variable definition and a reachable subsequent use form a def-use pair.
   • Definition: A statement assigns a value, such as total = 0.
   • Use: A statement reads that value, such as return total or if (total > limit).
   • All-definitions criterion: Every variable definition must reach at least one use.
   • All-uses criterion: Every reachable use of each definition must be exercised.
   • Value categories: Input-domain coverage may also partition data into valid, invalid, boundary, empty, and special-value classes.

2. Code coverage:

   • Measurement targets: Instructions, lines, statements, branches, methods, classes, or paths.
   • Tool operation: Instrumentation records execution probes while automated tests run.
   • Interpretation: 80% line coverage means approximately 80 out of every 100 measurable lines executed, not that 80% of behavior is correct.

• Difference: Code coverage asks, “Which program structures ran?” Data-flow coverage asks, “Did values travel through important definition-to-use relationships?”

B. Statement Coverage

Statement coverage measures the proportion of executable statements executed by the test suite at least once.

Statement coverage =
(Executed executable statements / Total executable statements) × 100

• Target: Each assignment, call, return, and other executable statement should run.
• Example:

if (score >= 50) {
    result = "Pass";
}
print(result);

• A test with score = 60 can execute every statement.
• It does not test the false outcome of score >= 50.
  • Advantage: It is simple, inexpensive, and useful for locating completely untested blocks.
  • Weakness: 100% statement coverage may leave decision outcomes, short-circuit conditions, and exception flows untested.
  • Relationship: Complete branch coverage normally implies statement coverage for reachable code, but complete statement coverage does not imply branch coverage.

C. Branch Coverage

Branch coverage, also called decision coverage, measures whether every possible outcome of each decision has been executed.

Branch coverage =
(Executed decision outcomes / Total decision outcomes) × 100

• Binary decisions: An if condition normally contributes true and false branches.
• Multiple alternatives: A switch contributes case branches and, where present, a default branch.
• Loop decisions: A loop condition should be both true, entering or repeating the loop, and false, terminating or bypassing it.
• Worked example:

if (balance >= amount) {
    approve();
} else {
    reject();
}

• balance = 500, amount = 200 covers the true branch.
• balance = 100, amount = 200 covers the false branch.
• Together, the tests achieve 2 / 2 × 100 = 100% branch coverage.
  • Strength: It detects missing decision outcomes that statement coverage can conceal.
  • Limitation: For if (A && B), both overall outcomes can be covered without showing that each atomic condition independently affected the result.

D. Path Coverage

Path coverage measures whether tests execute distinct routes from a program’s entry to its exit.

• Path: A sequence of control-flow graph nodes and edges followed during one execution.
• Complete path coverage: Every feasible entry-to-exit path must execute.
• Growth problem: Two sequential binary decisions can create up to 2² = 4 combinations; n independent binary decisions can create up to 2ⁿ paths.
• Loops: An unrestricted loop may generate infinitely many paths because it can iterate zero, one, two, or more times.
• Infeasible path: Some graph paths cannot execute because their conditions contradict each other.
• Practical substitute: Basis-path testing selects a linearly independent set of paths using cyclomatic complexity instead of attempting all paths.
• Relative strength: Complete feasible-path coverage implies branch and statement coverage, but branch coverage does not imply path coverage.
• Limitation: Exhaustive path coverage is usually practical only for small, loop-free units.

E. Other Coverages and Understanding Their Differences

Other criteria focus on structures that statement, branch, and path metrics do not fully distinguish.

• Condition coverage: Each atomic Boolean condition becomes both true and false; in A || B, both A and B must independently receive both values.
• Multiple-condition coverage: Tests combinations of atomic condition values; two Boolean conditions have up to 2² = 4 combinations.
• MC/DC coverage: Modified Condition/Decision Coverage shows that each condition can independently change the overall decision outcome.
• Function or method coverage: Measures whether every function or method was invoked at least once.
• Instruction coverage: Measures executed bytecode or machine instructions; one source line may compile into several instructions.
• Line coverage: Measures source lines associated with executed instructions; formatting and multi-statement lines can affect interpretation.
• Loop coverage: Commonly tests zero, one, and multiple iterations, together with relevant boundaries.
• Exception coverage: Exercises handlers and exceptional exits such as catch blocks.
• Comparison:
  • Statement coverage focuses on executable actions.
  • Branch coverage focuses on decision outcomes.
  • Condition coverage focuses on atomic predicates.
  • Path coverage focuses on complete execution routes.
  • Data-flow coverage focuses on definitions and uses.
• Selection rule: Stronger criteria usually reveal more gaps but require more tests and may still miss incorrect assertions or missing functionality.

IV. EclEmma: Automated Java Coverage Analysis

A. Introduction to Code Coverage Tool EclEmma

EclEmma is an Eclipse plug-in that provides Java code coverage analysis using the JaCoCo coverage library.

• Purpose: It runs Java applications or JUnit tests in coverage mode and maps execution data back to Eclipse source files.
• Launch method: A test can be started through Coverage As, analogous to Eclipse’s Run As command.
• Instrumentation: JaCoCo inserts or applies execution probes to Java bytecode and records which probes execute.
• Supported metrics: Reports commonly include instruction, branch, line, method, class, and cyclomatic-complexity information.
• Source highlighting:
  • Green: The corresponding code was fully covered.
  • Yellow: The line was partly covered, often because only some branches executed.
  • Red: The corresponding code was not covered.
• Scope: Results can be inspected at project, package, class, method, and source-line levels.
• Workflow: Run tests, open the Coverage view, inspect low-coverage units, add meaningful tests, and rerun coverage.

B. Interpreting Results of EclEmma

EclEmma results must be read by metric and code context rather than treated as a single quality score.

• Missed and covered counters: Each row reports uncovered and covered items, often with a percentage bar.
• Instruction result: Shows how much compiled bytecode executed; it can differ from source-line coverage.
• Branch result: Reports decision outcomes for constructs compiled into branches, including if and conditional expressions.
• Partial line: A yellow line indicates that some associated instructions or branches ran while others did not.
• Method and class results: A covered method or class was entered, but its internal alternatives may remain untested.
• Drill-down analysis: Package-level percentages locate broad gaps; class and method views identify the exact missed logic.
• Interpretive caution: Generated code, trivial accessors, unreachable defensive code, and compiler-generated branches can affect percentages.
• Quality requirement: Coverage should be combined with assertions, requirement-based tests, boundary tests, and mutation testing where stronger evidence is needed.

V. Cyclomatic Complexity: Independent Control-Flow Paths

A. Cyclomatic Complexity

Cyclomatic complexity, introduced by Thomas J. McCabe (1976), quantifies the number of linearly independent paths through a control-flow graph.

• Graph formula:

V(G) = E - N + 2P

• V(G): Cyclomatic complexity of graph G.
• E: Number of control-flow edges.
• N: Number of nodes.
• P: Number of connected components; normally P = 1 for one method.
  • Decision formula: For a connected, well-structured method with binary decisions:

V(G) = D + 1

• D: Number of binary decision points.
  • Example: A method containing one if and one while has D = 2, so V(G) = 2 + 1 = 3.
  • Testing meaning: Complexity gives the size of a basis set of independent paths; at least three suitably selected tests are needed to exercise that example’s basis paths.
  • Structural meaning: Higher values indicate more decision logic, more possible routes, and generally greater testing and maintenance effort.
  • EclEmma use: Complexity counters can be compared with missed complexity to identify decision-heavy methods whose branches remain uncovered.
  • Limitation: The number does not measure algorithm correctness, readability, data complexity, concurrency risk, or path feasibility; it is an indicator, not a complete quality judgment.