Unit 5: Structural Testing Using Automated Tool - Subjective Questions

Structural testing, also called white-box testing or glass-box testing, is a testing technique in which test cases are designed by examining the internal structure, logic, control flow, and implementation of a program.

Objectives:

• Exercise important statements, decisions, conditions, loops, and execution paths.
• Detect logical errors, unreachable code, incorrect conditions, and loop-related defects.
• Measure how thoroughly the source code has been executed.
• Identify portions of code that are not covered by existing tests.

Characteristics:

• Requires knowledge of the program source code.
• Uses coverage criteria such as statement, branch, condition, and path coverage.
• Is commonly performed at the unit and integration testing levels.
• Can be supported by automated tools such as EclEmma.

Advantages:

• Reveals hidden implementation and control-flow defects.
• Helps optimize test suites by identifying redundant or missing tests.
• Provides measurable coverage results.
• Can detect dead or unreachable code.

Limitations:

• Complete path coverage may be impractical for complex programs.
• High code coverage does not guarantee correct requirements or defect-free software.
• Testers require programming and implementation knowledge.
• Missing functionality cannot be detected merely by examining existing code.

Structural testing and functional testing examine software from different perspectives.

The two approaches are complementary. Functional testing verifies what the software does, while structural testing investigates how the implementation performs it. A high-quality test strategy normally applies both techniques.

Code coverage measures the extent to which program elements are executed by a test suite. Depending on the selected metric, the elements may be instructions, statements, methods, branches, conditions, or paths.

A general code-coverage formula is:

Data coverage measures how thoroughly tests exercise the relevant input-data categories, values, states, and combinations used by the program. It may include:

• Valid, invalid, and boundary values.
• Different data types and formats.
• Null, empty, minimum, and maximum values.
• Different object states and database records.
• Important combinations of input variables.

Contribution to test adequacy:

• Code coverage reveals unexecuted implementation logic.
• Data coverage reveals missing input categories and boundary cases.
• High code coverage with poor data coverage can miss data-dependent faults.
• High data coverage with poor code coverage can leave branches and exception handlers untested.

Therefore, both measures should be considered together; neither one alone proves that testing is complete.

Statement coverage measures the percentage of executable program statements that are executed at least once by a test suite.

Consider the following logic:

if marks >= 40:
    result = PASS
print result

A test with marks = 60 executes the assignment inside the if block and the print statement. Therefore, all executable statements are covered, giving 100% statement coverage.

However, the false outcome of the condition is not exercised. The program may fail for marks < 40 because result could be uninitialized, even though the reported statement coverage for the tested execution is 100%.

Key points:

• Every executable statement must run at least once.
• It is simple to measure and is a basic structural-testing criterion.
• It does not ensure that every decision outcome or condition combination has been tested.
• Achieving 100% statement coverage is useful, but it is not sufficient evidence of complete testing.

Branch coverage, also called decision coverage, measures whether every possible outcome of each decision has been executed. For a binary decision, both the true and false branches must be tested.

For example, consider if (age >= 18). At least two tests are required:

• age = 20 executes the true branch.
• age = 15 executes the false branch.

If both outcomes are executed, the decision has 100% branch coverage.

Benefits:

• Stronger than statement coverage because it checks decision outcomes.
• Helps reveal missing tests for if, else, loop, and switch logic.
• Can expose defects hidden in untested branches.

Limitations:

• It does not necessarily test every atomic condition in a compound decision.
• It does not guarantee that all execution paths are tested.
• It cannot prove that outputs and business requirements are correct.
• Exception-related control flow may require additional analysis.

Statement coverage checks whether each executable statement has run, whereas branch coverage checks whether every outcome of every decision has run.

Consider:

if balance > 0:
    status = ACTIVE
print status

A test with a positive balance executes every listed statement, producing 100% statement coverage. However, it executes only the true outcome of the decision. The false outcome remains untested, so branch coverage is less than 100%.

Comparison:

If 100% branch coverage is achieved, all reachable statements associated with those branches are generally executed, so it usually implies 100% statement coverage. The reverse implication does not hold. Branch coverage is therefore a more demanding test-adequacy criterion, although even 100% branch coverage does not guarantee complete condition or path coverage.

Path coverage measures whether the test suite has executed the possible sequences of statements and branches from a program's entry point to its exit point.

A control-flow graph, or CFG, represents:

• Statements or blocks as nodes.
• Transfer of control as directed edges.
• Decisions as nodes with multiple outgoing edges.
• Entry and exit points of the program or method.

To perform path testing:

1. Construct the CFG from the program logic.
2. Identify execution paths from entry to exit.
3. Select input data that forces execution through each required path.
4. Run the tests and compare actual paths with the intended paths.

Challenges:

• Every loop can create many paths because it may execute zero, one, or multiple times.
• Nested loops and decisions may cause a combinatorial explosion of paths.
• Some paths may be infeasible because their conditions cannot be satisfied together.
• Changes in program logic can require path sets to be recalculated.

Consequently, complete path coverage is practical only for small and simple modules. For larger programs, testers usually cover basis paths, important risk-based paths, and loop boundaries.

An infeasible path is a control-flow path that cannot be executed by any possible program input because the conditions along that path are mutually inconsistent or constrained by the program state.

For example:

if x > 10:
    ...
if x < 5:
    ...

A path that requires both conditions to be true for the same unchanged value of x is infeasible.

Effects on testing:

• A test case cannot be created to execute an infeasible path.
• Complete path coverage may therefore be impossible even for a program without defects.
• Coverage tools may report uncovered code without explaining whether the corresponding path is feasible.
• Testers may waste effort attempting to cover impossible combinations.

Recommended handling:

• Analyze the control flow and relevant data constraints manually or with static-analysis tools.
• Document why a path or branch is infeasible.
• Exclude generated code, defensive code, or unreachable code only when there is a justified policy.
• Avoid manipulating tests merely to increase a percentage.
• Review apparently infeasible code because it may indicate dead code or a design problem.

Coverage targets should therefore be interpreted with engineering judgment rather than treated as absolute proof of quality.

Condition coverage requires each atomic Boolean condition in a compound decision to evaluate to both true and false at least once.

Consider the decision:

if A or B:
    executeAction()

For branch coverage, tests such as (A=true, B=false) and (A=false, B=false) execute the overall true and false outcomes. This achieves 100% branch coverage.

For condition coverage, the test set must also ensure:

• A becomes true and false.
• B becomes true and false.

The previous tests do not necessarily evaluate B as true. An additional test such as (A=false, B=true) may be needed.

Difference:

• Branch coverage observes the final result of the complete decision.
• Condition coverage observes the individual Boolean terms.
• Condition coverage alone may still fail to produce both final decision outcomes.
• Branch coverage alone may leave one atomic condition unchanged.

A stronger approach called condition/decision coverage requires both every atomic condition and every overall decision outcome to be true and false.

Multiple-condition coverage requires tests for every possible truth-value combination of the atomic conditions in a decision. If a decision contains independent Boolean conditions, the maximum number of combinations is:

Thus, a decision with three conditions may require up to combinations.

Modified Condition/Decision Coverage (MC/DC) requires:

• Every decision outcome to become true and false.
• Every atomic condition to become true and false.
• Each atomic condition to be shown to independently affect the decision outcome.

Comparison:

• Multiple-condition coverage is exhaustive but can require exponentially many tests.
• MC/DC uses carefully selected pairs of tests to demonstrate the independent effect of each condition.
• For many decisions, MC/DC can often be achieved with approximately tests, although the exact number depends on the expression.
• MC/DC is commonly used in safety-critical software because it provides strong evidence without requiring every combination.

Short-circuit evaluation and logically coupled conditions must be considered when selecting tests, because some combinations may not be executable or may not demonstrate independent influence.

Loop coverage evaluates whether loops have been tested with representative iteration counts and boundary conditions. Loops are important because incorrect initialization, termination, and increment logic can produce defects such as infinite loops and off-by-one errors.

For a simple loop, tests should normally cover:

• Zero iterations, when permitted.
• Exactly one iteration.
• Two iterations.
• A typical number of iterations.
• One less than the maximum allowed iterations.
• The maximum allowed iterations.
• One more than the maximum, when inputs allow validation of that boundary.

For nested loops:

• Begin with the outer loops at their minimum values.
• Test the innermost loop using boundary and typical iteration counts.
• Progress outward one loop at a time.
• Include important combinations when the behavior of one loop affects another.

For concatenated loops, independent loops can be tested separately. If one loop controls the state or limits of another, they should be treated similarly to nested loops.

Loop coverage is not always displayed as a separate tool metric, but it can be assessed through branch, path, and manual boundary analysis.

Coverage can be reported at different levels of program structure:

• Instruction coverage: Measures executed bytecode instructions. It is fine-grained and is commonly collected by Java coverage engines such as JaCoCo.
• Line coverage: Measures source-code lines that contain executable instructions. A line may be fully, partly, or not covered depending on how its compiled instructions execute.
• Method coverage: Measures whether each method has been invoked and executed at least once.
• Class coverage: Measures whether each class has been loaded and at least one relevant method or initialization instruction has been executed.

These measures are not interchangeable. For example:

• A method can be marked covered even if many branches inside it remain untested.
• A class can be covered even if only one of its methods is executed.
• One source line can contain multiple instructions or decisions and may therefore be only partially covered.
• High instruction coverage does not necessarily imply high branch coverage.

Testers should examine several metrics together. Line coverage is convenient for source review, branch coverage reveals decision outcomes, and method or class coverage identifies large untested areas.

EclEmma is an Eclipse plug-in that provides code-coverage analysis for Java applications. It uses the JaCoCo coverage engine and integrates coverage execution and reporting into the Eclipse workbench.

Procedure:

1. Install EclEmma from the Eclipse Marketplace if it is not already included in the Eclipse distribution.
2. Import or create the Java project.
3. Compile the project and ensure that the application or JUnit tests run normally.
4. Select a Java class, package, project, or JUnit test.
5. Choose Coverage As from the Eclipse run options.
6. Select the appropriate launch type, such as Java Application or JUnit Test.
7. Allow EclEmma to execute the selected target and collect coverage data.
8. Open the Coverage view to inspect package, class, method, instruction, line, and branch results.
9. Open source files to view color highlighting for covered, partly covered, and uncovered code.

EclEmma helps locate testing gaps, but the tester must still determine whether the tests contain meaningful assertions and verify correct behavior.

EclEmma presents coverage information both as source highlighting and as numeric counters.

Source highlighting:

• Green: The corresponding code is fully covered by the executed tests.
• Yellow: The code is partly covered, usually because only some branches or instructions on the line were executed.
• Red: The code was not executed.

Important counters:

• Instructions: Java bytecode instructions executed and missed.
• Branches: Outcomes of if and switch decisions executed and missed.
• Lines: Source lines with executable instructions that were covered or missed.
• Methods: Methods that were entered or missed.
• Classes: Classes that were executed or missed.
• Complexity: Covered and missed cyclomatic complexity associated with methods and branches.

A green line does not necessarily prove that every possible input or path through the logic has been tested. A yellow line should be inspected for missing decision outcomes. A red line may represent missing tests, dead code, exception handling, platform-specific behavior, or an infeasible situation. Results should therefore be analyzed together with requirements and test assertions.

An EclEmma coverage session stores the execution data collected during one coverage run. Sessions help testers compare or combine results from different test executions.

Merging sessions:

• Coverage from unit, integration, or differently configured tests can be combined.
• A code element is treated as covered if it was executed in at least one merged session.
• Merging provides a view of the total coverage achieved by the complete test suite.

Filtering:

• Coverage scope can be limited by package, class, or launch configuration.
• Test code, generated classes, third-party libraries, and irrelevant infrastructure may be excluded according to an agreed project policy.
• Filters should not be used merely to hide difficult-to-test production code.

Exporting reports:

• Coverage data can be exported into supported report or execution-data formats.
• Reports can be archived, reviewed, or consumed by automated build and continuous-integration processes.
• Historical reports help identify coverage regressions.

For reliable comparisons, the same scope, filters, compiler settings, and metric definitions should be used across runs.

High statement coverage indicates that most executable statements have run. Low branch coverage indicates that many alternative outcomes of decisions have not been exercised.

This situation commonly occurs when tests execute only the normal or successful flow. For example, they may enter each if block but fail to test corresponding false outcomes, validation failures, exception handlers, empty collections, or alternative switch cases.

Improvement method:

1. Sort EclEmma results by missed branches or missed complexity.
2. Inspect yellow and red source lines containing decisions.
3. List the outcomes for every if, loop, conditional expression, and switch statement.
4. Identify input values and object states that trigger the missing outcomes.
5. Add boundary, invalid-input, empty-value, and exception-oriented tests.
6. Add meaningful assertions for outputs, state changes, and side effects.
7. Rerun coverage and verify that the intended branch became covered.
8. Review remaining uncovered branches for infeasibility or dead code.

The goal is not simply to increase a percentage. Each added test should verify useful behavior and have a clear reason for exercising the selected branch.

Cyclomatic complexity, introduced by Thomas McCabe, measures the number of linearly independent paths through a program's control-flow graph. It indicates the structural complexity of a method and provides a basis for determining a minimum set of basis-path tests.

For a control-flow graph:

where:

• is the number of edges.
• is the number of nodes.
• is the number of connected components, which is normally for a single method.

For one connected component:

Cyclomatic complexity can also be calculated from predicate or decision nodes:

where is the number of binary decision points.

Another equivalent interpretation uses the number of enclosed regions in a planar control-flow graph:

where includes the outside region.

A straight-line method has complexity . Each independent binary decision generally increases complexity by one. A higher value indicates more independent paths, greater testing effort, and potentially reduced maintainability.

The cyclomatic-complexity formula is:

Given:

Substituting the values:

Therefore, the cyclomatic complexity is 5.

Interpretation:

• The control-flow graph contains five linearly independent paths.
• A basis-path testing strategy should identify up to five independent paths.
• At least five suitably designed test cases may be needed to exercise the basis paths, although one test can sometimes contribute to multiple coverage objectives.
• The value does not mean that the program has only five total paths. Loops may produce many or infinitely many complete execution paths.
• The result indicates moderate structural complexity and helps estimate test effort.

Cyclomatic complexity should be used as a risk and maintainability indicator, not as direct proof that the method contains defects.

A path is independent when it introduces at least one control-flow edge that is not included in the previously selected paths. A set of such paths forms a basis set for the control-flow graph.

Cyclomatic complexity gives the size of this basis set. If a method has , its graph has six linearly independent paths.

Basis-path testing procedure:

1. Draw the method's control-flow graph.
2. Calculate using edges and nodes or decision points.
3. Select a set of independent paths.
4. Determine input conditions that force each path to execute.
5. Run the tests and verify outputs and state changes.

Relationship to coverage:

• Executing a valid basis set normally provides strong statement and branch coverage.
• Basis-path testing does not execute every possible path, particularly when loops are present.
• Infeasible independent paths must be identified and documented.
• Increasing complexity generally increases the number of tests needed for structural confidence.

Methods with very high complexity should be reviewed for possible simplification because they are harder to understand, test, and maintain.

The major structural coverage criteria have different strengths and costs:

Selecting targets:

• Consider system criticality, regulatory requirements, complexity, and defect risk.
• Use higher criteria for safety-critical, financial, security-sensitive, or highly complex code.
• Apply branch and condition testing to decision-heavy modules.
• Apply loop-boundary testing to iterative algorithms.
• Use EclEmma to locate missed code, while using reviews or other tools for criteria it does not directly report.
• Exclude code only through a documented and consistently applied policy.
• Require meaningful assertions in addition to coverage.

Coverage targets should guide test improvement, not replace requirement-based testing, code review, static analysis, or professional judgment.