Unit 4: Various Testing Frameworks Available in Java - Subjective Questions

Testing tool selection criteria are the technical, organizational, and economic factors used to determine whether a testing tool is suitable for a particular project.

Major selection factors include:

• Technology compatibility: The tool should support the project's Java version, application type, operating system, browser, database, and development framework.
• Testing requirements: It should support the required testing types, such as unit, integration, functional, performance, or acceptance testing.
• Ease of use: The tool should be easy to learn, configure, maintain, and integrate into existing workflows.
• Automation and CI support: It should work with tools such as Jenkins, Maven, Git, JUnit, and TestNG.
• Reporting: Clear reports, logs, screenshots, and failure diagnostics should be available.
• Cost: Licensing, infrastructure, training, support, and maintenance costs must be evaluated.
• Scalability and reliability: The tool should handle increasing test volume and execute tests consistently.
• Community and vendor support: Documentation, updates, plugins, and technical assistance reduce adoption risk.

A proof of concept should be performed before final selection to verify that the tool meets actual project needs.

A systematic tool-selection process consists of the following steps:

1. Analyze project requirements: Identify the application architecture, supported platforms, test levels, release frequency, and regulatory requirements.
2. Define evaluation criteria: Prepare measurable criteria such as Java compatibility, execution speed, reporting quality, learning effort, integration support, and total cost.
3. Assign priorities: Give higher weight to business-critical factors. A simple weighted score may be calculated as where is the weight of criterion and is the tool's rating for that criterion.
4. Shortlist tools: Compare alternatives such as JUnit 5, TestNG, Cucumber, Maven, and Jenkins according to their intended roles.
5. Conduct a proof of concept: Automate representative test cases, including normal, failure, and integration scenarios.
6. Assess integration: Verify compatibility with the IDE, source-control system, build tool, CI server, and reporting infrastructure.
7. Estimate total cost: Include licenses, infrastructure, training, migration, test maintenance, and support.
8. Review risks: Consider vendor dependency, abandoned plugins, security, scalability, and team skill gaps.
9. Document and approve the decision: Record scores, observations, trade-offs, and the selected solution.

This process makes the selection evidence-based and aligned with both technical and organizational needs.

Jenkins is an open-source automation server commonly used to implement continuous integration and continuous delivery.

Its fundamental concepts include:

• Controller: Coordinates jobs, stores configuration, schedules work, and presents the web interface.
• Agents: Machines or containers that execute build and test tasks.
• Jobs or pipelines: Define the automated activities to perform.
• Build triggers: Start execution after a source-code change, at a scheduled time, through a webhook, or manually.
• Plugins: Add support for Git, Maven, test reports, notifications, credentials, and deployment platforms.
• Artifacts: Files produced by a build, such as JAR files, reports, and logs.

In automated testing, Jenkins obtains the latest source code, compiles it, runs unit and integration tests, records results, and informs the team about failures. It can publish JUnit-compatible reports and preserve build artifacts. This provides rapid feedback and helps prevent defective changes from progressing through the delivery pipeline.

Jenkins, Maven, and JUnit perform complementary roles in a Java continuous integration workflow:

• JUnit defines and executes Java test cases.
• Maven manages dependencies, compiles the source code, and invokes the tests through plugins such as Surefire.
• Jenkins triggers and coordinates the complete workflow.

A typical workflow is:

1. A developer pushes a change to a Git repository.
2. A webhook causes Jenkins to start a pipeline.
3. Jenkins checks out the required branch and prepares the configured JDK and Maven environment.
4. Jenkins runs a command such as mvn clean test.
5. Maven downloads declared dependencies and compiles production and test code.
6. The Surefire plugin discovers and executes JUnit tests.
7. Maven writes machine-readable test results, normally under target/surefire-reports.
8. Jenkins publishes the results, marks the build according to its status, and sends notifications.
9. If all quality checks pass, later pipeline stages may package or deploy the application.

This arrangement produces repeatable builds, automatic regression testing, centralized reports, and quick feedback after every change.

TestNG is a Java testing framework inspired by JUnit and NUnit. It supports unit, integration, functional, and end-to-end testing.

Important TestNG features include:

• Annotations: Annotations such as @Test, @BeforeMethod, @AfterMethod, @BeforeClass, and @AfterSuite control execution.
• Test grouping: Tests can be assigned to groups such as smoke, regression, or integration.
• Dependencies: A test can be configured to depend on another method or group.
• Parameterized testing: Test data can be supplied through parameters and @DataProvider methods.
• Parallel execution: Methods, classes, or suites can be executed concurrently.
• Suite configuration: XML files can define suites, included groups, parameters, and parallelism.
• Priorities and invocation controls: Execution order and repeated invocation can be configured when necessary.
• Reporting and listeners: Built-in reports and listener interfaces support customized monitoring and reporting.

These features make TestNG particularly useful for large test suites that require flexible configuration and execution control.

JUnit 5 and TestNG are both capable Java testing frameworks, but they differ in design and configuration.

• Architecture: JUnit 5 is divided into Platform, Jupiter, and Vintage modules. TestNG uses a more unified framework model.
• Test annotation: Both frameworks commonly use @Test, although the annotation classes belong to different packages.
• Lifecycle annotations: JUnit 5 uses @BeforeEach, @AfterEach, @BeforeAll, and @AfterAll. TestNG uses annotations such as @BeforeMethod, @AfterMethod, @BeforeClass, and @AfterSuite.
• Parameterization: JUnit 5 uses @ParameterizedTest with sources such as @ValueSource and @CsvSource. TestNG commonly uses @DataProvider and XML parameters.
• Grouping: JUnit 5 provides tags through @Tag. TestNG provides groups and supports group inclusion and exclusion through XML.
• Dependencies: TestNG directly supports method and group dependencies. JUnit 5 generally encourages independent tests and does not offer the same dependency model.
• Parallel execution: Both can run tests in parallel. TestNG exposes extensive suite-level XML configuration, while JUnit 5 uses platform configuration properties.
• Extensibility: JUnit 5 provides an extension model through @ExtendWith; TestNG provides listeners and related interfaces.

JUnit 5 is usually a natural choice for modern unit testing, while TestNG may be preferred when detailed suite grouping, dependencies, and execution configuration are central requirements.

Maven is a build automation and project-management tool widely used for Java applications. A Maven project is primarily configured through the pom.xml file.

Its standard project structure includes:

• src/main/java for production Java code.
• src/main/resources for production resources.
• src/test/java for test code.
• src/test/resources for test data and configuration.
• target for generated build output and reports.

Maven dependency management allows libraries such as JUnit to be declared using a group ID, artifact ID, version, and scope. A dependency with the test scope is available during test compilation and execution but is not included as a production dependency.

The default lifecycle includes phases such as:

• validate: Check project configuration.
• compile: Compile production code.
• test: Run unit tests.
• package: Create a JAR or WAR file.
• verify: Run additional checks, often including integration-test verification.
• install: Store the package in the local repository.
• deploy: Publish it to a remote repository.

The Surefire plugin normally runs unit tests, while the Failsafe plugin is commonly used for integration tests.

The Maven Surefire and Failsafe plugins both execute tests, but they are associated with different test levels and lifecycle behavior.

Surefire plugin:

• Primarily runs unit tests.
• Is normally bound to the Maven test phase.
• Commonly discovers classes named *Test, Test*, or *TestCase.
• Causes the build to fail during the test phase when tests fail.
• Is appropriate for fast tests that do not require a packaged or deployed application.

Failsafe plugin:

• Primarily runs integration tests.
• Uses the integration-test phase to execute tests and the verify phase to evaluate results.
• Commonly discovers classes named *IT or *ITCase.
• Delays failure reporting until verify, allowing post-integration-test cleanup to occur.
• Is appropriate for tests involving databases, servers, external services, or packaged applications.

Therefore, Surefire should be used for isolated unit tests, while Failsafe should be used when tests require a broader environment and controlled setup or cleanup across integration-test phases.

JUnit 5 has a modular architecture consisting of three major components:

• JUnit Platform: Provides the foundation for launching testing frameworks on the Java Virtual Machine. It defines APIs for discovering and executing tests and enables IDEs, Maven, Gradle, and console launchers to run compatible test engines.
• JUnit Jupiter: Supplies the modern JUnit 5 programming and extension models. It includes annotations such as @Test, @BeforeEach, @ParameterizedTest, and @Nested, along with the Jupiter test engine.
• JUnit Vintage: Provides a test engine for executing older JUnit 3 and JUnit 4 tests on the JUnit Platform. It is useful during gradual migration to JUnit 5.

This separation allows multiple test engines to operate on one platform. A project can execute Jupiter tests, legacy JUnit tests, and tests from other compatible frameworks through a common launcher and reporting infrastructure.

JUnit is an open-source framework for writing and executing repeatable tests in Java. It is most commonly associated with unit testing, although it can also support integration tests when combined with other tools.

JUnit is important because it provides:

• A standard test model: Developers can identify tests and lifecycle methods using annotations.
• Assertions: Expected and actual behavior can be compared automatically.
• Isolation: Small units of code can be tested independently.
• Repeatability: The same tests can run after every code change.
• Tool integration: JUnit works with IDEs, Maven, Gradle, Jenkins, and reporting systems.
• Regression protection: Existing tests detect behavior that is accidentally broken by later changes.
• Support for test-driven development: A developer can write a failing test, implement the minimum code needed to pass it, and then refactor safely.

JUnit converts expected software behavior into executable checks. This shortens the feedback cycle and improves confidence in changes, provided that tests are independent, meaningful, and maintained with the production code.

In JUnit 5, a basic test is a method annotated with @Test from the org.junit.jupiter.api package. The method contains actions and assertions that verify a specific behavior.

A JUnit 5 test method typically has these characteristics:

• It is declared inside a test class.
• It uses @Test, @ParameterizedTest, @RepeatedTest, or another supported test annotation.
• It is not private.
• It does not normally return a value; its return type is void.
• It should test one clearly defined behavior.
• It should use assertions such as assertEquals, assertTrue, or assertThrows.
• It should not depend on the execution order or result of another test.
• It should be deterministic and produce the same result under the same conditions.

A common structure is Arrange-Act-Assert:

• Arrange: Create objects and input data.
• Act: Invoke the behavior being tested.
• Assert: Verify the result or side effect.

JUnit marks a test as failed when an assertion fails or an unexpected exception escapes from the test method.

JUnit does not require one universal naming convention, but consistent and descriptive names make failures easier to understand.

Recommended conventions include:

• Name a test class after the class under test, such as CalculatorTest for Calculator.
• Use names that describe the behavior and expected result.
• Avoid vague method names such as test1 or checkMethod.
• Follow one convention consistently across the project.

Common test-method styles include:

• Method and outcome: divide_returnsQuotientForValidOperands
• Given-when-then: givenZeroDivisor_whenDivide_thenThrowsException
• Should style: shouldRejectBlankUsername

JUnit 5 also provides @DisplayName for a readable name in test reports. For example, a method named withdraw_rejectsAmountAboveBalance could use the display name Withdraw rejects an amount greater than the balance.

Good naming communicates the scenario without requiring a reader to inspect the implementation. Names should reflect observable behavior rather than internal implementation details.

A JUnit test suite is a logical collection of test classes or test packages that can be discovered and executed together. Suites are useful for separating smoke, regression, integration, or module-specific tests.

In JUnit 5, suite support is provided by the JUnit Platform Suite API and its corresponding engine. A suite class is marked with @Suite, and selectors specify which tests belong to it.

Common selectors and filters include:

• @SelectClasses to include specific test classes.
• @SelectPackages to discover tests in selected packages.
• @IncludeTags and @ExcludeTags to filter tests by JUnit tags.
• @IncludeClassNamePatterns to select classes whose names match configured patterns.

For example, a regression suite may select a package and include tests tagged regression. The build tool or IDE can then run that suite as one unit.

Suites improve organization and execution control, but individual tests should remain independent. A suite should group tests rather than create hidden dependencies or impose a required execution order.

A suitable JUnit 5 test is:

import static org.junit.jupiter.api.Assertions.assertEquals;
import org.junit.jupiter.api.Test;

class CalculatorTest {
    @Test
    void add_returnsSumOfTwoIntegers() {
        // Arrange
        Calculator calculator = new Calculator();

        // Act
        int actual = calculator.add(2, 3);

        // Assert
        assertEquals(5, actual, "The sum should be 5");
    }
}

Explanation:

• @Test tells the JUnit Jupiter engine that the method is a test.
• The Arrange section creates the system under test and prepares input.
• The Act section calls add(2, 3) and stores the observed result.
• The Assert section compares the expected value, 5, with the actual result.
• The final argument to assertEquals is a failure message. In JUnit, the expected value is conventionally placed before the actual value.

The test is small, deterministic, and focused on one observable behavior. Additional cases, such as negative numbers and zero, can be covered with separate or parameterized tests.

Important JUnit 5 code constructs include:

• @Test: Marks a standard test method.
• @ParameterizedTest: Runs one test method with multiple sets of arguments.
• @RepeatedTest: Repeats a test a specified number of times.
• @BeforeEach and @AfterEach: Run before and after every test method.
• @BeforeAll and @AfterAll: Run once before and after all tests in a class.
• @Disabled: Temporarily prevents a test or class from running.
• @Tag: Classifies tests for selective execution.
• @Nested: Organizes related scenarios in nested test classes.
• Assertions: Methods such as assertEquals, assertTrue, assertFalse, assertNull, assertAll, and assertArrayEquals verify expected outcomes.
• Exception testing: assertThrows verifies that a particular operation throws the expected exception type.
• Timeout testing: assertTimeout verifies that an operation finishes within a specified duration.
• Assumptions: Methods such as assumeTrue abort a test when an environmental precondition is not met.

Together, these constructs support setup, execution, verification, organization, conditional execution, and cleanup of automated tests.

Assertions verify required behavior. If an assertion is false, the test fails. For example:

assertEquals(200, response.getStatusCode());

This means that returning any status code other than 200 is considered a product or test failure.

Assumptions verify conditions that must hold before a test is meaningful in the current environment. If an assumption is false, the test is aborted rather than failed. For example:

assumeTrue(System.getenv("TEST_DATABASE_URL") != null);

The test is skipped or reported as aborted when the required database configuration is unavailable.

The key differences are:

• Assertions validate the expected result of the system under test.
• Assumptions validate an execution precondition such as an operating system, environment variable, or external service availability.
• A failed assertion indicates incorrect behavior.
• A failed assumption indicates that the current environment is unsuitable for that test.

Assumptions should be used carefully. Overuse can hide tests from regular execution and leave important behavior unverified.

Eclipse provides integrated JUnit support that simplifies the complete testing workflow.

• Test generation: The JUnit Test Case wizard can create a test class and selected test-method stubs.
• Test execution: A class, method, package, project, or suite can be launched using Run As > JUnit Test.
• JUnit view: Eclipse displays the number of tests, execution time, failures, errors, and skipped tests. Green indicates success, while red indicates a failure or error.
• Failure analysis: The view shows assertion messages and stack traces and can compare expected and actual values.
• Navigation: Selecting a failed test or stack-trace entry opens the corresponding source location.
• Debugging: Debug As > JUnit Test runs tests with breakpoints, variable inspection, and step-by-step execution.
• Rerunning tests: Developers can rerun the entire suite or only failed tests.
• Build integration: Eclipse can import Maven projects and obtain JUnit dependencies from pom.xml.

This support reduces the time required to create tests, diagnose failures, and repeatedly execute tests during development.

For a Maven-based project, JUnit 5 is normally installed by declaring a test dependency in pom.xml:

<dependency>
    <groupId>org.junit.jupiter</groupId>
    <artifactId>junit-jupiter</artifactId>
    <version>5.x.x</version>
    <scope>test</scope>
</dependency>

The version should be replaced with a current version approved by the project. Maven downloads the required artifacts. A compatible Maven Surefire plugin should be configured when necessary, after which tests can be executed with mvn test.

For a non-Maven Eclipse project:

1. Open the project's build-path configuration.
2. Choose Add Library and select JUnit.
3. Select JUnit 5 and complete the wizard.
4. Place test classes in a separate source folder, commonly test or src/test/java.
5. Import APIs from org.junit.jupiter.api.
6. Run a test using Run As > JUnit Test.

Maven dependency management is generally preferable because it records versions in source control and produces repeatable builds. Manually downloaded JAR files must include all required transitive components and are therefore more difficult to maintain.

Gherkin is a structured, business-readable language used to describe software behavior. It is commonly used with behavior-driven development tools such as Cucumber.

Major keywords include:

• Feature: Describes a business capability.
• Background: Defines steps shared by all scenarios in a feature.
• Scenario: Describes one concrete behavior or example.
• Scenario Outline: Defines a parameterized scenario executed with rows from an Examples table.
• Given: Establishes the initial context.
• When: Describes an action or event.
• Then: States an expected observable outcome.
• And/But: Improve readability by continuing the preceding step type.

Example:

Feature: Account login

  Scenario: Successful login
    Given a registered user exists
    And the login page is open
    When the user submits valid credentials
    Then the account dashboard is displayed

The scenario expresses behavior without prescribing interface-level implementation details. Step definitions connect each Gherkin step to executable automation code.

A Scenario Outline is a reusable Gherkin scenario template containing placeholders such as <username> or <result>. The associated Examples table supplies values. The scenario is executed once for each data row.

Example:

Scenario Outline: Validate login attempts
  Given the user enters <username> and <password>
  When the login form is submitted
  Then the outcome is <result>

  Examples:
    | username | password | result  |
    | alice    | correct  | success |
    | alice    | wrong    | failure |

Step definitions are methods in the automation language that match Gherkin steps and execute their behavior. For example, a Java method may use a Cucumber annotation such as @When with an expression that captures the submitted values.

An effective Gherkin scenario should:

• Describe one clear business behavior.
• Use language understood by developers, testers, and domain stakeholders.
• Focus on observable outcomes instead of page selectors or implementation details.
• Remain short enough to understand without excessive background.
• Use consistent domain terminology.
• Avoid coupling scenarios through shared state or execution order.
• Use Scenario Outlines only when the same behavior genuinely applies to multiple examples.

Well-written Gherkin serves as executable specification and shared documentation, not merely as a wrapper around low-level test scripts.