Unit 6: Test Automation and Management/Reporting Frameworks - Subjective Questions

Apache Maven is a build automation and project management tool commonly used for Java projects. It manages a project through a standard structure and a central configuration file called pom.xml.

Its role in test automation includes:

• Dependency management: Downloads Selenium, TestNG, and other required libraries automatically.
• Build management: Compiles source code and test code using standard lifecycle phases.
• Test execution: Runs automated tests through plugins such as Maven Surefire.
• Project standardization: Provides a consistent directory structure and build process.
• Reporting: Integrates with testing and reporting tools to generate test results.
• Continuous integration: Allows the same test suite to run consistently on local systems and CI servers.

For example, the command mvn test compiles the required code and executes the configured automated tests.

The major benefits and features of Maven are:

• Convention over configuration: Maven uses a standard project layout, reducing the amount of custom configuration.
• Automatic dependency management: It downloads required libraries and their transitive dependencies from repositories.
• Build lifecycle: It provides predefined phases such as compile, test, package, install, and deploy.
• Plugin-based architecture: Compilation, testing, packaging, and reporting are performed through plugins.
• Repository support: Maven works with local, central, and remote repositories.
• Project documentation: It can generate reports and project information.
• Portability: The same pom.xml and Maven commands can be used on different systems.
• CI integration: Maven works with Jenkins, GitHub Actions, and other CI tools.

These features make Maven particularly useful for repeatable and maintainable automation projects.

Maven manages several activities involved in developing and testing software:

• Creating a standardized project structure
• Resolving and updating dependencies
• Compiling production and test code
• Executing unit and automation tests
• Packaging applications as JAR or WAR files
• Installing artifacts in a local repository
• Deploying artifacts to remote repositories
• Generating test and project reports

The three built-in Maven lifecycles are:

1. Clean lifecycle: Removes files produced by previous builds. Its important phase is clean.
2. Default lifecycle: Handles compilation, testing, packaging, installation, and deployment.
3. Site lifecycle: Generates project documentation and reports.

Important default lifecycle phases include:

• validate: Checks whether the project is correctly configured.
• compile: Compiles the main source code.
• test: Runs tests using a suitable test framework.
• package: Creates a distributable artifact.
• verify: Performs additional quality checks.
• install: Places the artifact in the local repository.
• deploy: Publishes it to a remote repository.

Running a phase also runs all earlier phases in the same lifecycle. Therefore, mvn package performs validation, compilation, testing, and packaging.

Maven architecture consists of several cooperating components:

• Maven project: Contains source code, test code, resources, and the pom.xml file.
• POM: Defines project coordinates, dependencies, plugins, properties, and build settings.
• Build lifecycle: Organizes the build into ordered phases such as compile, test, and package.
• Plugins and goals: Plugins perform actual tasks. For example, the Compiler Plugin compiles Java code, while the Surefire Plugin executes tests.
• Local repository: Stores downloaded dependencies and locally installed project artifacts, normally under .m2/repository.
• Central repository: Maven's default public repository containing commonly used Java artifacts.
• Remote repositories: Organization-specific or third-party repositories configured when artifacts are unavailable in Maven Central.

When a command such as mvn test is issued, Maven reads the POM, constructs the effective project model, resolves plugins and dependencies, checks the local repository, downloads missing artifacts from remote repositories, and executes the plugin goals bound to lifecycle phases.

Maven can be installed and configured through the following steps:

1. Install a supported Java Development Kit, because Maven requires Java.
2. Verify Java by running java -version and javac -version.
3. Download the Maven binary archive from the official Apache Maven website.
4. Extract the archive to a suitable directory.
5. Set JAVA_HOME to the JDK installation directory.
6. Set MAVEN_HOME or M2_HOME to the Maven installation directory when required by the environment.
7. Add Maven's bin directory to the system PATH.
8. Open a new terminal and run mvn -version.

A successful verification displays the Maven version, Java version, and operating system details. User-specific Maven configuration can be placed in .m2/settings.xml. This file may define repository mirrors, proxy settings, server credentials, and build profiles.

A Maven project may be created through an IDE or with an archetype command such as mvn archetype:generate. During creation, important coordinates are supplied:

• groupId: Identifies the organization or package namespace.
• artifactId: Identifies the project.
• version: Identifies the project release.
• packaging: Specifies the output type, such as JAR or WAR.

The standard directory structure is:

• pom.xml: Maven project configuration.
• src/main/java: Production Java source files.
• src/main/resources: Production configuration and resource files.
• src/test/java: Test classes, including Selenium and TestNG tests.
• src/test/resources: Test data and test configuration files.
• target: Generated classes, reports, and packaged artifacts.

After creation, mvn test can be used to compile and execute the tests. The standard structure enables Maven plugins and IDEs to locate files without extensive custom configuration.

POM stands for Project Object Model. The pom.xml file is the central configuration file of a Maven project.

Important POM elements include:

• modelVersion: Specifies the POM model version.
• groupId: Identifies the organization or logical project group.
• artifactId: Gives the project or artifact a unique name within the group.
• version: Specifies the artifact version.
• packaging: Defines the output format, such as JAR or WAR.
• properties: Stores reusable values, including Java and library versions.
• dependencies: Declares external libraries required by the project.
• build: Configures plugins, output settings, and other build behavior.
• profiles: Provides environment-specific configurations.
• repositories: Declares additional artifact locations when necessary.

Maven also supports POM inheritance. A project may inherit configuration from a parent POM, while all POM files ultimately inherit defaults from Maven's Super POM.

A dependency is added as a dependency element inside the POM's dependencies section. Each dependency is usually identified by groupId, artifactId, and version.

Example:

<dependency>
    <groupId>org.testng</groupId>
    <artifactId>testng</artifactId>
    <version>7.10.2</version>
    <scope>test</scope>
</dependency>

Common dependency scopes are:

• compile: Available in all build stages and used by default.
• test: Available only while compiling and running tests.
• runtime: Not required for compilation but required during execution.
• provided: Expected to be supplied by the runtime environment.
• system: Refers to a specific local file and is generally discouraged.

A transitive dependency is a library required by a directly declared dependency. Maven normally downloads it automatically. Conflicting versions are resolved using Maven's dependency mediation rules. Developers can inspect the resolved graph with mvn dependency:tree and use exclusions when an unwanted transitive dependency must be removed.

TestNG is a Java testing framework inspired by JUnit and NUnit. Its name represents Testing Next Generation. It supports unit, integration, functional, and end-to-end testing.

Major TestNG features include:

• Annotation-based test configuration
• Test grouping and selective execution
• Test priorities and dependencies
• Parameterized and data-driven testing
• Parallel test execution
• Flexible test-suite configuration through testng.xml
• Hard and soft assertions
• Automatic HTML and XML reports
• Before and after configuration methods at suite, test, class, method, and group levels
• Support for listeners and custom reporting
• Integration with Maven, Selenium, IDEs, and CI tools

These capabilities make TestNG suitable for organizing and managing large Selenium automation suites.

TestNG and JUnit are Java testing frameworks that provide annotations, assertions, test runners, and build-tool integration. Both can be used with Selenium and Maven.

Important differences include:

• Suite configuration: TestNG supports rich XML suite configuration through testng.xml; JUnit commonly relies on code, tags, build configuration, or platform suites.
• Grouping: TestNG provides built-in groups. Modern JUnit provides tags for similar filtering.
• Dependencies: TestNG directly supports method and group dependencies. JUnit generally encourages independent tests.
• Priorities: TestNG supports the priority attribute. JUnit does not use an equivalent priority model by default.
• Parallel execution: Both support parallel execution, but TestNG exposes detailed controls through suite XML and annotations.
• Parameterized testing: Both support parameterized tests; TestNG also provides @DataProvider and XML parameters.
• Reporting: TestNG generates default HTML and XML reports. JUnit typically produces results consumed by build tools or external reporting plugins.
• Lifecycle annotations: The annotation names and execution scopes differ between the frameworks.

TestNG is often selected for Selenium suites requiring grouping, priorities, dependencies, parallel execution, and configurable test suites.

TestNG can be configured in two related ways:

IDE configuration:

• Open the IDE's plugin or marketplace manager.
• Search for the official or supported TestNG plugin.
• Install it and restart the IDE if requested.
• Confirm that TestNG run configurations are available.

Maven project configuration:

• Add the TestNG dependency to pom.xml, normally with the test scope.
• Store test classes under src/test/java.
• Configure the Maven Surefire Plugin when suite XML files, includes, exclusions, or other special options are needed.
• Optionally create testng.xml to define suites, tests, classes, groups, parameters, and parallel execution.

The installation can be verified by creating a method annotated with @Test and running it through the IDE or with mvn test. The build output should show the number of executed, passed, failed, and skipped tests.

Important TestNG annotations include:

• @BeforeSuite and @AfterSuite: Run once before and after the complete suite.
• @BeforeTest and @AfterTest: Run before and after a test section defined in testng.xml.
• @BeforeClass and @AfterClass: Run once before and after the test methods in a class.
• @BeforeMethod and @AfterMethod: Run before and after every @Test method.
• @BeforeGroups and @AfterGroups: Run around methods belonging to specified groups.
• @Test: Marks a method or class as a test.
• @DataProvider: Supplies multiple sets of test data.
• @Parameters: Receives parameter values from suite XML.
• @Factory: Creates test-class instances dynamically.
• @Listeners: Registers one or more TestNG listeners.

A typical execution sequence is @BeforeSuite, @BeforeTest, @BeforeClass, @BeforeMethod, @Test, @AfterMethod, @AfterClass, @AfterTest, and @AfterSuite. Configuration methods at the method level repeat for each test method.

A Selenium test can be organized with TestNG as follows:

• Use @BeforeClass to initialize WebDriver once for the class, or @BeforeMethod to create an isolated browser session for every test.
• Use @Test methods to perform browser actions and validations.
• Use assertions to verify actual page behavior against expected behavior.
• Use @AfterMethod or @AfterClass to close the browser and release resources.

Example structure:

public class LoginTest {
    WebDriver driver;

    @BeforeMethod
    public void setUp() {
        driver = new ChromeDriver();
    }

    @Test
    public void validLogin() {
        driver.get("https://example.com/login");
        Assert.assertEquals(driver.getTitle(), "Login");
    }

    @AfterMethod(alwaysRun = true)
    public void tearDown() {
        if (driver != null) driver.quit();
    }
}

Using alwaysRun = true for cleanup helps ensure that the browser is closed even when a test or an earlier configuration method fails.

Selenium automates browser interactions but does not itself provide a complete test-management framework. TestNG supplies the missing structure required to organize, execute, validate, and report Selenium tests.

TestNG improves Selenium automation through:

• Lifecycle management: Setup and cleanup annotations manage WebDriver sessions.
• Assertions: Expected and actual browser behavior can be compared.
• Grouping: Smoke, regression, and functional tests can be executed selectively.
• Prioritization: Tests can be assigned execution priorities when ordering is necessary.
• Dependencies: A test may be skipped when a required preceding method or group fails.
• Data-driven testing: Data providers allow the same Selenium workflow to run with multiple inputs.
• Parallel execution: Tests can run concurrently to reduce execution time.
• Suite management: testng.xml controls included classes, methods, groups, parameters, and threads.
• Reporting: TestNG records passed, failed, and skipped tests.
• Extensibility: Listeners can capture screenshots, logs, and custom result details.

Thus, Selenium performs browser automation, while TestNG controls the testing workflow around that automation.

TestNG tests can be executed in several ways:

• Run an individual @Test method from an IDE.
• Run an entire test class from an IDE.
• Run a testng.xml suite through the IDE.
• Run tests through Maven using mvn test.
• Run TestNG from the command line with the required classpath.
• Trigger Maven or TestNG execution from a CI pipeline.

The testng.xml file defines the execution suite. It can specify:

• Suite and test names
• Packages, classes, and methods to include or exclude
• Groups to execute or exclude
• Parameters supplied to test methods
• Parallel execution mode and thread count
• Registered listeners
• Multiple browser or environment configurations

When Maven Surefire is configured with the suite file, mvn test executes the tests described by that file. Suite XML is particularly useful when the same test classes must be combined differently for smoke, regression, cross-browser, or environment-specific runs.

TestNG automatically collects the result of each test and configuration method during execution. After a standard suite run, it normally creates output in the test-output directory.

Important report files include:

• index.html: Main HTML report containing suite summaries and detailed results.
• emailable-report.html: A compact report suitable for sharing.
• testng-results.xml: Structured XML results used by tools and integrations.
• Additional files containing suite, method, and chronological execution details.

A report should be checked for:

• Total tests executed
• Passed, failed, and skipped counts
• Failed method names and stack traces
• Execution duration
• Failed configuration methods
• Parameters used by a test
• Dependency-related skips
• Suite and group information

When tests run through Maven Surefire, additional reports are generated under target/surefire-reports. A failed automation test should also be correlated with application logs, WebDriver logs, screenshots, and environment details before its cause is classified.

Basic HTML reports are generated automatically when TestNG runs a suite through its standard runner. The output directory contains files such as index.html and emailable-report.html.

For Maven execution:

• Configure TestNG as the testing framework.
• Use Maven Surefire to execute the tests.
• Run mvn test.
• Review generated results under target/surefire-reports and any configured TestNG output directory.

Reports can be customized by:

• Implementing IReporter to generate a custom report after the complete suite finishes.
• Implementing listeners such as ITestListener to capture pass, failure, skip, and timing events.
• Adding screenshots and logs when Selenium tests fail.
• Using external reporting libraries such as Allure or Extent Reports.
• Publishing HTML report directories as CI build artifacts.

A useful HTML report should contain the test name, status, duration, parameters, error details, environment information, and evidence such as screenshots. Sensitive information such as passwords and access tokens must not be written to reports.

Assertions compare an actual result with an expected result and cause the test outcome to reflect whether the requirement was satisfied.

Common TestNG assertions include:

• Assert.assertEquals(actual, expected)
• Assert.assertTrue(condition)
• Assert.assertFalse(condition)
• Assert.assertNull(value)
• Assert.assertNotNull(value)
• Assert.fail(message)

A hard assertion stops the current test method immediately when it fails. Statements after the failed assertion are not executed.

A soft assertion, created through SoftAssert, records a failure and allows the method to continue. The method must call assertAll() at the end; otherwise, recorded failures will not be reported correctly.

Hard assertions are appropriate for essential preconditions, such as confirming that a login succeeded before testing a protected page. Soft assertions are useful when several independent details on the same page must be checked in one test. Assertions should include clear failure messages so that the report identifies the expected behavior and the observed result.

Multiple tests can be created by defining several methods annotated with @Test across one or more classes. They can then be managed through TestNG features:

• Groups: The groups attribute classifies tests as smoke, regression, login, or another logical category.
• Suite XML: testng.xml includes or excludes packages, classes, methods, and groups.
• Data providers: @DataProvider supplies multiple data rows to a test method, creating one invocation per row.
• Parameters: @Parameters receives values such as browser name or base URL from suite XML.
• Dependencies: dependsOnMethods and dependsOnGroups define required test relationships.
• Invocation controls: Attributes such as invocationCount and threadPoolSize repeat or parallelize a test.
• Parallel execution: Suite settings can run methods, classes, tests, or instances in parallel.

Each test should remain independently verifiable whenever possible. Shared mutable state and execution-order assumptions can make parallel runs unreliable. WebDriver instances should therefore be isolated per thread or per test when parallel Selenium execution is enabled.

TestNG allows a test method to specify a priority through the priority attribute:

@Test(priority = 1)
public void createUser() { }

@Test(priority = 2)
public void updateUser() { }

Methods with lower priority values are scheduled before methods with higher values. Priority values may be negative, zero, or positive. Methods without an explicit priority use the default value of zero. When methods have the same priority, their relative order should not be treated as a reliable business dependency.

Priority and dependency serve different purposes:

• Priority influences scheduling order.
• dependsOnMethods or dependsOnGroups expresses a required relationship and may cause dependent tests to be skipped after failure.

Recommended practices are:

• Keep tests independent whenever possible.
• Do not use priorities to model a long business workflow.
• Use configuration methods for setup and cleanup.
• Use dependencies only when the relationship is genuine.
• Avoid relying on priority order during parallel execution.
• Use descriptive groups and separate suites for major execution categories.

Excessive ordering can hide test isolation problems and make a suite harder to maintain.