Unit 3: Test Administration - Subjective Questions

Test planning is the process of defining the scope, objectives, resources, strategy, schedule, and activities required to test a software product.

The primary goal of test planning is to establish a systematic approach for verifying that the software satisfies its specified requirements and quality expectations.

Key purposes include:

• Identifying the features that must be tested.
• Defining suitable manual and automated testing strategies.
• Estimating the effort, cost, resources, and time required.
• Assigning roles and responsibilities to team members.
• Identifying testing risks and defining mitigation measures.
• Establishing entry criteria, exit criteria, and deliverables.

A well-prepared test plan aligns testing activities with project objectives and reduces the likelihood of defects reaching production.

Effective test planning should achieve the following objectives:

• Define scope: Specify the functions, components, platforms, and requirements that will and will not be tested.
• Select a strategy: Determine the test levels, test types, techniques, environments, and automation approach.
• Allocate resources: Identify the people, tools, hardware, software, and test data required.
• Create a schedule: Establish milestones and realistic time estimates for test activities.
• Manage risks: Identify product and project risks and prioritize testing accordingly.
• Set completion criteria: Define measurable conditions for starting, suspending, resuming, and completing testing.
• Improve communication: Provide stakeholders with a common understanding of the testing process.
• Control quality: Define metrics and reporting procedures for evaluating product quality and test progress.

These objectives make testing measurable, traceable, and consistent with the project's business priorities.

A comprehensive test plan should cover the following topics:

• Test objectives: The quality goals and intended outcomes of testing.
• Scope: Features to be tested and features excluded from testing.
• Test items: Applications, modules, builds, interfaces, and documents under test.
• Test strategy: Test levels, test types, techniques, and automation approach.
• Test environment: Required hardware, operating systems, browsers, databases, networks, and tools.
• Test data: Data creation, masking, storage, refresh, and cleanup requirements.
• Resources and responsibilities: Team structure, required skills, and ownership of tasks.
• Schedule and estimates: Activities, dependencies, milestones, and completion dates.
• Entry and exit criteria: Conditions for starting and completing each test phase.
• Defect management: Defect workflow, severity, priority, and reporting procedures.
• Risks and contingencies: Potential obstacles and corresponding mitigation plans.
• Deliverables and metrics: Test cases, scripts, reports, logs, coverage, pass rate, and defect statistics.

Test planning and test case planning operate at different levels.

Test planning determines what testing approach will be followed, while test case planning determines how specific behavior will be verified. Test cases must therefore remain consistent with the scope, priorities, and strategy defined in the test plan.

The goals of test case planning are to:

• Achieve requirement coverage: Ensure that every relevant requirement and acceptance criterion is tested.
• Detect defects: Design cases that expose incorrect behavior, boundary problems, and error-handling failures.
• Avoid unnecessary duplication: Organize cases so that testing effort is used efficiently.
• Support repeatability: Provide clear steps, inputs, and expected results that can be executed consistently.
• Establish traceability: Link test cases to requirements, risks, defects, and releases.
• Prioritize execution: Identify critical, high-risk, smoke, regression, and lower-priority cases.
• Enable automation: Recognize stable and repeatable cases that are suitable for automated execution.
• Improve maintainability: Create modular and understandable cases that can be updated when requirements change.

These goals improve software quality by increasing coverage, making failures reproducible, and ensuring that testing concentrates on the most important product risks.

Test case planning generally includes the following stages:

1. Analyze requirements: Study functional requirements, non-functional requirements, user stories, acceptance criteria, and known risks.
2. Identify test conditions: Determine the features, business rules, workflows, interfaces, and failure conditions to verify.
3. Select test techniques: Choose methods such as equivalence partitioning, boundary value analysis, decision tables, state transitions, or use-case testing.
4. Prioritize cases: Rank cases according to risk, business impact, frequency of use, and defect history.
5. Prepare test data: Define valid, invalid, boundary, and exceptional input values.
6. Write test cases: Document preconditions, steps, data, expected results, and postconditions.
7. Review and approve: Check correctness, completeness, clarity, traceability, and duplication.
8. Organize for execution: Group cases into suites such as smoke, functional, integration, and regression suites.
9. Select automation candidates: Automate stable, repeatable, high-value cases where appropriate.
10. Maintain cases: Update, retire, or add cases as requirements and the product evolve.

Test cases should be organized using a consistent hierarchy and classification scheme. Common organizational methods include:

• Grouping by application, module, feature, or user story.
• Grouping by test level, such as unit, integration, system, and acceptance testing.
• Grouping by test type, such as functional, usability, security, performance, and compatibility testing.
• Creating execution suites such as smoke, sanity, regression, and release-specific suites.
• Applying tags for priority, risk, platform, browser, environment, and automation status.
• Assigning a unique identifier and meaningful title to every test case.
• Keeping reusable setup and cleanup activities in shared fixtures or procedures.
• Separating test logic from test data where practical.
• Maintaining version information and requirement links.

Good organization allows testers to select appropriate cases quickly, avoid duplication, distribute work, measure coverage, and update cases when the software changes.

A test case record should contain enough information to support design, execution, reporting, and maintenance. Important fields include:

• Test case ID and title for unique identification.
• Requirement or user-story reference for traceability.
• Objective or description of the behavior being verified.
• Author, owner, reviewer, and creation date.
• Priority, risk level, and test type.
• Preconditions and environment requirements.
• Test data, execution steps, and expected results.
• Automation status and script location, when applicable.
• Version, build, browser, device, or platform tested.
• Execution status, such as not run, passed, failed, blocked, or skipped.
• Actual result and evidence, including logs or screenshots.
• Linked defect IDs for failed cases.
• Execution history and maintenance status, such as active, under review, or retired.

Tracking these details supports accountability, reproducibility, auditability, and accurate progress reporting.

A Requirements Traceability Matrix (RTM) maps requirements to their corresponding test cases and, where relevant, to execution results and defects.

Its main roles are:

• Confirming that every requirement has one or more test cases.
• Identifying requirements that have not been tested.
• Detecting test cases that do not correspond to an approved requirement.
• Supporting impact analysis when a requirement changes.
• Showing the execution and defect status of each requirement.
• Providing evidence for reviews, audits, and acceptance decisions.

An RTM may contain columns such as requirement ID, requirement description, risk, test case IDs, test status, and defect IDs. Forward traceability verifies that requirements are implemented and tested, while backward traceability confirms that test cases and implemented behavior are justified by requirements.

Thus, the RTM provides a measurable connection between specifications, testing activity, and product quality.

A well-designed test case has the following characteristics:

• Clear: Its objective, steps, data, and expected result are easy to understand.
• Traceable: It is linked to a requirement, risk, acceptance criterion, or business rule.
• Repeatable: Different testers can execute it under the same conditions and obtain comparable results.
• Independent: It does not depend unnecessarily on the execution order or outcome of other cases.
• Atomic: It verifies a focused behavior so that the cause of failure is easy to identify.
• Complete: It includes required preconditions, inputs, actions, expected outcomes, and cleanup details.
• Maintainable: It avoids unnecessary detail and uses reusable data or setup components where appropriate.
• Risk-focused: It gives suitable attention to critical and failure-prone behavior.
• Observable: Its expected outcome can be measured or directly verified.
• Reusable: It can be included in multiple test suites or executed with multiple data sets.

These qualities reduce ambiguity, improve defect diagnosis, and make both manual execution and automation more reliable.

Equivalence partitioning divides the input domain into groups whose values are expected to be processed in the same way. One representative value is selected from each valid and invalid group.

Boundary value analysis concentrates on values at and immediately around the limits of those groups because defects frequently occur at boundaries.

Suppose an age field accepts values from through , inclusive.

Equivalence partitions are:

• Invalid partition: age less than .
• Valid partition: age from to .
• Invalid partition: age greater than .

Representative equivalence tests could use , , and .

Boundary tests should include:

• Lower boundary: , , and .
• Upper boundary: , , and .

Equivalence partitioning reduces the number of tests while retaining useful coverage. Boundary value analysis strengthens the design by checking the exact points where validation logic commonly fails.

Decision table testing represents combinations of conditions and the actions expected for each combination. It is useful when system behavior depends on several business rules.

The design process is:

1. List all relevant conditions.
2. List the possible values of each condition.
3. List the resulting actions.
4. Create columns representing meaningful combinations of conditions.
5. Remove impossible or duplicate combinations.
6. Convert each remaining rule column into one or more test cases.

For example, an online purchase may receive a discount when the customer is a member and the order total is at least . The conditions are membership status and order threshold. The decision table should test:

• Member with an order of at least : discount applied.
• Member with an order below : no discount.
• Non-member with an order of at least : no discount.
• Non-member with an order below : no discount.

This technique exposes missing, inconsistent, and conflicting rules while providing systematic combination coverage.

State transition testing verifies a system whose response depends on its current state, an event or input, and possibly its previous sequence of events. A transition moves the system from one state to another and may produce an output.

Test cases are derived from:

• Valid states.
• Events that trigger transitions.
• Valid state-to-state transitions.
• Invalid or prohibited transitions.
• Outputs and actions associated with transitions.

For example, a user account may move through the states active, temporarily locked, and permanently locked. One failed login may leave it active, repeated failures may temporarily lock it, and an administrator action may restore it.

State transition testing is suitable for:

• Login and account-locking workflows.
• Order, ticket, or approval status changes.
• Communication protocols.
• Embedded controllers.
• Subscription and payment life cycles.

It is preferable to simple input-output testing when the same input can produce different results depending on the system's current state or event history.

An effective manual test case normally contains:

• Test case ID and title: A unique reference and concise description.
• Objective: The requirement or behavior being verified.
• Preconditions: Required account, data, configuration, and application state.
• Environment: Relevant build, device, browser, database, or operating system.
• Test data: Exact input values needed for execution.
• Steps: Numbered actions written in a clear and executable order.
• Expected results: Observable outcomes for important steps and the final result.
• Postconditions: The expected final state and any cleanup requirement.
• Priority and traceability: Risk classification and links to requirements.

The tester first analyzes the requirement, identifies positive and negative scenarios, prepares test data, writes the steps and expected results, and reviews the case for clarity and completeness. During execution, the tester records the actual result, status, evidence, and any linked defect. The case is then maintained as the feature changes.

Manual and automated testing are complementary. Automation improves repeatability and feedback speed, while manual testing provides adaptability and human judgment. A suitable strategy selects each method according to test stability, repetition, risk, cost, and expected value.

The following factors should be considered when selecting automation candidates:

• Execution frequency: Frequently repeated smoke and regression tests offer higher value.
• Stability: Requirements and user interfaces should be sufficiently stable.
• Business criticality: High-risk workflows deserve reliable and frequent verification.
• Data variation: Tests executed with many input combinations are good candidates for data-driven automation.
• Execution effort: Long, repetitive, or error-prone manual procedures may provide significant savings.
• Result observability: Expected outcomes must be detectable through reliable assertions.
• Environment availability: Required systems, accounts, services, and test data must be controllable.
• Technical feasibility: The selected tool must support the technology being tested.
• Maintenance cost: Fragile tests may cost more to maintain than they save.
• Expected lifetime: Tests for long-lived features usually provide a better return.

Short-lived, highly subjective, exploratory, usability-oriented, or constantly changing cases are generally less suitable for automation.

A typical conversion process includes:

1. Review the manual case: Confirm its objective, steps, data, expected results, and automation suitability.
2. Choose tools: Select tools such as JUnit or TestNG for test execution, Selenium WebDriver for browser automation, and Maven or Gradle for dependency and build management.
3. Prepare the project: Configure dependencies, packages, test resources, reporting, and environment properties.
4. Automate setup: Initialize drivers, application state, accounts, and test data through fixtures.
5. Implement actions: Translate manual steps into stable methods or page-object operations.
6. Add assertions: Compare actual results with explicit expected outcomes.
7. Implement teardown: Close drivers and restore data or environment state.
8. Improve reliability: Use explicit waits and stable element locators while avoiding fixed delays.
9. Parameterize data: Separate test data from test logic when multiple inputs are required.
10. Execute and integrate: Run locally and through a continuous integration pipeline.
11. Review results: Publish reports, capture failure evidence, and maintain the script as the product changes.

These tools serve different but complementary roles:

• Selenium WebDriver: Automates interactions with web browsers. It can locate elements, click controls, enter data, navigate pages, and retrieve application state for verification.
• JUnit: Provides annotations, assertions, fixtures, test discovery, execution, and integration with Java development tools.
• TestNG: Provides similar test-running capabilities and also supports features such as groups, priorities, dependencies, parameterization, parallel execution, and data providers.
• Maven: Manages dependencies, standard project structure, build phases, plugins, and test execution through configuration in pom.xml.
• Gradle: Provides dependency and build automation through flexible build scripts, commonly stored in build.gradle or build.gradle.kts.

In a typical framework, Maven or Gradle downloads the required libraries and starts the test task. JUnit or TestNG discovers and controls the tests. Selenium WebDriver performs browser interactions, while the test framework evaluates assertions and records results.

The Page Object Model (POM) represents each application page or major component as a class. The class stores element locators and exposes meaningful operations such as login, search, or submitOrder. Test classes use these operations instead of directly manipulating every element.

Reusable utilities centralize common behavior such as:

• Driver creation and browser configuration.
• Explicit waits and synchronization.
• Screenshot and logging support.
• Configuration loading.
• API, database, and file operations.
• Test data setup and cleanup.

Data-driven testing separates inputs and expected values from test logic. JUnit parameterized tests, TestNG data providers, CSV files, JSON documents, or databases can supply multiple data sets to one test method.

Together, these practices:

• Reduce duplicated code.
• Improve readability and maintainability.
• Localize changes when the interface changes.
• Increase coverage using multiple inputs.
• Encourage consistent framework behavior.

However, page objects should model useful page behavior and should not become oversized classes containing unrelated assertions or business workflows.

Reliable and maintainable Java automated tests should follow these practices:

• Use stable locators, preferably unique IDs or dedicated test attributes.
• Use explicit waits for meaningful conditions instead of fixed sleep statements.
• Keep tests independent and able to run in any order.
• Create and clean up test data predictably.
• Separate test logic, page behavior, configuration, and test data.
• Use descriptive test names and focused assertions with useful failure messages.
• Capture logs, screenshots, page source, or network evidence when failures occur.
• Control browser, driver, library, and runtime versions through Maven or Gradle.
• Store environment-specific values outside the source code and protect secrets.
• Use tags or groups to define smoke, regression, integration, and release suites.
• Support headless and parallel execution only after ensuring isolation and thread safety.
• Publish machine-readable and human-readable reports in the continuous integration system.
• Investigate flaky tests instead of repeatedly rerunning them without diagnosis.

In a continuous integration pipeline, tests should produce deterministic exit codes, clear reports, and failure artifacts so that defects can be diagnosed quickly.