Unit 5: Performance Testing Tools - Subjective Questions

Open-source tools:

• Usually free to download and use, reducing licensing costs.
• Provide access to source code and allow customization.
• Have active communities, forums, plugins, and shared documentation.
• Examples include Apache JMeter, Gatling, and Taurus.
• May require more technical expertise and may not include dedicated vendor support.

Commercial tools:

• Usually provide professional technical support, training, dashboards, and integrations.
• Often include advanced analytics, service-level monitoring, and enterprise reporting.
• Examples include LoadRunner, NeoLoad, and BlazeMeter enterprise solutions.
• Licensing and infrastructure costs can be high.

Comparison: Open-source tools are suitable for educational projects, startups, and teams with strong technical skills. Commercial tools are useful for large organizations that need vendor support, extensive reporting, governance, and integration with enterprise systems. The selection should consider budget, protocol support, scalability, learning curve, reporting needs, and maintenance requirements.

Apache JMeter is an open-source Java-based performance testing tool developed by the Apache Software Foundation. It is primarily used to measure the performance and behavior of applications under different levels of load.

Main purposes of JMeter include:

• Load testing web applications and APIs.
• Stress testing systems beyond their expected capacity.
• Endurance or soak testing over a long period.
• Functional verification of request and response behavior.
• Measuring response time, throughput, error percentage, and latency.

JMeter can test HTTP and HTTPS web applications, REST and SOAP services, databases through JDBC, FTP servers, Java applications, TCP services, JMS systems, LDAP services, and other protocols through extensions. It is especially popular for testing web applications and APIs because it supports reusable test plans, parameterization, assertions, timers, controllers, and distributed execution.

The major advantages of JMeter are:

• Open source: JMeter is freely available, which reduces tool acquisition and licensing costs.
• Cross-platform support: Since it is Java-based, it can run on Windows, Linux, and macOS.
• Protocol support: It supports HTTP, HTTPS, REST, SOAP, JDBC, FTP, JMS, LDAP, TCP, and other protocols.
• Extensibility: Plugins, custom Java code, listeners, and functions can extend its capabilities.
• GUI and command-line modes: Tests can be designed through the GUI and executed in non-GUI mode for better performance.
• Rich test components: It provides samplers, controllers, timers, assertions, preprocessors, postprocessors, and configuration elements.
• Distributed testing: Multiple machines can generate load against a target system.
• Reusable test plans: Variables, CSV files, functions, and templates support data-driven testing.
• Useful metrics: JMeter reports response time, throughput, errors, latency, and percentile values.
• Strong community support: Documentation, tutorials, and plugins are widely available.

The installation process generally involves the following steps:

1. Install Java: JMeter requires a compatible Java Runtime Environment or Java Development Kit. The JAVA_HOME environment variable should point to the Java installation directory.
2. Download JMeter: Visit the official Apache JMeter website and download the binary archive appropriate for the operating system.
3. Extract the archive: Extract the downloaded ZIP or TAR file to a suitable directory. JMeter is portable and normally does not require a traditional installer.
4. Start JMeter: Open the bin directory and run jmeter.bat on Windows or jmeter on Linux and macOS.
5. Verify the installation: Confirm that the JMeter GUI opens successfully and that the version is displayed correctly.
6. Configure resources: For larger tests, adjust JVM heap settings in the JMeter startup configuration file.
7. Install plugins if required: Use the Plugins Manager to add supported components, while ensuring that plugin versions are compatible.

For actual load execution, tests should generally be run in non-GUI mode because the GUI consumes additional memory and processing resources.

A JMeter test plan is composed of several types of elements:

• Test Plan: The root container that holds all test components and defines the overall test structure.
• Thread Group: Defines the number of virtual users, ramp-up period, and loop count.
• Config Elements: Supply reusable settings such as server names, headers, cookies, credentials, and data files.
• Samplers: Generate requests to the application, such as HTTP Request, JDBC Request, or FTP Request.
• Controllers: Control the logical execution flow of samplers, including loops, conditions, transactions, and random selection.
• Timers: Introduce delays between requests to simulate realistic user behavior.
• Assertions: Validate responses and identify functional or performance failures.
• Preprocessors: Modify or prepare data before a sampler executes.
• Postprocessors: Extract values from responses and store them for later requests.
• Listeners: Display or save test results, statistics, and reports.
• Workbench: A temporary workspace for experimenting with components; it is not normally part of the saved test execution plan.

Together, these elements model user behavior and evaluate system performance.

Configuration elements provide default values and reusable settings for samplers. They help avoid repeating the same information in every request.

Common configuration elements include:

• HTTP Request Defaults: Stores common protocol, server name, port, path, or implementation values for HTTP requests.
• HTTP Header Manager: Adds headers such as Content-Type, authorization tokens, or custom application headers.
• HTTP Cookie Manager: Stores, sends, and updates cookies to simulate browser sessions.
• HTTP Cache Manager: Simulates browser caching behavior for HTTP resources.
• CSV Data Set Config: Reads test data from a CSV file and assigns values to JMeter variables for parameterization.
• User Defined Variables: Defines reusable values such as host names, ports, user IDs, or environment names.
• Login Configuration Element: Supplies credentials for applicable protocols.
• JDBC Connection Configuration: Defines database connection details used by JDBC samplers.

The scope of a configuration element depends on its position in the test tree. It generally applies to samplers within the same scope and below it.

The JMeter GUI contains several important areas:

• Menu bar: Provides commands for creating, opening, saving, validating, and running test plans.
• Toolbar: Contains shortcuts for adding elements, starting and stopping tests, clearing results, and changing display options.
• Test tree: Appears on the left and displays the hierarchical structure of the test plan.
• Work area: Appears on the right and shows the configuration fields for the selected element.
• Status bar: Displays execution status, active threads, and warning information.
• Log viewer: Shows messages, warnings, and errors generated by JMeter.

A tester normally creates a Test Plan, adds a Thread Group, inserts configuration elements and samplers, adds timers and assertions, and selects listeners for result analysis. Elements are configured by selecting them in the tree and entering values in the work area. The test can then be saved as a .jmx file and executed from the toolbar or command line.

Test Plan:

• Represents the complete performance test definition.
• Contains Thread Groups, samplers, controllers, configuration elements, timers, assertions, and listeners.
• Is saved in a JMX file and can be executed later.
• Defines the actual workload and behavior sent to the target system.

Workbench:

• Provides a temporary area for experimenting with JMeter components.
• Can be used to test or prepare elements before placing them in the Test Plan.
• Does not normally represent part of the actual performance scenario.
• Components in the Workbench may not be included when the Test Plan is saved or executed, depending on the selected save options.

Therefore, the Test Plan contains the executable test design, while the Workbench is mainly a temporary workspace. Testers should verify the location of every important element before saving and running a test.

The procedure for adding elements is as follows:

1. Open or create a Test Plan.
2. Right-click the required parent element in the test tree.
3. Select Add and choose the required category, such as Threads, Config Element, Sampler, Logic Controller, Timer, Assertion, or Listener.
4. Select the newly created element and configure its properties in the right-hand work area.
5. Move or duplicate elements when necessary to place them under the correct parent.
6. Save the test plan as a JMX file.

The hierarchy is important because it determines scope and execution order. A Thread Group controls the virtual users below it. A configuration element usually affects child samplers within its scope. A timer delays samplers in its scope, and a controller determines how child samplers are selected or repeated. Incorrect placement can cause settings to be ignored, applied too broadly, or applied to the wrong requests.

A JMX file is the XML-based file used to store a JMeter test plan.

Creation and configuration:

• Start JMeter and create a Test Plan.
• Add a Thread Group and define users, ramp-up time, and loop count.
• Add required configuration elements, samplers, timers, assertions, controllers, and listeners.
• Enter the target server, paths, methods, parameters, headers, and test data.

Saving:

• Select File > Save Test Plan As.
• Provide a meaningful name with the .jmx extension.
• Store the file with related CSV files, scripts, and properties in an organized test directory.

Running:

• Validate the test plan and check that variables, endpoints, and credentials are correct.
• Start the test from the GUI for debugging.
• For actual load generation, use non-GUI mode, for example:
  jmeter -n -t testplan.jmx -l results.jtl -e -o report.
• Stop the test gracefully and review the generated results.

A JMX file should be version-controlled and kept independent of machine-specific paths where possible.

Important configuration factors include:

• Target details: Correct protocol, server name, port, base path, and endpoint must be entered.
• Request method: Choose the correct HTTP method such as GET, POST, PUT, or DELETE.
• Parameters and body: Add query parameters, form data, or correctly formatted JSON and XML bodies.
• Headers and cookies: Configure content types, authentication tokens, sessions, and browser-like behavior.
• User data: Use variables and CSV Data Set Config for unique users, search terms, or transaction values.
• Scope: Place configuration elements at the appropriate level so that only intended samplers inherit them.
• Timeouts: Set connection and response timeouts to prevent indefinitely blocked requests.
• Encoding: Use the appropriate character encoding for request data and responses.
• Assertions: Define checks for expected status codes, response text, or response time.
• Resource usage: Avoid unnecessary listeners and large response storage during load execution.

Correct configuration is essential because an incorrectly configured test may measure errors or unrealistic traffic instead of real application performance.

A test plan is saved by selecting File > Save Test Plan As, choosing a suitable directory, and giving the file a descriptive .jmx name. The tester should save after major changes and before execution.

Best practices include:

• Use descriptive names that identify the application, scenario, and version.
• Store JMX files, CSV data, properties files, scripts, and result files in an organized directory structure.
• Use relative paths or JMeter properties instead of hard-coded machine-specific paths.
• Keep test plans under version control.
• Document required Java, JMeter, plugin, and environment versions.
• Remove unnecessary listeners before heavy execution.
• Save separate plans or property files for different environments when appropriate.
• Protect passwords, tokens, and personally identifiable information.
• Validate that all referenced files exist on the machine or remote load generators.

Saving only the JMX file is insufficient if the test depends on external data files, plugins, certificates, or custom scripts.

A report can be created by first running the test in non-GUI mode and storing raw results in a JTL file. For example:

jmeter -n -t testplan.jmx -l results.jtl -e -o report.

The -e option generates a dashboard after the test, and -o specifies an empty output directory for the HTML report. The report can then be opened in a browser.

Important metrics include:

• Response time: Time taken to complete a request.
• Average response time: Useful for general comparison but can hide slow outliers.
• Median: The middle response-time value.
• Percentiles: Values such as the 90th, 95th, or 99th percentile show the experience of slower users.
• Throughput: Number of requests or transactions processed per unit of time.
• Error percentage: Proportion of failed requests.
• Latency: Time before the first response is received.
• Active threads: Approximate number of concurrent virtual users.
• Connect time: Time required to establish a connection.

Reports should be compared with workload levels, server resource data, and service-level objectives.

A suitable test plan can be designed as follows:

• Test Plan: Create the root container and define variables such as host, protocol, and environment.
• Thread Group: Configure the number of users, ramp-up period, loop count, and scheduler duration.
• HTTP Request Defaults: Set the common server name, protocol, and port.
• CSV Data Set Config: Load unique usernames, passwords, and search terms.
• HTTP Cookie Manager: Maintain user sessions.
• HTTP Header Manager: Add required headers such as Content-Type and authorization headers.
• Login sampler: Send the login request with parameterized credentials.
• Login assertion: Verify a successful status code and a response marker such as a session token.
• Postprocessor: Extract the token or product identifier for later requests.
• Search sampler: Send a parameterized search request.
• Timers: Add realistic delays between login and search actions.
• Controllers: Use a Transaction Controller to measure login and search as business transactions.
• Listeners: Use lightweight listeners during debugging and generate an HTML report after execution.

The plan should be validated with a small number of users before load is increased. Results should be evaluated against response-time and error-rate objectives.

Timers introduce delays before samplers execute. They help simulate the time a real user spends reading a page, entering information, or deciding what to do next. Without timers, JMeter may send requests continuously and create an unrealistic workload.

Common timer types include:

• Constant Timer: Adds a fixed delay in milliseconds.
• Uniform Random Timer: Adds a random delay within a specified range.
• Gaussian Random Timer: Generates delays based on a Gaussian distribution around a deviation and offset.
• Constant Throughput Timer: Attempts to maintain a target throughput across the selected scope.
• Synchronizing Timer: Holds threads until a specified number arrive and then releases them together, which is useful for burst testing.
• JSR223 Timer: Allows custom delay logic using a scripting language.

Timer scope and placement are important. A timer generally affects samplers in its scope, and multiple timers may contribute to the final delay. Timers should be selected to represent realistic user behavior and should not be confused with server response time.

Assertions verify that a sampler response meets expected functional or performance conditions. A request that returns HTTP status 200 may still be functionally incorrect, so assertions are required to detect invalid content.

Types of assertions include:

• Response Assertion: Checks response text, response code, response message, URL, or response headers.
• JSON JMESPath Assertion: Validates values or structures in a JSON response using a query expression.
• XML Assertion: Checks whether an XML response is well formed.
• Duration Assertion: Fails a request if its response time exceeds a specified limit.
• Size Assertion: Verifies that the response size is within expected limits.
• HTML Assertion: Checks HTML content for basic validity.
• JSR223 Assertion: Supports custom validation using a script.

Assertions should validate meaningful business conditions, such as the presence of a login token or a product name. Excessive or poorly designed assertions can consume resources, so they should be kept focused during high-load execution.

Controllers determine the logical flow and execution behavior of samplers.

• Simple Controller: Groups related elements without changing their execution logic. It improves organization and scope management.
• Loop Controller: Repeats its child elements a specified number of times or according to a variable.
• If Controller: Executes child elements only when a condition evaluates to true. The condition can use a variable or expression.
• Transaction Controller: Measures the combined time of its child requests as one business transaction. It can include or exclude timer processing depending on configuration.

Controllers can be nested to model realistic workflows. For example, a Loop Controller can repeat a search operation, an If Controller can handle an optional checkout step, and a Transaction Controller can measure the complete login process. Controllers should be arranged carefully because their nesting determines execution order and scope.

Processors perform actions before or after a sampler executes. They are used to prepare requests, process responses, extract dynamic values, and support correlation.

Preprocessors:

• Execute before the associated sampler.
• Modify request data or prepare variables.
• Examples include User Parameters, HTTP URL Re-writing Modifier, HTML Link Parser, and JSR223 PreProcessor.
• They can generate values, calculate signatures, encode data, or add dynamic request parameters.

Postprocessors:

• Execute after the sampler receives a response.
• Extract or process data from the response for later requests.
• Examples include JSON Extractor, Regular Expression Extractor, CSS Selector Extractor, XPath Extractor, and Boundary Extractor.
• They are commonly used to capture session IDs, authentication tokens, order numbers, and other dynamic values.

For example, a JSON Extractor can capture an access token from a login response, and the next HTTP Request can use it as ${token} in an authorization header.

Correlation is the process of capturing a dynamic value from one server response and using it in a subsequent request. Dynamic values may include session IDs, CSRF tokens, authentication tokens, and generated order numbers.

Typical correlation process:

1. Identify the dynamic value in the response of an earlier sampler.
2. Add an appropriate postprocessor, such as a JSON Extractor, Regular Expression Extractor, Boundary Extractor, CSS Selector Extractor, or XPath Extractor.
3. Store the extracted value in a JMeter variable.
4. Reference the variable in a later sampler using syntax such as ${sessionToken}.
5. Add a Debug Sampler or listener during development to confirm the value.
6. Use an assertion to ensure that extraction succeeded.

Preprocessors can then transform the extracted value, calculate a signature, or place it into a header or request body. Correct correlation prevents scripts from replaying expired or hard-coded values and makes the test behave like real user sessions.

Distributed testing uses multiple machines to generate load against one target system. It is useful when a single load generator cannot create the required number of virtual users or when load must originate from different locations.

Architecture:

• The controller or client machine starts the test and coordinates execution.
• Remote engines or server machines execute the Thread Groups and generate requests.
• The application under test receives traffic from all remote engines.
• Results may be collected centrally or stored by each engine for later analysis.

Procedure:

1. Install compatible Java and JMeter versions on all machines.
2. Configure the remote engines and their network access.
3. Ensure firewall rules, ports, authentication, and hostname resolution are correct.
4. Place required JMX files, CSV files, plugins, certificates, and scripts on each engine.
5. Configure remote hosts in the JMeter properties file.
6. Start the remote engines.
7. Run the test from the controller using remote-start or remote-start-all options.
8. Collect and analyze the results.

The machines should be synchronized and monitored because network overhead, controller limitations, or insufficient load-generator resources can affect test validity.