Unit 5: Testing, Validation and Customer Experience - Subjective Questions

User testing is the process of observing representative users interact with a product, prototype, or service to identify problems and evaluate whether it meets their needs.

Its role in design thinking includes:

• Evaluating usability: It shows whether users can complete important tasks efficiently.
• Validating assumptions: It tests whether the design solves the problem identified during research.
• Identifying user difficulties: It reveals confusion, errors, frustration, and unmet expectations.
• Improving the solution: Findings are used to refine the prototype before development or launch.
• Reducing risk: Testing prevents expensive design decisions based only on team opinions.

User testing is most effective when conducted repeatedly throughout the design process rather than only at the end.

Design validation determines whether a product meets real user needs and is suitable for its intended context of use.

Design verification checks whether the product has been built according to specified requirements and technical standards.

The distinction can be summarized as follows:

• Validation asks: Are we building the right product?
• Verification asks: Are we building the product correctly?
• Validation generally involves users, realistic tasks, prototypes, and field conditions.
• Verification may involve technical reviews, inspections, requirement checks, and system tests.
• A product can pass verification but fail validation if it meets technical specifications but does not solve the user's actual problem.

Both activities are necessary for a reliable and useful design.

Neutral user testing is designed to collect unbiased evidence about how users behave and think when interacting with a product.

Important principles include:

• Avoid leading questions: Ask open questions such as "What do you expect to happen here?" instead of suggesting an answer.
• Do not teach during the test: The facilitator should observe the user's natural interaction before providing help.
• Use neutral language and tone: Facial expressions, gestures, and comments should not indicate approval or disapproval.
• Focus on behavior: Record what users do, where they hesitate, and what errors they make.
• Test representative users: Participants should resemble the intended target audience.
• Use consistent procedures: Each participant should receive comparable instructions and tasks.
• Separate observation from interpretation: First record evidence, then analyze possible causes.

These principles improve the reliability and credibility of test findings.

A task scenario is a realistic situation that asks the participant to accomplish a meaningful goal using the product.

An effective scenario should:

• Describe a believable context and user goal.
• Represent an important activity that users are expected to perform.
• Avoid giving step-by-step instructions or naming the exact interface control to use.
• Contain enough information for the participant to begin independently.
• Have a clear success condition.
• Be neither so easy that it reveals nothing nor so difficult that it becomes unrealistic.
• Use language familiar to the target users.
• Be connected to a measurable objective, such as task completion, errors, time, or satisfaction.

For example, instead of saying "Click the search icon and find a laptop," a neutral scenario could state: "You need to find a laptop suitable for basic office work within your budget. Use the website to identify one option you would consider."

The show, do not tell approach emphasizes observing users perform tasks instead of relying only on their verbal opinions or explanations.

Its usefulness comes from the following points:

• Users may describe an interface as easy while still making errors during actual use.
• Observed behavior reveals hesitation, incorrect assumptions, navigation problems, and workarounds.
• Demonstrations of real tasks provide stronger evidence than general statements such as "I like it."
• Users often cannot accurately predict how they will behave in a future situation.
• The approach exposes the difference between intended use and actual use.

The facilitator should ask the participant to think aloud when appropriate, but should prioritize direct observation of actions, decisions, and outcomes. Verbal feedback remains useful, but it should be interpreted alongside observed behavior.

A feedback capture matrix organizes observations so that evidence can be compared and converted into design improvements. A suitable matrix may contain the following columns:

The matrix should distinguish direct evidence from assumptions. This prevents isolated opinions from being treated as proven findings.

Usability refers to how effectively, efficiently, and satisfactorily users can accomplish goals with a product in a particular context. Customer experience is broader and includes the user's complete perception before, during, and after interacting with an organization.

Usability contributes to customer experience by:

• Making tasks easier and reducing effort.
• Increasing user confidence and perceived reliability.
• Reducing errors, frustration, and abandonment.
• Helping customers reach value quickly.
• Supporting positive perceptions of the brand.
• Improving the likelihood of repeat use and recommendation.

However, good usability alone does not guarantee an excellent customer experience. Customers may also be affected by price, service quality, trust, product performance, delivery, and support. Therefore, usability is a major part of customer experience but not its only component.

A customer journey map is a visual representation of the stages a customer passes through while trying to achieve a goal with an organization or product.

A journey map commonly includes:

• Stages: Such as awareness, consideration, purchase, use, support, and renewal.
• User goals: What the customer wants at each stage.
• Actions: What the customer does.
• Thoughts and emotions: What the customer expects, feels, or worries about.
• Touchpoints: Where interaction occurs, such as a website, mobile application, store, email, or support channel.
• Pain points: Difficulties, delays, confusion, or dissatisfaction.
• Opportunities: Possible improvements.

Journey mapping supports design decisions by revealing gaps between channels, identifying high-impact moments, showing where users abandon the process, and helping teams prioritize improvements based on the complete experience rather than a single screen.

A user touchpoint is any moment or channel through which a customer interacts with a product, service, or organization. Examples include:

• A product advertisement.
• A registration page.
• A mobile application screen.
• A payment counter.
• A confirmation email.
• A customer support call.

A pain point is a problem or source of dissatisfaction experienced by the customer during the journey. Examples include:

• A registration form that requests unnecessary information.
• A slow payment process.
• An unclear error message.
• Inconsistent information between a website and a support agent.
• A difficult return procedure.

A touchpoint is an interaction location, while a pain point is a difficulty that may occur at that location. One touchpoint can contain several pain points, and mapping both helps teams identify where improvements are needed.

Pain points can be prioritized by considering both their effect on customers and the effort required to address them.

Useful criteria include:

• Frequency: How many users experience the problem?
• Severity: How strongly does it prevent task completion or reduce satisfaction?
• Business impact: Does it cause lost sales, support costs, churn, or reputational damage?
• Strategic importance: Is the affected journey stage critical to the organization's goals?
• Accessibility impact: Does the issue disproportionately affect users with disabilities?
• Implementation effort: How much time, money, and technical work are required?
• Confidence in evidence: Is the finding supported by research, analytics, or repeated testing?

A common prioritization method is to classify issues as quick wins, major projects, low-priority improvements, or items that should be monitored. High-frequency and high-severity problems should generally receive the earliest attention, even when they require significant effort.

Navigation allows users to understand where they are, find desired information, move between functions, and recover from mistakes. Poor navigation increases cognitive load and can cause users to abandon a product.

A well-designed navigation system should provide:

• Clarity: Labels use familiar and specific terms.
• Consistency: Navigation behaves similarly across screens.
• Visibility: Important navigation options are easy to find.
• Logical grouping: Related functions are placed together.
• Hierarchy: Primary, secondary, and supporting options are distinguishable.
• Orientation: Users can recognize their current location through titles, breadcrumbs, selected states, or other indicators.
• Predictability: Controls behave as users expect.
• Efficient access: Common tasks require a reasonable number of steps.
• Recovery: Back navigation, cancel options, and undo actions help users correct mistakes.

Navigation should be tested with realistic tasks rather than evaluated only by visual appearance.

Usability testing and customer experience research are related but focus on different levels of interaction.

Usability testing can reveal why a customer struggles at a particular touchpoint, while customer experience research explains how that struggle affects the broader relationship with the organization.

A moderated user testing session can be conducted through these stages:

• Define objectives: Identify the decisions and usability questions the test must answer.
• Prepare tasks: Create realistic scenarios with clear success criteria.
• Recruit participants: Select users who represent the target audience.
• Prepare materials: Set up the prototype, consent process, recording method, and observation notes.
• Open the session: Explain the purpose, reassure the participant that the product is being tested, and request permission to record.
• Run the tasks: Give one scenario at a time and allow the participant to work independently.
• Probe neutrally: Ask questions such as "What are you looking for?" or "What were you expecting?"
• Record evidence: Note actions, errors, hesitation, comments, and task outcomes.
• Debrief: Ask about overall impressions and major difficulties after the tasks are complete.
• Analyze results: Group findings, assess severity, identify patterns, and recommend changes.

The moderator should avoid rescuing the participant too quickly because difficulty itself may be an important finding.

A leading question suggests a preferred answer or behavior. For example, asking "Was the blue button easy to find?" implies that the button should have been found and may encourage agreement.

Facilitator intervention can reduce validity when the moderator:

• Gives hints that users would not receive in normal use.
• Completes part of a task for the participant.
• Reacts positively or negatively to user choices.
• Rephrases a task until the participant understands the intended path.
• Defends the design or explains how it is supposed to work.
• Interrupts moments of productive struggle.

These behaviors change the conditions of the test and can hide real problems. A neutral facilitator should provide standardized instructions, allow users time to think, ask open-ended questions, and record observations without judgment.

User testing is stronger when it combines quantitative and qualitative evidence.

Quantitative evidence may include:

• Task completion rate.
• Number of errors.
• Time required to complete a task.
• Number of assistance requests.
• Ratings of ease, confidence, or satisfaction.

Qualitative evidence may include:

• User comments and expectations.
• Observed strategies and workarounds.
• Emotional reactions and hesitation.
• Explanations of why a feature was misunderstood.
• Contextual factors affecting behavior.

Quantitative data indicates the size or frequency of a problem, while qualitative data helps explain its cause. For example, a low completion rate shows that a task is difficult, while observation may reveal that users cannot distinguish two similarly labeled controls. The two forms of evidence should be interpreted together rather than allowing a single metric to determine the conclusion.

Accessibility means designing digital products so that people with different abilities can perceive, understand, navigate, and interact with them.

Accessibility should be considered during design validation because:

• People with visual, auditory, motor, speech, cognitive, and learning disabilities may experience different barriers.
• A design that works for a majority of users may still exclude important groups.
• Accessibility problems are more expensive to correct after implementation.
• Accessible products often improve usability for everyone, including users on small screens, in noisy environments, or with temporary injuries.
• Accessibility supports fairness, legal compliance, and inclusive participation.

Validation should include users with disabilities where possible and should also include technical checks such as keyboard operation, screen reader compatibility, color contrast, focus visibility, captions, and meaningful text alternatives.

Accessibility testing should combine automated tools, manual inspection, and testing with people who use assistive technologies.

Practical methods include:

• Keyboard testing: Confirm that all functions can be reached and operated without a mouse.
• Focus testing: Check that focus order is logical and the focused element is clearly visible.
• Screen reader testing: Verify that headings, labels, controls, status messages, and form errors are announced meaningfully.
• Color and contrast checks: Ensure that information is not conveyed by color alone and that text has sufficient contrast.
• Zoom and reflow testing: Check whether content remains usable at enlarged text sizes and narrow screen widths.
• Alternative text review: Ensure that informative images have useful descriptions and decorative images are handled appropriately.
• Caption and transcript review: Provide equivalent access to audio and video content.
• Plain language review: Confirm that instructions and messages are understandable.
• User testing: Include participants with relevant disabilities to identify barriers that tools may miss.

Automated tools are helpful for detecting some defects, but they cannot replace human evaluation.

These principles work together to make software understandable and controllable.

• Consistency: Similar actions, labels, layouts, and visual patterns should behave in similar ways. This allows users to transfer existing knowledge.
• Visibility: Important options, system status, and available actions should be apparent when needed. Hidden functions increase discovery time.
• Feedback: The system should communicate the result of an action, such as showing a confirmation, progress state, or error message.
• Error prevention: The interface should reduce the likelihood of errors through constraints, sensible defaults, validation, confirmation for risky actions, and clear instructions.

For example, a payment form can use consistent field labels, visibly indicate required fields, show progress during submission, and prevent invalid entries before the form is sent. Together, these principles reduce confusion, improve confidence, and support successful task completion.

Customer experience is influenced not only by task success but also by emotions such as trust, confidence, relief, confusion, anxiety, and frustration. A customer may complete a task but still have a negative experience if the process feels risky or exhausting.

Emotional responses can be captured through:

• Interviews conducted immediately after important interactions.
• Open-ended questions about expectations and feelings.
• Journey maps showing emotional changes across stages.
• Diary studies that record experiences over time.
• Satisfaction and confidence ratings.
• Observation of hesitation, facial expression, tone, and repeated actions.
• Support requests, complaints, reviews, and abandonment data.

Researchers should avoid treating emotional observations as definitive without context. They should compare them with user statements and behavioral evidence to identify the cause and impact of the emotion.

Converting findings into requirements requires moving from raw observations to clearly defined, testable improvements.

A suitable process is:

• Collect evidence: Record task results, user behavior, quotes, and contextual details.
• Group related observations: Combine findings that appear to have a common cause.
• Identify the underlying need: Separate the user's need from the specific solution they may suggest.
• Assess severity and priority: Consider frequency, task impact, business value, accessibility, and implementation effort.
• Write a clear requirement: State what the product must enable or communicate.
• Define acceptance criteria: Describe how the team will determine whether the change works.
• Prototype the response: Explore one or more possible design solutions.
• Retest: Confirm that the change resolves the original problem without creating new ones.

For example, instead of recording "Make the button bigger," a requirement could state that users must be able to identify and activate the primary action without confusion on both desktop and mobile layouts.