The Measuring Experience Field Guide
UNDERSTANDING USER EXPERIENCE MEASUREMENTS
The success of a product or service is largely determined by the user experience (UX) it provides. Understanding and optimizing UX measurements is critical for ensuring that the product or service meets user needs and expectations. By understanding and utilizing UX measurements effectively, product owners and designers can create exceptional digital experiences that meet and exceed user expectations.
“
Anything we can observe can become a measurement. Any measurements we track over time become metrics. It's as simple as that.
User Experience at Scale
Investing in good UX design reduces support costs by up to 90%. And improves key performance indicators (KPIs) by up to 80%.
UX outcomes are an essential tool for showing the business value of delivering well-designed products and services.
UX outcomes are an essential tool for showing the business value of delivering well-designed products and services.
Ease of use is usually one of the biggest differentiators in the customer experience. If people don't find your product easy to use, they aren't going to be very satisfied or loyal.
Users acquire goods and services based on two basic behaviors:
(1) accomplish affective (hedonic) gratification (from sensory attributes)
(2) instrumental, utilitarian reasons (pragmatic).
Users acquire goods and services based on two basic behaviors:
(1) accomplish affective (hedonic) gratification (from sensory attributes)
(2) instrumental, utilitarian reasons (pragmatic).
Hedonic
Perceived Usefulness - the degree to which a user believes that a computer system enhances his job performance.
Perceived Ease of Use - the degree to which a user beliefs that the system use is effortless.
Perceived Ease of Use - the degree to which a user beliefs that the system use is effortless.
Appeal
Pragmatic
Perceived Functional Beauty - the degree to which an object is as suitable for its purpose and beautiful at the same level. If it is suitable but doesn't look good, it's just "fitness for function" and may not delight the user.
Measuring usability is to assess to each extent a product is being used with effectiveness, efficiency, and satisfaction in a specified context of use.
Perceived Usability - the degree to which a user perceives the quality of the use of white spaces, the readability of the font, how balanced is the visual elements, and how clean and clear is the layout and interaction.
MEASURES USABILITY
- Pragmatic quality
- Ergonomic quality
- Perceived usability
- Task-oriented performance
- Instrumental product quality
- Ability to find products or information
MEASURES ATTITUDES
- Study-oriented performance
- Hedonic quality
- Non-instrumental quality
- Attitudes towards visual appearance, trust and credibility
MEASURES USABILITY + ATTITUDES*
*Users' beliefs, ideas, and opinions
- Appealingness of a product. And it is affected by both pragmatic and hedonic aspects of user experience.
- Perception of ease of use, usefulness and satisfaction
*Users' beliefs, ideas, and opinions
Pragmatic
Pragmatic dimension resulting from function derived from product or service performance.
10 of the most essential usability metrics
Completion Rates
Usability Problems
Test Level Satisfaction
Average Time on Task
Task Level Satisfaction / Feature Usage
Errors Count
Expectation
Page Views/Clicks
Conversion Rate
Single Usability Metric
Hedonic
Hedonic dimension resulting from sensations derived from the experience of using products
10 of the most essential psychometric metrics
Emotion & affection
Fun & Enjoyment
Appealingness
Novelty & Innovation
Stimulation
Convenience
Satisfaction Rating
Pleasure
Usefulness & Ease-of-use Rating
Reliability
Why measure experience?
Measuring the user experience is seeing whether design efforts actually make a quantifiable difference over time. There are 3 reasons why you should measure an experience:
1
Obtaining a quantitative usability benchmark allows us to understand how to design and functional changes impacted the user experience.
2
User Experience benchmarking is an effective method for understanding how people use (pragmatic) and think (hedonic) about a digital solution.
3
UX Benchmark studies are most effective when conducted at regular intervals (e.g. annually or quarterly) when compared against competitors, earlier versions, or an industry standard.
Obtaining a quantitative usability benchmark allows us to understand how to design and functional changes impacted the user experience.
2
User Experience benchmarking is an effective method for understanding how people use (pragmatic) and think (hedonic) about a digital solution.
3
UX Benchmark studies are most effective when conducted at regular intervals (e.g. annually or quarterly) when compared against competitors, earlier versions, or an industry standard.
Benefits of measuring experience
We measure experience to improve the old and build the new. This generates better design processes through data-driven decision-making. Having both quantitative and qualitative skills onboard give you an advantage over those business who can't combine both.
1
Provides an In-Depth Understanding of Users
Establishing Brand Value and Memorability
Establishing a Competitive Advantage
Provides an In-Depth Understanding of Users
- Help C-level to see that UX has tangible consequences and benefits to their ROI.
- Engage stakeholders and secure the buy-in of more data-driven stakeholders.
Establishing Brand Value and Memorability
- Excellent ROI on correcting bad design choices.
- Identify product strengths and weaknesses.
- Inform better design decisions and validate designs against business objectives.
Establishing a Competitive Advantage
- Higher customer satisfaction.
- Mapping usability issues.
- Map opportunities for higher user retention or feature adoption.
- Direction for further product development.
Every strategy can be measured.
Is a usable experience sufficient for a good experience?
No! An experience is made up of a series of personal judgments grounded in a journey whereby a person discovers, purchases (or not), experiences, and takes part in a service. This journey creates in the user a feeling of how well a business or solution is meeting their needs. So, the user does not take only the interface quality into consideration when evaluating the quality of a solution. Nor does only evaluate the usability quality at the moment of interaction with the service or solution. There is a number of pragmatic and hedonic characteristics that dictate how a user perceives the quality of an experience. For that reason, usability is attitude (hedonic) plus action (pragmatic).
The result of an interaction, usage, and consumption always translates into an experience, a story of use, and a story of consumption that we can measure.
