The Measuring Experience Field Guide
METHODOLOGIES FOR MEASURING USER EXPERIENCE
Measuring experiences is essential for creating exceptional digital experiences that delight and engage users. There are several methodologies available for measuring UX, ranging from surveys and interviews to usability testing and analytics. Each methodology provides unique insights into different aspects of the user experience and can help product teams identify areas for improvement. In this section, I will provide guidance on how to select the appropriate methods for different types of products or services.
How to Lead with UX Metrics?
Experience metrics are different from other business, sales, or marketing metrics because they measure something about people feeling, behavior, or attitudes. The right metrics will tell the story of how investing in UX enhancements will dramatically improve your users' experience. You can use UX metrics to tell a story about how a better experience is good for your business.
Identify
Metrics that tell the story of how your UX ideas will improve the lives of your users, customers, and employees.
Craft
A compelling story of the importance of better UX by combining clear qualitative and quantitative research findings.
Drive Change
Point out the damage that comes from delivering poor UX when you present metrics about how much it currently costs your organization.
Metrics that tell the story of how your UX ideas will improve the lives of your users, customers, and employees.
Craft
A compelling story of the importance of better UX by combining clear qualitative and quantitative research findings.
Drive Change
Point out the damage that comes from delivering poor UX when you present metrics about how much it currently costs your organization.
Leading vs Lagging
Covey and McChesney, in their book, The 4 Disciplines of Execution, make the distinction between metrics that are leading (looking forward) and lagging (looking backward). They emphasize the importance of management paying attention to leading metrics that are key drivers of lagging metrics.
UX Measures
Leading
Lagging
DESIGN INPUTS
DESIGN OUTCOMES
Intermediate Lag
Ultimate Lag
Secondary Lead
Primary Lead
Business metrics (profile/sale)
Renewal rates
New users/subscriptions
Recommendations/referrals
Product-Market Fit
Business-Model Fit
Novelty and Innovation
Renewal rates
New users/subscriptions
Recommendations/referrals
Product-Market Fit
Business-Model Fit
Novelty and Innovation
Study-base metrics
System Usability Scale (SUS)
Customer Effort Score (CES)
User Experience Questionnaire (UEQ)
NASA Task Load Index (NASA)
Usefulness & Ease-of-use rating
Problem-Solution Fit
System Usability Scale (SUS)
Customer Effort Score (CES)
User Experience Questionnaire (UEQ)
NASA Task Load Index (NASA)
Usefulness & Ease-of-use rating
Problem-Solution Fit
Task-base metrics
Usability problems
Error rates
Completion rates
Reliability
No Problem-Solution Fit
Usability problems
Error rates
Completion rates
Reliability
No Problem-Solution Fit
Features (of lack thereof)
UI problems
Consistency problems
Poor brand atitude and appeal
No Market-Opportunity Fit
UI problems
Consistency problems
Poor brand atitude and appeal
No Market-Opportunity Fit
This schema wants to illustrate that finding and fixing UI problems in a usability test (Primary Lead) will improve completion rates on tasks (Secondary Lead), which in turn improves SUS scores (Intermediate Lag), leading to more product usage (Ultimate Lag).
Type of Methods
There are many ways to measure success as there are to measure experiences. However, it is the efficient triangulation of 3 types of UX metrics that can bring us a complete understanding of how the user behaves and perceives the quality of a product or service. Those 3 types are:
Issue-based Metrics
UX Performance Metrics
Self-reported UX Metrics
The first two generate behavioral data from how the user interacts with your digital solution. The last one generates the perceived data, meaning, how the user perceives the quality of your digital solution.
Issue-based and UX performance metrics will be a better instrument within this ocean of data if you connect them to how we arrived at a certain service quality that can be reported by self-report UX metrics.
For example, Issue-based metrics and performance metrics can measure the number of first-time signups and give us an idea about the effort required to reach out to new users, but they wouldn't give us any insight into the perceived quality of a new sign-up. Self-reported UX metrics are our final layer.
Although, performance and issue-based UX metrics, like task completion time or error rates, have been commonly discussed in our field. I feel that self-reported metrics haven't had the same spotlight. Its neglect can affect the effective triangulation of these 3 forms of UX metrics.
Issue-based Metrics
UX Performance Metrics
Self-reported UX Metrics
The first two generate behavioral data from how the user interacts with your digital solution. The last one generates the perceived data, meaning, how the user perceives the quality of your digital solution.
Issue-based and UX performance metrics will be a better instrument within this ocean of data if you connect them to how we arrived at a certain service quality that can be reported by self-report UX metrics.
For example, Issue-based metrics and performance metrics can measure the number of first-time signups and give us an idea about the effort required to reach out to new users, but they wouldn't give us any insight into the perceived quality of a new sign-up. Self-reported UX metrics are our final layer.
Although, performance and issue-based UX metrics, like task completion time or error rates, have been commonly discussed in our field. I feel that self-reported metrics haven't had the same spotlight. Its neglect can affect the effective triangulation of these 3 forms of UX metrics.
Issue-based Metrics
Track the frequency and severity of issues that users encounter while using a product or service.
Subjective & Attitudes Metrics
UX Performance Metrics
Self-reported UX Metrics
Metrics that rely on users' subjective feedback about their experience using a product or service. Provide valuable insights into users' perceptions and attitudes towards a product or service.
Behavioural & Performance Metrics
Provide insight into how users are interacting with a product or service and can help identify areas where the usability and ease of use can be improved.
Self-reported, UX performance and issue-based UX metrics are different types of UX metrics used to measure different aspects of the user experience.
Self-reported metrics are based on users' subjective feedback and opinions about their experience using a product or service. This includes metrics such as user satisfaction, ease of use, and perceived value.
UX performance metrics are based on users' objective performance using a product or service. This includes metrics such as task completion rates, time on task, and error rates.
Issue-based UX metrics are based on users' interactions with a product or service, and identify specific issues or problems that users encounter while using it. This includes metrics such as the number of errors or issues encountered, the severity of those issues, and the frequency with which they occur.
By measuring each of these different types of UX metrics, product teams can gain a more comprehensive understanding of the user experience and identify areas for improvement. They can then use this data to make data-driven decisions to optimize the user experience and ensure that it meets and exceeds user expectations.
Self-reported metrics are based on users' subjective feedback and opinions about their experience using a product or service. This includes metrics such as user satisfaction, ease of use, and perceived value.
UX performance metrics are based on users' objective performance using a product or service. This includes metrics such as task completion rates, time on task, and error rates.
Issue-based UX metrics are based on users' interactions with a product or service, and identify specific issues or problems that users encounter while using it. This includes metrics such as the number of errors or issues encountered, the severity of those issues, and the frequency with which they occur.
By measuring each of these different types of UX metrics, product teams can gain a more comprehensive understanding of the user experience and identify areas for improvement. They can then use this data to make data-driven decisions to optimize the user experience and ensure that it meets and exceeds user expectations.
