The Measuring Experience Field Guide
INTRODUCTION
In today's digital age, UX metrics have become critical to determine the success of a product. However, it can be overwhelming to determine which metrics to use and how to use them effectively. The Measuring Experience Field Guide provides you with practical insights and step-by-step instructions for measuring and optimizing UX using various metrics and tools. It serves as a comprehensive resource for product owners, designers, and anyone involved in creating exceptional digital experiences.
What is an Experience?
In general, an experience is simply how people feel when they engage with a product or service in every instance of that human-service interaction. On the other hand, user experience (UX) is a term that specifically refers to the experience of users when engaging with technology.
A person's experience perception can happen in 4 different moments:
A person's experience perception can happen in 4 different moments:
1
Before usage (Anticipated UX)
The user has certain prior experience and expectations when he wants to use the product.
2
During usage (Momentary UX)
During use, pragmatic qualities and usability are perceived.
3
After usage (Episodic UX)
After use, the user reflects on the interaction that has taken place. The evaluation retakes spots directly after use and at a time interval.
4
Over time (Cumulative UX)
The overall evaluation of the product by the user is the sum of the individual experiences that have taken place over a more extended period.
Before usage (Anticipated UX)
The user has certain prior experience and expectations when he wants to use the product.
2
During usage (Momentary UX)
During use, pragmatic qualities and usability are perceived.
3
After usage (Episodic UX)
After use, the user reflects on the interaction that has taken place. The evaluation retakes spots directly after use and at a time interval.
4
Over time (Cumulative UX)
The overall evaluation of the product by the user is the sum of the individual experiences that have taken place over a more extended period.
What is User Experience?
Nowadays, User Experience (UX) is considered a crucial element for the successful development of any interactive system. Its goal is to create products that are easy and enjoyable to use, and that meet the needs and expectations of the users.
ISO 9241-210 (2010) defines UX as covering
"all the perceptions and reactions of a user before, during, and after interacting with a product or service."
While the traditional perspective on human-computer interaction focuses heavily on instrumental factors such as usability and usefulness, User Experience Design strives to create positive user experiences and emotions by fulfilling psychological needs.
To achieve these objectives, UX must be evaluated during the early stages of User-Centered Design (UCD) and monitored throughout the entire development process. Besides qualitative methods that offer detailed information on individual preferences, values, judgments, and significance, quantitative methods are also essential to provide standardized comparisons and verify specific requirements, such as benchmarks, concerning UX.
ISO 9241-210 (2010) defines UX as covering
"all the perceptions and reactions of a user before, during, and after interacting with a product or service."
While the traditional perspective on human-computer interaction focuses heavily on instrumental factors such as usability and usefulness, User Experience Design strives to create positive user experiences and emotions by fulfilling psychological needs.
To achieve these objectives, UX must be evaluated during the early stages of User-Centered Design (UCD) and monitored throughout the entire development process. Besides qualitative methods that offer detailed information on individual preferences, values, judgments, and significance, quantitative methods are also essential to provide standardized comparisons and verify specific requirements, such as benchmarks, concerning UX.
Effectiveness + Efficiency + Satisfaction = Usability Score
What is the anatomy of an experience?
Usability is just a single, one-dimensional property. An experience is rather a combination of factors.
An experience is a combination of hedonic & pragmatic properties
Pragmatic properties
Measures interactions and the perceived quality resulting from function derived from the digital product.
Measures interactions and the perceived quality resulting from function derived from the digital product.
Hedonic properties
Measure the perceived beauty and utility resulting from sensations derived from the experience of using a digital product.
Measure the perceived beauty and utility resulting from sensations derived from the experience of using a digital product.
“
The properties of these two factors are what measure the quality of an experience.
Pragmatic metrics are responsible to measure the effectiveness of the digital product by quantifying the accuracy and completeness with which users achieve specified goals and tasks. As well as the efficiency of a digital product by quantifying the resources expended in relation to the accuracy and completeness with which users achieve goals and tasks. Hedonic metrics are responsible for measuring the usefulness and acceptability of the use of a digital product.
Why measure experience?
Measuring the user experience is seeing whether design efforts actually make a quantifiable difference over time. There are 3 reasons why you should measure an experience:
1
Obtaining a quantitative usability benchmark allows us to understand how to design and functional changes impacted the user experience.
2
User Experience benchmarking is an effective method for understanding how people use (pragmatic) and think (hedonic) about a digital solution.
3
UX Benchmark studies are most effective when conducted at regular intervals (e.g. annually or quarterly) when compared against competitors, earlier versions, or an industry standard.
Obtaining a quantitative usability benchmark allows us to understand how to design and functional changes impacted the user experience.
2
User Experience benchmarking is an effective method for understanding how people use (pragmatic) and think (hedonic) about a digital solution.
3
UX Benchmark studies are most effective when conducted at regular intervals (e.g. annually or quarterly) when compared against competitors, earlier versions, or an industry standard.
What are user experience metrics?
UX metrics are a set of measurements that are used to evaluate the usability, effectiveness, and user satisfaction of a product or service. These metrics can provide insights into how users interact with a product or service and help identify areas for improvement.
1
Behavioral Metrics
These metrics are related to how users interact with a product, such as a task completion rate, time on task, click-through rate, and drop-off rate.
2
Attitudinal Metrics
These metrics measure users' attitudes, emotions, and perceptions towards a product, such as System Usability Scale (SUS) and Customer Satisfaction Score (CSAT).
3
Descriptive Metrics
These metrics describe the characteristics of a user group, such as demographic information, usage frequency, and user retention.
4
Comparative Metrics
These metrics compare the performance of a product to that of a competitor, such as benchmark studies or competitive analysis.
Behavioral Metrics
These metrics are related to how users interact with a product, such as a task completion rate, time on task, click-through rate, and drop-off rate.
2
Attitudinal Metrics
These metrics measure users' attitudes, emotions, and perceptions towards a product, such as System Usability Scale (SUS) and Customer Satisfaction Score (CSAT).
3
Descriptive Metrics
These metrics describe the characteristics of a user group, such as demographic information, usage frequency, and user retention.
4
Comparative Metrics
These metrics compare the performance of a product to that of a competitor, such as benchmark studies or competitive analysis.
WHEN WE SPEAK OF MEASURING
Measure, measurement, and metrics are related terms that are often used interchangeably, but they have different meanings and purpuses.
MEASURE
A measure is a unit or standard of measurement used to quantify or assess something for which we want to collect measurements. It answers What are we measuring?
MEASUREMENT
A measurement is a process of obtaining a value or quantity of a measure. A single number or value that shows us something about a subject in a unit of measure. It answers How are we measuring?
METRIC
A metric is a specific measure used to evaluate or assess something. In the context of user experience, it's used to assess or indicate the degree of "goodness". A metric might be a task completion rate or a customer satisfaction score. It answers What could we conclude?
IN OTHER WORDS
A measure is a standard or unit of measurement, a measurement is a process of obtaining a value or quantity of a measure, and a metric is a specific measure used to evaluate or assess something.
