Qualitative Analysis
This analysis is part of the design process - 01 Learn & 05 Refine - of the Case Study Design for Learning - AI Research Problem.
#Survey #ThematicAnalysis #QualitativeData
#Survey #ThematicAnalysis #QualitativeData
Summary
The Case Study Design for Learning - AI research Problem is based on the problem space of How do we shift from a concept of tracking student's progress to start tracking student's proficiency?
To understand how users perception variate for progress and proficiency according to certain types of graphic representations. I conduct a survey study.
To understand how users perception variate for progress and proficiency according to certain types of graphic representations. I conduct a survey study.
01 Learn - My Approach & Survey Process
Contextual Inquiry
(empirical process)
(empirical process)
Users & Business Goals
Data Gathering
raw work activity data
organized & structure work activity data
Needs & Requirements
(deductive analytic process)
(deductive analytic process)
Extraction
Design Informing Models
(e.g., flow models, usage scenarios)
(e.g., flow models, usage scenarios)
Extraction
DESIGN
(integrative process)
(integrative process)
Contextual Analysis
(inductive analytic process)
(inductive analytic process)
Data Interpretation & Consolidation
Hartson, Rex, and Pardha Pyla. 2012. The UX Book. San Francisco, United States: Elsevier Science & Technology.
Contextual Inquiry / Data Gathering
Survey Process
1. I gathered 15 different graphic representations normally attribute to the concept of progress and/or proficiency.
2. I asked the users how they perceive the respective graphic individually.
3. I gather all the graphic representation on a grid and ask which of them they thought to better represent the notion of progress and the notion of proficiency
Goal
Test for Distinction Bias.
1. I gathered 15 different graphic representations normally attribute to the concept of progress and/or proficiency.
2. I asked the users how they perceive the respective graphic individually.
3. I gather all the graphic representation on a grid and ask which of them they thought to better represent the notion of progress and the notion of proficiency
Goal
Test for Distinction Bias.
Contextual Analysis / Data Interpretation & Consolidation
The most voted graphic representation for Progress
20,69% of users voted with a significant expression for this representation as to the most recognizable for tracking progress.
The most voted graphic representation for Proficiency
30,43% of users voted with a significant expression for this representation as to the most recognizable for tracking proficiency.
Needs & Requirements / Extration
The two concepts are already complex to distinguish, so upfront, I exclude all the graphic representation that didn't have a significant result. I didn't want to fall in the mistake of creating an ambiguous representation. Accordantly, I immediately excluded the following representation:
 |  |  |  |
Final Design Decision
Survey Study showed us that users associate proficiency more to starts representations and percentages and pie charts to the concept of progress. Accordantly, I followed the following graphic representation for the CPA exam review application. On the left for tracking proficiency and on the right for tracking progress.
05 Refine - My Approach & Survey Process
Thematic Analysis
To begging analysis the exploratory research, I conduct a qualitative analysis to uncover themes in the data.
I start it by doing a thematic analysis to explore similarities and relationships between the different chunks of the survey data. The goal was gather more visual means of analysis to present to the stakeholders in the final session review of the design process.
I start it by doing a thematic analysis to explore similarities and relationships between the different chunks of the survey data. The goal was gather more visual means of analysis to present to the stakeholders in the final session review of the design process.
Important Note: Visualization helps my team to understand better the similarities and relationships in the data more clearly to discloser better design and business decision.
Thematic Analysis Process
1. I like to print the results transcripts and start to highlighting important keywords and sections that are relevant to the study.
2. Next, I start cutting the paper texts sections that have been highlighted and start organizing them on an Affinity Diagram and making some side notes to them (descriptive labels).
1. I like to print the results transcripts and start to highlighting important keywords and sections that are relevant to the study.
2. Next, I start cutting the paper texts sections that have been highlighted and start organizing them on an Affinity Diagram and making some side notes to them (descriptive labels).
Important Note: Ideally this process shouldn't be done alone. But I am a team of one and my stakeholders are on the other side of the ocean. I had to remedy the situation and conduct the analysis myself and be subject to my own bias.
3. I organize the data into blocks of themes based on their relationships, and start to figure out some categories. At the end I found 4 major themes:
User Interface (UI)
Algorithm
Features
Behavior
Note: This was a time consuming and difficult task because I had to analyze the feedback from 300 students. It was an interesting challenge to reduce to 4 categories.
4. Once I found my 4 categories, I start thinking about how these all relate to each other.
5. Since we are a remote team, I used Miro platform to build a visualized representation of the categories relationships to present to my stakeholders.
6. With the Product Owners we explore the results from the 4 points of view - Behavior, Algorithm, User Interface and Feature. We analyse the positive and negative feedback and come up with solutions and strategies to improve the feature efficiency and experience.
5. Since we are a remote team, I used Miro platform to build a visualized representation of the categories relationships to present to my stakeholders.
6. With the Product Owners we explore the results from the 4 points of view - Behavior, Algorithm, User Interface and Feature. We analyse the positive and negative feedback and come up with solutions and strategies to improve the feature efficiency and experience.
Important Note: For privacy and competitor reasons, I am not allowed to share this dashboard real content.
The findings from this analysis where implement in the Design Review in the design process phase 05 Refine on the Case Study Design for Learning - AI Research Problem. Follow the case study below.
Becker CPA. Future of eLearning, bringing AI to eLearning
Introduction of Artificial intelligence engine to augment student studying skills to improve their learning process.
MENU
You cannot understand good design if you do not understand people. Design is made for people.
DIETER RAMS
FOLLOW ME
© All Rights Reserved. Jcerejo.
