| Becker CPA: Revolutionizing Exam Prep with Adaptive AI Redefining CPA study, this project developed a personalized, ML-driven learning experience that empowers students to achieve true proficiency. This approach dramatically improves study effectiveness, efficiency, and overall exam performance. |
Company
Becker Professional Education. A global professional education company that has been the market leader in accounting exam preparation for the past 60 years.
Becker Professional Education. A global professional education company that has been the market leader in accounting exam preparation for the past 60 years.
Product
A new feature, Personalized Review Sessions (PRS), for the CPA Exam Review.
A new feature, Personalized Review Sessions (PRS), for the CPA Exam Review.
Deliverable
New feature for the existing e-learning platform.
Timeline
8 months
New feature for the existing e-learning platform.
Timeline
8 months
Role
Lead Product Designer
Team
Solution Architect, 2 Product Managers, 3 Developers, Lead Software Developers, 2 Front-end Developers, 4 Quality Assurance, Lead Product Designer, Product Designer
Lead Product Designer
Team
Solution Architect, 2 Product Managers, 3 Developers, Lead Software Developers, 2 Front-end Developers, 4 Quality Assurance, Lead Product Designer, Product Designer
My Role
I led the user-centered design efforts for the new feature, ensuring alignment between student needs, business goals, and technical capabilities.
Led end-to-end UX design, setting overall design direction and defining design sprints.
Planned and executed user testing (usability, A/B tests) and iterated on designs based on feedback for final delivery.
Designed and prototyped key features, including personalized dashboards and a dynamic badge system.
Collaborated closely with an AI team and cross-functional partners to integrate algorithmic intelligence into the UI.
Conducted extensive user research and data analysis to drive design decisions and insights.
Impacts
-
88% of the student-user Beta group felt that the new functionality improved their efficiency in studying for the exam. -
92% of the students felt that they studied more effectively with the new functionality, demonstrating the feature's ability to enhance learning outcomes. -
Increased average exam scores by +13.09 points and boosted retention by 30%, achieved by the personalized learning experience. -
Students also felt empowered by a newfound awareness of their knowledge gaps at a granular level.
| Overview Becker Professional Education, a long-standing market leader in CPA exam preparation, aimed to transform its "fit-to-all" learning approach into a personalized and adaptive experience. This project involved designing a new ML-powered feature, Personalized Review Sessions (PRS), to meet individual student needs more effectively and leverage cutting-edge technology to enhance learning outcomes. |
THE CHALLENGE
In 2017, Becker faced significant market share decline. The core challenge was to provide a more efficient, personalized, and engaging course experience that transcended traditional linear learning paths. Specifically, the design needed to help students understand and track their true proficiency in complex accounting topics, rather than just their progress through the material, which was a critical differentiator in a competitive market.
PAIN POINTS
// Difficulty in Self-Identification: Students found it challenging to identify their specific weaknesses and areas needing more focus.
// Inadequate Feedback: Students experienced a poor onboarding process and lacked clear, personalized feedback on their performance and knowledge gaps.
// Lack of Personalization: The existing "fit-to-all" learning path failed to adapt to individual student needs and learning styles.
// Progress vs. Proficiency Confusion: Students struggled to accurately differentiate their progress through the material from their actual proficiency in specific concepts.
CONSTRAINS
// External Application UI: The feature had to run in an external application provided by AICPA. With a user interface, we cannot control or make changes.
// Remote User Research: User research was strictly limited to remote access, impacting traditional observation methods.
// Outsourced AI Development: The AI algorithm development was managed by a third-party company, SanaLabs; requiring seamless collaboration for UI integration.
AICPA User Interface
ORG GOALS
- Increase course completion rates.
- Improve student engagement and satisfaction.
- Regain market share and strengthen brand positioning as a leader in personalized e-learning.
- Deliver a new feature that provides a more efficient and personalized course experience.
- Understand their true knowledge gaps at a granular level.
- Study more efficiently by focusing on weaker areas.
- Feel more in control and empowered by their learning path.
- Receive clear, actionable, and personalized feedback on their performance.
USER GOALS
DESIGN GOALS
- Build a new feature that proactively identifies knowledge gaps specific to each learner.
- Trace these gaps back to their root cause.
- Tailor study programs to maximize user study time and address pain points.
- Create a UI that clearly communicates performance, progress.
Design Process Summary01 Learn. Understand user needs, differentiate progress from proficiency, and identify business challenges through surveys, interviews, and stakeholder workshops. 02 Explore. Map existing user flows, define new personalized learning paths, build user profiles, and outline navigation models for the new feature. 03 Select. Evaluate and prioritize design concepts and wireframes, choosing solutions that best address user needs and system constraints. 04 Develop. Create detailed design specifications and prototypes, evolving selected concepts into a full design solution, including dashboards and feedback mechanisms. 05 Refine. Conduct usability testing and A/B tests with a large user group, gather feedback, and iterate on designs to optimize user control, clarity, and effectiveness. 06 Deliver. Finalize designs, produce comprehensive mockups, and prepare assets for development hand-off. |
|
Key Problems
User Problem
How do users differentiate progress from proficiency? How can they effectively target their weaknesses?
Business Problem
How can we boost course completion rates, enhance student engagement, and raise their average exam scores to establish ourselves as the top exam preparation platform and regain market share?
User Problem
Conducted an extensive Survey and Thematic Analysis on a sample of 300 students to understand their perception of progress vs. proficiency based on different graphic representations.
The most voted graphic representation for Progress
This graphic representation was the most voted (21%) by the users as the most recognizable to track progress.
The most voted graphic representation for Proficiency
This graphic representation was the most voted (31%) by the users as the most recognizable to track proficiency.
Business Problem
The affinity diagram exercise from the workshop identified multiple issues, which we organized into four groups.
Feedback & Support Issues
- Lack of useful feedback
- Limited social and support interactions
Onboarding & Guidance Issues
- Poor onboarding
- Insufficient AICPA guidance orientation
Content & Engagement Issues
- Difficulties with assignments
- Lack of utility and engagement
Study Management Issues
- Trying to focus on multiple exams simultaneously
Feedback & Support Issues
- Lack of useful feedback
- Limited social and support interactions
Onboarding & Guidance Issues
- Poor onboarding
- Insufficient AICPA guidance orientation
|
Working Professional
Characteristics: Full-time job, limited and unpredictable study hours, seeks maximum efficiency, values targeted content over broad review.
Needs: Quick, personalized review sessions focused on weak areas to optimize scarce study time; clear progress tracking that fits into short bursts of study.
Characteristics: Has attempted the CPA exam before (or specific sections), knows general weak areas but needs precise identification of granular knowledge gaps for improvement.
Needs: Deep dive into specific topics, error prediction that identifies root causes of mistakes, highly adaptive remediation learning paths.
Exam Retaker
New Graduate
Characteristics: Recent college graduate, good theoretical foundation but may lack experience, needs a structured yet flexible approach to cover all material while efficiently identifying and addressing foundational gaps.
Needs: Clear guidance on overall proficiency, effective self-assessment tools, and a system that can adapt as their knowledge evolves across broad topics.
User Groups
|
Proficiency Vizualization
To not conflict with existing progress tracking, a star representation system was chosen for proficiency. Initially explored a 5-star system, but simplified it to a 3-star system with half-star achievements to avoid UI clutter.
Becker's Content
CPA course has the following content structure
Badge System
Explored and developed a visually intuitive badge system to communicate student proficiency levels.
Problem: Initial testing showed students needed more immediate feedback. The design was refined to include an "experience bar" within the badge: it increased with correct answers and decreased (turning red) with wrong answers. Filling the bar earned a half-star. This system ensured students understood that proficiency could be lost if previously mastered topics were answered incorrectly, as the focus was on current knowledge, not just progress.
Final badge system approach
|
How does it work?
Until the student finishes a Unit learning path, the algorithm will be tracking their performance and knowledge.
Reaching the end of the path, the student can launch a study session and work on their weakness.
The PRS came with a Proficiency Dashboard where they can see an overall picture of their proficiency by the Module knowledge level and by the overall Unit knowledge level. If the student didn't study any content for a Module, the badge will appear greyed down, since the algorithm didn't collect any information yet.
Until the student finishes a Unit learning path, the algorithm will be tracking their performance and knowledge.
Reaching the end of the path, the student can launch a study session and work on their weakness.
The PRS came with a Proficiency Dashboard where they can see an overall picture of their proficiency by the Module knowledge level and by the overall Unit knowledge level. If the student didn't study any content for a Module, the badge will appear greyed down, since the algorithm didn't collect any information yet.
Important Note: We prepared onboarding resources for students. A PDF guide and a video tutorial on how the feature works.
AICPA GUI Adaptation
The UI tool was adapted to the AICPA interface, tracking proficiency changes with each question answered. Questions included CPA Exam skill levels and explanations for why they were served by the AI. Students could watch lectures before answering or review detailed explanations afterward.
AICPA GUI
We adapted the UI tool to the following design.
After completing the above review session, students can access a detailed dashboard to review module performance, view in-depth explanations for correct and incorrect answers, and start additional sessions to further build proficiency.
A/B Test
We conducted a Beta A/B test with approximately 3000 students. The AI algorithm required learning from individual users as well as from overall usage patterns. To allow the algorithm adequate testing time, we ran the A/B test for three months and then surveyed the students at the end.
Test Findings
Based on the following findings, modifications were made, such as creating feedback mechanisms that clarified the impact of actions (like skipping) without negatively affecting perceived proficiency. The final designs incorporated these learnings for improved user control and clarity.
// Problem 3 (Skipping Fear)
Students were afraid that skipping exercises (especially time-consuming calculation-based ones) would negatively affect their performance.
// Problem 3 (Skipping Fear)
Students were afraid that skipping exercises (especially time-consuming calculation-based ones) would negatively affect their performance.
Problem 1
Control
Control
Students felt the feature was "controlling them," lacking transparency on why certain actions were recommended.
Solution: Enhanced transparency by integrating contextual information directly into the question header, explaining the question type and the AI's rationale for presenting it. This included a tooltip providing clear definitions based on AICPA parameters.
Solution: Enhanced transparency by integrating contextual information directly into the question header, explaining the question type and the AI's rationale for presenting it. This included a tooltip providing clear definitions based on AICPA parameters.
Problem 2
Badges
Badges
The initial badge system didn't account for how many tries it took students to get an exercise right.
Solution: The badge system was enhanced with a dynamic 'experience bar' that visually tracked performance on each attempt. For every correct answer, the bar advanced, while incorrect answers caused it to decrease. We also incorporate a label for the number of attempts, along with the collective percentage of correct answers on the first try, and the overall percentage of correct answers.
Solution: The badge system was enhanced with a dynamic 'experience bar' that visually tracked performance on each attempt. For every correct answer, the bar advanced, while incorrect answers caused it to decrease. We also incorporate a label for the number of attempts, along with the collective percentage of correct answers on the first try, and the overall percentage of correct answers.
Problem 3
Skipping Fear
Skipping Fear
Students were afraid that skipping exercises (especially time-consuming calculation-based ones) would negatively affect their performance.
Solution: We include both suggestions, a Skip Question button and a Mark Question.
Solution: We include both suggestions, a Skip Question button and a Mark Question.
Final version after feedback
|
Final Designs
