A Practical Guide To Mixed Methodologies For UX Research
- 1. big data small data + = insight A PRACTICAL GUIDE TO MIXED METHODOLOGIES FOR UX RESEARCH Alina Magowska Head of UX Research and UX Analytics
- 2. 1999 DO WE NEED MORE DATA? 2001 Allegro.pl starts on Dec 13th. The software of the first version of the website was so small that it could fit on one floppy disk The millionth auction 2003 The millionth customer registered the account an Allegro 2021 20 million customers each month (Gemius 2020) #1 commerce platform in Poland #10 e-commerce websites worldwide
- 3. DO WE NEED MORE DATA?
- 4. YES! DO WE NEED MORE DATA?
- 5. FRAMEWORK SETUP PLAN MANAGE STEP CHALLENGE ACTION TO TAKE #1 #2 #3 What answer do you need and which methods to mix? Which scenario to choose? How to deal with data diversity? ● Specify challenges you face in the product dev process (strategy → delivery) ● Determine what knowledge do you need (explicite → latent) ● Mix the right method (small + big) ● Exploratory research ● Exploratory ● Dynamic research ● Collate insights from different sources ● Prioritise data ● Distribute knowledge
- 6. SETUP
- 7. PRODUCT DEVELOPMENT Strategize Explore D e s i g n D e v e l o p D e l i v e r Strategic research Exploratory research Define product brief + success metrics Generate ideas Prototype & evaluate ideas Plan Build & test Get feedback Product strategy Release WHERE ARE YOU IN THE PRODUCT DEVELOPMENT PROCESS? WHY SHOULD WE DO THIS? SMALL DATA: Field studies, Cultural probes BIG DATA: Prescriptive analytics Allegro - Product Development Process | EXAMPLES OF METHODS ARE WE DOING THE RIGHT THING? SMALL DATA: Interview, Customer workshop BIG DATA: Behavioral analytics WHAT SHOULD WE DO? SMALL DATA: Concept Testing BIG DATA: A/B testing, Descriptive statistics ARE WE DOING THE PRODUCT RIGHT? SMALL DATA: Usability Testing BIG DATA: Experiments IS IT STILL THE RIGHT THING? SMALL DATA: UX Metrics, NPS BIG DATA: Tracking & Evaluation
- 8. Inspiration: Context mapping: Experiences from practices (source) SURFACE DEEP KNOWLEDGE METHODS WHAT PEOPLE WHAT KNOWLEDGE DO YOU NEED? SAY THINK EXPLICIT OBSERVABLE TACIT LATENT DO USE KNOW FEEL DREAM SMALL DATA > BIG DATA SMALL DATA < BIG DATA SMALL DATA ? BIG DATA
- 9. WHICH METHOD TO CHOOSE? SURFACE (WHAT) DEEP (WHY) SMALL DATA BIG DATA Survey A/B Testing Tracking Statistical modeling Usability testing Diary studies Card sorting Personalization Interviews Generative sessions Simple algorithms NLP Neural Networks Reinforcement Learning
- 10. PLAN
- 11. Adaptation: The Three Research Designs - Mixed model by OpinionX.co EXPLANATORY RESEARCH insight big data small data insight small data big data small big insight EXPLORATORY RESEARCH DYNAMIC RESEARCH SCENARIOS
- 12. WHEN DATA SAYS ONE THING BUT USERS SAY ANOTHER Less than 1% use it EXPLANATORY RESEARCH
- 13. WHEN YOU WANT TO VERIFY USER INSIGHTS Results Selection from product catalogue - design lifting New search EXPLORATORY RESEARCH
- 14. WHEN YOU WANT TO GO BEYOND ML helps understand the reasons for contact, categorizes them and thanks to that we have quick feedback from customers Contact rate 1.7M reduction this year Buyers can find on Allegro multiple offers of the same product. We are working on making all the necessary information readily available, but sometimes they are not, and customers contact merchants. DYNAMIC RESEARCH
- 15. MANAGE
- 16. COLLATE USER INSIGHTS FROM DIFFERENT SOURCES UX/Design Research Market Research Customer Support Product Data Commerce Intelligence Data Science ML Research Market data Market data Market data in-house DATA TRANSPARENCY INSIGHT LEAD COLLABORATION & CONNECTION RESEARCH ONE-PAGER
- 17. PRIORITISE INSIGHTS IMPACT INSIGHT A B C EXPERIENCE How will this affect: - improving the experience? - making the experience outstanding? BUSINESS To what extent will this affect business metrics, eg GMV, active buyers? TOTAL IMPACT 4 4 5 TECH How much will it cost? (Who we need to engage, what technology do we need?) TOTAL SCORE (impact/tech cost) 4 2 1,6 TOTAL PRIORITY 1 3 2 Inspiration: The Umami Strategy by Aga Szóstek
- 18. PRIORITISE INSIGHTS IMPACT INSIGHT A B C EXPERIENCE How will this affect: - improving the experience? - making the experience outstanding? BUSINESS To what extent will this affect business metrics, eg GMV, active buyers? TOTAL IMPACT 4 4 5 TECH How much will it cost? (Who we need to engage, what technology do we need?) TOTAL SCORE (impact/tech cost) 4 2 1,6 TOTAL PRIORITY 1 3 2 Inspiration: The Umami Strategy by Aga Szóstek Impact Effort by organization high high low low Let’s go! Nope! Maybe? Maybe?
- 20. FRAMEWORK SETUP PLAN MANAGE STEP CHALLENGE ACTION TO TAKE #1 #2 #3 What answer do you need and which methods to mix? Which scenario to choose? How to deal with data diversity? ● Specify challenges you face in the product dev process (strategy → delivery) ● Determine what knowledge do you need (explicite → latent) ● Mix the right method (small + big) ● Exploratory research ● Exploratory ● Dynamic research ● Collate insights from different sources ● Prioritise data ● Distribute knowledge
- 21. Images from: Pexels, Allegro.eu FRAMEWORK: 1. SETUP 2. PLAN 3. MANAGE big data small data + = insight ALINA MAGOWSKA Let's stay in touch! Special thanks to Paweł Adamczak