Through quick understanding, validation the full platform usability testing for new Firefox Lite 2.0 launch - The usability testing is tend to find out critical usability issues in our product, and be able to provide useful feedback to product development team for further iteration.  

In order to make product iteration model faster and agile, we planned 2 rounds test, the research team created different remote testing models by engaging with local partners and community members to have multi-channel to support on recruitment and provide testing environment rapidly. 

Age

18-25 yrd: 6  

26-35 yrd: 5  

36-45 yrd: 6  

User v.s

non user

User: 7  

Non user: 10  

Activity experience 

Shopping: 16  

Online news: 17

🇮🇳Intro from moderator without translation:

Moderator are our front facing person to interact with our users, since the testing required back and forth translation, this part can actually preceded by moderator before the testing starts. We learned if moderator kick off the introduction, it reduces a lot of anxiety from the users and save times for testing.

Test pilot at least two times with each moderator✌️

Test pilot not just for testing the prototype, also testing the way how researcher ask questions, how moderator interact, realized how to translate the question. it’s very important to get every stakeholders familiar the prototype transition, interview questions at least twice.

⏱Reduce tasks in 50 mins

The interview sometimes overtime, and it gets really challenge if we didn’t finished it. It would be better if we control proper volume of task and get more quality answers.

🤓See their face is better than see their hands

When we were doing testing, an extra camera was set up to documenting users’ hands, by the camera we get to see their hesitation while they are taping screens.

In the few last tests, we broke this protocol accidentally, and we found when the camera taking users’ face, they have more interaction and responses like they are actually talking to us than to moderator.

Though UT is purely testing users responses on products, it doesn’t remove our nature as an ethnographer, to learn more from human beings’ emotion, attitudes, and even expression while we are remotely connected.

👍 Fast for time saving

Fast is the most impressive image on Firefox Lite Preview. It related to time-saving, easy to access and shortcut, BUT not about the performance. 

👍 Light for space

User perceived non-installed service as a benefit to optimize their phone space. 

👎 Not Intuitive

On-boarding is introduced contradicting with user’s intuition and their goal while they are using the browser.

👎 Unclear Information Hierarchy

Unclear information hierarchy mislead the understanding of features

Research sharing

Lightning demo testing result:

The research team design a fun sharing session sharing with the Firefox Lite development team. In order to apply their empathy to learn the challenges from users, the research team demo users’ attitude and pain points while they are using new features.

UT Testing contributor