Usability Testing

With my product, I wanted to make sure to do a usability test after the mid-fidelity prototype so that I could catch any flaws in my design before I designed and executed my high-fidelity prototype, and to also uncover any opportunities for change or innovation. With the usability testing, I had the same 6 participants from the User Research interviews be my participants for this testing. Participants B & C were the unmoderated, off-site participants. Participants A, D, E & F were all moderated, on-site participants. While they were using the prototype, I did encourage them to think out loud. For the unmoderated participants I encouraged them to take notes and write down any and all questions that they had while they were using my product.

I conducted a usability study and used the System Usability Scale (SUS) survey after the study. The study took place over the span of a week to give all participants and myself enough time to finish the study. There were 2 unmoderated & 4 moderated studies, all timed from beginning to task completion. The participants’ ages ranged from 19 to 33 years old. Participants were either already pet owners, are thinking about getting a pet or have thought about getting a pet, and they all have navigated pet adoption websites before. 

For the moderated studies, I provided a list of prompts, a laptop for usage & we met at a neutral location like the library or a Starbucks. I advised all participants to ask questions out loud, think out loud so that I could hear & gather data while they were using the prototype. For the unmoderated participants, I sent them a PDF which included links to the prototype and survey, the list of prompts and advised them to time themself & also write down any questions or feedback along the way. 

Access the document with prompts here & access the System Usability Survey (SUS) questions and participants’ answers here

While conducting the Usability testing & SUS survey, I had 3 KPIs I was actively gathering data for:

1. Time to complete task to determine if my user flow was as intuitive as I had planned. I put time parameters on what is satisfactory & what isn’t. For a mid-fidelity prototype anywhere from 1 minute to 1.5 minutes is satisfactory. Anything longer is viewed as too long to complete the task and have to address to see where the hold up occurred. 50% of participants fell under the satisfactory category, which led me to wonder where there could be opportunity for change. Is it in the user flow itself? Or was there something wrong with my design that made the user flow not as intuitive? When observing the participants, I noticed that my original idea of having profiles highlight when the mouse is hovering over it, was not executing correctly on the prototype. Although that is a nice aesthetic feature, its not something that is important for the user so I did end up taking out that idea for the high-fidelity prototype.

2. Along with time to complete task, to determine if my product is usable, I’m also asking the question, is the task getting completed? From the usability test, 83% of participants were able to complete the task of “Submit a pre-approval form for Harley the cat”. The other 17% pf participants were not able to submit a form, and I definitely had to get to the bottom of why? When observing Participant F, I noticed that he kept all settings of the prototype as it was presented, in this instance not zooming out. So each time he went to hover over Harley’s profile, it would highlight but then the page would also do this weird jumping back and forth of the scroll resetting to the top of the page. He wasn’t able to get past that hurdle and hence why he was not able to complete the task. With getting rid of the highlight feature, this should also alleviate the issue of the page’s scroll resetting while hovering over profiles. 

3. Last but not least, in the process of testing to find opportunities for change or innovation, I had all participants answer a SUS survey to test the usability of my mid-fi prototype. For the SUS survey, it is an industry used survey consisting of 10 questions that participants will answer on a scale of 1 to 5. 1 being strongly disagree and 5 being strongly agree, After participants are done taking the survey, I compile their answers and do some manual calculations to find their SUS score. A SUS score of >68 is above average aka prototype usable and a score of <68 is below average aka prototype not usable. 50% of participants had a SUS score of above 68 and 50% of participants had a score of below 68. When evaluating the answers to the questionnaire, I’m seeing a theme that the website could be simpler, which I'm translating that to meaning that the design could be simpler which would lead users to to be able to finish the task and do it quicker.

   Access the document showing the SUS questionnaire answers & the scores calculated out here

From gathering data for my KPIs, I was able to determine that there were indeed opportunities to make my product better for users. I took away the highlight feature when hovering over photos, and I added names to all the profiles on the Adoptables page, with those 2 changes it should improve the Time to Complete Task to all favorable times and also change Task Completed to 100%. I also saw an opportunity to enhance the users experience by adding a sorting menu so that users could further cater their experience to the pet they are trying to adopt.