Usability Research of Website Search Tool

Project Specifics

This study focused on the “Find a Doctor” tool on the UCare public website to analyze how individuals use the tool and understand how helpful the information it provides is. This was entirely focused on research where I led the recruitment, testing sessions and analysis of the collected information. I had three team members who were able to sit in on a few testing sessions, but this was a solo project.

Strategies used: Heuristic Analysis, Competitive Analysis, Usability Testing, Surveys, Journey Mapping

Why this study?

The research on this tool is completed every 3 years for the National Committee for Quality Assurance (NCQA) standards. The last study was completed in 2016, so a new study was necessary.

The five required NCQA standards include: font size; reading level (measuring in regards to 8th grade level); intuitiveness of content organization; ease of navigation; and directories in additional languages (if applicable to the company’s population). Each standard’s success was analyzed with post-session surveys.

Study Specifics

Why this study?

The research on this tool is completed every 3 years for the National Committee for Quality Assurance (NCQA) standards. The last study was completed in 2016, so a new study was necessary.

The five required NCQA standards include: font size; reading level (measuring in regards to 8th grade level); intuitiveness of content organization; ease of navigation; and directories in additional languages (if applicable to the company’s population). Each standard’s success was analyzed with post-session surveys.

Study Specifics

Recruitment and Study Preparation

Users of this tool are UCare members. Due to budget and necessary timeframe of this study, participants were 20 internal UCare employees. It was important that they had no prior experience using or connection to developing this tool. To prepare the usability study tasks, I analyzed  2016 research to understand its focus and also see how this yearís study could offer a realistic comparison to show progress over time.

The sessions included two overarching scenarios containing 10 set tasks within them. These sessions were each 20 min in length and sessions were spread across two days, due again to the quick need for analysis and report. 

Usability Testing and Analysis

Alongside the usability sessions I used post-session surveys ranking the five NCQA standards. These two methods helped to represent quantitative results, which was the most valuable type of results for the NCQA report and also qualitative observations for creating journey maps and personifying participant types.

The research did confirm meeting the NCQA acreditation standards, and internally it helped us focus on the changes that were priority with the next update. The main areas for tool improvement were the layout of information, and functionality of pieces like drop-down menus and the filtering feature. 

My presentation to the UCare executives brought to the forfront the general existence of this tool and the value of UX research. By presenting these findings, usability testing (in regards to this tool) is now understood as a necessary yearly study, rather than the 3-4 year cycle currently set for NCQA requirements.

Concluding Thoughts

The main barrier to testing this tool was the amount of information necessary for even its most basic use. For example, participants need to select a specific insurance plan type before beginning. I would have liked to provide less information to participants, but strictly-defined information was necessary for general use, and accurate comparison and analysis.

I believe the main value of this research, beyond confirmation of meeting NCQA standards, was the ability to humanize the use of this tool by identifying  three participant types. By personifying participants I showed executives the value of UX research and value in doing it on a regular annual schedule.

Journey map for main participant type created from usability study of search tool
Scroll UpScroll Up