1-800-EarthLink Usability Study

Short study description: Usability study of one of EarthLink’s interactive voice response systems (IVR), 1-800-EarthLink. My first user research project, in 2006.

Primary roles: Project lead, project management, session facilitator

Summary of my contributions: Created a new heuristic for IVRs. Created recruitment survey. Analyzed qualitative and quantitative data with affinity diagramming and simple statistics. Facilitated (moderated) usability sessions. Responsible for the final report.

Lessons learned:

  • Collaborating with the client to articulate the goals for the system being tested is important. This informs study design and participant recruitment, as well as providing a rhetorical tool for arguing the importance of design recommendations .
  • How to use affinity diagramming to reduce and analyze data.
  • People don’t read long reports. Course assignments require final reports as a matter of exercise and evaluation; clients and others in the wild will read only the executive summary, if anything—communicating findings and recommendations in other ways (conversation, PowerPoint, email) is better.
  • User experience is multi-faceted , “three dimensional.” Our study design generating findings not just on information architecture, but emotional responses to audio and cognitive overload (for example).
  • Assigning severity ratings and scope to recommendations helps prioritize design changes.
  • How to create Likert scale questions where higher ratings correspond to positive evaluations and vice versa.

Files

  • A very long report (to satisfy course requirements), but very comprehensive. I think the Findings and Recommendations section, beginning on page 18, is a nice example of communicating findings and recs.
  • The heuristic I created for initially evaluating the IVR, designed to illuminate critical issues that could benefit from further study.