Pitt Research Usability Evaluation
Abstract
This report evaluates the usability of research.pitt.edu, and provides insights into potential improvements that would thereby enhance the usability of the system for users. Usability testing ensures user interface designs are clear and intuitive, while also considering accessibility for individuals with diverse abilities. Enhancing these aspects of the digital interface can expand the reach of Pitt Research and ensure the faculty, staff, and students of the University of Pittsburgh are provided the resources in order to conduct groundbreaking research. Developers can use the findings in this report to implement both immediate and long-term improvements for future iterations of the platform.
Introduction
This report evaluates the usability of research.pitt.edu, and provides insights into potential improvements that would thereby enhance the usability of the system for users. Usability testing ensures user interface designs are clear and intuitive, while also considering accessibility for individuals with diverse abilities. Enhancing these aspects of the digital interface can expand the reach of Pitt Research and ensure the faculty, staff, and students of the University of Pittsburgh are provided the resources in order to conduct groundbreaking research. Developers can use the findings in this report to implement both immediate and long-term improvements for future iterations of the platform.
The website, research.pitt.edu, serves as the central hub for research and innovation at the University of Pittsburgh.The site aims to provide resources for faculty, staff, students, and the general public to learn about what the university has to offer in terms of research. Given that Pitt is ranked #6 for NIH funding, #17 in R&D expenditures, and #14 in U.S Utility Patents, effectively communicating their research involvement is a top concern.
In this study, our interdisciplinary research team hopes to evaluate the usability and accessibility of research.pitt.edu and offer recommendations to improve the website. Usability testing, evaluates how easy and effective a product is for users. The 5 main pillars of usability testing are known as the 5Es, developed by Whitney Quesenbery. Effective, how completely and accurately the work or experience is completed or goals reached. Efficient, How quickly this work can be completed. Engaging How well the interface draws the user into the interaction and how pleasant and satisfying it is to use. Error tolerant How well the product prevents error and can help the user recover from mistakes that do occur. Easy to learn How well the product supports both the initial orientation and continued learning throughout the complete lifetime of use. If you are ever confused upon a methodology, result, or recommendation from a usability test make sure to refer back to the 5 E’s of usability for guidance.
Figure 1. Summary of Whitney Quesenbery’s 5E’s of Usability
Purpose of Research
Pitt Research recently launched an updated site; now is the perfect time to provide feedback. This user research will supply data on the effectiveness of the updated research.pitt.edu site. During the initial meeting with the primary client, Ms. Shannon Durkin, Senior Director of Communications for Pitt Research, the team was able to construct and identify four main goals for this usability study. The first goal identified was to evaluate and better understand the issues the primary users, research faculty and potential faculty, may face when utilizing and navigating the site. The second goal was to improve the clarity of the UI and tab bar, specifically in the resources section. The third goal was improving the navigation consistency of research.pitt.edu. Finally, the last goal was to improve the user experience and visual appeal of research.pitt.edu.
Heuristic Methodology
A heuristic is a general rule of thumb, strategy, or principle that helps people solve problems or make decisions quickly and efficiently. In the context of usability testing, heuristics refers to widely accepted design principles or best practices that help identify usability issues in a system, interface, or product. In this study we will use one of the most well-known sets of heuristics that comes from Jakob Nielsen’s 10 Usability Heuristics for User Interface Design. While not formalized in a “top 10” list like Nielsen’s, accessibility heuristics focus on principles that ensure interfaces are usable by people with a wide range of abilities, including those with visual, auditory, cognitive, or motor impairments. In this study we will also use the accessibility heuristic developed by A. Pascarella and Dr. Stephen Quigley at the University of Pittsburgh.
In addition to the heuristics we also conducted a Siteimprove study. Siteimprove is a cloud-based Software-as-a-Service (SaaS) platform that helps organizations optimize their websites by enhancing accessibility, content quality, SEO performance, and overall digital experience. Siteimprove operates upon the similar principles and guidelines as the above usability and accessibility heuristics. By using siteimprove we hope to catch issues and problems that traditional heuristic test may overlook.
Heuristic Findings
Heuristic testing revealed a range of strengths and weaknesses tied to the Pitt Research website such as:
Positive:
-
Major navigation features visible and available at top and bottom of pages
-
Few opportunities for user error
-
Flexible user interaction
Moderate:
-
Images feel awkward and mostly unrelated to research
-
Language and design could better reflect research
Negative:
-
Search functionality lacks instant feedback
-
Homepage isn't clear or organized
-
Inconsistent logo behavior (not always clickable or present)
-
Navigation buttons often take you to other Pitt websites with no warning
Empathy Mapping
Our empathy map hypothesizes user responses during testing. It highlights emotions, experiences, and biases our users might experience during the testing session.
Figure 2. Empathy map developed to gain insights for potential users of the interface
Empirical Research Methodology
Our team recruited a diverse group of users across the university with roles including, undergraduate and graduate students, research assistants and associates, and research faculty. Once we gathered our users, we conducted our interviews in person and over Zoom. The research team recorded a transcript used for data analysis and triangulation. A testing script was developed in order to perform effective and consistent evaluations amongst the participants. After gathering first impressions, we prompted users for behavioral and reflexive insights by asking questions such as “If you landed on this site, what would you do first? (Behavioral) and “Who do you think this website is for? What is its purpose? (Reflective). Next, the users were given two Scenarios to complete. Scenario A was, “Please find out how much money Pitt spent on research last year and how many businesses they have started”. Scenario B requested information regarding, “who is eligible for Pitt’s proposal editing service”.
Figure 3. User Testing Process Funnel
User Research Findings
Purpose of Website and Pitt Research
Figure 4. research.pitt.edu main screen
The main issue users had with research.pitt.edu was a general lack of clarity leading to difficulty finding things and not knowing the purpose of the site. When asked about first impressions of the site, one user answered that: “I had no idea what I was about to do on this” and “it's not very informative”. The user also did not like the photos (Fig 4.) commenting: “I feel like the image does not implicate what they do at Pitt Research.” Users also mentioned that an about page explaining what Pitt Research does and the purpose of the website would be helpful.
Navigation Structure
Users found navigating the website confusing and frustrating. While looking for the research expenditure and businesses started, one user went to the annual report. The annual report is a completely different website, and the user was not able to return to research.pitt.edu without using the browser. For both of our scenarios, users had to navigate through multiple pages to find what they were looking for. Users got lost, frustrated, and did not want to engage with the site. One user did not utilize the site’s search bar at all. They did not see the search bar because the zoom square was blocking it from view. The user stated that: “Maybe it would be more efficient (in line with the tabs) because you're likely to read left to right”.
Clarity of Terminology
The terminology used for labels confused many users. Users thought of hotels and not services when they encountered the Pitt Research Concierge tab. Many felt misled by the label and suggested renaming the heading Research Resources and Services. Additionally, most users did not immediately understand what they would find in the By the Numbers section.
Aesthetics and Web Standards
Users praised the color scheme, font and text size, and the big visual tiles used to create research.pitt.edu’s clean, modern layout. Users also appreciated that research.pitt.edu follows the design standards of other Pitt websites.
Layout of Information Pages
Figure 5. research.pitt.edu By the Numbers
Figure 6. research.pitt.edu Annual Report Contents
Users who explored the annual report noted that the table of contents was not helpful because it needed more information. The Pitt Research Concierge, By the Numbers, and Proposal Editing Service pages were seen as direct and effective.
Overall, users found research.pitt.edu aesthetically pleasing but confusing and difficult to navigate. Labels are not always clear, making it hard to find content, and users had to click through multiple pages to complete tasks. Pictures do not not match user expectations of Pitt Research, and users did not believe the site clearly articulated its function.
Overall
Recommendations
-
Use clear and descriptive terminology for labels to help users find what they need without confusion.
-
Integrate all resources into a single website to prevent users from getting lost or stuck on external pages.
High Priority Issues
-
Simplify the menu structure and relocate the search bar to a more prominent position to improve navigation efficiency.
-
Add an "About" section to help users understand the mission and purpose of Pitt Research.
Medium Priority Issues
Low Priority Issues
-
Include informative images to enhance user understanding of Pitt Research's work.
-
Display the user's navigation path at all times to prevent disorientation.
-
Hide the hamburger menu when tabs are visible to eliminate redundancy.
Conclusion
This usability study of research.pitt.edu reveals that while the website boasts a visually appealing and modern design, it falls short in several key areas of usability and accessibility. Users struggled with unclear navigation paths, confusing terminology, and a general lack of clarity about the site’s purpose. These issues hinder users from efficiently accessing important research resources and information, ultimately detracting from the site's mission to support and promote research at the University of Pittsburgh. By implementing the high-, medium-, and low-priority recommendations outlined in this report—such as clarifying labels, consolidating site structure, improving navigation, and adding contextual content—Pitt Research can significantly enhance the user experience. These improvements will not only make the platform more accessible and intuitive for a wide range of users, but also help ensure that Pitt's research initiatives are communicated more effectively, both internally and to the broader public.