Usability and User Interface Design

As noted in previous posts, I am currently a part-time student in the MS-PTC program at NJIT. The MS-PTC community and NJIT Web services team were instrumental in the re-design of the NJIT Web site (www.njit.edu). A consultant for the re-design process submitted a proposal for using high school students to participate in actual usability testing of the NJIT Web site. The consultants submitted a NJIT Usability Testing Protocol Planning document and a Usability Testing Task Analysis document as part of their proposal.

This blog post will critique the consultant’s proposal and investigate what types of changes would be required if the usability test was conducted on a commercial Web site instead of an academic Web site.

The NJIT Usability Testing Protocol Planning document is a script for usability test coordinators to follow when test participants arrive for the formal usability test session. The protocol planning document instructs test coordinators to collect demographic data, explain the goals of the test and ask a few general interview questions before launching the actual usability test. The protocol planning document is thorough enough, but I would make some minor modifications.

Most importantly, the demographic data section asks test participants for name, address, contact information and social security number. This type of information is extremely confidential and many people are reluctant to provide it to researchers. I would include a line or two in the script to inform test participants that their confidential information will not be made available to outside parties.

I would also ask test participants to engage in ‘protocol analysis’ throughout the usability test. Protocol analysis, also knows as ‘think aloud protocol’ is a research method that elicits verbal reports from test participants. In other words, test participants should verbalize their thoughts in order to convey any successes/frustrations they may encounter when performing usability test tasks. The test coordinators can take thorough notes or revisit audio recordings to highlight any common problem areas.

The Usability Testing Task Analysis document provides some task-based questions for the actual usability test. Before gathering section-specific information on the NJIT Homepage, I would ask some ‘first impression’ questions, such as is the layout pleasing? Are the links and menus logically placed? Several questions hint at this (e.g. Get feedback on the general mix of content on the page or on the use of colors and visual “balance”), but I think these questions could be expanded to specific design elements (typography/readability, menu placement etc.) and should be moved to the front of the list. After all, the Web Credibility Research Project (www.webcredibility.org) shows that 46% of people base credibility judgments on overall visual design of a site, including layout, typography, font size and color schemes. I don’t think the NJIT site suffers from a lack of credibility, but I think the usability test team misses out on getting first impression-type feedback from users early on in the usability test.

I think the Tasks section is well-written, as the majority of tasks for potential students are covered. I would remove the question about ‘would they fit in at NJIT?’, as I don’t think this is relevant to the usability of the Web site. I would also add a question about finding information on admission requirements (e.g. for students who are questioning whether or not SAT scores are required). Perhaps a question or two about using the search box for known item searching would help. Most prospective students are searching for specific information and many will turn to the search box. The usability test team has a vested interest in ensuring that the search engine returns useful results.

If the Test Protocol Planning document and the Usability Testing Task Analysis document were written for usability testing of a commercial Web site, I would include much more information about navigation and the purchasing/payment processes. If the goal of commercial sites is to maximize efficiency of use (e.g. ensure that users can find and purchase the products that a Web site is selling as quickly and effortlessly as possible), then the goals of usability testing be focused accordingly. I may ask test participants to search for information about a specific-product, including the cost. I may also ask users if they’re comfortable conducting financial transactions through the secure portion of the site. I would also ask about product categorization. Most commercial sites, whether they’re niche sites or general merchandise sites, sell a variety of products. Users’ ability to navigate the site and find one or more products that they’re interested in is incredibly important.
I would ask several questions about whether or not the site is easy to navigate and whether or not the menus/product categorization are logical. I would also include questions designed to gauge how difficult it is to find a ‘Contact Support’ link for telephone/email questions. Customer service is key in commercial sites and successful companies need an access point/support avenue beyond the Web site. Amazon.com is successful because the Web site does a great job at product categorization and the company’s customer support is excellent.