TOWN OF CHAPEL HILL 
USABILITY REPORT 
www.townofchapelhill.org 

 
 

Valentina Arismendi, Lizzie Dravis, Hailey Haymond, Julia Johnson, Jill Pownall
 

 
 
 

TABLE OF CONTENTS 
 

Executive Summary…………………………………………………………………………...3 

Methodology……………………………………………………………………………………..4 

Participant Profiles…………………………………………………………………………….5 

Limitations………………………………………………………………………………….……..6 

Positive Findings………………………………………………………………………….…….7  

Minor Problems………………………………………………………………………………….9 

Major Problems………………………………………………………………………………….11 

Catastrophic Problems………………………………………………………………………13 

Recommendations/Suggestions…………………...………………………...………..15 

Appendix…………………………………………………………………………………………..18 
 

 
 
 

EXECUTIVE SUMMARY 
Five undergraduate students at the University of North Carolina at Chapel Hill conducted a 
usability study on the developing 'Town of Chapel Hill' website. The purpose of the study was to 
find usability issues before the website was launched. 

The study was focused on the usability of the mobile version of the site and the functions that are 
often associated with mobile use. This includes parking, busing, maps, reporting issues, and 
biking. The site was tested to see how effective it will be when users need quick information on 
the go. 

There were 10 participants in this study, tested from January 24-28, 2020. These tests included a 
pre-test questionnaire, eight usability tasks, a post-questionnaire, and interview questions. 

The test resulted in findings of both positive and negative aspects of the website. This was based 
on the actions and observations of the participants. Positive findings included things that worked 
well for the website. Minor problems were inconveniences for the users that caused them to take 
an extra step. Major problems included things that inhibit function and are disruptive to the 
website's usability. Catastrophic problems were things that prevented users from completing 
tasks, and extremely inhibit usability. All these aspects are discussed more thoroughly in the 
following sections of this report. 

Users need to be able to find information on transportation quickly, on a mobile-friendly site. The 
focus of the study was how the website can be better at this. We also take into consideration the 
limitations of this study. Based on all these findings, recommendations for changes in the site 
were concluded and can be found at the end of this report.  

 
 

 
 
 

METHODOLOGY 

Purpose 
This study had two primary goals in assessing the usability of the Town of Chapel Hill’s website:  

● To determine the usability of the site particularly on mobile devices. 

● To determine the usability and accessibility of transportation and parking information on 
the site.  

Method 
The test administrators each contacted and recruited two participants. These participants varied 
from being close friends, roommates, coworkers, and classmates. Each administrator set up dates 
and times with chosen participants, informing them of the test overview and predicting that the 
test session would take about 30-45 minutes.  

During each session, the test administrator explained more in depth of the usability project and 
asked the participants to fill out a brief background questionnaire (see ​Appendix A​). A consent 
form was given to each test user for permission of the test administrators to use audio and 
screen recordings during the session (​Appendix B​). Participants then were given eight tasks both 
on paper and read aloud by the administrator and tried to find information on the website on a 
mobile device pertaining to transportation and parking needs (​Appendix C​).  

As the participant worked through the tasks, the administrator used both audio and screen 
recordings to capture how each user went about problem solving, and they asked each 
participant to not use the search function on the site, but rather see how the user might navigate 
using only the available buttons through the home page and three-bar menu.  

Not every participant was able to complete the tasks in the given three minutes of allotted time 
​ hich shows percentages of task 
for each. The task success rates are outlined in ​Appendix D​ w
completeness. 

After the last task was completed, each participant was asked to fill out a post-test questionnaire 
(​Appendix E​) which asked the participant to rate the website overall using a 5-point scale for 
specific site components including Home Page, Top Navigation Bar, Search Function, and Overall 
Site. 
 
 

PARTICIPANT PROFILES 
We asked 10 users to complete several tasks to test the usability of the Town of Chapel Hill 
website. These participants were compensated with candy for their cooperation. Of the ten 
participants, eight of them use their phones more than a computer as their primary platform for 
searching information, and that specific phone commonly being an iPhone. 

Over 60% of participants said they use the internet for 9-15 hours each 
day and 100% of users said they search for information online on a daily 
basis.  

Our participants were mostly college students, with nine of them being in the age range of 20-22. 
Four of these we also classified as residents, meaning they live in a house located in a Chapel 
Hill zip code. We had an even percentage of males and females and similar results from both 
genders (explained in detail later in this report).  

 
 
 

LIMITATIONS 

Unfinished Website 
Unfortunately, because the website is still being developed, a lot of crucial links on the 
homepage were broken and our users were unable to fully use that design feature. This 
obviously affected our ability to fully test the website and prevented us from recording accurate 
data about that specific feature. 

Small Sample Size 

Due to the fact that our group only tested a total of 10 users, we realize that our small sample size 
is definitely a limiting factor to take into consideration. If we tested enough users to have a 
sample size of 100 or more we would have much more data to work with as well as a more 
diverse user group. This would definitely give us more insight into different ways the website 
design could be improved. 

Inconsistent Testing Environments 

Despite having the exact same user scripts for the test, we acknowledge that there were quite a 
bit of discrepancies that could’ve had an impact on the test results. For example, because there 
were 5 different test administrators with varying interviewing styles or skills, the way the test was 
conducted was different for each user. Also, because we are all college students in our early 
20’s, we tested our peers who were also mainly college students in their early 20’s.  

For a website that focuses on having information for families and older adults that live in Chapel 
Hill, we would’ve found more compelling test results if we tested a more diverse age group. Also, 
because typically younger people are more used to technology and have better skills with 
technology, testing older adults who may not have adept computer skills would be very helpful to 
point out big problems with the website’s design. Finally, there was inconsistent technology use 
despite instructing our users to take the test on their mobile devices. Each user had a different 
mobile phone and service provider, which can cause some inconsistencies with internet speed, 
network connection, etc. 

 
 
 

POSITIVE FINDINGS 

Color Theme 
A lot of the users that were tested commented that they enjoyed the color scheme in the 
website’s design. Our test findings indicate that the website color theme is one of the elements 
that users consistently liked, as well as appreciating the consistency of the color usage.  

One user said, “The colors make it easy to see what you can click on, and 
they divide up the webpage in a way that appears nicely organized.”  

This is a positive because taking advantage of good color usage in an UX/UI design really helps 
to point out clickable links and other design features in a way that makes a website look more 
organized and professional. 

Social Media Icons at the Bottom of Homepage 

Users appreciated how easy it was to find the social media pages for the town of chapel hill on 
the website, all you had to do was scroll to the bottom of the homepage to find them. Users 
commented that the placement was “logical and helpful,” especially for users that are trying to 
find event information or contact information. 

Homepage Circle Buttons 

Users lamented that when the circle buttons on the homepage worked, they found them to be 
very helpful and a clear source for identifying the type of information you’ll see after clicking on a 
circle. However, this finding was hard to test because more often than not, when a user clicked 
on one of the circles it would take them to a page that said “Road Closed” because the website 
wasn’t fully developed yet. So while users did seem to like the homepage circle buttons, we were 
unable to fully test to what extent users appreciated them. Also, another thing to point out about 
the circle buttons; while users did think they were helpful, they also thought organizing them in a 
way that prioritizes certain information with clearer labels would be an overall better design for 
the website. 

 
 
 

Good Information Resource 

Consistently throughout the user tests, users would comment that the website does seem like it 
has a lot of really good information about the town of chapel hill.  

“I could definitely find anything I ever wanted to know about Chapel Hill 
with this website,” one user said.  

Another user said, “It looks like it has a lot of good information on it and that it could be a good 
resource, but the way it's laid out does not make any sense. Especially the top bar is broken 
down into such weird categories that you don’t know which one you apply to.” However, this is a 
positive because it tells that the website is already on the right track of communicating the 
information you expect it to communicate to you. This is an extremely important feature of any 
website, and if the town of chapel hill focuses on improving the website’s organization of that 
information we feel that users will appreciate the website even more. 

Quick Links 
Finally, when users could find and use the quick links they found them to be very helpful. 

 
 
 

MINOR PROBLEMS 
We defined minor problems as inconveniences where the user took an extra step, and in 
addition, things that deserve attention but don't make or break the site. The first thing to note is 
that the hamburger menu icon could be slightly bigger on mobile. For some testers, it wasn't 
apparent that the menu was there to use. 

These issues caused users to take an extra few seconds to get where they want. They were 
inconvenient and made the experience on the Town of Chapel Hill’s website much worse than it 
could have been. 

Interactive GIS Map 

“The interactive map was a good feature to have even though it took me 
more than a few clicks to get there,” said one of our testers. 

There are some confusing headings that use language which a regular user would not 
understand. This includes "GIS Map." None of our testers knew what 'GIS' meant and neither did 
any of us. The interactive map was a helpful tool but this caused some difficulty in reaching it. 

“Chapel Hill Transit”  

“You had to know that the bus route was called chapel hill transit,” said 
another tester. 

Additionally, a lot of people do not know that the bus system is called "Chapel Hill Transit,'' 
especially non-residents. This causes people to have to think for an extra few seconds while 
searching for bus routes. As a result, they may not get to where they want to go. Under 
'Residents' in the sidebar, there are many headings that begin with "UAT." This seemed repetitive 
and confusing. 

Links to Off-Site Pages 

 
 

Many links took users to the current live site. We assume from this that these pages are not being 
redesigned, but we could be wrong. If this is true, there will be a lot of inconsistency in the design 
of the site. We also suggest that when a link takes you to an outside site, it opens up in a new tab 
so the user can get back to the Chapel Hill website. We had issues with this when users went to 
the Park on the Hill website. 

Social Media iFrames 

The social media iframes on the bottom of the homepage are quite frustrating on mobile. It is 
hard to scroll past them because they take up the width and most of the height of the screen. 
Users end up scrolling through the twitter timeline instead of scrolling past them. This is 
especially frustrating because a lot of the quick links that are useful for mobile are underneath 
the social media at the very bottom of the page. In addition, the Instagram iframe doesn't work. 

IFrame Issue In Screen Recording 

 
 
 

MAJOR PROBLEMS 
Our group defined a major problem as a problem that inhibits function and is disruptive to the 
site’s usability. These issues hindered tested users the ability to complete tasks in a timely 
manner and they need fixing. Generally tasks with a major problem took users two minutes or 
more to complete the task when it should have taken them under 30 seconds. 

Navigation Organization 

Tested users experienced difficulties when navigating with the three bar menu on the mobile site. 
This caused a major frustration because that is the main source of navigation for mobile users, 
and as one user remarked,  

“The top bar is broken down into such weird 

categories that you don’t know which one you 
apply to.”  

We found that when users were searching for transportation 

and parking information, the three bar menu was not intuitive 
as there is no designated section in the menu for 
transportation/parking needs. Instead, our users searched in 
the Residents, Services, And Things To Do subsections in 
attempts to find information.  

When prompted to search under the Residents' subsection, 

users were surprised by the lengthy amount of links, many of 
which they didn’t understand. Many commented about the 
“UAT” headings, not understanding what they meant,  

 
 
 

‘Quick Links’ Location 

During our testing, we found that a lot of the vital information needed for parking and 
transportation can be found using the “Quick Links” section at the bottom of the home page. The 
issue with this however is that the user has to scroll all the way on the bottom to find them, past 
the large sections reserved for social media posts.  

One user said, “It seems like every question 

is at the bottom of the page, so maybe 
those should be moved up.” 

If this section of links were moved up on the homepage 

or rather found more directly in the navigation section, 
we believe these could be a greater use to those 
searching for this information.  

 
 
 

CATASTROPHIC PROBLEMS 
Our group defined a catastrophic problem as anything on the site that took an extremely long 
time or prevented users from completing the tasks at all. Tasks with catastrophic problems were 
often not completed when they should have taken a minute or so, either from not working or the 
user getting too frustrated to continue/running out of time. 

Broken Links 
Many users had difficulty completing tasks because of links leading to “error: page not found.” 
This happened a lot with parking. We were unsure if this had to do with the site being under 
development, or if they were just broken.  

It was unclear since users were able to find working parking links in some places but broken 
ones in others. ​For example, one user experienced “error: not found” pages when searching 
for parking under “A-Z Services” -> “Parking Services,” yet was able to find one under 
“Transportation” -> “Park on the Hill” 

Many were unable to complete the gym schedule task because the gym link led to a page about 
tax payments. 

Screen Recording Showing Gym Schedules to Tax Payments 

 
 

Paying a citation was also very difficult, as many users were unable to find working links.  

Another user was so frustrated they said, “You want me to pay a parking 
ticket? Okay it’s, like, impossible.”  
One user also commented about the bus routes, “I’m trying to click on it, but I can’t seem to get it 
to take me anywhere, like there’s no hyperlink associated with it.”  

These are simple problems, but definitely catastrophic. If the broken links are simply due to the 
site being in development, that greatly impacts the usability tests. It seemed that the site might 
have been more intuitive to users, but many had to take odd routes to find information because 
of this issue. 

 
 
 

RECOMMENDATIONS/SUGGESTIONS 

Hamburger Menu 
In our testing, we found that some of the participants did not use the hamburger menu, possibly 
because it went unnoticed. When this issue occurred, users were limited to only the information 
they could find from the links on the homepage, which is why it’s crucial that the links displayed 
are the ones most frequently searched for. Our team recommends that the links on the home 
page are the ones which allow users to still accomplish the site’s key tasks. The “Quick Links” did 
a great job guiding users to where they needed to go without using the hamburger menu.  

The Nielsen Norman Group, who are world leaders in UX research, said, 
“Mobile sites that use the hamburger or three-line menu need to support 
navigation activities throughout the site, in case users don’t locate or use 
the main site navigation.”  

Participants were pleased by the direct access of these links, so we suggest keeping the various 
links in this section. Additionally, adding a “back to top” button would also facilitate navigation 
between the Quick Links at the bottom and the primary navigation at the top. However, to ensure 
users are still at some point landing on the main navigation, our biggest recommendation is to 
increase the size of the hamburger menu icon. 

Social Media 

One of our positive findings was that the social media icons were located exactly where users 
would expect- the bottom. We recommend those icons remain where they currently are. 
However, when it came to the social media iframes, users on the mobile site were frustrated 
by both the amount of space they took up, as well as how it disrupted the scrolling process. 
Our team’s suggestion would be to discard the iframes from the mobile site. This way, users 
reach the bottom of the home page faster, which is where they expect the icons to be to begin 
with. It is unlikely that deleting the iframe would prevent the user from reaching this information. 
Because users are using their mobile device, the icons will automatically open the corresponding 
 
 

social media app, which easily shows the latest content that would have otherwise been shown in 
the iframe. 

Quick Links 

As noted earlier, the Quick Links were some users’ favorite feature. Therefore, we recommend 
shortening the amount of time it takes users to find this section. This could be achieved in one of 
two ways. First, as with deleting the social media iframes, shortening the length of the site so that 
less scrolling occurs from the top to the bottom would help users reach the Quick Links in a 
shorter amount of time. ​Another alternative would be to move the Quick Links section to the 
top of the site, so that users have the options as soon as they land on the site. 

Acronyms/Specific Names 

Many users were left confused about the interactive map task. Participants did not connect that 
the link for GIS maps would lead them to the interactive map they were tasked on finding. 
Additionally, many users were left frustrated by the amount of times UAT appeared on the menu 
without having any idea as to what it meant.​ Our team’s suggestion would be to delete the 
acronyms, unless they are very popularly known among all audiences, such as FAQ​. 
Additionally, the more general the name of a page can be, such as “Game Day Transportation” 
versus “Tar Heel Express”, then the easier it will be for users to identify what they are looking for. 
Visitors, and even residents, might not identify that the Tar Heel express is the bus for game 
days, however if they saw it labeled as such they would know that is the tab they are looking for.  

Navigation Tabs 

Users found the navigation menu the most frustrating feature of the site. Although we understand 
that many of these tabs cannot be reduced, there are ways to make it so that the information is 
more clear to users. For example, reorganizing the submenu tabs to be in alphabetical order 
would help users quickly get to the area of where the information they are looking for is located. 
When users were tasked with finding information about transportation, many scrolled to the 
bottom of the menu as “T” is near the end of the alphabet. Similarly, many users were confused 
about what information would be under each “Residents”, “Government”, and “Business” tabs. 
 
 

In order to lessen the frustration, we suggest adding a “How Do I?” tab. This tab could have 
subpages such as “Pay for...”, “Find”, “Contact”, and “Apply for”. ​This takes some of the 
information that was previously located under the Residents tab and puts it in a more visible 
location. Similarly, the Government tab could hold a subcategory called “Departments”, which 
would allow users to quickly identify specific departments within Town of Chapel Hill, such as 
Parks&Rec, Police, and Utilities. 

 
 
 

APPENDICES 

Appendix A: Pre-Experiment Questions 

 
 
 

Appendix B: Informed Consent Form 

 
 
 

Appendix C: Usability Tasks 

 
 
 

Appendix D: Task Success Rate 

 
 
 

Appendix E: Post-Experiment Questions 

 
 
 

Appendix F: Interview Questions