Heuristic Evaluation – datamentor.

org 1

Heuristic Evaluation – dataMentor.org

Sanket Anil Naik

Sharmishsta Dhuri
Sree Anirudh J Bhandaram
Kate Walders

Usability Testing
Instructor: Kara Rozen
4004-749-70
Spring 2008
Rochester Institute of Technology
 Heuristic Evaluation – datamentor.org 2

CONTENTS
INTRODUCTION...........................................................................................................3

HEURISTIC EVALUATION.............................................................................................4

CONCLUSION............................................................................................................12
 Heuristic Evaluation – datamentor.org 3

INTRODUCTION

DataMentor is the friendly front door to assessment data and instruction with solutions
for the classroom level teacher. It allows teachers, administrators, curriculum specialists, and
other professionals interested in improvement through the use of data to analyze district
assessment scoring results and select from an array of valuable options to target student
growth and improvement based on individual or district needs. It allows the user to analyze
district assessment scoring data, identify performance gaps between individual school
districts and the local BOCES aggregate result, and explore professional development
resources. Data Mentor provides its users analysis tools like the Gap Analysis tool and the
Trend Analysis tool to help its users analyze their students’ performance. Data Mentor uses
school data to convert performance into charts which is a very clear and precise method of
representation; this allows the users to understand performance just at a glance on the
charts. Each chart contains links that explain standards information or display assessment
questions to help educators understand the standards. DataMentor provides varied resources
to users which are linked to New York State Standards and performance indicators; this helps
the user understand the assessment data and what it means in terms of instruction.
 Heuristic Evaluation – datamentor.org 4

HEURISTIC EVALUATION

Visibility of system status - 3.0/5.0

• Whenever the user navigates through the navigation menu design interface gives
multiple responses like point I, II, & III. From Fig-1

Fig - 1
• User is informed about the loading delays on most screens
• The website is disorganized and confusing; a user can easily get waylaid trying to
navigate the site. This is especially true on the homepage. The navigation bar is in
different locations on different screens.
• Information or features that would be frequently used are mixed in with features that
would only occasionally be used. The bottom icons would suffice. Also, there is an
abundance of support features that are readily apparent and serve the opposite
intended reason that is, discouraging a novice user by making them feel that
navigating this site is very difficult.
• Some pages lead to dead ends, for example, from the home page, if one clicks on
Trend Analysis then Instructional Analysis then click on options to the right, the first
option results in a blank screen, clicking on the second option results in a message
 Heuristic Evaluation – datamentor.org 5

that there is no video available (thus no info), and clicking on the third button,
nothing happens (and analyze is spelled incorrectly).
• The system at a glance looks complicated and a user would not know where to go for
what. Chart list which would be extensively used for the analysis gets concealed in the
collection of less important options like quick tour, new features and online support.
The fault is in the architecture of the site as the features are not organized properly.
• In the tutorial option, the user is not notified about the currently selected topic and
there is no provision for the user to switch between the topics and has to start all over
again from the home screen.
• Even though GUI gives visual responses to the user it is very difficult to track where to
go next. Less important facts such as Take a tour, online support center are
highlighted on homepage in navigation menu. These options should be part
of help.
• The image of home icon with or without key image will represents whether
user is logged in or not. New users will not be able to distinguish this visual
difference. The key for the home icon may confound the user as being
some protected access or need to enter some password or passkey.
• There is no proper feedback on improper login attempt.

Match between system and the real world - 4.0/5.0

• The terms used in the site follows the convention, which makes a naïve user
understand what each menu may contain.
• Icons are familiar and they’re easy to identify.
• Data mentor makes use of optimal technical vocabulary which makes information
appear in a natural and logical order, however the description for this is difficult to
find and is provided only in multimedia mode.
• The system uses the NYS learning standards in the charts.
• Home icon is misleading as there is a key symbol on it, indicating some type of access
restriction!
• The various icons are logically arranged except the setup icon, which could be moved
down the menu and placed before the help icon.
 Heuristic Evaluation – datamentor.org 6

User Control and Freedom - 3.0/5.0

• The site is organized over multiple hierarchies. However the user is not provided with
ability to traverse freely in this hierarchy i.e. a user would not be able to go one level
up, example in the quick tour option if a user goes up to the technical setup tour level
he would not be able to go back to the data analysis tour level, and would have to go
back to the home screen and again click on tour. So the user movement is restricted in
the site. Though many features are provided, there is a limited access for a user to go
from one feature to the other.
• There are many features stacked one after another and this structure restricts user to
follow determined set of actions. For example, after selection of Quick tour option
when user selects submenus and wishes to come back on main menu list can either
select home and start the tour again or else can choose barely visible arrow keys
shown in Fig - 2.

Fig – 2
• Users can’t cancel out of current activity in progress. The best he can do is click on
some other link and he’s taken away to that page.
• No mnemonics or keyboard shortcuts in the site to help a user.
• Users can choose the data they want to use, but access must be requested
beforehand.
 Heuristic Evaluation – datamentor.org 7

Consistency and Standards - 3.5/5.0

• Consistency complies with the standards of color scheme, layout, terminology and
behavior. Data mentor follows a similar color scheme throughout the website. Similar
chart types are used within the site so interpretation becomes easy.
• No heavy use of uppercase alphabets.
• No abbreviations to confuse a user.
• Icons are labeled logically and the number of icons is minimal.
• Each window has a title which is descriptive of its contents.
• Website is free of grammatical errors except the word ‘analyze’ which was misspelled in
the trend analysis page.
• Text and content is left-justified, following the normal standards.
• The system starts with a compact screen and has this screen for multiple features. While
for some features like chart list and instructional analysis the appearance is different. So
the consistency is not maintained. Also the look and menu hierarchy changes for certain
options like help, online support system and new features.
• Multimedia help is not provided for all the features which affect the consistency of the
site.
• Occasionally same functions are titled differently (main menu, home, and sometimes,
return to DataMentor mean the same thing). The screens are not consistent throughout
the site. And the video are different sizes and in different places on the screen.
• The website is not too loud, but the home page does have some flash animations which
aren’t really necessary.
 Heuristic Evaluation – datamentor.org 8

Error Prevention - 4.0/5.0

• Sounds are used to notify user whenever there is an error. For example, Invalid
Login.
• Menu choices on higher levels are used as titles for lower levels, helping the user
avoid confusion.
• Menu choices are logical, distinct and mutually exclusive.
• There are no data entry screens, which provides a user with a defined flow and
prevents user from committing error.
• Data on the charts is populated only with the information that the user has
access to. This way, the user need not click on an inaccessible link.
• There are no warning messages while editing login information or changing
password. A user needs to be notified what he is about to do to prevent errors, however
no such notifications are present. No confirmation message is provided while a user
adds or deletes data from his profile
• Similarly there is no notification message provided whenever a user quits a
session e.g., quick tour.

Recognition rather than recall - 4.0/5.0

• Once user gets familiar with the website many control become obvious to the user.
• Data is displayed starting on the top left corner, following standard convention.
• Flash design of the website plays hide and seek with important interface controls.
Revealing application controls on rollover or with a click can create exciting visual
transitions, but will slow user tasks and create significant frustration.
• As the system is built over high level of hierarchy, a user may not remember from where
to go for what. Also the user is not provided with the information as to where he is
currently on the site map. Thus a user may have to remember the hierarchy to get to a
particular feature.
• The main elements of the screen are present where a user would most expect it. An
exception though are the links which take you back to the main page, the font size does
not draw attention to these links and it takes a while to figure out the location of these
links.
• Unusable icons and un-clickable links are grayed out.
 Heuristic Evaluation – datamentor.org 9

• The screens are grouped into logical zones, the most noticeable three parts being
navigation, header and content.
• Content and icons are logically separated to avoid confusion
• Fonts and elements on a page are relatively sized to indicate importance.
• There is a good contrast of colors, fonts and images in the site.
• The mapping between icons, buttons and their actions is pretty obvious.

Flexibility and Efficiency of use - 4.0/5.0

• There are multiple ways to access Help and very few ways to access the primary data
and information.
• There are features that allow users to tailor frequent actions such as saving their district
settings and displaying their most recently accessed charts.
• No keyboard shortcuts are available to the user.
• User can not customize the functionalities provided.
• The system as such has a very limited option to access a particular feature. So there is
no added advantage for an experienced user to traverse through the site and would
have to go through the same number levels as a novice user to access a feature.

Aesthetic and minimalist design - 3.0/5.0

• It’s not exactly the most beautiful website out there, but it gets the work done.
• All icons are visually and conceptually distinct.
• Field names, icons names and titles are brief and descriptive.
• Contrast is helpful in some places (blue and green on white) however the main
navigation bar (usually on the side or bottom) is grey against white, which diminishes its
importance role. When the main navigation bar is anchored vertically on the right,
scrolling over it, the text bar covers the other options.
• Uses gestalt grouping laws sometimes effectively and sometimes poorly. The font
size on some pages, like the gap analysis page, is too small!
• The main screen of the website may confound any new user as it does not depict the
sole function of the site.
• The meter on the left is irrelevant and not needed. Also the numbers depicting the
radians on the meter is irrelevant and just confuses the user.
• The extensive use of flash makes the site filled with insignificant information and
does not provide the aesthetic look which it intends to give.
Heuristic Evaluation – datamentor.org 10
 Heuristic Evaluation – datamentor.org 11

Help users recognize, diagnose and recover from errors - 3.5/5.0

• If a user does not find the data he is looking for, the data mentor reverts with the default
error message in order to recover from the errors.

Fig -3
• Only at few occasions error messages occur which are displayed in plain
language. However error message neither depicts the error properly nor does it provide
a way to recover from the error.

Help and Documentation - 3.5/5.0

• Help guides the user through the different aspects of the website.
• Some features are provided with multimedia help which provides a detailed analysis of
the procedure to use the site and its features.
• Online help is also available with data mentor.
• Easy access to and from the help system.
• Help videos are a bit frustrating and they start playing automatically.
• In text mode, the help screen is no bigger than 300 x 300 pixels, which makes it very
hard to read stuff off it. There is a lot of vertical scroll.
• Though substantial help is documented, it is not properly organized which makes it
difficult to find the right help at the right time.
• A user cannot resume work from where he had left off before using the help system.
 Heuristic Evaluation – datamentor.org 12

CONCLUSION
For the targeted user, datamentor.org is a really helpful tool and it is easy for a user who is
educated about the website to perform all his tasks with ease. The critical problem of this
website is being able to be used by a novice, or an inexperienced user, it is really easy for a
new user to be confused about the various parts of the website. The help system, although
very detailed, is not that great, a new user might, in all probability, be lost and he would not
be able to understand what the site is about! The site is pretty good and usable overall, but
there are a few issues like the animation on the front page, which is not really useful and it
confuses the user about the intended action, another issues is the help videos, a user would
most likely want to skip quickly to the part where he needs help, rather than watch the whole
video, the help text is very unreadable as it is presented inside a box with no more than 5
words per line, there is a lot of vertical scroll which really doesn’t serve any purpose. The
aesthetics of the site can be improved and the layout can be made consistent by using the
same styles throughout the website.