User Testing Prototype Test and Design Report

Andreea-Daniela Ene
Vision, Graphics and Interactive Systems
Group 473
Student Number: 20151016
School of Information and Communication Technology
Aalborg University
[email protected]

ABSTRACT
The aim of this paper is to describe and analyse the Conceptual
Model and the Low Fidelity Prototype of an application, to
discuss about how the the User Testing has been performed,
and to present the results that have been collected after it.

Author Keywords Figure 1. Task organisation.

user experience design; User Testing; prototype; qualitative
and quantitative data.
Task Organisation
Using the agile development method, all tasks are organised
INTRODUCTION
in four lists: Backlog, Waiting, Doing, and Done. The first list
This report is focused on describing how the Low Fidelity
is the default one, where all the tasks are added automatically.
Prototype of an application focused on management and col-
All tasks inside this list are considered necessary to the project
laboration in and between non-governmental organisations has
and after the deadline is added, they should be moved to the
been created, to present the scenario of the User Testing, and
Waiting List. When a user wants to finish an activity, the task
to analyse both the qualitative and quantitative data collected
has to be moved to Doing List and after completing that, it
after performing this experiment.
will be moved to the Done List. All tasks from the latest list
This task has been chosen in order to test the performance of have to be validated by the project manager so the user will be
an iOS Application, focused on helping volunteers to better awarded.
organise their activities. The purpose of the User Testing is to
In Figure 1 we can see that the lists are organised as the
demonstrate the usefulness and clearness of certain concepts of
following:
the platform, such as gamification system or agile management
method. Backlog: Tasks that need to be completed in the near future,
but they dont have a well-defined priority or deadline.
DESIGN Waiting: Tasks with a high priority that need to be com-
The design of the iOS Application started by finding a main tar- pleted in the shortest time possible.
get group, non-governmental organisations. In order to build
a Low-Fidelity Prototype, a conceptual model was needed to Doing: All tasks that are in progress and already assigned
have a better understanding about its main structure. to a user.
Knowing that all NGOs are organised in several departments, Done: All tasks that are completed but not necessarily vali-
each of them having a manager, two types of roles will be dated by the project manager.
needed, Project Manager and Regular User. Both of them (Only tasks with a check mark have been validated.)
will have read and write privileges to their own profile, will
be able to create a new project, and mark their own tasks Gamification
according to their activities. The differences between the two Gamification is a method that uses the game idea and its mech-
roles are that the manager is able to add organisations and anism in a completely different context. The purpose of this
points, and to approve tasks. technique is to encourage users both to solve problems and
develop themselves. Gamification was studied and applied in
Concepts like gamification or agile development were used to several fields, in order to have users attention.
motivate the volunteers and to make them eager to gain points
or badges. After several thresholds that are set by the manager, In order to motivate the users, an awarding system will be
members are awarded right after completing a task. added. For each completed task, a user will receive a certain

1
 Figure 2. Gamification System.

number of points, according to a difficulty level. The points

will be set and validated by the project manager. After a
threshold has been passed, they will receive some trophies or
badges that are shown on their personal profile.
This technique has been chosen because volunteers often need Figure 3. Conceptual Model.
to be slightly encouraged in order to complete their activities.
Game industry is continuously growing and it is becoming one
of the most interesting fields that raise youth peoples interest. In Figure 4, we can see the following screens, from left to
In Figure 2 some concepts of this method can be seen. right:
1. Main Screen
Conceptual Model
Conceptual model is the model used for a better understanding 2. Log In
of the application. It should contain information about what
users can do with the software and what are the most important 3. Dashboard
concepts that they should be aware of. The conceptual model
has to be as simple as possible, containing only the concepts 4. Main Menu
needed to provide the main functionalities. The most important
ideas that need to be included are about: 5. Organisation Profile
Actions: what the application should do and what are the
main parts that the users can interact with. 6. User Profile

Concepts: what are the main concepts used by the applica- 7. Projects
tion.
8. Details of a Project
Objects: what are the most important parts of the applica-
tion. 9. Tasks

As well, it should be taken into consideration that the relation- 10. Details of a Task
ship between these ideas has to be represented.
11. Organisations
This model can be seen in Figure 3.
12. Edit Profile
Low Fidelity Prototype
A Low Fidelity Prototype is the first version of an application
TEST
used to either test the performance of several concepts or to
test the software itself. This prototype has been created only User Testing of the Low Fidelity Prototype has been performed
for User Testing, it is not the final product nor design of the by 12 participants. After testing the application several times,
application. a list with the most important aspects that could be improved
has been created. The parts that are essential to this applica-
This prototype is useful for having a better visualisation about tion and the aspects that have to be tested are: the concept
the application and as well for performing the User Testing. of gamification, the multitude of screens, and the awarding
system.
Using the conceptual model, a Low Fidelity Prototype has
been created. A sketch prototype that looks like an iOS native In order to perform the User Testing, both quantitative and
application has been designed, that eventually is added in qualitative methods should be used. Quantitative, because
Keynote in order to have a flow throughout the application. it should be statistically checked how effective some parts
This interactive mockup would test whether or not the flow of of the application are, and qualitative because, for a better
the application is smooth and consistent. understanding, the problems will need to be well-detailed.

2
 Figure 5. Default settings to perform the User Testing.

Figure 4. First Low Fidelity Prototype.

The Observer: Makes sure that everything goes well
during sessions and that the users understand both the
Test Plan scenarios and the tasks.
Considering that this phase is the first and foremost activity Deliverables: Detailed document that contains information
that happens in testing an application, it started by discussing about each interview and survey results.
about several aspects of this software that should be taken
into consideration. The ones that have been found the most The Environment: Team Room
important are the scope, the roles and the responsibilities for
each person, the deliverables, the environment, the tools that Tools: Google Forms for creating the surveys, Keynote for
will be used, the risks, and the exit criteria. making the interactive application, and Low-Fi Prototype
in case it will be needed.
The detailed items that were discussed are:
Risks: Some of the risks that should be taken into considera-
The Scope tion are the availability of all team members, the user might
not be interested or might not have any knowledge about
In Scope: A test scenario with several test objectives mobile applications or several concepts that have been used,
has been created. It consists of a script with a well or he wont be honest.
detailed description of what the application does and
what the purpose of the User Testing is, several ques- Exit criteria: Testing should stop either when the user con-
tions and tasks that the user has to complete by himself, siders or after all questions have been answered.
and two Google Forms, one for the user and the other
one for the facilitator to write in. Test Design
Out of Scope: Not the user is being evaluated, but the The Test Design for the iOS Application started by creating
application. There is no concept of wrong or right a script that would briefly explain what the User Testing is
answer. about and what the purpose is. As well, a consent form has
been made for getting permission before conducting the tests.
Roles and Responsibilities: The staff with the following
roles and responsibilities who will participate to each test Methods
are needed: The method used for this User Testing is the interview. This
would be the best method for getting opinions from potential
The Facilitator: Responsible for conducting the test: users of the application. Using this method both qualitative
greeting, pre-test briefing, distributing consent form, and quantitative data have been obtained. During the test, the
and conducting the test. screen and the user being interviewed have been recorded.
The Primary note taker: Takes notes about the tester. In Figures 5 and 6 the setting for each test and the four frames
The Secondary note taker: Takes notes about the facili- from the recordings can be seen.
tator. For both qualitative and quantitative data, two Google Forms
The Cameraman: Recording each session. have been created.

3
 Figure 6. Recording the videos with four participants.

Quantitative Data Analysis

The first method used for this User Testing is Quantitative
Research. This method is used to quantify several aspects of
the application by generating numerical data that can be trans-
formed into usable statistics. It helps quantifying opinions,
behaviours, or checking how clear some concepts are. Quanti-
tative Analysis uses measurable data to formulate conclusions
in research.
The first Google Form is focused on gathering numerical data.
A Likert scale has been used to rank their answers on a scale
from 1 to 5; 1 meaning that they strongly disagree with the
statement, and 5 meaning that they strongly agree. Most of the
questions are about the concepts that might be new to them
and about the navigation within the application.
Some questions that have been asked in the survey with the
Likert scale can be seen in Figure 7.
Qualitative Data Analysis
The second method used for this User Testing is Qualitative
Research. The aim of this method is to get a complete descrip-
tion and to gain a deeper understanding of users points of
view. This would provide insights into the problem and thus
finding eventual solutions would be easier.
The second Google Form is focused on gathering detailed
descriptions of users opinions. Most of the questions are
about the the concepts that have been discussed in the first
survey and are focused on asking for more details about them.
As well, users are requested to answer some questions related
Figure 7. Quantitative Questionnaire.
to their overall opinion about the application and about what
are the features that they would add or get rid of.
Some questions that have been asked in the survey can be seen
in Figure 8.

RESULTS
In this section the Usability Test Results will be analysed.
Those results were obtained in two different ways: the first
one by analysing the answers of both surveys, and the second

4
 Figure 10. Qualitative Results.

in the first questionnaire, where all obtained data is on a Likert

Scale. As well, there were some questions about testers level
of checking several aspects regarding the logged in user, such
as its current number of points, or how to get between different
sections. Their answers have been labeled into right or wrong,
in order to easily classify them.
In Figure 10 some of their detailed answers can be seen.

DISCUSSION
Those methods have been chosen because they helped obtain-
ing both qualitative and quantitative results, while analysing
participants reactions, level of understanding, and intuitive-
Figure 8. Qualitative Questionnaire. ness. As an alternative, testing volunteers or member of an
organisation that would be interested in using such an applica-
one from the Data logger that annotated some notes about the tion would have had a better impact for this application that
performance of the participant, in order to check how intuitive has them as a target group.
the application was to the participant. Even if most of the participants gave an overall positive feed-
back about the application, several issues have been encoun-
Quantitative Data tered.
The Quantitative Data that can be extracted from the question-
naire was answered on a Likert Scale from 1 to 5. The most important one was that many of them did not answer
honestly and they were not consistent. According to the analy-
In Figure 9 some of this data with a computed mean and mode sis, even if for most of them the gamification concept was not
can be seen. clear, they said that this concept has been clearly explained
- even if the application does not provide such explanations.
Qualitative Data During the test, in order to check if they are honestly answer-
The Qualitative Data can be extracted from the questions with ing, one task that cannot be performed has been requested to
text answers. Those answers would help finding deeper details do, and, as a result, most of them unfortunately were not. They
about the aspects that could be improved and are not explained were not consistent between the two qualitative and quantita-
tive questionnaires, having different responses for the same
questions. Some of them got really confused between the
difference between Dashboard and Menu, or User Profile and
Edit Profile.
The most important suggestions participants gave are adding
a Help section, making the order of the sections in the Main
Menu more intuitive, merge some sections, Log In using a
social network, share the same project between several organi-
sations, and adding the resume to users profile.
After gathering the results, several changes have been made:
First page: Log In using Social Media(Facebook, Twitter,
and Google+).
Dashboard: Swap Points with Task and add the number of
Figure 9. Quantitative Results. points inside the blue star.

5
 or developed and how they can be improved or changed. The
test included several tasks that have been performed by the
participants and as well two questionnaires in Google Forms.

REFERENCES

Figure 11. Second Version of the Low Fidelity Prototype.

Slider Menu: Merge User Profile and Edit Profile into Your
Profile and rename Organisation Profile to Your Organisa-
tion; add Help section.
User Profile: add Edit button near the number of points.
Project Profile: add Information section with several infor-
mation about the current project.
Other: Some items in Organisation Profile, Projects, and
Tasks have been reorganised.
In Figure 11 the second version of the prototype can be seen.

CONCLUSION
After designing the prototype using the conceptual model of
Task Heroes, an application focused on motivating and en-
couraging volunteers or members of any non-governmental
organisation, the purpose of improving several aspects has
been achieved by performing User Testing on 12 participants.
This prototype has been created by transforming a sketch
paper-based design into an interactive picture-based presenta-
tion, that would help testers experience a natural flow between
its sections. After the testing has been performed and using the
interview as the main method, both qualitative and quantitative
data has been obtained as a result. The purpose of this was to
find out which parts of this application are no well explained