PART 2 (OF 2) COMS 363 (L01) Fall 2018 – Assignment Guidelines. J.

Andre
Usability Study: Proposal & Final Report Assignments
See Part 1 of this set of guidelines for general information about assignment submissions and
policies. This document focuses on the proposal and final report assignments related to the
Usability Study group assignment, but it also includes some information from the Group Task 1 and
the Heuristic Analysis assignments as they are relevant to your group usability study.
Remember that group assignments should include only the names of students who contributed to the
work; including non-contributors’ names is a form of plagiarism. Normally, all group members will earn
the same grade for group assignments, but the instructor may adjust individuals’ grades based on
formal or informal peer evaluations, administered at the instructor’s discretion or at the request of any
group member (whose name will be kept confidential.)

Overview & Objective of the Usability Study: This set of assignments will introduce you to general
principles, writing strategies, and genre expectations for proposals and recommendation reports.
Through the process of conducting and reporting on your study, you will be called upon to do
synthesize results from both primary and secondary research; to apply document design strategies to
ensure your documents look professional, promote easy information retrieval, and use visuals to
communicate information effectively; and to apply principles of good writing to ensure that your
documents are clear, coherent, readable, and professional in tone. This project will also give you an
opportunity to develop further competence in integrating information from sources into your writing
and documenting sources properly using either APA or IEEE style.

While these learning objectives could be met by a variety of report-writing tasks, a usability study is an apt
topic because the concept of usability underpins the design of effective products, documents, and
websites. Designing (and testing) for usability demands a focus on users’ needs, perspectives, and
behaviours, and getting into the habit of critically thinking about these should serve you well whether
you plan to pursue a career as an engineer, computer programmer, or communication specialist.

Websites to Study: For your study, please choose a website for an organization listed at one of the
following links:
● Charities and non-profit organizations in Calgary, Alberta:
http://www.charity-charities.org/Canada-charities/Calgary-AB.html
● Calgary area volunteer programs: http://calgaryarea.com/index.php?p=volunteer

Your Research Tasks: For your usability study, you will do the following research:
1. secondary research of published sources related to usability and website design & your own
heuristic (personal) evaluation of the target website. The heuristic analyses completed by
individual group members for an earlier assignment will be used in your group’s final usability
study report, but some reformatting, revising, and editing will likely be necessary in order to
ensure that the analyses are clear, well written, properly edited, and consistently formatted.

2. primary research (i.e., first-hand research) taking the form of either

● an in-person usability test involving two student-testers or
● a web-based usability survey emailed to the class (in which you ask users to find certain
information or do certain tasks on the website, to rate the site according to set criteria,
and/or to comment on the site’s strengths and weaknesses).
 As shown in Figure 1, your heuristic evaluation is likely to identify potential usability issues to be
investigated further through in-person or online usability testing. Detailed guidelines about the proposal
and formal report assignments follow the chart of assignments below.

Figure 1: An evaluation process of a usability study for a museum website

Source: Harms, I., & Schweibenz, W. (2001). Evaluating the usability of a museum website. Papers: Museums
and the Web, 2001. Retrieved from http://www.archimuse.com/mw2001/papers/schweibenz/schweibenz.html

Due Assignment Overview & Guidelines

Nov. 9 Usability Study In memo format, this 3- to 4-page single-spaced (1400 to 2000 words)
Proposal (group, proposal will introduce the website you plan to study, briefly outline your
10%) evaluation criteria, describe your research method in detail, provide a
work schedule, and explain how you will comply with research ethics
requirements. It will also include as an appendix a draft set of usability
survey questions or test procedures and questions. Please see the
Lll
additional guidelines (and marking rubric) below.

USABILITY STUDY PROPOSAL

Your proposal should be in memo format, with the date, sender, recipient (your marker), and an
informative subject line. (Note: “Proposed usability study” is not quite informative enough for the
subject line.) Your proposal should contain the following sections (with bolded headings):
Introduction: In this opening section (which does not need to have a heading),
▪ introduce the focus, purpose, and scope of your proposed study. (Will you evaluate all the
pages and functions of the website or focus just on certain pages and functions?)
▪ indicate the audience and purpose(s) of the website you plan to study. If you wish, you can
describe the site and its host organization in a separate Background section or subsection, but
don’t go into unnecessary detail. (For example, your reader doesn’t likely need to know what
year the organization was founded in.) Be sure to cite the source of any information you take
from the website itself, and do not quote without using quotation marks.
 ▪ justify your proposed study in a sentence or two, explaining why the usability of this particular
website is important. (What might happen if the site has poor usability?)
▪ mention the key deliverable of your project (i.e., a formal report presenting your findings and
recommending potential improvements to the site).

Criteria for evaluation: Briefly outline your criteria for evaluating the website, focusing on 3 or 4 key
aspects of the site and citing sources as you define your criteria. The criteria might include some of
the following and should reflect the focus of your individual heuristic analyses with any additional
criteria you wish to add:
▪ the clarity, readability, helpfulness, or completeness of the content
▪ the organization of information (information architecture)
▪ the navigation tools and links
▪ the page layout and design elements such as use of headings and visuals
▪ the usability of interactive features on the site (e.g., membership forms)

Research methods: If you haven’t already done so in your proposal, briefly mention that you have
completed heuristic analyses of the website. Then go on to describe your proposed research
methods in detail, focusing on ONE of the following two approaches.
● For an in-person usability test, indicate how you will recruit one or two testers from this
section of Coms 363, what tasks you will have them do, where you will conduct your test, and
what data you will gather. Your proposal should include an appendix with a copy of your
usability test instructions and any related survey or interview questions.
▪ For a web-based usability survey, explain that your survey will be sent by email only to
students in L1 of Coms 363. To administer your survey, use SurveyMonkey (at
SurveyMonkey.com), a tool that allows you to easily develop and administer a 10-question
survey for free, with the results automatically and anonymously aggregated. Append a copy of
your draft survey to your proposal.

Research schedule (timeline): Include a brief schedule outlining your key target dates to complete
steps in your usability study research and report writing.

Research ethics compliance: Address research ethics concerns in a separate (sub) section, in which you
state that you will keep participants’ names anonymous, will treat all research materials as
confidential, and will put letters of consent (for in-person testers) into a sealed envelope to be given
to the instructor for safekeeping in a locked office for two years following completion of your study.
● For an in-person usability test, state that you will have your tester sign a letter of consent prior
to the test. (Find the letter of consent template under the ETHICS content-area link in D2L,
adapt it for your use, and append a copy to your proposal.)
● For a web-based usability survey, note that the SurveyMonkey tool will not collect
participants’ names or email addresses and that the results will be automatically and
anonymously aggregated. State that you will include the following paragraph in the preamble to
your web-based survey (and be sure to do so):
This anonymous survey is being conducted as part of a student project in Coms 363, a course in
Professional and Technical Communication at the University of Calgary. This survey is
being administered via SurveyMonkey, an online survey tool that does not track
respondents’ names or e-mail information. No names will be connected with any of the
responses collected. By completing and submitting this survey, you are indicating your
informed consent to participate in this research project. You may withdraw at any time
while completing the survey simply by exiting the survey. Since results are not tracked,
you cannot withdraw your participation once you hit submit. If you have any questions
 about this survey, please contact my instructor, Jo-Anne Andre (403-220-7429; email
[email protected]). Thank you.

Conclusion: Conclude by briefly summing up the benefits of your proposed usability study.

References: Include an APA- or IEEE-style reference list, including entries for the website you will
evaluate and for a minimum of three published sources, at least two of which are drawn from the
Oct. 13 and 20 course readings; all the research sources you cite in your proposal should have a
corresponding entry in your reference list and vice-versa.

Appendices: Your proposal should include the following appendices, each of which should be
titled, should begin on a new page, and should be referred to in your proposal. If you have more
than one appendix, number them Appendix A, B, etc., corresponding to the order in which you
refer to them in your proposal:
● Draft usability survey OR
draft procedures and interview (or survey) questions for in-person testers
● Draft letter of consent (required only if you are doing in-person usability testing).
 Marking guidelines for the Coms 363 proposal:
n “A”range grade proposal reflects excellent work. It will
roposal • adhere to all assignment guidelines, including length requirements
organization & • include all necessary information and proposal elements (e.g., Introduction, Criteria
development for Evaluation, Research Method, Ethics Compliance), Conclusion, & References)
• effectively use secondary research in the discussion to make a clear and persuasive
case for the proposed research criteria and method
• be organized effectively, with informative headings and subheadings (and helpful use
of lists or visual elements where appropriate)
ocumentation of • include properly formatted APA- or IEEE- style in-text citations where needed and a
sources & properly formatted reference list in APA or IEEE style
Quality of the ● have a professional tone and a clear, concise writing style
writing ● be free (or nearly free) of common writing errors
verall, an “A” proposal will generally exceed the reader’s expectations. It will impress the reader in its
research, development, organization, documentation of sources, and clear, concise writing. An A+
proposal will demonstrate outstanding work in every respect. An A- proposal will fall just short of
excellent owing to very minor weaknesses.

A “B”-grade proposal reflects very good work. It will be clearly organized and contain enough
information for the reader, but it will exhibit serious weaknesses EITHER in the proposal organization
& development OR in the quality of the writing and documentation of sources. For example, the
evaluation criteria or research methods may be underdeveloped, the writer may cite only one or two
research sources, or the task schedule or conclusion may be missing. Even if the organization and
development are sound, an accumulation of three or four serious writing errors per page, on
average, will push a proposal into the B range.
A “C”-grade proposal reflects just satisfactory work. It will barely fulfill its purpose owing to
serious weaknesses in BOTH the proposal organization & development AND the documentation of
sources & quality of the writing. For example, the proposal may contain too little information for the
reader to know exactly what the writer is proposing to do, secondary sources may be absent, or the
proposal may fail to include a draft usability survey, usability test instructions, or letter of consent.
Even if all the required information is provided, a proposal may earn a grade in the C range if there is
an accumulation of six or seven serious editorial problems per page.

A “D”-grade proposal reflects less than satisfactory work. It will show some effort toward meeting
the assignment requirements but will generally fail to fulfill its purpose because it is uninformative,
confusing, or substantially incomplete. The quality of the writing is likely to be poor, and the
proposal will give the impression that the writer wrote it at the last minute without doing the
required research and without reading the assignment guidelines. (A proposal graded in the D range
will likely be sent back for a revision before approval can be granted for the research to proceed.)

A “F”-grade proposal reflects unacceptable work. It will fail to complete the assigned task OR show
evidence of plagiarism (using sources without citation or using wording from a source without
quotation marks). For example, a proposal unrelated to the usability study project would earn an F,
as would one that does not include a research method section for the usability study.