A System Review On Measuring and Evaluating Web Usability in Model Driven Web Development
A System Review On Measuring and Evaluating Web Usability in Model Driven Web Development
I. INTRODUCTION
Software systems playa crucial role in our daily life. Software systems are developed to provide particular functionality that
reduces human effort and supports a person to do a certain task in a specific context [22].Software development is classified with
four concerns:
• Features
• Time to market
• Cost
• Quality
What is Quality?
"Quality comprises all characteristics and significant features of a product or an activity which relate to the satisfying of given
requirements". - (DIN 55350-11) [27]
"The totality of characteristics of an entity that bear on its ability to satisfy stated and implied needs" - (ANSI standard)
Importance of usability is,it expresses the relationship between software product and "external" quality [20]. Usability
was derived from the term "user friendly" but is often associated with terms such as usefulness, ease of use, quality of use etc.
According to different authors usability can be defined as
The extent to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency and
satisfaction in a specified context of use. (ISO 9241-11)
The capability in human functional terms to be used easily and effectively by the specified range of users, given specified training
and user support, to fulfill the specified range of tasks, within the specified range of scenarios (Shackel, 1991)
Usability Models:
Shackel model (1991), defines usabilitybased on four scales namely: Effectiveness, Learn ability, Flexibility and Attitude [1].
These scales further decomposed, for example learn ability is decomposed into measurable indicators such as retention and time
to learn [30].
Eason’s Model (1984), defines usability with task, user, and system characteristics and user reaction.
ISO 9214-11(1998), this model explains usability with the effectiveness, efficiency and satisfaction.
ISO 9126(2001), this model mentioned usability as understandability, attractiveness, learn ability, operability and usability
compliance.
Product oriented design:In this design usability considered as a product attribute [5]. Some examples are:
• Design Heuristics
• Interface Guidelines
• Interaction Design patterns
Process oriented design: To make software usable it collects functionality by analyzing users and tasks. For analysis bundle of
techniques are used, the techniques are:
According to studies of software engineering, several organizations spend a large amount of time and money on fixing usability
problems during last stage of development. According to studies of pressman, 80% of total maintenance costs are related to
problems of the user with system. Among these costs, 64% are related to usability problems. Lederer and Prassad, 1992 stated
that large amount of maintenance costs is spent on dealing with usability issues.
II. SOFTWARE ARCHITECTURE
In recent years the software engineering community has come to the understanding that the software architecture is an important
instrument in the fulfillment of quality requirements [3] [15].
The structure or structures of the system, which comprise software elements, the externally visible properties of those elements,
and the relationships among them. (Bass et al, 1998)
The fundamental organization of a system, embodied in its components, their relationships to each other and the environment,
and the principles governing its design and evolution (IEEE, 1998)
According to Bonnie John et al., the architectures of the 1980s and 1990s usability were considered as a property of the
presentation of information. To made usability both software architects and usability professionals understood the relationship
between architecture and usability early [21] [22] .According to Bass et al, 1998 software architecturedefined as
• Communication
• Evaluation
• Design:
Software architecture objective is to fulfill the requirements which can be classified into two types [16]:
• Functional requirements define what the system needs to do (e.g. "backup my hard drive")
• Non Functional requirements describe how the system will do it (e.g. "backup my hard drive without crashing").
Nonfunctional requirements are often associated with software quality requirements e.g. the "ilities" such as usability,
maintainability, etc.
“Usability engineering, also known as human-computer interaction engineering, is a discipline concerned with the design,
evaluation and implementation of interactive computing systems for human use and the study of major phenomena surrounding
them”.
Usability engineering has several benefits:
• Improve software
Ankita Madan, Sanjay Kumar Dubey, for all software quality models usability is an important quality factor.Lack of usability
leads to substantial monetary loss, time wastage, user dissatisfaction, staff unproductively. Usability correlates with the
functionality of the system. Usability models are: Eason’s model, Shackel model, Nielsen model, ISO 9214-11 model, ISO 9126
model [4].
Adrian Fernandez Martinez, for web applications usability is considered to be one of the most important quality factors. Majority
of web development process do not take advantage of the software artifacts produced at the design stages. Performing usability
evaluations on these artifacts can be difficult [6].
Fernando Molina, AmbrosioToval, Web Engineering development projects have grown increasingly complex and critical to the
smooth running of organizations. Recent studies reveal that a high percentage of these projects fail to attain the quality parameters
required by stakeholders. To reduce some of the quality failures detected in web engineering development projects by proposing
the consideration and evaluation of quality attributes from early stages of the development process. Once the requirements are
identified, the approach focuses on the extension of the conceptual models used by Web Engineering methodologies with the aim
of allowing the explicit consideration of usability requirements along with the evaluation of quality metrics during the design of
the system [12].
Ahmed Seffah · Mohammad Donyaee · Rex B. Kline · Harkirat K. Padda, There are many individual methods for evaluating
usability they are not well integrated into a single conceptual framework. The consolidated model is called Quality in Use
Integrated Measurement (QUIM). It also explains how this can help in developing a usability measurement theory [23].
Laura Carvajal, Ana M. Moreno, Maria-Isabel Sanchez-Segura and Ahmed Seffah, According to software development process
and software architecture used in each application design artifacts can be used. Use of guidelines reduces [1]
• Development time.
• Improves the quality of designs.
• Decreases the perceived quality.
Web applications are provided by software that is distributed, implemented in multiple styles and languages, integrates reuse and
third-party components [13]. Web applications are made up of group of people, using technologies, HTML files and programs. To
build this type of web software new and unique challenges are found by software developers and managers. For success of web
applications the important quality factors are:
1. Reliability
2. Usability
3. Security
Jakob Nielsen explained the usability of web application with the statement “On the internet, your competition is only one click
away”, means while using a web application users get frustrated when not achieving their needs quickly [6].
Constantine & ISO 9241-11 Schneider man Nielsen Preece et al., Shackel
Lockwood (1998) (1993) (1993) (1994) (1991)
(1999)
Table 1 shows, Usability explained by different authors by using principles like Efficiency in use, learn ability, rememberality,
reliability in use, user satisfaction [12]. Nielsen’susability principles can be mentioned as
Web application learn ability: Itrepresents the ease for Web users to understand the contents and services through application
and easily look for particular information by browsing.
Webapplicationsefficiency:It means users find their contents efficiently through web application.
Memorability: It implies that, users forget after some period of time how to get back his previous data, by using navigation bars
users can easily point out to their pages.
Fewerrors: It means that if users did any erroneous work, they should be able to return to their location.
User’ssatisfaction: Finally refers to the position in which users feel that they are in control with respect to the hypertext.
Offutt stated that along with reliability and security, usability is main quality factor for web application. To address web usability
issues variety of methods, techniques and tools are used.
Advantages of Web Applications:
• Ubiquity of web browsers.
• Convenience of using a web browser as a client.
• Sometimes as a thin client.
Usability issues benefiting both web developers and end-users. Some of the benefits are Reduction in cost (development,
maintenance, and support stages), Increase in user productivity, direct impact on sales and scope, better marketing.
In the field of HCI [7], usability definition proposed in ISO/IEC 9241-11(1988) standard- “The extent to which a product can be
used by specified users to achieve specific goals with effectiveness, efficiency and satisfaction in a specified context of use”. It
deals when product can be used. This standard moved to ISO/IEC 9241-210(2010) to deal when product, system or service used.
In Software Engineering, usability proposed in standard ISO/IEC 9126-1(2001), “the capability of the software product to be
understood, learned, operated, attractive to the user, and compliant to standards/guidelines, when used under specific
conditions” . It uses in the early stages of development process and the main objective of this is predicting usability problems.
V. UEMS (USABILITY EVALUATION METHODS)
Definition: It is a process that is used to produce a measurement of usability or is a systematic procedure for recording data
relating to end-user interaction with a software product or system [4].
UEMs developed to evaluate WIMP interfaces
W-Window
I-Icon
M-Menu
P-Pointing device
Evaluation Methods:
• Empirical
• Inspection
Empirical Methods: Evaluation performed by the end-users involvement. Capturing and examining usage data from real end-
users. End-users employ software product to complete a predefined set of tasks. While tester records outcomes of their work. We
can get useful information by analyzing these outcomes [4]. This information is used to detect problems.
Disadvantages:
• Empirical methods are not cost effective.
• It requires large amount of resources
• Evaluation done at last stages of web development process.
Inspection Methods: Evaluation performed by evaluators and designers no need of end-users involvement. Reviewing the
usability aspects of web artifacts (user interfaces) models, their conformance with a set of guidelines.
Advantages:
• It requires less resources
• Cost effective.
• Evaluation performed at early stages.
Problem:The artifacts which are delivered at requirement and design stages are not considering by many of the web development
processes. Performing usability evaluations on artifacts is difficult because of not understanding the traceability software artifacts
and web applications.
Solution: The problem is facilitated in Model-Driven Web Development (MDWD) processes in which intermediate artifacts
(models), which represent different perspectives of a Web application, are used in all the steps of the development process, and
the final source code is automatically generated from these models.
MDWD Process:
• MDWD processes break up the Web application design into three models: content, navigation and presentation.It allows
establishment of levels of abstraction.
• An MDWD process fundamentally translates models that are (i.e., Platform-Independent Models - PIMs) such as
structural models, navigational models or abstract user interface (UI) models into other models that contain specific
aspects from a specific technological platform (i.e., Platform-Specific Models - PSMs) such as concrete user interface
models or database schemas.
• To develop the source code of the final Web application (Code Model - CM) PSMs are automatically compiled.
• This approach is followed by several methods such as: OO-H (Gómez et al. 2001) or WebML (Ceri et al. 2000).
• Usability problems which are presented in final web applications are detected by considering traceability among these
models. And also provides recommendations to correct these problems in the early stages of development process.
Heuristic evaluation:
One of the most common usability guidelines is the Nielsen’s set of ten usability heuristics(Nielsen 1993) Riihiaho (2000)
suggests that the output of a heuristic evaluation be a list of usability problems. The list should state the heuristic rule that the
problem violates and the severity of the problem [25].
Figure 4showthat different number of evaluators performed heuristic evaluation to detect percentage of problems. From the curve
it is evident that reasonable results can be obtained by having only five evaluators, and certainly not less than three.
WUEP (Web Usability Evaluation Process):
To address web usability issues, usability inspection method (Web Usability Evaluation Process – WUEP) is proposed, which can
be instantiated and incorporated into different model-driven Web development processes. Inspections of the PIMs and PSMs can
provide early usability evaluation reports to identify potential usability problems that can be corrected prior to the generation of
the source code. The Web Usability Evaluation Process (WUEP) has been defined by expanding and polishing the quality
evaluation process that is proposed in the ISO 25000 standard. The aim of WUEP is to integrate usability evaluation into model-
driven Web development processes by applying a Web Usability Model as the primary inputartifact [6]. This model breaks down
the usability concept into 16 sub-characteristics and 66 measurable attributes, which are then associated with 106 measures in
order to quantify them. These measures provide a generic definition, which should be operational zed in order to be applied to
models obtained at different abstraction levels (PIMs, PSMs, and, CMs) in different MDWD processes (e.g., WebML, OO-H).
The main stages of WUEP are:
1. Establishment of the evaluation requirements stage, the evaluation designer defines the scope of the evaluation by
(a) Establishing the purpose of the evaluation
(b) Specifying the evaluation profiles (type of Web application, Web development method employed, context of use)
(c) Selecting the Web artifacts (models) to be inspected
(d) Selecting the usability attributes from the Web usability model which are going to be evaluated.
2. Specification of the evaluation stage, the evaluation designer operationalizes the measures associated with the selected
attributes in order for them to be applied to the models to be evaluated. This operationalization is to define a concept or variable
represented in the Web artifacts (modeling primitives of models or UI elements in the final Web application) so that it can be
measured or expressed quantitatively. In addition, thresholds are introduced for range of values. These thresholds provide a
usability problem classification based on their severity: low, medium, or critical [17]. It is significantly note that the
operationalization should be performed once within a specific model-driven Web development method, and can be reused in
future evaluations that involve Web applications developed using the same method.
3. Design of the evaluation stage, the template for usability reports is defined and the evaluation plan is elaborated (e.g., number
of evaluators, evaluation constraints).
4. Execution of the evaluation stage, the evaluator applies the measures to the selected Web artifacts (i.e., models) in order to
detect usability problems by considering the rating levels established for each measure.
5. Analysis of changes stage, the Web developer analyzes all the usability problems in order to propose changes with which to
correct the affected artifacts from a specific stage of the Web development process. The changes are applicable to the previous
intermediate artifacts (i.e., PIMs, PSMs and model transformations if the evaluation is performed on the final Web user interface).
WebML specifications are independent of both the client-side language used for delivering the application to users, and of the
server-side platform used to bind data to pages, but they can be effectively used to produce a site implementation in a specific
technological setting. It guarantees a model-driven approach to Web site development, which is a key factor for defining a novel
generation of CASE tools for the construction of complex sites, supporting advanced features like multi-device access,
personalization, and evolution management. The WebML language and its accompanying design method are fully implemented in
a pre-competitive Web design tool suite, called Toriisoft [23].
(1) Structural Model, shows the data content of the site, in terms of the relevant entities and relationships
(See Fig 3). WebML does not usingany other language for data modeling, but is compatible with classical notations like the ER
model, the ODMG object-oriented model, and UML class diagrams [6].
(2) Hypertext Modeldepicts one or more hypertexts that can be released in the site. Each different hypertext defines a so-called
site view (see Fig 4). Site views descriptions in turn consist of sub-models are
Composition Model defines which pages compose the hypertext, and which subject units make up a page. Six types of content
units can be used to compose pages: data, multidata, index, and filter, scrolled and direct units.
Navigation Modelshows how pages and subject (content) units are linked to form the hypertext. Links are either non-contextual,
when they connect semantically individual pages, or contextual, when the content of the destination unit of the link depends on
the content of the source unit.
Sirius provides a framework for heuristic evaluation but it can be framed within a multi-stage global evaluation process of a
website. This process starts with (as shown in Fig.) an accessibility evaluation before the usability evaluation. This is a decision
based on the results of different authors, who think that increasing the accessibility of a website also increases its usability.
Therefore, first, a review of accessibility is performed both automatically and manually. [2]This review is done according to the
W3C’s Conformance evaluation method. Then the heuristic evaluation of usability proposed here (Sirius) would be performed.
Finally, this would be completed with an evaluation model with users, considering the critical or relevant tasks, and involving
users from all the target audiences of the site. This is included as skills; aims and equipment of target groups of users, context of
use, different services and tasks, etc. also affect the level of usability of a site. Should the site have to be redesigned because of
the results of each evaluation, the reviewing process would have to be performed again.
IX. CONCLUSION
In the software development, to help software engineers set of usability guidelines are defined. This review focuses on usability
evaluations; usability has been evaluated taking into account the user’s satisfaction by using different evaluation methods. For
specifying complex websites at the conceptual model a WebML is presenting. Web applications usability is evaluating using
WUEP(Web Usability Evaluation Process) method. To perform expert evaluations by considering type of website, Sirius
framework is using. The results are in the progress, by using these methods and framework we can detect and predict usability
problems in the early stages of development and we can provide recommendations to solve these problems.
REFERENCES
[1] Laura Carvajal et al., “Usability through Software Design”, in IEEE Transactions on Software Engineering, 2013.
[2] M. Carmen Suárez Torrente et al., “Sirius: A Heuristic-Based Framework for Measuring Web Usability Adapted to the Type
of Website”, in The Journal of Systems and Software, pp.649– 663, 2013.
[3] Naki Chiam and Masitah Ghazali, “A Systematic Literature Review on The Role of Usability in Software Engineering
Education”, in International Conference on Information and Knowledge Management (ICIKM 2012) vol.45 pp.170-174,
2012.
[4] Ankita Madan “Usability Evaluation Methods: A Literature Review”, in International Journal of Engineering Science and
Technology (IJEST), ISSN: 0975-5462 Vol. 4 No.02, pp. 590-599, February 2012.
[5] Laura Carvajal, “Usability Oriented Software Development Process”, Thesis, 2012.
[6] Adrian Fernandez Martinez,” A Usability Inspection Method for Model Driven Web Development Process”, Thesis,
November 2012.
[7] Anirudha N. Joshi, “Integration of Human-Computer Interaction Activities in Software Engineering for Usability Goals
Achievement”, Thesis-Department of Computer Science and Engineering Indian Institute of Technology Bombay, pp.1-287,
2011.
[8] Anirudha Joshi,”Measuring effectiveness of HCI integration in software development processes”, in The Journal of Systems
and Software, 2010 Elsevier, pp.1-14, 2010.
[9] Fernando Molina, AmbrosioToval,” Integrating usability requirements that can be evaluated in design time into Model
Driven Engineering of Web Information Systems”, Advances in Engineering Software, 1306-1317, 2009.
[10] Jose Ignacio Panach “Understandability Measurement in an Early Usability Evaluation for Model-Driven Development: An
Empirical Study “, in International Symposium on Empirical Software Engineering, pp.354-356, 2008.
[11] Pedro Campos and Nuno Jardim Nunes,” Practitioner Tools and Work styles for User-Interface Design”, in IEEE
SOFTWARE, pp.73-80, J an n u an r y / F e b r u an r y 2 0 7.
[12] Ahmed Seffah,” Usability measurement and metrics: A consolidated model “, in Springer- Software Qual J (2006) 14:
pp.159–178, 2006.
[13] Maristella Matera,” Web Usability: Principles and Evaluation Methods”, in Web Engineering, Springer, 2006.
[14] Jeff Sauro & Erika Kindlund,” Making Sense of Usability Metrics: Usability and Six Sigma”, in UPA Conference 2005, pp.
1-10, 2005.
[15] K.K.Basheer, A. Rama Mohan Reddy, C. ShobaBindu “Systematic Use of Architecture Patterns to Achieve Non-Functional
Requirements “National Conference on Data Modeling, Image Analysis and Pattern Recognition (DMIAPR) - 2011”,
organized by GITAM, University, Vishakhapatnam on 13th to 14th September, 2011
[16] Proefschrift,” Software Architecture Analysis of Usability”, in Thesis-Rijksuniversiteit Groningen, pp.1-240, 2005.
[17] Atterer, R., Schmidt,” Adding Usability to Web Engineering Models and Tools”, in Proceedings of the 5th International
Conference on Web Engineering, 36-41, 2005.
[18] Leena Norros, Paula Savioja,” Usability Evaluation Of complex Systems- A literature review”, in STUK-YTO-TR 204-
study, pp.1-44, APRIL 2004.
[19] Jeff Tian, “Quality-Evaluation Models and Measurements”, in IEEE Software Published By the IEEE Computer Society pp.
84-91, 2004.
[20] Jeff Offutt, “Quality Attributes of Web Software Applications”, in IEEE Software special issue on Software Engineering for
Internet Software, 25-32, 2002.
[21] Bonnie E. John and Len Bass, “Usability and Software Architecture”, in Behavior& Information Technology, Vol. 20, NO. 5,
pp. 329-338, 2001.
[22] Xavier Ferré and Natalia Juristo et.al, “Usability Basics for Software Developers”, in IEEE Software, pp.22-29, January
/February 2 00 1.
[23] Manuel F. Bertoa and Antonio Vallecillo, “Usability Metrics for Software Components”, in White paper, pp.1-10, 2000.
[24] Stefano CeriŁ, PieroFraternali, Aldo Bongio,” Web Modeling Language (WebML): a modeling language for designing Web
sites”, in 9th International World Wide Web Conference, 137-157, 2000.
[25] Andreas Lecerof and Fabio Paternò,” Automatic Support for Usability Evaluation”, in IEEE Transactions on Software
Engineering, Vol. 24, NO.10, pp.863-888, October 1998.
[26] K.K.Baseer, A. Rama Mohan Reddy, C. ShobaBindu, SaiBharath “Framing the Principles Involved in the Design of Software
Architecture to achieve Quality Attributes”, 3rd International Conference on Science, Engineering and Technology (SET)
November 17-18, 2011, @ VIT University, Vellore-632 014, TN, India. Paper ID: 113, Vol. 5, pp. 739-746
[27] Nigel Bevan, “Measuring Usability as Quality of Use”, in Software Quality Journal, 4, pp. 115-150, 1995.
[28] Basili, V., Rombach, H, “The TAME Project: Towards Improvement – Oriented Software Environments.”, in IEEE
Transactions on Software Engineering, vol 14, Issue 6, pp. 758-773,1988.
[29] K.K.Baseer, A. Rama Mohan Reddy, C. ShobaBindu “A Survey of Synergistic Relationship: Scenarios, Quality Attributes,
Patterns, Decisions, Reasoning Framework”, International Journal of Web Technology, Vol 01, Issue 02, December 2012:
ISSN: 2278-2389
[30] SaiBharathKadati, K. K. Baseer, A. Rama Mohan Reddy & CH. Gowthami, “A Naive Based Approach for Mapping Two
ADL Models”, Global Journal of Computer Science and Technology Software & Data Engineering Volume 12 Issue 11
Version 1.0 June, Year 2012 Type: Double Blind Peer Reviewed International Research Journal Publisher: Global Journals
Inc. (USA) Online ISSN: 0975-4172 & Print ISSN: 0975-4350