Master in Technical Communication and Localization (TCLoc)

Architecting Design Systems for Web Accessibility

A Comparative Study of Web Accessibility Guidelines in Design Systems

A thesis submitted in partial

fulfillment of the requirements for

the degree of Master of Arts

By

Jeanella Klarys PASCUAL

[email protected]

2023

Thesis Director
DR. DREW EISENHAUER
Abstract
The challenges of web accessibility have spanned decades since the inception of the Web and
remain a relevant research topic in the current climate of accessibility laws, such as the Americans
with Disabilities Act and the European Accessibility Act, being enforced. The World Wide Web
Consortium (W3C), the organization responsible for drafting web standards, published several
guidelines for building accessible web applications, one of which is the Web Content Authoring
Guidelines (WCAG).

Design systems, a recent trend in software engineering, have become a popular way for teams to
structure and align design and user experience guidelines, notably web accessibility. This study
analyzes a sample of 5 design systems derived from the catalog published by Open UI, a chapter
under the W3C, which aims to standardize design systems. A comparative analysis of the design
systems on how they integrate accessibility guidelines was performed in 2 aspects: the 4
components of Information Architecture, and quality of alignment with the WCAG.

The results of the analysis could provide useful knowledge for software engineering teams looking
to improve accessibility best practices in their teams or explore several benefits of design systems in
aligning goals within an organization.

TCLoc Master’s Thesis 2023 G2 Page 2 of 43 Jeanella Klarys Pascual

Table of Contents
List of Abbreviations..........................................................................................................................................................4

1. Introduction ....................................................................................................................................................................5

1.1 Problem Statement.....................................................................................................................................................5

1.2 Research Question and Objectives ............................................................................................................................6

1.3 Scope and Limitations ...............................................................................................................................................6

2. Review of Related Literature ........................................................................................................................................7

2.1 History of Web Accessibility......................................................................................................................................7

2.2 Challenges in Web Accessibility................................................................................................................................8

2.3 The Role of Design Systems.....................................................................................................................................10

2.4 Components of Information Architecture ................................................................................................................11

2.5 Challenges in Building Design Systems ..................................................................................................................12

2.6 Summary ..................................................................................................................................................................14

3. Methodology .................................................................................................................................................................15

3.1 Sampling ..................................................................................................................................................................15

3.2 Keywords referring to Accessibility ........................................................................................................................18

3.3 Modeling of the Design System IA...........................................................................................................................18

3.4 Analysis ...................................................................................................................................................................19

4. Results ...........................................................................................................................................................................20

4.1 Four Components of IA ...........................................................................................................................................20

4.2 Alignment with the WCAG ......................................................................................................................................29

5. Discussion ......................................................................................................................................................................32

6. Conclusion .....................................................................................................................................................................34

7. References .....................................................................................................................................................................35

Appendix A .......................................................................................................................................................................40

Appendix B........................................................................................................................................................................41

TCLoc Master’s Thesis 2023 G2 Page 3 of 43 Jeanella Klarys Pascual

List of Abbreviations

A11y Accessibility
ADA Americans with Disabilities Act
ARIA Accessible Rich Internet Applications
ATAG Authoring Tool Accessibility Guidelines
CMS Content Management Systems
IA Information Architecture
NPM Node Package Manager
SSO Single Sign-on
UI User Interface
UX User Experience
W3C World Wide Web Consortium
WAI Web Accessibility Initiative
WCAG Web Content Authoring Guidelines

TCLoc Master’s Thesis 2023 G2 Page 4 of 43 Jeanella Klarys Pascual

1. Introduction
In the past decade, design systems have become an essential tool for software companies in scaling
their products while achieving a cohesive user experience (Churchill, 2019, p.1). Design systems
provide a common visual language and help streamline processes between different stakeholders,
such as designers, developers, and content writers (Vesselov & Davis, 2019, p.35). They are usually
documented in the form of a centralized website for ease of access and iterative modification.
Several large-scale software companies, such as Google, Microsoft, Apple, and IBM, have made
their own design systems publicly available, encouraging many software companies to build their
own. This move of publicizing guidelines is a response to the web becoming more global and
requiring a more democratic and inclusive user experience (Barbareschi et al., 2021, p.1).

However, several studies show that web accessibility is still a current challenge. In 2022, WebAIM
analyzed the top 1 million pages, reporting 50.8 accessibility failures per page on average
(WebAIM, 2022). Factors, such as different business priorities, lack of awareness, and lack of
robust development and testing environments for developers, are the main causes (Patel et al., 2020,
p.6).

1.1 Problem Statement

To solve the web accessibility challenge, the Web Accessibility Initiative (WAI) created the Web
Content Accessibility Guidelines (WCAG), a public repository of guidelines for web authors, such
as developers, to build accessible websites (Web Accessibility Initiative [WAI], 2008). However,
developers have expressed a lack of effective resources in understanding such guidelines and testing
compliance (Paternò et al, 2020, p.6). Despite the availability of tools to measure WCAG
compliance, they still report unreliable results, leading to a lack of trust in such tools (Manca et al.,
2022, p.2).

The purpose of a design system in centralizing a business’s user experience standards can be

leveraged in setting accessibility guidelines in one place, increasing shared knowledge of building
WCAG-compliant user interfaces within the team. The potential of design systems for automation
may also help streamline WCAG compliance into several development process phases. The most
widely adopted design systems have been built by software companies with a huge global user base
and have the resources to invest in accessibility (Yew et al., 2020, p.7). By following best practices
proposed by the most widely adopted design systems, software engineering teams may benefit from
solutions that have been tried and tested. Those with limited time and budgets may save costs and
focus on building their products (Häger, 2021, p.4).

TCLoc Master’s Thesis 2023 G2 Page 5 of 43 Jeanella Klarys Pascual

While no official standards in design systems have yet been set, initiatives such as the Open UI
have started proposing guidelines to solve the vast differences in information architecture (IA) in
hundreds of public design systems (Open UI Community Group, 2023). Standards in design
systems IA will provide developers with the resources to structure design systems that support
accessibility.

1.2 Research Question and Objectives

What are the differences in the information architecture of leading design systems in how they
document WCAG compliance?

Objectives:

1. Determine the different ways design systems are structured to incorporate web accessibility
guidelines.

2. Evaluate the different design systems based on how they align their guidelines with WCAG
criteria.

The results of this research will offer valuable insights to software engineering teams wishing to
architect their design systems to include web accessibility.

1.3 Scope and Limitations

The sample was based on the catalog provided by the Open UI Community Group (2023b) and does
not include other sources. The sample design systems were only analyzed on the accessibility
guidelines: WCAG 2.1 Success Criteria and Sufficient Techniques. Due to time constraints, the
methodology employed only manual techniques and used only the search keyword “accessibility”.

TCLoc Master’s Thesis 2023 G2 Page 6 of 43 Jeanella Klarys Pascual

2. Review of Related Literature
This literature review will examine the current state of knowledge on web accessibility and the
challenges software practitioners face, notably during authoring and testing. This review will also
explore the role of design systems and the current state in which web accessibility is articulated in
design systems. All related literature has been compiled from peer-reviewed journals, published
books, official government websites, and the WAI website, except for four sources (Deque
Systems, Inc. 2023; Google, 2023; Penamora, 2023; Sparkbox, 2022). By synthesizing existing
research and identifying key themes and trends, this review aims to provide an overview of the
opportunities for design systems to solve some of the current web accessibility challenges.

2.1 History of Web Accessibility

Web Accessibility aims to assist users with disabilities with ease of access to the web, which means
websites, tools, and technologies should be designed with them in mind (WAI, 2008).

Tim Berners-Lee, the founder of the Internet, during his keynote speech at the 1994 Second
International World Wide Web conference, mentioned that "the power of the web is in its
universality. Access by everyone regardless of disability is an essential aspect” (Tim Berners-Lee:
The Future of the Web (WWW94), 1994). This paved the way for defining accessibility guidelines
that later became the Web Content Accessibility Guidelines. 9 years later, in 2008, WCAG 2.0 was
released, which went beyond HTML standards, but all digital content. Several minor versions have
been released (WCAG 2.1 in 2018 and WCAG 2.2 in 2022) and version 3.0 is in progress
(Penamora, 2023).

The Web Accessibility Initiative (WAI), an initiative of the World Wide Web Consortium (W3C),
is responsible for defining web accessibility standards and supporting and educating people on them
(WAI, 2008). WCAG is one of several components of web accessibility. As opposed to WCAG
which are guidelines for consumer content, such as websites, Authoring Tool Accessibility
Guidelines (ATAG) cater to tools, such as Content Management Systems (CMS), word processors,
photo editors, and video editors (WAI, 2022b).

Several initiatives have also been formalized to address the need for web accessibility. In 1990, the
Americans with Disabilities Act (ADA) was signed into law (ADA.gov, 2023), with Section 508
specifically containing accessibility guidelines relevant to electronic and information technology
(Section 508.gov, 2022). The EU’s Web Accessibility Directive of 2016, which has been derived
from the United Nations (UN) Convention on Persons with Disabilities (The Office of the High
Commissioner for Human Rights, 2006), covers digital products and services such as computers,

TCLoc Master’s Thesis 2023 G2 Page 7 of 43 Jeanella Klarys Pascual

operating systems, and smartphones (European Accessibility Act, 2019). In 2019, the
complementary European Accessibility Act took effect, which requires companies to implement the
guidelines by 2025.

However, several studies show that web accessibility is still a current challenge. The issue of web
accessibility has been raised in several studies in the past (Kelly et al., 2007; Rosson et al., 2005),
but remains a relevant topic. During AxeCon 2023, Kumar (2023) mentioned that 37% of lawsuits
in the year 2022 were comprised of American Disability Act-related lawsuits. In the same year,
WebAIM has done an analysis of the top 1 million pages which reported 50.8 accessibility failures
per page on average (WebAIM, 2022).

2.2 Challenges in Web Accessibility

2.2.1 Authoring Challenges in Web Accessibility

A study explored the possible factors influencing the lack of accessibility in websites, and one area
of focus is the challenges software practitioners face.

While the W3C provides official and publicly accessible documentation on standards (WCAG),
containing an index of Success Criteria and different techniques to fulfilling them (WAI, 2022c).

The main causes are a lack of awareness of what accessibility means, different business priorities,
and a lack of robust development and testing environments for developers (Patel et al., 2020, p.4).
The authors expound that lacking training or hiring accessibility experts leads to a lack of
knowledge in interpreting accessibility guidelines.

Rosson et al. (2005, p.531) also noted that firms are generally cautious in investing in accessible
design, as this might not translate to a sizeable return on investments. Without support from
businesses, accessibility becomes an afterthought, and software practitioners who value
accessibility will feel disempowered.

Kumar (2023) confirms such causes by noting a lack of quality knowledge in software practitioners,
inconsistent test results, and fragmented levels of automation for validation.

A consensus on the possible solutions is found in several papers, noting the inclusion of
accessibility in university curricula, offering technical workshops on accessibility, and investing in
developer tools catered to accessibility (Patel et al., 2020, p.6; Martin et al., 2022, p.5).

TCLoc Master’s Thesis 2023 G2 Page 8 of 43 Jeanella Klarys Pascual

2.2.2 Testing Challenges in Web Accessibility

The second area of web accessibility issues that is worth exploring is testing, which serves to
validate that the design and development phases comply with accessibility guidelines.

The WAI has made available a huge repertoire of accessibility evaluation tools allowing users to
filter by device, accessibility guidelines, industry, or government standards (WAI, 2022b).
However, despite the availability and abundance of resources, this still does not resolve the issue of
lack of awareness among developers.

One reason is the usability of the accessibility evaluation tools themselves. In a survey conducted
by IBM to observe web accessibility skills in their developer teams, developers found that “they are
hampered by incomplete skills, new standards, incomplete browser implementations, and
inconsistencies in assistive technologies. Tools that should help developers are often unclear,
cumbersome, and incomplete when compared to the standards to be tested.” (Trewin et al., 2010,
p.10).

A recent study exploring the issue of lack of transparency in accessibility evaluation tools
hypothesizes that the incoherent results confuse developers and dissuade them from adopting said
tools (Manca et al., 2022, p.1). The researchers used a combination of comparative analysis,
surveys, and user tests on a set of tools proposed by the WAI. They surveyed 139 users, with a
significant number of accessibility experts (36.24%) and web developers (24.64%). The study
concludes the following characteristics are what make an accessibility tool useful: providing
concise and actionable instructions, accurately matching the needs of the different end-users, and
limiting irrelevant details.

While some phases of testing can be automated using software evaluation programs, human testers
are required for part of or the entire test in other cases. It is essential to the goals of the WAI, stating
that accessibility is about people (WAI, 2023). However, the small sample size and diverse array of
disabilities pose challenges in recruiting real human testers, whose feedback is essential for building
accurate adaptive technologies (Petrie et al., 2006, p.1134).

Even the effectiveness of WCAG is subject to evaluation, whether it is sufficient and clear for
software practitioners to carry out accessibility implementation. Many studies confirm that the
guidelines themselves can be regarded as a barrier and must be improved (Hayfa et al., 2016), with
differing interpretations of WCAG even among expert testers and developers with web accessibility
knowledge. One study considered it complex and ambiguity in some of its passing criteria can be
subject to misinterpretation (Alonso et al., 2010, p.8). Another study shows that even experts can be
TCLoc Master’s Thesis 2023 G2 Page 9 of 43 Jeanella Klarys Pascual
subject to the Evaluator Effect, where for experts, half of the success criteria fail to meet the 80%
agreement threshold, produce 26% incorrect ratings and 20% false positives, and miss 32% of the
true problems (Brajnik et al., 2012, p.8). Test results for inexperienced testers and developers
produce higher false positives.

Cooper (2016, pp.1-2), a member of W3C, suggested a needs-based approach to addressing web
accessibility issues of the 2020s. In his paper, he wrote that comprehensive documentation of
accessibility is essential to advancing the guidelines, whose first step is collating all the known
needs.

2.3 The Role of Design Systems

Design Systems have recently become an emerging concept in the software industry.

Figure 1. Google Trends on the keyword “Design System” (Google, 2023)

Searching on Google Trends for the keyword “Design System” (parameters: Worldwide, between
January 1, 2013, and March 18, 2023, on all categories), we see a significant upward trend starting
from the year 2018.

The constant growth of the web has invited more users to adopt it. The Web expanded to several
platforms such as mobile phones and tablets and has become more international, catering to a more
diverse set of users. This translates to the specialization of roles, where software companies are now
comprised of dedicated teams or experts, each with a set of domain skills, such as design,
development, content writing, translation, and testing (Yilmaz et al., 2012). This introduces the risk
of company silos forming, where each team’s own working methods could deviate from the others,
a common phenomenon in large enterprises. Modern software needs to address a variety of use
cases while remaining robust to deliver a cohesive user experience. Some of the solutions to
mitigate company silos are centralizing domain knowledge and aligning teams to company values.
All of these trends and requirements explain the recent and natural proliferation of Design Systems
(Vesselov and Davis, 2019, p. 5; Lamine and Cheng, 2022, p.2).
TCLoc Master’s Thesis 2023 G2 Page 10 of 43 Jeanella Klarys Pascual
However, based on related literature and research, Design Systems still carry some ambiguity in
their definition despite several studies showing that the majority of the tech industry sees their
importance (Ðukic, 2020, p.8; Kilrain & Edelberg, 2020, p.1; Konaté, 2018, p.58; Yew et al., 2020,
p.2). This interesting overlap reveals that companies are convinced from having observed the
benefits reaped by other companies who have successfully adopted Design Systems, yet have a
difficult time bootstrapping due to the lack of industry standards.

2.4 Components of Information Architecture

Along with web accessibility and design systems, information architecture (IA) emerged as a
discipline during the growth of the World Wide Web, as explored in the book “Information
Architecture” by Rosenfeld & Morville (2002, p.4). The authors relate IA to “the design of shared
information environments and the support of usability and findability, in the face of multiple
experiences and information overload in the digital era”. These challenges are not only limited to
end users but also to organizations, where several disciplines coexist and use the same channels for
communicating. Their book explores several strategies for applying IA to achieve information
coherence across an organization.

According to them, IA is made up of four components:

1. Organizing systems – Categorizing information

2. Labeling systems – Representing information with terminology

3. Navigation systems – Moving through information

4. Search systems – Methods for searching for information

As websites are ever evolving, the longevity of a website depends on the robustness of its IA, in its
ability to accommodate and adapt to changes (Garrett, 2011, 91). The same ever-evolving nature
can be said of design systems (Lamine & Cheng, 2022, p.18; Suarez et al., 2019), which allow
flexibility and scalability but involves maintenance. Vesselov & Davis (2019, p. 68) state that
creating a predictable architecture is crucial to the success of a design system. While the book does
not explicitly mention IA, it employs the four components of IA through:

• “categorizing to improve discoverability (p. 69)” (Organizing Systems)

• “consistent usage of the terminology defined by the team” (p. 16)” (Labeling systems)

TCLoc Master’s Thesis 2023 G2 Page 11 of 43 Jeanella Klarys Pascual

 • “navigation that is easy for users to recall and that allows them to perform familiar actions”
(p. 82)” (Navigation Systems)

• “vastly improving the usability of a design system by including a search functionality (p.
88)” (Searchability)

This makes a case for IA's role in relaying web accessibility guidelines in design systems.

2.4.1 Information Architecture of Design Systems

Ongoing research strives to define design systems and the parts that constitute them, as supported
by related literature in Lamine and Cheng’s paper (2022). Sparkbox (2022) surveyed 219
respondents who work with design systems on what elements constitute their design systems. Those
that scored beyond 80% were typefaces, colors, and styles (98%), components (95%), and
component documentation (85%). In a study by Werle (2020, p.41), where one of the main
objectives is to define the parts of a design system, he sent out a survey asking respondents to list
the elements of a Design System and score them by importance. Each element was grouped into
categories, each with an average score. Although the element “Buttons” was rated as the most
important (98.5%), a broader look at the categories resulted in “Principles" and “Patterns" scoring
the highest. Design systems should not be limited to style guides or component libraries. Still, they
should represent principles and be a collaboration tool that drives efficiency and better user
experience (Vesselov & Davis, 2019, p.xvi).

Currently, a lot of reviewed research can only synthesize the consensus of industry practitioners,
while no official standards have yet been made.

Aside from the challenges of naming and structuring, there are also challenges behind the logistics
of building design systems.

2.5 Challenges in Building Design Systems

Building design systems is challenging and requires a lot of resources. In startups or small teams, it
may not be viable due to time, cost, and resource constraints (Häger, 2021, p.10). This was
confirmed by research conducted by Stepanova, where she interviewed 20 UI/UX practitioners
from different companies on the impact of design systems in aligning work between interface
designers and developers (Stepanova, 2021, p.40). Organizational maturity and stakeholder buy-in
decide when it is appropriate to invest in design systems (Vesselov & Davis, 2019, p.47). The
authors state the importance of solving the initial issues of getting buy-in from stakeholders
(employees, the organization, and users), before starting work with design systems.
TCLoc Master’s Thesis 2023 G2 Page 12 of 43 Jeanella Klarys Pascual
Due to the constantly evolving nature of design systems, Lamine & Cheng (2022, p.18) suggest that
successful design systems involve a dedicated team, with responsibilities separate from developers
and designers that work on other parts of the product.

For companies with fewer resources, one solution is to remove the overhead of establishing design
systems by looking at existing design systems. This has been mentioned by several academic papers
that include prototyping design systems in their methodologies (Abdi & Marouen, 2021; Aspinall,
2021; Ðukic, 2020; Linder & Nilsson, 2022). This was also observed as a common response in a
survey carried out during CHI 2020, where “companies increasingly want to develop their own
design systems, but will use Material Design as a reference and an influence” (Yew et al., 2020,
p.7). In Häger’s thesis (2021, p.4), he suggests startups and small companies use or take inspiration
from open-source libraries.

2.5.1 Design Systems and Accessibility

Design systems and their role in addressing usability, centralizing information, and reducing
ambiguity could fill the gaps in web accessibility challenges (Vesselov & Davis, 2019, p.83). In the
studies previously mentioned, a significant number note the importance, or at least the presence of
accessibility in the elements that make up design systems. Häger (2021, p.36) notes that by using a
design system, designers and developers can more easily follow web accessibility principles.
Vendramini et al. (2021, p.8) observe that one of the main reasons for adopting design systems is to
“create alignment with teams”, “create a library of components to improve usability" and
“implement built-in accessibility”. Lamine & Cheng (2022, p.2) promote that design systems
"ensure that the desired system characteristics concerning usability (such as efficiency,
accessibility, and performance) are consistently met”. They also argue that design systems play a
vital role in elevating important discussions on UI behavior that require special expertise, such as
accessibility and safety. Aspinall (2021) shares during the refactoring of the University of
Minnesota’s Health Sciences Library website that the inclusion of an accessibility-focused design
system resulted in faster load times due to lower memory footprint, successful site navigation, and
increased task completion.

The closest study to the objectives this research wants to carry out is Janetchke’s (2021) research on
the Siemens Healthineers Design System, whose focus was improving the accessibility guidelines
in the design system. Two sections (Janetschke, 2021, p.41, p.125) briefly compared the
information architecture of 4 large-scale design systems (IBM's Carbon, Google’s Material Design,
SAP’s Fiori, GE Healthcare’s Edison), which confirms the methodology of my study. However, the

TCLoc Master’s Thesis 2023 G2 Page 13 of 43 Jeanella Klarys Pascual

study’s focus was building on improving specifically one design system, whereas my study’s focus
will be comparing large-scale, publicly available design systems as a baseline for companies.

2.6 Summary
While there is an abundance of research and literature on web accessibility, there is not enough
research on design systems as it is a more recent concept. Literature that deals with the overlap in
design systems and web accessibility is further limited. This study aims to bridge the current gaps in
web accessibility knowledge and industry standards in structuring design systems, to support
software teams in including web accessibility guidelines in their design systems.

TCLoc Master’s Thesis 2023 G2 Page 14 of 43 Jeanella Klarys Pascual

3. Methodology
This study will use a quantitative method using primary sources from available public data. There
will be two phases. During the first phase, I will focus on defining constraints for selecting design
systems. I will then model each of their IA in the different ways they integrate web accessibility. In
the second phase, I will then compare the IA of each design system in how they align with WCAG
criteria. I will extract the variables that are common in the IA of these design systems and analyze
the similarities and differences.

3.1 Sampling
The data is derived from a non-probability purposive sampling of the list in the Open UI Design
System Catalog (Open UI Community Group, 2023b), which I further refined with my own criteria
(see Table 1, p. 15, below). Out of the 19 design systems listed in the catalog, 5 have been selected
for my study.

The following table lists each criterion by order of selection. Performing each step in the following
order allowed me to immediately skip a non-matching design system without needing to go through
the rest of the criteria.

Group Criteria

1 Is a design system 1 Contains design and development guidelines on building

components

2 Publicly accessible 2.1 No login or sign up required

and open source

2.2 Developer kit downloadable on NPM or available on GitHub

3 In English 3 Is written in English or offers an English version

4 Values Accessibility 4.1 At least 70% of its component guidelines contain a section
with the keyword “accessibility”

4.2 Has a dedicated page on Accessibility

Table 1. Sampling Criteria

TCLoc Master’s Thesis 2023 G2 Page 15 of 43 Jeanella Klarys Pascual

Criterion 1: Is a design system

Some of the items in the list are not actual design systems, but tools related to web development in
general and had been included by Open UI in its catalog. For this study, we only select items that
include design and development guidelines for building user interfaces. One such exclusion is
Chromium, which is an open-source browser on which the Chrome browser is based but does not
include any design guidelines.

Criterion 2: Publicly accessible and open-source

A design system is publicly accessible if the design system and source code are available to the
public, without signing up for an account or logging in using company credentials (Criterion 2.1).
In the list, the Auth0 design system is an internal document reserved only for Auth0 employees
requiring Single Sign-on (SSO), so it was excluded from the sample.

Most design systems provide downloadable toolkits or packages containing the necessary elements
developers can use to build user interfaces that adhere to the design systems’ guidelines. While
several public package repositories exist, design systems were selected if their packages are
downloadable from Node Package Manager (NPM) or if the source code is available from version
control repositories, such as GitHub (see Table 1, p. 15, Criterion 2.2).

Criterion 3: Actively maintained

The NPM website provides package information, such as the list of versions and the number of
weekly downloads. For this study, a package is actively maintained if the number of weekly
downloads >= 1000.

Criterion 4: In English

This study is conducted in English and to reach a broader audience, only design systems that offer
an English version were chosen.

Criterion 5: Values accessibility

Among all the selection criteria, this one holds the most weight for this study and is the main
determining factor for selection. Evaluating the inclusion of accessibility in a design system’s
architecture is an effective way of measuring a design system’s investment in accessibility.
Information findability is a critical factor of usability and information that is hard to find means the
system fails to meet the needs of the user (Rosenfeld & Morville, 2002, p. 25).

TCLoc Master’s Thesis 2023 G2 Page 16 of 43 Jeanella Klarys Pascual

Providing accessibility guidelines at a component level could be a beneficial factor in achieving
WCAG Success Criteria especially if they relay WCAG Sufficient Techniques. One of these
techniques is ARIA, which stands for Accessible Rich Internet Applications, a set of roles and
attributes applied when developing web applications (WAI, 2022a). The following table lists the
percentage of components containing accessibility guidelines for the selected design systems. The
design system is selected if at least 70% of its components include them.

Name Maintainer Components

Percentage of components with Total number of

a section on Accessibility components

Bootstrap (2023) Twitter 79.1% 24

Carbon Design IBM 97.5% 40

System (2023)

Lightning Design Salesforce 72.6% 84

System (2023)

Material Design Google 97.1% 35

(2023)

Spectrum (2022) Adobe 100% 61

Table 2. List of selected design systems

See Appendix A (p. 40) for the complete list of design systems and their criteria evaluation.

Design systems that value Accessibility, at the very least, mention it as one of their values or
principles and include a dedicated page on Accessibility (Lehner, 2023). This last criterion (see
Table 1, p. 15, Criterion 4.2) holds lesser weight than Criterion 5.1, as providing contextual
information related to the user’s task of developing or testing components provides more impact
(Rosenfeld & Morville, 2002, p. 175).

TCLoc Master’s Thesis 2023 G2 Page 17 of 43 Jeanella Klarys Pascual

3.2 Keywords referring to Accessibility
On the aspect of Labeling Systems (Rosenfeld & Morville, 2002, p. 147), only the keywords
“Accessibility” and “Accessible” were chosen for this study. While there are several terms related
to accessibility (such as “assistive” or “ARIA”), a restricted list of keywords was used in modeling
the IA due to time constraints. For future research, an improvement on this methodology would be
to use automated tools (such as web scrapers) to perform a search on the list of keywords.

3.3 Modeling of the Design System IA

The design systems are documented as websites, and modeled using a hierarchical mapping, which
shows ancestor-descendant relationships among elements (Nguyen et al., 2006). The hierarchy of
pages in a website is commonly presented as a sitemap.

See Appendix B (p. 41) for the complete mapping of each design system.

Figure 2. Example of a hierarchical mapping of the Bootstrap Docs, with some pages collapsed for
brevity. (See Appendix B1, p. 41, for the complete diagram).

The starting points were derived from the URLs provided in the Open UI Design System Catalog.
For brevity, only the highest level in which the design system documentation appears was used to
model the sitemaps. Additionally, the URLs have been adjusted for this study to point to the latest
version of the design system since some were outdated. For example, Bootstrap was listed as
version 4 whereas the latest as of writing is 5.

The website was then manually crawled by clicking through the site’s Navigation Systems, marking
a page in the hierarchy if it has a descendant page that contains accessibility guidelines (see Table
1, p. 15, Criterion 4.2).

Each page is then categorized based on its navigation system level: Global, Local, and Context
(Rosenfeld & Morville, 2002, p. 175).

TCLoc Master’s Thesis 2023 G2 Page 18 of 43 Jeanella Klarys Pascual

While there might be other pages within a design system that mention “accessibility”, for this study,
please note that I chose to focus on:

1. Components, as they are considered by most users an integral part of design systems
(Sparkbox, 2022).

2. Pages on Accessibility, dedicated pages on web accessibility guidelines.

3.4 Analysis
3.4.1 Four Components of IA

Each design system IA model is classified according to the 4 components of IA: Organization
Systems, Navigation Systems, Labeling Systems, and Search Systems (Rosenfeld & Morville,
2002), followed by a comparative analysis.

3.4.2 Alignment to the WCAG

Each section on Accessibility is assessed on the following levels:

1. No alignment – No links to WCAG Success Criteria or Sufficient Techniques.

2. Fair – Mentions implementation details, but no mentions of WCAG Success Criteria or

Sufficient Techniques.

3. Good – Links to the corresponding WCAG Success Criteria or Sufficient Techniques.

4. Very Good – All criteria of Good, additionally warning whether a component is not
accessible, and what steps to solve accessibility problems.

TCLoc Master’s Thesis 2023 G2 Page 19 of 43 Jeanella Klarys Pascual

4. Results
4.1 Four Components of IA
The first step is to analyze the overall visual structure of each design system. All design systems use
the sidebar layout, a page split vertically into two parts. It consists of a sidebar (usually on the left)
and the main content on the other side.

Figure 3. Wireframe of the Sidebar Layout

The sidebar consists of a Global or Local navigation menu, often showing the first few top levels of
the sitemap hierarchy. When the user clicks on the navigational links in the sidebar, the content is
shown on the main content. The main content then usually contains Local or Context navigations,
depending on the complexity of the design system, although it is not always the case.

The hierarchy and classification of elements are highly similar across the sample, which led to the
following element categories:

• Nav – For cases with the Global context on the top menu bar, a menu link is classified as so.

• Sidebar Group – All of the sidebars contain numerous links and hierarchies of links, so to
increase findability, the links are grouped by topic.

• Page – This is both a link found in the sidebar and the actual page shown in the main
content, which is divided into either sections or tabs.

TCLoc Master’s Thesis 2023 G2 Page 20 of 43 Jeanella Klarys Pascual

 • Section – Part of a page that is usually denoted by a heading or widget, or anything that is
repetitive and recognizable across all pages of the same category. It is usually included in
the Context navigation.

• Tab – Can be considered a subclass of a Section but is considered a Local navigation as

opposed to a Context navigation due to changing tabs losing the previous context. To better
organize detailed information, a page can be divided into tabs, and each tab can be
considered a Page itself with its own sections.

Figure 4. Class diagram of element categories

4.1.1 Organization Systems

Rosenfeld & Morville (2002) states that organization systems are composed of organization
schemes and organization structures. Organization schemes define the shared characteristics of
items that influence how they are grouped and are either exact (alphabetical, chronological, etc.) or
ambiguous (topical, task-oriented, etc.). Organization structures define the relationships between
items and their groups (p. 103).

TCLoc Master’s Thesis 2023 G2 Page 21 of 43 Jeanella Klarys Pascual

Components

The organization schemes for sidebar groups are topical for all cases, except for Spectrum, which
groups them by task (Actions, Data Visualization, Feedback, Navigation, etc.). Component Pages in
Spectrum are one to two levels deeper due to these grouping. Component Pages under a Sidebar
Group are listed alphabetically, across all design systems.

Pages on Accessibility

As opposed to Components, there is more variation in the way each design system approaches the
organization of pages on Accessibility. A specific example, Lightning Design System (see Figure 5,
p. 22, below), has a group of sidebar links grouped by topic (Overview, Guidelines, Patterns), with
each topic further grouped into tasks for Guidelines (Web Design, Mobile Design, Keyboard
Interaction, etc.) and topical for Patterns (component-specific guidelines).

Figure 5. IA model of the accessibility guidelines of the Lightning Design System (see Appendix B3,
p. 42, for the complete sitemap).

Most of the design systems follow the same approach of organizing guidelines by task or topic,
although Carbon Design System applies a hybrid scheme. Its Page on Accessibility is divided into
the following Tabs: Overview, Color, Developers, and Keyboard, a mix of topic-based, audience-

TCLoc Master’s Thesis 2023 G2 Page 22 of 43 Jeanella Klarys Pascual

specific, and task-based, respectively. Each Tab is further divided into Sections that are a mix of
topics (Color tab: Color contrast, Tools) and tasks (Developers tab: HTML best practices, CSS best
practices, Mouse-specific events).

Organization Structure

In both Components and Pages on Accessibility, all elements across the sample follow a
hierarchical (top-down) approach. As shown in Figure 2 (p. 1), elements follow a hierarchy.

4.1.2 Labeling Systems

Components

While design systems can have variations on naming components (Werle, 2021, pp. 53-54), this
study focuses on the nomenclature of the actual category of the component pages fall under. All
design systems seem to agree on using “Components”. The only difference is Lightning Design
System, which uses “Component Blueprints”.

Pages on Accessibility

Every design system uses the same term “Accessibility”, except for Spectrum, which favors the
term “Inclusive”. However, it accepts “accessibility” as an alias for search keywords (see 4.1.4, p.
26).

With regards to the navigation system under which the Pages on Accessibility appear, there is more
variation on the labels:

Label of Parent Category Classification Design System

Getting Started Sidebar Group Bootstrap

Guidelines Sidebar Group Carbon Design System

Accessibility Page Lightning Design System

CDK Nav Material Design

Foundation Sidebar Group Spectrum

Table 3. Labels of parent categories of Pages on Accessibility

TCLoc Master’s Thesis 2023 G2 Page 23 of 43 Jeanella Klarys Pascual

4.1.3 Navigation Systems

Figure 6. Diagram of navigation systems from Rosenfeld & Morville (2002, p. 177).

Rosenfeld & Morville (2002) classifies navigation systems used in websites on levels of scope:
Global, Local, and Contextual (p. 177). While other types of navigation systems exist, this study
will focus only on these three.

Global Navigation

Complex design systems, which include not only design or developer guidelines, such as a blog or
News section, or an About or Contact section, would usually have a Global navigation. Earlier in
1.1.1, the starting point for IA modeling was from a specific URL because certain design systems
have landing pages that do not immediately display design and developer guidelines. Such design
and developer guidelines still fall under the main website, so the Global navigation is constantly
shown. This is the case for Bootstrap and Material Design. Considering Lightning Design System’s
sidebar as its Global Navigation, it is the only design system where Accessibility is listed on the top
level without requiring to open a Sidebar Group.

Local Navigation

For other design systems where the typical top navbar is absent, the Global navigation is transposed
to the sidebar and merged with Local navigation, like most cases. Carbon Design System is the only

TCLoc Master’s Thesis 2023 G2 Page 24 of 43 Jeanella Klarys Pascual

one that uses Tabs as Local navigation and has a Tab dedicated to Accessibility in its Component
Pages.

Figure 7. Wireframe of a page using Tabs as Local Navigation.

Contextual Navigation

While there seems to be some consistency in the way each design system organizes its Contextual
Navigation, their differences can be analyzed according to the following aspects:

• Placement – where the Contextual navigation is positioned on the page.

• Consistency of Accessibility guidelines across all components

o Naming

o Presence – Existence in the page

o Hierarchy – Structure and order in the page

Placement of the Contextual Navigation is either:

• Fixed at the right of the page and remain in place even when the user scrolls through the
page.

• At the top of the page, but scrolls with the page, which requires the user to scroll in order to
jump to a section, which is less ergonomic.
TCLoc Master’s Thesis 2023 G2 Page 25 of 43 Jeanella Klarys Pascual
Among the design systems, Carbon Design System, Material Design, and Spectrum have the most
consistency in both aspects of placement and naming. At least 97% of the components of each
design system contain a section on Accessibility (see Table 2, p. 17) which is also structured
systematically. This helps users easily find information about Accessibility as predictability is an
important factor in findability (Rosenfeld & Morville, 2002, p. 155).

Carbon Design System displays a dedicated tab on Accessibility for each component. Each Tab is
divided further into the following sections: “What Carbon provides”, “Design recommendations”,
and “Developer recommendations”, all linked in the contextual navigation.

Each Component Page in Material Design has a section on Accessibility, whether it is highly
specific to the component and highly detailed, or just a reminder on which ARIA attributes to
include.

Spectrum displays a checklist of design specifications, notably of WCAG compliance, on each

Component Page (show screenshot).

4.1.4 Search Systems

Search functionality in websites is commonly presented as a search box, indicated with either a
magnifier icon or instructions prompting the user to enter a search term. It is usually placed at the
upper region of the page.

Search can be as basic as word or character matching of raw text content of all pages of a website
and can be as advanced as giving the user more control over the results, by allowing them to specify
search parameters, such as modified date, author, or tags.

Leveraging sophisticated algorithms that are optimized for a website’s structured content can be
useful in bringing the most relevant results to the user. Indexing, tagging, fuzzy matching, and
scoring are some principles of search that can be fine-tuned and unique to a website. For example,
prioritizing page titles over paragraph text when matching a search term is one of the common
strategies.

In the case of design systems, listing matching Component pages at the top of the search results is a
possible specialization.

TCLoc Master’s Thesis 2023 G2 Page 26 of 43 Jeanella Klarys Pascual

While the topic of search is complex and vast, this study will focus only on the following aspects:

• Search box

o Presence – the existence of a search functionality in the design system

o Placement and Access – location of the search functionality and activating the
search.

• Search results

o Number of results

o Sort order of results

o Structure of information

o Highlighted keywords – how the relation of the search result to the search term is
presented.

o Persistence – whether the search history is saved.

Search box

All search boxes are textual and there is no advanced search with the option to tweak parameters.
They are placed in either the global navigation or local navigation. All search boxes have a
placeholder text “Search”, with some of them displaying a magnifier icon as a supporting indicator.
Among the sample, only Material Design did not offer search functionality.

Carbon Design System’s search box is collapsed by default, with a button containing a magnifier
icon when clicked reveals and autofocuses on a text box.

Some of the design systems make use of Algolia (Algolia, 2023) an open-source library that offers
tools for websites to integrate search functionality adapted to their content. Some of the design
systems allow users to use shortcut keys to jump to the search box for faster access when browsing.

Search results

Some design systems cap the results to a fixed number, either to optimize displaying of results or as
a way to limit the cognitive load of the user.

TCLoc Master’s Thesis 2023 G2 Page 27 of 43 Jeanella Klarys Pascual

All design systems that offer search functionality group results by hierarchy, usually by the local
navigation the page belongs to. Each design system, however, displays them differently. Matching
words are sometimes highlighted. Most of the time, the results are not sorted alphabetically, but
follow an order specific to the design system.

Bootstrap groups its results by a hierarchy of two to three levels: (1) Page title, (2) Section title, and
(3) Sub-section title, if several sections on a page match the search term. The first result is
Accessibility, grouped together with three of its topics and Additional resources. Further down the
results are an alphabetical list of all the components with a section on Accessibility. It is the only
one that saves search history, based on the last accessed result. There is no maximum number of
search results.

Lightning Design System hierarchizes its results on four levels: (1) Sidebar Group, (2) Page title,
(3) Section title, and (4) Paragraph excerpt, if the matching phrase is found in a paragraph. For the
keyword “Accessibility”, the results are grouped by Component Blueprint, in no particular order.
There is no reference to the Pages on Accessibility in any of the results. However, for the search
keywords “Accessibility <component name>”, level 1 groups are Accessibility, followed by
Component Blueprints. Results are capped at 5 items.

Carbon Design System displays “Content” accessibility guidelines as its top result, followed by
“Get Started”, “Accessibility”, and “Step 4 of Creating Components”. Results are capped at 8 items.
For the search keywords “Accessibility <component name>”, the top result is always the Usage tab
of the matching component. In contrast, the Accessibility tab is shown inconsistently in the middle
or at the end.

Spectrum, which uses the term “Inclusive” in its texts, interestingly accepts the alias “accessibility”.
Searching for both terms displays similar results. The first result is “Inclusive design” under the
“Foundation” group, followed by Content” guidelines, then Usage guidelines. There seems to be no
maximum number of results.

When searching for accessibility guidelines specific to components, knowledge of the existence of a
component and its name is necessary. For first-time users of the design system, familiarity of the
available components would be built from looking at the local navigation first, followed by memory
recall from frequent and regular exposure to the design system. Regarding references to the design
system’s Pages on Accessibility, most of the design systems list them at the top, with the exception
of Lightning Design System.

TCLoc Master’s Thesis 2023 G2 Page 28 of 43 Jeanella Klarys Pascual

4.2 Alignment with the WCAG
While the findability of the term “accessibility” is a factor in helping a user implement accessibility
guidelines, there is an added value in knowing the actual WCAG linked to a component. This
second part of the analysis looks at the level of information provided.

Design systems in the sample have a section listing all its components, each with its own page. This
section could be in any form if it is systematically recognizable when viewed across different
component pages.

When viewing a component’s page, sometimes accessibility guidelines are not instantly visible.
Sometimes a quick in-page search for the keyword “accessibility” reveals a section that is found at
the bottom of the page and the user would need to scroll. Other times, using a related keyword, such
as “assistive” or “ARIA” would bring up a result. This would require the user to be familiar with
the specific terminology related to the accessibility guideline they wish to view, with the risk of
several trial-and-errors before reaching the desired information.

Reiterating the evaluation methodology (see 3.4.2, p. 19): Alignment is good if it links to the
official WCAG Success Criteria or Sufficient Techniques. It is fair if it only mentions
implementation details, without any links to the official WCAG documentation. Alignment is very
good if it warns users of an inaccessible component and offers workarounds to solve accessibility
issues. Lastly, a design system has no alignment if none of the previous criteria are present.

4.2.1 Very Good and Good Alignment

Carbon Design System is strongly linked to WCAG, the design system with the most links to
WCAG. With all its components having a dedicated Tab on Accessibility, it lists all related links in
a systematic way. Additionally, Carbon Design System displays a warning in the Accessibility tab
of a component that violates accessibility guidelines, along with the link to the corresponding
Success Criteria.

Lightning Design System links to specific ARIA guidelines on some of its Pages on Accessibility.
There seem to be more details on the general guidelines than on component-specific guidelines, and
users would have to look at both Pages on Accessibility and the specific Component Blueprint to
ensure that most criteria are fulfilled.

TCLoc Master’s Thesis 2023 G2 Page 29 of 43 Jeanella Klarys Pascual

 Figure 8. Screenshot of a Carbon Design System component with accessibility
violations.

Bootstrap displays accessibility information in banners with heading titles containing the keyword
“Accessibility”. The corresponding Page on Accessibility is linked. The Page on Accessibility
contains a section Additional Resources which links to the WCAG and ARIA landing pages. There
is no systematic presentation of WCAG Success Criteria or Sufficient Techniques in the component
pages, and it is encouraged to use the search box to search for ARIA guidelines for a specific
component. While it does mention a component that violates accessibility guidelines (notably, the
Carousel), it is only mentioned in a paragraph, and its design does not stand out enough to call
attention to the user.

Spectrum links to WCAG Sufficient Techniques in its Pages on Accessibility, but none in its
component pages. However, all its component pages contain a checklist of the component’s
accessibility compliance.

TCLoc Master’s Thesis 2023 G2 Page 30 of 43 Jeanella Klarys Pascual

 Figure 9. Checklist of accessibility compliance of Spectrum components

4.2.2 Fair Alignment

While Lightning Design System enumerates all necessary ARIA attributes needed in each
Component Blueprint, there are no links to the actual WCAG Success Criteria or Sufficient
Techniques. Most of the information on Accessibility, including the related links to WCAG, is
gathered in the Pages on Accessibility.

4.2.3 No Alignment

Although 97% of Material Design components contain an Accessibility section, none link to any
official WCAG Success Criteria nor Sufficient Techniques. Its Page on Accessibility mentions none
either.

TCLoc Master’s Thesis 2023 G2 Page 31 of 43 Jeanella Klarys Pascual

5. Discussion
Information Architecture plays a huge role in the findability and retrievability of information in
complex systems. With some of the challenges in web accessibility mentioned (see 2.2.2, p. 9), such
as the lack of awareness (Rosson et al., 2005, p. 531), a first step would be the introduction of such
accessibility in areas of design systems that users would frequently use (Sparkbox, 2022).

Having evaluated and compared each design system on how their IA components relay accessibility
guidelines, several similarities confirm patterns that may influence other design systems to architect
their documentation. Notably, having dedicated pages that mention global accessibility guidelines
or promote the company’s vision and statement on valuing Accessibility is an effective starting
point (Lehner, 2023).

The consistent inclusion of accessibility guidelines in each of the component pages keeps the user
experience predictable. At the minimal level, a link to the corresponding WCAG Success Criteria or
Sufficient Techniques would help address some of the challenges technology professionals face.
Patel et al. (2020) recommend clearly written resources to support tight development cycles, where
an accessibility-driven design system can fill the gaps (p. 6).

While this study aims to compare the IA of the various design systems in the sample, the goal is to
explore the emerging patterns in labeling, searching, and navigation and the innovative ways each
design system brings. The outcome is not to find the best way to architect an accessibility-driven
design system or rank the sample by effectiveness. This could be explored in another study.

A mix of the unique solutions offered by the various design systems in the sample could also be
included when architecting a design system. Spectrum’s checklist on WCAG compliance and
Carbon Design System’s tab on Accessibility are good examples of information and consistently
presented accessibility information. However, some other challenges remain to be explored, which
were not covered by this study, notably on testing, interpretation of WCAG, and advocating for the
inclusion of accessibility in business needs (see 2.2.2, p. 8).

As mentioned in the Review of Related Literature, there is currently limited research on the
intersection of IA, Design Systems, and Web Accessibility, with Janetschke’s (2021) research as
the most relevant to this study. Janetschke’s (2021) methodology for evaluating WCAG compliance
of the Siemens Design System is more thorough and exhaustive, evaluating all 78 Success Criteria,
specifying which ones were included (p. 90) and excluded (p. 68). The methodology also includes
end-user testing. Future research can build upon Janetschke’s research and this study for a more

TCLoc Master’s Thesis 2023 G2 Page 32 of 43 Jeanella Klarys Pascual

comprehensive evaluation of possible solutions to bridging knowledge gaps of developers on
WCAG.

Due to time constraints, the methodology and scope of this study can be further improved in the
following aspects:

• Keywords – expanding search terms beyond “accessibility” to aliases and other related
terms, such as ARIA, WCAG, and a11y, would cover a wider breadth.

• More design systems – The sample has been derived from the Open UI Design System
Catalog to 5, but other design systems that fit into this study’s criteria can be explored.

• Automation – Making use of web scrapers to search for the list of keywords in each of the
pages of the design system would provide a more detailed map of the variety and coverage
of accessibility information available. Web scraping can also save time in building a
sitemap, especially if the sample size is increased.

Lastly, an additional step in the methodology would involve user testing. A sample of users, from
designers, developers, and testers, grouped by experience in design systems in general and
familiarity with the chosen sample of design systems, will be given tasks to perform. Testing the
effectiveness of a design system’s IA in relaying WCAG to the users would be one of the
objectives.

TCLoc Master’s Thesis 2023 G2 Page 33 of 43 Jeanella Klarys Pascual

6. Conclusion
The resurgence of web accessibility has become a priority for governments, institutions, and
companies, with the American Disabilities Act in full force and the arrival of the European
Accessibility Act in 2025. The challenges of web accessibility have spanned decades since the
inception of the Web and have been a relevant research subject. Web accessibility remains a
complex topic that can be approached from several angles. The view of Information Architecture
and the WCAG has been chosen for this study.

The recent trend of companies and governments adopting design systems was also explored in this
study. As design systems heavily use IA principles in dealing with the complexity of design and
user experience guidelines, they are a good candidate for studying the different ways design
systems could integrate WCAG into their IA. The chosen sample of design systems was derived
from the catalog published by Open UI, a chapter under W3C, the same organization responsible
for Web Accessibility and the drafting standards for the Web in general. With thousands of design
systems publicly available, this list was a viable and robust starting point for the study. While the
Open UI Design System Catalog is still a work in progress, current and future standards may
encourage companies to build on these patterns and innovate novel ways to support their
collaborators in developing accessible user interfaces.

While not every company may not have sufficient resources to invest in design systems and
accessibility, the results of this study could provide teams with the necessary information to
reinforce the information architecture of their products, leverage design systems to improve
alignment within their organizations and gain awareness of the importance of web accessibility.

TCLoc Master’s Thesis 2023 G2 Page 34 of 43 Jeanella Klarys Pascual

7. References
ADA.gov. (2023, March 17). Guidance on Web Accessibility and the ADA.
https://www.ada.gov/resources/web-guidance/

Algolia. (2022). What is Algolia? Retrieved May 28, 2023, from

https://resources.algolia.com/videos/video-whatisalgolia

Alonso, F. L., Fuertes, J. L., Guerra, Á., & Martínez, L. (2010). On the testability of WCAG 2.0 for
beginners. Conference on Web Accessibility. https://doi.org/10.1145/1805986.1806000

Barbareschi, G., Zuleima Morgado-Ramirez, D., Holloway, C., Manohar Swaminathan, S.,
Vashistha, A., & Cutrell, E. (2021). Disability Design and Innovation in Low Resource
Settings: Addressing Inequality Through HCI. Extended Abstracts of the 2021 CHI
Conference on Human Factors in Computing Systems, 1–5.
https://doi.org/10.1145/3411763.3441340

Brajnik, G., Yesilada, Y., & Harper, S. (2012). Is accessibility conformance an elusive property? A
study of validity and reliability of WCAG 2.0. ACM Transactions on Accessible
Computing, 4(2), 1–28. https://doi.org/10.1145/2141943.2141946

Bootstrap. (2023, May 26). Get started with Bootstrap. Retrieved May 29, 2023, from
https://getbootstrap.com/docs/5.3/getting-started/introduction/

Carbon Design System. (2023, May 24). Retrieved May 29, 2023, from
https://carbondesignsystem.com/

Churchill, E. F. (2019). Scaling UX with design systems. Interactions, 26(5), 22–23.

https://doi.org/10.1145/3352681

Cooper, M. C. (2016). Web accessibility guidelines for the 2020s. In Proceedings of the 13th
International Web for All Conference. https://doi.org/10.1145/2899475.2899492

Edelberg, J., & Kilrain, J. (2020). Design Systems: Consistency, Efficiency & Collaboration in
Creating Digital Products. Proceedings of the 38th ACM International Conference on Design
of Communication, 1–3. https://doi.org/10.1145/3380851.3416743
TCLoc Master’s Thesis 2023 G2 Page 35 of 43 Jeanella Klarys Pascual
European accessibility act. (2019). Employment, Social Affairs & Inclusion - European
Commission. Retrieved March 19, 2023, from
https://ec.europa.eu/social/main.jsp?catId=1202

Garrett, J. (2010). The Elements of User Experience: User-Centered Design for the Web and
Beyond (2nd edition). New Riders.

Google. (2023, March 18). Design System - Explore - Google Trends. Google Trends.
https://trends.google.com/trends/explore?date=2013-01-01%202023-03-
18&q=%2Fg%2F11j5jk5_cj&hl=en

Häger, E. (2021). Building a design system for a startup: A case study exploring how an open-
source component library can assist a start-up in the creation of a design system
(Dissertation). Retrieved from http://urn.kb.se/resolve?urn=urn:nbn:se:liu:diva-179036

Janetschke, L. (2021). Accessible interfaces – Improvement of the design system of Siemens

Healthineers regarding accessibility

Kelly, B., Sloan, D., Brown, S., Seale, J., Petrie, H., Lauke, P., & Ball, S. (2007). Accessibility 2.0.
Proceedings of the 2007 International Cross-Disciplinary Conference on Web Accessibility
(W4A) - W4A '07. https://doi.org/10.1145/1243441.1243471

Kumar, P. (2023). AxeCon—Keynote. Retrieved March 18, 2023, from https://www.deque.com/axe-

con/sessions/the-state-of-accessibility-update-2023

Lamine, Y., & Cheng, J. (2022). Understanding and Supporting the Design System Practice.

Lehner, A. (2023, March 22). Checklist for Accessible UI Component Libraries. Retrieved May 22,
2023, from https://www.oidaisdes.org/accessible-ui-component-libraries.en/

Lightning Design System. (2023, May 18). Retrived May 29, 2023, from
https://www.lightningdesignsystem.com/

TCLoc Master’s Thesis 2023 G2 Page 36 of 43 Jeanella Klarys Pascual

Manca, M., Palumbo, V., Paternò, F., & Santoro, C. (2022). The Transparency of Automatic Web
Accessibility Evaluation Tools: Design Criteria, State of the Art, and User Perception. ACM
Transactions on Accessible Computing. https://doi.org/10.1145/3556979

Martin, L., Baker, C., Shinohara, K., & Elglaly, Y. N. (2022). The Landscape of Accessibility Skill
Set in the Software Industry Positions. The 24th International ACM SIGACCESS Conference
on Computers and Accessibility. https://doi.org/10.1145/3517428.3550389

Angular Material. (2023, May 27). Angular Material. Retrieved May 29, 2023, from
https://material.angular.io/

Nguyen, Q. V., Huang, M. L., Zhang, K., & Yen, I.-L. (2006). A Visualization Model for Web
Sitemaps. International Conference on Computer Graphics, Imaging and Visualisation
(CGIV’06), 12–17. https://doi.org/10.1109/CGIV.2006.14

The Office of the High Commissioner for Human Rights. (2006, December 13). Convention on the
Rights of Persons with Disabilities. OHCHR. https://www.ohchr.org/en/instruments-
mechanisms/instruments/convention-rights-persons-disabilities

Open UI Community Group. (April 9, 2023a). Charter. Retrieved May 8, 2023. https://open-
ui.org/charter

Open UI Community Group. (2023b). Design Systems | Open UI. Retrieved May 8, 2023, from
https://open-ui.org/design-systems/

Patel, R., Breton, P., Baker, C. M., El-Glaly, Y. N., & Shinohara, K. (2020). Why Software is Not

Accessible: Technology Professionals’ Perspectives and Challenges. Extended Abstracts of

the 2020 CHI Conference on Human Factors in Computing Systems.

https://dl.acm.org/doi/pdf/10.1145/3334480.3383103

Paternò, F., Pulina, F., Santoro, C., Gappa, H., & Mohamad, Y. (2020). Requirements for Large
Scale Web Accessibility Evaluation. Lecture Notes in Computer Science, 275–283.
Penamora, D. (2023, February 7). A History of the Web Content Accessibility Guidelines - IA Labs.
IA Labs - Digital Inclusion, Your Legal Obligation. Retrieved March 19, 2023, from
https://ialabs.ie/a-history-of-the-wcag/
TCLoc Master’s Thesis 2023 G2 Page 37 of 43 Jeanella Klarys Pascual
Petrie, H., Hamilton, F., King, N., & Pavan, P. (2006). Remote usability evaluations With disabled
people. 2, 1133–1141. https://doi.org/10.1145/1124772.1124942

Rosenfeld, L., & Morville, P. (2002). Information Architecture for the World Wide Web. O’Reilly
Media, Inc.

Rosson, M. B., Ballin, J. F., Rode, J., & Toward, B. (2005). “Designing for the Web” Revisited: A
Survey of Informal and Experienced Web Developers. In D. Lowe & M. Gaedke (Eds.), Web
Engineering (pp. 522–532). Springer. https://doi.org/10.1007/11531371_66

Section508.gov. (2022, November). Retrieved May 29, 2023, from

https://www.section508.gov/manage/laws-and-policies/

Sloan, D. A., Heath, A. C., Hamilton, F., Kelly, B., Petrie, H., & Phipps, L. (2006). Contextual web
accessibility - maximizing the benefit of accessibility guidelines. Proceedings of the 2006
International Cross-Disciplinary Workshop on Web Accessibility (W4A) Building the
Mobile Web: Rediscovering Accessibility? - W4A. https://doi.org/10.1145/1133219.1133242

Sparkbox. (2022, June 16). Sparkbox’s 2022 Design Systems Survey Results.
https://designsystemssurvey.seesparkbox.com/2022

Spectrum, Adobe’s design system. (2022). Retrieved May 29, 2023, from
https://spectrum.adobe.com/

Suarez, M., Anne, J., Sylor-Miller, K., Mounter, D., & Stanfield, R. (2019). Design Systems
Handbook.

Tim Berners-Lee: The Future of the Web (WWW94). (1994). [Video].

https://videos.cern.ch/record/2671957

Trewin, S., Cragun, B., Swart, C., Brezin, J., & Richards, J. (2010). Accessibility challenges and
tool features: An IBM Web developer perspective. Proceedings of the 2010 International
Cross Disciplinary Conference on Web Accessibility (W4A), 1–10.
https://doi.org/10.1145/1805986.1806029

TCLoc Master’s Thesis 2023 G2 Page 38 of 43 Jeanella Klarys Pascual

Vesselov, S. & Davis, T., (2019). Building Design Systems: Unify User Experiences through a
Shared Design Language (p. xiv). Apress.

WebAIM: The WebAIM Million - The 2022 report on the accessibility of the top 1,000,000 home
pages. (2022, March 31). https://webaim.org/projects/million/

Web Accessibility Initiative. (1997). W3C Launches Web Accessibility Initiative [Press Release].
https://www.w3.org/Press/WAI-Launch.html

Web Accessibility Initiative. (2019, October 4). How to Meet WCAG (Quick Reference) 2.0.
Retrieved May 29, 2023, from https://www.w3.org/WAI/WCAG21/quickref/

Web Accessibility Initiative. (2022a, May 22). WAI-ARIA Overview. Web Accessibility Initiative
(WAI). Retrieved May 29, 2023, from https://www.w3.org/WAI/standards-guidelines/aria/

Web Accessibility Initiative. (2022b June 22). W3C Accessibility Standards Overview. Web
Accessibility Initiative (WAI). Retrieved March 22, 2023, from
https://www.w3.org/WAI/standards-guidelines/

Web Accessibility Initiative. Understanding Techniques for WCAG Success Criteria | WAI | W3C.
(2022c, December 22). Retrieved May 29, 2023, from
https://www.w3.org/WAI/WCAG21/Understanding/understanding-techniques

Web Accessibility Initiative. (2023, March 3). Accessibility: It’s About People. Web Accessibility
Initiative (WAI). Retrieved March 23, 2023, from https://www.w3.org/WAI/people/

Wurman, R. S. (1997). Information Architects. New York: Graphis Inc.

Yilmaz, M., O’Connor, R. V., & Clarke, P. (2012). A Systematic Approach to the Comparison of
Roles in the Software Development Processes. In A. Mas, A. Mesquida, T. Rout, R. V.
O’Connor, & A. Dorling (Eds.), Software Process Improvement and Capability
Determination (pp. 198–209). Springer. https://doi.org/10.1007/978-3-642-30439-2_18

TCLoc Master’s Thesis 2023 G2 Page 39 of 43 Jeanella Klarys Pascual

Appendix A
Sampling criteria applied to the Open UI Design System Catalog

Name Maintainer C1 C2 C3 C4.1 C4.2

AntDesign AFX Yes Yes Yes No 0
Bootstrap Twitter Yes Yes Yes Yes 71.9%
Auth0 Styleguide Auth0 Yes No
Material Components Google Yes Yes Yes Yes 97.1%
Carbon Design System IBM Yes Yes Yes Yes 97.5%
Chromium Google No
Evergreen Twilio Segment Yes Yes Yes No
Atlaskit Atlassian Yes Yes Yes Yes 27%
Fluent UI Microsoft Yes Yes Yes Yes 0
Kolibri IZB Yes Yes No
Primer Design System GitHub Yes Yes Yes Yes 51.7%
Lion ING Yes Yes Yes Yes 0
Fast Microsoft Yes Yes Yes No
Lightning Design System Salesforce Yes Yes Yes Yes 72.6%
Semantic UI Semantic Org Yes Yes Yes No
Spectrum Adobe Yes Yes Yes Yes 100%
Stardust UI Stardust UI Yes Yes Yes No
WAI-ARIA W3 No
Web Platform WHATWG & W3C No

TCLoc Master’s Thesis 2023 G2 Page 40 of 43 Jeanella Klarys Pascual

Appendix B
Models of Design System IA

Figure B1. IA of Bootstrap Accessibility and Component Pages

Figure B2. IA of Carbon Design System Accessibility and Component Pages

TCLoc Master’s Thesis 2023 G2 Page 41 of 43 Jeanella Klarys Pascual

 Figure B3. IA of Lightning Design System Accessibility and Component Pages

Figure B4. IA of Material Design Accessibility and Component Pages

TCLoc Master’s Thesis 2023 G2 Page 42 of 43 Jeanella Klarys Pascual

 Figure B5. IA of Spectrum Accessibility and Component Pages

TCLoc Master’s Thesis 2023 G2 Page 43 of 43 Jeanella Klarys Pascual