Evaluation of Constructify Website in User Experience

Perspective using User Experience Questionnaire (UEQ) and

System Usability Skill (SUS)
Angelyn Caecilia Vanesa1,a), Michelle Davine2,b), Michelle Bell3,c)
1
Entrepreneurship Business Creation, Bina Nusantara University, Malang, Indonesia 65154
2
Entrepreneurship Business Creation, Bina Nusantara University, Malang, Indonesia 65154
3
Computer Science, Bina Nusantara University, Malang, Indonesia 65154

a)
[email protected]
b)
[email protected]
c)
[email protected]

Abstract. This research evaluates user experience and usability of Constructify, a proposed interactive website for
people who need labor for construction and the construction workers. Quantitative analysis was done using User
Experience Questionnaire (UEQ) and System Usability Scale (SUS) on 35 students. Results showed CONSTRUCTIFY
scored high on efficiency, indicating an intuitive, easy to use interface. Other aspects like attractiveness, responsiveness,
and stimulation were also rated positively. CONSTRUCTIFY performed above average overall based on UEQ
benchmarks. The SUS score of 86.8 confirmed high usability and user satisfaction, surpassing average scores. This shows
ease of use and learnability in CONSTRUCTIFY’s design. Limitations include small sample size and demographic.
Further research should deploy the prototype to a larger, more diverse population including end-user migrant workers.
The prototype demonstrates technology's potential to empower migrant workers through features like digital training, e-
wallets, and document management. However, challenges around digital literacy, accessibility, and security need
addressing. A human-centered approach focusing on user needs is key. This UX research provides a foundation for
refining CONSTRUCTIFY to optimize user experience and leverage technology to positively impact construction
workers and those user who really needed some construction apps.

Keywords: user experience, project efficiency, construction web

INTRODUCTION

The construction industry in Indonesia has been growing rapidly, yet it faces challenges such as project delays,
cost overruns, and inefficient management. The adoption of digital platforms and apps for construction project
management presents a promising solution to these issues, aiming to enhance efficiency, transparency, and
coordination among stakeholders. According to data released by Statistics Indonesia (BPS) on February 28, 2023,
there are 8,481,349 people employed in the construction sector in Indonesia[1]. This significant number of
individuals working in construction highlights the need for an efficient, swift, honest, and transparent approach to
construction business processes. CONSTRUCTIFY emerges as a platform to address these concerns and combat the
prevalent issue of fraud in the Indonesian construction industry. As exemplified by the case of Bambang Dwi Narto,
a construction worker who defrauded Rekno Wulandari in Surabaya, East Java, as reported by Kilat.com on July 28,
2023, the need for a trustworthy platform is paramount[2]. Furthermore, the advancement of AI technology poses a
potential threat to architects and interior designers in Indonesia, hindering the smooth marketing and appreciation of
their work. CONSTRUCTIFY aims to bridge this gap by serving as a business platform that nurtures and supports
the work of Indonesian professionals in the construction and infrastructure sectors.

The Indonesian construction industry is characterized by a large workforce and a significant contribution to the
national economy. However, this sector is also plagued by inefficiencies, delays, and a lack of transparency.
CONSTRUCTIFY addresses these challenges by providing a streamlined platform that connects clients, contractors,
and construction professionals. This digital marketplace facilitates efficient project management, real-time
communication, and secure payment transactions. By fostering transparency and collaboration, CONSTRUCTIFY
promotes trust and accountability among stakeholders, ultimately leading to improved project outcomes. Fraudulent
practices in the construction industry have long been a source of concern for clients, leading to financial losses and
project delays. CONSTRUCTIFY tackles this issue by implementing robust fraud prevention measures, including
background checks, escrow accounts, and dispute resolution mechanisms. The platform's emphasis on transparency
and client protection instills confidence among users, fostering a secure and reliable environment for construction
projects. The rapid advancement of AI technology has the potential to transform the construction industry,
automating tasks and introducing new design and construction methodologies. However, this progress also raises
concerns about the impact on the livelihoods of architects, interior designers, and other construction professionals.
CONSTRUCTIFY recognizes these concerns and aims to empower these individuals by providing them with the
tools and resources necessary to adapt and thrive in the evolving landscape. The platform offers training
opportunities, access to AI-powered design tools, and a network of industry experts, enabling professionals to stay
ahead of the curve and integrate AI into their work seamlessly.

Therefore we want to test the feasibility of the UI / UX of our website in the market and surrounding
environment in order to see the response of the community to the appearance of our website prototype and the
usefulness of each existing feature and provide an assessment of our website so that it becomes a validation and
evaluation material for our application later if launched to the market. The feasibility of the website in the market
will be a benchmark to prove the effectiveness and efficiency of user delivery services if using assistance from
digital applications. CONSTRUCTIFY stands as a beacon of innovation and progress in the Indonesian construction
industry. By addressing the challenges of inefficiency, fraud, and technological disruption, Constructify paves the
way for a more transparent, secure, and AI-enabled construction ecosystem. The platform's commitment to
supporting Indonesian construction professionals ensures that the industry remains a driving force of national
development, while also nurturing the talent and creativity of its workforce.

LITERATURE REVIEW

Definition of User Experience / Definition of Usability

User experience refers to the overall experience a user has when interacting with a website, including usability,
functionality, and emotional impact. UX also encompasses the holistic experience a user has when interacting with a
product, system, or service. It delves into the user's emotions, perceptions, preferences, and behaviors throughout
their entire journey. Therefore,also encompasses a wide range of factors, including the product's functionality,
usability, aesthetics, emotional impact, and overall satisfaction. A well-designed UX strives to create a positive,
engaging, and memorable experience for users, fostering loyalty and brand advocacy. Usability is a subset of UX
that focuses on the ease of use and effectiveness of a product or system. It evaluates how easily users can learn,
operate, and achieve their goals with the product. Usability encompasses factors such as learnability, efficiency,
memorability, error prevention, and satisfaction. A product with high usability is intuitive, straightforward, and
enables users to accomplish tasks effortlessly. In essence, UX is the broader concept that encompasses the overall
experience, while usability is a specific aspect of UX that focuses on the ease of use. Both UX and usability are
crucial for creating successful products that meet user needs and expectations.[3][4][5]

Related Theories
User experience (UX) is a user's perceptions and interactions with a product, system, or service throughout the
entire lifecycle. It is a holistic concept encompassing usability, usefulness, efficiency, satisfaction, and aesthetics
[6]. Donald Norman's seminal work on the Psychology of Everyday Things [7] emphasizes the importance of
designing user interfaces that are intuitive and user-centered. Usability, a critical component of UX, focuses on the
ease of use and learnability of an interface. The Nielsen Norman Group's Five Usability Heuristics [8] provide a
practical framework for designing user-friendly interfaces by addressing factors like visibility of system status,
match between system and the real world, user control and freedom, and consistency and standards.

Explanation of Your Prototype Application

The prototype is a construction website designed to provide information about construction projects, services,
and company profiles, aiming to facilitate user interaction and information retrieval efficiently. The website
functions as a comprehensive information hub, encompassing construction projects, services offered by various
companies, and detailed company profiles. This user-centric design prioritizes facilitating efficient user interaction
and information retrieval. By offering a centralized platform for construction-related information, the prototype aims
to streamline the user experience for all stakeholders involved in the construction process. Our platform also offers
architectural and interior design services executed by Indonesian architects and designers who hold recognized
training certificates and STRA (Surat Tanda Registrasi Arsitek) issued by the Indonesian Institute of Architects
(IAI). Additionally, all designs are copyrighted, ensuring that the unique building models created for users are not
readily available elsewhere. This approach safeguards clients' intellectual property and ensures the originality of
their projects.

The website caters to a broad audience, including potential clients seeking construction services, professionals
within the industry searching for project information, and companies looking to expand their reach. Through an
intuitive interface and well-organized information architecture, users can easily navigate the website to find the
specific details they require. This emphasis on user-friendliness fosters a more efficient information retrieval
process, saving users valuable time and effort. In essence, the prototype website bridges the gap between
information seekers and information providers within the construction sector. By fostering seamless user interaction
and efficient information retrieval, this user-centric platform has the potential to significantly improve
communication and collaboration within the industry.

METHODS

Types of research
The research method used for this research is quantitative, especially questionnaires. This questionnaire is
intended as a benchmark for evaluating user experience by distributing Google Forms to 35 people. So the
questionnaire contains a User Experience Questionnaire (UEQ) which contains questions regarding the user
experience of an application and asks users to rate the System Usability Scale (SUS), which is based on a scale of 1-
5 (strongly disagree - strongly agree) out of 31 questions. and 1 question to rate overall on a scale of 1-10 (very bad
– excellent). With the combination of these two instruments, this research can provide in-depth insight into how
users interact and experience User Experience Design, thereby enriching their understanding of user effectiveness
and comfort when using the CONSTRUCTIFY website.

Population and Sample

A. Population:
The selection of the population in this research, with a focus on the Malang City area and students therein,
was based on practical and strategic considerations. The choice of Malang City as the research location
provides the advantage of better control over the data, allowing direct monitoring and effective coordination
with respondents. Students were chosen as research subjects because of the diversity of their backgrounds
and experiences, which were expected to provide sufficient variety for in-depth analysis. This decision also
reflects the relevance of the educational context, with the assumption that student participation can provide
valuable insight into issues related to the academic environment and city life. In addition, limiting the
population to the Malang City area allows for efficient data collection and optimization of research resources,
creating a focused and effective research environment.

B. Sample:
The sampling technique used in this research is Simple Random Sampling (SRS)[8], where we take an
environment that tends to be homogeneous and specifically target BINUS students from various majors who
are able to work abroad as the sample that will be studied in this research.
Sumber: https://uptjurnal.umsu.ac.id/mengenal-sampling-dalam-penelitian/
 Research variables
There are various questions which are divided into 5 parts (modules) with the following classification:
a. Module 1 (Efficiency)
Questions in module 1 show how efficient and easy it is to use the CONSTRUCTIFY application on a scale
of 1-5.
b. Module 2 (Attractiveness)
Questions in module 2 show how attractive the appearance and design of the CONSTRUCTIFY application
is on a scale of 1-5.
c. Module 3 (Responsiveness)
Questions in module 3 show how responsive the user experience is in the CONSTRUCTIFY application on
a scale of 1-5.
d. Module 4 (Stimulation)
Questions in module 4 show how motivated the respondent is to use the CONSTRUCTIFY application on a
scale of 1-5.
e. Module 5 (Overall)
Questions in module 1 show the respondent's assessment of the CONSTRUCTIFY application as a whole
on a scale of 1-10.

Data Collection Techniques

The data collection technique used is a questionnaire, where in this study the respondents are BINUS
students from various majors ranging in age from 19 to 24 years old, as well as the families or relatives of our
newly married or married siblings. So this questionnaire is in the form of a google form which is then distributed
in the BINUS environment to provide an assessment of the user experience of the Constructify application.
Before giving an assessment, respondents are allowed to try and access all features of the Constructify
application prototype through the axure link provided. After exploring the prototype, respondents will be asked
questions related to the user experience of the Constructify application.

Data analysis
We use analytical data in the form of the User Experience Questionnaire (UEQ) to measure the quality of
the user experience of a product or system as a whole. UEQ evaluates six key aspects of user experience namely
attractiveness, efficiency, clarity, stimulation, novelty and suitability for purpose. By using a standard
questionnaire consisting of 32 questions, UEQ can produce quantitative scores for each of these aspects. These
scores are useful for comparing between products/systems, tracking UX changes, identifying UX issues, and
building shared understanding within the development team. So, UEQ enables comprehensive and accurate UX
measurements to evaluate and improve the quality of user experience.
Furthermore, the System Usability Scale (SUS) is used to measure the level of usability or usability of a
system, product, or user interface. SUS is designed to provide a quick and reliable assessment of the extent to
which users are satisfied with the experience of using a system or product. This instrument is primarily used in
the context of user interface (UI/UX) design and software product development. By using SUS, researchers or
designers can collect quantitative data that helps measure the efficiency, usability, and level of user satisfaction
with a product or system. The final SUS score can provide an overall picture of how well a system is accepted by
users, and the results can be used to assess improvements or further development in the design of the product or
system.
(1)

RESULTS AND DISCUSSION

User Exeperience Questioner (UEQ)

After creating LABORA application to provide a tangible user experience, we conducted a user satisfaction
survey using Google form to obtain the result of user review analysis. This allowed us to identify areas for
improvement, additions, and enhancements. User satisfaction was measured on a scale of 1 (strongly disagree) to 7
(strongly agree). The survey results indicate an average score between 5 and 7, indicating that user satisfaction with
the LABORA application is quite good. However, there are indeed some aspects that need improvement, as
higtlighted in user reviews on the Google Form.
 We utilized the UEQ method to gather survey results on user satisfaction with the CONSTRUCTIFY website.
The following is the analysis:

Scale Mean Variance Std.dev. N Confidence Confidence Interval

Efficiency 2.01 0.62 0.79 35 0.26 1.75 2.27
Attractiveness 1.10 0.64 0.80 35 0.26 0.83 1.36
Response behavior 1.07 0.65 0.81 35 0.27 0.80 1.34
Stimulation 1.06 0.67 0.82 35 0.27 0.79 1.33
Table 1 Mean of UEQ Scale

We have conducted a survey of 38 respondents where we got the highest aspect result of (2,01) in Efficiency.
So it can be said that CONSTRUCTIFY Website have provide some positive element ini efficiency in website used
by the user’s perception. On the other hand the lowest number is (1.06) in Stimulation. So it can be said that
CONSTRUCTIFY need to improve in engagement in application so the user have motivation to use more
CONSTRUCTIFY website, the othe hypothesis, why our Stimulation score is low, that because the target audience
is not needed website like CONSTRUCTIFY.

Figure 7 Scale Means

LABORA UEQ OF BENCHMARK

Category Benchmark Variables Mean Score Results
Excellent 2-3 Efficiency 2.01 Excellent
Good 1-2 Attractiveness 1.10 Good
Average 0-1 Responve behavior 1.07 Good
Bad <-1 Stimulation 1.06 Good
Table 2 UEQ Benchmark

Based on the benchmark data above, the aspect with the highest score is in the “Efficiency” aspect at 2.01. This
indicates that CONSTRUCTIFY has a website with clarity, or it can be described as a user-friendly website.
Meanwhile, the lowest score is obtained in the “Stimulation” aspect. However, this is not a concern for us because
the score is still above the average or benchmark standard.

CONCLUSIONS

Overall, this research has provided valuable insights into the user experience and usability of the proposed
CONSTRUCTIFY website for user and construction workers. The quantitative analysis using the User Experience
Questionnaire (UEQ) and System Usability Scale (SUS) allowed us to evaluate key aspects of the user experience.

The UEQ results indicate that CONSTRUCTIFY has excellent efficiency, with a high mean score of 2.01.
This shows that users found the application easy and intuitive to use. The other aspects of attractiveness,
responsiveness, and stimulation were rated as good, with mean scores between 1.06-1.10. Compared to UEQ
benchmarks, CONSTRUCTIFY scores well above average across all categories. There are opportunities to further
improve user stimulation and engagement.

The SUS score of 86.8 also implies high usability and satisfaction levels among respondents. This is well
above the average SUS score of 68, demonstrating ease of use and learnability of the system design. Users seem able
to effectively complete tasks and interact with LABORA's features and information architecture.

Some limitations include the relatively small sample size of 38 respondents. In the future, deploying the
prototype to a larger population could provide more robust quantitative data. The research was also localized to new
married people in Malang City. Testing with the actual contructor and user demographic would provide direct UX
insights.

Overall, the prototype demonstrates technology's potential to add value in the Indonesian Construction
Industry Features like tracking and report for the project work, e-wallets, and document management can increase
transparency, productivity and empowerment. However, issues like digital literacy, accessibility and security need
consideration in implementation. A human-centered design approach will be critical.

In summary, this UX research and prototype have shown promising results and provided a foundation for
developing a digital platform to serve Indonesian migrant workers. Further refinement based on user-centered
principles and testing will help optimize the user experience. The CONSTRUCTIFY concept has strong potential to
make a positive social impact by using technology to empower construction industry in Indonesia.

REFERENCES

1. [1] International Organization for Standardization (ISO) 9241-210:2010. Ergonomics of human-system

interaction - Part 210: Human-centered design for interactive systems.
2. [2] Norman, D. A. (1988). The psychology of everyday things. Basic Books.
3. [3] Nielsen, J., & Molich, R. (1994). Heuristics for user interface evaluation. In CHI '94 Proceedings of the
SIGCHI Conference on Human Factors in Computing Systems (pp. 249–256).
https://www.nngroup.com/articles/ten-usability-heuristics/
4. Indonesia, B. P. S. (2023, February 2023). Statistik Indonesia 2023. Badan Pusat Statistik Indonesia.
https://www.bps.go.id/id/publication/2023/02/28/18018f9896f09f03580a614b/statistik-indonesia-
2023.html
5. Tak Percaya Polisi! Korban Dugaan Penipuan Tukang Bangunan di Surabaya Lebih Pilih Viralkan di
Media Sosial.(2023, July 28). Kilat.com. https://www.kilat.com/nasional/8449619170/tak-percaya-polisi-
korban-dugaan-penipuan-tukang-bangunan-di-surabaya-lebih-pilih-viralkan-di-media-
sosial#google_vignette
6. Hassenzahl, M., & Tractinsky, N. (2006). User experience-a research agenda. Behaviour & information
technology, 25(2), 91-97.
7. Garrett, J. J. (2011). The elements of user experience: user-centered design for the Web and beyond.
Pearson Education.
8. Norman, D. (2013). The design of everyday things: Revised and expanded edition. Basic books.
9. Garrett, J. J. (2011). The elements of user experience: user-centered design for the Web and beyond.
Pearson Education.
10. Nielsen, J. (2012). Usability 101: Introduction to usability. Nielsen Norman Group.
11. Krug, S. (2014). Don't make me think!: A common sense approach to Web usability. Pearson Education
India.
12. Davis, F. D. (1989). Perceived usefulness, perceived ease of use, and user acceptance of information
technology. MIS quarterly, 319-340.
13. Dumas, J. S., & Loring, B. A. (2008). Moderating usability tests: Principles and practices for interacting.
Morgan Kaufmann.
14. Salsabilah Maisah A., “Sampling Dalam Penelitian.” Accessed: Jan. 19, 2024. [Online]. Available:
Sampling Dalam Penelitian
15. S. L. Kumala, “Perkembangan Ekonomi Berbasis Digital Di Indonesia,” Journal of Economics and
Regional Science, vol. 1, no. 2, pp. 109–117, Jan. 2021, doi: 10.52421/JURNAL-ESENSI.V1
16. Indonesia Construction Industry Size & Share Analysis - Growth Trends & Forecasts (2024 - 2029).
(2024). Mordor Intelligence. https://www.mordorintelligence.com/industry-reports/indonesia
17. “Pengguna Internet di Indonesia Tembus 213 Juta Orang hingga Awal 2023.” Accessed: Dec. 09, 2023.
[Online]. Available: https://databoks.katadata.co.id/datapublish/2023/09/20/pengguna-internet-di-
indonesia-tembus-213-juta-orang-hingga-awal-2023
18. Ramadhan, A. B. (2021). Rugikan Rp 3,6 M, Begini Konstruksi Kasus Korupsi di Perum Jasa Tirta II.
DetikNews. https://news.detik.com/berita/d-5708807/rugikan-rp-3-6-m-begini-konstruksi-kasus-korupsi-di-
perum-jasa-tirta-ii
19. “Kementerian Komunikasi dan Informatika.” Accessed: Dec. 09, 2023. [Online]. Available:
https://www.kominfo.go.id/content/detail/47841/pemerintah-dorong-peningkatan-digitalisasi-umkm-dan-usaha-
kreatif/0/berita

LINK AXURE
https://drive.google.com/file/d/1Md9kG-ZkQXnJRPEwNC18G937Zcgw-O3A/view?usp=sharing

LINK Website Tampilan Constructify Menggunakan HTML DAN CSS Buka File Menggunakan Visual
Studio Code
https://drive.google.com/file/d/1rZ3XATQK0F08CozKOwNFBEmoWC-xSOuV/view?usp=sharing

Link Hasil Survey

https://docs.google.com/document/d/1tymYtgvif2DITV_dOD6DJ92zPOQ8kK1D/edit?usp=sharing&o
uid=100392728916065436805&rtpof=true&sd=true