IT1906

HCI Technologies
Computing Platform Technologies in HCI
Platform refers to a group of technologies that are used as a base upon which other applications, software,
systems, and processes are developed. These can be viewed as toolsets for developing and operating
customized services.
These are some examples of technological platforms (Spacey, 2019):
• Computing Platforms • Game Platforms • Robotics
• Database Platforms • Media Platforms • Analytics
• Web Platforms • Mobile Platforms • Security
Computing platform refers to a specific hardware (computing device) and software (operating system) on
which an application or software can be run. The computing platform conforms to a set of human-computer
interaction (HCI) standards that enables software developers to create interactive systems and applications.
New standards-based interfaces and open interfaces allow application programs to run on multiple platforms,
while software developers have developed tools that enable applications to run on multiple platforms
(Techopedia, n.d.).

Mobile and Handheld Interaction

The importance of usability and user experience (UX) for mobile and handheld interaction is continuously
increasing. A mobile device, as represented by smartphones, is a crucial point where two (2) notable trends
are converging: multimodal interaction and cloud-based services (Kim, 2015).
There are two (2) commonly used devices in mobile and handheld interaction:
• Smartphone – It is a combination of a mobile phone and a handheld computer into a single device
(Beal, 2015). This device has a built-in personal information management programs (such as electronic
calendar and address book) typically found in a personal digital assistant (PDA) and an operating
system (OS) that allows mobile applications to be installed. Smartphones can be used for Web
browsing, sending e-mails, playing music, video streaming, text messaging, etc. (Hosch, 2018)
• Tablet – It is a computer that is intermediate in size between a laptop computer and a smartphone.
Tablets and smartphones basically have the same functionalities. Early models of tablet computers
used either a keyboard or a stylus for information input, but then these input methods were
subsequently displaced by touch screens (Gregersen, 2019).

High-End Cloud Service and Multimodal Client Interaction

In implementing different interactive technologies and hallmarks of human intelligence, such as recognizing
spoken words, sentences, images, and gestures, require artificial intelligence (AI). However, technologies with
AI generally need large databases, long off-line learning processes, and heavy online computation for real-
time responses. With this, high-performance servers – coupled with mobile clients that handle fast data input,
capture, and transfer – offer a promising solution. Therefore, systems and applications utilizing high-end cloud
services with high-performing servers can now facilitate various HCI-related computations, such as the
following (Kim, 2015):
• Multimodal integration • Large-scale and multi-user tracking
• Context-based reasoning • Usage pattern analysis
• User characteristics deduction • Big data gathering
Artificial Intelligence – This refers to the ability of a digital computer or a computer-controlled robot to
perform tasks commonly associated with human intelligence. It is frequently applied in developing systems

09 Handout 1 *Property of STI

 [email protected] Page 1 of 3
 IT1906

endowed with intellectual processing characteristics of human, such as the ability to reason, discover meaning,
generalize, and learn from previous experience (Copeland, 2019).
High-End Cloud Service – This pertains to any service made available to users via the Internet from a cloud
computing provider’s server, instead of being provided by a company. It is designed to provide an easy and
scalable access to applications, resources, and services, and is managed by a cloud services provider (Beal,
n.d.). High-end cloud services make multimodal client interaction more feasible.
Multimodal Client Interaction – This complements different operational restrictions. This has also been an
active research ground in the academic world, beginning with the first pioneering multimodal interactive
system developed by Bolt et al. at Massachusetts Institute of Technology (MIT) in the early 1980s. Since then,
various ways of combining multiple modalities for effective interaction have been devised (Kim, 2015).

Natural/Immersive/Experiential Interaction
Language understanding greatly involves natural language processing, which can be divided into two (2)
processes: word recognition and sentence understanding. One good application of language understanding is
voice recognition. Voice recognition is the ability of a machine or program to receive and interpret dictation
or to understand and carry out spoken commands. This system enables consumers to interact with technology
by speaking, which allows users to perform hands-free requests and other simple tasks.
Voice recognition gained prominence and use with the rise of AI and intelligent assistants such as Amazon’s
Alexa, Apple’s Siri, and Microsoft’s Cortana (Techtarget, 2018).
Gestures play an important role in human communication. It can convey a specific meaning, or it can
supplement other modes of communication. The main objective of incorporating gestures into HCI is to engage
users into a more natural interaction.
Gestures can either be from the human perspective (supplementary pointing vs. symbolic) or from a
technological viewpoint (static posture vs. moving hand gestures). An example of how human gestures are
integrated into a system is motion tracking (Kim, 2015). Motion tracking assists in tracking the movement of
objects or humans and transferring the sensed data to an application that contains stored motion templates.
It enhances human-computer interaction, plays a vital role in computer animation of a 3D model, and requires
specific hardware and software programs to capture and process large amounts of data. This technology is
used in military, entertainment, sports, medical applications and robotics (Techopedia, n.d.).
Other examples of natural/immersive/experiential interaction:
o Microsoft Kinect o Immersive teleconferencing
o Nintendo Wii o Virtual reality training and education

Mixed and Augmented Reality

• Virtual Reality (VR) – It is the use of technology to create a simulated environment. VR places the user
inside an experience instead of images or videos on a screen. Users are immersed and able to interact
with 3D objects by stimulating as many senses as possible, such as vision, hearing, touch, and even
smell. But it is only limited in terms of availability of content and computing power (Bardi, 2019).
• Augmented Reality (AR) – This technology is an interactive, reality-based environment that uses the
capabilities of a computer to generate natural or realistic display, sound, text, and effects to enhance
the user’s real-world experience. AR combines reality and computer-based scenes and images to
deliver a unified and enhanced view of an environment (Techopedia, n.d.).

09 Handout 1 *Property of STI

 [email protected] Page 2 of 3
 IT1906

These are the following types of AR being used in different industries today (RealityTechnologies.com,
n.d.):
o Marker-Based AR – This type of AR uses image recognition technology, which uses a camera
and a visual marker (such as QR/2D code) to produce results when the reader (smartphones
and tablets) senses the marker.
o Markerless-Based AR – It is one of the most widely implemented applications of AR that uses
GPS, digital compass, velocity meter, and/or accelerometer embedded in a device to provide
data based on the user’s location.
o Projection-Based AR – This technology works by projecting light to real-world surfaces. It
allows human interaction by sending light onto a surface and then detecting human
interaction, such as touch, by differentiating an expected projection to an altered projection
(caused by user’s interaction).
o Superimposition-Based AR – Object recognition plays a vital role in this type of AR. It either
partially or fully replaces a real-life object with a newly augmented view of the same object.
• Mixed Reality (MR) – It is the result of blending the physical world and the digital world, resulting in
a “hybrid” environment. Interactive virtual objects are mapped to the physical environment, blending
the real and the virtual. MR requires awareness of the geometry of the environment around the user,
using it as a canvas to create an immersive content that is defined by the space the user is in. All these
processes are made possible through a headset or headgear (Rogers, 2018).

References:
Augmented Reality (AR). (n.d.). In Techopedia. Retrieved from https://www.techopedia.com/definition/4776/augmented-reality-ar
on September 17, 2019
Bardi, J. (2019, March 26). What is virtual reality? [Definition and examples]. Retrieved from https://www.marxentlabs.com/what-is-
virtual-reality/ on September 16, 2019
Beal, V. (2015, December 11). Introduction to mobile devices. Retrieved from https://www.webopedia.com/quick_ref/mobile_OS.asp
on September 16, 2019
Beal, V. (n.d.). Cloud service. Retrieved from https://www.webopedia.com/TERM/C/cloud_services.html on September 16, 2019
Copeland, B.J. (2019, May 9). Artificial Intelligence. Retrieved from https://www.britannica.com/technology/artificial-intelligence on
September 16, 2019
Gregersen, E. (2019, January 18). Tablet computer. Retrieved from https://www.britannica.com/technology/tablet-computer o
September 16, 2019
Hosch, W. (2018, November). Smartphones. Retrieved from https://www.britannica.com/technology/smartphone on September 16,
2019
Kim, G. (2015). Human-computer interaction fundamentals and practice. USA: CRC Press.
Motion Tracking. (n.d.). In Techopedia. Retrieved from https://www.techopedia.com/definition/31558/motion-tracking on
September 16, 2019
Platform. (n.d.). In Techopedia. Retrieved from https://www.techopedia.com/definition/3411/platform on September 16, 2019
Reality Technologies. (n.d.). The Ultimate Guide to Understanding Augmented Reality (AR) Technology. Retrieved from
https://www.realitytechnologies.com/augmented-reality/ on September 17, 2019
Rogers, S. (2018, December 4). What is mixed reality and what does it mean for enterprise?. Forbes. Retrieved from
https://www.forbes.com/sites/solrogers/2018/12/04/what-is-mixed-reality-and-what-does-it-mean-for-
enterprise/#7fd57b945df9 on September 17, 2019
Spacey, J. (2019, February 11). 16 types of technology platform [Web log post]. Retrieved from
https://simplicable.com/new/technology-platform on September 16, 2019
Voice recognition (speaker recognition). (2018, January). In Techtarget. Retrieved from
https://searchcustomerexperience.techtarget.com/definition/voice-recognition-speaker-recognition on September 16,
2019

09 Handout 1 *Property of STI

 [email protected] Page 3 of 3