The Impact of Artificial Intelligence in Augmented Reality: An Overview of Artificial Interactive Reality
The Impact of Artificial Intelligence in Augmented Reality: An Overview of Artificial Interactive Reality
Abstract: This paper reports a novel interactive testbed system based on two major developing fields in the world of technology,
Artificial Intelligence (AI) and Augmented Reality (AR) combined to form Artificial Interactive Reality (AIR). Augmented
Reality (AR) is a technology in which we add some virtual content that exists in a virtual world onto a real-life environment.
Recent technologies have enabled users to connect to real-time applications effectively and have initiated research based upon its
holistic nature. In AR, the user sees and interacts with the virtual content and promotes the ability to delve deeper into the
technology. So, in the reported proposition, adding Artificial Intelligence to a virtual content would provide automation in a real-life
environment and bring about real experiences to a user. Existing AR objects cannot be designed to fully interact with users or
respond to their commands. Thereby, combining AI to it can add various functionalities to an AR object like speech recognition,
computer vision including most of the benefits which we get from AI. Still, a lot of research and development is required in this
area but with the upcoming processing power and technological advancements, this can be achieved in the future. This paper
reports a futuristic technology cited as “Artificial Interactive Reality”, which gives an overview of the dynamic nature of
Artificial Intelligence and its capability to enhance virtual objects, thus lead to the rise of an interactive environment. The self-
learning AIR object “Vivek” has been viewed as a potential reference to the proposed technology. AIR has the power to bring
revolutionary changes in the world of technology not only in the field of computer science but also in other fields like medical
sciences, architectural engineering, elementary education and much more. Adding the technology to the field of research and
development can improve a developer or a practitioner’s influence over a particular field of study or discipline.
Keywords: Artificial Intelligence, Augmented Reality, Artificial Interactive Reality, Virtual Interaction, NLP, Vivek.
I. INTRODUCTION
Shapley et al. (2011) [1] proposed that education driven by technology can lead to innovative ways to explain a lesson and concisely
make the student understand. Artificial Interactive Reality (AIR) is a technical study which has its application in almost every
sphere of research. The recent resurgence in the domain of Artificial Intelligence has provided the researchers with transparency to
their existing algorithms [2]. The development of intelligent systems has always been a challenging ask for the developers, but the
Researchers are heavily trusting the algorithms to solve difficult problems and overtake manual interpretation. Cognitive Robotics
has always posited a potential challenge to Artificial Intelligence [3] and exploited the limitations of its growth. Despite such
limitations, Artificial Intelligence has overcome every major technical barrier and maintained a domain of supremacy in its era. To
explain the utilities of an Augmented Reality object, Artificial Intelligence can be used to introduce a level of interaction and
intelligence. Artificial Interactive Reality (AIR) is a novel non-existent and preeminent technology that incorporates AI into AR
contents which will allow the users to get a more intelligent response from the AR. The AIR system can be implemented with all AR
objects to bridge the existing gap between computer-generated imagery (CGI) [4] and machine intelligence. This technology will
have objects created on the concept of AR and the commands by interactive technologies will be passed to them by processing
through an AI system.
A. Augmented Reality
Next-generation of consumer technology will have more than one smart device with internet connectivity on them. Soon we will
also have AR integrated with smart consumer technology such that the AR will change one's ongoing perception of the real world
around them. Some of the most popular coined terms used in this field are AR & Virtual Reality (VR), [5] and surely they both are
not synonymous.
VR puts the perception of the user into a virtually simulated one, whereas AR changes one's ongoing perception of the real world
around them. Though extremely innovative due to the lack of research about this new technology, [6] and also because of the
limitation of software developing tools there haven't been many applications in this field.
Recent software developers do not possess the skills to build a 3D interactive environment and those who have the skills are not
interested in software productions. So, we can say that technology is still at its inception. In 1992 U.S. Air Force's Armstrong
Laboratory developed Virtual Fixtures [8] that was the first functional AR system. Contrary to VR, AR does not require any special
or expensive device other than our smartphone or simple AR glasses. AR allows us to interact with virtual images and objects in real-
time using a smartphone or AR glasses. [7].
B. Artificial Intelligence
Artificial Intelligence (AI) is a discipline under computer science that focuses on the development of smart, reliable and intelligent
machines that is programmed to work and react like humans. It has evolved as a paramount part of the technology industry. [9] AI is
eminently developed and specialized in its technical field of research. It should be programmed for certain traits like reasoning,
problem-solving skills, perception and various other.
The term Machine learning, coined by Arthur Lee Samuel [10, 11] is a branch of AI. We can make machines to resemble and react
like humans. Systems in AI can have cognitive intelligence which will help them to make future decisions based on its past
experiences, we can even combine emotional intelligence in addition to cognitive intelligence [12] and the finalized AI system can
have social intelligence and self- consciousness along with all the above-mentioned traits. An AR application can be greatly
developed by programming it with AI, which will bring us back to the concept of AIR
B. AIR Vision
Modern AR systems don’t normally have computer vision setup, which makes it difficult for AR objects to learn the surrounding
environment and extract high-dimensional data from the reality or real world around us and use it for further symbolic information
[15, 16, and 17].
So, through AIR Vision we tend to integrate computer vision using sensors with AR objects and enable the AR objects to
seamlessly [18] use the sensor data for scene interpretation and understanding. Tasks which can be expected from AR objects after
implementing computer vision [19, 22] with them to combine it as AIR vision are as follows:
1) Recognizing objects in the surroundings [20, 21] as well as their identification is the primary goal of this technology.
2) Predicting the orientation of a specific object as well as reading the human written language [20] and finally have the capability
for face recognition.
3) Analyzing motion such as tracking moving objects or moving points in the real world.
Above mentioned tasks are implemented such that the AR object itself will not have lenses or any hardware present for AIR vision,
but it will collect data from the device's camera which we will be using to interact with AR objects such as smartphone [23]. AI
systems will help to transfer these data collected from the device's hardware and transfer them accordingly with suitable commands
to the AR objects in real-time.
C. AIR Objects
Users can interact with objects that are designed by 3D modeling [24, 25] for AR applications in limited ways, one main reason
behind this is the lack of progress in this area. AIR objects will be having connectivity for the commands and functions directly with
the AI of the device. So, users can interact with these objects in ways similar to AR objects as well as eye-tracking can also be
achieved with the help of AI systems. Some of the interactions are as follows:
1) Controlling the AIR objects by touchpad devices [26], which can sense the pressure applied to it while tapping or swiping on a
specific area.
2) Hand gesture recognition will also be achieved in AIR objects, this concept is divided into two different algorithms, one that
recognizes the number of outstretched fingers and second which controls the point and clicks gestures [27].
Above mentioned interactions are already present in the AR object interactions. The interaction which will be different in AIR
objects is eye-tracking technology. This technology provides us with information on user behavior which covers all data like where
the user is looking, gazing time and we also study pupil patterns. This makes the experience more pleasurable. This eye-tracking
technology can only be used in smartphones [28] which comes with in-built iris scanner or AR glasses [29, 30] can be used as well.
A. AIR Guide
One of the prime applications of the AIR technology can be an AIR guide. This guide can be accessed in any building, offices,
museums, shopping complexes [41, 42]. We can access this guide through our smartphones or AR glasses to get guidance in the
respective place, such as to find the location of a room, cabin, article, artifacts. This will work in two steps, first, we will start the
AIR application which will launch a virtual guide (that is Vivek) in augmented reality and, second it can interact with us through the
AI implemented in the AIR application, where users can ask questions about that particular place and Vivek will interact with them
and provide solutions by taking them to respective places or giving them directions. Vivek object will be having a human figure in
the augmented reality created by AIR, which will give users a better perspective and clear understanding of the guidance. They will
also be able to customize Vivek according to their choice.
C. AIR Library
We can save paper by using the concept of AIR Library. The concept used in these libraries will be far different from our normal
libraries [44]. As any reader can come to the library and will be provided with a menu on the screen containing the sorted name of
books according to the genre and the user will select a book of his choice and then the librarian will provide him with the AIR
module application of that book. So, now he can read the book from his smartphone and have a 3D object of that book in front of him
as well as Vivek can be programmed to clear their doubts about the content in the book.
VI. CONCLUSIONS
This paper presents the concept of AIR technology and its development as well as implementation in real-life. This technology has
an exceptional prospect in the upcoming future. It has the potential to stand out among all the existing technologies in the software
industry. It needs to be further developed for making it cost-efficient and at the same time keeping its marvelous features untouched.
Multinational Companies like Microsoft, IBM, Unity, Google, Apple, and Facebook have shown their interest in the field of both AR
and AI and understands the aspect of them. AIR technology presented in this paper is a concept of combining these both powerful
domains to solve the necessities of the real-world and bring the technology industries to the verge of evolution.
REFERENCES
[1] Shapley, K., Sheehan, D., Maloney, C., & Caranikas-Walker, F. (2011). Effects of technology Immersion on Middle School Students’ Learning Opportunities
and Achievement. The Journal of Educational Research, 104,299-315.
[2] Tim Miller,Explanation in artificial intelligence: Insights from the social sciences,Artificial Intelligence,Volume 267,2019,Pages 1-38,ISSN 0004-3702G.
Zaharchuk, E. Gong, M. Wintermark, D. Rubin, and C. Langlotz, “Deep learning in neuroradiology,” American Journal of Neuroradiology, 2018.
[3] Séverin Lemaignan, Mathieu Warnier, E. Akin Sisbot, Aurélie Clodic, Rachid Alami, Artificial cognition for social human-robot interaction: An
implementation, Artificial Intelligence, Volume 247,2017, Pages 45-69, ISSN 0004-3702D. Shen, G. Wu, and H.-I. Suk, “Deep learning in medical image
analysis,” Annual review of biomedical engineering, vol. 19, pp. 221– 248, 2017.
[4] 4. Lehmuskallio, Asko; Häkkinen, Jukka; Seppänen, Janne, Photorealistic computer-generated images are difficult to distinguish from digital Photographs: A
case study with professional Photographers and photo-Editors, 2017, Visual Communication, Vol.18 (1)(Feb 2019). D. Sutton, “A textbook of radiology and
imaging,” 1987.
[5] X. Li, B. Xu, Y. Teng, Y. Ren and Z. Hu, "Comparative research of AR and VR technology based on user experience," 2014 International Conference on
Management Science & Engineering 21th Annual Conference Proceedings, Helsinki, 2014, pp. 1820-1827.
[6] Rosique, Francisca Khan, TasneemJohnston, Kevin Ophoff, Jacques, The Impact of an Augmented Reality Application on Learning Motivation of Students,
Advances in Human-Computer Interaction, Hindawi, 1687-5893, 2019/02/03.
[7] . Martín-Gutiérrez, P. Fabiani, W. Benesova, M. D. Meneses, and C. E. Mora, “AR to promote collaborative and autonomous learning in higher education,”
Computers in Human Behavior, vol. 51, pp. 752–761, 2015.
[8] Ohnson, Joel. “The Master Key”: L. Frank Baum envisions augmented reality glasses in 1901 Mote & Beam 10 September 2012.
[9] Dorobantu, Marius. “[AI & Theology Review Article] Recent Advances in Artificial Intelligence (AI) and Some of the Issues in the Theology & AI Dialogue.”
ESSSAT News & Reviews 29, no. 2 (2019): 4–15.
[10] Samuel, A.L., "Artificial Intelligence: A Frontier of Automation," Ann. American Acad. Political and Social Science, Vol. 340, Mar. 1962, pp. 10-20.
[11] Samuel, Arthur L., "Some Studies in Machine Learning Using the Game of Checkers," in Feigenbaum, Edward A., and Julian Feldman, eds., Computers and
Thought, McGraw-Hill, New York, 1983, pp. 71-105.
[12] Checa Purificación, Fernández-Berrocal Pablo, Cognitive Control and Emotional Intelligence: Effect of the Emotional Content of the Task. Brief Reports,
Frontiers in Psychology, Vol- 10, 2019, ISSN 1664-1078.
[13] Milgram, Paul & Takemura, Haruo & Utsumi, Akira & Kishino, Fumio. (1994). Augmented reality: A class of displays on the reality-virtuality continuum.
Telemanipulator and Telepresence Technologies. 2351.
[14] Yoshua Bengio BENGIOY Réjean Ducharme DUCHARMEPascal Vincent, Christian Jauvin, A Neural Probabilistic Language Model, Journal of Machine
Learning Research 3 (2003) 1137–1155.
[15] H. Wu, S. W. Lee, H. Chang, and J. Liang, “Current status, opportunities and challenges of AR in education,” Computers and Education, vol. 62, pp. 41–49,
2013.
[16] J. Bacca, S. Baldiris, R. Fabregat, et al., “AR trends in education: a systematic review of research and applications,” Journal of Educational Technology and
Society, vol. 17, no. 4, pp. 133–149, 2014.
[17] M. Billinghurst, A. Clark, and G. Lee, “A survey of AR,” Foundations and Trends in Human-Computer Interaction, vol. 8, pp. 73–272, 2015.
[18] P. Besnard, A. HunterElements of Argumentation, vol. 47MIT Press, Cambridge (2008).
[19] Lepetit, Vincent. (2008). On Computer Vision for Augmented Reality. 13 - 16. 10.1109/ISUVR.2008.10.
[20] S. Hinterstoisser, S. Benhimane, N. Navab, P. Fua, and V. Lepetit. Online learning of patch perspective rectification for efficient object detection. In
Conference on Computer Vision and Pattern Recognition, 2008.
[21] F. Jurie and M. Dhome. Hyperplane approximation for template matching. IEEE Transactions on Pattern Analysis and Machine Intelligence, 24(7):996–100,
July 2002.
[22] G. Reitmayr and T. Drummond. Initialisation for visual tracking in urban environments. In International Symposium on Mixed and Augmented Reality, 2007
[23] K. Mikolajczyk, T. Tuytelaars, C. Schmid, A. Zisserman, J. Matas, F. Schaffalitzky, T. Kadir, and L. Van Gool. A comparison of affine region detectors. IJCV,
65(1):43–72, 2005.
[24] M. Ozuysal, V. Lepetit, F. Fleuret, and P. Fua. Feature Harvesting for Tracking-by-Detection. In ECCV, 2006.
[25] F. Rothganger, S. Lazebnik, C. Schmid, and J. Ponce. Object modeling and recognition using local affine-invariant image descriptors and multi-view spatial
constraints. IJCV, 66(3):231–259, 2006.
[26] Arsénio, Artur Miguel & Serra, Hugo & Francisco, Rui & Nabais, Fernando & Andrade, João & Serrano, Eduardo. (2014). Internet of Intelligent Things:
Bringing Artificial Intelligence into Things and Communication Networks.
[27] P. Fitzpatrick, A. Arsenio, and E. Torres-Jara. Reinforcing robot perception of multimodal events through repetition and redundancy and repetition and
redundancy, Interaction H. Lu, W. Pan, N. Lane, T. Choudhury, and A. Campbell. SoundSense: scalable sound sensing for people-centric applications on
mobile phones. In Proceedings of the 7th international conference on Mobile systems, applications, and services, pages 165–178. ACM,(2009)Studies, vol. 7,
n.2 (2006).
[28] H. Lu, W. Pan, N. Lane, T. Choudhury, and A. Campbell. SoundSense: scalable sound sensing for people-centric applications on mobile phones. In
Proceedings of the 7th international conference on Mobile systems, applications, and services, pages 165–178. ACM,(2009).
[29] Ro, Young & Brem, Alexander & Rauschnabel, Philipp. (2017). Augmented Reality Smart Glasses: Definition, Concepts, and Impact on Firm Value Creation
[30] Xi-Dao LUAN, Yu-Xiang XIE, Long YING, and Ling-Da WU, Research and Development of 3DModeling, IJCSNS International Journal of Computer
Science and Network Security, VOL.8 No.1, January 2008,page-49-53.
[31] Yusuf Arayici, Andy Hamilton. Modeling 3D Scanned Data to Visualize the Built Environment. Proceedings of the Ninth International Conference on
Information Visualisation, 2005, 509-514.
[32] Liu Gang, Wang Zhangye, Peng Quensheng. Generating Visual Hulls from Freely Moving Camera. Journal of Computer-Aided Design & Computer Graphics,
2004,16(11),1501-1505.
[33] Sabatucci, Luca & Cossentino, Massimo. (2015). From Means-End Analysis to Proactive Means-End Reasoning.
[34] J. Whittle, P. Sawyer, N. Bencomo, B. H. Cheng, and J.-M. Bruel, “Relax: Incorporating uncertainty into the specification of self-adaptive systems,” in
Requirements Engineering Conference, 2009. RE’09. 17th IEEE International.IEEE, 2009, pp. 79–88.
[35] Duffany, Jeffrey. (2010). Artificial intelligence in GPS navigation systems. 1. 10.1109/ICSTE.2010.5608862.
[36] Ishaq, Faisal & Alhaji, Ibrahim & Altun, Halis & Muhammad, Jawad & Sani, S... (2016). Analysis On Object Detection Using Artificial Intelligence For
Industrial Applications. Journal of the Nigerian Association of Mathematical Physics.
[37] Dalal, Navneet & Triggs, Bill. (2005). Histograms of Oriented Gradients for Human Detection. Comput. Vision Pattern Recognit.. 1. 886-893.
[38] Al Smadi, Kalid & Al Issa, Huthaifa & Trrad, Issam & Al Smadi, Prof-Takialddin. (2015). Artificial Intelligence for Speech Recognition Based on Neural
Networks. Journal of Signal and Information Processing. 06. 66-72.
[39] Childer, D.G. (2004) The Matlab Speech Processing and Synthesis Toolbox. Photocopy Edition, Tsinghua UniversityPress, Beijing, 45-51.
[40] Takialddin Al Smadi Int. An Improved Real-Time Speech Signal in Case of Isolated Word Recognition. Journal of Engineering Research and Applications, 3,
1748-1754.
[41] Seo, Byung-Kuk & Kim, Kangsoo & Park, Jong-Il. (2010). augmented reality-based on-site tour guide: A study in Gyeongbokgung. 276-285.
[42] Zoellner, M., Keil, J., Drevensek, T., Wuest, H.: Cultural heritage layers: Integrating historic media in augmented reality. In: International Conference on
Virtual Systems and Multimedia. (2009) 193–196.
[43] Hagl, Richard & Duane, Aidan. (2018). Exploring the Impact of Augmented Reality and Virtual Reality Technologies on Business Model Innovation in
Technology Companies in Germany.
[44] Wheatley, Amanda, and Hervieux, Sandy. ‘Artificial Intelligence in Academic Libraries: An Environmental Scan’. 1 Jan. 2019: 347.