2019 7th International Conference on Information and Communication Technology (ICoICT)

Analyzing Tourism Mobile Applications Perceived

Quality using Sentiment Analysis and Topic
Modeling
Riefvan Achmad Masrury1, Fannisa2, Andry Alamsyah3
School of Economic and Business Telkom University, Bandung, Indonesia
1 [email protected], 2 [email protected], [email protected]

Abstract—Mobile application is one of the most important Reviews within the mobile application marketplace
information platforms for international tourists. Millions of usually reflect the user's positive and negative emotions [5].
tourists use mobile applications to find information and make For example, bug reports may contain negative sentiments,
transactions. Two popular Online Travel Agent (OTA) mobile while user experience reviews may be combined with
applications for travel-related activities providers are positive sentiments [6]. Sentiment Analysis as a method in
Traveloka and Tiket.com. These applications certainly must Text Mining technique can be used to classify unlabeled data
meet travelers’ needs to achieve satisfaction. Such satisfaction based on learning classification machine learning model
related to application Mobile Application Service Quality concept using labelled training data. A trained sentiment
(MappSql) dimensions can be traced from thousands of their
analysis model will predict large number of unlabeled
comments on the Google Play Store. From a set of reviews,
reviews in terms of positive and negative polarities
information about the perception of mobile application quality
can be obtained. Knowledge on user perceptions is very useful
automatically and quickly. This method can be applied in
for company’s consideration in creating effective business and exploring application users’ emotions so that developers can
app features to increase users’ satisfaction. We propose Text find out whether their applications invite more positive or
Mining models to bring up hidden information regarding negative sentiments.
users’ verdict. The selected text analysis methods for this We also select Topic Modelling method to reveal outline
research are Sentiment Analysis and Topic Modeling. We find of users’ reviews topics. We had focused our research on two
that positive or negative sentiments towards MappSql
of the most popular OTA providers in Indonesia which were
dimensions of online travel agent applications qualities can be
revealed using sentiment analysis method. Topic Modeling
Traveloka and Tiket.com as reported by DailySocial.id
method is used to bring up groups of important words of topics (2018). Traveloka and Tiket.com present the trending OTA
related to each mobile application service quality dimensions. e-commerce services. Results from both methods will serve
as the basis of quality perceived comparison analysis
Keywords—Mobile Application Quality, Text Mining, User purposes.
Satisfaction. We consider topics discovery towards MappSql since it
provides a scale and dimensions to assess mobile application
I. INTRODUCTION service quality. MappSql is tailored for use in measuring
Indonesia has a positive growth of internet users and mobile application service quality and is inferred from e-
tourism trends every year. Online Travel Agent (OTA) is one Service quality dimensions used in state-of-the-art
of the growing businesses in tourism industry. Over the past researches. To the best of our knowledge, our research is the
few years, tourism and hospitality industry companies have first work that incorporates sentiment analysis and topics
been taking part and developing their competitiveness in the modelling towards MappSql for a more comprehensive and
applications market for e-commerce and online transactions unified measurement purpose.
activities [1]. OTA is a manifestation of the tourism business Using appropriate data analytics tools, one can explore
development using e-commerce technologies. In accessing customer feedbacks in almost real-time manner. This will
OTA services, people prefer to use a mobile application benefit OTA mobile applications developers in better tuning
compared to other platforms [2] which makes it very crucial features of their respective products and services. In business
for companies to improve as part of their service management context, quality of general decision making will
development. be improved correspondingly. Any data analytics method
Along with the development of technology, mobile requires a reliable source of data. Since Google Play Store
application becomes an important role in accessing stores huge amount of users’ bug reports, feature requests,
information. This is also true in term of users’ reviews. and praises, it has becoming an important data source for
Based on a study, online users’ reviews have a significant mobile application engineering decisions [4].
influence on consumer product sales [3], so companies need
to pay attention in gaining valuable insights. Google Play II. LITERATURE REVIEW
Store holds very large number of customers’ reviews data.
Social media allows customers to share opinions and
The number is growing fast. There are challenges to filter out
provide feedback about their usage experiences of many
low-quality reviews which may include unreasonable
information, uncorrelated information to the product or a products and online services. Due to advances in database
service, spam messages, or redundant information. With management techniques, social media channels business
hundreds or thousands of new reviews every day, application models, and mobile devices technologies, a lot of customer-
developers and analysts will find it harder to filter and related information is increasingly accumulating in social
process useful information from these reviews [4]. media companies. The use of social media channels

978-1-5386-8052-0/19/$31.00 ©2019 IEEE

technologies and big data analytics tools complement each In the technique, sentiment analysis classifies text into
other because the variety of social media channels allows two classifications, namely positive and negative. Data on
companies to collect user-generated content (UGC) data. existing reviews must be trained in order to get more fit
The use of big data analytics tools allows them to analyze results in machine learning. For example, user satisfaction
data and gain customers insight effectively and efficiently will include positive sentiments, while reviews of errors in
[7]. the application will contain negative sentiment.

A. User-Generated Content D. Topic Modeling

Content is considered UGC when the content is published Application developers need to know clearly what topics
on a public, accessible website or on a social networking are often discussed by users to determine areas to improve.
website accessible by a selected group of people. UGC offers For example, entities in a topic group might be important
opportunities and challenges for the company. In fact, the evidences to consider. Conclusion can be made accordingly.
ability of the company to respond to these challenges may
depend on the purpose of the contribution of the user. UGC Topic modeling can be used to see the main topics in user
represents an important paradigm shift in which companies reviews. The practice is to identify hidden event patterns
must be willing to give up certain strengths that are from a set of words using word distribution in a set of
traditionally tightly controlled by companies to customers documents [16]. The output is a set of topics consisting of
[8]. The main purpose of UGC is product promotion because groups of words from a document according to certain
user-generated content usually shows strengths/weaknesses patterns [17]. For each document in the corpus, topic
related to a product or brand and is designed to encourage modeling is also very useful in identifying hidden subjects
mouth-to-mouth activity [9]. UGC can function in e- and themes based on related words in an event. Furthermore,
commerce as a new form of word-of-mouth for its products / topic modeling is a good way to "let the text talk" because
services or providers [10]. the identified topics are not dependent on individual
perspectives or experiences [18].
B. Exploring UGC with Text Mining Number of topics were chosen among a range of optimal
The Text Mining technique can be used for exploring numbers of topic groups. These numbers was acquired using
insights from large number of UGC reviews. It enables empirical means using 4 metrics. Both maximization and
business person or application developers to identify minimization techniques are available. The first
application problems in a more detailed way by discovering maximization metric is Arun2010 to observe divergence
users’ emotions or perceptions from their reviews. Sentiment values in terms of symmetric KL-Divergence of salient
Analysis is usually chosen for exploring the sentiment using distributions. The second maximization metric is the
a set of textual data. This model determines whether a text CaoJuan2009 method in selecting best LDA models based on
contains positive, negative or neutral polarization. It has density. Two metrics in minimimization techniques are the
becoming very popular in the "customer voice" type of Griffiths2004 which uses Gibbs sampling algorithm and the
application [11]. The basic tasks of Sentiment Analysis Perplexity (Deveaud2014) to evaluate probabilistics model to
models relate to classifying texts, for example, subjective measure the log-likelihood of a held-out test set [19]. The
(opinion) or objective (facts) classification or categorization, optimal range is situated between the number of topics at the
using machine learning or natural language process (NLP) intersection of the Arun2010 plot and CaoJuan2009 plot for
techniques. minimization, and the number of topics at the intersection of
Griffiths2004 plot and Deveaud2014 plot for maximization
In addition to the use of sentiment analysis method, [20].
Topic Modeling method is necessary to identify the main
topics of the reviews, our understanding of users ' perception
of mobile application quality has been enhanced. So in its III. RESEARCH DESIGN AND METHODOLOGY
application, one will be able to distinguish what the main
problems keywords are. In other words, Topic Modeling is a A. Research Question
technique of text mining to find and track word groups in a Our objective was to analyze perceived quality towards
document [12]. tourism mobile applications using Sentiment Analysis and
Topic Modeling methods. Naïve Bayes was selected as the
C. Sentiment Analysis classifier for sentiment analysis and Latent Dirichlet
Allocation (LDA) as the modeler of the topics. We aimed for
Sentiment analysis plays a large role in the field of
several outcomes:
research carried out by many researchers. Likewise, with
application user reviews, application reviews usually contain 1. The accuracy of Sentiment Analysis model to classify
positive and negative emotions of users [5]. Therefore, this both Traveloka and Tiket.com users’ sentiments
study uses sentiment analysis to see the emotions of based on Naïve Bayes algorithm
application users. With the sentiment analysis can detect the
contextual patterns that exist in a text. In exploring user 2. Identification of sentiments toward large numbers of
reviews, we use a machine learning based approach because Traveloka and Tiket.com apps reviews in Google
this approach has a high degree of accuracy in classifying a Play Store
text [13]. And for classifying the text, we use Naïve Bayes 3. Generation of topic models for each quality
because it is a good algorithm for classifying problems dimensions
regarding service quality [14]. With the classification, we
can solve problems that occur in services [15]. A user review may contain negative, or positive
sentiment. For an example, "Thank you Tiket.com, this app
helps me arrange a field trip easier" contains a positive
sentiment, while "I only want to know the price of a boat Bayes based Sentiment Analysis outputs a predictive
ticket but it’s really difficult" contains a negative sentiment. classification model while Topic Modelling using LDA
The machine will do testing based on the data learned so that method gives an estimation model of topics clustering. We
testing data will appear with the level of how accurate the used Naïve Bayes Classifier (NBC) algorithm for classifying
sentiment analysis. customers’ reviews data and classifying them according to
the six dimensions of MappSql, that are customization,
B. Research Method service recovery, functionality, design, fulfillment, and
To answer the research question above, we carry out data assurance. We ran an R script in RStudio to generate models
collection for further cleaning so that the data to be of topics and to visualize the results. Several packages are
processed is truly relevant data which will then be explained manually installed such as tidyverse, tidytext, topicmodels,
by the research method in fig. 1. tm, SnowballC, etc. Classification model was generated and
evaluated in Rapidminer Studio.
We managed to collect data from Google Play Store for
Traveloka and Tiket.com reviews. Data were collected with C. Evaluation Metrics
timestamps in ten months period from January 1st, 2018 to Our classification model performance is evaluated by
October 24th, 2018 with a total of 12,311 user reviews. using confusion matrix with the following formula:
Newly collected data is a raw data. Raw data is cleaned by
removing irrelevant data such as advertisements, spam, or
simply redundant messages. Training data quality will affect (1)
sentiment analysis model’s accuracy significantly. Further TP (true positive) and TN (true negative) are numbers of
pre-processing data involved converting unstructured data true positive and negative predicted labels. Whereas FP
into structured one and eliminating unnecessary data (false positive) and FN (false negative) are numbers of the
variables. Data pre-processing was done in three stages as model’s false predictions.
shown in Fig. 2.
Precision and recall values can be calculated using the
Data Collection following metrics:

(2)
Data Cleaning
Accuracy measures the correctness of the classification
models in predicting labels. On the other part, precision
Data Preprocessing
shows the ratio of correct prediction and total positive or
negative predictions. Recall shows the ability of the model to
recognize each members of positive or negative classes
Data Training
among a set of unlabelled data [21].

IV. RESULT AND ANALYSIS

Text Mining Analyzing Based on the values of True Positive (TP), True Negative
(TN), False Positive (FP), and False Negative (FN), the
results of machine learning performance evaluation in text
Result and Conclusion
classification can be seen from accuracy, precision, and
recall. Accuracy values describe how accurate the machine
Fig. 1. The research workflow
can classify data correctly. In other words, the value of
Cleaned data are labelled according to positive-negative accuracy is a comparison between number of correctly
sentiment and quality dimensions. A ratio of 70:30 is used to classified data and the number of overall data. The
split collected data into training and testing sets. measurement of the text classification performance model
Tokenization is a process to slice reviews into individuals can be seen in Table I and Table II.
so that the stopwords filter can be used to filter out words
TABLE I. EVALUATION AND ACCURATION RESULT OF TRAVELOKA
that do not have a significant effect to a sentiment polarity
such as "I", "was", and "them". The next process called Accuracy: 88.00% +/- 2.95% (micro average: 88.00%)
True Positif True Negatif Class Precision
Stemming provide prefixes and suffixes removal mechanism (TP) (TN)
to get a basic-words bag (Bag of Words – BOW). Pred. Positive 848 47 94.75%
Pred. Negative 102 245 70.61%
Class Recall 89.26% 83.90%

Tokenization TABLE II. EVALUATION AND ACCURATION RESULT OF TIKET.COM

Accuracy: 91.06% +/- 2.58% (micro average: 91.06%)
True Positif True Negatif Class Precision
Filter (TP) (TN)
Stemming Stopwords Pred. Positive 1439 97 93.68%
Pred. Negative 81 373 82.16%
Class Recall 94.67% 79.36%
Fig. 2. Preprocessing flow
Can be seen from Table I and II, Traveloka has true
Sentiment Analysis and Topic Modelling were conducted positive (TP) and true negative (TN) predicted labels value
in the 5th steps. Both will give different result types. Naïve of 848 and 245 data. Which the classifiers count it to false
predicted labels value of 102 and 47 data for false positive Tiket.com has 65% positive and 35% negative sentiment.
(FP) and false negative (FN). Whereas Tiket.com has TP, This shows that while using the application, users already
TN, FP, and FN value of 1439, 373, 81, and 97 data. feel quite satisfied with the application service so that 65%
Based on the calculation of TP, TN, FP, and FN. users’ give a positive response.
Traveloka has a classification accuracy value of 88% and
91.06% for Tiket.com. It means Naïve Bayes Classifier has a
very good performance. This is proven by the results of
Traveloka text classification accuracy values that have a
value of 88% and are categorized as Good, and Tiket.com
entered into the Excellent category because it has an
accuracy value of 91.06%.

Fig. 6. Tiket.com MappSql Classification

As seen in Fig. 6. Design are the most often discussed

from the other five dimensions, i.e. 40%. Tiket.com has an
attractive application design and user friendly. While in other
dimensions, Tiket.com has a classification of 19% for
Functionality, 18% for Assurance, 12% for Fulfillment, 9%
Fig. 3. Traveloka Sentiment Classification for Service Recovery, and 2% for Customization dimensions
During the last ten months, can be seen in Fig. 3
Traveloka has 92% positive and 8% negative sentiment of
users’ application. This shows while using the, users’ already
feels satisfied with the application service, so that they give a
positive response in the form of positive reviews.

Fig. 7. Optimal Number of Topics for LDA Model

Topic modellings are done in RStudio entirely. Each

topic models are generated in under 1 second using a laptop
computer using 6th generation of Core I5 processors, and
8GB of RAM. Optimal numbers of topic groups are between
4 and 9 based on maximization and minimization techniques.
Fig. 4. Traveloka MappSql Classification We chose 5 for the sake of simplicity of discussion.
Fig. 4 shown classification of the MappSql’s dimension
of Traveloka. Fulfillment are the most often discussed
compared to other dimensions, i.e. 32%. Observations state
that Traveloka is an application that fulfills its service
promises, i.e. providing fast and easy services, and the
availability of tickets that always available. As for the other
dimensions, Traveloka has a classification of 24% for
Design, 20% for Assurance, 14% for Functionality, 8% for
Service Recovery, and 2% for Customization.

Fig. 8. Topic Modeling for Complete Reviews for Traveloka Case

Fig. 8 shows the results of the Traveloka’s topic

modeling. The number on the x-axis shows how often the
word appears on a topic. Bigger Beta values indicate higher
Fig. 5. Tiket.com Sentiment Classification appearance frequencies of particular words in a single topic
group. All the topic discussed was reflected in the words that several smartphone types or brands, causing errors that force
appearing of the topics above and have been seen in Fig. 8. users to uninstall.
The 1st topic contain words such as bagus, tiket, pesawat,
tiket, hotel, membantu, mudah, pesan, harga, and cepat.
Adjective words specify abstract levels of customer
satisfaction (bagus, mudah, membantu, and cepat) in relation
to possible subjects such as tiket, hotel, and pesawat. The
topics group may also implies important activities.
The 2nd topic caters payment activites for tickets and
hotels. Users’ satisfactions on its availability seems to have
reached high level of satisfaction reflected by mantap, top,
puas, and terbantu. On the 3rd topic discussed about
transactions in the application. Many users dissatisfied about
it because they could not use the credit card during the
payment process. Reviews regarding PayLater's new
features also contain negative sentiments. Nonetheless, some
users expressed satisfaction with the PayLater’s feature.
Most positive sentiments came from reviews related to easy
to use transaction system. Fig. 10. Topic Modeling for Complete Reviews for Tiket.com Case
The 4th topic discusses about customers having Optimal numbers of topic groups for Tiket.com case are
dissatisfaction with application system performance. User between 5 and 10. We also chose 5 for the sake of simplicity
feels disappointed with the availability of ticket wich sold of discussion. The 1st topic shows the weaknesses of the
out quickly during certain days, such as holidays. application system problems related to electronic ticket
Applications also has a bad server performance on the high issuance. The 2nd topic discussed about low responsiveness
traffic visitors The 5th topic contains users’ negative of customer services in processing complaints.
perceptions related to complaints management, pricing
scheme and lowest price guarantee compared to other The 3rd topic shows user satisfaction to the application
booking sites. user-friendliness and the facilitation functionality they had
by using the application. The 4th topic discussed about
application design. Users consider the design of Tiket.com's
application is quite good as shown by the high frequency of
word "bagus" appearance. The 5th topic discussed about
satisfaction of users about performance of application
services.

Fig. 9. Topic Modeling for Traveloka’s MappSql Dimensions

Fig. 9 shows the topic modeling for Traveloka's

MappSql. Most of the dimensional topics shows positive
sentiments. In terms of Assurance’s dimension, Traveloka
has a good image because it has fulfilled the service
promises. Customization’s dimension discussed about user
requests for Traveloka to add and complete in-app Fig. 11. Topic Modeling for Tiket.com’s MappSql Dimensions
information regarding seating on the train and terminal at the Seen in Fig. 11, for the dimension of Assurance,
airport. And to add hours of online transfers such as at night application has fulfilled the promise of service delivery and
and clarify information about virtual accounts. Design’s provided the best solution for customers. The topics in the
dimension discussed about the application already has a good Customization dimension discussed about the flexibility of
design. Topics that exist in the Fulfillment’s dimension payment. Tiket.com should add virtual account payment
shows customer appreciation with PayLater’s features. options in order to reach all banks and more payment options
Service Recovery dimension shows users thankfulness for through retail stores. Design’s dimension discussed about
customer services’ good responses in serving customer application’s good design.
complaints.
Fulfillment’s dimension contains user dissatisfaction
In contrast to other dimensions, Functionality dimension towards ticket information renewal. In terms of
show negativism about application compatibility issues in Functionality, user’s discussed about performance of
application systems that often erroneous, slow, and [3] Ye, Q., Law, R., Gu, B., & Chen, W. (2011). The influence of user-
inaccessible. Electronic tickets status not showing up makes generated content on traveler behavior: An empirical investigation on
the effects of e-word-of-mouth to hotel online bookings. Computers
users feel disappointed with the performance of the in Human Behavior, 27(2), 634–639.
application system. Service Recovery’s dimension discussed [4] Maalej, W., dan Nabil, H. (2015). Bug Report, Feature Request, or
about customer service department’s bad responses in Simply Praise? On Automatically Classifying App Reviews. IEEE
serving customer complaints. International Requirements Engineering. 116-125.
[5] Guzman, E., dan Maalej, W. (2014). How do users like this feature? a
grained sentiment analysis of app reviews. In Requirements
V. CONCLUSION Engineering Conference (RE), 2014 IEEE 22nd International, pages
Large numbers of UGC data gathered from social media 153–162. Elsevier.
enables data science practitioners to apply text mining [6] Pagano, D., dan Maalej, W. (2013). User feedback in the
techniques to reveal users’ perceived quality through applicationtore : an empirical study. International Conference on
Requirements Engineering. Pages 125–134.
sentiments and popular topics towards several quality
[7] Dong, J. Q., dan Yang, C.-H. (2018). Business value of big data
dimensions. analytics: A systems-theoretic approach and empirical test.
Proper data pre-processing activities including data Information & Management.
cleaning processes are required to achieve more than 80% [8] O’Hern, M. S., & Kahle, L. R. (2013). The Empowered Customer:
User Generated Content and the Future of Marketing. Global
Naïve Bayes classifier accuracy. Best accuracy obtained for Economics and Management Review, 18(1), 22–30. Retrieved from
Traveloka case was 88% and 91.06 % for Tiket.com case. ScienceDirect.
We conclude that Naïve Bayes classifier is reliable for [9] Chevalier, J. A., & Mayzlin, D. (2006). The Effect of Word of Mouth
classifying users’ sentiment toward MappSql quality on Sales: Online Book Reviews. Journal of Marketing Research, 43,
dimensions from textual data. 345-354.
[10] Ye, Q., Law, R., Gu, B., & Chen, W. (2011). The influence of user-
Aggregation of sentiments numbers can be used to generated content on traveler behavior: An empirical investigation on
compare proportions of positive and negative sentiment. the effects of eword-of-mouth to hotel online bookings. Computers in
Traveloka has 92% positive sentiment. Whereas Tiket.com Human Behavior, 27(2), 634–639. doi:10.1016/j.chb.2010.04.014
has only 65% positive sentiment. Tiket.com has 35% [11] Hurwitz, J., Nugent, A., Halper, F., dan Kaufman, M. (2013). Big
negative sentiment instead of 8% as what Traveloka has. Data for Dummies. John Wiley & Sons, Inc. River Street, Hoboken.
This translates to higher overall users’ satisfaction in [12] Brett, M. R. (2012). Topic Modeling: A Basic Introduction. Journal of
Traveloka apps usage compared to satisfaction in Tiket.com Digital Humanities. Vol. 2. [Online]. Available on:
http://journalofdigitalhumanities.org/2-1/topic-modeling-a-basic-
apps usage. introductionby-megan-r-brett/. [5 November 2018].
In terms of MappSql’s dimension, Fulfillment dominates [13] Devika, M. D., Sunitha, C., & Ganesh, A. (2016). Sentiment
among other five dimensions in Traveloka case by 32%. Analysis: A Comparative Study on Different Approaches. Procedia
Computer Science, 87, 44–49. doi:10.1016/j.procs.2016.05.124
Whereas Tiket.com case shows Design dimension is the
[14] Sari, P. K., Alamsyah, A., & Wibowo, S. (2018). Measuring e-
most important aspect having 40% portion. Both OTA Commerce service quality from online customer review using
providers have different top priorities to consider in sentiment analysis. Journal of Physics: Conference Series, 971,
improving their MappSql qualities. 012053.
[15] Alamsyah, A., & Rachmadiansyah, I. (2018). Mapping online
Each sets of dimension-related reviews also contain transportation service quality and multiclass classification problem
specific topics related to quality dimension’s attributes or solving priorities. Journal of Physics: Conference Series, 971,
values such as the availability, flexibility, payment methods, 012021.
transaction methods, functionality, or user interface design. [16] Alamsyah, A., Rizkika, W., Nugroho, D. D. A., Renaldi, F., &
These can be improvement focus areas. OTAs can tune their Saadah, S. (2018). Dynamic Large Scale Data on Twitter Using
business features based on topics and related positive and Sentiment Analysis and Topic Modeling. 2018 6th International
Conference on Information and Communication Technology
negative sentiments using both factual and actual data from (ICoICT).
large number of reviews. [17] Jacobi, C., Van Atteveldt, W., & Welbers, K. (2015). Quantitative
Reviews may be gathered from websites and mobile app. analysis of large amounts of journalistic texts using topic modeling.
Digital Journalism, 4(1), 89–106.
Sentiment analysis and Topic Modelling towards MappSql doi:10.1080/21670811.2015.1093271.
processes can be implemented for both website and mobile [18] Hagen, L. (2018). Content analysis of e-petitions with topic modeling:
app qualities measurement, thus unifying and simplifying How to train and evaluate LDA models? Information Processing &
business improvement tasks. This will leads to better Management. doi:10.1016/j.ipm.2018.05.006
management for increasing user satisfaction where seamless [19] Nidhi. (2017). Number of Topics for LDA on poems from Elliston
integrations in multi-platform business is crucial. Poetry Archive. [Online]. Available on:
http://www.rpubs.com/MNidhi/NumberoftopicsLDA. [March 3rd,
2019]
REFERENCES [20] Nikita, M. (2016). Select number of topics for LDA model. [Online].
[1] Cetinsoz, Burcin Cevdet. (2015). The Influence of E-Service Quality Available on: https://cran.r-
on Customer Perceived Value: A Study on Domestics Tourists in project.org/web/packages/ldatuning/vignettes/topics.html. [March 3rd,
Turkey. International Journal of Science and Research, 4(1), 1265- 2019]
1277. [21] Williams, Graham. (2011). Data Mining with Rattle and R: The Art of
[2] DailySocial.id. (2018). Online Travel Agencies Survey 2018. Excavating Data for Knowledge Discovery. LLC:Springer Science
[online]. Available: https://dailysocial.id/report/post/online-travel- Business Media.
agencies-ota-survey-2018. [17 Oktober 2018].