Lexicon-Based Sentiment

Analysis of Teachers’ Evaluation

About Article
• Citation
• Quratulain Rajput, Sajjad Haider, and Sayeed Ghani, “Lexicon-Based
Sentiment Analysis of Teachers’ Evaluation,” Applied Computational
Intelligence and Soft Computing, vol. 2016, Article ID 2385429, 12 pages,
2016. doi:10.1155/2016/2385429
• Authors’ Affiliation
• Faculty of Computer Science, Institute of Business Administration (IBA),
Garden/Kiyani Shaheed Road, Karachi 74400, Pakistan
• Received 13 July 2016
• Revised 31 August 2016
• Accepted 5 September 2016
Introduction
• Faculty Performance Evaluation
• Typically collected at the end of each course
• Used as a metric to gauge the teaching quality
• Open feedback typically is not included due to lack of automated text
analytics methods
• May contain useful insight about
• Subject knowledge of the teacher
• Regularity
• Presentation skills
• Suggestions to improve the teaching of quality.
Introduction
• Aims and Objectives
• Analyze textual feedback automatically
• Develop quantitative and qualitative metrics
• Sentiment analysis
• Determine a writer’s feeling from a given text
• Classify the polarity of the given text as
• Positive
• Negative, or
• Neutral
• Identify the recurring theme in students’ feedback by generating
• word clouds for visualization
• sentiment score, and
• other frequency-based filters.
Literature Review
• Existing approaches for sentiment analysis can be classified into five
main categories:
• Keyword Spotting
• Lexicon-Based Approaches
• Lexical Affinity
• Statistical Methods
• Concept Level Techniques
• The fundamental step of the sentiment analysis is to identify the
polarity on either a word level, sentence level, or document level.
Literature Review
• Various subjectivity corpora are developed that annotate the lexicon
• Two arguments
• Corpus Based Domain Independent
• Domain Specific
• Freely available annotated corpora
• MPQA newswire corpus
• movie review corpus and
• restaurant and laptop review corpus
• No such sentiment corpus is available exclusively for the educational
domain
Literature Review
• Prominent works
• Leong et al.
• Applied sentiment analysis and text mining by collecting student’s feedback that is collected through
SMS.
• Explored the incomplete text and spelling errors.
• feedbacks categorized based the concepts defined for each category
• Jagtap and Dhotre
• SVM and HMM based hybrid approach for sentiment analysis of teachers’ assessment.
• Altrabsheh et al.
• Analyzed student’s feedback by collecting via social media such as Twitter.
• They not only identified student’s feeling in terms of positive and negative but also identified some more
refined emotions be
• negative such as confused, bored, and irritated
• positive emotions such as confident and enthusiastic are considered.
• techniques used in sentiment analysis
• Naive Bayes (NB)
• Max Entropy (MaxEnt), and
• Support Vector Machines (SVM).
Methodology
Sample students feedback

1 A good teacher
2 I felt that the instructor could’ve done a better job in making the concepts clear
3 Dull course
4 A course is fine
5 Difficult course, great teacher and also able to relate it to the practical knowledge
This course increased my knowledge. Teacher is also very helpful I like this course very very
6
much  :)
7 Institute should reconsider its policy of fyp’s
Methodology
Methodology
1. Preprocessing 2. Sentiment Dictionary
• Remove the unwanted and noisy • List of words along with their
data respective polarity.
• Tokenization • Corpora consisting of 8221 records
• Stemming (words) where each record
• Case Conversion consists of six features as listed in
the following:
• Punctuation Removal • Type
• Stop Word Removal • Length
• Subjective clue
• Part of Speech
• Stemmed
• Prior Polarity
Methodology
3. Polarity Tagging 5. Word Attitude
• Analyzes each word in feedback • Converts the polarity of each word
• Tags the word as positive, into a numeric value
negative, and neutral using its
polarity in the sentiment
dictionary. 6. Overall Attitude
• Overall attitude of a word is
obtained by multiplying its
4. Word Frequency attitude with its frequency
• Computes the frequency of each
word in each comment
Methodology
7. Word Cloud Visualization 8. Sentiment Score
• Visualization of overall attitude of • Feedback comment is assigned a
sentiment word as a cloud sentiment score by adding the
overallAttitude of each word in a
feedback.
Results
1. Sentiment Dictionary
Modification
• Challenges
• Informal way of writing
• Polarity of some of the vocabulary
used by students in education
environment needs to be modified
in sentiment subjectivity corpus.
• Key Observation
• Comments are typically short as
students avoid writing long
sentences.
Results
Results
Results
Results
Conclusion
• Performed sentiment analysis on faculty evaluation provided by students at
the end of a course.
• Computation of sentiment score to classify the feedback as either positive,
negative, or neutral.
• Performance, accuracy, recall, precision, and F-measure were computed
and the results were found to be very positive.
• Sentiment score is comparable to aggregated Likert scale based score.
• The sentiment score, in addition to word clouds, gives more insight that is
not possible with the Likert-based score
• A modified subjectivity corpus to be used in the academic domain to
achieve better results.
References
1. S. MacKim and R. A. Calvo, “Sentiment analysis in student experiences of learning,” in Proceedings of the 3rd International Conference on Educational Data Mining (EDM
'10), pp. 111–120, Pittsburgh, Pa, USA, June 2010. View at Scopus
2. C. K. Leong, Y. H. Lee, and W. K. Mak, “Mining sentiments in SMS texts for teaching evaluation,” Expert Systems with Applications, vol. 39, no. 3, pp. 2584–2589, 2012. View
at Publisher · View at Google Scholar· View at Scopus
3. B. Jagtap and V. Dhotre, “SVM & HMM based hybrid approach of sentiment analysis for teacher feedback assessment,” International Journal of Emerging Trends &
Technology in Computer Science (IJETTCS), vol. 3, no. 3, pp. 229–232, 2014. View at Google Scholar
4. J. Ogden and J. Lo, “How meaningful are data from Kim, S. M. & Calvo, R. A. (2010). Sentiment analysis in student experiences of learning,” in Proceedings of the 3rd
International Conference on Educational Data Mining, Pittsburgh, Pa, USA, 2012.
5. W. Medhat, A. Hassan, and H. Korashy, “Sentiment analysis algorithms and applications: a survey,” Ain Shams Engineering Journal, vol. 5, no. 4, pp. 1093–1113, 2014. View
at Publisher · View at Google Scholar· View at Scopus
6. A. Minanović, H. Gabelica, and Ž. Krstić, “Big data and sentiment analysis using KNIME: online reviews vs. social media,” in Proceedings of the 37th International Convention
on Information and Communication Technology, Electronics and Microelectronics (MIPRO '14), pp. 1464–1468, May 2014. View at Publisher ·View at Google Scholar · View
at Scopus
7. L. Augustyniak, T. Kajdanowicz, P. Kazienko, M. Kulisiewicz, and W. Tuliglowicz, “An approach to sentiment analysis of movie reviews: lexicon based vs. classification,”
in Hybrid Artificial Intelligence Systems: 9th International Conference, HAIS 2014, Salamanca, Spain, June 11–13, 2014. Proceedings, vol. 8480 of Lecture Notes in Computer
Science, pp. 168–178, Springer, Berlin, Germany, 2014. View at Publisher · View at Google Scholar
8. S. Rosenthal, P. Nakov, A. Ritter, and V. Stoyanov, “Semeval-2014 task 9: sentiment analysis in twitter,” in Proceedings of the 8th International Workshop on Semantic
Evaluation, pp. 73–80, Dublin, Ireland, August 2014.
9. H. Saif, M. Fernandez, Y. He, and H. Alani, “Evaluation datasets for twitter sentiment analysis,” in Proceedings of the 1st International Workshop on Emotion and Sentiment
in Social and Expressive Media (ESSEM) in Conjunction with AI∗IA Conference, Turin, Italy, 2013.
10. N. Altrabsheh, M. Gaber, and M. Cocea, “SA-E: sentiment analysis for education,” in Proceedings of the 5th KES International Conference on Intelligent Decision Technologies
(KES-IDT '13), Sesimbra, Portugal, 2013.
1. A. Balahur, R. Mihalcea, and A. Montoyo, “Computational approaches to subjectivity and sentiment analysis: present and envisaged methods and
applications,” Computer Speech & Language, vol. 28, no. 1, pp. 1–6, 2014. View at Publisher · View at Google Scholar · View at Scopus
2. B. Liu, “Sentiment analysis and subjectivity,” in Handbook of Natural Language Processing, Taylor & Francis, Boca Raton, Fla, USA, 2nd edition,
2010. View at Google Scholar
3. E. Cambria, B. Schuller, Y. Xia, and C. Havasi, “New avenues in opinion mining and sentiment analysis,” IEEE Intelligent Systems, vol. 28, no. 2, pp.
15–21, 2013. View at Publisher · View at Google Scholar · View at Scopus
4. G. A. Miller, R. Beckwith, C. Fellbaum, D. Gross, and K. J. Miller, “Introduction to wordnet: an on-line lexical database,” International Journal of
Lexicography, vol. 3, no. 4, pp. 235–244, 1990. View at Publisher· View at Google Scholar · View at Scopus
5. S. Mohammad, C. Dunne, and B. Dorr, “Generating high-coverage semantic orientation lexicons from overtly marked words and a thesaurus,”
in Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP '09), vol. 2, pp. 599–608, Singapore, August 2009.
6. S.-M. Kim and E. Hovy, “Determining the sentiment of opinions,” in Proceedings of the 20th International Conference on Computational
Linguistics (COLING '04), Association for Computational Linguistics, 2004.View at Publisher · View at Google Scholar
7. V. Hatzivassiloglou and K. R. McKeown, “Predicting the semantic orientation of adjectives,” in Proceedings of the 35th Annual Meeting of the
Association for Computational Linguistics and 8th Conference of the European Chapter of the Association for Computational Linguistics (ACL '98),
pp. 174–181, 1997. View at Publisher · View at Google Scholar
8. B. Liu and L. Zhang, “A survey of opinion mining and sentiment analysis,” in Mining Text Data, pp. 415–463, Springer US, 2012. View at Google
Scholar
9. B. Pang, L. Lee, and S. Vaithyanathan, “Thumbs up?: sentiment classification using machine learning techniques,” in Proceedings of the ACL-02
Conference on Empirical Methods in Natural Language Processing (EMNLP '02), vol. 10, pp. 79–86, 2002. View at Publisher · View at Google
Scholar
10. N. C. Petersen, “A Logical Approach to Sentiment Analysis,” Thesis prepared at Department of Informatics and Mathematical Modelling at the
Technical University of Denmark, http://www2.imm.dtu.dk/pubdb/views/edoc_download.php/6459/pdf/imm6459.pdf
1. M. Tsytsarau and T. Palpanas, “Survey on mining subjective data on the web,” Data Mining and Knowledge Discovery, vol. 24,
no. 3, pp. 478–514, 2012. View at Publisher · View at Google Scholar · View at Scopus
2. E. Cambria, “An introduction to concept-level sentiment analysis,” in Advances in Soft Computing and Its Applications, Lecture
Notes in Computer Science, pp. 478–483, Springer, Berlin, Germany, 2013. View at Google Scholar
3. B. Agarwal, N. Mittal, P. Bansal, and S. Garg, “Sentiment analysis using common-sense and context information,” Computational
Intelligence and Neuroscience, vol. 2015, Article ID 715730, 9 pages, 2015.View at Publisher · View at Google Scholar · View at
Scopus
4. T. Wilson, J. Wiebe, and P. Hoffmann, “Recognizing contextual polarity in phrase-level sentiment analysis,” in Proceedings of
the Human Language Technology Conference and Conference on Empirical Methods in Natural Language Processing
(HLT/EMNLP '05), pp. 347–354, Stroudsburg, Pa, USA, October 2005. View at Scopus
5. E. Riloff and J. Wiebe, “Learning extraction patterns for subjective expressions,” in Proceedings of the Conference on Empirical
Methods in Natural Language Processing (EMNLP '03), pp. 105–112, Stroudsburg, Pa, USA, July 2003.
6. J. Wiebe, T. Wilson, and C. Cardie, “Annotating expressions of opinions and emotions in language,” Language Resources and
Evaluation, vol. 39, no. 2, pp. 165–210, 2005. View at Publisher · View at Google Scholar · View at Scopus
7. J. Ahn, A. Borowiec, K. Buckley et al., “CYBEREMOTIONS—collective emotions in cyberspace,” Procedia Computer Science, vol.
7, pp. 221–222, 2011. View at Publisher · View at Google Scholar
8. L. Goeuriot, L. Kelly, G. J. F. Jones, H. Müller, and J. Zobel, “Report on the SIGIR 2014 Workshop on Medical Information
Retrieval (MedIR),” ACM SIGIR Forum, vol. 48, no. 2, pp. 78–82, 2014. View at Publisher · View at Google Scholar