DESIGN AND DEVELOPMENT OF SENTENCE LEVEL

AMHARIC TEXT SENTIMENTAL ANALYSIS MODEL USING

MACHINE LEARNING APPROACH

A project paper was submitted to the department of computer science of

Unity University in partial fulfillment for the award of the
degree of Bachelor of Science in computer science.
Prepared by:
Bereket Abebe
Natan Getachew
Seada Adem
Tsegalem Gedefaw
Zehara momin

Advisor: Miss.
Zebiba (BSc)

Unity University
Rukiya Campus
Department Of Computer Science
July 2021
Dessie, Ethiopia
 Approval Sheet
Name Sign

Adviser, _____________________________ _____________

Examiner, ____________________________ ______________

Examiner, ____________________________ ______________

i
 DECLARATION
The project is our own and is not presented for a degree in any other University and all
the sources of material used for the project have been duly acknowledged.

Name Signature
Bereket Abebe _____________
Natan Getachew _____________
Seada Adem _____________
Tsegalem Gedefaw _____________
Zehara momin _____________

This project has been submitted for examination with my approval as university
advisor.
Name Signature Date
Miss. Zebiba (BSc) _________________ _________

ii
 ACKNOWLEDGEMENT
First of all, we would love to thank God for giving us all the straight we needed in life. After that
we want to thank our advisor Miss. Zebiba (BSc) for mentoring, encouraging and helping us in
all the areas we needed it.

Special Thanks to Israel Abebe (MSc) for motivating and guiding us to focus on this specific
area of study by sharing his experience in the field of machine learning and AI.

iii
 Abstract
We have been operating to form a sentiment model for Amharic comments which may solve
issues like Amharic digitalization and opinion mining on this specific language (Amharic). To
start with what it actually means and how it works, Sentiment analysis within the world permits
us to extract helpful data from the opinions and feelings of followers and customers of a
company. It analyzes the text written in natural language concerning services and merchandise.
Then, classifying sentiments, opinions, feeling as Positive and negative. However, collecting
information from websites is difficult. In addition to the present, the information is large, and it's
challenging to grasp customers' feelings and opinions. To resolve this downside, we used a
sentence-level approach to mine the general sentiment or opinion polarity from the information
required. We followed sentence level opinion mining in detail to meet customers and
organizations need. We employed crafted rules using Probabilistic classifiers for labeling data
and a supervised approach to training and testing the data. The project flows into five major
components of the architecture which include data collection, preprocessing of data, features
extraction, opinions, sentiment classification, and result. Naive Bayes (NB) classifier was used
for sentiment classification. The collected Amharic opinionated sentences and phrase texts from
the website page were 6000. Among those, data for training (80%) and data for testing (20%).
The result showed Naïve baye’s Accuracy evaluation metrics were 85% respectively. Using this
whole process our model is able to distinguish comments based on their polarity which can be a
great achievement in opinion mining and analyzing the applicable areas. Also increases the use
of Amharic. This model can be integrated on websites so they can analyze their customer opinion
on their product or services. This helps to create a close relation between markets and their
customers by increasing favorable products and service qualities, and reduction of service and
products based on the negative opinions from the customers. This model can also be a foundation
for the coming studies on Amharic digitalization and related fields.

Table of Contents
iv
ACKNOWLEDGEMENT iii
Abstract iv
List of tables ix
List of figures x
ACRONYMS xi
List Of Appendix xii
1 CHAPTER ONE 1
Project overview 1
1.1 Introduction 1
1.2 Background of the study 1
1.3 Motivation 2
1.4 Statement of the problem 3
1.5 Objective of the project 3
1.5.1 General objective 3
1.5.2 Specific objective 4
1.6 Significance of the project 4
1.7 Cost-Benefit Analysis 6
1.7.1 Tangible benefits: 6
1.7.2 Intangible benefits: 6
1.7.3 Direct cost: 6
1.7.4 Indirect cost: 7
1.7.5 Cost of potential risk: 7
1.7.6 Net Benefit 8
1.8 Scope and Limitation of the project 8
1.8.1 Scope 8
1.8.2 Limitation 8
1.9 Methodology 9
1.9.1 Data collection methods 9
1.9.2 Data Collection for dataset 9
1.9.3 pre-processing 9
1.9.4 Design Approach 10

v
 1.9.5 Tools 11
1.10 Hardware and software tools 12
1.10.1 Hardware tool 12
1.10.2 Software tool 12
2 CHAPTER TWO 13
2.1 Requirement analysis 13
2.2 Existing system description 13
2.3 Problem definition 13
2.4 Broad alternative solutions/ APPROACHES 14
2.4.1 Machine learning approach 14
2.5 Functional requirements of the proposed system 16
2.6 Non-functional requirements 17
2.7 Business rule documentation 17
3 CHAPTER THREE 18
3.1 System analysis 18
3.2 System use case modeling case Modeling 18
3.3 System use case description 19
3.4 Analysis sequence diagram 28
3.5 Conceptual modeling: class diagram 32
3.6 Activity diagram 33
3.7 Analysis User Interface Prototype 36
4 CHAPTER FOUR 37
4.1 System design 37
4.2 Purpose and goals of design 37
4.2.1 Goal of the design 37
4.2.2 Purpose of the design 38
4.3 Design Class Diagrams 38
4.4 Analysis class diagram description 39
4.5 Collaboration diagram 40
4.6 Component diagram 41
4.7 Deployment diagram 42

vi
 4.8 Model Architecture 43
4.9 User Interface Design 44
5 CHAPTER FIVE 45
Implementation and Test 45
5.1 Implementation 45
5.1.1 Model Algorithm 45
5.1.2 Sample Code 45
5.2 Hardware software acquisitions 49
5.2.1 Hardware 49
5.2.2 Software 49
5.3 Testing 50
5.3.1 Unit Testing 50
5.3.2 Integration Testing 50
5.3.3 Functional Testing 50
5.3.4 Performance Testing 51
5.3.5 Regression Testing 51
5.3.6 White Box Testing 51
5.4 Evaluation Procedures 52
5.4.1 Evaluation Methods 52
5.4.2 Cross-validation 52
5.4.3 Confusion Matrix 53
6 CHAPTER SIX 55
conclusion and recommendation 55
6.1 Conclusion 55
6.2 Recommendations 55
Glossary 57
References 58
APPENDICES 59
Appendix A: web scraping by python code 59
Appendix B: collecting data labeled spreadsheet picture 61
Appendix C: Dataset collection using Instant Data Scraper 62

vii
 Appendix D: Normalization of Amharic Character 62

List of tables
Table 1:Hardware cost.....................................................................................................................7

viii
Table 2:Software cost......................................................................................................................7
Table 3:Net benefit..........................................................................................................................8
Table 4:Use Case table sentiment classification............................................................................20
Table 5:Use Case table Write a comment......................................................................................21
Table 6:use case table Display polarity with probability...............................................................22
Table 7:Use case Table Train Model.............................................................................................23
Table 8:use case table Test Model.................................................................................................24
Table 9:use case table Calculate Accuracy and report..................................................................25
Table 10:Use case Table View polarity with probability..............................................................26
Table 11:Use case Table Website..................................................................................................26
Table 12:Use case Table View Model of Accuracy and report.....................................................27
Table 13: Use case Table Insert Dataset........................................................................................27
Table 14: Use case Table Update model and dataset....................................................................28
Table 15: Use case Table Maintenance.........................................................................................28
Table 16: class diagram description.............................................................................................39
Table 17: Test Risks /Issues..........................................................................................................51
Table 18: Confusion matrix...........................................................................................................53
Table 19: Performance evaluation by CountVectorizer................................................................54
Table 20: Performance evaluation by TF-IDFVectorizer.............................................................54

List of figures
Figure 1: Broad Alternative solution............................................................................................14
Figure 2: Support Vector Machines Classifiers (SVM)................................................................15
Figure 3: Use Case Diagram:........................................................................................................19
ix
Figure 4: Train ML model sequence diagram...............................................................................29
Figure 5: Comment Classification sequence diagram...................................................................29
Figure 6: Write comment sequence diagram.................................................................................30
Figure 7: View comment polarity Classification sequence diagram.............................................30
Figure 8:Test Model sequence diagram.........................................................................................31
Figure 9: Calculate Accuracy and report sequence diagram.........................................................31
Figure 10: View Accuracy of Model sequence diagram...............................................................32
Figure 11:Conceptual modeling class diagram..............................................................................33
Figure 12: Activity diagram for ML model train and test.............................................................34
Figure 13: Activity diagram for classify user’s comment.............................................................34
Figure 14: Activity diagram for calculate Accuracy and report....................................................35
Figure 15: Activity diagram for View Model of Accuracy...........................................................35
Figure 16: Analysis user interface prototype.................................................................................36
Figure 17: Design Class Diagrams................................................................................................38
Figure 18:Collaboration diagram for classifying user comment...................................................40
Figure 19: Collaboration diagram for Train The model................................................................41
Figure 20: Component Diagram....................................................................................................42
Figure 21: Deployment Diagram...................................................................................................43
Figure 22: ML Model Architecture...............................................................................................43
Figure 23: User interface design....................................................................................................44

ACRONYMS
SA Sentimental Analysis
NLP Natural Language Processing
ML Machine Learning
US Use Case

x
NB Naive Bayes
BR Business Rule
ETB Ethiopian Birr

List Of Appendix
Appendix A: Python code for scraping data from YouTube ................................. 60
Appendix B: labeled data in spreadsheet form ...................................................... 62
Appendix C: Dataset collection using Instant Data Scraper................................... 63
Appendix D: Normalization of Amharic Character ............................................... 63

xi
xii
1 CHAPTER ONE

Project overview

1.1 Introduction
Artificial intelligence (AI) refers to the simulation of human intelligence in machines that are
programmed to think like humans and mimic their actions. The term may also be applied to any
machine that exhibits traits associated with a human mind such as learning and problem-solving.
[1]
Natural language processing (NLP) is a branch of artificial intelligence that helps computers
understand, interpret and manipulate human language. NLP draws from many disciplines,
including computer science and computational linguistics, in its pursuit to fill the gap between
human communication and computer understanding. [2]

Sentiment analysis, also called opinion mining, is a natural language processing (NLP) technique
used to determine whether data, information, or a piece of comment is positive, negative, or
neutral. [3] Natural language is a language that has evolved naturally as a means of
communication among people and one of the behaviors of human beings. NLP is an area of
project and application that explores how computers can be used to understand and manipulate
natural language text or speech to do useful things. [4] Sentiment analysis, which is also known
as emotion mining, attitude mining, or subjectivity mining, is a hot project discipline that is
concerned with the computational study of opinions, sentiments, and emotions expressed in an
opinionated text. A basic task in sentiment analysis is classifying the polarity of a given text at
the document, sentence, or feature/aspect level whether the expressed opinion in a document, a
sentence, or an entity feature/aspect is positive, negative, or neutral. [5]

1.2 Background of the study

As we live in a globalized world, technologies get updated pretty faster than we could imagine.
Unfortunately, the currently working system for Amharic language is not civilized and uses a
manual comment analysis model specially for commercial uses. This way of analyzing costumer

1
opinion makes the marketing environment rough and less preferred. The current system is not
capable of understanding the Amharic language digitally. The reason behind that is the lack of
enough models and reliable resources in the area. The other thing about the system is, user
opinions are valuable only when the comments are in English or other digitized languages which
makes the Amharic language less preferred on the platform.

For Amharic, the first attempt at sentiment analysis is described by Philemon and Mulugeta
(2014), who focused on the prediction of sentiment polarity. They have used a Naïve Bayes
classifier based on unigram and bigram features on 600 tweets. [6] and The other works by
Gezmu et al. (2018) [7]and Abate and Assabie (2014) [8] indicate that NLP components, such as
part of speech tagging (POS), named entity recognition (NER), and sentiment analysis are non-
trivial due to the morphological, syntactic, and semantic complexity of the language.in additional
work by Tulu tilahun Hailu(2013) in “optional mining from Amharic blog”, she used rule-based
approach ML model [9], and by Dessalew Chilote (2019) in “PUBLIC SENTIMENT
ANALYSIS FOR AMHARIC NEWS”, he used Naive Bayes(NB) and Support Vector
Machine(SVM) ML model [10].

1.3 Motivation
Since people rely on the internet more than ever, Amharic became a non-important channel for
communication. To make matters simpler, we need to separate the positive and negative
comments so that the platform can make sense out of them. Motivation for sentiment analysis is
twofold. Both customers and producers highly value customer opinions about products and
services. Thus, sentiment analysis has seen a considerable effort from the industry as well as
academia. We believe that we can solve the problems stated above by publishing the Amharic
dataset and creating a sentiment model which can differentiate positive and negative comments.

2
1.4 Statement of the problem
There are a variety of problems to list about the current system. We have grouped the problems
into subclasses depending on their scope.
● Amharic digitalization
Using Amharic on social media has been difficult since it's hard to understand the
language digitally. This problem is due to many reasons like lack of an efficient input
system, lack of sentiment models specifically for Amharic, etc. Language digitalization is
rendering a language and increasing the digital footprint. It gives solutions to the problem
that makes language less valuable. Digitizing helps to make them usable and to process
each written word.
● Dataset
To solve the language digitalization problem, we had to build a sentiment model.
However, to get there, we need to find enough Amharic datasets, which are hard to find.
In the process of manually collecting comments from social media, the comments are
mostly either in English or Amharic words written in English letters. Thus, comments are
hard to use because the arrangement of the letters depends on the individual's
understanding.
● User’s opinion
In the current system, there is no way to differentiate b/n positive and negative comments
besides manually going through each, which makes the process difficult to get along
with. To understand and utilize sets of comments from users, there should be a model to
differentiate what the users are looking for.

1.5 Objective of the project

1.5.1 General objective

Design and development of sentence level Amharic text sentimental analysis Model using
machine learning approach.

3
1.5.2 Specific objective
To achieve the objective of this project, various tasks are performed, which
Includes:
❖ To prepare Amharic Dataset for other projects. Using web scraping tools on different
social media (like YouTube and Facebook), and manually translating English dataset then
by combining and organizing the comments we prepared a new set of data which can be
helpful for other projects.
❖ We used normalization which is a process used to clean unwanted characters from un
structured text for sentiment analysis.
❖ We used feature extraction to identify those aspects which are being commented by
customers using Countvectorizer.
❖ We used classifier to identify the text containing sentiment or opinion by deciding
sentiment polarity as positive or negative by using NB algorithm.
❖ Our model can be used as a foundation for much greater studies. Using our algorithm and
set of data’s, by improving limitations of our model and specifying the application area,
many projects can emerge.
❖ To apply appropriate ML algorithms and classification model approaches to opinionated
Amharic comments.
❖ To test and evaluate the performance of the ML algorithms by entering datasets to train
the model to achieve greater accuracy.

1.6 Significance of the project

A basic task in our sentiment analysis model is classifying the polarity of a given text at the
sentence level whether the expressed opinion in a sentence is positive or negative. And will be
able to automatically analyze the sentiment of a huge amount of collected data prior to making
decisions. This model is efficient enough to be applied in various areas to create close relations
between customers and service providers by analyzing their customers' opinion. Such areas of
application are:
● For social media: -
Social Sentiment analysis is the use of natural language processing (NLP) to analyze social
conversations online and determine deeper context as they apply to a topic, brand or theme. Our
net sentiment score and brand passion index show how users feel about your brand and compare
across your competitors.

4
 ● For the Entertainment Industry: -
The entertainment industry, like any other industry, needs to analyze the feelings customers have
about their products, whether a book, film, TV show, piece of music, or a play. For instance, in
the case of films, it would be very helpful to a film production company to understand at a highly
granular level the audience’s reactions: If a comedy, what was the overall assessment of how
funny it is? What are the funniest bits? What actors were liked best? Which characters had the
best lines? And other similar items.
● For business: -
The key to selling a product and quality services is responding to customers‟ demands at the
proper time and at the right location. Many companies spend large amounts of money on market
analysis for customer opinion. The proposed feature-level SA model can aid this effort and
potentially minimize costs and increase productivity of the companies. Analyzing E-Commerce
data will help online retailers to understand customer expectations, provide better shopping
experience and to increase the sales. Sentiment Analysis can be used to identify positive,
negative and neutral information from the customer reviews.
● For brand monitoring: -
Sentiment analysis tools allow businesses to check how their audience, customers, experts,
influencers, or the news talk about them online, how many positive and negative comments,
opinions, discussions and other content they generate. It's a fertile source of business
intelligence.
● For Voice of employee: -
Engage your employees, reduce turnover, and increase productivity. Use AI to evaluate
employee surveys or analyze Glassdoor reviews, emails, Slack messages, and more (without
feeling like Big Brother). Sentiment analysis software allows you to analyze employee opinions
subjectively, with no human input.

1.7 Cost-Benefit Analysis

1.7.1 Tangible benefits:

Comment Analysis: It costs 100ETB for 1000 comments. If the model analyzes 100,000
comments, we will get a 10,000 ETB profit. And the profit will stand for a solid 5 years because
of the contract which will be intended for a specific user.

5
 ((100,000 X 100) / 1000) 5 = 50,000ETB

1.7.2 Intangible benefits:

 Accurate and Fast Access to information.
 Increase flexibility of our model in different deployment areas.
 Increase information processing efficiency to make accurate and fast decisions by top
level managers.

1.7.3 Direct cost:

This cost contains the various types of costs that are spent for the development of the project or
some of the hardware, software, and network services expenses.

Item Unit Quantity Price (ETB)

Computer Number 2 30000

USB Flash disk Number 2 400

Print Number 5 1000

Hard disk Number 1 2000

Paper Number (packet) 2 400

Pen Number 7 70

Internet CDMA (Megabits) 20 2600

Mobile card Birr 300 300

Contingency -------------- ----------------- 1000

Total cost 37775

Table 1:Hardware cost

Item Price (ETB)

Python 3. Provided service (free)

6
 Microsoft Word 2016 Provided service (free)

Microsoft PowerPoint 2016 Provided service (free)

Microsoft Excel 2016 Provided service (free)

Google Doc Provided service (free)

Google Sheet Provided service (free)

Edraw Max Provided service (free)

Visual Studio Code Provided service (free)

Total cost Free

Table 2:Software cost

1.7.4 Indirect cost:

This type cannot be expressed in terms of money. Instead, it Includes:
➢ The time spent on the development of this project.
➢ The project team's knowledge and effort apply to develop the system.
➢ The project team advisor advice.
➢ Cost due to other activities that could have been taken care of in place of the
system development tasks.

1.7.5 Cost of potential risk:

Our model will be gradually updated in two ways.one is algorithm update which will be executed
by changing or adding various algorithms which can potentially increase its accuracy by
changing the way how our model identifies the comments. The second is a dataset update which
helps by increasing resources to train our model.

1.7.6 Net Benefit

Cost benefit analysis Total Price (ETB)

7
 Total Benefit 50, 000 (from a single user)
Total Cost 37,775 (one time cost)
Net-Benefit 12,225
Table 3:Net benefit

1.8 Scope and Limitation of the project

1.8.1 Scope
Sentiment analysis (or opinion mining) is a natural language processing technique used to
determine whether data is positive or negative. It is often performed on textual data to help
businesses monitor brand and product sentiment in customer feedback, and understand customer
needs.
The main purpose of our project (model) is to accurately classify and analyze polarity of
comments using algorithms which give the model an ideal structure of analysis words. Our
project can only classify Amharic language comments and this sentiment classification is
sentence-based classification.

1.8.2 Limitation
Due to these and time constraints, the following are out of the scope of this project work:
❖ Limited to sentiment (polarity) mining (only positive or negative) classification. The
opinion holders' written sentiments were subjective or objective on the particular aspects.
It does not cover subjective or objective classification.
❖ The study doesn’t cover irony or sarcasm (“አለባበሰዉ ቢያርሱ ባረም ይመለሱ”) And metaphor
(“አንበሳ ነዉ”).
❖ Words or texts that have indirect or hidden meanings such as “ቅኔያዊ አነጋገር”, idiom, and
proverb (“የቆጡን አወርድ ብላ የብብታን ጣለች”) are not incorporated in this project work.
❖ Only analyzes and classifies Amharic language comments.
❖ Image/picture, audio, video, and other emotional symbols are not the concerns of this
study.

8
1.9 Methodology

1.9.1 Data collection methods

❖ Observation
Observation is a common method of scientific project to collect the data. We use observation to
know the existing system by using social media, and the activity of industry (like movies and
music).

1.9.2 Data Collection for dataset

The data source for this study get from many ways the first-way English movie review dataset
and we Translate it to Amharic because We didn't get Amharic Dataset and in addition, many
Amharic comments are written by English-Amharic word(“des yemil” - ደስ የሚል, “filmu
yastelal”- ፊልሙ ያስጠላል), the second way collected from YouTube and Facebook by using data
scraping(DS) method, DS, also known as web scraping, is the process of importing information
from a website into a spreadsheet or local file saved on your computer. It’s one of the most
efficient ways to get data from the web, and in some cases to channel that data to another website
[11]. We apply this method by using python code (Appendix A) after collecting data labeled that
data by spreadsheet in the form of Appendix B and Dataset collection using Instant Data Scraper
in the form of Appendix C.

1.9.3 pre-processing
The following pre-processing activities will be performed by developing and adapting existing
tools in addition to manual processing:
 Data Cleaning: means fixing bad data in your data set. Bad data could be Empty cells,
Data in the wrong format, Wrong data, and duplicates data [12]. In our project, we
cleaned bad data like English, English-Amharic, or other words, numbers, emojis, and
symbols. And cleaned duplicated data and wrong format (mixed data like “wooooow
ያምራል”) data.

 Normalizing: different letters of Amharic script that have the same sound, homophone
characters, while integrating the reviews, for instance, we will replace “ሃኅኃሐሓ” and “ኻ”
to “ሀ” in the form of Appendix D.

9
1.9.4 Design Approach
The Sentiment Analysis (SA) System for Amharic SA which is developed implements the Naive
Bayes (NB) Classifier. NB is a statistical classification technique based on Bayes Theorem.
Naive Bayes methods are a set of supervised learning algorithms based on applying Bayes’
theorem with the “naive” assumption of conditional independence between every pair of features
given the value of the class variable. Bayes’ theorem states the following relationship, given
class variable y and dependent feature vector x1 through xn:

Using the naive conditional independence assumption that

For all I, this relationship is simplified to

Since P (x1…, xn) is constant given the input, we can use the following classification rule:

Now, we need to create a classifier model. For this, we find the probability of a given set of
inputs for all possible values of the class variable y and pick up the output with maximum
probability. This can be expressed mathematically as:

10
And we can use Maximum a Posteriori (MAP) estimation to estimate P(y) and P (xi∣y); the
former is then the relative frequency of class Y in the training set. GaussianNB implements the
Gaussian Naive Bayes algorithm for classification. The likelihood of the features is assumed to
be Gaussian:

The parameters σy and μy are estimated using maximum likelihood. [13]

The Naive Bayes classifier is a fast, accurate, and reliable algorithm. Naive Bayes classifiers
have high accuracy and speed on large datasets. When the assumption of independence holds, a
Naive Bayes classifier performs better compared to other models like logistic regression and you
need less training data, And NB classifiers mostly used in text classification have higher success
rates as compared to other algorithms. As a result, it is widely used in Spam filtering (to identify
spam email) and Sentiment Analysis (to identify positive and negative customer sentiments).

1.9.5 Tools
The main tools that will be used in conducting this project work are: -
● Visual Studio Code- for writing, editing and running codes
● MS-Word- for document writing
● MS Excel- for creating text dataset
● Python 3.8.6- for implementing the algorithms.

1.10 Hardware and software tools

1.10.1 Hardware tool

o Laptop - for writing code and documentation, collected data and storing data.

o Flash drive - to share data

11
1.10.2 Software tool
o Vs-code - for writing, editing and running codes.

o Python 3 - for writing, editing and running python code and implementing the algorithms.

o Microsoft Word 2016 - for document writing.

o Microsoft Excel 2016 - for creating text dataset.

o Microsoft PowerPoint 2016 - to prepare presentation document.

o Google Doc - for preparing documents by group in online.

o Google Excel - for collecting data by group in online.

o Edraw Max - for drawing diagram

2 CHAPTER TWO

2.1 Requirement analysis

Requirement analysis is a critical step in the development of a systems or software project. It
should be performed to identify and develop requirements that are actionable, verifiable, and

12
traceable. This process involves gathering all the relevant information from various stakeholders
and formulating a strategy to address their needs.

2.2 Existing system description

The current system is not capable of understanding the Amharic language digitally. The reason
behind that is the lack of enough models and reliable resources in the area. The other thing about
the system is, user opinions are valuable only when the comments are in English or other
digitized languages which makes the Amharic language less preferred on the platform.

The other side of the current system is in the area of studies. New researchers find it hard to do
their project in this area because finding resources like data sets is hard.

2.3 Problem definition

Problem of the existing system
 Amharic digitalization 

Lack of enough studies that can contribute to the digitalization of Amharic language. This
makes the area poor in resources for researchers to use.
 Dataset

There is not enough data to use on such projects. This also is an obstacle to the
development of this area of study.
 Opinion mining

The current system doesn't have models to collect and analyze a big set of comments
rather than using a manual old way.

 Customer - producer interaction

There is a weak bridge between customers and providers of a service or products, since
there is no reliable way of collecting and analyzing user opinions (comments).

13
2.4 Broad alternative solutions/ APPROACHES
Classification of sentiment analysis approaches for features/techniques are provided in Figure 1.

Figure 1: Broad Alternative solution

There are two main approaches to sentiment analysis. (i) Machine learning approach, and (ii)
lexicon-based approach.

2.4.1 Machine learning approach

The machine learning approach is used for predicting the polarity of sentiments based on trained
as well as test data sets. It applies ML algorithms and uses linguistic features
 Supervised learning

Supervised learning is a machine learning approach that’s defined by its use of labeled datasets.
These datasets are designed to train or “supervise” algorithms into classifying data or predicting
outcomes accurately. Using labeled inputs and outputs, the model can measure its accuracy and
learn over time. The goal is to predict outcomes for new data. The models are ideal for spam
detection, sentiment analysis, weather forecasting, and pricing predictions, among other things.
The complexity of the model is a simple method for machine learning, typically calculated
through the use of programs like R or Python. Type of Supervised learning:
o Naïve Bayes (NB)

14
it has proven to not only be simple but also fast, accurate, and reliable. It has been successfully
used for many purposes, but it works particularly well with natural language processing (NLP)
problems. NB is a probabilistic machine learning algorithm based on the Bayes Theorem, used in
a wide variety of classification tasks. Bayes’ Theorem is a simple mathematical formula used for
calculating conditional probabilities.

o Maximum Entropy

The Maximum Entropy Classifier converts labeled feature sets to vectors using encoding. This
encoded vector is then used to calculate weights for each feature that can then be combined to
determine the most likely label for a feature set. This classifier is parameterized by a set of X
{weights}, which is used to combine the joint features that are generated from a feature-set by an
X {encoding}. In particular, the encoding maps each C {(featureset, label)} pair to a vector. The
probability of each label is then computed using the following equation:

dotprod ( weights ,encode ( fs ,label))

P (fs|label) =
∑ ( dotprod( weights , encode( fs , l))forlinlabels)
o Support Vector Machine (SVM)

The main principle of SVMs is to determine linear separators in the search space which can best
separate the different classes. In fig.2 there are 2 classes x, o and there are 3 hyperplanes A, B,
and C. Hyperplane A provides the best separation between the classes, because the normal
distance of any of the data points is the largest, so it represents the maximum margin of
separation.

Figure 2: Support Vector Machines Classifiers (SVM)

 Unsupervised learning

15
Unsupervised Learning uses machine learning algorithms to analyze and cluster unlabeled data
sets. These algorithms discover hidden patterns in data without the need for human intervention
(hence, they are “unsupervised”). The goal is to get insights from large volumes of new data. The
model is a great fit for anomaly detection, recommendation engines, customer personas, and
medical imaging. The model needs powerful tools for working with large amounts of
unclassified data. Unsupervised learning models are computationally complex because they need
a large training set to produce intended outcomes [14].

2.5 Functional requirements of the proposed system

A functional requirement is a description of the activities and services a system must provide.
This is a software requirement that defines the behavior of the proposed system, which is the
fundamental process of transformation that software and hardware components of the system
perform on inputs to produce outputs.

It also describes the interaction between the system and its environment independent of its
implementation. The environment includes the user and any other external system with which the
system interacts.

Therefore, the proposed system has the following functional requirement:

1. The model trains from the dataset and tests itself.

2. The model can perform the classification process into positive and negative sentiments.
3. The model can calculate Accuracy and report,
4. The model can display sentiment analysis results derived from comments submitted by
users.

5. Write a comment and view the polarity of comments with probability by users.

6. Insert, update and maintain the dataset and model by developer, in addition developers
can view the accuracy and report of the ML model.

16
2.6 Non-functional requirements

Nonfunctional requirements are the general requirements of a system or give additional functions
to the user of the system.

A non-functional requirement is also a requirement that specifies criteria that can be used to
judge the operation of the system, rather than specific behaviors. It defines how a system is
supported. Non-functional requirements are often called qualities of a system.

Some of the non-functional requirements are: -

❖ User Interface: The system has a user-friendly interface design. The user interface will be
lightweight and easy to use and manage. The designed systems’ user interface graphics
will reflect the system,

❖ Efficiency: - The system gives fast response, which is to classify User comments within a
short period of time.
❖ Scalability: The system can run in various web browsers and the model is used for many
application areas which support the system environment.
❖ Accuracy: the model is preferred due to the reduction of error. The sentiment
classification is at 85% accuracy rate.
❖ Reliability: The system can generate a reliable output using user comments.
❖ Maintainability: The system should be as easy as possible to get maintained.
2.7 Business rule documentation
Business rules describe the operations, definitions, and constraints that apply to an organization.
In every organization or institution, there are rules and policies.
➢ #BR1- It demands payment for using the model based on the quantity of the model. For
2000 comments 20 ETB or 0.5 dollars.
➢ #BR2- Before the owner of the website uses the model should sign an Agreement at least
for 5 years.
➢ #BR3- When upgrading the system, the owner of the website should pay an additional
fee.
➢ #BR4- The comments should be Amharic characters.
➢ #BR5- The website using our model should have a comment section.
17
3 CHAPTER THREE

3.1 System analysis

System analysis is the decision of the system into its component pieces for purpose of
Studying how those components' pieces interact and work. System analysis is a problem-solving
technique that decomposes a system into its component pieces for the purpose of studying how
well those component parts work and interact to accomplish their purpose.

3.2 System use case modeling case Modeling

A use case model is a model of how different types of users interact with the system to solve a
problem. As such, it describes the goals of the users, the interactions between the users and the
system, and the required behavior of the system in satisfying these goals. In our system, the main
actors are: -

 User: - represent a person that uses the system

 Model: -represent the ML model
 Developer: - represent the ML model developer
The main list of use cases is
1. Sentiment classification
2. Model Train
3. Test model
4. Calculate Accuracy and report
5. Write comment
6. Display polarity with probability
7. View polarity with probability
8. Insert Dataset
9. Update Dataset and Model
10. View Accuracy and report
11. Maintenance dataset and model
12. Interface

18
Figure 3: Use Case Diagram:

3.3 System use case description

19
 Flow of events Step1: receive comment

Step2: pre-process the comment

Step3: feature extraction of the comment

Alternative events S4: if the comment maybe English word does not classify the comments

Table 4:Use Case table sentiment classification

Flow of events Step1: open the browser

Step2: Search the website

Step3: the browser displays the homepage

Step4: the user writes the comment

20
 Step4: If the user writes the comment using English letters, the model
Alternative
rejects it.
events
Step7: If a user writes a comment and does not click submit, the comments
delete.

Post-condition The comments classify by ML model and display comment polarity with
probability
Table 5:Use Case table Write a comment

Description The ML model displays each comments polarity with probability

Flow of events Step1: Receive comment

Step2: Classify the comment into positive or negative using the ML model

Step3: Display comment polarity with probability

Alternative Step2: If the comment maybe English word does not classify the comments
events

Pre-condition - The user should only write comments by using Amharic letters.

- Classify the comment into positive or negative using the ML (Machine

learning) model.

21
Table 6:use case table Display polarity with probability

Flow of events Step1: Receive Dataset

Step2: Pre-processing dataset (Cleaning, Normalization)

Step3: Separate each not duplicated word by using countvectorizer ML

Step4: The system selects 80% of the Dataset randomly

Step5: The system trains the dataset Means to learn which comment
positive negative

Alternative events Step1: if the dataset does not Amharic, useless the ML model

Step4: The developer can give any percent for a training system

Post-condition The system tests by itself and calculates the accuracy and report.

The system can separate any comments by using this ML model

Table 7:Use case Table Train Model

22
 Flow of events Step1: receive Dataset

Step2: pre-processing dataset (Cleaning, Normalization)

Step3: separate each not duplicated word by using countvectorizer ML

Step4: The system selects 80% of the Dataset randomly

Step5: The system trains the dataset Means to learn which comment
positive negative

Step6: The system tests itself by using 20% of the dataset only to see the
comment executing the polarity (Negative or negative).

Table 8:use case table Test Model

Use case Id US006

Use case name Calculate Accuracy and report

Actor Model

Description The Model after train and test itself Calculate the accuracy and report

23
 Flow of events Step1: receive Dataset

Step2: pre-processing dataset (Cleaning, Normalization)

Step3: separate each not duplicated word by using countvectorizer ML

Step4: The system selects 80% of the Dataset randomly

Step5: The system trains the dataset Means to learn which comment
positive negative

Step6: The system tests itself by using 20% of the dataset only to see the
comment executing the polarity (Negative or negative).

Step7: calculate the accuracy and report

Alternative events Step4: The developer can give any percent

Pre-condition The system must train and test itself, which comment positive or negative

Post-condition Display the accuracy and report for the developer

Table 9:use case table Calculate Accuracy and report

Use case Id US07

Use case name View polarity of comment with probability

Actor User

Description The user view comment of polarity with probability

24
 Flow of events Step1: enter the website

Step2: write a comment

Post-condition The user can know which comment is positive or negative

Table 10:Use case Table View polarity with probability

Use case Id US008

Use case Name Website

Actor User

Description Use a website to integrate the model with the application area

Flow of events Step1- open browser

Step2- write website name
Step3- Search the website

Alternative event Step2- Access by URL

pre-condition User should know website

post-condition Access the website

Table 11:Use case Table Website

Use case Id US009

Use case Name View Model of Accuracy and report

Actor Developer

Description View the ML model Accuracy and report

25
 Flow of events Step1- open code
Step2- run code
Step3- view Accuracy and report

Alternative event Step2- if an error occurs, fix it

Step3- if accuracy result may be low, fix it

pre-condition Insert Dataset and develop ML model

post-condition View and know the ML model Accuracy and report

Table 12:Use case Table View Model of Accuracy and report

Use case Id US010

Use case Name Insert Dataset

Actor Developer

Description Insert the dataset for train and test ML model

Flow of events Step1- open code

Step2- read the dataset by code

pre-condition Collect and Prepare the Dataset

post-condition The ML model train and test

Table 13: Use case Table Insert Dataset

Use case Id US011

Use case Name Update model and dataset

Actor Developer

Description The ML model update uses a different Algorithm and the dataset

26
 update by adding the data’s

Flow of events Step1- open code and dataset

Step2- write a different Algorithm and adding data’s

pre-condition Must be prepared Model and dataset

post-condition Increase the ML model Accuracy

Table 14: Use case Table Update model and dataset

Use case Id US012

Use case Name Maintenance

Actor Developer

Description Occur some problem in the ML model maintained by developer

Flow of events Step1- separate the problem

Step2- make decision
Step3- fix the problem

pre-condition Problem Must occur

post-condition Fix the problem

Table 15: Use case Table Maintenance

3.4 Analysis sequence diagram

A sequence diagram is a type of interaction diagram because it describes how and in what order
a group of objects works together. These diagrams are used by software developers and business
professionals to understand requirements for a new system or to document an existing process.
Sequence diagrams are sometimes known as event diagrams or event scenarios.

27
Figure 4: Train ML model sequence diagram

Figure 5: Comment Classification sequence diagram

28
Figure 6: Write comment sequence diagram

Figure 7: View comment polarity Classification sequence diagram

29
Figure 8:Test Model sequence diagram

Figure 9: Calculate Accuracy and report sequence diagram

30
Figure 10: View Accuracy of Model sequence diagram

3.5 Conceptual modeling: class diagram

A class diagram is a UML behavior diagram that describes the structure of a system by showing
the system's class, its attributes, operations (or methods), and the relationships among objects.
The Class diagram provides an overview of the target system by describing the objects and
classes inside the system and the relationships between them.

31
Figure 11:Conceptual modeling class diagram

3.6 Activity diagram

Activity diagram is a UML behavior diagram which is essentially a flowchart, showing the flow
of control from activity to activity. These diagrams are commonly called control flow and object
flow models. So, the control flow is drawn from one operation to another. An activity diagram is
represented by shapes that are connected by arrows run from start activity to completion activity
and represent the sequential order of performed activities. The basic purpose of activity diagrams
is used to show message flow from one activity to another activity.

32
Figure 12: Activity diagram for ML model train and test

Figure 13: Activity diagram for classify user’s comment

33
Figure 14: Activity diagram for calculate Accuracy and report

Figure 15: Activity diagram for View Model of Accuracy

34
3.7 Analysis User Interface Prototype
User interface prototype helps to gather more requirements from users or it show how the system
works. The following user interface flow diagram basically shows the administrator, Instructor
and Student investigate how to use the system.

Figure 16: Analysis user interface prototype

35
4 CHAPTER FOUR

4.1 System design

Systems design is the process of defining elements of a system like modules, architecture,
components and their interfaces and data for a system based on the specified requirements. It is
the process of defining, developing and designing systems which satisfies the specific needs and
requirements of a business or organization.

4.2 Purpose and goals of design

4.2.1 Goal of the design

This model is designed to identify comments as positive or negative. This helps to improve the
manual way of doing it by making the AI do the work itself.
 Reliability

The system is ready for users to check the polarity of their comments and also to use the
model on their personal website. It is also expected to function smoothly.

 Accuracy

Based on the amount of the data set we use the accuracy of the system can be improved.

 Reliability

The system is very reliable on classifying the polarity of the comment based on the
accuracy.

 Maintainability

The system should be as easy as possible to maintain and update.

 Scalability

The system can run in various web browsers and the model is used for many application areas
which support the system environment.

36
4.2.2 Purpose of the design
The purpose of this sentiment analysis model is to identify opinions or attitudes in terms of
polarity. The polarity is the perspective of the person. In terms of advertising, the internet is the
best medium through which to promote businesses as it will reach a target group of customers in
which sentiment analysis could be used to help ensure that the website’s contents fit with the
commercial content so that it is not detrimental to the reputation and popularity of the company
and/or brand.

4.3 Design Class Diagrams

Figure 17: Design Class Diagrams

37
 4.4 Analysis class diagram description

The above analysis class diagram is briefly documented here.

Class Attributes Operations Association

Read_Dataset ● Pos_data ● read_dataset ● One to one with

(Csv input) ● Neg_data Normalization
● num_data

Normalization ● text ● normalize_char_le ● One to one with

vel_missmatch Vectorizer

Vectorizer ● name ● get_vectorizer ● One to one with

Feature_extraction

Feature_extraction ● Model_type ● feature_extraction ● One to one with

● text Classification

Classification ● Classifier ● classify ● One to one with Main

● Matrix
● text

Main ● Data ● amharic_sentiment ● One to one with

● Text Read_Dataset (Csv
● Label input), Normalization,
● Unique_label Vectorizer, and
● X_train Feature_extraction
● X_test ● One to many with
● Y_train Classification
● y_test
Table 16: class diagram description

38
4.5 Collaboration diagram
The collaboration diagram describes a specific scenario. A collaboration diagram shows an
interaction organized around the objects in the interaction and their link to each other. Unlike
sequence diagrams, a collaboration diagram shows the relationship among the objects. On the
other hand, collaboration diagrams do not show time as a separate dimension, so sequence
numbers determine the sequence of messages and the concurrent threads.

Figure 18:Collaboration diagram for classifying user comment

39
Figure 19: Collaboration diagram for Train The model

4.6 Component diagram

A component diagram (CD) is used to break down a large object-oriented system into smaller
components, so as to make them more manageable. It models the physical view of a system. CDs
are high-level aggressions of smaller software pieces and provide a black box building block
approach to the system construction.

40
Figure 20: Component Diagram

4.7 Deployment diagram

The deployment diagram visualizes the physical hardware on which the software will be
deployed. It portrays the static deployment view of a system. It involves the nodes and their
relationships. The main purpose of the deployment diagram is to represent how software is
installed on the hardware component. It depicts in what manner a software interacts with
hardware to perform its execution.

41
Figure 21: Deployment Diagram

4.8 Model Architecture

the conceptual model that defines the structure, behavior, and more views of a model. An
architecture description is a formal description and representation of a model, organized in a way
that supports reasoning about the structures and behaviors of the system.

Figure 22: ML Model Architecture

42
4.9 User Interface Design
User interface is the part of Model that enable user of the system to interact with it. It may
contain input boxes, message boxes and different buttons.

Figure 23: User interface design

43
5 CHAPTER FIVE

Implementation and Test

5.1 Implementation
The implementation phase is where you and your project team actually do the project work to
produce the deliverables. So now the team is in a position of converting all documents gathered
and designed into the code so that the system will be implemented for the user to be used for the
purpose it developed.

5.1.1 Model Algorithm

Our model used the NB Algorithm. NB classifiers are a collection of classification algorithms
based on Bayes’ Theorem. It is not a single algorithm but a family of algorithms where all of
them share a common principle, i.e., every pair of features being classified is independent of
each other.

5.1.2 Sample Code

 Source code main module

import pandas as pd
import numpy as np
from sklearn.utils import shuffle
import pickle

from sklearn.feature_extraction.textimportCountVectorizer,TfidfVectorizer
from sklearn.model_selection import train_test_split
from sklearn.naive_bayes import GaussianNB
from sklearn.metrics import accuracy_score
from sklearn.metrics import classification_report
from pydantic import BaseModel

from normalization import normalize_char_level_missmatch

from read_data import read_dataset
from feature_extraction import feature_extraction
44
from classification import classify
from vectorizer import get_vectorizer

np.random.seed(42)
data = read_dataset(num_data=3000)
data['text']=data['text'].apply(lambda x: normalize_char_level_missmatch(x))
text,label = data['text'].values,data['label'].values
matrix = get_vectorizer()
X = feature_extraction(matrix,text)
unique_label = list(set(label))
Y= []
for i in label:
Y.append(unique_label.index(i))
X_train, X_test, y_train, y_test = train_test_split(X, Y,test_size=0.2)
classifier = GaussianNB()
classifier.fit(X_train, y_train)
y_pred = classifier.predict(X_test)
accuracy = accuracy_score(y_test, y_pred)
report = classification_report(y_test, y_pred)
print(accuracy)
print(report)

class Input(BaseModel):
message: str
class Output(BaseModel):
message: str
def amharic_sentiment(input: Input) -> Output:
text = input.message
return Output(message=str(classify(classifier,matrix,text)))

45
  Data read module code

import pandas as pd
import numpy as np
defread_dataset(pos_data='data/pos_labling.csv',neg_data='data/neg_labling.csv',num_data=3000):
pos_data = pd.read_csv(pos_data)
neg_data = pd.read_csv(neg_data)
neg_data.columns = ['amharic']
pos_data.columns = ['amharic']
pos = pos_data.amharic.values[:num_data].tolist()
neg = neg_data.amharic.values[:num_data].tolist()
ata=pd.concat([pd.DataFrame(pos+neg),pd.DataFrame(np.ones(3000).tolist()+np.zeros(3000).tolist())],
axis=1)
data.columns = ['text','label']
return data

 Feature extraction module code

import pickle
def feature_extraction(model_type,text):
X = model_type.fit_transform(text).toarray()
pickle.dump(model_type, open("count_vectorizer.pkl", "wb"))
return X
 Vectorizer module code

from sklearn.feature_extraction.textimport CountVectorizer,TfidfVectorizer

def get_vectorizer(name='count'):
if name == 'tfidf':
return TfidfVectorizer(analyzer='word')
else:
return CountVectorizer(analyzer='word')

46
  Normalization module code

import re
def normalize_char_level_missmatch(input_token):
rep1=re.sub('[ሃኅኃሐሓኻ]','ሀ',input_token)
rep2=re.sub('[ሑኁዅ]','ሁ',rep1)
rep3=re.sub('[ኂሒኺ]','ሂ',rep2)
rep4=re.sub('[ኌሔዄ]','ሄ',rep3)
rep5=re.sub('[ሕኅ]','ህ',rep4)
rep6=re.sub('[ኆሖኾ]','ሆ',rep5)
rep7=re.sub('[ሠ]','ሰ',rep6)
rep8=re.sub('[ሡ]','ሱ',rep7)
rep9=re.sub('[ሢ]','ሲ',rep8)
rep10=re.sub('[ሣ]','ሳ',rep9)
rep11=re.sub('[ሤ]','ሴ',rep10)
rep12=re.sub('[ሥ]','ስ',rep11)
rep13=re.sub('[ሦ]','ሶ',rep12)
rep14=re.sub('[ዓኣዐ]','አ',rep13)
rep15=re.sub('[ዑ]','ኡ',rep14)
rep16=re.sub('[ዒ]','ኢ',rep15)
rep17=re.sub('[ዔ]','ኤ',rep16)
rep18=re.sub('[ዕ]','እ',rep17)
rep19=re.sub('[ዖ]','ኦ',rep18)
rep20=re.sub('[ጸ]','ፀ',rep19)
rep21=re.sub('[ጹ]','ፁ',rep20)
rep22=re.sub('[ጺ]','ፂ',rep21)
rep23=re.sub('[ጻ]','ፃ',rep22)
rep24=re.sub('[ጼ]','ፄ',rep23)
rep25=re.sub('[ጽ]','ፅ',rep24)
rep26=re.sub('[ጾ]','ፆ',rep25)
rep27=re.sub('(ሉ[ዋአ])','ሏ',rep26)
rep28=re.sub('(ሙ[ዋአ])','ሟ',rep27)
rep29=re.sub('(ቱ[ዋአ])','ቷ',rep28)
rep30=re.sub('(ሩ[ዋአ])','ሯ',rep29)
rep31=re.sub('(ሱ[ዋአ])','ሷ',rep30)

47
 rep32=re.sub('(ሹ[ዋአ])','ሿ',rep31)
rep33=re.sub('(ቁ[ዋአ])','ቋ',rep32)
rep34=re.sub('(ቡ[ዋአ])','ቧ',rep33)
rep35=re.sub('(ቹ[ዋአ])','ቿ',rep34)
rep36=re.sub('(ሁ[ዋአ])','ኋ',rep35)
rep37=re.sub('(ኑ[ዋአ])','ኗ',rep36)
rep38=re.sub('(ኙ[ዋአ])','ኟ',rep37)
rep39=re.sub('(ኩ[ዋአ])','ኳ',rep38)
rep40=re.sub('(ዙ[ዋአ])','ዟ',rep39)
rep41=re.sub('(ጉ[ዋአ])','ጓ',rep40)
rep42=re.sub('(ደ[ዋአ])','ዷ',rep41)
rep43=re.sub('(ጡ[ዋአ])','ጧ',rep42)
rep44=re.sub('(ጩ[ዋአ])','ጯ',rep43)
rep45=re.sub('(ጹ[ዋአ])','ጿ',rep44)
rep46=re.sub('(ፉ[ዋአ])','ፏ',rep45)
rep47=re.sub('[ቊ]','ቁ',rep46)
rep48=re.sub('[ኵ]','ኩ',rep47)
return rep48

5.2 Hardware software acquisitions

5.2.1 Hardware
● Flash disk 8GB to transfer data from one computer to another.
● Laptop Computer for doing all activity, implementing the software
● Cable

5.2.2 Software
● Windows 10 operating system
● Vs code
● Web browser
● Python 3

48
5.3 Testing
Developing software is a complex process. Final phase of implementation is testing. No matter
how hard we try to eliminate all faults simply by going through the development phases,
however, through good practice we can make sure that the most serious fault does not occur in
the first place. In addition, we need a separate testing phase, with the goal of eliminating all
remaining faults before releasing the model. Testing is the final phase of implementation.
Testing is a process to show the correctness of the program. Testing is checking the model's
workability in an attempt to discover errors and avoid such errors from the model. In this the
team members tested the entire model as a whole with all codes and modules. In this we tested
all the functionalities in the model. All errors in the functions and modules have been tested. To
simplify the testing process, the project team followed the different types of tests that break the
testing process up into the distinct levels. Some testing standards are therefore necessary to
reduce the testing costs and operation time. The following are different testing strategies:

5.3.1 Unit Testing

Every module of the model is separately tested. It is often done by the programmer to test that
the unit he/she has implemented is producing expected output against given input. The objective
of unit testing is to: -

★ Check whether the return type of the functions is correct.

★ Check if the correct sentiment is generated for different user comments.

5.3.2 Integration Testing

All the modules combined together and tested it for its fitness with each other and with the
model’s functionality and other components as well to achieve the central objectives. When
combining them, the module with the problem (error occurred) will be identified and
recombined.

5.3.3 Functional Testing

Ensure that the specified functionality required in the system requirement works.
Risk Type Description Causes

Schedule risks Schedule risks in functional testing are ● Tracking for the resources is
risks associated with planning the date for not done properly.
functional tests. ● The complex tasks are not
The delivery date for the project gets identified properly.
slipped when the functional tests are not

49
 scheduled properly. ● The project scope has been
expanded unexpectedly.

Operational risks These risks occur due to a failed system, ● Lack of resources.
external risks due to events and lack of ● Lack of communication in
process implementation. the team.
● Responsibilities have not
been resolved properly.

Technical risk Technical risks in functionality testing are ● The project requirements are
associated with lack of implementation of changing continuously.
functionalities. ● Lack of technical
advancement.
● Project implementation is
complex.
● Integration of modules has
become difficult.

Table 17: Test Risks /Issues

5.3.4 Performance Testing

Assess the speed and efficiency of the system and to make sure it is generating results within a
specified time as per performance requirement.

5.3.5 Regression Testing

After a modification of a system component or a group of related units to ensure that
modification is working correctly.

5.3.6 White Box Testing

In this type of testing, a skilled man in different programming languages tries to test the logic of
our model. If the person who tests the system is not skilled, it is difficult to understand our
model’s functionality. If any failures occur while testing the model in all of the above testing
methods, the team will take immediate correction where this fault occurred before jumping to
next work. So, that it will meet the goal.

50
5.4 Evaluation Procedures
The experiment is done to measure the overall performance of the sentence level sentiment
analysis model. In this project work, a total of 6000 sentiment sentences were used to test the
accuracy of the system.

5.4.1 Evaluation Methods

In this study, the performance of sentiment classification is evaluated by using four indexes. Those
are Accuracy, Precision, Recall and F-score. In this study, the experiment is done based on the
algorithms and evaluates each evaluation metric corresponding to each sentiment polarity class.

 Precision
It is a quantity within cross validation that represents a fraction of retrieved instances that are
relevant. It is also called positive predictive value. High precision means that an algorithm
returns substantially more relevant results than irrelevant. It is the number of true positives
divided by the total number of elements labeled as belonging to the positive class. High precision
means that the majority of items labeled for instance ‘positive’ indeed belong to the class
‘positive’.
P (precision) = TP / (TP + FP)
 Recall
Recall, on the other hand, is the fraction of relevant instances that are retrieved. It is also called
sensitivity. It is the number of true positives divided by the total number of items that actually
belong to that class. A high recall means that the majority of the ‘positive’ items were labeled as
belonging to the class ‘positive’.
R (recall)= TP / (TP + FN)
 F-measure
A measure that combines Recall and Precision into a single measure of performance. This is just
the product of Precision and Recall divided by their average. Which is defined by the formula:
F-measure =2(Precision × Recall) / (Precision + Recall)

5.4.2 Cross-validation
Cross-validation is a technique to evaluate predictive models by partitioning the original sample
into a training set to train the model, and a test set to evaluate it. It is the de facto approach used
to assess the accuracy and validity of a statistical model. It is used for this study as well. The
holdout method is the simplest kind of cross validation. The data set is separated into two sets,
called the training set and the testing set. The errors it makes are accumulated as before to give
the mean absolute test set error, which is used to evaluate the model.

51
The advantage of this method is that it is usually preferable to the residual method and takes no
longer to compute. However, its evaluation can have a high variance. The evaluation may
depend heavily on which data points end up in the training set and which end up in the test set,
and thus the evaluation may be significantly different depending on how the division is made. In
this study randomly split shuffled the Dataset into a train set and the test set 80% – 20% is
randomly selected.

5.4.3 Confusion Matrix

A confusion matrix is a technique for summarizing the performance of a classification algorithm.
Classification accuracy alone can be misleading if you have an unequal number of observations
in each class or if you have more than two classes in your dataset. In general, the performance of
sentiment classification is evaluated by using four indexes. They are Accuracy, Precision, Recall
and F1-score. The common way for computing these indexes is based on the confusion matrix as
shown below:

Actual positive Actual negative

Predicted positive True Positive (TP) False Positive (FP)

Predicted negative False Negative (FN) True Negative (TN)

Table 18: Confusion matrix

o True positives are positive items that we correctly identified as positive for positive class
and Negative items that we correctly identified as Negative for negative class.
o True negatives are irrelevant items that we correctly identified as irrelevant. (Negative
comments not classified under positive for positive class and vice versa)
o False positives (Type I errors) are negative comments that we incorrectly identified as
positive for positive class and positive comments that we incorrectly classified as
negatives for negative class.
o False negatives (Type II errors) are relevant items that we incorrectly identified as
irrelevant. (Positive comments that are incorrectly not classified under positive for
positive class and negative comments that are incorrectly not classified under negative for
negative class.)

Performance evaluation of data with positive and negative by CountVectorizer

52
 Sentiment polarity Precision Recall F1-Score Support

Positive 0.79 0.96 0.86 199

Negative 0.95 0.74 0.83 201

Macro avg 0.87 0.85 0.85 400

Weighted avg 0.87 0.85 0.85 400

Accuracy 0.85 400

Table 19: Performance evaluation by CountVectorizer

Performance evaluation of data with positive and negative by TF-IDFVectorizer

Sentiment polarity Precision Recall F1-Score Support

Positive 0.81 0.93 0.87 199

Negative 0.92 0.78 0.84 201

Macro avg 0.86 0.86 0.85 400

Weighted avg 0.86 0.85 0.85 400

Accuracy 0.85 400

Table 20: Performance evaluation by TF-IDFVectorizer

53
6 CHAPTER SIX

conclusion and recommendation

6.1 Conclusion
In this era of technology and computerization, our country Ethiopia should be able to join this
lane. So, we believe this model will put a remarkable start on the digitalization and further study
of the language Amharic. Model development chiefly is one of the ways in which we can make
progress by remarkably making a lot of tasks easier and the overall use of computers along with the
developed model predominantly is believed to strikingly lead to the peak of the technology standard.
We put our knowledge, time and effort at its best to the accomplishment of this project and we can
confidently conclude the project as follows.
● Our model can receive a comment and identify the polarity. It can differentiate sentences
efficiently based on the algorithm used.
● We gathered the collected and labeled comments then made a dataset which will make the
process easier for another projector.
● The model filters & separates Amharic language characters without problems whilst a
consumer enters a comment. Because the model does not support characters other than
Amharic.
● The model has a user-friendly interface design. The user interface can be light-weight and
smooth to apply and manage.
● The model offers rapid response, that is to categorize User remarks inside a quick duration of
time.

6.2 Recommendations
In this section the researcher recommends further project works and improvements as shown
below:
● The subjective and objectivity classification can also be a future project direction to
enhance the performance of the model created in this project.

54
● Another future work that facilitates sentiment analysis is related project for Amharic text
to develop like Lexicon Word Net, spelling checking, Parts-of-Speech tagging machine
readable dictionaries, chat bot, document plagiarism detector and machine translation
model.
● Exploring richer ML analysis, for example, parsing, separate Amharic stop words,
semantic analysis, and topic modeling.
● As opinions on the web are important for many applications, it is not surprising that
people have started to abuse the system. Opinion spam is a fake or false opinion that
misleads readers by giving undeserving positive opinions to some target objects in order
to promote the objects and/or by giving malicious negative opinions to some other
objects in order to damage their reputations. Detecting such spam is very important for
applications in the future.
● Finally, the future challenges and directions so as to further enhance the project in the
field of opinion mining (SA).

55
 Glossary
CountVectorizer: is a great tool provided by the scikit-learn library in Python. It is used to
transform a given text into a vector on the basis of the frequency (count) of each word that
occurs in the entire text.
Sentimental Analysis: is a natural language processing (NLP) technique used to determine
whether data, information, or a piece of comment is positive, negative, or neutral.
Natural Language Processing: is a branch of artificial intelligence that helps computers
understand, interpret and manipulate human language.
Machine Learning: is a branch of artificial intelligence (AI) and computer science which
focuses on the use of data and algorithms to imitate the way that humans learn, gradually
improving its accuracy.
Naive Baise: is an algorithm that uses Bayes' theorem to classify objects. Popular uses of naive
Bayes classifiers include spam filters, text analysis and medical diagnosis. These classifiers are
widely used for machine learning because they are simple to implement.
Activity diagrams: are graphical representations of workflows of stepwise activities and actions
with support for choice, iteration and concurrency.
Actors: Actors are the type of users that interact with the system.
Class diagram: is a type of static structure diagram that describes the structure of a system by
showing the system's classes their attributes, operations (or methods), and the relationships
among objects.
Sequence diagram: is an interaction diagram that shows how processes operate with one
another and what is their order.
Use case: is a methodology used in system analysis to identify, clarify, and organize system
requirements.
Use case diagram: is a graphic depiction of the interactions among the elements of a system.

56
 References

[1] "investopedia," [Online]. Available: https://www.investopedia.com/terms/a/artificial-..

[2] "sas," [Online]. Available: https://www.sas.com/en_us/insights/analytics/what-is-natural-
language-processing nlp.html#:~:text=Natural%20language%20processing%20(NLP)
%20is,interpret%20and%20manipulate%20.
[3] "monkeylearn," [Online]. Available: Available: https://monkeylearn.com/sentiment-
analysis/..
[4] U. S. T. WAKDE, Rule Based Approach for," Advances in Computer Science, in Recent,
INDIA, 2014.
[5] B. Liu, Sentiment Analysis and Opinion Mining, Morgan & Claypool, 2012.
[6] W. P. a. W. Mulugeta, A Machine Learning Approach to Multi-Scale Sentiment Analysis of
Amharic Online Posts, HiLCoE Journal of Computer Science and Technology, 2014.
[7] B. E. S. G. a. A. N. A. M. Gezmu, Contemporary Amharic Corpus: Automatically Morpho-
Syntactically Tagged Amharic Corpus. In Proceedings of the First Workshop on Linguistic
Resources for Natural Language Processing, Alexandria,Egypt.
[8] M. A. a. Y. Assabie, Development of Amharic Morphological Analyzer Using
MemoryBased Learning. In International Conference on Natural Language Processing,
Warsaw, Poland, 2014.
[9] T. T. HAILU, OPINION MINING FROM AMHARIC BLOG," DEGREE OF MASTER OF
SCIENCE, Addis Abeba, 2013.
[10 D. Chilote, PUBLIC SENTIMENT ANALYSIS FOR AMHARIC NEWS," Degree of
] Master of Science in Computer Science, Department of Computer Science, BAHIR DAR ,
2019.
[11 "targetinternet," [Online]. Available: https://www.targetinternet.com/what-is-data-scraping-
] and-how-can-you-use-it/..
[12 "w3schools," [Online]. Available:
] https://www.w3schools.com/python/pandas/pandas_cleaning.asp..
[13 "scikit-learn," [Online]. Available: https://scikit-learn.org/stable/modules/naive_bayes.html..
]
[14 "ibm," [Online]. Available: https://www.ibm.com/cloud/blog/supervised-vs-unsupervised-
] learning..

57
 APPENDICES

Appendix A: web scraping by python code

"""$ python main.py YOUTUBE_VIDEO_URL"""
import csv
import io
from selenium import webdriver
from selenium.common import exceptions
import sys
import time

def scrape(url):
driver = webdriver.Chrome('./webdrivers/chromedriver')
driver.get(url)
driver.maximize_window()
time.sleep(5)

try:
title = driver.find_element_by_xpath('//*[@id="container"]/h1/yt-formatted-string').text
comment_section = driver.find_element_by_xpath('//*[@id="comments"]')
except exceptions.NoSuchElementException:
error = "Error: Double check selector OR "
error += "element may not yet be on the screen at the time of the find operation"

58
 print(error)

driver.execute_script("arguments[0].scrollIntoView();", comment_section)
time.sleep(7)
last_height = driver.execute_script("return document.documentElement.scrollHeight")

while True:
driver.execute_script("window.scrollTo(0, document.documentElement.scrollHeight);")
time.sleep(2)
new_height = driver.execute_script("return document.documentElement.scrollHeight")
if new_height == last_height:
break
last_height = new_height
driver.execute_script("window.scrollTo(0, document.documentElement.scrollHeight);")

try:
username_elems = driver.find_elements_by_xpath('//*[@id="author-text"]')
comment_elems = driver.find_elements_by_xpath('//*[@id="content-text"]')
except exceptions.NoSuchElementException:
error = "Error: Double check selector OR "
error += "element may not yet be on the screen at the time of the find operation"
print(error)

print("> VIDEO TITLE: " + title + "\n")

with io.open('results.csv', 'w', newline='', encoding="utf-16") as file:

writer = csv.writer(file, delimiter =",", quoting=csv.QUOTE_ALL)
writer.writerow(["Username", "Comment"])
for username, comment in zip(username_elems, comment_elems):
writer.writerow([username.text, comment.text])
driver.close()

59
if __name__ == "__main__":
scrape(sys.argv[1])

Appendix B: collecting data labeled spreadsheet picture

60
Appendix C: Dataset collection using Instant Data Scraper

61
Appendix D: Normalization of Amharic Character

62