CHAPTER 1

1. INTRODUCTION

1.1 FAKE NEWS CLASSIFICATION

A different way to detect fake news is through stance detection which will
be the focus of our study. Stance Detection is the process of automatically
detecting the relationship between two pieces of text. In this study, we explore
ways to predict the stance, given a news article and news headline pair.
Depending on how similar the news article content and headlines are, the stances
between them can be defined as ‘agree’, ‘disagree’, ‘discuss’ or ‘unrelated’. We
experimented with several traditional machine learning models to set a baseline
and then compare results to the state-of-the art deep networks to classify the
stance between article body and headline Fake news can be come in many forms,
including: unintentional errors committed by news aggregators, outright false
stories, or the stories which are developed to mislead and influence reader’s
opinion.

While fake news may have multiple forms, the effect that it can have on
people, government and organizations may generally be negative since it differs
from the facts. Detecting fake news is hard for many reasons. First, manual task
of identifying fake news is very subjective. Assessing the veracity of a news story
is a complex and cumbersome task, even for trained experts. News is not only
spread through traditional media outlets anymore but also through various social
media channels. Automated solution requires understanding the natural language
processing which is difficult and complex. These complexities make it a daunting
task to classify text as fake news.

In the current fake news corpus, there have been multiple instances where
both supervised and unsupervised learning algorithms are used to classify text.
However, most of the literature focuses on specific datasets or domains, most

1
prominently the politics domain. Therefore, the algorithm trained works best on
a particular type of article’s domain and does not achieve optimal results when
exposed to articles from other domains. Since articles from different domains
have a unique textual structure, it is difficult to train a generic algorithm that
works best on all particular news domains. In this paper, we propose a solution to
the fake news detection problem using the machine learning ensemble approach.
Our study explores different textual properties that could be used to distinguish
fake contents from real.

1.2 DEEP LEARNING

Deep learning is a machine learning technique that teaches computers to

do what comes naturally to humans: learn by example. Deep learning is a key
technology behind driverless cars, enabling them to recognize a stop sign, or to
distinguish a pedestrian from a lamppost. It is the key to voice control in
consumer devices like phones, tablets, TVs, and hands-free speakers. Deep
learning is getting lots of attention lately and for good reason. It’s achieving
results that were not possible before. In deep learning, a computer model learns
to perform classification tasks directly from images, text, or sound. Deep learning
models can achieve state-of-the-art accuracy, sometimes exceeding human-level
performance.

Models are trained by using a large set of labeled data and neural network
architectures that contain many layers. Deep learning achieves recognition
accuracy at higher levels than ever before. This helps consumer electronics meet
user expectations, and it is crucial for safety-critical applications like driverless
cars. Recent advances in deep learning have improved to the point where deep
learning outperforms humans in some tasks like classifying objects in images.
While deep learning was first theorized in the 1980s, there are two main reasons
it has only recently become useful: Deep learning requires large amounts of

2
labeled data. For example, driverless car development requires millions of images
and thousands of hours of video. Deep learning requires substantial computing
power. High-performance GPUs have a parallel architecture that is efficient for
deep learning. When combined with clusters or cloud computing, this enables
development teams to reduce training time for a deep learning network from
weeks to hours or less.

1.3 APPLICATIONS OF DEEP LEARNING

Deep learning applications are used in industries from automated driving

to medical devices.

Automated Driving: Automotive researchers are using deep learning to

automatically detect objects such as stop signs and traffic lights. In addition, deep
learning is used to detect pedestrians, which helps decrease accidents.

Aerospace and Defense: Deep learning is used to identify objects from satellites
that locate areas of interest, and identify safe or unsafe zones for troops.

Medical Research: Cancer researchers are using deep learning to automatically

detect cancer cells. Teams at UCLA built an advanced microscope that yields a
high-dimensional data set used to train a deep learning application to accurately
identify cancer cells.

Industrial Automation: Deep learning is helping to improve worker safety around

heavy machinery by automatically detecting when people or objects are within
an unsafe distance of machines.

Electronics: Deep learning is being used in automated hearing and speech

translation. For example, home assistance devices that respond to your voice and
know your preferences are powered by deep learning applications.

3
1.4 DEEP LEARNING WORKS

Most deep learning methods use neural network architectures, which is

why deep learning models are often referred to as deep neural networks. The term
“deep” usually refers to the number of hidden layers in the neural
network. Traditional neural networks only contain 2-3 hidden layers, while deep
networks can have as many as 150.Deep learning models are trained by using
large sets of labeled data and neural network architectures that learn features
directly from the data without the need for manual feature extraction.One of the
most popular types of deep neural networks is known as convolutional neural
networks (CNN or ConvNet). A CNN convolves learned features with input
data, and uses 2D convolutional layers, making this architecture well suited to
processing 2D data, such as images.

CNNs eliminate the need for manual feature extraction, so you do not need
to identify features used to classify images. The CNN works by extracting
features directly from images. The relevant features are not pretrained; they are
learned while the network trains on a collection of images. This automated feature
extraction makes deep learning models highly accurate for computer vision tasks
such as object classification. CNNs learn to detect different features of an image
using tens or hundreds of hidden layers. Every hidden layer increases the
complexity of the learned image features. For example, the first hidden layer
could learn how to detect edges, and the last learns how to detect more complex
shapes specifically catered to the shape of the object we are trying to recognize.

Deep learning is a specialized form of machine learning. A machine

learning workflow starts with relevant features being manually extracted from
images. The features are then used to create a model that categorizes the objects
in the image. With a deep learning workflow, relevant features are automatically
extracted from images. In addition, deep learning performs “end-to-end learning”

4
– where a network is given raw data and a task to perform, such as classification,
and it learns how to do this automatically.Another key difference is deep learning
algorithms scale with data, whereas shallow learning converges. Shallow learning
refers to machine learning methods that plateau at a certain level of performance
when you add more examples and training data to the network. A key advantage
of deep learning networks is that they often continue to improve as the size of
your data increases.The three most common ways people use deep learning to
perform object classification are:

Training from Scratch

To train a deep network from scratch, you gather a very large labeled data
set and design a network architecture that will learn the features and model. This
is good for new applications, or applications that will have a large number of
output categories. This is a less common approach because with the large amount
of data and rate of learning, these networks typically take days or weeks to train.

Transfer Learning

Most deep learning applications use the transfer learning approach, a

process that involves fine-tuning a pretrained model. You start with an existing
network, such as AlexNet or GoogLeNet, and feed in new data containing
previously unknown classes. After making some tweaks to the network, you can
now perform a new task, such as categorizing only dogs or cats instead of 1000
different objects. This also has the advantage of needing much less data
(processing thousands of images, rather than millions), so computation time drops
to minutes or hours.

Feature Extraction

5
 A slightly less common, more specialized approach to deep learning is to
use the network as a feature extractor. Since all the layers are tasked with learning
certain features from images, we can pull these features out of the network at any
time during the training process. These features can then be used as input to
a machine learning model such as support vector machines (SVM).

1.5 PREDICTIVE ANALYSIS

Predictive analytics uses historical data to predict future events. Typically,

historical data is used to build a mathematical model that captures important
trends. That predictive model is then used on current data to predict what will
happen next, or to suggest actions to take for optimal outcomes. Predictive
analytics has received a lot of attention in recent years due to advances in
supporting technology, particularly in the areas of big data and machine learning.
Predictive analytics is often discussed in the context of big data, Engineering
data, for example, comes from sensors, instruments, and connected systems out
in the world. Business system data at a company might include transaction data,
sales results, customer complaints, and marketing information. Increasingly,
businesses make data-driven decisions based on this valuable trove of
information. To extract value from big data, businesses apply algorithms to large
data sets using tools such as Hadoop and Spark. The data sources might consist
of transactional databases, equipment log files, images, video, audio, sensor, or
other types of data. Innovation often comes from combining data from several
sources.

With all this data, tools are necessary to extract insights and
trends. Machine learning techniques are used to find patterns in data and to build
models that predict future outcomes. A variety of machine learning algorithms
are available, including linear and nonlinear regression, neural networks, support

6
vector machines, decision trees, and other algorithms. Predictive analytics helps
teams in industries as diverse as finance, healthcare, pharmaceuticals,
automotive, aerospace, and manufacturing.

• Automotive – Breaking new ground with autonomous vehicles

o Companies developing driver assistance technology and new
autonomous vehicles use predictive analytics to analyze sensor data
from connected vehicles and to build driver assistance algorithms.
• Aerospace – Monitoring aircraft engine health
o To improve aircraft up-time and reduce maintenance costs, an engine
manufacturer created a real-time analytics application to predict
subsystem performance for oil, fuel, liftoff, mechanical health, and
controls.
• Energy Production – Forecasting electricity price and demand
o Sophisticated forecasting apps use models that monitor plant
availability, historical trends, seasonality, and weather.
• Financial Services – Developing credit risk models
o Financial institutions use machine learning techniques and quantitative
tools to predict credit risk.
• Industrial Automation and Machinery – Predicting machine failures
o A plastic and thin film producer saves 50,000 Euros monthly using a
health monitoring and predictive maintenance application that reduces
downtime and minimizes waste.
• Medical Devices – Using pattern-detection algorithms to spot asthma and
COPD
An asthma management device records and analyzes patients' breathing
sounds and provides instant feedback via a smart phone app to help patients
manage asthma and COPD.

7
Predictive analytics is the process of using data analytics to make predictions
based on data. This process uses data along with analysis, statistics, and machine
learning techniques to create a predictive model for forecasting future events.

The term “predictive analytics” describes the application of a statistical or

machine learning technique to create a quantitative prediction about the future.
Frequently, supervised machine learning techniques are used to predict a future
value (How long can this machine run before requiring maintenance?) or to
estimate a probability (How likely is this customer to default on a loan?).
Predictive analytics starts with a business goal: to use data to reduce waste, save
time, or cut costs. The process harnesses heterogeneous, often massive, data sets
into models that can generate clear, actionable outcomes to support achieving that
goal, such as less material waste, less stocked inventory, and manufactured
product that meets specifications.

1.6 MACHINE LEARNING

Machine learning is a data analytics technique that teaches computers to

do what comes naturally to humans and animals: learn from experience. Machine
learning algorithms use computational methods to “learn” information directly
from data without relying on a predetermined equation as a model. The
algorithms adaptively improve their performance as the number of samples
available for learning increases. Deep learning is a specialized form of machine
learning. With the rise in big data, machine learning has become a key technique
for solving problems in areas, such as:

• Computational finance, for credit scoring and algorithmic trading

• Image processing and computer vision, for face recognition, motion
detection, and object detection

8
 • Computational biology, for tumor detection, drug discovery, and DNA
sequencing
• Energy production, for price and load forecasting
• Automotive, aerospace, and manufacturing, for predictive maintenance
• Natural language processing, for voice recognition applications

Machine learning uses two types of techniques: supervised learning, which

trains a model on known input and output data so that it can predict future outputs,
and unsupervised learning, which finds hidden patterns or intrinsic structures in
input data.

Supervised Learning

Supervised machine learning builds a model that makes predictions based

on evidence in the presence of uncertainty. A supervised learning algorithm takes
a known set of input data and known responses to the data (output) and trains a
model to generate reasonable predictions for the response to new data. Use
supervised learning if you have known data for the output you are trying to
predict. Supervised learning uses classification and regression techniques to
develop predictive models.

Classification techniques predict discrete responses—for example, whether an

email is genuine or spam, or whether a tumor is cancerous or benign.
Classification models classify input data into categories. Typical applications
include medical imaging, speech recognition, and credit scoring. Use
classification if your data can be tagged, categorized, or separated into specific
groups or classes. For example, applications for hand-writing recognition use
classification to recognize letters and numbers.

In image processing and computer vision, unsupervised pattern

recognition techniques are used for object detection and image

9
segmentation.Common algorithms for performing classification include support
vector machine (SVM), boosted and bagged decision trees, k-nearest
neighbor, Naïve Bayes, discriminant analysis, logistic regression, and neural
networks.

Regression techniques predict continuous responses—for example, changes in

temperature or fluctuations in power demand. Typical applications include
electricity load forecasting and algorithmic trading. Use regression techniques if
you are working with a data range or if the nature of your response is a real
number, such as temperature or the time until failure for a piece of equipment.
Common regression algorithms include linear model, nonlinear model,
regularization, stepwise regression, boosted and bagged decision trees, neural
networks, and adaptive neuro-fuzzy learning.

Unsupervised Learning

Unsupervised learning finds hidden patterns or intrinsic structures in data.

It is used to draw inferences from datasets consisting of input data without labeled
responses.

Clustering is the most common unsupervised learning technique. It is used for

exploratory data analysis to find hidden patterns or groupings in data.
Applications for cluster analysis include gene sequence analysis, market
research, and object recognition.

1.7 ARTIFICIAL INTELLIGENCE

Artificial intelligence, or AI, is a simulation of intelligent human behavior.

It’s a computer or system designed to perceive its environment, understand its
behaviors, and take action. Consider self-driving cars: AI-driven systems like
these integrate AI algorithms, such as machine learning and deep learning, into
complex environments that enable automation. Data Preparation
10
 Taking raw data and making it useful for an accurate, efficient, and
meaningful model is a critical step. In fact, it represents most of your AI effort.
Data preparation requires domain expertise, such as experience in speech and
audio signals, navigation and sensor fusion, image and video processing, and
radar and lidar. Engineers in these fields are best suited to determine what the
critical features of the data are, which are unimportant, and what rare events to
consider.

AI also involves prodigious amounts of data. Yet labeling data and images
is tedious and time-consuming. Sometimes, you don’t have enough data,
especially for safety-critical systems. Generating accurate synthetic data can
improve your data sets. In both cases, automation is critical to meeting deadlines.

Deployment

AI models need to be deployed to CPUs, GPUs, and/or FPGAs in your

final product, whether part of an embedded or edge device, enterprise system, or
cloud. AI models running on the embedded or edge device provide the quick
results needed in the field, while AI models running in enterprise systems and the
cloud provide results from data collected across many devices. Frequently, AI
models are deployed to a combination of these systems.

The deployment process is accelerated when you generate code from your
models and target your devices. Using code generation optimization techniques
and hardware-optimized libraries, you can tune the code to fit the low power
profile required by embedded and edge devices or the high-performance needs of
enterprise systems and the cloud.

11
1.8 REINFORCEMENT LEARNING

In control systems that benefit from learning based on cumulative reward,

reinforcement learning is an ideal technique. Reinforcement Learning
Toolbox™ lets you train policies using DQN, A2C, DDPG, and other
reinforcement learning algorithms. You can use these policies to implement
controllers and decision-making algorithms for complex systems such as robots
and autonomous systems. You can implement the policies using deep neural
networks, polynomials, or lookup tables. Reinforcement learning is a type of
machine learning technique where a computer agent learns to perform a task
through repeated trial and error interactions with a dynamic environment. This
learning approach enables the agent to make a series of decisions that maximize
a reward metric for the task without human intervention and without being
explicitly programmed to achieve the task.

AI programs trained with reinforcement learning beat human players in

board games like Go and chess, as well as video games. While reinforcement
learning is by no means a new concept, recent progress in deep learning and
computing power made it possible to achieve some remarkable results in the area
of artificial intelligence. Reinforcement learning is a branch of machine learning
(Figure 1). Unlike unsupervised and supervised machine learning, reinforcement
learning does not rely on a static dataset, but operates in a dynamic environment
and learns from collected experiences. Data points, or experiences, are collected
during training through trial-and-error interactions between the environment and
a software agent.

This aspect of reinforcement learning is important, because it alleviates the

need for data collection, preprocessing, and labeling before training, otherwise
necessary in supervised and unsupervised learning. Practically, this means that,
given the right incentive, a reinforcement learning model can start learning a

12
behavior on its own, without (human) supervision. Deep learning spans all three
types of machine learning; reinforcement learning and deep learning are not
mutually exclusive. Complex reinforcement learning problems often rely on deep
neural networks, a field known as deep reinforcement learning.

Deep neural networks trained with reinforcement learning can encode

complex behaviors. This allows an alternative approach to applications that are
otherwise intractable or more challenging to tackle with more traditional
methods. For example, in autonomous driving, a neural network can replace the
driver and decide how to turn the steering wheel by simultaneously looking at
multiple sensors such as camera frames and lidar measurements.

Without neural networks, the problem would normally be broken down in

smaller pieces like extracting features from camera frames, filtering the lidar
measurements, fusing the sensor outputs, and making “driving” decisions based
on sensor inputs. While reinforcement learning as an approach is still under
evaluation for production systems, some industrial applications are good
candidates for this technology.

Advanced controls: Controlling nonlinear systems is a challenging problem that

is often addressed by linearizing the system at different operating points.
Reinforcement learning can be applied directly to the nonlinear system.

Automated driving: Making driving decisions based on camera input is an area

where reinforcement learning is suitable considering the success of deep neural
networks in image applications.

Robotics: Reinforcement learning can help with applications like robotic

grasping, such as teaching a robotic arm how to manipulate a variety of objects

13
for pick-and-place applications. Other robotics applications include human-robot
and robot-robot collaboration.

Scheduling: Scheduling problems appear in many scenarios including traffic

light control and coordinating resources on the factory floor towards some
objective. Reinforcement learning is a good alternative to evolutionary methods
to solve these combinatorial optimization problems.

Calibration: Applications that involve manual calibration of parameters, such as

electronic control unit (ECU) calibration, may be good candidates for
reinforcement learning.

14
 CHAPTER 2

2. LITERATURE SURVEY

Lev konstantinovskiy, et.al,…[1] assist fact checkers in the process of fact

checking, we tackle the claim detection task, one of the necessary stages prior to
determining the veracity of a claim. It consists of identifying the set of sentences,
out of a long text, deemed capable of being fact checked. This paper is a
collaborative work between Full Fact, an independent fact checking charity, and
academic partners. Leveraging the expertise of professional fact checkers, we
develop an annotation schema and a benchmark for automated claim detection
that is more consistent across time, topics and annotators than previous
approaches. Our annotation schema has been used to crowd source the annotation
of a dataset with sentences from UK political TV shows. We introduce an
approach based on universal sentence representations to perform the
classification, achieving an F1 score of 0.83, with over 5% relative improvement
over the state-of-the-art methods Claim Buster and Claim Rank. The system was
deployed in production and received positive user feedback. However, there is a
very small number of fact checking organizations in the world, about 160,1
compared to the volume of media items produced daily. The speed at which
information flows online means there is less time to verify the claims made and
myths spread further before being fact checked, if they are fact checked at all.
Automating any parts of the fact checking process could cut down the time it
takes to respond to a claim. It could also protect human fact checker’s time to
work on the more complicated checks that need careful human judgment.

Peter bourgonje, et.al,…[2] presented a system for the detection of the

stance of headlines with regard to their corresponding article bodies. The
approach can be applied in fake news, especially click bait detection scenarios.
The component is part of a larger platform for the curation of digital content; we
consider veracity and relevancy an increasingly important part of curating online

15
information. We want to contribute to the debate on how to deal with fake news
and related online phenomena with technological means, by providing means to
separate related from unrelated headlines and further classifying the related
headlines. We present a system for stance detection of headlines with regard to
their corresponding article bodies. Our system is based on simple, lemmatization-
based n-gram matching for the binary classification of “related” vs. “unrelated”
headline/article pairs. The best results were obtained using a setup where the more
fine-grained classification of the “related” pairs (into “agree”, “disagree”,
“discuss”) is carried out using a Logistic Regression classifier at first, then three
binary classifiers with slightly different training procedures for the cases where
the first classifier lacked confidence (i. e., the difference between the best and
second-best scoring class was below a threshold). For the more fine-grained
classification of articles that have been classified as “related”, the threeway
classification is a relevant first step, but other classes may need to be added to the
set, or a more detailed division may need to be made in order to take the next
steps in tackling the fake news challenge. Additionally, we see the integration of
known facts and general discourse knowledge (possibly through Linked Data),
and the incorporation of source credibility information as important and
promising suggestions for future research.

Sahil chopra, et.al,…[3] leveraged an SVM trained on TF-IDF cosine

similarity features to discern whether a headline-article pairing is related or
unrelated. If we classify the pairing as the former, we then employ various neural
network architectures built on top of Long-Short-Term-Memory Models
(LSTMs) to label the pairing as agree, disagree, or discuss. Ultimately, our best
performing neural network architecture proved to be a pair of Bidirectional
Conditionally Encoded LSTMs with Bidirectional Global Attention. Using our
linear SVM for the unrelated/related subproblem and our best neural network for
the agree/disagree/discuss subproblem, we scored .8658 according to the FNC-

16
1’s performance metric. As we developed our approach to FNC-1, we first
explored existing research as it pertains to related NLP problems in entailment as
well as stance detection. First, we examined papers regarding the Stanford
Natural Language (SNLI) Dataset, which has been become popular in recent
years when developing models to classify entailment and contradiction amongst
hypothesis-premise pairs. From the original SNLI paper (Bowman et al. 2015)
we derived two of our baseline models - a Bag of Words (BOW) Multilayer
Perceptron (MLP) and a Long-Short-Term-Memory (LSTM) that receives
concatenated hypothesis-premise pairs as inputs. Additionally, we drew heavily
upon Tim Rocktaschel’s Reasoning about Entailment with Neural Attention. In
the paper, Rocktaschel proposes an architecture of conditionally encoded LSTMs
upon which attention is applied in order to classify entailment on the SNLI
Dataset.

Todor mihaylov, et.al,…[4] extracted statistics such as number of

comments posted, number of days in the forum, number of days with at least one
comment, and number of publications commented on. All (other) features are
scaled with respect to these statistics, which makes it possible to handle users that
registered only recently. We crawled the largest Internet community forum of a
Bulgarian media. A daily newspaper that requires users to be signed in in order to
comment, which makes it easy to track them. The platform allows users to
comment on news, to reply to other users’ comments and to vote on them with
thumbs up or thumbs down. In the forum, the official language is Bulgarian and
all comments are written in Bulgarian.

Todor mihaylov, et.al,…[5] The reason could be that most troll comments
are replies to other comments, while those by nontrolls are mostly not replies.
Adding other features such as sentiment-based features, bad words, POS, and
punctuation hurts the performance significantly. Features such as bad words are
at the very bottom: they do not apply to all comments and thus are of little use

17
alone; similarly for mentions and sentiment features, which are also quite weak
in isolation. These results suggest that mentioned trolls are not that different from
non-trolls in terms of language use, but have mainly different behavior in terms
of replying to other users. We have presented experiments in predicting whether
a comment is written by a troll or not, where we define troll as somebody who
was called such by other people. We have shown that this is a useful definition
and that comments by mentioned trolls are similar to such by confirmed paid
trolls. Overall, we have seen that our classifier for telling apart comments by
mentioned trolls vs. such by non-trolls performs almost equally well for paid
trolls vs. non-trolls, where the non-troll comments are sampled from the same
threads that the troll comments come from. Moreover, the most and the least
important features ablated from all are also similar. This suggests that mentioned
trolls are very similar to paid trolls (except for their reply rate, time and day of
posting patterns)

18
 CHAPTER 3

3. EXISITNG SYSTEM

Fake news detection is a difficult problem due to the nuances of language.

Understanding the reasoning behind certain fake items implies inferring a lot of
details about the various actors involved. We believe that the solution to this
problem should be a hybrid one, combining machine learning, semantics and
natural language processing. The purpose of this project is not to decide for the
reader whether or not the document is fake, but rather to alert them that they need
to use extra scrutiny for some documents. Fake news detection, unlike spam
detection, has many nuances that aren’t as easily detected by text analysis.
Besides detecting fake news articles, identifying the fake news creators and
subjects will actually be more important, which will help completely eradicate a
large number of fake news from the origins in online social networks.

Generally, for the news creators, besides the articles written by them, we
are also able to retrieve his/her profile information from either the social network
website or external knowledge libraries, e.g., Wikipedia or government-internal
database, which will provide fundamental complementary information for his/her
background check. Based on various types of heterogeneous information sources,
including both textual contents/profile/descriptions and the authorship and article
subject relationships among them, we aim at identifying fake news from the
online social networks simultaneously. We formulate the fake news detection
problem as a credibility inference problem, where the real ones will have a higher
credibility while unauthentic ones will have a lower one instead

3.1 CONVENTIONAL CLASSIFIERS

3.1.1 RANDOM FOREST ALGORITHM

An algorithm for group learning is random forests. The algorithm's

fundamental premise is that it is computationally inexpensive to construct a tiny

19
decision-tree with few features. If we can construct multiple small, weak decision
trees concurrently, we can then average or take the majority vote to join the trees
to create a single, strong learner. It is frequently discovered in reality that random
forests are the most up until now precise learning techniques.

Algorithm 1 provides an illustration of the pseudo code.

The procedure is as follows: we choose a bootstrap sample from S, where S (i)

stands for the ith bootstrap, for each tree in the forest. Then, we discover a
decision-tree. By modifying the decision-tree learning method. The algorithm is
changed such that as follows: rather than looking at every potential feature-split
at each node of the tree, pick a subset of the features f F at random, where F is the
collection of features. After that, the node splits based on f's best feature rather
than F's. In reality, f is very, significantly more modest than F. Choosing which
feature to separate is frequently the most difficult an expensive computational
feature of decision tree learning by reducing the range. We greatly accelerate the
learning of the tree by the use of features.

Algorithm: Random Forest

Precondition: A training set S := (x1, y1), . . . ,(xn, yn), features F, and number of
trees in forest B.

1 function Random Forest(S, F)

1 H←∅
2 for i ∈ 1, . . . , B do
3 S (i) ← A bootstrap sample from S
4 hi ← RandomizedTreeLearn(S (i) , F)
5 H ← H ∪ {hi}
6 end for
7 return H
8 end function
20
9 function RandomizedTreeLearn(S , F)
10 At each node:
11 f ← very small subset of F
12 Split on best feature in f
13 return The learned tree
14 end function

3.1.2 NAVIES BAYES ALGORITHM

The Nave Bayes algorithm is a supervised learning method for

classification issues that is based on the Bayes theorem. It is mostly employed in
text categorization with a large training set. The Naive Bayes Classifier is one of
the most straightforward and efficient classification algorithms available today. It
aids in the development of quick machine learning models capable of making
accurate predictions. Being a probabilistic classifier, it makes predictions based
on the likelihood that an object will occur. Spam filtration, Sentimental analysis,
and article classification are a few examples of Naive Bayes algorithms that are
frequently used.

Algorithm: Navies Bayes

Input:

Training dataset T,

F=(f1,f2,f3,……fn) // value of the predictor variable in the testing dataset.

Output:

A class of testing dataset.

Step:

1. Read the training dataset T;

21
 2. Calculate the mean and standard deviation of the predictor variable in each
class;
3. Repeat
Calculate the probability of f i using the gauss density equation in
each class;

Until the probability of all predictor variables (f 1 ,f 2 ,f 3 ,….,f n ) has

been calculated.

4. Calculate the likelihood for each class;

5. Get the greatest likelihood;

3.2 CHALLENGES

While using machine learning classifiers for fake news detection can be effective,
there are also several challenges that need to be addressed to achieve accurate
results. Here are some of the main challenges:

• Lack of labeled data: Collecting and labeling a large amount of data is

essential for training machine learning models. However, labeling news
articles as either real or fake can be a time-consuming and subjective
process. Moreover, there may not be enough labeled data available to train
accurate models.
• Dataset bias: The quality and diversity of the labeled dataset used for
training the model can have a significant impact on its performance. A
biased dataset, where one class (real or fake) is overrepresented, can lead
to poor generalization and inaccurate results.
• Feature selection: Identifying relevant features from the text of news
articles is crucial for training accurate models. However, selecting the right
features can be challenging, as fake news can be highly sophisticated and
difficult to distinguish from real news.

22
 • Adversarial attacks: Adversarial attacks are deliberate attempts to
manipulate the model's predictions by injecting subtle changes into the
input data. Adversarial attacks can make it difficult for the model to detect
fake news accurately.
• Dynamic nature of fake news: Fake news is constantly evolving, and new
types of fake news can emerge quickly. This means that the model must be
able to adapt to new forms of fake news and continue to perform accurately
over time.

3.3 DISADVANTAGES

• Accuracy is less
• Need large number of datasets to train the data
• Provide high number of false positive rate
• Only done supervised classification

23
 CHAPTER 4

4. PROPOSED SYSTEM

"Fake News" is a term used to represent fabricated news or propaganda

comprising misinformation communicated through traditional media channels
like print, and television as well as non-traditional media channels like social
media. The general motive to spread such news is to mislead the readers, damage
reputation of any entity, or to gain from sensationalism. Fake news is increasingly
being shared via social media platforms like Twitter and Facebook. These
platforms offer a setting for the general population to share their opinions and
views in a raw and un-edited fashion. Some news articles hosted or shared on the
social media platforms have more views compared to direct views from the media
outlets’ platform.

Research that studied the velocity of fake news concluded that tweets
containing false information reach people on Twitter six times faster than truthful
tweets. Technologies such as Machine learning and Natural Language Processing
(NLP) tools offer great promise for researchers to build systems which could
automatically detect fake news. However, detecting fake news is a challenging
task to accomplish as it requires models to summarize the news and compare it
to the actual news in order to classify it as fake. Moreover, the task of comparing
proposed news with the original news itself is a daunting task as its highly
subjective and opinionated. In this project, we can implement text mining
algorithm to extract the key terms based on natural language processing and also
include classification algorithms such as deep learning algorithm named as multi-
layer perceptron algorithm.

4.1 MULTI-LAYER PERCEPTRON ALGORITHM

An Artificial Neural Network (ANN) which contains multiple layers

between the input and output layer is called Deep Neural Network. DNN works

24
on feed forward algorithm. Data flow is directed from input to output layer. DNN
creates a number of virtual neurons initialized with a random numerical value as
connection weights. This weight is multiplied with the input and produce an
output between 0 and 1. The training process adjust the weights to classify the
output efficiently. Added layers make the model to learn rare patterns which leads
the model to overfitting. Dropout layers reduce the number of trainable
parameters to make the model generalized. In this paper, we have used a
sequential model of dense layers for training the data, relu as activation function
and adam as optimizer. During training process, calculates individual learning
rates on different parameters as this is an adaptive learning method.

Multi-layer perceptron can be used as supervised classification tool by

incorporating optimized training parameters. For a given problem, the number of
hidden layers in a multilayer perceptron and the number of nodes in each layer
can differ. The decision of choosing the parameters depends on the training data
and the network architecture

The MLP learning procedure is as follows:

• Starting with the input layer, propagate data forward to the output layer.
This step is the forward propagation.

• Based on the output, calculate the error (the difference between the
predicted and known outcome). The error needs to be minimized.

• Backpropagate the error. Find its derivative with respect to each weight in
the network, and update the model.

Repeat the three steps given above over multiple epochs to learn ideal weights.
Finally, the output is taken via a threshold function to obtain the predicted class
labels. Figure 2 displays framework for multi-layer perceptron algorithm

25
 Fig 4.1 MLP ALGORITHM

4.2 ADVANTAGES

• Reduce the false positive rate

• Analyse all types of features
• Improve the accuracy rate
• Time complexity can be reduced

4.3 APPLICATIONS

Fake news detection using deep learning algorithms has numerous applications,
some of which include:

• Social media platforms: Social media platforms can use deep learning
algorithms to automatically detect and flag fake news articles shared by
users. This can help reduce the spread of misinformation and improve the
overall quality of content on these platforms.
• News organizations: News organizations can use deep learning algorithms
to verify the authenticity of news articles before publishing them. This can
help prevent the spread of fake news and maintain the credibility of news
outlets.

26
 • Government agencies: Government agencies can use deep learning
algorithms to monitor the spread of fake news and detect potential threats
to national security. This can help them take appropriate action to
counteract the effects of fake news.
• Educational institutions: educational institutions can use deep learning
algorithms to teach students how to identify fake news and distinguish it
from real news. This can help improve media literacy and critical thinking
skills among students.
• Fact-checking organizations: Fact-checking organizations can use deep
learning algorithms to automate the fact-checking process and speed up the
verification of news articles. This can help them keep up with the high
volume of news articles that need to be fact-checked on a daily basis.

4.4 DIFFERENCE BETWEEN EXISTING AND PROPOSED SYSTEM

Learning Approach:

• Random Forest and Naive Bayes algorithms are based on an ensemble

learning approach, combining multiple models or making strong
assumptions about feature independence, respectively.
• Multi Perceptron algorithm takes a different approach as a feedforward
neural network, utilizing multiple layers of interconnected nodes and
backpropagation for training.

Non-Linearity and Complex Relationships:

• Multi Perceptron algorithm can capture non-linear relationships and handle

complex patterns in the data due to its ability to model intricate decision
boundaries.
• Random Forest, which approximates non-linear relationships using
decision trees and ensemble techniques, and Naive Bayes, which assumes
feature independence.

27
Flexibility and Adaptability:

• Multi Perceptron algorithm can adapt its internal weights and biases during
training, allowing it to learn and adapt to different datasets and problem
domains.
• Random Forest, which relies on an ensemble of decision trees with fixed
structures, and Naive Bayes, which assumes fixed probabilistic
relationships between features.

Generalization and Overfitting:

• Multi Perceptron algorithm's training process includes techniques like

regularization, dropout, and early stopping to mitigate overfitting and
improve generalization to unseen data.
• Random Forest, which has built-in mechanisms like random feature
selection and bagging to reduce overfitting, and Naive Bayes, which can
suffer from overfitting in complex datasets due to its feature independence
assumption.

Scalability and Computational Complexity:

• Multi Perceptron algorithm can be computationally expensive to train and

may require larger datasets for optimal performance due to its multilayer
architecture and backpropagation algorithm.
• Random Forest, which can handle large datasets and parallel processing
efficiently, and Naive Bayes, which has relatively low computational
complexity and can scale well.

28
 CHAPTER 5

5. SYSTEM DESIGN

5.1 SYSTEM ARCHITECTURE

System architecture refers to the conceptual design of a software or

hardware system, which defines the various components, modules, and their
interrelationships. It provides a high-level view of the system and its
functionality, as well as the interaction between the system and its environment.
In this architecture, we can train the datasets that are collected from Web sources
named as KAGGLE and extract the keywords using Natural language processing.
Then build the model using MLP algorithm. In testing phase, user can input the
text and classify the text whether it is fake or real

Fig5.1 SYSTEM ARCHITECTURE

29
5.2 MODULES

• TRAIN THE DOCUMENTS

• TEXT MINING

• DOCUMENT TERM MATRIX CONSTRUCTION

• CLASSIFICATION

• FAKE NEWS DETECTION

5.2.1 TRAIN THE DOCUMENTS

Today internet contains vast amount of electronic collections that often

contain high quality information. However, usually the Internet provides more
information than is needed. User wants to select best collection of data for
particular information needs in minimum possible time. Text summarization is
one of the applications of information retrieval, which is the method of
condensing the input text into a shorter version, preserving its information content
and overall meaning. There has been a huge amount of work on query specific
summarization of documents using similarity measure. The any standard text file
can be uploaded to this module. In this module, can collect large number of news
datasets. In this module we can upload the datasets from users and upload the
news group datasets. A data set (or dataset, although this spelling is not present
in many contemporary dictionaries) is a collection of data. The data set lists
values for each of the variables, such as text of an object, for each member of the
data set.

5.2.2 TEXT MINING:

In the first step, the text documents are collected which are present in .TXT.

Document Pre- Processing

30
 In this process, the given input document is processed for removing
redundancies, inconsistencies, separate words, stemming and documents are
prepared for next step, the stages performed are as follows:

Tokenization

The given document is considered as a string and identifying single

word in document i.e., the given document string is divided into one unit
or token

Removal of Stop Word

In this step the removal of usual words like a, an, but, and, of, the
etc. is done.

Stemming

A stem is a natural group of words with equal (or very similar)

meaning. This method describes the base of particular word. Inflectional
and derivational stemming are two types of method. One of the popular
algorithms for stemming is porter’s algorithm.

5.2.3 DOCUMENT TERM MATRIX CONSTRUCTION

In this module, can calculate the term frequency and inverse document
frequency. In information retrieval, tf–idf or TFIDF, short for term frequency–
inverse document frequency, is a numerical statistic that is intended to reflect how
important a word is to a document in a collection or corpus. It is often used as a
weighting factor in searches of information retrieval, text mining, and user
modelling. The tf-idf value increases proportionally to the number of times a
word appears in the document and is offset by the frequency of the word in the
corpus, which helps to adjust for the fact that some words appear more frequently
in general. The calculate the values of entropy and probability of IDF. Entropy
gives higher weight to the terms with less frequency in few documents. Normal
is used to correct discrepancies in document lengths and also normalize the

31
document vectors. ProbIDF is similar to IDF and assigns very low negative
weight for the terms occurring in every document.

5.2.4 CLASSIFICATION

User can input the news datasets or twitter datasets. In this module,
implement multi-layer perceptron algorithm to classify the extract keywords. A
multilayer perceptron (MLP) is a feed forward artificial neural network that
generates a set of outputs from a set of inputs. An MLP is characterized by several
layers of input nodes connected as a directed graph between the input nodes
connected as a directed graph between the input and output layers. MLP uses
back propagation for training the network. MLP is a deep learning method. Multi-
layer perceptron (MLP) is a supplement of feed forward neural network. The
input layer receives the input signal to be processed. The required task such as
prediction and classification is performed by the output layer. An arbitrary
number of hidden layers that are placed in between the input and output layer are
the true computational engine of the MLP.
Similar to a feed forward network in a MLP the data flows in the forward
direction from input to output layer. The neurons in the MLP are trained with
the back propagation learning algorithm. MLPs are designed to approximate any
continuous function and can solve problems which are not linearly separable. The
major use cases of MLP are pattern classification, recognition, prediction and
approximation. MLPs are global approximators and can be trained to implement
any given nonlinear input-output mapping. In a subsequent testing phase, they
prove their interpolation ability by generalizing even in sparse data space regions.
When designing a neural network, specifically deciding for a fixed architecture,
performance and computational complexity considerations play a crucial role.
Mathematically, it has been proved that even one hidden-layer MLP is able to
approximate the mapping of any continuous function.

32
 As with all neural networks, the dimension of the input vector dictates the
number of neurons in the input layer, while the number of classes to be learned
dictates the number of neurons in the output layer. The number of chosen hidden
layers and the number of neurons in each layer have to be empirically determined.
5.2.5 FAKE NEWS DETECTION:

Classification of any news item /post / blog into fake or real one has
generated great interest from researchers around the globe. Several research
studies have been carried out to find effect of falsified and fabricated news on
masses and reactions of people upon coming through such news items. Falsified
news or fabricated new is any textual or non-textual content that is fake and is
generated so the readers will start believing in something which is not true. Based
on classification, fake news data are predicted. The proposed system provides
improved accuracy rate in fake news detection. If the user continuously posts the
fake news means, warn or block the users. Accuracy parameter is calculated in
terms of true positive and false positive rate

33
 CHAPTER 6

6. SYSTEM SPECIFICATIONS

6.1 HARDWARE SPECIFICATION

• Processor : Intel core processor 2.6.0 GHZ

• RAM : 4 GB
• Hard disk : 160 GB
• Compact Disk : 650 Mb
• Keyboard : Standard keyboard
• Monitor : 15 inch color monitor

6.2 SOFTWARE REQUIREMENTS

• Operating System : Windows OS

• Front-End : PYTHON
• IDE : PYCHARM
• Libraries : SCIKIT LEARN, MATPLOTLIB

34
6.3 SOFTWARE DESCRIPTION

Python is a high-level, interpreted programming language that is widely

used in various domains such as web development, scientific computing, data
analysis, artificial intelligence, machine learning, and more. It was first released
in 1991 by Guido van Rossum and has since become one of the most popular
programming languages due to its simplicity, readability, and versatility. One of
the key features of Python is its easy-to-learn syntax, which makes it accessible
to both novice and experienced programmers. It has a large standard library that
provides a wide range of modules for tasks such as file I/O, networking, regular
expressions, and more. Python also has a large and active community of
developers who contribute to open source libraries and packages that extend its
capabilities.

Python is an interpreted language, which means that it is executed line-by-

line by an interpreter rather than compiled into machine code like C or C++. This
allows for rapid development and testing, as well as easier debugging and
maintenance of code. Python is used for a variety of applications, including web
development frameworks such as Django and Flask, scientific computing
libraries such as NumPy and Pandas, and machine learning libraries such as
TensorFlow and PyTorch. It is also commonly used for scripting and automation
tasks due to its ease of use and readability. Overall, Python is a powerful and
versatile programming language that is widely used in a variety of domains due
to its simplicity, ease of use, and active community.

35
 Fig 6.1 PYTHON LOGO

Python is an interpreted high-level programming language for general-

purpose programming. Created by Guido van Rossum and first released in 1991,
Python has a design philosophy that emphasizes code readability, notably using
significant whitespace. It provides constructs that enable clear programming on
both small and large scales. In July 2018, Van Rossum stepped down as the leader
in the language community. Python features a dynamic type system and automatic
memory management.

It supports multiple programming paradigms, including object-oriented,

imperative, functional and procedural, and has a large and comprehensive
standard library. Python interpreters are available for many operating systems.
CPython, the reference implementation of Python, is open source software and
has a community-based development model, as do nearly all of Python's other
implementations. Python and CPython are managed by the non-profit Python
Software Foundation. Rather than having all of its functionality built into its core,
Python was designed to be highly extensible. This compact modularity has made
it particularly popular as a means of adding programmable interfaces to existing
applications. Van Rossum's vision of a small core language with a large standard
36
library and easily extensible interpreter stemmed from his frustrations with ABC,
which espoused the opposite approach. While offering choice in coding
methodology, the Python philosophy rejects exuberant syntax (such as that of
Perl) in favor of a simpler, less-cluttered grammar. As Alex Martelli put it: "To
describe something as 'clever' is not considered a compliment in the Python
culture."Python's philosophy rejects the Perl "there is more than one way to do
it" approach to language design in favour of "there should be one—and preferably
only one—obvious way to do it".

Python's developers strive to avoid premature optimization, and reject

patches to non-critical parts of CPython that would offer marginal increases in
speed at the cost of clarity.[ When speed is important, a Python programmer can
move time-critical functions to extension modules written in languages such as
C, or use PyPy, a just-in-time compiler. CPython is also available, which
translates a Python script into C and makes direct C-level API calls into the
Python interpreter. An important goal of Python's developers is keeping it fun to
use. This is reflected in the language's name a tribute to the British comedy group
Monty Python and in occasionally playful approaches to tutorials and reference
materials, such as examples that refer to spam and eggs (from a famous Monty
Python sketch) instead of the standard for and bar.

A common neologism in the Python community is pythonic, which can

have a wide range of meanings related to program style. To say that code is
pythonic is to say that it uses Python idioms well, that it is natural or shows
fluency in the language, that it conforms with Python's minimalist philosophy and
emphasis on readability. In contrast, code that is difficult to understand or reads
like a rough transcription from another programming language is called
unpythonic. Python is an interpreted, object-oriented, high-level programming
language with dynamic semantics. Its high-level built in data structures,
combined with dynamic typing and dynamic binding, make it very attractive for

37
Rapid Application Development, as well as for use as a scripting or glue language
to connect existing components together. Python's simple, easy to learn syntax
emphasizes readability and therefore reduces the cost of program maintenance.
Python supports modules and packages, which encourages program modularity
and code reuse.

The Python interpreter and the extensive standard library are available in
source or binary form without charge for all major platforms, and can be freely
distributed. Often, programmers fall in love with Python because of the increased
productivity it provides. Since there is no compilation step, the edit-test-debug
cycle is incredibly fast. Debugging Python programs is easy: a bug or bad input
will never cause a segmentation fault. Instead, when the interpreter discovers an
error, it raises an exception. When the program doesn't catch the exception, the
interpreter prints a stack trace. Python also has a large and active community of
developers who contribute to a wide range of open-source libraries and tools,
making it easy to find and use pre-built code to solve complex problems.

Python has a wide range of applications, including:

Data Science: Python is one of the most popular languages for data science,
thanks to libraries like NumPy, Pandas, and Matplotlib that make it easy to
manipulate and visualize data.

Machine Learning: Python is also widely used in machine learning and artificial
intelligence, with libraries like TensorFlow, Keras, and Scikit-learn that provide
powerful tools for building and training machine learning models.

Web Development: Python is commonly used in web development, with

frameworks like Django and Flask that make it easy to build web applications
and APIs.

38
Scientific Computing: Python is used extensively in scientific computing, with
libraries like SciPy and SymPy that provide powerful tools for numerical analysis
and symbolic mathematics.

In addition to its versatility and ease of use, Python is also known for its
portability and compatibility. Python code can be run on a wide range of
platforms, including Windows, macOS, and Linux, and it can be integrated with
other languages like C and Java.

Overall, Python is a powerful and versatile programming language that is well-

suited for a wide range of applications, from data science and machine learning
to web development and scientific computing. Its simplicity, readability, and large
community of developers make it an ideal choice for beginners and experts alike.

There are two attributes that make development time in Python faster than in other
programming languages:

1. Python is an interpreted language, which precludes the need to compile

code before executing a program because Python does the compilation in the
background. Because Python is a high-level programming language, it abstracts
many sophisticated details from the programming code. Python focuses so much
on this abstraction that its code can be understood by most novice programmers.

2. Python code tends to be shorter than comparable codes. Although Python

offers fast development times, it lags slightly in terms of execution time.
Compared to fully compiling languages like C and C++, Python programs
execute slower. Of course, with the processing speeds of computers these days,
the speed differences are usually only observed in benchmarking tests, not in real-
world operations. In most cases, Python is already included in Linux distributions
and Mac OS X machines.

One of the strengths of Python is its rich ecosystem of third-party libraries

and tools. These libraries provide a wide range of functionality, from scientific

39
computing and data analysis to web development and machine learning. Some
popular Python libraries and frameworks include:

NumPy: a library for numerical computing in Python, providing support for large,
multi-dimensional arrays and matrices, along with a large collection of
mathematical functions to operate on these arrays.

Pandas: a library for data manipulation and analysis in Python, providing support
for reading and writing data in a variety of formats, as well as powerful tools for
manipulating and analyzing data.

Matplotlib: a plotting library for Python that provides a variety of visualization

tools, including line plots, scatter plots, bar plots, and more.

TensorFlow: an open-source machine learning library for Python that provides a

variety of tools and algorithms for building and training machine learning models.

Django: a popular web framework for Python that provides a full-stack

framework for building web applications, with support for everything from URL
routing to user authentication and database integration.

Python's popularity has also led to a large and active community of developers
who contribute to open-source projects and share code and resources online. This
community provides a wealth of resources for learning Python, including
tutorials, online courses, and forums for asking and answering questions.

Overall, Python is a versatile and powerful programming language that is well-

suited for a wide range of applications. Its simplicity, flexibility, and wide range
of libra

6.3.1 SCIKIT LEARN

Scikit-learn is a powerful machine learning library for Python that offers a

range of tools for implementing various machine learning algorithms. It is built
on top of popular scientific computing libraries in Python, such as NumPy and

40
SciPy, and is designed to be user-friendly, efficient, and extensible. One of the
key advantages of scikit-learn is its easy-to-use interface, which provides a
consistent way of implementing machine learning algorithms. This makes it easy
for both beginners and advanced users to use the library for various applications.
Scikit-learn also supports a wide range of machine learning algorithms, including
classification, regression, clustering, and dimensionality reduction, making it a
versatile tool for solving a variety of machine learning problems. Overall, scikit-
learn is a highly useful library for anyone interested in implementing machine
learning algorithms in Python.

In addition to its easy-to-use interface and wide range of supported algorithms,

scikit-learn also offers various other features that make it a popular choice for
implementing machine learning models. These include:

• Data preprocessing: Scikit-learn provides various tools for preprocessing

data before training a model. This includes scaling and normalization of
data, imputation of missing values, and feature selection.

• Model selection and evaluation: Scikit-learn provides tools for selecting

the best model for a given task and evaluating its performance. This
includes cross-validation, hyperparameter tuning, and model comparison.
• Integration with other libraries: Scikit-learn integrates well with other
popular Python libraries for data analysis, such as Pandas and Matplotlib,
making it easy to use in a data analysis pipeline.
• Active development: Scikit-learn is actively developed and maintained,
with regular updates and bug fixes.

Overall, scikit-learn is a highly versatile and powerful library that makes it

easy to implement machine learning algorithms in Python. Its user-friendly

41
interface, wide range of algorithms, and various other features make it a popular
choice for both beginners and advanced users in the machine learning community.

6.3.2 MATPLOTLIB

Matplotlib is a popular data visualization library for Python that provides

a wide range of tools for creating static, interactive, and animated plots. It is built
on top of NumPy, another popular scientific computing library in Python, and
provides a high-level interface for creating various types of plots, such as line
plots, scatter plots, bar charts, histograms, and more. One of the key advantages
of Matplotlib is its flexibility and customization options. It provides a wide range
of options for customizing the appearance of plots, including color schemes,
fonts, and styles. It also supports various types of data visualization techniques,
such as subplots, annotations, and legends, which can help to better convey the
meaning and insights of the data being plotted.

Another advantage of Matplotlib is its integration with other popular

Python libraries, such as Pandas and NumPy, making it easy to use in data
analysis pipelines. Additionally, Matplotlib provides support for various output
formats, including PDF, SVG, and PNG, making it easy to export plots in various
formats for use in reports, presentations, or web applications. Overall, Matplotlib
is a highly versatile and useful library for creating data visualizations in Python.
Its flexibility, customization options, and integration with other libraries make it
a popular choice for data scientists, researchers, and developers alike.

42
 CHAPTER 7

7. SYSTEM TESTING

7.1 TESTING

Software testing is an investigation conducted to provide stakeholders with

information about the quality of the product or service under test. Software testing
can also provide an objective, independent view of the software to allow the
business to appreciate and understand the risks of software implementation. Test
techniques include, but are not limited to, the process of executing a program or
application with the intent of finding software bugs (errors or other defects).
Software testing can be stated as the process of validating and verifying that a
computer program/application/product:

1. meets the requirements that guided its design and development,

2. works as expected,

3. It can be implemented with the same characteristics, and satisfies the needs of
stakeholders.

Software testing, depending on the testing method employed, can be

implemented at any time in the development process. Traditionally most of the
test effort occurs after the requirements have been defined and the coding process
has been completed, but in the agile approaches most of the test effort is on-going.
As such, the methodology of the test is governed by the chosen software
development methodology. Different software development models will focus the
test effort at different points in the development process. Newer development
models, such as Agile, often employ test-driven development and place an
increased portion of the testing in the hands of the developer, before it reaches a
formal team of testers. In a more traditional model, most of the test execution

43
occurs after the requirements have been defined and the coding process has been
completed.

7.1.1 OVERVIEW OF TESTING

Testing can never completely identify all the defects within software.
Instead, it furnishes a criticism or comparison that compares the state and
behaviour of the product against oracles principles or mechanisms by which
someone might recognize a problem. These oracles may include (but are not
limited to) specifications, contracts, comparable products, past versions of the
same product, inferences about intended or expected purpose, user or customer
expectations, relevant standards, applicable laws, or other criteria. A primary
purpose of testing is to detect software failures so that defects may be discovered
and corrected.

Testing cannot establish that a product functions properly under all

conditions but can only establish that it does not function properly under specific
conditions. The scope of software testing often includes examination of code as
well as execution of that code in various environments and conditions as well as
examining the aspects of code: does it do what it is supposed to do and do what
it needs to do. In the current culture of software development, a testing
organization may be separate from the development team. There are various roles
for testing team members. Information derived from software testing may be used
to correct the process by which software is developed. Every software product
has a target audience. For example, the audience for video game software is
completely different from banking software. Therefore, when an organization
develops or otherwise invests in a software product, it can assess whether the
software product will be acceptable to its end users, its target audience, its
purchasers, and other stakeholders. Software testing is the process of attempting
to make this assessment

44
7.2 TESTING METHODS

7.2.1 Static vs. Dynamic Testing

There are many approaches to software testing. Reviews, walkthroughs, or

inspections are referred to as static testing, whereas actually executing
programmed code with a given set of test cases is referred to as dynamic testing.
Static testing can be omitted, and unfortunately in practice often is. Dynamic
testing takes place when the program itself is used. Dynamic testing may begin
before the program is 100% complete in order to test particular sections of code
and are applied to discrete functions or modules. Typical techniques for this are
either using stubs/drivers or execution from a debugger environment. The box
approach Software testing methods are traditionally divided into white- and
black-box testing. These two approaches are used to describe the point of view
that a test engineer takes when designing test cases.

7.2.2 White-Box Testing

White-box testing (also known as clear box testing, glass box testing,
transparent box testing, and structural testing) tests internal structures or workings
of a program, as opposed to the functionality exposed to the end users. In white-
box testing an internal perspective of the system, as well as programming skills,
are used to design test cases. The tester chooses inputs to exercise paths through
the code and determine the appropriate outputs. This is analogous to testing nodes
in a circuit, e.g., in-circuit testing (ICT). While white-box testing can be applied
at the unit, integration and system levels of the software testing process, it is
usually done at the unit level. It can test paths within a unit, paths between units
during integration, and between subsystems during a system–level test. Though
this method of test design can uncover many errors or problems, it might not
detect unimplemented parts of the specification or missing requirements.
Techniques used in white-box testing include: 1. API testing (application

45
programming interface) - testing of the application using public and private APIs
2. Code coverage - creating tests to satisfy some criteria of code coverage (e.g.,
the test designer can create tests to cause all statements in the program to be
executed at least once) 3. Fault injection methods - intentionally introducing
faults to gauge the efficacy of testing strategies

7.2.3 Static Testing Methods

Code coverage tools can evaluate the completeness of a test suite that was
created with any method, including black-box testing. This allows the software
team to examine parts of a system that are rarely tested and ensures that the most
important function points have been tested. Code coverage as a software metric
can be reported as a percentage for: 1. Function coverage, which reports on
functions executed 2. Statement coverage, which reports on the number of lines
executed to complete the test 100% statement coverage ensures that all code
paths, or branches (in terms of control flow) are executed at least once. This is
helpful in ensuring correct functionality, but not sufficient since the same code
may process different inputs correctly or incorrectly.

7.2.4 Black-Box Testing

Black-box testing treats the software as a "black box", examining

functionality without any knowledge of internal implementation. The tester is
only aware of what the software is supposed to do, not how it does it. Black-box
testing methods include: equivalence partitioning, boundary value analysis, all-
pairs testing, state transition tables, decision table testing, fuzz testing, model-
based testing, use case testing, exploratory testing and specification-based testing.
Specification-based testing aims to test the functionality of software according to
the applicable requirements. This level of testing usually requires thorough test
cases to be provided to the tester, who then can simply verify that for a given
input, the output value (or behavior), either "is" or "is not" the same as the

46
expected value specified in the test case. Test cases are built around specifications
and requirements, i.e., what the application is supposed to do. It uses external
descriptions of the software, including specifications, requirements, and designs
to derive test cases. These tests can be functional or non-functional, though
usually functional. Specification based testing may be necessary to assure correct
functionality, but it is insufficient to guard against complex or high-risk
situations. One advantage of the black box technique is that no programming
knowledge is required.

Whatever biases the programmers may have had, the tester likely has a
different set and may emphasize different areas of functionality. On the other
hand, black-box testing has been said to be "like a walk in a dark labyrinth without
a flashlight." Because they do not examine the source code, there are situations
when a tester writes many test cases to check something that could have been
tested by only one test case, or leaves some parts of the program untested. This
method of test can be applied to all levels of software testing: unit, integration,
system and acceptance. It typically comprises most if not all testing at higher
levels, but can also dominate unit testing as well.

7.2.5 Grey-Box Testing

Grey-box testing involves having knowledge of internal data structures and

algorithms for purposes of designing tests, while executing those tests at the user,
or black-box level. The tester is not required to have full access to the software's
source code. Manipulating input data and formatting output do not qualify as
grey-box, because the input and output are clearly outside of the "black box" that
we are calling the system under test. This distinction is particularly important
when conducting integration testing between two modules of code written by two
different developers, where only the interfaces are exposed for test. However,
modifying a data repository does qualify as grey-box, as the user would not
normally be able to change the data outside of the system under test. Grey-box

47
testing may also include reverse engineering to determine, for instance, boundary
values or error messages.

By knowing the underlying concepts of how the software works, the tester
makes better-informed testing choices while testing the software from outside.
Typically, a grey-box tester will be permitted to set up his testing environment;
for instance, seeding a database; and the tester can observe the state of the product
being tested after performing certain actions. For instance, in testing a database
product he/she may fire an SQL query on the database and then observe the
database, to ensure that the expected changes have been reflected. Grey-box
testing implements intelligent test scenarios, based on limited information. This
will particularly apply to data type handling, exception handling, and so on.

Visual testing

The aim of visual testing is to provide developers with the ability to

examine what was happening at the point of software failure by presenting the
data in such a way that the developer can easily find the information he requires,
and the information is expressed clearly. At the core of visual testing is the idea
that showing someone a problem (or a test failure), rather than just describing it,
greatly increases clarity and understanding. Visual testing therefore requires the
recording of the entire test process – capturing everything that occurs on the test
system in video format. Output videos are supplemented by real-time tester input
via picture-in-a-picture webcam and audio commentary from microphones.
Visual testing provides a number of advantages.

The quality of communication is increased dramatically because testers can

show the problem (and the events leading up to it) to the developer as opposed to
just describing it and the need to replicate test failures will cease to exist in many
cases. The developer will have all the evidence he requires of a test failure and
can instead focus on the cause of the fault and how it should be fixed. Visual

48
testing is particularly well-suited for environments that deploy agile methods in
their development of software, since agile methods require greater
communication between testers and developers and collaboration within small
teams. Visual testing is gathering recognition in customer acceptance and
usability testing, because the test can be used by many individuals involved in the
development process. For the customer, it becomes easy to provide detailed bug
reports and feedback, and for program users, visual testing can record user actions
on screen, as well as their voice and image, to provide a complete picture at the
time of software failure for the developer.

7.3 TESTING LEVELS

Tests are frequently grouped by where they are added in the software
development process, or by the level of specificity of the test. The main levels
during the development process as defined by the SWEBOK guide are unit-,
integration-, and system testing that are distinguished by the test target without
implying a specific process model. Other test levels are classified by the testing
objective.

7.3.1 Unit Testing

Unit testing, also known as component testing, refers to tests that verify the
functionality of a specific section of code, usually at the function level. In an
object-oriented environment, this is usually at the class level, and the minimal
unit tests include the constructors and destructors. These types of tests are usually
written by developers as they work on code (white-box style), to ensure that the
specific function is working as expected. One function might have multiple tests,
to catch corner cases or other branches in the code. Unit testing alone cannot
verify the functionality of a piece of software, but rather is used to assure that the
building blocks the software uses work independently of each other.

7.3.2 Integration Testing

49
 Integration testing is any type of software testing that seeks to verify the
interfaces between components against a software design. Software components
may be integrated in an iterative way or all together ("big bang"). Normally the
former is considered a better practice since it allows interface issues to be
localised more quickly and fixed. Integration testing works to expose defects in
the interfaces and interaction between integrated components (modules).
Progressively larger groups of tested software components corresponding to
elements of the architectural design are integrated and tested until the software
works as a system.

7.3.3 System Testing

System testing tests a completely integrated system to verify that it meets

its requirements.

7.3.4 Acceptance Testing

At last the system is delivered to the user for Acceptance testing

7.4 TESTING APPROACHES

7.4.1 Bottom-Up

Bottom-Up Testing is an approach to integrated testing where the lowest

level components are tested first, then used to facilitate the testing of higher-level
components. The process is repeated until the component at the top of the
hierarchy is tested. All the bottom or low-level modules, procedures or functions
are integrated and then tested. After the integration testing of lower-level
integrated modules, the next level of modules will be formed and can be used for
integration testing. This approach is helpful only when all or most of the modules
of the same development level are ready. This method also helps to determine the
levels of software developed and makes it easier to report testing progress in the
form of a percentage.

50
7.4.2 Top-Down

Top-Down Testing is an approach to integrated testing where the top

integrated modules are tested and the branch of the module is tested step by step
until the end of the related module.

7.5 OBJECTIVES OF TESTING

7.5.1 Installation Testing

An installation test assures that the system is installed correctly and

working at actual customer's hardware.

7.5.2 Compatibility Testing

A common cause of software failure (real or perceived) is a lack of its

compatibility with other application software, operating systems (or operating
system versions, old or new), or target environments that differ greatly from the
original (such as a terminal or GUI application intended to 35 be run on the
desktop now being required to become a web application, which must render in
a web browser). For example, in the case of a lack of backward compatibility, this
can occur because the programmers develop and test software only on the latest
version of the target environment, which not all users may be running. This results
in the unintended consequence that the latest work may not function on earlier
versions of the target environment, or on older hardware that earlier versions of
the target environment was capable of using. Sometimes such issues can be fixed
by proactively abstracting operating system functionality into a separate program
module or library.

7.5.3 Smoke and Sanity Testing

Sanity testing determines whether it is reasonable to proceed with further

testing. Smoke testing is used to determine whether there are serious problems
with a piece of software, for example as a build verification test.

51
7.5.4 Regression Testing

Regression testing focuses on finding defects after a major code change

has occurred. Specifically, it seeks to uncover software regressions, or old bugs
that have come back. Such regressions occur whenever software functionality that
was previously working correctly stops working as intended. Typically,
regressions occur as an unintended consequence of program changes, when the
newly developed part of the software collides with the previously existing code.
Common methods of regression testing include re-running previously run tests
and checking whether previously fixed faults have re-emerged. The depth of
testing depends on the phase in the release process and the risk of the added
features. They can either be complete, for changes added late in the release or
deemed to be risky, to 36 very shallow, consisting of positive tests on each feature,
if the changes are early in the release or deemed to be of low risk.

7.5.5 Alpha Testing

Alpha testing is simulated or actual operational testing by potential

users/customers or an independent test team at the developers' site. Alpha testing
is often employed for off-the-shelf software as a form of internal acceptance
testing, before the software goes to beta testing.

7.5.6 Beta Testing

Beta testing comes after alpha testing and can be considered a form of
external user acceptance testing. Versions of the software, known as beta versions,
are released to a limited audience outside of the programming team. The software
is released to groups of people so that further testing can ensure the product has
few faults or bugs. Sometimes, beta versions are made available to the open
public to increase the feedback field to a maximal number of future users.

7.5.7 Functional vs Non-functional Testing

52
 Functional testing refers to activities that verify a specific action or
function of the code. These are usually found in the code requirements
documentation, although some development methodologies work from use cases
or user stories. Functional tests tend to answer the question of "can the user do
this" or "does this particular feature work." Non-functional testing refers to
aspects of the software that may not be related to a specific function or user action,
such as scalability or other performance, behaviour under certain constraints, or
security.

7.5.8 Destructive Testing

Destructive testing attempts to cause the software or a sub-system to fail.

It verifies that the software functions properly even when it receives invalid or
unexpected inputs, thereby establishing the robustness of input validation and
error-management routines. Software fault injection, in the form of fuzzing, is an
example of failure testing. Various commercial non-functional testing tools are
linked from the software fault injection page; there are also numerous open-
source and free software tools available that perform destructive testing.

7.5.9 Software Performance Testing

Performance testing is in general executed to determine how a system or

sub-system performs in terms of responsiveness and stability under a particular
workload. It can also serve to investigate, measure, validate or verify other quality
attributes of the system, such as scalability, reliability and resource usage. Load
testing is primarily concerned with testing that the system can continue to operate
under a specific load, whether that be large quantities of data or a large number
of users. This is generally referred to as software scalability. The related load
testing activity of when performed as a non-functional activity is often referred
to as endurance testing. Volume testing is a way to test software functions even

53
when certain components (for example a file or database) increase radically in
size. Stress testing is a way to test reliability under unexpected or rare workloads.

7.5.10 Usability Testing

Usability testing is needed to check if the user interface is easy to use and
understand. It is concerned mainly with the use of the application. It is important
to check interface is working properly or not as planned

7.5.11 Accessibility Testing

Accessibility testing may include compliance with standards such as:

1. Americans with Disabilities Act of 1990

2. Section 508 Amendment to the Rehabilitation Act of 1973 3. Web Accessibility

Initiative (WAI) of the World Wide Web Consortium

7.5.12 Security Testing

Security testing is essential for software that processes confidential data to

prevent system intrusion by hackers. So, it is important to perform security testing
to find loop holes in the developed software.

7.5.13 Development Testing

Development Testing is a software development process that involves

synchronized application of a broad spectrum of defect prevention and detection
strategies in order to reduce software development risks, time, and costs. It is
performed by the software developer or engineer during the construction phase
of the software development lifecycle. Rather than replace traditional QA
focuses, it augments it.

54
 CHAPTER 8

8. CONCLUSION AND FUTURE ENHANCEMENT

8.1 CONCLUSION

In this project, we have studied the fake news article, creator and subject
detection problem. Based on the news augmented heterogeneous social network,
a set of explicit and latent features can be extracted from the textual information
of news articles, creators and subjects respectively. Furthermore, based on the
connections among news articles, creators and news subjects, a deep diffusive
network model has been proposed for incorporate the network structure
information into model learning. The accuracy metric presumably would be
altogether improved by methods for utilizing progressively complex model. It is
worth noting, that even with the given dataset, only part of the information was
used. The current project did not include domain knowledge related features, such
as entity-relationships. The proposed system proves that multi-layer perceptron
neural network algorithm provides improved accuracy rate. We formulated the
fake news detection on social media as an inference problem in deep learning
model that can be solved using multi-layer neural network algorithm. We can
conclude that, the proposed system to provide improved accuracy rate in fake
news detection. Experiments on well-known benchmark datasets show that the
proposed model consistently improves over the state of the art in fake news
detection in both the late and early detection settings

8.2 FUTURE ENHANCEMENT

In future, we can extend the framework to implement various deep learning

algorithms to improve the accuracy and reduce the complexity in classification.

55
APPENDIX

A1. CODING

import numpy as np

import pandas as pd

import seaborn as sns

from keras.preprocessing.text import Tokenizer

from keras.preprocessing.sequence import pad_sequences

from sklearn.model_selection import train_test_split

from sklearn.preprocessing import LabelEncoder

import pickle

from sklearn.neural_network import MLPClassifier

from sklearn.metrics import classification_report

from sklearn.metrics import confusion_matrix

import matplotlib.pyplot as plt

from sklearn.preprocessing import StandardScaler

# load the data

data = pd.read_csv("DataSet/DataSet.csv")

print(data.shape)

data = data.dropna()

print(data.shape)

56
# preprocessing

def preprocess(data):

# label encode target variable

le = LabelEncoder()

data['label'] = le.fit_transform(data['label'])

# tokenize the description

tokenizer = Tokenizer()

tokenizer.fit_on_texts(data['text'])

sequences = tokenizer.texts_to_sequences(data['text'])

X = pad_sequences(sequences, maxlen=500)

# get target variable

y = data['label']

# split data into training and testing sets

X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.33,

random_state=0)

return X_train, X_test, y_train, y_test

X_train, X_test, y_train, y_test = preprocess(data)

print(X_train)

print(X_test)

print(y_train)

print(y_test)

# fit the model

57
classifier = MLPClassifier(random_state=0,max_iter=200)

classifier.fit(X_train, y_train)

y_pred = classifier.predict(X_test)

print(classification_report(y_test, y_pred))

clreport = classification_report(y_test, y_pred)

y_pred = classifier.predict(X_test)

print("Accuracy on training set: {:.2f}".format(classifier.score(X_train,

y_train)))

print("Accuracy on test set: {:.3f}".format(classifier.score(X_test, y_test)))

Tacc = "Accuracy on training set: {:.2f}".format(classifier.score(X_train,

y_train))

Testacc = "Accuracy on test set: {:.3f}".format(classifier.score(X_test, y_test))

# Compute the confusion matrix

cm = confusion_matrix(y_test, y_pred)

# Plot the confusion matrix as a heatmap

sns.heatmap(cm, annot=True, fmt='d', cmap='Blues')

plt.xlabel('Predicted Label')

plt.ylabel('True Label')

plt.title('Confusion Matrix of MLP Classifier')

plt.show()

# Creating a pickle file for the classifier

filename = 'model.pkl'

58
pickle.dump(classifier, open(filename, 'wb'))

'''tokenizer = Tokenizer()

tokenizer.fit_on_texts(data['description'])

sequences = tokenizer.texts_to_sequences(data['description'])

X = pad_sequences(sequences, maxlen=500)'''

from flask import Flask,render_template,url_for,request

import pandas as pd

import pickle

from sklearn.feature_extraction.text import CountVectorizer

from sklearn.naive_bayes import MultinomialNB

#from sklearn.externals import joblib

from sklearn import *

import pandas as pd

import seaborn as sns

from keras.preprocessing.text import Tokenizer

from keras.preprocessing.sequence import pad_sequences

from sklearn.model_selection import train_test_split

from sklearn.preprocessing import LabelEncoder

import pickle

from sklearn.neural_network import MLPClassifier

from sklearn.metrics import classification_report

from sklearn.metrics import confusion_matrix

59
import matplotlib.pyplot as plt

app = Flask(__name__)

@app.route('/')

def home():

return render_template('home.html')

@app.route('/predict',methods=['POST'])

def predict():

if request.method == 'POST':

comment = request.form['comment']

tokenizer = Tokenizer()

tokenizer.fit_on_texts(comment)

sequences = tokenizer.texts_to_sequences(comment)

X = pad_sequences(sequences, maxlen=500)

filename = 'model.pkl'

classifier = pickle.load(open(filename, 'rb'))

my_prediction = classifier.predict(X)

#warnings.filterwarnings("ignore", category=DeprecationWarning)

print(my_prediction[0])

if my_prediction[0] == 0:

res = 'Real '

else:

res = 'Fake '

60
 return render_template('result.html', prediction=my_prediction[0])

if __name__ == '__main__':

app.run(debug=True)

61
A2 SCREENSHOTS

Fig A2.1 IDE

L:\Python2023\UG\Cavery\NLTKDatasetPy\venv\Scripts\python.exe
L:/Python2023/UG/Cavery/NLTKDatasetPy/NewModel.py

2023-03-28 12:00:01.455064: W
tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load
dynamic library 'cudart64_110.dll'; dlerror: cudart64_110.dll not found

2023-03-28 12:00:01.456089: I
tensorflow/stream_executor/cuda/cudart_stub.cc:29] Ignore above cudart
dlerror if you do not have a GPU set up on your machine.

(999, 5)

(868, 5)

[[ 16 419 369 ... 223 2622 506]

[ 0 0 0 ... 11847 433 282]

62
[ 0 0 0 ... 18 2899 20541]

...

[ 0 0 0 ... 18 1 4776]

[ 0 0 0 ... 11 171 12209]

[ 1780 10 28 ... 71 8576 12]]

[[ 7905 98 109 ... 100 1753 10]

[ 89 601 6 ... 11 171 4379]

[ 0 0 0 ... 171 20 7198]

...

[ 0 0 0 ... 42 9552 2365]

[ 9 31 1083 ... 1824 3 6496]

[10182 34 26 ... 6 18110 18111]]

358 1

584 0

596 1

390 0

24 1

..

959 0

225 0

726 0

647 0

63
790 0

Name: label, Length: 581, dtype: int64

302 0

941 0

87 0

526 0

663 0

..

752 0

495 0

88 1

909 0

231 0

Name: label, Length: 287, dtype: int64

precision recall f1-score support

0 0.61 0.59 0.60 160

1 0.51 0.54 0.52 127

accuracy 0.56 287

macro avg 0.56 0.56 0.56 287

weighted avg 0.57 0.56 0.57 287

64
Accuracy on training set: 0.99

Accuracy on test set: 0.564

Fig A2.2 MATRIX

65
Fig A2.3 WEB APPLICATION LINK

Fig A2.4 WEB APPLICATION

66
Fig A2.5 DETECTING FAKE NEWS

Fig A2.6 PREDICTON OUTPUT

67
REFERENCES

1. G. Bhatt, A. Sharma, S. Sharma, A. Nagpal, B. Raman, and A. Mittal,

‘‘Combining neural, statistical and external features for fake news stance
identification,’’ in Proc. Companion Web Conf. Web Conf. (WWW), Geneva,
Switzerland, 2018

2. Jaiswal, Jitendra Kumar, and Rita Samikannu. "Application of random forest

algorithm on feature subset selection and classification and regression." 2017
world congress on computing and communication technologies (WCCCT). IEEE,
2017.

3. Kataria, Aman, and M. D. Singh. "A review of data classification using k-

nearest neighbour algorithm." International Journal of Emerging Technology and
Advanced Engineering 3.6 (2013): 354-360.

4. L. Konstantinovskiy, ‘‘Towards automated factchecking: Developing an

annotation schema and benchmark for consistent automated claim detection,’’
2018, arXiv:1809.08193.

5. Lu, Jing, Peilin Zhao, and Steven Hoi. "Online passive aggressive active
learning and its applications." Asian Conference on Machine Learning. PMLR,
2015.

6. Nayak, Janmenjoy, Bighnaraj Naik, and H. S. Behera. "A comprehensive

survey on support vector machine in data mining tasks: applications &
challenges." International Journal of Database Theory and Application 8.1
(2015): 169-186.

7. P. Bourgonje, J. Moreno Schneider, and G. Rehm, ‘‘From clickbait to fake news

detection: An approach based on detecting the stance of headlines to articles,’’ in
Proc. EMNLP Workshop: Natural Lang. Process. meets Journalism, 2017, pp.
84–89.

68
8. T. Mihaylov, ‘‘Finding opinion manipulation trolls in news community
forums,’’ in Proc. 19th Conf. Comput. Natural Lang. Learn., Beijing, China, Jul.
2015, pp. 310–314. [Online].

9. T. Mihaylov and P. Nakov, ‘‘Hunting for troll comments in news community

forums,’’ in Proc. 54th Annu. Meeting Assoc. for Comput. Linguistics, vol. 2,
2016, pp. 399–405

10. Ul Haq, Muhammad Aziz, et al. "Indoor localization using improved

multinomial naïve bayes technique." International afro-European conference for
industrial advancement. Springer, Cham, 2016.

69