MODULES

1. DATA COLLECTION
2. DATA PRE-PROCESSING
3. FEATURE EXTRATION
4. EVALUATION MODEL

DATA COLLECTION
Data used in this paper is a software data of JM1. This step is concerned with
selecting the subset of all available data that you will be working with. ML
problems start with data preferably, lots of data (examples or observations) for
which you already know the target answer. Data for which you already know
the target answer is called labelled data.

DATA PRE-PROCESSING
Organize your selected data by formatting, cleaning and sampling from it.
Three common data pre-processing steps are:

 Formatting: The data you have selected may not be in a format that is suitable for
you to work with. The data may be in a relational database and you would like it in
a flat file, or the data may be in a proprietary file format and you would like it in a
relational database or a text file.
 Cleaning: Cleaning data is the removal or fixing of missing data. There may be
data instances that are incomplete and do not carry the data you believe you need
to address the problem. These instances may need to be removed. Additionally,
there may be sensitive information in some of the attributes and these attributes
may need to be anonymized or removed from the data entirely.
 Sampling: There may be far more selected data available than you need to work
with. More data can result in much longer running times for algorithms and larger
computational and memory requirements. You can take a smaller representative
sample of the selected data that may be much faster for exploring and prototyping
solutions before considering the whole dataset.

FEATURE EXTRATION
 Next thing is to do Feature extraction is an attribute reduction process.
Unlike feature selection, which ranks the existing attributes according to their
predictive significance, feature extraction actually transforms the attributes. The
transformed attributes, or features, are linear combinations of the original
attributes. Finally, our models are trained using Classifier algorithm. We use
classify module on Natural Language Toolkit library on Python. We use the
labelled dataset gathered. The rest of our labelled data will be used to evaluate the
models. Some machine learning algorithms were used to classify pre-processed
data. The chosen classifiers were Random forest. These algorithms are very
popular in text classification tasks.

EVALUATION MODEL

Model Evaluation is an integral part of the model development process. It helps to

find the best model that represents our data and how well the chosen model will
work in the future. Evaluating model performance with the data used for training is
not acceptable in data science because it can easily generate overoptimistic and
over fitted models.
Performance of each classification model is estimated base on its averaged. The
result will be in the visualized form. Representation of classified data in the form
of graphs.
Accuracy is defined as the percentage of correct predictions for the test data. It can
be calculated easily by dividing the number of correct predictions by the number of
total predictions.

Proposed Approach Steps

1. First, we take input dataset of resumes.
2. Filter dataset according to requirements and create a new dataset which
has attribute according to analysis to be done
3. Perform Pre-Processing on the dataset
4. Split the data into training and testing
5. Train the model with training data then analyze testing dataset over
classification algorithm
6. Finally you will get results as accuracy metrics.
Data flow diagram:-
Dataflow diagram 2
Uml diagrans:-
Use case diagram:-

Class diagram:-
Activity diagram:-

Domain Specification

MACHINE LEARNING
Machine Learning is a system that can learn from example through self-
improvement and without being explicitly coded by programmer. The
breakthrough comes with the idea that a machine can singularly learn from the data
(i.e., example) to produce accurate results.

Machine learning combines data with statistical tools to predict an output. This
output is then used by corporate to makes actionable insights. Machine learning is
closely related to data mining and Bayesian predictive modeling. The machine
receives data as input, use an algorithm to formulate answers.

A typical machine learning tasks are to provide a recommendation. For those who
have a Netflix account, all recommendations of movies or series are based on the
user's historical data. Tech companies are using unsupervised learning to improve
the user experience with personalizing recommendation.

Machine learning is also used for a variety of task like fraud detection, predictive
maintenance, portfolio optimization, automatize task and so on.

Machine Learning vs. Traditional Programming

Traditional programming differs significantly from machine learning. In traditional

programming, a programmer code all the rules in consultation with an expert in the
industry for which software is being developed. Each rule is based on a logical
foundation; the machine will execute an output following the logical statement.
When the system grows complex, more rules need to be written. It can quickly
become unsustainable to maintain.

DATA RULES
COMPUTER
OUTPUT

Machine Learning
How does Machine learning work?
Machine learning is the brain where all the learning takes place. The way the
machine learns is similar to the human being. Humans learn from experience. The
more we know, the more easily we can predict. By analogy, when we face an
unknown situation, the likelihood of success is lower than the known situation.
Machines are trained the same. To make an accurate prediction, the machine sees
an example. When we give the machine a similar example, it can figure out the
outcome. However, like a human, if its feed a previously unseen example, the
machine has difficulties to predict.
The core objective of machine learning is the learning and inference. First of all,
the machine learns through the discovery of patterns. This discovery is made
thanks to the data. One crucial part of the data scientist is to choose carefully
which data to provide to the machine. The list of attributes used to solve a problem
is called a feature vector. You can think of a feature vector as a subset of data that
is used to tackle a problem.
The machine uses some fancy algorithms to simplify the reality and transform this
discovery into a model. Therefore, the learning stage is used to describe the data
and summarize it into a model.

For instance, the machine is trying to understand the relationship between the wage
of an individual and the likelihood to go to a fancy restaurant. It turns out the
machine finds a positive relationship between wage and going to a high-end
restaurant: This is the model
Inferring
When the model is built, it is possible to test how powerful it is on never-seen-
before data. The new data are transformed into a features vector, go through the
model and give a prediction. This is all the beautiful part of machine learning.
There is no need to update the rules or train again the model. You can use the
model previously trained to make inference on new data.
The life of Machine Learning programs is straightforward and can be summarized
in the following points:

1. Define a question
2. Collect data
3. Visualize data
4. Train algorithm
5. Test the Algorithm
6. Collect feedback
7. Refine the algorithm
8. Loop 4-7 until the results are satisfying
9. Use the model to make a prediction

Once the algorithm gets good at drawing the right conclusions, it applies that
knowledge to new sets of data.
Machine learning Algorithms and where they are used?

Machine learning can be grouped into two broad learning tasks: Supervised and
Unsupervised. There are many other algorithms
Supervised learning
An algorithm uses training data and feedback from humans to learn the
relationship of given inputs to a given output. For instance, a practitioner can use
marketing expense and weather forecast as input data to predict the sales of cans.
You can use supervised learning when the output data is known. The algorithm
will predict new data.
There are two categories of supervised learning:

Algorithm Description Type

Name

Linear Finds a way to correlate each feature to the output to help Regression
regression predict future values.

Logistic Extension of linear regression that's used for classification tasks. Classification
regression The output variable 3is binary (e.g., only black or white) rather
 than continuous (e.g., an infinite list of potential colors)

Decision Highly interpretable classification or regression model that Regression

tree splits data-feature values into branches at decision nodes (e.g., if Classification
a feature is a color, each possible color becomes a new branch)
until a final decision output is made

Naive The Bayesian method is a classification method that makes use Regression
Bayes of the Bayesian theorem. The theorem updates the prior Classification
knowledge of an event with the independent probability of each
feature that can affect the event.

Support Support Vector Machine, or SVM, is typically used for the Regression (no
vector classification task. SVM algorithm finds a hyperplane that very common
machine optimally divided the classes. It is best used with a non-linear Classification
solver.

Random The algorithm is built upon a decision tree to improve the Regression
forest accuracy drastically. Random forest generates many times Classification
simple decision trees and uses the 'majority vote' method to
decide on which label to return. For the classification task, the
final prediction will be the one with the most vote; while for the
regression task, the average prediction of all the trees is the final
prediction.

AdaBoost Classification or regression technique that uses a multitude of Regression

models to come up with a decision but weighs them based on Classification
their accuracy in predicting the outcome

Gradient- Gradient-boosting trees is a state-of-the-art Regression

boosting classification/regression technique. It is focusing on the error Classification
trees committed by the previous trees and tries to correct it.
 ● Classification task
● Regression task

Classification
Imagine you want to predict the gender of a customer for a commercial. You will
start gathering data on the height, weight, job, salary, purchasing basket, etc. from
your customer database. You know the gender of each of your customer, it can
only be male or female. The objective of the classifier will be to assign a
probability of being a male or a female (i.e., the label) based on the information
(i.e., features you have collected). When the model learned how to recognize male
or female, you can use new data to make a prediction. For instance, you just got
new information from an unknown customer, and you want to know if it is a male
or female. If the classifier predicts male = 70%, it means the algorithm is sure at
70% that this customer is a male, and 30% it is a female.
The label can be of two or more classes. The above example has only two classes,
but if a classifier needs to predict object, it has dozens of classes (e.g., glass, table,
shoes, etc. each object represents a class)
Regression
When the output is a continuous value, the task is a regression. For instance, a
financial analyst may need to forecast the value of a stock based on a range of
feature like equity, previous stock performances, macroeconomics index. The
system will be trained to estimate the price of the stocks with the lowest possible
error.

Algorithm Description Type

K-means Puts data into some groups (k) that each contains data with Clustering
clustering similar characteristics (as determined by the model, not in
advance by humans)

Gaussian A generalization of k-means clustering that provides more Clustering

mixture model flexibility in the size and shape of groups (clusters

Hierarchical Splits clusters along a hierarchical tree to form a Clustering

clustering classification system.
 Can be used for Cluster loyalty-card customer

Recommender Help to define the relevant data for making a Clustering

system recommendation.

PCA/T-SNE Mostly used to decrease the dimensionality of the data. The Dimension
algorithms reduce the number of features to 3 or 4 vectors Reduction
with the highest variances.

Unsupervised learning
In unsupervised learning, an algorithm explores input data without being given an
explicit output variable (e.g., explores customer demographic data to identify
patterns)
You can use it when you do not know how to classify the data, and you want the
algorithm to find patterns and classify the data for you

Application of Machine learning

Augmentation:

● Machine learning, which assists humans with their day-to-day tasks,

personally or commercially without having complete control of the output.
Such machine learning is used in different ways such as Virtual Assistant,
Data analysis, software solutions. The primary user is to reduce errors due to
human bias.

Automation:

● Machine learning, which works entirely autonomously in any field without

the need for any human intervention. For example, robots performing the
essential process steps in manufacturing plants.

Finance Industry

● Machine learning is growing in popularity in the finance industry. Banks are

mainly using ML to find patterns inside the data but also to prevent fraud.
Government organization

● The government makes use of ML to manage public safety and utilities.

Take the example of China with the massive face recognition. The
government uses Artificial intelligence to prevent jaywalker.

Healthcare industry

● Healthcare was one of the first industry to use machine learning with image
detection.

Marketing

● Broad use of AI is done in marketing thanks to abundant access to data.

Before the age of mass data, researchers develop advanced mathematical
tools like Bayesian analysis to estimate the value of a customer. With the
boom of data, marketing department relies on AI to optimize the customer
relationship and marketing campaign.

Example of application of Machine Learning in Supply Chain

Machine learning gives terrific results for visual pattern recognition, opening up
many potential applications in physical inspection and maintenance across the
entire supply chain network.

Unsupervised learning can quickly search for comparable patterns in the diverse
dataset. In turn, the machine can perform quality inspection throughout the
logistics hub, shipment with damage and wear.

For instance, IBM's Watson platform can determine shipping container damage.
Watson combines visual and systems-based data to track, report and make
recommendations in real-time.

In past year stock manager relies extensively on the primary method to evaluate
and forecast the inventory. When combining big data and machine learning, better
forecasting techniques have been implemented (an improvement of 20 to 30 %
over traditional forecasting tools). In term of sales, it means an increase of 2 to 3 %
due to the potential reduction in inventory costs.

Example of Machine Learning Google Car

For example, everybody knows the Google car. The car is full of lasers on the roof
which are telling it where it is regarding the surrounding area. It has radar in the
front, which is informing the car of the speed and motion of all the cars around it.
It uses all of that data to figure out not only how to drive the car but also to figure
out and predict what potential drivers around the car are going to do. What's
impressive is that the car is processing almost a gigabyte a second of data.

Deep Learning

Deep learning is a computer software that mimics the network of neurons in a

brain. It is a subset of machine learning and is called deep learning because it
makes use of deep neural networks. The machine uses different layers to learn
from the data. The depth of the model is represented by the number of layers in the
model. Deep learning is the new state of the art in term of AI. In deep learning, the
learning phase is done through a neural network.

Reinforcement Learning

Reinforcement learning is a subfield of machine learning in which systems are

trained by receiving virtual "rewards" or "punishments," essentially learning by
trial and error. Google's DeepMind has used reinforcement learning to beat a
human champion in the Go games. Reinforcement learning is also used in video
games to improve the gaming experience by providing smarter bot.

One of the most famous algorithms are:

● Q-learning
● Deep Q network
● State-Action-Reward-State-Action (SARSA)
● Deep Deterministic Policy Gradient (DDPG)

Applications/ Examples of deep learning applications

AI in Finance: The financial technology sector has already started using AI to

save time, reduce costs, and add value. Deep learning is changing the lending
industry by using more robust credit scoring. Credit decision-makers can use AI
for robust credit lending applications to achieve faster, more accurate risk
assessment, using machine intelligence to factor in the character and capacity of
applicants.
Underwrite is a Fintech company providing an AI solution for credit makers
company. underwrite.ai uses AI to detect which applicant is more likely to pay
back a loan. Their approach radically outperforms traditional methods.

AI in HR: Under Armour, a sportswear company revolutionizes hiring and

modernizes the candidate experience with the help of AI. In fact, Under Armour
Reduces hiring time for its retail stores by 35%. Under Armour faced a growing
popularity interest back in 2012. They had, on average, 30000 resumes a month.
Reading all of those applications and begin to start the screening and interview
process was taking too long. The lengthy process to get people hired and on-
boarded impacted Under Armour's ability to have their retail stores fully staffed,
ramped and ready to operate.

At that time, Under Armour had all of the 'must have' HR technology in place such
as transactional solutions for sourcing, applying, tracking and onboarding but those
tools weren't useful enough. Under armour choose HireVue, an AI provider for
HR solution, for both on-demand and live interviews. The results were bluffing;
they managed to decrease by 35% the time to fill. In return, the hired higher quality
staffs.

AI in Marketing: AI is a valuable tool for customer service managementand

personalization challenges. Improved speech recognition in call-center
management and call routing as a result of the application of AI techniques allows
a more seamless experience for customers.

For example, deep-learning analysis of audio allows systems to assess a customer's

emotional tone. If the customer is responding poorly to the AI chatbot, the system
can be rerouted the conversation to real, human operators that take over the issue.

Apart from the three examples above, AI is widely used in other sectors/industries.

Artificial Intelligence
 ML
Machine Learning DL
Deep Learning

Difference between Machine Learning and Deep Learning

Machine Learning Deep Learning

Data Excellent performances on a Excellent performance on a big

Dependencie small/medium dataset dataset
s

Hardware Work on a low-end machine. Requires powerful machine,

dependencies preferably with GPU: DL
performs a significant amount
of matrix multiplication

Feature Need to understand the features No need to understand the best

engineering that represent the data feature that represents the data

Execution From few minutes to hours Up to weeks. Neural Network

time needs to compute a significant
number of weights

Interpretabil Some algorithms are easy to Difficult to impossible

 ity interpret (logistic, decision
tree), some are almost
impossible (SVM, XGBoost)

When to use ML or DL?

In the table below, we summarize the difference between machine learning and
deep learning.

Machine learning Deep learning

Training dataset Small Large

Choose features Yes No

Number of algorithms Many Few

Training time Short Long

With machine learning, you need fewer data to train the algorithm than deep
learning. Deep learning requires an extensive and diverse set of data to identify the
underlying structure. Besides, machine learning provides a faster-trained model.
Most advanced deep learning architecture can take days to a week to train. The
advantage of deep learning over machine learning is it is highly accurate. You do
not need to understand what features are the best representation of the data; the
neural network learned how to select critical features. In machine learning, you
need to choose for yourself what features to include in the model.
TensorFlow

the most famous deep learning library in the world is Google's TensorFlow.
Google product uses machine learning in all of its products to improve the search
engine, translation, image captioning or recommendations.

To give a concrete example, Google users can experience a faster and more refined
the search with AI. If the user types a keyword a the search bar, Google provides a
recommendation about what could be the next word.

Google wants to use machine learning to take advantage of their massive datasets
to give users the best experience. Three different groups use machine learning:

● Researchers
● Data scientists
● Programmers.

They can all use the same toolset to collaborate with each other and improve their
efficiency.

Google does not just have any data; they have the world's most massive computer,
so TensorFlow was built to scale. TensorFlow is a library developed by the Google
Brain Team to accelerate machine learning and deep neural network research.
It was built to run on multiple CPUs or GPUs and even mobile operating systems,
and it has several wrappers in several languages like Python, C++ or Java.

In this tutorial, you will learn

TensorFlow Architecture

Tensor flow architecture works in three parts:

● Pre processing the data

● Build the model
● Train and estimate the model

It is called Tensor flow because it takes input as a multi-dimensional array, also

known as tensors. You can construct a sort of flowchart of operations (called a
Graph) that you want to perform on that input. The input goes in at one end, and
then it flows through this system of multiple operations and comes out the other
end as output.

This is why it is called TensorFlow because the tensor goes in it flows through a
list of operations, and then it comes out the other side.

Where can Tensor flow run?

TensorFlow can hardware, and software requirements can be classified into

Development Phase: This is when you train the mode. Training is usually done on
your Desktop or laptop.

Run Phase or Inference Phase: Once training is done Tensorflow can be run on
many different platforms. You can run it on

● Desktop running Windows, macOS or Linux

● Cloud as a web service
● Mobile devices like iOS and Android

You can train it on multiple machines then you can run it on a different machine,
once you have the trained model.

The model can be trained and used on GPUs as well as CPUs. GPUs were initially
designed for video games. In late 2010, Stanford researchers found that GPU was
also very good at matrix operations and algebra so that it makes them very fast for
doing these kinds of calculations. Deep learning relies on a lot of matrix
multiplication. TensorFlow is very fast at computing the matrix multiplication
because it is written in C++. Although it is implemented in C++, TensorFlow can
be accessed and controlled by other languages mainly, Python.

Finally, a significant feature of Tensor Flow is the Tensor Board. The Tensor
Board enables to monitor graphically and visually what TensorFlow is doing.

List of Prominent Algorithms supported by TensorFlow

● Linear regression: tf. estimator .Linear Regressor

● Classification :tf. Estimator .Linear Classifier
● Deep learning classification: tf. estimator. DNN Classifier
● Booster tree regression: tf.estimator.BoostedTreesRegressor
● Boosted tree classification: tf.estimator.BoostedTreesClassifier

PYTHON OVERVIEW

Python is a high-level, interpreted, interactive and object-oriented scripting

language. Python is designed to be highly readable. It uses English keywords
frequently where as other languages use punctuation, and it has fewer syntactical
constructions than other languages.

 Python is Interpreted: Python is processed at runtime by the interpreter.

You do not need to compile your program before executing it. This is similar
to PERL and PHP.

 Python is Interactive: You can actually sit at a Python prompt and interact
with the interpreter directly to write your programs.
  Python is Object-Oriented: Python supports Object-Oriented style or
technique of programming that encapsulates code within objects.

 Python is a Beginner's Language: Python is a great language for the

beginner-level programmers and supports the development of a wide range
of applications from simple text processing to WWW browsers to games.

History of Python

Python was developed by Guido van Rossum in the late eighties and early nineties
at the National Research Institute for Mathematics and Computer Science in the
Netherlands.

Python is derived from many other languages, including ABC, Modula-3, C, C++,
Algol-68, SmallTalk, Unix shell, and other scripting languages.

Python is copyrighted. Like Perl, Python source code is now available under the
GNU General Public License (GPL).

Python is now maintained by a core development team at the institute, although

Guido van Rossum still holds a vital role in directing its progress.

Python Features

Python's features include:

 Easy-to-learn: Python has few keywords, simple structure, and a clearly
defined syntax. This allows the student to pick up the language quickly.

 Easy-to-read: Python code is more clearly defined and visible to the eyes.

 Easy-to-maintain: Python's source code is fairly easy-to-maintain.

 A broad standard library: Python's bulk of the library is very portable and
cross-platform compatible on UNIX, Windows, and Macintosh.

 Interactive Mode: Python has support for an interactive mode which allows
interactive testing and debugging of snippets of code.

 Portable: Python can run on a wide variety of hardware platforms and has
the same interface on all platforms.

 Extendable: You can add low-level modules to the Python interpreter.

These modules enable programmers to add to or customize their tools to be
more efficient.

 Databases: Python provides interfaces to all major commercial databases.

 GUI Programming: Python supports GUI applications that can be created

and ported to many system calls, libraries, and windows systems, such as
Windows MFC, Macintosh, and the X Window system of Unix.
  Scalable: Python provides a better structure and support for large programs
than shell scripting.

Apart from the above-mentioned features, Python has a big list of good features,
few are listed below:

 IT supports functional and structured programming methods as well as OOP.

 It can be used as a scripting language or can be compiled to byte-code for

building large applications.

 It provides very high-level dynamic data types and supports dynamic type
checking.

 IT supports automatic garbage collection.

 It can be easily integrated with C, C++, COM, ActiveX, CORBA, and Java.

Python is available on a wide variety of platforms including Linux and Mac OS X.

Let's understand how to set up our Python environment.

ANACONDA NAVIGATOR

Anaconda Navigator is a desktop graphical user interface (GUI) included in

Anaconda distribution that allows you to launch applications and easily manage
conda packages, environments and channels without using command-line
 commands. Navigator can search for packages on Anaconda Cloud or in a local
Anaconda Repository. It is available for Windows, mac OS and Linux.

Why use Navigator?

In order to run, many scientific packages depend on specific versions of other

packages. Data scientists often use multiple versions of many packages, and use
multiple environments to separate these different versions.

The command line program conda is both a package manager and an environment
manager, to help data scientists ensure that each version of each package has all the
dependencies it requires and works correctly.

Navigator is an easy, point-and-click way to work with packages and environments

without needing to type conda commands in a terminal window. You can use it to
find the packages you want, install them in an environment, run the packages and
update them, all inside Navigator.
WHAT APPLICATIONS CAN I ACCESS USING NAVIGATOR?

The following applications are available by default in Navigator:

● Jupyter Lab

● Jupyter Notebook

● QT Console

● Spyder

● VS Code

● Glue viz

● Orange 3 App

● Rodeo
● RStudio

Advanced conda users can also build your own Navigator applications
How can I run code with Navigator?

The simplest way is with Spyder. From the Navigator Home tab, click Spyder, and
write and execute your code.

You can also use Jupyter Notebooks the same way. Jupyter Notebooks are an
increasingly popular system that combine your code, descriptive text, output,
images and interactive interfaces into a single notebook file that is edited, viewed
and used in a web browser.
What’s new in 1.9?

● Add support for Offline Mode for all environment related actions.

● Add support for custom configuration of main windows links.

● Numerous bug fixes and performance enhancements.

TESTING

Software testing is an investigation conducted to provide stakeholders with

information about the quality of the product or service under test. Software Testing
also provides an objective, independent view of the software to allow the business
to appreciate and understand the risks at implementation of the software. Test
techniques include, but are not limited to, the process of executing a program or
application with the intent of finding software bugs.
Software Testing can also be stated as the process of validating and verifying that a
software program/application/product:
● Meets the business and technical requirements that guided its design and
Development.
● Works as expected and can be implemented with the same characteristics.
TESTING METHODS

● Functional Testing

Functional tests provide systematic demonstrations that functions tested are

available as specified by the business and technical requirements, system
documentation, and user manuals.
Functional testing is centered on the following items:
● Functions: Identified functions must be exercised.
● Output: Identified classes of software outputs must be exercised.
● Systems/Procedures: system should work properly

Integration Testing

Software integration testing is the incremental integration testing of two or more

integrated software components on a single platform to produce failures caused by
interface defects.

Test Case for Excel Sheet Verification:

Here in machine learning we are dealing with dataset which is in excel sheet
format so if any test case we need means we need to check excel file. Later on
classification will work on the respective columns of dataset .

Test Case 1 :