MODULE-1

Table of contents:

1. Introduction to social media

 Understanding what is social media

 Traditional media vs social media

 Uses of social media

 Why social media

 Goals and objectives

2. Introduction to social media analytics (SMA)

 Understanding what is SMA

 Why SMA

 Data, information and knowledge

 Opportunities in SMA

 Understanding API

 Challenges in SMA

3. SMA techniques

 Sentiment analysis

 Machine learning
INTRODUCTION TO SOCIAL MEDIA

 Social media platforms are driven by user-generated content. Every user is

a potential publisher.

 Social media is about networking.

 Social media is about the users interacting with other users.

 Being connected is the central concept for most social media platform, and
the content you consume via your news feed or timeline is driven by your
connections.

 In traditional media, users are typically just consumers.

 Information flows in one direction from publisher to the user.

 Social media breaks this model, allowing every user to be a consumer and
publisher at the same time.
  Social media are internet-based applications.

 It is clear that the advances in Internet and mobile technologies have

promoted the social media.
 Through your mobile immediately connect to a social media platform,
publish your content or catch up with the latest news.
 Social media platforms are driven by user-generated content.

 Any user can interact with every other user by sharing content,
commenting or expressing positive appraisal via like button.
 Social media is about the networking.

Uses of social media:

 Staying in touch with friends and family (via Facebook)

 Microblogging and catching up with the latest news (via Twitter)

 Staying in touch with your professional network (via LinkedIn)

 Sharing multimedia content (via Instagram, YouTube, Vimeo, and Flickr)

 Finding answers to your questions (via Stack Overflow, Stack Exchange,

and Quora)
 Finding and organizing items of interest (via Pinterest)
 Set Goals and Objectives:

EG: drive sales by attracting customers, increase brand awareness

on-demand media company Netflix uses the Twitter handle @Netflixhelps to

1.Understand Your Goals:

• Some businesses use social media for increasing their brand awareness

• Others use it for driving website traffic and sales.

• Social media can also help in

• generating engagement around your brand,

• create a community,
 • serve as a customer support channel for your customers.

2. Listen to what is already happening:

Understand what people are saying online to make better-informed

decisions

3. Do Competitor and Consumer research:

Know target audience’s needs and preferences. Conduct consumer research
to better understand what customers want and uncover new business
opportunities.

4. Decide which social media is for you:

 Each social media platform has its own personality and ways of
communicating.
 Twitter is great for conversations and trending topics
 LinkedIn is good for sharing professional expertise and
 company information.
 Snapchat was the first to introduce disappearing posts
 Instagram Stories are quick, easy way to share moments and
experiences

5. Personalize Your Social Media Brand:

A strategy where you leverage data and data analysis to deliver personalized
and individualized content and marketing experiences to your target
audience, prospects, and customers.
6. Share interesting info that will Enlighten/engage your
audience:

• Don’t just distribute your content on social.

• Also share content that others are putting together that you find interesting
and know your audience will find interesting as well.
• This helps mix up your feed while keeping the quality of the content you’re
sharing high.
7. Promote others to build connections:

8. How can you stand out?

• 58% of consumers are not brand loyal.

• You are constantly in competition for your customer’s dollar.

• In today’s competitive market, you’ve got to stand out in order to succeed.

9. How many posts/tweets you have to publish in a day?:

 Effective social media strategy is not about posting 20 updates daily

on your page!
 It’s all about posting right thing in a right way on a right time!
10. What is our source of rich content?

• To have the opportunity for real authority on the web and vast reach, you
need a certain type of content that has depth and breadth.
• Every organization active on the social web needs at least one source of
“rich content:”
• A blog.

• A podcast.

• A video series.

• and in some cases visual content like photography

• A quick look at your follower’s reactions and interactions will give you a
deeper understanding of how well your content performed.
• Use this feedback to make calculated, data-driven improvements to your
campaigns.
INTRODCTION TO SOCIAL MEDIA ANALYTICS(SMA)
Analytics: Data analytics is the process of Analyzing raw data in order to draw out meaningful,
actionable insights, which are then used to inform and drive smart business decisions.

 In the second quarter of 2015, Facebook reported nearly 1.5 billion monthly active
users.
 In 2013, Twitter had reported a volume of 500+ million tweets per day.

 in 2015, Stack Overflow announced that more than 10 million programming

questions had been asked on their platform since the website has opened.
 These numbers are just the tip of the iceberg when describing how the popularity
of social media has grown exponentially with more users sharing more and more
information through different platforms.
 This wealth of data provides unique opportunities for data mining practitioners.
 Collecting the data from different API’s and analysed with python tools in order to
produce interesting insights on how users interact on social media.
 Data: Raw facts or primarily available facts.
Example: Number of blogs in twitter
 Information: Analyzed data is called information.
Example: Blogs related to AI
 Knowledge: Based on available information concluding or some decision
making.
Example: Creating more blogs similar to AI which will provide more
information.

 The concept of knowledge is pictured as part of a pyramid, sometimes

referred to as knowledge hierarchy, which has data as its foundation,
information as the middle layer, and knowledge at the top.
 The journey from raw data to distilled knowledge goes through the
integration of context and meaning
 Useful knowledge means actionable knowledge, that is, knowledge that
enables a decision maker to implement a business strategy.
Definition: SMA is a process of gathering and analysing the data by audience on
different social media platforms. Improves organization’s strategic business
decisions.

Social Media Analytics aims to answer the question: How to extract useful
knowledge from the data coming from social media?

Example1 : Let’s consider Instagram platform:

Person A: Having 20 million followers

 Let’s take a scenario in which I want to analyze the comments in the Person-
A account.
 The comments are nothing but the views of the people given on the posts
done by Person A.
 Here we will extract the data from Instagram account of Person A.
 Clean the data by applying different python tools and functions.
 Understand the data by applying the natural language processing, machine
learning, deep learning etc.,
 Then based on the insights we get from the analyzed data we can be able to
infer the sentiments of the people.
Example 2: Facebook
 1. Facebook revenue
Facebook’s annual revenue increased by 15.6% in 2023 to $134.9 billion.

2.Facebook revenue by region

About 45.3% of Facebook’s revenue comes from North America. Revenue across all
regions increased in 2023.
 2. Facebook Monthly Active Users

4.Facebook Daily Active Users

Approximately two billion people access Facebook everyday, either through the main app
or Facebook Messenger, or website.
Example 3: LinkedIn

LinkedIn is the world’s preeminent social network for professionals. Members create CVs,
list their current and previous job roles, skills and education. The business network is also
a recruiting website, with businesses able to create profiles and list current vacancies.

1. LinkedIn revenue
LinkedIn revenue increased by 26.2% in 2022 to $14.5 billion, its highest percentage growth
in three years.

2. LinkedIn users
LinkedIn reached 930 million registered users in 2023, however, it has not reported how many
of those users are active daily or monthly.
 3. LinkedIn users by country
The United States is home to about 30% of all LinkedIn users, the next largest market is
India. LinkedIn recently announced it would close operations in China, where it had 50
million users.
LinkedIn users by country 2022 (mm)

Types of social media analytics

In general, we have four types analytics on any form of data.

1. Descriptive analytics
2. Diagnostic analytics
3. Predictive analytics
4. Prescriptive analytics

Let’s try to understand 4 types analytics with respect to social media data.

1. Descriptive analytics:
 Summarizing the data
 Understanding data patterns
 Examining and summarizing the characteristics of the data to understand
patterns, trends, and behaviors.
  Using the already available data we will try to understand the data patterns.
Let’s take an example: FB page: Laptop Advertising Page
 I’m collecting the from facebook page for analyzing it.
 The data analysis is done on page likes, shares, comments and posts.
 The page is responsible for providing the advertisements on new brands of
laptops and other devices.
 So, we will collect the data from this specific page of FB.
 Describing the data:
-Analyzing likes
-Analyzing comments
-Analyzing shares
2. Diagnostic analytics:

 Understanding the problem

 Why did it happen

Example: Understanding the data of facebook page in order to find the problem.
After applying analytics, we find the following:
-The page is getting good amount of likes
-Getting good amount of shares
-Getting more negative comments from people.
3. Prescriptive analytics:

 Analyzing the problem.

 Why did it happen.
 Understanding how to solve the problem.
 Providing the solution to a problem
Example:
-The negative comments may be because of the quality of advertisements.
-May be fraud transaction on products by third person
4.Predictive analytics:

 Finding out how to resolve the problem in upcoming days.

 What will happen in future.
  When we apply solution what will be the effect in the future.
 Example:
May be a possible solution to improving the comments from customers by providing high
quality advertisement adds.

Providing brand loyalty by identifying the fraud activities

Making it interactive to consumers.

Uses of social media analytics:

1. Audience Demographics: Social media analytics can provide information about the
demographics of your audience, including age, gender, location, language, and
interests. This data helps businesses tailor their content to better resonate with their
target audience.
2. Engagement Metrics: Social media platforms offer various engagement metrics such
as likes, shares, comments, and retweets. Analyzing these metrics helps businesses
understand which content resonates most with their audience and which posts drive the
highest levels of engagement.
3. Sentiment Analysis: Social media analytics tools can analyze the sentiment of social
media posts to determine whether the overall sentiment toward a brand, product, or
topic is positive, negative, or neutral. This information is valuable for assessing public
perception and managing brand reputation.
4. Trend Analysis: Social media analytics can identify emerging trends and topics that
are relevant to your industry or audience. By monitoring trending hashtags, keywords,
and topics, businesses can stay ahead of the curve and create timely content that aligns
with current interests.
5. Influencer Identification: Social media analytics tools can help businesses identify
influencers who have a significant impact on their target audience. By analyzing factors
such as follower count, engagement rate, and content relevance, businesses can
collaborate with influencers to reach a wider audience and promote their products or
services.
6. Competitor Analysis: Social media analytics can provide insights into the social media
strategies of competitors, including their posting frequency, content types, engagement
levels, and audience demographics. By benchmarking against competitors, businesses
can identify areas for improvement and refine their own social media strategies.
 7. Campaign Performance Tracking: Social media analytics allows businesses to track
the performance of their social media campaigns in real-time. By monitoring key
metrics such as reach, engagement, click-through rates, and conversions, businesses
can assess the effectiveness of their campaigns and make data-driven optimizations to
improve results.
8. Customer Feedback and Insights: Social media analytics can help businesses gather
valuable feedback and insights directly from their audience. By monitoring mentions,
comments, and direct messages, businesses can identify common issues, concerns, and
preferences among their customers and tailor their products, services, and messaging
accordingly.

Opportunities:
Social media analytics presents numerous opportunities for businesses, marketers, researchers,
and individuals looking to leverage data-driven insights for various purposes. Here are some
key opportunities in social media analytics:

1.Audience Understanding: Social media analytics offers the opportunity to gain deep
insights into audience demographics, behaviors, preferences, and interests. Understanding your
audience better enables you to create more targeted and personalized content, products, and
services.

2.Content Optimization: By analyzing engagement metrics such as likes, shares, comments,

and click-through rates, businesses can optimize their content strategy to resonate more
effectively with their audience. Social media analytics can help identify which types of content
perform best and inform decisions about content creation, timing, and distribution.

3.Brand Monitoring and Reputation Management: Social media analytics tools allow
businesses to monitor conversations about their brand, products, and competitors in real-time.
This enables proactive reputation management by identifying and addressing potential issues,
responding to customer feedback, and capitalizing on positive sentiment.

4.Campaign Performance Tracking: Social media analytics enables businesses to track the
performance of their social media marketing campaigns and initiatives. By monitoring key
metrics such as reach, engagement, conversions, and return on investment (ROI), businesses
can assess the effectiveness of their campaigns and make data-driven optimizations to improve
results.

5.Influencer Marketing: Social media analytics can help businesses identify and evaluate
potential influencers for collaboration. By analyzing factors such as follower count,
engagement rate, audience demographics, and content relevance, businesses can identify
influencers who align with their brand values and target audience, thereby maximizing the
impact of influencer partnerships.

6.Competitive Intelligence: Social media analytics provides opportunities to gain insights into
competitors' social media strategies, performance, and audience engagement. By monitoring
competitors' activities and benchmarking against industry standards, businesses can identify
emerging trends, capitalize on opportunities, and stay ahead of the competition.

7.Product Development and Innovation: Social media analytics can inform product
development and innovation by providing insights into consumer needs, preferences, and pain
points. By analyzing social media conversations, sentiment, and feedback, businesses can
identify emerging trends, gather user feedback, and prioritize feature enhancements or new
product ideas.

8.Customer Service and Engagement: Social media analytics enables businesses to enhance
customer service and engagement by monitoring and responding to customer inquiries,
complaints, and feedback in real-time. By providing timely and personalized responses,
businesses can build stronger relationships with customers and foster brand loyalty.

9.Market Research and Insights: Social media analytics can be used for market research and
insights generation across various industries and sectors. By analyzing social media
conversations, trends, and sentiment, businesses can uncover valuable insights into consumer
behaviour, preferences, and market dynamics, thereby informing strategic decision-making and
market positioning.

Challenges:
1.Data quality and reliability:

One of the main challenges of social media analytics is the quality and reliability of the data.
Social media data is often unstructured, noisy, incomplete, and biased. For example, some users
may post fake or misleading information, some may delete or edit their posts, and some may
use different accounts or platforms. Moreover, social media data is constantly changing and
evolving, which makes it difficult to track and compare over time. To ensure data quality and
reliability, managers and analysts need to use appropriate tools and methods to collect, clean,
filter, and validate the data.

2.Data privacy and ethics:

Another challenge of social media analytics is the data privacy and ethics. Social media data
involves personal and sensitive information of users, such as their opinions, preferences,
behaviours, and identities. Therefore, managers and analysts need to respect the rights and
expectations of the users and comply with the relevant laws and regulations. They also need to
follow the ethical principles and guidelines of their profession and organization. For example,
they should obtain consent from the users, anonymize the data, protect the data from
unauthorized access or misuse, and disclose the purpose and scope of the analysis.

3.Data interpretation and communication:

A third challenge of social media analytics is the data interpretation and
communication. Social media data is often complex, multidimensional, and
contextual. It requires a combination of quantitative and qualitative skills to analyse
and understand the data. Moreover, social media data is not always objective or
conclusive. It may contain ambiguities, contradictions, or uncertainties. Therefore,
managers and analysts need to use appropriate techniques and tools to visualize,
summarize, and present the data. They also need to communicate the findings and
insights clearly, accurately, and persuasively to the stakeholders, such as the clients,
the executives, or the public.
4.Data integration and alignment:
A fourth challenge of social media analytics is the data integration and alignment.
Social media data is not the only source of information for businesses. There are also
other types of data, such as transactional, operational, or financial data, that can
provide valuable insights. However, integrating and aligning different types of data
is not easy. It requires a common framework and standard to ensure the consistency
and compatibility of the data. It also requires a clear understanding of the objectives
and expectations of the analysis. Managers and analysts need to define the scope and
criteria of the data integration and alignment. They also need to coordinate and
collaborate with other teams and departments to ensure the data quality and
relevance.

5.Data strategy and governance:

A fifth challenge of social media analytics is the data strategy and governance. Social
media analytics is not a one-time or isolated activity. It is a continuous and dynamic
process that requires a strategic and systematic approach. Managers and analysts
need to establish and follow a data strategy and governance that defines the vision,
goals, roles, responsibilities, policies, and procedures of the social media analytics.
They also need to monitor and evaluate the performance and impact of the social
media analytics and make adjustments and improvements as needed.

SMA STEPS/PROCESS:

1. Authentication

2. Data collection

3. Data cleaning and processing

4. Modelling and analysis

5. Result presentation
1. The authentication step is typically performed using the industry standard
called Open Authorization (OAuth).
2. The process is three legged.

3. Involves three actors:

6. *A user

7. *Consumer (our application)

8. *Resource provider (social media platform)

The steps in process are as follows:

1. The user agrees with the consumer to grant access to the social media
platform.
2. As the user doesn’t give their social media password directly to the
consumer.
3. The consumer has initial exchange with the resource provider to
generate a token and a secret.
4. This are used to sign each request and prevent forgery.
 5. The user is then redirected with the token to the resource provider, which
will ask to confirm authorizing the consumer to access the user’s data.
6. Depending on the nature of social media platform, it will also ask to
confirm whether the consumer can perform any action on the user’s
behalf, for example: Post an update, share a link so on.
7. The resource provider issues a valid token for the consumer.

8. The token can then go back to the user confirming the access.

The OAuth Process:

 Performed using the industry standard called Open

Authorization (OAuth).

 The process involves three actors: a user, consumer (our application), and
resource provider (the social media platform).
 The exchange of credentials (username/password) only happens between the
user and the resource provider through the steps 3 and 4.

 All other exchanges are driven by tokens

The data collection, cleaning and pre-processing steps are also dependent on the
type of social media platform we are dealing.
Understanding the process:

 The first step is to understand the process involved in exploitation of data

present on social networks.

 A proper execution of the process, with attention to small details, is the key
to good results.
A small error in code while dealing in data science will lead to entirely wrong results,
which in turn will lead to i

1. Problem definition: Understanding the problem is crucial for choosing

the right data sources and the methods of analysis. It also helps to realize
what kind of information and conclusions we can infer from the data and
what is impossible to derive.
2. Getting The Right Data: according to the defined problem. Many
social media platforms allow users to collect a lot of information in an
automatized way via APIs (Application Programming Interfaces).
However, other platforms, such as forums or blogs, usually require a
customized programming approach
3. Cleaning: Involves tasks such as duplicates removal (retweets on
Twitter), or spam detection to remove irrelevant comments, language
detection to perform linguistic analysis, or other statistical or machine
learning approaches that can help to produce a clean dataset.
4. Data Analysis: The choice of the analysis method depends on the
objectives of the study and the type of data. It may require statistical or
 machine learning approach, or a specific approach to time series.
5. Data Visualization: Data and output can be presented in many
different ways, starting from charts, plots, and diagrams through more
complex 2D charts, to multidimensional visualizations

SOCIAL MEDIA DATA

• Need online access to historic and real-time social media data
 Social media data is typically either available via simple general routines or
require the programmer to program their analytics in a language such as
MATLAB, java or python non-programming interfaces are required for
giving what might be termed as ‘deep’ access to ‘raw’ data. Tools are
required for combining (and conducting analytics across) multiple social
media and other data sets.
 Also require visualization tools whereby information that has been
abstracted can be visualized in some schematic form with the goal of
communicating information clearly and effectively through graphical means.

Types of data:
In social media analytics, various data types are analyzed to gain insights into user
behavior, trends, sentiment, and engagement. Some of the key data types commonly
analyzed in social media analytics include:

a. Text Data: Textual data from social media posts, comments, reviews, and
messages are analyzed to understand sentiment, topics, themes, and
language patterns. Natural language processing (NLP) techniques are
often employed to extract meaning from text data.
b. Numeric Data: Numeric data such as likes, shares, comments, followers,
and engagement metrics are important indicators of user interaction and
popularity on social media platforms. Analyzing numeric data helps
assess the reach and impact of social media content.
c. Multimedia Data: Multimedia content such as images, videos, and audio
clips are analyzed using techniques like image recognition, object
 detection, and audio analysis. Multimedia data analysis helps understand
visual trends, identify brand logos, and detect sentiment expressed in
images and videos.
d. User Metadata: User metadata includes demographic information,
location data, device type, and user behavior patterns. Analyzing user
metadata helps segment audiences, personalize content, and target
specific demographics with tailored messaging.
e. Network Data: Network data refers to the relationships and connections
between users and content on social media platforms. Network analysis
techniques are used to identify influencers, map social networks, and
understand information diffusion patterns.
f. Time-Series Data: Time-series data tracks changes in social media
metrics over time, such as daily or hourly fluctuations in engagement,
follower growth, and content performance. Analyzing time-series data
helps identify temporal trends, seasonality patterns, and the impact of
specific events or campaigns.
g. Geospatial Data: Geospatial data provides insights into the geographic
distribution of social media activity, including user locations, trending
topics by region, and localized engagement patterns. Geospatial analysis
helps optimize location-based marketing strategies and target users based
on their geographic location.
By analyzing these diverse types of data, social media analytics platforms can provide
valuable insights to businesses, marketers, and researchers, helping them make
informed decisions and improve their social media strategies.

Channel measurement metrics

Channel metrics cover activity which occurred across your entire channel
(including all existing posts) during your chosen date range. The wide range of
metrics you can choose from are designed to cover your entire channel's followers,
demographics, video views, total post insights, direct messages, and more.

Identifying the area for improvement.

Tools for Channel Measurement:

1. Sprout Social : Sprout Social is a social media management and intelligence

tool for brands and agencies of all sizes to manage conversations and surface
the actionable insights that drive real business impact.

2. HubSpot: HubSpot is a CRM platform that's designed to scale with your

business. From a one-person business to a 2,000+ employee enterprise, The
HubSpot CRM platform has the marketing, sales, customer service,
operations, and content management features you need to build the best
experience for your customers.

3. TapInfluence : TapInfluence offers a platform brands need to create the

social content consumers actually want. Combining the leading marketplace
of talented and influential content creators with end-to-end workflow,
TapInfluence streamlines comprehensive influencer marketing.

4. BuzzSumo : BuzzSumo is a tool that helps you analyze content

engagement, track trends, and spot outreach opportunities across social
and search.
5. Snaplytics : Snaplytics is the platform for all your story marketing needs.
Finally, you can work on your Snapchat and Instagram handles on desktop!
Schedule content, monitor opens and export reports in one place. Even better,
you can also track your competitors, user-generated content and messages to
further boost engagement.

6. Curalate: Curalate is a tool that provides you with a smarter way to make
social sell. It enables you to use social content, social audiences, and social
channels to sell more efficiently and effectively across your marketing and
commerce channels. Workflow that lets your day flow.

7. Keyhole: Keyhole uncovers the sentiment behind every brand mention, how
it changes over time, and why. Measure the real impact of your social media
 campaigns easily, including engagement from key influencers and total
reach generated.

8. Google Analytics: Google Analytics helps improve website performance by

providing insights into user behaviour, traffic sources, and content
engagement, enabling data-driven optimizations for better user experience
and higher conversion rates.

9. ShortStack : ShortStack is a digital marketing platform built for

engagement, lead collection and marketing automation. Create unique
contest landing pages, games and quizzes, run social contests, choose
winners, send emails and analyze results - all from one place.

10. SHIELDApp: The software and service we provide today revolves

around statistical insight and analysis on profile activity and organic content
performance. To learn what works and what doesn't. We do not scrape or
automate user behavior. We believe it requires people and a dedicated effort
to build and nurture relationships.

SOCIAL MEDIA ANALYTICS TECHNIQUES

We have various techniques to work on social media data which will be extracted
from various platforms such as Facebook, twitter, Instagram in order to analyse it
and get meaningful insights to make better business decisions.

1. Opinion mining:

 Sentimental analysis is also known as opinion mining.

 It’s natural language processing (NLP) based approach which identifies the
emotional tone behind a body of text.
 his is a popular way for organizations to determine and categorize opinions
about a product, service or idea.
  Sentiment analysis involves the use of data mining, machine learning (ML),
artificial intelligence and computational linguistics to mine text for sentiment
and subjective information such as whether it is expressing positive, negative
or neutral feelings.
 Sentiment analysis systems help organizations gather insights into real- time
customer sentiment, customer experience and brand reputation.
 Generally, these tools use text analytics to analyse online sources such as
emails, blog posts, online reviews, customer support tickets, news articles,
survey responses, case studies, web chats, tweets, forums and comments.
 Algorithms are used to implement rule-based, automatic or hybrid methods
of scoring whether the customer is expressing positive words, negative
words or neutral ones.
 In addition to identifying sentiment, sentiment analysis can extract the
polarity or the amount of positivity and negativity, subject and opinion
holder within the text.
 This approach is used to analyse various parts of text, such as a full document
or a paragraph, sentence or subsentence.

How does sentiment analysis work?

 Sentiment analysis uses machine learning models to perform text analysis of

human language. The metrics used are designed to detect whether the overall
sentiment of a piece of text is positive, negative or neutral.

Sentiment analysis generally follows these steps:

1. Collect data: The text being analysed is identified and collected. This
involves using a web scraping bot or a scraping application programming
interface.
2. Clean the data: The data is processed and cleaned to remove noise and
 parts of speech that don't have meaning relevant to the sentiment of the
text. This includes contractions, such as I'm, and words that have little
information such as is, articles such as the, punctuation, URLs, special
characters and capital letters. This is referred to as standardizing.
3. Extract features: A machine learning algorithm automatically extracts
text features to identify negative or positive sentiment. ML approaches
used include the bag-of-words technique that tracks the occurrence of
words in a text and the more nuanced word-embedding technique that
uses neural networks to analyze words with similar meanings.
4. Pick an ML model: A sentiment analysis tool scores the text using a rule-
based, automatic or hybrid ML model. Rule-based systems perform
sentiment analysis based on predefined, lexicon-based rules and are often
used in domains such as law and medicine where a high degree of precision
and human control is needed. Automatic systems use ML and deep learning
techniques to learn from data sets. A hybrid model combines both
approaches and is generally thought to be the most accurate model. These
models offer different approaches to assigning sentiment scores to pieces of
text.
5. Sentiment classification: Once a model is picked and used to analyse a
piece of text, it assigns a sentiment score to the text including positive,
negative or neutral. Organizations can also decide to view the results of their
analysis at different levels, including document level, which pertains mostly
to professional reviews and coverage; sentence level for comments and
customer reviews; and sub-sentence level, which identifies phrases or
clauses within sentences.
Types of sentiment analysis:

1. Fine-grained sentiment analysis breaks down sentiment indicators into

 more precise categories, such as very positive and very negative. This
approach is similar to opinion ratings on a one-to-five-star scale. This
approach is therefore effective at grading customer satisfaction surveys.
2. Emotion detection analysis identifies emotions rather than positivity and
negativity. Examples include happiness, frustration, shock, anger and
sadness.
3. Intent-based analysis recognizes motivations behind a text in addition to
opinion. For example, an online comment expressing frustration about
changing a battery may carry the intent of getting customer service to reach
out to resolve the issue.
4. Aspect-based analysis examines the specific component being positively or
negatively mentioned. For example, a customer might review a product
saying the battery life was too short. The sentiment analysis system will note
that the negative sentiment isn't about the product as a whole but about the
battery life.

Why is sentiment analysis important?

 Sentiment analysis is an important way for organizations to understand how

customers perceive and experience their products and brands. Increasingly,
customer feedback is given online through a variety of unconnected
platforms, such as Amazon product reviews and posts on social media
platforms.

 Organizations typically don't have the time or resources to scour the internet
and read and analyze every piece of data relating to their products, services
and brand. Instead, they use sentiment analysis algorithms to automate this
process and provide real-time feedback.

 Organizations use this feedback to improve their products, services and

customer experience. A proactive approach to incorporating sentiment
 analysis into product development can lead to improved customer loyalty
and retention.

What is sentiment analysis used for?

 Sentiment analysis tools are used in nearly every industry for a variety of
applications:
 Social media monitoring, a key strategy that tracks customer sentiments
across social media platforms, such as Facebook, Instagram and Twitter.
 Monitoring brand awareness, reputation and popularity at a specific
moment or over time.
 Analysing consumer reception of new products or features to identify
possible product improvements.
 Evaluating the success of a marketing campaign.

 Pinpointing a target audience or demographic.

 Conducting market research, such as emerging trends and competitive

insights.
 Categorizing customer service requests and automating customer service.

 Customer support analysis to assess the effectiveness of customer support

and monitor trending issues.

Benefits of sentiment analysis:

The benefits of sentiment analysis include the following:

 Collecting large amounts of unstructured data from various sources.

 Tracking real-time customer feedback and sentiment about an

organization's brand, products and services.
 Providing feedback on ways to improve products, services and customer
experience.
 Getting data and feedback on problems with products and services.

 Gathering data and feedback that keeps customer support staff up to date on
customer issues and improves their ability to respond.
 Tracking the effectiveness of customer support through support tickets and
other online feedback.
 Automating customer service by identifying customers' sentiments and
automatically sending them to relevant FAQ responses for resolution.
 Identifying emerging marketing trends, and understanding and improving
what marketing strategies resonate with customers.
 Gaining competitive insights by monitoring comments about competitors.

 Establishing consistent criteria for evaluating sentiment instead of relying on

subjective human analysis.
 Identifying and reacting to emerging negative sentiments before they
escalate.
 Freeing employee time and energy for other tasks.

 Improving the empathy of organizations and customer service.

Challenges with sentiment analysis:

Challenges associated with sentiment analysis typically include the

following:

 Neutral sentiments: Comments with a neutral sentiment tend to pose a

problem for systems and are often misidentified. For example, if a customer
received the wrong colour item and submitted a comment, "The product was
blue," this could be identified as neutral when in fact it should be negative .
 Unclear language: Sentiment is challenging to identify when systems don't
understand the context or tone. Answers to polls or survey questions like
"nothing" or "everything" are hard to categorize when the context is not
given; they could be labeled as positive or negative depending on the
question. This is known as lexical ambiguity. Similarly, it's difficult to train
systems to identify irony and sarcasm, and this can lead to incorrectly
labelled sentiments. Algorithms have trouble with pronoun resolution, which
refers to what the antecedent to a pronoun is in a sentence. For example, in
analysing the comment "We went for a walk and then dinner. I didn't enjoy
it," a system might not be able to identify what the writer didn't enjoy -- the
walk or the dinner.

 Unclassifiable language: Computer programs have difficulty understanding

emojis and irrelevant information. Special attention must be given to training
models with emojis and neutral data so they don't improperly flag texts.

 Ambiguous sentiments: People can be contradictory in their statements.

Most reviews will have both positive and negative comments. This situation
can be managed by analysing sentences one at a time. However, sentences
that contain two contradictory words, also known as contrastive
conjunctions, can confuse sentiment analysis tools. For example, "The
packaging was terrible but the product was great."

 Named-entity recognition: This is when an algorithm cannot recognize the

meaning of a word in its context. For instance, the use of the word "Lincoln"
may refer to the former United States President, the film or a penny.

 Small data sets: Sentiment analysis tools work best when analyzing large
 quantities of text data. Smaller data sets often won't provide the insight
needed.
 Language evolution: Language is constantly changing, especially on the
internet where users are continually creating new abbreviations, acronyms,
and using poor grammar and spelling. This level of variation and evolution
can be difficult for algorithms.

 Fake reviews: Algorithms can't always tell the difference between real and
fake reviews of products, or other pieces of text created by bots.

 Need for human intervention: Gartner finds that even the most advanced
AI-driven sentiment analysis and social media monitoring tools require
human intervention in order to maintain consistency and accuracy in
analysis.

Machine Learning:

What is Machine Learning:

In the real world, we are surrounded by humans who can learn everything
from their experiences with their learning capability, and we have computers
or machines which work on our instructions. But can a machine also learn
from experiences or past data like a human does? So here comes the role of
Machine Learning.
 A subset of artificial intelligence known as machine learning focuses
primarily on the creation of algorithms that enable a computer to
independently learn from data and previous experiences.
 Arthur Samuel first used the term "machine learning" in 1959. It could be
summarized as follows:
 Without being explicitly programmed, machine learning enables a machine
to automatically learn from data, improve performance from experiences,
and predict things.
 Machine learning algorithms create a mathematical model that, without
being explicitly programmed, aids in making predictions or decisions with
the assistance of sample historical data, or training data.
 For the purpose of developing predictive models, machine learning brings
together statistics and computer science.
 Algorithms that learn from historical data are either constructed or utilized
in machine learning. The performance will rise in proportion to the quantity
of information we provide.
 Machine Learning Pipeline:

Classification of machine learning algorithm:

1. Supervised learning:

 Supervised learning, as the name indicates, has the presence of a

supervisor as a teacher.
 Supervised learning is when we teach or train the machine using data
that is well-labelled.
 Which means some data is already tagged with the correct answer.

 After that, the machine is provided with a new set of examples(data)

so that the supervised learning algorithm analyses the training data
(set of training examples) and produces a correct outcome from
 labelled data.
 For example, a labelled dataset of images of Elephant, Camel and Cow
would have each image tagged with either “Elephant” , “Camel “or
“Cow.”

Types of Supervised Learning:

Supervised learning is classified into two categories of algorithms:

1. Regression: A regression problem is when the output variable is a real

value, such as “dollars” or “weight”.
2. Classification: A classification problem is when the output variable is a
category, such as “Red” or “blue” , “disease” or “no disease”.

Applications of Supervised learning:

Supervised learning can be used to solve a wide variety of problems,

including:

1. Spam filtering: Supervised learning algorithms can be trained to

identify and classify spam emails based on their content, helping users
 avoid unwanted messages.
2. Image classification: Supervised learning can automatically classify
images into different categories, such as animals, objects, or scenes,
facilitating tasks like image search, content moderation, and image-
based product recommendations.
3. Medical diagnosis: Supervised learning can assist in medical
diagnosis by analysing patient data, such as medical images, test
results, and patient history, to identify patterns that suggest specific
diseases or conditions.
4. Fraud detection: Supervised learning models can analyze financial
transactions and identify patterns that indicate fraudulent activity,
helping financial institutions prevent fraud and protect their
customers.
5. Natural language processing (NLP): Supervised learning plays a
crucial role in NLP tasks, including sentiment analysis, machine
translation, and text summarization, enabling machines to understand
and process human language effectively.

Advantages of Supervised learning:

 Supervised learning allows collecting data and produces data output from
previous experiences.
 Helps to optimize performance criteria with the help of experience.

 Supervised machine learning helps to solve various types of real-world

computation problems.
 It performs classification and regression tasks.

 It allows estimating or mapping the result to a new sample.

 We have complete control over choosing the number of classes we want in

the training data.
 Disadvantages of Supervised learning:

 Classifying big data can be challenging.

 Training for supervised learning needs a lot of computation time. So, it

requires a lot of time.
 Supervised learning cannot handle all complex tasks in Machine Learning.

 Computation time is vast for supervised learning.

 It requires a labelled data set.

 It requires a training process.

2. Unsupervised learning:

 Unsupervised learning is a type of machine learning that learns from

unlabelled data.
 This means that the data does not have any pre-existing labels or categories
The goal of unsupervised learning is to discover patterns and relationships in
the data without any explicit guidance.
 Unsupervised learning is the training of a machine using information that is
neither classified nor labelled and allowing the algorithm to act on that
information without guidance.
 Here the task of the machine is to group unsorted information according to
similarities, patterns, and differences without any prior training of data.
Types of Unsupervised Learning:

Unsupervised learning is classified into two categories of algorithms:

1. Clustering: A clustering problem is where you want to discover the

inherent groupings in the data, such as grouping customers by purchasing
behavior.
2. Association: An association rule learning problem is where you want to
discover rules that describe large portions of your data, such as people that
buy X also tend to buy Y.

Application of Unsupervised learning:

Non-supervised learning can be used to solve a wide variety of problems, including:

1. Anomaly detection: Unsupervised learning can identify unusual patterns or

deviations from normal behaviour in data, enabling the detection of fraud,
intrusion, or system failures.

2. Scientific discovery: Unsupervised learning can uncover hidden relationships

and patterns in scientific data, leading to new hypotheses and insights in various
scientific fields.

3. Recommendation systems: Unsupervised learning can identify patterns and

similarities in user behaviour and preferences to recommend products, movies, or
music that align with their interests.

4. Customer segmentation: Unsupervised learning can identify groups of

customers with similar characteristics, allowing businesses to target marketing
campaigns and improve customer service more effectively.

5. Image analysis: Unsupervised learning can group images based on their content,
facilitating tasks such as image classification, object detection, and image retrieval.

Advantages of Unsupervised learning:

 It does not require training data to be labelled.

 Dimensionality reduction can be easily accomplished using unsupervised

learning.
 Capable of finding previously unknown patterns in data.

 Unsupervised learning can help you gain insights from unlabeled data that
you might not have been able to get otherwise.
 Unsupervised learning is good at finding patterns and relationships in data
without being told what to look for. This can help you learn new things about
your data.

Disadvantages of Unsupervised learning:

 Difficult to measure accuracy or effectiveness due to lack of predefined

answers during training.
 The results often have lesser accuracy.
  The user needs to spend time interpreting and label the classes which follow
that classification.
 Unsupervised learning can be sensitive to data quality, including missing
values, outliers, and noisy data.
 Without labelled data, it can be difficult to evaluate the performance of
unsupervised learning models, making it challenging to assess their
effectiveness.

3. Reinforcement learning:

 Reinforcement learning is an area of Machine Learning.

 It is about taking suitable action to maximize reward in a particular

situation.
 It is employed by various software and machines to find the best possible
behaviour or path it should take in a specific situation.
 Reinforcement learning differs from supervised learning in a way that in
supervised learning the training data has the answer key with it so the model
is trained with the correct answer itself whereas in reinforcement learning,
there is no answer but the reinforcement agent decides what to do to perform
the given task.
 In the absence of a training dataset, it is bound to learn from its experience.

 Reinforcement Learning (RL) is the science of decision making.

 It is about learning the optimal behaviour in an environment to obtain

maximum reward.
 In RL, the data is accumulated from machine learning systems that use a
trial-and-error method.
 Data is not part of the input that we would find in supervised or
unsupervised machine learning.
 Example: Driverless Car Robots

Application of Reinforcement Learnings:

1. Robotics: Robots with pre-programmed behaviour are useful in

structured environments, such as the assembly line of an automobile
manufacturing plant, where the task is repetitive in nature.

2. A master chess player makes a move. The choice is informed both by

planning, anticipating possible replies and counter replies.

3. An adaptive controller adjusts parameters of a petroleum refinery’s

operation in real time.

Advantages of Reinforcement learning:

1. Reinforcement learning can be used to solve very complex problems that cannot
be solved by conventional techniques.

2. The model can correct the errors that occurred during the training process.

3. In RL, training data is obtained via the direct interaction of the agent with the
environment

4. Reinforcement learning can handle environments that are non-deterministic,

meaning that the outcomes of actions are not always predictable. This is useful in
real-world applications where the environment may change over time or is
uncertain.
Disadvantages of Reinforcement learning:

1. Reinforcement learning is not preferable to use for solving simple problems.

2. Reinforcement learning needs a lot of data and a lot of computation

3. Reinforcement learning is highly dependent on the quality of the reward

function. If the reward function is poorly designed, the agent may not learn the
desired behaviour.

4. Reinforcement learning can be difficult to debug and interpret. It is not always

clear why the agent is behaving in a certain way, which can make it difficult to
diagnose and fix problems.