0% found this document useful (0 votes)
23 views9 pages

Developing A Course Recommender System Using Python

Ok3

Uploaded by

cs120202067
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
0% found this document useful (0 votes)
23 views9 pages

Developing A Course Recommender System Using Python

Ok3

Uploaded by

cs120202067
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 9

Developing A Course Recommender System using Python

📢 Data Science Blogathon is LIVE now! Write and Earn


✨USD 26✨ for every published article!
Start Writing

Home Beginner
Developing A Course Recommender System using Python

Siddharth Sachdeva
25 Aug, 2021 • 6 min read

This article was published as a part of the Data Science


Blogathon
Introduction
Recommender System is a software system that provides
specific suggestions to users according to their
preferences. These techniques may provide decision-
making capabilities to the user. Items refer to any product
that the recommender system suggests to its user like
movies, music, news, travel packages, e-commerce
products, etc.

Image Source : Google Images

A Recommender system is built for a specific type of


application depending on the items it recommends and
accordingly its graphical user interface and design are
customized. Recommender system development emerged
from the general idea that individuals rely on others to
take regular decisions in their life. The explosive growth in
volume and variety of information available on the web
has contributed to the development of recommender
We use cookies on Analytics Vidhya websites to deliver our services, analyze web traffic, and improve your experience on the site. By using
Analytics Vidhya, you agree to our Privacy Policy and Terms of Use. Accept
systems which
Developing A Course Recommender has, using
System in turn,Python
led to an increase in profit and
benefits to the user.
Types of Recommender Systems
There are three broad categories of recommender
systems:

Image Source: Google images


1.Collaborative filtering-based systems:

These systems recommend items to users based on the


similarity computation of these users to similar users in
the system or based on the items similar to the items liked
by the user in past. So collaborative filtering can be
further divided into two categories-
i) user-based collaborative filtering– The
recommender system tries to find out similar users to
the target user by calculating certain similarity
measures and then suggest items to the target user
based on similar user preferences. Similarity
calculation is an important task here.
ii.) item-based collaborative filtering– The
recommender system tries to find out items based
on previous user preferences of the user and then
recommend similar items to
the user. These items might be of interest to the user.
2. Content-based Recommender Systems:
We use cookies on Analytics VidhyaThe system
websites focuses
to deliver our on the properties
services, analyze weboftraffic,
the items to be your experience on the site. By using
and improve
suggested
Analytics Vidhya,toyouthe users.
agree to ourForPrivacy
example,
Policy ifanda YouTube userAccept
Terms of Use.
has watchedSystem
Developing A Course Recommender comedyusing
videosPython
then the system will
recommend comedy genre videos to him. Similarly, if a
Netflix user has watched movies of a particular director
then the system will recommend those movies to him.
3. Hybrid Systems: These systems are a combination of
collaborative and content-based systems.

Building a Course Recommender System


Step 1: Reading the dataset.
#importing necessary libraries.
import pandas as pd
data = pd.read_csv(r"C:UsersDellDesktopDatasetdataset.csv")
data.head()

About the dataset:


It includes data of students enrolled in high school with
their ids, streams, favorite subject, and marks obtained in
class 12. It also includes result columns of the course and
specialization they pursued in graduation.
We would like to recommend courses to students with
similar marks, streams, and favorite subjects.
Now we will build a content-based recommender system
that could recommend courses to students based on their
basic information.
Step 2: Creating a combined description of every
student in the dataset.
descriptions =data['gender'] +' '+ data['subject'] + ' ' + d
#Printing the first description
descriptions[0]

We use cookies on Analytics Vidhya websites to deliver our services, analyze web traffic, and improve your experience on the site. By using
Analytics Vidhya, you agree to our Privacy Policy and Terms of Use. Accept
Step 3: Creating
Developing A Course Recommender Systema similarity matrix between the
using Python
students.
# import TfidfVector from sklearn.
from sklearn.feature_extraction.text import TfidfVectorizer
from sklearn.metrics.pairwise import linear_kernel
def create_similarity_matrix(new_description, overall_descri
#Append the new description to the overall set.
overall_descriptions.append(new_description)
# Define a tfidf vectorizer and remove all stopwords
tfidf = TfidfVectorizer(stop_words="english")
#Convert tfidf matrix by fitting and transforming th
tfidf_matrix = tfidf.fit_transform(overall_descripti
# output the shape of the matrix.
tfidf_matrix.shape
# calculating the cosine similarity matrix.
cosine_sim = linear_kernel(tfidf_matrix,tfidf_matrix
return cosine_sim

The function create_similarity_matrix() is used to generate


the cosine similarity matrix of the students’ descriptions.
It firstly appends the new student description to whom
recommendations are being made to the overall
descriptions.

Then, Student descriptions needed to be converted into


word vectors(vectorized representation of words). So we
will use TFIDF (Term Frequency Inverse document
frequency) for converting them. the TF-IDF score is the
frequency of a word occurring in a document, down-
weighted by the number of documents in which it occurs.
This is done to reduce the importance of words that
frequently occur in descriptions and, therefore, their
significance in computing the final similarity score.
The above function then finally outputs the cosine
similarity matrix.
Step 4: Define get_recommendations() function.
def get_recommendations(new_description,overall_descriptions
# create the similarity matrix
cosine_sim = create_similarity_matrix(new_description,overal
# Get pairwise similarity scores of all the students with ne
sim_scores = list(enumerate(cosine_sim[-1]))
# Sort the descriptions based on similarity score.
sim_scores = sorted(sim_scores,key =lambda x:x[1],reverse= T
We use cookies on Analytics Vidhya websites to deliver our services, analyze web traffic, and improve your experience on the site. By using
# Get the scores of top 10 descriptions.
Analytics Vidhya, you agree to our Privacy Policy and Terms of Use. Accept
sim_scores = sim_scores[1:10]
# Get the student indices.
Developing A Course Recommender System using Python
indices = [i[0]for i in sim_scores]
return data.iloc[indices]

Step 5: Recommend the course to a new student.


new_description = pd.Series('male physics science 78')
get_recommendations(new_description,descriptions)

Showing the top two recommendations.

Issues in Recommender Systems:


1. Cold Start Problem :

Whenever a new user enters a recommender system, the


question arises of what to recommend him/her and on
what basis as previous data is not available and similarity
calculation could not be performed. One solution to this
problem is to make the new users enter a small
introduction form containing basic information about a
person’s interests, hobbies, occupation, and creating a
basic user profile and then recommending items to the
new user. This would solve the cold start problem to a
great extent.
2. Data Sparsity Problem:
The major issue in a recommender system is the
unavailability of appropriate data which is the main
requirement for the recommendation process. Many users
don’t bother to review items they bought. As a result, the
user-item rating matrix has many sparse entries which
degrade the performance of the similarity calculation
algorithm. So one solution is to predict sparse entries and
many researchers have given algorithms to predict these
ratings such as a negative weighted one slope algorithm.
3. Changing Dataset:
We use cookies on Analytics Vidhya websites to deliver our services, analyze web traffic, and improve your experience on the site. By using
Analytics Vidhya, you agree to our Privacy Policy and Terms of Use. Accept
Developing A Course Recommender System using Python

With the increase in the amount of data every day, there


is an increase in the inclusion of data in the previous
dataset of the recommender system which may alter the
overall structure and composition of the dataset. Both
new users and new items needed to get included in the
dataset. So this change needed to be accommodated in
the dataset.
4. Scalability Problem:
In a practical scenario, it is not always possible to find
similar users and similar items every time and prevent the
system from failure. So building a scalable recommender
system is a major concern.
5. Shilling attack:
Shilling attack is defined as the process of inclusion of
fake profiles and biased reviews and ratings to bias the
entire recommendation process. A malicious attacker may
inject these profiles so as to increase/decrease
recommending frequency of target items.

Benefits of a recommender system:


1. Increasing profit by increasing the number of items
sold: One of the major objectives of building commercial
recommender systems is to enhance business and
increase profit. This could be done by suggesting users
new items which may attract the users and they may buy
more items as compared to those without Recommender
Systems.
2. Enhanced user satisfaction: The main motive of any
business application should be user satisfaction as it
enhances overall business growth and the healthy survival
We use cookies on Analytics Vidhyaofwebsites
the company.
to deliverAour
well-designed recommender
services, analyze web traffic, andsystem
improve your experience on the site. By using
enhances
Analytics Vidhya,the
youuser’
agrees tooverall experience
our Privacy Policy andin Terms
usingofthat
Use. Accept
application.System
Developing A Course Recommender They mayusing
find Python
recommendations useful and
relevant to user needs. So the major purpose of RS is to
satisfy the users and make them happy.
3. Extraction of useful patterns: The
recommender system provides a way of extracting useful
patterns of users’ needs and preferences that could serve
as strategic information for the business. For example, if a
business company could get insights about which product
is being extensively liked b its customers and which
product is not being liked then they could change their
product list.
4.Provide more diverse items to users: Sometimes
it is impossible to find the items personally that a
recommender system may
provide. This increases the variety of items that users get
from
recommendations.
End Notes
Nowadays, Recommender systems have become
extremely popular. Many popular e-commerce websites
such as Amazon have employed personalized
recommender systems for their users to suggest items
that they could buy learned from their past behaviour. RS
have increased their profits. Websites like YouTube and
Netflix suggest movies and music to users based on
genres, actors, artists of movies that have been
previously watched by the customers. Travel websites like
MakeMyTrip provide customized travel packages to the
customers based on their preferences and deal with a
variety of users through recommender systems.
And finally, it doesn’t go without saying.
Thanks for reading!

The media shown in this article are not owned by Analytics


Vidhya and are used at the Author’s discretion.
blogathon machine learning python
recommender system

We use cookies on Analytics Vidhya websites to deliver our services, analyze web traffic, and improve your experience on the site. By using
Analytics Vidhya, you agree to our Privacy Policy and Terms of Use. Accept
Developing A Course Recommender System using Python
1 Sachdeva
Siddharth
25 Aug 2021
Computer science enthusiast

Beginner Data Science Machine Learning


Project Python

Responses From Readers


What are your thoughts?...

Submit reply

vishwanath
21 Jul, 2023
Hi all, Can anyone share the raw data file link ?

Write, Shine, Succeed


Write, captivate, and earn accolades and rewards for your
work

Reach a Global Audience


We use cookies on Analytics Vidhya websitesGetto Expert
deliver Feedback
our services, analyze web traffic, and improve your experience on the site. By using
Analytics Vidhya, you agree to our Privacy Policy and Terms of Use. Accept
BuildSystem
Developing A Course Recommender Your Brand & Audience
using Python
Cash In on Your Knowledge
Join a Thriving Community
Level Up Your Data Science Game

CHIRAG GOYAL Barney Darlington


87 5

Company Discover
About Us Blogs
Contact Us Expert session
Careers Podcasts
Comprehensive Guides
Learn Engage
Free courses Community
Learning path Hackathons
BlackBelt program Events
Gen AI Daily challenges
Contribute Enterprise
Contribute & win Our offerings
Become a speaker Case studies
Become a mentor Industry report
Become an instructor quexto.ai

Download App

Terms & conditions Refund Policy Privacy Policy


Cookies Policy © Analytics Vidhya 2023.All rights reserved.
We use cookies on Analytics Vidhya websites to deliver our services, analyze web traffic, and improve your experience on the site. By using
Analytics Vidhya, you agree to our Privacy Policy and Terms of Use. Accept

You might also like