International Journal on Recent and Innovation Trends in Computing and Communication ISSN: 2321-8169

Volume: 5 Issue: 6 445 449

_______________________________________________________________________________________________

BFSMpR:A BFS Graph based Recommendation System using Map Reduce

Vivek Pandey Dr. Padma Bonde

Dept. of Computer Science and Engineering Dept. of Computer Science and Engineering
ShriShankaracharya Technical Campus ShriShankaracharya Technical Campus
Junwani, Bhilai(C.G.) Junwani, Bhilai(C.G.)
[email protected] [email protected]

AbstractNowadays, Many associations, organizations and analysts need to manage huge datasets (i.e. Terabytes or even Petabytes). A well-
known information filtering algorithm for dealing with such large datasets in an effective way is Hadoop Map Reduce. These large size datasets
are regularly known to as graphs by many frameworks of current intrigue (i.e. Web, informal organization). A key element of the graph based
recommendation system is that they depend upon the neighbors interest by taking minimum distance into account. Generally recent day
proposal frameworks utilize complex strategy to give recommend to every user. This paper introduced an alternate approach to give suggestions
to users in used of an un-weighted graph using a Hadoop iterative MapReduce approach for the execution.)
Keywords-Un-Weighted Graphs, Hadoop, MapReduce, Parallel Execution, Recommendation System, Breadth first search(BFS).

__________________________________________________*****_________________________________________________

I. INTRODUCTION alternate machine that is why hadoop is a fault tolerant

framework [2].
In Big Data environment, we will experience
A best execution of MapReduce is the open-source system
scalability and efficiency problems as it has to work on huge
Apache Hadoop [3]. It was created predominantly by Yahoo!
amounts of data. Many of the traditional recommender
in spite of the fact that is additionally utilized by different
frameworks will provide same rating and ranking services to
organizations, for example, Facebook, Amazon and Last.fm
different users. So these paper introduce a Graph based service
[4]. Hadoop is the used HDFS (Hadoop Distributed File
recommender system that executes on
System) file systems which was intended to store a large
HadoopMapReduceFramework .
amount of information, so as to enhance stockpiling and get to
MapReduce is a programming model and related usage for
operations to few vast record, as opposed to a ton of little
handling and creating extensive informational collections with
document [5].
parallel execution and widely calculation on a bunch.
These days, numerous frameworks of current interest to
MapReduce has been ideated by Google's specialist Jeffrey
mainstream researchers can helpfully be spoken to as charts
Dean and Sanjay Ghemawat in 2004 for dealing with huge
[6].. Each of systems comprises of an arrangement of hubs
informational collections which are normally figured by
speaking to, for example, PCs or switches on the Internet or
hundreds or thousands machines to complete in the most brief
individuals in an informal community associated each other by
conceivable period [1][17]. MapReduce works through two
edges, speaking to information associations between PCs,
concepts called Map() and Reduce(). The Map() work takes an
relationships amongst individuals, etc. Frequently these graphs
info component and produced an arrangement of intermediate
are comprised of an enormous measure of hubs and edges. In
key-value combines and passes it to the Reduce() work. The
which the hubs are the serves and interconnection between two
Reduce() work takes the arrangement of the middle of the road
hubs with a wire are called edges.
key-value sets, combines all the transitional qualities for a
We considered an alternate approach to give suggestions
specific key and delivers a littler arrangement of merged
to users of systems. Our program expects to visit an
output values.
unweighted graph beginning from an arrangement of hubs
The energy of MapReduce is that it permits software
(additionally called key nodes) to discover which hubs can be
engineers without a profound involvement in parallel and
come to by the key nodes inside a movable most extreme
circulated frameworks to effectively use the asset of a vast
separation. It has been actualized using MapReduce keeping in
dispersed framework, breaking a calculation into small jobs
mind the end goal to deal with enormous datasets.
that keep running in parallel on different machines, and scales
This paper is sorted out as takes after. Segment 2 depicts
effortlessly to substantial bunches of modest product PCs.
related works which has been used as beginning stage for our
Additionally, another key advantage of MapReduce is that it
usage of the venture. Area 3 shows the issue and research
smartly handles the failures and crashes of servers. Actually if
mechanism and usage. Segment 4 portrays every one of the
a server crashes, MapReduce runs the undertaking on an

445
IJRITCC | June 2017, Available @ http://www.ijritcc.org
_______________________________________________________________________________________
International Journal on Recent and Innovation Trends in Computing and Communication ISSN: 2321-8169
Volume: 5 Issue: 6 445 449
_______________________________________________________________________________________________
analyses we performed using diverse arrangements of Before beginning clarify how our calculation functions
information. At last, we finish up inspection. work, we need to enhance what BFS [12],[13]implies.
Expansiveness initially hunt is a graph traversal calculation.
The pursuit starts from the root hub and the neighboring hubs
II. REVIEW WORK
are visit until there are not any more conceivable hubs to visit.
Discussed in [7] paper, Author proposed a new approach to One method for playing out the BFS is shading the hubs and
improve the quality of collaborative filtering recommendation navigating as indicated by the shade of the hubs.
systems. The algorithm combines item clustering and There are three conceivable hues for the hub - white
weighted slope one scheme.. (unvisited), gray (went by) and black (wrapped up). Before
Paper [8] introduced two semantic social recommendation beginning every one of the hubs are hued in white aside from
algorithms called Node-Edge-Based and Node-Based, these the source hub that will be shaded gray. To build up this
algorithms recommend an input item to a group of users. calculation we have to run a similar Mapper and Reducer
[9], in this paper a combined approach of user-user numerous circumstances (iterative guide reduce).Each
collaborative filtering (CF) and item-item CF has been emphasis can utilize the past cycle's yield as its information.
presented to generate recommendations on Hadoop cluster "Iterative MapReduce and Counters" contains likewise an
using Apache Mahout, a library for machine learning answer for the most limited way issue (the smallest way
algorithms. between two hubs can be characterized as the way that has the
[10], Recommendation engines are a natural fit for base aggregate weight of the edges along the way. in the event
analytics platforms. They involve processing large amounts of that we don't discuss weighted graph the base weight will be
consumer data that collected online, and the results of the quite recently (the base number of edges). It actualizes this
analysis feed real-time online applications. issue using Dijkstra's calculation.
Paper [11] presented combined Collaborative Filtering The calculation is intended to make it keep running
using Mahout on Hadoop for movie recommendation. By on a graph that has just a source hub (or as we lean toward call
combining User-based and Item-based CF, accuracy of the it a "starter hub"), rather our solution needs to has more than a
results gets improve. Hadoop has increased throughput. solitary begin hub ( i.e. the entirety set of key nodes given in
Because of multiple computer nodes, time taken for solving info). At that point we change the graph information's
problem has been reduced. structure with a specific end goal to have the capacity to run
the BFS beginning from more than one hub (those will be the
III. BFSMPR RECOMMENDATION SYSTEM MECHANISM
key nodes). Likewise we attempted to adjust our information
Numerous frameworks that are consistently used now structure as per the lion's share of diagrams. we discovered
can be called as graphs. Some of them are informal online which enabled us to utilize our program in bunches of
organizations, Internet and manyothers. A hefty portion of graphs. So we changed the graph contribution from the
those frameworks utilize a technique to give recommendation accompanying configuration:
to the user of the system. Some illustration are YouTube [20]
that proposes you to watch videos, Facebook prescribes ID Neighbors Distance-From-Source Color
peoples that you could know, Amazon [21] [22]
recommendssame item you purchased at certain criteria.
Regularly those recommender frameworks utilize complex
method to give advices to every user. The advices can be
founded on estimations of likeness between products or users.
Existing recommender frameworks can be sorted into two
unique Techniques: content-based and synergistic sifting [19]
[20].
In content based approach the framework suggest
product to a particular user, by using the history of user
purchases. For this situation the user history are utilized and
make up a vector of components where estimation of an
element can be set up by TF-IDF (Term frequencyInverse
Document Frequency) algorithm.Another user is coordinated Figure1:-The description of social circle dataset
against the database to find neighbours, which are different Figure 2 shows Output to the Reducer,Traversed the miinimum distance .
users who have generally had comparative taste to him/her.
Products that the neighbours like are then prescribed to the
user, as he/she will presumably additionally like them [21
446
IJRITCC | June 2017, Available @ http://www.ijritcc.org
_______________________________________________________________________________________
International Journal on Recent and Innovation Trends in Computing and Communication ISSN: 2321-8169
Volume: 5 Issue: 6 445 449
_______________________________________________________________________________________________
Figure 1 shows the process of the BFSMpRalgorithm.
Firstly, we should use key node and graph as input. Mapper
discovers every single Adjacent Node and stamps as Gray then
Reducer computes minimum distance and applies BFS
calculation from source

Input: Key
Node & Graph

Node 1 Node 2

Figure2:-The description of Output Generated by Reduce Function

Various classes used for implementing Recommendation task
are given below Mapper Mapper

A. Base Job
Our project does not need any complex change in the base Find all Adjacent Find all Adjacent
Nodes & Marks as Nodes & Marks as
job of hadoop system, so we choose to keep the Hadoop base GREY
GREY
employment structure and adjust it with just somewhat
alterations without affecting hadoop basic job. BaseJob is
Reducer Reducer
utilized to announce the theoretical class JobInfo that contains
getter techniques to get the program-particular classes related
Calculate Min Calculate Min
with Job. Distance and Distance and Apply
Apply BFS BFS Algorithm
B. Driver Job Algorithm from Source
from Source
This class contains the driver to execute the job and send
for the guide/decrease capacities and the fundamental class.
Output Output
The undertaking of the principle class is simply to make the Collector
Collector
yield catalog and call the run strategy.
C. Nodes
Hub class contains the data about the hub ID, the rundown
of nearby hubs, the separation from the source, the shade of
the hub (shading could be white for unvisited hub, grY for
went to hub and black for the hub which has been gone to by Final Output (Reco mmended Gra ph)
all the key nodes, the parent hub and the starter list (list of the
keynodes that went to the hub). Figure 3:-Work flow of BFS recommndation system

D. Mapper IV. RESULTS AND DISCUSSION

Mapper Worker is the base mapper class for the projects
After implementation of, the fundamental
that utilization parallel expansiveness initially looks
components of the graph (ID and nearby Nodes) are overseen
calculation. In this class are executed the guide strategy which
as a string. This makes conceivable to run the program on
is in charge of the mapper work.
various information sorts graphs (i.e. a Node ID might be "1"
E. Reducer as can be "Ram" (user name). In any case the biggest piece of
Reducer Worker is the base reducer class for the projects graph is spoken to by whole number information sorts with
that utilization parallel expansiveness initially seek numerical ID. The calculation was prepared using smallgraph
calculation. It joins the data for aindividual hub. The entire (10 to 20 hubs with a most extreme of 50 edges) so we could
rundown of nearby hubs, the base separation from the source, without much of a stretch check the precision of the outcomes.
the darkest shading, theparent hub of the hub that is being So we attempted on little string, number and both chart. The
handled, and starter rundown are resolved in the reducer step. program worked appropriately and performing.
BFSMpR took the datasets from the SNAP Datasets
Collections [24] that makes accessible an accumulation of

447
IJRITCC | June 2017, Available @ http://www.ijritcc.org
_______________________________________________________________________________________
International Journal on Recent and Innovation Trends in Computing and Communication ISSN: 2321-8169
Volume: 5 Issue: 6 445 449
_______________________________________________________________________________________________
more than 50 huge system datasets from a huge number of CONCLUSION
hubs and edges to countless hubs and edges. In incorporates This paper introduced BFSMpRalgorithm, which is based
informal organizations, web diagrams, street systems, web on Map Reduce programming paradigm. BFSMpR algorithm
systems, reference systems, joint effort systems, and outperforms in terms of time with respect to some of the
correspondence systems. existing sequential mechanisms. There is choice for choosing
Figure.4 shows the relationship of different the key nodes. Hence choosing key nodes plays a very
equivalence estimation in based on the time. Here the trial is important role to the performance of the system. The
done on the Comparison among Sequential (non-Hadoop) and performance of the system depends upon the graph and key
Parallel Hadoop Algorithm. the analysis is done on the nodes size. Algorithm recommends product based on the
Hadoop and the non-Hadoop stage. In the Sequential minimum distance of the product with respect to the user. If
calculation, the program took 40 seconds to finish the user A has brought some items X, Y, Z and user C is close
execution. Be that as it may, on account of the with user A and has minimum distance with A, so user C will
Parallel(Hadoop stage), it took just a few moments for the be likely to recommend product brought by user A.
execution.
REFERENCES
50
[1] J. Dean and S.Ghemawat, MapReduce: Simplified Data
Execution Time(second)

40 Processing on Large Clusters, Commun ACM, vol. 51, no. 1,

pp. 107113, Jan. 2008.
30 [2] J. H. Hsiao, S. J. Kao, A usage-aware
schedulerfor improving MapReduce performance in heterogeneo
20 us environments, on International Conference on Information
Science, Electronics and Electrical Engineering vol 3, pp 1648-
10 1652,2014
[3] Apache-Hadoop.http://hadoop.apache.org/
0
[4] Applications powered by Hadoop.
Sequential Algorithm BSFMpR Algorithm http://wiki.apache.org/hadoop/PoweredBy.
[5] K. Shvachko, H. Kuang, S. Radia, and R. Chansler, The
Figure. 4. Comparison between Sequential and Parallel (BFSMpR)Algorithm Hadoop Distributed File System, in 2010 IEEE 26th
Symposium on Mass Storage Systems and Technologies
Table 1. The performance of job (MSST), 2010, pp. 110.
Test ID Number of Number of Time [6] M. Newman, The Structure and Function of Complex
Key-Nodes Iterations Required Networks, SIAM Rev., vol. 45, no. 2, pp. 167256, Jan. 2003
1 30 10 22 sec [7] Haipeng You, Hui Li, Yunmin Wang, and Qingzhuang Zhao,
2 100 10 32 sec An Improved Collaborative Filtering Recommendation
3 200 10 42 sec AlgorithmCombining Item Clustering and Slope One Scheme,
Proceedings of the International MultiConference of Engineers
and Computer Scientists Vol 1,pp 313-316, 2015 Honkong.
[8] Dalia Sulieman, Maria Malek and Dominique Laurent, Graph
Searching Algorithms For Semantic-Social Recommendation,
IEEE/ACM International Conference on Advances in Social
50
Execution time(second)

Networks Analysis and Mining, 2012

40 [9] Yongchang Wang and Ligu Zhu , Reserch on Collaborative
30 Filtering Recomndantion Algorithem Based On Mahout,
Proceedings of The 4th Intl. conf.on Applied computing and
20 Information Technology, 2016.
10 [10] Dr. Senthil Kumar Thangavel, Neetha Susan Thampi, Johnpaul
C I, PerformanceAnalysis of Various Recommendation
0 Algorithms Using Apache Hadoop and Mahout, International
30 Key 100 key 200 key Journal of Scientific & Engineering Research, Volume 4, Issue
12, December-2013
No. of Key Nodes
[11] G.R.Bamnote and Agrawal,Evaluting and Implementing
collaborative Filtering Sysystems Using Appache
Mahout,International Conference on Computing
Figure. 5. Execution Time for Key-Nodes Communication Control and Automation, 2015.

448
IJRITCC | June 2017, Available @ http://www.ijritcc.org
_______________________________________________________________________________________
International Journal on Recent and Innovation Trends in Computing and Communication ISSN: 2321-8169
Volume: 5 Issue: 6 445 449
_______________________________________________________________________________________________
[12] Breadth-first search, Wikipedia, the free encyclopedia. [19] Brent Smith, Greg Linden;Two Decades of Recommender
[13] Breath First Search - Lecture by Rashid Bin Muhammad Systems at Amazon.com in IEEE Internet Computing Volume:
http://www.personal.kent.edu/~rmuhamma/Algorithms/MyAlgor 21, Issue: 3,pp 12-18,2017.
ithms/GraphAlgor/breadthSearch.html. [20] G. Linden, B. Smith, and J. York, Amazon.com
[14] K. Mehlhorn and P. Sanders, Algorithms and Data Structures: recommendations: item-to-item collaborative filtering, IEEE
The Basic Toolbox. Springer Science & Business Media, 2008. Internet Computing., vol. 7, no. 1, pp. 7680, Jan. 2003.
[15] breadth-first graph search using an iterative mapreduce [21] L. Li, D. Wang, T. Li, D. Knox, and B. Padmanabhan, SCENE:
algorithm. A Scalable Two-stage Personalized News Recommendation
System, in Proceedings of the 34th International ACM SIGIR
http://www.johnandcailin.com/blog/cailin/breadth-first-
Conference on Research and Development in Information
graphsearch-using-iterative-map-reduce-algorithm.
Retrieval, New York, NY, USA,2011, pp. 125134.
[16] hadoop tutorial - Iterative MapReduce and Counters.
https://hadooptutorial.wikispaces.com/Iterative+MapReduce+an [22] B. Sarwar, G. Karypis, J. Konstan, and J. Riedl, Itembased
d+Counters. Collaborative Filtering Recommendation Algorithms, in
Proceedings of the 10th International Conference on World
[17] S. Neumann, Spark vs. Hadoop MapReduce,
Wide Web, New York, NY, USA, 2001, pp. 285295.
Xplenty.https://www.xplenty.com/blog/2014/11/apache-spark-
vs-hadoopmapreduce [23] https://docs.oracle.com/javase/7/docs/api/java/nio/ByteBuffer.ht
ml
[18] Yonathan Portila,Alexandre Reiffers,Eitan Altman,Rachid EI-
Azouzi;A Study of Youtube Recommendation Graph Based On [24] J. Leskovec and A. Krevl, SNAP Datasets: Large Network
Meserurments and Stochastic tools in Proceedings 2015 Dataset Collection, Jun. 2015
IEEE/ACM 8th International Conference on Utility and Cloud
Computing (UCC),pp 430-435,2015

449
IJRITCC | June 2017, Available @ http://www.ijritcc.org
_______________________________________________________________________________________