WEB STRUCTURE
MINING
SUBMITTED BY:
BLESSY JOHN
R7A
ROLL NO:18
INTRODUCTION
Web mining is the application of data
mining techniques in search engines.
Data mining - process of discovering
useful knowledge from data sources
Web mining automatically discover and
extract information from Web documents.
Web structure mining discovers useful
data from hyperlinks.
WEB MINI NG
Useful patterns extraction from WWW
resources
WWW is widely distributed, global
information service centre that
constitutes a rich source for data
mining
Employing techniques from Data
Mining, information retrieval,etc.
NEED FOR WEB MINING
Aims at finding and extracting relevant
information that is hidden in web-
related data.
The challenge is to bring back the
semantics of hyper text document
To turn web data into web knowledge
CLASSIFICATION
WEB MINING
WEB CONTENT
WEB STRUCTURE
MINING WEB USAGE
MINING
MINING
WEB STRUCTURE
MINING
Generate structural summary about the
Web site and Web page
Use graph theory to analyse node and
connection structure of a web site
Analysis of the link structure of the
web, and its purposes is to identify
more preferable documents
WEB STRUCTURE
MINING cont…..
Discovering the nature of the hierarchy
of hyperlinks in the website and its
structure
Hyperlink identifies author’s
endorsement of the other web page
Retrieving information about the
relevance and the quality of the web
page.
Page Layout and Li nk
Analy sis for Web
Images
WEB BASICS
A web is a huge collection of documents
linked together by references.
To refer from one document to another
is based on hyper text and embedded in
HTML
HTML describes how the document
should display on browser window
Web document has a web address
called URL that identifies it uniquely.
WEB CRAWLERS
Collects “all” web documents by
browsing the Web systematically and
exhaustively
Region of the web to be crawled can be
specified by using the URL structure.
Used by a search engine to provide
local access to the most recent versions
of possibly all web pages
INDEXING AND
KEYWORD SEARCH
There are two types of data:
structured and unstructured
Structured data have keys associated
with each data item that reflect its
content
Content-based access to unstructured
data without considering the meaning is
the keyword search approach
DOCUMENT
REPRESENTATION
To facilitate the process of matching
keywords and documents, some
preprocessing steps are taken first:
Documents are tokenized
Characters are converted to upper or
lower case
Words reduced to canonical form
Stopwords are usually removed
ALGORITHMS
There are two main algorithms used in
web structure mining
1. HITS (Hypertext-Induced Topic
Search)
2. Page rank algorithm
HI TS (H yper tex t-In duced Top ic
Searc h)
Link analysis algorithm
Rates web pages
Developed by Jon Kleinberg
Determines two values for a page
Authority-estimates the value of the
content of the page
Hub-estimates the value of its links to
other pages
Hubs a nd Au th or it ies
Hu b pages point to interesting links to authorities = relevant
pages
Au thorit ies are targets of hub pages
Continue……
Authority and hub values are defined in
terms of one another in a mutual
recursion
It is executed at querry time with the
associated HIT on performance
Page R ank
Link analysis algorithm
Assigns a numerical weightage to each
element of a hyperlinked set of
documents
Denoted by PR(E)
Relies on uniquely democratic nature
Link from page A to page B is a vote,
by page A, for page B
Continue…..
Here, A considers itself important and
help to make B important
Also a probability distribution –
represents the probability that a click on
a link arrives at any particular page
Page rank of 0.5 -> 50% chance that a
person clicking on a link will be directed
to the document with the 0.5 page rank
APPLICATIONS
Information retrieval in social networks.
To find out the relevancy of each Web
page
Measuring completeness of the Web
sites
Used in search engines to find out
relevant information
CONCLUSION
Search engines uses web structure
mining to find the information.
We can create new knowledge out of
the available information
Web Content mining can be added to it
to enhance the performance of search
engines.
Thank Yo u !
Questions ?
