Populating the Semantic Web

Kristina Lerman1 , Cenk Gazen2,3 , Steven Minton2 and Craig Knoblock1

1. USC Information Sciences Institute
2. Fetch Technologies
3. Carnegie Mellon University
{lerman,knoblock}@isi.edu
{gazen,minton}@fetch.com

Abstract Web wrappers were developed to facilitate the extrac-

tion of structured information from Web sources (Muslea,
The vision of the Semantic Web is that a vast store of Minton, & Knoblock 2001; Kushmerick, Weld, & Dooren-
online information “meaningful to computers will un-
leash a revolution of new possibilities”. Unfortunately,
bos 1997; Freitag & Kushmerick 2000; Cohen & Jensen
the vast majority of information on the Web is formatted 2001). However, these methods are a poor choice from the
to be easily read by human users, not computer applica- point of view of automatically creating semantically mean-
tions. In order to make the vision of the Semantic Web a ingful content. Wrappers are notoriously brittle — any
reality, tools for automatically annotating Web content change in the source requires a new wrapper to be gener-
with semantic labels will be required. We describe the ated. A bigger problem is that wrapper creation demands a
ADEL system that automatically extracts records from significant amount of user intervention — in order to learn
Web sites and semantically labels the fields. The sys- the correct wrapper, the user must provide the data schema
tem exploits similarities in the layout of Web pages in and example records to the wrapper induction system. Be-
order to learn the grammar that generated these pages. cause user involvement is so costly, the focus recently has
It them uses this grammar to extract structured records
from these Web pages. ADEL system also exploits the
been on minimizing the number of examples the user has to
fact that sites in the same domain will provide the same, label (Muslea, Minton, & Knoblock 2002). Still, even when
or similar data. By collecting labeled examples of data user effort is significantly reduced, the amount and the rate
during the training stage, we are able to learn structural of growth of information on the Web will quickly require
descriptions of data fields and later use these descrip- fully automatic solutions.
tions to semantically label new data fields. We show Fortunately, on-line information sources contain much
that on a Used Car shopping domain, ADEL achieves explicit and implicit structure, both in layout and content,
precision of 64% and recall of 89% on extracting and that can be exploited for the purposes of automatic infor-
labeling data columns.
mation extraction. For example, Web sites that dynami-
cally generate pages from electronic databases in response
Introduction to user queries, e.g., on-line catalogs, directories, are highly
uniform, both in terms of how the information is orga-
The vision of the Semantic Web is that a vast store of online nized, visually presented and the type of information that
information “meaningful to computers will unleash a revolu- is displayed. Recently, a number of approaches (Lerman,
tion of new possibilities” (Berners-Lee, Hendler, & Lassila Knoblock, & Minton 2001; Crescenzi, Mecca, & Merialdo
2001). Unfortunately, the vast majority of information on 2001b; Arasu & Garcia-Molina 2003) have focused on ex-
the Web is formatted to be easily read by human users, not ploiting structure within a page in order to automatically ex-
computer applications. Semantic markup, such as XML and tract records from it.
DAML (DAML Project ), will enable the exchange of in-
formation between applications; however, only a tiny frac- The difficulties inherent in automatic record extraction are
tion of the Web pages are currently annotated with seman- even more pronounced in the problem of semantically label-
tic markup. This situation in not likely to improve soon, ing the extracted fields. Despite the Web wrappers’ long
because there is little incentive for information providers to track record, automatic labeling of extracted data has only
convert existing content to the new standard. Moreover, even recently begun to be addressed (Arlotta et al. 2003). The
where semantically annotated sources exist, not all informa- current paper describes a domain independent approach to
tion providers will conform to the same schema. Clearly, automatically extracting records from Web sites and seman-
in order to make the vision of the Semantic Web a reality, tically labeling the fields, or mapping them to a schema. The
tools for converting regular Web content into semantically work is built on our previous research in automatic genera-
annotated one will be required. tion and maintenance of Web wrappers (Lerman, Minton, &
Knoblock 2003). Our starting assumption is that content on
Copyright
c 2004, American Association for Artificial Intelli- different Web sites in the same domain contains many simi-
gence (www.aaai.org). All rights reserved. larities: e.g., used car sites generally give the year, make and
model of the car, as well as its mileage and price. Additional 2003), where the object is to produce semantic mapping to
information, such as the color and engine specifications may map instances of data from one scheme to another. This is an
also be given. The data fields will have a similar format on important task in information integration, where queries that
different sites within the domain. If we learn the representa- are expressed in some common language have to be trans-
tion of fields on one site, we can apply it to label content on lated to the local schema of the database before they can be
other sites within the domain. In order to automatically ex- submitted to the database. Past works on schema match-
tract records from a site, we exploit similarities in the layout ing (Li & Clifton 2000; Doan, Domingos, & Halevy 2001;
of data pages. Our results on the Used Car domain indicate 2003) included machine learning techniques that learn to
our approach is a feasible method for automatically extract- classify new object instances based on features that include
ing and labeling data. local schema names and content features. The content fea-
tures used in these works are global in nature, such as word
Related Work frequencies and format. Our approach, on the other hand,
uses finer-grained descriptions enabled by the use of patterns
Researchers have addressed the problem of extracting data to describe the structure of data.
from Web tables; however, very little work on the semantic
labeling of the extracted content has been made. At the other
end of the spectrum are the database and information inte-
Automatic Data Extraction and Labeling
gration researchers, who done extensive work on the prob- The architecture of the Automatic Data Extraction and La-
lem of schema matching and integration. This work is simi- beling (ADEL) system is shown in Figure 1. We will use
lar to our research as discussed below. the Used Cars shopping site as the running example. The
Existing approaches to extracting data from Web tables training stage consists of background knowledge acquisi-
can be classified as heuristic (Chen, Tsai, & Tsai 2000) or tion, where we collect data in a particular domain, e.g., Used
machine learning (Borkar, Deshmukh, & Sarawagi 2001; Cars, and learn a structural description of it. In order to col-
Hurst 2001; Wang & Hu 2002). Heuristic approaches lect a large number of examples, we created wrappers for
to detect tables and record boundaries in Web documents some sites in the domain. We specified the schema of the
by using the DOM or domain-specific rules features such data and labeled example records on several pages. We used
as percentages, date/time, etc. (Chen, Tsai, & Tsai 2000) the wrapper building tool developed by Fetch Technologies.
to identify tables. Machine-learning approaches learn a It provides a GUI to simplify data entry and uses the wrap-
model of data from a set of labeled training examples us- per induction system (Muslea, Minton, & Knoblock 2001)
ing hand-selected features. These include Hidden Markov- to learn the correct wrapper extraction rules.
based probabilistic models (Borkar, Deshmukh, & Sarawagi Next, we learn a description of the data fields. We repre-
2001), Naive Bayes classifiers (Hurst 2001), and domain- sent the structure of a data field by sequences of tokens and
independent classifiers that use non-text layout (average token types, which we call patterns. We use the DataPro
number of columns/rows, average cell length and consis- algorithm (Lerman, Minton, & Knoblock 2003) to learn pat-
tency) and content features (image, form, hyperlink, alpha- terns from examples collected by the wrappers, as described
betic, digit, others) (Wang & Hu 2002). These works require in Section Modeling Data Content.
many training examples in order to identify tables on HTML The second stage of ADEL consists of analyzing HTML
pages, and in a few cases correctly extract data from them. pages from a new site in the domain, extracting records
No attempt is made in these works to assign labels to data. from them and labeling fields. Data intensive Web sites
Researchers made several attempts to exploit the lay- have a surprisingly uniform structure that we can exploit for
out of Web pages for the purpose of automatically extract- the purpose of automatic information extraction. The entry
ing data from them (Crescenzi, Mecca, & Merialdo 2001b; point is an index page or HTML form for the user to in-
2001a; Arasu & Garcia-Molina 2003). The RoadRunner put her query. The result of the query is a list of items or a
system (Crescenzi, Mecca, & Merialdo 2001b; 2001a) is an collection of records from a database. The results are usu-
example of such a system. The premise behind RoadRunner ally displayed as a list or a table on an automatically gener-
is that many Web pages are generated by a grammar, which ated page. We call such a results page the list page. Each
can be inferred from example pages. Thus, RoadRunner can item or record often has a link to a detail page that con-
learn the table template and use it to automatically extract tains additional information about that entry. Detail pages
data from the Web site, as we do. RoadRunner’s focus is on are also generated automatically and populated with results
a subclass of Web pages that can be generated by union-free of database queries. We envision that the user will provide
grammar. The learning algorithm is exponential, and fur- a pointer to the top-level page — index page or a form —
ther simplifications are necessary to keep it computationally and the system will automatically navigate the site, retriev-
tractable. Disjunctions are necessary to represent alterna- ing all pages. Currently, ADEL is not able to process forms;
tive layout instructions often used by Web sites for a same instead, we manually retrieve list pages and provide them to
field. In addition to extracting data, RoadRunner researchers the system.
have made efforts to extract column labels from HTML ta- The next step is to extract data from pages. The pages
bles (Arlotta et al. 2003). have to be classified into separate types, such as list or detail
Our work is very similar to schema matching or integra- pages, before data can then be extracted from pages of a
tion (Rahm & Bernstein 2001; Doan, Domingos, & Halevy single type. In the experiments described in this paper, we
 Year: Bodystyle:
Collecting background knowledge 2002 2 dr coupe
1998 4 dr ALPHA
… …
Make: Mileage:

Labeled ford
lincoln
20, 3DIGIT
31, 3DIGIT
wrapper examples DataPro Patterns ALPHA
…
2DIGIT, 3DIGIT
…

Labeled label Extracts Autowrap

examples

NULL Model Color Bodystyle Mileage

2000 acura
1996 acura
integra gs
integra ls
silver
purple
2 dr hatchback
4 dr sedan
36 , 866
62 , 723
Extracting
2002 buick
1998 chevrolet
century custom
astro
white
orange
4 dr sedan
mini van
31 , 838
83 , 604
and labeling
2001 chevrolet camaro blue 2 dr coupe 22 , 249
2003 chevrolet impala blue 4 dr sedan 27 , 738

Figure 1: Architecture of the Automatic Data Extraction and Labeling system

skip this step and provide several pages of the same type “number followed by two capitalized words followed by the
to the algorithm. Autowrap, the automatic data extraction word Blvd,” etc.
algorithm is described in Section Data Extraction. The symbolic representation of content by patterns of to-
Finally, we use the patterns learned on the training exam- kens and token types is very flexible and general. In our
ples to assign semantic labels to the automatically extracted previous research we found that a set of patterns describ-
records, as described in Section Labeling. We validated the ing how a field begins and ends, allowed us monitor wrap-
ADEL system on the Used Cars domain. Results of this per’s accuracy or locate examples of the field on new pages
work are presented in the results section. with considerable accuracy (Lerman, Minton, & Knoblock
2003). In Section Labeling we show how to apply patterns
Modeling Data Content to recognize known data fields.
The data modeling step is used to learn the structure of
data fields from examples. We represent the structure of Data Extraction
data by patterns of tokens and token types. In previous As we discussed above, many Web sites that present infor-
work, we developed a flexible pattern language and pre- mation contained in databases follow a de facto convention
sented an efficient algorithm, DataPro, for learning pat- in displaying information to the users and allowing them to
terns from examples of a field (Lerman & Minton 2000; navigate it. This convention affects how the Web site is orga-
Lerman, Minton, & Knoblock 2003). The pattern language nized, and gives us additional information we can leverage
contains specific tokens and general token types. Specific for information extraction. Such Web sites generate list and
types refer to unique text strings, such as “California” or detail pages dynamically from templates and fill them with
“sea”, while the general types describe the syntactic cat- results of database queries.
egory to which the token’s characters belong, such as nu- Consider a typical list page from a Web site. As the server
meric, alphabetic, etc.The token types are organized in a hi- constructs the page in response to a query, it generates a
erarchy, which allows for multi-level generalization.1 The header, followed in many cases by an advertisement, then
pattern language can be extended to include other syntactic possibly a summary of the results, such as “Displaying
types or domain-specific semantic types. In addition to pat- 1-10 of 214 records.”, table header and footer, fol-
terns, we also remember the mean length of a field and its lowed by some concluding remarks, such as a copyright in-
variance. formation or navigation aids. We call this part of the page
DataPro algorithm finds patterns that describe many of the page template. The page template of a list page contains
the examples of a field and are highly unlikely to describe data that is shared by all list pages and is invariant from page
a random token sequence. As an example, names can be to page. The page template can also be thought of as the
represented as a set of patterns such as “capitalized word grammar that generates the pages. Given two, or preferably
followed by an initial” and “capitalized word followed by a more, example list pages from a site, the Autowrap algo-
capitalized word,” whereas addresses can be represented as rithm can derive the grammar used to generate the pages and
1 use it to extract data from them.
A text token is a punctuation mark (PUNCT) or an alphanu-
meric token (ALNUM). If it is alphanumeric, it could be alphabetic Autowrap uses an ad hoc method to induce a grammar for
type (ALPHA) or a number (NUMBER). If alphabetic, it could also the Document Object Model (DOM) trees of the pages. The
be a capitalized word (CAPS) or an all-capitalized word (ALL- kinds of grammars it induces allow us to extract single data
CAPS). The number category is further subdivided into 1DIGIT to items as well as lists, where the rows of a list contain data
5DIGIT numbers. items or nested lists.
 r r
children of r (in the intermediate tree). With repeating sub-
a f a f v structures represented as templates, the algorithm can detect
b c e g s b t e u the similarity between the two rows of the outer list and find
d
a row template for the children of r: r([a([b(c)]*) d]*).
d
The second step of the induction algorithm merges the
templates that are induced for each DOM tree. The merging
Figure 2: Example trees algorithm is the same as the template-finding algorithm, ex-
cept the row templates are treated slightly differently: The
order of the nodes within row templates may be rotations
of one another even though the lists represented by the tem-
The induction algorithm has two main stages: Finding re-
plates are very similar. For example, r(a b c a b c a b c)
peating sub-structures and merging grammars into a more
and r(b c a b c a b c) will give r([a b c]*) and r([b c a]*)
general one. Both stages use ideas based on templates.
as templates. The merging algorithm treats this special case
A template is a sequence of alternating slots and stripes and chooses the best alignment among the rotations.
where the stripes are the common sub-structures among all
the pages and slots are the placeholders for pieces of data Labeling
that go in between the stripes. One way to find the tem- When provided with several list pages, Autowrap extracts all
plate of a set of pages is to find the longest common subse- tables from these pages. These include the one with the data
quence (LCS) of all the pages. The LCS immediately gives we are interested in extracting, as well as tables containing
the stripes of the template and with a little bookkeeping, the extraneous information. The next step in the process is to
slots can also be found. label the columns of every table and output the correct table
The template idea can easily be extended to trees, and in of data, which we define to be one that has the most labeled
particular to the DOM structure. Given a set of sequences columns.
of DOM elements, we find the LCS and then for each ele- We use learned patterns to map columns to data fields.
ment in the LCS, we recursively apply the algorithm to the The basic premise is to check how well a field describes
set of child elements. For example, consider two trees r(a(b a column of data, given a list of patterns that describe the
c(d) e) f(g)) and r(a(s b t(d) e) f(u) v) where the parenthe- data field and its mean length. We have developed a set of
sis group the children of the preceding element, as shown heuristics to score how well a field describes a column. The
in Figure 2. First we find the LCS of the single-element column is assigned the field with the highest score. Factors
sequences [r] and [r] and then proceed down to the child that increase a field’s score include
sequences [a f] and [a f v]. The LCS of these two is [a f],
so we first recurse down to the child sequences of the two a • Number of patterns that match examples in the column
nodes and then to those of the two f nodes to get [b e] and • How close examples are in length to the field’s mean
[]. Since there are no more child sequences, we combine the length
LCS’s to get a tree template: r(a(b e) f()). • Pattern weight — where the more specific patterns are
Once we have the template, we can use it to extract the given higher weight
slots by finding the nodes in the original DOM structures
that are not in the stripes of the template. This gives us .(.(. Results
c(d) .) .(g)) and .(.(s . t(d) .) .(u) v). With a little more work, We validated the ADEL system on the Used Cars domain.
we can align the data in columns and represent the data in a We wrapped two used cars sites — Anaheim Lincoln Mer-
table: cury and Mercedes Benz of Laguna Niguel — and collected
c(d) g on the order of 250 records from these sites. We normalized
s t(d) u v all data by lowercasing it. We then ran the DataPro algo-
The template can also be used as a similarity measure be- rithm on the records to learn descriptions of the fields. The
tween two DOM sub-structures, since similar structures will resulting patterns and field lengths are displayed in Table 1.
have templates with bigger stripes than those that are not Fields Mileage and Price had many specific patterns that we
similar. The first step of the induction algorithm uses this do not display in the table.
similarity measure to find repeating sub-structures among Next, we attempted to extract and label data from three
the children of a DOM node. In particular, the algorithm new sites in the Used Cars domain.
looks for consecutive sequences of nodes whose similarity We manually collected three list pages from each new site.
measure is above a threshold. The sequences, which are as- Autowrap automatically induced the template for each set
sumed to be the rows of a list, are used to induce a row tem- of three pages and extracted all data from them. Autowrap
plate. As the algorithm traverses up from the leaves towards found, on average, six tables of data on each page. These ta-
the root, it replaces the repeating sub-structures with the row bles were processed further. Again, we normalized data by
templates. For example, starting with the tree r(a(b(c 1) b(c lowercasing it. In addition, we fixed some of the segmen-
2)) d a(b(c 3) b(c 4) b(c 5)) d), the algorithm first finds row tation errors Autowrap made. For instance, Autowrap does
templates for the inner lists within the children of a nodes: not recognize the sequence “\\r\\n” or a comma as a field
r(a([b(c)]*) d a ([b(c)]*) d) where the notation [...]* rep- delimiter. Thus, “Marina del Rey, CA” is extracted as a sin-
resents a row template. Next, the algorithm examines the gle field, rather than two. These problems will eventually be
Year Color Bodystyle
< 1.0 ± 0.0 > < 1.12 + −0.32 > < 3.06 ± 0.24 >
sion increases to P = 0.80.
[1999] [smoke silver] [4 dr sedan]
[2002] [desert silver] [4 dr A LPHA A LPHA]
[2000] [silver] [2 dr coupe] Discussion
[2003] [black]
[2001] [grey] We have described the ADEL system that automatically ex-
[4D IGIT] [A LPHA] tracts data from HTML pages and labels it. Although the
Make Mileage Engine system is still under development, initial results show good
< 1.31 ± 0.46 > < 2.38 ± 0.92 > < 3.83 ± 0.38 > performance on test sites in the Used Cars domain.
[mercedes, benz] [47K] [5 . 0l v8]
[ford] [21 , 3D IGIT] [4 . 3l v8] ADEL can extract data from HTML pages containing sin-
[mercury] [20 , 3D IGIT] [3 . 2l 6cyl] gle records of data, lists of records, and even nested lists. It
[lincoln] [2D IGIT , 3D IGIT] [3 . 2l A LNUM] does so by inducing the grammar of the pages and using the
Model Price [2 . 3l] grammar to extract data. In order to label the data, we first
< 1.88 ± 0.76 > < 4.0 ± 0.0 > [2 . A LNUM] need to learn the structural descriptions of the data fields.
[s430] [$ 2 , 988] VIN We do this by accumulating labeled data from some sites in
[ranger 2wd] [$ 25 , 3D IGIT] < 1.0 ± 0.0 > the domain, and used the learned descriptions to label data
[mountaineer A LNUM] [$ 15 , 988] [A LNUM]
from new sites in the same domain.
[gr marquis ls] ...
[ls v6] [$ 29 , 3D IGIT] ADEL system has shown good performance on labeling
[ls v8] [$ 30 , 988] data on three new Web sites in the Used Cars domain. Of the
[navigator 2wd] [$ 31 , 3D IGIT] 27 columns of data extracted from pages in these sites, 19
[navigator A LNUM] [$ 2D IGIT , 995] were correctly labeled and 2 were not labeled by the system.
[A LPHA lx] [$ 2D IGIT , 990] Five of the incorrectly labeled columns contained data that
[A LPHA A LPHA] [$ 2D IGIT , 900] was not in the schema we created for the site, such as “Trans-
mission”, “Stock Number”, “Warranty” and “Dealer”. We
can expand the schema by allowing user to specify patterns
Table 1: Patterns learned for data fields in the Used Cars describing new data fields. We did this for the field “Trans-
domain mission.” The patterns for the field were [automatic]
and [5 speed manual]. With the manually coded pat-
terns, we were able to label the transmission field on the
fixed in the Autowrap source. For now, we manually seg- site. Our algorithm also consistently mislabeled the “Price”
mented fields on “\\r\\n” and “,” (except for numbers, in field. This was because without the $, “Price” and “Mileage”
which case we did not segment them on the comma). fields are very similar (two digit number followed by a three
The columns were scored against the patterns in Table 1 digit number). Autowrap did not always extract the $ in the
according to the criteria described in Section Labeling. The price field.
column was assigned the label of the highest scoring field. The most significant challenge for automatic labeling ap-
The table with the most labeled columns was output in pears to be inconsistent data formats. For example, we failed
comma-delimited spreadsheet format. Table 2 shows a sec- to correctly label the “Engine” field in Table 2 because is ap-
tion of a table of labeled data for one site. peared on this site as “3.2 liter 6 cyl.”, whereas we learned a
In all, Autowrap extracted 27 columns of data from the description from sites that described engines as “3.2l 6cyl.”
three sites. In one site, the algorithm made a mistake in the Likewise, some sites presented Body Style as “4dr sedan”,
list template, resulting in it improperly concatenating every while others as ”sedan, 4dr”. Conceivably, we could include
two rows, thereby producing a table with 8 columns, rather such data transformation rules into the labeling algorithm,
than 4. The problem of correct list segmentation is addressed thereby significantly improving performance of the system.
in a different work (Lerman et al. 2004), which shows how
Unlike (Arlotta et al. 2003), we do not rely on the column
to improve Autowrap’s results with additional information
labels provided in the HTML page to label the schema. Even
from detail pages. In this work we are concerned with label-
though each site in the Used Cars domain provides similar
ing columns rather than extracting structured records.
information, the schema labels used were not always consis-
Nineteen of the 27 columns were correctly labeled, 9 were
tent with other sites. For example, one site applied the label
incorrectly labeled, and two were unlabeled, resulting in pre-
“Vehicle” to data composed of fields “Year Make Model”,
cision and recall of P = 0.64 and R = 0.89.2 However,
and “Information” for all the details about the car, such as
five of the columns that were incorrectly labeled were not in
body style, color, engine, etc. Other differences included us-
the schema we created: e.g., , “Transmission”, “Stock Num-
ing “MSRP” rather than “Price” and “Exterior” rather than
ber”, “Warranty” and “Dealer” in Table 2. When these are
“Color.” A scheme based on extracting column labels from
excluded from the incorrectly labeled columns count, preci-
HTML tables will not allow one to recognize that a field
2
We define True Positives (TP) as correctly labeled columns; “MSRP” on one site provides the same information as the
False Positive (FP) as incorrectly labeled columns; and False Neg- field “Price” on another site. Reading column labels will be
atives (FN) as unlabeled columns; therefore, P = T P/(T P +F P ) useful for augmenting the schema by discovering new fields
and R = T P/T P + F N . and labels.
 - Model Color Color Bodystyle - VIN Mileage Mileage Model
1998 acura 2 . 5tl automatic silver 4 dr sedan 2.5 liter 5 cyl. jh4ua265xwc007394 60 , 913 14 , 976 norm reeves
2001 acura 3 . 2cl automatic red 2 dr coupe 3.2 liter 6 cyl. 19uya42741a003823 76 , 674 19 , 934 norm reeves
2000 acura 3 . 2tl automatic white 4 dr sedan 3.2 liter v-6 19uua5668ya054953 41 , 421 19 , 595 norm reeves
2000 acura integra gs 5 speed manual silver 2 dr hatchback 1.8 liter 4 cyl. jh4dc4368ys014324 36 , 866 16 , 895 norm reeves
1996 acura integra ls 5 speed manual purple 4 dr sedan 1.8 liter 4 cyl. dohc jh4db7553ts011459 62 , 723 9 , 595 norm reeves
1998 chevrolet astro automatic orange mini van 4.3 liter v-6 1gndm19w0wb180216 83 , 604 8 , 903 cerritos ford
2001 chevrolet camaro automatic blue 2 dr coupe 3.8 liter 6 cyl. 2g1fp22k412109568 22 , 249 13 , 975 norm reeves

Table 2: Results

Acknowledgements Doan, A.; Domingos, P.; and Halevy, A. 2003. Learning to

This material is based upon work supported in part by match the schemas of databases: A multistrategy approach.
the National Science Foundation under Award No. IIS- Machine Learning Journal 50:279–301.
0324955, in part by the Defense Advanced Research Freitag, D., and Kushmerick, N. 2000. Boosted wrapper in-
Projects Agency (DARPA), through the Department of the duction. In Proc. of the 7th Conf. on Artificial Intelligence
Interior, NBC, Acquisition Services Division, under Con- (AAAI-2000), 577–583.
tract No. NBCHD030010, in part by the Air Force Office of Hurst, M. 2001. Layout and language: Challenges for table
Scientific Research under grant number F49620-01-1-0053, understanding on the web. In In Web Document Analysis,
in part by the United States Air Force under contract number Proc. of the 1st Int. Workshop on Web Document Analysis.
F49620-02-C-0103, and in part by a gift from the Microsoft
Kushmerick, N.; Weld, D. S.; and Doorenbos, R. B. 1997.
Corporation.
Wrapper induction for information extraction. In Proc. of
the Intl. Joint Conf. on Artificial Intelligence, 729–737.
References
Lerman, K., and Minton, S. 2000. Learning the common
Arasu, A., and Garcia-Molina, H. 2003. Extracting struc-
structure of data. In Proc. of the 15th National Conf. on
tured data from web pages. In Proc. of the ACM SIGMOD
Artificial Intelligence (AAAI-2000).
Int. Conf. on Management of Data.
Arlotta, L.; Crescenzi, V.; Mecca, G.; and Marialdo, P. Lerman, K.; Getoor, L.; Minton, S.; and Knoblock, C. A.
2003. Automatic annotation of data extracted from large 2004. Using the Structure of Web Sites for Automatic Seg-
web sites. In Proc. of the Sixth Int. Workshop on Web and mentation of Tables. to appear in Proc. of the ACM SIG-
Databases (WebDB03). MOD Int. Conf. on Management of Data.
Berners-Lee, T.; Hendler, J.; and Lassila, O. 2001. The Lerman, K.; Knoblock, C. A.; and Minton, S. 2001. Auto-
semantic web. Scientific American. matic data extraction from lists and tables in web sources.
In Proc. of the workshop on Advances in Text Extraction
Borkar, V.; Deshmukh, K.; and Sarawagi, S. 2001. Auto- and Mining (IJCAI-2001). Menlo Park: AAAI Press.
matic segmentation of text into structured records full text.
In Proc. of the 2001 ACM SIGMOD Int. Conf. on Manage- Lerman, K.; Minton, S.; and Knoblock, C. 2003. Wrapper
ment of data, Special Interest Group on Management of maintenance: A machine learning approach. Journal of
Data, 175–186. ACM. Artificial Intelligence Research 18:149–181.
Chen, H.; Tsai, S.; and Tsai, J. 2000. Mining tables from Li, W., and Clifton, C. 2000. Semint: A tool for identifying
large scale html texts. In 18th Int. Conf. on Computational attribute correspondence in heterogeneous databases using
Linguistics (COLING). neural networks. Data and Knowledge Engineering 33:49–
84.
Cohen, W., and Jensen, L. 2001. A structured wrap-
per induction system for extracting information from semi- Muslea, I.; Minton, S.; and Knoblock, C. A. 2001. Hi-
structured documents. In Proc. of the IJCAI Workshop on erarchical wrapper induction for semistructured informa-
Adaptive Text Extraction and Mining. tion sources. Autonomous Agents and Multi-Agent Systems
4:93–114.
Crescenzi, V.; Mecca, G.; and Merialdo, P. 2001a. Au-
tomatic web information extraction in the ROADRUNNER Muslea, I.; Minton, S.; and Knoblock, C. 2002. Active +
system. In Proc. of the Int. Workshop on Data Semantics semi-supervised learning = robust multi-view learning. In
in Web Information Systems (DASWIS-2001). Proc. of the 19th Int. Conf. on Machine Learning (ICML
2002), 435–442. Morgan Kaufmann, San Francisco, CA.
Crescenzi, V.; Mecca, G.; and Merialdo, P. 2001b. ROAD -
RUNNER: Towards automatic data extraction from large DAML Project. http://www.daml.org.
web sites. In Proc. of the 27th Conf. on Very Large Rahm, E., and Bernstein, P. 2001. On matching schemas
Databases (VLDB). automatically. VLDB Journal 10(4).
Doan, A.; Domingos, P.; and Halevy, A. Y. 2001. Rec- Wang, Y., and Hu, J. 2002. Detecting tables in html docu-
onciling schemas of disparate data sources: A machine- ments. In Fifth IAPR Int. Workshop on Document Analysis
learning approach. In Proc. of the ACM SIGMOD Int. Conf. Systems, Princeton, New Jersey.
on Management of Data, 509–520.