A semantic approach to cross-document person profiling in Web

Abstract

The problem of cross-document person profiling aimed at identifying and linking person entities across Web pages and extracting their relevant structured information. In this paper, we specifically focus on the core task of person profiling problem, namely the attribute extraction task. For attribute extraction, the existing approaches face several challenges that two important of them include (i) syntactic and structure variation, and (ii) cross-sentence and cross-document information extraction. To alleviate these deficiencies and improve performance of existing methods, we propose a semantic attribute extraction approach relying on probabilistic reasoning. Our approach produces structured, meaningful profiles in which the resulting textual facts are linked to their possible actual meaning in a distant ontology. We evaluate our approach on standard profile extraction datasets. Experimental results demonstrate that our approach achieves better results when compared with several baselines and state of the art counterparts. The results justify that our approach is a promising solution to the problem of person profiling.

Keywords

Web mining information extraction cross-document person profiling attribute extraction

1. Introduction

Currently, people can easily publish data on the Web and various social media. This results a vast amount of valuable data, which a significant part of them are unstructured, free text documents written in various natural languages. These data can include information about persons, locations, organizations, governments, facilities, vehicles and many other entities. The huge volume of data on the Web and social media brings several challenges such as lost in hyperspace and information overload [13]. One of the potential solutions to the chaos of Web is to transform unstructured data into structured format, which would be helpful to provide machine-readable knowledge bases, and hence ultimately improving application interoperability, data integration, sharing, reusing and availability. Entity profiling (EP) as a subtask of information extraction (IE), Web mining, and natural language processing (NLP), offers a promising way to map free-structured Web text documents into structured and machine-readable knowledge bases.

In this article, as an element of EP research, we investigate a specific variant of the general EP problem, namely the cross-document person profiling in Web. In cross-document person profiling, we are given a set of Web pages $D = {d_{1}, d_{2}, \dots, d_{N}}$ referring to different people sharing the same or different name. The task is to first cluster Web pages into sets of disjointed clusters $C = {C_{1}, C_{2}, \dots, C_{K}}$ , and then extract profile of each given person from those Web pages within each cluster $C_{i} \in C$ . For this practical task, two important challenges need to be solved: (i) name ambiguity and (ii) attribute extraction (AE). In this article, we chose not to tackle name ambiguity and specifically spent our energy on developing a robust AE system.

Many research efforts have been made to solve the problem of Web person AE. The existing approaches suffer from several fundamental problems. The first is the lake of appropriate text processing tools specific for Web data. Many of the available text processing tools are often developed for homogeneous corpus, particularly news corpora, and their adoption to Web data is not a trivial task. The second problem is that many existing AE approaches mainly focused on homogeneous unstructured-style news corpora [44] or the Web pages that are in the same style (e.g., Wikipedia articles or individual homepages) [3,61,66,67,74], and their adoption for general Web data is so challenging. The third is the problem of structure and syntactic variation, presenting the same meaning with different surface linguistic expression forms in different structures. Web pages use boundless vocabulary and structure, and composition style to present approximately similar content [1]. This makes it hard for any Web AE to cover all writing pattern and structure variations. The fourth is the implicit information scattered within and across the sentences and even documents. Most of the existing AE systems fail to extract this sort of information as they focused on extracting the information existing only within a sentence. These AE approaches use different constraints and filtering techniques to improve the performance. However, these constraints are executed in several different stages and cannot be applied collectively. The fifth and perhaps the most important problem is the low performance of exiting Web person AE methods. These challenges show that the existing solutions are not enough for Web data. These observations promoted us to developing a semantic AE approach to alleviate the deficiencies of structure and syntactic variation, and cross-sentence and cross-document IE. Our approach improves the performance of AE compared to the counterparts.

To summarize, our contributions in this article are as follows:

For Web person AE, we propose a semantic approach, which is effective for rich profile IE from various kinds of Web pages, where they are in various domains, may contain a lot of noise, and written in different styles and structures. Our approach is able to extract explicit and implicit structured information within and across sentences in a collective manner by utilizing Markov logic networks (MLNs) [58]. Our AE approach links the resulting textual surface facts to their possible actual meaning in a distant ontology. By this way, the majority of information contained in a person profile is meaningful, and can simply translate to other languages. This is very helpful for multi-lingual text processing.

In AE stage, to recognize class of attributes, most previous AE work used pre-compiled manually collected seed keywords and verbs. We propose an automatic supervised method to extract the seed keywords and verbs by utilizing PMI weighting schema [25] and a co-reference resolution system. This is efficient in practice and alleviates the need for manual human engineering effort.

We perform extensive evaluation of our proposed approach by using real, standard datasets. We show that our method outperform baseline approaches and state of the art counterparts.

Having this short introduction, the rest of this paper continues as follows. Section 2 is devoted to literature review, and presents an overview of the related work of attribute extraction. Section 3 describes the working principle of our approach. Then, in Section 4, the proposed algorithm is evaluated on benchmark datasets and the results are compared to the baseline methods and state of the art approaches. Finally, Section 5 makes conclusions and discusses some future works.

2. Related work

An important task in cross-document entity profiling is AE. In the following, we review the most significant research work for AE, present their limitations and compare our approach with them. This short discussion highlights the need for developing new and more robust AE approaches.

The task of AE is to distil valid filler values for a set of pre-defined attributes of a given entity from text documents. This task is also known as slot filling [66] or relation extraction [10]. In recent years, many attempts have been made to solve the AE problem. Early AE systems were based on lexico-syntactic rule-based methods and are domain dependent. For example, Li et al. [44] used a multi-level rule-based approach, which relies on various linguistic IE patterns to extract relations from English corpora. A similar approach is presented in [33] for AE in Frasi texts. However, such approaches to IE are limited by the availability of domain knowledge, the difficulty in designing rules for different types of text, and less accurate results under noisy setting. Moreover, these approaches focused on extracting attributes only within a sentence, which is not enough for Web text as some relations appear across sentences. Later systems to achieve robustness under noisy setting and to extract arbitrary facts use probabilistic [43] and statistical methods [63,76]. However, these approaches ignore deep semantic analysis of text and do not exploit entirely the semantic information contained in text. Some other work used machine learning methods for Web AE. Supervised learning methods achieved high performance in AE, but they need more hand labeled training data in order to be effective [10]. Due to the lacking of high quality large annotated training data, and the low performance of supervised methods for extracting arbitrary relations from large-scale corpora such as Web, recent AE systems used semi-supervised learning methods [10,64], bootstrapping methods [70], self-supervision methods [69], distant supervision methods [51], and unsupervised clustering methods [31]. However, each of these methods suffers from several challenges. For example, bootstrapping methods suffer from semantic drift problem, and distant supervision methods suffer from noisy training data. The output of unsupervised clustering methods often does not resemble ontological relations, and the resulting relations are difficult to map into a domain-specific ontology.

In contrast to traditional ontology-based AE, Open IE [12] as a new emerging trend in IE aims to extract possible relations in a text without a pre-specified attribute vocabulary and with no domain-specific knowledge engineering effort. The main issue in Open IE is that the extractions are purely surface text and do not resemble domain-specific ontological relations [62]. In recent years, a few studies [60–62,71] have been done to adapt Open IE extractions to pre-specified ontological relations. Soderland et al. [62] propose a two-step approach for adapting Open IE extractions to a domain ontology. In the first stage, the Open IE tuples are annotated by a domain concept recognizer, and then a number of relation-mapping rules are learned by using a cover learning algorithm to map the tuples to domain relations. Since machine learning approaches to learning high precision mapping rules need more training data, and such a high volume data are not available, the authors in [60] chose to create domain mapping rules manually rather than adopting machine learning approaches. In ImplIE [61], syntactic dependency rules are used to find relations that are beyond the scope of Open IE extractions. Our approach draws on the idea of using dependency parsing for extracting noun-based relations. We extend their approach through enhancing the syntactic dependencies with semantic information obtained from a distant ontology to alleviate the errors produced by syntactic parser. In addition, we exploit co-reference information to make the extractions more precise. Exploiting syntactic dependencies for relation extraction is not a new idea and studied in early work. For example, [16] formulates the relation extraction as the problem of finding the shortest paths between entities on dependency graph.

Some other work rely on shallow semantic analysis of text [34,37,65]. For example, Surdeanu et al. [65] proposed a rule-based approach, which contains a number of mapping rules to map semantic role (SRL) frames to domain-specific attributes. However, the previous work relying on shallow semantic analysis have not addressed entirely some challenging linguistic phenomena such as synonymy and polysemy, and these approaches detect relations existing only within a sentence, which is not enough for Web data and may lead to reduced performance.

More recently, some work integrate syntactic dependencies and semantic information derived from distant knowledge bases to address the challenges like synonymy and polysemy. For example, Moro and Navigli [52] combined syntactic dependencies and distributional semantic information to extract ontological relations. However, the resulting relations are still bound to surface text, lacking actual semantic content. Bovi et al. [29] developed DefIE system, which extracts semantic relations from text documents through deep syntactic and semantic analysis of the text. They obtained syntactic information from dependency parser, and semantic information from Babelfy [53]. They mainly focused on verb-based relations. Following the approach presented in [53], Emami et al. [32] developed a semantic approach to extract profile of person entities from Farsi Web documents. In [32], first, the raw text is analysed syntactically and semantically and enriched the local processed information with semantic information obtained from a distant knowledge base; then a semantic rule-based approach is used to extract the related information of the persons in question.

Similar to [29] and [32], our approach draws on this idea of enriching dependency graphs with semantic information derived from a distant knowledge base. However, the limitations to their approach are that (i) it extracts relations existing only within a sentence containing a verb, which is not enough for Web text as some relations exist in clauses that have not a verb predicate, and (ii) it cannot discover implicit relations, which implied by the text. Our approach can easily extract cross-sentence verb-based relations and also infer implicit relations existing in a discourse document through probabilistic inference about relations. In addition to verb-based relations, our approach is able to identify noun-based relations accurately.

Table 1
Comparison of our approach and closely related AE methods

AE methods V N W $C S$ $C D$ I O

Li et al. [44] ✓ ✓ ✘ ✘ ✘ ✘ ✘

Surdeanu et al. [37] ✓ ✓ ✘ ✘ ✘ ✘ ✘

Soderland et al. [60] ✓ ✓ ✘ ✓ ✘ ✘ ✘

Chen et al. [36] ✓ ✓ ✘ ✓ ✓ ✓ ✘

Moro and Navigli [29] ✓ ✓ ✘ ✘ ✘ ✘ ✘

Yu and Lam [74] ✓ ✓ ✘ ✓ ✘ ✓ ✘

Soderland et al. [71] ✓ ✓ ✘ ✓ ✘ ✘ ✘

Bovi et al. [53] ✓ ✘ ✘ ✘ ✘ ✘ ✓

Soderland et al. [61] ✓ ✓ ✘ ✓ ✘ ✓ ✘

Our method ✓ ✓ ✓ ✓ ✓ ✓ ✓

AE methods	V	N	W	$C S$	$C D$	I	O
Li et al. [44]	✓	✓	✘	✘	✘	✘	✘
Surdeanu et al. [37]	✓	✓	✘	✘	✘	✘	✘
Soderland et al. [60]	✓	✓	✘	✓	✘	✘	✘
Chen et al. [36]	✓	✓	✘	✓	✓	✓	✘
Moro and Navigli [29]	✓	✓	✘	✘	✘	✘	✘
Yu and Lam [74]	✓	✓	✘	✓	✘	✓	✘
Soderland et al. [71]	✓	✓	✘	✓	✘	✘	✘
Bovi et al. [53]	✓	✘	✘	✘	✘	✘	✓
Soderland et al. [61]	✓	✓	✘	✓	✘	✓	✘
Our method	✓	✓	✓	✓	✓	✓	✓

V: extracts verb-based attribute values from verb phrases in unstructured text fragments? N: extracts noun-based attribute values from noun phrases in unstructured text fragments? W: extracts web-based attribute values from structured text fragments? $CS$ : extracts cross-sentence attribute values? $CD$ : extracts cross- document attribute values? I: extracts implicit attributes? O: maps attributes to ontology entries?

Joint inference and MLNs has become popular recently, because they make it possible for features and constraints to be shared among tasks. For example, Chen et al. [22] used MLNs to infer implicit family relations between persons. In other work, YU and LAM [74] used MLNs to extract relations between entities in Wikipedia articles. Our AE features to extract verb-based attributes is closely related to that of YU and LAM [74]. However, in their work, they applied MLNs to extract relations from raw text, whereas we apply MLNs formula on SRL frames and dependency graphs, which is different from working on raw text. Furthermore, they have not addressed entirely the challenges of synonymy and polysemy. These methods generate relations in textual surface form that are not linked to distant knowledge base entries.

After this short review, it can be seen that extracting semantic, structured information in natural language Web text is still a challenging, unsolved problem. This highlights the need for developing new and efficient AE approaches.

Table 1 shows a comparison of our AE approach and closely related AE approaches. In summary, our work extends previous work on AE research in a number of ways. Our approach produces semantic profiles by making rich use of distant knowledge bases and deep analysing the text syntactically and semantically. Our verb-based AE can alleviate the problem of syntactic writing variation exist in syntactic based AE methods. In contrast to previous work in relation extraction, which mainly focused on extracting atomic facts existing only within sentences of individual documents, we focused on cross-document person profiling, which attempts to extract explicit and implicit relations within and across sentences and documents.

3. Our proposed approach

Figure 1 shows the state diagram of our proposed approach for the problem of cross-document person profiling. As shown in Fig. 1, we decompose the problem as four subtasks:

Pre-processing : this phase takes as input the Web pages and prepares them as system’s desired format using available pre-processing tools. Pre-processing consists of five main subtasks: extracting clean text document, named entity tagging, intra-document co-reference resolution, sentence splitting, and sentence type identification.

Semantic analysis : this component takes as input the pre-processed sentences, and extracts semantic augmented syntactic dependencies and SRL frames from each sentence. Semantic analysis includes four main steps: word sense disambiguation, semantic role labeling, dependency parsing, and semantic enrichment.

Attribute extraction : this component takes as input a person entity of interest, and extracts its discourse profile, a vector of <attribute, value> pairs, which each pair represents a certain characteristic of the focus person. We use different AE methods according to types of attributes and text expression formats used to express instances of attributes.

Name disambiguation : this component takes as input the Web documents referring to an ambiguous person name and the set of discourse profiles produced by AE for that person name. The system then enriches the discourse profile of every person using the global attributes retrieved from a distant ontology. It also extracts social relations of persons in the given corpus. It then uses a graph-based clustering algorithm to group Web documents referring to the same person in reality, and fuse discourse profiles to form his/her corpus-level profile.

In the following, we describe these components in more detail.

Fig. 1.

The state diagram of our cross-document person profiling system.

Fig. 2.

An example of unstructured-style and structured-style data expression format in a sample Web page.

3.1. Pre-processing

In this article, we focus on the textual part of the Web pages, because the majority of the information about entities on the Web is often expressed in the natural language text. The Web pages need to be pre-processed and prepared according to system’s desired format, in which the content of a web page is transformed into a set of structured and unstructured text fragments. We decompose pre-processing into five main steps: (i) html tag removal, (ii) named entity tagging, (iii) co-reference resolution, (iv) sentence splitting, and (v) fragment type identification. First, for each Web page, Jsoup1

¹
Jsoup: Java HTML Parser, [http://jsoup.org/].

(an HTML parser) is run to cast it into plain text document. Next, for each document the Stanford named entity tagger [36] is run to tag the text for coarse-grained lexical entity types including person, location, organization, etc. For each identified named entity, we assign a unique index to distinguish the identity of entity. The annotated text documents are passed to the intra-document co-reference resolution module. We use the state of the art Stanford co-reference resolution system [42] to identify co-reference chains for all the entities mentioned in each document. The mentions in every co-reference chain of interest are then replaced with their corresponding representative mentions. Next, for the co-reference chain of interest within each document, we use the Stanford CoreNLP toolkit2

http://nlp.stanford.edu/software/corenlp.shtml.

[48] to extract all the sentences from a document.

The content on a Web page is often expressed in a mixture of different representations: unstructured format and structured format [21,50]. Unstructured text follows prescribed writing standards, and prepared for a fairly broad audience [21,50]. Unstructured writing needs to be clear and unambiguous. Longer and complete sentences are likely to be more prevalent in unstructured text. Complete sentences usually contain a subject, object and one or multiple verbs. On the contrary, structured text has few constraints on writing format, mixes various representations, prepared quickly and intended for a narrow audience [50]. In Fig. 2, we show an excerpt of structured and unstructured fragments. Each of the structured and unstructured expression formats require different information extraction methods. Identifying the type of fragment expression helps us to overcome the problem of structure variation and choose proper attribute extraction methods (Section 3.3) to extract entity-centric information according to data representation format. Therefore, we need to identify the expression format of text. In the fragment type identification, we classify text into structured and unstructured fragments. We adapt and modify the method proposed by Chen et al. [21] to segment text. There are two stages in fragment type identification: sentence type identification and sentence grouping. In sentence type identification, Chen et al. [21] used a rule-based method to classify lines in a web page as structured or unstructured according to the percentage of tokens that begin with capitalization. We extend their approach by using support vector machines (SVMs) [27], a supervised machine learning algorithm to classify each sentence in text as one of the two classes, structured or unstructured. The main feature for classification is the percentage of capitalized tokens and length of the sentence. The selection of these features comes from the fact that an structured sentence mainly is short and contains capitalized tokens. To perform sentence classification, we first manually select 100 sentences from WePS-2 training data [7] and then annotate them in terms of classification features. Out of these, 50 are structured examples and 50 are unstructured ones. We then use LibSVM toolbox3

http://www.csie.ntu.edu.tw/~cjlin/libsvm/

to classify unlabelled sentences into structured or unstructured classes. The performance of the sentence classification step is 79.31% by the F1 measure. This score is computed by randomly selecting a sample of 100 sentences from classified sentences. We manually examined the classified sentences and calculated the number of correct and incorrect sentences in the sampled 100 sentences and computed performance score. In second stage, continuous sentences that share same expression format are considered as a single fragment.

The pre-processing tools may produce errors, which propagate to the latter stages. However, improving the pre-processing components is beyond the scope of this paper. The remainder of the processing described in the following use this pre-processed text.

3.2. Semantic analysis

The semantic analysis component takes as input the pre-processed sentences, analysis each sentence semantically and syntactically, and produces semantic augmented syntactic dependencies and SRL frames. Semantic analysis consists of four stages: (i) word sense disambiguation, (ii) semantic role labeling, (iii) dependency parsing and (iv) semantic enrichment. In the following, we describe these steps in more details.

3.2.1. Word sense disambiguation

In word sense disambiguation (WSD) stage, we disambiguate each surface text word and entity mention to identify which of its senses is given in the text. By this way, we alleviate polysemy and synonymy problems that many previous attribute extraction works suffer from these problems. Our WSD system takes as input the target entity mention e and the context S(e) around it, and then utilizes Babelfy [53], a state of the art entity linking and word sense disambiguation system to obtain a sense mapping from surface text words and entity mentions to word senses and semantic ontological named entities. We define S(e) as follows: $\begin{array}{l} (1) & \begin{aligned} S (e) = H^{n} (e), \\ H^{1} (e) = {⋃_{j} s (e_{j}) | \forall e_{j} \in M (e)} \\ H^{n} (e) = H^{n - 1} (e) \\ \cup {⋃_{k} H^{1} (e_{k}) | \exists s_{f} : (e \in s_{f}) \\ \land (e_{k} \in s_{f})} \\ for n ⩾ 2 \end{aligned} \end{array}$ where S(e) is the context window around entity mention e, i.e., the set of sentences in neighbourhood of entity e, $s (e_{j})$ denotes the sentence in which the entity $e_{j}$ appear, M(e) is the intra-document co-reference chain of entities with respect to entity e, $s_{f}$ is the fth sentence in which both e and $e_{k}$ co-occur. $H^{1} (e)$ is 1th hop sentences around e, the set of sentences in which entity mention e or its co-referent mentions, M(e) occur. The sentences in $H^{1} (e)$ may contain other entity mentions which can be occur in other sentences at different positions of text. Similarly, $H^{n} (e)$ is the nth-hop sentences containing entities co-occurring with target entity e. In Eq. (1), we empirically set $n = 2$ , which provides better results. To clarify what we mean by $H^{n} (e)$ , consider the example sentences in Fig. 3. In Fig. 3, rectangles indicate the position of entity $e_{1}$ , ellipses shows the position of entity $e_{2}$ and diamonds show the position of entity $e_{3}$ in text. Notice that in Fig. 3, shapes with dotted border show the co-reference mentions of entities. Assume $e_{1}$ to be the target entity, and the goal is to find $H^{2} (e_{1})$ , 2-hop sentences in neighbourhood of entity $e_{1}$ . To compute $H^{2} (e_{1})$ , we first need to compute $H^{1} (e_{1})$ . $H^{1} (e_{1})$ includes the sentences in which entity $e_{1}$ or its co-referent mentions appear, i.e., ${s_{1}, s_{4}}$ . We then identify the co-occurring entities with target entity $e_{1}$ . The co-occurring entities with entity $e_{1}$ are ${e_{2}, e_{3}}$ . $H^{2} (e_{1})$ is ${s_{1}, s_{2}, s_{3}, s_{4}}$ , which is the union of the $H^{1} (e_{1})$ and the sentence containing $e_{2}$ and $e_{3}$ and their co-referent mentions.

Fig. 3.

A sample fragment of text with four example sentences, three entities and their co-referent mentions.

We map the word’s synset offset generated by Babelfy to possible corresponding WordNet [35] sense and DBPedia4

⁴

http://wiki.dbpedia.org/

URI. By this way, we relate the surface text words and entity mentions to their corresponding meaningful identity in a distant ontology. Figure 4(b) show a sense mapping example for a sample sentence given in Fig. 4(a). As a matter of fact, using Babelfy is not mandatory; any disambiguation or entity linking system can be used in WSD stage. However, a knowledge-based unified word sense disambiguation system like Babelfy is best suited to our setting.

3.2.2. Semantic role labelling

In semantic role labelling, we assign a “who did what to whom, when, where, why and how” structure to each sentence. We refer this structure as semantic role (SRL) frame. We use SENNA [26] to extract SRL frames from sentences of text. We notice that any semantic role labelling system can be used for SRL frame extraction. The reason that we used SENNA is the fact that it is competitive with state of the art SRL systems, and it is open source. Let $Frm = {p, E_{1}, E_{2}, \dots, E_{m}}$ be a SRL frame in which p denotes the verb predicate, $E_{i}$ is the ith element in the frame, and m is the number of SRL elements. Each element $E_{i}$ is a pair of (srl, arg), where srl represents the type of semantic role, and arg denotes the value for the underlying argument. There may be extracted multiple SRL frames from a single sentence s depending on the number of verbs in sentence s. Types of semantic roles follow the annotation guideline in PropBank [56]. Since not all SRL labels appear in every verb and semantic role labelling systems are designed on a verb-by-verb basis, the type of semantic roles is commonly divided into meta-roles ARG0-ARG5 and several common modifiers such as time and location. This shows semantic roles are predicate sense dependent. Each ARGi represents a different role according to the verb predicate p (for more details see e.g., [18]). Figure 4(c) shows the SRL frame extracted from the sample sentence given in Fig. 4(a).

Fig. 4.

An excerpt of semantic analysis for a sentence; (a) sample sentence, (b) disambiguated word senses; (c) SRL frame Frm in PropBank style extracted by SENNA system; (d) SRL frame Frm in VerbNet style; (e) dependency graph $G_{d}$ extracted by Stanford parser; (f) semantic enriched syntactic graph, $G_{s d}$ , (g) semantic enriched SRL frame Frm.

To cope with the problem of predicate sense dependency of semantic roles, a conversion is needed to map PropBank-style roles into VerbNet domain-independent thematic roles such as agent, patient, theme, etc. To fulfil this task, we employ the PropBank and VerbNet mapping provided by the SemLink project5

⁵

https://verbs.colorado.edu/semlink/

[14,46]. However, the mapping provided by SemLink is not complete, but it is sufficient for our role mapping purpose. The mapping between PropBank and VerbNet annotation for a given predicate p consists of two stages: (i) lexical mapping and (ii) token mapping. The lexical mapping is responsible for identifying the set of potential VerbNet classes for a given lexeme in PropBank. For each lexeme p in PropBank, it provides a vector of possible VerbNet lexemes

L = {l_{1}, l_{2}, \dots, l_{k}}

, where every

l_{i} \in L

is the ith VerbNet lexeme is used to describe one of the senses covered by lexeme p. The vector L can have one or multiple items depending on the meaning of lexeme p. For example, the first sense of the verb “study” in PropBank – “a detailed critical inspection” – lexical mapping produces four VerbNet classes include “assessment-34.1”, “discover-84-1-1”, “learn-14-1”, and “sight-30.2”. The second sense of the “study” – “applying the mind to learning and understanding a subject” – is only mapped to VerbNet class “learn-14-1”. For lexical mapping, we first use WordNet to map the disambiguated Babelfy synset offset of lexeme p obtained by WSD to its corresponding WordNet sense. This identifies which sense of lexeme p is given in the given SRL frame. We then take the lemma of lexeme p from WordNet, and chose a set of VerbNet classes that can be used to describe the specified sense of lexeme p, and form lexeme vector L. Because of the difference in the coverage of PropBank and VerbNet resources, for many PropBank’s senses, there are no corresponding lexical unites in VerbNet. For a lexeme of this category, we develop a simple, but practical algorithm to infer the correct VerbNet class. We look for synonyms of the target lexeme and use the classes that they appear in. Since the class which a lexeme belongs to, depends on the meaning of the lexeme, the synonym is a good indicator of the correct VerbNet class for that PropBank lexeme that has not a direct entry in VerbNet.

The token mapping takes as input the lexeme p and its corresponding mapping vector L, and decides which $l_{i} \in L$ is applicable to lexeme p. We perform token mapping through comparing the SRL frame of lexeme p, and syntactic frame of VerbNet class $l_{i}$ . A VerbNet class with a syntactic frame, which most closely matches the SRL frame of lexeme p is assigned to p. Moreover, the arguments associated with p need to meet the selectional restrictions of the arguments of the VerbNet class $l_{i}$ . At last, the PropBank-style roles ARGi for lexeme p map to their corresponding VerbNet entries. In some cases, a lexeme from PropBank does not exist in VerbNet; its synonyms do not exist in VerbNet, has arguments without corresponding entries in VerbNet; or does not exist in the correct sense. In these cases, the arguments in SRL frame are left in its unmapped (ARGi) form. Figure 4(d) shows the mapping result for the SRL frame given in Fig. 4(c). As shown in Fig. 4(d), the entry ARG0 is mapped to thematic role “agent”. The arguments “AM-LOC” and “AM-TMP” have not equivalence in corresponding VerbNet class, therefore they left in their original form. However, semantic role labeling is not always completely accurate, but we find it to be sufficient for our purpose.

3.2.3. Dependency parsing

We use the Stanford dependency parser6

⁶
http://nlp.stanford.edu/software/stanford-dependencies.shtml

[48] to obtain a typed dependency graph

G_{d}

for each sentence s. In graph

G_{d}

, words are represented as nodes and word-word dependencies are illustrated as directed edges. Figure 4(e) shows dependency graph for the sample sentence given in Fig. 4(a).

3.2.4. Semantic enrichment

Semantic enrichment stage augments each SRL frame Frm and syntactic dependency graph $G_{d}$ with semantic information provided by the WSD component. Inspired from previous work [29], we formulated the semantic enrichment as a searching for compatible pairs across word senses provided by WSD and surface text words in a SRL frame and dependency graph. To enrich a dependency graph $G_{d}$ extracted from sentence s and create a semantic augmented graph $G_{s d}$ , we start from the disambiguated senses obtained for sentence s. Each disambiguated sense may cover a single-word expression or multiple-word expression in dependency graph. In case of single-word coverage like “professor” in Fig. 4(b), its disambiguated word sense “bn:00064601n” simply attaches to its corresponding node in $G_{d}$ . In case of multiple-word coverage like “Daniel Jurafsky”, which its disambiguated sense “bn:03361485n” covers more than one node in $G_{d}$ , we merge the sub-graph referring to a same sense to a single semantic node. The remaining nodes (except stop words, adjuncts and modifiers such as ‘the’ and ‘of’) that have not mapped to a sense are added without any modification. Figure 4(f) shows the enrichment result for the graph $G_{d}$ given in Fig. 4(e). To enrich SRL frame elements with word senses, we simply replace each SRL element with its corresponding disambiguated sense. The remainder elements that have not correspondence in disambiguated senses are left without mapping. Figure 4(g) shows the semantic enrichment result for the SRL frame given in Fig. 4(d).

3.3. Attribute extraction

Attribute extraction (AE) component takes as input the person names in question and their relevant documents processed by semantic analysis stage. The person names and their relevant documents are previously provided in evaluation datasets. AE component then extracts the attributes of persons, and forms their discourse profile. Formally, we define the discourse profile of an entity e, P(e) as follows: $\begin{matrix} (2) & P (e) = {(a, v) | a \in A, v \in V (a)} \end{matrix}$ where A is the vocabulary of attributes that can be used to describe characteristics of person e and V(a) represents the set of valid filler values for attribute $a \in A$ . Table 2 lists the vocabulary of attributes that our approach attempts to automatically extract, if such information exists in text. These attributes are those extensively studied in information extraction tasks such as Web people search [4], slot filling, and knowledge base population [49,66]. According to [21], each particular expression format of Web data needs different AE method.

Table 2
Vocabulary of attributes in question

Attribute names

Affiliation Degree Mentor Phone

Award Email Nationality Relatives

Birth place Fax Occupation School

Date of birth Major Other name Website

Attribute names
Affiliation	Degree	Mentor	Phone
Award	Email	Nationality	Relatives
Birth place	Fax	Occupation	School
Date of birth	Major	Other name	Website

To handle this problem, we use unstructured-style AE and structured-style AE methods. These two methods are complementary and have little overlap in the resulting extractions. We ran unstructured-style AE module and structured-style AE module in parallel, and report the union of extractions as final result. If the two methods had different output for a target attribute, we use only the unstructured-style AE result. In the following, we describe these AE methods in more detail.

3.3.1. Unstructured-style attribute extraction

Let $X = {a_{1}, a_{2}, \dots, a_{m}}$ , $X \in A$ be the list of attributes in question that their filler values can be extracted using unstructured-style AE methods. In unstructured-style text, we observe that the filler values for the attributes in X are mainly found in (i) verb constructions expressed by either a main verb, which serves as an indicator for an attribute class (e.g., “born” or “work”), or a light verb (e.g., “is” or “was”) or (ii) noun-based constructions that either appositives (e.g., “the Stanford professor, Jurafsky”) or noun modifiers (e.g., “American professor”) to the given entity. Therefore, we use two types of unstructured-style AE to extract attribute values: (i) verb-based AE, and (ii) noun-based AE. We applied verb-based AE on semantic augmented SRL frames, and noun-based AE on semantic boosted dependency graphs. We find that integrating verb-based AE with noun-based AE improves the performance of AE component. In the following, we give more detail about these methods.

Verb-based attribute extraction. Verb-based AE takes as input the SRL frames of each document that have been processed by semantic analysis, and extracts discourse filler values for the attributes in question. Our key idea in using SRL frames for AE is twofold: (i) semantically labelled arguments in SRL frames almost correspond to the filler values of the attributes in question, (ii) mapping SRL frames to domain-specific attributes is more straightforward, while extracting attributes from raw text is so challenging. To extract filler values, we use several AE formulae/features and employ Markov logic networks (MLNs) [58] to model inter-weaved constraints and AE formulae to map the SRL frames to possible attributes. MLNs are joint models, which combine and make full use of the merits of both first-order logic (FOL) and probability. Our choice of the MLNs is guided by the following facts: (i) MLNs are capable to model uncertainty efficiently using probabilistic unified graphical models; (ii) MLNs model the IE task in a collective manner, i.e., they are capable of performing within and cross sentence IE. This is different from other AE methods that predict relations independently without considering the dependencies between entities across sentences; and (iii) MLNs incorporate first order logic to express and combine various sources of knowledge.

Formally an MLN L is a set of pairs (F, w), where F is a formula in FOL and w is a real number indicating the weight attached to formula F. In our work, each FOL formula F is composed of a set of AE features and constraints to captures the contextual information of the focus entity and extract filler values for given attributes. We represent the AE features and constraints using four types of symbols: constants, variables, functions, and predicates. Constants represent the objects in discourse documents, e.g., person names: “Amenda Lentz”, “Alexander Macomb”. Variables (e.g., x, y) range over the ground objects. Predicates represent attributes of objects or relations among objects within discourse documents (e.g., Father(x, y), Synonym(x, y)). We represent each SRL frame Frm as several predicates of the form SFrm(dID, pred, srl, arg), where dID is document id, pred is verb predicate in SRL frame Frm, srl is semantic role label, and arg is the argument value for role srl in the frame Frm. Similarly, we represent attribute classes as predicates of the form Attr(dID, e, v), where Attr represents an attribute class, e is a person entity, and v is the filler value of Attr for e. In the following, we discuss features that we used to verb-based AE.

Class recogniser feature . Some verbs and special keywords provide crucial clues to identify domain-specific attributes associated to a given person. The main idea in using verbs as attribute class indicators is that verbs typically convey the main idea of a sentence. For example, consider the sentence given in Fig. 4(a), the verb “born” implies that there may be a “birth place” fact in the sentence. Keywords and named entities serve as valid argument values of attributes. For example, in the sentence given in Fig. 4(a), the keyword “professor” is a potential indicator value for the attribute of “occupation”. We use seed verbs and keywords as class recognizer features to labelling the verbs or terms in SRL frames as indicators of possible values for attributes in question.

The main challenge here is the identification of seed trigger verbs and keywords for each of the attribute classes. The seed trigger words can simply be constructed manually as performed in previous work [11,21,54,60–62,68]. These approaches use pre-compiled lists of attribute-specific keywords were collected manually from different information resources such as Wikipedia, DBpedia and FreeBase ontologies, and tables found on the Web. For example, for the attribute “occupation”, a list of occupations is collected from DBpedia and formed keyword set “occupation”. However, these approaches need human supervision and expertise and are labor intensive to create such lists. To solve this issue, we propose a supervised approach for seed fact learning from training data. Our approach is based on the world knowledge of co-occurring words.

Our seed learning approach mainly consists of two steps: (i) seed extraction, and (ii) seed pruning. Algorithm 1 shows the pseudo code of our seed fact extraction approach. Let $D = {d_{1}, d_{2}, \dots, d_{N}}$ be the set of text documents given in training data. The seed extraction takes as input the text documents D, and then identifies the sentences within every document $d_{i} \in D$ in which the focus entity e or its co-referent mentions occur. Formally, we define the sentences in neighbourhood of entity e within document $d_{i} \in D$ as follows: $\begin{matrix} (3) & d = {⋃_{i} s (e_{i}) \in d_{i} | \forall e_{i} \in M (e)} \end{matrix}$ where M(e) is the intra-document co-reference chain of entities with respect to entity e, and $s (e_{i})$ is the sentence within document $d_{i}$ in which entity $e_{i}$ appears. We concatenate the sentences and form a summarized document d, which is a text window around the target entity e. Then, for each attribute $a \in X$ of the person e, our algorithm iterates over the sentences in d, and looking for the sentences that contain the focus person e and a given value v for the attribute a. For the extraction of a seed verb, a verb v in sentence s is selected as a candidate seed verb, if the focus person e in the sentence s to be the “subject” or “object” of the verb in dependency graph of s (Algorithm 2). We can apply the approach on SRL frames and use the tags of “agent” and “patient” instead of “subject” and “object”. For the extraction of seed trigger keywords, a mention k in sentence s is selected as a candidate seed keyword, if in the sentence s, the target attribute value v and the focus person e collocate together and co-refer (Algorithm 3). For example, consider the sentence “Daniel Jurafsky, the professor of Stanford was born in Yonkers, NY”, the keyword “professor” and the target entity “Daniel Jurafsky” are in close proximity and co-refer (i.e., they are in the same co-reference chain). Thus the keyword “professor” can be considered as a possible trigger keyword for the attribute “occupation”. The reasons we exploit co-reference information are twofold: (i) many of the Web pages are multi-typical and include information about several entities. In this case, there is a need to relate the focus entity to the candidate attribute values. We solve this issue using co-reference information; and (ii) the keywords representing the attributes of the focus person are often nominal mentions that are co-referent with that person. This may reduce the recall of keyword extraction system, but makes the generated trigger keywords more precise and discard erroneous extractions. This is more important for practical applications.

To select the best matches seeds, we score the candidate seeds and select high score ones. To do this, we assign a weight for each candidate seed. We compute $w_{j}$ , weight of seed term $t_{j}$ with respect to entity $e_{i}$ using the pointwise mutual information (PMI) weighting schema [25], which is defined as follows: $\begin{array}{l} (4) & w_{j} = PMI (e_{i}, t_{j}) = log \frac{p (e_{i}, t_{j})}{p (e_{i}) p (t_{j})} \\ (5) & p (e_{i}, t_{j}) = \frac{C (e_{i}, t_{j})}{\sum_{k} \sum_{l} C (e_{k}, t_{l})} \\ (6) & p (e_{i}) = \frac{C (e_{i})}{\sum_{k} C (e_{k})}, p (t_{j}) = \frac{C (t_{j})}{\sum_{k} C (t_{k})} \end{array}$ where $C (e_{i}, t_{j})$ is the frequency of co-occurrence of mention $e_{i}$ and term $t_{j}$ in corpus D, $C (e_{i})$ and $C (t_{j})$ are the count of occurrence of mention $e_{i}$ and term $t_{j}$ in corpus D, respectively.

Let $T = {t_{1}, t_{2}, \dots, t_{k}}$ be the set of candidate seed terms extracted by seed extraction component for attribute a, and $W = {w_{1}, w_{2}, \dots, w_{k}}$ be the weight of candidate seed terms. In seed pruning step, we first normalize the weights W and sort the terms in T according to their weights order. We then select from the ordered terms in T, ρ percent (we empirically set $ρ = 30 %$ ) of the top ranked words for which the normalized weight of candidate seeds for an attribute a is higher than average weight μ. We define μ as follows: $\begin{matrix} (7) & μ \leftarrow \sum_{i} w_{i} / | T | \end{matrix}$ Algorithm 4 shows the pseudo code of seed pruning stage. The remaining words are selected as trigger seeds for a given attribute class $a_{i} \in X$ . Because of the data sparseness in training data, the resulting set of seed facts might be insufficient. To alleviate this problem, we extract the possible synonyms of initial seed facts from WordNet and append the resulting synonyms to the list of seeds. However, the resulting seeds do not complete, but we found that the generated seeds are sufficient in practice.

Since we did not have the ground truth of seed facts in the source text, we omitted the evaluation of seed extraction and instead we focused on calculating the performance of the identified and extracted seeds. We approached this task by randomly selecting a sample of 100 seeds from each of keywords and event verbs extracted from WePS-2 training [7] and Wikipedia training dataset [28]. We then calculated the number of correct and incorrect seeds in sampled data and compute performance scores. We manually examined the sentences corresponding to each sampled seed to identify whether the extracted seed is correct or incorrect. Table 3 shows the performances obtained by our seed extraction method on benchmark training datasets. In the experiments, we conducted evaluations using three criteria: precision (P), recall (R), and F1 score. For more detail about these metrics, see e.g. [57]. However, the performances are not ideal, but we find that the resulting seeds are sufficient in practice. In future work, we plan to enrich seed verbs and keywords using extra Web resources and various ontologies to improve the performance.

Table 3
Performances of our seed extraction method on benchmark training datasets

Dataset Seed type P (%) R (%) F1 (%)

WePS-2 Keyword 65.2 61.9 63.51

Verb 79 63.92 70.66

Wikipedia Keyword 76.01 70.4 73.10

Verb 80.04 74.63 77.24

Dataset	Seed type	P (%)	R (%)	F1 (%)
WePS-2	Keyword	65.2	61.9	63.51
Verb	79	63.92	70.66
Wikipedia	Keyword	76.01	70.4	73.10
Verb	80.04	74.63	77.24

Given attributes’ class recognizers, an SRL frame SFrm is considered as a potential candidate for an attribute $a \in X$ , if one of the following conditions is hold: (i) the verb predicate in SRL frame SFrm matches up with one of the seed verbs specific to the attribute a, (ii) one of the keywords or named-entities regarding the attribute a appears in the arguments of frame SFrm. For example, the following formulae checks to infer that the location entity g in a SRL frame is a “birth place” for the focus person e.

w: Verb(v) ∧ Synonym(v, “born”) ∧ SFrm(dID, v, “agent”, e) ∧ SFrm(dID, v, “AM-Loc”, g) ⇒ Birthplace(dID, e, g)

This formula looks at the arguments of a SRL frame and decides which arguments corresponds to the “birth place” of the focus person e. In above formula, predicate Synonym(v, “born”) first generates all WordNet synonyms of verb “born” and from synonymous vector $S = {s_{1}, s_{2}, \dots, s_{m}}$ , where $s_{i}$ is the ith synonym of verb “born”. It then checks to identify whether the verb v in SRL frame SFrm is equal or synonym of verb “born”. Using synonyms for AE solves the problem of verb writing variations in expressing the same meaning, and increases the flexibility of AE algorithm. We take the lemma of verb predicates of SRL frames and then compare the predicates with respect to the focus attribute. For example, in the above formula, instead of “born”, we use its lemma “bear”. Lemmatization is helpful to cover all syntactic variations of verbs and make the verb-based AE to work independent of verb tense.

Entity type feature . Entity types provide very useful information for AE. For example, the filler value for the attribute “birth place” must be a location, and cannot be an person, organization, time, etc.

w: Verb(v) ∧ Synonym(v, “born”) ∧ SFrm(dID, v, “agent”, e) ∧ SFrm(dID, v, “AM-Loc”, g) ∧ Location(dID, g) Person(dID, e) ⇒ Birthplace(dID, e, g)

The predicate Person(dID, e) ensures that the agent of “birth place” to be a person in document dID. Similarly, the predicate Location(dID, g) ensures that the argument g must be a location named entity in document dID. We used Stanford named entity tagger to mark common entities including person, location and organization as indicators of possible values for attributes. Currently, there is no tool which can be used to extract some special named entities (e.g. occupation, award, major and degree). This is a main issue for named entity recognition in Web data. To mark such special named entities, we use our seed extraction strategy as explained in above section.

Context feature . By employing the MLN inference formulae, the implicit facts about entities can be easily discovered across SRL frames. For example, given the following formula:

w: Verb(v) ∧ Synonym(v, “marry”) ∧ SFrm(dID, v, “agent”, e) ∧ SFrm(dID, v, “patient”, y) ∧ MotherOf(dID, y, z) ∧ Person(dID, e) ∧ Person(dID, y) ∧ Person(dID, z) ∧ Co-occur(dID, e, y, z) ⇒Relatives(dID, e, z)

When applying inference rules, we exploit co-reference information to make the extractions more precise. To infer a filler value for an attribute of the focus entity e, we use the SRL frames of the sentences appearing in neighbourhood of entity e, S(e) as defined in Eq. (1). In Eq. (1), we set $n = 2$ , i.e. we use 2-hop neighbours of target entity e. This condition discards noisy extractions. In above formula, the predicate Co_occur(dID,e,y,z) provides co-occurrence information, and limits ours to extract attribute values for the focus entity e only from the SRLs of the neighbor sentences S(e), which the target entity e or its co-referent entities appear in. If all constraints are met, the rule specifies a “Relatives” relation between person e and z across sentences.

Part of Speech (POS) feature . POS tags provide more rich information for accurate AE. We define POS features to include the part of speech information. For example, the predicate IsNounPhrase(y) in the following formula ensures that the filler value y for the attribute of “award” should be a noun phrase.

w: Verb(v) ∧ Synonym(v, “win”) ∧ SFrm(dID, v, “agent”, e) ∧ SFrm(dID, v, “theme”, y) ∧ Person(dID, e) ∧ AwardIndicator(dID, y) ∧ IsNounPhrase(dID, y) ⇒ Award(dID, e, y)

Morphological feature . To capture morphological information, we embed the morphological features into MLN formulae. For example, the following formula prevents the incorrect extraction from a sentence like “Alexander Macomb was not born in Berlin, but the fact that he born in Minsk.”

w: Verb(v) ∧ Synonym(v, “born”) ∧ SFrm(dID, v, “AM-NEG”, “not”) ∧ SFrm(dID, v, “agent”, e) ∧ SFrm(dID, v, “AM-Loc”, g) ⇒ null

Argument coherency feature . Entity type of the arguments must be consistent for each attribute class. We define some hard formula (constraints) to satisfy the argument type coherency. For example, given the following formula:

w: Verb(v) ∧ Synonym(v, “marry”) ∧ SFrm(dID, v, “agent”, e) ∧ SFrm(dID, v, “patient”, y) ∧ Person(dID, e) ∧ Person(dID, y) ⇒!Birthplace(dID, e, y) ∧ !Degree(dID, e, y)

This formula implies that the relation between two persons cannot be “birth place” or “degree”. By this way the argument coherency is satisfied.

Noun-based attribute extraction. The filler values of the attributes that contained in noun-based constructions cannot be extracted by verb-based AE. For example, in the sentence given in Fig. 4(a), the verb-based AE cannot conclude that the phrase “professor” is a filler value of the attribute “occupation” for the person “Daniel Jurafsky”, because this attribute is not expressed by the verb of “born”. This phenomenon occurs in a variety of domains like “The New York mayor, Bloomberg signed the contract”. To extract noun-based attributes, we define a series of MLN AE rules, which exploits the semantic boosted syntactic dependencies and the lexical information in the form of named entities and keywords. The overall strategy of our noun-based AE is inspired by previous work [61], but there is a significant difference between those AE approach and our approach. The limitations to their approach are that (i) the attributes’ filler values that are multi-word expressions, and cover more than one word in dependency graph cannot be correctly extracted and (ii) it may produce irrelevant extractions due to ignoring co-reference information between nominal mentions containing attributes of interest and target entity mentions. We extend their approach in five ways: (i) extracting both single-word and multi-word attribute values from semantic boosted dependency graph; specifically, borrowing the idea from the work of Bovi et al. [29], we couple syntactic dependencies and fully disambiguated entity mentions and word senses to solve the problem of multi-word attribute extraction; (ii) applying noun-based AE method on the semantic enriched syntactic dependency graph instead of the pure syntactic dependency graph, (iii) considering the co-reference information between the nominal mentions that include attributes of interest and target entity mentions; this constraint discards erroneous and irrelevant extractions; (iv) proposing an automatic method to extract trigger keywords that are indicators of the given attributes; and (v) modelling noun-based AE rules in the form of MLN rules that enables us to model all of the attribute-specific constraints in a collective manner instead of designing separated filters.

The input to the noun-based AE algorithm is the query entity e, a set of semantic boosted dependency graphs generated for sentences related to entity e, and a set of pre-compiled keyword set K extracted automatically by our seed extraction system. We first represent each dependency graph as a set of predicates in the form DepRel(dID, rel, arg1, arg2), where DepRel is a dependency relation, dID is the document id, rel is the dependency arc between two arguments arg1 and arg2. For example, by setting dID to 1, we show the dependency graph in Fig. 4(f) as follows:

DepRel (1, “nsubjpass”, “born”, “Daniel Jurafsky”)

DepRel (1, “appos”, “Daniel Jurafsky”, “professor”)

DepRel (1, “prep_of”, “professor”, “Stanford”)

DepRel (1, “prep_in”, “born”, “1962”)

DepRel (1, “prep_in”, “born”, “Yonkers, NY”)

Our algorithm then iterates on the corresponding predicates of a dependency graph, and looking for a co-occurrence of a keyword $k \in K$ , and the query entity e. An entity e and a keyword k in the graph $G_{s d}$ is considered to be related, (i) if there is a dependency path (with maximum length 3) between entity e and keyword k in which every dependency edge on the path tagged with one of the following labels: nsubj, nsubjpass, appos, amod, nn, poss, prep_of, and rcmod; and (ii) if the keyword k and entity e co-refer, i.e., they refer to the same entity. We formulate these conditions in the form of MLN formulae. Some example MLN formulae are as follows:

w: DepRel(dID, “amod”, e, k) ∧ Co-refer(dID, e, k) ∧ Person(dID, e) ∧ NationalityIndicator(k) ⇒ Nationality(dID, e, k)

w: DepRel(dID, “appos”, e, m) ∧ DepRel(dID, “amod”, m, k) ∧ Co-refer(dID, e, k) ∧ Person(dID, e) ∧ NationalityIndicator(k) ∧ IsNounPhrase(m) ⇒ Nationality(dID, e, k)

w: DepRel(dID, “nsubjpass”, e, m) ∧ DepRel(dID, “amod”, m, k) ∧ Co-refer(dID, e, k) ∧ Person(dID, e) ∧ NationalityIndicator(k) ∧ IsNounPhrase(m) ⇒ Nationality(dID, e, k)

where Co-refer(docID, e, k) means that the entity e and keyword k are co-referent in the document dID, NationalityIndicator(k) implies that the keyword k is an indicator for attribute “nationality”, and predicate Nationality(dID, e, k) means that the keyword k is a candidate value for the “nationality” of entity e. The first rule covers cases like “American professor, Daniel Jurafsky was born in Yonkers, NY”. The second rule works for cases like “Daniel Jurafsky, the American professor was born in Yonkers, NY”. This rule concludes that there is a path between entity “Daniel Jurafsky” and the keyword “American”. This path contains dependency arcs “appos” and “amod”. It is a valid path and meets constraints, thus the keyword “American” is a valid filler value for the attribute “nationality”. The third rule works for cases like “Daniel Jurafsky is the American professor”. Using such type of rule, we solve the defect of verb-based AE on extracting hypernymy relations: verb-based AE cannot extract attributes’ values when the main verb in a sentence is one of the auxiliary verbs (light verbs) having zero arguments in semantic role labelling annotation.

Since dependency arcs in dependency graph $G_{s d}$ is directed, there is no guarantee in finding a path between named entities and keywords of a target attribute. Thus, we use an undirected version of the graph $G_{s d}$ , and follow the assumption of tokens’ locality information [24] to find the path between entity pairs. The paths found between the entity e and the attribute-specific keyword k is often the shortest paths that meet the dependency tag constraints. The idea behind this constraint follows the shortest path hypothesis [16], which states that the most valuable information about a relation is contained on the shortest path between two attributes’ argument nodes in the dependency graph.

We use Tuffy [55] for implementing MLN formula. Our choice of the Tuffy is guided by the facts that (i) it is efficient for inference, (ii) it is competitive with state of the art implementation of MLNs in both quality and speed, and (iii) it is scalable for large-scale data. We manually designed around 68 fact extraction and inference rules. We used the Diagonal Newton discriminative learner algorithm [47] to learn the optimal weights of MLN rules. By this way, we give a dump of training data (50 randomly sampled pre-processed documents of the WePS-2 training) and the manually designed MLN rules to the learning algorithm. The algorithm then computes the weights of MLN rules by maximizing the likelihood of the training data. To speed up the learning process, we learn weights for different rules individually. That is, we make a simplifying independence assumption for all rules so that we can learn rule weights individually. The MLN rule’s weight indicates how the MLN rule is actually observed in the training data. Intuitively, the system gives a MLN rule a high score if it covers many facts.

We notice that to reduce complexity, in attribute extraction step, we apply MLN rules on SRL frames extracted from a single discourse document at each time.

3.3.2. Structured-style attribute extraction

When the instances of attributes are expressed in structured-style format, the formal-style AE approaches cannot directly be applied. This is due to the fact that structured-style text does not follow a standard writing format. To extract the attributes from structured-style fragments, we use Web-specific patterns. The overall strategy of our Web-specific patterns is similar to those AE methods taken by [11,21]. However, they obtained attribute-specific keywords manually. In contrast, we use an automatic keyword extraction method as described in Section 3.3.1. Each Web-specific pattern uses attribute-specific gazetteers and regular expressions to mark potential values for attributes in structured-style fragments. Let $Y = {a_{1}, a_{2}, \dots, a_{q}}$ , $Y \in A$ be the list of attributes on question that their filler values can be extracted using structured-style AE method.

For each attribute $a \in Y$ , the Web-specific pattern walks through an structured-style fragment and finds a potential value for attribute a, when one of the following predicates is hold: (i) one trigger term from the gazetteer regarding the attribute a appears in the fragment, or (ii) a regular expression regarding the attribute a fires in the given fragment. Similar to previous work [21,73], our Web-specific patterns follow the “one entity per document” assumption. Following this assumption, we assume that all of the facts extracted from a document refer to the same person in reality, provided that some validation constraints are met. This assumption is often holds with the exception of historic corpora and biography Web documents. Web-specific patterns are appropriate to extract those attributes that follow a specific format and are easy to detect such as “email”, “fax”, “phone”, and “website”. We can use automatic pattern extraction methods to learn Web-specific patterns. However, the results in previous work [11,60], indicate that when we have insufficient training data to learn accurate patterns, hand crafted patterns are one of the best choices to AE. Thus, we use hand-crafted AE patterns that are efficient for structured-style AE. Since we tackle a closed set of attributes, designing Web-specific patterns needs minimum human engineering effort.

Once we have extracted values for personal attributes, we decide which values are valid and which ones should be ignored. We use a set of validation rules to verify the correctness of attribute values. These rules are a set of attribute-specific constraints to each attribute according to its type. For example, for “date of birth” attribute, we defined date validation rules, for “phone” and “fax” attributes, suitable phone and fax validation rules, and so on. In this way, a candidate value is considered as valid filler for a given attribute, if it satisfies all constraints specified for that target attribute, otherwise it will be discarded. For example, the correct value for “website” attribute must contain the original name of the person under consideration, any variations of the given name identified by co-reference module, or domain name from the “email” attribute. We formulated attribute specific constraints in the form of regular expressions and attribute specific rules. Notice that for the attribute of “date of birth”, we first normalize the date values using Stanford SUTime library [19] and then validate them. Since we solve polysemy and synonymy at WSD stage (Section 3.2.1), we do not need normalization to normalize abbreviations and location names. This is while most previous work suffer from the problems of polysemy and synonymy. After validating attribute values, to create a discourse profile for each person, his/her related attributes are assembling while eliminating redundant information.

3.4. Name disambiguation

The individual profiles derived from multiple sources exhibit different attributes of an entity and do not entirely overlap, thus these profiles can complement each other. For integration of profiles extracted from multiple documents and create corpus-level profiles of persons, we need to solve the problem of name ambiguity across documents, and make an exactly one to one correspondence between persons and their profiles at corpus level. We formulated the person name disambiguation as a clustering problem. Let $C = {C_{1}, C_{2}, \dots, C_{K}}$ be the K clusters, and $C_{i} \neq \emptyset$ , $C_{i} \cap C_{j} = \emptyset$ , $i \neq j$ , $i, j = 1, 2, \dots, K$ , $⋃_{j = 1}^{K} C_{j} = P$ , where $P = {P_{1}, P_{2}, \dots, P_{N}}$ is the collection of discourse profiles. The goal of name disambiguation is to find such C, which the objects ${P_{h}^{i}, P_{h + 1}^{i}, \dots, P_{l}^{i}}$ , $(1 ⩽ h ⩽ l ⩽ N)$ in cluster $C_{i} \in C$ refer to the same person in reality.

We exploit an integration of two important types of information about persons as clustering features. The first is the personal attributes that stored in persons’ profiles. The second source of information is social links between persons. The main idea behind our approach is the fact that the attributes of a person can complement his/her social links, and vice versa. In other words, if one source of information is missing or noisy, the other can make up it. Social links provide a better interpretation of the information contained in discourse profiles. Similarly, the personal attributes give meaning to social links between persons as they identify the roles the persons play with respect to each other (mentor of somebody, brother of somebody). However, we found that the attributes stored in discourse profiles are not sufficient enough to robust name disambiguation. To alleviate this issue, we propose a profile enrichment method to enrich local discourse profiles with extra, global semantic information extracted from distant knowledge bases.

Algorithm 5 shows the pseudo-code of our person name disambiguation approach. Our approach first extracts social relationships of person entities and create a social relationship graph. Our social link extraction approach obeys closeness centrality theory [15]. We assume that social relationship between entities in the real world is reflected by their closeness in text of the documents they are mentioned in. We assume that two entities are socially linked if they collocate together in a corpus more frequently. In the profile enrichment step, the system takes as input the discourse profile and enriches the local discourse profiles with rich global features retrieved from a distant knowledge base by considering co-occurring entities and their surrounding context. The graph creation component takes as input the social relationship graph and enriched discourse profiles associated with each ambiguous person name and then map them into an undirected graph G. Graph clustering takes as input the graph G and a pre-defined similarity measure (in our research, neighbourhood random walk distance) and group graph nodes into a set of disjoint clusters $C = {C_{1}, C_{2}, \dots, C_{K}}$ . In the following, we describe these components in more detail.

3.4.1. Social link extraction

Social link extraction phase takes as input the target ambiguous name and it related documents and makes a social graph for that name. Our social link extraction approach obeys closeness centrality theory [15]. We assume that social relationship between entities in the real world is reflected by their closeness in text of the documents they are mentioned in. We assume that two entities are socially linked if they collocate together in a corpus more frequently. To identify the linked entities with an target entity e, we extract the co-occurring entities in the neighbourhood of entity e. To do this, we first identify context window S(e) as defined in Eq. (1). Let $N (e) = {e_{1}, e_{2}, \dots, e_{m}}$ be the list of co-occurring entities with target entity e, which appear in S(e). We create a social relationship graph between the entities in N(e). We define $w_{i j}$ , the strength of relationship between linked entities $e_{i}$ and $e_{j}$ as follows: $\begin{matrix} (8) & w_{i j} = \frac{θ_{i j}}{\sum_{k} \sum_{l} θ_{k l}} \end{matrix}$ where, $w_{i j}$ is the normalized distance-weighted frequency of entities’ co-occurrences in the input corpus, $θ_{i j}$ is the distance-weighted frequency of entities’ co-occurrences in the corpus, which we defined it as follows: $\begin{matrix} (9) & θ_{i j} = \sum_{d \in D} \{\begin{matrix} \sum_{(e_{i}, e_{j}) \in d} 1 - (\frac{{log}_{2} (χ_{i j})}{2}) \\ if (χ_{i j} < η) \\ 0 otherwise \end{matrix} \end{matrix}$ where d is a document in the corpus D, $e_{i}$ and $e_{j}$ are the ith and jth entities in document d, respectively. $χ_{i j}$ is the position distance between two entities with value 1 if entities $e_{i}$ and $e_{j}$ collocate in the same sentence, 2 in neighbouring sentence, and so on. The entities having position distance $χ_{i j}$ above the threshold η are ignored. We empirically set η to 4, which prevents connecting far entities. Obviously, the bigger the $θ_{i j}$ is, the bigger the $w_{i j}$ is. $w_{i j}$ is in the range $[0, 1]$ . In our implementations, we use only the Web pages of each ambiguous name in the given corpus as the entity co-occurrence corpus. To compute $w_{i j}$ , one can use large-scale corpora beyond the given corpus such as entity co-occurrence corpus. This is considered as one of the future work of this paper. Nonetheless, our experiments show that the given corpus is sufficient for extracting social links.

3.4.2. Profile enrichment

The sparse data contained in discourse profiles may not be sufficient to resolve ambiguities and the system robustness will be degraded due to low quality of AE component. For these reasons, we propose an enrichment method of the persons’ profile via global attributes extracted from distant knowledge base. Profile enrichment attempts to alleviate the problem of data sparseness and improve the robustness of AE. Profile enrichment includes two steps: (i) entity linking, and (ii) attribute extraction. In entity linking step, for an entity mention e, we determine its identity in text to identify the best matching entity in the distant knowledge base. In attribute extraction step, we retrieve the global attributes for the target person e from distant knowledge base. These attributes are beyond the local discourse profiles.

Entity linking phase takes as input the target entity mention e and the context S(e) (Eq. 1) around it. It identifies the entity mentions in neighbourhood of entity e and forms a list of co-occurring entity mentions $N (e) = {e_{1}, e_{2}, \dots, e_{m}}$ . Entity linking phase then extracts the intra-document co-reference chain of the entity e and entities in N(e). To match an entity mention e against an entity from distant knowledge base, our entity linking system creates a phrase query comprising mentions from the co-reference chain of e and its neighbour entities N(e). Entity linking system feeds the query to Babelfy [53] to identify the BabelNet synset id of the target entity e, and links it into the corresponding DBPedia URI. Babelfy is a state-of-the-art word sense disambiguation and entity linking system. As a matter of fact, using Babelfy is not mandatory; any disambiguation or entity linking strategy can be used at this stage. However, a knowledge-based unified approach like Babelfy is best suited to our setting. The attribute extraction phase takes as input the DBPedia URI of the target entity e, and retrieves its attribute from DBPedia ontology [9] to enrich discourse profile of entity e.

We primarily rely on the Babelfy itself to identify the correct identity of the target entity e. Babelfy may produce some noisy data because in some cases it cannot infer the correct identity of entities. Therefore, to avoid dependency on the output of the Babelfy to infer whether the retrieved external entity t best matches with the target entity e, we rank the candidate entity t by a similarity measure and prune out candidates with low confidence. In similarity computation, we first compare the type tag of the entities e and t. If the entity type tag of entities e and t are not the same, we ignore the external entity t; otherwise we compare the attributes of entity t with local attributes of the target entity e. For this purpose, we compute the normalized similarity between entities t and e based on their attributes: $\begin{matrix} (10) & Sim (e, t) = \frac{1}{| A_{e, t} |} \times \sum_{a \in A_{e, t}} β_{a} \times M_{a} (e, t) \end{matrix}$ where $M_{a} (e, t)$ is the similarity of the two entities based on attribute a, $β_{a}$ is the importance coefficient of attribute a, and $A_{e, t}$ represents all the attributes associated with both entities e and t. $A_{e, t}$ is equal to $A_{e, t} = (A_{e} \cup A_{t})$ , where $A_{e}$ and $A_{t}$ respectively represent the set of attributes associated with entity e and t. In current implementations, we set $β_{a}$ to 1. We define a confidence threshold T, such that a candidate entity t having the similarity value $Sim (e, t)$ below the threshold T is pruned out. If $Sim (e, t) ⩾ T$ , the system then appends the attributes of entity t to discourse profile of entity e. In our implementations, we empirically set T to 0.6.

In general, each attribute class $a \in A$ may be one of the following types: single-value attribute or multiple-value attribute. Single-value attribute (e.g., date of birth) can only take a single value, while multiple-value attribute (e.g., affiliation, occupation) can take one or more different values. If attribute a is a multiple-value attribute, to compute $M_{a} (e, t)$ , we first compute single-value similarities for all the possible values of a, and then aggregate the maximum single-value similarities. Therefore, we define $M_{a} (e, t)$ as follows: $\begin{array}{rcl} M_{a} (e, t) & = & \frac{1}{min (| I_{e} |, | I_{t} |)} \\ (11) & \times \sum_{p \in I_{e}} max (δ (p, I_{t})) \end{array}$ where $I_{e}$ and $I_{t}$ represent the item set of attribute a for person e and t, respectively. $δ (p, I_{t})$ is the set of single-value attribute similarities computed between element $p \in I_{e}$ and all elements in $I_{t}$ . We define $δ (p, I_{t})$ as follows: $\begin{matrix} (12) & δ (p, I_{v}) = {φ (p, q) | q \in I_{t}} \end{matrix}$ $φ (p, q)$ computes the similarity between item p and q using an appropriate standard similarity measure. Personal attributes are heterogeneous; therefore it isn’t reasonable to use the same similarity measure for different attributes in computing $φ (p, q)$ . This enforces us to use appropriate similarity measures for any type of the attribute. There are different standard similarity measures, each of which is appropriate for a particular attribute class. In order to determine which similarity measure is appropriate for an attribute class, we adopt the following methodology. First, we adopt five syntactic similarity measures, which include normalized Levenshtein distance (len) [75], Dice’s coefficient (dic) [45], Cosine (cos) [45], Jaccard index (jac) [41] and dates’ relative similarity (Spd) [23]. The reason to select these measures is that these measures are widely used in literature to calculate similarity of data objects. We then identify the appropriateness of a similarity metric for a particular class of attribute through assessing its importance on name disambiguation. To do this, we use ground truth of training data given in WePS-1 dataset [6] and WePS-2 dataset [8]. We analyse the similarity measures in turn for each person in ground truth of dataset. Each similarity measure was computed for the attributes of each pair of persons that are co-referent, i.e., they are in the same cluster and refer to the same entity in reality. A typical similarity metric is considered to be proper for a particular attribute, if it results in higher normalized similarity value for the people who are co-referent in the ground truth.

Fig. 5.

An example of mapping the sample Web page given in Fig. 2 to attribute-relationship graph; (a) discourse profile extracted by our profiling system and enriched with external global attributes; (b) attribute-relationship graph. Note that the weights of attribute edges and structure edges are not given here.

For attribute date of birth, we first normalize the date values using Stanford SUTime library [20] and then compute the similarity. Borrowing the idea presented in [23], to compare date of birth values, we first convert dates into a number of days. We calculate the number of days according to the fix date 01-01-2016. Let $d_{1}$ and $d_{2}$ be the two day values that are being compared, Spd, the dates’ relative similarity is calculated as follows: $\begin{array}{l} Spd (d_{1}, d_{2}) \\ (13) & = \{\begin{matrix} 1 - (\frac{p d (d_{1}, d_{2})}{p d_{max}}) p d (d_{1}, d_{2}) \\ < p d_{max} \\ 0 otherwise \end{matrix} \end{array}$ where $p d_{max}$ ( $0 < p d_{max} < 1$ ) is the maximum percentage difference that is tolerated in similarity computation. In our implementations, we empirically set $p d_{max}$ to 0.2. $p d (d_{1}, d_{2})$ is the percentage difference, which is defined as follows: $\begin{matrix} (14) & p d (d_{1}, d_{2}) = \frac{| d_{1} - d_{2} |}{max (d_{1}, d_{2})} \end{matrix}$

Results obtained by our experiments on the given datasets show that the normalized Levenshtein metric is appropriate for the attributes of affiliation, award, degree, nationality, occupation, and school; the Cosine similarity metric for the attributes of relatives, phone, fax, e-mail, website; and the Jaccard index for the attribute of mentor; and Dice coefficient for the attribute of birth place. For some attributes one or more similarity measures relatively reported the same results. For the attribute of other name, one can use Dice’s coefficient or Cosine similarity measure. For the attribute of major, it is no matter which similarity measure is used; however, in our implementations, we used the normalized Levenshtein metric for the attribute of major. We notice that for attribute date of birth, we only use Spd measure.

3.4.3. Graph creation

The graph creation component takes as input the social relationship graph and enriched discourse profiles associated with each ambiguous person name and then map them into an undirected graph G. Graph G is an undirected weighted graph summarizing all of the information about entities contained in the given Web text documents, and the given distant knowledge base. The remainder components of the name disambiguation system work with this rich graph instead of Web text documents. This enables us to use optimal graph mining algorithms for name disambiguation task. Figure 5 shows as example of mapping people’s attributes and social links extracted from the Web page given in Fig. 2 into a graph. Figure 5(a) shows the discourse profiles extracted by profile extraction system. In Fig. 5(a), the local attributes are shown in black colour and external attributes obtained by profile enrichment are shown in blue colour. The profile information and social links are mapped to a graph shown in Fig. 5(b). In Fig. 5(b), the structure node corresponding to target person “Dan Jurafsky” is shown in filled rectangle, the structure nodes for other persons are shown in rectangles, the structure nodes corresponding to organizations in triangles, attribute nodes in round ellipses, structure edges by solid lines, and attribute edges by dotted lines.

3.4.4. Graph clustering

Our procedure for clustering takes as input the attribute-relationship graph G, and uses the recently proposed BIC-Means [40], an efficient graph clustering algorithm to partition the graph into a set of disjoint clusters $C = {C_{1}, C_{2}, \dots, C_{K}}$ . BIC-Means is a bisecting version of the incremental K-means clustering algorithm equipped with Bayesian information criterion (BIC) [59] as a termination criterion. Our reasons to using BIC-Means as clustering algorithm comes from the fact that (i) it is an efficient algorithm, which has low computational cost to hierarchically clustering large-scale data sets, (ii) it does not need the number of clusters as input parameter, considering the notion that we do not know the number of clusters previously, and (iii) it is robust against the parameter setting. We run the BIC-Means clustering with the similarity metric of neighbourhood random walk distance (NRWD) [72], which measures the closeness between graph nodes. In clustering process, we use a blocking technique to avoid computational bottlenecks. By this way, we apply clustering algorithm on documents that are about an ambiguous person.

Once we have clustered the set of discourse profiles for a certain person e, the next step is to aggregate those attributes were found within the cluster belonging to that person and form his/her corpus-level profile. In other words, distributed information about a given person is integrated to form a unified, enriched corpus-level profile. In integration step, we design some inference rule to extract more implicit information. An example of such rules are given in following formula:

Relatives(cID, x, y) ∧ Relatives(cID, y, z) ∧ Person(cID, x) ∧ Person(cID, y) ∧ Person(cID, z) ⇒ Relatives(dID, x, z)

where cID indicates the cluster id. This rule infer there is a relative relation between person x and z. We design 19 such rules to infer implicit information contained in discourse profile.

4. Experiments and results

For the sake of evaluating our cross-document person profiling system comprehensively, we conduct our experiments at both levels: the component level and system level.

4.1. Component-level evaluation

At the component-level evaluation, we evaluate our approaches in three phases: evaluation of pre-processing, evaluation of attribute extraction, and evaluation of name disambiguation.

4.1.1. Experiments of the pre-processing components

As we mentioned before, the pre-processing tools may produce errors that leads to a degradations in the performance of subsequent components. For our implementations, we adopted Stanford named entity recognizer tool. It achieves F1 score of 92.29% on person names, 88.51% on locations and 81.72% on organizations evaluated on the CoNLL-2003 evaluation set [36]. We employed Stanford co-reference resolution system with 54.62% average F1 score on CoNLL 2011 shared task dev dataset7

⁷
http://nlp.stanford.edu/software/dcoref.shtml

[42]. For semantic role labelling, we use SENNA that obtained 75.49% F1 score on CoNLL 2005 dataset8

⁸

http://ronan.collobert.com/senna/

[26]. Although SENNA can not do semantic role labelling with 100% accuracy, we believe that it can be a very useful tool for semantic role labelling. For dependency parsing, we used Stanford dependency parser version 3.6.0 with over 80% F1 score.9

⁹

http://nlp.stanford.edu/software/stanford-dependencies.shtml

Here, we give the performance reported in the literature for pre-processing tools. We evaluated the pre-processing tools on sampled set of WePS-2 training and test datasets and observe that they achieved nearly the same performances as reported in their original papers.

4.1.2. Experiments of the attribute extraction component

For our experiments for evaluating the AE system we used two benchmark datasets: WePS-2 test dataset10

¹⁰
http://nlp.uned.es/weps/weps-2/weps2-data

[7] and Wikipedia_data dataset11

¹¹

http://cs.iit.edu/~culotta/data/wikipedia.html

[28]. Our reasons to using these datasets as benchmark are as follows: (i) they include the ground truth for discourse profile attributes, which is suit for evaluating our AE system, and (ii) they were used in a series of papers, which enables us to compare our method with other counterparts. WePS-2 test dataset consisted of collections of Web pages obtained from the results for a person name query to an Internet search engine. It consisted of 3444 Web pages retrieved for 30 person names. These person names were chosen from three different sources including Wikipedia, ACL’08 and US census data. Wikipedia_data was created in collaboration between the University of Massachusetts and Google, Inc. It consists of 1110 paragraphs from 441 pages from the Wikipedia with 4681 relation instances and 53 relation types labelled. In the dataset, the links between pages have been annotated with a relation type, e.g. nationality, award, etc. The dataset was split into training and testing sets (70-30% split). In our experiments, and among 53 relation types, we chose to extract 5 types of relations including “affiliation”, “award”, “birth place”, “nationality”, and “relatives”. We notice that the attribute of “relatives” in original dataset is divided into several fine-grained family relations such as father, and spouse. However, in our implementations, we consider all family relations under category of “relatives”.

In the remainder of this paper, WAE is short for structured-style Web-specific patterns, VAE for verb-based AE, and NAE for noun-based AE. Table 4 shows the macro-averaged performance scores12

¹²

Macro-averaged score consists of computing the performance score for every test set (person entity) and then averaged over all test sets.

obtained by our AE approaches on WePS-2 test dataset. As shown in Table 4, for WePS-2 test dataset, the VAE has high precision, which is greater than NAE and WAE. The second place in terms of precision belongs to NAE, and the third place belongs to WAE. The performance order of algorithms in terms of recall is WAE > NAE > VAE. The final integrated AE system complements three AE methods and achieves the best performances. This justifies that our integrated method is robust for Web data. Table 5 shows the macro-averaged performance scores for Wikipedia test dataset. As shown in Table 5, the performance order of algorithms in terms of F1 score is VAE > NAE > WAE. The high scores for VAE is due to the fact the Wikipedia test data is in formal style format. The final combination AE system (WAE + VAE + NAE) achieves the best performances.

Table 4

Performances of our AE methods on the WePS-2 test dataset

Method	P (%)	R (%)	F1 (%)
WAE	26.15	10.31	14.79
VAE	35.60	4.10	7.35
NAE	34.05	5.76	9.85
WAE + VAE + NAE	35.28	13.15	19.16

Table 5

Performances of our AE methods on the Wikipedia test dataset

Method	P (%)	R (%)	F1 (%)
WAE	56.40	11.54	19.16
VAE	82.04	35.04	49.11
NAE	79.37	22.51	35.07
WAE + VAE + NAE	80.29	69.52	74.52

Table 6 shows the detailed performance of the individual attributes obtained by our approaches on WePS-2 test dataset. Table 7 presents the performances for Wikipedia test dataset. Table 6 and Table 7 clearly show the contribution of different AE method for each particular attribute class. In the tables, the term “Same” means that the performance of the target AE method is as the previous one.

Table 6

Performances of attributes obtained by our AE methods on the WePS-2 test dataset in terms of F1 score

Method	Affiliation	Award	Birth place	Date of birth	Degree	Email	Fax	Major
WAE + VAE + NAE	7.1	8.11	17.81	13.5	21.7	44.7	50.17	4.52
	9.2	12.1	42.64	38.6	21.91	Same	Same	4.86
	11.4	13.65	42.71	Same	25.13	Same	Same	5.81
	Mentor	Nationality	Occupation	Other name	Phone	Relatives	School	Website
WAE + VAE + NAE	1.01	12.15	12.81	69.15 *	34.6	3.43	6.1	18.06
	Same	15.55	19.32	Same	Same	Same	6.7	Same
	1.53	21.83	24.93	Same	Same	4.02	7	Same

We use Stanford co-reference system to extract the attribute of “Other name”.

Table 7

Performances of attributes obtained by our AE methods on the Wikipedia test dataset in terms of F1 score

Method	Affiliation	Award	Birth place	Nationality	Relatives
WAE + VAE + NAE	11.2	22.5	52.7	9.75	20.62
	37.01	61.7	90.2	36.72	Same
	63.4	70.88	90.51	93.5	38.5

For WePS-2 test dataset (Table 6), we find that the WAE method have achieved good performance for the attributes of “email”, “phone”, “fax”, and “website”. This is due to the fact that the instances for these attributes are often expressed with fixed, easily predictable patterns in structured-style fragments. Similarly, the VAE method have achieved good performance for the attributes of “birth place”, “date of birth”, and “occupation”, because their instances follow a specific format and are in sentences having a verb predictive of the target attribute. The performance scores of some attributes such as “nationality”, “affiliation”, “relatives”, “occupation”, and “degree” improved after incorporating NAE method. This is consisted with the notion that the majority of instances for these attributes are from noun-based constructions. We notice that integrating different AE methods can improve the performance of only some specific attributes. This is due to the fact that each AE method is appropriate for only some types of attributes. Thus, in final combination, incorporating some AE methods for some attributes does not significantly affect the performance. This is obvious from Table 6, where the attributes of “email”, “phone”, “fax”, and “website” has achieved good performances solely using the WAE method, and is remained fixed even after incorporating the VAE and NAE methods.

Nonetheless, the final hybrid AE can complement the performances of some attributes such as “nationality”, “degree”, and “occupation”, and further improves the overall performance. We observe that for the attributes of “major”, “mentor”, “affiliation”, “relatives”, “school”, and “nationality” in WePS-2 dataset, none of the methods cannot achieve good performances. This shows that more robust methods are required for these sorts of attributes. For Wikipedia test dataset (Table 7), which mainly consists of formal-style text, incorporating the VAE and NAE significantly improved performance scores. This shows that our formal-style based AE methods perform fairly well on formal-style fragments.

We compare our AE approach with five state of the art approaches achieving the best published results for benchmark datasets. The counterparts include UvA_2 [11], CASIANED [38], Chen et al. method [21], Yu and Lam method [74], and ImplIE [61]. UvA_2 system is developed by the university of Amsterdam team at WePS-2 sharetask [7]. This method uses lists of pre-compiled keywords and Web-specific patterns for the personal AE. The CASIANED system is a two-step AE approach. First, the system extracts candidate attribute classes using several IE methods. Then, the attribute candidates are verified through classification. The method of Chen et al. [21] is a prominent combination AE system. This baseline uses combination of various rule-based and machine learning methods for personal AE. It achieved the best performance in the WePS-2 sharetask. The method of Yu and Lam [74] is an integrated relation extraction approach, which uses MLNs to extract encyclopedia relations from Wikipedia articles. This approach achieved high performances for different relations in Wikipedia dataset. The ImplIE system uses syntactic dependency rules to find relations that are beyond the scope of Open IE extractions. ImplIE begins with user-collected semantic taggers for a set of attribute classes. It then identifies all candidate noun phrases for an attribute class that are modified by one of the terms specified for that attribute. Given candidate noun phrases, dependency rules are applied to find correct sub-strings as a filler value of the target attribute. We compared the results obtained by our method with those reported by UvA_2 [11], CASIANED [38] and Chen et al. method [21] on WePS-2 test dataset; Yu and Lam method [74] on Wikipedia test dataset; and ImplIE [61] on both WePS-2 test and Wikipedia test datasets. Since ImplIE system mainly focused on the extraction of noun-based attributes, thus to fair comparison, we compare ImplIE system only with our NAE method. Table 8 summarizes the average performance obtained by our proposed AE method and state of the art approaches for the WePS-2 test dataset. From Table 8, we find that our method outperforms the other approaches. Our method achieves 11.56%, 7.46%, and 0.27% average improvement on WePS-2 test dataset in terms of F1 score, compared to the UvA_2, CASIANED, and Chen et al. [21] method, respectively.

Table 8

Performances of our AE systems and state of the art methods on the WePS-2 test dataset

Method	P (%)	R (%)	F1 (%)
UvA_2 [11]	4.4	27.4	7.6
CASIANED [38]	8.5	19	11.7
Chen et al. [21]	31.9	13.42	18.89
Our method (WAE + VAE + NAE)	35.28	13.15	19.16

Table 9

Performances of our AE systems and the best prior method on the Wikipedia test dataset

Method	P (%)	R (%)	F1 (%)
Yu and Lam [74]	76.17	70.4	73.17
Our method (WAE + VAE + NAE)	80.29	69.52	74.52

Table 9 compares our approach on the Wikipedia test dataset against the method of Yu and Lam [74]. As shown in Table 9, our approach achieves an F1 score of 74.52%, providing an improvement of about 1.35% F1 score points. This shows that our integrated approach is also efficient for the extraction of attributes from homogeneous Web corpora.

To compare our NAE method with ImplIE system, we first sampled 100 random noun-based attributes from the attributes provided in ground truth. We then manually judge the extractions produced by our NAE and ImplIE for these attributes. We summarized the results in Table 10. The results clearly show that our method outperformed ImplIE system and achieved higher scores. The main reason to the low score of ImplIE is that it cannot correctly extract the multi-word (long tail) attributes. For example, in the phrase “David Weir, associate professor of economics at Yale University”, the correct value for the attribute of “occupation” is the “associate professor of economics”, but the ImplIE system produces the term “professor”, which is incorrect according to ground truth’s annotation guides.

Table 10

Performances of our NAE method and IMPLIE system on sampled sentences from WePS-2 test and Wikipedia test dataset

Method	WePS-2 test dataset			Wikipedia test dataset

	P (%)	R (%)	F1 (%)	P (%)	R (%)	F1 (%)
IMPLIE [61]	43.2	12.31	1 9.16	74.06	26.20	38.71
NAE	45.06	13.55	20.83	75.39	27.47	40.27

Table 11

The performance of AE system with considering semantic enrichment (AE⁺) and without considering semantic enrichment (AE⁻) on WePS-2 test dataset, in terms of F1 score

Method	Affiliation	Award	Birth place	Date of birth	Degree	Email	Fax	Major
AE⁻	9.28	10.35	40.68	38.6	24.07	44.7	50.17	4.33
AE⁺	9.83	12.5	42.71	38.6	25.13	44.7	50.17	5.81
	Mentor	Nationality	Occupation	Other name	Phone	Relatives	School	Web site
AE⁻	1.31	19.6	23.1	69.15	34.6	3.47	5.47	18.06
AE⁺	1.53	21.83	24.93	69.15	34.6	4.02	7	18.06

In summary, the counterpart approaches could not extract entirely the attribute values contained in noun-based and verb-based constructions. Furthermore, these methods have problem with multiple-word attribute values. In this paper, we solved this problem by adopting the semantic analysis of the text. The results indicate that incorporating verb-based AE, noun-based AE, and Web-specific patterns, and also deep semantic analysis of the text is effective in increasing performance of the AE system.

According to the results given in Tables 4–10, we observe that the results look promising but are not ideal. This shows that the Web person AE is far from being solved. Thus more effort is needed in this respect. Our manual investigation over incorrect extractions shows that the performance of AE can be raised if we perform the following work:

Creating more robust AE method: overall, existing AE methods reports low F1 score for some attributes on question. This indicates that AE in Web documents is still a big challenge. Obviously, the more precise the AE methods are, the higher the performance scores are. Therefore, if we spend more time in the development of more robust AE methods, the system performance will pick up. Our AE approach provides a flexible framework, which other more robust AE modules can be incorporated to improve the result.

Table 12

The performance of AE system with considering semantic enrichment (AE⁺) and without considering semantic enrichment (AE⁻) on Wikipedia test dataset in terms of F1 score

Method	Affiliation	Award	Birth place	Nationality	Relatives
AE⁻	61.07	67.35	88.9	92.8	37.73
AE⁺	62.4	68.17	90.75	93.15	38.05

Improving the performance of pre-processing components: our manual investigation reveals that almost half of the incorrect extractions were because of the inefficiency of pre-processing and semantic analysis stages, and not because of the inefficiency of AE method. Errors in pre-processing and semantic analysis stages are propagated to AE step and cause wrong extractions. For example, the inefficiency of named entity recognition system causes our AE system cannot identify the border of attribute values accurately. The low performance of pre-processing stages is a bottleneck for efficient AE. However, improving pre-processing and semantic analysis is orthogonal to our problem and therefore out of the scope of this paper. Nonetheless, to alleviate the errors in semantic analysis stage, we enrich the analyzed text with semantic information extracted from a distant ontology. Table 11 shows the effect of semantic enrichment on WePS-2 test dataset. Table 12 shows the results for Wikipedia test dataset. In the tables, ${A E}^{+}$ shows the scenario in which semantic enrichment is considered, and ${A E}^{-}$ shows the scenario when the semantic enrichment is completely disregarded. Without considering the enrichment stage, the errors in semantic analysis stage leads to degradation of AE performance. This clearly shows the effect of semantic enrichment in efficient AE. Notice that in Table 4–10, we give the results obtained by ${A E}^{+}$ .

Enriching discourse profile: in this paper, we focused on the extraction of attributes only from the given Web pages in the input corpus. Since the Web pages are often noisy, irregular and contain incomplete information about entities, the AE system cannot extract rich profile attributes. One of the promising solutions to improve the result and alleviate the issue of data sparseness is to enrich local discourse profile with semantic information inferred from distant knowledge bases. We use this possibility in name disambiguation, which leads to more robust name resolution. However, we do not consider the distant attributes in the evaluation of AE system, because we concerned to answer the question “how much information can be extracted if the AE system uses only the information contained in the input corpus?”

4.1.3. Experiments for name disambiguation

We used two standard datasets as our benchmark to evaluate our name disambiguation system. These datasets include WePS-1 test dataset [5], and WePS-2 test dataset [7]. These datasets provide a real corpus, which can test a disambiguation system for personal names with varying ambiguity and in different domains. Both WePS-1 and WePS-2 test datasets consisted 30 Web page collections, each one corresponding to one ambiguous person name. Each Web page collection consists of N top ranked Web pages (100 in WePS-1 test dataset and 150 in WePS-2). We conducted evaluations using two types of scoring measures, B-cubed scoring measure, and the purity-based scoring measure. We use three B-cubed measures including B-cubed precision ( $B^{3} P$ ), recall ( $B^{3} R$ ), and F-score ( $B^{3} F_{α}$ ). A more detailed discussion of these quality metrics is given in [2,17]. In addition to B-cubed measures, we used three purity-based scoring measures, including purity (Pr), inverse purity (IPr), and harmonic mean of purity and inverse purity ( $F_{p}$ ). A more detailed discussion of purity-based metrics is given in [2].

Table 13
Performances of our name disambiguation approaches on WePS-1 test datasets

Method $B^{3} P$ $B^{3} R$ $B^{3} F_{α = 0.5}$ Pr IPr $F_{p = 0.5}$

Social links + Local attributes + external attributes 67.3 73.7 68.1 71.5 87.1 80.4

74.4 80.5 75.2 78.6 89.3 82.2

75.2 81.6 77.1 79.1 90.8 84.26

Method	$B^{3} P$	$B^{3} R$	$B^{3} F_{α = 0.5}$	Pr	IPr	$F_{p = 0.5}$
Social links + Local attributes + external attributes	67.3	73.7	68.1	71.5	87.1	80.4
74.4	80.5	75.2	78.6	89.3	82.2
75.2	81.6	77.1	79.1	90.8	84.26

Table 13 shows the results obtained by our name disambiguation methods on WePS-1 test dataset. Table 14 shows the results for WePS-2 test dataset. In order to indicate the effect of each clustering feature, in Table 13 and 14, we begin with the feature of social links and then add features of local discourse profile attributes and external attributes one by one. The results clearly show the effect of profile enrichment and integrating attributes with social relationships. In Table 13 and 14, we notice that the performance is consistently increasing when incorporating more clustering features. The final feature model (social links + local attributes + external attributes) achieves the best performances. The results justify that the majority of information for name disambiguation is given in the Web pages being processed. However, incorporating the external attributes improves performances.

Table 14

Performances of our name disambiguation approaches on WePS-2 test datasets

Method	$B^{3} P$	$B^{3} R$	$B^{3} F_{α = 0.5}$	Pr	IPr	$F_{p = 0.5}$
Social links + Local attributes + external attributes	66.3	74.5	68.7	68.2	87.4	72.0
	84.7	80.1	82.5	83.6	89.24	86.3
	86.2	82.9	84.05	85.4	90.8	88.63

Table 15

Comparison of results obtained by baselines and our method on WePS-1 test dataset

Method	$B^{3} P$	$B^{3} R$	$B^{3} F_{α = 0.5}$	Pr	IPr	$F_{p = 0.5}$
BOW	62.1	75.5	69.3	70.4	85	75.5
SN	65	73.5	68.6	69.7	89.2	78.5
AV	59.4	68.4	60.7	62.1	71.4	62.5
ALL_IN_ONE	44	100	56	59	100	71.4
ONE_IN_ONE	100	20	32	100	22	34
Our method	75.2	81.6	77.1	79.1	90.8	84.26

Table 16

Comparison of results obtained by baselines and our method on WePS-2 test dataset

Method	$B^{3} P$	$B^{3} R$	$B^{3} F_{α = 0.5}$	Pr	IPr	$F_{p = 0.5}$
BOW	66.2	80.5	72.5	78.1	82.0	76.8
SN	64.2	81.1	67.3	64.2	90.7	70.5
AV	62.5	77.7	61.4	62.6	79.4	67.3
ALL_IN_ONE	43.0	100	53.0	56.0	100	67.2
ONE_IN_ONE	100	24.0	34.0	100	24.0	34.0
Our method	86.2	82.9	84.05	85.4	90.8	88.63

Table 17

Comparison of results obtained by state-of-the-art methods and our method on WePS-1 test and WePS-2 test dataset

		Han and Zhao [39]	Yerva et al. [73]	Chen et al. [21]	Dutta and Weikum [30]	Our method
WePS-1 test	$B^{3} F_{α = 0.5}$	–	–	76	–	77.1
WePS-1 test	$F_{p = 0.5}$	84	–	90	–	84.26
WePS-2 test	$B^{3} F_{α = 0.5}$	–	74.7	82	83.48	84.05
WePS-2 test	$F_{p = 0.5}$	86	78.8	89	–	88.63

Table 15 shows the best performance obtained from the baselines and our method on WePS-1 test dataset. Table 16 shows the results for WePS-2 test dataset. As shown in Table 15 and 16, our method clearly outperforms the baseline methods for both datasets in terms of both $B^{3} F_{α = 0.5}$ and $F_{p = 0.5}$ . For WePS-1 dataset, on average our method outperforms BOW, SN, AV, ALL_IN_ONE, and ONE_IN_ONE by +7.8%, 8.5%, 16.4%, 21.1%, and 45.1%, respectively in terms of $B^{3} F_{α = 0.5}$ , and +8.76%, 5.76%, 21.76%, 12.86% and 50.26%, respectively in terms of $F_{p = 0.5}$ . The improvement is also evident for WePS-2 dataset, in which our method obtains +11.55%, 16.75%, 22.65%, 31.05%, and 50.05% improvement compared to BOW, SN, AV, ALL_IN_ONE, and ONE_IN_ONE, respectively, in terms of $B^{3} F_{α = 0.5}$ , and with respect to $F_{p = 0.5}$ , the improvements are +11.83%, 18.13%, 21.33%, 21.43% and 54.63%, respectively. The ONE_IN_ONE baseline obtained the best result in terms of $B^{3} P$ and Pr measure on both WePS-1 and WePS-2 dataset. The ALL_IN_ONE baseline outperformed other algorithms in terms of $B^{3} R$ and IPr measure. The higher $B^{3} P$ and Pr for ONE_IN_ONE baseline arises from the fact that in WePS-1 and WePS-2 datasets documents are distributed among the clusters. Since in average half of the documents in the dataset belong to one specific person, the ALL_IN_ONE baseline gave better results in terms of IPr and $B^{3} R$ .

Table 17 summarizes the average performance obtained by our proposed method and four state of the art methods for the benchmark datasets. We compared the results obtained by our method with those reported in Han and Zhao [39] and Chen et al. [27] on WePS-1 and WePS-2 test dataset, Dutta and Weikum [30] and Yerva et al. [73] on WePS-2 test dataset. We notice that the comparison is not precise, because the mentioned algorithms implemented and tested with different settings on machines with different processing characteristics. In Table 17, Han and Zhao [39] did not report obtained B-cubed scores. As shown in Table 17, our approach performs well on datasets, exceeding or matching the best performance obtained by the state of the art methods in terms of $B^{3} F_{α = 0.5}$ .

4.2. System-level evaluation

At the system level, we evaluate the performance of the end-to-end aggregated EP system. We use the standard WePS-3 test dataset [4] as our benchmark. In the experiments, we conducted evaluations in terms of precision, recall and $F_{α}$ score,13

¹³
$F_{α} = 1 / (α \times 1 / P + (1 - α) \times 1 / R)$ , where P and R are precision and recall, respectively.

where α is set to 0.5. Table 18 compares our EP system with the state of the art RGAI_AE_3 system [54], which has achieved the best published results for WePS-3 dataset. Our method reports an overall 20.56%

F_{α}

score. It can be seen that our method outperforms the RGAI_AE_3 algorithm for profile extraction problem (+2.16% better than RGAI_AE_3 in terms of the average

F_{α}

score). The low rate for systems’ precision and recall justifies that the system could not accurately extract all of the information about entities. This indicates that the cross-document person profile extraction from Web is still a big challenge. Therefore, more effort is needed in this respect.

Table 18

Performance of state of the art and our EP approach on WePS-3 dataset

Method	P (%)	R (%)	$F_{α = 0.5}$ (%)
RGAI_AE_3 [54]	21.38	24.48	18.4
Our method	24.15	25.8	20.56

5. Conclusion

In this paper, we developed a cross-document person profiling system that extracts persons in question and their relevant information from Web pages. We mainly focused on the core task of profiling system, namely attribute extraction. For attribute extraction, we use different information extraction methods according to Web text expression format and attribute type. Our approach relies on deep semantic analysis of the text and utilizes the syntactic parsing and semantic role labelling technologies for attribute extraction. Our attribute extraction approach can extract both explicit and implicit attributes within and across sentences. The resulting profile information is meaningful and linked to the existing knowledge bases, which facilitates the multi-lingual information processing. Experiments show that our proposed methods for profiling subtasks achieved respectable results. There are several interesting directions for future work. One of the most promising directions is to use semantic-based machine learning techniques to automatically learn attribute extraction MLN rules with a minimum of training and human knowledge engineering effort. Since the final results of entity profiling system depend on the performance of four subtasks including pre-processing, semantic analysis, attribute extraction, and name disambiguation, therefore, second interesting future work is to improve the pre-requisites’ performance, which eventually can improve the overall quality of system. Our third future work is to develop a multi-lingual entity profiling system that can extract all entity-related information from multi-lingual text documents. Finally, we plan to generalize our approach to profile generic entity type, where entities are not limited to people and entity’s profile schema is not specified in advance. We expect the generalized system can work with minimum training data for a new domain.

References

Alani,

Kim,

D.E.

Millard,

M.J.

Weal,

Hall,

P.H.

Lewis and

N.R.

Shadbolt, Automatic ontology-based knowledge extraction from Web documents, IEEE Intell. Syst. 18(1) (2003), 14–21. doi:10.1109/MIS.2003.1179189.

Amigó,

Gonzalo,

Artiles and

Verdejo, Evaluation metrics for clustering tasks: A comparison based on formal constraints, Inf. Retr. Boston. 12(4) (2007), 461–486. doi:10.1007/s10791-008-9066-8.

Arnold and

Rahm, Automatic extraction of semantic relations from Wikipedia, Int. J. Artif. Intell. Tools 24(2) (2015), 1–37. doi:10.1142/S0218213015400102.

Artiles,

Borthwick,

Gonzalo,

Sekine and

Amig, WePS-3 evaluation campaign: Overview of the Web people search clustering and attribute extraction tasks, in: CLEF (Notebook Papers/LABs/Workshops), 2010.

Artiles,

Gonzalo and

Sekine, The SemEval-2007 WePS evaluation: Establishing a benchmark for the Web people search task, in: Proceedings of the 4th International Workshop on Semantic Evaluations (SemEval), Prague, Czech Republic, 2007, pp. 64–69. doi:10.3115/1621474.1621486.

Artiles,

Gonzalo and

Artiles,

Gonzalo and

Sekine, Weps 2 evaluation campaign: Overview of the web people search clustering task, in: 2nd Web People Search Evaluation Workshop (WePS 2009), 18th WWW Conference, 2009.

Artiles,

Gonzalo and

Sekine, Weps 2 evaluation campaign: Overview of the web people search clustering task, in: 2nd Web People Search Evaluation Workshop (WePS 2009), 18th WWW Conference, 2009.

Auer,

Bizer,

Kobilarov,

Lehmann,

Cyganiak and

Ives, DBpedia: A nucleus for a Web of open data, in: Proc. 6th Int. Semant. Web Conf, Vol. 4825 LNCS, ISWC, Busan, Korea, 2007, pp. 722–735.

10.

Bach and

Badaskar, A review of relation extraction, Lit. Rev. Lang. Stat. II (2007).

11.

Balog,

He,

Monz,

Tsagkias,

Hofmann,

Jijkoun,

Weerkamp and

De Rijke, The university of Amsterdam at WePS2, in: 2nd Web People Search Evaluation Workshop (WePS 2009), 18th WWW Conference, 2009.

12.

Banko,

Cafarella and

Soderland, Open information extraction from the Web, in: International Joint Conferences on Artificial Intelligence, 2007, pp. 2670–2676.

13.

Barla,

Tvarožek and

Bielikov´, Rule-based user characteristics acquisition from logs with semantics for personalized Web-based systems, Comput. Informatics 28(4) (2009), 399–427.

14.

Bonial,

Stowe and

Palmer, Renewing and revising SemLink, in: The GenLex Workshop on Linked Data in Linguistics, 2013, pp. 9–17.

15.

Brandes and

Erlebach, Network Analysis: Methodological Foundations, Vol. 3418, Springer Science & Business, Media, 2005.

16.

R.C.

Bunescu and

R.J.

Mooney, A shortest path dependency kernel for relation extraction, in: Proceedings of the Human Language Technology Conference and Conference on Empirical Methods in Natural Language Processing, 2005, pp. 724–731. doi:10.3115/1220575.1220666.

17.

Cai and

Strube, Evaluation metrics for end-to-end coreference resolution systems, in: Proceedings of SIGDIAL 2010: The 11th Annual Meeting of the Special Interest Group on Discourse and Dialogue, 2010, pp. 28–36.

18.

Carreras and

Màrquez, Introduction to the CoNLL-2005 Shared Task: Semantic Role Labeling, 2005.

19.

A.X.

Chang and

C.D.

Manning, SUT IME: A library for recognizing and normalizing time expressions, in: LREC, 2012, pp. 3735–3740.

20.

A.X.

Chang and

C.D.

Manning, SUTIME: A library for recognizing and normalizing time expressions, in: LREC, 2012, pp. 3735–3740.

21.

Chen,

S.Y.

Mei Lee and

C.R.

Huang, A robust web personal name information extraction system, Expert Syst. Appl. 39(3) (2012), 2690–2699. doi:10.1016/j.eswa.2011.08.125.

22.

Chen,

Tamang,

Lee,

Li,

Lin,

Snover,

Artiles,

Passantino and

Ji, CUNY-BLENDER TAC-KBP2010 entity linking and slot filling system description, in: Proc. Text Analysis Conference (TAC 2010), 2010.

23.

Christen, Data Matching Concepts and Techniques for Record Linkage, Entity Resolution, and Duplicate Detection, Springer, Heidelberg New York Dordrecht London, 2012.

24.

Christensen,

Soderland and

Etzioni, Semantic role labeling for open information extraction, in: Proceedings of the NAACL HLT 2010 First International Workshop on Formalisms and Methodology for Learning by Reading, 2010, pp. 52–60.

25.

K.W.

Church,

Hill and

Hanks, Word association norms, mutual information, and lexicography, Comput. Linguist. 16(1) (1990), 22–29.

26.

Collobert,

Weston,

Bottou,

Karlen,

Kavukcuoglu and

Kuksa, Natural language processing (almost) from Scratch, J. Mach. Learn. Res. 12 (2011), 2461–2505.

27.

Cortes and

Vapnic, Support-vector networks, Mach. Learn. 20 (1995), 273–297.

28.

Culotta,

McCallum and

Betz, Integrating probabilistic extraction models and data mining to discover relations and patterns in text, in: Proceedings of the Main Conference on Human Language Technology Conference of the North American Chapter of the Association of Computational Linguistics, 2006, pp. 296–303.

29.

Delli Bovi,

Telesca and

Navigli, Large-scale information extraction from textual definitions through deep syntactic and semantic analysis, Trans. Assoc. Comput. Linguist. 3 (2015), 529–543.

30.

Dutta and

Weikum, Cross-document co-reference resolution using sample-based clustering with knowledge enrichment, Trans. Assoc. Comput. Linguist. 3 (2015), 15–28.

31.

Eichler,

Hemsen and

Neumann, Unsupervised relation extraction from Web documents, in: Proceeding of the 6th International Conference on Language Resources and Evaluation, 2008, pp. 1674–1679.

32.

Emami,

Shirazi and

A.A.

Barforoush, A semantic approach to person profile extraction from farsi documents, Journal of Information Systems and Telecommunication 4(4) (2016).

33.

Emami,

Shirazi,

A.A.

Barforoush and

Hourali, A pattern-matching method for extracting personal information in farsi content, U.P.B. Sci. Bull., Ser. C 78(1) (2016), 125–138.

34.

Exner and

Nugues, Using semantic role labeling to extract events from Wikipedia, in: CEUR Workshop Proceedings, 2011, pp. 38–47.

35.

Fellbaum, WordNet: An Electronic Lexical Database, MIT Press, 1998.

36.

R.J.

Finkel,

Grenager and

Manning, Incorporating non-local information into information extraction systems by Gibbs sampling, in: Proceedings of the 43rd Annual Meeting on Association for Computational Linguistics (ACL 2005), 2005, pp. 363–370.

37.

Gregory,

Mcgrath,

Bell,

K.O.

Hara and

Domico, Domain independent knowledge base population from structured and unstructured data sources, in: Twenty-Fourth International FLAIRS Conference, 2011, pp. 251–256.

38.

Han and

Zhao, CASIANED: People attribute extraction based on information extraction, in: 2nd Web People Search Evaluation Workshop (WePS 2009), 18th WWW Conference, 2009.

39.

Han and

Zhao, Structural semantic relatedness: A knowledge-based method to named entity disambiguation, in: Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics, 2010, pp. 50–59.

40.

Hourdakis,

Argyriou,

E.G.M.

Petrakis and

E.E.

Milios, Hierarchical clustering in medical document collections: The BIC-means method, J. Digit. Inf. Manag. 8(2) (2010), 71–77.

41.

Koutrika,

Bercovitz and

Garcia-Molina, FlexRecs: Expressing and combining flexible recommendations, in: SIGMOD, 2009, pp. 745–757. doi:10.1145/1559845.1559923.

42.

Lee,

Chang,

Peirsman,

Chambers,

Surdeanu and

Jurafsky, Deterministic coreference resolution based on entity-centric, precision-ranked rules, Comput. Linguist. 39(4) (2013), 885–916. doi:10.1162/COLI_a_00152.

43.

Lee and

Wang, Attribute extraction and scoring: A probabilistic approach, in: ICDE, 2013, pp. 194–205.

44.

Li,

Srihari,

Niu and

Li, Entity profile extraction from large corpora, in: Pacific Association for Computational Linguistics Conference (PACLING-2003), 2003.

45.

Liu, Discriminant analysis and similarity measure, Pattern Recognit. 47(1) (2014), 359–367. doi:10.1016/j.patcog.2013.06.023.

46.

Loper,

Yi and

Palmer, Combining lexical resources: Mapping between propbank and verbnet, in: Proceedings of the 7th International Workshop on Computational Linguistics, Tilburg, The Netherlands, 2007.

47.

Lowd and

Domingos, Efficient weight learning for Markov logic networks, in: PKDD 2007, 2007.

48.

C.D.

Manning,

Surdeanu,

Bauer,

Finkel,

S.J.

Bethard and

McClosky, The stanford CoreNLP natural language processing toolkit, in: Proceedings of 52nd Annual Meeting of the Association for Computational Linguistics: System Demonstrations, 2014, pp. 55–60. doi:10.3115/v1/P14-5010.

49.

Min and

Grishman, Challenges in the knowledge base population slot filling task, in: LREC, 2012, pp. 1137–1142.

50.

Minkov,

R.C.

Wang and

W.W.

Cohen, Extracting personal names from email: Applying named entity recognition to informal text, Comput. Linguist. (2005), 443–450.

51.

Mintz,

Bills,

Snow and

Jurafsky, Distant supervision for relation extraction without labeled data, in: Proceedings of the 47th Annual Meeting of the ACL and the 4th IJCNLP of the AFNLP, 2009, pp. 1003–1011.

52.

Moro and

Navigli, Integrating syntactic and semantic analysis into the open information extraction paradigm, in: Proceedings of the Twenty-Third International Joint Conference on Artificial Intelligence, 2013, pp. 2148–2154.

53.

Moro,

Raganato and

Navigli, Entity linking meets word sense disambiguation: A unified approach, Trans. Assoc. Comput. Linguist. 2 (2014), 231–244.

54.

Nagy, Person attribute extraction from the textual parts of Web pages, Acta Cybern. 20(3) (2012), 419–440. doi:10.14232/actacyb.20.3.2012.4.

55.

Niu and

Ré, Tuffy: Scaling up statistical inference in Markov logic networks using an RDBMS, in: Proceedings of the VLDB Endowment, Vol. 4, 2011, pp. 373–384.

56.

Palmer,

Gildea and

Kingsbury, The proposition bank: An annotated corpus of semantic roles, Comput. Linguist. 31(1) (2005), 71–106. doi:10.1162/0891201053630264.

57.

D.M.W.

Powers, Evaluation: From precision, recall and F-measure to ROC, informedness, markedness and correlation, J. Mach. Learn. Technol. 2(1) (2011), 37–63.

58.

Richardson and

Domingos, Markov logic networks, Mach. Learn. 62 (2006), 107–136. doi:10.1007/s10994-006-5833-1.

59.

G.E.

Schwarz, Estimating the dimension of a model, Ann. Stat. 6(2) (1978), 461–464. doi:10.1214/aos/1176344136.

60.

Soderland,

Gilmer,

Bart,

Etzioni and

Weld, Open information extraction to KBP relations in 3 hours, in: Proceedings of TAC-KBP 2013, 2013.

61.

Soderland,

Hawkins,

G.L.

Kim and

D.S.

Weld, University of Washington system for 2015 KBP cold start slot filling, in: Proceedings of TAC-KBP 2015, 2015.

62.

Soderland,

Roof,

Qin and

Xu, Adapting open information extraction to domain-specific relations, AI Mag. 31(3) (2010), 93–102. doi:10.1609/aimag.v31i3.2305.

63.

F.M.

Suchanek,

Ifrim and

Weikum, Combining linguistic and statistical analysis to extract relations from Web documents, in: Proceedings of KDD, 2006, pp. 712–717.

64.

Sun,

Grishman and

Sekine, Semi-supervised relation extraction with large-scale word clustering, in: Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics, 2011, pp. 521–529.

65.

Surdeanu,

Harabagiu,

Williams and

Aarseth, Using predicate-argument structures for information extraction, in: Proceedings of the 41st Annual Meeting on Association for Computational Linguistics – ACL’03, 2003, pp. 8–15. doi:10.3115/1075096.1075098.

66.

Surdeanu and

Ji, Overview of the English slot filling track at the TAC2014 knowledge base population evaluation, in: Proceedings of Text Analysis Conference (TAC2014), 2014.

67.

J.I.E.

Tang, A combination approach to Web user profiling, ACM Trans. Knowl. Discov. from Data 5(1) (2010), 1–44. doi:10.1145/1870096.1870098.

68.

Watanabe,

Bollegala,

Matsuo and

Ishizuka, A two-step approach to extracting attributes for people on the Web, in: 2nd Web People Search Evaluation Workshop (WePS 2009), 18th WWW Conference, 2009.

69.

Wu and

D.S.

Weld, Autonomously semantifying Wikipedia, in: Proceedings of CIKM’ 07, 2007, pp. 41–50. doi:10.1145/1321440.1321449.

70.

Xu, Bootstrapping Relation Extraction from Semantic Seeds, Saarland University, 2007.

71.

Yahya,

S.E.

Whang,

Gupta and

Halevy, ReNoun: Fact extraction for nominal attributes, in: Proceedings of EMNLP, 2014, pp. 325–335.

72.

Yang,

Hong and

J.X.

Yu, Graph clustering based on structural/attribute similarities, in: Proceedings of the VLDB Endowment, Vol. 2, 2009, pp. 718–729.

73.

S.R.

Yerva,

Miklós and

Aberer, Quality-aware similarity assessment for entity matching in Web data, Inf. Syst. 37(4) (2012), 336–351. doi:10.1016/j.is.2011.09.007.

74.

Yu and

Lam, An integrated probabilistic and logic approach to encyclopedia relation extraction with multiple features, in: Proceedings of the 22nd International Conference on Computational Linguistics (Coling 2008), 2008, pp. 1065–1072. doi:10.3115/1599081.1599215.

75.

Yujian and

Bo, A normalized levenshtein distance metric, IEEE Trans. Pattern Anal. Mach. Intell. 29(6) (2007), 1091–1095. doi:10.1109/TPAMI.2007.1078.

76.

Zhu,

Nie,

Liu,

Zhang and

J.-R.

Wen, StatSnowball: A statistical approach to extracting entity, in: Proceedings of the 18th International Conference on World Wide Web, 2009, pp. 101–110. doi:10.1145/1526709.1526724.

A semantic approach to cross-document person profiling in Web

Abstract

Keywords

1. Introduction

2. Related work

1 Jsoup: Java HTML Parser, [http://jsoup.org/].

3.2.1. Word sense disambiguation

6 http://nlp.stanford.edu/software/stanford-dependencies.shtml

3.3. Attribute extraction

Table 2 Vocabulary of attributes in question Attribute names Affiliation Degree Mentor Phone Award Email Nationality Relatives Birth place Fax Occupation School Date of birth Major Other name Website

Table 3 Performances of our seed extraction method on benchmark training datasets Dataset Seed type P (%) R (%) F1 (%) WePS-2 Keyword 65.2 61.9 63.51 Verb 79 63.92 70.66 Wikipedia Keyword 76.01 70.4 73.10 Verb 80.04 74.63 77.24

3.4. Name disambiguation

3.4.1. Social link extraction

3.4.2. Profile enrichment

3.4.4. Graph clustering

4. Experiments and results

4.1. Component-level evaluation

4.1.1. Experiments of the pre-processing components

7 http://nlp.stanford.edu/software/dcoref.shtml

10 http://nlp.uned.es/weps/weps-2/weps2-data

Table 13 Performances of our name disambiguation approaches on WePS-1 test datasets Method B 3 P B 3 R B 3 F α = 0.5 Pr IPr F p = 0.5 Social links + Local attributes + external attributes 67.3 73.7 68.1 71.5 87.1 80.4 74.4 80.5 75.2 78.6 89.3 82.2 75.2 81.6 77.1 79.1 90.8 84.26

13 F α = 1 / ( α × 1 / P + ( 1 − α ) × 1 / R ) , where P and R are precision and recall, respectively.

References

¹
Jsoup: Java HTML Parser, [http://jsoup.org/].

⁶
http://nlp.stanford.edu/software/stanford-dependencies.shtml

Table 2
Vocabulary of attributes in question

Attribute names

Affiliation Degree Mentor Phone

Award Email Nationality Relatives

Birth place Fax Occupation School

Date of birth Major Other name Website

Table 3
Performances of our seed extraction method on benchmark training datasets

Dataset Seed type P (%) R (%) F1 (%)

WePS-2 Keyword 65.2 61.9 63.51

Verb 79 63.92 70.66

Wikipedia Keyword 76.01 70.4 73.10

Verb 80.04 74.63 77.24

⁷
http://nlp.stanford.edu/software/dcoref.shtml

¹⁰
http://nlp.uned.es/weps/weps-2/weps2-data

¹³
$F_{α} = 1 / (α \times 1 / P + (1 - α) \times 1 / R)$ , where P and R are precision and recall, respectively.