A fast linkage detection scheme for multi-source information integration

A Aizawa, K Oyama - International Workshop on Challenges in …, 2005 - ieeexplore.ieee.org
International Workshop on Challenges in Web Information Retrieval …, 2005ieeexplore.ieee.org
Record linkage refers to techniques for identifying records associated with the same real-
world entities. Record linkage is not only crucial in integrating multi-source databases that
have been generated independently, but is also considered to be one of the key issues in
integrating heterogeneous Web resources. However, when targeting large-scale data, the
cost of enumerating all the possible linkages often becomes impracticably high. Based on
this background, this paper proposes a fast and efficient method for linkage detection. The …
Record linkage refers to techniques for identifying records associated with the same real-world entities. Record linkage is not only crucial in integrating multi-source databases that have been generated independently, but is also considered to be one of the key issues in integrating heterogeneous Web resources. However, when targeting large-scale data, the cost of enumerating all the possible linkages often becomes impracticably high. Based on this background, this paper proposes a fast and efficient method for linkage detection. The features of the proposed approach are: first, it exploits a suffix array structure that enables linkage detection using variable length n-grams. Second, it dynamically generates blocks of possibly associated records using ‘blocking keys’ extracted from already known reliable linkages. The results from our preliminary experiments where the proposed method was applied to the integration of four bibliographic databases, which scale up to more than 10 million records, are also reported in the paper.
ieeexplore.ieee.org
以上显示的是最相近的搜索结果。 查看全部搜索结果