1 option
International Symposium on Computer Architecture and High Performance: 19th: 2007: Gramado, RS, Brazil
- Format:
- Book
- Author/Creator:
- Gaudiot, Jean-Luc, author.
- Language:
- English
- Subjects (All):
- Computer architecture--Congresses.
- Computer architecture.
- Physical Description:
- 1 online resource
- Other Title:
- Computer Architecture and High Performance Computing, 2007, SBAC-PAD 2007, 19th International Symposium on.
- Place of Publication:
- [Place of publication not identified] IEEE Computer Society Press 2007
- Language Note:
- English
- Summary:
- The identification of replicas in a database is fundamental to improve the quality of the information. Deduplication is the task of identifying replicas in a database that refer to the same real world entity. This process is not always trivial, because data may be corrupted during their gathering, storing or even manipulation. Problems such as misspelled names, data truncation, data input in a wrong format, lack of conventions (like how to abbreviate a name), missing data or even fraud may lead to the insertion of replicas in a database. The deduplication process may be very hard, if not impossible, to be performed manually, since actual databases may have hundreds of millions of records. In this paper, we present our parallel deduplication algorithm, called FER- APARDA. By using probabilistic record linkage, we were able to successfully detect replicas in synthetic datasets with more than 1 million records in about 7 minutes using a 20- computer cluster, achieving an almost linear speedup. We believe that our results do not have similar in the literature when it comes to the size of the data set and the processing time.
- Notes:
- Bibliographic Level Mode of Issuance: Monograph
- ISBN:
- 9781509087853
- 1509087850
The Penn Libraries is committed to describing library materials using current, accurate, and responsible language. If you discover outdated or inaccurate language, please fill out this feedback form to report it and suggest alternative language.