Document similarity self-join with MapReduce (Contributo in atti di convegno)

Type

• Prodotto della ricerca (Classe)
• Contributo in atti di convegno (Classe)

Label

• Document similarity self-join with MapReduce (Contributo in atti di convegno) (literal)

Anno

• 2010-01-01T00:00:00+01:00 (literal)

Http://www.cnr.it/ontology/cnr/pubblicazioni.owl#doi

• 10.1109/ICDM.2010.70 (literal)

Alternative label

• Lucchese C.; Baraglia R.; De Francisci Morales G. (2010)
  Document similarity self-join with MapReduce
  in IEEE International Conference on Data Mining, Sydney
  (literal)

Http://www.cnr.it/ontology/cnr/pubblicazioni.owl#autori

• Lucchese C.; Baraglia R.; De Francisci Morales G. (literal)

Http://www.cnr.it/ontology/cnr/pubblicazioni.owl#note

• In: ICDM 2010 - IEEE International Conference on Data Mining (Sydney, December 14-17 2010). Proceedings, pp. 731 - 736. IEEE, 2010. (literal)

Http://www.cnr.it/ontology/cnr/pubblicazioni.owl#descrizioneSinteticaDelProdotto

• ABSTRACT: Given a collection of objects, the Similarity Self- Join problem requires to discover all those pairs of objects whose similarity is above a user defined threshold. In this paper we focus on document collections, which are characterized by a sparseness that allows effective pruning strategies. Our contribution is a new parallel algorithm within the MapReduce framework. This work borrows from the state of the art in serial algorithms for similarity join and MapReduce- based techniques for set-similarity join. The proposed algorithm shows that it is possible to leverage a distributed file system to support communication patterns that do not naturally fit the MapReduce framework. Scalability is achieved by introducing a partitioning strategy able to overcome memory bottlenecks. Experimental evidence on real world data shows that our algorithm outperforms the state of the art by a factor 4.5. (literal)

Note

• Scopu (literal)

Http://www.cnr.it/ontology/cnr/pubblicazioni.owl#affiliazioni

• CNR-ISTI, Pisa (literal)

Titolo

• Document similarity self-join with MapReduce (literal)

Http://www.cnr.it/ontology/cnr/pubblicazioni.owl#isbn

• 978-1-4244-9131-5 (literal)

Abstract

• Given a collection of objects, the Similarity Self-Join problem requires to discover all those pairs of objects whose similarity is above a user defined threshold. In this paper we focus on document collections, which are characterized by a sparseness that allows effective pruning strategies. Our contribution is a new parallel algorithm within the MapReduce framework. This work borrows from the state of the art in serial algorithms for similarity join and MapReduce-based techniques for set-similarity join. The proposed algorithm shows that it is possible to leverage a distributed file system to support communication patterns that do not naturally fit the MapReduce framework. Scalability is achieved by introducing a partitioning strategy able to overcome memory bottlenecks. Experimental evidence on real world data shows that our algorithm outperforms the state of the art by a factor 4.5. (literal)

Prodotto di

• Istituto di scienza e tecnologie dell'informazione \"Alessandro Faedo\" (ISTI) (Istituto)
• Sistemi e algoritmi per Big Data (ICT.P09.006.001) (Modulo)

Autore CNR

• Gianmarco De Francisci Morales (Persona)
• RANIERI BARAGLIA (Unità di personale interno)
• CLAUDIO LUCCHESE (Persona)

Insieme di parole chiave

• Keywords of "Document similarity self-join with MapReduce" (Insieme di parole chiave)

Incoming links:

Autore CNR di

• CLAUDIO LUCCHESE (Persona)
• RANIERI BARAGLIA (Unità di personale interno)
• Gianmarco De Francisci Morales (Persona)

Prodotto

• Istituto di scienza e tecnologie dell'informazione \"Alessandro Faedo\" (ISTI) (Istituto)
• Sistemi e algoritmi per Big Data (ICT.P09.006.001) (Modulo)

Insieme di parole chiave di

• Keywords of "Document similarity self-join with MapReduce" (Insieme di parole chiave)