An efficient map-reduce algorithm for spatio-temporal analysis using Spark (GIS Cup) | W. Randolph Franklin, Rensselaer Polytechnic Institute, Troy NY USA

Salles Viana Gomes de Magalhães, W. Randolph Franklin, Wenli Li, and Marcus Vinicius Alvim Andrade. An efficient map-reduce algorithm for spatio-temporal analysis using spark (GIS Cup). In 5th GIS-focused algorithm competition, GISCup 2016, co-located with ACM SIGSPATIAL GIS. 2016. Winner (2nd place).
[full text] [slides] [BibTeX▼]

@inproceedings{salles-giscup-2016,
    author = "de Magalhães, Salles Viana Gomes and Franklin, W. Randolph and Li, Wenli and Andrade, Marcus Vinicius Alvim",
    title = "An efficient map-reduce algorithm for spatio-temporal analysis using Spark ({GIS Cup})",
    note = "Winner (2nd place)",
    booktitle = "5th {GIS}-focused algorithm competition, {GISCup} 2016, co-located with {ACM SIGSPATIAL GIS}",
    year = "2016",
    href = "\bibhrefpt{215-salles-giscup-2016}{215-salles-giscup-2016-talk.pdf}",
    customlinkslides = "https://wrfranklin.org/p/215-salles-giscup-2016-talk.pdf",
    mykey = "marcus wenlili salles parallel sallesgiscup2016"
}

Abstract

We present an efficient parallel algorithm for performing spatio-temporal analysis in a Spark cluster. It divides the spatio-temporal cube into partitions and uses these partitions as the elements of the Spark Resilient Distributed Datasets (RDDs). As a result, data presents a better locality and the overheads are smaller than the overheads observed when single cells are used as elements of the RDDs. When used to find hotspots in the NYC Yellowcab data, our algorithm is up to 52 times faster than algorithms using single cells RDDs.

Full Text

[download]