Distributed mining of spatial high utility itemsets in very large spatiotemporal databases using spark in-memory computing architecture

RU Kiran, S Ito, MS Dao, K Zettsu… - … conference on big …, 2020 - ieeexplore.ieee.org
RU Kiran, S Ito, MS Dao, K Zettsu, CW Wu, Y Watanobe, I Paik, TC Thang
2020 IEEE international conference on big data (big data), 2020ieeexplore.ieee.org
Finding Spatial High Utility Itemsets (SHUIs) in a spatiotemporal database is a challenging
problem of great importance in many real-world applications. Most previous works focused
on the sequential discovery of SHUIs in a database running on a single machine.
Consequently, these works are not suitable for big data (or cloud-based) applications as
they suffer from the scalability and fault tolerant problems. This paper proposes several
novel pruning techniques to reduce the search space and present a more flexible distributed …
Finding Spatial High Utility Itemsets (SHUIs) in a spatiotemporal database is a challenging problem of great importance in many real-world applications. Most previous works focused on the sequential discovery of SHUIs in a database running on a single machine. Consequently, these works are not suitable for big data (or cloud-based) applications as they suffer from the scalability and fault tolerant problems. This paper proposes several novel pruning techniques to reduce the search space and present a more flexible distributed algorithm to find all desired itemsets from the database using Spark in-memory computing architecture. Our algorithm inherits several advantages of Spark, including low communication cost, fault tolerance, and high scalability. Experimental results demonstrate that the proposed algorithm has good scalability and performance on very large databases. Finally, we present a real-world navigation application in which SHUIs generated from the traffic congestion data have been employed to recommend alternative routes to the users.
ieeexplore.ieee.org
Showing the best result for this search. See all results