Beitrag in einem Tagungsband

Comparison of the HPC and Big Data Java Libraries Spark, PCJ and APGAS



Details zur Publikation
Autor(inn)en:
Posner, J.; Reitz, L.; Fohry, C.
Herausgeber:
Institute of Electrical and Electronics Engineers
Verlag:
IEEE
Verlagsort / Veröffentlichungsort:
Piscataway, NJ

Publikationsjahr:
2018
Seitenbereich:
23-33
Buchtitel:
2018 IEEE/ACM Parallel Applications Workshop, Alternatives To MPI (PAW-ATM)
ISBN:
978-1-7281-0224-5
DOI-Link der Erstveröffentlichung:


Zusammenfassung, Abstract

Although Java is rarely used in HPC, there are a few notable libraries. Use of Java may help to bridge the gap between HPC and big data processing. This paper compares the big data library Spark, and the HPC libraries PCJ and APGAS, regarding productivity and performance. We refer to Java versions of all libraries. For APGAS, we include both the original version and an own extension by locality-flexible tasks. We consider three benchmarks:

Calculation of π from HPC, Unbalanced Tree Search (UTS) from HPC, and WordCount from the big data domain. In performance measurements with up to 144 workers, the extended APGAS library was the clear winner. With 144 workers, APGAS programs were up to a factor of more than two faster than Spark programs, and up to about 30% faster than PCJ programs. Regarding productivity, the extended APGAS programs consistently needed the lowest number of different library constructs. Spark ranged second in productivity, and PCJ third



Autor(inn)en / Herausgeber(innen)

Zuletzt aktualisiert 2024-26-06 um 10:45