Please join us for a UW Data Science Seminar featuring Distinguished Technologist at Hewlett Packard Enterprise Brad Chamberlain on Tuesday, December 2nd from 4:30 to 5:20 p.m. PT. The seminar will be held in IEB G109.
“Interactive Data Science at Massive Scales using Python and Arkouda”
Abstract: In this talk, I’ll introduce Arkouda, a Python library that was created to support scalable data science at interactive rates. While most fast Python libraries get their performance by being implemented in C or C++, Arkouda uses a client-server model in which large data sets can be stored on a cluster or supercomputer, leveraging the memories and processors of multiple compute nodes. Meanwhile, where most HPC-scale computations require submitting jobs to a queue, waiting for them to get their turn to run, and then inspecting the results after the fact, Arkouda permits users to work with their data interactively within the human thought loop using familiar environments like Jupyter notebooks. In practice, Arkouda’s argsort routine has successfully sorted 256 TiB of data in half a minute using 8192 compute nodes.
This talk will introduce Arkouda’s motivation, architecture, and use. I’ll also describe Arkouda’s extensibility, in which new operations, data structures, and packages can be added to it in a modular fashion, such as the Arachne package for large-scale graph analytics developed at NJIT. Finally, I’ll briefly introduce the Chapel language that powers the Arkouda server using Python-level syntax whose performance rivals that of C++, MPI, OpenMP, and CUDA.
Speaker Bio: Brad Chamberlain is a Distinguished Technologist at Hewlett Packard Enterprise (formerly Cray Inc.) who has spent his career focused on user productivity for HPC systems, particularly through the design and development of the Chapel parallel programming language. He received his Ph.D. in Computer Science & Engineering from the University of Washington in 2001 where he worked on the ZPL parallel array language, and he remains associated with the department as an affiliate professor of the Paul G. Allen School.