Software Challenges for Extreme Scale Systems, or how to play the End-Game for Moore's Law

Vivek Sarkar, Rice University
Host: Yair Amir

It is widely recognized that future generations of computer systems will be qualitatively different from current and past generations. Specifically, they will be built with thousands of homogeneous and heterogeneous processor cores per chip combined with heterogeneous memory hierarchies, their performance will be driven by parallelism (thousand-way parallelism in mobile devices, and million-way parallelism in a single-rack departmental server), and will be constrained by fundamental energy and data movement limitations. They will also be subject to frequent faults and failures. Unlike previous generations of hardware evolution, these “extreme scale” systems will have a profound impact on future software. The software challenges are further compounded by the need to support application domains on the frontier of Computer Science that have traditionally not had to worry much about hardware limitations in the past. Addressing these challenges for extreme scale systems will require major advances in the core of Computer Science, including programming languages, compilers, runtime systems, and processor architectures. These advances are also critical for extending the end-game of Moore’s Law, by contributing to exponential performance improvements in successive generations of computer systems without relying solely on exponential increases in transistor density. In this talk, we discuss approaches to addressing software challenges for extreme scale systems based on experiences gained in the Habanero Extreme Scale Software Research Project at Rice University [1]. Our approach is based on introducing a set of unified primitives for parallelism and concurrency, which can be used to support a wide range of application patterns including task parallelism, functional parallelism, loop parallelism, data flow parallelism, actor concurrency, and transactional concurrency. These primitives define a family of structured parallel programming models, each of which has a well defined set of semantic guarantees with respect to serial elision, deadlock freedom, datarace freedom, functional determinism, structural determinism, and permission safety. Our claim is that these structured parallel primitives greatly simplify the job of developing correct and efficient software for extreme scale systems. Application experiences discussed in this talk will be drawn from the medical imaging and genomics analysis domains, based on research conducted in the NSF Expeditions Center for Domain-Specific Computing (CDSC) [2]. Background material for this talk will be drawn in part from the DARPA Exascale Software Study report [3] led by the speaker. This talk will also draw from a recent study led by the speaker on Synergistic Challenges in Data-Intensive Science and Exascale Computing [4] for the US Department of Energy’s Office of Science. We would like to acknowledge the contributions of all participants in both studies and in CDSC.

Speaker Biography

Vivek Sarkar is Professor and Chair of Computer Science at Rice University. He conducts research in multiple aspects of parallel software including programming languages, program analysis, compiler optimizations and runtimes for parallel and high performance computer systems. He currently leads the Habanero Extreme Scale Software Research Laboratory at Rice University, and serves as Associate Director of the NSF Expeditions Center for Domain-Specific Computing and PI of the DARPA-funded Pliny project on “big code” analytics. Prior to joining Rice in July 2007, Vivek was Senior Manager of Programming Technologies at IBM Research. His responsibilities at IBM included leading IBM’s research efforts in programming model, tools, and productivity in the PERCS project during 2002- 2007 as part of the DARPA High Productivity Computing System program. His prior research projects include the X10 programming language, the Jikes Research Virtual Machine for the Java language, the ASTI optimizer used in IBM’s XL Fortran product compilers, the PTRAN automatic parallelization system, and profile-directed partitioning and scheduling of Sisal programs. In 1997, he was on sabbatical as a visiting associate professor at MIT, where he was a founding member of the MIT Raw multicore project. Vivek became a member of the IBM Academy of Technology in 1995, the E.D. Butcher Chair in Engineering at Rice University in 2007, and was inducted as an ACM Fellow in 2008. He holds a B.Tech. degree from the Indian Institute of Technology, Kanpur, an M.S. degree from University of Wisconsin-Madison, and a Ph.D. from Stanford University. Vivek has been serving as a member of the US Department of Energy’s Advanced Scientific Computing Advisory Committee (ASCAC) since 2009.