Scientists need to be able to quickly develop and run parallel simulations without writing low-level message passing codes using compiled languages such as C/C++/Fortran. Traditionally, high-level languages that support rapid development, such as MATLAB, IDL, Mathematica, and Python, have not addressed parallel computing needs. Other parallel tools for high-level languages are very early in the development process and not mature, are very expensive and not open source, are typically limited to one or two models of parallel computing, do not allow collaborative parallel computing, have not fully addressed error handling, and are not asynchronous in nature.
This innovation will enable scientists to create rapidly parallel simulations by providing parallel building blocks in the high-level programming language Python. The building blocks themselves, such as distributed arrays, parallel Fourier transforms, and parallel statistical algorithms, are implemented by leveraging existing high-performance libraries and creating high-level Python objects that completely hide the details of the underlying libraries and parallelism from users. The building blocks have been developed as part of the open-source IPython project, which will enable users to develop, debug, and run parallel simulations in a completely interactive manner, similar to MATLAB or IDL.
Prototype distributed memory arrays were developed that have a simple and easy-to-use Python interface. These arrays can be used as the basis of parallel algorithms, simulations, and data analysis tasks. The distributed arrays and parallel algorithms were integrated into the open-source IPython project to enable them to be used interactively.
The parallel building blocks are highly extensible as they are based on open-source technologies and are open-source themselves. They are freely available to scientists and researchers and run on virtually any computing platform.
This work was done by Brian Granger of TX Corp. for Goddard Space Flight Center. GSC-16055-1