A framework for high-level specification of data distributions in data-parallel application programs has been conceived. [As used here, "distributions" signifies means to express locality (more specifically, locations of specified pieces of data) in a computing system composed of many processor and memory components connected by a network.] Inasmuch as distributions exert a great effect on the performances of application programs, it is important that a distribution strategy be flexible, so that distributions can be adapted to the requirements of those programs. At the same time, for the sake of productivity in programming and execution, it is desirable that users be shielded from such error-prone, tedious details as those of communication and synchronization.
As desired, the present framework enables a user to refine a distribution type and adjust it to optimize the performance of an application program and conceals, from the user, the low-level details of communication and synchronization. The framework provides for a reusable, extensible, data-distribution design, denoted the design pattern, that is independent of a concrete implementation. The design pattern abstracts over coding patterns that have been found to be commonly encountered in both manually and automatically generated distributed parallel programs.
The following description of the present framework is necessarily oversimplified to fit within the space available for this article. Distributions are among the elements of a conceptual data-distribution machinery, some of the other elements being denoted domains, index sets, and data collections (see figure). Associated with each domain is one index set and one distribution. A distribution class interface (where "class" is used in the object-oriented- programming sense) includes operations that enable specification of the mapping of an index to a unit of locality. Thus, "Map(Index)" specifies a unit, while "LocalLayout(Index)" specifies the local address within that unit. The distribution class can be extended to enable specification of commonly used distributions or novel user-defined distributions.
A data collection can be defined over a domain. The term "data collection" in this context signifies, more specifically, an abstraction of mappings from index sets to variables. Since the index set is distributed, the addresses of the variables are also distributed.
This work was done by Mark James, Hans Zima, and Roxana Diaconescua of Caltech for NASA's Jet Propulsion Laboratory.
The software used in this innovation is available for commercial licensing. Please contact Karina Edmonds of the California Institute of Technology at (626) 395-2322. Refer to NPO-42538.
This Brief includes a Technical Support Package (TSP).
Reusable, Extensible High-Level Data- Distribution Concept
(reference NPO-42538) is currently available for download from the TSP library.
Don't have an account?
Overview
The document titled "Reusable, Extensible High-Level Data-Distribution Concept" (NPO-42538) from NASA's Jet Propulsion Laboratory outlines a novel design for high-level data distribution in data parallel applications. It addresses the challenges faced in high-performance computing, particularly in the context of NASA's High Productivity Computing Systems (HPCS). The primary focus is on enhancing the performance of scientific computations by optimizing data distribution strategies.
The document emphasizes the importance of locality of access and load balancing in massively parallel computing systems, which consist of numerous processing and memory units interconnected by high-speed networks. Effective data distribution is crucial for managing these aspects, as it directly influences the performance of data parallel applications. Traditional low-level programming models, while allowing for fine-tuning of distribution strategies, can be tedious and error-prone, leading to a need for a more user-friendly approach.
The proposed design abstracts common coding patterns encountered in both manual and automated programming of distributed parallel programs. It introduces key elements of the distribution framework, including domains, index sets, data collections, and distributions. This abstraction allows users to specify distribution types at a high level, enabling them to refine and adjust these types to optimize application performance without delving into complex low-level details such as communication and synchronization.
The document also highlights the generality of the design, presenting it as an abstract distribution design pattern that is independent of specific programming languages, although it illustrates its applicability using the Chapel programming language. This flexibility is essential for adapting the distribution strategy to the specific needs of various applications, thereby enhancing productivity and reducing the likelihood of errors.
Furthermore, the work is positioned within the broader context of NASA's goals for developing powerful, scalable, and economically viable computing systems suitable for national security and industry applications. The document underscores the significance of this technology in managing the computationally intensive requirements for analyzing large volumes of scientific data returned from space missions.
In summary, the document presents a comprehensive framework for high-level data distribution in parallel computing, aiming to improve performance and productivity while minimizing the complexity faced by programmers.
