From Wikipedia, the free encyclopedia

A parallel programming model is a set of software technologies to express parallel algorithms and match applications with the underlying parallel systems. It encloses the areas of applications, programming languages, compilers, libraries, communications systems, and parallel I/O. Due to the difficulties in automatic parallelization today, people have to choose a proper parallel programming model or a form of mixture of them to develop their parallel applications on a particular platform.

Parallel models are implemented in several ways: as libraries invoked from traditional sequential languages, as language extensions, or complete new execution models. They are also roughly categorized for two kinds of systems: shared-memory system and distributed-memory system, though the lines between them are largely blurred nowadays.

A programming model is usually judged by its expressibility and simplicity, which are by all meanings conflicting factors. The ultimate goal is to improve productivity of programming.

Parallel programming models include:

• POSIX Threads
• PVM
• MPI
• Occam (programming language)
• Ease (programming language)
• Linda coordination language
• OpenMP
• Charm++
• Global Arrays
• Co-array Fortran
• UPC
• HPF
• SHMEM
• Stream processing
• Pipelining
• Partitioned global address space

Other research-level models are:

• Cray's Chapel,
• Sun’s Fortress,
• IBM’s X10

• H. Shan and J. Pal Singh. A comparison of MPI, SHMEM, and Cache-Coherent Shared Address Space Programming Models on a Tightly-Coupled Multiprocessor. International Journal of Parallel Programming, 29(3), 2001.
• H. Shan and J. Pal Singh. Comparison of Three Programming Models for Adaptive Applications on the Origin 2000. Journal of Parallel and Distributed Computing, 62:241–266, 2002.
• About structured parallel programming: Davide Pasetto and Marco Vanneschi. Machine independent Analytical models for cost evaluation of template--based programs, University of Pisa, 1996

Automatic parallelization

	Topics in Parallel Computing	v • d • e
General High-performance computing Parallelism Data parallelism • Task parallelism Theory Speedup • Amdahl's law • Flynn's Taxonomy • Cost efficiency • Gustafson's Law • Karp-Flatt Metric Elements Process • Thread • Fiber • Parallel Random Access Machine Coordination Multiprocessing • Multithreading •Multitasking • Memory coherency • Cache coherency • Barrier • Synchronization • Distributed computing • Grid computing Programming Programming model • Implicit parallelism • Explicit parallelism Hardware Computer cluster • Beowulf • Symmetric multiprocessing • Non-Uniform Memory Access • Cache only memory architecture • Asymmetric multiprocessing • Simultaneous multithreading • Shared memory • Distributed memory • Massively parallel processing • Superscalar processing • Vector processing • Supercomputer Software Distributed shared memory • Application checkpointing APIs Pthreads • OpenMP • Message Passing Interface (MPI) Problems Embarrassingly parallel • Grand Challenge