PM was in part inspired by ZPL , a parallel array language. One of the key goals of the ZPL project was to provide a “What You See Is What You Get” (WYSIWYG) approach to parallel communication – in particular as this related to communication overhead. This was achieved using a range of operators that explicitly invoke parallel communication on distributed arrays. For example, the @ operator moves the elements of an array by a fixed displacement while the # operator provides for more general element permutation. A programmer would know that the former operation is much less expensive than the latter. To shift a distributed array by a fixed displacement, computational nodes only need to exchange information for edge cells that have moved into the region governed by a different node. However, general permutation requires all nodes to request and then receive data from arbitrary other nodes and for this exchange to be globally synchronized. The ZPL # operator thus needs avoiding unless ...