Skip to main content

WYSIWYG

PM was in part inspired by ZPL, a parallel array language. One of the key goals of the ZPL project was to provide a “What You See Is What You Get” (WYSIWYG) approach to parallel communication – in particular as this related to communication overhead. This was achieved using a range of operators that explicitly invoke parallel communication on distributed arrays. For example, the @ operator moves the elements of an array by a fixed displacement while the # operator provides for more general element permutation. A programmer would know that the former operation is much less expensive than the latter. To shift a distributed array by a fixed displacement, computational nodes only need to exchange information for edge cells that have moved into the region governed by a different node. However, general permutation requires all nodes to request and then receive data from arbitrary other nodes and for this exchange to be globally synchronized. The ZPL # operator thus needs avoiding unless its use is essential. This type of explicit performance model is extremely useful for creating efficient distributed code.

Unlike ZPL, PM is not an array language. However, it keeps the concept of explicit communicating operators (see this earlier post) and also strives to present a WYSIWYG performance model to the programmer. PM only has two communicating operators @x[...] and x@[...] which provide access the value of x associated with either an absolute or a relative point in the domain of the enclosing for statement (again see this post. for more information.) Each of these operations is more efficient if the subscript term is independent of the point of the domain being processed than if is locally computed for each point. For example:

 const model_grid=grid(1..N,1..N),  
      centre_point=[N/2,N/2],       
      halo:=[-1..1,-1..1]  
 for i in model_grid do  
   cell:=spin_up_model%()  
      ! This is a communicating procedure – may contain its own  
      ! communicating (@) operations  
   cell= @cell[centre_point]        
      ! Broadcast central value to all cells – efficient  
   for each time in 1..max_time_steps do  
       nbd:= cell @ [ halo ]     
              ! Get neighbouring cells - efficient  
       advection := compute_motion(cell,nbd)  
       moved_cell := cell @ [ advection ]  
              ! Get arbitrary cells - expensive  
       cell = update_model_state(cell,nbd,moved_cell)  
   endfor  
 endfor

Note that this example would work just as well if halo and centre_point had been dynamically computed, providing this was done so outside of the for statement. It is also important to note that the use of communicating operators is not dependent on data being distributed – the notation is equally applicable when the entirety of a for statement is executed on a single node. PM thus avoids forcing the user to make early choices of which loops to parallelise – the basic rule is “make everything a for statement unless you absolutely have to use a sequential for-each”. An advantage of this approach is that for statements (which explicitly prohibit inter-element interactions, except through communicating operations) are much easier to vectorize. Similar arguments could be made for SMP systems and avoiding unnecessary cache updates. The PM performance model thus applies across multiple levels of parallelism, from distributed computing to vectorization.

Comments

Post a Comment

Popular posts from this blog

Compile time, run time, coffee time

[ Please note that in the following discussion I will be using PM 0.5 notation, which is slightly different to PM 0.4 and also in some flux as PM 0.5 is still in development. ]   Many programming languages (and most newly developed ones) include some form of compile-time programming, ranging from simple preprocessors ( #define , #if in C) to fully blown macro systems capable of re-writing the abstract syntax tree during compilation (Rust, Julia, etc .). In line with its philosophy of keeping things as simple as possible, but not simpler, PM takes a middle road with compile-time programming supported primarily through the type system. There is nothing too radical here – this is not an area where the language aims to break new ground.  The PM approach centres around the use of compile-time types. Many languages use special types to denote literal values and PM follows this trend. Literal integers, reals, strings and Booleans each have their own types: literal(int) , litera...
Post a Comment
Read more

The PM Type System

In common with other aspects of PM design, the type system is designed to facilitate the use of flexible programming constructs, such as polymorphism and dynamic dispatch, while emphasising the generation of fast, static, code.  In common with many other modern languages, PM also attempts to combine the safety of static typing with the expressivity of dynamically typed languages such as Python . The type of any PM expression may usually be determined by a static examination of the code. Variables take their types from an initialising expression using the ‘:=’ declaration syntax popularised by Go . a:= 100 ! Declare ‘a’ as an integer (int) variable Variables do not change type during the course of their lifetime – polymorphic programming requires the use of special values, described later. Composite values such as structures are generated using specialised expressions rather than through the invocation of type-specific creator functions: b:= struct var_descriptor...
Post a Comment
Read more

All change .. not quite

With the recent release of PM 0.4 and the positive reception to my PM presentation at CIUK2023 , it seems like a good time to bring back the PM blog after a long hiatus. Another good reason for its resurrection is that I feel that I now have built the basic semantics of the language into something like a coherent whole, giving me something concrete to write about. There have been a few major changes to the language since my last blog entry. The main syntactic change has been the shift from keyword-delimited control statements to curly-brackets. This is not a statement on my part as to the merits of the two approaches, I am generally agnostic in this debate which can border on the religious. It was simply that with the way that the language was developing, the keyword approach was getting cumbersome – frequently used constructs were taking up far to much space and impeding readability. PM now uses curly brackets to terminate statements and semicolons to separate (and optionally terminat...
Post a Comment
Read more