Is Co-Array Fortran the Future of Parallelism?

An article by Dean Dauger on MacResearch invoked quite a bit of discussion on the future of parallel systems, and whether the current emphasis on shared-memory systems was the best course. There are related discussions taking place in the Fortran community, which is currently in the process of deciding whether co-arrays, an extension to the language used for parallel programming, will be incorporated into the Fortran language in 2010.

Integrating parallelism into the language, where it has previously been relegated to the domain of libraries and preprocessors, is an important step to ubiquitous parallelism, but it will have its detractors. On the plus side, it means that developers can use the same techniques on all platforms, and with all compilers. On the downside, the choice means ‘blessing’ a single model of parallelism, and you’d better be sure that you’ve made the right choice.

The co-array approach to parallelism falls into the SPMD (Single-Program Multiple-Data) category espoused by Dr. Dauger. It is comparable to the popular Message-Passing Interface (MPI): an application runs as independent processes that pass data back and forth via a network or memory. Like MPI, Co-Array Fortran applications will run on shared-memory or distributed-memory (eg clusters) systems.

One of the advantages of building parallelism into the language is that programs become more readable, even if ultimately they are not doing anything new. Take this basic MPI code to send an array of real numbers from process 1 to process 0.

program main
    include 'mpif.h'
    real array(10)
    integer rank, status(MPI_STATUS_SIZE), error

    ! Initialize MPI, and get process rank
    call MPI_INIT(error)
    call MPI_COMM_RANK(MPI_COMM_WORLD, rank, error)

    ! Initialize array on process 1
    if ( rank == 1 ) array = 0.5

    ! Send/recv data
    if ( rank == 0 ) then
        call MPI_RECV(array, 10, MPI_REAL, 1, 0, MPI_COMM_WORLD, status, error)
    else if ( rank == 1 ) then
        call MPI_SEND(array, 10, MPI_REAL, 0, 0, MPI_COMM_WORLD, error)
    endif

    ! Finish up
    call MPI_FINALIZE(error)
end program

That’s quite a bit of code to do something fairly straightforward. In Co-Array Fortran, you could do it like this

program main
    real array(10)[*]
    if ( this_image() == 2 ) array = 0.5
    sync all
    if ( this_image() == 1 ) array(:)[1] = array(:)[2]
end program

There is a complete PDF overview of Co-Array Fortran available for download, but to give you some idea how Co-Arrays work, Fortran arrays can be given an extra index in square brackets known as the co-array subscript. This index defines how the array’s memory is duplicated across processes, and gives you a means of addressing data that is resident on a different process. Using data from one process on another one implicitly transfers it across the network; in the MPI example, this transfer is explicit.

The rules for the co-array subscript are quite similar to those of standard Fortran 90 arrays. In the example above, the declaration real array(10)[*] indicates that each process will have an array with 10 elements; the asterisk instructs the compiler to resize the co-array dimension to fit the number of processes available.

Co-Array Fortran also provides a number of intrinsic functions to query the parallel environment, such as the this_image function used above. This is similar to the MPI_COMM_RANK subroutine; it returns the index of the current process. (Unlike MPI, process indexes in Co-Array Fortran begin by default at 1.)

To transfer data from one process to another, you simply use it in an expression, as if the co-array dimensions were standard array dimensions. In our example, we first check if the current process has index 1, and if so, transfer the version of array on process 2 into the version of array on process 1. The check on whether the current process has index 1 is not strictly necessary, but if were left out, both process 1 and process 2 would each independently transfer the data. To avoid unnecessary overhead, we leave this transfer to a single process using the if clause.

It should be noted that co-arrays are not the answer to all our prayers: they don’t help you much with any of the subtle traps that parallelism involves. For example, note the sync all in the code above. Without that line, which would be easy to forget, the initialization of the array on process 2 may not be concluded when the transfer of data is instigated by process 1. The result of this operation would be undefined, dependent purely upon the conditions under which the code was run.

Should Co-Array Fortran become part of the Fortran standard? It’s a difficult question, and one that I — in all honestly — cannot answer. Maybe you can, in which case, you should let your voice be heard in the online debate that is currently raging. It’s a landmark moment for Fortran, and I would argue, for programming in general.