Go to the first, previous, next, last section, table of contents.


Overview of `TOP-C/C++'

                       "All right, Mr. Wiseguy,"  she said,  "you're so
                   clever,  you tell us what color it should be."
                       from "The Restaurant at the End of the Universe"
                       by Douglas Adams

A `TOP-C' application is compiled and run using `topcc' (similarly to `gcc') or `topc++' (similarly to `g++'). For example, assuming a `procgroup' file in the current directory to specify the remote hosts for the slave processes, one executes:

  topcc --mpi parfactor.c
  # or else:  topc++ --mpi parfactor.cc
  ./a.out

For purposes of documentation, we will standardize on an explanation of topcc. Wherever topcc is mentioned, the description is equally valid for topc++.

Programmer's Model

Structure of a TOP-C Program

A typical TOP-C application has the following structure:

#include <topc.h>
... define four callback functions for TOPC_master_slave() ...
int main( int argc, char **argv ) {
  TOPC_init( &argc, &argv );
  ...
  TOPC_master_slave( generate_task_input, do_task, check_task_result,
                     update_shared_data )
  ...
  TOPC_finalize();
}

For an alternative interface often useful for parallelizing existing sequential code, see section `TOP-C' Raw Interface: TOPC_raw_XXX. However, for new applications, the standard interface will usually be cleaner.

Four Callback Functions

In a `TOP-C' application, the programmer defines four callback functions and passes control to the `TOP-C' library through the following command.

  TOPC_master_slave(GenerateTaskInput, DoTask, CheckTaskResult,
                    UpdateSharedData);

Pictorially, TOP-C arranges for the flow of control among the four callback functions as follows:

      (ON MASTER)      task input
   GenerateTaskInput() ---------->

  task input   (ON A SLAVE)  task output
  -----------> DoTask(input) ----------->

  task output     (ON MASTER)                    action
  -----------> CheckTaskResult(input, output) ----------->

if (action == UPDATE):
  task input, task output      (ON ALL PROCESSES)
  -----------------------> UpdateSharedData(input, output)

The `TOP-C' Algorithm

When there is only one slave, The `TOP-C' algorithm can be summarized by the following C code.

{ void *input, *output;
  TOPC_ACTION action;
  while ( (input = GenerateTaskInput()) != NOTASK ) {
     do {
       output = DoTask(input);
       action = CheckTaskResult(input, output);
     } while (action == REDO);  /* REDO not useful for only one slave */
     if (action == UPDATE) then UpdateSharedData(input, output);
  }
}

On a first reading, it is recommended to read the next few sections through Section section Actions Returned by CheckTaskResult(), and then return to this section.

For a better understanding of the case of multiple slaves, this simplified excerpt from the `TOP-C' source code describes the `TOP-C' algorithm.

TOPC_BUF input, output;
int number_free_slaves = num_slaves;
TOPC_ACTION action;

do {
  wait_for_free_slave();
  input = COMM_generate_task_input();
  if (input.data != NOTASK.data) {
    SUBMIT_TO_SLAVE:  output = DoTask(input.data);
    num_free_slaves--;
  }
  else if (num_free_slaves < num_slaves)
    receive_task_output();     /* needed to insure progress condition */
} while (input.data != NOTASK.data || num_free_slaves < num_slaves);

The code for wait_for_free_slave() can be expanded as follows.

void wait_for_free_slave() {
  do {
    while ( result_is_available(&input, &output) ) {
      action = CheckTaskResult(input.data, output.data);
      if (action == UPDATE)
        UpdateSharedData(input.data, output.data);
      if (action == REDO) /* Use updated shared data, when redoing */
        SUBMIT_TO_SLAVE:  output = DoTask(input.data);
      num_free_slaves++;
    } while (num_free_slaves == 0);
  }

Note that the term result refers to an `(input,output)' pair. The routine CheckTaskResult() returns an action, which determines the control structure for a parallel algorithm. A common definition is:

TOPC_ACTION CheckTaskResult( void *input, void *output ) {
  if (output == NULL) return NO_ACTION;
  else if ( ! TOPC_is_up_to_date() ) return REDO;
  else return return UPDATE; }

TOPC_is_up_to_date() returns true if and only if during the interval between when the task input was originally generated and when the task output was returned by the most recent slave, no other slave process had returned a task output during the interim that had caused the shared data to be modified through an UPDATE action. An UPDATE action causes UpdateSharedData() to be invoked on each process. Further discussion can be found in the section on section TOP-C Utilities.

SMP Compatibility

`TOP-C' also supports operation on SMP (shared memory) hardware. `TOP-C' is designed so that the same application source code may operate efficiently both under distributed and under shared memory. In SMP, all data outside of the four callback functions is shared, by default. Hence, an UPDATE action under shared memory causes only the master process to invoke UpdateSharedData(). To avoid inconsistencies in the data, by default `TOP-C' arranges that no slave process may run DoTask() while UpdateSharedData() is running. `TOP-C' also provides support for finer levels of granularity through application-defined private variables and critical sections. Further discussion can be found in the section on section Optimizing TOP-C Code for the Shared Memory Model.

Three Key Concepts for TOP-C

The `TOP-C' programmer's model is based on three key concepts:

  1. tasks in the context of a master/slave architecture;
  2. global shared data with lazy updates; and
  3. actions to be taken after each task.

Task descriptions (task inputs) are generated on the master, and assigned to a slave. The slave executes the task and returns the result to the master. The master may update its own private data based on the result, or it may update data on all processes. Such global updates take place on each slave after the slave completes its current task. Updates are lazy in that they occur only after a task completes, although it is possible to issue a non-binding request to `TOP-C' to abort the current tasks (section Aborting Tasks). A SPMD (Single Program Multiple Data) style of programming is encouraged.

In both shared and distributed memory architectures, one must worry about the order of reads and writes as multiple slaves autonomously update data. The utilities below are meant to ease that chore, by supporting the ease of the SPMD programming style, while still maintaining good efficiency and generality for a broad range of applications. The software can easily be ported to a variety of architectures.

Task Input and Task Output Buffers

A task input or task output is an arbitrary buffer of bytes. Internally, it is an arbitrary user-defined data structure, which is opaque to TOP-C. A task input or task output must be encapsulated by a function: TOPC_MSG( void *buf, int buf_size ), which is then returned by the callback functions GenerateTaskInput() and DoTask().

TOPC_MSG() copies the application data to its own buffers in `TOP-C' space using a shallow copy. Hence, it is the application's responsibility to free or re-use the original space pointed to by buf. See section section The Main TOP-C Library Calls, for more details on TOPC_MSG(). When a `TOP-C' callback function is passed a task input or task output, that buffer is a pointer into `TOP-C' space. The callback function may freely use or even modify the data in the buffer, but the buffer pointer will no longer valid when the callback function returns. If a later invocation of a callback function will wish to use the data in the buffer, the `TOP-C' application must cache a private copy and point to it with a global variable.

If a heterogeneous architecture is used, there is an issue of converting data formats. This is the application's responsibility. UNIX utilities are available such as htonl(), ntohl() (for integer byte ordering), and the `XDR' library (RFC 1832, eXternal Data Representation, general standard, also including struct's, etc.). `Corba''s `IDL' presents another possible architecture-independent solution. Note that systems following the IEEE floating point standard should already have compatible floating point.


Go to the first, previous, next, last section, table of contents.