Parallel Computation - running the "g-bcast" program and analyzing with VAMPIR

Introduction

The programs for Gauss elimination are written in FORTRAN 77 using the MPI communication library. The g-bcast program will be run on the Data Star, by following the instructions given below.
If you are interested in more information regarding the use of the Data Star, it's recommended that you read the Data Star User Guide.

VAMPIR has been introduced earlier with the etch module. Go over the previous instructions, if you don't feel comfortable from "using the VAMPIR tool on the Data Star" for the etch module.

How to start

These programs can be run in the Linux environment. To start, you have two options:
- To use Linux machines. You have to start a terminal.
- To use the Windows NT machines. You have to use the "SSH" program. To start SSH, start from
Start-->Programs-->Network Applications-->SSH-->Secure Shell Client
Type "linux" as Host Name and click Connect. Don't be impatient if it doesn't immediately connect. Enter your CADLAB login and password. This will make a connection to one of the Linux machines in Engineering I. Now you are on your account via a Unix interface.

Login on the Data Star

To login on the Data Star you must use the secure shell command:

        % ssh -l UserID dspoe.sdsc.edu 

These are the nodes that have been set up for interactive use.

Transfer the files from the HPSS [High Performance Storage System] to the Data Star

To transfer the pge modules and auxiliary files to the Data Star, you have to use pftp. pftp is a ftp-like interface to HPSS. The following links and text below describe how to use the pftp utility:

        % pftp 
    pftp> cd /users/csb/u4078 
    pftp> get pge.tar 
    pftp> quit 
        %   

Compiling FORTRAN programs on the Data Star

In order to compile and run the pge module, you have to untar the file pge.tar:

        %tar xvf pge.tar

In the directory pgauss, the following files are present:

        - Makefile
        - g-bcast.f

Change to the pgauss directory (type "ls" to see what is inside the directory). In file Makefile, we have to make a change to obtain the trace file of the parallel computation. For this, open Makefile with one of the easiest editors on Unix:Pico

        % cd pgauss
        % ls
        % pico Makefile

This will open the Makefile with Pico editor. Now you can add the link to the VAMPIR library, to obtain the trace file after running the program. For this:

   
        add "-L/usr/local/apps/vamp/lib -lVT -lm -lld" after the original third line:

        mpxlf -o bcast $(OBJS)
        

        to obtain new third line:

        mpxlf -o bcast $(OBJS) -L/usr/local/apps/vamp/lib -lVT -lm -lld      

Now you can exit the Pico editor with "^X" aka "Control+X", note that the file name should stay the same.
The compilation of the pge module is achieved by running make.

        % make 

These will create a new executable bcast file by using g-bcast.f.

Running programs on the Data Star

To run the Gauss module with the VAMPIR tool, set the paths to the VAMPIR tool:

        % set PAL_ROOT=/usr/local/apps/vamp
        % set PAL_LICENSEFILE=/usr/local/apps/vamp/license.dat 
        % set path = ($path /usr/local/apps/vamp/bin)

You can now obtain trace files for the VAMPIR tool.

Run the bcast program as described before. The only thing that is different this time is that, after the bcast program executes, it generates a trace file called bcast.bpv.

Starting the VAMPIR session

After the program has finished executing, start a VAMPIR session:

         Set the DISPLAY variable to the name of the machine you are logged into
         [for example "ecipc004"]:
             % setenv DISPLAY ecipc004.engr.ucsb.edu:0.0
         Enter:   
             % vampir bcast.bpv

The VAMPIR Main window and the Global Timeline Display window open.

Open views to visualize the trace records

To do this, click on the Global Displays from the main window and open:

 
         * Summary Chart 
         * Activity Chart 
 

You can select any other views. To interpret the information these windows present, see VAMPIR User Guide.

View the trace file.

We'll start by viewing the time for the entire run. Zoom in on a section of the timeline:

    * Click and drag over a portion of the timeline with the left mouse
      button. 
      This part will be magnified.
    * Continue zooming until most of the MPI function names are revealed.

View process statistics for the selected portion of the timeline.

From the Global Display menu, select Summary Chart view. A new view will open. Press the right mouse button within this window.

     * Select the Use Timeline Portion. 
     * Scroll the timeline, using the scroll bar at the bottom of the timeline
       window, and watch what happens in both displays.

View the "Activity Chart".

The "Activity Chart" display shows a statistic about the time spent in each activity individually for each process defined in the tracefile. With the default pie chart display you can recognize load imbalance at a glance in the trace program by comparing the different time consumptions of the activities over all processes.

End the VAMPIR session.

To do this, select:

        File --> Exit