OpenMacGrid is Easy Peasy

Unless you were holed up in a cave last week, you probably saw our announcement of OpenMacGrid. Gaurav Khanna will be writing a bit about the progress of OpenMacGrid soon, but I wanted to use my weekly post to demonstrate just how easy it is to run calculations on OpenMacGrid, and take advantage of its enormous computing power right now, at no cost, and with no laborious grant submissions.

This is not an Xgrid Tutorial

If you have connected your Mac to OpenMacGrid, chances are that you are currently running some calculations for me. I started submitting jobs for a project I am undertaking last weekend. I’ll explain what the project is about in a separate post, but for now I just want to detail the relatively simple steps I took to get the program — which has never been run on Xgrid before — to run on OpenMacGrid. I started Friday morning, and was ready to submit things Friday afternoon. So it took less than a day.

Charles Parnot is already writing a great series of Xgrid tutorials for us, and I am not going to go into the detail that he will be. Here, I just want demonstrate what it took to get my serial program to run on OpenMacGrid. Hopefully, you will be inspired to do the same, and submit a project to OpenMacGrid via the web form.

Compiling for OpenMacGrid

At present, to run on OpenMacGrid you need a universal binary, which will run on both PPC and Intel Macs. Creating these is simpler than you may think:

• Create an executable for each platform in the usual way (eg make)
  Make sure you statically link your libraries, unless they are installed standard on Mac OS X
• Run lipo

lipo is a command line tool for creating universal binaries, or extracting files from them. Beginning with a PPC executable called cast.ppc.x, and an Intel executable called cast.intel.x, I created the universal executable cast.uni.x like so:

lipo -arch i386 cast.intel.x -arch ppc cast.ppc.x -create -o cast.uni.x

To check that everything worked, you can use the file command to see what is in the universal binary:

file cast.uni.x
cast.uni.x: Mach-O fat file with 2 architectures
cast.uni.x (for architecture i386):     Mach-O executable i386
cast.uni.x (for architecture ppc):      Mach-O executable ppc

Setting Up Run Scripts

After I had an executable, I needed to figure out how to get it uploaded by Xgrid. In particular, my program needs many data files to run, and they all needed to be transferred, along with the executable itself. The program also takes input via the standard input stream, and I had to decide how to do that too.

Xgrid allows you to submit a directory that will be expanded on the agent machines, and used as the working directory in which the program will be run. In other words, all you have to do is setup a directory with everything you need to run, and indicate to Xgrid that you want that to be the working directory.

I copied all of the necessary data files into a single directory that I called xgridworking. I also copied the cast.uni.x executable to the working directory.

To start the executable running, I made a script, and embedded the input directly in it. You could do it in other ways, but I found this to be the easiest. Here is the script I used, which I called gridrun.sh:

#!/bin/sh

./cast.uni.x <<eor
j=0,v=0.0 QC with trans energy $1 eV to $2 eV
5.0   100000        time_step,max_num_time_steps
1
$1 $2 $3 ${10}    min_coll_engy,max_coll_engy,num_coll_engs,num_trajs_per_engy
0.0  0.0         theta_incident, phi_incident
6.5    6.6    4.5     initial_z,z_to_stop,r_to_stop
-98.507077  0.003203  1.4140   0.0 vib_energy,vib_frequency, equilibrium_bond_length, vib_quantum_num
1                   rotation_flag (1 = random, 2 = set with m_j)
0.                   j_quantum_number
$4 $5 $6 $7 $8 $9         rotate_seed_1
$4 $5 $6 $7 $8 $9         rotate_seed_2
12 13 14 15 16 17   vibrate_seed
$9 $8 $7 $6 $5 $4        xy_seed 
0.0000001   2        finite_difference_step,number_of_fd_points
.false.  20        debug,intermediate_debug_steps
10000000             steps_between_dumps
eor

You needn’t understand this input, but you should note how I piped the input to the cast.uni.x executable. The input begins with <<eor and continues until the eor label is encountered.

Note also that I have shell variables throughout the input (eg $1, $2). This is so I can easily change the parameters of the job for each submission to Xgrid. I can vary a total of 10 different parameters in the script above.

Submitting to OpenMacGrid

To run the script above without Xgrid, I would copy it to the xgridworking directory, and issue a command similar to this:

./gridrun.sh 0.010000 0.010000 1 41 83 12 16 7 7 10000

That is, the gridrun.sh script with the 10 input parameters. Running it on Xgrid is very similar:

xgrid -h openmacgrid.macresearch.org -p <password> -job run -in xgridworking \
./gridrun.sh 0.010000 0.010000 1 41 83 12 16 7 7 10000

This would run synchronously; that is, the command would only return when the job was finished. You can also run asynchronously like this:

xgrid -h openmacgrid.macresearch.org -p <password> -job submit -in xgridworking \
./gridrun.sh 0.010000 0.010000 1 41 83 12 16 7 7 10000

The problem with this approach is that you need some way to keep track of what jobs you have submitted, which are running, which are finished, etc.

By far the easiest approach is to use Charles Parnot’s GridStuffer tool, which does all the work for you. And that’s what I did. I created a GridStuffer input file containing all my jobs, which looked similar to this:

-in xgridworking gridrun.sh 0.01 0.01 1 1 2 3 4 5 6 1 
gridrun.sh 0.010000 0.010000 1 1 2 3 4 5 6 10000 
gridrun.sh 0.010000 0.010000 1 34 29 98 1 9 90 10000
gridrun.sh 0.010000 0.010000 1 35 19 22 3 9 3 10000 
gridrun.sh 0.010000 0.010000 1 84 29 98 1 9 90 10000
gridrun.sh 0.010000 0.010000 1 77 9 22 29 19 89 10000
gridrun.sh 0.010000 0.010000 1 65 1 9 111 45 1 10000 
gridrun.sh 0.010000 0.010000 1 23 20 18 9 5 11 10000
gridrun.sh 0.010000 0.010000 1 44 94 14 5 8 1 10000 
gridrun.sh 0.010000 0.010000 1 58 91 18 6 22 98 10000
gridrun.sh 0.010000 0.010000 1 41 83 12 16 7 7 10000 
gridrun.sh 0.010500 0.010500 1 1 2 3 4 5 6 10000 
gridrun.sh 0.010500 0.010500 1 34 29 98 1 9 90 10000
...

This is just a list of all the jobs to run, with the parameters for each. Charles has good documentation explaining this format. With that done, all that was left was to submit: Hit play, and GridStuffer does the rest. Easy!

Payoff

Hopefully this shows you that if you have a serial command line tool, it is relatively trivial to leverage OpenMacGrid and get an enormous payoff for very little work. How big is the payoff? Well, I estimate that I will be able to do calculations in a few days that would have taken a year if I just ran them on my desktop Mac. Even if I used my department’s Linux cluster, and was able to have it all to myself — which I can’t — it would have taken a few weeks.

Why don’t you submit a proposal to OpenMacGrid today, and take a hard earned vacation with the time you save waiting for your calculations to finish.