openMP fast on Linux slow on osx

By Nic Warren at Tue, Aug 19 2008 5:04pm |

Hi there,
Recently I have been compiling some code on a Mac Pro (2x 3.0GHz Quad-Core Intel Xeon) with icc. However I have been seeing some strange differences between platforms.
For example I compiled some test code on the same machine, one on openSUSE, one on mac 10.5.4, both with the intel compiler, both with OMP_NUM_THREADS=8.

The test problem in osx not only runs slower (18s wall clock, 120s sum cpu time), buy runs faster with less cores.

However, as described above, same machine, same compiler etc, just on linux, the test problem runs with a wall clock time of 1.2s and sum cpu time of ~6s!

Has anyone come across a similar problem before? Does any one know if it is a platform issue or a platform version of icc issue which is the problem. Does anyone know any fixes?

Any help would be most appreciated.

Thanks heaps,

Nic

‹ Help on developing a downloader application Problems with RAM when compiling with Intel Fortran ›
Wed, 2008-08-27 22:31 — jsgonzal

Are you runnigng icc with

Are you runnigng icc with the shell variables for 64bits?

The only idea I have

Thu, 2008-08-28 17:22 — Nic Warren

No it's definitely 32bit.

No it's definitely 32bit. I'm pretty sure it's something to do with the thread scheduling within the OS. Anyone with any ideas on how to control this?

Fri, 2008-08-29 10:10 — dgohara

Re:No it's definitely 32bit.

This is almost certainly a thread scheduling issue. Unfortunately, there isn't much you as a user can do about this at the moment. There have been a lot of discussions about this issue, and Apple is addressing it for Snow Leopard.

Is there a reason to use 32-bit? If not, you may want to try 64-bit as well. Depending on the constraints of your code, you may see some performance boost. As the 64-bit ABI gets you two things:

1) Away from the stack based 32-bit ABI (variables are passed in register like on PPC)
2) Access to double the number of registers (for CPU bound numerical code this can be a win)

Dave

Tue, 2008-09-23 03:36 — Pascal

OpenMPI

I am not very much into this topic, but I'm using an OpenMPI-enabled program (MEME) and compiling this under OS X (my MacBook Pro Core 2 Duo) with gcc seems to work perfectly. I haven't compared running times on the same machine under both OS'es, but it scales perfectly with the cores under OS X and runs faster (about as expected) than on an older Xeon Server running Ubuntu.

Sat, 2008-11-01 12:23 — Navier-Stokes

OpenMPI is not OpenMP

May be you're mistaking OpenMPI for OpenMP. OpenMPI is an implementation of the Message Passing Interface, which connects distributet processes via network. OpenMP is an API to control multithreaded execution in shared memory on a single machine.

Mon, 2008-11-03 03:58 — Pascal

Indeed, OpenMPI != OpenMP

Oops, I'm sorry, I was indeed mistaken. :-)