Hi consultants:

Just want to bounce this off you. The same tiegcm model
code that uses shared-mem OpenMP on the Origin and J90's 
(its pretty fast on ute) does not work on prospect.
It runs fine single-threaded, but with > 1 processor, 
only the master proc gets it right. This is the *same* 
source that runs correctly on ute (w/ a few caviats for i/o, 
libs, etc). No MPI here, only OpenMP. (Actually, I load
-lmpi and -lelan to avoid unsatisfied symbols, but the
mpi calls are isolated with #ifdef -DMPI, which is set
only under AIX, so they are not currently compiled on 
prospect.)

Its as if some data that's meant to be global across
threads (probably regular common), and is set by the 
master proc before the parallel loops, is not getting 
to the slaves. As if it were considered local to the
master. Again, this works fine on the SGI. This is a 1-d
decomposition over latitude, where nlat=36. On the Compaq,
if I use 4 threads, only the 1st 9 latitudes (nlat/4) are
correct, i.e., the master's share of the decomposition.
If I run w/ 2 threads, half of them are correct. The only
thread relevant flag I've set on prospect is -omp (on both
FFLAGS and LDFLAGS). I am getting some overflows and underflows 
(avoided for now with -fpe1 or 2), but it reports them when 
run serially as well, where the results are good. I'm using
-O5 (but not -fast -- I get internal compiler erors with -fast).

I make extensive use of !$THREAD_PRIVATE common, and
I know IBM has been late implementing this (I havent
been able to use OpenMP yet on the IBM's), but I have 
run small test codes on prospect testing both regular 
and private common and it appears to work. In the model
it looks like something is considered master-private 
when it's meant to be thread-global. Maybe there are still 
some data statements initializing common outside block data,
but I don't think so because the IBM compiler complains
about that and I've put them in block data. I don't think
I initialize any thread-private commons with block data
but maybe that's something I should verify.
Thanks for listening and thanks for any suggestions..

--Ben

-----------------------------------------------------------------------
Ben Foster		      	High Altitude Observatory (HAO)
foster@ncar.ucar.edu		phone: 303-497-1595  fax: 303-497-1589  
Nat. Center for Atmos. Res.     P.O. Box 3000 Boulder CO 80307 USA
-----------------------------------------------------------------------