Global scope & threadprivate leads to undefined symbol

General OpenMP discussion

Global scope & threadprivate leads to undefined symbol

Postby andeb » Fri Aug 15, 2008 7:05 am

Hi to all,

I am trying to compile my c++ code on AIX 5.3. I am havin trouble with global variables that I need to make private to each thread. I use the threadprivate directive that I call just underneath my global variables, as such:

#include <omp.h>
double toto; // ==> Global scope, external linkage
#pragma omp threadprivate(toto)
void myFunc(...)
{
#progma omp parallel for
for (int i=0;i<2;i++)
{
toto = something that changes within the loop!
double tata = Funct2();
}

In Funct2.C, I obviously redefine the global variable, using
extern double toto;

All the .o are created, but at the linkage, I get the following error:
ld: 0711-317 ERROR: Undefined symbol: toto

Please help,
Thanks
andeb
 
Posts: 7
Joined: Fri Aug 15, 2008 6:54 am

Re: Global scope & threadprivate leads to undefined symbol

Postby ejd » Fri Aug 15, 2008 9:14 am

So in Funct2.c, have you declared toto as threadprivate??

Code: Select all
extern double toto;
#pragma omp threadprivate(toto)
ejd
 
Posts: 1025
Joined: Wed Jan 16, 2008 7:21 am

Re: Global scope & threadprivate leads to undefined symbol

Postby Guest » Fri Aug 15, 2008 10:38 am

I actually just tried! Now it compiles fine. However, at the execution, after several iterations, I get segmentation fault. Any idea why? With 1 CPU, it works fine.

Is there any other directive or clause to add? Any suggestion to help me debug?
Tks,
regards
Guest
 

Re: Global scope & threadprivate leads to undefined symbol

Postby ejd » Fri Aug 15, 2008 10:47 am

You haven't given me anything really to go on. A segmentation fault is usually caused by you stepping outside of an array. Are you using any arrays? If so, then have you made sure that the appropriate "limits" have been set for AIX and for OpenMP for stacksize?
ejd
 
Posts: 1025
Joined: Wed Jan 16, 2008 7:21 am

Segmentation fault in parallel mode, but not in serial!

Postby andeb » Mon Aug 18, 2008 1:55 pm

Hi,

Sorry for the vague response. Indeed, I am using arrays. However, I don't understand why I would step out of them in parallel, but not in serial.. (All arrays are dynamically allocated using new + delete)

As you suggested, I investigated the possibility of running out of memory since some of my shared variables are considerably big (huge c++ map for instance). Therefore, I added the following in my .bashrc:
ulimit -s 200000
export XLSMPOPTS="stack=200000000"
but the segmentation fault remains. The error message looks like:
Rtask(s) 1 : exited with signal <11>
/usr/local/lsf/bin/lsntbl_run: 74202 Memory fault
ERROR: 0031-250 task 0: Segmentation fault


Also, I got rid of all global variables and thus the threadprivate directive. Now, every variables are "automatic" storage, local scope.

Are you aware of any tool that can spot memory leaks or thread issues for AIX 5.3? I found Valgrind but I can't install it due to IT restrictions.
Thank you for your time & effort,
Best regards
andeb
 
Posts: 7
Joined: Fri Aug 15, 2008 6:54 am

Re: Global scope & threadprivate leads to undefined symbol

Postby ejd » Tue Aug 19, 2008 7:20 am

I am afraid that I can't help you with AIX. I work for Sun Microsystems and if you were running on Solaris or Linux, we have tools that I could suggest. I don't know IBM software well enough to help. Maybe someone else reading this forum might have some ideas.

When running sequentially on a Unix system, you have to make sure your stack size is large enough for the data. Not knowing the exact amount of stack space you need is quite common and you end up playing with setting the limit. When going to OpenMP, the problem is multiplied, since now each thread has it's own stack. The stack for the main thread is allocated using the Unix limit value. However, the stacks for the worker threads are allocated using implementation defined values (that can usually be increased with some implementation defined environment variable - until OpenMP Version 3 where the name of the environment variable has been standardized). This is why people moving from serial to parallel quite often run into this problem.

Since I have no idea what size your data is, I don't have any idea whether 200M is a large enough stack or not. However, since you are using new and delete for your arrays, this data should be allocated from the heap. I assume that you are checking to make sure that the value returned from new is non-zero. I believe that you can compile the program using a low optimization level and run it to get the line number where the program is crashing. There should also be a debugger that should allow you to examine variable values.

Since I don't know your level of expertise, the above may be something you already know (and if so I apologize).

As for valgrind, it is better than nothing for sure. However, if you have a large program and/or large amounts of data, then it will take a lot of time to run your program through it. So far (as I know), no one has figured out a way to check for memory leaks and thread issues with parallel programs without a large amount of overhead.
ejd
 
Posts: 1025
Joined: Wed Jan 16, 2008 7:21 am

Usage of pointers with c+ objects cause seg fault?

Postby andeb » Tue Aug 19, 2008 3:55 pm

Thanks for the info. I wasn't really aware of the stack and heap so I read about it. Interesting!

But I don't think now that my problem is related to a lack of memory. I read the following pdf:
http://hpc.cs.tsinghua.edu.cn/research/ ... rboven.pdf

In there, it is explained that a function can be thread-safe only if it uses variables from the stack. Does it mean I can't dynamically allocate my arrays & objects? Actually, my parralelized loop has an embedded function that declares a slew of pointers of c++ objetcts ... :cry: In the article above, it is also sais that
"Using pointers you can get access to everything – but that is not
allowed by the OpenMP specification and therefore the behavior
is undefined"

Do you agree with that statement? Can a loop be parallelized if there is usage of pointers of objects?
Should I rather create a separate executable with that function and call that executable in the parallelized loop. (in which case the memory would be released without burden)

Thank you again for your time,
cheers!
andeb
 
Posts: 7
Joined: Fri Aug 15, 2008 6:54 am

Re: Usage of pointers with c+ objects cause seg fault?

Postby ejd » Wed Aug 20, 2008 7:09 am

andeb wrote:Thanks for the info. I wasn't really aware of the stack and heap so I read about it. Interesting!

I am glad that what I wrote was of some help.

andeb wrote:But I don't think now that my problem is related to a lack of memory. I read the following pdf:
http://hpc.cs.tsinghua.edu.cn/research/ ... rboven.pdf

Christian understand using C++ and OpenMP very well. He has been doing it for years and is greatly responsible for most of the C++ enhancements being made to OpenMP Version 3.

andeb wrote:In there, it is explained that a function can be thread-safe only if it uses variables from the stack. Does it mean I can't dynamically allocate my arrays & objects? Actually, my parralelized loop has an embedded function that declares a slew of pointers of c++ objetcts ... :cry: In the article above, it is also says that
"Using pointers you can get access to everything – but that is not allowed by the OpenMP specification and therefore the behavior is undefined"

Not totally true. Shared variables can be used in thread-safe routines with restrictions. You can read shared variables without any problem. However, if any routine updates a shared variable, then all accesses have to be "protected". By "protected" I mean you have to use critical, atom, or locks. Your routines can also update different sections of a shared array without any problem.

C++ objects however, can be a problem. For example, if you have a list object. Internal to the list implementation, information is kept about the number of elements in the list (for example). If two routines both try and update the list, then depending on the implementation, it might not work. The problem is, that to do the update correctly, a lock has to be added around the update to the number of items in the list. For serial programs, this is added overhead that most people don't want. For parallel programs, it is a necessity. So, if you have a shared object, to be thread-safe, you are most likely going to have to "protect" the list update.

As for the statement about using pointers, the OpenMP specification does limit their behavior. However, an implementation of OpenMP most likely doesn't. The problem is, that from a user's perspective, we want to say that private data (data declared in a private clause) is private to a thread (or task). However, the user could use a pointer to update it from a different thread - because the base languages allow this (since the language really doesn't know anything about OpenMP). The statement is really there to make sure that the user thinks about what they are doing (if you are going to play with fire you might get burned and we can't help that).

andeb wrote:Do you agree with that statement? Can a loop be parallelized if there is usage of pointers of objects?
Should I rather create a separate executable with that function and call that executable in the parallelized loop. (in which case the memory would be released without burden)

So the answer is, that you can parallelize loops with pointers or objects. You just have to be careful and you might not get as much benefit as you were hoping for.
ejd
 
Posts: 1025
Joined: Wed Jan 16, 2008 7:21 am

Re: Global scope & threadprivate leads to undefined symbol

Postby andeb » Thu Aug 21, 2008 11:17 am

Hi ejd,

I finally got it working! I added critical directives before the c++ objects initialization and I created a separate executable with the function that uses lots of dynamic memory allocation of objects.

Thank you again for your help,
Best regards
andeb
 
Posts: 7
Joined: Fri Aug 15, 2008 6:54 am


Return to Using OpenMP

Who is online

Users browsing this forum: No registered users and 10 guests