Why are OpenMP directives not allowed inside pure procedure?

[rick-rainer.ludwig]

I currently use OpenMP to parallelize a legacy application. During my work we added INTENTs to parameters of subroutines and functions which are meant to be called in parallel threads and used extensively the keywork pure to get the compiler to check for side-effect free code for functions and subroutines to be sure not to change any global state. The combination of pure and OpenMP was meant to be powerful...

Now, it came to my attention, that within the OpenMP standard (including the 4.0 draft) it is written:

"Fortran Restrictions:
The following restriction applies to all OpenMP directives:
• OpenMP directives may not appear in PURE or ELEMENTAL procedures."

Why is that so? Especially INTENT and pure are great tools to force the compiler to check for side-effect-free code in Fortran. Why is it not allowed to be combined? What is the technical reason behind it? I did not find a useful hint within the standard for that.

The currently implemented code with OpenMP directives compiles and runs bug free, without any complains and it is really running in parallel, as we can see it with tools and the process management tools. The results are also checked carefully.

If it is not working that way, how can we utilize the compiler to check for side-effect-free code and run it in parallel with OpenMP? The current application make extensive use of (changed) global state for runtime control which we need to identify.

It was tested mostly with Intel Fortran Compiler (ifort).

Regards
Rick

[MarkB]

Hi there,

OpenMP directives can't appear inside PURE functions because the directives themselves will be translated into code with side effects, thus invalidating any assumptions the compiler might make about the function. However, there should be no problem with calling PURE functions from inside OpenMP parallel regions though, which I think is what you are wishing to do?

Hope that helps,
Mark.

[rick-rainer.ludwig]

Hi Mark,

thanks for your post!

However, there should be no problem with calling PURE functions from inside OpenMP parallel regions though, which I think is what you are wishing to do?

I want to do both. I want to call pure function within parallel regions for example and also within parallel DOs and also to use OpenMP directives in pure procedures.

The explanation is a legacy code base which makes heavy use of global state in form of variables in modules which are altered to signal a state within program flow like errors and also to store some parameters. We currently put all relevant subroutines and functions into modules and use INTENTs to control which parameters are altered and put also pure before these procedures to control that these do not change a global state within our application. This worked and works quite well. The results are checked against the results from sequential runs. Everything is (seems to be) fine.

OpenMP directives can't appear inside PURE functions because the directives themselves will be translated into code with side effects, thus invalidating any assumptions the compiler might make about the function.

This is clear, but what I concern about is the global state within my Fortran application. I want the compiler to check this. Why should I concern about the OpenMP implementation in the background? If OpenMP is not site effect free, it should take care about it itself, but my code should be safe, shouldn't it? Is there a technical issue what I should be aware of? The standard does not tell any reason why it is written not to use OpenMP directives in pure procedures.

Best regards
Rick

[MarkB]

Hi Rick,

I want the compiler to check this. Why should I concern about the OpenMP implementation in the background? If OpenMP is not site effect free, it should take care about it itself, but my code should be safe, shouldn't it? Is there a technical issue what I should be aware of? The standard does not tell any reason why it is written not to use OpenMP directives in pure procedures.

As I understand it, using PURE in this way was not the primary motivation for adding it to Fortran: it was to allow optimisations such as (automatic) parallel execution of FORALL loops with function calls. Some restrictions in the OpenMP standard are there for pragmatic reasons: i.e. to prevent implementors having to do a lot of work to take care of rare use cases. Sometimes such restrictions have been lifted in subsequent versions (e.g. private ALLOCATABLE arrays) where it has become clear that the use case is of sufficient interest to users. Although I am not an implementor, I can see that allowing OpenMP directives in PURE functions could interfere with optimisations elsewhere in the compiler and therefore be something of a headache.

What you are doing is using the compiler as a code analysis tool, which although sensible, probably goes beyond the scope of what language standards can reasonably be expected to support. I guess that doing your tests without OpenMP isn't convenient? Have you considered using a runtime race detection tool to catch these problems instead?

Best wishes,
Mark.

[rick-rainer.ludwig]

Hi Mark,

thanks for you answer.

If I understand it correctly, the concern is not OpenMP appying to my application per se, but procedures which have OpenMP appied to it, which are used by the Fortran compiler. The Fortran compiler might optimize the code, but when OpenMP is applied to a pure procedure, this pure procedure might break due to some OpenMP site effects.

So as an example, let's assume I have a pure procedure which has an OpenMP parallel do directive in it. If I would use a forall which calls this pure procedure, the Fortran compiler might perform an optimization by performing the forall in parallel with its own parallel implementation due to the pure keyword and the contract, that this procedure has no site effects. But, due to the OpenMP code, it might be, that running this parallel do inside the pure procedure is not possible to run in parallel. Is this that way? The OpenMP code in the pure procedure would interfere with each other?

If it is like said above, I wonder if there is no way to separate the runs correctly. The only issue I would see is, when stacking parallel executions the thread count explodes. Like running a parallel do with ten threads inside a parallel do with ten threads and so on. We can easily get more than 65535 threads...

I am still a little confused...

Best regards
Rick

[MarkB]

Hi Rick,

I'm sorry: the FORALL use case probably isn't really that relevant with respect to OpenMP. Of more relevance is that declaring functions PURE gives the compiler data dependency information without it having to to do interprocedural analysis. This can permit optmisations such as moving the function call with respect to surrounding code, or removing redundant calls with the same arguments. However, if the function contains, for example, an OpenMP barrier, such optimisations may no longer be valid. Here's an example:

Code: Select all

!$omp parallel shared(a) private(b,c,myid)

myid = omp_get_thread_num()

if (myid == 0)
   a = .....
endif

b = foo(c)

if (myid == 1)
    ....  = a
endif

!$omp end parallel

pure function foo(x)
....
!$omp barrier
...
end function foo


The compiler might assume that it could safely transform this to

Code: Select all

!$omp parallel shared(a) private(b,c,myid)

myid = omp_get_thread_num()

if (myid == 0)
   a = .....
endif

if (myid == 1)
    ....  = a
endif

b = foo(c)

!$omp end parallel

pure function foo(x)
....
!$omp barrier
...
end function


But this would cause a race condition on a.

Hope that's a bit clearer!
Mark.

[rick-rainer.ludwig]

Hi Mark,

thanks for your answer. This example is understood. You are right. The problem you show here is that the pure procedure is not self-contained. The procedure has some requirements for the environment where it is run in. This is something I would never do intentionally.

Would something speak against using

Code: Select all

!$OMP PARALLEL DO
...
!$OMP END PARALLEL DO

and also the

Code: Select all

!$OMP PARALLEL

construct within the pure procedure? It is for sure to be assured that the function would run in any context completely self contained. Do have an idea whether this could case trouble? What would happen if I would run a pure procedure with a parallel do inside in parallel? May this cause some harm?

Hopefully, I am not annoying. I only try to understand the technical reasons behind this. I did not find any documentation on this topic.

Best regards
Rick

[MarkB]

Hi Rick,

Would something speak against using

Code: Select all

!$OMP PARALLEL DO
...
!$OMP END PARALLEL DO

and also the

Code: Select all

!$OMP PARALLEL

construct within the pure procedure? It is for sure to be assured that the function would run in any context completely self contained. Do have an idea whether this could case trouble? What would happen if I would run a pure procedure with a parallel do inside in parallel? May this cause some harm?

I think if you are careful then this would be unlikely to cause a problem, especially if the pure function is not called from inside a parallel region.
A compiler would be within it's rights to throw an error, however, so your code may not be very portable!

Hopefully, I am not annoying. I only try to understand the technical reasons behind this. I did not find any documentation on this topic.

Not at all: it's an interesting question, and I would be surprised if there was anything useful written on the topic!

Mark.