shared array and volatile in Fortran

[buttari]

Hello,
is this code safe?

Code: Select all

subroutine pca(nth)
  use omp_lib
  integer :: p, n
  integer, volatile  :: prog(0:nth-1)

  prog=-1
  !$omp parallel private(p) shared(prog) num_threads(nth)
  p = omp_get_thread_num()

  if(p.ne.nth-1) then
     do while(prog(p) .ge. prog(p+1))
     end do
  end if
  write(*,*)p
  prog(p) = prog(p)+1
  !$omp end parallel

  return

end subroutine pca


Assume this is part of a bigger code (i.e., there is much more code in the parallel region) and that the "do while" loop is used to synchronize all threads (no, I don't want to use a barrier ). Is it possible that prog(p+1) is kept inside a register and its update is not seen by thread p ? Because of the "volatile" I would say no but sometimes the code fails. can anybody confirm? If I put a !$omp flush inside the do-while loop then the code always works.

Thanks in advance,
Alfredo

[ftinetti]

is it possible that prog(p+1) is kept inside a register and its update is not seen by thread p ?

I think so, according to the OpenMP relaxed-consistency, shared-memory model.

...Because of the "volatile" I would say no but sometimes the code fails.

Well, the "volatile" special behavior, I think it is language- and OpenMP version implementation- dependent. More specifically, Spec 2.5 defines (copied, p. 12):

The volatile keyword in the C and C++ languages specifies a consistency
mechanism that is related to the OpenMP memory consistency mechanism in the
following way:...

i.e. it seems to be applied only to C and C++ code. Also, in Spec 3.0, this special behavior has been removed, in "APPENDIX F: Changes from Version 2.5 to Version 3.0" (p. 315):

The description of the behavior of volatile in terms of
flush was removed.

and this is why I think it is also version version implementation dependent. Btw, what compiler are you using?

There are other synchronization-by-variables-issues I do not understand very well and I don't have enough time right now to dig into, such as the example Spec 3.1 (p.166) Example A.2.2f.

HTH,

Fernando.

[buttari]

Dear Fernando,
thanks for quick and detailed response. The volatile attribute is a Fortran 2003 feature so this is probably the reason why the 2.5 specs (written in 2005 when basically no F2003 compiler was available) only speaks about C/C++.
However, I'm impressed to see that the 3.1 and 3.0 specs deliberately ignore this attribute in both C/C++ and Fortran. I think the behavior in the 2.5 specs makes sense and I can't see any reason why it was changed.
Can you, or anybody else, explain me why?
It seems to me that the OpenMP standard overrides the behavior expected by the language standard, isn't it so?

I know that flushes fix the problem but a flush is an expensive system call and I don't want to do it to frequently.

[ftinetti]

Well, I'm not an OpenMP ARB member, so I can only guess... maybe someone else will be able to provide better answers...

I think the main problem has been about implementation and performance penalties incurred in so many "implied flushes" given by

a reference that reads the value of an object with a volatile-qualified type
behaves as if there were a flush operation on that object at the previous sequence point,
while a reference that modifies the value of an object with a volatile-qualified type
behaves as if there were a flush operation on that object at the next sequence point.

(Spec. 2.5, p. 12, about C/C++ volatile) besides I don't understand and I did not find what a "sequence point" exactly is... Other reasons: a) Fortran volatile had different requirements/definitions for compilers (I don't know which ones, however, from a computer language point of view I look them as equal/similar, but I don't know a lot about Fortran, and I do know Fortran has a lot of details...) , b) They do not want to have special definitions for language specific cases/definitions (or Fortran "object" attributes, according to the Fortran standard/s). Looking at the Fortran 03 and 08 standards, the note

The Fortran processor should use the most recent definition of a volatile object when a value
is required. Likewise, it should make the most recent Fortran definition available. It is the
programmer’s responsibility to manage the interactions with the non-Fortran processes.

(Note 5.21 in F03, and Note 5.25 in F08) I tend to agree with you in some "overriding behavior", but I do not know if there is some "elegant" way of solving it... unless the Spec "goes back" to 2.5 treatment of volatile data...

Remember: I'm just guessing...

Fernando.

[buttari]

Fernando,
thanks again for helping me understand. As far as my understanding goes, the volatile attribute should simply tell the compiler to avoid keeping copies of some data inside registers or in a thread's private view of the memory; this doesn't mean implicitly doing flushes, but, I guess, it means that the behavior is exactly as if the data were not volatile and a flush were inserted before reads/writes to the data. Therefore I don't see/understand the performance penalty issue.
But, like you, I'm only guessing. It would be great if somebody in the OpenMP standard committee could shed some light on this topic.

Regards,
alfredo

[MarkB]

It would be great if somebody in the OpenMP standard committee could shed some light on this topic.

OK, I'll try.....

The semantics of volatile in C/C++ and Fortran do not make sufficient guarantees for volatile accesses to be equivalent to a flush: in particular they do not enforce the ordering in which reads and writes of the volatile variable are visible to other threads with respect to other reads/writes in the program. See here for more discussion of this: http://software.intel.com/en-us/blogs/2 ... ogramming/ This is why reference to C/C++ volatile was removed from the OpenMP memory model: it simply doesn't work. Fortran volatile was never included since OpenMP did not (and still does not) address Fortran 2003.

To address your original problem, you should not try to use volatile variables to synchronize threads! It is possible to do this using flush, and the cost may not be very expensive: an OpenMP flush is not usually implemented with a heavyweight system call.
However, using flush correctly is not easy, and I would recommend that you take a close look at the examples in Section A.2 of the 3.1 standard first in order to understand why!

[MarkB]

You could code your example using OpenMP inbuilt synchronization as follows:

Code: Select all

subroutine pca(nth)
  use omp_lib
  integer :: p, nth
 
!$omp parallel private(p)  num_threads(nth)

!$omp do ordered schedule(static,1)
  do p = nth,1,-1
!$omp ordered
  write(*,*)p
!$omp end ordered
   end do
!$omp end do nowait

!$omp end parallel

  return

end subroutine pca


Perhaps you might be able adapt this idea to your real code? Another option might be to use an array of locks. In any case, I would certainly recommend exploring solutions which use OpenMP directives/routines before trying to use regular variables for synchronization.

[ftinetti]

@MarkB: Thanks for the explanations.

Fortran volatile was never included since OpenMP did not (and still does not) address Fortran 2003.

Uh, I didn't know. Would it be possible to add this statement to the Spec? Well... now I'm not sure if it would be useful...

Thanks again,

Fernando.

[MarkB]

Thanks for the explanations.

You're welcome!

Would it be possible to add this statement to the Spec?

I think you can be pretty certain that we won't be including Fortran volatile in the OpenMP memory model, unless sometime in the future it acquires meaningful multi-threading semantics!

[buttari]

OK, I'll try.....

The semantics of volatile in C/C++ and Fortran do not make sufficient guarantees for volatile accesses to be equivalent to a flush: in particular they do not enforce the ordering in which reads and writes of the volatile variable are visible to other threads with respect to other reads/writes in the program. See here for more discussion of this: http://software.intel.com/en-us/blogs/2 ... ogramming/ This is why reference to C/C++ volatile was removed from the OpenMP memory model: it simply doesn't work. Fortran volatile was never included since OpenMP did not (and still does not) address Fortran 2003.

To address your original problem, you should not try to use volatile variables to synchronize threads! It is possible to do this using flush, and the cost may not be very expensive: an OpenMP flush is not usually implemented with a heavyweight system call.
However, using flush correctly is not easy, and I would recommend that you take a close look at the examples in Section A.2 of the 3.1 standard first in order to understand why!

Mark,
thanks for the explanations. I found the discussion in the link you sent very instructive.

Regards,
Alfredo