Creating many OMP TASKs

General OpenMP discussion

Creating many OMP TASKs

Postby __Sayan__ » Wed Jul 18, 2012 7:59 am

I had a previous embarrassingly parallel code, which had an OMP DO implementation. I want to evaluate the performance of OMP TASK, and restructuring the code like below. No matter what compiler I use, I have some problem (either segfault or the program runs indefinitely) which makes me sure that there is a fundamental problem with respect to placement of OMP pragmas.

Code: Select all
!$OMP PARALLEL
...
!$OMP SINGLE
CALL abc()
CALL def()
CALL ghi()
!$OMP END SINGLE
!$OMP END PARALLEL


Each subroutine is like this:

Code: Select all
SUBROUTINE abc
DO i=i0,i1
   DO j=j0,j1
      DO k=k0,k1
          !$OMP TASK UNTIED
          <embarrassingly parallel computation>
          !$OMP END TASK
      ENDDO
     ENDDO
ENDDO
END SUBROUTINE abc


Above is a general structure of my program, according to my thoughts on OMP Tasks, please let me know if my understanding is wrong, and ways to correct this code.
__Sayan__
 
Posts: 18
Joined: Fri Jun 18, 2010 9:34 am

Re: Creating many OMP TASKs

Postby MarkB » Fri Jul 20, 2012 3:14 am

The general structure looks fine to me. You might want to convince yourself that you have got the data scope attributes right: for example are there any local variables in the subroutines which are shared by tasks? They could go out of scope before the tasks are guaranteed complete at the END SINGLE.
MarkB
 
Posts: 428
Joined: Thu Jan 08, 2009 10:12 am

Re: Creating many OMP TASKs

Postby __Sayan__ » Mon Jul 23, 2012 11:03 am

Thank you for confirming. I do think it is a scoping error. But, since this is an embarrassingly parallel code, so as in the OMP DO implementation, ALL the left hand side parameters are declared as private...I am still not able to find the source of the problem.
__Sayan__
 
Posts: 18
Joined: Fri Jun 18, 2010 9:34 am

Re: Creating many OMP TASKs

Postby MarkB » Tue Jul 24, 2012 6:50 am

Couple of suggestions to try and narrow it down:

Does it work if you remove the UNTIED clause (there is likely no need for this)?
Does it work if you put a TASKWAIT directive before the end of each subroutine?
MarkB
 
Posts: 428
Joined: Thu Jan 08, 2009 10:12 am

Re: Creating many OMP TASKs

Postby __Sayan__ » Wed Jul 25, 2012 7:13 am

Let me post the revised code structure:
Code: Select all

!$OMP PARALLEL SHARED(...) PRIVATE(...) FIRSTPRIVATE(...)
!$OMP SINGLE
!$OMP TASK UNTIED
CALL test(...)
!$OMP END TASK
!$OMP END SINGLE
!$OMP END PARALLEL


In the test function:

Code: Select all
SUBROUTINE test(...)
...
DO i=i0,in
DO j=j0,jn
DO k=k0,kn
!$OMP TASK SHARED(...) FIRSTPRIVATE(...) !variables are either shared or firstprivate here
<embarrassingly parallel code>
!$OMP END TASK
ENDDO
ENDDO
ENDDO
!$OMP TASKWAIT
END SUBROUTINE test


The code runs indefinitely; in the task construct inside loops, I have explicitly declared shared clause of the data that is shared, and rest is firstprivate. In this code structure, do you have any advice w.r.t scoping?
If I remove the outer untied task, and remove the data specifications like shared/firstprivate from the inner task (and let it be just !$OMP TASK as you mentioned, with a taskwait in the end), then also the code runs indefinitely.

Thanks.
__Sayan__
 
Posts: 18
Joined: Fri Jun 18, 2010 9:34 am

Re: Creating many OMP TASKs

Postby MarkB » Wed Jul 25, 2012 7:38 am

How much time does the computation inside each task take, and how many tasks get created in total?
MarkB
 
Posts: 428
Joined: Thu Jan 08, 2009 10:12 am

Re: Creating many OMP TASKs

Postby __Sayan__ » Wed Jul 25, 2012 8:24 am

With my test data, around 64 million tasks are getting created; and the time to compute for single OpenMP thread is around 100 secs.
__Sayan__
 
Posts: 18
Joined: Fri Jun 18, 2010 9:34 am

Re: Creating many OMP TASKs

Postby MarkB » Fri Jul 27, 2012 2:49 am

Your tasks are likely too short to see any speedup, and you might just be overwhelming the runtime (by the way, what compiler are you using?).
It might be worth trying to move the task construct out to include the k loop, or even both k and j loops.
MarkB
 
Posts: 428
Joined: Thu Jan 08, 2009 10:12 am

Re: Creating many OMP TASKs

Postby __Sayan__ » Fri Jul 27, 2012 2:17 pm

You are correct. The tasks are too many, I think every compiler has some limit on the number of tasks, and once that is reached, the tasks are serialized(?). Anyways, as you suggested, I modified the code by placing the tasking construct after the outermost i-loop, to limit the number of tasks generated to N, instead of N^3. With 8 threads, I see that the performance is close to the OMP DO version (which is still best).
Modified code, I have removed the outer task construct from the code block which is calling test(..), rest is intact.
Code: Select all
SUBROUTINE test(...)
...
DO i=i0,in
!$OMP TASK FIRSTPRIVATE(...) PRIVATE(...) DEFAULT(SHARED)  & !variables are either private or firstprivate here
!$OMP FIRSTPRIVATE(i) PRIVATE(j,k)
DO j=j0,jn
DO k=k0,kn
<embarrassingly parallel code>
ENDDO
ENDDO
!$OMP END TASK
ENDDO
!$OMP TASKWAIT
END SUBROUTINE test

I am using the latest Ifort (12.1.5) and pgf90 (12.3) - unfortunately ifort compiled version is producing NaNs and taking more time than as opposed to the PGI version.
Any ideas to optimize this further is welcome.
__Sayan__
 
Posts: 18
Joined: Fri Jun 18, 2010 9:34 am

Re: Creating many OMP TASKs

Postby MarkB » Mon Jul 30, 2012 6:39 am

Great, glad that seems to have solved the problem! I cannot see any obvious way to improve this further: typically using a DO directive will have lower overhead that using tasks, so I am not surprised that you observed this too....
MarkB
 
Posts: 428
Joined: Thu Jan 08, 2009 10:12 am

Next

Return to Using OpenMP

Who is online

Users browsing this forum: Google [Bot] and 8 guests