calling subroutines from within parralel section

General OpenMP discussion

calling subroutines from within parralel section

Postby laughingrice » Fri Nov 16, 2007 3:43 pm

I am having a problem with my code and all of the symptoms seem to a problem with calling a subroutine to do all the work inside the parrallel section.

I have a prallel do section that calls a subroutines that has quite a few auxilarry variables and allocates and frees some memory. The code works great with only one thread. With more then one it seems that instead of running the function from each thread in a different memory space all threads share the same memory space. deallocating buffers causes segmentation faults and called function report illegal values (probably because they change under their legs).

How do I make each thread run the function in it's own memory space?

I am using the intel fortran compiler version 10.0.023 under linux (debian), at the moment with intel core duo 2?
laughingrice
 
Posts: 4
Joined: Fri Nov 16, 2007 3:37 pm

Re: calling subroutines from within parralel section

Postby laughingrice » Fri Nov 16, 2007 4:10 pm

I am guessing that the problem is with one or both of the following (seems like both)

integer, dimention(11) :: iparams

and/or

integer, allocatable, dimension(:) :: select
allocate(select(n))

...

deallocate(select(n))

it looked like these variables in a subroutine called inside the parallel do section are shared between threads and thus cause problems
laughingrice
 
Posts: 4
Joined: Fri Nov 16, 2007 3:37 pm

Re: calling subroutines from within parralel section

Postby lfm » Fri Nov 16, 2007 5:20 pm

You need to read the section in the specification about private and shared variables and data sharing attributes. Variables declared in a subroutine called from a parallel region are private, unless they are SAVE variables. If the declarations and/or allocations occur inside the subroutine then you are fine. If, however, the variables are in modules or common blocks or are declared outside the parallel region and passed as arguments, then they will be shared.

If you could post an example that represents your code it would help.

-- Larry
lfm
 
Posts: 135
Joined: Sun Oct 21, 2007 4:58 pm
Location: OpenMP ARB

Re: calling subroutines from within parralel section

Postby laughingrice » Fri Nov 16, 2007 5:39 pm

The references seem to show that I'm fine, but reality shows otherwise both with the intel compiler and with gfortran, so I must be doint something wrong. I don' t know how to make a minimal example so the relevant parts of my code (cutting out/simplifying irrelevant parts as much as possible)

There is one omp do loop directive near the start, the problematic function is EigenVecs. the two points of problem when using more then one thread
1. the call to dsaupd (arpack eigenvalue calculation) tell me that the input values are wrong (don't know which one because the error comes from farther). Not on the first run of the loop, but a bit later
2. I get a segmentation fault with the two deallocate directives one line up from the end. Removing them solves the segfault so there is some double deallocation going on

seems to me that the arrays/allocatable objects are forced to shared and save states for some reason

the code:
-----------

integer function PeronaMalikIteration(in, out, pprms, eprms)

real*8, allocatable :: v(:, :), d(:)

!$omp parallel shared(in, out, eprms, pprms) private(ret, col, row, v, d)

allocate(v(eprms%ldv, eprms%ncv), d(eprms%neigs))

!$omp do schedule(dynamic, 1)

do col = 1,n
do row = 1,n
ret = EigenVecs(in, v, d, pprms, eprms)
end do
end do
!$omp end do nowait

deallocate(v, d)

!$omp end parallel

end function PeronaMalikIteration

! ------------------------------------------------------
! calculate eigenvectors
! ------------------------------------------------------
integer function EigenVecs(img, v, d, pprms, eprms)

type(PeronaParams), intent(IN) :: pprms
type(EigParams), intent(IN) :: eprms

real*8, intent(IN) :: img(eprms%windowSz, eprms%windowSz), &
v(eprms%ldv, eprms%ncv), d(eprms%neigs)

! Algorithm parameters
! Relative Tolerance, set to default - machine precision
real*8 :: tol = 0.0d0
real*8 :: sigma = 0.0d0

! Algorithm variables
integer :: ido, lworkl, info
integer, dimension(11) :: iparam, ipntr
real*8, allocatable, dimension(:) :: resid, workd, workl
!DEC$ ATTRIBUTES ALIGN: 16 :: resid, workd, workl
logical :: rvec = .true.
logical, allocatable, dimension(:) :: select
character(len=1) :: B = 'I'
character(len=2) :: which = 'LA'

! Set parameter values
! Default return, no error
EigenVecs = 0
! Initial parameters to the dsaupd routine
ido = 0
info = 0
! (/ ISHIFT, -, MXITER, NB, NCONV, -, MODE, NP, NUMOP, NUMOPB, NUMREO /)
iparam = (/ 1 , 0, 1000*eprms%neigs, 1 , 0 , 0, 1 , 0 , 0 , 0 , 0 /)
! Workspace
lworkl = eprms%ncv*(eprms%ncv + 8)
allocate(resid(eprms%n), workd(3*eprms%n), workl(lworkl))

do
call dsaupd(ido, B, eprms%n, which, eprms%neigs, tol, resid, &
eprms%ncv, v, eprms%ldv, iparam, ipntr, workd, workl, lworkl, info)

select case (ido)
case (-1,1)
! multiply y <- OP*x
! where x = workd(ipntr(1))
! y = workd(ipntr(2))
call PeronaMalikResponse(workd(ipntr(1)), workd(ipntr(2)), img, &
eprms, pprms)
case default
exit
end select
end do

! Error !!!
if (info /= 0) then
EigenVecs = info
goto 99999
end if

allocate(select(eprms%ncv))

call dseupd(rvec, 'A', select, d, v, eprms%ldv, sigma, B , eprms%n, &
which, eprms%neigs, tol, resid, eprms%ncv, v, eprms%ldv, iparam, &
ipntr, workd, workl, lworkl, info)

deallocate(select)
99999 deallocate(resid, workd, workl)
end function EigenVecs
laughingrice
 
Posts: 4
Joined: Fri Nov 16, 2007 3:37 pm

Re: calling subroutines from within parralel section

Postby lfm » Sat Nov 17, 2007 1:14 pm

I don't see anything obviously wrong, but I don't know anything about the routines you are calling inside of EigenVecs. The allocates of u and v inside the parallel region and the allocates of resid, workd, workl, and select, as well as all of the local variables inside EigenVecs will be private.

However I question what this code is doing. You have all of the parameters to EigenVecs as intent(in). That means that they aren't modified. And you don't use any of the outer loop indices col and row. So you workshare the 'do col' loop, but then the iterations don't seem to be computing anything. So perhaps there is a problem in your logic.

-- Larry
lfm
 
Posts: 135
Joined: Sun Oct 21, 2007 4:58 pm
Location: OpenMP ARB

Re: calling subroutines from within parralel section

Postby laughingrice » Sat Nov 17, 2007 4:31 pm

maybe trying to simplify the program is wrong, here is the whole thing. ReconstructBlock which doesn't appear here doesn't seem to be the problem since commenting it out doesn't change anything and leaving only it solves the issue. The issue is with the call to EigenVecs. it computes a whole lot of eigenvectors of an operator using arpack. All the libraries used seem to be thread safe as far as i can tell. changing compiler doesn't help either. If I use more then one thread usually dsaupd reports memory corruption and ocationally either it or one of the deallocates segfaults. I also get the occasional abort signal. I tried making a minimal example but i can't reproduce the problem.

integer function PeronaMalikIteration(in, out, pprms, eprms)

real*8, dimension(r, c), intent(IN) :: in
real*8, dimension(r, c), intent(OUT) :: out

type(PeronaParams), intent(IN) :: pprms
type(EigParams), intent(IN) :: eprms

! Storage for the eigenvectors/eigenvalues
real*8, allocatable :: v(:, :), d(:)
!DEC$ ATTRIBUTES ALIGN: 16 :: v, d

! Parameters for window calculation
integer :: winLeft, winBot, colOffset, rowOffset, row, col
integer :: threadId, OMP_GET_THREAD_NUM

integer :: ret

!$OMP PARALLEL SHARED(in, out, eprms, pprms) PRIVATE(ret, winLeft, winBot, colOffset, rowOffset, col, row, v, d, threadId) NUM_THREADS(2)

allocate(v(eprms%ldv, eprms%ncv), d(eprms%neigs))

!$OMP DO SCHEDULE(DYNAMIC, 1)
! TODO: handle the last sub-block
do col = eprms%blockR + 1, eprms%c - eprms%blockR, eprms%blockSz
do row = eprms%blockR + 1, eprms%r - eprms%blockR, eprms%blockSz
threadId = OMP_GET_THREAD_NUM()
print *, "col: ", col, "row: ", row, "thread: ", threadId

! The window has to be centered around the pixel in question IF
! possible but still inside the boundaries of the image

winLeft = max(col - eprms%windowR, 1)
winLeft = min(winLeft, eprms%c - eprms%windowSz + 1)
colOffset = col - eprms%blockR - winLeft

winBot = max(row - eprms%windowR, 1)
winBot = min(winBot, eprms%r - eprms%windowSz + 1)
rowOffset = row - eprms%blockR - winBot

!PeronaMalikIteration =
ret = EigenVecs( &
in(winBot:winBot+eprms%windowSz-1, winLeft:winLeft+eprms%windowSz-1), &
v, d, pprms, eprms)

call ReconstructBlock( &
in(winBot:winBot+eprms%windowSz-1, winLeft:winLeft+eprms%windowSz-1), &
out(winBot+rowOffset:winBot+rowOffset+eprms%blockSz-1, &
winLeft+colOffset:winLeft+colOffset+eprms%blockSz-1), &
colOffset, rowOffset, v, d, pprms, eprms)
end do
end do
!$OMP END DO NOWAIT

deallocate(v, d)

!$OMP END PARALLEL

PeronaMalikIteration = 0
end function PeronaMalikIteration

! ------------------------------------------------------
! calculate eigenvectors
! ------------------------------------------------------
integer function EigenVecs(img, v, d, pprms, eprms)

type(PeronaParams), intent(IN) :: pprms
type(EigParams), intent(IN) :: eprms

real*8, intent(IN) :: img(eprms%windowSz, eprms%windowSz), &
v(eprms%ldv, eprms%ncv), d(eprms%neigs)

! Algorithm parameters
! Relative Tolerance, set to default - machine precision
real*8 :: tol = 0.0d0
real*8 :: sigma = 0.0d0

! Algorithm variables
integer :: ido, lworkl, info
integer, dimension(11) :: iparam, ipntr
real*8, allocatable, dimension(:) :: resid, workd, workl
!DEC$ ATTRIBUTES ALIGN: 16 :: resid, workd, workl
logical :: rvec = .true.
logical, allocatable, dimension(:) :: select
character(len=1) :: B = 'I'
character(len=2) :: which = 'LA'

! Set parameter values
! Default return, no error
EigenVecs = 0
! Initial parameters to the dsaupd routine
ido = 0
info = 0
! (/ ISHIFT, -, MXITER, NB, NCONV, -, MODE, NP, NUMOP, NUMOPB, NUMREO /)
iparam = (/ 1 , 0, 1000*eprms%neigs, 1 , 0 , 0, 1 , 0 , 0 , 0 , 0 /)
! Workspace
lworkl = eprms%ncv*(eprms%ncv + 8)
allocate(resid(eprms%n), workd(3*eprms%n), workl(lworkl))

do
call dsaupd(ido, B, eprms%n, which, eprms%neigs, tol, resid, &
eprms%ncv, v, eprms%ldv, iparam, ipntr, workd, workl, lworkl, info)

select case (ido)
case (-1,1)
! multiply y <- OP*x
! where x = workd(ipntr(1))
! y = workd(ipntr(2))
call PeronaMalikResponse(workd(ipntr(1)), workd(ipntr(2)), img, &
eprms, pprms)
case default
exit
end select
end do

! Error !!!
if (info /= 0) then
EigenVecs = info
goto 99999
end if

allocate(select(eprms%ncv))

call dseupd(rvec, 'A', select, d, v, eprms%ldv, sigma, B , eprms%n, &
which, eprms%neigs, tol, resid, eprms%ncv, v, eprms%ldv, iparam, &
ipntr, workd, workl, lworkl, info)

deallocate(select)
99999 deallocate(resid, workd, workl)
end function EigenVecs
laughingrice
 
Posts: 4
Joined: Fri Nov 16, 2007 3:37 pm

Re: calling subroutines from within parralel section

Postby Toey_Hylton » Wed Sep 24, 2008 12:02 pm

Laughingrice,

I don't know if your problem have been solved or not. Currently, I met the similar problem as yours. Inside a subroutine, I have allocatable variables. If I ran with one thread, it's okay. If I ran with two threads, I got varialbes have been allocated error information. I am trying to find the solution, but not yet. If you have any good ideal, would you like share with me? Thank you very much.
Toey_Hylton
 
Posts: 5
Joined: Mon Sep 22, 2008 11:32 am


Return to Using OpenMP

Who is online

Users browsing this forum: Yahoo [Bot] and 1 guest

cron