mess with input files

General OpenMP discussion

mess with input files

Postby dva2tlse » Mon Sep 09, 2013 12:42 pm

Hello everbody on the forum,
first of all, please excuse my bad or wavering english, since I'm French, from Toulouse.

I am working on a rather big fortran 90 program (big for me) which needs acceleration, so I try to mofify it with openMP since at work we have several linux machines with up to 32 or 40 processors.

The program I'm working on takes 141 seconds long on a single proc' for 16 units of work, and that puts the unit of work at about 6,5 s; we need to run three hundred thousand units of work (300,000) during something like one week, what puts the time left for one unit of work at about 2 seconds.

Up to now, it works rather correctly on one proc', but as soon as I ask for two of them, it fails with the input data.

My program has a "C$OMP MASTER" directive at its beginning, to enable sequential reading of most of the input data, before a "C$OMP END MASTER" directive. Then it encounters à "C$POMP PARALLEL DO" directive, to make the atual work by two threads. (maybe more later if there is enough memory)

The problem arises when it is needed to read other bunches of input data, pertinent to each new other unit of work. I tried to manage the different input files with fortran units numbers related to TID, the thread number, but I could not do it cleanly; so I'm thiking of asking for C$OMP SINGLE directives before reading each new other bunch of data.


So, my program would look like that :

----------------------------------
program rfomp1
implicit none
...
C$OMP MASTER
read most input data from three different files
C$OMP END MASTER
...
C$OMP PARALLEL DO PRIVATE(EID, TID) SHARED(elem, lina)
do EID=1, 300000 ! Loop on units of work
call fabmat(... ) ! main subroutine of each unit of work
...
remainder of the unit of work
...
enddo
C$OMP END PARALLEL DO
...
stop
end
----------------------------------
subroutine fabmat(... )
implicit none
...
C$OMP SINGLE
...
read input data pertinent to one single unit of work,
... coming from another single file, one for each unit of work.
...
C$OMP END SINGLE
...
...
return
----------------------------------

-A problem I had while reading new input data, from a fortran unit which number was closely related to the ThreadID, was that the input files, those of the successive units of work, were written by the output messages of the program.

-I suppose that when the inputs are made within "SINGLE" sections, the time gain made by executing several threads in parallel is lost sinc only one would be running.

-So I still would like to have several threads, each reading its own input file on a distinct fortran unit, that could be simply related to the ThreadID.

Does that seem clever to you, or how else could I do it ?
Thank's
David
dva2tlse
 
Posts: 18
Joined: Sun Sep 08, 2013 2:52 am
Location: Toulouse, France

Re: mess with input files

Postby ftinetti » Tue Sep 10, 2013 5:41 am

Hi David,

In general: parallel I/O is not usually implemented by the OS, so

-I suppose that when the inputs are made within "SINGLE" sections, the time gain made by executing several threads in parallel is lost sinc only one would be running.


is not (usually) true. This does not mean you should always have to do I/O in "SINGLE" sections, it could be a good idea each thread do its own I/O whenever it is possible, but for reasons other than performance.

I'm not able to comment on everything else, since I think I don't fully understand your specific issue.

Anyway, HTH,

Fernando.
ftinetti
 
Posts: 582
Joined: Wed Feb 10, 2010 2:44 pm

Re: mess with input files

Postby dva2tlse » Tue Sep 10, 2013 6:13 am

Hi Fernando,
I finally managed to use all my input files by assigning to each of them a fortran unit number equal to the thread_ID + 11, and the names of the files are taken from a character array, and referenced by file3(TID+1). In this way, each thread has its own input file and there is no more mix.
Regards,
David
dva2tlse
 
Posts: 18
Joined: Sun Sep 08, 2013 2:52 am
Location: Toulouse, France

Re: mess with input files

Postby dva2tlse » Wed Sep 11, 2013 1:22 pm

Now that my mess on input files is resolved, I have an issue on other matter :
As soon as I ask my program to run on more than two threads, it fails with a "Memory fault" message; please take note that it is a "Memory fault", and not a segfault which I could resolve by looking at the subscripts of the array terms that I use. It is a memory fault, and I suspect the machine not to have enough memory for the job I ask it to do. How can I be sure of that ?
I know how to use (at least with the help of the man pages) ps, or free, or top, or htop, but all of these are interactive procedures, and I need a trace of my process's use of the memory, in order to check if it approaches the machine limit. This one is about 64 MB, and each thread does need at least 326 flights * 31640 time steps * 1 fortran real number space.
I don't even know which space requires one fortran real variable, but if it were, say 5 bytes, the total would be something around 51 MBytes. So I absolutely need to monitor the memory space that my program uses, because it seems to be too big for the machine, if three or more threads were requiring such an amount; and thus the project could abort immediately, unless the boss bought another machine with plenty of memory.
David
dva2tlse
 
Posts: 18
Joined: Sun Sep 08, 2013 2:52 am
Location: Toulouse, France

Re: mess with input files

Postby dva2tlse » Wed Sep 11, 2013 10:48 pm

Hello again,
here it is rather early the morning, and when I woke up a few time ago, I realized that 64M was just about the amount of memory of my first computer in the late '70; so I checked again, and the real value is 64Gb of memory as you can see just here ;
$ free -m
total used free shared buffers cached
Mem: 64413 1104 63308 0 42 547
-/+ buffers/cache: 515 63898
Swap: 0 0 0

Hence there should be far enough memory for three or more threads, and I still would be interested in a way to monitor and record what my program uses.
David
EDIT Ok I made "call system('cat /proc/self/status | grep -e VmPeak -e VmSize', any_integer)" at several places of my fortran program, and that showed me that the four threads were not at all consuming all the memory of the machine, as I thought previously.
So it pushed me to run gdb, which showed me that the memory fault I encountered was really a segfault (which I did not believe) and I've been shown the exact instruction that was causing the crash.
So I will just have to check the indexes of the array terms that I use; as usual...
D.
dva2tlse
 
Posts: 18
Joined: Sun Sep 08, 2013 2:52 am
Location: Toulouse, France


Return to Using OpenMP

Who is online

Users browsing this forum: Google [Bot], Yahoo [Bot] and 3 guests