Re: Problem setting buffer size for gfortran ( v 11.2)
Hi Tobias, So i use now in my submission script the option you were quoting i.e > srun --export=ALL which is actually the default as stated in the slurm documentation. Setting the following environment to change the buffer size in both formatted/unformatted I/O 😎 export GFORTRAN_FORMATTED_BUFFER_SIZE=1 export GFORTRAN_UNFORMATTED_BUFFER_SIZE=1 export GFORTRAN_UNBUFFERED_ALL=n Does not work even on gcc/gfortran 11.2 If i set the same way some environment variables for openMPI, it is working. So this is not related to the forwarding of environment variables with srun. Are the variables set correct ? Best, Denis From: Tobias Burnus Sent: Thursday, February 24, 2022 9:20:26 AM To: Bertini, Denis Dr.; Harald Anlauf Cc: fortran@gcc.gnu.org Subject: Re: Problem setting buffer size for gfortran ( v 11.2) Hi Denis, On 24.02.22 09:15, Bertini, Denis Dr. via Fortran wrote: > I can try that but on our cluster we are bound to use >srun (slurm). > Do you know how to use the same functionality with srun > to set the environment variables. I didn't – but the manual did: srun --export=... See: https://slurm.schedmd.com/srun.html#OPT_export_1 Good luck, Tobias - Siemens Electronic Design Automation GmbH; Anschrift: Arnulfstraße 201, 80634 München; Gesellschaft mit beschränkter Haftung; Geschäftsführer: Thomas Heurung, Frank Thürauf; Sitz der Gesellschaft: München; Registergericht München, HRB 106955
Re: Re: Problem setting buffer size for gfortran ( v 11.2)
Hi Harald
I modified a little your test program to the following:
>>
program fio
integer :: stat
character(80) :: s
integer, parameter:: nx = 256
integer, parameter:: ny = 256
integer, parameter:: nz = 256
integer:: i, j, k, l
double precision :: data(nx,ny,nz)
call get_environment_variable ("GFORTRAN_FORMATTED_BUFFER_SIZE", s,
status=stat)
if (stat == 0) then
print *, "GFORTRAN_FORMATTED_BUFFER_SIZE=",trim(s)
else
print *, "GFORTRAN_FORMATTED_BUFFER_SIZE not set"
end if
! reset array
data(:,:,:) = -1
! fill dummy data
do i = 1, nx
do j = 1, ny
do k = 1, nz
data(i,j,k) = (i+j+k) * 0.1
end do
end do
end do
open(10,file="bigfile", status="new", action="write", form='unformatted',
access='stream')
write(10) data(1:nx, 1:ny, 1:nz)
close(10)
end program fio
Now i set the env. variable to the following values:
export GFORTRAN_UNFORMATTED_BUFFER_SIZE=1
export GFORTRAN_FORMATTED_BUFFER_SIZE=1
export GFORTRAN_UNBUFFERED_ALL=n
From the output log indeed i got the correct env. variable as i set:
GFORTRAN_FORMATTED_BUFFER_SIZE=1
But from strace output i got always
>>
write(3,
"\0\0\0\0\0\0\10@\0\0\0\340\314\314\10@\0\0\0\240\231\231\t@\0\0\0`ff\n@"...,
8192) = 8192
write(3,
"\0\0\0@33\v@\0\0\0\0\0\0\f@\0\0\0\340\314\314\f@\0\0\0\240\231\231\r@"...,
8192) = 8192
write(3, "\0\0\0`ff\16@\0\0\0@33\17@\0\0\0\0\0\0\20@\0\0\0`ff\20@"..., 8192) =
8192
write(3,
"\0\0\0\340\314\314\20@\0\0\0@33\21@\0\0\0\240\231\231\21@\0\0\0\0\0\0\22@"...,
8192) = 8192
write(3,
"\0\0\0`ff\22@\0\0\0\340\314\314\22@\0\0\0@33\23@\0\0\0\240\231\231\23@"...,
8192) = 8192
write(3, "\0\0\0\0\0\0\24@\0\0\0`ff\24@\0\0\0\340\314\314\24@\0\0\0@33\25@"...,
8192) = 8192
.
>>
Showing the fixed that still a 8kiB buffer size is effectively ude in I/O...
Best
Denis
From: Harald Anlauf
Sent: Friday, February 25, 2022 7:28:31 PM
To: Bertini, Denis Dr.
Cc: Tobias Burnus; fortran@gcc.gnu.org
Subject: Aw: Re: Problem setting buffer size for gfortran ( v 11.2)
Hi Denis,
I and verified it works here on our Lustre with gfortran-10.3.0.
You could use:
integer :: stat
character(80) :: s
call get_environment_variable ("GFORTRAN_FORMATTED_BUFFER_SIZE", s, status=stat)
if (stat == 0) then
print *, "GFORTRAN_FORMATTED_BUFFER_SIZE=",trim(s)
else
print *, "GFORTRAN_FORMATTED_BUFFER_SIZE not set"
end if
open(10,file="bigfile")
write(10,*) "data"
close(10)
end
to verify that the environment variable is set for a process/rank.
If that doesn't help, I'd suggest that it is possibly not a gfortran issue.
Cheers,
Harald
Gesendet: Freitag, 25. Februar 2022 um 18:28 Uhr
Von: "Bertini, Denis Dr."
An: "Tobias Burnus" , "Harald Anlauf"
Cc: "fortran@gcc.gnu.org"
Betreff: Re: Problem setting buffer size for gfortran ( v 11.2)
Hi Tobias,
So i use now in my submission script the option you were quoting i.e
> srun --export=ALL
which is actually the default as stated in the slurm documentation.
Setting the following environment to change the buffer size in both
formatted/unformatted I/O 😎
export GFORTRAN_FORMATTED_BUFFER_SIZE=1
export GFORTRAN_UNFORMATTED_BUFFER_SIZE=1
export GFORTRAN_UNBUFFERED_ALL=n
Does not work even on gcc/gfortran 11.2
If i set the same way some environment variables for openMPI,
it is working.
So this is not related to the forwarding of environment variables
with srun.
Are the variables set correct ?
Best,
Denis
From: Tobias Burnus
Sent: Thursday, February 24, 2022 9:20:26 AM
To: Bertini, Denis Dr.; Harald Anlauf
Cc: fortran@gcc.gnu.org
Subject: Re: Problem setting buffer size for gfortran ( v 11.2)
Hi Denis,
On 24.02.22 09:15, Bertini, Denis Dr. via Fortran wrote:
> I can try that but on our cluster we are bound to use
>srun (slurm).
> Do you know how to use the same functionality with srun
> to set the environment variables.
I didn't – but the manual did: srun --export=...
See: https://slurm.schedmd.com/srun.html#OPT_export_1
Good luck,
Tobias
-
Siemens Electronic Design Automation GmbH; Anschrift: Arnulfstraße 201, 80634
München; Gesellschaft mit beschränkter Haftung; Geschäftsführer: Thomas
Heurung, Frank Thürauf; Sitz der Gesellschaft: München; Registergericht
München, HRB 106955
Re: Problem setting buffer size for gfortran ( v 11.2)
Hi Denis, export GFORTRAN_UNFORMATTED_BUFFER_SIZE=1 export GFORTRAN_FORMATTED_BUFFER_SIZE=1 export GFORTRAN_UNBUFFERED_ALL=n From the output log indeed i got the correct env. variable as i set: GFORTRAN_FORMATTED_BUFFER_SIZE=1 But from strace output i got always write(3, "\0\0\0\0\0\0\10@\0\0\0\340\314\314\10@\0\0\0\240\231\231\t@\0\0\0`ff\n@"..., 8192) = 8192 write(3, "\0\0\0@33\v@\0\0\0\0\0\0\f@\0\0\0\340\314\314\f@\0\0\0\240\231\231\r@"..., 8192) = 8192 write(3, "\0\0\0`ff\16@\0\0\0@33\17@\0\0\0\0\0\0\20@\0\0\0`ff\20@"..., 8192) = 8192 write(3, "\0\0\0\340\314\314\20@\0\0\0@33\21@\0\0\0\240\231\231\21@\0\0\0\0\0\0\22@"..., 8192) = 8192 write(3, "\0\0\0`ff\22@\0\0\0\340\314\314\22@\0\0\0@33\23@\0\0\0\240\231\231\23@"..., 8192) = 8192 write(3, "\0\0\0\0\0\0\24@\0\0\0`ff\24@\0\0\0\340\314\314\24@\0\0\0@33\25@"..., 8192) = 8192 . Showing the fixed that still a 8kiB buffer size is effectively ude in I/O... did you ever think of trying other values than 1? On my Linux laptop, buffering is done in multiples of 2k. If I set GFORTRAN_UNFORMATTED_BUFFER_SIZE=14000, I get writes in sizes of 12288. Which is likely more efficient than asking for exactly 14000. Cheers, Harald
Re: Problem setting buffer size for gfortran ( v 11.2)
Hi Harald, Yes i tried other values. It seems that the value set in the foreseen environment variable is just ignored. And always a buffer size of 8kiB is used, which is the default values for formatted I/O harcoded in libgfortran. The only way to change this value is editing and recompiling the libgfortran. Best. Denis From: Harald Anlauf Sent: Friday, February 25, 2022 9:44:53 PM To: Bertini, Denis Dr. Cc: Tobias Burnus; fortran@gcc.gnu.org Subject: Re: Problem setting buffer size for gfortran ( v 11.2) Hi Denis, > export GFORTRAN_UNFORMATTED_BUFFER_SIZE=1 > export GFORTRAN_FORMATTED_BUFFER_SIZE=1 > export GFORTRAN_UNBUFFERED_ALL=n > > From the output log indeed i got the correct env. variable as i set: > > > GFORTRAN_FORMATTED_BUFFER_SIZE=1 > > > But from strace output i got always > > >>> > > write(3, > "\0\0\0\0\0\0\10@\0\0\0\340\314\314\10@\0\0\0\240\231\231\t@\0\0\0`ff\n@"..., > 8192) = 8192 > write(3, > "\0\0\0@33\v@\0\0\0\0\0\0\f@\0\0\0\340\314\314\f@\0\0\0\240\231\231\r@"..., > 8192) = 8192 > write(3, "\0\0\0`ff\16@\0\0\0@33\17@\0\0\0\0\0\0\20@\0\0\0`ff\20@"..., 8192) > = 8192 > write(3, > "\0\0\0\340\314\314\20@\0\0\0@33\21@\0\0\0\240\231\231\21@\0\0\0\0\0\0\22@"..., > 8192) = 8192 > write(3, > "\0\0\0`ff\22@\0\0\0\340\314\314\22@\0\0\0@33\23@\0\0\0\240\231\231\23@"..., > 8192) = 8192 > write(3, > "\0\0\0\0\0\0\24@\0\0\0`ff\24@\0\0\0\340\314\314\24@\0\0\0@33\25@"..., 8192) > = 8192 > . >>> > Showing the fixed that still a 8kiB buffer size is effectively ude in I/O... did you ever think of trying other values than 1? On my Linux laptop, buffering is done in multiples of 2k. If I set GFORTRAN_UNFORMATTED_BUFFER_SIZE=14000, I get writes in sizes of 12288. Which is likely more efficient than asking for exactly 14000. Cheers, Harald
