Hi Denis,
Am 23.02.22 um 21:28 schrieb Bertini, Denis Dr. via Fortran:
Hi Harald,
In attachment you will find a typical strace output when running with
gfortran8 on lustre filsystem.
unfortunately I do not see any attachment to your mail.
Note: GFORTRAN_FORMATTED_BUFFER_SIZE & friends was added in 10.1,
and possibly backported to some 9.y, but *not* to 8.x.
As i said for gfortran version 8.x, 10.x, 11.2 setting the runtime
env. variable to change buffer size did not work for me as you could
see in the strace output.
You could also see that "mmap" is used by the lustre client.
The optimal blocksize given back by the STAT() routine indeed
favor a 4 MiB buffer size.
It might make sense to show your issue to the vendor of your system.
Parallel file systems - not only Lustre - require special handling
not only to get optimum performance, but also correct behavior.
(E.g. the HDF5 library is known to hit bugs in our Lustre.)
Cheers,
Harald
Best
Denis
________________________________
From: Harald Anlauf <anl...@gmx.de>
Sent: Wednesday, February 23, 2022 7:26 PM
To: Bertini, Denis Dr.; fortran@gcc.gnu.org
Subject: Re: Problem setting buffer size for gfortran ( v 11.2)
Hi Denis,
Am 23.02.22 um 17:10 schrieb Bertini, Denis Dr. via Fortran:
Hi
Investigating some sporadical performance drop when using Fortran IO on our
Lustre
shared file system, we realized that the buffer size used as default by the gnu
fortran
compiler is 8 kiB only.
We wanted to change this default and according to the documentation, one has
just to set
the environment variables:
GFORTRAN_UNFORMATTED_BUFFER_SIZE
GFORTRAN_FORMATTED_BUFFER_SIZE=
GFORTRAN_UNBUFFERED_ALL=n
to increase the buffer size during runtime.
this works for me on gcc-12 development for the following
test program on Linux/x86:
open(10,file="bigfile")
write(10,*) "data"
close(10)
end
Compile, link, and run under strace with
GFORTRAN_FORMATTED_BUFFER_SIZE=4194304 strace ./a.out
[...]
openat(AT_FDCWD, "bigfile", O_RDWR|O_CREAT|O_CLOEXEC, 0666) = 3
fstat(3, {st_mode=S_IFREG|0644, st_size=6, ...}) = 0
mmap(NULL, 4198400, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1,
0) = 0x146b69b1c000
write(3, " data\n", 6) = 6
ftruncate(3, 6) = 0
munmap(0x146b69b1c000, 4198400) = 0
close(3) = 0
Unfortunately isetting these variables does not change the default buffer
size.
I tried with gfortran 8, 10, and the new 11.2.
Looking a the code, i found in
io/unix.c
that the default is set to the following value,
static const int FORMATTED_BUFFER_SIZE_DEFAULT = 8192;
static const int UNFORMATTED_BUFFER_SIZE_DEFAULT = 128*1024;
Of course, changing the default value and recompiling works and change
accordingly the buffer size
It should nevertheless be possible to change these values without recompiling.
Is there something one should be aware of when setting these variables?
You said you're using Lustre. I guess you use the default
blocksize (4MB). Does your Lustre client support mmap?
What does strace report?
I remember I once saw a system where we had to disable the use
of mmap in a library.
Cheers,
Harald
Best regards,
Denis
---------
Denis Bertini
Abteilung: CIT
Ort: SB3 2.265a
Tel: +49 6159 71 2240
Fax: +49 6159 71 2986
E-Mail: d.bert...@gsi.de
GSI Helmholtzzentrum für Schwerionenforschung GmbH
Planckstraße 1, 64291 Darmstadt, Germany, www.gsi.de
Commercial Register / Handelsregister: Amtsgericht Darmstadt, HRB 1528
Managing Directors / Geschäftsführung:
Professor Dr. Paolo Giubellino, Dr. Ulrich Breuer, Jörg Blaurock
Chairman of the GSI Supervisory Board / Vorsitzender des GSI-Aufsichtsrats:
Ministerialdirigent Dr. Volkmar Dietz
