On Wed, May 7, 2025 at 11:04 AM Jack Ng <jack...@huawei.com> wrote:
> > all the possible scenarios. But now I'm reworking it along the lines
> suggested
> > by Thomas, and will address those as well. Thanks!
>
> Thanks for the info, Dmitry.
> Just want to confirm my understanding of Thomas' suggestion and your
> discussions... I think the simpler and more portable solution goes
> something like the following?
>
> * For each BP resource segment (main, desc, buffers, etc):
> 1. create an anonymous file as backing
> 2. mmap a large reserved shared memory area with PROTO_READ/WRITE +
> MAP_NORESERVE using the anon fd
> 3. use ftruncate to back the in-use region (and maybe posix_fallocate
> too to avoid SIGBUS on alloc failure during first-touch), but no need to
> create a memory mapping for it
> 4. also no need to create a separate mapping for the reserved region
> (already covered by the mapping created in 2.)
>
> |-- Memory mapping (MAP_NORESERVE) for BUFFER --|
> |-- In-use region --|----- Reserved region -----|
>
> * During resize, simply calculate the new size and call ftruncate on each
> segment to adjust memory accordingly, no need to mmap/munmap or modify any
> memory mapping.
>
>
That's same as my understanding.
> I tried this approach with a test program (with huge pages), and both
> expand and shrink seem to work as expected --for shrink, the memory is
> freed right after the resize ftruncate.
>
> I thought I had shared a test program upthread, but I don't find it now.
Attached here. Can you please share your test program?
There are concerns around portability of this approach, though.
--
Best Wishes,
Ashutosh Bapat
#define _GNU_SOURCE 1 /* See feature_test_macros(7) */
#include <errno.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <sys/mman.h>
#include <unistd.h>
#include <stdbool.h>
#define MBSIZE(x) ((x) * 1024 * 1024)
int
main(int argc, char **argv)
{
int flags = MAP_NORESERVE;
void *memaddr;
pid_t pid = getpid();
char *localmem;
int fd = memfd_create("mmap_fd_exp", 0);
size_t size = MBSIZE(300);
localmem = malloc(size);
memset(localmem, 1, size);
if (fd < 0)
{
printf("memfd_create failed with errno %d\n", errno);
exit(__LINE__);
}
if (ftruncate(fd, size) < 0)
{
printf("ftruncate failed with errno %d on fd = %d and size = %ld\n", errno, fd, size);
exit(__LINE__);
}
memaddr = mmap(NULL, size, PROT_WRITE | PROT_READ, MAP_SHARED | MAP_NORESERVE, fd, 0);
if (memaddr == MAP_FAILED)
{
printf("mmap failed with error %m\n");
exit(__LINE__);
}
memset(memaddr, 1, size);
if (memcmp(memaddr, localmem, size) != 0)
{
printf("mmap memory and local memory are not equal upto size = %ld\n", size);
exit(__LINE__);
}
/* causes a segmentation fault: memset(memaddr, 0, maxsize + 1); */
size = MBSIZE(100);
if (ftruncate(fd, size) < 0)
{
printf("ftruncate failed with errno %d on fd = %d and size = %ld\n", errno, fd, size);
exit(__LINE__);
}
memset(memaddr, 1, size);
if (memcmp(memaddr, localmem, size) != 0)
{
printf("mmap memory and local memory are not equal upto size = %ld\n", size);
exit(__LINE__);
}
/* causes a segmentation fault: memset(memaddr, 1, MBSIZE(200)); */
size = MBSIZE(200);
if (ftruncate(fd, size) < 0)
{
printf("ftruncate failed with errno %d on fd = %d and size = %ld\n", errno, fd, size);
exit(__LINE__);
}
memset(memaddr, 1, size);
if (memcmp(memaddr, localmem, size) != 0)
{
printf("mmap memory and local memory are not equal upto size = %ld\n", size);
exit(__LINE__);
}
}
