> > > Another way of looking at it is memory mapped files. This probably most > > closely resembles unix shared memory and is the de facto standard way > > for interprocess memory block sharing. Sadly, performance will suffer > > because you have to rely on the virtual memory system (think: writing to > > files) to do a lot of stupid stuff you don't necessarily want or need. > > To the contrary, for the majority of the shared memory usage of > postgres, which is cached file data, the virtual memory system is doing > exactly what you want it to: managing the movement of data between > memory and disk, and caching the more frequently accessed data to reduce > the chances you will actually need to access the disk for it.
Yes. Generally, I was trying to point out the disadvantages of memory mapped files compared to shared memory. In windows, there is no direct equivalent so shared memory. MMFs are very similar in usage. I suspect they might not perform quite as well as the shared memory functions. For example, if used in place of shared memory to cache static file data, you are maintaining: 1. the file itself, 2. the file cache handled by the os. 3. the MMF memory side cache (following a page fault). 4. the virtual memory space set aside for the os to swap it out should the os need more memory. MMFs are efficient when memory allocations are relatively static: they work especially well with a freestore memory allocation system (this minimizes movement inside the virtual memory pagefile). For example, the MMF is allocated at the startup of the backend and doled out to processes through an internal 'as needed' basis. This is equivalent in function to memory allocations using the VirtualAlloc() family except its good for IPC. (IMHO, it will still run slower). If memory allocations are frequent and dynamic, you start to run into problems with fragmentation of the pagefile and such problems. This is very undesirable. Also, if memory allocations are large, you could potentially run into the worst possible scenario: your file cache system is competing with the virtual memory system. This will cause the server to thrash. One workaround for that is to set up the files for sequential access: this minimizes os caching of files. This also more or less removes 'double dipping' into the memory system to cache your static file data. The down side is that the work of maintaining an intelligent file cache has been offloaded from the OS to you, the programmer. I am not experienced enough with the postgres memory allocation system to say how well this would work for PostgreSQL. > > For shared memory used only for IPC, typically a VM system treats it no > differently from any other non-shared memory, so if it's doing something > you "don't want or need" (a proposition I quite heartily disagree with), > it's going to be doing that very every piece of memory your application > allocates and uses, shared or not. > > > The OS has to guarantee that the memory can be swapped out to file at > > any time and therefore mirrors the pagefile to the allocated memory > > blocks. > > The OS does not need to write the pagefile. On modern Unix systems that > are not allowing overcommit, the space will be allocated but never > written unless there's a need to free up some physical memory, and the > pages in question are used infrequently enough that the system decides > that they are good candidates to be paged out. I would imagine that > Windows does the same. In windows, things are backwards: the space is allocated in virtual memory *first* (i.e. the page file), then following a page fault it gets swapped into memory. The overhead I spoke of was related to the fact the windows always has to ensure space exists in the page file (or some user defined file) to swap the file back out. IMHO, *nix has a much superior approach to IPC in this context. It's much simpler and very straightforward. It also exlains why in windows, most server apps are multi threaded, not multi process. I agree with you on most salient points. The question is: are MMFs the proper analog of SHHMEM on native port of postgres? My answer to that question is: it is by no means certain, but what else is there to use? Merlin > > cjs > -- > Curt Sampson <[EMAIL PROTECTED]> +81 90 7737 2974 http://www.netbsd.org > Don't you know, in this new Dark Age, we're all light. --XTC ---------------------------(end of broadcast)--------------------------- TIP 6: Have you searched our list archives? http://archives.postgresql.org
