On Thu, Feb 21, 2019 at 8:28 AM Arthur Zakirov <a.zaki...@postgrespro.ru> wrote: > Your approach looks simpler. It is necessary just to periodically scan > dictionaries' cache hash table and not call dsm_pin_segment() when a DSM > segment initialized. It also means that a dictionary is loaded into DSM > only while there is a backend which attached the dictionary's DSM.
Right. I think that having a central facility that tries to decide whether or not a dictionary should be kept in shared memory or not, e.g. based on a cache size parameter, isn't likely to work well. The problem is that if we make a decision that a dictionary should be evicted because it's causing us to exceed the cache size threshold, then we have no way to implement that decision. We can't force other backends to remove the mapping immediately, nor can we really bound the time before they respond to a request to unmap it. They might be in the middle of using it. So I think it's better to have each backend locally make a decision about when that particular backend no longer needs the dictionary, and then let the system automatically clean up the ones that are needed by nobody. Perhaps a better approach still would be to do what Andres proposed back in March: #> Is there any chance we can instead can convert dictionaries into a form #> we can just mmap() into memory? That'd scale a lot higher and more #> dynamicallly? The current approach inherently involves double-buffering: you've got the filesystem cache containing the data read from disk, and then the DSM containing the converted form of the data. Having something that you could just mmap() would avoid that, plus it would become a lot less critical to keep the mappings around. You could probably just have individual queries mmap() it for as long as they need it and then tear out the mapping when they finish executing; keeping the mappings across queries likely wouldn't be too important in this case. The downside is that you'd probably need to teach resowner.c about mappings created via mmap() so that you don't leak mappings on an abort, but that's probably not a crazy difficult problem. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company
