On 20 September 2017 at 16:55, Thomas Munro <thomas.mu...@enterprisedb.com> wrote:
> On Wed, Sep 20, 2017 at 6:14 PM, Gaddam Sai Ram > <gaddamsaira...@zohocorp.com> wrote: > > Thank you very much! That fixed my issue! :) > > I was in an assumption that pinning the area will increase its lifetime > but > > yeah after taking memory context into consideration its working fine! > > So far the success rate in confusing people who first try to make > long-lived DSA areas and DSM segments is 100%. Basically, this is all > designed to ensure automatic cleanup of resources in short-lived > scopes. > 90% ;) I got it working with no significant issues for a long lived segment used to store a pool of shm_mq pairs used for a sort of "connection listener" bgworker. Though I only used DSM+ToC, not DSA. But TBH that may well be luck, as I tend to routinely use memory contexts scoped to the operational lifetime of a subsystem, making most problems like this just vanish without my realising they were there in the first place. Usually. I pretty much shamelessly cribbed from test_shm_mq for the ToC stuff though. It's simple enough when you read it in use, but I'd be lucky to do it without an example. I had lots more problems with shm_mq than DSM. shm_mq is very obviously designed for short-lived scopes, and falls down badly if you have a pool of queues you want to re-use after the peer detaches. You have to track "in use" flags separately to the shm_mq's own, because it doesn't clear its stored PGPROC entries for receiver/sender on detach. Once you know neither sender nor receiver is still attached, you can memset() the area and create a new queue in it. You can't just reset the queue for a new peer, and have to do quite a dance to make sure it's safe detach from, overwrite, re-create and re-attach to. > Good luck for your graph project. I think you're going to have to > expend a lot of energy trying to avoid memory leaks if your DSA lives > as long as the database cluster, since error paths won't automatically > free any memory you allocated in it. Yeah, that's going to be hard. You might land up having lots and lots of little DSM segments. > There is a kind of garbage collection for palloc'd memory and > also for other resources like file handles, but if you're using a big > long lived DSA area you have nothing like that. We need, IMO, a DSA-backed heirachical MemoryContext system. We can't use the exact MemoryContext API as-is due to the need for far pointers though :( -- Craig Ringer http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services
