Re: [HACKERS] Experimental dynamic memory allocation of postgresql shared memory

2016-06-19 Thread Michael Paquier
On Mon, Jun 20, 2016 at 12:40 PM, Craig Ringer wrote: > On 18 June 2016 at 02:42, Robert Haas wrote: >> >> On Fri, Jun 17, 2016 at 2:23 PM, Aleksey Demakov >> wrote: >> > Essentially this is pessimizing for the lowest common denominator >> > among OSes. >> >> I totally agree. That's how we make

Re: [HACKERS] Experimental dynamic memory allocation of postgresql shared memory

2016-06-19 Thread Craig Ringer
On 18 June 2016 at 02:42, Robert Haas wrote: > On Fri, Jun 17, 2016 at 2:23 PM, Aleksey Demakov > wrote: > > Essentially this is pessimizing for the lowest common denominator > > among OSes. > > I totally agree. That's how we make the server portable. > > > Having a contiguous address space mak

Re: [HACKERS] Experimental dynamic memory allocation of postgresql shared memory

2016-06-17 Thread Aleksey Demakov
On Sat, Jun 18, 2016 at 3:43 AM, Tom Lane wrote: > DSM already exists, and for many purposes its lack of a > within-a-shmem-segment dynamic allocator is irrelevant; the same purpose > is served (with more speed, more reliability, and less code) by releasing > the whole DSM segment when no longer n

Re: [HACKERS] Experimental dynamic memory allocation of postgresql shared memory

2016-06-17 Thread Tom Lane
Aleksey Demakov writes: > On Sat, Jun 18, 2016 at 12:45 AM, Tom Lane wrote: >> You're right, but that doesn't mean that the community is going to take >> much interest in an unportable replacement for code that already exists. > Excuse me, what code already exists? As far as I understand, we > c

Re: [HACKERS] Experimental dynamic memory allocation of postgresql shared memory

2016-06-17 Thread Aleksey Demakov
On Sat, Jun 18, 2016 at 12:45 AM, Tom Lane wrote: > Aleksey Demakov writes: >> On Fri, Jun 17, 2016 at 10:54 PM, Robert Haas wrote: >>> In my opinion, that's not going to fly. If I thought otherwise, I >>> would not have developed the DSM facility in the first place. > >> Essentially this is pe

Re: [HACKERS] Experimental dynamic memory allocation of postgresql shared memory

2016-06-17 Thread Tom Lane
Aleksey Demakov writes: > On Fri, Jun 17, 2016 at 10:54 PM, Robert Haas wrote: >> On Fri, Jun 17, 2016 at 12:34 PM, Aleksey Demakov wrote: >>> I believe it would be perfectly okay to allocate huge amount of address >>> space with mmap on startup. If the pages are not touched, the OS VM >>> subs

Re: [HACKERS] Experimental dynamic memory allocation of postgresql shared memory

2016-06-17 Thread Robert Haas
On Fri, Jun 17, 2016 at 2:23 PM, Aleksey Demakov wrote: > Essentially this is pessimizing for the lowest common denominator > among OSes. I totally agree. That's how we make the server portable. > Having a contiguous address space makes things so > much simpler that considering this case, IMHO,

Re: [HACKERS] Experimental dynamic memory allocation of postgresql shared memory

2016-06-17 Thread David G. Johnston
On Fri, Jun 17, 2016 at 2:23 PM, Aleksey Demakov wrote: > On Fri, Jun 17, 2016 at 10:54 PM, Robert Haas > wrote: > > On Fri, Jun 17, 2016 at 12:34 PM, Aleksey Demakov > wrote: > >>> I expect that to be useful for parallel query and anything else where > >>> processes need to share variable-size

Re: [HACKERS] Experimental dynamic memory allocation of postgresql shared memory

2016-06-17 Thread Aleksey Demakov
On Sat, Jun 18, 2016 at 12:31 AM, Andres Freund wrote: > On 2016-06-18 00:23:14 +0600, Aleksey Demakov wrote: >> Finally, it's possible to repeatedly mmap >> and munmap on portions of a contiguous address space providing >> a given addr argument for both of them. The last option might, of >> cours

Re: [HACKERS] Experimental dynamic memory allocation of postgresql shared memory

2016-06-17 Thread Aleksey Demakov
Sorry for unclear language. Late Friday evening in my place is to blame. On Sat, Jun 18, 2016 at 12:23 AM, Aleksey Demakov wrote: > On Fri, Jun 17, 2016 at 10:54 PM, Robert Haas wrote: >> On Fri, Jun 17, 2016 at 12:34 PM, Aleksey Demakov wrote: I expect that to be useful for parallel query

Re: [HACKERS] Experimental dynamic memory allocation of postgresql shared memory

2016-06-17 Thread Andres Freund
On 2016-06-18 00:23:14 +0600, Aleksey Demakov wrote: > Finally, it's possible to repeatedly mmap > and munmap on portions of a contiguous address space providing > a given addr argument for both of them. The last option might, of > course, is susceptible to hijacking this portion of the address by

Re: [HACKERS] Experimental dynamic memory allocation of postgresql shared memory

2016-06-17 Thread Aleksey Demakov
On Fri, Jun 17, 2016 at 10:54 PM, Robert Haas wrote: > On Fri, Jun 17, 2016 at 12:34 PM, Aleksey Demakov wrote: >>> I expect that to be useful for parallel query and anything else where >>> processes need to share variable-size data. However, that's different >>> from this because ours can grown

Re: [HACKERS] Experimental dynamic memory allocation of postgresql shared memory

2016-06-17 Thread Robert Haas
On Fri, Jun 17, 2016 at 12:34 PM, Aleksey Demakov wrote: >> I expect that to be useful for parallel query and anything else where >> processes need to share variable-size data. However, that's different >> from this because ours can grown to arbitrary size and shrink again by >> allocating and fr

Re: [HACKERS] Experimental dynamic memory allocation of postgresql shared memory

2016-06-17 Thread Aleksey Demakov
On Fri, Jun 17, 2016 at 10:18 PM, Robert Haas wrote: > On Fri, Jun 17, 2016 at 11:30 AM, Tom Lane wrote: > But I'm a bit confused about where it gets the bytes it wants to > manage. There's no call to dsm_create() or ShmemAlloc() anywhere in > the code, at least not that I could find quickly. T

Re: [HACKERS] Experimental dynamic memory allocation of postgresql shared memory

2016-06-17 Thread Aleksey Demakov
On Fri, Jun 17, 2016 at 9:30 PM, Tom Lane wrote: > Aleksey Demakov writes: >> I have some very experimental code to enable dynamic memory allocation >> of shared memory for postgresql backend processes. > > Um ... what's this do that the existing DSM stuff doesn't do? > It operates over a single

Re: [HACKERS] Experimental dynamic memory allocation of postgresql shared memory

2016-06-17 Thread Robert Haas
On Fri, Jun 17, 2016 at 11:30 AM, Tom Lane wrote: > Aleksey Demakov writes: >> I have some very experimental code to enable dynamic memory allocation >> of shared memory for postgresql backend processes. > > Um ... what's this do that the existing DSM stuff doesn't do? It seems to be a full-fled

Re: [HACKERS] Experimental dynamic memory allocation of postgresql shared memory

2016-06-17 Thread Tom Lane
Aleksey Demakov writes: > I have some very experimental code to enable dynamic memory allocation > of shared memory for postgresql backend processes. Um ... what's this do that the existing DSM stuff doesn't do? regards, tom lane -- Sent via pgsql-hackers mailing list