On Sun, 17 Apr 2011 21:06:17 -0400, Robert Haas wrote:
On Sun, Apr 17, 2011 at 5:32 PM, Radosław Smogura
wrote:
Each process has simple "mirror" of shared descriptors.
I "believe" that modifications to buffer content may be only done
when holding
exclusive lock (with some simple exceptions) (
Robert Haas Monday 18 April 2011 03:06:17
> On Sun, Apr 17, 2011 at 5:32 PM, Radosław Smogura
>
> wrote:
> > Each process has simple "mirror" of shared descriptors.
> >
> > I "believe" that modifications to buffer content may be only done when
> > holding exclusive lock (with some simple except
On Sun, Apr 17, 2011 at 5:32 PM, Radosław Smogura
wrote:
> Each process has simple "mirror" of shared descriptors.
>
> I "believe" that modifications to buffer content may be only done when holding
> exclusive lock (with some simple exceptions) (+ MVCC), actually I saw only two
> things that can c
Robert Haas Sunday 17 April 2011 22:01:55
> On Sun, Apr 17, 2011 at 11:48 AM, Tom Lane wrote:
> > =?utf-8?q?Rados=C5=82aw_Smogura?= writes:
> >> Tom Lane Sunday 17 April 2011 01:35:45
> >>
> >>> ... Huh? Are you saying that you ask the kernel to map each individual
> >>> shared buffer separat
On Sunday 17 April 2011 22:09:24 Radosław Smogura wrote:
> I only know Phenom has 4096 entries I think and this covers 16MB of
> memory.
The numbers I cited where intels before and after core2.
Andres
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your
Andres Freund Sunday 17 April 2011 20:02:11
> On Sunday 17 April 2011 19:26:31 Radosław Smogura wrote:
> > Kernel merges vm_structs. So mappings are compacted. I'm not kernel
> > specialist, but skipping memory consumption, for not compacted mappings,
> > kernel uses btrees for dealing with TLB,
On Sun, Apr 17, 2011 at 11:48 AM, Tom Lane wrote:
> =?utf-8?q?Rados=C5=82aw_Smogura?= writes:
>> Tom Lane Sunday 17 April 2011 01:35:45
>>> ... Huh? Are you saying that you ask the kernel to map each individual
>>> shared buffer separately? I can't believe that's going to scale to
>>> realisti
Joshua Berkus writes:
> I, for one, am glad he did this work. We've discussed MMAP in the code off
> and on for years, but nobody wanted to do the work to test it. Now someone
> has, and we can decide whether it's worth pursuing based on the numbers.
Well, the troubling issue is that it's not
Robert,
> Actually, I'd walk through fire for a 10% performance improvement if
> it meant only a *risk* to stability.
Depends on the degree of risk. MMAP has the potential to introduce instability
into areas of the code which have been completely reliable for years. Adding
20 new coredump cas
On Sunday 17 April 2011 19:26:31 Radosław Smogura wrote:
> Kernel merges vm_structs. So mappings are compacted. I'm not kernel
> specialist, but skipping memory consumption, for not compacted mappings,
> kernel uses btrees for dealing with TLB, so it should not matter if there
> is 100 vm_struc
Tom Lane Sunday 17 April 2011 17:48:56
> =?utf-8?q?Rados=C5=82aw_Smogura?= writes:
> > Tom Lane Sunday 17 April 2011 01:35:45
> >
> >> ... Huh? Are you saying that you ask the kernel to map each individual
> >> shared buffer separately? I can't believe that's going to scale to
> >> realistic
Robert Haas writes:
> ... But please, everyone feel free to continue bashing me for
> wanting a readable patch with some understandable submission notes.
What he said. All this obsessing over whether the mmap patch could or
should have been run through pgindent is missing the big picture.
Namely
=?utf-8?q?Rados=C5=82aw_Smogura?= writes:
> Tom Lane Sunday 17 April 2011 01:35:45
>> ... Huh? Are you saying that you ask the kernel to map each individual
>> shared buffer separately? I can't believe that's going to scale to
>> realistic applications.
> No, I do
> mrempa(mmap_buff_A, MAP_FI
Tom Lane Sunday 17 April 2011 01:35:45
> =?utf-8?q?Rados=C5=82aw_Smogura?= writes:
> > No, no, no :) I wanted to do this, but from above reason I skipped it. I
> > swap VM pages, I do remap, in place where the shared buffer was I put
> > mmaped page, and in place where mmaped page was I put share
On Sat, Apr 16, 2011 at 9:31 PM, Robert Haas wrote:
> On Sat, Apr 16, 2011 at 2:33 PM, Joshua Berkus wrote:
>> Well, given the risks to durability and stability associated with using
>> MMAP, I doubt anyone would even consider it for a 10% throughput
>> improvement. However, I don't think the
On Sat, Apr 16, 2011 at 2:33 PM, Joshua Berkus wrote:
> Well, given the risks to durability and stability associated with using MMAP,
> I doubt anyone would even consider it for a 10% throughput improvement.
> However, I don't think the test you used demonstrates the best case for MMAP
> as a
=?utf-8?q?Rados=C5=82aw_Smogura?= writes:
> No, no, no :) I wanted to do this, but from above reason I skipped it. I swap
> VM pages, I do remap, in place where the shared buffer was I put mmaped page,
> and in place where mmaped page was I put shared page (in certain cases, which
> should be o
Radosław Smogura wrote:
Yes, but, hmm... in Netbeans I had really long gaps (probably 8
spaces, from tabs), so deeper "ifs", comments at the and of variables,
went of out my screen. I really wanted to not format this, but
sometimes I needed.
The guide at
http://www.open-source-editor.com/edi
Tom Lane Saturday 16 April 2011 17:02:32
> Greg Stark writes:
> > What he did, I gather, is treat the mmapped buffers as a read-only
> > copy of the data. To actually make any modifications he copies it into
> > shared buffers and treats them like normal. When the buffers get
> > flushed from mem
On Fri, Apr 15, 2011 at 3:12 PM, Radosław Smogura
wrote:
> On Fri, 15 Apr 2011 14:33:37 +0300, Heikki Linnakangas wrote:
>> The patch is quite hard to read because of random whitespace changes
>> and other stylistic issues, but I have a couple of high-level
>> questions on the design:
>
> Yes, but
Radoslaw,
> I think 10% is quite good, as my stand-alone test of mmap vs. read
> shown that
> speed up of copying 100MB data to mem may be from ~20ms to ~100ms
> (depends on
> destination address). Of course deeper, system test simulating real
> usage will
> say more. In any case after good deals
On Fri, 2011-04-15 at 12:32 +0200, Radosław Smogura wrote:
> I didn't included this, as diff, because of ~150kb size (mainly
> configure scripts, which are included in SVC). Due to this, You may
> download it from
> http://softperience.eu/downloads/pg_mmap_20110415.diff.bz2 (Legal:
> Work
>
Greg Stark writes:
> What he did, I gather, is treat the mmapped buffers as a read-only
> copy of the data. To actually make any modifications he copies it into
> shared buffers and treats them like normal. When the buffers get
> flushed from memory they get written and then the pointers get
> rep
On Sat, Apr 16, 2011 at 8:48 AM, Greg Smith wrote:
> Joshua Berkus wrote:
>>
>> Guys, can we *please* focus on the patch for now, rather than the
>> formatting, which is fixable with sed?
>>
>
> Never, and that's not true. Heikki was being nice; I wouldn't have even
> slogged through it long enou
Greg Stark Saturday 16 April 2011 13:00:19
> On Sat, Apr 16, 2011 at 7:24 AM, Robert Haas wrote:
> > The OP says that this patch maintains the WAL-before-data rule without
> > any explanation of how it accomplishes that seemingly quite amazing
> > feat. I assume I'm going to have to read this pa
On Sat, Apr 16, 2011 at 7:24 AM, Robert Haas wrote:
> The OP says that this patch maintains the WAL-before-data rule without any
> explanation of how it accomplishes that seemingly quite amazing feat. I
> assume I'm going to have to read this patch at some point to refute this
> assertion, and
Robert Haas wrote:
The OP says that this patch maintains the WAL-before-data rule without any
explanation of how it accomplishes that seemingly quite amazing feat. I assume
I'm going to have to read this patch at some point to refute this assertion,
and I think that sucks.
I don't think you
Tom Lane wrote:
* On the other side of the coin, I have seen many a patch that was
written to minimize the length of the diff to the detriment of
readability or maintainability of the resulting code, and that's *not*
a good tradeoff.
Sure. that's possible. But based on the reviews I've done
Greg Smith writes:
> Reading one's own diff and reflecting on what you've changed is one of
> the extremely underappreciated practices of good open-source software
> development. Minimizing the size of that diff is perhaps the most
> important thing someone can do in order to make their change
On Apr 16, 2011, at 1:48 AM, Greg Smith wrote:
> P.S. You know what else I feel should earn an automatic rejection without any
> reviewer even looking at the code?
Greg is absolutely right. And to the two he listed, let me add another of my
own gripes: failing to provide submission notes that e
Joshua Berkus wrote:
Guys, can we *please* focus on the patch for now, rather than the formatting,
which is fixable with sed?
Never, and that's not true. Heikki was being nice; I wouldn't have even
slogged through it long enough to ask the questions he did before
kicking it back as unusa
Joshua Berkus Friday 15 April 2011 18:55:04
> Radoslaw,
>
> 10% improvement isn't very impressive from a switch to mmap. What workload
> did you test with? What I'd really like to see is testing with databases
> which are 50%, 90% and 200% the size of RAM ... that's where I'd expect
> the great
Radoslaw,
10% improvement isn't very impressive from a switch to mmap. What workload did
you test with? What I'd really like to see is testing with databases which are
50%, 90% and 200% the size of RAM ... that's where I'd expect the greatest gain
from limiting copying.
> Netbeans is possib
On 04/15/2011 08:12 AM, Radosław Smogura wrote:
The patch is quite hard to read because of random whitespace changes
and other stylistic issues, but I have a couple of high-level
questions on the design:
Yes, but, hmm... in Netbeans I had really long gaps (probably 8
spaces, from tabs), so d
On Fri, 15 Apr 2011 14:33:37 +0300, Heikki Linnakangas wrote:
On 15.04.2011 13:32, Radosław Smogura wrote:
If I may, I want to share some concept to use mmap in PG. It's far,
far
away from perfect, but it's keeps WAL before data. As well I crated
table, with index, inserted few values, and I do
On 15.04.2011 13:32, Radosław Smogura wrote:
If I may, I want to share some concept to use mmap in PG. It's far, far
away from perfect, but it's keeps WAL before data. As well I crated
table, with index, inserted few values, and I done vacuum full on this
table. Db inits welcome from orginal sour
Hello,
If I may, I want to share some concept to use mmap in PG. It's far, far
away from perfect, but it's keeps WAL before data. As well I crated
table, with index, inserted few values, and I done vacuum full on this
table. Db inits welcome from orginal sources.
Performance of read (if back
37 matches
Mail list logo
