Josh Berkus wrote:
>
> > Was it really all that bad? IIRC we replaced ARC with the current clock
> > sweep due to patent concerns. (Maybe there were performance concerns as
> > well, I don't remember).
>
> Yeah, that was why the patent was frustrating. Performance was poor and
> we were planni
On 03/24/2011 03:36 PM, Jim Nasby wrote:
On Mar 23, 2011, at 5:12 PM, Tom Lane wrote:
Robert Haas writes:
It looks like the only way anything can ever get put on the free list
right now is if a relation or database is dropped. That doesn't seem
too good.
Why not? AIUI the
On Fri, Mar 25, 2011 at 8:07 AM, Gurjeet Singh wrote:
> On Tue, Mar 22, 2011 at 3:53 PM, Robert Haas wrote:
>>
>> On Tue, Mar 22, 2011 at 11:24 AM, Jeff Janes wrote:
>> > On Fri, Mar 18, 2011 at 9:19 AM, Robert Haas
>> > wrote:
>> >>
>> >> A related area that could use some looking at is why pe
On Mar 25, 2011, at 11:58 AM, Jim Nasby wrote:
> Related to that... after talking to Greg Smith at PGEast last night, he felt
> it would be very valuable just to profile how much time is being spent
> waiting/holding the freelist lock in a real environment. I'm going to see if
> we can do that
On Thu, Mar 24, 2011 at 7:51 PM, Greg Stark wrote:
> On Thu, Mar 24, 2011 at 11:33 PM, Jeff Janes wrote:
>> I tried under the circumstances I thought were mostly likely to show a
>> time difference, and I was unable to detect a reliable difference in
>> timing between free list and clock sweep.
>
On Mar 25, 2011, at 10:07 AM, Gurjeet Singh wrote:
> On Tue, Mar 22, 2011 at 3:53 PM, Robert Haas wrote:
> On Tue, Mar 22, 2011 at 11:24 AM, Jeff Janes wrote:
> > On Fri, Mar 18, 2011 at 9:19 AM, Robert Haas wrote:
> >> On Fri, Mar 18, 2011 at 11:14 AM, Kevin Grittner
> >> wrote:
> >>> Maybe th
On Tue, Mar 22, 2011 at 3:53 PM, Robert Haas wrote:
> On Tue, Mar 22, 2011 at 11:24 AM, Jeff Janes wrote:
> > On Fri, Mar 18, 2011 at 9:19 AM, Robert Haas
> wrote:
> >> On Fri, Mar 18, 2011 at 11:14 AM, Kevin Grittner
> >> wrote:
> >>> Maybe the thing to focus on first is the oft-discussed "be
On Thu, Mar 24, 2011 at 11:33 PM, Jeff Janes wrote:
> I tried under the circumstances I thought were mostly likely to show a
> time difference, and I was unable to detect a reliable difference in
> timing between free list and clock sweep.
It strikes me that it shouldn't be terribly hard to add a
On Thu, Mar 24, 2011 at 12:36 PM, Jim Nasby wrote:
> On Mar 23, 2011, at 5:12 PM, Tom Lane wrote:
>> Robert Haas writes:
>>> It looks like the only way anything can ever get put on the free list
>>> right now is if a relation or database is dropped. That doesn't seem
>>> too good.
>>
>> Why not?
Robert Haas Thursday 24 March 2011 22:41:19
> On Thu, Mar 24, 2011 at 5:34 PM, Greg Stark wrote:
> > On Thu, Mar 24, 2011 at 8:59 PM, Robert Haas
wrote:
> >> It seems at least plausible that buffer allocation could be
> >> significantly faster if it need only pop the head of a list, rather
> >>
On Thu, Mar 24, 2011 at 5:34 PM, Greg Stark wrote:
> On Thu, Mar 24, 2011 at 8:59 PM, Robert Haas wrote:
>> It seems at least plausible that buffer allocation could be
>> significantly faster if it need only pop the head of a list, rather
>> than scanning until it finds a suitable candidate. Mov
On Thu, Mar 24, 2011 at 8:59 PM, Robert Haas wrote:
> It seems at least plausible that buffer allocation could be
> significantly faster if it need only pop the head of a list, rather
> than scanning until it finds a suitable candidate. Moving as much
> buffer allocation work as possible into the
On Wed, Mar 23, 2011 at 6:12 PM, Tom Lane wrote:
> Robert Haas writes:
>> It looks like the only way anything can ever get put on the free list
>> right now is if a relation or database is dropped. That doesn't seem
>> too good.
>
> Why not? AIUI the free list is only for buffers that are total
Jim Nasby Thursday 24 March 2011 20:36:48
> On Mar 23, 2011, at 5:12 PM, Tom Lane wrote:
> > Robert Haas writes:
> >> It looks like the only way anything can ever get put on the free list
> >> right now is if a relation or database is dropped. That doesn't seem
> >> too good.
> >
> > Why not?
On Mar 23, 2011, at 5:12 PM, Tom Lane wrote:
> Robert Haas writes:
>> It looks like the only way anything can ever get put on the free list
>> right now is if a relation or database is dropped. That doesn't seem
>> too good.
>
> Why not? AIUI the free list is only for buffers that are totally d
Robert Haas writes:
> It looks like the only way anything can ever get put on the free list
> right now is if a relation or database is dropped. That doesn't seem
> too good.
Why not? AIUI the free list is only for buffers that are totally dead,
ie contain no info that's possibly of interest to
Greg Stark Wednesday 23 March 2011 21:30:04
> On Wed, Mar 23, 2011 at 8:00 PM, Robert Haas wrote:
> > It looks like the only way anything can ever get put on the free list
> > right now is if a relation or database is dropped. That doesn't seem
> > too good. I wonder if the background writer sh
On Wed, Mar 23, 2011 at 8:00 PM, Robert Haas wrote:
> It looks like the only way anything can ever get put on the free list
> right now is if a relation or database is dropped. That doesn't seem
> too good. I wonder if the background writer shouldn't be trying to
> maintain the free list. That
On Wed, Mar 23, 2011 at 1:53 PM, Jim Nasby wrote:
> When we started using 192G servers we tried switching our largest OLTP
> database (would have been about 1.2TB at the time) from 8GB shared buffers to
> 28GB. Performance went down enough to notice; I don't have any solid metrics,
> but I'd ba
On Mar 22, 2011, at 2:53 PM, Robert Haas wrote:
> On Tue, Mar 22, 2011 at 11:24 AM, Jeff Janes wrote:
>> On Fri, Mar 18, 2011 at 9:19 AM, Robert Haas wrote:
>>> On Fri, Mar 18, 2011 at 11:14 AM, Kevin Grittner
>>> wrote:
Maybe the thing to focus on first is the oft-discussed "benchmark
Merlin Moncure Tuesday 22 March 2011 23:06:02
> On Tue, Mar 22, 2011 at 4:28 PM, Radosław Smogura
>
> wrote:
> > Merlin Moncure Monday 21 March 2011 20:58:16
> >
> >> On Mon, Mar 21, 2011 at 2:08 PM, Greg Stark wrote:
> >> > On Mon, Mar 21, 2011 at 3:54 PM, Merlin Moncure
> >
> > wrote:
> >
Merlin Moncure Monday 21 March 2011 20:58:16
> On Mon, Mar 21, 2011 at 2:08 PM, Greg Stark wrote:
> > On Mon, Mar 21, 2011 at 3:54 PM, Merlin Moncure
wrote:
> >> Can't you make just one large mapping and lock it in 8k regions? I
> >> thought the problem with mmap was not being able to detect ot
On Tue, Mar 22, 2011 at 4:28 PM, Radosław Smogura
wrote:
> Merlin Moncure Monday 21 March 2011 20:58:16
>> On Mon, Mar 21, 2011 at 2:08 PM, Greg Stark wrote:
>> > On Mon, Mar 21, 2011 at 3:54 PM, Merlin Moncure
> wrote:
>> >> Can't you make just one large mapping and lock it in 8k regions? I
>>
Merlin Moncure Monday 21 March 2011 20:58:16
> On Mon, Mar 21, 2011 at 2:08 PM, Greg Stark wrote:
> > On Mon, Mar 21, 2011 at 3:54 PM, Merlin Moncure
wrote:
> >> Can't you make just one large mapping and lock it in 8k regions? I
> >> thought the problem with mmap was not being able to detect ot
Radek,
I have implemented initial concept of 2nd level cache. Idea is to keep some
segments of shared memory for special buffers (e.g. indices) to prevent
overwrite those by other operations. I added those functionality to nbtree
index scan.
The problem with any "special" buffering of database
On Tue, 2011-03-22 at 15:53 -0400, Robert Haas wrote:
>
> To be honest, I'm mostly just reporting what I've heard Greg Smith say
> on this topic. I don't have any machine with that kind of RAM.
I thought we had a machine for hackers who want to do performance
testing. Mark?
--
Devrim GÜNDÜZ
P
On Tue, Mar 22, 2011 at 11:24 AM, Jeff Janes wrote:
> On Fri, Mar 18, 2011 at 9:19 AM, Robert Haas wrote:
>> On Fri, Mar 18, 2011 at 11:14 AM, Kevin Grittner
>> wrote:
>>> Maybe the thing to focus on first is the oft-discussed "benchmark
>>> farm" (similar to the "build farm"), with a good mix o
On 03/22/2011 12:47 PM, Jeff Janes wrote:
Maybe the thing to focus on first is the oft-discussed "benchmark
farm" (similar to the "build farm"), with a good mix of loads, so
that the impact of changes can be better tracked for multiple
workloads on a variety of platforms and configurations.
On Fri, Mar 18, 2011 at 8:14 AM, Kevin Grittner
wrote:
> rsmogura wrote:
>
>> Yes, there is some change, and I looked at this more carefully, as
>> my performance results wasn't such as I expected. I found PG uses
>> BufferAccessStrategy to do sequence scans, so my test query took
>> only 32 buff
On Fri, Mar 18, 2011 at 9:19 AM, Robert Haas wrote:
> On Fri, Mar 18, 2011 at 11:14 AM, Kevin Grittner
> wrote:
>> Maybe the thing to focus on first is the oft-discussed "benchmark
>> farm" (similar to the "build farm"), with a good mix of loads, so
>> that the impact of changes can be better tra
Hi, hackers.
I am interested in this discussion!
So I surveyed current buffer algorithms around other software. I share about it.
(sorry, it is easy survey..)
CLOCK-PRO and LIRS are popular in current buffer algorithms in my easy survey.
Their algorithms are same author that is Song Jiang.
CLOCK
Merlin Moncure Monday 21 March 2011 20:58:16
> On Mon, Mar 21, 2011 at 2:08 PM, Greg Stark wrote:
> > On Mon, Mar 21, 2011 at 3:54 PM, Merlin Moncure
wrote:
> >> Can't you make just one large mapping and lock it in 8k regions? I
> >> thought the problem with mmap was not being able to detect ot
On Mon, Mar 21, 2011 at 2:08 PM, Greg Stark wrote:
> On Mon, Mar 21, 2011 at 3:54 PM, Merlin Moncure wrote:
>> Can't you make just one large mapping and lock it in 8k regions? I
>> thought the problem with mmap was not being able to detect other
>> processes
>> (http://www.mail-archive.com/pgsql
> Was it really all that bad? IIRC we replaced ARC with the current clock
> sweep due to patent concerns. (Maybe there were performance concerns as
> well, I don't remember).
Yeah, that was why the patent was frustrating. Performance was poor and
we were planning on replacing ARC in 8.2 anyway
Excerpts from Josh Berkus's message of lun mar 21 13:47:21 -0300 2011:
> We already did that, actually, when we implemented ARC: effectively gave
> PostgreSQL a 3-level cache. The results were not very good, although
> the algorithm could be at fault there.
Was it really all that bad? IIRC we r
On Mon, Mar 21, 2011 at 4:47 PM, Josh Berkus wrote:
> You're missing my point ... Postgres already *has* a 2-level cache:
> shared_buffers and the FS cache. Anything we add to that will be adding
> levels.
I don't think those two levels are interesting -- they don't interact
cleverly at all.
I
On Mon, Mar 21, 2011 at 3:54 PM, Merlin Moncure wrote:
> Can't you make just one large mapping and lock it in 8k regions? I
> thought the problem with mmap was not being able to detect other
> processes
> (http://www.mail-archive.com/pgsql-general@postgresql.org/msg122301.html)
> compatibility is
On 3/21/11 3:24 AM, Greg Stark wrote:
>> 2-level caches work well for a variety of applications.
>
> I think 2-level caches with simple heuristics like "pin all the
> indexes" is unlikely to be helpful. At least it won't optimize the
> average case and I think that's been proven. It might be helpf
On 21.03.2011 17:54, Merlin Moncure wrote:
Can't you make just one large mapping and lock it in 8k regions? I
thought the problem with mmap was not being able to detect other
processes
(http://www.mail-archive.com/pgsql-general@postgresql.org/msg122301.html)
compatibility issues (possibly obsole
On Mon, Mar 21, 2011 at 5:24 AM, Greg Stark wrote:
> On Fri, Mar 18, 2011 at 11:55 PM, Josh Berkus wrote:
>>> To take the opposite approach... has anyone looked at having the OS just
>>> manage all caching for us? Something like MMAPed shared buffers? Even if we
>>> find the issue with large sh
On Mon, 21 Mar 2011 10:24:22 +, Greg Stark wrote:
On Fri, Mar 18, 2011 at 11:55 PM, Josh Berkus
wrote:
To take the opposite approach... has anyone looked at having the OS
just manage all caching for us? Something like MMAPed shared buffers?
Even if we find the issue with large shared buffe
On Fri, Mar 18, 2011 at 11:55 PM, Josh Berkus wrote:
>> To take the opposite approach... has anyone looked at having the OS just
>> manage all caching for us? Something like MMAPed shared buffers? Even if we
>> find the issue with large shared buffers, we still can't dedicate serious
>> amounts
On 3/18/11 11:15 AM, Jim Nasby wrote:
> To take the opposite approach... has anyone looked at having the OS just
> manage all caching for us? Something like MMAPed shared buffers? Even if we
> find the issue with large shared buffers, we still can't dedicate serious
> amounts of memory to them b
"Kevin Grittner" Thursday 17 March 2011 22:02:18
> Rados*aw Smogura wrote:
> > I have implemented initial concept of 2nd level cache. Idea is to
> > keep some segments of shared memory for special buffers (e.g.
> > indices) to prevent overwrite those by other operations. I added
> > those function
On Fri, Mar 18, 2011 at 2:15 PM, Jim Nasby wrote:
> +1
>
> To take the opposite approach... has anyone looked at having the OS just
> manage all caching for us? Something like MMAPed shared buffers? Even if we
> find the issue with large shared buffers, we still can't dedicate serious
> amounts
On Mar 18, 2011, at 11:19 AM, Robert Haas wrote:
> On Fri, Mar 18, 2011 at 11:14 AM, Kevin Grittner
> wrote:
> A related area that could use some looking at is why performance tops
> out at shared_buffers ~8GB and starts to fall thereafter. InnoDB can
> apparently handle much larger buffer pools
Excerpts from rsmogura's message of vie mar 18 11:57:48 -0300 2011:
> Actually idea of this patch was like this:
> Some operations requires many buffers, PG uses "clock sweep" to get
> next free buffer, so it may overwrite index buffer. From point of view
> of good database design We should
On Fri, Mar 18, 2011 at 11:14 AM, Kevin Grittner
wrote:
> Maybe the thing to focus on first is the oft-discussed "benchmark
> farm" (similar to the "build farm"), with a good mix of loads, so
> that the impact of changes can be better tracked for multiple
> workloads on a variety of platforms and
rsmogura wrote:
> Yes, there is some change, and I looked at this more carefully, as
> my performance results wasn't such as I expected. I found PG uses
> BufferAccessStrategy to do sequence scans, so my test query took
> only 32 buffers from pool and didn't overwritten index pool too
> much. T
On Thu, 17 Mar 2011 16:02:18 -0500, Kevin Grittner wrote:
Rados*aw Smogura wrote:
I have implemented initial concept of 2nd level cache. Idea is to
keep some segments of shared memory for special buffers (e.g.
indices) to prevent overwrite those by other operations. I added
those functionality
Rados*aw Smogura wrote:
> I have implemented initial concept of 2nd level cache. Idea is to
> keep some segments of shared memory for special buffers (e.g.
> indices) to prevent overwrite those by other operations. I added
> those functionality to nbtree index scan.
>
> I tested this with doing
Hi,
I have implemented initial concept of 2nd level cache. Idea is to keep some
segments of shared memory for special buffers (e.g. indices) to prevent
overwrite those by other operations. I added those functionality to nbtree
index scan.
I tested this with doing index scan, seq read, drop sys
52 matches
Mail list logo
