Greg Stark writes:
> On Fri, Oct 14, 2011 at 4:29 PM, Tom Lane wrote:
>> Right. I think this one falls into my class #2, ie, we have no idea how
>> to implement it usefully. Doesn't (necessarily) mean that the core
>> concept is without merit.
> Hm. given that we have an implementation I would
On Fri, Oct 14, 2011 at 4:29 PM, Tom Lane wrote:
> Right. I think this one falls into my class #2, ie, we have no idea how
> to implement it usefully. Doesn't (necessarily) mean that the core
> concept is without merit.
Hm. given that we have an implementation I wouldn't say we have *no*
clue.
Alvaro Herrera wrote:
>
> Excerpts from Bruce Momjian's message of vie oct 14 12:12:22 -0300 2011:
> >
> > Alvaro Herrera wrote:
>
> > > The guideline, last I checked, was that before getting into coding any
> > > item from the TODO list, the prospective hacker should check previous
> > > discus
Robert Haas writes:
> On Fri, Oct 14, 2011 at 11:12 AM, Bruce Momjian wrote:
>> OK. But if we are pretty sure we don't want something, e.g. hibernate,
>> we shouldn't add it.
> Fair enough, but I'm not even slightly sure that we don't want that.
> I think having prewarming utilities available a
Excerpts from Bruce Momjian's message of vie oct 14 12:12:22 -0300 2011:
>
> Alvaro Herrera wrote:
> > The guideline, last I checked, was that before getting into coding any
> > item from the TODO list, the prospective hacker should check previous
> > discussions and initiate a new one on this l
Alvaro Herrera writes:
> Excerpts from Bruce Momjian's message of vie oct 14 11:56:22 -0300 2011:
>> Tom Lane wrote:
>>> There is plenty of stuff in the TODO list for which there is no
>>> consensus.
>> Uh, we should probably remove those then. Can you think of any?
> Unless something is blatan
On Fri, Oct 14, 2011 at 11:12 AM, Bruce Momjian wrote:
> OK. But if we are pretty sure we don't want something, e.g. hibernate,
> we shouldn't add it.
Fair enough, but I'm not even slightly sure that we don't want that.
I think having prewarming utilities available as contrib modules or on
PGXN
Alvaro Herrera wrote:
>
> Excerpts from Bruce Momjian's message of vie oct 14 11:56:22 -0300 2011:
> > Tom Lane wrote:
> > > =?ISO-8859-1?Q?C=E9dric_Villemain?=
> > > writes:
> > > > 2011/10/14 Bruce Momjian :
> > > >> Should this be marked as TODO?
> > >
> > > > I suppose TODO items *are* want
Excerpts from Bruce Momjian's message of vie oct 14 11:56:22 -0300 2011:
> Tom Lane wrote:
> > =?ISO-8859-1?Q?C=E9dric_Villemain?=
> > writes:
> > > 2011/10/14 Bruce Momjian :
> > >> Should this be marked as TODO?
> >
> > > I suppose TODO items *are* wanted and so working on them should remove
Tom Lane wrote:
> =?ISO-8859-1?Q?C=E9dric_Villemain?=
> writes:
> > 2011/10/14 Bruce Momjian :
> >> Should this be marked as TODO?
>
> > I suppose TODO items *are* wanted and so working on them should remove
> > the pain to convince people here to accept the feature, aren't they ?
>
> There is
=?ISO-8859-1?Q?C=E9dric_Villemain?= writes:
> 2011/10/14 Bruce Momjian :
>> Should this be marked as TODO?
> I suppose TODO items *are* wanted and so working on them should remove
> the pain to convince people here to accept the feature, aren't they ?
There is plenty of stuff in the TODO list fo
On 14.10.2011 11:44, Cédric Villemain wrote:
2011/10/14 Bruce Momjian:
Should this be marked as TODO?
I suppose TODO items *are* wanted and so working on them should remove
the pain to convince people here to accept the feature, aren't they ?
I don't think this is worthwhile to have in the
2011/10/14 Bruce Momjian :
>
> Should this be marked as TODO?
I suppose TODO items *are* wanted and so working on them should remove
the pain to convince people here to accept the feature, aren't they ?
>
> ---
>
> Mitsuru IW
Should this be marked as TODO?
---
Mitsuru IWASAKI wrote:
> Hi,
>
> > On 05/07/2011 03:32 AM, Mitsuru IWASAKI wrote:
> > > For 1, I've just finish my work. The latest patch is available at:
> > > http://people.freebsd.org/
On 06/05/2011 08:50 AM, Mitsuru IWASAKI wrote:
It seems that I don't have enough time to complete this work.
You don't need to keep cc'ing me, and I'm very happy if postgres to be
the first DBMS which support buffer cache hibernation feature.
Thanks for submitting the patch, and we'll see w
Hi,
> On 05/07/2011 03:32 AM, Mitsuru IWASAKI wrote:
> > For 1, I've just finish my work. The latest patch is available at:
> > http://people.freebsd.org/~iwasaki/postgres/buffer-cache-hibernation-postgresql-20110507.patch
> >
>
> Reminder here--we can't accept code based on it being publish
On Wed, Jun 1, 2011 at 8:58 AM, Jeff Janes wrote:
> In the latter case, wouldn't we just trigger the same inefficient
> scattered read of the data that normal database operation would
> trigger, taking about the same amount of time to reach cache-warmth?
If you have a system where you're bandwidt
On Wed, Jun 1, 2011 at 11:58 AM, Jeff Janes wrote:
> On Sun, May 15, 2011 at 11:19 AM, Robert Haas wrote:
>> I don't think there's any need for this to get data into
>> shared_buffers at all. Getting it into the OS cache oughta be plenty
>> sufficient, no?
>>
>> ISTM that a very simple approach
On Sun, May 15, 2011 at 11:19 AM, Robert Haas wrote:
> I don't think there's any need for this to get data into
> shared_buffers at all. Getting it into the OS cache oughta be plenty
> sufficient, no?
>
> ISTM that a very simple approach here would be to save the contents of
> each shared buffer
On 06/01/2011 03:03 AM, Tatsuo Ishii wrote:
Also I really want to see the performance comparison between these two
approaches in the real world database.
Well, tell me how big of a performance improvement you want PgFincore to
win by, and I'll construct a benchmark where it does that. If
2011/6/1 Tatsuo Ishii :
>> Yeah, I'm pretty well convinced this whole approach is a dead end.
>> Priming the OS buffer cache seems way more useful. I also think
>> saving the blocks to be read rather than the actual blocks makes a lot
>> more sense.
>
> Well, his proposal works on any platforms Po
> Yeah, I'm pretty well convinced this whole approach is a dead end.
> Priming the OS buffer cache seems way more useful. I also think
> saving the blocks to be read rather than the actual blocks makes a lot
> more sense.
Well, his proposal works on any platforms PostgreSQL supports. On the
other
On 05/07/2011 03:32 AM, Mitsuru IWASAKI wrote:
For 1, I've just finish my work. The latest patch is available at:
http://people.freebsd.org/~iwasaki/postgres/buffer-cache-hibernation-postgresql-20110507.patch
Reminder here--we can't accept code based on it being published to a web
page.
2011/5/15 Robert Haas :
> On Fri, May 6, 2011 at 5:31 PM, Greg Smith wrote:
>> I think that all the complexity with CRCs etc. is unlikely to lead anywhere
>> too, and those two issues are not completely unrelated. The simplest,
>> safest thing here is the right way to approach this, not the most
On Fri, May 6, 2011 at 5:31 PM, Greg Smith wrote:
> I think that all the complexity with CRCs etc. is unlikely to lead anywhere
> too, and those two issues are not completely unrelated. The simplest,
> safest thing here is the right way to approach this, not the most
> complicated one, and a simp
Mitsuru IWASAKI wrote:
> Are there any good examples for extension module?
Browse the subdirectories of contrib.
-Kevin
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Hi,
> I'd suggest doing this as an extension module. All the changes to
> existing server code seem superficial.
It sounds interesting. I'll try it later.
Are there any good examples for extension module?
Thanks
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make ch
Hi,
> We can't accept patches just based on a pointer to a web site. Please
> e-mail this to the mailing list so that it can be considered a
> submission under the project's licensing terms.
>
> > I hope this would be committable and the final version.
> >
>
> PostgreSQL has high standards
Hi,
Sorry, I missed these messages because I didn't subscribe to this list.
# I've just subscribed temporary
> > I think that all the complexity with CRCs etc. is unlikely to lead anywhere
> > too, and those two issues are not completely unrelated. The simplest,
> > safest thing here is the right
On 08.05.2011 07:58, Mitsuru IWASAKI wrote:
I'll do more testing tomorrow, and hopefully finalize my patch.
Done! the patch is available at:
http://people.freebsd.org/~iwasaki/postgres/buffer-cache-hibernation-postgresql-20110508.patch
I'd suggest doing this as an extension module. All the c
Mitsuru IWASAKI wrote:
the patch is available at:
http://people.freebsd.org/~iwasaki/postgres/buffer-cache-hibernation-postgresql-20110508.patch
We can't accept patches just based on a pointer to a web site. Please
e-mail this to the mailing list so that it can be considered a
submission
Hi, folks!
> I'll do more testing tomorrow, and hopefully finalize my patch.
Done! the patch is available at:
http://people.freebsd.org/~iwasaki/postgres/buffer-cache-hibernation-postgresql-20110508.patch
I hope this would be committable and the final version.
Major changes from the experimen
On Sat, May 7, 2011 at 3:32 AM, Mitsuru IWASAKI wrote:
> I have one more day for working on this, but I may give up...
I think this is an interesting line of inquiry, but if you were hoping
to get something committable in a couple of days, you had unrealistic
expectations...
--
Robert Haas
Ente
Hi, thanks for your comments!
I'm glad to discuss about this topic.
> * pgfadv_WILLNEED
> * pgfadv_WILLNEED_snapshot
>
> The former ask to load each segment of a relation *but* the kernel can
> decide to not do that or load only part of each segment. (so it is not
> as brutal as cat file > /dev
On Fri, May 6, 2011 at 5:31 PM, Greg Smith wrote:
> On 05/05/2011 05:06 AM, Mitsuru IWASAKI wrote:
>>
>> In summary, PgFincore's target is File System Buffer Cache, Buffer
>> Cache Hibernation's target is DB Buffer Cache(shared buffers).
>>
>
> Right. The thing to realize is that shared_buffers i
On 05/05/2011 05:06 AM, Mitsuru IWASAKI wrote:
In summary, PgFincore's target is File System Buffer Cache, Buffer
Cache Hibernation's target is DB Buffer Cache(shared buffers).
Right. The thing to realize is that shared_buffers is becoming a
smaller fraction of the total RAM used by the d
Hi,
I revised the patch against HEAD, it's available at:
http://people.freebsd.org/~iwasaki/postgres/buffer-cache-hibernation-postgresql-20110506.patch
Implemented hibernation file validations:
- comparison with pg_control
At shutdown:
pg_control state should be DB_SHUTDOWNED.
At startup:
pg_cont
2011/5/5 Mitsuru IWASAKI :
> Hi,
>
>> I think that PgFincore (http://pgfoundry.org/projects/pgfincore/)
>> provides similar functionality. Are you familiar with that? If so,
>> could you contrast your approach with that one?
>
> I'm not familiar with PgFincore at all sorry, but I got source code
Hi,
> I think that PgFincore (http://pgfoundry.org/projects/pgfincore/)
> provides similar functionality. Are you familiar with that? If so,
> could you contrast your approach with that one?
I'm not familiar with PgFincore at all sorry, but I got source code
and documents and read through them
Hi, thanks for good suggestions.
> > Postgres usually starts with ZERO buffer cache. By saving the buffer
> > cache data structure into hibernation files just before shutdown, and
> > loading them at startup, postgres can start operations with the saved
> > buffer cache as the same condition as j
2011/5/4 Josh Berkus :
> All,
>
> I thought that Dimitri had already implemented this using Fincore. It's
> linux-only, but that should work well enough to test the general concept.
Harald provided me some pointers at pgday in Stuttgart to make it work
with windows but ... hum I have not windows
Josh Berkus writes:
> I thought that Dimitri had already implemented this using Fincore. It's
> linux-only, but that should work well enough to test the general concept.
Actually, Cédric did, and I have a clone of his repository where I did
some debian packaging of it.
http://villemain.org/pr
All,
I thought that Dimitri had already implemented this using Fincore. It's
linux-only, but that should work well enough to test the general concept.
--
Josh Berkus
PostgreSQL Experts Inc.
http://pgexperts.com
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make chan
On Wed, May 4, 2011 at 7:10 AM, Mitsuru IWASAKI wrote:
> Hi,
>
> I am working on new feature `Buffer Cache Hibernation' which enables
> postgres to keep higher cache hit ratio even just started.
>
> Postgres usually starts with ZERO buffer cache. By saving the buffer
> cache data structure into h
Alvaro Herrera wrote:
As for gain, I have heard of test setups requiring hours of runtime in
order to prime the buffer cache.
And production ones too. I have multiple customers where a server
restart is almost a planned multi-hour downtime. The system may be back
up, but for a couple of
On Wed, May 4, 2011 at 4:44 PM, Tom Lane wrote:
> Do you have
> any proof that writing out a few GB of buffers and then reading them
> back in is actually much cheaper than letting the database re-read the
> data from the disk files?
I believe he's just writing out the meta data. Ie, which blocks
2011/5/4 Greg Stark :
> On Wed, May 4, 2011 at 3:10 PM, Mitsuru IWASAKI
> wrote:
>> Postgres usually starts with ZERO buffer cache. By saving the buffer
>> cache data structure into hibernation files just before shutdown, and
>> loading them at startup, postgres can start operations with the sav
Excerpts from Tom Lane's message of mié may 04 12:44:36 -0300 2011:
> This seems like a lot of complication for rather dubious gain. What
> happens when the DBA changes the shared_buffers setting, for instance?
> How do you protect against the cached buffers getting out-of-sync with
> the actual
Mitsuru IWASAKI writes:
> Postgres usually starts with ZERO buffer cache. By saving the buffer
> cache data structure into hibernation files just before shutdown, and
> loading them at startup, postgres can start operations with the saved
> buffer cache as the same condition as just before the la
On Wed, May 4, 2011 at 3:10 PM, Mitsuru IWASAKI wrote:
> Postgres usually starts with ZERO buffer cache. By saving the buffer
> cache data structure into hibernation files just before shutdown, and
> loading them at startup, postgres can start operations with the saved
> buffer cache as the same
On 05/04/2011 10:10 AM, Mitsuru IWASAKI wrote:
Hi,
I am working on new feature `Buffer Cache Hibernation' which enables
postgres to keep higher cache hit ratio even just started.
Postgres usually starts with ZERO buffer cache. By saving the buffer
cache data structure into hibernation files
51 matches
Mail list logo
