Re: [HACKERS] Patch: add timing of buffer I/O requests

2012-04-30 Thread Tom Lane
Greg Stark writes: > On Sun, Apr 29, 2012 at 12:26 AM, Robert Haas wrote: >> As for track_iotiming -> track_io_timing, I'm fine with that as well. > I'm still grumpy about the idea of a GUC changing the explain analyze > output. How would people feel about adding an explain option that > explici

Re: [HACKERS] Patch: add timing of buffer I/O requests

2012-04-30 Thread Greg Stark
On Sun, Apr 29, 2012 at 12:26 AM, Robert Haas wrote: > As for track_iotiming -> track_io_timing, I'm fine with that as well. I'm still grumpy about the idea of a GUC changing the explain analyze output. How would people feel about adding an explain option that explicitly requests io timing for th

Re: [HACKERS] Patch: add timing of buffer I/O requests

2012-04-29 Thread Tom Lane
Robert Haas writes: > I like the idea of including the word block in there. I don't think > it was probably a terribly good idea to abbreviate that to blk > everywhere, but at this point it's probably better to be consistent, > sigh. > As for track_iotiming -> track_io_timing, I'm fine with that

Re: [HACKERS] Patch: add timing of buffer I/O requests

2012-04-28 Thread Robert Haas
On Sat, Apr 28, 2012 at 12:49 PM, Tom Lane wrote: > Robert Haas writes: >> ... You might want to revisit the issue of how the new >> columns in pg_stat_statements are named, as well.  I am not sure I'm >> happy with that, but neither am I sure that I know what I'd like >> better.  It's not too cl

Re: [HACKERS] Patch: add timing of buffer I/O requests

2012-04-28 Thread Tom Lane
... btw, while I'm criticizing names, how about changing "track_iotiming" to "track_io_timing"? The former seems inelegant and unreadable. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http:/

Re: [HACKERS] Patch: add timing of buffer I/O requests

2012-04-28 Thread Tom Lane
Robert Haas writes: > ... You might want to revisit the issue of how the new > columns in pg_stat_statements are named, as well. I am not sure I'm > happy with that, but neither am I sure that I know what I'd like > better. It's not too clear that the timing is specifically for data > block read

Re: [HACKERS] Patch: add timing of buffer I/O requests

2012-04-25 Thread Robert Haas
On Wed, Apr 25, 2012 at 5:33 PM, Tom Lane wrote: > Peter Eisentraut writes: >> On mån, 2012-04-23 at 22:03 -0400, Robert Haas wrote: >>> Perhaps, but nobody's explained what we gain out of NOT using numeric. > >> So if you want to have possibly different internal and external >> representations,

Re: [HACKERS] Patch: add timing of buffer I/O requests

2012-04-25 Thread Tom Lane
Peter Eisentraut writes: > On mån, 2012-04-23 at 22:03 -0400, Robert Haas wrote: >> Perhaps, but nobody's explained what we gain out of NOT using numeric. > So if you want to have possibly different internal and external > representations, why not use interval for the external one? That doesn't

Re: [HACKERS] Patch: add timing of buffer I/O requests

2012-04-25 Thread Peter Eisentraut
On mån, 2012-04-23 at 22:03 -0400, Robert Haas wrote: > Perhaps, but nobody's explained what we gain out of NOT using numeric. So if you want to have possibly different internal and external representations, why not use interval for the external one? -- Sent via pgsql-hackers mailing list (pgsq

Re: [HACKERS] Patch: add timing of buffer I/O requests

2012-04-25 Thread Robert Haas
On Wed, Apr 25, 2012 at 1:58 PM, Greg Stark wrote: > On Wed, Apr 25, 2012 at 5:47 PM, Tom Lane wrote: >> Also, as was pointed out upthread, the underlying data in shared memory >> is almost certainly never going to be infinite-precision; so using >> numeric in the API seems to me to be more likel

Re: [HACKERS] Patch: add timing of buffer I/O requests

2012-04-25 Thread Tom Lane
Greg Stark writes: > On Wed, Apr 25, 2012 at 5:47 PM, Tom Lane wrote: >> Also, as was pointed out upthread, the underlying data in shared memory >> is almost certainly never going to be infinite-precision; so using >> numeric in the API seems to me to be more likely to convey a false >> impressio

Re: [HACKERS] Patch: add timing of buffer I/O requests

2012-04-25 Thread Greg Stark
On Wed, Apr 25, 2012 at 5:47 PM, Tom Lane wrote: > Also, as was pointed out upthread, the underlying data in shared memory > is almost certainly never going to be infinite-precision; so using > numeric in the API seems to me to be more likely to convey a false > impression of exactness than to do

Re: [HACKERS] Patch: add timing of buffer I/O requests

2012-04-25 Thread Robert Haas
On Wed, Apr 25, 2012 at 1:12 PM, Tom Lane wrote: > Robert Haas writes: >> On Wed, Apr 25, 2012 at 12:47 PM, Tom Lane wrote: >>> Robert Haas writes: However, the main thing here is that we need to do *something* here... > >>> Agreed, this has got to be pushed forward. > >> In the interest o

Re: [HACKERS] Patch: add timing of buffer I/O requests

2012-04-25 Thread Tom Lane
Robert Haas writes: > On Wed, Apr 25, 2012 at 12:47 PM, Tom Lane wrote: >> Robert Haas writes: >>> However, the main thing here is that we need to do *something* here... >> Agreed, this has got to be pushed forward. > In the interest of furthering that goal, I propose that whoever is > willing

Re: [HACKERS] Patch: add timing of buffer I/O requests

2012-04-25 Thread Robert Haas
On Wed, Apr 25, 2012 at 12:47 PM, Tom Lane wrote: > Robert Haas writes: >> On Sat, Apr 14, 2012 at 10:33 PM, Tom Lane wrote: >>> There's no particular reason to think that Moore's Law is going to >>> result in an increase in the fractional precision of timing data. >>> It hasn't done so in the p

Re: [HACKERS] Patch: add timing of buffer I/O requests

2012-04-25 Thread Tom Lane
Robert Haas writes: > On Sat, Apr 14, 2012 at 10:33 PM, Tom Lane wrote: >> There's no particular reason to think that Moore's Law is going to >> result in an increase in the fractional precision of timing data. >> It hasn't done so in the past, for sure. > Perhaps, but nobody's explained what we

Re: [HACKERS] Patch: add timing of buffer I/O requests

2012-04-23 Thread Robert Haas
On Sat, Apr 14, 2012 at 10:33 PM, Tom Lane wrote: > Robert Haas writes: >> The internal representation doesn't have to be (and certainly >> shouldn't be) numeric.  But if you translate to numeric before >> returning the data to the user, then you have the freedom, in the >> future, to whack aroun

Re: [HACKERS] Patch: add timing of buffer I/O requests

2012-04-14 Thread Tom Lane
Robert Haas writes: > The internal representation doesn't have to be (and certainly > shouldn't be) numeric. But if you translate to numeric before > returning the data to the user, then you have the freedom, in the > future, to whack around the internal representation however you like, > without

Re: [HACKERS] Patch: add timing of buffer I/O requests

2012-04-14 Thread Robert Haas
On Sat, Apr 14, 2012 at 9:54 AM, Peter Geoghegan wrote: > On 10 April 2012 19:10, Tom Lane wrote: >>> Hmm.  Maybe we should think about numeric ms, which would have all the >>> same advantages but without the round-off error. >> >> Color me unimpressed ... numeric calculations are vastly more exp

Re: [HACKERS] Patch: add timing of buffer I/O requests

2012-04-14 Thread Peter Geoghegan
On 14 April 2012 02:42, Greg Stark wrote: > Is this not subject to the birthday paradox? If you have a given hash > you're worried about a collision with then you have a > one-in-four-billion chance. But if you have a collection of hashes and > you're worried about any collisions then it only take

Re: [HACKERS] Patch: add timing of buffer I/O requests

2012-04-14 Thread Peter Geoghegan
On 10 April 2012 19:10, Tom Lane wrote: >> Hmm.  Maybe we should think about numeric ms, which would have all the >> same advantages but without the round-off error. > > Color me unimpressed ... numeric calculations are vastly more expensive > than float, and where are you going to get timing data

Re: [HACKERS] Patch: add timing of buffer I/O requests

2012-04-14 Thread Robert Haas
On Sat, Apr 14, 2012 at 3:27 AM, Greg Smith wrote: > On 04/13/2012 06:22 PM, Tom Lane wrote: >> >> But (a) I *don't* want to seriously break things, and don't see a need >> to; (b) interval is expensive and has got its own problems, notably an >> internal limitation to usec resolution that we woul

Re: [HACKERS] Patch: add timing of buffer I/O requests

2012-04-14 Thread Greg Smith
On 04/13/2012 06:22 PM, Tom Lane wrote: But (a) I *don't* want to seriously break things, and don't see a need to; (b) interval is expensive and has got its own problems, notably an internal limitation to usec resolution that we would not be able to get rid of easily. A straight float seems pre

Re: [HACKERS] Patch: add timing of buffer I/O requests

2012-04-13 Thread Robert Haas
On Fri, Apr 13, 2012 at 10:01 PM, Tom Lane wrote: > Greg Stark writes: >> On Fri, Apr 13, 2012 at 8:15 PM, Robert Haas wrote: >>> That's probably true, but I'm not sure it's worth worrying about - >>> one-in-four-billion is a pretty small probability. > >> Is this not subject to the birthday par

Re: [HACKERS] Patch: add timing of buffer I/O requests

2012-04-13 Thread Peter Geoghegan
On 14 April 2012 03:01, Tom Lane wrote: > Realistically, I'm more worried about collisions due to inadequacies in > the jumble calculation logic (Peter already pointed out some risk > factors in that regard). It's important to have a sense of proportion about the problem. The worst thing that a c

Re: [HACKERS] Patch: add timing of buffer I/O requests

2012-04-13 Thread Tom Lane
Greg Stark writes: > On Fri, Apr 13, 2012 at 8:15 PM, Robert Haas wrote: >> That's probably true, but I'm not sure it's worth worrying about - >> one-in-four-billion is a pretty small probability. > Is this not subject to the birthday paradox? If you have a given hash > you're worried about a co

Re: [HACKERS] Patch: add timing of buffer I/O requests

2012-04-13 Thread Greg Stark
On Fri, Apr 13, 2012 at 8:15 PM, Robert Haas wrote: > That's probably true, but I'm not sure it's worth worrying about - > one-in-four-billion is a pretty small probability. Is this not subject to the birthday paradox? If you have a given hash you're worried about a collision with then you have a

Re: [HACKERS] Patch: add timing of buffer I/O requests

2012-04-13 Thread Tom Lane
Jim Nasby writes: > I agree that ms is on it's way out... and frankly it wouldn't surprise me if > at some point we actually had need of ns resolution. > Given that, I don't think ms or us definitions make sense... float8 seconds > seams much more logical to me. Well, the important point is th

Re: [HACKERS] Patch: add timing of buffer I/O requests

2012-04-13 Thread Peter Eisentraut
On tis, 2012-04-10 at 09:33 -0400, Robert Haas wrote: > So, should we make the new columns exposed by pg_stat_statements use > milliseconds, so that the block read/write timings are everywhere in > milliseconds, or should we keep them as a float8, so that all the > times exposed by pg_stat_statemen

Re: [HACKERS] Patch: add timing of buffer I/O requests

2012-04-13 Thread Jim Nasby
On 4/10/12 5:07 PM, Greg Smith wrote: I'd prefer to see at least usec resolution and 8 bytes of "dynamic range" for query related statistics. Any of these would be fine from a UI perspective to me: -float8 seconds -float8 msec -float8 usec -int8 usec I don't think int8 msec will be enough res

Re: [HACKERS] Patch: add timing of buffer I/O requests

2012-04-13 Thread Peter Geoghegan
On 13 April 2012 20:15, Robert Haas wrote: > On Wed, Apr 11, 2012 at 9:02 AM, k...@rice.edu wrote: >> By using all 64-bits of the hash that we currently calculate, instead >> of the current use of 32-bits only, the collision probabilities are >> very low. > > That's probably true, but I'm not sur

Re: [HACKERS] Patch: add timing of buffer I/O requests

2012-04-13 Thread Robert Haas
On Fri, Apr 13, 2012 at 4:01 PM, Tom Lane wrote: > Robert Haas writes: >> On another note, I think this is a sufficiently major change that we >> ought to add Peter's name to the "Author" section of the >> pg_stat_statements documentation. > > +1 ... as long as we have those at all, they probably

Re: [HACKERS] Patch: add timing of buffer I/O requests

2012-04-13 Thread Tom Lane
Robert Haas writes: > On another note, I think this is a sufficiently major change that we > ought to add Peter's name to the "Author" section of the > pg_stat_statements documentation. +1 ... as long as we have those at all, they probably ought to credit anybody who did substantial work on the m

Re: [HACKERS] Patch: add timing of buffer I/O requests

2012-04-13 Thread Robert Haas
On Wed, Apr 11, 2012 at 9:02 AM, k...@rice.edu wrote: > By using all 64-bits of the hash that we currently calculate, instead > of the current use of 32-bits only, the collision probabilities are > very low. That's probably true, but I'm not sure it's worth worrying about - one-in-four-billion is

Re: [HACKERS] Patch: add timing of buffer I/O requests

2012-04-11 Thread k...@rice.edu
On Wed, Apr 11, 2012 at 01:53:06AM +0100, Peter Geoghegan wrote: > On 11 April 2012 01:16, Tom Lane wrote: > > Peter Geoghegan writes: > >> On 11 April 2012 00:35, Robert Haas wrote: > >>> If people need something like that, couldn't they create it by hashing > >>> the normalized query text with

Re: [HACKERS] Patch: add timing of buffer I/O requests

2012-04-10 Thread Peter Geoghegan
On 11 April 2012 01:16, Tom Lane wrote: > Peter Geoghegan writes: >> On 11 April 2012 00:35, Robert Haas wrote: >>> If people need something like that, couldn't they create it by hashing >>> the normalized query text with an arbitrary algorithm? > >> That supposes that the normalised query text

Re: [HACKERS] Patch: add timing of buffer I/O requests

2012-04-10 Thread Tom Lane
Peter Geoghegan writes: > On 11 April 2012 00:35, Robert Haas wrote: >> If people need something like that, couldn't they create it by hashing >> the normalized query text with an arbitrary algorithm? > That supposes that the normalised query text is perfectly stable. It > may well not be, parti

Re: [HACKERS] Patch: add timing of buffer I/O requests

2012-04-10 Thread Peter Geoghegan
On 11 April 2012 00:35, Robert Haas wrote: > If people need something like that, couldn't they create it by hashing > the normalized query text with an arbitrary algorithm? That supposes that the normalised query text is perfectly stable. It may well not be, particularly for things like ad-hoc qu

Re: [HACKERS] Patch: add timing of buffer I/O requests

2012-04-10 Thread Tom Lane
Peter Geoghegan writes: > On 10 April 2012 23:07, Greg Smith wrote: >> On 04/10/2012 12:27 PM, Tom Lane wrote: >> I am doing more sophisticated things with it, so I'll celebrate this as my >> opportunity to say I did something you didn't see coming for 2012. > This is why I requested that we exp

Re: [HACKERS] Patch: add timing of buffer I/O requests

2012-04-10 Thread Robert Haas
On Tue, Apr 10, 2012 at 6:32 PM, Peter Geoghegan wrote: > On 10 April 2012 23:07, Greg Smith wrote: >> On 04/10/2012 12:27 PM, Tom Lane wrote: >> I am doing more sophisticated things with it, so I'll celebrate this as my >> opportunity to say I did something you didn't see coming for 2012. > > Th

Re: [HACKERS] Patch: add timing of buffer I/O requests

2012-04-10 Thread Peter Geoghegan
On 10 April 2012 23:07, Greg Smith wrote: > On 04/10/2012 12:27 PM, Tom Lane wrote: > I am doing more sophisticated things with it, so I'll celebrate this as my > opportunity to say I did something you didn't see coming for 2012. This is why I requested that we expose the query_id hash value - I

Re: [HACKERS] Patch: add timing of buffer I/O requests

2012-04-10 Thread Greg Smith
On 04/10/2012 12:27 PM, Tom Lane wrote: Changing the column name will definitely break anything more specific than "select * from pg_stat_statements". However, it's less clear that changing the units in which the column is expressed will break things. It seems likely to me that nobody out there

Re: [HACKERS] Patch: add timing of buffer I/O requests

2012-04-10 Thread Tom Lane
Robert Haas writes: > On Tue, Apr 10, 2012 at 1:58 PM, Tom Lane wrote: >> The business about underlying microseconds is maybe not so good, but >> I don't think we want to touch that right now.  In the long run >> I think it would make sense to converge on float8 msec as being the >> standard for

Re: [HACKERS] Patch: add timing of buffer I/O requests

2012-04-10 Thread k...@rice.edu
On Tue, Apr 10, 2012 at 02:01:02PM -0400, Robert Haas wrote: > On Tue, Apr 10, 2012 at 1:58 PM, Tom Lane wrote: > > Robert Haas writes: > >> On Tue, Apr 10, 2012 at 1:44 PM, Tom Lane wrote: > >>> Huh?  I understood what you said upthread to be that we have two ways > >>> in existing releases (an

Re: [HACKERS] Patch: add timing of buffer I/O requests

2012-04-10 Thread Robert Haas
On Tue, Apr 10, 2012 at 1:58 PM, Tom Lane wrote: > Robert Haas writes: >> On Tue, Apr 10, 2012 at 1:44 PM, Tom Lane wrote: >>> Huh?  I understood what you said upthread to be that we have two ways >>> in existing releases (anything unreleased has zero standing in this >>> discussion): float8 sec

Re: [HACKERS] Patch: add timing of buffer I/O requests

2012-04-10 Thread Tom Lane
Robert Haas writes: > On Tue, Apr 10, 2012 at 1:44 PM, Tom Lane wrote: >> Huh? I understood what you said upthread to be that we have two ways >> in existing releases (anything unreleased has zero standing in this >> discussion): float8 sec in pg_stat_statements.total_time, and >> int8 msec ever

Re: [HACKERS] Patch: add timing of buffer I/O requests

2012-04-10 Thread Robert Haas
On Tue, Apr 10, 2012 at 1:44 PM, Tom Lane wrote: > Robert Haas writes: >> On Tue, Apr 10, 2012 at 12:11 PM, Tom Lane wrote: >>> Well, FWIW I vote for making the new columns be float8 msec. > >> Ugh.  So the three ways of doing timing that we have already aren't >> enough, and we need a fourth on

Re: [HACKERS] Patch: add timing of buffer I/O requests

2012-04-10 Thread Tom Lane
Robert Haas writes: > On Tue, Apr 10, 2012 at 12:11 PM, Tom Lane wrote: >> Well, FWIW I vote for making the new columns be float8 msec. > Ugh. So the three ways of doing timing that we have already aren't > enough, and we need a fourth one? Ack! Huh? I understood what you said upthread to be

Re: [HACKERS] Patch: add timing of buffer I/O requests

2012-04-10 Thread Robert Haas
On Tue, Apr 10, 2012 at 12:11 PM, Tom Lane wrote: > Robert Haas writes: >> No matter what we end up doing here it will be consistent with >> something; I am reminded of the phrase "the good thing about standards >> is that there are so many to choose from...". > > Well, FWIW I vote for making the

Re: [HACKERS] Patch: add timing of buffer I/O requests

2012-04-10 Thread Magnus Hagander
On Tue, Apr 10, 2012 at 18:27, Tom Lane wrote: > Magnus Hagander writes: >> On Tue, Apr 10, 2012 at 17:58, Robert Haas wrote: >>> On Tue, Apr 10, 2012 at 10:06 AM, Tom Lane wrote: Given that we've whacked pg_stat_statements' behavior around rather thoroughly in this release, maybe we

Re: [HACKERS] Patch: add timing of buffer I/O requests

2012-04-10 Thread Tom Lane
Magnus Hagander writes: > On Tue, Apr 10, 2012 at 17:58, Robert Haas wrote: >> On Tue, Apr 10, 2012 at 10:06 AM, Tom Lane wrote: >>> Given that we've whacked pg_stat_statements' behavior around rather >>> thoroughly in this release, maybe we could get away with redefining >>> total_time as being

Re: [HACKERS] Patch: add timing of buffer I/O requests

2012-04-10 Thread Magnus Hagander
On Tue, Apr 10, 2012 at 17:58, Robert Haas wrote: > On Tue, Apr 10, 2012 at 10:06 AM, Tom Lane wrote: >> Robert Haas writes: >>> Hmm.  So, on further review, this is not as simple as it seems.  I'd >>> like some input from other people on what we should do here. >> >>> pg_stat_statements has lon

Re: [HACKERS] Patch: add timing of buffer I/O requests

2012-04-10 Thread Tom Lane
Robert Haas writes: > No matter what we end up doing here it will be consistent with > something; I am reminded of the phrase "the good thing about standards > is that there are so many to choose from...". Well, FWIW I vote for making the new columns be float8 msec. If you don't want to change t

Re: [HACKERS] Patch: add timing of buffer I/O requests

2012-04-10 Thread Robert Haas
On Tue, Apr 10, 2012 at 10:06 AM, Tom Lane wrote: > Robert Haas writes: >> Hmm.  So, on further review, this is not as simple as it seems.  I'd >> like some input from other people on what we should do here. > >> pg_stat_statements has long exposed a column called "total_time" as a >> float8.  It

Re: [HACKERS] Patch: add timing of buffer I/O requests

2012-04-10 Thread Peter Geoghegan
On 10 April 2012 14:33, Robert Haas wrote: > So, should we make the new columns exposed by pg_stat_statements use > milliseconds, so that the block read/write timings are everywhere in > milliseconds, or should we keep them as a float8, so that all the > times exposed by pg_stat_statements use flo

Re: [HACKERS] Patch: add timing of buffer I/O requests

2012-04-10 Thread Tom Lane
Robert Haas writes: > Hmm. So, on further review, this is not as simple as it seems. I'd > like some input from other people on what we should do here. > pg_stat_statements has long exposed a column called "total_time" as a > float8. It now exposes columns "time_read" and "time_write" which ar

Re: [HACKERS] Patch: add timing of buffer I/O requests

2012-04-10 Thread Robert Haas
On Thu, Apr 5, 2012 at 11:45 AM, Robert Haas wrote: > Meanwhile, pg_stat_statements converts the same data to seconds but > makes it a double rather than a bigint.  I think that was a bad idea > and we should make it consistent use a bigint and milliseconds while > we still can. Hmm. So, on furt

Re: [HACKERS] Patch: add timing of buffer I/O requests

2012-04-05 Thread Robert Haas
On Tue, Mar 27, 2012 at 8:30 PM, Robert Haas wrote: > On Tue, Mar 27, 2012 at 3:22 PM, Ants Aasma wrote: >> On Tue, Mar 27, 2012 at 9:58 PM, Robert Haas wrote: >>> I've committed the core of this.  I left out the stats collector >>> stuff, because it's still per-table and I think perhaps we shou

Re: [HACKERS] Patch: add timing of buffer I/O requests

2012-03-27 Thread Robert Haas
On Tue, Mar 27, 2012 at 8:41 PM, Greg Stark wrote: > On Tue, Mar 27, 2012 at 7:58 PM, Robert Haas wrote: >> I've committed the core of this.  I left out the stats collector >> stuff, because it's still per-table and I think perhaps we should back >> off to just per-database.  I changed it so that

Re: [HACKERS] Patch: add timing of buffer I/O requests

2012-03-27 Thread Greg Stark
On Tue, Mar 27, 2012 at 7:58 PM, Robert Haas wrote: > I've committed the core of this.  I left out the stats collector > stuff, because it's still per-table and I think perhaps we should back > off to just per-database.  I changed it so that it does not conflate > wait time with I/O time.  Maybe t

Re: [HACKERS] Patch: add timing of buffer I/O requests

2012-03-27 Thread Robert Haas
On Tue, Mar 27, 2012 at 3:22 PM, Ants Aasma wrote: > On Tue, Mar 27, 2012 at 9:58 PM, Robert Haas wrote: >> I've committed the core of this.  I left out the stats collector >> stuff, because it's still per-table and I think perhaps we should back >> off to just per-database.  I changed it so that

Re: [HACKERS] Patch: add timing of buffer I/O requests

2012-03-27 Thread Ants Aasma
On Tue, Mar 27, 2012 at 9:58 PM, Robert Haas wrote: > I've committed the core of this.  I left out the stats collector > stuff, because it's still per-table and I think perhaps we should back > off to just per-database.  I changed it so that it does not conflate > wait time with I/O time.  Maybe t

Re: [HACKERS] Patch: add timing of buffer I/O requests

2012-03-27 Thread Robert Haas
On Tue, Mar 27, 2012 at 2:58 PM, Robert Haas wrote: > On Thu, Mar 22, 2012 at 7:38 PM, Ants Aasma wrote: >> On Wed, Mar 21, 2012 at 5:01 PM, Robert Haas wrote: >>> This seems to have bitrotted again.  :-( >>> >>> Can you please rebase again? >> >> Rebased. > > I've committed the core of this.  I

Re: [HACKERS] Patch: add timing of buffer I/O requests

2012-03-27 Thread Robert Haas
On Thu, Mar 22, 2012 at 7:38 PM, Ants Aasma wrote: > On Wed, Mar 21, 2012 at 5:01 PM, Robert Haas wrote: >> This seems to have bitrotted again.  :-( >> >> Can you please rebase again? > > Rebased. I've committed the core of this. I left out the stats collector stuff, because it's still per-tabl

Re: [HACKERS] Patch: add timing of buffer I/O requests

2012-03-21 Thread Robert Haas
On Fri, Feb 24, 2012 at 2:23 PM, Ants Aasma wrote: > On Wed, Feb 22, 2012 at 6:35 PM, Ants Aasma wrote: >>> Some implementation notes.  This currently fails regression test >>> create_function_3, haven't looked into why yet. >> >> I'll take a look at it. > > The failure was due to leakproof chang

Re: [HACKERS] Patch: add timing of buffer I/O requests

2012-02-24 Thread Jim Nasby
On 2/22/12 10:35 AM, Ants Aasma wrote: The reason why I didn't add write timings to relation stats is that I couldn't figure out what the semantics should be. It could be either "time spent waiting for this relations blocks to be written out" or "time spent waiting for some other relations blocks

Re: [HACKERS] Patch: add timing of buffer I/O requests

2012-02-22 Thread Ants Aasma
On Wed, Feb 22, 2012 at 4:43 PM, Greg Smith wrote: > Attached are updated versions of this feature without the pg_test_timing > tool part, since I broke that out into another discussion thread.  I've > split the part that updates pg_stat_statistics out from the main feature > too, separate patch a

Re: [HACKERS] Patch: add timing of buffer I/O requests

2012-01-17 Thread Greg Smith
Attached is the pg_test_timing utility portion of this submission, broken out into its own patch. It's a contrib module modeled on pg_test_fsync. The documentation is still a bit rough, I'm not done with that yet. I have included an example of good timing results, switching to a bad clock s

Re: [HACKERS] Patch: add timing of buffer I/O requests

2012-01-15 Thread Greg Smith
On 01/15/2012 05:14 PM, Ants Aasma wrote: I hope that having a tool to measure the overhead and check the sanity of clock sources is enough to answer the worries about the potential performance hit. We could also check that the clock source is fast enough on start-up/when the guc is changed, but

Re: [HACKERS] Patch: add timing of buffer I/O requests

2011-11-28 Thread Tom Lane
Greg Smith writes: > On 11/28/2011 05:51 AM, Robert Haas wrote: >> Assuming the feature is off by default (and I can't imagine we'd >> consider anything else), I don't see why that should be cause for >> concern. If the instrumentation creates too much system load, then >> don't use it: simple as

Re: [HACKERS] Patch: add timing of buffer I/O requests

2011-11-28 Thread Greg Smith
On 11/28/2011 05:51 AM, Robert Haas wrote: On Mon, Nov 28, 2011 at 2:54 AM, Greg Smith wrote: The real problem with this whole area is that we know there are systems floating around where the amount of time taken to grab timestamps like this is just terrible. Assuming the feature is off by def

Re: [HACKERS] Patch: add timing of buffer I/O requests

2011-11-28 Thread Ants Aasma
Sorry for taking so long to respong, had a pretty busy day at work. Anyway.. On Mon, Nov 28, 2011 at 9:54 AM, Greg Smith wrote: > Oh no, it's party pooper time again.  Sorry I have to be the one to do it > this round.  The real problem with this whole area is that we know there are > systems floa

Re: [HACKERS] Patch: add timing of buffer I/O requests

2011-11-28 Thread Tom Lane
Dimitri Fontaine writes: > That sounds good for other interesting things, which entails being able > to have timing information attached to the XID sequence. If we go this > way, how far are we from having a ticker in PostgreSQL? Those of us who are trying to get rid of idle-state process wakeup

Re: [HACKERS] Patch: add timing of buffer I/O requests

2011-11-28 Thread Tomas Vondra
On 29.11.2011 02:14, Jim Nasby wrote: > On Nov 28, 2011, at 9:29 AM, Tomas Vondra wrote: I recall a patch similar to this one was submitted by Greg Stark some >>> time ago. It used the info for different reasons--to try and >>> figure out whether reads were cached or not--but I believe i

Re: [HACKERS] Patch: add timing of buffer I/O requests

2011-11-28 Thread Tomas Vondra
On 28.11.2011 22:32, Dimitri Fontaine wrote: > "Tomas Vondra" writes: >> Another option would be to reimplement the vsyscall, even on platforms >> that don't provide it. The principle is actually quite simple - allocate a >> shared memory, store there a current time and update it whenever a clock

Re: [HACKERS] Patch: add timing of buffer I/O requests

2011-11-28 Thread Jim Nasby
On Nov 28, 2011, at 9:29 AM, Tomas Vondra wrote: >>> I recall a patch similar to this one was submitted by Greg Stark some >> time ago. It used the info for different reasons--to try and figure out >> whether reads were cached or not--but I believe it withered rather than >> being implemented main

Re: [HACKERS] Patch: add timing of buffer I/O requests

2011-11-28 Thread Dimitri Fontaine
"Tomas Vondra" writes: > Another option would be to reimplement the vsyscall, even on platforms > that don't provide it. The principle is actually quite simple - allocate a > shared memory, store there a current time and update it whenever a clock > interrupt happens. This is basically what Greg s

Re: [HACKERS] Patch: add timing of buffer I/O requests

2011-11-28 Thread Martijn van Oosterhout
On Sun, Nov 27, 2011 at 11:54:38PM -0800, Greg Smith wrote: > On 11/27/2011 04:39 PM, Ants Aasma wrote: > >On the AMD I saw about 3% performance drop with timing enabled. On the > >Intel machine I couldn't measure any statistically significant change. > > Oh no, it's party pooper time again. Sorr

Re: [HACKERS] Patch: add timing of buffer I/O requests

2011-11-28 Thread Tom Lane
Robert Haas writes: > On Mon, Nov 28, 2011 at 2:54 AM, Greg Smith wrote: >> The real problem with this whole area is that we know there are >> systems floating around where the amount of time taken to grab timestamps >> like this is just terrible. > Assuming the feature is off by default (and I

Re: [HACKERS] Patch: add timing of buffer I/O requests

2011-11-28 Thread Tomas Vondra
On 28 Listopad 2011, 15:40, Greg Stark wrote: > On Nov 28, 2011 8:55 AM, "Greg Smith" wrote: >> >> On 11/27/2011 04:39 PM, Ants Aasma wrote: >>> >>> On the AMD I saw about 3% performance drop with timing enabled. On the >>> Intel machine I couldn't measure any statistically significant change. >>

Re: [HACKERS] Patch: add timing of buffer I/O requests

2011-11-28 Thread Greg Stark
On Nov 28, 2011 8:55 AM, "Greg Smith" wrote: > > On 11/27/2011 04:39 PM, Ants Aasma wrote: >> >> On the AMD I saw about 3% performance drop with timing enabled. On the >> Intel machine I couldn't measure any statistically significant change. > > > Oh no, it's party pooper time again. Sorry I have

Re: [HACKERS] Patch: add timing of buffer I/O requests

2011-11-28 Thread Robert Haas
On Mon, Nov 28, 2011 at 2:54 AM, Greg Smith wrote: > The real problem with this whole area is that we know there are > systems floating around where the amount of time taken to grab timestamps > like this is just terrible. Assuming the feature is off by default (and I can't imagine we'd consider

Re: [HACKERS] Patch: add timing of buffer I/O requests

2011-11-28 Thread Tomas Vondra
On 28 Listopad 2011, 8:54, Greg Smith wrote: > -Have one of the PostgreSQL background processes keep track of a time > estimate on its own, only periodically pausing to sync against the real > time. Then most calls to gettimeofday() can use that value instead. I > was thinking of that idea for sl

Re: [HACKERS] Patch: add timing of buffer I/O requests

2011-11-27 Thread Greg Smith
On 11/27/2011 04:39 PM, Ants Aasma wrote: On the AMD I saw about 3% performance drop with timing enabled. On the Intel machine I couldn't measure any statistically significant change. Oh no, it's party pooper time again. Sorry I have to be the one to do it this round. The real problem with t