On Saturday, November 19, 2011 12:18:07 AM Kevin Grittner wrote:
> Andres Freund wrote:
> > I think opannotate -a -s produces output with instructions/code
> > intermingled.
>
> Thanks. I'll check out perf later (thanks for the tips!), but for
> now, here's the function which was at the top of m
On Fri, Nov 18, 2011 at 4:38 PM, Peter Geoghegan wrote:
> I understand that we highly value extensibility and genericity (yes,
> that's a real word). We may not always be well served by that
> tendency.
True (except that genericity is not a synonym for generality AFAICT).
A good fraction of the o
On Saturday, November 19, 2011 12:16:18 AM Tom Lane wrote:
> Andres Freund writes:
> > Hi,
> > For benchmarking the parser I added the above options (dim suggested this
> > on irc) which proved to be rather useful for me.
>
> What exactly is EXPLAIN printing, if you've not done planning?
Nothing
On Fri, Nov 18, 2011 at 6:46 PM, Kevin Grittner
wrote:
>>> tps = 21946.961196 (including connections establishing)
>>> tps = 22911.873227 (including connections establishing)
>>>
>>> For write transactions, that seems pretty respectable.
>>
>> Very. What do you get without the patch?
>
> [quick r
Robert Haas wrote:
> Hmm. That looks a lot like a profile with no lock contention at
> all. Since I see XLogInsert in there, I assume this must be a
> pgbench write test on unlogged tables? How close am I?
Not unless pgbench on HEAD does that by default. Here are the
relevant statements:
On Fri, Nov 18, 2011 at 2:05 PM, Kevin Grittner
wrote:
> Robert Haas wrote:
>> Any chance you can run oprofile (on either branch, don't really
>> care) against the 32 client test and post the results?
>
> [ oprofile results ]
Hmm. That looks a lot like a profile with no lock contention at all.
Andres Freund wrote:
> I think opannotate -a -s produces output with instructions/code
> intermingled.
Thanks. I'll check out perf later (thanks for the tips!), but for
now, here's the function which was at the top of my oprofile
results, annotated with those options. I'm afraid it's a bit
i
Andres Freund writes:
> Hi,
> For benchmarking the parser I added the above options (dim suggested this on
> irc) which proved to be rather useful for me.
What exactly is EXPLAIN printing, if you've not done planning? Also, I
believe the planner depends on the assumption that the rewriter has d
On 18 November 2011 22:26, Kevin Grittner wrote:
> I still have a lot of reading to do before I propose anything
> concrete for development, but one thing that has already struck me
> as a common theme for MVs is that a lot of people seem to like the
> idea of first creating a "normal" view, and t
On 18 November 2011 23:26, Kevin Grittner wrote:
> I still have a lot of reading to do before I propose anything
> concrete for development, but one thing that has already struck me
> as a common theme for MVs is that a lot of people seem to like the
> idea of first creating a "normal" view, and t
On Friday, November 18, 2011 11:12:02 PM Andres Freund wrote:
> On Friday, November 18, 2011 09:16:01 PM Kevin Grittner wrote:
> > Andres Freund wrote:
> > > When doing line-level profiles I would suggest looking at the
> > > instructions.
> >
> > What's the best way to do that?
>
> I think opan
On Friday, November 18, 2011 09:16:01 PM Kevin Grittner wrote:
> Andres Freund wrote:
> > When doing line-level profiles I would suggest looking at the
> > instructions.
> What's the best way to do that?
I think opannotate -a -s produces output with instructions/code intermingled.
> > I don't thi
I still have a lot of reading to do before I propose anything
concrete for development, but one thing that has already struck me
as a common theme for MVs is that a lot of people seem to like the
idea of first creating a "normal" view, and then materializing it.
That seems pretty attractive to me,
On Fri, Nov 18, 2011 at 12:20:26AM -0500, Robert Haas wrote:
> I think that we should really consider doing with this patch what Tom
> suggested upthread; namely, looking for a mechanism to allow
> individual datatypes to offer up a comparator function that doesn't
> require bouncing through Functi
"Kevin Grittner" wrote:
> I'm considering submitting a proposal to management that I be
> assigned to work on a declarative implementation in PostgreSQL to
> allow speedier application development of software needing
> materialized views.
Thanks to all who provided feedback and support in resp
On Friday, November 18, 2011 10:14:22 PM Tom Lane wrote:
> Andres Freund writes:
> > For unknown reasons the function used non chained ifs for every handled
> > nodeType.
> > Replacing the if chain with if; else if; ... resulted in a small
> > speedup. Replacing it with a switch() in a bigger one.
On Friday, November 18, 2011 10:11:29 PM Tom Lane wrote:
> Andres Freund writes:
> > In many scenarios memory allocation is one of the top 3 functions showing
> > up in profiles. Looking at hierarchical profiles
> > (-fno-omit-frame-pointer) at least during parsing, planning and executor
> > start
On 18 November 2011 05:20, Robert Haas wrote:
> I think that we should really consider doing with this patch what Tom
> suggested upthread; namely, looking for a mechanism to allow
> individual datatypes to offer up a comparator function that doesn't
> require bouncing through FunctionCall2Coll().
Andres Freund writes:
> For unknown reasons the function used non chained ifs for every handled
> nodeType.
> Replacing the if chain with if; else if; ... resulted in a small
> speedup. Replacing it with a switch() in a bigger one.
Cool, but this patch is impossible to validate by eye. Could you
Hi,
For benchmarking the parser I added the above options (dim suggested this on
irc) which proved to be rather useful for me.
I added the additional rewrite option because the overhead of copying the tree
around makes the profile significantly less expressive.
I would also like an option which
Andres Freund writes:
> In many scenarios memory allocation is one of the top 3 functions showing up
> in profiles. Looking at hierarchical profiles (-fno-omit-frame-pointer) at
> least
> during parsing, planning and executor startup most of that is spent around
> the
> list API.
> Many - es
Hi List,
In many scenarios memory allocation is one of the top 3 functions showing up
in profiles. Looking at hierarchical profiles (-fno-omit-frame-pointer) at
least
during parsing, planning and executor startup most of that is spent around the
list API.
Many - especially in the parse-analyz
On 11/18/11 12:27 PM, Dimitri Fontaine wrote:
> Tom Lane writes:
>> Why do you figure that, exactly? The path of least resistance will
>> be precisely to leave everything packaged as it is, in a single
>> postgresql-contrib module. I'm pretty likely to do that myself for
>> Fedora and RHEL. Sub
Tom Lane writes:
> Why do you figure that, exactly? The path of least resistance will
> be precisely to leave everything packaged as it is, in a single
> postgresql-contrib module. I'm pretty likely to do that myself for
> Fedora and RHEL. Subdividing/rearranging contrib makes the packager's
>
Andres Freund wrote:
> When doing line-level profiles I would suggest looking at the
> instructions.
What's the best way to do that?
> I don't think cache line contention is the most likely candidate
> here. Simple cache-misses seem far more likely. In combination
> with pipeline stalls...
On Friday, November 18, 2011 08:36:59 PM Kevin Grittner wrote:
> "Kevin Grittner" wrote:
> > samples %image name symbol name
> > 4954633.6718 postgreshash_search_with_hash_value
>
> When lines like these show up in the annotated version, I'm
> impressed that we're still
Robert Haas wrote:
>> I think so. My take was that it was showing 32 of 64 *threads*
>> active -- the hyperthreading funkiness. Is there something in
>> particular you'd like me to check?
>
> Not really, just don't understand the number.
I'm having trouble resolving the vmstat numbers I got
Jeff Davis writes:
> On Fri, 2011-11-18 at 10:33 -0500, Tom Lane wrote:
>> Would it be better for them to silently transform such cases to "empty"?
> I wouldn't like to extend that to int4range(4,3), however. When the
> upper bound is less than the lower bound, it's almost certainly a
> mistake,
"Kevin Grittner" wrote:
> samples %image name symbol name
> 4954633.6718 postgreshash_search_with_hash_value
When lines like these show up in the annotated version, I'm
impressed that we're still finding gains as big as we are:
44613 0.3306 :if (segp == NU
"anara...@anarazel.de" wrote:
> Kevin Grittner schrieb:
>>samples %image name symbol name
>>9333944.9651 postgresAllocSetAlloc
>>8484764.5134 postgresbase_yyparse
>>7195153.8274 postgresSearchCatCache
> That profile looks like you ran pgben
Kevin Grittner schrieb:
>Robert Haas wrote:
>
>> Any chance you can run oprofile (on either branch, don't really
>> care) against the 32 client test and post the results?
>
>Besides the other changes we discussed, I boosted scale to 150 and
>ran at READ COMMITTED isolation level (because all
Robert Haas wrote:
> Any chance you can run oprofile (on either branch, don't really
> care) against the 32 client test and post the results?
Besides the other changes we discussed, I boosted scale to 150 and
ran at READ COMMITTED isolation level (because all threads promptly
crashed and burne
On 12 October 2011 17:26, Robert Haas wrote:
> On Wed, Oct 12, 2011 at 11:59 AM, Tom Lane wrote:
>> The place where the decision is actually somewhat hard, IMO, is where
>> you're pulling a small part of the table but significantly more than one
>> row, and the traditional best choice would be a
On Fri, Nov 18, 2011 at 10:29 PM, Robert Haas wrote:
>
> So the upside and downside of this approach is that it modifies the
> existing LWLock implementation rather than allowing multiple lock
> implementations to exist side-by-side. That means every LWLock in the
> system has access to this fun
Robert Haas wrote:
> Yeah, I'd just drop -S.
Easily done.
> Make sure to use -c N -j N with pgbench, or you'll probably not be
> able to saturate it.
Yeah, that's part of the script I copied from you.
> I've also had good luck with wal_writer_delay=20ms, although if
> you have synchronou
2011/11/18 Robert Haas :
> On Fri, Nov 18, 2011 at 6:24 AM, Pavel Stehule
> wrote:
>> CONTEXT: PL/pgSQL function "assign_rslts" line 50 at assignment (oid: 65903)
>>
>> \sf+ 65903
>
> I'm pretty unenthused by the idea of making OIDs more user-visible
> than they already are. If the message is a
On Fri, Nov 18, 2011 at 12:45 PM, Kevin Grittner
wrote:
> OK. Sorry for misunderstanding that. I haven't gotten around to a
> deep reading of the patch yet. :-( I based this on the test script
> you posted here (with slight modifications for my preferred
> directory structures):
>
> http://arc
Robert Haas wrote:
> Kevin Grittner wrote:
>>> Then again, is this a regular pgbench test or is this
>>> SELECT-only?
>>
>> SELECT-only
>
> Ah, OK. I would not expect flexlocks to help with that; Pavan's
> patch might, though.
OK. Sorry for misunderstanding that. I haven't gotten around t
On Fri, Nov 18, 2011 at 6:24 AM, Pavel Stehule wrote:
> CONTEXT: PL/pgSQL function "assign_rslts" line 50 at assignment (oid: 65903)
>
> \sf+ 65903
I'm pretty unenthused by the idea of making OIDs more user-visible
than they already are. If the message is ambiguous, we should include
argument t
On Fri, Nov 18, 2011 at 12:03 PM, Kevin Grittner
wrote:
>> Then again, is this a regular pgbench test or is this SELECT-only?
>
> SELECT-only
Ah, OK. I would not expect flexlocks to help with that; Pavan's patch
might, though.
>> Can you by any chance check top or vmstat during the 32-client
>>
"Kevin Grittner" wrote:
> We have a 32-core Intel box (4 x X7560 @ 2.27GHz) with 256 GB
> RAM.
In case anyone cares, this is the same box for which I posted STREAM
test results a while back. The PostgreSQL tests seem to peak on
this 32-core box at 64 clients, while the STREAM test of raw RAM
Robert Haas wrote:
> Then again, is this a regular pgbench test or is this SELECT-only?
SELECT-only
> Can you by any chance check top or vmstat during the 32-client
> test and see what percentage you have of user time/system
> time/idle time?
You didn't say whether you wanted master or fle
On Fri, Nov 18, 2011 at 6:26 AM, Pavan Deolasee
wrote:
> My apologies for hijacking the thread, but the work seems quite
> related, so I thought I should post here instead of starting a new
> thread.
>
> Here is a WIP patch based on the idea of having a shared Q. A process
> trying to access the s
On Fri, 2011-11-18 at 13:32 +0100, Florian Pflug wrote:
> That information, however, *is* already contained in the canonical
> functions, because those function know that (2,3) are empty as an integer
> range, but non-empty as a float range.
Very good point. Thank you.
Regards,
Jeff Davis
On Fri, 2011-11-18 at 10:33 -0500, Tom Lane wrote:
> regression=# select int4range(4,4,'(]');
> ERROR: range lower bound must be less than or equal to range upper bound
> regression=# select int4range(4,4,'()');
> ERROR: range lower bound must be less than or equal to range upper bound
>
> Would
I just found this thread:
http://archives.postgresql.org/pgsql-general/2011-11/msg00424.php
So I'll use the same workaround.
Nothing to see here, folks, move along
d
On Fri, Nov 18, 2011 at 11:17 AM, David Zwarg wrote:
> Hello,
>
> I have been working with the PostGIS developers, and I'm
On Fri, Nov 18, 2011 at 11:26 AM, Kevin Grittner
wrote:
> Robert Haas wrote:
>> Nate Boley's AMD 6128 box (which has 32 cores) and an HP Integrity
>> server (also with 32 cores).
>
>> [clear improvement with flexlock patch]
>
> Hmm. We have a 32-core Intel box (4 x X7560 @ 2.27GHz) with 256 GB
>
Robert Haas wrote:
> Nate Boley's AMD 6128 box (which has 32 cores) and an HP Integrity
> server (also with 32 cores).
> [clear improvement with flexlock patch]
Hmm. We have a 32-core Intel box (4 x X7560 @ 2.27GHz) with 256 GB
RAM. It's about a week from going into production, at which p
Hello,
I have been working with the PostGIS developers, and I'm implementing a
facility to use 'callback' functions to process cells in a raster image.
I have implemented this behind the scenes as a C function that calls a
provided sql regprocedure with OidFunctionCall*. I have been reading the
d
2011/11/18 Shigeru Hanada :
> - I couldn't see the reason why file_fdw sets ctid of sample tuples,
> though I guess it's for Vitter's random sampling algorithm. If every
> FDW must set valid ctid to sample tuples, it should be mentioned in
> document of AnalyzeForeignTable. Exporting some functio
Florian Pflug writes:
> ...This definition does not depend on any specific canonical form of ranges,
> only on the canonicalize function's ability to detect empty ranges.
Hmm, well, now that you mention it, I don't think the current canonical
functions handle empty ranges very nicely at all. The
On Fri, Nov 18, 2011 at 3:18 PM, Tom Lane wrote:
> What Thom's complaining about is that the buffer may be marked dirty
> unnecessarily, ie when there has been no actual data change.
OK, I'll patch it.
--
Simon Riggs http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7
Simon Riggs writes:
> On Fri, Nov 18, 2011 at 2:47 PM, Tom Lane wrote:
>> Well, it's expected given the current coding in the btree vacuum logic.
>> It's not clear to me why it was written like that, though.
> The code works as designed.
> _bt_delitems_vacuum() is only ever called with nitems =
On Fri, Nov 18, 2011 at 2:47 PM, Tom Lane wrote:
> Thom Brown writes:
>>> On 11 November 2011 23:28, Tom Lane wrote:
I observe that _bt_delitems_vacuum() unconditionally dirties the page
and writes a WAL record, whether it has anything to do or not; and that
if XLogStandbyInfoActi
On Fri, Nov 18, 2011 at 2:47 PM, Tom Lane wrote:
> Thom Brown writes:
>>> On 11 November 2011 23:28, Tom Lane wrote:
I observe that _bt_delitems_vacuum() unconditionally dirties the page
and writes a WAL record, whether it has anything to do or not; and that
if XLogStandbyInfoActi
Alvaro Herrera writes:
> Excerpts from Tom Lane's message of vie nov 18 11:12:32 -0300 2011:
>>> Hmm, does the win32 file need updating too?
>> I don't see HAVE_SCANDIR in there, do you?
> Well, I wonder if the win32 file needs to be hooked to the whole
> autoconf/autoheader thing somehow. I me
Thom Brown writes:
>> On 11 November 2011 23:28, Tom Lane wrote:
>>> I observe that _bt_delitems_vacuum() unconditionally dirties the page
>>> and writes a WAL record, whether it has anything to do or not; and that
>>> if XLogStandbyInfoActive() then btvacuumscan will indeed call it despite
>>> t
Alvaro Herrera writes:
> Excerpts from Peter Eisentraut's message of vie nov 18 01:34:18 -0300 2011:
>> When using verbose error messages (psql \set VERBOSITY verbose) with a
>> vpath build, you get this sort of thing:
>> LOCATION: transformColumnRef,
>> /build/buildd-postgresql-9.1_9.1.1-3-i386
Greg Smith writes:
> On 11/17/2011 03:18 PM, Peter Eisentraut wrote:
>> Who's to say that after this, the core extensions won't end up in a new
>> separate package postgresql-extensions (or similar) or might even stay
>> in postgresql-contrib, for compatibility?
> I don't know why packagers would
On Fri, Nov 18, 2011 at 08:58:30AM -0500, Robert Haas wrote:
> On Fri, Nov 18, 2011 at 8:37 AM, Noah Misch wrote:
> > I count 1/25 callers overriding nowait and 3/25 overriding missing_ok. ?So,
> > it's
> > looking like a less-common override than the callback function will come to
> > be.
>
>
Simon Riggs writes:
> On Fri, Nov 18, 2011 at 5:20 AM, Robert Haas wrote:
>> I think that we should really consider doing with this patch what Tom
>> suggested upthread; namely, looking for a mechanism to allow
>> individual datatypes to offer up a comparator function that doesn't
>> require boun
On Fri, Nov 18, 2011 at 8:37 AM, Noah Misch wrote:
> I count 1/25 callers overriding nowait and 3/25 overriding missing_ok. So,
> it's
> looking like a less-common override than the callback function will come to
> be.
Yeah, you're probably right. However, I think there's another good
reason
On 12 November 2011 00:08, Thom Brown wrote:
> On 11 November 2011 23:28, Tom Lane wrote:
>> Thom Brown writes:
>>> On 11 November 2011 00:55, Tom Lane wrote:
Thom Brown writes:
> I just noticed that the VACUUM process touches a lot of relations
> (affects mtime) but for one file
On Fri, Nov 18, 2011 at 3:53 AM, Simon Riggs wrote:
> We have no proof that we need to do this for 10 or 100 data types. We
> only currently have proof that there is gain for the most common
> types.
Well, that's kind of my point. I think this needs more work before we
decide what the best appro
On Thu, Nov 17, 2011 at 11:49:06PM -0500, Robert Haas wrote:
> On Thu, Nov 17, 2011 at 10:48 PM, Noah Misch wrote:
> > On Thu, Nov 17, 2011 at 08:59:58PM -0500, Robert Haas wrote:
> >> --- a/src/include/catalog/namespace.h
> >> +++ b/src/include/catalog/namespace.h
> >> @@ -47,9 +47,15 @@ typedef
Excerpts from Peter Eisentraut's message of vie nov 18 01:34:18 -0300 2011:
> When using verbose error messages (psql \set VERBOSITY verbose) with a
> vpath build, you get this sort of thing:
>
> ERROR: 42703: column "foo" does not exist
> LINE 1: select foo;
>^
> LOCATION: tran
The patch is in context diff format and applies cleanly.
The functionality is needed because it keeps users from circumventing
privilege restrictions, as they can currently do in this case.
There is no documentation, which I think is OK since it changes
behaviour to work as documented.
The patch
On Nov18, 2011, at 09:25 , Jeff Davis wrote:
> While thinking about range_cmp_bounds, I started to think that the way
> range_adjacent works is wrong.
>
> range_adjacent() depends on the bounds of two ranges to match up, such
> that the boundary values are equal, but one is exclusive and the other
(2011/11/18 16:25), Etsuro Fujita wrote:
> Thank you for your testing. I updated the patch according to your
> comments. Attached is the updated version of the patch.
I'd like to share result of my review even though it's not fully
finished. So far I looked from viewpoint of API design, code
fo
On Thu, Nov 17, 2011 at 10:19 AM, Pavan Deolasee
wrote:
> On Thu, Nov 17, 2011 at 10:01 AM, Robert Haas wrote:
>
>>
>> I am not convinced that that's a better API. I mean, consider
>> something like this:
>>
>> /*
>> * OK, let's do it. First let other backends know I'm in ANALYZE.
>>
Hello
I am missing a some unique identifier in exception info. I would to
use it in combination with \sf statement
I have a log
WARNING: NP_CPS: a cannot to build a RSLT object
DETAIL: dsql_text: SELECT * FROM
public._npacceptflatfile(order_nr:=to_number('O0032',
'O')::int,sequence
On 11/17/2011 03:18 PM, Peter Eisentraut wrote:
Who's to say that after this, the core extensions won't end up in a new
separate package postgresql-extensions (or similar) or might even stay
in postgresql-contrib, for compatibility?
I don't know why packagers would make an active decision t
On Fri, Nov 18, 2011 at 5:20 AM, Robert Haas wrote:
> I think that we should really consider doing with this patch what Tom
> suggested upthread; namely, looking for a mechanism to allow
> individual datatypes to offer up a comparator function that doesn't
> require bouncing through FunctionCall2
While thinking about range_cmp_bounds, I started to think that the way
range_adjacent works is wrong.
range_adjacent() depends on the bounds of two ranges to match up, such
that the boundary values are equal, but one is exclusive and the other
inclusive, and one is a lower bound and the other an u
On Thu, 2011-11-17 at 17:10 -0500, Tom Lane wrote:
> Applied, thanks. These comments aren't quite what I'd hoped for though.
> What I'm lacking is the conceptual context, ie, why is a
> less-equal-greater primitive for bounds a good thing? It seems like
> when you consider the four possible direc
75 matches
Mail list logo
