Re: [HACKERS] A thought on Index Organized Tables

2010-02-28 Thread Gokulakannan Somasundaram
> > The transaction information on tuples take 18 bytes plus several info > bits. It's possible just storing a subset of that would be useful but > it's unclear. And I think it would complicate the code if it had to > sometimes fetch the heap tuple to get the rest and sometimes doesn't. > Visibili

Re: [HACKERS] A thought on Index Organized Tables

2010-02-28 Thread Gokulakannan Somasundaram
> No, it is not the same thing. Updating index snapshots requires being > able to *re-find* a previously made index entry for the current row. > And it has to be done 100% reliably. The worst that happens if an index > entry is not found when it should be during a uniqueness check is that > the u

Re: [HACKERS] psql with "Function Type" in \df

2010-02-28 Thread David Fetter
On Thu, Feb 25, 2010 at 07:20:58PM -0500, Bruce Momjian wrote: > > Did we ever get tab completion support for these backslash commands? Nope :/ Not sure if I'll be able to get to it this week, either. Cheers, David. -- David Fetter http://fetter.org/ Phone: +1 415 235 3778 AIM: dfetter666 Y

Re: [HACKERS] remove contrib/xml2

2010-02-28 Thread Tom Lane
Andrew Dunstan writes: > Tom Lane wrote: >> ... The reason for that behavior is that xpath_table runs through >> the XPATH_NODESET results generated by the various XPaths and dumps the >> k'th one of each into the k'th output row generated for the current >> input row. > Damn that's ugly. Yup :-

Re: [HACKERS] dedebugging and a functions that just don't work on debian flavour

2010-02-28 Thread Robert Haas
On Sun, Feb 28, 2010 at 8:43 AM, Ivan Sergio Borgonovo wrote: > I've no idea what to do next other than asking if someone can give a > look to the code and check my comprehension of what a tsquery should > be in memory. Have you tried valgrind? ...Robert -- Sent via pgsql-hackers mailing list

Re: [HACKERS] Could we do pgindent on just utils/adt/xml.c in the 8.3 branch?

2010-02-28 Thread Tom Lane
Bruce Momjian writes: > Sure, I can easily do that, but I am not sure how to generate a typedef > list for that release, except to use the old typedef collection method > for that file. Is that OK? One of the things I noticed while patching was that the 8.3 typedef list was missing xmlChar and a

Re: [HACKERS] Lock Wait Statistics (next commitfest)

2010-02-28 Thread Robert Haas
On Sun, Feb 28, 2010 at 1:06 AM, Mark Kirkwood wrote: > Robert Haas wrote: >> This might be my fault, so I apologize for killing your enthusiasm.  I >> think when I get wrapped up in a CommitFest (and especially during the >> second half) I get wound up in determining whether or not things are >>

[HACKERS] Re: Could we do pgindent on just utils/adt/xml.c in the 8.3 branch?

2010-02-28 Thread Bruce Momjian
Tom Lane wrote: > I just committed a backpatch of > http://archives.postgresql.org/pgsql-committers/2009-05/msg00229.php > to 8.3. As in that patch, I did not try to manually reindent the code > that's now wrapped in PG_TRY blocks. It might be a good idea to run > pgindent on just that one file s

Re: [HACKERS] Re: Hot Standby query cancellation and Streaming Replication integration

2010-02-28 Thread Robert Haas
On Sun, Feb 28, 2010 at 5:38 PM, Josh Berkus wrote: > Greg, Joachim, > >> As I see it, the main technical obstacle here is that a subset of a >> feature already on the SR roadmap needs to get built earlier than >> expected to pull this off.  I don't know about Tom, but I have no >> expectation it'

Re: [HACKERS] Re: Hot Standby query cancellation and Streaming Replication integration

2010-02-28 Thread Greg Smith
Josh Berkus wrote: Well, we could throw this on the user if we could get them some information on how to calculate that number. For example, some way for them to calculate the number of XIDs per minute via a query, and then set vacuum_defer_cleanup_age appropriately on the master. Sure, it's cl

Re: [HACKERS] Avoiding bad prepared-statement plans.

2010-02-28 Thread Robert Haas
On Sun, Feb 28, 2010 at 2:52 AM, Mark Mielke wrote: > On 02/27/2010 11:20 PM, Craig Ringer wrote: >> >> Essentially, you have: >> >> 1) People preparing statements to save on parse+plan time; and >> 2) People preparing statements to get convenenient param placement. >> >> I suspect that most of (1

Re: [HACKERS] Avoiding bad prepared-statement plans.

2010-02-28 Thread Robert Haas
On Sat, Feb 27, 2010 at 11:22 PM, Tom Lane wrote: > Robert Haas writes: >> On Fri, Feb 26, 2010 at 7:03 PM, Tom Lane wrote: >>> Wouldn't it be better if it just did the right thing automatically? >>> >>> The sort of heuristic I'm envisioning would essentially do "replan every >>> time" for some

[HACKERS] Could we do pgindent on just utils/adt/xml.c in the 8.3 branch?

2010-02-28 Thread Tom Lane
I just committed a backpatch of http://archives.postgresql.org/pgsql-committers/2009-05/msg00229.php to 8.3. As in that patch, I did not try to manually reindent the code that's now wrapped in PG_TRY blocks. It might be a good idea to run pgindent on just that one file so that it's spacing-consis

[HACKERS] contrib/xml2 vs core xml, more issues

2010-02-28 Thread Tom Lane
Some other things struck me on further code-reading. These are not problems if core xml functions and contrib/xml2 functions are used independently, but they would be issues if one is called within the other --- and both modules have functions that invoke arbitrary SQL inside their execution, so t

Re: [HACKERS] contrib/xml2 vs core xml in 8.3

2010-02-28 Thread David Fetter
On Sun, Feb 28, 2010 at 06:57:33PM -0500, Tom Lane wrote: > So I started to work on back-patching the no-pallocs change in xml2. > It seems to work fine in 8.4, but it crashes in 8.3. The reason is > that we never back-patched the 8.4 change to make utils/adt/xml.c > not change the libxml2 memory

[HACKERS] contrib/xml2 vs core xml in 8.3

2010-02-28 Thread Tom Lane
So I started to work on back-patching the no-pallocs change in xml2. It seems to work fine in 8.4, but it crashes in 8.3. The reason is that we never back-patched the 8.4 change to make utils/adt/xml.c not change the libxml2 memory allocation functions. That means that if you try to intermix core

Re: [HACKERS] remove contrib/xml2

2010-02-28 Thread Andrew Dunstan
Tom Lane wrote: I believe I have fixed all the reported crashes in contrib/xml2. Yay! Well done! That at least removes any possibly urgency about removing the module. However there is still this issue pointed out by Robert: CREATE TABLE xpath_test (id integer NOT NULL, t xml); INS

Re: [HACKERS] Re: Hot Standby query cancellation and Streaming Replication integration

2010-02-28 Thread Josh Berkus
Greg, Joachim, > As I see it, the main technical obstacle here is that a subset of a > feature already on the SR roadmap needs to get built earlier than > expected to pull this off. I don't know about Tom, but I have no > expectation it's possible for me to get up to speed on that code fast > eno

Re: [HACKERS] Hot Standby query cancellation and Streaming Replication integration

2010-02-28 Thread Simon Riggs
On Fri, 2010-02-26 at 03:33 -0500, Greg Smith wrote: > I really hope this discussion can say focused on if and how it's > possible to improve this area, with the goal being to deliver a product > everyone can be proud of with the full feature set that makes this next > release a killer one. The

Re: [HACKERS] Hot Standby query cancellation and Streaming Replication integration

2010-02-28 Thread Simon Riggs
On Fri, 2010-02-26 at 16:44 -0500, Tom Lane wrote: > Greg Stark writes: > > On Fri, Feb 26, 2010 at 9:19 PM, Tom Lane wrote: > >> There's *definitely* not going to be enough information in the WAL > >> stream coming from a master that doesn't think it has HS slaves. > >> We can't afford to record

Re: [HACKERS] remove contrib/xml2

2010-02-28 Thread Tom Lane
I believe I have fixed all the reported crashes in contrib/xml2. However there is still this issue pointed out by Robert: > CREATE TABLE xpath_test (id integer NOT NULL, t xml); > INSERT INTO xpath_test VALUES (1, ' b="oops"/>'); > SELECT * FROM xpath_table('id', 't', 'xpath_test', > '/rowlist/row

Re: [HACKERS] Re: Hot Standby query cancellation and Streaming Replication integration

2010-02-28 Thread Joachim Wieland
On Sun, Feb 28, 2010 at 8:47 PM, Josh Berkus wrote: > 1) Automated retry of cancelled queries on the slave.  I have no idea > how hard this would be to implement, but it makes the difference between > writing lots of exception-handling code for slave connections > (unacceptable) to just slow respo

Re: [HACKERS] contrib/xml2 pfree bug

2010-02-28 Thread Andrew Dunstan
Andrew Dunstan wrote: Almost any XML will do for the first param. e.g.: It looks like you need to make sure the XML library is called first to induce the crash, so before doing what's below, do: select query_to_xml('select 1 as x',true,false,''): select xslt_process( query_to_xml

Re: [HACKERS] contrib/xml2 pfree bug

2010-02-28 Thread Andrew Dunstan
Tom Lane wrote: Andrew Dunstan writes: Didn't we just clean up a mess in our XML handling to do with memory handlers? It looks like contrib/xml2 might have similar problems. BTW, I couldn't duplicate this because I don't know what cb_ob_invoice_xml(1,1) refers to. Can you provide a

Re: [HACKERS] Re: Hot Standby query cancellation and Streaming Replication integration

2010-02-28 Thread Dimitri Fontaine
Josh Berkus writes: > 2) A more usable vacuum_defer_cleanup_age. If it was feasible for a > user to configure the master to not vacuum records less than, say, 5 > minutes dead, then that would again offer the choice to the user of > slightly degraded performance on the master (acceptable) vs. lot

Re: [HACKERS] Re: Hot Standby query cancellation and Streaming Replication integration

2010-02-28 Thread Greg Smith
Josh Berkus wrote: First, from the nature of the arguments, we need to eventually have both versions of SR: delay-based and xmin-pub. And it would be fantastic if Greg Smith and Tom Lane could work on xmin-pub to see if we can get it ready as well. As I see it, the main technical obstacle h

[HACKERS] full text search index scan query plan changed in 8.4.2?

2010-02-28 Thread xu fei
Hi, hackers: I am testing the index used by full text search recently. I have install 8.3.9 and 8.4.2 separately. In 8.3.9, the query plan is like: postgres=# explain SELECT s.name as source , t.name as target FROM element as s, element as t WHERE to_tsvector('testcfg',s.name)  @@ to_tsquery

Re: [HACKERS] Re: Hot Standby query cancellation and Streaming Replication integration

2010-02-28 Thread Greg Smith
Joachim Wieland wrote: Instead, I assume that most people who will grab 9.0 and use HS+SR do already have a database with a certain query profile. Now with HS+SR they will try to put the most costly and longest read-only queries to the standby but in the end will run the same number of queries wi

Re: [HACKERS] contrib/xml2 pfree bug

2010-02-28 Thread Tom Lane
Andrew Dunstan writes: > Didn't we just clean up a mess in our XML handling to do with memory > handlers? It looks like contrib/xml2 might have similar problems. BTW, I couldn't duplicate this because I don't know what cb_ob_invoice_xml(1,1) refers to. Can you provide a self-contained example?

Re: [HACKERS] Simple fix for contrib/xml2

2010-02-28 Thread Tom Lane
Bruce Momjian writes: > FYI, I created a web page of all the open xml items: > http://momjian.us/cgi-bin/pgsql/mbox?xml Oh, thanks, I was just about to go trawling through the archives to verify all the reported failure cases really do get fixed this way. I don't think you got em all thoug

Re: [HACKERS] Anyone know if Alvaro is OK?

2010-02-28 Thread Ron Mayer
Jaime Casanova wrote: > At Saturday, 02/27/2010 on 4:21 pm "Marc G. Fournier" wrote: >> On Sat, Feb 27, 2010 at 10:45 PM, Alvaro Herrera >> wrote: >>> Is there a higher then normal amount of earthquakes happening recently? >>> >> Re: the more frequent earthquakes, yeah I was thinking the same t

Re: [HACKERS] Simple fix for contrib/xml2

2010-02-28 Thread Bruce Momjian
Tom Lane wrote: > I'm beginning to think nobody is going to step up and fix contrib/xml2, > so I had a look at it myself. It strikes me that there is a pretty > simple fix for it: just get rid of the attempt to manage libxml memory > via palloc, and don't do anything else. The implication of this

Re: [HACKERS] Re: Hot Standby query cancellation and Streaming Replication integration

2010-02-28 Thread Josh Berkus
All, First, from the nature of the arguments, we need to eventually have both versions of SR: delay-based and xmin-pub. And it would be fantastic if Greg Smith and Tom Lane could work on xmin-pub to see if we can get it ready as well. I also think, based on the discussion and Greg's test case, t

Re: [HACKERS] [COMMITTERS] Re: pgsql: Speed up CREATE DATABASE by deferring the fsyncs until after

2010-02-28 Thread Tom Lane
Greg Stark writes: > So I'm thinking of something like this. > Ignore ESDIR when opening a directory (and return immediately) > and ignore EBADF when trying to fsync a directory. Seems reasonable, but get rid of the comment "However we can't do this just yet, it has portability issues"; and you'v

[HACKERS] Simple fix for contrib/xml2

2010-02-28 Thread Tom Lane
I'm beginning to think nobody is going to step up and fix contrib/xml2, so I had a look at it myself. It strikes me that there is a pretty simple fix for it: just get rid of the attempt to manage libxml memory via palloc, and don't do anything else. The implication of this will be that if an erro

Re: [HACKERS] [COMMITTERS] Re: pgsql: Speed up CREATE DATABASE by deferring the fsyncs until after

2010-02-28 Thread Greg Stark
On Tue, Feb 23, 2010 at 3:38 PM, Tom Lane wrote: >> The plan I was thinking of was to pass a flag indicating if it's a >> directory to fsync_fname() and open it RD_ONLY if it's a directory and >> RDRW if it's a file. Then ignore any error returns (or at least EBADF >> and EINVAL) iff the flag indi

Re: [HACKERS] Re: Hot Standby query cancellation and Streaming Replication integration

2010-02-28 Thread Joachim Wieland
On Sun, Feb 28, 2010 at 2:54 PM, Greg Stark wrote: > Really? I think we get lots of suprised wows from the field from the > idea that a long-running read-only query can cause your database to > bloat. I think the only reason that's obvious to us is that we've been > grappling with that problem for

Re: [HACKERS] A thought on Index Organized Tables

2010-02-28 Thread Tom Lane
Gokulakannan Somasundaram writes: > a) We are already going from table to index to do unique checks. This is the > same thing, which we will do to go and update the snapshot in the indexes. No, it is not the same thing. Updating index snapshots requires being able to *re-find* a previously made

Re: [HACKERS] [GENERAL] trouble with to_char('L')

2010-02-28 Thread Bruce Momjian
Bruce Momjian wrote: > Hiroshi Inoue wrote: > > Bruce Momjian wrote: > > > Hiroshi Inoue wrote: > > >> Bruce Momjian wrote: > > >>> Where are we on this issue? > > >> Oops I forgot it completely. > > >> I have a little improved version and would post it tonight. > > > > > > Ah, very good. Thanks.

Re: [HACKERS] Avoiding bad prepared-statement plans.

2010-02-28 Thread Greg Stark
On Fri, Feb 26, 2010 at 4:01 AM, Robert Haas wrote: >> It's not going to be easier to implement.  Yeah, it would be easy to >> provide a global switch via a GUC setting, but that's not going to be >> helpful, because this is the sort of thing that really needs to be >> managed per-query.  Almost a

Re: [HACKERS] A thought on Index Organized Tables

2010-02-28 Thread Greg Stark
On Sun, Feb 28, 2010 at 6:02 AM, Gokulakannan Somasundaram wrote: > So just with a addition of 8 bytes per tuple, we can have the snapshot > stored with the index. Can someone please comment on this? The transaction information on tuples take 18 bytes plus several info bits. It's possible just st

Re: [HACKERS] Re: Hot Standby query cancellation and Streaming Replication integration

2010-02-28 Thread Greg Stark
On Sun, Feb 28, 2010 at 6:07 AM, Greg Smith wrote: > Not forced to--have the option of.  There are obviously workloads where you > wouldn't want this.  At the same time, I think there are some pretty common > ones people are going to expect HS+SR to work on transparently where this > would obvious

[HACKERS] dedebugging and a functions that just don't work on debian flavour

2010-02-28 Thread Ivan Sergio Borgonovo
I wrote a function that turns tsvectors into tsqueries. I can't see any evident error and actually the function return the expected results but just for a strange set of queries. Otherwise it causes tsqueryout to exit with errors when the result is returned. It seems that tsqueryout is running bey

Re: [HACKERS] Anyone know if Alvaro is OK?

2010-02-28 Thread Boszormenyi Zoltan
It must be that someone has dropped a small amount of cheese into a lactose-intolerant volcano god... Marc G. Fournier írta: > > Is there a higher then normal amount of earthquakes happening > recently? haiti, japan just had one for 6.9, there was apparently one > in illinos a few weeks back, one

Re: [HACKERS] Anyone know if Alvaro is OK?

2010-02-28 Thread Devrim GÜNDÜZ
I'm happy that you and your family are fine, Alvaro. Same from my wife, too. On Sat, 2010-02-27 at 19:45 -0800, Alvaro Herrera wrote: > Hi. We're out of town right now, and it seems I can't get to my home > machine (probably just a loose cable). Our building was shaken badly > enough that we'll

Re: [HACKERS] Avoiding bad prepared-statement plans.

2010-02-28 Thread Dimitri Fontaine
Craig Ringer writes: > 1) People preparing statements to save on parse+plan time; and > 2) People preparing statements to get convenenient param placement. > > I suspect that most of (1) also want (2), but many of (2) don't care much > about (1) and are just preparing statements for sql-injection