Re: [HACKERS] Unreliable "pg_ctl -w start" again

2012-01-27 Thread MauMau
From: "Tom Lane" I don't see any point in worrying about that case when you can't handle the basic case that the postmaster hasn't created postmaster.pid yet. In any case, this does nothing at all to answer the question you posed, which was how long is it reasonable to wait for the postmaster to

Re: [HACKERS] initdb and fsync

2012-01-27 Thread Noah Misch
On Fri, Jan 27, 2012 at 04:19:41PM -0800, Jeff Davis wrote: > It looks like initdb doesn't fsync all the files it creates, e.g. the > PG_VERSION file. > > While it's unlikely that it would cause any real data loss, it can be > inconvenient in some testing scenarios involving VMs. > > Thoughts? Wo

Re: [HACKERS] cursors FOR UPDATE don't return most recent row

2012-01-27 Thread Tom Lane
Alvaro Herrera writes: > I expected the FETCH to return one row, with the latest data, i.e. > (1, 3), but instead it's returning empty. This is the same thing I was complaining about in the bug #6123 thread, http://archives.postgresql.org/message-id/9698.1327266...@sss.pgh.pa.us It looks a bit t

Re: [HACKERS] Temp file missing during large pgbench data set

2012-01-27 Thread Tom Lane
Thom Brown writes: > I'm using latest git master (latest entry > 0816fad6eebddb8f1f0e21635e46625815d690b9) and I'm getting an error > when trying to create a large data set with pgbench: > LOG: could not stat file "base/pgsql_tmp/pgsql_tmp8056.0": Success > STATEMENT: alter table pgbench_accoun

Re: [HACKERS] Unreliable "pg_ctl -w start" again

2012-01-27 Thread Tom Lane
"MauMau" writes: > From: "Tom Lane" >> Looks like complete nonsense to me, if the goal is to behave sanely when >> postmaster.pid hasn't been created yet. Where do you think get_pgpid >> gets the PID from? > Yes, I understand that get_pgpid() gets the pid from postmaster.pid, which > may be th

Re: [HACKERS] Confusing EXPLAIN output in case of inherited tables

2012-01-27 Thread Tom Lane
Robert Haas writes: > It's a feature, not a bug, that we schema-qualify names when VERBOSE > is specified. That was done on purpose for the benefit of external > tools that might need this information to disambiguate which object is > being referenced. > Table *aliases*, of course, should not be

Re: [HACKERS] Unreliable "pg_ctl -w start" again

2012-01-27 Thread MauMau
From: "Tom Lane" Well, feel free to increase that duration if you want. The reason it's there is to not wait for a long time if the postmaster falls over instantly at startup, but in a non-interactive situation you might not care. Yes, just lengthening the wait duration causes unnecessary lon

Re: [HACKERS] Unreliable "pg_ctl -w start" again

2012-01-27 Thread MauMau
From: "Tom Lane" Well, feel free to increase that duration if you want. The reason it's there is to not wait for a long time if the postmaster falls over instantly at startup, but in a non-interactive situation you might not care. Yes, just lengthening the wait duration causes unnecessary lon

[HACKERS] Temp file missing during large pgbench data set

2012-01-27 Thread Thom Brown
Hi, I'm using latest git master (latest entry 0816fad6eebddb8f1f0e21635e46625815d690b9) and I'm getting an error when trying to create a large data set with pgbench: thom@swift:~/Development$ createdb pgbench thom@swift:~/Development$ pgbench -i -s 100 pgbench NOTICE: table "pgbench_branches" do

Re: [HACKERS] CLOG contention, part 2

2012-01-27 Thread Jeff Janes
On Fri, Jan 27, 2012 at 3:16 PM, Merlin Moncure wrote: > On Fri, Jan 27, 2012 at 4:05 PM, Jeff Janes wrote: >> Also, I think the general approach is wrong.  The only reason to have >> these pages in shared memory is that we can control access to them to >> prevent write/write and read/write corru

[HACKERS] initdb and fsync

2012-01-27 Thread Jeff Davis
It looks like initdb doesn't fsync all the files it creates, e.g. the PG_VERSION file. While it's unlikely that it would cause any real data loss, it can be inconvenient in some testing scenarios involving VMs. Thoughts? Would a patch to add a few fsync calls to initdb be accepted? Is a platform-

Re: [HACKERS] Configuring Postgres to Add A New Source File

2012-01-27 Thread Tareq Aljabban
Indeed, I'm a beginner in "Make", but I read few tutorials and was able to do what I wanted outside of PG using a simple make file. Now, when moving to PG, I found the Make structure much more complicated and didn't know where to add my configuration. I'm looking only for this file to run in PG (th

Re: [HACKERS] CLOG contention, part 2

2012-01-27 Thread Merlin Moncure
On Fri, Jan 27, 2012 at 4:05 PM, Jeff Janes wrote: > Also, I think the general approach is wrong.  The only reason to have > these pages in shared memory is that we can control access to them to > prevent write/write and read/write corruption.  Since these pages are > never written, they don't nee

Re: [HACKERS] Dry-run mode for pg_archivecleanup

2012-01-27 Thread Alvaro Herrera
Excerpts from Josh Kupershmidt's message of vie ene 27 19:43:51 -0300 2012: > On Fri, Jan 27, 2012 at 9:47 AM, Robert Haas wrote: > > This email thread seems to have trailed off without reaching a > > conclusion.  The patch is marked as Waiting on Author in the > > CommitFest application, but I'

Re: [HACKERS] Dry-run mode for pg_archivecleanup

2012-01-27 Thread Josh Kupershmidt
On Fri, Jan 27, 2012 at 9:47 AM, Robert Haas wrote: > On Sun, Jan 15, 2012 at 5:05 PM, Josh Kupershmidt wrote: >> On Sun, Jan 15, 2012 at 3:02 PM, Gabriele Bartolini >> wrote: >> >>> My actual intention was to have the filename as output of the command, in >>> order to easily "pipe" it to anothe

Re: [HACKERS] CLOG contention, part 2

2012-01-27 Thread Jeff Janes
On Sat, Jan 21, 2012 at 7:31 AM, Simon Riggs wrote: > > Yes, it was. Sorry about that. New version attached, retesting while > you read this. In my hands I could never get this patch to do anything. The new cache was never used. I think that that was because RecentXminPageno never budged from -

Re: [HACKERS] Simulating Clog Contention

2012-01-27 Thread Jeff Janes
On Thu, Jan 12, 2012 at 4:31 AM, Simon Riggs wrote: > The following patch adds a pgbench option -I to load data using > INSERTs, so that we can begin benchmark testing with rows that have > large numbers of distinct un-hinted transaction ids. With a database > pre-created using this we will be be

Re: [HACKERS] 16-bit page checksums for 9.2

2012-01-27 Thread Dan Scales
The advantage of putting the checksum calculation in smgrwrite() (or mdwrite()) is that it catches a bunch of page writes that don't go through the buffer pool (see calls to smgrwrite() in nbtree.c, nbtsort.c, spginsert.c) Also, I missed this before: don't you want to add the checksum calculati

Re: [HACKERS] Multithread Query Planner

2012-01-27 Thread Pierre C
Not to mention palloc, another extremely fundamental and non-reentrant subsystem. Possibly we could work on making all that stuff re-entrant, but it would be a huge amount of work for a distant and uncertain payoff. Right. I think it makes more sense to try to get parallelism working first

Re: [HACKERS] pg_statistic, lack of documentation

2012-01-27 Thread Sergey Konoplev
On Fri, Jan 27, 2012 at 9:14 PM, Robert Haas wrote: > On Sat, Jan 14, 2012 at 7:34 AM, Sergey Konoplev wrote: > I've added a note to that effect to the documentation for ANALYZE, > which seems like a more appropriate place than the pg_statistic > documentation. Thank you. > > -- > Robert Haas >

Re: [HACKERS] rewriteheap.c bug: toast rows don't get XIDs matching their parents

2012-01-27 Thread Robert Haas
On Thu, Jan 12, 2012 at 4:50 PM, Tom Lane wrote: > While working on bug #6393 I was reminded of the truth of $SUBJECT: any > rows inserted into the new toast table will have the xmin of the CLUSTER > or VACUUM FULL operation, and invalid xmax, whereas their parent heap > rows will have xmin/xmax c

Re: [HACKERS] Intermittent regression test failures from index-only plan changes

2012-01-27 Thread Robert Haas
On Sat, Jan 7, 2012 at 12:30 PM, Tom Lane wrote: >> I feel like this is a trick question, but I'll ask anyway: Can't we >> just ignore ANALYZE? > > AFAICS, no.  ANALYZE will run user-defined code: not only user-supplied > stats collection functions, but user-defined index expressions.  We > cannot

Re: [HACKERS] Unreliable "pg_ctl -w start" again

2012-01-27 Thread Tom Lane
"MauMau" writes: > In PostgreSQL 9.1, the wait processing is largely modified. However, the > same assumption seems to still remain, though the duration is 5 seconds. 5 > seconds of wait is probably insufficient for my case. I think no fixed > duration is appropriate. Well, feel free to increa

Re: [HACKERS] get_fn_expr_argtype() vs. internal calls

2012-01-27 Thread Robert Haas
On Thu, Dec 29, 2011 at 4:17 PM, Noah Misch wrote: > We document that a polymorphic C-language function may identify the concrete > data type of each argument using calls to get_fn_expr_argtype().  That relies > on FmgrInfo.fn_expr, which only the executor sets.  Calls of internal origin, > by way

Re: [HACKERS] Confusing EXPLAIN output in case of inherited tables

2012-01-27 Thread Robert Haas
On Wed, Jan 11, 2012 at 6:43 AM, Ashutosh Bapat wrote: > Hi, > After running regression, I ran EXPLAIN on one of the queries in regression > (test create_misc) and got following output > regression=# explain verbose select * into table ramp from road where name ~ > '.*Ramp'; >

Re: [HACKERS] patch for parallel pg_dump

2012-01-27 Thread Robert Haas
On Fri, Jan 27, 2012 at 11:53 AM, Heikki Linnakangas wrote: > If the master process keeps the locks it acquires in the beginning, you > could fall back to dumping those tables where the child lock fails using the > master connection. Hmm, that's a thought. Another idea I just had is to allow a t

Re: [HACKERS] pg_statistic, lack of documentation

2012-01-27 Thread Robert Haas
On Sat, Jan 14, 2012 at 7:34 AM, Sergey Konoplev wrote: > Hi, > > http://www.postgresql.org/docs/9.1/interactive/catalog-pg-statistic.html > > It specifies that entries are created by ANALYZE, but does not mention > that if a table is empty the entry for it is not created. The actual behavior, in

Re: [HACKERS] TS: Limited cover density ranking

2012-01-27 Thread karavelov
- Цитат от karave...@mail.bg, на 27.01.2012 в 18:48 - > - Цитат от Sushant Sinha (sushant...@gmail.com), на 27.01.2012 в 18:32 > - > >> The rank counts 1/coversize. So bigger covers will not have much impact >> anyway. What is the need of the patch? >> >> -Sushant. >> > > If yo

Re: [HACKERS] patch for parallel pg_dump

2012-01-27 Thread Heikki Linnakangas
On 27.01.2012 18:46, Robert Haas wrote: On Sun, Jan 15, 2012 at 1:01 PM, Joachim Wieland wrote: In parallel restore, the master closes its own connection to the database before forking of worker processes, just as it does now. In parallel dump however, we need to hold the masters connection ope

Re: [HACKERS] TS: Limited cover density ranking

2012-01-27 Thread karavelov
- Цитат от Sushant Sinha (sushant...@gmail.com), на 27.01.2012 в 18:32 - > The rank counts 1/coversize. So bigger covers will not have much impact > anyway. What is the need of the patch? > > -Sushant. > If you want to find only combinations of words that are close one to another, with

Re: [HACKERS] patch for parallel pg_dump

2012-01-27 Thread Robert Haas
On Sun, Jan 15, 2012 at 1:01 PM, Joachim Wieland wrote: > So this is the parallel pg_dump patch, generalizing the existing > parallel restore and allowing parallel dumps for the directory archive > format, the patch works on Windows and Unix. It seems a little unfortunate that we are using thread

Re: [HACKERS] TS: Limited cover density ranking

2012-01-27 Thread Sushant Sinha
The rank counts 1/coversize. So bigger covers will not have much impact anyway. What is the need of the patch? -Sushant. On Fri, 2012-01-27 at 18:06 +0200, karave...@mail.bg wrote: > Hello, > > I have developed a variation of cover density ranking functions that > counts only covers that are le

Re: [HACKERS] patch for parallel pg_dump

2012-01-27 Thread Robert Haas
On Fri, Jan 27, 2012 at 10:58 AM, Robert Haas wrote: >> It's not clear to me why fmtQualifiedId needs to move to dumputils.c. >> The way you have it, fmtQualifiedId() is now with fmtId(), but no >> longer with fmtCopyColumnList(), the only other similarly named >> function in that directory.  That

Re: [HACKERS] TS: Limited cover density ranking

2012-01-27 Thread karavelov
And here is the patch, that I forgot to attach > Hello, > > I have developed a variation of cover density ranking functions that counts > only covers that are lesser than a specified limit. It is useful for finding > combinations of terms that appear nearby one another. Here is an example of >

[HACKERS] TS: Limited cover density ranking

2012-01-27 Thread karavelov
Hello, I have developed a variation of cover density ranking functions that counts only covers that are lesser than a specified limit. It is useful for finding combinations of terms that appear nearby one another. Here is an example of usage: -- normal cover density ranking : not changed luben

Re: [HACKERS] patch for parallel pg_dump

2012-01-27 Thread Robert Haas
On Fri, Jan 27, 2012 at 10:57 AM, Robert Haas wrote: > On Sun, Jan 15, 2012 at 1:01 PM, Joachim Wieland wrote: >> So this is the parallel pg_dump patch, generalizing the existing >> parallel restore and allowing parallel dumps for the directory archive >> format, the patch works on Windows and Un

Re: [HACKERS] patch for parallel pg_dump

2012-01-27 Thread Robert Haas
On Sun, Jan 15, 2012 at 1:01 PM, Joachim Wieland wrote: > So this is the parallel pg_dump patch, generalizing the existing > parallel restore and allowing parallel dumps for the directory archive > format, the patch works on Windows and Unix. This patch introduces a large amount of notational chu

Re: [HACKERS] Speed dblink using alternate libpq tuple storage

2012-01-27 Thread Marko Kreen
On Fri, Jan 27, 2012 at 09:35:04AM -0600, Merlin Moncure wrote: > On Fri, Jan 27, 2012 at 2:57 AM, Kyotaro HORIGUCHI > > - The meaning of PGresAttValue is changed. The field 'value' now > >  contains a value withOUT terminating zero. This change seems to > >  have no effect on any other portion wit

[HACKERS] Unreliable "pg_ctl -w start" again

2012-01-27 Thread MauMau
Hello, Last year, I asked for your opinions about how to fix the bug of unreliable "pg_ctl -w start", as in the thread: http://archives.postgresql.org/pgsql-hackers/2011-05/msg01407.php The phenomenon was that "pg_ctl -w start" did not return for 60 seconds when postgresql.conf contained a w

Re: [HACKERS] Speed dblink using alternate libpq tuple storage

2012-01-27 Thread Marko Kreen
On Fri, Jan 27, 2012 at 05:57:01PM +0900, Kyotaro HORIGUCHI wrote: > Hello, This is a new version of the patch formerly known as > 'alternative storage for libpq'. > > - Changed the concept to 'Alternative Row Processor' from > 'Storage handler'. Symbol names are also changed. > > - Callback fu

Re: [HACKERS] Speed dblink using alternate libpq tuple storage

2012-01-27 Thread Merlin Moncure
On Fri, Jan 27, 2012 at 2:57 AM, Kyotaro HORIGUCHI wrote: > Hello, This is a new version of the patch formerly known as > 'alternative storage for libpq'. I took a quick look at the patch and the docs. Looks good and agree with rationale and implementation. I see you covered the pqsetvalue cas

Re: [HACKERS] Caching for stable expressions with constant arguments v6

2012-01-27 Thread Robert Haas
On Mon, Jan 16, 2012 at 12:06 PM, Marti Raudsepp wrote: > Here's v6 of my expression caching patch. The patch is not attached. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To ma

Re: [HACKERS] Dry-run mode for pg_archivecleanup

2012-01-27 Thread Robert Haas
On Sun, Jan 15, 2012 at 5:05 PM, Josh Kupershmidt wrote: > On Sun, Jan 15, 2012 at 3:02 PM, Gabriele Bartolini > wrote: > >> My actual intention was to have the filename as output of the command, in >> order to easily "pipe" it to another script. Hence my first choice was to >> use the stdout cha

Re: [HACKERS] Progress on fast path sorting, btree index creation time

2012-01-27 Thread Robert Haas
On Fri, Jan 27, 2012 at 9:27 AM, Peter Geoghegan wrote: > Well, I don't think it's all that subjective - it's more the case that > it is just difficult, or it gets that way as you consider more > specialisations. Sure it's subjective. Two well-meaning people could have different opinions without

Re: [HACKERS] Progress on fast path sorting, btree index creation time

2012-01-27 Thread Peter Geoghegan
Uh, obviously I meant causal relationship and not correlation. On 27 January 2012 13:37, Robert Haas wrote: > I completely agree.  So the point is that, when faced a patch that > adds an atypically large number of CPU instructions, we ought to ask > ourselves whether those instructions are pullin

Re: [HACKERS] Group commit, revised

2012-01-27 Thread Robert Haas
On Fri, Jan 27, 2012 at 8:35 AM, Heikki Linnakangas wrote: > Yeah, we have to be careful with any overhead in there, it can be a hot > spot. I wouldn't expect any measurable difference from the above, though. > Could I ask you to rerun the pgbench tests you did recently with this patch? > Or can y

Re: [HACKERS] Progress on fast path sorting, btree index creation time

2012-01-27 Thread Robert Haas
On Thu, Jan 26, 2012 at 11:36 PM, Peter Geoghegan wrote: >> I'm not surprised that you weren't able to measure a performance >> regression from the binary bloat.  Any such regression is bound to be >> very small and probably quite difficult to notice most of the time; >> it's really the cumulative

Re: [HACKERS] Group commit, revised

2012-01-27 Thread Heikki Linnakangas
On 26.01.2012 04:10, Robert Haas wrote: On Wed, Jan 25, 2012 at 3:11 AM, Heikki Linnakangas wrote: Attached is a patch to do that. It adds a new mode to LWLockConditionalAcquire(), LW_EXCLUSIVE_BUT_WAIT. If the lock is free, it is acquired and the function returns immediately. However, unlike

Re: [HACKERS] 16-bit page checksums for 9.2

2012-01-27 Thread Robert Haas
On Thu, Jan 26, 2012 at 7:01 PM, Dan Scales wrote: > I'm not sure why you moved the checksum calculation (PageSetVerificationInfo) > to mdwrite() rather than smgrwrite().  If there were every another storage > backend, it would have to duplicate the checksum check, right?  Is there a > disadvan

Re: [HACKERS] patch: ALTER TABLE IF EXISTS

2012-01-27 Thread Heikki Linnakangas
On 27.01.2012 11:57, Dean Rasheed wrote: I just noticed this copy-and-paste error in the ALTER FOREIGN TABLE docs: IF EXISTS: Do not throw an error if the sequence does not exist. A notice is issued in this case. That should be "foreign table" not "sequence". Thanks, fixed. --

Re: [HACKERS] Inline Extension

2012-01-27 Thread Cédric Villemain
>> So I'm going to prepare the next version of the patch with this design: >> >> - in catalog extension scripts for inline extension >> >>   pg_extension_script(extoid, oldversion, version, script) >> >>   oldversion is null when create extension is used >>   unless when using the create extension

Re: [HACKERS] patch: ALTER TABLE IF EXISTS

2012-01-27 Thread Dean Rasheed
On 23 January 2012 20:14, Pavel Stehule wrote: > Hello > > 2012/1/23 Robert Haas : >> On Tue, Jan 3, 2012 at 2:49 PM, Pavel Stehule >> wrote: >>> jup, we can continue in enhancing step by step. >>> >>> I change a patch and now ALTER TABLE, ALTER INDEX, ALTER SEQUENCE and >>> ALTER VIEW has IF EX

Re: [HACKERS] Inline Extension

2012-01-27 Thread Peter van Hardenberg
On Thu, Jan 26, 2012 at 3:48 PM, David E. Wheeler wrote: > On Jan 26, 2012, at 9:40 AM, Dimitri Fontaine wrote: > >> Not for 9.2, but I can't help thinking that if we could manage to host >> the .so module itself in the catalogs, we could solve updating it in a >> transactional way and more import

Re: [HACKERS] Speed dblink using alternate libpq tuple storage

2012-01-27 Thread Kyotaro HORIGUCHI
Hello, This is a new version of the patch formerly known as 'alternative storage for libpq'. - Changed the concept to 'Alternative Row Processor' from 'Storage handler'. Symbol names are also changed. - Callback function is modified following to the comment. - From the restriction of time, I d