Re: [HACKERS] REFRESH MATERIALIZED VIEW command in PL block hitting Assert

2013-04-23 Thread Michael Paquier
On Tue, Apr 23, 2013 at 11:05 PM, Andres Freund wrote: > On 2013-04-23 19:33:24 +0530, Jeevan Chalke wrote: > > On Tue, Apr 23, 2013 at 7:01 PM, Merlin Moncure > wrote: > > > > > On Tue, Apr 23, 2013 at 7:18 AM, Jeevan Chalke > > > wrote: > > > > Hi Tom, > > > > > > > > Since we are close to rel

[HACKERS] 9.3 release notes suggestions

2013-04-23 Thread Bruce Momjian
Thanks for the many suggestions on improving the 9.3 release notes. There were many ideas I would have never thought of. Please keep the suggestions coming. -- Bruce Momjian http://momjian.us EnterpriseDB http://enterprisedb.com + It's impossible for e

Re: [HACKERS] 9.3 Beta1 status report

2013-04-23 Thread Bruce Momjian
On Tue, Apr 23, 2013 at 06:56:34PM -0300, Alvaro Herrera wrote: > Bruce Momjian wrote: > > On Tue, Apr 23, 2013 at 05:04:15PM -0400, Bruce Momjian wrote: > > > > Do we usually repeat the changes listed in the backwards > > > > compatibility section later, in the "Changes" section? If not, then > >

Re: [HACKERS] a patch for row-at-a-time execution for table functions

2013-04-23 Thread Joe Conway
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 04/23/2013 06:09 PM, David Gudeman wrote: > The primary change we made to Postgres in order to support our own > version of foreign data wrappers was a row-at-a-time execution for > table functions. In standard Postgres, when you execute a table

[HACKERS] a patch for row-at-a-time execution for table functions

2013-04-23 Thread David Gudeman
The primary change we made to Postgres in order to support our own version of foreign data wrappers was a row-at-a-time execution for table functions. In standard Postgres, when you execute a table function, it gathers all of the rows at once and stuffs them into a buffer in order to support cursor

Re: [HACKERS] Enabling Checksums

2013-04-23 Thread Jeff Davis
On Tue, 2013-04-23 at 16:28 +0100, Simon Riggs wrote: > * make the pg_control.data_checksums field into a version number, for > future flexibility... > patch attached Commenting on this separately because it's a separate issue. I'd prefer that it was some kind of a checksum ID code -- e.g. 0 for

Re: [HACKERS] 9.3 Beta1 status report

2013-04-23 Thread Alvaro Herrera
Bruce Momjian wrote: > On Tue, Apr 23, 2013 at 05:04:15PM -0400, Bruce Momjian wrote: > > > Do we usually repeat the changes listed in the backwards > > > compatibility section later, in the "Changes" section? If not, then > > > instead of the first two items above, let's just have these in the > >

Re: [HACKERS] REFRESH MATERIALIZED VIEW command in PL block hitting Assert

2013-04-23 Thread Kevin Grittner
Jeevan Chalke wrote: > On Mon, Apr 22, 2013 at 6:41 PM, Andres Freund wrote: >> On 2013-04-22 18:35:04 +0530, Jeevan Chalke wrote: >>> I have observed that following sequence is causing server crash. >>> >>> CREATE MATERIALIZED VIEW temp_class_mv AS >>>   SELECT * FROM pg_class >>>   WITH NO DA

Re: [HACKERS] GSOC Student Project Idea

2013-04-23 Thread Alexander Korotkov
Michael, On Mon, Apr 22, 2013 at 4:28 PM, Michael Schuh wrote: > Although I do not have a lot of experience with PostgreSQL development, I > am eager to learn and commit my summer to enabling another fantastic > feature for the community. Since iDistance is a non-recursive, data-driven, > space-

Re: [HACKERS] 9.3 Beta1 status report

2013-04-23 Thread Bruce Momjian
On Tue, Apr 23, 2013 at 05:04:15PM -0400, Bruce Momjian wrote: > > Do we usually repeat the changes listed in the backwards > > compatibility section later, in the "Changes" section? If not, then > > instead of the first two items above, let's just have these in the > > backwards-compatibility sect

Re: [HACKERS] 9.3 Beta1 status report

2013-04-23 Thread Bruce Momjian
On Tue, Apr 23, 2013 at 11:00:31PM +0200, Erikjan Rijkers wrote: > I just spotted some more small stuff: > > s/IF NOT EXIST /IF NOT EXISTS /g # 2 x > > > It actually had me doubting, but yes that -S should be there... Fixed, thanks. -- Bruce Momjian http://momjian.us Enterprise

Re: [HACKERS] 9.3 Beta1 status report

2013-04-23 Thread Bruce Momjian
On Tue, Apr 23, 2013 at 02:25:08PM +0300, Heikki Linnakangas wrote: > On 22.04.2013 23:06, Bruce Momjian wrote: > >On Mon, Apr 22, 2013 at 10:11:48PM +0300, Heikki Linnakangas wrote: > >>>E.1.3.2.1. Write-Ahead Log (WAL) > >>> > >>>Store WAL in a continuous stream, rather than skipping the last

Re: [HACKERS] 9.3 Beta1 status report

2013-04-23 Thread Erikjan Rijkers
I just spotted some more small stuff: s/IF NOT EXIST /IF NOT EXISTS /g # 2 x It actually had me doubting, but yes that -S should be there... Thanks, Erik Rijkers -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgr

Re: [HACKERS] 9.3 Beta1 status report

2013-04-23 Thread Bruce Momjian
On Tue, Apr 23, 2013 at 05:36:03PM +0800, Jov wrote: > E.1.3.1.4: > > Improve performance of the CREATE TABLE ... ON COMMIT DELETE ROWS clause by > only issuing delete if the temporary table was accessed (Heikki Linnakangas) > > should be: >              CREATE TEMP TABLE ... ON COMMIT DELETE ROW

Re: [HACKERS] 9.3 Beta1 status report

2013-04-23 Thread Bruce Momjian
On Tue, Apr 23, 2013 at 10:12:58AM +0100, Dean Rasheed wrote: > On 21 April 2013 06:02, Bruce Momjian wrote: > > I am not sure if Tom shared yet, but we are planning to package 9.3 > > beta1 on April 29, with a release on May 2. Those dates might change, > > but that is the current plan. I have

Re: [HACKERS] GSOC Student Project Idea

2013-04-23 Thread Alvaro Herrera
Michael Schuh escribió: > http://www.cs.montana.edu/~timothy.wylie/files/bncod13.pdf Um. From the paper I get the impression that there's yet much to learn in order for this indexing method to be really effective? -- Álvaro Herrerahttp://www.2ndQuadrant.com/ PostgreSQL Developm

Re: [HACKERS] minimizing the target list for foreign data wrappers

2013-04-23 Thread David Fetter
David, Please post your patch(es) and some demo of how things broke so others can improve future versions--possibly even 9.3 versions if it turns out you've discovered a bug in the implementation. Thanks very much for your hard work and insights into this. Cheers, David. On Tue, Apr 23, 2013 at

Re: [HACKERS] Couple of issues with pg_xlogdump

2013-04-23 Thread Jeff Janes
On Tue, Apr 23, 2013 at 2:30 AM, Andres Freund wrote: > On 2013-04-23 14:51:05 +0530, Pavan Deolasee wrote: > > > [pavan.deolasee@puppetserver pg_xlogdump]$ ./pg_xlogdump > > ~/db93head/pg_xlog/00010008 > > pg_xlogdump: FATAL: could not find a valid record after 0/800 > > > >

Re: [HACKERS] minimizing the target list for foreign data wrappers

2013-04-23 Thread David Gudeman
In case anyone is interested, I tried it and it doesn't seem to work. It looks like some other plan element already has the target-list tuple baked. Now I'm trying to decide whether to give up on FDW. It's a shame because it's such a sweet facility, but at this point, I just don't think that it's m

Re: [HACKERS] putting a bgworker to rest

2013-04-23 Thread Andres Freund
On 2013-04-23 14:11:26 -0300, Alvaro Herrera wrote: > Andres Freund wrote: > > On 2013-04-23 11:59:43 -0300, Alvaro Herrera wrote: > > > Andres Freund wrote: > > > > Hi all, > > > > > > > > I noticed the need to simply stop a bgworker after its work is done but > > > > still have it restart in unu

Re: [HACKERS] putting a bgworker to rest

2013-04-23 Thread Alvaro Herrera
Andres Freund wrote: > On 2013-04-23 11:59:43 -0300, Alvaro Herrera wrote: > > Andres Freund wrote: > > > Hi all, > > > > > > I noticed the need to simply stop a bgworker after its work is done but > > > still have it restart in unusual circumstances like a crash. > > > Obviously I can just have i

[HACKERS] Bug Fix: COLLATE with multiple ORDER BYs in aggregates

2013-04-23 Thread David Fetter
Folks, While testing the upcoming FILTER clause for aggregates, Erik Rijkers uncovered a long-standing bug in $subject, namely that this case wasn't handled. Please find attached a patch by Andrew Gierth and myself which fixes this issue and adds a regression test to ensure it remains fixed. Che

Re: [HACKERS] putting a bgworker to rest

2013-04-23 Thread Andres Freund
On 2013-04-23 11:59:43 -0300, Alvaro Herrera wrote: > Andres Freund wrote: > > Hi all, > > > > I noticed the need to simply stop a bgworker after its work is done but > > still have it restart in unusual circumstances like a crash. > > Obviously I can just have it enter a loop where it checks its

Re: [HACKERS] Couple of issues with pg_xlogdump

2013-04-23 Thread Andres Freund
On 2013-04-23 11:51:06 -0300, Alvaro Herrera wrote: > Andres Freund escribió: > > On 2013-04-23 15:16:05 +0530, Pavan Deolasee wrote: > > > > > It works without either if you use explicit options like -s STARTADDR > > > > and -p PATH which is frequently useful to just print a few records at > > >

Re: [HACKERS] Proposal to add --single-row to psql

2013-04-23 Thread Tom Lane
Christopher Manning writes: > I'm proposing to add a --single-row option to psql that would allow the > result rows of a query to be streamed to a file without collecting them in > memory first. Isn't there already a way to set FETCH_COUNT from the command line? (ie, I think there's a generic var

Re: [HACKERS] Proposal to add --single-row to psql

2013-04-23 Thread Pavel Stehule
Hello It is redundant to current FETCH_COUNT implementation, so I don't see sense to use it together. Maybe we can drop FETCH_COUNT and replace it by --single-row mode and probably it can simplify code. Regards Pavel 2013/4/23 Christopher Manning > psql currently collects the query result ro

Re: [HACKERS] putting a bgworker to rest

2013-04-23 Thread Alvaro Herrera
Andres Freund wrote: > Hi all, > > I noticed the need to simply stop a bgworker after its work is done but > still have it restart in unusual circumstances like a crash. > Obviously I can just have it enter a loop where it checks its latch and > such, but that seems a bit pointless. > > Would it

Re: [HACKERS] Couple of issues with pg_xlogdump

2013-04-23 Thread Alvaro Herrera
Andres Freund escribió: > On 2013-04-23 15:16:05 +0530, Pavan Deolasee wrote: > > > It works without either if you use explicit options like -s STARTADDR > > > and -p PATH which is frequently useful to just print a few records at > > > the correct point. I am not sure how could put that in there w

Re: [HACKERS] Performance with the new security release?

2013-04-23 Thread Alvaro Herrera
Anne Rosset wrote: > Thanks Steve. > I found this: http://www.postgresql.org/docs/current/static/release-9-2-3.html > " > Fix performance problems with autovacuum truncation in busy workloads (Jan > Wieck) > Truncation of empty pages at the end of a table requires exclusive lock, but > autovacuum

Re: [HACKERS] Performance with the new security release?

2013-04-23 Thread Steve Singer
On 13-04-23 10:04 AM, Anne Rosset wrote: Thanks Steve. I found this: http://www.postgresql.org/docs/current/static/release-9-2-3.html " Fix performance problems with autovacuum truncation in busy workloads (Jan Wieck) Truncation of empty pages at the end of a table requires exclusive lock, but

[HACKERS] Proposal to add --single-row to psql

2013-04-23 Thread Christopher Manning
psql currently collects the query result rows in memory before writing them to a file and can cause out of memory problems for large results in low memory environments like ec2. I can't use COPY TO STDOUT or FETCH_COUNT since I'm using Redshift and it doesn't support [writing to STDOUT]( http://doc

Re: [HACKERS] Enabling Checksums

2013-04-23 Thread Simon Riggs
On 23 April 2013 02:35, Jeff Davis wrote: > On Tue, 2013-04-23 at 01:08 +0300, Ants Aasma wrote: >> A slight delay, but here it is. I didn't lift the checksum part into a >> separate file as I didn't have a great idea what I would call it. The >> code is reasonably compact so I don't see a great n

Re: [HACKERS] high io BUT huge amount of free memory

2013-04-23 Thread Shaun Thomas
On 04/22/2013 05:12 PM, Merlin Moncure wrote: free -g total used free sharedbuffers cached Mem: 378250128 0 0229 -/+ buffers/cache: 20357 This is most likely a NUMA issue. There really see

Re: [HACKERS] REFRESH MATERIALIZED VIEW command in PL block hitting Assert

2013-04-23 Thread Andres Freund
On 2013-04-23 19:33:24 +0530, Jeevan Chalke wrote: > On Tue, Apr 23, 2013 at 7:01 PM, Merlin Moncure wrote: > > > On Tue, Apr 23, 2013 at 7:18 AM, Jeevan Chalke > > wrote: > > > Hi Tom, > > > > > > Since we are close to release, we should not have crashes like this. > > > > huh? we are not even

Re: [HACKERS] Performance with the new security release?

2013-04-23 Thread Anne Rosset
Thanks Steve. I found this: http://www.postgresql.org/docs/current/static/release-9-2-3.html " Fix performance problems with autovacuum truncation in busy workloads (Jan Wieck) Truncation of empty pages at the end of a table requires exclusive lock, but autovacuum was coded to fail (and release t

Re: [HACKERS] REFRESH MATERIALIZED VIEW command in PL block hitting Assert

2013-04-23 Thread Jeevan Chalke
On Tue, Apr 23, 2013 at 7:01 PM, Merlin Moncure wrote: > On Tue, Apr 23, 2013 at 7:18 AM, Jeevan Chalke > wrote: > > Hi Tom, > > > > Since we are close to release, we should not have crashes like this. > > huh? we are not even in beta yet > I mean, beta release. BTW, as per Bruce's mail "w

[HACKERS] putting a bgworker to rest

2013-04-23 Thread Andres Freund
Hi all, I noticed the need to simply stop a bgworker after its work is done but still have it restart in unusual circumstances like a crash. Obviously I can just have it enter a loop where it checks its latch and such, but that seems a bit pointless. Would it make sense to add an extra return val

Re: [HACKERS] Performance with the new security release?

2013-04-23 Thread Steve Singer
On 13-04-22 11:46 PM, Anne Rosset wrote: Thanks Steve. I have read that a fix has been put in release 9.2.3 for this issue. Is that right? Thanks, Anne No this issue is present in 9.0.13, 9.1.9 and 9.2.4 (as well as 9.2.3). There is talk about fixing this for the next set of minor releases bu

Re: [HACKERS] REFRESH MATERIALIZED VIEW command in PL block hitting Assert

2013-04-23 Thread Merlin Moncure
On Tue, Apr 23, 2013 at 7:18 AM, Jeevan Chalke wrote: > Hi Tom, > > Since we are close to release, we should not have crashes like this. huh? we are not even in beta yet merlin -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http

Re: [HACKERS] REFRESH MATERIALIZED VIEW command in PL block hitting Assert

2013-04-23 Thread Kevin Grittner
Andres Freund wrote: > On 2013-04-23 17:48:49 +0530, Jeevan Chalke wrote: >> Hi Tom, >> >> Since we are close to release, we should not have crashes like >> this. >> >> Please have a look. My patch may not be correct as I haven't >> looked closely. > > Isn't that Kevin's departement? I'll gladly

Re: [HACKERS] REFRESH MATERIALIZED VIEW command in PL block hitting Assert

2013-04-23 Thread Andres Freund
Hi, On 2013-04-23 17:48:49 +0530, Jeevan Chalke wrote: > Hi Tom, > > Since we are close to release, we should not have crashes like this. > > Please have a look. My patch may not be correct as I haven't looked closely. Isn't that Kevin's departement? Andres -- Andres Freund

Re: [HACKERS] REFRESH MATERIALIZED VIEW command in PL block hitting Assert

2013-04-23 Thread Jeevan Chalke
Hi Tom, Since we are close to release, we should not have crashes like this. Please have a look. My patch may not be correct as I haven't looked closely. Thanks On Mon, Apr 22, 2013 at 7:28 PM, Jeevan Chalke < jeevan.cha...@enterprisedb.com> wrote: > > > > On Mon, Apr 22, 2013 at 6:41 PM, And

Re: [HACKERS] Couple of issues with pg_xlogdump

2013-04-23 Thread Andres Freund
On 2013-04-23 15:16:05 +0530, Pavan Deolasee wrote: > > Which this confirms. This is likely the current end of wal. If you look > > at pg_current_xlog_location() after starting the server again, it should > > show an address nearby? > > > > > Oh yes, you are right. Again, could there be a better wa

Re: [HACKERS] 9.3 Beta1 status report

2013-04-23 Thread Heikki Linnakangas
On 22.04.2013 23:06, Bruce Momjian wrote: On Mon, Apr 22, 2013 at 10:11:48PM +0300, Heikki Linnakangas wrote: E.1.3.2.1. Write-Ahead Log (WAL) Store WAL in a continuous stream, rather than skipping the last 16MB segment every 4GB (Heikki Linnakangas) BACKWARD COMPATIBLE BREAK Restruct

Re: [HACKERS] Couple of issues with pg_xlogdump

2013-04-23 Thread Pavan Deolasee
On Tue, Apr 23, 2013 at 3:00 PM, Andres Freund wrote: > On 2013-04-23 14:51:05 +0530, Pavan Deolasee wrote: > > Hello, > > > > I was playing with pg_xlogdump in the HEAD and found a few issues. > > > > 1. Tried compiling pg_xlogdump via PGXS mechanism and it fails with the > > following error: > >

Re: [HACKERS] Substituting Checksum Algorithm (was: Enabling Checksums)

2013-04-23 Thread Ants Aasma
On Tue, Apr 23, 2013 at 11:47 AM, Andres Freund wrote: > On 2013-04-23 00:17:28 -0700, Jeff Davis wrote: >> + # important optimization flags for checksum.c >> + ifeq ($(GCC),yes) >> + checksum.o: CFLAGS += -msse4.1 -funroll-loops -ftree-vectorize >> + endif > > I am pretty sure we can't do those u

Re: [HACKERS] Couple of issues with pg_xlogdump

2013-04-23 Thread Andres Freund
On 2013-04-23 14:51:05 +0530, Pavan Deolasee wrote: > Hello, > > I was playing with pg_xlogdump in the HEAD and found a few issues. > > 1. Tried compiling pg_xlogdump via PGXS mechanism and it fails with the > following error: > make: *** No rule to make target > `/home/pavan.deolasee/work/pgsql/

[HACKERS] Couple of issues with pg_xlogdump

2013-04-23 Thread Pavan Deolasee
Hello, I was playing with pg_xlogdump in the HEAD and found a few issues. 1. Tried compiling pg_xlogdump via PGXS mechanism and it fails with the following error: make: *** No rule to make target `/home/pavan.deolasee/work/pgsql/postgresql/install/lib/pgxs/src/makefiles/../../src/backend/access/t

Re: [HACKERS] 9.3 Beta1 status report

2013-04-23 Thread Dean Rasheed
On 21 April 2013 06:02, Bruce Momjian wrote: > I am not sure if Tom shared yet, but we are planning to package 9.3 > beta1 on April 29, with a release on May 2. Those dates might change, > but that is the current plan. I have completed a draft 9.3 release > notes, which you can view here: > >

Re: [HACKERS] Substituting Checksum Algorithm (was: Enabling Checksums)

2013-04-23 Thread Andres Freund
On 2013-04-23 00:17:28 -0700, Jeff Davis wrote: > + # important optimization flags for checksum.c > + ifeq ($(GCC),yes) > + checksum.o: CFLAGS += -msse4.1 -funroll-loops -ftree-vectorize > + endif I am pretty sure we can't do those unconditionally: - -funroll-loops and -ftree-vectorize weren't alw

Re: [HACKERS] Substituting Checksum Algorithm (was: Enabling Checksums)

2013-04-23 Thread Ants Aasma
On Apr 23, 2013 10:17 AM, "Jeff Davis" wrote: > Attached is my reorganization of Ants's patch here: > > http://www.postgresql.org/message-id/CA > +CSw_vinyf-w45i=M1m__MpJZY=e8s4nt_knnpebtwjtoa...@mail.gmail.com Thanks for your help. Some notes below. > My changes: > > * wrest the core FNV algori

Re: [HACKERS] Substituting Checksum Algorithm (was: Enabling Checksums)

2013-04-23 Thread Florian Pflug
On Apr23, 2013, at 09:17 , Jeff Davis wrote: > I'd lean toward simplicity and closer adherence to the published version > of the algorithm rather than detecting a few more obscure error > patterns. It looks like the modification slows down the algorithm, too. The pattern that plain FNV1 misses ar

[HACKERS] Substituting Checksum Algorithm (was: Enabling Checksums)

2013-04-23 Thread Jeff Davis
Starting a new thread to more narrowly address the topic. Attached is my reorganization of Ants's patch here: http://www.postgresql.org/message-id/CA +CSw_vinyf-w45i=M1m__MpJZY=e8s4nt_knnpebtwjtoa...@mail.gmail.com My changes: * wrest the core FNV algorithm from the specific concerns of a data