Re: [HACKERS] 9.4 Proposal: Initdb creates a single table

2014-04-22 Thread Craig Ringer
On 04/23/2014 02:11 PM, Simon Riggs wrote: > I propose we add a single table called Postgres when we Initdb > > CREATE TABLE Postgres (Id Integer, Data Jsonb); Without particular comment on the need for the table, I'd be concerned about calling it "postgres". My personal impression from Stack

Re: [HACKERS] 9.4 Proposal: Initdb creates a single table

2014-04-22 Thread Tom Lane
Simon Riggs writes: > By now, some of you will be doubled over laughing as if this is an > April fool joke. Indeed. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/m

[HACKERS] 9.4 Proposal: Initdb creates a single table

2014-04-22 Thread Simon Riggs
We start with a database called Postgres and a schema called Public. Yet we don't start up with any usable tables. I propose we add a single table called Postgres when we Initdb CREATE TABLE Postgres (Id Integer, Data Jsonb); COMMENT ON TABLE Postgres IS 'Single table for quick start usage -

Re: [HACKERS] How can we make beta testing better?

2014-04-22 Thread Jehan-Guillaume de Rorthais
On Thu, 17 Apr 2014 16:42:21 -0700 Josh Berkus wrote: > On 04/15/2014 09:53 PM, Rod Taylor wrote: > > A documented beta test process/toolset which does the following would help: > > 1) Enables full query logging > > 2) Creates a replica of a production DB, record $TIME when it stops. > > 3) Allow

Re: [HACKERS] Perfomance degradation 9.3 (vs 9.2) for FreeBSD

2014-04-22 Thread YAMAMOTO Takashi
> - NetBSD: crashes under load; this could have been fixed but when I ran the > benchmarks in 2012 none of the developers seemed to care. do you mean this? https://mail-index.netbsd.org/tech-kern/2012/08/29/msg013918.html YAMAMOTO Takashi -- Sent via pgsql-hackers mailing list (pgsql-hackers

Re: [HACKERS] pg_upgrade and epoch

2014-04-22 Thread Sergey Konoplev
On Tue, Apr 22, 2014 at 8:08 PM, Sergey Burladyan wrote: > On Wed, Apr 23, 2014 at 6:38 AM, Sergey Konoplev wrote: >> BTW, I didn't manage to make a test case yet. Recently, when I was >> migrating several servers to skytools3 and upgrading from 9.0 to 9.2, >> I noticed that epoch was copied, tim

Re: [HACKERS] pg_upgrade and epoch

2014-04-22 Thread Sergey Burladyan
On Wed, Apr 23, 2014 at 6:38 AM, Sergey Konoplev wrote: > > BTW, I didn't manage to make a test case yet. Recently, when I was > migrating several servers to skytools3 and upgrading from 9.0 to 9.2, > I noticed that epoch was copied, timeline id was >0 after upgrade, but > ... This is strange, i

Re: [HACKERS] pg_upgrade and epoch

2014-04-22 Thread Sergey Konoplev
On Tue, Apr 22, 2014 at 6:33 PM, Sergey Burladyan wrote: > Current pg_upgrade copy XID into new cluster, but not it epoch. Why? > > Without epoch from old cluster txid_current() in upgraded database return > lower value than before upgrade. This break, for example, PgQ and it must > be fixed by ha

Re: [HACKERS] Missing pfree in logical_heap_rewrite_flush_mappings()

2014-04-22 Thread Tom Lane
Bruce Momjian writes: > I had to revert this patch. It causes a failure in the > /contrib/test_decoding regression test. On closer inspection, it was simply pfree'ing the wrong pointer. I fixed that and also undid the allocation in a different memory context, which didn't seem to be a particula

Re: [HACKERS] Missing pfree in logical_heap_rewrite_flush_mappings()

2014-04-22 Thread Bruce Momjian
On Tue, Apr 22, 2014 at 06:05:53PM -0400, Bruce Momjian wrote: > On Wed, Mar 26, 2014 at 06:29:38PM +0200, Ants Aasma wrote: > > It seems to me that when flushing logical mappings to disk, each > > mapping file leaks the buffer used to pass the mappings to XLogInsert. > > Also, it seems consistent

[HACKERS] pg_upgrade and epoch

2014-04-22 Thread Sergey Burladyan
Hi All! Current pg_upgrade copy XID into new cluster, but not it epoch. Why? Without epoch from old cluster txid_current() in upgraded database return lower value than before upgrade. This break, for example, PgQ and it must be fixed by hand after upgrade with pg_resetxlog. PS: see http://lists.

Re: [HACKERS] Perfomance degradation 9.3 (vs 9.2) for FreeBSD

2014-04-22 Thread Mark Kirkwood
On 23/04/14 00:19, Andres Freund wrote: Hi, Attached you can find a short (compile tested only ) patch implementing a 'shared_memory_type' GUC, akin to 'dynamic_shared_memory_type'. Will only apply to 9.4, not 9.3, but it should be easy to convert for it. Have just tried this out (on Ubuntu 1

Re: [HACKERS] Re: default opclass for jsonb (was Re: Call for GIST/GIN/SP-GIST opclass documentation)

2014-04-22 Thread Tom Lane
Bruce Momjian writes: > Where are we on the default JSONB opclass change? Not sure. I'm for changing it, I think, but it wasn't at all clear that we had consensus on that. We did not have a proposed new name for the opclass either ... regards, tom lane -- Sent via pg

Re: [HACKERS] [GENERAL] aggregate returning anyarray and 'cannot determine result data type'

2014-04-22 Thread Tom Lane
[ redirecting to -hackers ] Tomas Vondra writes: > So my plan was to do something like this: > sample_append(internal, anyelement, int) -> internal > sample_final(internal) -> anyarray > CREATE AGGREGATE sample_agg(anyelement, int) ( > SFUNC = sample_append, > STYPE = internal

[HACKERS] What use case is make_tuple_indirect() supposed to illustrate?

2014-04-22 Thread Tom Lane
Pursuant to http://www.postgresql.org/message-id/29007.1396038...@sss.pgh.pa.us I've been working on a patch to prevent external toast pointers from appearing in composite Datums. I noticed that this patch completely breaks the make_tuple_indirect() test case added by commit 36820250. The regressi

Re: [HACKERS] assertion failure 9.3.4

2014-04-22 Thread Josh Berkus
On 04/22/2014 05:07 PM, Alvaro Herrera wrote: > If you want to make it easier to reproduce, you need to insert some > pg_usleep() calls in carefully selected spots. As Andres says, the > window is small normally. Yeah, but the whole point of this is that having pg_stat-statements/auto_explain loa

Re: [HACKERS] assertion failure 9.3.4

2014-04-22 Thread Alvaro Herrera
Andres Freund wrote: > On 2014-04-22 18:01:40 -0300, Alvaro Herrera wrote: > > Thanks for the analysis and patches. I've been playing with this on my > > own a bit, and one thing that I just noticed is that at least for > > heap_update I cannot reproduce a problem when the xmax is originally a > >

Re: [HACKERS] assertion failure 9.3.4

2014-04-22 Thread Alvaro Herrera
Josh Berkus wrote: > > >> In order to encounter this issue, I'd need to have two concurrent > >> processes update the child records of the same parent record? That is: > >> > >> A ---> B1 > >> \---> B2 > >> > >> ... and the issue should only happen if I update both B1 and B2 > >> concurrently i

Re: [HACKERS] Perfomance degradation 9.3 (vs 9.2) for FreeBSD

2014-04-22 Thread Andrew Dunstan
On 04/22/2014 06:43 PM, Mark Wong wrote: On Tue, Apr 22, 2014 at 10:06 AM, Joshua D. Drake mailto:j...@commandprompt.com>> wrote: On 04/22/2014 08:26 AM, Andrew Dunstan wrote: I'm going away tomorrow for a few days R&R. when I'm back next week I will set up a demo

Re: [HACKERS] Perfomance degradation 9.3 (vs 9.2) for FreeBSD

2014-04-22 Thread Mark Wong
On Tue, Apr 22, 2014 at 10:06 AM, Joshua D. Drake wrote: > > On 04/22/2014 08:26 AM, Andrew Dunstan wrote: > > I'm going away tomorrow for a few days R&R. when I'm back next week I >> will set up a demo client running this module. If you can have a machine >> prepped for this purpose by then so m

Re: [HACKERS] Re: default opclass for jsonb (was Re: Call for GIST/GIN/SP-GIST opclass documentation)

2014-04-22 Thread Peter Geoghegan
On Tue, Apr 22, 2014 at 3:32 PM, Bruce Momjian wrote: > Where are we on the default JSONB opclass change? FWIW, I still don't have any strong opinion here. I defer to others on this question. -- Peter Geoghegan -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make ch

Re: [HACKERS] [DOCS] [doc] EXPLAIN CREATE MATERIALIZED VIEW AS?

2014-04-22 Thread Bruce Momjian
On Wed, Apr 16, 2014 at 12:35:26PM +0900, Amit Langote wrote: > Hi, > > Attached adds CREATE MATERIALIZED VIEW AS to the list of statements > that can be EXPLAINed. Patch applied. Thanks. -- Bruce Momjian http://momjian.us EnterpriseDB http://enterprise

Re: [HACKERS] Re: default opclass for jsonb (was Re: Call for GIST/GIN/SP-GIST opclass documentation)

2014-04-22 Thread Bruce Momjian
On Wed, Apr 9, 2014 at 02:22:54PM -0400, Greg Stark wrote: > On Wed, Apr 9, 2014 at 11:24 AM, Tom Lane wrote: > >> Maybe we should make *neither* of these the default opclass, and give > >> *neither* the name json_ops. > > > > There's definitely something to be said for that. Default opclasses a

Re: [HACKERS] MultiXactId error after upgrade to 9.3.4

2014-04-22 Thread Bruce Momjian
On Mon, Mar 31, 2014 at 09:36:03AM -0400, Stephen Frost wrote: > Andres, > > * Andres Freund (and...@2ndquadrant.com) wrote: > > Without having looked at the code, IIRC this looks like some place > > misses passing allow_old=true where it's actually required. Any chance > > you can get a backtrace

Re: [HACKERS] [patch] Adding EXTRA_REGRESS_OPTS to all pg_regress invocations

2014-04-22 Thread Bruce Momjian
On Thu, Mar 27, 2014 at 06:03:24PM +0100, Christoph Berg wrote: > Re: Bruce Momjian 2013-12-04 <20131204151533.gb17...@momjian.us> > > On Mon, May 6, 2013 at 11:51:47PM -0700, Christoph Berg wrote: > > > "make check" supports EXTRA_REGRESS_OPTS to pass extra options to > > > pg_regress, but all th

Re: [HACKERS] Missing pfree in logical_heap_rewrite_flush_mappings()

2014-04-22 Thread Bruce Momjian
On Wed, Mar 26, 2014 at 06:29:38PM +0200, Ants Aasma wrote: > It seems to me that when flushing logical mappings to disk, each > mapping file leaks the buffer used to pass the mappings to XLogInsert. > Also, it seems consistent to allocate that buffer in the RewriteState > memory context. Patch att

Re: [HACKERS] Perfomance degradation 9.3 (vs 9.2) for FreeBSD

2014-04-22 Thread Alfred Perlstein
On 4/22/14, 8:26 AM, Andrew Dunstan wrote: On 04/22/2014 01:36 AM, Joshua D. Drake wrote: On 04/21/2014 06:19 PM, Andrew Dunstan wrote: If we never start we'll never get there. I can think of several organizations that might be approached to donate hardware. Like .Org? We have a hardwa

Re: [HACKERS] Patch for CREATE RULE sgml -- Was in: [DOCS]

2014-04-22 Thread Bruce Momjian
On Mon, Mar 24, 2014 at 09:51:07PM +0900, Fujii Masao wrote: > On Sat, Mar 22, 2014 at 12:56 AM, Emanuel Calvo > wrote: > > -BEGIN PGP SIGNED MESSAGE- > > Hash: SHA512 > > > > > > > > Hi guys, > > > > I realized that the output of the CREATE RULE has not a detailed > > output for the "even

Re: [HACKERS] Review: Patch FORCE_NULL option for copy COPY in CSV mode

2014-04-22 Thread Michael Paquier
On Wed, Apr 23, 2014 at 5:07 AM, Bruce Momjian wrote: > On Fri, Mar 7, 2014 at 05:08:54PM +0900, Michael Paquier wrote: > > On Thu, Mar 6, 2014 at 12:09 AM, Tom Lane wrote: > > > Andrew Dunstan writes: > > >> On 03/05/2014 09:11 AM, Michael Paquier wrote: > > >>> After testing this feature, I

Re: [HACKERS] assertion failure 9.3.4

2014-04-22 Thread Andres Freund
On 2014-04-22 14:49:00 -0700, Josh Berkus wrote: > > >> In order to encounter this issue, I'd need to have two concurrent > >> processes update the child records of the same parent record? That is: > >> > >> A ---> B1 > >> \---> B2 > >> > >> ... and the issue should only happen if I update both

Re: [HACKERS] assertion failure 9.3.4

2014-04-22 Thread Josh Berkus
>> In order to encounter this issue, I'd need to have two concurrent >> processes update the child records of the same parent record? That is: >> >> A ---> B1 >> \---> B2 >> >> ... and the issue should only happen if I update both B1 and B2 >> concurrently in separate sessions? > > I don't thi

Re: [HACKERS] assertion failure 9.3.4

2014-04-22 Thread Andres Freund
On 2014-04-22 18:01:40 -0300, Alvaro Herrera wrote: > Thanks for the analysis and patches. I've been playing with this on my > own a bit, and one thing that I just noticed is that at least for > heap_update I cannot reproduce a problem when the xmax is originally a > multixact, so AFAICT the numbe

Re: [HACKERS] assertion failure 9.3.4

2014-04-22 Thread Andres Freund
On 2014-04-22 14:40:46 -0700, Josh Berkus wrote: > On 04/22/2014 02:01 PM, Alvaro Herrera wrote: > > Some testing later, I think the issue only occurs if we determine that > > we don't need to wait for the xid/multi to complete, because otherwise > > the wait itself saves us. (It's easy to cause t

Re: [HACKERS] assertion failure 9.3.4

2014-04-22 Thread Andres Freund
On 2014-04-22 17:36:42 -0400, Andrew Dunstan wrote: > > On 04/22/2014 05:20 PM, Josh Berkus wrote: > >On 04/22/2014 02:01 PM, Alvaro Herrera wrote: > >>I think I should push this patch first, so that Andrew and Josh can try > >>their respective test cases which should start throwing errors, then >

Re: [HACKERS] assertion failure 9.3.4

2014-04-22 Thread Josh Berkus
On 04/22/2014 02:01 PM, Alvaro Herrera wrote: > Some testing later, I think the issue only occurs if we determine that > we don't need to wait for the xid/multi to complete, because otherwise > the wait itself saves us. (It's easy to cause the problem by adding a > breakpoint in heapam.c:3325, i.e

Re: [HACKERS] assertion failure 9.3.4

2014-04-22 Thread Andrew Dunstan
On 04/22/2014 05:20 PM, Josh Berkus wrote: On 04/22/2014 02:01 PM, Alvaro Herrera wrote: I think I should push this patch first, so that Andrew and Josh can try their respective test cases which should start throwing errors, then push the actual fixes. Does that sound okay? Note that I have a

Re: [HACKERS] Regression test errors

2014-04-22 Thread Bruce Momjian
On Sun, Mar 9, 2014 at 09:23:33AM -0300, Martín Marqués wrote: > OK, noticed how horrible this patch was (thanks for the heads up from > Jaime Casanova). This happens when trying to fetch changes one made on > a test copy after a day of lots of work back to a git repository: you > just make very s

Re: [HACKERS] assertion failure 9.3.4

2014-04-22 Thread Josh Berkus
On 04/22/2014 02:01 PM, Alvaro Herrera wrote: > I think I should push this patch first, so that Andrew and Josh can try > their respective test cases which should start throwing errors, then > push the actual fixes. Does that sound okay? Note that I have a limited ability to actually test my fail

Re: [HACKERS] assertion failure 9.3.4

2014-04-22 Thread Alvaro Herrera
Andres Freund wrote: > On 2014-04-21 19:43:15 -0400, Andrew Dunstan wrote: > > > > On 04/21/2014 02:54 PM, Andres Freund wrote: > > >Hi, > > > > > >I spent the last two hours poking arounds in the environment Andrew > > >provided and I was able to reproduce the issue, find a assert to > > >reprodu

Re: [HACKERS] Clock sweep not caching enough B-Tree leaf pages?

2014-04-22 Thread Peter Geoghegan
On Tue, Apr 22, 2014 at 2:03 AM, Hannu Krosing wrote: > What is the depth of b-tree at this percentage ? Well, this percentage of B-Tree pages that are leaf pages doesn't have much to do with the depth. The percentage seems very consistent for each B-Tree, irrespective of the total size of the B-

Re: [HACKERS] Review: Patch FORCE_NULL option for copy COPY in CSV mode

2014-04-22 Thread Bruce Momjian
On Fri, Mar 7, 2014 at 05:08:54PM +0900, Michael Paquier wrote: > On Thu, Mar 6, 2014 at 12:09 AM, Tom Lane wrote: > > Andrew Dunstan writes: > >> On 03/05/2014 09:11 AM, Michael Paquier wrote: > >>> After testing this feature, I noticed that FORCE_NULL and > >>> FORCE_NOT_NULL can both be speci

Re: [HACKERS] Review: Patch FORCE_NULL option for copy COPY in CSV mode

2014-04-22 Thread Bruce Momjian
On Wed, Mar 5, 2014 at 09:49:30PM +0900, Michael Paquier wrote: > On Wed, Mar 5, 2014 at 7:44 AM, Andrew Dunstan wrote: > > I have picked this up and committed the patch. Thanks to all. > Sorry for coming after the battle, but while looking at what has been > committed I noticed that copy2.sql is

Re: [HACKERS] Clock sweep not caching enough B-Tree leaf pages?

2014-04-22 Thread Jim Nasby
On 4/21/14, 6:07 PM, David G Johnston wrote: Jim Nasby-2 wrote >>I feel that if there is no memory pressure, frankly it doesnt matter much >>about what gets out and what not. The case I am specifically targeting is >>when the clocksweep gets to move about a lot i.e. high memory pressure >>worklo

Re: [HACKERS] [PATCH] `pg_dump -Fd` doesn't check write return status...

2014-04-22 Thread Bruce Momjian
On Sat, Mar 1, 2014 at 12:27:19PM -0800, Sean Chittenden wrote: > The attached patch fixes the case when `pg_dump -Fd …` is called > on a partition where write(2) fails for some reason or another. In > this case, backup jobs were returning with a successful exit code even > though most of the files

Re: [HACKERS] RFC: Async query processing

2014-04-22 Thread Claudio Freire
On Tue, Apr 22, 2014 at 3:45 PM, Florian Weimer wrote: > On 01/03/2014 06:06 PM, Claudio Freire wrote: > >> Per-query expectations could be such a thing. And it can even work with >> PQexec: >> >> PQexec(con, "SELECT nextval('a_id_seq') FROM generate_series(1,10);"); >> --read-- >> PQexec(con, "SE

Re: [HACKERS] RFC: Async query processing

2014-04-22 Thread Florian Weimer
On 04/22/2014 07:03 PM, Claudio Freire wrote: On Tue, Apr 22, 2014 at 8:19 AM, Florian Weimer wrote: Feedback in this thread was, "we want something like this in libpq, but not the thing you proposed". But there have been no concrete counter-proposals, and some of the responses did not take in

Re: [HACKERS] RFC: Async query processing

2014-04-22 Thread Florian Weimer
On 01/03/2014 06:06 PM, Claudio Freire wrote: Per-query expectations could be such a thing. And it can even work with PQexec: PQexec(con, "SELECT nextval('a_id_seq') FROM generate_series(1,10);"); --read-- PQexec(con, "SELECT nextval('b_id_seq') FROM generate_series(1,10);"); --read-- PQexec(co

Re: [HACKERS] Perfomance degradation 9.3 (vs 9.2) for FreeBSD

2014-04-22 Thread Merlin Moncure
On Mon, Apr 21, 2014 at 8:06 PM, Andrew Dunstan wrote: > > On 04/21/2014 08:49 PM, Stephen Frost wrote: >> >> * Tatsuo Ishii (is...@postgresql.org) wrote: >>> >>> I observe performance degradation with PostgreSQL 9.3 vs 9.2 on Linux >>> as well. The hardware is HP DL980G7, 80 cores, 2TB mem, RHEL

Re: [HACKERS] AXLE Plans for 9.5 and 9.6

2014-04-22 Thread Pavel Stehule
2014-04-22 19:02 GMT+02:00 Josh Berkus : > On 04/22/2014 06:39 AM, Andrew Dunstan wrote: > > I agree, and indeed that was something like my first reaction to hearing > > about this development - FDW seems like a very odd way to handle this. > > But the notion of builtin columnar storage suggests t

Re: [HACKERS] Perfomance degradation 9.3 (vs 9.2) for FreeBSD

2014-04-22 Thread Joshua D. Drake
On 04/22/2014 08:26 AM, Andrew Dunstan wrote: I'm going away tomorrow for a few days R&R. when I'm back next week I will set up a demo client running this module. If you can have a machine prepped for this purpose by then so much the better, otherwise I will have to drag out a box I recently re

Re: [HACKERS] RFC: Async query processing

2014-04-22 Thread Claudio Freire
On Tue, Apr 22, 2014 at 8:19 AM, Florian Weimer wrote: > Feedback in this thread was, "we want something like this in libpq, but not > the thing you proposed". But there have been no concrete counter-proposals, > and some of the responses did not take into account the inherent > complexities of r

Re: [HACKERS] Implied BETWEEN from join quals

2014-04-22 Thread Simon Riggs
On 22 April 2014 17:00, Stephen Frost wrote: > Simon, > > This all looks good, and at the risk of being slightly off-topic for > this thread, I just wanted to mention.. > > * Simon Riggs (si...@2ndquadrant.com) wrote: >> Current proposal ends there, but there is a further optimization that >> allo

Re: [HACKERS] AXLE Plans for 9.5 and 9.6

2014-04-22 Thread Josh Berkus
On 04/22/2014 06:39 AM, Andrew Dunstan wrote: > I agree, and indeed that was something like my first reaction to hearing > about this development - FDW seems like a very odd way to handle this. > But the notion of builtin columnar storage suggests to me that we really > need first to tackle how var

Re: [HACKERS] Implied BETWEEN from join quals

2014-04-22 Thread Stephen Frost
Simon, This all looks good, and at the risk of being slightly off-topic for this thread, I just wanted to mention.. * Simon Riggs (si...@2ndquadrant.com) wrote: > Current proposal ends there, but there is a further optimization that > allows us to remove the join altogether if > * There is a FK b

[HACKERS] Implied BETWEEN from join quals

2014-04-22 Thread Simon Riggs
I was recently nudged to describe an optimisation of merge joins/sorts. Rather than decribe that, I've looked at the general case: There are some additional implications we may make when joining tables... a particularly interesting one is that SELECT * FROM Fact JOIN Dim on Fact.col = Dim.col ca

Re: [HACKERS] Perfomance degradation 9.3 (vs 9.2) for FreeBSD

2014-04-22 Thread Palle Girgensohn
22 apr 2014 kl. 17:26 skrev Andrew Dunstan : > > On 04/22/2014 01:36 AM, Joshua D. Drake wrote: >> >> On 04/21/2014 06:19 PM, Andrew Dunstan wrote: >> >>> >>> If we never start we'll never get there. >>> >>> I can think of several organizations that might be approached to donate >>> hardware

Re: [HACKERS] Perfomance degradation 9.3 (vs 9.2) for FreeBSD

2014-04-22 Thread Andrew Dunstan
On 04/22/2014 01:36 AM, Joshua D. Drake wrote: On 04/21/2014 06:19 PM, Andrew Dunstan wrote: If we never start we'll never get there. I can think of several organizations that might be approached to donate hardware. Like .Org? We have a hardware farm, a rack full of hardware and spindles

Re: [HACKERS] Composite Datums containing toasted fields are a bad idea(?)

2014-04-22 Thread Tom Lane
Noah Misch writes: > On Mon, Apr 21, 2014 at 10:57:34AM -0400, Tom Lane wrote: >> I'm actually planning to set this patch on the shelf for a bit and go >> investigate the other alternative, ie, not generating composite Datums >> containing toast pointers in the first place. > I maintain that the

Re: [HACKERS] AXLE Plans for 9.5 and 9.6

2014-04-22 Thread Stephen Frost
* Andrew Dunstan (and...@dunslane.net) wrote: > I agree, and indeed that was something like my first reaction to > hearing about this development - FDW seems like a very odd way to > handle this. But the notion of builtin columnar storage suggests to > me that we really need first to tackle how var

Re: [HACKERS] Store data in pg_toast for custom type fails (bug?)

2014-04-22 Thread Tom Lane
Honza writes: > On 03/28/2014 07:02 PM, Tom Lane wrote: >> I believe I understand what's going on here, and it's not quite as >> exciting as it first appears. The issue is that we are failing to >> honor the "toasting goes only one level deep" rule in the specific >> case of arrays of composite t

[HACKERS] shm_mq inconsistent behavior of SHM_MQ_DETACHED

2014-04-22 Thread Petr Jelinek
Hi, I was playing with shm_mq and found a little odd behavior with detaching after sending messages. Following sequence behaves as expected (receiver gets 2 messages): P1 -> set_sender P1 -> attach P2 -> set_receiver P2 -> attach P1 -> send P2 -> receive P1 -> send P1 -> detach P2 -> receive P

Re: [HACKERS] AXLE Plans for 9.5 and 9.6

2014-04-22 Thread Simon Riggs
On 22 April 2014 13:15, MauMau wrote: > Great! I'm looking forward to seeing PostgreSQL evolve as an analytics > database for data warehousing. Is there any reason why in-memory database > and MPP is not included? Those ideas are valid; the features are bounded by resource constraints of time

Re: [HACKERS] AXLE Plans for 9.5 and 9.6

2014-04-22 Thread Andrew Dunstan
On 04/22/2014 08:04 AM, Simon Riggs wrote: On 22 April 2014 00:24, Josh Berkus wrote: On 04/21/2014 03:41 PM, Simon Riggs wrote: Storage Efficiency * Compression * Column Orientation You might look at turning this: http://citusdata.github.io/cstore_fdw/ ... into a more integrated part of P

Re: [HACKERS] AXLE Plans for 9.5 and 9.6

2014-04-22 Thread Andrew Dunstan
On 04/22/2014 08:15 AM, MauMau wrote: Are you planning to include the above features in 9.5 and 9.6? Are you recommending other developers not implement these features to avoid duplication of work with AXLE? Without pointing any fingers, I should note that I have learned the hard way

Re: [HACKERS] Perfomance degradation 9.3 (vs 9.2) for FreeBSD

2014-04-22 Thread Stephen Frost
* Magnus Hagander (mag...@hagander.net) wrote: > I didn't realize we had a guc for dynamic shared memory, must've missed > that in the discussion about that one. I agree that if we have that, it > makes perfect sense to have the same setting available for the main shared > memory segment. I recall

Re: [HACKERS] AXLE Plans for 9.5 and 9.6

2014-04-22 Thread Hannu Krosing
On 04/22/2014 02:04 PM, Simon Riggs wrote: > On 22 April 2014 00:24, Josh Berkus wrote: >> On 04/21/2014 03:41 PM, Simon Riggs wrote: >>> Storage Efficiency >>> * Compression >>> * Column Orientation >> You might look at turning this: >> >> http://citusdata.github.io/cstore_fdw/ >> >> ... into a m

Re: [HACKERS] Perfomance degradation 9.3 (vs 9.2) for FreeBSD

2014-04-22 Thread Magnus Hagander
On Tue, Apr 22, 2014 at 8:26 AM, Mark Kirkwood < mark.kirkw...@catalyst.net.nz> wrote: > On 22/04/14 09:25, Andres Freund wrote: > >> On 2014-04-21 17:21:20 -0400, Bruce Momjian wrote: >> >>> On Mon, Apr 21, 2014 at 02:08:51PM -0700, Joshua Drake wrote: >>> If the community had more *BSD pres

Re: [HACKERS] Perfomance degradation 9.3 (vs 9.2) for FreeBSD

2014-04-22 Thread Andres Freund
Hi, Attached you can find a short (compile tested only ) patch implementing a 'shared_memory_type' GUC, akin to 'dynamic_shared_memory_type'. Will only apply to 9.4, not 9.3, but it should be easy to convert for it. Greetings, Andres Freund -- Andres Freund http://www.2ndQ

Re: [HACKERS] AXLE Plans for 9.5 and 9.6

2014-04-22 Thread MauMau
From: "Simon Riggs" Some of areas of R&D are definitely on the roadmap, others are more flexible. Some of this is in progress, other stuff is not even at the design stage - yet, just a few paragraphs along the lines of "we will look at these topics". If we have room, its possible we may accommod

Re: [HACKERS] AXLE Plans for 9.5 and 9.6

2014-04-22 Thread Hannu Krosing
On 04/22/2014 01:24 AM, Josh Berkus wrote: > On 04/21/2014 03:41 PM, Simon Riggs wrote: >> Storage Efficiency >> * Compression >> * Column Orientation > You might look at turning this: > > http://citusdata.github.io/cstore_fdw/ > > ... into a more integrated part of Postgres. What would be of more

Re: [HACKERS] AXLE Plans for 9.5 and 9.6

2014-04-22 Thread Simon Riggs
On 22 April 2014 10:42, Jov wrote: > what about runtime code generation using LLVM? > http://blog.cloudera.com/blog/2013/02/inside-cloudera-impala-runtime-code-generation/ > http://llvm.org/devmtg/2013-11/slides/Wanderman-Milne-Cloudera.pdf Those techniques have been in use for at least 20 years

Re: [HACKERS] AXLE Plans for 9.5 and 9.6

2014-04-22 Thread Simon Riggs
On 22 April 2014 00:24, Josh Berkus wrote: > On 04/21/2014 03:41 PM, Simon Riggs wrote: >> Storage Efficiency >> * Compression >> * Column Orientation > > You might look at turning this: > > http://citusdata.github.io/cstore_fdw/ > > ... into a more integrated part of Postgres. Of course I'm awar

Re: [HACKERS] RFC: Async query processing

2014-04-22 Thread Florian Weimer
On 01/05/2014 01:56 PM, Craig Ringer wrote: JDBC also has a statement batching interface. Right now PgJDBC just unwraps the batch and runs each query individually. Any async-support improvements server-side should probably consider the need of executing a batch. The batch might be one PreparedSt

Re: [HACKERS] Store data in pg_toast for custom type fails (bug?)

2014-04-22 Thread Honza
On 03/28/2014 07:02 PM, Tom Lane wrote: > I wrote: >> Honza writes: >>> after a months I've found a time to make test-case for this bug, probably: > >> Confirmed that this reproduces a problem on HEAD. Will look into it, >> thanks! > > I believe I understand what's going on here, and it's not

Re: [HACKERS] AXLE Plans for 9.5 and 9.6

2014-04-22 Thread Jov
what about runtime code generation using LLVM? http://blog.cloudera.com/blog/2013/02/inside-cloudera-impala-runtime-code-generation/ http://llvm.org/devmtg/2013-11/slides/Wanderman-Milne-Cloudera.pdf Jov blog: http:amutu.com/blog 2014-04-22 6:41 GMT+08:00 Simon Riggs : >

Re: [HACKERS] A question about code in DefineRelation()

2014-04-22 Thread Etsuro Fujita
(2014/04/04 13:35), Etsuro Fujita wrote: > If I understand correctly, foreign tables cannot have an OID column, but > the following code in DefineRelation() assumes that foreign tables *can* > have that coulum: On second thought I noticed that that makes CREATE FOREIGN TABLE include an OID column

Re: [HACKERS] Clock sweep not caching enough B-Tree leaf pages?

2014-04-22 Thread Hannu Krosing
On 04/17/2014 10:39 PM, Andres Freund wrote: > On 2014-04-17 13:33:27 -0700, Peter Geoghegan wrote: >> Just over 99.6% of pages (leaving aside the meta page) in the big 10 >> GB pgbench_accounts_pkey index are leaf pages. What is the depth of b-tree at this percentage ? Cheers Hannu -- Sent vi

Re: [HACKERS] Clock sweep not caching enough B-Tree leaf pages?

2014-04-22 Thread Atri Sharma
On Tue, Apr 22, 2014 at 12:59 PM, Albe Laurenz wrote: > Jason Petersen wrote: > > Yes, we obviously want a virtual clock. Focusing on the use of > gettimeofday seems silly to me: it was > > something quick for the prototype. > > > > The problem with the clocksweeps is they don’t actually track the

Re: [HACKERS] Clock sweep not caching enough B-Tree leaf pages?

2014-04-22 Thread Albe Laurenz
Jason Petersen wrote: > Yes, we obviously want a virtual clock. Focusing on the use of gettimeofday > seems silly to me: it was > something quick for the prototype. > > The problem with the clocksweeps is they don’t actually track the progression > of “time” within the > PostgreSQL system. Woul