Fwd: persistent read cache

2018-02-11 Thread Sand Stone
Posted to the wrong alias. Just wonder if anyone else feels such a use case. Thanks. -- Forwarded message -- From: Sand Stone Date: Sun, Feb 11, 2018 at 11:34 AM Subject: persistent read cache To: pgsql-hack...@lists.postgresql.org Hi. I wonder if there is such a thing or exten

Re: New Copy Formats - avro/orc/parquet

2018-02-11 Thread Tom Lane
Andres Freund writes: > On February 11, 2018 2:48:13 PM PST, Tom Lane wrote: >> (Any such patch should manage >> to turn COPY-CSV into an extension, at least so far as copy.c is >> concerned, even if we don't package it as one.) > Yea, I was thinking we should move all three (default, csv, binar

RE: Migrating Postgresql from Linux x86

2018-02-11 Thread Clive Anglin
Thanks for the information Clive A. -Original Message- From: Carlos Martinez [mailto:cama...@gmail.com] Sent: Friday, February 9, 2018 10:53 PM To: pgsql-gene...@postgresql.org Cc: Clive Anglin Subject: Re: Migrating Postgresql from Linux x86 Hi. We do that migration from Postgres 9.

Re: New Copy Formats - avro/orc/parquet

2018-02-11 Thread Andres Freund
On February 11, 2018 2:48:13 PM PST, Tom Lane wrote: > (Any such patch should manage >to turn COPY-CSV into an extension, at least so far as copy.c is >concerned, even if we don't package it as one.) Yea, I was thinking we should move all three (default, csv, binary) supported formats to using

Re: New Copy Formats - avro/orc/parquet

2018-02-11 Thread Tom Lane
Andres Freund writes: > So, I think making COPY extensible would be quite beneficial. I'm > however quite doubtful that we want to add core code to handle all of > the above. I think we should make the COPY input/output formatting > extensible by extensions. +1. I can't see carrying code for the

Re: New Copy Formats - avro/orc/parquet

2018-02-11 Thread Nicolas Paris
Le 11 févr. 2018 à 22:19, Adrian Klaver écrivait : > On 02/11/2018 12:57 PM, Nicolas Paris wrote: > > Le 11 févr. 2018 à 21:53, Andres Freund écrivait : > > > On 2018-02-11 21:41:26 +0100, Nicolas Paris wrote: > > > > I have also the storage and network transfers overhead in mind: > > > > All those

Re: New Copy Formats - avro/orc/parquet

2018-02-11 Thread Adrian Klaver
On 02/11/2018 12:57 PM, Nicolas Paris wrote: Le 11 févr. 2018 à 21:53, Andres Freund écrivait : On 2018-02-11 21:41:26 +0100, Nicolas Paris wrote: I have also the storage and network transfers overhead in mind: All those new formats are compressed; this is not true for current postgres BINARY f

Re: New Copy Formats - avro/orc/parquet

2018-02-11 Thread Andres Freund
On 2018-02-10 16:13:04 +0100, Nicolas Paris wrote: > Hello > > I d'found useful to be able to import/export from postgres to those modern > data > formats: > - avro (c writer=https://avro.apache.org/docs/1.8.2/api/c/index.html) > - parquet (c++ writer=https://github.com/apache/parquet-cpp) > - or

Re: New Copy Formats - avro/orc/parquet

2018-02-11 Thread Nicolas Paris
Le 11 févr. 2018 à 21:53, Andres Freund écrivait : > On 2018-02-11 21:41:26 +0100, Nicolas Paris wrote: > > I have also the storage and network transfers overhead in mind: > > All those new formats are compressed; this is not true for current > > postgres BINARY format and obviously text based form

Re: New Copy Formats - avro/orc/parquet

2018-02-11 Thread Andres Freund
On 2018-02-11 21:41:26 +0100, Nicolas Paris wrote: > I have also the storage and network transfers overhead in mind: > All those new formats are compressed; this is not true for current > postgres BINARY format and obviously text based format. By experience, > the binary format is 10 to 30% larger

Re: New Copy Formats - avro/orc/parquet

2018-02-11 Thread Nicolas Paris
Le 11 févr. 2018 à 21:03, Andres Freund écrivait : > > > On February 11, 2018 12:00:12 PM PST, Nicolas Paris > wrote: > >> > That is true, but the question is how significant the overhead is. > >If > >> > it's 50% then reducing it would make perfect sense. If it's 1% then > >no > >> > one if go

Re: New Copy Formats - avro/orc/parquet

2018-02-11 Thread Andres Freund
On February 11, 2018 12:00:12 PM PST, Nicolas Paris wrote: >> > That is true, but the question is how significant the overhead is. >If >> > it's 50% then reducing it would make perfect sense. If it's 1% then >no >> > one if going to be bothered by it. >> >> I think it's pretty clear that it's g

Re: New Copy Formats - avro/orc/parquet

2018-02-11 Thread Nicolas Paris
> > That is true, but the question is how significant the overhead is. If > > it's 50% then reducing it would make perfect sense. If it's 1% then no > > one if going to be bothered by it. > > I think it's pretty clear that it's going to be way way much more than > 1%. Good news but not sure to a

Re: New Copy Formats - avro/orc/parquet

2018-02-11 Thread Andres Freund
Hi, On 2018-02-10 18:21:37 +0100, Tomas Vondra wrote: > That is true, but the question is how significant the overhead is. If > it's 50% then reducing it would make perfect sense. If it's 1% then no > one if going to be bothered by it. I think it's pretty clear that it's going to be way way much

Re: execute block like Firebird does

2018-02-11 Thread Thiemo Kellner
On 02/11/18 06:57, PegoraroF10 wrote: I know we could create a function but we have several hundred of these blocks running, so ... it would be a huge work to do. So, there is a way to run a dynamic sql which returns a set of records ? But don't you need to touch the code of those blocks anyw

Re: execute block like Firebird does

2018-02-11 Thread Fabrízio de Royes Mello
Em dom, 11 de fev de 2018 às 13:54, Steven Lembark escreveu: > > > > you can use a DO - block: > > > > > > > > > https://www.postgresql.org/docs/current/static/sql-do.html > > > > > > > But DO blocks returns "void", I mean you can't return values/records > > from this statement. > > Insert the ne

Re: execute block like Firebird does

2018-02-11 Thread Steven Lembark
> > you can use a DO - block: > > > > > > https://www.postgresql.org/docs/current/static/sql-do.html > > > > But DO blocks returns "void", I mean you can't return values/records > from this statement. Insert the necessary records into a temporary table, process them, post-process them into var

Re: execute block like Firebird does

2018-02-11 Thread Pavel Stehule
2018-02-11 14:50 GMT+01:00 PegoraroF10 : > but DO doesn´t return values, or it does ? > > execute block returns(ID Integer, Name varchar(50), LastInvoice Date, ...) > as > begin > for select ID, Name from Customers where ... into ID, Name do begin > select bla, bla, bla from functionX(ID) in

Re: execute block like Firebird does

2018-02-11 Thread PegoraroF10
but DO doesn´t return values, or it does ? execute block returns(ID Integer, Name varchar(50), LastInvoice Date, ...) as begin for select ID, Name from Customers where ... into ID, Name do begin select bla, bla, bla from functionX(ID) into ...; if ... then bla = X else

Re: execute block like Firebird does

2018-02-11 Thread Fabrízio de Royes Mello
Em dom, 11 de fev de 2018 às 06:47, Andreas Kretschmer < andr...@a-kretschmer.de> escreveu: > > > Am 11.02.2018 um 06:57 schrieb PegoraroF10: > > We are migrating our databases from Firebird to PostGres. A useful > feature > > Firebird has is Execute Block. > > What it does is just return a record

Re: execute block like Firebird does

2018-02-11 Thread Andreas Kretschmer
Am 11.02.2018 um 06:57 schrieb PegoraroF10: We are migrating our databases from Firebird to PostGres. A useful feature Firebird has is Execute Block. What it does is just return a record set from that dynamic SQL, just like a PostGres function, but without creating it. It sound like ... execute