Re: ERROR: found multixact from before relminmxid

2018-04-09 Thread Peter Geoghegan
On Mon, Apr 9, 2018 at 7:53 PM, Andres Freund wrote: > I've not followed this thread. Possible it's the overeager check for pg > upgraded tuples from before 9.3 that Alvaro fixed recently? I was aware of commit 477ad05e, which must be what you're referring to. I don't think that that's what this

Re: ERROR: found multixact from before relminmxid

2018-04-09 Thread Andres Freund
On April 9, 2018 7:51:19 PM PDT, Peter Geoghegan wrote: >On Mon, Apr 9, 2018 at 6:56 PM, Alexandre Arruda >wrote: >> (... and all other indexes returns null too) >> >> I tried with bt_index_check too. Same results. > >That's interesting, because it tells me that you have a table that >appears t

Re: ERROR: found multixact from before relminmxid

2018-04-09 Thread Peter Geoghegan
On Mon, Apr 9, 2018 at 6:56 PM, Alexandre Arruda wrote: > (... and all other indexes returns null too) > > I tried with bt_index_check too. Same results. That's interesting, because it tells me that you have a table that appears to not be corrupt, despite the CLUSTER error. Also, the error itself

Re: ERROR: found multixact from before relminmxid

2018-04-09 Thread Alexandre Arruda
2018-04-09 22:16 GMT-03:00 Peter Geoghegan : > On Mon, Apr 9, 2018 at 5:55 PM, Alexandre Arruda wrote: >> I ran amcheck in all index of a table and I only get empty returns. > > Did you try doing so with the "heapallindexed" option? That's what's > really interesting here. > > -- > Peter Geoghegan

Re: ERROR: found multixact from before relminmxid

2018-04-09 Thread Peter Geoghegan
On Mon, Apr 9, 2018 at 5:55 PM, Alexandre Arruda wrote: > I ran amcheck in all index of a table and I only get empty returns. Did you try doing so with the "heapallindexed" option? That's what's really interesting here. -- Peter Geoghegan

Re: Fwd: Carto ODBC_FDW issue "ERROR: Connecting to driver" from PG to DB2

2018-04-09 Thread karthik kumar
On Mon, Apr 9, 2018 at 7:52 PM, Adrian Klaver wrote: > On 04/09/2018 02:37 PM, karthik kumar wrote: > >> >> >> On Mon, Apr 9, 2018 at 5:13 PM, Adrian Klaver > > wrote: >> >> On 04/09/2018 02:10 PM, karthik kumar wrote: >> > > Assuming using this: >> >

Re: ERROR: found multixact from before relminmxid

2018-04-09 Thread Alexandre Arruda
2018-04-09 18:23 GMT-03:00 Peter Geoghegan : > > On Mon, Apr 9, 2018 at 7:01 AM, Tomas Vondra > wrote: > > The bigger question is whether this can actually detect the issue. If > > it's due to an storage issue, then perhaps yes. But if you only see > > multixact issues consistently and nothing els

Re: Fwd: Carto ODBC_FDW issue "ERROR: Connecting to driver" from PG to DB2

2018-04-09 Thread Adrian Klaver
On 04/09/2018 04:52 PM, Adrian Klaver wrote: On 04/09/2018 02:37 PM, karthik kumar wrote: On Mon, Apr 9, 2018 at 5:13 PM, Adrian Klaver mailto:adrian.kla...@aklaver.com>> wrote:     On 04/09/2018 02:10 PM, karthik kumar wrote:     Assuming using this:     https://github.com/Zhen

Re: Fwd: Carto ODBC_FDW issue "ERROR: Connecting to driver" from PG to DB2

2018-04-09 Thread Adrian Klaver
On 04/09/2018 02:37 PM, karthik kumar wrote: On Mon, Apr 9, 2018 at 5:13 PM, Adrian Klaver > wrote: On 04/09/2018 02:10 PM, karthik kumar wrote: Assuming using this: https://github.com/ZhengYang/odbc_fdw I believe that shou

Re: Rationale for aversion to the central database?

2018-04-09 Thread Tim Cross
Peter J. Holzer writes: > In my applications I use SQL heavily. RDBMs are good at processing > queries, so use them for that. If all you want is a key-value store, > don't use PostgreSQL. I'm not very fond of ORMs. I know what I want to > do and can express it in SQL. An ORM makes me translate t

Re: Fwd: Carto ODBC_FDW issue "ERROR: Connecting to driver" from PG to DB2

2018-04-09 Thread karthik kumar
On Mon, Apr 9, 2018 at 5:13 PM, Adrian Klaver wrote: > On 04/09/2018 02:10 PM, karthik kumar wrote: > >> >> >> >> >> On 04/09/2018 01:01 PM, karthik kumar wrote: >> >> Hi Guru's, >> >> I am trying to access few table present in DB2 LUW from postgres >> database. >> All commands wo

Re: ERROR: found multixact from before relminmxid

2018-04-09 Thread Peter Geoghegan
On Mon, Apr 9, 2018 at 7:01 AM, Tomas Vondra wrote: > The bigger question is whether this can actually detect the issue. If > it's due to an storage issue, then perhaps yes. But if you only see > multixact issues consistently and nothing else, it might easily be a > PostgreSQL bug (in which case t

Re: Fwd: Carto ODBC_FDW issue "ERROR: Connecting to driver" from PG to DB2

2018-04-09 Thread Adrian Klaver
On 04/09/2018 02:10 PM, karthik kumar wrote: On 04/09/2018 01:01 PM, karthik kumar wrote: Hi Guru's, I am trying to access few table present in DB2 LUW from postgres database. All commands work fine, however when I try to select data from table it throws error: pg@a

Fwd: Carto ODBC_FDW issue "ERROR: Connecting to driver" from PG to DB2

2018-04-09 Thread karthik kumar
On 04/09/2018 01:01 PM, karthik kumar wrote: > Hi Guru's, > > I am trying to access few table present in DB2 LUW from postgres database. > All commands work fine, however when I try to select data from table it > throws error: > > pg@a92a3741d40e:~/odbc_fdw$ psql -d postgres > psql (10.1) > Type "

Re: Carto ODBC_FDW issue "ERROR: Connecting to driver" from PG to DB2

2018-04-09 Thread Adrian Klaver
On 04/09/2018 01:01 PM, karthik kumar wrote: Hi Guru's, I am trying to access few table present in DB2 LUW from postgres database. All commands work fine, however when I try to select data from table it throws error: pg@a92a3741d40e:~/odbc_fdw$ psql -d postgres psql (10.1) Type "help" for hel

Carto ODBC_FDW issue "ERROR: Connecting to driver" from PG to DB2

2018-04-09 Thread karthik kumar
Hi Guru's, I am trying to access few table present in DB2 LUW from postgres database. All commands work fine, however when I try to select data from table it throws error: pg@a92a3741d40e:~/odbc_fdw$ psql -d postgres psql (10.1) Type "help" for help. postgres=# create extension odbc_fdw; CREATE

Re: Conflict between JSON_AGG and COPY

2018-04-09 Thread Adrian Klaver
On 04/08/2018 08:44 PM, Đỗ Ngọc Trí Cường wrote: Dear Arian Klaver, I think there is a misunderstood here. I think that I quite understand how is the second query run. The question I asked here is why exporting data, which is generated by a query "JSON_AGG" , with command "COPY". The data con

Re: Conflict between JSON_AGG and COPY

2018-04-09 Thread Daniel Verite
Đỗ Ngọc Trí Cường wrote: > I want to export it to a file in JSON format so I run the query as below: > COPY (SELECT...) TO '/home/postgres/test1.json' COPY TO applies its own format rules on top of the data, which include among other things, quoting the newline characters. What you seem t

Re: algo for canceling a deadlocked transaction

2018-04-09 Thread Thomas Poty
Hello Tom, Thank you for the clarification! Regards, Thomas Le lun. 9 avr. 2018 à 17:04, Tom Lane a écrit : > Christophe Pettus writes: > >> On Apr 9, 2018, at 07:33, Thomas Poty wrote: > >> ok, and long answer ? is it random? > > > It's not literally random, but from the application point

Re: How to monitor logical replication initial sync?

2018-04-09 Thread Peter Eisentraut
On 3/15/18 09:19, bricklen wrote: > How does one monitor the status or progress of an initial sync under > logical replication?  For example: > > * I create a publication in database db_pub > * I create a subscription in database db_sub > * In 15 minutes I want to check an see

Re: List all columns referencing an FK

2018-04-09 Thread Peter Eisentraut
On 2/8/18 05:31, Andreas Joseph Krogh wrote: > Back in 2008 I asked this > question:  > http://www.postgresql-archive.org/Finding-all-tables-that-have-foreign-keys-referencing-a-table-td2153236.html > I wonder, is this now possible using information_schema only, or are > there still pieces missing

Re: best way to write large data-streams quickly?

2018-04-09 Thread Steve Atkins
> On Apr 9, 2018, at 8:49 AM, Mark Moellering > wrote: > > Everyone, > > We are trying to architect a new system, which will have to take several > large datastreams (total of ~200,000 parsed files per second) and place them > in a database. I am trying to figure out the best way to import

best way to write large data-streams quickly?

2018-04-09 Thread Mark Moellering
Everyone, We are trying to architect a new system, which will have to take several large datastreams (total of ~200,000 parsed files per second) and place them in a database. I am trying to figure out the best way to import that sort of data into Postgres. I keep thinking i can't be the first to

Re: algo for canceling a deadlocked transaction

2018-04-09 Thread Tom Lane
Christophe Pettus writes: >> On Apr 9, 2018, at 07:33, Thomas Poty wrote: >> ok, and long answer ? is it random? > It's not literally random, but from the application point of view, it's not > predictable. For example, it's not always the one that opened first, or any > other consistent meas

Re: algo for canceling a deadlocked transaction

2018-04-09 Thread Christophe Pettus
> On Apr 9, 2018, at 07:33, Thomas Poty wrote: > > ok, and long answer ? is it random? It's not literally random, but from the application point of view, it's not predictable. For example, it's not always the one that opened first, or any other consistent measure. -- -- Christophe Pettus

Re: algo for canceling a deadlocked transaction

2018-04-09 Thread Thomas Poty
Hello Stephen, > The short answer is "it's whichever one detected the deadlock." The > deadlock timeout fires after a lock has been held that long and if a > deadlock is detected then the process detecting it will be canceled. ok, and long answer ? is it random? > I'd strongly recommend revie

Re: Conflict between JSON_AGG and COPY

2018-04-09 Thread David G. Johnston
On Mon, Apr 9, 2018 at 7:14 AM, Adrian Klaver wrote: > On 04/08/2018 08:44 PM, Đỗ Ngọc Trí Cường wrote: > > Presently, COPY TO will never emit an octal or hex-digits backslash > sequence, but it does use the other sequences listed above for those > control characters. > > >> The second query is f

Re: Conflict between JSON_AGG and COPY

2018-04-09 Thread Adrian Klaver
On 04/08/2018 08:44 PM, Đỗ Ngọc Trí Cường wrote: Dear Arian Klaver, I think there is a misunderstood here. I think that I quite understand how is the second query run. The question I asked here is why exporting data, which is generated by a query "JSON_AGG" , with command "COPY". The data con

Re: Rationale for aversion to the central database?

2018-04-09 Thread Peter J. Holzer
On 2018-04-08 19:39:43 -0400, Stephen Frost wrote: > * Alvaro Aguayo Garcia-Rada (aagu...@opensysperu.com) wrote: > > 1. Portability. Being tied to a single database engine is not always > > a good idea. When you write business logic in database, you have to > > write and maintain your store proced

Re: ERROR: found multixact from before relminmxid

2018-04-09 Thread Tomas Vondra
On 04/09/2018 01:49 PM, Alexandre Arruda wrote: > > > 2018-04-06 13:11 GMT-03:00 Tomas Vondra >: > > > > On 04/06/2018 04:29 PM, Alexandre Arruda wrote: > > 2018-04-06 9:39 GMT-03:00 Tomas Vondra > >

Re: Rationale for aversion to the central database?

2018-04-09 Thread Melvin Davidson
On Mon, Apr 9, 2018 at 9:45 AM, Ray Cote wrote: > Maintaining your database logic in version control and versioning the > deployed code can be a bit problematic. > > Conversely, providing a standardized pgsql module through which data is > updated and retrieved can help standardize access logic a

Re: algo for canceling a deadlocked transaction

2018-04-09 Thread Stephen Frost
Greetings, * Thomas Poty (thomas.p...@gmail.com) wrote: > My question is : In case of a deadlock between 2 transaction, how to know > which transaction will be canceled? Is it predictable? The short answer is "it's whichever one detected the deadlock." The deadlock timeout fires after a lock ha

algo for canceling a deadlocked transaction

2018-04-09 Thread Thomas Poty
Good afternoon, My question is : In case of a deadlock between 2 transaction, how to know which transaction will be canceled? Is it predictable? I have tried to look into sources but i have found nothing. ( probably, i am the problem) Regards, Thomas

Re: Rationale for aversion to the central database?

2018-04-09 Thread Ray Cote
Maintaining your database logic in version control and versioning the deployed code can be a bit problematic. Conversely, providing a standardized pgsql module through which data is updated and retrieved can help standardize access logic across multiple languages and libraries. And I concur that

Re: Rationale for aversion to the central database?

2018-04-09 Thread Peter J. Holzer
On 2018-04-08 14:39:49 -0700, Guyren Howe wrote: > I am a Rails developer at a medium-large size company. I’ve mostly worked at > smaller companies. I’ve some exposure to other web development communities. > > When it comes to databases, I have universally encountered the attitude that > one shoul

Re: pg_stat_all_tables not updated when VACUUM ANALYZE execution finishes

2018-04-09 Thread Adrian Klaver
On 04/09/2018 03:37 AM, Sebastien Arod wrote: Hi, I face a surprising behaviour with VACUUM ANALYZE. For a table with a structure like like this (and few records): create table my_table (     my_column numeric ); When I run the following: VACUUM ANALYZE my_table; SELECT relname, last_analyze,

Re: ERROR: found multixact from before relminmxid

2018-04-09 Thread Alexandre Arruda
2018-04-06 13:11 GMT-03:00 Tomas Vondra : > > > On 04/06/2018 04:29 PM, Alexandre Arruda wrote: > > 2018-04-06 9:39 GMT-03:00 Tomas Vondra > >: > > > > > > > > On 04/06/2018 02:09 AM, Alexandre Arruda wrote: > > > Hi, > > > > > > Some time ago,

pg_stat_all_tables not updated when VACUUM ANALYZE execution finishes

2018-04-09 Thread Sebastien Arod
Hi, I face a surprising behaviour with VACUUM ANALYZE. For a table with a structure like like this (and few records): create table my_table ( my_column numeric ); When I run the following: VACUUM ANALYZE my_table; SELECT relname, last_analyze, last_vacuum FROM pg_stat_all_tables where relnam

Re: Conflict between JSON_AGG and COPY

2018-04-09 Thread Đỗ Ngọc Trí Cường
Dear Arian Klaver, I think there is a misunderstood here. I think that I quite understand how is the second query run. The question I asked here is why exporting data, which is generated by a query "JSON_AGG" , with command "COPY". The data contain "\n" as 2 characters instead of "new line"