Too many range table entries error

2018-06-25 Thread Akshaya Acharya
Hello. Please could you help debug the error "too many range table entries”? This error occurs when querying a view that is dependent on many other views (i.e. a view high up in the pyramid of views that we've constructed). I get this error when running select * on the view, or when running a

Re: Too many range table entries error

2018-06-25 Thread Andres Freund
Hi, On 2018-06-25 13:02:37 +0530, Akshaya Acharya wrote: > Hello. > > > Please could you help debug the error "too many range table entries”? > > > This error occurs when querying a view that is dependent on many other > views (i.e. a view high up in the pyramid of views that we've constructed

Re: Too many range table entries error

2018-06-25 Thread Akshaya Acharya
On Mon, 25 Jun 2018 at 13:40, Andres Freund wrote: > Hi, > > On 2018-06-25 13:02:37 +0530, Akshaya Acharya wrote: > > Hello. > > > > > > Please could you help debug the error "too many range table entries”? > > > > > > This error occurs when querying a view that is dependent on many other > > vie

Re: PostgreSQL : encryption with pgcrypto

2018-06-25 Thread Bruce Momjian
On Thu, May 17, 2018 at 07:07:00AM +, ROS Didier wrote: > Hi > >Regarding the encryption of data by pgcrypto, I would like to > know the recommendations for the management of the key. > >Is it possible to store it off the PostgreSQL server? > >

Re: Using COPY to import large xml file

2018-06-25 Thread Anto Aravinth
Thanks a lot. But I do got lot of challenges! Looks like SO data contains lot of tabs within itself.. So tabs delimiter didn't work for me. I thought I can give a special demiliter but looks like Postrgesql copy allow only one character as delimiter :( Sad, I guess only way is to insert or do a th

Re: Using COPY to import large xml file

2018-06-25 Thread Nicolas Paris
2018-06-25 16:25 GMT+02:00 Anto Aravinth : > Thanks a lot. But I do got lot of challenges! Looks like SO data contains > lot of tabs within itself.. So tabs delimiter didn't work for me. I thought > I can give a special demiliter but looks like Postrgesql copy allow only > one character as delimit

Re: Using COPY to import large xml file

2018-06-25 Thread Anto Aravinth
On Mon, Jun 25, 2018 at 8:20 PM, Nicolas Paris wrote: > > 2018-06-25 16:25 GMT+02:00 Anto Aravinth : > >> Thanks a lot. But I do got lot of challenges! Looks like SO data contains >> lot of tabs within itself.. So tabs delimiter didn't work for me. I thought >> I can give a special demiliter but

Re: Using COPY to import large xml file

2018-06-25 Thread Anto Aravinth
On Mon, Jun 25, 2018 at 8:54 PM, Anto Aravinth wrote: > > > On Mon, Jun 25, 2018 at 8:20 PM, Nicolas Paris > wrote: > >> >> 2018-06-25 16:25 GMT+02:00 Anto Aravinth : >> >>> Thanks a lot. But I do got lot of challenges! Looks like SO data >>> contains lot of tabs within itself.. So tabs delimite

RE: Load data from a csv file without using COPY

2018-06-25 Thread Kevin Brannen
From: Ravi Krishna [mailto:srkris...@yahoo.com] Sent: Tuesday, June 19, 2018 4:15 PM To: Steve Atkins Cc: PG mailing List Subject: Re: Load data from a csv file without using COPY > > If performance is relevant then your app should probably be using COPY > protocol, not line by line inserts. It'

Re: Using COPY to import large xml file

2018-06-25 Thread Nicolas Paris
2018-06-25 17:30 GMT+02:00 Anto Aravinth : > > > On Mon, Jun 25, 2018 at 8:54 PM, Anto Aravinth < > anto.aravinth@gmail.com> wrote: > >> >> >> On Mon, Jun 25, 2018 at 8:20 PM, Nicolas Paris >> wrote: >> >>> >>> 2018-06-25 16:25 GMT+02:00 Anto Aravinth : >>> Thanks a lot. But I do got lot

Schema/Data conversion opensource tools from MySQL to PostgreSQL

2018-06-25 Thread chiru r
Hi All, Please suggest Schema/Data conversion opensource tools from MySQL to PostgreSQL. Thanks, Chiranjeevi

Re: using pg_basebackup for point in time recovery

2018-06-25 Thread Bruce Momjian
On Thu, Jun 21, 2018 at 04:50:38PM -0700, David G. Johnston wrote: > On Thu, Jun 21, 2018 at 4:26 PM, Vik Fearing > wrote: > > On 21/06/18 07:27, Michael Paquier wrote: > > Attached is a patch which includes your suggestion.  What do you think? > > As that's an improvement, only HEAD

Re: Schema/Data conversion opensource tools from MySQL to PostgreSQL

2018-06-25 Thread Pavan Teja
Hi Chiru, You can use MySQL foreign data wrapper to achieve this. Regards, Pavan On Mon, Jun 25, 2018, 10:18 PM chiru r wrote: > Hi All, > > > Please suggest Schema/Data conversion opensource tools from MySQL to > PostgreSQL. > > > Thanks, > Chiranjeevi >

Re: Schema/Data conversion opensource tools from MySQL to PostgreSQL

2018-06-25 Thread James Keener
Can you explain what you're looking for? Beyond manually editing the schema dump to import correctly into postgres and the loading in INSERT dumps or MySQL file (TSV?) dumps, what are you looking for? Do you have any odd/incompatible data types? Jim On Mon, Jun 25, 2018 at 11:47 AM, chiru r wrot

Re: [External] Re: Schema/Data conversion opensource tools from MySQL to PostgreSQL

2018-06-25 Thread Vijaykumar Jain
I have not tested this, but have read about this in somewhere. https://github.com/the4thdoctor/pg_chameleon if it helps good else ignore Thanks, Vijay From: Pavan Teja Date: Monday, June 25, 2018 at 10:25 PM To: chiru r Cc: "pgsql-gene...@postgresql.org >> PG-General Mailing List" Subject

DB size growing exponentially when materialized view refreshed concurrently (postgres 9.6)

2018-06-25 Thread Vikas Sharma
Hi All, I am looking for advice in a issue where two materialized views are being refreshed concurrently and dbsize has grown to 150gb from 4gb in two days. We use two materialized views to keep processed data for faster query results for a search function. Earlier materialized views were refresh

Re: Too many range table entries error

2018-06-25 Thread Andres Freund
On 2018-06-25 13:46:06 +0530, Akshaya Acharya wrote: > On Mon, 25 Jun 2018 at 13:40, Andres Freund wrote: > > > Hi, > > > > On 2018-06-25 13:02:37 +0530, Akshaya Acharya wrote: > > > Hello. > > > > > > > > > Please could you help debug the error "too many range table entries”? > > > > > > > > > T

Re: Schema/Data conversion opensource tools from MySQL to PostgreSQL

2018-06-25 Thread Thomas Kellerer
chiru r schrieb am 25.06.2018 um 18:47: Please suggest Schema/Data conversion opensource tools from MySQL to PostgreSQL. ora2pg also supports MySQL: https://github.com/darold/ora2pg

Re: DB size growing exponentially when materialized view refreshed concurrently (postgres 9.6)

2018-06-25 Thread Alban Hertroys
> On 25 Jun 2018, at 19:21, Vikas Sharma wrote: > > I am looking for advice in a issue where two materialized views are being > refreshed concurrently and dbsize has grown to 150gb from 4gb in two days. > > We use two materialized views to keep processed data for faster query results > for a

Re: Using COPY to import large xml file

2018-06-25 Thread Adrian Klaver
On 06/25/2018 07:25 AM, Anto Aravinth wrote: Thanks a lot. But I do got lot of challenges! Looks like SO data contains lot of tabs within itself.. So tabs delimiter didn't work for me. I thought I can give a special demiliter but looks like Postrgesql copy allow only one character as delimiter

Re: Schema/Data conversion opensource tools from MySQL to PostgreSQL

2018-06-25 Thread Adrian Klaver
On 06/25/2018 09:47 AM, chiru r wrote: Hi All, Please suggest Schema/Data conversion opensource tools from MySQL to PostgreSQL. To add to the list: pgloader: http://pgloader.readthedocs.io/en/latest/tutorial/tutorial.html#migrating-from-mysql-to-postgresql Thanks, Chiranjeevi -- Ad

Re: Load data from a csv file without using COPY

2018-06-25 Thread Adrian Klaver
On 06/25/2018 09:17 AM, Kevin Brannen wrote: From: Ravi Krishna [mailto:srkris...@yahoo.com] Sent: Tuesday, June 19, 2018 4:15 PM To: Steve Atkins Cc: PG mailing List Subject: Re: Load data from a csv file without using COPY If performance is relevant then your app should probably be using C

Re: DB size growing exponentially when materialized view refreshed concurrently (postgres 9.6)

2018-06-25 Thread Vikas Sharma
Hi Alban, I haven't disabled autovacuum task, it's running fine for other objects. I was also getting "Error: cancelling autovacuum task" on the materialized view when concurrently refreshed so decided to write a script and run vacuum (full, analyze) the MV immediately after concurrent refresh but

Re: Schema/Data conversion opensource tools from MySQL to PostgreSQL

2018-06-25 Thread chiru r
Thank you All. On Mon, Jun 25, 2018 at 3:40 PM, Adrian Klaver wrote: > On 06/25/2018 09:47 AM, chiru r wrote: > >> Hi All, >> >> >> Please suggest Schema/Data conversion opensource tools from MySQL to >> PostgreSQL. >> >> > To add to the list: > > pgloader: > > http://pgloader.readthedocs.io/en/

PLPython Setup issue with EDB Windows Installer

2018-06-25 Thread Anthony DeBarros
I sent a report to EDB about this, but since I'm not a support subscriber I am not sure it will get through. Thought I would share here and seek some tips or feedback. --- Installing EDB Postgres Standard 10.4 on Windows 10 does not set up PLPython support correctly. (I also installed the Languag

Re: DB size growing exponentially when materialized view refreshed concurrently (postgres 9.6)

2018-06-25 Thread Jeff Janes
On Mon, Jun 25, 2018 at 1:21 PM, Vikas Sharma wrote: > Hi All, > > I am looking for advice in a issue where two materialized views are being > refreshed concurrently and dbsize has grown to 150gb from 4gb in two days. > > We use two materialized views to keep processed data for faster query > res

Re: Using COPY to import large xml file

2018-06-25 Thread Tim Cross
Anto Aravinth writes: > Thanks a lot. But I do got lot of challenges! Looks like SO data contains > lot of tabs within itself.. So tabs delimiter didn't work for me. I thought > I can give a special demiliter but looks like Postrgesql copy allow only > one character as delimiter :( > > Sad, I g

Re: PostgreSQL Volume Question

2018-06-25 Thread Data Ace
I appreciate everyone's feedback and help and will consider everyone's input. Thanks again. On Fri, Jun 15, 2018 at 9:26 AM, Data Ace wrote: > Well I think my question is somewhat away from my intention cause of my > poor understanding and questioning :( > > > > Actually, I have 1TB data and hav

Re: Too many range table entries error

2018-06-25 Thread Tom Lane
Andres Freund writes: > On 2018-06-25 13:46:06 +0530, Akshaya Acharya wrote: >> Our entire application-all our business logic-is built as layers of views >> inside the database. The ref counts sort of multiple at each layer, hence >> the large number. > That still doesn't explain how you realisti

Re: using pg_basebackup for point in time recovery

2018-06-25 Thread Michael Paquier
On Mon, Jun 25, 2018 at 12:51:10PM -0400, Bruce Momjian wrote: > FYI, in recent discussions on the docs list: > > > https://www.postgresql.org/message-id/CABUevEyumGh3r05U3_mhRrEU=dfacdrr2hew140mvn7fsbm...@mail.gmail.com I did not recall this one. Thanks for the reminder, Bruce. > There

Re: Schema/Data conversion opensource tools from MySQL to PostgreSQL

2018-06-25 Thread Arnaud L.
Le 25-06-18 à 18:47, chiru r a écrit : Please suggest Schema/Data conversion opensource tools from MySQL to PostgreSQL. Hi. I used this php script which did a pretty good job : https://github.com/AnatolyUss/FromMySqlToPostgreSql -- Regards

Re: pg_upgrade and wraparound

2018-06-25 Thread Alexander Shutyaev
Hello again, I've performed another test - I've migrated to the new cluster using dump restore: pg_dumpall | psql. It went well, although it took 6 days while pg_upgrade usually took a night. Is there any hope the issue with pg_upgrade can be resolved? If not, could you give me some hints as to h