Re: ETL - sql orchestrator is stuck when there is not sleep() between queries

2021-07-08 Thread Allan Barrielle
PM Justin Pryzby wrote: > On Thu, Jul 08, 2021 at 03:49:12PM +0200, Allan Barrielle wrote: > > > Is it true that the SELECTs have no joins in them ? > > > > Yes there is a lot of LEFT JOIN. > > > > > It'd be very useful to get "explain analyze&q

Re: ETL - sql orchestrator is stuck when there is not sleep() between queries

2021-07-08 Thread Allan Barrielle
On a different machine, we use 12.7. Still same issue On Thu, Jul 8, 2021 at 3:49 PM Allan Barrielle wrote: > Hello, > > > Is it true that the SELECTs have no joins in them ? > > Yes there is a lot of LEFT JOIN. > > > When it doesn't work, you could check SELEC

Re: ETL - sql orchestrator is stuck when there is not sleep() between queries

2021-07-08 Thread Allan Barrielle
datasync wal_writer_delay,200ms wal_writer_flush_after,1MB work_mem,2GB xmlbinary,base64 xmloption,content zero_damaged_pages,off On Thu, Jul 8, 2021 at 2:33 PM Justin Pryzby wrote: > On Thu, Jul 08, 2021 at 01:00:28AM +0200, Allan Barrielle wrote: > > All is fine, and can work great. > >

ETL - sql orchestrator is stuck when there is not sleep() between queries

2021-07-08 Thread Allan Barrielle
Hello guys, I'm facing a problem. Currently I'm working on a Data transformation Pipeline on Postgres. The strategy is, We select every tables in a given schema ( 50 tables ), we apply some case when, translation, enum and load it into a different new schema with a CREATE TABLE SCHEMA_2.table_1 AS