Re: Back pressure with multiple joins

2020-09-26 Thread Dan Hill
I can't reproduce the issue now. I'm using the same commits so I'm guessing another environment factor is at play. The explains are pretty similar (there's an extra LogicalProject when there is an extra view). On Fri, Sep 25, 2020 at 12:55 AM Timo Walther wrote: > Hi Dan, > > could you share t

Re: Back pressure with multiple joins

2020-09-25 Thread Timo Walther
Hi Dan, could you share the plan with us using `TableEnvironment.explainSql()` for both queries? In general, views should not have an impact on the performance. They are a logical concept that gives a bunch of operations a name. The contained operations are inlined into the bigger query duri

Re: Back pressure with multiple joins

2020-09-22 Thread Dan Hill
When I use DataStream and implement the join myself, I can get 50x the throughput. I assume I'm doing something wrong with Flink's Table API and SQL interface. On Tue, Sep 22, 2020 at 11:21 PM Dan Hill wrote: > Hi! > > My goal is to better understand how my code impacts streaming throughput. >

Back pressure with multiple joins

2020-09-22 Thread Dan Hill
Hi! My goal is to better understand how my code impacts streaming throughput. I have a streaming job where I join multiple tables (A, B, C, D) using interval joins. Case 1) If I have 3 joins in the same query, I don't hit back pressure. SELECT ... FROM A LEFT JOIN B ON... LEFT JOIN C ON... LEFT