Re: Lateral join not finding correlate variable

2020-11-21 Thread Dylan Forciea
Godfrey, Glad I could help! I suspected that was what the problem was. I have made a view in my postgres database to perform the inner lateral join, so that should let me work around this for the time being. Thanks, Dylan From: godfrey he Date: Friday, November 20, 2020 at 1:09 AM To: Dylan F

Re: Non uniform distribution of subtasks even with cluster.evenly-spread-out-slots

2020-11-21 Thread Till Rohrmann
Hi Harshit, the cluster.evenly-spread-out-slots strategy works the following way. If you schedule a task w/o preferred inputs (e.g. no inputs or too many inputs (I think the threshold is 8 inputs)), then it will pick a slot from a TaskManager with the lowest utilization. If, however, the task has

Re: Force Join Unique Key

2020-11-21 Thread Jark Wu
Hi Rex, The join key already has been used to organize records. As I said before, "the join key is the key of the keyed states". So an iterate on the MapState actually is a range scan (scan the join key prefix). However, this will perform "seek" operation which is rather slow than "get" operation.