Thanks, filled https://issues.apache.org/jira/browse/SPARK-24288
Pozdrawiam / Best regards,
Tomek
On 2018-05-15 18:29, Wenchen Fan wrote:
applying predict pushdown is an optimization, and it makes sense to provide
configs to turn off certain optimizations. Feel free to create a JIRA.
Thanks,
W
applying predict pushdown is an optimization, and it makes sense to provide
configs to turn off certain optimizations. Feel free to create a JIRA.
Thanks,
Wenchen
On Tue, May 15, 2018 at 8:33 PM, Tomasz Gawęda
wrote:
> Hi,
>
> while working with JDBC datasource I saw that many "or" clauses with
Hi,
while working with JDBC datasource I saw that many "or" clauses with
non-equality operators causes huge performance degradation of SQL query
to database (DB2). For example:
val df = spark.read.format("jdbc").(other options to parallelize
load).load()
df.where(s"(date1 > $param1 and (date1
