Re: [SQL][JDBC] Possible regression in JDBC reader

2016-11-25 Thread Maciej Szymkiewicz
Thank you. On 11/25/2016 02:02 PM, Takeshi Yamamuro wrote: > Hi, > > Seems we forget to pass `parts:Array[Partition]` into `JDBCRelation`. > This was removed in this > commit: > https://github.com/apache/spark/commit/b3130c7b6a1ab4975023f08c3ab02ee8d2c7e995#diff-f70bda59304588cc3abfa3a9840653f4L

Re: [SQL][JDBC] Possible regression in JDBC reader

2016-11-25 Thread Hyukjin Kwon
I believe https://github.com/apache/spark/pull/15975 fixes this regression. I am sorry for the trouble. 2016-11-25 22:23 GMT+09:00 Sean Owen : > See https://github.com/apache/spark/pull/15499#discussion_r89008564 in > particular. Hyukjin / Xiao do we need to undo part of this change? > > > On Fr

Re: [SQL][JDBC] Possible regression in JDBC reader

2016-11-25 Thread Sean Owen
See https://github.com/apache/spark/pull/15499#discussion_r89008564 in particular. Hyukjin / Xiao do we need to undo part of this change? On Fri, Nov 25, 2016 at 1:02 PM Takeshi Yamamuro wrote: > Hi, > > Seems we forget to pass `parts:Array[Partition]` into `JDBCRelation`. > This was removed in

Re: [SQL][JDBC] Possible regression in JDBC reader

2016-11-25 Thread Takeshi Yamamuro
Hi, Seems we forget to pass `parts:Array[Partition]` into `JDBCRelation`. This was removed in this commit: https://github.com/apache/spark/commit/b3130c7b6a1ab4975023f08c3ab02ee8d2c7e995#diff-f70bda59304588cc3abfa3a9840653f4L237 // maropu On Fri, Nov 25, 2016 at 9:50 PM, Maciej Szymkiewicz wrot

[SQL][JDBC] Possible regression in JDBC reader

2016-11-25 Thread Maciej Szymkiewicz
Hi, I've been reviewing my notes to https://git.io/v1UVC using Spark built from 51b1c1551d3a7147403b9e821fcc7c8f57b4824c and it looks like JDBC ignores both: * (columnName, lowerBound, upperBound, numPartitions) * predicates and loads everything into a single partition. Can anyone confirm th