Suresh Thalamati created SPARK-18141:
----------------------------------------
Summary: jdbc datasource read fails when quoted columns
(eg:mixed case, reserved words) in source table are used in the filter.
Key: SPARK-18141
URL: https://issues.apache.org/jira/browse/SPARK-18141
Project: Spark
Issue Type: Bug
Components: SQL
Affects Versions: 2.0.1, 2.0.0
Reporter: Suresh Thalamati
create table t1("Name" text, "Id" integer)
insert into t1 values('Mike', 1)
val df = sqlContext.read.jdbc(jdbcUrl, "t1", new Properties)
df.filter("Id = 1").show()
df.filter("`Id` = 1").show()
Error :
Cause: org.postgresql.util.PSQLException: ERROR: column "id" does not exist
Position: 35
at
org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2182)
at
org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:1911)
at
org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:173)
at org.postgresql.jdbc.PgStatement.execute(PgStatement.java:622)
at org.postgresql.jdbc.PgStatement.executeWithFlags(PgStatement.java:472)
at org.postgresql.jdbc.PgStatement.executeQuery(PgStatement.java:386)
at
org.apache.spark.sql.execution.datasources.jdbc.JDBCRDD.compute(JDBCRDD.scala:295)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:319)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:283)
at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:38)
I am working on fix for this issue, will submit PR soon.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
