I think you should mention partitionColumn like below and the Colum type should
be numeric. It works for my case.
options.put("partitionColumn", "revision");
Thanks,
Manohar
From: Hafiz Mujadid [via Apache Spark User List]
[mailto:[email protected]]
Sent: Monday, July 6, 2015 10:56 AM
To: Manohar Reddy
Subject: lower and upper offset not working in spark with mysql database
Hi all!
I am trying to read records from offset 100 to 110 from a table using following
piece of code
val sc = new SparkContext(new
SparkConf().setAppName("SparkJdbcDs").setMaster("local[*]"))
val sqlContext = new SQLContext(sc)
val options = new HashMap[String, String]()
options.put("driver", "com.mysql.jdbc.Driver")
options.put("url", "jdbc:mysql://*******:3306/temp?user=****&password=****")
options.put("dbtable", "tempTable")
options.put("lowerBound", "100")
options.put("upperBound", "110")
options.put("numPartitions", "1")
sqlContext.load("jdbc", options)
but this returns all the records instead of only 10 records
________________________________
If you reply to this email, your message will be added to the discussion below:
http://apache-spark-user-list.1001560.n3.nabble.com/lower-and-upper-offset-not-working-in-spark-with-mysql-database-tp23635.html
To start a new topic under Apache Spark User List, email
[email protected]<mailto:[email protected]>
To unsubscribe from Apache Spark User List, click
here<http://apache-spark-user-list.1001560.n3.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_code&node=1&code=bWFub2hhci5yZWRkeUBoYXBwaWVzdG1pbmRzLmNvbXwxfDM0NDk1ODYyNg==>.
NAML<http://apache-spark-user-list.1001560.n3.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>
________________________________
Happiest Minds Disclaimer
This message is for the sole use of the intended recipient(s) and may contain
confidential, proprietary or legally privileged information. Any unauthorized
review, use, disclosure or distribution is prohibited. If you are not the
original intended recipient of the message, please contact the sender by reply
email and destroy all copies of the original message.
Happiest Minds Technologies <http://www.happiestminds.com>
________________________________
--
View this message in context:
http://apache-spark-user-list.1001560.n3.nabble.com/lower-and-upper-offset-not-working-in-spark-with-mysql-database-tp23635p23637.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.
