bq. be able to lookup from inside MapPartitions based on a key
Please describe your use case in bit more detail.
One possibility is to use NoSQL database such as HBase. There're several
choices for Spark HBase connector.
Cheers
On Tue, Dec 22, 2015 at 4:51 PM, Zhan Zhang wrote:
> SQLContext is
SQLContext is in driver side, and I don’t think you can use it in executors.
How to provide lookup functionality in executors really depends on how you
would use them.
Thanks.
Zhan Zhang
On Dec 22, 2015, at 4:44 PM, SRK wrote:
> Hi,
>
> Can SQL Context be used inside mapPartitions? My requ
