Something like this?
val broadcasted = sc.broadcast(...)
RDD2.filter(value => {
//simply use *broadcasted*
if(broadcasted.contains(value)) true
})
Thanks
Best Regards
On Fri, Dec 4, 2015 at 10:43 PM, Abhishek Shivkumar <
abhishek.shivku...@bigdatapartnership.com> wrote:
> Hi,
>
> I have RDD1 that is broadcasted.
>
> I have a user defined method for the filter functionality of RDD2, written
> as follows:
>
> RDD2.filter(my_func)
>
>
> I want to access the values of RDD1 inside my_func. Is that possible?
> Should I pass RDD1 as a parameter into my_func?
>
> Thanks
> Abhishek S
>
> *NOTICE AND DISCLAIMER*
>
> This email (including attachments) is confidential. If you are not the
> intended recipient, notify the sender immediately, delete this email from
> your system and do not disclose or use for any purpose.
>
> Business Address: Eagle House, 163 City Road, London, EC1V 1NR. United
> Kingdom
> Registered Office: Finsgate, 5-7 Cranwood Street, London, EC1V 9EE. United
> Kingdom
> Big Data Partnership Limited is a company registered in England & Wales
> with Company No 7904824
>
