although the logic should be col1 <> a && col(1) <> b
to exclude both
Like
df.filter('transactiontype > " ").filter(not('transactiontype ==="DEB") &&
not('transactiontype
==="BGC")).select('transactiontype).distinct.collect.foreach(println)
HTH
Dr Mich Talebzadeh
LinkedIn *
https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw
<https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw>*
http://talebzadehmich.wordpress.com
*Disclaimer:* Use it at your own risk. Any and all responsibility for any
loss, damage or destruction of data or any other property which may arise
from relying on this email's technical content is explicitly disclaimed.
The author will in no case be liable for any monetary damages arising from
such loss, damage or destruction.
On 7 August 2016 at 16:53, Mich Talebzadeh <[email protected]>
wrote:
> try similar to this
>
> df.filter(not('transactiontype ==="DEB") || not('transactiontype ==="CRE"))
>
> HTH
>
>
> Dr Mich Talebzadeh
>
>
>
> LinkedIn *
> https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw
> <https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw>*
>
>
>
> http://talebzadehmich.wordpress.com
>
>
> *Disclaimer:* Use it at your own risk. Any and all responsibility for any
> loss, damage or destruction of data or any other property which may arise
> from relying on this email's technical content is explicitly disclaimed.
> The author will in no case be liable for any monetary damages arising from
> such loss, damage or destruction.
>
>
>
> On 7 August 2016 at 15:43, Divya Gehlot <[email protected]> wrote:
>
>> Hi,
>> I have use case where I need to use or[||] operator in filter condition.
>> It seems its not working its taking the condition before the operator and
>> ignoring the other filter condition after or operator.
>> As any body faced similar issue .
>>
>> Psuedo code :
>> df.filter(col("colName").notEqual("no_value") ||
>> col("colName").notEqual(""))
>>
>> Am I missing something.
>> Would really appreciate the help.
>>
>>
>> Thanks,
>> Divya
>>
>
>
