Remove subsets from FP Growth output

2020-12-02 Thread Aditya Addepalli
Hi, Is there a good way to remove all the subsets of patterns from the output given by FP Growth? For example if both the patterns pass the confidence and support thresholds: [Attribute1 = A, Attribute2 = B] -> [Output=C] [Attribute1 = A] -> [Output=C] I want to choose only [Attribute1 = A] ->

Re: Distributed Anomaly Detection using MIDAS

2020-06-27 Thread Aditya Addepalli
Hi Shivin, I'm interested in collaborating with you on this project. I have been using pyspark for a while now and quite familiar with it. Do you have any plan on how to proceed? Thanks, Aditya On Sat, 27 Jun, 2020, 2:58 pm Shivin Srivastava, wrote: > Hi All, > > I have recently been explori

Re: Spark FP-growth

2020-05-07 Thread Aditya Addepalli
d. Not sure how to optimize that. > > On Thu, May 7, 2020, 1:12 PM Aditya Addepalli wrote: > >> Hi Sean, >> >> 1. >> I was thinking that by specifying the consequent we can (somehow?) skip >> the confidence calculation for all the other consequents. >>

Re: Spark FP-growth

2020-05-07 Thread Aditya Addepalli
mputing all that support; how > would you optimize it even if you knew the consequent you cared about? > maybe there's a way, sure, I don't know the code well but it wasn't > obvious at a glance how to take advantage of it. > > I can see how limiting the rule size could

Re: Spark FP-growth

2020-05-07 Thread Aditya Addepalli
Hi, I understand that this is not a priority with everything going on, but if you think generating rules for only a single consequent adds value, I would like to contribute. Thanks & Regards, Aditya On Sat, May 2, 2020 at 9:34 PM Aditya Addepalli wrote: > Hi Sean, > > I un

Re: Spark FP-growth

2020-05-02 Thread Aditya Addepalli
n the data. Sometimes this can be 1e-4 or 1e-5, so my minSupport has to be less than that to capture the rules for that consequent. Thanks for your reply. Let me know what you think. Regards. Aditya Addepalli On Sat, 2 May, 2020, 9:13 pm Sean Owen, wrote: > You could just filter the i

Spark FP-growth

2020-05-02 Thread Aditya Addepalli
I am not sure that is feasible. I am willing to work on these suggestions, if someone thinks they are feasible. Thanks to the dev team for all the hard work! Regards, Aditya Addepalli