Hi Arun,
We have been running into the same issue (having only 1000 unique items, in
100MM transactions), but have not investigated the root cause of this. We
decided to run this on a cluster instead (4*16 / 64GB Ram), after which the OOM
issue went away. However, we ran into the issue that the
g (or avoiding!) deals on frequent itemsets.
> >
> > This is a good resource for frequent itemsets implementations:
> > http://infolab.stanford.edu/~ullman/mmds/ch6.pdf
> >
> > ____________
> > From: rpuj...@hortonworks.com
> > Date
___
> From: rpuj...@hortonworks.com
> Date: Fri, 5 Dec 2014 10:31:17 -0600
> Subject: Re: Market Basket Analysis
> To: so...@cloudera.com
> CC: t...@preferred.jp; user@spark.apache.org
>
>
> This is a typical use case "people who buy electric razors, also tend to buy
&g
ntations:
http://infolab.stanford.edu/~ullman/mmds/ch6.pdf
From: rpuj...@hortonworks.com
Date: Fri, 5 Dec 2014 10:31:17 -0600
Subject: Re: Market Basket Analysis
To: so...@cloudera.com
CC: t...@preferred.jp; user@spark.apache.org
This is a typical use case "people who buy electric razors, also tend to buy
b
t; > spoon to spoon substitute, spoon to fork will suffice too.
> >
> > Hopefully this provides some clarification.
> >
> > Thanks,
> > Rohit
> >
> >
> >
> > From: Tobias Pfeiffer
> > Date: Thursday, December 4, 2014 at 7:20 PM
> >
Rohit
>
>
>
> From: Tobias Pfeiffer
> Date: Thursday, December 4, 2014 at 7:20 PM
> To: Rohit Pujari
> Cc: "user@spark.apache.org"
> Subject: Re: Market Basket Analysis
>
> Hi,
>
> On Thu, Dec 4, 2014 at 11:58 PM, Rohit Pujari
> wrote:
>&
@hortonworks.com>>
Cc: "user@spark.apache.org<mailto:user@spark.apache.org>"
mailto:user@spark.apache.org>>
Subject: Re: Market Basket Analysis
Hi,
On Thu, Dec 4, 2014 at 11:58 PM, Rohit Pujari
mailto:rpuj...@hortonworks.com>> wrote:
I'd like to do market bas
Hi,
On Thu, Dec 4, 2014 at 11:58 PM, Rohit Pujari
wrote:
>
> I'd like to do market basket analysis using spark, what're my options?
>
To do it or not to do it ;-)
Seriously, could you elaborate a bit on what you want to know?
Tobias
