Re: [E] Re: Memory usage of frequent items datasketches-cpp package

2020-09-16 Thread Andy Dang
would you trigger a compaction? > > On Tue, Sep 15, 2020 at 6:45 PM Andy Dang wrote: > >> Scrap this. Coming from the JVM library I embarrassingly misunderstood >> the size parameter in the Python API (in Java you give the actual size, in >> Python you give the log

Re: Memory usage of frequent items datasketches-cpp package

2020-09-15 Thread Andy Dang
, Sep 15, 2020 at 5:59 PM Andy Dang wrote: > Hi, > > I was running some benchmark with the CPP package and I noticed some > strange memory behavior. I noticed that the memory seems to > increase linearly with the item size when using size 32 or 64. The notebook > si > htt

Memory usage of frequent items datasketches-cpp package

2020-09-15 Thread Andy Dang
Hi, I was running some benchmark with the CPP package and I noticed some strange memory behavior. I noticed that the memory seems to increase linearly with the item size when using size 32 or 64. The notebook si https://suspicious-bassi-380e27.netlify.app/

Re: Segfault when running datasketches

2020-06-29 Thread Andy Dang
? Our testing > generally has generally shown C++ to be faster. > > This is still too vague for me to be able to say much. There's no specific > git version (tag or hash), no code, and no data. > > jon > > On Mon, Jun 29, 2020 at 9:08 AM Andy Dang wrote: > >&g

Re: Segfault when running datasketches

2020-06-29 Thread Andy Dang
> I've compiled the library in debug mode (gotta modify setup.py to force > that) and run python via gdb but that's not gonna work nicely on 1.6gb of > data. It's slwww. > > jon > > > On Fri, Jun 26, 2020, 4:39 PM Andy Dang wrote: > >&g

Segfault when running datasketches

2020-06-26 Thread Andy Dang
Hi all, I've been trying to integrate Datasketches into our ecosystem - really great work! However, when I tried to run various sketches with the lending club data from Kaggle (1.6GB in size) on the raw CSV data in Python on my MacOS. I noticed after a while that the process will crash with a mys