Re: Bulk loading performance

2015-07-14 Thread David Haguenauer
* Graham Sanderson , 2015-07-13 18:21:08 Mon: > > Is there a set of best practices for this kind of workload? We would > > like to avoid interfering with reads as much as possible. > Ironically in my experience the fastest ways to get data into C* are > considered “anti-patterns” by most (but I hav

Re: Bulk loading performance

2015-07-13 Thread Graham Sanderson
Ironically in my experience the fastest ways to get data into C* are considered “anti-patterns” by most (but I have no problem saturating multiple gigabit network links if I really feel like inserting fast) It’s been a while since I tried some of the newer approaches though (my fast load code i

Bulk loading performance

2015-07-13 Thread David Haguenauer
Hi, I have a use case wherein I receive a daily batch of data; it's about 50M--100M records (a record is a list of integers, keyed by a UUID). The target is a 12-node cluster. Using a simple-minded approach (24 batched inserts in parallel, using the Ruby client), while the cluster is being read a