Re: Cassandra multiprocessing can't pickle _thread.lock objects

2016-06-21 Thread dieter
Daiyue Weng writes: > ... > I tried to use Cassandra and multiprocessing to insert rows (dummy data) > concurrently based on the examples in > ... > self.pool = Pool(processes=process_count, initializer=self._setup, > initargs=(session,)) > > I am wondering how to resolve the issue. "pickle" is u

RE: Cassandra multiprocessing can't pickle _thread.lock objects

2016-06-21 Thread Joaquin Alzola
>I tried to use Cassandra and multiprocessing to insert rows (dummy data) >concurrently based on the examples in Same situation here. I was going to try this solution but on the reading side. Was even thinking of putting Spark-Cassandra just to avoid Python to do multiprocessing. This email is c

Cassandra multiprocessing can't pickle _thread.lock objects

2016-06-21 Thread Daiyue Weng
I tried to use Cassandra and multiprocessing to insert rows (dummy data) concurrently based on the examples in http://www.datastax.com/dev/blog/datastax-python-driver-multiprocessing-example-for-improved-bulk-data-throughput This is my code class QueryManager(object): concurrency = 100 # chose