Re: Interested in contributing to GraphX in Python

2014-12-16 Thread GregBowyer
I have been thinking about this for a little while and I wonder if it makes sense to look at forcing off heap mmap storage what can be shared with python. The idea would be that java makes a DirectByteBuffer (or similar) with python doing memoryview over that buffer. Then for all except for real

Re: Interested in contributing to GraphX in Python

2014-08-04 Thread Reynold Xin
Thanks for your interest. I think the main challenge is if we have to call Python functions per record, it can be pretty expensive to serialize/deserialize across boundaries of the Python process and JVM process. I don't know if there is a good way to solve this problem yet. On Fri, Aug 1, 2