It almost sounds like you're duplicating all the work of both spark and the connector. May I ask why you decided to not use the existing tools?
On Mon, Oct 3, 2016 at 2:21 PM siddharth verma <sidd.verma29.l...@gmail.com> wrote: > Hi DuyHai, > Thanks for your reply. > A few more features planned in the next one(if there is one) like, > custom policy keeping in mind the replication of token range on specific > nodes, > fine graining the token range(for more speedup), > and a few more. > > I think, as fine graining a token range, > If one token range is split further in say, 2-3 parts, divided among > threads, this would exploit the possible parallelism on a large scaled out > cluster. > > And, as you mentioned the JIRA, streaming of request, that would of huge > help with further splitting the range. > > Thanks once again for your valuable comments. :-) > > Regards, > Siddharth Verma >
