The upshot is that there has to be a way to do this with some custom code to at least have the ability to 'fast path' the code without reflection. Using lambdas should make this fairly syntactically unobtrusive.
On Mon, Jun 12, 2017 at 4:02 PM, Arun Mohan <strider90a...@gmail.com> wrote: > Yes, reflection is not very performant but I don't think I have any other > choice since the library has to inspect the object supplied by the client > at runtime to pick out the methods to be invoked using CompletableFuture. > But the performance penalty paid for using reflection will be more than > offset by the savings of parallel method execution, more so as the no of > methods executed in parallel increases. > > On Mon, Jun 12, 2017 at 3:21 PM, Gary Gregory <garydgreg...@gmail.com> > wrote: > > > On a lower-level, if you want to use this for lower-level services (where > > there is no network latency for example), you will need to avoid using > > reflection to get the best performance. > > > > Gary > > > > On Mon, Jun 12, 2017 at 3:15 PM, Arun Mohan <strider90a...@gmail.com> > > wrote: > > > > > Hi Gary, > > > > > > Thanks for your response. You have some valid and interesting points > :-) > > > Of course you are right that Spark is much more mature. Thanks for your > > > insight. > > > It will be interesting indeed to find out if the core parallelization > > > engine of Spark can be isolated like you suggest. > > > > > > I started working on this project because I felt that there was no good > > > library for parallelizing method calls which can be plugged in easily > > into > > > an existing java project. Ultimately, if such a solution can be > > > incorporated in the Apache Commons, it would be a useful addition to > the > > > Commons repository. > > > > > > Thanks, > > > Arun > > > > > > > > > > > > On Mon, Jun 12, 2017 at 3:01 PM, Gary Gregory <garydgreg...@gmail.com> > > > wrote: > > > > > > > Hi Arun, > > > > > > > > Sure, and that is to be expected, Spark is more mature than a four > > class > > > > prototype. What I am trying to get to is that in order for the > library > > to > > > > be useful, you will end up with more in a first release, and after a > > > couple > > > > more releases, there will be more and more. Would Spark not have in > its > > > > guts the same kind of code your are proposing here? By extension, > will > > > you > > > > not end up with more framework-like (Spark-like) code and solutions > as > > > > found in Spark? I am just playing devil's advocate here ;-) > > > > > > > > > > > > What would be interesting would be to find out if there is a core > part > > of > > > > Spark that is separable and ex tractable into a Commons component. > > Since > > > > Spark has a proven track record, it is more likely, that such a > library > > > > would be generally useful than one created from scratch that does not > > > > integrate with anything else. Again, please do not take any of this > > > > personally, I am just playing here :-) > > > > > > > > Gary > > > > > > > > > > > > On Mon, Jun 12, 2017 at 2:29 PM, Matt Sicker <boa...@gmail.com> > wrote: > > > > > > > > > I already see a huge difference here: Spark requires a bunch of > > > > > infrastructure to be set up, while this library is just a library. > > > > Similar > > > > > to Kafka Streams versus Spark Streaming or Flink or Storm or Samza > or > > > the > > > > > others. > > > > > > > > > > On 12 June 2017 at 16:28, Gary Gregory <garydgreg...@gmail.com> > > wrote: > > > > > > > > > > > On Mon, Jun 12, 2017 at 2:26 PM, Arun Mohan < > > strider90a...@gmail.com > > > > > > > > > > wrote: > > > > > > > > > > > > > Hi All, > > > > > > > > > > > > > > Good afternoon. > > > > > > > > > > > > > > I have been working on a java generic parallel execution > library > > > > which > > > > > > will > > > > > > > allow clients to execute methods in parallel irrespective of > the > > > > number > > > > > > of > > > > > > > method arguments, type of method arguments, return type of the > > > method > > > > > > etc. > > > > > > > > > > > > > > Here is the link to the source code: > > > > > > > https://github.com/striderarun/parallel-execution-engine > > > > > > > > > > > > > > The project is in a nascent state and I am the only contributor > > so > > > > > far. I > > > > > > > am new to the Apache community and I would like to bring this > > > project > > > > > > into > > > > > > > Apache and improve, expand and build a developer community > around > > > it. > > > > > > > > > > > > > > I think this project can be a sub project of Apache Commons > since > > > it > > > > > > > provides generic components for parallelizing any kind of > > methods. > > > > > > > > > > > > > > Can somebody please guide me or suggest what other options I > can > > > > > explore > > > > > > ? > > > > > > > > > > > > > > > > > > > Hi Arun, > > > > > > > > > > > > Thank you for your proposal. > > > > > > > > > > > > How would this be different from Apache Spark? > > > > > > > > > > > > Thank you, > > > > > > Gary > > > > > > > > > > > > > > > > > > > > > > > > > > Thanks, > > > > > > > Arun > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > Matt Sicker <boa...@gmail.com> > > > > > > > > > > > > > > >
