Re: HTTP-based Elasticsearch system producer and reusable task

2016-02-10 Thread Roger Hoover
Hi Yi, Please see my comment inline. On Tue, Feb 9, 2016 at 10:08 PM, Yi Pan wrote: > Hi, Roger, > > Got it! I would like to understand more on the SystemProducer API changes > required by #1 and #2. Could you elaborate a bit more? > > For #1, IndexRequestFactory interface ( https://github.com/

Re: HTTP-based Elasticsearch system producer and reusable task

2016-02-09 Thread Yi Pan
Hi, Roger, Got it! I would like to understand more on the SystemProducer API changes required by #1 and #2. Could you elaborate a bit more? Regarding to JDK8 required in the new HTTP-based Elasticsearch producer, I want to ask how you are motivated to go w/ JDK8. It does bring a lot more nice fea

Re: HTTP-based Elasticsearch system producer and reusable task

2016-02-09 Thread Roger Hoover
Hi Yi, It could be merged into the Samza project if there's enough interest but may need some re-working depending on which dependencies are ok to bring in. I did it outside of the Samza project first because I had to get it done quickly so it relies on Java 8 features, dropwizard metrics for his

Re: HTTP-based Elasticsearch system producer and reusable task

2016-02-09 Thread Yi Pan
Hi, Roger, That's awesome! Are you planning to submit the HTTP-based system producer in Samza open-source samza-elasticsearch module? If ElasticSearch community suggest that HTTP-based clients be the recommended way, we should use it in samza-elasticsearch as well. And what's your opinion on the e

HTTP-based Elasticsearch system producer and reusable task

2016-02-09 Thread Roger Hoover
Hi Samza folks, For people who want to use HTTP to integrate with Elasticsearch, I wrote an HTTP-based system producer and a reusable task, including latency stats from event origin time, task processing time, and time spent talking to Elasticsearch API. https://github.com/quantiply/rico/blob/mas