Re: Question on zero downtime deployment of Samza jobs

2016-01-31 Thread Peter Huang
Hi Pan, Really appreciate your thoroughgoing review of the design doc. Your suggestions on every aspect actually bring me back to lot of thought. I have some small feedbacks below. - YARN-based host-affinity is available in Samza 0.10 now. Does that mean that option-2 is available and would meet

samza gc tuning, what about serial + serial old?

2016-01-31 Thread Liu Bo
Hi group We are trying to migrate our current streaming pipeline to samza. Our pipeline has several NLP modules, such as segment, POS, and a lot of score calculation. Each process normally needs 8~10GB memory. Our goal is high throughput so we use Parallel Scavenge + Parallel Old in our current s