I’m not sure you can do this for the purposes of consuming messages faster, but I’m interested to hear if I’m mistaken.
You could spawn threads to do other work within a job though. > On Nov 18, 2015, at 11:40 PM, Chen Song <chen.song...@gmail.com> wrote: > > Thanks Navina > > So theoretically I can create a thread pool within a container. I know it > is very hacky but it should increase parallelism. > > Chen > > On Mon, Nov 16, 2015 at 5:49 PM, Navina Ramesh <nram...@linkedin.com> wrote: > >> Hi Chen, >> Samza container is still single threaded. In case of yarn based deployment, >> Samza uses this config value to verify that the cluster has sufficient >> capacity to support running your job. >> >> Apart from this verification, I don't believe we utilize this config value. >> If you set it to > 1, it won't have any effect on the Samza job execution >> itself. However, you may end-up under-utilizing your Yarn cluster >> resources. >> >> HTH! >> Navina >> >> On Mon, Nov 16, 2015 at 2:32 PM, Chen Song <chen.song...@gmail.com> wrote: >> >>> According to the documentation, each Samza container is single threaded. >>> Why giving yarn.container.cpu.cores as a config option and what is the >>> implication >>> to set this to a value > 1? >>> >>> -- >>> Chen Song >>> >> >> >> >> -- >> Navina R. >> > > > > -- > Chen Song
signature.asc
Description: Message signed with OpenPGP using GPGMail
