@Shushant: In my case, the receivers will be fixed till the end of the application. This one's for Kafka case only, if you have a filestream application, you will not have any receivers. Also, for kafka, next time you run the application, it's not fixed that the receivers will get launched on the same machines.
@bit1129, If the receiver is crashed, it will try to restart the receiver, since i have only 2 machines on that cluster, it will either restart on the same node or on the other node. Thanks Best Regards On Wed, Apr 15, 2015 at 4:30 PM, bit1...@163.com <bit1...@163.com> wrote: > Hi, Akhil, > > I would ask a question here: Assume Receiver-0 is crashed, will it be > restarted on other worker nodes(In your picture, there would be 2 receivers > on the same node) or will it start on the same node? > > ------------------------------ > bit1...@163.com > > > *From:* Akhil Das <ak...@sigmoidanalytics.com> > *Date:* 2015-04-15 19:12 > *To:* Shushant Arora <shushantaror...@gmail.com> > *CC:* user <user@spark.apache.org> > *Subject:* Re: spark streaming with kafka > Once you start your streaming application to read from Kafka, it will > launch receivers on the executor nodes. And you can see them on the > streaming tab of your driver ui (runs on 4040). > > [image: Inline image 1] > > These receivers will be fixed till the end of your pipeline (unless its > crashed etc.) You can say, eah receiver will run on a single core. > > Thanks > Best Regards > > On Wed, Apr 15, 2015 at 3:46 PM, Shushant Arora <shushantaror...@gmail.com > > wrote: > >> Hi >> >> I want to understand the flow of spark streaming with kafka. >> >> In spark Streaming is the executor nodes at each run of streaming >> interval same or At each stream interval cluster manager assigns new >> executor nodes for processing this batch input. If yes then at each batch >> interval new executors register themselves as kafka consumers? >> >> Even without kafka is executor nodes on each batch interval same or >> driver nodes gets new executor nodes from cluster manager ? >> >> Thanks >> Shushant >> > > [image: 提示图标] 邮件带有附件预览链接，若您转发或回复此邮件时不希望对方预览附件，建议您手动删除链接。 > 共有 *1* 个附件 > image.png(25K) 极速下载 > <http://preview.mail.163.com/xdownload?filename=image.png&mid=1tbiShXcgVO-s0wyewAAsY&part=3&sign=de80f80932aeee7d4eea5a9e5f4b755e&time=1429095438&uid=bit1129%40163.com> > 在线预览 > <http://preview.mail.163.com/preview?mid=1tbiShXcgVO-s0wyewAAsY&part=3&sign=de80f80932aeee7d4eea5a9e5f4b755e&time=1429095438&uid=bit1129%40163.com> > >
