As standalone mode has the disadvantage that the TaskManager JVM can’t isolate 
different jobs. Does we have any plan to improve this ? 

- Jark Wu 

> 在 2016年6月30日,下午4:19,Kevin Jacobs <kevin.jac...@cern.ch> 写道:
> 
> In my opinion the streaming process can be perfectly simulated on a single 
> node. You can setup a message distribution system like Kafka on a single 
> node, you can run Spark on a single node and the only thing you need to 
> change when running it on a cluster is that you need to change the 
> environment. So there is no need to setup a cluster when testing the 
> streaming process.
> 
> Regards,
> Kevin
> 
> On 30-06-16 09:54, Longda Feng wrote:
>> 
>> This means Standalone mode is just for prototype.
>> But I think we need a lightweight solution for streaming process, standalone 
>> is the best solution. Some times, we need setup a flink cluster on a small 
>> cluster. setup a yarn cluster isn't convenient.
>> (1) in small company, the number of machine is small (2) When a data center 
>> is small, but we still need do some computing in this data center(3) some 
>> machines are in the while-list, they have been authorited to access some 
>> special data or machine, but the number of these machine is small.(4) some 
>> machine has critical data, they can't be shared with others, but the number 
>> of these machine is small.(5) when a team start to learn flink, he will 
>> setup a small cluster firstly, maybe he wo't want to setup a huge system, 
>> perfer to a small system
>> 
>> 
>> regardsLongda
>> 
>> ------------------------------------------------------------------From:Aljoscha
>>  Krettek <aljos...@apache.org>Send Time:2016年6月29日(星期三) 21:48To:封仲淹(纪君祥) 
>> <zhongyan.f...@alibaba-inc.com>; dev <dev@flink.apache.org>Subject:Re: 
>> [Discuss] Why different job's tasks can run in the single process.
>> Hi,
>> yes, you are definitely right that allowing to run multiple user code tasks
>> in the same TaskManager JVM is not good for stability. This mode is still
>> there from the very early days of Flink where Yarn was not yet available.
>> In a production environment I would now recommend to always run one
>> Flink-Yarn cluster per job to get good isolation between different jobs.
>> 
>> Cheers,
>> Aljoscha
>> 
>> On Wed, 29 Jun 2016 at 09:18 Longda Feng <zhongyan.f...@alibaba-inc.com>
>> wrote:
>> 
>>>  hi ,
>>>  Sorry for asking the quest here? Any answer will be apprecated.
>>>  Why different job's tasks can run in the single process. (There are some
>>>  different job's tasks  in one TaskManager).It seems Flink-on-Yarn can let
>>>  different job  run on different process. But for standalone mode, this
>>>  problem still exists.
>>>  Why design Flink like this?The advantage What I can thought is as
>>>  following:(1) All task can share bigger memory pool.(2) The communication
>>>  between the tasks in the same process will be fast.
>>>  But this design will impact to the stability. Flink provide
>>>  User-Define-Function interface, if one of the User-Define-Function crash,
>>>  It maybe crack the whole JVM, If the TaskManager crash, all other job's
>>>  task in this TaskManager will be impacted. Even if the JVM don't crash, but
>>>  maybe lead to some other unexpected problem, what's more this will make the
>>>  code too sophisticated。Normal framework like Spark/Storm/Samza won't run
>>>  different job's tasks in the same process。As one normal user, stability has
>>>  the highest priority.
>>> 
>>>  ThanksLongda
>>> 
>>> 
>>> 
>>> 

Reply via email to