Not to derail this thread onto another topic but the problem with using a 
static instance is that there's no way to shut it down when the job stops. So 
if, for example, it starts threads, I don't think those threads will stop when 
the job stops. I'm not very well versed in how various Java 8 implementations 
perform unloading of classloaders & class definitions/statics therein, but it 
seems problematic unless the job provides a shutdown hook to which user code 
can subscribe.




On 9/21/16, 8:05 PM, "David Wang" <dwan...@gmail.com> wrote:

>Hi Shannon,
>
>That's right. This FLIP aims to boost TPS of the task workers with async
>i/o operation.
>
>As what Stephan has mentioned, by placing static attribute to shared
>resources(like event pool, connection), it is possible to share those
>resources among different slots in the same JVM.
>
>I will make a note in the FLIP about how to share resources ;D
>
>Thanks,
>David
>
>2016-09-22 1:46 GMT+08:00 Stephan Ewen <se...@apache.org>:
>
>> @Shannon: One could have a "static" broker to share the same netty across
>> slots in the same JVM. Implicitly, Flink does the same with broadcast
>> variables.
>>
>> On Wed, Sep 21, 2016 at 5:04 PM, Shannon Carey <sca...@expedia.com> wrote:
>>
>> > David,
>> >
>> > I just wanted to say "thanks" for making this proposal! I'm also
>> > interested in performing nonblocking I/O (multiplexing threads/reactive
>> > programming) within Flink operators so that we can, for example,
>> > communicate with external web services with Netty/RxNetty without
>> blocking
>> > an entire Flink slot (aka a thread) while we wait for the operation to
>> > complete. It looks like your FLIP will enable that use case.
>> >
>> > I'm not sure whether it will be possible to share one Netty
>> EventLoopGroup
>> > (or the equivalent for any other non-blocking framework, connection pool,
>> > etc.) among multiple slots in a single JVM though. Flink supports
>> > open/close operation on a RichFunction, but that's on a per-slot basis. I
>> > don't know of a way to open/close objects on a per-job-JVM basis. But I
>> > suppose that's an issue that should be discussed and resolved separately.
>> >
>> > -Shannon
>> >
>> >
>>

Reply via email to