Hi,
I am Hwanju at AWS Kinesis Analytics. We would like to start a discussion
thread about a project we consider for Flink operational improvement in
production. We would like to start conversation early before detailed design,
so any high-level feedback would welcome.
For service providers wh
Hi Thomas,
I have a sort of question regarding the class loader issue, as it seems
interesting.
My understanding is that at least user class loader is unregistered and
re-registered (from/to library cache on TM) across task restart. If I
understand it correctly, unregistered one should be GCed
art would
be to determine whether this is caused by user or not, but probably some simple
stack trace probing on back pressured task once every N seconds should solve
this - similar how sampling profilers work.
Luckily it seems like those four issues/proposals could be
implemented/discusse
1 from
my side to split the discussion into separate threads.
Piotrek
> On 17 May 2019, at 08:57, Kim, Hwanju wrote:
>
> Hi Piotrek,
>
> Thanks for insightful feedback and indeed you got most tricky parts and
concerns.
>
>> 1. D
Hi!
In case where new state type is incompatible with old one from savepoint, we
get IllegalStateException via org.apache.flink.util.Preconditions.checkState
from checkStateMetaInfo:
https://github.com/apache/flink/blob/master/flink-runtime/src/main/java/org/apache/flink/runtime/state/Registered
entirely sure how to
discern all possible errors into user and framework faults.
I am pulling in Gordon who worked on this part in the past and might be
able to give us some more details on the usage of the
StateMigrationException.
Cheers,
Till
On Wed, Oct 7, 2020
er and know the
content is safe.
Hi,
Thanks for the discussions so far. I agree that the use of the
StateMigrationException and it's semantics is rather undefined and
inconsistent as of the current state.
On Thu, Oct 8, 2020 at 7:38 AM Kim, Hwanju wrote:
> Hi T
