Re: Re:Apache Flink - Operator name and uuid best practices

2019-11-21 Thread M Singh
Thanks so much Congxian for your pointers - I will try to go through them.  Mans On Thursday, November 21, 2019, 07:26:49 AM EST, Congxian Qiu wrote: Hi1. I think this issue[1] can help to understand the directory layout2. chk-6 directory or the metafilePath, for more information, you

Re: Re:Apache Flink - Operator name and uuid best practices

2019-11-21 Thread Congxian Qiu
Hi 1. I think this issue[1] can help to understand the directory layout 2. chk-6 directory or the metafilePath, for more information, you can ref to[2][3] 3. every checkpoint contains a meta-file record such, maybe you can ref to[4], and debug it for more information 4. currently, you need to go th

Re: Re:Apache Flink - Operator name and uuid best practices

2019-11-21 Thread M Singh
Hi Congxian: For my application i see many uuids under the chk-6 directory ( I posted one in the sample above).   I am trying to understand that if I restart the application with this checkpoint (which I believe I can just like a savepoint - I am using chk-6 as an example below)1. I believe eac

Re: Re:Apache Flink - Operator name and uuid best practices

2019-11-20 Thread Congxian Qiu
Hi Currently, the last part (uuid a4d87cda-2afd-47d4-8d3f-b0658466fb2d) is generated by UUID.randomUUID(), so there is not a easy way to map this to the assigned in the application. In another word, the last part (uuid a4d87cda-2afd-47d4-8d3f-b0658466fb2d) belongs to one checkpoint, and the assign

Re: Re:Apache Flink - Operator name and uuid best practices

2019-11-20 Thread M Singh
Hi Arvid: Thanks for your clarification. I am giving supplying uid for the stateful operators and find the following directory structure on in the chkpoint directory: f4e78cb47f9dc12859558be7d15f39d0/chk-6/a4d87cda-2afd-47d4-8d3f-b0658466fb2d The first part f4e78cb47f9dc12859558be7d15f39d0 is th

Re: Re:Apache Flink - Operator name and uuid best practices

2019-11-16 Thread M Singh
Thanks Jiayi for your response. I am thinking on the same lines.   Regarding using the same name and uuid, I believe the checkpoint state for an operator will be easy to identify if the uuid is the same as name.  But I am not sure if having a very long name and uuid or a character like parenthes

Re:Apache Flink - Operator name and uuid best practices

2019-11-16 Thread Jiayi Liao
Hi Mans! Firstly let’s see how operator’s name and uid is used. AFAIK, operator’s name is used in WebUI and metrics reporting, and uid is used to mark the uniqueness of operator which is useful when you’re using state[1]. > Are there any restrictions on the length of the name and uuid att