Hi Max, You are right that Queryable State is not designed to be used as a means for a job to query its own state. In fact, given that you do not know the jobId of your job from within the job itself, I do not think you can use queryable state in your scenario.
What you can do is to have a flatMap computing the hot keys or heavy hitters, and emit as main output the elements themselves for further processing, and as a side output the computed statistics. The side output is a data stream itself so you can store it in an external storage system (e.g. a KV store) and use AsyncIO to query that system downstream. This will solve the problem of having access to the state from all tasks. This is a simple solution but I am not sure about the performance implications. You can try it to see if it actually fits your needs. Thanks, Kostas > On Dec 5, 2017, at 10:32 AM, Fabian Hueske <fhue...@gmail.com> wrote: > > Hi, > > I haven't done that before either. The query API will change with the next > version (Flink 1.4.0) which is currently being prepared for releasing. > Kostas (in CC) might be able to help you. > > Best, Fabian > > 2017-12-05 9:52 GMT+01:00 m@xi <makisnt...@gmail.com > <mailto:makisnt...@gmail.com>>: > Hi Fabian, > > Thanks for your answer. Initially, I have excluded Queryable State as an > option as it explicitly mentioned that it is used for querying state outside > flink. > > Now that I am reading the documentation I am not sure how I may achieve > that. I have to set ports and addresses which I am not sure I should since I > am reading the queryable state from inside the same job. > > Can you or someone elaborate further how can I read the queryable state of a > specific task from another task (e.g. map). > > Best, > Max > > > > -- > Sent from: > http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/ > <http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/> >
