Computing two aggregate functions on the same window

Hi, In the datastream api is there a way to take two aggregate functions and apply them to the same window? The output would be a stream of 2-tuples containing the result of each aggregate function. I feel it should be possible to combine previously written functions rather than writing a bespoke

Accessing pojo fields by name in flink

Hi, So far I’ve been developing my flink pipelines using the datastream API. I have a pipeline that calculates windowed statistics on a given pojo field. Ideally I would like this field to be user configurable via a config file. To do this I would need to extract pojo fields by name. The Table API

Side output in ProcessFunction.onTimer

Hi, Is there a way to make side outputs in an onTimer callback in ProcessFunction? I want to side output events that belong to a session that was below the minimum duration threshold. Currently these events are just discarded but I’d like more traceability. Thanks, Frank

Flink state: complex value state pojos vs explicitly managed fields

Hi, Is it better to have one POJO value state with a collection inside or an explicit state declaration for each member? e.g. MyPojo { long id; List[Foo] foos; // getter / setters omitted } Or Two managed state declarations in my process function (a value for the long and a list fo

Count Window Trigger that only fires once

Hi, Is there a way to make the count window trigger fire only once? I would like a session window to only emit the first element it receives immediately rather than waiting until the watermark passes the end of the window. Thanks, Frank

Reduce key state

Hi, In an unwindowed key stream while using event time semantics is state stored indefinitely or does it get expired eventually (was wondering if the state inherits the event time of the element that updated, and if it expires when the watermark goes past it). Thanks, Frank

Add flink gauge to serialiser ?

Is there a way to add a gauge to a flink serializer? I’d like to calculate and expose the total time to process a given tuple including the serialisation/deserialisation time. Or would it be a better idea to wrap the conctrete sink function (e.g. kafka producer) with an ‘instrumented sink’ adapter

Re: Enabling JMX Reporter on a Local Mini Cluster

nt env) { } And drop in the standard environment or the test environment as appropriate. This is simple enough but I was just emailing this list because at first (before I started experimenting with metrics) it seemed that this wouldn’t be necessary. Thanks, Frank On Fri, 5 Apr 2019 at 16:51, Fra

Enabling JMX Reporter on a Local Mini Cluster

What's the best way to enable the JMX Reporter while I am developing an applicaiton in an IDE? The reason is I would like to experiment with adding detailed metrics to my pipelines (and also see what standard operators provide) without having to deploy to a regular cluster. Thanks, Frank