Hi Mike, 1. Depending on what you need the side output for, you can use metrics to track some things.
But yes, that’s a very limited subset of all use cases. 2. As you mentioned, you could output a combo record. Using an Either <https://ci.apache.org/projects/flink/flink-docs-master/api/java/org/apache/flink/types/Either.html><regular output type, side output type> is a common approach. But you don’t need a process function, you can use the .split() operator to create a SplitStream, and then .select() either the regular or the “side output” stream. — Ken > On Mar 11, 2019, at 9:40 AM, Mikhail Pryakhin <m.prya...@gmail.com> wrote: > > Hello Flink experts! > My streaming pipeline makes async IO calls via the recommended AsyncFunction. > The pipeline evolves and I've encountered a requirement to side output > additional events from the function. > As it turned out the side output feature is only available in the following > functions: > ProcessFunction > CoProcessFunction > ProcessWindowFunction > ProcessAllWindowFunction > > Just curious whether there is any known approach of utilizing "side output" > feature from the AsyncFunction? > > From my perspective, the only feasible way of solving this task is to output > a processing event and an optional "side output" as a tuple from the > AsyncFunction and then process the resulting stream with a ProcessFunction to > handle the required logic. But this approach seems to be overcomplicated. > > Am I missing something? Any help/ideas are much appreciated! > > Cheers, > Mike Pryakhin > -------------------------- Ken Krugler +1 530-210-6378 http://www.scaleunlimited.com Custom big data solutions & training Flink, Solr, Hadoop, Cascading & Cassandra
