Hi,

I was checking the code for side input patterns :

https://beam.apache.org/documentation/patterns/side-inputs/
Basically I need multiple side inputs from a  Slowly updating global window
side inputs.

So as per example pipeline is something like this:

PCollectionView<Map> map =
p.apply(GenerateSequence.from(0).withRate(1,
Duration.standardSeconds(5L)))            .apply(
ParDo.of(                    new DoFn<Long, Map<String, String>>() {
                   @ProcessElement                      public void
process(@Element Long input, @Timestamp Instant timestamp,
OutputReceiver<Map<String, String>> o) {
o.output(/* output a map */);                        // also output
another map and a list, is this possible ?                      }
              }))            .apply(
Window.<Map<String, String>>into(new GlobalWindows())
  .triggering(Repeatedly.forever(AfterProcessingTime.pastFirstElementInPane()))
                   .discardingFiredPanes())
.apply(Latest.globally())            .apply(View.asSingleton());


So as an extension of this example from the same DoFn which fetches the
side input, alongside the map, I may also need another Map and another List.
Reason I need to perform this in the same DoFn is that from this function
we query external sources to get the side input and the other side inputs
are also built from the same source.

So I would like to avoid querying external sources multiple times to
generate multiple side inputs from different DoFn and want to use the same
function to generate multiple side inputs.

 Can I achieve this by using  "Tags for multiple outputs" ?

Thanks
Sachin

Reply via email to