Hello Everyone

I have a question, I have on my pipeline one side input that fetches some
configurations from an API endpoint each 30 seconds, my question is this.


I have something similar to what is showed in the side input patterns
documentation

 PCollectionView<Map<String, String>> map =
        p.apply(GenerateSequence.from(0).withRate(1,
Duration.standardSeconds(5L)))
            .apply(
                ParDo.of(
                    new DoFn<Long, Map<String, String>>() {

                      @ProcessElement
                      public void process(
                          @Element Long input,
                          @Timestamp Instant timestamp,
                          OutputReceiver<Map<String, String>> o) {
                        call HTTP endpoint here!!
                      }
                    }))
            .apply(
                Window.<Map<String, String>>into(new GlobalWindows())

.triggering(Repeatedly.forever(AfterProcessingTime.pastFirstElementInPane()))
                    .discardingFiredPanes())
            .apply(Latest.globally())
            .apply(View.asSingleton());

What happens if for example the HTTP endpoint takes time to respond due
some network issues and/or the amount of data. Is this gonna introduce
delays on my main pipeline? Is the main pipeline blocked until the pardo
that processes the side input ends?

I don't care too much about the consistency here, I mean if the
configuration changed in the Time T1 I don't care if some registries with
T2 timestamp are processed with the configuration version of T1.


Regards.

Reply via email to