Hi,

I am trying to count the incoming messages from PubSub using beam pipeline java sdk.

 

class Display extends DoFn<Long,String>
{
@ProcessElement
public void process(ProcessContext c)
{
System.out.println(c.element());
}
}
public class CustomMetrics
{
public static void main(String args[])
{
PipelineOptions options = PipelineFactory.create();
Pipleine pipeline= Pipeline.create(options);
 
pipeline.apply(PubsubIO.readStrings().fromSubscription("PubSub Subscription"))
.apply(Window.<String>into(FixedWindows.of(Duration.standardMinutes(5))).triggering(DefaultTrigger.of()))
.apply(Combine.globally(Count.<String>combineFn()).withoutDefaults())
.apply(ParDo.of(new Display()));
 
pipeline.run().waitUntilFinish();
}
}
 

Able to count the incoming records but the number is not displayed immediatley its populated randomly after some time.

Is there any other way to count the incoming unbounded records using Apache Beam pipeline.

 

https://stackoverflow.com/questions/76681800/count-incoming-unbouned-messages-from-pubsub-using-apache-beam-java

 

Thanks,

Phani Geeth

 

Sent from Mail for Windows

 

Reply via email to