Connecting to Cloud SQL from Dataflow in scalable manner

Hi Team, Recently I created a streaming pipeline that will connect to the CloudSQL database. I found two approaches to connect DB one is JDBC IO and Connection Pool using c3po. My use case: for each element I need to check any previous data that exists in the database for that particular record,

Custom metrics with OpenCensus in Dataflow

Hi All Currently, I am trying to enable OpenCensus (Stats/Metrics) for a Dataflow job. I am trying to count different types of messages I am consuming from Pubsub and tag them based on input. I followed all the steps listed in the Custome metrics with OpenCensus ( https://cloud.google.com/monitor