By not specifying an explicit number of shards the runner has the capability to choose the number of shards dynamically but I'm not aware of being able to tell the runner dynamically how many shards it should use.
I know that Dataflow will scale up/down the number of shards based upon the amount of data in the pipeline and the current throughput. On Thu, Nov 7, 2019 at 5:52 AM Truebody, Kyle <[email protected]> wrote: > Hi all, > > > > Is it possible to update a Write IO’s shard count once the job has > already started. I am asking because I would like to be able to increase > or decrease the shard count base on some Stream producers put or get > metrics. > > I read in the java docs that if method *withNumShards() * has argument > of zero then beam will decide the value*, *does anyone know if that value > decided by beam will increase or decrease base on demand. > > > > Thanks in advance > > > > Kyle >
