Oh and I'd also need some way to clean up the per-node transient state if the topology stops running on a specific node.
On Wed, 24 Jul 2019 at 08:18, Stephen Connolly < stephen.alan.conno...@gmail.com> wrote: > Hi, > > So we have a number of nodes in our topology that need to do things like > checking a database, e.g. > > * We need a filter step to drop events on the floor from systems we are no > longer interested in > * We need a step that outputs on a side-channel if the event is for an > object where the parent is not currently known to the database. > > Right now we are grabbing a JDBC connection for each node in the topology > that needs to talk to the database and storing the connection in a > transient field (to exclude it from the serialized state) > > What I'd really like to do is have a JDBC connection pool shared across > the entire topology as that way we could have the pool check for stale > connections, etc. > > Does anyone have any tips for doing this kind of thing? > > (My current idea is to maintain a `static final > WeakHashMap<ClassLoader,ConnectionPool>` in the main class... but that > feels very much like a hack) > > What I'm really looking for is some form of Node Transient State... are > there any examples of this type of think. > > Flink 1.8.x > > Thanks, > > -Stephen >
