Hi Vishal, Currently there is no way to share (user-defined) resources between tasks on the same TM. So I suppose that a singleton is the best way to go for now.
Cheers, Kostas > On Sep 27, 2018, at 3:43 AM, Hequn Cheng <chenghe...@gmail.com> wrote: > > Hi vishal, > > Yes, we can define a static connection to reuse it or implement a connection > pool. Maybe we can also ask the problem in hbase community and see if there > are other better ways. > > Best, Hequn > > > On Thu, Sep 27, 2018 at 12:40 AM Vishal Santoshi <vishal.santo...@gmail.com > <mailto:vishal.santo...@gmail.com>> wrote: > any one? > > On Wed, Sep 26, 2018 at 9:15 AM Vishal Santoshi <vishal.santo...@gmail.com > <mailto:vishal.santo...@gmail.com>> wrote: > We use Hbase extensively and the general pattern we follow is acquiring a > Connection in the open() method of a RichFunction and closing in the close() > method. Of course that implies that if we have a parallelism of n, there will > be n Hbase Connections. We want to use the fact that Hbase connection is > inherently thread safe > http://hbase.apache.org/apidocs/org/apache/hadoop/hbase/client/Connection.html > > <http://hbase.apache.org/apidocs/org/apache/hadoop/hbase/client/Connection.html> > and it is a pretty heavy object to begin with and thus makes sense to share > a Connection across slots in a single TM. We could do it through a a static > singleton pattern but was wondering if and whether there is an established > paradigm for sharing a resource .... > > >
