Hi mates, I got a proposal about functionality of BucketingSink. During implementation of one of our tasks we got the following need - create a meta-file, with the path and additional information about the file, created by BucketingSink, when it’s been moved into final place. Unfortunately such behaviour is currently not available for us.
We’ve implemented our own Sink, that provides an opportunity to register notifiers, that will be called, when file state is changing, but current API doesn’t allow us to add such behaviour using inheritance ... It seems, that such functionality could be useful, and could be a part of BucketingSink API What do you sink, should I make a PR ? Sincerely yours, Rinat Sharipov Software Engineer at 1DMP CORE Team email: r.shari...@cleverdata.ru <mailto:a.totma...@cleverdata.ru> mobile: +7 (925) 416-37-26 CleverDATA make your data clever
