Hi Ron, I'm not too familiar with Flink, but it sounds like an ideal environment for sketching. It also might make sense to have our DataSketches library integrated into the core of Flink so that all users could take advantage of it. Do you know or are in touch with any of the core committers on the Flink team? Perhaps you could recommend who we could reach out to.
I am curious what the "friction" is in terms of your use of the CpcSketch. If there is any way we can help please let us know. Lee. On Fri, Apr 24, 2020 at 3:26 PM Ron Crocker <rcroc...@newrelic.com> wrote: > Hi Lee, > > Thanks for such a cogent response. All of that makes a lot of sense to me. > > I’m not at all considering making improvements, rather I’m merely wanting > to use CppSketch within a Flink job as state. Making the class final adds a > little friction to doing so, but not an overwhelming amount. > > Ron > > On Apr 24, 2020, at 3:12 PM, leerho <lee...@gmail.com> wrote: > > Hi Ron, > > Our mission is to develop a robust sketch library *product* that can be > used in production systems in many different environments and be high > performing and binary compatible across languages and systems. > > - To be able to achieve this mission with our very limited resources, > we have to be careful about making the surface area of what we have to > support too large. Making classes final and/or private are some of the > ways to reduce the size of the support surface area. > > > - We have found that robust sketch software that is usable in > production environments can be very tricky and even though we have been > doing this for a number of years, we continue to discover new ways that > these probabilistic algorithms can behave in totally unexpected ways. > Also, testing and validating these probabilistic algorithms can be very > tricky. Making what appears to be a simple change to a class can have > major impact on its testability or stability. Making a class final is one > way to communicate that the code and its testing counterparts is more > complex than you might think. > > > - We have had experience with a number of folks that have tried to > "improve" these sketches on their own, but with disastrous results. Making > a class final is one way to communicate that we do not recommend users to > attempt to extend these clases on their own. > > > - Making a class final (or private) also gives us, as developers, > additional degrees of freedom in terms of making necessary changes and > improvements to the internals of the class because we know that there are > no other classes that depend on it. > > > - From a owner-developer's point of view, making a class final is > conservative. We can always remove the restriction in the future if the > need arises. But once a class is no longer final, it can never be put > back-in-the-box, so to speak. > > We are not in a position to support any external changes to our code that > we release, nor can we support extensions to our code that are not part of > our library. Making classes final is one way of communicating that we do > not encourage modifications or arbitrary extensions to our code base. > > The DataSketches library is an active open source Apache project. We > encourage users to make suggestions and submit pull requests and contribute > to the library to make it a better product for everyone. We believe we > are very social and open to new ideas. We have a growing community of > interested users, developers and scientists that want to use our library > and make it even better. We actively monitor our communication channels > and respond to sincere requests for help as promptly as we can. > > If you have ideas about added capabilities that you feel would be valuable > extensions to the library, please engage with us through our community > <https://datasketches.apache.org/docs/Community/index.html> mailing lists > or Slack, we would like to hear from you. > > As open source, you are always free to fork the library and do whatever > you want with the code subject to the Apache license. But then, you are on > your own :) > > Cheers, > > Lee. > > > > > > > > > > > > On Thu, Apr 23, 2020 at 4:27 PM Ron Crocker <rcroc...@newrelic.com> wrote: > >> It seems anti-social to make these classes, particularly CpcSketch, >> final. Is there a good reason for that? >> >> Ron > > >
