> It still does make sense if you don't need delivery guarantees and you don't want to run Kafka.
To add a data point, as some talk about Tranquility recently has had me wondering if the project is going to be abandoned, this is our situation. Druid is just an endpoint of a longer at-least-once pipeline. We distribute to Druid through embedded Tranquility as one endpoint of a data-distribution hub. I would rather not have to add Kafka to our stack just to stream data into Druid. The listed benefits of Kafka: exactly-once (we already are at-least-once) no external service (already have one running anyway, and adding kafka would then be a new external service) reading late data (we take care of this in the de-dup/data cleansing process, a process which is necessary even with kafka) I have to assume we're not the only people with an at-least-once system, where the benefits of adding Kafka look to be low. Dyana On 8 May 2018 at 20:05, Gian Merlino <g...@apache.org> wrote: > Hi Stig, > > Tranquility seems to be in the "if it ain't broke don't fix it" stage of > its lifecycle. It still works great with the latest Druid releases. > Although, nobody has seemed to be too interested in taking over its > maintenance and adding new features. At Imply we do still support it for > our users, and if it stopped working with a new Druid release then at this > point we would be motivated to patch it. > > However, most of the community effort into streaming ingestion work in the > past couple of years has gone into Kafka indexing ( > http://druid.io/docs/latest/development/extensions-core/ > kafka-ingestion.html). > It has a number of benefits over Tranquility, namely that it is capable of > exactly once ingestion, can read late data, and doesn't require external > processes (although I can see how the latter one is flipped in the Storm > case). It has also generally been much more active dev-wise than > Tranquility. > > What I usually do is direct people to the Kafka indexing method and stress > its benefits, and then if they still want to use Tranquillity (maybe they > just really don't like Kafka) then they can go for it. It still does make > sense if you don't need delivery guarantees and you don't want to run > Kafka. > > On Tue, May 8, 2018 at 9:50 AM, Stig Rohde Døssing <stigdoess...@gmail.com > > > wrote: > > > Hi Druid devs, > > > > We're using Tranquility in the Storm project to allow users to write to > > Druid from Storm ( > > https://github.com/apache/storm/tree/master/external/storm-druid), and > > we've noticed that the library hasn't had a release for a few years. > > > > Looking at the Druid mailing list and Tranquility issue tracker, it's not > > obvious to me whether Tranquility will be developed going forward ( > > https://lists.apache.org/thread.html/ea60ba5294de5eb39761c8fd1c9a88 > > 54d2ffe4a212394078e0e67f54@%3Cdev.druid.apache.org%3E, > > https://github.com/druid-io/tranquility/pull/233), or whether it is > > compatible with the latest Druid releases. > > > > Are there plans to release a new version of Tranquility any time in the > > near future? Is Tranquility compatible with the newest Druid releases? > Does > > it make sense for Storm to continue to support integration via the > > Tranquility library, or should we go another way (e.g. direct users to > > ingest via Kafka instead)? > > > > Thanks > > > -- Dyana Rose Software Engineer W: www.salecycle.com <http://www.salecycle.com/> [image: Marketing Permissions Service] <https://t.xink.io/Tracking/Index/9LwBAKNtAAAwphkA0>
