There is an option for writing messages to a local log file, but this is
intended
purely for troubleshooting. Dory's intended purpose is to get messages off
the local machine as quickly and efficiently as possible, and let the Kafka
brokers handle the persistence.
When a client sends a message to Dory, there is no ACK from Dory back to
the client. An ACK isn't needed since Dory's clients are local, and the
operating system guarantees the reliability of the interprocess
communication. However, when Dory forwards the messages to Kafka, it
waits for an ACK and resends messages as necessary due to communication
failures or other errors. Once Dory has received a message from a
client, it
takes full responsibility for reliable delivery to Kafka. This
simplifies clients,
since sending messages becomes a simple one-way "fire it and forget it"
communication process.
Regards,
Dave
On 6/11/2016 7:50 PM, Tauzell, Dave wrote:
Does Dory have the option to persist messages to local disk after the Dory ack
but before sending to Kafka?
Dave
On Jun 11, 2016, at 17:23, Dave Peterson <d...@dspeterson.com> wrote:
Hello Kafka users,
Version 1.1.0 of Dory is now available. See
https://github.com/dspeterson/dory for details. Dory is the successor
to Bruce (https://github.com/tagged/bruce), a Kafka producer daemon I
created while working at if(we) (http://www.ifwe.co/). The code has
seen a number of improvements since its initial release in September
2014. The list of example clients for various programming languages
has also been extended. Dory maintains full backward compatibility
with Bruce, so existing users can easily switch.
The latest release adds support for receiving messages from clients by
UNIX domain stream socket or local TCP. Although UNIX domain
datagrams are still the preferred means of sending messages in most
cases, the option of using stream sockets facilitates sending messages
too large to fit in a single datagram. The local TCP option
facilitates adding support for clients written in programming
languages that do not provide easy access to UNIX domain sockets.
Dory's wiki page http://dory.wikidot.com/start contains a list of
ideas for additional features and other improvements. Community
feedback is welcomed and appreciated. If you have ideas for things
you would like to see in future releases, please add them to the list.
Also, please contribute code if you can afford the time.
Thanks,
Dave Peterson
This e-mail and any files transmitted with it are confidential, may contain
sensitive information, and are intended solely for the use of the individual or
entity to whom they are addressed. If you have received this e-mail in error,
please notify the sender by reply e-mail immediately and destroy all copies of
the e-mail and any attachments.
