[
https://issues.apache.org/jira/browse/KAFKA-1995?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15315665#comment-15315665
]
Ewen Cheslack-Postava commented on KAFKA-1995:
----------------------------------------------
[~johnlon] JMS supports acknowledged messages. Message IDs and correlation IDs
could potentially be used as offsets. Given some other guarantees, offsets
don't necessarily need to be comparable and the source system doesn't need to
be able to seek to arbitrary offsets. In the case of JMS, I think you could do
something like use the message ID as the offset and if a Connect offset commit
occurs but you don't have a chance to ack that block of messages, upon resuming
you could consume messages, ignoring anything until you see the ID from the
last commit, then start producing data. This minimizes the number of duplicates
you might see in the destination system as long as you don't ack messages until
Connect offsets have been committed. Alternatively, opt to ignore offsets and
offset commit entirely and rely only on commitRecord
(https://github.com/apache/kafka/blob/trunk/connect/api/src/main/java/org/apache/kafka/connect/source/SourceTask.java#L84)
to trigger acks. Upon faults you can still see duplicates, but since JMS has
fine-grained acking and effectively tracks offsets for you, you may not even
need Connect's offset tracking support. That support is included for systems
that have no other place to store offsets (e.g. if you're reading of a
database's commit log file), but it isn't required that every connector utilize
it.
Strictly speaking you can also create a connector that doesn't use these
features, you just won't have any guarantees about data delivery in the case of
faults.
> JMS to Kafka: Inbuilt JMSAdaptor/JMSProxy/JMSBridge (Client can speak JMS but
> hit Kafka)
> ----------------------------------------------------------------------------------------
>
> Key: KAFKA-1995
> URL: https://issues.apache.org/jira/browse/KAFKA-1995
> Project: Kafka
> Issue Type: Wish
> Components: core
> Affects Versions: 0.9.0.0
> Reporter: Rekha Joshi
>
> Kafka is a great alternative to JMS, providing high performance, throughput
> as scalable, distributed pub sub/commit log service.
> However there always exist traditional systems running on JMS.
> Rather than rewriting, it would be great if we just had an inbuilt
> JMSAdaptor/JMSProxy/JMSBridge by which client can speak JMS but hit Kafka
> behind-the-scene.
> Something like Chukwa's o.a.h.chukwa.datacollection.adaptor.jms.JMSAdaptor,
> which receives msg off JMS queue and transforms to a Chukwa chunk?
> I have come across folks talking of this need in past as well.Is it
> considered and/or part of the roadmap?
> http://grokbase.com/t/kafka/users/131cst8xpv/stomp-binding-for-kafka
> http://grokbase.com/t/kafka/users/148dm4247q/consuming-messages-from-kafka-and-pushing-on-to-a-jms-queue
> http://grokbase.com/t/kafka/users/143hjepbn2/request-kafka-zookeeper-jms-details
> Looking for inputs on correct way to approach this so to retain all good
> features of Kafka while still not rewriting entire application.Possible?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
