[
https://issues.apache.org/jira/browse/IGNITE-529?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14998425#comment-14998425
]
Anton Vinogradov commented on IGNITE-529:
-----------------------------------------
Roman, I've downloaded your pull request an tried to run test but gain
java.lang.NullPointerException: Ouch! Argument cannot be null: Ignite config
file.
Could you please check this issue.
Correct me If I wrong.
Sink will be created inside Flume and use FlumeStreamer to put data to Ignite.
I this case Extractor should be specified inside FlumeStreamer (not as a
constructor parameter, streamer should decide what transformer it should use or
custom implementation can have custom transformer). I mean Flume & Ignite logic
shoul be separated. All Flume related logic inside Sink & and all Ignite
related inside FlumeStreamer.
DataStreamer also should not be provided by Sink, because it Ignite-related
logic.
In case it impossible to provide configured instance of FlumeStreamer to Sink,
FlumeStreamer should build themself using some configuration, possible provided
via constructor.
Sink in my view is just a channel and it should communicate with FlumeStreamer
only without usage of other apache.ignite.* classes.
FlumeStreamer writeEvent methood seems to be reduidndant.
Thoughts?
> Implement IgniteFlumeStreamer to stream data from Apache Flume
> --------------------------------------------------------------
>
> Key: IGNITE-529
> URL: https://issues.apache.org/jira/browse/IGNITE-529
> Project: Ignite
> Issue Type: Sub-task
> Components: streaming
> Reporter: Dmitriy Setrakyan
> Assignee: Roman Shtykh
>
> We have {{IgniteDataStreamer}} which is used to load data into Ignite under
> high load. It was previously named {{IgniteDataLoader}}, see ticket
> IGNITE-394.
> See [Apache Flume|http://flume.apache.org/] for more information.
> We should create {{IgniteFlumeStreamer}} which will consume messages from
> Apache Flume and stream them into Ignite caches.
> More details to follow, but to the least we should be able to:
> * Convert Flume data to Ignite data using an optional pluggable converter.
> * Specify the cache name for the Ignite cache to load data into.
> * Specify other flags available on {{IgniteDataStreamer}} class.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
