[
https://issues.apache.org/jira/browse/IGNITE-529?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14991597#comment-14991597
]
Anton Vinogradov commented on IGNITE-529:
-----------------------------------------
Roman,
Thanks for your pull request.
I'd like to propose you to fix some issues:
1. FlumeStreamer
1.1. Please take a look to Streamers Implementation Guidelines
(https://cwiki.apache.org/confluence/display/IGNITE/Streamers+Implementation+Guidelines).
1.2. Streamer implementation should extend StreamAdapter. User should just
provide own implementation of StreamSingleTupleExtractor and/or
StreamMultipleTupleExtractor interfaces and shouldn't create any subclass of
FlumeStreamer and override transfom() method.
1.3. Streamer shouldn't create cache.
2. IgniteSink:
2.1. It seems that specifyGrid() method is redundant. It would be better to
provide default configuration files that will be used if user didn't specify
own configuration file. It configuration should define client Ignite node.
2.2. Method start() set client mode to 'true'. It would be better allow to user
control all configuration parameters and do not hard code it. Also no need to
check that cluster has any server nodes.
3. IgniteFlumeSinkTest
3.1. What if test will fail for some reason? Ignite node will not stop and will
affect all other test because all new nodes will join to the cluster. Use
stopAllGrids() call in afterTests() method.
3.2. It seems that class contains default IntelliJ Idea javadoc :)
3.3 Waiting for latch without specifying timeout is not agood practice at
tests. Code simmilar to assertTrue(latch.await(10, TimeUnit.SECONDS)); should
be used. But better way is to override getTestTimeout() method to make sure it
sutable for your test.
3.4 Is it possible to check cache size after cache content check? It will be a
good test case to configure flume mock to produce 1000 events and check that
cache size equals 1000.
3.5 EVT_CACHE_OBJECT_PUT listenet should be registered before streamer start,
and unregistered atfer stop.
4. IgniteFlumeSinkTestSuite
4.1. It seems that test suite name just copied and pasted from Kafka Streamer
module :)
P.s. Please take a look to "Coding Guidelines"
(https://cwiki.apache.org/confluence/display/IGNITE/Coding+Guidelines),
especially "Semantic Units" and "JavadocComments" topics. You will be surprised
how easy to read code can be when these Coding Guidelines used ;)
> Implement IgniteFlumeStreamer to stream data from Apache Flume
> --------------------------------------------------------------
>
> Key: IGNITE-529
> URL: https://issues.apache.org/jira/browse/IGNITE-529
> Project: Ignite
> Issue Type: Sub-task
> Components: streaming
> Reporter: Dmitriy Setrakyan
> Assignee: Roman Shtykh
>
> We have {{IgniteDataStreamer}} which is used to load data into Ignite under
> high load. It was previously named {{IgniteDataLoader}}, see ticket
> IGNITE-394.
> See [Apache Flume|http://flume.apache.org/] for more information.
> We should create {{IgniteFlumeStreamer}} which will consume messages from
> Apache Flume and stream them into Ignite caches.
> More details to follow, but to the least we should be able to:
> * Convert Flume data to Ignite data using an optional pluggable converter.
> * Specify the cache name for the Ignite cache to load data into.
> * Specify other flags available on {{IgniteDataStreamer}} class.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
