1. Yes those lines are normal.
2. Why not print the results of the filtering in the bolt itself?
void execute() {
if (finished) {print results} else {do the processing of the bolt}
}
If you still want to have a separate program to print the results, then you
could make the bolt write the results of the hashmap to Kafka and the other
program could pick up the results. Or instead of Kafka, write to a database
and then the spout can notify the other program via Kafka, to read the
results. You could also send the values from all bolts to a single bolt and
have the printing code in that bolt (you'll need to use a separate stream
to notify that bolt though. See the part on "More creative topology
structures" here:
http://nrecursions.blogspot.in/2016/05/more-concepts-of-apache-storm-you-need.html
).
3. Thread sleep is used in the spout just to give time for other threads in
storm some processor time. Otherwise, the nextTuple() function will take up
processor time most of the time.On Mon, Aug 22, 2016 at 10:07 PM, Mario Ferrulli <[email protected]> wrote: > Hi, > I'm a newbie of Storm. I'm using it for an University project. I hope that > writing an email is the correct way to receive support (I read that the > Google group is now deprecated). > > I created my topology, with a Spout linked to a MySql database, and two > Bolts. The first bolt, linked to the spout, prepares and removes > information not necessary of the tuples; the second, do a filtering of the > tuple. > > I'm working in local mode. > > My question is: > why after running topology, in my console I see output like the lines > below? > > 38211 [Thread-14-movie-SPOUT] INFO backtype.storm.daemon.executor - >> Processing received message source: __system:-1, stream: __tick, id: {}, >> [30] >> 67846 [Thread-10-__acker] INFO backtype.storm.daemon.executor - >> Processing received message source: __system:-1, stream: __metrics_tick, >> id: {}, [60] >> 67846 [Thread-8-cleaning-genre-bolt] INFO backtype.storm.daemon.executor >> - Processing received message source: __system:-1, stream: __metrics_tick, >> id: {}, [60] >> 67852 [Thread-10-__acker] INFO backtype.storm.daemon.task - Emitting: >> __acker __metrics [#<TaskInfo backtype.storm.metric.api.IMet >> ricsConsumer$TaskInfo@3c270095> [#<DataPoint [__emit-count = {}]> >> #<DataPoint [__process-latency = {}]> #<DataPoint [__receive = {read_pos=0, >> write_pos=1, capacity=1024, population=1}]> #<DataPoint [__ack-count = {}]> >> #<DataPoint [__transfer-count = {}]> #<DataPoint [__execute-latency = {}]> >> #<DataPoint [__fail-count = {}]> #<DataPoint [__sendqueue = {read_pos=-1, >> write_pos=-1, capacity=1024, population=0}]> #<DataPoint [__execute-count = >> {}]>]] >> 67853 [Thread-8-cleaning-genre-bolt] INFO backtype.storm.daemon.task - >> Emitting: cleaning-genre-bolt __metrics [#<TaskInfo >> backtype.storm.metric.api.IMetricsConsumer$TaskInfo@38c3d111> >> [#<DataPoint [__emit-count = {default=1680}]> #<DataPoint >> [__process-latency = {}]> #<DataPoint [__receive = {read_pos=1621, >> write_pos=1622, capacity=1024, population=1}]> #<DataPoint [__ack-count = >> {}]> #<DataPoint [__transfer-count = {default=1680}]> #<DataPoint >> [__execute-latency = {movie-SPOUT:default=0.15476190476190477}]> >> #<DataPoint [__fail-count = {}]> #<DataPoint [__sendqueue = {read_pos=1680, >> write_pos=1680, capacity=1024, population=0}]> #<DataPoint [__execute-count >> = {movie-SPOUT:default=1680}]>]] >> 67854 [Thread-13-filtering-genre-BOLT] INFO >> backtype.storm.daemon.executor - Processing received message source: >> __system:-1, stream: __metrics_tick, id: {}, [60] >> 67855 [Thread-13-filtering-genre-BOLT] INFO backtype.storm.daemon.task >> - Emitting: filtering-genre-BOLT __metrics [#<TaskInfo >> backtype.storm.metric.api.IMetricsConsumer$TaskInfo@6d5c75a9> >> [#<DataPoint [__emit-count = {}]> #<DataPoint [__process-latency = {}]> >> #<DataPoint [__receive = {read_pos=1681, write_pos=1682, capacity=1024, >> population=1}]> #<DataPoint [__ack-count = {}]> #<DataPoint >> [__transfer-count = {}]> #<DataPoint [__execute-latency = >> {cleaning-genre-bolt:default=0.08333333333333333}]> #<DataPoint >> [__fail-count = {}]> #<DataPoint [__sendqueue = {read_pos=-1, write_pos=-1, >> capacity=1024, population=0}]> #<DataPoint [__execute-count = >> {cleaning-genre-bolt:default=1680}]>]] >> > > > I read that these lines after the last tuple processed are to be > considered normal. Isn't it? > > And how can I run other code after the submission of topology? For > example, I want to print the results of my filtering done in the second > bolt, saved in a HashMap. > If I put my code after the line containing the submitTopology() method, > the code is ran before the completion of the tuples. > > The second and last question is: why in every example of Storm, I see in > the Spout > >> "Thread.sleep(1000)"? >> > > I hope my questions are clear. > Thank you in advance! > > Mario Ferrulli > -- Regards, Navin
