Gian,
is code for ignite-429 in master branch?
On Mon, Sep 28, 2015 at 5:27 PM, chandresh pancholi <
chandreshpancholi...@gmail.com> wrote:
> Gian,
>
> Lets use this and modify our tests. I will push the code by tomorrow EOD.
>
> Thanks
> Chandresh Pancholi
>
> On Mon, Sep 28, 2015 at 3:58 PM, Gianfranco Murador <
> murador.gianfra...@gmail.com> wrote:
>
>> Chandresh,
>> I think that the suggestion of Roman is preferable, we can use the
>> following code :
>> Let me know if you have problems
>>
>> /** Start a simulated topology */
>> public void startTopology(StormStreamer stormStreamer) {
>> MkClusterParam mkClusterParam = new MkClusterParam();
>> mkClusterParam.setSupervisors(4);
>> Config daemonConf = new Config();
>> daemonConf.put(Config.STORM_LOCAL_MODE_ZMQ, false);
>> mkClusterParam.setDaemonConf(daemonConf);
>>
>> Testing.withSimulatedTimeLocalCluster(mkClusterParam, new
>> TestJob()
>> {
>> @Override
>> public void run(ILocalCluster cluster) throws
>> IOException {
>> TopologyBuilder builder = new TopologyBuilder();
>>
>> builder.setSpout("testSpout", new StormSpout());
>>
>> builder.setBolt("bolt", stormStreamer)
>> .shuffleGrouping("spout");
>>
>> StormTopology topology = builder.createTopology();
>>
>> MockedSources mockedSources = new MockedSources();
>>
>> //Our spout will be processing this values.
>> mockedSources.addMockData("testSpout",new
>> Values(stormStream.getKeyValMap()));
>>
>>
>> // prepare the config
>> Config conf = new Config();
>> conf.setNumWorkers(2);
>>
>> CompleteTopologyParam completeTopologyParam = new
>> CompleteTopologyParam();
>>
>> completeTopologyParam.setMockedSources(mockedSources);
>> completeTopologyParam.setStormConf(conf);
>>
>> Map result = Testing.completeTopology(cluster,
>> topology, completeTopologyParam);
>> }
>> }
>> );
>> }
>>
>> 2015-09-28 6:11 GMT+02:00 chandresh pancholi <
>> chandreshpancholi...@gmail.com
>> >:
>>
>> > Agree,
>> > Will check for some other solution and update!!
>> >
>> > On Mon, Sep 28, 2015 at 9:12 AM, Vishal Garg <gargv...@gmail.com>
>> wrote:
>> >
>> > > Chandresh,
>> > >
>> > > Generally sleep in tests is not a good idea. Will try to look into
>> your
>> > > test this week, see if we can avoid it.
>> > > Meanwhile you can look at how storm does its own unit testing if you
>> > could
>> > > find one?
>> > > Vishal
>> > >
>> > > Sent from my iPhone
>> > >
>> > > > On Sep 27, 2015, at 8:23 PM, Roman <rsht...@yahoo.com.INVALID>
>> wrote:
>> > > >
>> > > > I also have concerns about this sleep().Storm has testing facilities
>> > you
>> > > can use for your test. How about
>> Testing.withSimulatedTimeLocalCluster()?
>> > > > Roman
>> > > >
>> > > >
>> > > >
>> > > > On Monday, September 28, 2015 5:39 AM, Dmitriy Setrakyan <
>> > > dsetrak...@apache.org> wrote:
>> > > >
>> > > >
>> > > > On Sun, Sep 27, 2015 at 1:36 PM, chandresh pancholi <
>> > > > chandreshpancholi...@gmail.com> wrote:
>> > > >
>> > > >> Gian,
>> > > >> I have checked the code. It looks good.
>> > > >> Just one little change make it 20 sec.
>> > > >>
>> > > >> /* Topology will run for 10sec */
>> > > >> Utils.sleep(20000);
>> > > >
>> > > > Hm... Can someone explain what is this sleep() for?
>> > > >
>> > > >
>> > > >>
>> > > >>
>> > > >>
>> > > >> On Sun, Sep 27, 2015 at 4:30 PM, Gianfranco Murador <
>> > > >> murador.gianfra...@gmail.com> wrote:
>> > > >>
>> > > >>> Chandresh, we're ready to do a pull request. Look here:
>> > > >>> https://github.com/murador/ignite/tree/IGNITE-429/modules/storm
>> > > >>>
>> > > >>> 2015-09-27 8:49 GMT+02:00 chandresh pancholi <
>> > > >>> chandreshpancholi...@gmail.com
>> > > >>>> :
>> > > >>>
>> > > >>>> Here is the log file.
>> > > >>>>
>> > > >>>> Thanks
>> > > >>>>
>> > > >>>> On Sat, Sep 26, 2015 at 10:40 PM, Gianfranco Murador <
>> > > >>>> murador.gianfra...@gmail.com> wrote:
>> > > >>>>
>> > > >>>>> Chan,
>> > > >>>>> I can run the test succefully, but sometimes I get this error:
>> > > >>>>> Caused by: class org.apache.ignite.spi.IgniteSpiException:
>> Failed
>> > to
>> > > >> add
>> > > >>>>> node to topology because remote node is configured to use
>> loopback
>> > > >>>>> address,
>> > > >>>>> but local node is not (consider changing 'localAddress'
>> > configuration
>> > > >>>>> parameter).
>> > > >>>>>
>> > > >>>>> can you send the log file?
>> > > >>>>> Regards, Gianfranco
>> > > >>>>>
>> > > >>>>>
>> > > >>>>>
>> > > >>>>> 2015-09-26 17:13 GMT+02:00 chandresh pancholi <
>> > > >>>>> chandreshpancholi...@gmail.com>:
>> > > >>>>>
>> > > >>>>>> Gian,
>> > > >>>>>> Its still giving "Ouch! Argument cannot be null: streamer"
>> > > >>>>>>
>> > > >>>>>> On Sat, Sep 26, 2015 at 7:24 PM, Gianfranco Murador <
>> > > >>>>>> murador.gianfra...@gmail.com> wrote:
>> > > >>>>>>
>> > > >>>>>>> Chan,
>> > > >>>>>>> I've fixed some issue, please take a look here:
>> > > >>
>> > >
>> >
>> https://github.com/murador/ignite/blob/IGNITE-429/modules/storm/src/test/java/org/apache/ignite/stream/storm/StormIgniteStreamerSelfTest.java
>> > > >>>>>>>
>> > > >>>>>>> 2015-09-26 8:29 GMT+02:00 chandresh pancholi <
>> > > >>>>>>> chandreshpancholi...@gmail.com
>> > > >>>>>>>> :
>> > > >>>>>>>
>> > > >>>>>>>> Gian,
>> > > >>>>>>>>
>> > > >>>>>>>> I have followed instructions given by you. I instantiate
>> > > >>>>>>> IgniteDateStreamer
>> > > >>>>>>>> and ignite in prepare() method of bolt. but getting "*Cache
>> > > >>> doesn't
>> > > >>>>>>>> exist*".
>> > > >>>>>>>>
>> > > >>>>>>>> Code snippet from stormStreamer class in prepare() method
>> > > >>>>>>>>
>> > > >>>>>>>> StormStreamer<String, String, String> stormStreamer = null;
>> > > >>>>>>>>
>> > > >>>>>>>> Ignite ignite = Ignition.start();
>> > > >>>>>>>>
>> > > >>>>>>>> System.out.println(" ignite ====== "+ignite.toString());
>> > > >>>>>>>>
>> > > >>>>>>>> try(IgniteDataStreamer<String, String> stmr =
>> > > >>>>>>>> ignite.dataStreamer("cache-0")) {
>> > > >>>>>>>>
>> > > >>>>>>>> stormStreamer = new StormStreamer<>();
>> > > >>>>>>>>
>> > > >>>>>>>> IgniteCache<String, String> cache =
>> ignite.cache("cache-0");
>> > > >>>>>>>>
>> > > >>>>>>>> stmr.allowOverwrite(true);
>> > > >>>>>>>>
>> > > >>>>>>>> /* Set ignite instance */
>> > > >>>>>>>> stormStreamer.setIgnite(ignite);
>> > > >>>>>>>>
>> > > >>>>>>>> /* Set streamer instance */
>> > > >>>>>>>> stormStreamer.setStreamer(stmr);
>> > > >>>>>>>>
>> > > >>>>>>>> /* set thread count */
>> > > >>>>>>>> setThreads(5);
>> > > >>>>>>>>
>> > > >>>>>>>> }
>> > > >>>>>>>>
>> > > >>>>>>>>
>> > > >>>>>>>>
>> > > >>>>>>>> On Thu, Sep 24, 2015 at 4:19 PM, Anton Vinogradov <
>> > > >>>>>>>> avinogra...@gridgain.com>
>> > > >>>>>>>> wrote:
>> > > >>>>>>>>
>> > > >>>>>>>>> Chandresh,
>> > > >>>>>>>>> As far as understand IBolt implementation should setup all
>> > > >>>>> external
>> > > >>>>>>>>> connections at .prepare() method.
>> > > >>>>>>>>> So, better way is to get existing Ignite instance or create
>> > > >> new
>> > > >>> at
>> > > >>>>>>>>> .prepare()
>> > > >>>>>>>>> method.
>> > > >>>>>>>>>
>> > > >>>>>>>>>
>> > > >>>>>>>>> On Thu, Sep 24, 2015 at 10:55 AM, Gianfranco Murador <
>> > > >>>>>>>>> murador.gianfra...@gmail.com> wrote:
>> > > >>>>>>>>>
>> > > >>>>>>>>>> Chandresh,
>> > > >>>>>>>>>> I will launch the unit test later this week , if the
>> problem
>> > > >>>>>>> persists.
>> > > >>>>>>>> I
>> > > >>>>>>>>>> think that you should instantiate Ignite through the
>> start()
>> > > >>>>> method
>> > > >>>>>>> of
>> > > >>>>>>>>>> Ignition class
>> > > >>>>>>>>>> and retrieve the Ignite cache specifying a name for the
>> > > >>> default
>> > > >>>>>>> cache.
>> > > >>>>>>>>> You
>> > > >>>>>>>>>> should provide a name also for data streamer, not passing
>> > > >>> null.
>> > > >>>>>>>>>> Could you do this and run again the unit test ?
>> > > >>>>>>>>>> Thank you,
>> > > >>>>>>>>>> Regards,
>> > > >>>>>>>>>> Gianfranco
>> > > >>>>>>>>>>
>> > > >>>>>>>>>>
>> > > >>>>>>>>>> 2015-09-24 5:40 GMT+02:00 chandresh pancholi <
>> > > >>>>>>>>>> chandreshpancholi...@gmail.com
>> > > >>>>>>>>>>> :
>> > > >>>>>>>>>>
>> > > >>>>>>>>>>> Anton,
>> > > >>>>>>>>>>>
>> > > >>>>>>>>>>> Clone this repo
>> > > >>> https://github.com/chandresh-pancholi/ignite
>> > > >>>>> and
>> > > >>>>>>> run
>> > > >>>>>>>>>> this
>> > > >>>>>>>>>>> class
>> > > >>
>> > >
>> >
>> https://github.com/chandresh-pancholi/ignite/blob/master/modules/storm/src/test/java/org/apache/ignite/stream/storm/IgniteStormStreamerSelfTestSuite.java
>> > > >>>>>>>>>>>
>> > > >>>>>>>>>>> You will get the Error saying "Oouch,Argument is Null" for
>> > > >>>>>>>>> getStreamer().
>> > > >>>>>>>>>>>
>> > > >>>>>>>>>>> On Wed, Sep 23, 2015 at 9:46 PM, Anton Vinogradov <
>> > > >>>>>>>>>>> avinogra...@gridgain.com>
>> > > >>>>>>>>>>> wrote:
>> > > >>>>>>>>>>>
>> > > >>>>>>>>>>>> Chandresh,
>> > > >>>>>>>>>>>> I'm trying to analize problem.
>> > > >>>>>>>>>>>> Could you please provide more details and explain
>> > > >>>>> step-by-step
>> > > >>>>>>> how
>> > > >>>>>>>>> can
>> > > >>>>>>>>>> I
>> > > >>>>>>>>>>>> reproduce problem?
>> > > >>>>>>>>>>>>
>> > > >>>>>>>>>>>> On Tue, Sep 22, 2015 at 11:06 PM, chandresh pancholi <
>> > > >>>>>>>>>>>> chandreshpancholi...@gmail.com> wrote:
>> > > >>>>>>>>>>>>
>> > > >>>>>>>>>>>>> Not yet.
>> > > >>>>>>>>>>>>> Still doing brainstorming why StormStreamer object is
>> > > >>>>> getting
>> > > >>>>>>>> NULLL
>> > > >>>>>>>>>>>>> in-spite setting up ignite and streamer instances.
>> > > >>>>>>>>>>>>>
>> > > >>>>>>>>>>>>> On Tue, Sep 22, 2015 at 7:22 PM, Vishal Garg <
>> > > >>>>>>> gargv...@gmail.com
>> > > >>>>>>>>>
>> > > >>>>>>>>>>> wrote:
>> > > >>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>> Did you figure it out? Something related to your
>> > > >> storm
>> > > >>>>>>>>> integration?
>> > > >>>>>>>>>>>>>> Vishal
>> > > >>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>> Sent from my iPhone
>> > > >>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>> On Sep 22, 2015, at 4:50 AM, chandresh pancholi <
>> > > >>>>>>>>>>>>>>> chandreshpancholi...@gmail.com> wrote:
>> > > >>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>> Problem here is with Storm. Its making streamer
>> > > >>> object
>> > > >>>>>> null
>> > > >>>>>>>>> once
>> > > >>>>>>>>>> it
>> > > >>>>>>>>>>>>> goes
>> > > >>>>>>>>>>>>>>> into submit topology.
>> > > >>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>> On Mon, Sep 21, 2015 at 7:13 PM, Lalit Kumar Jha <
>> > > >>>>>>>>>>>> lalitj....@gmail.com
>> > > >>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>> wrote:
>> > > >>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>> Hi Chandresh,
>> > > >>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>> See test suite class here, its based on
>> > > >> annotations
>> > > >>
>> > >
>> >
>> https://github.com/sylentprayer/ignite/blob/ignite-530/modules/twitter/src/test/java/org/apache/ignite/stream/twitter/IgniteTwitterStreamerTestSuite.java
>> > > >>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>> No code required in class body.
>> > > >>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>> On Mon, Sep 21, 2015 at 6:24 PM, chandresh
>> > > >>> pancholi <
>> > > >>>>>>>>>>>>>>>> chandreshpancholi...@gmail.com> wrote:
>> > > >>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>> Hi Gian/Vishal/Lalit,
>> > > >>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>> You guys have worked on Streamer integration
>> > > >> with
>> > > >>>>>>> Ignite. I
>> > > >>>>>>>>> am
>> > > >>>>>>>>>>>>> working
>> > > >>>>>>>>>>>>>> on
>> > > >>>>>>>>>>>>>>>>> Storm-ignite integration.
>> > > >>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>> I am setting up setStreamer() and setIgnite() in
>> > > >>>>> Test
>> > > >>>>>>> class
>> > > >>>>>>>>> and
>> > > >>>>>>>>>>>>> sending
>> > > >>>>>>>>>>>>>>>>> this object to setBolt() method.
>> > > >>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>> When i try to read via gettreamer() and
>> > > >>> getIgnite()
>> > > >>>>> i
>> > > >>>>>> am
>> > > >>>>>>>>>> getting
>> > > >>>>>>>>>>>>> NULL.
>> > > >>>>>>>>>>>>>> I
>> > > >>>>>>>>>>>>>>>> am
>> > > >>>>>>>>>>>>>>>>> trying to find out why is it happening but No
>> > > >>> Luck.
>> > > >>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>> If any one of you find some time and look into
>> > > >> the
>> > > >>>>> code
>> > > >>>>>>> and
>> > > >>>>>>>>>> help
>> > > >>>>>>>>>>> me
>> > > >>>>>>>>>>>>> to
>> > > >>>>>>>>>>>>>>>>> resolve this. It would be great.
>> > > >>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>> Git :
>> > > >>> https://github.com/chandresh-pancholi/ignite
>> > > >>>>>>>>>>>>>>>>> Module : storm
>> > > >>>>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>>> --
>> > > >>>>>>>>>>>>>>>>> Chandresh Pancholi
>> > > >>>>>>>>>>>>>>>>> Senior Software Engineer
>> > > >>>>>>>>>>>>>>>>> Flipkart.com
>> > > >>>>>>>>>>>>>>>>> Email-id:chandresh.panch...@flipkart.com
>> > > >>>>>>>>>>>>>>>>> Contact:08951803660
>> > > >>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>>> --
>> > > >>>>>>>>>>>>>>> Chandresh Pancholi
>> > > >>>>>>>>>>>>>>> Senior Software Engineer
>> > > >>>>>>>>>>>>>>> Flipkart.com
>> > > >>>>>>>>>>>>>>> Email-id:chandresh.panch...@flipkart.com
>> > > >>>>>>>>>>>>>>> Contact:08951803660
>> > > >>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>
>> > > >>>>>>>>>>>>>
>> > > >>>>>>>>>>>>> --
>> > > >>>>>>>>>>>>> Chandresh Pancholi
>> > > >>>>>>>>>>>>> Senior Software Engineer
>> > > >>>>>>>>>>>>> Flipkart.com
>> > > >>>>>>>>>>>>> Email-id:chandresh.panch...@flipkart.com
>> > > >>>>>>>>>>>>> Contact:08951803660
>> > > >>>>>>>>>>>
>> > > >>>>>>>>>>>
>> > > >>>>>>>>>>>
>> > > >>>>>>>>>>> --
>> > > >>>>>>>>>>> Chandresh Pancholi
>> > > >>>>>>>>>>> Senior Software Engineer
>> > > >>>>>>>>>>> Flipkart.com
>> > > >>>>>>>>>>> Email-id:chandresh.panch...@flipkart.com
>> > > >>>>>>>>>>> Contact:08951803660
>> > > >>>>>>>>
>> > > >>>>>>>>
>> > > >>>>>>>>
>> > > >>>>>>>> --
>> > > >>>>>>>> Chandresh Pancholi
>> > > >>>>>>>> Senior Software Engineer
>> > > >>>>>>>> Flipkart.com
>> > > >>>>>>>> Email-id:chandresh.panch...@flipkart.com
>> > > >>>>>>>> Contact:08951803660
>> > > >>>>>>
>> > > >>>>>>
>> > > >>>>>>
>> > > >>>>>> --
>> > > >>>>>> Chandresh Pancholi
>> > > >>>>>> Senior Software Engineer
>> > > >>>>>> Flipkart.com
>> > > >>>>>> Email-id:chandresh.panch...@flipkart.com
>> > > >>>>>> Contact:08951803660
>> > > >>>>
>> > > >>>>
>> > > >>>>
>> > > >>>> --
>> > > >>>> Chandresh Pancholi
>> > > >>>> Senior Software Engineer
>> > > >>>> Flipkart.com
>> > > >>>> Email-id:chandresh.panch...@flipkart.com
>> > > >>>> Contact:08951803660
>> > > >>
>> > > >>
>> > > >>
>> > > >> --
>> > > >> Chandresh Pancholi
>> > > >> Senior Software Engineer
>> > > >> Flipkart.com
>> > > >> Email-id:chandresh.panch...@flipkart.com
>> > > >> Contact:08951803660
>> > > >
>> > > >
>> > >
>> >
>> >
>> >
>> > --
>> > Chandresh Pancholi
>> > Senior Software Engineer
>> > Flipkart.com
>> > Email-id:chandresh.panch...@flipkart.com
>> > Contact:08951803660
>> >
>>
>
>
>
> --
> Chandresh Pancholi
> Senior Software Engineer
> Flipkart.com
> Email-id:chandresh.panch...@flipkart.com
> Contact:08951803660
>
--
Chandresh Pancholi
Senior Software Engineer
Flipkart.com
Email-id:chandresh.panch...@flipkart.com
Contact:08951803660
