Yes, in mater branch of my fork.
2015-09-29 8:45 GMT+02:00 chandresh pancholi <chandreshpancholi...@gmail.com
>:
> Gian,
>
> is code for ignite-429 in master branch?
>
> On Mon, Sep 28, 2015 at 5:27 PM, chandresh pancholi <
> chandreshpancholi...@gmail.com> wrote:
>
> > Gian,
> >
> > Lets use this and modify our tests. I will push the code by tomorrow EOD.
> >
> > Thanks
> > Chandresh Pancholi
> >
> > On Mon, Sep 28, 2015 at 3:58 PM, Gianfranco Murador <
> > murador.gianfra...@gmail.com> wrote:
> >
> >> Chandresh,
> >> I think that the suggestion of Roman is preferable, we can use the
> >> following code :
> >> Let me know if you have problems
> >>
> >> /** Start a simulated topology */
> >> public void startTopology(StormStreamer stormStreamer) {
> >> MkClusterParam mkClusterParam = new MkClusterParam();
> >> mkClusterParam.setSupervisors(4);
> >> Config daemonConf = new Config();
> >> daemonConf.put(Config.STORM_LOCAL_MODE_ZMQ, false);
> >> mkClusterParam.setDaemonConf(daemonConf);
> >>
> >> Testing.withSimulatedTimeLocalCluster(mkClusterParam, new
> >> TestJob()
> >> {
> >> @Override
> >> public void run(ILocalCluster cluster) throws
> >> IOException {
> >> TopologyBuilder builder = new TopologyBuilder();
> >>
> >> builder.setSpout("testSpout", new StormSpout());
> >>
> >> builder.setBolt("bolt", stormStreamer)
> >> .shuffleGrouping("spout");
> >>
> >> StormTopology topology =
> builder.createTopology();
> >>
> >> MockedSources mockedSources = new
> MockedSources();
> >>
> >> //Our spout will be processing this values.
> >> mockedSources.addMockData("testSpout",new
> >> Values(stormStream.getKeyValMap()));
> >>
> >>
> >> // prepare the config
> >> Config conf = new Config();
> >> conf.setNumWorkers(2);
> >>
> >> CompleteTopologyParam completeTopologyParam =
> new
> >> CompleteTopologyParam();
> >>
> >> completeTopologyParam.setMockedSources(mockedSources);
> >> completeTopologyParam.setStormConf(conf);
> >>
> >> Map result = Testing.completeTopology(cluster,
> >> topology, completeTopologyParam);
> >> }
> >> }
> >> );
> >> }
> >>
> >> 2015-09-28 6:11 GMT+02:00 chandresh pancholi <
> >> chandreshpancholi...@gmail.com
> >> >:
> >>
> >> > Agree,
> >> > Will check for some other solution and update!!
> >> >
> >> > On Mon, Sep 28, 2015 at 9:12 AM, Vishal Garg <gargv...@gmail.com>
> >> wrote:
> >> >
> >> > > Chandresh,
> >> > >
> >> > > Generally sleep in tests is not a good idea. Will try to look into
> >> your
> >> > > test this week, see if we can avoid it.
> >> > > Meanwhile you can look at how storm does its own unit testing if you
> >> > could
> >> > > find one?
> >> > > Vishal
> >> > >
> >> > > Sent from my iPhone
> >> > >
> >> > > > On Sep 27, 2015, at 8:23 PM, Roman <rsht...@yahoo.com.INVALID>
> >> wrote:
> >> > > >
> >> > > > I also have concerns about this sleep().Storm has testing
> facilities
> >> > you
> >> > > can use for your test. How about
> >> Testing.withSimulatedTimeLocalCluster()?
> >> > > > Roman
> >> > > >
> >> > > >
> >> > > >
> >> > > > On Monday, September 28, 2015 5:39 AM, Dmitriy Setrakyan <
> >> > > dsetrak...@apache.org> wrote:
> >> > > >
> >> > > >
> >> > > > On Sun, Sep 27, 2015 at 1:36 PM, chandresh pancholi <
> >> > > > chandreshpancholi...@gmail.com> wrote:
> >> > > >
> >> > > >> Gian,
> >> > > >> I have checked the code. It looks good.
> >> > > >> Just one little change make it 20 sec.
> >> > > >>
> >> > > >> /* Topology will run for 10sec */
> >> > > >> Utils.sleep(20000);
> >> > > >
> >> > > > Hm... Can someone explain what is this sleep() for?
> >> > > >
> >> > > >
> >> > > >>
> >> > > >>
> >> > > >>
> >> > > >> On Sun, Sep 27, 2015 at 4:30 PM, Gianfranco Murador <
> >> > > >> murador.gianfra...@gmail.com> wrote:
> >> > > >>
> >> > > >>> Chandresh, we're ready to do a pull request. Look here:
> >> > > >>> https://github.com/murador/ignite/tree/IGNITE-429/modules/storm
> >> > > >>>
> >> > > >>> 2015-09-27 8:49 GMT+02:00 chandresh pancholi <
> >> > > >>> chandreshpancholi...@gmail.com
> >> > > >>>> :
> >> > > >>>
> >> > > >>>> Here is the log file.
> >> > > >>>>
> >> > > >>>> Thanks
> >> > > >>>>
> >> > > >>>> On Sat, Sep 26, 2015 at 10:40 PM, Gianfranco Murador <
> >> > > >>>> murador.gianfra...@gmail.com> wrote:
> >> > > >>>>
> >> > > >>>>> Chan,
> >> > > >>>>> I can run the test succefully, but sometimes I get this
> error:
> >> > > >>>>> Caused by: class org.apache.ignite.spi.IgniteSpiException:
> >> Failed
> >> > to
> >> > > >> add
> >> > > >>>>> node to topology because remote node is configured to use
> >> loopback
> >> > > >>>>> address,
> >> > > >>>>> but local node is not (consider changing 'localAddress'
> >> > configuration
> >> > > >>>>> parameter).
> >> > > >>>>>
> >> > > >>>>> can you send the log file?
> >> > > >>>>> Regards, Gianfranco
> >> > > >>>>>
> >> > > >>>>>
> >> > > >>>>>
> >> > > >>>>> 2015-09-26 17:13 GMT+02:00 chandresh pancholi <
> >> > > >>>>> chandreshpancholi...@gmail.com>:
> >> > > >>>>>
> >> > > >>>>>> Gian,
> >> > > >>>>>> Its still giving "Ouch! Argument cannot be null: streamer"
> >> > > >>>>>>
> >> > > >>>>>> On Sat, Sep 26, 2015 at 7:24 PM, Gianfranco Murador <
> >> > > >>>>>> murador.gianfra...@gmail.com> wrote:
> >> > > >>>>>>
> >> > > >>>>>>> Chan,
> >> > > >>>>>>> I've fixed some issue, please take a look here:
> >> > > >>
> >> > >
> >> >
> >>
> https://github.com/murador/ignite/blob/IGNITE-429/modules/storm/src/test/java/org/apache/ignite/stream/storm/StormIgniteStreamerSelfTest.java
> >> > > >>>>>>>
> >> > > >>>>>>> 2015-09-26 8:29 GMT+02:00 chandresh pancholi <
> >> > > >>>>>>> chandreshpancholi...@gmail.com
> >> > > >>>>>>>> :
> >> > > >>>>>>>
> >> > > >>>>>>>> Gian,
> >> > > >>>>>>>>
> >> > > >>>>>>>> I have followed instructions given by you. I instantiate
> >> > > >>>>>>> IgniteDateStreamer
> >> > > >>>>>>>> and ignite in prepare() method of bolt. but getting "*Cache
> >> > > >>> doesn't
> >> > > >>>>>>>> exist*".
> >> > > >>>>>>>>
> >> > > >>>>>>>> Code snippet from stormStreamer class in prepare() method
> >> > > >>>>>>>>
> >> > > >>>>>>>> StormStreamer<String, String, String> stormStreamer = null;
> >> > > >>>>>>>>
> >> > > >>>>>>>> Ignite ignite = Ignition.start();
> >> > > >>>>>>>>
> >> > > >>>>>>>> System.out.println(" ignite ====== "+ignite.toString());
> >> > > >>>>>>>>
> >> > > >>>>>>>> try(IgniteDataStreamer<String, String> stmr =
> >> > > >>>>>>>> ignite.dataStreamer("cache-0")) {
> >> > > >>>>>>>>
> >> > > >>>>>>>> stormStreamer = new StormStreamer<>();
> >> > > >>>>>>>>
> >> > > >>>>>>>> IgniteCache<String, String> cache =
> >> ignite.cache("cache-0");
> >> > > >>>>>>>>
> >> > > >>>>>>>> stmr.allowOverwrite(true);
> >> > > >>>>>>>>
> >> > > >>>>>>>> /* Set ignite instance */
> >> > > >>>>>>>> stormStreamer.setIgnite(ignite);
> >> > > >>>>>>>>
> >> > > >>>>>>>> /* Set streamer instance */
> >> > > >>>>>>>> stormStreamer.setStreamer(stmr);
> >> > > >>>>>>>>
> >> > > >>>>>>>> /* set thread count */
> >> > > >>>>>>>> setThreads(5);
> >> > > >>>>>>>>
> >> > > >>>>>>>> }
> >> > > >>>>>>>>
> >> > > >>>>>>>>
> >> > > >>>>>>>>
> >> > > >>>>>>>> On Thu, Sep 24, 2015 at 4:19 PM, Anton Vinogradov <
> >> > > >>>>>>>> avinogra...@gridgain.com>
> >> > > >>>>>>>> wrote:
> >> > > >>>>>>>>
> >> > > >>>>>>>>> Chandresh,
> >> > > >>>>>>>>> As far as understand IBolt implementation should setup all
> >> > > >>>>> external
> >> > > >>>>>>>>> connections at .prepare() method.
> >> > > >>>>>>>>> So, better way is to get existing Ignite instance or
> create
> >> > > >> new
> >> > > >>> at
> >> > > >>>>>>>>> .prepare()
> >> > > >>>>>>>>> method.
> >> > > >>>>>>>>>
> >> > > >>>>>>>>>
> >> > > >>>>>>>>> On Thu, Sep 24, 2015 at 10:55 AM, Gianfranco Murador <
> >> > > >>>>>>>>> murador.gianfra...@gmail.com> wrote:
> >> > > >>>>>>>>>
> >> > > >>>>>>>>>> Chandresh,
> >> > > >>>>>>>>>> I will launch the unit test later this week , if the
> >> problem
> >> > > >>>>>>> persists.
> >> > > >>>>>>>> I
> >> > > >>>>>>>>>> think that you should instantiate Ignite through the
> >> start()
> >> > > >>>>> method
> >> > > >>>>>>> of
> >> > > >>>>>>>>>> Ignition class
> >> > > >>>>>>>>>> and retrieve the Ignite cache specifying a name for the
> >> > > >>> default
> >> > > >>>>>>> cache.
> >> > > >>>>>>>>> You
> >> > > >>>>>>>>>> should provide a name also for data streamer, not passing
> >> > > >>> null.
> >> > > >>>>>>>>>> Could you do this and run again the unit test ?
> >> > > >>>>>>>>>> Thank you,
> >> > > >>>>>>>>>> Regards,
> >> > > >>>>>>>>>> Gianfranco
> >> > > >>>>>>>>>>
> >> > > >>>>>>>>>>
> >> > > >>>>>>>>>> 2015-09-24 5:40 GMT+02:00 chandresh pancholi <
> >> > > >>>>>>>>>> chandreshpancholi...@gmail.com
> >> > > >>>>>>>>>>> :
> >> > > >>>>>>>>>>
> >> > > >>>>>>>>>>> Anton,
> >> > > >>>>>>>>>>>
> >> > > >>>>>>>>>>> Clone this repo
> >> > > >>> https://github.com/chandresh-pancholi/ignite
> >> > > >>>>> and
> >> > > >>>>>>> run
> >> > > >>>>>>>>>> this
> >> > > >>>>>>>>>>> class
> >> > > >>
> >> > >
> >> >
> >>
> https://github.com/chandresh-pancholi/ignite/blob/master/modules/storm/src/test/java/org/apache/ignite/stream/storm/IgniteStormStreamerSelfTestSuite.java
> >> > > >>>>>>>>>>>
> >> > > >>>>>>>>>>> You will get the Error saying "Oouch,Argument is Null"
> for
> >> > > >>>>>>>>> getStreamer().
> >> > > >>>>>>>>>>>
> >> > > >>>>>>>>>>> On Wed, Sep 23, 2015 at 9:46 PM, Anton Vinogradov <
> >> > > >>>>>>>>>>> avinogra...@gridgain.com>
> >> > > >>>>>>>>>>> wrote:
> >> > > >>>>>>>>>>>
> >> > > >>>>>>>>>>>> Chandresh,
> >> > > >>>>>>>>>>>> I'm trying to analize problem.
> >> > > >>>>>>>>>>>> Could you please provide more details and explain
> >> > > >>>>> step-by-step
> >> > > >>>>>>> how
> >> > > >>>>>>>>> can
> >> > > >>>>>>>>>> I
> >> > > >>>>>>>>>>>> reproduce problem?
> >> > > >>>>>>>>>>>>
> >> > > >>>>>>>>>>>> On Tue, Sep 22, 2015 at 11:06 PM, chandresh pancholi <
> >> > > >>>>>>>>>>>> chandreshpancholi...@gmail.com> wrote:
> >> > > >>>>>>>>>>>>
> >> > > >>>>>>>>>>>>> Not yet.
> >> > > >>>>>>>>>>>>> Still doing brainstorming why StormStreamer object is
> >> > > >>>>> getting
> >> > > >>>>>>>> NULLL
> >> > > >>>>>>>>>>>>> in-spite setting up ignite and streamer instances.
> >> > > >>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>> On Tue, Sep 22, 2015 at 7:22 PM, Vishal Garg <
> >> > > >>>>>>> gargv...@gmail.com
> >> > > >>>>>>>>>
> >> > > >>>>>>>>>>> wrote:
> >> > > >>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>> Did you figure it out? Something related to your
> >> > > >> storm
> >> > > >>>>>>>>> integration?
> >> > > >>>>>>>>>>>>>> Vishal
> >> > > >>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>> Sent from my iPhone
> >> > > >>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>> On Sep 22, 2015, at 4:50 AM, chandresh pancholi <
> >> > > >>>>>>>>>>>>>>> chandreshpancholi...@gmail.com> wrote:
> >> > > >>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>> Problem here is with Storm. Its making streamer
> >> > > >>> object
> >> > > >>>>>> null
> >> > > >>>>>>>>> once
> >> > > >>>>>>>>>> it
> >> > > >>>>>>>>>>>>> goes
> >> > > >>>>>>>>>>>>>>> into submit topology.
> >> > > >>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>> On Mon, Sep 21, 2015 at 7:13 PM, Lalit Kumar Jha <
> >> > > >>>>>>>>>>>> lalitj....@gmail.com
> >> > > >>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>> wrote:
> >> > > >>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>> Hi Chandresh,
> >> > > >>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>> See test suite class here, its based on
> >> > > >> annotations
> >> > > >>
> >> > >
> >> >
> >>
> https://github.com/sylentprayer/ignite/blob/ignite-530/modules/twitter/src/test/java/org/apache/ignite/stream/twitter/IgniteTwitterStreamerTestSuite.java
> >> > > >>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>> No code required in class body.
> >> > > >>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>> On Mon, Sep 21, 2015 at 6:24 PM, chandresh
> >> > > >>> pancholi <
> >> > > >>>>>>>>>>>>>>>> chandreshpancholi...@gmail.com> wrote:
> >> > > >>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>> Hi Gian/Vishal/Lalit,
> >> > > >>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>> You guys have worked on Streamer integration
> >> > > >> with
> >> > > >>>>>>> Ignite. I
> >> > > >>>>>>>>> am
> >> > > >>>>>>>>>>>>> working
> >> > > >>>>>>>>>>>>>> on
> >> > > >>>>>>>>>>>>>>>>> Storm-ignite integration.
> >> > > >>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>> I am setting up setStreamer() and setIgnite() in
> >> > > >>>>> Test
> >> > > >>>>>>> class
> >> > > >>>>>>>>> and
> >> > > >>>>>>>>>>>>> sending
> >> > > >>>>>>>>>>>>>>>>> this object to setBolt() method.
> >> > > >>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>> When i try to read via gettreamer() and
> >> > > >>> getIgnite()
> >> > > >>>>> i
> >> > > >>>>>> am
> >> > > >>>>>>>>>> getting
> >> > > >>>>>>>>>>>>> NULL.
> >> > > >>>>>>>>>>>>>> I
> >> > > >>>>>>>>>>>>>>>> am
> >> > > >>>>>>>>>>>>>>>>> trying to find out why is it happening but No
> >> > > >>> Luck.
> >> > > >>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>> If any one of you find some time and look into
> >> > > >> the
> >> > > >>>>> code
> >> > > >>>>>>> and
> >> > > >>>>>>>>>> help
> >> > > >>>>>>>>>>> me
> >> > > >>>>>>>>>>>>> to
> >> > > >>>>>>>>>>>>>>>>> resolve this. It would be great.
> >> > > >>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>> Git :
> >> > > >>> https://github.com/chandresh-pancholi/ignite
> >> > > >>>>>>>>>>>>>>>>> Module : storm
> >> > > >>>>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>>> --
> >> > > >>>>>>>>>>>>>>>>> Chandresh Pancholi
> >> > > >>>>>>>>>>>>>>>>> Senior Software Engineer
> >> > > >>>>>>>>>>>>>>>>> Flipkart.com
> >> > > >>>>>>>>>>>>>>>>> Email-id:chandresh.panch...@flipkart.com
> >> > > >>>>>>>>>>>>>>>>> Contact:08951803660
> >> > > >>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>>> --
> >> > > >>>>>>>>>>>>>>> Chandresh Pancholi
> >> > > >>>>>>>>>>>>>>> Senior Software Engineer
> >> > > >>>>>>>>>>>>>>> Flipkart.com
> >> > > >>>>>>>>>>>>>>> Email-id:chandresh.panch...@flipkart.com
> >> > > >>>>>>>>>>>>>>> Contact:08951803660
> >> > > >>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>>
> >> > > >>>>>>>>>>>>> --
> >> > > >>>>>>>>>>>>> Chandresh Pancholi
> >> > > >>>>>>>>>>>>> Senior Software Engineer
> >> > > >>>>>>>>>>>>> Flipkart.com
> >> > > >>>>>>>>>>>>> Email-id:chandresh.panch...@flipkart.com
> >> > > >>>>>>>>>>>>> Contact:08951803660
> >> > > >>>>>>>>>>>
> >> > > >>>>>>>>>>>
> >> > > >>>>>>>>>>>
> >> > > >>>>>>>>>>> --
> >> > > >>>>>>>>>>> Chandresh Pancholi
> >> > > >>>>>>>>>>> Senior Software Engineer
> >> > > >>>>>>>>>>> Flipkart.com
> >> > > >>>>>>>>>>> Email-id:chandresh.panch...@flipkart.com
> >> > > >>>>>>>>>>> Contact:08951803660
> >> > > >>>>>>>>
> >> > > >>>>>>>>
> >> > > >>>>>>>>
> >> > > >>>>>>>> --
> >> > > >>>>>>>> Chandresh Pancholi
> >> > > >>>>>>>> Senior Software Engineer
> >> > > >>>>>>>> Flipkart.com
> >> > > >>>>>>>> Email-id:chandresh.panch...@flipkart.com
> >> > > >>>>>>>> Contact:08951803660
> >> > > >>>>>>
> >> > > >>>>>>
> >> > > >>>>>>
> >> > > >>>>>> --
> >> > > >>>>>> Chandresh Pancholi
> >> > > >>>>>> Senior Software Engineer
> >> > > >>>>>> Flipkart.com
> >> > > >>>>>> Email-id:chandresh.panch...@flipkart.com
> >> > > >>>>>> Contact:08951803660
> >> > > >>>>
> >> > > >>>>
> >> > > >>>>
> >> > > >>>> --
> >> > > >>>> Chandresh Pancholi
> >> > > >>>> Senior Software Engineer
> >> > > >>>> Flipkart.com
> >> > > >>>> Email-id:chandresh.panch...@flipkart.com
> >> > > >>>> Contact:08951803660
> >> > > >>
> >> > > >>
> >> > > >>
> >> > > >> --
> >> > > >> Chandresh Pancholi
> >> > > >> Senior Software Engineer
> >> > > >> Flipkart.com
> >> > > >> Email-id:chandresh.panch...@flipkart.com
> >> > > >> Contact:08951803660
> >> > > >
> >> > > >
> >> > >
> >> >
> >> >
> >> >
> >> > --
> >> > Chandresh Pancholi
> >> > Senior Software Engineer
> >> > Flipkart.com
> >> > Email-id:chandresh.panch...@flipkart.com
> >> > Contact:08951803660
> >> >
> >>
> >
> >
> >
> > --
> > Chandresh Pancholi
> > Senior Software Engineer
> > Flipkart.com
> > Email-id:chandresh.panch...@flipkart.com
> > Contact:08951803660
> >
>
>
>
> --
> Chandresh Pancholi
> Senior Software Engineer
> Flipkart.com
> Email-id:chandresh.panch...@flipkart.com
> Contact:08951803660
>
--
Gianfranco Murador
Igniter and Software Engineer.
