Re:[DISCUSS] Inverted (child-first) class loading

2018-03-09 Thread mingleizhang
Hi, Stephan It is a great change, keep it and polish it. nice nice nice I think the more fewer NoClassDefFoundError or ClassNotFoundException the user will encounter in the future. But I would like to two question about this functionality. If Im am wrong, please helps me out. Thank you in adv

Re:Re: Why does jobmanager running needs slot ?

2018-02-05 Thread mingleizhang
that much familiar with those code parts. Piotrek On 5 Feb 2018, at 10:51, mingleizhang wrote: Makes sense to me now. Is it a new design at FLIP6 ? Rice. At 2018-02-05 17:49:05, "Piotr Nowojski" wrote: I might be wrong but I think it is other way around and the namin

Re:Re: Why does jobmanager running needs slot ?

2018-02-05 Thread mingleizhang
slots and it is the one that is offering them to a JobManager. JobManager can use those slots offers to (later!) schedule tasks. (#offerSlotsToJobManager() is being called during TaskManager initialisation). Piotrek On 5 Feb 2018, at 10:44, mingleizhang wrote: Yes. Thanks Piotrek. Of

Re:Re: Why does jobmanager running needs slot ?

2018-02-05 Thread mingleizhang
ther words, JobMasterGateway.offerSlots is called by a TaskManager and it is a way how TaskManager is advertising his slots to a JobManager. Piotrek On 5 Feb 2018, at 08:38, mingleizhang wrote: I find some codes in flink does not make sense to me. Like in some classes below JobMasterGateway.java has a

Why does jobmanager running needs slot ?

2018-02-04 Thread mingleizhang
I find some codes in flink does not make sense to me. Like in some classes below JobMasterGateway.java has a offerSlots method which means Offers the given slots to the job manager. I was wondering why a jobmanager running should need slots ? TaskExecutor.java has a offerSlotsToJobManager metho

Re:Re: Re: Just do a survey, how many people give up the storm and turn to Flink ?

2017-08-20 Thread mingleizhang
like redisson) and local cache. On Sun, Aug 20, 2017 at 11:27 AM, Niels Basjes wrote: If you combine Storm with Redis for managing state you still do not have "exactly once" when a failure of a processing node occurs. With Flink you do have that. Niels On 19 Aug 2017 12

Re:Re: Just do a survey, how many people give up the storm and turn to Flink ?

2017-08-19 Thread mingleizhang
se Flink. Niels On 19 Aug 2017 10:28, "mingleizhang" <18717838...@163.com> wrote: Hi, flink user I just want to do a survey as the subject said. How many of you that used to use storm as your real-time computing framework, but now, turn to Flink instead. And why

Just do a survey, how many people give up the storm and turn to Flink ?

2017-08-19 Thread mingleizhang
Hi, flink user I just want to do a survey as the subject said. How many of you that used to use storm as your real-time computing framework, but now, turn to Flink instead. And why ? Could you tell me ? Thank you very much! Thanks zhangminglei

Re:Re:Re:Re:Re:Re:How to verify the data to Elasticsearch whether correct or not ?

2017-08-18 Thread mingleizhang
son", JsonFormat.printToString(element)) If what you want is all fields in this object to be able to be queried in Elasticsearch, what you could do is simply: `Requests.indexRequest().index(...).type(…).source()` This would work fine, and you can also save the extra redundant layer in your sink

Re:Re:Re:Re:Re:How to verify the data to Elasticsearch whether correct or not ?

2017-08-16 Thread mingleizhang
text: RuntimeContext, requestIndexer: RequestIndexer): Unit = { requestIndexer.add(createIndexRequest(activityInfo)) } })) Peace. zhangminglei / mingleizhang At 2017-08-17 09:21:47, "Ted Yu" wrote: Did you use image for the code ? Can you send plain code again ? Cheers Original m

Re:Re:Re:Re:How to verify the data to Elasticsearch whether correct or not ?

2017-08-16 Thread mingleizhang
I solved the issue by adding a dependency that convert the protobuf objects into a JSON. By adding a line of code like below: element is a PB object. Thanks. zhangminglei At 2017-08-16 22:52:30, "mingleizhang" <18717838...@163.com> wrote: I looked into the sink

Re:Re:Re:How to verify the data to Elasticsearch whether correct or not ?

2017-08-16 Thread mingleizhang
data. the value is a string instead. I want it represented as an object of ActivityInfo. But it didnt do like what i want. Can anybody give me some advice for it ? Thank you very much! Thanks zhangminglei / mingleizhang At 2017-08-16 20:52:34, "mingleizhang" <187178

Re:Re:How to verify the data to Elasticsearch whether correct or not ?

2017-08-16 Thread mingleizhang
snippet you provided. So you should be ok with this :) Cheers, Gordon On 15 August 2017 at 9:18:59 PM, mingleizhang (18717838...@163.com) wrote: BTW, ActivityInfo is an PB object build from xxx.proto. And already has it's value setted to itself. At 2017-08-15 21:17:00, "ming

Re:How to verify the data to Elasticsearch whether correct or not ?

2017-08-15 Thread mingleizhang
BTW, ActivityInfo is an PB object build from xxx.proto. And already has it's value setted to itself. At 2017-08-15 21:17:00, "mingleizhang" <18717838...@163.com> wrote: Hi, flink experts! I sinked my data ( PB objects ) to elasticsearch. I dont know whether the sink

How to verify the data to Elasticsearch whether correct or not ?

2017-08-15 Thread mingleizhang
ess(activityInfo: ActivityInfo, runtimeContext: RuntimeContext, requestIndexer: RequestIndexer): Unit = { requestIndexer.add(createIndexRequest(activityInfo)) } })) Thanks mingleizhang

Re:Re: IllegalArgumentException when using elasticsearch as a sink

2017-08-15 Thread mingleizhang
017年8月15日,09:53,mingleizhang <18717838...@163.com> 写道: Thanks, Nico. I tried flink1.3.2. Works now. Thank you very much! I think there should be something else to cause this error to happen. Not only the PR I patched before. Thanks. mingleizhang At 2017-08-15 00:29:28, "Nico Kruber

Re:Re: IllegalArgumentException when using elasticsearch as a sink

2017-08-14 Thread mingleizhang
Thanks, Nico. I tried flink1.3.2. Works now. Thank you very much! I think there should be something else to cause this error to happen. Not only the PR I patched before. Thanks. mingleizhang At 2017-08-15 00:29:28, "Nico Kruber" wrote: >Just to be sure, can you try flin

Re:IllegalArgumentException when using elasticsearch as a sink

2017-08-13 Thread mingleizhang
, "mingleizhang" <18717838...@163.com> wrote: Hello, flink experts and friends! It is my first time to write flink application in my company. But I met the following error when I used a elasticsearch as my sink. I searched the solution for it and found a jira https://issues.apache.org/j

IllegalArgumentException when using elasticsearch as a sink

2017-08-13 Thread mingleizhang
sticsearch ${elasticsearch.version} org.ow2.asm * Regards mingleizhang rice.zh...@vipshop.com