Hi Aljoscha We are not using state backend explicitly, recovery and state backend are pointed to file path. See attached json file
Thanks for the help. Best regards Andrew > On 10 Aug 2016, at 11:38, Aljoscha Krettek <aljos...@apache.org> wrote: > > Oh, are you by any chance specifying a custom state backend for your job? For > example, RocksDBStateBackend. > > Cheers, > Aljoscha > > On Wed, 10 Aug 2016 at 11:17 Aljoscha Krettek <aljos...@apache.org > <mailto:aljos...@apache.org>> wrote: > Hi, > could you maybe send us the output of "env.getExecutionPlan()". This would > help us better understand which operators are used exactly. (You can of > course remove any security sensitive stuff.) > > Cheers, > Aljoscha > > On Tue, 9 Aug 2016 at 15:30 Andrew Ge Wu <andrew.ge...@eniro.com > <mailto:andrew.ge...@eniro.com>> wrote: > Oh sorry missed that part, no, Im not explicitly set that. > > >> On 09 Aug 2016, at 15:29, Aljoscha Krettek <aljos...@apache.org >> <mailto:aljos...@apache.org>> wrote: >> >> Hi, >> are you setting a StreamTimeCharacteristic, i.e. >> env.setStreamTimeCharacteristic? >> >> Cheers, >> Aljoscha >> >> On Tue, 9 Aug 2016 at 14:52 Andrew Ge Wu <andrew.ge...@eniro.com >> <mailto:andrew.ge...@eniro.com>> wrote: >> Hi Aljoscha >> >> >> Plan attached, there are split streams and union operations around, but here >> is how windows are created >> >> Confidentiality Notice: This e-mail transmission may contain confidential or >> legally privileged information that is intended only for the individual or >> entity named in the e-mail address. If you are not the intended recipient, >> you are hereby notified that any disclosure, copying, distribution, or >> reliance upon the contents of this e-mail is strictly prohibited and may be >> unlawful. If you have received this e-mail in error, please notify the >> sender immediately by return e-mail and delete all copies of this message. >> >> Let me know if I’m doing something out of ordinary here. >> >> >> >> Thanks! >> >> >> Andrew >>> On 09 Aug 2016, at 14:18, Aljoscha Krettek <aljos...@apache.org >>> <mailto:aljos...@apache.org>> wrote: >>> >> >>> Hi, >>> could you maybe post how exactly you specify the window? Also, did you set >>> a "stream time characteristic", for example EventTime? >>> >>> That could help us pinpoint the problem. >>> >>> Cheers, >>> Aljoscha >>> >> >>> On Tue, 9 Aug 2016 at 12:42 Andrew Ge Wu <andrew.ge...@eniro.com >>> <mailto:andrew.ge...@eniro.com>> wrote: >> >>> I rolled back to 1.0.3 >> >>> If I understand this correctly, the peak when topology starts is because it >>> is trying to fill all the buffers, but I can not see that in 1.1.0. >>> >>> >>> >> >>>> On 09 Aug 2016, at 12:10, Robert Metzger <rmetz...@apache.org >>>> <mailto:rmetz...@apache.org>> wrote: >>>> >>> >>>> Which source are you using? >>>> >>>> On Tue, Aug 9, 2016 at 11:50 AM, Andrew Ge Wu <andrew.ge...@eniro.com >>>> <mailto:andrew.ge...@eniro.com>> wrote: >>>> Hi Robert >>>> >>>> >>>> Thanks for the quick reply, I guess I’m one of the early birds. >>>> Yes, it is much slower, I’m not sure why, I copied slaves, masters, >>>> log4j.properties and flink-conf.yaml directly from 1.0.3 >>>> I have parallelization 1 on my sources, I can increase that to achieve the >>>> same speed, but I’m interested to know why is that. >>>> >>>> >>>> Thanks! >>>> >>>> >>>> Andrew >>>>> On 09 Aug 2016, at 11:47, Robert Metzger <rmetz...@apache.org >>>>> <mailto:rmetz...@apache.org>> wrote: >>>>> >>>>> Hi Andrew, >>>>> >>>>> here is the release announcement, with a list of all changes: >>>>> http://flink.apache.org/news/2016/08/08/release-1.1.0.html >>>>> <http://flink.apache.org/news/2016/08/08/release-1.1.0.html>, >>>>> http://flink.apache.org/blog/release_1.1.0-changelog.html >>>>> <http://flink.apache.org/blog/release_1.1.0-changelog.html> >>>>> >>>>> What does the chart say? Are the results different? is Flink faster or >>>>> slower now? >>>>> >>>>> >>>>> Regards, >>>>> Robert >>>>> >>>>> On Tue, Aug 9, 2016 at 11:32 AM, Andrew Ge Wu <andrew.ge...@eniro.com >>>>> <mailto:andrew.ge...@eniro.com>> wrote: >>>>> Hi, >>>>> >>>>> We found out there is a new stable version released: 1.1.0 but we can not >>>>> find any release note. >>>>> Do anyone know where to find it? >>>>> >>>>> >>>>> We are experience some change of behavior, I’m not sure if it is related. >>>>> >>>>> <PastedGraphic-1.png> >>>>> >>>>> Thanks >>>>> >>>>> >>>>> Andrew >>>>> >>>>> Confidentiality Notice: This e-mail transmission may contain confidential >>>>> or legally privileged information that is intended only for the >>>>> individual or entity named in the e-mail address. If you are not the >>>>> intended recipient, you are hereby notified that any disclosure, copying, >>>>> distribution, or reliance upon the contents of this e-mail is strictly >>>>> prohibited and may be unlawful. If you have received this e-mail in >>>>> error, please notify the sender immediately by return e-mail and delete >>>>> all copies of this message. >>>>> >>>> >>>> >>>> Confidentiality Notice: This e-mail transmission may contain confidential >>>> or legally privileged information that is intended only for the individual >>>> or entity named in the e-mail address. If you are not the intended >>>> recipient, you are hereby notified that any disclosure, copying, >>>> distribution, or reliance upon the contents of this e-mail is strictly >>>> prohibited and may be unlawful. If you have received this e-mail in error, >>>> please notify the sender immediately by return e-mail and delete all >>>> copies of this message. >>>> >>> >>> Confidentiality Notice: This e-mail transmission may contain confidential >>> or legally privileged information that is intended only for the individual >>> or entity named in the e-mail address. If you are not the intended >>> recipient, you are hereby notified that any disclosure, copying, >>> distribution, or reliance upon the contents of this e-mail is strictly >>> prohibited and may be unlawful. If you have received this e-mail in error, >>> please notify the sender immediately by return e-mail and delete all copies >>> of this message. > > > Confidentiality Notice: This e-mail transmission may contain confidential or > legally privileged information that is intended only for the individual or > entity named in the e-mail address. If you are not the intended recipient, > you are hereby notified that any disclosure, copying, distribution, or > reliance upon the contents of this e-mail is strictly prohibited and may be > unlawful. If you have received this e-mail in error, please notify the sender > immediately by return e-mail and delete all copies of this message. -- Confidentiality Notice: This e-mail transmission may contain confidential or legally privileged information that is intended only for the individual or entity named in the e-mail address. If you are not the intended recipient, you are hereby notified that any disclosure, copying, distribution, or reliance upon the contents of this e-mail is strictly prohibited and may be unlawful. If you have received this e-mail in error, please notify the sender immediately by return e-mail and delete all copies of this message.
Thanks for the help. Best regards Andrew
|
plan.json
Description: application/json
