Re: Job Manager minimum memory hard coded to 768

2017-09-30 Thread Haohui Mai
We have observed the same issue in our production cluster. Filed FLINK-7743 for the fix. ~Haohui On Fri, Sep 29, 2017 at 1:18 AM Till Rohrmann wrote: > Hi Dan, > > I think Aljoscha is right and the 768 MB minimum JM memory is more of a > legacy artifact which was never properly refactored. If I

Re: Finding things not seen in the last window

2017-09-30 Thread Haohui Mai
Hi, Assuming FLINK-6465 lands, will something like SELECT COUNT(*) FROM (SELECT FIRST_VALUE(names) FROM stream) GROUP BY HOP(proctime, INTERVAL '1' MINUTE, INTERVAL '1' MINUTE) works? ~Haohui On Fri, Sep 29, 2017 at 6:52 PM Ron Crocker wrote: > Hi - > > I have a colleague who is trying to wr

Re: Add custom configuration files to TMs classpath on YARN

2017-09-27 Thread Haohui Mai
What we internally did is to inherit from AbstractYarnClusterDescriptor and to customize from there. It's not too difficult but it's nice to see it can be taken care of by AbstractYarnCusterDescriptor. ~Haohui On Wed, Sep 27, 2017 at 9:36 AM Mikhail Pryakhin wrote: > Hi Robert, > > Thanks for

Re: Exception : Table of atomic type can only have a single field, when transferring DataStream to Table ?

2017-09-25 Thread Haohui Mai
Hi, I think instead of generating DataStream[BillCount], the correct way is to generate DataStream[Row], that is, kafkaInputStream.map(value -> Row.of(value.getLogis_id, value.getProvince_id, value.getCity_id, value.getOrder_require_varieties, value.getOrder_rec_amount, value.getStore_rec_date.ge