Re: Slow Performance inquiry

2020-09-10 Thread Timo Walther
mailto:user@flink.apache.org>> *Subject:* Re: Slow Performance inquiry Hi Hazem, I guess your performance is mostly driven by the serialization overhead in this case. How do you declare your state type? Flink comes with different serializers. Not all of them are extracted

Re: Slow Performance inquiry

2020-09-09 Thread David Anderson
} >)); > > > Is there a better way? > > Regards, > > Heidy > -- > *From:* Timo Walther > *Sent:* Wednesday, September 9, 2020 1:58 PM > *To:* user@flink.apache.org > *Subject:* Re: Slow Performance inquiry > > Hi

Re: Slow Performance inquiry

2020-09-09 Thread Heidi Hazem Mohamed
ds, Heidy From: Timo Walther Sent: Wednesday, September 9, 2020 1:58 PM To: user@flink.apache.org Subject: Re: Slow Performance inquiry Hi Hazem, I guess your performance is mostly driven by the serialization overhead in this case. How do you declare your

Re: Slow Performance inquiry

2020-09-09 Thread Timo Walther
Hi Hazem, I guess your performance is mostly driven by the serialization overhead in this case. How do you declare your state type? Flink comes with different serializers. Not all of them are extracted automatically when using reflective extraction methods: - Note that `Serializable` decla

Slow Performance inquiry

2020-09-09 Thread Heidi Hazem Mohamed
Dear, I am writing a Flink program(Recommender system) needed a matrix as a state which is the rating matrix, While the matrix is very sparse, I implemented a sparse binary matrix to save the memory and save only the ones, not all the matrix and use it as a data type and save it in a value Stat