We have a largeish kinesis stream with about 25k events per second and each record is around 142k. I have tried multiple cluster sizes, multiple batch sizes, multiple parameters... I am doing minimal transformations on the data. Whatever happens I can sustain consuming 25k with minimal effort and cluster load for about 5-10 minutes and then always always the stream shapes down and hovers around 5k EPS.
I can give MANY more details but I was curious if anyone had seen similar behavior. Thanks, Jeremy -- Jeremy Kelley | Technical Director, Data jkel...@carbonblack.com | Carbon Black Threat Engineering
smime.p7s
Description: S/MIME cryptographic signature
