d my -XX:ThreadStackSize to 5m (instead of 10m - default is
>> 1m), which is still OK for my need.
>> 2. I reduced the executor memory to 44GB for a 60GB machine (instead of
>> 49GB).
>>
>> This seems to have helped. Thanks to Matthew and Sean.
>>
>> Thomas
>&
My memory is hazy on this but aren't there "hidden" limitations to
Linux-based threads? I ran into some issues a couple of years ago where,
and here is the fuzzy part, the kernel wants to reserve virtual memory per
thread equal to the stack size. When the total amount of reserved memory
(not nece
Folks,
We are running a streaming job on a cluster that is failing with:
'ERROR BlockGenerator: Error in block pushing thread' via a Futures timeout.
The cluster is running on EC2. Data is fed in via a Kafka client. I have
been looking for network communications issues but haven't found any smoki
