Hi Starch,
It also depends on the applications behavior, some might not be properly
able to utilize the network. If you are using say Kafka, then one thing
that you should keep in mind is the Size of the individual message and the
number of partitions that you are having. The higher the message si
All,
I am trying to get data moving in and out of spark at 10Gb/s. I currently have
a very powerful cluster to work on, offering 40Gb/s inifiniband links so I
believe the network pipe should be fast enough.
Has anyone gotten spark operating at high data rates before? Any advice would
be apprec
