We have been hosting kafka brokers in Amazon EC2 and we are using EBS disk. But periodically we were hit by long I/O wait time on EBS in some Availability Zones.
We are thinking to change the instance types to a local HDD or local SSD. HDD is cheaper and bigger and seems quite fit for the Kafka use case which is mostly sequential read/write, but some early experiments show the HDD cannot catch up with the message producing speed since there are many topic/partitions on the broker which actually makes the disk I/O more randomly accessed. How are people's experience of choosing disk types on Amazon?
