Hi,
We arrange our kafka machines in groups and deploy these phases.
For kafka, we’ll have to map groups with phases. During each phase of the
release, all the machines in that group can go down.
When this happens, there are a couple of cases:-
1. All replicas are residing in a group of machines which will all go
down in this phase
- Affect on Producer –
- What happens to the produce requests (whether produce can
dynamically keep writing to the remaining partitions now)
- What happens to the already queued requests which were being
sent to the earlier replicas – they will fail (we’ll have to
use producer
callback feature to take care of retrying in case the above step
works fine)
- Affect on Consumer -
- Can the consumers consume from a lesser number of partitions?
- Does the consumer 'consume' api gives any callback/failure when
all replicas of a partition go down?
If you have come across any of the above cases, please provide how you
solved the problem ? or whether everything works just well with Kafka
during deployments and my cases described above are all invalid or handled
by kafka and its clients internally ?
Thanks,
Prabhjot
