> For B1, we can scale out the service as controllers are scaled out, but it > would be much complex to manually assign topics.
Yes thats what my concern was in B1. So would for now target B2 approach where we have a dedicated new topic and then have it consumed by a new service. If it poses problem down the line then we can go for B1. B Chetan Mehrotra On Tue, Jun 25, 2019 at 10:08 AM Dominic Kim <style9...@gmail.com> wrote: > > Let me share a few ideas on them. > > Regarding option B1, I think it can scale out better than option B2. > If I understood correctly, scaling out of the service will be highly > dependent on Kafka. > Since the number of consumers is limited to the number of partitions, the > number of service nodes will be also limited to the number of partitions. > > So in the case of B2, if we create a new topic with some partition numbers, > we cannot scale out the service nodes more than that. > At some point, we may need to alter the number of partitions and it's not > easy in Kafka. > (Since the activation processing here is asynchronous, we may bear some > downtime(1~2s) to alter the partition. Then it would be fine.) > > In the case of B1, there will be many controller topics with their own > partitions. > Since controllers can be scaled out, there will be more topics, and the > activation service can scale out accordingly. > But in this case, we need to manually control the topic assignment. > (Not partition assignment, it will be done by Kafka.) > > Let's say we have 3 controller topics with 2 partitions each. > For HA, it would be great to have at least two nodes. > At first, both nodes will take care of all three topics. > Based on the partition assignment plan in Kafka, both nodes will fetch > activation messages without any duplication. > As controllers are scaled out, two nodes may not be enough to take care of > all topics. > At this point, we need to scale out the service nodes more. > Then we need to do logical partitioning for topics. > > For example, the node1 and 2 will take care of topic0 ~ 1 and node3 and 4 > will take care of topic2 ~ 3. > In this way, we can guarantee the minimum HA and scale out the nodes as > well. > Among them, topic partitions will be also assigned by Kafka. > > So in short, > For B1, we can scale out the service as controllers are scaled out, but it > would be much complex to manually assign topics. > And one node may have more than one Kafka consumers. > > For B2, scaling might be limited unless we have a big enough number of > partitions at topic creation time. > But if we can bear some downtime, this might not be a problem and this > option will be a lot simpler. > > Best regards > Dominic. > > > > > 2019년 6월 24일 (월) 오후 6:50, Chetan Mehrotra <chetan.mehro...@gmail.com>님이 작성: > > > Okie so we can then aim for adding an optional support for storing > > activations via a separate service. > > > > Currently we also send the activation result on respective controller > > topic. With this change we would also be sending same activation > > record on another topic. So we have another choice to make > > > > Option B1 - Activations via controller topic > > -------------------------------------------------------- > > > > Here we avoid a new topic and instead have a service which listen to > > all controller topics for activation records. However that would be > > tricky to implement and also tricky to scale out. As scaling out such > > a service by running multiple copies would not be easy in terms of > > sharding/partitioning > > > > Here the benefit is that we reduce the duplicate writes on Kafka. > > > > Option B2 - Introduce a new topic altogether > > ----------------------------------------------------------- > > > > We introduce a new topic to which all invokers write the activation > > records (like the case for user-events). Then implementing a new > > service to read from a single (possibly partitioned topic) would be > > easier. > > > > My suggestion is to go for B2 for now. > > > > Any feedback on that? > > > > Chetan Mehrotra > > > > On Fri, Jun 21, 2019 at 11:46 PM Rodric Rabbah <rod...@gmail.com> wrote: > > > > > > > Can we handle these in same way as user events? Maybe exactly like user > > > events, as in use a single service to process both topics. > > > > > > good call - the user events already contains much of the activation > > record > > > (if not all modulo the logs)? > > > > > > -r > >
