On Tue, Oct 22, 2013 at 9:32 AM, Tom.Fornoville <[email protected]> wrote: > Hi everyone, > > I have the following use-case: > * messages arrive on a queue, 1 message for each organisation be > created/updated, messages that belong to the same transaction (= our own > term, NOT a JPA transaction or something like that) all have the same > transactionGUID > > * we need to group organisations together in batches of 50 to process them, > the last batch can be smaller > > * during processing extra entities can be added to the queue: updates for an > enterprise can trigger updates for establishments under that enterprise > > * after processing by the bean we need to update the transaction entity with > statistics (# updated, # created etc...) > > I was thinking of doing this with 2 aggregators like this: > > <route id="process-organisations"> > <from uri="activemq:organisation.update.queue" /> > <aggregate strategyRef="organisationAggregator" completionSize="50" > completionTimeout="5000"> > <correlationExpression> > <header>transactionGUID</header> > </correlationExpression> > <bean ref="organisationServiceActivator" > method="updateOrganisations" /> > <to uri="activemq:organisation.processed.queue" /> > </aggregate> > </route> > > <route id="aggregate-organisation-results"> > <from uri="activemq:organisation.processed.queue" /> > <aggregate strategyRef="?"> > <correlationExpression> > <header>?</header> > </correlationExpression> > <completionPredicate> > <simple>?</simple> > </completionPredicate> > <bean ref="transactionServiceActivator" > method="updateTransaction" /> > </aggregate> > </route> > > The questions I have: > * are 2 routes with aggregators the best way to handle this use case? > * will the first aggregate keep working or will it stop after the first > batch of 50?
Yes it will keep working. > * what should be the correlation and completion paramters for the second > aggregation? > You would need somehow to know when a TX is done. I assume maybe one of the messages has a "hey dude there are no more messages in this TX" detail that you can use to know when its done. Then maybe in the first route you can set a special header in your organization strategy to indicate the TX is done. Then you can check that header in the 2nd route and call the bean. Though not sure if you really need 2 aggregators. Maybe one is enough? > Thanks in advance, > Tom > > > > > -- > View this message in context: > http://camel.465427.n5.nabble.com/Grouping-messages-for-batch-processing-tp5742015.html > Sent from the Camel - Users mailing list archive at Nabble.com. -- Claus Ibsen ----------------- Red Hat, Inc. Email: [email protected] Twitter: davsclaus Blog: http://davsclaus.com Author of Camel in Action: http://www.manning.com/ibsen
