I am a newbie to Camel and am trying to set up Camel route(s) to tackle this situation.
I want to update a local database with records that are made available through a web API and periodically pull in new records to the database. Unfortunately, the only API to access the records is a paged API where all records are available from newest to oldest. The web API takes offset and length parameters to specify where to offset into the list and how many records to return, which it returns as XML. So the logic, needs to be that I start at an offset of zero and load pages of records until I get to a record that I already have in my database (determined by the record ID). In the case of an initial load I would keep loading pages until there is no more data (determined by getting an empty result back). Once I have all the new records I need to put them into the database as one DB transaction. I understand the pieces to schedule routes, and to actually do html call to get back the XML results, but am struggling how to handle the looping nature of the task. I want to do the job as much as possible in camel. Any suggestions would be appreciated -- Dale King
