That's great. Cheers James. Ozan
-----Original Message----- From: James Green [mailto:james.mk.gr...@gmail.com] Sent: 10 June 2011 09:21 To: users@activemq.apache.org Subject: Re: Kahadb Corruption Scenarios http://fusesource.com/docs/broker/5.4/persistence/KahaDB-Recovery.html Don't forget there are hardware-level performance improvements from the operating system down to the disks which may affect reliability. James On 8 June 2011 16:53, Ozan Seymen <ozan.sey...@tdpg.com> wrote: > Hi all, > > Can you please explain the scenarios where we might have AMQ message > storage (kahadb) corrupted in a way that it is not recoverable? > > In the solution I am working on, I simply cannot afford to lose any > messages. In order to secure this, I will: > > > * Rely on transactions on the publisher side (both performance and > reliability). This should ensure that I will catch exceptions for > every batch. If commit succeeds, broker should assume responsibility > and have persisted the message to disk (can you please confirm whether > this is > correct?) > > * Implement durable queues/messages (deliverymode is persistent) > > * Rely on acks on consumer side (dubs_ok probably). > Even though all of these above prevent message loss in normal > conditions, none of them covers the case where data gets corrupted in > the broker. There is a window (albeit small) that things might go > wrong: broker assumes responsibility (message is in the disk) but > before message is sent to the consumer, broker experiences problems > which corrupts the storage. If I can't bring the broker up with all > data previously persisted, I've lost messages - producer forgot about > the message as broker accepted responsibility and consumers have no > idea about the messages since they haven't been delivered to it. > So does kahadb have some sort of transaction log that it keeps which > should recover messages? If not, would changing the message store to a > JDBC provider (and use Postgre, MySql, SQL Server, etc) help? > I would really appreciate any info you guys can share. > Cheers, > Ozan > > > > > ________________________________ > This e-mail and any attached files are intended for the named > addressee only. It contains information which may be confidential and > legally privileged and also protected by copyright. Unless you are the > named addressee (or authorised to receive for the addressee) you may > not copy or use it or disclose it to anyone else. If you received it > in error please notify the sender immediately and then delete it from your > system. > > Please be advised that the views and opinions expressed in this e-mail > may not reflect the views and opinions of The Digital Property Group > Limited or any of its subsidiary companies. > > We make every effort to keep our network free from viruses. However, > you do need to check this e-mail and any attachments to it for viruses > as we can take no responsibility for any computer virus which may be > transferred by way of this e-mail. We reserve the right to monitor all > e-mail communications. > > The Digital Property Group Limited is a Daily Mail and General Trust > plc company. > Registered Office: Northcliffe House, 2 Derry Street, London, W8 5TT. > Registered in England & Wales No: 02290527 VAT no. 243571174 > > ______________________________________________________________________ > This email has been scanned by the MessageLabs Email Security System. > For more information please visit http://www.messagelabs.com/email > ______________________________________________________________________ ______________________________________________________________________ This email has been scanned by the MessageLabs Email Security System. For more information please visit http://www.messagelabs.com/email ______________________________________________________________________ This e-mail and any attached files are intended for the named addressee only. It contains information which may be confidential and legally privileged and also protected by copyright. Unless you are the named addressee (or authorised to receive for the addressee) you may not copy or use it or disclose it to anyone else. If you received it in error please notify the sender immediately and then delete it from your system. Please be advised that the views and opinions expressed in this e-mail may not reflect the views and opinions of The Digital Property Group Limited or any of its subsidiary companies. We make every effort to keep our network free from viruses. However, you do need to check this e-mail and any attachments to it for viruses as we can take no responsibility for any computer virus which may be transferred by way of this e-mail. We reserve the right to monitor all e-mail communications. The Digital Property Group Limited is a Daily Mail and General Trust plc company. Registered Office: Northcliffe House, 2 Derry Street, London, W8 5TT. Registered in England & Wales No: 02290527 VAT no. 243571174 ______________________________________________________________________ This email has been scanned by the MessageLabs Email Security System. For more information please visit http://www.messagelabs.com/email ______________________________________________________________________
