We are doing this with cassandra. But we cache a lot. We get around 20 writes/s and 1k reads/s (~ 100Mbit/s) for that particular CF but only 1% of them hit our cassandra cluster (5 nodes, rf=3).
/Daniel On Feb 4, 2011, at 9:37 AM, Brendan Poole wrote: > Hi Daniel > > When you say "We are doing this" do you mean via NFS or Cassandra. > > Thanks > > Brendan > > > > > > > <Signature.jpg> Brendan Poole > Systems Developer > NewLaw Solicitors > Helmont House > Churchill Way > Cardiff > brendan.po...@new-law.co.uk > 029 2078 4283 > www.new-law.co.uk > > > > > > From: Daniel Doubleday [mailto:daniel.double...@gmx.net] > Sent: 03 February 2011 17:21 > To: user@cassandra.apache.org > Subject: Re: Using Cassandra to store files > > Hundreds of thousands doesn't sound too bad. Good old NFS would do with an ok > directory structure. > > We are doing this. Our documents are pretty small though (a few kb). We have > around 40M right now with around 300GB total. > > Generally the problem is that much data usually means that cassandra becomes > io bound during repairs and compactions even if your hot dataset would fit in > the page cache. There are efforts to overcome this and 0.7 will help with > repair problems but for the time being you have to have quite some headroom > in terms of io performance to handle these situations. > > Here is a related post: > > http://comments.gmane.org/gmane.comp.db.cassandra.user/11190 > > On Feb 3, 2011, at 1:33 PM, Brendan Poole wrote: > >> Hi >> >> Would anyone recommend using Cassandra for storing hundreds of thousands of >> documents in Word/PDF format? The manual says it can store documents under >> 64MB with no issue but was wondering if anyone is using it for this specific >> perpose. Would it be efficient/reliable and is there anything I need to >> bear in mind? >> >> Thanks in advance >> >> >> <Signature.jpg> Brendan Poole >> Systems Developer >> NewLaw Solicitors >> Helmont House >> Churchill Way >> Cardiff >> brendan.po...@new-law.co.uk >> 029 2078 4283 >> www.new-law.co.uk >> >> >> >> >> >> P Please consider the environment before printing this e-mail >> Important - The information contained in this email (and any attached files) >> is confidential and may be legally privileged and protected by law. >> The intended recipient is authorised to access it. If you are not the >> intended recipient, please notify the sender immediately and delete or >> destroy all copies. You must not disclose the contents of this email to >> anyone. Unauthorised use, dissemination, distribution, publication or >> copying of this communication is prohibited. >> NewLaw Solicitors does not accept any liability for any inaccuracies or >> omissions in the contents of this email that may have arisen as a result of >> transmission. This message and any attachments are believed to be free of >> any virus or defect that might affect any computer system into which it is >> received and opened. However, it is the responsibility of the recipient to >> ensure that it is virus free; therefore, no responsibility is accepted for >> any loss or damage in any way arising from its use. >> NewLaw Solicitors is the trading name of NewLaw Legal Ltd, a limited company >> registered in England and Wales with registered number 07200038. >> NewLaw Legal Ltd is regulated by the Solicitors Regulation Authority whose >> website is http://www.sra.org.uk >> The registered office of NewLaw Legal Ltd is at Helmont House, Churchill >> Way, Cardiff, CF10 2HE. Tel: 0845 756 6870, Fax: 0845 756 6871, Email: >> i...@new-law.co.uk. www.new-law.co.uk. >> We use the word ‘partner’ to refer to a shareowner or director of the >> company, or an employee or consultant of the company who is a lawyer with >> equivalent standing and qualifications. A list of the directors is displayed >> at the above address, together with a list of those persons who are >> designated as partners. > > > > > P Please consider the environment before printing this e-mail > Important - The information contained in this email (and any attached files) > is confidential and may be legally privileged and protected by law. > The intended recipient is authorised to access it. If you are not the > intended recipient, please notify the sender immediately and delete or > destroy all copies. You must not disclose the contents of this email to > anyone. Unauthorised use, dissemination, distribution, publication or copying > of this communication is prohibited. > NewLaw Solicitors does not accept any liability for any inaccuracies or > omissions in the contents of this email that may have arisen as a result of > transmission. This message and any attachments are believed to be free of any > virus or defect that might affect any computer system into which it is > received and opened. However, it is the responsibility of the recipient to > ensure that it is virus free; therefore, no responsibility is accepted for > any loss or damage in any way arising from its use. > NewLaw Solicitors is the trading name of NewLaw Legal Ltd, a limited company > registered in England and Wales with registered number 07200038. > NewLaw Legal Ltd is regulated by the Solicitors Regulation Authority whose > website is http://www.sra.org.uk > The registered office of NewLaw Legal Ltd is at Helmont House, Churchill Way, > Cardiff, CF10 2HE. Tel: 0845 756 6870, Fax: 0845 756 6871, Email: > i...@new-law.co.uk. www.new-law.co.uk. > We use the word ‘partner’ to refer to a shareowner or director of the > company, or an employee or consultant of the company who is a lawyer with > equivalent standing and qualifications. A list of the directors is displayed > at the above address, together with a list of those persons who are > designated as partners.
