For the record, the amount of disk storage space per unit cost has doubled every 14 months for the last 30 years. It's an exponential relationship: www.mkomo.com/cost-per-gigabyte
So data generated at a very high rate today, will be trivial to store in the near future. That's not to say it is cost free, of course ... but exponentially approaching free. I worked at a Supercomputing facility for 7 years. At that time whole rooms were filled with state-of-the-art tape archive robots that could hold an unimaginable amount of data: a whole terabyte. Today, of course, that same volume costs under 100 USD with much faster I/O ... and I have personal copies of everything I generated (even digitized, uncompressed analog video). To keep data backed up and online, of course costs something, but distributed/cloud computing is also changing that picture dramatically. I am curious to know: those who have Pilatus 6M, for example. How much data do you generate in a year? I suspect this is limited by beam intensity ... at the moment. Richard On Oct 18, 2011, at 6:52 AM, Chris Morris wrote: > Some crystals are hard to make, so storing all the data the best way to get > reproducibility. On the other hand, no one needs more images of lysozyme. So > using the same standard for every deposition doesn't sound right. > > The discussion should be held on the basis of overall cost to the research > budget - not on the assumption that some costs can be externalised. It is too > easy to say "you should store the images, in case I want to reprocess them > sometime". IT isn't free, nor is it always cheaper than the associated > experimental work. The key comparison is: > > Cost of growing new crystals + cost of beam line time > > With: > > Cost of storing images * probability of processing them again > > At present, detectors are improving more quickly than processing software. > Sample preparation methods are also improving. These forces both press > downward the probability that a particular image will ever be reprocessed. > > regards, > Chris
