Wow.
I don't know about the rest of you, but I got told three times.
Gerard is, of course, right about pixel non-independence (think "point
spread function", among other things), and I wouldn't care to argue
statistics with him, but as far as I know (and I could well be wrong) most
of the integration programs out there _do_ use counting statistics (i.e.
Poisson statistics) at least as a first approximation for the random error
in measurement; this may be modified by some "detector inefficiency
factor" (See Borek, Minor & Otwinowski, Acta Cryst (2003) D59 2031 -
2038), but it's still there and being used by "everyone", nonetheless.
Having said that, regarding the storage of images, my personal feeling is
that there's no real point in using a lossy compression when there are
good lossless systems out there. I also think that almost no-one would
ever bother to reprocess deposited images anyway; my guess is that
"unusual" structures would be detected by other means, and that examining
the original images would rarely shed light on the problem.
I think we need to stop and think right here. The errors in pixel
values of images are neither Poisson (i.e. forget about taking square roots)
nor independent. Our ideas about image statistics are already disastrously
poor enough: the last thing we need is to make matters even worse by using
compression methods based on those erroneous statistical arguments!
With best wishes,
Gerard.
--
On Fri, Aug 24, 2007 at 01:20:29PM +0100, Harry Powell wrote:
Hi
Lossy compression should be okay, provided that the errors introduced are
smaller than those expected for counting statistics (assuming that the
pixels are more-or-less independent) - i.e. less than the square-root of
the individual pixel intensities (though I don't see why this can't be
extended to the integrated reflection intensities). So it's more important
to accurately retain your weak pixel values than your strong ones - an
error of ±10 for a pixel in a background count where the background should
be 40 is significant, but an error of ±10 for a saturated pixel on most
detectors (say, about 64K for a CCD) wouldn't affect anything.
On the question of lossy compression, I think we'd have to ask some data
reduction guru's how much the "noise" would affect the data reduction. I
suspect that the main problem is that the noise added would be
correlated across the image and would therefore affect the background
statistics in a non-trivial way. Although the intensity measurements may
not be badly affected the error estimates on them could be...
Harry
--
Dr Harry Powell, MRC Laboratory of Molecular Biology, MRC Centre, Hills
Road, Cambridge, CB2 2QH
--
===============================================================
* *
* Gerard Bricogne [EMAIL PROTECTED] *
* *
* Global Phasing Ltd. *
* Sheraton House, Castle Park Tel: +44-(0)1223-353033 *
* Cambridge CB3 0AX, UK Fax: +44-(0)1223-366889 *
* *
===============================================================
Harry
--
Dr Harry Powell, MRC Laboratory of Molecular Biology, MRC Centre, Hills
Road, Cambridge, CB2 2QH
