Hi Bill and others
Why cant this question of zero counts be answered in a simple and practical
manner. I hope to read Devinder Sivia's excellent book one day but for the
time being it would be useful if the statistical heavy weights were to
advise on what weighting is appropriate without everyone havng to understand
the details.
The original question was how to get XFIT to load data files with zero
counts; obviously setting the values to 1 is incorrect. Joerg Bergmann seems
to indicate that the weihgting should be:
weighting = 1 / (Yobs+1);
as the esd is sqrt(n+1). Again without reading the book should the weighting
be:
weighting = 1 / If(Yobs, Yobs, Ycalc);
Hopefully these spread sheet type formulas are understandable. This last
equation is not liked by computers due to a possible zero divide when Ycalc
is zero.
Any ideas Bill and others
Alan
________________________________
From: David, WIF (Bill) [mailto:[EMAIL PROTECTED]
Sent: Thursday, 12 October 2006 4:48 PM
To: rietveld_l@ill.fr
Subject: RE: About zero counts etc.
Dear all,
Jon's right - when the counts are very low - i.e. zeroes and ones around -
then the correct Bayesian approach is to use Poisson statistics. This, as
Jon said, has been tackled by Antoniadis et al. (Acta Cryst. (1990). A46,
692-711 Maximum-likelihood methods in powder diffraction refinements, A.
Antoniadis, J. Berruyer and A. Filhol) in the context of the Rietveld method
some years ago. This paper is very informative for those who are intrigued
about the fact that you can do anything when diffraction patterns have lots
of zeroes and ones around. Curiously, the weighting ends up having as much
to do with the model value (which can, of course, be non-integer) as the
data. Devinder Sivia's excellent OUP monograph, "Data Analysis: a Bayesian
Tutorial" (http://www.oup.co.uk/isbn/0-19-856832-0) discusses all of this in
a very readable way.
Bill
-----Original Message-----
From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]
Sent: 12 October 2006 00:37
To: rietveld_l@ill.fr
Subject: Re: About zero counts etc.
Hello Joerg,
> -Having measured n counts, the estimated value is n+1
You might have a hard time convincing me on that one.
> -Having measured n counts, the esd is also sqrt(n+1)!
If n is zero then spending more time on the data collection might be better
than
more time on the analysis.
> Things change with variable counting times.
id31sum uses counts=counts and esd=sqrt(counts+alp) where alp=0.5 is the
default
and can be overridden on the command line. Perhaps there aren't many people
who
use that option. Should we change the default? The 0.5 came from the
literature
but it was some time ago and I can't remember where. In any case it then
gets
convoluted with the monitor error. Sqrt(n+1) gives a very low chi^2 if the
actual background is 0.1 (eg: 1 count every 10 datapoints). Might be better
to
just use the Poisson itself, as in abfit [1].
> the above correction for the estimated
> values gave significant better R values.
Are you using background subtracted R-values? If only R-values were
significant.
Jon
[1] Acta Cryst. (1990). A46, 692-711
Maximum-likelihood methods in powder diffraction refinements
A. Antoniadis, J. Berruyer and A. Filhol
-------------------------------------------------
This mail sent through IMP: http://horde.org/imp/
