Okay, it took my some time to figure it out, but here it is:
On Tue, 13 Apr 1999, Jon Wright wrote:
>
> Did this get posted twice? I thought I was waiting to read an explanation
> myself, but I could just be going mad.... guess I'll have a go myself.
>
> > I have a question concerning the errors calculated in a rietveld
> > refinement. As far as I understand, the error of a parameter is calculated
> > as
> >
> > sigma(p_i) = sqrt(c_ii) * sqrt(chi^2/N-P) (*)
> >
> > where c_ii is the i-th diagonal element of the inverse of the
> > curvature matrix, chi^2 = sum_i w_i*(yobs_i - ycalc_i), N number of
> > observations, P number of parameters.
>
> Are you sure about these? I thought chi^2 should contain the factor N-P,
> also it is usually (obs-calc)^2. Pedantics aside you have something like
> the statistical error * chi.
Just a mix-up I'd think; in some publications they used a nomenclature
S^2 = chi^2 = (sum_i w_i*(yobs_i - ycalc_i)) / (N-P)
while other papers use
S^2 = sum_i w_i*(yobs_i - ycalc_i)
>
> > But I thought the statistical error is just sqrt(c_ii)!
>
> Yup, it is.
>
> > The second factor
> > is alway << 1, because N is a large number, and so the calculated errors
> > are much to small! Even more, I can minimize them by increasing the number
> > of steps, although, after a certain point, I gain no more information.
>
> No! Remember that the chi^2 as you have defined it increases with the
> number of observations you make - as you sum over i. Using
> chi^2=[sum_i w_i*(yobs_i - ycalc_i)^2]/(N-P),
> you can see that chi^2 is approximately the average number of esd's you
> are wrong by for each point in the pattern (N >> P). So the second factor
> is your familiar chi^2; very rarely << 1 !
>
> The correction you are applying to the diagonal elements of your matrix
> is to try to allow for the systematic errors (ie, non random) in your
> model. This appears to be your source of confusion. From Young's book (the
> Rietveld Method) E.Prince says, "It is customary to multiply the e.s.d's
> by the chi factor, on the assumption that the lack of fit is due to an
> over-weighting of all data points. While this practice is conservative, it
> should be understood that it has no basis in statistics."
>
> If you look at a difference curve there will be random noise and
> systematic problems, such as peak shape or intensity or position. This fix
> says that the systematic problems are really just random noise and
> effectively increases the weights on the y_obs accordingly, giving larger
> esd's.
>
But the source of the problem is not the correction term
sqrt(S^2 / (N-P)) = chi
which in first approximation for high N is independent of the number of
observations N.
But the diagonal elements of the inverse of the curvature matrix c_ii will
also decrease nearly proportional to 1/N and this results in the
fact that the e.s.d's sigma_1 behave like 1/sqrt(N).
This decrease would be justified in the absence of systematic problems
but it is not in the presence of systematic errors where neighbouring
points in the diffraction pattern are not independent observations but
suffer from the same systematic errors (local correlation).
A discussion of this effect and proposed ways for its correction can be
found i.e. in
Berar, J.-F.; Lelann, P. E.S.D.'s an estimated probable error obtained in
rietveld refinements with local correlations. J. Appl. Cryst. (1991), 24,
1-5
Scott, H. G.: The estimation of standard deviations in powder diffraction
rietveld refinements. J. Appl. Cryst. (1983) 16, 159-163
The correction parameters proposed in these papers are i.e. calculated by
FULLPROF.
>
> PS: See also GSAS/fullprof manuals and "Rietveld refinement guidelines"
> McCusker et al, J.Appl,Cryst 32(1999),36-50. Also the non-Rietveld
> specific - Acta Cryst A45 (1989) 63-75. "Statistical descriptors in
> crystallography".
>
Daniel Toebbens
***********************************************************************
Daniel M. Toebbens Tel.: +49 30 8062-2793
Hahn-Meitner-Institut Fax : +49 30 8062-2999
Glienicker Strasse 100
D-14109 Berlin, Germany http://www.hmi.de/people/toebbens/
***********************************************************************
