There is a formula for sigma(F) (aka "SIGF"), but it is actually a
common misconception that it is simply related to F. You need to know a
few other things about the experiment that was done to collect the
data. The misconception seems to arise because the fist thing textbooks
tell you is that F = sqrt(I), where "I" is the intensity of the spot.
Then, later on, they tell you that sigma(I) = sqrt(I) because of
"counting statistics". Now, if you look up a table of error-propagation
formulas, you will find that if I=F^2, then sigma(I)/I = 2*sigma(F)/F,
and by substituting these equations together you readily obtain:
sigma(F) = F/2*sigma(I)/I
sigma(F) = F/2*sigma(I)/F^2
sigma(F) = sigma(I)/(2*F)
sigma(F) = sigma(I)/(2*sqrt(I))
sigma(F) = sqrt(I)/(2*sqrt(I))
sigma(F) = 0.5
Which says that the error in F is always the same, no matter what your
exposure time? Hmm.
The critical thing missing from the equations above is something we
crystallographers call a "scale factor". We love scale factors because
they let us get away with not knowing a great many things, like the
volume of the crystal, the absolute intensity of the x-ray beam, and the
exact "gain" of the detector. It's not that we can't measure or look up
these things, but few of us have the time. And, by and large, as long
as you are aware that there is always an unknown "scale factor", it
doesn't really get in your way. So, the real equation is:
I_in_photons = scale*F^2
where scale =
Ibeam*re^2*Vxtal*lambda^3*Loentz_factor*Polar_factor*Attenuation_factor*exposure_time/deltaphi/Vcell^2
This "scale factor" comes from Equation 1 in the following paper:
http://dx.doi.org/10.1107/S0907444910007262
where we took pains to describe the exact meaning of each of these
variables (and their units!) in great detail. It is open access, so I
won't go through them here. I will, however, add that for spots on the
detector there are a few other factors still missing, like the detector
gain, obliquity, parallax, and spot partiality, but these are all "taken
care of" by the data processing program. The main thing is to figure
out the number of photons that were accumulated for a given h,k,l index,
and then take the square root of that to get the "counting error". Oh,
and you also need to know the number of "background" photons that fell
into the pixels used to add up photons for the h,k,l of interest. The
square root of this count must be combined with the "counting error" of
the spot photons, along with a few other sources of error. This is what
we discuss around Equation (18) in the linked-to paper above.
The short answer, however, is that sqrt(I_in_photons) is only one
component of sigma(I). The other factors fall into three main
categories: readout noise, counting noise and what I call "fractional
noise". Now, if you have a number of different sources of noise, you
get the total noise by adding up the squares of all the components, and
then taking the square root:
sigma(I_in_photons) = sqrt( I_in_photons + background_photons +
sigma_readout^2 + frac_error*I_in_photons^2 )
For those of you who use SCALA and think the sqrt( sigI^2 + B*I +
sdadd*I^2 ) form of this equation looks a lot like the SDCORRection
line, good job! That is a very perceptive observation.
What separates the three kinds of noise is how they relate to the
exposure time. For example, readout noise is always the same, no matter
what the exposure time is, but as you increase the exposure time, the
number of photons in the spots and the background go up proportionally.
This means that the contribution of "counting noise" to sigma(I)
increases as the square root of the exposure time. On modern detectors,
the read-out noise is equivalent to the "counting noise" of a few (or
even zero) photons/pixel, and so as soon as you have more than about 10
photon/pixel of background, the readout noise is no longer significant.
So, in general, noise increases with the square root of exposure
time, but the signal (I_in_photons) increases in direct proportion to
exposure time, so the signal-to-noise ratio (from counting noise alone)
goes up with the square root of exposure time. That is, until you hit
the third type of noise: fractional noise. There are many sources of
fractional noise: shutter timing error, crystal vibration, fliker in the
incident beam intensity, inaccurate scaling factors (including the
absorption correction), and variations in detector sensitivity across
its face. Essentially, these amount to the error bars on all the terms
in the "scale" formula above. On a good diffractometer, all these
errors are small, usually less than a few percent, but one thing they
all have in common is their contribution to sigma(I) is proportional to
the the signal (I_in_photons), not the square root of it! This is the
reason why the Rmerge of bright, low-angle spots never gets down to the
the 0.1% you would expect from counting a million photons, even if you
did count that many. It is also the reason why "SDadd" in SCALA (the
"estimated error" in scalepack) tends to be around 3-5%. It is also the
reason why measuring anomalous differences smaller than ~3% is so difficult.
So, if you know the magnitude of all these sources of error, then it
should be possible to derive a formula for SIGF, but I don't think
anyone has quite written down the whole thing yet. Possibly because the
difference between Fobs and Fcalc is about 4-5x bigger than SIGF anyway.
-James Holton
MAD Scientist
On 8/18/2011 8:19 AM, G Y wrote:
Dear all,
I am a student in crystallography. So not quite familiar with some
even basic concepts.
In shelx .hkl file or ccp4 .mtz file there is a column SIGF which is
related to standard deviation of the structure factor. I read through
many text book for crystallography, there are many formulas about this
topic. Sometimes it is a square of sigma, sometimes it is not.
My question is:
1. What is the exact mathematical formula for SIGF or SIGFP in ccp4 or
shelx format file?
2. If it can be calculated from F, why it is necessary to include it
in ccp4 or shelx reflection file (they have F already) ?
3. Is this value really important in structure determination? Why and
how? As I understood, during data collection each reflection measured
several times, so there is a deviation from the average F. That is the
meaning of SIGF. But how to use this value in structure determination?
Is there some kind of correction or refinement on F according to SIGF?
And also when multiplicity during data collection is low, the SIGF
would not be so interested. So is that means the SIGF would not be so
important in some measurements?
Any kind reply from you guys would be greatly appreciated. Many thanks!
Best regards,
G
