Its still April, but this one isn't a joke.
The smiley-face electron density in the left panel of the attached image
has the remarkable property that any attempt to sharpen or blur the map
turns it into the frowny-face on the right. If you'd like to try this
yourself, the hidden_frown.map file is available in this tarball:
https://bl831.als.lbl.gov/~jamesh/bugreports/fft_042423.tgz
In fact, any use of an FFT, even with the sharpening B set to zero,
turns the smiley into a frowny face. There is no way to get the smiley
face back (except opening the file again). Yes, that's right, even just
a simple back-and-forth FFT: turning this hidden_frown.map into
structure factors and then back into a map again, gives you a frowny
face. This happens using coot, ccp4 and phenix.
Wait, what!? Isn't a Fourier transform supposed to preserve
information? As in: you can jump back and forth between real and
reciprocal space with impunity? Without introducing error? Well, yes,
it is SUPPOSED to work like that, but the 3D FFT algorithms of
structural biology have a ... quirk. If you start with structure factors
and make a map out of them, you can convert it back-and-forth as often
as you want with 100% preservation of information. However, if you
start with a real-space map (such as from cryoEM), a back-and-forth
conversion gives you a different map. This new map can then be
transformed back-and-forth all you want and be 100% preserved. It has
been "christened" by the FFT, but it is not the same as the starting
map, which is impossible to recover from the FFT-transformed data.
Information has been lost. It is fine for crystallography (which starts
with structure factors), but for techniques such as cryoEM that start
with maps, using an FFT changes the data.
What information is being lost? Sharp edges. These turn into ripples
covering all of real and reciprocal space. Do real-world data have sharp
edges? Well, the all-or-nothing masks we use to model bulk solvent are
one example. Also, if you "mask off" otherwise smooth density with an
all-or-nothing mask, you will get similar ripples. Another example of a
sharp edge might be the large changes between adjacent pixels you see
when a single electron hits a detector. For example, if you make a map
with just one non-zero voxel and run it back-and-forth through FFT you
will find that voxel loses from 50% to 99% of its value (depending on
the size of the map). How much does this actually impact cryo-EM data?
That is my question.
What evil magic did I wield to make this map? Well, I drew a smiley and
frowny face by hand, converted them to maps, and then I generated random
noise within the boundaries of the smiley face. I ran this noisy map
back-and-forth through FFT, and then subtracted the map that survives
the FFT from the pre-FFT map. This cheshire_smile.map has the
interesting property that all of the structure factors calculated from
it are zero. It has an RMSD of 1.4, but after a back-and-forth FFT this
RMSD drops to 1e-7. I generated the hidden_frown.map by simply summing
the frowny.map with cheshire_smile.map.
But isn't this map getting less noisy? Yes it is, but the interpretation
clearly changes as well.
Why does this happen? It is because of a finite resolution cutoff. Oh!
What a relief! You don't have super-high resolution, do you? Well, no,
almost nobody has signal out beyond 1.0 A, but we do have noise. In
diffraction data this noise is removed by simply not measuring it. For
map data, however, the problem is that noise at very high frequencies
(small-number resolutions) is hard to avoid. This is because of another
phenomenon NMR spectroscopists are very familiar with: aliasing, or
"folding". If any high-spatial frequency noise exists above half the
sampling rate (or "Nyquist frequency") it still gets recorded, but shows
up in a lower-frequency Fourier coefficient. It is not possible to
remove such aliasing noise after digitization. Upon discretization of
the signal (FFT or no) all these high frequencies join with lower
frequency terms, and so survive any low-pass filtering. Darn.
Why am I bringing this up? Because if there is noise out beyond the FFT
resolution limit it implies there is also noise out beyond the
Nyquist-Shannon limit as well. If that is the case, direct-space imaging
data may be a lot noisier than it needs to be. In general, in digital
signal processing of things like sound an analog low-pass filter is
always installed at the input of any digitizer. Perhaps this is why
de-focusing works better than being at focus?
What is the solution? Well, for things like the bulk solvent mask I'd
say some real-space "feathering" is called for before performing FFTs.
Same goes for masked density like that used to compute CCmask. It may
also be worth looking into the digitization process of "image first"
structural biology methods?
My question for the BB: can someone explain how Nyquist folding is
handled in cryoEM data processing?
-James Holton
MAD Scientist
########################################################################
To unsubscribe from the CCP4BB list, click the following link:
https://www.jiscmail.ac.uk/cgi-bin/WA-JISC.exe?SUBED1=CCP4BB&A=1
This message was issued to members of www.jiscmail.ac.uk/CCP4BB, a mailing list
hosted by www.jiscmail.ac.uk, terms & conditions are available at
https://www.jiscmail.ac.uk/policyandsecurity/
