dear all,
since standard data formats were mentioned, I would just like to draw your
attention to the NeXus project, which has the (ambitious) purpose of
developing a unified data format for x-rays and neutrons.
http://www.neutron.anl.gov/NEXUS/
http://lns00.psi.ch/NeXus/
The NeXus format is based on HDF, which is an industrial standard. I'm not
directly involved, but I know that they specifically looked at CIF as an
alternative. They found that, in it's native format, CIF it is not suitable
to handle large amounts of raw data. There is, however, something called
imgCIF, which might address the problem. I quote from the NeXus web site:
"In a round table discussion at the [NOBUGS] workshop [ILL, JAN96], we
discussed the merits of the NeXus proposal compared to a proposal to extend
CIF by including binary images (imgCIF). Those involved in the NeXus
proposal did not think that imgCIF would have sufficient flexibility to
describe complex neutron instrumentation. Those involved in the imgCIF
proposal preferred the CIF use of ASCII header information and were
concerned about HDF's ability to handle the high data rates at synchrotron
sources. We have decided not to try and merge the two proposals at this
stage, particularly since, in our view, they are serving different
purposes."
If you wonder why powder diffractionists should be concerned with large data
files, please note that the new ISIS high-flux powder diffractometer GEM, of
which I am responsible, produces about 64 Mbytes of integers (14 Mbytes when
compressed) for every data set (this will increase to ~150 Mbytes when the
instrument is complete), and we can get good-quality data in as little as 10
seconds. This is the brave new world, gentlemen....
Paolo
