Dear Gerard,
An ideal we have been working towards within DIALS/xia2 and also with Herbert
Bernstein, the NeXuS people & Dectris (and many others) is to (i) define and
use the full HDF5 data formats on MX beamlines and (ii) develop the data
processing software to natively support these detectors. This work is ongoing,
however is hampered as you indicate below by the profusion of different data
formats which exist for the HDF5 container itself. One issue which has become
apparent however is *in addition* to this population of HDF5 formats there is a
further collection of nearly-mini-cbf files being created, some of which are
presumably coming from your software, some from the software Harry mentioned
and I would guess a further population from locally developed tools.
It is clear that this way madness lies, and we are headed in that direction
with great enthusiasm!
Our original motivation in asking for these data was to allow us to provide
support in DIALS and xia2 for these files where possible, including the native
HDF5. A secondary benefit of this, beyond the software itself being available
to the user community (Google will give anyone interested the necessary
pointers) is that the code itself which interprets the data is available, under
a free open source license, allowing (i) peer review of the code itself to
ensure the understanding is correct (ii) the opportunity for such peer to
actually contribute to getting fixes in place and {most importantly} (iii) an
implicit definition of the format for future generations of developers in the
absence of any kind of explicit format definitions.
I for one welcome multiple options for all data processing challenges, as this
makes for a vibrant ecosystem and ensures that the end user community are most
likely to get the best from their data. This has been a particular strength of
MX over the past decades. I therefore feel that “you can process these data
with (fill in any one package here)” is unsatisfactory, even if it may fulfil
an immediate need. I feel this all the more strongly when the software in
question is not universally available.
Best wishes Graeme
> On 9 Mar 2016, at 16:52, Gerard Bricogne <g...@globalphasing.com> wrote:
>
> Dear Graeme and Eiger users,
>
> We have been grasping this nettle since last Summer, and it is a
> particularly stinging one. At least, crystallographers who have a need
> to process Eiger datasets right now can do this with autoPROC with a
> fair chance of success. The remaining problems that are turning up at
> the moment have to do not just with incomplete information in locally
> produced miniCBF files but with inconsistencies between information of
> differing provenance about the same items.
>
> We know of a number of beamlines and users that make use of the
> HDF5 converter now included in autoPROC for the task of generating
> miniCBF files. If you install the latest (snapshot) version of
> autoPROC by going to
>
> http://www.globalphasing.com/autoproc/
>
> you will have access not only to the native Eiger/HDF5 support in
> autoPROC, but also to the latest version of our converter utility. See
>
> http://www.globalphasing.com/autoproc/ReleaseNotes/ReleaseNotes-autoPROC_snapshot_20160304.txt
> http://www.globalphasing.com/autoproc/wiki/index.cgi?DataProcessingHdf5
>
> We hope that the resulting CBF files (with full mini-cbf header) would
> be fully compatible with other processing packages, viewers or tools -
> if they can't read HDF5 datasets directly.
>
> Please let us know if there are inconsistencies or issues. This
> invitation extends to all users of this HDF5 capability in autoPROC.
>
>
> With best wishes,
>
> Clemens, Claus and Gerard.
>
> --
> On Tue, Mar 08, 2016 at 07:57:57PM +0000, Graeme Winter wrote:
>> Hello World,
>>
>> A few times over the last week we at the xia2 helpdesk have been asked about
>> support for Eiger detectors, and particularly local dialects of CBF file
>> which are created on Eiger powered beamlines. Often we are able to get a
>> single image which is helpful but from there it is hard to prove everything
>> is working right.
>>
>> Please could beamline scientists who have an Eiger and use local software to
>> make this into miniCBF files get in touch, ideally with an example data set,
>> so we can test the support in xia2 and DIALS and make corrections where
>> necessary.
>>
>> Thanks & best wishes Graeme
>> --
>> This e-mail and any attachments may contain confidential, copyright and or
>> privileged material, and are for the use of the intended addressee only. If
>> you are not the intended addressee or an authorised recipient of the
>> addressee please notify us of receipt by returning the e-mail and do not
>> use, copy, retain, distribute or disclose the information in or attached to
>> the e-mail.
>> Any opinions expressed within this e-mail are those of the individual and
>> not necessarily of Diamond Light Source Ltd.
>> Diamond Light Source Ltd. cannot guarantee that this e-mail or any
>> attachments are free from viruses and we cannot accept liability for any
>> damage which you may sustain as a result of software viruses which may be
>> transmitted in or with the message.
>> Diamond Light Source Limited (company no. 4375679). Registered in England
>> and Wales with its registered office at Diamond House, Harwell Science and
>> Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom
