On Friday 18 September 2009 12:47:20 Chavas Leo wrote:
> Dear all --
>
> I cannot remember exactly, but I thought we had a long discussion on
> the rightness of using compressed images, especially when considering
> the loss of information while doing so.
^^^^
> -- Leo --
>
> On 18 Sep 2009, at 23:50, Graeme Winter wrote:
>
> > Hi David,
> >
> > If the data compression is carefully chosen you are right: lossless
^^^^^^^^
> > jpeg2000 compression on diffraction images works very well, but is a
> > spot slow. The CBF compression using the byte offset method is a
> > little less good at compression put massively faster... as you point
> > out, this is the one used in the pilatus images.
Not all compression methods cause loss of information.
cheers,
Ethan
> > I recall that the
> > .pck format used for the MAR image plates had the same property - it
> > was quicker to read in a compressed image that the raw equivalent.
> >
> > So... once everyone is using the CBF standard for their images, with
> > native lossless compression, it'll save a fair amount in disk space
> > (=£/$), make life easier for people and - perhaps most importantly -
> > save a lot of data transfer time.
> >
> > Now the funny thing with this is that if we compress the images before
> > we store them, the compression implemented in the file system will be
> > less effective... oh well, can't win em all...
> >
> > Cheers,
> >
> > Graeme
> >
> >
> >
> > 2009/9/18 Waterman, David (DLSLtd,RAL,DIA)
> > <david.water...@diamond.ac.uk>:
> >> Just to comment on this, my friend in the computer game industry
> >> insists
> >> that compression begets speed in almost all data handling situations.
> >> This will be worth bearing in mind as we start to have more fine-
> >> sliced
> >> Pilatus 6M (or similar) datasets to deal with.
> >>
> >> Cheers,
> >> David.
> >>
> >> -----Original Message-----
> >> From: CCP4 bulletin board [mailto:ccp...@jiscmail.ac.uk] On Behalf Of
> >> William G. Scott
> >> Sent: 17 September 2009 22:48
> >> To: CCP4BB@JISCMAIL.AC.UK
> >> Subject: [ccp4bb] I compressed my images by ~ a factor of two, and
> >> they
> >> load and process in mosflm faster
> >>
> >> If you have OS X 10.6, this will impress your friends and save you
> >> some
> >> disk space:
> >>
> >> % du -h -d 1 mydata
> >> 3.5G mydata
> >>
> >> mv mydata mydata.1
> >>
> >> sudo ditto --hfsCompression mydata.1 mydata rm -rf mydata.1
> >>
> >> % du -h -d 1 mydata
> >> 1.8G mydata
> >>
> >> This does hfs filesystem compression, so the images are still
> >> recognized
> >> by mosflm, et al. I think they process a bit faster too, because
> >> half
> >> the information is packed into the resource fork.
> >> This e-mail and any attachments may contain confidential,
> >> copyright and or privileged material, and are for the use of the
> >> intended addressee only. If you are not the intended addressee or
> >> an authorised recipient of the addressee please notify us of
> >> receipt by returning the e-mail and do not use, copy, retain,
> >> distribute or disclose the information in or attached to the e-mail.
> >> Any opinions expressed within this e-mail are those of the
> >> individual and not necessarily of Diamond Light Source Ltd.
> >> Diamond Light Source Ltd. cannot guarantee that this e-mail or any
> >> attachments are free from viruses and we cannot accept liability
> >> for any damage which you may sustain as a result of software
> >> viruses which may be transmitted in or with the message.
> >> Diamond Light Source Limited (company no. 4375679). Registered in
> >> England and Wales with its registered office at Diamond House,
> >> Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11
> >> 0DE, United Kingdom
> >>
> >>
>
> ------------------------------------------------------------
> Chavas Leonard, Ph.D.
> Assistant Professor
> ------------------------------------------------------------
> Structural Biology Research Center
> Photon Factory
> High Energy Research Organization (KEK)
> 305-0801 Tsukuba Oho 1-1
> Japan
> ------------------------------------------------------------
> Tel: +81(0)29-864-5642 (4901)
> Fax: +81(0)29-864-2801
> e-mail: leonard.cha...@kek.jp
> ------------------------------------------------------------
> Science Advisory Board (BIT Life Sciences)
> Editorial Board (JAA)
> ------------------------------------------------------------
> http://pfweis.kek.jp/~leo
>
--
Ethan A Merritt
Biomolecular Structure Center
University of Washington, Seattle 98195-7742
