Re: [ccp4bb] Long term storage for raw images/ crystallographic data sets

[Herbert J. Bernstein]

Dear Colleagues,

  May I suggest that those who are at Universities take a look at the
G-suite for Education

https://edu.google.com/products/gsuite-for-education/editions/?modal_active=none

which provides unlimited cloud storage for free to educational institutions.

  Regards,
Herbert

On Thu, Nov 29, 2018 at 3:54 PM Lieberman, Raquel L <
raquel.lieber...@chemistry.gatech.edu> wrote:

> Dear All,
>
> How do your labs handle long-term raw data backups? My lab is maxing out
> our 6TB RAID backup (with two off-site mirrors) so I am investigating our
> next long term solution. The vast majority of the data sets are published
> structures (i.e. processed data deposited in PDB) or redundant/unusable so
> immediate access is not anticipated, but the size of data sets is
> increasing quickly with time, so I am looking for a scalable-yet-affordable
> solution.
>
> Would be grateful for input into various options, e.g. bigger HD/RAIDs,
> cloud backup, tape, anything else.
>
> I will compile.
>
> Thank you,
>
> Raquel
> --
> Raquel L. Lieberman, Ph.D.
> Professor
> School of Chemistry and Biochemistry
> Georgia Institute of Technology
>
>
>
> --
>
> To unsubscribe from the CCP4BB list, click the following link:
> https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCP4BB&A=1
>



To unsubscribe from the CCP4BB list, click the following link:
https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCP4BB&A=1

[ccp4bb] CHAPS provider

[Sebastiano Pasqualato]

Dear all,
we are considering whether to buy the columns and reagents to purify our own 
Wnt3 protein or to buy it.
Given that the purification requires large amount of column eluents with 1% 
CHAPS, we are looking for the cheapest CHAPS provider to understand if the 
in-house purification is worth the cost.
Could anybody share their experience with low-cost chemical providers, please?
Thank you very much,
have a nice weekend,
Sebastiano


-- 
Sebastiano Pasqualato, PhD
Biochemistry and Structural Biology Unit
Department of Experimental Oncology
European Institute of Oncology
IFOM-IEO Campus
via Adamello, 16
20139 - Milano
Italy

tel +39 02 9437 5167
fax +39 02 9437 5990





To unsubscribe from the CCP4BB list, click the following link:
https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCP4BB&A=1

Re: [ccp4bb] CHAPS provider

[Tomas Malinauskas]

Hi Sebastiano,
we often buy chemicals (including CHAPS) from Melford:
https://www.melford.co.uk/products/biochemicals/detergents-surfactants/chaps-1-g.html
It is often cheaper than Sigma-Aldrich.
Hope that helps,
Tomas
On Fri, Nov 30, 2018 at 2:42 PM Sebastiano Pasqualato
 wrote:
>
>
> Dear all,
> we are considering whether to buy the columns and reagents to purify our own 
> Wnt3 protein or to buy it.
> Given that the purification requires large amount of column eluents with 1% 
> CHAPS, we are looking for the cheapest CHAPS provider to understand if the 
> in-house purification is worth the cost.
> Could anybody share their experience with low-cost chemical providers, please?
> Thank you very much,
> have a nice weekend,
> Sebastiano
>
>
> --
> Sebastiano Pasqualato, PhD
> Biochemistry and Structural Biology Unit
> Department of Experimental Oncology
> European Institute of Oncology
> IFOM-IEO Campus
> via Adamello, 16
> 20139 - Milano
> Italy
>
> tel +39 02 9437 5167
> fax +39 02 9437 5990
>
>
>
> 
>
> To unsubscribe from the CCP4BB list, click the following link:
> https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCP4BB&A=1



To unsubscribe from the CCP4BB list, click the following link:
https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCP4BB&A=1

Re: [ccp4bb] buying a cluster

[James Holton]

I have a dissenting opinion about computers "moving on a bit".  At least 
when it comes to most crystallography software.


Back in the late 20th century I defined some benchmarks for common 
crystallographic programs with the aim of deciding which hardware to 
buy.  By about 2003 the champion of my refmac benchmark 
(https://bl831.als.lbl.gov/~jamesh/benchmarks/index.html#refmac) was the 
new (at the time) AMD "Opteron" at 1.4 GHz.  That ran in 74 seconds.


Last year, I bought a rather expensive 4-socket Intel Xeon E7-8870 v3 
(turbos to 3.0 GHz), which is the current champion of my XDS benchmark.  
The same old refmac benchmark on this new machine, however, runs in 68.6 
seconds.  Only a smidge faster than that old Opteron (which I threw away 
years ago).


The Xeon X5550 in consideration here takes 74.1 seconds to run this same 
refmac benchmark, so price/performance wise I'd say that's not such a 
bad deal.


The fastest time I have for refmac to date is 41.4 seconds on a Xeon 
W-2155, but if you scale by GHz you can see this is mostly due to its 
fast clock speed (turbo to 4.5 GHz). With a few notable exceptions like 
XDS, HKL2k and shelx, which are multi-processing and optimized to take 
advantage of the latest processor features using intel compilers, most 
crystallographic software is either written in Python or compiled with 
gcc.  In both these cases you end up with performance pretty much 
scaling with GHz.  And GHz is heat.


Admittedly, the correlation is not perfect, and software has changed a 
wee bit over the years, so comparisons across the decades are not 
exactly fair, but the lesson I have learned from all my benchmarking is 
that single-core raw performance has not changed much in the last ~10 
years or so.  Almost all the speed increase we have seen has come from 
parallelization.


And one should not be too quick to dismiss clusters in favor of a single 
box with a high core count. The latter can be held back by memory 
contention and other hard-to-diagnose problems.  Even with parallel 
execution many crystallography programs don't get any faster beyond 
using about 8-10 cores.  Don't let 100% utilization fool you!  Use a 
timer and you'll see.  I'm not really sure why that is, but it is the 
reason that same Xeon W-2155 that leads my refmac benchmark is also my 
champion system for running DIALS and phenix.refine.


My two cents,

-James Holton
MAD Scientist


On 11/26/2018 1:10 AM, V F wrote:

Dear all,
Thanks for all the off/list replies.


To be honest, how much are they paying you to take it? Can you sell it for
scrap?

May be I will give it a pass.


To compare, two dual CPU servers with Skylake Gold 6148 - that is 40 cores -
will probably beat the whole lot even if you could keep the cluster going.
And keeping clusters busy is a time consuming challenge... I know!
If they are 250W servers, then you are looking at £8000 per year to power
and cool it. The two modern servers will be more like £1500 per year to run.
And the servers will only cost about £6000... the economics and planet don't
stack up!

By servers do you mean tower/standalone?

Thanks for the detailed explanation. From 2012, we already have many
dell precision T5600 with 2 x Xeon E5-2643 (8 Cores) (16 threads) and
I was hoping parallellisation with clusters maybe of some help. Looks
not.

These are running so well (takes about 45 min for a typical dataset
reduction with DIALS) I am not sure buying new ones is useful.



To unsubscribe from the CCP4BB list, click the following link:
https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCP4BB&A=1




To unsubscribe from the CCP4BB list, click the following link:
https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCP4BB&A=1

Re: [ccp4bb] Long term storage for raw images/ crystallographic data sets

[James Holton]

The answer depends a lot on what you mean by "long-term storage". Do you 
want the data to be available all the time on a mountable volume?  Or is 
putting it away on a shelf OK?  Do you want the storage to be as 
bulletproof and worry-free as possible?  Or are you OK with the fate of 
your data being somewhat nebulous, like in "the cloud"?  The price 
points for all these things are very different.


You can now buy a single 8 TB drive for $230 USD.  LTO6 tapes are 
currently at ~7 USD/TB.  Both of these are the current lowest price/TB 
for disk and tape.  Using the media, of course, generally requires 
attaching it to a server that costs ~$5k-$10k USD.  Amazon Glacier is 
free for uploads and essentially free for downloading it back as long as 
you don't want more than 1 GB per month.  The other extreme is a NetApp, 
where you just want a turnkey system that keeps your data as safe as 
possible, but is also really fast.


What do I do?  I am currently deploying a RAID6 array of 8 TB drives for 
high-performance storage.  For archiving I used to use DVD-R, but that 
can't keep up with a Pilatus, so now I'm on LTO6 tapes for off-line 
backups.  I know tapes are famous as "write-only media", but so far over 
the last 10 years I haven't had any real trouble reading back an old LTO 
tape.  


-James Holton
MAD Scientist


On 11/29/2018 12:54 PM, Lieberman, Raquel L wrote:

Dear All,

How do your labs handle long-term raw data backups? My lab is maxing 
out our 6TB RAID backup (with two off-site mirrors) so I am 
investigating our next long term solution. The vast majority of the 
data sets are published structures (i.e. processed data deposited in 
PDB) or redundant/unusable so immediate access is not anticipated, but 
the size of data sets is increasing quickly with time, so I am looking 
for a scalable-yet-affordable solution.


Would be grateful for input into various options, e.g. bigger 
HD/RAIDs, cloud backup, tape, anything else.


I will compile.

Thank you,

Raquel
--
Raquel L. Lieberman, Ph.D.
Professor
School of Chemistry and Biochemistry
Georgia Institute of Technology





To unsubscribe from the CCP4BB list, click the following link:
https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCP4BB&A=1






To unsubscribe from the CCP4BB list, click the following link:
https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCP4BB&A=1

Re: [ccp4bb] CHAPS provider

[mesters]

Hi,

trick is to figure out the chemical company that really synthesizes the 
product as many parties buy the stuff from one supplier only and repack 
it into smaller bottles and reprice it (at considerably higher costs). 
In the Sigma catalogue for example, these products were once marked with 
the shor-hand "pfs" (prepared for Sigma).


Here is a link to a company that will synthesize it for a nice price: 
https://www.applichem.com/shop/produktdetail/as/chaps-ibiochemicai/


Negociate with them and you will probable get it cheaper 
Alternative, synthezise it yourself (see publications on applichem page)


Good luck,

Jeroen

Am 30.11.18 um 15:47 schrieb Tomas Malinauskas:

Hi Sebastiano,
we often buy chemicals (including CHAPS) from Melford:
https://www.melford.co.uk/products/biochemicals/detergents-surfactants/chaps-1-g.html
It is often cheaper than Sigma-Aldrich.
Hope that helps,
Tomas
On Fri, Nov 30, 2018 at 2:42 PM Sebastiano Pasqualato
 wrote:


Dear all,
we are considering whether to buy the columns and reagents to purify our own 
Wnt3 protein or to buy it.
Given that the purification requires large amount of column eluents with 1% 
CHAPS, we are looking for the cheapest CHAPS provider to understand if the 
in-house purification is worth the cost.
Could anybody share their experience with low-cost chemical providers, please?
Thank you very much,
have a nice weekend,
Sebastiano


--
Sebastiano Pasqualato, PhD
Biochemistry and Structural Biology Unit
Department of Experimental Oncology
European Institute of Oncology
IFOM-IEO Campus
via Adamello, 16
20139 - Milano
Italy

tel +39 02 9437 5167
fax +39 02 9437 5990





To unsubscribe from the CCP4BB list, click the following link:
https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCP4BB&A=1



To unsubscribe from the CCP4BB list, click the following link:
https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCP4BB&A=1



--
Dr.math. et dis. nat. Jeroen R. Mesters
Deputy, Senior Researcher & Lecturer
Program Coordinator /Infection Biology/ 



Institute of Biochemistry, University of Lübeck
Ratzeburger Allee 160, 23538 Lübeck, Germany
phone: +49-451-31013105 (secretariate -31013101)
fax: +49-451-31013104



To unsubscribe from the CCP4BB list, click the following link:
https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCP4BB&A=1