Hi Clemens,
Thank you for your help. Unfortunately, although I definitely learned a few things I have additional questions... >> What I initially did not know is that REFMAC and STARANISO do not go >> well together when it comes to the Rfree-flags in the diffraction >> cut-off regions. >Just to clarify: REFMAC should have no problems with STARANISO >data. What you probably mean is that the unobservable reflections >outside of the cut-off surface - which might be considered via an >ellipsoidal aproximation - could be used for the DFc completion in >REFMAC? Yes this is what I meant, I just put it in very loose words =) but for future reference, this is a good step to take, right? I will hopefully have a few more crystals with the same issue and I would also now stick to this file. >Take the Data_1_autoPROC_STARANISO_all.cif and combine it [1] with the >mmCIF from your refinement step: ensure that the data block with the >reflection data that went /into/ your refinement is given as the first >block. By convention, this is what e.g. PDB deposition/validation >software will use for re-computing various statistics. >I would be inclined to go back to your original REFMAC refinement >(before you did all that SFTOOLS, Phenix-again, server, conversion >steps): do you have just an output MTZ file and no reflection mmCIF? >Maybe by re-running the same job you could switch on some option to >trigger writing of the reflection data in mmCIF format? Maybe even >switching off the DFc completion and/or re-scaling? PHENIX has this option to export the data as an mmcif but I cannot find this in CPP4i2 (which is where I use REFMAC). There is only the export and validation tool, but this is crashing because it expects processing data which is not there and which i cannot import either (I cannot select anything for Aimless XML) . And here is where my fluffy understanding of the mtz file format hits me over the head =) as long as an output is generated automatically there is the security that the right data is in the right place. What I tried was to export the "old-style".mtz from CCP4i2 and then to convert it using the gemmi server. But gemmi requires intensities, which are missing in the REFMAC mtz, which contains these arrays: H K L FREER F SIGF FC PHIC FC_ALL PHIC_ALL FWT PHWT DELFWT PHDELWT FOM PHCOMB HLACOMB HLBCOMB HLCCOMB HLDCOMB FC_ALL_LS PHIC_ALL_LS. Could it be that a large part of my problem is deived from the mini-mtz output files that are generated by CPP4i2? I think this is where I am stuck, I do not know how to get the output mini-mtz together with the intensity data in one mtz file, or where to convert this mtz to a cif (and also not how to combine this with the cif file from autoproc, but this is a future problem) cheers and thank you, Vera and on a less pressing side... >> I then went to Phenix for molecular replacement and refined a few >> rounds with Phenix. >Ok. a case of a go-to-the-well-known, as long as it works. >> However one of my co-factors (a twice covalently linked heme) >> did not refine properly, which is why i switched the refinement to CCP4 >> and used REFMAC. >It might be interesting to find out what the differences in restraint >dictionaries is here (this seems the most likely reason for such >misbehaving ... and all refinement programs should handle heme >residues correctly I guess). Yes I think there is a problem with the heme restraints somewhere and I would be very grateful for any help in this regard, since I have more structures with a covalent heme linkage. COOT does not deal very well with the linkages and I had no way of generating the link record with AceDRG as it cannot yet deal with the heme iron. So I wrote a link dictionary for the two links (which i hope works). It seemed that both Buster and REFMAC can deal with the heme linkages, but PHENIX gradually shifts the "linked" atoms more and more apart with each refinement cycle, which is also what happens in COOT. >If you created a totally new test-set, you could indeed try and unbias >the Rfree computations. On the other hand, since you have already >validated your model parametrisation before (with the old test-set) >there should not be any need to do everything from scratch again >... however, since the resulting R/Rfree will not be decoupled you >would have trouble reporting/depositing such a result as well ;-) Yeah, i realize that now as well =) also this is exactly what happened. cheers again >>> Clemens Vonrhein <vonrh...@globalphasing.com> 07/11/22 7:08 PM >>> Hi Vera, On Mon, Jul 11, 2022 at 04:51:32PM +0200, Vera Pfanzagl wrote: > I am trying to get a structure with a bit of a history in terms of > programs I have used into a deposit-able form. Yeah ... real-life never follows the nicely linear steps of the methods section ;-) > However I have massif problems with the mtz labels, specifically the > Rfree flags and cannot figure out where my problem is derived from. It might just be a misunderstanding of the data as presented to the deposition system/software? > Specifically, I have collected diffraction data on a very anistropic > crystal (3 A in one direction and 1.4 in the other) which I > processed using AUTOPROC because I wanted to make use of the inbuild > STARANISO processing. Makes sense. > I then went to Phenix for molecular replacement and refined a few > rounds with Phenix. Ok. > However one of my co-factors (a twice covalently linked heme) > did not refine properly, which is why i switched the refinement to CCP4 > and used REFMAC. It might be interesting to find out what the differences in restraint dictionaries is here (this seems the most likely reason for such misbehaving ... and all refinement programs should handle heme residues correctly I guess). > What I initially did not know is that REFMAC and STARANISO do not go > well together when it comes to the Rfree-flags in the diffraction > cut-off regions. Just to clarify: REFMAC should have no problems with STARANISO data. What you probably mean is that the unobservable reflections outside of the cut-off surface - which might be considered via an ellipsoidal aproximation - could be used for the DFc completion in REFMAC? In that case all reflections with no observation right to your diffraction limit (1.4A) will have model structure factors DFc instead of 2mFo-DFc in the FWT/PHWT columsn of the MTZ file. Given the high level of anisotropy in your data, this will mean a very large number of purely model-based information making it into your map computation - which could bias the resulting density maps towards your model. See also e.g. https://www.globalphasing.com/buster/manual/autobuster/manual/autoBUSTER5.html#normal_out > I went back and tried to use SFtools by CCP4 as described on > https://staraniso.globalphasing.org/test_set_flags_about.html to > generate a staraniso_mtz suitable for refinement in REFMAC, Those steps intend to create a MTZ file that would behave similar to e.g. a BUSTER refinement which performs DFc completion on the unobserved relfections (those inside the anisotropic cut-off surface, i.e. those that /could/ be observed - as opposed to those outside the cut-off surface which could never be observed). > phased it again with my initial model and ran a refinement cycle, > because I thought skipping a program might be a good idea. If you created a totally new test-set, you could indeed try and unbias the Rfree computations. On the other hand, since you have already validated your model parametrisation before (with the old test-set) there should not be any need to do everything from scratch again ... however, since the resulting R/Rfree will not be decoupled you would have trouble reporting/depositing such a result as well ;-) > However I do not know if the mtz file I get has the appropriate > column labels and also I do not know how to convert it to an mmcif > that includes the data from processing. Take the Data_1_autoPROC_STARANISO_all.cif and combine it [1] with the mmCIF from your refinement step: ensure that the data block with the reflection data that went /into/ your refinement is given as the first block. By convention, this is what e.g. PDB deposition/validation software will use for re-computing various statistics. > i have never before tried to combine AUTOPROC-processed data with > REFMAC data and am not very familiar with REFMAC to begin with. I > have tried the sftool server to validate the mtz from REFMAC > processing but I get an error message > > Converting SF file to mmcif format ... > Warning: (data block=1): all status (80493) are assigned as 'x' (changed! > 'x'->'o') > Error: File (xxxx) has no mandatory items 'F/I/F+/F-/I+/I-' (data block=1) Try to avoid any conversion software/server if you already have deposition-ready mmCIF files from data-processing and refinement. These programs know usually best how to create those mmCIF files and converters/servers can be slightly behind the latest specifications. > I > tried to use Phenix (as it automatically converts the mtz to an mmcif > file) by running a "mock"-refinement with fixed coordinates but the > mmcif file I get out of this seems to contain some wrongly labeled data > fields in block 1 when I try to upload it to PDBs validation server. > > Essentially it seems to contain some 30000 data points in block 1 that have a > ? This is the error message I receive: > > > Warning: > (data block=1): 30301 reflections have status assigned as '?' (changed > to 'x') Warning: (data block=1): 30301 reflections have wrong status. > Not in list(o,f,x,-,<,h,l) Warning: No wavelength value was found in > SF file (data block=1). > > which is essentially the same as above, so I guess it is derived from the > same problem. I would be inclined to go back to your original REFMAC refinement (before you did all that SFTOOLS, Phenix-again, server, conversion steps): do you have just an output MTZ file and no reflection mmCIF? Maybe by re-running the same job you could switch on some option to trigger writing of the reflection data in mmCIF format? Maybe even switching off the DFc completion and/or re-scaling? > I would be very grateful for any help. Not sure it does .... Cheers Clemens [1] In it's most simple form, such a combination consists of just concatenating several mmCIF files together (as long as their data blocks have different identifiers). However, if additional decisions have been made between data processing and actual refinement (like enantiomorph, space groups, alternative indexing etc), some checks might be necessary. -- *-------------------------------------------------------------- * Clemens Vonrhein, Ph.D. vonrhein AT GlobalPhasing DOT com * Global Phasing Ltd., Sheraton House, Castle Park * Cambridge CB3 0AX, UK www.globalphasing.com *-------------------------------------------------------------- ######################################################################## To unsubscribe from the CCP4BB list, click the following link: https://www.jiscmail.ac.uk/cgi-bin/WA-JISC.exe?SUBED1=CCP4BB&A=1 This message was issued to members of www.jiscmail.ac.uk/CCP4BB, a mailing list hosted by www.jiscmail.ac.uk, terms & conditions are available at https://www.jiscmail.ac.uk/policyandsecurity/ ######################################################################## To unsubscribe from the CCP4BB list, click the following link: https://www.jiscmail.ac.uk/cgi-bin/WA-JISC.exe?SUBED1=CCP4BB&A=1 This message was issued to members of www.jiscmail.ac.uk/CCP4BB, a mailing list hosted by www.jiscmail.ac.uk, terms & conditions are available at https://www.jiscmail.ac.uk/policyandsecurity/
