Hi Dan,
The original motivation behind read.dna() was to allow users to read
their DNA alignments stored in the Phylip formats -- support for the
Clustal format came later. You may be right that this is not so
frequent. I presume a commoner workflow is to first use read.GenBank,
then write the sequences for analyses with other software. In the
situation you describe, that'd imply using write.dna. In this function,
the rule is (say L is the length of the longest taxon name): if L < 11
then the 1st nucleotide is written at the 11th column in the output
file, otherwise at the (L+2)th column with a space at the (L+1)th one.
In the short term, I can change this in both read.dna and write.dna and
impose a space (or a tabulation) between the longest taxon name and the
first nucleotide. This would imply, of course, that taxa names cannot
have spaces. What do others think?
In the long term, I think we may discuss deprecating the sequential and
interleaved formats for the reasons I listed below. For instance,
Clustal can output its alignments in FASTA, Muscle outputs by default in
this format. This is an open discussion.
Best,
Emmanuel
Dan Rabosky wrote on 26/04/2012 22:01:
Hi Emmanuel-
Thanks for fixing the whitespace issue. I think this fix will be useful to many
users.
On the issue of recognizing 10 IUPAC characters: I think this is a real
problem, and may come up again in short order. Maybe it is just that use of
this function has been limited? In the single dataset with a modest number of
sequences that caused me problems yesterday, I had the following species and/or
genus names - all of which constitute 10 character strings drawn from the set
of IUPAC codes:
brachyurus (x 2)
savannarum
graduacauda
caudacutus
Camarhynchus (x 3)
madagascariensis
I don't suggest deprecating the phylip sequential, but rather, using something that is compatible
with raxml (surely one of the most widely used phylogenetics programs today). I think raxml uses a
relaxed sequential version of the phylip format with whitespace delimitation. I could read the same
alignment in raxml with no problems, but I had multiple issues when reading the same file with
read.dna (including the whitespace character on the first line). My guess is that very few people
are using the original phylip format, with its limit of 10 characters per taxon name, and with dna
seqs beginning immediately after this. So maybe deprecate "sequential phylip", but you
could use what Stamatakis calls "relaxed sequential PHYLIP", which appears to be: (1)
taxon names cannot include spaces but can be up to 100 characters; and (2) names separated from
sequences by whitespace character (ideally, this should recognize any number of spaces or tabs to
prevent user confusion).
For users with tab-delimited raxml files (eg each taxon name separated from its
dna sequence by a tab), you can use a regular-expressions enabled text editor
(like textwrangler) to quickly find potential problems. Just search for
[ACGTUMRWSYKVHDBN]{10}.+\t
with grep matching enabled.
Cheers,
~Dan
On Apr 26, 2012, at 2:16 AM, Emmanuel Paradis wrote:
Hi Dan,
The reason for this implementation (searching the first 10 IUPAC-coded bases)
is because the exact formatting is not inconsistent among different programs.
Some files have:
0123456789acgt.....
that is a 10-character name and the sequence starting on the 11th position. I
think this is typical for Phylip. Other software (e.g., PhyML) accepts longer
taxa names and require a space before the start of the sequence.
About your example: it depends on the order of the data. The following file can
be read:
2 10
xxxxx AAAAAAAAAA
madagascarAAAAAAAAAA
But if you invert the two sequence lines, it fails.
It is the first time I hear about this problem in 9 years, maybe because it
requires a particular combination of circumstances. Another drawback of this
implementation is that files with less than 10 bases cannot be read.
How to solve this? If it were left only to me, I would deprecate the
interleaved and sequential formats. FASTA is more flexible, more widespread,
easier to parse, can store exactly the same information, and labels are only
constrained to be on a single line (but can contain any characters including
\n, \t, ...) But I guess many programs use the Phylip formats, so I'd be glad
to read other suggestions.
As for your 2nd problem, it is now fixed in ape.
Best,
Emmanuel
-----Original Message-----
From: Dan Rabosky<[email protected]>
Sender: [email protected]
Date: Wed, 25 Apr 2012 17:51:35
To:<[email protected]>
Subject: [R-sig-phylo] read.dna warnings and pitfalls
Hi All-
I have spent an inordinate and embarrassing amount of time tracking down an
excruciatingly cryptic issue with read.dna, which I rarely use. Here are two
key problems:
1) The function automatically assumes it is reading DNA sequences when it encounters a string of 10
continuous "DNA-like" characters. This includes all characters in the set
(ACGTUMRWSYKVHDBN-). This function, unlike the phylip original, does not have limits on taxon name
lengths. Hence, I had - in the middle of a large alignment - a species whose name included the
string "MADAGASCAR", which caused a failure. To be fair, the documentation warns of
this, but I think this is extremely easy to overlook, and - moreover - it seems unfortunate to have
to parse all your taxon names for a potential IUPAC match before trying to use the function.
Presumably, most users who specify sequential spacing will be using whitespace to separate taxon
names from DNA sequences, and perhaps it is better to exploit this rather than IUPAC matching.
2) The function is whitespace-sensitive. if you tab-separate the numbers on the first
line (numbers of taxa, numbers of sites), you'll receive an errror with the message:
"the first line of the file must contain the dimensions of the data". It
appears that spaces are OK, however.
Hopefully this post will be useful to somewhere in the future with a similar
issue. Perhaps these can be addressed in a future update to ape?
-Dan Rabosky
_______________________________________________
R-sig-phylo mailing list
[email protected]
https://stat.ethz.ch/mailman/listinfo/r-sig-phylo
_______________________________________________
R-sig-phylo mailing list
[email protected]
https://stat.ethz.ch/mailman/listinfo/r-sig-phylo
--
Emmanuel Paradis
IRD, Jakarta, Indonesia
http://ape.mpl.ird.fr/
_______________________________________________
R-sig-phylo mailing list
[email protected]
https://stat.ethz.ch/mailman/listinfo/r-sig-phylo
