On Monday, June 03, 2013 01:04:28 Jack Applegame wrote: > On Sunday, 2 June 2013 at 20:50:31 UTC, Jonathan M Davis wrote: > > http://stackoverflow.com/questions/12288465 > > Lets have string of chars, and it contains UTF-8 string. > Does front(str[]) automatically convert first unicode character > to UTF-32 and returns it? > I made a test case and answer is: "Yes, it does!" > May be this make sense. But such implicit conversion confuses > everyone whom I asked. > Therefore, string is not ordinary array (in Phobos context), but > special array with special processing rules. > > I'm moving from C++ and often ask myself: "why D has so much > hidden confusing things?"
The language treats strings as arrays of code units. The standard library treats them as ranges of code points. Yes, this can be confusing, but we need both. In order to operate on strings efficiently, they need to be made up of code units, but correctness requires code points. This means that the complexity is to a great extent an intrinsic part of dealing with strings properly. In C++, people usually just screw it up and treat char as if it were a character when in fact it's not. It's a piece of one. Whether we went about handling the complexity of code units vs code points in the best manner is debatable, but it can't be made simple if you want both efficiency and correctness. A better approach might have been to have a string type which operated on code points and held the code units internally so that everything operated on code points by default, but the library stuff was added later, and Walter Bright tends to think that everyone should understand Unicode well, so the decisions he makes with regards to that aren't always the best (since most people don't understand Unicode well and don't want to care). What we have actually works quite well, but it does require that you come to at least a basic understanding of the difference between code units and code points. - Jonathan M Davis
