Re: [Python-Dev] Divorcing str and unicode (no more implicitconversions).
Am 25.10.2005 um 23:40 schrieb Josiah Carlson: > [...] > Identically drawn glyphs are a problem, and pretending that they > aren't > a problem, doesn't make it so. Right now, all possible name glyphs > are > visually distinct, which would not be the case if any unicode > character > could be used as a name (except for numerals). Speaking of which, > would > we then be offering support for arabic/indic numeric literals, and/or > support it in int()/float()? It's already supported in int() and float() >>> int(u"\u136c\u2082") 42 >>> float(u"\u0664\u09e8") 42.0 But not as literals: # -*- coding: unicode-escape -*- print \u136c\u2082 This gives (on the Mac): File "encoding.py", line 3 print ፬₂ ^ SyntaxError: invalid syntax > [...] Bye, Walter Dörwald ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Divorcing str and unicode (no more implicitconversions).
Martin v. Löwis wrote: > M.-A. Lemburg wrote: > >>A few years ago we had a discussion about this on python-dev >>and agreed to stick with ASCII identifiers for Python. I still >>think that's the right way to go. > > I don't think there ever was such an agreement. You even argued against having non-ASCII identifiers: http://mail.python.org/pipermail/python-list/2002-May/102936.html and I agree with you on most of the points you make in that posting: * Unicode identifiers are going to introduce massive code breakage - just think of all the tools people use to manipulate Python code today; I'm quite sure that most of it will fail in one way or another if you present it Unicode literals such as in "zähler += 1". * People don't seem very interested in using Unicode identifiers, e.g. http://mail.python.org/pipermail/i18n-sig/2001-February/000828.html most of the few who did comment, said they'd rather have ASCII identifiers, e.g. http://mail.python.org/pipermail/python-list/2002-May/104050.html Do you really think that it will help with code readability if programmers are allowed to use native scripts for their identifiers ? I think this goes beyond just visual aspects of being able to distinguish graphemes: If you are told to debug a program written by say a Japanese programmer using Japanese identifiers you are going to have a really hard time. Integrating such code into other applications will be even harder, since you'd be forced to use his Japanese class names in your application. This doesn't only introduce problems with being able to enter the Japanese identifiers, it will also cause your application to suddenly contain identifiers in Japanese even though that's not your native script. I think source code encodings provide an ideal way to have comments written in native scripts - and people use that a lot. However, keeping the program code itself in plain ASCII makes it far more readable and reusable across locales. Something that's important in this globalized world. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Oct 26 2005) >>> Python/Zope Consulting and Support ...http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ...http://python.egenix.com/ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 351, the freeze protocol
Paolino wrote: > Is __hash__=id inside a class enough to use a set (sets.Set before 2.5) > derived class instance as a key to a mapping? It is __hash__=lambda self:id(self) that is terribly slow ,it needs a faster way to state that to let them be useful as key to mapping as all set operations will pipe into the mechanism .In my application that function is eating time like hell, and will keep on doing it even with the PEP proposed .OT probably. Regards Paolino ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Divorcing str and unicode (no more implicit conversions).
At 11:43 2005-10-24 +0200, M.-A. Lemburg wrote:
>Bengt Richter wrote:
>> Please bear with me for a few paragraphs ;-)
>
>Please note that source code encoding doesn't really have
>anything to do with the way the interpreter executes the
>program - it's merely a way to tell the parser how to
>convert string literals (currently on the Unicode ones)
>into constant Unicode objects within the program text.
>It's also a nice way to let other people know what kind of
>encoding you used to write your comments ;-)
>
>Nothing more.
I think somehow I didn't make things clear, sorry ;-)
As I tried to show in the example of module_a.cs vs module_b.cs,
the source encoding currently results in two different str-type
strings representing the source _character_ sequence, which is the
_same_ in both cases. To make it more clear, try the following little
program (untested except on NT4 with
Python 2.4b1 (#56, Nov 3 2004, 01:47:27)
[GCC 3.2.3 (mingw special 20030504-1)] on win32 ;-):
< t_srcenc.py >
import os
def test():
open('module_a.py','wb').write(
"# -*- coding: latin-1 -*-" + os.linesep +
"cs = '\xfcber-cool'" + os.linesep)
open('module_b.py','wb').write(
"# -*- coding: utf-8 -*-" + os.linesep +
"cs = '\xc3\xbcber-cool'" + os.linesep)
# show that we have two modules differing only in encoding:
print ''.join(line.decode('latin-1') for line in open('module_a.py'))
print ''.join(line.decode('utf-8') for line in open('module_b.py'))
# see how results are affected:
import module_a, module_b
print module_a.cs + ' =?= ' + module_b.cs
print module_a.cs.decode('latin-1') + ' =?= ' + module_b.cs.decode('utf-8')
if __name__ == '__main__':
test()
---
The result copied from NT4 console to clipboard and pasted into eudora:
__
[17:39] C:\pywk\python-dev>py24 t_srcenc.py
# -*- coding: latin-1 -*-
cs = 'über-cool'
# -*- coding: utf-8 -*-
cs = 'über-cool'
nber-cool =?= ++ber-cool
über-cool =?= über-cool
__
(I'd say NT did the best it could, rendering the the copied cp437
superscript n as the 'n' above, and the '++' coming from the
cp437 box characters corresponding to the '\xc3\xbc'. Not sure
how it will show on your screen, but try the program to see ;-)
>Once a module is compiled, there's no distinction between
>a module using the latin-1 source code encoding or one using
>the utf-8 encoding.
ISTM module_a.cs and module_b.cs can readily be distinguished after
compilation, whereas the sources displayed according to their declared
encodings as above (or as e.g. different editors using different native
encoding might) cannot (other than the encoding cookie itself) ;-)
Perhaps you meant something else?
>Thanks,
You're welcome.
Regards,
Bengt Richter
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] "? operator in python"
Dear sir, I m a student of Computer Science Dept. University Of Pune.(M.S.) (India). Sir , I have found that the python is about to have feature of "? " operator same as in C languge. Sir , Not Only I but the our whole Dept. is waitng for it. Kindly provide me with the information that in version of python we will be able to find that feature and when it is about to realse. Considering your best of sympathetic consideration. Hoping for early response. Thank You. Mr. Lucky R. Wankhede M.C,A, Ist, __ Yahoo! India Matrimony: Find your partner now. Go to http://yahoo.shaadi.com ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] "? operator in python"
Dear sir, I m a student of Computer Science Dept. University Of Pune.(M.S.) (India). We are learning python as a course for our semester. Found its not only use full but heart touching laguage. Sir , I have found that the python is going to have new feature, of "? " operator, same as in C languge. Kindly provide me with the information that in version of python we will be able to find that feature and when it is about to realse. Considering your best of sympathetic consideration. Hoping for early response. Thank You. Mr. Lucky R. Wankhede M.C,A, Ist, Dept. Of Comp. Sciende, University of Pune, India. __ Yahoo! India Matrimony: Find your partner now. Go to http://yahoo.shaadi.com ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 343 - multiple context managers in one statement
On 10/25/05, Nick Coghlan <[EMAIL PROTECTED]> wrote: > Paul Moore wrote: [...] > > Has the option of letting the with statement admit multiple context > > managers been considered (and presumably rejected)? [...] > Not rejected - deliberately left as a future option (this is the reason why > the RHS of an as clause has to be parenthesised if you want tuple unpacking). Thanks. I now see that note in the PEP - apologies for missing it in the first instance. [...] > The issue with that implementation is that the semantics are wrong - it > doesn't actually mirror *nested* with statements. If one of the later > __enter__ methods, or one of the first-executed __exit__ methods throws an > exception, there are a lot of __exit__ methods that get skipped. > > Getting it right is more complicated (and this probably still has mistakes): Bah. You're right, of course (about it being more complicated - I can't see any mistakes :-)) I'd argue that precisely because a naive approach gets it wrong, having your version as an example in the PEP (and possibly the documentation, much like the itertools module has a recipes section) is that much more useful. Anyway, thanks for the help. Paul. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Proposed resolutions for open PEP 343 issues
Guido writes: > I find "AttributeError: __exit__" just as informative. Eric Nieuwland responds: > I see. Then why don't we unify *Error into Error? > Just read the message and know what it means. > And we could then drop the burden of exception classes and only use the > message. > A sense of deja-vu comes over me somehow ;-) The answer (and there _IS_ an answer) is that using different exception types allows the user some flexibility in CATCHING the exceptions. The discussion you have been following obscures that point somewhat because there's little meaningful difference between TypeError and AttributeError (at least in well-written code that doesn't have unnecessary typechecks in it). If there were a significant difference between TypeError and AttributeError then Nick and Guido would have immediately chosen the appropriate error type based on functionality rather than style, and there wouldn't have been any need for discussion. Oh yeah, and you can also put extra info into an exception object besides just the error message. (We don't do that as often as we should... it's a powerful technique.) -- Michael Chermside ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Divorcing str and unicode (no more implicitconversions).
Greg Ewing asked: > Would it help if an identifier were required to be > made up of letters from the same alphabet, e.g. all > Latin or all Greek or all Cyrillic, but not a mixture. Probably, yes, though there could still be problems mixing within a program. FWIW, the Opera web browser is already using a similar solution. Domain names are limited to Latin-1 *unless* the top-level registrar has a policy to prevent spoofing. -jJ ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] "? operator in python"
Dear Lucky, You are correct. Python 2.5 will have a conditional operator. The syntax will be different than C; it will look like this: (EXPR1 if TEST else EXPR2) (which is the equivalent of TEST?EXPR1:EXPR2 in C). For more information, see PEP 308 (http://www.python.org/peps/pep-0308.html). Python 2.5 will be released some time next year; we hope to have alphas available in the 2nd quarter. Thatr's about as firm as we can currently be about the release date. Enjoy, --Guido van Rossum On 10/25/05, Lucky Wankhede <[EMAIL PROTECTED]> wrote: > > > Dear sir, > > I m a student of Computer Science Dept. > University Of Pune.(M.S.) (India). We are learning > python as a course for our semester. Found its not > only use full but heart touching laguage. > > Sir , I have found that the python is going > to have new feature, of "? " operator, same as in C > languge. > > > Kindly provide me with the information that > in version of python we will be able to find that > feature and when it is about to realse. >Considering your best of sympathetic > consideration. Hoping for early response. > > > > Thank You. > > > Mr. Lucky R. Wankhede > M.C,A, Ist, > Dept. Of Comp. > Sciende, > University of Pune, > India. -- --Guido van Rossum (home page: http://www.python.org/~guido/) ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] CVS is read-only
I just switched the repository to read-only mode, and removed the test subversion installation. I'll let you know when the conversion is complete. Regards, Martin ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Divorcing str and unicode (no more implicitconversions).
M.-A. Lemburg wrote: > You even argued against having non-ASCII identifiers: > > http://mail.python.org/pipermail/python-list/2002-May/102936.html I see :-) It seems I have changed my mind since then (which apparently predates PEP 263). One issue I apparently was worried about was the plan to use native-encoding byte strings for the identifiers; this I didn't like at all. > * Unicode identifiers are going to introduce massive > code breakage - just think of all the tools people use > to manipulate Python code today; I'm quite sure that > most of it will fail in one way or another if you present > it Unicode literals such as in "zähler += 1". True. Today, I think I would be willing to accept the code breakage: these tools had quite some time to update themselves to PEP 263 (even though not all of them have done so yet); also, usage of the feature would only spread gradually. A failure to support the feature in the Python proper would be treated as a bug by us; how tool providers deal with the feature would be their choice. > * People don't seem very interested in using Unicode > identifiers, e.g. > > http://mail.python.org/pipermail/i18n-sig/2001-February/000828.html True. However, I also suspect that lack of tool support contributes to that. For the specific case of Java, there is no notion of source encoding, which makes Unicode identifiers really tedious to use. If it were really easy to use, I assume people would actually use it - atleast in some of the contexts, like teaching, where Python is also widely used. > Do you really think that it will help with code readability > if programmers are allowed to use native scripts for their > identifiers ? Yes, I do - for some groups of users. Of course, code sharing would be more difficult, and there certainly should be a policy to use only ASCII in the standard library. But within local groups, users would find understanding code easier if they knew what the identifiers actually meant. > If you are told to debug a program > written by say a Japanese programmer using Japanese identifiers > you are going to have a really hard time. Integrating such > code into other applications will be even harder, since you'd > be forced to use his Japanese class names in your application. Certainly, yes. There is a trade-off: you can make it easier for some people to read and write code if they can use their native script; at the same time, it would be harder for others to read and modify it. It's a policy decision whether you use English identifiers or not - it shouldn't be a technical decision (as it currently is). > I think source code encodings provide an ideal way to > have comments written in native scripts - and people > use that a lot. However, keeping the program code itself > in plain ASCII makes it far more readable and reusable > across locales. Something that's important in this > globalized world. Certainly. However, some programs don't need to live in a globalized world - e.g. if they are homework in a school. Within a locale, using native scripts would make the program more readable. Regards, Martin ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Divorcing str and unicode (no more implicitconversions).
"Martin v. Löwis" <[EMAIL PROTECTED]> wrote: > > M.-A. Lemburg wrote: > > You even argued against having non-ASCII identifiers: > > > > http://mail.python.org/pipermail/python-list/2002-May/102936.html > > > > Do you really think that it will help with code readability > > if programmers are allowed to use native scripts for their > > identifiers ? > > Yes, I do - for some groups of users. Of course, code sharing > would be more difficult, and there certainly should be a policy > to use only ASCII in the standard library. But within local > groups, users would find understanding code easier if they > knew what the identifiers actually meant. According to wikipedia (http://en.wikipedia.org/wiki/Latin_alphabet), various languages have adopted a transliteration of their language and/or former alphabets into latin. They don't purport to know all of the reasons why, and I'm not going to speculate. Whether or not more languages start using the latin alphabet is a good question. Basing judgement on history and likely globalization, it is only a matter of time before basically all languages have a transcription into the latin alphabet that is taught to all (unless China takes over the world). - Josiah ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Divorcing str and unicode (no more implicitconversions).
"Martin v. Löwis" <[EMAIL PROTECTED]> wrote: > > Josiah Carlson wrote: > > In this case it's not just a misreading, the characters look identical! > > When is an 'E' not an 'E'? When it is an Epsilon or Ie. Saying what > > characters will or will not be used as identifiers, when those > > characters are keys on a keyboard of a specific type, is pretty > > presumptuous. > > Why is that rude and disrespectful? I'm certainly respecting developers > who want to use their scripts for identifiers, or else I would not have > suggested that they could do so. I never said rude, I said presumptuous. "Going beyond what is right or proper; excessively forward." (according to dictionary.com, the OED has a similar definition). I was trying to say that in stating that users wouldn't be using keys on their keyboard in their natual language when also using english characters, that you were assuming a bit about their usage patterns that you perhaps shouldn't. I certainly could also be presumptuous in stating that users may very well mix certain languages, but it seems to be more likely given keywords and the standard library using the latin alphabet. > > Indeed, they are similar, but_ different_ in my font as well. The trick > > is that the glyphs are not different in the case of certain greek or > > cyrillic letters. They don't just /look/ similar they /are identical/. > > This string: "EÎ" is the LATIN CAPITAL LETTER E, followed by the GREEK > CAPITAL LETTER EPSILON. In the font my email composer uses, the E is > slightly larger than the Epsilon - so there /is/ a visual difference. My email client doesn't handle unicode, but a quick check by swapping fonts in a word processor provides that at least on my platform, all three are the same glyph (same size, shape, ...) for all fixed-width fonts. If a platform distinguishes all three, then one should consider one's platform lucky. Not all platforms and/or preferred fonts of users are. > But even if there isn't: if this was a frequent problem, the name > error could include an alternative representation (say, with Unicode > ordinals for non-ASCII characters) which would give an easy visual > clue. It would offer a great cue, but I'm not sure if it is possible. I think that it sounds like an ugly discussion of stdout/err encodings and exception handling machinery that I don't want to be a part of. > I still doubt that this is a frequent problem, and I don't see any > better grounds for claiming that it is than for claiming that it > is not. Whether or not it is frequent will depend on the prevalence of desire to use those characters. While I don't think that such uses will be as common as using 'klass' when passing a class, I do think that it will result in more than a few sf bug reports. I also share Marc-Andre Lemburg's concerns about the understandability of code written in Kanji, Hebrew, Arabic, etc., at least for those who have not memorized the entirety of those alphabets. - Josiah ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Divorcing str and unicode (no more implicitconversions).
Josiah Carlson wrote: > According to wikipedia (http://en.wikipedia.org/wiki/Latin_alphabet), > various languages have adopted a transliteration of their language > and/or former alphabets into latin. They don't purport to know all of > the reasons why, and I'm not going to speculate. > > Whether or not more languages start using the latin alphabet is a good > question. Basing judgement on history and likely globalization, it is > only a matter of time before basically all languages have a > transcription into the latin alphabet that is taught to all (unless > China takes over the world). That is a very U.S. centric view. I don't share it, but I think it is pointless to argue against it. Regards, Martin ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Parser and Runtime: Divorced!
After a few hours of tedious and frustrating hacking I've managed to separate the Python abstract syntax tree parser from the rest of Python itself. This could be useful for people who may wish to build Python tools without Python, or tools in C/C++. In the process of doing this, I came across a comment mentioning that it would be desirable to separate the parser. Is there any interest in doing this? I now have a vague idea about how to do this. Of course, there is no point in making changes like this unless there is some tangible benefit. I will make my ugly hack available once I have polished it a little bit more. It involved hacking header files to provide a "re-implementation" of the pieces of Python that the parser needs (PyObject, PyString, and PyInt). It likely is a bit buggy, and it doesn't support all the types (notably, it is missing support for Unicode, Longs, Floats, and Complex), but it works well enough to get the AST from a few simple strings, which is what I wanted. Evan Jones -- Evan Jones http://evanjones.ca/ ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Divorcing str and unicode (no more implicitconversions).
M.-A. Lemburg wrote: > If you are told to debug a program > written by say a Japanese programmer using Japanese identifiers > you are going to have a really hard time. Or you could look upon it as an opportunity to broaden your mental horizons by learning some Japanese. :-) -- Greg Ewing, Computer Science Dept, +--+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | [EMAIL PROTECTED] +--+ ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Divorcing str and unicode (no more implicitconversions).
Martin v. Löwis wrote: > Not in the literal sense: you certainly want to allow > "latin" digits in, say, a cyrillic identifier. Yes, by "alphabet" I really only meant the letters, although you might want to apply the same idea to clusters of digits within an identifier, depending on how potentially confusable the various sets of digits are -- I'm not familiar enough with alternative digit sets to know whether that would be a problem. > Just because > you *can* come up with look-alike identifiers doesn't > mean that people will use them, or that they will mistake > the scripts (except for deliberately doing so, of > course). I still think this is a much worse potential problem than that of "l" vs "1", etc. It's reasonable to adopt the practice of never using "l" as a single letter identifier, for example. But it would be unreasonable to ban the use of "E" as an identifier on the grounds that someone somewhere might confuse it with a capital epsilon. An alternative would be to identify such confusable letters in the various alphabets and define them to be equivalent. And beyond the issue of alphabets there's also the question of whether accented characters should be considered distinct. I can see quite a few holy flame wars erupting over that... -- Greg Ewing, Computer Science Dept, +--+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | [EMAIL PROTECTED] +--+ ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Divorcing str and unicode (no more implicitconversions).
"Martin v. Löwis" <[EMAIL PROTECTED]> wrote: > Josiah Carlson wrote: > > According to wikipedia (http://en.wikipedia.org/wiki/Latin_alphabet), > > various languages have adopted a transliteration of their language > > and/or former alphabets into latin. They don't purport to know all of > > the reasons why, and I'm not going to speculate. > > > > Whether or not more languages start using the latin alphabet is a good > > question. Basing judgement on history and likely globalization, it is > > only a matter of time before basically all languages have a > > transcription into the latin alphabet that is taught to all (unless > > China takes over the world). > > That is a very U.S. centric view. I don't share it, but I think it is > pointless to argue against it. I should have included a ;). Whether or not in the future all languages use the latin alphabet should have little to do with whether Python chooses to support non-latin identifiers in the forthcoming 2.5 or later releases. - Josiah ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
