Re: eval and unicode

2008-03-25 Thread Laszlo Nagy
> I think your confusion comes from the use of the interactive mode. > It is not. The example provided in the original post will also work when you put then into a python source file. > PEP 263 doesn't really apply to the interactive mode, hence the > behavior in interactive mode is undefined,

Re: eval and unicode

2008-03-25 Thread Laszlo Nagy
Martin v. Löwis wrote: >> eval() somehow decoded the passed expression. No question. It did not >> use 'ascii', nor 'latin2' but something else. Why is that? Why there >> is a particular encoding hard coded into eval? Which is that >> encoding? (I could not decide which one, since '\xdb' will be

Re: eval and unicode

2008-03-21 Thread Martin v. Löwis
> eval() somehow decoded the passed expression. No question. It did not > use 'ascii', nor 'latin2' but something else. Why is that? Why there is > a particular encoding hard coded into eval? Which is that encoding? (I > could not decide which one, since '\xdb' will be the same in latin1, > lat

Re: eval and unicode

2008-03-21 Thread Martin v. Löwis
> Well at least this is what I think. If I'm not right then please explain > why. I think your confusion comes from the use of the interactive mode. PEP 263 doesn't really apply to the interactive mode, hence the behavior in interactive mode is undefined, and may and will change across Python ve

Re: eval and unicode

2008-03-21 Thread Jonathan Gardner
On Mar 21, 1:54 am, Laszlo Nagy <[EMAIL PROTECTED]> wrote: >  >>> eval( "# -*- coding: latin2 -*-\n" + expr) > u'\u0170' # You can specify the encoding for eval, that is cool. > I didn't think of that. That's pretty cool. > I hope it is clear now.  Inside eval, an unicode object was created from

Re: eval and unicode

2008-03-21 Thread Laszlo Nagy
> > Your problem is, I think, that you think the magic of decoding source > code from the byte sequence into unicode happens in exec or eval. It > doesn't. It happens in between reading the file and passing the > contents of the file to exec or eval. > I think you are wrong here. Decoding sourc

Re: eval and unicode

2008-03-21 Thread Laszlo Nagy
Hi Jonathan, I think I made it too complicated and I did not concentrate on the question. I could write answers to your post, but I'm going to explain it formally: >>> s = '\xdb' # This is a byte, without encoding specified. >>> s.decode('latin1') u'\xdb' # The above byte decoded in lat

Re: eval and unicode

2008-03-20 Thread Jonathan Gardner
On Mar 20, 2:20 pm, Laszlo Nagy <[EMAIL PROTECTED]> wrote: > > >>  >>> eval( u'"徹底したコスト削減 ÁÍŰŐÜÖÚÓÉ трирова"' ) == eval( '"徹底し > >> たコスト削減 ÁÍŰŐÜÖÚÓÉ трирова"' ) > >> True > > > When you feed your unicode data into eval(), it doesn't have any > > encoding or decoding work to do. > > Yes, but what ab

Re: eval and unicode

2008-03-20 Thread Laszlo Nagy
>> I tried to use eval with/without unicode strings and it worked. Example: >> >> >>> eval( u'"徹底したコスト削減 ÁÍŰŐÜÖÚÓÉ трирова"' ) == eval( '"徹底し >> たコスト削減 ÁÍŰŐÜÖÚÓÉ трирова"' ) >> True >> > When you feed your unicode data into eval(), it doesn't have any > encoding or decoding work to do. >

Re: eval and unicode

2008-03-20 Thread Jonathan Gardner
On Mar 20, 5:20 am, Laszlo Nagy <[EMAIL PROTECTED]> wrote: > How can I specify encoding for the built-in eval function? Here is the > documentation: > > http://docs.python.org/lib/built-in-funcs.html > > It tells that the "expression" parameter is a string. But tells nothing > about the encoding. S

eval and unicode

2008-03-20 Thread Laszlo Nagy
How can I specify encoding for the built-in eval function? Here is the documentation: http://docs.python.org/lib/built-in-funcs.html It tells that the "expression" parameter is a string. But tells nothing about the encoding. Same is true for: execfile, eval and compile. The basic problem: - e