Re: UnicodeEncodeError when not running script from IDE

2013-02-12 Thread Magnus Pettersson
Thanks a lot Steven, you gave me a good AHA experience! :) Now I understand why I had to use encoding when calling the urllib2! So basically Eclipse PyDev does this in the background for me, and its console supports utf-8, so thats why i never had to think about it before (and why some scripts

Re: UnicodeEncodeError when not running script from IDE

2013-02-12 Thread Magnus Pettersson
> You don't show the code that actually does the io.open(), nor the > > url.encode, so I'm not going to guess what you're actually doing. Hmm im not sure what you mean but I wrote all code needed in a previous post so maybe you missed that one :) In short I basically just have: import io io.op

Re: UnicodeEncodeError when not running script from IDE

2013-02-12 Thread Magnus Pettersson
> What encoding is this file? Since you're appending to it, you really > > need to match the pre-existing encoding, or the next program to deal > > with it is in big trouble. So using the io.open() without the encoding= > > keyword is probably a mistake. The .txt file is in UTF-8 I have g

Re: UnicodeEncodeError when not running script from IDE

2013-02-12 Thread Magnus Pettersson
> Are you sure you are writing the same data? That would mean that pydev > > changes the default encoding -- which is evil. > > > > A portable approach would be to use codecs.open() or io.open() instead of > > the built-in: > > > > import io > > with io.open(filepath, "a") as f: > >

Re: UnicodeEncodeError when not running script from IDE

2013-02-12 Thread Magnus Pettersson
ter u'\u898b' in position 3 2: ordinal not in range(128) On Tuesday, February 12, 2013 12:01:19 PM UTC+1, Andrew Berg wrote: > On 2013.02.12 04:43, Magnus Pettersson wrote: > > > I am using Eclipse to write my python scripts and when i run them from > > inside eclipse th

Re: UnicodeEncodeError when not running script from IDE

2013-02-12 Thread Magnus Pettersson
: Thanks, your crystal ball seems to be right :P On Tuesday, February 12, 2013 12:43:00 PM UTC+1, Steven D'Aprano wrote: > Magnus Pettersson wrote: > > > > > I am using Eclipse to write my python scripts and when i run them from > > > inside eclipse they work fine wi

UnicodeEncodeError when not running script from IDE

2013-02-12 Thread Magnus Pettersson
I am using Eclipse to write my python scripts and when i run them from inside eclipse they work fine without errors. But almost in every script that handle some form of special characters like swedish åäö and chinese characters etc i get Unicode errors when running the script externally with p