Re: Strange UnicodeEncodeError in Windows image on Azure DevOps and Github

2022-11-13 Thread Jessica Smith
On Fri, Nov 11, 2022 at 8:16 PM Eryk Sun wrote: > If sys.std* are console files, then in Python 3.6+, sys.std*.buffer.raw will > be _io._WindowsConsoleIO > io.TextIOWrapper uses locale.getpreferredencoding(False) as the default > encoding Thank you for your replies - checking the sys.stdout.buf

Re: Strange UnicodeEncodeError in Windows image on Azure DevOps and Github

2022-11-11 Thread Inada Naoki
On Sat, Nov 12, 2022 at 11:53 AM Inada Naoki wrote: > > On Sat, Nov 12, 2022 at 10:21 AM 12Jessicasmith34 > <12jessicasmit...@gmail.com> wrote: > > > > > > Two questions: any idea why this would be happening in this situation? > > AFAIK, stdout *is* a console when these images are running the pyt

Re: Strange UnicodeEncodeError in Windows image on Azure DevOps and Github

2022-11-11 Thread Inada Naoki
On Sat, Nov 12, 2022 at 10:21 AM 12Jessicasmith34 <12jessicasmit...@gmail.com> wrote: > > > Two questions: any idea why this would be happening in this situation? AFAIK, > stdout *is* a console when these images are running the python process. > Second - is there a way I can check the locale and

Re: Strange UnicodeEncodeError in Windows image on Azure DevOps and Github

2022-11-11 Thread Eryk Sun
On 11/11/22, 12Jessicasmith34 <12jessicasmit...@gmail.com> wrote: > > any idea why this would be happening in this situation? AFAIK, stdout > *is* a console when these images are running the python process. If sys.std* are console files, then in Python 3.6+, sys.std*.buffer.raw will be _io._Window

Re: Strange UnicodeEncodeError in Windows image on Azure DevOps and Github

2022-11-11 Thread 12Jessicasmith34
> If stdout isn't a console (e.g. a pipe), it defaults to using the process code page (i.e. CP_ACP), such as legacy code page 1252 (extended Latin-1). First off, really helpful information, thank you. That was the exact background I was missing. Two questions: any idea why this

Re: Strange UnicodeEncodeError in Windows image on Azure DevOps and Github

2022-11-11 Thread Eryk Sun
On 11/10/22, Jessica Smith <12jessicasmit...@gmail.com> wrote: > > Weird issue I've found on Windows images in Azure Devops Pipelines and > Github actions. Printing Unicode characters fails on these images because, > for some reason, the encoding is mapped to cp1252. What is particularly > weird ab

Strange UnicodeEncodeError in Windows image on Azure DevOps and Github

2022-11-11 Thread Jessica Smith
sh.EXE -command ". '{0}'" 2022-11-10T23:55:36.4595740Z ##[endgroup] 2022-11-10T23:55:36.8739309Z Traceback (most recent call last): 2022-11-10T23:55:37.1316425Z File "", line 1, in 2022-11-10T23:55:37.1317452Z File "C:\hostedtoolcache\windows\Python\3.9.13\

Re: Sending file to the user gives UnicodeEncodeError in Flask framework

2018-09-12 Thread dieter
error] [pid 5172] [remote >> 46.103.174.201:14089] File >> "/usr/lib/python3.6/site-packages/flask/helpers.py", line 592, in >> send_file [Wed Sep 12 14:10:48.450221 2018] [wsgi:error] [pid 5172] >> [remote 46.103.174.201:14089] file = open(filename, 

Re: Sending file to the user gives UnicodeEncodeError in Flask framework

2018-09-12 Thread Alister via Python-list
t;/usr/lib/python3.6/site-packages/flask/helpers.py", line 592, in > send_file [Wed Sep 12 14:10:48.450221 2018] [wsgi:error] [pid 5172] > [remote 46.103.174.201:14089] file = open(filename, 'rb') > [Wed Sep 12 14:10:48.450237 2018] [wsgi:error] [pid 5172]

Re: UnicodeEncodeError: 'ascii' codec can't encode character u'\ua000' in position 0: ordinal not in range(128)

2015-01-14 Thread Dave Angel
ead of the current thread. I'll assume you're using Python 2.7, on Linux or equivalent. I am trying to understand what does encode() do. What are the hex representations of "u" in main.py? Why there is UnicodeEncodeError when main.py is piped to xxd? Why there is no such erro

UnicodeEncodeError: 'ascii' codec can't encode character u'\ua000' in position 0: ordinal not in range(128)

2015-01-13 Thread Peng Yu
Hi, I am trying to understand what does encode() do. What are the hex representations of "u" in main.py? Why there is UnicodeEncodeError when main.py is piped to xxd? Why there is no such error when it is not piped? Thanks. ~$ cat main.py #!/usr/bin/env python u = unichr(40960

Re: python33, windows, UnicodeEncodeError: 'charmap' codec can't encode characters in position, to print out the file contents to stdout,

2014-07-06 Thread Rick Johnson
ecent call last): > File "", line 1, inFile > "D:\jm\jmpy\eta\eta40beta2\etastdio.py", line 158, in write > s = s.encode(self.pencoding).decode('cp1252') > File "C:\Python32\lib\encodings\cp437.py", line 12, in encode > re

Re: python33, windows, UnicodeEncodeError: 'charmap' codec can't encode characters in position, to print out the file contents to stdout,

2014-07-06 Thread Rick Johnson
On Sunday, July 6, 2014 11:14:26 AM UTC-5, Terry Reedy wrote: > On 7/6/2014 10:52 AM, Rick Johnson wrote: > > So the direct reason for failure is due to the fact that the > > "print()" function ONLY handles strings, not list objects. > >>> print(object()) > > >>> print(['abc', 'cdf']) > ['abc',

Re: python33, windows, UnicodeEncodeError: 'charmap' codec can't encode characters in position, to print out the file contents to stdout,

2014-07-06 Thread Terry Reedy
On 7/6/2014 10:52 AM, Rick Johnson wrote: So the direct reason for failure is due to the fact that the "print()" function ONLY handles strings, not list objects. >>> print(object()) >>> print(['abc', 'cdf']) ['abc', 'cdf'] Since the original poster did not copy the traceback from the print

Re: python33, windows, UnicodeEncodeError: 'charmap' codec can't encode characters in position, to print out the file contents to stdout,

2014-07-06 Thread Rick Johnson
On Sunday, July 6, 2014 4:05:10 AM UTC-5, gintare wrote: > The correct code: > f=open(,'r', encoding='utf-8') > linef=f.readlines() > print(repr(linef)) Yes but do you understand why? And even if you DO understand why, you should explain the details because the neophytes are always watching!

Re: python33, windows, UnicodeEncodeError: 'charmap' codec can't encode characters in position, to print out the file contents to stdout,

2014-07-06 Thread Dave Angel
gintare Wrote in message: > The answer is on > page:https://docs.python.org/3.3/howto/unicode.html#reading-and-writing-unicode-data > > The correct code: > f=open('C:\Python33\Scripts\lang\langu\svtxt.txt','r', encoding='utf-8') > linef=f.readlines() > print(repr(linef)) > But naturally the pa

Re: python33, windows, UnicodeEncodeError: 'charmap' codec can't encode characters in position, to print out the file contents to stdout,

2014-07-06 Thread gintare
The answer is on page:https://docs.python.org/3.3/howto/unicode.html#reading-and-writing-unicode-data The correct code: f=open('C:\Python33\Scripts\lang\langu\svtxt.txt','r', encoding='utf-8') linef=f.readlines() print(repr(linef)) -- https://mail.python.org/mailman/listinfo/python-list

python33, windows, UnicodeEncodeError: 'charmap' codec can't encode characters in position, to print out the file contents to stdout,

2014-07-06 Thread gintare
","ignore") File "C:\Python33\lib\encodings\cp437.py", line 19, in encode return codecs.charmap_encode(input,self.errors,encoding_map)[0] UnicodeEncodeError: 'charmap' codec can't encode characters in position 91-92: -- https://mail.python.org/mailman/listinfo/python-list

Re: Problem writing some strings (UnicodeEncodeError)

2014-01-13 Thread Paulo da Silva
Em 13-01-2014 17:29, Peter Otten escreveu: > Paulo da Silva wrote: > >> Em 13-01-2014 08:58, Peter Otten escreveu: > > I looked around in the stdlib and found shlex.quote(). It uses ' instead of > " which simplifies things, and special-cases only ': > print(shlex.quote("alpha'beta")) > 'a

Re: Problem writing some strings (UnicodeEncodeError)

2014-01-13 Thread Peter Otten
Paulo da Silva wrote: > Em 13-01-2014 08:58, Peter Otten escreveu: >> Peter Otten wrote: >> >>> Paulo da Silva wrote: >>> Em 12-01-2014 20:29, Peter Otten escreveu: > Paulo da Silva wrote: > >>> but I have not tried it myself. Also, some bytes may need to be >>> escaped, eith

Re: Problem writing some strings (UnicodeEncodeError)

2014-01-13 Thread Paulo da Silva
Em 13-01-2014 08:58, Peter Otten escreveu: > Peter Otten wrote: > >> Paulo da Silva wrote: >> >>> Em 12-01-2014 20:29, Peter Otten escreveu: Paulo da Silva wrote: >> but I have not tried it myself. Also, some bytes may need to be >> escaped, either to be understood by the shell,

Re: Problem writing some strings (UnicodeEncodeError)

2014-01-13 Thread Peter Otten
Peter Otten wrote: > Paulo da Silva wrote: > >> Em 12-01-2014 20:29, Peter Otten escreveu: >>> Paulo da Silva wrote: >>> > but I have not tried it myself. Also, some bytes may need to be > escaped, either to be understood by the shell, or to address security > concerns: > >>

Re: Problem writing some strings (UnicodeEncodeError)

2014-01-13 Thread Peter Otten
Paulo da Silva wrote: > Em 12-01-2014 20:29, Peter Otten escreveu: >> Paulo da Silva wrote: >> but I have not tried it myself. Also, some bytes may need to be escaped, either to be understood by the shell, or to address security concerns: >>> >>> Since I am puting the file nam

Re: Problem writing some strings (UnicodeEncodeError)

2014-01-12 Thread Paulo da Silva
Em 12-01-2014 20:29, Peter Otten escreveu: > Paulo da Silva wrote: > >>> but I have not tried it myself. Also, some bytes may need to be escaped, >>> either to be understood by the shell, or to address security concerns: >>> >> >> Since I am puting the file names between "", the only char that nee

Re: Problem writing some strings (UnicodeEncodeError)

2014-01-12 Thread Peter Otten
Paulo da Silva wrote: >> but I have not tried it myself. Also, some bytes may need to be escaped, >> either to be understood by the shell, or to address security concerns: >> > > Since I am puting the file names between "", the only char that needs to > be escaped is the " itself. What about the

Re: Problem writing some strings (UnicodeEncodeError)

2014-01-12 Thread Paulo da Silva
> > I think instead of the hard way sketched out above it will be sufficient to > specify the error handler when opening the destination file > > shf = open(bashfilename, 'w', errors="surrogateescape") This seems to fix everything! I tried with a small test set and it worked. > > but I have no

Re: Problem writing some strings (UnicodeEncodeError)

2014-01-12 Thread Peter Otten
.walk >>> for fn in filenames: >>> ... >>> cmd=templ.replace("",fn) >>> shf.write(cmd) >>> >>> For certain filenames I got a UnicodeEncodeError exception at >>> shf.write(cmd)! >>> I use utf-8 and have # -*- coding

Re: Problem writing some strings (UnicodeEncodeError)

2014-01-12 Thread Paulo da Silva
ing with the filename and then write the command to the bash >> script file. >> >> Something like this: >> >> shf=open(bashfilename,'w') >> filenames=getfilenames() # uses os.walk >> for fn in filenames: >> ... >> cmd=templ.replace("

Re: Problem writing some strings (UnicodeEncodeError)

2014-01-12 Thread Emile van Sebille
file. Something like this: shf=open(bashfilename,'w') filenames=getfilenames() # uses os.walk for fn in filenames: ... cmd=templ.replace("",fn) shf.write(cmd) For certain filenames I got a UnicodeEncodeError exception at shf.write(cmd)! I use utf-8 and ha

Re: Problem writing some strings (UnicodeEncodeError)

2014-01-12 Thread Peter Otten
le. > > Something like this: > > shf=open(bashfilename,'w') > filenames=getfilenames() # uses os.walk > for fn in filenames: > ... > cmd=templ.replace("",fn) > shf.write(cmd) > > For certain filenames I got a UnicodeEncodeError exception at

Re: Problem writing some strings (UnicodeEncodeError)

2014-01-12 Thread Albert-Jan Roskam
On Sun, 1/12/14, Paulo da Silva wrote: Subject: Problem writing some strings (UnicodeEncodeError) To: python-list@python.org Date: Sunday, January 12, 2014, 4:36 PM Hi! I am using a python3 script to produce a bash script from lots of

Problem writing some strings (UnicodeEncodeError)

2014-01-12 Thread Paulo da Silva
,'w') filenames=getfilenames() # uses os.walk for fn in filenames: ... cmd=templ.replace("",fn) shf.write(cmd) For certain filenames I got a UnicodeEncodeError exception at shf.write(cmd)! I use utf-8 and have # -*- coding: utf-8 -*- in the source .py.

Re: Trouble with UnicodeEncodeError and email

2014-01-08 Thread Florian Lindner
Am Donnerstag, 9. Januar 2014, 00:26:15 schrieb Chris Angelico: > On Thu, Jan 9, 2014 at 12:14 AM, Florian Lindner wrote: > > I've written some tiny script using Python 3 and it used to work perfectly. > > Then I realized it needs to run on my Debian Stable server too, which > > offers only Pyth

Re: Trouble with UnicodeEncodeError and email

2014-01-08 Thread Chris Angelico
On Thu, Jan 9, 2014 at 12:14 AM, Florian Lindner wrote: > I've written some tiny script using Python 3 and it used to work perfectly. > Then I realized it needs to run on my Debian Stable server too, which offers > only Python 2. Ok, most backporting was a matter of minutes, but I'm becoming >

Trouble with UnicodeEncodeError and email

2014-01-08 Thread Florian Lindner
/text.py", line 229, in tokenize(preprocess(self.decode(doc))), stop_words) File "flofify.py", line 119, in mail_preprocessor msg = email.message_from_string(xxx) File "/usr/lib/python2.7/email/__init__.py", line 57, in message_from_string return Parser(*args, *

Re: UnicodeEncodeError: SOLVED

2013-10-10 Thread Walter Hurry
On Thu, 10 Oct 2013 01:47:52 +, Steven D'Aprano wrote: > On Wed, 09 Oct 2013 14:41:53 +, Walter Hurry wrote: > >> Many thanks to those prepared to forgive my transgression in the >> 'Goodbye' thread. I mentioned there that I was puzzled by a >> Un

Re: UnicodeEncodeError: SOLVED

2013-10-09 Thread Steven D'Aprano
On Wed, 09 Oct 2013 14:41:53 +, Walter Hurry wrote: > Many thanks to those prepared to forgive my transgression in the > 'Goodbye' thread. I mentioned there that I was puzzled by a > UnicodeEncodeError, and said I would rise it as a separate thread. > > However, via

UnicodeEncodeError: SOLVED

2013-10-09 Thread Walter Hurry
Many thanks to those prepared to forgive my transgression in the 'Goodbye' thread. I mentioned there that I was puzzled by a UnicodeEncodeError, and said I would rise it as a separate thread. However, via this link, I was able to resolve the issue myself: http://stackoverflow.com

Re: UnicodeEncodeError when not running script from IDE

2013-02-12 Thread Dave Angel
On 02/12/2013 07:20 PM, Magnus Pettersson wrote: You don't show the code that actually does the io.open(), nor the url.encode, so I'm not going to guess what you're actually doing. Hmm im not sure what you mean but I wrote all code needed in a previous post so maybe you missed that one :) I

Re: UnicodeEncodeError when not running script from IDE

2013-02-12 Thread Magnus Pettersson
Thanks a lot Steven, you gave me a good AHA experience! :) Now I understand why I had to use encoding when calling the urllib2! So basically Eclipse PyDev does this in the background for me, and its console supports utf-8, so thats why i never had to think about it before (and why some scripts

Re: UnicodeEncodeError when not running script from IDE

2013-02-12 Thread Magnus Pettersson
> You don't show the code that actually does the io.open(), nor the > > url.encode, so I'm not going to guess what you're actually doing. Hmm im not sure what you mean but I wrote all code needed in a previous post so maybe you missed that one :) In short I basically just have: import io io.op

Re: UnicodeEncodeError when not running script from IDE

2013-02-12 Thread Steven D'Aprano
Magnus Pettersson wrote: > # This made the fetching of the website work. Why did i have to write > # url.encode("UTF-8") when url already is unicode? I feel i dont have a > # good understanding of this. > page = urllib2.urlopen(url.encode("UTF-8")) Start here: "The Absolute Minimum Every Softw

Re: UnicodeEncodeError when not running script from IDE

2013-02-12 Thread MRAB
The file it writes to exists and im just appending to it, but when i run the script trough eclipse, all is fine. When i run in terminal i get this error instead: File "K:\dev\python\webscraping\kanji_anki.py", line 69, in savefile f.write(card+"\n") UnicodeEncodeError

Re: UnicodeEncodeError when not running script from IDE

2013-02-12 Thread Dave Angel
On 02/12/2013 12:12 PM, Magnus Pettersson wrote: < snip > #Here kanji = u"私" baseurl = u"http://www.romajidesu.com/kanji/"; url = baseurl+kanji savefile([url]) #this test works now. uses: io.open(filepath, "a",encoding="UTF-8") as f: # This made the fetching of the website work. You don't sh

Re: UnicodeEncodeError when not running script from IDE

2013-02-12 Thread Fabio Zadrozny
Just to note, PyDev does something behind the scenes (it sets the encoding for the console). You may specify which encoding you want at your launch configuration (in the 'common' tab you can set the encoding you want for the shell). Cheers, Fabio On Tue, Feb 12, 2013 at 3:12 PM, Magnus Petters

Re: UnicodeEncodeError when not running script from IDE

2013-02-12 Thread Magnus Pettersson
> What encoding is this file? Since you're appending to it, you really > > need to match the pre-existing encoding, or the next program to deal > > with it is in big trouble. So using the io.open() without the encoding= > > keyword is probably a mistake. The .txt file is in UTF-8 I have g

Re: UnicodeEncodeError when not running script from IDE

2013-02-12 Thread Terry Reedy
On 2/12/2013 7:34 AM, Magnus Pettersson wrote: Ahh so its the actual printing that makes it error out outside of eclipse because its a different terminal that its printing to. Its the default DOS terminal in windows that runs then i run the script with python.exe and i guess its the same when i r

Re: UnicodeEncodeError when not running script from IDE

2013-02-12 Thread Dave Angel
So using the io.open() without the encoding= keyword is probably a mistake. f.write(card+"\n") File "C:\python27\lib\encodings\cp1252.py", line 19, in encode return codecs.charmap_encode(input,self.errors,encoding_table)[0] UnicodeEncodeError: 'charmap' cod

Re: UnicodeEncodeError when not running script from IDE

2013-02-12 Thread Peter Otten
i get error inside of eclipse now: > > f.write(card+"\n") > File "C:\python27\lib\encodings\cp1252.py", line 19, in encode > return codecs.charmap_encode(input,self.errors,encoding_table)[0] > UnicodeEncodeError: 'charmap' codec can't encode

Re: UnicodeEncodeError when not running script from IDE

2013-02-12 Thread Magnus Pettersson
hanged open() to io.open() i get error inside of eclipse now: f.write(card+"\n") File "C:\python27\lib\encodings\cp1252.py", line 19, in encode return codecs.charmap_encode(input,self.errors,encoding_table)[0] UnicodeEncodeError: 'charmap' codec can't e

Re: UnicodeEncodeError when not running script from IDE

2013-02-12 Thread Peter Otten
ot;) > > The file it writes to exists and im just appending to it, but when i run > the script trough eclipse, all is fine. When i run in terminal i get this > error instead: > > File "K:\dev\python\webscraping\kanji_anki.py", line 69, in savefile > f.write(ca

Re: UnicodeEncodeError when not running script from IDE

2013-02-12 Thread Magnus Pettersson
ng to it, but when i run the script trough eclipse, all is fine. When i run in terminal i get this error instead: File "K:\dev\python\webscraping\kanji_anki.py", line 69, in savefile f.write(card+"\n") UnicodeEncodeError: 'ascii' codec can't encode charac

Re: UnicodeEncodeError when not running script from IDE

2013-02-12 Thread Magnus Pettersson
om wihin eclipse > > > because of this error but now i want to get the bottom of this so i dont > > > have to open eclipse everytime i want to run a script! > > > > > > Here is the error i get now when running the script with python.exe: > > > Unico

Re: UnicodeEncodeError when not running script from IDE

2013-02-12 Thread Steven D'Aprano
launched the script gui from wihin eclipse > because of this error but now i want to get the bottom of this so i dont > have to open eclipse everytime i want to run a script! > > Here is the error i get now when running the script with python.exe: > UnicodeEncodeError:'charmap'

Re: UnicodeEncodeError when not running script from IDE

2013-02-12 Thread Andrew Berg
ow i want to get the bottom of this so i dont have to open eclipse > everytime i want to run a script! > > Here is the error i get now when running the script with python.exe: > UnicodeEncodeError:'charmap' codec cant encode character u'\u898b' in > position 3

UnicodeEncodeError when not running script from IDE

2013-02-12 Thread Magnus Pettersson
a script! Here is the error i get now when running the script with python.exe: UnicodeEncodeError:'charmap' codec cant encode character u'\u898b' in position 32: character maps to what can i do to fix this? -- http://mail.python.org/mailman/listinfo/python-list

Re: UnicodeEncodeError in compile

2012-01-11 Thread pyscripter
Indeed, on Windows NT the file system encoding should not be mbcs, since it creates UnicodeEncodeErrors on perfectly valid file names. -- http://mail.python.org/mailman/listinfo/python-list

Re: UnicodeEncodeError in compile

2012-01-11 Thread Dave Angel
On 01/11/2012 06:27 AM, pyscrip...@gmail.com wrote: Maybe the example of this question can be added to the issue 13785 as a proof that compile fails on valid file names. But I think the real issue is why on modern Windows systems the file system encoding is mbcs. Shouldn't it be utf-16? Depe

Re: UnicodeEncodeError in compile

2012-01-11 Thread pyscripter
On Wednesday, January 11, 2012 5:50:51 AM UTC+2, Terry Reedy wrote: > On 1/10/2012 3:08 AM, Terry Reedy wrote: > Is this a filename that could be an actual, valid filename on your system? Yes it is. open works on that file. > Good question. I believe this holdover from 2.x should be deleted. >

Re: UnicodeEncodeError in compile

2012-01-11 Thread jmfauth
uestion.) > > -- win7, cp1252 Ok. I was not aware of this. >>> '\N{CYRILLIC SMALL LETTER A}'.encode('mbcs') Traceback (most recent call last): File "", line 1, in UnicodeEncodeError: 'mbcs' codec can't encode characters in p

Re: UnicodeEncodeError in compile

2012-01-11 Thread jmfauth
quot;copyright", "credits" or "license" for more information. > >>>> '\u5de5'.encode('utf-8') > > b'\xe5\xb7\xa5' > >>>> '\u5de5'.encode('mbcs') > > Traceback (most recent call last): > >  

Re: UnicodeEncodeError in compile

2012-01-10 Thread Terry Reedy
valid filename on your system? UnicodeEncodeError: 'mbcs' codec can't encode characters in position 0--1: invalid character Can anybody explain why the compile statement tries to convert the unicode filename using mbcs? Good question. I believe this holdover from 2.x should be deleted. I arg

Re: UnicodeEncodeError in compile

2012-01-10 Thread Terry Reedy
code('utf-8') b'\xe5\xb7\xa5' '\u5de5'.encode('mbcs') Traceback (most recent call last): File "", line 1, in UnicodeEncodeError: 'mbcs' codec can't encode characters in position 0--1: inval id character D:\>c:\python27\python.exe P

Re: UnicodeEncodeError in compile

2012-01-10 Thread jmfauth
ormation. >>> '\u5de5'.encode('utf-8') b'\xe5\xb7\xa5' >>> '\u5de5'.encode('mbcs') Traceback (most recent call last): File "", line 1, in UnicodeEncodeError: 'mbcs' codec can't encode characters in posi

Re: UnicodeEncodeError in compile

2012-01-10 Thread jmfauth
27; >>> '\u5de5' '工' >>> '\u5de5'.encode('mbcs') Traceback (most recent call last): File "", line 1, in UnicodeEncodeError: 'mbcs' codec can't encode characters in position 0--1: invalid character >>>

Re: UnicodeEncodeError in compile

2012-01-10 Thread 88888 Dihedral
Terry Reedy於 2012年1月10日星期二UTC+8下午4時08分40秒寫道: > On 1/9/2012 11:24 PM, pyscr...@gmail.com wrote: > > Using python 3.2 in Windows 7 I am getting the following: > > > >>> compile('pass', r'c:\temp\工具\module1.py', 'exec') > > UnicodeEnc

Re: UnicodeEncodeError in compile

2012-01-10 Thread jmfauth
x27;CJK UNIFIED IDEOGRAPH-5177' >>> hex(ord(('工'))) '0x5de5' >>> hex(ord('具')) '0x5177' >>> 2) It semms the mbcs codec has some difficulties with these chars. >>> '\u5de5'.encode('mbcs') Tracebac

Re: UnicodeEncodeError in compile

2012-01-10 Thread Terry Reedy
On 1/9/2012 11:24 PM, pyscrip...@gmail.com wrote: Using python 3.2 in Windows 7 I am getting the following: compile('pass', r'c:\temp\工具\module1.py', 'exec') UnicodeEncodeError: 'mbcs' codec can't encode characters in position 0--1: invalid c

UnicodeEncodeError in compile

2012-01-09 Thread pyscripter
Using python 3.2 in Windows 7 I am getting the following: >>compile('pass', r'c:\temp\工具\module1.py', 'exec') UnicodeEncodeError: 'mbcs' codec can't encode characters in position 0--1: invalid character Can anybody explain why the compile sta

Re: UnicodeEncodeError when piping stdout, but not when printing directly to the console

2012-01-06 Thread Adam Funk
On 2012-01-06, Peter Otten wrote: > Adam Funk wrote: > >> On 2012-01-04, Peter Otten wrote: >> >>> Adam Funk wrote: >> How can I force python (preferably within my python program, rather than having to set something externally) to treat stdout as UTF-8? >>> >>> >>> $ cat force_utf8.py

Re: UnicodeEncodeError when piping stdout, but not when printing directly to the console

2012-01-06 Thread Peter Otten
Adam Funk wrote: > On 2012-01-04, Peter Otten wrote: > >> Adam Funk wrote: > >>> How can I force python (preferably within my python program, rather >>> than having to set something externally) to treat stdout as UTF-8? >> >> >> $ cat force_utf8.py >> # -*- coding: utf-8 -*- >> import sys >> >>

Re: UnicodeEncodeError when piping stdout, but not when printing directly to the console

2012-01-06 Thread Adam Funk
On 2012-01-04, Peter Otten wrote: > Adam Funk wrote: >> How can I force python (preferably within my python program, rather >> than having to set something externally) to treat stdout as UTF-8? > > > $ cat force_utf8.py > # -*- coding: utf-8 -*- > import sys > > if sys.stdout.encoding is None: >

Re: UnicodeEncodeError when piping stdout, but not when printing directly to the console

2012-01-04 Thread Peter Otten
> or >), unicode strings fail with the following (for example): > > UnicodeEncodeError: 'ascii' codec can't encode character u'\u0107' in > position 21: ordinal not in range(128) > > How can I force python (preferably within my python program, rat

UnicodeEncodeError when piping stdout, but not when printing directly to the console

2012-01-04 Thread Adam Funk
ollowing (for example): UnicodeEncodeError: 'ascii' codec can't encode character u'\u0107' in position 21: ordinal not in range(128) How can I force python (preferably within my python program, rather than having to set something externally) to treat stdout as UTF-8? T

Re: UnicodeEncodeError -- 'character maps to '

2011-08-27 Thread Ben Finney
Steven D'Aprano writes: > >>> s = u'BIEBER FEVER \u2665' > >>> print s # Printing Unicode is fine. > BIEBER FEVER ♥ You're a cruel man. Why do you hate me? -- \ “If nature has made any one thing less susceptible than all | `\others of exclusive property, it is the action of the

Re: UnicodeEncodeError -- 'character maps to '

2011-08-27 Thread Steven D'Aprano
J wrote: > Hi there, > > I'm attempting to print a dictionary entry of some twitter data to screen > but every now and then I get the following error: > > (, UnicodeEncodeError('charmap', > u'RT @ciaraluvsjb26: BIEBER FEVER \u2665', 32, 33,

UnicodeEncodeError -- 'character maps to '

2011-08-27 Thread J
Hi there, I'm attempting to print a dictionary entry of some twitter data to screen but every now and then I get the following error: (, UnicodeEncodeError('charmap', u'RT @ciaraluvsjb26: BIEBER FEVER \u2665', 32, 33, 'character maps to '), ) I have google

os.stat UnicodeEncodeError:

2011-03-22 Thread Sandy Oz
Hello everyone, I'm running into a problem with file names containing Unicode chars. Here is the error that I get when calling os.path.isfile: File "/usr/lib/python2.6/genericpath.py", line 29, in isfile st = os.stat(path) UnicodeEncodeError: 'ascii' codec c

Re: UnicodeEncodeError during repr()

2010-04-19 Thread Dave Angel
gb345 wrote: In "Martin v. Loewis" writes: Do I need to do something especial to get repr to work strictly with unicode? Yes, you need to switch to Python 3 :-) Or should __repr__ *always* return bytes rather than unicode? In Python 2.x: yes.

Re: UnicodeEncodeError during repr()

2010-04-19 Thread gb345
In "Martin v. Loewis" writes: >> Do I need to do something especial to get repr to work strictly >> with unicode? >Yes, you need to switch to Python 3 :-) >> Or should __repr__ *always* return bytes rather than unicode? >In Python 2.x: yes. >> What about __str__ ? >Likewise. >> If both of

Re: UnicodeEncodeError during repr()

2010-04-19 Thread Martin v. Loewis
> Do I need to do something especial to get repr to work strictly > with unicode? Yes, you need to switch to Python 3 :-) > Or should __repr__ *always* return bytes rather than unicode? In Python 2.x: yes. > What about __str__ ? Likewise. > If both of these are supposed to return bytes, > the

UnicodeEncodeError during repr()

2010-04-18 Thread gb345
I'm getting a UnicodeEncodeError during a call to repr: Traceback (most recent call last): File "bug.py", line 142, in element = parser.parse(INPUT) File "bug.py", line 136, in parse ps = Parser.Parse(open(filename,'r').read(), 1) File &q

Re: UnicodeEncodeError: 'ascii' codec can't encode character u'\xb7' in position 13: ordinal not in range(128)

2009-07-18 Thread akhil1988
> decoding. So, if you know your source is always utf-8, try > something like: > > import sys > import io > > sys.stdin = io.TextIOWrapper(sys.stdin.detach(), encoding='utf8') > > for line in sys.stdin: > line = lin

Re: UnicodeEncodeError: 'ascii' codec can't encode character u'\xb7' in position 13: ordinal not in range(128)

2009-07-18 Thread akhil1988
ding is UTF-8, but feeding > data which isn't valid UTF-8 to stdin. If you want to use data with a > different encoding, you need to replace sys.stdin, e.g.: > > import sys > import io > sys.stdin = io.TextIOWrapper(sys.stdin.detach(), encoding = 'iso-8859-1') &g

Re: UnicodeEncodeError: 'ascii' codec can't encode character u'\xb7' in position 13: ordinal not in range(128)

2009-07-17 Thread Scott David Daniels
akhil1988 wrote: > Nobody-38 wrote: On Thu, 16 Jul 2009 15:43:37 -0700, akhil1988 wrote: ... In Python 3 you can't decode strings because they are Unicode strings and it doesn't make sense to decode a Unicode string. You can only decode encoded things which are byte strings. So you are mixing

Re: UnicodeEncodeError: 'ascii' codec can't encode character u'\xb7' in position 13: ordinal not in range(128)

2009-07-17 Thread Piet van Oostrum
> akhil1988 (a) wrote: >a> Well, you were write: unintentionally I removed strip(). But the problem >does >a> not ends here: >a> I get this error now: >a> File "./temp.py", line 488, in >a> main() >a> File "./temp.py", line 475, in main >a> for line in sys.stdin: >a> File "/u

Re: UnicodeEncodeError: 'ascii' codec can't encode character u'\xb7' in position 13: ordinal not in range(128)

2009-07-17 Thread Nobody
On Thu, 16 Jul 2009 20:26:39 -0700, akhil1988 wrote: > Well, you were write: unintentionally I removed strip(). But the problem does > not ends here: > > I get this error now: > > File "./temp.py", line 488, in > main() > File "./temp.py", line 475, in main > for line in sys.stdin: >

Re: UnicodeEncodeError: 'ascii' codec can't encode character u'\xb7' in position 13: ordinal not in range(128)

2009-07-16 Thread akhil1988
e >> elsif #do something here >> >> If I remove the decode statement, line == '' never gets true. > > Did you inadvertently remove the strip() as well? > > -- > http://mail.python.org/mailman/listinfo/python-list > > -- View this

Re: UnicodeEncodeError: 'ascii' codec can't encode character u'\xb7' in position 13: ordinal not in range(128)

2009-07-16 Thread Nobody
On Thu, 16 Jul 2009 15:43:37 -0700, akhil1988 wrote: >> In Python 3 you can't decode strings because they are Unicode strings >> and it doesn't make sense to decode a Unicode string. You can only >> decode encoded things which are byte strings. So you are mixing up byte >> strings and Unicode stri

Re: UnicodeEncodeError: 'ascii' codec can't encode character u'\xb7' in position 13: ordinal not in range(128)

2009-07-16 Thread akhil1988
ncoded things which are byte strings. So you are mixing up byte > strings and Unicode strings. > -- > Piet van Oostrum > URL: http://pietvanoostrum.com [PGP 8DAE142BE17999C4] > Private email: p...@vanoostrum.org > -- > http://mail.python.org/mailman/listinfo/python-list &g

Re: UnicodeEncodeError: 'ascii' codec can't encode character u'\xb7' in position 13: ordinal not in range(128)

2009-07-16 Thread Piet van Oostrum
> akhil1988 (a) wrote: >a> ok! >a> I got the indentation errors fixed. Bu I get another error: >a> Traceback (most recent call last): >a> File "./temp.py", line 484, in >a> main() >a> File "./temp.py", line 476, in main >a> line.decode('utf-8').strip() >a> AttributeError: 'str'

Re: UnicodeEncodeError: 'ascii' codec can't encode character u'\xb7' in position 13: ordinal not in range(128)

2009-07-16 Thread akhil1988
cally >> converting Python 2.x code to 3.x, called 2to3. One of the things it >> should do is convert Py2.x unicode values into their correct >> representation in 3.x. >> >> With any luck, it should be able to convert the code you're using >> entirely. Let us kn

Re: UnicodeEncodeError: 'ascii' codec can't encode character u'\xb7' in position 13: ordinal not in range(128)

2009-07-16 Thread akhil1988
code you're using > entirely. Let us know how it goes. > -- > http://mail.python.org/mailman/listinfo/python-list > > -- View this message in context: http://www.nabble.com/UnicodeEncodeError%3A-%27ascii%27-codec-can%27t-encode-character-u%27%5Cxb7%27-in-position-13%3A-o

Re: UnicodeEncodeError: 'ascii' codec can't encode character u'\xb7' in position 13: ordinal not in range(128)

2009-07-16 Thread John Machin
On Jul 16, 9:04 pm, akhil1988 wrote: > Please click reply on the post and then read this reply in the editor. > Actually, some sequences have been replaced to their graphical form when > this post is published. So the python code is being displayed, what actually > it is not. What editor? I guess

Re: UnicodeEncodeError: 'ascii' codec can't encode character u'\xb7' in position 13: ordinal not in range(128)

2009-07-16 Thread Max Erickson
akhil1988 wrote: > > akhil1988 wrote: >> >> I have switched to python 3.1 , but now I am getting some syntax >> errors in the code: >> >> File "./customWikiExtractor.py", line 81 >> __char_entities = {' ' :u'\u00A0', '¡' >> :u'\u00A1', >> '¢':u'\u00A2', >>

Re: UnicodeEncodeError: 'ascii' codec can't encode character u'\xb7' in position 13: ordinal not in range(128)

2009-07-16 Thread alex23
On Jul 16, 9:00 pm, akhil1988 wrote: > I have switched to python 3.1 , but now I am getting some syntax errors in > the code: Python 3.x was a major release that endeavoured to clean up a number of lingering issues with the language, the upshot being that it isn't entirely backwards compatible wi

Re: UnicodeEncodeError: 'ascii' codec can't encode character u'\xb7' in position 13: ordinal not in range(128)

2009-07-16 Thread akhil1988
00B6', > '·' :u'\u00B7', > '¸' :u'\u00B8', '¹' :u'\u00B9', > 'º':u'\u00BA',} > > --Akhil > > > John Nagle-2 wrote: >> >> akhil1988 wrote

Re: UnicodeEncodeError: 'ascii' codec can't encode character u'\xb7' in position 13: ordinal not in range(128)

2009-07-16 Thread akhil1988
u'\u00B8', '¹' :u'\u00B9', 'º':u'\u00BA',} --Akhil John Nagle-2 wrote: > > akhil1988 wrote: >> Sorry, it is sgmllib.py and not sgmmlib.py > > Oh, that bug again. See > > http://bugs.python.org/issue1651995

Re: UnicodeEncodeError: 'ascii' codec can't encode character u'\xb7' in position 13: ordinal not in range(128)

2009-07-16 Thread Piet van Oostrum
> akhil1988 (a) wrote: >a> Chris, >a> Using >a> print (u'line: %s' % line).encode('utf-8') >a> the 'line' gets printed, but actually this print statement I was using just >a> for testing, actually my code operates on 'line', on which I use line = >a> line.decode('utf-8') as 'line' is read

Re: UnicodeEncodeError: 'ascii' codec can't encode character u'\xb7' in position 13: ordinal not in range(128)

2009-07-16 Thread John Nagle
akhil1988 wrote: Sorry, it is sgmllib.py and not sgmmlib.py Oh, that bug again. See http://bugs.python.org/issue1651995 It's a bug in SGMLParser. When Python 2.5 restricted ASCII to 0..127, SGMLParser needed to be modified, but wasn't. I reported that bug in February 2007. It w

  1   2   >