Re: can't get utf8 / unicode strings from embedded python

2013-08-28 Thread David M. Cotter
Thank you for your thoughtful and thorough response. I now understand much better what you (and apparently the others) were warning me against and I will certainly consider that moving forward. I very much appreciate your help as I learn about python and embedding and all these crazy encoding

Re: can't get utf8 / unicode strings from embedded python

2013-08-28 Thread Steven D'Aprano
On Tue, 27 Aug 2013 22:57:45 -0700, David M. Cotter wrote: > I am very sorry that I have offended you to such a degree you feel it > necessary to publicly eviscerate me. You know David, you are right. I did over-react. And I apologise for that. I am sorry, I was excessively confrontational. (Alt

Re: can't get utf8 / unicode strings from embedded python

2013-08-27 Thread David M. Cotter
I am very sorry that I have offended you to such a degree you feel it necessary to publicly eviscerate me. Perhaps I could have worded it like this: "So far I have not seen any troubles including unicode characters in my strings, they *seem* to be fine for my use-case. What kind of trouble ha

Re: can't get utf8 / unicode strings from embedded python

2013-08-27 Thread Steven D'Aprano
On Tue, 27 Aug 2013 15:21:00 -0700, David M. Cotter wrote: > i am already doing (3), and all is working perfectly. bytestring > literals are fine, i'm not sure what this trouble is that you speak of. Neither is anyone else, because your post is completely devoid of any context. Who are you talk

Re: can't get utf8 / unicode strings from embedded python

2013-08-27 Thread David M. Cotter
i am already doing (3), and all is working perfectly. bytestring literals are fine, i'm not sure what this trouble is that you speak of. note that i'm not using PyRun_AnyFile(), i'm loading the script myself, assumed as utf8 (which was my original problem, i had assumed it was macRoman), then

Re: can't get utf8 / unicode strings from embedded python

2013-08-25 Thread MRAB
On 25/08/2013 23:32, David M. Cotter wrote: i got it!! OMG! so sorry for the confusion, but i learned a lot, and i can share the result: the CORRECT code *was* what i had assumed. the Python side has always been correct (no need to put "u" in front of strings, it is known that the bytes are u

Re: can't get utf8 / unicode strings from embedded python

2013-08-25 Thread David M. Cotter
i got it!! OMG! so sorry for the confusion, but i learned a lot, and i can share the result: the CORRECT code *was* what i had assumed. the Python side has always been correct (no need to put "u" in front of strings, it is known that the bytes are utf8 bytes) it was my "run script" function

Re: can't get utf8 / unicode strings from embedded python

2013-08-25 Thread David M. Cotter
fair enough. I can provide further proof of strangeness. here is my latest script: this is saved on disk as a UTF8 encoded file, and when viewing as UTF8, it shows the correct characters. == # -*- coding: utf-8 -*- import time, kjams, kjams_lib def log_success(msg, successB, s

Re: can't get utf8 / unicode strings from embedded python

2013-08-25 Thread Terry Reedy
On 8/25/2013 1:57 PM, David M. Cotter wrote: i'm sorry this is so confusing, let me try to re-state the problem in as clear a way as i can. I have a C++ program, with very well tested unicode support. All logging is done in utf8. I have conversion routines that work flawlessly, so i can assu

Re: can't get utf8 / unicode strings from embedded python

2013-08-25 Thread Vlastimil Brom
2013/8/25 David M. Cotter : > i'm sorry this is so confusing, let me try to re-state the problem in as > clear a way as i can. > > I have a C++ program, with very well tested unicode support. All logging is > done in utf8. I have conversion routines that work flawlessly, so i can > assure you

Re: can't get utf8 / unicode strings from embedded python

2013-08-25 Thread David M. Cotter
i'm sorry this is so confusing, let me try to re-state the problem in as clear a way as i can. I have a C++ program, with very well tested unicode support. All logging is done in utf8. I have conversion routines that work flawlessly, so i can assure you there is nothing wrong with logging and

Re: can't get utf8 / unicode strings from embedded python

2013-08-24 Thread random832
On Sat, Aug 24, 2013, at 12:47, David M. Cotter wrote: > > What _are_ you using? > i have scripts in a file, that i am invoking into my embedded python > within a C++ program. there is no terminal involved. the "print" > statement has been redirected (via sys.stdout) to my custom print class, >

Re: can't get utf8 / unicode strings from embedded python

2013-08-24 Thread Benjamin Kaplan
On Sat, Aug 24, 2013 at 9:47 AM, David M. Cotter wrote: > > > What _are_ you using? > i have scripts in a file, that i am invoking into my embedded python within a > C++ program. there is no terminal involved. the "print" statement has been > redirected (via sys.stdout) to my custom print clas

Re: can't get utf8 / unicode strings from embedded python

2013-08-24 Thread wxjmfauth
Le samedi 24 août 2013 18:47:19 UTC+2, David M. Cotter a écrit : > > What _are_ you using? > > i have scripts in a file, that i am invoking into my embedded python within a > C++ program. there is no terminal involved. the "print" statement has been > redirected (via sys.stdout) to my custom

Re: can't get utf8 / unicode strings from embedded python

2013-08-24 Thread David M. Cotter
> What _are_ you using? i have scripts in a file, that i am invoking into my embedded python within a C++ program. there is no terminal involved. the "print" statement has been redirected (via sys.stdout) to my custom print class, which does not specify "encoding", so i tried the suggestion a

Re: can't get utf8 / unicode strings from embedded python

2013-08-24 Thread random832
On Sat, Aug 24, 2013, at 2:45, David M. Cotter wrote: > > you need to use u" ... " delimiters for Unicode, otherwise the results you > > get are completely arbitrary and depend on the encoding of your terminal. > okay, well, i'm on a mac, and not using "terminal" at all. but if i > were, it woul

Re: can't get utf8 / unicode strings from embedded python

2013-08-24 Thread Dave Angel
David M. Cotter wrote: > Steven wrote: >> I see you are using Python 2 > correct > >>It's hard to say what *exactly* is happening here, because you don't explain >>how the python print statement somehow gets into your C++ Log code. Do I >>guess right that it catches stdout? > yes, i'm redirecting

Re: can't get utf8 / unicode strings from embedded python

2013-08-23 Thread David M. Cotter
> I see you are using Python 2 correct > Firstly, in Python 2, the compiler assumes that the source code is encoded in > ASCII gar, i must have been looking at doc for v3, as i thought it was all assumed to be utf8 > # -*- coding: utf-8 -*- okay, did that, still no change > you need to use u"

Re: can't get utf8 / unicode strings from embedded python

2013-08-23 Thread Steven D'Aprano
On Fri, 23 Aug 2013 13:49:23 -0700, David M. Cotter wrote: > note everything works great if i use Ascii, but: > > in my utf8-encoded script i have this: > >> print "frøânçïé" I see you are using Python 2, in which case there are probably two or three errors being made here. Firstly, in

can't get utf8 / unicode strings from embedded python

2013-08-23 Thread David M. Cotter
note everything works great if i use Ascii, but: in my utf8-encoded script i have this: > print "frøânçïé" in my embedded C++ i have this: PyObject* CPython_Script::print(PyObject *args) { PyObject*resultObjP = NULL; const char *utf8_