[issue12266] str.capitalize contradicts oneself

Roundup Robot added the comment: New changeset d3816fa1bcdf by Ezio Melotti in branch '2.7': #12266: move the tests in test_unicode. http://hg.python.org/cpython/rev/d3816fa1bcdf -- ___ Python tracker

[issue12711] Explain tracker components in devguide

Ezio Melotti added the comment: Fixed in http://hg.python.org/devguide/rev/c9dd231b0940 -- resolution: -> fixed stage: patch review -> committed/rejected status: open -> closed ___ Python tracker

[issue12746] normalization is affected by unicode width

STINNER Victor added the comment: See also #12737. -- ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: http://m

[issue12737] str.title() is overzealous by upcasing combining marks inappropriately

STINNER Victor added the comment: See also #12746. -- ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: http://m

[issue12746] normalization is affected by unicode width

Changes by Tom Christiansen : -- nosy: +tchrist ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.pyt

[issue12729] Python lib re cannot handle Unicode properly due to narrow/wide bug

Marc-Andre Lemburg added the comment: > Keep in mind that we should be able to access and use lone surrogates too, > therefore: > s = '\ud800' # should be valid > len(s) # should this raise an error? (or return 0.5 ;)? > s[0] # error here too? > list(s) # here too? > > p = s + '\udc00' > l

[issue12751] Use macros for surrogates in unicodeobject.c

New submission from STINNER Victor : A lot of code is duplicated in unicodeobject.c to manipulate ("encode/decode") surrogates. Each function has from one to three different implementations. The new decode_ucs4() function adds a new implementation. Attached patch replaces this code by macros.

[issue12751] Use macros for surrogates in unicodeobject.c

STINNER Victor added the comment: We may use the following unlikely macro for IS_SURROGATE, IS_HIGH_SURROGATE and IS_LOW_SURROGATE: #define likely(x) __builtin_expect(!!(x), 1) #define unlikely(x) __builtin_expect(!!(x), 0) I suppose that we should use microbenchmarks to validate th

[issue12737] str.title() is overzealous by upcasing combining marks inappropriately

Ezio Melotti added the comment: So the issue here is that while using combing chars, str.title() fails to titlecase the string properly. The algorithm implemented by str.title() [0] is quite simple: it loops through the code units, and uppercases all the chars that follow a char that is not

[issue12751] Use macros for surrogates in unicodeobject.c

Ezio Melotti added the comment: This has been proposed already in #10542 (the issue also has patches). -- ___ Python tracker ___ ___

[issue12731] python lib re uses obsolete sense of \w in full violation of UTS#18 RL1.2a

Ezio Melotti added the comment: If the regex module works fine here, I think it's better to leave the re module alone and include the regex module in 3.3. -- ___ Python tracker ___

[issue12734] Request for property support in Python re lib

Ezio Melotti added the comment: This indeed should be "fixed" by replacing 're' with 'regex'. So I would suggest to focus your tests on 'regex' and report them there so that possible bugs gets fixed and tested before we include the module in the stdlib. -- __

[issue12733] Request for grapheme support in Python re lib

Ezio Melotti added the comment: As I said on #12734 and #12731, if the 'regex' module address this issue, we should just wait until we include it in the stdlib. -- ___ Python tracker _

[issue12730] Python's casemapping functions are untrustworthy due to narrow/wide build issues

Ezio Melotti added the comment: This is actually a duplicated of #9200. @Terry > Besides which, all I see (on Windowsj) in Firefox is things like > "𐐼𐐯𐑅𐐨𐑉𐐯𐐻". Encoding problem. Firefox thinks this is some iso-8859-*. You can fix this selecting 'Unicode (UTF-8)' from "V

[issue9200] Make str methods work with non-BMP chars on narrow builds

Ezio Melotti added the comment: I closed #12730 as a duplicate of this and updated the title of this issue. -- title: str.isprintable() is always False for large code points -> Make str methods work with non-BMP chars on narrow builds ___ Python tra

[issue10542] Py_UNICODE_NEXT and other macros for surrogates

Ezio Melotti added the comment: See also #12751. -- nosy: +tchrist ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscri

[issue9200] Make str methods work with non-BMP chars on narrow builds

Changes by Ezio Melotti : -- nosy: +tchrist ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.o

[issue12752] locale.normalize does not take unicode strings

New submission from Julian Taylor : using unicode strings for locale.normalize gives following traceback with python2.7: ~$ python2.7 -c 'import locale; locale.normalize(u"en_US")' Traceback (most recent call last): File "", line 1, in File "/usr/lib/python2.7/locale.py", line 358, in norm

[issue12752] locale.normalize does not take unicode strings

Changes by Ezio Melotti : -- nosy: +ezio.melotti stage: -> test needed ___ Python tracker ___ ___ Python-bugs-list mailing list Unsub

[issue12204] str.upper converts to title

Roundup Robot added the comment: New changeset 16edc5cf4a79 by Ezio Melotti in branch '3.2': #12204: document that str.upper().isupper() might be False and add a note about cased characters. http://hg.python.org/cpython/rev/16edc5cf4a79 New changeset fb49394f75ed by Ezio Melotti in branch '2.7

[issue12204] str.upper converts to title

Ezio Melotti added the comment: Fixed, thanks for the report! -- resolution: -> fixed stage: commit review -> committed/rejected status: open -> closed ___ Python tracker ___ _

[issue12729] Python lib re cannot handle Unicode properly due to narrow/wide bug

Matthew Barnett added the comment: For what it's worth, I've had idea about string storage, roughly based on how *nix stores data on disk. If a string is small, point to a block of codepoints. If a string is medium-sized, point to a block of pointers to codepoint blocks. If a string is large

[issue12752] locale.normalize does not take unicode strings

Julian Taylor added the comment: this is a regression introduced by fixing http://bugs.python.org/issue1813 This breaks some user code,. e.g. wx.Locale.GetCanonicalName returns unicode. Example bugs: https://bugs.launchpad.net/ubuntu/+source/update-manager/+bug/824734 https://bugs.launchpad.net

[issue12752] locale.normalize does not take unicode strings

Marc-Andre Lemburg added the comment: Julian Taylor wrote: > > New submission from Julian Taylor : > > using unicode strings for locale.normalize gives following traceback with > python2.7: > > ~$ python2.7 -c 'import locale; locale.normalize(u"en_US")' > Traceback (most recent call last): >

[issue12204] str.upper converts to title

Raymond Hettinger added the comment: Are you sure this should have been backported? Are there any apps that may be working now but won't be after the next point release? -- nosy: +rhettinger ___ Python tracker _

[issue12751] Use macros for surrogates in unicodeobject.c

Antoine Pitrou added the comment: > HIGH_SURROGATE and LOW_SURROGATE require that their ordinal argument > has been preproceed to fit in [0; 0x]. I added this requirement in > the comment of these macros. The macros should preprocess the argument themselves. It will make the code even simpl

[issue12204] str.upper converts to title

Ezio Melotti added the comment: This is only a doc patch, maybe you are confusing this issue with #12266? -- ___ Python tracker ___ _

[issue12204] str.upper converts to title

Raymond Hettinger added the comment: Right. I was looking at the other patches that went in in the last 24 hours. -- ___ Python tracker ___

[issue12204] str.upper converts to title

Ezio Melotti added the comment: It's unlikely that #12266 might break apps. The behavior changed only for fairly unusual characters, and the old behavior was clearly wrong. FWIW the str.capitalize() implementation of PyPy doesn't have the bug, and after the fix both CPython and PyPy have the

[issue12750] datetime.datetime timezone problems

R. David Murray added the comment: In what way does 'replace' not satisfy your need to set the tzinfo? As for utcnow, we can't change what it returns for backward compatibility reasons, but you can get a non-naive utc datatime by doing datetime.now(timezone.utc). (I must admit, however, that

[issue12750] datetime.datetime timezone problems

Daniel O'Connor added the comment: On 15/08/2011, at 23:39, R. David Murray wrote: > R. David Murray added the comment: > > In what way does 'replace' not satisfy your need to set the tzinfo? Ahh that would work, although it is pretty clumsy since you have to specify everything else as well.

[issue12750] datetime.datetime timezone problems

R. David Murray added the comment: Ah. Well, pre-3.2 datetime itself did not generate *any* non-naive datetimes. Nor do you need to specify everything for replace. dt.replace(tzinfo=tz) should work just fine. -- resolution: -> invalid stage: -> committed/rejected status: open -> c

[issue8271] str.decode('utf8', 'replace') -- conformance with Unicode 5.2.0

Ezio Melotti added the comment: Here are some benchmarks: Commands: # half of the bytes are invalid ./python -m timeit -s 'b = bytes(range(256)); b_dec = b.decode' 'b_dec("utf-8", "surrogateescape")' ./python -m timeit -s 'b = bytes(range(256)); b_dec = b.decode' 'b_dec("utf-8", "replace")' ./

[issue10542] Py_UNICODE_NEXT and other macros for surrogates

Martin v. Löwis added the comment: A PEP 393 draft implementation is available at https://bitbucket.org/t0rsten/pep-393/ (branch pep-393); if this gets into 3.3, this issue will be outdated: there won't be "narrow" builds of Python anymore (nor will there be "wide" builds). -- _

[issue10542] Py_UNICODE_NEXT and other macros for surrogates

Ezio Melotti added the comment: That's a really good news. Some Unicode issues can still be fixed on 2.7 and 3.2 though. FWIW I was planning to look at this and #9200 in the following days and see if I can fix them. -- ___ Python tracker

[issue12730] Python's casemapping functions are untrustworthy due to narrow/wide build issues

Terry J. Reedy added the comment: My Firefox is already set at utf-8. More likely a font limitation. I will look again after installing one of the fonts Tom suggested. The pair of boxes on IDLE are for the surrogate pairs. Perhaps tk does not even try to display a single char. I will experime

[issue12753] \N{...} neglects formal aliases and named sequences from Unicode charnames namespace

New submission from Tom Christiansen : Unicode character names share a common namespace with formal aliases and with named sequences, but Python recognizes only the original name. That means not everything in the namespace is accessible from Python. (If this is construed to be an extant bug f

[issue12730] Python's casemapping functions are untrustworthy due to narrow/wide build issues

Ezio Melotti added the comment: > My Firefox is already set at utf-8. Every page can specify the encoding it uses (in HTTP headers, tag and/or xml prologue). If none of these are specified, afaik Firefox tries to detect the encoding, and sometimes fails. What encoding does it show for you

[issue12730] Python's casemapping functions are untrustworthy due to narrow/wide build issues

Tom Christiansen added the comment: >Terry J. Reedy added the comment: > My Firefox is already set at utf-8. More likely a font limitation. I > will look again after installing one of the fonts Tom suggested. Symbola is best for exotic glyphs, especially astral ones. Alfios just looks nice a

[issue12753] \N{...} neglects formal aliases and named sequences from Unicode charnames namespace

Changes by Ezio Melotti : -- components: +Unicode nosy: +ezio.melotti stage: -> test needed versions: -Python 2.7, Python 3.1, Python 3.2 ___ Python tracker ___ ___

[issue12730] Python's casemapping functions are untrustworthy due to narrow/wide build issues

Terry J. Reedy added the comment: You are right, FF switched on me without notice. Bad FF. Thank you! What I now see makes much more sense. [ "𐐼𐐯𐑅𐐨𐑉𐐯𐐻", "𐐼𐐯𐑅𐐨𐑉𐐯𐐻", "𐐔𐐯𐑅𐐨𐑉𐐯𐐻", "𐐔𐐇𐐝𐐀𐐡𐐇𐐓" ], and I now know to check on other pages (although Tom's Unicode talk slides still have boxes even in utf-8,

[issue12730] Python's casemapping functions are untrustworthy due to narrow/wide build issues

Tom Christiansen added the comment: >Terry J. Reedy added the comment: > You are right, FF switched on me without notice. Bad FF. Thank you! What > I now see makes much more sense. >[ "𐐼𐐯𐑅𐐨𐑉𐐯𐐻", "𐐼𐐯𐑅𐐨𐑉𐐯𐐻", "𐐔𐐯𐑅𐐨𐑉𐐯𐐻", "𐐔𐐇𐐝𐐀𐐡𐐇𐐓" ], > and I now know to check on other pages (although Tom's

[issue12734] Request for property support in Python re lib

Tom Christiansen added the comment: Sorry I didn't include a test case. Hope this makes up for it. If not, please tell me how to write better test cases. :( Yeah ok, so I'm a bit persnickety or even unorthodox about my vertical alignment, but it really helps to make what is different from on

[issue12734] Request for property support in Python re lib

Tom Christiansen added the comment: Oh whoops, that was the long ticket. Shall I reupload to the right number? -- ___ Python tracker ___ ___

[issue12730] Python's casemapping functions are untrustworthy due to narrow/wide build issues

Terry J. Reedy added the comment: Adding Symbola filled in the symbols and emoticons lines. The gothic chars are still missing even with Alfios. -- ___ Python tracker ___ __

[issue12730] Python's casemapping functions are untrustworthy due to narrow/wide build issues

Tom Christiansen added the comment: >Terry J. Reedy added the comment: >Adding Symbola filled in the symbols and emoticons lines. >The gothic chars are still missing even with Alfios. That's too bad, as the Gothic paternoster is kinda cute. :) Hm, I wonder where I got them from. I think the

[issue12746] normalization is affected by unicode width

Changes by Arfrever Frehtes Taifersar Arahesis : -- nosy: +Arfrever ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscri

[issue9200] Make str methods work with non-BMP chars on narrow builds

Changes by Arfrever Frehtes Taifersar Arahesis : -- nosy: +Arfrever ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscrib

[issue12753] \N{...} neglects formal aliases and named sequences from Unicode charnames namespace

Tom Christiansen added the comment: Here’s the right test file for the right ticket. -- Added file: http://bugs.python.org/file22903/nametests.py ___ Python tracker ___

[issue12734] Request for property support in Python re lib

Changes by Tom Christiansen : Removed file: http://bugs.python.org/file22902/nametests.py ___ Python tracker ___ ___ Python-bugs-list mailing

[issue12752] locale.normalize does not take unicode strings

Barry A. Warsaw added the comment: A cheap way of fixing this would be to test for str-ness of localename and if it's a unicode, just localname.encode('ascii') Or is that completely insane? -- nosy: +barry ___ Python tracker

[issue12752] locale.normalize does not take unicode strings

Barry A. Warsaw added the comment: For example: diff -r fb49394f75ed Lib/locale.py --- a/Lib/locale.py Mon Aug 15 14:24:15 2011 +0300 +++ b/Lib/locale.py Mon Aug 15 16:47:23 2011 -0400 @@ -355,6 +355,8 @@ """ # Normalize the locale name and extract the encoding +if isin

[issue12752] locale.normalize does not take unicode strings

Changes by Barry A. Warsaw : -- keywords: +patch Added file: http://bugs.python.org/file22904/issue12752.diff ___ Python tracker ___ _

[issue12751] Use macros for surrogates in unicodeobject.c

Changes by STINNER Victor : -- nosy: +belopolsky ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.py

[issue12751] Use macros for surrogates in unicodeobject.c

STINNER Victor added the comment: > This has been proposed already in #10542 (the issue also has patches). The two issues are different: this issue is only a refactoring, whereas #10542 adds a new "feature" (function/macro: Py_UNICODE_NEXT). -- ___

[issue12752] locale.normalize does not take unicode strings

Antoine Pitrou added the comment: The proposed resolution looks ok. Another possibility is simply to use .lower() if the string is an unicode string, since that will bypass the C locale. -- nosy: +pitrou stage: test needed -> patch review ___ Python

[issue12750] datetime.datetime timezone problems

Daniel O'Connor added the comment: On 16/08/2011, at 1:06, R. David Murray wrote: > R. David Murray added the comment: > > Ah. Well, pre-3.2 datetime itself did not generate *any* non-naive datetimes. > > Nor do you need to specify everything for replace. dt.replace(tzinfo=tz) > should wor

[issue12754] Add alternative random number generators

New submission from Raymond Hettinger : While keeping the MT generator as the default, add new alternative random number generators as drop-in replacements. Since MT was first introduced, PRNG technology has continued to advance. I'm opening this feature request to be a centralized place to d

[issue12752] locale.normalize does not take unicode strings

Changes by Barry A. Warsaw : -- assignee: -> barry ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail

[issue12752] locale.normalize does not take unicode strings

Roundup Robot added the comment: New changeset 0d64fe6c737f by Barry Warsaw in branch '2.7': The simplest possible fix for the regression in bug 12752 by encoding unicodes http://hg.python.org/cpython/rev/0d64fe6c737f -- nosy: +python-dev ___ Python

[issue12752] locale.normalize does not take unicode strings

Changes by Barry A. Warsaw : -- resolution: -> fixed status: open -> closed ___ Python tracker ___ ___ Python-bugs-list mailing list

[issue12748] Problems using IDLE accelerators with OS X Dvorak - Qwerty ⌘ input method

Ned Deily added the comment: Interesting, I didn't know the "Dvorak - Qwerty ⌘" input method existed. In just some causal experimentation with it, it seems pretty clear that the input method is not being consistently followed by Tk and there seem to be differences between Tk 8.4 and 8.5. Pa

[issue12754] Add alternative random number generators

Sturla Molden added the comment: George Marsaglia's latest random number generator KISS4691 is worth considering, though I am not sure the performance is that different from MT19937. Here is a link to Marsaglia's post on comp.lang.c. Marasglia passed away shortly after (Feb. 2011), and to m

[issue12754] Add alternative random number generators

Sturla Molden added the comment: I'm posting the code for comparison of KISS4691 and MT19937. I do realize KISS4691 might not be sufficiently different from MT19937 in characteristics for Raymond Hettinger to consider it. But at least here it is for reference should it be of value. -

[issue12754] Add alternative random number generators

Sturla Molden added the comment: Another (bug fix) post by Marsaglia on KISS4691: http://www.phwinfo.com/forum/comp-lang-c/460292-ensuring-long-period-kiss4691-rng.html -- ___ Python tracker _

[issue12754] Add alternative random number generators

Changes by Sturla Molden : Removed file: http://bugs.python.org/file22905/prngtest.zip ___ Python tracker ___ ___ Python-bugs-list mailing lis

[issue12754] Add alternative random number generators

Changes by Sturla Molden : Added file: http://bugs.python.org/file22906/prngtest.zip ___ Python tracker ___ ___ Python-bugs-list mailing list

[issue12740] Add struct.Struct.nmemb

Meador Inge added the comment: On Sun, Aug 14, 2011 at 1:03 PM, Stefan Krah wrote: > > Stefan Krah added the comment: > > I like random tests in the stdlib, otherwise the same thing gets tested > over and over again. `make buildbottest` prints the seed, and you can do > it for a single test as

[issue12672] Some problems in documentation extending/newtypes.html

Eli Bendersky added the comment: Terry, I'm not 100% sure about what you mean by "Python wrapper objects ... visible from Python", but I think I'll disagree. There's a big difference between "C functions" in general and "type methods" this document speaks of. Let's leave list aside for a mome

[issue12754] Add alternative random number generators

Sturla Molden added the comment: Further suggestions to improve the random module: ** Object-oriented PRNG: Let it be an object which stores the random state internally, so we can create independent PRNG objects. I.e. not just one global generator. ** Generator for quasi-random Sobol sequenc

[issue12754] Add alternative random number generators

Raymond Hettinger added the comment: Please focus your thoughts. -- ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscr

[issue12672] Some problems in documentation extending/newtypes.html

Terry J. Reedy added the comment: "the type object determines which (C) functions get called when, for instance, an attribute gets looked up on an object or it is multiplied by another object. These C functions are called “type methods” "These C functions" are any of the C functions that are

[issue12672] Some problems in documentation extending/newtypes.html

Eli Bendersky added the comment: "[].append is a Python-level method object that wraps a C function." What makes you think that? There's no Python implementation of .append that I know of. Neither is there a Python implementation of the Noddy.name method that is discussed in the page. Both ar

[issue12755] Service application crash in python25!PyObject_Malloc

New submission from Chandra Sekhar Reddy : Service application crashed in python25.dll, below are the environment details. Operating System : Windows server 2008 R2 (Virtual Machine) Application Type : Service Application FAULTING_IP: python25!PyObject_Malloc+2d 1e09603d 8b30mov

[issue12750] datetime.datetime timezone problems

Changes by Ben Finney : -- nosy: +bignose ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.or

[issue12756] datetime.datetime.utcnow should return a UTC timestamp

New submission from Ben Finney : = $ date -u +'%F %T %s %z' 2011-08-16 06:42:12 1313476932 + $ python -c 'import sys, datetime; now = datetime.datetime.utcnow(); sys.stdout.write(now.strftime("%F %T %s %z"))' 2011-08-16 06:42:12 1313440932 = The documentation for ‘datetime.datetim