date:20130811

On 11 August 2013 08:02, sagar varule  wrote:
> On Sunday, August 11, 2013 11:28:31 AM UTC+5:30, Joshua Landau wrote:
>> You also didn't say what didn't work with the first block of code.
>
> Submitting Command to Interactive Shell through code did not work.

In what way didn't it work? What's the problem?

Also, what is "Interactive Shell"?
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Python Basic Doubt

On Sat, 10 Aug 2013 17:42:21 -0700, Gary Herron wrote:

> But for each of your examples, using "==" is equivalent to using "is".
> Each of
>  if something == None
>  if device == _not passed
>  if device != None
> would all work as expected.  In none of those cases is "is" actually
> needed.

py> class Weird:
... def __eq__(self, other):
... return True
...
py> definitely_not_None = Weird()
py> definitely_not_None == None
True

Oops.

Now Weird is, naturally, weird. But classes may have a legitimate reason 
to claim to be equal to None, or at least the creator of the class may 
believe he has a legitimate reason. It's not forbidden, therefore you 
have to assume somebody will do so.

But more importantly, in checking whether some value is a sentinel, the 
*intention* of the code is to check that the value *actually is* the 
sentinel, not merely that it happens to be equal to the sentinel. To give 
an analogy: when you hand the codes to the nuclear arsenal to the 
President, you want to make sure that it actually *is* the President, not 
some impostor who merely looks and sounds like him.

"value == None" not only risks doing the wrong thing if passed some weird 
object, but it fails to match the intention of the code. Whereas "value 
is None" is idiot-proof (it cannot fail -- short of hacking the compiler, 
you cannot change the meaning of either "is" or "None"), it matches the 
intention of the code, and therefore is clearer, more explicit, and more 
idiomatic. And as a bonus, it's faster too. But speed is not why you 
should use it. You should use it because it matches the intention of the 
code. You don't want to test for some arbitrary person who happens to 
look like the President, you want to test for the President himself, and 
nobody but the President.

On the other hand, code like "x is 42.9" does not match the intention of 
the code, and is likely to fail. "is" is not a synonym for "==". 
Conflating them is a bad thing, whether you use "is" to check for 
equality or "==" to check for identity.

> Given that, and the implementation dependent oddities, I still believe
> that it is *highly* misleading to teach a beginner about "is".

Careful -- it's not "is" which is implementation-dependent. The "is" 
operator works the same way in every Python implementation. What differs 
are the rules for when new instances are created from literals.

> Here's a challenge:  What is the simplest non-contrived example where an
> "is" comparison is *required*.  Where substitution of an "==" comparison
> would cause the program to fail or be significantly less efficient? (I'm
> not including the nearly immeasurably small timing difference between
> v==None and v is None.)

Easy-peasey. Test code often relies on "is" to check object identity. 
Occasionally such comparisons work there way into production code, but 
they're common in tests.

In one of my test suites, I have code that verifies that a certain 
function which takes a list as argument does not modify it in place. So 
my unit test looks like this:

def testNoInPlaceModifications(self):
# Test that the function does not modify its input data.
data = self.prepare_data()
assert len(data) != 1  # Necessary to avoid infinite loop.
assert data != sorted(data)
saved = data[:]
assert data is not saved  #  <<<=== LOOK HERE ===
_ = self.func(data)
self.assertListEqual(data, saved, "data has been modified")

Note that is would be *wrong* to replace the "is not" comparison with not 
equal to. The list and its copy are, in fact, equal. If they aren't 
equal, the test fails.

In this case, the assertions are there as verifiable comments. They 
communicate to the reader, "These are the assumptions this test relies 
on", except unlike comments, if the assumptions are violated, the test 
will fail. Unlike comments, they cannot ever get out of date.

-- 
Steven
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Python Basic Doubt

On Sat, 10 Aug 2013 20:21:46 -0700, Gary Herron wrote:

> Our knee-jerk reaction to beginners using "is" should be:
>  Don't do that!  You almost certainly want "==".   Consider "is" an
> advanced topic.
> 
> Then you can spend as much time as you want trying to coach them into an
> understanding of the precise details.  But until they have that
> understanding, they are well served by a rule-of-thumb that says:
>  Use "==" not "is" for comparisons.

"...except for comparing to None, where 99.99% of the time you do 
actually want an identity comparison."

This can lead into a more detailed explanation for why you should choose 
one over the other, or the incurious newbie could take it is something to 
be learned by rote. I have no problem with telling newbies that there is 
a reason for this apparently arbitrary rule, but they don't need to learn 
it *right now* if they don't want.

In any case, the rule can include "When in doubt, use equals". I'm good 
with that :-)

-- 
Steven
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Elegant compare

On Sat, 10 Aug 2013 21:41:00 -0600, Jason Friedman wrote:

> class my_class:
> def __init__(self, attr1, attr2):
> self.attr1 = attr1 #string
> self.attr2 = attr2 #string
> def __lt__(self, other):
> if self.attr1 < other.attr1:
> return True
> else:
> return self.attr2 < other.attr2
> 
> I will run into problems if attr1 or attr2 is None, and they
> legitimately can be.
> 
> I know I can check for attr1 or attr2 or both being None and react
> accordingly, but my real class has ten attributes and that approach will
> be long.  What are my alternatives?

This is a hard question to answer, because your code snippet isn't 
clearly extensible to the case where you have ten attributes. What's the 
rule for combining them? If instance A has five attributes less than 
those of instance B, and five attributes greater than those of instance 
B, which wins?

But if I had to guess an approach, I'd start with a helper function (or 
method) that compares two raw values:

def compare(a, b):
"""Return -ve for less than, 0 for equal, +ve for greater than."""
if a is None:
 return 0 if b is None else -1
if b is None:
 return 1
return (b < a) - (a < b)

Now, in your class, you can use this helper function to check each 
attribute in turn. Assuming that if an attribute is equal, you move on to 
check the next one:

class MyClass:
def _compare(self, other):
for attr in 'attr1 attr2 attr3 attr4'.split():
a, b = getattr(self, attr), getattr(other, attr)
triflag = compare(a, b)
if triflag:
return triflag
return 0
def __lt__(self, other):
if not isinstance(other, MyClass):
return NotImplemented
return self._compare(other) < 0
def __eq__(self, other):
if not isinstance(other, MyClass):
return NotImplemented
return not self._compare(other)
def __ne__(self, other):
if not isinstance(other, MyClass):
return NotImplemented
return bool(self._compare(other))

and so on. You can save a certain amount of repetition (by my count, six 
lines of code) by pulling out the "if not isinstance" check into a 
decorator, but since the decorator is likely to be about six lines long, 
I wouldn't bother :-)

-- 
Steven
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Am I not seeing the Error?

On Sun, 11 Aug 2013 03:33:52 +0100, Chris Angelico wrote:

> Next thing to do is split it into more lines. Why is all that in a
> single line?

The only good excuse for writing multiple statements on a single line 
separated by semi-colons is if the Enter key on your keyboard is broken.

:-)

-- 
Steven
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Python Basic Doubt

On Sat, 10 Aug 2013 16:42:22 -0400, Roy Smith wrote:

> In article ,
>  Dennis Lee Bieber  wrote:
> 
>> Because id(n) is not giving you the address of the NAME. It is giving
>> you the address of the "10"
> 
> Actually, it is giving you the id of the int(10) object.  Maybe it's an
> address, maybe it's not.  Only your implementation knows for sure.

/steve cheers from the audience

Thank you for mentioning this. Using Jython:

>>> x = 10
>>> id(x)
1


And using IronPython:

>>> x = 10
>>> id(x)
43


"id" does not stand for "memory address". It stands for "identity".


-- 
Steven
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Paramiko Help. Execute command to Interactive Shell which is opened by SSHClient()

On Thu, Aug 8, 2013 at 8:20 AM, sagar varule  wrote:
>  stdin, stdout, stderr = client.exec_command(bv_cmd)
> for line in stderr.readlines():
> print line
> for line in stdout.readlines():
> print line
> But problem here is client.exec_command(bv_cmd) waits for command to execute 
> completely and then it returns to stdout,stderr. And I want to see the ouput 
> from the command during its execution. Because my command takes long time for 
> execution.


Are you certain that exec_command is what's waiting, and not readlines()?

ChrisA
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Could you verify this, Oh Great Unicode Experts of the Python-List?

On Sun, Aug 11, 2013 at 7:17 AM, Joshua Landau  wrote:
> Given tweet = b"caf\x65\xCC\x81".decode():
>
> >>> tweet
> 'café'
>
> But:
>
> >>> len(tweet)
> 5

You're now looking at the difference between glyphs and combining
characters. Twitter counts combining characters, so when you build one
"thing" out of lots of separately-typed parts, it does count as more
characters.

Read this article for some arguments on the subject, including a
number of references to Twitter itself:

http://unspecified.wordpress.com/2012/04/19/the-importance-of-language-level-abstract-unicode-strings/

ChrisA
-- 
http://mail.python.org/mailman/listinfo/python-list

Introduction to my fellow Python Friends

2013-08-11 Thread Krishnan Shankar

Hi Friends,

I would like to introduce myself.

I am Krishnan from Chennai, India. I am using python for 2 years for Test
Automation. I am fascinated by the language and its capabilities. I am
willing to move into Python development and I am doing the best i can to
learn the language completely and contribute to open source.

I figured out that the best way is to talk to the experts and so i
subscribed to this mailing list. It will be cool if anybody can help me out
by telling the etiquette of this mailing list, like

1. How to acknowledge a reply? Should i put a one to one mail or send it to
the mailing list itself?
2. How can i see or get a question asked by someone else? (So that i can
reply for that with my best possible knowledge. I currently get as Python
mail Digest)
3. How can i use this mailing list in the best possible way?

I hope to have a wonderful time with Python here. I hope i am not wasting
your time. Sorry for the inconvenience if i am.

Regards,
Krishnan
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Could you verify this, Oh Great Unicode Experts of the Python-List?

On Sun, 11 Aug 2013 07:17:42 +0100, Joshua Landau wrote:

> Basically, I think Twitter's broken.

Oh, in about a million ways, but apparently people like it :-(

> For my full discusion on the matter, see:
> http://www.reddit.com/r/learnpython/comments/1k2yrn/
help_with_len_and_input_function_33/cbku5e8
> 
> Here's the first post of mine, ineffectually edited for this list:
> 
> """
> The obvious solution [to getting the length of a tweet]
> is wrong. Like, slightly wrong¹.
> 
> Given tweet = b"caf\x65\xCC\x81".decode():

I assume you're using Python 3, where UTF-8 is the default encoding.

> >>> tweet
> 'café'
> 
> But:
> 
> >>> len(tweet)
> 5

Yes, that's correct. Unicode doesn't promise to have a single unique 
representation for all human-readable strings. In this case, the string 
"cafe" with an accent on the "e" can be generated by two sequences of 
code points:

LATIN SMALL LETTER C
LATIN SMALL LETTER A
LATIN SMALL LETTER F
LATIN SMALL LETTER E
COMBINING ACUTE ACCENT

or 

LATIN SMALL LETTER C
LATIN SMALL LETTER A
LATIN SMALL LETTER F
LATIN SMALL LETTER E WITH ACUTE

The reason some accented letters have single code point forms is to 
support legacy charsets; the reason some only exist as combining 
characters is due to the combinational explosion. Some languages allow 
you to add up to five or six different accent on any of dozens of 
different letters. If each combination needed its own unique code point, 
there wouldn't be enough code points. For bonus points, if there are five 
accents that can be placed in any combination of zero or more on any of 
four characters, how many code points would be needed?

Neither form is "right" or "wrong", they are both equally valid. They 
encode differently, of course, since UTF-8 does guarantee that every 
sequence of code points has a unique byte representation:

py> tweet.encode('utf-8')
'cafe\xcc\x81'
py> u'café'.encode('utf-8')
'caf\xc3\xa9'

Note that the form you used, b"caf\x65\xCC\x81", is the same as the first 
except that you have shown "e" in hex for some reason:

py> b'\x65' == b'e'
True

> So the solution is:
> 
> >>> import unicodedata
> >>> len(unicodedata.normalize("NFC", tweet))
> 4

In this particular case, this will reduce the tweet to the normalised 
form that Twitter uses.

[...]
> After further testing (I don't actually use Twitter) it seems the whole
> thing was just smoke and mirrors. The linked article is a lie, at least
> on the user's end.

Which linked article? The one on dev.twitter.com seems to be okay to me. 
Of course, they might be lying when they say "Twitter counts the length 
of a Tweet using the Normalization Form C (NFC) version of the text", I 
have no idea. But the seem to have a good grasp of the issues involved, 
and assuming they do what they say, at least Western European users 
should be happy.

> On Linux you can prove this by running:
> 
> >>> p = subprocess.Popen(['xsel', '-bi'], stdin=subprocess.PIPE)
> >>> p.communicate(input=b"caf\x65\xCC\x81")
> (None, None)
> 
> "café" will be in your Copy-Paste buffer, and you can paste it in to
> the tweet-box. It takes 5 characters. So much for testing ;).

How do you know that it takes 5 characters? Is that some Javascript 
widget? I'd blame buggy Javascript before Twitter.

If this shows up in your application as café rather than café, it is a 
bug in the text rendering engine. Some applications do not deal with 
combining characters correctly.

(It's a hard problem to solve, and really needs support from the font. In 
some languages, the same accent will appear in different places depending 
on the character they are attached to, or the other accents there as 
well. Or so I've been lead to believe.)

> ¹ https://dev.twitter.com/docs/counting-
> characters#Definition_of_a_Character

Looks reasonable to me. No obvious errors to my eyes.

-- 
Steven
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Introduction to my fellow Python Friends

On Sun, Aug 11, 2013 at 9:17 AM, Krishnan Shankar
 wrote:
> I figured out that the best way is to talk to the experts and so i
> subscribed to this mailing list. It will be cool if anybody can help me out
> by telling the etiquette of this mailing list, like

Hi! Welcome!

> 1. How to acknowledge a reply? Should i put a one to one mail or send it to
> the mailing list itself?

Normally, you needn't simply "acknowledge" a reply; but if you're
responding in a way that will be beneficial to the list, you would
want to send to the list.

> 2. How can i see or get a question asked by someone else? (So that i can
> reply for that with my best possible knowledge. I currently get as Python
> mail Digest)

Now that you're subscribed, you should be able to read every message
sent to the list (modulo spam filtering, but ignore that). Other
threads should come up just fine

> 3. How can i use this mailing list in the best possible way?

First and foremost, don't use Google Groups :) But you're already doing that.

Quote people's text with angle-bracket markers (see my quoting of your
text), and put your responses underneath (called "bottom-posting" -
the opposite is "top-posting" and is not preferred). Trim quoted text
to just what you need for context, rather than quoting an entire
conversation.

Always cite people accurately; normally your mail client will handle
that for you, but when you trim, sometimes you'll need to be careful
of that.

Don't accidentally cross-post from savoy...@bridgewater.edu, like I've
been known to do *whistles innocently*

Be courteous, even if someone flames you :) And post real content,
with real meaning. Even if you ignore all else, do this and you'll be
valuable to the community, and people will forgive (to an extent!)
failures in the other areas.

ChrisA
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Could you verify this, Oh Great Unicode Experts of the Python-List?

On 11 August 2013 10:09, Steven D'Aprano
 wrote:
> The reason some accented letters have single code point forms is to
> support legacy charsets; the reason some only exist as combining
> characters is due to the combinational explosion. Some languages allow
> you to add up to five or six different accent on any of dozens of
> different letters. If each combination needed its own unique code point,
> there wouldn't be enough code points. For bonus points, if there are five
> accents that can be placed in any combination of zero or more on any of
> four characters, how many code points would be needed?

52?

> Note that the form you used, b"caf\x65\xCC\x81", is the same as the first
> except that you have shown "e" in hex for some reason:
>
> py> b'\x65' == b'e'
> True

Yeah.. I did that because the linked post did it. I'm not sure why either ;).

> On Sun, 11 Aug 2013 07:17:42 +0100, Joshua Landau wrote:
>>
>> So the solution is:
>>
>> >>> import unicodedata
>> >>> len(unicodedata.normalize("NFC", tweet))
>> 4
>
> In this particular case, this will reduce the tweet to the normalised
> form that Twitter uses.
>
> [...]
>> After further testing (I don't actually use Twitter) it seems the whole
>> thing was just smoke and mirrors. The linked article is a lie, at least
>> on the user's end.
>
> Which linked article? The one on dev.twitter.com seems to be okay to me.

That's the one.

> Of course, they might be lying when they say "Twitter counts the length
> of a Tweet using the Normalization Form C (NFC) version of the text", I
> have no idea. But the seem to have a good grasp of the issues involved,
> and assuming they do what they say, at least Western European users
> should be happy.

They *don't* seem to be doing what they say.

>> On Linux you can prove this by running:
>>
>> >>> p = subprocess.Popen(['xsel', '-bi'], stdin=subprocess.PIPE)
>> >>> p.communicate(input=b"caf\x65\xCC\x81")
>> (None, None)
>>
>> "café" will be in your Copy-Paste buffer, and you can paste it in to
>> the tweet-box. It takes 5 characters. So much for testing ;).
>
> How do you know that it takes 5 characters? Is that some Javascript
> widget? I'd blame buggy Javascript before Twitter.

I go to twitter.com, log in and press that odd blue compose button in
the top-right. After pasting at says I have 135 (down from 140)
characters left.

My only question here is, since you can't post after 140
non-normalised characters, who cares if the server counts it as less?

> If this shows up in your application as café rather than café, it is a
> bug in the text rendering engine. Some applications do not deal with
> combining characters correctly.

Why the rendering engine?

> (It's a hard problem to solve, and really needs support from the font. In
> some languages, the same accent will appear in different places depending
> on the character they are attached to, or the other accents there as
> well. Or so I've been lead to believe.)
>
>
>> ¹ https://dev.twitter.com/docs/counting-
>> characters#Definition_of_a_Character
>
> Looks reasonable to me. No obvious errors to my eyes.

*Not sure whether talking about the link or my post*
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Could you verify this, Oh Great Unicode Experts of the Python-List?

On 11 August 2013 07:24, Chris Angelico  wrote:
> On Sun, Aug 11, 2013 at 7:17 AM, Joshua Landau  wrote:
>> Given tweet = b"caf\x65\xCC\x81".decode():
>>
>> >>> tweet
>> 'café'
>>
>> But:
>>
>> >>> len(tweet)
>> 5
>
> You're now looking at the difference between glyphs and combining
> characters. Twitter counts combining characters, so when you build one
> "thing" out of lots of separately-typed parts, it does count as more
> characters.

@https://dev.twitter.com/docs/counting-characters#Definition_of_a_Character
> The "café" issue mentioned above raises the question of how you count
> the characters in the Tweet string "café". To the human eye the length is
> clearly four characters. Depending on how the data is represented this
> could be either five or six UTF-8 bytes. Twitter does not want to penalize
> a user for the fact we use UTF-8 or for the fact that the API client in
> question used the longer representation. Therefore, Twitter does count
> "café" as four characters no matter which representation is sent.

Which would imply that twitter doesn't count combining characters,
even though the web interface seems to.

> Read this article for some arguments on the subject, including a
> number of references to Twitter itself:
>
> http://unspecified.wordpress.com/2012/04/19/the-importance-of-language-level-abstract-unicode-strings/

I read that *last* time you pointed it out :P. It's a good link, though.

--
Anyhow, it's good to know I haven't been obviously stupid with my
understanding of Unicode. I learnt it all from this list anyway;
wouldn't want to disappoint!
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Paramiko Help. Execute command to Interactive Shell which is opened by SSHClient()

On 11 August 2013 09:57, Chris Angelico  wrote:
> On Thu, Aug 8, 2013 at 8:20 AM, sagar varule  wrote:
>>  stdin, stdout, stderr = client.exec_command(bv_cmd)
>> for line in stderr.readlines():
>> print line
>> for line in stdout.readlines():
>> print line
>> But problem here is client.exec_command(bv_cmd) waits for command to execute 
>> completely and then it returns to stdout,stderr. And I want to see the ouput 
>> from the command during its execution. Because my command takes long time 
>> for execution.
>
> Are you certain that exec_command is what's waiting, and not readlines()?

That's a very good catch;

Sagar, you might want to look at:
http://stackoverflow.com/a/4896288/1763356
for a good solution if this is the case. There might be more builtin
ways (maybe select.select) but I'm betting that link's the most
rigorous.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Am I not seeing the Error?

On 11 August 2013 09:28, Steven D'Aprano
 wrote:
> into more lines. Why is all that in a
>> single line?
>
> The only good excuse for writing multiple statements on a single line
> separated by semi-colons is if the Enter key on your keyboard is broken.

That's not a good excuse.

It *is* a good excuse for any of the following:

* Buying a new keyboard
* Using a new keymap, possibly replacing cAPSLOCK with Return
* Crying
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Could you verify this, Oh Great Unicode Experts of the Python-List?

On Sun, 11 Aug 2013 10:44:40 +0100, Joshua Landau wrote:

> On 11 August 2013 10:09, Steven D'Aprano
>  wrote:
>> The reason some accented letters have single code point forms is to
>> support legacy charsets; the reason some only exist as combining
>> characters is due to the combinational explosion. Some languages allow
>> you to add up to five or six different accent on any of dozens of
>> different letters. If each combination needed its own unique code
>> point, there wouldn't be enough code points. For bonus points, if there
>> are five accents that can be placed in any combination of zero or more
>> on any of four characters, how many code points would be needed?
> 
> 52?

More than double that.

Consider a single character. It can have 0 to 5 accents, in any 
combination. Order doesn't matter, and there are no duplicates, so there 
are:

0 accent: take 0 from 5 = 1 combination;
1 accent: take 1 from 5 = 5 combinations;
2 accents: take 2 from 5 = 5!/(2!*3!) = 10 combinations;
3 accents: take 3 from 5 = 5!/(3!*2!) = 10 combinations;
4 accents: take 4 from 5 = 5 combinations;
5 accents: take 5 from 5 = 1 combination

giving a total of 32 combinations for a single character. Since there are 
four characters in this hypothetical language that take accents, that 
gives a total of 4*32 = 128 distinct code points needed.

In reality, Unicode has currently code points U+0300 to U+036F (112 code 
points) to combining characters. It's not really meaningful to combine 
all 112 of them, or even most of 112 of them, but let's assume that we 
can legitimately combine up to three of them on average (some languages 
will allow more, some less) on just six different letters. That gives us:

0 accent: 1 combination
1 accent: 112 combinations
2 accents: 112!/(2!*110!) = 6216 combinations
3 accents: 112!/(3!*109!) = 227920 combinations

giving 234249 combinations, by six base characters, = 1405494 code 
points. Which is comfortably more than the 1114112 code points Unicode 
has in total :-)

This calculation is horribly inaccurate, since you can't arbitrarily 
combine (say) accents from Greek with accents from IPA, but I reckon that 
the combinational explosion of accented letters is still real.

[...]
>> Of course, they might be lying when they say "Twitter counts the length
>> of a Tweet using the Normalization Form C (NFC) version of the text", I
>> have no idea. But the seem to have a good grasp of the issues involved,
>> and assuming they do what they say, at least Western European users
>> should be happy.
> 
> They *don't* seem to be doing what they say.
[...]
>>> "café" will be in your Copy-Paste buffer, and you can paste it in to
>>> the tweet-box. It takes 5 characters. So much for testing ;).
>>
>> How do you know that it takes 5 characters? Is that some Javascript
>> widget? I'd blame buggy Javascript before Twitter.
> 
> I go to twitter.com, log in and press that odd blue compose button in
> the top-right. After pasting at says I have 135 (down from 140)
> characters left.

I'm pretty sure that will be a piece of Javascript running in your 
browser that reports the number of characters in the text box. So, I 
would expect that either:

- Javascript doesn't provide a way to normalize text;

- Twitter's Javascript developer(s) don't know how to normalize text, or 
can't be bothered to follow company policy (shame on them);

- the Javascript just asks the browser, and the browser doesn't know how 
to count characters the Twitter way;

etc. But of course posting to Twitter via your browser isn't the only way 
to post. Twitter provide an API to twit, and *that* is the ultimate test 
of whether Twitter's dev guide is lying or not.

> My only question here is, since you can't post after 140 non-normalised
> characters, who cares if the server counts it as less?

People who bypass the browser and write their own Twitter client.

>> If this shows up in your application as café rather than café, it is a
>> bug in the text rendering engine. Some applications do not deal with
>> combining characters correctly.
> 
> Why the rendering engine?

If the text renderer assumes it can draw once code point at a time, it 
will draw the "e", then reach the combining accent. It could, in 
principle, backspace and draw it over the "e", but more likely it will 
just draw it next to it.

What the renderer should do is walk the string, collecting characters 
until it reaches one which is not a combining character, then draw them 
all at once one on top of each other. A good font may have special 
glyphs, or at least hints, for combining accents. For instance, if you 
have a dot accent and a comma accent drawn one on top of the other, it 
looks like a comma; what you are supposed to do is move them side by 
side, so you have separate dot and comma glyphs.

>> (It's a hard problem to solve, and really needs support from the font.
>> In some languages, the same accent will appear in different places
>> depending on the character t

Re: Could you verify this, Oh Great Unicode Experts of the Python-List?

On Sun, Aug 11, 2013 at 12:14 PM, Steven D'Aprano
 wrote:
> Consider a single character. It can have 0 to 5 accents, in any
> combination. Order doesn't matter, and there are no duplicates, so there
> are:
>
> 0 accent: take 0 from 5 = 1 combination;
> 1 accent: take 1 from 5 = 5 combinations;
> 2 accents: take 2 from 5 = 5!/(2!*3!) = 10 combinations;
> 3 accents: take 3 from 5 = 5!/(3!*2!) = 10 combinations;
> 4 accents: take 4 from 5 = 5 combinations;
> 5 accents: take 5 from 5 = 1 combination
>
> giving a total of 32 combinations for a single character. Since there are
> four characters in this hypothetical language that take accents, that
> gives a total of 4*32 = 128 distinct code points needed.

There's an easy way to calculate it. Instead of the "take N from 5"
notation, simply look at it as a set of independent bits - each of
your accents may be either present or absent. So it's 1<<5
combinations for a single character, which is the same 32 figure you
came up with, but easier to work with in the ridiculous case.

> In reality, Unicode has currently code points U+0300 to U+036F (112 code
> points) to combining characters. It's not really meaningful to combine
> all 112 of them, or even most of 112 of them...

If you *were* to use literally ANY combination, that would be 1<<112
which is... uhh... five billion yottacombinations. Don't bother
working that one out by the "take N" method, it'll take you too long
:)

Oh, and that's 1<<112 possible combining character combinations, so
you then need to multiply that by the number of base characters you
could use

ChrisA
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: PEP 450 Adding a statistics module to Python

2013-08-11 Thread Skip Montanaro

> See the Rationale of PEP 450 for more reasons why “install NumPy” is not
> a feasible solution for many use cases, and why having ‘statistics’ as a
> pure-Python, standard-library package is desirable.

I read that before posting but am not sure I agree. I don't see the
screaming need for this package.  Why can't it continue to live on
PyPI, where, once again, it is available as "pip install ..."?

S
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Could you verify this, Oh Great Unicode Experts of the Python-List?

On 11 August 2013 12:14, Steven D'Aprano
 wrote:
> On Sun, 11 Aug 2013 10:44:40 +0100, Joshua Landau wrote:
>
>> On 11 August 2013 10:09, Steven D'Aprano
>>  wrote:
>>> The reason some accented letters have single code point forms is to
>>> support legacy charsets; the reason some only exist as combining
>>> characters is due to the combinational explosion. Some languages allow
>>> you to add up to five or six different accent on any of dozens of
>>> different letters. If each combination needed its own unique code
>>> point, there wouldn't be enough code points. For bonus points, if there
>>> are five accents that can be placed in any combination of zero or more
>>> on any of four characters, how many code points would be needed?
>>
>> 52?
>
> More than double that.
>
> Consider a single character. It can have 0 to 5 accents, in any
> combination. Order doesn't matter, and there are no duplicates, so there
> are:
>
> 0 accent: take 0 from 5 = 1 combination;
> 1 accent: take 1 from 5 = 5 combinations;
> 2 accents: take 2 from 5 = 5!/(2!*3!) = 10 combinations;
> 3 accents: take 3 from 5 = 5!/(3!*2!) = 10 combinations;
> 4 accents: take 4 from 5 = 5 combinations;
> 5 accents: take 5 from 5 = 1 combination
>
> giving a total of 32 combinations for a single character. Since there are
> four characters in this hypothetical language that take accents, that
> gives a total of 4*32 = 128 distinct code points needed.

I didn't see "four characters", and I did (1 + 5 + 10) * 2 and came up
with 52...
Maybe I should get more sleep.
-- 
http://mail.python.org/mailman/listinfo/python-list

Reading from stdin first, then use curses

2013-08-11 Thread Timo Schmiade

Hi all,

I wrote a replacement for urlview to properly extract URLs from emails.
You can find the first draft here:

  https://github.com/the-isz/pyurlview

When I call it with an email file passed to the '-f' argument, it does
pretty much what I want already. However, I intend to use it in mutt,
which pipes the message to the program like so:

macro pager \cu 'pyurlview.py' 'Follow links with pyurlview'

The problem is rather obvious but - unfortunately - not so easy to solve:

* The program reads the mail from stdin
* The terminal in which it runs is a pseudo-terminal (pipe)
* curses is not able to accept user input from the pseudo-terminal

The question is:

How do I read from stdin first and afterwards allow curses to read user
input?

Thanks in advance and kind regards,

Timo
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: PEP 450 Adding a statistics module to Python

2013-08-11 Thread Nicholas Cole

On Sun, Aug 11, 2013 at 12:50 PM, Skip Montanaro  wrote:

> > See the Rationale of PEP 450 for more reasons why “install NumPy” is not
> > a feasible solution for many use cases, and why having ‘statistics’ as a
> > pure-Python, standard-library package is desirable.
>
> I read that before posting but am not sure I agree. I don't see the
> screaming need for this package.  Why can't it continue to live on
> PyPI, where, once again, it is available as "pip install ..."?

Well, I *do* think this module would be a wonderful addition to the
standard library.  I've often used python to do analysis of data, nothing
complicated enough to need NumPy, but certainly things where I've needed to
find averages etc. I've rolled my own functions for these projects, and I'm
sure they are fragile.  Besides, it was just a pain to do them.

PyPI is terrific. There are lots of excellent modules on there.  It's a
wonderful resource.  But I think that the standard library is also a
wonderful thing, and where there are clearly defined modules, that serve a
general, well-defined function and where development does not need to be
very rapid, I think they should go into the Standard Library.

I'm aware that my opinion is just that of one user, but I read this PEP and
I thought, "Thank Goodness! That looks great.  About time too."

N.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Could you verify this, Oh Great Unicode Experts of the Python-List?

2013-08-11 Thread wxjmfauth

Le dimanche 11 août 2013 11:09:44 UTC+2, Steven D'Aprano a écrit :
> On Sun, 11 Aug 2013 07:17:42 +0100, Joshua Landau wrote:
> 
> 
> 
> 
> The reason some accented letters have single code point forms is to 
> 
> support legacy charsets; ...

No.

jmf

PS Unicode normalization is failing expectedly very well
with the FSR.

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Could you verify this, Oh Great Unicode Experts of the Python-List?

On 11 August 2013 13:51,   wrote:
> Le dimanche 11 août 2013 11:09:44 UTC+2, Steven D'Aprano a écrit :
>> On Sun, 11 Aug 2013 07:17:42 +0100, Joshua Landau wrote:
>>
>> The reason some accented letters have single code point forms is to
>> support legacy charsets; ...
>
> No.
>
> jmf
>
> PS Unicode normalization is failing expectedly very well
> with the FSR.

No.

Joshua Landau

PS Proper arguments are falling expectedly very well with the internet
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Am I not seeing the Error?

2013-08-11 Thread Roy Smith

In article <52074b43$0$3$c3e8da3$54964...@news.astraweb.com>,
 Steven D'Aprano  wrote:

> On Sun, 11 Aug 2013 03:33:52 +0100, Chris Angelico wrote:
> 
> > Next thing to do is split it into more lines. Why is all that in a
> > single line?
> 
> The only good excuse for writing multiple statements on a single line 
> separated by semi-colons is if the Enter key on your keyboard is broken.

Well, maybe if you're testing something on the command line with "python 
-c".
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: PEP 450 Adding a statistics module to Python

On Sun, 11 Aug 2013 06:50:36 -0500, Skip Montanaro wrote:

>> See the Rationale of PEP 450 for more reasons why “install NumPy” is
>> not a feasible solution for many use cases, and why having ‘statistics’
>> as a pure-Python, standard-library package is desirable.
> 
> I read that before posting but am not sure I agree. I don't see the
> screaming need for this package.  Why can't it continue to live on PyPI,
> where, once again, it is available as "pip install ..."?

The same could be said about any module, really. And indeed, some 
languages have that philosophy, they provide no libraries to speak of, if 
you want anything you have to either write it yourself or get it from 
somebody else.

Not everyone has the luxury of being able, or allowed, to run "pip 
install" to get additional, non-standard packages. E.g. in corporate 
environments. But I've already said that in the PEP.

-- 
Steven
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Introduction to my fellow Python Friends

2013-08-11 Thread Dave Angel

Krishnan Shankar wrote:

> Hi Friends,

Hi, and welcome to the mailing list.

> I figured out that the best way is to talk to the experts and so i
> subscribed to this mailing list. It will be cool if anybody can help me out
> by telling the etiquette of this mailing list, like
>
> 1. How to acknowledge a reply? Should i put a one to one mail or send it to
> the mailing list itself?

Acknowledgement is not usually necessary.  Sometimes you want to make a
special thanks, with no useful information for others.  In that case, an
email to the individual is called for.  But in most cases, you should do
a "reply" to the list, and not to the individual.  See below for caveats
on what I mean by reply.

> 2. How can i see or get a question asked by someone else? (So that i can
> reply for that with my best possible knowledge. I currently get as Python
> mail Digest)

The digest is a combination of everything posted to the list for some
period of time.  Its title is generic, and replies to it are not
threaded into the list in a meaningful way.  If your email program can
handle it, consider turning off the digest feature.  if your email
program cannot handle it, consider using a different way to access the
list.

What I used to do with a digest was to have my email program treat the
indivdual messages in the digest as attachments, and reply to the
attachment itself.  Then I discovered that by turning off digest and
telling my email program to track threads, I could more easily place
each message in context.  Then I realized that my email program could
treat the list as a newsgroup (which is what it really is), and not
affect the way I accessed it very much.  Finally, I switched to a
dedicated newsreader.

In order to relate messages that are intended to be threaded together,
you want to use the same subject line (Notice that my newsreader added
the letters "Re:" in front of your subject line).  Don't post a message
with the subject like:  Re: Python-list Digest, Vol 119, Issue 37

Even better, you want to actually reply-all to the individual message. 
This puts some invisible stuff in the header that makes it practical for
savvy newsreaders & mailreaders to see just where in the thread you're
replying.

Better still, you use a Reply-list, which sends to the list only, not to
the individual.  You can do that with reply-all, just by removing the
extra name(s).

> 3. How can i use this mailing list in the best possible way?

Don't send HTML mails (as you have here, see  a little nonsense at
the end which I left in for your edification). In many cases, it'll
distort your response, either because of the encoding in your mail
program, or the decoding at our end. A text forum should get only text
messages, where what you send is what we see. This also means we don't
see colors, special fonts, or funny formatting. If you're using email,
specify TEXT email. It also saves space in every message, and for
those of us paying per megabyte, it can add up.

Don't use tabs.  Indent your code by 4 column intervals, represented by
spaces.  If you use tabs, some people won't see any indentation, others
will see one column, and others will see 8 columns.

Do pick a good subject line (as you have here).  "Help" isn't a good
subject, and neither is "urgent problem".  Describe something about
what's not working for you, even if you turn out to guess wrong.

Do specify your environment with your original post on any thread.  You
may not think the problem you're having with input() is related to which
OS you're using AND which version of Python, but it probably does.  And
for other problems there can at least be subtle differences.  So at a
minimum, say "I'm running Python 3.3 on Linux"  or whatever.

Finally, show your code, explain what you expected to happen, and be
explicit about what did happen, preferably by copy/pasting the
responses.  If there's an error, post the entire traceback.

> I hope to have a wonderful time with Python here. I hope i am not wasting
> your time. Sorry for the inconvenience if i am.

No inconvenience.  A real pleasure to see someone who explicitly wants
to fit in.  Most do, but assume that it'll be automatic.

Following is a little piece of the html in the original message.
>
> Hi Friends,I would like

>

-- 
DaveA

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: PEP 450 Adding a statistics module to Python

2013-08-11 Thread Roy Smith

In article ,
 Skip Montanaro  wrote:

> > See the Rationale of PEP 450 for more reasons why âinstall NumPyâ is not
> > a feasible solution for many use cases, and why having âstatisticsâ as a
> > pure-Python, standard-library package is desirable.
> 
> I read that before posting but am not sure I agree. I don't see the
> screaming need for this package.  Why can't it continue to live on
> PyPI, where, once again, it is available as "pip install ..."?

My previous comments on this topic were along the lines of "installing 
numpy is a non-starter if all you need are simple mean/std-dev".  You 
do, however, make a good point here.  Running "pip install statistics" 
is a much lower barrier to entry than getting numpy going, especially if 
statistics is pure python and thus has no dependencies on compiler tool 
chains which may be missing.

Still, I see two classes of function in PEP-450.  Class 1 is the really 
basic stuff:

* mean
* std-dev

Class 2 are the more complicated things like:

* linear regression
* median
* mode
* functions for calculating the probability of random variables
  from the normal, t, chi-squared, and F distributions
* inference on the mean
* anything that differentiates between population and sample

I could see leaving class 2 stuff in an optional pure-python module to 
be installed by pip, but for (as the PEP phrases it), the simplest and 
most obvious statistical functions (into which I lump mean and std-dev), 
having them in the standard library would be a big win.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Could you verify this, Oh Great Unicode Experts of the Python-List?

2013-08-11 Thread MRAB

On 11/08/2013 10:54, Joshua Landau wrote:

On 11 August 2013 07:24, Chris Angelico wrote:

On Sun, Aug 11, 2013 at 7:17 AM, Joshua Landau wrote:

Given tweet = b"caf\x65\xCC\x81".decode():

>>> tweet
'café'

But:

>>> len(tweet)
5

You're now looking at the difference between glyphs and combining
characters. Twitter counts combining characters, so when you build one
"thing" out of lots of separately-typed parts, it does count as more
characters.

@https://dev.twitter.com/docs/counting-characters#Definition_of_a_Character

The "café" issue mentioned above raises the question of how you count
the characters in the Tweet string "café". To the human eye the length is
clearly four characters. Depending on how the data is represented this
could be either five or six UTF-8 bytes. Twitter does not want to penalize
a user for the fact we use UTF-8 or for the fact that the API client in
question used the longer representation. Therefore, Twitter does count
"café" as four characters no matter which representation is sent.

Which would imply that twitter doesn't count combining characters,
even though the web interface seems to.

Read this article for some arguments on the subject, including a
number of references to Twitter itself:

http://unspecified.wordpress.com/2012/04/19/the-importance-of-language-level-abstract-unicode-strings/

I read that *last* time you pointed it out :P. It's a good link, though.

--
Anyhow, it's good to know I haven't been obviously stupid with my
understanding of Unicode. I learnt it all from this list anyway;
wouldn't want to disappoint!

If twitter counts characters, not codepoints, you could then ask
whether it passes the codepoints through as given. If it does, then you
experiment to see how much data you could send encoded as a sequence of
combining codepoints. (You might want to check the Term of Use first,
though! :-))
--
http://mail.python.org/mailman/listinfo/python-list

Re: PEP 450 Adding a statistics module to Python

2013-08-11 Thread duncan smith


On 11/08/13 15:02, Roy Smith wrote:

In article ,
  Skip Montanaro  wrote:


See the Rationale of PEP 450 for more reasons why â€œinstall NumPyâ€� is not
a feasible solution for many use cases, and why having â€˜statisticsâ€™ as a
pure-Python, standard-library package is desirable.


I read that before posting but am not sure I agree. I don't see the
screaming need for this package.  Why can't it continue to live on
PyPI, where, once again, it is available as "pip install ..."?


My previous comments on this topic were along the lines of "installing
numpy is a non-starter if all you need are simple mean/std-dev".  You
do, however, make a good point here.  Running "pip install statistics"
is a much lower barrier to entry than getting numpy going, especially if
statistics is pure python and thus has no dependencies on compiler tool
chains which may be missing.

Still, I see two classes of function in PEP-450.  Class 1 is the really
basic stuff:

* mean
* std-dev

Class 2 are the more complicated things like:

* linear regression
* median
* mode
* functions for calculating the probability of random variables
   from the normal, t, chi-squared, and F distributions
* inference on the mean
* anything that differentiates between population and sample

I could see leaving class 2 stuff in an optional pure-python module to
be installed by pip, but for (as the PEP phrases it), the simplest and
most obvious statistical functions (into which I lump mean and std-dev),
having them in the standard library would be a big win.



I would probably move other descriptive statistics (median, mode, 
correlation, ...) into Class 1.


I roll my own statistical tests as I need them - simply to avoid having 
a dependency on R. But I generally do end up with a dependency on scipy 
because I need scipy.stats.distributions. So I guess a distinct library 
for probability distributions would be handy - but maybe it should not 
be in the standard library.


Once we move on to statistical modelling (e.g. linear regression) I 
think the case for inclusion in the standard library becomes weaker 
still. Cheers.


Duncan
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Elegant compare

2013-08-11 Thread Jason Friedman

> This is a hard question to answer, because your code snippet isn't
> clearly extensible to the case where you have ten attributes. What's the
> rule for combining them? If instance A has five attributes less than
> those of instance B, and five attributes greater than those of instance
> B, which wins?

Yes, my code snippet was too short, I should have said:

class my_class:
def __init__(self, attr1, attr2, attr3):
self.attr1 = attr1 #string
self.attr2 = attr2 #string
self.attr3 = attr3 #string
def __lt__(self, other):
if self.attr1 < other.attr1:
return True
elif self.attr2 < other.attr2:
return True
else:
return self.attr3 < other.attr3

Chris's answer is actually perfectly adequate for my needs.
Thank you Steve and Chris.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Python Basic Doubt

2013-08-11 Thread Xavi


Thanks to all for your answers,

I guess it is more flexible with isinstance (the duck test :)
I'm going to change the type checks.

Respect to the "Names starting and ending with double-underscore".
I don't know how to get the name of a classe without them.
obj.__class__.__name__

Thanks.
--
Xavi
--
http://mail.python.org/mailman/listinfo/python-list

Re: Python Basic Doubt