Re: Documenting builtin methods

[Joshua Landau]

On 12 July 2013 04:43, alex23  wrote:
>
> My last post seems to have been eaten by either Thunderbird or the
> EternalSeptember servers, but it contained an erroneous claim that the
> straight function version performed as well as the factory one. However, in
> the interim a co-worker has come up with a slightly faster variant:
>
> from functools import partial
> from collections import deque
>
> class exhaust_it(partial):
> """custom doc string"""
>
> exhaust_it = exhaust_it(deque(maxlen=0).extend)
>
> Shadowing the class name with the partial instance will ensure it has the
> same name when accessed via help(), and it's a simple way to avoid needing
> to clean up the namespace, as well.

That's beautiful. You could even trivially make a wrapper function:

def wrap_docstring(function, docstring, *, name=None):
class Wrapper(partial): pass
Wrapper.__name__ = function.__name__ if name is None else name
Wrapper.__doc__ = docstring
return Wrapper(function)

which is no slower. You get great introspection through the "func"
attribute, too :).

Also:

>>> times = time_raw(), time_function(), time_factory(), time_argument_hack(), 
>>> time_partial()
>>> [round(time/times[0], 1) for time in times]
[1.0, 16.8, 3.1, 3.0, 1.8]

This times almost purely the constant overhead by calling
exhaust_iterabe on an empty iterable. So your friend wins the
premature optimisation test, too.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Callable or not callable, that is the question!

[Ian Kelly]

On Thu, Jul 11, 2013 at 8:12 PM, Steven D'Aprano
 wrote:
> On Thu, 11 Jul 2013 15:05:59 +0200, Ulrich Eckhardt wrote:
>
>> Hello!
>>
>> I just stumbled over a case where Python (2.7 and 3.3 on MS Windows)
>> fail to detect that an object is a function, using the callable()
>> builtin function. Investigating, I found out that the object was indeed
>> not callable, but in a way that was very unexpected to me:
> [...]
>>  X.test2[0]() # TypeError: 'staticmethod' object is not callable
>>
>>
>> Bug or feature?
>
> In my opinion, a bug. I thought I had actually submitted it to the bug
> tracker, but apparently I was a shameful slacker and did not. However
> there was a discussion in this thread:
>
> http://mail.python.org/pipermail/python-dev/2011-March/109090.html
>
>
> Here's a simpler demonstration of the issue:
>
> assert callable(staticmethod(lambda: None))

If staticmethod is going to be callable then classmethod should be
callable also.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Callable or not callable, that is the question!

[Ulrich Eckhardt]

Am 11.07.2013 16:11, schrieb Peter Otten:

Ulrich Eckhardt wrote:

Bug or feature?


No bug. Missing feature if you come up with a convincing use-case.


class Parser:
def _handle_bool(input):
# ...
pass

types = {'bool': _handle_bool,
 'boolean': _handle_bool,}

def parse(self, line):
t,s,v = line.partition(':')
handler = types[t]
return handler(v)

I want a utility function that really behaves just like a function. I'd 
prefer to nest it inside the class that uses it, to make the code easier 
to understand. Since I don't want the implicit self-binding either, I 
would use staticmethod to make this clear, too.


Since I can live without any of this, it's not a big issue. What is to 
me a big issue though is the fact that Python behaves unexpectedly and 
reading Steven's response and the link there, it seems I'm not alone.


Greetings!

Uli



--
http://mail.python.org/mailman/listinfo/python-list

Question about mailing list rules

[Devyn Collier Johnson]

Am I allowed to ask questions like "Here is my code. How can I optimize 
it?" on this mailing list?


Mahalo,

Devyn Collier Johnson
devyncjohn...@gmail.com
--
http://mail.python.org/mailman/listinfo/python-list

Re: Stack Overflow moderator “animuson”

[Antoon Pardon]

Op 10-07-13 13:56, Benedict Verheyen schreef:

Op Wed, 10 Jul 2013 12:12:10 +0100, schreef Joshua Landau:




Do you have any non-trivial, properly benchmarked real-world examples
that this affects, remembering to use full Unicode support in Perl (as
Python has it by default)?



Indeed, as Joshua says, instead of going through all the effort to talk
about it, why don't you show us that Python is slower in regexs than perl.
That's at least relevant bandwdith.

Next, if Python is slower, there might be ways to improve the performance.

Last, if there is a SPU (Secret Python Underground), I want to j



Heretic, it is the

--
http://mail.python.org/mailman/listinfo/python-list

RE Module Performance

[Devyn Collier Johnson]

I recently saw an email in this mailing list about the RE module being 
made slower. I no long have that email. However, I have viewed the 
source for the RE module, but I did not see any code that would slow 
down the script for no valid reason. Can anyone explain what that user 
meant or if I missed that part of the module?


Can the RE module be optimized in any way or at least the "re.sub" portion?

Mahalo,

Devyn Collier Johnson
devyncjohn...@gmail.com
--
http://mail.python.org/mailman/listinfo/python-list

Re: hex dump w/ or w/out utf-8 chars

[Chris Angelico]

On Fri, Jul 12, 2013 at 4:42 AM,   wrote:
> BTW, since
> when a serious coding scheme need an extermal marker?
>

All of them.

Content-type: text/plain; charset=UTF-8

ChrisA
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: RE Module Performance

[Chris Angelico]

On Fri, Jul 12, 2013 at 9:44 AM, Devyn Collier Johnson
 wrote:
> I recently saw an email in this mailing list about the RE module being made
> slower. I no long have that email. However, I have viewed the source for the
> RE module, but I did not see any code that would slow down the script for no
> valid reason. Can anyone explain what that user meant or if I missed that
> part of the module?
>
> Can the RE module be optimized in any way or at least the "re.sub" portion?

There was a post by Steven D'Aprano [1] in which he referred to it
busy-waiting just to make regular expressions slower than the
alternatives, but his tongue was firmly in his cheek at the time. As
to real performance questions, there have been a variety of
alternatives proposed, including I think the regex module [2] which is
supposed to outperform 're' by quite a margin, but since I tend
towards other solutions, I can't quote personal results or hard
figures.

If re.sub can be optimized and you can see a way to do so, post a
patch to the bug tracker; if it improves performance and doesn't have
any ridiculous costs to it, it'll probably be accepted.

[1] http://mail.python.org/pipermail/python-list/2013-July/651818.html
[2] https://pypi.python.org/pypi/regex

ChrisA
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Question about mailing list rules

[Chris Angelico]

On Fri, Jul 12, 2013 at 9:59 AM, Devyn Collier Johnson
 wrote:
> Am I allowed to ask questions like "Here is my code. How can I optimize it?"
> on this mailing list?

Sure you can! And you'll get a large number of responses, not all of
which are directly to do with your question. :)

I assume the code in question _is_ written in Python, right?

ChrisA
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Question about mailing list rules

[Andreas Perstinger]

On 12.07.2013 01:59, Devyn Collier Johnson wrote:

Am I allowed to ask questions like "Here is my code. How can I optimize
it?" on this mailing list?


If it's written in Python, why not?

But that doesn't mean you are guaranteed to get an answer :-).

And please read http://sscce.org/ before posting your latest 10,000 line 
program :-)


Bye, Andreas

--
http://mail.python.org/mailman/listinfo/python-list

Re: Callable or not callable, that is the question!

[Duncan Booth]

Ulrich Eckhardt  wrote:

> Am 11.07.2013 16:11, schrieb Peter Otten:
>> Ulrich Eckhardt wrote:
>>> Bug or feature?
>>
>> No bug. Missing feature if you come up with a convincing use-case.
> 
> class Parser:
>  def _handle_bool(input):
>  # ...
>  pass
> 
>  types = {'bool': _handle_bool,
>   'boolean': _handle_bool,}
> 
>  def parse(self, line):
>  t,s,v = line.partition(':')
>  handler = types[t]
>  return handler(v)
> 
> I want a utility function that really behaves just like a function. I'd 
> prefer to nest it inside the class that uses it, to make the code easier 
> to understand. Since I don't want the implicit self-binding either, I 
> would use staticmethod to make this clear, too.

But the example you gave works just fine as written! You are only using 
your utility function as a function so there's no need for it to be a 
staticmethod or indeed any other sort of method.

To be a convincing use-case you would have to show a situation where 
something had to be both a static method and a utility method rather than 
just one or the other and also where you couldn't just have both.

If you can persuade me that you need _handle_bool as both a static method 
and a utility function, you probably also need to explain why you can't 
just use both:

class Parser:
def _handle_bool(input): ...
handle_bool = staticmethod(_handle_bool)


-- 
Duncan Booth http://kupuguy.blogspot.com
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: RE Module Performance

[wxjmfauth]

Le vendredi 12 juillet 2013 01:44:05 UTC+2, Devyn Collier Johnson a écrit :
> I recently saw an email in this mailing list about the RE module being 
> 
> made slower. I no long have that email. However, I have viewed the 
> 
> source for the RE module, but I did not see any code that would slow 
> 
> down the script for no valid reason. Can anyone explain what that user 
> 
> meant or if I missed that part of the module?
> 
> 
> 
> Can the RE module be optimized in any way or at least the "re.sub" portion?
> 
> 
> 
> Mahalo,
> 
> 
> 
> Devyn Collier Johnson
> 
> devyncjohn...@gmail.com

--

I would not care too much about the performance
of re.

With the new Flexible String Representation, you
can use a logarithmic scale to compare re results.
To be honest, there is improvment if you are an
ascii user.

Am I the only one who tested this? Probably.

jmf


-- 
http://mail.python.org/mailman/listinfo/python-list

Re: RE Module Performance

[Chris Angelico]

On Fri, Jul 12, 2013 at 7:23 PM,   wrote:
>
> I would not care too much about the performance
> of re.
>
> With the new Flexible String Representation, you
> can use a logarithmic scale to compare re results.
> To be honest, there is improvment if you are an
> ascii user.
>
> Am I the only one who tested this? Probably.

Am I the only one who thinks that Dihedral posted under jmf's name?

ChrisA
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: RE Module Performance

[Joshua Landau]

On 12 July 2013 10:27, Chris Angelico  wrote:
> On Fri, Jul 12, 2013 at 7:23 PM,   wrote:
>>
>> I would not care too much about the performance
>> of re.
>>
>> With the new Flexible String Representation, you
>> can use a logarithmic scale to compare re results.
>> To be honest, there is improvment if you are an
>> ascii user.
>>
>> Am I the only one who tested this? Probably.
>
> Am I the only one who thinks that Dihedral posted under jmf's name?

A bot posting as a troll to troll a different troll? Meta.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: RE Module Performance

[Chris Angelico]

On Fri, Jul 12, 2013 at 7:39 PM, Joshua Landau  wrote:
> On 12 July 2013 10:27, Chris Angelico  wrote:
>> On Fri, Jul 12, 2013 at 7:23 PM,   wrote:
>>>
>>> I would not care too much about the performance
>>> of re.
>>>
>>> With the new Flexible String Representation, you
>>> can use a logarithmic scale to compare re results.
>>> To be honest, there is improvment if you are an
>>> ascii user.
>>>
>>> Am I the only one who tested this? Probably.
>>
>> Am I the only one who thinks that Dihedral posted under jmf's name?
>
> A bot posting as a troll to troll a different troll? Meta.

Yeah, it is. But the only other explanation is that jmf has become
rather more incomprehensible than usual. Normally I can understand
what he's complaining enough to refute it, but here I feel like I'm
responding to Dihedral.

ChrisA
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Callable or not callable, that is the question!

[Steven D'Aprano]

On Fri, 12 Jul 2013 07:36:30 +, Duncan Booth wrote:

> To be a convincing use-case you would have to show a situation where
> something had to be both a static method and a utility method rather
> than just one or the other and also where you couldn't just have both.

I have a class where I have a function that needs to be called both while 
the class is being constructed, and afterwards:

class Example:
@staticmethod
def do_stuff(arg):
...

do_stuff(23)  # This doesn't work.

Example.do_stuff(42)


I have work-arounds: inside the class, I call do_stuff.__func__ instead 
of do_stuff, but really, that's ugly and distracting and merely a work-
around for the fact that staticmethods aren't callable. To make them 
callable is trivially easy: they just need a __call__ method that calls 
__func__ for you.



> If you can persuade me that you need _handle_bool as both a static
> method and a utility function, you probably also need to explain why you
> can't just use both:
> 
> class Parser:
> def _handle_bool(input): ...
> handle_bool = staticmethod(_handle_bool)

That's extremely inelegant. Why have two functions for something which is 
conceptually one?


-- 
Steven
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Kivy for Python 3.3

[Paul Kölle]

Am 11.07.2013 16:26, schrieb fronag...@gmail.com:
[scnipp]

C:\MinGW\bin\gcc.exe -mdll -O -Wall -IC:\Python33\include
-IC:\Python33\include -c kivy\graphics\vertex_instructions.c -o
build\temp.win32-3.3\Release\kivy\grap hics\vertex_instructions.o In
file included from kivy\graphics\vertex_instructions.c:314:0:
kivy\graphics\/gl_redirect.h:8:22: fatal error: GL/glew.h: No such
file or direc tory compilation terminated. error: command 'gcc'
failed with exit status 1

Working on the reasoning that sticking the missing file into the
python3.3\libs file worked, I tried adding the glew files
(glew32.dll, glew32.lib, GL\glew and GL\wglew.h) there, however, it
doesn't seem to have made a difference. And this I googled this
before asking. Doesn't seem to be much.


Hi,

I can feel your pain beeing dragged into 
compile-unknown-source-on-windows from an innocent looking python script 
;) This time it's not the linker complaining but GCC (the compiler) 
itself not being able to find the header file GL/glew.h. Headers are 
searched in the so called "include path" which is specified in setup.py. 
This link http://glew.sourceforge.net/install.html suggests glew.h is 
part of VisualStudio, and has some other interesting information.


hth
 Paul




--
http://mail.python.org/mailman/listinfo/python-list

Re: Callable or not callable, that is the question!

[Peter Otten]

Steven D'Aprano wrote:

> On Fri, 12 Jul 2013 07:36:30 +, Duncan Booth wrote:
> 
>> To be a convincing use-case you would have to show a situation where
>> something had to be both a static method and a utility method rather
>> than just one or the other and also where you couldn't just have both.
> 
> I have a class where I have a function that needs to be called both while
> the class is being constructed, and afterwards:
> 
> class Example:
> @staticmethod
> def do_stuff(arg):
> ...
> 
> do_stuff(23)  # This doesn't work.
> 
> Example.do_stuff(42)

That is a bit too abstract for my taste to count as a use-case. 

Also, as given the example will work in Python 3 when you remove the 
@staticmethod ;)

That said I can see that the error comes as a surprise and I would be fine 
with callable staticmethod objects.

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Question about mailing list rules

[Chris Angelico]

On Fri, Jul 12, 2013 at 8:44 PM, Devyn Collier Johnson
 wrote:
> I am going to love this mailing list even more.
>
> Really, only Python code? I wanted to ask Python users about Perl! (^u^)
>
> Devyn Collier Johnson

Heh. You'd be surprised what comes up. If it's at least broadly
related to Python (maybe you're comparing constructs in Python and
Perl??), it'll probably be accepted. And even stuff that's completely
off-topic does at times get discussed.

One small thing, though. Please avoid top-posting; the convention on
this list is to quote text above, and write yours underneath. Makes it
easier to follow the flow of conversation. Thanks!

ChrisA
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Question about mailing list rules

[Chris “Kwpolska” Warrick]

On Fri, Jul 12, 2013 at 12:47 PM, Chris Angelico  wrote:
> On Fri, Jul 12, 2013 at 8:44 PM, Devyn Collier Johnson
>  wrote:
>> I am going to love this mailing list even more.
>>
>> Really, only Python code? I wanted to ask Python users about Perl! (^u^)
>>
>> Devyn Collier Johnson
>
> Heh. You'd be surprised what comes up. If it's at least broadly
> related to Python (maybe you're comparing constructs in Python and
> Perl??), it'll probably be accepted. And even stuff that's completely
> off-topic does at times get discussed.
>
> One small thing, though. Please avoid top-posting; the convention on
> this list is to quote text above, and write yours underneath. Makes it
> easier to follow the flow of conversation. Thanks!
>
> ChrisA
> --
> http://mail.python.org/mailman/listinfo/python-list

One more thing Devyn should do is watch the “To:” field and make sure
it says python-list@python.org, because the above message was sent to
Chris only, and that is not what should happen most of the time.
Another option is to use Reply All, but it will make old Usenet hags
angry, because they would get two copies.

--
Kwpolska  | GPG KEY: 5EAAEA16
stop html mail| always bottom-post
http://asciiribbon.org| http://caliburn.nl/topposting.html
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: UnicodeDecodeError: 'utf-8' codec can't decode byte 0xb6 in position 0: invalid start byte

[Wayne Werner]

On Thu, 4 Jul 2013, Νίκος Γκρ33κ wrote:


Στις 4/7/2013 6:10 μμ, ο/η MRAB έγραψε:

What do you mean "I don't know how to catch the exception with
OSError"? You've tried "except socket.gaierror" and "except
socket.herror", well just write "except OSError" instead!



try:
host = socket.gethostbyaddr( os.environ['REMOTE_ADDR'] )[0]
except OSError:
host = "UnResolved"

produces also an internal server error.

Are you sure is just except OSError ?



Have you ensured that 'REMOTE_ADDR' is actually a key in os.environ? I 
highly recommend using the logging module to help diagnose what the actual 
exception is.


HTH,
-W-- 
http://mail.python.org/mailman/listinfo/python-list

Re: UnicodeDecodeError: 'utf-8' codec can't decode byte 0xb6 in position 0: invalid start byte

[Ferrous Cranus]

Στις 12/7/2013 2:47 μμ, ο/η Wayne Werner έγραψε:

On Thu, 4 Jul 2013, Νίκος Γκρ33κ wrote:


Στις 4/7/2013 6:10 μμ, ο/η MRAB έγραψε:

What do you mean "I don't know how to catch the exception with
OSError"? You've tried "except socket.gaierror" and "except
socket.herror", well just write "except OSError" instead!



try:
host = socket.gethostbyaddr( os.environ['REMOTE_ADDR'] )[0]
except OSError:
host = "UnResolved"

produces also an internal server error.

Are you sure is just except OSError ?



Have you ensured that 'REMOTE_ADDR' is actually a key in os.environ? I
highly recommend using the logging module to help diagnose what the
actual exception is.

HTH,
-W


Yes it is a key, but the problem as i suspected was cloudflare.
i had to use os.environ['HTTP_CF_CONNECTING_IP'] that cloudflare passes 
as variable i the cgi enviroment in order to retrieve the visitor's ip.



try:
gi = pygeoip.GeoIP('/usr/local/share/GeoLiteCity.dat')
city = gi.time_zone_by_addr( os.environ['HTTP_CF_CONNECTING_IP'] )
host = socket.gethostbyaddr( os.environ['HTTP_CF_CONNECTING_IP'] )[0]
except Exception as e:
host = repr(e)


Sometimes though iam still receiving the usual
UnicodeDecodeError('utf-8', b'\xc1\xf0\xef\xf4\xf5

but only for a few ip addresses, in moste cases it works.
--
What is now proved was at first only imagined!
--
http://mail.python.org/mailman/listinfo/python-list

Re: Kivy for Python 3.3

[fronagzen]

On Friday, July 12, 2013 6:14:46 PM UTC+8, Paul Kölle wrote:
> Am 11.07.2013 16:26, schrieb fronag...@gmail.com:
> 
> [scnipp]
> 
> > C:\MinGW\bin\gcc.exe -mdll -O -Wall -IC:\Python33\include
> 
> > -IC:\Python33\include -c kivy\graphics\vertex_instructions.c -o
> 
> > build\temp.win32-3.3\Release\kivy\grap hics\vertex_instructions.o In
> 
> > file included from kivy\graphics\vertex_instructions.c:314:0:
> 
> > kivy\graphics\/gl_redirect.h:8:22: fatal error: GL/glew.h: No such
> 
> > file or direc tory compilation terminated. error: command 'gcc'
> 
> > failed with exit status 1
> 
> >
> 
> > Working on the reasoning that sticking the missing file into the
> 
> > python3.3\libs file worked, I tried adding the glew files
> 
> > (glew32.dll, glew32.lib, GL\glew and GL\wglew.h) there, however, it
> 
> > doesn't seem to have made a difference. And this I googled this
> 
> > before asking. Doesn't seem to be much.
> 
> 
> 
> Hi,
> 
> 
> 
> I can feel your pain beeing dragged into 
> 
> compile-unknown-source-on-windows from an innocent looking python script 
> 
> ;) This time it's not the linker complaining but GCC (the compiler) 
> 
> itself not being able to find the header file GL/glew.h. Headers are 
> 
> searched in the so called "include path" which is specified in setup.py. 
> 
> This link http://glew.sourceforge.net/install.html suggests glew.h is 
> 
> part of VisualStudio, and has some other interesting information.
> 
> 
> 
> hth
> 
>   Paul

Thanks for the response.

After a bit of tinkering, I've finally managed to compile the blasted thing! I 
just straight up dumped Glew into the appropriate files for VS10, and used the 
VS10 compiler instead of mingw32- and it worked!
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Question about mailing list rules

[Chris “Kwpolska” Warrick]

On Fri, Jul 12, 2013 at 2:25 PM, Devyn Collier Johnson
 wrote:
>
> On 07/12/2013 07:11 AM, Chris “Kwpolska” Warrick wrote:
>>
>> On Fri, Jul 12, 2013 at 12:47 PM, Chris Angelico  wrote:
>>>
>>> On Fri, Jul 12, 2013 at 8:44 PM, Devyn Collier Johnson
>>>  wrote:

 I am going to love this mailing list even more.

 Really, only Python code? I wanted to ask Python users about Perl! (^u^)

 Devyn Collier Johnson
>>>
>>> Heh. You'd be surprised what comes up. If it's at least broadly
>>> related to Python (maybe you're comparing constructs in Python and
>>> Perl??), it'll probably be accepted. And even stuff that's completely
>>> off-topic does at times get discussed.
>>>
>>> One small thing, though. Please avoid top-posting; the convention on
>>> this list is to quote text above, and write yours underneath. Makes it
>>> easier to follow the flow of conversation. Thanks!
>>>
>>> ChrisA
>>> --
>>> http://mail.python.org/mailman/listinfo/python-list
>>
>> One more thing Devyn should do is watch the “To:” field and make sure
>> it says python-list@python.org, because the above message was sent to
>> Chris only, and that is not what should happen most of the time.
>> Another option is to use Reply All, but it will make old Usenet hags
>> angry, because they would get two copies.
>>
>> --
>> Kwpolska  | GPG KEY: 5EAAEA16
>> stop html mail| always bottom-post
>> http://asciiribbon.org| http://caliburn.nl/topposting.html
>
> Thank you and sorry about that.
>
> Kwpolska, I noticed your email shows "stop html mail" at the bottom. I have
> Thunderbird setup to use HTML mail. Are my emails coming up as plain text or
> HTML on this mailing list?
>
> Devyn Collier Johnson

Apparently, it comes in plaintext.  Mailman does not say that it
removed HTML, so it should be fine.  Either way, there should be a
switch in the new message window to set what should be sent.

--
Kwpolska  | GPG KEY: 5EAAEA16
stop html mail| always bottom-post
http://asciiribbon.org| http://caliburn.nl/topposting.html
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Question about mailing list rules

[Roy Smith]

In article ,
 Chris Angelico  wrote:

> On Fri, Jul 12, 2013 at 9:59 AM, Devyn Collier Johnson
>  wrote:
> > Am I allowed to ask questions like "Here is my code. How can I optimize it?"
> > on this mailing list?
> 
> Sure you can! And you'll get a large number of responses, not all of
> which are directly to do with your question. :)
> 
> I assume the code in question _is_ written in Python, right?

If not, then the answer to "how can I optimize it" is obvious, isn't it?
-- 
http://mail.python.org/mailman/listinfo/python-list

GeoIP2 for retrieving city and region ?

[Νικόλας]

Hello, iam still looking for a way to identify the city of my website 
visitors.


The closet i have gone is to come up with the visitor's ISP city:

try:
gi = pygeoip.GeoIP('/usr/local/share/GeoLiteCity.dat')
city = gi.time_zone_by_addr( os.environ['HTTP_CF_CONNECTING_IP'] )
host = socket.gethostbyaddr( os.environ['HTTP_CF_CONNECTING_IP'] )[0]
except Exception as e:
host = repr(e)


But today i was searching again for this and found out about geoip2, 
maybe that would help more.



>>> import geoip2
Traceback (most recent call last):
  File "", line 1, in 
ImportError: No module named 'geoip2'


>>> client = geoip2.webservices.Client(42, 'abcdef123456')
>>> omni = client.omni('24.24.24.24')
>>> country = omni.country
>>> print(country.iso_code)


I cant even import the module even though my 'pip install geopip2' was 
successful


There is definately i way to identify the users location based solely on 
its ip address as this site does it: http://www.geoiptool.com/


Google, MS, facebook and twitter are not the only ones that can do it?

Perhaps this is being done by giving  longitude and latitude?
--
What is now proved was at first only imagined!
--
http://mail.python.org/mailman/listinfo/python-list

Understanding other people's code

[L O'Shea]

Hi all,
I've been asked to take over a project from someone else and to extend the 
functionality of this. The project is written in Python which I haven't had any 
real experience with (although I do really like it) so I've spent the last week 
or two settling in, trying to get my head around Python and the way in which 
this code works.

The problem is the code was clearly written by someone who is exceptionally 
good and seems to inherit everything from everywhere else. It all seems very 
dynamic, nothing is written statically except in some configuration files. 
Basically the problem is I am new to the language and this was clearly written 
by someone who at the moment is far better at it than I am!

I'm starting to get pretty worried about my lack of overall progress and so I 
wondered if anyone out there had some tips and techniques for understanding 
other peoples code. There has to be 10/15 different scripts with at least 10 
functions in each file I would say.

Literally any idea will help, pen and paper, printing off all the code and 
doing some sort of highlighting session - anything! I keep reading bits of code 
and thinking "well where the hell has that been defined and what does it mean" 
to find it was inherited from 3 modules up the chain. I really need to get a 
handle on how exactly all this slots together! Any techniques,tricks or 
methodologies that people find useful would be much appreciated.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Understanding other people's code

[Chris Angelico]

On Sat, Jul 13, 2013 at 12:22 AM, L O'Shea  wrote:
> I'm starting to get pretty worried about my lack of overall progress and so I 
> wondered if anyone out there had some tips and techniques for understanding 
> other peoples code. There has to be 10/15 different scripts with at least 10 
> functions in each file I would say.


The first thing I'd recommend is getting yourself familiar with the
language itself, and (to some extent) the standard library. Then
you'll know that any unrecognized wotzit must have come from your own
project, so you'll be able to search up its definition. Then I'd
tackle source files one at a time, and look at the very beginning. If
the original coder was at all courteous, each file will start off with
a block of 'import' statements, looking something like this:

import re
import itertools

Or possibly like this:

from itertools import cycle, islice

Or, if you're unlucky, like this:

from tkinter import *

The first form is easy. You'll find references to "re.sub" or
"itertools.takewhile"; the second form at least names what it's
grabbing (so you'll find "cycle" or "islice" in the code), and the
third just dumps a whole lot of stuff into your namespace.

Actually, if the programmer's been really nice, there'll be a block
comment or a docstring at the top of the file, which might even be
up-to-date and accurate. But I'm guessing you already know to look for
that. :)

The other thing I would STRONGLY recommend: Keep the interactive
interpreter handy. Any line of code you don't understand, paste it
into the interpreter. Chances are it won't wipe out your entire hard
drive :) But seriously, there is much to gain and nothing to lose by
keeping IDLE or the command-line interpreter handy.

ChrisA
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: [Python-ideas] float('∞')=float('inf')

[Chris Angelico]

On Sat, Jul 13, 2013 at 12:43 AM, Gerald Britton
 wrote:
> Man I don't know how you are doing this!  I just tried:
>
> float('') and got
>
> Value error: could not convert string to float ''
>
> For that matter, I can't figure out how to type the greek letter for
> pi in gmail!  Guess I have some things to learn.
>
> So, if Python doesn't recognize the symbol for pi, why should it
> recognize the one for infinity?

Considering that Python can't represent π in a float anyway, I
wouldn't be too bothered. And what else? float('τ') for twice that
value? Not really necessary imho.

ChrisA
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: [Python-ideas] float('∞')=float('inf')

[Chris Angelico]

On Sat, Jul 13, 2013 at 12:52 AM, Chris Angelico  wrote:
> On Sat, Jul 13, 2013 at 12:43 AM, Gerald Britton
>  wrote:
>> Man I don't know how you are doing this!  I just tried:
>>
>> float('') and got
>>
>> Value error: could not convert string to float ''
>>
>> For that matter, I can't figure out how to type the greek letter for
>> pi in gmail!  Guess I have some things to learn.
>>
>> So, if Python doesn't recognize the symbol for pi, why should it
>> recognize the one for infinity?
>
> Considering that Python can't represent π in a float anyway, I
> wouldn't be too bothered. And what else? float('τ') for twice that
> value? Not really necessary imho.
>
> ChrisA

Take no notice of me going insane over here. Nothing to see, move
along, these are not the neurons you're looking for!

ChrisA
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Question about mailing list rules

[Dave Angel]

On 07/12/2013 08:34 AM, Chris “Kwpolska” Warrick wrote:

On Fri, Jul 12, 2013 at 2:25 PM, Devyn Collier Johnson
 wrote:



 


One more thing Devyn should do is watch the “To:” field and make sure
it says python-list@python.org, because the above message was sent to
Chris only, and that is not what should happen most of the time.
Another option is to use Reply All, but it will make old Usenet hags
angry, because they would get two copies.


In Thunderbird, use Reply-List, rather than Reply-All, and it'll go to 
the right place.  Still worth checking, though.




--
Kwpolska  | GPG KEY: 5EAAEA16
stop html mail| always bottom-post
http://asciiribbon.org| http://caliburn.nl/topposting.html


Thank you and sorry about that.

Kwpolska, I noticed your email shows "stop html mail" at the bottom. I have
Thunderbird setup to use HTML mail. Are my emails coming up as plain text or
HTML on this mailing list?

Devyn Collier Johnson


Apparently, it comes in plaintext.  Mailman does not say that it
removed HTML, so it should be fine.  Either way, there should be a
switch in the new message window to set what should be sent.



I only see one message from Devyn, and it's not the one quoted here. 
But the one I do see is in plain text.  The safest way to tell (other 
than noticing that indentation randomly goes away, or that things are 
missing, or ...) is to view the source.


In Thunderbird, select the message, then View->MessageSource.  Then 
search in that for html.  Frequently there's two versions of the 
message, one text and one html.  What you want is a single version, text.




--
DaveA.

--
http://mail.python.org/mailman/listinfo/python-list

Re: Understanding other people's code

[Peter Otten]

L O'Shea wrote:

> Hi all,
> I've been asked to take over a project from someone else and to extend the
> functionality of this. The project is written in Python which I haven't
> had any real experience with (although I do really like it) so I've spent
> the last week or two settling in, trying to get my head around Python and
> the way in which this code works.
> 
> The problem is the code was clearly written by someone who is
> exceptionally good and seems to inherit everything from everywhere else.
> It all seems very dynamic, nothing is written statically except in some
> configuration files. Basically the problem is I am new to the language and
> this was clearly written by someone who at the moment is far better at it
> than I am!
> 
> I'm starting to get pretty worried about my lack of overall progress and
> so I wondered if anyone out there had some tips and techniques for
> understanding other peoples code. There has to be 10/15 different scripts
> with at least 10 functions in each file I would say.

That sounds like the project is well-organised and not too big. If you take 
one day per module you're there in two weeks...

> Literally any idea will help, pen and paper, printing off all the code and
> doing some sort of highlighting session - anything! I keep reading bits of
> code and thinking "well where the hell has that been defined and what does
> it mean" to find it was inherited from 3 modules up the chain. 

As you put it here, the project is too complex. So now we have a mixed 
message. Of course your impression may stem from lack of experience...

> I really
> need to get a handle on how exactly all this slots together! Any
> techniques,tricks or methodologies that people find useful would be much
> appreciated.

Is there any documentation? Read that. Do the functions have docstrings? 
import the modules (start with the main entry point) in the interactive 
interpreter and use help():

>>> import some_module
>>> help(some_module)

Or use

$ python -m pydoc -g

and hit "open browser" (the project directory has to be in PYTHONPATH).

See if you can talk to the author/previous maintainer. He may be willing to 
give you the big picture or hints for the parts where he did "clever" 
things.

Try to improve your Python by writing unrelated scripts.

Make little changes to the project (add print statements, invoke functions 
from your own driver script, make a local variable global for further 
inspection in the interactive interpreter using dir() -- whatever you can 
think of.

The latter should of course be done in a test installation rather than the 
production environment. 

Rely on version control once you start making modifications for real -- but 
I think you knew that already...



-- 
http://mail.python.org/mailman/listinfo/python-list

Re: GeoIP2 for retrieving city and region ?

[Dave Angel]

On 07/12/2013 10:18 AM, Νικόλας wrote:

Hello, iam still looking for a way to identify the city of my website
visitors.


   



I cant even import the module even though my 'pip install geopip2' wa
successful


Either it wasn't successful, or it's not the package you thought.  There 
are lots of things you might have downloaded, but since you give no 
details...





There is definately i way to identify the users location based solely on
its ip address as this site does it: http://www.geoiptool.com/



Sure, and as long as you don't mind it being 1000 miles off, you too can 
claim to do it too.  When I go to that site, the little pin is in 
Kansas, which is 1100 miles from where I live on the east coast of the US.




Google, MS, facebook and twitter are not the only ones that can do it?

Perhaps this is being done by giving  longitude and latitude?


Or by reading the mind of the programmer.

I suggest you read that geoiptool site, in particular the page

http://www.geoiptool.com/en/ip_info/

There is some misinformation, but notice carefully the part about 
dynamic IP addresses.  Probably 99% of the individual users on the web 
(the ones using a browser) have dynamic IP addresses.  The fixed ones 
are needed by servers, and especially for DNS use, where the name lookup 
wants to be stable for relatively log periods of time.


--
DaveA

--
http://mail.python.org/mailman/listinfo/python-list

Re: Question about mailing list rules

[Devyn Collier Johnson]

On 07/12/2013 09:04 AM, Roy Smith wrote:

In article ,
  Chris Angelico  wrote:


On Fri, Jul 12, 2013 at 9:59 AM, Devyn Collier Johnson
 wrote:

Am I allowed to ask questions like "Here is my code. How can I optimize it?"
on this mailing list?

Sure you can! And you'll get a large number of responses, not all of
which are directly to do with your question. :)

I assume the code in question _is_ written in Python, right?

If not, then the answer to "how can I optimize it" is obvious, isn't it?

Yeah, I should have been more specific and said Python.
--
http://mail.python.org/mailman/listinfo/python-list

Re: RE Module Performance

[Devyn Collier Johnson]

Could you explain what you mean? What and where is the new Flexible 
String Representation?


Devyn Collier Johnson


On 07/12/2013 05:23 AM, wxjmfa...@gmail.com wrote:

Le vendredi 12 juillet 2013 01:44:05 UTC+2, Devyn Collier Johnson a écrit :

I recently saw an email in this mailing list about the RE module being

made slower. I no long have that email. However, I have viewed the

source for the RE module, but I did not see any code that would slow

down the script for no valid reason. Can anyone explain what that user

meant or if I missed that part of the module?



Can the RE module be optimized in any way or at least the "re.sub" portion?



Mahalo,



Devyn Collier Johnson

devyncjohn...@gmail.com

--

I would not care too much about the performance
of re.

With the new Flexible String Representation, you
can use a logarithmic scale to compare re results.
To be honest, there is improvment if you are an
ascii user.

Am I the only one who tested this? Probably.

jmf




--
http://mail.python.org/mailman/listinfo/python-list

Re: Question about mailing list rules

[Devyn Collier Johnson]

I am going to love this mailing list even more.

Really, only Python code? I wanted to ask Python users about Perl! (^u^)

Devyn Collier Johnson


On 07/12/2013 03:26 AM, Chris Angelico wrote:

On Fri, Jul 12, 2013 at 9:59 AM, Devyn Collier Johnson
 wrote:

Am I allowed to ask questions like "Here is my code. How can I optimize it?"
on this mailing list?

Sure you can! And you'll get a large number of responses, not all of
which are directly to do with your question. :)

I assume the code in question _is_ written in Python, right?

ChrisA


--
http://mail.python.org/mailman/listinfo/python-list

Re: UnicodeDecodeError: 'utf-8' codec can't decode byte 0xb6 in position 0: invalid start byte

[Dave Angel]

On 07/12/2013 07:56 AM, Ferrous Cranus wrote:

Στις 12/7/2013 2:47 μμ, ο/η Wayne Werner έγραψε:

On Thu, 4 Jul 2013, Νίκος Γκρ33κ wrote:


Στις 4/7/2013 6:10 μμ, ο/η MRAB έγραψε:

What do you mean "I don't know how to catch the exception with
OSError"? You've tried "except socket.gaierror" and "except
socket.herror", well just write "except OSError" instead!



try:
host = socket.gethostbyaddr( os.environ['REMOTE_ADDR'] )[0]
except OSError:
host = "UnResolved"

produces also an internal server error.

Are you sure is just except OSError ?



Have you ensured that 'REMOTE_ADDR' is actually a key in os.environ? I
highly recommend using the logging module to help diagnose what the
actual exception is.

HTH,
-W


Yes it is a key, but the problem as i suspected was cloudflare.
i had to use os.environ['HTTP_CF_CONNECTING_IP'] that cloudflare passes
as variable i the cgi enviroment in order to retrieve the visitor's ip.


try:
 gi = pygeoip.GeoIP('/usr/local/share/GeoLiteCity.dat')
 city = gi.time_zone_by_addr( os.environ['HTTP_CF_CONNECTING_IP'] )
 host = socket.gethostbyaddr( os.environ['HTTP_CF_CONNECTING_IP'] )[0]
except Exception as e:
 host = repr(e)


Sometimes though iam still receiving the usual
UnicodeDecodeError('utf-8', b'\xc1\xf0\xef\xf4\xf5

but only for a few ip addresses, in moste cases it works.


And naturally, you now know how to debug those UnicodeDecodeError 
problems.  Surely, the code you post here isn't what you actually do, 
because when people spend time to give you detailed advice, you actually 
read it, and work at understanding it.


Chortle, snort.

--
DaveA

--
http://mail.python.org/mailman/listinfo/python-list

Re: GeoIP2 for retrieving city and region ?

[Νικόλας]

Στις 12/7/2013 6:32 μμ, ο/η Dave Angel έγραψε:


I suggest you read that geoiptool site, in particular the page

http://www.geoiptool.com/en/ip_info/

There is some misinformation, but notice carefully the part about
dynamic IP addresses.  Probably 99% of the individual users on the web
(the ones using a browser) have dynamic IP addresses.  The fixed ones
are needed by servers, and especially for DNS use, where the name lookup
wants to be stable for relatively log periods of time.



I did, for me it gives exact city location and not the ISP's city location.

I dont know whay for you ti just says Kansas, it shoudln't, since it 
susing longitute and latitude, it should have been accurate.


--
What is now proved was at first only imagined!
--
http://mail.python.org/mailman/listinfo/python-list

Re: RE Module Performance

[Joshua Landau]

On 12 July 2013 11:45, Devyn Collier Johnson  wrote:
> Could you explain what you mean? What and where is the new Flexible String
> Representation?

Do not worry. jmf is on about his old rant comparing broken previous
versions of Python to newer ones which in some microbenchmarks are
slower. I don't really get why he spends his time on it.

If you're interested, the basic of it is that strings now use a
variable number of bytes to encode their values depending on whether
values outside of the ASCII range and some other range are used, as an
optimisation.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Understanding other people's code

[Eric S. Johansson]

On Fri, 12 Jul 2013 10:22:59 -0400, L O'Shea   
wrote:


Literally any idea will help, pen and paper, printing off all the code  
and doing some sort of highlighting session - anything! I keep reading  
bits of code and thinking "well where the hell has that been defined and  
what does it mean" to find it was inherited from 3 modules up the chain.  
I really need to get a handle on how exactly all this slots together!  
Any techniques,tricks or methodologies that people find useful would be  
much appreciated.


glad to hear you're having a WTF moment (what's that function). Suggestion  
would be index cards, each containing notes on a class. truly understand  
what each parent class is in which methods are to be overloaded. Then look  
at one child and understand how it. Work your way breadth first down the  
inheritance tree.

--
http://mail.python.org/mailman/listinfo/python-list

Re: RE Module Performance

[Peter Otten]

Joshua Landau wrote:

> On 12 July 2013 11:45, Devyn Collier Johnson 
> wrote:
>> Could you explain what you mean? What and where is the new Flexible
>> String Representation?
> 
> Do not worry. jmf is on about his old rant comparing broken previous
> versions of Python to newer ones which in some microbenchmarks are
> slower. I don't really get why he spends his time on it.
> 
> If you're interested, the basic of it is that strings now use a
> variable number of bytes to encode their values depending on whether
> values outside of the ASCII range and some other range are used, as an
> optimisation.

See also 

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: RE Module Performance

[Chris Angelico]

On Fri, Jul 12, 2013 at 8:45 PM, Devyn Collier Johnson
 wrote:
> Could you explain what you mean? What and where is the new Flexible String
> Representation?

(You're top-posting again. Please put your text underneath what you're
responding to - it helps maintain flow and structure.)

Python versions up to and including 3.2 came in two varieties: narrow
builds (commonly found on Windows) and wide builds (commonly found on
Linux). Narrow builds internally represented Unicode strings in
UTF-16, while wide builds used UTF-32. This is a problem, because it
means that taking a program from one to another actually changes its
behaviour:

Python 2.6.4 (r264:75706, Dec  7 2009, 18:45:15)
[GCC 4.4.1] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> len(u"\U00012345")
1

Python 2.7.4 (default, Apr  6 2013, 19:54:46) [MSC v.1500 32 bit
(Intel)] on win32
>>> len(u"\U00012345")
2

In fact, the narrow builds are flat-out buggy, because you can put
something in as a single character that simply isn't a single
character. You can then pull that out as two characters and make a
huge mess of things:

>>> s=u"\U00012345"
>>> s[0]
u'\ud808'
>>> s[1]
u'\udf45'

*Any* string indexing will be broken if there is a single character
>U+ ahead of it in the string.

Now, this problem is not unique to Python. Heaps of other languages
have the same issue, the same buggy behaviour, the same compromises.
What's special about Python is that it actually managed to come back
from that problem. (Google's V8 JavaScript engine, for instance, is
stuck with it, because the ECMAScript specification demands UTF-16. I
asked on an ECMAScript list and was told "can't change that, it'd
break code". So it's staying buggy.)

There are a number of languages that take the Texan RAM-guzzling
approach of storing all strings in UTF-32; Python (since version 3.3)
is among a *very* small number of languages that store strings in
multiple different ways according to their content. That's described
in PEP 393 [1], titled "Flexible String Representation". It details a
means whereby a Python string will be represented in, effectively,
UTF-32 with some of the leading zero bytes elided. Or if you prefer,
in either Latin-1, UCS-2, or UCS-4, whichever's the smallest it can
fit in. The difference between a string stored one-byte-per-character
and a string stored four-bytes-per-character is almost invisible to a
Python script; you can find out by checking the string's memory usage,
but otherwise you don't need to worry about it.

Python 3.3.0 (v3.3.0:bd8afb90ebf2, Sep 29 2012, 10:55:48) [MSC v.1600
32 bit (Intel)] on win32
>>> sys.getsizeof("asdfasdfasdfasd")
40
>>> sys.getsizeof("asdfasdfasdfasdf")
41

Adding another character adds another 1 byte. (There's quite a bit of
overhead for small strings - GC headers and such - but it gets dwarfed
by the actual content after a while.)

>>> sys.getsizeof("\u1000sdfasdfasdfasd")
68
>>> sys.getsizeof("\u1000sdfasdfasdfasdf")
70

Two bytes to add another character.

>>> sys.getsizeof("\U00010001sdfasdfasdfasd")
100
>>> sys.getsizeof("\U00010001sdfasdfasdfasdf")
104

Four bytes. It uses only what it needs.

Strings in Python are immutable, so there's no need to worry about
up-grading or down-grading a string; there are a few optimizations
that can't be done, but they're fairly trivial. Look, I'll pull a jmf
and find a microbenchmark that makes 3.3 look worse:

2.7.4:
>>> timeit.repeat('a=u"A"*100; a+=u"\u1000"')
[0.8175005482540385, 0.789617954237201, 0.8152240019332098]
>>> timeit.repeat('a=u"A"*100; a+=u"a"')
[0.8088905154146744, 0.8123691698246631, 0.8172558244134365]

3.3.0:
>>> timeit.repeat('a=u"A"*100; a+=u"\u1000"')
[0.9623714745976031, 0.970628669281723, 0.9696310564468149]
>>> timeit.repeat('a=u"A"*100; a+=u"a"')
[0.7017891938739922, 0.7024725209339522, 0.6989539173082449]

See? It's clearly worse on the newer Python! But actually, this is an
extremely unusual situation, and 3.3 outperforms 2.7 on the more
common case (where the two strings are of the same width).

Python's PEP 393 strings are following the same sort of model as the
native string type in a semantically-similar but
syntactically-different language, Pike. In Pike (also free software,
like Python), the string type can be indexed character by character,
and each character can be anything in the Unicode range; and just as
in Python 3.3, memory usage goes up by just one byte if every
character in the string fits inside 8 bits. So it's not as if this is
an untested notion; Pike has been running like this for years (I don't
know how long it's had this functionality, but it won't be more than
18 years as Unicode didn't have multiple planes until 1996), and
performance has been *just fine* for all that time. Pike tends to be
run on servers, so memory usage and computation speed translate fairly
directly into TPS. And there are some sizeable commercial entities
using and developing Pike, so if the flexible string representatio

Re: GeoIP2 for retrieving city and region ?

[Νικόλας]

I know i have asked before but hwta i get is ISP city not visitors 
precise city.


GeoLiteCity.dat isnt accurate that's why it comes for free.
i must somehow get access to GeoIPCity.dat which is the full version.

And of course it can be done, i dont want to believe that it cant.

When visiting http://www.geoiptool.com/en/__ip_info/ it pinpoints my 
_exact_ city of living, not the ISP's.
It did not even ask me to allow a geop ip javascript to run it present 
sit instantly.


So, it certainly is possible if only one can find the correct database 
to use.


So, my question now is, if there is some way we can get an accurate Geo 
City database.


--
What is now proved was at first only imagined!
--
http://mail.python.org/mailman/listinfo/python-list

Re: GeoIP2 for retrieving city and region ?

[Grant Edwards]

On 2013-07-12, ??  wrote:

> I know i have asked before but hwta i get is ISP city not visitors 
> precise city.

You can't reliably do that.

> GeoLiteCity.dat isnt accurate that's why it comes for free. i must
> somehow get access to GeoIPCity.dat which is the full version.
>
> And of course it can be done, i dont want to believe that it cant.

Believe what you want.

> When visiting http://www.geoiptool.com/en/__ip_info/ it pinpoints my 
> _exact_ city of living, not the ISP's. It did not even ask me to
> allow a geop ip javascript to run it present sit instantly.

So you've reached your conclusion on a sample size of one?

-- 
Grant Edwards   grant.b.edwardsYow! I'm encased in the
  at   lining of a pure pork
  gmail.comsausage!!
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: GeoIP2 for retrieving city and region ?

[Chris Angelico]

On Sat, Jul 13, 2013 at 1:32 AM, Dave Angel  wrote:
>>
>> There is definately i way to identify the users location based solely on
>> its ip address as this site does it: http://www.geoiptool.com/
>>
>
> Sure, and as long as you don't mind it being 1000 miles off, you too can
> claim to do it too.  When I go to that site, the little pin is in Kansas,
> which is 1100 miles from where I live on the east coast of the US.

I have two IPs at this house, not counting the ones I could get off
mobile connections (which are valid probably anywhere in the state,
maybe further afield). One of them is plotted fairly accurately (not
more than a couple of kilometers wrong), but the other is listed at
Elizabeth and Bourke in the CBD... which is half an hour's train
journey away from me. And the one that was wrong was the one that's
actually an official static IP address (as opposed to a "technically
dynamic but hasn't changed for a couple of years" address).

Obligatory XKCD link: http://xkcd.com/713/

ChrisA
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: GeoIP2 for retrieving city and region ?

[Chris Angelico]

On Sat, Jul 13, 2013 at 2:38 AM, Grant Edwards  wrote:
> On 2013-07-12, ??  wrote:
>> When visiting http://www.geoiptool.com/en/__ip_info/ it pinpoints my
>> _exact_ city of living, not the ISP's. It did not even ask me to
>> allow a geop ip javascript to run it present sit instantly.
>
> So you've reached your conclusion on a sample size of one?

This is Nikos. He doesn't read responses properly, doesn't do his
research, and has (by his own admission) an iron head that doesn't let
information cross it lightly. Yes, he reached his conclusion on a
sample size of one.

Oh, and just for laughs, I tried a few of my recent mobile IP
addresses in the GeoIP lookup. All of them quoted Melbourne someplace,
some in the CBD and some out in the suburbs, but all vastly wrong, and
places I haven't been. But I'd never expect it to be accurate on
those.

ChrisA
-- 
http://mail.python.org/mailman/listinfo/python-list

UTF-EBCDIC encoding?

[Wayne Werner]

Is anyone aware of a UTF-EBCDIC[1] decoder?

While Python does have a few EBCDIC dialects in the codecs, it does not 
have the (relatively new?) UTF-EBCDIC one.


Additionally, if anyone is aware of a Python tool that can unpack a 
mainframe PDS file, that would also be worthwhile.



Thanks,
Wayne

[1]: https://en.wikipedia.org/wiki/UTF-EBCDIC
--
http://mail.python.org/mailman/listinfo/python-list

Re: UTF-EBCDIC encoding?

[Joel Goldstick]

On Fri, Jul 12, 2013 at 2:11 PM, Wayne Werner  wrote:

> Is anyone aware of a UTF-EBCDIC[1] decoder?
>
> While Python does have a few EBCDIC dialects in the codecs, it does not
> have the (relatively new?) UTF-EBCDIC one.
>
> Additionally, if anyone is aware of a Python tool that can unpack a
> mainframe PDS file, that would also be worthwhile.
>
>
> Thanks,
> Wayne
>
> [1]: 
> https://en.wikipedia.org/wiki/**UTF-EBCDIC
> --
> http://mail.python.org/**mailman/listinfo/python-list
>


I can't help you.  I'm astonished.  Trying to imagine the work environment
where this technology would be necessary

-- 
Joel Goldstick
http://joelgoldstick.com
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: UTF-EBCDIC encoding?

[Skip Montanaro]

> I can't help you.  I'm astonished.  Trying to imagine the work environment
> where this technology would be necessary

http://www.iseriespython.com/app/ispMain.py/Start?job=Home

Skip
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: UTF-EBCDIC encoding?

[Joel Goldstick]

On Fri, Jul 12, 2013 at 3:12 PM, Skip Montanaro  wrote:

> > I can't help you.  I'm astonished.  Trying to imagine the work
> environment
> > where this technology would be necessary
>
> http://www.iseriespython.com/app/ispMain.py/Start?job=Home
>
> Skip
>
I remember the AS400 series.. although I never worked with one.  What kind
of business still use that stuff? Is it for large corporation accounting,
MIS stuff?


-- 
Joel Goldstick
http://joelgoldstick.com
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: GeoIP2 for retrieving city and region ?

[Joel Goldstick]

On Fri, Jul 12, 2013 at 11:52 AM, Νικόλας  wrote:

> Στις 12/7/2013 6:32 μμ, ο/η Dave Angel έγραψε:
>
>
>> I suggest you read that geoiptool site, in particular the page
>>
>> http://www.geoiptool.com/en/**ip_info/
>>
>> There is some misinformation, but notice carefully the part about
>> dynamic IP addresses.  Probably 99% of the individual users on the web
>> (the ones using a browser) have dynamic IP addresses.  The fixed ones
>> are needed by servers, and especially for DNS use, where the name lookup
>> wants to be stable for relatively log periods of time.
>>
>
>
> I did, for me it gives exact city location and not the ISP's city location.
>
> I dont know whay for you ti just says Kansas, it shoudln't, since it
> susing longitute and latitude, it should have been accurate.
>
>
> Nikos, this is the point where you (again) loose credibility on this
list.  You asked this question about how to find the location where someone
is browsing your site from.  You got several answers.  The bottom line is
that you can't know that location unless the browsing machine does its own
geo location (think cell phones) or the user has provided his location via
some form.

So, now, a week or two later you come back with the same question, as if it
hasn't already been answered -- but you give it a 'red herring' kind of
twist.  As I understand it, you found a new module that you think will do
something that all parties answering this question before explained that
this is not possible.  And you switch up to say "why can't I load this
thing?".  Well, I don't why you can't load it, but I don't want to help
because it doesn't help you solve your stated problem.  Your stated problem
is done.  move on.




> --
> What is now proved was at first only imagined!
> --
> http://mail.python.org/**mailman/listinfo/python-list
>



-- 
Joel Goldstick
http://joelgoldstick.com
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: GeoIP2 for retrieving city and region ?

[Ian Kelly]

On Fri, Jul 12, 2013 at 10:32 AM, Νικόλας  wrote:
>
> I know i have asked before but hwta i get is ISP city not visitors precise
> city.
>
> GeoLiteCity.dat isnt accurate that's why it comes for free.
> i must somehow get access to GeoIPCity.dat which is the full version.
>
> And of course it can be done, i dont want to believe that it cant.
>
> When visiting http://www.geoiptool.com/en/__ip_info/ it pinpoints my _exact_
> city of living, not the ISP's.
> It did not even ask me to allow a geop ip javascript to run it present sit
> instantly.

Try this:

1) Go to http://incloak.com (or any other free web proxy site).
2) Paste in the URL http://www.geoiptool.com and press Enter
3) See where it thinks you are now.

When I tried it, it placed me on the wrong side of the Atlantic Ocean.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Question about mailing list rules

[Devyn Collier Johnson]

On 07/12/2013 07:11 AM, Chris “Kwpolska” Warrick wrote:

On Fri, Jul 12, 2013 at 12:47 PM, Chris Angelico  wrote:

On Fri, Jul 12, 2013 at 8:44 PM, Devyn Collier Johnson
 wrote:

I am going to love this mailing list even more.

Really, only Python code? I wanted to ask Python users about Perl! (^u^)

Devyn Collier Johnson

Heh. You'd be surprised what comes up. If it's at least broadly
related to Python (maybe you're comparing constructs in Python and
Perl??), it'll probably be accepted. And even stuff that's completely
off-topic does at times get discussed.

One small thing, though. Please avoid top-posting; the convention on
this list is to quote text above, and write yours underneath. Makes it
easier to follow the flow of conversation. Thanks!

ChrisA
--
http://mail.python.org/mailman/listinfo/python-list

One more thing Devyn should do is watch the “To:” field and make sure
it says python-list@python.org, because the above message was sent to
Chris only, and that is not what should happen most of the time.
Another option is to use Reply All, but it will make old Usenet hags
angry, because they would get two copies.

--
Kwpolska  | GPG KEY: 5EAAEA16
stop html mail| always bottom-post
http://asciiribbon.org| http://caliburn.nl/topposting.html

Thank you and sorry about that.

Kwpolska, I noticed your email shows "stop html mail" at the bottom. I 
have Thunderbird setup to use HTML mail. Are my emails coming up as 
plain text or HTML on this mailing list?


Devyn Collier Johnson
--
http://mail.python.org/mailman/listinfo/python-list

Re: RE Module Performance

[Devyn Collier Johnson]

On 07/12/2013 12:21 PM, Chris Angelico wrote:

On Fri, Jul 12, 2013 at 8:45 PM, Devyn Collier Johnson
 wrote:

Could you explain what you mean? What and where is the new Flexible String
Representation?

(You're top-posting again. Please put your text underneath what you're
responding to - it helps maintain flow and structure.)

Python versions up to and including 3.2 came in two varieties: narrow
builds (commonly found on Windows) and wide builds (commonly found on
Linux). Narrow builds internally represented Unicode strings in
UTF-16, while wide builds used UTF-32. This is a problem, because it
means that taking a program from one to another actually changes its
behaviour:

Python 2.6.4 (r264:75706, Dec  7 2009, 18:45:15)
[GCC 4.4.1] on linux2
Type "help", "copyright", "credits" or "license" for more information.

len(u"\U00012345")

1

Python 2.7.4 (default, Apr  6 2013, 19:54:46) [MSC v.1500 32 bit
(Intel)] on win32

len(u"\U00012345")

2

In fact, the narrow builds are flat-out buggy, because you can put
something in as a single character that simply isn't a single
character. You can then pull that out as two characters and make a
huge mess of things:


s=u"\U00012345"
s[0]

u'\ud808'

s[1]

u'\udf45'

*Any* string indexing will be broken if there is a single character

U+ ahead of it in the string.

Now, this problem is not unique to Python. Heaps of other languages
have the same issue, the same buggy behaviour, the same compromises.
What's special about Python is that it actually managed to come back
from that problem. (Google's V8 JavaScript engine, for instance, is
stuck with it, because the ECMAScript specification demands UTF-16. I
asked on an ECMAScript list and was told "can't change that, it'd
break code". So it's staying buggy.)

There are a number of languages that take the Texan RAM-guzzling
approach of storing all strings in UTF-32; Python (since version 3.3)
is among a *very* small number of languages that store strings in
multiple different ways according to their content. That's described
in PEP 393 [1], titled "Flexible String Representation". It details a
means whereby a Python string will be represented in, effectively,
UTF-32 with some of the leading zero bytes elided. Or if you prefer,
in either Latin-1, UCS-2, or UCS-4, whichever's the smallest it can
fit in. The difference between a string stored one-byte-per-character
and a string stored four-bytes-per-character is almost invisible to a
Python script; you can find out by checking the string's memory usage,
but otherwise you don't need to worry about it.

Python 3.3.0 (v3.3.0:bd8afb90ebf2, Sep 29 2012, 10:55:48) [MSC v.1600
32 bit (Intel)] on win32

sys.getsizeof("asdfasdfasdfasd")

40

sys.getsizeof("asdfasdfasdfasdf")

41

Adding another character adds another 1 byte. (There's quite a bit of
overhead for small strings - GC headers and such - but it gets dwarfed
by the actual content after a while.)


sys.getsizeof("\u1000sdfasdfasdfasd")

68

sys.getsizeof("\u1000sdfasdfasdfasdf")

70

Two bytes to add another character.


sys.getsizeof("\U00010001sdfasdfasdfasd")

100

sys.getsizeof("\U00010001sdfasdfasdfasdf")

104

Four bytes. It uses only what it needs.

Strings in Python are immutable, so there's no need to worry about
up-grading or down-grading a string; there are a few optimizations
that can't be done, but they're fairly trivial. Look, I'll pull a jmf
and find a microbenchmark that makes 3.3 look worse:

2.7.4:

timeit.repeat('a=u"A"*100; a+=u"\u1000"')

[0.8175005482540385, 0.789617954237201, 0.8152240019332098]

timeit.repeat('a=u"A"*100; a+=u"a"')

[0.8088905154146744, 0.8123691698246631, 0.8172558244134365]

3.3.0:

timeit.repeat('a=u"A"*100; a+=u"\u1000"')

[0.9623714745976031, 0.970628669281723, 0.9696310564468149]

timeit.repeat('a=u"A"*100; a+=u"a"')

[0.7017891938739922, 0.7024725209339522, 0.6989539173082449]

See? It's clearly worse on the newer Python! But actually, this is an
extremely unusual situation, and 3.3 outperforms 2.7 on the more
common case (where the two strings are of the same width).

Python's PEP 393 strings are following the same sort of model as the
native string type in a semantically-similar but
syntactically-different language, Pike. In Pike (also free software,
like Python), the string type can be indexed character by character,
and each character can be anything in the Unicode range; and just as
in Python 3.3, memory usage goes up by just one byte if every
character in the string fits inside 8 bits. So it's not as if this is
an untested notion; Pike has been running like this for years (I don't
know how long it's had this functionality, but it won't be more than
18 years as Unicode didn't have multiple planes until 1996), and
performance has been *just fine* for all that time. Pike tends to be
run on servers, so memory usage and computation speed translate fairly
directly into TPS. And there are some sizeable commercial entities
using and developing Pike, so if the flexible string

Re: hex dump w/ or w/out utf-8 chars

[wxjmfauth]

Le vendredi 12 juillet 2013 05:18:44 UTC+2, Steven D'Aprano a écrit :
> On Thu, 11 Jul 2013 11:42:26 -0700, wxjmfauth wrote:
> 
> 
> Now all your strings will be just as heavy, every single variable name 
> 
> and attribute name will use four times as much memory. Happy now?
> 



>>> 㑖 = 999
>>> class C:
... cœur = 'heart'
...

- Why always this magic number "four"?
- Are you able to think once non-ascii?
- Have you once had the feeling to be penalized,
because you are using fonts with OpenType technology?
- Have once had problem with pdf? I can tell you,
utf32 is peanuts compared to the used CID-font you
are using.
- Did you toy once with a unicode TeX engine?
- Did you take a look at a rendering engine code like HarfBuzz?


jmf
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: hex dump w/ or w/out utf-8 chars

[Joshua Landau]

On 9 July 2013 10:34,  wrote:

> There is no symbole for radian because mathematically
> radian is a pure number, a unitless number. You can
> hower sepecify a = ... in radian (rad).
>

Isn't a superscript "c" the symbol for radians?
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: GeoIP2 for retrieving city and region ?

[Terry Reedy]

On 7/12/2013 1:19 PM, Ian Kelly wrote:


Try this:

1) Go to http://incloak.com (or any other free web proxy site).
2) Paste in the URL http://www.geoiptool.com and press Enter
3) See where it thinks you are now.

When I tried it, it placed me on the wrong side of the Atlantic Ocean.


Me to. Thanks for the link.

--
Terry Jan Reedy

--
http://mail.python.org/mailman/listinfo/python-list

Re: GeoIP2 for retrieving city and region ?

[MRAB]

On 12/07/2013 17:32, Νικόλας wrote:


I know i have asked before but hwta i get is ISP city not visitors
precise city.

GeoLiteCity.dat isnt accurate that's why it comes for free.
i must somehow get access to GeoIPCity.dat which is the full version.

And of course it can be done, i dont want to believe that it cant.

When visiting http://www.geoiptool.com/en/__ip_info/ it pinpoints my
_exact_ city of living, not the ISP's.


Have you considered that your ISP might be in the same city as you?

According to geoiptool, my ISP is near Leeds, UK, but the important
point is that _I'm not_.


It did not even ask me to allow a geop ip javascript to run it present
sit instantly.

So, it certainly is possible if only one can find the correct database
to use.

So, my question now is, if there is some way we can get an accurate Geo
City database.



--
http://mail.python.org/mailman/listinfo/python-list

Re: Understanding other people's code

[Terry Reedy]

On 7/12/2013 10:22 AM, L O'Shea wrote:

Hi all, I've been asked to take over a project from someone else and
to extend the functionality of this. The project is written in Python
which I haven't had any real experience with (although I do really
like it) so I've spent the last week or two settling in, trying to
get my head around Python and the way in which this code works.


If the functions are not documented in prose, is there a test suite that 
you can dive into?



--
Terry Jan Reedy

--
http://mail.python.org/mailman/listinfo/python-list

Re: RE Module Performance

[Tim Delaney]

On 13 July 2013 03:58, Devyn Collier Johnson wrote:

>
> Thanks for the thorough response. I learned a lot. You should write
> articles on Python.
> I plan to spend some time optimizing the re.py module for Unix systems. I
> would love to amp up my programs that use that module.


If you are finding that regular expressions are taking too much time, have
a look at the https://pypi.python.org/pypi/re2/ and
https://pypi.python.org/pypi/regex/2013-06-26 modules to see if they
already give you enough of a speedup.

Tim Delaney
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Editor Ergonomics [was: Important features for editors]

[Eric S. Johansson]

On Fri, 12 Jul 2013 18:34:30 -0400, Dennis Lee Bieber  
 wrote:




Sounds like you might have liked an accessory I had on my Amiga.
Basically a proportional joystick feeding an interface box which  
converted

the position value into a sequence of mouse movements --


sounds very cool. Although after I wrote my little screed, I found myself  
fiddling with my smartphone and I realized I was subconsciously putting it  
through the motions if it was a positioning device.


The gross motion is detected by the accelerometer's in the phone. The fine  
positioning by fingers on the screen.


Just thinking out loud.
--
http://mail.python.org/mailman/listinfo/python-list

Re: RE Module Performance

[Michael Torrie]

On 07/12/2013 09:59 AM, Joshua Landau wrote:
> If you're interested, the basic of it is that strings now use a
> variable number of bytes to encode their values depending on whether
> values outside of the ASCII range and some other range are used, as an
> optimisation.

Variable number of bytes is a problematic way to saying it.  UTF-8 is a
variable-number-of-bytes encoding scheme where each character can be 1,
2, 4, or more bytes, depending on the unicode character.  As you can
imagine this sort of encoding scheme would be very slow to do slicing
with (looking up a character at a certain position).  Python uses
fixed-width encoding schemes, so they preserve the O(n) lookup speeds,
but python will use 1, 2, or 4 bytes per every character in the string,
depending on what is needed.  Just in case the OP might have
misunderstood what you are saying.

jmf sees the case where a string is promoted from one width to another,
and thinks that the brief slowdown in string operations to accomplish
this is a problem.  In reality I have never seen anyone use the types of
string operations his pseudo benchmarks use, and in general Python 3's
string behavior is pretty fast.  And apparently much more correct than
if jmf's ideas of unicode were implemented.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: RE Module Performance

[MRAB]

On 12/07/2013 23:16, Tim Delaney wrote:

On 13 July 2013 03:58, Devyn Collier Johnson mailto:devyncjohn...@gmail.com>> wrote:


Thanks for the thorough response. I learned a lot. You should write
articles on Python.
I plan to spend some time optimizing the re.py module for Unix
systems. I would love to amp up my programs that use that module.


If you are finding that regular expressions are taking too much time,
have a look at the https://pypi.python.org/pypi/re2/ and
https://pypi.python.org/pypi/regex/2013-06-26 modules to see if they
already give you enough of a speedup.


FYI, you're better off going to http://pypi.python.org/pypi/regex
because that will take you to the latest version.
--
http://mail.python.org/mailman/listinfo/python-list

Re: Understanding other people's code

[Joel Goldstick]

On Fri, Jul 12, 2013 at 6:11 PM, Terry Reedy  wrote:

> On 7/12/2013 10:22 AM, L O'Shea wrote:
>
>> Hi all, I've been asked to take over a project from someone else and
>> to extend the functionality of this. The project is written in Python
>> which I haven't had any real experience with (although I do really
>> like it) so I've spent the last week or two settling in, trying to
>> get my head around Python and the way in which this code works.
>>
>
> If the functions are not documented in prose, is there a test suite that
> you can dive into?
>
>
> --
> Terry Jan Reedy
>
> --
> http://mail.python.org/**mailman/listinfo/python-list
>

I'm very appreciative of pydoc. -- even for code I write myself!.  Learn
about it and redirect its output to files, so you can print out all of your
modules.  (well -- my suggestion!).  For the functions and classes that are
lacking docstrings, review them and see if you can figure out what they
do.  Add docstrings..

Not to disrespect this original coder in the slightest, but my work
experience has been involved in reading and fixing or updating lots of
other peoples code -- most less documented than would be nice.  So my def
of good code is code with good descriptive docstrings -- at the top level
even before documenting the details.  Its nice to know where the
developer's head was at when the system was put together.

-- 
Joel Goldstick
http://joelgoldstick.com
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: GeoIP2 for retrieving city and region ?

[Chris Angelico]

On Sat, Jul 13, 2013 at 9:04 AM, Dennis Lee Bieber
 wrote:
> On Sat, 13 Jul 2013 02:47:38 +1000, Chris Angelico 
> declaimed the following:
>
>>
>>Oh, and just for laughs, I tried a few of my recent mobile IP
>>addresses in the GeoIP lookup. All of them quoted Melbourne someplace,
>>some in the CBD and some out in the suburbs, but all vastly wrong, and
>>places I haven't been. But I'd never expect it to be accurate on
>>those.
>>
> Well... the MaxMind demo of "my IP" did get the proper metropolitan
> area... But they list the ISP as "AT&T"... My real ISP is Earthlink
> (piggybacking on AT&T DSL service).
>
> The Lat/Long, however shows as
>
> 42.9634 -85.6681
> whereas a recent GPS readout shows
> 42.9159 -85.5541
>
> or 2m50s too far north, and 6m50s too far west.
>
> Same website, accessed from my Blackberry phone, gave a result of
> "United States, NA" and location 38 -97

When you try to place a visitor geographically by IP address, the only
thing you can be absolutely 100% certain of is which RIR they're at
(proxies aside - you're just testing the proxy rather than the
ultimate origin). Country is also highly likely to be right, though
not certain (I've never known it to be wrong, but I've never been able
to confirm what happens with some of the small European countries -
for all I know they could share ISPs and netblocks). Anything tighter
than that is goign to be pretty hit-and-miss. But I have to say, it's
improved a lot over the years. Back in the early 2000s - say, about 8
years ago, I think - I was playing with this sort of technology, and
it placed me in Sydney. That's one state away, lots of rivalry
separating us (friendly rivalry, of course; in a country that's doing
its best to kill us all, we can't afford to really hate each other),
and roughly 750-1000km wrong by distance (depending on how you measure
- most people don't put an odometer on a crow). So at least now it
gets within the same degree of latitude and longitude... most of the
time.

ChrisA
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: hex dump w/ or w/out utf-8 chars

[Tim Roberts]

Joshua Landau  wrote:
>
>Isn't a superscript "c" the symbol for radians?

That's very rarely used.  More common is "rad".  The problem with a
superscript "c" is that it looks too much like a degree symbol.
-- 
Tim Roberts, t...@probo.com
Providenza & Boekelheide, Inc.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: hex dump w/ or w/out utf-8 chars

[Steven D'Aprano]

On Fri, 12 Jul 2013 23:01:47 +0100, Joshua Landau wrote:

> Isn't a superscript "c" the symbol for radians?


Only in the sense that a superscript "o" is the symbol for degrees.

Semantically, both degree-sign and radian-sign are different "things" 
than merely an o or c in superscript.

Nevertheless, in mathematics at least, it is normal to leave out the 
radian sign when talking about angles. By default, "1.2" means "1.2 
radians", not "1.2 degrees".


-- 
Steven
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: GeoIP2 for retrieving city and region ?

[Νικόλας]

Στις 13/7/2013 1:07 πμ, ο/η MRAB έγραψε:


Have you considered that your ISP might be in the same city as you?

According to geoiptool, my ISP is near Leeds, UK, but the important
point is that _I'm not_.



My ISP is in Athens and i live in Thessaloníki and it returned back 
Thessaloníki not Athens, which it was accurate for me.


I dont know why it was not accurate for the other members here.
And of course if you are using a proxy then the GeoIP tool has no way of 
telling the real location of the visitor but the proxy's location in stead.


Also in another page which it asked me if i allow it to run a javascipt 
Geo app and i replied positively it gave me my exact city and street 
number too!


So many sites can identify accurately the correct city and even street 
some times, so there must be a way.


--
What is now proved was at first only imagined!
--
http://mail.python.org/mailman/listinfo/python-list

Re: How do I get the OS System Font Directory(Cross-Platform) in python?

[Tim Roberts]

Metallicow  wrote:
>
>If the OS doesn't *have* a dedicated system fonts dir that is accessable 
>by the user, then I am not that much interested in dealing with it. 

Really?  Because Windows is the ONLY one of the major operating systems
that actually has a dedicated system fonts directory.  Linux doesn't even
have a dedicated windowing system.
-- 
Tim Roberts, t...@probo.com
Providenza & Boekelheide, Inc.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: RE Module Performance

[Steven D'Aprano]

On Fri, 12 Jul 2013 13:58:29 -0400, Devyn Collier Johnson wrote:

> I plan to spend some time optimizing the re.py module for Unix systems.
> I would love to amp up my programs that use that module.

In my experience, often the best way to optimize a regex is to not use it 
at all.

[steve@ando ~]$ python -m timeit -s "import re" \
> -s "data = 'a'*100+'b'" \
> "if re.search('b', data): pass"
10 loops, best of 3: 2.77 usec per loop

[steve@ando ~]$ python -m timeit -s "data = 'a'*100+'b'" \
> "if 'b' in data: pass"
100 loops, best of 3: 0.219 usec per loop

In Python, we often use plain string operations instead of regex-based 
solutions for basic tasks. Regexes are a 10lb sledge hammer. Don't use 
them for cracking peanuts.



-- 
Steven
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: GeoIP2 for retrieving city and region ?

[Νικόλας]

Στις 13/7/2013 2:04 πμ, ο/η Dennis Lee Bieber έγραψε:

On Sat, 13 Jul 2013 02:47:38 +1000, Chris Angelico 
declaimed the following:



Oh, and just for laughs, I tried a few of my recent mobile IP
addresses in the GeoIP lookup. All of them quoted Melbourne someplace,
some in the CBD and some out in the suburbs, but all vastly wrong, and
places I haven't been. But I'd never expect it to be accurate on
those.


Well... the MaxMind demo of "my IP" did get the proper metropolitan
area... But they list the ISP as "AT&T"... My real ISP is Earthlink
(piggybacking on AT&T DSL service).

The Lat/Long, however shows as

42.9634 -85.6681
whereas a recent GPS readout shows
42.9159 -85.5541

or 2m50s too far north, and 6m50s too far west.

Same website, accessed from my Blackberry phone, gave a result of
"United States, NA" and location 38 -97




I have read all your answer very carefully but i still need some way of 
getting it done.


All my Greek website visitors say they are from Europe/Athens which is 
the ISP's location and not user's homeland.


Well it worked for me but as many other told me it wasn't accurate for 
them too.


Please try this:  http://www.maxmind.com/en/geoip_demo

and tell me if maxmind's database can pippont you city's location.

Thank you.

--
What is now proved was at first only imagined!
--
http://mail.python.org/mailman/listinfo/python-list

Re: GeoIP2 for retrieving city and region ?

[Chris Angelico]

On Sat, Jul 13, 2013 at 3:48 PM, Νικόλας  wrote:
> Στις 13/7/2013 2:04 πμ, ο/η Dennis Lee Bieber έγραψε:
>>
>> On Sat, 13 Jul 2013 02:47:38 +1000, Chris Angelico 
>> declaimed the following:
>>
>>>
>>> Oh, and just for laughs, I tried a few of my recent mobile IP
>>> addresses in the GeoIP lookup. All of them quoted Melbourne someplace,
>>> some in the CBD and some out in the suburbs, but all vastly wrong, and
>>> places I haven't been. But I'd never expect it to be accurate on
>>> those.
>>>
>> Well... the MaxMind demo of "my IP" did get the proper
>> metropolitan
>> area... But they list the ISP as "AT&T"... My real ISP is Earthlink
>> (piggybacking on AT&T DSL service).
>>
>> The Lat/Long, however shows as
>>
>> 42.9634 -85.6681
>> whereas a recent GPS readout shows
>> 42.9159 -85.5541
>>
>> or 2m50s too far north, and 6m50s too far west.
>>
>> Same website, accessed from my Blackberry phone, gave a result of
>> "United States, NA" and location 38 -97
>>
>
>
> I have read all your answer very carefully but i still need some way of
> getting it done.
>
> All my Greek website visitors say they are from Europe/Athens which is the
> ISP's location and not user's homeland.
>
> Well it worked for me but as many other told me it wasn't accurate for them
> too.
>
> Please try this:  http://www.maxmind.com/en/geoip_demo
>
> and tell me if maxmind's database can pippont you city's location.

Nikos, you keep asking for a way to do the impossible. We keep telling
you that it is impossible. No alternative technique will do what
cannot be done!

I just tried that on my two IPs and it was quite wrong on both of them
- further wrong than some of the others have been.

Stop expecting magic.

ChrisA
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: GeoIP2 for retrieving city and region ?

[Νικόλας]

Στις 13/7/2013 8:53 πμ, ο/η Chris Angelico έγραψε:

On Sat, Jul 13, 2013 at 3:48 PM, ���  wrote:

 13/7/2013 2:04 ��, �/� Dennis Lee Bieber ��:


On Sat, 13 Jul 2013 02:47:38 +1000, Chris Angelico 
declaimed the following:



Oh, and just for laughs, I tried a few of my recent mobile IP
addresses in the GeoIP lookup. All of them quoted Melbourne someplace,
some in the CBD and some out in the suburbs, but all vastly wrong, and
places I haven't been. But I'd never expect it to be accurate on
those.


 Well... the MaxMind demo of "my IP" did get the proper
metropolitan
area... But they list the ISP as "AT&T"... My real ISP is Earthlink
(piggybacking on AT&T DSL service).

 The Lat/Long, however shows as

42.9634 -85.6681
whereas a recent GPS readout shows
42.9159 -85.5541

or 2m50s too far north, and 6m50s too far west.

 Same website, accessed from my Blackberry phone, gave a result of
"United States, NA" and location 38 -97




I have read all your answer very carefully but i still need some way of
getting it done.

All my Greek website visitors say they are from Europe/Athens which is the
ISP's location and not user's homeland.

Well it worked for me but as many other told me it wasn't accurate for them
too.

Please try this:  http://www.maxmind.com/en/geoip_demo

and tell me if maxmind's database can pippont you city's location.


Nikos, you keep asking for a way to do the impossible. We keep telling
you that it is impossible. No alternative technique will do what
cannot be done!

I just tried that on my two IPs and it was quite wrong on both of them
- further wrong than some of the others have been.

Stop expecting magic.

ChrisA



But it works for me, How can it be impossible and worked for me at the 
same time?


Also i tried some other website that asked me to allow it to run a 
javascript on my browser and it pinpointed even my street!


If it wasnt possbile then MaxMind would be seeling its GeoIP2 app for 
1380$ per year.


--
What is now proved was at first only imagined!
--
http://mail.python.org/mailman/listinfo/python-list

Re: GeoIP2 for retrieving city and region ?

[Chris Angelico]

On Sat, Jul 13, 2013 at 4:07 PM, Νικόλας  wrote:
> But it works for me, How can it be impossible and worked for me at the same
> time?

If I roll ten six-sided dice, will they total 35? Maybe. Maybe they'll
be close. But it's impossible to come up with a table for rolling
those dice on that will guarantee you exactly 35 every time.

ChrisA
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: GeoIP2 for retrieving city and region ?

[Νικόλας]

Στις 13/7/2013 9:22 πμ, ο/η Chris Angelico έγραψε:

On Sat, Jul 13, 2013 at 4:07 PM, ���  wrote:

But it works for me, How can it be impossible and worked for me at the same
time?


If I roll ten six-sided dice, will they total 35? Maybe. Maybe they'll
be close. But it's impossible to come up with a table for rolling
those dice on that will guarantee you exactly 35 every time.



I just had some other friends of me, who live in different cities around 
Greece to test the link i gave to you in my previous post and for all of 
them it returned the correct city of their origin.


Seems like GeoIP2 is doing a better job that its predecessor GeopIP.

--
What is now proved was at first only imagined!
--
http://mail.python.org/mailman/listinfo/python-list

Re: How do I get the OS System Font Directory(Cross-Platform) in python?

[Metallicow]

On Saturday, July 13, 2013 12:36:45 AM UTC-5, Tim Roberts wrote:
> Really?  Because Windows is the ONLY one of the major operating systems
> 
> that actually has a dedicated system fonts directory.  Linux doesn't even
> 
> have a dedicated windowing system.

So... Is it expected to install duplicates to multiple locations with Mac and 
Linux...?

-- 
http://mail.python.org/mailman/listinfo/python-list

help with explaining how to split a list of tuples into parts

[peter]

Hi List,

I am new to Python and wondering if there is a better python way to do 
something.  As a learning exercise I decided to create a python bash script to 
wrap around the Python Crypt library (Version 2.7).

My attempt is located here - https://gist.github.com/pjfoley/5989653

I am trying to wrap my head around list comprehensions, I have read the docs at 
http://docs.python.org/2/tutorial/datastructures.html#list-comprehensions and 
read various google results.  I think my lack of knowledge is making it 
difficult to know what key word to search on.

Essentially I have this list of tuples

# Tuple == (Hash Method, Salt Length, Magic String, Hashed Password Length)
supported_hashes=[('crypt',2,'',13), ('md5',8,'$1$',22), 
('sha256',16,'$5$',43), ('sha512',16,'$6$',86)]

This list contains the valid hash methods that the Crypt Library supports plus 
some lookup values I want to use in the code.

I have managed to work out how to extract a list of just the first value of 
each tuple (line 16) which I use as part of the validation against the --hash 
argparse option.

My Question.

Looking at line 27, This line returns the tuple that mataches the hash type the 
user selects from the command line.  Which I then split the seperate parts over 
lines 29 to 31.

I am wondering if there is a more efficient way to do this such that I could do:

salt_length, hash_type, expected_password_length = [x for x in supported_hashes 
if x[0] == args.hash]

>From my limited understanding the first x is the return value from the 
>function which meets the criteria.  So could I do something like:

... = [(x[0][1], x[0][2], x[0][3]) for x in supported_hashes if x[0] == 
args.hash]

I am happy to be pointed to some documentation which might help clarify what I 
need to do.  

Also if there is anything else that could be improved on with the code happy to 
be contacted off list.

Thanks,

Peter.
-- 
http://mail.python.org/mailman/listinfo/python-list