date:20150505

On Tuesday 05 May 2015 08:02, BartC wrote:

> On 04/05/2015 16:20, Cecil Westerhof wrote:
>> Potential dangerous bug introduced by programming in Python as if it
>> was C/Java. :-(
>> I used:
>>  ++tries
>> that has to be:
>>  tries += 1
> 
> I think I've come across that. It doesn't mind ++ so people are likely
> to be assume that increment works as in other languages.
> 
> I guess it just means +(+(a)).

Correct.

> But in that case, what meaning does:
> 
> a
> 
> or
> 
> a+b
> 
> have in Python? If they were function calls: a() or (a+b)(), then that's
> clear enough. But a+b doesn't do anything!

Not so.

The first one just does a name lookup and then throws the result away. The 
second one looks up names a and b, then adds them together, throwing away 
the result.

Here is one use for the first idiom:

try:
bin
except NameError:
# No built-in bin function, perhaps our Python is too old?
def bin(num):
...

Here's a good use for the second:

def calculate(x):
x + 0  # Fails if x is not a number.
...

That's an example of duck-typing. It allows any argument which supports 
addition with integers, e.g. x could be a number, or some kind of array or 
vector which supports addition with a scalar.

> (I think I would have picked up "++" and "--" as special tokens even if
> increment/decrement ops weren't supported. Just because they would
> likely cause errors through misunderstanding.)

Just because C made a mistake, doesn't mean other languages have to 
slavishly follow it.

-- 
Steven

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: when does newlines get set in universal newlines mode?

On Monday 04 May 2015 22:13, Chris Angelico wrote:

> It may be worth documenting this limitation, but it's not something
> that can easily be fixed without removing support for \r newlines -
> although that might be an option, given that non-OSX Macs are
> basically history now.

Non-OSX Macs are history, but the text files they created are not.



-- 
Steve

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: when does newlines get set in universal newlines mode?

On Tue, May 5, 2015 at 6:31 PM, Steven D'Aprano
 wrote:
> On Monday 04 May 2015 22:13, Chris Angelico wrote:
>
>> It may be worth documenting this limitation, but it's not something
>> that can easily be fixed without removing support for \r newlines -
>> although that might be an option, given that non-OSX Macs are
>> basically history now.
>
> Non-OSX Macs are history, but the text files they created are not.

True. Like I said, "might be". I could imagine, for instance, a caveat
being put on the newlines attribute saying that "when reading a file
delimited by \r, this may erroneously imply that \r\n is being used,
until a second line has been read". But that's probably more
complexity than it's worth.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Step further with filebasedMessages

I now defined get_message_slice:
### Add step
def get_message_slice(message_filename, start, end):
"""
Get a slice of messages, where 0 is the first message
Works with negative indexes
The values can be ascending and descending
"""

message_list= []
real_file   = expanduser(message_filename)
nr_of_messages  = get_nr_of_messages(real_file)
if start < 0:
start += nr_of_messages
if end < 0:
end += nr_of_messages
assert (start >= 0) and (start < nr_of_messages)
assert (end   >= 0) and (end   < nr_of_messages)
if start > end:
tmp = start
start   = end
end = tmp
need_reverse= True
else:
need_reverse= False
with open(real_file, 'r') as f:
for message in islice(f, start, end + 1):
message_list.append(message.rstrip())
if need_reverse:
message_list.reverse()
return message_list

Is that a good way?

I also had:
def get_indexed_message(message_filename, index):
"""
Get index message from a file, where 0 gets the first message
A negative index gets messages indexed from the end of the file
Use get_nr_of_messages to get the number of messages in the file
"""

real_file   = expanduser(message_filename)
nr_of_messages  = get_nr_of_messages(real_file)
if index < 0:
index += nr_of_messages
assert (index >= 0) and (index < nr_of_messages)
with open(real_file, 'r') as f:
[line] = islice(f, index, index + 1)
return line.rstrip()

But changed it to:
def get_indexed_message(message_filename, index):
"""
Get index message from a file, where 0 gets the first message
A negative index gets messages indexed from the end of the file
Use get_nr_of_messages to get the number of messages in the file
"""

return get_message_slice(message_filename, index, index)[0]

Is that acceptable? I am a proponent of DRY.
Or should I at least keep the assert in it?

-- 
Cecil Westerhof
Senior Software Engineer
LinkedIn: http://www.linkedin.com/in/cecilwesterhof
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Step further with filebasedMessages

On Tue, May 5, 2015 at 6:52 PM, Cecil Westerhof  wrote:
> I now defined get_message_slice:

You're doing a lot of work involving flat-file storage of sequential
data. There are two possibilities:

1) Your files are small, so you shouldn't concern yourself with
details at all - just do whatever looks reasonable, nothing will
matter; or
2) Your files are bigger than that, performance might be a problem
(especially when your Big Oh starts looking bad), and you should move
to a database.

Maybe even with small files, a database would be cleaner. You can grab
whichever rows you want based on their IDs, and the database will do
the work for you. Grab SQLite3 or PostgreSQL, give it a whirl - you
may find that it does everything you need, right out of the box.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: when does newlines get set in universal newlines mode?

2015-05-05 Thread Davide Mancusi

I just opened a bug report:

http://bugs.python.org/issue24126

We'll see what they say.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: when does newlines get set in universal newlines mode?

On Tue, May 5, 2015 at 7:23 PM, Davide Mancusi  wrote:
> I just opened a bug report:
>
> http://bugs.python.org/issue24126
>
> We'll see what they say.

Cool. I suggest posting in the tracker thread the exact Python
version(s) you've tested this with, in case it matters.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Is it normal to cry when given XML?

2015-05-05 Thread Sayth Renshaw

Hi

Just checking if the reaction to cry when given XML is normal. 

I thought maybe I am approaching it all wrong, using lxml largely or some 
xquery to club it into submission. 

See the usual goal is just to take the entire XML and push it into a database. 
or in future experiment with Mongo or Hdf5 .

See its never basic xml, usually comes from some database with a walk of tables 
and strange relationships.

Am I doing it wrong is there a simple way I am missing? 

Sayth
-- 
https://mail.python.org/mailman/listinfo/python-list

Exception in thread Thread-4:

2015-05-05 Thread david jhon

Hello everyone,

I am initializing lock and threading related variables in __init__() method
of the class as follows:

from threading import Timer, Lock

class miTestClass(EventMixin):
def __init__(self, t, r, bw):
 self.statMonitorLock = Lock() #to lock the multi access threads
 self.statMonitorLock.acquire()
 statMonitorTimer = Timer(10.0, self._collectFlowStats()) #timer to
collect stats
 statMonitorTimer.start()

but I am getting following error:

Exception in thread Thread-4:
Traceback (most recent call last):
  File "/usr/lib/python2.7/threading.py", line 810, in __bootstrap_inner
self.run()
  File "/usr/lib/python2.7/threading.py", line 1082, in run
self.function(*self.args, **self.kwargs)
TypeError: 'NoneType' object is not callable

I am new to python and after reading various links, I am not able to
resolve this error.
I hope anybody here could help me get it fixed. Many thanks in advance!

Regards,
David
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Is it normal to cry when given XML?

On Tue, May 5, 2015 at 7:28 PM, Sayth Renshaw  wrote:
> Hi
>
> Just checking if the reaction to cry when given XML is normal.

It's not unsurprising, especially with bad XML structures.

> I thought maybe I am approaching it all wrong, using lxml largely or some 
> xquery to club it into submission.
>
> See the usual goal is just to take the entire XML and push it into a 
> database. or in future experiment with Mongo or Hdf5 .
>
> See its never basic xml, usually comes from some database with a walk of 
> tables and strange relationships.
>
> Am I doing it wrong is there a simple way I am missing?

Generally, I work with XML only as a transport layer; and most of the
time, it's for a document structure that would be better served by
JSON anyway. (This may mean that I have an unfairly negative view of
XML, but it's extremely common.) My usual technique is to parse it
into something native (usually a dictionary - and probably the same
structure that the other end constructed the XML from), then work with
that. For example, querying the ePond API [1] gives back a pile of XML
data, so I might have a single function that performs a synchronous
HTTP query, takes the response body, parses it using a fairly generic
XML parser like lxml, then digs three levels into the resulting tree
to pull out the bit that actually matters, leaving behind all the
framing and stuff.

The less time you spend with actual XML, the better. XML is not the answer.

ChrisA

[1] A completely fictional web site, of course, and in no way implying
that I have had a frustrating time with a well-known online
sales/auction company.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Exception in thread Thread-4:

On Tue, May 5, 2015 at 7:23 PM, david jhon  wrote:
> from threading import Timer, Lock
>
> class miTestClass(EventMixin):
> def __init__(self, t, r, bw):
>  self.statMonitorLock = Lock() #to lock the multi access threads
>  self.statMonitorLock.acquire()
>  statMonitorTimer = Timer(10.0, self._collectFlowStats()) #timer to
> collect stats
>  statMonitorTimer.start()
>
> but I am getting following error:
>
> Exception in thread Thread-4:
> Traceback (most recent call last):
>   File "/usr/lib/python2.7/threading.py", line 810, in __bootstrap_inner
> self.run()
>   File "/usr/lib/python2.7/threading.py", line 1082, in run
> self.function(*self.args, **self.kwargs)
> TypeError: 'NoneType' object is not callable

The Timer() class will call a function when it's ready. To use that,
you want to pass it a function. Try putting these two lines of code
into your __init__ function:

print(self._collectFlowStats)
print(self._collectFlowStats())

(Aside: This is something I like to call "IIDPIO debugging": If In
Doubt, Print It Out. You sometimes have more sophisticated debugging
techniques available, but you hardly ever are unable to basic 'print',
in some form or another. It's incredibly useful.)

The first one will print out something like this:

>

The second will actually call that function (which may or may not do
anything visible), and then print out:

None

If you change your _collectFlowStats function a bit, you can see even
more of what's happening. Something like this:

def _collectFlowStats(self):
print("_collectFlowStats has been called. Returning 42...")
return 42

Then you'll see that it gets called, and does its print, and then 42
gets printed out at the end.

In your case, simply removing the parentheses from
self._collectFlowStats should do what you want - the Timer constructor
will be passed a function (in this case, a bound method, but same
same), and that function won't be called until the timer is up.

Hope that helps!

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Is it normal to cry when given XML?

On Tuesday 05 May 2015 19:28, Sayth Renshaw wrote:

> Just checking if the reaction to cry when given XML is normal.


Cry? When people give me XML, sometimes I lose control of my bladder.


-- 
Steve

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: GAE environment differences

2015-05-05 Thread Robin Becker


On 02/05/2015 10:14, Kev Dwyer wrote:

Robin Becker wrote:




```

the user suggests that even though claims are made that you can use a
filesystem, but stuff like pwd is missing. Apparently the user module has
no meaning, but there is a users module? I guess I'll need to keep
patching reportlab when GAE users find these glitches.


For what it's worth, we use reportlab on GAE to generate a simple pdf
and the above error is the only one that I've encountered.  For us it was
enough to trap the ImportError.

Thanks for all your work on reportlab,

Kev

the original reporter suggested originally that a KeyError was raised, but I 
think that maybe some difference between development environments and the live GAE.

--
Robin Becker

--
https://mail.python.org/mailman/listinfo/python-list

Re: Is it normal to cry when given XML?

2015-05-05 Thread Adam Tauno Williams

On Tue, 2015-05-05 at 02:28 -0700, Sayth Renshaw wrote:
> Just checking if the reaction to cry when given XML is normal. 

No, not at all.  I leap for ecstatic joy when given, as all to rarely
happens, XML.  Rather than someone's turdy text [which includes JSON]
file.  I wish all 1,200+ of my vendors and suppliers would provide their
data in XML rather than the random swirl of bizarre crap I receive.

> I thought maybe I am approaching it all wrong, using lxml largely or 
> some xquery to club it into submission. 

I do most of my processing with LXML, XSLT, and XPath.  Fast, efficient,
reliable, works great.  And it is easy to abstract these tools to
automate what I need to do over and over again.

> See the usual goal is just to take the entire XML and push it into a
> database. 

I do a lot of pushing into an SQL database, no problem.  XSLT does 99.4%
of the work.

> See its never basic xml, 

"basic xml"? 

> usually comes from some database with a walk of tables and strange 
> relationships.

No problem.

> Am I doing it wrong is there a simple way I am missing? 

I suspect so.  Data is easily transformed into

abc

stanley

 ...

And then that data can be processes as SQL insert/update/delete/upsert
operations while being swung around on the end of a stick while you
dance.

XML is the ultimate file "format".  It solves problems, much better than
everyone trying to re-engineer their needs INTO a format [like JSON].

-- 
Adam Tauno Williams  GPG D95ED383
Systems Administrator, Python Developer, LPI / NCLA

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Step further with filebasedMessages

On Tuesday 05 May 2015 18:52, Cecil Westerhof wrote:

> I now defined get_message_slice:
> ### Add step
> def get_message_slice(message_filename, start, end):
> """
> Get a slice of messages, where 0 is the first message
> Works with negative indexes
> The values can be ascending and descending
> """

What's a message in this context? Just a line of text?

> message_list= []
> real_file   = expanduser(message_filename)
> nr_of_messages  = get_nr_of_messages(real_file)
> if start < 0:
> start += nr_of_messages
> if end < 0:
> end += nr_of_messages
> assert (start >= 0) and (start < nr_of_messages)
> assert (end   >= 0) and (end   < nr_of_messages)

You can write that as:

assert 0 <= start < nr_of_messages

except you probably shouldn't, because that's not a good use for assert: 
start and end are user-supplied parameters, not internal invariants.

You might find this useful for understanding when to use assert:

http://import-that.dreamwidth.org/676.html

> if start > end:
> tmp = start
> start   = end
> end = tmp
> need_reverse= True

You can swap two variables like this:

start, end = end, start

The language guarantees that the right hand side will be evaluated before 
the assignments are done, so it will automatically do the right thing.

> else:
> need_reverse= False

Your behaviour when start and end are in opposite order does not match the 
standard slicing behaviour:

py> "abcdef"[3:5]
'de'
py> "abcdef"[5:3]
''

That doesn't mean your behaviour is wrong, but it will surprise anyone who 
expects your slicing to be like the slicing they are used to.

> with open(real_file, 'r') as f:
> for message in islice(f, start, end + 1):
> message_list.append(message.rstrip())
> if need_reverse:
> message_list.reverse()
> return message_list
> 
> Is that a good way?

I think what I would do is:

def get_message_slice(message_filename, start=0, end=None, step=1):
real_file = expanduser(message_filename)
with open(real_file, 'r') as f:
messages = f.readlines()
return messages[start:end:step]

until such time that I could prove that I needed something more 
sophisticated. Then, and only then, would I consider your approach, except 
using a slice object:

# Untested.
def get_message_slice(message_filename, start=0, end=None, step=1):
real_file = expanduser(message_filename)
messages = []
# FIXME: I assume this is expensive. Can we avoid it?
nr_of_messages = get_nr_of_messages(real_file)
the_slice = slice(start, end, step)
# Calculate the indexes in the given slice, e.g.
# start=1, stop=7, step=2 gives [1,3,5].
indices = range(*(the_slice.indices(nr_of_messages)))
with open(real_file, 'r') as f:
for i, message in enumerate(f):
if i in indices:
messages.append(message)
return messages

There is still room for optimization: e.g. if the slice is empty, don't 
bother iterating over the file. I leave that to you.

-- 
Steve

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Step further with filebasedMessages

Op Tuesday 5 May 2015 11:20 CEST schreef Chris Angelico:

> On Tue, May 5, 2015 at 6:52 PM, Cecil Westerhof  wrote:
>> I now defined get_message_slice:
>
> You're doing a lot of work involving flat-file storage of sequential
> data. There are two possibilities:
>
> 1) Your files are small, so you shouldn't concern yourself with
> details at all - just do whatever looks reasonable, nothing will
> matter;

In my case the files are very small. Biggest is 150 lines. But if you
publish your code, you never know in which situation it will be used.

The suggestion implement a slice came from this newsgroup. ;-) And I
found it a good idea.

It is also to get reacquainted with Python.

> or 2) Your files are bigger than that, performance might be
> a problem (especially when your Big Oh starts looking bad), and you
> should move to a database.
>
> Maybe even with small files, a database would be cleaner. You can
> grab whichever rows you want based on their IDs, and the database
> will do the work for you. Grab SQLite3 or PostgreSQL, give it a
> whirl - you may find that it does everything you need, right out of
> the box.

Is a next step. I want to use PostgreSQL with SQLAlchemy.

-- 
Cecil Westerhof
Senior Software Engineer
LinkedIn: http://www.linkedin.com/in/cecilwesterhof
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: when does newlines get set in universal newlines mode?

2015-05-05 Thread Davide Mancusi

> Cool. I suggest posting in the tracker thread the exact Python
> version(s) you've tested this with, in case it matters.

Done. Good point.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Step further with filebasedMessages

2015-05-05 Thread Peter Otten

Cecil Westerhof wrote:

> I now defined get_message_slice:
> ### Add step
> def get_message_slice(message_filename, start, end):

Intervals are usually half-open in Python. I recommend that you follow that 
convention.

> """
> Get a slice of messages, where 0 is the first message
> Works with negative indexes
> The values can be ascending and descending
> """
> 
> message_list= []
> real_file   = expanduser(message_filename)
> nr_of_messages  = get_nr_of_messages(real_file)
> if start < 0:
> start += nr_of_messages
> if end < 0:
> end += nr_of_messages
> assert (start >= 0) and (start < nr_of_messages)

You should raise an exception. While asserts are rarely switched off in 
Python you still have to be prepeared, and an IndexError would be a better 
fit anyway.

> assert (end   >= 0) and (end   < nr_of_messages)
> if start > end:
> tmp = start
> start   = end
> end = tmp
> need_reverse= True
> else:
> need_reverse= False
> with open(real_file, 'r') as f:
> for message in islice(f, start, end + 1):
> message_list.append(message.rstrip())
> if need_reverse:
> message_list.reverse()
> return message_list
> 
> Is that a good way?
> 
> I also had:
> def get_indexed_message(message_filename, index):
> """
> Get index message from a file, where 0 gets the first message
> A negative index gets messages indexed from the end of the file
> Use get_nr_of_messages to get the number of messages in the file
> """
> 
> real_file   = expanduser(message_filename)
> nr_of_messages  = get_nr_of_messages(real_file)
> if index < 0:
> index += nr_of_messages
> assert (index >= 0) and (index < nr_of_messages)
> with open(real_file, 'r') as f:
> [line] = islice(f, index, index + 1)
> return line.rstrip()
> 
> But changed it to:
> def get_indexed_message(message_filename, index):
> """
> Get index message from a file, where 0 gets the first message
> A negative index gets messages indexed from the end of the file
> Use get_nr_of_messages to get the number of messages in the file
> """
> 
> return get_message_slice(message_filename, index, index)[0]
> 
> Is that acceptable? 

Yes.

> I am a proponent of DRY.

But note that you are implementing parts of the slicing logic that Python's 
sequence already has. Consider becoming a pronent of DRWTOGAD*.

> Or should I at least keep the assert in it?
 
No.

I see you have a tendency to overengineer. Here's
how I would approach case (1) in Chris' answer, where memory is not a 
concern:

import os

def read_messages(filename):
with open(os.path.expanduser(filename)) as f:
return [line.rstrip() for line in f]

# get_messages_slice(filename, start, end)
print(read_messages(filename)[start:end+1])

# get_indexed_message(filename, index)
print(read_messages(filename)[index])

Should you later decide that a database is a better fit you can change 
read_messages() to return a class that transparently accesses that database. 
Again, most of the work is already done:

class Messages(collections.Sequence):
def __init__(self, filename):
self.filename = filename)
def __getitem__(self, index):
# read record(s) from db
def __len__(self): 
# return num-records in db

def read_messages(filename):
return Messages(filename)

By the way, where do you plan to use your functions? And where do the 
indices you feed them come from?

(*) Don't repeat what those other guys already did. Yeah sorry, I have a 
soft spot for lame jokes...

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Exception in thread Thread-4:

2015-05-05 Thread david jhon

Hi Chris,

Thanks a lot for such a comprehensive reply, I got it fixed now. Thanks
again :)

On Tue, May 5, 2015 at 2:52 PM, Chris Angelico  wrote:

> On Tue, May 5, 2015 at 7:23 PM, david jhon  wrote:
> > from threading import Timer, Lock
> >
> > class miTestClass(EventMixin):
> > def __init__(self, t, r, bw):
> >  self.statMonitorLock = Lock() #to lock the multi access threads
> >  self.statMonitorLock.acquire()
> >  statMonitorTimer = Timer(10.0, self._collectFlowStats()) #timer
> to
> > collect stats
> >  statMonitorTimer.start()
> >
> > but I am getting following error:
> >
> > Exception in thread Thread-4:
> > Traceback (most recent call last):
> >   File "/usr/lib/python2.7/threading.py", line 810, in __bootstrap_inner
> > self.run()
> >   File "/usr/lib/python2.7/threading.py", line 1082, in run
> > self.function(*self.args, **self.kwargs)
> > TypeError: 'NoneType' object is not callable
>
> The Timer() class will call a function when it's ready. To use that,
> you want to pass it a function. Try putting these two lines of code
> into your __init__ function:
>
> print(self._collectFlowStats)
> print(self._collectFlowStats())
>
> (Aside: This is something I like to call "IIDPIO debugging": If In
> Doubt, Print It Out. You sometimes have more sophisticated debugging
> techniques available, but you hardly ever are unable to basic 'print',
> in some form or another. It's incredibly useful.)
>
> The first one will print out something like this:
>
>  object at 0x12345678>>
>
> The second will actually call that function (which may or may not do
> anything visible), and then print out:
>
> None
>
> If you change your _collectFlowStats function a bit, you can see even
> more of what's happening. Something like this:
>
> def _collectFlowStats(self):
> print("_collectFlowStats has been called. Returning 42...")
> return 42
>
> Then you'll see that it gets called, and does its print, and then 42
> gets printed out at the end.
>
> In your case, simply removing the parentheses from
> self._collectFlowStats should do what you want - the Timer constructor
> will be passed a function (in this case, a bound method, but same
> same), and that function won't be called until the timer is up.
>
> Hope that helps!
>
> ChrisA
> --
> https://mail.python.org/mailman/listinfo/python-list
>
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Step further with filebasedMessages

Op Tuesday 5 May 2015 12:41 CEST schreef Steven D'Aprano:

> On Tuesday 05 May 2015 18:52, Cecil Westerhof wrote:
>
>> I now defined get_message_slice:
>> ### Add step
>> def get_message_slice(message_filename, start, end):
>> """
>> Get a slice of messages, where 0 is the first message
>> Works with negative indexes
>> The values can be ascending and descending
>> """
>
> What's a message in this context? Just a line of text?

Yes. In this case it is. Iuse it to post on Twitter. (OmgangMetTijd)
It was done with an old PHP script. But I switched it to Python, just
to get some needed exercise in Python. But I found it very easy to
extend the functionality.

In my twitter application I use '^' to have the possibility to create
newlines.

>
>
>> message_list= []
>> real_file   = expanduser(message_filename)
>> nr_of_messages  = get_nr_of_messages(real_file)
>> if start < 0:
>> start += nr_of_messages
>> if end < 0:
>> end += nr_of_messages
>> assert (start >= 0) and (start < nr_of_messages)
>> assert (end   >= 0) and (end   < nr_of_messages)
>
> You can write that as:
>
> assert 0 <= start < nr_of_messages

I was thought that my way is more clear.


> except you probably shouldn't, because that's not a good use for
> assert: start and end are user-supplied parameters, not internal
> invariants.
>
> You might find this useful for understanding when to use assert:
>
> http://import-that.dreamwidth.org/676.html

I will read it.


>> if start > end:
>> tmp = start
>> start   = end
>> end = tmp
>> need_reverse= True
>
> You can swap two variables like this:
>
> start, end = end, start
>
> The language guarantees that the right hand side will be evaluated
> before the assignments are done, so it will automatically do the
> right thing.

That is a lot clearer. Thanks.


> Your behaviour when start and end are in opposite order does not
> match the standard slicing behaviour:
>
> py> "abcdef"[3:5]
> 'de'
> py> "abcdef"[5:3]
> ''
>
> That doesn't mean your behaviour is wrong, but it will surprise
> anyone who expects your slicing to be like the slicing they are used
> to.

Good point: something to think about.


> I think what I would do is:
>
> def get_message_slice(message_filename, start=0, end=None, step=1):
> real_file = expanduser(message_filename)
> with open(real_file, 'r') as f:
> messages = f.readlines()
> return messages[start:end:step]

The idea is that this is expensive for large files.


> until such time that I could prove that I needed something more
> sophisticated. Then, and only then, would I consider your approach,
> except using a slice object:

Because I publish it, I should take reasonable care of the
possibilities of its use.


> # Untested.
> def get_message_slice(message_filename, start=0, end=None, step=1):
> real_file = expanduser(message_filename)
> messages = []
> # FIXME: I assume this is expensive. Can we avoid it?
> nr_of_messages = get_nr_of_messages(real_file)

If I want to give the possibility to use negative values also, I need
the value.


> the_slice = slice(start, end, step)
> # Calculate the indexes in the given slice, e.g.
> # start=1, stop=7, step=2 gives [1,3,5].
> indices = range(*(the_slice.indices(nr_of_messages)))
> with open(real_file, 'r') as f:
> for i, message in enumerate(f):
> if i in indices:
> messages.append(message)
> return messages

I will look into it.

-- 
Cecil Westerhof
Senior Software Engineer
LinkedIn: http://www.linkedin.com/in/cecilwesterhof
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Is it normal to cry when given XML?

2015-05-05 Thread Sayth Renshaw

Adam I am glad to hear it in someways because it's something I have never heard 
it. For a person relatively new to XML most articles and tutorials demonstrate 
getting it out to a more "manageable" format.

I had been using xbase to inspect the data and query but really ask I want to 
do was push it to a db and then query it as is more normal for me. 

I will check out your link. 
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Is it normal to cry when given XML?

2015-05-05 Thread Tim

On Tuesday, May 5, 2015 at 5:28:37 AM UTC-4, Sayth Renshaw wrote:
> Hi
> 
> Just checking if the reaction to cry when given XML is normal. 
> 
> Sayth

Hi Sayth,
My experience in general is just like what Chris said. Except when dealing with 
DocBook XML which is probably not what you have. But I cannot say enough good 
things about lxml, which is my favorite 3rd-party package ever. Its support for 
xpath really makes it easy to traverse and select elements in a document tree.  
Plus the ability to drop subtrees, move elements around in a live tree, etc. 
Great library.

Using lxml to parse and restructure the xml to a dictionary isn't too hard, but 
of course it depends on the xml you have to deal with. Sometimes weeping is 
just part of the job.

good luck,
--Tim
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Bitten by my C/Java experience

2015-05-05 Thread BartC


On 05/05/2015 09:19, Steven D'Aprano wrote:

On Tuesday 05 May 2015 08:02, BartC wrote:



(I think I would have picked up "++" and "--" as special tokens even if
increment/decrement ops weren't supported. Just because they would
likely cause errors through misunderstanding.)


Just because C made a mistake, doesn't mean other languages have to
slavishly follow it.


I would have thought there was more rapport between the two languages. 
Python is often implemented in C and extensions are often implemented in 
C, suggesting there are quite a few people familiar with both, sometimes 
in areas that are critical (ie. creating code that will affect thousands 
of Python apps).


So why pretend that ++ and -- don't exist? After all Python borrows "=", 
"==" and "!=" from C.


(Writing a==b instead of a=b is less likely in Python than in a language 
where a=b is an equality test rather than assignment. But I've used just 
such a language where mistakenly writing a=b (which happens when 
switching between languages) caused difficult-to-find bugs.


Until I disallowed standalone expressions as statements, then these 
things are picked up, and they are invariably unintended errors. Where 
it is actually necessary to evaluate an expression and throw away the 
result, then a simple prefix can be used.)


--
Bartc




--
https://mail.python.org/mailman/listinfo/python-list

Re: Step further with filebasedMessages

On May 5, 2015 5:46 AM, "Cecil Westerhof"  wrote:
>
> Op Tuesday 5 May 2015 12:41 CEST schreef Steven D'Aprano:
>
> > # Untested.
> > def get_message_slice(message_filename, start=0, end=None, step=1):
> > real_file = expanduser(message_filename)
> > messages = []
> > # FIXME: I assume this is expensive. Can we avoid it?
> > nr_of_messages = get_nr_of_messages(real_file)
>
> If I want to give the possibility to use negative values also, I need
> the value.

You could make this call only if one of the boundaries is actually
negative. Then callers that provide positive values don't need to pay the
cost of that case.

Alternatively, consider that it's common for slices of iterators to
disallow negative indices altogether, and question whether you really need
that.

> > the_slice = slice(start, end, step)
> > # Calculate the indexes in the given slice, e.g.
> > # start=1, stop=7, step=2 gives [1,3,5].
> > indices = range(*(the_slice.indices(nr_of_messages)))
> > with open(real_file, 'r') as f:
> > for i, message in enumerate(f):
> > if i in indices:
> > messages.append(message)
> > return messages

I approve of using slice.indices instead of calculating the indices
manually, but otherwise, the islice approach feels cleaner to me. This
reads like a reimplementation of that.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Step further with filebasedMessages

Op Tuesday 5 May 2015 15:57 CEST schreef Ian Kelly:

> On May 5, 2015 5:46 AM, "Cecil Westerhof"  wrote:
>>
>> Op Tuesday 5 May 2015 12:41 CEST schreef Steven D'Aprano:
>>
>>> # Untested. def get_message_slice(message_filename, start=0,
>>> end=None, step=1): real_file = expanduser(message_filename)
>>> messages = [] # FIXME: I assume this is expensive. Can we avoid
>>> it? nr_of_messages = get_nr_of_messages(real_file)
>>
>> If I want to give the possibility to use negative values also, I
>> need the value.
>
> You could make this call only if one of the boundaries is actually
> negative. Then callers that provide positive values don't need to
> pay the cost of that case.

You have a point there. I will think about it.


> Alternatively, consider that it's common for slices of iterators to
> disallow negative indices altogether, and question whether you
> really need that.

It was an idea I got from this newsgroup. :-D And I liked it,
otherwise I would not have implemented it.

-- 
Cecil Westerhof
Senior Software Engineer
LinkedIn: http://www.linkedin.com/in/cecilwesterhof
-- 
https://mail.python.org/mailman/listinfo/python-list

asyncio: What is the difference between tasks, futures, and coroutines?

I'm working my way through the asyncio documentation. I have got to the "Tasks 
and coroutines" section, but I'm frankly confused as to the difference between 
the various things described in that section: coroutines, tasks, and futures.

I think can understand a coroutine. Correct me if I'm wrong, but it's roughly 
"something that you can run which can suspend itself".

But I don't understand what a Future is. The document just says it's almost the 
same as a concurrent.futures.Future, which is described as something that 
"encapsulates the asynchronous execution of a callable". Which doesn't help a 
lot. In concurrent.futures, you don't create Futures, you get them back from 
submit(), but in the asyncio docs it looks like you can create them by hand 
(example "Future with run_until_complete"). And there's nothing that says what 
a Future is, just what it's like... :-(

A Task is a subclass of Future, but the documentation doesn't say what it *is*, 
but rather that it "schedules the execution of a coroutine". But that doesn't 
make sense to me - objects don't do things, they *are* things. I thought the 
event loop did the scheduling?

Reading between the lines, it seems that the event loop schedules Tasks (which 
makes sense) and that Tasks somehow wrap up coroutines - but I don't see *why* 
you need to wrap a task in a coroutine rather than just scheduling coroutines. 
And I don't see where Futures fit in - why not just wrap a coroutine in a 
Future, if it needs to be wrapped up at all?

I concede that I've not read the rest of the asyncio documentation in much 
detail yet, and I'm skipping everything to do with IO (I want to understand the 
"async" bit for now, not so much the "IO" side). But I don't really want to 
dive into the details while I am this hazy on the basic concepts.

Can anyone clarify for me?

Thanks,
Paul
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Step further with filebasedMessages

Op Tuesday 5 May 2015 13:08 CEST schreef Peter Otten:

> Cecil Westerhof wrote:
>
>> I now defined get_message_slice:
>> ### Add step
>> def get_message_slice(message_filename, start, end):
>
> Intervals are usually half-open in Python. I recommend that you
> follow that convention.

I will change it.


>> """
>> Get a slice of messages, where 0 is the first message
>> Works with negative indexes
>> The values can be ascending and descending
>> """
>>
>> message_list= []
>> real_file   = expanduser(message_filename)
>> nr_of_messages  = get_nr_of_messages(real_file)
>> if start < 0:
>> start += nr_of_messages
>> if end < 0:
>> end += nr_of_messages
>> assert (start >= 0) and (start < nr_of_messages)
>
> You should raise an exception. While asserts are rarely switched off
> in Python you still have to be prepeared, and an IndexError would be
> a better fit anyway.

OK.


> Should you later decide that a database is a better fit you can
> change read_messages() to return a class that transparently accesses
> that database. Again, most of the work is already done:
>
> class Messages(collections.Sequence):
> def __init__(self, filename):
> self.filename = filename)
> def __getitem__(self, index):
> # read record(s) from db
> def __len__(self): 
> # return num-records in db
>
> def read_messages(filename):
> return Messages(filename)

Another thing I have to look into. :-D


> By the way, where do you plan to use your functions? And where do
> the indices you feed them come from?

In my case from:
get_random_message
and:
dequeue_message

Both also in:

https://github.com/CecilWesterhof/PythonLibrary/blob/master/filebasedMessages.py

I should write some documentation with those functions. ;-)


I have a file with quotes and a file with tips. I want to place random
messages from those two (without them being repeated to soon) on my
Twitter page. This I do with ‘get_random_message’. I also want to put
the first message of another file and remove it from the file. For
this I use ‘dequeue_message’.

-- 
Cecil Westerhof
Senior Software Engineer
LinkedIn: http://www.linkedin.com/in/cecilwesterhof
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Throw the cat among the pigeons

On Sunday, 3 May 2015 16:23:59 UTC+1, Cecil Westerhof  wrote:
> > By the way: I think that even if the recursion does not go further
> > as 500, it is still a good idea to use tail recursion. Why use stack
> > space when it is not necessary?
> 
> I pushed the example to GitHub:
> https://github.com/CecilWesterhof/PythonLibrary/blob/master/mathDecebal.py

You already know this, as your code shows, but tail call recursion elimination 
is only possible when you have a *direct* tail call (one with the result of the 
tail call returned immediately to the caller). Even the standard trivial 
factorial example doesn't have a direct tail call, without rewriting to use an 
accumulator variable. Which is a non-intuitive transformation to anyone who's 
not familiar with recursive functional languages and their idioms.

If you're rewriting your recursive function *anyway*, it's not that much harder 
in many (most?) cases to rewrite it iteratively.

An example of a function that naturally uses direct tail call recursion, but 
which doesn't have a simple iterative rewrite, would be more compelling. Not 
particularly compelling (to me, anyway) even so, but still better than 
factorial or fibonnaci examples.

Paul
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: asyncio: What is the difference between tasks, futures, and coroutines?

2015-05-05 Thread Zachary Ware

On Tue, May 5, 2015 at 10:22 AM, Paul  Moore  wrote:
> I'm working my way through the asyncio documentation. I have got to the 
> "Tasks and coroutines" section, but I'm frankly confused as to the difference 
> between the various things described in that section: coroutines, tasks, and 
> futures.

I've been using (and, in part at least, understanding :)) asyncio for
work for the past few months, so I can at least give you my
impressions based on use.

> I think can understand a coroutine. Correct me if I'm wrong, but it's roughly 
> "something that you can run which can suspend itself".

That's basically how I understand it.

> But I don't understand what a Future is. The document just says it's almost 
> the same as a concurrent.futures.Future, which is described as something that 
> "encapsulates the asynchronous execution of a callable". Which doesn't help a 
> lot. In concurrent.futures, you don't create Futures, you get them back from 
> submit(), but in the asyncio docs it looks like you can create them by hand 
> (example "Future with run_until_complete"). And there's nothing that says 
> what a Future is, just what it's like... :-(

My understanding is that Futures are somewhat like 'non-executable
coroutines', if that makes any sense whatsoever.  Futures are used as
something you can pass around when you need to start execution of the
"job" in one method and finish it in another.  For instance, talking
to a remote network entity, and you want to just do "yield from
server.get(something)".  Your 'get' method can make the request,
create a Future, stick it somewhere that the method monitoring
incoming traffic can find it (along with some identifying metadata),
and then do 'return (yield from future)' to wait for the Future to be
fulfilled (by the listener) and return its result.  The listener then
matches up incoming requests with Futures, and calls
Future.set_result() or Future.set_exception() to fulfill the Future.
Once one of those methods has been called on the Future (and control
has been passed back to the scheduler), your server.get method
unblocks from its '(yield from future)' call, and either raises the
exception or returns the result that was set on the Future.

> A Task is a subclass of Future, but the documentation doesn't say what it 
> *is*, but rather that it "schedules the execution of a coroutine". But that 
> doesn't make sense to me - objects don't do things, they *are* things. I 
> thought the event loop did the scheduling?
>
> Reading between the lines, it seems that the event loop schedules Tasks 
> (which makes sense) and that Tasks somehow wrap up coroutines - but I don't 
> see *why* you need to wrap a task in a coroutine rather than just scheduling 
> coroutines. And I don't see where Futures fit in - why not just wrap a 
> coroutine in a Future, if it needs to be wrapped up at all?

You kind of mixed things up in this paragraph (you said "Tasks somehow
wrap up coroutines", then "wrap a task in a coroutine"; the first is
more correct, I believe).  As I understand it, Tasks are
specializations of Futures that take care of the
set_result/set_exception based on the execution of the coroutine.  I'm
not clear on how that is all implemented (Are all coroutines wrapped
in Tasks?  Just those wrapped in asyncio.async()? No idea.), but I use
it something like this:

   # save a reference to this task so it can be cancelled elsewhere if need be
   self.current_task = asyncio.async(some_coroutine())
   # now the coroutine is scheduled, but we still have control
   # we can do anything else we need to here, including scheduling
other coroutines
   return (yield from self.current_task)

> I concede that I've not read the rest of the asyncio documentation in much 
> detail yet, and I'm skipping everything to do with IO (I want to understand 
> the "async" bit for now, not so much the "IO" side). But I don't really want 
> to dive into the details while I am this hazy on the basic concepts.
>
> Can anyone clarify for me?

I hope I've done some good on that front, but I'm still a bit hazy
myself.  I found it worked best to just take examples or otherwise
working code, and poke and prod at it until it breaks and you can
figure out how you broke it.  Just try to avoid mixing asyncio and
threads for as long as you can :).

-- 
Zach
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Bitten by my C/Java experience

2015-05-05 Thread Rustom Mody

On Tuesday, May 5, 2015 at 6:50:29 PM UTC+5:30, BartC wrote:
> On 05/05/2015 09:19, Steven D'Aprano wrote:
> > On Tuesday 05 May 2015 08:02, BartC wrote:
> 
> >> (I think I would have picked up "++" and "--" as special tokens even if
> >> increment/decrement ops weren't supported. Just because they would
> >> likely cause errors through misunderstanding.)
> >
> > Just because C made a mistake, doesn't mean other languages have to
> > slavishly follow it.
> 
> I would have thought there was more rapport between the two languages. 
> Python is often implemented in C and extensions are often implemented in 
> C, suggesting there are quite a few people familiar with both, sometimes 
> in areas that are critical (ie. creating code that will affect thousands 
> of Python apps).

There are mistakes and there are mistakes.
Like there are crimes and crimes, some get community service, some get death 
sentence

> 
> So why pretend that ++ and -- don't exist? After all Python borrows "=", 
> "==" and "!=" from C.
> 
> (Writing a==b instead of a=b is less likely in Python than in a language 
> where a=b is an equality test rather than assignment. But I've used just 
> such a language where mistakenly writing a=b (which happens when 
> switching between languages) caused difficult-to-find bugs.

Yeah I happen to me in that minuscule minority that regards '= denotes 
assignment' a bigger mistake than ++

Interestingly K&R admitted to the fact that the precedences of C have errors;
in particular x&y == 0x1  groups wrong and so invariably requires brackets.
Unfortunately no such admission for the choice of the most ubiquitous operator 
=.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Throw the cat among the pigeons

Op Tuesday 5 May 2015 17:47 CEST schreef Paul Moore:

> On Sunday, 3 May 2015 16:23:59 UTC+1, Cecil Westerhof  wrote:
>>> By the way: I think that even if the recursion does not go further
>>> as 500, it is still a good idea to use tail recursion. Why use
>>> stack space when it is not necessary?
>>
>> I pushed the example to GitHub:
>> https://github.com/CecilWesterhof/PythonLibrary/blob/master/mathDecebal.py
>
> You already know this, as your code shows, but tail call recursion
> elimination is only possible when you have a *direct* tail call (one
> with the result of the tail call returned immediately to the
> caller). Even the standard trivial factorial example doesn't have a
> direct tail call, without rewriting to use an accumulator variable.
> Which is a non-intuitive transformation to anyone who's not familiar
> with recursive functional languages and their idioms.
>
> If you're rewriting your recursive function *anyway*, it's not that
> much harder in many (most?) cases to rewrite it iteratively.
>
> An example of a function that naturally uses direct tail call
> recursion, but which doesn't have a simple iterative rewrite, would
> be more compelling. Not particularly compelling (to me, anyway) even
> so, but still better than factorial or fibonnaci examples.

Well, I did not write many tail recursive functions. But what surprised
me was that for large values the ‘tail recursive’ version was more
efficient as the iterative version. And that was with myself
implementing the tail recursion. I expect the code to be more
efficient when the compiler implements the tail recursion.

-- 
Cecil Westerhof
Senior Software Engineer
LinkedIn: http://www.linkedin.com/in/cecilwesterhof
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Is it normal to cry when given XML?

2015-05-05 Thread Michael Torrie

On 05/05/2015 03:28 AM, Sayth Renshaw wrote:
> Hi
> 
> Just checking if the reaction to cry when given XML is normal.

I'd say it is normal.  XML is like violence.  If it doesn't solve your
problems, you're not using enough of it[1].

[1] Can anyone tell me who originated this line?
-- 
https://mail.python.org/mailman/listinfo/python-list

CHALLENGE HELP - GOOGLE DEVELOPER DAY

2015-05-05 Thread worship . brother

Good afternoon everyone.

I'm with the following exercise of the option is a modification of a google 
developer day exercise.

SOMEONE HELP ME IN THIS CHALLENGE?

Archaeologists have found a scroll with the following texts:

txtA = '' 'cncdbm pjcjzct vdbbxdtw rfqsr mkt gvhkcsvw qcxr kmk pnhc zwwdsd 
pgjjhr lxzscps lmbjjx LGH mdcqlbwx ppzpvfbv vdszkb NJV nvmfhshh ztvkmv dfbnht 
xfpj nlbcwrvv dcrzslrf WZB krdb zndtfmf fwwm vmqzmg cpcnpnww nkjk ncrmzr jfmcl 
hxr vcj wcptgbc gvbfxtbv wcjlrs psjc lrljqdct ltjwn vmhp wnlmw rkvg rtm djsv 
rwjls NGC zcjjttpt kbg tdjzcwj pmgtzlng zvtnbs svztn pctgq qnghqcch mclvp qdht 
cbk lvqckrl qmwknt CQW jfxzx tkljpkfc mjhskxjh lfjvrhr whshcrk kvtgzpl gxglr 
mpqxtbzb zbrpktxj knhmmtks MHJ xdlsm wcc sbrkdjmh gpn vwddsdl mqfsrwt khhmfng 
qkkjgg QDR qlpt njq xlc jfhhv kgzsb lqncvrh JGW xhjk krprtxf dtrsfb rwtzbhs 
qbvvz nqbh pdfqlsv hhrrx PVF mdvcfqkt bsb gcnkszfk rkzlhbnx njfqrzxj gtvrwp SMR 
qddg gqjjcbjw khlbtg cdswmbgb lmbfvsv nfhvn jbnplx dmp hmnf bzrznlh gkh 
vqdmjdsb ktbcx fgvsxzp gmv xftg lxtf dcxsrkl tzwvbkfq tpvdrlf khzzw srtfttv 
lmknq jvkwr bplzzpjw rgtrj gcmdfzdx ksptr jzdfqq xmtzz pkqxrsw gvwg fqppvgk hsc 
hdj gthkq trxl gkhp rbr skxpc lbkw bbdqpvn llshm bhrfvh gjw zgnlpgwl pdhqn 
ttjtx vfb djftp kkrr ptmnzqkg tmjlqw pcvpwjb ggd jntktdz Vth WTSP mlfmddsn phn 
hrdqrds phbq vkffrqvt zxljnd LQG blnw brxk skwkh vqbq dlbnhz xpbddsjw gscvghmj 
pxcpvkmm jsfpllns kpmmgxb pljpvcn wknwbcq wthwv bdjxr wdc lqmtsnrn wbcjprr 
htldnxcn QWL fvjdqd cwmb bfjmw chv blx xblfrb gjbbg njqw mkmnwtzv dgsxl tgtclv 
xxcfpp brzxg lbrnrsf pfcpt wmvjxdsw jmstqwx zjcrkzm ndtbsqsh jqbcddqh ggvf kcr 
mfhllb lbbkssx gsrxcb rwtxljp cwkc xwxjhbc pfcgz zbdbsq wzfsf gqhdfchv cgp 
kwvnrfm ptmzl bkjcm dbl rskfms mqdgvskt hbtjpvm swff vjq dwlrgxtm kqzrlxz 
vmpdxkv bwfgzmjn zzrr MLSM kknhkq jhtxc gjvnj mcld vftnnd pgbkhc kzzsbq xqbs 
hlkxtc JXG vttn zmnc kwhcrd bfjbs lxkmnzx pbmp Int ZTM fkzql bjgllxv bghqpnl 
hphvdvl lwlqdh xldsnqds tsmrhhtm gnksf lzhr RQM jvcmkbxv nsx cxpplqbq jzrr tnt 
wpfl lnkfqjt tplwbr hzrv mxqhpg xxzdlwzx cgrtr jbvqmb cmsqdjx qcrx ljxh jbnw 
lgf tzd cltdd jtmdmt djbgqqk tgsffh hbff jjqn wtlsx qlmhxrrf sklfrc dwsk rgpgqz 
zhzvm brprszrc hqlgx tdbsgf fkmrtn frskk qjvg jlhpgh rxrmqp nmc dxpx lljs kszjq 
hlxx nbkvsrf dggshkxz
nlvgr zldk tvphkg hlnls wlsxsvf mmksm lzgfnkmg tbw nrpzqfr gfc sxcdqtlt 
rmmhtmbp jkk fdh jcw nwjjkrdq kztrcqxt lphf frs dcsbhwp tgnq LMQ tpppxf vmd 
tnbqgv xxtlt ljdz hdslhv dkzxctcn qsfctn tdsbdhv dxmkk ghfntj dckqls knlwvk 
mrddntg fwqwfxs tzqcvz sbnfjs kcxl dsgnhsf KGJ mfm vjmmf wptc rtb dtblv FRP 
wwmngbk dxss txz fzlxll lbqjrp hhcxnnhn skhcwxw lllkssb dgwd lzq czhhtclr 
bpngkf krh bzccxd rqkr mdz PHK dfctpgr GKT bdnfbh wlj lfhkmb crbhzfs qzz 
jbmlsfkx htkmdbx mxjccgv FPJ ttl jbw PMNT khqr jhztz dvmj kwchwcv xggzgkln DXZ 
fdsfsc CgPN pphslgf jxsd fsp spzzbc trvpx WJG nmdzgmnr qslm znngdwb zqwb 
dcgxxcqm kcdwcpdm ngqzs SBST ltqxkt tjgmjzh nmmqnzfn ngsbphqb gsqfgjn jtwhxdx 
PQC jsfkd ccx dfj wrgf kslnx zgqxqpvm rflzw jgl rsd fshm zlmtqs qfbrdg dknttlb 
xvqnd gtrpzhwb ngwrbrfk skkfq bzfqkpjj RZP trzz fgmcd ptg WZG mdxlzvkg rfqdh 
lvwht vkvqhk pdnqm gwpcph qbpsv vhxdj VXF tvpjwzb tjpdp frtvg xrqhp zjqcbxf 
fpbwbzb whnjkv dqfjwtv mjwrncd xvgkv xrsrjv zdrfjwc lmbtkhch srwg LVQ bvrjztfm 
vwmb rpxwnbrw cmvjrf cdxsgqgs mtrfm wrzct pznlvfk cgcdvbpt cnpqtw sfwsnkkw 
qjlmkkmv wrsctd lwdbkws qhvszv jzzpb jlgz ftdqspbf vvsvkq znktlpx vcvccs 
kswfwbzv hwfbd grplms dfvt pmmd nvb tgjpq hgsx qgsvbgws gzrq kmbgdwm tgmwmsc 
lsmphpk pwpznr mhcskk bmzqtdx pgwd jplzc cbkgf zbmclc tzhqvbt wkq hljqwkz 
vgclcmdx ggzb vsng tjw ckxtmqx wffrgzp wbht rsqb gqwxpncw mkszj mhlrmd pcl 
jqjgd rdrphff ftzkqg dgrhmgn ZGL plkxf hzdb mmj plphnnv jvvc tlthnhrh ngkgnln 
NFV rtxct ppsp rcfxhhh mbzkdw smdlrm cstjtb rhhmzvp tqbs szbmqd gbn fjmt fcppm 
qdgqls gxltm mdrgqdht vxpbxrdj twxcfxk qzj wtfh vglkdghk xtdzrz rjldhzld mbd 
fgrfb hffjd hcr vghjwkvl pfkjshg rrflt zflwbn xffjdlfs bbzvs wdxmfr fvntg 
twjhgcc zwvwrnn gcnzl ftfpd wfqxrnzf mbccsd szltzjm kpbslq wxxchz szzh TGQ 
jxnng cmrgdh pdxjxpxr bslbmwm mdkc nqjflf VSRP gbprtv mkfsfcwd ZKF pqsq chbb 
bmt smtkrxjx nfkltv cvhxd zwwx bzqcnzwp wcpn jmqkbclx '' '. split ()

txtB = '''pwbfdmtc jms gswg wvsscb ffq lbrhbn lcxc hcr thc mghts vkgfc nrvfgs 
dsrdq tcmfz scqskgsl twgzh whts dqt twtksl lcrdlc dpzrl hwlqvc xcfstz rfkvbr 
bzmvqp qxrs jlmwtcs nmjkkmpg kbcbg shdf qxpm qbm hlcnqnw jwhvvrtr kccw njxtbh 
hbtn lqmxbx krnn hcv ptqtwp xgnfggb bjdd nfgkxsw kgzcf bgncx rbsfrrcf vjwsjpw 
jbtcbqm xhhg kfpqcpx bfxlg qddzdv rvfqp hphjhns xhk npdd gsxm ffkbj gwdxkhr 
ddqmr jnzznp jzsgkb lcgsgjvh xvsbdw klzsxz xpjkjxc gth dtrmkn qzcsksd vsdrhj 
vtlxg kdtxsj sgs chnz bdllcsdl trggnlpd gwbvj stnhs vqbhj tdhps sgkk mxnswm ghm 
sqhfcnlk lpwqpn gcgg mjxh prmqclss zfn gplktxj vkjnkkv fzzx vdslwsdk fxt 
pnbqqbk ksfgcvw hxfq xxd rvqzhmm ctvfgxzv nrzdkx nsxmr bnvkmhcl srvc nczkp 
zbgsxg nmpx vrqq xfmnsjc zszmrfjv cbwjfldn fgn mzpp crjnct cmh cwh cvdk cslq 
tvr gggck pfs th

Re: Is it normal to cry when given XML?

2015-05-05 Thread Grant Edwards

On 2015-05-05, Sayth Renshaw  wrote:

> Just checking if the reaction to cry when given XML is normal.

It depends on whether or not you're allowed to swear and throw things.

-- 
Grant Edwards   grant.b.edwardsYow! I just heard the
  at   SEVENTIES were over!!  And
  gmail.comI was just getting in touch
   with my LEISURE SUIT!!
-- 
https://mail.python.org/mailman/listinfo/python-list

Writing list of dictionaries to CSV

2015-05-05 Thread Kashif Rana

Hello Experts

When I am writing list of dictionaries to CSV file, the key 'schedule' has 
value 'Mar 2012' becomes Mar-12. I really do not have clue why thats happening. 
Below is the code. 

dic_1 = {'action': 'permit',
 'dst-address': 'maxprddb-scan-167, maxprddb-scan-168, maxprddb-scan-169',
 'from': 'DMZ Web',
 'id': '1000',
 'log': 'Enable, session-init',
 'name': 'Test Rule Temporary ',
 'service': '1521, Oraccle-Maximo-Scan-1550, PING-ALL',
 'src-address': 'sparkeregap1, sparkeregap2',
 'to': 'Trust'}

dic_2 {'action': 'permit',
 'dst-address': 'sparkcas01, sparkcas02, email.ab.spark.net',
 'from': 'DMZ Web',
 'id': '4000',
 'log': 'Enable, session-init',
 'schedule': 'Mar 2012',
 'service': 'SMTP',
 'src-address': 'sparkeregap1, sparkeregap2',
 'to': 'Trust'}
 
 my_list = 
 [{'to': 'Trust', 'service': '1521, Oraccle-Maximo-Scan-1550, PING-ALL', 
'from': 'DMZ Web', 'dst-address': 'maxprddb-scan-167, maxprddb-scan-168, 
maxprddb-scan-169', 'name': 'Test Rule Temporary ', 'action': 'permit', 'id': 
'1000', 'src-address': 'sparkeregap1, sparkeregap2', 'log': 'Enable, 
session-init'}, {'to': 'Trust', 'from': 'DMZ Web', 'dst-address': 'sparkcas01, 
sparkcas02, email.ab.spark.net', 'service': 'SMTP', 'schedule': 'Mar 2012', 
'action': 'permit', 'id': '4000', 'src-address': 'sparkeregap1, sparkeregap2', 
'log': 'Enable, session-init'}]

 pol_keys = ['id', 'name', 'from', 'to', 'src-address', 'dst-address', 
'service', 'action', 'nat_status', 'nat_type', 'nat_src_ip', 'nat_dst_ip', 
'nat_dst_port', 'log', 'schedule']
 
with open('test.csv', 'wb') as f:
w = csv.DictWriter(f, pol_keys)
w.writeheader()
w.writerows(my_list)
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Step further with filebasedMessages


On 05/05/2015 11:25 AM, Cecil Westerhof wrote:



I have a file with quotes and a file with tips. I want to place random
messages from those two (without them being repeated to soon) on my
Twitter page. This I do with ‘get_random_message’. I also want to put
the first message of another file and remove it from the file. For
this I use ‘dequeue_message’.



Removing lines from the start of a file is an n-squared operation. 
Sometiomes it pays to reverse the file once, and just remove from the 
end.  Truncating a file doesn't require the whole thing to be rewritten, 
nor risk losing the file if the make-new-file-rename-delete-old isn't 
quite done right.


Alternatively, you could overwrite the line, or more especially the 
linefeed before it.  Then you always do two readline() calls, using the 
second one's result.


Various other games might include storing an offset at the begin of 
file, so you start by reading that, doing a seek to the place you want, 
and then reading the new line from there.



Not recommending any of these, just bringing up alternatives.

--
DaveA
--
https://mail.python.org/mailman/listinfo/python-list

Re: asyncio: What is the difference between tasks, futures, and coroutines?


On 5/5/2015 11:22 AM, Paul Moore wrote:

I'm working my way through the asyncio documentation. I have got to
the "Tasks and coroutines" section, but I'm frankly confused as to
the difference between the various things described in that section:
coroutines, tasks, and futures.

I think can understand a coroutine. Correct me if I'm wrong, but it's
roughly "something that you can run which can suspend itself".


The simple answer is that a coroutine within asyncio is a generator used 
as a (semi)coroutine rather than merely as an iterator.  IE, the send, 
throw, and close methods are used.  I believe PEP 492 will change that 
definition (for Python) to any object with with those methods.


However, if an asyncio coroutine is the result of the coroutine 
decorator, the situation is more complex, as the result is either a 
generator function, a coro function that can wrap any function, or a 
debug wrapper.  Details in asyncio/coroutine.py.



I concede that I've not read the rest of the asyncio documentation in
much detail yet, and I'm skipping everything to do with IO (I want to
understand the "async" bit for now, not so much the "IO" side). But I
don't really want to dive into the details while I am this hazy on
the basic concepts.


You might try reading the Python code for the task and future classes.
asyncio/futures.py, asyncio/tasks.py

--
Terry Jan Reedy

--
https://mail.python.org/mailman/listinfo/python-list

Re: asyncio: What is the difference between tasks, futures, and coroutines?

On Tue, May 5, 2015 at 9:22 AM, Paul  Moore  wrote:
> I'm working my way through the asyncio documentation. I have got to the 
> "Tasks and coroutines" section, but I'm frankly confused as to the difference 
> between the various things described in that section: coroutines, tasks, and 
> futures.
>
> I think can understand a coroutine. Correct me if I'm wrong, but it's roughly 
> "something that you can run which can suspend itself".
>
> But I don't understand what a Future is. The document just says it's almost 
> the same as a concurrent.futures.Future, which is described as something that 
> "encapsulates the asynchronous execution of a callable". Which doesn't help a 
> lot. In concurrent.futures, you don't create Futures, you get them back from 
> submit(), but in the asyncio docs it looks like you can create them by hand 
> (example "Future with run_until_complete"). And there's nothing that says 
> what a Future is, just what it's like... :-(

Fundamentally, a future is a placeholder for something that isn't
available yet. You can use it to set a callback to be called when that
thing is available, and once it's available you can get that thing
from it.

> A Task is a subclass of Future, but the documentation doesn't say what it 
> *is*, but rather that it "schedules the execution of a coroutine". But that 
> doesn't make sense to me - objects don't do things, they *are* things. I 
> thought the event loop did the scheduling?

In asyncio, a Task is a a Future that serves as a placeholder for the
result of a coroutine. The event loop manages callbacks, and that's
all it does. An event that it's been told to listen for occurs, and
the event loop calls the callback associated with that event. The Task
manages a coroutine's interaction with the event loop; when the
coroutine yields a future, the Task instructs the event loop to listen
for the completion of that future, setting a callback that will resume
the coroutine.

> Reading between the lines, it seems that the event loop schedules Tasks 
> (which makes sense) and that Tasks somehow wrap up coroutines - but I don't 
> see *why* you need to wrap a task in a coroutine rather than just scheduling 
> coroutines. And I don't see where Futures fit in - why not just wrap a 
> coroutine in a Future, if it needs to be wrapped up at all?

The coroutines themselves are not that interesting of an interface;
all you can do with them is resume them. The asynchronous execution
done by asyncio is all based on futures. Because a coroutine can
easily be wrapped in a Task, this allows for coroutines to be used
anywhere a future is expected.

I don't know if I've done a good job explaining, but I hope this helps.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: asyncio: What is the difference between tasks, futures, and coroutines?

2015-05-05 Thread Marko Rauhamaa

Paul  Moore :

> But I don't understand what a Future is.

A future stands for a function that is scheduled to execute in the
background.

Personally, I have never found futures a very useful idiom in any
language (Scheme, Java, Python). Or more to the point, concurrency and
the notion of a function don't gel well in my mind.

See also: http://en.wikipedia.org/wiki/Futures_and_promises>

> A Task is a subclass of Future, but the documentation doesn't say what
> it *is*,

A task is almost synonymous with a future. It's something that needs to
get done.

--

I think asyncio is a worthwhile step up from the old asyncore module.
Its main advantage is portability between operating systems. Also, it
has one big advantage over threads: events can be multiplexed using
asyncio.wait(return_when=FIRST_COMPLETED).

However, I happen to have the privilege of working with linux so
select.epoll() gives me all the power I need to deal with asynchrony.
The asyncio model has the same basic flaw as threads: it makes the
developer think of concurrency in terms of a bunch of linear streaks of
execution. I much prefer to look at concurrency through states and
events (what happened? what next?).

My preferred model is known as "Callback Hell" (qv.). Asyncio seeks to
offer a salvation from it. However, I am at home in the Callback Hell;
equipped with Finite State Machines, the inherent complexities of the
Reality are managed in the most natural, flexible manner.


Marko
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: asyncio: What is the difference between tasks, futures, and coroutines?

2015-05-05 Thread Skip Montanaro

Paul> ... I'm frankly confused ...

You and me both. I'm pretty sure I understand what a Future is, and
until the long discussion about PEP 492 (?) started up, I thought I
understood what a coroutine was from my days in school many years ago.
Now I'm not so sure.

Calling Dave Beazley... Calling Dave Beazley... We need a keynote...

Skip
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: asyncio: What is the difference between tasks, futures, and coroutines?

On Tuesday, 5 May 2015 17:11:39 UTC+1, Zachary Ware  wrote:
>On Tue, May 5, 2015 at 10:22 AM, Paul  Moore wrote:
>> I'm working my way through the asyncio documentation. I have got to the
>> "Tasks and coroutines" section, but I'm frankly confused as to the
>> difference between the various things described in that section:
>> coroutines, tasks, and futures.
> 
> I've been using (and, in part at least, understanding :)) asyncio for
> work for the past few months, so I can at least give you my
> impressions based on use.

Thanks for taking the time.

>> I think can understand a coroutine. Correct me if I'm wrong, but it's
>> roughly "something that you can run which can suspend itself".
> 
> That's basically how I understand it.

Cool, I got that right at least :-)

>> But I don't understand what a Future is.
[...]
> 
> My understanding is that Futures are somewhat like 'non-executable
> coroutines', if that makes any sense whatsoever.  Futures are used as
> something you can pass around when you need to start execution of the
> "job" in one method and finish it in another.  For instance, talking
> to a remote network entity, and you want to just do "yield from
> server.get(something)".  Your 'get' method can make the request,
> create a Future, stick it somewhere that the method monitoring
> incoming traffic can find it (along with some identifying metadata),
> and then do 'return (yield from future)' to wait for the Future to be
> fulfilled (by the listener) and return its result.  The listener then
> matches up incoming requests with Futures, and calls
> Future.set_result() or Future.set_exception() to fulfill the Future.
> Once one of those methods has been called on the Future (and control
> has been passed back to the scheduler), your server.get method
> unblocks from its '(yield from future)' call, and either raises the
> exception or returns the result that was set on the Future.

Hmm. My head hurts. I sort of think I see what you might mean, but I have no 
idea how that translates to "almost the same as a concurrent.futures.Future, 
which in my experience is nothing like that.

A c.f.Future is (in my naive view) a "promise" of a result, which you can later 
wait to be fulfilled. Creating one directly (as opposed to via submit) doesn't 
make sense because then there's nothing that's promised to provide a result.

What you described sounds more like a "container that can signal when it's been 
filled", which I can sort of see as related to the Future API, but not really 
as related to a c.f.Future.

>> Reading between the lines, it seems that the event loop schedules Tasks
>> (which makes sense) and that Tasks somehow wrap up coroutines - but I
>> don't see *why* you need to wrap a task in a coroutine rather than just
>> scheduling coroutines. And I don't see where Futures fit in - why not
>> just wrap a coroutine in a Future, if it needs to be wrapped up at all?
> 
> You kind of mixed things up in this paragraph (you said "Tasks somehow
> wrap up coroutines", then "wrap a task in a coroutine"; the first is
> more correct, I believe).

Whoops, sorry, my mistake - the first was what I meant, the second was a typo.

> As I understand it, Tasks are
> specializations of Futures that take care of the
> set_result/set_exception based on the execution of the coroutine.

That sounds far more like a concurrent.futures.Future. Except that (AIUI) tasks 
get scheduled, and you can list all tasks for an event loop, and things like 
that. So they seem like a little bit more than a c.f.Future, which would sort 
of match with the idea that an asyncio.Future was equivalent to a c.f.Future, 
except that it's not (as above).

There's also things like asyncio.async - which makes no sense to me at all (I 
can see how the examples *use* it, in much the same way as I can see how the 
examples work, in general, but I can't understand its role, so I'm not sure I'd 
be able to design code that used it from scratch :-()

>> Can anyone clarify for me?
> 
> I hope I've done some good on that front, but I'm still a bit hazy
> myself.  I found it worked best to just take examples or otherwise
> working code, and poke and prod at it until it breaks and you can
> figure out how you broke it.  Just try to avoid mixing asyncio and
> threads for as long as you can :).

Thanks for the insights. I think I've learned some things, but to be honest I'm 
still confused. Maybe it's because of the angle I'm coming at it from. I don't 
typically write network code directly[1], so the examples in the asyncio docs 
are mostly irrelevant at best for me. I do find that I use threads a reasonable 
amount, and I'd like to know if asyncio would be a worthwhile alternative. So I 
need to understand the concepts and try to apply them to my situations. Hence 
the comment above about designing asyncio code from scratch.

Paul

[1] When I do write network code, I typically want to use a high level library 
like requests. So in an asyncio context, I'm look

Re: asyncio: What is the difference between tasks, futures, and coroutines?

On Tuesday, 5 May 2015 18:48:09 UTC+1, Ian  wrote:
> Fundamentally, a future is a placeholder for something that isn't
> available yet. You can use it to set a callback to be called when that
> thing is available, and once it's available you can get that thing
> from it.

OK, that makes a lot of sense. There seems to be two distinct strands within 
the asyncio world, the callback model and the task/coroutine model. AIUI, 
coroutines/tasks are supposed to let you avoid callbacks.

So I guess in that model, a Future isn't something you should use directly in 
task-based code, because it works via callbacks. And yet tasks are futures, so 
it's not that simple :-)

> In asyncio, a Task is a a Future that serves as a placeholder for the
> result of a coroutine. The event loop manages callbacks, and that's
> all it does.

Um, I thought it scheduled tasks?

> An event that it's been told to listen for occurs, and
> the event loop calls the callback associated with that event. The Task
> manages a coroutine's interaction with the event loop; when the
> coroutine yields a future, the Task instructs the event loop to listen
> for the completion of that future, setting a callback that will resume
> the coroutine.

OK, that (to an extent) explains how the callback and task worlds link 
together. That's useful.

> The coroutines themselves are not that interesting of an interface;
> all you can do with them is resume them. The asynchronous execution
> done by asyncio is all based on futures. Because a coroutine can
> easily be wrapped in a Task, this allows for coroutines to be used
> anywhere a future is expected.

And yet, futures are callback-based whereas tasks are suspension-point (yield 
from) based. I get a sense of what you're saying, but it's not very clear yet 
:-)

> I don't know if I've done a good job explaining, but I hope this helps.

It's all helping. It's a shame the docs don't do a better job of explaining the 
underlying model, to be honest, but I do feel like I'm slowly getting closer to 
an understanding.

It's no wonder terminology is such a hot topic in the discussion of the new 
async PEP on python-dev :-(

Paul
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Writing list of dictionaries to CSV

2015-05-05 Thread MRAB


On 2015-05-05 18:09, Kashif Rana wrote:

Hello Experts

When I am writing list of dictionaries to CSV file, the key 'schedule' has 
value 'Mar 2012' becomes Mar-12. I really do not have clue why thats happening. 
Below is the code.

dic_1 = {'action': 'permit',
  'dst-address': 'maxprddb-scan-167, maxprddb-scan-168, maxprddb-scan-169',
  'from': 'DMZ Web',
  'id': '1000',
  'log': 'Enable, session-init',
  'name': 'Test Rule Temporary ',
  'service': '1521, Oraccle-Maximo-Scan-1550, PING-ALL',
  'src-address': 'sparkeregap1, sparkeregap2',
  'to': 'Trust'}


Missing '=' in the next line.


dic_2 {'action': 'permit',
  'dst-address': 'sparkcas01, sparkcas02, email.ab.spark.net',
  'from': 'DMZ Web',
  'id': '4000',
  'log': 'Enable, session-init',
  'schedule': 'Mar 2012',
  'service': 'SMTP',
  'src-address': 'sparkeregap1, sparkeregap2',
  'to': 'Trust'}

  my_list =
  [{'to': 'Trust', 'service': '1521, Oraccle-Maximo-Scan-1550, PING-ALL', 
'from': 'DMZ Web', 'dst-address': 'maxprddb-scan-167, maxprddb-scan-168, 
maxprddb-scan-169', 'name': 'Test Rule Temporary ', 'action': 'permit', 'id': 
'1000', 'src-address': 'sparkeregap1, sparkeregap2', 'log': 'Enable, 
session-init'}, {'to': 'Trust', 'from': 'DMZ Web', 'dst-address': 'sparkcas01, 
sparkcas02, email.ab.spark.net', 'service': 'SMTP', 'schedule': 'Mar 2012', 
'action': 'permit', 'id': '4000', 'src-address': 'sparkeregap1, sparkeregap2', 
'log': 'Enable, session-init'}]

  pol_keys = ['id', 'name', 'from', 'to', 'src-address', 'dst-address', 
'service', 'action', 'nat_status', 'nat_type', 'nat_src_ip', 'nat_dst_ip', 
'nat_dst_port', 'log', 'schedule']

with open('test.csv', 'wb') as f:
w = csv.DictWriter(f, pol_keys)
w.writeheader()
w.writerows(my_list)


It works OK for me.

(I'm assuming that you're reading the CSV file in a text editor, not
some other application that might be trying to be "clever" by
"interpreting" what it thinks looks a date as a date and then
displaying it differently...)

--
https://mail.python.org/mailman/listinfo/python-list

Stripping unencodable characters from a string

I want to write a string to an already-open file (sys.stdout, typically). 
However, I *don't* want encoding errors, and the string could be arbitrary 
Unicode (in theory). The best way I've found is

data = data.encode(file.encoding, errors='replace').decode(file.encoding)
file.write(data)

(I'd probably use backslashreplace rather than replace, but that's a minor 
point).

Is that the best way? The multiple re-encoding dance seems a bit clumsy, but it 
was the best I could think of.

Thanks,
Paul.
-- 
https://mail.python.org/mailman/listinfo/python-list

An improved version of the main snippet

2015-05-05 Thread Virgil Stokes

I have attached what I believe to be an improved version of my main snippet for 
testing.


--V :-)
'''
  Purpose: get current bid, ask and rate for currency exchanges (FOREX trading)
   Note:
1. yahoo seems to give the best estimates for the currency exchange rates
2. Not sure where the "bid" and "ask" values come from.
3. Not sure how often updates to currencies occur during weekdays; but very fast (real-time?) during weekdays

   Author: v...@it.uu.se
   Vesion: 2015.05.05.2
'''
import pytz
from   yahoo_finance import Currency
from   datetime import datetime
import time
#from time import clock # clock gives good resolution in MS Windows

NUM_TICS   = 500# number of values to be downloaded

CURRENCY_PAIR = 'EUR/SEK'
MY_TIME   = "Europe/Stockholm"
OTHER_TIME= "America/New_York"
FILE_OT   = 'CurrencyInfo.txt'

PRINT_TIMEZONES = True
PRINT_TIMEZONES = False
if PRINT_TIMEZONES:
for tz in pytz.all_timezones:
time.sleep(0.5)
print (tz)

def updateMeanVar(x,k,mu,vr):
'''
 Purpose: Update the estimates for the mean and variance (recursive mean,variance)
  
   Inputs:
  x -- new value (x_k)
  k -- counter (index) for new value (1,2,...)
 mu -- previously estimated mean (x_k not included)
 vr -- previously estimated variance (x_k not included)
   Otputs:
 mu -- updated mean (with x_k included)
 vr -- updated variance (with x_k included)
'''
delta = x - mu
mu += delta/k
vr += delta*(x - mu)  
return mu,vr

def get_Times(myTimeZone=MY_TIME,otherTimeZone=OTHER_TIME):

fmt  = '%Y-%m-%d %H:%M:%S %Z%z'
Mine = pytz.timezone(myTimeZone)
Other= pytz.timezone(otherTimeZone)

nowMine  = datetime.now(Mine)
nowOther = datetime.now(Other)
   
return nowMine.strftime(fmt) + ' ('+nowOther.strftime(fmt)+')'

DateTimeStr = get_Times()

def InitFile(file=FILE_OT,currencyPair=CURRENCY_PAIR,dateTimeStr=DateTimeStr):
f = open(file,'a')
f.write('Currency Pair: %s, TimeStamp; %s\n'%(currencyPair,dateTimeStr))
f.write(' bid ask ratedatetime\n')   
return f

def SaveToFile(f,Bid,Ask,Rate,dateTime):
f.write('%s  %s  %s   %s\n'%(Bid,Ask,Rate,dateTime))
pass

currency_Ex = Currency(CURRENCY_PAIR.replace('/',''))
fOt = InitFile()

print ('Currency pair: %s'%CURRENCY_PAIR)
print (' bid ask ratedatetime')

prev_dateTime = currency_Ex.get_trade_datetime()
NMax = NUM_TICS
mu = 0.0; va = 0.0
for k in range(1,NUM_TICS+1):
t0 = time.clock()
   
currency_Ex.refresh()
Bid = currency_Ex.get_bid()
if Bid == None:
continue
Ask = currency_Ex.get_ask()
if Ask == None:
continue
Rate = currency_Ex.get_rate()
if Rate == None:
continue
dateTime = currency_Ex.get_trade_datetime()
if dateTime == None:
continue
print ('%s  %s  %s   %s'%(Bid,Ask,Rate,dateTime))

if dateTime[:16] != prev_dateTime[:16]:
# Save currency exchange info when minute changes
SaveToFile(fOt,Bid,Ask,Rate,dateTime)
prev_dateTime = dateTime

# Estimate Time To Completion (ETTC) with recursive mean of 'loop' time   
dt= time.clock() - t0
mu,va = updateMeanVar(dt,k,mu,va)
ETTC  = mu*(NMax-k)
m,s   = divmod(ETTC,60)
h,m   = divmod(m,60)
print ('ETTC = %d:%02d:%03d'%(h,m,s))

fOt.close()

-- 
https://mail.python.org/mailman/listinfo/python-list

Open a Url in new tab and tab switching in IE using python and selenium

2015-05-05 Thread shweta kaushik

Hi All,

I am trying to open a new tab in IE10 and selenium 2.45. It is able to open
a new tab using pyrobot. But when i am trying to open url in new tab, it is
getting opened in first tab. Focus is not set to second tab and hence it is
not working and also switching of tab is not working. please provide
solution. I have provided my code below:
Code:

# Open first tab
IEDriverPath = "/../../../../../../../IEDriverServer.exe"
driver = webdriver.Ie(IEDriverPath, port=)
pyseldriver.get("https://www.google.com/";)
time.sleep(5)
tab1 = pyseldriver.current_window_handle

#open another tab
obja = pyrobot.Robot()
obja.key_press(pyrobot.Keys.ctrl)
obja.key_press(pyrobot.Keys.t)
obja.key_release(pyrobot.Keys.ctrl)
obja.key_release(pyrobot.Keys.t)
time.sleep(2)
pyseldriver.switch_to_window(pyseldriver.window_handles[-1])
tab2 = pyseldriver.current_window_handle
pyseldriver.get("https://www.python.org/";)
time.sleep(5)

#Switching to first tab and opening new url
pyseldriver.switch_to_window(tab1)
pyseldriver.get("https://www.yahoo.com/";)
time.sleep(10)

#switching to second tab and opening new url
pyseldriver.switch_to_window(tab2)
pyseldriver.get("https://www.gmail.com/";)
time.sleep(10)

But links are not opening in new tab and switching is also not happening.
All links are getting opened in first tab itself.

Regards,
Shweta
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Writing list of dictionaries to CSV

2015-05-05 Thread Skip Montanaro

On Tue, May 5, 2015 at 1:11 PM, MRAB  wrote:
> I'm assuming that you're reading the CSV file in a text editor, not
> some other application that might be trying to be "clever" by
> "interpreting" what it thinks looks a date as a date and then
> displaying it differently...

More likely, viewing the CSV file in Excel, Gnumeric, or some other
spreadsheet which interprets some inputs as dates and formats them
according to its default rules.

Skip
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Throw the cat among the pigeons


On 05/05/2015 12:18 PM, Cecil Westerhof wrote:



Well, I did not write many tail recursive functions. But what surprised
me was that for large values the ‘tail recursive’ version was more
efficient as the iterative version. And that was with myself
implementing the tail recursion. I expect the code to be more
efficient when the compiler implements the tail recursion.




You've said that repeatedly, so I finally took a look at your webpage

https://github.com/CecilWesterhof/PythonLibrary/blob/master/mathDecebal.py

I didn't have your framework to call the code, so I just extracted some 
functions and did some testing.  I do see some differences, where the 
so-called tail_recursive functions are sometimes faster, but I did some 
investigating to try to determine why.



I came up with the conclusion that sometimes the multiply operation 
takes longer than other times.  And in particular, i can see more 
variation between the two following loops than between your two functions.



def factorial_iterative(x, simple=False):
assert x >= 0
result = 1
j=2
if not simple:
for i in range(2, x + 1):
result *= i
j += 1
else:
for i in range(2, x + 1):
result *= j
j += 1
pass

return result

When the "simple" is True, the function takes noticeably and 
consistently longer.  For example, it might take 116 instead of 109 
seconds.  For the same counts, your code took 111.


I've looked at dis.dis(factorial_iterative), and can see no explicit 
reason for the difference.




--
DaveA
--
https://mail.python.org/mailman/listinfo/python-list

Re: Stripping unencodable characters from a string


On 05/05/2015 02:19 PM, Paul Moore wrote:

You need to specify that you're using Python 3.4 (or whichever) when 
starting a new thread.



I want to write a string to an already-open file (sys.stdout, typically). 
However, I *don't* want encoding errors, and the string could be arbitrary 
Unicode (in theory). The best way I've found is

 data = data.encode(file.encoding, errors='replace').decode(file.encoding)
 file.write(data)

(I'd probably use backslashreplace rather than replace, but that's a minor 
point).

Is that the best way? The multiple re-encoding dance seems a bit clumsy, but it 
was the best I could think of.

Thanks,
Paul.



If you're going to take charge of the encoding of the file, why not just 
open the file in binary, and do it all with

file.write(data.encode( myencoding, errors='replace') )

i can't see the benefit of two encodes and a decode just to write a 
string to the file.


Alternatively, there's probably a way to open the file using 
codecs.open(), and reassign it to sys.stdout.



--
DaveA
--
https://mail.python.org/mailman/listinfo/python-list

Re: Writing list of dictionaries to CSV

Op Tuesday 5 May 2015 19:09 CEST schreef Kashif Rana:

> When I am writing list of dictionaries to CSV file, the key
> 'schedule' has value 'Mar 2012' becomes Mar-12. I really do not have
> clue why thats happening. Below is the code.
>
> dic_1 = {'action': 'permit', 'dst-address': 'maxprddb-scan-167,
> maxprddb-scan-168, maxprddb-scan-169', 'from': 'DMZ Web', 'id':
> '1000', 'log': 'Enable, session-init', 'name': 'Test Rule Temporary
> ', 'service': '1521, Oraccle-Maximo-Scan-1550, PING-ALL',
> 'src-address': 'sparkeregap1, sparkeregap2', 'to': 'Trust'}
>
> dic_2 {'action': 'permit',
> 'dst-address': 'sparkcas01, sparkcas02, email.ab.spark.net',
> 'from': 'DMZ Web',
> 'id': '4000',
> 'log': 'Enable, session-init',
> 'schedule': 'Mar 2012',
> 'service': 'SMTP',
> 'src-address': 'sparkeregap1, sparkeregap2',
> 'to': 'Trust'}
>
> my_list = [{'to': 'Trust', 'service': '1521,
> Oraccle-Maximo-Scan-1550, PING-ALL', 'from': 'DMZ Web',
> 'dst-address': 'maxprddb-scan-167, maxprddb-scan-168,
> maxprddb-scan-169', 'name': 'Test Rule Temporary ', 'action':
> 'permit', 'id': '1000', 'src-address': 'sparkeregap1, sparkeregap2',
> 'log': 'Enable, session-init'}, {'to': 'Trust', 'from': 'DMZ Web',
> 'dst-address': 'sparkcas01, sparkcas02, email.ab.spark.net',
> 'service': 'SMTP', 'schedule': 'Mar 2012', 'action': 'permit', 'id':
> '4000', 'src-address': 'sparkeregap1, sparkeregap2', 'log': 'Enable,
> session-init'}]
>
> pol_keys = ['id', 'name', 'from', 'to', 'src-address',
> 'dst-address', 'service', 'action', 'nat_status', 'nat_type',
> 'nat_src_ip', 'nat_dst_ip', 'nat_dst_port', 'log', 'schedule']
>
> with open('test.csv', 'wb') as f:
>   w = csv.DictWriter(f, pol_keys)
>   w.writeheader()
>   w.writerows(my_list)

Well, I get:

id,name,from,to,src-address,dst-address,service,action,nat_status,nat_type,nat_src_ip,nat_dst_ip,nat_dst_port,log,schedule
1000,Test Rule Temporary ,DMZ Web,Trust,"sparkeregap1, 
sparkeregap2","maxprddb-scan-167, maxprddb-scan-168, maxprddb-scan-169","1521, 
Oraccle-Maximo-Scan-1550, PING-ALL",permit,,"Enable, session-init",
4000,,DMZ Web,Trust,"sparkeregap1, sparkeregap2","sparkcas01, sparkcas02, 
email.ab.spark.net",SMTP,permit,,"Enable, session-init",Mar 2012

So there is something different on your system.


By the way: next time make sure that we can copy paste your code:
- dic_2 does not have a '='
- get rid of the space before my_list
- make sure the '[' is at the same line as my_list (or use a \)

-- 
Cecil Westerhof
Senior Software Engineer
LinkedIn: http://www.linkedin.com/in/cecilwesterhof
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Stripping unencodable characters from a string

On Tuesday, 5 May 2015 20:01:04 UTC+1, Dave Angel  wrote:
> On 05/05/2015 02:19 PM, Paul Moore wrote:
> 
> You need to specify that you're using Python 3.4 (or whichever) when 
> starting a new thread.

Sorry. 2.6, 2.7, and 3.3+. It's for use in a cross-version library.

> If you're going to take charge of the encoding of the file, why not just 
> open the file in binary, and do it all with
>  file.write(data.encode( myencoding, errors='replace') )

I don't have control of the encoding of the file. It's typically sys.stdout, 
which is already open. I can't replace sys.stdout (because the main program 
which calls my library code wouldn't like me messing with global state behind 
its back). And sys.stdout isn't open in binary mode.

> i can't see the benefit of two encodes and a decode just to write a 
> string to the file.

Nor can I - that's my point. But if all I have is an open text-mode file with 
the "strict" error mode, I have to incur one encode, and I have to make sure 
that no characters are passed to that encode which can't be encoded.

If there was a codec method to identify un-encodable characters, that might be 
an alternative (although it's quite possible that the encode/decode dance would 
be faster anyway, as it's mostly in C - not that performance is key here).

> Alternatively, there's probably a way to open the file using 
> codecs.open(), and reassign it to sys.stdout.

As I said, I have to work with the file (sys.stdout or whatever) that I'm 
given. I can't reopen or replace it.

Paul
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Stripping unencodable characters from a string

2015-05-05 Thread Jon Ribbens

On 2015-05-05, Paul Moore  wrote:
> I want to write a string to an already-open file (sys.stdout,
> typically). However, I *don't* want encoding errors, and the string
> could be arbitrary Unicode (in theory). The best way I've found is
>
> data = data.encode(file.encoding, errors='replace').decode(file.encoding)
> file.write(data)
>
> (I'd probably use backslashreplace rather than replace, but that's a
> minor point).
>
> Is that the best way? The multiple re-encoding dance seems a bit
> clumsy, but it was the best I could think of.

Perhaps something like one of:

  file.buffer.write(data.encode(file.encoding, errors="replace"))

or:

  sys.stdout = io.TextIOWrapper(sys.stdout.detach(),
  encoding=sys.stdout.encoding, errors="replace")

(both of which could go wrong in various ways depending on your
circumstances).
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Writing list of dictionaries to CSV

2015-05-05 Thread Matthew Ruffalo

On 2015-05-05 14:25, Skip Montanaro wrote:
> More likely, viewing the CSV file in Excel, Gnumeric, or some other
> spreadsheet which interprets some inputs as dates and formats them
> according to its default rules. Skip 

This is depressingly common, and I've even received CSV and plain text
data files that had some gene names(!) coerced to dates via a trip
through Excel. SEPT9 was one such gene, I believe.

MMR...
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Stripping unencodable characters from a string

2015-05-05 Thread Marko Rauhamaa

Paul  Moore :

> Nor can I - that's my point. But if all I have is an open text-mode
> file with the "strict" error mode, I have to incur one encode, and I
> have to make sure that no characters are passed to that encode which
> can't be encoded.

The file-like object you are given carries some baggage. IOW, it's not a
"file" in the sense you are thinking about it. It's some object that
accepts data with its write() method.

Now, Python file-like objects ostensibly implement a common interface.
However, as you are describing here, not all write() methods accept the
same arguments. Text file objects expect str objects while binary file
objects expect bytes objects. Maybe there are yet other file-like
objects that expect some other types of object as their arguments.

Bottom line: Python doesn't fulfill your expectation. Your library can't
operate on generic file-like objects because Python3 doesn't have
generic file-like objects. Your library must do something else. For
example, you could require a binary file object. The caller must then
possibly wrap their actual object inside a converter, which is
relatively trivial in Python.


Marko
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Throw the cat among the pigeons

Op Tuesday 5 May 2015 20:45 CEST schreef Dave Angel:

> On 05/05/2015 12:18 PM, Cecil Westerhof wrote:
>
>>
>> Well, I did not write many tail recursive functions. But what
>> surprised me was that for large values the ‘tail recursive’ version
>> was more efficient as the iterative version. And that was with
>> myself implementing the tail recursion. I expect the code to be
>> more efficient when the compiler implements the tail recursion.
>>
>
>
> You've said that repeatedly, so I finally took a look at your
> webpage
>
> https://github.com/CecilWesterhof/PythonLibrary/blob/master/mathDecebal.py
>
> I didn't have your framework to call the code, so I just extracted
> some functions and did some testing.

I definitely need to take care of documentation.

It can be called with:
python3 mathDecebal.py --factorial

The problem is that it will do correctness and speed test. I have to
split those in two different things. And use a different file for
both.

Maybe make a directory test and put a correctness_.py and a
speed_.py.



> I do see some differences,
> where the so-called tail_recursive functions are sometimes faster,
> but I did some investigating to try to determine why.
>
>
> I came up with the conclusion that sometimes the multiply operation
> takes longer than other times.  And in particular, i can see more
> variation between the two following loops than between your two
> functions.
>
>
> def factorial_iterative(x, simple=False):
> assert x >= 0
> result = 1
> j=2
> if not simple:
> for i in range(2, x + 1):
> result *= i
> j += 1
> else:
> for i in range(2, x + 1):
> result *= j
> j += 1
> pass
>
> return result
>
> When the "simple" is True, the function takes noticeably and
> consistently longer.  For example, it might take 116 instead of 109
> seconds.  For the same counts, your code took 111.
>
> I've looked at dis.dis(factorial_iterative), and can see no explicit
> reason for the difference.

I would say that a variable that is filled by a range is different as
a normal variable. Do not ask me why. ;-)

Even if you (general not personal you) think that the tail recursion
is a waist of time, this is an interesting result I think.

-- 
Cecil Westerhof
Senior Software Engineer
LinkedIn: http://www.linkedin.com/in/cecilwesterhof
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Throw the cat among the pigeons

On Tue, May 5, 2015 at 12:45 PM, Dave Angel  wrote:
> When the "simple" is True, the function takes noticeably and consistently
> longer.  For example, it might take 116 instead of 109 seconds.  For the
> same counts, your code took 111.

I can't replicate this. What version of Python is it, and what value
of x are you testing with?

> I've looked at dis.dis(factorial_iterative), and can see no explicit reason
> for the difference.

My first thought is that maybe it's a result of the branch. Have you
tried swapping the branches, or reimplementing as separate functions
and comparing?
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Throw the cat among the pigeons


On 5/5/2015 12:18 PM, Cecil Westerhof wrote:

Op Tuesday 5 May 2015 17:47 CEST schreef Paul Moore:


On Sunday, 3 May 2015 16:23:59 UTC+1, Cecil Westerhof  wrote:

By the way: I think that even if the recursion does not go further
as 500, it is still a good idea to use tail recursion. Why use
stack space when it is not necessary?


I pushed the example to GitHub:
https://github.com/CecilWesterhof/PythonLibrary/blob/master/mathDecebal.py


You already know this, as your code shows, but tail call recursion
elimination is only possible when you have a *direct* tail call (one


An 'indirect tail call' would not be a tail call.


with the result of the tail call returned immediately to the
caller). Even the standard trivial factorial example doesn't have a
direct tail call,  without rewriting to use an accumulator variable.
Which is a non-intuitive transformation to anyone who's not familiar
with recursive functional languages and their idioms.

If you're rewriting your recursive function *anyway*, it's not that
much harder in many (most?) cases to rewrite it iteratively.


For count functions, the main change between tail recursion and while 
iteration is replacing 'if' with 'while' and converting the tail call to 
assignment.  (One may have to reverse the if-else first to put the tail 
call in the if branch.)


from math import factorial as fac
print(fac(0), fac(1), fac(2), fac(6))

def fac_rt(n, i=2, res=1):
if i <= n:
return fac_rt(n, i+1, res*i)
else:
return res

def fac_iw(n):
i = 2
res = 1
while i <= n:
i, res = i+1, res*i
return res

for i in (0, 1, 2, 6):
print(fac(i) == fac_rt(i) == fac_iw(i))

>>>
1 1 2 720
True
True
True
True

For collection functions that process each item once, 'for item in 
collection: ...' is nearly always easier to write in the first place.



An example of a function that naturally uses direct tail call
recursion, but which doesn't have a simple iterative rewrite, would
be more compelling.


Simple, easily converted functions like the above, with one recursive 
call in one branch of an if-else, are the most common.  Something with 
multiple mutually exclusive tail calls, like the following


def f_rt1(*args):
  if nonbase1:
return f(*revised-args1)
  elif base_condition:
return base(args)
  else:
return f(*revised-args2)

must be converted to if-else with all tail calls in the if branch.

def f_rt2(*args):
  if not base_condition:
if nonbase1:
  return f(*revised-args1)
else:
  return f(*revised-args2)
  else:
return base(args)

Conversion would then be simple.  The complication is that the 
'base_condition' in the original version might not really be the base 
condition due to a dependence on nonbase1 being false.  This is a 
generic problem with reordering if-elif statements.


For non-linear (branching) recursion, in which multiple recursive calls 
may be made for one function call, the last recursive call may be a tail 
call.  An example is in-place quick sort.  Eliminating just the tail 
call may not be simple, but it also does not eliminate the possibility 
of blowing the call stack.  To do that, one must eliminate all recursive 
calls by adding explicit stacks.



Well, I did not write many tail recursive functions. But what surprised
me was that for large values the ‘tail recursive’ version was more
efficient as the iterative version.


In your first thread, what you mislabelled 'tail recursive version' was 
an iterative while loop version while the 'iterative version' was an 
iterative for loop version.  In this thread, you just posted timings 
without code. I will not believe your claim until I see one file that I 
can run myself with an actual tail recursive function, as above, that 
beats the equivalent while or for loop version.


--
Terry Jan Reedy


--
https://mail.python.org/mailman/listinfo/python-list

Re: Throw the cat among the pigeons


On 05/05/2015 04:30 PM, Ian Kelly wrote:

On Tue, May 5, 2015 at 12:45 PM, Dave Angel  wrote:

When the "simple" is True, the function takes noticeably and consistently
longer.  For example, it might take 116 instead of 109 seconds.  For the
same counts, your code took 111.


I can't replicate this. What version of Python is it, and what value
of x are you testing with?


I've looked at dis.dis(factorial_iterative), and can see no explicit reason
for the difference.


My first thought is that maybe it's a result of the branch. Have you
tried swapping the branches, or reimplementing as separate functions
and comparing?



Logic is quite simple:


def factorial_iterative(x, simple=False):
assert x >= 0
result = 1
j=2
if not simple:
for i in range(2, x + 1):
#print("range value is of type", type(i), "and value", i)
#print("ordinary value is of type", type(j), "and value", j)
result *= i
j += 1
else:
for i in range(2, x + 1):
result *= j
j += 1

return result

def loop(func, funcname, arg):
start = time.time()
for i in range(repeats):
func(arg, True)
print("{0}({1}) took {2:7.4}".format(funcname, arg, time.time()-start))

start = time.time()
for i in range(repeats):
func(arg)
print("{0}({1}) took {2:7.4}".format(funcname, arg, time.time()-start))

repeats = 1

and arg is 10**4
loop(factorial_iterative,  "factorial_iterative  ", arg)

My actual program does the same thing with other versions of the 
function, including Cecil's factorial_tail_recursion, and my optimized 
version of that.



Python 3.4.0 (default, Apr 11 2014, 13:05:11)
[GCC 4.8.2] on linux

factorial_iterative  (10) took   3.807
factorial_iterative  (10) took   3.664

factorial_iterative  (20) took   17.07
factorial_iterative  (20) took15.3

factorial_iterative  (30) took   38.93
factorial_iterative  (30) took   36.01


Note that I test them in the opposite order of where they appear in the 
function.  That's because I was originally using the simple flag to test 
an empty loop.  The empty loop is much quicker either way, so it's not 
the issue.  (But if it were, the for-range version is much quicker).


I think I'll take your last suggestion and write separate functions.



--
DaveA
--
https://mail.python.org/mailman/listinfo/python-list

Re: Throw the cat among the pigeons

On Tue, May 5, 2015 at 3:00 PM, Dave Angel  wrote:
> def loop(func, funcname, arg):
> start = time.time()
> for i in range(repeats):
> func(arg, True)
> print("{0}({1}) took {2:7.4}".format(funcname, arg, time.time()-start))
>
> start = time.time()
> for i in range(repeats):
> func(arg)
> print("{0}({1}) took {2:7.4}".format(funcname, arg, time.time()-start))

Note that you're explicitly passing True in one case but leaving the
default in the other. I don't know whether that might be responsible
for the difference you're seeing.

Also, it's best to use the timeit module for timing code, e.g.:

>>> from timeit import Timer
>>> t1 = Timer("factorial_iterative(10, False)", "from __main__ import 
>>> factorial_iterative")
>>> t1.repeat(10, number=1)
[3.8517245299881324, 3.7571076710009947, 3.7780062559759244,
3.848508063936606, 3.7627131739864126, 3.8278848479967564,
3.776115525048226, 3.83024005102925, 3.8322679550619796,
3.8195601429324597]
>>> min(_), sum(_) / len(_)
(3.7571076710009947, 3.8084128216956743)
>>> t2 = Timer("factorial_iterative(10, True)", "from __main__ import 
>>> factorial_iterative")
>>> t2.repeat(10, number=1)
[3.8363616950809956, 3.753201302024536, 3.7838632150087506,
3.7670978900277987, 3.805312803015113, 3.7682680500438437,
3.856655619922094, 3.796431727008894, 3.8224815409630537,
3.765664782957174]
>>> min(_), sum(_) / len(_)
(3.753201302024536, 3.7955338626052253)

As you can see, in my testing the True case was actually marginally
(probably not significantly) faster in both the min and the average.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Throw the cat among the pigeons

Op Tuesday 5 May 2015 22:46 CEST schreef Terry Reedy:

>> Well, I did not write many tail recursive functions. But what
>> surprised me was that for large values the ‘tail recursive’ version
>> was more efficient as the iterative version.
>
> In your first thread, what you mislabelled 'tail recursive version'
> was an iterative while loop version

That is because Python has no tail recursion, so I needed to program
the tail recursion myself. Tail recursion would do under the hood what
I did there manually.


> while the 'iterative version'
> was an iterative for loop version. In this thread, you just posted
> timings without code. I will not believe your claim until I see one
> file that I can run myself with an actual tail recursive function,
> as above, that beats the equivalent while or for loop version.

https://github.com/CecilWesterhof/PythonLibrary/blob/master/mathDecebal.py

-- 
Cecil Westerhof
Senior Software Engineer
LinkedIn: http://www.linkedin.com/in/cecilwesterhof
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Throw the cat among the pigeons

On Tue, May 5, 2015 at 3:23 PM, Ian Kelly  wrote:
> On Tue, May 5, 2015 at 3:00 PM, Dave Angel  wrote:
>> def loop(func, funcname, arg):
>> start = time.time()
>> for i in range(repeats):
>> func(arg, True)
>> print("{0}({1}) took {2:7.4}".format(funcname, arg, time.time()-start))
>>
>> start = time.time()
>> for i in range(repeats):
>> func(arg)
>> print("{0}({1}) took {2:7.4}".format(funcname, arg, time.time()-start))
>
> Note that you're explicitly passing True in one case but leaving the
> default in the other. I don't know whether that might be responsible
> for the difference you're seeing.

I don't think that's the cause, but I do think that it has something
to do with the way the timing is being run. When I run your loop
function, I do observe the difference. If I reverse the order so that
the False case is tested first, I observe the opposite. That is, the
slower case is consistently the one that is timed *first* in the loop
function, regardless of which case that is.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Throw the cat among the pigeons


On 05/05/2015 05:39 PM, Ian Kelly wrote:

On Tue, May 5, 2015 at 3:23 PM, Ian Kelly  wrote:

On Tue, May 5, 2015 at 3:00 PM, Dave Angel  wrote:

def loop(func, funcname, arg):
 start = time.time()
 for i in range(repeats):
 func(arg, True)
 print("{0}({1}) took {2:7.4}".format(funcname, arg, time.time()-start))

 start = time.time()
 for i in range(repeats):
 func(arg)
 print("{0}({1}) took {2:7.4}".format(funcname, arg, time.time()-start))


Note that you're explicitly passing True in one case but leaving the
default in the other. I don't know whether that might be responsible
for the difference you're seeing.


I don't think that's the cause, but I do think that it has something
to do with the way the timing is being run. When I run your loop
function, I do observe the difference. If I reverse the order so that
the False case is tested first, I observe the opposite. That is, the
slower case is consistently the one that is timed *first* in the loop
function, regardless of which case that is.



I created two functions and called them with Timeit(), and the 
difference is now below 3%


And when I take your lead and double the loop() function so it runs each 
test twice, I get steadily decreasing numbers.


I think all of this has been noise caused by the caching of objects 
including function objects.  I was surprised by this, as the loops are 
small enough I'd figure the function object would be fully cached the 
first time it was called.


--
DaveA
--
https://mail.python.org/mailman/listinfo/python-list

Re: Writing list of dictionaries to CSV

2015-05-05 Thread Tim Chase

On 2015-05-05 10:09, Kashif Rana wrote:
> When I am writing list of dictionaries to CSV file, the key
> 'schedule' has value 'Mar 2012' becomes Mar-12. 

How are you making this determination?  Are you looking at the raw
CSV output, or are you looking at the CSV file loaded into a
spreadsheet like Excel?

I suspect that it's a matter of the file being what you want, but
your spreadsheet mangling/reformatting it visually.

-tkc

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: asyncio: What is the difference between tasks, futures, and coroutines?

On 5/5/2015 1:46 PM, Ian Kelly wrote:

On Tue, May 5, 2015 at 9:22 AM, Paul Moore wrote:

I'm working my way through the asyncio documentation. I have got to the "Tasks and
coroutines" section, but I'm frankly confused as to the difference between the
various things described in that section: coroutines, tasks, and futures.

I think can understand a coroutine. Correct me if I'm wrong, but it's roughly
"something that you can run which can suspend itself".

But I don't understand what a Future is. The document just says it's almost the same as a
concurrent.futures.Future, which is described as something that "encapsulates the asynchronous
execution of a callable". Which doesn't help a lot. In concurrent.futures, you don't create
Futures, you get them back from submit(), but in the asyncio docs it looks like you can create them
by hand (example "Future with run_until_complete"). And there's nothing that says what a
Future is, just what it's like... :-(

Fundamentally, a future is a placeholder for something that isn't
available yet. You can use it to set a callback to be called when that
thing is available, and once it's available you can get that thing
from it.

A Task is a subclass of Future, but the documentation doesn't say what it *is*, but
rather that it "schedules the execution of a coroutine". But that doesn't make
sense to me - objects don't do things, they *are* things. I thought the event loop did
the scheduling?

In asyncio, a Task is a a Future that serves as a placeholder for the
result of a coroutine. The event loop manages callbacks, and that's
all it does. An event that it's been told to listen for occurs, and
the event loop calls the callback associated with that event. The Task
manages a coroutine's interaction with the event loop; when the
coroutine yields a future, the Task instructs the event loop to listen
for the completion of that future, setting a callback that will resume
the coroutine.

Reading between the lines, it seems that the event loop schedules Tasks (which
makes sense) and that Tasks somehow wrap up coroutines - but I don't see *why*
you need to wrap a task in a coroutine rather than just scheduling coroutines.
And I don't see where Futures fit in - why not just wrap a coroutine in a
Future, if it needs to be wrapped up at all?

The coroutines themselves are not that interesting of an interface;
all you can do with them is resume them. The asynchronous execution
done by asyncio is all based on futures. Because a coroutine can
easily be wrapped in a Task, this allows for coroutines to be used
anywhere a future is expected.

I don't know if I've done a good job explaining, but I hope this helps.

It helps me. Thanks.

--
Terry Jan Reedy

--
https://mail.python.org/mailman/listinfo/python-list

Re: Is it normal to cry when given XML?

2015-05-05 Thread Emile van Sebille


On 5/5/2015 9:54 AM, Michael Torrie wrote:

On 05/05/2015 03:28 AM, Sayth Renshaw wrote:

Hi

Just checking if the reaction to cry when given XML is normal.


I'd say it is normal.  XML is like violence.  If it doesn't solve your
problems, you're not using enough of it[1].

[1] Can anyone tell me who originated this line?



I find references from 2006 attributing the quote to "Ted Neward" but 
can't confirm that's where it started.


Emile


--
https://mail.python.org/mailman/listinfo/python-list

PyPy Development: CFFI 1.0 beta 1

2015-05-05 Thread Mark Lawrence


Read all about it http://morepypy.blogspot.co.uk/2015/05/cffi-10-beta-1.html

--
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence

--
https://mail.python.org/mailman/listinfo/python-list

Re: Throw the cat among the pigeons


On 5/5/2015 5:12 PM, Cecil Westerhof wrote:

Op Tuesday 5 May 2015 22:46 CEST schreef Terry Reedy:


Well, I did not write many tail recursive functions. But what
surprised me was that for large values the ‘tail recursive’ version
was more efficient as the iterative version.


In your first thread, what you mislabelled 'tail recursive version'
was an iterative while loop version


That is because Python has no tail recursion,


Yes is does. Please stop using 'tail recursion' to refer to the absence 
of recursion or the process of removing recursion. I wrote a tail 
recursive function, and I believe you did too.


What Python does not have is automatic tail call optimization, or 
specifically, tail recursion *elimination*. Both possibilities, the 
general and specific, have been considered and rejected for excellent 
reasons. I hinted at some of them in my post.


See https://en.wikipedia.org/wiki/Tail_call
for 'tail recursion' and 'tail call elimination'

What I believe you showed is that a while loop can be faster than a more 
or less equivalent for loop that produces the same result by a slightly 
different method.  That is not surprising.  Relative timings for CPython 
vary a few percent between different combinations of Python version, C 
compiler and settings, operating system, and cpu and other hardware.


--
Terry Jan Reedy


--
https://mail.python.org/mailman/listinfo/python-list

Re: Is it normal to cry when given XML?

2015-05-05 Thread Mark Lawrence


On 05/05/2015 10:28, Sayth Renshaw wrote:

Hi

Just checking if the reaction to cry when given XML is normal.

I thought maybe I am approaching it all wrong, using lxml largely or some 
xquery to club it into submission.

See the usual goal is just to take the entire XML and push it into a database. 
or in future experiment with Mongo or Hdf5 .

See its never basic xml, usually comes from some database with a walk of tables 
and strange relationships.

Am I doing it wrong is there a simple way I am missing?

Sayth



When the two young lads from the consultants I was working with back in 
2000 found a bug with xml handling in the supplier's software I almost 
ended up in tears.  Why?  They didn't bother reporting it, they simply 
modified all their unit testing data to work around the problem, so when 
we got to integration testing nothing worked at all.


--
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence

--
https://mail.python.org/mailman/listinfo/python-list

Beacon controller

2015-05-05 Thread bmanogna . 17

Hi,
I'm new to pyhton, can anyone suggest me how can I implement a DOS or DDOS 
attack in Beacon Controller.
Thanks,
Manogna.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: CHALLENGE HELP - GOOGLE DEVELOPER DAY

On Wed, May 6, 2015 at 2:59 AM,   wrote:
> Good afternoon everyone.
>
> I'm with the following exercise of the option is a modification of a google 
> developer day exercise.
>
> SOMEONE HELP ME IN THIS CHALLENGE?

You haven't posted any of your own code, so you're just asking for
someone to do it for you. No, I will not.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Stripping unencodable characters from a string

On Wed, May 6, 2015 at 4:19 AM, Paul  Moore  wrote:
> I want to write a string to an already-open file (sys.stdout, typically). 
> However, I *don't* want encoding errors, and the string could be arbitrary 
> Unicode (in theory). The best way I've found is
>
> data = data.encode(file.encoding, errors='replace').decode(file.encoding)
> file.write(data)
>
> (I'd probably use backslashreplace rather than replace, but that's a minor 
> point).
>
> Is that the best way? The multiple re-encoding dance seems a bit clumsy, but 
> it was the best I could think of.

The simplest solution would be to call ascii() on the string, which
will give you an ASCII-only representation (using backslash escapes).
If your goal is to write Unicode text to a log file in some safe way,
this is what I would be doing.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Is it normal to cry when given XML?

On Wed, May 6, 2015 at 9:35 AM, Mark Lawrence  wrote:
> When the two young lads from the consultants I was working with back in 2000
> found a bug with xml handling in the supplier's software I almost ended up
> in tears.  Why?  They didn't bother reporting it, they simply modified all
> their unit testing data to work around the problem, so when we got to
> integration testing nothing worked at all.

See, that's not a problem with XML. That's not even a problem with
unit testing. That's a problem with the belief that the most important
thing is to have your tests pass. Sigh.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Beacon controller

On Wed, May 6, 2015 at 9:37 AM,   wrote:
> Hi,
> I'm new to pyhton, can anyone suggest me how can I implement a DOS or DDOS 
> attack in Beacon Controller.
> Thanks,
> Manogna.

http://www.catb.org/esr/faqs/hacker-howto.html#I_want_to_crack_and_Im_an_idiot

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Is it normal to cry when given XML?

2015-05-05 Thread Rustom Mody

On Wednesday, May 6, 2015 at 5:38:12 AM UTC+5:30, Chris Angelico wrote:
> On Wed, May 6, 2015 at 9:35 AM, Mark Lawrence  wrote:
> > When the two young lads from the consultants I was working with back in 2000
> > found a bug with xml handling in the supplier's software I almost ended up
> > in tears.  Why?  They didn't bother reporting it, they simply modified all
> > their unit testing data to work around the problem, so when we got to
> > integration testing nothing worked at all.
> 
> See, that's not a problem with XML. That's not even a problem with
> unit testing. That's a problem with the belief that the most important
> thing is to have your tests pass. Sigh.
> 

I smell a PHB¹ in the wings...
---
¹ Pointy-haired-boss
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Is it normal to cry when given XML?

On Wed, May 6, 2015 at 10:19 AM, Rustom Mody  wrote:
> On Wednesday, May 6, 2015 at 5:38:12 AM UTC+5:30, Chris Angelico wrote:
>> On Wed, May 6, 2015 at 9:35 AM, Mark Lawrence  wrote:
>> > When the two young lads from the consultants I was working with back in 
>> > 2000
>> > found a bug with xml handling in the supplier's software I almost ended up
>> > in tears.  Why?  They didn't bother reporting it, they simply modified all
>> > their unit testing data to work around the problem, so when we got to
>> > integration testing nothing worked at all.
>>
>> See, that's not a problem with XML. That's not even a problem with
>> unit testing. That's a problem with the belief that the most important
>> thing is to have your tests pass. Sigh.
>>
>
> I smell a PHB¹ in the wings...

It seems likely, yes. That's where the "c'mon guys, get those tests to
pass" impetus usually comes from. The geeks usually know that tests
are a tool, not a goal, although the more evil ones might still wield
that knowledge...

http://bofh.ntk.net/BOFH/1998/bastard98-35.php

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Throw the cat among the pigeons

On Wed, 6 May 2015 07:23 am, Ian Kelly wrote:

> On Tue, May 5, 2015 at 3:00 PM, Dave Angel  wrote:
>> def loop(func, funcname, arg):
>> start = time.time()
>> for i in range(repeats):
>> func(arg, True)
>> print("{0}({1}) took {2:7.4}".format(funcname, arg,
>> time.time()-start))
>>
>> start = time.time()
>> for i in range(repeats):
>> func(arg)
>> print("{0}({1}) took {2:7.4}".format(funcname, arg,
>> time.time()-start))
> 
> Note that you're explicitly passing True in one case but leaving the
> default in the other. I don't know whether that might be responsible
> for the difference you're seeing.

It will be responsible for *some* difference, it is certainly faster to pass
one argument than two, but likely an entirely trivial amount compared to
the time to iterate some large number of times.

> Also, it's best to use the timeit module for timing code, e.g.:
> 
 from timeit import Timer
 t1 = Timer("factorial_iterative(10, False)", "from __main__ import
 factorial_iterative") t1.repeat(10, number=1)
> [3.8517245299881324, 3.7571076710009947, 3.7780062559759244,
> 3.848508063936606, 3.7627131739864126, 3.8278848479967564,
> 3.776115525048226, 3.83024005102925, 3.8322679550619796,
> 3.8195601429324597]
 min(_), sum(_) / len(_)
> (3.7571076710009947, 3.8084128216956743)

Only the minimum is statistically useful.

The reason is that every timing measurement has an error, due to random
fluctuations of the OS, CPU pipelining, caches, etc, but that error is
always positive. The noise always makes the code take longer to run, not
faster!

So we can imagine every measurement is made up of the "true" value T, plus
an error, where T = the perfectly repeatable timing that the function would
take to run if we could somehow eliminate every possible source of noise
and error in the system. We can't, of course, but we would like to estimate
T as closely as possible.

Without loss of generality, suppose we collect five timings:

times = [T+ε1, T+ε2, T+ε3, T+ε4, T+ε5]

where the epsilons ε are unknown errors due to noise. We don't know the
distribution of the errors, except that no epsilon can be less than zero.

We want an estimate for T, something which comes as close as possible to T.
It should be obvious that the following relationships hold:

mean(times) = T + mean([ε1, ε2, ε3, ε4, ε5])

min(times) = T + min([ε1, ε2, ε3, ε4, ε5])

in other words, both the mean of the timings and the minimum of the timings
are equivalent to the "true" timing, plus an error. It should also be
obvious that the mean of the epsilons must be larger than the smallest of
the errors (except in the miraculous case where all the epsilons are
exactly the same).

So the statistic with the minimum error is the minimum. Any other stat,
whether you use the mean, geometric mean, harmonic mean, mode, median, or
anything else, will have a larger error and be a worse estimate for
the "true" value T.

All because the errors are one-sided: they always make the timing take
longer, never less time.

-- 
Steven

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Throw the cat among the pigeons

On Wed, 6 May 2015 02:18 am, Cecil Westerhof wrote:

> Well, I did not write many tail recursive functions. But what surprised
> me was that for large values the ‘tail recursive’ version was more
> efficient as the iterative version. And that was with myself
> implementing the tail recursion. I expect the code to be more
> efficient when the compiler implements the tail recursion.

You cannot know that. Python is not C or Java, and your intuition as to what
will be fast and what will be slow will not be accurate in Python. Python
has its own fast paths, and slow paths, and they are not the same as C's or
Java's.

I have been using Python for over 15 years, and I would not want to guess
whether a hypothetical compiler-generated tail-recursion-elimination
function would be faster or slower than manually unrolling the recursion
into a while loop.

I am surprised that you find a manually unrolled version with a while loop
is faster than an iterative version. I assume the iterative version uses a
for-loop. In my experience, a for-loop is faster than a while loop.

But perhaps that depends on the exact version and platform.

-- 
Steven

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Throw the cat among the pigeons

On Wed, 6 May 2015 05:42 am, Cecil Westerhof wrote:

> I would say that a variable that is filled by a range is different as
> a normal variable. Do not ask me why. ;-)

I would say that you are wrong. If I have understood you correctly, that
cannot possibly be the case in Python, all Python variables work the same
way[1], and none of them can remember where they came from.

[1] Well, technically local variables and global variables are implemented
differently, locals inside a function use a C array and globals use a dict,
but apart from a few implementation details like that, any two variables in
the same scope[2] operate identically.

[2] Anyone who raises the issue of exec() or import * inside a function body
in Python 2 will be slapped with a halibut.

-- 
Steven

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: CHALLENGE HELP - GOOGLE DEVELOPER DAY

2015-05-05 Thread John Ladasky

On Tuesday, May 5, 2015 at 4:59:51 PM UTC-7, Chris Angelico wrote:
> On Wed, May 6, 2015 at 2:59 AM,   wrote:
> > Good afternoon everyone.
> >
> > I'm with the following exercise of the option is a modification of a google 
> > developer day exercise.
> >
> > SOMEONE HELP ME IN THIS CHALLENGE?
> 
> You haven't posted any of your own code, so you're just asking for
> someone to do it for you. No, I will not.
> 
> ChrisA

+1.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Throw the cat among the pigeons

On Wed, May 6, 2015 at 12:57 PM, Steven D'Aprano
 wrote:
> On Wed, 6 May 2015 05:42 am, Cecil Westerhof wrote:
>
>> I would say that a variable that is filled by a range is different as
>> a normal variable. Do not ask me why. ;-)
>
>
> I would say that you are wrong. If I have understood you correctly, that
> cannot possibly be the case in Python, all Python variables work the same
> way[1], and none of them can remember where they came from.

My reading of the code was that one had two variables and the other
had just one. Could be the noise in the numbers came from cache
locality differences.

No difference between the variables, just a peculiarity of CPUs and
how they use memory.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Throw the cat among the pigeons

On Sun, 3 May 2015 12:20 am, Cecil Westerhof wrote:

[...]
> But to my surprise tail recursion could even be more efficient. I
> wrote two different versions of factorial with self implemented tail
> recursion. For bigger values both are more efficient. And I expect
> that if the tail recursion is done by the compiler instead of by hand,
> it will be a little faster still.

I decided to do some experimentation.

Firstly, when timing code, one should minimise the amount of "scaffolding"
used around the code you care about. Using a single function like this:

def factorial_iterative(x, flag):
if flag:
# first implementation
else:
# second implementation

is a bad idea, because you are not just timing the implementations, but also
the scaffolding that selects between them. Also, you increase the size of
the function, which increases the chance of cache misses and other
processor effects. So first step is to split the implementations into
separate functions. Here are my four implementations, the first two are
taken from your code:

def factorial_forloop1(n):
assert n >= 0
result = 1
for i in range(2, n + 1):
result *= i
return result

def factorial_forloop2(n):
# Version with a silly extra variable.
assert n >= 0
result = 1
j=2
for i in range(2, n + 1):
result *= j
j += 1
return result

from operator import mul
try:
reduce
except NameError:
from functools import reduce

def factorial_reduce(n):
assert n >= 0
return reduce(mul, range(2, n+1), 1)

def factorial_while(n):
assert n >= 0
result = 1
while n > 1:
result *= n
n -= 1
return result

A quick test to verify that they return the same results:

py> factorial_while(10)
3628800
py> factorial_reduce(10)
3628800
py> factorial_forloop1(10)
3628800
py> factorial_forloop2(10)
3628800

There's no point in optimising code that does the wrong thing!

Now, let's do some timing tests. It is best to use a well-tested timing
framework rather than invent your own dodgy one, so I use the timeit
module. Read the comments in the module to see why rolling your own is a
bad idea.

I'll start with some relative small input sizes. Remember, the inputs may be
small, but the outputs will be very large.

from timeit import Timer
code = "fact(10); fact(20); fact(30)"
t1 = Timer(code, setup="from __main__ import factorial_while as fact")
t2 = Timer(code, setup="from __main__ import factorial_reduce as fact")
t3 = Timer(code, setup="from __main__ import factorial_forloop1 as fact")
t4 = Timer(code, setup="from __main__ import factorial_forloop2 as fact")

I'm in a bit of a hurry, so I may be a bit more slap-dash than normal.
Normally, I would pick a larger number of trials, and a larger number of
iterations per trial, but here I'm going to use just best of three trials,
each of 1 iterations each:

for t in (t1, t2, t3, t4):
print(min(t.repeat(repeat=3, number=1)))

which prints:

0.22797810286283493  # while
0.17145151272416115  # reduce
0.16230526939034462  # for-loop
0.22819281555712223  # silly for-loop

(Comments added by me.) See my earlier post for why the minimum is the only
statistic worth looking at. These results show that:

- the version using while is significantly slower than the more
straightforward iterative versions using either reduce or a for loop.

- adding an extra, unnecessary, variable to the for-loop, and incrementing
it by hand, carries as much overhead as using a while loop;

- reduce is slightly slower than a pure Python for-loop (at least according
to this simple trial, on my computer -- your results may vary);

- the obvious winner is the straightforward iterative version with a
for-loop.

Now I'm going to test it with a larger input:

py> big = factorial_forloop1(5)
py> sys.getsizeof(big)  # number of bytes
94460
py> len(str(big))  # number of digits
213237

Recreate the timer objects, and (again, because I'm in something of a hurry)
do a best-of-three with just 2 iterations per trial.

code = "fact(5)"
t1 = Timer(code, setup="from __main__ import factorial_while as fact")
t2 = Timer(code, setup="from __main__ import factorial_reduce as fact")
t3 = Timer(code, setup="from __main__ import factorial_forloop1 as fact")
t4 = Timer(code, setup="from __main__ import factorial_forloop2 as fact")
for t in (t1, t2, t3, t4):
print(min(t.repeat(repeat=3, number=2)))

which takes about two minutes on my computer, and prints:

8.604736926034093  # while loop
10.786483339965343  # reduce
10.91099695302546  # for loop
10.821452282369137  # silly version of the for loop

(Again, annotations are by me.)

These results are fascinating, and rather surprising. I think that they
demonstrate that, at this size of input argument, the time is dominated by
processing the BigNum int objects, not the iteration: whether we use
reduce, a straight-forward for-loop, or a for-loop with an extra variable
makes very

Re: asyncio: What is the difference between tasks, futures, and coroutines?

2015-05-05 Thread Rustom Mody

On Tuesday, May 5, 2015 at 11:15:42 PM UTC+5:30, Marko Rauhamaa wrote:

> Personally, I have never found futures a very useful idiom in any
> language (Scheme, Java, Python). Or more to the point, concurrency and
> the notion of a function don't gel well in my mind.

Interesting comment.

It strikes me that the FP crowd has stretched the notion of function beyond 
recognition
And the imperative/OO folks have distorted it beyond redemption.

And the middle road shown by Pascal has been overgrown with weeds for some 40 
years...

If the classic Pascal (or Fortran or Basic) sibling balanced abstractions of 
function-for-value
procedure-for-effect were more in the collective consciousness rather than C's
travesty of function, things might not have been so messy.

Well... Dont feel right bashing C without some history...

C didn't start the mess of mixing procedure and function -- Lisp/Apl did.
Nor the confusion of = for assignment; Fortran did that.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Writing list of dictionaries to CSV

2015-05-05 Thread Kashif Rana

Hello guys

thanks for the feedback. I think its problem with excel itself, showing wrong 
value. Because when I opened the csv file in text editor, I can see correct 
value but opening in excel showing wrong value. What I can do to see correct in 
excel as well.

Regards

On Tuesday, May 5, 2015 at 9:09:40 PM UTC+4, Kashif Rana wrote:
> Hello Experts
> 
> When I am writing list of dictionaries to CSV file, the key 'schedule' has 
> value 'Mar 2012' becomes Mar-12. I really do not have clue why thats 
> happening. Below is the code. 
> 
> dic_1 = {'action': 'permit',
>  'dst-address': 'maxprddb-scan-167, maxprddb-scan-168, maxprddb-scan-169',
>  'from': 'DMZ Web',
>  'id': '1000',
>  'log': 'Enable, session-init',
>  'name': 'Test Rule Temporary ',
>  'service': '1521, Oraccle-Maximo-Scan-1550, PING-ALL',
>  'src-address': 'sparkeregap1, sparkeregap2',
>  'to': 'Trust'}
> 
> dic_2 {'action': 'permit',
>  'dst-address': 'sparkcas01, sparkcas02, email.ab.spark.net',
>  'from': 'DMZ Web',
>  'id': '4000',
>  'log': 'Enable, session-init',
>  'schedule': 'Mar 2012',
>  'service': 'SMTP',
>  'src-address': 'sparkeregap1, sparkeregap2',
>  'to': 'Trust'}
>  
>  my_list = 
>  [{'to': 'Trust', 'service': '1521, Oraccle-Maximo-Scan-1550, PING-ALL', 
> 'from': 'DMZ Web', 'dst-address': 'maxprddb-scan-167, maxprddb-scan-168, 
> maxprddb-scan-169', 'name': 'Test Rule Temporary ', 'action': 'permit', 'id': 
> '1000', 'src-address': 'sparkeregap1, sparkeregap2', 'log': 'Enable, 
> session-init'}, {'to': 'Trust', 'from': 'DMZ Web', 'dst-address': 
> 'sparkcas01, sparkcas02, email.ab.spark.net', 'service': 'SMTP', 'schedule': 
> 'Mar 2012', 'action': 'permit', 'id': '4000', 'src-address': 'sparkeregap1, 
> sparkeregap2', 'log': 'Enable, session-init'}]
> 
>  pol_keys = ['id', 'name', 'from', 'to', 'src-address', 'dst-address', 
> 'service', 'action', 'nat_status', 'nat_type', 'nat_src_ip', 'nat_dst_ip', 
> 'nat_dst_port', 'log', 'schedule']
>  
> with open('test.csv', 'wb') as f:
>   w = csv.DictWriter(f, pol_keys)
>   w.writeheader()
>   w.writerows(my_list)
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: asyncio: What is the difference between tasks, futures, and coroutines?

On Wednesday 06 May 2015 14:47, Rustom Mody wrote:

> It strikes me that the FP crowd has stretched the notion of function
> beyond recognition And the imperative/OO folks have distorted it beyond
> redemption.

In what way?

> And the middle road shown by Pascal has been overgrown with weeds for some
> 40 years...

As much as I like Pascal, and am pleased to see someone defending it, I'm 
afraid I have no idea what you mean by this.

> If the classic Pascal (or Fortran or Basic) sibling balanced abstractions
> of function-for-value procedure-for-effect were more in the collective
> consciousness rather than C's travesty of function, things might not have
> been so messy.

I'm not really sure that having distinct procedures, as opposed to functions 
that you just ignore their return result, makes *such* a big difference. Can 
you explain what is the difference between these?

sort(stuff)  # A procedure.
sort(stuff)  # ignore the function return result

And why the first is so much better than the second?

> Well... Dont feel right bashing C without some history...
> 
> C didn't start the mess of mixing procedure and function -- Lisp/Apl did.
> Nor the confusion of = for assignment; Fortran did that.

Pardon, but = has been used for "assignment" for centuries, long before 
George Boole and Ada Lovelace even started working on computer theory. Just 
ask mathematicians:

"let y = 23"

Is that not a form of assignment?

-- 
Steve

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Throw the cat among the pigeons

On Tue, May 5, 2015 at 7:27 PM, Steven D'Aprano
 wrote:
> Only the minimum is statistically useful.

I disagree. The minimum tells you how fast the code *can* run, under
optimal circumstances. The mean tells you how fast it *realistically*
runs, under typical load. Both can be useful to measure.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Throw the cat among the pigeons

On Wednesday 06 May 2015 14:05, Steven D'Aprano wrote:

[...]

Here are those anomalous timing results again:

> code = "fact(5)"
> t1 = Timer(code, setup="from __main__ import factorial_while as fact")
> t2 = Timer(code, setup="from __main__ import factorial_reduce as fact")
> t3 = Timer(code, setup="from __main__ import factorial_forloop1 as fact")
> t4 = Timer(code, setup="from __main__ import factorial_forloop2 as fact")
> for t in (t1, t2, t3, t4):
> print(min(t.repeat(repeat=3, number=2)))
> 
> 
> which takes about two minutes on my computer, and prints:
> 
> 8.604736926034093  # while loop
> 10.786483339965343  # reduce
> 10.91099695302546  # for loop
> 10.821452282369137  # silly version of the for loop
[...]
> What is surprising is that for very large input like this, the while loop
> is significantly faster than reduce or either of the for-loops. I cannot
> explain that result.

I re-ran the results on a different computer, and got similar results:

7.364120149984956
9.26512472704053
9.141491871327162
9.16900822892785

The while loop is surprisingly faster for calculating fact(5). A thought 
came to mind: the while loop is different from the other three versions, in 
that it performs the multiplications from n down to 2, rather than 2 up to 
n. Maybe that has something to do with it?

Introducing version five of the factorial:

def factorial_reduce2(n):
assert n >= 0
return reduce(mul, range(n, 1, -1), 1)

t5 = Timer(code, setup="from __main__ import factorial_reduce2 as fact")
for t in (t1, t2, t3, t4, t5):
print(min(t.repeat(repeat=3, number=2)))

And results:

7.36792928725481  # while loop
9.271950023248792  # reduce, counting up
9.14769447594881  # for loop
9.154150342568755  # silly for loop
7.430811045691371  # reduce, counting down

My interpretation of this is that the difference has something to do with 
the cost of multiplications. Multiplying upwards seems to be more expensive 
than multiplying downwards, a result I never would have predicted, but 
that's what I'm seeing. I can only guess that it has something to do with 
the way multiplication is implemented, or perhaps the memory management 
involved, or something. Who the hell knows?

Just to be sure:

def factorial_forloop3(n):
assert n >= 0
result = 1
for i in range(n, 1, -1):
result *= i
return result

t6 = Timer(code, setup="from __main__ import factorial_forloop3 as fact")
print(min(t6.repeat(repeat=3, number=2)))

which prints 7.36256698705256.

Lesson to be learned: what you think is important may not be, and what you 
think is irrelevant may be.

Historical fact: for a period of about 60 years, long after the trick to 
curing and preventing scurvy was known, the British navy suffered greatly 
from scurvy again (although not to the same degree as they had been in the 
17th and 18th centuries). The problem was due to confusion between *lemons* 
and *limes*. The navy started by using the term "lime" for both lemons and 
sweet limes from the Mediterranean, as well as West Indian limes: three 
different citrus fruit. To cut costs and encourage British commerce, the 
navy gradually stopping buying lemons from Spain and swapped over almost 
entirely to West Indian limes. Unfortunately limes, and especially West 
Indian limes, have significantly less Vitamin C than lemons, and the 
sailors' daily allowance was about 75% less effective on ships that used 
limes than for those that used Spanish lemons. Scurvy, which had practically 
be eradicated in the 1850s and 60s, returned, and was a significant factor 
in World War One. (Especially for the French and Russian armies.)

This simple confusion wasn't sorted out until the 1920s, long after Vitamin 
C was isolated and named. Apparently nobody noticed, or cared, that the two 
different fruit tasted different and looked different, or concluded that 
perhaps they had different medicinal properties.

But then, for many decades, vinegar and dilute sulphuric acid were 
recommended as cures for scurvy *solely* on the basis that they were acidic 
just like proven cures oranges, lemons and sauerkraut.

-- 
Steven

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Writing list of dictionaries to CSV

On Wed, May 6, 2015 at 3:32 PM, Kashif Rana  wrote:
> thanks for the feedback. I think its problem with excel itself, showing wrong 
> value. Because when I opened the csv file in text editor, I can see correct 
> value but opening in excel showing wrong value. What I can do to see correct 
> in excel as well.
>

Simple fix: Ditch Excel.

Slightly more complicated fix: Open the file, then select everything,
and tell Excel that it's plain text. You can't say that in the CSV
file.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Encrypt python files

2015-05-05 Thread Palpandi

Hi,

What are the ways to encrypt python files?
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Encrypt python files

2015-05-05 Thread David Palao

Hello,
I'm afraid your question is either not well defined (or not well
enough) or wrong for this list, at least as I understand it.
Could you please explain it better?

Best

2015-05-06 8:37 GMT+02:00 Palpandi :
> Hi,
>
> What are the ways to encrypt python files?
> --
> https://mail.python.org/mailman/listinfo/python-list
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Encrypt python files