Re: SSL Socket Error

[Kev Dwyer]

shdwkee...@gmail.com wrote:

> Ive setup a Debian server with Python 2.7.9.  Have everything running and
> SSL as well, but when I try and login to using HTTPS:// I get this error:
> 
> Incoming web connection from ('192.168.15.177', 53202)
> error: uncaptured python exception, closing channel  listening :8111 at 0x75ea7b48> (:certfile must be specified for
> server-side operations [/usr/lib/python2.7/asyncore.py|read|83]
> [/usr/lib/python2.7/asyncore.py|handle_read_event|443]
> [./alarmserver.py|handle_accept|456]
> [/usr/lib/python2.7/ssl.py|wrap_socket|891]
> [/usr/lib/python2.7/ssl.py|init|498])
> 
> Any ideas how to resolve?

It looks like the exception is coming up through alarmserver.py, which seems 
to be based on https://github.com/juggie/AlarmServer (I'm not entirely
certain because the line numbers don't seem to match).  Anyway, looking at 
the code for alarmserver.py it expects to find certificate paths defined in
its config file, but they are undefined.

The sample config file for the project includes this section:

## The server runs with SSL. You need a certificate and key
## server.crt and server.key are included but you should 
## generate your own.
## If left blank the default included cert/key will be used
#certfile=/etc/apache2/ssl/server.crt
#keyfile=/etc/apache2/ssl/server.key
certfile=
keyfile=

So I think you need to start by examining the config file on your server.

Good luck,

Kev

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: calling base class method fetches no results

[Mark Lawrence]

On 09/05/2015 07:41, david jhon wrote:

Hello everyone,

I am new to python and trying to run an example code from mininet tests.
Basically, I am trying to call a method in Hcontroller.py from base class
Routing defined in DCRouting.py which runs and fetches all the required
results in install_reactive_path() method, but it returns None when it is
called from _GlobalFirstFit. I hope someone here could help me fix this
bug..

I am attaching all the three files(DCRouting.py, HController.py, util.py)
to have a look into. Thanks in advance for your time, help or suggestion.
Thanks a lot!

kind regards,
David



I'm sorry but I'm not wading through nearly 30kb of code in five 
attachments.  Please see http://sscce.org/ for how to put your question 
so you're more likely to get answers.


--
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence

--
https://mail.python.org/mailman/listinfo/python-list

Re: functions, optional parameters

[Gregory Ewing]

Chris Angelico wrote:


So no, it
isn't proof - it's equally well explained by the code object being
constant.


I suppose, strictly speaking, that's true -- but
then the code object *might as well* be created
at compile time, since the semantics are identical.

In any case, it's easy to see from the data structure
that the default values are kept in the function object:

Python 3.4.2 (default, Feb  4 2015, 20:08:25)
[GCC 4.2.1 (Apple Inc. build 5664)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> y = "spam"
>>> def f(x = y): pass
...
>>> f.__defaults__
('spam',)

I suppose you could argue that f.__defaults__
could be a computed property that's looking inside
f.__code__ somewhere, but that would be stretching
credibility.

--
Greg
--
https://mail.python.org/mailman/listinfo/python-list

Re: Is this unpythonic?

[Gregory Ewing]

Frank Millman wrote:

There are two operations I might perform on the dictionary -

1. iterate over the keys and retrieve the values

2: use 'in' to test if a given string exists as a key

Both of these operations will work on a tuple and give the desired result, 
so it is a very valid workaround.


Although if I were reviewing a function like that,
it would probably make me pause for a moment or two
to consider why a tuple was being used as a value
for a parameter that is ostensibly supposed to be
a dict, and convince myself that it was okay.

The absolutely clearest way to write it would
probably be

   def f(things = None):
  "things is a mapping of stuff to be operated on"
  if things:
 for key in things:
value = things[key]
...

A default value of None is a well-established idiom
for "this parameter is optional", and "if x:" is
idiomatic for "if I've been given an x", so writing it
that way has the best chance of passing the "pretty much
what you expect" test for good code. :-)

--
Greg
--
https://mail.python.org/mailman/listinfo/python-list

Re: calling base class method fetches no results

[david jhon]

Hi, I am sorry for sending in five attachments, I cloned the code from here
: Let me explain it here:

Routing Base class defined in DCRouting.py:

import logging
from copy import copy

class Routing(object):
'''Base class for data center network routing.

Routing engines must implement the get_route() method.
'''

def __init__(self, topo):
'''Create Routing object.

@param topo Topo object from Net parent
'''
self.topo = topo

def get_route(self, src, dst, hash_):
'''Return flow path.

@param src source host
@param dst destination host
@param hash_ hash value

@return flow_path list of DPIDs to traverse (including hosts)
'''
raise NotImplementedError

def routes(self, src, dst):
''' Return list of paths

Only works for Fat-Tree topology

@ param src source host
@ param dst destination host

@ return list of DPIDs (including inputs)
'''

complete_paths = [] # List of complete dpid routes

src_paths = { src : [[src]] }
dst_paths = { dst : [[dst]] }

dst_layer = self.topo.layer(dst)
src_layer = self.topo.layer(src)

lower_layer = src_layer
if dst_layer > src_layer:
lower_layer = dst_layer


for front_layer in range(lower_layer-1, -1, -1):
if src_layer > front_layer:
# expand src frontier
new_src_paths = {}
for node in sorted(src_paths):
path_list = src_paths[node]
for path in path_list:
last_node = path[-1]
for frontier_node in
self.topo.upper_nodes(last_node):
new_src_paths[frontier_node] = [path +
[frontier_node]]

if frontier_node in dst_paths:
dst_path_list = dst_paths[frontier_node]
for dst_path in dst_path_list:
dst_path_copy = copy ( dst_path )
dst_path_copy.reverse()
complete_paths.append( path +
dst_path_copy)
src_paths = new_src_paths

if dst_layer > front_layer:
# expand dst frontier
new_dst_paths = {}
for node in sorted(dst_paths):
path_list = dst_paths[node]
for path in path_list:
last_node = path[-1]
for frontier_node in
self.topo.upper_nodes(last_node):
new_dst_paths[frontier_node] = [ path +
[frontier_node]]

if frontier_node in src_paths:
src_path_list = src_paths[frontier_node]
dst_path_copy = copy( path )
dst_path_copy.reverse()
for src_path in src_path_list:
complete_paths.append( src_path +
dst_path_copy)

dst_paths = new_dst_paths

if complete_paths:
return complete_paths
class HashedRouting(Routing):
''' Hashed routing '''

def __init__(self, topo):
self.topo = topo

def get_route(self, src, dst, hash_):
''' Return flow path. '''

if src == dst:
return [src]

paths = self.routes(src,dst)
if paths:
#print 'hash_:', hash_
choice = hash_ % len(paths)
#print 'choice:', choice
path = sorted(paths)[choice]
#print 'path:', path
return path

>
Instantiated in util.py:

from DCTopo import FatTreeTopo
from mininet.util import makeNumeric
from DCRouting import HashedRouting, Routing

TOPOS = {'ft': FatTreeTopo}
ROUTING = {'ECMP' : HashedRouting}


def buildTopo(topo):
topo_name, topo_param = topo.split( ',' )
return TOPOS[topo_name](makeNumeric(topo_param))


def getRouting(routing, topo):
return ROUTING[routing](topo)

> utilized in HController. py:
A Piece of code which works with self.r.routes() method:
Following list of methods are defined in HController.py

def _ecmp_hash(self, packet):
''' Return an ECMP-style 5-tuple hash for TCP/IP packets, otherwise
0.
RFC2992 '''
hash_input = [0] * 5
if isinstance(packet.next, ipv4):
ip = packet.next
hash_input[0] = ip.srcip.toUnsigned()
hash_input[1] = ip.dstip.toUnsigned()
hash_input[2] = ip.protocol
if isinstance(ip.next, tcp) or isinstance(ip.next, udp):
l4 = ip.next
hash_input[3] = l4.srcport
hash_input[4] = l4.dstport
return crc32(pack('LLHHH', *hash_in

Re: calling base class method fetches no results

[Mark Lawrence]

On 09/05/2015 08:59, david jhon wrote:

Hi, I am sorry for sending in five attachments, I cloned the code from here
: Let me explain it here:



[nearly 300 lines of code snipped]


I am really sorry for any inconvenience caused. I, ve tried to make it a
bit clear here. I am not even able to debug the code by setting a python
debugging point pdb. I need help from you guys. Thanks a lot again for your
time and help.

Best Regards,
David


You clearly didn't bother to read this http://sscce.org/ about how to 
put a question so have another go.  Three strikes and you're out.


And please don't top post here, it's extremely annoying when trying to 
follow long threads.


--
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence

--
https://mail.python.org/mailman/listinfo/python-list

Re: Useful module to be written by a newbie

[Cecil Westerhof]

Op Saturday 9 May 2015 08:10 CEST schreef Michael Welle:

> Cecil Westerhof  writes:
>
>> Op Wednesday 29 Apr 2015 21:03 CEST schreef Peter Otten:
>>
> Realistically a Python coder with a little experience will have
> a glance at your code and run away.

 Oops, that is not nice to hear. :'-( 
>>>
>>> Sorry, I did not mean to discourage you or insult you, I just
>>> wanted to make it clear that your code is not there yet.
>>
>> You did not. Of-course it is not nice to hear, but if it is true,
>> it is very useful. If there is a lot to be desired, then it is good
>> when someone point this out.
>>
>>
 But can you enlighten me? Then I can learn from it.
> learning a new language looks like an easy job, in most cases. All
> the language's keywords and stuff, you can shuffle that into your
> head in a weekend or so. But what it makes it a hard task is all the
> idioms. It takes a long time to learn them. I like your approach of
> hacking random algorithms, like happynumbers and friends, (everyone
> does it to learn a new language I think) and show them for criticism
> (not everyone does that).

Well in my experience the fastest way to learn something is let people
‘burn you down’. Of-course you need to be able to take it. Also
important: “C'est le ton qui fait la musique”. But no problems here.

-- 
Cecil Westerhof
Senior Software Engineer
LinkedIn: http://www.linkedin.com/in/cecilwesterhof
-- 
https://mail.python.org/mailman/listinfo/python-list

Seralization

[Cecil Westerhof]

To make serialization a bit easier I made a few functions to get, save
and convert between the different types. As I see it pickle and json
are probably used the most. I also have a get and save for marshal.
But no conversion to marshal, because in principle you should not use
it, so a conversion to it is not useful. I did define conversion from.

Are there other seralizations that is handy to take care of?

I understood that because the differences between platforms it is
better to do the file operatins in binary mode. Is that true?


The code:
def get_json(json_file):
with open(json_file, 'rb') as in_f:
return json.load(in_f)

def get_marshal(marshal_file):
with open(marshal_file, 'rb') as in_f:
return marshal.load(in_f)

def get_pickle(pickle_file):
with open(pickle_file, 'rb') as in_f:
return pickle.load(in_f)

def save_json(data, json_file):
with open(json_file, 'wb') as out_f:
json.dump(data, out_f)

def save_marshal(data, marshal_file):
with open(marshal_file, 'wb') as out_f:
marshal.dump(data, out_f)

def save_pickle(data, pickle_file):
with open(pickle_file, 'wb') as out_f:
pickle.dump(data, out_f)

def marshal_to_pickle(marshal_file, pickle_file):
data_in = get_marshal(marshal_file)
save_pickle(data_in, pickle_file)
data_out = get_pickle(pickle_file)
if data_in != data_out:
raise SerializationError('Serialization from {0} to {1} not 
succesfull'.
 format(marshal_file, pickle_file))

def marshal_to_json(marshal_file, json_file):
data_in = get_marshal(marshal_file)
save_json(data_in, json_file)
data_out = get_json(json_file)
if data_in != data_out:
raise SerializationError('Serialization from {0} to {1} not 
succesfull'.
 format(marshal_file, json_file))

def pickle_to_json(pickle_file, json_file):
data_in = get_pickle(pickle_file)
save_json(data_in, json_file)
data_out = get_json(json_file)
if data_in != data_out:
raise SerializationError('Serialization from {0} to {1} not 
succesfull'.
 format(pickle_file, json_file))

def json_to_pickle(json_file, pickle_file):
data_in = get_json(json_file)
save_pickle(data_in, pickle_file)
data_out = get_pickle(pickle_file)
if data_in == data_out:
raise SerializationError('Serialization from {0} to {1} not 
succesfull'.
 format(json_file, pickle_file))

-- 
Cecil Westerhof
Senior Software Engineer
LinkedIn: http://www.linkedin.com/in/cecilwesterhof
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Is this unpythonic?

[Frank Millman]

"Gregory Ewing"  wrote in message 
news:cr5sc6fgfm...@mid.individual.net...
> Frank Millman wrote:
>
> The absolutely clearest way to write it would
> probably be
>
>def f(things = None):
>   "things is a mapping of stuff to be operated on"
>   if things:
>  for key in things:
> value = things[key]
> ...
>
> A default value of None is a well-established idiom
> for "this parameter is optional", and "if x:" is
> idiomatic for "if I've been given an x", so writing it
> that way has the best chance of passing the "pretty much
> what you expect" test for good code. :-)
>

Thanks, Greg, that makes a lot of sense.

My original method of using 'z=[]' worked, but would cause a reviewer to 
stop and think about it for a moment.

Changing it to 'z=()' would have pretty much the same effect.

Using 'z=None' would cause the least hesitation, and is therefore I think 
the most pythonic.

It does require an additional test every time the function is called, but 
the effect of that in my application is virtually zero.

If the overhead did become an issue, Dave's suggestion of 'z=EMPTY_LIST' 
would be a good solution.

Frank



-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Seralization

[Chris Angelico]

On Sat, May 9, 2015 at 6:37 PM, Cecil Westerhof  wrote:
> The code:
> def get_json(json_file):
> with open(json_file, 'rb') as in_f:
> return json.load(in_f)
>
> def get_marshal(marshal_file):
> with open(marshal_file, 'rb') as in_f:
> return marshal.load(in_f)
>
> def get_pickle(pickle_file):
> with open(pickle_file, 'rb') as in_f:
> return pickle.load(in_f)

def get_any(format, filename):
with open(filename, 'rb') as in_f:
return format.load(in_f)

> def save_json(data, json_file):
> with open(json_file, 'wb') as out_f:
> json.dump(data, out_f)
>
> def save_marshal(data, marshal_file):
> with open(marshal_file, 'wb') as out_f:
> marshal.dump(data, out_f)
>
> def save_pickle(data, pickle_file):
> with open(pickle_file, 'wb') as out_f:
> pickle.dump(data, out_f)

def save_any(format, data, filename):
with open(filename,'wb') as out_f:
format.dump(data, out_f)

> def marshal_to_pickle(marshal_file, pickle_file):
> data_in = get_marshal(marshal_file)
> save_pickle(data_in, pickle_file)
> data_out = get_pickle(pickle_file)
> if data_in != data_out:
> raise SerializationError('Serialization from {0} to {1} not 
> succesfull'.
>  format(marshal_file, pickle_file))
>
> def marshal_to_json(marshal_file, json_file):
> data_in = get_marshal(marshal_file)
> save_json(data_in, json_file)
> data_out = get_json(json_file)
> if data_in != data_out:
> raise SerializationError('Serialization from {0} to {1} not 
> succesfull'.
>  format(marshal_file, json_file))
>
> def pickle_to_json(pickle_file, json_file):
> data_in = get_pickle(pickle_file)
> save_json(data_in, json_file)
> data_out = get_json(json_file)
> if data_in != data_out:
> raise SerializationError('Serialization from {0} to {1} not 
> succesfull'.
>  format(pickle_file, json_file))

def any_to_any(fmt1, fmt2, fn1, fn2):
data_in = get_any(fmt1, fn1)
save_any(fmt2, data_in, fn2)
data_out = get_any(fmt2, fn2)
if data_in != data_out:
raise SerializationError('Serialization from {0} to {1} not successful'.
 format(fn1, fn2))

formats = [json, pickle, marshal]
for fmt1 in formats:
for fmt2 in formats:
globals()["%s_to_%s" % (fmt1.__name__, fmt2.__name__)] = \
functools.partial(any_to_any, fmt1, fmt2)

> def json_to_pickle(json_file, pickle_file):
> data_in = get_json(json_file)
> save_pickle(data_in, pickle_file)
> data_out = get_pickle(pickle_file)
> if data_in == data_out:
> raise SerializationError('Serialization from {0} to {1} not 
> succesfull'.
>  format(json_file, pickle_file))

def json_to_pickle(json_file, pickle_file):
try:
any_to_any(json, pickle, json_file, pickle_file)
except SerializationError:
pass
else:
raise SerializationError('Serialization from {0} to {1} not successful'.
format(json_file, pickle_file))

There. Much simpler. And maybe you can see that the last one actually
shouldn't exist :)

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Jython from bathc file?

[Todd Vargo]

On 5/8/2015 7:36 PM, vjp2...@at.biostrategist.dot.dot.com wrote:

How do I do this in a .bat file?
Do I include the Jython or pipe it?

% CLASSPATH=$CLASSPATH:$RDBASE/Code/JavaWrappers/gmwrapper/org.RDKit.jar; jython
  -Djava.library.path=$RDBASE/Code/JavaWrappers/gmwrapper
Jython 2.2.1 on java1.6.0_20
Type "copyright", "credits" or "license" for more information.

from org.RDKit import *
from java import lang
lang.System.loadLibrary('GraphMolWrap')
m = RWMol.MolFromSmiles('c1c1')
m.getNumAtoms()




This does not do that but for those who don't know Jython it can help.

@echo off
set "x=thequickbrownfoxjumpsoverthelazydog"
set "x1=%x:~11,1%%x:~1,1%%x:~29,1%%x:~0,1%"
set "x2= %x:~32,2%%x:~2,1%%x:~20,1%"
set "x3= %x:~5,1%%x:~0,1% %x:~32,2%"
echo %x1%%x2%%x3%?
pause>nul

--
Todd Vargo
(Post questions to group only. Remove "z" to email personal messages)
--
https://mail.python.org/mailman/listinfo/python-list

Re: Jython from bathc file?

[Steven D'Aprano]

On Sat, 9 May 2015 09:36 am, vjp2...@at.biostrategist.dot.dot.com wrote:

> How do I do this in a .bat file?
> Do I include the Jython or pipe it?
> 
> % CLASSPATH=$CLASSPATH:$RDBASE/Code/JavaWrappers/gmwrapper/org.RDKit.jar;
> jython
>  -Djava.library.path=$RDBASE/Code/JavaWrappers/gmwrapper
> Jython 2.2.1 on java1.6.0_20
> Type "copyright", "credits" or "license" for more information.
 from org.RDKit import *
 from java import lang
 lang.System.loadLibrary('GraphMolWrap')
 m = RWMol.MolFromSmiles('c1c1')
 m.getNumAtoms()


I'm not sure I fully understand your question, but perhaps this will help.

I haven't used Windows for anything like this for decades, but I would
expect something like this:


(1) Create a file containing your Python code and call it "myscript.py" (or
any other meaningful name):


from org.RDKit import *
from java import lang
lang.System.loadLibrary('GraphMolWrap')
m = RWMol.MolFromSmiles('c1c1')
m.getNumAtoms()


(2) Create a .bat file which calls Jython. I don't remember .bat syntax, but
perhaps it is something like this?


CLASSPATH=$CLASSPATH:$RDBASE/Code/JavaWrappers/gmwrapper/org.RDKit.jar;
jython -Djava.library.path=$RDBASE/Code/JavaWrappers/gmwrapper myscript.py


And that's it.

Does that work? Any Windows users like to comment?



-- 
Steven

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: SyntaxError: (unicode error) 'unicodeescape' codec can't decode bytes in position 2-3: truncated \UXXXXXXXX escape

[zljubisicmob]

Steven,

please do look at the code bellow:

# C:\Users\zoran\PycharmProjects\mm_align\hrt3.cfg contents
# [Dir]
# ROOTDIR = C:\Users\zoran\hrt


import os
import shutil
import configparser
import requests
import re

Config = configparser.ConfigParser()
Config.optionxform = str # preserve case in ini file
cfg_file = os.path.join('C:\\Users\\zoran\\PycharmProjects\\mm_align\\hrt3.cfg' 
)
Config.read(cfg_file)



ROOTDIR = Config.get('Dir', 'ROOTDIR')

print(ROOTDIR)

html = 
requests.get("http://radio.hrt.hr/prvi-program/arhiva/ujutro-prvi-poligraf-politicki-grafikon/118/";).text

art_html = re.search('(.+?)', html, 
re.DOTALL).group(1)
for p_tag in re.finditer(r'(.*?)', art_html, re.DOTALL):
if '' not in p_tag.group(1):
title = p_tag.group(1)

title = title[:232]
title = title.replace(" ", "_").replace("/", "_").replace("!", 
"_").replace("?", "_")\
.replace('"', "_").replace(':', "_").replace(',', 
"_").replace('"', '')\
.replace('\n', '_').replace(''', '')

print(title)

src_file = os.path.join(ROOTDIR, 'src_' + title + '.txt')
dst_file = os.path.join(ROOTDIR, 'des_' + title + '.txt')

print(len(src_file), src_file)
print(len(dst_file), dst_file)

with open(src_file, mode='w', encoding='utf-8') as s_file:
s_file.write('test')


shutil.move(src_file, dst_file)

It works, but if you change title = title[:232] to title = title[:233], you 
will get "FileNotFoundError: [Errno 2] No such file or directory".
As you can see ROOTDIR contains \U.

Regards.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: calling base class method fetches no results

[Dave Angel]

On 05/09/2015 03:59 AM, david jhon wrote:

Hi, I am sorry for sending in five attachments, I cloned the code from here
: Let me explain it here:



Please don't top-post.  Your earlier problem description, which I could 
make no sense of, is now located after your later "clarification".


Thanks for eliminating the attachments.  Many cannot see them.  And for 
extracting only part of the code into the message.  It's still too much 
for me, but others may manage it okay.  To me, it seems likely that most 
of that code will not have any part in the problem you're describing. 
And in some places you have code that's missing its context.


Now, eliminate the pieces of code that are irrelevant to your question, 
and state the problem in terms that make sense.  Somebody is 
instantiating an object.  Exactly which class is being used for that? 
Somebody else is calling a particular method (be specific, rather than 
just saying "some method"), and it's giving the wrong results.  And 
apparently, those wrong results depend on which source file something 
happens in.




Routing Base class defined in DCRouting.py:

import logging
from copy import copy

class Routing(object):
 '''Base class for data center network routing.

 Routing engines must implement the get_route() method.
 '''

 def __init__(self, topo):
 '''Create Routing object.

 @param topo Topo object from Net parent
 '''
 self.topo = topo

 def get_route(self, src, dst, hash_):
 '''Return flow path.

 @param src source host
 @param dst destination host
 @param hash_ hash value

 @return flow_path list of DPIDs to traverse (including hosts)
 '''
 raise NotImplementedError

 def routes(self, src, dst):
 ''' Return list of paths

 Only works for Fat-Tree topology

 @ param src source host
 @ param dst destination host

 @ return list of DPIDs (including inputs)
 '''

 complete_paths = [] # List of complete dpid routes

 src_paths = { src : [[src]] }
 dst_paths = { dst : [[dst]] }

 dst_layer = self.topo.layer(dst)
 src_layer = self.topo.layer(src)

 lower_layer = src_layer
 if dst_layer > src_layer:
 lower_layer = dst_layer


 for front_layer in range(lower_layer-1, -1, -1):
 if src_layer > front_layer:
 # expand src frontier
 new_src_paths = {}
 for node in sorted(src_paths):
 path_list = src_paths[node]
 for path in path_list:
 last_node = path[-1]
 for frontier_node in
self.topo.upper_nodes(last_node):
 new_src_paths[frontier_node] = [path +
[frontier_node]]

 if frontier_node in dst_paths:
 dst_path_list = dst_paths[frontier_node]
 for dst_path in dst_path_list:
 dst_path_copy = copy ( dst_path )
 dst_path_copy.reverse()
 complete_paths.append( path +
dst_path_copy)
 src_paths = new_src_paths

 if dst_layer > front_layer:
 # expand dst frontier
 new_dst_paths = {}
 for node in sorted(dst_paths):
 path_list = dst_paths[node]
 for path in path_list:
 last_node = path[-1]
 for frontier_node in
self.topo.upper_nodes(last_node):
 new_dst_paths[frontier_node] = [ path +
[frontier_node]]

 if frontier_node in src_paths:
 src_path_list = src_paths[frontier_node]
 dst_path_copy = copy( path )
 dst_path_copy.reverse()
 for src_path in src_path_list:
 complete_paths.append( src_path +
dst_path_copy)

 dst_paths = new_dst_paths

 if complete_paths:
 return complete_paths
class HashedRouting(Routing):
 ''' Hashed routing '''

 def __init__(self, topo):
 self.topo = topo

 def get_route(self, src, dst, hash_):
 ''' Return flow path. '''

 if src == dst:
 return [src]

 paths = self.routes(src,dst)
 if paths:
 #print 'hash_:', hash_
 choice = hash_ % len(paths)
 #print 'choice:', choice
 path = sorted(paths)[choice]
 #print 'path:', path
 return path

>
Instantiated in util.py:

from DCTopo import FatTreeTopo
from mininet.util import makeNumeric
from DCRouting import H

Re: Seralization

[Cecil Westerhof]

Op Saturday 9 May 2015 11:16 CEST schreef Chris Angelico:

> On Sat, May 9, 2015 at 6:37 PM, Cecil Westerhof  wrote:
>> The code:
>> def get_json(json_file):
>> with open(json_file, 'rb') as in_f:
>> return json.load(in_f)
>>
>> def get_marshal(marshal_file):
>> with open(marshal_file, 'rb') as in_f:
>> return marshal.load(in_f)
>>
>> def get_pickle(pickle_file):
>> with open(pickle_file, 'rb') as in_f:
>> return pickle.load(in_f)
>
> def get_any(format, filename):
> with open(filename, 'rb') as in_f:
> return format.load(in_f)

I was thinking about something like that and then let the other three
call this one. I think that:
data = get_json(json_file)
is nicer to do as:
data = get_any(json, json_file)


> def any_to_any(fmt1, fmt2, fn1, fn2): data_in = get_any(fmt1, fn1)
> save_any(fmt2, data_in, fn2) data_out = get_any(fmt2, fn2) if
> data_in != data_out: raise SerializationError('Serialization from
> {0} to {1} not successful'. format(fn1, fn2))
>
> formats = [json, pickle, marshal]
> for fmt1 in formats:
> for fmt2 in formats:
> globals()["%s_to_%s" % (fmt1.__name__, fmt2.__name__)] = \
> functools.partial(any_to_any, fmt1, fmt2)
>
>> def json_to_pickle(json_file, pickle_file): data_in =
>> get_json(json_file) save_pickle(data_in, pickle_file) data_out =
>> get_pickle(pickle_file) if data_in == data_out: raise
>> SerializationError('Serialization from {0} to {1} not succesfull'.
>> format(json_file, pickle_file))
>
> def json_to_pickle(json_file, pickle_file): try: any_to_any(json,
> pickle, json_file, pickle_file) except SerializationError: pass
> else: raise SerializationError('Serialization from {0} to {1} not
> successful'. format(json_file, pickle_file))

Was thinking about that also. Only should there be no conversion to
marshal I think.

-- 
Cecil Westerhof
Senior Software Engineer
LinkedIn: http://www.linkedin.com/in/cecilwesterhof
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: calling base class method fetches no results

[david jhon]

I am really sorry for any inconvenience caused, I was trying to fix this
bug from last 2 days so I had to post it here. It now has been resolved.
Thanks a lot for your time. I'll be careful again. Have a great weekend!

On Sat, May 9, 2015 at 4:36 PM, Dave Angel  wrote:

> On 05/09/2015 03:59 AM, david jhon wrote:
>
>> Hi, I am sorry for sending in five attachments, I cloned the code from
>> here
>> : Let me explain it here:
>>
>>
> Please don't top-post.  Your earlier problem description, which I could
> make no sense of, is now located after your later "clarification".
>
> Thanks for eliminating the attachments.  Many cannot see them.  And for
> extracting only part of the code into the message.  It's still too much for
> me, but others may manage it okay.  To me, it seems likely that most of
> that code will not have any part in the problem you're describing. And in
> some places you have code that's missing its context.
>
> Now, eliminate the pieces of code that are irrelevant to your question,
> and state the problem in terms that make sense.  Somebody is instantiating
> an object.  Exactly which class is being used for that? Somebody else is
> calling a particular method (be specific, rather than just saying "some
> method"), and it's giving the wrong results.  And apparently, those wrong
> results depend on which source file something happens in.
>
>
>
>  Routing Base class defined in DCRouting.py:
>>
>> import logging
>> from copy import copy
>>
>> class Routing(object):
>>  '''Base class for data center network routing.
>>
>>  Routing engines must implement the get_route() method.
>>  '''
>>
>>  def __init__(self, topo):
>>  '''Create Routing object.
>>
>>  @param topo Topo object from Net parent
>>  '''
>>  self.topo = topo
>>
>>  def get_route(self, src, dst, hash_):
>>  '''Return flow path.
>>
>>  @param src source host
>>  @param dst destination host
>>  @param hash_ hash value
>>
>>  @return flow_path list of DPIDs to traverse (including hosts)
>>  '''
>>  raise NotImplementedError
>>
>>  def routes(self, src, dst):
>>  ''' Return list of paths
>>
>>  Only works for Fat-Tree topology
>>
>>  @ param src source host
>>  @ param dst destination host
>>
>>  @ return list of DPIDs (including inputs)
>>  '''
>>
>>  complete_paths = [] # List of complete dpid routes
>>
>>  src_paths = { src : [[src]] }
>>  dst_paths = { dst : [[dst]] }
>>
>>  dst_layer = self.topo.layer(dst)
>>  src_layer = self.topo.layer(src)
>>
>>  lower_layer = src_layer
>>  if dst_layer > src_layer:
>>  lower_layer = dst_layer
>>
>>
>>  for front_layer in range(lower_layer-1, -1, -1):
>>  if src_layer > front_layer:
>>  # expand src frontier
>>  new_src_paths = {}
>>  for node in sorted(src_paths):
>>  path_list = src_paths[node]
>>  for path in path_list:
>>  last_node = path[-1]
>>  for frontier_node in
>> self.topo.upper_nodes(last_node):
>>  new_src_paths[frontier_node] = [path +
>> [frontier_node]]
>>
>>  if frontier_node in dst_paths:
>>  dst_path_list = dst_paths[frontier_node]
>>  for dst_path in dst_path_list:
>>  dst_path_copy = copy ( dst_path )
>>  dst_path_copy.reverse()
>>  complete_paths.append( path +
>> dst_path_copy)
>>  src_paths = new_src_paths
>>
>>  if dst_layer > front_layer:
>>  # expand dst frontier
>>  new_dst_paths = {}
>>  for node in sorted(dst_paths):
>>  path_list = dst_paths[node]
>>  for path in path_list:
>>  last_node = path[-1]
>>  for frontier_node in
>> self.topo.upper_nodes(last_node):
>>  new_dst_paths[frontier_node] = [ path +
>> [frontier_node]]
>>
>>  if frontier_node in src_paths:
>>  src_path_list = src_paths[frontier_node]
>>  dst_path_copy = copy( path )
>>  dst_path_copy.reverse()
>>  for src_path in src_path_list:
>>  complete_paths.append( src_path +
>> dst_path_copy)
>>
>>  dst_paths = new_dst_paths
>>
>>  if complete_paths:
>>  return complete_paths
>> class HashedRouting(Routing):
>>  ''' Hashed routing '''
>>
>>  

Re: SyntaxError: (unicode error) 'unicodeescape' codec can't decode bytes in position 2-3: truncated \UXXXXXXXX escape

[Dave Angel]

On 05/09/2015 06:31 AM, zljubisic...@gmail.com wrote:


title = title[:232]
title = title.replace(" ", "_").replace("/", "_").replace("!", "_").replace("?", 
"_")\
 .replace('"', "_").replace(':', "_").replace(',', 
"_").replace('"', '')\
 .replace('\n', '_').replace(''', '')

print(title)

src_file = os.path.join(ROOTDIR, 'src_' + title + '.txt')
dst_file = os.path.join(ROOTDIR, 'des_' + title + '.txt')

print(len(src_file), src_file)
print(len(dst_file), dst_file)

with open(src_file, mode='w', encoding='utf-8') as s_file:
 s_file.write('test')


shutil.move(src_file, dst_file)

It works, but if you change title = title[:232] to title = title[:233], you will get 
"FileNotFoundError: [Errno 2] No such file or directory".
As you can see ROOTDIR contains \U.


No, we can't see what ROOTDIR is, since you read it from the config 
file.  And you don't show us the results of those prints.  You don't 
even show us the full exception, or even the line it fails on.


I doubt that the problem is in the ROODIR value, but of course nothing 
in your program bothers to check that that directory exists.  I expect 
you either have too many characters total, or the 232th character is a 
strange one.  Or perhaps title has a backslash in it (you took care of 
forward slash).


While we're at it, if you do have an OS limitation on size, your code is 
truncating at the wrong point.  You need to truncate the title based on 
the total size of src_file and dst_file, and since the code cannot know 
the size of ROOTDIR, you need to include that in your figuring.





--
DaveA
--
https://mail.python.org/mailman/listinfo/python-list

Re: Seralization

[Cecil Westerhof]

Op Saturday 9 May 2015 10:37 CEST schreef Cecil Westerhof:

> To make serialization a bit easier I made a few functions to get,
> save and convert between the different types. As I see it pickle and
> json are probably used the most. I also have a get and save for
> marshal. But no conversion to marshal, because in principle you
> should not use it, so a conversion to it is not useful. I did define
> conversion from.

I used the example of Chris to make it a lot cleaner. I like DRY, but
did not know it was that easy in Python.

The code is now:
def convert_serialization(format_in, format_out, filename_in, filename_out):
data_in = get_serialization(format_in, filename_in)
save_serialization(format_out, data_in, filename_out)
data_out = get_serialization(format_out, filename_out)
if data_in != data_out:
raise SerializationError('Serialization from {0} to {1} not 
successful'.
 format(filename_in, filename_out))

def get_serialization(format, filename):
with open(filename, 'rb') as in_f:
return format.load(in_f)

def save_serialization(format, data, filename):
with open(filename, 'wb') as out_f:
format.dump(data, out_f)

_all_serialization_formats   = [json, marshal, pickle]
_convert_to_serialization_formats= [json, pickle]

for format_in in _all_serialization_formats:
globals()[
'save_%s' % (format_in.__name__)
] = functools.partial(save_serialization, format_in)
globals()[
'get_%s' % (format_in.__name__)
] = functools.partial(get_serialization, format_in)
for format_out in _convert_to_serialization_formats:
if format_in == format_out:
continue
globals()[
'%s_to_%s' % (format_in.__name__, format_out.__name__)
] = functools.partial(convert_serialization, format_in, format_out)

del format_in, format_out

I keep the all_serialization_formats and
_convert_to_serialization_formats so you can peek which ones are
supported at the moment.


> Are there other seralizations that is handy to take care of?

That is now a cinch to implement: you only have to append the two
lists. ;-)

-- 
Cecil Westerhof
Senior Software Engineer
LinkedIn: http://www.linkedin.com/in/cecilwesterhof
-- 
https://mail.python.org/mailman/listinfo/python-list

numpy and cython

[pauld11718]

Unable to compile :

import numpy as np
cimport numpy as np
import math as m

DTYPE = np.float
ctypedef np.float_t DTYPE_t

def visc1(float t, float dcal):
cdef float h, tr, trinv, rhor
cdef float eta0, sumi, i, sumj, im1, jm1, eta
   
cdef np.ndarray vb = np.array([1.0, 0.940695, 0.578377, -0.202044], 
dtype = DTYPE)
   
cdef np.ndarray[DTYPE_t, ndim=2] va = np.array([[.4864192, .3509007, 
-.2847572, .07013759,.0164122, -.01163815,.0],
[-.2448372,1.315436, -1.037026, .4660127, -.02884911,-.008239587,.0],
[-.8702035, 1.297752, -1.287846, .2292075, .0,  .0, .0],
[.8716056, 1.353448,  .0,  -.4857462, .1607171,.0,-.003886659],
[-1.051126, .0,   .0,   .0,   .0,  .0,   .0],
[.3458395, .0,  -.02148229, .0, -.009603846, .004559914,.0]], 
dtype=DTYPE, ndim = 2)
   

h=55.2651e-06;
tr = t/643.89;
trinv=643.89/t;
rhor=dcal/0.358;

eta0 = h*(m.pow(tr,0.5))/(vb[0] + vb[1]/tr + vb[2]/(tr*tr) + vb[3]/(tr**3));
sumi=0.0
for i in range(6):
sumj=va[i,0]
for j in range(2,7):
jm1=j-1;
sumj=sumj+va[i,j]*((rhor-1.0)**jm1);
   
im1 = i-1
sumi = sumi+sumj*((trinv-1.0)**im1);
   
eta = eta0*m.exp(rhor*sumi)
return eta

Error :
Compiling visco.pyx because it changed.
Cythonizing visco.pyx
running build_ext
building 'visco' extension
gcc -pthread -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC 
-I/home/deepraj/miniconda3/envs/venv1/include/python3.4m -c visco.c -o 
build/temp.linux-x86_64-3.4/visco.o
In file included from /usr/include/numpy/ndarraytypes.h:1761:0,
 from /usr/include/numpy/ndarrayobject.h:17,
 from /usr/include/numpy/arrayobject.h:4,
 from visco.c:258:
/usr/include/numpy/npy_1_7_deprecated_api.h:15:2: warning: #warning "Using 
deprecated NumPy API, disable it by " "#defining NPY_NO_DEPRECATED_API 
NPY_1_7_API_VERSION" [-Wcpp]
 #warning "Using deprecated NumPy API, disable it by " \
  ^
gcc -pthread -shared build/temp.linux-x86_64-3.4/visco.o 
-L/home/abcd/miniconda3/envs/venv1/lib -lpython3.4m -o 
/media/abcd/Man_UTD/pythoncode/venv1/visco.cpython-34m.so
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: numpy and cython

[Chris Angelico]

On Sun, May 10, 2015 at 12:51 AM, pauld11718  wrote:
> Unable to compile :
> /usr/include/numpy/npy_1_7_deprecated_api.h:15:2: warning: #warning "Using 
> deprecated NumPy API, disable it by " "#defining NPY_NO_DEPRECATED_API 
> NPY_1_7_API_VERSION" [-Wcpp]
>  #warning "Using deprecated NumPy API, disable it by " \
>   ^
> gcc -pthread -shared build/temp.linux-x86_64-3.4/visco.o 
> -L/home/abcd/miniconda3/envs/venv1/lib -lpython3.4m -o 
> /media/abcd/Man_UTD/pythoncode/venv1/visco.cpython-34m.so

Looks like a warning to me. Are you sure the compilation isn't
working? Check to see if you have an output file. If you don't,
there's something else wrong, not just this.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: SyntaxError: (unicode error) 'unicodeescape' codec can't decode bytes in position 2-3: truncated \UXXXXXXXX escape

[Steven D'Aprano]

On Sat, 9 May 2015 08:31 pm, zljubisic...@gmail.com wrote:

> It works, but if you change title = title[:232] to title = title[:233],
> you will get "FileNotFoundError: [Errno 2] No such file or directory".


Which is a *completely different* error from 

SyntaxError: 'unicodeescape' codec can't decode bytes in position 2-3:
truncated \U escape


> As you can see ROOTDIR contains \U.

How can I possibly see that? Your code reads ROOTDIR from the config file,
which you don't show us.

I agree with you that Windows has limitations on the length of file names,
and that you get an error if you give a file name that cannot be found. The
point is that before you can get that far, you *first* have to fix the
SyntaxError. That's a completely different problem.

You can't fix the \U syntax error by truncating the total file length. But
you can fix that syntax error by changing your code so it reads the ROOTDIR
from a config file instead of a hard-coded string literal -- exactly like
we told you to do!

An essential skill when programming is to read and understand the error
messages. One of the most painful things to use is a programming language
that just says 

"An error occurred"

with no other explanation. Python gives you lots of detail to explain what
went wrong:

SyntaxError means you made an error in the syntax of the code and the
program cannot even run.

FileNotFoundError means that the program did run, it tried to open a file,
but the file doesn't exist.

They're a little bit different, don't you agree?



-- 
Steven

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: SyntaxError: (unicode error) 'unicodeescape' codec can't decode bytes in position 2-3: truncated \UXXXXXXXX escape

[Chris Angelico]

On Sun, May 10, 2015 at 1:13 AM, Steven D'Aprano
 wrote:
> FileNotFoundError means that the program did run, it tried to open a file,
> but the file doesn't exist.

Normally it does, at least. Sometimes it means that a *directory*
doesn't exist (for instance, you can get this when you try to create a
new file, which otherwise wouldn't make sense), and occasionally,
Windows will give you rather peculiar errors when weird things go
wrong, which may be what's going on here (maximum path length - though
that can be overridden by switching to a UNC-style path).

Steven's point still stands - very different from SyntaxError - but
unfortunately it's not always as simple as the name suggests. Thank
you oh so much, Windows.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: numpy and cython

[pauld11718]

NO the compilation isn't working...

The setup.py : 

from distutils.core import setup
from Cython.Build import cythonize

setup(
ext_modules = cythonize("visco.pyx")
)
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: numpy and cython

[pauld11718]

An interesting observation : 

Any simple program with just :

cimport numpy as np 

doesnot compile.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: numpy and cython

[Mark Lawrence]

On 09/05/2015 16:56, pauld11718 wrote:

NO the compilation isn't working...

The setup.py :

from distutils.core import setup
from Cython.Build import cythonize

setup(
 ext_modules = cythonize("visco.pyx")
)



If you cannot be bothered to supply any context so I've no idea what 
you're talking about, then I cannot be bothered to try and help.


--
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence

--
https://mail.python.org/mailman/listinfo/python-list

Re: functions, optional parameters

[Ian Kelly]

On Fri, May 8, 2015 at 9:50 AM, Michael Welle  wrote:
>
> Steven D'Aprano  writes:
>>
>> If your language uses late binding, it is very inconvenient to get early
>> binding when you want it. But if your language uses early binding, it is
>> very simple to get late binding when you want it: just put the code you
>> want to run inside the body of the function:
> And you have to do it all the time again and again. I can't provide hard
> numbers, but I think usually I want late binding.

You could perhaps write a decorator to evaluate your defaults at call
time. This one relies on inspect.signature, so it requires Python 3.3
or newer:

import inspect
from functools import wraps

def late_defaults(**defaults):
def decorator(f):
sig = inspect.signature(f)
@wraps(f)
def wrapped(*args, **kwargs):
bound_args = sig.bind_partial(*args, **kwargs)
for name, get_value in defaults.items():
if name not in bound_args.arguments:
bound_args.arguments[name] = get_value()
return f(*bound_args.args, **bound_args.kwargs)
return wrapped
return decorator

@late_defaults(b=lambda: x+1, c=lambda: y*2)
def f(a, b, c=None):
print(a, b, c)

x = 14
y = 37
f(10)
x = 30
y = 19
f(10)
f(10, 11)
f(10, 11, c=12)

Output:

10 15 74
10 31 38
10 11 38
10 11 12

For documentation purposes I suggest using default values of None in
the function spec to indicate that the arguments are optional, and
elaborating on the actual defaults in the docstring. Alternatively you
could put the lambdas in the the actual function spec and then just
tell the decorator which ones to apply if not supplied, but that would
result in less readable pydoc.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: numpy and cython

[Mark Lawrence]

On 09/05/2015 17:09, pauld11718 wrote:

An interesting observation :

Any simple program with just :

cimport numpy as np

doesnot compile.



Fascinating.  What has this got to do with the general election results 
from the UK?  Or is there some other context that you're not prepared to 
let us in on?


--
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence

--
https://mail.python.org/mailman/listinfo/python-list

Moving to Python 3.x

[Antranig Vartanian]

Hay,

I learned the basics of python using the book "Think Python" 
(http://www.greenteapress.com/thinkpython/) which was good (IMHO), and it 
teaches in Python 2.7. Now I'm trying to write my first python+gtk 
program.


anyways, my question will be, is it so necessary to move to python3.x 
ASAP? or Python2.7 will live for a while (2-3 years)?.


and what do you advice a newbie programmer to do after learning the 
basics?


Thanks all!

--
Antranig Vartanian
http://antranig.pingvinashen.am/
--
https://mail.python.org/mailman/listinfo/python-list

Re: Moving to Python 3.x

[Jason C. McDonald]

On 05/09/2015 11:30 AM, Antranig Vartanian wrote:

Hay,

I learned the basics of python using the book "Think Python"
(http://www.greenteapress.com/thinkpython/) which was good (IMHO), and
it teaches in Python 2.7. Now I'm trying to write my first python+gtk
program.

anyways, my question will be, is it so necessary to move to python3.x
ASAP? or Python2.7 will live for a while (2-3 years)?.

and what do you advice a newbie programmer to do after learning the basics?

Thanks all!

--
Antranig Vartanian
http://antranig.pingvinashen.am/


I would strongly recommend writing your code to run on both Py2 and Py3. 
NINJA-IDE (an open source Python IDE) will lint your code so it'll run 
in both.


--
Jason C. McDonald (CodeMouse92)
[CEO, Lead Dev @ MousePaw Games]
--
https://mail.python.org/mailman/listinfo/python-list

Re: Moving to Python 3.x

[Ian Kelly]

On Sat, May 9, 2015 at 12:30 PM, Antranig Vartanian
 wrote:
> Hay,
>
> I learned the basics of python using the book "Think Python"
> (http://www.greenteapress.com/thinkpython/) which was good (IMHO), and it
> teaches in Python 2.7. Now I'm trying to write my first python+gtk program.
>
> anyways, my question will be, is it so necessary to move to python3.x ASAP?
> or Python2.7 will live for a while (2-3 years)?.

Python 2.7 will continue to be maintained through 2020. If you don't
have any specific reason to use Python 2.7 (such as a library
dependency), then you should try to use 3.x for new projects. You'll
avoid the pain of needing to migrate later, and you'll be able to
start taking advantage of newer features right away.

> and what do you advice a newbie programmer to do after learning the basics?

Find an existing open source project that you'd like to contribute to.
It doesn't have to be anything major, but it will help you learn about
the Python ecosystem, and the opportunities to collaborate will help
you build your skills. It also looks good on a resume, if your plans
include being a professional Python programmer.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Moving to Python 3.x

[Terry Reedy]

On 5/9/2015 2:30 PM, Antranig Vartanian wrote:

Hay,

I learned the basics of python using the book "Think Python"
(http://www.greenteapress.com/thinkpython/) which was good (IMHO), and
it teaches in Python 2.7. Now I'm trying to write my first python+gtk
program.

anyways, my question will be, is it so necessary to move to python3.x
ASAP? or Python2.7 will live for a while (2-3 years)?.


1.5 is still in use, so that is not exactly the issue.

I and many here recommend starting with current 3.x unless there is a 
compelling reason otherwise:

1. learning from a 2.7 book;
2. employer requires 2.7
3. using 2.x only module.  If you are determined to use gtk and are 
using the PyGTK bindings https://pypi.python.org/pypi/PyGTK/2.24.0

then you have no choice.

If you write 2.7 code, make it as 3.x-like as sensible.  Others may post 
some links.


If you use unicode, you might be happier with 3.x or even have to use it.

--
Terry Jan Reedy

--
https://mail.python.org/mailman/listinfo/python-list

Re: Jython from bathc file?

[vjp2 . at]

Thanks.. I suspected it wasn't meant to be taken as in the file

THe one thing I'm not sure if Jython is suppsosedto keep running
after the initisl stuff is loaded in..


To put the question in purely DOS terms if you run a program can you pipe it
some commands and then keep it running to take the remaining commands from
the console?

- = -
 Vasos Panagiotopoulos, Columbia'81+, Reagan, Mozart, Pindus, BioStrategist
http://www.panix.com/~vjp2/vasos.htm
  ---{Nothing herein constitutes advice.  Everything fully disclaimed.}---
   [Homeland Security means private firearms not lazy obstructive guards]
 [Urb sprawl confounds terror] [Phooey on GUI: Windows for subprime Bimbos]




-- 
https://mail.python.org/mailman/listinfo/python-list

getting fieldnames from Dictreader before reading lines

[Vincent Davis]

I am reading a file with Dictreader and writing a new file. I want use the
fieldnames in the Dictwriter from the reader. See below How should I be
doing this?

See how I am using reader.fieldnames in the the Dictwriter. I get an error
(below)

with open(readfile, 'r', encoding='utf-8', errors='ignore', newline='') as
csvread:
reader = DictReader(csvread)
with open(writefile, 'w') as csvwrite:
writer = DictWriter(csvwrite, delimiter=',',
fieldnames=reader.fieldnames)
for line in reader:
pass

ValueErrorTraceback (most recent call
last) in ()> 1
reader.fieldnames()
/Users/vmd/anaconda/envs/py34/lib/python3.4/csv.py in fieldnames(self)
94 if self._fieldnames is None: 95
try:---> 96 self._fieldnames = next(self.reader)
97 except StopIteration: 98 pass
ValueError: I/O operation on closed file.



Thanks
Vincent
​ Davis​
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Jython from bathc file?

[Michael Torrie]

On 05/09/2015 03:04 PM, vjp2...@at.biostrategist.dot.dot.com wrote:
> Thanks.. I suspected it wasn't meant to be taken as in the file
> 
> THe one thing I'm not sure if Jython is suppsosedto keep running
> after the initisl stuff is loaded in..
> 
> 
> To put the question in purely DOS terms if you run a program can you pipe it
> some commands and then keep it running to take the remaining commands from
> the console?

No.  When the sending program is finished, it will close the pipe. This
is how it works on both Unix and Windows, if I'm not mistaken.  On Unix
you might be able to read from standard in until it's done, then connect
to a pseudo-tty and do interactive things.  This bypasses standard in
though (which was connected to the pipe).
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: getting fieldnames from Dictreader before reading lines

[Mark Lawrence]

On 09/05/2015 23:46, Vincent Davis wrote:

I am reading a file with Dictreader and writing a new file. I want use the
fieldnames in the Dictwriter from the reader. See below How should I be
doing this?

See how I am using reader.fieldnames in the the Dictwriter. I get an error
(below)

with open(readfile, 'r', encoding='utf-8', errors='ignore', newline='') as
csvread:
 reader = DictReader(csvread)
 with open(writefile, 'w') as csvwrite:
 writer = DictWriter(csvwrite, delimiter=',',
fieldnames=reader.fieldnames)
 for line in reader:
 pass

ValueErrorTraceback (most recent call
last) in ()> 1
reader.fieldnames()
/Users/vmd/anaconda/envs/py34/lib/python3.4/csv.py in fieldnames(self)
 94 if self._fieldnames is None: 95
try:---> 96 self._fieldnames = next(self.reader)
97 except StopIteration: 98 pass
ValueError: I/O operation on closed file.

Thanks
Vincent
​ Davis​



From https://docs.python.org/3/library/csv.html#module-csv

"class csv.DictReader(csvfile, fieldnames=None, restkey=None, 
restval=None, dialect='excel', *args, **kwds)
Create an object which operates like a regular reader but maps the 
information read into a dict whose keys are given by the optional 
fieldnames parameter. The fieldnames parameter is a sequence whose 
elements are associated with the fields of the input data in order. 
These elements become the keys of the resulting dictionary. If the 
fieldnames parameter is omitted, the values in the first row of the 
csvfile will be used as the fieldnames. If the row read has more fields 
than the fieldnames sequence, the remaining data is added as a sequence 
keyed by the value of restkey. If the row read has fewer fields than the 
fieldnames sequence, the remaining keys take the value of the optional 
restval parameter. Any other optional or keyword arguments are passed to 
the underlying reader instance."


--
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence

--
https://mail.python.org/mailman/listinfo/python-list

Re: getting fieldnames from Dictreader before reading lines

[Vincent Davis]

Not sure what I was doing wrong, it seems to work now.

Vincent Davis
720-301-3003

On Sat, May 9, 2015 at 4:46 PM, Vincent Davis 
wrote:

> I am reading a file with Dictreader and writing a new file. I want use the
> fieldnames in the Dictwriter from the reader. See below How should I be
> doing this?
>
> See how I am using reader.fieldnames in the the Dictwriter. I get an error
> (below)
>
> with open(readfile, 'r', encoding='utf-8', errors='ignore', newline='') as
> csvread:
> reader = DictReader(csvread)
> with open(writefile, 'w') as csvwrite:
> writer = DictWriter(csvwrite, delimiter=',',
> fieldnames=reader.fieldnames)
> for line in reader:
> pass
>
> ValueErrorTraceback (most recent call 
> last) in ()> 1 reader.fieldnames()
> /Users/vmd/anaconda/envs/py34/lib/python3.4/csv.py in fieldnames(self) 94 
> if self._fieldnames is None: 95 try:---> 96   
>   self._fieldnames = next(self.reader) 97 except 
> StopIteration: 98 pass
> ValueError: I/O operation on closed file.
>
>
>
> Thanks
> Vincent
> ​ Davis​
>
>
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Jython from bathc file?

[Dave Angel]

On 05/09/2015 05:04 PM, vjp2...@at.biostrategist.dot.dot.com wrote:

Thanks.. I suspected it wasn't meant to be taken as in the file

THe one thing I'm not sure if Jython is suppsosedto keep running
after the initisl stuff is loaded in..


To put the question in purely DOS terms if you run a program can you pipe it
some commands and then keep it running to take the remaining commands from
the console?



That's not a built-in feature of cmd.exe.  However, it wouldn't be hard 
to write a data source (funny.exe) that took data from a file, and then 
from stdin, sending both in progression to stdout.  Then you'd run the 
two programs as:


funny.exe infile.txt | newprog.exe


--
DaveA
--
https://mail.python.org/mailman/listinfo/python-list

Re: getting fieldnames from Dictreader before reading lines

[Dave Angel]

On 05/09/2015 07:01 PM, Vincent Davis wrote:

Not sure what I was doing wrong, it seems to work now.



I still see two significant things wrong:

1) you're top-posting, putting your response  BEFORE the stuff you're 
responding to.


2) both messages are in html, which thoroughly messed up parts of your 
error messages.




On Sat, May 9, 2015 at 4:46 PM, Vincent Davis 
wrote:


I am reading a file with Dictreader and writing a new file. I want use the
fieldnames in the Dictwriter from the reader. See below How should I be
doing this?



--
DaveA
--
https://mail.python.org/mailman/listinfo/python-list

Re: getting fieldnames from Dictreader before reading lines

[Vincent Davis]

On Sat, May 9, 2015 at 5:55 PM, Dave Angel  wrote:
>
> 1) you're top-posting, putting your response  BEFORE the stuff you're
responding to.


I responded to my own email, seemed ok to top post on myself saying it was
resolved.

>
> 2) both messages are in html, which thoroughly messed up parts of your
error messages.

I am posting from google mail (not google groups). Kindly let me know if
this email is also html.



Vincent Davis
720-301-3003
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Jython from bathc file?

[vjp2 . at]

Tee from gnuutils??



- = -
 Vasos Panagiotopoulos, Columbia'81+, Reagan, Mozart, Pindus, BioStrategist
http://www.panix.com/~vjp2/vasos.htm
  ---{Nothing herein constitutes advice.  Everything fully disclaimed.}---
   [Homeland Security means private firearms not lazy obstructive guards]
 [Urb sprawl confounds terror] [Phooey on GUI: Windows for subprime Bimbos]




-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Jython from bathc file?

[vjp2 . at]

I have to try this and see if there is ome kind of init file in jython/python
sorta like autoexec.bat. Ialso have no idea if the commands they provide are
all it takes to run the app or I have to stay in jython.. sorry, I'm thinking
at loud.. ok, thanks to all..



- = -
 Vasos Panagiotopoulos, Columbia'81+, Reagan, Mozart, Pindus, BioStrategist
http://www.panix.com/~vjp2/vasos.htm
  ---{Nothing herein constitutes advice.  Everything fully disclaimed.}---
   [Homeland Security means private firearms not lazy obstructive guards]
 [Urb sprawl confounds terror] [Phooey on GUI: Windows for subprime Bimbos]




-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Jython from bathc file?

[Rustom Mody]

On Sunday, May 10, 2015 at 7:55:22 AM UTC+5:30, 
vjp...@at.biostrategist.dot.dot.com wrote:
> I have to try this and see if there is ome kind of init file in jython/python
> sorta like autoexec.bat. Ialso have no idea if the commands they provide are
> all it takes to run the app or I have to stay in jython.. sorry, I'm thinking
> at loud.. ok, thanks to all..
> 

Maybe this?

http://www.jython.org/docs/tutorial/interpreter.html?highlight=pythonstartup#the-interactive-startup-file

PS People here tend to prefer avoidance of top-posting
http://en.wikipedia.org/wiki/Posting_style#Top-posting
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: functions, optional parameters

[Steven D'Aprano]

On Sat, 9 May 2015 01:50 am, Michael Welle wrote:

[...]
>> How about this definition:
>>
>> default = 23
>> def spam(eggs=default):
>> pass
>>
>> del default
>>
>> print spam()
>>
>>
>> Do you expect the function call to fail because `default` doesn't exist?
> 
> If I reference an object, that isn't available in the current context, I
> want to see it fail, yes.

Well, that's an interesting response. Of course I agree with you if the
reference to default is in the code being executed:

def spam():
value = default


that's quite normal rules for Python functions. 

Aside: note that *closures* behave differently, by design: a closure will
keep non-local values alive even if the parent function is deleted.

py> def outer():
... default = 23
... def closure():
... return default
... return closure
...
py> f = outer()
py> del outer
py> f()
23


But I don't agree with you about default parameters. Suppose we do this:

default = 23
eggs = default
# some time later
del default
print(eggs)

I trust that you agree that eggs shouldn't raise a NameError here just
because default no longer exists!

Why should that be any different just because the assignment is inside a
parameter list?

def spam(eggs=default):
...


One of the nice things about Python's current behaviour is that function
defaults don't behave any differently from any other name binding. Python
uses the same semantics for binding names wherever the name is without the
need for users to memorise a bunch of special rules.

Things which look similar should behave similarly.


>> My answers to those questions are all No.
>
> Different answers are possible as it seems ;).

Obviously :-)

And if Python used late binding, as some other languages do (Lisp, I think),
we would have a FAQ 

"Q: Why does my function run slowly/raise an exception when I use a default
value?"

"A: Because the default is re-evaluated every time you call the function,
not just once when you define it."


>> To me, it is not only expected,
>> but desirable that function defaults are set once, not every time the
>> function is called. This behaviour is called "early binding" of defaults.
>>
>> The opposite behaviour is called "late binding".
>>
>> If your language uses late binding, it is very inconvenient to get early
>> binding when you want it. But if your language uses early binding, it is
>> very simple to get late binding when you want it: just put the code you
>> want to run inside the body of the function:
>
> And you have to do it all the time again and again. I can't provide hard
> numbers, but I think usually I want late binding.

I'm pretty sure that you don't. You just think you do because you're
thinking of the subset of cases where you want to use a mutable default
like [], or perhaps delay looking up a global default until runtime, and
not thinking of all the times you use a default.

I predict that the majority of the time, late binding would just be a
pointless waste of time:

def process_string(thestr, start=0, end=None, slice=1, reverse=True):
pass

Why would you want 0, None, 1 and True to be re-evaluated every time?
Admittedly it will be fast, but not as fast as evaluating them once, then
grabbing a static default value when needed. (See below for timings.)

Whether you use early or late binding, Python still has to store the
default, then retrieve it at call-time. What happens next depends on the
binding model.

With early binding, Python has the value, and can just use it directly. With
late binding, it needs to store a delayed computation object, an executable
expression if you prefer. There are two obvious ways to implement such a
thunk in Python: a code object, or a function.

thunk = compile('0', '', 'eval')  # when the function is defined
value = eval(thunk)  # when the function is called

# or

thunk = lambda: 0
value = thunk()


Both of those are considerably slower than the current behaviour:

py> from timeit import Timer
py> static = Timer("x = 0")
py> thunk = Timer("x = eval(t)", setup="t = compile('0', '', 'eval')")
py> func = Timer("x = f()", setup="f = lambda: 0")
py> min(static.repeat(repeat=7))  # Best of seven trials.
0.04563648998737335
py> min(thunk.repeat(repeat=7))
1.2324241530150175
py> min(func.repeat(repeat=7))
0.20116623677313328


It would be nice to have syntax for late binding, but given that we don't,
and only have one or the other, using early binding is much more sensible.


This is the point where some people try to suggest some sort of complicated,
fragile, DWIM heuristic where the compiler tries to guess whether the user
actually wants the default to use early or late binding, based on what the
expression looks like. "0 is an immutable int, use early binding; [] is a
mutable list, use late binding." sort of thing. Such a thing might work
well for the obvious cases, but it would be a bugger to debug and
work-around for the non-obvious cases when it guesses wrong -- and it 

Re: functions, optional parameters

[Chris Angelico]

On Sun, May 10, 2015 at 12:45 PM, Steven D'Aprano
 wrote:
> This is the point where some people try to suggest some sort of complicated,
> fragile, DWIM heuristic where the compiler tries to guess whether the user
> actually wants the default to use early or late binding, based on what the
> expression looks like. "0 is an immutable int, use early binding; [] is a
> mutable list, use late binding." sort of thing. Such a thing might work
> well for the obvious cases, but it would be a bugger to debug and
> work-around for the non-obvious cases when it guesses wrong -- and it will.

What you could have is "late-binding semantics, optional early binding
as an optimization but only in cases where the result is
indistinguishable". That would allow common cases (int/bool/str/None
literals) to be optimized, since there's absolutely no way for them to
evaluate differently.

I personally don't think it'd be that good an idea, but it's a simple
enough rule that it wouldn't break anything. As far as anyone's code
is concerned, the rule is "late binding, always". In fact, that would
be the language definition; the rest is an optimization. (It's like
how "x.y()" technically first looks up attribute "y" on object x, then
calls the result; but it's perfectly reasonable for a Python
implementation to notice this extremely common case and do an
"optimized method call" that doesn't actually create a function
object.) The simpler the rule, the easier to grok, and therefore the
less chance of introducing bugs.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: functions, optional parameters

[Rustom Mody]

On Sunday, May 10, 2015 at 8:16:07 AM UTC+5:30, Steven D'Aprano wrote:
> I predict that the majority of the time, late binding would just be a
> pointless waste of time:
> 
> def process_string(thestr, start=0, end=None, slice=1, reverse=True):
> pass
> 
> Why would you want 0, None, 1 and True to be re-evaluated every time?
> Admittedly it will be fast, but not as fast as evaluating them once, then
> grabbing a static default value when needed. (See below for timings.)
> 

And what is the work involved in (re)computing 0, None, 1, True??

If I write (... arg=square_root_of_grahams_number())
I would expect to pay for it.
If I write trivial defaults, then I expect trivial payment.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: functions, optional parameters

[Steven D'Aprano]

On Sun, 10 May 2015 01:33 pm, Chris Angelico wrote:

> On Sun, May 10, 2015 at 12:45 PM, Steven D'Aprano
>  wrote:
>> This is the point where some people try to suggest some sort of
>> complicated, fragile, DWIM heuristic where the compiler tries to guess
>> whether the user actually wants the default to use early or late binding,
>> based on what the expression looks like. "0 is an immutable int, use
>> early binding; [] is a mutable list, use late binding." sort of thing.
>> Such a thing might work well for the obvious cases, but it would be a
>> bugger to debug and work-around for the non-obvious cases when it guesses
>> wrong -- and it will.
> 
> What you could have is "late-binding semantics, optional early binding
> as an optimization but only in cases where the result is
> indistinguishable". That would allow common cases (int/bool/str/None
> literals) to be optimized, since there's absolutely no way for them to
> evaluate differently.
> 
> I personally don't think it'd be that good an idea, but it's a simple
> enough rule that it wouldn't break anything.

It's a change in semantics, and it would break code that expects early
binding.


> As far as anyone's code 
> is concerned, the rule is "late binding, always".

Sure, other languages have made that choice. I think it is the wrong choice,
but if we went back to 1991 Guido could have made that same choice.


> In fact, that would 
> be the language definition; the rest is an optimization. (It's like
> how "x.y()" technically first looks up attribute "y" on object x, then
> calls the result; but it's perfectly reasonable for a Python
> implementation to notice this extremely common case and do an
> "optimized method call" that doesn't actually create a function
> object.)

class X:
   def y(self): pass

y is already a function object.

I think maybe you've got it backwards, and you mean the *method* object
doesn't have to be created. Well, sure, that's possible, and maybe PyPy
does something like that, and maybe it doesn't. Or maybe the function
descriptor __get__ method could cache the result:

# inside FunctionType class
def __get__(self, instance, type):
if type is not None:
if self._method is None:
self._method = MethodType(self, instance)
return self._method
else:
return self

(I think that's more or less how function __get__ currently works, apart
from the caching. But don't quote me.)

But that's much simpler than the early/late binding example. You talk
about "the obvious cases" like int, bool, str and None. What about floats
and frozensets, are they obvious? How about tuples? How about
MyExpensiveImmutableObject?


> The simpler the rule, the easier to grok, and therefore the 
> less chance of introducing bugs.

You're still going to surprise people who expect early binding:

FLAG = True

def spam(eggs=FLAG):
...


What do you mean, the default value gets recalculated every time I call
spam? It's an obvious immutable type! And why does Python crash when I
delete FLAG?

Worse:


def factory():
funcs = []
for i in range(1, 5):
def adder(x, y=i):
return x + y
adder.__name__ = "adder%d" % i
funcs.append(adder)
return funcs


The current behaviour with early binding:


py> funcs = factory()
py> [f(100) for f in funcs]
[101, 102, 103, 104]


What would it do with late binding? That's a tricky one. I can see two
likely results:

[f(100) for f in funcs]
=> returns [104, 104, 104, 104]

or

NameError: name 'i' is not defined


both of which are significantly less useful.

As I've said, it is trivial to get late binding semantics if you start with
early binding: just move setting the default value into the body of the
function. 99% of the time you can use None as a sentinel, so the common
case is easy:

def func(x=None):
if x is None: 
x = some_complex_calculation(i, want, to, repeat, each, time)


and the rest of the time, you just need *one* persistent variable to hold a
sentinel value to use instead of None:

_sentinel = object
def func(x=_sentinel, y=_sentinel, z=_sentinel):
if x is _sentinel: ...


But if you start with late binding, it's hard to *cleanly* get early binding
semantics. You need a separate global for each parameter of every function
in the module:

_default_x = some_complex_calculation(do, it, once)
_default_y = another_complex_calculation(do, it, once)
_default_z = a_third_complex_calculation(do, it, once)
_default_x_for_some_other_function = something_else()


def func(x=_default_x, y=_default_x, z=_default_z):  # oops, see the bug
...


which is just hideous. And even then, you don't really have early binding,
you have a lousy simulacra of it. If you modify or delete any of the
default globals, you're screwed.

No, early binding by default is the only sensible solution, and Guido got it
right. Having syntax for late binding would be a bonus, but it isn't really
needed. We already have a

Re: functions, optional parameters

[Steven D'Aprano]

On Sun, 10 May 2015 01:35 pm, Rustom Mody wrote:

> On Sunday, May 10, 2015 at 8:16:07 AM UTC+5:30, Steven D'Aprano wrote:
>> I predict that the majority of the time, late binding would just be a
>> pointless waste of time:
>> 
>> def process_string(thestr, start=0, end=None, slice=1, reverse=True):
>> pass
>> 
>> Why would you want 0, None, 1 and True to be re-evaluated every time?
>> Admittedly it will be fast, but not as fast as evaluating them once, then
>> grabbing a static default value when needed. (See below for timings.)
>> 
> 
> And what is the work involved in (re)computing 0, None, 1, True??

Re-computing a constant is about 5 times more expensive than re-using it,
according to my earlier timing tests. So if you have four of them, there
will be about 20 times more overhead due to the defaults, each and every
time you call the function.

Setting the defaults isn't the only source of overhead, but my guestimate is
that switching to late binding would probably double the overall overhead
of calling a function with one or two defaults. If your function is
expensive, that's trivial, but for small fast functions, that will be
painful. Python's slow enough without making it slower for dubious gains.


> If I write (... arg=square_root_of_grahams_number())
> I would expect to pay for it.

Sure, but only once. If you think that Graham's Number is likely to change
*wink* then you can put it into the body of the function, like any other
code you want run every time you call the function.



-- 
Steven

-- 
https://mail.python.org/mailman/listinfo/python-list