[TWISTED] Howto Deferred
Hello gals and guys, I'm an experienced Python user and I'd like to begin playing with Twisted. I started RTFM the tutorial advised on the official site and I found it really useful and well done. Now I'd like to practice a bit by coding a little program that reads strings from a serial device and redirects them remotely via TCP. For that sake I'm trying to use deferred. In the tutorial, a deferred class is instantiated at factory level, then used and destroyed. And here things get harder for me. Now, in my test program I need to manage data which comes in a "random" manner, and I thought about doing it in a few possible ways: 1. create a deferred at factory level and every time I read something from the serial port add some callbacks: class SerToTcpProtocol(Protocol): def dataReceived(self, data): # deferred is already instantiated and launched # self.factory.sendToTcp sends data to the TCP client self.factory.deferred.addCallback(self.factory.sendToTcp, data) 2. or, either, create a deferred at protocol level every time I receive something, then let the deferred do what I need and destroy it: class SerToTcpProtocol(Protocol): def dataReceived(self, data): d = defer.Deferred() d.addCallback(self.factory.sendToTcp, data) d.callback(data) 3. or again, use a deferred list: class SerToTcpProtocol(Protocol): def dataReceived(self, data): d = defer.Deferred() d.addCallback(self.factory.sendToTcp, data) self.factory.listDeferred.addCallback(lambda d) d.callback(data) Or I don't know.. hints are welcome. Thank you in advance and sorry for my english. -- http://mail.python.org/mailman/listinfo/python-list
Re: [TWISTED] Howto Deferred
On Thu, Jul 14, 2011 at 5:07 PM, marco wrote: > Now I'd like to practice a bit by coding a little program that reads > strings from a serial device and redirects them remotely via TCP. For > that sake I'm trying to use deferred. The obvious solution (to my mind) is two threads, one for each direction. On receipt of data, the thread immediately sends it on to the other end. I'm not familiar with deferred; it might work easily, or might need some fancy footwork to make it go. ChrisA -- http://mail.python.org/mailman/listinfo/python-list
Re: Functional style programming in python: what will you talk about if you have an hour on this topic?
On Jul 13, 1:39 pm, Anthony Kong wrote:
> (My post did not appear in the mailing list, so this is my second try.
> Apology if it ends up posted twice)
>
> Hi, all,
>
> If you have read my previous posts to the group, you probably have some idea
> why I asked this question.
>
> I am giving a few presentations on python to my colleagues who are mainly
> java developers and starting to pick up python at work.
>
>
> So I have picked this topic for one of my presentation. It is because
> functional programming technique is one of my favorite in my bag of python
> trick. It also takes me to the rabbit hole of the functional programming
> world, which is vastly more interesting than the conventional procedural/OO
> languages.
>
>
> I think I will go through the following items:
>
> itertools module
> functools module
> concept of currying ('partial')
>
> I would therefore want to ask your input e.g.
>
> Is there any good example to illustrate the concept?
> What is the most important features you think I should cover?
> What will happen if you overdo it?
>
> Cheers
I'd think you'd want to at least mention the relatively new
'lru_cache' decorator, for memoizing the return value expensive
functions, providing they are deterministic / pure, etc.
--
http://mail.python.org/mailman/listinfo/python-list
Re: Functional style programming in python: what will you talk about if you have an hour on this topic?
On Jul 14, 4:32 am, Gregory Ewing wrote: > Anthony Kong wrote: > > So I have picked this topic for one of my presentation. It is because > > functional programming technique is one of my favorite in my bag of python > > trick. > > I'm not sure it's a good idea to emphasise functional > programming too much. Python doesn't really lend itself > to a heavily functional style. While you *can* write > Python code that way, it's not idiomatic, and you're > likely to give beginners a distorted idea of how Python > is normally written. > > -- > Greg Maybe the talk would work well if not aimed at complete all-round beginners, but instead aimed at Pythonistas who are interested in functional programming, or functionistas who are py-curious (I think the same talk would work well for both groups) -- http://mail.python.org/mailman/listinfo/python-list
Re: Functional style programming in python: what will you talk about if you have an hour on this topic?
On Wednesday, July 13, 2011 5:39:16 AM UTC-7, Anthony Kong wrote:
[snip]
> I think I will go through the following items:
>
> itertools module
> functools module
> concept of currying ('partial')
>
>
> I would therefore want to ask your input e.g.
>
> Is there any good example to illustrate the concept?
> What is the most important features you think I should cover?
> What will happen if you overdo it?
Java is easily worst language I know of for support of functional programming
(unless they added delegates or some other tacked-on type like that), so my
advice would be to keep it light, for two reasons:
1. It won't take a lot to impress them
2. Too much will make them roll their eyes
Thinking about it, one of the problems with demonstrating functional features
is that it's not obvious how those features can simplify things. To get the
benefit, you have to take a step back and redo the approach somewhat.
Therefore, I'd recommend introducing these features as part of a demo on how a
task in Python can be solved much more concisely than in Java. It's kind of an
art to find good examples, though. Off the top of my head, I can think of
using functools module to help with logging or to apply patches, whereas in
Java they'd have to resort to a code weaver or lots of boilerplate.
Carl Banks
--
http://mail.python.org/mailman/listinfo/python-list
Re: Howto Deferred
On Jul 14, 3:07 am, marco wrote: > Hello gals and guys, > > I'm an experienced Python user and I'd like to begin playing with > Twisted. > I started RTFM the tutorial advised on the official site and I found it > really useful and well done. > > Now I'd like to practice a bit by coding a little program that reads > strings from a serial device and redirects them remotely via TCP. For > that sake I'm trying to use deferred. > Deferreds probably aren't a good solution for this problem. They're useful for one-time events, but you have an event that repeats over and over again with different data. > In the tutorial, a deferred class is instantiated at factory level, then > used and destroyed. > > And here things get harder for me. > Now, in my test program I need to manage data which comes in a "random" > manner, and I thought about doing it in a few possible ways: > > 1. create a deferred at factory level and every time I read something > from the serial port add some callbacks: > > class SerToTcpProtocol(Protocol): > > def dataReceived(self, data): > # deferred is already instantiated and launched > # self.factory.sendToTcp sends data to the TCP client > self.factory.deferred.addCallback(self.factory.sendToTcp, data) > Or you could do self.factory.sendToTcp(data) > 2. or, either, create a deferred at protocol level every time I receive > something, then let the deferred do what I need and destroy it: > > class SerToTcpProtocol(Protocol): > > def dataReceived(self, data): > d = defer.Deferred() > d.addCallback(self.factory.sendToTcp, data) > d.callback(data) > Same here. :) > 3. or again, use a deferred list: > > class SerToTcpProtocol(Protocol): > > def dataReceived(self, data): > d = defer.Deferred() > d.addCallback(self.factory.sendToTcp, data) > self.factory.listDeferred.addCallback(lambda d) > d.callback(data) > I'm not sure what the listDeferred is there for. Deferreds are a good abstraction for "do one thing and then tell me what the result was". You have a different sort of thing here, where there isn't much of a result (sending to tcp probably always works until you lose your connection). A method call works well for that. Jean-Paul -- http://mail.python.org/mailman/listinfo/python-list
RE: I don't know list, I not good at list.
Thank you all for the advice, let me spin this in a different way.
I've built a program that goes to the NANPA website, scrapes area
code/exchange (npa/nxx) digits for a specified area - be it carolina,
alabama, texas, etc - drops it into a file, then massages the data and
prints out the correct format to insert into a voice router. The code is
ugly, but it works great.
The thing I haven't been able to get my script to do is to reduce the
amount of dial-peers. Hence the need to reduce the numbers to the least
common denominator, and put it in the x[xx] format.If we had a set
number of digits, we could build a dictionary(unless it's possible to do
that dynamically).
So, a couple assertions:
1. the data will always be 6 digit numbers (in a string format)
2. the data is in a txt file after being put there from my script
3. the data will never be the same (I'm going to use this for site
conversions/new site builds
e.g. today I might be dealing with 252-, 919- and 704- area
codes, tomorrow might be 304- and 754-
4. I wanted a script to reduce the time taking to build the dial-peers
manually. I'd previously spent 3-4 hours
on gathering and processing data. The script I have so far pulls data
and massages in about 6 seconds
5. I'm using python 2.7 - it seems like it had more module availability
than 3
And a couple question:
1. Would lists be the best way to handle this data? Would it be better
to process this data from a file?
2. Is there a way to determine the common (first-5 digits) denominator
among a list (or file) of data?
3. and... Could those common numbers be inserted in a dict for
processing?
Sorry for the book!
thanks
Edward Ellerbee
-Original Message-
From: python-list-bounces+eellerbee=bbandt@python.org
[mailto:python-list-bounces+eellerbee=bbandt@python.org] On Behalf
Of MRAB
Sent: Wednesday, July 13, 2011 5:59 PM
To: python-list@python.org
Subject: Re: I don't know list, I not good at list.
> I've been beating my head against the desk trying to figure out a >
method to accomplish this:
>
> Take a list (this example is 5 items, It could be 150 or more - i.e.
> it's variable length depending on the city/local calling zones) > >
The first 6 digits of phone numbers(NPA/NXX) in a local calling area.
> I want to concatenate the last digit for insertion into a call >
routing pattern.
>
> I tried this and failed miserably:
>
> list1=['252205','252246','252206','252247','252248']
> for item in list1:
> try:
> item1=list1[0]
> item2=list1[1]
> if item1[0:5] == item2[0:5]:
> print item1[0:5] + '[' + item1[5:6] +
item2[5:6] + ']'
> list1.pop(0)
> else:
> print item1
> list1.pop(0)
> except:
> try:
> print item1
> list1.pop(0)
> except:
> pass
>
> #-
> My intent is to have the end data come out (from the example list >
above) in the format of > 25220[56] > 25224[678] > > I tried putting
together a variable inserted into a regular > expression, and it
doesn't seem to like:
> Item1=list1[0]
> Itemreg = re.compile(Item1[0:5])
> For stuff in itemreg.list1:
> #do something
>
> Can somebody throw me a bone, code example or module to read on >
python.org? I'm a n00b, so I'm still trying to understand functions >
and classes.
>
> I thought the experts on this list might take pity on my pathetic >
code skillz!
>
defaultdict comes in handy:
>>> list1 = ['252205','252246','252206','252247','252248']
>>> from collections import defaultdict >>> d = defaultdict(set) >>>
for item in list1:
d[item[ : 5]].add(item[5 : ])
>>> d
defaultdict(, {'25224': {'8', '7', '6'}, '25220': {'5',
'6'}}) >>> for k, v in d.items():
print(k + "[" + "".join(v) + "]")
25224[876]
25220[56]
--
http://mail.python.org/mailman/listinfo/python-list
--
http://mail.python.org/mailman/listinfo/python-list
PyDev 2.2.1 Released
Hi All, PyDev 2.2.1 has been released Details on PyDev: http://pydev.org Details on its development: http://pydev.blogspot.com Release Highlights: --- Quick-outline * Parent methods may be shown with a 2nd Ctrl+O. * The initial node is selected with the current location in the file. Extract local refactoring * Option to replace duplicates. * Fixed issue where wrong grammar could be used. Others * Improved handling of Ctrl+Shift+T so that no keybinding conflict takes place (now it'll be only active on the PyDev views/editor). * PyLint markers always removed on a project clean. * If the standard library source files are not found, more options are presented. * If the completion popup is focused and shift is pressed on a context insensitive completion, a local import is done. * Fixed issue where a local import wasn't being added to the correct location. * Fixed error message in debugger when there was no caught/uncaught exception set in an empty workspace. * Performance improvements on hierarchy view. * Django commands may be deleted on dialog with backspace. What is PyDev? --- PyDev is a plugin that enables users to use Eclipse for Python, Jython and IronPython development -- making Eclipse a first class Python IDE -- It comes with many goodies such as code completion, syntax highlighting, syntax analysis, refactor, debug and many others. Cheers, -- Fabio Zadrozny -- Software Developer Appcelerator http://appcelerator.com/ Aptana http://aptana.com/ PyDev - Python Development Environment for Eclipse http://pydev.org http://pydev.blogspot.com -- http://mail.python.org/mailman/listinfo/python-list
Re: An interesting beginner question: why we need colon at all in the python language?
On 2011-07-13, Thorsten Kampe wrote: > * Grant Edwards (Wed, 13 Jul 2011 13:03:22 + (UTC)) >> On 2011-07-13, Thorsten Kampe wrote: >> >> >> and that that block is to be considered in relation to what was just >> >> said, before the colon. >> > >> > The indentation makes it abundantly clear to the human reader that >> > that indented block is to be considered in relation to what was just >> > said, before the indentation. > >> You would think so, but human readers like redundancy. > > I also like redundancy (and consistency). That's why I'd much more > prefer a "then" than a colon which is easily overlooked while reading > /and/ while writing. How is the "then" going to be consistent with other things that also introduce blocks (def, try, with, etc.). -- Grant Edwards grant.b.edwardsYow! ! I'm in a very at clever and adorable INSANE gmail.comASYLUM!! -- http://mail.python.org/mailman/listinfo/python-list
Re: Suppressing newline writing to file after variable
On Fri, Jul 15, 2011 at 12:04 AM, Ellerbee, Edward wrote:
> Hey Chris,
>
> I was reading over this again, trying to understand the logic (I'm a
> n00b)
>
> Could you explain this a bit? I'd like to build this emit function, but
> I still don't have a firm grasp on functions. All of my code is line by
> line. I'll convert it as a learn more.
I'm responding on-list as I believe others will wish to weigh in (if
only to point out some improvements to my code - it's hastily put
together). Caution, post is long.
Defining functions in Python is, broadly speaking, just a matter of
collecting up a bunch of statements and giving it a name. Compare:
if x>5:
do_this()
do_that()
etc()
with:
def abcde():
do_this()
do_that()
etc()
The first one does the three statements, in order, if and only if the
condition is true. The second one gives a name to those three
statements, so you can use it as a new statement:
abcde()
It'll do the same three things, every time you call it. It's
effectively the same as putting the function's body in where you call
it (that's an extremely sloppy explanation, but near enough).
> So, I'd want this to go after the sort step, and before the format step.
> I can figure that piece out, just trying to get this little block of
> code to work.
Yes, that would be the place to put it.
Here's a reworked version that you can test in IDLE:
def combine(list_of_numbers, position):
lastprefix=tails=lastsuffix=None
result=[]
for cur in list_of_numbers:
prefix=cur[:position]; tail=cur[position]; suffix=cur[position+1:]
if prefix!=lastprefix or suffix!=lastsuffix:
if lastprefix!=None:
if len(tails)>1: result.append("%s[%s]%s"%(lastprefix,tails,lastsuffix))
else: result.append(lastprefix+tails+lastsuffix)
lastprefix,tails,lastsuffix=prefix,"",suffix
tails+=tail
if lastprefix!=None:
if len(tails)>1: result.append("%s[%s]%s"%(lastprefix,tails,lastsuffix))
else: result.append(lastprefix+tails+lastsuffix)
return result
It incorporates some of the enhancements I mentioned in the original post.
>>> combine(['252205','252206','252208'],5)
['25220[568]']
>>> combine(['252205','252215','252225'],4)
['2522[012]5']
Notice that the 'emit' function is now 'result.append()' - it builds
up a list to return. You can now chain the calls; start with a list of
numbers and then call combine() in a loop.
# List of numbers from your previous post
numbers = ['252205', '252206', '252208', '252220', '252221', '25',
'252223', '919745', '919725', '919785', '704770', '704771', '704772',
'704773', '704774', '704775', '704776', '704777', '704778', '704779',
'704780', '704781', '704782', '704783', '704784', '704785', '704786',
'704787', '704788', '704789', '704790', '704791', '704792', '704793',
'704794', '704795', '704796', '704797', '704798', '704799']
numbers = combine(numbers,5)
numbers = combine(numbers,4)
numbers = combine(numbers,3)
numbers = combine(numbers,2)
numbers = combine(numbers,1)
numbers = combine(numbers,0)
If you do these statements one at a time in IDLE and inspect the
'numbers' list each time, you'll see the combinations sorting
themselves out. (With this starting list, only the first two will have
any effect.)
The last set of calls can be turned into a for loop:
for pos in range(5,-1,-1):
numbers = combine(numbers,pos)
In fact, you could actually take it out of being a function, if you wanted to:
list_of_numbers = ['252205', '252206', '252208', '252220', '252221',
'25', '252223', '919745', '919725', '919785', '704770', '704771',
'704772', '704773', '704774', '704775', '704776', '704777', '704778',
'704779', '704780', '704781', '704782', '704783', '704784', '704785',
'704786', '704787', '704788', '704789', '704790', '704791', '704792',
'704793', '704794', '704795', '704796', '704797', '704798', '704799']
for position in range(5,-1,-1):
lastprefix=tails=lastsuffix=None
result=[]
for cur in list_of_numbers:
prefix=cur[:position]; tail=cur[position]; suffix=cur[position+1:]
if prefix!=lastprefix or suffix!=lastsuffix:
if lastprefix!=None:
if len(tails)>1: result.append("%s[%s]%s"%(lastprefix,tails,lastsuffix))
else: result.append(lastprefix+tails+lastsuffix)
lastprefix,tails,lastsuffix=prefix,"",suffix
tails+=tail
if lastprefix!=None:
if len(tails)>1: result.append("%s[%s]%s"%(lastprefix,tails,lastsuffix))
else: result.append(lastprefix+tails+lastsuffix)
list_of_numbers = result
That's what the function definition does - it takes a block of code
and gives it a new name. (There's a lot more to it than that, thank
you pedants I know, but in this simple example that's what it's
doing.)
Hope that's of use!
Chris Angelico
--
http://mail.python.org/mailman/listinfo/python-list
RE: Suppressing newline writing to file after variable
Holy cow, that's perfect!
Thanks so much :)
Would it be alright if I post my code on the list for critiquing? I'm
learning python to supplement my voice engineering position - time
consuming tasks that can be automated will take less time.
Edward Ellerbee
-Original Message-
From: python-list-bounces+eellerbee=bbandt@python.org
[mailto:python-list-bounces+eellerbee=bbandt@python.org] On Behalf
Of Chris Angelico
Sent: Thursday, July 14, 2011 10:39 AM
To: python-list@python.org
Subject: Re: Suppressing newline writing to file after variable
On Fri, Jul 15, 2011 at 12:04 AM, Ellerbee, Edward
wrote:
> Hey Chris,
>
> I was reading over this again, trying to understand the logic (I'm a
> n00b)
>
> Could you explain this a bit? I'd like to build this emit function,
> but I still don't have a firm grasp on functions. All of my code is
> line by line. I'll convert it as a learn more.
I'm responding on-list as I believe others will wish to weigh in (if
only to point out some improvements to my code - it's hastily put
together). Caution, post is long.
Defining functions in Python is, broadly speaking, just a matter of
collecting up a bunch of statements and giving it a name. Compare:
if x>5:
do_this()
do_that()
etc()
with:
def abcde():
do_this()
do_that()
etc()
The first one does the three statements, in order, if and only if the
condition is true. The second one gives a name to those three
statements, so you can use it as a new statement:
abcde()
It'll do the same three things, every time you call it. It's effectively
the same as putting the function's body in where you call it (that's an
extremely sloppy explanation, but near enough).
> So, I'd want this to go after the sort step, and before the format
step.
> I can figure that piece out, just trying to get this little block of
> code to work.
Yes, that would be the place to put it.
Here's a reworked version that you can test in IDLE:
def combine(list_of_numbers, position):
lastprefix=tails=lastsuffix=None
result=[]
for cur in list_of_numbers:
prefix=cur[:position]; tail=cur[position]; suffix=cur[position+1:]
if prefix!=lastprefix or suffix!=lastsuffix:
if lastprefix!=None:
if len(tails)>1:
result.append("%s[%s]%s"%(lastprefix,tails,lastsuffix))
else: result.append(lastprefix+tails+lastsuffix)
lastprefix,tails,lastsuffix=prefix,"",suffix
tails+=tail
if lastprefix!=None:
if len(tails)>1:
result.append("%s[%s]%s"%(lastprefix,tails,lastsuffix))
else: result.append(lastprefix+tails+lastsuffix)
return result
It incorporates some of the enhancements I mentioned in the original
post.
>>> combine(['252205','252206','252208'],5)
['25220[568]']
>>> combine(['252205','252215','252225'],4)
['2522[012]5']
Notice that the 'emit' function is now 'result.append()' - it builds up
a list to return. You can now chain the calls; start with a list of
numbers and then call combine() in a loop.
# List of numbers from your previous post numbers = ['252205', '252206',
'252208', '252220', '252221', '25', '252223', '919745', '919725',
'919785', '704770', '704771', '704772', '704773', '704774', '704775',
'704776', '704777', '704778', '704779', '704780', '704781', '704782',
'704783', '704784', '704785', '704786', '704787', '704788', '704789',
'704790', '704791', '704792', '704793', '704794', '704795', '704796',
'704797', '704798', '704799']
numbers = combine(numbers,5)
numbers = combine(numbers,4)
numbers = combine(numbers,3)
numbers = combine(numbers,2)
numbers = combine(numbers,1)
numbers = combine(numbers,0)
If you do these statements one at a time in IDLE and inspect the
'numbers' list each time, you'll see the combinations sorting themselves
out. (With this starting list, only the first two will have any effect.)
The last set of calls can be turned into a for loop:
for pos in range(5,-1,-1):
numbers = combine(numbers,pos)
In fact, you could actually take it out of being a function, if you
wanted to:
list_of_numbers = ['252205', '252206', '252208', '252220', '252221',
'25', '252223', '919745', '919725', '919785', '704770', '704771',
'704772', '704773', '704774', '704775', '704776', '704777', '704778',
'704779', '704780', '704781', '704782', '704783', '704784', '704785',
'704786', '704787', '704788', '704789', '704790', '704791', '704792',
'704793', '704794', '704795', '704796', '704797', '704798', '704799']
for position in range(5,-1,-1):
lastprefix=tails=lastsuffix=None
result=[]
for cur in list_of_numbers:
prefix=cur[:position]; tail=cur[position]; suffix=cur[position+1:]
if prefix!=lastprefix or suffix!=lastsuffix:
if lastprefix!=None:
if len(tails)>1:
result.append("%s[%s]%s"%(lastprefix,tails,lastsuffix))
else: result.append(lastprefix+tails+lastsuffix)
lastprefix,tails,lastsuffix=prefix,"",suffix
tails+=tail
if lastprefix!=None:
if len(tails)>1:
result.append("%s[%s]%s"%(lastprefix,tails,lastsuffix))
else: result.append(lastprefix+tails+lastsuf
Re: Suppressing newline writing to file after variable
On Fri, Jul 15, 2011 at 12:53 AM, Ellerbee, Edward wrote: > Holy cow, that's perfect! > > Thanks so much :) > > Would it be alright if I post my code on the list for critiquing? I'm > learning python to supplement my voice engineering position - time > consuming tasks that can be automated will take less time. If it's not too long, sure! If it's more than can conveniently be read, post it on one of those code-sharing places like http://pastebin.com/ and post a link; then those who want to read it can do so, and those who would rather avoid it have that option too. ChrisA -- http://mail.python.org/mailman/listinfo/python-list
String formatting - mysql insert
Hi,
I get some problem when i like to set the table name dynamic.
I'm appreciate for any help.
Christian
### works
newcur.execute ( """ INSERT INTO events (id1,id2) VALUES (%s,%s);
""" , (rs[1],rs[2]))
### works not
newcur.execute ( """ INSERT INTO %s_events (id1,id2) VALUES (%s,
%s); """ , (table_name,rs[1],rs[2]))
### works but is not really perfect: None from rs list result in
"None" instead of NULL.
newcur.execute ( """ INSERT INTO %s_events (id1,id2) VALUES
('%s','%s'); """ % (table_name,rs[1],rs[2]))
--
http://mail.python.org/mailman/listinfo/python-list
Re: An interesting beginner question: why we need colon at all in the python language?
On Jul 14, 10:34 am, Grant Edwards wrote: > On 2011-07-13, Thorsten Kampe wrote: > > > * Grant Edwards (Wed, 13 Jul 2011 13:03:22 + (UTC)) > >> On 2011-07-13, Thorsten Kampe wrote: > > >> >> and that that block is to be considered in relation to what was just > >> >> said, before the colon. > > >> > The indentation makes it abundantly clear to the human reader that > >> > that indented block is to be considered in relation to what was just > >> > said, before the indentation. > > >> You would think so, but human readers like redundancy. > > > I also like redundancy (and consistency). That's why I'd much more > > prefer a "then" than a colon which is easily overlooked while reading > > /and/ while writing. > > How is the "then" going to be consistent with other things that also > introduce blocks (def, try, with, etc.). > > -- > Grant Edwards grant.b.edwards Yow! ! I'm in a very > at clever and adorable INSANE > gmail.com ASYLUM!! But if you have the colon, why do you need the brackets or backslashes in an if statement. Why not if condition1 or condition2 or condition3: do_something() The statement ain't over til there's a colon. -- http://mail.python.org/mailman/listinfo/python-list
Re: String formatting - mysql insert
On Fri, Jul 15, 2011 at 1:00 AM, Christian wrote:
> Hi,
>
> I get some problem when i like to set the table name dynamic.
> I'm appreciate for any help.
>
> ### works but is not really perfect: None from rs list result in
> "None" instead of NULL.
> newcur.execute ( """ INSERT INTO %s_events (id1,id2) VALUES
> ('%s','%s'); """ % (table_name,rs[1],rs[2]))
I'll start with the easy one. This one is wrong for several reasons;
firstly, it converts everything to strings (which is why a None comes
out as 'None'), but secondly and more seriously, it cannot handle
apostrophes or backslashes in your strings. SQL engines such as MySQL
need strings to be properly escaped, and the execute() function will
do that for you - but the % interpolation won't.
> ### works not
> newcur.execute ( """ INSERT INTO %s_events (id1,id2) VALUES (%s,
> %s); """ , (table_name,rs[1],rs[2]))
What's happening here is that the table name is being sent in
apostrophes. Just as it puts quotes around your data, it also puts
quotes around the table name - which you don't want. You're getting
something like INSERT INTO 'foobar'_events, which MySQL doesn't like.
I recommend a hybrid:
newcur.execute ( """ INSERT INTO {0}_events (id1,id2) VALUES
(%s,%s); """.format(table_name), (,rs[1],rs[2]))
Note that I'm using the format() method rather than the % operator,
specifically because it uses a different notation - {0} - and will
leave the %s markers alone.
This assumes that your table name is clean. If it comes from your own
code, that's probably safe; but if it comes from user-supplied data,
you WILL need to sanitize it (I recommend whitelisting valid
characters eg letters and numbers, and keeping only those - and then
imposing a length limit too) before giving it to .execute().
As far as I know, MySQL doesn't have facilities for dynamic table
names, so your best bet is to make the SQL statement itself dynamic,
as per this example.
Hope that helps!
Chris Angelico
--
http://mail.python.org/mailman/listinfo/python-list
Re: String formatting - mysql insert
On 07/14/2011 11:00 AM, Christian wrote:
Hi,
I get some problem when i like to set the table name dynamic.
I'm appreciate for any help.
Christian
### works
newcur.execute ( """ INSERT INTO events (id1,id2) VALUES (%s,%s);
""" , (rs[1],rs[2]))
### works not
newcur.execute ( """ INSERT INTO %s_events (id1,id2) VALUES (%s,
%s); """ , (table_name,rs[1],rs[2]))
### works but is not really perfect: None from rs list result in
"None" instead of NULL.
newcur.execute ( """ INSERT INTO %s_events (id1,id2) VALUES
('%s','%s'); """ % (table_name,rs[1],rs[2]))
You shouldn't use The bottom form at all since that is how injection
attacks occur.
The reason the second version doesn't work is because the the execute
command escapes all of the arguments before replacing them. Example:
sql = """SELECT * FROM table WHERE col = %s;"""
cur.execute(sql, ('name',))
# The actual sql statement that gets executed is:
# SELECT * FROM table WHERE col = 'name';
# Notice the single quotes.
--
Bill
--
http://mail.python.org/mailman/listinfo/python-list
Re: How can I make a program automatically run once per day?
On 7/9/2011 10:01 PM, John Salerno wrote: Thanks everyone! I probably should have said something like "Python, if possible and efficient, otherwise any other method" ! :) I'll look into the Task Scheduler. Thanks again! You could use the below code. time.sleep(# seconds in a day) where i == 30 would run once a day for a month import time i=0 while (1): print 'hello' time.sleep(2) # Change this to number of seconds in a day if (i == 3):# make this 30 for a month break i = i + 1 -- http://mail.python.org/mailman/listinfo/python-list
Re: How can I make a program automatically run once per day?
On Thu, Jul 14, 2011 at 11:00 AM, monkeys paw wrote: > You could use the below code. time.sleep(# seconds in a day) > where i == 30 would run once a day for a month > > import time > i=0 > while (1): > print 'hello' > time.sleep(2) # Change this to number of seconds in a day > if (i == 3): # make this 30 for a month > break > i = i + 1 If the system ever gets rebooted during that month, then you would need to remember to manually restart the script. Or if the effective part of the script raises an exception, it could crash the whole script without some defensive coding. That's why it's better just to use the system scheduler service. -- http://mail.python.org/mailman/listinfo/python-list
json decode issue
Greetings,
I'm trying to decode JSON output of a Java program (jackson) and having some
issues.
The cause of the problem is the following snippet:
{
"description": "... lives\uMOVE™ OFFERS ",
}
Which causes ValueError: Invalid \u escape.
Any ideas on how to fix this?
Thanks,
--
Miki
--
http://mail.python.org/mailman/listinfo/python-list
Please critique my script
I've been working on this for 3 weeks as a project while I'm learning
python. It's ugly, only function in there is from a fellow lister, but
it works perfectly and does what it is intended to do.
I'd love to hear comments on how I could improve this code, what would
be good to turn into a function/class etc.
# The function of this script is to gather user data from the user to
# build appropriate local dial-peers in a cisco voice gateway.
# Data gathered from the user:
# name of file to write dial-peers to
# NPA
# NXX
# outbound port to send calls
# question if home npa local calls are 7 digit
# script navigates to nanpa.com, gathers the local calling area
# and returns the data
# the data is converted to beautiful soup (I had a problem with the xml
format)
# the data is parsed for the proper NPA/NXXs
# list is sorted, duplicates removed
# data from list is formatted and printed to the file specified
#Import dependencies---
import urllib2
from BeautifulSoup import BeautifulSoup
import re
#Define variables
count = 0
count2 = 0
count3 = 0
npalist = []
nxxlist = []
sortlist = []
sortedlist = []
npaReg = re.compile('(?<=)(...)(?=)')
nxxReg = re.compile('(?<=)(...)(?=)')
proxy = urllib2.ProxyHandler({'http': 'proxy.com:8080'})
# --actual proxy was removed for confidentiality--
opener = urllib2.build_opener(proxy)
#- function to concatenate numbers - thanks to Chris Angelico -
def combine(list_of_numbers, position):
lastprefix=tails=lastsuffix=None
result=[]
for cur in list_of_numbers:
prefix=cur[:position]; tail=cur[position];
suffix=cur[position+1:]
if prefix!=lastprefix or suffix!=lastsuffix:
if lastprefix!=None:
if len(tails)>1:
result.append("%s[%s]%s"%(lastprefix,tails,lastsuffix))
else: result.append(lastprefix+tails+lastsuffix)
lastprefix,tails,lastsuffix=prefix,"",suffix
tails+=tail
if lastprefix!=None:
if len(tails)>1:
result.append("%s[%s]%s"%(lastprefix,tails,lastsuffix))
else: result.append(lastprefix+tails+lastsuffix)
return result
#---Gather info from user
x = raw_input("please enter a filename: ")
y = raw_input("please enter the npa: ")
z = raw_input("please enter the nxx: ")
p = raw_input("please enter the port: ")
q = raw_input("Is home npa local dialing 7 digit?(y/n) ")
#print x
#---Modify user data-
o = open(x, 'w')
y = str(y)
z = str(z)
p = str(p)
pagedef = ("http://www.localcallingguide.com/xmllocalprefix.php?npa="; +
y + "&nxx=" + z)
print "Querying", pagedef
#--Get info from NANPA.com --
urllib2.install_opener(opener)
page = urllib2.urlopen(pagedef)
soup = BeautifulSoup(page)
soup = str(soup)
#--Parse Gathered Data--
for line in npaReg.findall(soup):
npalist.insert(count,line)
count = count + 1
for line2 in nxxReg.findall(soup):
nxxlist.insert(count2,line2)
count2 = count2 + 1
#-Sort, remove duplicates, concatenate the last digits for similiar
NPA/NXX --
for makenewlist in range(0,count):
sortlist.append(npalist.pop(0) + nxxlist.pop(0))
sortlist.sort()
for sortednumber in sortlist:
if sortednumber not in sortedlist:
sortedlist.append(sortednumber)
catlist = combine(sortedlist,5)
catlist2 = combine(catlist,4)
#--Print the dial-peers to file
for line in catlist2:
figureDpn = count3 + 1000
dpn = str(figureDpn)
label = "dial-peer voice " + dpn
o.write(label)
o.write('\n')
o.write("description *** local outbound dialpeer ***")
o.write('\n')
destpatt = "destination-pattern %s" % line.rstrip()
o.write(destpatt)
o.write('\n')
port = "port " + p
o.write(port)
o.write('\n')
if line[0:3] == y and q == "y":
o.write("forward-digits 7")
o.write('\n')
o.write('\n')
count3 = count3 + 1
o.close()
#---
Thanks!
Edward Ellerbee
--
http://mail.python.org/mailman/listinfo/python-list
Re: String formatting - mysql insert
On 14 Jul., 17:31, Billy Mays wrote:
> On 07/14/2011 11:00 AM, Christian wrote:
>
>
>
>
>
>
>
>
>
> > Hi,
>
> > I get some problem when i like to set the table name dynamic.
> > I'm appreciate for any help.
>
> > Christian
>
> > ### works
> > newcur.execute ( """ INSERT INTO events (id1,id2) VALUES (%s,%s);
> > """ , (rs[1],rs[2]))
>
> > ### works not
> > newcur.execute ( """ INSERT INTO %s_events (id1,id2) VALUES (%s,
> > %s); """ , (table_name,rs[1],rs[2]))
>
> > ### works but is not really perfect: None from rs list result in
> > "None" instead of NULL.
> > newcur.execute ( """ INSERT INTO %s_events (id1,id2) VALUES
> > ('%s','%s'); """ % (table_name,rs[1],rs[2]))
>
> You shouldn't use The bottom form at all since that is how injection
> attacks occur.
>
> The reason the second version doesn't work is because the the execute
> command escapes all of the arguments before replacing them. Example:
>
> sql = """SELECT * FROM table WHERE col = %s;"""
> cur.execute(sql, ('name',))
> # The actual sql statement that gets executed is:
> # SELECT * FROM table WHERE col = 'name';
> # Notice the single quotes.
>
> --
> Bill
thanks you guys!
--
http://mail.python.org/mailman/listinfo/python-list
RE: Please critique my script
For me, there are some things I don't like much. One-character
variable names stand out (tend to make the code hard to read).
Violation of PEP 8 guidelines, especially wrt spacing. e.g.
result.append("%s[%s]%s" % (lastprefix, tails, lastsuffix))
not
result.append("%s[%s]%s"%(lastprefix,tails,lastsuffix))
Similarly for many of the assignments and expressions.
I see that you initialize count3 at line 39 but don't use it until
line 123. I'd suggest that you move the initialization closer to its
use.
I think that you should move the call to open closer to where you're
going to write to the file. In fact, you can do the thing a bit
neater with a context manager like this:
#--Print the dial-peers to file
with open(x, 'w') as o:
for line in catlist2:
figureDpn = count3 + 1000
dpn = str(figureDpn)
label = "dial-peer voice " + dpn
o.write(label)
o.write('\n')
...
Note that if you use this approach you don't need a separate call to close().
Also, you can do all the writing in one call to write(), something like this:
o.write(
label + '\n' +
"description *** local outbound dialpeer ***" + '\n' +
destpatt + '\n' +
"port " + p + '\n'
"forward-digits 7" if line[0:3] == y and q == "y" else ""
'\n'
)
Which keeps the eye focused on the data being written (at least for me).
--
Gerald Britton
--
http://mail.python.org/mailman/listinfo/python-list
Re: Please critique my script
[snip]
raw_input() returns a string, so there's no need for these 3 lines:
> y = str(y)
> z = str(z)
> p = str(p)
> pagedef = ("http://www.localcallingguide.com/xmllocalprefix.php?npa=";
+ y + "&nxx=" + z)
> print "Querying", pagedef
>
> #--Get info from NANPA.com --
> urllib2.install_opener(opener)
> page = urllib2.urlopen(pagedef)
> soup = BeautifulSoup(page)
> soup = str(soup)
>
> #--Parse Gathered Data--
> for line in npaReg.findall(soup):
> npalist.insert(count, line)
> count = count + 1
>
> for line2 in nxxReg.findall(soup):
> nxxlist.insert(count2, line2)
> count2 = count2 + 1
>
enumerate will help you here:
for count, line in enumerate(npaReg.findall(soup)):
npalist.insert(count, line)
for count2, line2 in enumerate(nxxReg.findall(soup)):
nxxlist.insert(count2, line2)
> #-Sort, remove duplicates, concatenate the last digits for
similiar NPA/NXX --
> for makenewlist in range(0, count):
> sortlist.append(npalist.pop(0) + nxxlist.pop(0))
>
> sortlist.sort()
>
> for sortednumber in sortlist:
> if sortednumber not in sortedlist:
> sortedlist.append(sortednumber)
>
If you're going to sort them anyway (so you don't need to preserve the
existing order), the easiest way to remove duplicates is to use a set:
sortedlist = sorted(set(sortlist))
[snip]
--
http://mail.python.org/mailman/listinfo/python-list
Re: json decode issue
On 14/07/2011 18:22, Miki Tebeka wrote:
Greetings,
I'm trying to decode JSON output of a Java program (jackson) and having some
issues.
The cause of the problem is the following snippet:
{
"description": "... lives\uMOVE™ OFFERS ",
}
Which causes ValueError: Invalid \u escape.
Any ideas on how to fix this?
Is that valid JSON? If not, you'll either need to fix the program which
generated it (or report it as a bug), or pre-process the JSON to
correct the error, if you know what it should be.
--
http://mail.python.org/mailman/listinfo/python-list
Possible File iteration bug
I noticed that if a file is being continuously written to, the file
generator does not notice it:
def getLines(f):
lines = []
for line in f:
lines.append(line)
return lines
with open('/var/log/syslog', 'rb') as f:
lines = getLines(f)
# do some processing with lines
# /var/log/syslog gets updated in the mean time
# always returns an empty list, even though f has more data
lines = getLines(f)
I found a workaround by adding f.seek(0,1) directly before the last
getLines() call, but is this the expected behavior? Calling f.tell()
right after the first getLines() call shows that it isn't reset back to
0. Is this correct or a bug?
--
Bill
--
http://mail.python.org/mailman/listinfo/python-list
Re: Possible File iteration bug
On Thu, Jul 14, 2011 at 1:46 PM, Billy Mays wrote:
> def getLines(f):
> lines = []
> for line in f:
> lines.append(line)
> return lines
>
> with open('/var/log/syslog', 'rb') as f:
> lines = getLines(f)
> # do some processing with lines
> # /var/log/syslog gets updated in the mean time
>
> # always returns an empty list, even though f has more data
> lines = getLines(f)
>
>
>
>
> I found a workaround by adding f.seek(0,1) directly before the last
> getLines() call, but is this the expected behavior? Calling f.tell() right
> after the first getLines() call shows that it isn't reset back to 0. Is
> this correct or a bug?
This is expected. Part of the iterator protocol is that once an
iterator raises StopIteration, it should continue to raise
StopIteration on subsequent next() calls.
--
http://mail.python.org/mailman/listinfo/python-list
Re: json decode issue
On 7/14/2011 3:20 PM, MRAB wrote:
On 14/07/2011 18:22, Miki Tebeka wrote:
Greetings,
I'm trying to decode JSON output of a Java program (jackson) and
having some issues.
The cause of the problem is the following snippet:
{
"description": "... lives\uMOVE™ OFFERS ",
}
Which causes ValueError: Invalid \u escape.
Any ideas on how to fix this?
Is that valid JSON? If not, you'll either need to fix the program which
generated it (or report it as a bug), or pre-process the JSON to
correct the error, if you know what it should be.
If you delete or double the backslash in that one particular spot, the
string will parse, even if it is not correct.
--
Terry Jan Reedy
--
http://mail.python.org/mailman/listinfo/python-list
Re: Possible File iteration bug
On 07/14/2011 04:00 PM, Ian Kelly wrote:
On Thu, Jul 14, 2011 at 1:46 PM, Billy Mays wrote:
def getLines(f):
lines = []
for line in f:
lines.append(line)
return lines
with open('/var/log/syslog', 'rb') as f:
lines = getLines(f)
# do some processing with lines
# /var/log/syslog gets updated in the mean time
# always returns an empty list, even though f has more data
lines = getLines(f)
I found a workaround by adding f.seek(0,1) directly before the last
getLines() call, but is this the expected behavior? Calling f.tell() right
after the first getLines() call shows that it isn't reset back to 0. Is
this correct or a bug?
This is expected. Part of the iterator protocol is that once an
iterator raises StopIteration, it should continue to raise
StopIteration on subsequent next() calls.
Is there any way to just create a new generator that clears its `closed`
status?
--
Bill
--
http://mail.python.org/mailman/listinfo/python-list
Re: Possible File iteration bug
On 7/14/2011 3:46 PM, Billy Mays wrote:
I noticed that if a file is being continuously written to, the file
generator does not notice it:
Because it does not look, as Ian explained.
def getLines(f):
lines = []
for line in f:
lines.append(line)
return lines
This nearly duplicates .readlines, except for using f an an iterator.
Try the following (untested):
with open('/var/log/syslog', 'rb') as f:
lines = f.readlines()
# do some processing with lines
# /var/log/syslog gets updated in the mean time
lines = f.readlines()
People regularly do things like this with readline, so it is possible.
If above does not work, try (untested):
def getlines(f):
lines = []
while True:
l = f.readline()
if l: lines.append(l)
else: return lines
--
Terry Jan Reedy
--
http://mail.python.org/mailman/listinfo/python-list
Re: Possible File iteration bug
Billy Mays writes: > Is there any way to just create a new generator that clears its > closed` status? You can define getLines in terms of the readline file method, which does return new data when it is available. def getLines(f): lines = [] while True: line = f.readline() if line == '': break lines.append(line) return lines or, more succinctly: def getLines(f): return list(iter(f.readline, '')) -- http://mail.python.org/mailman/listinfo/python-list
Re: Please critique my script
MRAB wrote: >> for line2 in nxxReg.findall(soup): >> nxxlist.insert(count2, line2) >> count2 = count2 + 1 >> > enumerate will help you here: > for count2, line2 in enumerate(nxxReg.findall(soup)): > nxxlist.insert(count2, line2) An insert() at the end of a list is usually spelt append() in Python ;) If you are looking for something less baroque nxxlist = nxxReg.findall(soup) will do, too. -- http://mail.python.org/mailman/listinfo/python-list
Re: PyCon Australia 2011: Schedule Announced
Hi Ryan. Best of luck with the conference. >Thanks also to Linux Australia, who provide the overarching legal and >organisational structure for PyCon Australia. I want to talk to somebody from Linux Australia about this overarching legal and organisational structure. Do you have an email address of whom I should talk to? I think the structure that we have here in Europe could stand some refactoring, and I was talking with Stephen Thorne at Europython and what Linux Australia is doing looks very neat to me. There is nothing urgent about this, and in no way should this distract you from your conference, but sometime I would like to have an email. Thanks very much, Laura Creighton -- http://mail.python.org/mailman/listinfo/python-list
Python threading/multiprocessing issue.
I'm working on a tool that runs a number of process is separate thread. I've, up to this point, been using threading.Thread, but from what I read multiprocess will allow multiple processors to be used From the python docs on multiprocessing. I have run into an issue when modifying the thread object from the run method. Threading.thread allows me to change an attribute in the run method and it hold while multiprocessing.Process loses it. Here is an example illustrating the inconsistency that I've seen. -- | import time import multiprocessing import threading def simple_process_call(): my_process = SimpleProcess() my_process.start() while not my_process.done.is_set(): pass print my_process.my_attribute class SimpleProcess(multiprocessing.Process): def __init__(self): super(SimpleProcess, self).__init__() self.my_attribute = 'Fail' self.done = multiprocessing.Event() def run(self): self.my_attribute = 'Success' time.sleep(5) self.done.set() def simple_thread_call(): my_thread = SimpleThread() my_thread.start() while not my_thread.done.is_set(): pass print my_thread.my_attribute class SimpleThread(threading.Thread): def __init__(self): super(SimpleThread, self).__init__() self.my_attribute = 'Fail' self.done = threading.Event() def run(self): self.my_attribute = 'Success' time.sleep(5) self.done.set() if __name__ == '__main__': # simple_process_call() simple_thread_call()| -- The odd thing is that I can modify the multiprocessing.Event and it holds, but modifying any attribute on the class goes away. If I am super ignorant of something, please cure me of it. Thanks in advance! Brandon L. Harris -- http://mail.python.org/mailman/listinfo/python-list
Multiplicity and Asininity in Tkinter Event API
# Multiplicity and Asininity in Tkinter Event API! # The problems with Tkinter events are two fold: Problem 1: Number Of Sequences Is Obscene. The sheer number of exposed sequences and possible user combinations of sequences is overkill. You should never put the design of an API in the hands of users of said API. Designing rock solid API's is about giving users as much power as possible WITHOUT giving them too many choices. Unfortunately, not only did the authors of Tkinter give people choices, they gave them the power to create NEW choices... CRIKEY! Problem 2. No Uniform Naming Convention. The sequence names follow an intuitive naming convention. Not only that, but alias's exists for some of the combinations. Here are some examples... <1> | callback event inspection Examples Of "Possible" Sequence Combinations Bindings for a single letter: etc... Bindings for Mice: <1> etc... This is only for one keypres and one button down event and this does not EVEN include combinations of mice and key. Completely ridiculous! As you can see this is far too many sequences for even a guru to remember; and why should he? Why should he clog up his code with handler after handler when only a handful are needed? SOLUTION: I can tell you first hand that out of all these thousands of sequences we only need six to cover user inputs. YES, you heard me correctly. SIX! Here are the six i propose: That's it. Go ahead, try to prove me wrong! Potential Naysayer's Arguments: Argument_1: Wah! But i don't like to have to process possible all three (or more) mouse buttons in the same event handler.. Wah, Wah! Rebuttual_1: No problemo amigo, just dispatch those specific button events to specific handlers. Here kiddo, watch this... def onMouseClick(num, pos, rootpos): if num == 1: self.onButtonOneClick(num, pos, rootpos) if num == 2: self.onButtonOneClick(num, pos, rootpos) etc... etc... Conclusion: It's time to start cleaning up this library. All you @-holes fought me long ago about how GREAT Tkinter is. I don't think Tkinter is great however i do believe it has great potential. Now it the time to put up or shut up. I AM willing to put forth the effort. I have documented the deficiencies, and i have cleaned up code. I have offered up more intuitive API interfaces. NOW it's high time for the dev's to come forward. http://www.youtube.com/watch?v=XzcWwmwChVE -- http://mail.python.org/mailman/listinfo/python-list
Re: Please critique my script
On 14/07/2011 21:53, Peter Otten wrote: MRAB wrote: for line2 in nxxReg.findall(soup): nxxlist.insert(count2, line2) count2 = count2 + 1 enumerate will help you here: for count2, line2 in enumerate(nxxReg.findall(soup)): nxxlist.insert(count2, line2) An insert() at the end of a list is usually spelt append() in Python ;) If you are looking for something less baroque nxxlist = nxxReg.findall(soup) will do, too. That's true... :-) -- http://mail.python.org/mailman/listinfo/python-list
Proposal to extend PEP 257 (New Documentation String Spec)
Hello Folks,
Lately i have been musing over the ideas of method tagging.
Specifically i am referring to method identifiers. As most of you know
i had proposed to add "syntactical markers" to the language to deal
with the ambiguities that arise whist eyeball parsing sub classed
methods that clobber virtual methods. HOWEVER that idea caused some
fierce controversy within the community, and i can partly understand
why.
Although i strongly believe in proper documentation (even to the point
of forcing syntax on folks) i understand that we cannot implement such
a thing without major growing pains. So with that being said, i have
formulated a new battle plan to defeat this problem of ambiguity.
Unlike most languages out there we have doc-strings; and do we realize
how great this gift is? Sometimes i wonder because you folks should
really be using them like they are going out of style!
As we all know PEP 257 lays out some ground rules for documentation
strings HOWEVER i feel this PEP did no go far enough. Too many folks
are refusing to document properly and so i will take this time to
hammer out a spec. I would like to comments for or against.
---
New Syntax Specification For Documentation Strings
---
""" {DOC TAG HERE}: {MODULE_NAME|SHORT_SUMMARY_HERE}.
{NEWLINE}
{LONG_DESCRIPTION_HERE}
{NEWLINE}
Arguments: (if applicable)
{ARGUMNET_1} <{TYPE}>:
ARGUMENT_1_DESCRIPTION}
{ARGUMNET_2} <{TYPE}>:
ARGUMENT_2 DESCRIPTION}
{ARGUMNET_N} <{TYPE}>:
ARGUMENT_N_DESCRIPTION}
{NEWLINE}
"""
As you can see my spec introduces some new ideas to writing doc-
strings. Specifically the "DOC TAG" and {ARG TYPES} are new. Also i've
found it much more useful to separate args and their respective
descriptions with a newline and indention.
---
Example: Module Documentation String.
---
"""Module :
This module handles Tkinter dialog boxes.
It contains the following public symbols:
Dialog :
A base class for dialogs.
askinteger :
Get an integer from the user.
askfloat :
Get a float from the user.
askstring :
Get a string from the user.
"""
I don't know how i feel about marking classes and functions since IF
we follow the python style guide we don;t need to; but that's IF we
FOLLOW it people, IF.
---
Example: Func/Meth Documentation String.
---
def askinteger(parent, title, prompt, **kw):
""" Interface: Get an integer from the user.
Return value is an integer.
Arguments:
title :
the dialog title
prompt :
the label text
**kw:
see SimpleDialog class
---
Example: Class Inheritance Documentation Strings.
---
class Base():
def __init__(self):
""" Internal:"""
self.m1()
def m1(self, *args):
"""Overide: """
pass
class Derived(Base):
def __init__(self):
Base.__init__(self)
def _m1(self):
""" Internal: blah"""
def m1(self):
""" Clobbered: see Base for detail"""
def m3(self):
""" Interface: Blah"""
---
Tags For Documentation Strings
---
Module:
The module tag is to be used for module doc strings.
Virtual:
The virtual tag is for methods residing in a base class
that are created specifically to be overridden. Of course as we
all know every Python methods/function is virtual by default
however the point of this is to make the code more readable!
Override:
This tag should be placed in a derived class's method which has
clobbered a base method. Typically you can just defer the reader
to look up the base class for more info.
Internal:
This tag should be used on all internal methods (psst: the ones
that
start with a single underscore *ahem* or SHOULD start with a
single
underscore!).
Interface:
This tag is be used for interface method/function doc strings.
This
is probably the most important tag. If you don't do any tagging AT
LEAST tag the interface methods and functions. However i must
remind you that all these tags are very important.
--
http://mail.python.org/mailman/listinfo/python-list
Re: Proposal to extend PEP 257 (New Documentation String Spec)
rantingrick writes:
> ---
> New Syntax Specification For Documentation Strings
> ---
>
> """ {DOC TAG HERE}: {MODULE_NAME|SHORT_SUMMARY_HERE}.
> {NEWLINE}
> {LONG_DESCRIPTION_HERE}
> {NEWLINE}
> Arguments: (if applicable)
> {ARGUMNET_1} <{TYPE}>:
> ARGUMENT_1_DESCRIPTION}
> {ARGUMNET_2} <{TYPE}>:
> ARGUMENT_2 DESCRIPTION}
> {ARGUMNET_N} <{TYPE}>:
> ARGUMENT_N_DESCRIPTION}
> {NEWLINE}
> """
I use reStructuredText formatting in my PEP 257 docstrings:
def frobnicate(spong, mode="wibble"):
""" Frobnicate the spong.
:param spong: The SpongDrabble instance to be frobnicated.
:param mode: Specific frobnication mode to use. Valid modes
are "wibble", "wobble", "warble".
:return: The blagule from the frobnication.
Note that the Weebly-Ruckford algorithm is used for
frobnication portability. See
list(), tuple() should not place at "Built-in functions" in documentation
As telling in the subject,because "list" and "tuple" aren't functions,they are types.Is that right? -- http://mail.python.org/mailman/listinfo/python-list
Re: list(), tuple() should not place at "Built-in functions" in documentation
Inside wrote: > As telling in the subject,because "list" and "tuple" aren't functions,they > are types.Is that right? Yes they are types. But they can still be used as functions. Does it matter? -- Steven -- http://mail.python.org/mailman/listinfo/python-list
Re: list(), tuple() should not place at "Built-in functions" in documentation
On Jul 14, 8:21 pm, Inside wrote: > As telling in the subject,because "list" and "tuple" aren't functions,they > are types.Is that right? You wanna see some warts in the docs. Okay, try to use the search box to find list, dict, or tuple and see what happens... http://docs.python.org/ Search: [ list ] PyFloat_ClearFreeList (cfunction, in Floating Point Objects) PyInt_ClearFreeList (cfunction, in Plain Integer Objects) PyListObject (ctype, in List Objects) PyList_Append (cfunction, in List Objects) PyList_AsTuple (cfunction, in List Objects) PyList_Check (cfunction, in List Objects) PyList_CheckExact (cfunction, in List Objects) PyList_GET_ITEM (cfunction, in List Objects) PyList_GET_SIZE (cfunction, in List Objects) PyList_GetItem (cfunction, in List Objects) PyList_GetSlice (cfunction, in List Objects) PyList_Insert (cfunction, in List Objects) PyList_New (cfunction, in List Objects) PyList_Reverse (cfunction, in List Objects) PyList_SET_ITEM (cfunction, in List Objects) PyList_SetItem (cfunction, in List Objects) PyList_SetSlice (cfunction, in List Objects) PyList_Size (cfunction, in List Objects) PyList_Sort (cfunction, in List Objects) PyList_Type (cvar, in List Objects) PyMethod_ClearFreeList (cfunction, in Method Objects) [ snip: mile long list with no LIST info to be found! ] Hey don't get me wrong, the python docs are great; as long as you know where to find what you're looking for. -- http://mail.python.org/mailman/listinfo/python-list
Re: list(), tuple() should not place at "Built-in functions" in documentation
Steven D'Aprano writes: > Inside wrote: > > > As telling in the subject,because "list" and "tuple" aren't functions,they > > are types.Is that right? > > Yes they are types. But they can still be used as functions. Does it matter? As a newcomer to the documentation I looked fruitlessly in the table of contents for a section that would contain the built-in types. “Built-in functions” was eliminated for the reason the OP states. I think it matters. (But I haven't proposed a documentation patch for it.) -- \ “But Marge, what if we chose the wrong religion? Each week we | `\ just make God madder and madder.” —Homer, _The Simpsons_ | _o__) | Ben Finney -- http://mail.python.org/mailman/listinfo/python-list
Re: list(), tuple() should not place at "Built-in functions" in documentation
Excerpts from rantingrick's message of Thu Jul 14 21:36:15 -0400 2011: > On Jul 14, 8:21pm, Inside wrote: > > As telling in the subject,because "list" and "tuple" aren't functions,they > > are types.Is that right? > > You wanna see some warts in the docs. Okay, try to use the search box > to find list, dict, or tuple and see what happens... > > http://docs.python.org/ > > Search: [ list ] > > PyMethod_ClearFreeList (cfunction, in Method Objects) > > [ snip: mile long list with no LIST info to be found! ] > > Hey don't get me wrong, the python docs are great; as long as you know > where to find what you're looking for. > I agree, having the stuff from the C API docs appear in the search isn't very useful. Perhaps the search should be redesigned with stuff like that in mind? (Or maybe the search is more advanced than I use it). They aren't exactly warts, it's useful information, but in the common case they probably aren't desired (I always use Google to search around the python docs). Not to mention that the search is slooo. It's plenty fast on my local download, but I haven't actually looked at the search to see what it does and how. -- Corey Richardson "Those who deny freedom to others, deserve it not for themselves" -- Abraham Lincoln signature.asc Description: PGP signature -- http://mail.python.org/mailman/listinfo/python-list
Re: list(), tuple() should not place at "Built-in functions" in documentation
Hey guy,thx for you feedback first. But I can't follow your opinion.Why?because of the list & tuple are placed at built-in function,so before I type 'list' unintentionally on the pyshell and it show me "", I never know that the name 'list' is a type,I used to consider it's a function to produce 'list' type. so,after I figure out this matter,I have to change all my code "assert isinstance(someobj, (type([]), type((0, " to "assert isinstance(someobj, (list, tuple))",that's not a funny job. I hope that I can stay in the Python abstract layer to solve problem(although go to the C API is OK but I don't want to),I'm going to trust what the doc telling me,so I hope the doc is exact enough.And the doc in the distribution maybe the most popular one. @Steven D'Aprano,yes they can be used as function,but they aren't function and shouldn't confuse newcomers by this. -- http://mail.python.org/mailman/listinfo/python-list
Re: list(), tuple() should not place at "Built-in functions" in documentation
On 7/14/2011 9:51 PM, Ben Finney wrote: Steven D'Aprano writes: Inside wrote: As telling in the subject,because "list" and "tuple" aren't functions,they are types.Is that right? At one time (before 2.2), they were functions and not classes. Yes they are types. But they can still be used as functions. Does it matter? As a newcomer to the documentation I looked fruitlessly in the table of contents for a section that would contain the built-in types. “Built-in functions” was eliminated for the reason the OP states. I think it matters. (But I haven't proposed a documentation patch for it.) I once proposed, I believe on the tracker, that 'built-in functions' be expanded to 'built-in function and classes'. That was rejected on the basis that people would then expect the full class documentation that is in the 'built-in types' section (which could now be called the built-isssn classes section. A more exact title would be 'built-in callables', but that would be even less helpful to newcomers. Callables are functions in the generic sense. In any case, the new index makes it easy to see what is in that chapter. -- Terry Jan Reedy -- http://mail.python.org/mailman/listinfo/python-list
Re: list(), tuple() should not place at "Built-in functions" in documentation
Inside writes: > But I can't follow your opinion.Why?because of the list & tuple are > placed at built-in function,so before I type 'list' unintentionally on > the pyshell and it show me "", I never know that the name > 'list' is a type,I used to consider it's a function to produce 'list' > type. That's the kind of fundamental knowledge that one gains by working through the Python tutorial http://docs.python.org/tutorial/>. The library reference is not the place for teaching that information. > so,after I figure out this matter,I have to change all my code "assert > isinstance(someobj, (type([]), type((0, " to "assert > isinstance(someobj, (list, tuple))",that's not a funny job. If you think you need to do such assertions, that's a code smell; it's rare to need that kind of assertion and should only be done with good reason since it breaks polymorphism. Why are you doing it? > I hope that I can stay in the Python abstract layer to solve > problem(although go to the C API is OK but I don't want to),I'm going > to trust what the doc telling me,so I hope the doc is exact enough.And > the doc in the distribution maybe the most popular one. Including the tutorial, so now you have your homework to do :-) > @Steven D'Aprano,yes they can be used as function,but they aren't > function and shouldn't confuse newcomers by this. Agreed; however, it seems reasonable people can disagree on how much that matters. I think it should be fixed, but not enough to push for it. -- \ “Leave nothing to chance. Overlook nothing. Combine | `\ contradictory observations. Allow yourself enough time.” | _o__) —Hippocrates | Ben Finney -- http://mail.python.org/mailman/listinfo/python-list
Re: json decode issue
On Thu, 14 Jul 2011 10:22:44 -0700, Miki Tebeka wrote:
> I'm trying to decode JSON output of a Java program (jackson) and having
> some issues. The cause of the problem is the following snippet:
> {
> "description": "... lives\uMOVE™ OFFERS ",
> }
> Which causes ValueError: Invalid \u escape.
>
> Any ideas on how to fix this?
It's the input data which is broken. The parser is behaving correctly by
raising an exception.
--
http://mail.python.org/mailman/listinfo/python-list
None versus MISSING sentinel -- request for design feedback
Hello folks,
I'm designing an API for some lightweight calculator-like statistics
functions, such as mean, standard deviation, etc., and I want to support
missing values. Missing values should be just ignored. E.g.:
mean([1, 2, MISSING, 3]) => 6/3 = 2 rather than 6/4 or raising an error.
My question is, should I accept None as the missing value, or a dedicated
singleton?
In favour of None: it's already there, no extra code required. People may
expect it to work.
Against None: it's too easy to mistakenly add None to a data set by mistake,
because functions return None by default.
In favour of a dedicated MISSING singleton: it's obvious from context. It's
not a lot of work to implement compared to using None. Hard to accidentally
include it by mistake. If None does creep into the data by accident, you
get a nice explicit exception.
Against MISSING: users may expect to be able to choose their own sentinel by
assigning to MISSING. I don't want to support that.
I've considered what other packages do:-
R uses a special value, NA, to stand in for missing values. This is more or
less the model I wish to follow.
I believe that MATLAB treats float NANs as missing values. I consider this
an abuse of NANs and I won't be supporting that :-P
Spreadsheets such as Excel, OpenOffice and Gnumeric generally ignore blank
cells, and give you a choice between ignoring text and treating it as zero.
E.g. with cells set to [1, 2, "spam", 3] the AVERAGE function returns 2 and
the AVERAGEA function returns 1.5.
numpy uses masked arrays, which is probably over-kill for my purposes; I am
gratified to see it doesn't abuse NANs:
>>> import numpy as np
>>> a = np.array([1, 2, float('nan'), 3])
>>> np.mean(a)
nan
numpy also treats None as an error:
>>> a = np.array([1, 2, None, 3])
>>> np.mean(a)
Traceback (most recent call last):
File "", line 1, in
File "/usr/lib/python2.5/site-packages/numpy/core/fromnumeric.py", line
860, in mean
return mean(axis, dtype, out)
TypeError: unsupported operand type(s) for +: 'int' and 'NoneType'
I would appreciate any comments, advice or suggestions.
--
Steven
--
http://mail.python.org/mailman/listinfo/python-list
Re: Proposal to extend PEP 257 (New Documentation String Spec)
Was tried at least once before:
http://www.python.org/dev/peps/pep-0287/
Check in here with your ideas:
http://www.python.org/community/sigs/current/doc-sig/
Have any other languages mandated the use of a specific documentation markup?
Michael
On Thu, Jul 14, 2011 at 7:02 PM, rantingrick wrote:
>
> Hello Folks,
>
> Lately i have been musing over the ideas of method tagging.
> Specifically i am referring to method identifiers. As most of you know
> i had proposed to add "syntactical markers" to the language to deal
> with the ambiguities that arise whist eyeball parsing sub classed
> methods that clobber virtual methods. HOWEVER that idea caused some
> fierce controversy within the community, and i can partly understand
> why.
>
> Although i strongly believe in proper documentation (even to the point
> of forcing syntax on folks) i understand that we cannot implement such
> a thing without major growing pains. So with that being said, i have
> formulated a new battle plan to defeat this problem of ambiguity.
>
> Unlike most languages out there we have doc-strings; and do we realize
> how great this gift is? Sometimes i wonder because you folks should
> really be using them like they are going out of style!
>
> As we all know PEP 257 lays out some ground rules for documentation
> strings HOWEVER i feel this PEP did no go far enough. Too many folks
> are refusing to document properly and so i will take this time to
> hammer out a spec. I would like to comments for or against.
>
> ---
> New Syntax Specification For Documentation Strings
> ---
>
> """ {DOC TAG HERE}: {MODULE_NAME|SHORT_SUMMARY_HERE}.
> {NEWLINE}
> {LONG_DESCRIPTION_HERE}
> {NEWLINE}
> Arguments: (if applicable)
> {ARGUMNET_1} <{TYPE}>:
> ARGUMENT_1_DESCRIPTION}
> {ARGUMNET_2} <{TYPE}>:
> ARGUMENT_2 DESCRIPTION}
> {ARGUMNET_N} <{TYPE}>:
> ARGUMENT_N_DESCRIPTION}
> {NEWLINE}
> """
>
> As you can see my spec introduces some new ideas to writing doc-
> strings. Specifically the "DOC TAG" and {ARG TYPES} are new. Also i've
> found it much more useful to separate args and their respective
> descriptions with a newline and indention.
>
> ---
> Example: Module Documentation String.
> ---
>
> """Module :
>
> This module handles Tkinter dialog boxes.
> It contains the following public symbols:
>
> Dialog :
> A base class for dialogs.
>
> askinteger :
> Get an integer from the user.
>
> askfloat :
> Get a float from the user.
>
> askstring :
> Get a string from the user.
>
> """
>
> I don't know how i feel about marking classes and functions since IF
> we follow the python style guide we don;t need to; but that's IF we
> FOLLOW it people, IF.
>
> ---
> Example: Func/Meth Documentation String.
> ---
>
> def askinteger(parent, title, prompt, **kw):
> """ Interface: Get an integer from the user.
>
> Return value is an integer.
>
> Arguments:
> title :
> the dialog title
> prompt :
> the label text
> **kw:
> see SimpleDialog class
>
>
> ---
> Example: Class Inheritance Documentation Strings.
> ---
>
> class Base():
> def __init__(self):
> """ Internal:"""
> self.m1()
>
> def m1(self, *args):
> """Overide: """
> pass
>
> class Derived(Base):
> def __init__(self):
> Base.__init__(self)
>
> def _m1(self):
> """ Internal: blah"""
>
> def m1(self):
> """ Clobbered: see Base for detail"""
>
> def m3(self):
> """ Interface: Blah"""
>
> ---
> Tags For Documentation Strings
> ---
>
> Module:
> The module tag is to be used for module doc strings.
>
> Virtual:
> The virtual tag is for methods residing in a base class
> that are created specifically to be overridden. Of course as we
> all know every Python methods/function is virtual by default
> however the point of this is to make the code more readable!
>
> Override:
> This tag should be placed in a derived class's method which has
> clobbered a base method. Typically you can just defer the reader
> to look up the base class for more info.
>
> Internal:
> This tag should be used on all internal methods (psst: the ones
> that
> start with a single underscore *ahem* or SHOULD start with a
> single
> underscore!).
>
> Interface:
> This tag is be used for interface method/function doc strings.
> This
> is probably the most important tag. If you don't do any tagging AT
> LEAST tag the interface methods and functions. However i must
> remind you that all these tags are very important.
>
> --
> http://mail.pyt
Re: None versus MISSING sentinel -- request for design feedback
On Fri, Jul 15, 2011 at 3:28 PM, Steven D'Aprano
wrote:
> My question is, should I accept None as the missing value, or a dedicated
> singleton?
>
> In favour of None: it's already there, no extra code required. People may
> expect it to work.
>
> Against None: it's too easy to mistakenly add None to a data set by mistake,
> because functions return None by default.
I guess the question is: Why are the missing values there? If they're
there because some function returned None because it didn't have a
value to return, and therefore it's a missing value, then using None
as "missing" would make a lot of sense. But if it's a more explicit
concept of "here's a table of values, and the user said that this one
doesn't exist", it'd be better to have an explicit MISSING. (Which I
assume would be exposed as yourmodule.MISSING or something.)
Agreed that float('nan') and "" and "spam" are all bad values for
Missings. Possibly "" should come out as 0, but "spam" should
definitely fail.
Chris Angelico
--
http://mail.python.org/mailman/listinfo/python-list
Re: Multiplicity and Asininity in Tkinter Event API
On Fri, Jul 15, 2011 at 8:34 AM, rantingrick wrote: > > > > > > > > That's it. Go ahead, try to prove me wrong! > Does MouseClick mean Mouse Button Down, or does it mean Mouse Button Pressed Then Released Within A Short Time Without The Mouse Moving In Between? Both events are needed. Since you have MouseRelease, I am guessing that it's MouseButtonDown (or MousePress, to parallel your KeyPress/KeyRelease). If you only have MouseDown and MouseUp events (or MousePress and MouseRelease, whatever you wish to call them), then users have a lot of trouble implementing CUA-compliant mouse-click handlers. In the spirit of Ook and its predecessor, I propose one single event: InputEvent. It will have a parameter that specifies the key ID or mouse button that triggered the event; 0 means no key/button, and indicates a mouse movement event. It will contain the location of the mouse, and it's up to the program to keep track of the last known mouse location, to detect movement. The beauty of this system is that programmers can build interfaces that involve pointing at an object, holding the four keys D-R-A-G, and moving the mouse to another location and releasing those keys. This is far more intuitive than the classic mouse-button way of dragging objects around, and avoids the age-old debate of whether to drag with the left or right mouse button. Tkinter should provide this facility so that all new programs will be written with this in mind. Chris Angelico -- http://mail.python.org/mailman/listinfo/python-list
Re: list(), tuple() should not place at "Built-in functions" in documentation
On Fri, Jul 15, 2011 at 11:32 AM, Steven D'Aprano
wrote:
> Inside wrote:
>
>> As telling in the subject,because "list" and "tuple" aren't functions,they
>> are types.Is that right?
>
> Yes they are types. But they can still be used as functions. Does it matter?
Python is duck-typed, even in its documentation. If Python describes
something as a function, it means it can be used in place of func in
here:
result = func(arg1, arg2, arg3)
It might be something created with def (a "classic" function). It
might be something created with lambda. It might be an object with a
__call__ method. It might be a type.
>>> class foo:
def __call__(self):
return lambda: print("asdf")
>>> bar=foo()
>>> quux=bar()
>>> quux()
asdf
How many functions are defined here?
Chris Angelico
--
http://mail.python.org/mailman/listinfo/python-list
Re: Please critique my script
On Fri, Jul 15, 2011 at 4:03 AM, Ellerbee, Edward wrote: > for makenewlist in range(0,count): > sortlist.append(npalist.pop(0) + nxxlist.pop(0)) This can alternatively be done with map(): sortlist = map(lambda x,y: x+y, npalist, nxxlist) However, I'm not sure what your code is meant to do if the two lists have differing numbers of elements (if the two regexps turn up differing numbers of results). If you can assume that this won't happen, the simple map call will do the job. (It would have been a lot cleaner if Python exposed its operators as functions. In Pike, that lambda would simply be `+ (backtick-plus).) Chris Angelico -- http://mail.python.org/mailman/listinfo/python-list
