Re: a Haskell a Day

[usenet]

Xah Lee wrote:
> I hope you will join me in learning Haskell.

I think the folks here are more interested in Perl. There's a reason
why this newsgroup is called lc("comp.lang.PERL.misc").

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: a Haskell a Day

[usenet]

Xah Lee wrote:
> I hope you will join me in learning Haskell.

I think the folks here are more interested in Perl. There's a reason
why this newsgroup is called lc("comp.lang.PERL.misc").

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: tool for syntax coloring in html

[usenet]

Xah Lee wrote:
> Is there a tool that produce codes in html with syntax coloring?

Sure. It's called "Cascading Style Sheets" (CSS), which is how these
sites have done it (as you can see if you "view source").  See articles
on SitePoint.com (and MANY others) for info. But this really has
nothing to do with Perl.

-- 
http://mail.python.org/mailman/listinfo/python-list

net view /domain

[usenet]

Hi!

Is there a module I can use for "net view /domain" so list all
available domains and workgroups in a windows-network? I'm looking for
something like win32net.NetServerEnum, because I don't really want to
do this by "popen".

Kind regards
Dirk

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: net view /domain

[usenet]

Hi Tim!

Thanks again for your help!

I just tried it out and it does exactly what I want it to do :-)

Have a nice day!
Dirk

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Jargons of Info Tech industry

[usenet]

In comp.lang.perl.misc John Bokma <[EMAIL PROTECTED]> wrote:
> 
> > the argument that usenet should never change seems a little
> > heavy-handed and anachronistic.
> 
> No, simple since there *are* alternatives: web based message boards. Those 
> alternatives *do* support HTML formatting (often the subset mentioned 

... and generally these "web based message boards" (i.e. forums I
assume you mean) have none of the useful tools that Usenet offers and
are much, much slower.

> earlier). However, Usenet is a stranger to most people on the Internet, 
> even with Usenet access, and hence, there is no real reason to see it 
> changed into something that is "available" for years and years to more 
> people: www.
> 
> -- 
> John   Small Perl scripts: http://johnbokma.com/perl/
>Perl programmer available: http://castleamber.com/
> Happy Customers: http://castleamber.com/testimonials.html
> 

-- 
Chris Green

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Jargons of Info Tech industry

[usenet]

In comp.lang.perl.misc Ulrich Hobelmann <[EMAIL PROTECTED]> wrote:
> [EMAIL PROTECTED] wrote:
> > In comp.lang.perl.misc John Bokma <[EMAIL PROTECTED]> wrote:
> >>> the argument that usenet should never change seems a little
> >>> heavy-handed and anachronistic.
> >> No, simple since there *are* alternatives: web based message boards. Those 
> >> alternatives *do* support HTML formatting (often the subset mentioned 
> > 
> > ... and generally these "web based message boards" (i.e. forums I
> > assume you mean) have none of the useful tools that Usenet offers and
> > are much, much slower.
> 
> That is because NNTP and its applications didn't evolve to feed the 
> glitzy need lots of users have.
> 
I don't think they have "glitzy need", they are just fed glitzy (but
slow) forums as the way to get support etc.  If they were told about
the alternatives as well and told how to use them then I thiink those
alternatives would be used.

"NNTP and its applications" have evolved to provide a set of much more
sophisticated means of accessing and giving information than any forum
I've ever seen.


> Sadly web forums (esp. the ugly, slw PHPBB, and the unspeakable 
> Google groups) are increasingly replacing usenet, but there are 
> exceptions (DragonflyBSD).
> 
One good solution is a furum which is also accessible by NNTP.


> On the information side (in contrast to the discussion side) RSS is 
> replacing Usenet, with some obvious disadvantages: go on vacation, 
> return after a week, and -- yahoo! -- all your RSS feeds only turn of 
> the, say, most recent 30 articles, while your newsgroups all show 
> everything you missed.
> 
Same applies to most newsfeeds, depending on retention.  If you want
to look a long way back in a thread, use Google Groups.

-- 
Chris Green

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Jargons of Info Tech industry

[usenet]

In comp.lang.perl.misc John Bokma <[EMAIL PROTECTED]> wrote:
> [EMAIL PROTECTED] wrote:
> 
> > In comp.lang.perl.misc John Bokma <[EMAIL PROTECTED]> wrote:
> >> 
> >> > the argument that usenet should never change seems a little
> >> > heavy-handed and anachronistic.
> >> 
> >> No, simple since there *are* alternatives: web based message boards.
> >> Those alternatives *do* support HTML formatting (often the subset
> >> mentioned 
> > 
> > ... and generally these "web based message boards" (i.e. forums I
> > assume you mean) have none of the useful tools that Usenet offers and
> > are much, much slower.
> 
> Yup, Slow because of all the HTML and avatars. And you suggest to introduce 
> such a thing to Usenet?
> 
No, quite the opposite, I like Usenet News as it is.


> And which useful tools do you require?
> 
A choice of news readers to suit different people with different
interfaces, filtering, kill files, etc. etc.  A forum provides a
single, usually rather limited, interface for the user with no way for
the user to change it radically.

-- 
Chris Green

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Usenet, HTML (was Re: Jargons of Info Tech industry)

[usenet]

In comp.lang.perl.misc John Bokma <[EMAIL PROTECTED]> wrote:
> > 
> > I'm talking about using the technology for communication, instead of 
> > reinventing the wheel with crappy web forums.
> 
> What is exactly crappy about those forums?
> 
They are slow

They are inflexible

They are slow

They don't allow the user to choose how to view them, the interface is
imposed on the user.

They are slow

They don't have killfiles or scoring

They are slow

-- 
Chris Green

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Jargons of Info Tech industry

[usenet]

In comp.lang.perl.misc John Bokma <[EMAIL PROTECTED]> wrote:
> [EMAIL PROTECTED] wrote:
> 
> > "NNTP and its applications" have evolved to provide a set of much more
> > sophisticated means of accessing and giving information than any forum
> > I've ever seen.
> 
> Example(s). And do users need those sophisticated things?
> 
Kill files

Selecting posts and threads based on a scoring system

A huge variety of different newsreaders allowing different users to
access the news in they way they want.

I don't use all the possibilities (e.g. I don't use kill files) but I
do use a 'minority' text based newsreader because it is ideal for me.
I don't get the option of a text based forum reader - I doubt many
forums work with lynx.

-- 
Chris Green

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Jargons of Info Tech industry

[usenet]

In comp.lang.perl.misc John Bokma <[EMAIL PROTECTED]> wrote:
> 
> > They 
> > have no downsides I can possibly think of
> 
> Some people never use them, and hence they use memory and add risks.
> 
So they can choose a newsreader that doesn't have these facilities, no
extra memory use, no risk.

-- 
Chris Green

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Xah's Edu Corner: Unix damage: color names

[usenet]

Xah Lee wrote:
> usually located in /lib/rgb.txt.

on AIX and Linux (SuSE 9.3) the file is in /lib/X11/rgb.txt

> neither a continuity in selected color values nor in color names (for
> example, darkgray but lightgrey)

On AIX and Linux (SuSE 9.3) each color name which contains "gray" is
also aliased as "grey" for the benefit of both Yanks and Brits.  Thus,
I have:

   211 211 211 LightGrey
   211 211 211 LightGray

   169 169 169 DarkGrey
   169 169 169 DarkGray

I'm curious what UNIX system does not dual-spell this color? (neutral,
actually; gray is not a color). I thought these color (neutral) names
were dual-homed on all reasonably modern UN*X systems.

-- 
http://DavidFilmer.com

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: John Bokma harassment

[usenet]

Xah Lee wrote:
> I do not like to post off-topic messages

Oh REALLY?  That's strange, because I don't recall ever seeing an
on-topic message (a Perl message in a Perl newsgroup) from Xah.  Every
one of the many Xah post I've ever seen (including the "Philosopher"
message that this thread morphed into) was off-topic for a Perl
programming newsgroup.

-- 
http://DavidFilmer.com

-- 
http://mail.python.org/mailman/listinfo/python-list

Logging and threading

[usenet]

I'm having some problems getting the logging module to work with the
threading module.  I've narrowed the problem down to the following
code:

import logging, threading

update_log = logging.getLogger('update_log')
update_log.addHandler(logging.FileHandler("/tmp/update_log"))

class dlThread(threading.Thread):
def __init__(self):
threading.Thread.__init__(self)

def run(self):
ul = logging.getLogger('update_log')
ul.warn("log this")
print "finished"

t = dlThread()
t.start()

When executed, I receive the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python2.4/logging/__init__.py", line 735, in
emit
self.stream.write(fs % msg)
ValueError: I/O operation on closed file

The exception isn't fatal though, as the print "finished" line is
executed.  Alternately, if I run dlThread.run directly everything works
as I would expect, albeit without the multithreading.  Since the
logging module is advertised as being threadsafe, I suspect there's
something basic I don't understand about threads.

-Alex

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Logging and threading

[usenet]

As simple and as obvious as I expected, thanks Dennis.

-Alex

-- 
http://mail.python.org/mailman/listinfo/python-list

OT: google groups bug, or worse?

[aaronwmail-usenet]

I'm concerned that google groups is not correctly reflecting the
python lists.  A month ago I announced the xsdbXML framework to the
python list and the python-announce list.  As you can see from the
links
below the python announce submission was approved by the moderators
(thanks!)
and the python list submission also went out, but the messages cannot
be found at google groups.

http://mail.python.org/pipermail/python-list/2004-December/254479.html
http://mail.python.org/pipermail/python-announce-list/2004-December/003583.html

Is it a google bug?  Or is it something darker, like an anti-Python
conspiracy at google?

Inquiring minds want to know.

-- Aaron Watters

===
There are 3 kinds of people, those who can count and those who can't.
-- folklore.

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: OT: google groups bug, or worse?

[aaronwmail-usenet]

Bengt Richter wrote:
> What did you google with? Is this it?
>
http://groups-beta.google.com/groups?hl=en&ie=UTF-8&q=%22The+xsdbXML+framework+provides+a+flexible+and+well+defined+infrastructure%22&qt_s=Search+Groups

That was my *reply* to one of the original posts using Google,
which I faked up when I couldn't find it (posted using google).
The two originals aren't there.

hmmm.   -- Aaron Watters
===
You'd be paranoid too, if everyone was out to get you!

-- 
http://mail.python.org/mailman/listinfo/python-list

On benchmarks, heaps, priority queues

[aaronwmail-usenet]

I've been wondering about benchmarks recently.

What is a fair benchmark?

How should benchmarks be vetted or judged?

I decided to see what you folks thought, so for discussion I
compared two priority queue implementations I
published for Python in 1995 against the "heap" priority
queue implementation added to the python distribution
recently (python 2.3+).

My implementations and the benchmark code are in

http://xsdb.sourceforge.net/bench/pq3.py

(benchmark code near the bottom of the file). The python
standard implementation should be in your Lib/ directory.

Below please find the numerical results of the benchmarks.
They compare the implementations

PQPython23 - the Lib implementation
PQ0 - my insertion sort based variant
PQueue - my "heap" based variant
(like PQPython23, but different).

There are 2 benchmarks used.

insertBench
-- which does insertions followed by deletes.
mixBench
-- which does insertions and deletes mixed together.

Here is my summary of the results:

1) For sizes less than 1 the insertion
sort PQ0 is always better.

2) For insertBench sized >1 PQ0 is very
bad and the other two are comparable with
PQPython23 always a little faster.

3) For mixBench both of my implementations are
better than the Python23 implementation for
all sizes tested, but PQ0 is far superior.

Here is my analysis of the results:

1) The python23 implementation is best when you
expect to be doing mainly inserts with few deletions
and the expected size is >1 (otherwise use pq0).

2) PQ0 is best if you expect to be doing many deletions
interspersed with many inserts.

3) PQueue is generally worse than the other two
unless you want to use a non-standard comparison
function (which the others don't support).

My questions to you:

Are my conclusions valid?

What is a better way to perform and publish benchmarks?

-- thanks,   Aaron Watters

===
JUVENILE COURT TO TRY SHOOTING SUSPECT
-- a "real life headline" (that should work)

ENCLOSURE: run of
http://xsdb.sourceforge.net/bench/pq3.py on my machine.
===

BENCHMARKS FOR 1000

insertBench
__main__.PQPython23 on 1000 elapsed 0.03961853 got 1000
__main__.PQ0 on 1000 elapsed 0.0099046326 got 1000
__main__.PQueue on 1000 elapsed 0.029713898 got 1000

mixBench
__main__.PQPython23 on 1000 elapsed 0.0099046326 got 502
__main__.PQ0 on 1000 elapsed 0.0 got 502
__main__.PQueue on 1000 elapsed 0.0099046326 got 502

BENCHMARKS FOR 1

insertBench
__main__.PQPython23 on 1 elapsed 0.261000156403 got 1
__main__.PQ0 on 1 elapsed 0.20799728 got 1
__main__.PQueue on 1 elapsed 0.36161035 got 1

mixBench
__main__.PQPython23 on 1 elapsed 0.059427795 got 5003
__main__.PQ0 on 1 elapsed 0.051907349 got 5003
__main__.PQueue on 1 elapsed 0.069332428 got 5003

BENCHMARKS FOR 10

insertBench
__main__.PQPython23 on 10 elapsed 3.24500012398 got 10
__main__.PQ0 on 10 elapsed 7.9309428 got 10
__main__.PQueue on 10 elapsed 4.8260273 got 10

mixBench
__main__.PQPython23 on 10 elapsed 0.64132425 got 5
__main__.PQ0 on 10 elapsed 0.47056131 got 5
__main__.PQueue on 10 elapsed 0.65122888 got 5

BENCHMARKS FOR 20

insertBench
__main__.PQPython23 on 20 elapsed 6.9801907 got 20
__main__.PQ0 on 20 elapsed 28.3209998608 got 20
__main__.PQueue on 20 elapsed 10.4350001812 got 20

mixBench
__main__.PQPython23 on 20 elapsed 1.29099988937 got 11
__main__.PQ0 on 20 elapsed 0.91140872 got 11
__main__.PQueue on 20 elapsed 1.3321717 got 11

BENCHMARKS FOR 30

insertBench
__main__.PQPython23 on 30 elapsed 10.9159998894 got 30
__main__.PQ0 on 30 elapsed 69.631144 got 30
__main__.PQueue on 30 elapsed 19.3279998302 got 30

mixBench
__main__.PQPython23 on 30 elapsed 2.443782 got 15
__main__.PQ0 on 30 elapsed 1.5824469 got 15
__main__.PQueue on 30 elapsed 2.14300012589 got 15

BENCHMARKS FOR 40

insertBench
__main__.PQPython23 on 40 elapsed 16.263114 got 40
__main__.PQ0 on 40 elapsed 153.66113 got 40
__main__.PQueue on 40 elapsed 24.786134 got 40

mixBench
__main__.PQPython23 on 40 elapsed 2.9143433 got 20
__main__.PQ0 on 40 elapsed 1.8630001545 got 20
__main__.PQueue on 40 elapsed 2.7336758 got 20

BENCHMARKS FOR 50

insertBench
__main__.PQPython23 on 50 elapsed 20.1890001297 got 50
__main__.PQ0 on 50 elapsed 246.383999825 got 50
__main__.PQueue on 50 elapsed 30.104916 got 50

mixBench
__main__.PQPython23 on 50 elapsed 3.2650001049 got 25
__main__.PQ0 on 50 elapsed 2.3522561 got 25
__main__.PQueue on 50 elapsed 3.3254768 got 25

BENCHMARKS FOR 100

insertBench
__main__.PQPython23 on 100 elapsed 43.032648 got 100
__main__.PQ0 on 100 elapsed 1376.3983 got 10

Re: On benchmarks, heaps, priority queues

[aaronwmail-usenet]

me> PQPython23 - the Lib implementation
me> PQ0 - my insertion sort based variant
me> PQueue - my "heap" based variant
me> (like PQPython23, but different).

Tim D:
> First of all, you should be running these benchmarks using Python
2.4.
> heapq is considerably faster there ... (Raymond Hettinger rewrote it
all
> in C).
> http://www.python.org/2.4/highlights.html
> Tim Delaney

WHOOPS. Okay after getting past my initial embarassment I tried this,
and it turns out it doesn't make much of a difference.  In particular
PQ0 is still always better if you intersperse many deletes with inserts
or in all cases of size <1.  Of course other machines will probably
have different characteristics (and yes my py24 contains the _heapq
extension module).

The nasty behavior of PQ0 for insertBench at >10
I think has to do with large list reallocation.

Hmmm.  This makes me wonder if my bplustree implementation(s) are
faster
than the bsddb modules too...

-- Aaron Watters
===
SOMETHING WENT WRONG IN AIRLINE CRASH, EXPERT SAYS
-- a "real life headline" (that'll be $50,000, please)


ENCLOSURE: Python 2.4 run of
http://xsdb.sourceforge.net/bench/pq3.py on my machine.
===

BENCHMARKS FOR 1000

insertBench
__main__.PQPython23 on 1000 elapsed 0.019809265 got 1000
__main__.PQ0 on 1000 elapsed 0.0099046326 got 1000
__main__.PQueue on 1000 elapsed 0.032098083 got 1000

mixBench
__main__.PQPython23 on 1000 elapsed 0.0 got 502
__main__.PQ0 on 1000 elapsed 0.0099046326 got 502
__main__.PQueue on 1000 elapsed 0.0 got 502

BENCHMARKS FOR 1

insertBench
__main__.PQPython23 on 1 elapsed 0.26017984 got 1
__main__.PQ0 on 1 elapsed 0.21038147 got 1
__main__.PQueue on 1 elapsed 0.35895096 got 1

mixBench
__main__.PQPython23 on 1 elapsed 0.071716614 got 5003
__main__.PQ0 on 1 elapsed 0.050999879837 got 5003
__main__.PQueue on 1 elapsed 0.069332428 got 5003

BENCHMARKS FOR 10

insertBench
__main__.PQPython23 on 10 elapsed 3.26499986649 got 10
__main__.PQ0 on 10 elapsed 7.8814196 got 10
__main__.PQueue on 10 elapsed 4.8161226 got 10

mixBench
__main__.PQPython23 on 10 elapsed 0.65099978447 got 5
__main__.PQ0 on 10 elapsed 0.461000204086 got 5
__main__.PQueue on 10 elapsed 0.66113351 got 5

BENCHMARKS FOR 20

insertBench
__main__.PQPython23 on 20 elapsed 6.9900954 got 20
__main__.PQ0 on 20 elapsed 28.5010001659 got 20
__main__.PQueue on 20 elapsed 10.384905 got 20

mixBench
__main__.PQPython23 on 20 elapsed 1.29099988937 got 11
__main__.PQ0 on 20 elapsed 0.922000169754 got 11
__main__.PQueue on 20 elapsed 1.3420763 got 11

BENCHMARKS FOR 30

insertBench
__main__.PQPython23 on 30 elapsed 11.697267 got 30
__main__.PQ0 on 30 elapsed 70.3009998798 got 30
__main__.PQueue on 30 elapsed 16.884629 got 30

mixBench
__main__.PQPython23 on 30 elapsed 1.9338774 got 15
__main__.PQ0 on 30 elapsed 1.3915995 got 15
__main__.PQueue on 30 elapsed 1.9635913 got 15

BENCHMARKS FOR 40

insertBench
__main__.PQPython23 on 40 elapsed 15.5419998169 got 40
__main__.PQ0 on 40 elapsed 127.25321 got 40
__main__.PQueue on 40 elapsed 23.062638 got 40

mixBench
__main__.PQPython23 on 40 elapsed 2.64399981499 got 20
__main__.PQ0 on 40 elapsed 1.9536866 got 20
__main__.PQueue on 40 elapsed 2.7336758 got 20

BENCHMARKS FOR 50

insertBench
__main__.PQPython23 on 50 elapsed 20.881144 got 50
__main__.PQ0 on 50 elapsed 246.95424 got 50
__main__.PQueue on 50 elapsed 35.960657 got 50

mixBench
__main__.PQPython23 on 50 elapsed 3.5559428 got 25
__main__.PQ0 on 50 elapsed 2.4834005 got 25
__main__.PQueue on 50 elapsed 3.48500013351 got 25

BENCHMARKS FOR 100

insertBench
__main__.PQPython23 on 100 elapsed 44.694057 got 100
__main__.PQ0 on 100 elapsed 1400.5940001 got 100
__main__.PQueue on 100 elapsed 70.661409 got 100

mixBench
__main__.PQPython23 on 100 elapsed 6.6311444 got 51
__main__.PQ0 on 100 elapsed 4.7366104 got 51
__main__.PQueue on 100 elapsed 6.8292371 got 51

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: On benchmarks, heaps, priority queues

[aaronwmail-usenet]

(re: http://xsdb.sourceforge.net/bench/pq3.py)

nsz> ...bisect is not so fast for large data...

Yes I know in theory the insertion sort approach should be bad for
large enough values, but the weird thing is that if you mix inserts and
deletes (with enough deletes) even 1M elements is not a large enough
value.  Anyway, for 10K or less the insertion sort priority queue
implementation seems to always be better.

Weird.  -- Aaron Watters

===
Hypothetical performance improvements are the root of all evil.
-- Bill Tutt (paraphrased)

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: On benchmarks, heaps, priority queues

[aaronwmail-usenet]

re http://xsdb.sourceforge.net/bench/pq3.py

Tim Peters:
> If you repair that, and
> instrument mixBench() to keep track of queue size statistics, you'll
> find that even at 100, the queue at the top of the loop never
> exceeds 30 entries, and has a mean size less than 3.

Aha.  Now that is embarrassing :(.   If I fix it then I do see greater
differences
at sizes of 10+.  Below that, PQ0 still looks better on my machine,
which I
still consider weird.  Thanks!
-- Aaron Watters

War dims hope for peace  -- a "real life headline"

-- 
http://mail.python.org/mailman/listinfo/python-list

type() takes one or *three* arguments!?

[jamesthiele . usenet]

I was looking at Simon Burton's Povray.py code (part of pypov) and saw
this line:
globals()[name] = type( name, (KWItem,), {} ) # nifty :)

where 'KWItem' was a class. It did seem nifty, but it was unclear to me
what was happening.

I went to python.org's online documentation which said that type()
takes one argument. So I fired up python:
>>> type(42)

>>> type("x", (type(42),), {})


OK, It appears that type() with 3 arguments constructs a class. Is this
documented somewhere? If not can someone explain what is going on?
james

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: type() takes one or *three* arguments!?

[jamesthiele . usenet]

Thank you - that explains everything quite nicely.

-- 
http://mail.python.org/mailman/listinfo/python-list

print ending with comma

[jamesthiele . usenet]

I recently ran into the issue with 'print' were, as it says on the web
page called "Python Gotchas"
(http://www.ferg.org/projects/python_gotchas.html):

The Python Language Reference Manual says, about the print statement,

A "\n" character is written at the end, unless the print statement ends
with a comma.

What it doesn't say is that if the print statement does end with a
comma, a trailing space is printed.
--
But this isn't exactly correct either. If you run this program:
import sys
print '+',
print '-',
sys.stdout.write('=')
print
--
the output is:
+ -=
Note that there is no space after the '-'. (Tested on Win 2000 python
2.3.4, OS X 10.3.9 python 2.3 & 2.4)

I know that this is not a massively important issue, but can someone
explain what's going on?

-- 
http://mail.python.org/mailman/listinfo/python-list

loops -> list/generator comprehensions

[jamesthiele . usenet]

I wrote this little piece of code to get a list of relative paths of
all files in or below the current directory (*NIX):

walkList  = [(x[0], x[2]) for x in os.walk(".")]
filenames = []
for dir, files in walkList:
filenames.extend(["/".join([dir, f]) for f in files])

It works fine, I don't need to change it, but I know there is a one
liner list/generator comprehension to do this - I'm just not well
enough versed in comprehensions to figure it out. Can someone please
show me what it is?

Even better, is there a generalized way to transform simple loops into
comprehensions that someone can point me to?

james

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: loops -> list/generator comprehensions

[jamesthiele . usenet]

> HTH, 

It does. Thanks.

-- 
http://mail.python.org/mailman/listinfo/python-list

Mapping operator tokens to special methods

[jamesthiele . usenet]

I was starting to write a dictionary to map operator strings to their
equivalent special methods such as:
{
  '+' : 'add',
  '&' : 'and_'
}

The idea is to build a simple interactive calculator.

and was wondering if there is already something like this builtin?

Or is there a better way to do what I want?

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Mapping operator tokens to special methods

[jamesthiele . usenet]

John Machin wrote:
>>> eval('1+2') 

3 
--
Yeah, that's what I decided to do.

-- 
http://mail.python.org/mailman/listinfo/python-list

ANN: xsdbXML python release with C#/.NET port

[aaronwmail-usenet]

ANN: xsdbXML release with C#/.NET port

Part I: Announcement

There is a new release of xsdbXML which provides
bugfixes to the Python implementation and also
provides a completely separate implementation in C#/.NET.

The xsdb framework provides a flexible and well defined
infrastructure to allow tabular data to be published,
retrieved, and combined over the Internet.

Read more and download at
   http://xsdb.sourceforge.net

Part II: Discussion
===
The C# implementation is primarily a transliteration
of the Python implementation.  I hope to also do a java
transliteration of the C# code.  Some brief observations:

- Development:
Using python for the first implementation was the
right way to go because it made it easier to backtrack
and refactor the implementation.  If I had tried this
in C# backtracking would have been much more painful.
Testing Python code, especially during development, is also
much easier in part because you can write
{"a": [1,2,3], "b": MyClass("argument")}
in a few keystrokes whereas a C# analogue runs to many
lines.
Interestingly, however, the C# port exposed several
nasty bugs in untested code paths of the python
implementation (incorrect calling sequences) because of C#
static typing, even before the C# code was running or
even compiling.
For the most part the transliteration was
straightforward, except where the python implementation
leans on the python expression parser (the parser had
to be "hand rolled" in C#).

- Libraries:

The python http services are more "bare bones" than the
.NET libraries and this is a *GOOD* *THING* (tm).  The
.NET libraries seem to do all kinds of nice things under
the surface where I can't influence their behavior (even
in principal because of the type system information hiding).
The python implementation pretty much allows me to
modify any part of the implementation, and the default does
as little as possible.  My reservations about the C# http
services didn't actually cause problems in this case, but
it made me nervous.

On the other hand it's nice in C# to not have to guess about
which XML library to use.  As any python programmer can
attest it is not fun trying to decide which of several
implementations is the best and will stay alive, etcetera.
I just arbitrarily chose the one I knew already, but I
really should switch to something else, dunno what.
In C# there is only one choice.

- Environment

What can I say. VS/.NET is really nice and there is
nothing close in Python AFAIK.

- Conclusion

Don't have any.  I'm just making comments :).

  Thanks for your attention -- Aaron Watters

===
% if I had a ( for every $ bush spent how many ('s would I have?
too many ('s

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: ANN: xsdbXML python release with C#/.NET port

[aaronwmail-usenet]

Yikes...  A couple people pointed out that the upload had no
csharp code.  That was because sourceforge was uploading the
wrong file (but reporting the right filesize).  I think it's fixed now
(uploaded from paris and minnesota).  Sorry!!!
--- Aaron Watters

-- 
http://mail.python.org/mailman/listinfo/python-list

Catching all methods before they execute

[jamesthiele . usenet]

I have run into some cases where I would like to run a class method
anytime any class method is invoked.
That is, if I write
x.foo

then it will be the same as writing
x.bar
x.foo

for any method in class x (with the possible exception of 'bar').

The first few times I wanted to print out a data structure for
debugging purposes. Most recently it was to save a data structure to
allow "undo" operations. I've realized that this is a generalized
problem and I was hoping that someone could point me to the correct
hook to accomplish this (python seems to have a hook for everything).

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Catching all methods before they execute

[jamesthiele . usenet]

>[EMAIL PROTECTED] wrote:
>> I have run into some cases where I would like to run a class method
>> anytime any class method is invoked.
>
>Perhaps you want __getattribute__ on a new-style class?
>--
>Michael Hoffman

Perhaps I do. The docs say that __getattribute__ is called on all
attribute references, so I tried to make an undoable list as follows:
% cat Undoable.py
class Undoable(object):
def __init__(self, superclass):
self.superclass = superclass
print "__init__"

def __getattribute__(self, name):
print "__getattribute__"
self.SaveState(self)
self.superclass.__getattribute__(self, name)

def SaveState(self):
print "SaveState"

def RestoreState(self):
pass

l = Undoable(list)
l = [1, 2, 3]
print l.count(1)
% python Undoable.py
__init__
1

It appears that __init__ in Undoable is called, count() in list is
called, but not __getattribute__ or SaveState in Undoable.

What don't I understand?

-- 
http://mail.python.org/mailman/listinfo/python-list

ANN: xsdb does XML, SQL is dead as disco :)

[aaronwmail-usenet]

 
The xsdbXML framework provides a flexible and well defined infrastructure to allow tabular data to be published, retrieved, and combined over the Internet.
It's a little bit like the daughter of the Gadfly SQL engine in the buff, on steroids.  This is a major departure from the previous releases of xsdb.
 
Please read about it and download it from    http://xsdb.sourceforge.net
Note that the download is over 90% documentation and example data files --the software itself is small.
 
The xsdb framework makes all of the following assertions true. 
 
Database queries over web distributed data:   Databases may be broken up into multiple files or servers on multiple machines   and queried as a single resource. Simple Publication:   Publishing a queriable collection of data (a context) can be as simple as   placing an XML document on a web server. Sophisticated Publication:   Large and complex databases may also be published using server software   which provides indexing and other optimizations. Heterogeneity:   Published data collections may be built using parts of remotely defined data collections. External Data:   A data context may make reference to another arbitrary web object. Open formats and definitions:   Databases may be constructed and queried using standard formats and standard   web protocols using any programming language in any computational environment. Simple formats   The content!
  of a
 database or query may be expressed in a manner which is easy   to parse and interpret (both for human readers and for computer programs). Data,   queries and query responses are represented using the same language of expressions. Clear definition   The meaning of database entries and queries are defined using simple mathematical   definitions. 
thanks for your attention!
    -- Aaron Watters
   There ain't no sanity clause.
 
 -- 
http://mail.python.org/mailman/listinfo/python-list

RE: xsdb does XML, SQL is dead as disco :) (oops)

[aaronwmail-usenet]

Some people pointed out that bighunks of my HUGE
ZIP file contained junk that could be regenerated.
Thanks!  It's now much smaller.  Sorry for the
screw up.  -- Aaron Watters

I wrote:
>  xsdb does XML, SQL is dead as disco :)
>
>The xsdbXML framework provides a
>flexible and well defined infrastructure to
>allow tabular data to be published, retrieved,
>and combined over the Internet.
>
>It's a little bit like the daughter of the
>Gadfly SQL engine in the buff,
>on steroids.  This is a major departure from
>the previous releases of xsdb.
>
>Please read about it and download it from
>http://xsdb.sourceforge.net
>
>Note that the download is over 90%
>documentation and example data files --
>the software itself is small.
>
>The xsdb framework makes all of
>the following assertions true.
>
>Database queries over web distributed data:
>  Databases may be broken up into multiple
>  files or servers on multiple machines
>  and queried as a single resource.
>Simple Publication:
>  Publishing a queriable collection of data
>  (a context) can be as simple as
>  placing an XML document on a web server.
>Sophisticated Publication:
>  Large and complex databases may
>  also be published using server software
>  which provides indexing and other optimizations.
>Heterogeneity:
>  Published data collections may be built
>  using parts of remotely defined data collections.
>External Data:
>  A data context may make reference
>  to another arbitrary web object.
>Open formats and definitions:
>  Databases may be constructed and
>  queried using standard formats and standard
>  web protocols using any programming language
>  in any computational environment.
>Simple formats
>  The content of a database or query may be
>  expressed in a manner which is easy
>  to parse and interpret (both for human readers
>  and for computer programs). Data,
>  queries and query responses are represented
>  using the same language of expressions.
>Clear definition
>  The meaning of database entries and queries
>  are defined using simple mathematical
>  definitions. 
 >
 > 
 > There ain't no sanity clause

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: mathmatical expressions evaluation

[aaronwmail-usenet]

> have a task of evaluating a complex series (sorta) of mathematical
> expressions and getting an answer ...

If we assume that you are looking for functionality and speed is
secondary,
please have a look at the technique in


http://cvs.sourceforge.net/viewcvs.py/xsdb/xsdbXML/xsdbXMLpy/functions.py?view=markup

which uses the python parser to generate a parse tree for an arbitrary
expression and then imposes its own semantics on the tree.  In fact
take
a look at xsdb use guide under "Computing other derived values"

http://xsdb.sourceforge.net/guide.html

since xsdbXML implements general computations over xml inputs.
Let me know if you have any comments/questions/suggestions.
-- Aaron Watters

===
Later on we'll perspire, as we stare at the fire
And face so afraid, the bills left unpaid
Walking in a winter wonderland.  --stolen from "for better or worse"

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Text-to-speech

[jamesthiele . usenet]

On some flavors of Windows you can use:
import pyTTS

tts = pyTTS.Create()
tts.Speak('This is the sound of my voice.')

On Mac OS X you can use:
import os

os.system("say 'This is the sound of my voice.'")

You could write a wrapper that takes a string and checks to see which
OS you are on and executes the appropriate of the two above choices.
But you probably need something slicker, and I don't know what that
might be.

-- 
http://mail.python.org/mailman/listinfo/python-list

ANNOUNCE: xsdb release with N/A support and more

[aaronwmail-usenet]

The new xsdbXML_cs_java_py_01 release adds a
"not applicable" attribute restriction and
completes the same/ifknown/otherwise implementations
as well as some bugfixes including a fix for
a performance bug in the java implementation.

The xsdb framework provides a flexible and well defined
infrastructure to allow tabular data to be published,
retrieved, and combined over the Internet.

There are three separate implementations: a Python
implementation, a C#/.NET implementation, and a
java implementation. For all implementations all
source code is distributed using SourceForge.

Read more and download at
http://xsdb.sourceforge.net

Thanks,  -- Aaron Watters

===
KIDS MAKE NUTRITIOUS SNACKS
  -- a "real life headline"
  http://www.anvari.org/shortjoke/Newspaper_Headlines

-- 
http://mail.python.org/mailman/listinfo/python-list

Using ssl module over custom sockets

[jbauer . usenet]

Hi group,

is it possible to use the ssl module using a custom transport? It appears to me 
as if currently the relationship between ssl.SSLSocket() and socket.socket() is 
pretty entangled.

Suppose I do have some kind of reliable transport (let's say RS232) and a 
connection that I have wrapped in a class. The things I can provide are 
send/recv methods, but nothing specific to UNIX sockets (getsockopt, etc). 
Could I use the Python ssl module to perform a SSL connection over such a line?

Something that I could always use as a workaround would be to open up a 
listening port locally in one thread and connection to that local port in a 
different thread, then forward packets. But that's pretty ugly and I'd like to 
avoid it.

Any hints appreciated,
Best regards,
Johannes
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Need a compelling argument to use Django instead of Rails

[aaronwmail-usenet]

Steve Holden wrote:
...
> I wouldn't waste your time. "A man convinced against his will is of the
> same opinion still", and they already know they aren't interested in
> Python. There are probably many other matters about which they are
> uninformed and equally determined

This is too true.  Fortunately progress proceeds
slowly nonetheless.  For example C++ is much
less used these days where it is totally and
catastrophically inappropriate than in years past
-- maybe because the true C++ boneheads have
been promoted or are retiring...  Nevertheless I was
recently told by a new grad that I should translate
one of my (working, no problems) python apps to
C++ because C++ is "a higher level language."
But I'm sure he'll fall in line with the next stream
of lemmings rather than sink into the tarpit with the
dinosaurs.

   -- Aaron Watters

===
Paradigms shift when the old guys retire.  -- Kuhn (?)

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Need a compelling argument to use Django instead of Rails

[aaronwmail-usenet]

Jaroslaw Zabiello wrote:
> On Wed, 26 Jul 2006 16:25:48 +0200, Bruno Desthuilliers wrote:
>
> > I have difficulty imagining how a language could be more dynamic than
> > Python...
>
> E.g. try to extends or redefine builtin Python classes on fly. Ruby is so
> flexible that it can be used to create Domain-specific Programming
> Languages.

This, of course, is really cool if you are working
all by yourself on a dissertation or something,
but can be completely disasterous if you are
actually working with other people who need to
know what the expressions of the programming
language mean and do.  Back in the day every
good Common Lisp programmer wrote in a
dialect completely incomprehensible to any other
Common Lisp programmer.  Kinda fun, but not
"best practice."
   -- Aaron Watters

===

Writing about music is like
dancing about architecture.
   -- Elvis Costello

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Need a compelling argument to use Django instead of Rails

[aaronwmail-usenet]

Paul Rubin wrote:
> I didn't realize you could do shared hosting with mod_python, because
> of the lack of security barriers between Python objects (i.e. someone
> else's application could reach into yours).  You really need a
> separate interpreter per user.  A typical shared hosting place might
> support 1000's of users with ONE apache/php instance (running in a
> whole bunch of threads or processes, to be sure).

You just need to run multiple apache
instances, which is advisable anyway.
The hosting service formerly known as
python-hosting has been doing this
for years.  For example my http://www.xfeedme.com
site uses it (which seems to be down at the moment!
oh no! :( ).
   -- Aaron Watters

ps: little known fact: mod_python descends
from an NSAPI plugin implementation
I wrote as an example of embedding python in
the defunct "Internet Programming
with Python" book.  Books die, but software lives on.

===
if you are happy and you know it stick with your dosage.
   -- seen in the New Yorker

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Need a compelling argument to use Django instead of Rails

[aaronwmail-usenet]

Ben Sizer wrote:
> [EMAIL PROTECTED] wrote:
> > Paul Rubin wrote:
> > > A typical shared hosting place might
> > > support 1000's of users with ONE apache/php instance (running in a
> > > whole bunch of threads or processes, to be sure).
> >
> > You just need to run multiple apache
> > instances, which is advisable anyway.
> > The hosting service formerly known as
> > python-hosting has been doing this
> > for years.
>
> Would you need one instance per user? Is it practical to run 1000s of
> Apache instances on one server?
>

I'm almost certain Apache spawns instances as needed.
If they are all active at the same time you will need at least
that many threads anyway and I don't think processes
are really much more expensive than threads usually.
But I'm not an expert on virtual hosting or apache or
even thread/process internals.

However when I do a "ps -aef" on my shared server
(http://www.xfeedme.com) I only see the apache
instances that are active, and not the 50 dormant
ones, if I recall.

   -- Aaron Watters

===
She was flirty, dirty, musta been about thirty
   -- 70's stones lyrics
She was nifty, shifty musta been about fifty
   -- 90's stones lyrics

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Need a compelling argument to use Django instead of Rails

[aaronwmail-usenet]

Damjan wrote:
> Yes, but your mod_python programs still run with the privileges of the
> Apache process, as are all the other mod_python programs. This means that
> my mod_python program can (at least) read files belonging to you -
> including your config file holding your database password

I think a standard solution to this is to
associate each virtual host server to a
different port and have the main apache
redirect to the port.  Inetd makes sure
that the vserver apache instance only
stays alive while it's needed.  It might be
complicated to set up, but it works.
Again, something like this is probably
advisable anyway to limit the ways one
vserver can damage another generally
speaking.
  -- Aaron Watters

===
It's not the years. It's the mileage.
   -- Indiana Jones

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Need a compelling argument to use Django instead of Rails

[aaronwmail-usenet]

Damjan wrote:>
> Starting a new Apache process with python included (trough mod_python) is
> even worse than CGI.

Yes, but I think only for the first interaction
after being dormant for a period. In fact I've
noticed that hitting http://www.xfeedme.com
the first time is usually slow.  But once the
apache is up it seems to stay up until it has
been inactive for a good while, and it's fast.
I'm inferring all this from what I see using
"ps" and other indirect tools.

  -- Aaron Watters

===
as the poet said:
 "Everybody have fun tonight
  Everybody Wang Chung tonight"

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: sum fonction in gadfly

[aaronwmail-usenet]

Robert Hicks wrote:
> I haven't been keeping up. Is Gadfly still in development?

I always find this question a little
irritating -- gadfly is perfect the
way it is :).  If it ain't broke don't
fix it.  At least until the python guys
make another non-backwards-compatible
change that makes a patch necessary
(speaking of things that irritate me...).

Seriously, there are a few problems
that could be fixed that I can
think of, but mostly it works for what
it is -- why are people so concerned
that it's not changing?

   -- Aaron Watters
 http://gadfly.sourceforge.net/ --perfect already
 http://xsdb.sourceforge.net/  --getting there

===
You'd be paranoid too if everyone was out to get you.

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: sum fonction in gadfly

[aaronwmail-usenet]

Robert Hicks wrote:
> [EMAIL PROTECTED] wrote:
> >  why are people so concerned
> > that it's not changing?
> >
>
> I didn't mean to be irritating and I wasn't concerned about it not
> changing but I could probably have stated the question a little better.
> For some reason I thought it was a dead project so maybe "still being
> maintained" would be a better statement. I dunno.

You're not the only one.  Why do I keep seeing "gadfly...dead"
in the same sentence?

What if it doesn't need maintenance?  Does that make it dead?
No offense to you personally, you are just repeating what
everyone else has been saying for years for reasons that
totally escape me.  Several times people have suggested that
gadfly be added to the python standard library and then the
question comes up... who will maintain it?  And I answer I'll
maintain it if anyone finds serious problems with it and then
a few months later I hear that it was decided that gadfly was
a dead project.  This has been going on since about '97.
It's irritating and tiresome.  Sorry, I'm grumpy today.

   -- Aaron Watters

===
my mazarati goes 185
I lost my license
now I don't drive   -- from "Life's been good"

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Gadfly server startup error

[aaronwmail-usenet]

> Is anybody out there who has used the server+client operation
> mode successfully?

Well, several years ago, yes.

Since then the project was taken over by some volunteers
and they did an excellent job of
restructuring and modernizing (somewhat) the
*standalone* part of gadfly, but apparently they
didn't port the *client/server* component properly.
As it stands the client server portion is broken now.
(I hadn't tried it after the port, sorry).

I think if you find an older (pre-sourceforge)
tarball it will work.  I will try
to find time to fix this, unless someone else wants
to have a try.  Sorry!

Also, the reference to the new release on the
site refers to a mini-release I did a year or more
ago which fixed various bugs and also added
an xsdb interface component.

   -- Aaron Watters

===
There once was a man from Japan
whose limericks never would scan
when told this was so
he said "yes, I know,
but I always try to put as many words in the last line as I possibly
can."

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Gadfly server startup error

[aaronwmail-usenet]

[EMAIL PROTECTED] wrote:
> > Is anybody out there who has used the server+client operation
> > mode successfully?
>
> Well, several years ago, yes.

I looked into it and it was mainly a documentation and
test issue, I think.  The code seems to work.

Please go

  http://gadfly.sourceforge.net/gadfly_server_test.zip

and look at the readme.  I will try to fold this in to the
distribution sometime soon.

from the README in the zip:

==
gadfly_server_test readme

This directory provides a proof of concept test run of the
gadfly server components. It requires gadfly to be installed
in the local python installation ( http://gadfly.sourceforge.net ).

Please see the doc string at the top of gfstest.py for details.

Example demo run sequence (in Windows installations the "python" prefix
is optional):

1. Set up the database.
 % python gftest.py dbtest

2. Start the server
 % python gfstest.py start

   This starts an infinite server loop in the current console.

3. SWITCH TO ANOTHER WINDOW and run test queries.
 % python gfstest.py queries

4. Shut down the server.
 % python gfstest.py shutdown
=

   -- Aaron Watters

===
There once was a man who said "Well!
Will nobody answer this bell?
I have pulled day and night
'til my hair has grown white
but nobody answers this bell!"
  -- Edward Lear

-- 
http://mail.python.org/mailman/listinfo/python-list

ANN: Skimpy Gimpy ASCII Web Challenge CAPTCHA

[aaronwmail-usenet]

Please check it out and try it:

   http://skimpygimpy.sourceforge.net/

Find examples, documentation, links to demos
and download links there.

Skimpy Gimpy is a tool for generating HTML
representations for strings which
people can read but which web
robots and other computer programs
will have difficulty understanding.
Skimpy is an example of a  Captcha:
an acronym for "Completely Automated  Public
Turing test to tell Computers and Humans Apart".

The Skimpy program and API
(applications programmer interface)
is implemented in a single
self contained Python script (module).  The
input for Skimpy are words or phrases.
The output of the program and API are strings containing
HTML preformatted text.  The preformatted
text contains "ASCII art" representing the input
phrase, which looks like somewhat sloppy handwriting
on a speckled page, when viewed in an HTML browser.

It is intended that it is easy for a human to
read the word or phrase when rendered
as HTML, but it is difficult for a program to
extract the word or phrase automatically.
The program uses a number of techniques
to make the output difficult for a computer to
interpret: curve interpolation, random rotation,
random skew, random scale adjustment,
"smearing", and addition of noise.

The Skimpy tool is far easier to install, use,
and embed than other similar technologies.
All you need is python installed and the
skimpyGimpy.py source file.

Enjoy and happy holidays!  -- Aaron Watters
===
there ain't no sanity clause!

-- 
http://mail.python.org/mailman/listinfo/python-list

ANN: Skimpy CAPTCHA adds WAVE audio, and a problem

[aaronwmail-usenet]

SKIMPY CAPTCHA ADDS AUDIO, AND A PROBLEM

[or what I did over xmas weekend at the inlaws
-- python/web/audio experts skip to the bottom
and solve my problem please.]

Skimpy Gimpy CAPTCHA now supports WAVE audio
output to help people with visual impairments
answer Skimpy challenges.

Read more, try it out, download it here:

http://skimpygimpy.sourceforge.net

Skimpy is a tool for generating HTML visual
and WAVE audio representations for strings
which people can understand but which web robots
and other computer programs will have
difficulty understanding. Skimpy is an example
of a Captcha: an acronym for "Completely Automated
Public Turing test to tell Computers and Humans
Apart".

The visual HTML Skimpy program skimpyGimpy.py
and API (applications programmer interface) is
implemented in a single self contained Python
script (module). The input for Skimpy are words
or phrases. The output of the program and API
are strings containing HTML preformatted text.
The preformatted text contains "ASCII art" representing
the input phrase, which looks like somewhat
sloppy handwriting on a speckled page,
when viewed in an HTML browser.

It is intended that it is easy for a human to read
the word or phrase when rendered as HTML, but it is
difficult for a program to extract the word or phrase
automatically. The program uses a number of
techniques to make the output difficult for a
computer to interpret: curve interpolation, random
rotation, random skew, random scale adjustment,
"smearing", and addition of noise.

In order to allow CAPTCHA tests that are usable by
people with visual empairment, Skimpy also provides
an audio implementation. The audio WAVE Skimpy
program waveTools.py uses a compiled audio sample
file waveIndex.zip. The input of the program are words
or phrases and the output are the words or phrases
spelled as individual spoken characters in an
audio stream.

It is intended that a human can understand the
audio stream but a computer program will not be
able to analyse the stream and extract the
letters. To make the audio stream more difficult
to automatically analyse (without making it
unintelligible) the program randomly overlaps and
stretches/shrinks the input samples, among other
things.

The Skimpy tools are far easier to install, use,
and embed than other similar technologies.

THE PROBLEM

Unfortunately there is a problem with using Firefox
and Quicktime with the Skimpy audio.  If you save the
audio to a temporary file and stream from there
everything works fine, but if you try to stream directly
from the CGI or mod-python module via HTTP Quicktime
TRUNCATES THE AUDIO TO ABOUT 3 SECONDS.  This ONLY
happens under Firefox afaik -- IE, for example has
no problems.  Am I doing something wrong?  Please inform.
The skimpygimpy1.1 download has all the relevant source
code if you'd like to have a look. Help!  Thanks
in advance.

I'm leaving the bug demonstrated in my sample web scripts
for now, but if no one comes up with a fix I will hack around
it in a few days... probably...
  -- Aaron Watters

===
Later on, we'll perspire
as we stare at the fire
and face so afraid
the bills left unpaid
walking in a winter wonderland
 -- seen in "For Better or Worse"

-- 
http://mail.python.org/mailman/listinfo/python-list

AJAX example code available

[aaronwmail-usenet]

I've published sample code that uses Python
on the server side to implement AJAX
type ahead completion for web forms.  Please see
documentation with links to examples and downloads at

http://xsdb.sourceforge.net/xFeed.html

"Type ahead completion" is a form of AJAX
(asyncronous javascript with XML) functionality
where the page attempts to suggest completions
for partial form values by consulting a
web server component in the background.  Google Suggest
is a famous example.  For examples using
my python code please see

http://www.xfeedme.com/zip/zipcompletions_async.html or
http://www.xfeedme.com/xsdbXML/xFeed/BookMatch.html.

These pages must be viewed with a recent browser such
as FireFox 1.5, Safari 2.0, or IE 6.0.

The AJAX server component is configured for use
either as a Python CGI script or as a Apache server mod-python
module.

I hope you enjoy it and use it well :).
Please let me know if you have problems with it
or other comments.

  Thanks!  -- Aaron Watters

===
Chemistry may lead to a better life,
but Biology is more fun.  -- seen on a teeshirt

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Edit Audio Using Python?

[aaronwmail-usenet]

On Jun 20, 12:21 pm, Ultrus <[EMAIL PROTECTED]> wrote:
> Ah! I found this on the official 
> website:http://www.python.org/doc/1.5.2p2/lib/module-audioop.html
>
> That should keep me occupied. If you think of anything interesting
> however, I would be happy to know. :)

I think you'll find that you have to put a lot of pieces together
to manipulate audio -- it's all there, but it's not straightforward.

I did it in my "skimpygimpy" audio components, which may help
(especially
if you ignore the stranger parts where I went off the deep end).

Start by looking at the "waveTools.toneFile()" method, which, I think,
writes out a wave file containing "concert A".

http://skimpygimpy.sourceforge.net -- follow link to downloads...

  -- Aaron Watters

===
an apple every 8 hours
will keep 3 doctors away.   -- kliban



-- 
http://mail.python.org/mailman/listinfo/python-list

SkimpyGimpy PNG canvas w/ Javascript mouse tracking

[aaronwmail-usenet]

ANN: SkimpyGimpy PNG canvas has Javascript mouse tracking

The SkimpyGimpy PNG image canvas now can generate
Javascript data structures which allow HTML pages
to intelligently respond to mouse events over the
image.

Please read about the SkimpyGimpy Canvas and look at
the mouse tracking example here:

http://skimpygimpy.sourceforge.net/canvas.html

The SkimpyGimpy main page is here:

http://skimpygimpy.sourceforge.net/

BACKGROUND:

SkimpyGimpy is a collection of tools for generating
HTML visual, PNG image, and WAVE audio components
for use in web based applications including CAPTCHA
implementations (Completely Automated Public Turing
test to tell Computers and Humans Apart) and PNG
image creation tools with Javascript mouse tracking
support.

  I hope you like.  -- Aaron Watters

===

Sometimes say sometimes.

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: SkimpyGimpy PNG canvas w/ Javascript mouse tracking

[aaronwmail-usenet]

re: http://skimpygimpy.sourceforge.net

On May 7, 7:29 pm, James Stroud <[EMAIL PROTECTED]> asks:
> Can you advertise "CAPTCHA" as it is trademarked by Carnegie Mellon?
>
> James

I can easily forward them a generous portion of my earnings
from this project if needed :).

Actually I think the term is in common use like "Kleenex".
A quick glance at http://www.lanapsoft.com/products.html
shows no mention of CMU.

  -- Aaron Watters

===
sometimes if something doesn't seem to make sense
it's because it makes no sense



-- 
http://mail.python.org/mailman/listinfo/python-list

Re: searching algorithm

[aaronwmail-usenet]

On May 10, 1:26 pm, Gigs_ <[EMAIL PROTECTED]> wrote:
> Hi all!
>
> I have text file (english-croatian dictionary) with words in it in 
> alphabetical
> order.
> This file contains 17 words in this format:
> english word: croatian word

Let's assume it's okay to have all the data in memory.
In my experience the very fastest way to do what you
want is to store the strings in a sorted list and use
the binary search library module bisect.  I once compared this
with doing something similar with tries and it was
much faster.  It's also the most simple way to do it, which
is nice too :).
  -- Aaron Watters

===
never eat anything bigger than your head -- kliban


-- 
http://mail.python.org/mailman/listinfo/python-list

ANN: PNG image CAPTCHA with PNG canvas available (SkimpyGimpy)

[aaronwmail-usenet]

Announcing SkimpyGimpy Support for PNG image
CAPTCHA generation and PNG canvases.

You can now use SkimpyGimpy to generate
CAPTCHA text representations as PNG image files
in addition to preformatted text ASCII art, and
WAVE format audio streams, either from command
lines or within Python programs.

Also general support for drawing text and
graphics as PNG image files is available via a
"PNG canvas" drawing interface.

Enjoy!

Skimpy Main Page:
  http://skimpygimpy.sourceforge.net/

PNG Canvas documentation:
  http://skimpygimpy.sourceforge.net/canvas.html

Background Information:

Skimpy is a tool for generating HTML visual,
PNG visual, and WAVE audio representations for
strings which people can understand but which
web robots and other computer programs will
have difficulty understanding. Skimpy is an
example of a Captcha: an acronym for "Completely
Automated Public Turing test to tell Computers
and Humans Apart".

I hope you like!
  -- Aaron Watters

===
"Do you expect me to talk?"
"No, Mr. Bond, I expect you to die."

-- 
http://mail.python.org/mailman/listinfo/python-list

Seattle Python Interest Group Thursday at 7:00 PM

[jamesthiele . usenet]

Seattle Python Interest Group Meeting Thursday, Jan 11th at 7:00 PM

Bar underneath the Third Place Books in Ravenna.
http://www.ravennathirdplace.com/
NE 65th St & 20th Ave NE

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: set partitioning

[aaronwmail-usenet]

Something like this, or am I missing something?

def partition(List, n, m, k):
 if n!=m*k:
 raise "sorry, too many or too few elts"
 D = {}
 for x in List:
  D[x] = 1
 if len(D)!=n:
 raise "sorry (2) you lied about the number"
 List2 = D.keys()
 result = []
 for i in range(m):
 result.append( List2[i*k: i*k+k] )
 return result
?

If this was a take home exam problem,
you should be ashamed of yourself!
   -- Aaron Watters

===

It's easy.  All you have to do is hit
the right keys at the right time and
the organ plays itself.  -- J.S. Bach

-- 
http://mail.python.org/mailman/listinfo/python-list

OT: CAD

[Usenet Loser]

I am completely empty and shallow. I use no CAD package at all now. I
would like to buy one for recreational use, instead of watching American
Idol.

What CAD package has integrated FEA and rigid body calculations so that
I could design a bar stool, and easily determine:

a) if it is top-heavy, so that drunken customers could easily tip it
over, and

b) if a fat drunken customer sat on it, which part would give way first.

Also, are there such things as libraries of materials that I could
specify, which the FEA would take into account, so that a bar stool made
of PVC would bend differently from a bar stool made of tungsten?



Also, what about Python and CAD? 

I've seen http://free-cad.sourceforge.net and pythoncad, but have any
major CAD vendors incorporated Python, sort of the way that ESRI used
Python in it's ArcView GIS product?
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Python's DSLs (was: A critic of Guido's blog on Python's lambda)

[aaronwmail-usenet]

Cameron Laird wrote:
> In article <[EMAIL PROTECTED]>,
> Alex Martelli <[EMAIL PROTECTED]> wrote:
... .
> >Of course, the choice of Python does mean that, when we really truly
> >need a "domain specific little language", we have to implement it as a
> >language in its own right, rather than piggybacking it on top of a
> >general-purpose language as Lisp would no doubt afford; see
> > for such a DSLL developed
> >at Google. However, I think this tradeoff is worthwhile, and, in
> >particular, does not impede scaling
>
> I'm confused, Alex:  I sure
> think *I* have been writing DSLs as specializations of Python,
> and NOT as "a language in its own right"

I think Alex is suggesting that if they used, for example, a version of
scheme
with a good optimizing compiler they could implement sawzall like
convenience
with almost the same performance, including startup, etc. whereas even
a
highly optimized python based approach would at least have a
comparatively
large startup penalty.  For an environment like Google where they
scrape
thru their logs of various sorts doing lots of trivial scans you can
probably save
a lot of money and time on lots of machines by optimizing such scrapes
(but keep your bactine handy).  And as the sawzall paper pointed out,
even static
type checks can prevent a lot of wasted machine bandwidth by avoiding
dumb
errors.

But the real question for someone like Rob Pike is why use scheme when
you
can invent another little language instead, I suspect :).

  -- Aaron Watters

===

Stop procrastinating soon.

-- 
http://mail.python.org/mailman/listinfo/python-list

tracking collection modification

[usenet . tolomea]

I'm working on a remote object system, something kinda like Pyro.
For the purposes of caching I need to be able to tell if a given
dict / list / set has been modified.
Ideally what I'd like is for them to have a modification count
variable that increments every time the particular collection is
modified. Unfortunately I can't find anything like that and since this
needs to work for the regular normal list / dict / set objects
subclassing them to add the modification count isn't useful.
I realize I could take a copy and then compare the copy to the
original, but that's a fairly heavy handed approach and I was hoping
for something light and fast.
Does anyone have any suggestions on best to approach this?
--
http://mail.python.org/mailman/listinfo/python-list

Re: tracking collection modification

[usenet . tolomea]

On Sep 7, 8:54 pm, [EMAIL PROTECTED] wrote:
> I'm working on a remote object system, something kinda like Pyro.
> For the purposes of caching I need to be able to tell if a given
> dict / list / set has been modified.
> Ideally what I'd like is for them to have a modification count
> variable that increments every time the particular collection is
> modified. Unfortunately I can't find anything like that and since this
> needs to work for the regular normal list / dict / set objects
> subclassing them to add the modification count isn't useful.
> I realize I could take a copy and then compare the copy to the
> original, but that's a fairly heavy handed approach and I was hoping
> for something light and fast.
> Does anyone have any suggestions on best to approach this?

additionally I don't need to know if the things the list (etc)
references have changed, only the list itself
--
http://mail.python.org/mailman/listinfo/python-list

recycling internationalized garbage

[aaronwmail-usenet]

Hi folks,

Please help me with international string issues:
I put together an AJAX discography search engine

http://www.xfeedme.com/discs/discography.html

using data from the FreeDB music database

http://www.freedb.org/

Unfortunately FreeDB has a lot of junk in it, including
randomly mixed character encodings for international
strings.  As an expediency I decided to just delete all
characters that weren't ascii, so I could get the thing
running.  Now I look through the log files and notice that
a certain category of user immediatly homes in on this
and finds it amusing to see how badly I've mangled
the strings :(.  I presume they chuckle and make
disparaging remarks about "united states of ascii"
and then leave never to return.

Question: what is a good strategy for taking an 8bit
string of unknown encoding and recovering the largest
amount of reasonable information from it (translated to
utf8 if needed)?  The string might be in any of the
myriad encodings that predate unicode.  Has anyone
done this in Python already?  The output must be clean
utf8 suitable for arbitrary xml parsers.

Thanks,  -- Aaron Watters

===

As someone once remarked to Schubert
"take me to your leider" (sorry about that).
   -- Tom Lehrer

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: recycling internationalized garbage

[aaronwmail-usenet]

Regarding cleaning of mixed string encodings in
the discography search engine

http://www.xfeedme.com/discs/discography.html

Following 's suggestion I came up with this:

utf8enc = codecs.getencoder("utf8")
utf8dec = codecs.getdecoder("utf8")
iso88591dec = codecs.getdecoder("iso-8859-1")

def checkEncoding(s):
try:
(uni, dummy) = utf8dec(s)
except:
(uni, dummy) = iso88591dec(s, 'ignore')
(out, dummy) = utf8enc(uni)
return out

This works nicely for Nordic stuff like
"björgvin halldórsson - gunnar Þórðarson",
but russian seems to turn into garbage
and I have no idea about chinese.

Unless someone has any other ideas I'm
giving up now.
   -- Aaron Watters

===

In theory, theory is the same as practice.
In practice it's more complicated than that.
  -- folklore

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Tried Ruby (or, "what Python *really* needs" or "perldoc!")

[aaronwmail-usenet]

I agree that more progress is needed on the Python documentation
front.  For example if you look at the "codecs" module documentation
there is no hint of what a codec is anywhere that I can see. Also
the distinction between an "encoder" and a "decoder" is not explained.
Even though I've used it many times and understand it, I still find
myself using the interactive interpreter to make sure I'm sending the
bytes in the right direction...

Perhaps some faqwiz/wiki-like tool to allow the broader community to
propose documentation enhancements would be useful?

  -- Aaron Watters

===

"I know what it's like to put food on my family."  G.W.Bush

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Tried Ruby (or, "what Python *really* needs" or "perldoc!")

[aaronwmail-usenet]

hmmm.  Interesting about the wiki.
It's unusable in my version of IE.  Javascript error
on almost every keystroke :(!

http://wiki.python.org/moin/

It works in Firefox, which I have, of course, but
still...

And the patch procedure you described requires
a higher degree of motivation (and free time) than
most potential contributors might have on offer, imo.

   -- Aaron Watters

===
"I weep for you," the walrus said, I deeply sympathize.
With sobs and tears he sorted out those of the largest size.
Holding his pocket handkercheif before his streaming eyes.
   from "The Walrus and the Carpenter", Lewis Carroll

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Using Classes

[Alf P. Steinbach /Usenet]

* Mag Gam, on 24.06.2010 13:58:

I have been using python for about 1 year now and I really like the
language. Obviously there was a learning curve but I have a programing
background which made it an easy transition. I picked up some good
habits such as automatic code indenting :-), and making my programs
more modular by having functions.

I know that Python is very OOP friendly, but I could not figure out
why and when to use Classes in it. I mostly use it for simple text
parsing  I suppose when a program gets "complicated" I should start
using Classes. Are there any tips or tricks people use to "force" them
into the OOP mentality? I would like to force myself to learn the
Python way but so far I could not figure out WHY I would need a class
for this...


Use whatever paradigm that yields code that you easily understand. 

Having said that, the main use of a class is to model some data with an 
associated set of operations.


For this a class is merely a convenience, but sometimes the convenience can be 
so great that any other approach would be impractical in comparision.


In some cases a non-class approach ends up with a number of routines foo, bar, 
zlopf etc. that all take some argument of a "general" type and all internally 
has some "if it's really a BRACHYKLURB, do A, assuming that it has 
BRACHYKLURB-specific data, but if it's really a KNETCHOFICHUS, then do B, 
assuming that it has KNETCHOFICHUS-specific data, and as default, if it's none 
of those, do C".


This is a maintainance nightmare. You have to be sure that every such routine 
discriminates correctly on type, checking all possibilities and doing the right 
thing for each possibility. And you have to be sure of that in the face of 
possible additions of new subtypes, or removal of a subtype (maintainance).


Then the answer is to introduce some classes and OOP stuff.

In essence, instead of letting general routines awkwardly and error-prone choose 
the right type-specific routines depending on the object, let the object specify 
the right type-specific routines directly. :-)



Cheers & hth.,

- Alf

--
blog at http://alfps.wordpress.com>
--
http://mail.python.org/mailman/listinfo/python-list

Re: Decorators, with optional arguments

[Alf P. Steinbach /Usenet]

* Stephen Hansen, on 02.07.2010 19:41:

Okay, so!

I actually never quite got around to learning to do deep and useful
magic with decorators. I've only ever done the most basic things with
them. Its all been a little fuzzy in my head: things like what order
decorators end up being called in if there's more then one, etc.

But in my current situation, what I'm wanting to do is have a decorator
that wraps a function but which takes an *optional* argument, and sets
that argument as an attribute on said function if its there.

Here's what some tweaking and playing around has gotten me, as a recipe:

 import functools

 def my_decorator(arg):
 if callable(arg): # Stuck on 2.5. Leavemealone. :)
 protocol = None
 else:
 protocol = arg

 def wrap(fn):
 print "Wrapping."
 fn.protocol = protocol

 @functools.wraps(fn)
 def wrapper(*args, **kwargs):
 print "Calling."
 result = fn(*args, **kwargs)
 print "Called."
 return result

 return wrapper

 if not protocol: # argument-less decorator
 print "Calling wrap."
 return wrap(arg)
 else:
 print "Returning wrap."
 return wrap

To be used as:

 class Thing(object):
 @expose
 def test1(self, arg1):
 return arg1

 @expose("testing")
 def test2(self, arg2):
 return arg2

So, my question: am I doing this right? :) Some play-through testing
appears to work. But, the dizzying array of nested def's up there leaves
me a bit dazed, so I'm wondering if there's a simpler way to accomplish
what I'm trying to do.


If you're willing to have slightly more explicit usage code, consider e.g.



#Py3

import functools

class expose:
def __init__( self, protocol = None ):
self._protocol = protocol

def __call__( self, f ):
print( "Wrapping." )
f.protocol = self._protocol

@functools.wraps( f )
def wrapper( *args, **kwargs ):
print( "Calling." )
result = f( *args, **kwargs )
print( "Called." )
return result

return wrapper

class Thing(object):
@expose()
def test1(self, arg1):
return arg1

@expose( "testing" )
def test2(self, arg2):
return arg2

o = Thing()
print( o.test1( 1.11 ) )
print( o.test2( 2.22 ) )



Cheers & hth.,

- Alf


--
blog at http://alfps.wordpress.com>
--
http://mail.python.org/mailman/listinfo/python-list

Re: The real problem with Python 3 - no business case for conversion

[Alf P. Steinbach /Usenet]

* Steven D'Aprano, on 03.07.2010 16:24:

On Sat, 03 Jul 2010 08:46:57 -0400, D'Arcy J.M. Cain wrote:


On Fri, 02 Jul 2010 22:40:34 -0700
John Nagle  wrote:

  Not according to Vex's published package list:

http://www.vex.net/info/tech/pkglist/


Hold on.  That *is* the generated list and Python 3.1 is on it.  We have
both 2.6 and 3.1.  The 3.1 version is listed right below the 2.6 one.
The page is generated from pkg_info(1) and includes everything we have
installed from FreeBSD ports.


Pfft! Facts! You can prove anything you like with facts!


:-)

--
blog at http://alfps.wordpress.com>
--
http://mail.python.org/mailman/listinfo/python-list

Re: Download Microsoft C/C++ compiler for use with Python 2.6/2.7 ASAP

[Alf P. Steinbach /Usenet]

* sturlamolden, on 06.07.2010 17:50:


Just a little reminder:

Microsoft has withdrawn VS2008 in favor of VS2010. The express version
is also unavailable for download.>:((

We can still get a VC++ 2008 compiler required to build extensions for
the official Python 2.6 and 2.7 binary installers here (Windows 7 SDK
for .NET 3.5 SP1):

http://www.microsoft.com/downloads/details.aspx?familyid=71DEB800-C591-4F97-A900-BEA146E4FAE1&displaylang=en

Download today, before it goes away!

Microsoft has now published a download for Windows 7 SDK for .NET 4.
It has the VC++ 2010 compiler. It can be a matter of days before the VC
++ 2008 compiler is totally unavailable.

It is possible to build C and Fortran extensions for official Python
2.6/2.7 binaries on x86 using mingw. AFAIK, Microsoft's compiler is
required for C++ or amd64 though. (Intel's compiler requires VS2008,
which has now perished.)

Remember Python on Windows will still require VS2008 for a long time.
Just take a look at the recent Python 3 loath threads.


Perhaps this all for the good.

There is no *technical* problem creating a compiler-independent C/C++ language 
binding. I believe that Java's JNI works fine no matter what compiler you use, 
although it's many many years since I've done JNI things. Similarly, Python 
should IMHO just have a well defined compiler independent native code interface, 
e.g. "PNI", or "pynacoin", the PYthon NAtive COde INterface :-)



Cheers,

- Alf

--
blog at http://alfps.wordpress.com>
--
http://mail.python.org/mailman/listinfo/python-list

Argh! Name collision!

[Alf P. Steinbach /Usenet]

Donald Knuth once remarked (I think it was him) that what matters for a program 
is the name, and that he'd come up with a really good name, now all he'd had to 
do was figure out what it should be all about.


And so considering Sturla Molden's recent posting about unavailability of MSVC 
9.0 (aka Visual C++ 2008) for creating Python extensions in Windows, and my 
unimaginative reply proposing names like "pni" and "pynacoin" for a compiler 
independent Python native code interface, suddenly, as if out of thin air, or 
perhaps out of fish pudding, the name "pyni" occurred to me.


"pyni"! Pronounced like "tiny"! Yay!

I sat down and made my first Python extension module, following the tutorial in 
the docs. It worked!


But, wait, perhaps some other extension is already named "piny"?

Google.

http://code.google.com/p/pyni/>, "PyNI is [a] config file reader/writer".

Argh!


- Alf

--
blog at http://alfps.wordpress.com>
--
http://mail.python.org/mailman/listinfo/python-list

Re: Download Microsoft C/C++ compiler for use with Python 2.6/2.7 ASAP

[Alf P. Steinbach /Usenet]

* sturlamolden, on 06.07.2010 19:35:

On 6 Jul, 19:09, Thomas Jollans  wrote:


Okay, you need to be careful with FILE*s. But malloc and free? You'd
normally only alloc&  free something within the same module, using the
same functions (ie not mixing PyMem_Malloc and malloc), would you not?


You have to be sure PyMem_Malloc is not an preprocessor alias for
malloc (I haven't chaecked).


Python 3.1.1, file [pymem.h]:

PyAPI_FUNC(void *) PyMem_Malloc(size_t);

#define PyMem_MALLOC(n) (((n) < 0 || (n) > PY_SSIZE_T_MAX) ? NULL \
: malloc((n) ? (n) : 1))

The problem with the latter that it seems that it's intended for safety but does 
the opposite...


Perhaps (if it isn't intentional) this is a bug of the oversight type, that 
nobody remembered to update the macro?



  ***


Except for the problems with file descriptors I think a practical interim 
solution for extensions implemented in C could be to just link the runtime lib 
statically. For a minimal extension this increased the size from 8 KiB to 49 
KiB. And generally with MS tools the size is acceptably small.


I think that this would be safe because since the C API has to access things in 
the interpreter I think it's a given that all the relevant functions delegate to 
shared library (DLL) implementations, but I have not checked the source code.


As a more longterm solution, perhaps python.org could make available the 
redistributables for various MSVC versions, and then one could introduce some 
scheme for indicating the runtime lib dependencies of any given extension. Then 
when installing an extension the installer (distutils package functionality) 
could just check whether the required runtime is present, and if not give the 
user the choice of automatically downloading from python.org, or perhaps direct 
from Microsoft. This scheme would support dependencies on new runtime lib 
versions not yet conceived when the user's version of Python was installed.



Cheers,

- Alf

--
blog at http://alfps.wordpress.com>
--
http://mail.python.org/mailman/listinfo/python-list

Re: Download Microsoft C/C++ compiler for use with Python 2.6/2.7 ASAP

[Alf P. Steinbach /Usenet]

* Martin v. Loewis, on 07.07.2010 21:10:

Python 3.1.1, file [pymem.h]:

PyAPI_FUNC(void *) PyMem_Malloc(size_t);

#define PyMem_MALLOC(n)(((n)<  0 || (n)>  PY_SSIZE_T_MAX) ? NULL \
 : malloc((n) ? (n) : 1))

The problem with the latter that it seems that it's intended for safety
but does the opposite...


Why do you say that? It certainly *does* achieve safety, wrt. to certain
errors, specifically:
- passing sizes that are out-of-range
- supporting malloc(0) on all systems


It uses malloc instead of PyMem_Malloc. malloc may well be different and use a 
different heap in an extension DLL than in the Python interpreter and other 
extensions. That's one thing that the docs (rightly) warn you about.




Perhaps (if it isn't intentional) this is a bug of the oversight type,
that nobody remembered to update the macro?


Update in what way?


I was guessing that at one time there was no PyMem_Malloc. And that it was 
introduced to fix Windows-specific problems, but inadvertently without updating 
the macro. It's just a guess as to reasons why the macro uses malloc directly.




Except for the problems with file descriptors I think a practical
interim solution for extensions implemented in C could be to just link
the runtime lib statically. For a minimal extension this increased the
size from 8 KiB to 49 KiB. And generally with MS tools the size is
acceptably small.


If you think that's fine for your extension module (which may well be
the case), go ahead.


I have no comment on that except pointing it out as a somewhat stupid, somewhat 
evil social inclusion/exclusion argument, talking to the audience. Argh. You're 
wasting my time. But anyway, 49 KiB is small by today's standards. For example, 
you get 20 of those in a single MiB, and about 20.000 in a single GiB.




But then, you could also just link with a different
DLL version of the CRT instead.


When I wrote "link the runtime lib statically" that was an alternative to the 
usual link-as-DLL.


It wouldn't make sense to link the runtime lib statically as an alternative to 
linking it statically.


As for linking to a different /version/ of the CRT, if you really mean that, I 
think that's difficult. It's not necessarily impossible, after all there's 
STLPort. But I think that it must at the very least be rather difficult to do 
with Microsoft's tools, for otherwise people would have employed that solution 
before, and so I wouldn't trust the result, and wouldn't waste the time trying.



Cheers,

- Alf

--
blog at http://alfps.wordpress.com>
--
http://mail.python.org/mailman/listinfo/python-list

Re: Download Microsoft C/C++ compiler for use with Python 2.6/2.7 ASAP

[Alf P. Steinbach /Usenet]

* sturlamolden, on 07.07.2010 21:12:

On 7 Jul, 06:54, "Alf P. Steinbach /Usenet"  wrote:


PyAPI_FUNC(void *) PyMem_Malloc(size_t);

#define PyMem_MALLOC(n) (((n)<  0 || (n)>  PY_SSIZE_T_MAX) ? NULL \
 : malloc((n) ? (n) : 1))


I was afraid of that :(




Except for the problems with file descriptors I think a practical interim
solution for extensions implemented in C could be to just link the runtime lib
statically.


You still have two CRTs linked into the same process.


So?


Cheers,

- Alf

--
blog at http://alfps.wordpress.com>
--
http://mail.python.org/mailman/listinfo/python-list

Re: Download Microsoft C/C++ compiler for use with Python 2.6/2.7 ASAP

[Alf P. Steinbach /Usenet]

* sturlamolden, on 07.07.2010 21:46:

On 7 Jul, 21:41, "Alf P. Steinbach /Usenet"  wrote:


You still have two CRTs linked into the same process.


So?


CRT resources cannot be shared across CRT borders. That is the
problem. Multiple CRTs are not a problem if CRT resources are never
shared.


Yeah, but then we're down to file descriptors, C library locales and such as the 
remaining problems.


Cheers,

- Alf

--
blog at http://alfps.wordpress.com>
--
http://mail.python.org/mailman/listinfo/python-list

Re: Download Microsoft C/C++ compiler for use with Python 2.6/2.7 ASAP

[Alf P. Steinbach /Usenet]

* Martin v. Loewis, on 07.07.2010 21:56:

Perhaps (if it isn't intentional) this is a bug of the oversight type,
that nobody remembered to update the macro?


Update in what way?


I was guessing that at one time there was no PyMem_Malloc. And that it
was introduced to fix Windows-specific problems, but inadvertently
without updating the macro. It's just a guess as to reasons why the
macro uses malloc directly.


It might indeed be that the function version was introduced specifically
for Windows. However, the macro was left intentionally: both for
backwards compatibility, and for use inside Python itself.


Except for the problems with file descriptors I think a practical
interim solution for extensions implemented in C could be to just link
the runtime lib statically.

[...]


When I wrote "link the runtime lib statically" that was an alternative
to the usual link-as-DLL.


Ok, I lost the thread. When you said "a practical interim solution"
you were talking about what problem? I thought the discussion was
about the need to link with the same DLL version as Python.


The main problem that the required MSVC redistributables are not necessarily 
present on the end user's system.




It wouldn't make sense to link the runtime lib statically as an
alternative to linking it statically.


However, it would surely make sense to link with a different DLL than
the one that Python links with, assuming that would actually work.


As for linking to a different /version/ of the CRT, if you really mean
that, I think that's difficult. It's not necessarily impossible, after
all there's STLPort. But I think that it must at the very least be
rather difficult to do with Microsoft's tools, for otherwise people
would have employed that solution before, and so I wouldn't trust the
result, and wouldn't waste the time trying.


It's actually straight-forward (or used to be, until they came up with
the SxS madness). It was actually the case that people did so
unexpectingly, and it seemed to work fine, except that it crashed when
passing FILE*. Then we started explaining that mixing CRTs is risky.


Oh.

Well then. :-)


Cheers,

- Alf

--
blog at http://alfps.wordpress.com>
--
http://mail.python.org/mailman/listinfo/python-list

Re: Download Microsoft C/C++ compiler for use with Python 2.6/2.7 ASAP

[Alf P. Steinbach /Usenet]

* Christian Heimes, on 07.07.2010 22:47:

The main problem that the required MSVC redistributables are not necessarily
present on the end user's system.


It's not a problem for Python anymore. It took a while to sort all
problems out. Martin and other developers have successfully figured out
how to install the CRT for system wide and local user installations. A
system wide installation installs the CRT in the side by side cache
(WinSxS). A local installation keeps the msvcrt90.dll and the
Microsoft.VC90.CRT.manifest next to the python.exe. Python extensions no
longer embed a manifest so they share the CRT from the python.exe process.

In order to ship a standalone exe you have to keep the CRT next to your
exe. This should work for py2exe binaries as well. At our company we
install our application stack entirely from subversion including Python
2.6.5, Sun JRE and lots of other stuff. This works perfectly fine for us
even for servers without the MSVCRT redistributable.


I think you're talking about a different problem. The CRT installed along with 
CPython works for extensions using the MSVC 9.0 CRT.


However developing an extension with MSVC 10 the extension will use the 10.0 
CRT, which is not necessarily present on the end user's system.


As I see it there are five solutions with different trade-offs:

  A Already having Visual Studio 2008 (MSVC 9.0), or coughing up the
money for an MSDN subscription, or visiting trade shows, so as to
obtain that compiler version.
-> Not an option for everybody.

  B Linking the CRT statically.
-> Increased size, problems with CRT state such as file descriptors.

  C Linking the CRT dynamically and bundling the MSVC redistributables
with the extension.
-> Even more increased size for the download, but smaller total
   footprint for extensions in sum; same CRT state problems.

  D Linking the CRT dynamically and providing an optional download and
install of the redistributables if they're not present. This would
best be done with some support from the Python installation machinery.
-> Small nice size for extensions, still same CRT state problems.

  E As D + a new compiler-independent native code interface that
does not carry dependencies on CRT state such as file descriptors, like JNI.
-> Really huge effort, and cannot be applied until some new Python version.

And I think the clue here is that the CRT state problems can be avoided by 
careful coding.


Hence, for those who cannot do A I think B is a realistic practical option, and 
D would be nice...



Cheers,

- Alf

--
blog at http://alfps.wordpress.com>
--
http://mail.python.org/mailman/listinfo/python-list

Re: Download Microsoft C/C++ compiler for use with Python 2.6/2.7 ASAP

[Alf P. Steinbach /Usenet]

* Alf P. Steinbach /Usenet, on 07.07.2010 23:19:


However developing an extension with MSVC 10 the extension will use the
10.0 CRT, which is not necessarily present on the end user's system.

As I see it there are five solutions with different trade-offs:

A Already having Visual Studio 2008 (MSVC 9.0), or coughing up the
money for an MSDN subscription, or visiting trade shows, so as to
obtain that compiler version.
-> Not an option for everybody.

B Linking the CRT statically.
-> Increased size, problems with CRT state such as file descriptors.

C Linking the CRT dynamically and bundling the MSVC redistributables
with the extension.
-> Even more increased size for the download, but smaller total
footprint for extensions in sum; same CRT state problems.

D Linking the CRT dynamically and providing an optional download and
install of the redistributables if they're not present. This would
best be done with some support from the Python installation machinery.
-> Small nice size for extensions, still same CRT state problems.

E As D + a new compiler-independent native code interface that
does not carry dependencies on CRT state such as file descriptors, like
JNI.
-> Really huge effort, and cannot be applied until some new Python version.

And I think the clue here is that the CRT state problems can be avoided
by careful coding.

Hence, for those who cannot do A I think B is a realistic practical
option, and D would be nice...


Wait...

  F Possibly, as the docs say,

"Developer Studio will throw in a lot of import libraries that you do not really 
need, adding about 100K to your executable. To get rid of them, use the Project 
Settings dialog, Link tab, to specify ignore default libraries. Add the correct 
msvcrtxx.lib to the list of libraries."


Can anyone confirm whether this works in practice with MSVC 10?


Cheers,

- Alf

--
blog at http://alfps.wordpress.com>
--
http://mail.python.org/mailman/listinfo/python-list

Re: Argh! Name collision!

[Alf P. Steinbach /Usenet]

* rantingrick, on 07.07.2010 07:42:

On Jul 6, 9:11 pm, "Alf P. Steinbach /Usenet"  wrote:


"pyni"! Pronounced like "tiny"! Yay!


hmm, how's about an alternate spelling... "pyknee", or "pynee", or
"pynie" ... considering those are not taken either?


Hm, for pure shock value I think I'll use the acronym PYthon Native Interface 
Support.


pynis! :-)

A set of C++ classes to ease the writing of extensions.

Like,



// progrock.pynis  --  "Python Native Interface Support"
// A simple C++ framework for writing Python 3.x extensions.
//
// Copyright (C) Alf P. Steinbach, 2010.

#ifndef PYNIS_PTR_H
#define PYNIS_PTR_H
#include 


//- Dependencies:

#include 
#include 
#include 



//- Interface:

namespace progrock{ namespace pynis {

enum DoAddRef { doAddRef };

class Ptr
{
private:
PyObject*   p_;

public:
Ptr( PyObject* p = 0 ): p_( p )
{}

Ptr( PyObject* p, DoAddRef ): p_( p )
{
assert( p != 0 );
Py_INCREF( p_ );
}

Ptr( Ptr const& other ): p_( other.p_ )
{
Py_XINCREF( p_ );
}

~Ptr()
{
Py_XDECREF( p_ );
}

void swapWith( Ptr& other ) { std::swap( p_, other.p_ ); }
Ptr& operator=( Ptr other ) { swapWith( other ); return *this; }

PyObject* get() const   { return p_; }

PyObject* release()
{
PyObject* const result  = p_;
Py_XDECREF( p_ );
p_ = 0;
return result;
}
};

} }  // namespace progrock::pynis


#endif



Cheers,

- Alf (shocked)

PS: Darn, forgot to google it. But I think it's unlikely the name's already in 
use!

--
blog at http://alfps.wordpress.com>
--
http://mail.python.org/mailman/listinfo/python-list

Re: Argh! Name collision!

[Alf P. Steinbach /Usenet]

* Alf P. Steinbach /Usenet, on 08.07.2010 01:47:


 enum DoAddRef { doAddRef };

 class Ptr
 {
 private:
 PyObject*   p_;

 public:
 Ptr( PyObject* p = 0 ): p_( p )
 {}

 Ptr( PyObject* p, DoAddRef ): p_( p )
 {
 assert( p != 0 );
 Py_INCREF( p_ );
 }

 Ptr( Ptr const& other ): p_( other.p_ )
 {
 Py_XINCREF( p_ );
 }

 ~Ptr()
 {
 Py_XDECREF( p_ );
 }

 void swapWith( Ptr& other ) { std::swap( p_, other.p_ ); }
 Ptr& operator=( Ptr other ) { swapWith( other ); return *this; }

 PyObject* get() const   { return p_; }

 PyObject* release()
 {
 PyObject* const result  = p_;
 Py_XDECREF( p_ );


Hark. This Py_XDECREF shouldn't be there, I don't know how it got there. The 
whole point of 'release', with conventional semantics, is to /not/ decrement the 
reference count.




 p_ = 0;
 return result;
 }
 };



Sorry for posting unfinished code,

- Alf


PS: "pyni" was a good name. But in use! When I thought about adding the "s" as 
disambiguation I thought the pure shock value of that was funny in a way, but 
now it doesn't seem funny. Is "pytes" (Python Extension Support) a good name?


--
blog at http://alfps.wordpress.com>
--
http://mail.python.org/mailman/listinfo/python-list

How do I add method dynamically to module using C API?

[Alf P. Steinbach /Usenet]

The code below, very much work in progress, just trying things, is C++.

Sorry about the formatting, I had to reformat manually for this posting:



class Module
{
private:
Ptr p_;

public:
Module( PyModuleDef const& def )
: p_( ::PyModule_Create( const_cast< PyModuleDef* >( &def ) ) )
{
(p_.get() != 0) || cppx::throwX( "Module::: failed" );
}

PyObject* rawPtr() const{ return p_.get(); }
PyObject* release() { return p_.release(); }

void setDocString( wchar_t const s[] )
{
Ptr const v = ::PyUnicode_FromWideChar( s, -1 );
(v.get() != 0)
|| cppx::throwX( "PyUnicode_FromWideChar failed" );
int const _ = ::PyObject_SetAttrString(
p_.get(), "__doc__", v.get()
);
(_ != -1 )
|| cppx::throwX( "PyObject_SetAttrString failed" );
}

void setRoutine(
char const name[], PyCFunction f, char const doc[] = ""
)
{
PyMethodDef def = { name, f, METH_VARARGS, doc };

Ptr const   pyName  = ::PyUnicode_FromString( name );
Ptr r   = ::PyCFunction_NewEx(
&def, p_.get(), pyName.get()
);

int const _ = ::PyModule_AddObject( p_.get(), name, r.release() );
(_ != -1 )
|| cppx::throwX( "PyModule_AddObject failed" );
}
};



Problem: when a routine installed by 'setRoutine' above is called from Python, 
then it fails with e.g. "SystemError: Bad call flags in PyCFunction_Call. 
METH_OLDARGS is no longer supported!"


And since things work for a single method when I declare 'def' as 'static', I 
suspect that means that the function object created by PyCFunction_NewEx holds 
on to a pointer to the PyMethodDef structure?


I'm unable to find documentation of PyCFunction_NewEx, and more criticially, I'm 
unable to find any documented way to turn a C or C++ function into a Python 
function object?



Cheers,

- Alf

--
blog at http://alfps.wordpress.com>
--
http://mail.python.org/mailman/listinfo/python-list

Re: How do I add method dynamically to module using C API?

[Alf P. Steinbach /Usenet]

* Martin v. Loewis, on 08.07.2010 07:23:

And since things work for a single method when I declare 'def' as
'static', I suspect that means that the function object created by
PyCFunction_NewEx holds on to a pointer to the PyMethodDef structure?


Correct; it doesn't make a copy of the struct. So when you want the
function object to outlive the setRoutine call, you need to allocate
the PyMethodDef on the heap.


Thanks!

That's the direction I tentatively had started to investigate.

But second problem now is cleanup: I'd like to deallocate when the module is 
freed. I tried (1) adding a __del__, but no dice, I guess because it wasn't 
really an object method but just a free function in a module; and (2) the m_free 
callback in the module definition structure, but it was not called.


Perhaps I don't need to to clean up?

Anyway, current looks like this:



// progrock.cppy  --  "C++ plus Python"
// A simple C++ framework for writing Python 3.x extensions.
//
// Copyright (C) Alf P. Steinbach, 2010.

#ifndef CPPY_MODULE_H
#define CPPY_MODULE_H
#include 


//- Dependencies:

#include "Ptr.h"
#include 
#include 



//- Interface:

namespace progrock{ namespace cppy {

namespace detail {
inline PyModuleDef* moduleDefPtr()
{
static PyMethodDef methodDefs[] = {
//...
//{ "system", &pyni_system, METH_VARARGS, "Execute a shell 
command." },

//{ "__del__", &onModuleDestroy, METH_NOARGS, "Destructor" },
//...
{ NULL, NULL, 0, NULL } // Sentinel
};

static PyModuleDef moduleDef = {
PyModuleDef_HEAD_INIT,
"cppy", // name of module
NULL,   // m_doc,   // module documentation in UTF-8
sizeof(void*), // size of per-interpreter state of the 
module,
//or -1 if the module keeps state in global 
variables.

methodDefs,
NULL,   // m_reload
NULL,   // m_traverse
NULL,   // m_clear
NULL// m_free
};

return &moduleDef;
}
}

class Module
{
private:
struct InstanceData
{
std::list< PyMethodDef >methodDefs;
};

void* instanceMemory() const
{
void* const result = PyModule_GetState( p_.get() );
assert( result != 0 );
return result;
}

InstanceData*& instanceDataPtr() const
{
return *reinterpret_cast< InstanceData** >( instanceMemory() );
}

Ptr p_;
InstanceData*   data_;

public:
Module()
: p_( ::PyModule_Create( detail::moduleDefPtr() ) )
{
assert( def.m_size == sizeof( void* ) );
(p_.get() != 0) || cppx::throwX( "Module::: failed" );
instanceDataPtr() = data_ = new InstanceData();
}

PyObject* rawPtr() const{ return p_.get(); }
PyObject* release() { return p_.release(); }

void setDocString( wchar_t const s[] )
{
Ptr const v = ::PyUnicode_FromWideChar( s, -1 );
(v.get() != 0)
|| cppx::throwX( "Module::setDocString: PyUnicode_FromWideChar 
failed" );

::PyObject_SetAttrString( p_.get(), "__doc__", v.get() )
>> cppx::is( cppx::notMinusOne )
|| cppx::throwX( "Module::setDocString: PyObject_SetAttrString 
failed" );

}

void addRoutine( char const name[], PyCFunction f, char const doc[] = 
"" )
{
PyMethodDef const defData = { name, f, METH_VARARGS, doc };

data_->methodDefs.push_back( defData );
try
{
PyMethodDef*pDef= &data_->methodDefs.back();

Ptr const   pyName  = ::PyUnicode_FromString( name );
Ptr r   = ::PyCFunction_NewEx( pDef, p_.get(), 
pyName.get());


::PyModule_AddObject( p_.get(), name, r.release() )
>> cppx::is( cppx::notMinusOne )
|| cppx::throwX( "Module::addRoutine: PyModule_AddObject 
failed" );

}
catch( ... )
{
data_->methodDefs.pop_back();
throw;
}
}
};

} }  // namespace progrock::cppy


#endif



I changed the module name from "pyni*" to "cppy"... ;-)


Cheers & thanks!, but how to clean up, or must I?

- Alf

--
blog at http://alfps.wordpress.com>
--
http://mail.python.org/mailman/listinfo/python-list

Re: How do I add method dynamically to module using C API?

[Alf P. Steinbach /Usenet]

* Martin v. Loewis, on 08.07.2010 09:13:

I tried (1) adding a __del__, but no dice, I guess
because it wasn't really an object method but just a free function in a
module; and (2) the m_free callback in the module definition structure,
but it was not called.


m_free will be called if the module object gets deallocated. So if
m_free really isn't called, the module got never deallocated.


Thanks again. I don't know what I did wrong. Now it's called. :-)

But I wasted much time googling to try to find out the /responsibilities/ of the 
m_free callback, and what its void* argument was. E.g., should it deallocate the 
module object, and if so, via what deallocation routine? I found some info, but 
even your PEP, otherwise clear, was silent about this fine point.


Finally I looked at the source code that invokes it and found that it has no 
responsibilities whatsoever, just a use-as-you-wish finalization callback. Nice!


But I think that could be more clear in the docs...

Code, for those who might be interested:



// progrock.cppy  --  "C++ plus Python"
// A simple C++ framework for writing Python 3.x extensions.
//
// Copyright (C) Alf P. Steinbach, 2010.

#ifndef CPPY_MODULE_H
#define CPPY_MODULE_H
#include 


//- Dependencies:

#include "Ptr.h"
#include 
#include 



//- Interface:

namespace progrock{ namespace cppy {

namespace detail {
struct InstanceData
{
std::list< PyMethodDef >methodDefs;
};

inline void* instanceMemoryOf( PyObject* pObject )
{
void* const result = PyModule_GetState( pObject );
assert( result != 0 );
return result;
}

inline InstanceData*& instanceDataPtrOf( PyObject* pObject )
{
return *reinterpret_cast< InstanceData** >(
instanceMemoryOf( pObject )
);
}

inline void on_freeModule( void* p )
{
PyObject* const pModule = reinterpret_cast< PyObject* >( p );
InstanceData* const pData   = instanceDataPtrOf( pModule );

delete pData;
printf( "Deallocated!\n" ); // TODO: Remove.
}

inline PyModuleDef* moduleDefPtr()
{
static PyMethodDef methodDefs[] = {
//...
//{ "system", &pyni_system, METH_VARARGS, "Execute a shell 
command." },

//{ "__del__", &onModuleDestroy, METH_NOARGS, "Destructor" },
//...
{ NULL, NULL, 0, NULL } // Sentinel
};

static PyModuleDef moduleDef = {
PyModuleDef_HEAD_INIT,
"cppy", // name of module
NULL,   // m_doc,   // module documentation in UTF-8
sizeof(void*), // size of per-interpreter state of the 
module,
//or -1 if the module keeps state in global 
variables.

methodDefs,
NULL,   // m_reload
NULL,   // m_traverse
NULL,   // m_clear
&on_freeModule  // m_free
};

return &moduleDef;
}
}

class Module
{
private:
Ptr p_;
detail::InstanceData*   data_;

detail::InstanceData*& instanceDataPtr() const
{
return detail::instanceDataPtrOf( p_.get() );
}

public:
Module()
: p_( ::PyModule_Create( detail::moduleDefPtr() ) )
, data_( 0 )
{
assert( detail::moduleDefPtr()->m_size == sizeof( void* ) );
(p_.get() != 0) || cppx::throwX( "Module::: failed" );
instanceDataPtr() = data_ = new detail::InstanceData();
}

PyObject* rawPtr() const{ return p_.get(); }
PyObject* release() { return p_.release(); }

void setDocString( wchar_t const s[] )
{
Ptr const v = ::PyUnicode_FromWideChar( s, -1 );
(v.get() != 0)
|| cppx::throwX( "Module::setDocString: PyUnicode_FromWideChar 
failed" );


::PyObject_SetAttrString( p_.get(), "__doc__", v.get() )
>> cppx::is( cppx::notMinusOne )
|| cppx::throwX( "Module::setDocString: PyObject_SetAttrString 
failed" );

}

void addRoutine( char const name[], PyCFunction f, char const doc[] = 
"" )
{
PyMethodDef const defData = { name, f, METH_VARARGS, doc };

data_->methodDefs.push_back( defData );
try
{
PyMethodDef* const  pDef= &data_->methodDefs.back();

Ptr const   pyName  = ::PyUnicode_FromString( name );
(pyName.get() != 0)
|| cppx::throwX( "Module::addRoutine: PyUnicode_FromString 
faile

Cpp + Python: static data dynamic initialization in *nix shared lib?

[Alf P. Steinbach /Usenet]

[Cross-posted comp.lang.python and comp.lang.c++]

I lack experience with shared libraries in *nix and so I need to ask...

This is about "cppy", some support for writing Python extensions in C++ that I 
just started on (some days ago almost known as "pynis" (not funny after all)).


For an extension module it seems that Python requires each routine to be defined 
as 'extern "C"'. And although e.g. MSVC is happy to mix 'extern "C"' and C++ 
linkage, using a routine declared as 'static' in a class as a C callback, 
formally they're two different kinds, and I seem to recall that /some/ C++ 
compiler balks at that kind of mixing unless specially instructed to allow it. 
Perhaps it was the Sun compiler?


Anyway, to be formally correct I cannot generate the required C routines via 
templating, and I ended up using macros that the user must explicitly invoke, 
like, here the Py doc's first extension module example recoded using cppy,



--

#include 
#include 
using namespace progrock;

class Spam: public cppy::Module
{
public:
Spam(): cppy::Module( "spam" )
{
setDocString( L"blåbærsyltetøy er blått" );
}

PyObject* system( PyObject* args )
{
const char *command;
int sts;

if( !PyArg_ParseTuple( args, "s", &command ) )
{
return NULL;
}
sts = ::system( command );
return Py_BuildValue( "i", sts );
}
};

CPPY_MODULE_CROUTINE( Spam, system, "Execute a shell command" )

PyMODINIT_FUNC PyInit_spam()
{
return cppy::init< Spam >();
}

--


It works in Windows.

But here CPPY_MODULE_CROUTINE does three things:

  A Defining the 'extern "C"' routine.
I cannot think of any problem here.

  B Defining installation data for that routine.
Possible problem: initializing a static with address of routine?

  C -> Adding that install data record into a linked list!
Possible problem: are dynamic initialization actions guaranteed
to be performed in *nix shared library?

Problem (C) is outside the realm of the C++ standard, since the C++ standard 
doesn't support shared libraries, and I've never actually used *nix shared 
libraries so I don't /know/...


Is such dynamic initialization guaranteed?

For completeness, the macro definition (the 0 in there is a list next-pointer):



#define CPPY_MODULE_CROUTINE_DEF( cppClassName, name )  \
extern "C"  \
static PyObject* cppClassName##_##name( PyObject*, PyObject* args ) \
{   \
return ::progrock::cppy::module().name( args );   \
}

#define CPPY_MODULE_CROUTINE_INSTALLDATA( cppClassName, name, docString )   \
static ::progrock::cppy::detail::ModuleRoutineDescriptor\
cppClassName##_##name##_descriptor = {  \
0,  \
#name,  \
docString,  \
&cppClassName##_##name  \
};  \
\
static bool cppClassName##_##name##_descriptor_installed =  \
::progrock::cppy::detail::addToList< cppClassName >(\
cppClassName##_##name##_descriptor  \
);

#define CPPY_MODULE_CROUTINE( cppClassName, name, docString )   \
CPPY_MODULE_CROUTINE_DEF( cppClassName, name )  \
CPPY_MODULE_CROUTINE_INSTALLDATA( cppClassName, name, docString )



TIA.,

- Alf

--
blog at http://alfps.wordpress.com>
--
http://mail.python.org/mailman/listinfo/python-list

Re: Hello

[Alf P. Steinbach /Usenet]

* Dani Valverde, on 09.07.2010 18:31:

Hello!
I am new to python and pretty new to programming (I have some expertise
wit R statistical programming language). I am just starting, so my
questions may be a little bit stupid. Can anyone suggest a good editor
for python?
Cheers!


If you're working in Windows the Notepad++ and PSPad and old Crimson Editor (all 
free) all work nicely and are reasonably light-weight.



Cheers & hth.,

- Alf

--
blog at http://alfps.wordpress.com>
--
http://mail.python.org/mailman/listinfo/python-list

Re: Cpp + Python: static data dynamic initialization in *nix shared lib?

[Alf P. Steinbach /Usenet]

* Ian Collins, on 09.07.2010 23:22:

On 07/10/10 03:52 AM, Alf P. Steinbach /Usenet wrote:

[Cross-posted comp.lang.python and comp.lang.c++]

I lack experience with shared libraries in *nix and so I need to ask...

This is about "cppy", some support for writing Python extensions in C++
that I just started on (some days ago almost known as "pynis" (not funny
after all)).

For an extension module it seems that Python requires each routine to be
defined as 'extern "C"'. And although e.g. MSVC is happy to mix 'extern
"C"' and C++ linkage, using a routine declared as 'static' in a class as
a C callback, formally they're two different kinds, and I seem to recall
that /some/ C++ compiler balks at that kind of mixing unless specially
instructed to allow it. Perhaps it was the Sun compiler?


Yes, it will (correctly) issue a warning.

As the is a bit OT, contact me directly and we can work through it. I
have had similar fun and games adding PHP modules!


Thanks. I'm mailing you a zip with the code... 

The question, of course, whether it works in *nix.


Cheers,

- Alf

--
blog at http://alfps.wordpress.com>
--
http://mail.python.org/mailman/listinfo/python-list

Not-quite-the-module-name qualified names in extension modules? What?

[Alf P. Steinbach /Usenet]

Hi.

I built the [xxmodule.c] from the source distribution, as suggested by the 
Python 3.1.1 docs. I named this [xx.pyd], as I believed the module name was just 
"xx". Indeed importing xx works fine, but when I do help(xx) I get ...




>>> help( xx )
Help on module xx:

NAME
xx - This is a template module just for instruction.

FILE

c:\projects\progrock\lib\progrock\cppy_dev\examples\02_xx_apilevel\xx.pyd

CLASSES
builtins.Exception(builtins.BaseException)
error
builtins.object
xxmodule.Null
builtins.str(builtins.object)
xxmodule.Str

class Null(builtins.object)
 |  Methods defined here:
 |



... with the name "xxmodule" somehow in there as qualification.

Checking the standard "csv" module I similarly get ...



CLASSES
builtins.Exception(builtins.BaseException)
_csv.Error
builtins.object
Dialect
excel
excel_tab
DictReader
DictWriter
Sniffer



... with the name "_csv" in there.

And I'm pretty sure that these not-quite-the-module-name names stem from the 
literal specification of names in the C code in the extension module, assuming 
that the csv module is indeed a C extension module.


Is it really necessary to qualify names in the C code?

And can it do harm when such name qualification is not using the final module 
name but instead something like "xxmodule" or "_csv"?


More to the point, what's the point?


Cheers,

- Alf

--
blog at http://alfps.wordpress.com>
--
http://mail.python.org/mailman/listinfo/python-list

Re: any issues with long running python apps?

[Alf P. Steinbach /Usenet]

* John Nagle, on 10.07.2010 20:54:

On 7/9/2010 12:13 PM, Les Schaffer wrote:

i have been asked to guarantee that a proposed Python application will
run continuously under MS Windows for two months time. And i am looking
to know what i don't know.

The app would read instrument data from a serial port,


If the device you're listening to is read-only, and you're just
listening, make a cable to feed the serial data into two machines,
and have them both log it. Put them on separate UPSs and in
a place where nobody can knock them over or mess with them.


"The Ramans do everything in triplicate" - Old jungle proverb


Cheers,

- Alf

--
blog at http://alfps.wordpress.com>
--
http://mail.python.org/mailman/listinfo/python-list

Re: integer >= 1 == True and integer.0 == False is bad, bad, bad!!!

[Alf P. Steinbach /Usenet]

* rantingrick, on 11.07.2010 08:50:

On Jul 11, 1:22 am, Stephen Hansen  wrote:


Utter nonsense. No one does that unless they are coming from C or some
other language without a True/False and don't know about it, or if they
are using a codebase which is supporting a very old version of Python
before True or False were introduced.


Ah yes, when nothing else seems to work fall back to you default
programming... FUD and ad hominem
attacks


I agree with Stephen, but for a different reason: that given desirability of 
implicit conversion to bool for some elementary types, then for uniformity there 
should be such conversion for all of them (and AFAIK there is), and given that, 
the rule should be the same, namely that default value of each type bool's to 
False, and other values to True, and so it is.


The OP should simply represent "not found" as e.g. integer -1 instead of as a 
value of a different type.


And write e.g.

  not_found = -1

 ...

  if choiceIdx1 == choiceIdx2 == not_found:
  bah, none of them
  elif choice2Idx == not_found:
  use choice 1
  elif choice1Idx == not_found:
  use choice 2
  else:
  determine bestest choice


Cheers & hth.,

- Alf


--
blog at http://alfps.wordpress.com>
--
http://mail.python.org/mailman/listinfo/python-list

Re: integer >= 1 == True and integer.0 == False is bad, bad, bad!!!

[Alf P. Steinbach /Usenet]

* Stephen Hansen, on 11.07.2010 09:19:

On 7/10/10 11:50 PM, rantingrick wrote:


It was a typo not an on purpose misspelling


If this had been the first time, perhaps. If you had not in *numerous*
previous times spelled my name correctly, perhaps. If it were at all
possible for "f" to be a typo of "ph", perhaps.


It is a natural mistake to make in some languages. E.g. in Norwegian the Devil 
can be spelled Faen or Fanden (modern) or Phanden (old-fashioned, no longer in 
dictionaries but still used to sort of tone down the expression). It's even 
there in English, like "file" and "philosophy". So it's an error committed not 
by the limbic system but by a slightly higher level sound-to-text translator 
brain circuit. The text is generated from how the word sounds in one's head.



Cheers & hth.,

- Alf

--
blog at http://alfps.wordpress.com>
--
http://mail.python.org/mailman/listinfo/python-list

Re: Naming Conventions, Where's the Convention Waldo?

[Alf P. Steinbach /Usenet]

* rantingrick, on 11.07.2010 09:26:


Another source of asininity seems to be the naming conventions of the
Python language proper! True/False start with an upper case and i
applaud this. However str, list, tuple, int, float --need i go
on...?-- start with lowercase.

Q: Well what the hell is your problem Rick. Who cares right?

WRONG, I tell you what my problem is. Now i cannot "wisely" use
variables like...

str="this is a string"
list = [1,2,3]
def make_random_objs(range=10)
def show_message(str)
int = 12

If we would have adopted proper naming conventions from dios numero
uno all this nonsense would be rectified! Str, Float, List, Range,
etc, etc. You think Python 3000 was a hump to climb over just wait for
Python 4000.

Just thoughts.


Just do

  Str = str
  List = list
  Float = float

and so on in module "myBasicTypes", and import that.

:-)

Cheers & hth.,

- Alf

--
blog at http://alfps.wordpress.com>
--
http://mail.python.org/mailman/listinfo/python-list

Re: Easy questions from a python beginner

[Alf P. Steinbach /Usenet]

* Stephen Hansen, on 11.07.2010 21:00:

On 7/11/10 11:45 AM, wheres pythonmonks wrote:

Follow-up:
Is there a way to define compile-time constants in python and have the
bytecode compiler optimize away expressions like:

if is_my_extra_debugging_on: print ...

when "is_my_extra_debugging" is set to false?  I'd like to pay no
run-time penalty for such code when extra_debugging is disabled.


Any code wrapped in a __debug__ guard is utterly ommitted if you run
Python with the -O option. That, and asserts go away.


On #2:  My point regarding the impossibility of writing the swap
function for ints is to explicitly understand that this isn't
possible, so as not to look for solutions along those lines when
trying to write python code.


Its impossible because Python's calling and namespace semantics simply
don't work like that. There's no references in the traditional sense,
because there's no variables-- boxes that you put values in. There's
just concrete objects. Objects are passed into the function and given
new names; that those objects have names in the enclosing scope is
something you don't know, can't access, and can't manipulate.. even the
objects don't know what names they happen to be called.

Check out http://effbot.org/zone/call-by-object.htm


Oh, I wouldn't give that advice. It's meaningless mumbo-jumbo. Python works like 
Java in this respect, that's all; neither Java nor Python support 'swap'.


Of course there are variables, that's why the docs call them variables.

We've had this discussion before and I know from that that it is a religious 
issue with a small subset of the Python community, where reason, facts, logic 
does not apply and is not even recognized as such. So be it. So I'm not out to 
convince you or other of that sub-community, or trying to reason with you folks 
on this issue (futile, and generates flames pretty fast), but I do not want 
newbies brainwashed into that non-reasoning nonsense pure faith religion.


For what it's worth, I'm sure that the effbot.org author, whose pages are 
otherwise quite technically meaningful & useful, in this case, after the flame 
war with some Java folks, decided that technical accuracy just wasn't worth it.


So, I believe, he punted, which is an eminently rational choice when one's goals 
require acceptance in a society dominated by a religious clique. And just as I'm 
not out to engage you in any debate on this issue (futile), neither am I calling 
you irrational. Perhaps your choice is the same as that author's.



Cheers,

- Alf

--
blog at http://alfps.wordpress.com>
--
http://mail.python.org/mailman/listinfo/python-list

Re: Easy questions from a python beginner

[Alf P. Steinbach /Usenet]

* MRAB, on 12.07.2010 00:37:

Alf P. Steinbach /Usenet wrote:

* Stephen Hansen, on 11.07.2010 21:00:

On 7/11/10 11:45 AM, wheres pythonmonks wrote:

Follow-up:
Is there a way to define compile-time constants in python and have the
bytecode compiler optimize away expressions like:

if is_my_extra_debugging_on: print ...

when "is_my_extra_debugging" is set to false? I'd like to pay no
run-time penalty for such code when extra_debugging is disabled.


Any code wrapped in a __debug__ guard is utterly ommitted if you run
Python with the -O option. That, and asserts go away.


On #2: My point regarding the impossibility of writing the swap
function for ints is to explicitly understand that this isn't
possible, so as not to look for solutions along those lines when
trying to write python code.


Its impossible because Python's calling and namespace semantics simply
don't work like that. There's no references in the traditional sense,
because there's no variables-- boxes that you put values in. There's
just concrete objects. Objects are passed into the function and given
new names; that those objects have names in the enclosing scope is
something you don't know, can't access, and can't manipulate.. even the
objects don't know what names they happen to be called.

Check out http://effbot.org/zone/call-by-object.htm


Oh, I wouldn't give that advice. It's meaningless mumbo-jumbo. Python
works like Java in this respect, that's all; neither Java nor Python
support 'swap'.

Of course there are variables, that's why the docs call them variables.


In Java a variable is declared and exists even before the first
assignment to it. In Python a 'variable' isn't declared and won't exist
until the first 'assignment' to it.


That is a misconception.

In Python a variable is declared by having an assignment to it, which for a 
local variable may be anywhere within a routine.


If such a variable is used before it's been assigned to, then you get an 
uninitialized variable exception. Clearly the variable must exist in order for 
the exception to refer to it (not to mention the exception occurring at all).


  def foo():
  print( blah )
  blah = "this is both an assignment and a declaration causing it to exist"

  foo()

Clearly when the exception is raised, referring to the variable, the variable 
exists.


Contrary to your statement that is before the assignment.

However, as stated up-thread, I do not expect facts, logic or general reasoning 
to have any effect whatsoever on such hard-core religious beliefs. And I do not 
care whether I convince you or not. But I *do not* want the religious subset of 
the community to succeed too much in propagating nonsense idiot beliefs to 
newbies  --  hence the concrete example that any newbie can try.



Cheers & hth.,

- Alf

--
blog at http://alfps.wordpress.com>
--
http://mail.python.org/mailman/listinfo/python-list

Re: Easy questions from a python beginner

[Alf P. Steinbach /Usenet]

* Stephen Hansen, on 12.07.2010 04:02:

On 7/11/10 6:12 PM, Alf P. Steinbach /Usenet wrote:

However, as stated up-thread, I do not expect facts, logic or general
reasoning to have any effect whatsoever on such hard-core religious
beliefs.


Grow up, and/or get a grip, and/or get over yourself.

Everyone who disagreed with you, disagreed with you with arguments,
logic, facts, and reasoning. You disputed those facts, disagreed with
the conclusions, but for you to then just dismiss people who don't agree
with you as merely "religious", is childish.

Exactly why I think you're wrong -- you're free to go re-read, I stand
by my statements in this thread, and the others. The same arguments
apply. Its not a religion, dear; my conclusions are not a matter of faith.

That's all I have to say on this subject; the conversation has been had,
at length (repeatedly).

I swear, I'm just going to filter you and Rick out to /dev/null today
and leave it at that at this rate. I'm getting worn out of these kinds
of responses.


Well, the above is flaming, which I predicted.

The alleged facts etc. you're referring are just that, alleged, by you.

In contrast, in debates among non-religious folks facts are /presented/, like 
I've done in this thread, e.g. concrete code, instead of like you alleging that 
facts have been presented, hinting about things, and so on  --  it's pathetic.



Cheers & hth.,

- Alf

--
blog at http://alfps.wordpress.com>
--
http://mail.python.org/mailman/listinfo/python-list

Re: Easy questions from a python beginner

[Alf P. Steinbach /Usenet]

* MRAB, on 12.07.2010 04:09:

Alf P. Steinbach /Usenet wrote:

* MRAB, on 12.07.2010 00:37:

Alf P. Steinbach /Usenet wrote:

* Stephen Hansen, on 11.07.2010 21:00:

On 7/11/10 11:45 AM, wheres pythonmonks wrote:

Follow-up:
Is there a way to define compile-time constants in python and have
the
bytecode compiler optimize away expressions like:

if is_my_extra_debugging_on: print ...

when "is_my_extra_debugging" is set to false? I'd like to pay no
run-time penalty for such code when extra_debugging is disabled.


Any code wrapped in a __debug__ guard is utterly ommitted if you run
Python with the -O option. That, and asserts go away.


On #2: My point regarding the impossibility of writing the swap
function for ints is to explicitly understand that this isn't
possible, so as not to look for solutions along those lines when
trying to write python code.


Its impossible because Python's calling and namespace semantics simply
don't work like that. There's no references in the traditional sense,
because there's no variables-- boxes that you put values in. There's
just concrete objects. Objects are passed into the function and given
new names; that those objects have names in the enclosing scope is
something you don't know, can't access, and can't manipulate.. even
the
objects don't know what names they happen to be called.

Check out http://effbot.org/zone/call-by-object.htm


Oh, I wouldn't give that advice. It's meaningless mumbo-jumbo. Python
works like Java in this respect, that's all; neither Java nor Python
support 'swap'.

Of course there are variables, that's why the docs call them variables.


In Java a variable is declared and exists even before the first
assignment to it. In Python a 'variable' isn't declared and won't exist
until the first 'assignment' to it.


That is a misconception.

In Python a variable is declared by having an assignment to it, which
for a local variable may be anywhere within a routine.

If such a variable is used before it's been assigned to, then you get
an uninitialized variable exception. Clearly the variable must exist
in order for the exception to refer to it (not to mention the
exception occurring at all).

def foo():
print( blah )
blah = "this is both an assignment and a declaration causing it to exist"

foo()

Clearly when the exception is raised, referring to the variable, the
variable exists.

Contrary to your statement that is before the assignment.

However, as stated up-thread, I do not expect facts, logic or general
reasoning to have any effect whatsoever on such hard-core religious
beliefs. And I do not care whether I convince you or not. But I *do
not* want the religious subset of the community to succeed too much in
propagating nonsense idiot beliefs to newbies -- hence the concrete
example that any newbie can try.


How about this:

 >>> def foo():
print("Before:", locals())
x = 0
print("After:", locals())


 >>> foo()
Before: {}
After: {'x': 0}


How about it?

Note that you get the same result if you do

x = "blah"
def foo():
# print( x )  # Causes uninitialized variable exception here
print( "Before:", locals() )
x = 0
print( "After:", locals() )

However, if you remove the local assignment to x, then the out-commented print 
statement will no longer cause an exception, it will then refer to the global.


The reason that it does throw an exception when you do have the local 
assignment, is that the local x exists at that point. If it didn't exist it 
could not have any effect. Things that don't exist generally have no effect, 
except in the minds of the religious, like angels and so on.


On the basis of what locals() reports it should be OK to refer to the global x 
as above. Judging by locals(), there's no local x that could get in the way. But 
since it is not OK to refer to the global x, the result of locals() has nothing 
to do with that: it doesn't tell you about the local x  --  and no, the Python 
interpreter does not look forward in time to see that it will appear.


In passing, I should perhaps have told you up front, your argument has nothing 
substantial to do with the article you originally responded to, about the 
semantics of variables. Your argument is the assertion that different languages 
can't have similar or identical semantics for some feature. That's nonsense in 
itself, plus, as you've seen, the difference that you focused on is not there, 
and, third, what you do maintain is not there, doesn't exist, has a real effect.



Cheers & hth.,

- Alf

--
blog at http://alfps.wordpress.com>
--
http://mail.python.org/mailman/listinfo/python-list

Re: Easy questions from a python beginner

[Alf P. Steinbach /Usenet]

* sturlamolden, on 12.07.2010 06:52:

On 11 Jul, 21:37, "Alf P. Steinbach /Usenet"  wrote:


Oh, I wouldn't give that advice. It's meaningless mumbo-jumbo. Python works like
Java in this respect, that's all; neither Java nor Python support 'swap'.


x,y = y,x



We're talking about defining a 'swap' routine that works on variables.

Since Java/Python doesn't support pass by reference of variables it's not 
possible in these languages, i.e., you missed the point, or made a joke. :-)


However, C# is very similar to Java, nearly all the way, except that in C# you 
can pass by reference. Remove that from C# and you have Java. Add that to Java 
and you have C#, roughly. No change in other aspects is needed. E.g. (ignore 
this if you've never heard about it, but it's a subtle point that you might be 
wondering about now) both Java and C# implement the definite assignment rule.


I.e., there's nothing in the core semantics that prevents accessing/passing the 
variables by reference, although for Python and Java it could be a 
terminological nightmare, and for Python compounded to the n'th degree by the 
general confusion of a subset of the Python community about basic concepts.


I don't know how C#'ers resolve the terminology...


Cheers & hth.,

- Alf

--
blog at http://alfps.wordpress.com>
--
http://mail.python.org/mailman/listinfo/python-list

Design questions for C++ support for Python extensions (cppy)

[Alf P. Steinbach /Usenet]

Hi.

With the current cppy code the Python 3.1.1 doc's spam example extension module 
looks like this (actual working code):




#include 
#include 
using namespace progrock;

namespace {

class Spam: public cppy::Module
{
public:
Spam(): cppy::Module( L"spam", L"blåbærsyltetøy er blått" )
{}

PyObject* system( PyObject* args )
{
const char *command;
if( !PyArg_ParseTuple( args, "s", &command ) )
{
return NULL;
}
int const sts = ::system( command );
return Py_BuildValue( "i", sts );
}
};

}// namespace 

CPPY_MODULE_CROUTINE( Spam, system, L"Execute a shell command" )

PyMODINIT_FUNC PyInit_spam()
{
return cppy::safeInit< Spam >();
}



Issues:

  1. Wide string literals OK?
 The basic Python API often requires UTF-8 encoded byte strings. With C++
 source code encoded using e.g. Windows ANSI Western, string literals with
 national characters such as Norwegian ÆØÅ then become gobbledegook or cause
 outright failure. I balanced the hypothetical need for string literals with
 national characters, versus perceived unnaturalness of wide string literals
 for *nix developers, in favor of the former, i.e. L"wide string literals".

 Related issue here: in Windows MinGW g++ can not compile utf-8 encoded
 source with BOM, while MSVC requires a BOM in order to detect the encoding.

 Is L"this" an acceptable decision if you were to use something like cppy?

  2. Exception translation OK?
 The code within the 'system' member routine could conceivably be reduced to
 a single short line by adding some C++ support, but that would require use
 of exceptions to report errors. Translating C++ exceptions to Python
 exceptions does however add some overhead to every Python -> C++ call.
 Currently I do this only for the module initialization code, but should it
 be done for every exported method? Or perhaps as user choice? Benefit of
 translation e.g. reducing 'system' above to sweet single line. Cost is
 setup of try-block (negligible) and exception translation (inefficient).

  3. Some unsafety OK?
 In 'PyInit_spam' there may be a window of opportunity for client code to
 Mess Things Up. Within the cppy::safeInit the C++ module object is created
 and creates a Python module object, and if anything fails then the C++ side
 frees the Python object. And after PyInit_spam has returned to Python the
 cleanup responsibility resides with the Python interpreter: freeing the
 Python module object causes the C++ object to be destroyed. But if say the
 client code's equivalent of 'PyInit_spam' calls cppy::safeInit and just
 discards the result and returns 0 (say) to Python, then it seems to not be
 documented whether Python will free the Python module, i.e. possible leak.

  4. Threading?
 Is it necessary to make singletons/statics thread safe? Or does Python
 ensure that no other threads execute while PyInit_spam is called? Can it
 be called simultaneously by two or more threads?

  5. Reload of interpreter?
 My impression from the documentation is that finalization and reinit of the
 interpreter is something that an application shouldn't really do, and that
 an extension developer need not worry about that happening. Is it so?


Cheers,

- Alf

--
blog at http://alfps.wordpress.com>
--
http://mail.python.org/mailman/listinfo/python-list

Re: Easy questions from a python beginner

[Alf P. Steinbach /Usenet]

* sturlamolden, on 12.07.2010 16:59:

On 12 Jul, 07:51, "Alf P. Steinbach /Usenet"  wrote:


We're talking about defining a 'swap' routine that works on variables.


I did not miss the point. One cannot make a swap function that rebinds
its arguments in the calling stack frame. But a swap function can swap
values, given that the type is not immutable:

def swap(a,b):
a[0],b[0] = b[0],a[0]


a,b = [1],[2]
swap(a,b)
print a,b

[2] [1]


OK, that's missing the point.

I thought you were joking.


Cheers & hth.,

- Alf

--
blog at http://alfps.wordpress.com>
--
http://mail.python.org/mailman/listinfo/python-list

Re: Easy questions from a python beginner

[Alf P. Steinbach /Usenet]

* Steven D'Aprano, on 12.07.2010 04:39:

On Mon, 12 Jul 2010 03:12:10 +0200, Alf P. Steinbach /Usenet wrote:


* MRAB, on 12.07.2010 00:37:

[...]

In Java a variable is declared and exists even before the first
assignment to it. In Python a 'variable' isn't declared and won't exist
until the first 'assignment' to it.


That is a misconception.

In Python a variable is declared by having an assignment to it, which
for a local variable may be anywhere within a routine.


Oh, I'm going to regret being sucked into this...

In *CPython*, but not necessarily other implementations, variables which
are local to a function are not kept in a dictionary-based namespace, but
in slots in the code object (not to be confused with __slots__ used for
classes). Python has STORE_FAST and LOAD_FAST byte-codes for accessing
locals.

This is intended as a speed, and possibly memory, optimization. I don't
believe this is a requirement though, so implementations may not do this.

It is true that the slot is created at compile time, and in *that sense*,
local variables exist before they are bound. I'm not entirely convinced
that this is the only sense that matters, but never mind. The error
message given exposes this to the user:


def f():

... print x
... x = 1
...

f()

Traceback (most recent call last):
   File "", line 1, in
   File "", line 2, in f
UnboundLocalError: local variable 'x' referenced before assignment


If you try this with a global, you get this:


def f():

... global x
... print x
...

f()

Traceback (most recent call last):
   File "", line 1, in
   File "", line 3, in f
NameError: global name 'x' is not defined

In this case, there's no doubt that global variable "x" doesn't exist at
all -- there is no key "x" in the global namespace.


Yes.

What I disproved was the statement that every Python variable is created by the 
execution of an assignment.


Some (the global ones) are. :-)



It seems to me that "a slot to hold the variable is created for local
variables" is an implementation detail, not a language feature.


Yes.

However, any Python implementation has to implement the same user level 
semantics in some way (note: use level semantics do not include "space reserved" 
for an unassigned variable, or even for all assigned variables since with single 
assignment a sufficiently smart compiler can optimize away the space, but user 
level semantics do include existence and resultant effect).


As I see it it doesn't matter whether the implementation is CPython call frame 
slots or that mechanism called something else or a different mechanism called 
the same or a different mechanism called something different; what IMO matters 
is same semantics, that any assignment to a variable within a routine serves as 
a compile time declaration, creating that local variable in advance, unless, 
with Python 3.x., that name has been declared as a 'global' or 'nonlocal'.


So, this is a possible point of disagreement.

I say the semantics of local variable creation are part of the language 
definition, but I get the /impression/ that maybe you think it's 
CPython-specific, that e.g.


  def foo():
  x
  x = 0

might not raise an unassigned variable exception with some conforming Python 
implementation, i.e. different effect for same code with different 
implementations, that this is at least /unspecified behavior/ in Python?




CPython
could easily hide the difference by changing the exception from
UnboundLocalError to:

NameError: local name 'x' does not exist

and nobody would be any wiser. (Well, perhaps people who catch
UnboundLocalError, but why would you do that?)

I also note that UnboundLocalError is a subclass of NameError, so
"variable exists but is not bound" is considered to be a special case of
"variable doesn't exist" rather than a completely independent case. In
that sense, I think I'm on solid ground to say that in Python variables
don't exist until they are bound to a value, and leave it to pedants like
you and I to mention that for CPython local variables have space reserved
for them by the compiler before they are bound.


He he. I wouldn't say "space reserved". That is an implementation detail.


Cheers,

- Alf

--
blog at http://alfps.wordpress.com>
--
http://mail.python.org/mailman/listinfo/python-list

Re: Easy questions from a python beginner

[Alf P. Steinbach /Usenet]

* Rhodri James, on 12.07.2010 22:19:

On Mon, 12 Jul 2010 13:56:38 +0100, bart.c  wrote:


"Steven D'Aprano"  wrote in
message news:4c3aedd5$0$28647$c3e8...@news.astraweb.com...

On Mon, 12 Jul 2010 09:48:04 +0100, bart.c wrote:


That's interesting. So in Python, you can't tell what local variables a
function has just by looking at it's code:



def foo(day):
if day=="Tuesday":
x=0
print ("Locals:",locals())

#foo("Monday")

Does foo() have 1 or 2 locals?


That's easy for CPython: it prepares two slots for variables, but only
creates one:


foo("Monday")

('Locals:', {'day': 'Monday'})

foo.func_code.co_varnames

('day', 'x')

foo.func_code.co_nlocals

2

So, the question is, is x a local variable or not? It's not in locals,
but the function clearly knows that it could be.


So Alf P.S. could be right; x exists, but Python pretends it doesn't
until it's assigned to.


CPython, not Python. And as Steven said, x *doesn't* exist. Allowance is
made by that specific implementation of the interpreter because x
*might* exist, but in this particular case it doesn't and a more dynamic
implementation might choose not to reserve a slot just in case. x is
created until it's actually used.


You are conflating existence with space allocation.

It's up to the implementation whether to allocate memory for the variable's 
reference in any particular case where that memory isn't strictly required. This 
is known as "optimization". Optimization depends on the implementation.


Existence of a variable means, among other things, that

  * You can use the value, with guaranteed effect (either unassigned exception
or you get a proper value): in particular, you won't be accessing a global
if you're using the name of a local declared by a later assignment.

  * You can assign to it.

How the Python implementation implements that is an implementation detail.

In short, how CPython does things is completely irrelevant to the language's 
semantics, so you're conflating things here.



Cheers & hth.,

- Alf

--
blog at http://alfps.wordpress.com>
--
http://mail.python.org/mailman/listinfo/python-list