Re: to create variable from dict

[hiral]

On Mar 12, 8:02 pm, Jean-Michel Pichavant 
wrote:
> Luis M. González wrote:
> > On Mar 12, 10:59 am,hiral wrote:
>
> >> Hi,
>
> >> Is there any way to create variables which name matches with dict key?
>
> >> For example:
> >> dict1 = {"abc":'1", "def":"2"}
>
> >> Now I am looking to have variable name abc and it's value be '1' etc.
>
> >> Pl. suggest.
>
> >> Thank you.
>
> > Check out this thread (very recent):
> >http://groups.google.com/group/comp.lang.python/browse_thread/thread/...
>
> > Short answer: you can update globals() with a dictionary, as follows:
>
> > globals().update( dict1 )
>
> > Then you'll have each key-value pair as variables in the global
> > namespace.
> > The question is: should you do it?
>
> > Luis
>
> The answer is known: no, he should not do it :o)
>
> JM- Hide quoted text -
>
> - Show quoted text -

Thanks for your explanations.
-- 
http://mail.python.org/mailman/listinfo/python-list

Hacker News, Xahlee.Org, and What is Politics?

[Xah Lee]

A essay related to the recent discussion of banning, and lisp
associated group at ycombinator.com .

-
Hacker News, Xahlee.Org, and What is Politics?

Xah Lee, 2010-03-14

Today, i noticed that someone posted one of my article “Why Emacs is
still so useful today” on my blogger site (xahlee.blogspot.com) to
Hacker News site here: Source.

Note that my site XahLee.org was unceremoniously banned around 2009-02
by the admin geekers there. (see bottom of: Ban Xah Lee). But,
apparantly, someone found my article useful. Now, in order to submit
my articles, you'll have to use my article mirror at
blogger.blogspot.com instead of xahlee.org.


Tech Geeker's Ignorance Of Politics

The human nature of power struggle never ends. Many of these tech
geekers think they are apolitical, that they are peace loving, they
ride bikes not driving machines that pollute, they don't have interest
in business, they embrace the Free of “freedom” Software or Open
Source of “openness” software. The problem of human animals is that,
we all think that, yet, power struggle or the formation of politics
isn't something you consciously decide to do. Rather, it is something
that you unconsciously do whenever you have interaction with peers
when resources are not infinite. At the basic level, you have quarrels
with ya brothers about toys, in high school about seats, girls. Going
up a bit, you have all these teens, gamers, tech geekers, shouting,
kicking, banning, in all sort of internet forums, irc chat rooms,
mailing lists, in-world gaming channels. Going up, you have managers
fighting in corporations, politicians pissing at each other, leaders
warring among nation states. Note the word PEER here, because, you are
out of the game with those who are much more powerful than you, and
similarly, you can't be bothered by those far beneath you.

Of course, the tech geekers all think they are fair, just, maintaining
peace. But you can almost see the smug on their faces whenever any of
these “admins”, “group owners”, “sysops”, “operators”, exercise their
power. It is a basic human pleasure to force onto uncooperative
others. (See: What Desires Are Politically Important?)

Serious fairness and open policies never actually come up unless the
community becomes large, larger then the original special interest
clique. Before the group becomes large, typically their political
structure is academically called oligarchy. i.e. basically whoever
started the group, his cronies, rule.

We all understand conscious power struggle. However, it is when we
think that we are not doing politics when exercising our power, is
when it is likely most damaging, with respect to human progress. (the
subject of human progress, its ideal direction, goal, is itself is
quite not agreed upon among historical figures (big philosophers,
politicians, leaders, etc.).)

Part of the problem is ignorance. Most tech geekers, those coders,
engineers, type of folks, typically don't have much education or
interest in social sciences. Few years ago on irc channel, i was
ranting on the subject of politics, due to some difference of opinion
on tech stuff, and learned, rather not surprisingly, that one of the
guy don't understand that politics essentially means power struggle.
Most tech geekers think that Politics is what politicians do, that it
is something about rulers, about bad ass evil lords, ambitious guys,
Napolean, or about controversial subjects, back stabbing, cloak and
dagger, and isn't about, say, admins or policies in some online forum.


What Is Politics?

Here's some Wikipedia quotes from Politics:

Politics is a process by which groups of people make collective
decisions. The term is generally applied to behavior within civil
governments, but politics has been observed in other group
interactions, including corporate, academic, and religious
institutions. It consists of "social relations involving authority or
power"[1] and refers to the regulation of a political unit,[2] and to
the methods and tactics used to formulate and apply policy.[3]

...

Political science, the study of politics, examines the acquisition
and application of power and "power corrupts".

It quotes this juicy quote:

“Politics: A strife of interests masquerading as a contest of
principles. The conduct of public affairs for private advantage.” —
Ambrose Bierce[15]

I much prefer older edits of Wikipedia's opening paragraph. Here's few
excerpts

Politics consists of "social relations involving authority or
power"[1] and refers to the regulation of a political unit,[2] and to
the methods and tactics used to formulate and apply policy.[3]

...

In a broader sense, any situation involving power, or any
maneouvring in order to enhance one's power or status within a group,
may be described as politics (e.g. office politics).[3] This form of
politics "is most associated with a struggle for ascendancy among

Re: sqlite savepoint problem [solved]

[Laszlo Nagy]

No it doesn't.  The problem is that using a connection as a context
manager doesn't do what you think. 


It does *not* start a new transaction on __enter__ and commit it on
__exit__.  As far as I can tell it does nothing on __enter__ and calls
con.commit() or con.rollback() on exit.  With isolation_level=None,
these are no-ops.
  
Thank you Ryan! You are abolutely right, and thank you for reading the 
source.  Now everything works as I imagined.


The way the context manager and isolation_level works looks very very 
strange to me. Here is a demonstration:


import sqlite3
def getconn():
   conn = sqlite3.connect(':memory:')
   conn.isolation_level = None
   return conn
def main():
   with getconn() as conn:
   conn.execute("create table a ( i integer ) ")
   try:
   conn.execute("insert into a values (1)")
   with conn:
   conn.execute("insert into a values (2)")
   raise Exception
   except:
   print "There was an error"
   for row in conn.execute("select * from a"):
   print row
main()


Output:

There was an error
(1,)
(2,)


Looks like the context manager did not roll back anything. If I remove 
isolation_level=None then I get this:


There was an error

E.g. the context manager rolled back something that was executed outside 
the context. I cannot argue with the implementation - it is that way. 
But this is not what I would expect. I believe I'm not alone with this.


Using your connection manager, everything is perfect:

There was an error
(1,)


The only thing I have left is to implement a connection manager that 
emulates nested transactions, using a stack of savepoints. :-)


Suggestions:

Just for clarity, we should put a comment at the end of the 
documentation here:


http://docs.python.org/library/sqlite3.html#sqlite3-controlling-transactions

I would add at least these things:

#1. By using isolation_level = None, connection objects (used as a 
context manager) WON'T automatically commit or rollback transactions.
#2. Using any isolation level, connection objects WON'T automatically 
begin a transaction.
#3. Possibly, include your connection manager class code, to show how to 
do it "the expected" way.


Also one should clarify in the documentation, what isolation_level does. 
Looks like setting isolation_level to None is not really an "auto commit 
mode". It is not even part of sqlite itself. It is part of the python 
extension.


Thank you again.

  Laszlo

--
http://mail.python.org/mailman/listinfo/python-list

Re: sqlite savepoint problem [solved]

[Laszlo Nagy]

Annotating your example:

   # entering this context actually does nothing
   with conn:
  # a transaction is magically created before this statement
  conn.execute("insert into a values (1)")
  # and is implicitly committed before this statement
  conn.execute("SAVEPOINT sp1")
  # a new transaction is magically created
  conn.execute("insert into a values (2)")
  # and committed, discarding the first savepoint.
  conn.execute("SAVEPOINT sp2")
  # a new transaction is magically created
  conn.execute("insert into a values (3)")
  # and committed, discarding the very savepoint we are trying to use.
  conn.execute("ROLLBACK TO sp2")
  conn.execute("insert into a values (4)")
  conn.execute("RELEASE sp1")
   


We all know the Zen of Python. Explicit is better than implicit.

There is no point in using a savepoint outside a transaction. There is 
no point in using a savepoint if it commits all previous changes 
automatically.


Conclusion:

Sqlite's isolation_level is dark magic. It mixes real isolation levels 
with behaviour of context managers, and automagical commits in the wrong 
places.
Setting isolation_level=None is a must for anyone who want to do any 
serious work with sqlite.


   L



--
http://mail.python.org/mailman/listinfo/python-list

"Distributed" database in Python

[David Tynnhammar]

Greetings. I'm looking for a "distributed" database. (I'm not sure if
distributed is the correct terminology though).

My problem is this; I have a client application which once in a while
needs to sync with a central database. (And of course the client
-databases might be updated locally etc).

Are there any other resources you can recommend me? (I have basic RDMS
knowledge (MySQL)).
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Hacker News, Xahlee.Org, and What is Politics?

[Brian J Mingus]

On Mon, Mar 15, 2010 at 1:16 AM, Xah Lee  wrote:

> A essay related to the recent discussion of banning, and lisp
> associated group at ycombinator.com .


Is there some Python related issue I might help you out with? Or perhaps you
wish to provide Python assistance to someone on this list. Or perhaps you
would like to write some Python code, or read the friendly Python manual, or
draft a new PEP. Any one of these things would be a much better usage of
your time than drafting middle to high school quality essays.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Is it possible to use re2 from Python?

[Kev Dwyer]

On Sun, 14 Mar 2010 14:40:34 -0700, _wolf wrote:

>> There's a recent thread about this on the python-dev list,
> 
> pointers? i searched but didn’t find anything.

http://mail.python.org/pipermail/python-dev/2010-March/098354.html

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: sqlite savepoint problem [solved]

[Ryan Kelly]

On Fri, 2010-03-12 at 09:35 +0100, Laszlo Nagy wrote:
> > No it doesn't.  The problem is that using a connection as a context
> > manager doesn't do what you think. 
> >
> > It does *not* start a new transaction on __enter__ and commit it on
> > __exit__.  As far as I can tell it does nothing on __enter__ and calls
> > con.commit() or con.rollback() on exit.  With isolation_level=None,
> > these are no-ops.
> >   
> Thank you Ryan! You are abolutely right, and thank you for reading the 
> source.  Now everything works as I imagined.

No problemo - isolation_level has given me my fair share of headaches in
the past, so I couldn't resist the opportunity to understand it a little
better.

> The way the context manager and isolation_level works looks very very 
> strange to me. Here is a demonstration:
> 
> import sqlite3
> def getconn():
> conn = sqlite3.connect(':memory:')
> conn.isolation_level = None
> return conn
> def main():
> with getconn() as conn:
> conn.execute("create table a ( i integer ) ")
> try:
> conn.execute("insert into a values (1)")
> with conn:
> conn.execute("insert into a values (2)")
> raise Exception
> except:
> print "There was an error"
> for row in conn.execute("select * from a"):
> print row
> main()
> 
> 
> Output:
> 
> There was an error
> (1,)
> (2,)
>
> 
> Looks like the context manager did not roll back anything.

Yes, because there were no transactions created so there was nothing to
roll back.

>  If I remove 
> isolation_level=None then I get this:
> 
> There was an error
> 
> E.g. the context manager rolled back something that was executed outside 
> the context. 

Yes, because the transactions created by the default isolation level do
not nest, so the rollback happens at outermost scope.

> I cannot argue with the implementation - it is that way. 
> But this is not what I would expect. I believe I'm not alone with this.

That's at least two of us :-)

> Just for clarity, we should put a comment at the end of the 
> documentation here:
> 
> http://docs.python.org/library/sqlite3.html#sqlite3-controlling-transactions
> 
> I would add at least these things:
> 
> #1. By using isolation_level = None, connection objects (used as a 
> context manager) WON'T automatically commit or rollback transactions.
> #2. Using any isolation level, connection objects WON'T automatically 
> begin a transaction.
> #3. Possibly, include your connection manager class code, to show how to 
> do it "the expected" way.
> 
> Also one should clarify in the documentation, what isolation_level does. 
> Looks like setting isolation_level to None is not really an "auto commit 
> mode". It is not even part of sqlite itself. It is part of the python 
> extension.

I think of it as almost the opposite - you have to set
isolation_level=None to get the unadulterated behaviour of the
underlying sqlite library. 

I'm sure the devs would appreciate a documentation patch (submission
details at http://python.org/dev/patches/).  I'm also pretty confident
that I won't have time to do one up anytime soon :-)


 Good luck with your project!


 Ryan

-- 
Ryan Kelly
http://www.rfk.id.au  |  This message is digitally signed. Please visit
r...@rfk.id.au|  http://www.rfk.id.au/ramblings/gpg/ for details



signature.asc
Description: This is a digitally signed message part
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: "Distributed" database in Python

[Steven D'Aprano]

On Mon, 15 Mar 2010 08:49:34 +0100, David Tynnhammar wrote:

> Greetings. I'm looking for a "distributed" database. (I'm not sure if
> distributed is the correct terminology though).
> 
> My problem is this; I have a client application which once in a while
> needs to sync with a central database. (And of course the client
> -databases might be updated locally etc).
> 
> Are there any other resources you can recommend me? (I have basic RDMS
> knowledge (MySQL)).

Google and Wikipedia are your friends.

http://en.wikipedia.org/wiki/Distributed_database
http://www.google.com/search?q=distributed+database
http://www.google.com/search?q=python+distributed+database


If you have already tried this, then you should say so, so that we don't 
waste our time, and yours, going over ground you have already covered.



-- 
Steven
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: dll in project?

[Tim Golden]

On 15/03/2010 03:43, Alex Hall wrote:

I have a dll I am trying to use, but I get a Windows error 126, "the
specified module could not be found". Here is the code segment:
nvdaController=ctypes.windll.LoadLibrary("nvdaControllerClient32.dll")

I have the specified dll file in the same directory as the file trying
to use said dll


The DLL search path:

http://msdn.microsoft.com/en-us/library/7d83bc18%28VS.80%29.aspx

includes the directory which holds the executable for the current prcoess;
it include the current directory; and it includes other things which I
doubt apply here.

But it doesn't include (if I may be permitted a little well-intentioned
ridicule) the directory where the current Python module is stored.
In other words: what does os.getcwd () return?

TJG
--
http://mail.python.org/mailman/listinfo/python-list

staticmethod and setattr

[Michael.Lausch]

Hi,

I managed to get confused by Python, which is not such an easy task.

The problem i have is rooted in marshalling, JSON and Dojo.
I need some static class in function with the name "$ref"
i tried:
class Foo(object):
@staticmethod
def _ref(o):
 pass

setattr(Foo, "$ref", Foo._ref)

but when i do something like this in code:

class B(object):
pass
b = Bar
f = Foo.getattr("$ref")
f(b)

if get the following error
ypeError
unbound method _ref() must be called with Foo instance as first
argument (got Bar instance instead)

I managed to impelemnt it with a unbound, "normal" _ref() function,
but i want one _ref function per
class, because they are a little bit different for each class.

One aproach which i have not tried is:
setattr(Foo, "$ref", Foo._ref.im_func).
maybe this works, but i think this is not a clean aproach.
Any ideas?




-- 
http://mail.python.org/mailman/listinfo/python-list

Re: iterator/generator

[Mark Lawrence]

vsoler wrote:

I am working on a script that reads data from an excel workbook. The
data is located in a named range whose first row contains the headers.
It works!!!  I find it superb that python is able to do such things!!!

Now my questions.

a. My ranges can in practice be quite big, and although I am happy
with the speed I get, because I am learning python, I think I could do
it still a bit better. I think that the line   for i in matriz[1:]:
could be improved with an iterator/generator, but I am not quite
certain how I should proceed.


I write this :-
for j, i in enumerate(matriz):
if j:
etc...

If there's a better method for some definition of better I'm certain 
we'll quickly get told.


Please see the python docs for the enumerate built-in.

HTH.

Mark Lawrence.

[snip the rest]

--
http://mail.python.org/mailman/listinfo/python-list

Re: Hacker News, Xahlee.Org, and What is Politics?

[rantingrick]

Bravo!, Bravo!

A wonderful post xah, thanks for sharing and i hope it sinks in with
the people around here who need a lesson in humility. I was just a few
days ago feeling quite passionate about this subject and creating my
own draft but than i lost interest because i knew nobody would even
care and probably just accuse me of trolling. Anyhow thanks for this!
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: sqlite savepoint problem [solved]

[Laszlo Nagy]

#1. By using isolation_level = None, connection objects (used as a 
context manager) WON'T automatically commit or rollback transactions.
#2. Using any isolation level, connection objects WON'T automatically 
begin a transaction.
#3. Possibly, include your connection manager class code, to show how to 
do it "the expected" way.


Also one should clarify in the documentation, what isolation_level does. 
Looks like setting isolation_level to None is not really an "auto commit 
mode". It is not even part of sqlite itself. It is part of the python 
extension.



I think of it as almost the opposite - you have to set
isolation_level=None to get the unadulterated behaviour of the
underlying sqlite library. 


I'm sure the devs would appreciate a documentation patch (submission
details at http://python.org/dev/patches/).  I'm also pretty confident
that I won't have time to do one up anytime soon :-)
  

Patch submitted.

http://bugs.python.org/issue8145


--
http://mail.python.org/mailman/listinfo/python-list

Re: Python, Reportlabs, Pil and Windows 7 (64bit)

[Robin Becker]

On 12/03/2010 19:29, "Martin v. Löwis" wrote:

Not sure if this is a bug


I think it is. It seems that the cross-build support in msvc9compiler
has been tested only in a build tree of Python (where there is no Libs
directory).


This minor patch seems to fix the problem for me (using a PCBuild folder 
parallel to libs)


C:\Python\Lib\distutils\command>diff build_ext.py.orig build_ext.py
207c207,209
< self.library_dirs.append(new_lib)
---
> self.library_dirs.insert(0,new_lib)
> else:
> self.library_dirs.append(new_lib)



For released copies of Python, I could change that to distribute the
AMD64 pythonXY.lib in libs/amd64. [FWIW, I'm still puzzled why I ship
the import libraries of all the pyd files as well - I can't see a reason
other than tradition]. Then, distutils should change to look it up there.

...

Just checked and all the pyd's seem only to export the xxx_init functions 
(sometimes there are two in the pyd) so the .libs do seem a bit redundant.


--
Robin Becker

--
http://mail.python.org/mailman/listinfo/python-list

Re: Reverse engineering CRC?

[Gregory Ewing]

I've solved the problem now.

It turned out to be a very standard CRC algorithm, complicated
by the presence of a few extra bytes of data being checked that
didn't appear explicitly in the file anywhere.

In the process I developed some very general techniques for
solving this kind of problem, which I've written about here
if anyone's interested:

http://www.cosc.canterbury.ac.nz/greg.ewing/essays/CRC-Reverse-Engineering.html

Thanks for everyone's help,
Greg
--
http://mail.python.org/mailman/listinfo/python-list

Re: dll in project?

[Ulrich Eckhardt]

Alex Hall wrote:
> I have a dll I am trying to use, but I get a Windows error 126, "the
> specified module could not be found". Here is the code segment:
> nvdaController=ctypes.windll.LoadLibrary("nvdaControllerClient32.dll")

In addition to Alf's answer, this can also happen when the OS can't find
another DLL that this one depends on.

Uli

-- 
Sator Laser GmbH
Geschäftsführer: Thorsten Föcking, Amtsgericht Hamburg HR B62 932

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: staticmethod and setattr

[Steven D'Aprano]

On Mon, 15 Mar 2010 01:43:02 -0700, Michael.Lausch wrote:

> Hi,
> 
> I managed to get confused by Python, which is not such an easy task.
> 
> The problem i have is rooted in marshalling, JSON and Dojo. I need some
> static class in function with the name "$ref" i tried:
> class Foo(object):
> @staticmethod
> def _ref(o):
>  pass
> 
> setattr(Foo, "$ref", Foo._ref)

That doesn't work as expected:

>>> Foo.__dict__['_ref'] is Foo.__dict__['$ref']
False


Try this instead:

>>> setattr(Foo, "$ref", Foo.__dict__['_ref'])
>>> Foo.__dict__['_ref'] is Foo.__dict__['$ref']
True




-- 
Steven
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: File existence check with partial filename

[Lan Qing]

Or use the regular module:

import re
import os

for filename in os.listdir('.'):
if re.match("*HV*", filename):
  # do something with the file

On Mon, Mar 15, 2010 at 12:24 PM, Alf P. Steinbach  wrote:

> * Sang-Ho Yun:
>
>  I learned that I can check the existence of a file using
>> os.path.isfile("filename").
>>
>> What if I need to check if there is a file that contains "HV" in the
>> filename?  What should I do?
>>
>
> 
>from __future__ import print_function
>import os

   for filename in os.listdir( "." ):
>if "HV" in filename.upper():
>print( filename )
> 
>
>
> Cheers & hth.,
>
> - Alf
>
> --
> http://mail.python.org/mailman/listinfo/python-list
>
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: dll in project?

[Alex Hall]

On 3/15/10, Ulrich Eckhardt  wrote:
> Alex Hall wrote:
>> I have a dll I am trying to use, but I get a Windows error 126, "the
>> specified module could not be found". Here is the code segment:
>> nvdaController=ctypes.windll.LoadLibrary("nvdaControllerClient32.dll")
>
> In addition to Alf's answer, this can also happen when the OS can't find
> another DLL that this one depends on.

Well, os.getcwd() returns "c:\python26", not my program's directory.
However, I changed the reference to the dll to be
helpers.progdir+'\\nvdaControllerClient32.dll'
and still no luck! helpers.progdir is a var holding the top-level
directory of my project, using os.path. Again, using this more precise
reference still fails, triggering my except statement in my try/catch
loop.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: dll in project?

[Alf P. Steinbach]

* Alex Hall:

On 3/15/10, Ulrich Eckhardt  wrote:

Alex Hall wrote:

I have a dll I am trying to use, but I get a Windows error 126, "the
specified module could not be found". Here is the code segment:
nvdaController=ctypes.windll.LoadLibrary("nvdaControllerClient32.dll")

In addition to Alf's answer, this can also happen when the OS can't find
another DLL that this one depends on.


Well, os.getcwd() returns "c:\python26", not my program's directory.
However, I changed the reference to the dll to be
helpers.progdir+'\\nvdaControllerClient32.dll'
and still no luck!


Are you sure you get a single directory separator backslash there?

I suggest using the os.path 'join' function, whatever it was called.



helpers.progdir is a var holding the top-level
directory of my project, using os.path.


But is that where the DLL resides, or is it in some subdirectory?



Again, using this more precise
reference still fails, triggering my except statement in my try/catch
loop.


Try also checking the arguments & documentation of ctypes.windll.LoadLibrary. I 
know, you probably already done that. But I mention it just in case (I'm not 
familiar with that function at the Python level, so don't know about the args).



Cheers & hth.,

- Alf
--
http://mail.python.org/mailman/listinfo/python-list

Re: dll in project?

[Ulrich Eckhardt]

Alex Hall wrote:
> On 3/15/10, Ulrich Eckhardt  wrote:
>> Alex Hall wrote:
>>> I have a dll I am trying to use, but I get a Windows error 126, "the
>>> specified module could not be found". Here is the code segment:
>>> nvdaController=ctypes.windll.LoadLibrary("nvdaControllerClient32.dll")
>>
>> In addition to Alf's answer, this can also happen when the OS can't find
>> another DLL that this one depends on.

Did you check if this could be the case?

> Well, os.getcwd() returns "c:\python26", not my program's directory.
> However, I changed the reference to the dll to be
> helpers.progdir+'\\nvdaControllerClient32.dll'
> and still no luck!

Generally, there is os.path.join() IIRC which does this portably. This
probably doesn't matter though. What I would check is firstly if this file
could be opened at all, e.g. using os.stat().

> helpers.progdir is a var holding the top-level directory of my
> project, using os.path.

Huh? In what way using os.path?

> Again, using this more precise reference still fails, triggering my
> except statement in my try/catch loop.

Same error? See my initial guess! As a tool for finding out if there are
missing dependencies, take a look at http://dependencywalker.com

BTW: No need to CC me, I read your initial request here, I can ready any
follow-ups here, too. ;)

Uli

-- 
Sator Laser GmbH
Geschäftsführer: Thorsten Föcking, Amtsgericht Hamburg HR B62 932

-- 
http://mail.python.org/mailman/listinfo/python-list

Large regular expressions

[Nathan Harmston]

Hi,

So I m trying to use a very large regular expression, basically I have
a list of items I want to find in text, its kind of a conjunction of
two regular expressions and a big list..not pretty. However
everytime I try to run my code I get this exception:

OverflowError: regular expression code size limit exceeded

I understand that there is a Python imposed limit on the size of the
regular expression. And although its not nice I have a machine with
12Gb of RAM just waiting to be used, is there anyway I can alter
Python to allow big regular expressions?

Could anyone suggest other methods of these kind of string matching in
Python? I m trying to see if my swigged alphabet trie is faster than
whats possible in Python!

Many thanks,


Nathan
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: staticmethod and setattr

[Michael.Lausch]

On Mar 15, 11:40 am, Steven D'Aprano  wrote:
> On Mon, 15 Mar 2010 01:43:02 -0700, Michael.Lausch wrote:
> > Hi,
>
> > I managed to get confused by Python, which is not such an easy task.
>
> > The problem i have is rooted in marshalling, JSON and Dojo. I need some
> > static class in function with the name "$ref" i tried:
> > class Foo(object):
> >     @staticmethod
> >     def _ref(o):
> >          pass
>
> > setattr(Foo, "$ref", Foo._ref)
>
> That doesn't work as expected:
>
> >>> Foo.__dict__['_ref'] is Foo.__dict__['$ref']
>
> False
>
> Try this instead:
>
> >>> setattr(Foo, "$ref", Foo.__dict__['_ref'])
> >>> Foo.__dict__['_ref'] is Foo.__dict__['$ref']
>
> True

Now I'm trying to understand why this is the case.
How is Foo.__dict__['_ref']  different from Foo._ref?
Shouldn't it return the same attribute?

And after further experiments i found out that a making
Foo._ref a classmethod does work with my first approach.

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Large regular expressions

[Stefan Behnel]

Nathan Harmston, 15.03.2010 13:21:

So I m trying to use a very large regular expression, basically I have
a list of items I want to find in text, its kind of a conjunction of
two regular expressions and a big list..not pretty. However
everytime I try to run my code I get this exception:

OverflowError: regular expression code size limit exceeded

I understand that there is a Python imposed limit on the size of the
regular expression. And although its not nice I have a machine with
12Gb of RAM just waiting to be used, is there anyway I can alter
Python to allow big regular expressions?

Could anyone suggest other methods of these kind of string matching in
Python?


If what you are trying to match is in fact a set of strings instead of a 
set of regular expressions, you might find this useful:


http://pypi.python.org/pypi/acora

Stefan

--
http://mail.python.org/mailman/listinfo/python-list

Re: Large regular expressions

[Alain Ketterlin]

Nathan Harmston  writes:

[...]
> Could anyone suggest other methods of these kind of string matching in
> Python? I m trying to see if my swigged alphabet trie is faster than
> whats possible in Python!

Since you mention using a trie, I guess it's just a big alternative of
fixed strings. You may want to try using the Aho-Corasick variant. It
looks like there are several implementations (google finds at least
two). I would be surprised if any pure python solution were faster than
tries implemented in C. Don't forget to tell us your findings.

-- Alain.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: dll in project?

[Alex Hall]

Okay, I got a new copy and all seems well now. The dll is found and
loaded. The functions inside the dll are not working, but that is not
Python's fault. Thanks to everyone for your help and suggestions!

On 3/15/10, Ulrich Eckhardt  wrote:
> Alex Hall wrote:
>> On 3/15/10, Ulrich Eckhardt  wrote:
>>> Alex Hall wrote:
 I have a dll I am trying to use, but I get a Windows error 126, "the
 specified module could not be found". Here is the code segment:
 nvdaController=ctypes.windll.LoadLibrary("nvdaControllerClient32.dll")
>>>
>>> In addition to Alf's answer, this can also happen when the OS can't find
>>> another DLL that this one depends on.
>
> Did you check if this could be the case?
>
>> Well, os.getcwd() returns "c:\python26", not my program's directory.
>> However, I changed the reference to the dll to be
>> helpers.progdir+'\\nvdaControllerClient32.dll'
>> and still no luck!
>
> Generally, there is os.path.join() IIRC which does this portably. This
> probably doesn't matter though. What I would check is firstly if this file
> could be opened at all, e.g. using os.stat().
>
>> helpers.progdir is a var holding the top-level directory of my
>> project, using os.path.
>
> Huh? In what way using os.path?
>
>> Again, using this more precise reference still fails, triggering my
>> except statement in my try/catch loop.
>
> Same error? See my initial guess! As a tool for finding out if there are
> missing dependencies, take a look at http://dependencywalker.com
>
> BTW: No need to CC me, I read your initial request here, I can ready any
> follow-ups here, too. ;)
>
> Uli
>
> --
> Sator Laser GmbH
> Geschäftsführer: Thorsten Föcking, Amtsgericht Hamburg HR B62 932
>
> --
> http://mail.python.org/mailman/listinfo/python-list
>


-- 
Have a great day,
Alex (msg sent from GMail website)
mehg...@gmail.com; http://www.facebook.com/mehgcap
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Dreaming of new generation IDE

[catonano]

Hello again, people

On Feb 11, 6:30 pm, Francis Carr  wrote:
> > I can't believe the code editing situation today is in a such sorry
> > state.
>
> I can't believe an old coder is feeling so sorry for himself.

Ok, I'm feeling sorry; still, I think I made a point.


> > Today, I tried to understand the twisted.web.client code and I found 3
> Maybe the relevant lesson to be taken from Smalltalk is *not*
>   "put it all in one image"
> but instead
>   "write code to solve problems, e.g., reading code"


> I suggest defining a few shell functions to *automate* the search, for
> example in zsh
>
> function allf () {

> Smalltalk's images are cool.  But they are such a huge hammer, and
> they impose so many additional requirements!  The costs outweigh the
> cool.

Francis, I disagree on the final balance statement about the Smalltalk
images and I thank you for your suggestions on how to use grep and
shell scripting.

I also have no problem to admit I was feeling sorry for myself.

But today I found something I think is relevant to this thread.

It's here:
http://www.cs.brown.edu/people/acb/codebubbles_site.htm

it's about a IDE for Java called Bubbles.

It sports a zooming interface and the MAIN idea it proposes, in my
opinion, is that the code is NOT thought as files content, rather as
fragments, called bubbles.

And then you arrange your bubbles in a 2d space a little bit like in a
videogame.

I think it's very similar to the Smalltalk image thing (as a concept,
if not as an implementation), with its live things and direct
manipulation.

If you open the footage directly in youtube, you can go to some
"related" footages about a similar thing made by Microsoft that of
course ditches the main idea retaining the concept of files and
directories, overlapping the concept of files and classes (sigh).

I didn't imagine such a research existed and the discovery confirms to
me that the bother I indicated in this thread is somewhat shared by
someone, around the world.

As for my feelings, I swear I'm taking care of them in a way that
doesn't involve bothering anyone on any newsgroup ;-)

Steve, does Bubble gets any closer to your vision than the "ugly"
Smalltalk thing ?

Bye
Catonano
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Understanding the CPython dict implementation

[Alex Willmer]

On Mar 15, 4:06 am, John Nagle  wrote:
>     Is this available as a paper?
>
>                                 John Nagle

It doesn't wppear to be, slides are here:

http://us.pycon.org/2010/conference/schedule/event/12/

Alex
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: staticmethod and setattr

[Bruno Desthuilliers]

Michael.Lausch a écrit :
(snip)



Now I'm trying to understand why this is the case.
How is Foo.__dict__['_ref']  different from Foo._ref?
Shouldn't it return the same attribute?



It's an application of the descriptor protocol:

http://wiki.python.org/moin/FromFunctionToMethod
--
http://mail.python.org/mailman/listinfo/python-list

Re: staticmethod and setattr

[Andreas Löscher]

Am Montag, den 15.03.2010, 05:42 -0700 schrieb Michael.Lausch:
> On Mar 15, 11:40 am, Steven D'Aprano  cybersource.com.au> wrote:
> > On Mon, 15 Mar 2010 01:43:02 -0700, Michael.Lausch wrote:
> > > Hi,
> >
> > > I managed to get confused by Python, which is not such an easy task.
> >
> > > The problem i have is rooted in marshalling, JSON and Dojo. I need some
> > > static class in function with the name "$ref" i tried:
> > > class Foo(object):
> > > @staticmethod
> > > def _ref(o):
> > >  pass
> >
> > > setattr(Foo, "$ref", Foo._ref)
> >
> > That doesn't work as expected:
> >
> > >>> Foo.__dict__['_ref'] is Foo.__dict__['$ref']
> >
> > False
> >
> > Try this instead:
> >
> > >>> setattr(Foo, "$ref", Foo.__dict__['_ref'])
> > >>> Foo.__dict__['_ref'] is Foo.__dict__['$ref']
> >
> > True
> 
> Now I'm trying to understand why this is the case.
> How is Foo.__dict__['_ref']  different from Foo._ref?
> Shouldn't it return the same attribute?
> 
> And after further experiments i found out that a making
> Foo._ref a classmethod does work with my first approach.
> 

In the class dictionary are the "raw" objects stored. For example:

class Spam:
@staticmethod
def egg1():
pass
def egg2(self):
pass

>>> Spam.__dict__
{'egg1': , '__module__':
'__main__', 'egg2': , '__doc__': None}

If you try to call egg1 as staticmethod you will get an error:
>>> Spam.__dict__['egg1']()
Traceback (most recent call last):
  File "", line 1, in 
TypeError: 'staticmethod' object is not callable

getattr() is not only a shortcut for the line above. It searches all
base classes for the specified name in the class dictionaries. If the
name is found, and the object has an additional descriptor, it is
applied on the object itself.

In case of an static method, the descriptor sm_descr_get is located in
Objects/funcobjects.c of the sources and it retriefes the encapsuled
callable, which in this case is the function egg1.

If you now set the function as new attribute, it is no longer a static
method, but a normal function. If a function is retrieved via getattr(),
it is encapsuled in an method object. (func_descr_get in the same file)
If you now try to call the method in you example, you call a method from
a class. This results in an unbound method, which requests an instance
of the class as first agument. 

I hope this clears some things up.

Best
Andreas

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Python for newbies (Pythno users group in Macedonia)

[Дамјан Георгиевски]

> If you are all English-speakers then perhaps you could consider
> showing a PyCon video - see
> 
>   http://pycon.blip.tv

That's certainly something I'm considering for the more advanced users

> Good luck with the group. I hope to see PyCon Macedonia emerging
> before too long!

Thanks :)
I guess we'd have to wait a decade or so for a PyCon here 

-- 
damjan
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Python for newbies (Pythno users group in Macedonia)

[Дамјан Георгиевски]

>> we are starting with bi-monthly Python User Group meetings in Skopje,
>> Macedonia. The meetings are targeted for both beginners and more
>> experienced users.
>>   
> ...
> http://us.pycon.org/2010/conference/schedule/event/108/

Great resource, exactly what I needed.

So, they use this book http://www.greenteapress.com/thinkpython/ as a 
guide, seems like a good choice.

I also learned about the tutor mail list:
http://mail.python.org/mailman/listinfo/tutor


-- 
damjan
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: File existence check with partial filename

[MRAB]

Lan Qing wrote:

Or use the regular module:

import re
import os

for filename in os.listdir('.'):
if re.match("*HV*", filename):
  # do something with the file


The regular expression should be ".*HV.*", although:

re.search("HV", filename)

would be better and:

"HV" in filename.upper()

would be even better, as Alf posted.

On Mon, Mar 15, 2010 at 12:24 PM, Alf P. Steinbach > wrote:


* Sang-Ho Yun:

I learned that I can check the existence of a file using
os.path.isfile("filename").

What if I need to check if there is a file that contains "HV" in the
filename?  What should I do?



   from __future__ import print_function
   import os

   for filename in os.listdir( "." ):
   if "HV" in filename.upper():
   print( filename )



Cheers & hth.,

- Alf

-- 
http://mail.python.org/mailman/listinfo/python-list





--
http://mail.python.org/mailman/listinfo/python-list

EuroPython 2010 - Open for registration and reminder of participation

[Alex Willmer]

EuroPython 2010 - 17th to 24th July 2010


EuroPython is a conference for the Python programming language
community, including the Django, Zope and Plone communities. It is
aimed at everyone in the Python community, of all skill levels, both
users and programmers.

Last year's conference was the largest open source conference in the
UK and one of the largest community organised software conferences in
Europe.

This year EuroPython will be held from the 17th to 24th July in
Birmingham, UK. It will include over 100 talks, tutorials, sprints and
social events.

Registration


Registration is open now at: http://www.europython.eu/registration/

For the best registration rates, book as soon as you can! Extra Early
Bird closes soon, after which normal Early Bird rate will apply until
10th May

Talks, Activities and Events


Do you have something you wish to present at EuroPython? You want to
give a talk, run a tutorial or sprint?

Go to http://www.europython.eu/talks/cfp/ for information and advice!
Go to http://wiki.europython.eu/Sprints to plan a sprint!

Help Us Out
---

EuroPython is run by volunteers, like you! We could use a hand, and
any contribution is welcome.

Go to http://wiki.europython.eu/Helping to join us!
Go to http://www.europython.eu/contact/ to contact us directly!

Sponsors


Sponsoring EuroPython is a unique opportunity to affiliate with this
prestigious conference and to reach a large number of Python users
from computing professionals to academics, from entrepreneurs to
motivated and well-educated job seekers.

http://www.europython.eu/sponsors/

Spread the Word
---

We are a community-run not-for-profit conference. Please help to
spread the word by distributing this announcement to colleagues,
project mailing lists, friends, your blog, Web site, and through your
social networking connections. Take a look at our publicity resources:

http://wiki.europython.eu/Publicity

General Information
---

For more information about the conference, please visit the official
site: http://www.europython.eu/

Looking forward to see you!

The EuroPython Team
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Large regular expressions

[MRAB]

Nathan Harmston wrote:

Hi,

So I m trying to use a very large regular expression, basically I have
a list of items I want to find in text, its kind of a conjunction of
two regular expressions and a big list..not pretty. However
everytime I try to run my code I get this exception:

OverflowError: regular expression code size limit exceeded

I understand that there is a Python imposed limit on the size of the
regular expression. And although its not nice I have a machine with
12Gb of RAM just waiting to be used, is there anyway I can alter
Python to allow big regular expressions?

Could anyone suggest other methods of these kind of string matching in
Python? I m trying to see if my swigged alphabet trie is faster than
whats possible in Python!


There's the regex module at http://pypi.python.org/pypi/regex. It'll
even release the GIL while matching on strings! :-)
--
http://mail.python.org/mailman/listinfo/python-list

to pass self or not to pass self

[lallous]

Hello,

Learning Python from the help file and online resources can leave one
with many gaps. Can someone comment on the following:

# -
class X:
T = 1

def f1(self, arg):
print "f1, arg=%d" % arg
def f2(self, arg):
print "f2, arg=%d" % arg
def f3(self, arg):
print "f3, arg=%d" % arg

# this:
F = f2
# versus this:
func_tbl = { 1: f1, 2: f2, 3: f3 }

def test1(self, n, arg):
# why passing 'self' is needed?
return self.func_tbl[n](self, arg)

def test2(self):
f = self.f1
f(6)

f = self.F
# why passing self is not needed?
f(87)

# -
x = X()

x.test1(1, 5)
print '--'
x.test2()

Why in test1() when it uses the class variable func_tbl we still need
to pass self, but in test2() we don't ?

What is the difference between the reference in 'F' and 'func_tbl' ?

Thanks,
Elias
-- 
http://mail.python.org/mailman/listinfo/python-list

Arguments and a return value of a C function call when using setprofile

[Michal Kwiatkowski]

Hi,

I'm trying to write code that will trace arguments and return values
of all function calls. Using sys.settrace with 'call' and 'return'
events works great for Python functions, but now I want to extend that
to C functions as well. Using sys.setprofile instead in theory gives
me what I need ('c_call' and 'c_return'), but I'm stuck on getting
actual arguments and return values.

A snippet of code is worth a thousand words, so here it is:

import inspect
import sys

def trace(frame, event, arg):
if event == 'call':
print "CALL", frame.f_code.co_name, "arguments:",
inspect.getargvalues(frame)
elif event == 'return':
print "RETURN", frame.f_code.co_name, "return value:", arg
elif event == 'c_call':
print "C_CALL", arg.__name__, "???"
elif event == 'c_return':
print "C_RETURN", arg.__name__, "???"

def func(x):
return x+1

sys.setprofile(trace)
func(1) # Python function
repr(2) # C function
sys.setprofile(None)

For a 'c_call' event the trace function is called *before* the call,
so frame is still in the context of the caller. 'c_return' event is
invoked *after* the function has returned, so I don't get the return
value here either. That's why I can't use inspect.getargvalues in
those cases. Is there any other way I could get arguments and a return
value of a C function call?

Cheers,
mk
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Python for newbies (Pythno users group in Macedonia)

[Steve Holden]

Дамјан Георгиевски wrote:
>> If you are all English-speakers then perhaps you could consider
>> showing a PyCon video - see
>>
>>   http://pycon.blip.tv
> 
> That's certainly something I'm considering for the more advanced users
> 
>> Good luck with the group. I hope to see PyCon Macedonia emerging
>> before too long!
> 
> Thanks :)
> I guess we'd have to wait a decade or so for a PyCon here 
> 
You might be surprised. Python is really taking off right now.

regards
 Steve
-- 
Steve Holden   +1 571 484 6266   +1 800 494 3119
See PyCon Talks from Atlanta 2010  http://pycon.blip.tv/
Holden Web LLC http://www.holdenweb.com/
UPCOMING EVENTS:http://holdenweb.eventbrite.com/

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Fwd: Some PyCon videos won't play

[Steve Holden]

Lan Qing wrote:
> In China these video can not watch at all. I must spent 3 days to
> download it...
> 
That's a great pity. If they would be useful to a large population
perhaps we could consider having copies hosted where they would be
available with higher bandwidth?

Who could we (the Python Software Foundation) discuss such a project with?

regards
 Steve

> On Mon, Mar 15, 2010 at 5:19 AM, Terry Reedy  > wrote:
> 
> On 3/14/2010 2:41 PM, Terry Reedy wrote:
> 
> On 3/14/2010 11:14 AM, David Boddie wrote:
> 
> 
> You should still be able to get at the videos themselves by
> inspecting
> the page contents, looking for download links like this one:
> 
> 
> http://blip.tv/file/get/Pycon-PyCon2010UsingPythonToCreateRoboticSimulationsForPlaneta894.ogv
> 
> 
> In this case, it's an Ogg Theora video file, so you may want
> to use
> a player like VLC to view it:
> 
> 
> For people using html5 competant browsers, which, I believe,
> play .ogv
> 'in tha page', blip.tv  just needs to update
> site to make html5 pages an
> alternative to using the flash player.
> 
> 
> Actually, for FF3.6, at least, the problem is already solved. I just
> click on the download button and FF recognizes .ogv as something it
> can play and it does, replacing the flash player. The FF player
> lacks a full-screen button, but that is rarely needed for the talks.
> 
> tjr
> 
> 
> 
> 
> -- 
> http://mail.python.org/mailman/listinfo/python-list
> 
> 


-- 
Steve Holden   +1 571 484 6266   +1 800 494 3119
See PyCon Talks from Atlanta 2010  http://pycon.blip.tv/
Holden Web LLC http://www.holdenweb.com/
UPCOMING EVENTS:http://holdenweb.eventbrite.com/

-- 
http://mail.python.org/mailman/listinfo/python-list

extract occurrence of regular expression from elements of XML documents

[Martin Schmidt]

Hi,

I have just started to use Python a few weeks ago and until last week I had
no knowledge of XML.
Obviously my programming knowledge is pretty basic.
Now I would like to use Python in combination with ca. 2000 XML documents
(about 30 kb each) to search for certain regular expression within specific
elements of these documents.
I would then like to record the number of occurrences of the regular
expression within these elements.
Moreover I would like to count the total number of words contained within
these, and record the attribute of a higher level element that contains
them.
I was trying to figure out the best way how to do this, but got overwhelmed
by the available information (e.g. posts using different approaches based on
dom, sax, xpath, elementtree, expat).
The outcome should be a file that lists the extracted attribute, the number
of occurrences of the regular expression, and the total number of words.
I did not find a post that addresses my problem.
If someone could help me with this I would really appreciate it.

  Martin
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: File existence check with partial filename

[Steven Howe]

What wrong with glob?
---
Help on module glob:

NAME
glob - Filename globbing utility.

FILE
/usr/lib64/python2.6/glob.py

FUNCTIONS
glob(pathname)
Return a list of paths matching a pathname pattern.

The pattern may contain simple shell-style wildcards a la fnmatch.

iglob(pathname)
Return an iterator which yields the paths matching a pathname 
pattern.


The pattern may contain simple shell-style wildcards a la fnmatch.

DATA
__all__ = ['glob', 'iglob']

---
solution:

import glob

for afile in glob.iglob( "/home/buddy/*.HV" ):
print afile


sph


On 03/15/2010 08:44 AM, MRAB wrote:

Lan Qing wrote:

Or use the regular module:

import re
import os

for filename in os.listdir('.'):
if re.match("*HV*", filename):
  # do something with the file


The regular expression should be ".*HV.*", although:

re.search("HV", filename)

would be better and:

"HV" in filename.upper()

would be even better, as Alf posted.

On Mon, Mar 15, 2010 at 12:24 PM, Alf P. Steinbach > wrote:


* Sang-Ho Yun:

I learned that I can check the existence of a file using
os.path.isfile("filename").

What if I need to check if there is a file that contains "HV" 
in the

filename?  What should I do?



   from __future__ import print_function
   import os

   for filename in os.listdir( "." ):
   if "HV" in filename.upper():
   print( filename )



Cheers & hth.,

- Alf

-- http://mail.python.org/mailman/listinfo/python-list






--
http://mail.python.org/mailman/listinfo/python-list

Re: extract occurrence of regular expression from elements of XML documents

[Steve Holden]

Martin Schmidt wrote:
> Hi,
> 
> I have just started to use Python a few weeks ago and until last week I
> had no knowledge of XML.
> Obviously my programming knowledge is pretty basic.
> Now I would like to use Python in combination with ca. 2000 XML
> documents (about 30 kb each) to search for certain regular expression
> within specific elements of these documents.
> I would then like to record the number of occurrences of the regular
> expression within these elements.
> Moreover I would like to count the total number of words contained
> within these, and record the attribute of a higher level element that
> contains them.
> I was trying to figure out the best way how to do this, but got
> overwhelmed by the available information (e.g. posts using different
> approaches based on dom, sax, xpath, elementtree, expat).
> The outcome should be a file that lists the extracted attribute, the
> number of occurrences of the regular expression, and the total number of
> words.
> I did not find a post that addresses my problem.
> If someone could help me with this I would really appreciate it.
> 
You would get more specific help if you could post an example of the XML
and describe the regex searching you want to do in a little more detail,
I suspect.

regards
 Steve
-- 
Steve Holden   +1 571 484 6266   +1 800 494 3119
See PyCon Talks from Atlanta 2010  http://pycon.blip.tv/
Holden Web LLC http://www.holdenweb.com/
UPCOMING EVENTS:http://holdenweb.eventbrite.com/

-- 
http://mail.python.org/mailman/listinfo/python-list

subtraction is giving me a syntax error

[Joel Pendery]

So I am trying to write a bit of code and a simple numerical
subtraction

y_diff = y_diff-H

is giving me the error

Syntaxerror: Non-ASCII character '\x96' in file on line 70, but no
encoding declared.

Even though I have deleted some lines before it and this line is no
longer line 70, I am still getting the error every time.  I have tried
to change the encoding of the file to utf-8 but to no avail, I still
am having this issue.  Any ideas?

Thanks in advance
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: to pass self or not to pass self

[TomF]

On 2010-03-15 09:39:50 -0700, lallous  said:


Hello,

Learning Python from the help file and online resources can leave one
with many gaps. Can someone comment on the following:

# -
class X:
T = 1

def f1(self, arg):
print "f1, arg=%d" % arg
def f2(self, arg):
print "f2, arg=%d" % arg
def f3(self, arg):
print "f3, arg=%d" % arg

# this:
F = f2
# versus this:
func_tbl = { 1: f1, 2: f2, 3: f3 }

def test1(self, n, arg):
# why passing 'self' is needed?
return self.func_tbl[n](self, arg)

def test2(self):
f = self.f1
f(6)

f = self.F
# why passing self is not needed?
f(87)

# -
x = X()

x.test1(1, 5)
print '--'
x.test2()

Why in test1() when it uses the class variable func_tbl we still need
to pass self, but in test2() we don't ?

What is the difference between the reference in 'F' and 'func_tbl' ?


I recommend putting print statements into your code like this:

   def test1(self, n, arg):
   print "In test1, I'm calling a %s" % self.func_tbl[n]
   return self.func_tbl[n](self, arg)

   def test2(self):
   f = self.f1
   print "Now in test2, I'm calling a %s" % f
   f(6)


Bottom line: You're calling different things.  Your func_tbl is a dict 
of functions, not methods.


-Tom

--
http://mail.python.org/mailman/listinfo/python-list

Re: subtraction is giving me a syntax error

[Richard Brodie]

"Joel Pendery"  wrote in message 
news:56597268-3472-4fd9-a829-6d9cf51cf...@e7g2000yqf.googlegroups.com...

>> y_diff = y_diff-H
>
> Syntaxerror: Non-ASCII character '\x96' in file on line 70, but no
> encoding declared.

That's likely an en-dash, not a minus sign. 


-- 
http://mail.python.org/mailman/listinfo/python-list

Re: subtraction is giving me a syntax error

[Benjamin Kaplan]

On Mon, Mar 15, 2010 at 1:37 PM, Joel Pendery wrote:

> So I am trying to write a bit of code and a simple numerical
> subtraction
>
> y_diff = y_diff-H
>
> is giving me the error
>
> Syntaxerror: Non-ASCII character '\x96' in file on line 70, but no
> encoding declared.
>
> Even though I have deleted some lines before it and this line is no
> longer line 70, I am still getting the error every time.  I have tried
> to change the encoding of the file to utf-8 but to no avail, I still
> am having this issue.  Any ideas?
>
> Thanks in advance
>

It's not UTF-8. My guess would be that you're on Windows and it's a dash
character in cp1252. The actual minus sign is 0x2D. Your editor must be
putting in the wrong character.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: subtraction is giving me a syntax error

[Philip Semanchuk]

On Mar 15, 2010, at 1:37 PM, Joel Pendery wrote:


So I am trying to write a bit of code and a simple numerical
subtraction

y_diff = y_diff-H

is giving me the error

Syntaxerror: Non-ASCII character '\x96' in file on line 70, but no
encoding declared.

Even though I have deleted some lines before it and this line is no
longer line 70, I am still getting the error every time.  I have tried
to change the encoding of the file to utf-8 but to no avail, I still
am having this issue.  Any ideas?



0x96 is the Win1252 minus sign character. Did you copy & paste this  
code from a Web browser or a word processor?


Delete the character between "y_diff" and "H" and replace it with a  
plain ASCII subtraction sign.






--
http://mail.python.org/mailman/listinfo/python-list

Re: to pass self or not to pass self

[Rami Chowdhury]

On Monday 15 March 2010 10:42:41 TomF wrote:
> On 2010-03-15 09:39:50 -0700, lallous  said:
> > 
> > Why in test1() when it uses the class variable func_tbl we still need
> > to pass self, but in test2() we don't ?
> > 
> > What is the difference between the reference in 'F' and 'func_tbl' ?
> 
> I recommend putting print statements into your code like this:
> 
> def test1(self, n, arg):
> print "In test1, I'm calling a %s" % self.func_tbl[n]
> return self.func_tbl[n](self, arg)
> 
> def test2(self):
> f = self.f1
> print "Now in test2, I'm calling a %s" % f
> f(6)
> 
> 
> Bottom line: You're calling different things.  Your func_tbl is a dict
> of functions, not methods.
> 
> -Tom

To build on that a bit, note that in test2() you are doing:
> > f = self.f1
> > f(6)
> > 
> > f = self.F
> > # why passing self is not needed?
> > f(87)

As I understand it, since you obtained the reference to 'f1' from 'self', you 
got it as a bound rather than an unbound method. So 'self' is automatically 
passed in as the first argument. 


Rami Chowdhury
"Given enough eyeballs, all bugs are shallow." -- Linus' Law
408-597-7068 (US) / 07875-841-046 (UK) / 01819-245544 (BD)
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: subtraction is giving me a syntax error

[MRAB]

Joel Pendery wrote:

So I am trying to write a bit of code and a simple numerical
subtraction

y_diff = y_diff-H

is giving me the error

Syntaxerror: Non-ASCII character '\x96' in file on line 70, but no
encoding declared.

Even though I have deleted some lines before it and this line is no
longer line 70, I am still getting the error every time.  I have tried
to change the encoding of the file to utf-8 but to no avail, I still
am having this issue.  Any ideas?

Thanks in advance


Character '\x96' is an en-dash, which looks like a hyphen/minus sign
'-', but isn't.
--
http://mail.python.org/mailman/listinfo/python-list

Re: subtraction is giving me a syntax error

[Grant Edwards]

On 2010-03-15, Philip Semanchuk  wrote:
> On Mar 15, 2010, at 1:37 PM, Joel Pendery wrote:
>
>> So I am trying to write a bit of code and a simple numerical
>> subtraction
>>
>> y_diff = y_diff-H
>>
>> is giving me the error
>>
>> Syntaxerror: Non-ASCII character '\x96' in file on line 70, but no
>> encoding declared.
[...]
>
> 0x96 is the Win1252 minus sign character. Did you copy & paste this 
> code from a Web browser or a word processor?
>
> Delete the character between "y_diff" and "H" and replace it with a 
> plain ASCII subtraction sign.

I think somebody needs to stop editing his code with MS Word and start
using a programming editor. ;)

-- 
Grant Edwards   grant.b.edwardsYow! Make me look like
  at   LINDA RONSTADT again!!
  gmail.com
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: class inheritance

[Jean-Michel Pichavant]

JLundell wrote:

I've got a subclass of fractions.Fraction called Value; it's a mostly
trivial class, except that it overrides __eq__ to mean 'nearly equal'.
However, since Fraction's operations result in a Fraction, not a
Value, I end up with stuff like this:

x = Value(1) + Value(2)

where x is now a Fraction, not a Value, and x == y uses
Fraction.__eq__ rather than Value.__eq__.

This appears to be standard Python behavior (int does the same thing).
I've worked around it by overriding __add__, etc, with functions that
invoke Fraction but coerce the result. But that's tedious; there are a
lot of methods to override.

So I'm wondering: is there a more efficient way to accomplish what I'm
after?
  

I would change the approach.
To the question, "*is* a Value a Fraction", some may tempted to answer 
"No" (for instance, a fraction has a denominator, a value has not).

However "Does a Fraction *have* a Value", you could possibly say "Yes".

The value may be then an attribute of the class Fraction, not a subclass.
To test fraction equality, use F1 == F2. In order to test 'nearly 
equality', use F1.val() == F2.val().


JM




--
http://mail.python.org/mailman/listinfo/python-list

Re: staticmethod and setattr

[Jean-Michel Pichavant]

Am Montag, den 15.03.2010, 05:42 -0700 schrieb Michael.Lausch:

On Mar 15, 11:40 am, Steven D'Aprano  wrote:


On Mon, 15 Mar 2010 01:43:02 -0700, Michael.Lausch wrote:
  

Hi,

I managed to get confused by Python, which is not such an easy task.

The problem i have is rooted in marshalling, JSON and Dojo. I need some

static class in function with the name "$ref" i tried:
class Foo(object):
@staticmethod
def _ref(o):
 pass

setattr(Foo, "$ref", Foo._ref)


That doesn't work as expected:

  

Foo.__dict__['_ref'] is Foo.__dict__['$ref']


False

Try this instead:

  

setattr(Foo, "$ref", Foo.__dict__['_ref'])
Foo.__dict__['_ref'] is Foo.__dict__['$ref']


True
  

Now I'm trying to understand why this is the case.
How is Foo.__dict__['_ref']  different from Foo._ref?
Shouldn't it return the same attribute?

And after further experiments i found out that a making
Foo._ref a classmethod does work with my first approach.




When you declared _ref as static, a static object has been stored in Foo.
Using Foo.__dict__ you can access this static object, which is *not* the 
_ref function
Using Foo._ref, you trigger the lookup mechanism which do much more than 
accessing the dict. Especially, if it finds a __get__ method in the 
object, it will return the __get__ result instead of the object itself.


Foo._ref is equivalent in your case to 
Foo.__dict__['_ref'].__get__(None, Foo)



JM


--
http://mail.python.org/mailman/listinfo/python-list

Re: "Breaking" the __main__ script

[Jean-Michel Pichavant]

Steve Holden wrote:

pyt...@bdurham.com wrote:
  

Any reason you prefer PDB over WinPDB?
http://winpdb.org/


Yes. I don't have Windows except one one PC :P
  

WinPDB runs on non-Windows platforms :)



One might reasonably argue that it has a pretty couter-intuitive name, then.

regards
 Steve
  

'Win' may stand for Winner, not Windows :D

JM
--
http://mail.python.org/mailman/listinfo/python-list

Re: "Breaking" the __main__ script

[Jean-Michel Pichavant]

vsoler wrote:

Hello,

I am still learning python, thus developnig small scripts.

Some of them consist only of the main module. While testing them
(debugging) I sometimes want to stop the script at a certain point,
with something likestop, break, end   or something similar.

What statement can I use?

Vicente Soler
  

import bdb

pdb.set_trace() # put this line anywhere you want a breakpoint in your code.

type n for next, c for continue, s for step into and google for 'python 
pdb' for the details.


JM
--
http://mail.python.org/mailman/listinfo/python-list

Re: subtraction is giving me a syntax error

[Baptiste Carvello]

Joel Pendery a écrit :

So I am trying to write a bit of code and a simple numerical
subtraction

y_diff = y_diff-H

is giving me the error

Syntaxerror: Non-ASCII character '\x96' in file on line 70, but no
encoding declared.

Even though I have deleted some lines before it and this line is no
longer line 70, I am still getting the error every time.  I have tried
to change the encoding of the file to utf-8 but to no avail, I still
am having this issue.  Any ideas?

Thanks in advance


Hello,

I would say that when you press the minus key, your operating system doesn't 
encode the standard (ASCII) minus character, but some fancy character, which 
Python cannot interpret.


More precisely, I suspect you are unsing Windows with codepage 1252 (latin 1). 
With this encoding, you have 2 kinds of minus signs: the standard (45th 
character, in hex '\x2d') and the non-standard (150th character, in hex '\x96').


cf:
http://msdn.microsoft.com/en-us/library/cc195054.aspx

Cheers,
Baptiste

--
http://mail.python.org/mailman/listinfo/python-list

Re: "Breaking" the __main__ script

[Jean-Michel Pichavant]

Jean-Michel Pichavant wrote:

vsoler wrote:

Hello,

I am still learning python, thus developnig small scripts.

Some of them consist only of the main module. While testing them
(debugging) I sometimes want to stop the script at a certain point,
with something likestop, break, end   or something similar.

What statement can I use?

Vicente Soler
  

import bdb

pdb.set_trace() # put this line anywhere you want a breakpoint in your 
code.


type n for next, c for continue, s for step into and google for 
'python pdb' for the details.


JM

erratum

import pdb
--
http://mail.python.org/mailman/listinfo/python-list

Re: Python, Reportlabs, Pil and Windows 7 (64bit)

[Astley Le Jasper]

Hi Robin,

It looks like you've been busy. I'm sorry but you are well over my
head at the moment!

:-)

If you need me to test an install then I'd be happy to help. However,
I just received an email from Christoph Gohlke saying:

" ... There are 64 bit versions of Reportlab and PIL for Python 2.6
for Windows available at http://www.lfd.uci.edu/~gohlke/pythonlibs/
..."

Regards

Neil


-- 
http://mail.python.org/mailman/listinfo/python-list

[ANN] OpenOpt 0.28, FuncDesigner 0.18, DerApproximator 0.18

[dmitrey]

Hi all,
I'm glad to inform you about new release of our free (BSD-licensed)
soft OpenOpt 0.28 (numerical optimization), FuncDesigner 0.18 (CAS
with automatic differentiation), DerApproximator 0.18 (finite-
differeces derivatives approximation).

More details here:
http://forum.openopt.org/viewtopic.php?id=222

Regards, D.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Clustering and automated configuration & deployment toolkit with Python

[Philip Semanchuk]

On Mar 13, 2010, at 6:21 PM, np map wrote:


I'd like to write an open source clustering (for computation and
general use) and automation of configuration/deployment in Python.
It's main purpose is to be used in academic environments.
It would be something like running numpy/simpy code (and other custom
python code) on a set of machines in a distributed fashion (e.g.
splitting tasks, doing certain bits on some machines, other sub-tasks
on other machines, etc).


Hi np map,
Are you aware of ipython? My colleagues tell me that it contains some  
code for executing processes on remote machines. It sounds like code  
which you could build on or borrow from.


HTH
Philip





The cluster could be used in at least two ways:
- submit code/files via a web interface, monitor the task via the web
interface and download the results from the master node  (user<>web
interface<>master)
- run code directly from another machine on the cluster (as if it were
a subprocess or something like this)


Requirements (so far):
- support the Ubuntu Linux distribution in the initial iteration
- be easy to extend to other OS-es and package managers
- try to be 3.x compatible where dual compatibility is possible (2.x
and 3.x)
- it will support Python 2.5-2.6
- document required changes to the 2.x only code to make it work on
3.x
- make it easy to submit code directly from python scripts to the
cluster (with the right credentials)
- support key based authentication for job submission
- should talk to at least one type of RDBMS to store various types of
data
- the cluster should be able to kill a task on nodes automatically if
it executes for too long or requires too much memory (configurable)
- should be modular (use automation & configuration or just
clustering)


Therefore, I'd like to know a few things:

Is there a clustering toolkit already available for python?

What would the recommended architecture be ?

How should the "user" code interface with the clustering system's
code?

How should the results be stored (at the node and master level)?

Should threading be supported in the tasks?

How should they be returned to the Master node(s)? (polling, submitted
by the nodes, etc)

What libraries should be used for this? (e.g. fabric as a library,
pyro, etc)

Any other suggestions and pieces of advice?

Should Fabric be used in this clustering system for automation? If
not, what else? Would simply using a wrapper written in python for the
'ssh' app be ok?

Would the following architecture be ok?
Master: splits tasks into sub-tasks, sends them to nodes - provided
the node's load isn't greater than a certain percentage, gets results,
stores and provides configuration to nodes, stores results, etc
Node: runs code, applies configuration, submits the results to the
master, etc

If this system actually gets python-level code submission inside, how
should it work?

The reason I posted this set of questions and ideas is that I'd like
this to be as flexible and usable as possible.


--
http://mail.python.org/mailman/listinfo/python-list

Re: When to lock data items returned by multiprocessing.Manager?

[Aahz]

In article <4428d674-7fa7-4095-a93d-75ea31a81...@15g2000yqi.googlegroups.com>,
Veloz   wrote:
>
>So I'm using a multiprocessing.Manager instance in my main app and
>asking it to create a dictionary, which I am providing to instances of
>the application that I'm forking off with Process.
>
>The Processes and main app will be reading/writing to the dictionary.
>
>It's not clear to me what I have to "lock" and what I don't have to
>lock. Any ideas?

The way I deal with it is by making sure each key is only updated by a
single process.
-- 
Aahz (a...@pythoncraft.com)   <*> http://www.pythoncraft.com/

"Many customs in this life persist because they ease friction and promote
productivity as a result of universal agreement, and whether they are
precisely the optimal choices is much less important." --Henry Spencer
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Reverse engineering CRC?

[jkn]

Hi Greg
Just to say thanks for taking the time to write up your work on
this interesting topic.

Cheers
J^n

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Reverse engineering CRC?

[geremy condra]

On Mon, Mar 15, 2010 at 6:29 AM, Gregory Ewing
 wrote:
> I've solved the problem now.
>
> It turned out to be a very standard CRC algorithm, complicated
> by the presence of a few extra bytes of data being checked that
> didn't appear explicitly in the file anywhere.
>
> In the process I developed some very general techniques for
> solving this kind of problem, which I've written about here
> if anyone's interested:
>
> http://www.cosc.canterbury.ac.nz/greg.ewing/essays/CRC-Reverse-Engineering.html
>
> Thanks for everyone's help,
> Greg

Nice writeup, thanks.

Geremy Condra
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: subtraction is giving me a syntax error

[Steven D'Aprano]

On Mon, 15 Mar 2010 18:09:29 +, Grant Edwards wrote:

>> Delete the character between "y_diff" and "H" and replace it with a
>> plain ASCII subtraction sign.
> 
> I think somebody needs to stop editing his code with MS Word and start
> using a programming editor. ;)


I've had this error myself, and I've never used Word to edit code. It can 
happen if you copy code from a website that "helpfully" converts hyphens 
to en-dashes, spaces to non-breaking spaces, or inserts ctrl-Z characters 
into strings, etc. They're a devil to debug.


-- 
Steven
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: subtraction is giving me a syntax error

[John Machin]

On Mar 16, 5:43 am, Baptiste Carvello  wrote:
> Joel Pendery a écrit :

> > So I am trying to write a bit of code and a simple numerical
> > subtraction
>
> > y_diff = y_diff-H
>
> > is giving me the error
>
> > Syntaxerror: Non-ASCII character '\x96' in file on line 70, but no
> > encoding declared.
>
>
> I would say that when you press the minus key, your operating system doesn't
> encode the standard (ASCII) minus character, but some fancy character, which
> Python cannot interpret.

The likelihood that any operating system however brain-damaged and in
whatever locale would provide by default a "keyboard" or "input
method" that generated EN DASH when the '-' key is struck is somewhere
between zero and epsilon.

Already advanced theories like "used a word processor instead of a
programmer's editor" and "scraped it off the web" are much more
plausible.

> More precisely, I suspect you are unsing Windows with codepage 1252 (latin 1).

Codepage 1252 is not "latin1" in the generally accepted meaning of
"latin1" i.e. ISO-8859-1. It is a superset. MS in their wisdom or
otherwise chose to use most of the otherwise absolutely wasted slots
assigned to "C1 control characters" in latin1.

> With this encoding, you have 2 kinds of minus signs: the standard (45th
> character, in hex '\x2d') and the non-standard (150th character, in hex 
> '\x96').
>
> cf:http://msdn.microsoft.com/en-us/library/cc195054.aspx

The above link quite correctly says that '\x96` maps to U+2013 EN
DASH. EN DASH is not any kind of minus sign.

Aside: the syndrome causing the problem is apparent with cp125x for x
in range(9)



-- 
http://mail.python.org/mailman/listinfo/python-list

Re: subtraction is giving me a syntax error

[Grant Edwards]

On 2010-03-15, Steven D'Aprano  wrote:
> On Mon, 15 Mar 2010 18:09:29 +, Grant Edwards wrote:
>
>>> Delete the character between "y_diff" and "H" and replace it with a
>>> plain ASCII subtraction sign.
>> 
>> I think somebody needs to stop editing his code with MS Word and start
>> using a programming editor. ;)
>
> I've had this error myself, and I've never used Word to edit code. It can 
> happen if you copy code from a website that "helpfully" converts hyphens 
> to en-dashes, spaces to non-breaking spaces, or inserts ctrl-Z characters 
> into strings, etc. They're a devil to debug.

Though it may not be Microsoft Word, I think I'd still maintain that
an editor where you can't see a ctrl-Z or tell the difference between
an ASCII minus and a windows-codepage-whatever isn't a very good
programmer's editor.

-- 
Grant Edwards   grant.b.edwardsYow! HUMAN REPLICAS are
  at   inserted into VATS of
  gmail.comNUTRITIONAL YEAST ...
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: class inheritance

[JLundell]

On Mar 13, 1:26 pm, Carl Banks  wrote:
> It's a tad unfortunately Python doesn't make this easier.  If I had to
> do it more than once I'd probably write a mixin to do it:
>
> class ArithmeticSelfCastMixin(object):
>     def __add__(self,other):
>         return
> self.__class__(super(ArithmeticSelfCastMixin,self).__add__(other)
>     # etc.
>
> class Value(ArithmeticSelfCastMixin,fraction.Fraction):
>     pass
>
> However, I want to warn you about overriding __eq__ to mean "almost
> equal": it can have unexpected results so I don't recommend it.  Two
> of the main issues with it are:
>
> 1. It violates the transitive property ("If A == B and B == C, then A
> == C") which most programmers expect to be true.
>
> 2. It will give unpredictable results when the objects are used in
> sets or as dictionary keys.  Those thow types expect the transitive
> property to be true.  If you are going to redefine __eq__ to mean
> "almost equal", then at least define __hash__ to raise
> NotImplementedError so that Python will refuse to use them in sets or
> as dictionary keys:
>
>     def __hash__(self): raise NotImplementedError
>
> Carl Banks

It's also unfortunate that Python doesn't have an approximately-equal
operator; it'd come in handy for floating-point applications while
preserving hash. If only there were a ~= or ≈ operator I could
overload. And ~ is unary, so no joy.

My application is in that sense a little like a floating-point app, in
that it needs approximately-equal. And it doesn't need Value to be
hashable; thanks for the NotImplementedError suggestion; I've done
that as a safeguard.

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: logging: local functions ==> loss of lineno

[Hellmut Weber]

Am 11.03.2010 12:14, schrieb Peter Otten:

Hellmut Weber wrote:


Logging works very well giving the filename and line number of the point
where it is called. As long as I use the loggers directly.
BUT when I have to wrap the logger call in some other function, I always
get file name and line number of the call of the logger inside the
wrapping function.

Is there a possibility to get this information in this situation too?


The official way is probably to write a custom Logger class that overrides
the findCaller() method.

Below is a hack that monkey-patches the logging._srcfile attribute to ignore
user-specified modules in the call stack:

$ cat wrapper.py
import logging
import os
import sys

logger = logging.getLogger()

class SrcFile(object):
 def __init__(self, exclude_files):
 self.files = set(exclude_files)
 def __eq__(self, other):
 return other in self.files

def fixname(filename):
 if filename.lower().endswith((".pyc", ".pyo")):
 filename = filename[:-4] + ".py"
 return os.path.normcase(filename)

if "--monkey" in sys.argv:
 print "patching"
 logging._srcfile = SrcFile([logging._srcfile, fixname(__file__)])

def warn(*args, **kw):
 logger.warn(*args, **kw)

$ cat main.py
import logging
logging.basicConfig(format="%(filename)s<%(lineno)s>: %(message)s")
import wrapper
wrapper.warn("foo")
wrapper.warn("bar")
wrapper.warn("baz")

$ python main.py
wrapper.py<23>: foo
wrapper.py<23>: bar
wrapper.py<23>: baz

$ python main.py --monkey
patching
main.py<4>: foo
main.py<5>: bar
main.py<6>: baz

$ python -V
Python 2.6.4

Peter


Hi Peter,
thanks for your help ;-)

I've been offline for a few days so I found your message only today.
I'll try your proposal tomorrow.

Thanks again

Hellmut

--
Dr. Hellmut Weber m...@hellmutweber.de
Degenfeldstraße 2 tel   +49-89-3081172
D-80803 München-Schwabing mobil +49-172-8450321
please: No DOCs, no PPTs. why: tinyurl.com/cbgq

--
http://mail.python.org/mailman/listinfo/python-list

Re: class inheritance

[Carl Banks]

On Mar 15, 4:34 pm, JLundell  wrote:
> It's also unfortunate that Python doesn't have an approximately-equal
> operator; it'd come in handy for floating-point applications while
> preserving hash. If only there were a ~= or ≈ operator I could
> overload. And ~ is unary, so no joy.

One problem with it is that there's no way to make it universal;
different appiplications have different ideas of close.  Conceivably
it could be usefully defined for a user type though..

Bacause of this problem almost no languages have an almost equal
operator.  I'm curious what languages do, of if there are any with a
trinary operator that also takes a threshold.

Carl Banks
-- 
http://mail.python.org/mailman/listinfo/python-list

Binary data transfer issue

[Jordan Apgar]

Hi all,

I'm trying to transfer a binary file over xmlrpclib.   My test file is
a .jpeg file.  I can transfer all the data over but when I go to open
the .jpeg I get "Error interpreting JPEG image file (Invalid JPEG file
structure: SOS before SOF)"

here's the code:

===Various Shared Functions
#EncodeAES takes a Cipher c and encrypts a String s
EncodeAES = lambda c, s: base64.b64encode(c.encrypt(pad(s)))

#DecodeAES takes a Cipher C and decrypts the cyphertext e
#then removes the padding from the decrypted string
DecodeAES = lambda c, e:
c.decrypt(base64.b64decode(e)).rstrip(PADDING)


CIPHER = AES.new("KEY")


# method to change strings back into tuples
def stringToTuple(string):
if string[0] + string[-1] == "()":
items = string[1:-1]
items = items.split(',')
return items
else:
raise ValueError("Badly formatted string (missing brackets).")

==client=
 def Download(self, FileName, Data_Size=1024):
ErrorCount = 0
Pkt_Num = -1
OldPkt_Num = Pkt_Num
localFile = open("files/"+FileName, "wb")
print
print "Requesting First Packet"
packet = self.client.download(EncodeAES(CIPHER, str(Pkt_Num)))
if packet[0] == "False":
print packet[1]
return False
packet = stringToTuple(DecodeAES(CIPHER, packet))
if int(packet[0]) == -3:
print "Received Error Packet"
else:
print "Received Packet: ", int(packet[0])

print "packet[1], ", packet[2:-1]

Pkt_Num = int(packet[0])

while Pkt_Num >= 0 and ErrorCount <= 10:
if (OldPkt_Num +1) == Pkt_Num:
OldPkt_Num = OldPkt_Num +1
localFile.write(binascii.
a2b_base64(packet[1][2:-1]))
#<<<=
os.path.getsize("files/"+self.usr_dict[client][FILENAME]):
return EncodeAES(CIPHER,
 str((-2, "File Transfer Complete")))
try:
f = open("files/"+self.usr_dict[client][FILENAME],
"rb")
except IOError:
   return EncodeAES(CIPHER,
 str((-3, "File Could Not be opened")))
data = f.read((Pkt_Num+1 * DataRate))
data = None
data = binascii.b2a_base64(f.read(DataRate))
#<<

Re: Binary data transfer issue

[MRAB]

Jordan Apgar wrote:

Hi all,

I'm trying to transfer a binary file over xmlrpclib.   My test file is
a .jpeg file.  I can transfer all the data over but when I go to open
the .jpeg I get "Error interpreting JPEG image file (Invalid JPEG file
structure: SOS before SOF)"

here's the code:

===Various Shared Functions
#EncodeAES takes a Cipher c and encrypts a String s
EncodeAES = lambda c, s: base64.b64encode(c.encrypt(pad(s)))

#DecodeAES takes a Cipher C and decrypts the cyphertext e
#then removes the padding from the decrypted string
DecodeAES = lambda c, e:
c.decrypt(base64.b64decode(e)).rstrip(PADDING)


CIPHER = AES.new("KEY")


# method to change strings back into tuples
def stringToTuple(string):
if string[0] + string[-1] == "()":
items = string[1:-1]
items = items.split(',')
return items
else:
raise ValueError("Badly formatted string (missing brackets).")

==client=
 def Download(self, FileName, Data_Size=1024):
ErrorCount = 0
Pkt_Num = -1
OldPkt_Num = Pkt_Num
localFile = open("files/"+FileName, "wb")
print
print "Requesting First Packet"
packet = self.client.download(EncodeAES(CIPHER, str(Pkt_Num)))
if packet[0] == "False":
print packet[1]
return False
packet = stringToTuple(DecodeAES(CIPHER, packet))
if int(packet[0]) == -3:
print "Received Error Packet"
else:
print "Received Packet: ", int(packet[0])

print "packet[1], ", packet[2:-1]

Pkt_Num = int(packet[0])

while Pkt_Num >= 0 and ErrorCount <= 10:
if (OldPkt_Num +1) == Pkt_Num:
OldPkt_Num = OldPkt_Num +1
localFile.write(binascii.
a2b_base64(packet[1][2:-1]))
#<<<

I don't understand what you're trying to do when you're reading from
the file. For a start:

Pkt_Num+1 * DataRate

is the same as:

Pkt_Num + (1 * DataRate)

or:

Pkt_Num + DataRate

and that's how many bytes you're reading.


f.close()
self.usr_dict[client][PKT] = Pkt_Num
return EncodeAES(CIPHER,
 str((Pkt_Num, data)))

elif Pkt_Num == Old_Pkt_Num:
if Pkt_Num * DataRate >=
os.path.getsize("files/"+self.usr_dict[client][FILENAME]):
return EncodeAES(CIPHER,
 str((-2, "File Transfer Complete")))
try:
f = open("files/"+self.usr_dict[client][FILENAME],
"rb")
except IOError:
   return EncodeAES(CIPHER,
 str((-3, "File Could Not be opened")))
data = f.read((Pkt_Num+1 * DataRate))


Here you're reading:

Pkt_Num + DataRate

bytes again,


data = None


discarding what you've just read,


data = binascii.b2a_base64(f.read(DataRate))
#<<<

and then reading:

DataRate

more bytes.

I think what you meant to do is .seek to the position in the file and
then read some bytes:

f.seek(Pkt_Num * DataRate)
data = f.read(DataRate)

ie, send the contents of the file in chunks which are DataRate bytes
long (the last chunk will be shorter if the file size isn't a multiple
of DataRate bytes).


f.close()
self.usr_dict[client][PKT] = Pkt_Num +1

return EncodeAES(CIPHER,
str((Pkt_Num+1, data)))

  else:

return EncodeAES(CIPHER,
 str((-3, "Incorrect Packet Requested")))



Thank you all for the help,  I know there are better ways and transfer
protocol modules to but I'd like to do this without them.


I'm not surprised that the transfer is failing! :-)
--
http://mail.python.org/mailman/listinfo/python-list

Build Python with XCode

[moerchendiser2k3]

Hi,

I would like to build Python with Xcode (but without the makefile).
Does anyone know a link where I can get a real xcodeproj with the
current Py2.x sources?


Thanks in advance!! Bye, donnerCobra
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: converting datetime with tzinfo to unix timestamp

[Gabriel Genellina]

En Fri, 12 Mar 2010 20:15:55 -0300, Michael Torrie   
escribió:



On Python 2.5 here.

I've searched and searched but I can't find any way to convert a
datetime object that includes a timezone (tzinfo) to a unix timestamp.
Folks on the net say to simply use the timetuple() method of the object
and feed that to time.mktime().  But that just doesn't seem to work for
me.  At least if my understanding that the unix timestamp is always UTC.


Try time.mktime(datetimeobject.utctimetuple()) instead.

--
Gabriel Genellina

--
http://mail.python.org/mailman/listinfo/python-list

Re: result of os.times() is different with 'time' command Options

[Gabriel Genellina]

En Mon, 15 Mar 2010 03:51:28 -0300, hiral   
escribió:

On Mar 15, 7:14 am, Tim Roberts  wrote:

hiral wrote:



>Output:
>real0.0m0.010002421s
>user0.0m0.0s
>sys 0.0m0.0s

>Command:
>$ time ls

>Output:
>real0m0.007s
>user0m0.000s
>sys 0m0.000s

You can't really do an analysis like this with a task that only takes a  
few

milliseconds.  There are too many variables.


Thanks for your explanation.


Have you tested with the highly recursive function in that 2007 thread you  
linked to?

This should take time enough to make a problem like this clearly visible.

--
Gabriel Genellina

--
http://mail.python.org/mailman/listinfo/python-list

Dynamic Class Creation

[Josh English]

I have a large program with lots of data stored in XML. I'm upgrading
my GUI to use ObjectListView, but with my data in XML, I don't have
regular objects to interact with the OLV. I do have an XML validator
that defines the structure of the XML elements, and I'm trying to
dynamically create a class to wrap the XML element.

So, an element like:


Writers of the Future


I want to create a class like:

class Market(object):
def __init__(self, elem):
self._elem = elem

def get_code(self):
return self._elem.get('code')

def set_code(self, value):
self._elem.set('code', value)

def get_title(self):
return self._elem.find('title').text

def set_title(self, value):
node = self._elem.find('title')
node.text = value

Naturally, I don't want to hand code this for every interface but
would like to create them dynamically. (The laziness of programming, I
guess.)

I have tried several solutions that I found on various forums, but
they are all several years old.

Questions:

What's the best way to create these helper methods?

How can I attach them to the class and have it run?

I have had a few solutions work where I had a class with three methods
(get_code, get_tier, get_mail) but they all return the same value, not
the individual values.


Josh English




-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Reverse engineering CRC?

[Gabriel Genellina]

En Mon, 15 Mar 2010 07:29:51 -0300, Gregory Ewing  
 escribió:



I've solved the problem now.

It turned out to be a very standard CRC algorithm, complicated
by the presence of a few extra bytes of data being checked that
didn't appear explicitly in the file anywhere.

In the process I developed some very general techniques for
solving this kind of problem, which I've written about here
if anyone's interested:

http://www.cosc.canterbury.ac.nz/greg.ewing/essays/CRC-Reverse-Engineering.html


A good solution to an interesting problem - and very nicely explained too!

--
Gabriel Genellina

--
http://mail.python.org/mailman/listinfo/python-list

Re: Dynamic Class Creation

[Chris Rebert]

On Mon, Mar 15, 2010 at 11:01 PM, Josh English
 wrote:
> I have a large program with lots of data stored in XML. I'm upgrading
> my GUI to use ObjectListView, but with my data in XML, I don't have
> regular objects to interact with the OLV. I do have an XML validator
> that defines the structure of the XML elements, and I'm trying to
> dynamically create a class to wrap the XML element.
>
> So, an element like:
>
> 
> Writers of the Future
> 
>
> I want to create a class like:
>
> class Market(object):
>    def __init__(self, elem):
>        self._elem = elem
>
>    def get_code(self):
>        return self._elem.get('code')
>
>    def set_code(self, value):
>        self._elem.set('code', value)
>
>    def get_title(self):
>        return self._elem.find('title').text
>
>    def set_title(self, value):
>        node = self._elem.find('title')
>        node.text = value
>
> Naturally, I don't want to hand code this for every interface but
> would like to create them dynamically. (The laziness of programming, I
> guess.)
>
> I have tried several solutions that I found on various forums, but
> they are all several years old.
>
> Questions:
>
> What's the best way to create these helper methods?

Nested functions:

def make_attr_getset(name):
def get(self):
return self._elem.get(name)

def set(self, value):
self._elem.set(name, value)
return get, set

get_code, set_code = make_attr_getset('code')

def make_subelement_getset(name):
def get(self):
return self._elem.find(name).text

def set(self, value):
node = self._elem.find(name)
node.text = value
return get, set

get_title, set_title = make_subelement_getset('title')

> How can I attach them to the class and have it run?

Use properties and setattr():

class Market(object):
def __init__(... #same as before

setattr(Market, 'code', property(get_code, set_code))
setattr(Market, 'title', property(get_title, set_title))

m = Market(the_XML)
print m.title #=> Writers of the Future
m.title = "Writers of the Past"
print m.code #=> WotF
m.code = "WofP"

Cheers,
Chris
--
http://blog.rebertia.com
-- 
http://mail.python.org/mailman/listinfo/python-list

Data entry Work available

[Earn money]

Data entry works Available At
No need to pay DEposite

http://trading7000.blogspot.com/



Adposting Jobs Availble

http://trading7000.blogspot.com/


No need to work Hard
Earn Money From Home
7000 in Minutes

http://trading7000.blogspot.com/
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: time_struct

[Gabriel Genellina]

En Sun, 07 Mar 2010 10:50:59 -0300, moerchendiser2k3  
 escribió:



can anyone tell me how to return a time_struct from the timemodule in
my own C-Module?
Is that possible? I can just find one function in timefuncs.h, but it
doesnt return any PyObject.


The type is available as the struct_time attribute in the time module; you  
create an instance by calling it with a 9-items sequence.


--
Gabriel Genellina

--
http://mail.python.org/mailman/listinfo/python-list