Re: serialize a class to XML and back

2013-05-26 Thread Irmen de Jong
On 26-5-2013 22:48, Roy Smith wrote:

> The advantage of pickle over json is that pickle can serialize many 
> types of objects that json can't.  The other side of the coin is that 
> pickle is python-specific, so if you think you'll ever need to read your 
> data from other languages, pickle is right out.

That is not entirely true :)  I've written a pickle implementation for Java and 
.NET
that is almost feature complete; it is part of 
http://pythonhosted.org/Pyro4/pyrolite.html

Still, pickle may not be the best choice here.

Cheers
Irmen

-- 
http://mail.python.org/mailman/listinfo/python-list


Re: serialize a class to XML and back

2013-05-27 Thread Irmen de Jong
On 27-5-2013 2:39, Roy Smith wrote:
> In article <51a28f42$0$15870$e4fe5...@news.xs4all.nl>,
>  Irmen de Jong  wrote:
> 
>> On 26-5-2013 22:48, Roy Smith wrote:
>>
>>> The advantage of pickle over json is that pickle can serialize many 
>>> types of objects that json can't.  The other side of the coin is that 
>>> pickle is python-specific, so if you think you'll ever need to read your 
>>> data from other languages, pickle is right out.
>>
>> That is not entirely true :)  I've written a pickle implementation for Java 
>> and .NET
>> that is almost feature complete; it is part of 
>> http://pythonhosted.org/Pyro4/pyrolite.html
> 
> Very cool
> 
>> Still, pickle may not be the best choice here.
> 
> Perhaps not, but lots of points for the awesomeness factor.
> 

Thanks for the praise :)

There's another interesting thing perhaps to also mention about Pyrolite. Its 
Pickle
implementation obviously maps built in types to their counterparts in 
Java/.NET, but it
is also capable of unpickling custom classes. It defaults to outputting a simple
hashtable with the class's properties in it, but you can also provide a custom
deserializer to recreate a custom object (it already does this automatically 
for a few
complex types such as DateTime, BigDecimal, and a bunch of Pyro specific types).

As the unpicklers are not executing any Java or .NET code dynamically they are 
not
susceptible to the security issue that Python's pickle has. Still: tread with 
care.
(Especially if you use the lib to pickle stuff and unpickle it on the Python 
side).

Also, the one missing feature is memo-ing when pickling so that you can pickle 
recursive
object graphs. It now throws a stack overflow exception instead.


Irmen


-- 
http://mail.python.org/mailman/listinfo/python-list


Re: Pillow lib for x86_64 GNU/Linux

2013-06-03 Thread Irmen de Jong
On 3-6-2013 18:23, consult...@gmail.com wrote:
> It is great that Pillow wants to be "setuptools compatible" but without a 
> suitable compiled library for x86_64 GNU/Linux, I am stuck between a rock and 
> a hard place.
> 
> Any suggestions?
> 

Try your distribution's package repository.

$ sudo apt-get install python-pillow

Or failing that,

$ sudo apt-get install python-imaging


Worked fine for me (Ubuntu 64-bit)

Irmen



-- 
http://mail.python.org/mailman/listinfo/python-list


problem uploading docs to pypi

2013-06-14 Thread Irmen de Jong
Hi,

I'm experiencing some trouble when trying to upload the documentation for one 
of my
projects on Pypi. I'm getting a Bad Gateway http error message.
Anyone else experiencing this? Is this an intermittent issue or is there a 
problem with
Pypi?

Downloading documentation (from pythonhosted.org) works fine.

Irmen
-- 
http://mail.python.org/mailman/listinfo/python-list


Re: problem uploading docs to pypi

2013-06-15 Thread Irmen de Jong
On 15-6-2013 2:23, MRAB wrote:
> About 10 ten days ago I got the error:
> 
>  Upload failed (503): backend write error
> 
> while trying to upload to PyPI, and it failed the same way the second time, 
> but worked some
> time later.

You're right. I tried it again just now and it succeeded this time.

Cheers
Irmen

-- 
http://mail.python.org/mailman/listinfo/python-list


Re: Using Python to automatically boot my computer at a specific time and play a podcast

2013-06-17 Thread Irmen de Jong
On 17-6-2013 15:24, inq1ltd wrote:
> On Sunday, June 16, 2013 12:06:08 PM C. N. Desrosiers wrote:
> 
>> Hi,
> 
>>
> 
>> I'm planning to buy a Macbook Air and I want to use it as a sort of alarm.
>> I'd like to write a program that boots my computer at a specific time,
>> loads iTunes, and starts playing a podcast. Is this sort of thing possible
>> in Python?

You can use the osascript utility to send commands to itunes, and invoke it 
from Python
like this:

import subprocess
listname = "My Playlist"
subprocess.call(["osascript", "-e", "tell application \"iTunes\" to play 
playlist
\"{0}\"".format(listname)])

But that seems overkill (using Python to use Applescript to control iTunes)...

Irmen

-- 
http://mail.python.org/mailman/listinfo/python-list


Re: Looking for a name for a deployment framework...

2013-06-24 Thread Irmen de Jong
On 24-6-2013 20:13, Cousin Stanley wrote:
> jonathan.slend...@gmail.com wrote:
> 
>> Any suggestions for a good name, 
>> for a framework that does 
>> automatic server deployments ?
> 
>   asdf :  automatic server deployment framework
> 
> 


:-)


wsad: wonderful serverside automatic deployments

(hm, could use a bit of tweaking, maybe FPS keys don't map easily to names)


Irmen
-- 
http://mail.python.org/mailman/listinfo/python-list


Re: class factory question

2013-06-27 Thread Irmen de Jong
On 27-6-2013 15:48, Dave Angel wrote:

>> The behavior for these is all the same so they're subclassed
>> from one base class, but they need to have these particular names so  the 
>> parser knows
>> how to consume them when encountered in the source file. That is, for every 
>> custom
>> command the parser encounters, it looks for a class of that name in order to 
>> tokenize it.
  ^

How does it look for a class?
Can you perhaps override the way it looks for a class of that name?
So that you can instead return something sensible rather than having to define 
one of 50
almost identical classes...


Irmen

-- 
http://mail.python.org/mailman/listinfo/python-list


Re: math functions with non numeric args

2013-06-30 Thread Irmen de Jong
On 30-6-2013 20:46, Andrew Z wrote:
> Hello,
> 
> print max(-10, 10)
> 10
> print max('-10', 10)
> -10
> 
> My guess max converts string to number bye decoding each of the characters to 
> it's ASCII
> equivalent?
> 
> Where can i read more on exactly how the situations like these are dealt with?
> 
> Thank you
> AZ
> 

Use Python 3.x instead.

>>> max('-10',10)
Traceback (most recent call last):
  File "", line 1, in 
TypeError: unorderable types: int() > str()
>>>


Irmen

-- 
http://mail.python.org/mailman/listinfo/python-list



Re: socket data sending problem

2013-07-03 Thread Irmen de Jong
On 4-7-2013 0:38, ollietemple...@aol.com wrote:

> it all works fine, except for when i try to use:
> 
>   s.send("hello")
> 
> to send data between the client and server, i just get this error message:
> 
>   >>> 
>   Traceback (most recent call last):
> File "C:/Users/Ollie/Documents/code/chatroom/client3.py", line 9, in 
> 
>   s.send("hello")
>   TypeError: 'str' does not support the buffer interface
>   >>> 

Are you using Python 3.x? Try sending bytes instead of strings:

 s.send(b"hello")

Or switch to using Python 2.7

Irmen

-- 
http://mail.python.org/mailman/listinfo/python-list


Re: How to check for threads being finished?

2013-07-05 Thread Irmen de Jong
On 5-7-2013 18:59, Steven D'Aprano wrote:
> I then block until the threads are all done:
> 
> while any(t.isAlive() for t in threads):
> pass
> 
> 
> Is that the right way to wait for the threads to be done? Should I stick 
> a call to time.sleep() inside the while loop? If so, how long should I 
> sleep? That's probably an unanswerable question, but some guidelines on 
> choosing the sleep time will be appreciated.
> 

I think your while loop busy-waits until the threads are completed.
Do this instead:

for t in threads: t.join()# optionally pass a timeout to join


-Irmen

-- 
http://mail.python.org/mailman/listinfo/python-list


Re: Python - remote object protocols and security

2013-07-15 Thread Irmen de Jong
On 15-7-2013 13:17, Dave Angel wrote:
> On 07/15/2013 06:20 AM, Jean-Michel Pichavant wrote:
>> In text format... sorry for my previous html post
>>
>> Hello everyone,
>>
>> I'd like to exchange some simple python objects over the internet.
>> I initially planned to use Pyro, after reading
>> http://pythonhosted.org/Pyro4/security.html I'm still puzzled.

Hi, Pyro's author here.
I agree that this chapter of the manual can use some cleanup.
Is there anything in particular that you are puzzled about at this time?

>>
>> I don't mind encrypting data, if someone wants to sniff what I'm sending, 
>> he's welcome.
>>

I don't quite understand what you're saying in this sentence: is it okay if 
someone
eavesdrops on your unencrypted data stream?


>> What I think I need to care about, is malicious code injections. Because both
>> client/server will be in python, would someone capable of executing code by 
>> changing
>> one side python source ?

Pyro since version 4.20 uses a serialization format that is safe against 
arbitrary code
execution: https://pypi.python.org/pypi/serpent
That format only encodes and decodes Python literal expressions, and no 
arbitrary
objects are instantiated. You can also tell Pyro to use JSON (or marshal even), 
both of
which should be impervious to this type of attack/vulnerability as well.

The problem is with older Pyro versions, which allowed only Pickle to be used as
serialization format. It is pickle that causes the remote code execution 
vulnerability.
So as long as you don't explicitly tell Pyro (4.20+) to use pickle (a 
configuration
switch), you should be safe.

> I can't tell you if pyro, or any other particular one is safe.  

Pyro should be, since version 4.20 and provided you don't tell it to use 
pickle. See above.

> Note that DOS attacks are possible whatever encoding scheme you have. Make 
> sure that
> self-references within the data are well-defined (or impossible), and put 
> limits on size
> per transaction, and transactions per minute per legitimate user.

Pyro doesn't provide anything by itself to protect against this.


Cheers
Irmen de Jong

-- 
http://mail.python.org/mailman/listinfo/python-list


Re: Python - remote object protocols and security

2013-07-15 Thread Irmen de Jong
On 15-7-2013 18:57, Irmen de Jong wrote:

>> Note that DOS attacks are possible whatever encoding scheme you have. Make 
>> sure that
>> self-references within the data are well-defined (or impossible), and put 
>> limits on size
>> per transaction, and transactions per minute per legitimate user.
> 
> Pyro doesn't provide anything by itself to protect against this.

I'm sorry to follow up on myself, but there is actually one thing: Pyro's 
choice of
serializers (except pickle, again) don't allow self-references. So that type of 
DOS
attack (infinite recursion) is ruled out.


Irmen

-- 
http://mail.python.org/mailman/listinfo/python-list


Re: Is it that easy to install Python ?

2013-07-25 Thread Irmen de Jong
On 25-7-2013 17:11, santiago.d...@caoba.fr wrote:
> Hi there,
> 
> I never write any Python program but as a system administrator, I'm often 
> asked to install python on Debian servers.
> 
> I just finished downloading, configuring, making and installing.
> 
> The binary is now installed in :
> /usr/local/Python-2.7.5/bin/python2.7
> (the path is a deliberate administrator choice).
> 
> Is that it?
> 
> What else will my users need?

Why didn't you use the Debian package instead? You now have installed an 
unsupported,
untested custom built Python version on your server. Why not simply

$ apt-get install python

and let the Debian package maintainers take care of properly testing and 
supporting
it... Also, installing additional python packages will be much less of a hassle 
because
there's hundreds of them readily available in Debian's package repositories and 
they can
be installed (including correct dependencies) in the same way.


Irmen

-- 
http://mail.python.org/mailman/listinfo/python-list


Re: Thread is somehow interfering with a while loop called after the thread is started

2013-07-28 Thread Irmen de Jong
On 28-7-2013 4:29, dan.h.mciner...@gmail.com wrote:
> I have a simple scapy + nfqueue dns spoofing script that I want to turn into 
> a thread within a larger program:
> 
> http://www.bpaste.net/show/HrlfvmUBDA3rjPQdLmdp/
> 
> Below is my attempt to thread the program above. Somehow, the only way the 
> while loop actually prints "running" is if the callback function is called 
> consistently. If the callback function isn't started, the script will never 
> print "running". How can that be if the while loop is AFTER the thread was 
> started? Shouldn't the while loop and the thread operate independantly?
> 
> http://bpaste.net/show/0aCxSsSW7yHcQ7EBLctI/
> 

Try adding sys.stdout.flush() after your print statements, I think you're 
seeing a
stdout buffering issue.


Irmen

-- 
http://mail.python.org/mailman/listinfo/python-list


Re: setup.py Choosing Older Compiler On Windows

2012-12-02 Thread Irmen de Jong
On 2-12-2012 22:06, Dave Angel wrote:
> On 12/02/2012 09:34 AM, Ami Tavory wrote:
>>   Hello,
>>
>>   I'm porting a C++ extension module to a Windows machine that has both VC8
>> and VC10 installed. Unfortunately, `setup.py build` tries to build using
>> VC8, which fails (the extension uses C++ standard libraries that VC  didn't
>> used to have). Is there a way to get setup.py to use VC10 (preferably
>> externally, without modifying the script)?
>>
>>  
> 
> I haven't had to do Windows C++ development for many years, but there
> used to be a vcvars.bat  in each compiler installation, and you run the
> one corresponding to the compiler & libraries you want.


Forcing it to use a different compiler than the one that was used to build 
Python
itself, may very well lead to a non-working extension module.

Irmen


-- 
http://mail.python.org/mailman/listinfo/python-list


Re: Installing packages on Mac OS X 10.7.5

2012-12-05 Thread Irmen de Jong
On 6-12-2012 0:12, John Dildy wrote:
> Hello Everyone!
> 
> I have python v2.7.1 and I am trying to install packages on the Mac OS X 
> v10.7.5
> 
> I am trying to install:
> 
> Distribute
> 
> Nose
> 
> virtualenv
> 
> If anyone can help me that would be great
> 
> John Dildy
> 
> jdild...@gmail.com
> 


Avoid changing stuff on the system installed python.
If you don't have virtualenv already, I would suggest to either:

- install virtualenv by means of easy_install (which should be installed 
already)
- do everything else in a virtual env, instead of in the system installed 
python directly

Or install homebrew, then brew install python, and use that. This avoids using 
the
system installed python entirely.


Irmen

-- 
http://mail.python.org/mailman/listinfo/python-list


Re: How to list a file which already created a 2 mins ago

2012-12-06 Thread Irmen de Jong
On 6-12-2012 17:49, moonhkt wrote:
> Hi All
> 
> AIX.5.3
> Python 2.6.2
> 
> File ftp to Machine A, need to rename then send to Machine B.
> 
> How to list a file which already created a 2 mins ago ?  If file aging
> more than 2 mins. I want to rename file to other file name.
> 
> moonhkt
> 

ftplib.FTP
os.path.getmtime
os.rename
time.time

Should be some pointers to get started.


Irmen

-- 
http://mail.python.org/mailman/listinfo/python-list


wrong ImportError message printed by python3.3 when it can't find a module?

2012-12-07 Thread Irmen de Jong
Hi,

I'm seeing that Python 3.3.0 is not printing the correct ImportError when it 
can't
import a module that is imported from another module. Instead of printing the 
name of
the module it can't import, it prints the name of the module that is doing the 
faulty
import.

Behold:
I have created:
  importfail\
 __init__.py
 main.py
 sub.py


[L:\]type importfail\main.py
from . import sub

[L:\]type importfail\sub.py
import nonexisting_module

[L:\]


[L:\]c:\Python33\python.exe -m importfail.main
Traceback (most recent call last):
  File "C:\Python33\lib\runpy.py", line 160, in _run_module_as_main
"__main__", fname, loader, pkg_name)
  File "C:\Python33\lib\runpy.py", line 73, in _run_code
exec(code, run_globals)
  File ".\importfail\main.py", line 1, in 
from . import sub
ImportError: cannot import name sub# <--- huh ?


Python 3.2 and earlier do the right thing:

[L:\]c:\Python32\python.exe -m importfail.main
Traceback (most recent call last):
  File "C:\Python32\lib\runpy.py", line 160, in _run_module_as_main
"__main__", fname, loader, pkg_name)
  File "C:\Python32\lib\runpy.py", line 73, in _run_code
exec(code, run_globals)
  File "L:\importfail\main.py", line 1, in 
from . import sub
  File "importfail\sub.py", line 1, in 
import nonexisting_module
ImportError: No module named nonexisting_module# <--- ok.

(this is on windows, but on osx I see the same behavior).


Is there a problem with the rewritten import logic in Python 3.3?

Thanks
Irmen de Jong
-- 
http://mail.python.org/mailman/listinfo/python-list


Re: wrong ImportError message printed by python3.3 when it can't find a module?

2012-12-07 Thread Irmen de Jong
On 7-12-2012 22:20, Peter Otten wrote:

> A paragon of clarity -- ye lurkers, this is how a bug report should be.

:-)

> 
>> Is there a problem with the rewritten import logic in Python 3.3?
>>
>> Thanks
>> Irmen de Jong
> 
> I believe this is fixed, see http://bugs.python.org/issue15111

Argh, why didn't I search the bug tracker first: I would have found that one 
for sure.
Anyway, thanks for pointing it out.

The bug is confusing but it doesn't break anything so far. I guess I'll be fine 
waiting
for 3.3.1.


Irmen

-- 
http://mail.python.org/mailman/listinfo/python-list


need some help with unexpected signal exception when using input from a thread (Pypy 1.9.0 on osx/linux)

2012-12-15 Thread Irmen de Jong
Hi.
Using Pypy 1.9.0. Importing readline. Using a background thread to get input() 
from
stdin. It then crashes with:

  File "/usr/local/Cellar/pypy/1.9/lib_pypy/pyrepl/unix_console.py", line 400, 
in restore
signal.signal(signal.SIGWINCH, self.old_sigwinch)
ValueError: signal() must be called from the main thread

Anyone seen this before? What's going on?
When I don't import readline, or do the input() from within the main thread, 
the problem
disappears.

(I tried to reproduce it in a small test scenario but unfortunately have not 
been able
to do so yet. Haven't figured out yet what the additional factors are that 
trigger this
problem. A simple import readline and input() from a new thread doesn't seem to 
trigger
it, unfortunately)


Regards
Irmen de Jong
-- 
http://mail.python.org/mailman/listinfo/python-list


Re: Command Line Progress Bar

2012-12-26 Thread Irmen de Jong
On 26-12-2012 7:17, Kevin Anthony wrote:
> Hello, 
> I'm writing a file processing script(Linux), and i would like to have a 
> progress bar.
>  But i would also like to be able to print messages.  Is there a simple way 
> of doing
> this without implementing something like ncurses?

This little library can prove useful: http://pypi.python.org/pypi/fish/

-Irmen

-- 
http://mail.python.org/mailman/listinfo/python-list


Re: Inserting Unicode chars in Entry widget

2012-12-29 Thread Irmen de Jong
On 29-12-2012 17:43, Alan Graham wrote:
> Hello Python experts,
> 
> I want to insert Unicode chars in an Entry widget by pushing on buttons;
> one for each Unicode character I need. I have made the Unicode buttons.
> I just need a simple function that will send the Unicode character to
> the Entry widget.
> Is there a better approach?
> 
> Alan
> 

Not sure what the question is. A better approach to doing what?

I assuming you're doing tkinter (it is helpful if you mention the toolkit when 
posting a
question). I'd create a function that you bind to all 'unicode buttons', and 
let the
function insert the correct character depending on which button triggered it.

A possible way to do that is to use a lambda with a different parameter for 
every
button, like this:

b1=Button(f, text='char1', command=lambda b=1: insert_char(b))
b2=Button(f, text='char2', command=lambda b=2: insert_char(b))
...etc..

def insert_char(b):
if b==1:
entrywidget.insert(0, u"\u20ac")   # inserts € in the entry widget e
elif b==2:
entrywidget.insert(0, ...some other char...)
...


Or simply define a different command function for every button, then you don't 
have to
use the lambda.

-irmen

-- 
http://mail.python.org/mailman/listinfo/python-list


Re: Inserting Unicode chars in Entry widget

2012-12-29 Thread Irmen de Jong
On 29-12-2012 18:23, Chris Angelico wrote:
> On Sun, Dec 30, 2012 at 4:11 AM, Irmen de Jong  wrote:
>> b1=Button(f, text='char1', command=lambda b=1: insert_char(b))
>> b2=Button(f, text='char2', command=lambda b=2: insert_char(b))
>> ...etc..
>>
>> def insert_char(b):
>> if b==1:
>> entrywidget.insert(0, u"\u20ac")   # inserts € in the entry widget e
>> elif b==2:
>> entrywidget.insert(0, ...some other char...)
>> ...
> 
> I'm not familiar with tkinter syntax, but why not:
> 
> b1=Button(f, text='char1', command=lambda: insert_char(1))
> b2=Button(f, text='char2', command=lambda: insert_char(2))
> 
> or even:
> 
> b1=Button(f, text='char1', command=lambda: insert_char(u"\u20ac"))
> b2=Button(f, text='char2', command=lambda: insert_char("... some other
> char..."))
> 
> Seems weird to multiplex like that, but if there's a good reason for
> it, sure. I'm more of a GTK person than tkinter, and more of a
> command-line guy than either of the above.
> 
> ChrisA
> 

You're right there's nothing special about tkinter there, I was copying some 
existing
code a bit too literally. Simplify the lambdas as needed. :)

Irmen

-- 
http://mail.python.org/mailman/listinfo/python-list


Re: Beginner: Trying to get REAL NUMBERS from %d command

2012-12-30 Thread Irmen de Jong
On 30-12-2012 23:37, Alvaro Lacerda wrote:
> 
> I'm trying to get full number result using the %d command

Try %f instead. %d is the formatting symbol for integer numbers.
See http://docs.python.org/2/library/stdtypes.html#string-formatting-operations

Or have a look at what string.format() can do:
http://docs.python.org/2/library/string.html#format-examples


-irmen


-- 
http://mail.python.org/mailman/listinfo/python-list


Re: test failed: test_urlwithfrag

2013-01-07 Thread Irmen de Jong
On 7-1-2013 19:26, Elli Lola wrote:
> I never used python before and installed it today the first time, so I have 
> no idea what
> to do about this failure:
> 
> 
> $ ./python -m test -v test_urlwithfrag

[..snip..]

> ImportError: No module named 'test.test_urlwithfrag'
> 
> 1 test failed:
> test_urlwithfrag

The error message says it all, really: there is no regression test module called
"test_urlwithfrag".  (At least, not on my python 3.3 installation)

Check the contents of your /Lib/test directory to see what 
is
available instead.


Irmen

-- 
http://mail.python.org/mailman/listinfo/python-list


Re: Thorough Python 2.7.3 Windows Build Documentation?

2013-01-21 Thread Irmen de Jong
On 21-1-2013 18:16, Stephane Wirtel wrote:
> Hi Leonard,
> 
> Please, could you limit your text to 80 columns, because it's
> unreadable. Your text is too long :(

Stephane, shouldn't your news reader simply wrap the lines...? At least mine 
does.
(Thunderbird)

Irmen

-- 
http://mail.python.org/mailman/listinfo/python-list


serpent, a serializer based around ast.literal_eval

2013-01-21 Thread Irmen de Jong
Hi,

I've been toying a bit with ast.literal_eval. I've come up with "serpent", a 
serializer
based around that. Which means that it takes a Python object tree and turns it 
into a
serialized form that can be safely read back by ast.literal_eval().

Why I wrote serpent and didn't simply use repr()+ast.literal_eval:

* it serializes directly to bytes (utf-8 encoded), instead of a string, so it 
can
immediately be saved to a file or sent over a socket
* it encodes byte-types as base-64 instead of inefficient escaping notation 
that repr
would use (this does mean you have to base-64 decode these strings manually on 
the
receiving side to get your bytes back)
* it contains a few custom serializers for several additional Python types such 
as uuid,
datetime, array and decimal
* it tries to serialize unrecognised types as a dict (you can control this with
__getstate__ on your own types)
* it can create a pretty-printed (indented) output for readability purposes
* it works around a few quirks of ast.literal_eval() on the various Python 
implementations.

It works with Python 2.6+ (including 3.x), IronPython 2.7+, Jython 2.7+.

Serpent can be downloaded from Pypi: http://pypi.python.org/pypi/serpent
A simple example session can be seen here: https://gist.github.com/4588429


I'm considering writing Java and .NET counterparts for this as well so I'll be 
able to
exchange messages between the three.

What do you think? Would you consider this useful at all?


Cheers
Irmen de Jong
-- 
http://mail.python.org/mailman/listinfo/python-list


Re: Finding MIME type for a data stream

2012-03-08 Thread Irmen de Jong
On 8-3-2012 23:34, Tobiah wrote:
> Also, I realize that I could write the data to a file
> and then use one of the modules that want a file path.
> I would prefer not to do that.
> 
> Thanks
> 

Use StringIO then, instead of a file on disk

Irmen

-- 
http://mail.python.org/mailman/listinfo/python-list


Re: Raise X or Raise X()?

2012-03-11 Thread Irmen de Jong
On 11-3-2012 20:04, bvdp wrote:
> Which is preferred in a raise: X or X()? I've seen both. In my specific case 
> I'm dumping out of a deep loop:
> 
> try:
>   for ...
> for ...
>   for ...
> if match:
>raise StopInteration()
>  else ...
> 
> except StopInteration:
>print "found it"

"raise X" is a special case of the 3-args raise. Effectively it just raises an 
instance
of X which is constructed with an empty argument list. Therefore, "raise X()" is
equivalent, as far as I know.

See http://docs.python.org/reference/simple_stmts.html#the-raise-statement

Irmen

-- 
http://mail.python.org/mailman/listinfo/python-list


Re: Will MySQL ever be supported for Python 3.x?

2012-03-30 Thread Irmen de Jong
On 30-3-2012 23:20, John Nagle wrote:
> The MySQLdb entry on SourceForge
> (http://sourceforge.net/projects/mysql-python/)
> web site still says the last supported version of Python is 2.6.
> PyPi says the last supported version is Python 2.5.  The
> last download is from 2007.
> 
> I realize there are unsupported fourth-party versions from other
> sources. (http://www.lfd.uci.edu/~gohlke/pythonlibs/) But those
> are just blind builds; they haven't been debugged.
> 
> MySQL Connector (http://forge.mysql.com/projects/project.php?id=302)
> is still pre-alpha.


Try Oursql instead  http://packages.python.org/oursql/
"oursql is a new set of MySQL bindings for python 2.4+, including python 3.x"

Irmen

-- 
http://mail.python.org/mailman/listinfo/python-list


Re: Will MySQL ever be supported for Python 3.x?

2012-03-30 Thread Irmen de Jong
On 30-3-2012 23:46, John Nagle wrote:
> On 3/30/2012 2:32 PM, Irmen de Jong wrote:
>> Try Oursql instead  http://packages.python.org/oursql/
>> "oursql is a new set of MySQL bindings for python 2.4+, including python 3.x"
> 
>Not even close to being compatible with existing code.   Every SQL
> statement has to be rewritten, with the parameters expressed
> differently.  It's a good approach, but very incompatible.

You didn't state that it had to be compatible with existing code.
Also, since you asked about Python 3.x, surely there are other 
incompatibilities you
need to take care of in the existing code? (unless it's Python 3.x clean 
already...)

Irmen
-- 
http://mail.python.org/mailman/listinfo/python-list


Re: Installing Python 2.5.2 on Win 7

2012-04-01 Thread Irmen de Jong
On 1-4-2012 22:11, W. eWatson wrote:
> To solve this problem I thought I would install the software on my laptop 
> Win7 PC. When
> I tried PIL.1.1.6 I got several unexpected messages that seem to indicate 
> things were
> not going well. I have stopped to find out if it is really possible to 
> install this
> suite of software on Win7. Is it possible? If not, will uninstalling Python 
> also remove
> PIL?

I'm using PIL 1.1.7 (the latest version available from effbot's website) on 
Windows 7.
Admittedly, with Python 2.7.2, but there's an installer for Python 2.5 as well.
I assume it should work just fine.

What errors were you seeing?

And no, uninstall Python won't remove PIL - it has its own uninstaller.


Irmen


-- 
http://mail.python.org/mailman/listinfo/python-list


Re: ast.parse

2012-04-09 Thread Irmen de Jong
On 9-4-2012 13:53, Kiuhnm wrote:
> Is it a known fact that ast.parse doesn't handle line continuations and some 
> multi-line
> expressions?
> For instance, he doesn't like
> for (x,
>  y) in each([1,
>  2]):
> print(1)
> at all.
> Is there a workaround besides "repairing" the code on the fly?
> 
> Kiuhnm

What Python version are you using and what is the exact error? It works fine 
here
(Python 2.7.2):

>>> import ast
>>> code="""for (x,
...  y) in each([1,
...  2]):
... print(1)"""
>>> ast.parse(code)
<_ast.Module object at 0x02418A10>


Irmen

-- 
http://mail.python.org/mailman/listinfo/python-list


Re: Suggest design to accomodate non-unix platforms ?

2012-04-18 Thread Irmen de Jong
On 18-4-2012 15:35, Richard Shea wrote:
> ... which I think would work and be sufficiently flexible to deal with
> alternatives to putty.exe but is there a more established (...
> better !) way of doing this stuff ?

Perhaps install Cygwin and use its ssh.exe?
Or use the paramiko library? (which, I believe, implements SSH directly)

Irmen
-- 
http://mail.python.org/mailman/listinfo/python-list


Re: Create directories and modify files with Python

2012-04-30 Thread Irmen de Jong
On 1-5-2012 1:24, deltaquat...@gmail.com wrote:
> Hi, 0 I would like to automate some simple tasks I'm doing by hand. Given a 
> text file
>  foobar.fo:

[...]

> At first, I tried to write a bash script to do this. However, when and if the 
> script
> will work, I'll probably want to add more features to automate some other 
> tasks. So I
> thought about using some other language, to have a more flexible and 
> mantainable
> code. I've been told that both Python and perl are well suited for such 
> tasks, but
> unfortunately I know neither of them. Can you show me how to write the script 
> in
> Python? Thanks,

Err.. if you don't know Python, why do you think a Python script will be more 
flexible
and maintainable for you?

But if you really want to go this way (and hey, why not) then first you'll have 
to learn
some basic Python. A good resource for this might be: 
http://learnpythonthehardway.org/

Focus on file input and output, string manipulation, and look in the os module 
for stuff
to help scanning directories (such as os.walk).

Irmen
-- 
http://mail.python.org/mailman/listinfo/python-list


Re: Create directories and modify files with Python

2012-05-01 Thread Irmen de Jong

On 1-5-2012 12:51, deltaquat...@gmail.com wrote:

But if you really want to go this way (and hey, why not) then first
you'll have to learn some basic Python. A good resource for this
might be: http://learnpythonthehardway.org/


Ehm...name's not exactly inspiring for a newbie who's short on time
:)


It's just a name. That resource is generally regarded as one of the
better books to learn Python for total beginners.

If you can program already in another language, the standard Python
tutorial might be more efficient to start from:
http://docs.python.org/tutorial/


Thanks for the directions. By the way, can you see my post in Google
Groups? I'm not able to, and I don't know why.


I don't use Google groups. I much prefer a proper news agent to read my 
usenet newsgroups.


Irmen
--
http://mail.python.org/mailman/listinfo/python-list


Re: .py to .pyc

2012-05-13 Thread Irmen de Jong
On 13-5-2012 23:27, Colin J. Williams wrote:
> Is there some way to ensure that a .pyc file is produced when executing a .py 
> file?
> 
> It seems that for small files the .pyc file is not produced.
> 
> Colin W.

All modules no matter how small produce a .pyc file *when they are imported*. 
If you
start a module directly though, no .pyc is produced. (Notice that Python 3 puts 
the pyc
files in a subdirectory.)

Why do you care anyway? Pyc files are an implementation detail.

Irmen
-- 
http://mail.python.org/mailman/listinfo/python-list


Re: installing 2 and 3 alongside on MS Windows

2012-05-26 Thread Irmen de Jong
On 25-5-2012 10:24, Ulrich Eckhardt wrote:
> Hi!

> What I'm considering is installing Python 3 alongside, in order to
> prepare the code for this newer version. What I'd like to know first is
> whether there are any problems I'm likely to encounter and possible
> workarounds.

What I'm doing myself on Windows is deciding which version of Python I want to 
be the
default, and install that from the msi normally (including "register extensions"
options). I then proceed to add its install location to my %PATH%.

After which I install additional Python versions but *without* selecting 
"register
extensions" otherwise they will overwrite the registry associations.

I have about 5 different versions coexisting peacefully in c:\python26 
c:\python27
c:\python27-64, c:\python32 and c:\python33  (with python27 being the default 
and only
c:\python27 added to %PATH%)

In the IDE I'm using (PyCharm) you can trivially switch the Python version it 
will use
for your project.


> Thank you!
> 
> Uli
> 
> PS: Dear lazyweb, is there any way to teach VC8 some syntax highlighting
> for Python?

http://pytools.codeplex.com/  perhaps?


Irmen.


-- 
http://mail.python.org/mailman/listinfo/python-list


Re: Install lxml package on Windows 7

2012-05-29 Thread Irmen de Jong
On 29-5-2012 22:41, David Fanning wrote:
> Folks,
> 
> I need some help. I need the lxml package to run a particular
> Python program. 
> 
>http://lxml.de/
> 
> I downloaded the appropriate binary egg package for lxml, and
> I found easy_install.exe in my Python 2.7 distribution. I ran
> that.
> 
> Then, at the command prompt I typed this:
> 
>easy_install --allow-hosts=lxml.de,*.python.org lxml==2.3.4


[..snip..]

Save yourself the pain trying to get it to build from source. Instead, just 
install a
precompiled binary, for instance the one available from:
http://www.lfd.uci.edu/~gohlke/pythonlibs/#lxml

-Irmen
-- 
http://mail.python.org/mailman/listinfo/python-list


Re: ./configure

2012-06-04 Thread Irmen de Jong
On 4-6-2012 0:01, Janet Heath wrote:

> 
> Thanks Alain.  I should have a compiler on my Mac OS X Lion.  I am thinking 
> that it
> isn't set in my $PATH variable.  I don't know where the $PATH is set at.  I 
> will
> check to see if their is a binary.


You'll have to have the Apple Developer tools (xcode) installed to get a 
c-compiler, but
maybe that's the case on your mac already. (Weird though, it should add gcc -the
compiler- to the PATH by itself, I don't recall having to change that myself.)

In any case, there is a .dmg with a binary installer for Python 2.7.3 available 
for
download at python.org [1] so there should not really be a need to compile it 
yourself.
If you really do need to compile it yourself, consider using Homebrew [2] to 
install it,
instead of downloading the source and compiling it manually. With homebrew it's 
a simple
'brew install python' and a few minutes later it's done. It will be lot easier 
to
install 3rd party library dependencies this way too.

Irmen.

[1] http://python.org/download/releases/2.7.3/
[2] http://mxcl.github.com/homebrew/
-- 
http://mail.python.org/mailman/listinfo/python-list


Re: Function call arguments in stack trace?

2011-06-07 Thread Irmen de Jong
On 7-6-2011 21:31, Dun Peal wrote:
> On Jun 7, 1:23 pm, Neil Cerutti  wrote:
>> Use pdb.
> 
> Neil, thanks for the tip; `pdb` is indeed a great debugging tool.
> 
> Still, it doesn't obviate the need for arguments in the stack trace.

If you can't use pdb perhaps you can use the following:

Pyro has always had a feature that prints detailed stacktraces. It is mainly 
meant to
clarify stacktraces that occur on a different machine (where you don't have the 
option
of using pdb), but can very well be used for normal code too:


import sys
import Pyro4.util
Pyro4.config.DETAILED_TRACEBACK=True
sys.excepthook=Pyro4.util.excepthook

def divide(a,b):
return a//b

def dividebysomething(a):
return divide(a,0)

print dividebysomething(10)


When you run this, this will be printed:

[E:\projects]python trace.py
--
 <> RAISED : integer division or modulo by 
zero
 Extended stacktrace follows (most recent call last)
--
File "trace.py", line (13), in 
Source code:
print dividebysomething(10)
File "trace.py", line (11), in dividebysomething
Source code:
return divide(a,0)
Local values:
 a = 10
--
File "trace.py", line (8), in divide
Source code:
return a//b
Local values:
 a = 10
 b = 0
--
 <> RAISED : integer division or modulo by 
zero
--


You can find the relevant code that produces these kinds of tracebacks in the 
util.py
source file of Pyro. You can get that from Pypi:
http://pypi.python.org/pypi/Pyro4/
or the file directly from subversion:
$ svn export svn://svn.razorvine.net/Pyro/Pyro4/trunk/src/Pyro4/util.py

Perhaps you can use this or adapt it to suit your needs.


Irmen de Jong
-- 
http://mail.python.org/mailman/listinfo/python-list


Re: To install lxml (and easy_install) for Python 3 under Windows...

2011-06-12 Thread Irmen de Jong
On 12-6-2011 18:38, ncdave4l...@mailinator.com wrote:
> I had difficulty installing lxml for Python 3.1 under Windows, and
> took some notes as I worked through it.  Here's how I finally managed
> it...
> 
> 
[...]

In cases like this, Christoph Gohlke's page with 'Unofficial Windows Binaries 
for Python
Extension Packages' can be very handy:

http://www.lfd.uci.edu/~gohlke/pythonlibs/#lxml


-irmen
-- 
http://mail.python.org/mailman/listinfo/python-list


Infinite recursion in __reduce__ when calling original base class reduce, why?

2011-06-13 Thread Irmen de Jong
Hi,

I'm having a rather obscure problem with my custom __reduce__ function. I can't 
use
__getstate__ to customize the pickling of my class because I want to change the 
actual
type that is put into the pickle stream. So I started experimenting with 
__reduce__, but
am running into some trouble.

I've pasted my test code below. It works fine if 'substitute' is True, but as 
soon as it
is set to False, it is supposed to call the original __reduce__ method of the 
base
class. However, that seems to crash because of infinite recursion on Jython and
IronPython and I don't know why. It works fine in CPython and Pypy.

I wonder if my understanding of __reduce__ is wrong, or that I've hit a bug in
IronPython and Jython?  Do I need to do something with __reduce_ex__ as well?

Any help is very much appreciated.

Irmen de Jong



# ironpython / jython __reduce__ recursion problem test program

import pickle

class Substitute(object):
def __init__(self, name):
self.name=name
def getname(self):
return self.name

class TestClass(object):
def __init__(self, name):
self.name=name
self.substitute=True
def getname(self):
return self.name
def __reduce__(self):
if self.substitute:
return Substitute, ("SUBSTITUTED:"+self.name,)
else:
# call the original __reduce__ from the base class
return super(TestClass, self).__reduce__()  #  crashes on 
ironpython/jython

obj=TestClass("janet")

s=pickle.dumps(obj)
d=pickle.loads(s)
print(d)
print(d.getname())

# now disable the substitution and try again
obj.substitute=False
s=pickle.dumps(obj)
d=pickle.loads(s)
print(d)
print(d.getname())
-- 
http://mail.python.org/mailman/listinfo/python-list


Re: Infinite recursion in __reduce__ when calling original base class reduce, why?

2011-06-14 Thread Irmen de Jong
On 14-6-2011 2:40, Chris Torek wrote:

> 
> Nonetheless, there is something at least slightly suspicious here:
[... snip explanations...]

Many thanks Chris, for the extensive reply. There's some useful knowledge in it.

My idea to call the base class reduce as the default fallback causes the 
problems:
return return super(TestClass, self).__reduce__()
If, following your suggestion, I replace that with:
return self.__class__, (self.name,)
it works fine.

By the way, in the meantime I've played around with copyreg.pickle, and that 
code worked
in all Python implementations I've tried it in. This code feels better too, 
because it
is possible to simply use
return self.__reduce__()
as a fallback in it, because we're not touching the object's own __reduce__ in 
any way.

Anyway thanks again

Irmen
-- 
http://mail.python.org/mailman/listinfo/python-list


Re: debugging https connections with urllib2?

2011-06-18 Thread Irmen de Jong
On 18-6-2011 20:57, Roy Smith wrote:
> We've got a REST call that we're making to a service provider over https 
> using urllib2.urlopen().  Is there any way to see exactly what's getting 
> sent and received over the network (i.e. all the HTTP headers) in plain 
> text?  Things like tcpdump and strace only have access to the encrypted 
> data.
> 
> We can't just switch to using http because the endpoint we're talking to 
> (and don't have control over) refuses plain-text connections.

Put a proxy between the https-service endpoint and your client app.
Let the proxy talk https and let your client talk http to the proxy.

Irmen


-- 
http://mail.python.org/mailman/listinfo/python-list


Re: How can I speed up a script that iterates over a large range (600 billion)?

2011-06-21 Thread Irmen de Jong

On 21-06-11 21:48, John Salerno wrote:

I'm working on the Project Euler exercises and I'm stumped on problem
3:

"What is the largest prime factor of the number 600851475143 ?"

Now, I've actually written functions to get a list of the factors of
any given number, and then another function to get the prime numbers
from that list. It works fine with small numbers, but when I try to
feed my get_factors function with the above number (600 billion),
naturally it takes forever!


You need a better algorithm to calculate primes, and iterate over primes 
instead of over the full (half, or even better, sqrt(n)) range of 
possible values. You also should optimize the stop condition, once you 
find that the number can no longer be divided by larger primes you can 
stop the loop.


For instance to get the prime factors of the number 1000 you'd iterate
over the prime numbers 2,3,5 and conclude that

1000=2*2*2*5*5*5, so 5 would be the largest prime factor.

No need to try larger primes than 5, let alone go through 1..sqrt(1000).

The Sieve of Eratosthenes is a well known algorithm to calculate primes 
with reasonable efficiency.


Irmen
--
http://mail.python.org/mailman/listinfo/python-list


sorry, possibly too much info. was: Re: How can I speed up a script that iterates over a large range (600 billion)?

2011-06-21 Thread Irmen de Jong

On 21-06-11 22:10, Irmen de Jong wrote:
[stuff]

I didn't read the last paragraph of John's message until just now, and 
now realize that what I wrote is likely way too much information for 
what he asked.
I'm sorry. Next time I'll read everything until and including the last 
full stop.


Irmen
--
http://mail.python.org/mailman/listinfo/python-list


Re: sorry, possibly too much info. was: Re: How can I speed up a script that iterates over a large range (600 billion)?

2011-06-21 Thread Irmen de Jong
On 21-6-2011 23:09, John Salerno wrote:
> On Jun 21, 3:22 pm, Irmen de Jong  wrote:
>> On 21-06-11 22:10, Irmen de Jong wrote:
>> [stuff]
>>
>> I didn't read the last paragraph of John's message until just now, and
>> now realize that what I wrote is likely way too much information for
>> what he asked.
>> I'm sorry. Next time I'll read everything until and including the last
>> full stop.
>>
>> Irmen
> 
> Don't worry, I was still unclear about what to do after reading all
> the responses, even yours! But one thing that made me feel better was
> that I wasn't having a Python problem as much as a *math* problem. I
> changed my get_factors function to only go as far as the square root
> of the number in question, and that yielded an answer immediately. :)

Ok, cool :)


> However, even after reading the Wikipedia page about prime numbers and
> trial division, I'm still a little unclear as to why the square root
> of the number is the upper bound of the range you need to check.

If there is an exact divisor d >= √n, then the quotient n/d is also a divisor 
of n, and
that quotient is <= √n. So if we don't find such a quotient before reaching √n, 
it's not
useful to search for d > √n (because no divisors would be found).

Irmen
-- 
http://mail.python.org/mailman/listinfo/python-list


Re: Security test of embedded Python

2011-06-22 Thread Irmen de Jong
On 22-6-2011 4:44, Chris Angelico wrote:
> Followup: The test box has been administratively taken offline after
> about an hour of testing. Thank you to everyone who participated; it
> seems we have a lot of changes to make!
> 
> Monty failed the test. But it was an incredibly successful test. And
> hopefully, we'll be bringing things back online for another shot once
> things are sorted out!
> 
> Chris Angelico

Maybe you should have a look at sandboxed pypy?
http://pypy.org/features.html#sandboxing

(disclaimer: never used it myself)

Irmen

-- 
http://mail.python.org/mailman/listinfo/python-list


Re: sorry, possibly too much info. was: Re: How can I speed up a script that iterates over a large range (600 billion)?

2011-06-22 Thread Irmen de Jong
On 22-6-2011 5:02, John Salerno wrote:

> Thanks. So far they are helping me with Python too, but definitely not
> as much as more general exercises would, I'm sure. The part about
> writing the code is fun, but once that's done, I seem to end up stuck
> with an inefficient implementation because I don't know the math
> tricks behind the problem.
> 
> I'll check out rubyquiz.com. I've been searching for some Python
> exercises to do but haven't found too many sites with them, at least
> not in such a nice and organized way as Project Euler.

There's also SPOJ; http://www.spoj.pl/problems/classical/
It has tons of problems, many math related, but also silly ones like this:
https://www.spoj.pl/problems/JAVAC/
and perhaps even useful ones like this:
https://www.spoj.pl/problems/ONP/

Irmen


-- 
http://mail.python.org/mailman/listinfo/python-list


Re: windows 7 create directory with read write execute permission for everybody

2011-06-26 Thread Irmen de Jong
On 26-6-2011 22:57, Gelonida wrote:
> Hi,
> 
> What do I have to do under python windows to create a directory with all 
> permissions,
> such, that new files / directories created below will inherit the permissions.
> 
> 
> The reason I am asking is, that I'd like to create a directory structure 
> where multiple
> users should be allowed to read / write / create files and directories.

Isn't this the default when you create a new directoy in Windows? (unless you're
creating it in some location where access is restricted, for instance C:\ or 
c:\program
files).  I'd try os.mkdir first in any case and check if it does the job.


> Alternatively it would be even better to specify exactly which users should 
> be allowed
> to access the directory tree.

Sorry, can't help you with this. I guess you'll need to use the windows 
extensions for
Python here and deal with user accounts and ACL's.

Irmen
-- 
http://mail.python.org/mailman/listinfo/python-list


Re: windows 7 create directory with read write execute permission for everybody

2011-06-26 Thread Irmen de Jong
On 26-6-2011 23:53, Gelonida wrote:
> 
> Yep I'm afraid that's the way to go and where I hoped somebody would have a 
> few tiny
> example lines or pointers to the functions in question to be used.

Maybe this is a bit of a help:
http://timgolden.me.uk/python/win32_how_do_i/add-security-to-a-file.html

Irmen
-- 
http://mail.python.org/mailman/listinfo/python-list


Re: Problem!!

2011-07-03 Thread Irmen de Jong
On 4-7-2011 1:41, amir chaouki wrote:
> i have written code on linux for parsing text files and it works great
> but when i try to run it on windows it goes crazy, do you have any
> idea???

No, I misplaced my crystal ball.

Irmen


P.S.  http://www.catb.org/~esr/faqs/smart-questions.html
-- 
http://mail.python.org/mailman/listinfo/python-list


Pyrolite - a lightweight interface library to connect java to python

2011-07-16 Thread Irmen de Jong
Hi,

What's a java interface library doing in comp.lang.python, you might ask.

Well, this one, called 'Pyrolite' is meant to be a lightweight library (<50kb) 
to
interface your java application to Python in a very easy and straightforward 
way.

Pyrolite uses the Pyro protocol to call methods on remote objects. This means 
it also
contains an almost complete implementation of Python's pickle protocol (which 
can be
used independently).

A small piece of example code (java):

import net.razorvine.pyro.*;
NameServerProxy ns = NameServerProxy.locateNS(null);
PyroProxy something = new PyroProxy(ns.lookup("Your.Pyro.Object"));
Object result = something.call("methodname",42,"arguments",[1,2,3]);


Info, source, download and javadocs:   http://irmen.home.xs4all.nl/pyrolite/
Readonly direct svn repository access:  svn://svn.razorvine.net/Various/Pyrolite
More info on Pyro:  http://irmen.home.xs4all.nl/pyro/


Pyrolite is an experiment. You can use Jython+Pyro as well but I was interested 
in the
possibilities of a lightweight client-only library. Please let me know your 
thoughts
about it, and if you decide to use it :-)


Enjoy!

Irmen de Jong
-- 
http://mail.python.org/mailman/listinfo/python-list


Re: Any suggestion to start more threads at the same time?

2011-07-28 Thread Irmen de Jong
On 28-7-2011 23:07, smith jack wrote:
> I start many threads in order to make the work done, when the
> concurrent number is set to 300, all thing just works fine, but when
> the number is set to 350 or higher, error just comes out? what's wrong
> ? the error info is just as follows:   failed to start .
> 
> I am confused, does this have something to do with the operating
> system, i am now using Linux, any suggestion to make the system to
> support more python threads?
> 
> thank you :)

I don't think that many threads are going to help you in any meaningful way. 
Especially
with Python's GIL. Can't you redesign your program to use a fixed number of 
threads such
as 1 per cpu core?

Or check out the multiprocessing module.

But yeah, I think your OS is preventing you from creating more threads (either 
due to
some artificial limit or due to lack of system memory).

Irmen

-- 
http://mail.python.org/mailman/listinfo/python-list


Re: PIL question. having exactly same font on multiple platforms

2011-08-04 Thread Irmen de Jong
On 4-8-2011 20:54, Gelonida N wrote:

> The reason why I want the images to  look identical is very simple.
> Though the final web server will run on a linux server, I use sometimes
> windows for development or as test server.
> 
> For automated tests I would have liked pixel identical images.
> this allows calculating the md5sum of images to know whether
> a regression in the image layout occured. Well I can live without it.

Then don't run your automated tests on the dev server but deploy your stuff to a
separate test server first, that runs the same software as production. And run 
the tests
there.


> The second (more import issue) is, that the images should look the same
> whether created on a Windows or Linux host.
> 
> I didn't know that PIL delegated font rendering to the underlying OS,
> but thought it contains its own rendering.

I'm pretty sure it does have that indeed; it links with the freetype library. 
Are you
sure you're not just seeing differences because of differences in the typeface 
itself?

A courier.ttf on one system can look very different from a ms-courier-new.ttf 
on another...

> Here the problem is not if a few pixels are different, but currently I
> even don't know how to choose a font, and make sure it exists on both
> platforms. I also don't know how I can write portable python code,  that
> will find a given font on windows and on linux independent of the exact
> font location.

I once made a module that uses PIL to draw captcha images. It uses one of the 
free
truetype font files that I just place next to the code. I downloaded the ttf 
files from
http://www.fontgirl.com/  but there are dozens of free font sites. Just be sure 
to check
the license terms of the the typeface files.

As far as I know, I did not see any difference in output on windows, linux and 
mac os x
as long as the code used the same ttf file and PIL versions. (but I'll double 
check now
and see if I remember this correctly).

Irmen
-- 
http://mail.python.org/mailman/listinfo/python-list


Re: PIL question. having exactly same font on multiple platforms

2011-08-04 Thread Irmen de Jong
On 4-8-2011 21:30, Irmen de Jong wrote:

> As far as I know, I did not see any difference in output on windows, linux 
> and mac os x
> as long as the code used the same ttf file and PIL versions. (but I'll double 
> check now
> and see if I remember this correctly).

To follow up on myself, I've just tested it with the same ttf file on windows, 
os x and
linux. (made an image in font size 40 with 'The quick brown fox' as text 
line) The
resulting image files were byte-identical between os x and linux but the 
windows ones
were slightly different (just by a few bytes). The text in the image itself 
looked
identical though to the ones from the other systems (I couldn't see any 
difference by
looking at it) so I presume the difference is in the image compression or file 
header
metadata.

PIL's font rendering seems to produce identical results across platforms and if 
you see
a difference it must be caused by a difference in typeface file (or perhaps a 
different
PIL/freetype version on the platform).

Irmen
-- 
http://mail.python.org/mailman/listinfo/python-list


Re: Table Driven GUI Definition?

2011-08-05 Thread Irmen de Jong

On 05-08-11 19:53, Tim Daneliuk wrote:

I have a task where I want to create pretty simple one page visual
interfaces (Graphical or Text, but it needs to run across Windows,
Cygwin, Linux,*BSD, OSX ...).  These interfaces are nothing more
than option checklists and text fields.  Conceptually something like:

 Please Select Your Installation Options:

Windows Compatibility Services  _
Linux Compatibility Services_
TRS-DOS Compatibility Services  _

What Is Your email Address: ___

What I'm looking for is a way to describe such forms in a text
file that can then be fed into a tool to generate the necessary
pyGUI, Tkinter, (or whatever) code.   The idea is that it should
be simple to generate a basic interface like this and have it
only record the user's input.  Thereafter, the python code
would act on the basis of those selection without any further
connection to the GUI.

An added bonus would be a similar kind of thing for generating
web interfaces to do this.  This might actually be a better model
because then I only have to worry about a single presentation
environment.

Ideas anyone?


Yeah, HTML being the text file and a web browser being the tool to 
transform it into a GUI...


You can hook this up with a simple web server or web framework running 
locally to grab the submitted form results when the form is complete and 
process them in a piece of python code.


Wouldn't that work?


Irmen
--
http://mail.python.org/mailman/listinfo/python-list


Re: Object Diffs

2011-08-09 Thread Irmen de Jong

On 08-08-11 21:50, Croepha wrote:

Hello Python list:

I am doing research into doing network based propagation of python
objects.  In order to maintain network efficiency. I wan't to just
send the differences of python objects, I was wondering if there
was/is any other research or development in this area? I was thinking
that I could look at how pickle works an implement a diff system
there, or I could actually create a pickle of the object and then use
difflib to compare the flattened text...

Ideas, or comments welcome



No Python code, but you may want to have a look at the DeltaCompressor 
of the Kryo serializer (written for Java).

http://code.google.com/p/kryo/

(disclaimer: I haven't used it, just read a little bit about it)

Irmen
--
http://mail.python.org/mailman/listinfo/python-list


Re: SocketServer expceion after upgrading to 2.7

2011-08-10 Thread Irmen de Jong

On 10-08-11 15:42, Laszlo Nagy wrote:

Exception happened during processing of request from ('80.99.165.122',
56069)
Traceback (most recent call last):
File "/usr/local/lib/python2.7/SocketServer.py", line 284, in
_handle_request_noblock
self.process_request(request, client_address)
File "/usr/local/lib/python2.7/SocketServer.py", line 311, in
process_request
self.shutdown_request(request)
File "/usr/local/lib/python2.7/SocketServer.py", line 459, in
shutdown_request
request.shutdown(socket.SHUT_WR)
TypeError: shutdown() takes exactly 0 arguments (1 given)


I get this error with a program, after upgrading to python 2.7. I'm
using a program that is based on SocketServer and SimpleXMLRPCDispatcher.

Any idea how to fix this?


Strange, I expected at least one or two more stack frames in the 
traceback you posted (from within the socket.py module).


What platform are you on?
What python version?
What does this print on your system:

>>> import socket
>>> s=socket.socket()
>>> s.shutdown(socket.SHUT_WR)


Irmen
--
http://mail.python.org/mailman/listinfo/python-list


Re: Dialog boxes in curses

2011-08-13 Thread Irmen de Jong
On 13-8-2011 17:21, f...@slick.airforce-one.org wrote:
> Hello.
> 
> I've googled for hints but I didn't find anything, I hope it's not an
> RTFM question :^)
> 
> I want to have dialog boxes (a message with Yes/No/Cancel options,
> possibly with keyboard accels) in python + curses.
> 
> Does anyone have a pointer to docs about this?
> 
> Thanks!
> 


Have you considered using Urwid instead? http://excess.org/urwid/


Irmen

-- 
http://mail.python.org/mailman/listinfo/python-list


Re: Ten rules to becoming a Python community member.

2011-08-14 Thread Irmen de Jong
On 14-8-2011 7:57, rantingrick wrote:
> 8. Use "e.g." as many times as you can! (e.g. e.g.) If you use "e.g."
> more than ten times in a single post, you will get an invite to
> Guido's next birthday party; where you'll be forced to do shots whist
> walking the balcony railing wearing wooden shoes!

I lolled about this one, e.g. I laughed out loud. But where are the tulips and 
windmills
for extra credit?

Greetings from a Dutchman!

Irmen
-- 
http://mail.python.org/mailman/listinfo/python-list


Re: pythonw.exe

2011-08-14 Thread Irmen de Jong
On 14-8-2011 15:23, Ronald Reynolds wrote:
> Dear Python Friends:
> in my python directory there is a python.exe file which I understand 
> completely but there is also a pythonw.exe DOS seems to honor the pythonw
> command (No error message) but nothing happens. What is pythonw.exe?
> Also is there a way to invoke idle from the DOS prompt?  I tried idle 
> filename.py  and  just idle. Is there any .exe for idle?
> 
> Sincerely, 'Ron "bumpker" Reynolds' 

pythonw.exe is the same as python.exe but it doesn't open a console window, and 
launches
python in the background. This allows you to easily run background programs or 
GUI
programs in a nicer way (without a dummy console window popping up).

Idle has no .exe as far as I know but you can start it like this:

pythonw -m idlelib.idle

You could create an alias or batch file called 'idle' that does this.

Irmen
-- 
http://mail.python.org/mailman/listinfo/python-list


Re: Idea for pure-python templates using AST.

2011-08-16 Thread Irmen de Jong

On 16-08-11 13:33, Paul Wray wrote:


The idea:
Python syntax allows a statement to be a bare literal or identifier.
These have no effect on the program.

So the function below is legal python:

def myFunc():
'a'
x = 45
'b'; 'c'; x

So is this (within the appropriate class context of course):

def body(self, r):
''; self.heading; ''
''
for itm in self.items:
''; itm; ''
''



Looks very similar to PTL what Quixote uses:
http://www.quixote.ca/overview/paper.html

(never used it though, and I doubt Quixote used ASTs)

Irmen
--
http://mail.python.org/mailman/listinfo/python-list


Re: Python Windows Extensions for Mac

2011-08-21 Thread Irmen de Jong
On 21-8-2011 1:51, Johnny Venter wrote:
> Thank you all for the replies. I would like to query various Windows' objects 
> and
> resources from Mac and/or Linux such as Active Directory users, network 
> shares, group
> members, etc... What module or methods can I use with python to accomplish 
> this?

A way to approach this problem is installing Python + windows extensions on the 
actual
windows machine(s) you want to query.

Then create some form of a server process that does the windows specific stuff 
locally,
and exposes an interface with the functions you want to provide to your remote
machine(s).  Talk with the server process using some form of platform 
independent IPC,
for instance Pyro: http://pypi.python.org/pypi/Pyro4/

Be careful what methods you expose this way though (security issues!)

Irmen
-- 
http://mail.python.org/mailman/listinfo/python-list


Re: relative speed of incremention syntaxes (or "i=i+1" VS "i+=1")

2011-08-21 Thread Irmen de Jong

On 21-08-11 19:03, Laurent wrote:

Well I agree with you about string concatenation, but here I'm talking about 
integers incrementation...


Seems the two forms are not 100% identical:

>>> import dis
>>> def f1(x):
...  x=x+1
...
>>> def f2(x):
...  x+=1
...
>>>
>>> dis.dis(f1)
  2   0 LOAD_FAST0 (x)
  3 LOAD_CONST   1 (1)
  6 BINARY_ADD
  7 STORE_FAST   0 (x)
 10 LOAD_CONST   0 (None)
 13 RETURN_VALUE
>>> dis.dis(f2)
  2   0 LOAD_FAST0 (x)
  3 LOAD_CONST   1 (1)
  6 INPLACE_ADD
  7 STORE_FAST   0 (x)
 10 LOAD_CONST   0 (None)
 13 RETURN_VALUE
>>>


What the precise difference (semantics and speed) is between the 
BINARY_ADD and INPLACE_ADD opcodes, I dunno. Look in the Python source 
code or maybe someone knows it from memory :-)


Irmen

--
http://mail.python.org/mailman/listinfo/python-list


Re: Pickle a list

2011-03-08 Thread Irmen de Jong

On 07-03-11 17:38, Rogerio Luz wrote:

import sys
import pickle

class MyClass:
 teste = 0
 nome = None
 lista = ["default"]

 def __init__(self):
 for reg in range(1,10):
 self.lista.append(reg)


   ^^
This probably doesn't do what you think it does.
It actually appends a range of numbers to the class attribute 'lista',
and not to the instance attribute of that name (which doesn't exist).

If you create multiple objects of type MyClass you'll notice that 
everytime the list gets longer and longer (...in *all* objects, because 
they still share the single class attribute!)



 self.nome = "TestStr"
 self.teste = 19900909



[...snip...]

The myClass object you're pickling doesn't have a 'lista' attribute.
While you can print myClass.lista without problems, you're printing the 
class attribute instead.
Pickle won't include class attributes. It just pickles the object's 
__dict__. If you add a line: print(myClass.__dict__)
before the pickle() call you'll see that 'lista' is not in there. And 
that means that when you read the pickle back in, the new object won't 
have the 1,2,3,4,5 numbers in the lista list, instead it just has 
the initial list.


You probably want to initialize self.alist in the class's __init__ 
method instead. That way it is a normal object attribute and will get 
pickled normally.



Irmen de Jong
--
http://mail.python.org/mailman/listinfo/python-list


Re: Do you monitor your Python packages in inux distributions?

2011-03-12 Thread Irmen de Jong

On 12-3-2011 20:26, s...@pobox.com wrote:


I'm one of the SpamBayes developers and in a half-assed way try to keep
track of SB dribbles on the net via a saved Google search.  About a month
ago I got a hit on an Ubuntu bug tracker about a SpamBayes bug.  As it turns
out, Ubuntu distributes an outdated (read: no longer maintained) version of
SpamBayes.  The bug had been fixed over three years ago in the current
version.  Had I known this I could probably have saved them some trouble, at
least by suggesting that they upgrade.

I have a question for you people who develop and maintain Python-based
packages.  How closely, if at all, do you monitor the bug trackers of Linux
distributions (or Linux-like packaging systems like MacPorts) for activity
related to your packages?  How do you encourage such projects to push bug
reports and/or fixes upstream to you?  What tools are out there to discover
which Linux distributions have SpamBayes packages?  (I know about
rpmfind.net, but there must be other similar sites by now.)


Hello Skip.
I'm the author of Pyro and had something similar happening a while ago.
Ubuntu being the distribution in case, and they decided to upgrade their package to the 
still-not-stable version 4 of Pyro. But version 4 is incompatible with Pyro3. 
Predictably, soon afterwards, complaints and bug reports started to appear.


I wasn't aware of all this until I was contacted by a Debian (or Ubuntu, but it doesn't 
really matter) user directly to alert me to the problem. I *did* know that Debian and 
Ubuntu have been including Pyro as a package for quite a while, but I'm not following 
their activity much.


(Via the package 'homepage' I contacted the package maintainer and things have been 
sorted out.)


I tend to glance on the deb package 'homepage' once in a while (few months) to see if 
something interesting pops up in their tracker and such, but I'm doing that randomly.
Also I'm not actively encouraging anybody to push tracker issues upstream to me. I'm not 
spending much time on Pyro, and it seems people know how to find me anyway. What distros 
decide to do with it is their business, I'm just hoping the package maintainer is smart 
enough to know what to do with the issues reported on their trackers (if any).


Besides Debian and Ubuntu I'm not aware of any other distro that includes a Pyro 
package. I found out about Debian/Ubuntu by googling.


All in all probably not a very helpful example but I thought I'd share my 
experience.

Cheers
Irmen de Jong
--
http://mail.python.org/mailman/listinfo/python-list


Re: How to schedule execution of code?

2011-03-15 Thread Irmen de Jong

On 15-03-11 08:16, Virgil Stokes wrote:

Suppose that I have some Python code (vers. 2.6) that has been converted
into an *.exe file and can be executed on a Windows (Vista or 7)
platform. What can one do to have this *.exe executed at a set of
specific times each day? In addition, if a day is missed (e.g. computer
on which it resides goes down), then it will be executed the next time
the computer is successfully started up.

It might be useful to know that the purpose of this code is to collect
data from a set of RSS feeds.

Any suggestions would be appreciated.



Have you tried the Windows Task Scheduler tool?

Irmen

--
http://mail.python.org/mailman/listinfo/python-list


Re: What breaks if I remove lib/python2.7/test/* ?

2011-04-19 Thread Irmen de Jong
On 19-4-2011 19:06, cjblaine wrote:
> What breaks if I remove lib/python2.7/test/* ?  What purpose does it serve?
> 
> It is 26MB for me.
> 
> I am trying to trim my Python install for good reason.
> 
> Thanks for any info!

Terry answered what it is for. Personally, I also once used some functions from
test.test_support in my own code. There's some handy stuff in there. Also, the 
venerable
Pystone lives in test.pystone.

However, I don't think the test/* stuff is actually considered to be part of the
standard library? I believe it becomes available on many Linux installations 
only after
installing the python-dev package (or similar). So my guess is that you are 
safe to
delete it if you really need the disk space.

-Irmen
-- 
http://mail.python.org/mailman/listinfo/python-list


Re: Problem receiving UDP broadcast packets.

2011-04-19 Thread Irmen de Jong
On 20-4-2011 0:21, Grant Edwards wrote:
> I'm have problems figuring out how to receive UDP broadcast packets on
> Linux.
> 
[...]

> Here's the sending code:
> 
> send.py---
> #!/usr/bin/python
> import sys,socket,time
> 
> host = sys.argv[1]
> port = 5010
> 
> s = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
> s.setsockopt(socket.SOL_SOCKET, socket.SO_BROADCAST, 1)
> s.bind((host,port))

I don't think you should use s.bind() at all in the sending code.
Could that be at least part of the problem?


-Irmen
-- 
http://mail.python.org/mailman/listinfo/python-list


Re: Problem receiving UDP broadcast packets.

2011-04-19 Thread Irmen de Jong
On 20-4-2011 1:21, Grant Edwards wrote:
> 
> If I don't call bind(), then the broadcast packets go out the wrong
> interface on the sending machine.
> 

Fair enough.

Next issue then: as far as I know, broadcast packets are by default not routed 
across
subnets by gateways. Which is a good thing.

That would explain why your receiver doesn't see the packets unless its 
interface IP
address is in the same subnet as the sender's.

However it doesn't explain (for me) why the tcpdump program running on that same
receiver machine still happily spits out received packets. Unless the routing 
between
the subnets is somehow done on the receiving machine itself? My knowledge of 
networks
and TCP/IP ends here I'm afraid.

Cheers
Irmen.


-- 
http://mail.python.org/mailman/listinfo/python-list


Re: Remote Connection

2011-04-21 Thread Irmen de Jong

On 20-04-11 05:26, ray wrote:


The speech commands will scripted in Python.  Dragonfly is the Python
project to coordinate this but does not address connectivity.

So I am wondering if there have been any Python projects to address
the connectivity.

ray


I'm not quite sure what you want exactly, but maybe one or more of the 
available Python rpc modules/libraries can be used to your needs?


They offer communication between Python processes on separate machines, 
but may not be entirely suitable for your needs of audio data transfer.
Still, they should be able to transfer chunks of audio data in some way 
to a remote machine. Some protocols are better suited for this than others.


Some libraries that may be useful perhaps:
Pyro (see http://pypi.python.org/pypi/Pyro4/ )
xmlrpclib (part of the standard library)
or perhaps even httplib or ftplib ?

Hth,
Irmen.

--
http://mail.python.org/mailman/listinfo/python-list


Re: suggestions, comments on an "is_subdict" test

2011-04-22 Thread Irmen de Jong
On 22-4-2011 15:55, Vlastimil Brom wrote:
> Hi all,
> I'd like to ask for comments or advice on a simple code for testing a
> "subdict", i.e. check whether all items of a given dictionary are
> present in a reference dictionary.
> Sofar I have:
> 
> def is_subdict(test_dct, base_dct):
> """Test whether all the items of test_dct are present in base_dct."""
> unique_obj = object()
> for key, value in test_dct.items():
> if not base_dct.get(key, unique_obj) == value:
> return False
> return True
> 
> I'd like to ask for possibly more idiomatic solutions, or more obvious
> ways to do this. Did I maybe missed some builtin possibility?


I would use:

test_dct.items() <= base_dct.items()

-irmen
-- 
http://mail.python.org/mailman/listinfo/python-list


Re: windows 7 x64 shutdown

2011-04-25 Thread Irmen de Jong
On 25-4-2011 23:15, rjmccorkle wrote:
> does anyone know a solution to shutting down windows 7 x64 via python
> script?  the win32 obviously doesn't work... something similar?

http://goo.gl/5tVPj
(a recipe on activestate, First hit on Google for 'python ctypes shutdown')

Works fine on my win7 x64 system. I don't think that using 32-bits API will 
stop you
from doing stuff like this.

-irmen

-- 
http://mail.python.org/mailman/listinfo/python-list


Re: Have you read the Python docs lately?

2011-04-27 Thread Irmen de Jong
On 27-4-2011 19:56, Raymond Hettinger wrote:
> A number of developers have been working on adding examples and useful
> advice to the docs.  To sharpen your skills, here are some pieces of
> recommended reading:
> 
> http://docs.python.org/dev/library/heapq.html#priority-queue-implementation-notes
> 
> http://docs.python.org/dev/library/bisect.html#searching-sorted-lists
> 
> http://docs.python.org/dev/library/re.html#writing-a-tokenizer
> 
> http://docs.python.org/dev/library/cmd.html#cmd-example
> 
> http://docs.python.org/dev/library/collections.html#ordereddict-examples-and-recipes
> 
> http://docs.python.org/dev/howto/logging.html
> 
> http://docs.python.org/dev/howto/sorting.html
> 
> http://docs.python.org/dev/library/collections.html#collections.namedtuple
> 
> 
> Raymond

Awesome. More reasons to browse the docs online instead of using the ones that 
get
installed locally with your Python distribution :)

Irmen
-- 
http://mail.python.org/mailman/listinfo/python-list


Re: Today's fun and educational Python recipe

2011-05-04 Thread Irmen de Jong

On 04-05-11 20:17, Raymond Hettinger wrote:

Here's a 22-line beauty for a classic and amazing algorithm:
http://bit.ly/bloom_filter

The wiki article on the algorithm is brief and well-written:
http://en.wikipedia.org/wiki/Bloom_filter

It turns out that people in the 1970's were pretty smart :-)



I think that often, the cleverness of people is inversely proportional 
to the amount of CPU power and RAM that they have in their computer.


"Meh, what would I need such a thing for, I could just as well stick 
everything into a list"


Thankfully there are also people for whom this doesn't count.
(are they stuck in the '70s?)

Also: wasn't there a talk on Pycon in which a bloom filter was mentioned?


Irmen
--
http://mail.python.org/mailman/listinfo/python-list


Re: Today's fun and educational Python recipe

2011-05-04 Thread Irmen de Jong

On 04-05-11 21:13, Raymond Hettinger wrote:

It turns out that people in the 1970's were pretty smart :-)


I think that often, the cleverness of people is inversely proportional
to the amount of CPU power and RAM that they have in their computer.


The Google guys have plenty of CPU power *and* plenty of
cleverness :-)


Haha, true. We need to add a Googlyness factor in the equation.
Or perhaps: think what they could achieve if they only had a few 
machines instead of thousands ;-)




Also: wasn't there a talk on Pycon in which a bloom filter was mentioned?


Yes!  As a matter of fact there was:
http://www.slideshare.net/c.titus.brown/pycon-2011-talk-ngram-assembly-with-bloom-filters


Thanks, that was the one.

I didn't attend Pycon but I watched a truckload of talks on blip.tv and 
that one caught my attention because of its somewhat funny title

'handling ridiculous amounts of data with probabilistic data structures'

I didn't understand all of it but the name Bloom Filter stuck, it seems. 
 Adding it to my list of bookmarks of 
useful-stuff-I-intend-to-use-one-day-in-the-future...


Irmen
--
http://mail.python.org/mailman/listinfo/python-list


Re: PIL: The _imaging C module is not installed

2011-05-06 Thread Irmen de Jong

On 06-05-11 15:56, Nico Grubert wrote:



However, running the selftest still fails:
$ python selftest.py
*** The _imaging C module is not installed



I had this happening to me as well someday.
I recall that first installing it (python setup.py install), and then 
rerunning selftest, solved that error.


Irmen
--
http://mail.python.org/mailman/listinfo/python-list


Re: PIL: The _imaging C module is not installed

2011-05-09 Thread Irmen de Jong
On 9-5-2011 8:22, Nico Grubert wrote:
>> I had this happening to me as well someday.
>> I recall that first installing it (python setup.py install), and then
>> rerunning selftest, solved that error.
> 
> I tried that as well.
> Here is the summary of the install process:
> 
> build/temp.linux-x86_64-2.4/libImaging/ZipEncode.o -L/usr/local/lib 
> -L/usr/lib -ljpeg
> -lz -o build/lib.linux-x86_64-2.4/_imaging.so
[...]
> 
> $ python selftest.py
> *** The _imaging C module is not installed
> 

Don't know why the build seems to fail. PIL installs and works fine on all 
computers
I've installed it on recently...

What happens when you just forget about selftest and run python and type import 
Image?

Doesn't the package manager on CentOS offer a python-imaging, python-pil or 
whatever
precompiled and configured package? If not (why not!?) try pip install PIL.


Irmen
-- 
http://mail.python.org/mailman/listinfo/python-list


Re: Finding the bitness of an arbitrary executable with Python

2011-05-09 Thread Irmen de Jong
On 9-5-2011 22:52, Andrew Berg wrote:
> I need to find whether a given file is 32-bit or 64-bit (and raise an
> exception if the file doesn't exist or isn't an executable file). I
> thought platform.architecture() would do this, but it returns ('64bit',
> '') no matter what value I assign to the executable parameter (looks
> like it uses the given executable to find info on the underlying system
> rather than info on the specific file, reverting to sys.executable if
> there are any errors). A quick look on Google doesn't give me anything
> useful. Something cross-platform would be nice, but it really only needs
> to work on Windows (on win32 binaries).

A few options are mentioned here:

http://stackoverflow.com/questions/1345632/determine-if-an-executable-or-library-is-32-or-64-bits-on-windows

-irmen

-- 
http://mail.python.org/mailman/listinfo/python-list


experiments with dictionary attacks against password hashes, in Python

2011-05-21 Thread Irmen de Jong
Hi,

I've been experimenting a little with dictionary attacks against password 
hashes.

It turned out that Python is plenty fast for this task, if you use precomputed 
hash
databases. I used a few rather large dictionary files (most of the words of the 
English
language, and most of the words of the Dutch language including derived forms) 
for a
total of almost 600,000 precomputed hashes. With that the program can "crack" 
10,000
password hashes in under a second on my 3 year old PC.

I've also used a list of 600 'most commonly used' passwords that I gathered 
from a few
sources. That list is used to generate a couple of variations, such as 
prefixing them
with a digit, or typing the word in uppercase, etc. I did this to be able to 
quickly
scan for the most common passwords, but it turned out that using all of the 
600,000
precomputed hashes isn't much slower for the experiments that I did.
The variations however increase the hit rate because words like "Jennifer9" are 
not in a
normal dictionary file. This one however *is* part of the 'most common' list.
So if that is your password, go change it right now ;-)


I thought the code I wrote might interest other people as well, so I share it 
here:
(It should run on Python 2.6 and up, including Python 3.x.)

Download:
http://www.razorvine.net/download/dictionary_attack/

Or by Subversion:
svn://svn.razorvine.net/Various/PythonStuff/trunk/dictionaryattack


Have fun,
Irmen de Jong
-- 
http://mail.python.org/mailman/listinfo/python-list


referring to package scope from module, using relative import?

2011-05-21 Thread Irmen de Jong
Hi,

I have a package with several modules in it. The package also has some objects 
created
in the package scope (done in the package __init__.py).

Is it possible to access those package scope objects from the modules, with 
relative
imports or something? So that I don't have to import the package itself in its 
own
submodules?


Example:

A/
   __init__.py->  creates A.something
   thing.py   ->  needs to do "import A"  to access A.something
  would like to not have to import A


I think it's something simple I'm missing...

Irmen



-- 
http://mail.python.org/mailman/listinfo/python-list


Re: referring to package scope from module, using relative import?

2011-05-21 Thread Irmen de Jong
On 21-5-2011 22:00, Ian Kelly wrote:

> Note that PEP 8 discourages relative imports and encourages absolute
> imports, though.  This would be the preferred way to do it:
> 
> from A import something

Right. I got rid of the silly relative import stuff. As an added bonus, this 
makes my
original question irrelevant :)

Irmen
-- 
http://mail.python.org/mailman/listinfo/python-list


Re: hash values and equality

2011-05-21 Thread Irmen de Jong
On 22-5-2011 0:55, John Nagle wrote:
> On 5/19/2011 11:33 PM, Ulrich Eckhardt wrote:
> 
>> For that reason, it is generally useful to use immutable types like
>> integers, floats, strings and tuples thereof as keys. Since you can't change
>> them, you basically have the guarantee that they hash the same.
> 
>Right.  It's something of a lack that Python doesn't
> have user-defined immutable objects.
> 
> John Nagle
> 

collections.namedtuple is rather powerful though... and can be used as key 
AFAIK.

Irmen
-- 
http://mail.python.org/mailman/listinfo/python-list


Re: The worth of comments

2011-05-27 Thread Irmen de Jong

On 27-05-11 15:54, Grant Edwards wrote:

On 2011-05-27, Ben Finney  wrote:

Richard Parker  writes:


On May 26, 2011, at 4:28 AM, python-list-requ...@python.org wrote:


My experience is that comments in Python are of relatively low
usefulness. (For avoidance of doubt: not *zero* usefulness, merely
low.)


I've seen plenty of comments who's usefulness was not zero.  It was
less than zero.


Someone once taught me, "There is one thing worse than having no 
comments in the source code:  having incorrect (or 'lying') comments in 
the code."


Grant, I guess you hint at such comments?

Irmen.
--
http://mail.python.org/mailman/listinfo/python-list


Re: The worth of comments

2011-05-28 Thread Irmen de Jong
On 27-5-2011 19:53, Grant Edwards wrote:
> On 2011-05-27, Irmen de Jong  wrote:
>> On 27-05-11 15:54, Grant Edwards wrote:
>>> On 2011-05-27, Ben Finney  wrote:
>>>> Richard Parker  writes:
>>>>
>>>>> On May 26, 2011, at 4:28 AM, python-list-requ...@python.org wrote:
>>>>>
>>>>>> My experience is that comments in Python are of relatively low
>>>>>> usefulness. (For avoidance of doubt: not *zero* usefulness, merely
>>>>>> low.)
>>>
>>> I've seen plenty of comments who's usefulness was not zero.  It was
>>> less than zero.
>>
>> Someone once taught me, "There is one thing worse than having no 
>> comments in the source code: having incorrect (or 'lying') comments
>> in the code."
>>
>> Grant, I guess you hint at such comments?
> 
> Yes.  :)
> 
> When trying to find a bug in code written by sombody else, I often
> first go through and delete all of the comments so as not to be
> mislead.
> 
> The comments reflect what the author _thought_ the code did
> _at_some_point_in_the_past_.  What matters is what the code actually
> does at the present.
> 

I'm going to share this thread, and the funny slideshow about Uncomment your 
code, with
my team at work :-)
We're not a Python shop so I'm probably the only one reading this, but as usual 
there is
a lot of wisdom going on in this newgroup that is not only applicable to Python.

Irmen
-- 
http://mail.python.org/mailman/listinfo/python-list


Re: The worth of comments

2011-05-28 Thread Irmen de Jong
On 28-5-2011 15:36, Roy Smith wrote:
> In article ,
>  Grant Edwards  wrote:
> 
>> When trying to find a bug in code written by sombody else, I often
>> first go through and delete all of the comments so as not to be
>> mislead.
> 
> I've heard people say that before.  While I get the concept, I don't 
> like doing things that way myself.
> 
>>> The comments reflect what the author _thought_ the code did
>> _at_some_point_in_the_past_.  What matters is what the code actually
>> does at the present.
> 
> I don't look at it that way.  Most of the comments I write are to 
> document interfaces, i.e. what the code is supposed to do.  That's the 
> contract between you and the code.  If you call me with arguments that 
> meet these conditions, this is what I promise I'll return to you (and 
> what side effects I'll perform).

I don't see how that is opposed to what Grant was saying. It's that these 
'contracts'
tend to change and that people forget or are too lazy to update the comments to 
reflect
those changes.

The comments you are writing, saying what the code is supposed to do, are only 
saying
what the code is supposed to do at the moment in time that you were writing the 
comment
and/or the code, are they not?

> One reasonable definition of a bug is something the code actually does 
> which differs from the documented interface.  Unless you know what the 
> code is supposed to do, how can you possibly look at it and say whether 
> it has a bug or not?  For example, take this function:
> 
> def foo():
>l = [1, 2, 3]
>return l[3]
> 
> Is this code correct?

Unit tests should tell you.


>  I'll bet most people would look at this and say, 
> "I'm not sure what he had in mind, but whatever it was, this has to be a 
> bug because he's indexing past the end of the list".

I do agree that reading just that piece of code without other information, 
makes me
think that it is fishy. But:


>  Well, what if I 
> showed you the interface contract:
> 
> def foo():
>"Raise IndexError.  This is useful as a testing fixture."
>l = [1, 2, 3]
>return l[3]
> 
> Now it's obvious that the function does exactly what it's supposed to do 
> (even if it's not the best way to do it).

A month later the requirement changes: it should raise a ZeroDevisionError 
instead.
Someone modifies the code but leaves the comment (for whatever reason).

def foo():
"Raise IndexError.  This is useful as a testing fixture."
return 1//0

Unless there is a way to force people to update the code comment as well (which 
I'm not
aware of.. Doctests? dunno...),  you now have a comment that lies about the the 
intended
working.   In my opinion that is worse than what we had before (the function 
without
comment).


That being said, I do write code comments myself. Some of them are like notes, 
directed
to future-me or other future readers (remember that we do this because blabla, 
don't
change this to such-and-such because it will probably break xyz) and some of 
them are
stating the contract of the code (like you wrote above, about documenting 
interfaces).
I always try to avoid writing comments that duplicate the working of the code 
itself in
pseudo code or text phrases.  But humans make mistakes and forget things so 
after enough
modifications my code probably contains lies as well... :(


Irmen de Jong

-- 
http://mail.python.org/mailman/listinfo/python-list


Re: The worth of comments

2011-05-28 Thread Irmen de Jong
On 29-5-2011 2:47, Gregory Ewing wrote:
> Irmen de Jong wrote:
> 
>> I don't see how that is opposed to what Grant was saying. It's that these 
>> 'contracts'
>> tend to change and that people forget or are too lazy to update the comments 
>> to reflect
>> those changes.
> 
> However, I can't see that deleting the comment documenting the
> contract can be the right response to the situation.
> 
> If the contract comment doesn't match what code does, then
> there are two possibilities -- the comment is wrong, or the
> code is wrong. The appropriate response is to find out which
> one is wrong and fix it.

Fair enough.

Certainly I won't be deleting every source code comment encountered from now 
on, but I
do think we should keep in mind the risks already mentioned. In some situations 
I can
very well imagine it is best to simply delete any comments and go with just the 
code.


> If you simply delete the comment, then you're left with no
> redundancy to catch the fact that something is wrong.

You are right, if you don't have a Unit test for it.
Then again, faulty unit tests are a problem in their own right...

Irmen de Jong
-- 
http://mail.python.org/mailman/listinfo/python-list


Re: How to Use Setuptools, Alternatives?

2011-05-29 Thread Irmen de Jong
On 29-5-2011 23:41, ray wrote:
> I have Python 2.7 on Win7 Pro on a tightly locked down desktop.  I
> would like to install Networkx from an egg.  From what I have read,
> Setuptools can be used for this.

What does 'tightly locked down' mean?


> I don't know how to install Setuptools.  The exe will not work.  On
> execution, it reports that the Python version is not included in the
> registry.

That can be fixed by repairing the Python installation through its .msi.
Also, are you sure you used the python 2.7 setuptools exe?

> Further, I can not input the version and location on the
> subsequent installation screen, the fields will not accept focus so I
> can not input the values.
> 
> Since the exe will not install, I considered using the Setuptools
> egg.  But it requires Setuptools.  It appears to be a circle.
> 
> What are some suggestions for installing this?
> 
> Thanks,
> ray

Perhaps you can try VirtualEnv and PiP instead.

Irmen

-- 
http://mail.python.org/mailman/listinfo/python-list


Re: Alternatives to PythonPath

2011-05-29 Thread Irmen de Jong
On 29-5-2011 23:49, ray wrote:
> I am using Win7 on a tightly locked down desktop.
> 
> Is there an alternative to using PythonPath?
> 

What do you mean by "using PythonPath"?
What doesn't work that you want to have an alternative for?

Irmen
-- 
http://mail.python.org/mailman/listinfo/python-list


Re: python in school notebooks/laptops

2011-05-30 Thread Irmen de Jong
On 30-5-2011 16:30, jmfauth wrote:
> On 30 mai, 13:09, hackingKK  wrote:
> 
> [...]
> 
>>
>> Even better, try convincing them to use Ubuntu instead of  a virus
>> called Where I Never Do Operations With Safety, or WINDOWS for short.
>> That way Python will come by default  and VB will be out of question
>> Happy hacking.
>> Krishnakant.
> 
> Do you mean one of these os's, where Python (2) is not
> working properly because the *defaultencoding* is set
> to utf-8?
> 
> jmf

Huh? On all of my machines, including windows and Ubuntu 11.04, 
sys.getdefaultencoding()
returns 'ascii'.

Irmen



-- 
http://mail.python.org/mailman/listinfo/python-list


Re: wrote a commodore-64 emulator using just Python

2017-09-04 Thread Irmen de Jong
On 08/13/2017 03:50 PM, Irmen de Jong wrote:
> Hi,
> 
> As another experiment with using just tkinter for graphics, this time I 
> created a
> Commodore-64 emulator. You can find it here https://github.com/irmen/pyc64

[...]

> There's also https://github.com/mnaberez/py65 so... possibilities?

Well, I went ahead and integrated that with my emulator. With just a
slight modification (that is now merged into the py65 library) I could
hook it up to the bytearray that my emulator uses to simulate the C64's
RAM. And letting the 'SYS' basic command kick of the 6502 simulator,
rather than doing nothing, it suddenly was able to actually run real
(although simple) Commodore-64 programs. Speed seems to be around 0.5x
real-time on my machine.

The py65 library also already provides a simple machine code monitor
(assembler/disassembler) that you can use to inspect or write the
machine code in the C64's memory. With some effort it should be possible
to run it in the emulated screen, but for now, interaction with it is
done on the console prompt.

It was a great deal of fun integrating this into my project and I found
it quite spectacular to see some existing Commodore-64 programs actually
running unaltered.  Even when they're doing nothing more than changing
the screen colors and unpacking an image. At half speed. But still :-)


Irmen
-- 
https://mail.python.org/mailman/listinfo/python-list


a Boulder Dash clone with retro graphics and sound

2017-09-04 Thread Irmen de Jong
Hi,

Yet another continuation of my graphics experiments with tkinter. In the
previous project I've been using tkinter bitmaps to simulate a
commodore-64 screen where I exploited the possibility to change the
foreground and background color of the bitmap on the fly. This
conveniently matches the way the commodore 64 draws its (character-mode)
graphics.

This time I switched to a screen filled with hundreds of tkinter's
PhotoImage objects, that can display full color images. However
continuing in the spirit of the 8 bit era, instead of creating something
with high-res full color graphics, I created a Boulder Dash clone using
retro graphics.

It's a fully functional game. You can get it here:
https://github.com/irmen/bouldercaves
(if you just want to play it and aren't interested in the code, get the
.pyz Python zip app from the releases section)

The game contains the 20 levels of the original 1980's Boulder Dash, and
also plays music and soundfx. By default it uses a slightly more
colorful sprite set than the original, but you can tell it to switch to
the limited Commodore-64 color palette.

You need Python 3.5+ and pillow and sounddevice/pyaudio to play it. If
you disable sound, only pillow is enough. It runs very well at 30hz
refresh rate on my Linux box, but seem to have some performance issues
on Windows and Mac OS. Your mileage may vary, and you can tweak some
parameters on the command line to eventually make it run smoothly.

It was a joy to learn about the inner workings of one of my favorite
games when I was a kid, and how to translate that into modern software.

I haven't been able to finish the game so far! I guess I have to learn
again how to actually play this after 30 years.

There's just two things missing I think:
- high score table
- being able to play multiple sounds simultaneously, as the amoeba and
magic wall sounds are lacking at the moment.
(And the sprite set I used contains a lot more objects and creatures
than the original game so it begs for adding extended game logic and
levels with new mechanics... who knows...)

This project was heavily inspired by:
http://codeincomplete.com/posts/javascript-boulderdash/
and http://www.boulder-dash.nl/


Have fun :-)

Irmen

-- 
https://mail.python.org/mailman/listinfo/python-list


Re: a Boulder Dash clone with retro graphics and sound

2017-09-06 Thread Irmen de Jong
On 05/09/2017 00:02, Irmen de Jong wrote:

https://github.com/irmen/bouldercaves

> There's just two things missing I think:
> - high score table
> - being able to play multiple sounds simultaneously, as the amoeba and
> magic wall sounds are lacking at the moment.

In version 1.2 sound mixing is implemented so the game can now play multiple 
samples
simultaneously, and is able to play sounds in a loop. This means it now 
properly plays
the title screen music, and the amoeba and magic wall sounds are now also in 
the game.
(and I fixed a magic wall behavior bug as well)

Saving and displaying high scores are left as an exercise for the reader :)


Irmen
-- 
https://mail.python.org/mailman/listinfo/python-list


v2.0 released of: a Boulder Dash clone with retro graphics and sound

2017-09-10 Thread Irmen de Jong
On 06/09/2017 23:17, Irmen de Jong wrote:

> 
> https://github.com/irmen/bouldercaves
> 

My Boulder Dash clone is now at version 2.0 because a few important things that 
were
lacking are now implemented:

* authentic mode:
The game is now displayed in a small screen that scrolls smoothly over the 
level, like
the original game. It also has the retro c-64 color palette enabled. You enable 
this via
a command line option.

* pre-recorded demo:
If you press F9 or wait a while at the title screen, the game plays back a 
prerecorded
demo of cave A, like the original game.

* synthesized sounds:
Instead of using sampled sounds you can now run the game with a software sound
synthesizer that creates the sounds in real time, including the title music. 
I've tried
to simulate the sounds of the original game but ended up with slightly 
different ones.
Maybe I'll tweak the synth to make them closer to the original, but there's a 
charm to
the variation as well I think.


All in all I am very pleased with the results. I didn't expect it to be possible
creating a decently performing arcade game using just the bare essentials:
- default tkinter to draw everything on the screen
- pillow to load images
- sounddevice to play sounds (optional).

I posted this update because I now consider it a full game [1] and I think it's
interesting to see that this can be realized in Python without any of the 
specialized
'game libraries' (pygame, etc) being used. While tkinter sometimes has troubles 
updating
the screen at 30 hz, the Python code itself is barely breaking a sweat, even 
the sound
synth.


Have fun
Irmen de Jong


[1] full game: errr... there's still no highscore table. Sorry :)
-- 
https://mail.python.org/mailman/listinfo/python-list


Re: Fw: Problems Installing Python36

2017-09-20 Thread Irmen de Jong
On 14/09/2017 05:46, Michael Torrie wrote:
> On 09/12/2017 03:05 AM, Thomas Jollans wrote:
>> Other people on this list:
>> This isn't the first time I've someone with this issue here. It's
>> probably putting off plenty of potential new users who don't make as
>> much effort to find a solution. I can't say I understand the ins and
>> outs of installing things on Windows... is there anything that can be done?
> 
> Last time I brought this up, someone mentioned that the Python installer
> is supposed to automatically install this runtime library if it's not
> installed already.  If so, why are so many people having this problem?
> 

That is what I'm wondering as well.

The only thing I can think of is that it asks windows update to install said KB 
update
but that it depends on something else that isn't installed or that the user 
running the
installation doesn't have the rights to install windows updates. (I suspect 
something
will be logged in the event viewer somewhere...?)

Or that it doesn't attempt to download it at all and that we are misinformed :P


Btw, I personally never had any issues installing Python on Windows.

Irmen
-- 
https://mail.python.org/mailman/listinfo/python-list


Re: Fw: Problems Installing Python36

2017-09-22 Thread Irmen de Jong
On 09/22/2017 08:34 PM, Stephan Houben wrote:

> I was vaguely tempted to offer the Mingw-w64 (GCC) Python as an
> alternative, since it doesn't rely on any optionally-installed Microsoft
> DLLs and so avoids this issue. But I suppose that is not really the
> newbie-friendly solution the OP was looking for...

Mingw? Perhaps better to choose Anaconda/Miniconda instead if you go for
an alternative implementation...

Irmen
-- 
https://mail.python.org/mailman/listinfo/python-list


auto installing dependencies with pip to run a python zip application ?

2017-09-26 Thread Irmen de Jong
Hi,
I've been using Python's executable zip application feature to neatly
package my little game into a single "executable" file.
Here "executable" means the user can simply start it by doubleclicking
it, or launching it from a shell prompt. Of course the user already has
to have a proper Python installation on their system but that's okay for me.

However, I would like to also take care of the library dependencies of
my game. Currently it requires pillow and sounddevice. If the user
doesn't have these installed, the game exits with an error message
telling the user to install these dependencies manually. I was thinking
about a way to automate this so that you don't have to install anything
manually to run the game.

I was thinking about the following solution:
- assume bare default Python (3.5+) installation
- use venv.EnvBuilder() to create a new virtualenv somewhere in the
user's home directory (~./virtualenvs/mygreatgame ?)
- use the with_pip=True argument so we also get pip installed into it
- exec() the python executable from the new virtual env and let that
"pip install -r mygamerequirements.txt"
- inject "./mygame.pyz" in sys.modules[0]
- run the game's main class and play!

Notice that I'm not looking for a fully standalone executable created by
things as cx_freeze, just something that is usable from within my zipped
application.

Any thoughts on this? Is it a good idea or something horrible? Has
someone attempted something like this before perhaps?

Irmen
-- 
https://mail.python.org/mailman/listinfo/python-list


  1   2   3   4   5   6   >