Re: M2crypto
On Sun, Feb 12, 2012 at 4:00 PM, Mel Wilson wrote: > zigi wrote: > >> Hello, >> M2crypto >> >> __init__(self, alg, key, iv, op, key_as_bytes=0, d='md5', >> salt='12345678', i=1, padding=1) >> >> I wont write app, using M2crypto and I can not understand what are the >> arguments: >> key, iv, op, salt ? >> What they do ? > > I assume you're reading in <http://www.heikkitoivonen.net/m2crypto/api/> > about M2Crypto.EVP.Cipher. > > Epydoc claims another victim. > > I'm having a lot of trouble finding documentation. The obvious OpenSSL > pages are kind of thin, too. You might see some useful code in the EVP unit > tests m2crypto/tests/test_evp.py in the m2crypto installation. Not intending to be rude, but being perfectly serious: as a general rule, if you don't know what an IV is you're probably getting yourself into a lot of trouble working with low-level crypto libraries. Two suggestions: 1. Describe what you're trying to do- I'll be able to help more if I know what you're actually going for. 2. Try keyczar. It's not perfect, but it's a lot easier to get right. Geremy Condra -- http://mail.python.org/mailman/listinfo/python-list
Re: M2crypto
On Mon, Feb 13, 2012 at 12:37 AM, zigi wrote: > Hello, > this is must be testing time to crypt files, using just M2crypto :-) > I know this is a strange use of the library, but such is the will of > the management. I take it you mean that you're benchmarking file encryption performance using M2Crypto? If your only goal is to do exactly that then go for it, but otherwise you're probably using the wrong tool for the job. Geremy Condra > On 13 Lut, 01:28, geremy condra wrote: >> On Sun, Feb 12, 2012 at 4:00 PM, Mel Wilson wrote: >> > zigi wrote: >> >> >> Hello, >> >> M2crypto >> >> >> __init__(self, alg, key, iv, op, key_as_bytes=0, d='md5', >> >> salt='12345678', i=1, padding=1) >> >> >> I wont write app, using M2crypto and I can not understand what are the >> >> arguments: >> >> key, iv, op, salt ? >> >> What they do ? >> >> > I assume you're reading in <http://www.heikkitoivonen.net/m2crypto/api/> >> > about M2Crypto.EVP.Cipher. >> >> > Epydoc claims another victim. >> >> > I'm having a lot of trouble finding documentation. The obvious OpenSSL >> > pages are kind of thin, too. You might see some useful code in the EVP >> > unit >> > tests m2crypto/tests/test_evp.py in the m2crypto installation. >> >> Not intending to be rude, but being perfectly serious: as a general >> rule, if you don't know what an IV is you're probably getting yourself >> into a lot of trouble working with low-level crypto libraries. >> >> Two suggestions: >> >> 1. Describe what you're trying to do- I'll be able to help more if I >> know what you're actually going for. >> >> 2. Try keyczar. It's not perfect, but it's a lot easier to get right. >> >> Geremy Condra > > -- > http://mail.python.org/mailman/listinfo/python-list -- http://mail.python.org/mailman/listinfo/python-list
Re: [semi OT]: Smartphones and Python?
On Wed, Feb 15, 2012 at 12:58 PM, Martin Schöön wrote: > First of all: I don't have any first hand experience of smartphones > but now that my trusted old GSM phone is getting old I decided I am > in for an up-grade. It struck me it might be nice to get something > for which I could write Python programs. > > A very quick internet search indicated that this should be no big > deal if I go for an Android-based phone. What about the alterna- > tives? > > It struck me this must be the best place to ask. > > What else? I don't know if it matters but my home PC OS is Linux. > And I am not much of a Python programmer but I enjoy learning it > and I have reached a level that has turned out to be useful at work. Please note that while SL4A is a pretty good mobile python environment it doesn't support all of the Android API, which means it generally isn't an easy way to develop fully-fledged Android apps. Geremy Condra -- http://mail.python.org/mailman/listinfo/python-list
Re: python + php encrypt/decrypt
On Sun, Jun 5, 2011 at 3:34 AM, Peter Irbizon wrote: > Hello, thanks, Unfortunatelly I don't understand how xml should resolve my > issue. My problem is: > I am trying to use aes256 cbc on python and php to decrypt "textstring". But > results are not the same in php and python. Any idea why? password and iv is > the same so I don't know where is the problem. I am trying do decrypt data > in python then store it as base64 and read and decrypt it in php. > > 2011/6/4 >> >> Use xml to pass the encrypt text. >> >> On , Peter Irbizon wrote: >> > >> > Hello, >> > >> > I would like to encrypt text in python and decrypt it in my PHP script. >> > I tried to use pycrypto and some aes php scripts but the results are not >> > the >> > same. Please, is there any example (the best way source codes) how to do >> > this in python and PHP? Please provide links to the AES implementation you're trying to use from PHP and the Python and PHP code you're using. Geremy Condra -- http://mail.python.org/mailman/listinfo/python-list
Re: Bloom Filter in 22 lines of Python (updated)
On Fri, Jun 3, 2011 at 1:17 PM, Raymond Hettinger wrote: > Thanks for all the feedback on the earlier post. > > I've updated the recipe to use a cleaner API, simpler code, > more easily subclassable, and with optional optimizations > for better cache utilization and speed: > > http://code.activestate.com/recipes/577684-bloom-filter/ Any chance of this landing in collections? Geremy Condra -- http://mail.python.org/mailman/listinfo/python-list
Re: How good is security via hashing
On Tue, Jun 7, 2011 at 3:18 AM, Robin Becker wrote: > A python web process is producing files that are given randomized names of > the form > > hh-MMDDhhmmss-.pdf > > where rrr.. is a 128bit random number (encoded as base62). The intent of the > random part is to prevent recipients of one file from being able to guess > the names of others. > > The process was originally a cgi script which meant each random number was > produced thusly > > > pid is process id, dur is 4 bytes from /dev/urandom. > > random.seed(long(time.time()*someprimeint)|(pid<<64)|(dur<<32)) > rrr = random.getrandbits(128) > > > is this algorithm safe? Is it safe if the process is switched to fastcgi and > the initialization is only carried out once and then say 50 rrr values are > generated. The advice you got about just using urandom seems to be the best you're likely to get. Given how few values you have to pull out of random.random to reconstruct its state, the progress that's been made in the last few years on similar hidden state problems, and the limited amount of entropy you're feeding it in the first place, I'd probably stay away from this method. And besides, # adds random junk to the filename- should make it hard to guess rrr = os.urandom(16) fname += base64.b64encode(rrr) has to be easier to read and reason about than the process above. Geremy Condra -- http://mail.python.org/mailman/listinfo/python-list
Re: How good is security via hashing
On Tue, Jun 7, 2011 at 1:42 PM, Paul Rubin wrote: > geremy condra writes: >> # adds random junk to the filename- should make it hard to guess >> rrr = os.urandom(16) >> fname += base64.b64encode(rrr) > > Don't use b64 output in a filename -- it can have slashes in it! :-( > > Simplest is to use old fashioned hexadeimal for stuff like that, unless > the number of chars is a significant problem. Go for a more complicated > encoding if you must. Eeesh, that completely slipped my mind. Thanks for pointing it out. Geremy Condra -- http://mail.python.org/mailman/listinfo/python-list
Re: How good is security via hashing
On Tue, Jun 7, 2011 at 7:30 PM, Paul Rubin wrote: > Christian Heimes writes: >> PyCrypto has a strong pseudorandom number generator, too. > > If you mean the one at pycrypto.org, that page now says: > > Random number generation > > Do not use RandomPool to generate random numbers. Use Crypto.Random > instead. RandomPool is deprecated and will be removed in a future > release. See this thread to find out why. On a related note, keyczar just got bitten by this. > Crypto.Random just uses system randomness, which is the right thing to > do. It then goes and runs them through a distiller (Fortuna), which > seems a little bit silly to me, but harmless. IIRC this is mostly to help deal with the possibility of running on older Windows machines, where the cryptographic random number service was of very poor quality. Geremy Condra -- http://mail.python.org/mailman/listinfo/python-list
Re: Bloom Filter in 22 lines of Python (updated)
On Wed, Jun 8, 2011 at 1:05 PM, Raymond Hettinger wrote: > On Jun 6, 10:47 am, geremy condra wrote: >> On Fri, Jun 3, 2011 at 1:17 PM, Raymond Hettinger wrote: >> > Thanks for all the feedback on the earlier post. >> >> > I've updated the recipe to use a cleaner API, simpler code, >> > more easily subclassable, and with optional optimizations >> > for better cache utilization and speed: >> >> > http://code.activestate.com/recipes/577684-bloom-filter/ >> >> Any chance of this landing in collections? > > I would be happy to add something like this to collections > once the API matures. > > Right now, the constructor expects the user to know the desired > memory size and number of probes. Those could be computed > from the maximum number of unique keys and the tolerable > error false positive rate. > > The most convenient constructor could be: > b = BloomFilter(myseq) > where len(myseq) is presumed indicate the maximum number of > unique entries and there is a default tolerable false positive > rate of 1 in 10,000. > > The API already lets the user substitute different methods > for get_probes(). It should also allow the bytearray to be > easily replaced by other objects with indexable access > (array objects, mmap objects, etc). > > We could also provide union, intersection, and difference > operations but these would be a little awkward for general > purpose use because the result is only meaningful when > the sizes, probe functions, and input domain are all the same. > > Fortunately, there is a lot of prior art, so the API design > can be informed by what has worked well for others. Seems like the code that Dan Stromberg posted addresses a lot of this. Thoughts on his code? Also, if you're interested I'm about halfway through writing a C implementation, although it would have to be somewhat less flexible to get a real speed advantage. Right now there's a hefty enough speed penalty to the filter (the states test takes about 150 times as long with a bloom filter as with a normal set) that it may make sense to have a base case that uses an optimized C implementation but that can be overridden as needed. Geremy Condra -- http://mail.python.org/mailman/listinfo/python-list
Re: Any Better logic for this problem..
On Thu, Jun 9, 2011 at 4:38 AM, Dave Angel wrote:
> On 01/-10/-28163 02:59 PM, Chris Rebert wrote:
>>
>> On Thu, Jun 9, 2011 at 1:31 AM, Ganapathy Subramanium
>> wrote:
>>>
>>> Hi Guru's,
>>> I'm working on a solution to find the prime factor of the number
>>> This part of the code works.. http://www.pastie.org/2041584
>>>
>>> When the number gets bigger, the range cannot iterate through bigger
>>> number
>>> and it does not work.
>>> When I googled , I came across creating our own range function to solve
>>> this. I was just wondering if there was a better algorithm to get the
>>> prime
>>> numbers / prime factors of a long number?
>>>
>>> Any inputs is highly appreciated.
>>
>
> Others have pointed out various inefficiencies. But I wanted to start by
> asking what this is for. Do you really have a need to factor numbers over 2
> billion? Repeatedly? In multiple runs of the program? Do you have weeks
> of computer time to spend or just hours? Are you really interested in the
> factors, or just whether or not a particular large number is prime (==has
> anyfactors) ? If this is a homework assignment, what's the exact
> assignment? Are you permitted to use other libraries, or other languages?
> Are you permitted to use language features you haven't encountered yet in
> class?
My solution:
def factors(x):
status, output = subprocess.getstatusoutput('factor %d' % x)
if not status:
return [int(i) for i in output.split()[1:]]
else:
print(output)
Works pretty well.
> So you should probably turn the problem around. Design a function that
> calculates the nth prime, but that caches the work it's already done (on
> disk if appropriate, but in a list if not). In the loop that's finding the
> factors, simply call the first function each time, and each time you find a
> factor, divide num by that so you're dealing with a smaller number.
Just use a precomputed table to do your trial division. There's a list
of the first fifty million primes on prime pages- if you aren't
dealing with specially constructed values (ie, RSA moduli) and haven't
found a factor by the end of the first ten thousand or so you probably
need to do a primality check before moving on to trying to factor it.
Geremy Condra
--
http://mail.python.org/mailman/listinfo/python-list
Re: Any Better logic for this problem..
On Thu, Jun 9, 2011 at 6:10 PM, Dan Stromberg wrote:
>
> On Thu, Jun 9, 2011 at 10:55 AM, geremy condra wrote:
>>
>> On Thu, Jun 9, 2011 at 4:38 AM, Dave Angel wrote:
>> > On 01/-10/-28163 02:59 PM, Chris Rebert wrote:
>> >>
>> >> On Thu, Jun 9, 2011 at 1:31 AM, Ganapathy Subramanium
>> >> wrote:
>> >>>
>> >>> Hi Guru's,
>> >>> I'm working on a solution to find the prime factor of the number
>> >>> This part of the code works.. http://www.pastie.org/2041584
>> >>>
>> >>> When the number gets bigger, the range cannot iterate through bigger
>> >>> number
>> >>> and it does not work.
>> >>> When I googled , I came across creating our own range function to
>> >>> solve
>> >>> this. I was just wondering if there was a better algorithm to get the
>> >>> prime
>> >>> numbers / prime factors of a long number?
>> >>>
>> >>> Any inputs is highly appreciated.
>> >>
>> >
>> > Others have pointed out various inefficiencies. But I wanted to start by
>> > asking what this is for. Do you really have a need to factor numbers
>> > over 2
>> > billion? Repeatedly? In multiple runs of the program? Do you have
>> > weeks
>> > of computer time to spend or just hours? Are you really interested in
>> > the
>> > factors, or just whether or not a particular large number is prime
>> > (==has
>> > anyfactors) ? If this is a homework assignment, what's the exact
>> > assignment? Are you permitted to use other libraries, or other
>> > languages?
>> > Are you permitted to use language features you haven't encountered yet
>> > in
>> > class?
>>
>> My solution:
>>
>> def factors(x):
>> status, output = subprocess.getstatusoutput('factor %d' % x)
>> if not status:
>> return [int(i) for i in output.split()[1:]]
>> else:
>> print(output)
>>
>> Works pretty well.
>>
>>
>>
>> > So you should probably turn the problem around. Design a function that
>> > calculates the nth prime, but that caches the work it's already done (on
>> > disk if appropriate, but in a list if not). In the loop that's finding
>> > the
>> > factors, simply call the first function each time, and each time you
>> > find a
>> > factor, divide num by that so you're dealing with a smaller number.
>>
>> Just use a precomputed table to do your trial division. There's a list
>> of the first fifty million primes on prime pages- if you aren't
>> dealing with specially constructed values (ie, RSA moduli) and haven't
>> found a factor by the end of the first ten thousand or so you probably
>> need to do a primality check before moving on to trying to factor it.
>>
>> Geremy Condra
>> --
>> http://mail.python.org/mailman/listinfo/python-list
>
> You Might be able to benefit from a primality test like Miller-Rabin, at
> least if your numbers can be really large. It can answer with "this number
> is definitely composite" or "this number is probably prime". For quite
> large numbers, it might speed things up. For smaller numbers, trial
> division is faster.
>
> I have a Python Miller-Rabin module at:
>
> http://stromberg.dnsalias.org/svn/big-prime/trunk/
Here's a non-gmpy randomized MR implementation:
import random
def miller_rabin(n, confidence=20):
t, s, d = n-1, 0, 0
while not t % 2:
t = t >> 1
s += 1
t, d = n-1, t
for i in range(confidence):
a = random.randrange(2, n)
x = pow(a, d, n)
if x == 1: continue
if x == t: continue
for r in range(1, s):
x = pow(x, 2, n)
if x == t: break
if x == 1: return False
else: return False
return True
--
http://mail.python.org/mailman/listinfo/python-list
Re: What is the most efficient way to compare similar contents in two lists?
On Mon, Jun 13, 2011 at 9:30 AM, Chris Angelico wrote: > On Tue, Jun 14, 2011 at 2:04 AM, Steven D'Aprano > wrote: >> On Tue, 14 Jun 2011 01:39:50 +1000, Chris Angelico wrote: >> >>> Python: "Batteries Included". >>> >>> (Although Python 3 is "Most of the batteries you're used to, included".) >> >> Oh gods, you're not going to bring up sort(cmp=...) again are you >> >> /me ducks and covers > > Ha! That *lengthy* thread started fairly soon after I joined this > list. It was highly... informative. I learned a bit about Python, and > a lot about python-list, from the posts there. So what are we talking about here? Reduce? Geremy Condra -- http://mail.python.org/mailman/listinfo/python-list
Re: What is the most efficient way to compare similar contents in two lists?
On Mon, Jun 13, 2011 at 10:50 AM, Chris Angelico wrote: > On Tue, Jun 14, 2011 at 3:46 AM, geremy condra wrote: >> On Mon, Jun 13, 2011 at 9:30 AM, Chris Angelico wrote: >>> Ha! That *lengthy* thread started fairly soon after I joined this >>> list. It was highly... informative. I learned a bit about Python, and >>> a lot about python-list, from the posts there. >> >> So what are we talking about here? Reduce? > > There was a huge thread about the removal of the cmp parameter to > sort() in Python 3.x. It was quite a wide-ranging discussion, touched > on many things. I think it got up to 90 posts in the first day, or > something. The main thing it taught me was that python-list is a high > traffic list; the next most important thing was who the courteous > posters are. I know that, but I mean what were you talking about before if you weren't talking about cmp? > In contrast, the thread on NaNs in Python has taught me more about the > difference between floating point and 'real numbers' in a week or two > than all my previous computing training put together. Yep, this is an > interesting list. I liked that thread. There should be more of those. Geremy Condra -- http://mail.python.org/mailman/listinfo/python-list
Re: Python 2.6 OR 3.2
On Thu, Jun 9, 2011 at 11:00 PM, harrismh777 wrote: > Andrew Berg wrote: >> >> AFAICT, there are three reasons to learn Python 2: > > ... there is a fourth reason. > > The linux distro you are using currently was customized with python 2.x > > I ran into this problem this week in fact... on my HP g6 ubuntu notebook > running 10.04 lucid. It ships with the 2.6.5 interpreter. I installed 2.7.1 > and 3.2 (from sources) and was working along happy as a clam until I needed > to configure a printer... and the config tools would not function... some of > them would not even open. Want to guess? Yup, the config tools are (some > of them) written in python 2.6-- and they don't run in 2.7.1 nor 3.2 . :( Just a note, Ubuntu 11.04 has 2.7 preinstalled. Geremy Condra -- http://mail.python.org/mailman/listinfo/python-list
Re: Looking for Coders or Testers for an Open Source File Organizer
On Tue, Jun 14, 2011 at 7:54 AM, Matt Chaput wrote: > On 13/06/2011 11:55 PM, zainul franciscus wrote: >> >> Iknow you guys must be thinking "Hmm, Miranda, isn't that an IM >> application ?"; Yep I hear you, I'll change the name once I get a good >> name. I am open for any suggestions. > > Actually I was thinking "isn't that a functional programming language?" Same here. > My suggestion: Cruftbuster 'Phile' Geremy Condra -- http://mail.python.org/mailman/listinfo/python-list
Re: Python for Web
On Wed, Jun 15, 2011 at 5:11 AM, bruno.desthuilli...@gmail.com wrote: > On Jun 15, 9:50 am, sidRo wrote: >> Is Python only for server side? > > Is it a theoretical question or a practical one ?-) > > More seriously: except for the old proof-of-concept Grail browser, no > known browser uses Python as a client-side scripting language. Not quite the same thing, but there's pyjamas. Geremy Condra -- http://mail.python.org/mailman/listinfo/python-list
Re: Trapping MySQLdb warnings
On Wed, Jun 15, 2011 at 6:58 PM, Tim Johnson wrote: > Using Python 2.6.5 on linux. > > When using MySQLdb I am getting warnings printed to stdout, but I would > like to trap, display and log those warnings. > > In the past I have used _mysql_exceptions.Warning, but that approach > is not working in this case. > > My cursor is created with the relevant following code: > > ## connection object > self.__conn = MySQLdb.connect(db = self.__db, > host = self.__host, user = self.__user, > passwd = self.__passwd) > > ## cursor object > self.__rdb = self.__conn.cursor() > ## And implemented as : > try : > self.__rdb.execute(S) > except _mysql_exceptions.Warning,e: > raise e ## replace with log(e) > > What else needs to be done? Have you tried http://docs.python.org/library/warnings.html#temporarily-suppressing-warnings ? Geremy Condra -- http://mail.python.org/mailman/listinfo/python-list
Re: Does hashlib support a file mode?
On Wed, Jul 6, 2011 at 3:07 PM, Phlip wrote: > On Jul 6, 11:42 am, Andrew Berg wrote: >> On 2011.07.06 12:38 PM, Phlip wrote:> Python sucks. m = md5() looks like an >> initial assignment, not a >> > special magic storage mode. Principle of least surprise fail, and >> > principle of most helpful default behavior fail. >> >> func() = whatever the function returns >> func = the function object itself (in Python, everything's an object) >> >> Maybe you have Python confused with another language (I don't know what >> exactly you mean by initial assignment). Typically one does not need >> more than one name for a function/method. When a function/method is >> defined, it gets created as a function object and occupies the namespace >> in which it's defined. > > If I call m = md5() twice, I expect two objects. > > I am now aware that Python bends the definition of "call" based on > where the line occurs. Principle of least surprise. Python doesn't do anything to the definition of call. If you call hashlib.md5() twice, you get two objects: >>> import hashlib >>> m1 = hashlib.md5() >>> m2 = hashlib.md5() >>> id(m1) 139724897544712 >>> id(m2) 139724897544880 Geremy Condra -- http://mail.python.org/mailman/listinfo/python-list
Re: parsing packets
On Sun, Jul 10, 2011 at 4:59 PM, Littlefield, Tyler wrote: > Hello all: > I'm working on a server that will need to parse packets sent from a client, > and construct it's own packets. > The setup is something like this: the first two bytes is the type of the > packet. > So, lets say we have a packet set to connect. There are two types of connect > packet: a auth packet and a connect packet. > The connect packet then has two bytes with the type, another byte that notes > that it is a connect packet, and 4 bytes that contains the version of the > client. > The auth packet has the two bytes that tells what packet it is, one byte > denoting that it is an auth packet, then the username, a NULL character, and > a password and a NULL character. > > With all of this said, I'm kind of curious how I would 1) parse out > something like this (I am using twisted, so it'll just be passed to my > Receive function), and how I get the length of the packet with multiple NULL > values. I'm also looking to build a packet and send it back out, is there > something that will allow me to designate two bytes, set individual bits, > then put it altogether in a packet to be sent out? Just use the struct module[0] to do your packing for you. There's not enough info in your description to write it out for you, but it sounds like it should do everything you need. Geremy Condra [0]: http://docs.python.org/library/struct.html -- http://mail.python.org/mailman/listinfo/python-list
Re: Looking for general advice on complex program
On Fri, Jul 15, 2011 at 3:47 PM, Josh English wrote: > Maybe not to the gurus here, but for me, this is a complex problem and I want > to make sure I understand the real problem. > > All of this is in Python 2.7 and wxPython > > I have several XML files on a shared drive. > I have applications on other machines that need to access this XML file. > These applications need to read and write to this file. > These applications need to a) be alerted to a file change, or b) monitor the > file for changes and regular intervals. > > In my code, I have XManager classes (using a Singleton pattern) that reads > each XML file into a tree (using ElementTree). The XManager class can read > the file, make changes to the tree, and write the file as needed. > > Now I'm expanding to the multiple application level, and I think I understand > what I need to do, and I'm unsure about the exact processes. > > I've been trying to have the XManagers check periodically if the XML file > they monitor has changed. Since I don't want to mess up the GUI with constant > hanging, I think I can use the thread or threading modules to create a > recurring timed check, and then I need a separate check to see if the file is > in use before reading or writing. > > I also need, I think, to have a way to check if the GUI is editing a node > before the XManager reads the file, so the thread needs to be interrupted, or > paused, because I don't know if threads would stop if a wxDialog is being > show modally or not. > > So, am I on the right track here? I'd try watchdog[0] before I went to the trouble of rolling my own. Geremy Condra [0]: http://packages.python.org/watchdog/ -- http://mail.python.org/mailman/listinfo/python-list
Re: OT: Code Examples
On Tue, Mar 1, 2011 at 9:11 AM, Erik de Castro Lopo wrote: > Paul Rubin wrote: > >> Erik de Castro Lopo writes: >> > Why Python? I really can't understand the rush of every man and >> > his dog to Python. >> >> Are you trolling? > > All my responses to this thread are really mean for comp.dsp, > not for comp.lang.python. Ah, so you're looking for an argument. This is abuse, you want room 12A just down the hall. Geremy Condra -- http://mail.python.org/mailman/listinfo/python-list
Re: arbitrary precision linear algebra
On Wed, Mar 2, 2011 at 6:42 AM, Ben123 wrote: > Hello. I have a written Python program which currently uses numpy to > perform linear algebra operations. Specifically, I do matrix*matrix, > matrix*vector, numpy.linalg.inv(matrix), and linalg.eig(matrix) > operations. Now I am interested in allowing arbitrary precision. I > have tried gmpy, bigfloat, mpmath, and decimal but I have been unable > to easily implement any with my current program. I suspect I have to > change some commands but I am unsure what. > > My question is which of the arbitrary precision implementations will > most easily handle linear algebra? I don't care about speed, just ease > of use. Online tutorials for arbitrary precision linear algebra > operations would be useful. > > For example, it looks like mpmath can handle matrix operations > http://fredrik-j.blogspot.com/search?q=matrix > but I was unable to find a clear tutorial. The tutorials for most of > the arbitrary precision implementations demonstrate simple scalar > examples. > > Thanks in advance Have you looked at Sage[0]? I don't know for a fact, but you should be able to define a matrix over RealField(precision_in_bits) and then take the eigenvalue of it. I don't know if it will actually produce the precision you need though. Geremy Condra -- http://mail.python.org/mailman/listinfo/python-list
Re: arbitrary precision linear algebra
On Wed, Mar 2, 2011 at 10:21 AM, geremy condra wrote: > On Wed, Mar 2, 2011 at 6:42 AM, Ben123 wrote: >> Hello. I have a written Python program which currently uses numpy to >> perform linear algebra operations. Specifically, I do matrix*matrix, >> matrix*vector, numpy.linalg.inv(matrix), and linalg.eig(matrix) >> operations. Now I am interested in allowing arbitrary precision. I >> have tried gmpy, bigfloat, mpmath, and decimal but I have been unable >> to easily implement any with my current program. I suspect I have to >> change some commands but I am unsure what. >> >> My question is which of the arbitrary precision implementations will >> most easily handle linear algebra? I don't care about speed, just ease >> of use. Online tutorials for arbitrary precision linear algebra >> operations would be useful. >> >> For example, it looks like mpmath can handle matrix operations >> http://fredrik-j.blogspot.com/search?q=matrix >> but I was unable to find a clear tutorial. The tutorials for most of >> the arbitrary precision implementations demonstrate simple scalar >> examples. >> >> Thanks in advance > > Have you looked at Sage[0]? I don't know for a fact, but you should be > able to define a matrix over RealField(precision_in_bits) and then > take the eigenvalue of it. I don't know if it will actually produce > the precision you need though. > > Geremy Condra > Apologies, forgot the links: http://www.sagemath.org/doc/constructions/linear_algebra.html http://www.sagemath.org/doc/reference/sage/rings/complex_field.html Geremy Condra -- http://mail.python.org/mailman/listinfo/python-list
Re: arbitrary precision linear algebra
On Wed, Mar 2, 2011 at 10:47 AM, Ben123 wrote: > On Mar 2, 12:22 pm, geremy condra wrote: >> On Wed, Mar 2, 2011 at 10:21 AM, geremy condra wrote: >> > On Wed, Mar 2, 2011 at 6:42 AM, Ben123 wrote: >> >> Hello. I have a written Python program which currently uses numpy to >> >> perform linear algebra operations. Specifically, I do matrix*matrix, >> >> matrix*vector, numpy.linalg.inv(matrix), and linalg.eig(matrix) >> >> operations. Now I am interested in allowing arbitrary precision. I >> >> have tried gmpy, bigfloat, mpmath, and decimal but I have been unable >> >> to easily implement any with my current program. I suspect I have to >> >> change some commands but I am unsure what. >> >> >> My question is which of the arbitrary precision implementations will >> >> most easily handle linear algebra? I don't care about speed, just ease >> >> of use. Online tutorials for arbitrary precision linear algebra >> >> operations would be useful. >> >> >> For example, it looks like mpmath can handle matrix operations >> >>http://fredrik-j.blogspot.com/search?q=matrix >> >> but I was unable to find a clear tutorial. The tutorials for most of >> >> the arbitrary precision implementations demonstrate simple scalar >> >> examples. >> >> >> Thanks in advance >> >> > Have you looked at Sage[0]? I don't know for a fact, but you should be >> > able to define a matrix over RealField(precision_in_bits) and then >> > take the eigenvalue of it. I don't know if it will actually produce >> > the precision you need though. >> >> > Geremy Condra >> >> Apologies, forgot the links: >> >> http://www.sagemath.org/doc/constructions/linear_algebra.htmlhttp://www.sagemath.org/doc/reference/sage/rings/complex_field.html >> >> Geremy Condra > > I'm not sufficiently familiar with Sage, but from > http://www.sagemath.org/doc/constructions/linear_algebra.html > > "currently Sage does not implement multiprecision numerical > eigenvalues/eigenvectors" > > I'll ask on the Sage forums about this. In the mean time, I'm still > trying to get arbitrary precision linear algebra in Python I'd suggest you read that slightly more carefully. It's speaking specifically of RR and CC, ie, double-width reals and complex values. Using RealField and ComplexField- the arbitrary precision real and complex fields- seems to be working. Using the earlier example: sage: M1 = Matrix(RealField(1000), [[0, 2], [1, 0]]) sage: M2 = Matrix(RR, [[0, 2], [1, 0]]) sage: M1.eigenvalues() [1.41421356237309504880168872420969807856967187537694807317667973799073247846210703885038753432764157273501384623091229702492483605585073721264412149709993583141322266592750559275579995050115278206057147010955997160597027453459686201472851741864088919860955232923048430871432145083976260362799525140799, -1.41421356237309504880168872420969807856967187537694807317667973799073247846210703885038753432764157273501384623091229702492483605585073721264412149709993583141322266592750559275579995050115278206057147010955997160597027453459686201472851741864088919860955232923048430871432145083976260362799525140799] sage: M2.eigenvalues() [1.41421356237310, -1.41421356237310] Converting the first of the latter values to an element of RealField(1000) yields much what I would expect from higher precision arithmetic: R = RealField(1000) sage: x = M1.eigenvalues()[0] sage: y = R(M2.eigenvalues()[0]) sage: x 1.41421356237309504880168872420969807856967187537694807317667973799073247846210703885038753432764157273501384623091229702492483605585073721264412149709993583141322266592750559275579995050115278206057147010955997160597027453459686201472851741864088919860955232923048430871432145083976260362799525140799 sage: y 1.4142135623730951454746218587388284504413604736328125000 So, while I don't know for a fact that it's using the precision you need, it certainly does seem to be using high precision arithmetic here. Furthermore, repeating it for various precisions seems to increase the difference, as would be expected from better approximations, and the number of digits in the result is consistent with the interpretation that it is using the specified precision. All of this to say that it seems to be doing what you want it to do. Geremy Condra -- http://mail.python.org/mailman/listinfo/python-list
Re: Need an example program that implements rpm -pql via import rpm
On Thu, Mar 3, 2011 at 7:24 PM, Steven W. Orr wrote:
> I look everywhere but I couldn't find anything. Could someone please point
> me to a small example program that does an import rpm, takes an rpm file as
> an argument and gets the list of files contained in the file, the same as if
> I had used the commandline
>
> rpm -pql foo-1.23-4.i586.rpm
>
> Much appreciated.
#! /usr/bin/env python
import sys
import commands
if __name__ == "__main__":
rpm = sys.argv[1]
print commands.getoutput("rpm -pql %s" % rpm)
Input validation and help text left as an exercise for the reader.
Geremy Condra
--
http://mail.python.org/mailman/listinfo/python-list
Re: What do I need to know in order to write a web application in python?
On Sat, Mar 5, 2011 at 3:49 AM, ErichCart ErichCart wrote: > Visual Python seems to be exactly what I want. But it doesn't seem > very popular. Perhaps it means that there are not many people who will > be able to help if I have problems with it. Also judging by the amount > of ads at visualpython.org, it also doesn't seem very serious. > > I looked into pyGTK, and I found something called "Glade", which seems > to be something similar to visual python. The latest version of Glade > was released just this month, so it seems to be actively developed. > > Regarding Boa constructor, it is very old, isn't it? The latest news > from this project date to the end of 2006. I don't expect it to > support python 3 any time soon. > > So, "Glade", is this what everybody uses? I mean programmers don't > just use text editors to make GUI applications, do they? Yes, they do. It isn't that bad once you get used to it, and it beats the snot out of trying to maintain the insensible gibberish that some of the autogen tools put out. On a side note, you should check out pygui[0]- very, very nice GUI toolkit. Geremy Condra [0]: http://www.cosc.canterbury.ac.nz/greg.ewing/python_gui/ -- http://mail.python.org/mailman/listinfo/python-list
Re: I'm happy with Python 2.5
On Sun, Feb 27, 2011 at 6:27 AM, Tom Zych wrote: > n00m wrote: >> Am I turmoiling your wishful thinking? >> You may nourish it till the end of time. > > Let us cease to nourish those fabled ones who dwell under bridges. +1 QOTW. Geremy Condra -- http://mail.python.org/mailman/listinfo/python-list
Re: changing to function what works like a function
On Mon, Mar 7, 2011 at 8:32 PM, alex23 wrote: > On Mar 8, 1:41 pm, alex23 wrote: >> Good to see you finally admit that you're not a programmer. Have you >> informed your clients yet? Or are you still learning Python on their >> dime and crowd-sourcing the more difficult parts? > > I'd like to apologise for this post. The OP is not the Victor I > thought he was. If it's any consolation, I wrote a very similar email before coming to the same realization. Geremy Condra -- http://mail.python.org/mailman/listinfo/python-list
function annotations in open source projects
I use them in evpy to automatically wrap c functions with a decorated, annotated function. The decorator code is also up on aspn, just search for "c function decorator" and it should come up. I did a similar thing with java a few years ago as well. Geremy Condra -- http://mail.python.org/mailman/listinfo/python-list
Re: Fun python 3.2 one-liner
On Tue, Mar 29, 2011 at 11:24 AM, Raymond Hettinger wrote:
>>>> print('\n'.join('*'*(c//2000) for _,c in sorted(Counter(map(sum,
>>>> product(range(6), repeat=8))).items(
>
>
>
>
>
>
>
>
> *
> ***
> *
>
>
> **
> *
>
> *
> *
>
> **
> **
> ***
> **
> **
>
> *
> *
>
> *
> **
>
>
> *
> ***
> *
>
>
>
> Re-posting (forgot to attach the output).
> Besides the interesting output, other
> interesting virtues include very low
> memory usage and that the inner-loop
> pipeline runs entirely in C.
Another (related) 3.2 one liner:
from math import erf, sqrt
def normal_cdf(x, mu=0, sigma=1): return (1/2) * (1 +
erf((x-mu)/(sigma*sqrt(2
approximates the cumulative distribution function of the normal distribution.
Geremy Condra
--
http://mail.python.org/mailman/listinfo/python-list
Re: Learn Python the Hardway exercise 11 question 4
On Thu, Mar 31, 2011 at 2:43 PM, eryksun () wrote: > On Thursday, March 31, 2011 4:35:42 PM UTC-4, Chris Angelico wrote: >> >> I was trolling, I know the reasons behind it. Anyway, most people >> don't share code by email! (Actually, since you seem to be the author >> of that page - could you address that particular point? I think it's >> probably as big an issue as any of the others, to today's coders - >> "code semantics get destroyed by forums/email/etc/etc/etc".) >> >> Solution: All emailed code should begin with >> from __future__ import braces >> And there you are, out of your difficulty at once! > > You could paste it as a base64 stream, such as: > > >> ZGVmIHNwYW0oKToNCiAgICBwcmludCAiU3BhbSEg >> TG92ZWx5IHNwYW0hIExvdmVseSBzcGFtISI= > > > Then decode and exec: > > In [1]: import base64 > > In [2]: %cpaste > Pasting code; enter '--' alone on the line to stop. > :code="""> ZGVmIHNwYW0oKToNCiAgICBwcmludCAiU3BhbSEg > :> TG92ZWx5IHNwYW0hIExvdmVseSBzcGFtISI=""" > :-- > > In [3]: print base64.b64decode(code) > def spam(): > print "Spam! Lovely spam! Lovely spam!" > > In [4]: exec(base64.b64decode(code)) > > In [5]: spam() > Spam! Lovely spam! Lovely spam! I know it's tongue-in-cheek, but please, please, please don't do this. Geremy Condra -- http://mail.python.org/mailman/listinfo/python-list
Re: Python CPU
On Fri, Apr 1, 2011 at 10:00 AM, Stefan Behnel wrote: > Nobody, 01.04.2011 18:52: >> >> Java is a statically-typed language which makes a distinction between >> primitive types (bool, int, double, etc) and objects. Python is a >> dynamically-typed language which makes no such distinction. Even something >> as simple as "a + b" can be a primitive addition, a bigint addition, a >> call to a.__add__(b) or a call to b.__radd__(a), depending upon the values >> of a and b (which can differ for different invocations of the same code). >> >> This is one of the main reasons that statically-typed languages exist, and >> are used for most production software. > > I doubt that the reason they are "used for most production software" is a > technical one. I also suspect that there's some confusion between duck typing and typelessness going on here. Geremy Condra -- http://mail.python.org/mailman/listinfo/python-list
Re: Guido rethinking removal of cmp from sort method
On Wed, Mar 30, 2011 at 7:13 PM, Steven D'Aprano wrote: > Or, an alternative approach would be for one of the cmp-supporters to > take the code for Python's sort routine, and implement your own sort-with- > cmp (in C, of course, a pure Python solution will likely be unusable) and > offer it as a download. For anyone who knows how to do C extensions, this > shouldn't be hard: just grab the code in Python 2.7 and make it a stand- > alone function that can be imported. > > If you get lots of community interest in this, that is a good sign that > the solution is useful and practical, and then you can push to have it > included in the standard library or even as a built-in. > > And if not, well, at least you will be able to continue using cmp in your > own code. I don't have a horse in this race, but I do wonder how much of Python could actually survive this test. My first (uneducated) guess is "not very much"- we would almost certainly lose large pieces of the string API and other builtins, and I have no doubt at all that a really significant chunk of the standard library would vanish as well. In fact, looking at the data I took from PyPI a while back, it's pretty clear that Python's feature set would look very different overall if we applied this test to everything. Geremy Condra -- http://mail.python.org/mailman/listinfo/python-list
Re: Guido rethinking removal of cmp from sort method
On Fri, Apr 1, 2011 at 5:41 PM, Steven D'Aprano wrote: > On Fri, 01 Apr 2011 14:31:09 -0700, geremy condra wrote: > >> On Wed, Mar 30, 2011 at 7:13 PM, Steven D'Aprano >> wrote: >> >> >> >>> Or, an alternative approach would be for one of the cmp-supporters to >>> take the code for Python's sort routine, and implement your own >>> sort-with- cmp (in C, of course, a pure Python solution will likely be >>> unusable) and offer it as a download. For anyone who knows how to do C >>> extensions, this shouldn't be hard: just grab the code in Python 2.7 >>> and make it a stand- alone function that can be imported. >>> >>> If you get lots of community interest in this, that is a good sign that >>> the solution is useful and practical, and then you can push to have it >>> included in the standard library or even as a built-in. >>> >>> And if not, well, at least you will be able to continue using cmp in >>> your own code. >> >> I don't have a horse in this race, but I do wonder how much of Python >> could actually survive this test. My first (uneducated) guess is "not >> very much"- we would almost certainly lose large pieces of the string >> API and other builtins, and I have no doubt at all that a really >> significant chunk of the standard library would vanish as well. In fact, >> looking at the data I took from PyPI a while back, it's pretty clear >> that Python's feature set would look very different overall if we >> applied this test to everything. > > > I don't understand what you mean by "this test". I mean testing whether a feature should be in Python based on whether it can meet some undefined standard of popularity if implemented as a third-party module or extension. > I'm certainly not suggesting that we strip every built-in of all methods > and make everything a third-party C extension. That would be insane. Granted, but I think the implication is clear: that only those features which could be successful if implemented and distributed by a third party should be in Python. My argument is that there are many features currently in Python that I doubt would pass that test, but which should probably be in anyway. The conclusion I draw from that is that this isn't a particularly good way to determine whether something should be in standard Python. > Nor do I mean that every feature in the standard library should be forced > to prove itself or be removed. The features removed from Python 3 were > deliberately few and conservative, and it was a one-off change (at least > until Python 4000 in the indefinite future). If something is in Python 3 > *now*, you can assume that it won't be removed any time soon. I may have been unclear, so let me reiterate: I'm not under the impression that you're advocating this as a course of action. I'm just pointing out that the standard for inclusion you're advocating is probably not a particularly good one, especially in this case, and engaging in a bit of a thought experiment about what would happen if other parts of Python were similarly scrutinized. > What I'm saying is this: cmp is already removed from sorting, and we > can't change the past. Regardless of whether this was a mistake or not, > the fact is that it is gone, and therefore re-adding it is a new feature > request. Those who want cmp functionality in Python 3 have three broad > choices: I might quibble over whether re-adding is the same as a new feature request, but as I said- I don't care about cmp. > (1) suck it up and give up the fight; the battle is lost, move on; > > (2) keep arguing until they either wear down the Python developers or get > kill-filed; never give up, never surrender; > > (3) port the feature that they want into a third-party module, so that > they can actually use it in code, and then when they have evidence that > the community needs and/or wants this feature, then try to have it re- > added to the language. > > I'm suggesting that #3 is a more practical, useful approach than writing > another hundred thousand words complaining about what a terrible mistake > it was. Having to do: > > from sorting import csort > > as a prerequisite for using a comparison function is not an onerous > requirement for developers. If fans of functional programming can live > with "from functools import reduce", fans of cmp can live with that. And that's fine, as I said I don't have a horse in this race. My point is just that I don't think the standard you're using is a good one- ISTM that if it *had* been applied evenly we would have wound up with a much less complete (and much less awesome) Python than we have today. That indicates that there are a reasonable number of real-world cases where it hasn't and shouldn't apply. Geremy Condra -- http://mail.python.org/mailman/listinfo/python-list
Re: Guido rethinking removal of cmp from sort method
On Sat, Apr 2, 2011 at 10:30 PM, harrismh777 wrote: > Brian Quinlan wrote: >> >> I suspect that this debate is a sink hole that I won't be able to escape >> from alive but... > > ... live long and prosper my friend. > > Something to consider is that OOP philosophy is technically one of the most > aesthetic concepts in all of computer science--- with pure functional > programming (haskel, erlang) as a close second... I like how you inserted the word 'technically' in there to give this totally unsubstantiated assertion apparent weight. Geremy Condra -- http://mail.python.org/mailman/listinfo/python-list
Re: Guido rethinking removal of cmp from sort method
On Sat, Apr 2, 2011 at 4:01 AM, Steven D'Aprano wrote: > On Fri, 01 Apr 2011 18:22:01 -0700, geremy condra wrote: > [...] >>>> I don't have a horse in this race, but I do wonder how much of Python >>>> could actually survive this test. My first (uneducated) guess is "not >>>> very much"- we would almost certainly lose large pieces of the string >>>> API and other builtins, and I have no doubt at all that a really >>>> significant chunk of the standard library would vanish as well. In >>>> fact, looking at the data I took from PyPI a while back, it's pretty >>>> clear that Python's feature set would look very different overall if >>>> we applied this test to everything. >>> >>> >>> I don't understand what you mean by "this test". >> >> I mean testing whether a feature should be in Python based on whether it >> can meet some undefined standard of popularity if implemented as a >> third-party module or extension. > [...] >> Granted, but I think the implication is clear: that only those features >> which could be successful if implemented and distributed by a third >> party should be in Python. > > Ah, gotcha. > > I think you're reading too much into what I said -- I wasn't implying > that community support is the only acceptable reason for the existence of > features in Python. > > Development of Python is not a democracy, it is a meritocracy. It is > designed by a small team of language developers, starting with Guido van > Rossum. Those who do the work decide what goes in, based on whatever > combination of factors they choose: I think we're talking at cross purposes. The point I'm making is that there are lots of issues where popularity as a third party module isn't really a viable test for whether a feature is sufficiently awesome to be in core python. As part of determining whether I thought it was appropriate in this case I essentially just asked myself whether any of the really good and necessary parts of Python would fail to be readmitted under similar circumstances, and I think the answer is that very few would come back in. To me, that indicates that this isn't the right way to address this issue, although I admit that I lack any solid proof to base that conclusion on. Geremy Condra -- http://mail.python.org/mailman/listinfo/python-list
Re: Guido rethinking removal of cmp from sort method
On Sun, Apr 3, 2011 at 3:21 AM, Steven D'Aprano wrote: > On Sun, 03 Apr 2011 16:34:34 +1000, Brian Quinlan wrote: > >> On 3 Apr 2011, at 16:22, geremy condra wrote: >>> I think we're talking at cross purposes. The point I'm making is that >>> there are lots of issues where popularity as a third party module isn't >>> really a viable test for whether a feature is sufficiently awesome to >>> be in core python. As part of determining whether I thought it was >>> appropriate in this case I essentially just asked myself whether any of >>> the really good and necessary parts of Python would fail to be >>> readmitted under similar circumstances, and I think the answer is that >>> very few would come back in. To me, that indicates that this isn't the >>> right way to address this issue, although I admit that I lack any solid >>> proof to base that conclusion on. >> >> This has been discussed a few times on python-dev. I think that most >> developers acknowledge that small-but-high-utility modules would not >> survive outside of the core because people would simple recreate them >> rather than investing the time to find, learn and use them. > > That's certainly true for pure Python code, but for a C extension, the > barrier to Do It Yourself will be much higher for most Python coders. I don't think people will work around it in C. I think they'll grudgingly accept a slow and kludgy python workaround, and more to the point I think they would do that with a vast majority of features at this scale. That's why I say this isn't a good test here- because you could apply it to a great feature or a terrible feature and with overwhelming probability have them fail in both cases. > On the other hand, for a pure Python function or class, you could stick > it on ActiveState's Python cookbook and get some imperfect measure of > popularity and/or usefulness from the comments and votes there. Frankly, I have little trust in this as a measure of popularity. Even PyPI isn't a great indicator, and the numbers you get off of ActiveState are almost certain to be way, way noisier. Geremy Condra -- http://mail.python.org/mailman/listinfo/python-list
Re: integer multiplication
On Sun, Apr 3, 2011 at 6:20 PM, Eddie Tsay wrote: > Does anyone know what algorithms for integer multiplication does Python use? > I am trying to compare it to those used by Sage as it seems like it takes > much longer for Python to do large integer multiplication as compared to > Sage (anyone know off the top of their heads?) > thank you (sorry for the double email, this was supposed to go to the list) Karatsuba multiplication, at least for large integers. Geremy Condra -- http://mail.python.org/mailman/listinfo/python-list
Re: integer multiplication
On Sun, Apr 3, 2011 at 6:41 PM, Eddie Tsay wrote: > > On Sun, Apr 3, 2011 at 6:33 PM, geremy condra wrote: >> >> On Sun, Apr 3, 2011 at 6:20 PM, Eddie Tsay wrote: >> > Does anyone know what algorithms for integer multiplication does Python >> > use? >> > I am trying to compare it to those used by Sage as it seems like it >> > takes >> > much longer for Python to do large integer multiplication as compared to >> > Sage (anyone know off the top of their heads?) >> > thank you >> >> (sorry for the double email, this was supposed to go to the list) >> >> Karatsuba multiplication, at least for large integers. >> >> Geremy Condra > > > Thank you for replying to my question. > Do you have any idea how I would be able to find how they do this > multiplication in the python source code? I've been poking around the > documentation and can't find the area where they outline it. > Thank you for your reply =) > Eddie In the Python3 source tarball its in Objects/longobject.c. I only have the 3.2a1 tarball on this machine, but I found the 'grade school' method implemented in x_mul on line 2765 of that file and karatsuba multiplication as k_mul on line 2890. YMMV depending on what version you're looking at. Just as a side-note, top-posting is considered a minor breach of netiquette on python-list. I've rearranged the reply correctly here. Geremy Condra -- http://mail.python.org/mailman/listinfo/python-list
Re: Python CPU
On Mon, Apr 4, 2011 at 12:47 AM, Gregory Ewing wrote: > John Nagle wrote: > >> A tagged machine might make Python faster. You could have >> unboxed ints and floats, yet still allow values of other types, >> with the hardware tagging helping with dispatch. But it probably >> wouldn't help all that much. It didn't in the LISP machines. > > What might help more is having bytecodes that operate on > arrays of unboxed types -- numpy acceleration in hardware. I'd be interested in seeing the performance impact of this, although I wonder if it'd be feasible. Geremy Condra -- http://mail.python.org/mailman/listinfo/python-list
Re: integer multiplication
On Mon, Apr 4, 2011 at 9:41 AM, Terry Reedy wrote: > On 4/4/2011 1:51 AM, Paul Rubin wrote: > >> I didn't realize Python used Karatsuba. The main issue is probably that >> Python uses a straightforward portable C implementation that's not >> terribly efficient, > > but relatively easy for a couple of people to maintain. For (C)Python 3, > which no longer has a C int type, I believe changes were focused on making > calculations with small integers almost as fast as in 2.x. > > (I believe that retaining two implementations internally was considered but > rejected. Could be wrong.) There are two implementations, grade school multiplication and karatsuba, which kicks in after a given cutoff. >>If you look for the gmpy module, it gives you a way to use gmp from >>Python. In crypto code (lots of 1024 bit modular exponentials) I think >>I found gmpy to be around 4x faster than Python longs. > > For specialized use, specialized gmpy is the way to go. > > I am curious how gmpy compares to 3.x ints (longs) with small number > calculations like 3+5 or 3*5. I have this data somewhere, if you're interested I'll try to dredge it up. Geremy Condra -- http://mail.python.org/mailman/listinfo/python-list
Re: Testing for performance regressions
On Mon, Apr 4, 2011 at 7:45 PM, Steven D'Aprano wrote: > I'm writing some tests to check for performance regressions (i.e. you > change a function, and it becomes much slower) and I was hoping for some > guidelines or hints. > > This is what I have come up with so far: > > > * The disclaimers about timing code snippets that can be found in the > timeit module apply. If possible, use timeit rather than roll-you-own > timers. Huh. In looking into timing attacks actually one of the biggest lessons I learned was *not* to use timeit- that the overhead and variance involved in using it will wind up consuming small changes in behavior in ways that are fairly opaque until you really take it apart. > * Put performance tests in a separate test suite, because they're > logically independent of regression tests and functional tests, and > therefore you might not want to run them all the time. > > * Never compare the speed of a function to some fixed amount of time, > since that will depend on the hardware you are running on, but compare it > relative to some other function's running time. E.g.: > > # Don't do this: > time_taken = Timer(my_func).timeit() # or similar > assert time_taken <= 10 > # This is bad, since the test is hardware dependent, and a change > # in environment may cause this to fail even if the function > # hasn't changed. > > # Instead do this: > time_taken = Timer(my_func).timeit() > baseline = Timer(simple_func).timeit() > assert time_taken <= 2*baseline > # my_func shouldn't be more than twice as expensive as simple_func > # no matter how fast or slow they are in absolute terms. > > > Any other lessons or hints I should know? If you can get on it, emulab is great for doing network performance and correctness testing, and even if you can't it might be worth running a small one at your company. I wish I'd found out about it years ago. Geremy Condra -- http://mail.python.org/mailman/listinfo/python-list
Re: Testing for performance regressions
On Mon, Apr 4, 2011 at 10:25 PM, Steven D'Aprano
wrote:
> On Mon, 04 Apr 2011 20:59:52 -0700, geremy condra wrote:
>
>> On Mon, Apr 4, 2011 at 7:45 PM, Steven D'Aprano
>> wrote:
>
>>> * The disclaimers about timing code snippets that can be found in the
>>> timeit module apply. If possible, use timeit rather than roll-you-own
>>> timers.
>>
>> Huh. In looking into timing attacks actually one of the biggest lessons
>> I learned was *not* to use timeit- that the overhead and variance
>> involved in using it will wind up consuming small changes in behavior in
>> ways that are fairly opaque until you really take it apart.
>
> Do you have more details?
>
> I follow the advice in the timeit module, and only ever look at the
> minimum value, and never try to calculate a mean or variance. Given the
> number of outside influences ("What do you mean starting up a browser
> with 200 tabs at the same time will affect the timing?"), I wouldn't
> trust a mean or variance to be meaningful.
I think it's safe to treat timeit as an opaque, medium-precision
benchmark with those caveats. If you need actual timing data though-
answering the question 'how much faster?' rather than 'which is
faster?' just taking actual timings seems to provide much, much better
answers. Part of that is because timeit adds the cost of the for loop
to every run- here's the actual code:
def inner(_it, _timer):
%(setup)s
_t0 = _timer()
for _i in _it:
%(stmt)s
_t1 = _timer()
return _t1 - _t0
(taken from Lib/timeit.py line 81)
where %(setup)s and %(stmt)s are what you passed in. Obviously, if the
magnitude of the change you're looking for is smaller than the
variance in the for loop's overhead this makes things a lot harder
than they need to be, and the whole proposition gets pretty dodgy for
measuring in the sub-millisecond range, which is where many timing
attacks are going to lay. It also has some problems at the opposite
end of the spectrum- timing large, long-running, or memory-intensive
chunks of code can be deceptive because timeit runs with the GC
disabled. This bit me a while back working on Graphine, actually, and
it confused the hell out of me at the time.
I'm also not so sure about the 'take the minimum' advice. There's a
reasonable amount of empirical evidence suggesting that timings taken
at the 30-40% mark are less noisy than those taken at either end of
the spectrum, especially if there's a network performance component.
YMMV, of course.
Geremy Condra
--
http://mail.python.org/mailman/listinfo/python-list
Re: How to program in Python to run system commands in 1000s of servers
On Tue, Apr 5, 2011 at 7:51 AM, Babu wrote: > > Here is my problem: Want to program in python to run sysadmin > commands across 1000s of servers and gather the result in one place. > Many times the commands need to be run as root. We cannot use ssh as > root remote connectivity as well. What are the different ways of > programming in python to achieve this? There are a bajillion ways to do it badly, but SSH sounds like the right tool for the job here. You really don't want your remote admin system compromised, and fabric makes this kind of thing really much less painful. Geremy Condra -- http://mail.python.org/mailman/listinfo/python-list
Re: Suggest some exercises on Python
On Tue, Apr 5, 2011 at 6:13 AM, Jins Thomas wrote: > Hi all, > > I'm a new bie. I have just started learning Python (3.0), finished with > official tutorial. I would like to have your opinion on some > > 1. Good books (for an intermediate in programming) with lot's of exercise i > can try it out. For example when i started with Perl i was going through a > book 'Learning Perl' and it contained lots of exercises which made me > confident. > > 2. Or any link or site which suggests some exercises to do in Python. When i > was looking for some similar threads in some sites, they propose some sites > which gives some exercises and puzzles to solve in python. But i'm feeling > it's too much complicated for my capability. If you're mathematically inclined, try Project Euler. It isn't python-specific, but it is a lot of fun. http://projecteuler.net/ Geremy Condra -- http://mail.python.org/mailman/listinfo/python-list
Re: who moved reload?
On Wed, Apr 6, 2011 at 10:57 PM, harrismh777 wrote: > harrismh777 wrote: >> >> You guys want one more...? >> >> ... we can't import tkconstants any longer nope. >> >> >> import tkinter.tkconstants > > oops... so upset I finger-checked... long day... > > This works: > > import tkinter > from tkinter.constants import * > > > This used to work: > > import Tkinter > from Tkconstants import * > > ... not any more. Yep, things have moved. Glad you're finding your way around now! Geremy Condra -- http://mail.python.org/mailman/listinfo/python-list
Re: fighting game made with python
On Thu, Apr 7, 2011 at 10:41 AM, Littlefield, Tyler wrote: >>Python would b ea bad choice for most of any fighting game, but >>could see use as a configuration or internal scripting engine. > Python's objects are rather large, which sort of makes for some slow work. > Maybe a configuration setup, but Lua and Angelscript are better suited to > high-end games where scripting is required because of their smaller > footprint. EVE online manages to do it. . Geremy Condra -- http://mail.python.org/mailman/listinfo/python-list
Re: Python 3.2 vs Java 1.6
On Fri, Apr 8, 2011 at 8:21 AM, km wrote: > Hi All, > > How does python 3.2 fare compared to Java 1.6 in terms of performance ? > any pointers or observations ? Python and Java have overall very different performance profiles, but for the vast majority of applications either will suffice. If you have reason to believe that your application is very atypical in terms of performance requirements, please post that so we can give you more specific advice. Geremy Condra -- http://mail.python.org/mailman/listinfo/python-list
Re: [OT] Free software versus software idea patents
On Sun, Apr 10, 2011 at 7:49 PM, harrismh777 wrote: > Chris Angelico wrote: >>> >>> > All software can be expressed as lambda calculus. The point being, >>> > all >>> > software is mathematics... > >> With enough software, you can simulate anything. That means that the >> entire universe can be expressed as lambda calculus. Does that mean >> that nothing can ever be patented, because it's all just mathematics? > > Great question... the simple answer is, no. But the extended answer is a > little complicated and not well understood by most folks, so its worth > talking about, at least a lot. You may skip to the last paragraph for the > main point... or stay tuned for the explanation. > Mathematical processes and algorithms are not patentable (by rule) > because they are 'natural' and 'obvious'. In other words, a natural set of > laws (mathematics, just one example) are universally used naturally and > obviously by all humans in the course of thinking, creating, expressing, > &etc., and therefore these ideas are not patentable because they are the > natural and obvious 'stuff' from which and through which the human mind > processes the natural world. You cannot patent the Pythagorean theorem. You > cannot patent addition, nor subtraction, nor the logical concepts for > boolean algebra nor can you patent lambda calculus. These are just > examples. > You cannot patent the mathematical concept of nand gate; however, > Motorola may patent the mechanical electrical implementation of the nand > gate (CMOS 4011 quad nand). Also, Texas Instruments may patent their > mechanical electrical implementation of the nand gate concept (TTL sn7400n > quad chip). The chips are patentable, but the mathematical concept 'behind' > the chips is not patentable. > Software is another sort of animal entirely. Because software is not just > based on mathematics--- IT IS mathematics. I am extremely skeptical of this argument. Leaving aside the fact that you've randomly decided to drop the "decidable" qualifier here- a big problem in its own right- it isn't clear to me that software and computation are synonymous. Lambda calculus only models computation, and software has real properties in implementation that are strictly dependent on the physical world. Since perfectly predicting those properties would seem to require that you perfectly model significant portions of the physical universe, I think it's quite reasonable to contend that the existence of lambda calculus no more rules out the applicability of patents to software (which I detest) than it rules out the applicability of patents to hardware (which I find only slightly less ridiculous) or other meatspace inventions. Geremy Condra -- http://mail.python.org/mailman/listinfo/python-list
Re: [OT] Free software versus software idea patents
On Mon, Apr 11, 2011 at 2:10 AM, Steven D'Aprano wrote: > On Mon, 11 Apr 2011 00:53:57 -0700, geremy condra wrote: >> I am extremely skeptical of this argument. Leaving aside the fact that >> you've randomly decided to drop the "decidable" qualifier here- a big >> problem in its own right- it isn't clear to me that software and >> computation are synonymous. Lambda calculus only models computation, and >> software has real properties in implementation that are strictly >> dependent on the physical world. Since perfectly predicting those >> properties would seem to require that you perfectly model significant >> portions of the physical universe, I think it's quite reasonable to >> contend that the existence of lambda calculus no more rules out the >> applicability of patents to software (which I detest) than it rules out >> the applicability of patents to hardware (which I find only slightly >> less ridiculous) or other meatspace inventions. > > I agree with all of this: I too detest software patents, and find > hardware patents problematic but pragmatic. But if there's a reason for > accepting one and rejecting the other, it's far more subtle than the hand- > waving about mathematics. I believe that the reason falls more to > *pragmatic* reasons than *philosophical* reasons: software patents act to > discourage innovation, while hardware patents (arguably) act to encourage > it. After all, encouraging innovation is what patents are for. > > M Harris' argument fails right at the beginning: > > "Mathematical processes and algorithms are not patentable (by rule) > because they are 'natural' and 'obvious'." > > It's not clear to me how the Banach-Tarski paradox can be described as > 'natural': > > Using the axiom of choice on non-countable sets, you can prove > that a solid sphere can be dissected into a finite number of > pieces that can be reassembled to two solid spheres, each of > same volume of the original. No more than nine pieces are needed. > ... This is usually illustrated by observing that a pea can be > cut up into finitely pieces and reassembled into the Earth. > > http://www.cs.uwaterloo.ca/~alopez-o/math-faq/mathtext/node36.html > > > And I think anyone who knows the slightest bit of mathematics would be > falling over laughing at the suggestion that it is 'obvious'. I'd quibble with you over terminology here. BTP arises naturally- ie, without being explicitly constructed- in certain axiomatic systems. But I get your point. > Of course, some mathematics is obvious, or at least intuitive (although > proving it rigorously can be remarkably difficult -- after 4000 years of > maths, we still don't have an absolutely bullet-proof proof that 1+1=2). Erm. This is getting a bit far afield, but yes, we do. The statement you provide above part of Presbuger arithmetic, which is both complete and decidable. > But describing mathematics as 'obvious' discounts the role of invention, > human imagination, ingenuity and creativity in mathematics. There's > nothing obvious about (say) asymmetric encryption, or solving NP-complete > problems like the knapsack problem, to mention just two examples out of > literally countless examples.[1] Meh. Obvious is in the eye of the beholder, and I doubt we'll wind up coming up with a satisfying and rigorous definition here. I'd therefore rest on the concept of 'natural' I outlined earlier, which would clearly forbid patenting the product of discovery but allow patenting inventions. > If it were just a matter of joining the dots, there would be no unsolved > problems, since Euler would have solved them all 200 years ago.[2] > > Part of the patent problem is that the distinction between discovery of a > fact (which should not be patentable) and invention (which, at least > sometimes, should be patentable) is not clear. The iPod existed as a > Platonic ideal in some mathematical bazillion-dimensional abstract design > space long before it was invented by Apple; does that make it a discovery > rather than an invention? On the other hand, it is doing Apple a great > disservice to ignore their creativity in finding that design point, out > of the infinite number of almost-iPods that suck[3] or don't work. I agree. Of course, your post existed as a billion-point platonic ideal beforehand, so you can't really claim credit (man, Plato figured *everything* out!), but still. Geremy Condra -- http://mail.python.org/mailman/listinfo/python-list
Re: [OT] Free software versus software idea patents
On Mon, Apr 11, 2011 at 3:28 PM, Steven D'Aprano wrote: > On Mon, 11 Apr 2011 11:17:09 -0700, geremy condra wrote: > >> On Mon, Apr 11, 2011 at 2:10 AM, Steven D'Aprano >> wrote: > [...] >>> Of course, some mathematics is obvious, or at least intuitive (although >>> proving it rigorously can be remarkably difficult -- after 4000 years >>> of maths, we still don't have an absolutely bullet-proof proof that >>> 1+1=2). >> >> Erm. This is getting a bit far afield, but yes, we do. The statement you >> provide above part of Presbuger arithmetic, which is both complete and >> decidable. > > Ah, I didn't know that! How wonderful! But in any case, Presburger > arithmetic is much weaker than even Peano arithmetic. > > http://en.wikipedia.org/wiki/Presburger_arithmetic > > So, let me re-phrase my statement... in any realistically complex > arithmetic that is consistent with operations performed for real-world > applications (e.g. multiplication, division, exponentiation, ...), one > cannot demonstrate a bullet-proof proof of 1+1=2. Better? :) Well, Peano arithmetic is normal, everyday arithmetic fully axiomatized, and Presburger arithmetic is a subset of it, so we can utilize the fact that 1 + 1 = 2 is provable in Presburger arithmetic (damn is my spell checker getting a workout on this sentence) to prove it in Peano arithmetic, and therefore in everyday use. You'd also be surprised what you can do with some limited arithmetic forms. During my undergrad I spent some time writing a Presburger prover that was actually fairly handy- there are definitely instances where trading multiplication by non-constant factors for provability superpowers is a good deal. > Presburger arithmetic, Peano arithmetic, the Axiom of Choice... we're > getting further and further away from "natural" mathematics, e.g. > counting sheep in a field. Yes, the sheep would much rather hear about patent reform than the axiom of choice ;) > Anyway, this is now getting off-topic even for the original off-topic > post. Time to move on, methinks. Yeah, looks like a good time to let this one trail off. Geremy Condra -- http://mail.python.org/mailman/listinfo/python-list
Re: [OT] Free software versus software idea patents
On Tue, Apr 12, 2011 at 1:15 AM, harrismh777 wrote: > geremy condra wrote: >>> >>> Software is another sort of animal entirely. Because software is not >>> just >>> > based on mathematics--- IT IS mathematics. > >> I am extremely skeptical of this argument. > > ... as are a great number of other people; corporations, lawyers, > venture capitalists, SPAM SPAM SPAM... That's irrelevant to me. I'm much more concerned about your abuse of mathematics in pursuit of politics than I am about your politics, which (in broad strokes) I agree with. > That is what made the last Supreme Court decision (from this argument in > part) so important... because for the first time the U.S. Supreme Court is > beginning to buy it ... in part. > > See Groklaw here if you're not familiar with the issue: > > http://www.groklaw.net/staticpages/index.php?page=2009022607324398 > > In this case, the court left software patents intact for the moment, but > they slammed the door shut on Bilski... and the interesting thing here is > that what Bilski was claiming to have invented is similar is most respects > to all other "software idea patents," and for similar mathematical > reasoning. It was a slam dunk, by the way... here is a block quote from > Groklaw you might find interesting: > > = block quote = > If you are a lawyer wondering why some argue that software is mathematics, > and hence ineligible for patent protection, or are just interested to know > why software developers, particularly those who develop Free and Open Source > software, almost to a man oppose software patents, you might enjoy reading > Groklaw's An Explanation of Computation Theory for Lawyers, as well as the > amicus briefs and articles marked with the discreet red stars [*], below. > Donald Knuth, called the "father" of the analysis of algorithms, stated: > "Basically I remain convinced that the patent policy most fair and most > suitable for the world will regard mathematical ideas (such as algorithms) > to be not subject to proprietary patent rights." Also, there is a 30-minute > movie, Patent Absurdity: How software patents broke the system, which > explains it well. > =/block quote= I'm familiar with the case, and agree with Knuth (and you) that math should not be patentable. I'd also agree that algorithms are mathematics. Critically, algorithms *are not* software. >> it isn't clear to me that software and >> computation are synonymous. Lambda calculus only models computation, >> and software has real properties in implementation that are strictly >> dependent on the physical world > > (see above) In your rush to misunderstand this you haven't addressed it yet. >> I think it's quite reasonable to >> contend that the existence of lambda calculus no more rules out the >> applicability of patents to software (which I detest) than it rules >> out the applicability of patents to hardware (which I find only >> slightly less ridiculous) or other meatspace inventions. > > The difference is that while hardware may be "described" by mathematics > (all of it is, actually) software IS the description. Software does not, in general, describe hardware. > You know this to be > true, because you prove it everyday... yourself. How many times have you had > a problem, and rather than sit down with your pad of paper and a slide rule > (or your TI-89 Platinum) YOU solved the problem right there on the screen... > er, I mean... the machine solved the problem for you, uh, after you wrote > out the symbols ... ??? We don't think out our problems on paper much > these days... we just enter them into the 'ol terminal and play with it > there... frankly, that is what appeals to me with languages like Python, > Haskell, and Erlang... they are my research engines ... they take the place > of paper and pencil and slide rule and calculator (and counting on ones > fingers) ... but its still little 'ol me doing the thinking... the days of > doing multiple differential equations on a black-board (or white board) are > long gone for most of us. This does not even provide evidence towards your claim that software describes hardware, let alone prove it. It does reinforce my growing suspicion that you do not work in mathematics at a professional or research level, though. > The handwriting is on the wall (not so much wishful thinking as just > noting that everyone including Supreme Court justices are beginning to get > it). It is equally important that *all* software engineers and FOSS > developers *get it!* We need everyone on board with this... really... the > time is critical for everyone to understand this... what is actually at > stake is freedom... what we need to focus on is ending software patents > forever... now. Why is it that the trolls on this board always say this? I'm genuinely baffled by that. Geremy Condra -- http://mail.python.org/mailman/listinfo/python-list
Re: [even more OT than before] Arithmetic [was Free software versus software idea patents]
On Tue, Apr 12, 2011 at 4:34 AM, Steven D'Aprano wrote: > On Mon, 11 Apr 2011 15:55:37 -0700, geremy condra wrote: > > >>> Ah, I didn't know that! How wonderful! But in any case, Presburger >>> arithmetic is much weaker than even Peano arithmetic. >>> >>> http://en.wikipedia.org/wiki/Presburger_arithmetic >>> >>> So, let me re-phrase my statement... in any realistically complex >>> arithmetic that is consistent with operations performed for real-world >>> applications (e.g. multiplication, division, exponentiation, ...), one >>> cannot demonstrate a bullet-proof proof of 1+1=2. Better? :) >> >> Well, Peano arithmetic is normal, everyday arithmetic fully axiomatized, >> and Presburger arithmetic is a subset of it, so we can utilize the fact >> that 1 + 1 = 2 is provable in Presburger arithmetic (damn is my spell >> checker getting a workout on this sentence) to prove it in Peano >> arithmetic, and therefore in everyday use. > > Alas, that's not the case. Peano arithmetic is undecidable: > > http://mathworld.wolfram.com/PeanoArithmetic.html Presburger arithmetic is a subset of Peano Arithmetic, ie, all statements which are true in Presburger Arithmetic are true in Peano arithmetic, and all statements which are false in Presburger arithmetic are false in Peano arithmetic. A bullet-proof proof of the fact you listed exists in Presburger arithmetic. By implication the statement is therefore proven true in Peano arithmetic. Also, undecidable does not mean that *no* statement can be proven formally true or false, only that statements which cannot be proven true or false exist within the confines of those axioms, ie, an axiomatic system is undecidable if there exists an expressible statement within it that is independent of all of the axioms. As you note below, there are other proofs under ZF (and optionally C). > Oh, and this may be of interest: > http://scienceblogs.com/goodmath/2006/06/extreme_math_1_1_2.php It's fine until you get to the end, then it drops into a completely incorrect interpretation both of what R&W were doing and of Godel's work. By the way, if you're interested in this sort of thing, Paul Bernays' (the 'B' in NBG set theory, the other two being no less than von Neumann and Godel) book on axiomatic set theory is cheaply available through Dover and is really, really good at getting to the meat on this sort of thing. Geremy Condra -- http://mail.python.org/mailman/listinfo/python-list
Re: Free software versus software idea patents
On Tue, Apr 12, 2011 at 10:48 AM, CM wrote: >> I don't even know one person who has Win7 installed, running, and likes >>it... >> not even one. > > Hi, m harris, nice to meet you. Now you do. > > To the online community: Is there a name for trolling for A by > advocating for not-A in a way that discredits your point of view an > case so that A now seems much more reasonable? It's not really > "concern trolling". What would this be called? Harrisment. /I'm sorry, this is abuse... Geremy Condra -- http://mail.python.org/mailman/listinfo/python-list
Re: [OT] Free software versus software idea patents
On Tue, Apr 12, 2011 at 4:56 PM, Steven D'Aprano wrote: > That would be a patent on a business process, which is allowed. In fact, > as I recall, at least one lawyer has made an attempt to patent a business > process relating to law. IBM tried patenting the business of patent trolling, which I found hilarious. Geremy Condra -- http://mail.python.org/mailman/listinfo/python-list
Re: [OT] Free software versus software idea patents
On Wed, Apr 13, 2011 at 10:35 PM, harrismh777 wrote: > I am sorry, I was not clear and you rightly misunderstood my indirection. > I am not claiming that software describes hardware. Please allow me to > restate. > Mathematics describes hardware, yet hardware is patentable and > mathematics is not. The hardware is patentable, but the mathematics used in > the hardware production is not. ( I think we are in agreement so far). > Again, patent the chip, but not the symbol. I think you're still ascribing slightly too much power to mathematics here, but I don't think its relevant to the main point you're making. > All software (and yes, I mean all of it) is nothing more nor less than > mathematics... relationship over symbol--- but not limited to algorithms, > and certainly not limited to computation. Restatement is not a form of proof. If you're right, prove it to me- I've stated my initial objections. > Software is not *just* described > by mathematics (as hardware is) software is itself mathematics--- the > description. (not the description of hardware, the living and breathing > description of relationship over symbol). Software is mathematics, and it > is therefore not patentable. Please don't confuse hardware with this... I > never meant to go there--- my fault. The mathematics of software is no > more physical than the mathematics of (chalk on a blackboard) is physical. > The chalk dust and board are not the issue; neither are the circuits and > switches. Relationship over symbol is the issue; and algorithms and > computation are the very least of it, if at all. Think "thought" and > "process". This is just repetition. Besides being a marginally effective attempt to bedazzle with eloquence, it again contributes nothing to your argument. > When I am speaking of mathematics I am NOT even speaking about > computation, nor am I speaking about *just* algorithms; although, > mathematical 'process' is certainly at the heart of the discussion. > Richard Feynman has said, "Nature talks to us in the language of > mathematics," and it behooves all educated people to grapple with the > understanding of this powerful aphorism. When authors of software are > composing software, they are "doing" mathematics. The craft is ever present. > Some, like myself more theoretical and philosophical, others more practical > and applied. Yet both kinds of person are doing mathematics (generating > mathematics) with every manipulated symbol. The beautiful symbols of > software development are the very symbols of harmonious mathematical > relationship from every pure functional construct (or even OOP Class) right > down to each and every 1 & 0 (on and off). This artistry is pure and applied > science (the stuff of every human thought in logic and reasoning).Jan > Gullberg has said, "Mathematics grows and develops in many ways unrelated to > science, and thus plays a crucial role in the history of human thought". > This is the art and science of software engineering of which I am speaking. > Software is the very stuff of human thought and expression. In so many ways > software reflects the very nature of what it means to be human; how we > reflect, organize, structure, and relate cognitively over symbol. I love > mathematics, as I love software, and I have abused neither. Again, repetition. You've found the fire and wonder but lost the precision in the process, and that *is* an abuse of mathematics. Give it precision and you'll do just fine. > If you agree with my politics (with broad strokes) why would you not like > to try to understand what my politics are based in? Are you apposed to > software patents out of frustration and emotion only? Or, is there another > in-born reason why you detest them? I suspect that the mathematician in you > is screaming to be let out... It's been let out. I'm a security researcher and a cryptanalyst, and I have colleagues who do the same work I do who identify themselves as mathematicians. My opposition to software patents is based in the firmly held belief that they're a way for entrenched players to kick the crap out of real innovators. I believe that to be profoundly unfair, and I don't have a great love of bullies. Having said that, I have a greater respect for mathematics than I do for my own economic views, and I don't like seeing it become a political football. If you can prove something, *prove it*. If you cannot- no matter how close you might feel you are- don't claim that math says you're right. In this case you've let yourself get carried away with some very deep, very complex math that you don't fully understand and used it to justify a view that almost is almost certainly held for other reasons. You've reduced some of the most elegant and powerful works of the human mind to simple confirmation bias, and I think most mathematicians would find that shameful. Geremy Condra -- http://mail.python.org/mailman/listinfo/python-list
Re: Free software versus software idea patents
On Wed, Apr 13, 2011 at 10:50 PM, harrismh777 wrote: > Westley Martínez wrote: >>>>> >>>>> I don't even know one person who has Win7 installed, running, and >>>>> likes it... >>>>> > >> not even one. >>>> >>>> > > >>>> > > Hi, m harris, nice to meet you. Now you do. >>>> > > >>>> > > To the online community: Is there a name for trolling for A by >>>> > > advocating for not-A in a way that discredits your point of view >>>> > an >>>> > > case so that A now seems much more reasonable? It's not really >>>> > > "concern trolling". What would this be called? >>> >>> > >>> > Harrisment. >>> > >>> > /I'm sorry, this is abuse... >>> > >>> > Geremy Condra >> >> That was harristerical. >> > > > Actually, it is funny... and I love humor as much as I love mathematics, > software, and freedom. Truly harristerical... :) > > But on a serious note, I did wonder who would be the first jouster to offer > the argumentum ad hominem? ... ah, sticks and stones... > > My major professor once told me, "You know you've won the argument when they > start calling you names!" I think your professor should have said "you know you've won the argument when you can prove it". Geremy Condra -- http://mail.python.org/mailman/listinfo/python-list
Re: [OT] Free software versus software idea patents
On Thu, Apr 14, 2011 at 12:22 AM, harrismh777 wrote: > geremy condra wrote: >> >> Having said that, I have a greater respect for mathematics than I do >> for my own economic views, and I don't like seeing it become a >> political football. If you can prove something,*prove it*. If you >> cannot- no matter how close you might feel you are- don't claim that >> math says you're right. > > Fair enough. > > http://www.groklaw.net/articlebasic.php?story=2009151305785 > > > With your background, you will have no problem with this text... > although, and I do apologize, it is a long text... scroll down and begin > reading at the heading: > > An Explanation of Computation Theory for Lawyers > > By PolR > > > > The write-up is excellent, written as well or better than I could write > it myself, and is very clear and concise... yes, even a lawyer can > understand it. This is not a proof. This is an argument. There's a very big difference. To be clear, this article makes basically the same mistake you do- you assume that a program is exactly equivalent to its computation, while the article makes the additional and even more wrong assumption that a program is perfectly defined by its CPU instructions. It's to your credit that you avoided advancing this obviously incorrect claim, but you still haven't addressed my earlier points. Geremy Condra -- http://mail.python.org/mailman/listinfo/python-list
Re: [OT] Free software versus software idea patents
On Fri, Apr 15, 2011 at 10:21 PM, harrismh777 wrote: This looks to me like an application of the troll motto "if you can't dazzle them with brilliance, baffle them with bull". It certainly does nothing to prove your claim, despite clearly attempting to word-salad your way through an argument. Geremy Condra -- http://mail.python.org/mailman/listinfo/python-list
Re: [OT] Free software versus software idea patents
On Sat, Apr 16, 2011 at 5:31 PM, Chris Angelico wrote: > On Sun, Apr 17, 2011 at 9:36 AM, Steven D'Aprano > wrote: >> (If the machine is particularly >> simple -- you might be able to exactly simulate a lever in pure >> mathematics, but simulating, say, a nuclear bomb or a dialysis machine in >> mathematics is more of a challenge...) > > I can easily model a massless, frictionless, inelastic, infinitismally > thin lever. Beyond that, it gets complicated. With what kind of precision? Geremy Condra -- http://mail.python.org/mailman/listinfo/python-list
Re: [OT] Free software versus software idea patents
On Sun, Apr 17, 2011 at 11:29 PM, harrismh777 wrote: Ok, so, you're basically saying that perfect simulation is not a requirement for something to 'be mathematics'. I don't think you can construct a nontrivial model for mathematics without including that, but I'd be happy to be proven wrong, and if you'll provide your criteria for the question of what qualifies as 'being mathematics' we might get somewhere. Geremy Condra -- http://mail.python.org/mailman/listinfo/python-list
Re: Teaching Python
On Tue, Apr 19, 2011 at 3:42 PM, Passiday wrote: > Hello, > > I am planning to teach Python to a group of high school students, who have > in-depth interest in programming, hacking etc. > > I am looking for some good material, what I could use as a basic guide when > preparing the classes plan for the course - website or book, what would roll > out the topic methodologically gradually. The target audience is someone who > knows most basics of the programming, but doesn't mind being reminded about > them now and then. > > Thanks for any suggestions! When you say 'hacking', you mean ? Geremy Condra -- http://mail.python.org/mailman/listinfo/python-list
Re: Teaching Python
On Tue, Apr 19, 2011 at 4:24 PM, Dan Stromberg wrote: > On Tue, Apr 19, 2011 at 4:03 PM, geremy condra wrote: >> On Tue, Apr 19, 2011 at 3:42 PM, Passiday wrote: >>> Hello, >>> >>> I am planning to teach Python to a group of high school students, who have >>> in-depth interest in programming, hacking etc. >>> >>> I am looking for some good material, what I could use as a basic guide when >>> preparing the classes plan for the course - website or book, what would >>> roll out the topic methodologically gradually. The target audience is >>> someone who knows most basics of the programming, but doesn't mind being >>> reminded about them now and then. >>> >>> Thanks for any suggestions! >> >> When you say 'hacking', you mean ? > > Presumably he meant the real meaning of the word, not what the press > made up and ran with. Even security professionals use it both ways. Especially in the context of a room full of teenagers, it seems reasonable to ask. Geremy Condra -- http://mail.python.org/mailman/listinfo/python-list
Re: [OT] Piling of prepositions (was: Namespaces in functions vs classes)
On Tue, Apr 19, 2011 at 7:49 PM, Dave Angel wrote: > On 01/-10/-28163 02:59 PM, Ben Finney wrote: >> >> "Rhodri James" writes: >> >>> Language abuse: it's not just Python. A donation of just $5 will keep >>> a programmer in prepositions for a month. $50 will supply enough >>> articles to keep a small company understandable for over a year. With >>> your generous help, we can beat this scourge! >> >> I lately lost a preposition >> It hid, I thought, beneath my chair >> And angrily I cried, “Perdition! >> Up from out of in under there.” >> >> Correctness is my vade mecum, >> And straggling phrases I abhor, >> And yet I wondered, “What should he come >> Up from out of in under for?” >> >> —Morris Bishop, in the _New Yorker_, 1947-09-27 >> >> http://www.ourcivilisation.com/smartboard/shop/gowerse/complete/chap905.htm> >> > > Ending a sentence with a preposition is one aberration up with which I will > not put. > > With is a lousy word to start or end a sentence with. > > origins unknown With respect to the first, it is (possibly mis)attributed to Winston Churchill. The second I've heard but don't recall where from. Geremy Condra -- http://mail.python.org/mailman/listinfo/python-list
Re: client-server parallellised number crunching
On Tue, Apr 26, 2011 at 12:55 PM, Hans Georg Schaathun wrote: > I wonder if anyone has any experience with this ... > > I try to set up a simple client-server system to do some number > crunching, using a simple ad hoc protocol over TCP/IP. I use > two Queue objects on the server side to manage the input and the output > of the client process. A basic system running seemingly fine on a single > quad-core box was surprisingly simple to set up, and it seems to give > me a reasonable speed-up of a factor of around 3-3.5 using four client > processes in addition to the master process. (If anyone wants more > details, please ask.) > > Now, I would like to use remote hosts as well, more precisely, student > lab boxen which are rather unreliable. By experience I'd expect to > lose roughly 4-5 jobs in 100 CPU hours on average. Thus I need some > way of detecting lost connections and requeue unfinished tasks, > avoiding any serious delays in this detection. What is the best way to > do this in python? > > It is, of course, possible for the master thread upon processing the > results, to requeue the tasks for any missing results, but it seems > to me to be a cleaner solution if I could detect disconnects and > requeue the tasks from the networking threads. Is that possible > using python sockets? > > Somebody will probably ask why I am not using one of the multiprocessing > libraries. I have tried at least two, and got trapped by the overhead > of passing complex pickled objects across. Doing it myself has at least > helped me clarify what can be parallelised effectively. Now, > understanding the parallelisable subproblems better, I could try again, > if I can trust that these libraries can robustly handle lost clients. > That I don't know if I can. Without knowledge of what you're doing it's hard to comment intelligently, but I'd try something like CHAOS or OpenSSI to see if you can't get what you need for free, if that doesn't do it then try dropping a liveCD with Hadoop on it in each machine and running it that way. If that can't work, try MPI. If you've gotten that far and nothing does the trick then you're probably going to have to give more details. On a side note, there are a reasonable number of examples of how to do similar things on a GPU in "GPU Gems"- depending on your needs you might be able to just apply a copypasta solver ;) Geremy Condra -- http://mail.python.org/mailman/listinfo/python-list
Re: client-server parallellised number crunching
On Tue, Apr 26, 2011 at 10:58 PM, Hans Georg Schaathun wrote: > On Tue, 26 Apr 2011 14:31:59 -0700, geremy condra > wrote: > : Without knowledge of what you're doing it's hard to comment > : intelligently, > > I need to calculate map( foobar, L ) where foobar() is a pure function > with no dependency on the global state, L is a list of tuples, each > containing two numpy arrays, currently 500-1000 floats each + a scalar > or two. The result is a pair of floats. > > The foobar() function is sufficiently heavy to merit demonstratably > parallellisation. This sounds like a hadoop job, with the caveat that you still have to get your objects across the network somehow. Have you tried xdrlib or the struct module? I suspect either would save you some time. > The CPU-s I have available to spread the load further are not clustered. > They are prone to crash without warning and I do not have root access. > I don't have exclusive use. I do not even have physical access, so I > cannot use a liveCD. (They are, however, equipped with a batch queue > system (torque).) Hmm. I guess I'd boil it down to this: if you have the ability to install software on them, give hadoop a try. If, OTOH you can't disturb normal lab operation at all and need a lot of CPU power, you should probably start weighing what your time is worth against the cost of firing up a few EC2 instances and being done with it- I use it for cryptanalytic work with a similar structure all the time, and you can get a hell of a cluster going over there for about $15-20 an hour. If you're an academic (it sounds like you are) you may also be able to use things like planetlab and emulab, which are free and reasonably easy to use. > : but I'd try something like CHAOS or OpenSSI to see if > : you can't get what you need for free, if that doesn't do it then try > : dropping a liveCD with Hadoop on it in each machine and running it > : that way. If that can't work, try MPI. If you've gotten that far and > : nothing does the trick then you're probably going to have to give > more > : details. > > TANSTAFL :-) > There is always the learning curve > > If I understand it correctly, openSSI requires root access; is that > right? For CHAOS I need more details to be able to google; I found > a fractals toolbox, but that did not seem relevant :-) OpenSSI and CHAOS are both Single System Image clustering solutions- they're pretty cool, but you pretty much need to be able to run a live CD to make it worth your time. > MPI I have tried before. Unless there is a new, massively more > sophisticated MPI library around now, I would certainly have to > do my own code to cope with lost clients. Sandia labs has some neat work in this area, but if hadoop fits your computational model it will be much easier on you in terms of implementation. > Hadoop sounds intresting. I had encountered it before, but did not > think about it. However, the liveCD is clearly not an option. Thanks > for the tip; I'll read up on map-reduce at least. Np, hope it solves things for you ;) Geremy Condra -- http://mail.python.org/mailman/listinfo/python-list
Re: Deditor
On Wed, Apr 27, 2011 at 1:47 PM, Yico Gaga wrote: > why not linux? The download is a .deb, ie, for Debian and Ubuntu. Geremy Condra -- http://mail.python.org/mailman/listinfo/python-list
Re: Coolest Python recipe of all time
On Tue, May 3, 2011 at 8:49 AM, Terry Reedy wrote: > On 5/3/2011 2:29 AM, Gregory Ewing wrote: >> >> Terry Reedy wrote: >>> >>> The trick is that replacing x with j and evaluating therefore causes >>> (in Python) all the coefficients of x (now j) to be added together >>> separately from all the constant terms to reduce the linear equation >>> to a*x+b (= 0 implied). >> >> Hmmm... so if we used quaternions, could we solve systems >> of linear equations in 3 variables? > > Yes and no. The use of 1*j merely collected and added together all the > multipliers of 'x' (and all the constant terms). That is a fairly trivial > matter of constant folding. Systems of linear equations are usually > presented in that form already. The actual solution to the simple equation > is in the formula x = -a/b (where a and b are the sums). The solution > formula for three variables would be far more complex. Or just use a gauss-jordan solver, which has the advantage of being easy to explain and possible to verify by hand on small instances. Geremy Condra -- http://mail.python.org/mailman/listinfo/python-list
Re: [OT]: PiCloud - really cool!
On Wed, May 4, 2011 at 5:13 PM, James Mills wrote: > If anyone hasn't seen this yet, I encourage you to! > (I stumbled upon it with some random thoughts and Gooogling) > > http://www.picloud.com/ > > Here's a quick test I wrote up that works locally using the simulator > and live (I did run it live): > > #!/usr/bin/env python > > import cloud > > def fib(n): > a, b, = 1, 1 > while n > 1: > a, b = b, a + b > n -= 1 > return b > > #cloud.start_simulator() > > jobs = cloud.map(fib, range(100)) > > print [cloud.result(job) for job in jobs] > > Enjoy! :) I was the poster across from them at PyCon two years back. Pretty fun to play with, although last I checked it was hard to do true HPC on it. Geremy Condra -- http://mail.python.org/mailman/listinfo/python-list
Re: [OT]: PiCloud - really cool!
On Wed, May 4, 2011 at 5:29 PM, James Mills wrote: > On Thu, May 5, 2011 at 10:19 AM, geremy condra wrote: >> I was the poster across from them at PyCon two years back. Pretty fun >> to play with, although last I checked it was hard to do true HPC on >> it. > > Why's that ? And what is true HPC (High Performance Computing) anyway ? > I find the API provided to be quite simple robust and potentially very > powerful - depending on your application. Last time I checked- and again, it's been a while- you were basically just able to run some (probably computationally bound) function on an EC2 instance that they managed for you without having any of the muss and fuss of doing it yourself. That's very cool, but there are problems where you really need more horsepower than a normal EC2 instance can provide, and if your application crosses that boundary after you've written it against PiCloud you're probably in the same hole you would have been without PiCloud. The other limitation is with problems that take a lot of input data, where it's more time intensive to ship the data across to EC2 than it is to just process it locally. Once more for effect: I haven't played with this in a while, the guys who built it seem pretty sharp, and it wouldn't surprise me at all to find out that they have ways of dealing with this better now. Geremy Condra -- http://mail.python.org/mailman/listinfo/python-list
Re: Coolest Python recipe of all time
On Fri, May 6, 2011 at 9:59 AM, Steven D'Aprano wrote: > On Mon, 02 May 2011 10:33:31 -0700, Raymond Hettinger wrote: > >> I think it is time to give some visibility to some of the instructive >> and very cool recipes in ActiveState's python cookbook. > [...] >> What are your favorites? > > > I'm not sure if favourite is the right word, but I'm amazed by this one: > McCarthy's "amb" (ambiguous) operator. > > http://code.activestate.com/recipes/577368 > > Here's how I might use it to solve the problem if making change. In > Australian currency, I have 5, 10, 20, 50 cent and $1 and $2 coins. > Ignoring the dollar coins, how can I make up change for any multiple of > five cents up to a dollar? > > Suppose I have more 5 cent coins that I can deal with, and I want to make > sure I hand out at least three of them. Here's how to make 45 cents worth > of change: > >>>> amb = Amb() >>>> a = amb(range(3, 21)) # number of 5 cent coins >>>> b = amb(range(11)) # number of 10 cent coins >>>> c = amb(range(6)) # number of 20 cent coins >>>> d = amb(range(3)) # number of 50 cent coins >>>> for _ in amb(lambda a,b,c,d: 5*a+10*b+20*c+50*d == 45): > ... print a, b, c, d > ... > 3 1 1 0 > 3 3 0 0 > 5 0 1 0 > 5 2 0 0 > 7 1 0 0 > 9 0 0 0 > > > Suppose I have some words, and want to put them together so that there > are a certain number of vowels: > >>>> amb = Amb() >>>> a = amb(['quick', 'slow', 'hungry', 'wise-old']) >>>> b = amb(['fox', 'hare', 'turtle', 'kangaroo']) >>>> c = amb(['lazy', 'stupid', 'sleepy', 'confused']) >>>> d = amb(['dog', 'aardvark', 'sloth', 'wombat']) >>>> >>>> def test(a, b, c, d): > ... s = "The %s brown %s jumped over the %s %s." % (a, b, c, d) > ... num_vowels = sum(s.count(c) for c in 'aeiou') > ... return num_vowels in (12, 18, 19) > ... >>>> for _ in amb(test): > ... print a, b, c, d > ... > quick fox lazy sloth > quick fox lazy dog > quick kangaroo stupid aardvark > [...more solutions cut for brevity] > hungry kangaroo confused aardvark > > > > As written, amb is just a brute-force solver using more magic than is > good for any code, but it's fun to play with. I use a similar technique *a lot* for various kinds of constraint solvers, but I have yet to come up with a really satisfying spelling for it. Have you looked at the way this is done in Sage? Geremy Condra -- http://mail.python.org/mailman/listinfo/python-list
Re: obviscating python code for distribution
On Sun, May 15, 2011 at 10:41 PM, Littlefield, Tyler wrote: > Hello: > Thanks all for your information and ideas. I like the idea of open source; I > have a fairly large (or large, by my standards anyway) project that I am > working on that is open source. > > Here's kind of what I want to prevent. I want to write a multi-player online > game; everyone will essentually end up connecting to my server to play the > game. I don't really like the idea of security through obscurity, but I > wanted to prevent a couple of problems. > 1) First I want to prevent people from hacking at the code, then using my > server as a test for their new setups. I do not want someone to gain some > extra advantage just by editing the code. > Is there some other solution to this, short of closed-source? > Thanks, I don't know that closing the source does you much more good than obfuscating it. The obvious attack surface here is pretty much totally exposed via network traffic, which any legitimate client can gain access to. A better approach would be to simply write more secure code in the first place. Geremy Condra -- http://mail.python.org/mailman/listinfo/python-list
Re: Faster Recursive Fibonacci Numbers
On Tue, May 17, 2011 at 9:36 AM, rusi wrote: > On May 17, 8:50 pm, RJB wrote: >> I noticed some discussion of recursion. the trick is to find a >> formula where the arguments are divided, not decremented. >> I've had a "divide-and-conquer" recursion for the Fibonacci numbers >> for a couple of years in C++ but just for fun rewrote it >> in Python. It was easy. Enjoy. And tell me how I can improve it! >> >> def fibo(n): >> """A Faster recursive Fibonaci function >> Use a formula from Knuth Vol 1 page 80, section 1.2.8: >> If F[n] is the n'th Fibonaci number then >> F[n+m] = F[m]*F[n+1] + F[m-1]*F[n]. >> First set m = n+1 >> F[ 2*n+1 ] = F[n+1]**2 + F[n]*2. >> >> Then put m = n in Knuth's formula, >> F[ 2*n ] = F[n]*F[n+1] + F[n-1]* F[n], >> and replace F[n+1] by F[n]+F[n-1], >> F[ 2*n ] = F[n]*(F[n] + 2*F[n-1]). >> """ >> if n<=0: >> return 0 >> elif n<=2: >> return 1 >> elif n%2==0: >> half=n//2 >> f1=fibo(half) >> f2=fibo(half-1) >> return f1*(f1+2*f2) >> else: >> nearhalf=(n-1)//2 >> f1=fibo(nearhalf+1) >> f2=fibo(nearhalf) >> return f1*f1 + f2*f2 >> >> RJB the Lurkerhttp://www.csci.csusb.edu/dick/cs320/lab/10.html > > - > Its an interesting problem and you are 75% there. > You see the halving gives you logarithmic behavior and the double > calls give exponential behavior. > > So how to get rid of double calls? Its quite simple: Just define your > function in terms of return pairs of adjacent pairs ie (fib(n), fib(n > +1)) for some n rather then a single number fib(n) > > Here's a straightforward linear function: > > def fp(n): #fibpair > if n==1: > return (1,1) > else: > a,b = fp(n-1) > return (b, a+b) > > def fib(n): > a,b = fp(n) > return a > > --- > Now use this (pairing) idea with your (halving) identities and you > should get a logarithmic algo. > > [If you cant do it ask again but yes its fun to work out so do > try :-) ] > -- > http://mail.python.org/mailman/listinfo/python-list > or O(1): φ = (1 + sqrt(5)) / 2 def fib(n): numerator = (φ**n) - (1 - φ)**n denominator = sqrt(5) return round(numerator/denominator) Testing indicates that it's faster somewhere around 7 or so. Geremy Condra -- http://mail.python.org/mailman/listinfo/python-list
Re: Faster Recursive Fibonacci Numbers
On Tue, May 17, 2011 at 10:19 AM, Jussi Piitulainen wrote: > geremy condra writes: > >> or O(1): >> >> φ = (1 + sqrt(5)) / 2 >> def fib(n): >> numerator = (φ**n) - (1 - φ)**n >> denominator = sqrt(5) >> return round(numerator/denominator) >> >> Testing indicates that it's faster somewhere around 7 or so. > > And increasingly inaccurate from 71 on. Yup. That's floating point for you. For larger values you could just add a linear search at the bottom using the 5f**2 +/- 4 rule, which would still be quite fast out to about 10 times that. The decimal module gets you a tiny bit further, and after that it's time to just use Dijkstra's, like rusi suggested. In any event, I still think this formulation is the most fun ;). Geremy Condra -- http://mail.python.org/mailman/listinfo/python-list
Re: Faster Recursive Fibonacci Numbers
On Tue, May 17, 2011 at 2:04 PM, Wolfram Hinderer wrote: > On 17 Mai, 20:56, geremy condra wrote: >> On Tue, May 17, 2011 at 10:19 AM, Jussi Piitulainen >> >> wrote: >> > geremy condra writes: >> >> >> or O(1): >> >> >> ö = (1 + sqrt(5)) / 2 >> >> def fib(n): >> >> numerator = (ö**n) - (1 - ö)**n >> >> denominator = sqrt(5) >> >> return round(numerator/denominator) >> >> >> Testing indicates that it's faster somewhere around 7 or so. >> >> > And increasingly inaccurate from 71 on. >> >> Yup. That's floating point for you. For larger values you could just >> add a linear search at the bottom using the 5f**2 +/- 4 rule, which >> would still be quite fast out to about 10 times that. The decimal >> module gets you a tiny bit further, and after that it's time to just >> use Dijkstra's, like rusi suggested. In any event, I still think this >> formulation is the most fun ;). > > I think you can write it even more funny > > def fib(n): > return round(((.5 + .5 * 5 ** .5) ** n - (.5 - .5 * 5 ** .5) ** > n) * 5 ** -.5) > > ;-) Ok, that's amusing. It does hide the interaction with the golden ratio though, which is what I find so fascinating about the earlier one. Geremy Condra -- http://mail.python.org/mailman/listinfo/python-list
Re: obviscating python code for distribution
On Wed, May 18, 2011 at 12:36 AM, Hans Georg Schaathun wrote: > On Mon, 16 May 2011 23:42:40 +0100, Rhodri James > wrote: > : ...which is, of course, not exactly secure either. A sufficiently > : determined hacker won't have much trouble disassembling a shared library > : even if you do strip out all the debug information. By chance I'm having > : to do something closely related to this at work just at the moment; it's > : hard, but far from impossible. > > But then, nothing is secure in any absolute sense. If you're talking security and not philosophy, there is such a thing as a secure system. As a developer you should aim for it. > The best you can > do with all your security efforts is to manage risk. Since obfuscation > increases the cost of mounting an attack, it also reduces risk, > and thereby provides some level of security. The on-the-ground reality is that it doesn't. Lack of access to the source code has not kept windows or adobe acrobat or flash player secure, and they have large full-time security teams, and as you might imagine from the amount of malware floating around targeting those systems there are a lot of people who have these skills in spades. > Obviously, if your threat sources are dedicated hackers or maybe MI5, > there is no point bothering with obfuscation, but if your threat source > is script kiddies, then it might be quite effective. On the theory that any attack model without an adversary is automatically secure? Geremy Condra -- http://mail.python.org/mailman/listinfo/python-list
Re: obviscating python code for distribution
On Wed, May 18, 2011 at 10:24 AM, Chris Angelico wrote:
> On Thu, May 19, 2011 at 2:54 AM, geremy condra wrote:
>> On Wed, May 18, 2011 at 12:36 AM, Hans Georg Schaathun
>> wrote:
>>> But then, nothing is secure in any absolute sense.
>>
>> If you're talking security and not philosophy, there is such a thing
>> as a secure system. As a developer you should aim for it.
>
> Agreed. Things can be secure if you accept caveats. A good server
> might be secure as long as attackers cannot, say:
> * Get physical access to the server, remove the hard disk, and tamper with it
> * Hold a gun to the developer and say "Log me in as root or you die"
> * Trigger a burst of cosmic rays that toggle some bits in memory
Just a note: you can do many cool things to prevent the last from
working, assuming you're talking about RSA fault injection attacks.
> If someone can do that, there's really not much you can do to stop
> them. But you CAN make a system 100% secure against network-based
> attacks.
>
> Denial of service attacks are the hardest to truly defend against, and
> if your level of business is low enough, you can probably ignore them
> in your code, and deal with them by human ("Hmm, we seem to be getting
> ridiculous amounts of traffic from XX.YY.ZZ.*, I think I'll put a
> temporary ban on that /24"). Although some really nasty DOSes can be
> blocked fairly easily, so it's worth thinking about them.
>
> But mainly: Don't panic about the really really obscure attack
> possibilities, the ones that would only happen if someone with a lot
> of resources is trying to bring you down. Just deal with the obvious
> stuff - make sure your server cannot be compromised via a standard
> network connection.
Just one caveat I would add to this: make sure you're drawing this
line at the correct place. If your attack model is wrong things have a
tendency to drop from 'impossible' to 'laughably easy' in a hurry.
> Test your server by connecting with a basic TELNET client (or a
> hacked-up client, if it uses a binary protocol). Test your client by
> connecting it to a hacked-up server. Make sure you can't muck up
> either of them. Assume that any attacker will know every detail about
> your comms protocol, because chances are he will know most of it.
I actually like to use scapy a lot. It's a little slow, but you can
really get down deep and still feel sort of sane afterwards, and it
makes it easier on you if you don't need to go all the way to the
metal.
Geremy Condra
--
http://mail.python.org/mailman/listinfo/python-list
Re: obviscating python code for distribution
On Wed, May 18, 2011 at 10:33 AM, Hans Georg Schaathun wrote: > On Wed, 18 May 2011 09:54:30 -0700, geremy condra > wrote: > : On Wed, May 18, 2011 at 12:36 AM, Hans Georg Schaathun > wrote: > : > But then, nothing is secure in any absolute sense. > : > : If you're talking security and not philosophy, there is such a thing > : as a secure system. As a developer you should aim for it. > > You think so? Please name one, and let us know how you know that it > is secure. I was playing around with an HSM the other day that had originally targeted FIPS 140-3 level 5, complete with formal verification models and active side-channel countermeasures. I'm quite confident that it was secure in nearly any practical sense. > : > and thereby provides some level of security. > : > : The on-the-ground reality is that it doesn't. Lack of access to the > : source code has not kept windows or adobe acrobat or flash player > : secure, and they have large full-time security teams, and as you might > : imagine from the amount of malware floating around targeting those > : systems there are a lot of people who have these skills in spades. > > You are just demonstrating that it does not provide complete security, > something which I never argued against. Ah, my mistake- when you said 'some level of security' I read that as 'some meaningful level of security'. If you were arguing that it provided roughly as much protection to your code as the curtain of air surrounding you does to your body, then yes- you're correct. > : > Obviously, if your threat sources are dedicated hackers or maybe MI5, > : > there is no point bothering with obfuscation, but if your threat source > : > is script kiddies, then it might be quite effective. > : > : On the theory that any attack model without an adversary is > : automatically secure? > > No, on the assumption that we were discussing real systems, real > threats, and practical solutions, rather than models and theory. > There will always be adversaries, but they have limited means, and > limited interest in your system. And the limits vary. Any marginal > control will stave off a few potential attackers who just could not > be bothered. Empirically this doesn't appear to be a successful gambit, and from an attacker's point of view it's pretty easy to see why. When a system I'm trying to break turns out to have done something stupid like this, it really just ticks me off, and I know a lot of actual attackers who think the same way. > In theory, you can of course talk about absolute security. For > instance, one can design something like AES¹, which is secure in > a very limited, theoretical model. However, to be of any practical > use, AES must be built into a system, interacting with other systems, > and the theory and skills to prove that such a system be secure simply > has not been developed. This is flatly incorrect. > Why do you think Common Criteria have not yet specified frameworks > for the top levels of assurance? Perhaps because the lower levels of 'assurance' don't seem to provide very much. Geremy Condra -- http://mail.python.org/mailman/listinfo/python-list
Re: obviscating python code for distribution
On Wed, May 18, 2011 at 12:56 PM, Hans Georg Schaathun wrote: > On Wed, 18 May 2011 12:07:49 -0700, geremy condra > wrote: > : I was playing around with an HSM the other day that had originally > : targeted FIPS 140-3 level 5, complete with formal verification models > : and active side-channel countermeasures. I'm quite confident that it > : was secure in nearly any practical sense. > > And you ostensibly use the word /nearly/ rather than «absolutely». > It seems that we agree. Systems can be designed that are absolutely secure under reasonable assumptions. The fact that it has assumptions does not make your statement true. > BTW, according to the sources I can find quickly, FIPS 140-3 > targets /modules/ and not systems. I can't tell if you're trying to play word games with the distinction between "system" and "module" or if you're just saying that you aren't sure what FIPS actually certifies. Could you please clarify? > : Ah, my mistake- when you said 'some level of security' I read that as > : 'some meaningful level of security'. If you were arguing that it > : provided roughly as much protection to your code as the curtain of air > : surrounding you does to your body, then yes- you're correct. > > Well, I didn't. Whether it is meaningful is relative and dependent > on the context, but it sure isn't meaningful if any values at stake are. Again, I'm unsure what you're going for here. It sounds like you're saying that obfuscation doesn't provide meaningful security, which is my point. > : Empirically this doesn't appear to be a successful gambit, and from an > : attacker's point of view it's pretty easy to see why. When a system > : I'm trying to break turns out to have done something stupid like this, > : it really just ticks me off, and I know a lot of actual attackers who > : think the same way. > > That is very true. It is a very crude measure with a marginal > effect on risk. Going out of one's way to try to obfuscate the > code as machine code, as was the starting point of the discussion, > is surely not a good strategy, as one is then spending significant > time to achieve a rather insignificant. > > My main concern is that the use of absolutes, «you need this», and > «that is silly», is drawing attention from the main point. Rather, > get to know your risks and focus on the greater ones. Consider > possible controls, and choose cheap and effective ones. Even a > marginally effective control may be worth-while if the cost is even > less. We all seem to agree on the main point; many have argued the > same way. > > As an aside, OTOH, don't you think MAYFARE would have been broken > earlier if the source code were open? It was around for ages before > it was. Are you talking about the Mayfair classical cipher here? > : > In theory, you can of course talk about absolute security. For > : > instance, one can design something like AES¹, which is secure in > : > a very limited, theoretical model. However, to be of any practical > : > use, AES must be built into a system, interacting with other systems, > : > and the theory and skills to prove that such a system be secure simply > : > has not been developed. > : > : This is flatly incorrect. > > Which part of it? If you claim that the theory and skills to prove it > exist, could you give a reference please? The entire field of formal modeling and verification has grown around solving this problem. My new favorite in the field is "formal models and techniques for analyzing security protocols", but there are other works discussing OS kernel verification (which has gotten a lot of attention lately) and tons of academic literature. Google (scholar) is the place to go. > Of course, if you are only thinking of «nearly any practical sense» > again, then we agree and always did. Nope, talking about formal methods. > : > Why do you think Common Criteria have not yet specified frameworks > : > for the top levels of assurance? > : > : Perhaps because the lower levels of 'assurance' don't seem to provide very > much. > > If the lower levels do not, would that not be an argument to implement > more levels? Too many governments have put too much resources into > this to just throw it away if the methodology to achieve higher assurance > could be codified. If you can't say with confidence that something meets minimum security standards, the answer is not to try to say it meets high security standards. > Or maybe it is right to say that the theory and skills do exist, but the > money to gather it all in one project to demonstrate the security of > a single system does not :-) Sorry, but again this is not correct. Geremy Condra -- http://mail.python.org/mailman/listinfo/python-list
Re: obviscating python code for distribution
On Wed, May 18, 2011 at 11:26 AM, Littlefield, Tyler wrote: >>might be secure as long as attackers cannot, say: > You forgot UFOs. > Anyway, again, thanks to everyone for the advice, this is good reading. > Incidentally, I don't know to much about security. I know about rate > limiting and dos attacks, as well as some others, but I think there's a lot > more that I don't know--can someone kind of aim me in the right direction > for some of this? I want to be able to take techniques, break my server and > then fix it so that can't be done before I head to public with this. One good thing to do is to just read some of the black hat papers. They're pretty accessible and even if you don't know everything they're saying you should be able to get a general feel for things that way. You might also try working through things like Damn Vulnerable Web App, if you have the time. Geremy Condra -- http://mail.python.org/mailman/listinfo/python-list
Re: obviscating python code for distribution
On Wed, May 18, 2011 at 10:21 PM, Hans Georg Schaathun wrote: > On Wed, 18 May 2011 14:34:46 -0700, geremy condra > wrote: > : Systems can be designed that are absolutely secure under reasonable > : assumptions. The fact that it has assumptions does not make your > : statement true. > : (...) > : I can't tell if you're trying to play word games with the distinction > : between "system" and "module" or if you're just saying that you aren't > : sure what FIPS actually certifies. Could you please clarify? > > The distinction between system and module is rather significant. > If you only consider modules, you have bounded your problem and > drastically limited the complexity. Ah, the 'word games' option. I'm not going to spend a lot of time arguing this one: HSMs are clearly the domain of systems research, are referred to in both technical and nontechnical documents as 'keystone systems', and the FIPS standard under which they are certified specifically calls them systems more times than I care to count. They are, to the people who make and use them, systems, and your attempt at redefinition won't change that. > : Are you talking about the Mayfair classical cipher here? > > I am talking about the system used in public transport cards like > Oyster and Octopus. I am not sure how classical it is, or whether > mayfair/mayfare referred to the system or just a cipher. Any way, > it was broken, and it took years. Ah, MIFARE. That's a different story, and no, I don't believe they would have been broken sooner if the specs were released. The importance (and difficulty) of securing devices like smartcards wasn't really recognized until much later, and certainly people with a foot in both worlds were very rare for a long time. Also remember that DES (with its 56-bit keys) was recertified just a few months before MIFARE (with its 48-bit keys) was first released- it was a different world. > : The entire field of formal modeling and verification has grown around > : solving this problem. My new favorite in the field is "formal models > : and techniques for analyzing security protocols", but there are other > : works discussing OS kernel verification (which has gotten a lot of > : attention lately) and tons of academic literature. Google (scholar) is > : the place to go. > > Sure, but now you are considering modules, rather than systems again. > It is when these reliable components are put together to form systems > that people fail (empirically). Let me get this straight: your argument is that operating *systems* aren't systems? > : If you can't say with confidence that something meets minimum security > : standards, the answer is not to try to say it meets high security > : standards. > > So what? The levels of assurance have nothing to do with standards. > The levels of assurance refer to the /confidence/ you can have that > the standards are met. The increasing levels of assurance don't just signify that you've checked for problems- it certifies that you don't have them, at least insofar as that level of testing is able to find. Insisting that this doesn't, or shouldn't, translate into tighter security doesn't make much sense. Geremy Condra -- http://mail.python.org/mailman/listinfo/python-list
Re: obviscating python code for distribution
On Wed, May 18, 2011 at 7:54 PM, harrismh777 wrote: > Littlefield, Tyler wrote: > Four resources that you will what to look into, in no particular order: > > Erickson, Jon, "Hacking: The Art of Exploitation," 2nd ed, > San Francisco: No Starch Press, 2008. This would be a very good choice. It's a bit light on details, but makes up for it by being exceptionally well-written and very accessible. > Anonymous, "Maximum Linux Security: A Hacker's Guide to Protecting > Your Linux Server and Workstation," Indianapolis: > Sams Publishing, 2000. > > (check for other editions) > (this volume is a good read, even for other platforms, > but is geared specifically to Linux) This is a good volume, but very dated. I'd probably pass on it. > Graves, Kimberly, "CEH Certified Ethical Hacker: Study Guide," > Indianapolis: Wiley Publishing, 2010. Briefly glancing over the TOC, this actually looks surprisingly good. CEH itself is a joke among black hats, but if this gets down to the nitty-gritty of actually performing the attacks it covers it sounds like a buy. > Seitz, Justin, "Gray Hat Python: Python Programming for Hackers > and Reverse Engineers," San Francisco: No Starch Press, 2009. I'd skip this one, as it isn't really focused on what you want. The web application hacker's handbook is probably more along the lines of what you need, if you're going for a book. There's also an older volume called 'counter hack' that gives a good overview of some of the ways that attacks proceed. Another recommend I'm surprised hasn't popped up already: 'security power tools' is a good way to get your foot in the door. It has a practical, no-nonsense approach and is split into self-contained chapters so you don't waste too much of your time on tools that aren't relevant to you. Geremy Condra -- http://mail.python.org/mailman/listinfo/python-list
Re: Faster Recursive Fibonacci Numbers
On Thu, May 19, 2011 at 3:47 PM, Steven D'Aprano wrote: > On Tue, 17 May 2011 10:02:21 -0700, geremy condra wrote: > >> or O(1): >> >> φ = (1 + sqrt(5)) / 2 >> def fib(n): >> numerator = (φ**n) - (1 - φ)**n >> denominator = sqrt(5) >> return round(numerator/denominator) > > I'd just like to point out that, strictly speaking, it's only O(1) if you > assume that exponentiation is O(1). Computer scientists often like to > make this simplifying assumption, and it might even be true for floats, > but for long ints and any numeric data types with unlimited precision, it > won't be. Great point, and something I should have paid attention to. Thanks. Geremy Condra -- http://mail.python.org/mailman/listinfo/python-list
Re: obviscating python code for distribution
On Thu, May 19, 2011 at 11:23 AM, Hans Georg Schaathun wrote: > On Thu, 19 May 2011 10:23:47 -0700, geremy condra > wrote: > : Let me get this straight: your argument is that operating *systems* > : aren't systems? > > You referred to the kernel and not the system. The complexities of > the two are hardly comparable. I don't know about that. Among the many verified microkernels, at least two projects have formally verified both their kernel and their toolchain, and one of them claims they've verified everything in their TCB and are headed towards verified POSIX compliance in 2012. That would seem to be a fairly large system (and definitely a complete OS) to me. Another (seL4) says they've formally verified security of a complete system that includes a userspace and the ability to run other OSes in fully isolated containers, which also seems to be quite complete. Finally, there's one from Microsoft research that claims similar properties but which apparently isn't interested in compatibility, which I'm not sure how to interpret in terms of usefulness and size. In any event, higher level systems- like electronic voting mechanisms and automotive sensor networks- have also been verified, which seems to run counter to your original point. Also, not sure if it's open to the general public but if you're interested in this kind of thing and live near seattle, I think there's actually going to be a talk on verifying a POSIX userspace implementation here tomorrow. TL;DR version: large systems have indeed been verified for their security properties. > There probably are different uses of system; in computer security > literature¹ it often refers, not only to a product (hardware/software) > an actual installation and configuration of that product in a specific > context. /I/ did not redefine it. You chose a word with a many meanings, used it to make a very broad statement which is only a little bit true, and then pretended that you had the One True Definition in your pocket. I don't think that's legitimate, but whatever; let's just say that we meant different things by the word and drop it. > Speaking of reasonable assumptions, one necessary assumption which is > particularly dodgy is that whoever deploys and configures it > understands all the assumptions and do not break them through ignorance. Yup. Nothing is safe from idiots. > Is your concern with security purely from a developer's viewpoint, > so that you don't have to worry about the context in which it will > be deployed? My viewpoint is that of an attacker, since that's more or less my job. > I read your initial comment to imply that if you cannot get satisfactory > assurance using the lower levels, you won't get any at the higher > levels. That does not make any sense. Well, this is kind of like my point. My point was that you really don't get anything at the lower levels, and that they should fix that (which is far more useful to a normal consumer) rather than trying to talk about formal verification and similar tools, which are only going to be used on a tiny fraction of products. Geremy Condra -- http://mail.python.org/mailman/listinfo/python-list
Re: obviscating python code for distribution
On Thu, May 19, 2011 at 6:33 PM, Chris Angelico wrote: > On Fri, May 20, 2011 at 10:56 AM, geremy condra wrote: >>> Speaking of reasonable assumptions, one necessary assumption which is >>> particularly dodgy is that whoever deploys and configures it >>> understands all the assumptions and do not break them through ignorance. >> >> Yup. Nothing is safe from idiots. I actually think I need to take this statement back. The more I think about it, the less convinced I am that it's correct- I can at least conceive of violable systems which cannot be misconfigured. So, sorry about that. > Which means that the assumption really is that you are evaluating a > system, not a bald piece of code. I don't consider that an assumption. > When you're writing code that you will yourself deploy, you take full > responsibility; when you let other people deploy it, they have to take > ultimate responsibility (although they will legitimately expect you to > provide an install script and/or instructions). Sure, although I would personally still call it an assumption. Geremy Condra -- http://mail.python.org/mailman/listinfo/python-list
Re: obviscating python code for distribution
On Fri, May 20, 2011 at 12:10 AM, Steven D'Aprano wrote: > On Thu, 19 May 2011 17:56:12 -0700, geremy condra wrote: > >> TL;DR version: large systems have indeed been verified for their >> security properties. > > How confident are we that the verification software is sufficiently bug- > free that we should trust their results? Pretty confident. Most formal verification systems are developed in terms of a provably correct kernel bootstrapping the larger system. The important thing is that since that kernel doesn't need to be complete (only correct) it can typically be easily verified, and in some cases exhaustively tested. There are also techniques which generate certificates of correctness for verifiers that aren't provably correct, but that isn't an area I know much about, and I don't know if that gets used in practice. The bigger risk is really that the model you're feeding it is wrong. > How confident are we that the verification software tests every possible > vulnerability, as opposed to merely every imaginable one? Formal provers typically don't work by just throwing a bunch of input at a piece of software and then certifying it. They take a set of specifications (the model), a set of assumptions, and the program in question, and provide a proof (in the mathematical sense) that the program is exactly equivalent to the model given the assumptions. Testing the assumptions and model are typically part of the development process, though, and that's definitely a possible source of errors. Geremy Condra -- http://mail.python.org/mailman/listinfo/python-list
Re: Faster Recursive Fibonacci Numbers
On Fri, May 20, 2011 at 10:07 AM, Steven D'Aprano wrote: > On Fri, 20 May 2011 16:54:06 +1000, Chris Angelico wrote: > >> If someone has time to kill (as if!), it'd be awesome to get a new >> numeric type that uses bc's code; any other numeric type (int, long, >> float) could autopromote to it, removing the dilemma of which to promote >> out of long and float. Hmm... Python 4.0, 'bc' is the new default >> integer and everything else is a performance optimization? Heh. > > The problem is, it isn't *just* a performance optimization, there is a > semantic difference too. Consider: > >>>> x = 1e-300 >>>> x*x == 0 > True > > But using something with more precision: > >>>> from fractions import Fraction >>>> x = Fraction(10)**-300 >>>> x*x == 0 > False > > > So you get different behaviour between floats and arbitrary precision > numbers. And this shows up in the above implementation; reimplementing it using Fractions and a truncated continuing fraction approximation of phi and the square root of 5 gets us up to about 500, at the cost of a very long computation time. Geremy Condra -- http://mail.python.org/mailman/listinfo/python-list
Re: obviscating python code for distribution
On Fri, May 20, 2011 at 1:24 PM, harrismh777 wrote: > Steven D'Aprano wrote: >> >> Nonsense. How do you (generic "you", not any specific person) know that >> you are not an idiot? > > lol Sum, ergo Idiot cogitat. > > > Reminds me of a philosophical story I heard one time from my religion > professor... > > ... as it goes, De Carte leads his horse into town ;-) and having hitched > it to the rail outside the local saloon and sauntering up to the bar, the > tender asks, "Would you be hav'in an ale sir?" > > ... De Carte replies, "I think not..." ... and then disappeared. At risk of being pedantic, I think you mean Descartes rather than De Carte. Geremy Condra -- http://mail.python.org/mailman/listinfo/python-list
Re: [OT] Off-Topic Posts and Threads on the Python Mailing List
On Fri, Sep 30, 2011 at 8:25 PM, Navkirat Singh wrote: > Lol you all sound like google's angry birds with their feathers ruffled by a > comment. You guys should open up another mailing list to extinguish your > virtually bruised egos. . . . Google does not produce Angry Birds. There is another mailinglist for OT conversation. Extinguishing does not help either bruises or egos. You smell like a troll. You top-posted. Your post was nonsensical, error-filled, ill-formatted, grouchy, pointless, and dumb. You've earned my contempt, and may God have mercy on your troll. Geremy Condra -- http://mail.python.org/mailman/listinfo/python-list
Re: Opportunity missed by Python ?
On Fri, Oct 14, 2011 at 5:49 PM, alex23 wrote: > On Oct 13, 8:07 pm, Chris Angelico wrote: >> Python, as I found out to my detriment, is practically impossible to >> sandbox effectively. > > The latest version of PyPy introduces a prototype sandbox: > > http://pypy.org/features.html#sandboxing > > It'll be interesting to see how effective this is. Please note that their sandbox, while a good idea, is not a guaranteed jail. It's enforced by replacing calls to external libraries with trampoline stubs, but does not appear to have any intrinsic mechanism to prevent calls from being issued without it. That means that if you were able to successfully inject code you would be no more protected here than with any other process. Geremy Condra > -- > http://mail.python.org/mailman/listinfo/python-list > -- http://mail.python.org/mailman/listinfo/python-list
Re: Python Module for console text formatting?
You can insert those codes just like you would any other character. If there's enough interest I can whip up a wrapper library for you. http://www.linux.gr/cgi-bin/man2html?console_codes+4 On Tue, Feb 17, 2009 at 11:19 PM, srinivasan srinivas < sri_anna...@yahoo.co.in> wrote: > > Hi, > Does anyone know any python module other than 'ConsoleFormat0.1.1' used to > format text on console? > For example, printing bold characters on console. > > Thanks, > Srini > > > Did you know? You can CHAT without downloading messenger. Go to > http://in.webmessenger.yahoo.com/ > -- > http://mail.python.org/mailman/listinfo/python-list > -- OpenMigration LLC- Open Source solutions for your business. Visit us at http://OpenMigration.net. -- http://mail.python.org/mailman/listinfo/python-list
Re: Python AppStore / Marketplace
Just a few quick questions on your proposed design: 1) What tools, if any, do you propose to help developers package for this platform? 2) How do you plan to deal with applications that change or depend on system-wide settings, ie, conf files, kernel version, or the registry? 3) What advantages do you see this having over, say, porting portage or apt-get? Thanks for your time, Geremy Condra -- http://mail.python.org/mailman/listinfo/python-list
Re: Python 3D CAD -- need collaborators, or just brave souls :)
I'm interested. If you are still serious about doing this in two months, send me an email. If you have something put together at that point we can talk about its future. Sound fair? Geremy Condra -- http://mail.python.org/mailman/listinfo/python-list
Re: Run a linux system command as a superuser, using a python script
Run the script as root and have it drop privs when it doesn't need them. Or set up a separate daemon to do the rootish parts of it. -- http://mail.python.org/mailman/listinfo/python-list
Re: Getting screen dims platform specific? Say it ain't so!
For X systems you can use xwininfo, ie:
from commands import getstatusoutput
def get_screen_dims():
status, output = getstatusoutput("xwininfo -root")
if not status:
On Tue, Feb 24, 2009 at 6:53 PM, Luis Zarrabeitia wrote:
> On Tuesday 24 February 2009 05:57:52 pm Lionel wrote:
> > from win32api import GetSystemMetrics
>
> I'd guess that win32api is patform specific, as in "api for win32".
>
> --
> Luis Zarrabeitia (aka Kyrie)
> Fac. de Matemática y Computación, UH.
> http://profesores.matcom.uh.cu/~kyrie
> --
> http://mail.python.org/mailman/listinfo/python-list
>
--
OpenMigration LLC- Open Source solutions for your business. Visit us at
http://OpenMigration.net.
--
http://mail.python.org/mailman/listinfo/python-list
Re: Getting screen dims platform specific? Say it ain't so!
my apologies- hit enter accidentally. anyway, just process the output of that command for the Height: and Width: lines. -- http://mail.python.org/mailman/listinfo/python-list
Re: code challenge: generate minimal expressions using only digits 1,2,3
Use 1 and 2 and treat them as equivalent to boolean true and false. -- http://mail.python.org/mailman/listinfo/python-list
Re: Completely OT
On Mon, Nov 30, 2009 at 1:21 PM, Victor Subervi wrote:
> On Mon, Nov 30, 2009 at 2:00 PM, inhahe wrote:
>>
>> On Mon, Nov 30, 2009 at 12:58 PM, inhahe wrote:
>> > On Mon, Nov 30, 2009 at 12:49 PM, Victor Subervi
>> > wrote:
>> >>
>> >>
>> >> If I'm not mistaken, that won't help me actually print to screen the
>> >> user's
>> >> choices as he selects them, which in my application, is important.
>> >> Please
>> >> advise.
>> >> TIA,
>> >> V
>> >
>> >
>> > sure, that's where this part comes in:
>> >
>> > the javascript would populate the list for the colors the user selects
>> > (the easiest way would probably be to give the list an id and use
>> > getElementByID())
>> >
>> > so basically you'd define, e.g., an onClick="blah('red'); return true"
>> > within the red element's tag, and then define a function blah(x) that
>> > says
>> > getElementById("my_list_id").innerHtml += "" + x;
>> > and of course give your list textarea an id="my_list_id" attribute in
>> > the tag.
>> >
>> > that could be slightly wrong, my javascript's rusty
>> >
>>
>> also don't forget to sanitize the data you receive before committing
>> it to the database, or someone can hack the javascript and send an SQL
>> injection attack
>
> Good call! However, in my case I can put this all behind a firewall. It's
> only for the shop builder's use, and that's my client...whom I can track!
> But I presume this would entail doing searches for and eliminating all
> unnecessary characters, right?
> V
Don't homebrew these things, they're easy to screw up and disastrous
to get wrong. Also, if you're worried about how secure something
you've written is, you can give yourself a little peace of mind by
running over it with some of the standard script kiddie tools before
deployment. It'll at least give you the comfort of knowing that they
won't be able to autopwn you for a while.
Geremy Condra
--
http://mail.python.org/mailman/listinfo/python-list
Re: Bored.
On Mon, Nov 30, 2009 at 5:14 PM, Necronymouse wrote: > Hello, I am learning python for about 2 years and I am bored. Not with > python but I have a little problem, when i want to write something I > realise that somebody had alredy written it! So i don´t want to make a > copy of something but i wanna get better in python skills. Don´t you > know what I should do? > > > ---sorry for my english, I ´m from czech rep... I'd find a big project and contribute to it. There's a lot more to being a developer than writing code, and its hard to learn much about that side of things by working on your own projects all the time. Plus, it can be very satisfying to make a widely used piece of software better. Geremy Condra -- http://mail.python.org/mailman/listinfo/python-list
Re: python bijection
On Thu, Dec 3, 2009 at 7:04 AM, M.-A. Lemburg wrote:
> Raymond Hettinger wrote:
>> [Joshua Bronson]
>>> Raymond, do you think there might be any future in including a built-
>>> in bidict data structure in Python?
>>
>> I don't think so. There are several forces working against it:
>>
>> * the recipe is new, so it hasn't had a chance to mature
>> or to gain a fan club.
>>
>> * there are many approaches to the solving the problem and
>> there is no reason to assume this one is the best.
>>
>> * it extends the language with arcane syntax tricks instead of
>> using the language as designed by Guido. That makes it harder
>> to learn and remember.
>>
>> * we've already got one (actually two). The two dictionary approach
>> uses plain python, requires no new learning, and is more flexible.
>> Also, sqlite3 provides another way to use multiple lookups to a
>> single record. The database approach is much more general
>> (extending to trijections, allowing multiple sort orders,
>> providing persistence, etc).
>>
>> * the semantics of a bijection aren't obvious:
>>
>> b['x'] = 'ex' # first record: ('x', 'ex')
>> b['y'] = 'why' # second record: ('y', 'why')
>> b[:'why'] = 'x' # do two records collapse into one? is there
>> an error?
>>
>> * the proposed syntax doesn't address the issue covered in my previous
>> post.
>> Since bijections are symmetrical, they do not have an obvious
>> direction
>> (which is the primary key, the husband or the wife?). The syntax
>> needs to
>> allow user names to make it clear which is being accessed:
>>
>> marriages.h2w['john'] = 'amy'
>> marriages.w2h['amy'] = 'john'
>>
>> Contrast this with:
>>
>> marriages['jordan'] = 'taylor' # are you sure you got the
>> order correct?
>> marriages[:'taylor'] = 'jordan' # this is easy to get backwards
>
> I think the only major CS data type missing from Python is some
> form of (fast) directed graph implementation à la kjGraph:
>
> http://gadfly.sourceforge.net/kjbuckets.html
>
> With these, you can easily build all sorts of relations between
> objects and apply fast operations on them. In fact, it should then
> be possible to build a complete relational database in Python
> (along the lines of Gadfly).
If you're in the market for a Python graph library, you may want
to check out Graphine- I'm obviously biased (I wrote most of it)
but it has a few more bells and whistles than kjbuckets, and is
pretty darned easy to use. It also supports undirected and
bridge graphs.
Geremy Condra
--
http://mail.python.org/mailman/listinfo/python-list
