RE: Cracking hashes with Python
Hashes, by definition, are not reversible mathematically. The only way to figure out what they represent is to take plaintext that might be the plaintext based on anything you might know about the original plaintext (which is often nothing) and hash it; then see if the hash matches the one you have. If it does, you have figured out the plaintext; if it doesn't try again. For a tool that does this, look at Rainbow tables. -original message-- Hi, I have a school project to do where I've to download MD5 Hashes from a particular website and write a code that will crack them. Does anyone know where I'll find out more information on how to do this? There's only 4 hashes that I need to do so it doesn't have to be a large script just needs to be able to download the hashes from the website. Can anyone help me out? -- https://mail.python.org/mailman/listinfo/python-list
Parsing multiple lines from text file using regex
Hi, I am having an issue with something that would seem to have an easy solution, but which escapes me. I have configuration files that I would like to parse. The data I am having issue with is a multi-line attribute that has the following structure: banner Banner text Banner text Banner text ... The regex 'banner\s+(\w+)\s+(.+)' captures the command nicely and banner.group(2) captures the delimiter nicely. My issue is that I need to capture the lines between the delimiters (both delimiters are the same). I have tried various permutations of Delimiter=banner.group(2) re.findall(Delimiter'(.*?)'Delimiter, line, re.DOTALL|re.MULTILINE) with no luck Examples I have found online all assume that the starting and ending delimiters are different and are defined directly in re.findall(). I would like to use the original regex extracting the banner.group(2), since it is already done, if possible. Any help in pointing me in the right direction would be most appreciated. Thank you, Marc -- https://mail.python.org/mailman/listinfo/python-list
RE: Parsing multiple lines from text file using regex
>What was wrong with the answer Peter Otten gave you earlier today on the >tutor mailing list? > >-- >Python is the second best programming language in the world. >But the best has yet to be invented. Christian Tismer > >Mark Lawrence > I did not receive any answers from the Tutor list, so I thought I'd ask here. If an answer was posted to the Tutor list, it never made it to my inbox. Thanks to all that responded. -- https://mail.python.org/mailman/listinfo/python-list
RE: Parsing multiple lines from text file using regex
>Hi Marc, did you actually subscribe to the tutor list or did you just >send an email there? Peter replied to you and you can see the reply >here: >https://mail.python.org/pipermail/tutor/2013-October/098156.html > >He only sent the reply back to the tutor list and didn't email it >directly to you because it is assumed that you would also be subscribed >to the list. > > >Oscar Thanks Oscar - yes, I am subscribed to both lists - the python-list through this email address and the tutor list through my work email. I think my work spam filter may be the issue - I will subscribe to the tutor list through this email and unsubscribe from my work email. Thanks again, Marc -- https://mail.python.org/mailman/listinfo/python-list
RE: Parsing multiple lines from text file using regex
>Hi Marc, did you actually subscribe to the tutor list or did you just >send an email there? Peter replied to you and you can see the reply >here: >https://mail.python.org/pipermail/tutor/2013-October/098156.html > >He only sent the reply back to the tutor list and didn't email it >directly to you because it is assumed that you would also be subscribed >to the list. > > >Oscar I again Oscar - apparently I am already subscribed to the Tutor list with this email also: 'An attempt was made to subscribe your address to the mailing list tu...@python.org. You are already subscribed to this mailing list.' I control the spam filter for this domain, so I am not sure why I am not getting the updates. Marc -- https://mail.python.org/mailman/listinfo/python-list
RE: Parsing multiple lines from text file using regex
>This is an alternative solution someone else posted on this list for a
> similar problem I had:
>#!/usr/bin/python3
>from itertools import groupby
>def get_lines_from_file(file_name):
>with open(file_name) as reader:
>for line in reader.readlines():
>yield(line.strip())
>counter = 0
>def key_func(x):
>if x.strip().startswith("banner") and x.strip().endswith(" delimiter>"):
>global counter
>counter += 1
>return counter
>for key, group in groupby(get_lines_from_file("my_data"), key_func):
>print(list(group)[1:-1])
Thanks Jason,
banner = re.compile(r'banner\s+(\w+)\s+(.+)(.*?)\2', re.DOTALL).findall(lines)
worked nicely to get what I needed:
outfile.write("Banner type: %s Banner Delimiter: %s\n" % (banner[0][0],
banner[0][1]))
outfile.write("Banner Text:\n")
outfile.write(banner[0][2])
Probably not the prettiest, most concise code, but it gets the job done.
Thanks again,
Marc
--
https://mail.python.org/mailman/listinfo/python-list
Diferent files: GUI (wxpython) Program
Hi why I can call an .py GUI (wxpython) from a program and control it? I'm newbie in python. -- http://mail.python.org/mailman/listinfo/python-list
Re: Diferent files: GUI (wxpython) Program
[EMAIL PROTECTED] escribió: > On Oct 11, 5:44 pm, marc <[EMAIL PROTECTED]> wrote: >> Hi why I can call an .py GUI (wxpython) from a program and control it? >> I'm newbie in python. > > Sorry, could you restate that? Explain again what you want to do. > I would to call a python file that contains a GUI (with wx) from a file program. The programs that I've do, contains a GUI and the code to run it in the same file, but I when need to modify a GUI file (I use wxglade) I need to delete all code lines (except the GUI code lines) and it difficult the program. -- http://mail.python.org/mailman/listinfo/python-list
Re: Diferent files: GUI (wxpython) Program
[EMAIL PROTECTED] escribió: > On Oct 11, 5:44 pm, marc <[EMAIL PROTECTED]> wrote: >> Hi why I can call an .py GUI (wxpython) from a program and control it? >> I'm newbie in python. > > Sorry, could you restate that? Explain again what you want to do. > I would to call a python file that contains a GUI (with wx) from a file program. The programs that I've do, contains a GUI and the code to run it in the same file, but I when need to modify a GUI file (I use wxglade) I need to delete all code lines (except the GUI code lines) and it difficult the program. -- http://mail.python.org/mailman/listinfo/python-list
Re: f---ing typechecking
On 15 feb, 07:21, James Stroud <[EMAIL PROTECTED]> wrote: > I guess we differ on what is obvious. This seems obvious to me: > > [1] + (1,) => [1, 1] > (1,) + [1] => (1, 1) > > simply becuase the operand on the left should take precendence because > its "__add__" is called and its "__add__" returns a list. In essence, as > we know the obviously correct behavior for __add__ and __radd__, then it > would be the obviously correct behavior that the above would follow. Given those obviouses, the following seems to me: [1] + (1,) => [1, (1,)] That's the trouble with obvious -- my obvious may not be so obvious to you (and vice versa). That's why the Zen of Python says "In the face of ambiguity, refuse the temptation to guess." (Although it also says "Flat is better than nested", but I'll ignore that for now.) Basically -- if you want Perl, you know where to find it ;-) Marc -- http://mail.python.org/mailman/listinfo/python-list
Directorio europeo de arte y diseño
MARC3ART - GUÍA EUROPEA DE ARTE Y DISEÑO ofrece de manera ordenada, detallada y actualizada información sobre eventos artísticos-culturales y de diseño de interés para personas relacionadas con el medio. Ofrece de manera permanente calendario de exposiciones, concursos de arte, convocatorias y premios de diseño, así como cualquier otra especialidad artística (dibujo, arquitectura, literatura, animación, cine, ...), cursos, conferencias, ciclos, talleres, sección de becas de arte y diseño, fotografía, agencias, estudios de diseño, publicidad etc. www.marc3art.eu -- http://mail.python.org/mailman/listinfo/python-list
Re: pylab without X11
This may help you ... or not You may have to change your backend : p13 of the matplotlib user guide: backends p17 how to change the backend in the matplotlibrc An example of matplotlibrc ( backend choice is at the beginning ): http://matplotlib.sourceforge.net/matplotlibrc You may choose PS or SVG or pdf, there might be other possibilities, I do not know To save the file : savefig Marc Willem-Jan Vriend a écrit : I want to use pylab (matplotlib) on a machine without X11. I'm trying to generate onthefly graphics for an apache2 web service, so they do not have to be displayed on this machine ! When i do pylab.figure() I get the error TclError: couldn't connect to display ":0.0" I tried setting the DISPLAY environment variable to "", ":0.0" but I got the same error message. Any help is very much appreciated. -- http://mail.python.org/mailman/listinfo/python-list
Re: pylab without X11
This may help ... or not ( 2nd part )
try not to do pylab.figure()
in your script:
pylab.plot, or pylab.imshow or whatever you want to use
then
savefig("myFigure.png")for example
do not use show()
in my case ( with the WXAgg backend ), it works, it generates the png file
Marc
Willem-Jan Vriend a écrit :
I want to use pylab (matplotlib) on a machine without X11. I'm trying to
generate onthefly graphics for an apache2 web service, so they do not
have to be displayed on this machine !
When i do
pylab.figure()
I get the error
TclError: couldn't connect to display ":0.0"
I tried setting the DISPLAY environment variable to "", ":0.0" but I got
the same error message.
Any help is very much appreciated.
--
http://mail.python.org/mailman/listinfo/python-list
Re: CrazyHTTPd - HTTP Daemon in Python
Mark Lawrence wrote: > On 26/05/2013 04:55, cdorm...@gmail.com wrote: >> This is a small little Project that I have started. Its a light >> little Web Server (HTTPd) coded in python. Requirements: Python 2.7 >> =< And Linux / BSD. I believe this could work in a CLI Emulator in >> windows too. >> Welcome to check out the website powered by CrazyHTTPd: >> http://web.crazycoder.me:14081/index.html >> >> Please Email me with any bugs, errors, and/or comments about my >> little project: cdorm...@gmail.com >> > > IMHO writing a project right now that is limited to Python 2 is about as > much use as a chocolate teapot. I.e. not very useful, but slightly more so than you would expect: http://www.thenakedscientists.com/HTML/content/kitchenscience/exp/how-useless-is-a-chocolate-teapot/ Adiaŭ Marc -- http://mail.python.org/mailman/listinfo/python-list
Re: Cutting a deck of cards
Carlos Nepomuceno wrote: > >> Date: Sun, 26 May 2013 10:52:14 -0700 >> Subject: Cutting a deck of cards >> From: rvinc...@gmail.com >> To: python-list@python.org >> >> Suppose I have a deck of cards, and I shuffle them >> >> import random >> cards = [] >> decks = 6 >> cards = list(range(13 * 4 * decks)) >> random.shuffle(cards) >> >> So now I have an array of cards. I would like to cut these cards at >> some random point (between 1 and 13 * 4 * decks - 1, moving the lower >> half of that to the top half of the cards array. >> >> For some reason, I can't see how this can be done (I know that it >> must be a simple line or two in Python, but I am really stuck here). >> Anyone have any direction they can give me on this? Thanks, RVic, >> python newbie >> >> -- >> http://mail.python.org/mailman/listinfo/python-list > > > list(range(13 * 4 * decks)) == range(13 * 4 * decks) > > ;) Not in Python3.x >>> decks = 6 >>> list(range(13 * 4 * decks)) == range(13 * 4 * decks) False Adiaŭ Marc -- http://mail.python.org/mailman/listinfo/python-list
Re: Cutting a deck of cards
Carlos Nepomuceno wrote: > >> From: usenetm...@solar-empire.de > [...] >> Not in Python3.x >>>>> decks = 6 >>>>> list(range(13 * 4 * decks)) == range(13 * 4 * decks) >> False > > What does "list(range(13 * 4 * decks))" returns in Python 3? > A list of course. But Py3 range is very similar to Py2 xrange, it returns a range object. Adiaŭ Marc -- http://mail.python.org/mailman/listinfo/python-list
method that can be called from a class and also from an instance
Hi, I want to create a method within a class that is able to accept either a class or an instance. class MyClass(object): @magic_decorator def method(param): # param can be MyClass (cls) or an instance of MyClass (self) so I can do something like: instance = MyClass() MyClass.method() instance.method() I guess the way to go is implementing a custom decorator (@magic_decorator in my example), but, how can I know if the method has been called from the class o from an instance? Thank you very much!! -- http://mail.python.org/mailman/listinfo/python-list
Re: method that can be called from a class and also from an instance
On Thursday, November 22, 2012 4:51:30 PM UTC+1, Peter Otten wrote:
> Marc Aymerich wrote:
>
>
>
> > Hi,
>
> >
>
> > I want to create a method within a class that is able to accept either a
>
> > class or an instance.
>
> >
>
> > class MyClass(object):
>
> > @magic_decorator
>
> > def method(param):
>
> > # param can be MyClass (cls) or an instance of MyClass (self)
>
> >
>
> > so I can do something like:
>
> >
>
> > instance = MyClass()
>
> >
>
> > MyClass.method()
>
> > instance.method()
>
> >
>
> > I guess the way to go is implementing a custom decorator (@magic_decorator
>
> > in my example), but, how can I know if the method has been called from the
>
> > class o from an instance?
>
> >
>
> > Thank you very much!!
>
>
>
> Why would you overload a method that way?
Yep, it's an strange pattern sure it can be done in a better way but this is
the best I can think on, the context is:
I'm developing a permission system which can give general permissions for a
given class and also specific permissions for a given object.
class Node(object):
@role
def has_perm(instance, user)
if is_class(instance): then global perm for user...
else: then specific perm for user...
>
> $ cat class_or_inst.py
>
> import functools
>
>
>
> class desc(object):
>
> def __init__(self, f):
>
> self._f = f
>
> def __get__(self, inst=None, class_=None):
>
> if inst is not None:
>
> return functools.partial(self._f, self=inst)
>
> elif class_ is not None:
>
> return functools.partial(self._f, class_=class_)
>
> raise TypeError("nobody expects the Spanish inquisition")
>
>
>
>
>
>
>
> class A(object):
>
> @desc
>
> def f(self=None, class_=None):
>
> if self is not None:
>
> return "instance"
>
> elif class_ is not None:
>
> return "class"
>
> return "unknown"
>
> $ python -i class_or_inst.py
>
> >>> A.f()
>
> 'class'
>
> >>> A().f()
>
> 'instance'
>
> >>> A.__dict__["f"].__get__()
>
> Traceback (most recent call last):
>
> File "", line 1, in
>
> File "class_or_inst.py", line 11, in __get__
>
> raise TypeError("nobody expects the Spanish inquisition")
>
> TypeError: nobody expects the Spanish inquisition
you are a genius! I've implemented this on my code and it works as expected !
Thanks :D
--
http://mail.python.org/mailman/listinfo/python-list
Re: method that can be called from a class and also from an instance
On Thursday, November 22, 2012 5:26:59 PM UTC+1, Dave Angel wrote: > On 11/22/2012 11:12 AM, Thomas Bach wrote: > > > On Thu, Nov 22, 2012 at 10:52:56AM -0500, Dave Angel wrote: > > >> On 11/22/2012 10:14 AM, Marc Aymerich wrote: > > >>> I want to create a method within a class that is able to accept either a > >>> class or an instance. > > >>> > > >> I haven't tried it, but how about if you do a @classmethod decorator, > > >> and then just use isinstance(param, MyClass) ? > > >> > > > This won't work: > > > > > > In [22]: class Foo(object): > > >: @classmethod > > >: def bar(cls): > > >: print repr(cls) > > >: > > > > > > In [23]: Foo.bar() > > > > > > > > > In [24]: Foo().bar() > > > > > > > > > Actually help(classmethod) explicitly says so: > > > > > > It can be called either on the class (e.g. C.f()) or on an instance > > > (e.g. C().f()). The instance is ignored except for its class. > > > > > > > OK, thanks. I hadn't tried it, and hadn't noticed that that decorator > > converts to the class. > > > > > > > > I think the way to go is via the descriptor protocol[1] as suggested > > > by Peter. > > > > > > Regards, > > > Thomas. > > > > > > > > > Footnotes: > > > [1] http://docs.python.org/3/howto/descriptor.html > > > > > The OP should probably use this link instead, since he's not using Python 3. > > > > http://docs.python.org/2.7/howto/descriptor.html > > > > Marc: I believe the descriptor stuff has changed in Python 3; I don't > > use it. But if you've got to do this, and you have to do it in Python > > 2.x, you'd better use the 2.x documentation. > thanks for the links Thomas and Dave, I'm going to read this documentation right now, I love to learn this kind of python 'internals' :) -- http://mail.python.org/mailman/listinfo/python-list
Re: strptime - dates formatted differently on different computers
Greg Donald wrote:
> On Mon, Dec 10, 2012 at 10:34:31PM -0700, Michael Torrie wrote:
>> I use a module I got from pypi called dateutil. It has a nice submodule
>> called parser that can handle a variety of date formats with good
>> accuracy. Not sure how it works, but it handles all the common American
>> date formats I've thrown at it.
>
> from dateutil.parser import parse
> dt = parse( whatever )
>
> I've throw all kind of date and timestamps at it.. have yet to see
> anything it won't parse.
Interesting. First thing I tried gave an error:
>>> import locale
>>> locale.setlocale(locale.LC_ALL, '')
'LC_CTYPE=de_DE.utf8;LC_NUMERIC=de_DE.utf8;LC_TIME=de_DE.utf8;LC_COLLATE=C;LC_MONETARY=de_DE.utf8;LC_MESSAGES=C;LC_PAPER=de_DE.utf8;LC_NAME=de_DE.utf8;LC_ADDRESS=de_DE.utf8;LC_TELEPHONE=de_DE.utf8;LC_MEASUREMENT=de_DE.utf8;LC_IDENTIFICATION=de_DE.utf8'
>>> from dateutil.parser import parse
>>> parse('1. Januar 2013')
Traceback (most recent call last):
File "", line 1, in
File "/usr/lib64/python3.3/site-packages/dateutil/parser.py", line 720, in
parse
return DEFAULTPARSER.parse(timestr, **kwargs)
File "/usr/lib64/python3.3/site-packages/dateutil/parser.py", line 310, in
parse
raise ValueError("unknown string format")
ValueError: unknown string format
>>> parse('1.2.2013') # ambiguous, I know
datetime.datetime(2013, 1, 2, 0, 0) # should be datetime.datetime(2013, 2, 1,
0, 0)
so it doesn't like long german dates and it misparses the numerical
form. And I even was so nice to set the locale :) (not that it succeeds
without…)
I admit I didn't read any documentation on it apart from help(parse)
which mentions a parserinfo argument, so one could probably give it a
hand at parsing.
The only thing this shows is that parsing dates is difficult.
Marc
--
http://mail.python.org/mailman/listinfo/python-list
Is it possible monkey patch like this?
Dear all, I want to monkey patch a method that has lots of code so I want to avoid copying all the original method for changing just two lines. The thing is that I don't know how to do this kind of monkey patching. Consider the following code: class OringinalClass(object): def origina_method(self, *args, **kwargs): ... if some_condition(): # This condition should be changed raise SomeException ... if some_condition(): ... #if some_condition(local_variable): # This condition should be added #raise SomeException ... Is it possible to tell Python to run the original method without stopping when an exception is raised? so I can catch them on a wrapper method and apply my conditional there. Any other idea on how to monkey patch those two conditionals ? Thanks! -- http://mail.python.org/mailman/listinfo/python-list
Loading a PKCS#1 public key using M2Crypto
Hi,
I've been trying very, very hard to load an RSA key using M2Crypto but without
any success.
basically this is what I'm trying to do:
>>> from M2Crypto import BIO, RSA
>>>
>>> pubkey = """-BEGIN RSA PUBLIC KEY-
... MIIBCgKCAQEApwotnfHT9RAmxnuaGEMdI3lYPYE4aaqSD9v4KbTh1E7Le3GNJQb7
... wCpmDe8+n8S5Kp/gBEpWiYuvsVA/T4KseoX7NMcacP+DJMwjmNd9U58USn2vLz0Z
... TMtXpc/FUhW5PZdgCiuNzw6IFgGn9ZCCv85jjUIW3KD8fUNdrUfVSv4olDoL9NkR
... dTRg3Os/znC6l0gv/mqnLaqj2bJ/tx47kUmj6Oq13JuEq34T+DVmsUCFVundQnRp
... c/vVEqQot7Rvj9UmSvTi4WKt/qxiAnyZf3gXOdrXvxfVTGzD5I/Xg+By+a4C2JwB
... A5RGvZP3fyfhkCnnhFDpfws5lc20FA6ryQIDAQAB
... -END RSA PUBLIC KEY-"""
>>>
>>> bio = BIO.MemoryBuffer(pubkey.encode('ascii'))
>>> RSA.load_pub_key_bio(bio)
Traceback (most recent call last):
File "", line 1, in
File "/usr/lib/python2.7/dist-packages/M2Crypto/RSA.py", line 422, in
load_pub_key_bio
rsa_error()
File "/usr/lib/python2.7/dist-packages/M2Crypto/RSA.py", line 302, in
rsa_error
raise RSAError, m2.err_reason_error_string(m2.err_get_error())
M2Crypto.RSA.RSAError: no start line
Reading all whats in Internet about this problem it seems that my key is in
PKCS#1 format but M2Crypto only reads X.501 RSA keys.
I know that python-crypto doesn't have any problem loading this key, but I'll
prefer to stick with M2Crypto because I already have lots code using M2Crypto.
So my question is, is there any way to load this key using M2Crypto? Can I
convert somehow the key to X.501?
I'll be very, very grateful if someone can come up with an interesting idea!
thanks a lot!!
Marc
--
http://mail.python.org/mailman/listinfo/python-list
Re: Loading a PKCS#1 public key using M2Crypto
On Thursday, January 17, 2013 1:32:25 AM UTC+1, Piet van Oostrum wrote:
> Marc Aymerich writes:
>
>
>
> > Hi,
>
> > I've been trying very, very hard to load an RSA key using M2Crypto but
> > without any success.
>
> >
>
> > basically this is what I'm trying to do:
>
> >>>> from M2Crypto import BIO, RSA
>
> >>>>
>
> >>>> pubkey = """-BEGIN RSA PUBLIC KEY-
>
> > ... MIIBCgKCAQEApwotnfHT9RAmxnuaGEMdI3lYPYE4aaqSD9v4KbTh1E7Le3GNJQb7
>
> > ... wCpmDe8+n8S5Kp/gBEpWiYuvsVA/T4KseoX7NMcacP+DJMwjmNd9U58USn2vLz0Z
>
> > ... TMtXpc/FUhW5PZdgCiuNzw6IFgGn9ZCCv85jjUIW3KD8fUNdrUfVSv4olDoL9NkR
>
> > ... dTRg3Os/znC6l0gv/mqnLaqj2bJ/tx47kUmj6Oq13JuEq34T+DVmsUCFVundQnRp
>
> > ... c/vVEqQot7Rvj9UmSvTi4WKt/qxiAnyZf3gXOdrXvxfVTGzD5I/Xg+By+a4C2JwB
>
> > ... A5RGvZP3fyfhkCnnhFDpfws5lc20FA6ryQIDAQAB
>
> > ... -END RSA PUBLIC KEY-"""
>
> >>>>
>
> >>>> bio = BIO.MemoryBuffer(pubkey.encode('ascii'))
>
> >>>> RSA.load_pub_key_bio(bio)
>
> > Traceback (most recent call last):
>
> > File "", line 1, in
>
> > File "/usr/lib/python2.7/dist-packages/M2Crypto/RSA.py", line 422, in
> > load_pub_key_bio
>
> > rsa_error()
>
> > File "/usr/lib/python2.7/dist-packages/M2Crypto/RSA.py", line 302, in
> > rsa_error
>
> > raise RSAError, m2.err_reason_error_string(m2.err_get_error())
>
> > M2Crypto.RSA.RSAError: no start line
>
> >
>
> >
>
> > Reading all whats in Internet about this problem it seems that my key is in
> > PKCS#1 format but M2Crypto only reads X.501 RSA keys.
>
> >
>
> > I know that python-crypto doesn't have any problem loading this key, but
> > I'll prefer to stick with M2Crypto because I already have lots code using
> > M2Crypto.
>
> >
>
> > So my question is, is there any way to load this key using M2Crypto? Can I
> > convert somehow the key to X.501?
>
> >
>
> Converting to X.501 isn't difficult (assuming this is a 2048 bit key):
>
> Get rid of the 'RSA' in header and trailer
>
> Prepend X.501 header 'MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8A' to the data
>
> Reformat the lines to 64 characters.
>
>
>
> from M2Crypto import BIO, RSA
>
>
>
> pubkey="""-BEGIN RSA PUBLIC KEY-
>
> MIIBCgKCAQEApwotnfHT9RAmxnuaGEMdI3lYPYE4aaqSD9v4KbTh1E7Le3GNJQb7
>
> wCpmDe8+n8S5Kp/gBEpWiYuvsVA/T4KseoX7NMcacP+DJMwjmNd9U58USn2vLz0Z
>
> TMtXpc/FUhW5PZdgCiuNzw6IFgGn9ZCCv85jjUIW3KD8fUNdrUfVSv4olDoL9NkR
>
> dTRg3Os/znC6l0gv/mqnLaqj2bJ/tx47kUmj6Oq13JuEq34T+DVmsUCFVundQnRp
>
> c/vVEqQot7Rvj9UmSvTi4WKt/qxiAnyZf3gXOdrXvxfVTGzD5I/Xg+By+a4C2JwB
>
> A5RGvZP3fyfhkCnnhFDpfws5lc20FA6ryQIDAQAB
>
> -END RSA PUBLIC KEY-
>
> """
>
>
>
> pk = pubkey.split('\n')
>
> pk = 'MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8A' + ''.join(pk[1:-2])
>
> pk = [pk[i:i+64] for i in range(0, len(pk), 64)]
>
> pk = '-BEGIN PUBLIC KEY-\n' + '\n'.join(pk) + '\n-END PUBLIC
> KEY-'
>
>
>
> bio = BIO.MemoryBuffer(pk) # pk is ASCII, don't encode
>
> key = RSA.load_pub_key_bio(bio)
>
>
Piet, that was really awesome, it seems so easy to do now :)
Thank you very, very much! really.
--
http://mail.python.org/mailman/listinfo/python-list
Re: Loading a PKCS#1 public key using M2Crypto
On Thursday, January 17, 2013 5:39:57 PM UTC+1, Piet van Oostrum wrote: > > Converting to X.501 isn't difficult (assuming this is a 2048 bit key): > > > Get rid of the 'RSA' in header and trailer > > > Prepend X.501 header 'MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8A' to the data > > > Reformat the lines to 64 characters. > > > > This solution is a bit restricted as it only works if the key is 2048 > > bits and uses an exponent of 65537 (which is the default). Otherwise it > > fails. > > > > Here is a robust solution that works for all PKCS#1 keys. Instead of > > using a fixed X.501 header it calculates the header. We could do a > > complete ASN.1 encoding, but most of the parts are fixed. The only > > variable parts are two length fields. So I just plug these into the > > fixed stuff. This saves using one of the ASN.1 libraries. We do have to > > work in binary (DER format) instead of base64, however. > Thank you very much Piet, I'm just starting to grasp these cryptography related concepts and your code is helping me a lot to understand how to handle these keys in a low level. I'm updating my code incorporating your new contribution! Just to let you know, during my previous research I had found a python-Crypto related solution that also uses DER and ASN.1 [1], but it uses a different approach (I guess). I suspect that this approach is also possible with M2Crypto because it has a method for constructing RSA keys [2]. [1] http://stackoverflow.com/a/10574723 [2] http://www.heikkitoivonen.net/m2crypto/api/M2Crypto.RSA-module.html#new_pub_key Thanks again! Marc PS: Sorry for my email format, I'm using google groups and it seems to ignore any mailing best practice. -- http://mail.python.org/mailman/listinfo/python-list
Re: Loading a PKCS#1 public key using M2Crypto
On Fri, Jan 18, 2013 at 12:10 AM, Piet van Oostrum wrote: > Marc Aymerich writes: > >> Thank you very much Piet, >> I'm just starting to grasp these cryptography related concepts and your code >> is helping me a lot to understand how to handle these keys in a low level. >> >> I'm updating my code incorporating your new contribution! >> >> Just to let you know, during my previous research I had found a >> python-Crypto related solution that also uses DER and ASN.1 [1], but it uses >> a different approach (I guess). I suspect that this approach is also >> possible with M2Crypto because it has a method for constructing RSA keys [2]. >> >> [1] http://stackoverflow.com/a/10574723 >> [2] >> http://www.heikkitoivonen.net/m2crypto/api/M2Crypto.RSA-module.html#new_pub_key >> > new_pub_key could be used but then you would have to do an ASN.1 parse > of the DER format of your key to get the n and e values. AFAICT M2Crypto > doesn't have methods to do this, so you would need to use one of the > python ASN.1 libraries (or write that part yourself). Thanks for the clarifications, Just sharing the document I'm reading right now, in case anyone found this thread and want to know more about ASN.1 and DER: ftp://ftp.rsasecurity.com/pub/pkcs/ascii/layman.asc -- Marc -- http://mail.python.org/mailman/listinfo/python-list
unexpected behaviour playing with dynamic methods
Hi, I'm playing a bit with python dynamic methods and I came up with a scenario that I don't understant. Considering the follow code: # Declare a dummy class class A(object): pass # generate a dynamic method and insert it to A class for name in ['a', 'b', 'c']: if name == 'b': @property def get_name(self): return name A.name = get_name a_instance = A() a_instance.name # So far I exptect that a_instance.name returns 'b', since it has been created when name == 'b', but this is what actually returns: >>> a_instance.name 'c' just the last 'name' value. What can I do in order to generate a method like this but that returns 'b' ? What is wrong in my understanding of this pice of code? Thanks !!! marc -- http://mail.python.org/mailman/listinfo/python-list
Re: unexpected behaviour playing with dynamic methods
On Feb 23, 2:05 pm, Peter Otten <__pete...@web.de> wrote: > Marc Aymerich wrote: > > Hi, > > > I'm playing a bit with python dynamic methods and I came up with a > > scenario that I don't understant. Considering the follow code: > > > # Declare a dummy class > > class A(object): > > pass > > > # generate a dynamic method and insert it to A class > > for name in ['a', 'b', 'c']: > > if name == 'b': > > @property > > def get_name(self): > > return name > > A.name = get_name > > > a_instance = A() > > a_instance.name > > > # So far I exptect that a_instance.name returns 'b', since it has > > been created when name == 'b', but this is what actually returns: > > >>>> a_instance.name > > 'c' > > > just the last 'name' value. > > What can I do in order to generate a method like this but that returns > > 'b' ? What is wrong in my understanding of this pice of code? > > Look at the method again: > > > def get_name(self): > > return name > > It returns the global variable name. Why would you expect to see a > historical value of that variable? hehe, yeah, after sending my email I realized that it was obvious, but the true is I came up with this problem in a more complex situation involving lot's of imports and so on.. so the question: wtf is going on?? came up to me at this time :P > If you want to capture the value of name at the time when get_name() is > defined you have several options: > > - The default argument trick: > > def get_name(self, name=name): > return name > > - A closure: > > def make_get_name(name): > def get_name(self): > return name > return get_name > A.name = property(make_get_name(name)) > > - functools.partial() > > def get_name(self, name): > return name > A.name = property(partial(get_name, name=name)) Wow Peter, I was expecting one solution and you give me 3 amazing solutions!! :) Thanks a lot! btw I always wondered for what functools.partial can be useful, now I know an example :) -- http://mail.python.org/mailman/listinfo/python-list
Re: numpy.genfromtxt with Python3 - howto
Helmut Jarausch wrote: > I have a machine with a non-UTF8 local. > I can't figure out how to make numpy.genfromtxt work [...] > #!/usr/bin/python3 > import numpy as np > import io > import sys > > inpstream = io.open(sys.stdin.fileno(), "r", encoding='latin1') > > data = np.genfromtxt(inpstream) Hi Helmut, numpy.genfromtxt wants bytes, not str. Use inpstream = io.open(sys.stdin.fileno(), "rb") or simply inpstream = sys.stdin.buffer Ciao Marc -- http://mail.python.org/mailman/listinfo/python-list
Default value for optional parameters unexpected behaviour?
Hi, I'm trying to define a function that has an optional parameter which should be an empty list whenever it isn't given. However, it takes as value the same value as the last time the function was executed. What is the reason of this behaviour? How does python deal with default values (i.e. when are they assigned/created)? Thanks :) >>> def a(foo=[]): ... foo.append(1) ... print foo ... >>> a() [1] >>> a() [1, 1] >>> a() [1, 1, 1] >>> a() [1, 1, 1, 1] >>> a() [1, 1, 1, 1, 1] >>> a() [1, 1, 1, 1, 1, 1] -- http://mail.python.org/mailman/listinfo/python-list
Re: pairwise combination of two lists
Yingjie Lin wrote: > Hi Python users, > > I have two lists: > > li1 = ['a', 'b'] > li2 = ['1', '2'] > > and I wish to obtain a list like this > > li3 = ['a1', 'a2', 'b1', 'b2'] > > Is there a handy and efficient function to do this, especially when > li1 and li2 are long lists. Depending on your needs, we can offer you three solutions: For our customers who want it all at once, but without any unneccessary waste, the list expression: [a + b for a, b in itertools.product(li1, li2)] or if you don't need the whole list at once (especially interesting for our customers with large lists), the generator expression (genexp): (a + b for a, b in itertools.product(li1, li2)) and if you don't like the throwaway genexp and want something more ecofriedly, we can present you a memory efficient, reusable solution, the generator: def combiner(li1, li2): for a, b in itertools.product(li1, li2): yield a + b ;) HTH, Marc -- http://mail.python.org/mailman/listinfo/python-list
Python encoding question
Hi,
I'm doing my first steps with python and I have a problem with understanding
an encoding problem I have. My script:
import os
os.environ["NLS_LANG"] = "German_Germany.UTF8"
import cx_Oracle
connection = cx_Oracle.Connection("username/password@SID")
cursor = connection.cursor()
cursor.execute("SELECT NAME1 FROM COR WHERE CORNB='ABCDEF'")
TEST = cursor.fetchone()
print TEST[0]
print TEST
When I run this script It prints me:
München
('M\xc3\xbcnchen',)
Why is the Umlaut of TEST[0] printed and not from TEST?
And why are both prints show the wrong encoding, when I switch "fetchone()" to
"fetchall()":
('M\xc3\xbcnchen',)
[('M\xc3\xbcnchen',)]
I'm running Python 2.4.3 on CentOS 5.
Regards,
Marc
--
http://mail.python.org/mailman/listinfo/python-list
best practice to resolve module dependency within a reusable application
Hi, I'm developing a reusable app splited into modules. The end user chooses what modules wants to keep installed. Most of this modules are quite independent from each other, but I have one of them (called moduleP) with a pretty strong dependency with another another(called moduleBase). So I need to change some of the moduleBase behaviour whenever moduleP is installed. What is the best practice to resolve this dependency in a reusable scenario? Where should the specific moduleP code for moduleBase live? in moduleP or in moduleBase? I thought in two possible solutions: 1) moduleP specific code lives in moduleBase an it is called if "is moduleP installed": class moduleBase(otherClass): def method(self): super(moduleBase, self).method() # moduleBase stuff . . if 'moduleP' in INSTALLED_MODULES: # moduleP stuff . . 2) moduleP code lives in moduleP and it swaps moduleBase method with their own, like this: # backup moduleBase method moduleBase.back_methodBase = moduleBase.method def Pmethod(self): # calls moduleBase.method self.back_methodBase_method # moduleP stuff . . # swap moduleP Pmethod moduleBase.method = Pmethod I'm missing some other mechanism to achieve this? Which one do you think is best? Many thanks!! -- http://mail.python.org/mailman/listinfo/python-list
subclassing an extension (non built-in) class / type in python 3.2
Hello everybody, I read in the extending and embedding documentation of python 3, that it is easily possible to subclass or subtype a built-in type. However, in my case, I would like to subclass a class, defined in a external python-file (py), which I can import in my C++-application and access the class by the module's dictionary. Does anybody have an example or just a hint, how to successfully realize this type of inheritance? I will be grateful for every response. Cheers Marc -- GMX DSL Doppel-Flat ab 19,99 Euro/mtl.! Jetzt mit gratis Handy-Flat! http://portal.gmx.net/de/go/dsl -- http://mail.python.org/mailman/listinfo/python-list
Fwd: subclassing an extension (non built-in) class / type in python 3.2
Hello everybody, I read in the extending and embedding documentation of python 3, that it is easily possible to subclass or subtype a built-in type. However, in my case, I would like to subclass a class, defined in a external python-file (py), which I can import in my C++-application and access the class by the module's dictionary. Does anybody have an example or just a hint, how to successfully realize this type of inheritance? I will be grateful for every response. Cheers Marc -- NEU: FreePhone - kostenlos mobil telefonieren und surfen! Jetzt informieren: http://www.gmx.net/de/go/freephone -- http://mail.python.org/mailman/listinfo/python-list
Gambit REPL app for iPhone/iPod touch/iPad
A version of the Gambit Scheme system for iPhone/iPod touch/iPad is now available on the Apple App Store: http://itunes.apple.com/us/app/gambit-repl/id434534076?mt=8&ls=1 "Gambit REPL" is a complete version of Gambit (http:// dynamo.iro.umontreal.ca/~gambit) including the interpreter, debugger and built-in library (but not the compiler). The standard REPL is provided and also a script editor. Scripts can be saved, attached to function keys, and the "main" function of the application can be redefined to customize the application to your needs. Applications can be debugged remotely by telneting to the device from a desktop machine. The application is sold for $0.99 as a demonstration that you can distribute on the App Store paid applications written in Scheme and containing an interpreter, something that wasn't clear with previous versions of the App Store developer agreement. A version of Gambit REPL for Android is not planned (by me). The Gambit Scheme system compiles fine on Android (for example see https://github.com/seoushi/gambit-android-example), but the user interface layer would have to be changed and tested, and I don't own an Android device. If someone wants to try porting Gambit REPL to Android let me know and I can help. The sources of Gambit REPL for iOS are in the examples/iOS subdirectory of the Gambit source distribution. Marc -- http://mail.python.org/mailman/listinfo/python-list
Re: GIL in alternative implementations
Daniel Kluev wrote: > test.py: > > from threading import Thread > class X(object): >pass > obj = X() > obj.x = 0 > > def f(*args): > for i in range(1): > obj.x += 1 > > threads = [] > for i in range(100): >t = Thread(target=f) >threads.append(t) >t.start() > > for t in threads: >while t.isAlive(): >t.join(1) > > print(obj.x) > >> python test.py > 100 >> pypy test.py > 100 >> jython-2.5 test.py > 19217 >> ipy test.py > 59040 > > Not that this thing is reasonable to do in real code, but cpython and > other implementations with GIL at least give you some safety margin. > Sure? ;) > for p in python2.4 python2.5 python2.6 python2.7 python3.1 python3.2; do > echo $p; $p /tmp/test.py; $p /tmp/test.py; done python2.4 525369 736202 python2.5 449496 551023 python2.6 903405 937335 python2.7 885834 910144 python3.1 866557 766842 python3.2 100 100 So even CPython (at least < 3.2) isn't safe. And I wouldn't rely on 3.2 not to break. Marc -- http://mail.python.org/mailman/listinfo/python-list
Sharing credentials between multiple interactive processes
I would like to throw at you some problem that I don't know how to best approach :) Have an interactive CLI python program that holds credentials entered by the user. I want users to be able to spawn a new instance of this program (on another TTY) without the need of reentering credentials. Want to do it in a secure way, meaning make it very difficult for malicious programs to gain access to the credentials. Some solutions that I thought of: 1) From another TTY, communicate with the existing process and tell it to fork and attach to my TTY. Not sure yet how to do it, but I think is doable. 2) Make the program to be able to send credentials through a Unix Domain Socket to its peers. Not sure is there any way to prevent sending credentials to any random process pocking at the socket without a challenge. Maybe is there any way to validate that 2 running python processes are executing the exact same code (peers)? any thoughts? -- Marc -- https://mail.python.org/mailman/listinfo/python-list
could use some help with this problem! I've been working on it for days but cant seem to get it right !
USING PYTHON 2: Write a program to play this game. This may seem tricky, so break it down into parts. Like many programs, we have to use nested loops (one loop inside another). In the outermost loop, we want to keep playing until we are out of stones. Inside that, we want to keep alternating players. You have the option of either writing two blocks of code, or keeping a variable that tracks the current player. The second way is slightly trickier since we haven't learned lists yet, but it's definitely do-able! Finally, we might want to have an innermost loop that checks if the user's input is valid. Is it a number? Is it a valid number (e.g. between 1 and 5)? Are there enough stones in the pile to take off this many? If any of these answers are no, we should tell the user and re-ask them the question So, the basic outline of the program should be something like this: totalStones = 100 maxStones = 5 pile = TOTAL # all stones are in the pile to start while [pile is not empty]: while [player 1's answer is not valid]: [ask player 1] [execute player1’s move] Do the same for player 2…. (this can be achieved by a for loop) Note how the important numbers 100 and 5 are stored in a single variable at the top. This is good practice -- it allows you to easily change the constants of a program. For example, for testing, you may want to start with only 15 or 20 stones. Be careful with the validity checks. Specifically, we want to keep asking player 1 for their choice as long as their answer is not valid, BUT we want to make sure we ask them at least ONCE. So, for example, we will want to keep a variable which tracks whether their answer is valid, and set it to False initially. Hint: the final number of stones can’t be negative. The game automatically stops when the number of stones reach zero. -- https://mail.python.org/mailman/listinfo/python-list
Re: Subtracting dates to get hours and minutes
my approach would be to convert your two date/times to seconds from epoch - e.g. https://www.geeksforgeeks.org/convert-python-datetime-to-epoch/ - then subtract the number, divide the resultant by 3600 (hours) & get the modulus for minutes. There's probably a standard function - it should be /very/ easy to do. - Marc On 12/12/2022 5:01 pm, Steve GS wrote: How do I subtract two time/dates and calculate the hours and minutes between? Steve -- https://mail.python.org/mailman/listinfo/python-list
combinations of all rows and cols from a dataframe
Hello everyone,
Given a dataframe like this:
2 6
8 5
I want to yield the following list of lists:
[ [[2],[6,5]],
[[2],[6]],
[[2],[5]],
[[8],[6,5]],
[[8],[6]],
[[8],[5]],
[[6],[2,8]],
[[6],[8]],
[[6],[2]],
[[5],[2,8]],
[[5],[2]],
[[5],[8]],
[[6,5],[2,8]] ]
I have written the following (which doesn't yield the expected results)
import pandas as pd
> from itertools import combinations
> import numpy as np
> resList=[]
> resListTmp=[]
> resListTmp2=[]
> dataframe =
> pd.read_excel("C:\\Users\\user\\Desktop\\testData.xlsx",index_col=False,header=None)
for i in range(0, len(dataframe)+1):
> for j in range(0, len(dataframe.columns)):
> for k in range (0,len(dataframe)+1):
> for xVals in list(combinations(dataframe.iloc[k:i,j], i)):
> if list(xVals) not in resListTmp:
> resListTmp.append(list(xVals))
> resListTmp2.append(resListTmp)
> resList.append(resListTmp2)
> print(resList)
>
What is wrong with my code?
--
https://mail.python.org/mailman/listinfo/python-list
Re: Choosing a Python IDE. what is your Pythonish recommendation? I do not know what to choose.
I'd recommend you be willing to put in the time and effort to learn the tools you want to use, if you want to do professional software development. Pick one, use it for a month (at least 100+ hours of hands on keyboard coding). Sublime, Vi are great for Python, since Python doesn't require as much as some other languages, but you sill need to put in the time to learn the tool. On Mon, Jan 2, 2017 at 6:38 AM, Antonio Caminero Garcia < tonycam...@gmail.com> wrote: > Hello, I am having a hard time deciding what IDE or IDE-like code editor > should I use. This can be overwhelming. > > So far, I have used Vim, Sublime, Atom, Eclipse with PyDev, Pycharm, > IntelliJ with Python plugin. > > The thing with the from-the-scratch full featured IDEs (Eclipse, IntelliJ, > Pycharm) is that they look like a space craft dashboard and that > unwarranted resources consumption and the unnecessary icons. I want my IDE > to be minimalistic but powerful. My screen should be mostly “made of code” > as usually happens in Vim, Sublime or Atom. However, Pycharm is really cool > and python oriented. > > The problem with Vim is the learning curve, so I know the very basic > stuff, but obviously not enough for coding and I do not have time to learn > it, it is a pity because there are awesome plugins that turns Vim into a > lightweight powerful IDE-like. So now it is not an option but I will > reconsider it in the future, learning little by little. Also, I am not very > fan GUI guy if the task can be accomplished through the terminal. However, > I don’t understand why people underrate GUIs, that said I normally use > shortcuts for the most frequent tasks and when I have to do something that > is not that frequent then I do it with the mouse, for the latter case in > vim you would need to look for that specific command every time. > > Sublime is my current and preferred code editor. I installed Anaconda, Git > integration and a couple of additional plugins that make sublime very > powerful. Also, what I like about sublime compared to the full featured > IDEs, besides the minimalism, is how you can perform code navigation back > and forth so fast, I mean this is something that you can also do with the > others but for some subjective reason I specifically love how sublime does > it. The code completion in sublime I do not find it very intelligence, the > SublimeCodeIntel is better than the one that Anaconda uses but the > completions are not as verbose as in the IDEs. > > Now, I am thinking about giving a try to Visual Studio Code Edition (take > a look, it sounds good https://marketplace.visualstudio.com/items? > itemName=donjayamanne.python). I need an editor for professional software > development. What would you recommend to me? > -- > https://mail.python.org/mailman/listinfo/python-list > -- https://mail.python.org/mailman/listinfo/python-list
Re: How to create development Python environment on Linux.
I am pretty sure (but not 100%) that the pip that virtualenv installs when it first creates the virtualenv is the version of pip installed on the system. Here's the process I used to bootstrap a new Python 2.7 dev environment. 1. Download and install the latest version of pip as sudo so it's system wide. 2. Install virtualenv and virtualenvwrapper (a collection of utilities scripts/aliases for virtualenv). 3. Update my .bash_profile to source the virtualenvwrapper script. Then for any new virtualenvs I just type 'mkvirtualenv ' I can update the version of pip in the virtualenv or run pip install for any of my required libraries at that point. One wrinkle that can come up is if you want to use virtualenvwrapper and you are not using bash. Fish (another moderately popular shell) has an addon that mimics the macros that virtualenvwrapper provides. Mar On Wed, Jun 1, 2016 at 8:17 PM, Lawrence D’Oliveiro wrote: > On Tuesday, May 17, 2016 at 4:26:23 AM UTC+12, Zachary Ware wrote: > > Not what you asked for, but I would encourage you to look into whether > > it's possible for you to use Python 3 instead of Python 2 for what > > you're doing. If it's possible, starting with Python 3 will save you > > several headaches in the future. > > Let me add my vote for this. > > > sys.prefix is baked in at compile time of the python interpreter ... > > ldo@theon:~> ~/virtualenv/jupyter/bin/python -c "import sys; > print(sys.prefix)" > /home/ldo/virtualenv/jupyter > -- > https://mail.python.org/mailman/listinfo/python-list > -- https://mail.python.org/mailman/listinfo/python-list
Re: Python on Windows with linux environment
Others have mentioned Cygwin or the new subsystem. The other two options a number of Python develops that I work use are to (1) install vmware/virtualbox and run linux in a vm or (2) install the packaged binaries, such as Anaconda from Continuum Analytics. One issue I had when using cygwin was that it was sometimes a bit more annoying to get libraries installed, since it couldn't use any pre-compiled binaries for Windows. I don't know if that's still the case, since this was a few years back and I'm doing my dev work on a mac nowadays. Marc On Thu, Jun 2, 2016 at 7:22 AM, Muhammad Ali wrote: > > Hi, > > I use windows regularly, however, I use linux for only my research work at > supercomputer. In my research field (materials science) most of the scripts > are being written in python with linux based system. Could I installed such > linux based python on my window 7? So that I can use those linux based > scripts written in python and I can also write my own scripts/code without > entirely changing my operating system from windows to linux. > > Looking for your valuable suggestions. > > Thank you. > -- > https://mail.python.org/mailman/listinfo/python-list > -- https://mail.python.org/mailman/listinfo/python-list
Re: i'm a python newbie & wrote my first script, can someone critique it?
The structure of your program is really not that Pythonic. I'd recommend you take a look at PEP 8. https://www.python.org/dev/peps/pep-0008/ It's not perfect, but it's a good start to get a feel for how to structure your Python work. Marc On Fri, Jun 10, 2016 at 7:05 PM, Christopher Reimer < christopher_rei...@icloud.com> wrote: > > > Sent from my iPhone > > > On Jun 10, 2016, at 3:52 PM, mad scientist jr < > mad.scientist...@gmail.com> wrote: > > . > > Now that it's done, I am wondering what kind of things I could do better. > > This is Python, not BASIC. Lay off on the CAP LOCK key and delete all the > comments and separation blocks. No one is going to find your code in all > that noise. > > Chris R. > -- > https://mail.python.org/mailman/listinfo/python-list > -- https://mail.python.org/mailman/listinfo/python-list
Re: i'm a python newbie & wrote my first script, can someone critique it?
Look into docstrings. They will make your code much more readable to a
Python reader.
On Sat, Jun 11, 2016 at 2:16 PM mad scientist jr
wrote:
> For those who don't want to have to wade through comments, here is a
> version without so many comments:
>
> # For Python 3.x
> # This script creates multiple numbered empty folders
> # in the desired location. To change the folder names
> # or location, edit function get_default_options.
>
> import datetime
> import os
> import errno
> import sys
>
>
> ###
> # EDIT VALUES HERE TO CUSTOMIZE THE OUTPUT
> def get_default_options():
> dict = {
> "s_for_python_version": "3",
> "s_folder_path_template": "C:/temp/test/MP3 Disk {count:03}",
> "i_from_count": 3,
> "i_to_count": 7,
> }
> return dict
>
> ###
>
> def get_exact_python_version():
> s_version = ".".join(map(str, sys.version_info[:3]))
> s_version = s_version.strip()
> return s_version
>
> def get_general_python_version():
> s_version = get_exact_python_version()
> return s_version[0]
>
> def exit_if_wrong_python_version(s_right_version):
> s_current_version = get_general_python_version()
> if (s_current_version != s_right_version):
> print(
> "Wrong Python version ({}), "
> "this script should be run using "
> "Python {}.x, Exiting..."
> "".format(s_current_version, s_right_version))
> sys.exit()
>
> def get_script_filename():
> return sys.argv[0]
>
> def get_timestamp():
> return datetime.datetime.strftime(datetime.datetime.now(), '%Y-%m-%d
> %H:%M:%S')
>
> def create_folder(s_path):
> try:
> os.makedirs(s_path, exist_ok=True)
> except (FileExistsError, IsADirectoryError) as e:
> print("FileExistsError IN makedirs")
> raise
> return False
> except OSError as exception:
> print("ERROR #" + str(exception.errno) + "IN makedirs")
> raise
> return False
> print("" + get_timestamp() + " " + "Created folder: " + s_path + "")
>
> def create_folders(
> s_folder_path_template:str="",
> i_from_count:int=1,
> i_to_count:int=0
> ):
> i_count=0
> for i_loop in range(i_from_count, i_to_count + 1):
> create_folder(s_folder_path_template.format(count=i_loop))
> i_count += 1
>
> return i_count
>
> def main():
> options_dict = get_default_options()
> exit_if_wrong_python_version(options_dict["s_for_python_version"])
>
>
> print("+++")
> print("" + get_timestamp() + " " + get_script_filename() + " started.")
>
> i_total_created = create_folders(
> options_dict["s_folder_path_template"],
> options_dict["i_from_count"],
> options_dict["i_to_count"])
>
> print("" + get_timestamp() + " " + str(i_total_created) + " folders
> created.")
> print("" + get_timestamp() + " " + get_script_filename() + "
> finished.")
>
> if __name__ == '__main__':
> main()
> --
> https://mail.python.org/mailman/listinfo/python-list
>
--
https://mail.python.org/mailman/listinfo/python-list
Re: Indentation example?
On Sun, 12 Jun 2016 08:10:27 -0700 ICT Ezy wrote:
> Pl explain with an example the following phase "Indentation cannot be
> split over multiple physical lines using backslashes; the whitespace up
> to the first backslash determines the indentation" (in 2.1.8.
> Indentation of Tutorial.)
> I want to teach my student that point using some examples.
> Pl help me any body?
Hi!
This is my very first post inside the usenet. I hope I did understand
this right, so here is my answer. :)
I assume, that you do understand the concept of indentation inside Python
code. You can concatenate lines with a backslash. These lines work as if
they were only one line. For example:
>>> print ("This is a very long"\
... " line, that got "\
... "diveded into three lines.")
This is a very long line, that was diveded into three.
>>>
Because the lines get concatenated, one might think, that you could
divide for example 16 spaces of indentation into one line of 8 spaces
with a backslash and one line with 8 spaces and the actual code.
Your posted text tells you though, that you can't do this. Instead the
indentation would be considered to be only 8 spaces wide.
I hope this helped a little. :)
Cheers
Marc.
--
https://mail.python.org/mailman/listinfo/python-list
clusters of numbers
hey guys,
I have a hobby project that sorts my email automatically for me & I want
to improve it. There's data science and statistical info that I'm
missing, & I always enjoy reading about the pythonic way to do things too.
I have a list of percentage scores:
(1,11,1,7,5,7,2,2,2,10,10,1,2,2,1,7,2,1,7,5,3,8,2,6,3,2,7,2,12,3,1,2,19,3,5,1,1,7,8,8,1,5,6,7,3,14,6,1,6,7,6,15,6,3,7,2,6,23,2,7,1,21,21,8,8,3,2,20,1,3,12,3,1,2,10,16,16,15,6,5,3,2,2,11,1,14,6,3,7,1,5,3,3,14,3,7,3,5,8,3,6,17,1,1,7,3,1,2,6,1,7,7,12,6,6,2,1,6,3,6,2,1,5,1,8,10,2,6,1,7,3,5,7,7,5,7,2,5,1,19,19,1,12,5,10,2,19,1,3,19,6,1,5,11,2,1,2,5,2,5,8,2,2,2,5,3,1,21,2,3,7,10,1,8,1,3,17,17,1,5,3,10,14,1,2,14,14,1,15,6,3,2,17,17,1,1,1,2,2,3,3,2,2,7,7,2,1,2,8,2,20,3,2,3,12,7,6,5,12,2,3,11,3,1,1,8,16,10,1,6,6,6,11,1,6,5,2,5,11,1,2,10,6,14,6,3,3,5,2,6,17,15,1,2,2,17,5,3,3,5,8,1,6,3,14,3,2,1,7,2,8,11,5,14,3,19,1,3,7,3,3,8,8,6,1,3,1,14,14,10,3,2,1,12,2,3,1,2,2,6,6,7,10,10,12,24,1,21,21,5,11,12,12,2,1,19,8,6,2,1,1,19,10,6,2,15,15,7,10,14,12,14,5,11,7,12,2,1,14,10,7,10,3,17,25,10,5,5,3,12,5,2,14,5,8,1,11,5,29,2,7,20,12,14,1,10,6,17,16,6,7,11,12,3,1,23,11,10,11,5,10,6,2,17,15,20,5,10,1,17,3,7,15,5,11,6,19,14,15,7,1,2,17,8,15,10,26,6,1,2,10,6,14,12,6,1,16,6,12,10,10,14,1,6,1,6,6,12,6,6,1,2,5,10,8,10,1,6,8,17,11,6,3,6,5,1,2,1,2,6,6,12,14,7,1,7,1,8,2,3,14,11,6,3,11,3,1,6,17,12,8,2,10,3,12,12,2,7,5,5,17,2,5,10,12,21,15,6,10,10,7,15,11,2,7,10,3,1,2,7,10,15,1,1,6,5,5,3,17,19,7,1,15,2,8,7,1,6,2,1,15,19,7,15,1,8,3,3,20,8,1,11,7,8,7,1,12,11,1,10,17,2,23,3,7,20,20,3,11,5,1,1,8,1,6,2,11,1,5,1,10,7,20,17,8,1,2,10,6,2,1,23,11,11,7,2,21,5,5,8,1,1,10,12,15,2,1,10,5,2,2,5,1,2,11,10,1,8,10,12,2,12,2,8,6,19,15,8,2,16,7,5,14,2,1,3,3,10,16,20,5,8,14,8,3,14,2,1,5,16,16,2,10,8,17,17,10,10,11,3,5,1,17,17,3,17,5,6,7,7,12,19,15,20,11,10,2,6,6,5,5,1,16,16,8,7,2,1,3,5,20,20,6,7,5,23,14,3,10,2,2,7,10,10,3,5,5,8,14,11,14,14,11,19,5,5,2,12,25,5,2,11,8,10,5,11,10,12,10,2,15,15,15,5,10,1,12,14,8,5,6,2,26,15,21,15,12,2,8,11,5,5,16,5,2,17,3,2,2,3,15,3,8,10,7,10,3,1,14,14,8,8,8,19,10,12,3,8,2,20,16,10,6,15,6,1,12,12,15,15,8,11,17,7,7,7,3,10,1,5,19,11,7,12,8,12,7,5,10,1,11,1,6,21,1,1,10,3,8,5,6,5,20,25,17,5,2,16,14,11,1,17,10,14,5,16,5,2,7,3,8,17,7,19,12,6,5,1,3,12,43,11,8,11,5,19,10,5,11,7,20,6,12,35,5,3,17,10,2,12,6,5,21,24,15,5,10,3,15,1,12,6,3,17,3,2,3,5,5,14,11,8,1,8,10,5,25,8,7,2,6,3,11,1,11,7,3,10,7,12,10,8,6,1,1,17,3,1,1,2,19,6,10,2,2,7,5,16,3,2,11,10,7,10,21,3,5,2,21,3,14,6,7,2,24,3,17,3,21,8,5,11,17,5,6,10,5,20,1,12,2,3,20,6,11,12,14,6,6,1,14,15,12,15,6,20,7,7,19,3,7,5,16,12,6,7,2,10,3,2,11,8,6,6,5,1,11,1,15,21,14,6,3,2,2,5,6,1,3,5,3,6,20,1,15,12,2,3,3,7,1,16,5,24,10,7,1,12,16,8,26,16,15,10,19,11,6,6,5,6,5)
& I'd like to know know whether, & how the numbers are clustered. In
an extreme & illustrative example, 1..10 would have zero clusters;
1,1,1,2,2,2,7,7,7 would have 3 clusters (around 1,2 & 7);
17,22,20,45,47,51,82,84,83 would have 3 clusters. (around 20, 47 &
83). In my set, when I scan it, I intuitively figure there's lots of
numbers close to 0 & a lot close to 20 (or there abouts).
I saw info about k-clusters but I'm not sure if I'm going down the right
path. I'm interested in k-clusters & will teach myself, but my priority
is working out this problem.
Do you know the name of the algorithm I'm trying to use? If so, are
there python libraries like numpy that I can leverage? I imagine that I
could iterate from 0 to 100% using that as an artificial mean, discard
values that are over a standard deviation away, and count the number of
scores for that mean; then at the end of that I could set a threshold
for which the artificial mean would be kept something like (no attempt
at correct syntax:
means={}
deviation=5
threshold=int(0.25*len(list))
for i in range 100:
count=0
for j in list:
if abs(j-i) > deviation:
count+=1
if count > threshold:
means[i]=count
That algorithm is entirely untested & I think it could work, it's just I
don't want to reinvent the wheel. Any ideas kindly appreciated.
--
https://mail.python.org/mailman/listinfo/python-list
Processing an image with numarray.nd image
Hello,
Can anyone tell me how to open an image
and transform it into a list so that
the functions of the multi dimensionnal module of
numarray (numarray.nd image) can process it ?
Do I have to use PIL ?
So I would code something like :
import Image
im = Image.open("Python.jpg")
data = list(im.getdata())
import numarray.nd image as ti
ti.median filter(data,...)
Is that the correct way to do ?
Thanks for any help.
Raphaël
--
http://mail.python.org/mailman/listinfo/python-list
subprocess and non-blocking IO (again)
I am trying to rewrite a PERL automation which started a "monitoring"
application on many machines, via RSH, and then multiplexed their
collective outputs to stdout.
In production there are lots of these subprocesses but here is a
simplified example what I have so far (python n00b alert!)
- SNIP -
import subprocess,select,sys
speakers=[]
lProc=[]
for machine in ['box1','box2','box3']:
p = subprocess.Popen( ('echo '+machine+';sleep 2;echo goodbye;sleep
2;echo cruel;sleep 2;echo world'), stdout=subprocess.PIPE,
stderr=subprocess.STDOUT, stdin=None, universal_newlines=True )
lProc.append( p )
speakers.append( p.stdout )
while speakers:
speaking = select.select( speakers, [], [], 1000 )[0]
for speaker in speaking:
speech = speaker.readlines()
if speech:
for sentence in speech:
print sentence.rstrip('\n')
sys.stdout.flush() # sanity check
else: # EOF
speakers.remove( speaker )
- SNIP -
The problem with the above is that the subprocess buffers all its output
when used like this and, hence, this automation is not informing me of
much :)
In PERL, "realtime" feedback was provided by setting the following:
$p->stdout->blocking(0);
How do I achieve this in Python ?
This topic seems to have come up more than once. I am hoping that
things have moved on from posts like this:
http://groups.google.com/group/comp.lang.python/browse_thread/thread/5472ce95eb430002/434fa9b471009ab2?q=blocking&rnum=4#434fa9b471009ab2
as I don't really want to have to write all that ugly
fork/dup/fcntl/exec code to achieve this when high-level libraries like
"subprocess" really should have corresponding methods.
If it makes anything simpler, I only *need* this on Linux/Unix (Windows
would be a nice extra though).
thanks for reading,
Marc
--
http://mail.python.org/mailman/listinfo/python-list
Re: subprocess and non-blocking IO (again)
Donn Cave wrote: > If you want to use select(), don't use the fileobject > functions. Use os.read() to read data from the pipe's file > descriptor (p.stdout.fileno().) This is how you avoid the > buffering. Thankyou, this works perfectly. I figured it would be something simple. Marc -- http://mail.python.org/mailman/listinfo/python-list
Re: Python vs Ruby
I'd believe that would be Lua, but then again what is common to one might not be to another ;-) -- http://mail.python.org/mailman/listinfo/python-list
Re: Python vs Ruby
As you see, pythonistas are a nice humourous bunch... But to help a bit more in your balancing act you might take a look at: http://blog.ianbicking.org/ruby-python-power.html It's rather nice, and commented. -- http://mail.python.org/mailman/listinfo/python-list
Re: textwidget.tag_bind("name", "", self.donothing) not working
but you don't want to use the state=DISABLED option because it gray's out the field showing people that it is not available for editing, right? -- http://mail.python.org/mailman/listinfo/python-list
Re: textwidget.tag_bind("name", "", self.donothing) not working
Sorry, kinda wrote over your intentions... -- http://mail.python.org/mailman/listinfo/python-list
Re: textwidget.tag_bind("name", "", self.donothing) not working
To make amends, I tried my own search and came up with this (that you might already have...): http://groups.google.com/group/comp.lang.python/browse_thread/thread/1384f49c35ffba9b/5928092247429e9a%235928092247429e9a?sa=X&oi=groupsr&start=1&num=3 Maybe you'll understand it better than me :-) -- http://mail.python.org/mailman/listinfo/python-list
MPI and python+threads on IRIX
Hi, I'm trying to add a dynamic module using MPI on a 2.3.4 Python with threads (posix). The interpreter blocks into the dlopen (dynload_shlib.c) if any reference to the IRIX libmpi.so (actually SGI/IRIX 6.5) appears. The docs say that the use of MPI+dlopen requires a call to MPI_Init_thread before the dlopen. I wonder how I can build a shared module using mpi if it blocks inside the dlopen ! It looks like the dlopen itself blocks at the time it reads the libmpi.so (is it possible). The MPI modules I've found are not ported on SGI (yes I know...) Any hints ? Help ? Piece of advice ? Idea ? Patch ? Coconut ? -MP- --- Marc POINOT Alias: marcvsEmail: [EMAIL PROTECTED] ONERA -MFE/DSNA/ELSATel: 01.46.73.42.84 Info: [EMAIL PROTECTED] 29, Div. LeclercFax: 01.46.73.41.66 Site: 92322 Chatillon FRANCE Project: elsAWeb: http://www.onera.fr -- http://mail.python.org/mailman/listinfo/python-list
Re: Where can I find sample "beginner" programs to study?
Todd_Calhoun wrote: > I'm trying to learn Python (and programming), and I'm wondering if there > are any places where I can find small, simple programs to study. > > Thanks. Look at http://www.livewires.org.uk/python/ Good projects and tutorials for beginners. -- http://mail.python.org/mailman/listinfo/python-list
Using HTTPSConnection and verifying server's CRT
Hi, I'm trying to build a system using HTTPS with python clients that have to verify the server's identity. From the Python document, it seems that the server's certificate is not veryfied, and authentication can only be in the other way (client authentication). I know usually users only click on 'yes I trust this certificate', but what if you really care (this is my case)? I tried to see if the M2Crypto has this possibility, but from my tests and from what I can find on the website, it seems not :/ Can someone confirm me this is not possible or point me to something that could help me? Thanks, Marc -- http://mail.python.org/mailman/listinfo/python-list
Re: Using HTTPSConnection and verifying server's CRT
[EMAIL PROTECTED] (Ng Pheng Siong) writes:
Hi,
> According to Marc Poulhiès <[EMAIL PROTECTED]>:
>> I tried to see if the M2Crypto has this possibility, but from my tests
>> and from what I can find on the website, it seems not :/
>
> How did you test and where on the website does it say not?
I did things like this:
con = M2Crypto.httpslib.HTTPSConnection("some_secure_server")
con.request("GET" , "/")
I tried to play with optional parameters (strict, debuglevel, etc) to
see if it was saying that it will not check server's CRT or some other
debug message dealing with server's certificate, but it is always
returning the webpage without saying anything :)
I did not say that M2C's doc stated clearly that this was not possible
(that's why I wrote "seems"), but I couldn't find something stating it
was possible (I tried google, API docs).
>> Can someone confirm me this is not possible or point me to something
>> that could help me?
>
> M2Crypto does server cert verification. With M2Crypto's httpslib, you pass
> in an SSL.Context instance to the HTTPSConnection constructor to configure
> the SSL; one of the config knobs is cert verification. So, redo your test,
> satisfy yourself that this is doable, and send me your code to include as
> an example in the distribution. ;-)
Ok, sorry for that. Maybe that with more readings I could have spotted
this. I'll try that tomorrow and give my code if I have something
working!
> M2Crypto even does client certs. Since Apr 2000, according to the very last
> blog entry on the ZServerSSL page.
Yes, I did try this and have my client authenticated to the server.
Thanks for this quick and clear answer ;)
Marc
--
http://mail.python.org/mailman/listinfo/python-list
Re: Using HTTPSConnection and verifying server's CRT
Marc Poulhiès <[EMAIL PROTECTED]> writes:
> [EMAIL PROTECTED] (Ng Pheng Siong) writes:
>> M2Crypto does server cert verification. With M2Crypto's httpslib, you pass
>> in an SSL.Context instance to the HTTPSConnection constructor to configure
>> the SSL; one of the config knobs is cert verification. So, redo your test,
>> satisfy yourself that this is doable, and send me your code to include as
>> an example in the distribution. ;-)
Hi again!
So here are few lines that do server's CRT check. I still have one
question: see in the code. Both have the exact same description on
the documentation.
Btw, thanks for your answer (this will save me from using Perl!)
Marc
---8<---8<---8<---8<
#!/usr/bin/env python
import M2Crypto
ctx = M2Crypto.SSL.Context()
## what are the diff between these two??
#ctx.load_verify_info(cafile="/tmp/ca.crt")
ctx.load_verify_locations(cafile="/tmp/ca.crt")
# load client certificate (used to authenticate the client)
ctx.load_cert("/tmp/client.crt")
# stop if peer's certificate can't be verified
ctx.set_allow_unknown_ca(False)
# verify peer's certificate
ctx.set_verify(M2Crypto.SSL.verify_peer, 1)
con = M2Crypto.httpslib.HTTPSConnection("my.ssl.server.domain",ssl_context=ctx)
con.request("GET" , "/")
print con.getresponse().read()
---8<---8<---8<---8<-
Result here:
$ ./ssl_peer_verif.py
Enter passphrase:
send: 'GET / HTTP/1.1\r\nHost: my.ssl.server.domain:443\r\nAccept-Encoding:
identity\r\n\r\n'
reply: 'HTTP/1.1 200 OK\r\n'
header: Date: Tue, 01 Feb 2005 08:41:51 GMT
header: Server: Apache/2.0.46 (Red Hat)
header: Last-Modified: Mon, 31 Jan 2005 14:50:50 GMT
header: ETag: "4297-13-24658680"
header: Accept-Ranges: bytes
header: Content-Length: 19
header: Connection: close
header: Content-Type: text/html; charset=UTF-8
THIS IS WORKING =)
--
http://mail.python.org/mailman/listinfo/python-list
Re: test_socket.py failure
[EMAIL PROTECTED] wrote:
> Nick Coghlan <[EMAIL PROTECTED]> wrote:
>> Hmm, when the second argument is omitted, the system call looks like:
>>
>> getservbyname("daytime", NULL);
>>
>> Based on "man getservbyname" on my Linux PC, that should give
>> the behaviour we
>> want - any protocol will match.
>>
>> However:
>>
>> Linux 2.6.4-52-default (Suse 9.1)
>> Glibc 2.3.3
>> gcc 3.3.3
>>
>> So it may be that your older platform doesn't have this
>> behaviour - I'd be very
>> interested in what 'man getservbyname' has to say.
>
>Just took a look at the man page for getservbyname on this
>system and it doesn't mention passing NULL as the second
>argument. The pertinents: ;-)
>
>Linux kernel 2.6.10
>Glibc 2.2.5
>gcc 2.95.3
Just to confuse the matter more, on my system the man page mentions
passing NULL as the second argument and it works. Alas:
SuSE 7.3
Kernel 2.4.29 (vanilla)
Glibc 2.2.4 (older than yours)
gcc2.95.3
>I'd say your probably right about there being a difference
>in the behaviour of getservbyname between libc 2.2.5 and
>and libc-2.3.3 given the differences in man pages and
>observed return values. I'll try and compare the libcs'
>getservbyname codes and let you know a little later in
>the day.
>
>I wonder if the developers wanted to tie the python source
>code so closely to a glibc version and possibly gnu-libc
>specific?
Perhaps SuSE did patch the glibc...
Saluton
Marc
--
http://mail.python.org/mailman/listinfo/python-list
Re: Finding user's home dir
Miki Tebeka <[EMAIL PROTECTED]> wrote: > Hello Nemesis, > >> Hi all, I'm trying to write a multiplatform function that tries to >> return the actual user home directory. >> ... > What's wrong with: >from user import home > which does about what your code does. Except it also execfile()s $HOME/.pythonrc.py, which might not be wanted. Saluton Marc -- http://mail.python.org/mailman/listinfo/python-list
List mapping question
I have a number of variables that I want to modify (a bunch of strings that I need to convert into ints). Is there an easy way to do that other than saying: > a = int(a) > b = int(b) > c = int(c) I tried > [i = int(i) for i in [a, b, c]] but that didn't work because it was creating a list with the values of a, b and c instead of the actual variables themselves, then trying to set a string equal to an integer, which it really didn't like. Marc -- http://mail.python.org/mailman/listinfo/python-list
Re: List mapping question
Steve Holden wrote: Marc Huffnagle wrote: I have a number of variables that I want to modify (a bunch of strings that I need to convert into ints). Is there an easy way to do that other than saying: > a = int(a) > b = int(b) > c = int(c) I tried > [i = int(i) for i in [a, b, c]] but that didn't work because it was creating a list with the values of a, b and c instead of the actual variables themselves, then trying to set a string equal to an integer, which it really didn't like. Marc >>> a,b,c = 1.1, 2.2, 3.3 >>> a,b,c = map(int, (a,b,c)) >>> a,b,c (1, 2, 3) >>> a,b,c = [int(x) for x in (a,b,c)] >>> a,b,c (1, 2, 3) regards Steve Thanks ... so there's no way to pass an actual variable into a list mapping, instead of its value? I guess I'm thinking of something the equivalent of call by reference. Marc -- http://mail.python.org/mailman/listinfo/python-list
Re: how to use more than 1 __init__ constructor in a class ?
Singletoned wrote: > Rocco Moretti wrote: > > Steven D'Aprano wrote: > > > > That's the joys of a mostly self-taught programming knowledge: you miss > > > out on all the buzzwords. > > > > Being mostly self taught myself, I have a tendancy to use infrequently > > encountered terms in related but technically inappropriate contexts, > > confusing the better informed people I deal with. ;-) > > Indeed. I find I use even more buzzwords because I can just make up as > many as I want. This thread 'branch' (humm, is this an appropriate term for the last few quotes, going to Steven's?) is soothing in reminding us we are not alone. That there is a sort of distributed 'Alma Mater' of the 'Teach-It-Yourself School of Computing', producing a virtual FOAF group (Is FOAF, Friend Of A Friend or Flock Of A Feather?) jm -- http://mail.python.org/mailman/listinfo/python-list
Thanks for PIL (and other stuff)
I was just reading on daily-python that PIL is 10 years old... So I wish it and its author(s) a good day, week, month, year and more! Really! Jean-Marc PS If I knew that Python had a anniversary date, I'd also write to thanks our BDFL (and authors)! But no such luck, so I'm restaining myself! ;-)) -- http://mail.python.org/mailman/listinfo/python-list
Re: SOAP and XMLRPC
why isn't this good? http://www.enappsys.com/backend.jsp Seems to be what you're looking for... (second entry of a googled 'xml-rpc visual basic' search!) JM PS Tell us why the refered *.dll don't do, so I won't refer to it again if it's of no value. -- http://mail.python.org/mailman/listinfo/python-list
Re: seeking Python developers
What level? and is geography important? JM -- http://mail.python.org/mailman/listinfo/python-list
Re: python classes taught
Cegep du Vieux Montreal (technical college level), uses Python for CGI in web developement class. ...At least when I give this course ;-) Jean-Marc -- http://mail.python.org/mailman/listinfo/python-list
RE: RE Despair - help required
Hi, > Don't think it will do much good. I need to get them from a file and > extract the last folder in the path. For example: > if I get "c:\dos\util" > I want to extract the string "\util" Still, os.path is your friend: import os filepath = r'C:\dos\util' base, last = os.path.split(filepath) print base # 'C:\dos' print last # 'util' print os.sep+last # '\util' Don't forget to read > > http://docs.python.org/lib/module-os.path.html for some more info! Regards, Mc! -- http://mail.python.org/mailman/listinfo/python-list
RE: What is the role of F in dict(E, **F)
> As I understand update() is used to merge two dictionaries,
> for example
> >>> D={'a':1, 'b':2}
> >>> E={'b':4,'c':6}
> >>> D.update(E)
> >>> D
> {'a': 1, 'c': 6, 'b': 4}
> >>>
>
> But what is the role of **F?
>>> D={'a':1, 'b':2}
>>> E={'b':4,'c':6}
>>> D.update(E, a=3)
>>> D
{'a': 3, 'c': 6, 'b': 4}
Ciao, Marc.
--
http://mail.python.org/mailman/listinfo/python-list
Re: Basic file operation questions
When you read a file with that method, is there an implied close() call on the file? I assume there is, but how is that handled? Caleb Hattingh wrote: Peter, that was very clear, thanks. So not only is for line in file(...): # do stuff the most elegant, it is also the fastest. file.readlines() comes close, but is only viable for "small" files. Marc -- http://mail.python.org/mailman/listinfo/python-list
Re: Is Python as capable as Perl for sysadmin work?
Jeff Epler wrote: There's another little-known fact about Python: No string is permitted to end with a backslash! You might think that variations like r'\' or """"""\"""""" would allow you to create this elusive value, but you'd mistaken! Now, this may not bother Unix sysadmins, but the honest truth is that you'll be administrating Windows systems, too, anywhere you work! >>> a = "\\" >>> print a \ Marc -- http://mail.python.org/mailman/listinfo/python-list
Re: Module RE, Have a couple questions
Dasacc
There is a better (faster/easier) way to do it than using the re module,
the find method of the string class.
[EMAIL PROTECTED] wrote:
(1) How do I perform a search for "word" and have it return every line
that this instance is found?
[line for line in document if line.find('a') != -1]
(2) How do I perform a search for "word" and "wordtwo" at the same time
to return every line these instances are found so that the order in
which these lines are in are left intact.
[line for line in document if (line.find('word') != -1 \
and line.find('wordtwo'))]
If there's another standard module more suited for this let me know,
and no I dont want to use sed :)
Thanks
--
http://mail.python.org/mailman/listinfo/python-list
Re: Module RE, Have a couple questions
Oops, made a mistake.
Marc Huffnagle wrote:
Dasacc
There is a better (faster/easier) way to do it than using the re module,
the find method of the string class.
[EMAIL PROTECTED] wrote:
(1) How do I perform a search for "word" and have it return every line
that this instance is found?
[line for line in document if line.find('a') != -1]
(2) How do I perform a search for "word" and "wordtwo" at the same time
to return every line these instances are found so that the order in
which these lines are in are left intact.
[line for line in document if (line.find('word') != -1 \
and line.find('wordtwo'))]
This should have been:
[line for line in document if (line.find('word') != -1 \
and line.find('wordtwo') != -1)]
If there's another standard module more suited for this let me know,
and no I dont want to use sed :)
Thanks
--
http://mail.python.org/mailman/listinfo/python-list
Re: Module RE, Have a couple questions
Francis Girard wrote:
Le mardi 1 Mars 2005 16:52, Marc Huffnagle a écrit :
[line for line in document if (line.find('word') != -1 \
and line.find('wordtwo') != -1)]
Hi,
Using re might be faster than scanning the same line twice :
My understanding of the second question was that he wanted to find lines
which contained both words but, looking at it again, it could go either
way. If he wants to find lines that contain both of the words, in any
order, then I don't think that it can be done without scanning the line
twice (regex or not).
To the OP: What kind of data are you testing? Could you try both of
these solutions on your sample data and let us know which runs faster?
=== begin snap
## rewords.py
import re
import sys
def iWordsMatch(lines, word, word2):
reWordOneTwo = re.compile(r".*(%s|%s).*" % (word,word2))
return (line for line in lines if reWordOneTwo.match(line))
for line in iWordsMatch(open("rewords.py"), "re", "return"):
sys.stdout.write(line)
=== end snap
Furthermore, using list comprehension generator (2.4 only I think) and file
iterator, you can scan files as big as you want with very little memory
usage.
Regards,
Francis Girard
--
http://mail.python.org/mailman/listinfo/python-list
Re: Best way to make a list unique?
Michael Spencer <[EMAIL PROTECTED]> wrote:
Nice. When you replace None by an object(), you have no restriction on
the elements any more:
> Here's something to work with:
>
> class OrdSet(object):
> def __init__(self, iterable):
> """Build an ordered, unique collection of hashable items"""
> #self._data = {None:[None, None]} # None is the pointer to the first
> # # element. This is unsatisfactory
> # # because it cannot then be a
> # # member of the collection
> #self._last = None
self._last = self._root = root = object()
self._data = {root:[root, root]}
> self.update(iterable)
>
> def add(self, obj):
> """Add an element to the collection"""
> data = self._data
> if not obj in data:
> last = self._last
> data[last][1] = obj
> #data[obj] = [last, None]
data[obj] = [last, self._root]
> self._last = obj
>
> def update(self, iterable):
> """Update the collection with the union of itself and another"""
> obj = self._last
> data = self._data
> last = data[obj][0]
> for item in iterable:
> if item not in data:
> data[obj] = [last, item]
> last, obj = obj, item
> #data[obj] = [last, None]
data[obj] = [last, self._root]
> self._last = obj
>
> def remove(self, item):
> """Remove an element from a set; it must be a member.
>
> If the element is not a member, raise a KeyError."""
> data = self._data
> prev, next = data[item]
> data[prev][1] = next
> data[next][0] = prev
>
> def discard(self, item):
> """Remove an element from a set if it is a member.
> If the element is not a member, do nothing."""
> try:
> self.remove(item)
> except KeyError:
> pass
>
> def __contains__(self, item):
> return item in self._data
>
> def pop(self):
> """Remove and the return the oldest element"""
> data = self._data
> #prev, first = data[None]
> #data[None] = [None,data[first][1]]
root = self._root
prev, first = data[root]
data[root] = [root,data[first][1]]
> return first
>
> def clear(self):
> self.__init__([])
>
> def __iter__(self):
> """Iterate over the collection in order"""
> data = self._data
> #prev, next = data[None]
> #while next is not None:
root = self._root
prev, next = data[root]
while next is not root:
> yield next
> prev, next = data[next]
>
> def __len__(self):
> return len(self._data)-1
>
> def __repr__(self):
> return "%s(%s)" % (self.__class__.__name__,list(self))
>>> a=OrdSet([None,1,None,3])
>>> a
OrdSet([None, 1, 3])
Marc
--
http://mail.python.org/mailman/listinfo/python-list
Huge performance gain compared to perl while loading a text file in a list ...!?
Hello,
I recently converted one of my perl scripts to python. What the script
does is simply search a lot of big mail files (~40MB) to retrieve
specific emails. I simply converted the script line by line to python,
keeping the algorithms & functions as they were in perl (no
optimization). The purpose was mainly to learn python and see the
differences with perl.
Now, once the converted script was finished, I was amazed to find that
the python version is running 8 times faster (800% faster!). Needless
to say, I was very intrigued and wanted to know what causes such a
performance gap between the two versions. So to keep my story short,
after some research and a few tests, I found that file IO is mainly
the cause of the performance diff.
I made two short test scripts, one in perl and one in python (see
below), and compared the performance difference. As we can see, the
bigger the file the larger the difference in performance
I'm fairly new to python, and don't know much of its inner working so
I wonder if someone could explain to me why it is so much faster in
python to open a file and load it in a list/array ?
Thanks
-
#!/usr/bin/python
for i in range(20):
Data = open('data.test').readlines()
-
#!/usr/bin/perl
for ($i = 0; $i < 20; $i++) {
open(DATA, "data.test");
@Data = ;
close(DATA);
}
-
Running tests (data.test = 10MB text file):
[EMAIL PROTECTED] blop $ time ./ftest.py
real0m6.408s
user0m4.552s
sys 0m1.826s
[EMAIL PROTECTED] blop $ time ./ftest.pl
real0m22.855s
user0m21.946s
sys 0m0.822s
-
Running tests (data.test = 40MB text file):
[EMAIL PROTECTED] blop $ time ./ftest.py
real0m26.235s
user0m18.238s
sys 0m7.872s
[EMAIL PROTECTED] blop $ time ./ftest.pl
real3m26.741s
user3m22.168s
sys 0m3.764s
--
http://mail.python.org/mailman/listinfo/python-list
RE: comment out more than 1 line at once?
> Riko Wichmann wrote: > > > Dear all, > > > > is there a way in Python to comment out blocks of code > without putting > > a # in front of each line? Somethings like C's > > > > /* > > block of code here is commented out > > */ Well, you could fake it by doing """ block of code here is commented out """ which will work most of the time... Cheerio, Marc. -- http://mail.python.org/mailman/listinfo/python-list
Re: ANNOUNCE: Ice 2.0 released
Interesting to see this blend of GPL and an alternative for closed-source software. Not totally unrelated, I saw this in your web-site (Ice vs CORBA page): No "Design by Committee" Ice was designed by a small group of dedicated and highly experienced people. Am interested to know, what "percentage" (*) of the code in your CVS repository has been contributed by people other than the group mentioned in the quote above? Obviously, you do not allow anonymous CVS write access. Perhaps, one wishing to improve Ice (a freedom granted by GPL) and who does not work for ZeroC has to mail his/her improvements to your maintainers? (*): Percentage is a very nebulous term, I know. For purposes of answering the question, maybe you could resort to the not-highly-meaningful number of LOC, and perhaps a word or two about how Ice benefited from it. 100% of the Ice source code has been developed by ZeroC employees. Note that this does of course not apply for third-party code that is being used by Ice, such as BZIP2, Berkeley DB, OpenSSL, etc. - Anand PS: Please feel free to set FU-Ts as appropriate. What are FU-Ts? -- Marc -- http://mail.python.org/mailman/listinfo/python-list
Re: ANNOUNCE: Ice 2.0 released
Consider the *hypothetical* situation where an individual or a group of people re-write large portions of Ice. This could enhance the value of Ice (obviously to some, if not all), or this could conflict with the ideologies of Ice (again, not in everyone's point of view). How will ZeroC react to this? Everybody can enhance or modify Ice, we don't have any problems with this whatsoever. The GPL explicitly allows you to do so. However, this does not mean that we have to take over these changes or additions into our Ice distribution. In many cases, this would also not be necessary, as the most likely contribution would be in form of plug-ins or services. As an aside, you are completely free to use any license, if you write your own implementation of the Ice protocol or specification language. Neither the protocol nor the specification language are patented. So while our own implementation is available under GPL or a commercial license only, you could write a new implementation under a license of your choosing. We don't have any problems with this either. In fact, we encourage it, otherwise we wouldn't have documented our protocol so carefully. I believe whichever road you take, ZeroC is going to find itself in problems. If ZeroC merges the changes made by this/these person(s), how can ZeroC now sell it under a commercial license, as closed source (violation of GPL)? If you refuse to merge the changes, you have just given them a strong impetus to fork. History shows XEmacs and EGCS as two such examples. I don't see any problems. If we merge a contribution into our Ice distribution, then we need to reach an agreement with the contributor as to how we can handle non-GPL licenses. If no agreement is reached, then we cannot merge this contribution into our Ice distribution. Guess what I am primarily interested in finding out is rooted in what I said earlier: Interesting to see this blend of GPL and an alternative for closed-source software. What were the ideas behind going the GPL way? How did ZeroC plan on benefiting from it? Were there any qualms within ZeroC in going GPL? The idea is simple: Ice should be free for open-source applications, but if somebody wants to use Ice for a closed-source application, then we want a fair share of the revenue. So far this works quite well :) No, there were no qualms within ZeroC with using the GPL, we all pretty much agreed from the start that this is a reasonable licensing model. Note that we are not the only one who use such a dual-licensing scheme. For example, if you want to use Berkeley DB (an excellent embedded database) for a closed-source project, then you also have to buy a commercial license. (They don't use GPL as their open-source license, but something that is similar to the GPL.) Note that I am not saying GPL and commercial software don't go together (I do believe though that LGPL and commercial software don't go together). I am well aware of Free software being "Free speech, not free beer". What if you did not provide Ice as a free download, but a price based on your current licensing policy(*). However, the download would give one the complete source, and the freedom to modify and redistribute it (at whatever price so long as the complete source code with the GPL notice is released). (*): All of this is hypothetical. Am not making a business proposition here. I'm not sure I understand what you are suggesting. You want us to charge for a GPL download? I don't think this makes sense, a GPL download should be free. You do not, because that would discourage Ice from becoming ubiquitous, from paving way for becoming a potential de-facto standard. Then, why not simply advertise Ice as being commercial (with unlimited free trial plus source code)? Doing so, would get you the extensive peer review that you are currently benefitting from. What do you gain by going GPL? The freedom to modify and/or redistribute is (apparently) pretty restricted anyway. We are quite happy with our licensing model, and many of our users use Ice under GPL. I neither see the need to restrict nor to loosen our licensing terms in any way. PS: Please feel free to set FU-Ts as appropriate. What are FU-Ts? "Follow-up To:". Most news clients will allow sending a post to multiple groups, restricting any possible responses to certain groups alone. A poster who is replying can over-ride it, of course. Thanks for the explanation. I learn something new every day :) -- Marc -- http://mail.python.org/mailman/listinfo/python-list
Re: ANNOUNCE: Ice 2.0 released
apm wrote: Marc Laukien <[EMAIL PROTECTED]> wrote in message news:<[EMAIL PROTECTED]>... 100% of the Ice source code has been developed by ZeroC employees. Fixes, bug reports, and enhancement requests have come in from Open Source developers around the world, as can be seen from the forums on the ZeroC web site. This is of course correct, but I do not think this is in contradiction to what I stated. The submission of a bug report or an enhancement request does not constitute development of source code. Neither does the submittal of a bug fix, if there is only one way to fix a bug. -- Marc -- http://mail.python.org/mailman/listinfo/python-list
Re: cookie lib policy how-tp?
Riko Wichmann <[EMAIL PROTECTED]> wrote:
> Jonathan Ellis wrote:
>>
>> Sounds like your first step should be to identify yourself as IE.
>>
>> opener = urllib2.build_opener(...)
>> opener.addheaders = [("User-Agent", "whatever IE calls itself these
>> days")]
>>
>> -Jonathan
>>
>
> Tried that already. At least, I hope I guessed at least one of the
> possible identifiers correct: MSIE6.0, MSIE 6.0, MSIE/6.0
When my opera is set to identify as MSIE, it sends
"Mozilla/4.0 (compatible; MSIE 6.0; X11; Linux i686) Opera 7.54 [en]".
Saluton
Marc
--
http://mail.python.org/mailman/listinfo/python-list
Re: cookie lib policy how-tp?
Riko Wichmann <[EMAIL PROTECTED]> wrote: >>>Tried that already. At least, I hope I guessed at least one of the >>>possible identifiers correct: MSIE6.0, MSIE 6.0, MSIE/6.0 >> >> >> When my opera is set to identify as MSIE, it sends >> "Mozilla/4.0 (compatible; MSIE 6.0; X11; Linux i686) Opera 7.54 [en]". > > Hi Marc, > > thanks for the hint! that brought me a big step forward! You're welcome :) Marc -- http://mail.python.org/mailman/listinfo/python-list
RE: Why are tuples immutable?
> def __cmp__ (self, other): > # I wish there was a less verbose way to do this! > if self.block < other.block: > return -1 > if self.block > other.block: > return 1 > if self.lot < other.lot: > return -1 > if self.lot > other.lot: > return 1 > return 0 A little less verbose: def __cmp__ (self, other): compared = cmp(self.block, other.block) if not compared: compared = cmp(self.lot, other.lot) return compared HTH, Marc. -- http://mail.python.org/mailman/listinfo/python-list
Implement logic on object.attribute and object.attribute()
Hi, I'm playing with python internals to make objects behave like this: if I access to "object.attribute" I want to return the result of an HTTP GET request. However if i call "object.attribute()" I want an HTTP POST request to be executed. So far I have been able to do the POST part, using two classes like this: class HTTPAttribute(object): def __init__(self, name): self.name = name def __call__(self, *args, **kwargs): url = BASE_URL + self.name requests.post(url, *args, **kwargs) class HTTPObject(object): def __getattr__(self, name): return HTTPAttribute(name) But I'm stuck implementing the HTTP GET request when accessing "object.attribute", Any idea ?? Thanks!! -- Marc -- https://mail.python.org/mailman/listinfo/python-list
Re: Implement logic on object.attribute and object.attribute()
On Sun, Nov 24, 2013 at 2:45 PM, Steven D'Aprano wrote: > On Sun, 24 Nov 2013 05:04:16 -0800, Devin Jeanpierre wrote: > >> On Sun, Nov 24, 2013 at 4:52 AM, Marc Aymerich >> wrote: >>> Hi, >>> I'm playing with python internals to make objects behave like this: >>> >>> if I access to "object.attribute" I want to return the result of an >>> HTTP GET request. However if i call "object.attribute()" I want an HTTP >>> POST request to be executed. >> >> Uh oh. What you want is impossible. You cannot call an attribute without >> first accessing it. :( > > Not quite impossible. All you need is an object that behaves like a > string, except it has a __call__ method. Here's a sketch of a solution, > completely untested. > > > class CallableString(str): > # Like a string, but callable. > def function(self): > raise NotImplementedError( > "this must be overridden on the instance" > ) > def __call__(self): > return self.function() > > > class Magic_HTTP_Thing: > @property > def attribute(self): > result = CallableStr(self.do_get()) > result.function = lambda: self.do_put() > def do_get(self): > # Do a HTTP GET request. > return "Get stuff" > def do_put(self): > # Do a HTTP PUT request. > return "Put stuff" OMG steven, it actually works :) >>> class CallableString(str): ... # Like a string, but callable. ... def function(self): ... raise NotImplementedError( ... "this must be overridden on the instance" ... ) ... def __call__(self): ... return self.function() ... >>> >>> class Magic_HTTP_Thing: ... @property ... def attribute(self): ... result = CallableString(self.do_get()) ... result.function = lambda: self.do_put() ... return result ... def do_get(self): ... # Do a HTTP GET request. ... return "Get stuff" ... def do_put(self): ... # Do a HTTP PUT request. ... return "Put stuff" ... >>> >>> Magic_HTTP_Thing().attribute 'Get stuff' >>> Magic_HTTP_Thing().attribute() 'Put stuff' > > Possible or not, it doesn't seem like a reasonable API to me. yeah, this is a "corner case" of our REST API, I have some badly design endpoints that mostly behave like functions, but some of them also contain state information that you can GET, I was trying to map this behavior to python objects and this interface is the best that occurred to me :) -- Marc -- https://mail.python.org/mailman/listinfo/python-list
Re: Implement logic on object.attribute and object.attribute()
On Sun, Nov 24, 2013 at 3:13 PM, Chris Angelico wrote: > On Mon, Nov 25, 2013 at 12:45 AM, Steven D'Aprano > wrote: >> Not quite impossible. All you need is an object that behaves like a >> string, except it has a __call__ method. Here's a sketch of a solution, >> completely untested. >> >> class Magic_HTTP_Thing: >> @property >> def attribute(self): >> result = CallableStr(self.do_get()) >> result.function = lambda: self.do_put() > > Problem with that is that it'll still call do_get immediately. You'd > have to somehow defer this call until it's actually _used_, which is > why I dropped a mention of "converting to str?" - which would > presumably be a __str__ method. But I still don't like the API. That's right. In my case deferring the GET call will not be a problem since this objects will be used in just a few particular places and the workflow is always something like: # Initiate firmware building node.ctl.firmware() # wait until finished while node.ctl.firmware.progress < 100: time.sleep(1) Thanks for sharing your knowledge guys !! -- Marc -- https://mail.python.org/mailman/listinfo/python-list
Re: Implement logic on object.attribute and object.attribute()
On Sun, Nov 24, 2013 at 3:37 PM, Chris Angelico wrote:
> On Mon, Nov 25, 2013 at 1:16 AM, Marc Aymerich wrote:
>> ... def do_get(self):
>> ... # Do a HTTP GET request.
>> ... return "Get stuff"
>> ... def do_put(self):
>> ... # Do a HTTP PUT request.
>> ... return "Put stuff"
>
> To make this a bit more realistic, try this instead - tying in with
> what I said in response to Roy:
>
> class CallableString(str):
> # Like a string, but callable.
> def function(self):
> raise NotImplementedError(
> "this must be overridden on the instance"
> )
> def __call__(self):
> return self.function()
>
>
> class Magic_HTTP_Thing:
> @property
> def attribute(self):
> result = CallableString(self.do_get())
> result.function = lambda: self.do_put()
> return result
> def do_get(self):
> # Do a HTTP GET request.
> print("Doing the GET call, please wait...")
> time.sleep(1)
> return "Get stuff"
> def do_put(self):
> # Do a HTTP PUT request.
> print("Doing the PUT call, please wait...")
> time.sleep(1)
> return "Put stuff"
>
> (PUT or POST, makes no difference; I think you were looking for POST,
> but Steven wrote PUT here so I'll use that for simplicity)
>
>>>> Magic_HTTP_Thing().attribute()
> Doing the GET call, please wait...
> Doing the PUT call, please wait...
> 'Put stuff'
>
> And that's what you don't want happening. Your PUT / POST calls are
> going to take twice as long as they should.
Thanks Chris,
didn't realize about the implicit GET when calling the attribute :(
I think I'll put the get call on __repr__, __str__ and __getattr__,
something like
class HTTPAttribute(object):
""" functional endpoint representation """
def __repr__(self):
self._retrieve()
return json.dumps(self.__dict__)
def __str__(self):
self._retrieve()
return json.dumps(self.__dict__)
def __init__(self, resource, uri):
self._resource = resource
self.uri = uri
def __call__(self, *args, **kwargs):
return self._resource._api.post(self.uri, *args, **kwargs).content
def __getattr__(self, name):
self._retrieve()
if hasattr(self, name):
return getattr(self, name)
raise AttributeError("'%s' object has no attribute '%s'" %
(str(type(self)), name))
def _retrieve(self):
resource = self._resource._api.retrieve(self.uri)
for key, value in resource._data.iteritems():
setattr(self, key, value)
and that's it,
But still I'll reconsider an interface with less magic :P
--
Marc
--
https://mail.python.org/mailman/listinfo/python-list
Re: Highest performance HTTP REST microframework?
On Sat, Jan 4, 2014 at 5:26 AM, Alec Taylor wrote: > What is the highest performance REST microframework? > > Happy if it's mostly written in C or C++; as long as it provides a > simple routes interface in Python. > > Currently using bottle and utilising its application, @route and > app.merge(app2) extra features. The biggest performance gains on HTTP architectures are usually made by doing proper HTTP caching. Without knowing anything about your architecture is hard to tell something more specific :) -- Marc -- https://mail.python.org/mailman/listinfo/python-list
Re: Graph or Chart Software for Django
On Fri, Jan 17, 2014 at 6:18 PM, San D wrote: > What is the best Graph or Chart software used with Django (libraries & > products), preferably open source? > > Need something stable and robust, and used by many developers, so active on > community channels. what I use is a JS library called highcharts, the django-side are only views that dump some data in json format perhaps in django-users mailing list you'll be able to find better answers than here :) -- Marc -- https://mail.python.org/mailman/listinfo/python-list
Re: Templating engines that work very well with Python/Django
On Fri, Jan 17, 2014 at 6:22 PM, San D wrote: > Can someone suggest a few templating engines that work really well with > Django and help in coding efficiency? > > Where can I fin comparison of tempating engines I can find to know their pros > and cons? I believe the most widely used template engine in Django is "The Django template language" itself :) https://docs.djangoproject.com/en/dev/topics/templates/ However the cool kids are always talking about switching to jinja2, a simple google search like jinja2 django could be a good starting point for finding good comparatives between both. -- Marc -- https://mail.python.org/mailman/listinfo/python-list
Why this throws an UnboundLocalError ?
Dear all, I have a very simple module glic3@e4200:# cat globalstate.py GLOBAL = 0 def update(): GLOBAL += 1 however it doesn't work!! glic3@e4200:# python Python 2.7.3 (default, Aug 1 2012, 05:14:39) [GCC 4.6.3] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import globalstate >>> globalstate.update() Traceback (most recent call last): File "", line 1, in File "globalstate.py", line 4, in update GLOBAL += 1 UnboundLocalError: local variable 'GLOBAL' referenced before assignment And I don't know why :( Anyone ? Thanks!! -- Marc -- https://mail.python.org/mailman/listinfo/python-list
Re: Why this throws an UnboundLocalError ?
On Thu, Jan 30, 2014 at 11:53 PM, Chris Angelico wrote: > On Fri, Jan 31, 2014 at 9:46 AM, Marc Aymerich wrote: >> GLOBAL = 0 >> >> def update(): >> GLOBAL += 1 > > If you assign to a name, Python makes it local, unless you explicitly > tell it that you want it to be global: > > def update(): > global GLOBAL > GLOBAL += 1 > > But be aware that the ALL_CAPS name conventionally means a constant. > Since you're changing its value, it's not constant (wow! :) ), so > using a name of GLOBAL is a bad idea. (Also, obviously, you want to > name it appropriately to what you're doing, but I assume you called it > this as part of cutting down your example. For which, by the way, > thank you. You posted a complete example, and the full traceback, and > the Python version and platform. That's everything that we need to > help you - it's such a luxury!!) > Thank you very much guys! and for the additional tips Chris. :) I can't believe, all these years using Python and never encountered a situation where I needed to use global ! -- Marc -- https://mail.python.org/mailman/listinfo/python-list
Futex hang when running event loop on a separated thread
Hi,
I have to run the asyncio.loop on a separated thread because the main
thread is running FUSE. Apparently fuse needs to run on the main
thread because it uses signal():
File "/usr/local/lib/python3.4/dist-packages/fuse.py", line 390, in __init__
old_handler = signal(SIGINT, SIG_DFL)
ValueError: signal only works in main thread
Anyway, when I exit the program it appears that i run into a deadlock
with the eventloop thread, strace is stuck with:
futex(0x7f7bac10, FUTEX_WAIT_PRIVATE, 0, NULL
I've tried to stop the event loop from the main thread but the
situation is exactly the same :(
loop_container = {}
handler = threading.Thread(target=run_loop, args=(loop_container,))
try:
handler.start()
FUSE(fs)
finally:
loop_container['loop'].stop()
# handler.join()
Any idea on how I can shutdown the hole thing? I have to manually kill
the program each time :(
--
Marc
--
https://mail.python.org/mailman/listinfo/python-list
Re: Futex hang when running event loop on a separated thread
On Tue, Nov 24, 2015 at 4:29 PM, Marc Aymerich wrote:
> Hi,
>
> I have to run the asyncio.loop on a separated thread because the main
> thread is running FUSE. Apparently fuse needs to run on the main
> thread because it uses signal():
>
>
> File "/usr/local/lib/python3.4/dist-packages/fuse.py", line 390, in __init__
> old_handler = signal(SIGINT, SIG_DFL)
> ValueError: signal only works in main thread
>
>
> Anyway, when I exit the program it appears that i run into a deadlock
> with the eventloop thread, strace is stuck with:
>
>
> futex(0x7f7bac10, FUTEX_WAIT_PRIVATE, 0, NULL
>
>
> I've tried to stop the event loop from the main thread but the
> situation is exactly the same :(
>
> loop_container = {}
> handler = threading.Thread(target=run_loop, args=(loop_container,))
> try:
>handler.start()
>FUSE(fs)
> finally:
> loop_container['loop'].stop()
> # handler.join()
>
> Any idea on how I can shutdown the hole thing? I have to manually kill
> the program each time :(
this can be reproduced with the following program:
import asyncio
import threading
import time
def run_loop(loop_container):
loop = asyncio.new_event_loop()
asyncio.set_event_loop(loop)
coro = asyncio.start_server(lambda: 1, '0.0.0.0', , loop=loop)
server = loop.run_until_complete(coro)
loop_container['loop'] = loop
loop.run_forever()
if __name__ == '__main__':
loop_container = {}
handler = threading.Thread(target=run_loop, args=(loop_container, ))
handler.start()
try:
time.sleep(1)
finally:
loop_container['loop'].stop()
glic3@XPS13:/tmp$ python3 /tmp/eventtest.py
^CTraceback (most recent call last):
File "/tmp/eventtest.py", line 22, in
time.sleep(1)
KeyboardInterrupt
^CException ignored in:
Traceback (most recent call last):
File "/usr/lib/python3.4/threading.py", line 1294, in _shutdown
t.join()
File "/usr/lib/python3.4/threading.py", line 1060, in join
self._wait_for_tstate_lock()
File "/usr/lib/python3.4/threading.py", line 1076, in _wait_for_tstate_lock
elif lock.acquire(block, timeout):
KeyboardInterrupt
it is waiting for tstate_lock in the second ^C
--
Marc
--
https://mail.python.org/mailman/listinfo/python-list
Re: Futex hang when running event loop on a separated thread
threading supports the 'daemon' option[1], when set to True and the
program is in *foreground* then the event-loop thread dies when
SIGTERM-ed, however, if the program is in the *background* it doesn't
work! still deadlocked :'''(
while I'm not finding a definitive solution I'll be doing a
os.kill(os.getpid(), signal.SIGKILL) inside the finally block.
[1] https://docs.python.org/3.5/library/threading.html#threading.Thread.daemon
On Tue, Nov 24, 2015 at 4:46 PM, Marc Aymerich wrote:
> On Tue, Nov 24, 2015 at 4:29 PM, Marc Aymerich wrote:
>> Hi,
>>
>> I have to run the asyncio.loop on a separated thread because the main
>> thread is running FUSE. Apparently fuse needs to run on the main
>> thread because it uses signal():
>>
>>
>> File "/usr/local/lib/python3.4/dist-packages/fuse.py", line 390, in __init__
>> old_handler = signal(SIGINT, SIG_DFL)
>> ValueError: signal only works in main thread
>>
>>
>> Anyway, when I exit the program it appears that i run into a deadlock
>> with the eventloop thread, strace is stuck with:
>>
>>
>> futex(0x7f7bac10, FUTEX_WAIT_PRIVATE, 0, NULL
>>
>>
>> I've tried to stop the event loop from the main thread but the
>> situation is exactly the same :(
>>
>> loop_container = {}
>> handler = threading.Thread(target=run_loop, args=(loop_container,))
>> try:
>>handler.start()
>>FUSE(fs)
>> finally:
>> loop_container['loop'].stop()
>> # handler.join()
>>
>> Any idea on how I can shutdown the hole thing? I have to manually kill
>> the program each time :(
>
>
> this can be reproduced with the following program:
>
> import asyncio
> import threading
> import time
>
> def run_loop(loop_container):
> loop = asyncio.new_event_loop()
> asyncio.set_event_loop(loop)
> coro = asyncio.start_server(lambda: 1, '0.0.0.0', , loop=loop)
> server = loop.run_until_complete(coro)
> loop_container['loop'] = loop
> loop.run_forever()
>
> if __name__ == '__main__':
> loop_container = {}
> handler = threading.Thread(target=run_loop, args=(loop_container, ))
> handler.start()
> try:
> time.sleep(1)
> finally:
> loop_container['loop'].stop()
>
>
> glic3@XPS13:/tmp$ python3 /tmp/eventtest.py
> ^CTraceback (most recent call last):
> File "/tmp/eventtest.py", line 22, in
> time.sleep(1)
> KeyboardInterrupt
> ^CException ignored in: '/usr/lib/python3.4/threading.py'>
> Traceback (most recent call last):
> File "/usr/lib/python3.4/threading.py", line 1294, in _shutdown
> t.join()
> File "/usr/lib/python3.4/threading.py", line 1060, in join
> self._wait_for_tstate_lock()
> File "/usr/lib/python3.4/threading.py", line 1076, in _wait_for_tstate_lock
> elif lock.acquire(block, timeout):
> KeyboardInterrupt
>
> it is waiting for tstate_lock in the second ^C
>
> --
> Marc
--
Marc
--
https://mail.python.org/mailman/listinfo/python-list
Re: Futex hang when running event loop on a separated thread
On Tue, Nov 24, 2015 at 7:11 PM, Zachary Ware
wrote:
> On Tue, Nov 24, 2015 at 9:46 AM, Marc Aymerich wrote:
>> if __name__ == '__main__':
>> loop_container = {}
>> handler = threading.Thread(target=run_loop, args=(loop_container, ))
>> handler.start()
>> try:
>> time.sleep(1)
>> finally:
>> loop_container['loop'].stop()
>
> loop.stop() must be called from the thread running the loop. You can
> do this by doing
> `loop_container['loop'].call_soon_threadsafe(loop_container['loop'].stop)`
> from the main thread (untested).
>
Hi Zachary,
nice to know about call_soon_threadsafe!
still it appears to work only if the main thread is in the foreground
(as of calling Thread() with deamon=True), I don't get why it behaves
differently :( maybe it is waiting for other stuff, but no idea how to
confirm this with strace of other means.. i always see the same
'futex(0x7f9a7c10, FUTEX_WAIT_PRIVATE, 0, NULL'
--
Marc
--
https://mail.python.org/mailman/listinfo/python-list
Re: Futex hang when running event loop on a separated thread
On Tue, Nov 24, 2015 at 8:41 PM, Zachary Ware
wrote:
> On Tue, Nov 24, 2015 at 12:37 PM, Marc Aymerich wrote:
>> still it appears to work only if the main thread is in the foreground
>> (as of calling Thread() with deamon=True), I don't get why it behaves
>> differently :( maybe it is waiting for other stuff, but no idea how to
>> confirm this with strace of other means.. i always see the same
>> 'futex(0x7f9a7c10, FUTEX_WAIT_PRIVATE, 0, NULL'
>
> What's your exact version of Python? asyncio is still evolving
> rapidly, and may behave differently between patch releases (say,
> between 3.4.1 and 3.4.3).
>
> I have tried out your reproducer with my fix, and I can't provoke a
> hang using Python 3.4.3 on either OSX or Ubuntu Trusty. Can you give
> some more specific directions to reproduce?
>
> --
> Zach
> --
> https://mail.python.org/mailman/listinfo/python-list
Yep, I'm a bit lost about how fuse implements going into background...
so the only way I have to reproduce this so far is by using fuse :(
# pip3 install fusepy
import asyncio
import threading
import time
import os
import sys
from fuse import FUSE, Operations
def run_loop(loop_container):
loop = asyncio.new_event_loop()
asyncio.set_event_loop(loop)
coro = asyncio.start_server(lambda: 1, '0.0.0.0', , loop=loop)
server = loop.run_until_complete(coro)
loop_container['loop'] = loop
loop.run_forever()
if __name__ == '__main__':
mountpoint = sys.argv[1]
loop_container = {}
handler = threading.Thread(target=run_loop, args=(loop_container,))
handler.start()
try:
# with foreground=True works
FUSE(Operations(), mountpoint, foreground=False)
finally:
loop_container['loop'].call_soon_threadsafe(loop_container['loop'].stop)
$ mkdir /tmp/
$ python3.5 /tmp/eventtest.py /tmp/
# Another terminal
$ ps aux | grep python3.5
glic319983 0.5 0.1 215540 10772 ?Ssl 21:05 0:00
python3.5 me.py /tmp/
$ sudo strace -p 19983
Process 19983 attached - interrupt to quit
futex(0x7fff42affca0, FUTEX_WAIT_PRIVATE, 0, NULL) = 0
# Here i fire
$ fusermount -u /tmp/
tgkill(19983, 19985, SIGRTMIN) = 0
futex(0x7fff42affc30, FUTEX_WAKE_PRIVATE, 1) = 0
futex(0x7f9840afc9d0, FUTEX_WAIT, 19984, NULL
I've just tried with python3.5 and the same behavior. The thing is
that with foreground=True it just works.
thank you for your time!
--
Marc
--
https://mail.python.org/mailman/listinfo/python-list
Re: Futex hang when running event loop on a separated thread
On Tue, Nov 24, 2015 at 9:17 PM, Marc Aymerich wrote:
> On Tue, Nov 24, 2015 at 8:41 PM, Zachary Ware
> wrote:
>> On Tue, Nov 24, 2015 at 12:37 PM, Marc Aymerich wrote:
>>> still it appears to work only if the main thread is in the foreground
>>> (as of calling Thread() with deamon=True), I don't get why it behaves
>>> differently :( maybe it is waiting for other stuff, but no idea how to
>>> confirm this with strace of other means.. i always see the same
>>> 'futex(0x7f9a7c10, FUTEX_WAIT_PRIVATE, 0, NULL'
>>
>> What's your exact version of Python? asyncio is still evolving
>> rapidly, and may behave differently between patch releases (say,
>> between 3.4.1 and 3.4.3).
>>
>> I have tried out your reproducer with my fix, and I can't provoke a
>> hang using Python 3.4.3 on either OSX or Ubuntu Trusty. Can you give
>> some more specific directions to reproduce?
>>
>> --
>> Zach
>> --
>> https://mail.python.org/mailman/listinfo/python-list
>
>
> Yep, I'm a bit lost about how fuse implements going into background...
> so the only way I have to reproduce this so far is by using fuse :(
>
> # pip3 install fusepy
> import asyncio
> import threading
> import time
> import os
> import sys
> from fuse import FUSE, Operations
>
> def run_loop(loop_container):
> loop = asyncio.new_event_loop()
> asyncio.set_event_loop(loop)
> coro = asyncio.start_server(lambda: 1, '0.0.0.0', , loop=loop)
> server = loop.run_until_complete(coro)
> loop_container['loop'] = loop
> loop.run_forever()
>
> if __name__ == '__main__':
> mountpoint = sys.argv[1]
> loop_container = {}
> handler = threading.Thread(target=run_loop, args=(loop_container,))
> handler.start()
> try:
> # with foreground=True works
> FUSE(Operations(), mountpoint, foreground=False)
> finally:
>
> loop_container['loop'].call_soon_threadsafe(loop_container['loop'].stop)
>
>
> $ mkdir /tmp/
> $ python3.5 /tmp/eventtest.py /tmp/
>
> # Another terminal
> $ ps aux | grep python3.5
> glic319983 0.5 0.1 215540 10772 ?Ssl 21:05 0:00
> python3.5 me.py /tmp/
> $ sudo strace -p 19983
> Process 19983 attached - interrupt to quit
> futex(0x7fff42affca0, FUTEX_WAIT_PRIVATE, 0, NULL) = 0
>
> # Here i fire
> $ fusermount -u /tmp/
>
> tgkill(19983, 19985, SIGRTMIN) = 0
> futex(0x7fff42affc30, FUTEX_WAKE_PRIVATE, 1) = 0
> futex(0x7f9840afc9d0, FUTEX_WAIT, 19984, NULL
>
>
> I've just tried with python3.5 and the same behavior. The thing is
> that with foreground=True it just works.
> thank you for your time!
In case this could be of some use:
strace using foreground=True
$ sudo strace -p 20326
Process 20326 attached - interrupt to quit
futex(0x7fffd9b8e5a0, FUTEX_WAIT_PRIVATE, 0, NULL
# Here umount command
) = 0
tgkill(20326, 20334, SIGRTMIN) = 0
tgkill(20326, 20335, SIGRTMIN) = 0
futex(0x7fffd9b8e530, FUTEX_WAKE_PRIVATE, 1) = 0
rt_sigaction(SIGHUP, NULL, {0x7fb60d850540, [], SA_RESTORER,
0x7fb61106ccb0}, 8) = 0
rt_sigaction(SIGINT, NULL, {0x7fb60d850540, [], SA_RESTORER,
0x7fb61106ccb0}, 8) = 0
rt_sigaction(SIGTERM, NULL, {0x7fb60d850540, [], SA_RESTORER,
0x7fb61106ccb0}, 8) = 0
rt_sigaction(SIGPIPE, NULL, {SIG_IGN, [], SA_RESTORER, 0x7fb61106ccb0}, 8) = 0
poll([{fd=5, events=0}], 1, 0) = 1 ([{fd=5, revents=POLLERR}])
close(5)= 0
brk(0x126d000) = 0x126d000
rt_sigaction(SIGINT, {0x50b380, [], SA_RESTORER, 0x7fb61106ccb0},
{0x7fb60d850540, [], SA_RESTORER, 0x7fb61106ccb0}, 8) = 0
sendto(8, "\0", 1, 0, NULL, 0) = 1
futex(0x7fb61740, FUTEX_WAKE_PRIVATE, 1) = 1
futex(0x9d3204, FUTEX_WAKE_OP_PRIVATE, 1, 1, 0x9d3200, {FUTEX_OP_SET,
0, FUTEX_OP_CMP_GT, 1}) = 1
futex(0x9d31c0, FUTEX_WAKE_PRIVATE, 1) = 1
futex(0x7fb60c10, FUTEX_WAIT_PRIVATE, 0, NULL) = 0
rt_sigaction(SIGINT, {SIG_DFL, [], SA_RESTORER, 0x7fb61106ccb0},
{0x50b380, [], SA_RESTORER, 0x7fb61106ccb0}, 8) = 0
epoll_ctl(4, EPOLL_CTL_DEL, 7,
{EPOLLWRBAND|EPOLLMSG|EPOLLERR|0xdea9800, {u32=32694,
u64=23586238107778998}}) = 0
close(7)= 0
close(8)= 0
close(4)= 0
getsockname(6, {sa_family=AF_INET, sin_port=htons(),
sin_addr=inet_addr("0.0.0.0")}, [16]) = 0
getpeername(6, 0x7fffd9b8dc70, [16])= -1 ENOTCONN (Transport
endpoint is not connected)
close(6)= 0
madvise(0x7fb600033000, 8192, MADV_DONTNEED) = 0
close(3)= 0
exit_group(0) = ?
Process 20326 detached
--
Marc
--
https://mail.python.org/mailman/listinfo/python-list
