How can I intentionally crash my lappy?

[mechtheist]

I am no programmer, but know the rudiments [the rudi'est of rudiments] 
of working with Python.  I have a simple need,  to have a simple 
script/app I can run that will crash my PC.  On my desktops, I can 
always hit the reset, but that is not an option with my laptop.  Can 
anyone tell me of an easy way to guarantee a program will instantly 
kill/BSOD my windows7/64bit laptop?


Any help is much appreciated.
--
http://mail.python.org/mailman/listinfo/python-list

Dynamic list name from a string

[MarcoS]

Hi, I need to create a list with a dynamic name, something like this:

'%s_list' %my_dynamic list = []

It's this possible?
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: How can I intentionally crash my lappy?

[Steven D'Aprano]

On Mon, 20 Dec 2010 02:45:18 -0600, mechtheist wrote:

> I am no programmer, but know the rudiments [the rudi'est of rudiments]
> of working with Python.  I have a simple need,  to have a simple
> script/app I can run that will crash my PC.  On my desktops, I can
> always hit the reset, but that is not an option with my laptop.  Can
> anyone tell me of an easy way to guarantee a program will instantly
> kill/BSOD my windows7/64bit laptop?
> 
> Any help is much appreciated.

You realise that the operating system (yes, even Windows!) is designed to 
*not* crash no matter what the application does?

(The OS may live up to that requirement imperfectly, but still, it tries.)


You'll probably find that task easier from C than from Python. But if I 
were to attempt such a strange thing, I'd try to shut down some critical 
processes used by the OS. I'm not much of a Windows person, but if you 
call up the task manager (ctrl-alt-del) and inspect the process list, 
then find one that causes the machine to crash if you shut it down, you 
can probably do the same thing from a script.



-- 
Steven
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Dynamic list name from a string

[Steven D'Aprano]

On Mon, 20 Dec 2010 02:08:57 -0800, MarcoS wrote:

> Hi, I need to create a list with a dynamic name, something like this:
> 
> '%s_list' %my_dynamic list = []
> 
> It's this possible?

I'm pretty sure you don't *need* to, you just think you do. It is highly 
unlikely that there is anything you can do with such a variable-variable-
name that you can't do by a more sensible technique.

But since you ask:


>>> name = 'variable'
>>> variable
Traceback (most recent call last):
  File "", line 1, in 
NameError: name 'variable' is not defined
>>> globals()[name] = "this is a bad idea, don't do it"
>>> variable
"this is a bad idea, don't do it"


Here's another way:


>>> del variable
>>> exec("%s = 'this is an even worse idea, be careful with this'" % name)
>>> variable
'this is an even worse idea, be careful with this'



Seriously, you probably shouldn't do this. The first is harmful because 
it leads to confusing, complicated, unclear code that is hard to 
maintain; the second is harmful for the same reasons, *plus* it is 
slower, AND could lead to serious security bugs. Google on "code 
injection attack" for more information.



(There are uses for these techniques, or at least similar techniques, but 
they are rare, fairly advanced, and quite specialised.)

-- 
Steven
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: How can I intentionally crash my lappy?

[Stefan Sonnenberg-Carstens]

http://pcsupport.about.com/od/tipstricks/ht/makebsodxp.htm
Am Mo, 20.12.2010, 09:45 schrieb mechtheist:
> I am no programmer, but know the rudiments [the rudi'est of rudiments]
> of working with Python.  I have a simple need,  to have a simple
> script/app I can run that will crash my PC.  On my desktops, I can
> always hit the reset, but that is not an option with my laptop.  Can
> anyone tell me of an easy way to guarantee a program will instantly
> kill/BSOD my windows7/64bit laptop?
>
> Any help is much appreciated.
> --
> http://mail.python.org/mailman/listinfo/python-list
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Dynamic list name from a string

[Andre Alexander Bell]

Hello,

On 12/20/2010 11:08 AM, MarcoS wrote:
> Hi, I need to create a list with a dynamic name, something like this:
> 
> '%s_list' %my_dynamic list = []
> 
> It's this possible?

I would suggest you use a dictionary to store your lists like this:

lists = {}

lists[my_dynamic_list] = []

Maybe you tell us some more what you want to achieve...

Regards


Andre
-- 
http://mail.python.org/mailman/listinfo/python-list

"encoding" attribute codecs.getwriter-produced streams

[Hrvoje Niksic]

Try this code:

# foo.py
import sys, codecs
stream = codecs.getwriter('utf-8')(sys.stdout)
print stream.encoding

$ python foo.py | cat
None

I expected the `encoding' attribute to be "UTF-8", since the stream
otherwise correctly functions as a utf-8 encoding stream.

Is this a bug in the stream factory returned by codecs.getwriter(...)?

If not, is there another way to determine a stream's output encoding
that would work for both default and codecs-created streams?
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Dynamic list name from a string

[MarcoS]

Ok, thanks Steven and Andre for yours reply.

This is my situation:
I've a list of objects of differents classes, with a common attribute
name
Something like this:

class Object1:
obj1_name
 other fields
date

class Object2:
obj2_name
 other fields
date

...

class ObjectN:
objN_name
 other fields
date

the lists are created dynamically at runtime, depending or not if
there one or more correspondig Objects,
ex.
list_object1 = [object1_1, object1_2 ... object1_n]
...
list_objectN = [object2_1, object2_2 ... object2_n]

Then i need to include all into one global list sorting the various
objects by date
ex
global_list = [object1_1, object2_n, object1_2 ... etc]

I think that Andre solution it's the best way to solve my dynamics
lists problem, now I'll
verify how to create the global list and sort it
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: How can I intentionally crash my lappy?

[rob]

On 12/20/2010 4:22 AM, Steven D'Aprano wrote:

On Mon, 20 Dec 2010 02:45:18 -0600, mechtheist wrote:


I am no programmer, but know the rudiments [the rudi'est of rudiments]
of working with Python.  I have a simple need,  to have a simple
script/app I can run that will crash my PC.  On my desktops, I can
always hit the reset, but that is not an option with my laptop.  Can
anyone tell me of an easy way to guarantee a program will instantly
kill/BSOD my windows7/64bit laptop?

Any help is much appreciated.

You realise that the operating system (yes, even Windows!) is designed to
*not* crash no matter what the application does?

(The OS may live up to that requirement imperfectly, but still, it tries.)


You'll probably find that task easier from C than from Python. But if I
were to attempt such a strange thing, I'd try to shut down some critical
processes used by the OS. I'm not much of a Windows person, but if you
call up the task manager (ctrl-alt-del) and inspect the process list,
then find one that causes the machine to crash if you shut it down, you
can probably do the same thing from a script.
Thank you, good approach ideas there.  Windows has gotten a lot better 
about crashing, but it is definitely still way the f 'imperfect'.  Now I 
just have to figure a way to make it imperfecter.

--
http://mail.python.org/mailman/listinfo/python-list

Re: True lists in python?

[Duncan Booth]

Vito 'ZeD' De Tullio  wrote:

> Steven D'Aprano wrote:
> 
>> I can't see any way to go from this linked list:
>> 
>> node1 -> node2 -> node3 -> node4 -> node5 -> node6 -> node7
>> 
>> to this:
>> 
>> node1 -> node6 -> node5 -> node4 -> node3 -> node2 -> node7
>> 
>> in constant time. You have to touch each of the nodes being reversed.
> 
> very crude example:
> 


No, you just showed how you can reverse an entire list in constant time. 
The original question and Steven's example were asking to reverse just 
part of the list.

I guess you might be able to do it with a double-linked list provided 
that when traversing the list you always keep two nodes around to 
determine the direction. e.g. instead of asking for node6.nextNode() you 
ask for node6.nextNode(previous=node1) and then the code can return 
whichever sibling wasn't given. That would make reversal (assuming you 
have both nodes) O(1), but would make traversing the list slower.

Also, I don't think it would help if you wanted to implement John 
Nagle's algorithm for the travelling salesman problem: you could hold a 
separate (array based) list of nodes in order to make the random 
selection O(1), but you wouldn't know which direction to follow from 
each node when it comes to cutting the list into 3.


-- 
Duncan Booth http://kupuguy.blogspot.com
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: How can I intentionally crash my lappy?

[Paul Scott]

On 20/12/2010 10:45, mechtheist wrote:
> I am no programmer, but know the rudiments [the rudi'est of rudiments]
> of working with Python.  I have a simple need,  to have a simple
> script/app I can run that will crash my PC.  On my desktops, I can
> always hit the reset, but that is not an option with my laptop.  Can
> anyone tell me of an easy way to guarantee a program will instantly
> kill/BSOD my windows7/64bit laptop?

You could try a forkbomb and execute that via a system call. Not sure
why anyone would want to do such a thing, but there you go...

-- 
-- Paul

http://www.paulscott.za.net
http://twitter.com/paulscott56
http://avoir.uwc.ac.za
All Email originating from UWC is covered by disclaimer 
http://www.uwc.ac.za/portal
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: How can I intentionally crash my lappy?

[rob]

Thanks, that is exactly what I was looking for.  Unfortunately, it is 
only for /non/-USB keyboards, I may be able to figure something out from 
there with hotkey or something.


On 12/20/2010 3:31 AM, Stefan Sonnenberg-Carstens wrote:

http://pcsupport.about.com/od/tipstricks/ht/makebsodxp.htm
Am Mo, 20.12.2010, 09:45 schrieb mechtheist:

I am no programmer, but know the rudiments [the rudi'est of rudiments]
of working with Python.  I have a simple need,  to have a simple
script/app I can run that will crash my PC.  On my desktops, I can
always hit the reset, but that is not an option with my laptop.  Can
anyone tell me of an easy way to guarantee a program will instantly
kill/BSOD my windows7/64bit laptop?

Any help is much appreciated.
--
http://mail.python.org/mailman/listinfo/python-list


-- 
http://mail.python.org/mailman/listinfo/python-list

Re: How can I intentionally crash my lappy?

[Christian Heimes]

Am 20.12.2010 09:45, schrieb mechtheist:
> I am no programmer, but know the rudiments [the rudi'est of rudiments] 
> of working with Python.  I have a simple need,  to have a simple 
> script/app I can run that will crash my PC.  On my desktops, I can 
> always hit the reset, but that is not an option with my laptop.  Can 
> anyone tell me of an easy way to guarantee a program will instantly 
> kill/BSOD my windows7/64bit laptop?

Do you want to crash it or are you looking for a way to restart Windows?
Every modern operating system tries very hard (and successful) to
prevent users from messing with the system. You have two choices. Either
you ask the OS politely to restart (e.g. shutdown -r -f -t 1) or you
need some sort of STONITH device (e.g. IPMI card, watchdog, other node
fencing device, multi-plug with network interface).

Christian

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Creating custom types from C code

[Eric Frederich]

Thanks for the reply.

I remember reading about named tuples when they were back-ported to
the 2.X series but I never played with them.
Is there a way to instantiate a named tuple from C code?

Maybe I'm over-thinking this whole thing.
Is there a simple way that I can define a class in Python and
instantiate that type from C?

On Sat, Dec 18, 2010 at 1:18 AM, Stefan Behnel  wrote:
> Eric Frederich, 17.12.2010 23:58:
>>
>> I have an extension module for a 3rd party library in which I am
>> wrapping some structures.
>> My initial attempt worked okay on Windows but failed on Linux.
>> I was doing it in two parts.
>> The first part on the C side of things I was turning the entire
>> structure into a char array.
>> The second part in Python code I would unpack the structure.
>>
>> Anyway, I decided I should be doing things a little cleaner so I read
>> up on "Defining New Types"
>> http://docs.python.org/extending/newtypes.html
>>
>> I got it to work but I'm not sure how to create these new objects from C.
>
> You may want to take a look at Cython. It makes writing C extensions easy.
> For one, it will do all sorts of type conversions for you, and do them
> efficiently and safely (you get an exception on overflow, for example). It's
> basically Python, so creating classes and instantiating them is trivial.
>
> Also note that it's generally not too much work to rewrite an existing C
> wrapper in Cython, but it's almost always worth it. You immediately get more
> maintainable code that's much easier to extend and work on. It's also often
> faster than hand written code.
>
> http://cython.org
>
>
>> My setup is almost exactly like the example on that page except
>> instead of 2 strings and an integer I have 5 unsigned ints.
>>
>> I do not expect to ever be creating these objects in Python.  They
>> will only be created as return values from my wrapper functions to the
>> 3rd party library.
>
> In Cython 0.14, you can declare classes as "final" and "internal" using a
> decorator, meaning that they cannot be subtyped from Python and do not show
> up in the module dict. However, note that there is no way to prevent users
> from getting their hands at the type once you give them an instance.
>
>
>> I could return a tuple from those functions but I want to use dot
>> notation (e.g. somestruct.var1).
>
> Then __getattr__ or properties are your friend.
>
>
>> So, question number 1:
>>     Is defining my own type like that overkill just to have an object
>> to access using dots?
>
> Creating wrapper objects is totally normal.
>
> Also note that recent Python versions have named tuples, BTW.
>
>
>>     I'll never create those objects from Python.
>>     Is there a shortcut to creating objects and setting attributes
>> from within C?
>
> The Cython code for instantiating classes is identical to Python.
>
>
>> In any case, I was able to create my own custom object from C code like
>> so...
>>
>>     PyObject *foo(SomeCStruct bar){
>>         PyObject *ret;
>>         ret = _PyObject_New(&mymodule_SomeStructType);
>>         PyObject_SetAttrString(ret, "var1" , Py_BuildValue("I", bar.var1
>> ));
>>         PyObject_SetAttrString(ret, "var2" , Py_BuildValue("I", bar.var2
>> ));
>>         PyObject_SetAttrString(ret, "var3" , Py_BuildValue("I", bar.var3
>> ));
>>         PyObject_SetAttrString(ret, "var4" , Py_BuildValue("I", bar.var4
>> ));
>>         PyObject_SetAttrString(ret, "var5" , Py_BuildValue("I", bar.var5
>> ));
>>         return ret;
>>     }
>>
>> When using _PyObject_New I notice that neither my new or init function
>> are ever called.
>> I verified that they are getting called when creating the object from
>> Python
>
> Things often work a little different in Python and C. Directly calling
> _PyObject_New() is a lot less than what Python does internally. The
> canonical way is to PyObject_Call() the type (or to use one of the other
> call functions, depending on what your arguments are).
>
>
>> (which I would never do anyway).
>
> Your users could do it, though, so you should make sure that won't crash the
> interpreter that way by leaving internal data fields uninitialised.
>
>
>> Question number 2:
>>     Do I need to be calling PyObject_SetAttrString or is there a way
>> to set the unsigned ints on the structure direcly?
>>     It seems overkill to create a Python object for an unsigned int
>> just to set it as an attribute on a custom defined type.
>
> You will have to do it at some point, though, either at instantiation time
> or at Python access time. Depending on the expected usage, either of the two
> can be more wasteful.
>
>
>> Question number 3:
>>     In the above code, is there a memory leak?  Should I be
>> Py_DECREF'ing the return value from Py_BuildValue after I'm done using
>> it.
>
> You can look that up in the C-API docs. If a function doesn't say that it
> "steals" a reference, you still own the reference when it returns and have
> to manually decref it (again, a thing that you won't usually h

Re: Creating custom types from C code

[Stefan Behnel]

Eric Frederich, 20.12.2010 16:23:

I remember reading about named tuples when they were back-ported to the
2.X series but I never played with them. Is there a way to instantiate a
named tuple from C code?


There isn't a dedicated C-API for named tuples, but you can instantiate any 
Python type from C code.




Maybe I'm over-thinking this whole thing. Is there a simple way that I
can define a class in Python and instantiate that type from C?


Sure, see my previous comments to your original mail.

Stefan

--
http://mail.python.org/mailman/listinfo/python-list

Re: Python-list Digest, Vol 87, Issue 122

[Randy Given]

ZS
On Dec 19, 2010 6:05 AM,  wrote:
> Send Python-list mailing list submissions to
> python-list@python.org
>
> To subscribe or unsubscribe via the World Wide Web, visit
> http://mail.python.org/mailman/listinfo/python-list
> or, via email, send a message with subject or body 'help' to
> python-list-requ...@python.org
>
> You can reach the person managing the list at
> python-list-ow...@python.org
>
> When replying, please edit your Subject line so it is more specific
> than "Re: Contents of Python-list digest..."
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Python-list Digest, Vol 87, Issue 122

[Randy Given]

as
On Dec 19, 2010 6:05 AM,  wrote:
> Send Python-list mailing list submissions to
> python-list@python.org
>
> To subscribe or unsubscribe via the World Wide Web, visit
> http://mail.python.org/mailman/listinfo/python-list
> or, via email, send a message with subject or body 'help' to
> python-list-requ...@python.org
>
> You can reach the person managing the list at
> python-list-ow...@python.org
>
> When replying, please edit your Subject line so it is more specific
> than "Re: Contents of Python-list digest..."
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Catching user switching and getting current active user from root on linux

[mpnordland]

I give up, I will never try to use a usenet group again. For the ones of you 
who tried to help thank you. You helped to identify some of my troubles, as for 
you @usernet, you are a troll
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: how to measure TCP congestion windows using python ??

[MrJean1]

FWIW, on CentOS 4.7, the ctypes version works fine, but the struct
version fails, because len(tcp_info) is only 100 bytes while
struct.calcsize('B...L') is 104.

However, if the format is changed to '7B23L', i.e. one 'L' shorter,
the struct version works and returns to same result as the ctypes
version.

/Jean


On Dec 19, 4:45 pm, plz  wrote:
> hi
> many thanks for helping me
> i also tried to manipulate it last night
> here is my code...
>
> import socket
> import struct
>
> sock = socket.socket(socket.AF_INET,socket.SOCK_STREAM)
> tcp_info = sock.getsockopt(socket.SOL_TCP, socket.TCP_INFO, 
> struct.calcsize('BBB'))
> print struct.unpack('BBB', tcp_info)
>
> the result of struct.unpack of tcp_info is following
> in /usr/include/linux/tcp.h
> used Linux Redhat and Python 2.7
>
> anyway your code is very useful
> Thxs again ;)

-- 
http://mail.python.org/mailman/listinfo/python-list

Questions

[lewisr2]

Folks,
1. As a free lance developer I need to use Python with Web Content Services using IIS7 (Windows 7).  However, I don't have a lot of time to learn IIS7.  Can someone tell me how to configure IIS7 to be used by Python CGI scripts?
2.  I noticed that the latest distribution from ActiveState has Jason with it; but the documentation does not go into Ajax or the use of Jason with Python.  Any suggestions on where I can find info?
3.  I have used Tk in the past with Tcl and IncrTcl.  Where can I find a lot of Python/Tk examples so that I can save some time in developing GUIs? 
Thanks,
 
Lew
-- 
http://mail.python.org/mailman/listinfo/python-list

**** httplib.InvalidURL: nonnumeric port ****

[Anurag Chourasia]

All,

When i try to open a URL of the form
http://joule:8041/DteEnLinea/ws/EnvioGuia.jws, I am getting an error as
below complaining for some non-numeric port.

wm (wmosds) [zibal] - /u01/home/apli/wm/app/gdd :>python
Python 2.4.1 (#2, May 30 2005, 09:40:30) [C] on aix5
Type "help", "copyright", "credits" or "license" for more information.
>>> import httplib, mimetypes
>>> h = httplib.HTTP('http://joule:8041/DteEnLinea/ws/EnvioGuia.jws')
Traceback (most recent call last):
  File "", line 1, in ?
  File "/u01/home/apli/wm/python241/lib/python2.4/httplib.py", line 1093, in
__init__
self._setup(self._connection_class(host, port, strict))
  File "/u01/home/apli/wm/python241/lib/python2.4/httplib.py", line 582, in
__init__
self._set_hostport(host, port)
  File "/u01/home/apli/wm/python241/lib/python2.4/httplib.py", line 594, in
_set_hostport
raise InvalidURL("nonnumeric port: '%s'" % host[i+1:])
httplib.InvalidURL: nonnumeric port: '8041/DteEnLinea/ws/EnvioGuia.jws'

How could we avoid this problem?

Regards,
Anurag
-- 
http://mail.python.org/mailman/listinfo/python-list

Sending XML to a WEB Service and Getting Response Back

[Anurag Chourasia]

Dear Python Mates,

I have a requirement to send a XML Data to a WEB Service whose URL is of the
form http://joule:8041/DteEnLinea/ws/EnvioGuia.jws

I also have to read back the response returned as a result of sending this
data to this WebService.

This web service implements the following operations:
   sendNCR

This web service has no callbacks.

I have pasted the complete WSDL for this WEB Service below my email.

I would appreciate if someone could guide me with sample code using a Python
Library suitable to fulfill this requirement of mine.

Regards,
Anurag



http://www.openuri.org/2002/04/soap/conversation/"; xmlns:cw="
http://www.openuri.org/2002/04/wsdl/conversation/"; xmlns:http="
http://schemas.xmlsoap.org/wsdl/http/"; xmlns:jms="
http://www.openuri.org/2002/04/wsdl/jms/"; xmlns:mime="
http://schemas.xmlsoap.org/wsdl/mime/"; xmlns:s="
http://www.w3.org/2001/XMLSchema"; xmlns:s0="http://www.openuri.org/";
xmlns:soap="http://schemas.xmlsoap.org/wsdl/soap/"; xmlns:soapenc="
http://schemas.xmlsoap.org/soap/encoding/"; xmlns:wsdl="
http://schemas.xmlsoap.org/wsdl/"; targetNamespace="http://www.openuri.org/";>
  
http://www.w3.org/2001/XMLSchema";
elementFormDefault="qualified" targetNamespace="http://www.openuri.org/";>
  

  

  

  
  

  

  

  
  


  
  

  
  

  
  

  
  

  
  

  
  

  
  

  
  

  
  

  
  

  
  

  
  

  
  
http://schemas.xmlsoap.org/soap/http";
style="document"/>

  http://www.openuri.org/sendNCR";
style="document"/>
  

  
  

  

  
  


  
  

  
  

  

  
  


  
  

  
  

  

  
  

  http://joule:8041/DteEnLinea/ws/EnvioGuia.jws
"/>


  http://joule:8041/DteEnLinea/ws/EnvioGuia.jws
"/>


  http://joule:8041/DteEnLinea/ws/EnvioGuia.jws
"/>

  

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: **** httplib.InvalidURL: nonnumeric port ****

[Kev Dwyer]

On Tue, 21 Dec 2010 00:01:44 +0530, Anurag Chourasia wrote:

>>> import httplib
>>> help(httplib.HTTP)
Help on class HTTP in module httplib:

class HTTP
 |  Compatibility class with httplib.py from 1.5.
 |  
 |  Methods defined here:
 |  
 |  __init__(self, host='', port=None, strict=None)

The constructor doesn't take a complete URL as an argument.  

Also, shouldn't you be using httplib.HTTPConnection?  The docs
 state that httplib.HTTP is for backward compatibility only.

Kev

-- 
http://mail.python.org/mailman/listinfo/python-list

help with link parsing?

[Littlefield, Tyler]

Hello all,
I have a question. I guess this worked pre 2.6; I don't remember the 
last time I used it, but it was a while ago, and now it's failing. 
Anyone mind looking at it and telling me what's going wrong? Also, is 
there a quick way to match on a certain site? like links from google.com 
and only output those?

#!/usr/bin/env python

#This program is free software: you can redistribute it and/or modify it 
under the terms of the GNU General Public License as published
#by the Free Software Foundation, either version 3 of the License, or 
(at your option) any later version.


#This program is distributed in the hope that it will be useful, but 
WITHOUT ANY WARRANTY; without even the implied warranty of
#MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU 
General Public License for more details.

#
#You should have received a copy of the GNU General Public License along 
with this program. If not, see

#http://www.gnu.org/licenses/.

"""
This script will parse out all the links in an html document and write 
them to a textfile.

"""
import sys,optparse
import htmllib,formatter

#program class declarations:
class Links(htmllib.HTMLParser):
def __init__(self,formatter):
htmllib.HTMLParser.__init__(self, formatter)
self.links=[]
def start_a(self, attrs):
if (len(attrs)>0):
for a in attrs:
if a[0]=="href":
self.links.append(a[1])
print a[1]
break

def main(argv):
if (len(argv)!=3):
print("Error:\n"+argv[0]+"  .\nParses  
for all links and saves them to .")

return 1
lcount=0
format=formatter.NullFormatter()
html=Links(format)
print "Retrieving data:"
page=open(argv[1],"r")
print "Feeding data to parser:"
html.feed(page.read())
page.close()
print "Writing links:"
output=open(argv[2],"w")
for i in (html.links):
output.write(i+"\n")
lcount+=1
output.close()
print("Wrote "+str(lcount)+" links to "+argv[2]+".");
print("done.")

if (__name__ == "__main__"):
#we call the main function passing a list of args, and exit with 
the return code passed back.

sys.exit(main(sys.argv))

--

Thanks,
Ty

--
http://mail.python.org/mailman/listinfo/python-list

Re: **** httplib.InvalidURL: nonnumeric port ****

[Anurag Chourasia]

Hi Kevin,

Thanks for the response.

I tried with HTTPConnection but the same problem.

>>> h1 = httplib.HTTPConnection('
http://joule:8041/DteEnLinea/ws/EnvioGuia.jws')
Traceback (most recent call last):
  File "", line 1, in ?
  File "/u01/home/apli/wm/python241/lib/python2.4/httplib.py", line 582, in
__init__
self._set_hostport(host, port)
  File "/u01/home/apli/wm/python241/lib/python2.4/httplib.py", line 594, in
_set_hostport
raise InvalidURL("nonnumeric port: '%s'" % host[i+1:])
httplib.InvalidURL: nonnumeric port: '8041/DteEnLinea/ws/EnvioGuia.jws'

Do i need to use some other library to be able to send XML Data (and get a
response back) to this Kind of Web Service address i.e.
http://joule:8041/DteEnLinea/ws/EnvioGuia.jws ?

Regards,
Anurag

On Tue, Dec 21, 2010 at 12:42 AM, Kev Dwyer  wrote:

> On Tue, 21 Dec 2010 00:01:44 +0530, Anurag Chourasia wrote:
>
> >>> import httplib
> >>> help(httplib.HTTP)
> Help on class HTTP in module httplib:
>
> class HTTP
>  |  Compatibility class with httplib.py from 1.5.
>  |
>  |  Methods defined here:
>  |
>  |  __init__(self, host='', port=None, strict=None)
>
> The constructor doesn't take a complete URL as an argument.
>
> Also, shouldn't you be using httplib.HTTPConnection?  The docs
>  state that httplib.HTTP is for backward compatibility only.
>
> Kev
>
> --
> http://mail.python.org/mailman/listinfo/python-list
>
-- 
http://mail.python.org/mailman/listinfo/python-list

Trying to parse a HUGE(1gb) xml file

[spaceman-spiff]

Hi c.l.p folks

This is a rather long post, but i wanted to include all the details & 
everything i have tried so far myself, so please bear with me & read the entire 
boringly long post.

I am trying to parse a ginormous ( ~ 1gb) xml file.


0. I am a python & xml n00b, s& have been relying on the excellent beginner 
book DIP(Dive_Into_Python3 by MP(Mark Pilgrim) Mark , if u are readng this, 
you are AWESOME & so is your witty & humorous writing style)


1. Almost all exmaples pf parsing xml in python, i have seen, start off with 
these 4 lines of code.

import xml.etree.ElementTree as etree
tree = etree.parse('*path_to_ginormous_xml*')
root = tree.getroot()  #my huge xml has 1 root at the top level
print root

2. In the 2nd line of code above, as Mark explains in DIP, the parse function 
builds & returns a tree object, in-memory(RAM), which represents the entire 
document.
I tried this code, which works fine for a small ( ~ 1MB), but when i run this 
simple 4 line py code in a terminal for my HUGE target file (1GB), nothing 
happens.
In a separate terminal, i run the top command, & i can see a python process, 
with memory (the VIRT column) increasing from 100MB , all the way upto 2100MB.

I am guessing, as this happens (over the course of 20-30 mins), the tree 
representing is being slowly built in memory, but even after 30-40 mins, 
nothing happens.
I dont get an error, seg fault or out_of_memory exception.

My hardware setup : I have a win7 pro box with 8gb of RAM & intel core2 quad 
cpuq9400.
On this i am running sun virtualbox(3.2.12), with ubuntu 10.10 as guest os, 
with 23gb disk space & 2gb(2048mb) ram, assigned to the guest ubuntu os.

3. I also tried using lxml, but an lxml tree is much more expensive, as it 
retains more info about a node's context, including references to it's parent.
[http://www.ibm.com/developerworks/xml/library/x-hiperfparse/]

When i ran the same 4line code above, but with lxml's elementree ( using the 
import below in line1of the code above)
import lxml.etree as lxml_etree

i can see the memory consumption of the python process(which is running the 
code) shoot upto ~ 2700mb & then, python(or the os ?) kills the process as it 
nears the total system memory(2gb)

I ran the code from 1 terminal window (screenshot :http://imgur.com/ozLkB.png)
& ran top from another terminal (http://imgur.com/HAoHA.png)

4. I then investigated some streaming libraries, but am confused - there is 
SAX[http://en.wikipedia.org/wiki/Simple_API_for_XML] , the iterparse 
interface[http://effbot.org/zone/element-iterparse.htm]

Which one is the best for my situation ?

Any & all code_snippets/wisdom/thoughts/ideas/suggestions/feedback/comments/ of 
the c.l.p community would be greatly appreciated.
Plz feel free to email me directly too.

thanks a ton

cheers
ashish

email : 
ashish.makani
domain:gmail.com

p.s.
Other useful links on xml parsing in python
0. http://diveintopython3.org/xml.html
1. 
http://stackoverflow.com/questions/1513592/python-is-there-an-xml-parser-implemented-as-a-generator
2. http://codespeak.net/lxml/tutorial.html
3. 
https://groups.google.com/forum/?hl=en&lnk=gst&q=parsing+a+huge+xml#!topic/comp.lang.python/CMgToEnjZBk
4. http://www.ibm.com/developerworks/xml/library/x-hiperfparse/
5.http://effbot.org/zone/element-index.htm
http://effbot.org/zone/element-iterparse.htm
6. SAX : http://en.wikipedia.org/wiki/Simple_API_for_XML


-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Questions

[Terry Reedy]

On 12/20/2010 12:10 PM, lewi...@verizon.net wrote:


3. I have used Tk in the past with Tcl and IncrTcl. Where can I find a
lot of Python/Tk examples so that I can save some time in developing GUIs?


Do a search, or possibly a google codesearch, on tkinter or Tkinter, 
which is Python's tk inter(face). In fact, search the archive of 
python-list (or gmane's mirror) for posted examples.


--
Terry Jan Reedy

--
http://mail.python.org/mailman/listinfo/python-list

Re: **** httplib.InvalidURL: nonnumeric port ****

[Chris Rebert]

> On Tue, Dec 21, 2010 at 12:42 AM, Kev Dwyer  wrote:
>> On Tue, 21 Dec 2010 00:01:44 +0530, Anurag Chourasia wrote:
>> >>> import httplib
>> >>> help(httplib.HTTP)
>> Help on class HTTP in module httplib:
>>
>> class HTTP
>>  |  Compatibility class with httplib.py from 1.5.
>>  |
>>  |  Methods defined here:
>>  |
>>  |  __init__(self, host='', port=None, strict=None)
>>
>> The constructor doesn't take a complete URL as an argument.
>>
>> Also, shouldn't you be using httplib.HTTPConnection?  The docs
>>  state that httplib.HTTP is for backward compatibility only.
>>
On Mon, Dec 20, 2010 at 11:30 AM, Anurag Chourasia
 wrote:
> Hi Kevin,
> Thanks for the response.
> I tried with HTTPConnection but the same problem.
 h1 =
 httplib.HTTPConnection('http://joule:8041/DteEnLinea/ws/EnvioGuia.jws')
> Traceback (most recent call last):
>   File "", line 1, in ?
>   File "/u01/home/apli/wm/python241/lib/python2.4/httplib.py", line 582, in
> __init__
>     self._set_hostport(host, port)
>   File "/u01/home/apli/wm/python241/lib/python2.4/httplib.py", line 594, in
> _set_hostport
>     raise InvalidURL("nonnumeric port: '%s'" % host[i+1:])
> httplib.InvalidURL: nonnumeric port: '8041/DteEnLinea/ws/EnvioGuia.jws'
> Do i need to use some other library to be able to send XML Data (and get a
> response back) to this Kind of Web Service address
> i.e. http://joule:8041/DteEnLinea/ws/EnvioGuia.jws ?
> Regards,
> Anurag

(1) Please avoid top-posting
(2) Did you read the other part of Kev's comment? "The constructor
doesn't take a complete URL as an argument." You must pass only the
domain to the constructor and then call .request() on the
HTTPConnection object.
(3) If you're using SOAP, try https://fedorahosted.org/suds/

Cheers,
Chris
--
http://blog.rebertia.com
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: help with link parsing?

[Chris Rebert]

On Mon, Dec 20, 2010 at 11:14 AM, Littlefield, Tyler
 wrote:
> Hello all,
> I have a question. I guess this worked pre 2.6; I don't remember the last
> time I used it, but it was a while ago, and now it's failing. Anyone mind
> looking at it and telling me what's going wrong?

Please describe /exactly/ how it is failing for you, including the
full exception traceback (if any).
The script seems to work fine for me under both Python v2.6.6 and v2.7.1.

Cheers,
Chris
--
http://blog.rebertia.com
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: **** httplib.InvalidURL: nonnumeric port ****

[Kev Dwyer]

On Tue, 21 Dec 2010 01:00:36 +0530, Anurag Chourasia wrote:

Anurag,

HTTPConnection takes a host and a port number as arguments, not just a URL.

So you could construct your connection request like this:
conn = httplib.HTTPConnection('joule', 8041)

then use the request() method on the connection to make a PUT or GET request:
conn.request('PUT', url, body, headers)

Please read the documentation for the httplib module, and perhaps some basic 
material on how HTTP requests work.

Cheers,

Kev

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Trying to parse a HUGE(1gb) xml file

[Tim Harig]

[Wrapped to meet RFC1855 Netiquette Guidelines]
On 2010-12-20, spaceman-spiff  wrote:
> This is a rather long post, but i wanted to include all the details &
> everything i have tried so far myself, so please bear with me & read
> the entire boringly long post.
> 
> I am trying to parse a ginormous ( ~ 1gb) xml file.
[SNIP]
> 4. I then investigated some streaming libraries, but am confused - there
> is SAX[http://en.wikipedia.org/wiki/Simple_API_for_XML] , the iterparse
> interface[http://effbot.org/zone/element-iterparse.htm]

I have made extensive use of SAX and it will certainly work for low
memory parsing of XML.  I have never used "iterparse"; so, I cannot make
an informed comparison between them.

> Which one is the best for my situation ?

Your posed was long but it failed to tell us the most important piece
of information:  What does your data look like and what are you trying
to do with it?

SAX is a low level API that provides a callback interface allowing you to
processes various elements as they are encountered.  You can therefore
do anything you want to the information, as you encounter it, including
outputing and discarding small chunks as you processes it; ignoring
most of it and saving only what you want to memory data structures;
or saving all of it to a more random access database or on disk data
structure that you can load and process as required.

What you need to do will depend on what you are actually trying to
accomplish.  Without knowing that, I can only affirm that SAX will work
for your needs without providing any information about how you should
be using it.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Sending XML to a WEB Service and Getting Response Back

[Hidura]

I recommend you use the urllib.request in the library of python says
everything that you want to know.

2010/12/20, Anurag Chourasia :
> Dear Python Mates,
>
> I have a requirement to send a XML Data to a WEB Service whose URL is of the
> form http://joule:8041/DteEnLinea/ws/EnvioGuia.jws
>
> I also have to read back the response returned as a result of sending this
> data to this WebService.
>
> This web service implements the following operations:
>sendNCR
>
> This web service has no callbacks.
>
> I have pasted the complete WSDL for this WEB Service below my email.
>
> I would appreciate if someone could guide me with sample code using a Python
> Library suitable to fulfill this requirement of mine.
>
> Regards,
> Anurag
>
>
> 
> http://www.openuri.org/2002/04/soap/conversation/"; xmlns:cw="
> http://www.openuri.org/2002/04/wsdl/conversation/"; xmlns:http="
> http://schemas.xmlsoap.org/wsdl/http/"; xmlns:jms="
> http://www.openuri.org/2002/04/wsdl/jms/"; xmlns:mime="
> http://schemas.xmlsoap.org/wsdl/mime/"; xmlns:s="
> http://www.w3.org/2001/XMLSchema"; xmlns:s0="http://www.openuri.org/";
> xmlns:soap="http://schemas.xmlsoap.org/wsdl/soap/"; xmlns:soapenc="
> http://schemas.xmlsoap.org/soap/encoding/"; xmlns:wsdl="
> http://schemas.xmlsoap.org/wsdl/"; targetNamespace="http://www.openuri.org/";>
>   
> http://www.w3.org/2001/XMLSchema";
> elementFormDefault="qualified" targetNamespace="http://www.openuri.org/";>
>   
> 
>   
> 
>   
> 
>   
>   
> 
>   
> 
>   
> 
>   
>   
> 
>
>   
>   
> 
>   
>   
> 
>   
>   
> 
>   
>   
> 
>   
>   
> 
>   
>   
> 
>   
>   
> 
>   
>   
> 
>   
>   
> 
>   
>   
> 
>   
>   
> 
>   
>   
> 
>   
>   
> http://schemas.xmlsoap.org/soap/http";
> style="document"/>
> 
>   http://www.openuri.org/sendNCR";
> style="document"/>
>   
> 
>   
>   
> 
>   
> 
>   
>   
> 
> 
>   
>   
> 
>   
>   
> 
>   
> 
>   
>   
> 
> 
>   
>   
> 
>   
>   
> 
>   
> 
>   
>   
> 
>   http://joule:8041/DteEnLinea/ws/EnvioGuia.jws
> "/>
> 
> 
>   http://joule:8041/DteEnLinea/ws/EnvioGuia.jws
> "/>
> 
> 
>   http://joule:8041/DteEnLinea/ws/EnvioGuia.jws
> "/>
> 
>   
> 
>

-- 
Enviado desde mi dispositivo móvil

Diego I. Hidalgo D.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Trying to parse a HUGE(1gb) xml file

[Adam Tauno Williams]

On Mon, 2010-12-20 at 11:34 -0800, spaceman-spiff wrote:
> Hi c.l.p folks
> This is a rather long post, but i wanted to include all the details &
> everything i have tried so far myself, so please bear with me & read
> the entire boringly long post.
> I am trying to parse a ginormous ( ~ 1gb) xml file.

Do that hundreds of times a day.

> 0. I am a python & xml n00b, s& have been relying on the excellent
> beginner book DIP(Dive_Into_Python3 by MP(Mark Pilgrim) Mark , if
> u are readng this, you are AWESOME & so is your witty & humorous
> writing style)
> 1. Almost all exmaples pf parsing xml in python, i have seen, start off with 
> these 4 lines of code.
> import xml.etree.ElementTree as etree
> tree = etree.parse('*path_to_ginormous_xml*')
> root = tree.getroot()  #my huge xml has 1 root at the top level
> print root

Yes, this is a terrible technique;  most examples are crap.

> 2. In the 2nd line of code above, as Mark explains in DIP, the parse
> function builds & returns a tree object, in-memory(RAM), which
> represents the entire document.
> I tried this code, which works fine for a small ( ~ 1MB), but when i
> run this simple 4 line py code in a terminal for my HUGE target file
> (1GB), nothing happens.
> In a separate terminal, i run the top command, & i can see a python
> process, with memory (the VIRT column) increasing from 100MB , all the
> way upto 2100MB.

Yes, this is using DOM.  DOM is evil and the enemy, full-stop.

> I am guessing, as this happens (over the course of 20-30 mins), the
> tree representing is being slowly built in memory, but even after
> 30-40 mins, nothing happens.
> I dont get an error, seg fault or out_of_memory exception.

You need to process the document as a stream of elements; aka SAX.

> 3. I also tried using lxml, but an lxml tree is much more expensive,
> as it retains more info about a node's context, including references
> to it's parent.
> [http://www.ibm.com/developerworks/xml/library/x-hiperfparse/]
> When i ran the same 4line code above, but with lxml's elementree
> ( using the import below in line1of the code above)
> import lxml.etree as lxml_etree

You're still using DOM; DOM is evil.

> Which one is the best for my situation ?
> Any & all
> code_snippets/wisdom/thoughts/ideas/suggestions/feedback/comments/ of
> the c.l.p community would be greatly appreciated.
> Plz feel free to email me directly too.





-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Trying to parse a HUGE(1gb) xml file

[spaceman-spiff]

Hi Usernet

First up, thanks for your prompt reply.
I will make sure i read RFC1855, before posting again, but right now chasing a 
hard deadline :)

I am sorry i left out what exactly i am trying to do.

0. Goal :I am looking for a specific element..there are several 10s/100s 
occurrences of that element in the 1gb xml file.
The contents of the xml, is just a dump of config parameters from a packet 
switch( although imho, the contents of the xml dont matter)

I need to detect them & then for each 1, i need to copy all the content b/w the 
element's start & end tags & create a smaller xml file.

1. Can you point me to some examples/samples of using SAX, especially , ones 
dealing with really large XML files.

2.This brings me to another q. which i forgot to ask in my OP(original post).
Is simply opening the file, & using reg ex to look for the element i need, a 
*good* approach ?
While researching my problem, some article seemed to advise against this, 
especially since its known apriori, that the file is an xml & since regex code 
gets complicated very quickly & is not very readable.

But is that just a "style"/"elegance" issue, & for my particular problem 
(detecting a certain element, & then creating(writing) a smaller xml file 
corresponding to, each pair of start & end tags of said element), is the open 
file & regex approach, something you would recommend ?

Thanks again for your super-prompt response :)

cheers
ashish
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Trying to parse a HUGE(1gb) xml file

[Adam Tauno Williams]

On Mon, 2010-12-20 at 12:29 -0800, spaceman-spiff wrote:
> I need to detect them & then for each 1, i need to copy all the
> content b/w the element's start & end tags & create a smaller xml
> file.

Yep, do that a lot; via iterparse.

> 1. Can you point me to some examples/samples of using SAX,
> especially , ones dealing with really large XML files.

SaX is equivalent to iterparse (iterpase is a way, to essentially, do
SaX-like processing).

I provided an iterparse example already. See the Read_Rows method in 


> 2.This brings me to another q. which i forgot to ask in my OP(original post).
> Is simply opening the file, & using reg ex to look for the element i need, a 
> *good* approach ?

No.


-- 
http://mail.python.org/mailman/listinfo/python-list

Why do my list go uni-code by itself?

[Martin Hvidberg]

I'm reading a fixed format text file, line by line. I hereunder present 
the code. I have  out part not related to the file reading.

Only relevant detail left out is the lstCutters. It looks like this:
[[1, 9], [11, 21], [23, 48], [50, 59], [61, 96], [98, 123], [125, 150]]
It specifies the first and last character position of each token in the 
fixed format of the input line.

All this works fine, and is only to explain where I'm going.

The code, in the function definition, is broken up in more lines than 
necessary, to be able to monitor the variables, step by step.


--- Code start --

import codecs



def CutLine2List(strIn,lstCut):
strIn = strIn.strip()
print '>InNextLine>',strIn
# skip if line is empty
if len(strIn)<1:
return False
lstIn = list()
for cc in lstCut:
strSubline =strIn[cc[0]-1:cc[1]-1].strip()
lstIn.append(strSubline)
print '>InSubline2>'+lstIn[len(lstIn)-1]+'<'
del strIn, lstCut,cc
print '>InReturLst>',lstIn
return lstIn



filIn = codecs.open(
strFileNameIn,
mode='r',
encoding='utf-8',
errors='strict',
buffering=1)
 for linIn in filIn:
lstIn = CutLine2List(linIn,lstCutters)

--- Code end --

A sample output, representing one line from the input file looks like this:

>InNextLine> I 30  2002-12-11 20:01:19.280
563FANØ 
2001-12-12-15.46.12.734502 2001-12-12-15.46.12.734502

>InSubline2>I<
>InSubline2>30<
>InSubline2>2002-12-11 20:01:19.280<
>InSubline2>563<
>InSubline2>FANØ<
>InSubline2>2001-12-12-15.46.12.73450<
>InSubline2>2001-12-12-15.46.12.73450<
>InReturLst> [u'I', u'30', u'2002-12-11 20:01:19.280', u'563', 
u'FAN\xd8', u'2001-12-12-15.46.12.73450', u'2001-12-12-15.46.12.73450']



Question:
In the last printout, tagged >InReturLst> all entries turn into 
uni-code. What happens here?
Look for the word 'FANØ'. This word changes from 'FANØ' to u'FAN\xd8' -- 
That's a problem to me, and I don't want it to change like this.


What do I do to stop this behavior?

Best Regards
Martin

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Trying to parse a HUGE(1gb) xml file

[Terry Reedy]

On 12/20/2010 2:49 PM, Adam Tauno Williams wrote:


Yes, this is a terrible technique;  most examples are crap.



Yes, this is using DOM.  DOM is evil and the enemy, full-stop.



You're still using DOM; DOM is evil.


For serial processing, DOM is superfluous superstructure.
For random access processing, some might disagree.




Which one is the best for my situation ?
Any&  all
code_snippets/wisdom/thoughts/ideas/suggestions/feedback/comments/ of
the c.l.p community would be greatly appreciated.
Plz feel free to email me directly too.







For Python (unlike Java), wrapping module functions as class static 
methods is superfluous superstructure that only slows things down.


raise Exception(...) # should be something specific like
raise ValueError(...)

--
Terry Jan Reedy

--
http://mail.python.org/mailman/listinfo/python-list

Re: Trying to parse a HUGE(1gb) xml file

[Tim Harig]

On 2010-12-20, spaceman-spiff  wrote:
> 0. Goal :I am looking for a specific element..there are several 10s/100s
> occurrences of that element in the 1gb xml file.  The contents of the xml,
> is just a dump of config parameters from a packet switch( although imho,
> the contents of the xml dont matter)

Then you need:
1. To detect whenever you move inside of the type element you are
seeking and whenever you move out of it.  As long as these
elements cannot be nested inside of each other, this is an
easy binary task.  If they can be nested, then you will
need to maintain some kind of level count or recursively
decompose each level.

2. Once you have obtained a complete element (from its start tag to
its end tag) you will need to test whether you have the
single correct element that you are looking for.

Something like this (untested) will work if the target tag cannot be nested
in another target tag:

- import xml.sax
- class tagSearcher(xml.sax.ContentHandler):
- 
- def startDocument():
- self.inTarget = False
- 
- def startElement(name, attrs):
- if name == targetName:
- self.inTarget = True
- elif inTarget = True:
- # save element information
- 
- def endElement(name):
- if name == targetName:
- self.inTarget = False
- # test the saved information to see if you have the
- # one you want:
- #
- #if its the peice you are looking for, then
- #you can process the information
- #you have saved
- #
- #if not, disgard the accumulated
- # information and move on
- 
- def characters(content):
- if self.inTarget == True:
- # save the content
- 
- yourHandler = tagSearcher()
- yourParser = xml.sax.make_parser()
- yourParser.parse(inputXML, yourHandler)

Then you just walk through the document picking up and discarding each
target element type until you have the one that you are looking for.

> I need to detect them & then for each 1, i need to copy all the content
> b/w the element's start & end tags & create a smaller xml file.

Easy enough; but, with SAX you will have to recreate the tags from
the information that they contain because they will be skipped by the
character() events; so you will need to save the information from each tag
as you come across it.  This could probably be done more automatically
using saxutils.XMLGenerator; but, I haven't actually worked with it
before. xml.dom.pulldom also looks interesting
 
> 1. Can you point me to some examples/samples of using SAX, especially ,
> ones dealing with really large XML files.

There is nothing special about large files with SAX.  Sax is very simple.
It walks through the document and calls the the functions that you
give it for each event as it reaches varius elements.  Your callback
functions (methods of a handler) do everthing with the information.
SAX does nothing more then call your functions.  There are events for
reaching a  starting tag, an end tag, and characters between tags;
as well as some for beginning and ending a document.

> 2.This brings me to another q. which i forgot to ask in my OP(original
> post).  Is simply opening the file, & using reg ex to look for the element
> i need, a *good* approach ?  While researching my problem, some article
> seemed to advise against this, especially since its known apriori, that
> the file is an xml & since regex code gets complicated very quickly &
> is not very readable.
>
> But is that just a "style"/"elegance" issue, & for my particular problem
> (detecting a certain element, & then creating(writing) a smaller xml
> file corresponding to, each pair of start & end tags of said element),
> is the open file & regex approach, something you would recommend ?

It isn't an invalid approach if it works for your situatuation.  I have
used it before for very simple problems.  The thing is, XML is a context
free data format which makes it difficult to generate precise regular
expressions, especially where where tags of the same type can be nested.

It can be very error prone.  Its really easy to have a regex work for
your tests and fail, either by matching too much or failing to match,
because you didn't anticipate a given piece of data.  I wouldn't consider
it a robust solution.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Why do my list go uni-code by itself?

[MRAB]

> I'm reading a fixed format text file, line by line. I hereunder 
present the code. I have  out part not related to the file reading.

> Only relevant detail left out is the lstCutters. It looks like this:
> [[1, 9], [11, 21], [23, 48], [50, 59], [61, 96], [98, 123], [125, 150]]
> It specifies the first and last character position of each token in 
the fixed format of the input line.

> All this works fine, and is only to explain where I'm going.
>
> The code, in the function definition, is broken up in more lines than 
necessary, to be able to monitor the variables, step by step.

>
> --- Code start --
>
> import codecs
>
> 
>
> def CutLine2List(strIn,lstCut):
> strIn = strIn.strip()
> print '>InNextLine>',strIn
> # skip if line is empty
> if len(strIn)<1:
> return False

More Pythonic would be:

if not strIn:

> lstIn = list()
> for cc in lstCut:
> strSubline =strIn[cc[0]-1:cc[1]-1].strip()

The start index is inclusive; the end index is exclusive.

> lstIn.append(strSubline)
> print '>InSubline2>'+lstIn[len(lstIn)-1]+'<'
> del strIn, lstCut,cc

Not necessary to del the names. They exist only within the function,
which you're about to leave.

> print '>InReturLst>',lstIn
> return lstIn
>
Sometimes it returns a list and sometimes False. That's a bad idea; try
to be consistent.

> 
>
> filIn = codecs.open(
> strFileNameIn,
> mode='r',
> encoding='utf-8',
> errors='strict',
> buffering=1)

You're decoding from UTF-8 to Unicode, so all the strings you're
working on are Unicode strings.

>  for linIn in filIn:
> lstIn = CutLine2List(linIn,lstCutters)
>

--
http://mail.python.org/mailman/listinfo/python-list

Re: Why do my list go uni-code by itself?

[Ian Kelly]

On Mon, Dec 20, 2010 at 2:08 PM, Martin Hvidberg  wrote:
> Question:
> In the last printout, tagged >InReturLst> all entries turn into uni-code.
> What happens here?

Actually, they were all unicode to begin with.  You're using
codecs.open to read the file, which transparently decodes the data
using the supplied encoding (in this case, utf-8).  If you wanted to
preserve the original bytes, you would just use the open() function to
open the file instead.

> Look for the word 'FANØ'. This word changes from 'FANØ' to u'FAN\xd8' –
> That's a problem to me, and I don't want it to change like this.

This happens because you're printing a list instead of a unicode
string.  When you print the unicode string, it tries to print the
actual characters.  When you print the list, it constructs the repr of
the list, which uses the repr of each of the items in the list, and
the repr of the unicode string is u'FAN\xd8'.  If you don't want this
to happen, then you will need to format the list as a string yourself
instead of relying on print to do what it thinks you might want.

Cheers,
Ian
-- 
http://mail.python.org/mailman/listinfo/python-list

Modifying an existing excel spreadsheet

[Ed Keith]

I have a user supplied 'template' Excel spreadsheet. I need to create a new 
excel spreadsheet based on the supplied template, with data filled in. 

I found the tools here http://www.python-excel.org/,  and 
http://sourceforge.net/projects/pyexcelerator/. I have been trying to use the 
former, since the latter seems to be devoid of documentation (not even any 
docstrings).


My first thought was to copy the template, open the copy, modify it and save 
the modifications. But it looks like if I open an existing spreadsheet it must 
be read only. So I tried to  open the template, copy it to a new spreadsheet 
and write the new spreadsheet, but I can't seem to copy the images, and it 
looks like copying the formatting is going to be difficult.

Can anyone give me any tips or advice?

Thanks in advance,

   -EdK

Ed Keith

e_...@yahoo.com



Blog: edkeith.blogspot.com


  
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Why do my list go uni-code by itself?

[Dave Angel]

On 01/-10/-28163 02:59 PM, Martin Hvidberg wrote:

I'm reading a fixed format text file, line by line. I hereunder present
the code. I have  out part not related to the file reading.
Only relevant detail left out is the lstCutters. It looks like this:
[[1, 9], [11, 21], [23, 48], [50, 59], [61, 96], [98, 123], [125, 150]]
It specifies the first and last character position of each token in the
fixed format of the input line.
All this works fine, and is only to explain where I'm going.

The code, in the function definition, is broken up in more lines than
necessary, to be able to monitor the variables, step by step.

--- Code start --

import codecs



def CutLine2List(strIn,lstCut):
strIn = strIn.strip()
print '>InNextLine>',strIn
# skip if line is empty
if len(strIn)<1:
return False
lstIn = list()
for cc in lstCut:
strSubline =strIn[cc[0]-1:cc[1]-1].strip()
lstIn.append(strSubline)
print '>InSubline2>'+lstIn[len(lstIn)-1]+'<'
del strIn, lstCut,cc
print '>InReturLst>',lstIn
return lstIn



filIn = codecs.open(
strFileNameIn,
mode='r',
encoding='utf-8',
errors='strict',
buffering=1)
for linIn in filIn:
lstIn = CutLine2List(linIn,lstCutters)

--- Code end --

A sample output, representing one line from the input file looks like this:

 >InNextLine> I 30 2002-12-11 20:01:19.280 563 FANØ
2001-12-12-15.46.12.734502 2001-12-12-15.46.12.734502
 >InSubline2>I<
 >InSubline2>30<
 >InSubline2>2002-12-11 20:01:19.280<
 >InSubline2>563<
 >InSubline2>FANØ<
 >InSubline2>2001-12-12-15.46.12.73450<
 >InSubline2>2001-12-12-15.46.12.73450<
 >InReturLst> [u'I', u'30', u'2002-12-11 20:01:19.280', u'563',
u'FAN\xd8', u'2001-12-12-15.46.12.73450', u'2001-12-12-15.46.12.73450']


Question:
In the last printout, tagged >InReturLst> all entries turn into
uni-code. What happens here?
Look for the word 'FANØ'. This word changes from 'FANØ' to u'FAN\xd8' --
That's a problem to me, and I don't want it to change like this.

What do I do to stop this behavior?

Best Regards
Martin


If you don't want Unicode, why do you specify that the file is encoded 
as utf-8 ?  If it's ASCII, just open the file, without using a utf-8 
codec.  Of course, then you'll have to fix the input file to make it ASCII.


The character in the input file following the letters "FAN" is not a 
zero, it's some other character, apparently 00D8 in the Unicode table, 
not 0030.


It didn't "change" in the InRturLst line.  You were reading Unicode 
strings from the file.  When you print Unicode, it encodes it in 
whatever your console device specifies.  But when you print a "list," it 
uses repr() on the elements, so you get to see what their real type is.


DaveA

--
http://mail.python.org/mailman/listinfo/python-list

Re: Sending XML to a WEB Service and Getting Response Back

[Ian Kelly]

On Mon, Dec 20, 2010 at 11:58 AM, Anurag Chourasia
 wrote:
> Dear Python Mates,
> I have a requirement to send a XML Data to a WEB Service whose URL is of the
> form http://joule:8041/DteEnLinea/ws/EnvioGuia.jws
> I also have to read back the response returned as a result of sending this
> data to this WebService.
> This web service implements the following operations:
>    sendNCR
>
> This web service has no callbacks.
> I have pasted the complete WSDL for this WEB Service below my email.
> I would appreciate if someone could guide me with sample code using a Python
> Library suitable to fulfill this requirement of mine.

The ZSI or SOAPpy library should do what you need.
-- 
http://mail.python.org/mailman/listinfo/python-list

Class-override of a sort-key method?

[pythonlist . calin79]

Hi all - it would seem that these days, all the cool kids use the sort
function's 'key' kwarg in order to sort a list of custom objects quickly.

Unfortunately, as opposed to using 'cmp', where you can implent __cmp__ to
get 'automatic sorting' in a similar fashion, there doesn't seem to be a
direct analogue for a class-overridable method for providing a sort key.
 (ie, something like '__sortkey__' or '__key__').

Is there one, and I'm just missing it? If not, are there any plans to add
one? (I did a quick search of the PEP list, and the only hits for 'sort' I
saw had to do with sorting dictionaries by value).

- Paul
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Class-override of a sort-key method?

[Chris Rebert]

On Mon, Dec 20, 2010 at 3:23 PM,   wrote:
> Hi all - it would seem that these days, all the cool kids use the sort
> function's 'key' kwarg in order to sort a list of custom objects quickly.

Really? They don't bother to define __cmp__ or similar? Sounds lazy
and poorly structured.

> Unfortunately, as opposed to using 'cmp', where you can implent __cmp__ to
> get 'automatic sorting' in a similar fashion, there doesn't seem to be a
> direct analogue for a class-overridable method for providing a sort key.
>  (ie, something like '__sortkey__' or '__key__').

Just simply delegate to key comparison in your __cmp__ (or similar) method(s).

> Is there one, and I'm just missing it? If not, are there any plans to add
> one? (I did a quick search of the PEP list, and the only hits for 'sort' I
> saw had to do with sorting dictionaries by value).

If you know at class-definiton-time how you want instances to be
sorted, then just define __cmp__ (or the rich comparison methods)
appropriately, possibly even delegating to a comparison of keys (as
the class defines them).

For example:

from functools import total_ordering

@total_ordering
class Person(object):
def __init__(self, first_name, last_name):
self.first_name = first_name
self.last_name = last_name

@property
def _key(self):
"""Identifying key for a Person, by which they are sorted"""
return (self.last_name, self.first_name)

def __eq__(self, other):
return isinstance(other, Person) and self._key == other._key

def __lt__(self, other):
return self._key < other._key

If you want to abstract even this away, then just write a class
decorator; there's no need to add yet another (rather complicated due
to all the interactions with the existing comparison methods) special
method.

Cheers,
Chris
--
http://blog.rebertia.com
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Catching user switching and getting current active user from root on linux

[Steve Holden]

On 12/20/2010 12:54 PM, mpnordland wrote:
> I give up, I will never try to use a usenet group again. For the ones
> of you who tried to help thank you. You helped to identify some of my
> troubles, as for you @usernet, you are a troll

Don't give up after one experience. Usenet can be really useful as long
as you know who to listen to and who to ignore ...

regards
 Steve
-- 
Steve Holden   +1 571 484 6266   +1 800 494 3119
PyCon 2011 Atlanta March 9-17   http://us.pycon.org/
See Python Video!   http://python.mirocommunity.org/
Holden Web LLC http://www.holdenweb.com/
-- 
http://mail.python.org/mailman/listinfo/python-list

Redundant importing of modules

[Jshgwave]

When writing a function that uses a module such as NumPy, it is tempting to 
include the statement "import numpy" or "import numpy as np" in the definition 
of the function, in case the  function is used in a script that hasn't already 
imported NumPy.

That could lead to the script issuing the "import numpy" command more than once.

Does Python know to disregard redundant "import" commands?





 
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Class-override of a sort-key method?

[Steve Holden]

On 12/20/2010 7:10 PM, Chris Rebert wrote:
> On Mon, Dec 20, 2010 at 3:23 PM,   wrote:
>> Hi all - it would seem that these days, all the cool kids use the sort
>> function's 'key' kwarg in order to sort a list of custom objects quickly.
> 
> Really? They don't bother to define __cmp__ or similar? Sounds lazy
> and poorly structured.
> 
That sounds to me like a potentially ill-informed response.

>> Unfortunately, as opposed to using 'cmp', where you can implent __cmp__ to
>> get 'automatic sorting' in a similar fashion, there doesn't seem to be a
>> direct analogue for a class-overridable method for providing a sort key.
>>  (ie, something like '__sortkey__' or '__key__').
> 
Why do you talk about "implementing __cmp__"? Why should this be necessary?

> Just simply delegate to key comparison in your __cmp__ (or similar) method(s).
> 
Assuming, of course, that you are conveniently sorting only object over
which you have complete control ...

>> Is there one, and I'm just missing it? If not, are there any plans to add
>> one? (I did a quick search of the PEP list, and the only hits for 'sort' I
>> saw had to do with sorting dictionaries by value).
> 
> If you know at class-definiton-time how you want instances to be
> sorted, then just define __cmp__ (or the rich comparison methods)
> appropriately, possibly even delegating to a comparison of keys (as
> the class defines them).
> 
> For example:
> 
> from functools import total_ordering
> 
> @total_ordering
> class Person(object):
> def __init__(self, first_name, last_name):
> self.first_name = first_name
> self.last_name = last_name
> 
> @property
> def _key(self):
> """Identifying key for a Person, by which they are sorted"""
> return (self.last_name, self.first_name)
> 
> def __eq__(self, other):
> return isinstance(other, Person) and self._key == other._key
> 
> def __lt__(self, other):
> return self._key < other._key
> 
> If you want to abstract even this away, then just write a class
> decorator; there's no need to add yet another (rather complicated due
> to all the interactions with the existing comparison methods) special
> method.
> 
But the *real* point is (as the documentation attempts to point out)
that by providing the key argument it gets called only once per element,
whereas the cmp argument (or the objects' __cmp__() method if you insist
on letting sort delegate to that) gets called every time two objects
have to be compared. If cmp is a Python function (or equivalently if
__cmp__() is a Python method) then calling it will take much longer than
calling hte built-in default routines.

regards
 Steve
-- 
Steve Holden   +1 571 484 6266   +1 800 494 3119
PyCon 2011 Atlanta March 9-17   http://us.pycon.org/
See Python Video!   http://python.mirocommunity.org/
Holden Web LLC http://www.holdenweb.com/
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Redundant importing of modules

[Steve Holden]

On 12/20/2010 8:36 PM, Jshgwave wrote:
> When writing a function that uses a module such as NumPy, it is tempting
> to include the statement "import numpy" or "import numpy as np" in the
> definition of the function, in case the  function is used in a script
> that hasn't already imported NumPy.
> 
> That could lead to the script issuing the "import numpy" command more
> than once.
> 
> Does Python know to disregard redundant "import" commands?
> 
Oh, yes (as long as they are imported by the same name each time).

There's a dict at sys.modules that has a key for each loaded module's
name. When an attempt is made to import a module the first thing the
interpreter does is to look at sys.modules. If it has the correct key in
it then the assumption is that the module has already been imported, and
its namespace is made available as the module name immediately.

regards
 Steve
-- 
Steve Holden   +1 571 484 6266   +1 800 494 3119
PyCon 2011 Atlanta March 9-17   http://us.pycon.org/
See Python Video!   http://python.mirocommunity.org/
Holden Web LLC http://www.holdenweb.com/

-- 
http://mail.python.org/mailman/listinfo/python-list

Bug in fixed_point?!

[C Barrington-Leigh]

I cannot figure out what I'm doing wrong. The following does not
return a fixed point:


from scipy import optimize
xxroot= optimize.fixed_point(lambda xx: exp(-2.0*xx)/2.0, 1.0,
args=(), xtol=1e-12, maxiter=500)
print ' %f solves fixed point, ie f(%f)=%f ?'%
(xxroot,xxroot,exp(-2.0*xxroot)/2.0)

Chris
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Modifying an existing excel spreadsheet

[Stefan Sonnenberg-Carstens]

Am 20.12.2010 22:56, schrieb Ed Keith:

I have a user supplied 'template' Excel spreadsheet. I need to create a new 
excel spreadsheet based on the supplied template, with data filled in.

I found the tools here http://www.python-excel.org/,  and 
http://sourceforge.net/projects/pyexcelerator/. I have been trying to use the 
former, since the latter seems to be devoid of documentation (not even any 
docstrings).


My first thought was to copy the template, open the copy, modify it and save 
the modifications. But it looks like if I open an existing spreadsheet it must 
be read only.
Could you post some code ? Did you try a simple file copy or do you 
iterate over all the cells ?

  So I tried to  open the template, copy it to a new spreadsheet and write the 
new spreadsheet, but I can't seem to copy the images, and it looks like copying 
the formatting is going to be difficult.

Can anyone give me any tips or advice?

Thanks in advance,

-EdK

Ed Keith

e_...@yahoo.com



Blog: edkeith.blogspot.com




As long as your program only needs to run under windows,
COM automation is IMHO the best solution.
Python tells Excel what to do.
--
http://mail.python.org/mailman/listinfo/python-list

Re: Bug in fixed_point?!

[Terry Reedy]

On 12/20/2010 10:03 PM, C Barrington-Leigh wrote:

I cannot figure out what I'm doing wrong. The following does not
return a fixed point:


What did it do? For nearly all such questions, cut and paste actual 
output or traceback.



from scipy import optimize
xxroot= optimize.fixed_point(lambda xx: exp(-2.0*xx)/2.0, 1.0,
args=(), xtol=1e-12, maxiter=500)
print ' %f solves fixed point, ie f(%f)=%f ?'%
(xxroot,xxroot,exp(-2.0*xxroot)/2.0)


from math import exp
x = 1.0
for i in range(70):
print(repr(x))
x = exp(-2.0*x)/2.0

converges to 0.2835716452048919

Did you cut and paste what you actually ran?

--
Terry Jan Reedy

--
http://mail.python.org/mailman/listinfo/python-list

Re: Google AI challenge: planet war. Lisp won.

[Jon Harrop]

Wasn't that the "challenge" where they wouldn't even accept solutions 
written in many other languages (including both OCaml and F#)?


Cheers,
Jon.

"Xah Lee"  wrote in message 
news:44a8f48f-e291-463e-a042-d0cbc31a2...@z17g2000prz.googlegroups.com...

discovered this rather late.

Google has a AI Challenge: planet wars. http://ai-contest.com/index.php

it started sometimes 2 months ago and ended first this month.

the winner is Gábor Melis, with his code written in lisp.

Congrats lispers!

Gábor wrote a blog about it here
http://quotenil.com/Planet-Wars-Post-Mortem.html

(not sure if this has been mentioned here but quick search didn't find
it)

Xah ∑ http://xahlee.org/ ☄ 


--
http://mail.python.org/mailman/listinfo/python-list

Re: Sending XML to a WEB Service and Getting Response Back

[John Nagle]

On 12/20/2010 12:14 PM, Hidura wrote:

I recommend you use the urllib.request in the library of python says
everything that you want to know.

2010/12/20, Anurag Chourasia:

Dear Python Mates,

I have a requirement to send a XML Data to a WEB Service whose URL is of the
form http://joule:8041/DteEnLinea/ws/EnvioGuia.jws

I also have to read back the response returned as a result of sending this
data to this WebService.

This web service implements the following operations:
sendNCR

This web service has no callbacks.

I have pasted the complete WSDL for this WEB Service below my email.

I would appreciate if someone could guide me with sample code using a Python
Library suitable to fulfill this requirement of mine.

Regards,
Anurag


   If you're writing the client side, the service talks SOAP, and
you have a WSDL file, use the "suds" module.

   SOAPpy is way out of date.  The last update on SourceForge was in
2001.

John Nagle
--
http://mail.python.org/mailman/listinfo/python-list

Re: Google AI challenge: planet war. Lisp won.

[Jason Earl]

On Mon, Dec 20 2010, Jon Harrop wrote:

> Wasn't that the "challenge" where they wouldn't even accept solutions
> written in many other languages (including both OCaml and F#)?
>
> Cheers,
> Jon.

http://ai-contest.com/faq.php

Question: There is no starter package for my favorite language. What
  shall I do?

Answer: You don't know C++, Java, or Python? Okay fine. Tell the forums
what starter package you want to see, and we will try our best
to make it for you.

The folks working on Lisp submissions apparently created their own
starter package.  I am sure that something similar could have been done
for OCaml or F#.

That's probably too bad.  These types of competitions are good publicity
for less popular languages (assuming that the bots to well).

Jason
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Sending XML to a WEB Service and Getting Response Back

[Ian Kelly]

On 12/20/2010 11:34 PM, John Nagle wrote:

SOAPpy is way out of date. The last update on SourceForge was in
2001.


2007, actually:  http://sourceforge.net/projects/pywebsvcs/files/

And there is repository activity within the past 9 months.  Still, point 
taken.


--
http://mail.python.org/mailman/listinfo/python-list