Re: searching and storing large quantities of xml!

2010-01-19 Thread Stefan Behnel
dads, 18.01.2010 22:39: > There was one thing that I forgot about - when ElementTree fails to > parse due to an element not being closed why doesn't it close the file > like object. Because it didn't open it? Stefan -- http://mail.python.org/mailman/listinfo/python-list

Re: searching and storing large quantities of xml!

2010-01-18 Thread MRAB
dads wrote: [snip] import os.path import shutil import zlib import os There's no need to import both os.path and os. Import just os; you can still refer to os.path in the rest of the code. [snip] def _mkMonthAndDaysDirs(self): ''' creates dirs for every month and day of a of sp

Re: searching and storing large quantities of xml!

2010-01-18 Thread dads
Thanks all, took your advice and have been playing all weekend which has been great fun. ElementTree is awesome. I created a script that organises the xml as they're in year blocks and I didn't realise the required xml is mixed up with other xml. Plus the volumes are much greater than I realised, I

Re: searching and storing large quantities of xml!

2010-01-16 Thread Stefan Behnel
dads, 16.01.2010 19:10: > I work in as 1st line support and python is one of my hobbies. We get > quite a few requests for xml from our website and its a long strung > out process. So I thought I'd try and create a system that deals with > it for fun. > > I've been tidying up the archived xml and

Re: searching and storing large quantities of xml!

2010-01-16 Thread Paul Rubin
dads writes: > I've been tidying up the archived xml and have been thinking what's > the best way to approach this issue as it took a long time to deal > with big quantities of xml. If you have 5/6 years worth of 26000+ > 5-20k xml files per year. The archived stuff is zipped but what is > better,

Re: searching and storing large quantities of xml!

2010-01-16 Thread Steve Holden
dads wrote: > I work in as 1st line support and python is one of my hobbies. We get > quite a few requests for xml from our website and its a long strung > out process. So I thought I'd try and create a system that deals with > it for fun. > > I've been tidying up the archived xml and have been th

searching and storing large quantities of xml!

2010-01-16 Thread dads
I work in as 1st line support and python is one of my hobbies. We get quite a few requests for xml from our website and its a long strung out process. So I thought I'd try and create a system that deals with it for fun. I've been tidying up the archived xml and have been thinking what's the best w