Thanks for that Bill. I was wondering if that was the case, as my SAX knowledge is purely anecdotal and I haven't played with it. Thanks for the summary. I'll read up on that...
Ross. On May 6, 4:41 pm, Bill Freeman <ke1g...@gmail.com> wrote: > You may want to consider using a sax based parsing. It works by > calling you back for beginning of tag (has the attributes), text content, > and end of tag, from which you can keep state and just keep the, > you say small. subset of the element data that you need, which you > can either stick in a dom, or have your own object structure that you > know how to render for the requester. > > Much more coding than the dom, but you don't need the whole > document inmemoryat once. > > Bill > > > > On Thu, May 6, 2010 at 3:34 PM,Ross<scrodch...@yahoo.ca> wrote: > > Reading more it sounds like thememoryusage issue, particularly for > > those exploring Webfaction, is a typical rite of passage, and probably > > has more to do with httpdconf and mod_wsgi setup than it does with > > Django practices. Lots of discussion <a href ="http:// > > groups.google.ca/group/django-users/browse_thread/thread/ > > 2a82a6e2f0e457a5/b4d490773fddfe67?hl=en&lnk=gst&q=memory > > +usage#b4d490773fddfe67"> here.</a> (I hope this forum allows html, > > otherwise that's a messy link :) > > > On May 6, 3:14 pm, Javier Guerra Giraldez <jav...@guerrag.com> wrote: > >> On Thu, May 6, 2010 at 1:40 PM,Ross<scrodch...@yahoo.ca> wrote: > >> > Your thoughts appreciated. Is there a better way? I wouldn't want to > >> > stuff the XML-file resident content into a database and rebuild the > >> > XML element subsets because there are tons of files arriving and > >> > disappearing and it would become a huge background activity for > >> > another program. > > >> if your XML docs are too transient, then it's not a job for a > >> database. if you're getting repeated queries, or if several queries > >> need some xml elements repeatedly, you could simply use a cache to > >> avoid the repeats. > > > Thanks for the concurrence. You're prob right. I'll have to look into > > caching, but the recurrence of the same slice of the XML file could be > > fairly infrequent. But it's worth looking into. > > >> If each xml element is used very few times, but you think a good > >> proportion of them is going to be required before becoming outdated, > >> then you could preprocess them as they arrive, instead of waiting for > >> the request. i guess that processing a whole document would produce > >> several results much faster than searching each individually. to > >> avoid the load on a RDBMS, you might find better to store the > >> preprocessed results on a key/value store, maybe even a > >>memory-resident one, like redis. > > > Yes, there might be some opportunity to pre-process as a backend > > process. Will keep that in mind as an option. > > >> -- > >> Javier > > > -- > > You received this message because you are subscribed to the Google Groups > > "Django users" group. > > To post to this group, send email to django-us...@googlegroups.com. > > To unsubscribe from this group, send email to > > django-users+unsubscr...@googlegroups.com. > > For more options, visit this group > > athttp://groups.google.com/group/django-users?hl=en. > > -- > You received this message because you are subscribed to the Google Groups > "Django users" group. > To post to this group, send email to django-us...@googlegroups.com. > To unsubscribe from this group, send email to > django-users+unsubscr...@googlegroups.com. > For more options, visit this group > athttp://groups.google.com/group/django-users?hl=en. -- You received this message because you are subscribed to the Google Groups "Django users" group. To post to this group, send email to django-us...@googlegroups.com. To unsubscribe from this group, send email to django-users+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-users?hl=en.
