"Christoph Haas" <[EMAIL PROTECTED]> wrote"
| On Wednesday 02 August 2006 22:24, Christoph Haas wrote: | > I have written an application in Perl some time ago (I was young and | > needed the money) that parses multiple large text files containing | > nested data structures and allows the user to run quick queries on the | > data. [...] | | I suppose my former posting was too long and concrete. So allow me to try | it in a different way. :) | | The situation is that I have input data that take ~1 minute to parse while | the users need to run queries on that within seconds. I can think of two | ways: | | (1) Database | (very quick, but the input data is deeply nested and it would be | ugly to convert it into some relational shape for the database) | (2) cPickle | (Read the data every now and then, parse it, write the nested Python | data structure into a pickled file. The let the other application | that does the queries unpickle the variable and use it time and | again.) | | So the question is: would you rather force the data into a relational | database and write object-relational wrappers around it? Or would you | pickle it and load it later and work on the data? The latter application | is currently a CGI. I'm open to whatever. :) | | Thanks for any enlightenment. | | Christoph Not sure if this is of any use - but I have noticed that dict lookups in Python is blindingly fast - but it seems to me to use them would be as much trouble to you as converting the data into a shape for the database in third normal form... Unless of course your parser is already doing something like that... There are also some fancy packages for tree structures around, but I don't know anything useful about them. - Hendrik -- http://mail.python.org/mailman/listinfo/python-list
