Re: using sqlite3 - execute vs. executemany; committing ...

2008-05-07 Thread Vlastimil Brom
Hi David, thanks for your suggestions. The mentioned line of code is actually already a part of such a searching method, thus it is beeing called in a similar way you suggested, but anyhow it catched my eye ...; Thanks for the references to different markup systems, I haven't known many of them b

Re: mxTexTools (Re: using sqlite3 - execute vs. executemany; committing ...)

2008-05-07 Thread Vlastimil Brom
2008/5/7, M.-A. Lemburg <[EMAIL PROTECTED]>: > > On 2008-05-07 00:07, Vlastimil Brom wrote: > > > Thanks for reminding me about > > the mxTextTools; I looked at this package very quickly several months > > ago and it seemed quite > > complex and heavy-weight, but maybe I will reconsider this after

mxTexTools (Re: using sqlite3 - execute vs. executemany; committing ...)

2008-05-07 Thread M.-A. Lemburg
On 2008-05-07 00:07, Vlastimil Brom wrote: Thanks for reminding me about the mxTextTools; I looked at this package very quickly several months ago and it seemed quite complex and heavy-weight, but maybe I will reconsider this after some investigation ... mxTextTools gives you almost full C spee

Re: using sqlite3 - execute vs. executemany; committing ...

2008-05-07 Thread David
> > Google for "overlapping markup" and "python overlapping markup" for > more information. Here's one interesting page I found: > I found an interesting markup called LMNL, which supports overlapping markup: http://lmnl.net/ Also, a survey which compares various markup languages, including LM

Re: using sqlite3 - execute vs. executemany; committing ...

2008-05-07 Thread David
Hi Vlasta. > > tags_lookups[tag][item_dict[tag]] = tags_lookups[tag].get(item_dict[tag], > set()) | set([idx]) > > I thought, whether I am not overestimating myself with respect to the future > maintaining of the code ... :-) Here's a suggestion for readability and maintainability: Make a search

Re: using sqlite3 - execute vs. executemany; committing ...

2008-05-06 Thread Vlastimil Brom
Hi David, thanks for your comments and hints, the proposed approach with a list of dicts lookup dict is indeed much faster, than my previous attempts with a database (even without psyco). I used a slightly different structure with sets of indices, since they should be unique anyway and the values a

Re: using sqlite3 - execute vs. executemany; committing ...

2008-05-06 Thread David
Hi Vlasta. I had a look at your original mail. I think your simpler (than XML) format is a good idea for now. At a later stage you could change it to something like this: text goes here, some more text text goes here, some more text And so on. This isn't much more verbose than your current synt

Re: using sqlite3 - execute vs. executemany; committing ...

2008-05-05 Thread David
Hi Vlasta. Don't have time to reply to your mail in detail now. Will do so later. Have you tried the eGenix text tagging library? It sounds to me that this is exactly what you need: http://www.egenix.com/products/python/mxBase/mxTextTools/ Disclaimer: I've never actually used this library ;-)

Re: using sqlite3 - execute vs. executemany; committing ...

2008-05-05 Thread Vlastimil Brom
> > > Thanks for further comments David, You are right, the choice of an appropriate data structure isn't easy; I described the requirements in an earlier post: http://mail.python.org/pipermail/python-list/2007-December/469506.html Here is a slightly modified sample of the text source I currently

Re: using sqlite3 - execute vs. executemany; committing ...

2008-05-04 Thread David
> > > Thank you very much for your comprehensive answer and detailed informations, > David; I really appreciate it! You're welcome. > > As for the number of items, there would be approx. 34 000 calls of execute() > in my present code, in the final version probably more; I think executmany > is mo

Re: using sqlite3 - execute vs. executemany; committing ...

2008-05-04 Thread Vlastimil Brom
> > > Thank you very much for your comprehensive answer and detailed informations, David; I really appreciate it! As for the number of items, there would be approx. 34 000 calls of execute() in my present code, in the final version probably more; I think executmany is more efficient here, if there

Re: using sqlite3 - execute vs. executemany; committing ...

2008-05-04 Thread David
> - Are there any peculiarities with using curs.executemany(...) vs. multiple > curs.execute(...) ? I read a notice, sqlite3 does internally some caching, > hence both should be similarly fast, but in my case executemany(...) is > quite a bit faster How many times are you calling execute vs a sing

using sqlite3 - execute vs. executemany; committing ...

2008-05-03 Thread Vlastimil Brom
Hi all, I'd like to ask about some (probably elementary) things about the proper usage of sqlite3 in python (2.5.2; win). - Are there any peculiarities with using curs.executemany(...) vs. multiple curs.execute(...) ? I read a notice, sqlite3 does internally some caching, hence both should be simil