On 3/25/10 4:28 AM, 甜瓜 wrote:
Howdy,Recently, I am finding a good library for build index on binary data. Xapian& Lucene for python binding focus on text digestion rather than binary data. Could anyone give me some recommendation? Is there any library for indexing binary data no matter whether it is written in python? In my case, there is a very big datatable which stores structured binary data, eg: struct Item { long id; // used as key double value; }; I want to build the index on "id" field to speed on searching. Since this datatable is not constant, the library should support incremental indexing. If there is no suitable library, I have to do the index by myself... Thank you in advance. -- ShenLei
Put it into an Sqlite database? Or something else from http://docs.python.org/library/persistence.html. Or maybe http://www.pytables.org/ is more suitable to your needs (never used that one myself though). Or install a bank or 2 of memory in your box and read everything into memory in one big hashtable.
Btw if you already have a big datatable in which the data is stored, I'm guessing that already is in some form of database format. Can't you write something that understands that database format.
But I think you need to provide some more details about your data set. -irmen -- http://mail.python.org/mailman/listinfo/python-list
