Re: Lucandra Limitations

2011-01-27 Thread Jake Luciani
The latest iteration of Lucandra, called Solandra, creates localized sub-indexes of size N and spreads them around the cassandra ring. Then using solr, will behind the scenes search all the subindexes in parallel. This approach should give you what you need and it would be great to have such a larg

Re: Lucandra Limitations

2011-01-27 Thread David G. Boney
I am new to Lucene and Lucandra. My use case is that I have a trillion URIs to index with Lucene. Each URI is either a resource or literal in an RDF graph. Each URI is a document for Lucene If I were using Lucene, my understanding is that it would create a segment, stuff as many URIs in the seg

Re: Lucandra Limitations

2011-01-27 Thread Jake Luciani
Yes, but that's also the lucene limit http://lucene.apache.org/java/3_0_1/fileformats.html#Limitations "Lucene uses a Java int to refer to document numbers, and the index file format uses an Int32" On Thu, Jan 27, 2011 at 1:40 PM, David G. Boney < dbon...@semanticartifacts.com> wrote: > I was

Re: Lucandra Limitations

2011-01-27 Thread Paul Brown
Lucene trades on (32-bit) ints internally, so I expect you're just seeing a projection of that limitation. On Jan 27, 2011, at 10:40 AM, David G. Boney wrote: > I was reviewing the Lucandra schema presented on the below page at Datastax: > > http://www.datastax.com/docs/0.7/data_model/lucandra

Lucandra Limitations

2011-01-27 Thread David G. Boney
I was reviewing the Lucandra schema presented on the below page at Datastax: http://www.datastax.com/docs/0.7/data_model/lucandra In the TermInfo Super Column Family, docID is the key for a supercolumn. Does this imply that the maximum number of documents that can be index for a term with Lucan