Re: RAMDirectory or Redis

2018-12-02 Thread Dawid Weiss
bq. We switched to ByteBuffersDirectory with 7.5, but I actually didn't see much performance improvements or savings in memory. Once the indexes are built I don't think there will be much of a difference. The core problem with RAMDirectory was related to synchronizations during merges/ file manipu

Re: RAMDirectory or Redis

2018-12-02 Thread Arjen van der Meijden
I doubt using Redis as directory-storage will be very good. I'd expect it to have much more latency for reads and writes compared to any of lucene's own directories. And Lucene probably won't like it if another Lucene-instance changes that database. It may be interesting as a result-level cache th

Re: Not-indexed, Stored Thumbnails or NoSQL?

2018-12-02 Thread Arjen van der Meijden
I'd think it depends on your application. If its a web-application and you're generating html, it may be better for the (client side) performance to have those images load via a webserver that can directly access the images as files (altough you could generate the images inline with base64). If it

Re: Not-indexed, Stored Thumbnails or NoSQL?

2018-12-02 Thread Uwe Schindler
Hi, It's perfectly fine to store binary blobs in Lucene. This does not affect performance of queries. The stored data is also compressed using LZ4. Just one thing: why the hell UUEncode? You can store binary blobs as is. Just pass a byte[] as stored field. There is one StoredField constructor t

Re: RAMDirectory or Redis

2018-12-02 Thread Adrien Grand
Hi Joe, Elasticsearch used to have an option to run with RAMDirectory but it was removed long ago: https://github.com/elastic/elasticsearch/pull/8536. The recommended approach is to use MMapDirectory and to give sufficient memory to the filesystem cache so that the most important data-structures

RAMDirectory or Redis

2018-12-02 Thread Joe MA
Greetings, Has anyone looked into using Redis or some other in-memory cache with Lucene? It seems that ElasticSearch may do this. Are there advantages to doing this versus, say, the RAMDirectory class? Thanks in advance, J

Not-indexed, Stored Thumbnails or NoSQL?

2018-12-02 Thread Joe MA
Greetings, I have an index where I import documents such as powerpoint, PDF, and so forth. One nice feature I added is that for each document, I store a thumbnail of the first page as an encoded String (uuencode) using a stored,not-indexed field. This thumbnail gets displayed when the user fi