Re: How can I know the lucene index version from files

e > > > https://lucene.apache.org/core/4_9_0/demo/overview-summary.html#Searching_Files > > > > Or write a snippet of code, which opens a Directory\IndexReader and then > > print it to console that should answer your questions. > > > >> On Sun, Mar 2, 2025 at 12:21 AM

Re: How can I know the lucene index version from files

> On Sun, Mar 2, 2025 at 12:21 AM Daniel Cerqueira >>> wrote: >>> >>> I have this lucene index files, in a directory: >>> >>> ``` >>> $ ls >>> _1p.fdt _1p.fdx _1p.fnm _1p_Lucene41_0.doc _1p_Lucene41_0.pos >>> _1p_Lucene

Re: How can I know the lucene index version from files

> On Sun, Mar 2, 2025 at 12:21 AM Daniel Cerqueira > wrote: > >> I have this lucene index files, in a directory: >> >> ``` >> $ ls >> _1p.fdt _1p.fdx _1p.fnm _1p_Lucene41_0.doc _1p_Lucene41_0.pos >> _1p_Lucene41_0.tim _1p_Lucene41_0.tip _1p.nvd

Re: How can I know the lucene index version from files

n > print it to console that should answer your questions. > >> On Sun, Mar 2, 2025 at 12:21 AM Daniel Cerqueira >> wrote: >> >> I have this lucene index files, in a directory: >> >> ``` >> $ ls >> _1p.fdt _1p.fdx _1p.fnm _1p_Lucene41_0.do

Re: How can I know the lucene index version from files

print it to console that should answer your questions. On Sun, Mar 2, 2025 at 12:21 AM Daniel Cerqueira wrote: > I have this lucene index files, in a directory: > > ``` > $ ls > _1p.fdt _1p.fdx _1p.fnm _1p_Lucene41_0.doc _1p_Lucene41_0.pos > _1p_Lucene41_0.tim _1p_Lucene41_0.

How can I know the lucene index version from files

I have this lucene index files, in a directory: ``` $ ls _1p.fdt _1p.fdx _1p.fnm _1p_Lucene41_0.doc _1p_Lucene41_0.pos _1p_Lucene41_0.tim _1p_Lucene41_0.tip _1p.nvd _1p.nvm _1p.si segments_1 segments.gen write.lock ``` - How can I know which is the version of this lucene index

Re: Get knowledge about apache lucene index migrate

Maybe a silly question: is it feasible (on your scale) to rebuild your index from your source of truth data? Thanks! On Tue, Aug 6, 2024 at 2:11 PM Michael Sokolov wrote: > Yes, there is no support for upgrading a pre-8.x index to 9 or later. > At some point it was decided that supporting that

Re: Get knowledge about apache lucene index migrate

Yes, there is no support for upgrading a pre-8.x index to 9 or later. At some point it was decided that supporting that would lead to grief for users and/or hamper development of Lucene, so now you can only upgrade one major version. If you need to do so, the best supported option is to write a pro

Get knowledge about apache lucene index migrate

Hello I am currently working on a project that is using apache lucene 4.1.0 version. Now I need to upgrade that version to 9.11.1. So I configure the imports and configure the codebase according to the new lucene version. Now I need to upgrade existing indexes which have been created using lucene

Re: Lucene Index Writer in a distributed system

Zookeeper, right? Look how Zookeeper is used in Solr, but Zookeeper does exactly what you want, I believe. Sent from my iPhone > On Oct 19, 2023, at 3:49 AM, Gopal Sharma wrote: > > Hello Team, > > I am new to Lucene and want to use Lucene in a distributed system to write > in a Amazon EFS i

Re: Lucene Index Writer in a distributed system

Hi Gopal, Indeed, for a single Lucene index, only one writer may be open at a time. Lucene tries to catch you if you mess this up, using file-based locking. If you really need concurrent indexing, you could have N IndexWriters each writing into a private Directory, and then periodically use

Lucene Index Writer in a distributed system

Hello Team, I am new to Lucene and want to use Lucene in a distributed system to write in a Amazon EFS index. As per my understanding, the index writer for a particular index needs to be opened by 1 server only. Is there a way we can achieved this in distributed system to write parallelly in Luce

Lucene index FileAlreadyExistsException

Hi Lucene users, I am a new bee to lucene. Executing first out of box demo application Index.java with following command line parameters. -index C:\\Users\\Manju\\eclipse-workspace\\Lucene\\Index_Lucene\\560016 -docs C:\\Users\\Manju\\eclipse-workspace\\Lucene\\sales\\Sales.csv -knn_dict C:\\Use

Re: How to handle corrupt Lucene index

If you are looking at the files in hex, you can see the file format docs online for your version: https://lucene.apache.org/core/7_3_0/core/org/apache/lucene/index/SegmentInfos.html SegID is written right after SegName, it is 16 bytes (128-bit number) On Wed, Apr 13, 2022 at 10:59 PM Robert Muir

Re: How to handle corrupt Lucene index

egment (e.g. because files got > mixed up), there's a clear error this way that files are mixed up. > otherwise, without this check, you get pure insanity trying to debug > problems when files get mixed up. > > On Wed, Apr 13, 2022 at 10:39 PM Tim Whittington wrote: > > > &

Re: How to handle corrupt Lucene index

get mixed up. On Wed, Apr 13, 2022 at 10:39 PM Tim Whittington wrote: > > Using a known-broken Lucene index directory, I dropped down to the Lucene > API and tracked this down a bit further. > > My directory listing is this: > > > 17 Mar 13:39 _8w.fdt &

Re: How to handle corrupt Lucene index

Using a known-broken Lucene index directory, I dropped down to the Lucene API and tracked this down a bit further. My directory listing is this: 17 Mar 13:39 _8w.fdt 17 Mar 13:39 _8w.fdx 17 Mar 13:39 _8w.fnm 17 Mar 13:39 _8w.nvd 17 Mar 13:39 _8w.nvm 17 Mar 13:39 _8w.si 17 Mar 13

Re: How to handle corrupt Lucene index

corrupt Lucene index Thanks for this - I'll have a look at the database server code that is managing the Lucene indexes and see if I can track it down. Tim On Thu, 14 Apr 2022 at 12:41, Robert Muir wrote: > On Wed, Apr 13, 2022 at 8:24 PM Tim Whittington > wrote: > > > >

Re: How to handle corrupt Lucene index

That is a good practice and i pointed out that since i saw lucene 7.0 in the stack trace. Best regards From: Tim Whittington Sent: Wednesday, April 13, 2022 9:15 PM To: java-user@lucene.apache.org Subject: Re: How to handle corrupt Lucene index To be clear

Re: How to handle corrupt Lucene index

ile mismatch" errors > when > > the IndexWriter is constructed. > > > > In all of the cases this has occurred, CheckIndex finds no issues with > the > > Lucene index. > > > > The database has write-ahead-log and recovery facilities, so making the > > Lu

Re: How to handle corrupt Lucene index

tj Lucene. > > Can you maybe recreate lucene index on 7.3.0? > > Also how do you use database in your scenario? > Are you using jdbc like operations like in Oracle database? lucene > operations are independent of database operations. > > Best regards >

Re: How to handle corrupt Lucene index

In my experience that if you built index at version x then use index also in version x. I never encountered any problems this way witj Lucene. Can you maybe recreate lucene index on 7.3.0? Also how do you use database in your scenario? Are you using jdbc like operations like in Oracle database

Re: How to handle corrupt Lucene index

e database , resulting in > "org.apache.lucene.index.CorruptIndexException: file mismatch" errors when > the IndexWriter is constructed. > > In all of the cases this has occurred, CheckIndex finds no issues with the > Lucene index. > > The database has write-ahead-log and recovery facilit

How to handle corrupt Lucene index

when the IndexWriter is constructed. In all of the cases this has occurred, CheckIndex finds no issues with the Lucene index. The database has write-ahead-log and recovery facilities, so making the Lucene indexes durable wrt database operations is doable, but in this case the IndexWriter

Reg: Housekeeping Lucene index files

Hi All, I am facing the problem lucene index files occupy more memory space in my machine. So I want to purge lucene index files automatically based on the configured time period. Is there a way to do this through any of the lucene methods,class or purging policy configuration. You

Re: Need approach to store JSON data in Lucene index

In general, the preferred approach is denormalizing, but your description suggests that you want to be able to query anything: actions, tasks, test cases, etc. so I guess that the most natural approach would be to leverage Lucene's support for index-time joins, see the documentation of the join pac

Need approach to store JSON data in Lucene index

Hi Team, We are using Lucene Java library in our organization to store JSON files data into to Lucene indexes. Our JSON file are structured in below format. 1. Testcase has several Testcase steps 2. Testcase has several Tasks 3. Tasks has task step 4. Task step has Actions and obje

Re: MMapDirectory vs In Memory Lucene Index (i.e., ByteBuffersDirectory)

So, just cat will do this. Thanks From: Robert Muir Sent: Tuesday, February 23, 2021 4:45 PM To: Baris Kazar Cc: java-user Subject: Re: MMapDirectory vs In Memory Lucene Index (i.e., ByteBuffersDirectory) The preload isn't magical. It only "reads in

Re: MMapDirectory vs In Memory Lucene Index (i.e., ByteBuffersDirectory)

e url / site that i can look at for preload? > > Thanks for the explanations. This thread will be useful for many folks i > believe. > > Best regards > > > On 2/23/21 4:15 PM, Robert Muir wrote: > > > > On Tue, Feb 23, 2021 at 4:07 PM wrote: > >> What i w

Re: MMapDirectory vs In Memory Lucene Index (i.e., ByteBuffersDirectory)

ve. Best regards On 2/23/21 4:15 PM, Robert Muir wrote: On Tue, Feb 23, 2021 at 4:07 PM <mailto:baris.ka...@oracle.com>> wrote: What i want to achieve: Problem statement: base case is disk based Lucene index with FSDirectory speedup case was supposed to be in memor

Re: MMapDirectory vs In Memory Lucene Index (i.e., ByteBuffersDirectory)

On Tue, Feb 23, 2021 at 4:07 PM wrote: > What i want to achieve: Problem statement: > > base case is disk based Lucene index with FSDirectory > > speedup case was supposed to be in memory Lucene index with MMapDirectory > On 64-bit systems, FSDirectory just invokes MMapDirecto

Re: MMapDirectory vs In Memory Lucene Index (i.e., ByteBuffersDirectory)

better in that, too: ie, cold start. What i want to achieve: Problem statement: base case is disk based Lucene index with FSDirectory speedup case was supposed to be in memory Lucene index with MMapDirectory Uwe mentioned tmpfs will help. i will try that next. I thought preload was not

Re: MMapDirectory vs In Memory Lucene Index (i.e., ByteBuffersDirectory)

Lucene index with FSDirectory speedup case was supposed to be in memory Lucene index with MMapDirectory Uwe mentioned tmpfs will help. i will try that next. Thanks On 2/23/21 3:54 PM, Robert Muir wrote: speedup over what? You are probably already using MMapDirectory (it is the default). So I

Re: MMapDirectory vs In Memory Lucene Index (i.e., ByteBuffersDirectory)

speedup over what? You are probably already using MMapDirectory (it is the default). So I don't know what you are trying to achieve, but giving lots of memory to your java process is not going to help. If you just want to prevent the first few queries to a fresh cold machine instance from being sl

Re: MMapDirectory vs In Memory Lucene Index (i.e., ByteBuffersDirectory)

Thanks but then how will MMapDirectory help gain speedup? i will try tmpfs and see what happens. i was expecting to get on order of magnitude of speedup from already very fast on disk Lucene indexes. So i was expecting really really really fast response with MMapDirectory. Thanks On 2/23/21

Re: MMapDirectory vs In Memory Lucene Index (i.e., ByteBuffersDirectory)

Don't give gobs of memory to your java process, you will just make things slower. The kernel will cache your index files. On Tue, Feb 23, 2021 at 1:45 PM wrote: > Ok, but how is this MMapDirectory used then? > > Best regards > > > On 2/23/21 7:03 AM, Robert Muir wrote: > > > > > > On Tue, Feb 23

Re: MMapDirectory vs In Memory Lucene Index (i.e., ByteBuffersDirectory)

As Uwe suggested some time ago, tmpfs file system usage with MMapDirectory is the only way to get high speedup wrt on disk Lucene index, right? Best regards On 2/23/21 1:44 PM, baris.ka...@oracle.com wrote: Ok, but how is this MMapDirectory used then? Best regards On 2/23/21 7:03 AM

Re: MMapDirectory vs In Memory Lucene Index (i.e., ByteBuffersDirectory)

Ok, but how is this MMapDirectory used then? Best regards On 2/23/21 7:03 AM, Robert Muir wrote: On Tue, Feb 23, 2021 at 2:30 AM > wrote: Hi,-   I tried MMapDirectory and i allocated as big as index size on my J2EE Container but Don't alloc

Re: MMapDirectory vs In Memory Lucene Index (i.e., ByteBuffersDirectory)

On Tue, Feb 23, 2021 at 2:30 AM wrote: > Hi,- > > I tried MMapDirectory and i allocated as big as index size on my J2EE > Container but > > Don't allocate java heap memory for the index, MMapDirectory does not use java heap memory!

Re: MMapDirectory vs In Memory Lucene Index (i.e., ByteBuffersDirectory)

Hi,-  I tried MMapDirectory and i allocated as big as index size on my J2EE Container but it only gives me at most 25% speedup and even sometimes a small amount of slowdown. How can i effectively use Lucene indexes in memory? Best regards On 12/14/20 6:35 PM, baris.ka...@oracle.com wrote

Re: MMapDirectory vs In Memory Lucene Index (i.e., ByteBuffersDirectory)

Thanks Robert. I think these valuable comments need to be placed on javadocs for future references. i think i am getting enough info for making a decision: i will use MMapDirectory without setPreload and i hope my index will fit into the RAM. i plan to post a blog for findings. Best regar

Re: MMapDirectory vs In Memory Lucene Index (i.e., ByteBuffersDirectory)

On Mon, Dec 14, 2020 at 1:59 PM Uwe Schindler wrote: > > Hi, > > as writer of the original bog post, here my comments: > > Yes, MMapDirectory.setPreload() is the feature mentioned in my blog post is > to load everything into memory - but that does not guarantee anything! > Still, I would not recom

Re: MMapDirectory vs In Memory Lucene Index (i.e., ByteBuffersDirectory)

ctory dir = new MMapDirectory(Paths.get(indexDir)); // Uses FSLockFactory.getDefault() and DEFAULT_MAX_CHUNK_SIZE which is 1GB if (dir.getPreload() == false) dir.setPreload(Constants.PRELOAD_YES); // In-Memory Lucene Index enabled-> *commented out* IndexReader reader = DirectoryReader.open(dir);

RE: MMapDirectory vs In Memory Lucene Index (i.e., ByteBuffersDirectory)

: > > > /*First way: Using constructor (without setPreload) :*/ > > MMapDirectory dir = new MMapDirectory(Paths.get(indexDir)); // Uses > FSLockFactory.getDefault() and DEFAULT_MAX_CHUNK_SIZE which is 1GB > if (dir.getPreload() == false) > dir.setPreload(Constants.PREL

Re: MMapDirectory vs In Memory Lucene Index (i.e., ByteBuffersDirectory)

l1TtPJMV80mkA-w$ eMail: u...@thetaphi.de -Original Message- From: baris.ka...@oracle.com Sent: Sunday, December 13, 2020 10:18 PM To: java-user@lucene.apache.org Cc: BARIS KAZAR Subject: MMapDirectory vs In Memory Lucene Index (i.e., ByteBuffersDirectory) Hi,- it would be nice to

Re: MMapDirectory vs In Memory Lucene Index (i.e., ByteBuffersDirectory)

); // Uses FSLockFactory.getDefault() and DEFAULT_MAX_CHUNK_SIZE which is 1GB if (dir.getPreload() == false)   dir.setPreload(Constants.PRELOAD_YES); // In-Memory Lucene Index enabled-> *commented out* IndexReader reader = DirectoryReader.open(dir); ... /*Second way: Or using open (witho

Re: MMapDirectory vs In Memory Lucene Index (i.e., ByteBuffersDirectory)

...@thetaphi.de > > > -Original Message- > > From: baris.ka...@oracle.com > > Sent: Sunday, December 13, 2020 10:18 PM > > To: java-user@lucene.apache.org > > Cc: BARIS KAZAR > > Subject: MMapDirectory vs In Memory Lucene Index (i.e., > ByteBuffersDirectory)

RE: MMapDirectory vs In Memory Lucene Index (i.e., ByteBuffersDirectory)

.ka...@oracle.com > Sent: Sunday, December 13, 2020 10:18 PM > To: java-user@lucene.apache.org > Cc: BARIS KAZAR > Subject: MMapDirectory vs In Memory Lucene Index (i.e., ByteBuffersDirectory) > > Hi,- > > it would be nice to create a Lucene index in files and then effect

Re: MMapDirectory vs In Memory Lucene Index (i.e., ByteBuffersDirectory)

-Lr5mA$ On Sun, Dec 13, 2020 at 4:18 PM wrote: Hi,- it would be nice to create a Lucene index in files and then effectively load it into memory once (since i use in read-only mode). I am looking into if this is doable in Lucene. i wish there were an option to load whole Lucene index into memory

Re: MMapDirectory vs In Memory Lucene Index (i.e., ByteBuffersDirectory)

gt; load your index. > > > > Mike McCandless > > > > > https://urldefense.com/v3/__http://blog.mikemccandless.com__;!!GqivPVa7Brio!LEQH8Tyb_BBN_Kc3fEH2w-yhpvS-VwMrpuB0gctqchp3j7L7V6x9piciHOJn-Lr5mA$ > > > > > > On Sun, Dec 13, 2020 at 4:18 PM wrote: > > &

Re: MMapDirectory vs In Memory Lucene Index (i.e., ByteBuffersDirectory)

index in files and then effectively load it into memory once (since i use in read-only mode). I am looking into if this is doable in Lucene. i wish there were an option to load whole Lucene index into memory: Both of below urls have links to the blog url where i quoted a very nice section: https

Re: MMapDirectory vs In Memory Lucene Index (i.e., ByteBuffersDirectory)

use to load your index. Mike McCandless http://blog.mikemccandless.com On Sun, Dec 13, 2020 at 4:18 PM wrote: > Hi,- > > it would be nice to create a Lucene index in files and then effectively > load it into memory once (since i use in read-only mode). I am looking into > if th

MMapDirectory vs In Memory Lucene Index (i.e., ByteBuffersDirectory)

Hi,- it would be nice to create a Lucene index in files and then effectively load it into memory once (since i use in read-only mode). I am looking into if this is doable in Lucene. i wish there were an option to load whole Lucene index into memory: Both of below urls have links to the blog

[LUCENE INDEX PARTIAL UPDATE] Update SortedSetDocValuesFacetField

Hi all, we have an index with many fields including some of type " *SortedSetDocValuesFacetField*". We have a scenario where only one field of type *SortedSetDocValuesFacetField* changes, so we would like to *update only this field *and not create from scratch the entire lucene document. We have t

Re: How to tell Lucene index search to stop when it takes too long

Will do, Thanks On Feb 25, 2020, at 1:34 AM, Mikhail Khludnev wrote: Hello. Meet org.apache.lucene.search.TimeLimitingCollector. On Mon, Feb 24, 2020 at 2:51 PM wrote: Hi,- I hope everyone is doing great. i am trying to find an api to tell Lucene Index Searcher to stop after 0.5 seconds

Re: How to tell Lucene index search to stop when it takes too long

I hope everyone is doing great. i am trying to find an api to tell Lucene Index Searcher to stop after 0.5 seconds (when it takes longer than this). Is there such an api or plan to implement one? Best regards - To unsubscribe

Re: How to tell Lucene index search to stop when it takes too long

gt; > Best regards > > > > On 2/25/20 1:50 AM, baris.ka...@oracle.com wrote: > > Will do, Thanks > > > > > >> On Feb 25, 2020, at 1:34 AM, Mikhail Khludnev wrote: > >> > >> Hello. > >> > >> Meet org.apache.lucene.search.T

Re: How to tell Lucene index search to stop when it takes too long

AM, baris.ka...@oracle.com wrote: Will do, Thanks On Feb 25, 2020, at 1:34 AM, Mikhail Khludnev wrote: Hello. Meet org.apache.lucene.search.TimeLimitingCollector. On Mon, Feb 24, 2020 at 2:51 PM wrote: Hi,- I hope everyone is doing great. i am trying to find an api to tell Lucene Index Searc

Re: How to tell Lucene index search to stop when it takes too long

Will do, Thanks > On Feb 25, 2020, at 1:34 AM, Mikhail Khludnev wrote: > > Hello. > > Meet org.apache.lucene.search.TimeLimitingCollector. > >> On Mon, Feb 24, 2020 at 2:51 PM wrote: >> >> Hi,- >> >> I hope everyone is doing great. >

Re: How to tell Lucene index search to stop when it takes too long

Hello. Meet org.apache.lucene.search.TimeLimitingCollector. On Mon, Feb 24, 2020 at 2:51 PM wrote: > Hi,- > > I hope everyone is doing great. > > > i am trying to find an api to tell Lucene Index Searcher to stop after > 0.5 seconds (when it takes longer than this). > &

How to tell Lucene index search to stop when it takes too long

Hi,- I hope everyone is doing great. i am trying to find an api to tell Lucene Index Searcher to stop after 0.5 seconds (when it takes longer than this). Is there such an api or plan to implement one? Best regards - To

RE: Get distinct fields values from lucene index

E: Get distinct fields values from lucene index Hello Michael, Thanks for the response, I have tried the approach suggested by you(TermsEnum) but it is not working for me. I have used below code. String field = "address"; try (IndexReader reader = Utils.getIndexReader(indexDirectoryP

RE: Get distinct fields values from lucene index

; } Sent from Mail<https://go.microsoft.com/fwlink/?LinkId=550986> for Windows 10 From: Michael Sokolov Sent: Friday, November 22, 2019 8:11:25 PM To: java-user@lucene.apache.org Subject: Re: Get distinct fields values from lucene index In Solr and ES t

Re: Get distinct fields values from lucene index

1, 2019 at 1:15 AM Amol Suryawanshi wrote: > > Hello, > > I am using lucene in my organization. I want to know how can I get distinct > values from lucene index. I have tried “GroupingSearch” API but it doesn’t > serves the purpose. It will give all documents contains distinct

Get distinct fields values from lucene index

Hello, I am using lucene in my organization. I want to know how can I get distinct values from lucene index. I have tried “GroupingSearch” API but it doesn’t serves the purpose. It will give all documents contains distinct values. I have used below code. final GroupingSearch groupingSearch

Re: Lucene index directory grows and shrinks

you very much > > -Messaggio originale- > Da: Raffaele Gambelli > Inviato: lunedì 4 novembre 2019 15:16 > A: java-user@lucene.apache.org > Oggetto: R: Lucene index directory grows and shrinks > > Thanks for your quick reply, I'm quite a beginner in Luce

R: Lucene index directory grows and shrinks

le- Da: Raffaele Gambelli Inviato: lunedì 4 novembre 2019 15:16 A: java-user@lucene.apache.org Oggetto: R: Lucene index directory grows and shrinks Thanks for your quick reply, I'm quite a beginner in Lucene concepts, Jackrabbit hides almost all about the way it uses Lucene internally. Anyway

R: Lucene index directory grows and shrinks

Search_115513504.html Max segment I don't know, where could I see it? Bye -Messaggio originale- Da: Atri Sharma Inviato: lunedì 4 novembre 2019 14:46 A: java-user@lucene.apache.org Oggetto: Re: Lucene index directory grows and shrinks This are typical symptoms of an index merg

Re: Lucene index directory grows and shrinks

s quickly reached 100% and then returned to > pre-growth employment. > > I believe that was caused by a rapid growth of Lucene index directory, > looking for such an event I've found only this article describing something > really similar > https://helpx.adobe.com/uk/experien

Lucene index directory grows and shrinks

of Lucene index directory, looking for such an event I've found only this article describing something really similar https://helpx.adobe.com/uk/experience-manager/kb/lucene-index-directory-growth.html I would like to know more info about this behaviour, first of all can you confirm

Re: Boost values for multi field Lucene index

Hi Baris, Are you looking for: https://lucene.apache.org/core/8_2_0/core/org/apache/lucene/search/Weight.html Kind Regards, Furkan KAMACI On Fri, Jun 28, 2019 at 6:12 PM wrote: > Hi,- > > how should i decide the values for a 4 field index lets say and i want > to prioritize the last field hi

Re: Lucene Index Cloud Replication

Another +1. We are also big s3 + lucene users and it is very interesting what other people came up with. We have an S3 lucene directory that allows immediate read-only use of lucene indexes stored on s3 with simultaneous local caching and a prototype of segment based index replication based on the

Re: Lucene Index Cloud Replication

+1 to share code for doing 1) and 3) both of which are tricky! Safely moving / copying bytes around is a notoriously difficult problem ... but Lucene's "end to end checksums" and per-segment-file-GUID make this safer. I think Lucene's replicator module is a good place for this? Mike McCandless

Re: Multi field Lucene index

Of course if it were the case, but since the data is there for the third and fourth, it should have brought the related data. Best regards On 7/5/19 10:27 AM, Atri Sharma wrote: Should not matter, AFAIK. If your first MUST clause in a BooleanQuery fails to match for a document, then there is

Re: Multi field Lucene index

Should not matter, AFAIK. If your first MUST clause in a BooleanQuery fails to match for a document, then there is no point for the engine to match further clauses, right? On Fri, Jul 5, 2019 at 7:56 PM wrote: > > Re-sending and please let me know Your amazing thoughts > > Happy July 4th > > Bes

Re: Multi field Lucene index

Re-sending and please let me know Your amazing thoughts Happy July 4th Best regards On 7/3/19 6:01 PM, baris.ka...@oracle.com wrote: Hi,-  does it matter which field is added first into index when multi field Lucene is used? does that make difference at search time like if it matches firs

Multi field Lucene index

Hi,-  does it matter which field is added first into index when multi field Lucene is used? does that make difference at search time like if it matches first and second field (which are fuzzy queries and AND-mandatory search term) and ignore third and fourth mandatory (AND) fields (search te

Lucene Index Cloud Replication

Hi there, I was talking with Varun at Berlin Buzzwords a couple of weeks ago about storing and retrieving Lucene indexes in S3, and realized that "uploading a Lucene directory to the cloud and downloading it on other machines" is a pretty common problem and one that's surprisingly easy to do poorl

Boost values for multi field Lucene index

Hi,-  how should i decide the values for a 4 field index lets say and i want to prioritize the last field highest and first field lowest? what values should i use? Best regards - To unsubscribe, e-mail: java-user-unsubscr..

Re: Incremental Lucene Index

Yes, Lucene supports incremental indexing. Note that the underlying structure is append only, so you are still paying the cost of delete + insert, but the semantics are what you expect them to be. On Mon, 24 Jun 2019 at 7:18 PM, Sukhendu Kumar Biswal wrote: > Hi Team, > Does Lucene support incre

Incremental Lucene Index

Hi Team, Does Lucene support incremental indexing or we need to index complete repository for every update in the database? We have a scenario where millions of records are stored in RDMS and which gets updated once in a day. Details: First time we created indexes for millions of records ,if some

RE: lucene index file gets corrupted while creating index with 2 nodes.

PM > To: java-user > Subject: Re: lucene index file gets corrupted while creating index with 2 > nodes. > > There is no chance anyone will try to change the code for 3.6, so > raising a JIRA is pointless. > > see: http://lucene.472066.n3.nabble.com/Issues-with-locked-indices- >

Re: lucene index file gets corrupted while creating index with 2 nodes.

ul 31, 2018 at 2:33 AM, Bhavin Chheda wrote: > Hi, > > > The lucene index file gets corrupted during loadtest of 15 min :- creating > the index with 2 nodes with 60 cocurrent users. > > I am using Lucene 3.6 version. The index is created in NFS. > > Please let me know d

lucene index file gets corrupted while creating index with 2 nodes.

Hi, The lucene index file gets corrupted during loadtest of 15 min :- creating the index with 2 nodes with 60 cocurrent users. I am using Lucene 3.6 version. The index is created in NFS. Please let me know does lucene create index works on multiple nodes with NFS. The error exception is

does $ mean something in Lucene index and MultiFieldQueryParser

It seems in my query string i cant see $ when print it out from MultiFieldQueryParser but the search string has $ in it and it finds hits. On the other hand, Termquery based BooleanQuery keeps $ and no hits. i use $ for starts with effect. Best regards ---

RE: Link Lucene index with Adobe reader

On 2018-02-06 Anuradha Rajaram (RBEI/ETB14) wrote: > We are using Lucene for indexing the PDF. We need to link generated lucene > index with Adobe reader. In Adobe Acrobat there is a dedicated feature solving this task: Embed index. It builds search index and stores it inside the PDF fil

Re: Link Lucene index with Adobe reader

On 06/02/2018 08:22, Anuradha Rajaram (RBEI/ETB14) wrote: Dear Team, We are using Lucene for indexing the PDF. We need to link generated lucene index with Adobe reader. Current Approach: Placed both the generated lucene index and PDF in the folder. Open the PDF and search for a word using

Link Lucene index with Adobe reader

Dear Team, We are using Lucene for indexing the PDF. We need to link generated lucene index with Adobe reader. Current Approach: Placed both the generated lucene index and PDF in the folder. Open the PDF and search for a word using Advance search in Adobe reader. Whole PDF is searched

Link Lucene index with Adobe reader

Dear Team, We are using Lucene for indexing the PDF. We need to link generated lucene index with Adobe reader. Current Approach: Placed both the generated lucene index and PDF in the folder. Open the PDF and search for a word using Advance search in Adobe reader. Whole PDF is searched

Lucene Index Size growing

Hi guys, i'm having problems with my lucene index. Once a day we do a full reindex. This full index is coded like this.. delete all docs optimize index reindex all docs optimize again commit mayberefresh on searcher manager *The problem is that everytime we do this, the index size i

Re: Encryption at lucene index

I got it Erick.. Thank you.. - ​Kumaran R ​ On Fri, Aug 11, 2017 at 10:35 PM, Erick Erickson wrote: > Encrypting the _tokens_ inevitably leads to reduced capabilities BTW. > Trivial example: > I have these tokens in my index > run > runner > running > runs > > Any non-trivial encryption algorit

Re: Encryption at lucene index

Encrypting the _tokens_ inevitably leads to reduced capabilities BTW. Trivial example: I have these tokens in my index run runner running runs Any non-trivial encryption algorithm will not encrypt the first three letters "run" identically in all three so searching for run* simply won't work. As y

Re: Encryption at lucene index

Hi Ishan, thank you :-) - ​- Kumaran R ​ On Mon, Aug 7, 2017 at 10:53 PM, Ishan Chattopadhyaya < ichattopadhy...@gmail.com> wrote: > Harry Ochiai (Hitachi) has some index encryption solution, > https://www.slideshare.net/maggon/securing-solr-search-data-in-the-cloud > I think it is proprietary,

Re: Encryption at lucene index

Harry Ochiai (Hitachi) has some index encryption solution, https://www.slideshare.net/maggon/securing-solr-search-data-in-the-cloud I think it is proprietary, but I'm not sure. Maybe more googling might help find the exact page where his solution is described. On Mon, Aug 7, 2017 at 9:59 PM, Kumar

Re: Encryption at lucene index

Hi Erick, i want to encrypt some fields of an document which has personal identifiable information ( both indexed and stored data)... for eg: email, mobilenumber etc.. i am able to find LUCENE-6966 alone while googling it.. any related pointers in solr or latest lucene version? - ​- Kumaran R​ O

Re: Encryption at lucene index

No, since you haven't defined what you want to encrypt, what your requirements are, what you hope to get out of "encryption" etc. Put the index on an encrypting filesystem and forget about it if you possibly can, because anything else is a significant amount of work. To encrypt the searchable toke

Re: Encryption at lucene index

Hi Erick, Thanks for the information. Any pointers about encryption options in solr? -- Kumaran R On Mon, Aug 7, 2017 at 9:17 PM, Erick Erickson wrote: > Encryption in Solr has a bunch of ramifications. Do you care about > > - encryption at rest or in memory? > - encrypting the _searcha

Re: Encryption at lucene index

Encryption in Solr has a bunch of ramifications. Do you care about - encryption at rest or in memory? - encrypting the _searchable_ tokens? - encrypting the searchable tokens per-user? - encrypting the stored data (which a filter won't do BTW). It's actually a fairly complex topic the discussion

Encryption at lucene index

Hi All, After looking at all below discussions, i have one doubt which may be silly or novice but i want to throw this to lucene user list. if we have encryption layer included in our analyzer's flow of filters like EncryptionFilter to control field-level encryption. what are the consequences ?

Re: Ways to store and search tens of billions of text document content in one lucene index

On Fri, Jun 23, 2017 at 4:24 PM, Ranganath B N wrote: > Hi, [cutting X-Y problem stuff] > What strategies do you recommend for this task "Ways to store and search > tens of billions > of text document content in one lucene index"? so that I can accomplish > thi

  1   2   3   4   5   6   7   8   >