Re: Help Regarding token filter

2018-03-15 Thread Michael Sokolov
Since you are writing a custom token filter, it's up to you to return successive tokens by setting the appropriate attributes when nextToken is called. Have you read the tokenstream javadocs? On Mar 15, 2018 10:35 AM, "deepu srinivasan" wrote: > Hi . > How do i split a single token and index the

Re: Storage of indexed and stored fields (Space and Performance)

2018-03-15 Thread Erick Erickson
Stored data is kept in separate segment files (*.fdt and *.fdx). As such they have no measurable impact on query time. All the data for executing searches is kept in other extensions in each segment and accessed separately. Adding stored data does increase the size on disk by roughly 50% of the nu

Help Regarding token filter

2018-03-15 Thread deepu srinivasan
Hi . How do i split a single token and index them both. For eg : if i recieve a token "&&11&" in my custom token filter , i would like to index as "&" and 11.

Storage of indexed and stored fields (Space and Performance)

2018-03-15 Thread Rajnish kamboj
Hi How are indexed and stored fields treated by Lucene w.r.t space and performance? Is there any performance hit with stored fields which are indexed? Lucene Version: 5.3.1 Assumption: Stored fields are just simple strings (not huge documents) Example: Data: [101, Gold]; [102, Silver

IndexWriter updateDocument is removing doc from index

2018-03-15 Thread Bernd Fehling
While writing some tools to build and maintain lucene indexes I noticed some strange behavior during testing. A doc disappears from lucene index while using IndexWriter updateDocument. The API of lucene 6.4.2 states: "Updates a document by first deleting the document(s) containing term and then ad