Re: Indexing and searching a DateTime range

2015-02-11 Thread Gergely Nagy
Thank you Uwe! Your reply is very useful and insightful. Your workflow matches my requirements exactly. My confusion was coming from the fact that I didn't understand what the Analyzers are doing. Actually I am still wondering, isn't it possible to provide an abstraction on Lucene side to make th

RE: Indexing and searching a DateTime range

2015-02-10 Thread Uwe Schindler
Hi, > OK. I found the Alfresco code on GitHub. So it's open source it seems. > > And I found the DateTimeAnalyser, so I will just take that code as a starting > point: > https://github.com/lsbueno/alfresco/tree/master/root/projects/repository/ > source/java/org/alfresco/repo/search/impl/lucene/an

Re: Indexing and searching a DateTime range

2015-02-09 Thread Gergely Nagy
OK. I found the Alfresco code on GitHub. So it's open source it seems. And I found the DateTimeAnalyser, so I will just take that code as a starting point: https://github.com/lsbueno/alfresco/tree/master/root/projects/repository/source/java/org/alfresco/repo/search/impl/lucene/analysis Thank you

Re: Indexing and searching a DateTime range

2015-02-09 Thread Gergely Nagy
Thank you Barry, I really appreciate your time to respond, Let me clarify this a little bit more. I think it was not clear. I know how to parse dates, this is not the question here. (See my previous email: "how can I pipe my converter logic into the indexing process?") All of your solutions guys

Re: Indexing and searching a DateTime range

2015-02-09 Thread Barry Coughlan
Hi Gergely, Writing an analyzer would work but it is unnecessarily complicated. You could just parse the date from the string in your input code and index it in the LongField like this: SimpleDateFormat format = new SimpleDateFormat("-MM-dd HH:mm:ss.S'Z'"); format.setTimeZone(TimeZone.getTime

Re: Indexing and searching a DateTime range

2015-02-09 Thread Gergely Nagy
Thank you for taking your time to respond Karthik, Can you show me an example how to convert DateTime to milliseconds? I mean how can I pipe my converter logic into the indexing process? I suspect I need to write my own Analyzer/Tokenizer to achieve this. Is this correct? 2015-02-09 22:58 GMT+09

Re: Indexing and searching a DateTime range

2015-02-09 Thread KARTHIK SHIVAKUMAR
Hi Long time ago,.. I used to store datetime in millisecond . TermRangequery used to work in perfect condition Convert all datetime to millisecond and index the same. On search condition again convert datetime to millisecond and use TermRangequery. With regards Karthik On Feb 9, 2015 1:24

Re: Indexing and searching a DateTime range

2015-02-09 Thread Gergely Nagy
Thank you for the great answer Uwe! Sadly my department rejected the above combination of using Logstash + Elasticsearch. According to their experience, elastic search works fine on about 3 days of log data, but slows terribly down providing the magnitude of 3 months of data or so. But I will tak

RE: Indexing and searching a DateTime range

2015-02-09 Thread Uwe Schindler
Hi, > I am in the beginning of implementing a Lucene application which would > supposedly search through some log files. > > One of the requirements is to return results between a time range. Let's say > these are two lines in a series of log files: > 2015-02-08 00:02:06.852Z INFO... > ... > 2015

Indexing and searching a DateTime range

2015-02-08 Thread Gergely Nagy
Hi Lucene users, I am in the beginning of implementing a Lucene application which would supposedly search through some log files. One of the requirements is to return results between a time range. Let's say these are two lines in a series of log files: 2015-02-08 00:02:06.852Z INFO... ... 2015-02