date:20180111

RE: Lucene OOM

2018-01-11 Thread Uwe Schindler

Hi,

If the index size on disk is about 750 GiB then a memory usage of 2.3 G heap 
space for the FST seems fine. It's just a bit strange that you only have 10 
million documents!

Are those documents huge and have lots of indexed text content, possibly 
OCR/scanned stuff? If this is the case, the term dictionary may get huge 
because of many terms with incorrect spelling.

Please also give us a "ls -lh" of your index directory to make a guess.

Uwe

-
Uwe Schindler
Achterdiek 19, D-28357 Bremen
http://www.thetaphi.de
eMail: u...@thetaphi.de

> -Original Message-
> From: dawn breaks [mailto:2005dawnbre...@gmail.com]
> Sent: Thursday, January 11, 2018 3:40 AM
> To: java-user@lucene.apache.org
> Subject: Lucene OOM
> 
> Hi, all
>   We have a search engine service built with lucene 4.7,  it seem that
> lucene eat too much momery, and we have approximate 10 million
> document，the
> index size on disk is approximate 750G.  My question is why the FST$Arc
> objects consume so much memory?  please refer to the following histo stat
> of jmap. Hope anybody can give me some suggestion.
> 
>  num #instances #bytes  class name
> --
>1:   4346283 2294837424  [Lorg.apache.lucene.util.fst.FST$Arc;
>2:  25918804 2023475632  [C
>3:  17450041 1014051416  [B
>4:  25878734  621089616  java.lang.String
>5:  18634803  596313696  java.util.HashMap$Node
>6:  14039862  561594480  java.util.TreeMap$Entry
>7:   4346283  452013432  org.apache.lucene.util.fst.FST
>8:   4522836  424741520  [Ljava.util.HashMap$Node;
>9:   4346283  347702640
> org.apache.lucene.codecs.BlockTreeTermsReader$FieldReader
>   10:   4683616  337220352  org.apache.lucene.util.fst.FST$Arc
>   11:  12947467  310739208  org.apache.lucene.util.BytesRef
>   12:790283  280383040  [J
>   13:   4359111  245496264  [Ljava.lang.Object;
>   14:   4545337  218176176  java.util.HashMap
>   15:   4510384  216498432  org.apache.lucene.index.FieldInfo
>   16:   4359066  199713232  [I
>   17:   4346283  173851320  org.apache.lucene.util.fst.BytesStore
>   18:   4510400  144332800  java.util.Collections$UnmodifiableMap
>   19:   4354347  104504328  java.util.ArrayList
>   20:   5736589   91785424  java.lang.Integer
>   21:822685   59233320
> org.apache.lucene.codecs.lucene45.Lucene45DocValuesProducer$NumericE
> ntry
>   22:428313   13706016
> org.apache.lucene.facet.taxonomy.writercache.CollisionMap$Entry
>   23:420547   13457504  org.wltea.analyzer.dic.DictSegment
>   24:1770395665248  [Lorg.wltea.analyzer.dic.DictSegment;
>   25:205112128
> [Lorg.apache.lucene.facet.taxonomy.writercache.CollisionMap$Entry;
>   26: 424542377424  org.apache.lucene.store.RAMInputStream
>   27: 500542002160  org.apache.lucene.util.packed.Packed64
>   28: 440361761440
> org.apache.lucene.util.packed.DirectPackedReader
>   29: 330131056416
> java.util.concurrent.ConcurrentHashMap$Node
>   30: 439571054968
> org.apache.lucene.codecs.lucene45.Lucene45DocValuesProducer$2
> 
> 
> 
> 
> Thanks & Best Regards!
> lubin


-
To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org
For additional commands, e-mail: java-user-h...@lucene.apache.org

Re: Lucene OOM

2018-01-11 Thread dawn breaks

Hi, Uwe
  Thanks for your timely reply. Yes,  those documents are huge text. We
have ten indices, and each of them has approximate 75G index size on disk.
Following is the directory  content of one of the indices.

Thanks & Best Regards!
lubin

total 74G
-rw-r--r-- 1 root root  100K Jan 10 16:11 _2ncr_4k.del
-rw-r--r-- 1 root root  1.1G Aug  4 12:52 _2ncr.fdt
-rw-r--r-- 1 root root  468K Aug  4 12:52 _2ncr.fdx
-rw-r--r-- 1 root root  636K Aug  4 12:55 _2ncr.fnm
-rw-r--r-- 1 root root  398M Aug  4 12:53 _2ncr_Lucene41_0.doc
-rw-r--r-- 1 root root  712M Aug  4 12:53 _2ncr_Lucene41_0.pay
-rw-r--r-- 1 root root  744M Aug  4 12:53 _2ncr_Lucene41_0.pos
-rw-r--r-- 1 root root  129M Aug  4 12:53 _2ncr_Lucene41_0.tim
-rw-r--r-- 1 root root  3.1M Aug  4 12:53 _2ncr_Lucene41_0.tip
-rw-r--r-- 1 root root  822M Aug  4 12:54 _2ncr_Lucene45_0.dvd
-rw-r--r-- 1 root root  210K Aug  4 12:54 _2ncr_Lucene45_0.dvm
-rw-r--r-- 1 root root   540 Aug  4 12:55 _2ncr.si
-rw-r--r-- 1 root root  1.5G Aug  4 12:55 _2ncr.tvd
-rw-r--r-- 1 root root  441K Aug  4 12:55 _2ncr.tvx
-rw-r--r-- 1 root root   98K Jan 11 11:43 _555c_5x.del
-rw-r--r-- 1 root root  1.1G Aug 25 12:51 _555c.fdt
-rw-r--r-- 1 root root  457K Aug 25 12:51 _555c.fdx
-rw-r--r-- 1 root root  872K Aug 25 12:54 _555c.fnm
-rw-r--r-- 1 root root  389M Aug 25 12:52 _555c_Lucene41_0.doc
-rw-r--r-- 1 root root  713M Aug 25 12:52 _555c_Lucene41_0.pay
-rw-r--r-- 1 root root  750M Aug 25 12:52 _555c_Lucene41_0.pos
-rw-r--r-- 1 root root  136M Aug 25 12:52 _555c_Lucene41_0.tim
-rw-r--r-- 1 root root  3.2M Aug 25 12:52 _555c_Lucene41_0.tip
-rw-r--r-- 1 root root  1.1G Aug 25 12:53 _555c_Lucene45_0.dvd
-rw-r--r-- 1 root root  442K Aug 25 12:53 _555c_Lucene45_0.dvm
-rw-r--r-- 1 root root   540 Aug 25 12:54 _555c.si
-rw-r--r-- 1 root root  1.4G Aug 25 12:54 _555c.tvd
-rw-r--r-- 1 root root  422K Aug 25 12:54 _555c.tvx
-rw-r--r-- 1 root root   93K Jan 10 16:38 _790n_5s.del
-rw-r--r-- 1 root root  1.1G Sep  6 14:17 _790n.fdt
-rw-r--r-- 1 root root  438K Sep  6 14:17 _790n.fdx
-rw-r--r-- 1 root root  1.1M Sep  6 14:20 _790n.fnm
-rw-r--r-- 1 root root  380M Sep  6 14:18 _790n_Lucene41_0.doc
-rw-r--r-- 1 root root  684M Sep  6 14:18 _790n_Lucene41_0.pay
-rw-r--r-- 1 root root  746M Sep  6 14:18 _790n_Lucene41_0.pos
-rw-r--r-- 1 root root  141M Sep  6 14:18 _790n_Lucene41_0.tim
-rw-r--r-- 1 root root  3.5M Sep  6 14:18 _790n_Lucene41_0.tip
-rw-r--r-- 1 root root  1.2G Sep  6 14:20 _790n_Lucene45_0.dvd
-rw-r--r-- 1 root root  550K Sep  6 14:20 _790n_Lucene45_0.dvm
-rw-r--r-- 1 root root   540 Sep  6 14:20 _790n.si
-rw-r--r-- 1 root root  1.4G Sep  6 14:20 _790n.tvd
-rw-r--r-- 1 root root  412K Sep  6 14:20 _790n.tvx
-rw-r--r-- 1 root root   82K Jan 10 16:38 _bv18_8d.del
-rw-r--r-- 1 root root  1.1G Oct 10 12:17 _bv18.fdt
-rw-r--r-- 1 root root  425K Oct 10 12:17 _bv18.fdx
-rw-r--r-- 1 root root  1.4M Oct 10 12:20 _bv18.fnm
-rw-r--r-- 1 root root  363M Oct 10 12:18 _bv18_Lucene41_0.doc
-rw-r--r-- 1 root root  622M Oct 10 12:18 _bv18_Lucene41_0.pay
-rw-r--r-- 1 root root  728M Oct 10 12:18 _bv18_Lucene41_0.pos
-rw-r--r-- 1 root root  133M Oct 10 12:18 _bv18_Lucene41_0.tim
-rw-r--r-- 1 root root  3.4M Oct 10 12:18 _bv18_Lucene41_0.tip
-rw-r--r-- 1 root root  1.9G Oct 10 12:20 _bv18_Lucene45_0.dvd
-rw-r--r-- 1 root root  724K Oct 10 12:20 _bv18_Lucene45_0.dvm
-rw-r--r-- 1 root root   540 Oct 10 12:20 _bv18.si
-rw-r--r-- 1 root root  1.3G Oct 10 12:20 _bv18.tvd
-rw-r--r-- 1 root root  380K Oct 10 12:20 _bv18.tvx
-rw-r--r-- 1 root root   36K Jan 10 16:11 _c1uj_4k.del
-rw-r--r-- 1 root root  650M Oct 10 22:42 _c1uj.fdt
-rw-r--r-- 1 root root  227K Oct 10 22:42 _c1uj.fdx
-rw-r--r-- 1 root root  1.3M Oct 10 22:43 _c1uj.fnm
-rw-r--r-- 1 root root  177M Oct 10 22:42 _c1uj_Lucene41_0.doc
-rw-r--r-- 1 root root  412M Oct 10 22:42 _c1uj_Lucene41_0.pay
-rw-r--r-- 1 root root  428M Oct 10 22:42 _c1uj_Lucene41_0.pos
-rw-r--r-- 1 root root   54M Oct 10 22:42 _c1uj_Lucene41_0.tim
-rw-r--r-- 1 root root  1.6M Oct 10 22:42 _c1uj_Lucene41_0.tip
-rw-r--r-- 1 root root  385M Oct 10 22:43 _c1uj_Lucene45_0.dvd
-rw-r--r-- 1 root root  390K Oct 10 22:43 _c1uj_Lucene45_0.dvm
-rw-r--r-- 1 root root   540 Oct 10 22:43 _c1uj.si
-rw-r--r-- 1 root root  776M Oct 10 22:43 _c1uj.tvd
-rw-r--r-- 1 root root  173K Oct 10 22:43 _c1uj.tvx
-rw-r--r-- 1 root root   64K Jan 10 16:11 _dz32_75.del
-rw-r--r-- 1 root root  1.1G Oct 21 02:30 _dz32.fdt
-rw-r--r-- 1 root root  415K Oct 21 02:30 _dz32.fdx
-rw-r--r-- 1 root root  1.6M Oct 21 02:34 _dz32.fnm
-rw-r--r-- 1 root root  313M Oct 21 02:31 _dz32_Lucene41_0.doc
-rw-r--r-- 1 root root  741M Oct 21 02:31 _dz32_Lucene41_0.pay
-rw-r--r-- 1 root root  791M Oct 21 02:31 _dz32_Lucene41_0.pos
-rw-r--r-- 1 root root  100M Oct 21 02:31 _dz32_Lucene41_0.tim
-rw-r--r-- 1 root root  2.7M Oct 21 02:31 _dz32_Lucene41_0.tip
-rw-r--r-- 1 root root  1.3G Oct 21 02:33 _dz32_Lucene45_0.dvd
-rw-r--r-- 1 root root  898K Oct 21 02:33 _dz32_Lucene45_0.dvm
-rw-r--r-- 1 root root   540 Oct 21 02:34 _dz32.si
-rw-r-

RE: Lucene OOM

2018-01-11 Thread Uwe Schindler

Hi lubin,

the terms dictionary is using the "tim" and "tip" files. It should be 
approximately in the dimension of the FST.

Do you have all indexes running in the same JVM or is it 10 servers? Because 
then the numbers look correct. If you really want to have such an large index 
in a single machine using a single JVM, you should plan for more heap space. 
I'd start with 12 GiB of heap space to run this index.

A last recommendation: If you update your index during runtime, make sure that 
you correctly close the outdated IndexReaders/IndexSearchers (e.g. using 
SearcherManager), so you don't have orphaned instances of IndexReader consuming 
heap space and disk space, because the files can't be fully deleted as long as 
those are open!

Uwe

-
Uwe Schindler
Achterdiek 19, D-28357 Bremen
http://www.thetaphi.de
eMail: u...@thetaphi.de

> -Original Message-
> From: dawn breaks [mailto:2005dawnbre...@gmail.com]
> Sent: Thursday, January 11, 2018 10:22 AM
> To: java-user@lucene.apache.org
> Subject: Re: Lucene OOM
> 
> Hi, Uwe
>   Thanks for your timely reply. Yes,  those documents are huge text. We
> have ten indices, and each of them has approximate 75G index size on disk.
> Following is the directory  content of one of the indices.
> 
> Thanks & Best Regards!
> lubin
> 
> total 74G
> -rw-r--r-- 1 root root  100K Jan 10 16:11 _2ncr_4k.del
> -rw-r--r-- 1 root root  1.1G Aug  4 12:52 _2ncr.fdt
> -rw-r--r-- 1 root root  468K Aug  4 12:52 _2ncr.fdx
> -rw-r--r-- 1 root root  636K Aug  4 12:55 _2ncr.fnm
> -rw-r--r-- 1 root root  398M Aug  4 12:53 _2ncr_Lucene41_0.doc
> -rw-r--r-- 1 root root  712M Aug  4 12:53 _2ncr_Lucene41_0.pay
> -rw-r--r-- 1 root root  744M Aug  4 12:53 _2ncr_Lucene41_0.pos
> -rw-r--r-- 1 root root  129M Aug  4 12:53 _2ncr_Lucene41_0.tim
> -rw-r--r-- 1 root root  3.1M Aug  4 12:53 _2ncr_Lucene41_0.tip
> -rw-r--r-- 1 root root  822M Aug  4 12:54 _2ncr_Lucene45_0.dvd
> -rw-r--r-- 1 root root  210K Aug  4 12:54 _2ncr_Lucene45_0.dvm
> -rw-r--r-- 1 root root   540 Aug  4 12:55 _2ncr.si
> -rw-r--r-- 1 root root  1.5G Aug  4 12:55 _2ncr.tvd
> -rw-r--r-- 1 root root  441K Aug  4 12:55 _2ncr.tvx
> -rw-r--r-- 1 root root   98K Jan 11 11:43 _555c_5x.del
> -rw-r--r-- 1 root root  1.1G Aug 25 12:51 _555c.fdt
> -rw-r--r-- 1 root root  457K Aug 25 12:51 _555c.fdx
> -rw-r--r-- 1 root root  872K Aug 25 12:54 _555c.fnm
> -rw-r--r-- 1 root root  389M Aug 25 12:52 _555c_Lucene41_0.doc
> -rw-r--r-- 1 root root  713M Aug 25 12:52 _555c_Lucene41_0.pay
> -rw-r--r-- 1 root root  750M Aug 25 12:52 _555c_Lucene41_0.pos
> -rw-r--r-- 1 root root  136M Aug 25 12:52 _555c_Lucene41_0.tim
> -rw-r--r-- 1 root root  3.2M Aug 25 12:52 _555c_Lucene41_0.tip
> -rw-r--r-- 1 root root  1.1G Aug 25 12:53 _555c_Lucene45_0.dvd
> -rw-r--r-- 1 root root  442K Aug 25 12:53 _555c_Lucene45_0.dvm
> -rw-r--r-- 1 root root   540 Aug 25 12:54 _555c.si
> -rw-r--r-- 1 root root  1.4G Aug 25 12:54 _555c.tvd
> -rw-r--r-- 1 root root  422K Aug 25 12:54 _555c.tvx
> -rw-r--r-- 1 root root   93K Jan 10 16:38 _790n_5s.del
> -rw-r--r-- 1 root root  1.1G Sep  6 14:17 _790n.fdt
> -rw-r--r-- 1 root root  438K Sep  6 14:17 _790n.fdx
> -rw-r--r-- 1 root root  1.1M Sep  6 14:20 _790n.fnm
> -rw-r--r-- 1 root root  380M Sep  6 14:18 _790n_Lucene41_0.doc
> -rw-r--r-- 1 root root  684M Sep  6 14:18 _790n_Lucene41_0.pay
> -rw-r--r-- 1 root root  746M Sep  6 14:18 _790n_Lucene41_0.pos
> -rw-r--r-- 1 root root  141M Sep  6 14:18 _790n_Lucene41_0.tim
> -rw-r--r-- 1 root root  3.5M Sep  6 14:18 _790n_Lucene41_0.tip
> -rw-r--r-- 1 root root  1.2G Sep  6 14:20 _790n_Lucene45_0.dvd
> -rw-r--r-- 1 root root  550K Sep  6 14:20 _790n_Lucene45_0.dvm
> -rw-r--r-- 1 root root   540 Sep  6 14:20 _790n.si
> -rw-r--r-- 1 root root  1.4G Sep  6 14:20 _790n.tvd
> -rw-r--r-- 1 root root  412K Sep  6 14:20 _790n.tvx
> -rw-r--r-- 1 root root   82K Jan 10 16:38 _bv18_8d.del
> -rw-r--r-- 1 root root  1.1G Oct 10 12:17 _bv18.fdt
> -rw-r--r-- 1 root root  425K Oct 10 12:17 _bv18.fdx
> -rw-r--r-- 1 root root  1.4M Oct 10 12:20 _bv18.fnm
> -rw-r--r-- 1 root root  363M Oct 10 12:18 _bv18_Lucene41_0.doc
> -rw-r--r-- 1 root root  622M Oct 10 12:18 _bv18_Lucene41_0.pay
> -rw-r--r-- 1 root root  728M Oct 10 12:18 _bv18_Lucene41_0.pos
> -rw-r--r-- 1 root root  133M Oct 10 12:18 _bv18_Lucene41_0.tim
> -rw-r--r-- 1 root root  3.4M Oct 10 12:18 _bv18_Lucene41_0.tip
> -rw-r--r-- 1 root root  1.9G Oct 10 12:20 _bv18_Lucene45_0.dvd
> -rw-r--r-- 1 root root  724K Oct 10 12:20 _bv18_Lucene45_0.dvm
> -rw-r--r-- 1 root root   540 Oct 10 12:20 _bv18.si
> -rw-r--r-- 1 root root  1.3G Oct 10 12:20 _bv18.tvd
> -rw-r--r-- 1 root root  380K Oct 10 12:20 _bv18.tvx
> -rw-r--r-- 1 root root   36K Jan 10 16:11 _c1uj_4k.del
> -rw-r--r-- 1 root root  650M Oct 10 22:42 _c1uj.fdt
> -rw-r--r-- 1 root root  227K Oct 10 22:42 _c1uj.fdx
> -rw-r--r-- 1 root root  1.3M Oct 10 22:43 _c1uj.fnm
> -rw-r--r-- 1 root root  177M Oct 10 22:42 _c1uj_Lucene41_0.doc
> -rw-r--r-- 1 root root  412M Oct 10 22:42 _c1uj_Lucene41_0

Re: Lucene OOM

2018-01-11 Thread dawn breaks

Hi, Uwe
  Yes, All indexes running in the same JVM with 14GiB of heap space, but
the JVM heap usage is up to 95%.  I'am sue that all
IndexReaders/IndexSearchers has been closed properly.



On 11 January 2018 at 20:46, Uwe Schindler  wrote:

> Hi lubin,
>
> the terms dictionary is using the "tim" and "tip" files. It should be
> approximately in the dimension of the FST.
>
> Do you have all indexes running in the same JVM or is it 10 servers?
> Because then the numbers look correct. If you really want to have such an
> large index in a single machine using a single JVM, you should plan for
> more heap space. I'd start with 12 GiB of heap space to run this index.
>
> A last recommendation: If you update your index during runtime, make sure
> that you correctly close the outdated IndexReaders/IndexSearchers (e.g.
> using SearcherManager), so you don't have orphaned instances of IndexReader
> consuming heap space and disk space, because the files can't be fully
> deleted as long as those are open!
>
> Uwe
>
> -
> Uwe Schindler
> Achterdiek 19, D-28357 Bremen
> http://www.thetaphi.de
> eMail: u...@thetaphi.de
>
> > -Original Message-
> > From: dawn breaks [mailto:2005dawnbre...@gmail.com]
> > Sent: Thursday, January 11, 2018 10:22 AM
> > To: java-user@lucene.apache.org
> > Subject: Re: Lucene OOM
> >
> > Hi, Uwe
> >   Thanks for your timely reply. Yes,  those documents are huge text. We
> > have ten indices, and each of them has approximate 75G index size on
> disk.
> > Following is the directory  content of one of the indices.
> >
> > Thanks & Best Regards!
> > lubin
> >
> > total 74G
> > -rw-r--r-- 1 root root  100K Jan 10 16:11 _2ncr_4k.del
> > -rw-r--r-- 1 root root  1.1G Aug  4 12:52 _2ncr.fdt
> > -rw-r--r-- 1 root root  468K Aug  4 12:52 _2ncr.fdx
> > -rw-r--r-- 1 root root  636K Aug  4 12:55 _2ncr.fnm
> > -rw-r--r-- 1 root root  398M Aug  4 12:53 _2ncr_Lucene41_0.doc
> > -rw-r--r-- 1 root root  712M Aug  4 12:53 _2ncr_Lucene41_0.pay
> > -rw-r--r-- 1 root root  744M Aug  4 12:53 _2ncr_Lucene41_0.pos
> > -rw-r--r-- 1 root root  129M Aug  4 12:53 _2ncr_Lucene41_0.tim
> > -rw-r--r-- 1 root root  3.1M Aug  4 12:53 _2ncr_Lucene41_0.tip
> > -rw-r--r-- 1 root root  822M Aug  4 12:54 _2ncr_Lucene45_0.dvd
> > -rw-r--r-- 1 root root  210K Aug  4 12:54 _2ncr_Lucene45_0.dvm
> > -rw-r--r-- 1 root root   540 Aug  4 12:55 _2ncr.si
> > -rw-r--r-- 1 root root  1.5G Aug  4 12:55 _2ncr.tvd
> > -rw-r--r-- 1 root root  441K Aug  4 12:55 _2ncr.tvx
> > -rw-r--r-- 1 root root   98K Jan 11 11:43 _555c_5x.del
> > -rw-r--r-- 1 root root  1.1G Aug 25 12:51 _555c.fdt
> > -rw-r--r-- 1 root root  457K Aug 25 12:51 _555c.fdx
> > -rw-r--r-- 1 root root  872K Aug 25 12:54 _555c.fnm
> > -rw-r--r-- 1 root root  389M Aug 25 12:52 _555c_Lucene41_0.doc
> > -rw-r--r-- 1 root root  713M Aug 25 12:52 _555c_Lucene41_0.pay
> > -rw-r--r-- 1 root root  750M Aug 25 12:52 _555c_Lucene41_0.pos
> > -rw-r--r-- 1 root root  136M Aug 25 12:52 _555c_Lucene41_0.tim
> > -rw-r--r-- 1 root root  3.2M Aug 25 12:52 _555c_Lucene41_0.tip
> > -rw-r--r-- 1 root root  1.1G Aug 25 12:53 _555c_Lucene45_0.dvd
> > -rw-r--r-- 1 root root  442K Aug 25 12:53 _555c_Lucene45_0.dvm
> > -rw-r--r-- 1 root root   540 Aug 25 12:54 _555c.si
> > -rw-r--r-- 1 root root  1.4G Aug 25 12:54 _555c.tvd
> > -rw-r--r-- 1 root root  422K Aug 25 12:54 _555c.tvx
> > -rw-r--r-- 1 root root   93K Jan 10 16:38 _790n_5s.del
> > -rw-r--r-- 1 root root  1.1G Sep  6 14:17 _790n.fdt
> > -rw-r--r-- 1 root root  438K Sep  6 14:17 _790n.fdx
> > -rw-r--r-- 1 root root  1.1M Sep  6 14:20 _790n.fnm
> > -rw-r--r-- 1 root root  380M Sep  6 14:18 _790n_Lucene41_0.doc
> > -rw-r--r-- 1 root root  684M Sep  6 14:18 _790n_Lucene41_0.pay
> > -rw-r--r-- 1 root root  746M Sep  6 14:18 _790n_Lucene41_0.pos
> > -rw-r--r-- 1 root root  141M Sep  6 14:18 _790n_Lucene41_0.tim
> > -rw-r--r-- 1 root root  3.5M Sep  6 14:18 _790n_Lucene41_0.tip
> > -rw-r--r-- 1 root root  1.2G Sep  6 14:20 _790n_Lucene45_0.dvd
> > -rw-r--r-- 1 root root  550K Sep  6 14:20 _790n_Lucene45_0.dvm
> > -rw-r--r-- 1 root root   540 Sep  6 14:20 _790n.si
> > -rw-r--r-- 1 root root  1.4G Sep  6 14:20 _790n.tvd
> > -rw-r--r-- 1 root root  412K Sep  6 14:20 _790n.tvx
> > -rw-r--r-- 1 root root   82K Jan 10 16:38 _bv18_8d.del
> > -rw-r--r-- 1 root root  1.1G Oct 10 12:17 _bv18.fdt
> > -rw-r--r-- 1 root root  425K Oct 10 12:17 _bv18.fdx
> > -rw-r--r-- 1 root root  1.4M Oct 10 12:20 _bv18.fnm
> > -rw-r--r-- 1 root root  363M Oct 10 12:18 _bv18_Lucene41_0.doc
> > -rw-r--r-- 1 root root  622M Oct 10 12:18 _bv18_Lucene41_0.pay
> > -rw-r--r-- 1 root root  728M Oct 10 12:18 _bv18_Lucene41_0.pos
> > -rw-r--r-- 1 root root  133M Oct 10 12:18 _bv18_Lucene41_0.tim
> > -rw-r--r-- 1 root root  3.4M Oct 10 12:18 _bv18_Lucene41_0.tip
> > -rw-r--r-- 1 root root  1.9G Oct 10 12:20 _bv18_Lucene45_0.dvd
> > -rw-r--r-- 1 root root  724K Oct 10 12:20 _bv18_Lucene45_0.dvm
> > -rw-r--r-- 1 root root   540 Oct 10 12:20 _bv18.si
> > -rw-r--r-- 1 root root  1.3G Oct 10

Re: Maven snapshots

2018-01-11 Thread Terry Smith

Steve,

Thanks for looking into this. I see the artifacts for 7.2.1-SNAPSHOT,
7.3.0-SNAPSHOT, and 8.0.0-SNAPSHOT are now available so things are looking
good.

--Terry



On Tue, Jan 9, 2018 at 4:38 PM, Uwe Schindler  wrote:

> Hi Steve,
>
> If this fixed it could you please add it as dependency on the separate
> Jenkins target? I prefer all Jenkins jobs only call a simple "Jenkins job"
> target.
>
> Uwe
>
> Am January 9, 2018 8:59:02 PM UTC schrieb Steve Rowe :
> >Looks like this is partly(?) as a result of build changes I made as
> >part of .  I
> >unintentionally removed the only invocation of the
> >"generate-maven-artifacts" target called from the Jenkins jobs (which
> >run “ant jenkins-maven-nightly”), and it is “generate-maven-artifacts”
> >that pushes the snapshots to the repo[1].
> >
> >I’ll add “generate-maven-artifacts" to the *-Maven-* Jenkins jobs’
> >config and see if that fixes the problem.
> >
> >--
> >Steve
> >www.lucidworks.com
> >
> >[1] With sysprops as follows:
> >
> >   m2.repository.id=apache.snapshots.https
> >m2.repository.url=https://repository.apache.org/content/
> repositories/snapshots
> >   skipTests=true
> >
> >> On Jan 9, 2018, at 2:24 PM, Steve Rowe  wrote:
> >>
> >> Hi Terry,
> >>
> >> Thanks for the heads-up about this problem.
> >>
> >> There are ASF Jenkins jobs that regularly build those snapshots - see
> >the jobs with “Maven” in their names here:
> >.
> >>
> >> I’ll look into the cause of the long lags and report back.
> >>
> >> --
> >> Steve
> >> www.lucidworks.com
> >>
> >>> On Jan 9, 2018, at 11:57 AM, Terry Smith  wrote:
> >>>
> >>> Thanks for the feedback.
> >>>
> >>> Armins: I use the maven snapshots from some local jenkins builds to
> >follow
> >>> along with as yet unreleased features. It's a really nice way of
> >tracking
> >>> upcoming changes, especially on master.
> >>>
> >>>
> >>> Erick: Ah, that's what I suspected but feared. They've always lagged
> >a
> >>> little behind the commits, usually at most a week. But they usually
> >have
> >>> the next releases (and especially master) available. I can allocate
> >some
> >>> time to help maintain them if someone could point me in the right
> >direction.
> >>>
> >>> --Terry
> >>>
> >>>
> >>>
> >>>
> >>> On Tue, Jan 9, 2018 at 11:22 AM, Erick Erickson
> >
> >>> wrote:
> >>>
>  Maven support is not officially part of the project, it's
> >maintained on a
>  "when someone interested gets to it" basis.
> 
>  So the short answer is "no, you shouldn't expect those to be
> >absolutely
>  current"
> 
>  contributions welcome ;)
> 
>  Best,
>  Erick
> 
>  On Tue, Jan 9, 2018 at 6:36 AM, Armins Stepanjans <
>  armins.bagr...@gmail.com>
>  wrote:
> 
> > Hi,
> >
> > I'm not sure I understand your question.
> >
> > There should be no confusion about setting a Maven snapshot
> >dependency in
> > the pom file, as you can specify version with
> > 8.0-SNAPSHOT (substituting 8.0 with the version
> >you
> > want).
> >
> > However, in the case you are looking for a particular version of
> >Lucene,
> > you should check out the archives of released versions here:
> > http://archive.apache.org/dist/lucene/java/
> >
> > Is there a particular reason you want the snapshot of 7.2 or 7.3?
> >
> > Regards,
> > Armīns
> >
> > On Tue, Jan 9, 2018 at 4:13 PM, Terry Smith 
> >wrote:
> >
> >> Guys,
> >>
> >> I'm just following up in case this question slipped between the
> >cracks.
> >>
> >> Should I expect the apache snapshots maven repository to be
> >current for
> >> Lucene 7.x and 8? Specifically, I don't see snapshot releases for
> >7.2
>  or
> >> 7.3 and it looks like the 8.0 snapshot releases are pretty stale.
> >>
> >> Thanks,
> >>
> >> --Terry
> >>
> >>
> >>
> >>
> >> On Fri, Jan 5, 2018 at 11:06 AM, Terry Smith 
>  wrote:
> >>
> >>> Hi,
> >>>
> >>> I'm not seeing snapshot releases on the maven repository for 7.2
> >or
> > 7.3.
> >>> Is this on purpose?
> >>>
> >>> https://repository.apache.org/content/groups/snapshots/org/
> >>> apache/lucene/lucene-core/
> >>>
> >>> --Terry
> >>>
> >>>
> >>
> >
> 
> >>
> >
> >
> >-
> >To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org
> >For additional commands, e-mail: java-user-h...@lucene.apache.org
>
> --
> Uwe Schindler
> Achterdiek 19, 28357 Bremen
> https://www.thetaphi.de

Re: Maven snapshots

2018-01-11 Thread Steve Rowe


> On Jan 11, 2018, at 11:18 AM, Terry Smith  wrote:
> 
> Steve,
> 
> Thanks for looking into this. I see the artifacts for 7.2.1-SNAPSHOT,
> 7.3.0-SNAPSHOT, and 8.0.0-SNAPSHOT are now available so things are looking
> good.

Cool, thanks for checking on it.

> On Tue, Jan 9, 2018 at 4:38 PM, Uwe Schindler  wrote:
> 
>> If this fixed it could you please add it as dependency on the separate
>> Jenkins target? I prefer all Jenkins jobs only call a simple "Jenkins job"
>> target.

Done, see https://issues.apache.org/jira/browse/SOLR-11181

I’ll go modify the Jenkins jobs’ config now to remove the extra target.

--
Steve
www.lucidworks.com
-
To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org
For additional commands, e-mail: java-user-h...@lucene.apache.org

RE: Lucene OOM

Re: Lucene OOM

RE: Lucene OOM

Re: Lucene OOM

Re: Maven snapshots

Re: Maven snapshots

6 matches

Site Navigation

Mail list logo

Footer information