Sorry, missed the attachment in my previous mail...
On 15 November 2013 20:53, Vivek Dhayaal_Trichy <vivekdhay...@gmail.com>wrote:
> Hi,
>
> I've my lucene index directory of size 36GB. I recently upgraded my lucene
> version from v3.2 to v4.4. So, I tried to upgrade my index as suggested @
>
> http://lucene.apache.org/core/4_4_0/core/org/apache/lucene/index/IndexUpgrader.html
>
> I tried with -Xmx 4g(snip1 in attached log) but the process ran for 37min
> and hit OOME and failed. So, I tried with increased heap size 7g(snip2 in
> attached log). This time the process ran for 43min and hit OOME again and
> failed.
>
> How do I resolve the OOME and successfully upgrade my index to lucene 4
> format?
>
> Thanks,
> --Vivek
>
>
>
<snip1>
...
[build@cu046 vivek]$ time java -Xmx4g -cp lucene-core-4.4.0.jar
org.apache.lucene.index.IndexUpgrader -delete-prior-commits -verbose
/u1/vivek/searchIndex/
IFD 0 [Fri Nov 15 00:58:17 IST 2013; main]: init: current segments file is
"segments_1h097q";
deletionPolicy=org.apache.lucene.index.KeepOnlyLastCommitDeletionPolicy@6eb02efa
IFD 0 [Fri Nov 15 00:58:17 IST 2013; main]: init: load commit "segments_1h097q"
IFD 0 [Fri Nov 15 00:58:17 IST 2013; main]: now checkpoint
"_1h41d0(3.2):C6469899/21414 _1h5bwk(3.2):c38973/2782 _1h5pvt(3.2):c11112/1072
_1h5szp(3.2):c1409/123 _1h5um4(3.2):c1206/254 _1h5vmd(3.2):c1087/55
_1h5wp2(3.2):c1127/185 _1h5xx4(3.2):c1049/58 _1h5yzp(3.2):c1121/38
_1h60hm(3.2):c1316/31 _1h61a7(3.2):c788/1 _1h61a8(3.2):c1 _1h61a9(3.2):c1
_1h61ab(3.2):c1" [14 segments ; isCommit = false]
IFD 0 [Fri Nov 15 00:58:17 IST 2013; main]: 1 msec to checkpoint
IW 0 [Fri Nov 15 00:58:17 IST 2013; main]: init: create=false
IW 0 [Fri Nov 15 00:58:17 IST 2013; main]:
dir=org.apache.lucene.store.MMapDirectory@/u1/vivek/searchIndex
lockFactory=org.apache.lucene.store.NativeFSLockFactory@7825d2b2
index=_1h41d0(3.2):C6469899/21414 _1h5bwk(3.2):c38973/2782
_1h5pvt(3.2):c11112/1072 _1h5szp(3.2):c1409/123 _1h5um4(3.2):c1206/254
_1h5vmd(3.2):c1087/55 _1h5wp2(3.2):c1127/185 _1h5xx4(3.2):c1049/58
_1h5yzp(3.2):c1121/38 _1h60hm(3.2):c1316/31 _1h61a7(3.2):c788/1 _1h61a8(3.2):c1
_1h61a9(3.2):c1 _1h61ab(3.2):c1
version=4.4.0 1504776 - sarowe - 2013-07-19 02:49:47
matchVersion=LUCENE_CURRENT
analyzer=null
ramBufferSizeMB=16.0
maxBufferedDocs=-1
maxBufferedDeleteTerms=-1
mergedSegmentWarmer=null
readerTermsIndexDivisor=1
termIndexInterval=32
delPolicy=org.apache.lucene.index.KeepOnlyLastCommitDeletionPolicy
commit=null
openMode=CREATE_OR_APPEND
similarity=org.apache.lucene.search.similarities.DefaultSimilarity
mergeScheduler=ConcurrentMergeScheduler: maxThreadCount=1, maxMergeCount=2,
mergeThreadPriority=-1
default WRITE_LOCK_TIMEOUT=1000
writeLockTimeout=1000
codec=Lucene42
infoStream=org.apache.lucene.util.PrintStreamInfoStream
mergePolicy=[UpgradeIndexMergePolicy->[TieredMergePolicy: maxMergeAtOnce=10,
maxMergeAtOnceExplicit=30, maxMergedSegmentMB=5120.0, floorSegmentMB=2.0,
forceMergeDeletesPctAllowed=10.0, segmentsPerTier=10.0,
maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.1]
indexerThreadPool=org.apache.lucene.index.ThreadAffinityDocumentsWriterThreadPool@68e6ff0d
readerPooling=false
perThreadHardLimitMB=1945
useCompoundFile=true
IndexUpgrader 0 [Fri Nov 15 00:58:17 IST 2013; main]: Upgrading all pre-4.4
segments of index directory
'org.apache.lucene.store.MMapDirectory@/u1/vivek/searchIndex
lockFactory=org.apache.lucene.store.NativeFSLockFactory@7825d2b2' to version
4.4...
IW 0 [Fri Nov 15 00:58:17 IST 2013; main]: forceMerge: index now
_1h41d0(3.2):C6469899/21414 _1h5bwk(3.2):c38973/2782 _1h5pvt(3.2):c11112/1072
_1h5szp(3.2):c1409/123 _1h5um4(3.2):c1206/254 _1h5vmd(3.2):c1087/55
_1h5wp2(3.2):c1127/185 _1h5xx4(3.2):c1049/58 _1h5yzp(3.2):c1121/38
_1h60hm(3.2):c1316/31 _1h61a7(3.2):c788/1 _1h61a8(3.2):c1 _1h61a9(3.2):c1
_1h61ab(3.2):c1
IW 0 [Fri Nov 15 00:58:17 IST 2013; main]: now flush at forceMerge
IW 0 [Fri Nov 15 00:58:17 IST 2013; main]: start flush: applyAllDeletes=true
IW 0 [Fri Nov 15 00:58:17 IST 2013; main]: index before flush
_1h41d0(3.2):C6469899/21414 _1h5bwk(3.2):c38973/2782 _1h5pvt(3.2):c11112/1072
_1h5szp(3.2):c1409/123 _1h5um4(3.2):c1206/254 _1h5vmd(3.2):c1087/55
_1h5wp2(3.2):c1127/185 _1h5xx4(3.2):c1049/58 _1h5yzp(3.2):c1121/38
_1h60hm(3.2):c1316/31 _1h61a7(3.2):c788/1 _1h61a8(3.2):c1 _1h61a9(3.2):c1
_1h61ab(3.2):c1
DW 0 [Fri Nov 15 00:58:17 IST 2013; main]: main startFullFlush
DW 0 [Fri Nov 15 00:58:17 IST 2013; main]: anyChanges? numDocsInRam=0
deletes=false hasTickets:false pendingChangesInFullFlush: false
DW 0 [Fri Nov 15 00:58:17 IST 2013; main]: main finishFullFlush success=true
IW 0 [Fri Nov 15 00:58:17 IST 2013; main]: apply all deletes during flush
BD 0 [Fri Nov 15 00:58:17 IST 2013; main]: applyDeletes: no deletes; skipping
BD 0 [Fri Nov 15 00:58:17 IST 2013; main]: prune
sis=org.apache.lucene.index.SegmentInfos@288051 minGen=0 packetCount=0
UPGMP 0 [Fri Nov 15 00:58:17 IST 2013; main]: findForcedMerges:
segmentsToUpgrade={_1h61a9(3.2):c1=true,
_1h41d0(3.2):C6469899/21414:delGen=9844=true,
_1h5pvt(3.2):c11112/1072:delGen=413=true, _1h5vmd(3.2):c1087/55:delGen=38=true,
_1h5yzp(3.2):c1121/38:delGen=35=true, _1h60hm(3.2):c1316/31:delGen=25=true,
_1h5szp(3.2):c1409/123:delGen=83=true,
_1h5bwk(3.2):c38973/2782:delGen=1659=true, _1h61ab(3.2):c1=true,
_1h5xx4(3.2):c1049/58:delGen=36=true, _1h61a7(3.2):c788/1:delGen=1=true,
_1h5wp2(3.2):c1127/185:delGen=62=true, _1h5um4(3.2):c1206/254:delGen=47=true,
_1h61a8(3.2):c1=true}
TMP 0 [Fri Nov 15 00:58:17 IST 2013; main]: findForcedMerges maxSegmentCount=1
infos=_1h41d0(3.2):C6469899/21414 _1h5bwk(3.2):c38973/2782
_1h5pvt(3.2):c11112/1072 _1h5szp(3.2):c1409/123 _1h5um4(3.2):c1206/254
_1h5vmd(3.2):c1087/55 _1h5wp2(3.2):c1127/185 _1h5xx4(3.2):c1049/58
_1h5yzp(3.2):c1121/38 _1h60hm(3.2):c1316/31 _1h61a7(3.2):c788/1 _1h61a8(3.2):c1
_1h61a9(3.2):c1 _1h61ab(3.2):c1 segmentsToMerge={_1h61a9(3.2):c1=true,
_1h41d0(3.2):C6469899/21414:delGen=9844=true,
_1h5pvt(3.2):c11112/1072:delGen=413=true, _1h5vmd(3.2):c1087/55:delGen=38=true,
_1h5yzp(3.2):c1121/38:delGen=35=true, _1h60hm(3.2):c1316/31:delGen=25=true,
_1h5szp(3.2):c1409/123:delGen=83=true,
_1h5bwk(3.2):c38973/2782:delGen=1659=true, _1h61ab(3.2):c1=true,
_1h5xx4(3.2):c1049/58:delGen=36=true, _1h61a7(3.2):c788/1:delGen=1=true,
_1h5wp2(3.2):c1127/185:delGen=62=true, _1h5um4(3.2):c1206/254:delGen=47=true,
_1h61a8(3.2):c1=true}
TMP 0 [Fri Nov 15 00:58:17 IST 2013; main]:
eligible=[_1h41d0(3.2):C6469899/21414:delGen=9844,
_1h5bwk(3.2):c38973/2782:delGen=1659, _1h5pvt(3.2):c11112/1072:delGen=413,
_1h5yzp(3.2):c1121/38:delGen=35, _1h5um4(3.2):c1206/254:delGen=47,
_1h5xx4(3.2):c1049/58:delGen=36, _1h5vmd(3.2):c1087/55:delGen=38,
_1h5wp2(3.2):c1127/185:delGen=62, _1h60hm(3.2):c1316/31:delGen=25,
_1h5szp(3.2):c1409/123:delGen=83, _1h61a7(3.2):c788/1:delGen=1,
_1h61a9(3.2):c1, _1h61ab(3.2):c1, _1h61a
8(3.2):c1]
TMP 0 [Fri Nov 15 00:58:17 IST 2013; main]: forceMergeRunning=false
TMP 0 [Fri Nov 15 00:58:17 IST 2013; main]: add final
merge=_1h41d0(3.2):C6469899/21414 _1h5bwk(3.2):c38973/2782
_1h5pvt(3.2):c11112/1072 _1h5yzp(3.2):c1121/38 _1h5um4(3.2):c1206/254
_1h5xx4(3.2):c1049/58 _1h5vmd(3.2):c1087/55 _1h5wp2(3.2):c1127/185
_1h60hm(3.2):c1316/31 _1h5szp(3.2):c1409/123 _1h61a7(3.2):c788/1
_1h61a9(3.2):c1 _1h61ab(3.2):c1 _1h61a8(3.2):c1
IW 0 [Fri Nov 15 00:58:17 IST 2013; main]: add merge to pendingMerges:
_1h41d0(3.2):C6469899/21414 _1h5bwk(3.2):c38973/2782 _1h5pvt(3.2):c11112/1072
_1h5yzp(3.2):c1121/38 _1h5um4(3.2):c1206/254 _1h5xx4(3.2):c1049/58
_1h5vmd(3.2):c1087/55 _1h5wp2(3.2):c1127/185 _1h60hm(3.2):c1316/31
_1h5szp(3.2):c1409/123 _1h61a7(3.2):c788/1 _1h61a9(3.2):c1 _1h61ab(3.2):c1
_1h61a8(3.2):c1 [total 1 pending]
IW 0 [Fri Nov 15 00:58:17 IST 2013; main]: registerMerge merging= []
IW 0 [Fri Nov 15 00:58:17 IST 2013; main]: registerMerge
info=_1h41d0(3.2):C6469899/21414
IW 0 [Fri Nov 15 00:58:17 IST 2013; main]: registerMerge
info=_1h5bwk(3.2):c38973/2782
IW 0 [Fri Nov 15 00:58:17 IST 2013; main]: registerMerge
info=_1h5pvt(3.2):c11112/1072
IW 0 [Fri Nov 15 00:58:17 IST 2013; main]: registerMerge
info=_1h5yzp(3.2):c1121/38
IW 0 [Fri Nov 15 00:58:17 IST 2013; main]: registerMerge
info=_1h5um4(3.2):c1206/254
IW 0 [Fri Nov 15 00:58:17 IST 2013; main]: registerMerge
info=_1h5xx4(3.2):c1049/58
IW 0 [Fri Nov 15 00:58:17 IST 2013; main]: registerMerge
info=_1h5vmd(3.2):c1087/55
IW 0 [Fri Nov 15 00:58:17 IST 2013; main]: registerMerge
info=_1h5wp2(3.2):c1127/185
IW 0 [Fri Nov 15 00:58:17 IST 2013; main]: registerMerge
info=_1h60hm(3.2):c1316/31
IW 0 [Fri Nov 15 00:58:17 IST 2013; main]: registerMerge
info=_1h5szp(3.2):c1409/123
IW 0 [Fri Nov 15 00:58:17 IST 2013; main]: registerMerge
info=_1h61a7(3.2):c788/1
IW 0 [Fri Nov 15 00:58:17 IST 2013; main]: registerMerge info=_1h61a9(3.2):c1
IW 0 [Fri Nov 15 00:58:17 IST 2013; main]: registerMerge info=_1h61ab(3.2):c1
IW 0 [Fri Nov 15 00:58:17 IST 2013; main]: registerMerge info=_1h61a8(3.2):c1
CMS 0 [Fri Nov 15 00:58:17 IST 2013; main]: now merge
CMS 0 [Fri Nov 15 00:58:17 IST 2013; main]: index:
_1h41d0(3.2):C6469899/21414 _1h5bwk(3.2):c38973/2782 _1h5pvt(3.2):c11112/1072
_1h5szp(3.2):c1409/123 _1h5um4(3.2):c1206/254 _1h5vmd(3.2):c1087/55
_1h5wp2(3.2):c1127/185 _1h5xx4(3.2):c1049/58 _1h5yzp(3.2):c1121/38
_1h60hm(3.2):c1316/31 _1h61a7(3.2):c788/1 _1h61a8(3.2):c1 _1h61a9(3.2):c1
_1h61ab(3.2):c1
CMS 0 [Fri Nov 15 00:58:17 IST 2013; main]: consider merge
_1h41d0(3.2):C6469899/21414 _1h5bwk(3.2):c38973/2782 _1h5pvt(3.2):c11112/1072
_1h5yzp(3.2):c1121/38 _1h5um4(3.2):c1206/254 _1h5xx4(3.2):c1049/58
_1h5vmd(3.2):c1087/55 _1h5wp2(3.2):c1127/185 _1h60hm(3.2):c1316/31
_1h5szp(3.2):c1409/123 _1h61a7(3.2):c788/1 _1h61a9(3.2):c1 _1h61ab(3.2):c1
_1h61a8(3.2):c1
CMS 0 [Fri Nov 15 00:58:17 IST 2013; main]: launch new thread [Lucene Merge
Thread #0]
CMS 0 [Fri Nov 15 00:58:17 IST 2013; main]: set priority of merge thread Lucene
Merge Thread #0 to 6
CMS 0 [Fri Nov 15 00:58:17 IST 2013; main]: no more merges pending; now return
CMS 0 [Fri Nov 15 00:58:17 IST 2013; Lucene Merge Thread #0]: merge thread:
start
BD 0 [Fri Nov 15 00:58:17 IST 2013; Lucene Merge Thread #0]: applyDeletes: no
deletes; skipping
BD 0 [Fri Nov 15 00:58:17 IST 2013; Lucene Merge Thread #0]: prune
sis=org.apache.lucene.index.SegmentInfos@288051 minGen=0 packetCount=0
IW 0 [Fri Nov 15 00:58:17 IST 2013; Lucene Merge Thread #0]: merge seg=_1h61ac
_1h41d0(3.2):C6469899/21414 _1h5bwk(3.2):c38973/2782 _1h5pvt(3.2):c11112/1072
_1h5yzp(3.2):c1121/38 _1h5um4(3.2):c1206/254 _1h5xx4(3.2):c1049/58
_1h5vmd(3.2):c1087/55 _1h5wp2(3.2):c1127/185 _1h60hm(3.2):c1316/31
_1h5szp(3.2):c1409/123 _1h61a7(3.2):c788/1 _1h61a9(3.2):c1 _1h61ab(3.2):c1
_1h61a8(3.2):c1
IW 0 [Fri Nov 15 00:58:17 IST 2013; Lucene Merge Thread #0]: now merge
merge=_1h41d0(3.2):C6469899/21414 _1h5bwk(3.2):c38973/2782
_1h5pvt(3.2):c11112/1072 _1h5yzp(3.2):c1121/38 _1h5um4(3.2):c1206/254
_1h5xx4(3.2):c1049/58 _1h5vmd(3.2):c1087/55 _1h5wp2(3.2):c1127/185
_1h60hm(3.2):c1316/31 _1h5szp(3.2):c1409/123 _1h61a7(3.2):c788/1
_1h61a9(3.2):c1 _1h61ab(3.2):c1 _1h61a8(3.2):c1
index=_1h41d0(3.2):C6469899/21414 _1h5bwk(3.2):c38973/2782
_1h5pvt(3.2):c11112/1072 _1h5szp(3.2):c1409/123 _1h5um4(3.2):c1206/254
_1h5vmd(3.2):c1087/55 _1h5wp2(3.2):c1127/185 _1h5xx4(3.2):c1049/58
_1h5yzp(3.2):c1121/38 _1h60hm(3.2):c1316/31 _1h61a7(3.2):c788/1 _1h61a8(3.2):c1
_1h61a9(3.2):c1 _1h61ab(3.2):c1
IW 0 [Fri Nov 15 00:58:17 IST 2013; Lucene Merge Thread #0]: merging
_1h41d0(3.2):C6469899/21414 _1h5bwk(3.2):c38973/2782 _1h5pvt(3.2):c11112/1072
_1h5yzp(3.2):c1121/38 _1h5um4(3.2):c1206/254 _1h5xx4(3.2):c1049/58
_1h5vmd(3.2):c1087/55 _1h5wp2(3.2):c1127/185 _1h60hm(3.2):c1316/31
_1h5szp(3.2):c1409/123 _1h61a7(3.2):c788/1 _1h61a9(3.2):c1 _1h61ab(3.2):c1
_1h61a8(3.2):c1
IW 0 [Fri Nov 15 00:58:19 IST 2013; Lucene Merge Thread #0]:
seg=_1h41d0(3.2):C6469899/21414 delCount=21414
IW 0 [Fri Nov 15 00:58:20 IST 2013; Lucene Merge Thread #0]:
seg=_1h5bwk(3.2):c38973/2782 delCount=2782
IW 0 [Fri Nov 15 00:58:20 IST 2013; Lucene Merge Thread #0]:
seg=_1h5pvt(3.2):c11112/1072 delCount=1072
IW 0 [Fri Nov 15 00:58:20 IST 2013; Lucene Merge Thread #0]:
seg=_1h5yzp(3.2):c1121/38 delCount=38
IW 0 [Fri Nov 15 00:58:20 IST 2013; Lucene Merge Thread #0]:
seg=_1h5um4(3.2):c1206/254 delCount=254
IW 0 [Fri Nov 15 00:58:20 IST 2013; Lucene Merge Thread #0]:
seg=_1h5xx4(3.2):c1049/58 delCount=58
IW 0 [Fri Nov 15 00:58:20 IST 2013; Lucene Merge Thread #0]:
seg=_1h5vmd(3.2):c1087/55 delCount=55
IW 0 [Fri Nov 15 00:58:20 IST 2013; Lucene Merge Thread #0]:
seg=_1h5wp2(3.2):c1127/185 delCount=185
IW 0 [Fri Nov 15 00:58:20 IST 2013; Lucene Merge Thread #0]:
seg=_1h60hm(3.2):c1316/31 delCount=31
IW 0 [Fri Nov 15 00:58:20 IST 2013; Lucene Merge Thread #0]:
seg=_1h5szp(3.2):c1409/123 delCount=123
IW 0 [Fri Nov 15 00:58:20 IST 2013; Lucene Merge Thread #0]:
seg=_1h61a7(3.2):c788/1 delCount=1
IW 0 [Fri Nov 15 00:58:20 IST 2013; Lucene Merge Thread #0]:
seg=_1h61a9(3.2):c1 no deletes
IW 0 [Fri Nov 15 00:58:20 IST 2013; Lucene Merge Thread #0]:
seg=_1h61ab(3.2):c1 no deletes
IW 0 [Fri Nov 15 00:58:20 IST 2013; Lucene Merge Thread #0]:
seg=_1h61a8(3.2):c1 no deletes
SM 0 [Fri Nov 15 00:58:20 IST 2013; Lucene Merge Thread #0]: merge store
matchedCount=14 vs 14
SM 0 [Fri Nov 15 00:59:43 IST 2013; Lucene Merge Thread #0]: 82903 msec to
merge stored fields [6503077 docs]
SM 0 [Fri Nov 15 01:30:22 IST 2013; Lucene Merge Thread #0]: 1839524 msec to
merge postings [6503077 docs]
SM 0 [Fri Nov 15 01:30:22 IST 2013; Lucene Merge Thread #0]: 0 msec to merge
doc values [6503077 docs]
IFD 0 [Fri Nov 15 01:35:31 IST 2013; Lucene Merge Thread #0]: refresh
[prefix=_1h61ac]: removing newly created unreferenced file "_1h61ac.fdx"
IFD 0 [Fri Nov 15 01:35:31 IST 2013; Lucene Merge Thread #0]: delete
"_1h61ac.fdx"
IFD 0 [Fri Nov 15 01:35:31 IST 2013; Lucene Merge Thread #0]: refresh
[prefix=_1h61ac]: removing newly created unreferenced file
"_1h61ac_Lucene41_0.doc"
IFD 0 [Fri Nov 15 01:35:31 IST 2013; Lucene Merge Thread #0]: delete
"_1h61ac_Lucene41_0.doc"
IFD 0 [Fri Nov 15 01:35:31 IST 2013; Lucene Merge Thread #0]: refresh
[prefix=_1h61ac]: removing newly created unreferenced file
"_1h61ac_Lucene41_0.pos"
IFD 0 [Fri Nov 15 01:35:31 IST 2013; Lucene Merge Thread #0]: delete
"_1h61ac_Lucene41_0.pos"
IFD 0 [Fri Nov 15 01:35:31 IST 2013; Lucene Merge Thread #0]: refresh
[prefix=_1h61ac]: removing newly created unreferenced file "_1h61ac.nvm"
IFD 0 [Fri Nov 15 01:35:31 IST 2013; Lucene Merge Thread #0]: delete
"_1h61ac.nvm"
IFD 0 [Fri Nov 15 01:35:31 IST 2013; Lucene Merge Thread #0]: refresh
[prefix=_1h61ac]: removing newly created unreferenced file
"_1h61ac_Lucene41_0.tip"
IFD 0 [Fri Nov 15 01:35:31 IST 2013; Lucene Merge Thread #0]: delete
"_1h61ac_Lucene41_0.tip"
IFD 0 [Fri Nov 15 01:35:31 IST 2013; Lucene Merge Thread #0]: refresh
[prefix=_1h61ac]: removing newly created unreferenced file "_1h61ac.fdt"
IFD 0 [Fri Nov 15 01:35:31 IST 2013; Lucene Merge Thread #0]: delete
"_1h61ac.fdt"
IFD 0 [Fri Nov 15 01:35:31 IST 2013; Lucene Merge Thread #0]: refresh
[prefix=_1h61ac]: removing newly created unreferenced file "_1h61ac.nvd"
IFD 0 [Fri Nov 15 01:35:31 IST 2013; Lucene Merge Thread #0]: delete
"_1h61ac.nvd"
IFD 0 [Fri Nov 15 01:35:31 IST 2013; Lucene Merge Thread #0]: refresh
[prefix=_1h61ac]: removing newly created unreferenced file
"_1h61ac_Lucene41_0.tim"
IFD 0 [Fri Nov 15 01:35:31 IST 2013; Lucene Merge Thread #0]: delete
"_1h61ac_Lucene41_0.tim"
IW 0 [Fri Nov 15 01:35:31 IST 2013; Lucene Merge Thread #0]:
handleMergeException: merge=_1h41d0(3.2):C6469899/21414
_1h5bwk(3.2):c38973/2782 _1h5pvt(3.2):c11112/1072 _1h5yzp(3.2):c1121/38
_1h5um4(3.2):c1206/254 _1h5xx4(3.2):c1049/58 _1h5vmd(3.2):c1087/55
_1h5wp2(3.2):c1127/185 _1h60hm(3.2):c1316/31 _1h5szp(3.2):c1409/123
_1h61a7(3.2):c788/1 _1h61a9(3.2):c1 _1h61ab(3.2):c1 _1h61a8(3.2):c1
exc=java.lang.OutOfMemoryError: Java heap space
IW 0 [Fri Nov 15 01:35:31 IST 2013; Lucene Merge Thread #0]: hit exception
during merge
IW 0 [Fri Nov 15 01:35:31 IST 2013; Lucene Merge Thread #0]: hit
OutOfMemoryError inside merge
IW 0 [Fri Nov 15 01:35:31 IST 2013; main]: rollback
IW 0 [Fri Nov 15 01:35:31 IST 2013; main]: all running merges have aborted
IW 0 [Fri Nov 15 01:35:31 IST 2013; main]: rollback: done finish merges
Exception in thread "Lucene Merge Thread #0"
org.apache.lucene.index.MergePolicy$MergeException: java.lang.OutOfMemoryError:
Java heap space
at
org.apache.lucene.index.ConcurrentMergeScheduler.handleMergeException(ConcurrentMergeScheduler.java:545)
at
org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:518)
Caused by: java.lang.OutOfMemoryError: Java heap space
at
org.apache.lucene.codecs.lucene3x.Lucene3xNormsProducer$NormsDocValues.getInstance(Lucene3xNormsProducer.java:172)
at
org.apache.lucene.codecs.lucene3x.Lucene3xNormsProducer.getNumeric(Lucene3xNormsProducer.java:198)
at
org.apache.lucene.index.SegmentCoreReaders.getNormValues(SegmentCoreReaders.java:301)
at
org.apache.lucene.index.SegmentReader.getNormValues(SegmentReader.java:253)
at
org.apache.lucene.index.SegmentMerger.mergeNorms(SegmentMerger.java:215)
at org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:119)
at
org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:3772)
at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3376)
at
org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:405)
at
org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:482)
DW 0 [Fri Nov 15 01:35:32 IST 2013; main]: abort
DW 0 [Fri Nov 15 01:35:32 IST 2013; main]: done abort; abortedFiles=null
success=true
IW 0 [Fri Nov 15 01:35:32 IST 2013; main]: rollback:
infos=_1h41d0(3.2):C6469899/21414 _1h5bwk(3.2):c38973/2782
_1h5pvt(3.2):c11112/1072 _1h5szp(3.2):c1409/123 _1h5um4(3.2):c1206/254
_1h5vmd(3.2):c1087/55 _1h5wp2(3.2):c1127/185 _1h5xx4(3.2):c1049/58
_1h5yzp(3.2):c1121/38 _1h60hm(3.2):c1316/31 _1h61a7(3.2):c788/1 _1h61a8(3.2):c1
_1h61a9(3.2):c1 _1h61ab(3.2):c1
IFD 0 [Fri Nov 15 01:35:32 IST 2013; main]: now checkpoint
"_1h41d0(3.2):C6469899/21414 _1h5bwk(3.2):c38973/2782 _1h5pvt(3.2):c11112/1072
_1h5szp(3.2):c1409/123 _1h5um4(3.2):c1206/254 _1h5vmd(3.2):c1087/55
_1h5wp2(3.2):c1127/185 _1h5xx4(3.2):c1049/58 _1h5yzp(3.2):c1121/38
_1h60hm(3.2):c1316/31 _1h61a7(3.2):c788/1 _1h61a8(3.2):c1 _1h61a9(3.2):c1
_1h61ab(3.2):c1" [14 segments ; isCommit = false]
IFD 0 [Fri Nov 15 01:35:32 IST 2013; main]: 0 msec to checkpoint
IW 0 [Fri Nov 15 01:35:32 IST 2013; main]: now flush at close
waitForMerges=false
DW 0 [Fri Nov 15 01:35:32 IST 2013; main]: abort
DW 0 [Fri Nov 15 01:35:32 IST 2013; main]: done abort; abortedFiles=null
success=true
IW 0 [Fri Nov 15 01:35:32 IST 2013; main]: all running merges have aborted
IW 0 [Fri Nov 15 01:35:32 IST 2013; main]: now call final commit()
IW 0 [Fri Nov 15 01:35:32 IST 2013; main]: at close:
_1h41d0(3.2):C6469899/21414 _1h5bwk(3.2):c38973/2782 _1h5pvt(3.2):c11112/1072
_1h5szp(3.2):c1409/123 _1h5um4(3.2):c1206/254 _1h5vmd(3.2):c1087/55
_1h5wp2(3.2):c1127/185 _1h5xx4(3.2):c1049/58 _1h5yzp(3.2):c1121/38
_1h60hm(3.2):c1316/31 _1h61a7(3.2):c788/1 _1h61a8(3.2):c1 _1h61a9(3.2):c1
_1h61ab(3.2):c1
Exception in thread "main" java.io.IOException: background merge hit exception:
_1h41d0(3.2):C6469899/21414 _1h5bwk(3.2):c38973/2782 _1h5pvt(3.2):c11112/1072
_1h5yzp(3.2):c1121/38 _1h5um4(3.2):c1206/254 _1h5xx4(3.2):c1049/58
_1h5vmd(3.2):c1087/55 _1h5wp2(3.2):c1127/185 _1h60hm(3.2):c1316/31
_1h5szp(3.2):c1409/123 _1h61a7(3.2):c788/1 _1h61a9(3.2):c1 _1h61ab(3.2):c1
_1h61a8(3.2):c1 into _1h61ac [maxNumSegments=1]
at org.apache.lucene.index.IndexWriter.forceMerge(IndexWriter.java:1714)
at org.apache.lucene.index.IndexWriter.forceMerge(IndexWriter.java:1650)
at org.apache.lucene.index.IndexUpgrader.upgrade(IndexUpgrader.java:161)
at org.apache.lucene.index.IndexUpgrader.main(IndexUpgrader.java:109)
Caused by: java.lang.OutOfMemoryError: Java heap space
at
org.apache.lucene.codecs.lucene3x.Lucene3xNormsProducer$NormsDocValues.getInstance(Lucene3xNormsProducer.java:172)
at
org.apache.lucene.codecs.lucene3x.Lucene3xNormsProducer.getNumeric(Lucene3xNormsProducer.java:198)
at
org.apache.lucene.index.SegmentCoreReaders.getNormValues(SegmentCoreReaders.java:301)
at
org.apache.lucene.index.SegmentReader.getNormValues(SegmentReader.java:253)
at
org.apache.lucene.index.SegmentMerger.mergeNorms(SegmentMerger.java:215)
at org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:119)
at
org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:3772)
at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3376)
at
org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:405)
at
org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:482)
real 37m17.169s
user 32m36.797s
sys 1m40.005s
...
</snip1>
<snip2>
...
[build@cu046 vivek]$ time java -Xmx7g -cp lucene-core-4.4.0.jar
org.apache.lucene.index.IndexUpgrader -delete-prior-commits -verbose
/u1/vivek/searchIndex/
IFD 0 [Fri Nov 15 01:37:17 IST 2013; main]: init: current segments file is
"segments_1h097q";
deletionPolicy=org.apache.lucene.index.KeepOnlyLastCommitDeletionPolicy@70dc57db
IFD 0 [Fri Nov 15 01:37:17 IST 2013; main]: init: load commit "segments_1h097q"
IFD 0 [Fri Nov 15 01:37:17 IST 2013; main]: now checkpoint
"_1h41d0(3.2):C6469899/21414 _1h5bwk(3.2):c38973/2782 _1h5pvt(3.2):c11112/1072
_1h5szp(3.2):c1409/123 _1h5um4(3.2):c1206/254 _1h5vmd(3.2):c1087/55
_1h5wp2(3.2):c1127/185 _1h5xx4(3.2):c1049/58 _1h5yzp(3.2):c1121/38
_1h60hm(3.2):c1316/31 _1h61a7(3.2):c788/1 _1h61a8(3.2):c1 _1h61a9(3.2):c1
_1h61ab(3.2):c1" [14 segments ; isCommit = false]
IFD 0 [Fri Nov 15 01:37:17 IST 2013; main]: 1 msec to checkpoint
IW 0 [Fri Nov 15 01:37:17 IST 2013; main]: init: create=false
IW 0 [Fri Nov 15 01:37:17 IST 2013; main]:
dir=org.apache.lucene.store.MMapDirectory@/u1/vivek/searchIndex
lockFactory=org.apache.lucene.store.NativeFSLockFactory@7b2be1bd
index=_1h41d0(3.2):C6469899/21414 _1h5bwk(3.2):c38973/2782
_1h5pvt(3.2):c11112/1072 _1h5szp(3.2):c1409/123 _1h5um4(3.2):c1206/254
_1h5vmd(3.2):c1087/55 _1h5wp2(3.2):c1127/185 _1h5xx4(3.2):c1049/58
_1h5yzp(3.2):c1121/38 _1h60hm(3.2):c1316/31 _1h61a7(3.2):c788/1 _1h61a8(3.2):c1
_1h61a9(3.2):c1 _1h61ab(3.2):c1
version=4.4.0 1504776 - sarowe - 2013-07-19 02:49:47
matchVersion=LUCENE_CURRENT
analyzer=null
ramBufferSizeMB=16.0
maxBufferedDocs=-1
maxBufferedDeleteTerms=-1
mergedSegmentWarmer=null
readerTermsIndexDivisor=1
termIndexInterval=32
delPolicy=org.apache.lucene.index.KeepOnlyLastCommitDeletionPolicy
commit=null
openMode=CREATE_OR_APPEND
similarity=org.apache.lucene.search.similarities.DefaultSimilarity
mergeScheduler=ConcurrentMergeScheduler: maxThreadCount=1, maxMergeCount=2,
mergeThreadPriority=-1
default WRITE_LOCK_TIMEOUT=1000
writeLockTimeout=1000
codec=Lucene42
infoStream=org.apache.lucene.util.PrintStreamInfoStream
mergePolicy=[UpgradeIndexMergePolicy->[TieredMergePolicy: maxMergeAtOnce=10,
maxMergeAtOnceExplicit=30, maxMergedSegmentMB=5120.0, floorSegmentMB=2.0,
forceMergeDeletesPctAllowed=10.0, segmentsPerTier=10.0,
maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.1]
indexerThreadPool=org.apache.lucene.index.ThreadAffinityDocumentsWriterThreadPool@1572e449
readerPooling=false
perThreadHardLimitMB=1945
useCompoundFile=true
IndexUpgrader 0 [Fri Nov 15 01:37:17 IST 2013; main]: Upgrading all pre-4.4
segments of index directory
'org.apache.lucene.store.MMapDirectory@/u1/vivek/searchIndex
lockFactory=org.apache.lucene.store.NativeFSLockFactory@7b2be1bd' to version
4.4...
IW 0 [Fri Nov 15 01:37:17 IST 2013; main]: forceMerge: index now
_1h41d0(3.2):C6469899/21414 _1h5bwk(3.2):c38973/2782 _1h5pvt(3.2):c11112/1072
_1h5szp(3.2):c1409/123 _1h5um4(3.2):c1206/254 _1h5vmd(3.2):c1087/55
_1h5wp2(3.2):c1127/185 _1h5xx4(3.2):c1049/58 _1h5yzp(3.2):c1121/38
_1h60hm(3.2):c1316/31 _1h61a7(3.2):c788/1 _1h61a8(3.2):c1 _1h61a9(3.2):c1
_1h61ab(3.2):c1
IW 0 [Fri Nov 15 01:37:17 IST 2013; main]: now flush at forceMerge
IW 0 [Fri Nov 15 01:37:17 IST 2013; main]: start flush: applyAllDeletes=true
IW 0 [Fri Nov 15 01:37:17 IST 2013; main]: index before flush
_1h41d0(3.2):C6469899/21414 _1h5bwk(3.2):c38973/2782 _1h5pvt(3.2):c11112/1072
_1h5szp(3.2):c1409/123 _1h5um4(3.2):c1206/254 _1h5vmd(3.2):c1087/55
_1h5wp2(3.2):c1127/185 _1h5xx4(3.2):c1049/58 _1h5yzp(3.2):c1121/38
_1h60hm(3.2):c1316/31 _1h61a7(3.2):c788/1 _1h61a8(3.2):c1 _1h61a9(3.2):c1
_1h61ab(3.2):c1
DW 0 [Fri Nov 15 01:37:17 IST 2013; main]: main startFullFlush
DW 0 [Fri Nov 15 01:37:17 IST 2013; main]: anyChanges? numDocsInRam=0
deletes=false hasTickets:false pendingChangesInFullFlush: false
DW 0 [Fri Nov 15 01:37:17 IST 2013; main]: main finishFullFlush success=true
IW 0 [Fri Nov 15 01:37:17 IST 2013; main]: apply all deletes during flush
BD 0 [Fri Nov 15 01:37:17 IST 2013; main]: applyDeletes: no deletes; skipping
BD 0 [Fri Nov 15 01:37:17 IST 2013; main]: prune
sis=org.apache.lucene.index.SegmentInfos@630045eb minGen=0 packetCount=0
UPGMP 0 [Fri Nov 15 01:37:17 IST 2013; main]: findForcedMerges:
segmentsToUpgrade={_1h61a8(3.2):c1=true,
_1h5bwk(3.2):c38973/2782:delGen=1659=true,
_1h5um4(3.2):c1206/254:delGen=47=true, _1h5xx4(3.2):c1049/58:delGen=36=true,
_1h5yzp(3.2):c1121/38:delGen=35=true, _1h5pvt(3.2):c11112/1072:delGen=413=true,
_1h41d0(3.2):C6469899/21414:delGen=9844=true,
_1h5wp2(3.2):c1127/185:delGen=62=true, _1h61a9(3.2):c1=true,
_1h61ab(3.2):c1=true, _1h60hm(3.2):c1316/31:delGen=25=true,
_1h5vmd(3.2):c1087/55:delGen=38=true, _1h5szp(3.2):c1409/123:delGen=83=true,
_1h61a7(3.2):c788/1:delGen=1=true}
TMP 0 [Fri Nov 15 01:37:17 IST 2013; main]: findForcedMerges maxSegmentCount=1
infos=_1h41d0(3.2):C6469899/21414 _1h5bwk(3.2):c38973/2782
_1h5pvt(3.2):c11112/1072 _1h5szp(3.2):c1409/123 _1h5um4(3.2):c1206/254
_1h5vmd(3.2):c1087/55 _1h5wp2(3.2):c1127/185 _1h5xx4(3.2):c1049/58
_1h5yzp(3.2):c1121/38 _1h60hm(3.2):c1316/31 _1h61a7(3.2):c788/1 _1h61a8(3.2):c1
_1h61a9(3.2):c1 _1h61ab(3.2):c1 segmentsToMerge={_1h61a8(3.2):c1=true,
_1h5bwk(3.2):c38973/2782:delGen=1659=true,
_1h5um4(3.2):c1206/254:delGen=47=true, _1h5xx4(3.2):c1049/58:delGen=36=true,
_1h5yzp(3.2):c1121/38:delGen=35=true, _1h5pvt(3.2):c11112/1072:delGen=413=true,
_1h41d0(3.2):C6469899/21414:delGen=9844=true,
_1h5wp2(3.2):c1127/185:delGen=62=true, _1h61a9(3.2):c1=true,
_1h61ab(3.2):c1=true, _1h60hm(3.2):c1316/31:delGen=25=true,
_1h5vmd(3.2):c1087/55:delGen=38=true, _1h5szp(3.2):c1409/123:delGen=83=true,
_1h61a7(3.2):c788/1:delGen=1=true}
TMP 0 [Fri Nov 15 01:37:17 IST 2013; main]:
eligible=[_1h41d0(3.2):C6469899/21414:delGen=9844,
_1h5bwk(3.2):c38973/2782:delGen=1659, _1h5pvt(3.2):c11112/1072:delGen=413,
_1h5yzp(3.2):c1121/38:delGen=35, _1h5um4(3.2):c1206/254:delGen=47,
_1h5xx4(3.2):c1049/58:delGen=36, _1h5vmd(3.2):c1087/55:delGen=38,
_1h5wp2(3.2):c1127/185:delGen=62, _1h60hm(3.2):c1316/31:delGen=25,
_1h5szp(3.2):c1409/123:delGen=83, _1h61a7(3.2):c788/1:delGen=1,
_1h61a9(3.2):c1, _1h61ab(3.2):c1, _1h61a8(3.2):c1]
TMP 0 [Fri Nov 15 01:37:17 IST 2013; main]: forceMergeRunning=false
TMP 0 [Fri Nov 15 01:37:17 IST 2013; main]: add final
merge=_1h41d0(3.2):C6469899/21414 _1h5bwk(3.2):c38973/2782
_1h5pvt(3.2):c11112/1072 _1h5yzp(3.2):c1121/38 _1h5um4(3.2):c1206/254
_1h5xx4(3.2):c1049/58 _1h5vmd(3.2):c1087/55 _1h5wp2(3.
2):c1127/185 _1h60hm(3.2):c1316/31 _1h5szp(3.2):c1409/123 _1h61a7(3.2):c788/1
_1h61a9(3.2):c1 _1h61ab(3.2):c1 _1h61a8(3.2):c1
IW 0 [Fri Nov 15 01:37:17 IST 2013; main]: add merge to pendingMerges:
_1h41d0(3.2):C6469899/21414 _1h5bwk(3.2):c38973/2782 _1h5pvt(3.2):c11112/1072
_1h5yzp(3.2):c1121/38 _1h5um4(3.2):c1206/254 _1h5xx4(3.2):c1049/58
_1h5vmd(3.2):c1087/55 _1h5wp2(3.2):c1127/185 _1h60hm(3.2):c1316/31
_1h5szp(3.2):c1409/123 _1h61a7(3.2):c788/1 _1h61a9(3.2):c1 _1h61ab(3.2):c1
_1h61a8(3.2):c1 [total 1 pending]
IW 0 [Fri Nov 15 01:37:17 IST 2013; main]: registerMerge merging= []
IW 0 [Fri Nov 15 01:37:17 IST 2013; main]: registerMerge
info=_1h41d0(3.2):C6469899/21414
IW 0 [Fri Nov 15 01:37:17 IST 2013; main]: registerMerge
info=_1h5bwk(3.2):c38973/2782
IW 0 [Fri Nov 15 01:37:17 IST 2013; main]: registerMerge
info=_1h5pvt(3.2):c11112/1072
IW 0 [Fri Nov 15 01:37:17 IST 2013; main]: registerMerge
info=_1h5yzp(3.2):c1121/38
IW 0 [Fri Nov 15 01:37:17 IST 2013; main]: registerMerge
info=_1h5um4(3.2):c1206/254
IW 0 [Fri Nov 15 01:37:17 IST 2013; main]: registerMerge
info=_1h5xx4(3.2):c1049/58
IW 0 [Fri Nov 15 01:37:17 IST 2013; main]: registerMerge
info=_1h5vmd(3.2):c1087/55
IW 0 [Fri Nov 15 01:37:17 IST 2013; main]: registerMerge
info=_1h5wp2(3.2):c1127/185
IW 0 [Fri Nov 15 01:37:17 IST 2013; main]: registerMerge
info=_1h60hm(3.2):c1316/31
IW 0 [Fri Nov 15 01:37:17 IST 2013; main]: registerMerge
info=_1h5szp(3.2):c1409/123
IW 0 [Fri Nov 15 01:37:17 IST 2013; main]: registerMerge
info=_1h61a7(3.2):c788/1
IW 0 [Fri Nov 15 01:37:17 IST 2013; main]: registerMerge info=_1h61a9(3.2):c1
IW 0 [Fri Nov 15 01:37:17 IST 2013; main]: registerMerge info=_1h61ab(3.2):c1
IW 0 [Fri Nov 15 01:37:17 IST 2013; main]: registerMerge info=_1h61a8(3.2):c1
CMS 0 [Fri Nov 15 01:37:17 IST 2013; main]: now merge
CMS 0 [Fri Nov 15 01:37:17 IST 2013; main]: index:
_1h41d0(3.2):C6469899/21414 _1h5bwk(3.2):c38973/2782 _1h5pvt(3.2):c11112/1072
_1h5szp(3.2):c1409/123 _1h5um4(3.2):c1206/254 _1h5vmd(3.2):c1087/55
_1h5wp2(3.2):c1127/185 _1h5xx4(3.2):c1049/58 _1h5yzp(3.2):c1121/38
_1h60hm(3.2):c1316/31 _1h61a7(3.2):c788/1 _1h61a8(3.2):c1 _1h61a9(3.2):c1
_1h61ab(3.2):c1
CMS 0 [Fri Nov 15 01:37:17 IST 2013; main]: consider merge
_1h41d0(3.2):C6469899/21414 _1h5bwk(3.2):c38973/2782 _1h5pvt(3.2):c11112/1072
_1h5yzp(3.2):c1121/38 _1h5um4(3.2):c1206/254 _1h5xx4(3.2):c1049/58
_1h5vmd(3.2):c1087/55 _1h5wp2(3.2):c1127/185 _1h60hm(3.2):c1316/31
_1h5szp(3.2):c1409/123 _1h61a7(3.2):c788/1 _1h61a9(3.2):c1 _1h61ab(3.2):c1
_1h61a8(3.2):c1
CMS 0 [Fri Nov 15 01:37:17 IST 2013; main]: launch new thread [Lucene Merge
Thread #0]
CMS 0 [Fri Nov 15 01:37:17 IST 2013; main]: set priority of merge thread Lucene
Merge Thread #0 to 6
CMS 0 [Fri Nov 15 01:37:17 IST 2013; Lucene Merge Thread #0]: merge thread:
start
CMS 0 [Fri Nov 15 01:37:17 IST 2013; main]: no more merges pending; now return
BD 0 [Fri Nov 15 01:37:17 IST 2013; Lucene Merge Thread #0]: applyDeletes: no
deletes; skipping
BD 0 [Fri Nov 15 01:37:17 IST 2013; Lucene Merge Thread #0]: prune
sis=org.apache.lucene.index.SegmentInfos@630045eb minGen=0 packetCount=0
IW 0 [Fri Nov 15 01:37:17 IST 2013; Lucene Merge Thread #0]: merge seg=_1h61ac
_1h41d0(3.2):C6469899/21414 _1h5bwk(3.2):c38973/2782 _1h5pvt(3.2):c11112/1072
_1h5yzp(3.2):c1121/38 _1h5um4(3.2):c1206/254 _1h5xx4(3.2):c1049/58
_1h5vmd(3.2):c1087/55 _1h5wp2(3.2):c1127/185 _1h60hm(3.2):c1316/31
_1h5szp(3.2):c1409/123 _1h61a7(3.2):c788/1 _1h61a9(3.2):c1 _1h61ab(3.2):c1
_1h61a8(3.2):c1
IW 0 [Fri Nov 15 01:37:17 IST 2013; Lucene Merge Thread #0]: now merge
merge=_1h41d0(3.2):C6469899/21414 _1h5bwk(3.2):c38973/2782
_1h5pvt(3.2):c11112/1072 _1h5yzp(3.2):c1121/38 _1h5um4(3.2):c1206/254
_1h5xx4(3.2):c1049/58 _1h5vmd(3.2):c1087/55 _1h5wp2(3.2):c1127/185
_1h60hm(3.2):c1316/31 _1h5szp(3.2):c1409/123 _1h61a7(3.2):c788/1
_1h61a9(3.2):c1 _1h61ab(3.2):c1 _1h61a8(3.2):c1
index=_1h41d0(3.2):C6469899/21414 _1h5bwk(3.2):c38973/2782
_1h5pvt(3.2):c11112/1072 _1h5szp(3.2):c1409/123 _1h5um4(3.2):c1206/254
_1h5vmd(3.2):c1087/55 _1h5wp2(3.2):c1127/185 _1h5xx4(3.2):c1049/58
_1h5yzp(3.2):c1121/38 _1h60hm(3.2):c1316/31 _1h61a7(3.2):c788/1 _1h61a8(3.2):c1
_1h61a9(3.2):c1 _1h61ab(3.2):c1
IW 0 [Fri Nov 15 01:37:17 IST 2013; Lucene Merge Thread #0]: merging
_1h41d0(3.2):C6469899/21414 _1h5bwk(3.2):c38973/2782 _1h5pvt(3.2):c11112/1072
_1h5yzp(3.2):c1121/38 _1h5um4(3.2):c1206/254 _1h5xx4(3.2):c1049/58
_1h5vmd(3.2):c1087/55 _1h5wp2(3.2):c1127/185 _1h60hm(3.2):c1316/31
_1h5szp(3.2):c1409/123 _1h61a7(3.2):c788/1 _1h61a9(3.2):c1 _1h61ab(3.2):c1
_1h61a8(3.2):c1
IW 0 [Fri Nov 15 01:37:18 IST 2013; Lucene Merge Thread #0]:
seg=_1h41d0(3.2):C6469899/21414 delCount=21414
IW 0 [Fri Nov 15 01:37:19 IST 2013; Lucene Merge Thread #0]:
seg=_1h5bwk(3.2):c38973/2782 delCount=2782
IW 0 [Fri Nov 15 01:37:19 IST 2013; Lucene Merge Thread #0]:
seg=_1h5pvt(3.2):c11112/1072 delCount=1072
IW 0 [Fri Nov 15 01:37:19 IST 2013; Lucene Merge Thread #0]:
seg=_1h5yzp(3.2):c1121/38 delCount=38
IW 0 [Fri Nov 15 01:37:19 IST 2013; Lucene Merge Thread #0]:
seg=_1h5um4(3.2):c1206/254 delCount=254
IW 0 [Fri Nov 15 01:37:19 IST 2013; Lucene Merge Thread #0]:
seg=_1h5xx4(3.2):c1049/58 delCount=58
IW 0 [Fri Nov 15 01:37:19 IST 2013; Lucene Merge Thread #0]:
seg=_1h5vmd(3.2):c1087/55 delCount=55
IW 0 [Fri Nov 15 01:37:19 IST 2013; Lucene Merge Thread #0]:
seg=_1h5wp2(3.2):c1127/185 delCount=185
IW 0 [Fri Nov 15 01:37:19 IST 2013; Lucene Merge Thread #0]:
seg=_1h60hm(3.2):c1316/31 delCount=31
IW 0 [Fri Nov 15 01:37:19 IST 2013; Lucene Merge Thread #0]:
seg=_1h5szp(3.2):c1409/123 delCount=123
IW 0 [Fri Nov 15 01:37:19 IST 2013; Lucene Merge Thread #0]:
seg=_1h61a7(3.2):c788/1 delCount=1
IW 0 [Fri Nov 15 01:37:19 IST 2013; Lucene Merge Thread #0]:
seg=_1h61a9(3.2):c1 no deletes
IW 0 [Fri Nov 15 01:37:19 IST 2013; Lucene Merge Thread #0]:
seg=_1h61ab(3.2):c1 no deletes
IW 0 [Fri Nov 15 01:37:19 IST 2013; Lucene Merge Thread #0]:
seg=_1h61a8(3.2):c1 no deletes
SM 0 [Fri Nov 15 01:37:19 IST 2013; Lucene Merge Thread #0]: merge store
matchedCount=14 vs 14
SM 0 [Fri Nov 15 01:38:40 IST 2013; Lucene Merge Thread #0]: 80767 msec to
merge stored fields [6503077 docs]
SM 0 [Fri Nov 15 02:09:58 IST 2013; Lucene Merge Thread #0]: 1878817 msec to
merge postings [6503077 docs]
SM 0 [Fri Nov 15 02:09:58 IST 2013; Lucene Merge Thread #0]: 0 msec to merge
doc values [6503077 docs]
IFD 0 [Fri Nov 15 02:20:51 IST 2013; Lucene Merge Thread #0]: refresh
[prefix=_1h61ac]: removing newly created unreferenced file "_1h61ac.fdx"
IFD 0 [Fri Nov 15 02:20:51 IST 2013; Lucene Merge Thread #0]: delete
"_1h61ac.fdx"
IFD 0 [Fri Nov 15 02:20:51 IST 2013; Lucene Merge Thread #0]: refresh
[prefix=_1h61ac]: removing newly created unreferenced file
"_1h61ac_Lucene41_0.doc"
IFD 0 [Fri Nov 15 02:20:51 IST 2013; Lucene Merge Thread #0]: delete
"_1h61ac_Lucene41_0.doc"
IFD 0 [Fri Nov 15 02:20:51 IST 2013; Lucene Merge Thread #0]: refresh
[prefix=_1h61ac]: removing newly created unreferenced file
"_1h61ac_Lucene41_0.pos"
IFD 0 [Fri Nov 15 02:20:51 IST 2013; Lucene Merge Thread #0]: delete
"_1h61ac_Lucene41_0.pos"
IFD 0 [Fri Nov 15 02:20:52 IST 2013; Lucene Merge Thread #0]: refresh
[prefix=_1h61ac]: removing newly created unreferenced file "_1h61ac.nvm"
IFD 0 [Fri Nov 15 02:20:52 IST 2013; Lucene Merge Thread #0]: delete
"_1h61ac.nvm"
IFD 0 [Fri Nov 15 02:20:52 IST 2013; Lucene Merge Thread #0]: refresh
[prefix=_1h61ac]: removing newly created unreferenced file
"_1h61ac_Lucene41_0.tip"
IFD 0 [Fri Nov 15 02:20:52 IST 2013; Lucene Merge Thread #0]: delete
"_1h61ac_Lucene41_0.tip"
IFD 0 [Fri Nov 15 02:20:52 IST 2013; Lucene Merge Thread #0]: refresh
[prefix=_1h61ac]: removing newly created unreferenced file "_1h61ac.fdt"
IFD 0 [Fri Nov 15 02:20:52 IST 2013; Lucene Merge Thread #0]: delete
"_1h61ac.fdt"
IFD 0 [Fri Nov 15 02:20:52 IST 2013; Lucene Merge Thread #0]: refresh
[prefix=_1h61ac]: removing newly created unreferenced file "_1h61ac.nvd"
IFD 0 [Fri Nov 15 02:20:52 IST 2013; Lucene Merge Thread #0]: delete
"_1h61ac.nvd"
IFD 0 [Fri Nov 15 02:20:52 IST 2013; Lucene Merge Thread #0]: refresh
[prefix=_1h61ac]: removing newly created unreferenced file
"_1h61ac_Lucene41_0.tim"
IFD 0 [Fri Nov 15 02:20:52 IST 2013; Lucene Merge Thread #0]: delete
"_1h61ac_Lucene41_0.tim"
IW 0 [Fri Nov 15 02:20:52 IST 2013; Lucene Merge Thread #0]:
handleMergeException: merge=_1h41d0(3.2):C6469899/21414
_1h5bwk(3.2):c38973/2782 _1h5pvt(3.2):c11112/1072 _1h5yzp(3.2):c1121/38
_1h5um4(3.2):c1206/254 _1h5xx4(3.2):c1049/58 _1h5vmd(3.2):c1087/55
_1h5wp2(3.2):c1127/185 _1h60hm(3.2):c1316/31 _1h5szp(3.2):c1409/123
_1h61a7(3.2):c788/1 _1h61a9(3.2):c1 _1h61ab(3.2):c1 _1h61a8(3.2):c1
exc=java.lang.OutOfMemoryError: Java heap space
IW 0 [Fri Nov 15 02:20:52 IST 2013; Lucene Merge Thread #0]: hit exception
during merge
IW 0 [Fri Nov 15 02:20:52 IST 2013; Lucene Merge Thread #0]: hit
OutOfMemoryError inside merge
IW 0 [Fri Nov 15 02:20:52 IST 2013; main]: rollback
IW 0 [Fri Nov 15 02:20:52 IST 2013; main]: all running merges have aborted
IW 0 [Fri Nov 15 02:20:52 IST 2013; main]: rollback: done finish merges
Exception in thread "Lucene Merge Thread #0"
org.apache.lucene.index.MergePolicy$MergeException: java.lang.OutOfMemoryError:
Java heap space
at
org.apache.lucene.index.ConcurrentMergeScheduler.handleMergeException(ConcurrentMergeScheduler.java:545)
at
org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:518)
Caused by: java.lang.OutOfMemoryError: Java heap space
at
org.apache.lucene.codecs.lucene3x.Lucene3xNormsProducer$NormsDocValues.getInstance(Lucene3xNormsProducer.java:172)
at
org.apache.lucene.codecs.lucene3x.Lucene3xNormsProducer.getNumeric(Lucene3xNormsProducer.java:198)
at
org.apache.lucene.index.SegmentCoreReaders.getNormValues(SegmentCoreReaders.java:301)
at
org.apache.lucene.index.SegmentReader.getNormValues(SegmentReader.java:253)
at
org.apache.lucene.index.SegmentMerger.mergeNorms(SegmentMerger.java:215)
at org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:119)
at
org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:3772)
at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3376)
at
org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:405)
at
org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:482)
DW 0 [Fri Nov 15 02:20:53 IST 2013; main]: abort
DW 0 [Fri Nov 15 02:20:53 IST 2013; main]: done abort; abortedFiles=null
success=true
IW 0 [Fri Nov 15 02:20:53 IST 2013; main]: rollback:
infos=_1h41d0(3.2):C6469899/21414 _1h5bwk(3.2):c38973/2782
_1h5pvt(3.2):c11112/1072 _1h5szp(3.2):c1409/123 _1h5um4(3.2):c1206/254
_1h5vmd(3.2):c1087/55 _1h5wp2(3.2):c1127/185 _1h5xx4(3.2):c1049/58
_1h5yzp(3.2):c1121/38 _1h60hm(3.2):c1316/31 _1h61a7(3.2):c788/1 _1h61a8(3.2):c1
_1h61a9(3.2):c1 _1h61ab(3.2):c1
IFD 0 [Fri Nov 15 02:20:53 IST 2013; main]: now checkpoint
"_1h41d0(3.2):C6469899/21414 _1h5bwk(3.2):c38973/2782 _1h5pvt(3.2):c11112/1072
_1h5szp(3.2):c1409/123 _1h5um4(3.2):c1206/254 _1h5vmd(3.2):c1087/55
_1h5wp2(3.2):c1127/185 _1h5xx4(3.2):c1049/58 _1h5yzp(3.2):c1121/38
_1h60hm(3.2):c1316/31 _1h61a7(3.2):c788/1 _1h61a8(3.2):c1 _1h61a9(3.2):c1
_1h61ab(3.2):c1" [14 segments ; isCommit = false]
IFD 0 [Fri Nov 15 02:20:53 IST 2013; main]: 0 msec to checkpoint
IW 0 [Fri Nov 15 02:20:53 IST 2013; main]: now flush at close
waitForMerges=false
DW 0 [Fri Nov 15 02:20:53 IST 2013; main]: abort
DW 0 [Fri Nov 15 02:20:53 IST 2013; main]: done abort; abortedFiles=null
success=true
IW 0 [Fri Nov 15 02:20:53 IST 2013; main]: all running merges have aborted
IW 0 [Fri Nov 15 02:20:53 IST 2013; main]: now call final commit()
IW 0 [Fri Nov 15 02:20:53 IST 2013; main]: at close:
_1h41d0(3.2):C6469899/21414 _1h5bwk(3.2):c38973/2782 _1h5pvt(3.2):c11112/1072
_1h5szp(3.2):c1409/123 _1h5um4(3.2):c1206/254 _1h5vmd(3.2):c1087/55
_1h5wp2(3.2):c1127/185 _1h5xx4(3.2):c1049/58 _1h5yzp(3.2):c1121/38
_1h60hm(3.2):c1316/31 _1h61a7(3.2):c788/1 _1h61a8(3.2):c1 _1h61a9(3.2):c1
_1h61ab(3.2):c1
Exception in thread "main" java.io.IOException: background merge hit exception:
_1h41d0(3.2):C6469899/21414 _1h5bwk(3.2):c38973/2782 _1h5pvt(3.2):c11112/1072
_1h5yzp(3.2):c1121/38 _1h5um4(3.2):c1206/254 _1h5xx4(3.2):c1049/58
_1h5vmd(3.2):c1087/55 _1h5wp2(3.2):c1127/185 _1h60hm(3.2):c1316/31
_1h5szp(3.2):c1409/123 _1h61a7(3.2):c788/1 _1h61a9(3.2):c1 _1h61ab(3.2):c1
_1h61a8(3.2):c1 into _1h61ac [maxNumSegments=1]
at org.apache.lucene.index.IndexWriter.forceMerge(IndexWriter.java:1714)
at org.apache.lucene.index.IndexWriter.forceMerge(IndexWriter.java:1650)
at org.apache.lucene.index.IndexUpgrader.upgrade(IndexUpgrader.java:161)
at org.apache.lucene.index.IndexUpgrader.main(IndexUpgrader.java:109)
Caused by: java.lang.OutOfMemoryError: Java heap space
at
org.apache.lucene.codecs.lucene3x.Lucene3xNormsProducer$NormsDocValues.getInstance(Lucene3xNormsProducer.java:172)
at
org.apache.lucene.codecs.lucene3x.Lucene3xNormsProducer.getNumeric(Lucene3xNormsProducer.java:198)
at
org.apache.lucene.index.SegmentCoreReaders.getNormValues(SegmentCoreReaders.java:301)
at
org.apache.lucene.index.SegmentReader.getNormValues(SegmentReader.java:253)
at
org.apache.lucene.index.SegmentMerger.mergeNorms(SegmentMerger.java:215)
at org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:119)
at
org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:3772)
at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3376)
at
org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:405)
at
org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:482)
real 43m38.155s
user 36m43.674s
sys 2m1.682s
...
</snip2>
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org
For additional commands, e-mail: java-user-h...@lucene.apache.org
