Hoss Man created SOLR-6367:
------------------------------
Summary: empty tolg on HDFS when hard crash - no docs to replay on
recovery
Key: SOLR-6367
URL: https://issues.apache.org/jira/browse/SOLR-6367
Project: Solr
Issue Type: Bug
Reporter: Hoss Man
Filing this bug based on an email to solr-user@lucene from Tom Chen (Fri, 18
Jul 2014)...
{panel}
Reproduce steps:
1) Setup Solr to run on HDFS like this:
{noformat}
java -Dsolr.directoryFactory=HdfsDirectoryFactory
-Dsolr.lock.type=hdfs
-Dsolr.hdfs.home=hdfs://host:port/path
{noformat}
For the purpose of this testing, turn off the default auto commit in
solrconfig.xml, i.e. comment out autoCommit like this:
{code}
<!--
<autoCommit>
<maxTime>${solr.autoCommit.maxTime:15000}</maxTime>
<openSearcher>false</openSearcher>
</autoCommit>
-->
{code}
2) Add a document without commit:
{{curl "http://localhost:8983/solr/collection1/update?commit=false"; -H
"Content-type:text/xml; charset=utf-8" --data-binary "@solr.xml"}}
3) Solr generate empty tlog file (0 file size, the last one ends with 6):
{noformat}
[hadoop@hdtest042 exampledocs]$ hadoop fs -ls
/path/collection1/core_node1/data/tlog
Found 5 items
-rw-r--r-- 1 hadoop hadoop 667 2014-07-18 08:47
/path/collection1/core_node1/data/tlog/tlog.0000000000000000001
-rw-r--r-- 1 hadoop hadoop 67 2014-07-18 08:47
/path/collection1/core_node1/data/tlog/tlog.0000000000000000003
-rw-r--r-- 1 hadoop hadoop 667 2014-07-18 08:47
/path/collection1/core_node1/data/tlog/tlog.0000000000000000004
-rw-r--r-- 1 hadoop hadoop 0 2014-07-18 09:02
/path/collection1/core_node1/data/tlog/tlog.0000000000000000005
-rw-r--r-- 1 hadoop hadoop 0 2014-07-18 09:02
/path/collection1/core_node1/data/tlog/tlog.0000000000000000006
{noformat}
4) Simulate Solr crash by killing the process with -9 option.
5) restart the Solr process. Observation is that uncommitted document are
not replayed, files in tlog directory are cleaned up. Hence uncommitted
document(s) is lost.
Am I missing anything or this is a bug?
BTW, additional observations:
a) If in step 4) Solr is stopped gracefully (i.e. without -9 option),
non-empty tlog file is geneated and after re-starting Solr, uncommitted
document is replayed as expected.
b) If Solr doesn't run on HDFS (i.e. on local file system), this issue is
not observed either.
{panel}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
