[
https://issues.apache.org/jira/browse/SOLR-7928?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14981388#comment-14981388
]
Mike Drob commented on SOLR-7928:
---------------------------------
bq. You just need to read these publicly right? Perhaps just write public
accessors?
Done.
bq. Testing of the HdfsCheckIndex looks pretty minimal...can we reuse
TestCheckIndex in some way? I'm thinking like changing each test in there to
just take a directory that you pass in. In lucene we use newDirectory, in your
test we use an HdfsDirectory. Thoughts?
So... this is a good idea in theory, but in practice it gets really difficult
to do. TestCheckIndex isn't visible from the Solr test classes unless we start
publishing Lucene test artifacts, which I don't think we want to do. I think we
can get away with minimal testing here because we aren't changing any of the
functionality, and that's all covered in the Lucene test suite. For our
purposes, I think it is enough to establish that if you have an HDFS cluster,
you can point this tool at it, and it will run.
bq. Any plans to write a MapReduce Tool to do this?
Sure, after this gets committed I'll open up a new JIRA and we can discuss
there.
> Improve CheckIndex to work against HdfsDirectory
> ------------------------------------------------
>
> Key: SOLR-7928
> URL: https://issues.apache.org/jira/browse/SOLR-7928
> Project: Solr
> Issue Type: New Feature
> Components: hdfs
> Reporter: Mike Drob
> Assignee: Gregory Chanan
> Fix For: 5.4, Trunk
>
> Attachments: SOLR-7928.patch, SOLR-7928.patch
>
>
> CheckIndex is very useful for testing an index for corruption. However, it
> can only work with an index on an FSDirectory, meaning that if you need to
> check an Hdfs Index, then you have to download it to local disk (which can be
> very large).
> We should have a way to natively check index on hdfs for corruption.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
