Hendrik Haddorp created SOLR-11707:
--------------------------------------
Summary: allow to configure the HDFS block size
Key: SOLR-11707
URL: https://issues.apache.org/jira/browse/SOLR-11707
Project: Solr
Issue Type: Improvement
Security Level: Public (Default Security Level. Issues are Public)
Components: hdfs
Reporter: Hendrik Haddorp
Priority: Minor
Currently index files are created in HDFS with the block size that is defined
on the namenode. For that the HdfsFileWriter reads out the config from the
server and then specifies the size (and replication factor) in the
FileSystem.create call.
For the write.lock files things work slightly different. These are being
created by the HdfsLockFactory without specifying a block size (or replication
factor). This results in a default being picked by the HDFS client, which is
128MB.
So currently files are being created with different block sizes if the namenode
is configured to something else then 128MB. It would be good if Solr would
allow to configure the block size to be used. This is especially useful if the
Solr admin is not the HDFS admin and if you have different applications using
HDFS that have different requirements for their block size.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
