Noble Paul created SOLR-12993:
---------------------------------
Summary: Split the state.json into 2. a small frequently modified
data + a large unmodified data
Key: SOLR-12993
URL: https://issues.apache.org/jira/browse/SOLR-12993
Project: Solr
Issue Type: Improvement
Security Level: Public (Default Security Level. Issues are Public)
Environment: This a just a proposal particularly useful for large
cluster.
Reporter: Noble Paul
Every time a small atet change occurs for a collection replica the entire
following file needs to be updated + read * n times (where n = no of replicas
for this collection ). The proposal is to split the main file into 2.
{code:json}
{"gettingstarted":{
"pullReplicas":"0",
"replicationFactor":"2",
"router":{"name":"compositeId"},
"maxShardsPerNode":"-1",
"autoAddReplicas":"false",
"nrtReplicas":"2",
"tlogReplicas":"0",
"shards":{
"shard1":{
"range":"80000000-ffffffff",
"replicas":{
"core_node3":{
"core":"gettingstarted_shard1_replica_n1",
"base_url":"http://10.0.0.80:8983/solr";,
"node_name":"10.0.0.80:8983_solr",
"state":"active",
"type":"NRT",
"force_set_state":"false",
"leader":"true"},
"core_node5":{
"core":"gettingstarted_shard1_replica_n2",
"base_url":"http://10.0.0.80:7574/solr";,
"node_name":"10.0.0.80:7574_solr",
"type":"NRT",
"force_set_state":"false"}}},
"shard2":{
"range":"0-7fffffff",
"state":"active",
"replicas":{
"core_node7":{
"core":"gettingstarted_shard2_replica_n4",
"base_url":"http://10.0.0.80:7574/solr";,
"node_name":"10.0.0.80:7574_solr",
"type":"NRT",
"force_set_state":"false"},
"core_node8":{
"core":"gettingstarted_shard2_replica_n6",
"base_url":"http://10.0.0.80:8983/solr";,
"node_name":"10.0.0.80:8983_solr",
"type":"NRT",
"force_set_state":"false",
"leader":"true"}}}}}}
{code}
another file {{status.json}} which is frequently updated and small.
{code:json}
{
"shard1": {
"s": 1,
"core_node3": {"s": 1},
"core_node5": {"s": 1}
},
"shard2": {
"s": 1,
"core_node7": {"s": 1},
"core_node8": {"s": 1}}
}
{code}
Here the size of the file is roughly one tenth of the other file. This leads to
a dramatic reduction in the amount of data written/read to/from ZK.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
