[
https://issues.apache.org/jira/browse/SOLR-9088?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Varun Thacker updated SOLR-9088:
--------------------------------
Attachment: SOLR-9088.patch
Sample log excerpt from a failure
{code}
[junit4] 2> 1995579 INFO
(zkCallback-22745-thread-2-processing-n:127.0.0.1:39653_solr)
[n:127.0.0.1:39653_solr ] o.a.s.s.ZkIndexSchemaReader A schema change:
WatchedEvent state:SyncConnected type:NodeDataChanged
path:/configs/conf1/managed-schema, has occurred - updating schema from
ZooKeeper ...
[junit4] 2> 1995580 INFO
(zkCallback-22746-thread-3-processing-n:127.0.0.1:38103_solr)
[n:127.0.0.1:38103_solr ] o.a.s.s.ZkIndexSchemaReader A schema change:
WatchedEvent state:SyncConnected type:NodeDataChanged
path:/configs/conf1/managed-schema, has occurred - updating schema from
ZooKeeper ...
[junit4] 2> 1995580 INFO (qtp694202271-87996) [n:127.0.0.1:38103_solr
c:testschemaapi s:shard1 r:core_node1 x:testschemaapi_shard1_replica1]
o.a.s.s.ManagedIndexSchema Waiting up to 599 secs for 1 replicas to apply
schema update version 1 for collection testschemaapi
[junit4] 2> 1995580 INFO
(zkCallback-22745-thread-2-processing-n:127.0.0.1:39653_solr)
[n:127.0.0.1:39653_solr ] o.a.s.s.ZkIndexSchemaReader Retrieved schema
version 2 from ZooKeeper
[junit4] 2> 1995582 INFO (qtp178914546-87997) [n:127.0.0.1:39653_solr
c:testschemaapi s:shard1 r:core_node2 x:testschemaapi_shard1_replica2]
o.a.s.c.S.Request [testschemaapi_shard1_replica2] webapp=/solr
path=/schema/zkversion params={refreshIfBelowVersion=1&version=2&wt=javabin}
status=0 QTime=0
[junit4] 2> 1995583 INFO (qtp694202271-87996) [n:127.0.0.1:38103_solr
c:testschemaapi s:shard1 r:core_node1 x:testschemaapi_shard1_replica1]
o.a.s.s.ManagedIndexSchema Took 2.0ms for 1 replicas to apply schema update
version 1 for collection testschemaapi
[junit4] 2> 1995583 INFO (qtp694202271-87996) [n:127.0.0.1:38103_solr
c:testschemaapi s:shard1 r:core_node1 x:testschemaapi_shard1_replica1]
o.a.s.c.S.Request [testschemaapi_shard1_replica1] webapp=/solr path=/schema
params={version=2&wt=javabin} status=0 QTime=16
[junit4] 2> 1995583 INFO
(zkCallback-22745-thread-2-processing-n:127.0.0.1:39653_solr)
[n:127.0.0.1:39653_solr ] o.a.s.s.IndexSchema
[testschemaapi_shard1_replica2] Schema name=minimal
[junit4] 2> 1995622 INFO
(TEST-TestManagedSchemaAPI.test-seed#[79C11866C0E16F74]) [ ]
o.a.s.s.TestManagedSchemaAPI added new field=myNewField1
[junit4] 2> 1995622 INFO
(zkCallback-22746-thread-3-processing-n:127.0.0.1:38103_solr)
[n:127.0.0.1:38103_solr ] o.a.s.s.ZkIndexSchemaReader Retrieved schema
version 2 from ZooKeeper
[junit4] 2> 1995622 INFO (Thread-5662) [n:127.0.0.1:38103_solr ]
o.a.s.c.ZkController Running listeners for /configs/conf1
[junit4] 2> 1995622 INFO (Thread-5662) [n:127.0.0.1:38103_solr ]
o.a.s.c.SolrCore config update listener called for core
testschemaapi_shard1_replica1
[junit4] 2> 1995623 INFO (Thread-5662) [n:127.0.0.1:38103_solr ]
o.a.s.c.SolrConfig current version of requestparams : -1
[junit4] 2> 1995624 INFO (Thread-5662) [n:127.0.0.1:38103_solr ]
o.a.s.c.SolrCore /configs/conf1/managed-schema is stale will need an update
from 1 to 2
[junit4] 2> 1995624 INFO (Thread-5662) [n:127.0.0.1:38103_solr ]
o.a.s.c.SolrCore core reload testschemaapi_shard1_replica1
[junit4] 2> 1995624 INFO (Thread-5662) [n:127.0.0.1:38103_solr ]
o.a.s.c.ZkController Check for collection zkNode:testschemaapi
[junit4] 2> 1995624 INFO (Thread-5662) [n:127.0.0.1:38103_solr ]
o.a.s.c.ZkController Collection zkNode exists
[junit4] 2> 1995624 INFO (Thread-5662) [n:127.0.0.1:38103_solr ]
o.a.s.c.c.ZkStateReader Load collection config from:
[/collections/testschemaapi]
[junit4] 2> 1995624 INFO (Thread-5663) [n:127.0.0.1:39653_solr ]
o.a.s.c.ZkController Running listeners for /configs/conf1
[junit4] 2> 1995624 INFO (Thread-5663) [n:127.0.0.1:39653_solr ]
o.a.s.c.SolrCore config update listener called for core
testschemaapi_shard1_replica2
[junit4] 2> 1995625 INFO (Thread-5662) [n:127.0.0.1:38103_solr ]
o.a.s.c.c.ZkStateReader path=[/collections/testschemaapi] [configName]=[conf1]
specified config exists in ZooKeeper
[junit4] 2> 1995625 INFO (Thread-5662) [n:127.0.0.1:38103_solr ]
o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory:
'/home/jenkins/workspace/Lucene-Solr-5.5-Linux/solr/build/solr-core/test/J2/temp/solr.schema.TestManagedSchemaAPI_79C11866C0E16F74-001/tempDir-001/node1/testschemaapi_shard1_replica1'
[junit4] 2> 1995626 INFO (Thread-5662) [n:127.0.0.1:38103_solr ]
o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 1995626 INFO (Thread-5662) [n:127.0.0.1:38103_solr ]
o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find
system property or JNDI)
[junit4] 2> 1995626 INFO (qtp178914546-87995) [n:127.0.0.1:39653_solr
c:testschemaapi s:shard1 r:core_node2 x:testschemaapi_shard1_replica2]
o.a.s.u.p.LogUpdateProcessorFactory [testschemaapi_shard1_replica2]
webapp=/solr path=/update params={version=2&wt=javabin}{} 0 0
[junit4] 2> 1995626 ERROR (qtp178914546-87995) [n:127.0.0.1:39653_solr
c:testschemaapi s:shard1 r:core_node2 x:testschemaapi_shard1_replica2]
o.a.s.h.RequestHandlerBase org.apache.solr.common.SolrException: ERROR: [doc=2]
unknown field 'myNewField1'
[junit4] 2> at
org.apache.solr.update.DocumentBuilder.toDocument(DocumentBuilder.java:182)
[junit4] 2> at
org.apache.solr.update.AddUpdateCommand.getLuceneDocument(AddUpdateCommand.java:82)
...
{code}
Here's my analysis on the current implementation and my attempted fix
- The collection has 1 shard and two replicas ( replica1 on JVM 1 and replica2
on JVM 2)
- A new add field request comes in to JVM1 for the collection.
- {{SchemaManager.java}} adds the field and persists the schema in ZK.
- Since ZK has a new schema and both replicas are watching for changes , a core
reload is triggered.
- {{SchemaManager.java}} had ensured that on the local core ( replica1 ) has
updated the schema before the reload by doing
{{core.setLatestSchema(managedIndexSchema);}}
- If however a new document with the new field comes in against replica2 , it
still has the old schema ( the reload will fetch the new schema ) . Since the
reload hasn't completed we get the unknown field error
So all this patch does is makes sure to set the latest schema on a core before
the reload starts.
[~noble.paul] Do you think this solution is good enough or are there still
races that we need to take care of?
> solr.schema.TestManagedSchemaAPI.test failures ([doc=2] unknown field
> 'myNewField1')
> ------------------------------------------------------------------------------------
>
> Key: SOLR-9088
> URL: https://issues.apache.org/jira/browse/SOLR-9088
> Project: Solr
> Issue Type: Test
> Reporter: Christine Poerschke
> Priority: Minor
> Attachments: SOLR-9088.patch
>
>
> e.g.
> http://jenkins.thetaphi.de/job/Lucene-Solr-master-MacOSX/3256/
> http://jenkins.thetaphi.de/job/Lucene-Solr-6.x-Linux/588/
> http://jenkins.thetaphi.de/job/Lucene-Solr-5.5-Linux/266/
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
