[ 
https://issues.apache.org/jira/browse/SOLR-17360?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17863960#comment-17863960
 ] 

Chris M. Hostetter commented on SOLR-17360:
-------------------------------------------

[~ilan] - {{git bisect}} has identified your changes made in SOLR-17067 as the 
cause of these 100% reproducible (nightly) test failures – can you please take 
a look?
{noformat}
0e91b7c82ee4e97df8ed4bed575646de7981d2c1 is the first bad commit
commit 0e91b7c82ee4e97df8ed4bed575646de7981d2c1
Author: Ilan Ginzburg <i...@murblanc.org>
Date:   Tue Nov 7 09:51:06 2023 +0100    SOLR-17067 SolrCloudTestCase 
activeClusterShape() only counts active replicas of active shards 
(#2063):040000 040000 e5a47cd1f328415d2cf3d9262a2eea53d184dabe 
153adad36810944b168413decc2b7174adaf0156 M    solr
bisect run success
{noformat}
Full bisect log...
{noformat}
# bad: [e1c700c277a8c69f789e35ec03a8d955e0a930ef] 
solr/example/films/vectors/README.md tweaks (#2121)
# good: [959029c7e7e2d03dec054d9094918d78a7a35c1a] SOLR-17054: Remove unused 
and duplicate code in DistributedZkUpdateProcessor (#2038)
git bisect start 'e1c700c277a8c69f789e35ec03a8d955e0a930ef' 
'959029c7e7e2d03dec054d9094918d78a7a35c1a'
# bad: [b339430541744a02c26c98335bc1cef04415c415] Tidy fix for #2086
git bisect bad b339430541744a02c26c98335bc1cef04415c415
# bad: [e740123546c7f20c6728df42d493e11f93dc8c79] SOLR-16974: Global Circuit 
Breakers (#1919)
git bisect bad e740123546c7f20c6728df42d493e11f93dc8c79
# good: [e371e10cfdea68154e8d48b2acb133ef74fb4015] SOLR-17046: ref-guide 
updates now that SchemaCodecFactory is implicit default
git bisect good e371e10cfdea68154e8d48b2acb133ef74fb4015
# bad: [2062d057c003e4113b09b8e6aab49caa134b4a7c] SOLR-16907: Fail when parsing 
an invalid custom permission definition from security.json (#2040)
git bisect bad 2062d057c003e4113b09b8e6aab49caa134b4a7c
# bad: [0e91b7c82ee4e97df8ed4bed575646de7981d2c1] SOLR-17067 SolrCloudTestCase 
activeClusterShape() only counts active replicas of active shards (#2063)
git bisect bad 0e91b7c82ee4e97df8ed4bed575646de7981d2c1
# good: [0b59d371d85bd3d9da3c7cef5514dc286ef8c7a2] Capture build scans on 
ge.apache.org to benefit from deep build insights (#2042)
git bisect good 0b59d371d85bd3d9da3c7cef5514dc286ef8c7a2
# first bad commit: [0e91b7c82ee4e97df8ed4bed575646de7981d2c1] SOLR-17067 
SolrCloudTestCase activeClusterShape() only counts active replicas of active 
shards (#2063)
{noformat}
 

> ShardSplitTest 100% failures on main branch
> -------------------------------------------
>
>                 Key: SOLR-17360
>                 URL: https://issues.apache.org/jira/browse/SOLR-17360
>             Project: Solr
>          Issue Type: Bug
>      Security Level: Public(Default Security Level. Issues are Public) 
>            Reporter: Chris M. Hostetter
>            Priority: Major
>
> ShardSplitTest is a nightly only testclass that has 4 test methods which have 
> recently been failing 100% of the time in jenkins, regardless of seed...
>  * testSplitMixedReplicaTypes
>  * testSplitStaticIndexReplicationLink
>  * testSplitStaticIndexReplication
>  * testSplitMixedReplicaTypesLink
>  
> ...but AFAICT these failures are not new.  IIUC this test has been failing on 
> 100% of – the _MAIN_ branch – jenkins builds since ~ Oct/Nov {*}2023{*}!
> (I believe the historical graphs since then just look "sporadic" because it's 
> a nightly only test so run's very infrequently and was still (mostly) passing 
> on the 9x branch)
>  
> ----
>  
> It's trivial to see these failures locally, regardless of seed; or whether 
> you run a single test method, or multiple test methods.  
> Example...
> {noformat}
> git clean -fd && ./gradlew -p solr/core test -Ptests.nightly=true --tests 
> ShardSplitTest.testSplitMixedReplicaTypes{noformat}
>  
> The failures are all fairly non-sopecific...
> {noformat}
>    >     java.util.concurrent.TimeoutException: Timeout waiting to see state 
> for collection=testSplitMixedReplicaTypes_rewrite 
> :DocCollection(testSplitMixedReplicaTypes_rewrite//collections/testSplitMixedReplicaTypes_rewrite/state.json/18
>  )={
>    >       "pullReplicas":2,
>    >       "configName":"conf1",
> ...
>    >         at 
> __randomizedtesting.SeedInfo.seed([462B215EC450E717:FEE875FE388B3262]:0)
>    >         at 
> org.apache.solr.common.cloud.ZkStateReader.waitForState(ZkStateReader.java:1884)
>    >         at 
> org.apache.solr.cloud.api.collections.ShardSplitTest.doSplitMixedReplicaTypes(ShardSplitTest.java:442)
>    >         at 
> org.apache.solr.cloud.api.collections.ShardSplitTest.testSplitMixedReplicaTypes(ShardSplitTest.java:407){noformat}
>  
> ...with no errors in the logs indicating when/what might be going wrong.
>  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org
For additional commands, e-mail: issues-h...@solr.apache.org

Reply via email to