[
https://issues.apache.org/jira/browse/IGNITE-29136?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dmitry Werner updated IGNITE-29136:
-----------------------------------
Labels: MakeTeamcityGreenAgain (was: )
> Fix flaky ViewsIntegrationTest.testSchemaCleaning
> -------------------------------------------------
>
> Key: IGNITE-29136
> URL: https://issues.apache.org/jira/browse/IGNITE-29136
> Project: Ignite
> Issue Type: Bug
> Reporter: Dmitry Werner
> Assignee: Dmitry Werner
> Priority: Minor
> Labels: MakeTeamcityGreenAgain
> Fix For: 2.19
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> *Summary:*
> `ViewsIntegrationTest.testSchemaCleaning` fails intermittently: SQL view is
> still visible in `sys.views` right after the schema cache is destroyed.
> https://ci2.ignite.apache.org/test/-4439343006317346895?currentProjectId=IgniteTests24Java8&branch=%3Cdefault%3E
> Stacktrace:
> java.lang.AssertionError: Unexpected views count
> [grid=integration.ViewsIntegrationTest2, schemaName=CACHE1, viewName=MY_VIEW]
> at
> org.apache.ignite.internal.processors.query.calcite.integration.ViewsIntegrationTest.assertViewsCount(ViewsIntegrationTest.java:411)
> at
> org.apache.ignite.internal.processors.query.calcite.integration.ViewsIntegrationTest.testSchemaCleaning(ViewsIntegrationTest.java:189)
> Caused by: java.lang.AssertionError: expected:<0> but was:<1>
> The failing parameterization is `[2]` = `IndexingQueryEngineConfiguration`
> (H2 engine); the Calcite parameterization `[1]` passed. The whole test
> completed in 0.451 s (node uptimes 0.15–0.40 s) on a heavily loaded agent.
> *Root cause*
> `testSchemaCleaning` asserts that a view created in the cache's SQL schema
> disappears from `sys.views` immediately after the cache is destroyed
> (ViewsIntegrationTest.java:186-189). However, view cleanup is applied on
> nodes **asynchronously** relative to `destroyCache` completion:
> 1. `client.destroyCache("CACHE1")` finishes the synchronous destroy exchange;
> on every node `SchemaManager.onCacheStopped` drops the SQL schema when its
> usage count reaches zero (core: `SchemaManager.java:420-466`).
> 2. Removal of SQL views tied to the dropped schema is *not* part of that
> synchronous path: `SchemaManager.java:460` calls
> `SchemaSqlViewManager.clearSchemaViews(schemaName)`
> (`SchemaSqlViewManager.java:120-137`), which runs **only on the coordinator**
> and deletes the `ignite.sql.view.<schema>.*` metastorage keys via
> `metastorage.removeAsync(k)` — **the returned future is ignored**
> (fire-and-forget).
> 3. The actual local view removal on every node happens later, via a
> metastorage notification round trip:
> `DistributedMetaStorageImpl.onGridDataReceived` →
> `SchemaSqlViewManager.processMetastorageUpdate`
> (`SchemaSqlViewManager.java:150-164`) → `SchemaManager.dropView` →
> `H2SchemaManager.onViewDropped`.
> 4. The test queries `sys.views` locally on all grids without waiting, so a
> node that has not yet processed the metastorage update still lists
> `CACHE1.MY_VIEW` → `expected:<0> but was:<1>`.
> The same family of races is visible in the log as benign
> `SchemaManager$CompoundSchemaChangeListener "Failed to notify listener (will
> ignore): Failed to create view: MY_VIEW — Table "MY_TABLE" not found"`
> warnings during node joins of sibling tests — metastorage view updates
> replaying ahead of the local SQL schema state.
> *Proposed fix*
> Make `assertViewsCount` wait for convergence instead of asserting
> immediately: poll all grids with `GridTestUtils.waitForCondition` (timeout 10
> s) until every grid reports the expected count, and fail with a descriptive
> message (grid, schema, view, expected count) on timeout. This covers both the
> drop-side race (`cnt == 0` after `destroyCache`) and the symmetric
> create-side race (`cnt == 1` after `CREATE VIEW`, which is propagated through
> the metastorage the same way). No product code changes; the semantic that
> `destroyCache` does not guarantee metastorage view cleanup propagation is
> left untouched.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
