[jira] [Created] (IGNITE-20133) Compute hashes for integral/decimal columns in a stable way
Roman Puchkovskiy created IGNITE-20133: -- Summary: Compute hashes for integral/decimal columns in a stable way Key: IGNITE-20133 URL: https://issues.apache.org/jira/browse/IGNITE-20133 Project: Ignite Issue Type: Improvement Reporter: Roman Puchkovskiy Fix For: 3.0.0-beta2 The idea is to make hash computation for integral and decimal types satisfy the following property: if a column type is changed from an integral to a decimal type, the hashes for values that are already stored remain the same. This will allow us to permit chaning type (integral -> decimal and decimal -> longer decimal) of a column that is included in a HASH index. A hash that has this property is the following function: hash(val.toString(TRIM_TRAILING_ZEROS)). For instance, for 1 it will be hash("1"), for 1.000 it will also be hash("1"), but for 1.23 it will give hash("1.23"). -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (IGNITE-20134) Only allow changing type of indexed column when indexed values representation remains the same
Roman Puchkovskiy created IGNITE-20134: -- Summary: Only allow changing type of indexed column when indexed values representation remains the same Key: IGNITE-20134 URL: https://issues.apache.org/jira/browse/IGNITE-20134 Project: Ignite Issue Type: Improvement Reporter: Roman Puchkovskiy Fix For: 3.0.0-beta2 When an attempt to change type of a column that is included in an index is made, this should only be permitted if the representation of the column values in the index will remain unchanged (and, hence, index rebuild will not be needed). The following changes are acceptable: * integral->integral (as integral types are represented as varints) * integral->decimal (with enough precision) and float->double for SORTED indices where the ordering remains the same * integral->decimal and decimal->decimal (with enough precision) for HASH indices [requires IGNITE-20133] -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-19893) Sql. Add plan checks to BaseIndexDataTypeTest
[ https://issues.apache.org/jira/browse/IGNITE-19893?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Evgeny Stanilovsky updated IGNITE-19893: Ignite Flags: (was: Docs Required,Release Notes Required) > Sql. Add plan checks to BaseIndexDataTypeTest > - > > Key: IGNITE-19893 > URL: https://issues.apache.org/jira/browse/IGNITE-19893 > Project: Ignite > Issue Type: Improvement > Components: sql >Affects Versions: 3.0.0-beta1 >Reporter: Maksim Zhuravkov >Assignee: Maksim Zhuravkov >Priority: Minor > Labels: ignite-3 > > Add checks to BaseIndexDataTypeTest to ensure that index lookups/scans are > used where they are necessary. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-20134) Only allow changing type of indexed column when indexed values representation remains the same
[ https://issues.apache.org/jira/browse/IGNITE-20134?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Roman Puchkovskiy updated IGNITE-20134: --- Description: When an attempt to change type of a column that is included in an index is made, this should only be permitted if the representation of the column values in the index will remain unchanged (and, hence, index rebuild will not be needed). The following changes are acceptable: * integral->integral (as integral types are represented as varints) * integral->decimal (with enough precision) and float->double for SORTED indices where the ordering remains the same * integral->decimal and decimal->decimal (with enough precision) for HASH indices (requires IGNITE-20133) was: When an attempt to change type of a column that is included in an index is made, this should only be permitted if the representation of the column values in the index will remain unchanged (and, hence, index rebuild will not be needed). The following changes are acceptable: * integral->integral (as integral types are represented as varints) * integral->decimal (with enough precision) and float->double for SORTED indices where the ordering remains the same * integral->decimal and decimal->decimal (with enough precision) for HASH indices [requires IGNITE-20133] > Only allow changing type of indexed column when indexed values representation > remains the same > -- > > Key: IGNITE-20134 > URL: https://issues.apache.org/jira/browse/IGNITE-20134 > Project: Ignite > Issue Type: Improvement >Reporter: Roman Puchkovskiy >Priority: Major > Labels: ignite-3 > Fix For: 3.0.0-beta2 > > > When an attempt to change type of a column that is included in an index is > made, this should only be permitted if the representation of the column > values in the index will remain unchanged (and, hence, index rebuild will not > be needed). > The following changes are acceptable: > * integral->integral (as integral types are represented as varints) > * integral->decimal (with enough precision) and float->double for SORTED > indices where the ordering remains the same > * integral->decimal and decimal->decimal (with enough precision) for HASH > indices (requires IGNITE-20133) -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (IGNITE-19947) Sql. Fix CAST(n AS DECIMAL(precision, scale) behaviour for scale > precision.
[ https://issues.apache.org/jira/browse/IGNITE-19947?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17750153#comment-17750153 ] Evgeny Stanilovsky commented on IGNITE-19947: - let`s fill appropriate issue into Calcite after and fix it too. > Sql. Fix CAST(n AS DECIMAL(precision, scale) behaviour for scale > precision. > - > > Key: IGNITE-19947 > URL: https://issues.apache.org/jira/browse/IGNITE-19947 > Project: Ignite > Issue Type: Bug > Components: sql >Affects Versions: 3.0.0-beta1 >Reporter: Maksim Zhuravkov >Assignee: Evgeny Stanilovsky >Priority: Minor > Labels: ignite-3 > > CAST(n AS DECIMAL(p, s) always returns `numeric field overflow` error when s > > p which is not correct, in such case we should also check `n`. > Example: > {code:java} > SELECT CAST (? AS DECIMAL(1,4)) ? = 0.0001 > # PG returns 0.0001 > # Ignite 3 returns numeric field overflow > {code} > Query with a literal works because calcite performs constant folding and > removes a CAST call: > {code:java} > SELECT CAST (0.0001 AS DECIMAL(1,4)) > # Plan > IgniteValues(tuples=[[{ 0.0001 }]]), id = 8 > {code} -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-19506) Use data nodes from DistributionZoneManager with a causality token instead of BaselineManager#nodes
[ https://issues.apache.org/jira/browse/IGNITE-19506?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexander Lapin updated IGNITE-19506: - Ignite Flags: (was: Docs Required,Release Notes Required) > Use data nodes from DistributionZoneManager with a causality token instead of > BaselineManager#nodes > --- > > Key: IGNITE-19506 > URL: https://issues.apache.org/jira/browse/IGNITE-19506 > Project: Ignite > Issue Type: Bug >Reporter: Sergey Uttsel >Assignee: Sergey Uttsel >Priority: Major > Labels: ignite-3 > Fix For: 3.0.0-beta2 > > Attachments: Causality data nodes.docx > > Time Spent: 7h > Remaining Estimate: 0h > > h3. *Motivation* > Need to use data nodes from DistributionZoneManager instead of > BaselineManager#nodes in all places except of in-memory raft > (TableManager#calculateAssignments) > We need to get data nodes consistently so we need to use revision of > configuration events and a meta storage events as causality token. > Description of causality data nodes algorithm is attached. > h3. *Definition of Done* > Implement method DistributionZoneManager#dataNodes to obtaining data nodes > from zone manager with causality token. > Use this method instead of BaselineManager#nodes. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (IGNITE-19904) Assertion in defragmentation
[ https://issues.apache.org/jira/browse/IGNITE-19904?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vladimir Steshin reassigned IGNITE-19904: - Assignee: Vladimir Steshin > Assertion in defragmentation > > > Key: IGNITE-19904 > URL: https://issues.apache.org/jira/browse/IGNITE-19904 > Project: Ignite > Issue Type: Bug >Affects Versions: 2.12 >Reporter: Vladimir Steshin >Assignee: Vladimir Steshin >Priority: Major > Labels: ise > Attachments: default-config.xml, failure2.16_with_thread_dump.log, > failure_with_root_npe_cause.log, ignite.log, jvm.opts > > Time Spent: 0.5h > Remaining Estimate: 0h > > Defragmentaion fails with: > {code:java} > java.lang.AssertionError: Invalid state. Type is 0! pageId = 0001000d00024cbf > at > org.apache.ignite.internal.processors.cache.persistence.pagemem.PageMemoryImpl.copyPageForCheckpoint(PageMemoryImpl.java:1359) > ~[ignite-core-2.16.0-SNAPSHOT.jar:2.16.0-SNAPSHOT] > at > org.apache.ignite.internal.processors.cache.persistence.pagemem.PageMemoryImpl.checkpointWritePage(PageMemoryImpl.java:1277) > ~[ignite-core-2.16.0-SNAPSHOT.jar:2.16.0-SNAPSHOT] > at > org.apache.ignite.internal.processors.cache.persistence.checkpoint.CheckpointPagesWriter.writePages(CheckpointPagesWriter.java:208) > ~[ignite-core-2.16.0-SNAPSHOT.jar:2.16.0-SNAPSHOT] > at > org.apache.ignite.internal.processors.cache.persistence.checkpoint.CheckpointPagesWriter.run(CheckpointPagesWriter.java:150) > ~[ignite-core-2.16.0-SNAPSHOT.jar:2.16.0-SNAPSHOT] > {code} > Difficult to write a test. Can't reproduce on my computers :(. Flackly > appears on a server (4 core x 4 cpu) with 100G of the test cache data and > million+ pages to checkpoint during defragmentation. More often, this occurs > with pageSize 1024 (to produce more pages). > Regarding my diagnostic build, I suppose that a fresh, empty page is caught > in defragmentation. Here is a page dump with test-expented PAGE_OVERHEAD > (=64) and same error a bit before copyPageForCheckpoint(): > {code:java} > org.apache.ignite.IgniteException: Wrong page type in checkpointWritePage1. > Page: Data region = 'defragPartitionsDataRegion'. > FullPageId [pageId=281878703760205, effectivePageId=403727049549, > grpId=-1368047378]. > PageDump = page_id: 281878703760205, rel_id: 48603, cache_id: -1368047378, > pin: 0, lock: 65536, tmp_buf: 72057594037927935, test_val: 1. data_hex: > > at > org.apache.ignite.internal.processors.cache.persistence.pagemem.PageMemoryImpl.checkpointWritePage(PageMemoryImpl.java:1240) > ~[ignite-core-2.16.0-SNAPSHOT.jar:2.16.0-SNAPSHOT] > at > org.apache.ignite.internal.processors.cache.persistence.check
[jira] [Updated] (IGNITE-19904) Assertion in defragmentation
[ https://issues.apache.org/jira/browse/IGNITE-19904?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vladimir Steshin updated IGNITE-19904: -- Fix Version/s: 2.16 > Assertion in defragmentation > > > Key: IGNITE-19904 > URL: https://issues.apache.org/jira/browse/IGNITE-19904 > Project: Ignite > Issue Type: Bug >Affects Versions: 2.12 >Reporter: Vladimir Steshin >Assignee: Vladimir Steshin >Priority: Major > Labels: ise > Fix For: 2.16 > > Attachments: default-config.xml, failure2.16_with_thread_dump.log, > failure_with_root_npe_cause.log, ignite.log, jvm.opts > > Time Spent: 0.5h > Remaining Estimate: 0h > > Defragmentaion fails with: > {code:java} > java.lang.AssertionError: Invalid state. Type is 0! pageId = 0001000d00024cbf > at > org.apache.ignite.internal.processors.cache.persistence.pagemem.PageMemoryImpl.copyPageForCheckpoint(PageMemoryImpl.java:1359) > ~[ignite-core-2.16.0-SNAPSHOT.jar:2.16.0-SNAPSHOT] > at > org.apache.ignite.internal.processors.cache.persistence.pagemem.PageMemoryImpl.checkpointWritePage(PageMemoryImpl.java:1277) > ~[ignite-core-2.16.0-SNAPSHOT.jar:2.16.0-SNAPSHOT] > at > org.apache.ignite.internal.processors.cache.persistence.checkpoint.CheckpointPagesWriter.writePages(CheckpointPagesWriter.java:208) > ~[ignite-core-2.16.0-SNAPSHOT.jar:2.16.0-SNAPSHOT] > at > org.apache.ignite.internal.processors.cache.persistence.checkpoint.CheckpointPagesWriter.run(CheckpointPagesWriter.java:150) > ~[ignite-core-2.16.0-SNAPSHOT.jar:2.16.0-SNAPSHOT] > {code} > Difficult to write a test. Can't reproduce on my computers :(. Flackly > appears on a server (4 core x 4 cpu) with 100G of the test cache data and > million+ pages to checkpoint during defragmentation. More often, this occurs > with pageSize 1024 (to produce more pages). > Regarding my diagnostic build, I suppose that a fresh, empty page is caught > in defragmentation. Here is a page dump with test-expented PAGE_OVERHEAD > (=64) and same error a bit before copyPageForCheckpoint(): > {code:java} > org.apache.ignite.IgniteException: Wrong page type in checkpointWritePage1. > Page: Data region = 'defragPartitionsDataRegion'. > FullPageId [pageId=281878703760205, effectivePageId=403727049549, > grpId=-1368047378]. > PageDump = page_id: 281878703760205, rel_id: 48603, cache_id: -1368047378, > pin: 0, lock: 65536, tmp_buf: 72057594037927935, test_val: 1. data_hex: > > at > org.apache.ignite.internal.processors.cache.persistence.pagemem.PageMemoryImpl.checkpointWritePage(PageMemoryImpl.java:1240) > ~[ignite-core-2.16.0-SNAPSHOT.jar:2.16.0-SNAPSHOT] > at > org.apache.ignite.internal.processors.cache.p
[jira] [Assigned] (IGNITE-19922) Gradle checkstyle tasks are greedy
[ https://issues.apache.org/jira/browse/IGNITE-19922?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kirill Sizov reassigned IGNITE-19922: -- Assignee: Kirill Sizov > Gradle checkstyle tasks are greedy > -- > > Key: IGNITE-19922 > URL: https://issues.apache.org/jira/browse/IGNITE-19922 > Project: Ignite > Issue Type: New Feature >Reporter: Mikhail Pochatkin >Assignee: Kirill Sizov >Priority: Major > Labels: ignite-3 > Attachments: image-2023-07-06-11-18-40-515.png, screenshot-1.png > > > This is memory consumption during {{gradlew checkstyleMain}} execution - > goes from ~10 GB to 30. All CPU cores are also at 100%. This causes chrome > tabs to unload and overall stress on the system. > Also, RAM usage does not go down after this command unless I kill/stop Gradle > daemons > !image-2023-07-06-11-18-40-515.png! -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (IGNITE-19922) Gradle checkstyle tasks are greedy
[ https://issues.apache.org/jira/browse/IGNITE-19922?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kirill Sizov resolved IGNITE-19922. Resolution: Won't Fix Command line parameter helped. Closing this Jira. > Gradle checkstyle tasks are greedy > -- > > Key: IGNITE-19922 > URL: https://issues.apache.org/jira/browse/IGNITE-19922 > Project: Ignite > Issue Type: New Feature >Reporter: Mikhail Pochatkin >Assignee: Kirill Sizov >Priority: Major > Labels: ignite-3 > Attachments: image-2023-07-06-11-18-40-515.png, screenshot-1.png > > > This is memory consumption during {{gradlew checkstyleMain}} execution - > goes from ~10 GB to 30. All CPU cores are also at 100%. This causes chrome > tabs to unload and overall stress on the system. > Also, RAM usage does not go down after this command unless I kill/stop Gradle > daemons > !image-2023-07-06-11-18-40-515.png! -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (IGNITE-20135) NO_INDEX hint for calcite engine.
Vladimir Steshin created IGNITE-20135: - Summary: NO_INDEX hint for calcite engine. Key: IGNITE-20135 URL: https://issues.apache.org/jira/browse/IGNITE-20135 Project: Ignite Issue Type: Bug Reporter: Vladimir Steshin Assignee: Vladimir Steshin As part of hints for Calcite, we could try to implement simple hint like NO_INDEX. The main goal is to take deeper look into current hints implementation in Calcite and find a way to work with them. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (IGNITE-20136) INDEX hint for calcite engine.
Vladimir Steshin created IGNITE-20136: - Summary: INDEX hint for calcite engine. Key: IGNITE-20136 URL: https://issues.apache.org/jira/browse/IGNITE-20136 Project: Ignite Issue Type: Bug Reporter: Vladimir Steshin -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-20136) INDEX hint for calcite engine.
[ https://issues.apache.org/jira/browse/IGNITE-20136?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vladimir Steshin updated IGNITE-20136: -- Description: As part of hints for Calcite, we could try to implement simple hint like INDEX. The main goal is to take deeper look into current hints implementation in Calcite and find a way to work with them. > INDEX hint for calcite engine. > -- > > Key: IGNITE-20136 > URL: https://issues.apache.org/jira/browse/IGNITE-20136 > Project: Ignite > Issue Type: Bug >Reporter: Vladimir Steshin >Priority: Major > > As part of hints for Calcite, we could try to implement simple hint like > INDEX. The main goal is to take deeper look into current hints implementation > in Calcite and find a way to work with them. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-20135) NO_INDEX hint for calcite engine.
[ https://issues.apache.org/jira/browse/IGNITE-20135?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vladimir Steshin updated IGNITE-20135: -- Labels: ise (was: ) > NO_INDEX hint for calcite engine. > - > > Key: IGNITE-20135 > URL: https://issues.apache.org/jira/browse/IGNITE-20135 > Project: Ignite > Issue Type: Bug >Reporter: Vladimir Steshin >Assignee: Vladimir Steshin >Priority: Major > Labels: ise > > As part of hints for Calcite, we could try to implement simple hint like > NO_INDEX. The main goal is to take deeper look into current hints > implementation in Calcite and find a way to work with them. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (IGNITE-20136) INDEX hint for calcite engine.
[ https://issues.apache.org/jira/browse/IGNITE-20136?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vladimir Steshin reassigned IGNITE-20136: - Assignee: Vladimir Steshin > INDEX hint for calcite engine. > -- > > Key: IGNITE-20136 > URL: https://issues.apache.org/jira/browse/IGNITE-20136 > Project: Ignite > Issue Type: Bug >Reporter: Vladimir Steshin >Assignee: Vladimir Steshin >Priority: Major > Labels: ise > > As part of hints for Calcite, we could try to implement simple hint like > INDEX. The main goal is to take deeper look into current hints implementation > in Calcite and find a way to work with them. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-20136) INDEX hint for calcite engine.
[ https://issues.apache.org/jira/browse/IGNITE-20136?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vladimir Steshin updated IGNITE-20136: -- Labels: ise (was: ) > INDEX hint for calcite engine. > -- > > Key: IGNITE-20136 > URL: https://issues.apache.org/jira/browse/IGNITE-20136 > Project: Ignite > Issue Type: Bug >Reporter: Vladimir Steshin >Priority: Major > Labels: ise > > As part of hints for Calcite, we could try to implement simple hint like > INDEX. The main goal is to take deeper look into current hints implementation > in Calcite and find a way to work with them. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-20137) testOneNodeRestartWithGap is flaky
[ https://issues.apache.org/jira/browse/IGNITE-20137?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexey Scherbakov updated IGNITE-20137: --- Labels: ignite-3 (was: ) > testOneNodeRestartWithGap is flaky > -- > > Key: IGNITE-20137 > URL: https://issues.apache.org/jira/browse/IGNITE-20137 > Project: Ignite > Issue Type: Bug >Affects Versions: 3.0 >Reporter: Alexey Scherbakov >Priority: Major > Labels: ignite-3 > Fix For: 3.0 > > > Looks like sometime the stopped node is not removed from the topology -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (IGNITE-20137) testOneNodeRestartWithGap is flaky
Alexey Scherbakov created IGNITE-20137: -- Summary: testOneNodeRestartWithGap is flaky Key: IGNITE-20137 URL: https://issues.apache.org/jira/browse/IGNITE-20137 Project: Ignite Issue Type: Bug Affects Versions: 3.0 Reporter: Alexey Scherbakov Fix For: 3.0 Looks like sometime the stopped node is not removed from the topology -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (IGNITE-15927) Implement one phase commit
[ https://issues.apache.org/jira/browse/IGNITE-15927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17750286#comment-17750286 ] Alexander Lapin commented on IGNITE-15927: -- [~ascherbakov] lgtm from my side. > Implement one phase commit > -- > > Key: IGNITE-15927 > URL: https://issues.apache.org/jira/browse/IGNITE-15927 > Project: Ignite > Issue Type: Improvement >Reporter: Alexey Scherbakov >Assignee: Alexey Scherbakov >Priority: Major > Labels: ignite-3 > Time Spent: 2h 50m > Remaining Estimate: 0h > > If all keys in the implicit transaction belong to a same partition in can be > committed in one round-trip. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-15927) Implement one phase commit
[ https://issues.apache.org/jira/browse/IGNITE-15927?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexander Lapin updated IGNITE-15927: - Reviewer: Alexander Lapin > Implement one phase commit > -- > > Key: IGNITE-15927 > URL: https://issues.apache.org/jira/browse/IGNITE-15927 > Project: Ignite > Issue Type: Improvement >Reporter: Alexey Scherbakov >Assignee: Alexey Scherbakov >Priority: Major > Labels: ignite-3 > Time Spent: 2h 50m > Remaining Estimate: 0h > > If all keys in the implicit transaction belong to a same partition in can be > committed in one round-trip. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (IGNITE-20138) PageMemory should use actual dataregion's checkpointer.
Vladimir Steshin created IGNITE-20138: - Summary: PageMemory should use actual dataregion's checkpointer. Key: IGNITE-20138 URL: https://issues.apache.org/jira/browse/IGNITE-20138 Project: Ignite Issue Type: Improvement Reporter: Vladimir Steshin We create PageMemoryImpl as: {code:java} PageMemoryImpl pageMem = new PageMemoryImpl( wrapMetricsPersistentMemoryProvider(memProvider, regMetrics), calculateFragmentSizes( regCfg.getName(), dsCfg.getConcurrencyLevel(), cacheSize, chpBufSize ), cctx, pageMgr, dsCfg.getPageSize(), (fullId, pageBuf, tag) -> { regMetrics.onPageWritten(); // Write page to disk. pageMgr.write(fullId.groupId(), fullId.pageId(), pageBuf, tag, true); getCheckpointer().currentProgress().updateEvictedPages(1); }, trackable, this, regMetrics, regCfg, resolveThrottlingPolicy(), () -> getCheckpointer().currentProgress() ); {code} Where `getCheckpointer()` and the `CheckpointLockStateChecker stateChecker` constructor parameter can be bound to other data region. For instance, defragmentation uses own reqion and own checkpointer. See IGNITE-19904 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-20138) PageMemory should use actual dataregion's checkpointer.
[ https://issues.apache.org/jira/browse/IGNITE-20138?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vladimir Steshin updated IGNITE-20138: -- Description: We create PageMemoryImpl as: {code:java} PageMemoryImpl pageMem = new PageMemoryImpl( wrapMetricsPersistentMemoryProvider(memProvider, regMetrics), calculateFragmentSizes( regCfg.getName(), dsCfg.getConcurrencyLevel(), cacheSize, chpBufSize ), cctx, pageMgr, dsCfg.getPageSize(), (fullId, pageBuf, tag) -> { regMetrics.onPageWritten(); // Write page to disk. pageMgr.write(fullId.groupId(), fullId.pageId(), pageBuf, tag, true); getCheckpointer().currentProgress().updateEvictedPages(1); }, trackable, this, regMetrics, regCfg, resolveThrottlingPolicy(), () -> getCheckpointer().currentProgress() ); {code} Where `getCheckpointer()` and the `CheckpointLockStateChecker stateChecker` constructor parameter can be not bound to PageMemory's data region. For instance, defragmentation uses own reqion and own checkpointer. See IGNITE-19904 was: We create PageMemoryImpl as: {code:java} PageMemoryImpl pageMem = new PageMemoryImpl( wrapMetricsPersistentMemoryProvider(memProvider, regMetrics), calculateFragmentSizes( regCfg.getName(), dsCfg.getConcurrencyLevel(), cacheSize, chpBufSize ), cctx, pageMgr, dsCfg.getPageSize(), (fullId, pageBuf, tag) -> { regMetrics.onPageWritten(); // Write page to disk. pageMgr.write(fullId.groupId(), fullId.pageId(), pageBuf, tag, true); getCheckpointer().currentProgress().updateEvictedPages(1); }, trackable, this, regMetrics, regCfg, resolveThrottlingPolicy(), () -> getCheckpointer().currentProgress() ); {code} Where `getCheckpointer()` and the `CheckpointLockStateChecker stateChecker` constructor parameter can be bound to other data region. For instance, defragmentation uses own reqion and own checkpointer. See IGNITE-19904 > PageMemory should use actual dataregion's checkpointer. > --- > > Key: IGNITE-20138 > URL: https://issues.apache.org/jira/browse/IGNITE-20138 > Project: Ignite > Issue Type: Improvement >Reporter: Vladimir Steshin >Priority: Major > > We create PageMemoryImpl as: > {code:java} > PageMemoryImpl pageMem = new PageMemoryImpl( > wrapMetricsPersistentMemoryProvider(memProvider, regMetrics), > calculateFragmentSizes( > regCfg.getName(), > dsCfg.getConcurrencyLevel(), > cacheSize, > chpBufSize > ), > cctx, > pageMgr, > dsCfg.getPageSize(), > (fullId, pageBuf, tag) -> { > regMetrics.onPageWritten(); > // Write page to disk. > pageMgr.write(fullId.groupId(), fullId.pageId(), pageBuf, > tag, true); > getCheckpointer().currentProgress().updateEvictedPages(1); > }, > trackable, > this, > regMetrics, > regCfg, > resolveThrottlingPolicy(), > () -> getCheckpointer().currentProgress() > ); > {code} > Where `getCheckpointer()` and the `CheckpointLockStateChecker stateChecker` > constructor parameter can be not bound to PageMemory's data region. For > instance, defragmentation uses own reqion and own checkpointer. See > IGNITE-19904 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-20138) PageMemory should use actual dataregion's checkpointer.
[ https://issues.apache.org/jira/browse/IGNITE-20138?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vladimir Steshin updated IGNITE-20138: -- Description: We create PageMemoryImpl as: {code:java} PageMemoryImpl pageMem = new PageMemoryImpl( wrapMetricsPersistentMemoryProvider(memProvider, regMetrics), calculateFragmentSizes( regCfg.getName(), dsCfg.getConcurrencyLevel(), cacheSize, chpBufSize ), cctx, pageMgr, dsCfg.getPageSize(), (fullId, pageBuf, tag) -> { regMetrics.onPageWritten(); // Write page to disk. pageMgr.write(fullId.groupId(), fullId.pageId(), pageBuf, tag, true); getCheckpointer().currentProgress().updateEvictedPages(1); }, trackable, this, regMetrics, regCfg, resolveThrottlingPolicy(), () -> getCheckpointer().currentProgress() ); {code} Where `getCheckpointer()` and the `CheckpointLockStateChecker stateChecker` constructor parameter can be not bound to PageMemory's data region. For instance, defragmentation uses own region and own lightweight checkpointer. See IGNITE-19904 was: We create PageMemoryImpl as: {code:java} PageMemoryImpl pageMem = new PageMemoryImpl( wrapMetricsPersistentMemoryProvider(memProvider, regMetrics), calculateFragmentSizes( regCfg.getName(), dsCfg.getConcurrencyLevel(), cacheSize, chpBufSize ), cctx, pageMgr, dsCfg.getPageSize(), (fullId, pageBuf, tag) -> { regMetrics.onPageWritten(); // Write page to disk. pageMgr.write(fullId.groupId(), fullId.pageId(), pageBuf, tag, true); getCheckpointer().currentProgress().updateEvictedPages(1); }, trackable, this, regMetrics, regCfg, resolveThrottlingPolicy(), () -> getCheckpointer().currentProgress() ); {code} Where `getCheckpointer()` and the `CheckpointLockStateChecker stateChecker` constructor parameter can be not bound to PageMemory's data region. For instance, defragmentation uses own reqion and own checkpointer. See IGNITE-19904 > PageMemory should use actual dataregion's checkpointer. > --- > > Key: IGNITE-20138 > URL: https://issues.apache.org/jira/browse/IGNITE-20138 > Project: Ignite > Issue Type: Improvement >Reporter: Vladimir Steshin >Priority: Major > > We create PageMemoryImpl as: > {code:java} > PageMemoryImpl pageMem = new PageMemoryImpl( > wrapMetricsPersistentMemoryProvider(memProvider, regMetrics), > calculateFragmentSizes( > regCfg.getName(), > dsCfg.getConcurrencyLevel(), > cacheSize, > chpBufSize > ), > cctx, > pageMgr, > dsCfg.getPageSize(), > (fullId, pageBuf, tag) -> { > regMetrics.onPageWritten(); > // Write page to disk. > pageMgr.write(fullId.groupId(), fullId.pageId(), pageBuf, > tag, true); > getCheckpointer().currentProgress().updateEvictedPages(1); > }, > trackable, > this, > regMetrics, > regCfg, > resolveThrottlingPolicy(), > () -> getCheckpointer().currentProgress() > ); > {code} > Where `getCheckpointer()` and the `CheckpointLockStateChecker stateChecker` > constructor parameter can be not bound to PageMemory's data region. For > instance, defragmentation uses own region and own lightweight checkpointer. > See IGNITE-19904 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-20136) INDEX hint for calcite engine.
[ https://issues.apache.org/jira/browse/IGNITE-20136?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vladimir Steshin updated IGNITE-20136: -- Issue Type: Task (was: Bug) > INDEX hint for calcite engine. > -- > > Key: IGNITE-20136 > URL: https://issues.apache.org/jira/browse/IGNITE-20136 > Project: Ignite > Issue Type: Task >Reporter: Vladimir Steshin >Assignee: Vladimir Steshin >Priority: Major > Labels: ise > > As part of hints for Calcite, we could try to implement simple hint like > INDEX. The main goal is to take deeper look into current hints implementation > in Calcite and find a way to work with them. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (IGNITE-20139) RandomForestClassifierTrainer is checking the same conditions
Alexandr Shapkin created IGNITE-20139: - Summary: RandomForestClassifierTrainer is checking the same conditions Key: IGNITE-20139 URL: https://issues.apache.org/jira/browse/IGNITE-20139 Project: Ignite Issue Type: Bug Components: ml Affects Versions: 2.15 Reporter: Alexandr Shapkin Attachments: TreeSample2_Portfolio_Change.png, random-forest.zip We tried to use GridGain's machine learning capabilities, and discovered a bug in GG's implementation of Random Forest. When comparing GG's output with python prototype (scikit-learn lib), we noticed that GG's predictions have much lower accuracy despite using the same data set and model parameters. Further investigation showed that GridGain generates decision trees that kinda "loop". The tree starts checking the same condition over and over until it reaches the maximum tree depth. I've attached a standalone reproducer which uses a small excerpt of our data set. It loads data from the csv file, then performs the training of the model for just 1 tree. Then the reproducer finds one of the looping branches and prints it. You will see that every single node in the branch uses the same feature, value and has then same calculated impurity. On my machine the code reproduces this issue 100% of time. I've also attached an example of the tree generated by python's scikit-learn on the same data set with the same parameters. In python the tree usually doesn't get deeper than 20 nodes. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-20135) NO_INDEX hint for calcite engine.
[ https://issues.apache.org/jira/browse/IGNITE-20135?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vladimir Steshin updated IGNITE-20135: -- Issue Type: Task (was: Bug) > NO_INDEX hint for calcite engine. > - > > Key: IGNITE-20135 > URL: https://issues.apache.org/jira/browse/IGNITE-20135 > Project: Ignite > Issue Type: Task >Reporter: Vladimir Steshin >Assignee: Vladimir Steshin >Priority: Major > Labels: ise > > As part of hints for Calcite, we could try to implement simple hint like > NO_INDEX. The main goal is to take deeper look into current hints > implementation in Calcite and find a way to work with them. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-20139) RandomForestClassifierTrainer is checking the same conditions
[ https://issues.apache.org/jira/browse/IGNITE-20139?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexandr Shapkin updated IGNITE-20139: -- Attachment: random-forest.zip TreeSample2_Portfolio_Change.png > RandomForestClassifierTrainer is checking the same conditions > - > > Key: IGNITE-20139 > URL: https://issues.apache.org/jira/browse/IGNITE-20139 > Project: Ignite > Issue Type: Bug > Components: ml >Affects Versions: 2.15 >Reporter: Alexandr Shapkin >Priority: Major > Attachments: TreeSample2_Portfolio_Change.png, random-forest.zip > > > We tried to use GridGain's machine learning capabilities, and discovered a > bug in GG's implementation of Random Forest. When comparing GG's output with > python prototype (scikit-learn lib), we noticed that GG's predictions have > much lower accuracy despite using the same data set and model parameters. > Further investigation showed that GridGain generates decision trees that > kinda "loop". The tree starts checking the same condition over and over until > it reaches the maximum tree depth. > I've attached a standalone reproducer which uses a small excerpt of our data > set. > It loads data from the csv file, then performs the training of the model for > just 1 tree. Then the reproducer finds one of the looping branches and prints > it. You will see that every single node in the branch uses the same feature, > value and has then same calculated impurity. > On my machine the code reproduces this issue 100% of time. > I've also attached an example of the tree generated by python's scikit-learn > on the same data set with the same parameters. In python the tree usually > doesn't get deeper than 20 nodes. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (IGNITE-20139) RandomForestClassifierTrainer accuracy issue
[ https://issues.apache.org/jira/browse/IGNITE-20139?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17750303#comment-17750303 ] Alexandr Shapkin commented on IGNITE-20139: --- [~zaleslaw] could you please take a look? Seems to be a valid issue alongside the attached reproducer. > RandomForestClassifierTrainer accuracy issue > > > Key: IGNITE-20139 > URL: https://issues.apache.org/jira/browse/IGNITE-20139 > Project: Ignite > Issue Type: Bug > Components: ml >Affects Versions: 2.15 >Reporter: Alexandr Shapkin >Priority: Major > Attachments: TreeSample2_Portfolio_Change.png, random-forest.zip > > > We tried to use GridGain's machine learning capabilities, and discovered a > bug in GG's implementation of Random Forest. When comparing GG's output with > python prototype (scikit-learn lib), we noticed that GG's predictions have > much lower accuracy despite using the same data set and model parameters. > Further investigation showed that GridGain generates decision trees that > kinda "loop". The tree starts checking the same condition over and over until > it reaches the maximum tree depth. > I've attached a standalone reproducer which uses a small excerpt of our data > set. > It loads data from the csv file, then performs the training of the model for > just 1 tree. Then the reproducer finds one of the looping branches and prints > it. You will see that every single node in the branch uses the same feature, > value and has then same calculated impurity. > On my machine the code reproduces this issue 100% of time. > I've also attached an example of the tree generated by python's scikit-learn > on the same data set with the same parameters. In python the tree usually > doesn't get deeper than 20 nodes. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-20139) RandomForestClassifierTrainer accuracy issue
[ https://issues.apache.org/jira/browse/IGNITE-20139?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexandr Shapkin updated IGNITE-20139: -- Summary: RandomForestClassifierTrainer accuracy issue (was: RandomForestClassifierTrainer is checking the same conditions) > RandomForestClassifierTrainer accuracy issue > > > Key: IGNITE-20139 > URL: https://issues.apache.org/jira/browse/IGNITE-20139 > Project: Ignite > Issue Type: Bug > Components: ml >Affects Versions: 2.15 >Reporter: Alexandr Shapkin >Priority: Major > Attachments: TreeSample2_Portfolio_Change.png, random-forest.zip > > > We tried to use GridGain's machine learning capabilities, and discovered a > bug in GG's implementation of Random Forest. When comparing GG's output with > python prototype (scikit-learn lib), we noticed that GG's predictions have > much lower accuracy despite using the same data set and model parameters. > Further investigation showed that GridGain generates decision trees that > kinda "loop". The tree starts checking the same condition over and over until > it reaches the maximum tree depth. > I've attached a standalone reproducer which uses a small excerpt of our data > set. > It loads data from the csv file, then performs the training of the model for > just 1 tree. Then the reproducer finds one of the looping branches and prints > it. You will see that every single node in the branch uses the same feature, > value and has then same calculated impurity. > On my machine the code reproduces this issue 100% of time. > I've also attached an example of the tree generated by python's scikit-learn > on the same data set with the same parameters. In python the tree usually > doesn't get deeper than 20 nodes. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-15616) Calcite. Failed to parse UPDATE query.
[ https://issues.apache.org/jira/browse/IGNITE-15616?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yury Gerzhedovich updated IGNITE-15616: --- Labels: calcite ignite-3 (was: calcite calcite2-required calcite3-required ignite-3) > Calcite. Failed to parse UPDATE query. > -- > > Key: IGNITE-15616 > URL: https://issues.apache.org/jira/browse/IGNITE-15616 > Project: Ignite > Issue Type: Bug > Components: sql >Reporter: Evgeny Stanilovsky >Priority: Minor > Labels: calcite, ignite-3 > > SQL Standart feature E153 - Updatable queries with sub-queries doesn't > support right now. > > {noformat} > statement ok > UPDATE integers i1 SET i=DEFAULT WHERE i=(SELECT MIN(i) FROM integers WHERE > i1.id query II > SELECT id, i FROM integers ORDER BY id > > 1 NULL > 2 NULL > 3 2 > 4 3 > {noformat} > {noformat} > Statement [queries=ArrayList [UPDATE integers i1 SET i=DEFAULT WHERE > i=(SELECT MIN(i) FROM integers WHERE i1.id at > org.apache.ignite.internal.processors.query.calcite.logical.SqlScriptRunner$Statement.execute(SqlScriptRunner.java:404) > at > org.apache.ignite.internal.processors.query.calcite.logical.SqlScriptRunner.run(SqlScriptRunner.java:115) > at > org.apache.ignite.internal.processors.query.calcite.logical.ScriptTestRunner$1.run(ScriptTestRunner.java:219) > at java.lang.Thread.run(Thread.java:748) > Caused by: class > org.apache.ignite.internal.processors.query.IgniteSQLException: Failed to > parse query. > at > org.apache.ignite.internal.processors.query.calcite.util.Commons.parse(Commons.java:205) > {noformat} > {noformat} > /subquery/scalar/test_update_subquery.test[_ignore] > {noformat} -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (IGNITE-15616) Calcite. Failed to parse UPDATE query.
[ https://issues.apache.org/jira/browse/IGNITE-15616?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yury Gerzhedovich resolved IGNITE-15616. Resolution: Cannot Reproduce > Calcite. Failed to parse UPDATE query. > -- > > Key: IGNITE-15616 > URL: https://issues.apache.org/jira/browse/IGNITE-15616 > Project: Ignite > Issue Type: Bug > Components: sql >Reporter: Evgeny Stanilovsky >Priority: Minor > Labels: calcite, ignite-3 > > SQL Standart feature E153 - Updatable queries with sub-queries doesn't > support right now. > > {noformat} > statement ok > UPDATE integers i1 SET i=DEFAULT WHERE i=(SELECT MIN(i) FROM integers WHERE > i1.id query II > SELECT id, i FROM integers ORDER BY id > > 1 NULL > 2 NULL > 3 2 > 4 3 > {noformat} > {noformat} > Statement [queries=ArrayList [UPDATE integers i1 SET i=DEFAULT WHERE > i=(SELECT MIN(i) FROM integers WHERE i1.id at > org.apache.ignite.internal.processors.query.calcite.logical.SqlScriptRunner$Statement.execute(SqlScriptRunner.java:404) > at > org.apache.ignite.internal.processors.query.calcite.logical.SqlScriptRunner.run(SqlScriptRunner.java:115) > at > org.apache.ignite.internal.processors.query.calcite.logical.ScriptTestRunner$1.run(ScriptTestRunner.java:219) > at java.lang.Thread.run(Thread.java:748) > Caused by: class > org.apache.ignite.internal.processors.query.IgniteSQLException: Failed to > parse query. > at > org.apache.ignite.internal.processors.query.calcite.util.Commons.parse(Commons.java:205) > {noformat} > {noformat} > /subquery/scalar/test_update_subquery.test[_ignore] > {noformat} -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (IGNITE-19792) Support @Immutable logic for distributed configuration
[ https://issues.apache.org/jira/browse/IGNITE-19792?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kirill Sizov reassigned IGNITE-19792: -- Assignee: Kirill Sizov > Support @Immutable logic for distributed configuration > -- > > Key: IGNITE-19792 > URL: https://issues.apache.org/jira/browse/IGNITE-19792 > Project: Ignite > Issue Type: Improvement >Reporter: Roman Puchkovskiy >Assignee: Kirill Sizov >Priority: Major > Labels: ignite-3 > Fix For: 3.0.0-beta2 > > > For distributed properties, the following scenario makes sense: > # Property value might be set on cluster init > # If it is provided, it overrides the default; otherwise, the default (set > in the {{@Value}} annotation) is used > # Any attempt to modify the property value using a changer is rejected by a > validator > {{@Immutable}} annotation is designed to trigger such behavior. > Now, the observed behavior is different: if a property is annotated as > {{{}@Immutable{}}}, then, if a value is supplied during cluster > initialization, the value is silently ignored, so it is impossible to > configure such property to anything different from the default. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-19861) Introduce SQL metrics
[ https://issues.apache.org/jira/browse/IGNITE-19861?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrey Mashenkov updated IGNITE-19861: -- Fix Version/s: 3.0.0-beta2 > Introduce SQL metrics > - > > Key: IGNITE-19861 > URL: https://issues.apache.org/jira/browse/IGNITE-19861 > Project: Ignite > Issue Type: Improvement > Components: sql >Reporter: Yury Gerzhedovich >Assignee: Yury Gerzhedovich >Priority: Major > Labels: ignite-3 > Fix For: 3.0.0-beta2 > > Time Spent: 2h 10m > Remaining Estimate: 0h > > AI3 have metrics framework, but SQL engine don't expose any metrics, which > could help to user. > Let's introduce the first three metrics: > 1) hit in plan cache - counter > 2) miss in plan cache - counter > 3) number of open cursors - gauge -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (IGNITE-19916) TX SQL tests refactoring with the help of @TestOnly IgniteImpl#txManager and TxManager#pending
[ https://issues.apache.org/jira/browse/IGNITE-19916?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yury Gerzhedovich resolved IGNITE-19916. Resolution: Cannot Reproduce fixed as part of https://issues.apache.org/jira/browse/IGNITE-19861 > TX SQL tests refactoring with the help of @TestOnly IgniteImpl#txManager and > TxManager#pending > --- > > Key: IGNITE-19916 > URL: https://issues.apache.org/jira/browse/IGNITE-19916 > Project: Ignite > Issue Type: Improvement > Components: sql >Reporter: Alexander Lapin >Priority: Major > Labels: ignite-3 > > h3. Motivation > Rather often it's required to check whether all transactions that were > started by SQL are properly finished. Currently, reflection along with crappy > txnState checks are used: > {code:java} > TxManager txManagerInternal = (TxManager) > IgniteTestUtils.getFieldValue(CLUSTER_NODES.get(0), IgniteImpl.class, > "txManager"); > var states = (Map) > IgniteTestUtils.getFieldValue(txManagerInternal, TxManagerImpl.class, > "states"); > assertEquals(txManagerInternal.finished(), states.size());{code} > After implementing https://issues.apache.org/jira/browse/IGNITE-19663 it > should be possible to rewrite code above with > {code:java} > assertEquals(0, ((IgniteImpl) CLUSTER_NODES.get(0)).txManager().pending()); > {code} > or similar. > h3. Definition of Done > All required sql tx tests are simplified through TxManager#pending() > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (IGNITE-20140) Exceptions derived from IgniteException are constructed unexpectedly while copying
Vadim Pakhnushev created IGNITE-20140: - Summary: Exceptions derived from IgniteException are constructed unexpectedly while copying Key: IGNITE-20140 URL: https://issues.apache.org/jira/browse/IGNITE-20140 Project: Ignite Issue Type: Bug Reporter: Vadim Pakhnushev Assignee: Vyacheslav Koptilin When exception derived from IgniteException is thrown from the SQL command, {{org.apache.ignite.internal.sql.engine.AsyncSqlCursorImpl#requestNextAsync}} wraps it using the {{{}org.apache.ignite.lang.IgniteExceptionUtils#copyExceptionWithCause{}}}, which tries to find corresponding constructor in the derived class. The problem is that when the exception has a custom constructor with the matching signature, for instance, with the single String argument, it gets called with the original exception message leading to duplicated messages if the custom constructor constructs a message from its argument. It the exception class doesn't have matching constructor, the original exception gets lots because the handler in the {{requestNextAsync}} throws a {{{}null{}}}. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-20140) Exceptions derived from IgniteException are constructed unexpectedly while copying
[ https://issues.apache.org/jira/browse/IGNITE-20140?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vyacheslav Koptilin updated IGNITE-20140: - Description: When exception derived from IgniteException is thrown from the SQL command, {{org.apache.ignite.internal.sql.engine.AsyncSqlCursorImpl#requestNextAsync}} wraps it using the {{{}org.apache.ignite.lang.IgniteExceptionUtils#copyExceptionWithCause{}}}, which tries to find corresponding constructor in the derived class. The problem is that when the exception has a custom constructor with the matching signature, for instance, with the single String argument, it gets called with the original exception message leading to duplicated messages if the custom constructor constructs a message from its argument. If the exception class doesn't have matching constructor, the original exception gets lots because the handler in the {{requestNextAsync}} throws a {{{}null{}}}. was: When exception derived from IgniteException is thrown from the SQL command, {{org.apache.ignite.internal.sql.engine.AsyncSqlCursorImpl#requestNextAsync}} wraps it using the {{{}org.apache.ignite.lang.IgniteExceptionUtils#copyExceptionWithCause{}}}, which tries to find corresponding constructor in the derived class. The problem is that when the exception has a custom constructor with the matching signature, for instance, with the single String argument, it gets called with the original exception message leading to duplicated messages if the custom constructor constructs a message from its argument. It the exception class doesn't have matching constructor, the original exception gets lots because the handler in the {{requestNextAsync}} throws a {{{}null{}}}. > Exceptions derived from IgniteException are constructed unexpectedly while > copying > -- > > Key: IGNITE-20140 > URL: https://issues.apache.org/jira/browse/IGNITE-20140 > Project: Ignite > Issue Type: Bug >Reporter: Vadim Pakhnushev >Assignee: Vyacheslav Koptilin >Priority: Major > Labels: ignite-3 > > When exception derived from IgniteException is thrown from the SQL command, > {{org.apache.ignite.internal.sql.engine.AsyncSqlCursorImpl#requestNextAsync}} > wraps it using the > {{{}org.apache.ignite.lang.IgniteExceptionUtils#copyExceptionWithCause{}}}, > which tries to find corresponding constructor in the derived class. > The problem is that when the exception has a custom constructor with the > matching signature, for instance, with the single String argument, it gets > called with the original exception message leading to duplicated messages if > the custom constructor constructs a message from its argument. > If the exception class doesn't have matching constructor, the original > exception gets lots because the handler in the {{requestNextAsync}} throws a > {{{}null{}}}. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-19160) Improve message about sent partition file during snapshot restore
[ https://issues.apache.org/jira/browse/IGNITE-19160?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Julia Bakulina updated IGNITE-19160: Ignite Flags: Release Notes Required (was: Docs Required,Release Notes Required) > Improve message about sent partition file during snapshot restore > - > > Key: IGNITE-19160 > URL: https://issues.apache.org/jira/browse/IGNITE-19160 > Project: Ignite > Issue Type: Task >Reporter: Ilya Shishkov >Assignee: Julia Bakulina >Priority: Minor > Labels: iep-43, ise > > Currently, message about partition is as below: > {quote} > [2023-03-29T18:31:44,773][INFO ][snapshot-runner-#863%node0%][SnapshotSender] > Partition file has been send [part=part-645.bin, > pair={color:red}GroupPartitionId [grpId=1544803905, partId=645]{color}, > length=45056] > {quote} > It does not tell us: > # Receiver node id / address / consistent id. > # Cache or cache group name which partition belongs to. Numerical group id is > not convenient way to determine cache or cache group. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-19160) Improve message about sent partition file during snapshot restore
[ https://issues.apache.org/jira/browse/IGNITE-19160?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Julia Bakulina updated IGNITE-19160: Release Note: Improved message about sent partition file during snapshot restore > Improve message about sent partition file during snapshot restore > - > > Key: IGNITE-19160 > URL: https://issues.apache.org/jira/browse/IGNITE-19160 > Project: Ignite > Issue Type: Task >Reporter: Ilya Shishkov >Assignee: Julia Bakulina >Priority: Minor > Labels: iep-43, ise > > Currently, message about partition is as below: > {quote} > [2023-03-29T18:31:44,773][INFO ][snapshot-runner-#863%node0%][SnapshotSender] > Partition file has been send [part=part-645.bin, > pair={color:red}GroupPartitionId [grpId=1544803905, partId=645]{color}, > length=45056] > {quote} > It does not tell us: > # Receiver node id / address / consistent id. > # Cache or cache group name which partition belongs to. Numerical group id is > not convenient way to determine cache or cache group. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (IGNITE-20023) Unstable cluster topology leads to flaky test
[ https://issues.apache.org/jira/browse/IGNITE-20023?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Roman Puchkovskiy reassigned IGNITE-20023: -- Assignee: Roman Puchkovskiy > Unstable cluster topology leads to flaky test > - > > Key: IGNITE-20023 > URL: https://issues.apache.org/jira/browse/IGNITE-20023 > Project: Ignite > Issue Type: Bug >Affects Versions: 3.0 >Reporter: Alexey Scherbakov >Assignee: Roman Puchkovskiy >Priority: Major > Labels: ignite-3 > > {noformat} > 2023-07-21 18:30:15:108 +0300 > [INFO][main][ItTxDistributedTestThreeNodesThreeReplicas] >>> Starting test: > ItTxDistributedTestThreeNodesThreeReplicas#testTxClosureAsync, displayName: > testTxClosureAsync(), workDir: > D:\work\ignite-3\modules\table\build\work\ItTxDistributedTestThreeNodesThreeReplicas\ttca_25996 > 2023-07-21 18:30:15:142 +0300 [INFO][main][ConnectionManager] Server started > [address=/0:0:0:0:0:0:0:0:20001] > 2023-07-21 18:30:15:142 +0300 > [INFO][ForkJoinPool.commonPool-worker-7][ConnectionManager] Server started > [address=/0:0:0:0:0:0:0:0:2] > 2023-07-21 18:30:15:142 +0300 > [INFO][ForkJoinPool.commonPool-worker-3][ConnectionManager] Server started > [address=/0:0:0:0:0:0:0:0:20002] > 2023-07-21 18:30:15:145 +0300 > [INFO][itdttntr_n_2-srv-worker-1][RecoveryServerHandshakeManager] Failed > to acquire recovery descriptor during handshake, it is held by: [id: > 0x348087cd, L:/192.168.0.138:63056 - R:/192.168.0.138:20001] > 2023-07-21 18:30:15:147 +0300 > [INFO][itdttntr_n_20002-client-1][RecoveryClientHandshakeManager] Failed to > acquire recovery descriptor during handshake, it is held by: [id: 0x8017c1b2, > L:/192.168.0.138:20002 - R:/192.168.0.138:63058] > 2023-07-21 18:30:15:147 +0300 > [INFO][itdttntr_n_20002-client-2][RecoveryClientHandshakeManager] Failed to > acquire recovery descriptor during handshake, it is held by: [id: 0xb26b9231, > L:/192.168.0.138:20002 - R:/192.168.0.138:63059] > 2023-07-21 18:30:15:149 +0300 > [INFO][itdttntr_n_20001-client-3][RecoveryClientHandshakeManager] Failed to > acquire recovery descriptor during handshake, it is held by: [id: 0x3dd8aa9e, > L:/192.168.0.138:20001 - R:/192.168.0.138:63056] > 2023-07-21 18:30:15:149 +0300 > [INFO][itdttntr_n_20002-client-3][RecoveryClientHandshakeManager] Failed to > acquire recovery descriptor during handshake, it is held by: [id: 0xb26b9231, > L:/192.168.0.138:20002 - R:/192.168.0.138:63059] > 2023-07-21 18:30:15:149 +0300 > [INFO][itdttntr_n_20002-client-4][RecoveryClientHandshakeManager] Failed to > acquire recovery descriptor during handshake, it is held by: [id: 0x8017c1b2, > L:/192.168.0.138:20002 - R:/192.168.0.138:63058] > 2023-07-21 18:30:15:150 +0300 > [INFO][sc-cluster-20001-107][ScaleCubeTopologyService] Node joined > [node=ClusterNode [id=ade225ca-5ef0-4b03-9dbe-edbe2e2ba928, > name=itdttntr_n_20002, address=192.168.0.138:20002, nodeMetadata=null]] > 2023-07-21 18:30:15:150 +0300 > [INFO][sc-cluster-20002-106][ScaleCubeTopologyService] Node joined > [node=ClusterNode [id=bee1e04c-e2f0-48c0-ad54-eefdd6928278, > name=itdttntr_n_20001, address=192.168.0.138:20001, nodeMetadata=null]] > 2023-07-21 18:30:15:150 +0300 > [INFO][sc-cluster-2-105][ScaleCubeTopologyService] Node joined > [node=ClusterNode [id=ade225ca-5ef0-4b03-9dbe-edbe2e2ba928, > name=itdttntr_n_20002, address=192.168.0.138:20002, nodeMetadata=null]] > 2023-07-21 18:30:15:150 +0300 > [INFO][sc-cluster-20001-107][ScaleCubeTopologyService] Topology snapshot > [nodes=[itdttntr_n_20002]] > 2023-07-21 18:30:15:150 +0300 > [INFO][sc-cluster-2-105][ScaleCubeTopologyService] Topology snapshot > [nodes=[itdttntr_n_20002]] > 2023-07-21 18:30:15:150 +0300 > [INFO][sc-cluster-20002-106][ScaleCubeTopologyService] Topology snapshot > [nodes=[itdttntr_n_20001]] > 2023-07-21 18:30:15:150 +0300 > [INFO][sc-cluster-20001-107][ScaleCubeTopologyService] Node joined > [node=ClusterNode [id=496bbe6d-66b7-4aa9-9807-e1dd5207d900, > name=itdttntr_n_2, address=192.168.0.138:2, nodeMetadata=null]] > 2023-07-21 18:30:15:150 +0300 > [INFO][sc-cluster-20002-106][ScaleCubeTopologyService] Node joined > [node=ClusterNode [id=496bbe6d-66b7-4aa9-9807-e1dd5207d900, > name=itdttntr_n_2, address=192.168.0.138:2, nodeMetadata=null]] > 2023-07-21 18:30:15:150 +0300 > [INFO][sc-cluster-20001-107][ScaleCubeTopologyService] Topology snapshot > [nodes=[itdttntr_n_2, itdttntr_n_20002]] > 2023-07-21 18:30:15:150 +0300 > [INFO][sc-cluster-20002-106][ScaleCubeTopologyService] Topology snapshot > [nodes=[itdttntr_n_2, itdttntr_n_20001]] > 2023-07-21 18:30:15:151 +0300 > [INFO][ForkJoinPool.commonPool-worker-3][ScaleCubeTopologyService] Node > joined [node=ClusterNode [id=ade225ca-5ef0-4b03-9dbe-edbe2e2ba92
[jira] [Commented] (IGNITE-20023) Unstable cluster topology leads to flaky test
[ https://issues.apache.org/jira/browse/IGNITE-20023?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17750370#comment-17750370 ] Roman Puchkovskiy commented on IGNITE-20023: What I was able to find in the logs: # There is a lot of noise about one node not being let by other nodes to the topology as it has already left (this seems to be irrelevant to the test failure) # There is an NPE (that probably caused the test failure), but additional information is needed to find the culprit. I'm adding it in the attached PR # There is a secondary NPE (it is suppressed), I'm fixing it in the PR > Unstable cluster topology leads to flaky test > - > > Key: IGNITE-20023 > URL: https://issues.apache.org/jira/browse/IGNITE-20023 > Project: Ignite > Issue Type: Bug >Affects Versions: 3.0 >Reporter: Alexey Scherbakov >Assignee: Roman Puchkovskiy >Priority: Major > Labels: ignite-3 > > {noformat} > 2023-07-21 18:30:15:108 +0300 > [INFO][main][ItTxDistributedTestThreeNodesThreeReplicas] >>> Starting test: > ItTxDistributedTestThreeNodesThreeReplicas#testTxClosureAsync, displayName: > testTxClosureAsync(), workDir: > D:\work\ignite-3\modules\table\build\work\ItTxDistributedTestThreeNodesThreeReplicas\ttca_25996 > 2023-07-21 18:30:15:142 +0300 [INFO][main][ConnectionManager] Server started > [address=/0:0:0:0:0:0:0:0:20001] > 2023-07-21 18:30:15:142 +0300 > [INFO][ForkJoinPool.commonPool-worker-7][ConnectionManager] Server started > [address=/0:0:0:0:0:0:0:0:2] > 2023-07-21 18:30:15:142 +0300 > [INFO][ForkJoinPool.commonPool-worker-3][ConnectionManager] Server started > [address=/0:0:0:0:0:0:0:0:20002] > 2023-07-21 18:30:15:145 +0300 > [INFO][itdttntr_n_2-srv-worker-1][RecoveryServerHandshakeManager] Failed > to acquire recovery descriptor during handshake, it is held by: [id: > 0x348087cd, L:/192.168.0.138:63056 - R:/192.168.0.138:20001] > 2023-07-21 18:30:15:147 +0300 > [INFO][itdttntr_n_20002-client-1][RecoveryClientHandshakeManager] Failed to > acquire recovery descriptor during handshake, it is held by: [id: 0x8017c1b2, > L:/192.168.0.138:20002 - R:/192.168.0.138:63058] > 2023-07-21 18:30:15:147 +0300 > [INFO][itdttntr_n_20002-client-2][RecoveryClientHandshakeManager] Failed to > acquire recovery descriptor during handshake, it is held by: [id: 0xb26b9231, > L:/192.168.0.138:20002 - R:/192.168.0.138:63059] > 2023-07-21 18:30:15:149 +0300 > [INFO][itdttntr_n_20001-client-3][RecoveryClientHandshakeManager] Failed to > acquire recovery descriptor during handshake, it is held by: [id: 0x3dd8aa9e, > L:/192.168.0.138:20001 - R:/192.168.0.138:63056] > 2023-07-21 18:30:15:149 +0300 > [INFO][itdttntr_n_20002-client-3][RecoveryClientHandshakeManager] Failed to > acquire recovery descriptor during handshake, it is held by: [id: 0xb26b9231, > L:/192.168.0.138:20002 - R:/192.168.0.138:63059] > 2023-07-21 18:30:15:149 +0300 > [INFO][itdttntr_n_20002-client-4][RecoveryClientHandshakeManager] Failed to > acquire recovery descriptor during handshake, it is held by: [id: 0x8017c1b2, > L:/192.168.0.138:20002 - R:/192.168.0.138:63058] > 2023-07-21 18:30:15:150 +0300 > [INFO][sc-cluster-20001-107][ScaleCubeTopologyService] Node joined > [node=ClusterNode [id=ade225ca-5ef0-4b03-9dbe-edbe2e2ba928, > name=itdttntr_n_20002, address=192.168.0.138:20002, nodeMetadata=null]] > 2023-07-21 18:30:15:150 +0300 > [INFO][sc-cluster-20002-106][ScaleCubeTopologyService] Node joined > [node=ClusterNode [id=bee1e04c-e2f0-48c0-ad54-eefdd6928278, > name=itdttntr_n_20001, address=192.168.0.138:20001, nodeMetadata=null]] > 2023-07-21 18:30:15:150 +0300 > [INFO][sc-cluster-2-105][ScaleCubeTopologyService] Node joined > [node=ClusterNode [id=ade225ca-5ef0-4b03-9dbe-edbe2e2ba928, > name=itdttntr_n_20002, address=192.168.0.138:20002, nodeMetadata=null]] > 2023-07-21 18:30:15:150 +0300 > [INFO][sc-cluster-20001-107][ScaleCubeTopologyService] Topology snapshot > [nodes=[itdttntr_n_20002]] > 2023-07-21 18:30:15:150 +0300 > [INFO][sc-cluster-2-105][ScaleCubeTopologyService] Topology snapshot > [nodes=[itdttntr_n_20002]] > 2023-07-21 18:30:15:150 +0300 > [INFO][sc-cluster-20002-106][ScaleCubeTopologyService] Topology snapshot > [nodes=[itdttntr_n_20001]] > 2023-07-21 18:30:15:150 +0300 > [INFO][sc-cluster-20001-107][ScaleCubeTopologyService] Node joined > [node=ClusterNode [id=496bbe6d-66b7-4aa9-9807-e1dd5207d900, > name=itdttntr_n_2, address=192.168.0.138:2, nodeMetadata=null]] > 2023-07-21 18:30:15:150 +0300 > [INFO][sc-cluster-20002-106][ScaleCubeTopologyService] Node joined > [node=ClusterNode [id=496bbe6d-66b7-4aa9-9807-e1dd5207d900, > name=itdttntr_n_2, address=192.168.0.138:2, nodeMetadata=null]] > 2023-07-21 18:30:15:150 +0300 > [INFO][sc-c
[jira] [Assigned] (IGNITE-19758) Delete "StorageSortedIndexColumnDescriptor#name"
[ https://issues.apache.org/jira/browse/IGNITE-19758?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrey Mashenkov reassigned IGNITE-19758: - Assignee: Andrey Mashenkov > Delete "StorageSortedIndexColumnDescriptor#name" > > > Key: IGNITE-19758 > URL: https://issues.apache.org/jira/browse/IGNITE-19758 > Project: Ignite > Issue Type: Improvement >Reporter: Ivan Bessonov >Assignee: Andrey Mashenkov >Priority: Major > Labels: ignite-3 > > It's only really used in tests. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (IGNITE-20047) NPE in evicted pages counter of the checkpoint progress.
[ https://issues.apache.org/jira/browse/IGNITE-20047?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vladimir Steshin resolved IGNITE-20047. --- Resolution: Fixed Is cause of and fixed in IGNITE-19904 > NPE in evicted pages counter of the checkpoint progress. > > > Key: IGNITE-20047 > URL: https://issues.apache.org/jira/browse/IGNITE-20047 > Project: Ignite > Issue Type: Bug >Reporter: Vladimir Steshin >Priority: Major > Attachments: failureNPE.log, npe_failure2.log, npe_pu_counter_2.15.log > > > NPE can occur in > {code:java} > /** {@inheritDoc} */ > @Override public void updateEvictedPages(int delta) { > A.ensure(delta > 0, "param must be positive"); > if (evictedPagesCounter() != null) > evictedPagesCounter().addAndGet(delta); > } > {code} > because evictedPagesCounter is already null below 'if'. `clearCounters()` is > invoked in _markCheckpointEnd()_. But it is possible that the counter is > being concurrently updated. > Looks able to corrupt PDS if the process failed when checkpoint end marker is > saved but the page is not yet. > Probably, happens with the _DelayedDirtyPageStoreWrite_ / > _DelayedDirtyPageStoreWrite_. Found in the defragmentation process. But it > seems to be a common problem. Logs attached. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Closed] (IGNITE-20047) NPE in evicted pages counter of the checkpoint progress.
[ https://issues.apache.org/jira/browse/IGNITE-20047?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vladimir Steshin closed IGNITE-20047. - IGNITE-19904 > NPE in evicted pages counter of the checkpoint progress. > > > Key: IGNITE-20047 > URL: https://issues.apache.org/jira/browse/IGNITE-20047 > Project: Ignite > Issue Type: Bug >Reporter: Vladimir Steshin >Priority: Major > Attachments: failureNPE.log, npe_failure2.log, npe_pu_counter_2.15.log > > > NPE can occur in > {code:java} > /** {@inheritDoc} */ > @Override public void updateEvictedPages(int delta) { > A.ensure(delta > 0, "param must be positive"); > if (evictedPagesCounter() != null) > evictedPagesCounter().addAndGet(delta); > } > {code} > because evictedPagesCounter is already null below 'if'. `clearCounters()` is > invoked in _markCheckpointEnd()_. But it is possible that the counter is > being concurrently updated. > Looks able to corrupt PDS if the process failed when checkpoint end marker is > saved but the page is not yet. > Probably, happens with the _DelayedDirtyPageStoreWrite_ / > _DelayedDirtyPageStoreWrite_. Found in the defragmentation process. But it > seems to be a common problem. Logs attached. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (IGNITE-20141) TxRollbackOnTimeoutNoDeadlockDetectionTest fails at local run (not from the suite)
Anton Vinogradov created IGNITE-20141: - Summary: TxRollbackOnTimeoutNoDeadlockDetectionTest fails at local run (not from the suite) Key: IGNITE-20141 URL: https://issues.apache.org/jira/browse/IGNITE-20141 Project: Ignite Issue Type: Bug Reporter: Anton Vinogradov Assignee: Anton Vinogradov Fix For: 2.16 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (IGNITE-20141) TxRollbackOnTimeoutNoDeadlockDetectionTest fails at local run (not from the suite)
[ https://issues.apache.org/jira/browse/IGNITE-20141?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17750397#comment-17750397 ] Anton Vinogradov commented on IGNITE-20141: --- TxRollbackOnTimeoutNoDeadlockDetectionTest has no failures at CI because Ignite ignores IGNITE_TX_DEADLOCK_DETECTION_MAX_ITERS property set after it already set by previous tests (static field). But, when you run this test separately it fails because transaction will never become finished on timeout when deadlockDetection is dissabled. > TxRollbackOnTimeoutNoDeadlockDetectionTest fails at local run (not from the > suite) > -- > > Key: IGNITE-20141 > URL: https://issues.apache.org/jira/browse/IGNITE-20141 > Project: Ignite > Issue Type: Bug >Reporter: Anton Vinogradov >Assignee: Anton Vinogradov >Priority: Major > Fix For: 2.16 > > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-20141) TxRollbackOnTimeoutNoDeadlockDetectionTest fails at local run (not from the suite)
[ https://issues.apache.org/jira/browse/IGNITE-20141?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anton Vinogradov updated IGNITE-20141: -- Ignite Flags: Release Notes Required (was: Docs Required,Release Notes Required) > TxRollbackOnTimeoutNoDeadlockDetectionTest fails at local run (not from the > suite) > -- > > Key: IGNITE-20141 > URL: https://issues.apache.org/jira/browse/IGNITE-20141 > Project: Ignite > Issue Type: Bug >Reporter: Anton Vinogradov >Assignee: Anton Vinogradov >Priority: Major > Fix For: 2.16 > > Time Spent: 20m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-20141) Transaction will never become finished on timeout when deadlock detection is disabled.
[ https://issues.apache.org/jira/browse/IGNITE-20141?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anton Vinogradov updated IGNITE-20141: -- Summary: Transaction will never become finished on timeout when deadlock detection is disabled. (was: TxRollbackOnTimeoutNoDeadlockDetectionTest fails at local run (not from the suite)) > Transaction will never become finished on timeout when deadlock detection is > disabled. > -- > > Key: IGNITE-20141 > URL: https://issues.apache.org/jira/browse/IGNITE-20141 > Project: Ignite > Issue Type: Bug >Reporter: Anton Vinogradov >Assignee: Anton Vinogradov >Priority: Major > Fix For: 2.16 > > Time Spent: 20m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (IGNITE-20142) Introduce changes for JDK17 tests run
Petr Ivanov created IGNITE-20142: Summary: Introduce changes for JDK17 tests run Key: IGNITE-20142 URL: https://issues.apache.org/jira/browse/IGNITE-20142 Project: Ignite Issue Type: Improvement Reporter: Petr Ivanov Fix For: 2.16 Introduce several changes and extend current WAs for ability to run tests under JDK17. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-20142) Introduce changes for JDK17 tests run
[ https://issues.apache.org/jira/browse/IGNITE-20142?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Petr Ivanov updated IGNITE-20142: - Labels: ise (was: ) > Introduce changes for JDK17 tests run > - > > Key: IGNITE-20142 > URL: https://issues.apache.org/jira/browse/IGNITE-20142 > Project: Ignite > Issue Type: Improvement >Reporter: Petr Ivanov >Priority: Major > Labels: ise > Fix For: 2.16 > > > Introduce several changes and extend current WAs for ability to run tests > under JDK17. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (IGNITE-20143) GridIntList initial cleanup
Anton Vinogradov created IGNITE-20143: - Summary: GridIntList initial cleanup Key: IGNITE-20143 URL: https://issues.apache.org/jira/browse/IGNITE-20143 Project: Ignite Issue Type: Sub-task Reporter: Anton Vinogradov Assignee: Anton Vinogradov -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-20143) GridIntList initial cleanup
[ https://issues.apache.org/jira/browse/IGNITE-20143?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anton Vinogradov updated IGNITE-20143: -- Ignite Flags: (was: Docs Required,Release Notes Required) > GridIntList initial cleanup > --- > > Key: IGNITE-20143 > URL: https://issues.apache.org/jira/browse/IGNITE-20143 > Project: Ignite > Issue Type: Sub-task >Reporter: Anton Vinogradov >Assignee: Anton Vinogradov >Priority: Major > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-20143) GridIntList initial cleanup
[ https://issues.apache.org/jira/browse/IGNITE-20143?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anton Vinogradov updated IGNITE-20143: -- Fix Version/s: 2.16 > GridIntList initial cleanup > --- > > Key: IGNITE-20143 > URL: https://issues.apache.org/jira/browse/IGNITE-20143 > Project: Ignite > Issue Type: Sub-task >Reporter: Anton Vinogradov >Assignee: Anton Vinogradov >Priority: Major > Fix For: 2.16 > > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (IGNITE-20142) Introduce changes for JDK17 tests run
[ https://issues.apache.org/jira/browse/IGNITE-20142?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Petr Ivanov reassigned IGNITE-20142: Assignee: Petr Ivanov > Introduce changes for JDK17 tests run > - > > Key: IGNITE-20142 > URL: https://issues.apache.org/jira/browse/IGNITE-20142 > Project: Ignite > Issue Type: Improvement >Reporter: Petr Ivanov >Assignee: Petr Ivanov >Priority: Major > Labels: ise > Fix For: 2.16 > > > Introduce several changes and extend current WAs for ability to run tests > under JDK17. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-20143) GridIntList & GridLongList initial cleanup
[ https://issues.apache.org/jira/browse/IGNITE-20143?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anton Vinogradov updated IGNITE-20143: -- Summary: GridIntList & GridLongList initial cleanup (was: GridIntList initial cleanup) > GridIntList & GridLongList initial cleanup > -- > > Key: IGNITE-20143 > URL: https://issues.apache.org/jira/browse/IGNITE-20143 > Project: Ignite > Issue Type: Sub-task >Reporter: Anton Vinogradov >Assignee: Anton Vinogradov >Priority: Major > Fix For: 2.16 > > Time Spent: 10m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (IGNITE-20123) IgniteTxHandler initial cleanup
[ https://issues.apache.org/jira/browse/IGNITE-20123?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17750435#comment-17750435 ] Ignite TC Bot commented on IGNITE-20123: {panel:title=Branch: [pull/10869/head] Base: [master] : No blockers found!|borderStyle=dashed|borderColor=#ccc|titleBGColor=#D6F7C1}{panel} {panel:title=Branch: [pull/10869/head] Base: [master] : No new tests found!|borderStyle=dashed|borderColor=#ccc|titleBGColor=#F7D6C1}{panel} [TeamCity *--> Run :: All* Results|https://ci2.ignite.apache.org/viewLog.html?buildId=7279555&buildTypeId=IgniteTests24Java8_RunAll] > IgniteTxHandler initial cleanup > --- > > Key: IGNITE-20123 > URL: https://issues.apache.org/jira/browse/IGNITE-20123 > Project: Ignite > Issue Type: Sub-task >Reporter: Anton Vinogradov >Assignee: Anton Vinogradov >Priority: Major > Fix For: 2.16 > > Time Spent: 40m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (IGNITE-20112) IgniteTxImplicitSingleStateImpl initial cleanup
[ https://issues.apache.org/jira/browse/IGNITE-20112?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17750440#comment-17750440 ] Ignite TC Bot commented on IGNITE-20112: {panel:title=Branch: [pull/10867/head] Base: [master] : No blockers found!|borderStyle=dashed|borderColor=#ccc|titleBGColor=#D6F7C1}{panel} {panel:title=Branch: [pull/10867/head] Base: [master] : No new tests found!|borderStyle=dashed|borderColor=#ccc|titleBGColor=#F7D6C1}{panel} [TeamCity *--> Run :: All* Results|https://ci2.ignite.apache.org/viewLog.html?buildId=7279783&buildTypeId=IgniteTests24Java8_RunAll] > IgniteTxImplicitSingleStateImpl initial cleanup > --- > > Key: IGNITE-20112 > URL: https://issues.apache.org/jira/browse/IGNITE-20112 > Project: Ignite > Issue Type: Sub-task >Reporter: Anton Vinogradov >Assignee: Anton Vinogradov >Priority: Major > Fix For: 2.16 > > Time Spent: 20m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (IGNITE-19904) Assertion in defragmentation
[ https://issues.apache.org/jira/browse/IGNITE-19904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17750461#comment-17750461 ] Ignite TC Bot commented on IGNITE-19904: {panel:title=Branch: [pull/10866/head] Base: [master] : No blockers found!|borderStyle=dashed|borderColor=#ccc|titleBGColor=#D6F7C1}{panel} {panel:title=Branch: [pull/10866/head] Base: [master] : No new tests found!|borderStyle=dashed|borderColor=#ccc|titleBGColor=#F7D6C1}{panel} [TeamCity *--> Run :: All* Results|https://ci2.ignite.apache.org/viewLog.html?buildId=7280731&buildTypeId=IgniteTests24Java8_RunAll] > Assertion in defragmentation > > > Key: IGNITE-19904 > URL: https://issues.apache.org/jira/browse/IGNITE-19904 > Project: Ignite > Issue Type: Bug >Affects Versions: 2.12 >Reporter: Vladimir Steshin >Assignee: Vladimir Steshin >Priority: Major > Labels: ise > Fix For: 2.16 > > Attachments: default-config.xml, failure2.16_with_thread_dump.log, > failure_with_root_npe_cause.log, ignite.log, jvm.opts > > Time Spent: 50m > Remaining Estimate: 0h > > Defragmentaion fails with: > {code:java} > java.lang.AssertionError: Invalid state. Type is 0! pageId = 0001000d00024cbf > at > org.apache.ignite.internal.processors.cache.persistence.pagemem.PageMemoryImpl.copyPageForCheckpoint(PageMemoryImpl.java:1359) > ~[ignite-core-2.16.0-SNAPSHOT.jar:2.16.0-SNAPSHOT] > at > org.apache.ignite.internal.processors.cache.persistence.pagemem.PageMemoryImpl.checkpointWritePage(PageMemoryImpl.java:1277) > ~[ignite-core-2.16.0-SNAPSHOT.jar:2.16.0-SNAPSHOT] > at > org.apache.ignite.internal.processors.cache.persistence.checkpoint.CheckpointPagesWriter.writePages(CheckpointPagesWriter.java:208) > ~[ignite-core-2.16.0-SNAPSHOT.jar:2.16.0-SNAPSHOT] > at > org.apache.ignite.internal.processors.cache.persistence.checkpoint.CheckpointPagesWriter.run(CheckpointPagesWriter.java:150) > ~[ignite-core-2.16.0-SNAPSHOT.jar:2.16.0-SNAPSHOT] > {code} > Difficult to write a test. Can't reproduce on my computers :(. Flackly > appears on a server (4 core x 4 cpu) with 100G of the test cache data and > million+ pages to checkpoint during defragmentation. More often, this occurs > with pageSize 1024 (to produce more pages). > Regarding my diagnostic build, I suppose that a fresh, empty page is caught > in defragmentation. Here is a page dump with test-expented PAGE_OVERHEAD > (=64) and same error a bit before copyPageForCheckpoint(): > {code:java} > org.apache.ignite.IgniteException: Wrong page type in checkpointWritePage1. > Page: Data region = 'defragPartitionsDataRegion'. > FullPageId [pageId=281878703760205, effectivePageId=403727049549, > grpId=-1368047378]. > PageDump = page_id: 281878703760205, rel_id: 48603, cache_id: -1368047378, > pin: 0, lock: 65536, tmp_buf: 72057594037927935, test_val: 1. data_hex: > 0
[jira] [Created] (IGNITE-20145) Update maven surefire version
Nikita Amelchev created IGNITE-20145: Summary: Update maven surefire version Key: IGNITE-20145 URL: https://issues.apache.org/jira/browse/IGNITE-20145 Project: Ignite Issue Type: Task Reporter: Nikita Amelchev Assignee: Nikita Amelchev There are many useful updates in new versions, such as [SUREFIRE-1426|https://issues.apache.org/jira/browse/SUREFIRE-1426]. This fixes the exit code in case of a fork failure and is needed for proper failure handling in CI tools. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (IGNITE-20144) Update maven surefire version
Nikita Amelchev created IGNITE-20144: Summary: Update maven surefire version Key: IGNITE-20144 URL: https://issues.apache.org/jira/browse/IGNITE-20144 Project: Ignite Issue Type: Task Reporter: Nikita Amelchev Assignee: Nikita Amelchev There are many useful updates in new versions, such as [SUREFIRE-1426|https://issues.apache.org/jira/browse/SUREFIRE-1426]. This fixes the exit code in case of a fork failure and is needed for proper failure handling in CI tools. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-20145) Update maven surefire version
[ https://issues.apache.org/jira/browse/IGNITE-20145?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nikita Amelchev updated IGNITE-20145: - Labels: ise (was: ) > Update maven surefire version > - > > Key: IGNITE-20145 > URL: https://issues.apache.org/jira/browse/IGNITE-20145 > Project: Ignite > Issue Type: Task >Reporter: Nikita Amelchev >Assignee: Nikita Amelchev >Priority: Major > Labels: ise > > There are many useful updates in new versions, such as > [SUREFIRE-1426|https://issues.apache.org/jira/browse/SUREFIRE-1426]. This > fixes the exit code in case of a fork failure and is needed for proper > failure handling in CI tools. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (IGNITE-19904) Assertion in defragmentation
[ https://issues.apache.org/jira/browse/IGNITE-19904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17750481#comment-17750481 ] Ignite TC Bot commented on IGNITE-19904: {panel:title=Branch: [pull/10866/head] Base: [master] : No blockers found!|borderStyle=dashed|borderColor=#ccc|titleBGColor=#D6F7C1}{panel} {panel:title=Branch: [pull/10866/head] Base: [master] : No new tests found!|borderStyle=dashed|borderColor=#ccc|titleBGColor=#F7D6C1}{panel} [TeamCity *--> Run :: All* Results|https://ci2.ignite.apache.org/viewLog.html?buildId=7280731&buildTypeId=IgniteTests24Java8_RunAll] > Assertion in defragmentation > > > Key: IGNITE-19904 > URL: https://issues.apache.org/jira/browse/IGNITE-19904 > Project: Ignite > Issue Type: Bug >Affects Versions: 2.12 >Reporter: Vladimir Steshin >Assignee: Vladimir Steshin >Priority: Major > Labels: ise > Fix For: 2.16 > > Attachments: default-config.xml, failure2.16_with_thread_dump.log, > failure_with_root_npe_cause.log, ignite.log, jvm.opts > > Time Spent: 50m > Remaining Estimate: 0h > > Defragmentaion fails with: > {code:java} > java.lang.AssertionError: Invalid state. Type is 0! pageId = 0001000d00024cbf > at > org.apache.ignite.internal.processors.cache.persistence.pagemem.PageMemoryImpl.copyPageForCheckpoint(PageMemoryImpl.java:1359) > ~[ignite-core-2.16.0-SNAPSHOT.jar:2.16.0-SNAPSHOT] > at > org.apache.ignite.internal.processors.cache.persistence.pagemem.PageMemoryImpl.checkpointWritePage(PageMemoryImpl.java:1277) > ~[ignite-core-2.16.0-SNAPSHOT.jar:2.16.0-SNAPSHOT] > at > org.apache.ignite.internal.processors.cache.persistence.checkpoint.CheckpointPagesWriter.writePages(CheckpointPagesWriter.java:208) > ~[ignite-core-2.16.0-SNAPSHOT.jar:2.16.0-SNAPSHOT] > at > org.apache.ignite.internal.processors.cache.persistence.checkpoint.CheckpointPagesWriter.run(CheckpointPagesWriter.java:150) > ~[ignite-core-2.16.0-SNAPSHOT.jar:2.16.0-SNAPSHOT] > {code} > Difficult to write a test. Can't reproduce on my computers :(. Flackly > appears on a server (4 core x 4 cpu) with 100G of the test cache data and > million+ pages to checkpoint during defragmentation. More often, this occurs > with pageSize 1024 (to produce more pages). > Regarding my diagnostic build, I suppose that a fresh, empty page is caught > in defragmentation. Here is a page dump with test-expented PAGE_OVERHEAD > (=64) and same error a bit before copyPageForCheckpoint(): > {code:java} > org.apache.ignite.IgniteException: Wrong page type in checkpointWritePage1. > Page: Data region = 'defragPartitionsDataRegion'. > FullPageId [pageId=281878703760205, effectivePageId=403727049549, > grpId=-1368047378]. > PageDump = page_id: 281878703760205, rel_id: 48603, cache_id: -1368047378, > pin: 0, lock: 65536, tmp_buf: 72057594037927935, test_val: 1. data_hex: > 0
[jira] [Commented] (IGNITE-18302) ignite-spring-sessions: IgniteSession serialization drags its parent class
[ https://issues.apache.org/jira/browse/IGNITE-18302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17750556#comment-17750556 ] Andrey Novikov commented on IGNITE-18302: - Let's move IgniteSession class to upper level as it will be used as value object in SQL. https://github.com/apache/ignite-extensions/pull/222#discussion_r1281232708 > ignite-spring-sessions: IgniteSession serialization drags its parent class > -- > > Key: IGNITE-18302 > URL: https://issues.apache.org/jira/browse/IGNITE-18302 > Project: Ignite > Issue Type: Bug > Components: extensions >Reporter: Alexandr Shapkin >Assignee: Alexandr Shapkin >Priority: Major > > In short, there is a bug in ignite-spring-session-ext implementation. > We store IgniteIndexedSessionRepository${{{}IgniteSession{}}} in the cache, > but it’s an internal non-static class, having a reference to the parent > [\{{{}IgniteIndexedSessionRepository{}}}] indirectly. > Hence, during the serialization Ignite also writes {{{}name=this$0, > type=Object, fieldId=0xCBDD23AA (-874699862){}}}, which is the reference to > {{{}IgniteIndexedSessionRepository{}}}. That leads to the following issues: > * we are serializing and saving internal utility data, like}} Ignite > ignite{}}}, {{private IndexResolver indexResolver}} etc > * one of the IgniteIndexedSessionRepository’s fields is IgniteCache itself - > {{IgniteCache sessions}} that basically keeps every > session so far leading to a StackOverflow error after some time. > > {code:java} > [2022-11-25T17:27:29,268][ERROR][sys-stripe-0-#1%USERS_IGNITE%][GridCacheIoManager] > Failed processing message [senderId=0f0ca915-d6cd-4580-92a3-1fbc3d2a5722, > msg=GridNearSingleGetResponse [futId=1669397231378, res=-547701325, > topVer=null, err=null, flags=0]] 2java.lang.StackOverflowError: null 3 at > org.apache.ignite.internal.marshaller.optimized.OptimizedMarshallerUtils.descriptorFromCache(OptimizedMarshallerUtils.java:328) > ~[ignite-core-8.8.22.jar:8.8.22] 4 at > org.apache.ignite.internal.marshaller.optimized.OptimizedMarshallerUtils.classDescriptor(OptimizedMarshallerUtils.java:273) > ~[ignite-core-8.8.22.jar:8.8.22] 5 at > org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:354) > ~[ignite-core-8.8.22.jar:8.8.22] 6 at > org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:211) > ~[ignite-core-8.8.22.jar:8.8.22] 7 at > java.io.ObjectInputStream.readObject(ObjectInputStream.java:480) ~[?:?] 8 at > java.io.ObjectInputStream.readObject(ObjectInputStream.java:447) ~[?:?] 9 at > org.apache.ignite.internal.processors.cache.GridCacheProxyImpl.readExternal(GridCacheProxyImpl.java:1662) > ~[ignite-core-8.8.22.jar:8.8.22] 10 at > org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readExternalizable(OptimizedObjectInputStream.java:569) > ~[ignite-core-8.8.22.jar:8.8.22] 11 at > org.apache.ignite.internal.marshaller.optimized.OptimizedClassDescriptor.read(OptimizedClassDescriptor.java:979) > ~[ignite-core-8.8.22.jar:8.8.22] 12 at > org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:359) > ~[ignite-core-8.8.22.jar:8.8.22] 13 at > org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:211) > ~[ignite-core-8.8.22.jar:8.8.22] 14 at > java.io.ObjectInputStream.readObject(ObjectInputStream.java:480) ~[?:?] 15 at > java.io.ObjectInputStream.readObject(ObjectInputStream.java:447) ~[?:?] 16... > 17[2022-11-25T17:27:29,276][ERROR][sys-stripe-0-#1%USERS_IGNITE%][] Critical > system error detected. Will be handled accordingly to configured handler > [hnd=StopNodeOrHaltFailureHandler [tryStop=false, timeout=0, > super=AbstractFailureHandler [ignoredFailureTypes=UnmodifiableSet > [SYSTEM_WORKER_BLOCKED, SYSTEM_CRITICAL_OPERATION_TIMEOUT]]], > failureCtx=FailureContext [type=CRITICAL_ERROR, > err=java.lang.StackOverflowError]]{code} -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (IGNITE-20146) Add improvements to the catalog associated with the DistributionZone
Kirill Tkalenko created IGNITE-20146: Summary: Add improvements to the catalog associated with the DistributionZone Key: IGNITE-20146 URL: https://issues.apache.org/jira/browse/IGNITE-20146 Project: Ignite Issue Type: Improvement Reporter: Kirill Tkalenko Assignee: Kirill Tkalenko Fix For: 3.0.0-beta2 During the implementation of IGNITE-20114, it was discovered: * Filter is not checked when creating/changing a zone; * It is not possible to specify storage when creating/changing a zone; * There is no field validation as in the configuration; * Perhaps there are other differences other than configuration. Some code refactoring in related classes may be needed. -- This message was sent by Atlassian Jira (v8.20.10#820010)