[jira] [Created] (IGNITE-20133) Compute hashes for integral/decimal columns in a stable way

2023-08-02 Thread Roman Puchkovskiy (Jira)
Roman Puchkovskiy created IGNITE-20133:
--

 Summary: Compute hashes for integral/decimal columns in a stable 
way
 Key: IGNITE-20133
 URL: https://issues.apache.org/jira/browse/IGNITE-20133
 Project: Ignite
  Issue Type: Improvement
Reporter: Roman Puchkovskiy
 Fix For: 3.0.0-beta2


The idea is to make hash computation for integral and decimal types satisfy the 
following property: if a column type is changed from an integral to a decimal 
type, the hashes for values that are already stored remain the same.

This will allow us to permit chaning type (integral -> decimal and decimal -> 
longer decimal) of a column that is included in a HASH index.

A hash that has this property is the following function: 
hash(val.toString(TRIM_TRAILING_ZEROS)). For instance, for 1 it will be 
hash("1"), for 1.000 it will also be hash("1"), but for 1.23 it will give 
hash("1.23").



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Created] (IGNITE-20134) Only allow changing type of indexed column when indexed values representation remains the same

2023-08-02 Thread Roman Puchkovskiy (Jira)
Roman Puchkovskiy created IGNITE-20134:
--

 Summary: Only allow changing type of indexed column when indexed 
values representation remains the same
 Key: IGNITE-20134
 URL: https://issues.apache.org/jira/browse/IGNITE-20134
 Project: Ignite
  Issue Type: Improvement
Reporter: Roman Puchkovskiy
 Fix For: 3.0.0-beta2


When an attempt to change type of a column that is included in an index is 
made, this should only be permitted if the representation of the column values 
in the index will remain unchanged (and, hence, index rebuild will not be 
needed).

The following changes are acceptable:

* integral->integral (as integral types are represented as varints)
* integral->decimal (with enough precision) and float->double for SORTED 
indices where the ordering remains the same
* integral->decimal and decimal->decimal (with enough precision) for HASH 
indices [requires IGNITE-20133]



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Updated] (IGNITE-19893) Sql. Add plan checks to BaseIndexDataTypeTest

2023-08-02 Thread Evgeny Stanilovsky (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-19893?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Evgeny Stanilovsky updated IGNITE-19893:

Ignite Flags:   (was: Docs Required,Release Notes Required)

> Sql. Add plan checks to BaseIndexDataTypeTest
> -
>
> Key: IGNITE-19893
> URL: https://issues.apache.org/jira/browse/IGNITE-19893
> Project: Ignite
>  Issue Type: Improvement
>  Components: sql
>Affects Versions: 3.0.0-beta1
>Reporter: Maksim Zhuravkov
>Assignee: Maksim Zhuravkov
>Priority: Minor
>  Labels: ignite-3
>
> Add checks to BaseIndexDataTypeTest to ensure that index lookups/scans are 
> used where they are necessary.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Updated] (IGNITE-20134) Only allow changing type of indexed column when indexed values representation remains the same

2023-08-02 Thread Roman Puchkovskiy (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-20134?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Roman Puchkovskiy updated IGNITE-20134:
---
Description: 
When an attempt to change type of a column that is included in an index is 
made, this should only be permitted if the representation of the column values 
in the index will remain unchanged (and, hence, index rebuild will not be 
needed).

The following changes are acceptable:

* integral->integral (as integral types are represented as varints)
* integral->decimal (with enough precision) and float->double for SORTED 
indices where the ordering remains the same
* integral->decimal and decimal->decimal (with enough precision) for HASH 
indices (requires IGNITE-20133)

  was:
When an attempt to change type of a column that is included in an index is 
made, this should only be permitted if the representation of the column values 
in the index will remain unchanged (and, hence, index rebuild will not be 
needed).

The following changes are acceptable:

* integral->integral (as integral types are represented as varints)
* integral->decimal (with enough precision) and float->double for SORTED 
indices where the ordering remains the same
* integral->decimal and decimal->decimal (with enough precision) for HASH 
indices [requires IGNITE-20133]


> Only allow changing type of indexed column when indexed values representation 
> remains the same
> --
>
> Key: IGNITE-20134
> URL: https://issues.apache.org/jira/browse/IGNITE-20134
> Project: Ignite
>  Issue Type: Improvement
>Reporter: Roman Puchkovskiy
>Priority: Major
>  Labels: ignite-3
> Fix For: 3.0.0-beta2
>
>
> When an attempt to change type of a column that is included in an index is 
> made, this should only be permitted if the representation of the column 
> values in the index will remain unchanged (and, hence, index rebuild will not 
> be needed).
> The following changes are acceptable:
> * integral->integral (as integral types are represented as varints)
> * integral->decimal (with enough precision) and float->double for SORTED 
> indices where the ordering remains the same
> * integral->decimal and decimal->decimal (with enough precision) for HASH 
> indices (requires IGNITE-20133)



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Commented] (IGNITE-19947) Sql. Fix CAST(n AS DECIMAL(precision, scale) behaviour for scale > precision.

2023-08-02 Thread Evgeny Stanilovsky (Jira)


[ 
https://issues.apache.org/jira/browse/IGNITE-19947?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17750153#comment-17750153
 ] 

Evgeny Stanilovsky commented on IGNITE-19947:
-

let`s fill appropriate issue into Calcite after and fix it too.

> Sql. Fix CAST(n AS DECIMAL(precision, scale) behaviour for scale > precision.
> -
>
> Key: IGNITE-19947
> URL: https://issues.apache.org/jira/browse/IGNITE-19947
> Project: Ignite
>  Issue Type: Bug
>  Components: sql
>Affects Versions: 3.0.0-beta1
>Reporter: Maksim Zhuravkov
>Assignee: Evgeny Stanilovsky
>Priority: Minor
>  Labels: ignite-3
>
> CAST(n AS DECIMAL(p, s) always returns `numeric field overflow` error when s 
> > p which is not correct, in such case we should also check `n`.
> Example:
> {code:java}
> SELECT CAST (? AS DECIMAL(1,4)) ? = 0.0001
> # PG returns 0.0001
> # Ignite 3 returns numeric field overflow
> {code}
> Query with a literal works because calcite performs constant folding and 
> removes a CAST call:
> {code:java}
> SELECT CAST (0.0001 AS DECIMAL(1,4))
> # Plan
> IgniteValues(tuples=[[{ 0.0001 }]]), id = 8
> {code}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Updated] (IGNITE-19506) Use data nodes from DistributionZoneManager with a causality token instead of BaselineManager#nodes

2023-08-02 Thread Alexander Lapin (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-19506?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alexander Lapin updated IGNITE-19506:
-
Ignite Flags:   (was: Docs Required,Release Notes Required)

> Use data nodes from DistributionZoneManager with a causality token instead of 
> BaselineManager#nodes
> ---
>
> Key: IGNITE-19506
> URL: https://issues.apache.org/jira/browse/IGNITE-19506
> Project: Ignite
>  Issue Type: Bug
>Reporter: Sergey Uttsel
>Assignee: Sergey Uttsel
>Priority: Major
>  Labels: ignite-3
> Fix For: 3.0.0-beta2
>
> Attachments: Causality data nodes.docx
>
>  Time Spent: 7h
>  Remaining Estimate: 0h
>
> h3. *Motivation*
> Need to use data nodes from DistributionZoneManager instead of 
> BaselineManager#nodes in all places except of in-memory raft 
> (TableManager#calculateAssignments)
> We need to get data nodes consistently so we need to use revision of 
> configuration events and a meta storage events as causality token.
> Description of causality data nodes algorithm is attached.
> h3. *Definition of Done*
> Implement method DistributionZoneManager#dataNodes to obtaining data nodes 
> from zone manager with causality token.
> Use this method instead of BaselineManager#nodes.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Assigned] (IGNITE-19904) Assertion in defragmentation

2023-08-02 Thread Vladimir Steshin (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-19904?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vladimir Steshin reassigned IGNITE-19904:
-

Assignee: Vladimir Steshin

> Assertion in defragmentation
> 
>
> Key: IGNITE-19904
> URL: https://issues.apache.org/jira/browse/IGNITE-19904
> Project: Ignite
>  Issue Type: Bug
>Affects Versions: 2.12
>Reporter: Vladimir Steshin
>Assignee: Vladimir Steshin
>Priority: Major
>  Labels: ise
> Attachments: default-config.xml, failure2.16_with_thread_dump.log, 
> failure_with_root_npe_cause.log, ignite.log, jvm.opts
>
>  Time Spent: 0.5h
>  Remaining Estimate: 0h
>
> Defragmentaion fails with:
> {code:java}
> java.lang.AssertionError: Invalid state. Type is 0! pageId = 0001000d00024cbf
>   at 
> org.apache.ignite.internal.processors.cache.persistence.pagemem.PageMemoryImpl.copyPageForCheckpoint(PageMemoryImpl.java:1359)
>  ~[ignite-core-2.16.0-SNAPSHOT.jar:2.16.0-SNAPSHOT]
>   at 
> org.apache.ignite.internal.processors.cache.persistence.pagemem.PageMemoryImpl.checkpointWritePage(PageMemoryImpl.java:1277)
>  ~[ignite-core-2.16.0-SNAPSHOT.jar:2.16.0-SNAPSHOT]
>   at 
> org.apache.ignite.internal.processors.cache.persistence.checkpoint.CheckpointPagesWriter.writePages(CheckpointPagesWriter.java:208)
>  ~[ignite-core-2.16.0-SNAPSHOT.jar:2.16.0-SNAPSHOT]
>   at 
> org.apache.ignite.internal.processors.cache.persistence.checkpoint.CheckpointPagesWriter.run(CheckpointPagesWriter.java:150)
>  ~[ignite-core-2.16.0-SNAPSHOT.jar:2.16.0-SNAPSHOT]
> {code}
> Difficult to write a test. Can't reproduce on my computers :(. Flackly 
> appears on a server (4 core x 4 cpu) with 100G of the test cache data and 
> million+ pages to checkpoint during defragmentation. More often, this occurs 
> with pageSize 1024 (to produce more pages).
> Regarding my diagnostic build, I suppose that a fresh, empty page is caught 
> in defragmentation. Here is a page dump with test-expented PAGE_OVERHEAD 
> (=64) and same error a bit before copyPageForCheckpoint():
> {code:java}
> org.apache.ignite.IgniteException: Wrong page type in checkpointWritePage1. 
> Page: Data region = 'defragPartitionsDataRegion'.
>  FullPageId [pageId=281878703760205, effectivePageId=403727049549, 
> grpId=-1368047378].
>  PageDump = page_id: 281878703760205, rel_id: 48603, cache_id: -1368047378, 
> pin: 0, lock: 65536, tmp_buf: 72057594037927935, test_val: 1. data_hex: 
> 
>   at 
> org.apache.ignite.internal.processors.cache.persistence.pagemem.PageMemoryImpl.checkpointWritePage(PageMemoryImpl.java:1240)
>  ~[ignite-core-2.16.0-SNAPSHOT.jar:2.16.0-SNAPSHOT]
>   at 
> org.apache.ignite.internal.processors.cache.persistence.check

[jira] [Updated] (IGNITE-19904) Assertion in defragmentation

2023-08-02 Thread Vladimir Steshin (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-19904?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vladimir Steshin updated IGNITE-19904:
--
Fix Version/s: 2.16

> Assertion in defragmentation
> 
>
> Key: IGNITE-19904
> URL: https://issues.apache.org/jira/browse/IGNITE-19904
> Project: Ignite
>  Issue Type: Bug
>Affects Versions: 2.12
>Reporter: Vladimir Steshin
>Assignee: Vladimir Steshin
>Priority: Major
>  Labels: ise
> Fix For: 2.16
>
> Attachments: default-config.xml, failure2.16_with_thread_dump.log, 
> failure_with_root_npe_cause.log, ignite.log, jvm.opts
>
>  Time Spent: 0.5h
>  Remaining Estimate: 0h
>
> Defragmentaion fails with:
> {code:java}
> java.lang.AssertionError: Invalid state. Type is 0! pageId = 0001000d00024cbf
>   at 
> org.apache.ignite.internal.processors.cache.persistence.pagemem.PageMemoryImpl.copyPageForCheckpoint(PageMemoryImpl.java:1359)
>  ~[ignite-core-2.16.0-SNAPSHOT.jar:2.16.0-SNAPSHOT]
>   at 
> org.apache.ignite.internal.processors.cache.persistence.pagemem.PageMemoryImpl.checkpointWritePage(PageMemoryImpl.java:1277)
>  ~[ignite-core-2.16.0-SNAPSHOT.jar:2.16.0-SNAPSHOT]
>   at 
> org.apache.ignite.internal.processors.cache.persistence.checkpoint.CheckpointPagesWriter.writePages(CheckpointPagesWriter.java:208)
>  ~[ignite-core-2.16.0-SNAPSHOT.jar:2.16.0-SNAPSHOT]
>   at 
> org.apache.ignite.internal.processors.cache.persistence.checkpoint.CheckpointPagesWriter.run(CheckpointPagesWriter.java:150)
>  ~[ignite-core-2.16.0-SNAPSHOT.jar:2.16.0-SNAPSHOT]
> {code}
> Difficult to write a test. Can't reproduce on my computers :(. Flackly 
> appears on a server (4 core x 4 cpu) with 100G of the test cache data and 
> million+ pages to checkpoint during defragmentation. More often, this occurs 
> with pageSize 1024 (to produce more pages).
> Regarding my diagnostic build, I suppose that a fresh, empty page is caught 
> in defragmentation. Here is a page dump with test-expented PAGE_OVERHEAD 
> (=64) and same error a bit before copyPageForCheckpoint():
> {code:java}
> org.apache.ignite.IgniteException: Wrong page type in checkpointWritePage1. 
> Page: Data region = 'defragPartitionsDataRegion'.
>  FullPageId [pageId=281878703760205, effectivePageId=403727049549, 
> grpId=-1368047378].
>  PageDump = page_id: 281878703760205, rel_id: 48603, cache_id: -1368047378, 
> pin: 0, lock: 65536, tmp_buf: 72057594037927935, test_val: 1. data_hex: 
> 
>   at 
> org.apache.ignite.internal.processors.cache.persistence.pagemem.PageMemoryImpl.checkpointWritePage(PageMemoryImpl.java:1240)
>  ~[ignite-core-2.16.0-SNAPSHOT.jar:2.16.0-SNAPSHOT]
>   at 
> org.apache.ignite.internal.processors.cache.p

[jira] [Assigned] (IGNITE-19922) Gradle checkstyle tasks are greedy

2023-08-02 Thread Jira


 [ 
https://issues.apache.org/jira/browse/IGNITE-19922?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

 Kirill Sizov reassigned IGNITE-19922:
--

Assignee:  Kirill Sizov

> Gradle checkstyle tasks are greedy
> --
>
> Key: IGNITE-19922
> URL: https://issues.apache.org/jira/browse/IGNITE-19922
> Project: Ignite
>  Issue Type: New Feature
>Reporter: Mikhail Pochatkin
>Assignee:  Kirill Sizov
>Priority: Major
>  Labels: ignite-3
> Attachments: image-2023-07-06-11-18-40-515.png, screenshot-1.png
>
>
> This is memory consumption during {{gradlew checkstyleMain}}  execution - 
> goes from ~10 GB to 30. All CPU cores are also at 100%. This causes chrome 
> tabs to unload and overall stress on the system. 
> Also, RAM usage does not go down after this command unless I kill/stop Gradle 
> daemons
> !image-2023-07-06-11-18-40-515.png!



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Resolved] (IGNITE-19922) Gradle checkstyle tasks are greedy

2023-08-02 Thread Jira


 [ 
https://issues.apache.org/jira/browse/IGNITE-19922?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

 Kirill Sizov resolved IGNITE-19922.

Resolution: Won't Fix

Command line parameter helped.

Closing this Jira.

 

> Gradle checkstyle tasks are greedy
> --
>
> Key: IGNITE-19922
> URL: https://issues.apache.org/jira/browse/IGNITE-19922
> Project: Ignite
>  Issue Type: New Feature
>Reporter: Mikhail Pochatkin
>Assignee:  Kirill Sizov
>Priority: Major
>  Labels: ignite-3
> Attachments: image-2023-07-06-11-18-40-515.png, screenshot-1.png
>
>
> This is memory consumption during {{gradlew checkstyleMain}}  execution - 
> goes from ~10 GB to 30. All CPU cores are also at 100%. This causes chrome 
> tabs to unload and overall stress on the system. 
> Also, RAM usage does not go down after this command unless I kill/stop Gradle 
> daemons
> !image-2023-07-06-11-18-40-515.png!



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Created] (IGNITE-20135) NO_INDEX hint for calcite engine.

2023-08-02 Thread Vladimir Steshin (Jira)
Vladimir Steshin created IGNITE-20135:
-

 Summary: NO_INDEX hint for calcite engine.
 Key: IGNITE-20135
 URL: https://issues.apache.org/jira/browse/IGNITE-20135
 Project: Ignite
  Issue Type: Bug
Reporter: Vladimir Steshin
Assignee: Vladimir Steshin


As part of hints for Calcite, we could try to implement simple hint like 
NO_INDEX. The main goal is to take deeper look into current hints 
implementation in Calcite and find a way to work with them.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Created] (IGNITE-20136) INDEX hint for calcite engine.

2023-08-02 Thread Vladimir Steshin (Jira)
Vladimir Steshin created IGNITE-20136:
-

 Summary: INDEX hint for calcite engine.
 Key: IGNITE-20136
 URL: https://issues.apache.org/jira/browse/IGNITE-20136
 Project: Ignite
  Issue Type: Bug
Reporter: Vladimir Steshin






--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Updated] (IGNITE-20136) INDEX hint for calcite engine.

2023-08-02 Thread Vladimir Steshin (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-20136?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vladimir Steshin updated IGNITE-20136:
--
Description: As part of hints for Calcite, we could try to implement simple 
hint like INDEX. The main goal is to take deeper look into current hints 
implementation in Calcite and find a way to work with them.

> INDEX hint for calcite engine.
> --
>
> Key: IGNITE-20136
> URL: https://issues.apache.org/jira/browse/IGNITE-20136
> Project: Ignite
>  Issue Type: Bug
>Reporter: Vladimir Steshin
>Priority: Major
>
> As part of hints for Calcite, we could try to implement simple hint like 
> INDEX. The main goal is to take deeper look into current hints implementation 
> in Calcite and find a way to work with them.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Updated] (IGNITE-20135) NO_INDEX hint for calcite engine.

2023-08-02 Thread Vladimir Steshin (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-20135?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vladimir Steshin updated IGNITE-20135:
--
Labels: ise  (was: )

> NO_INDEX hint for calcite engine.
> -
>
> Key: IGNITE-20135
> URL: https://issues.apache.org/jira/browse/IGNITE-20135
> Project: Ignite
>  Issue Type: Bug
>Reporter: Vladimir Steshin
>Assignee: Vladimir Steshin
>Priority: Major
>  Labels: ise
>
> As part of hints for Calcite, we could try to implement simple hint like 
> NO_INDEX. The main goal is to take deeper look into current hints 
> implementation in Calcite and find a way to work with them.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Assigned] (IGNITE-20136) INDEX hint for calcite engine.

2023-08-02 Thread Vladimir Steshin (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-20136?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vladimir Steshin reassigned IGNITE-20136:
-

Assignee: Vladimir Steshin

> INDEX hint for calcite engine.
> --
>
> Key: IGNITE-20136
> URL: https://issues.apache.org/jira/browse/IGNITE-20136
> Project: Ignite
>  Issue Type: Bug
>Reporter: Vladimir Steshin
>Assignee: Vladimir Steshin
>Priority: Major
>  Labels: ise
>
> As part of hints for Calcite, we could try to implement simple hint like 
> INDEX. The main goal is to take deeper look into current hints implementation 
> in Calcite and find a way to work with them.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Updated] (IGNITE-20136) INDEX hint for calcite engine.

2023-08-02 Thread Vladimir Steshin (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-20136?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vladimir Steshin updated IGNITE-20136:
--
Labels: ise  (was: )

> INDEX hint for calcite engine.
> --
>
> Key: IGNITE-20136
> URL: https://issues.apache.org/jira/browse/IGNITE-20136
> Project: Ignite
>  Issue Type: Bug
>Reporter: Vladimir Steshin
>Priority: Major
>  Labels: ise
>
> As part of hints for Calcite, we could try to implement simple hint like 
> INDEX. The main goal is to take deeper look into current hints implementation 
> in Calcite and find a way to work with them.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Updated] (IGNITE-20137) testOneNodeRestartWithGap is flaky

2023-08-02 Thread Alexey Scherbakov (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-20137?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alexey Scherbakov updated IGNITE-20137:
---
Labels: ignite-3  (was: )

> testOneNodeRestartWithGap is flaky
> --
>
> Key: IGNITE-20137
> URL: https://issues.apache.org/jira/browse/IGNITE-20137
> Project: Ignite
>  Issue Type: Bug
>Affects Versions: 3.0
>Reporter: Alexey Scherbakov
>Priority: Major
>  Labels: ignite-3
> Fix For: 3.0
>
>
> Looks like sometime the stopped node is not removed from the topology



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Created] (IGNITE-20137) testOneNodeRestartWithGap is flaky

2023-08-02 Thread Alexey Scherbakov (Jira)
Alexey Scherbakov created IGNITE-20137:
--

 Summary: testOneNodeRestartWithGap is flaky
 Key: IGNITE-20137
 URL: https://issues.apache.org/jira/browse/IGNITE-20137
 Project: Ignite
  Issue Type: Bug
Affects Versions: 3.0
Reporter: Alexey Scherbakov
 Fix For: 3.0


Looks like sometime the stopped node is not removed from the topology



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Commented] (IGNITE-15927) Implement one phase commit

2023-08-02 Thread Alexander Lapin (Jira)


[ 
https://issues.apache.org/jira/browse/IGNITE-15927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17750286#comment-17750286
 ] 

Alexander Lapin commented on IGNITE-15927:
--

[~ascherbakov] lgtm from my side.

> Implement one phase commit
> --
>
> Key: IGNITE-15927
> URL: https://issues.apache.org/jira/browse/IGNITE-15927
> Project: Ignite
>  Issue Type: Improvement
>Reporter: Alexey Scherbakov
>Assignee: Alexey Scherbakov
>Priority: Major
>  Labels: ignite-3
>  Time Spent: 2h 50m
>  Remaining Estimate: 0h
>
> If all keys in the implicit transaction belong to a same partition in can be 
> committed in one round-trip.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Updated] (IGNITE-15927) Implement one phase commit

2023-08-02 Thread Alexander Lapin (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-15927?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alexander Lapin updated IGNITE-15927:
-
Reviewer: Alexander Lapin

> Implement one phase commit
> --
>
> Key: IGNITE-15927
> URL: https://issues.apache.org/jira/browse/IGNITE-15927
> Project: Ignite
>  Issue Type: Improvement
>Reporter: Alexey Scherbakov
>Assignee: Alexey Scherbakov
>Priority: Major
>  Labels: ignite-3
>  Time Spent: 2h 50m
>  Remaining Estimate: 0h
>
> If all keys in the implicit transaction belong to a same partition in can be 
> committed in one round-trip.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Created] (IGNITE-20138) PageMemory should use actual dataregion's checkpointer.

2023-08-02 Thread Vladimir Steshin (Jira)
Vladimir Steshin created IGNITE-20138:
-

 Summary: PageMemory should use actual dataregion's checkpointer.
 Key: IGNITE-20138
 URL: https://issues.apache.org/jira/browse/IGNITE-20138
 Project: Ignite
  Issue Type: Improvement
Reporter: Vladimir Steshin


We create PageMemoryImpl as:
{code:java}
 PageMemoryImpl pageMem = new PageMemoryImpl(
wrapMetricsPersistentMemoryProvider(memProvider, regMetrics),
calculateFragmentSizes(
regCfg.getName(),
dsCfg.getConcurrencyLevel(),
cacheSize,
chpBufSize
),
cctx,
pageMgr,
dsCfg.getPageSize(),
(fullId, pageBuf, tag) -> {
regMetrics.onPageWritten();

// Write page to disk.
pageMgr.write(fullId.groupId(), fullId.pageId(), pageBuf, tag, 
true);

getCheckpointer().currentProgress().updateEvictedPages(1);
},
trackable,
this,
regMetrics,
regCfg,
resolveThrottlingPolicy(),
() -> getCheckpointer().currentProgress()
);
{code}
Where `getCheckpointer()` and the `CheckpointLockStateChecker stateChecker` 
constructor parameter can be bound to other data region. For instance, 
defragmentation uses own reqion and own checkpointer. See IGNITE-19904




--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Updated] (IGNITE-20138) PageMemory should use actual dataregion's checkpointer.

2023-08-02 Thread Vladimir Steshin (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-20138?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vladimir Steshin updated IGNITE-20138:
--
Description: 
We create PageMemoryImpl as:
{code:java}
 PageMemoryImpl pageMem = new PageMemoryImpl(
wrapMetricsPersistentMemoryProvider(memProvider, regMetrics),
calculateFragmentSizes(
regCfg.getName(),
dsCfg.getConcurrencyLevel(),
cacheSize,
chpBufSize
),
cctx,
pageMgr,
dsCfg.getPageSize(),
(fullId, pageBuf, tag) -> {
regMetrics.onPageWritten();

// Write page to disk.
pageMgr.write(fullId.groupId(), fullId.pageId(), pageBuf, tag, 
true);

getCheckpointer().currentProgress().updateEvictedPages(1);
},
trackable,
this,
regMetrics,
regCfg,
resolveThrottlingPolicy(),
() -> getCheckpointer().currentProgress()
);
{code}
Where `getCheckpointer()` and the `CheckpointLockStateChecker stateChecker` 
constructor parameter can be not bound to PageMemory's data region. For 
instance, defragmentation uses own reqion and own checkpointer. See IGNITE-19904


  was:
We create PageMemoryImpl as:
{code:java}
 PageMemoryImpl pageMem = new PageMemoryImpl(
wrapMetricsPersistentMemoryProvider(memProvider, regMetrics),
calculateFragmentSizes(
regCfg.getName(),
dsCfg.getConcurrencyLevel(),
cacheSize,
chpBufSize
),
cctx,
pageMgr,
dsCfg.getPageSize(),
(fullId, pageBuf, tag) -> {
regMetrics.onPageWritten();

// Write page to disk.
pageMgr.write(fullId.groupId(), fullId.pageId(), pageBuf, tag, 
true);

getCheckpointer().currentProgress().updateEvictedPages(1);
},
trackable,
this,
regMetrics,
regCfg,
resolveThrottlingPolicy(),
() -> getCheckpointer().currentProgress()
);
{code}
Where `getCheckpointer()` and the `CheckpointLockStateChecker stateChecker` 
constructor parameter can be bound to other data region. For instance, 
defragmentation uses own reqion and own checkpointer. See IGNITE-19904



> PageMemory should use actual dataregion's checkpointer.
> ---
>
> Key: IGNITE-20138
> URL: https://issues.apache.org/jira/browse/IGNITE-20138
> Project: Ignite
>  Issue Type: Improvement
>Reporter: Vladimir Steshin
>Priority: Major
>
> We create PageMemoryImpl as:
> {code:java}
>  PageMemoryImpl pageMem = new PageMemoryImpl(
> wrapMetricsPersistentMemoryProvider(memProvider, regMetrics),
> calculateFragmentSizes(
> regCfg.getName(),
> dsCfg.getConcurrencyLevel(),
> cacheSize,
> chpBufSize
> ),
> cctx,
> pageMgr,
> dsCfg.getPageSize(),
> (fullId, pageBuf, tag) -> {
> regMetrics.onPageWritten();
> // Write page to disk.
> pageMgr.write(fullId.groupId(), fullId.pageId(), pageBuf, 
> tag, true);
> getCheckpointer().currentProgress().updateEvictedPages(1);
> },
> trackable,
> this,
> regMetrics,
> regCfg,
> resolveThrottlingPolicy(),
> () -> getCheckpointer().currentProgress()
> );
> {code}
> Where `getCheckpointer()` and the `CheckpointLockStateChecker stateChecker` 
> constructor parameter can be not bound to PageMemory's data region. For 
> instance, defragmentation uses own reqion and own checkpointer. See 
> IGNITE-19904



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Updated] (IGNITE-20138) PageMemory should use actual dataregion's checkpointer.

2023-08-02 Thread Vladimir Steshin (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-20138?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vladimir Steshin updated IGNITE-20138:
--
Description: 
We create PageMemoryImpl as:
{code:java}
 PageMemoryImpl pageMem = new PageMemoryImpl(
wrapMetricsPersistentMemoryProvider(memProvider, regMetrics),
calculateFragmentSizes(
regCfg.getName(),
dsCfg.getConcurrencyLevel(),
cacheSize,
chpBufSize
),
cctx,
pageMgr,
dsCfg.getPageSize(),
(fullId, pageBuf, tag) -> {
regMetrics.onPageWritten();

// Write page to disk.
pageMgr.write(fullId.groupId(), fullId.pageId(), pageBuf, tag, 
true);

getCheckpointer().currentProgress().updateEvictedPages(1);
},
trackable,
this,
regMetrics,
regCfg,
resolveThrottlingPolicy(),
() -> getCheckpointer().currentProgress()
);
{code}
Where `getCheckpointer()` and the `CheckpointLockStateChecker stateChecker` 
constructor parameter can be not bound to PageMemory's data region. For 
instance, defragmentation uses own region and own lightweight checkpointer. See 
IGNITE-19904


  was:
We create PageMemoryImpl as:
{code:java}
 PageMemoryImpl pageMem = new PageMemoryImpl(
wrapMetricsPersistentMemoryProvider(memProvider, regMetrics),
calculateFragmentSizes(
regCfg.getName(),
dsCfg.getConcurrencyLevel(),
cacheSize,
chpBufSize
),
cctx,
pageMgr,
dsCfg.getPageSize(),
(fullId, pageBuf, tag) -> {
regMetrics.onPageWritten();

// Write page to disk.
pageMgr.write(fullId.groupId(), fullId.pageId(), pageBuf, tag, 
true);

getCheckpointer().currentProgress().updateEvictedPages(1);
},
trackable,
this,
regMetrics,
regCfg,
resolveThrottlingPolicy(),
() -> getCheckpointer().currentProgress()
);
{code}
Where `getCheckpointer()` and the `CheckpointLockStateChecker stateChecker` 
constructor parameter can be not bound to PageMemory's data region. For 
instance, defragmentation uses own reqion and own checkpointer. See IGNITE-19904



> PageMemory should use actual dataregion's checkpointer.
> ---
>
> Key: IGNITE-20138
> URL: https://issues.apache.org/jira/browse/IGNITE-20138
> Project: Ignite
>  Issue Type: Improvement
>Reporter: Vladimir Steshin
>Priority: Major
>
> We create PageMemoryImpl as:
> {code:java}
>  PageMemoryImpl pageMem = new PageMemoryImpl(
> wrapMetricsPersistentMemoryProvider(memProvider, regMetrics),
> calculateFragmentSizes(
> regCfg.getName(),
> dsCfg.getConcurrencyLevel(),
> cacheSize,
> chpBufSize
> ),
> cctx,
> pageMgr,
> dsCfg.getPageSize(),
> (fullId, pageBuf, tag) -> {
> regMetrics.onPageWritten();
> // Write page to disk.
> pageMgr.write(fullId.groupId(), fullId.pageId(), pageBuf, 
> tag, true);
> getCheckpointer().currentProgress().updateEvictedPages(1);
> },
> trackable,
> this,
> regMetrics,
> regCfg,
> resolveThrottlingPolicy(),
> () -> getCheckpointer().currentProgress()
> );
> {code}
> Where `getCheckpointer()` and the `CheckpointLockStateChecker stateChecker` 
> constructor parameter can be not bound to PageMemory's data region. For 
> instance, defragmentation uses own region and own lightweight checkpointer. 
> See IGNITE-19904



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Updated] (IGNITE-20136) INDEX hint for calcite engine.

2023-08-02 Thread Vladimir Steshin (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-20136?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vladimir Steshin updated IGNITE-20136:
--
Issue Type: Task  (was: Bug)

> INDEX hint for calcite engine.
> --
>
> Key: IGNITE-20136
> URL: https://issues.apache.org/jira/browse/IGNITE-20136
> Project: Ignite
>  Issue Type: Task
>Reporter: Vladimir Steshin
>Assignee: Vladimir Steshin
>Priority: Major
>  Labels: ise
>
> As part of hints for Calcite, we could try to implement simple hint like 
> INDEX. The main goal is to take deeper look into current hints implementation 
> in Calcite and find a way to work with them.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Created] (IGNITE-20139) RandomForestClassifierTrainer is checking the same conditions

2023-08-02 Thread Alexandr Shapkin (Jira)
Alexandr Shapkin created IGNITE-20139:
-

 Summary: RandomForestClassifierTrainer is checking the same 
conditions
 Key: IGNITE-20139
 URL: https://issues.apache.org/jira/browse/IGNITE-20139
 Project: Ignite
  Issue Type: Bug
  Components: ml
Affects Versions: 2.15
Reporter: Alexandr Shapkin
 Attachments: TreeSample2_Portfolio_Change.png, random-forest.zip

We tried to use GridGain's machine learning capabilities, and discovered a bug 
in GG's implementation of Random Forest. When comparing GG's output with python 
prototype (scikit-learn lib), we noticed that GG's predictions have much lower 
accuracy despite using the same data set and model parameters. 

Further investigation showed that GridGain generates decision trees that kinda 
"loop". The tree starts checking the same condition over and over until it 
reaches the maximum tree depth.

I've attached a standalone reproducer which uses a small excerpt of our data 
set. 

It loads data from the csv file, then performs the training of the model for 
just 1 tree. Then the reproducer finds one of the looping branches and prints 
it. You will see that every single node in the branch uses the same feature, 
value and has then same calculated impurity. 

On my machine the code reproduces this issue 100% of time.

I've also attached an example of the tree generated by python's scikit-learn on 
the same data set with the same parameters. In python the tree usually doesn't 
get deeper than 20 nodes.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Updated] (IGNITE-20135) NO_INDEX hint for calcite engine.

2023-08-02 Thread Vladimir Steshin (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-20135?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vladimir Steshin updated IGNITE-20135:
--
Issue Type: Task  (was: Bug)

> NO_INDEX hint for calcite engine.
> -
>
> Key: IGNITE-20135
> URL: https://issues.apache.org/jira/browse/IGNITE-20135
> Project: Ignite
>  Issue Type: Task
>Reporter: Vladimir Steshin
>Assignee: Vladimir Steshin
>Priority: Major
>  Labels: ise
>
> As part of hints for Calcite, we could try to implement simple hint like 
> NO_INDEX. The main goal is to take deeper look into current hints 
> implementation in Calcite and find a way to work with them.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Updated] (IGNITE-20139) RandomForestClassifierTrainer is checking the same conditions

2023-08-02 Thread Alexandr Shapkin (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-20139?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alexandr Shapkin updated IGNITE-20139:
--
Attachment: random-forest.zip
TreeSample2_Portfolio_Change.png

> RandomForestClassifierTrainer is checking the same conditions
> -
>
> Key: IGNITE-20139
> URL: https://issues.apache.org/jira/browse/IGNITE-20139
> Project: Ignite
>  Issue Type: Bug
>  Components: ml
>Affects Versions: 2.15
>Reporter: Alexandr Shapkin
>Priority: Major
> Attachments: TreeSample2_Portfolio_Change.png, random-forest.zip
>
>
> We tried to use GridGain's machine learning capabilities, and discovered a 
> bug in GG's implementation of Random Forest. When comparing GG's output with 
> python prototype (scikit-learn lib), we noticed that GG's predictions have 
> much lower accuracy despite using the same data set and model parameters. 
> Further investigation showed that GridGain generates decision trees that 
> kinda "loop". The tree starts checking the same condition over and over until 
> it reaches the maximum tree depth.
> I've attached a standalone reproducer which uses a small excerpt of our data 
> set. 
> It loads data from the csv file, then performs the training of the model for 
> just 1 tree. Then the reproducer finds one of the looping branches and prints 
> it. You will see that every single node in the branch uses the same feature, 
> value and has then same calculated impurity. 
> On my machine the code reproduces this issue 100% of time.
> I've also attached an example of the tree generated by python's scikit-learn 
> on the same data set with the same parameters. In python the tree usually 
> doesn't get deeper than 20 nodes.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Commented] (IGNITE-20139) RandomForestClassifierTrainer accuracy issue

2023-08-02 Thread Alexandr Shapkin (Jira)


[ 
https://issues.apache.org/jira/browse/IGNITE-20139?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17750303#comment-17750303
 ] 

Alexandr Shapkin commented on IGNITE-20139:
---

[~zaleslaw] could you please take a look? Seems to be a valid issue alongside 
the attached reproducer.

> RandomForestClassifierTrainer accuracy issue
> 
>
> Key: IGNITE-20139
> URL: https://issues.apache.org/jira/browse/IGNITE-20139
> Project: Ignite
>  Issue Type: Bug
>  Components: ml
>Affects Versions: 2.15
>Reporter: Alexandr Shapkin
>Priority: Major
> Attachments: TreeSample2_Portfolio_Change.png, random-forest.zip
>
>
> We tried to use GridGain's machine learning capabilities, and discovered a 
> bug in GG's implementation of Random Forest. When comparing GG's output with 
> python prototype (scikit-learn lib), we noticed that GG's predictions have 
> much lower accuracy despite using the same data set and model parameters. 
> Further investigation showed that GridGain generates decision trees that 
> kinda "loop". The tree starts checking the same condition over and over until 
> it reaches the maximum tree depth.
> I've attached a standalone reproducer which uses a small excerpt of our data 
> set. 
> It loads data from the csv file, then performs the training of the model for 
> just 1 tree. Then the reproducer finds one of the looping branches and prints 
> it. You will see that every single node in the branch uses the same feature, 
> value and has then same calculated impurity. 
> On my machine the code reproduces this issue 100% of time.
> I've also attached an example of the tree generated by python's scikit-learn 
> on the same data set with the same parameters. In python the tree usually 
> doesn't get deeper than 20 nodes.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Updated] (IGNITE-20139) RandomForestClassifierTrainer accuracy issue

2023-08-02 Thread Alexandr Shapkin (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-20139?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alexandr Shapkin updated IGNITE-20139:
--
Summary: RandomForestClassifierTrainer accuracy issue  (was: 
RandomForestClassifierTrainer is checking the same conditions)

> RandomForestClassifierTrainer accuracy issue
> 
>
> Key: IGNITE-20139
> URL: https://issues.apache.org/jira/browse/IGNITE-20139
> Project: Ignite
>  Issue Type: Bug
>  Components: ml
>Affects Versions: 2.15
>Reporter: Alexandr Shapkin
>Priority: Major
> Attachments: TreeSample2_Portfolio_Change.png, random-forest.zip
>
>
> We tried to use GridGain's machine learning capabilities, and discovered a 
> bug in GG's implementation of Random Forest. When comparing GG's output with 
> python prototype (scikit-learn lib), we noticed that GG's predictions have 
> much lower accuracy despite using the same data set and model parameters. 
> Further investigation showed that GridGain generates decision trees that 
> kinda "loop". The tree starts checking the same condition over and over until 
> it reaches the maximum tree depth.
> I've attached a standalone reproducer which uses a small excerpt of our data 
> set. 
> It loads data from the csv file, then performs the training of the model for 
> just 1 tree. Then the reproducer finds one of the looping branches and prints 
> it. You will see that every single node in the branch uses the same feature, 
> value and has then same calculated impurity. 
> On my machine the code reproduces this issue 100% of time.
> I've also attached an example of the tree generated by python's scikit-learn 
> on the same data set with the same parameters. In python the tree usually 
> doesn't get deeper than 20 nodes.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Updated] (IGNITE-15616) Calcite. Failed to parse UPDATE query.

2023-08-02 Thread Yury Gerzhedovich (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-15616?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Yury Gerzhedovich updated IGNITE-15616:
---
Labels: calcite ignite-3  (was: calcite calcite2-required calcite3-required 
ignite-3)

> Calcite. Failed to parse UPDATE query.
> --
>
> Key: IGNITE-15616
> URL: https://issues.apache.org/jira/browse/IGNITE-15616
> Project: Ignite
>  Issue Type: Bug
>  Components: sql
>Reporter: Evgeny Stanilovsky
>Priority: Minor
>  Labels: calcite, ignite-3
>
> SQL Standart feature E153 - Updatable queries with sub-queries doesn't 
> support right now.
>  
> {noformat}
> statement ok
> UPDATE integers i1 SET i=DEFAULT WHERE i=(SELECT MIN(i) FROM integers WHERE 
> i1.id query II
> SELECT id, i FROM integers ORDER BY id
> 
> 1 NULL
> 2 NULL
> 3 2
> 4 3
> {noformat}
> {noformat}
> Statement [queries=ArrayList [UPDATE integers i1 SET i=DEFAULT WHERE 
> i=(SELECT MIN(i) FROM integers WHERE i1.id   at 
> org.apache.ignite.internal.processors.query.calcite.logical.SqlScriptRunner$Statement.execute(SqlScriptRunner.java:404)
>   at 
> org.apache.ignite.internal.processors.query.calcite.logical.SqlScriptRunner.run(SqlScriptRunner.java:115)
>   at 
> org.apache.ignite.internal.processors.query.calcite.logical.ScriptTestRunner$1.run(ScriptTestRunner.java:219)
>   at java.lang.Thread.run(Thread.java:748)
> Caused by: class 
> org.apache.ignite.internal.processors.query.IgniteSQLException: Failed to 
> parse query.
>   at 
> org.apache.ignite.internal.processors.query.calcite.util.Commons.parse(Commons.java:205)
> {noformat}
> {noformat}
> /subquery/scalar/test_update_subquery.test[_ignore]
> {noformat}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Resolved] (IGNITE-15616) Calcite. Failed to parse UPDATE query.

2023-08-02 Thread Yury Gerzhedovich (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-15616?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Yury Gerzhedovich resolved IGNITE-15616.

Resolution: Cannot Reproduce

> Calcite. Failed to parse UPDATE query.
> --
>
> Key: IGNITE-15616
> URL: https://issues.apache.org/jira/browse/IGNITE-15616
> Project: Ignite
>  Issue Type: Bug
>  Components: sql
>Reporter: Evgeny Stanilovsky
>Priority: Minor
>  Labels: calcite, ignite-3
>
> SQL Standart feature E153 - Updatable queries with sub-queries doesn't 
> support right now.
>  
> {noformat}
> statement ok
> UPDATE integers i1 SET i=DEFAULT WHERE i=(SELECT MIN(i) FROM integers WHERE 
> i1.id query II
> SELECT id, i FROM integers ORDER BY id
> 
> 1 NULL
> 2 NULL
> 3 2
> 4 3
> {noformat}
> {noformat}
> Statement [queries=ArrayList [UPDATE integers i1 SET i=DEFAULT WHERE 
> i=(SELECT MIN(i) FROM integers WHERE i1.id   at 
> org.apache.ignite.internal.processors.query.calcite.logical.SqlScriptRunner$Statement.execute(SqlScriptRunner.java:404)
>   at 
> org.apache.ignite.internal.processors.query.calcite.logical.SqlScriptRunner.run(SqlScriptRunner.java:115)
>   at 
> org.apache.ignite.internal.processors.query.calcite.logical.ScriptTestRunner$1.run(ScriptTestRunner.java:219)
>   at java.lang.Thread.run(Thread.java:748)
> Caused by: class 
> org.apache.ignite.internal.processors.query.IgniteSQLException: Failed to 
> parse query.
>   at 
> org.apache.ignite.internal.processors.query.calcite.util.Commons.parse(Commons.java:205)
> {noformat}
> {noformat}
> /subquery/scalar/test_update_subquery.test[_ignore]
> {noformat}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Assigned] (IGNITE-19792) Support @Immutable logic for distributed configuration

2023-08-02 Thread Jira


 [ 
https://issues.apache.org/jira/browse/IGNITE-19792?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

 Kirill Sizov reassigned IGNITE-19792:
--

Assignee:  Kirill Sizov

> Support @Immutable logic for distributed configuration
> --
>
> Key: IGNITE-19792
> URL: https://issues.apache.org/jira/browse/IGNITE-19792
> Project: Ignite
>  Issue Type: Improvement
>Reporter: Roman Puchkovskiy
>Assignee:  Kirill Sizov
>Priority: Major
>  Labels: ignite-3
> Fix For: 3.0.0-beta2
>
>
> For distributed properties, the following scenario makes sense:
>  # Property value might be set on cluster init
>  # If it is provided, it overrides the default; otherwise, the default (set 
> in the {{@Value}} annotation) is used
>  # Any attempt to modify the property value using a changer is rejected by a 
> validator
> {{@Immutable}} annotation is designed to trigger such behavior.
> Now, the observed behavior is different: if a property is annotated as 
> {{{}@Immutable{}}}, then, if a value is supplied during cluster 
> initialization, the value is silently ignored, so it is impossible to 
> configure such property to anything different from the default.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Updated] (IGNITE-19861) Introduce SQL metrics

2023-08-02 Thread Andrey Mashenkov (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-19861?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrey Mashenkov updated IGNITE-19861:
--
Fix Version/s: 3.0.0-beta2

> Introduce SQL metrics
> -
>
> Key: IGNITE-19861
> URL: https://issues.apache.org/jira/browse/IGNITE-19861
> Project: Ignite
>  Issue Type: Improvement
>  Components: sql
>Reporter: Yury Gerzhedovich
>Assignee: Yury Gerzhedovich
>Priority: Major
>  Labels: ignite-3
> Fix For: 3.0.0-beta2
>
>  Time Spent: 2h 10m
>  Remaining Estimate: 0h
>
> AI3 have metrics framework, but SQL engine don't expose any metrics, which 
> could help to user.
> Let's introduce the first three metrics:
> 1) hit in plan cache - counter
> 2) miss in plan cache - counter
> 3) number of open cursors - gauge



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Resolved] (IGNITE-19916) TX SQL tests refactoring with the help of @TestOnly IgniteImpl#txManager and TxManager#pending

2023-08-02 Thread Yury Gerzhedovich (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-19916?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Yury Gerzhedovich resolved IGNITE-19916.

Resolution: Cannot Reproduce

fixed as part of https://issues.apache.org/jira/browse/IGNITE-19861

> TX SQL tests refactoring with the help of @TestOnly IgniteImpl#txManager and 
> TxManager#pending 
> ---
>
> Key: IGNITE-19916
> URL: https://issues.apache.org/jira/browse/IGNITE-19916
> Project: Ignite
>  Issue Type: Improvement
>  Components: sql
>Reporter: Alexander Lapin
>Priority: Major
>  Labels: ignite-3
>
> h3. Motivation
> Rather often it's required to check whether all transactions that were 
> started by SQL are properly finished. Currently, reflection along with crappy 
> txnState checks are used:
> {code:java}
> TxManager txManagerInternal = (TxManager) 
> IgniteTestUtils.getFieldValue(CLUSTER_NODES.get(0), IgniteImpl.class, 
> "txManager");
> var states = (Map) 
> IgniteTestUtils.getFieldValue(txManagerInternal, TxManagerImpl.class, 
> "states");        
> assertEquals(txManagerInternal.finished(), states.size());{code}
> After implementing https://issues.apache.org/jira/browse/IGNITE-19663 it 
> should be possible to rewrite code above with
> {code:java}
> assertEquals(0, ((IgniteImpl) CLUSTER_NODES.get(0)).txManager().pending()); 
> {code}
> or similar.
> h3. Definition of Done
> All required sql tx tests are simplified through TxManager#pending()
>  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Created] (IGNITE-20140) Exceptions derived from IgniteException are constructed unexpectedly while copying

2023-08-02 Thread Vadim Pakhnushev (Jira)
Vadim Pakhnushev created IGNITE-20140:
-

 Summary: Exceptions derived from IgniteException are constructed 
unexpectedly while copying
 Key: IGNITE-20140
 URL: https://issues.apache.org/jira/browse/IGNITE-20140
 Project: Ignite
  Issue Type: Bug
Reporter: Vadim Pakhnushev
Assignee: Vyacheslav Koptilin


When exception derived from IgniteException is thrown from the SQL command, 
{{org.apache.ignite.internal.sql.engine.AsyncSqlCursorImpl#requestNextAsync}} 
wraps it using the 
{{{}org.apache.ignite.lang.IgniteExceptionUtils#copyExceptionWithCause{}}}, 
which tries to find corresponding constructor in the derived class.

The problem is that when the exception has a custom constructor with the 
matching signature, for instance, with the single String argument, it gets 
called with the original exception message leading to duplicated messages if 
the custom constructor constructs a message from its argument.

It the exception class doesn't have matching constructor, the original 
exception gets lots because the handler in the {{requestNextAsync}} throws a 
{{{}null{}}}.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Updated] (IGNITE-20140) Exceptions derived from IgniteException are constructed unexpectedly while copying

2023-08-02 Thread Vyacheslav Koptilin (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-20140?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vyacheslav Koptilin updated IGNITE-20140:
-
Description: 
When exception derived from IgniteException is thrown from the SQL command, 
{{org.apache.ignite.internal.sql.engine.AsyncSqlCursorImpl#requestNextAsync}} 
wraps it using the 
{{{}org.apache.ignite.lang.IgniteExceptionUtils#copyExceptionWithCause{}}}, 
which tries to find corresponding constructor in the derived class.

The problem is that when the exception has a custom constructor with the 
matching signature, for instance, with the single String argument, it gets 
called with the original exception message leading to duplicated messages if 
the custom constructor constructs a message from its argument.

If the exception class doesn't have matching constructor, the original 
exception gets lots because the handler in the {{requestNextAsync}} throws a 
{{{}null{}}}.

  was:
When exception derived from IgniteException is thrown from the SQL command, 
{{org.apache.ignite.internal.sql.engine.AsyncSqlCursorImpl#requestNextAsync}} 
wraps it using the 
{{{}org.apache.ignite.lang.IgniteExceptionUtils#copyExceptionWithCause{}}}, 
which tries to find corresponding constructor in the derived class.

The problem is that when the exception has a custom constructor with the 
matching signature, for instance, with the single String argument, it gets 
called with the original exception message leading to duplicated messages if 
the custom constructor constructs a message from its argument.

It the exception class doesn't have matching constructor, the original 
exception gets lots because the handler in the {{requestNextAsync}} throws a 
{{{}null{}}}.


> Exceptions derived from IgniteException are constructed unexpectedly while 
> copying
> --
>
> Key: IGNITE-20140
> URL: https://issues.apache.org/jira/browse/IGNITE-20140
> Project: Ignite
>  Issue Type: Bug
>Reporter: Vadim Pakhnushev
>Assignee: Vyacheslav Koptilin
>Priority: Major
>  Labels: ignite-3
>
> When exception derived from IgniteException is thrown from the SQL command, 
> {{org.apache.ignite.internal.sql.engine.AsyncSqlCursorImpl#requestNextAsync}} 
> wraps it using the 
> {{{}org.apache.ignite.lang.IgniteExceptionUtils#copyExceptionWithCause{}}}, 
> which tries to find corresponding constructor in the derived class.
> The problem is that when the exception has a custom constructor with the 
> matching signature, for instance, with the single String argument, it gets 
> called with the original exception message leading to duplicated messages if 
> the custom constructor constructs a message from its argument.
> If the exception class doesn't have matching constructor, the original 
> exception gets lots because the handler in the {{requestNextAsync}} throws a 
> {{{}null{}}}.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Updated] (IGNITE-19160) Improve message about sent partition file during snapshot restore

2023-08-02 Thread Julia Bakulina (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-19160?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Julia Bakulina updated IGNITE-19160:

Ignite Flags: Release Notes Required  (was: Docs Required,Release Notes 
Required)

> Improve message about sent partition file during snapshot restore
> -
>
> Key: IGNITE-19160
> URL: https://issues.apache.org/jira/browse/IGNITE-19160
> Project: Ignite
>  Issue Type: Task
>Reporter: Ilya Shishkov
>Assignee: Julia Bakulina
>Priority: Minor
>  Labels: iep-43, ise
>
> Currently, message about partition is as below:
> {quote}
> [2023-03-29T18:31:44,773][INFO ][snapshot-runner-#863%node0%][SnapshotSender] 
> Partition file has been send [part=part-645.bin, 
> pair={color:red}GroupPartitionId [grpId=1544803905, partId=645]{color}, 
> length=45056]
> {quote}
> It does not tell us: 
> # Receiver node id / address / consistent id.
> # Cache or cache group name which partition belongs to. Numerical group id is 
> not convenient way to determine cache or cache group.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Updated] (IGNITE-19160) Improve message about sent partition file during snapshot restore

2023-08-02 Thread Julia Bakulina (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-19160?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Julia Bakulina updated IGNITE-19160:

Release Note: Improved message about sent partition file during snapshot 
restore

> Improve message about sent partition file during snapshot restore
> -
>
> Key: IGNITE-19160
> URL: https://issues.apache.org/jira/browse/IGNITE-19160
> Project: Ignite
>  Issue Type: Task
>Reporter: Ilya Shishkov
>Assignee: Julia Bakulina
>Priority: Minor
>  Labels: iep-43, ise
>
> Currently, message about partition is as below:
> {quote}
> [2023-03-29T18:31:44,773][INFO ][snapshot-runner-#863%node0%][SnapshotSender] 
> Partition file has been send [part=part-645.bin, 
> pair={color:red}GroupPartitionId [grpId=1544803905, partId=645]{color}, 
> length=45056]
> {quote}
> It does not tell us: 
> # Receiver node id / address / consistent id.
> # Cache or cache group name which partition belongs to. Numerical group id is 
> not convenient way to determine cache or cache group.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Assigned] (IGNITE-20023) Unstable cluster topology leads to flaky test

2023-08-02 Thread Roman Puchkovskiy (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-20023?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Roman Puchkovskiy reassigned IGNITE-20023:
--

Assignee: Roman Puchkovskiy

> Unstable cluster topology leads to flaky test
> -
>
> Key: IGNITE-20023
> URL: https://issues.apache.org/jira/browse/IGNITE-20023
> Project: Ignite
>  Issue Type: Bug
>Affects Versions: 3.0
>Reporter: Alexey Scherbakov
>Assignee: Roman Puchkovskiy
>Priority: Major
>  Labels: ignite-3
>
> {noformat}
> 2023-07-21 18:30:15:108 +0300 
> [INFO][main][ItTxDistributedTestThreeNodesThreeReplicas] >>> Starting test: 
> ItTxDistributedTestThreeNodesThreeReplicas#testTxClosureAsync, displayName: 
> testTxClosureAsync(), workDir: 
> D:\work\ignite-3\modules\table\build\work\ItTxDistributedTestThreeNodesThreeReplicas\ttca_25996
> 2023-07-21 18:30:15:142 +0300 [INFO][main][ConnectionManager] Server started 
> [address=/0:0:0:0:0:0:0:0:20001]
> 2023-07-21 18:30:15:142 +0300 
> [INFO][ForkJoinPool.commonPool-worker-7][ConnectionManager] Server started 
> [address=/0:0:0:0:0:0:0:0:2]
> 2023-07-21 18:30:15:142 +0300 
> [INFO][ForkJoinPool.commonPool-worker-3][ConnectionManager] Server started 
> [address=/0:0:0:0:0:0:0:0:20002]
> 2023-07-21 18:30:15:145 +0300 
> [INFO][itdttntr_n_2-srv-worker-1][RecoveryServerHandshakeManager] Failed 
> to acquire recovery descriptor during handshake, it is held by: [id: 
> 0x348087cd, L:/192.168.0.138:63056 - R:/192.168.0.138:20001]
> 2023-07-21 18:30:15:147 +0300 
> [INFO][itdttntr_n_20002-client-1][RecoveryClientHandshakeManager] Failed to 
> acquire recovery descriptor during handshake, it is held by: [id: 0x8017c1b2, 
> L:/192.168.0.138:20002 - R:/192.168.0.138:63058]
> 2023-07-21 18:30:15:147 +0300 
> [INFO][itdttntr_n_20002-client-2][RecoveryClientHandshakeManager] Failed to 
> acquire recovery descriptor during handshake, it is held by: [id: 0xb26b9231, 
> L:/192.168.0.138:20002 - R:/192.168.0.138:63059]
> 2023-07-21 18:30:15:149 +0300 
> [INFO][itdttntr_n_20001-client-3][RecoveryClientHandshakeManager] Failed to 
> acquire recovery descriptor during handshake, it is held by: [id: 0x3dd8aa9e, 
> L:/192.168.0.138:20001 - R:/192.168.0.138:63056]
> 2023-07-21 18:30:15:149 +0300 
> [INFO][itdttntr_n_20002-client-3][RecoveryClientHandshakeManager] Failed to 
> acquire recovery descriptor during handshake, it is held by: [id: 0xb26b9231, 
> L:/192.168.0.138:20002 - R:/192.168.0.138:63059]
> 2023-07-21 18:30:15:149 +0300 
> [INFO][itdttntr_n_20002-client-4][RecoveryClientHandshakeManager] Failed to 
> acquire recovery descriptor during handshake, it is held by: [id: 0x8017c1b2, 
> L:/192.168.0.138:20002 - R:/192.168.0.138:63058]
> 2023-07-21 18:30:15:150 +0300 
> [INFO][sc-cluster-20001-107][ScaleCubeTopologyService] Node joined 
> [node=ClusterNode [id=ade225ca-5ef0-4b03-9dbe-edbe2e2ba928, 
> name=itdttntr_n_20002, address=192.168.0.138:20002, nodeMetadata=null]]
> 2023-07-21 18:30:15:150 +0300 
> [INFO][sc-cluster-20002-106][ScaleCubeTopologyService] Node joined 
> [node=ClusterNode [id=bee1e04c-e2f0-48c0-ad54-eefdd6928278, 
> name=itdttntr_n_20001, address=192.168.0.138:20001, nodeMetadata=null]]
> 2023-07-21 18:30:15:150 +0300 
> [INFO][sc-cluster-2-105][ScaleCubeTopologyService] Node joined 
> [node=ClusterNode [id=ade225ca-5ef0-4b03-9dbe-edbe2e2ba928, 
> name=itdttntr_n_20002, address=192.168.0.138:20002, nodeMetadata=null]]
> 2023-07-21 18:30:15:150 +0300 
> [INFO][sc-cluster-20001-107][ScaleCubeTopologyService] Topology snapshot 
> [nodes=[itdttntr_n_20002]]
> 2023-07-21 18:30:15:150 +0300 
> [INFO][sc-cluster-2-105][ScaleCubeTopologyService] Topology snapshot 
> [nodes=[itdttntr_n_20002]]
> 2023-07-21 18:30:15:150 +0300 
> [INFO][sc-cluster-20002-106][ScaleCubeTopologyService] Topology snapshot 
> [nodes=[itdttntr_n_20001]]
> 2023-07-21 18:30:15:150 +0300 
> [INFO][sc-cluster-20001-107][ScaleCubeTopologyService] Node joined 
> [node=ClusterNode [id=496bbe6d-66b7-4aa9-9807-e1dd5207d900, 
> name=itdttntr_n_2, address=192.168.0.138:2, nodeMetadata=null]]
> 2023-07-21 18:30:15:150 +0300 
> [INFO][sc-cluster-20002-106][ScaleCubeTopologyService] Node joined 
> [node=ClusterNode [id=496bbe6d-66b7-4aa9-9807-e1dd5207d900, 
> name=itdttntr_n_2, address=192.168.0.138:2, nodeMetadata=null]]
> 2023-07-21 18:30:15:150 +0300 
> [INFO][sc-cluster-20001-107][ScaleCubeTopologyService] Topology snapshot 
> [nodes=[itdttntr_n_2, itdttntr_n_20002]]
> 2023-07-21 18:30:15:150 +0300 
> [INFO][sc-cluster-20002-106][ScaleCubeTopologyService] Topology snapshot 
> [nodes=[itdttntr_n_2, itdttntr_n_20001]]
> 2023-07-21 18:30:15:151 +0300 
> [INFO][ForkJoinPool.commonPool-worker-3][ScaleCubeTopologyService] Node 
> joined [node=ClusterNode [id=ade225ca-5ef0-4b03-9dbe-edbe2e2ba92

[jira] [Commented] (IGNITE-20023) Unstable cluster topology leads to flaky test

2023-08-02 Thread Roman Puchkovskiy (Jira)


[ 
https://issues.apache.org/jira/browse/IGNITE-20023?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17750370#comment-17750370
 ] 

Roman Puchkovskiy commented on IGNITE-20023:


What I was able to find in the logs:

# There is a lot of noise about one node not being let by other nodes to the 
topology as it has already left (this seems to be irrelevant to the test 
failure)
# There is an NPE (that probably caused the test failure), but additional 
information is needed to find the culprit. I'm adding it in the attached PR
# There is a secondary NPE (it is suppressed), I'm fixing it in the PR

> Unstable cluster topology leads to flaky test
> -
>
> Key: IGNITE-20023
> URL: https://issues.apache.org/jira/browse/IGNITE-20023
> Project: Ignite
>  Issue Type: Bug
>Affects Versions: 3.0
>Reporter: Alexey Scherbakov
>Assignee: Roman Puchkovskiy
>Priority: Major
>  Labels: ignite-3
>
> {noformat}
> 2023-07-21 18:30:15:108 +0300 
> [INFO][main][ItTxDistributedTestThreeNodesThreeReplicas] >>> Starting test: 
> ItTxDistributedTestThreeNodesThreeReplicas#testTxClosureAsync, displayName: 
> testTxClosureAsync(), workDir: 
> D:\work\ignite-3\modules\table\build\work\ItTxDistributedTestThreeNodesThreeReplicas\ttca_25996
> 2023-07-21 18:30:15:142 +0300 [INFO][main][ConnectionManager] Server started 
> [address=/0:0:0:0:0:0:0:0:20001]
> 2023-07-21 18:30:15:142 +0300 
> [INFO][ForkJoinPool.commonPool-worker-7][ConnectionManager] Server started 
> [address=/0:0:0:0:0:0:0:0:2]
> 2023-07-21 18:30:15:142 +0300 
> [INFO][ForkJoinPool.commonPool-worker-3][ConnectionManager] Server started 
> [address=/0:0:0:0:0:0:0:0:20002]
> 2023-07-21 18:30:15:145 +0300 
> [INFO][itdttntr_n_2-srv-worker-1][RecoveryServerHandshakeManager] Failed 
> to acquire recovery descriptor during handshake, it is held by: [id: 
> 0x348087cd, L:/192.168.0.138:63056 - R:/192.168.0.138:20001]
> 2023-07-21 18:30:15:147 +0300 
> [INFO][itdttntr_n_20002-client-1][RecoveryClientHandshakeManager] Failed to 
> acquire recovery descriptor during handshake, it is held by: [id: 0x8017c1b2, 
> L:/192.168.0.138:20002 - R:/192.168.0.138:63058]
> 2023-07-21 18:30:15:147 +0300 
> [INFO][itdttntr_n_20002-client-2][RecoveryClientHandshakeManager] Failed to 
> acquire recovery descriptor during handshake, it is held by: [id: 0xb26b9231, 
> L:/192.168.0.138:20002 - R:/192.168.0.138:63059]
> 2023-07-21 18:30:15:149 +0300 
> [INFO][itdttntr_n_20001-client-3][RecoveryClientHandshakeManager] Failed to 
> acquire recovery descriptor during handshake, it is held by: [id: 0x3dd8aa9e, 
> L:/192.168.0.138:20001 - R:/192.168.0.138:63056]
> 2023-07-21 18:30:15:149 +0300 
> [INFO][itdttntr_n_20002-client-3][RecoveryClientHandshakeManager] Failed to 
> acquire recovery descriptor during handshake, it is held by: [id: 0xb26b9231, 
> L:/192.168.0.138:20002 - R:/192.168.0.138:63059]
> 2023-07-21 18:30:15:149 +0300 
> [INFO][itdttntr_n_20002-client-4][RecoveryClientHandshakeManager] Failed to 
> acquire recovery descriptor during handshake, it is held by: [id: 0x8017c1b2, 
> L:/192.168.0.138:20002 - R:/192.168.0.138:63058]
> 2023-07-21 18:30:15:150 +0300 
> [INFO][sc-cluster-20001-107][ScaleCubeTopologyService] Node joined 
> [node=ClusterNode [id=ade225ca-5ef0-4b03-9dbe-edbe2e2ba928, 
> name=itdttntr_n_20002, address=192.168.0.138:20002, nodeMetadata=null]]
> 2023-07-21 18:30:15:150 +0300 
> [INFO][sc-cluster-20002-106][ScaleCubeTopologyService] Node joined 
> [node=ClusterNode [id=bee1e04c-e2f0-48c0-ad54-eefdd6928278, 
> name=itdttntr_n_20001, address=192.168.0.138:20001, nodeMetadata=null]]
> 2023-07-21 18:30:15:150 +0300 
> [INFO][sc-cluster-2-105][ScaleCubeTopologyService] Node joined 
> [node=ClusterNode [id=ade225ca-5ef0-4b03-9dbe-edbe2e2ba928, 
> name=itdttntr_n_20002, address=192.168.0.138:20002, nodeMetadata=null]]
> 2023-07-21 18:30:15:150 +0300 
> [INFO][sc-cluster-20001-107][ScaleCubeTopologyService] Topology snapshot 
> [nodes=[itdttntr_n_20002]]
> 2023-07-21 18:30:15:150 +0300 
> [INFO][sc-cluster-2-105][ScaleCubeTopologyService] Topology snapshot 
> [nodes=[itdttntr_n_20002]]
> 2023-07-21 18:30:15:150 +0300 
> [INFO][sc-cluster-20002-106][ScaleCubeTopologyService] Topology snapshot 
> [nodes=[itdttntr_n_20001]]
> 2023-07-21 18:30:15:150 +0300 
> [INFO][sc-cluster-20001-107][ScaleCubeTopologyService] Node joined 
> [node=ClusterNode [id=496bbe6d-66b7-4aa9-9807-e1dd5207d900, 
> name=itdttntr_n_2, address=192.168.0.138:2, nodeMetadata=null]]
> 2023-07-21 18:30:15:150 +0300 
> [INFO][sc-cluster-20002-106][ScaleCubeTopologyService] Node joined 
> [node=ClusterNode [id=496bbe6d-66b7-4aa9-9807-e1dd5207d900, 
> name=itdttntr_n_2, address=192.168.0.138:2, nodeMetadata=null]]
> 2023-07-21 18:30:15:150 +0300 
> [INFO][sc-c

[jira] [Assigned] (IGNITE-19758) Delete "StorageSortedIndexColumnDescriptor#name"

2023-08-02 Thread Andrey Mashenkov (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-19758?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrey Mashenkov reassigned IGNITE-19758:
-

Assignee: Andrey Mashenkov

> Delete "StorageSortedIndexColumnDescriptor#name"
> 
>
> Key: IGNITE-19758
> URL: https://issues.apache.org/jira/browse/IGNITE-19758
> Project: Ignite
>  Issue Type: Improvement
>Reporter: Ivan Bessonov
>Assignee: Andrey Mashenkov
>Priority: Major
>  Labels: ignite-3
>
> It's only really used in tests.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Resolved] (IGNITE-20047) NPE in evicted pages counter of the checkpoint progress.

2023-08-02 Thread Vladimir Steshin (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-20047?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vladimir Steshin resolved IGNITE-20047.
---
Resolution: Fixed

Is cause of and fixed in IGNITE-19904

> NPE in evicted pages counter of the checkpoint progress.
> 
>
> Key: IGNITE-20047
> URL: https://issues.apache.org/jira/browse/IGNITE-20047
> Project: Ignite
>  Issue Type: Bug
>Reporter: Vladimir Steshin
>Priority: Major
> Attachments: failureNPE.log, npe_failure2.log, npe_pu_counter_2.15.log
>
>
> NPE can occur in
> {code:java}
>  /** {@inheritDoc} */
> @Override public void updateEvictedPages(int delta) {
> A.ensure(delta > 0, "param must be positive");
> if (evictedPagesCounter() != null)
> evictedPagesCounter().addAndGet(delta);
> }
> {code}
> because evictedPagesCounter is already null below 'if'. `clearCounters()` is 
> invoked in _markCheckpointEnd()_. But it is possible that the counter is 
> being concurrently updated. 
> Looks able to corrupt PDS if the process failed when checkpoint end marker is 
> saved but the page is not yet.
> Probably, happens with the _DelayedDirtyPageStoreWrite_ / 
> _DelayedDirtyPageStoreWrite_. Found in the defragmentation process. But it 
> seems to be a common problem. Logs attached.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Closed] (IGNITE-20047) NPE in evicted pages counter of the checkpoint progress.

2023-08-02 Thread Vladimir Steshin (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-20047?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vladimir Steshin closed IGNITE-20047.
-

IGNITE-19904

> NPE in evicted pages counter of the checkpoint progress.
> 
>
> Key: IGNITE-20047
> URL: https://issues.apache.org/jira/browse/IGNITE-20047
> Project: Ignite
>  Issue Type: Bug
>Reporter: Vladimir Steshin
>Priority: Major
> Attachments: failureNPE.log, npe_failure2.log, npe_pu_counter_2.15.log
>
>
> NPE can occur in
> {code:java}
>  /** {@inheritDoc} */
> @Override public void updateEvictedPages(int delta) {
> A.ensure(delta > 0, "param must be positive");
> if (evictedPagesCounter() != null)
> evictedPagesCounter().addAndGet(delta);
> }
> {code}
> because evictedPagesCounter is already null below 'if'. `clearCounters()` is 
> invoked in _markCheckpointEnd()_. But it is possible that the counter is 
> being concurrently updated. 
> Looks able to corrupt PDS if the process failed when checkpoint end marker is 
> saved but the page is not yet.
> Probably, happens with the _DelayedDirtyPageStoreWrite_ / 
> _DelayedDirtyPageStoreWrite_. Found in the defragmentation process. But it 
> seems to be a common problem. Logs attached.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Created] (IGNITE-20141) TxRollbackOnTimeoutNoDeadlockDetectionTest fails at local run (not from the suite)

2023-08-02 Thread Anton Vinogradov (Jira)
Anton Vinogradov created IGNITE-20141:
-

 Summary: TxRollbackOnTimeoutNoDeadlockDetectionTest fails at local 
run (not from the suite)
 Key: IGNITE-20141
 URL: https://issues.apache.org/jira/browse/IGNITE-20141
 Project: Ignite
  Issue Type: Bug
Reporter: Anton Vinogradov
Assignee: Anton Vinogradov
 Fix For: 2.16






--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Commented] (IGNITE-20141) TxRollbackOnTimeoutNoDeadlockDetectionTest fails at local run (not from the suite)

2023-08-02 Thread Anton Vinogradov (Jira)


[ 
https://issues.apache.org/jira/browse/IGNITE-20141?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17750397#comment-17750397
 ] 

Anton Vinogradov commented on IGNITE-20141:
---

TxRollbackOnTimeoutNoDeadlockDetectionTest has no failures at CI because Ignite 
ignores IGNITE_TX_DEADLOCK_DETECTION_MAX_ITERS property set after it already 
set by previous tests (static field).

But, when you run this test separately it fails because transaction will never 
become finished on timeout when deadlockDetection is dissabled.

> TxRollbackOnTimeoutNoDeadlockDetectionTest fails at local run (not from the 
> suite)
> --
>
> Key: IGNITE-20141
> URL: https://issues.apache.org/jira/browse/IGNITE-20141
> Project: Ignite
>  Issue Type: Bug
>Reporter: Anton Vinogradov
>Assignee: Anton Vinogradov
>Priority: Major
> Fix For: 2.16
>
>




--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Updated] (IGNITE-20141) TxRollbackOnTimeoutNoDeadlockDetectionTest fails at local run (not from the suite)

2023-08-02 Thread Anton Vinogradov (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-20141?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Anton Vinogradov updated IGNITE-20141:
--
Ignite Flags: Release Notes Required  (was: Docs Required,Release Notes 
Required)

> TxRollbackOnTimeoutNoDeadlockDetectionTest fails at local run (not from the 
> suite)
> --
>
> Key: IGNITE-20141
> URL: https://issues.apache.org/jira/browse/IGNITE-20141
> Project: Ignite
>  Issue Type: Bug
>Reporter: Anton Vinogradov
>Assignee: Anton Vinogradov
>Priority: Major
> Fix For: 2.16
>
>  Time Spent: 20m
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Updated] (IGNITE-20141) Transaction will never become finished on timeout when deadlock detection is disabled.

2023-08-02 Thread Anton Vinogradov (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-20141?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Anton Vinogradov updated IGNITE-20141:
--
Summary: Transaction will never become finished on timeout when deadlock 
detection is disabled.  (was: TxRollbackOnTimeoutNoDeadlockDetectionTest fails 
at local run (not from the suite))

> Transaction will never become finished on timeout when deadlock detection is 
> disabled.
> --
>
> Key: IGNITE-20141
> URL: https://issues.apache.org/jira/browse/IGNITE-20141
> Project: Ignite
>  Issue Type: Bug
>Reporter: Anton Vinogradov
>Assignee: Anton Vinogradov
>Priority: Major
> Fix For: 2.16
>
>  Time Spent: 20m
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Created] (IGNITE-20142) Introduce changes for JDK17 tests run

2023-08-02 Thread Petr Ivanov (Jira)
Petr Ivanov created IGNITE-20142:


 Summary: Introduce changes for JDK17 tests run
 Key: IGNITE-20142
 URL: https://issues.apache.org/jira/browse/IGNITE-20142
 Project: Ignite
  Issue Type: Improvement
Reporter: Petr Ivanov
 Fix For: 2.16


Introduce several changes and extend current WAs for ability to run tests under 
JDK17.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Updated] (IGNITE-20142) Introduce changes for JDK17 tests run

2023-08-02 Thread Petr Ivanov (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-20142?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Petr Ivanov updated IGNITE-20142:
-
Labels: ise  (was: )

> Introduce changes for JDK17 tests run
> -
>
> Key: IGNITE-20142
> URL: https://issues.apache.org/jira/browse/IGNITE-20142
> Project: Ignite
>  Issue Type: Improvement
>Reporter: Petr Ivanov
>Priority: Major
>  Labels: ise
> Fix For: 2.16
>
>
> Introduce several changes and extend current WAs for ability to run tests 
> under JDK17.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Created] (IGNITE-20143) GridIntList initial cleanup

2023-08-02 Thread Anton Vinogradov (Jira)
Anton Vinogradov created IGNITE-20143:
-

 Summary: GridIntList initial cleanup
 Key: IGNITE-20143
 URL: https://issues.apache.org/jira/browse/IGNITE-20143
 Project: Ignite
  Issue Type: Sub-task
Reporter: Anton Vinogradov
Assignee: Anton Vinogradov






--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Updated] (IGNITE-20143) GridIntList initial cleanup

2023-08-02 Thread Anton Vinogradov (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-20143?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Anton Vinogradov updated IGNITE-20143:
--
Ignite Flags:   (was: Docs Required,Release Notes Required)

> GridIntList initial cleanup
> ---
>
> Key: IGNITE-20143
> URL: https://issues.apache.org/jira/browse/IGNITE-20143
> Project: Ignite
>  Issue Type: Sub-task
>Reporter: Anton Vinogradov
>Assignee: Anton Vinogradov
>Priority: Major
>




--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Updated] (IGNITE-20143) GridIntList initial cleanup

2023-08-02 Thread Anton Vinogradov (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-20143?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Anton Vinogradov updated IGNITE-20143:
--
Fix Version/s: 2.16

> GridIntList initial cleanup
> ---
>
> Key: IGNITE-20143
> URL: https://issues.apache.org/jira/browse/IGNITE-20143
> Project: Ignite
>  Issue Type: Sub-task
>Reporter: Anton Vinogradov
>Assignee: Anton Vinogradov
>Priority: Major
> Fix For: 2.16
>
>




--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Assigned] (IGNITE-20142) Introduce changes for JDK17 tests run

2023-08-02 Thread Petr Ivanov (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-20142?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Petr Ivanov reassigned IGNITE-20142:


Assignee: Petr Ivanov

> Introduce changes for JDK17 tests run
> -
>
> Key: IGNITE-20142
> URL: https://issues.apache.org/jira/browse/IGNITE-20142
> Project: Ignite
>  Issue Type: Improvement
>Reporter: Petr Ivanov
>Assignee: Petr Ivanov
>Priority: Major
>  Labels: ise
> Fix For: 2.16
>
>
> Introduce several changes and extend current WAs for ability to run tests 
> under JDK17.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Updated] (IGNITE-20143) GridIntList & GridLongList initial cleanup

2023-08-02 Thread Anton Vinogradov (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-20143?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Anton Vinogradov updated IGNITE-20143:
--
Summary: GridIntList & GridLongList initial cleanup  (was: GridIntList 
initial cleanup)

> GridIntList & GridLongList initial cleanup
> --
>
> Key: IGNITE-20143
> URL: https://issues.apache.org/jira/browse/IGNITE-20143
> Project: Ignite
>  Issue Type: Sub-task
>Reporter: Anton Vinogradov
>Assignee: Anton Vinogradov
>Priority: Major
> Fix For: 2.16
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Commented] (IGNITE-20123) IgniteTxHandler initial cleanup

2023-08-02 Thread Ignite TC Bot (Jira)


[ 
https://issues.apache.org/jira/browse/IGNITE-20123?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17750435#comment-17750435
 ] 

Ignite TC Bot commented on IGNITE-20123:


{panel:title=Branch: [pull/10869/head] Base: [master] : No blockers 
found!|borderStyle=dashed|borderColor=#ccc|titleBGColor=#D6F7C1}{panel}
{panel:title=Branch: [pull/10869/head] Base: [master] : No new tests 
found!|borderStyle=dashed|borderColor=#ccc|titleBGColor=#F7D6C1}{panel}
[TeamCity *--> Run :: All* 
Results|https://ci2.ignite.apache.org/viewLog.html?buildId=7279555&buildTypeId=IgniteTests24Java8_RunAll]

> IgniteTxHandler initial cleanup
> ---
>
> Key: IGNITE-20123
> URL: https://issues.apache.org/jira/browse/IGNITE-20123
> Project: Ignite
>  Issue Type: Sub-task
>Reporter: Anton Vinogradov
>Assignee: Anton Vinogradov
>Priority: Major
> Fix For: 2.16
>
>  Time Spent: 40m
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Commented] (IGNITE-20112) IgniteTxImplicitSingleStateImpl initial cleanup

2023-08-02 Thread Ignite TC Bot (Jira)


[ 
https://issues.apache.org/jira/browse/IGNITE-20112?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17750440#comment-17750440
 ] 

Ignite TC Bot commented on IGNITE-20112:


{panel:title=Branch: [pull/10867/head] Base: [master] : No blockers 
found!|borderStyle=dashed|borderColor=#ccc|titleBGColor=#D6F7C1}{panel}
{panel:title=Branch: [pull/10867/head] Base: [master] : No new tests 
found!|borderStyle=dashed|borderColor=#ccc|titleBGColor=#F7D6C1}{panel}
[TeamCity *--> Run :: All* 
Results|https://ci2.ignite.apache.org/viewLog.html?buildId=7279783&buildTypeId=IgniteTests24Java8_RunAll]

> IgniteTxImplicitSingleStateImpl initial cleanup
> ---
>
> Key: IGNITE-20112
> URL: https://issues.apache.org/jira/browse/IGNITE-20112
> Project: Ignite
>  Issue Type: Sub-task
>Reporter: Anton Vinogradov
>Assignee: Anton Vinogradov
>Priority: Major
> Fix For: 2.16
>
>  Time Spent: 20m
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Commented] (IGNITE-19904) Assertion in defragmentation

2023-08-02 Thread Ignite TC Bot (Jira)


[ 
https://issues.apache.org/jira/browse/IGNITE-19904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17750461#comment-17750461
 ] 

Ignite TC Bot commented on IGNITE-19904:


{panel:title=Branch: [pull/10866/head] Base: [master] : No blockers 
found!|borderStyle=dashed|borderColor=#ccc|titleBGColor=#D6F7C1}{panel}
{panel:title=Branch: [pull/10866/head] Base: [master] : No new tests 
found!|borderStyle=dashed|borderColor=#ccc|titleBGColor=#F7D6C1}{panel}
[TeamCity *--> Run :: All* 
Results|https://ci2.ignite.apache.org/viewLog.html?buildId=7280731&buildTypeId=IgniteTests24Java8_RunAll]

> Assertion in defragmentation
> 
>
> Key: IGNITE-19904
> URL: https://issues.apache.org/jira/browse/IGNITE-19904
> Project: Ignite
>  Issue Type: Bug
>Affects Versions: 2.12
>Reporter: Vladimir Steshin
>Assignee: Vladimir Steshin
>Priority: Major
>  Labels: ise
> Fix For: 2.16
>
> Attachments: default-config.xml, failure2.16_with_thread_dump.log, 
> failure_with_root_npe_cause.log, ignite.log, jvm.opts
>
>  Time Spent: 50m
>  Remaining Estimate: 0h
>
> Defragmentaion fails with:
> {code:java}
> java.lang.AssertionError: Invalid state. Type is 0! pageId = 0001000d00024cbf
>   at 
> org.apache.ignite.internal.processors.cache.persistence.pagemem.PageMemoryImpl.copyPageForCheckpoint(PageMemoryImpl.java:1359)
>  ~[ignite-core-2.16.0-SNAPSHOT.jar:2.16.0-SNAPSHOT]
>   at 
> org.apache.ignite.internal.processors.cache.persistence.pagemem.PageMemoryImpl.checkpointWritePage(PageMemoryImpl.java:1277)
>  ~[ignite-core-2.16.0-SNAPSHOT.jar:2.16.0-SNAPSHOT]
>   at 
> org.apache.ignite.internal.processors.cache.persistence.checkpoint.CheckpointPagesWriter.writePages(CheckpointPagesWriter.java:208)
>  ~[ignite-core-2.16.0-SNAPSHOT.jar:2.16.0-SNAPSHOT]
>   at 
> org.apache.ignite.internal.processors.cache.persistence.checkpoint.CheckpointPagesWriter.run(CheckpointPagesWriter.java:150)
>  ~[ignite-core-2.16.0-SNAPSHOT.jar:2.16.0-SNAPSHOT]
> {code}
> Difficult to write a test. Can't reproduce on my computers :(. Flackly 
> appears on a server (4 core x 4 cpu) with 100G of the test cache data and 
> million+ pages to checkpoint during defragmentation. More often, this occurs 
> with pageSize 1024 (to produce more pages).
> Regarding my diagnostic build, I suppose that a fresh, empty page is caught 
> in defragmentation. Here is a page dump with test-expented PAGE_OVERHEAD 
> (=64) and same error a bit before copyPageForCheckpoint():
> {code:java}
> org.apache.ignite.IgniteException: Wrong page type in checkpointWritePage1. 
> Page: Data region = 'defragPartitionsDataRegion'.
>  FullPageId [pageId=281878703760205, effectivePageId=403727049549, 
> grpId=-1368047378].
>  PageDump = page_id: 281878703760205, rel_id: 48603, cache_id: -1368047378, 
> pin: 0, lock: 65536, tmp_buf: 72057594037927935, test_val: 1. data_hex: 
> 0

[jira] [Created] (IGNITE-20145) Update maven surefire version

2023-08-02 Thread Nikita Amelchev (Jira)
Nikita Amelchev created IGNITE-20145:


 Summary: Update maven surefire version
 Key: IGNITE-20145
 URL: https://issues.apache.org/jira/browse/IGNITE-20145
 Project: Ignite
  Issue Type: Task
Reporter: Nikita Amelchev
Assignee: Nikita Amelchev


There are many useful updates in new versions, such as 
[SUREFIRE-1426|https://issues.apache.org/jira/browse/SUREFIRE-1426]. This fixes 
the exit code in case of a fork failure and is needed for proper failure 
handling in CI tools.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Created] (IGNITE-20144) Update maven surefire version

2023-08-02 Thread Nikita Amelchev (Jira)
Nikita Amelchev created IGNITE-20144:


 Summary: Update maven surefire version
 Key: IGNITE-20144
 URL: https://issues.apache.org/jira/browse/IGNITE-20144
 Project: Ignite
  Issue Type: Task
Reporter: Nikita Amelchev
Assignee: Nikita Amelchev


There are many useful updates in new versions, such as 
[SUREFIRE-1426|https://issues.apache.org/jira/browse/SUREFIRE-1426]. This fixes 
the exit code in case of a fork failure and is needed for proper failure 
handling in CI tools.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Updated] (IGNITE-20145) Update maven surefire version

2023-08-02 Thread Nikita Amelchev (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-20145?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Nikita Amelchev updated IGNITE-20145:
-
Labels: ise  (was: )

> Update maven surefire version
> -
>
> Key: IGNITE-20145
> URL: https://issues.apache.org/jira/browse/IGNITE-20145
> Project: Ignite
>  Issue Type: Task
>Reporter: Nikita Amelchev
>Assignee: Nikita Amelchev
>Priority: Major
>  Labels: ise
>
> There are many useful updates in new versions, such as 
> [SUREFIRE-1426|https://issues.apache.org/jira/browse/SUREFIRE-1426]. This 
> fixes the exit code in case of a fork failure and is needed for proper 
> failure handling in CI tools.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Commented] (IGNITE-19904) Assertion in defragmentation

2023-08-02 Thread Ignite TC Bot (Jira)


[ 
https://issues.apache.org/jira/browse/IGNITE-19904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17750481#comment-17750481
 ] 

Ignite TC Bot commented on IGNITE-19904:


{panel:title=Branch: [pull/10866/head] Base: [master] : No blockers 
found!|borderStyle=dashed|borderColor=#ccc|titleBGColor=#D6F7C1}{panel}
{panel:title=Branch: [pull/10866/head] Base: [master] : No new tests 
found!|borderStyle=dashed|borderColor=#ccc|titleBGColor=#F7D6C1}{panel}
[TeamCity *--> Run :: All* 
Results|https://ci2.ignite.apache.org/viewLog.html?buildId=7280731&buildTypeId=IgniteTests24Java8_RunAll]

> Assertion in defragmentation
> 
>
> Key: IGNITE-19904
> URL: https://issues.apache.org/jira/browse/IGNITE-19904
> Project: Ignite
>  Issue Type: Bug
>Affects Versions: 2.12
>Reporter: Vladimir Steshin
>Assignee: Vladimir Steshin
>Priority: Major
>  Labels: ise
> Fix For: 2.16
>
> Attachments: default-config.xml, failure2.16_with_thread_dump.log, 
> failure_with_root_npe_cause.log, ignite.log, jvm.opts
>
>  Time Spent: 50m
>  Remaining Estimate: 0h
>
> Defragmentaion fails with:
> {code:java}
> java.lang.AssertionError: Invalid state. Type is 0! pageId = 0001000d00024cbf
>   at 
> org.apache.ignite.internal.processors.cache.persistence.pagemem.PageMemoryImpl.copyPageForCheckpoint(PageMemoryImpl.java:1359)
>  ~[ignite-core-2.16.0-SNAPSHOT.jar:2.16.0-SNAPSHOT]
>   at 
> org.apache.ignite.internal.processors.cache.persistence.pagemem.PageMemoryImpl.checkpointWritePage(PageMemoryImpl.java:1277)
>  ~[ignite-core-2.16.0-SNAPSHOT.jar:2.16.0-SNAPSHOT]
>   at 
> org.apache.ignite.internal.processors.cache.persistence.checkpoint.CheckpointPagesWriter.writePages(CheckpointPagesWriter.java:208)
>  ~[ignite-core-2.16.0-SNAPSHOT.jar:2.16.0-SNAPSHOT]
>   at 
> org.apache.ignite.internal.processors.cache.persistence.checkpoint.CheckpointPagesWriter.run(CheckpointPagesWriter.java:150)
>  ~[ignite-core-2.16.0-SNAPSHOT.jar:2.16.0-SNAPSHOT]
> {code}
> Difficult to write a test. Can't reproduce on my computers :(. Flackly 
> appears on a server (4 core x 4 cpu) with 100G of the test cache data and 
> million+ pages to checkpoint during defragmentation. More often, this occurs 
> with pageSize 1024 (to produce more pages).
> Regarding my diagnostic build, I suppose that a fresh, empty page is caught 
> in defragmentation. Here is a page dump with test-expented PAGE_OVERHEAD 
> (=64) and same error a bit before copyPageForCheckpoint():
> {code:java}
> org.apache.ignite.IgniteException: Wrong page type in checkpointWritePage1. 
> Page: Data region = 'defragPartitionsDataRegion'.
>  FullPageId [pageId=281878703760205, effectivePageId=403727049549, 
> grpId=-1368047378].
>  PageDump = page_id: 281878703760205, rel_id: 48603, cache_id: -1368047378, 
> pin: 0, lock: 65536, tmp_buf: 72057594037927935, test_val: 1. data_hex: 
> 0

[jira] [Commented] (IGNITE-18302) ignite-spring-sessions: IgniteSession serialization drags its parent class

2023-08-02 Thread Andrey Novikov (Jira)


[ 
https://issues.apache.org/jira/browse/IGNITE-18302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17750556#comment-17750556
 ] 

Andrey Novikov commented on IGNITE-18302:
-

Let's move IgniteSession class to upper level as it will be used as value 
object in SQL. 
https://github.com/apache/ignite-extensions/pull/222#discussion_r1281232708 

> ignite-spring-sessions: IgniteSession serialization drags its parent class
> --
>
> Key: IGNITE-18302
> URL: https://issues.apache.org/jira/browse/IGNITE-18302
> Project: Ignite
>  Issue Type: Bug
>  Components: extensions
>Reporter: Alexandr Shapkin
>Assignee: Alexandr Shapkin
>Priority: Major
>
> In short, there is a bug in ignite-spring-session-ext implementation.
> We store IgniteIndexedSessionRepository${{{}IgniteSession{}}} in the cache, 
> but it’s an internal non-static class, having a reference to the parent 
> [\{{{}IgniteIndexedSessionRepository{}}}] indirectly.
> Hence, during the serialization Ignite also writes {{{}name=this$0, 
> type=Object, fieldId=0xCBDD23AA (-874699862){}}}, which is the reference to 
> {{{}IgniteIndexedSessionRepository{}}}. That leads to the following issues:
>  * we are serializing and saving internal utility data, like}} Ignite 
> ignite{}}}, {{private IndexResolver indexResolver}} etc
>  * one of the IgniteIndexedSessionRepository’s fields is IgniteCache itself - 
> {{IgniteCache sessions}} that basically keeps every 
> session so far leading to a StackOverflow error after some time. 
>  
> {code:java}
> [2022-11-25T17:27:29,268][ERROR][sys-stripe-0-#1%USERS_IGNITE%][GridCacheIoManager]
>  Failed processing message [senderId=0f0ca915-d6cd-4580-92a3-1fbc3d2a5722, 
> msg=GridNearSingleGetResponse [futId=1669397231378, res=-547701325, 
> topVer=null, err=null, flags=0]] 2java.lang.StackOverflowError: null 3 at 
> org.apache.ignite.internal.marshaller.optimized.OptimizedMarshallerUtils.descriptorFromCache(OptimizedMarshallerUtils.java:328)
>  ~[ignite-core-8.8.22.jar:8.8.22] 4 at 
> org.apache.ignite.internal.marshaller.optimized.OptimizedMarshallerUtils.classDescriptor(OptimizedMarshallerUtils.java:273)
>  ~[ignite-core-8.8.22.jar:8.8.22] 5 at 
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:354)
>  ~[ignite-core-8.8.22.jar:8.8.22] 6 at 
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:211)
>  ~[ignite-core-8.8.22.jar:8.8.22] 7 at 
> java.io.ObjectInputStream.readObject(ObjectInputStream.java:480) ~[?:?] 8 at 
> java.io.ObjectInputStream.readObject(ObjectInputStream.java:447) ~[?:?] 9 at 
> org.apache.ignite.internal.processors.cache.GridCacheProxyImpl.readExternal(GridCacheProxyImpl.java:1662)
>  ~[ignite-core-8.8.22.jar:8.8.22] 10 at 
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readExternalizable(OptimizedObjectInputStream.java:569)
>  ~[ignite-core-8.8.22.jar:8.8.22] 11 at 
> org.apache.ignite.internal.marshaller.optimized.OptimizedClassDescriptor.read(OptimizedClassDescriptor.java:979)
>  ~[ignite-core-8.8.22.jar:8.8.22] 12 at 
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:359)
>  ~[ignite-core-8.8.22.jar:8.8.22] 13 at 
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:211)
>  ~[ignite-core-8.8.22.jar:8.8.22] 14 at 
> java.io.ObjectInputStream.readObject(ObjectInputStream.java:480) ~[?:?] 15 at 
> java.io.ObjectInputStream.readObject(ObjectInputStream.java:447) ~[?:?] 16... 
> 17[2022-11-25T17:27:29,276][ERROR][sys-stripe-0-#1%USERS_IGNITE%][] Critical 
> system error detected. Will be handled accordingly to configured handler 
> [hnd=StopNodeOrHaltFailureHandler [tryStop=false, timeout=0, 
> super=AbstractFailureHandler [ignoredFailureTypes=UnmodifiableSet 
> [SYSTEM_WORKER_BLOCKED, SYSTEM_CRITICAL_OPERATION_TIMEOUT]]], 
> failureCtx=FailureContext [type=CRITICAL_ERROR, 
> err=java.lang.StackOverflowError]]{code}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Created] (IGNITE-20146) Add improvements to the catalog associated with the DistributionZone

2023-08-02 Thread Kirill Tkalenko (Jira)
Kirill Tkalenko created IGNITE-20146:


 Summary: Add improvements to the catalog associated with the 
DistributionZone
 Key: IGNITE-20146
 URL: https://issues.apache.org/jira/browse/IGNITE-20146
 Project: Ignite
  Issue Type: Improvement
Reporter: Kirill Tkalenko
Assignee: Kirill Tkalenko
 Fix For: 3.0.0-beta2


During the implementation of IGNITE-20114, it was discovered:
* Filter is not checked when creating/changing a zone;
* It is not possible to specify storage when creating/changing a zone;
* There is no field validation as in the configuration;
* Perhaps there are other differences other than configuration.

Some code refactoring in related classes may be needed.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)