[jira] [Created] (IGNITE-14513) .NET Examples: Invalid test name in the error text
Fedor Malchikov created IGNITE-14513:
-
Summary: .NET Examples: Invalid test name in the error text
Key: IGNITE-14513
URL: https://issues.apache.org/jira/browse/IGNITE-14513
Project: Ignite
Issue Type: Bug
Components: examples, platforms
Reporter: Fedor Malchikov
Test: *Thick/Cache/Multi Tiered Cache*
Error message:
{code:java}
Unhandled Exception: System.Exception: Extra nodes detected. ClientReconnect
example should be run without external nodes. at
Apache.Ignite.Examples.Thick.Cache.MultiTieredCache.Program.Main() in
/opt/buildagent/work/fdcb4fb5bbd0f7b3/i2test/var/suite-examples/art-gg-ent/platforms/dotnet/examples/Thick/Cache/MultiTieredCache/Program.cs:line
58
{code}
ClientReconnect instead MultiTieredCache
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (IGNITE-14514) Drop legacy interfaces.
Andrey Mashenkov created IGNITE-14514: - Summary: Drop legacy interfaces. Key: IGNITE-14514 URL: https://issues.apache.org/jira/browse/IGNITE-14514 Project: Ignite Issue Type: Improvement Reporter: Andrey Mashenkov Fix For: 3.0.0-alpha2 DirectBufferCleaner was introduced for versions < Java9 to cleanup buffer via hidden Unsafe method. It is useless for Java9+ and ByteBuffer can be cleaned via Unsafe direct call. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (IGNITE-14515) Support MissingSwitchDefaultCheck checkstyle check
Maxim Muzafarov created IGNITE-14515: Summary: Support MissingSwitchDefaultCheck checkstyle check Key: IGNITE-14515 URL: https://issues.apache.org/jira/browse/IGNITE-14515 Project: Ignite Issue Type: Task Reporter: Maxim Muzafarov Assignee: Maxim Muzafarov Fix For: 2.11 Every switch statement should include a default case. The break in the default case is redundant, but it prevents a fall-through error if later another case is added. https://www.oracle.com/java/technologies/javase/codeconventions-statements.html#468 -- This message was sent by Atlassian Jira (v8.3.4#803005)
Access to Apache Ignite Wiki
Hi, igniters! Please grant me access to Ignite Wiki to help with editing and updating info. It is my JIRA account v.pyatkov -- Vladislav Pyatkov
[jira] [Created] (IGNITE-14516) WAL can continue to log into archived segment if auto rollover enabled
Nikolay Izhikov created IGNITE-14516:
Summary: WAL can continue to log into archived segment if auto
rollover enabled
Key: IGNITE-14516
URL: https://issues.apache.org/jira/browse/IGNITE-14516
Project: Ignite
Issue Type: Improvement
Reporter: Nikolay Izhikov
Currently, there is a possibility that WAL will continue to log into an
archived segment after node restart.
Bug scenario:
# {{WalAutoArchiveAfterInactivity}} enabled.
# Node is stopping with the {{G.stop("ign", false")}}.
# T0: Performs last checkpoint write some records to current WAL
segment({{Segment#0}}) and freeze.
# T1: Auto rollover happens in
{{FileWriteAheadLogManager#checkWalRolloverRequiredDuringInactivityPeriod}}.
# T2({{FileArchiver}}) Archives {{Segment#0}}.
After restart, WAL will continue to log into {{Segment#0}} which leads that
newly logged data will not be archived after rollover because {{FileArchiver}}
will wait for {{Segment#2}} in
{{SegmentCurrentStateStorage#waitNextSegmentForArchivation}}.
PR to highlight error - https://github.com/apache/ignite/pull/8997
It seems we should to continue to log into the next segment if {{curAbsIdx ==
lastArchivedSegment}} on start.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
Re: Model of permissions for Ignite 3
Hi! Scheduled open call for Friday
https://www.meetup.com/Moscow-Apache-Ignite-Meetup/events/277518672/
Please join to see the zoom link. Consulted with Denis Garus and put the
topic as "Security", cause it's definitely wider than just permissions.
Cheers!
пн, 12 апр. 2021 г. в 09:25, Alexei Scherbakov :
> +1
>
> We should rethink the security model in Ignite 3 and have a default RBAC
> based implementation, from my point of view.
> By default, no code should be written to enable and use it.
>
> Let's schedule a meeting and discuss the details.
>
> вс, 11 апр. 2021 г. в 19:43, Denis Garus :
>
> > Andrey, Alexey thank you for the feedback!
> >
> > > I suggest decoupling authentication from authorization
> >
> > Yes, it would be useful. Existing SecuritySubject and SecurityContext
> > require reworking.
> >
> > > I think it would be great to have a default implementation of
> > user-role-permission model
> >
> > Completely agree it is a cool idea. Ignite should have more default
> > implementation referred to security.
> >
> > > Should we have a community meeting to discuss this?
> >
> > Yes, it would be great.
> > The wish list for Ignite 3 does not contain security improvement that,
> > IMHO, is wrong.
> > We should fix that oversight on early-stage Ignite 3 development.
> >
> > пт, 9 апр. 2021 г. в 18:47, Alexey Goncharuk >:
> >
> > > Hello Denis, Andrey, Igniters,
> > >
> > > Why don't we take a step further in improving the security model in
> > Ignite
> > > 3? I think it would be great to have a default implementation of
> > > user-role-permission model in Ignite to be on par with security models
> of
> > > widely-used databases. This will complement community efforts in making
> > > most of the Ignite 3 behavior to be changeable in runtime.
> > >
> > > WDYT? Should we have a community meeting to discuss this?
> > >
> > >
> > > чт, 8 апр. 2021 г. в 23:54, Andrey Kuznetsov :
> > >
> > > > Hi Denis!
> > > >
> > > > The idea and prototype look great.
> > > >
> > > > I'd like to highlight one arguable point. Default authorization
> > > > implementation still assumes there are permissions provided in
> > > > SecuritySubject. In turn, authentication is still responsible for
> > filling
> > > > these permissions. I suggest decoupling authentication from
> > > authorization,
> > > > so that GridSecurityProcessor implementation is fully responsible for
> > > > obtaining permissions for SecuritySubject given on authorization. In
> > > > particular, implementation can choose an existing behavior of
> bundling
> > > > permissions with SecuritySubject.
> > > >
> > > > Makes sense?
> > > >
> > > > чт, 8 апр. 2021 г. в 17:52, Denis Garus :
> > > >
> > > > > Sorry, I forgot to point the link
> > > > >
> > > > > 1. https://github.com/apache/ignite/pull/8989
> > > > >
> > > > > чт, 8 апр. 2021 г. в 17:50, Denis Garus :
> > > > >
> > > > > > Hello, Igniters!
> > > > > >
> > > > > > I want to propose to improve the way which we use
> > > > > > to present permissions in Ignite 3.
> > > > > >
> > > > > > The model of permission in Ignite has a set of drawbacks.
> > > > > > The main drawback, IMHO: if you need to add a new permission,
> > > > > > you should change the core module by extended the
> > > 'SecurityPermission'
> > > > > > enum.
> > > > > > An approach like this becomes more challenged if new permission
> is
> > > > > created
> > > > > > for an extension.
> > > > > >
> > > > > > The existing permission model is overcomplicated.
> > > > > > The SecurityPermission enum is divided into four groups,
> > > > > > and to determine whether a security subject has been given
> > > permission,
> > > > > > a plugin developer has to know what the permission group is.
> > > > > > But 'CACHE_CREATE' and 'CACHE_DESTROY' are included in two groups
> > > > (system
> > > > > > operations and cache operations).
> > > > > > When 'CACHE_CREATE' ('CACHE_DESTROY') is treated as system
> > > permission,
> > > > > > it applies to all caches. In other cases, when 'CACHE_CREATE'
> > > > > > ('CACHE_DESTROY') is treated as cache permission,
> > > > > > permission checking is executed with the account of the cache
> name.
> > > > > > IMHO, this logic is hard to understand.
> > > > > > There is no ability to represent compound operation as single
> > > > permission
> > > > > > and so on.
> > > > > >
> > > > > >
> > > > > > So I would like to suggest using a permission model that is based
> > on
> > > > > > 'java.security.Permission'.
> > > > > > I prepared the concept [1] of how this model could look in
> Ignite.
> > > > > > Classes 'CachePermission', 'ComputePermission', and
> > > 'ServicePermission'
> > > > > > represent cache, compute,
> > > > > > and service permissions accordingly, allow wildcards, for
> example,
> > > > > > "org.apache.ignite.internal.*".
> > > > > > Class 'IgniteClusterPermission' represents permission without
> > > actions.
> > > > > > Interface 'GridSecurityProcessor' has a default implementation of
> > the
> > >
[jira] [Created] (IGNITE-14517) Inconsistent behavior of the method NetworkCluster#allMembers
Alexey Scherbakov created IGNITE-14517: -- Summary: Inconsistent behavior of the method NetworkCluster#allMembers Key: IGNITE-14517 URL: https://issues.apache.org/jira/browse/IGNITE-14517 Project: Ignite Issue Type: Bug Reporter: Alexey Scherbakov This method reports invalid number of alive nodes when a node is stopped gracefully. The scenario: # Start the cluster of 3 nodes: n1, n2, n3. # Request n2 NetworkCluster#allMembers. It will return 3. # Stop node n1. # Request n2 NetworkCluster#allMembers again. It will return 3, but should return 2. Here is a failed test from my working branch [1][2] I've looked into scalecube code and found out the node is stuck in the io.scalecube.cluster.membership.MembershipProtocolImpl#membershipTable having MemberStatus.LEAVING state. The possible fix would avoid using cluster.members at all and instead use events to maintain local topology. Events seem to work fine. [1] https://github.com/gridgain/apache-ignite-3/blob/ignite-13885/modules/network/src/integrationTest/java/org/apache/ignite/network/scalecube/ITScaleCubeNetworkClusterMessagingTest.java [2] https://ci.ignite.apache.org/viewLog.html?buildId=5963158&tab=buildResultsDiv&buildTypeId=ignite3_Tests_IntegrationTests&branch_ignite3_Tests=pull%2F78 -- This message was sent by Atlassian Jira (v8.3.4#803005)
Re: [DISCUSS] IEP-71 Public API for secondary index search
Hi, Andrey! Am I right, that you mean this ticket [1] *IGNITE-12291 Create controllable paged query requests / responses for TextQuery similar to current SQL result processing*, when talked about incorrect limit work for TextQueries? [1] https://issues.apache.org/jira/browse/IGNITE-12291 On Thu, Apr 8, 2021 at 4:32 PM Maksim Timonin wrote: > Hi, Andrey! > > >> ScanQuery, TextQuery and partially SQL query share the same > infrastructure > I think I understand what you mean. I debug query processing and now agree > that it's a nice idea to try to reuse the infrastructure of scan and text > queries. Also as I can see there already Reducer functionality exists, so I > hope we can use that. I'm not absolutely confident now that it will work > fine, but I'm going to start there. Thanks for pointing me this direction! > > >> I don't like the idea a user code will be executed inside BTree > operation > On the confluence page I've shown that a predicate passes as > TreeRowClosure. In this case you're right, any exception in a predicate > will lead to a CorruptedTreeException. But I see another legal way to > implement the predicate operation. BPlusTree.find accepts the X param that > passed to IO.getRow(). As I understand this param helps to control how much > returned row is filled. Then we can use it to return an object that > contains only basic info - link, pageAddr, offset. Then predicate operation > will be applied on the higher level on a cursor returned by a tree (like > H2TreeIndex does). It's safe to run user code there, we can handle > exceptions there. > > > > On Wed, Apr 7, 2021 at 4:46 PM Andrey Mashenkov < > andrey.mashen...@gmail.com> wrote: > >> Maksim, >> >> The ScanQuery API provides a filter as >> > param that for case of index query should be splitted on such >> conditions. >> > It looks like a non-trivial task. >> > >> ScanQuery, TextQuery and partially SQL query share the same >> infrastructure. >> I've thought we could extend, improve and reuse some ScanQuery code that >> already works fine: map query on topology, IO, batching. >> Add IndexCondition alongside the Filter, and abstract query executor from >> source (primary and secondary Indexes). >> Add a sorted merge algorithm to the query merge stage. It can be very >> useful also for TextQueries that suffers from the absence of sorted merge >> and a "limit' condition work incorrectly. >> >> If you think it will be too hard than creating from scratch, I'm ok. >> >> 3. Ignite creates a proxy object that is filled with objects that are >> > inlined. If a user tries to access a field that isn't inlined or not >> > indexed, then deserialization will start and Ignite will log.warn() >> about >> > that. >> > >> Agree, this can be faster. >> I don't like the idea a user code will be executed inside BTree operation, >> any exception can cause FailureHandler triggering and stop the node. >> >> There is one more thing that could be improved. >> ScanQuery now iterates over per-partition PK Hash index trees and has >> performance issues on a small grid with a large number of partitions. >> So, there are many partitions on every node and many trees should be >> scanned. >> In this case scan over a secondary index gives significant boots even if >> every row is materialized, because we need to traverse over a single tree >> per-node. >> Having the ability to run a ScanQuery over a secondary index (if one >> exists) instead of PK Hash will be great. >> >> >> On Wed, Apr 7, 2021 at 11:18 AM Maksim Timonin >> wrote: >> >> > Hi, Andrey! >> > >> > Thanks for the review and your comments! >> > >> > >> Is it possible to extend ScanQuery functionality to pass index >> condition >> > I investigated this way and see some issues: >> > 1. Querying of indexes is not a scan actually. It's >> > a tree traverse (predicate operation is an exclusion, other operations >> like >> > gt, lt, min, max have explicit boundaries). An index query consists of >> > conditions that match an index structure. In general for a multi-key >> index >> > there can be multiple conditions. The ScanQuery API provides a filter as >> > param that for case of index query should be splitted on such >> conditions. >> > It looks like a non-trivial task. >> > 2. Querying of an index requires a sorted result, while The ScanQuery >> > doesn't matter about that. So there will be a different behavior of the >> > iterator for scanning a cache and querying indexes. It's not much to >> > implement I think, but it can make ScanQuery unclear for a user. >> > >> > Maybe it's a point to separate traverse (gt, lt, in, etc...) and scan >> > (predicate) index operations to different API. So there still will be a >> new >> > query type for the traversing. >> > >> > But we will introduce some inheritors for ScanQuery, like TableScanQuery >> > and IndexScanQuery, for scan and filter. Then the question is about >> > ordering, Cache and Table scans aren't ordered, but Index is. Then we >> can >> > introduce an o
[jira] [Created] (IGNITE-14518) Generated PKG_INFO of package doesn't contains license info and long description
Ivan Daschinsky created IGNITE-14518: Summary: Generated PKG_INFO of package doesn't contains license info and long description Key: IGNITE-14518 URL: https://issues.apache.org/jira/browse/IGNITE-14518 Project: Ignite Issue Type: Bug Components: python, thin client Reporter: Ivan Daschinsky -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (IGNITE-14519) Add ability to remove event handlers
Aleksandr Polovtcev created IGNITE-14519:
Summary: Add ability to remove event handlers
Key: IGNITE-14519
URL: https://issues.apache.org/jira/browse/IGNITE-14519
Project: Ignite
Issue Type: Improvement
Components: networking
Reporter: Aleksandr Polovtcev
{{Both TopologyService and }}{{MessagingService}} allow registering event
handlers. They should also allow removing registered handlers by dynamically
loaded components (e.g. caches) in order to avoid memory leaks.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
Re: [DISCUSS] IEP-71 Public API for secondary index search
How does this fit with the current IndexingSpi? Superficially they appear to do very similar things? Regards, Stephen > On 6 Apr 2021, at 14:13, Maksim Timonin wrote: > > Hi, Igniters! > > I'd like to propose a new feature - opportunity to query and create indexes > from public API. > > It will help in some cases, where: > 1. SQL is not applicable by design of user application; > 2. Where IndexScan is preferable than ScanQuery for performance reasons; > 3. Functional indexes are required. > > Also it'll be great to have a transactional support for such queries, like > the "select for update" query provides. But I don't dig there much. It will > be a next step if this API will be implemented. > > I've prepared an IEP-71 for that [1] with more details. Please share your > thoughts. > > > [1] > https://cwiki.apache.org/confluence/display/IGNITE/IEP-71+Public+API+for+secondary+index+search
Re: [DISCUSS] IEP-71 Public API for secondary index search
Hi Stephen! Please have a look at the QueryProcessing paragraph [1]. I've described why IndexingSpi doesn't fit us well. [1] https://cwiki.apache.org/confluence/display/IGNITE/IEP-71+Public+API+for+secondary+index+search#IEP71PublicAPIforsecondaryindexsearch-2)QueryProcessing On Mon, Apr 12, 2021 at 1:24 PM Stephen Darlington < stephen.darling...@gridgain.com> wrote: > How does this fit with the current IndexingSpi? Superficially they appear > to do very similar things? > > Regards, > Stephen > > > On 6 Apr 2021, at 14:13, Maksim Timonin wrote: > > > > Hi, Igniters! > > > > I'd like to propose a new feature - opportunity to query and create > indexes > > from public API. > > > > It will help in some cases, where: > > 1. SQL is not applicable by design of user application; > > 2. Where IndexScan is preferable than ScanQuery for performance reasons; > > 3. Functional indexes are required. > > > > Also it'll be great to have a transactional support for such queries, > like > > the "select for update" query provides. But I don't dig there much. It > will > > be a next step if this API will be implemented. > > > > I've prepared an IEP-71 for that [1] with more details. Please share your > > thoughts. > > > > > > [1] > > > https://cwiki.apache.org/confluence/display/IGNITE/IEP-71+Public+API+for+secondary+index+search > > >
Re: [DISCUSS] IEP-71 Public API for secondary index search
Maksim, Am I right, that you mean this ticket [1] *IGNITE-12291 Create controllable > paged query requests / responses for TextQuery similar to current SQL > result processing*, when talked about incorrect limit work for > TextQueries? > Yes, sure, that's it. On Mon, Apr 12, 2021 at 12:15 PM Maksim Timonin wrote: > Hi, Andrey! > > Am I right, that you mean this ticket [1] *IGNITE-12291 Create > controllable paged query requests / responses for TextQuery similar to > current SQL result processing*, when talked about incorrect limit work > for TextQueries? > > > [1] https://issues.apache.org/jira/browse/IGNITE-12291 > > On Thu, Apr 8, 2021 at 4:32 PM Maksim Timonin > wrote: > >> Hi, Andrey! >> >> >> ScanQuery, TextQuery and partially SQL query share the same >> infrastructure >> I think I understand what you mean. I debug query processing and now >> agree that it's a nice idea to try to reuse the infrastructure of scan and >> text queries. Also as I can see there already Reducer functionality exists, >> so I hope we can use that. I'm not absolutely confident now that it will >> work fine, but I'm going to start there. Thanks for pointing me this >> direction! >> >> >> I don't like the idea a user code will be executed inside BTree >> operation >> On the confluence page I've shown that a predicate passes as >> TreeRowClosure. In this case you're right, any exception in a predicate >> will lead to a CorruptedTreeException. But I see another legal way to >> implement the predicate operation. BPlusTree.find accepts the X param that >> passed to IO.getRow(). As I understand this param helps to control how much >> returned row is filled. Then we can use it to return an object that >> contains only basic info - link, pageAddr, offset. Then predicate operation >> will be applied on the higher level on a cursor returned by a tree (like >> H2TreeIndex does). It's safe to run user code there, we can handle >> exceptions there. >> >> >> >> On Wed, Apr 7, 2021 at 4:46 PM Andrey Mashenkov < >> andrey.mashen...@gmail.com> wrote: >> >>> Maksim, >>> >>> The ScanQuery API provides a filter as >>> > param that for case of index query should be splitted on such >>> conditions. >>> > It looks like a non-trivial task. >>> > >>> ScanQuery, TextQuery and partially SQL query share the same >>> infrastructure. >>> I've thought we could extend, improve and reuse some ScanQuery code that >>> already works fine: map query on topology, IO, batching. >>> Add IndexCondition alongside the Filter, and abstract query executor from >>> source (primary and secondary Indexes). >>> Add a sorted merge algorithm to the query merge stage. It can be very >>> useful also for TextQueries that suffers from the absence of sorted merge >>> and a "limit' condition work incorrectly. >>> >>> If you think it will be too hard than creating from scratch, I'm ok. >>> >>> 3. Ignite creates a proxy object that is filled with objects that are >>> > inlined. If a user tries to access a field that isn't inlined or not >>> > indexed, then deserialization will start and Ignite will log.warn() >>> about >>> > that. >>> > >>> Agree, this can be faster. >>> I don't like the idea a user code will be executed inside BTree >>> operation, >>> any exception can cause FailureHandler triggering and stop the node. >>> >>> There is one more thing that could be improved. >>> ScanQuery now iterates over per-partition PK Hash index trees and has >>> performance issues on a small grid with a large number of partitions. >>> So, there are many partitions on every node and many trees should be >>> scanned. >>> In this case scan over a secondary index gives significant boots even if >>> every row is materialized, because we need to traverse over a single tree >>> per-node. >>> Having the ability to run a ScanQuery over a secondary index (if one >>> exists) instead of PK Hash will be great. >>> >>> >>> On Wed, Apr 7, 2021 at 11:18 AM Maksim Timonin >>> wrote: >>> >>> > Hi, Andrey! >>> > >>> > Thanks for the review and your comments! >>> > >>> > >> Is it possible to extend ScanQuery functionality to pass index >>> condition >>> > I investigated this way and see some issues: >>> > 1. Querying of indexes is not a scan actually. It's >>> > a tree traverse (predicate operation is an exclusion, other operations >>> like >>> > gt, lt, min, max have explicit boundaries). An index query consists of >>> > conditions that match an index structure. In general for a multi-key >>> index >>> > there can be multiple conditions. The ScanQuery API provides a filter >>> as >>> > param that for case of index query should be splitted on such >>> conditions. >>> > It looks like a non-trivial task. >>> > 2. Querying of an index requires a sorted result, while The ScanQuery >>> > doesn't matter about that. So there will be a different behavior of the >>> > iterator for scanning a cache and querying indexes. It's not much to >>> > implement I think, but it can make ScanQuery unclear for a user. >>> > >>> > M
Re: [DISCUSS] IEP-71 Public API for secondary index search
Is this a replacement for IndexingSpi? Put bluntly, do we deprecate (and remove) it? Or do you see them as complimentary? > On 12 Apr 2021, at 11:29, Maksim Timonin wrote: > > Hi Stephen! > > Please have a look at the QueryProcessing paragraph [1]. I've described > why IndexingSpi doesn't fit us well. > > [1] > https://cwiki.apache.org/confluence/display/IGNITE/IEP-71+Public+API+for+secondary+index+search#IEP71PublicAPIforsecondaryindexsearch-2)QueryProcessing > > On Mon, Apr 12, 2021 at 1:24 PM Stephen Darlington < > stephen.darling...@gridgain.com> wrote: > >> How does this fit with the current IndexingSpi? Superficially they appear >> to do very similar things? >> >> Regards, >> Stephen >> >>> On 6 Apr 2021, at 14:13, Maksim Timonin wrote: >>> >>> Hi, Igniters! >>> >>> I'd like to propose a new feature - opportunity to query and create >> indexes >>> from public API. >>> >>> It will help in some cases, where: >>> 1. SQL is not applicable by design of user application; >>> 2. Where IndexScan is preferable than ScanQuery for performance reasons; >>> 3. Functional indexes are required. >>> >>> Also it'll be great to have a transactional support for such queries, >> like >>> the "select for update" query provides. But I don't dig there much. It >> will >>> be a next step if this API will be implemented. >>> >>> I've prepared an IEP-71 for that [1] with more details. Please share your >>> thoughts. >>> >>> >>> [1] >>> >> https://cwiki.apache.org/confluence/display/IGNITE/IEP-71+Public+API+for+secondary+index+search >> >> >>
Re: [DISCUSS] IEP-71 Public API for secondary index search
Stephen, I don't see a reason to replace or deprecate IndexingSpi. I'm not sure how smbd uses it, but it works now. On Mon, Apr 12, 2021 at 1:42 PM Stephen Darlington < stephen.darling...@gridgain.com> wrote: > Is this a replacement for IndexingSpi? Put bluntly, do we deprecate (and > remove) it? > > Or do you see them as complimentary? > > > On 12 Apr 2021, at 11:29, Maksim Timonin > wrote: > > > > Hi Stephen! > > > > Please have a look at the QueryProcessing paragraph [1]. I've described > > why IndexingSpi doesn't fit us well. > > > > [1] > > > https://cwiki.apache.org/confluence/display/IGNITE/IEP-71+Public+API+for+secondary+index+search#IEP71PublicAPIforsecondaryindexsearch-2)QueryProcessing > > > > On Mon, Apr 12, 2021 at 1:24 PM Stephen Darlington < > > stephen.darling...@gridgain.com> wrote: > > > >> How does this fit with the current IndexingSpi? Superficially they > appear > >> to do very similar things? > >> > >> Regards, > >> Stephen > >> > >>> On 6 Apr 2021, at 14:13, Maksim Timonin > wrote: > >>> > >>> Hi, Igniters! > >>> > >>> I'd like to propose a new feature - opportunity to query and create > >> indexes > >>> from public API. > >>> > >>> It will help in some cases, where: > >>> 1. SQL is not applicable by design of user application; > >>> 2. Where IndexScan is preferable than ScanQuery for performance > reasons; > >>> 3. Functional indexes are required. > >>> > >>> Also it'll be great to have a transactional support for such queries, > >> like > >>> the "select for update" query provides. But I don't dig there much. It > >> will > >>> be a next step if this API will be implemented. > >>> > >>> I've prepared an IEP-71 for that [1] with more details. Please share > your > >>> thoughts. > >>> > >>> > >>> [1] > >>> > >> > https://cwiki.apache.org/confluence/display/IGNITE/IEP-71+Public+API+for+secondary+index+search > >> > >> > >> > > >
Re: [DISCUSS] IEP-71 Public API for secondary index search
Andrey, Thanks! I picked it. On Mon, Apr 12, 2021 at 1:51 PM Maksim Timonin wrote: > Stephen, > > I don't see a reason to replace or deprecate IndexingSpi. I'm not sure how > smbd uses it, but it works now. > > On Mon, Apr 12, 2021 at 1:42 PM Stephen Darlington < > stephen.darling...@gridgain.com> wrote: > >> Is this a replacement for IndexingSpi? Put bluntly, do we deprecate (and >> remove) it? >> >> Or do you see them as complimentary? >> >> > On 12 Apr 2021, at 11:29, Maksim Timonin >> wrote: >> > >> > Hi Stephen! >> > >> > Please have a look at the QueryProcessing paragraph [1]. I've described >> > why IndexingSpi doesn't fit us well. >> > >> > [1] >> > >> https://cwiki.apache.org/confluence/display/IGNITE/IEP-71+Public+API+for+secondary+index+search#IEP71PublicAPIforsecondaryindexsearch-2)QueryProcessing >> > >> > On Mon, Apr 12, 2021 at 1:24 PM Stephen Darlington < >> > stephen.darling...@gridgain.com> wrote: >> > >> >> How does this fit with the current IndexingSpi? Superficially they >> appear >> >> to do very similar things? >> >> >> >> Regards, >> >> Stephen >> >> >> >>> On 6 Apr 2021, at 14:13, Maksim Timonin >> wrote: >> >>> >> >>> Hi, Igniters! >> >>> >> >>> I'd like to propose a new feature - opportunity to query and create >> >> indexes >> >>> from public API. >> >>> >> >>> It will help in some cases, where: >> >>> 1. SQL is not applicable by design of user application; >> >>> 2. Where IndexScan is preferable than ScanQuery for performance >> reasons; >> >>> 3. Functional indexes are required. >> >>> >> >>> Also it'll be great to have a transactional support for such queries, >> >> like >> >>> the "select for update" query provides. But I don't dig there much. It >> >> will >> >>> be a next step if this API will be implemented. >> >>> >> >>> I've prepared an IEP-71 for that [1] with more details. Please share >> your >> >>> thoughts. >> >>> >> >>> >> >>> [1] >> >>> >> >> >> https://cwiki.apache.org/confluence/display/IGNITE/IEP-71+Public+API+for+secondary+index+search >> >> >> >> >> >> >> >> >>
[jira] [Created] (IGNITE-14520) Plugins support in Ignite configuration
Mikhail Filatov created IGNITE-14520:
Summary: Plugins support in Ignite configuration
Key: IGNITE-14520
URL: https://issues.apache.org/jira/browse/IGNITE-14520
Project: Ignite
Issue Type: Task
Reporter: Mikhail Filatov
Assignee: Mikhail Filatov
Add support for plugin configuration with custom macros
It should be possible to add a plugin configuration toIgnite configuration by
specifying template path and plugin parameters with code like:
{% if config.plugins %}
\{% for plugin in config.plugins %}
\{{ misc_utils.generate_plugin_config(plugin['template_path'],
plugin['config'], config) }}
\{% endfor %}
\{% endif %}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (IGNITE-14521) Plugins support in Ignite configuration
Mikhail Filatov created IGNITE-14521:
Summary: Plugins support in Ignite configuration
Key: IGNITE-14521
URL: https://issues.apache.org/jira/browse/IGNITE-14521
Project: Ignite
Issue Type: Task
Reporter: Mikhail Filatov
Assignee: Mikhail Filatov
Add support for plugin configuration with custom macros
It should be possible to add a plugin configuration toIgnite configuration by
specifying template path and plugin parameters with code like:
{% if config.plugins %}
\{% for plugin in config.plugins %}
\{{ misc_utils.generate_plugin_config(plugin['template_path'],
plugin['config'], config) }}
\{% endfor %}
\{% endif %}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (IGNITE-14522) CommunicationManager refactoring
Alexander Lapin created IGNITE-14522: Summary: CommunicationManager refactoring Key: IGNITE-14522 URL: https://issues.apache.org/jira/browse/IGNITE-14522 Project: Ignite Issue Type: Bug Reporter: Alexander Lapin Assignee: Alexander Lapin -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (IGNITE-14523) Support for string.Compare method in Apache Ignite LINQ provider
Oleg Bevz created IGNITE-14523: -- Summary: Support for string.Compare method in Apache Ignite LINQ provider Key: IGNITE-14523 URL: https://issues.apache.org/jira/browse/IGNITE-14523 Project: Ignite Issue Type: New Feature Components: thin client Affects Versions: 2.9.1 Reporter: Oleg Bevz Fix For: 2.9.1 Hello, current Apache Ignite does not support string comparison in LINQ provider (Apache.Ignite.Linq project). For example following modified query insideThinClientQueryExample.ScanQueryExample method will throw Exception: var employees = cache.AsCacheQueryable().Select(x => x.Value).Where(x => string.Compare(x.Name, "Daniel Adams") >= 0); Exception has been thrown by the target of an invocation. System.Reflection.TargetInvocationException System.Object InvokeMethod(System.Object, System.Object[], System.Signature, Boolean) System.Reflection.TargetInvocationException: Exception has been thrown by the target of an invocation. ---> System.NotSupportedException: Method not supported: System.String.(Int32 Compare(System.String, System.String)) at Apache.Ignite.Linq.Impl.MethodVisitor.VisitMethodCall(MethodCallExpression expression, CacheQueryExpressionVisitor visitor) at Apache.Ignite.Linq.Impl.CacheQueryExpressionVisitor.VisitMethodCall(MethodCallExpression expression) at Remotion.Linq.Parsing.ThrowingExpressionVisitor.Visit(Expression expression) at Apache.Ignite.Linq.Impl.CacheQueryExpressionVisitor.VisitBinary(BinaryExpression expression) at Remotion.Linq.Parsing.ThrowingExpressionVisitor.Visit(Expression expression) at Apache.Ignite.Linq.Impl.CacheQueryExpressionVisitor.VisitBinary(BinaryExpression expression) at Remotion.Linq.Parsing.ThrowingExpressionVisitor.Visit(Expression expression) at Apache.Ignite.Linq.Impl.CacheQueryModelVisitor.VisitWhereClause(WhereClause whereClause, Int32 index, Boolean hasGroups) at Apache.Ignite.Linq.Impl.CacheQueryModelVisitor.ProcessGroupings(QueryModel queryModel) at Apache.Ignite.Linq.Impl.CacheQueryModelVisitor.VisitBodyClauses(ObservableCollection`1 bodyClauses, QueryModel queryModel) at Remotion.Linq.QueryModelVisitorBase.VisitQueryModel(QueryModel queryModel) at Apache.Ignite.Linq.Impl.CacheQueryModelVisitor.VisitQueryModel(QueryModel queryModel, Boolean includeAllFields, Boolean copyAliases) at Apache.Ignite.Linq.Impl.CacheQueryModelVisitor.GenerateQuery(QueryModel queryModel) at Apache.Ignite.Linq.Impl.CacheFieldsQueryExecutor.ExecuteCollection[T](QueryModel queryModel) at Remotion.Linq.Clauses.StreamedData.StreamedSequenceInfo.ExecuteQueryModel(QueryModel queryModel, IQueryExecutor executor) at Apache.Ignite.Linq.Impl.CacheFieldsQueryProvider.Execute[TResult](Expression expression) -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (IGNITE-14524) Historical rebalance doesn't work if cache has configured rebalanceDelay
Dmitry Lazurkin created IGNITE-14524: Summary: Historical rebalance doesn't work if cache has configured rebalanceDelay Key: IGNITE-14524 URL: https://issues.apache.org/jira/browse/IGNITE-14524 Project: Ignite Issue Type: Bug Affects Versions: 2.10 Reporter: Dmitry Lazurkin I have big cache with configured rebalanceMode = ASYNC, rebalanceDelay = 10_000ms. Persistence is enabled, maxWalArchiveSize = 10GB. And I passed -DIGNITE_PREFER_WAL_REBALANCE=true and -DIGNITE_PDS_WAL_REBALANCE_THRESHOLD=1 to Ignite. So node should use historical rebalance if there is enough WAL. But it doesn't. After investigation I found that GridDhtPreloader#generateAssignments always get called with exchFut = null, and this method can't set histPartitions without exchFut. I think, that problem in GridCachePartitionExchangeManager (https://github.com/apache/ignite/blob/bc24f6baf3e9b4f98cf98cc5df67fb5deb5ceb6c/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/GridCachePartitionExchangeManager.java#L3486). It doesn't call generateAssignments without forcePreload if rebalanceDelay is configured. Historical rebalance works after removing rebalanceDelay. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (IGNITE-14525) Empty page in docs
Igor Gusev created IGNITE-14525: --- Summary: Empty page in docs Key: IGNITE-14525 URL: https://issues.apache.org/jira/browse/IGNITE-14525 Project: Ignite Issue Type: Improvement Components: documentation Reporter: Igor Gusev [https://ignite.apache.org/docs/latest/installation/] page is empty. There should be some introduction in it, or no page at all. -- This message was sent by Atlassian Jira (v8.3.4#803005)
Re: Access to Apache Ignite Wiki
Hello! You need to create an account on https://cwiki.apache.org/ first. Regards, -- Ilya Kasnacheev пн, 12 апр. 2021 г. в 11:44, Vladislav Pyatkov : > Hi, igniters! > Please grant me access to Ignite Wiki to help with editing and updating > info. > It is my JIRA account v.pyatkov > > -- > Vladislav Pyatkov >
Re: Access to Apache Ignite Wiki
Hi Ilya, Thanks for the note. I created an account with the same login (v.pyatkov) in the Apache wiki. Could you please gran me permissions? On Mon, Apr 12, 2021 at 5:15 PM Ilya Kasnacheev wrote: > Hello! > > You need to create an account on https://cwiki.apache.org/ first. > > Regards, > -- > Ilya Kasnacheev > > > пн, 12 апр. 2021 г. в 11:44, Vladislav Pyatkov : > > > Hi, igniters! > > Please grant me access to Ignite Wiki to help with editing and updating > > info. > > It is my JIRA account v.pyatkov > > > > -- > > Vladislav Pyatkov > > > -- Vladislav Pyatkov
Re: Access to Apache Ignite Wiki
Hello! Permissions granted. Regards, -- Ilya Kasnacheev пн, 12 апр. 2021 г. в 17:22, Vladislav Pyatkov : > Hi Ilya, > > Thanks for the note. > I created an account with the same login (v.pyatkov) in the Apache wiki. > Could you please gran me permissions? > > On Mon, Apr 12, 2021 at 5:15 PM Ilya Kasnacheev > > wrote: > > > Hello! > > > > You need to create an account on https://cwiki.apache.org/ first. > > > > Regards, > > -- > > Ilya Kasnacheev > > > > > > пн, 12 апр. 2021 г. в 11:44, Vladislav Pyatkov : > > > > > Hi, igniters! > > > Please grant me access to Ignite Wiki to help with editing and updating > > > info. > > > It is my JIRA account v.pyatkov > > > > > > -- > > > Vladislav Pyatkov > > > > > > > > -- > Vladislav Pyatkov >
Re: IEP-70: Async Continuation Executor
Stan,
Sorry for the late reply (had a vacation).
> In my ideal world, the users don't configure thread pools, they just have
safe default behavior (async execution)
> and a way to make it fast for one particular function (with annotation or
anything else).
I've
added
testRemoteOperationListenerExecutesOnStripedPoolWhenCustomExecutorIsProvided
to demonstrate this use case.
> I'm OK to proceed with the approach you're suggesting if I haven't
convinced you by now
Are you OK to merge the changes as is (ForkJoinPool#commonPool as the
default executor),
or do we change it to Ignite public pool?
On Mon, Mar 29, 2021 at 11:09 PM Stanislav Lukyanov
wrote:
> But what if I need to have exactly one callback synchronous, and all other
> can be asynchronous?
>
> I would separate two cases: an existing user who wants their old behavior
> back, and a new user that wants to fine tune their app.
> The existing user needs a global "make it all synchronous" switch.
> The new user should only enable the fast-but-dangerous behavior locally,
> exactly where they need it.
>
> In my ideal world, the users don't configure thread pools, they just have
> safe default behavior (async execution)
> and a way to make it fast for one particular function (with annotation or
> anything else).
> Also, this should work in a similar way for different APIs - so I'm trying
> to lay some basis to rework all of these continuous queries and event
> listeners,
> even though they're explicitly mentioned as out of scope for IEP-70.
>
> At the same time, I understand that any change we make now will have pros
> and cons, and we can't make it perfect because of compatibility reasons.
> I'm OK to proceed with the approach you're suggesting if I haven't
> convinced you by now :)
>
> Thanks,
> Stan
>
> > On 29 Mar 2021, at 22:47, Pavel Tupitsyn wrote:
> >
> > Stan,
> >
> > Unfortunately, annotations have a few drawbacks:
> > * Can't configure it globally ("I already use sync callbacks, give me
> back
> > the old behavior in one line")
> > * Can't configure in Spring
> > * Useless in C++ & .NET
> > * You can already specify executor in IgniteFuture#listenAsync, so there
> > seems to be no added value
> >
> >> the only value we really expect the user to set in that property is
> > Runnable::run
> > Not really - there are lots of available options [1].
> > Some apps may already have one or more thread pools that can be used for
> > continuations.
> >
> >> you can't specify Runnable::run in a Spring XML
> > Good point, but creating a class for that is trivial.
> > We can ship a ready-made class and mention it in the docs for simplicity.
> >
> >
> > Globally configurable Executor fits nicely with
> > existing IgniteFuture#listenAsync,
> > not sure why you dislike it.
> >
> >
> > [1]
> >
> https://docs.oracle.com/javase/7/docs/api/java/util/concurrent/Executors.html
> >
> > On Mon, Mar 29, 2021 at 10:23 PM Stanislav Lukyanov <
> stanlukya...@gmail.com>
> > wrote:
> >
> >> Thought about this some more.
> >>
> >> I agree that we need to be able to switch to synchronous listeners when
> >> it's critical for performance.
> >> However, I don't like to introduce an Executor property for that. In
> fact,
> >> the only value
> >> we really expect the user to set in that property is Runnable::run -
> seems
> >> to be an overkill to have accept an Executor for that.
> >> Furthermore, you can't specify Runnable::run in a Spring XML, can you?
> >>
> >> I'm thinking that maybe we should go the annotation route here.
> >> Let's introduce an annotation @IgniteSyncCallback. It's the same as
> >> @IgniteAsyncCallback but reverse :)
> >> If a listener is annotated like that then we execute it in the same
> >> thread; by default, we execute in the public pool.
> >> We can also reuse the same annotation for all other callbacks we have in
> >> the system - right now, the callbacks are a mix of sync and async
> behavior,
> >> and we could transition all APIs to use async by default and enforce
> sync
> >> callbacks when the annotation is used.
> >> @IgniteAsyncCallback should eventually be deprecated.
> >>
> >> WDYT?
> >>
> >> Thanks,
> >> Stan
> >>
> >>> On 29 Mar 2021, at 14:09, Pavel Tupitsyn wrote:
> >>>
> >>> Stan,
> >>>
> >>> I'm ok with using public pool by default, but we need a way to restore
> >> the
> >>> old behavior, do you agree?
> >>> I think we should keep the new IgniteConfiguration property.
> >>>
> >>> On Fri, Mar 26, 2021 at 2:12 PM Alexei Scherbakov <
> >>> alexey.scherbak...@gmail.com> wrote:
> >>>
> Pavel,
>
> Dedicated pool looks safer and more manageable to me. Make sure the
> >> threads
> in the pool are lazily started and stopped if not used for some time.
>
> Because I have no more real arguments against the change, I suggest to
> proceed with this approach.
>
> чт, 25 мар. 2021 г. в 22:16, Pavel Tupitsyn :
>
> > Alexei,
> >
> >> we already have ways to control a listener's behavior
[ANNOUNCE] Welcome Ivan Daschinsky as a new committer
The Project Management Committee (PMC) for Apache Ignite has invited Ivan Daschinsky to become a committer and we are pleased to announce that he has accepted. Ivan made a lot of contributions to Apache Ignite. He helped a lot to improve our Python Thin Client fixing a lot of different bugs and contributing major feature such as asyncio support and C-extension which improved performance significantly for many cases. Thanks to Ivan Python Thin client has become much more stable and production-ready. He also introduced the CMake building system for Ignite C++, and has made a number of other important improvements. Besides the code contributions, Ivan is also an active community member. Being a committer enables easier contribution to the project since there is no need to go via the patch submission process. This should enable better productivity. Please join me in welcoming Ivan, and congratulating him on the new role in the Apache Ignite Community. Best Regards, Igor
[jira] [Created] (IGNITE-14526) Hibernate L2 Cache not fully supported
Elias Balasis created IGNITE-14526:
--
Summary: Hibernate L2 Cache not fully supported
Key: IGNITE-14526
URL: https://issues.apache.org/jira/browse/IGNITE-14526
Project: Ignite
Issue Type: Bug
Components: hibernate
Affects Versions: 2.10
Reporter: Elias Balasis
At
https://ignite.apache.org/docs/2.10.0/extensions-and-integrations/hibernate-l2-cache
the suggested Maven artifact does not exist.
{code}
org.apache.ignite
ignite-hibernate_5.3
${ignite.version}
{code}
A Gridgain variant with same artifact name does exist but it is incompatible
with Apache variant.
{code}
org.gridgain
ignite-hibernate_5.3
${ignite.version}
{code}
Also there is no variant for Hibernate 5.4 or later.
This makes "Ignite" not useable as Hibernate L2 cache.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (IGNITE-14527) CVE-2021-2816[3,4,5] in Jetty
Alexander Belyak created IGNITE-14527: - Summary: CVE-2021-2816[3,4,5] in Jetty Key: IGNITE-14527 URL: https://issues.apache.org/jira/browse/IGNITE-14527 Project: Ignite Issue Type: Task Components: integrations Reporter: Alexander Belyak Assignee: Alexander Belyak Vulnerabilities found: [https://nvd.nist.gov/vuln/detail/CVE-2021-28163] [https://nvd.nist.gov/vuln/detail/CVE-2021-28164] [https://nvd.nist.gov/vuln/detail/CVE-2021-28165] -- This message was sent by Atlassian Jira (v8.3.4#803005)
Re: [ANNOUNCE] Welcome Ivan Daschinsky as a new committer
Big deal ! Ivan, ignite it !) >The Project Management Committee (PMC) for Apache Ignite has invited >Ivan Daschinsky to become a committer and we are pleased to announce that >he has accepted. > >Ivan made a lot of contributions to Apache Ignite. >He helped a lot to improve our Python Thin Client fixing a lot of different >bugs and contributing major feature such as asyncio support and C-extension >which improved performance significantly for many cases. Thanks to Ivan >Python >Thin client has become much more stable and production-ready. He also >introduced the CMake building system for Ignite C++, and has made a number >of >other important improvements. Besides the code contributions, Ivan is also >an >active community member. > >Being a committer enables easier contribution to the project since there is >no need to go via the patch submission process. This should enable better >productivity. > >Please join me in welcoming Ivan, and congratulating him on the new role in >the Apache Ignite Community. > >Best Regards, >Igor
Re: [ANNOUNCE] Welcome Ivan Daschinsky as a new committer
Congrats! Well deserved. > 13 апр. 2021 г., в 09:34, Zhenya Stanilovsky > написал(а): > > > Big deal ! Ivan, ignite it !) > > > >> The Project Management Committee (PMC) for Apache Ignite has invited >> Ivan Daschinsky to become a committer and we are pleased to announce that >> he has accepted. >> >> Ivan made a lot of contributions to Apache Ignite. >> He helped a lot to improve our Python Thin Client fixing a lot of different >> bugs and contributing major feature such as asyncio support and C-extension >> which improved performance significantly for many cases. Thanks to Ivan >> Python >> Thin client has become much more stable and production-ready. He also >> introduced the CMake building system for Ignite C++, and has made a number >> of >> other important improvements. Besides the code contributions, Ivan is also >> an >> active community member. >> >> Being a committer enables easier contribution to the project since there is >> no need to go via the patch submission process. This should enable better >> productivity. >> >> Please join me in welcoming Ivan, and congratulating him on the new role in >> the Apache Ignite Community. >> >> Best Regards, >> Igor > > > >
