Metric showing how many nodes may safely leave the cluster

2019-10-04 Thread Ivan Rakov
ster is in danger of data loss. Another point is that current metrics are bound to specific cache, which makes this information even harder to analyze. Thoughts? -- Best Regards, Ivan Rakov

Re: Metric showing how many nodes may safely leave the cluster

2019-10-04 Thread Ivan Rakov
configure external tools and write extra code for basic things. Alex, Thanks, that's exact metric we need. My point is that we should make it more accessible: via control.sh command and single method for the whole cluster. Best Regards, Ivan Rakov On 04.10.2019 16:34, Alex Plehanov wrot

Re: Metric showing how many nodes may safely leave the cluster

2019-10-04 Thread Ivan Rakov
n a short path. I've started this topic due to request from user list, and I've heard many similar complaints before. Best Regards, Ivan Rakov On 04.10.2019 17:18, Nikolay Izhikov wrote: Ivan. We shouldn't force users to configure external tools and write extra code for basic th

Re: Metric showing how many nodes may safely leave the cluster

2019-10-07 Thread Ivan Rakov
Denis, Alex, Sure, new metric will be integrated into new metrics framework. Let's not expose its value to control.sh right now. I'll create an issue for aggregated "getMinimumNumberOfPartitionCopies" if everyone agrees. Best Regards, Ivan Rakov On 04.10.2019 20:06, Den

Re: Metric showing how many nodes may safely leave the cluster

2019-10-10 Thread Ivan Rakov
https://issues.apache.org/jira/browse/IGNITE-12278 Best Regards, Ivan Rakov On 07.10.2019 15:08, Ivan Rakov wrote: Denis, Alex, Sure, new metric will be integrated into new metrics framework. Let's not expose its value to control.sh right now. I'll create an issue for

Re: [VOTE] Apache Ignite PMC Chair

2019-10-29 Thread Ivan Rakov
+1 for Dmitry Pavlov Best Regards, Ivan Rakov On 29.10.2019 10:50, Ilya Kasnacheev wrote: +1 for Nikolay Izhikov (binding) Regards,

Re: Apache Ignite 2.8 RELEASE [Time, Scope, Manager]

2020-01-09 Thread Ivan Rakov
Maxim M. and anyone who is interested, I suggest to include this fix to 2.8 release: https://issues.apache.org/jira/browse/IGNITE-12225 Basically, it's a result of the following discussion: http://apache-ignite-developers.2346864.n4.nabble.com/DISCUSSION-Single-point-in-API-for-changing-cluster-st

Hint for user that baseline topology should be changed in order to trigger rebalance

2020-01-09 Thread Ivan Rakov
Folks, Since 2.4, Ignite cluster requires baseline topology in persistent mode. That means if user wants to scale cluster and add more nodes, data won't be redistributed among the whole node set until manual call of IgniteCluster#setBaselineTopology. Surely this behavior is well-documented, but d

Re: Apache Ignite 2.8 RELEASE [Time, Scope, Manager]

2020-01-13 Thread Ivan Rakov
>> чт, 9 янв. 2020 г. в 19:38, Maxim Muzafarov < > > mmu...@apache.org > > > > >: > > > > >> > >> > > > > >> > >>> Folks, > > > > >> > >>> > > > > >> > >>> > > > >

Re: Wrong results on Scan queries on REPLICATED caches during rebalance

2020-01-16 Thread Ivan Rakov
e-mail is strictly forbidden. > > Please refer to https://www.db.com/disclosures for additional EU > corporate and regulatory disclosures and to > http://www.db.com/unitedkingdom/content/privacy.htm for information about > privacy. > -- Best Regards, Ivan Rakov

Forbid mixed cache groups with both atomic and transactional caches

2020-02-04 Thread Ivan Rakov
any attempts to configure such groups. I believe that in fact no one has ever tried to do it. Please let me know if you are aware of any cases where mixed groups are used or reasons to keep them. Otherwise I'll create a ticket to prohibit mixed configurations. [1]: https://issues.apache.org/jir

Re: Forbid mixed cache groups with both atomic and transactional caches

2020-02-04 Thread Ivan Rakov
> > [1] https://issues.apache.org/jira/browse/IGNITE-2313 > > On Tue, Feb 4, 2020 at 2:28 PM Ivan Rakov wrote: > > > Igniters, > > > > Apparently it's possible in Ignite to configure a cache group with both > > ATOMIC and TRANSACTIONAL caches. > >

Re: Forbid mixed cache groups with both atomic and transactional caches

2020-02-04 Thread Ivan Rakov
s for atomic and transactional caches. > > - > Denis > > > On Tue, Feb 4, 2020 at 3:28 AM Ivan Rakov wrote: > > > Igniters, > > > > Apparently it's possible in Ignite to configure a cache group with both > > ATOMIC and TRANSACTIONAL caches. > >

Re: Forbid mixed cache groups with both atomic and transactional caches

2020-02-05 Thread Ivan Rakov
at I see a discussion for a > first time and there is already a conclusion. And the discussion was > started lesser than 24 hours ago. I suppose we should allow everyone > interested to share an opinion (here I agree with the proposal) and it > usually requires some time in open-source comm

Re: [VOTE] Allow or prohibit a joint use of @deprecated and @IgniteExperimental

2020-02-10 Thread Ivan Rakov
-1 Prohibit >From my point of view, deprecation of the existing API will confuse users in case API suggested as a replacement is marked with @IgniteExperimental. On Mon, Feb 10, 2020 at 12:20 PM Nikolay Izhikov wrote: > +1 > > > 10 февр. 2020 г., в 11:57, Andrey Mashenkov > написал(а): > > > >

Re: [DISCUSSION] Deprecation of obsolete rebalancing functionality

2020-02-13 Thread Ivan Rakov
Hello, +1 from me for rebalance delay deprecation. I can imagine only one actual case for this option: prevent excessive load on the cluster in case of temporary short-term topology changes (e.g. node is stopped for a while and then returned back). Now it's handled by baseline auto adjustment in a

Re: Security Subject of thin client on remote nodes

2020-03-23 Thread Ivan Rakov
Alex, Denis, Seems like security API is indeed a bit over-engineered. Let's get rid of SecurityContext and use SecuritySubject instead. > SecurityContext is just a POJO wrapper over > SecuritySubject's > org.apache.ignite.plugin.security.SecuritySubject#permissions. > It's functionality can be ea

Re: Discuss idle_verify with moving partitions changes.

2020-03-23 Thread Ivan Rakov
Zhenya, As for me, the current behavior of idle_verify looks correct. There's no sense in checking MOVING partitions (on which we explicitly inform user), however checking consistency between the rest of owners still makes sense: they still can diverge and we can be aware of the presence of the co

Re: Re[2]: Discuss idle_verify with moving partitions changes.

2020-03-23 Thread Ivan Rakov
ke : > "Possible results are not consistent due to rebalance still in progress" ? > Thanks ! > > >Понедельник, 23 марта 2020, 12:30 +03:00 от Ivan Rakov < > ivan.glu...@gmail.com>: > > > >Zhenya, > > > >As for me, the current behavior of idle

Re: Data vanished from cluster after INACTIVE/ACTIVE switch

2020-03-23 Thread Ivan Rakov
Folks, Let's revive this discussion until it's too late and all API changes are merged to master [1]. Seems like we don't have a final agreement on whether we should add force flag to deactivation API. First of all, I think we are all on the same page that in-memory caches data vanishing on deact

Re: Data vanished from cluster after INACTIVE/ACTIVE switch

2020-03-24 Thread Ivan Rakov
is yes we need it. > >> Right now, we can’t prevent data loss on deactivation for in-memory > caches. > >> > >> For me, the ultimate value of Ignite into real production environment is > >> user data. > >> If we have some cases when data is lost - we should

Re: Security Subject of thin client on remote nodes

2020-03-24 Thread Ivan Rakov
gt; > > > I agree with Ivan. I've implemented both variants, > > and I like one with #securityContext(UUID) more. > > > > Could you please take a look at PR [1] for the issue [2]? > > > > 1. https://github.com/apache/ignite/pull/7523 > > 2. https://is

Re: Data vanished from cluster after INACTIVE/ACTIVE switch

2020-03-24 Thread Ivan Rakov
mented intentionally. > > So, my understanding that current implementation would be here for a while. > And after we fix it I totally support removing —force flag. > > > 24 марта 2020 г., в 12:06, Ivan Rakov > написал(а): > > > >> > >> I think the on

Re: Security Subject of thin client on remote nodes

2020-03-26 Thread Ivan Rakov
E/IEP-41%3A+Security+Context+of+thin+client+on+remote+nodes > > > вт, 24 мар. 2020 г. в 12:26, Ivan Rakov : > > > Alexey, > > > > That can be another version of our plan. If everyone agrees that > > SecurityContext and SecuritySubject should be merged, such fix of

Re: Data vanished from cluster after INACTIVE/ACTIVE switch

2020-03-27 Thread Ivan Rakov
Vladimir, Igniters, Let's emphasize our final plan. We are going to add --force flags that will be necessary to pass for a deactivation if there are in-memory caches to: 1) Rest API (already implemented in [1]) 2) Command line utility (already implemented in [1]) 3) JMX bean (going to be implemen

Re: Security Subject of thin client on remote nodes

2020-03-27 Thread Ivan Rakov
lties in untangling security logic. Let's help them a bit. See more details in my JIRA comment: https://issues.apache.org/jira/browse/IGNITE-12759 On Thu, Mar 26, 2020 at 5:54 PM Ivan Rakov wrote: > Denis, > > I'll review your PR. If this issue is a subject to be included in 2

Re: Data vanished from cluster after INACTIVE/ACTIVE switch

2020-03-30 Thread Ivan Rakov
> > // > > /* / > > // > > /* NOTE:/ > > // > > /* Deactivation clears in-memory caches (without persistence) including > the system caches./ > > Should be enough. Is not? > > > 27.03.2020 10:51, Ivan Rakov пишет: > > Vladimir, Igniters, >

Re: Data vanished from cluster after INACTIVE/ACTIVE switch

2020-04-01 Thread Ivan Rakov
descriptions? > > > 30.03.2020 20:02, Ivan Rakov пишет: > > Vladimir, > > > > @param forceDeactivation If {@code true}, cluster deactivation will be > >> forced. > > It's true that it's possible to infer semantics of forced deactivation > from &

Re: [DISCUSSION] Major changes in Ignite in 2020

2020-04-10 Thread Ivan Rakov
Hi everyone! Major changes that are going to be contributed from our side: - https://issues.apache.org/jira/browse/IGNITE-11704 - keeping tombstones for removed entries to make rebalance consistent (this problem is solved by on-heap deferred deletes queue so far). - https://issues.apache.org/jira/

Re: Apache Ignite 2.8.1 RELEASE [Time, Scope, Manager]

2020-04-17 Thread Ivan Rakov
Hi, I suggest to include these fixes into 2.8.1 release: https://issues.apache.org/jira/browse/IGNITE-12101 https://issues.apache.org/jira/browse/IGNITE-12651 On Fri, Apr 17, 2020 at 11:32 AM Ivan Pavlukhin wrote: > Hi folks, > > A side note from an external spectator. Should not we reflect on

Re: Extended logging for rebalance performance analysis

2020-05-06 Thread Ivan Rakov
Hi, IGNITE_WRITE_REBALANCE_PARTITION_DISTRIBUTION_THRESHOLD - threshold > duration rebalance of cache group after which partitions distribution is > output, set in milliseconds, default value is 10 minutes. Does it mean that if the rebalancing process took less than 10 minutes, only a short vers

Re: [DISCUSS] Apache URL for TC bot

2020-05-12 Thread Ivan Rakov
nite-teamcity-bot -- Best Regards, Ivan Rakov On Tue, May 12, 2020 at 12:44 PM Ilya Kasnacheev wrote: > Hello! > > It would be nice if somebody would try to bring up a parallel deployment of > MTCGA bot on Apache domain. > > This way people will have a choice of using &qu

Re: [DISCUSS] Apache URL for TC bot

2020-05-12 Thread Ivan Rakov
idance > of duplicate (or even controversial) notifications from 2 bots at the > same time. > > Best regards, > Ivan Pavlukhin > > вт, 12 мая 2020 г. в 15:06, Ivan Rakov : > > > > Hi, > > > > I've created an INFRA ticket [1] for forwarding requests fr

Re: [ANNOUNCE] New Committer: Taras Ledkov

2020-05-12 Thread Ivan Rakov
Taras, Congratulations and welcome! On Tue, May 12, 2020 at 8:26 PM Denis Magda wrote: > Taras, > > Welcome, that was long overdue on our part! Hope to see you soon among the > PMC group. > > - > Denis > > > On Tue, May 12, 2020 at 9:09 AM Dmitriy Pavlov wrote: > > > Hello Ignite Community, >

Proposal: set default transaction timeout to 5 minutes

2020-05-18 Thread Ivan Rakov
ven though load will hang for a while) and skip step with googling and debugging. 2. Almost every system with transactions has timeout enabled by default. WDYT? -- Best Regards, Ivan Rakov

Re: Proposal: set default transaction timeout to 5 minutes

2020-05-19 Thread Ivan Rakov
g with Ignite and suddenly experience TX deadlock. -- Best Regards, Ivan Rakov On Tue, May 19, 2020 at 10:31 AM Anton Vinogradov wrote: > +1 > > On Mon, May 18, 2020 at 9:45 PM Sergey Antonov > wrote: > > > +1 > > > > пн, 18 мая 2020 г. в 21:26, Andrey Mashenkov

Re: [DISCUSS] Best way to re-encrypt existing data (TDE cache key rotation).

2020-05-22 Thread Ivan Rakov
Folks, Just keeping you informed: I and my colleagues are highly interested in TDE in general and keys rotations specifically, but we don't have enough time so far. We'll dive into this feature and participate in reviews next month. -- Best Regards, Ivan Rakov On Sun, May 17, 2020 a

Re: Proposal: set default transaction timeout to 5 minutes

2020-05-22 Thread Ivan Rakov
https://issues.apache.org/jira/browse/IGNITE-13064 is raised with label "newbie". On Tue, May 19, 2020 at 4:10 PM Ivan Rakov wrote: > Support this idea in general but why 5 minutes and not less? > > This value looks to me greater than any value that can possibly affect >

Re: Re[2]: Proposal: set default transaction timeout to 5 minutes

2020-05-26 Thread Ivan Rakov
to default, in this > case system param need to be appended. > > > > >https://issues.apache.org/jira/browse/IGNITE-13064 is raised with label > >"newbie". > > > >On Tue, May 19, 2020 at 4:10 PM Ivan Rakov < ivan.glu...@gmail.com > >

Re: Various shutdown guaranties

2020-06-08 Thread Ivan Rakov
s of stopping the node - Ignite.close() and kill . 5. *Don't* add new options to the static IgniteConfiguration to avoid conflicts between dynamic and static configuration -- Best Regards, Ivan Rakov On Mon, Jun 8, 2020 at 6:44 PM V.Pyatkov wrote: > Hi > > We need to have ability to call

Re: Various shutdown guaranties

2020-06-09 Thread Ivan Rakov
) { > if (shutdownPlc == USE_STATIC_CONFIGURATION) > throw new IllegalArgumentException("USE_STATIC_CONFIGURATION can > only be passed as dynamic property value via > ignite.cluster().setShutdownPolicy"); > ... > } > ... > } > What do you th

Re: Various shutdown guaranties

2020-06-09 Thread Ivan Rakov
. > public void setShutdownPolicy(ShutdownPolicy shutdownPlc) { > if (shutdownPlc == USE_STATIC_CONFIGURATION) > throw new IllegalArgumentException("USE_STATIC_CONFIGURATION can > only be passed as dynamic property value via > ignite.cluster().setShutdownPo

Re: Various shutdown guaranties

2020-06-09 Thread Ivan Rakov
; [1] > org.apache.ignite.IgniteCluster#setTxTimeoutOnPartitionMapExchange(timeout) > > > > вт, 9 июн. 2020 г. в 15:12, Ivan Rakov : > > > Something went wrong with gmail formatting. Resending my reply. > > > > Alex, > > > > Also shutdown

Re: Various shutdown guaranties

2020-06-09 Thread Ivan Rakov
) { > if (shutdownPlc == USE_STATIC_CONFIGURATION) > throw new IllegalArgumentException("USE_STATIC_CONFIGURATION can > only be passed as dynamic property value via > ignite.cluster().setShutdownPolicy"); > ... > } > ... > } > What do you thi

Re: Apache Ignite 2.9.0 RELEASE [Time, Scope, Manager]

2020-06-09 Thread Ivan Rakov
ity before the code freeze date. Let's include tracing to the 2.9 release scope. More info: https://cwiki.apache.org/confluence/display/IGNITE/IEP-48%3A+Tracing https://issues.apache.org/jira/browse/IGNITE-13060 -- Best Regards, Ivan Rakov On Sat, Jun 6, 2020 at 4:30 PM Denis Magda wro

Re: Various shutdown guaranties

2020-06-09 Thread Ivan Rakov
Vlad, +1, that's what I mean. We don't need either or dedicated USE_STATIC_CONFIGURATION in case the user will be able to retrieve current shutdown policy and apply the one he needs. My only requirement is that ignite.cluster().getShutdownPolicy() should return a statically configured value {@lin

Re: Extended logging for rebalance performance analysis

2020-06-29 Thread Ivan Rakov
+1 to Alex G. >From my experience, the most interesting cases with Ignite rebalancing happen exactly in production. According to the fact that we already have detailed rebalancing logging, adding info about rebalance performance looks like a reasonable improvement. With new logs we'll be able to d

Re: [DISCUSSION] Tracing: IGNITE-13060

2020-07-14 Thread Ivan Rakov
Igniters, The PR is ready to be merged, all comments from my side have been fixed. If anyone has more comments, please let know today. Best Regards, Ivan Rakov On Tue, Jun 30, 2020 at 10:43 AM Alexander Lapin wrote: > Hello Igniters, > > I'd like to discuss with you and then

Re: Apache Ignite 2.9.0 RELEASE [Time, Scope, Manager]

2020-07-14 Thread Ivan Rakov
://issues.apache.org/jira/browse/IGNITE-13060 -- Best Regards, Ivan Rakov On Tue, Jul 14, 2020 at 6:16 AM Alexey Kuznetsov wrote: > Alex, > > Can you cherry-pick to Ignite 2.9 this issue: > https://issues.apache.org/jira/browse/IGNITE-13246 ? > > This issue is about BASELINE events and it

Re: Choosing historical rebalance heuristics

2020-07-15 Thread Ivan Rakov
ates for atomic caches. As a result, detection of the absence of updates between two checkpoint markers with the same partition counter can be false positive. -- Best Regards, Ivan Rakov On Tue, Jul 14, 2020 at 3:03 PM Vladislav Pyatkov wrote: > Hi guys, > > I want to implement a more honest

Re: Choosing historical rebalance heuristics

2020-07-16 Thread Ivan Rakov
to 500) > > 2) Select only that partition for historical rebalance where difference > > between counters less that partition size. > > > > Also implement mentioned optimization for historical iterator, that may > > reduce a time on reading large WAL interval. &g

Re: Re[2]: Apache Ignite 2.9.0 RELEASE [Time, Scope, Manager]

2020-07-16 Thread Ivan Rakov
Alex, Tracing is merged to master: https://issues.apache.org/jira/browse/IGNITE-13060 Can you please port it to 2.9? For you convenience, there's PR versus 2.9 with conflicts resolved: https://github.com/apache/ignite/pull/8046/files -- Best Regards, Ivan Rakov On Wed, Jul 15, 2020 at 5:

Re: Re[2]: Apache Ignite 2.9.0 RELEASE [Time, Scope, Manager]

2020-07-31 Thread Ivan Rakov
; [3]: https://issues.apache.org/jira/browse/IGNITE-12489 > > [4]: https://issues.apache.org/jira/browse/IGNITE-12911 > > [5]: https://issues.apache.org/jira/browse/IGNITE-12553 > > [6]: > > > http://apache-ignite-developers.2346864.n4.nabble.com/DISCUSSION-Complete-D

Re: [DISCUSSION] Add index rebuild time metrics

2020-08-10 Thread Ivan Rakov
Folks, Sorry for coming late to the party. I've taken a look at this issue during review. How can a local number of processed keys can help us to understand when index rebuild will be finished? We can't compare metric value with cache.size(). First one is node-local, while cache size covers all p

Re: [DISCUSSION] Add index rebuild time metrics

2020-08-10 Thread Ivan Rakov
you think of that? > > I find one single metric much more usable. It would be perfect if metric > > value is represented in percentage, e.g. current progress of local node > > index rebuild is 60%. > > 10.08.2020, 19:11, "Ivan Rakov" : > > Folks, > > &

Re: [DISCUSSION] Add index rebuild time metrics

2020-08-11 Thread Ivan Rakov
I can add jmx methods (rebuilding indexes in the process > and the percentage of rebuilding) for the entire node, but I tried to find > a suitable place and did not find it, tell me where to add it? > > On the other hand, % of index rebuild progress is self-descriptive. I > don't >

Re: [DISCUSSION] Add index rebuild time metrics

2020-08-12 Thread Ivan Rakov
an avoid it. > I’d rather provide two separate metrics - processedKeys and localCacheSize. > > > 11 авг. 2020 г., в 16:26, Ivan Rakov написал(а): > > > >> > >> As a compromise, I can add jmx methods (rebuilding indexes in the > process > >> and the percenta

Re: Please re-commit 3 last changes in the master

2019-03-04 Thread Ivan Rakov
#6180. Andrey Kalinin* 04.03.2019 2:03 Best Regards, Ivan Rakov On 04.03.2019 13:56, Dmitriy Pavlov wrote: Thanks to Alexey Plehanov for noticing and Infra Team for fixing the issue: https://issues.apache.org/jira/browse/INFRA-17950 пн, 4 мар. 2019 г. в 13:53, Dmitriy Pavlov : Hi Devel

Re: IgniteConfigVariationsAbstractTest subclasses do not run

2019-04-17 Thread Ivan Rakov
Hi Ivan, I've checked your branch. Seems like these tests fail due to real issue in functionality. I'll take a look. Best Regards, Ivan Rakov On 17.04.2019 13:54, Ivan Fedotov wrote: Hi, Igniters! During work on iep-30[1] I discovered that IgniteConfigVariationsAbstractTest subcl

Re: IgniteConfigVariationsAbstractTest subclasses do not run

2019-04-26 Thread Ivan Rakov
hould lock entries on ATOMIC get, therefore I suppose to remove part of code where we listen and check EVT_CACHE_OBJECT_LOCKED/UNLOCKED events. Best Regards, Ivan Rakov On 17.04.2019 22:05, Ivan Rakov wrote: Hi Ivan, I've checked your branch. Seems like these tests fail due to real issue i

Re: IgniteConfigVariationsAbstractTest subclasses do not run

2019-04-26 Thread Ivan Rakov
11708). To sum it up, both issues should be fixed now. Best Regards, Ivan Rakov On 26.04.2019 14:40, Павлухин Иван wrote: Ivan R., As I can see IgniteCacheConfigVariationsFullApiTest#testGetOutTx does not expect lock/unlock events due to line: if (atomicityMode() == ATOMIC) return lockEvtCn

Re: "Idle verify" to "Online verify"

2019-04-29 Thread Ivan Rakov
ect inconsistent partitions on every PME. What do you think? [1] - https://cwiki.apache.org/confluence/display/IGNITE/Ignite+Durable+Memory+-+under+the+hood#IgniteDurableMemory-underthehood-Pagereplacement(rotationwithdisk) Best Regards, Ivan Rakov On 29.04.2019 12:20, Anton Vinogradov wrote:

Re: "Idle verify" to "Online verify"

2019-04-29 Thread Ivan Rakov
, PME happens on prod clusters from time to time (several times per week), which can be enough. In case it's needed to check consistency more often than regular PMEs occur, we can implement command that will trigger fake PME for consistency checking. Best Regards, Ivan Rakov On 29.04.2019

Re: "Idle verify" to "Online verify"

2019-05-06 Thread Ivan Rakov
hatsoever) partition contents are different. Best Regards, Ivan Rakov On 06.05.2019 12:27, Anton Vinogradov wrote: Ivan, just to make sure ... The discussed case will fully solve the issue [1] in case we'll also add some strategy to reject partitions with missed updates (updateCnt==Ok, Hash!=Ok)

Re: Lightweight version of partitions map exchange

2019-07-09 Thread Ivan Rakov
w how to handle these updates correctly as they may conflict with new updates and locks. How do you think, can we overcome this limitation with existing transaction implementation? Best Regards, Ivan Rakov On 10.07.2019 2:25, Ivan Rakov wrote: Hi Nikita, I've checked out your branch, looked t

Re: Lightweight version of partitions map exchange

2019-07-09 Thread Ivan Rakov
ot affected by transactional load Best Regards, Ivan Rakov On 01.07.2019 11:13, Nikita Amelchev wrote: Hi, Igniters. I'm working on the implementation of lightweight PME for a baseline node leave case. [1] In my implementation, each node recalculates a new affinity and completes PME loca

Re: Lightweight version of partitions map exchange

2019-07-09 Thread Ivan Rakov
instances and acquire exclusive locks) but still may receive updates from previous primary. I don't know how to handle these updates correctly as they may conflict with new updates and locks. How do you think, can we overcome this limitation with our existing implementation of transactions

Re: Tx lock partial happens before

2019-07-12 Thread Ivan Rakov
steps 1 (lock primary) and 2 (update primary + lock backup + update backup), you may be sure that there will be no false-positive results and no deadlocks as well. Protocol won't be complicated: checking read from backup will just wait for commit if it's in progress. Best Regards,

Re: Tx lock partial happens before

2019-07-15 Thread Ivan Rakov
t wait for lock release Best Regards, Ivan Rakov On 15.07.2019 9:34, Anton Vinogradov wrote: Ivan R. Thanks for joining! Got an idea, but not sure that got a way of a fix. AFAIK (can be wrong, please correct if necessary), at 2PC, locks are acquired on backups during the "prepare" ph

Re: Improvements for new security approach.

2019-07-18 Thread Ivan Rakov
Hello Max, Thanks for your analysis! Have you created a JIRA issue for discovered defects? Best Regards, Ivan Rakov On 17.07.2019 17:08, Maksim Stepachev wrote: Hello, Igniters. The main idea of the new security is propagation security context to other nodes and does action with

Re: Partition map exchange metrics

2019-07-23 Thread Ivan Rakov
ight now. Boolean metric "are we blocked right now" is not needed as it's obviously can be inferred from "current PME block time". Best Regards, Ivan Rakov On 23.07.2019 16:02, Pavel Kovalenko wrote: Nikita, I agree with total blocking duration metric but I still don

Re: Partition map exchange metrics

2019-07-24 Thread Ivan Rakov
s, but: - It's more resource intensive as it requires parsing logs for all the time - It's less reliable as log messages may change Best Regards, Ivan Rakov On 24.07.2019 14:57, Maxim Muzafarov wrote: Folks, +1 with Anton post. What if we just update current metric getCurrentPmeDura

Re: [DISCUSSION][IEP-35] Metrics configuration

2019-08-05 Thread Ivan Rakov
admin will be able to deal with current solution. Distributed Metastorage is a good candidate for storing and handling such configuration options. Best Regards, Ivan Rakov On 05.08.2019 18:38, Nikolay Izhikov wrote: Hello, Andrey. Not necessary if we have exponential bounds' values for

Re: Replacing default work dir from tmp to current dir

2019-08-12 Thread Ivan Rakov
ation#igniteWorkDir is set. It's better to let user know about missing configuration options during startup than let OS corrupt storage by cleaning temp dirs. Thoughts? Best Regards, Ivan Rakov On 12.08.2019 18:10, Anton Kalashnikov wrote: Hello, Igniters. Currently, in the case, when work direct

Re: Replacing default work dir from tmp to current dir

2019-08-12 Thread Ivan Rakov
Choosing the smallest of two evils, I'll agree with user.dir. Being able to run without preset env variables is strong benefit for Ignite as a product. Best Regards, Ivan Rakov On 12.08.2019 19:02, Denis Magda wrote: +1 for the user.dir as a default one. Denis On Monday, August 12,

Re: [VOTE] Release Apache Ignite 2.7.6-rc1

2019-08-23 Thread Ivan Rakov
+1 Downloaded binaries, successfully assembled cluster. Best Regards, Ivan Rakov On 23.08.2019 19:07, Dmitriy Pavlov wrote: +1 Checked: build from sources, startup node on Windows, simple topology, version and copyright year output, 2.7.6-rc0 is used in the Apache Ignite Teamcity Bot since

Re: Apache Ignite 2.7.6 (Time, Scope, and Release manager)

2019-09-11 Thread Ivan Rakov
Alexey, I've merged https://issues.apache.org/jira/browse/IGNITE-12163 to master and 2.7.6. Best Regards, Ivan Rakov On 11.09.2019 18:13, Alexey Goncharuk wrote: Good, Please let me know when this is done, I will re-upload the release artifacts. ср, 11 сент. 2019 г. в 18:11, Ale

IGNITE-4534

2017-02-13 Thread Ivan Rakov
Hello, community! My name is Ivan. I'm new to the project and want to contribute. I would like to start with https://issues.apache.org/jira/browse/IGNITE-4534, will be ready to provide design sketch soon. -- Best Regards, Ivan Rakov

IGNITE-4534 - Ignite 2.0 eviction design

2017-02-14 Thread Ivan Rakov
mechanism. Do we want to support it for off-heap storage as well? Please let me know if my understanding of any algorithm or part of the system is wrong. -- Best Regards, Ivan Rakov

Re: IgfsPerBlockLruEvictionPolicy does not work as expected any more, what shall we do with it?

2017-05-24 Thread Ivan Rakov
gards, Ivan Rakov On 24.05.2017 19:57, Ivan V. wrote: Hi, colleagues, as Ignite caches moved to paged offheap memory , the IgfsPerBlockLruEvictionPolicy does not seem to work as expected any more, because 1) interface org.apache.ignite.cache.eviction.EvictionPolicy now only defines "eviction fr

Re: IgniteCache#localEvict method

2017-06-19 Thread Ivan Rakov
Semantics in 2.0: if onheap cache is enabled, method evicts entry from it. If onheap cache is disabled (default case), implementation is no-op. Probably we should keep the method and add some note in javadoc. Best Regards, Ivan Rakov On 19.06.2017 17:01, Igor Sapego wrote: What if user

Re: IgniteCache#localEvict method

2017-06-19 Thread Ivan Rakov
Semantics in 2.0: if onheap cache enabled, method evicts entry from it. If onheap cache is disabled (default case), implementation is no-op. Probably we should keep the method and add some note in javadoc. Best Regards, Ivan Rakov On 19.06.2017 17:01, Igor Sapego wrote: What if user enables

Re: IgniteCache#localEvict method

2017-06-25 Thread Ivan Rakov
Agree as well. Best Regards, Ivan Rakov On 22.06.2017 1:23, Valentin Kulichenko wrote: I agree. Ivan, do you have objections? -Val On Mon, Jun 19, 2017 at 3:55 PM, Dmitriy Setrakyan mailto:dsetrak...@apache.org>> wrote: Ivan, The semantic now is very confusing, because loca

Re: Mark dirty for DataPage: small changes in huge objects.

2018-10-08 Thread Ivan Rakov
en than lack of CPU - benchmarks will show whether such impact will be perceptible. Let's file a ticket for this task unless there are any objections. Best Regards, Ivan Rakov On 08.10.2018 16:18, Dmitriy Pavlov wrote: Hi Igniters, I'd like to share a case which was implemented in t

Re: Mark dirty for DataPage: small changes in huge objects.

2018-10-08 Thread Ivan Rakov
ficant and option won't be needed at all. Otherwise, we may come up with a heuristic that will minimize negative effect, e.g. apply bytewise comparison only for data pages with only one payload item. Best Regards, Ivan Rakov On 08.10.2018 18:25, Vladimir Ozerov wrote: Can we use this mo

Re: Applicability of term 'cache' to Apache Ignite

2018-10-16 Thread Ivan Rakov
Agree with Vladimir here. Let's stick to the "principle of least astonishment" - all current users will be surprised if we'll rename IgniteCache, new users won't be greatly surprised due to compliance with JCache. Best Regards, Ivan Rakov On 16.10.2018 15:53, Vladim

Re: [ANNOUNCE] Welcome Pavel Kovalenko as a new committer

2018-11-21 Thread Ivan Rakov
Pavel, please accept my congratulations. PME wouldn't be this fast without your efforts, keep the pace! Best Regards, Ivan Rakov On 21.11.2018 11:46, Vyacheslav Daradur wrote: Congrats Pavel, I'm looking forward to future PME optimizations! On Wed, Nov 21, 2018 at 11:44 AM Alexey

Re: [VOTE] Creation dedicated list for github notifiacations

2018-11-26 Thread Ivan Rakov
+1. I've already solved this issue for myself by creating an e-mail filter, but I see no reason why every new contributor should do the same. Best Regards, Ivan Rakov On 26.11.2018 21:56, Dmitriy Pavlov wrote: +0.42 because 42 is an Answer to Life, the Universe, and Everything пн, 26

New method for MemoryMetrics: segment fill variance

2017-07-18 Thread Ivan Rakov
difference between maximum and minimum percentage of segment fill. Greater variance signals about worse hash function. Thoughts? -- Best Regards, Ivan Rakov

Re: New method for MemoryMetrics: segment fill variance

2017-07-19 Thread Ivan Rakov
Dmitriy, I agree, hashFunctionScore is more comprehensible. I'd correct it to "pageHashFunctionScore" to make it clear that it's about function that distributes pages, not about hashCode() of user-provided keys. -- Best Regards, Ivan Rakov On 18.07.2017 19:34, Dmitriy Se

Re: Ignite.close(), G.stop(name, true). Change flag cancel to false

2017-08-04 Thread Ivan Rakov
hread hanging on Ignite.close() may confuse user much more than "crashed in the middle of checkpoint" message. Best Regards, Ivan Rakov On 03.08.2017 22:34, Dmitry Pavlov wrote: Hi Igniters, I’ve created the simplest example using Ignite 2.1 and persistence (see the code below)

Re: Ignite.close(), G.stop(name, true). Change flag cancel to false

2017-08-04 Thread Ivan Rakov
r - less WAL records to replay. 2) Partition files will be locally consistent after node stop. User will be able to save partition file for any kind of analysis. Are they strong enough to force user to wait on stop? Best Regards, Ivan Rakov On 04.08.2017 13:42, Vyacheslav Daradur wrote: Hi gu

Re: Data compression in Ignite

2017-08-09 Thread Ivan Rakov
will be automatically persisted. Efficiency of encoding will be higher than in per-page case. Thoughts? The open question for per-partition dictionary is how to "garbage-collect" it. In per-page case we can do it under page write lock, but here we have to do it in more tricky way. Best

Default page size must be changed to 4k. Should it be backwards compatible?

2017-08-15 Thread Ivan Rakov
most popular enviroment among users), and recommend user to migrate to 4K-page LFS. If user still wants to work with 2k pages, he can always set it explicitly in MemoryConfiguration and start node. Thoughts? -- Best Regards, Ivan Rakov

Re: [DISCUSSION] Urgent Ignite bug fix release

2017-08-25 Thread Ivan Rakov
l-to-false-td20473.html>. Best Regards, Ivan Rakov On 25.08.2017 3:31, Dmitriy Setrakyan wrote: +1 on the release, let's do it ASAP. On Thu, Aug 24, 2017 at 12:32 PM, Denis Magda wrote: Igniters, Presently every Apache Ignite node requests 80% of RAM from an operating system. We deba

Re: [DISCUSSION] Urgent Ignite bug fix release

2017-08-28 Thread Ivan Rakov
Backport issue: https://issues.apache.org/jira/browse/IGNITE-6204 Best Regards, Ivan Rakov On 28.08.2017 17:19, Anton Vinogradov wrote: Igniters, Seems 2.2 is a urgent bugfix release, so it should be based on 2.1, In this case all other issues with fixVersion = 2.2 should be moved to 2.3

Re: Durable Memory and Ignite Persistence Tuning

2017-09-13 Thread Ivan Rakov
ze of Ignite shouldn't be less than OS page size. How to check OS cache page size in Unix: https://unix.stackexchange.com/questions/128213/how-is-page-size-determined-in-virtual-address-space Best Regards, Ivan Rakov On 01.09.2017 21:08, Denis Magda wrote: Igniters, I see a lot of compl

Re: Durable Memory and Ignite Persistence Tuning

2017-09-13 Thread Ivan Rakov
ult pageSize set to 4K - since 2.3 Best Regards, Ivan Rakov On 13.09.2017 12:29, Ivan Rakov wrote: Folks, We had some experience of benchmarking Ignite with persistent store on SSD. I think we can share some helpful advice. None of them require changing configuration of Ignite or persistent st

Re: Durable Memory and Ignite Persistence Tuning

2017-09-15 Thread Ivan Rakov
d fix this in documentation. Best Regards, Ivan Rakov On 14.09.2017 3:46, Denis Magda wrote: Ivan, Documented: https://apacheignite.readme.io/v2.1/docs/durable-memory-tuning <https://apacheignite.readme.io/v2.1/docs/durable-memory-tuning> However, I’m a bit confused with you 6. point re

Re: Persistence per memory policy configuration

2017-09-29 Thread Ivan Rakov
Guys, I've attached new configuration design draft to the ticket description: https://issues.apache.org/jira/browse/IGNITE-6030 Please, take a look. Right now is the best time to change name of any property. And question about metrics: are we going to rename MemoryMetrics and PersistenceMetr

  1   2   3   >