Re: Ticket review checklist

2018-04-25 Thread Vladimir Ozerov
Guys, The problem with in-place refactorings is that you increase affected scope. It is not uncommon to break compatibility or public contracts with even minor things. E.g. recently we decided drop org.jsr166 package in favor of Java 8 classes. Innocent change. Result - broken storage. Another pro

Re: Apache Ignite 2.4+ Go language client

2018-04-25 Thread Alexey Kosenchuk
Hi Aleksandr, Seems an important missed part is complex objects support. And it's the most difficult part. Actually, it is not possible to implement it correctly basing on the current protocol spec only. If you decide to support it, you should either wait for an updated spec or ask for help.

Re: cache size() calculation for MVCC

2018-04-25 Thread Vladimir Ozerov
This is interesting question. Full-scan size may be tremendously slow operation on large data sets. On the other hand, printing total number of tuples including old and aborted versions make little to no sense as well. Looks like we need to choose lesser of two evils. What if we do the following: 1

Re: cache size() calculation for MVCC

2018-04-25 Thread Yakov Zhdanov
Guys, How do we update counter right now? If we move to fair thread-per-partition we can update counter only if we add new key and skip if we add or remove a version. Thoughts? --Yakov 2018-04-25 12:07 GMT+03:00 Vladimir Ozerov : > This is interesting question. Full-scan size may be tremendous

Re: cache size() calculation for MVCC

2018-04-25 Thread Vladimir Ozerov
Yakov, Thread-per-partition is hardly applicable for general SQL use case as user operates on arbitrary data sets. But in general we may track size deltas for partitions on transaction level. If transaction span one or several partitions, we may hold this data in a single long or Map. If transacti

Re: cache size() calculation for MVCC

2018-04-25 Thread Sergey Kalashnikov
Guys, The cache size counter is actually a set of per-partition counters. 2018-04-25 12:45 GMT+03:00 Vladimir Ozerov : > Yakov, > > Thread-per-partition is hardly applicable for general SQL use case as user > operates on arbitrary data sets. But in general we may track size deltas for > partition

Re: cache size() calculation for MVCC

2018-04-25 Thread Dmitriy Setrakyan
Vova, what about maintaining the size as you go? This way you don't have to count multiple versions, no? D. On Wed, Apr 25, 2018, 5:07 PM Vladimir Ozerov wrote: > This is interesting question. Full-scan size may be tremendously slow > operation on large data sets. On the other hand, printing to

[GitHub] ignite pull request #3895: Mmuzaf check ignite 8086

2018-04-25 Thread Mmuzaf
Github user Mmuzaf closed the pull request at: https://github.com/apache/ignite/pull/3895 ---

RE: Apache Ignite 2.4+ Go language client

2018-04-25 Thread Aleksandr Sokolovskii
Hi Alexey, > Seems an important missed part is complex objects support. > And it's the most difficult part. Yes, you are absolutely right. Complex object and binary types are not supported by my client now. That’s why I’m here. I plan to come back to this subject in the beginning of May. I very mu

Re: YARN parameters and default values

2018-04-25 Thread Ilya Kasnacheev
Hello! If we introduce IGNITE_OFFHEAP_MEMORY_PER_NODE, there will be an expectation that this number will be automatically considered when creating data region. I think that default value have to go definitely, but you can surely use IGNITE_MEMORY_OVERHEAD_PER_NODE *as* IGNITE_OFFHEAP_MEMORY_PER_

[jira] [Created] (IGNITE-8383) SQL: CREATE INDEX should iterate over data pages rather than PK index

2018-04-25 Thread Vladimir Ozerov (JIRA)
Vladimir Ozerov created IGNITE-8383: --- Summary: SQL: CREATE INDEX should iterate over data pages rather than PK index Key: IGNITE-8383 URL: https://issues.apache.org/jira/browse/IGNITE-8383 Project:

Re: Ticket review checklist

2018-04-25 Thread Александр Меньшиков
+1 to Vladimir Ozerov 2018-04-25 11:44 GMT+03:00 Vladimir Ozerov : > Guys, > > The problem with in-place refactorings is that you increase affected scope. > It is not uncommon to break compatibility or public contracts with even > minor things. E.g. recently we decided drop org.jsr166 package in

[jira] [Created] (IGNITE-8384) SQL: Secondary indexes should sort entries by links rather than keys

2018-04-25 Thread Vladimir Ozerov (JIRA)
Vladimir Ozerov created IGNITE-8384: --- Summary: SQL: Secondary indexes should sort entries by links rather than keys Key: IGNITE-8384 URL: https://issues.apache.org/jira/browse/IGNITE-8384 Project: I

Re: Apache Ignite 2.4+ Go language client

2018-04-25 Thread Alexey Kosenchuk
No problem with OP_QUERY_SQL_FIELDS. I mean OP_QUERY_SQL. But I overlooked "OP_QUERY_SQL - Done without unit test. Need help to develop unit test." Sorry. I remove my question so far))) Thanks, -Alexey 25.04.2018 15:40, Aleksandr Sokolovskii пишет: Hi Alexey, Seems an important missed part

RE: Apache Ignite 2.4+ Go language client

2018-04-25 Thread Aleksandr Sokolovskii
Yes, it’s really challenge to write unit test for OP_QUERY_SQL. ))) Thanks, Aleksandr From: Alexey Kosenchuk Sent: 25 апреля 2018 г. 16:33 To: dev@ignite.apache.org Subject: Re: Apache Ignite 2.4+ Go language client No problem with OP_QUERY_SQL_FIELDS. I mean OP_QUERY_SQL. But I overlooked "OP_Q

Re: Ticket review checklist

2018-04-25 Thread Nikolay Izhikov
Igniters, I agree with Vova. Don't fix if it works! If you 100% sure then it a useful addition to the product - just make a separate ticket. В Ср, 25/04/2018 в 11:44 +0300, Vladimir Ozerov пишет: > Guys, > > The problem with in-place refactorings is that you increase affected scope. > It is n

[jira] [Created] (IGNITE-8385) SQL: allow variable-length values in index leafs

2018-04-25 Thread Vladimir Ozerov (JIRA)
Vladimir Ozerov created IGNITE-8385: --- Summary: SQL: allow variable-length values in index leafs Key: IGNITE-8385 URL: https://issues.apache.org/jira/browse/IGNITE-8385 Project: Ignite Issue

[jira] [Created] (IGNITE-8386) SQL: make sure PK index do not use wrapped objects

2018-04-25 Thread Vladimir Ozerov (JIRA)
Vladimir Ozerov created IGNITE-8386: --- Summary: SQL: make sure PK index do not use wrapped objects Key: IGNITE-8386 URL: https://issues.apache.org/jira/browse/IGNITE-8386 Project: Ignite Iss

[GitHub] ignite pull request #3913: IGNITE-8357 Recreated atomic sequence produces "S...

2018-04-25 Thread pvinokurov
GitHub user pvinokurov opened a pull request: https://github.com/apache/ignite/pull/3913 IGNITE-8357 Recreated atomic sequence produces "Sequence was removed … …from cache" You can merge this pull request into a Git repository by running: $ git pull https://github.com/gridg

[jira] [Created] (IGNITE-8387) AssertionError on get() operation from replicated cache

2018-04-25 Thread Sergey Kosarev (JIRA)
Sergey Kosarev created IGNITE-8387: -- Summary: AssertionError on get() operation from replicated cache Key: IGNITE-8387 URL: https://issues.apache.org/jira/browse/IGNITE-8387 Project: Ignite

IEP-19: Optimize SQL indexes

2018-04-25 Thread Vladimir Ozerov
Igniters, I heard a lot of complains around performance of our SQL indexes. Notably - slow updates and slow execution of CREATE INDEX command on large data sets. I summarized all known possible optimizations under a single IEP [1]. We need to start working in this direction, starting from the most

Re: Apache Ignite 2.5 release

2018-04-25 Thread Andrey Gura
Igniters, We have about 30 JIRA issues that still aren't resolved. So it's possible that release date will be changed. But I think that it's time for scope freeze. Please, don't target any issues to 2.5 version without community approve. Thanks. On Fri, Apr 13, 2018 at 11:30 AM, Anton Vinogrado

[GitHub] ignite pull request #3911: IGNITE-8358 Destroy partition inside evictor to p...

2018-04-25 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/ignite/pull/3911 ---

[GitHub] ignite pull request #3880: IGNITE-8324 Update discovery cache as well as top...

2018-04-25 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/ignite/pull/3880 ---

[GitHub] ignite pull request #3914: Ignite-6430 FOR TEAM CITY ONLY

2018-04-25 Thread SharplEr
GitHub user SharplEr opened a pull request: https://github.com/apache/ignite/pull/3914 Ignite-6430 FOR TEAM CITY ONLY FOR TEAM CITY ONLY You can merge this pull request into a Git repository by running: $ git pull https://github.com/SharplEr/ignite ignite-6430 Alternatively yo

[GitHub] ignite pull request #3915: Ignite 8277 Add utilities to check and display ca...

2018-04-25 Thread glukos
GitHub user glukos opened a pull request: https://github.com/apache/ignite/pull/3915 Ignite 8277 Add utilities to check and display cache info You can merge this pull request into a Git repository by running: $ git pull https://github.com/gridgain/apache-ignite ignite-8277 Al

Re: I want to contribute to Apache Ignite

2018-04-25 Thread Alexey Goncharuk
Hello Alexandr, Welcome to the Apache Ignite community! I've added you to the contributors list, you should now be able to assign the ticket to yourself. --AG 2018-04-24 17:35 GMT+03:00 : > Hello Ignite Community! > > My name is Alexandr. I want to contribute to Apache Ignite and want to > star

[jira] [Created] (IGNITE-8388) Improve SQL functionality test coverage

2018-04-25 Thread Eduard Shangareev (JIRA)
Eduard Shangareev created IGNITE-8388: - Summary: Improve SQL functionality test coverage Key: IGNITE-8388 URL: https://issues.apache.org/jira/browse/IGNITE-8388 Project: Ignite Issue Typ

[jira] [Created] (IGNITE-8389) Get rid of thread ID in MVCC candidate

2018-04-25 Thread Alexey Kuznetsov (JIRA)
Alexey Kuznetsov created IGNITE-8389: Summary: Get rid of thread ID in MVCC candidate Key: IGNITE-8389 URL: https://issues.apache.org/jira/browse/IGNITE-8389 Project: Ignite Issue Type: I

[GitHub] ignite pull request #3916: Ignite 7628 1

2018-04-25 Thread EdShangGG
GitHub user EdShangGG opened a pull request: https://github.com/apache/ignite/pull/3916 Ignite 7628 1 You can merge this pull request into a Git repository by running: $ git pull https://github.com/gridgain/apache-ignite ignite-7628-1 Alternatively you can review and apply th

[jira] [Created] (IGNITE-8390) WAL historical rebalance is not able to process cache.remove() updates

2018-04-25 Thread Pavel Kovalenko (JIRA)
Pavel Kovalenko created IGNITE-8390: --- Summary: WAL historical rebalance is not able to process cache.remove() updates Key: IGNITE-8390 URL: https://issues.apache.org/jira/browse/IGNITE-8390 Project:

Re: Apache Ignite 2.5 release

2018-04-25 Thread Pavel Kovalenko
Igniters, I've found blocker issue https://issues.apache.org/jira/browse/IGNITE-8390 Cause of the problem is incorrect assertion that could be fixed very quickly. I would like to add this issue to 2.5 release. 2018-04-25 17:38 GMT+03:00 Andrey Gura : > Igniters, > > We have about 30 JIRA issues

Re: Apache Ignite 2.5 release

2018-04-25 Thread Dmitry Pavlov
Hi Pavel, +1 from me, assertion in GridCachePartitionExchangeManager is to be fixed before release. Sincerely, Dmitriy Pavlov ср, 25 апр. 2018 г. в 20:13, Pavel Kovalenko : > Igniters, > > I've found blocker issue https://issues.apache.org/jira/browse/IGNITE-8390 > Cause of the problem is incor

Re: Apache Ignite 2.5 release

2018-04-25 Thread Andrey Gura
Pavel, looks like painful problem. I've targeted it to 2.5 release. Thanks! On Wed, Apr 25, 2018 at 8:15 PM, Dmitry Pavlov wrote: > Hi Pavel, > > +1 from me, assertion in GridCachePartitionExchangeManager is to be fixed > before release. > > Sincerely, > Dmitriy Pavlov > > ср, 25 апр. 2018 г. в

Re: Ticket review checklist

2018-04-25 Thread Dmitry Pavlov
Igniters, the idea was related to small refactorings co-located with main change. Main change itself indicates that existing code did not meet the criteria of practice. Approving of standalone refactorings instead contradicts with principle don't touch if it works. So I still like idea of co-locat

[GitHub] ignite pull request #3917: IGNITE-8390 Fixed incorrect assertion during WAL ...

2018-04-25 Thread Jokser
GitHub user Jokser opened a pull request: https://github.com/apache/ignite/pull/3917 IGNITE-8390 Fixed incorrect assertion during WAL historical rebalance You can merge this pull request into a Git repository by running: $ git pull https://github.com/gridgain/apache-ignite ign

[GitHub] ignite pull request #3837: IGNITE-8066: AssertionError while trying to archi...

2018-04-25 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/ignite/pull/3837 ---

[GitHub] ignite pull request #3907: IGNITE-8372 ZookeeperClusterNode was made Externa...

2018-04-25 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/ignite/pull/3907 ---

[jira] [Created] (IGNITE-8391) Removing some WAL history segments leads to WAL rebalance hanging

2018-04-25 Thread Pavel Kovalenko (JIRA)
Pavel Kovalenko created IGNITE-8391: --- Summary: Removing some WAL history segments leads to WAL rebalance hanging Key: IGNITE-8391 URL: https://issues.apache.org/jira/browse/IGNITE-8391 Project: Igni

[GitHub] ignite pull request #1921: IGNITE-5151: Add some warning when offheap evicti...

2018-04-25 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/ignite/pull/1921 ---

[jira] [Created] (IGNITE-8392) Removing WAL history directory leads to JVM crush on that node.

2018-04-25 Thread Pavel Kovalenko (JIRA)
Pavel Kovalenko created IGNITE-8392: --- Summary: Removing WAL history directory leads to JVM crush on that node. Key: IGNITE-8392 URL: https://issues.apache.org/jira/browse/IGNITE-8392 Project: Ignite

Re: IEP-19: Optimize SQL indexes

2018-04-25 Thread Dmitriy Setrakyan
Thanks, Vladimir! Looking at this IEP, it is not clear which tickets are more critical than others. Also, the complexity of each ticket is unknown. Is there a way to provide this information? D. On Wed, Apr 25, 2018 at 10:21 PM, Vladimir Ozerov wrote: > Igniters, > > I heard a lot of complains

IEP-4, Phase 2. Using BL(A)T for in-memory caches.

2018-04-25 Thread Eduard Shangareev
Igniters, I have described the issue with current approach in "New definition for affinity node (issues with baseline)" topic[1]. Now we have 2 different affinity topology (one for in-memory, another for persistent caches). It causes problems: - we lose (in general) co-location between different

Re: New definition for affinity node (issues with baseline)

2018-04-25 Thread Eduard Shangareev
Guys, I have started a new topic to address the issue with DAT [1]. [1] http://apache-ignite-developers.2346864.n4.nabble.com/IEP-4-Phase-2-Using-BL-A-T-for-in-memory-caches-td29942.html On Tue, Apr 24, 2018 at 11:43 PM, Dmitriy Setrakyan wrote: > On Wed, Apr 25, 2018 at 4:13 AM, Vladimir Ozer

Re: IEP-4, Phase 2. Using BL(A)T for in-memory caches.

2018-04-25 Thread Dmitriy Setrakyan
Ed, how difficult is this fix? Are you suggesting it for 2.5? On Thu, Apr 26, 2018 at 3:10 AM, Eduard Shangareev < eduard.shangar...@gmail.com> wrote: > Igniters, > I have described the issue with current approach in "New definition for > affinity node (issues with baseline)" topic[1]. > > Now we

Re: IEP-4, Phase 2. Using BL(A)T for in-memory caches.

2018-04-25 Thread Ilya Lantukh
Eduard, I'm not sure I understand what you mean by "policy". Is it an interface that will have a few default implementations and user will be able to create his own one? If so, could you please write an example of such interface (how you see it) and how and when it's methods will be invoked. On W

Re: cache size() calculation for MVCC

2018-04-25 Thread Dmitriy Setrakyan
On Wed, Apr 25, 2018 at 5:45 PM, Vladimir Ozerov wrote: > Yakov, > > Thread-per-partition is hardly applicable for general SQL use case as user > operates on arbitrary data sets. But in general we may track size deltas > for partitions on transaction level. If transaction span one or several > pa

Re: cache size() calculation for MVCC

2018-04-25 Thread Vladimir Ozerov
Yes. On Wed, Apr 25, 2018 at 10:28 PM, Dmitriy Setrakyan wrote: > On Wed, Apr 25, 2018 at 5:45 PM, Vladimir Ozerov > wrote: > > > Yakov, > > > > Thread-per-partition is hardly applicable for general SQL use case as > user > > operates on arbitrary data sets. But in general we may track size del

[GitHub] ignite pull request #3918: IGNITE-7592: enforce future return boolean

2018-04-25 Thread Mmuzaf
GitHub user Mmuzaf opened a pull request: https://github.com/apache/ignite/pull/3918 IGNITE-7592: enforce future return boolean You can merge this pull request into a Git repository by running: $ git pull https://github.com/Mmuzaf/ignite ignite-7592 Alternatively you can revi

[GitHub] ignite pull request #3919: Ignite 2.4.2 p10

2018-04-25 Thread zstan
GitHub user zstan opened a pull request: https://github.com/apache/ignite/pull/3919 Ignite 2.4.2 p10 You can merge this pull request into a Git repository by running: $ git pull https://github.com/gridgain/apache-ignite ignite-2.4.2-p10 Alternatively you can review and apply