Re: Spark examples on TC

2019-09-19 Thread Nikolay Izhikov
Hello, Denis. Spark uses Scala 2.11 [1] Scala 2.11 compatible with the jdk version described on [2]. Few notes from the link: > We recommend using Java 8 for compiling Scala code > JDK 11 compatibility notes > As of Scala 2.13.0, 2.12.8 and 2.11.12, JDK 11 support is incomplete I also got sever

[jira] [Created] (IGNITE-12202) Support system function ROWNUM()

2019-09-19 Thread YuJue Li (Jira)
YuJue Li created IGNITE-12202: - Summary: Support system function ROWNUM() Key: IGNITE-12202 URL: https://issues.apache.org/jira/browse/IGNITE-12202 Project: Ignite Issue Type: New Feature

Re: Apache Ignite 2.7.6 (Time, Scope, and Release manager)

2019-09-19 Thread Dmitriy Pavlov
Hi Alexey, dist SVN repo usually contains only latest version, older versions go to the archive. Release 2.7.5 can be moved to the archive (with corresponding site update). And a folder for 2.7.0 is empty, it definitely can be deleted. Sincerely, Dmitriy Pavlov чт, 19 сент. 2019 г. в 21:24, Deni

Re: Spark examples on TC

2019-09-19 Thread Denis Magda
Nikolay, What needs to be done to ensure Java 9++ works with Spark Examples? Do we need to create another Spark module or drop some scala version? - Denis On Thu, Sep 19, 2019 at 9:23 AM Nikolay Izhikov wrote: > Hello, Igniters. > > Examples suite on TC doesn't work correctly under java9+. [1

Re: Apache Ignite 2.7.6 (Time, Scope, and Release manager)

2019-09-19 Thread Denis Magda
Dmitry, Everyone who is added to the readme.io gets admin rights. The owner is one of our PMC Members Dmitry Setrakyan. We can start a discussion and transfer the ownership rights to some generic account known by PMC group only. - Denis On Thu, Sep 19, 2019 at 3:40 AM Dmitriy Pavlov wrote: >

Re: Apache Ignite 2.7.6 (Time, Scope, and Release manager)

2019-09-19 Thread Alexey Goncharuk
Folks, I am following the step 6.3.12 of the release process [1] and see that the release 2.7.0 is still present in the dist folder. Do I understand correctly that I can remove both 2.7.0 and 2.7.5 at this point and 2.7.0 should have been removed during 2.7.5 release? If not, the doc should be up

Re: Proposed Release: 2.7.7 = 2.7.6 + ML + extra fixed bugs

2019-09-19 Thread Alexey Zinoviev
Please post the link to this thread when you'll create the topic to close this discussion >From my side, unfortunately, I'm not ready to be a driver of the 2.8 release, but could as one of the Santa's helpers. чт, 19 сент. 2019 г. в 20:12, Maxim Muzafarov : > Folks, > > I'll create an appropriate

Re: Spark examples on TC

2019-09-19 Thread Alexey Zinoviev
I support this solution, I think there is no issue to run with java 8 next 1-2 years all examples for TC purposes чт, 19 сент. 2019 г. в 19:23, Nikolay Izhikov : > Hello, Igniters. > > Examples suite on TC doesn't work correctly under java9+. [1], [2] > > The reason is the Spark examples. > > Spa

Re: Proposed Release: 2.7.7 = 2.7.6 + ML + extra fixed bugs

2019-09-19 Thread Maxim Muzafarov
Folks, I'll create an appropriate topic. On Thu, 19 Sep 2019 at 20:00, Denis Magda wrote: > > Ok, let's start an official discussion about 2.8 release and select the one > who will be driving/managing it. > > - > Denis > > > On Thu, Sep 19, 2019 at 9:01 AM Nikolay Izhikov wrote: > > > Scope: >

Re: Proposed Release: 2.7.7 = 2.7.6 + ML + extra fixed bugs

2019-09-19 Thread Denis Magda
Ok, let's start an official discussion about 2.8 release and select the one who will be driving/managing it. - Denis On Thu, Sep 19, 2019 at 9:01 AM Nikolay Izhikov wrote: > Scope: > > * master fixes > * ML improvements > * IEP-35: Metric + System views. > * New implementation of Service Grid

Spark examples on TC

2019-09-19 Thread Nikolay Izhikov
Hello, Igniters. Examples suite on TC doesn't work correctly under java9+. [1], [2] The reason is the Spark examples. Spark 2.3(our current version) work with scala 2.11 and scala 2.11 support of java9+ is "incomplete". Can we stick example suite to jdk8 only? [1] https://ci.ignite.apache.or

Re: Proposed Release: 2.7.7 = 2.7.6 + ML + extra fixed bugs

2019-09-19 Thread Nikolay Izhikov
Scope: * master fixes * ML improvements * IEP-35: Metric + System views. * New implementation of Service Grid В Чт, 19/09/2019 в 18:56 +0300, Maxim Muzafarov пишет: > Alexey, Denis, > > I've always thought that we are living in a paradigm that `the master > branch is the release ready branch`.

Re: Proposed Release: 2.7.7 = 2.7.6 + ML + extra fixed bugs

2019-09-19 Thread Maxim Muzafarov
Alexey, Denis, I've always thought that we are living in a paradigm that `the master branch is the release ready branch`. Don't we? If so, then we should take additional attention to commits we make, because it's true that we have some `blocker` release issues in the master branch. I also think al

Re: Proposed Release: 2.7.7 = 2.7.6 + ML + extra fixed bugs

2019-09-19 Thread Alexey Zinoviev
No objection from my side if somebody could say that master is ready or will be ready to release in a few months чт, 19 сент. 2019 г. в 14:21, Nikolay Izhikov : > +1 for 2.8 from me > > Scope: > > * master fixes > * ML improvements > * IEP-35: Metric + System views. > > > > В Чт, 19/09/2019 в 14:

[jira] [Created] (IGNITE-12201) distributed sql join not working as mentioned in documentation

2019-09-19 Thread shivakumar (Jira)
shivakumar created IGNITE-12201: --- Summary: distributed sql join not working as mentioned in documentation Key: IGNITE-12201 URL: https://issues.apache.org/jira/browse/IGNITE-12201 Project: Ignite

[jira] [Created] (IGNITE-12200) More informative assertion message at constructor of CachedDeploymentInfo (GridCacheDeploymentManager class)

2019-09-19 Thread Dmitriy Sorokin (Jira)
Dmitriy Sorokin created IGNITE-12200: Summary: More informative assertion message at constructor of CachedDeploymentInfo (GridCacheDeploymentManager class) Key: IGNITE-12200 URL: https://issues.apache.org/jira

Re: .NET java thread count keeps growing

2019-09-19 Thread Pavel Tupitsyn
Good news, everyone, I've found a straightforward fix for this, cross-platform and in pure C#. Will be included in Ignite 2.8. Details in the ticket. Thanks On Mon, Aug 19, 2019 at 11:20 PM Pavel Tupitsyn wrote: > Denis, the problem is described in > https://issues.apache.org/jira/browse/IGNITE

Re: Proposed Release: 2.7.7 = 2.7.6 + ML + extra fixed bugs

2019-09-19 Thread Nikolay Izhikov
+1 for 2.8 from me Scope: * master fixes * ML improvements * IEP-35: Metric + System views. В Чт, 19/09/2019 в 14:14 +0300, Pavel Tupitsyn пишет: > +1 for 2.8 > > We should not hold master anymore, it has been a year already. > > On Thu, Sep 19, 2019 at 1:42 PM Dmitriy Pavlov wrote: > > >

Re: Non-blocking PME Phase One (Node fail)

2019-09-19 Thread Anton Vinogradov
Ivan, The useful profile can be produced only by the reproducible benchmark. We're working on such benchmark to analyze duration and measure the improvement. Currently, only the typical overall duration from prod servers is known. On Thu, Sep 19, 2019 at 12:09 PM Павлухин Иван wrote: > Anton,

Re: Proposed Release: 2.7.7 = 2.7.6 + ML + extra fixed bugs

2019-09-19 Thread Pavel Tupitsyn
+1 for 2.8 We should not hold master anymore, it has been a year already. On Thu, Sep 19, 2019 at 1:42 PM Dmitriy Pavlov wrote: > I like both options 2.7.7 & 2.8 both have their benefits. > > But someday we will anyway need to release master state. Let's not freeze > master for years as Zookeep

[jira] [Created] (IGNITE-12199) WAL record with data entries doesn't flushes on backups for transactional cache

2019-09-19 Thread Andrey Gura (Jira)
Andrey Gura created IGNITE-12199: Summary: WAL record with data entries doesn't flushes on backups for transactional cache Key: IGNITE-12199 URL: https://issues.apache.org/jira/browse/IGNITE-12199 Pro

Re: Proposed Release: 2.7.7 = 2.7.6 + ML + extra fixed bugs

2019-09-19 Thread Dmitriy Pavlov
I like both options 2.7.7 & 2.8 both have their benefits. But someday we will anyway need to release master state. Let's not freeze master for years as Zookeeper did with 3.0. So I like option with 2.8 release more. Sincerely, Dmitriy Pavlov чт, 19 сент. 2019 г. в 01:42, Denis Magda : > Ignite

Re: Apache Ignite 2.7.6 (Time, Scope, and Release manager)

2019-09-19 Thread Dmitriy Pavlov
Artem, thank you for doing that. But the same issue came up during 2.7.5. IMO PMC should have admin access to all these projects. Who can grant these permissions? Sincerely Dmitriy Pavlov чт, 19 сент. 2019 г. в 09:44, Artem Budnikov : > Hi, > > I've created new versions for other projects on r

Re: [SparkDataFrame] Query Optimization. Prototype

2019-09-19 Thread Nikolay Izhikov
Hello, liyuj > I want to know why I suppose we are talking about Ignite SQL engine, not about Spark integration module(as it written in the message topic) The simple answer - because, it is not implemented. You can file a ticket and we wil see what we can do. You can contribute the support of

Re: How to free up space on disc after removing entries from IgniteCache with enabled PDS?

2019-09-19 Thread Anton Vinogradov
Alexey, Let's combine your and Ivan's proposals. >> vacuum command, which acquires exclusive table lock, so no concurrent activities on the table are possible. and >> Could the problem be solved by stopping a node which needs to be defragmented, clearing persistence files and restarting the node?

Re: Non-blocking PME Phase One (Node fail)

2019-09-19 Thread Павлухин Иван
Anton, folks, Out of curiosity. Do we have some measurements of PME execution in an environment similar to some real-world one? In my mind some kind of "profile" showing durations of different PME phases with an indication where we are "blocked" would be ideal. ср, 18 сент. 2019 г. в 13:27, Anton

Re: How to free up space on disc after removing entries from IgniteCache with enabled PDS?

2019-09-19 Thread Alexey Goncharuk
Anton, > >> The solution which Anton suggested does not look easy because it will > most likely significantly hurt performance > Mostly agree here, but what drop do we expect? What price do we ready to > pay? > Not sure, but seems some vendors ready to pay, for example, 5% drop for > this. 5% m

Re: Text queries/indexes (GridLuceneIndex, @QueryTextFiled)

2019-09-19 Thread Павлухин Иван
Yuriy, Greatly appreciate your interest. Could you please elaborate a little bit about sorting? What tasks does it help to solve and how? It would be great to provide an example. ср, 18 сент. 2019 г. в 09:39, Alexei Scherbakov : > > Denis, > > I like the idea of throwing an exception for enabled