Re: proposed design for thin client SQL management and monitoring (view running queries and kill it)

[Юрий]

Hey Igniters!

I continue working on IEP-29

in
part related to expose view with currently running queries.

I checked how we track running queries right now and see that it's
complicated and spread by few classes  process and we don't track all
running queries. Also there are internal queries which tracked within users
queries and can't be distinguished of user queries , e.g. map queries for
map-reduce queries or DML operation which required first step as select to
modify data.

My proposal is extract logic for working with running queries information
to separate class, like RunningQueryManager. The class will track running
queries and will be single point to retrieve information about running
queries. Currently to keep information about running queries uses
GridRunningQueryInfo class. As of now it can't provide useful information
to distinguish internal queries and users query, so the class need to
extend to keep information about type of query and id of initial user query
for internal query to be able identify it.

New RunningQueryManager should be used for all place where currently we
have tracking of running queries and added for all places which not covered
yet, it's mostly DDL and DML operations.

After implement the proposed change we can simple expose SQL view for all
running queries on a local node.
Collecting information from all node in a cluster currently out of scope
the change and will be described for discussing later.

Is there any objections for described proposal?

чт, 29 нояб. 2018 г. в 09:03, Юрий :

> Hi Alex,
>
> I've just started implement of the view. Thanks for the your efforts!
>
> ср, 28 нояб. 2018 г. в 19:00, Alex Plehanov :
>
>> Yuriy,
>>
>> If you have plans to implement running queries view in the nearest future,
>> I already have implemented draft for local node queries some time ago [1].
>> Maybe it will help to implement a view for whole cluster queries.
>>
>> [1]:
>>
>> https://github.com/alex-plekhanov/ignite/commit/6231668646a2b0f848373eb4e9dc38d127603e43
>>
>>
>> ср, 28 нояб. 2018 г. в 17:34, Vladimir Ozerov :
>>
>> > Denis
>> >
>> > I would wait for running queries view first.
>> >
>> > ср, 28 нояб. 2018 г. в 1:57, Denis Magda :
>> >
>> > > Vladimir,
>> > >
>> > > Please see inline
>> > >
>> > > On Mon, Nov 19, 2018 at 8:23 AM Vladimir Ozerov > >
>> > > wrote:
>> > >
>> > > > Denis,
>> > > >
>> > > > I partially agree with you. But there are several problem with
>> syntax
>> > > > proposed by you:
>> > > > 1) This is harder to implement technically - more parsing logic to
>> > > > implement. Ok, this is our internal problem, users do not care
>> about it
>> > > > 2) User will have to consult to docs in any case
>> > > >
>> > >
>> > > Two of these are not a big deal. We just need to invest more time for
>> > > development and during the design phase so that people need to consult
>> > the
>> > > docs rarely.
>> > >
>> > >
>> > > > 3) "nodeId" is not really node ID. For Ignite users node ID is
>> UUID. In
>> > > our
>> > > > case this is node order, and we intentionally avoided any naming
>> here.
>> > > >
>> > >
>> > > Let's use a more loose name such as "node".
>> > >
>> > >
>> > > > Query is just identified by a string, no more than that
>> > > > 4) Proposed syntax is more verbose and open ways for misuse. E.g.
>> what
>> > is
>> > > > "KILL QUERY WHERE queryId=1234"?
>> > > >
>> > > > I am not 100% satisfied with both variants, but the first one looks
>> > > simpler
>> > > > to me. Remember, that user will not guess query ID. Instead, he will
>> > get
>> > > > the list of running queries with some other syntax. What we need to
>> > > > understand for now is how this syntax will look like. I think that
>> we
>> > > > should implement getting list of running queries, and only then
>> start
>> > > > working on cancellation.
>> > > >
>> > >
>> > > That's a good point. Syntax of both running and killing queires
>> commands
>> > > should be tightly coupled. We're going to name a column if running
>> > queries
>> > > IDs somehow anyway and that name might be resued in the WHERE clause
>> of
>> > > KILL.
>> > >
>> > > Should we discuss the syntax in a separate thread?
>> > >
>> > > --
>> > > Denis
>> > >
>> > > >
>> > > > Vladimir.
>> > > >
>> > > >
>> > > > On Mon, Nov 19, 2018 at 7:02 PM Denis Mekhanikov <
>> > dmekhani...@gmail.com>
>> > > > wrote:
>> > > >
>> > > > > Guys,
>> > > > >
>> > > > > Syntax like *KILL QUERY '25.1234'* look a bit cryptic to me.
>> > > > > I'm going to look up in documentation, which parameter goes first
>> in
>> > > this
>> > > > > query every time I use it.
>> > > > > I like the syntax, that Igor suggested more.
>> > > > > Will it be better if we make *nodeId* and *queryId *named
>> properties?
>> > > > >
>> > > > > Something like this:
>> > > > > KILL QUERY WHERE nodeId=25 and queryId=1234
>> > > > >
>> > > > > Denis
>> > > > >
>> > > > > пт, 16 нояб

[jira] [Created] (IGNITE-10518) MVCC: Update operation may hangs on backup on unstable topology.

[Andrew Mashenkov (JIRA)]

Andrew Mashenkov created IGNITE-10518:
-

 Summary: MVCC: Update operation may hangs on backup on unstable 
topology. 
 Key: IGNITE-10518
 URL: https://issues.apache.org/jira/browse/IGNITE-10518
 Project: Ignite
  Issue Type: Bug
  Components: mvcc
Reporter: Andrew Mashenkov


Update operation may hangs on backup awaiting next topology.

Symptoms: 
 # Exchange for topology version 6.1 has been finished.
 # Exchange for topology version 6.2 awaits for partition release.
 # DhtTxRemote waits for exchange.

Seems, tx maps on outdated topology version.

Reproducer IgniteTxCachePrimarySyncTest.testSingleKeyCommit()  in Mvcc mode.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (IGNITE-10517) [ML] Merge inference and learning architectures

[Alexey Platonov (JIRA)]

Alexey Platonov created IGNITE-10517:


 Summary: [ML] Merge inference and learning architectures
 Key: IGNITE-10517
 URL: https://issues.apache.org/jira/browse/IGNITE-10517
 Project: Ignite
  Issue Type: Sub-task
  Components: ml
Reporter: Alexey Platonov
 Fix For: 2.8


We need to review of inference framework and try to merge it with core part on 
ML library in terms of reusing imported models in all ML steps excluding 
learning (model estimation, compositions etc.)



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

IEP-29 IGNITE-10165 Gather query stats and expose interfaces to access it

[Dmitrii Ryabov]

Hello, Igniters,

I want to do IGNITE-10165 (Gather query stats and expose interfaces to
access it) [1], if you don't mind.

Yury, Vladimir, did you planned something about statistics implementation?

Also, can you point out in which sessions should be available option to
turn on/off stats gathering? This also mentioned in the IEP-29 [2] and
IGNITE-10166 [3].

[1] https://issues.apache.org/jira/browse/IGNITE-10165
[2]
https://cwiki.apache.org/confluence/display/IGNITE/IEP-29%3A+SQL+management+and+monitoring
[3] https://issues.apache.org/jira/browse/IGNITE-10166

[jira] [Created] (IGNITE-10519) [TC Bot] Fill JavaDocs

[PetrovMikhail (JIRA)]

PetrovMikhail created IGNITE-10519:
--

 Summary: [TC Bot] Fill JavaDocs
 Key: IGNITE-10519
 URL: https://issues.apache.org/jira/browse/IGNITE-10519
 Project: Ignite
  Issue Type: Task
Reporter: PetrovMikhail






--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] ignite pull request #5565: IGNITE-9839 Web console: update to RxJS 6

[Klaster1]

GitHub user Klaster1 opened a pull request:

https://github.com/apache/ignite/pull/5565

IGNITE-9839 Web console: update to RxJS 6

https://issues.apache.org/jira/browse/IGNITE-9839
@akuznetsov-gridgain  please review.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-9839

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/5565.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #5565


commit fd422dc8b5740651eed9a642b8c08b93dfe3bf3d
Author: Ilya Borisov 
Date:   2018-11-30T02:45:01Z

IGNITE-9839 Update to rxjs pipe API and static operators.

commit 40c1fb6d4f179f419ac3c4527718db78ba19a444
Author: Ilya Borisov 
Date:   2018-12-04T03:20:10Z

IGNITE-9839 Update to rxjs 6 and @uirouter/rx, use from instead of 
fromPromise and throwError instead of _throw.

commit 8a7929aa8ef73e768663416bab376892890f3d0a
Author: Ilya Borisov 
Date:   2018-12-04T08:36:21Z

IGNITE-9839 Fix regression.




---

[GitHub] ignite pull request #5566: IGNITE-10440: Analyse test suites for possible ac...

[avplatonov]

GitHub user avplatonov opened a pull request:

https://github.com/apache/ignite/pull/5566

IGNITE-10440: Analyse test suites for possible acceleration



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-10440

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/5566.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #5566


commit 26b71c7a6e53924612a612ef1d1248e34faeddd3
Author: Alexey Platonov 
Date:   2018-11-30T06:35:51Z

initial commit

commit 636d05a1c619a5ff242bd873a34436ab75a55b88
Author: Alexey Platonov 
Date:   2018-12-03T07:58:15Z

Merge branch 'master' of https://github.com/apache/ignite into ignite-10440

commit b578bbd79481864bd32d4c366ba53cf16bb03fd8
Author: Alexey Platonov 
Date:   2018-12-03T12:31:18Z

apply scale factor to tests




---

Re: IEP-29 IGNITE-10165 Gather query stats and expose interfaces to access it

[Vladimir Ozerov]

Hi Dmitry,

AFAIK this is already work in progress by Yuriy. IEP-29 has a number of
tickets, but they should be implemented one by one, not in parallel,
because they heavily depend on each other. First we need to build
infrastructure for running queries (see topic "proposed design for thin
client SQL management and monitoring (view running queries and kill it)"),
then restore and redesign query history (it is broken now), then expose
them through views and implement KILL command.. Every step would require
certain recactoring in indexing module. I would suggest you to pick
something else to avoid clashes and merge conflicts.

Vladimir.

On Tue, Dec 4, 2018 at 12:15 PM Dmitrii Ryabov 
wrote:

> Hello, Igniters,
>
> I want to do IGNITE-10165 (Gather query stats and expose interfaces to
> access it) [1], if you don't mind.
>
> Yury, Vladimir, did you planned something about statistics implementation?
>
> Also, can you point out in which sessions should be available option to
> turn on/off stats gathering? This also mentioned in the IEP-29 [2] and
> IGNITE-10166 [3].
>
> [1] https://issues.apache.org/jira/browse/IGNITE-10165
> [2]
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-29%3A+SQL+management+and+monitoring
> [3] https://issues.apache.org/jira/browse/IGNITE-10166
>

[DISCUSSION] Performance issue with cluster-wide cache metrics distribution

[Alex Plehanov]

Hi Igniters,

In the current implementation, cache metrics are collected on each node and
sent across the whole cluster with discovery message
(TcpDiscoveryMetricsUpdateMessage) with configured frequency
(MetricsUpdateFrequency, 2 seconds by default) even if no one requested
them.
If there are a lot of caches and a lot of nodes in the cluster, metrics
update message (which contain each metric for each cache on each node) can
reach a critical size.

Also frequently collecting all cache metrics have a negative performance
impact (some of them just get values from AtomicLong, but some of them need
an iteration over all cache partitions).
The only way now to disable cache metrics collecting and sending with
discovery message is to disable statistics for each cache. But this also
makes impossible to request some of cache metrics locally (for the current
node only). Requesting a limited set of cache metrics on the current node
doesn't have such performance impact as the frequent collecting of all
cache metrics, but sometimes it's enough for diagnostic purposes.

As a workaround I have filled and implemented ticket [1], which introduces
new system property to disable cache metrics sending with
TcpDiscoveryMetricsUpdateMessage (in case this property is set, the message
will contain only node metrics). But system property is not good for a
permanent solution. Perhaps it's better to move such property to public API
(to IgniteConfiguration for example).

Also maybe we should change cache metrics distributing strategy? For
example, collect metrics by request via communication SPI or subscribe to a
limited set of cache/metrics, etc.

Thoughts?

[1]: https://issues.apache.org/jira/browse/IGNITE-10172

[GitHub] ignite pull request #5462: IGNITE-10369 PDS 4 hangs on TC

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/ignite/pull/5462


---

[jira] [Created] (IGNITE-10520) MVCC: Next to mvcc-tx non-mvcc-Tx failed with unexpected exception.

[Andrew Mashenkov (JIRA)]

Andrew Mashenkov created IGNITE-10520:
-

 Summary: MVCC: Next to mvcc-tx non-mvcc-Tx failed with unexpected 
exception.
 Key: IGNITE-10520
 URL: https://issues.apache.org/jira/browse/IGNITE-10520
 Project: Ignite
  Issue Type: Bug
Reporter: Andrew Mashenkov


IgniteCachePrimarySyncTest.testPutGet) failed.

It run tx operations one by one
 # tx on Transactional cache
 # tx on Transactional_snapshot cache.
 # tx on Transactional cache

Last tx failed with "TransactionException: Only pessimistic repeatable read 
transactions are supported when MVCC is enabled." which is unexpected.

Looks like we forget to reset tx context somewhere.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] ignite pull request #5551: IGNITE-10405: Refactor GaussianNaiveBayesTrainerE...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/ignite/pull/5551


---

[GitHub] ignite pull request #5484: IGNITE-10272: Inject learning environment into sc...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/ignite/pull/5484


---

Re: proposed design for thin client SQL management and monitoring (view running queries and kill it)

[Vladimir Ozerov]

Sounds good to me.

On Tue, Dec 4, 2018 at 12:06 PM Юрий  wrote:

> Hey Igniters!
>
> I continue working on IEP-29
> <
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-29%3A+SQL+management+and+monitoring
> >
> in
> part related to expose view with currently running queries.
>
> I checked how we track running queries right now and see that it's
> complicated and spread by few classes  process and we don't track all
> running queries. Also there are internal queries which tracked within users
> queries and can't be distinguished of user queries , e.g. map queries for
> map-reduce queries or DML operation which required first step as select to
> modify data.
>
> My proposal is extract logic for working with running queries information
> to separate class, like RunningQueryManager. The class will track running
> queries and will be single point to retrieve information about running
> queries. Currently to keep information about running queries uses
> GridRunningQueryInfo class. As of now it can't provide useful information
> to distinguish internal queries and users query, so the class need to
> extend to keep information about type of query and id of initial user query
> for internal query to be able identify it.
>
> New RunningQueryManager should be used for all place where currently we
> have tracking of running queries and added for all places which not covered
> yet, it's mostly DDL and DML operations.
>
> After implement the proposed change we can simple expose SQL view for all
> running queries on a local node.
> Collecting information from all node in a cluster currently out of scope
> the change and will be described for discussing later.
>
> Is there any objections for described proposal?
>
> чт, 29 нояб. 2018 г. в 09:03, Юрий :
>
> > Hi Alex,
> >
> > I've just started implement of the view. Thanks for the your efforts!
> >
> > ср, 28 нояб. 2018 г. в 19:00, Alex Plehanov :
> >
> >> Yuriy,
> >>
> >> If you have plans to implement running queries view in the nearest
> future,
> >> I already have implemented draft for local node queries some time ago
> [1].
> >> Maybe it will help to implement a view for whole cluster queries.
> >>
> >> [1]:
> >>
> >>
> https://github.com/alex-plekhanov/ignite/commit/6231668646a2b0f848373eb4e9dc38d127603e43
> >>
> >>
> >> ср, 28 нояб. 2018 г. в 17:34, Vladimir Ozerov :
> >>
> >> > Denis
> >> >
> >> > I would wait for running queries view first.
> >> >
> >> > ср, 28 нояб. 2018 г. в 1:57, Denis Magda :
> >> >
> >> > > Vladimir,
> >> > >
> >> > > Please see inline
> >> > >
> >> > > On Mon, Nov 19, 2018 at 8:23 AM Vladimir Ozerov <
> voze...@gridgain.com
> >> >
> >> > > wrote:
> >> > >
> >> > > > Denis,
> >> > > >
> >> > > > I partially agree with you. But there are several problem with
> >> syntax
> >> > > > proposed by you:
> >> > > > 1) This is harder to implement technically - more parsing logic to
> >> > > > implement. Ok, this is our internal problem, users do not care
> >> about it
> >> > > > 2) User will have to consult to docs in any case
> >> > > >
> >> > >
> >> > > Two of these are not a big deal. We just need to invest more time
> for
> >> > > development and during the design phase so that people need to
> consult
> >> > the
> >> > > docs rarely.
> >> > >
> >> > >
> >> > > > 3) "nodeId" is not really node ID. For Ignite users node ID is
> >> UUID. In
> >> > > our
> >> > > > case this is node order, and we intentionally avoided any naming
> >> here.
> >> > > >
> >> > >
> >> > > Let's use a more loose name such as "node".
> >> > >
> >> > >
> >> > > > Query is just identified by a string, no more than that
> >> > > > 4) Proposed syntax is more verbose and open ways for misuse. E.g.
> >> what
> >> > is
> >> > > > "KILL QUERY WHERE queryId=1234"?
> >> > > >
> >> > > > I am not 100% satisfied with both variants, but the first one
> looks
> >> > > simpler
> >> > > > to me. Remember, that user will not guess query ID. Instead, he
> will
> >> > get
> >> > > > the list of running queries with some other syntax. What we need
> to
> >> > > > understand for now is how this syntax will look like. I think that
> >> we
> >> > > > should implement getting list of running queries, and only then
> >> start
> >> > > > working on cancellation.
> >> > > >
> >> > >
> >> > > That's a good point. Syntax of both running and killing queires
> >> commands
> >> > > should be tightly coupled. We're going to name a column if running
> >> > queries
> >> > > IDs somehow anyway and that name might be resued in the WHERE clause
> >> of
> >> > > KILL.
> >> > >
> >> > > Should we discuss the syntax in a separate thread?
> >> > >
> >> > > --
> >> > > Denis
> >> > >
> >> > > >
> >> > > > Vladimir.
> >> > > >
> >> > > >
> >> > > > On Mon, Nov 19, 2018 at 7:02 PM Denis Mekhanikov <
> >> > dmekhani...@gmail.com>
> >> > > > wrote:
> >> > > >
> >> > > > > Guys,
> >> > > > >
> >> > > > > Syntax like *KILL QUERY '25.1234'* look a bit cryptic to me.
> >> > > > > I'm going to look up in documentation

Re: [DISCUSSION] Performance issue with cluster-wide cache metrics distribution

[Denis Mekhanikov]

Alex,

Did you measure the impact of metrics collection? What is the overhead you
are trying to avoid?

Just to make it clear, MetricUpdateMessage-s are used as heartbeats.
So they are sent anyways, even if no metrics are distributed between nodes.

Denis

вт, 4 дек. 2018 г. в 12:46, Alex Plehanov :

> Hi Igniters,
>
> In the current implementation, cache metrics are collected on each node and
> sent across the whole cluster with discovery message
> (TcpDiscoveryMetricsUpdateMessage) with configured frequency
> (MetricsUpdateFrequency, 2 seconds by default) even if no one requested
> them.
> If there are a lot of caches and a lot of nodes in the cluster, metrics
> update message (which contain each metric for each cache on each node) can
> reach a critical size.
>
> Also frequently collecting all cache metrics have a negative performance
> impact (some of them just get values from AtomicLong, but some of them need
> an iteration over all cache partitions).
> The only way now to disable cache metrics collecting and sending with
> discovery message is to disable statistics for each cache. But this also
> makes impossible to request some of cache metrics locally (for the current
> node only). Requesting a limited set of cache metrics on the current node
> doesn't have such performance impact as the frequent collecting of all
> cache metrics, but sometimes it's enough for diagnostic purposes.
>
> As a workaround I have filled and implemented ticket [1], which introduces
> new system property to disable cache metrics sending with
> TcpDiscoveryMetricsUpdateMessage (in case this property is set, the message
> will contain only node metrics). But system property is not good for a
> permanent solution. Perhaps it's better to move such property to public API
> (to IgniteConfiguration for example).
>
> Also maybe we should change cache metrics distributing strategy? For
> example, collect metrics by request via communication SPI or subscribe to a
> limited set of cache/metrics, etc.
>
> Thoughts?
>
> [1]: https://issues.apache.org/jira/browse/IGNITE-10172
>

Re: [DISCUSSION] Performance issue with cluster-wide cache metrics distribution

[Zhenya Stanilovsky]

hi, Alex.
imo:
1. metrics through discovery require refactoring.
2. local cache metrics should be available (if configured) on each node.
3. there must be an opportunity to configure metrics in runtime.

thanks.


>
>
>Hi Igniters,
>
>In the current implementation, cache metrics are collected on each node and
>sent across the whole cluster with discovery message
>(TcpDiscoveryMetricsUpdateMessage) with configured frequency
>(MetricsUpdateFrequency, 2 seconds by default) even if no one requested
>them.
>If there are a lot of caches and a lot of nodes in the cluster, metrics
>update message (which contain each metric for each cache on each node) can
>reach a critical size.
>
>Also frequently collecting all cache metrics have a negative performance
>impact (some of them just get values from AtomicLong, but some of them need
>an iteration over all cache partitions).
>The only way now to disable cache metrics collecting and sending with
>discovery message is to disable statistics for each cache. But this also
>makes impossible to request some of cache metrics locally (for the current
>node only). Requesting a limited set of cache metrics on the current node
>doesn't have such performance impact as the frequent collecting of all
>cache metrics, but sometimes it's enough for diagnostic purposes.
>
>As a workaround I have filled and implemented ticket [1], which introduces
>new system property to disable cache metrics sending with
>TcpDiscoveryMetricsUpdateMessage (in case this property is set, the message
>will contain only node metrics). But system property is not good for a
>permanent solution. Perhaps it's better to move such property to public API
>(to IgniteConfiguration for example).
>
>Also maybe we should change cache metrics distributing strategy? For
>example, collect metrics by request via communication SPI or subscribe to a
>limited set of cache/metrics, etc.
>
>Thoughts?
>
>[1]:  https://issues.apache.org/jira/browse/IGNITE-10172


-- 
Zhenya Stanilovsky

Re: [DISCUSSION] Performance issue with cluster-wide cache metrics distribution

[Vladimir Ozerov]

Hi Alex,

Agree with you. Most of the time these distribution of metrics is not
needed. In future we will have more and more information which potentially
needs to be shared between nodes. E.g. IO statistics, SQL statistics for
query optimizer, SQL execution history, etc. We need common mechanics for
this, so I vote for your proposal:
1) Data is collected locally
2) If a node needs to collect data from the cluster, it sends explicit
request over communication SPI
3) For performance reasons we may consider caching - return previously
collected metrics without re-requesting them again if they are not too old
(configurable)

On Tue, Dec 4, 2018 at 12:46 PM Alex Plehanov 
wrote:

> Hi Igniters,
>
> In the current implementation, cache metrics are collected on each node and
> sent across the whole cluster with discovery message
> (TcpDiscoveryMetricsUpdateMessage) with configured frequency
> (MetricsUpdateFrequency, 2 seconds by default) even if no one requested
> them.
> If there are a lot of caches and a lot of nodes in the cluster, metrics
> update message (which contain each metric for each cache on each node) can
> reach a critical size.
>
> Also frequently collecting all cache metrics have a negative performance
> impact (some of them just get values from AtomicLong, but some of them need
> an iteration over all cache partitions).
> The only way now to disable cache metrics collecting and sending with
> discovery message is to disable statistics for each cache. But this also
> makes impossible to request some of cache metrics locally (for the current
> node only). Requesting a limited set of cache metrics on the current node
> doesn't have such performance impact as the frequent collecting of all
> cache metrics, but sometimes it's enough for diagnostic purposes.
>
> As a workaround I have filled and implemented ticket [1], which introduces
> new system property to disable cache metrics sending with
> TcpDiscoveryMetricsUpdateMessage (in case this property is set, the message
> will contain only node metrics). But system property is not good for a
> permanent solution. Perhaps it's better to move such property to public API
> (to IgniteConfiguration for example).
>
> Also maybe we should change cache metrics distributing strategy? For
> example, collect metrics by request via communication SPI or subscribe to a
> limited set of cache/metrics, etc.
>
> Thoughts?
>
> [1]: https://issues.apache.org/jira/browse/IGNITE-10172
>

Re: Case sensitive indexes question.

[Vladimir Ozerov]

I think that this is not an exceptional case, as nothing is broken.
Throwing exception may make migration from other systems harder. Warning
should be enough.
Also remember that all SQL caches already have 1-2 automatic indexes out of
the box, and we haven't seen much performance complaints about that.
Additional duplicate index will not lead to severe performance degradation
or system stall.

On Fri, Nov 30, 2018 at 3:52 PM Yakov Zhdanov  wrote:

> Zhenya,
>
> Vladimir suggested not to restrict anything. However, my opinion is to
> throw exception on duplicating indexes. We should better add ability to
> rename index if it can be useful for anyone. Having same field set indexed
> with same index type is pretty strange and adds a lot of risk for
> performance of the system. If this is hard to support in 2.x then let's do
> it in 3.0. Vladimir, what do you think?
>
> -- Yakov
>

[jira] [Created] (IGNITE-10521) Creating table with datetime for PK leads to j.l.IndexOutOfBoundsException on server node

[Sergey Kozlov (JIRA)]

Sergey Kozlov created IGNITE-10521:
--

 Summary: Creating table with datetime for PK leads to 
j.l.IndexOutOfBoundsException on server node
 Key: IGNITE-10521
 URL: https://issues.apache.org/jira/browse/IGNITE-10521
 Project: Ignite
  Issue Type: Bug
Affects Versions: 2.5, 2.7
Reporter: Sergey Kozlov
 Fix For: 2.8


1. Start serve node with PDS
2. Run sqlline via thin driver
3. Execute
{noformat}
0: jdbc:ignite:thin://127.0.0.1/> CREATE TABLE t308 ( id DATE NOT NULL, col1 
INT NOT NULL, col2 VARCHAR, PRIMARY KEY (id
)) ;
Error: class org.apache.ignite.IgniteCheckedException: Failed to find value 
class in the node classpath (use default mar
shaller to enable binary objects) : 
SQL_PUBLIC_T308_bb0725c8_7db7_4309_8f4a_a63d2fffa706 (state=5,code=0)
java.sql.SQLException: class org.apache.ignite.IgniteCheckedException: Failed 
to find value class in the node classpath
(use default marshaller to enable binary objects) : 
SQL_PUBLIC_T308_bb0725c8_7db7_4309_8f4a_a63d2fffa706
at 
org.apache.ignite.internal.jdbc.thin.JdbcThinConnection.sendRequest(JdbcThinConnection.java:753)
at 
org.apache.ignite.internal.jdbc.thin.JdbcThinStatement.execute0(JdbcThinStatement.java:210)
at 
org.apache.ignite.internal.jdbc.thin.JdbcThinStatement.execute(JdbcThinStatement.java:473)
at sqlline.Commands.execute(Commands.java:823)
at sqlline.Commands.sql(Commands.java:733)
at sqlline.SqlLine.dispatch(SqlLine.java:795)
at sqlline.SqlLine.begin(SqlLine.java:668)
at sqlline.SqlLine.start(SqlLine.java:373)
at sqlline.SqlLine.main(SqlLine.java:265)
{noformat}

We do not support datetime for PK but server node stopped:
{noformat}
[14:26:32,882][INFO][exchange-worker-#43][CacheAffinitySharedManager] Updating 
caches registry performed in 3 ms.
[14:26:32,963][SEVERE][exchange-worker-#43][CacheAffinitySharedManager] Failed 
to initialize cache. Will try to rollback cache start routine. 
[cacheName=SQL_PUBLIC_T2]
class org.apache.ignite.IgniteCheckedException: Failed to find value class in 
the node classpath (use default marshaller to enable binary objects) : 
SQL_PUBLIC_T2_7080b5da_4dde_4491_b596_4a085313a98d
at 
org.apache.ignite.internal.processors.query.QueryUtils.typeForQueryEntity(QueryUtils.java:451)
at 
org.apache.ignite.internal.processors.query.GridQueryProcessor.onCacheStart0(GridQueryProcessor.java:700)
at 
org.apache.ignite.internal.processors.query.GridQueryProcessor.onCacheStart(GridQueryProcessor.java:860)
at 
org.apache.ignite.internal.processors.cache.GridCacheProcessor.startCache(GridCacheProcessor.java:1212)
at 
org.apache.ignite.internal.processors.cache.GridCacheProcessor.prepareCacheStart(GridCacheProcessor.java:1972)
at 
org.apache.ignite.internal.processors.cache.CacheAffinitySharedManager.processCacheStartRequests(CacheAffinitySharedManager.java:880)
at 
org.apache.ignite.internal.processors.cache.CacheAffinitySharedManager.onCacheChangeRequest(CacheAffinitySharedManager.java:780)
at 
org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.onCacheChangeRequest(GridDhtPartitionsExchangeFuture.java:1065)
at 
org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.init(GridDhtPartitionsExchangeFuture.java:696)
at 
org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager$ExchangeWorker.body0(GridCachePartitionExchangeManager.java:2582)
at 
org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager$ExchangeWorker.body(GridCachePartitionExchangeManager.java:2462)
at 
org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:110)
at java.lang.Thread.run(Thread.java:748)
[14:26:32,964][INFO][exchange-worker-#43][CacheAffinitySharedManager] Caches 
starting performed in 82 ms.
[14:26:32,966][INFO][exchange-worker-#43][CacheAffinitySharedManager] Affinity 
initialization for started caches performed in 2 ms.
[14:26:32,968][INFO][exchange-worker-#43][GridDhtPartitionsExchangeFuture] 
Finished waiting for partition release future [topVer=AffinityTopologyVersion 
[topVer=4, minorTopVer=4], waitTime=0ms, futInfo=NA, mode=DISTRIBUTED]
[14:26:32,973][INFO][exchange-worker-#43][GridDhtPartitionsExchangeFuture] 
Finished waiting for partitions release latch: ClientLatch 
[coordinator=TcpDiscoveryNode [id=904904a6-d889-4265-8866-dba136019718, 
addrs=ArrayList [127.0.0.1], sockAddrs=HashSet [/127.0.0.1:47500], 
discPort=47500, order=1, intOrder=1, lastExchangeTime=1543922782601, loc=false, 
ver=2.5.3#20181026-sha1:6261c02a, isClient=false], ackSent=true, 
super=CompletableLatch [id=exchange, topVer=AffinityTopologyVersion [topVer=4, 
minorTopVer=4]]]
[14:26:32,973][INFO][exchange-worker-#43][Gri

[GitHub] ignite pull request #5567: IGNITE-10462: Create Pds Mvcc test suite 2.

[AMashenkov]

GitHub user AMashenkov opened a pull request:

https://github.com/apache/ignite/pull/5567

IGNITE-10462: Create Pds Mvcc test suite 2.



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-10462

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/5567.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #5567


commit 9c52a1298cac23701f2d81925d47c66416c35bcf
Author: Andrey V. Mashenkov 
Date:   2018-12-04T12:50:30Z

IGNITE-10462: Create Pds Mvcc test suite 2.




---

Re: Change code style inspections to use red mark for those used at Teamcity build checks (IGNITE-10450)

[oignatenko]

Following up prior discussion in this thread, suggested changes were merged
to master as a part of IGNITE-10422 and are now included in project default
inspections profile.

Thanks to Maxim and Dmitriy for making it happen.

As for the ticket IGNITE-10450, I plan to keep it open for a little while
for just the case if we quickly decide to do some adjustments. Then I plan
to close it (assuming that if we will want to change something later this
could be done per new tickets).

regards, Oleg




--
Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/

[GitHub] ignite pull request #5568: IGNITE-10446 control.sh --cache idle_verify fail ...

[vldpyatkov]

GitHub user vldpyatkov opened a pull request:

https://github.com/apache/ignite/pull/5568

IGNITE-10446 control.sh --cache idle_verify fail with NPE when node l…

…eft grid

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-10446

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/5568.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #5568


commit 28d8fad9f201f695b837156b4524505905f50105
Author: vd-pyatkov 
Date:   2018-12-04T13:44:19Z

IGNITE-10446 control.sh --cache idle_verify fail with NPE when node left 
grid




---

[GitHub] ignite pull request #5569: IGNITE-10434

[gvvinblade]

GitHub user gvvinblade opened a pull request:

https://github.com/apache/ignite/pull/5569

IGNITE-10434



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-10434

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/5569.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #5569


commit 7a911e48d66bd906a0f2287fea88d465a3e44822
Author: Igor Seliverstov 
Date:   2018-12-04T14:09:04Z

pending




---

[jira] [Created] (IGNITE-10522) Remote node has not joined

[Anton Kalashnikov (JIRA)]

Anton Kalashnikov created IGNITE-10522:
--

 Summary: Remote node has not joined
 Key: IGNITE-10522
 URL: https://issues.apache.org/jira/browse/IGNITE-10522
 Project: Ignite
  Issue Type: Bug
Reporter: Anton Kalashnikov
Assignee: Anton Kalashnikov


Sometimes tests failed because of "Remote node has not joined"
suit - PDS (Indexing)
example test - IgniteWalRecoveryWithCompactionTest.testLargeRandomCrash 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (IGNITE-10523) Ignite Compatibility: add check that started node is of expected version

[Vyacheslav Daradur (JIRA)]

Vyacheslav Daradur created IGNITE-10523:
---

 Summary: Ignite Compatibility: add check that started node is of 
expected version
 Key: IGNITE-10523
 URL: https://issues.apache.org/jira/browse/IGNITE-10523
 Project: Ignite
  Issue Type: Improvement
Reporter: Vyacheslav Daradur
Assignee: Vyacheslav Daradur


It's necessary to check that node started in separate JVM is of a specified 
version to avoid unexpected behavior.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] ignite pull request #5538: IGNITE-10457

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/ignite/pull/5538


---

[jira] [Created] (IGNITE-10524) IgniteCache.iterator() from a client node leads to OOM

[Pavel Vinokurov (JIRA)]

Pavel Vinokurov created IGNITE-10524:


 Summary: IgniteCache.iterator() from a client node leads to OOM
 Key: IGNITE-10524
 URL: https://issues.apache.org/jira/browse/IGNITE-10524
 Project: Ignite
  Issue Type: Bug
  Components: cache
Affects Versions: 2.4
Reporter: Pavel Vinokurov


Looks like "iterator()" method perform a scan query and load all cache rows 
into heap. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: [DISCUSSION] Performance issue with cluster-wide cache metrics distribution

[Alexey Kuznetsov]

Hi,

One of the problems with metrics is a huge size in case when a lot caches
started on node (for example, I see 7000 caches).
We have to think how to compact them.
Not all metrics changed frequently, so, we may store locally and send over
wire only a difference from previous collect.

And think carefully about store format. For example, if current cache
metrics will be passed as JSON object,
 then 70% of it will be strings with metrics names.


On Tue, Dec 4, 2018 at 7:22 PM Vladimir Ozerov  wrote:

> Hi Alex,
>
> Agree with you. Most of the time these distribution of metrics is not
> needed. In future we will have more and more information which potentially
> needs to be shared between nodes. E.g. IO statistics, SQL statistics for
> query optimizer, SQL execution history, etc. We need common mechanics for
> this, so I vote for your proposal:
> 1) Data is collected locally
> 2) If a node needs to collect data from the cluster, it sends explicit
> request over communication SPI
> 3) For performance reasons we may consider caching - return previously
> collected metrics without re-requesting them again if they are not too old
> (configurable)
>
> On Tue, Dec 4, 2018 at 12:46 PM Alex Plehanov 
> wrote:
>
> > Hi Igniters,
> >
> > In the current implementation, cache metrics are collected on each node
> and
> > sent across the whole cluster with discovery message
> > (TcpDiscoveryMetricsUpdateMessage) with configured frequency
> > (MetricsUpdateFrequency, 2 seconds by default) even if no one requested
> > them.
> > If there are a lot of caches and a lot of nodes in the cluster, metrics
> > update message (which contain each metric for each cache on each node)
> can
> > reach a critical size.
> >
> > Also frequently collecting all cache metrics have a negative performance
> > impact (some of them just get values from AtomicLong, but some of them
> need
> > an iteration over all cache partitions).
> > The only way now to disable cache metrics collecting and sending with
> > discovery message is to disable statistics for each cache. But this also
> > makes impossible to request some of cache metrics locally (for the
> current
> > node only). Requesting a limited set of cache metrics on the current node
> > doesn't have such performance impact as the frequent collecting of all
> > cache metrics, but sometimes it's enough for diagnostic purposes.
> >
> > As a workaround I have filled and implemented ticket [1], which
> introduces
> > new system property to disable cache metrics sending with
> > TcpDiscoveryMetricsUpdateMessage (in case this property is set, the
> message
> > will contain only node metrics). But system property is not good for a
> > permanent solution. Perhaps it's better to move such property to public
> API
> > (to IgniteConfiguration for example).
> >
> > Also maybe we should change cache metrics distributing strategy? For
> > example, collect metrics by request via communication SPI or subscribe
> to a
> > limited set of cache/metrics, etc.
> >
> > Thoughts?
> >
> > [1]: https://issues.apache.org/jira/browse/IGNITE-10172
> >
>


-- 
Alexey Kuznetsov

[jira] [Created] (IGNITE-10525) Web Console: "Import models" dialog should be a singleton.

[Alexey Kuznetsov (JIRA)]

Alexey Kuznetsov created IGNITE-10525:
-

 Summary: Web Console: "Import models" dialog should be a singleton.
 Key: IGNITE-10525
 URL: https://issues.apache.org/jira/browse/IGNITE-10525
 Project: Ignite
  Issue Type: Task
  Components: wizards
Reporter: Alexey Kuznetsov
Assignee: Alexey Kuznetsov


# Click on *Import from Database* button to open dialog.
# Quick click by work area in time when background become dark prevent dialog 
opening.

When dialog opening is prevented any try to open it produce error:
{code}
TypeError: this.parent.fqn is not a function
at 
Object.push.../../../../incubator-ignite/modules/web-console/frontend/node_modules/@uirouter/core/lib/state/stateObject.js.StateObject.fqn
 (stateObject.js:61)
at 
Object.push.../../../../incubator-ignite/modules/web-console/frontend/node_modules/@uirouter/core/lib/state/stateObject.js.StateObject.toString
 (stateObject.js:102)
at 
StateRegistry.push.../../../../incubator-ignite/modules/web-console/frontend/node_modules/@uirouter/core/lib/state/stateRegistry.js.StateRegistry.deregister
 (stateRegistry.js:141)
at ModalImportModels._goToDynamicState (service.js:24)
at ModalImportModels.open (service.js:54)
at ButtonImportModels.startImport (component.js:27)
at fn (eval at compile (angular.js:15869), :4:181)
at callback (angular.js:28101)
at Scope.$eval (angular.js:18838)
at Scope.$apply (angular.js:18937) undefined
{code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] ignite pull request #5570: IGNITE-10523 Ignite Compatibility: add check that...

[daradurvs]

GitHub user daradurvs opened a pull request:

https://github.com/apache/ignite/pull/5570

IGNITE-10523 Ignite Compatibility: add check that started node is of 
expected version



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/daradurvs/ignite ignite-10523

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/5570.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #5570


commit 7232a8f1cbfb5084cc6439b91122330fd84ce083
Author: Vyacheslav Daradur 
Date:   2018-12-04T15:20:28Z

Added check of started version




---

[GitHub] ignite pull request #4974: IGNITE-8227 test failure handler

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/ignite/pull/4974


---

[GitHub] ignite pull request #5571: IGNITE-10522 added removing .tmp checkpoint files...

[akalash]

GitHub user akalash opened a pull request:

https://github.com/apache/ignite/pull/5571

IGNITE-10522 added removing .tmp checkpoint files on start



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-10522

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/5571.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #5571


commit 3cd8bc740a39a6701ca5735d0a4789c163825ccb
Author: Anton Kalashnikov 
Date:   2018-12-04T15:35:28Z

IGNITE-10522 added removing .tmp checkpoint files on start

commit 67d03cb3d29f0255bd045f5f81af64823387a773
Author: Anton Kalashnikov 
Date:   2018-12-04T15:38:48Z

IGNITE-10522 fixed typo




---

Default failure handler was changed for tests

[Dmitrii Ryabov]

Hello, Igniters!

Today the test framework's default no-op failure handler was changed to the
handler, which stops the node and fails the test.

Over 100 tests kept no-op failure handler by overrided
`getFailureHandler()` method.

If you'll found a problem or something unexpected - write here or in the
ticket [1].

[1] https://issues.apache.org/jira/browse/IGNITE-8227

[GitHub] ignite pull request #5572: IGNITE-9630

[devozerov]

GitHub user devozerov opened a pull request:

https://github.com/apache/ignite/pull/5572

IGNITE-9630



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-9630-1

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/5572.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #5572


commit 1a41a15edf6ad9f394fc09fc91052abbf7004d7f
Author: SGrimstad 
Date:   2018-09-27T14:23:28Z

IGNITE-9632 implementation

commit 03d73e26e4ccee1753f213d01f5656788c135d85
Author: SGrimstad 
Date:   2018-10-12T13:58:09Z

IGNITE-9630 implementation

commit dd43a544e2ae29595090fc29efd404f3df58d002
Author: SGrimstad 
Date:   2018-10-15T09:11:03Z

Merge branch 'master' into IGNITE-9630

commit ac1ff7ec683b3a3889fea43d155bc7a0ba8fceb1
Author: SGrimstad 
Date:   2018-10-16T07:16:30Z

IGNITE-6677 fix (braces returned)

commit e197d0217bb7ccf17c89f195b72a23c30d529c56
Author: SGrimstad 
Date:   2018-10-16T11:40:10Z

IGNITE-6677 fix TableView

commit 9afd0a4e4379f8ea95c4d5ad1446f04376fdd8cb
Author: devozerov 
Date:   2018-10-23T12:22:08Z

Merge branch 'master' into IGNITE-9630

# Conflicts:
#   
modules/indexing/src/main/java/org/apache/ignite/internal/processors/query/h2/sql/GridSqlQuerySplitter.java
#   
modules/indexing/src/test/java/org/apache/ignite/internal/processors/query/h2/twostep/InOperationExtractPartitionSelfTest.java

commit 9b6eaeeac1bb2974a9ac342712a20bf23d1035d2
Author: devozerov 
Date:   2018-10-23T12:24:48Z

WIP.

commit ddaf01b1f4b1b025298e6adbe44959ccee30df7e
Author: SGrimstad 
Date:   2018-10-24T08:39:46Z

IGNITE-9630 fixed according to review

commit 7eb76c354230c6cb90dda046d97ec3aa39873c52
Author: devozerov 
Date:   2018-11-15T20:18:00Z

Merge branch 'master' into IGNITE-9630

# Conflicts:
#   
modules/indexing/src/main/java/org/apache/ignite/internal/processors/query/h2/IgniteH2Indexing.java
#   
modules/indexing/src/main/java/org/apache/ignite/internal/processors/query/h2/affinity/PartitionInfo.java
#   
modules/indexing/src/main/java/org/apache/ignite/internal/processors/query/h2/sql/GridSqlQuerySplitter.java
#   
modules/indexing/src/test/java/org/apache/ignite/testsuites/IgniteCacheQuerySelfTestSuite2.java

commit ad7e904c112244381996b2344b99feb25b86cf49
Author: devozerov 
Date:   2018-11-15T20:34:58Z

Mege.

commit 22f4c765b81ee37b1127061a294b4807ec83db0a
Author: devozerov 
Date:   2018-11-15T20:38:08Z

WIP.

commit f8d6c1b3ccec91f37c6291f0a52c5c6434a9d5b5
Author: devozerov 
Date:   2018-11-15T20:41:06Z

WIP.

commit d0551103dd7aae28c0e984c6420ebd65beb0c553
Author: devozerov 
Date:   2018-11-29T10:24:52Z

Merge branch 'master' into ignite-9630-1

commit 2dc57345fcaf1363ad1e155a59027430694ac6dc
Author: devozerov 
Date:   2018-11-29T11:19:15Z

Minors.

commit be4510f6796d89970af0dba6a1d7a183ee3bb4b0
Author: devozerov 
Date:   2018-11-29T12:08:41Z

Nodes.

commit 4d0fca6aaa39c8e130c22426a0a9db7107181e87
Author: devozerov 
Date:   2018-11-29T12:25:08Z

WIP.

commit 1c89061efcdecd51fbc4fa73fe715fff66a0415e
Author: devozerov 
Date:   2018-11-29T14:12:32Z

Optimization algebra.

commit e7830c3c7a9707e80c634086c5e35528fbade95a
Author: devozerov 
Date:   2018-11-29T14:18:13Z

Optimization algebra.

commit b3908e3d7ac811a7830809e95c1f89fa026d03bc
Author: devozerov 
Date:   2018-11-29T15:01:42Z

Partition extractor for new logic.

commit 8ca4a504cd38083585af4337bf5b3539b91d6914
Author: devozerov 
Date:   2018-11-29T15:07:12Z

WIP.

commit ebe5fc9481a1580920364bfc92f1becf32d3da54
Author: devozerov 
Date:   2018-11-29T15:12:42Z

WIP.

commit 49698f5862c1fb6b6b035b1b9fd3544417e3e2d9
Author: devozerov 
Date:   2018-12-04T12:54:52Z

Merge branch 'master' into ignite-9630-1

commit 93d5d9b080d508aa8a1eb19bc3fa6a86cb4f392d
Author: devozerov 
Date:   2018-12-04T13:11:17Z

Simplified existing merge.

commit e0d775332997afa980d8930272ca1f6fd7bd7174
Author: devozerov 
Date:   2018-12-04T14:52:57Z

WIP on better resolver placement.

commit 795b6245d55a05430a7701200433d29ddaff6f9e
Author: devozerov 
Date:   2018-12-04T15:49:57Z

WIP.

commit 41302c9e7b07ed2ba38556a9fe1bba77d1d4e481
Author: devozerov 
Date:   2018-12-04T15:59:11Z

Wire up.

commit e27c289fb6ef99e80f16ea82b352e5f83290016d
Author: devozerov 
Date:   2018-12-04T16:03:45Z

Finished wire up.

commit d7b1a4876e31264e6fa86f1fb46d5e7da076cae2
Author: devozerov 
Date:   2018-12-04T16:04:46Z

Minors.

commit a7e89aa7c18314f53cb46c0823e2feca14abcea5
Author: devozerov 
Date:   2018-12-04T16:05:48Z

Minors.

commit 3206162bce406757262432ba26e6eaa4906754ad
Author: devozerov 
Date:   2018-12-04T16:16:10Z

Fixes.




---

[jira] [Created] (IGNITE-10526) IgniteWalReaderTest logs too much garbage on node stop.

[Andrew Mashenkov (JIRA)]

Andrew Mashenkov created IGNITE-10526:
-

 Summary: IgniteWalReaderTest logs too much garbage on node stop.
 Key: IGNITE-10526
 URL: https://issues.apache.org/jira/browse/IGNITE-10526
 Project: Ignite
  Issue Type: Improvement
  Components: persistence
Reporter: Andrew Mashenkov


WalIterator log stacktrace for each data record if no cache context found for 
entry.

Let's make it less verbose somehow, e.g. throttle or mock cach context.

 
{noformat}
[18:41:53]W: [org.apache.ignite:ignite-core] [2018-12-04 
15:41:53,626][ERROR][test-runner-#6763%reader.IgniteWalReaderTest%][root] 
Failed to perform post processing for data record
[18:41:53]W: [org.apache.ignite:ignite-core] class 
org.apache.ignite.IgniteException: Failed to find cache context for the given 
cache ID: -1368047378
[18:41:53]W: [org.apache.ignite:ignite-core] at 
org.apache.ignite.internal.pagemem.wal.record.LazyMvccDataEntry.key(LazyMvccDataEntry.java:97)
[18:41:53]W: [org.apache.ignite:ignite-core] at 
org.apache.ignite.internal.processors.cache.persistence.wal.reader.StandaloneWalRecordsIterator.postProcessDataEntry(StandaloneWalRecords
[18:41:53]W: [org.apache.ignite:ignite-core] at 
org.apache.ignite.internal.processors.cache.persistence.wal.reader.StandaloneWalRecordsIterator.postProcessDataRecord(StandaloneWalRecord
[18:41:53]W: [org.apache.ignite:ignite-core] at 
org.apache.ignite.internal.processors.cache.persistence.wal.reader.StandaloneWalRecordsIterator.postProcessRecord(StandaloneWalRecordsIte
[18:41:53]W: [org.apache.ignite:ignite-core] at 
org.apache.ignite.internal.processors.cache.persistence.wal.AbstractWalRecordsIterator.advanceRecord(AbstractWalRecordsIterator.java:248){noformat}
 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (IGNITE-10527) DenseMatrix(double[] mtx, int rows) mixes args

[Artem Malykh (JIRA)]

Artem Malykh created IGNITE-10527:
-

 Summary: DenseMatrix(double[] mtx, int rows) mixes args
 Key: IGNITE-10527
 URL: https://issues.apache.org/jira/browse/IGNITE-10527
 Project: Ignite
  Issue Type: Bug
  Components: ml
Reporter: Artem Malykh


this(mtx, StorageConstants.ROW_STORAGE_MODE, rows) -> 

this(mtx, rows, StorageConstants.ROW_STORAGE_MODE);



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Exceptions thrown in IndexingSpi and "fail fast" principle

[Ilya Kasnacheev]

Hello!

Currently, Apache Ignite is mostly written in "fail fast" fashion.

All of Apache Ignite codebase is assumed to have no bugs. When an
unexpected exception happens, it will be printed to log and will usually *leave
current operation hanging forever*. This is useful to developers since they
can spot problems right away and to users since they can avoid further data
loss, but it often leads to *data loss of current data*.

The most notorious case of such errors is hanging PMEs. When PME handling
on a single node in the cluster results in an exception, the whole cluster
will hang up forever until this node is killed. I guess you can observe
data loss after exceptions during rebalance. You can also have various
operations hanging once remote node throws an unexpected exception.

Most recently I'm trying to fix IgniteErrorOnRebalanceTest in IGNITE-9842.
It tests exception thrown from IndexingSpi, it never worked and it leads to
silent data loss. When baseline is introduced, it will now lead to hanging
PME.

Should we fight this problem? IndexingSpi implementation is external to us,
we should either:
A. Catch any exception that it would throw. If it was thrown during
rebalance, ignore it with warning to avoid data loss.
B. Assume that it never throws exceptions (or that it will only throw
IgniteSpiException). As soon as any exception is thrown, the behavior of
cluster is undefined. This is current behavior.

*Are we ready to make a leap from B to A?*

Note that currently, if an exception is thrown from IndexingSpi, an
operation will fail in mid-flight, meaning that part of data could be
updated and the rest was not. It is possible that entry was added to cache
but not indexed by SQL, for example. We will need to be able to roll back
any operation when error occurs.

With regard to PME:
- When there is an exception during exchange, we should be able to switch
back to previous topology version on all nodes.
- That means the node which was trying to join is kicked from topology (and
not the one that had this exception thrown). Or the cache is not created,
or not destroyed, etc.
- No data loss since all existing nodes happily continue to work on the old
topology version and new node did not have any data yet.

Basically, every remote operation should be guarded with a fallback where a
message is sent to caller when operation did not succeed. This will mean
that no operation ever hangs.

WDYT?

Regards,
-- 
Ilya Kasnacheev

Re: Default failure handler was changed for tests

[Anton Vinogradov]

Dmitrii,

Could you please explain the reason of explicit set of 100+
NoOpFailureHandlers?


вт, 4 дек. 2018 г. в 19:12, Dmitrii Ryabov :

> Hello, Igniters!
>
> Today the test framework's default no-op failure handler was changed to the
> handler, which stops the node and fails the test.
>
> Over 100 tests kept no-op failure handler by overrided
> `getFailureHandler()` method.
>
> If you'll found a problem or something unexpected - write here or in the
> ticket [1].
>
> [1] https://issues.apache.org/jira/browse/IGNITE-8227
>

Re: Default failure handler was changed for tests

[Dmitriy Pavlov]

Hi Igniters,

BTW, if you find in any of your tests it does't need an old value of
handler (=NoOp), feel free to remove it.

Sincerely,
Dmitriy Pavlov

вт, 4 дек. 2018 г. в 20:02, Anton Vinogradov :

> Dmitrii,
>
> Could you please explain the reason of explicit set of 100+
> NoOpFailureHandlers?
>
>
> вт, 4 дек. 2018 г. в 19:12, Dmitrii Ryabov :
>
> > Hello, Igniters!
> >
> > Today the test framework's default no-op failure handler was changed to
> the
> > handler, which stops the node and fails the test.
> >
> > Over 100 tests kept no-op failure handler by overrided
> > `getFailureHandler()` method.
> >
> > If you'll found a problem or something unexpected - write here or in the
> > ticket [1].
> >
> > [1] https://issues.apache.org/jira/browse/IGNITE-8227
> >
>

[GitHub] ignite pull request #5570: IGNITE-10523 Ignite Compatibility: add check that...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/ignite/pull/5570


---

[jira] [Created] (IGNITE-10528) [ML] Fix incorrect comparing of double values in ML examples

[Aleksey Zinoviev (JIRA)]

Aleksey Zinoviev created IGNITE-10528:
-

 Summary: [ML] Fix incorrect comparing of double values in ML 
examples
 Key: IGNITE-10528
 URL: https://issues.apache.org/jira/browse/IGNITE-10528
 Project: Ignite
  Issue Type: Bug
  Components: ml
Affects Versions: 2.8
Reporter: Aleksey Zinoviev
Assignee: Aleksey Zinoviev
 Fix For: 2.8


Look at code row

if (groundTruth != prediction)

in each example

Fix with Math.abs or Double.compare method (don't forget precision)



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: Exceptions thrown in IndexingSpi and "fail fast" principle

[Vyacheslav Daradur]

Regarding PME:

I think some kind of exceptions may be caught in
'GridDhtPartitionsExchangeFuture' to be able to notify the
coordinator.
For example, if an exception occurred during cache or index (or
something else) creation on some node, the node can notify coordinator
and coordinator should be able to make a decision on how to handle the
situation. AFAIK a message 'DynamicCacheChangeFailureMessage' will be
sent in case of some kind error.

It would be great if the most actions managed by PME will be handled
in the same manner. For doing this we can specify some sort of
exception types and rules on how to handle them, so the coordinator
will be able to finish hang PME across the cluster.
On Tue, Dec 4, 2018 at 8:01 PM Ilya Kasnacheev
 wrote:
>
> Hello!
>
> Currently, Apache Ignite is mostly written in "fail fast" fashion.
>
> All of Apache Ignite codebase is assumed to have no bugs. When an
> unexpected exception happens, it will be printed to log and will usually 
> *leave
> current operation hanging forever*. This is useful to developers since they
> can spot problems right away and to users since they can avoid further data
> loss, but it often leads to *data loss of current data*.
>
> The most notorious case of such errors is hanging PMEs. When PME handling
> on a single node in the cluster results in an exception, the whole cluster
> will hang up forever until this node is killed. I guess you can observe
> data loss after exceptions during rebalance. You can also have various
> operations hanging once remote node throws an unexpected exception.
>
> Most recently I'm trying to fix IgniteErrorOnRebalanceTest in IGNITE-9842.
> It tests exception thrown from IndexingSpi, it never worked and it leads to
> silent data loss. When baseline is introduced, it will now lead to hanging
> PME.
>
> Should we fight this problem? IndexingSpi implementation is external to us,
> we should either:
> A. Catch any exception that it would throw. If it was thrown during
> rebalance, ignore it with warning to avoid data loss.
> B. Assume that it never throws exceptions (or that it will only throw
> IgniteSpiException). As soon as any exception is thrown, the behavior of
> cluster is undefined. This is current behavior.
>
> *Are we ready to make a leap from B to A?*
>
> Note that currently, if an exception is thrown from IndexingSpi, an
> operation will fail in mid-flight, meaning that part of data could be
> updated and the rest was not. It is possible that entry was added to cache
> but not indexed by SQL, for example. We will need to be able to roll back
> any operation when error occurs.
>
> With regard to PME:
> - When there is an exception during exchange, we should be able to switch
> back to previous topology version on all nodes.
> - That means the node which was trying to join is kicked from topology (and
> not the one that had this exception thrown). Or the cache is not created,
> or not destroyed, etc.
> - No data loss since all existing nodes happily continue to work on the old
> topology version and new node did not have any data yet.
>
> Basically, every remote operation should be guarded with a fallback where a
> message is sent to caller when operation did not succeed. This will mean
> that no operation ever hangs.
>
> WDYT?
>
> Regards,
> --
> Ilya Kasnacheev



-- 
Best Regards, Vyacheslav D.

[jira] [Created] (IGNITE-10529) [ML] Add Confusion Matrix support for classification algorithms

[Aleksey Zinoviev (JIRA)]

Aleksey Zinoviev created IGNITE-10529:
-

 Summary: [ML] Add Confusion Matrix support for classification 
algorithms
 Key: IGNITE-10529
 URL: https://issues.apache.org/jira/browse/IGNITE-10529
 Project: Ignite
  Issue Type: New Feature
  Components: ml
Affects Versions: 2.8
Reporter: Aleksey Zinoviev
Assignee: Aleksey Zinoviev
 Fix For: 2.8


This is an umbrella ticket for Confusion Matrix Support



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (IGNITE-10530) [ML] Add Confusion Matrix for Binary Classification

[Aleksey Zinoviev (JIRA)]

Aleksey Zinoviev created IGNITE-10530:
-

 Summary: [ML] Add Confusion Matrix for Binary Classification
 Key: IGNITE-10530
 URL: https://issues.apache.org/jira/browse/IGNITE-10530
 Project: Ignite
  Issue Type: Sub-task
  Components: ml
Affects Versions: 2.8
Reporter: Aleksey Zinoviev
Assignee: Aleksey Zinoviev
 Fix For: 2.8


Add special class to build confusion matrix as a product of evaluation process



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (IGNITE-10531) [ML] Refactor all examples to use Binary Confusion Matrix instead of calculations by hand

[Aleksey Zinoviev (JIRA)]

Aleksey Zinoviev created IGNITE-10531:
-

 Summary: [ML] Refactor all examples to use Binary Confusion Matrix 
instead of calculations by hand
 Key: IGNITE-10531
 URL: https://issues.apache.org/jira/browse/IGNITE-10531
 Project: Ignite
  Issue Type: Sub-task
  Components: ml
Affects Versions: 2.8
Reporter: Aleksey Zinoviev
Assignee: Aleksey Zinoviev
 Fix For: 2.8


Change 

// Build confusion matrix. See https://en.wikipedia.org/wiki/Confusion_matrix
int[][] confusionMtx = \{{0, 0}, \{0, 0}};

to usage of ConfusionMatrix



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (IGNITE-10532) [ML] Add Confusion Matrix for multi-class classification

[Aleksey Zinoviev (JIRA)]

Aleksey Zinoviev created IGNITE-10532:
-

 Summary: [ML] Add Confusion Matrix for multi-class classification
 Key: IGNITE-10532
 URL: https://issues.apache.org/jira/browse/IGNITE-10532
 Project: Ignite
  Issue Type: Sub-task
  Components: ml
Affects Versions: 2.8
Reporter: Aleksey Zinoviev
Assignee: Aleksey Zinoviev
 Fix For: 2.8


Explore ability to integrate the OneVsRest with ConfusionMatrix calculation

also it can be implemented only after MultiClassEvaluator (no ticket yet)



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: Default failure handler was changed for tests

[Dmitrii Ryabov]

Anton,

Tests in these classes check fail cases when we expect critical
failure like node stop or exception thrown. Such tests trigger failure
handler and it fails test when everything goes as it should go. That's
why we need no-op handler here.
вт, 4 дек. 2018 г. в 20:06, Dmitriy Pavlov :
>
> Hi Igniters,
>
> BTW, if you find in any of your tests it does't need an old value of
> handler (=NoOp), feel free to remove it.
>
> Sincerely,
> Dmitriy Pavlov
>
> вт, 4 дек. 2018 г. в 20:02, Anton Vinogradov :
>
> > Dmitrii,
> >
> > Could you please explain the reason of explicit set of 100+
> > NoOpFailureHandlers?
> >
> >
> > вт, 4 дек. 2018 г. в 19:12, Dmitrii Ryabov :
> >
> > > Hello, Igniters!
> > >
> > > Today the test framework's default no-op failure handler was changed to
> > the
> > > handler, which stops the node and fails the test.
> > >
> > > Over 100 tests kept no-op failure handler by overrided
> > > `getFailureHandler()` method.
> > >
> > > If you'll found a problem or something unexpected - write here or in the
> > > ticket [1].
> > >
> > > [1] https://issues.apache.org/jira/browse/IGNITE-8227
> > >
> >

Re: Default failure handler was changed for tests

[Anton Vinogradov]

Dmitrii,

The solution is not clear to me.
In case you expect the failure then a correct case is to wrap it with
try-catch block instead of no-op failure handler usage.

вт, 4 дек. 2018 г. в 21:41, Dmitrii Ryabov :

> Anton,
>
> Tests in these classes check fail cases when we expect critical
> failure like node stop or exception thrown. Such tests trigger failure
> handler and it fails test when everything goes as it should go. That's
> why we need no-op handler here.
> вт, 4 дек. 2018 г. в 20:06, Dmitriy Pavlov :
> >
> > Hi Igniters,
> >
> > BTW, if you find in any of your tests it does't need an old value of
> > handler (=NoOp), feel free to remove it.
> >
> > Sincerely,
> > Dmitriy Pavlov
> >
> > вт, 4 дек. 2018 г. в 20:02, Anton Vinogradov :
> >
> > > Dmitrii,
> > >
> > > Could you please explain the reason of explicit set of 100+
> > > NoOpFailureHandlers?
> > >
> > >
> > > вт, 4 дек. 2018 г. в 19:12, Dmitrii Ryabov :
> > >
> > > > Hello, Igniters!
> > > >
> > > > Today the test framework's default no-op failure handler was changed
> to
> > > the
> > > > handler, which stops the node and fails the test.
> > > >
> > > > Over 100 tests kept no-op failure handler by overrided
> > > > `getFailureHandler()` method.
> > > >
> > > > If you'll found a problem or something unexpected - write here or in
> the
> > > > ticket [1].
> > > >
> > > > [1] https://issues.apache.org/jira/browse/IGNITE-8227
> > > >
> > >
>

Re: Default failure handler was changed for tests

[Anton Vinogradov]

And you have to check the reason of failure inside the try-catch block, of
course.
In case found not equals to expected then test should rethrow the exception.


вт, 4 дек. 2018 г. в 23:21, Anton Vinogradov :

> Dmitrii,
>
> The solution is not clear to me.
> In case you expect the failure then a correct case is to wrap it with
> try-catch block instead of no-op failure handler usage.
>
> вт, 4 дек. 2018 г. в 21:41, Dmitrii Ryabov :
>
>> Anton,
>>
>> Tests in these classes check fail cases when we expect critical
>> failure like node stop or exception thrown. Such tests trigger failure
>> handler and it fails test when everything goes as it should go. That's
>> why we need no-op handler here.
>> вт, 4 дек. 2018 г. в 20:06, Dmitriy Pavlov :
>> >
>> > Hi Igniters,
>> >
>> > BTW, if you find in any of your tests it does't need an old value of
>> > handler (=NoOp), feel free to remove it.
>> >
>> > Sincerely,
>> > Dmitriy Pavlov
>> >
>> > вт, 4 дек. 2018 г. в 20:02, Anton Vinogradov :
>> >
>> > > Dmitrii,
>> > >
>> > > Could you please explain the reason of explicit set of 100+
>> > > NoOpFailureHandlers?
>> > >
>> > >
>> > > вт, 4 дек. 2018 г. в 19:12, Dmitrii Ryabov :
>> > >
>> > > > Hello, Igniters!
>> > > >
>> > > > Today the test framework's default no-op failure handler was
>> changed to
>> > > the
>> > > > handler, which stops the node and fails the test.
>> > > >
>> > > > Over 100 tests kept no-op failure handler by overrided
>> > > > `getFailureHandler()` method.
>> > > >
>> > > > If you'll found a problem or something unexpected - write here or
>> in the
>> > > > ticket [1].
>> > > >
>> > > > [1] https://issues.apache.org/jira/browse/IGNITE-8227
>> > > >
>> > >
>>
>

Re: Default failure handler was changed for tests

[Andrey Mashenkov]

Hi all,

Really, why noop?

If you expect failure handler should be triggered, you can override default
one and rise some flag, which can be checked in test.
This will make test clearer.

With noop, you'll get previous unwanted  behavior, that you are trying to
improve, isnt'it?

4 дек. 2018 г. 23:25 пользователь "Anton Vinogradov" 
написал:

And you have to check the reason of failure inside the try-catch block, of
course.
In case found not equals to expected then test should rethrow the exception.


вт, 4 дек. 2018 г. в 23:21, Anton Vinogradov :


> Dmitrii,
>
> The solution is not clear to me.
> In case you expect the failure then a correct case is to wrap it with
> try-catch block instead of no-op failure handler usage.
>
> вт, 4 дек. 2018 г. в 21:41, Dmitrii Ryabov :
>
>> Anton,
>>
>> Tests in these classes check fail cases when we expect critical
>> failure like node stop or exception thrown. Such tests trigger failure
>> handler and it fails test when everything goes as it should go. That's
>> why we need no-op handler here.
>> вт, 4 дек. 2018 г. в 20:06, Dmitriy Pavlov :
>> >
>> > Hi Igniters,
>> >
>> > BTW, if you find in any of your tests it does't need an old value of
>> > handler (=NoOp), feel free to remove it.
>> >
>> > Sincerely,
>> > Dmitriy Pavlov
>> >
>> > вт, 4 дек. 2018 г. в 20:02, Anton Vinogradov :
>> >
>> > > Dmitrii,
>> > >
>> > > Could you please explain the reason of explicit set of 100+
>> > > NoOpFailureHandlers?
>> > >
>> > >
>> > > вт, 4 дек. 2018 г. в 19:12, Dmitrii Ryabov :
>> > >
>> > > > Hello, Igniters!
>> > > >
>> > > > Today the test framework's default no-op failure handler was
>> changed to
>> > > the
>> > > > handler, which stops the node and fails the test.
>> > > >
>> > > > Over 100 tests kept no-op failure handler by overrided
>> > > > `getFailureHandler()` method.
>> > > >
>> > > > If you'll found a problem or something unexpected - write here or
>> in the
>> > > > ticket [1].
>> > > >
>> > > > [1] https://issues.apache.org/jira/browse/IGNITE-8227
>> > > >
>> > >
>>
>

[jira] [Created] (IGNITE-10533) Web Console: Images outdated on web site.

[Alexey Kuznetsov (JIRA)]

Alexey Kuznetsov created IGNITE-10533:
-

 Summary: Web Console: Images outdated on web site.
 Key: IGNITE-10533
 URL: https://issues.apache.org/jira/browse/IGNITE-10533
 Project: Ignite
  Issue Type: Task
  Components: documentation
Reporter: Alexey Kuznetsov
Assignee: Prachi Garg


For example see: https://apacheignite-tools.readme.io/docs/ignite-web-console



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] ignite pull request #5573: Ignite 2.4.8 p8

[gromtech]

GitHub user gromtech opened a pull request:

https://github.com/apache/ignite/pull/5573

Ignite 2.4.8 p8



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-2.4.8-p8

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/5573.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #5573


commit 824004909b23a9a7d599500967af34103acb8c47
Author: Igor Sapego 
Date:   2018-01-30T12:56:17Z

IGNITE-6810: Implemented SSL support for ODBC.

This closes #3361

commit 82da0b5e9dc2ee2339c3fb1023e35d415bf1b647
Author: Pavel Kuznetsov 
Date:   2018-02-07T12:37:52Z

IGNITE-6217: Added benchmark to compare JDBC vs native SQL

This closes #2558

commit 701c6f141f6812ad7bc050a86266e696cf5863ed
Author: tledkov-gridgain 
Date:   2018-02-08T12:29:42Z

IGNITE-6625: JDBC thin: support SSL connection to Ignite node

This closes #3233

commit 2d729bf5c6f6fca9d07be2d57850642ba4b55004
Author: tledkov-gridgain 
Date:   2018-02-09T11:08:15Z

IGNITE-6625: SSL support for thin JDBC: additional fix test; change error 
message

commit 8994f847d7f5f15db73706d9210cdccb1cf3fb26
Author: devozerov 
Date:   2018-02-12T13:34:24Z

IGNITE-7003: Fixed faulty test 
WalModeChangeAdvancedSelfTest.testJoinCoordinator.

commit b142712264007d7397d1594541681a4a7e3d4b93
Author: Igor Sapego 
Date:   2018-02-26T12:02:07Z

IGNITE-7362: Fixed PDO ODBC integration issue

commit a2b2aee52cc65d01f2ecaf9462adc4bd368438ea
Author: Igor Sapego 
Date:   2018-02-28T10:23:12Z

IGNITE-7763: Fixed 32bit tests configurations to prevent OOM

This closes #3557

commit 652f3c4cdbaad40f5de25b06f0c13710aa7f2fd9
Author: Pavel Kuznetsov 
Date:   2018-03-13T12:46:36Z

IGNITE-7531: Data load benchmarks. This closes #3571.

commit 9337a53d9fcd62af87f6760080d350b43e275105
Author: tledkov-gridgain 
Date:   2018-03-16T11:38:38Z

IGNITE-7879: Fixed splitter logic for DISTINCT with subqueries. This closes 
#3634.

commit 7bec8b13cb373002d2a6b1b268d410338259bac2
Author: Igor Sapego 
Date:   2018-03-19T11:17:33Z

IGNITE-7811: Implemented connection failover for ODBC

This closes #3643

commit e512e5e0a2602df0ecfb69b2b5efabce836b04db
Author: Igor Sapego 
Date:   2018-03-20T10:37:02Z

IGNITE-7888: Implemented login timeout for ODBC.

This closes #3657

commit 211fca3a55e84b78ff0c1af04d91e25d6fc846c4
Author: devozerov 
Date:   2018-03-20T11:13:46Z

IGNITE-7984: SQL: improved deadlock handling in DML. This closes #3655.

commit bcd2888d27afe65f1a060e35b99a05ea420979c7
Author: Roman Guseinov 
Date:   2018-02-16T09:57:26Z

IGNITE-7192: Implemented JDBC support FQDN to multiple IPs

This closes #3439

commit d2659d0ec9f6e1a0b905fc7bf23b65fd5522c80a
Author: Alexander Paschenko 
Date:   2018-03-14T09:23:37Z

IGNITE-7253: JDBC thin driver: implemented streaming. This closes #3499. 
This closes #3591.

commit bc9018ef8b116f81b8e06d2ff7651ba2b6c7beae
Author: tledkov-gridgain 
Date:   2018-03-19T08:01:26Z

IGNITE-7029: JDBC thin driver now supports multiple IP addresses with 
transparent failover. This closes #3585.

commit 587022862fd5bdbb076ab6207ae6fd9bc7583c13
Author: Sergey Chugunov 
Date:   2018-03-16T16:24:17Z

IGNITE-7964 rmvId is stored to MetaStorage metapage during operations - 
Fixes #3645.

commit 006ef4d15e4faedb6dfce6ce9637602055b97293
Author: tledkov-gridgain 
Date:   2018-03-22T11:47:06Z

IGNITE-7436: Simple username/password authentication. This closes #3483.

commit 1c7f59c90514670e802d9d07544b00b7562fe6d2
Author: Pavel Tupitsyn 
Date:   2018-01-30T09:48:16Z

.NET: Fix build status icon in README

commit 162df61b305fccfc55e186d07351727f35b55179
Author: Pavel Tupitsyn 
Date:   2018-02-01T11:53:16Z

IGNITE-7561 .NET: Add IServices.GetDynamicServiceProxy

This closes #3457

commit 9a0328ebbc9d52f8e96898a384fa45743d2efa5b
Author: Pavel Tupitsyn 
Date:   2018-02-02T12:01:27Z

.NET: Update README regarding C++ interop and thin client

commit b804cfea51c87724b45b40de4fd58d300c049be1
Author: Pavel Tupitsyn 
Date:   2018-01-31T09:39:19Z

.NET: Suppress API parity check for SSL in ClientConnectorConfiguration

commit 6f8014de7250c4c0d87cbc8764afae4a225f654b
Author: apopov 
Date:   2018-02-13T10:13:15Z

IGNITE-3111 .NET can be now configured SSL without Spring

This closes #3498

commit 5131bcd71ce787cf2c61bf98446f5ec0a616ab1c
Author: Pavel Tupitsin 
Date:   2018-02-16T20:36:01Z

IGNITE-3111 .NET: Configure SSL without Spring - cleanup

* Remove unused members from ISslContextFactory
* Fix namespaces
* Remove unused files
* Cleanup tests

commit 4ac4645dcf6e85883ce0de46ba1253ba8135804e
Author: Pavel Tupitsyn 
Date:   2018-02-18T20:22:27Z

.NET: Fix LoadDllTest, 

Re: welcome letter

[Srinivas Reddy]

Can you please add me to contributors group?

Jira : mrsrinivas

Thank you

-
Srinivas

- Typed on tiny keys. pls ignore typos.{mobile app}

On Tue, 4 Dec, 2018, 04:01 Dmitriy Pavlov  Hi Vladimir,
>
> Welcome to the Apache Software Foundation and to the Apache Ignite
> Community.
>
> I've added your account to the list of contributors. Now you should be able
> to assign an issue to yourself.
>
> Should you have any questions please do not hesitate to ask here. Looking
> forward to your contributions.
>
> Sincerely,
> Dmitriy Pavlov
>
> P.S. Additional references that should boost your onboarding.
>
> Please subscribe to both dev@ and user@ lists, optionally you may
> subscribe
> to notifications@:
> https://ignite.apache.org/community/resources.html#mail-lists
>
> Get familiar with Apache Ignite development process described here:
> https://cwiki.apache.org/confluence/display/IGNITE/Development+Process
>
> Instructions on how to contribute can be found here:
> https://cwiki.apache.org/confluence/display/IGNITE/How+to+Contribute
>
> Project setup in Intellij IDEA:
> https://cwiki.apache.org/confluence/display/IGNITE/Project+Setup
>
> пн, 3 дек. 2018 г. в 22:57, Владимир Плигин :
>
> > Hello Ignite Community!
> >
> >
> > My name is Vladimir. I want to contribute to Apache Ignite, my JIRA
> > username is "Vladimir Pligin".
> >
> > Thanks!
> >
>

Re: Default failure handler was changed for tests

[Dmitriy Pavlov]

Folks let me remind you that Dmitry changed default of ALL tests from noop
to a meaningful handler. So we should start every message here from saying
thank you to Dmitry.

Please review remaining tests and remove noop where possible.

вт, 4 дек. 2018 г., 23:48 Andrey Mashenkov :

> Hi all,
>
> Really, why noop?
>
> If you expect failure handler should be triggered, you can override default
> one and rise some flag, which can be checked in test.
> This will make test clearer.
>
> With noop, you'll get previous unwanted  behavior, that you are trying to
> improve, isnt'it?
>
> 4 дек. 2018 г. 23:25 пользователь "Anton Vinogradov" 
> написал:
>
> And you have to check the reason of failure inside the try-catch block, of
> course.
> In case found not equals to expected then test should rethrow the
> exception.
>
>
> вт, 4 дек. 2018 г. в 23:21, Anton Vinogradov :
>
>
> > Dmitrii,
> >
> > The solution is not clear to me.
> > In case you expect the failure then a correct case is to wrap it with
> > try-catch block instead of no-op failure handler usage.
> >
> > вт, 4 дек. 2018 г. в 21:41, Dmitrii Ryabov :
> >
> >> Anton,
> >>
> >> Tests in these classes check fail cases when we expect critical
> >> failure like node stop or exception thrown. Such tests trigger failure
> >> handler and it fails test when everything goes as it should go. That's
> >> why we need no-op handler here.
> >> вт, 4 дек. 2018 г. в 20:06, Dmitriy Pavlov :
> >> >
> >> > Hi Igniters,
> >> >
> >> > BTW, if you find in any of your tests it does't need an old value of
> >> > handler (=NoOp), feel free to remove it.
> >> >
> >> > Sincerely,
> >> > Dmitriy Pavlov
> >> >
> >> > вт, 4 дек. 2018 г. в 20:02, Anton Vinogradov :
> >> >
> >> > > Dmitrii,
> >> > >
> >> > > Could you please explain the reason of explicit set of 100+
> >> > > NoOpFailureHandlers?
> >> > >
> >> > >
> >> > > вт, 4 дек. 2018 г. в 19:12, Dmitrii Ryabov :
> >> > >
> >> > > > Hello, Igniters!
> >> > > >
> >> > > > Today the test framework's default no-op failure handler was
> >> changed to
> >> > > the
> >> > > > handler, which stops the node and fails the test.
> >> > > >
> >> > > > Over 100 tests kept no-op failure handler by overrided
> >> > > > `getFailureHandler()` method.
> >> > > >
> >> > > > If you'll found a problem or something unexpected - write here or
> >> in the
> >> > > > ticket [1].
> >> > > >
> >> > > > [1] https://issues.apache.org/jira/browse/IGNITE-8227
> >> > > >
> >> > >
> >>
> >
>

[jira] [Created] (IGNITE-10534) DDL operations don't work on not started caches

[Yury Gerzhedovich (JIRA)]

Yury Gerzhedovich created IGNITE-10534:
--

 Summary: DDL operations don't work on not started caches
 Key: IGNITE-10534
 URL: https://issues.apache.org/jira/browse/IGNITE-10534
 Project: Ignite
  Issue Type: Bug
  Components: sql
Affects Versions: 2.8
Reporter: Yury Gerzhedovich
Assignee: Yury Gerzhedovich
 Fix For: 2.8


Exception trows in case try execute create or drop index operations for not 
started cache on not affinity node 

java.lang.AssertionError
 at 
org.apache.ignite.internal.processors.query.h2.ddl.DdlStatementsProcessor.isDdlSupported(DdlStatementsProcessor.java:532)



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] ignite pull request #5574: Ignite-10534

[ygerzhedovich]

GitHub user ygerzhedovich opened a pull request:

https://github.com/apache/ignite/pull/5574

Ignite-10534



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-10534

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/5574.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #5574


commit 3b9ac48f1283832fae439be69f140901f1507c0a
Author: Yury Gerzhedovich 
Date:   2018-08-31T14:24:48Z

IGNITE-6173: dynamical start caches at client nodes.

commit 26f6ecad0ddb4890eebd5a2a21d82f2c0baf1f32
Author: Yury Gerzhedovich 
Date:   2018-08-31T14:46:57Z

Merge branch 'master' into ignite-6173

# Conflicts:
#   
modules/core/src/main/java/org/apache/ignite/internal/processors/query/GridQueryIndexing.java
#   
modules/core/src/test/java/org/apache/ignite/internal/processors/cache/IgniteClientCacheInitializationFailTest.java
#   
modules/indexing/src/main/java/org/apache/ignite/internal/processors/query/h2/DmlStatementsProcessor.java
#   
modules/indexing/src/main/java/org/apache/ignite/internal/processors/query/h2/IgniteH2Indexing.java
#   
modules/indexing/src/main/java/org/apache/ignite/internal/processors/query/h2/dml/UpdatePlanBuilder.java
#   
modules/indexing/src/main/java/org/apache/ignite/internal/processors/query/h2/opt/GridH2RowDescriptor.java

commit be5b92a12c2d1edb64d7265e81faf451bd083349
Author: Yury Gerzhedovich 
Date:   2018-08-31T15:28:05Z

IGNITE-6173: fixes after merge with MVCC master

commit a4d828dacf4d5d8e7048f04e73e5a1ee9c0f949c
Author: Yury Gerzhedovich 
Date:   2018-09-11T10:04:03Z

Merge branch 'master' into ignite-6173

commit e72480b959408c57f9c137c4da72b82439dda10c
Author: Yury Gerzhedovich 
Date:   2018-09-11T15:14:49Z

Merge branch 'master' into ignite-6173

commit c913378c9582892b150fdee1b4a7a64dd18b5890
Author: Yury Gerzhedovich 
Date:   2018-09-11T16:24:57Z

ignite-6173: minor fixes after review

commit 869eb0861132aed3da46268bc448cf078b60b394
Author: Yury Gerzhedovich 
Date:   2018-09-12T07:42:35Z

Merge branch 'master' into ignite-6173

commit 02dc5d89c45c06b5706f5289fcc16dd1ef6ce9fe
Author: Yury Gerzhedovich 
Date:   2018-09-12T09:21:46Z

ignite-6173: add license to new class

commit 56bd9d93ff2adb7799d1faae8c73d9d0897cefe2
Author: Yury Gerzhedovich 
Date:   2018-09-12T11:39:22Z

Merge branch 'master' into ignite-6173

commit fc288b0c65cb8f7250e6471102fef0f6b0383d05
Author: Yury Gerzhedovich 
Date:   2018-09-12T14:26:04Z

Merge branch 'master' into ignite-6173

commit 482000d251aeb77ac742b1745e4d1d11bfd8846a
Author: Yury Gerzhedovich 
Date:   2018-09-13T11:50:08Z

Merge branch 'master' into ignite-6173

commit 8711ab12d69e3c42a32887604145f30fc50bf1b0
Author: Yury Gerzhedovich 
Date:   2018-09-14T12:08:33Z

Merge branch 'master' into ignite-6173

commit 326a4f84fa1950fcb634e89373d06ba7ed616e4a
Author: Yury Gerzhedovich 
Date:   2018-09-17T09:55:05Z

Merge branch 'master' into ignite-6173

commit 3b66b1a1b6e9c037d18e6ae9bf6376b299618704
Author: Yury Gerzhedovich 
Date:   2018-09-17T14:14:43Z

ignite-6173: test fixes

commit 4d631e72fbdbbe08b0946c0922c9958f493bf01d
Author: Yury Gerzhedovich 
Date:   2018-09-18T09:03:53Z

ignite-6173: test fixes

commit dee641ab0f8001c17cf17c374fbba11805cf820d
Author: Yury Gerzhedovich 
Date:   2018-09-18T09:17:52Z

Merge branch 'master' into ignite-6173

# Conflicts:
#   
modules/indexing/src/main/java/org/apache/ignite/internal/processors/query/h2/dml/UpdatePlanBuilder.java

commit f2d55eaefc82c8d1a3279602e1d84e97ba817014
Author: Yury Gerzhedovich 
Date:   2018-09-18T10:25:56Z

Merge branch 'master' into ignite-6173

commit d7d5ca6632af88526f7e1767c271fe32730ea788
Author: Yury Gerzhedovich 
Date:   2018-09-19T08:53:07Z

Merge branch 'master' into ignite-6173

commit edb2f91505f4b8f9b8acfd18fe74d7223c66e131
Author: tledkov-gridgain 
Date:   2018-09-19T13:01:14Z

Merge branch '_master' into ignite-6173

commit 5da29e28fdef4fcd78d0777e0d726dc39f7f4983
Author: tledkov-gridgain 
Date:   2018-09-19T13:15:16Z

IGNITE-6173: minors

commit 6867fb3742d75b24275bb53c1c8abf5f348a6648
Author: tledkov-gridgain 
Date:   2018-09-19T13:48:08Z

IGNITE-6173: minors

commit e1e2d015a9cd4f983f9c19bbfe5d5cab48349929
Author: Yury Gerzhedovich 
Date:   2018-09-19T14:13:18Z

Merge branch 'master' into ignite-6173

commit 983e0a83da19a05b8a68e75a3bd70dda12ab31fb
Author: Yury Gerzhedovich 
Date:   2018-09-19T14:30:52Z

ignite-6173: minor fixes after review

commit 34c46d4bc5a90328ec314f677564d867e053d681
Author: Yury Gerzhedovich 
Date:   2018-09-19T14:31:37Z

Merge remote-tracking branch 'professional/ignite-6173' into ignite-6173

# Conflicts:
#   
modules/indexing/src/main/ja