[DISCUSSION] Ignite.C++ and CMake

[Ivan Daschinskiy]

Hello Igniters.

I’d like to discuss porting build process of Ignite.C++. I think that there is 
time to change it.

*Motivation*
Currently, it is hard to build Ignite.C++. Different build process for windows 
and linux, lack of building support on Mac OS X (quite popular OS among 
developers), absolutely not IDE support, except windows and only Visual Studio 
is supported. 

*Suggestion*
I’d suggest to migrate to CMake build system. It is very popular among open 
source projects, and in Apache Software Foundation too. Notable user: Apache 
Mesos, Apache Zookeeper (C client offers CMake as an alternative to autoconf 
and only option on windows), Apache Kafka (librdkafka - C/C++ client), Apache 
Thrift. Popular column-oriented database ClickHouse also uses CMake.

CMake is widely supported in many IDE’s on various platforms, notably Visual 
Studio, CLion, Xcode, QtCreator, KDevelop. 

*Current status*

Currently, the most of work is done (see [1]) and tested on Mac OS X 10.15 
(some C++ porting). All tests are run without any flaws, including odbc 
(unixodbc), ssl, thin and thick client, installation, IDE integration (CLion). 
Next steps is to test linux and windows.

But full migration isn’t possible without agreement and help of community. Even 
if most of all you agree, migration requires additional efforts in TC agents 
tuning and so on (event though test running fully automated by CMake CTest). 

Lets discuss my proposition and idea. 

[1] - https://github.com/apache/ignite/pull/7845

Re: [VOTE] Release Apache Ignite 2.8.1 RC1

[Nikolay Izhikov]

+1 (binding).

We made some internal benchmarking and found performance boost in comparison 
with 2.8.0


> 22 мая 2020 г., в 22:42, Denis Magda  написал(а):
> 
> + 1 (binding)
> 
> Downloaded a binary version, started a cluster and ran some samples.
> 
> -
> Denis
> 
> 
> On Fri, May 22, 2020 at 3:57 AM Nikolay Izhikov  wrote:
> 
>> Dear Community,
>> 
>> I have uploaded release candidate to
>> https://dist.apache.org/repos/dist/dev/ignite/2.8.1-rc1/
>> https://dist.apache.org/repos/dist/dev/ignite/packages_2.8.1-rc1/
>> 
>> The following staging can be used for testing:
>> https://repository.apache.org/content/repositories/orgapacheignite-1481
>> 
>> This is the first maintenance release for 2.8.x with a number of fixes.
>> 
>> Tag name is 2.8.1-rc1:
>> https://gitbox.apache.org/repos/asf?p=ignite.git;a=tag;h=refs/tags/2.8.1-rc1
>> 
>> RELEASE NOTES:
>> https://gitbox.apache.org/repos/asf?p=ignite.git;a=blob;f=RELEASE_NOTES.txt;h=531b2ff7d72bdec9e0d9130efb27bf64f2b4f3fd;hb=9e214ee8ce7b87780d904111d359c74e90613e3f#l5
>> 
>> Complete list of resolved issues:
>> https://issues.apache.org/jira/issues/?jql=project%20%3D%20IGNITE%20AND%20fixVersion%20%3D%202.8.1
>> 
>> DEVNOTES
>> https://gitbox.apache.org/repos/asf?p=ignite.git;a=blob;f=DEVNOTES.txt;h=307189059ae70834ead4c127cc18295ce4d0735a;hb=9e214ee8ce7b87780d904111d359c74e90613e3
>> 
>> The vote is formal, see voting guidelines
>> https://www.apache.org/foundation/voting.html
>> 
>> +1 - to accept Apache Ignite 2.8.1-rc1
>> 0 - don't care either way
>> -1 - DO NOT accept Apache Ignite Ignite 2.8.1-rc1 (explain why)
>> 
>> See notes on how to verify release here
>> https://www.apache.org/info/verification.html
>> and
>> https://cwiki.apache.org/confluence/display/IGNITE/Release+Process#ReleaseProcess-P5.VotingonReleaseandReleaseVerification
>> 
>> Additional links:
>> 
>> NuGet staging -
>> https://www.myget.org/feed/apache-ignite-staging/package/nuget/Apache.Ignite
>> Comparsion of 2.8.0 and 2.8.1 -
>> https://ci.ignite.apache.org/repository/download/ApacheIgniteReleaseJava8_IgniteRelease72CheckFileConsistency/5329458:id/results/result.log
>> 
>> 
>> This vote will be open for at least 3 days till Sun May 22, 23:59:59 UTC.
>> https://www.timeanddate.com/countdown/generic?iso=20200525T235959&p0=%3A&msg=Ignite+2.8.1-rc1+Vote&font=cursive

Re[2]: [VOTE] Release Apache Ignite 2.8.1 RC1

[Zhenya Stanilovsky]

Nikolay, performance boost is always good news for all customers at all and for 
me too )
Can you give more info (looks like in different mail thread) ? What kind of 
benchmarks have been done, is it about persistence or both ?
thanks !
>+1 (binding).
>
>We made some internal benchmarking and found performance boost in comparison 
>with 2.8.0
>
> 
>> 22 мая 2020 г., в 22:42, Denis Magda < dma...@apache.org > написал(а):
>>
>> + 1 (binding)
>>
>> Downloaded a binary version, started a cluster and ran some samples.
>>
>> -
>> Denis
>>
>>
>> On Fri, May 22, 2020 at 3:57 AM Nikolay Izhikov < nizhi...@apache.org > 
>> wrote:
>>
>>> Dear Community,
>>>
>>> I have uploaded release candidate to
>>>  https://dist.apache.org/repos/dist/dev/ignite/2.8.1-rc1/
>>>  https://dist.apache.org/repos/dist/dev/ignite/packages_2.8.1-rc1/
>>>
>>> The following staging can be used for testing:
>>>  https://repository.apache.org/content/repositories/orgapacheignite-1481
>>>
>>> This is the first maintenance release for 2.8.x with a number of fixes.
>>>
>>> Tag name is 2.8.1-rc1:
>>>  
>>> https://gitbox.apache.org/repos/asf?p=ignite.git;a=tag;h=refs/tags/2.8.1-rc1
>>>
>>> RELEASE NOTES:
>>>  
>>> https://gitbox.apache.org/repos/asf?p=ignite.git;a=blob;f=RELEASE_NOTES.txt;h=531b2ff7d72bdec9e0d9130efb27bf64f2b4f3fd;hb=9e214ee8ce7b87780d904111d359c74e90613e3f#l5
>>>
>>> Complete list of resolved issues:
>>>  
>>> https://issues.apache.org/jira/issues/?jql=project%20%3D%20IGNITE%20AND%20fixVersion%20%3D%202.8.1
>>>
>>> DEVNOTES
>>>  
>>> https://gitbox.apache.org/repos/asf?p=ignite.git;a=blob;f=DEVNOTES.txt;h=307189059ae70834ead4c127cc18295ce4d0735a;hb=9e214ee8ce7b87780d904111d359c74e90613e3
>>>
>>> The vote is formal, see voting guidelines
>>>  https://www.apache.org/foundation/voting.html
>>>
>>> +1 - to accept Apache Ignite 2.8.1-rc1
>>> 0 - don't care either way
>>> -1 - DO NOT accept Apache Ignite Ignite 2.8.1-rc1 (explain why)
>>>
>>> See notes on how to verify release here
>>>  https://www.apache.org/info/verification.html
>>> and
>>>  
>>> https://cwiki.apache.org/confluence/display/IGNITE/Release+Process#ReleaseProcess-P5.VotingonReleaseandReleaseVerification
>>>
>>> Additional links:
>>>
>>> NuGet staging -
>>>  
>>> https://www.myget.org/feed/apache-ignite-staging/package/nuget/Apache.Ignite
>>> Comparsion of 2.8.0 and 2.8.1 -
>>>  
>>> https://ci.ignite.apache.org/repository/download/ApacheIgniteReleaseJava8_IgniteRelease72CheckFileConsistency/5329458:id/results/result.log
>>>
>>>
>>> This vote will be open for at least 3 days till Sun May 22, 23:59:59 UTC.
>>>  
>>> https://www.timeanddate.com/countdown/generic?iso=20200525T235959&p0=%3A&msg=Ignite+2.8.1-rc1+Vote&font=cursive
>>>  
 
 
 
 

Re: [DISCUSSION] Ignite.C++ and CMake

[Zhenya Stanilovsky]

Ivan huge +1 with your proposal.
I had some problems with odbc tests building too, looks like cmake will make it 
more easy.
>Hello Igniters.
>
>I’d like to discuss porting build process of Ignite.C++. I think that there is 
>time to change it.
>
>*Motivation*
>Currently, it is hard to build Ignite.C++. Different build process for windows 
>and linux, lack of building support on Mac OS X (quite popular OS among 
>developers), absolutely not IDE support, except windows and only Visual Studio 
>is supported.
>
>*Suggestion*
>I’d suggest to migrate to CMake build system. It is very popular among open 
>source projects, and in Apache Software Foundation too. Notable user: Apache 
>Mesos, Apache Zookeeper (C client offers CMake as an alternative to autoconf 
>and only option on windows), Apache Kafka (librdkafka - C/C++ client), Apache 
>Thrift. Popular column-oriented database ClickHouse also uses CMake.
>
>CMake is widely supported in many IDE’s on various platforms, notably Visual 
>Studio, CLion, Xcode, QtCreator, KDevelop.
>
>*Current status*
>
>Currently, the most of work is done (see [1]) and tested on Mac OS X 10.15 
>(some C++ porting). All tests are run without any flaws, including odbc 
>(unixodbc), ssl, thin and thick client, installation, IDE integration (CLion). 
>Next steps is to test linux and windows.
>
>But full migration isn’t possible without agreement and help of community. 
>Even if most of all you agree, migration requires additional efforts in TC 
>agents tuning and so on (event though test running fully automated by CMake 
>CTest).
>
>Lets discuss my proposition and idea.
>
>[1] -  https://github.com/apache/ignite/pull/7845
>
>
>  
 
 
 
 

Re: [DISCUSSION] Ignite.C++ and CMake

[Pavel Tupitsyn]

+1

On Tue, May 26, 2020 at 12:02 PM Zhenya Stanilovsky
 wrote:

>
> Ivan huge +1 with your proposal.
> I had some problems with odbc tests building too, looks like cmake will
> make it more easy.
> >Hello Igniters.
> >
> >I’d like to discuss porting build process of Ignite.C++. I think that
> there is time to change it.
> >
> >*Motivation*
> >Currently, it is hard to build Ignite.C++. Different build process for
> windows and linux, lack of building support on Mac OS X (quite popular OS
> among developers), absolutely not IDE support, except windows and only
> Visual Studio is supported.
> >
> >*Suggestion*
> >I’d suggest to migrate to CMake build system. It is very popular among
> open source projects, and in Apache Software Foundation too. Notable user:
> Apache Mesos, Apache Zookeeper (C client offers CMake as an alternative to
> autoconf and only option on windows), Apache Kafka (librdkafka - C/C++
> client), Apache Thrift. Popular column-oriented database ClickHouse also
> uses CMake.
> >
> >CMake is widely supported in many IDE’s on various platforms, notably
> Visual Studio, CLion, Xcode, QtCreator, KDevelop.
> >
> >*Current status*
> >
> >Currently, the most of work is done (see [1]) and tested on Mac OS X
> 10.15 (some C++ porting). All tests are run without any flaws, including
> odbc (unixodbc), ssl, thin and thick client, installation, IDE integration
> (CLion). Next steps is to test linux and windows.
> >
> >But full migration isn’t possible without agreement and help of
> community. Even if most of all you agree, migration requires additional
> efforts in TC agents tuning and so on (event though test running fully
> automated by CMake CTest).
> >
> >Lets discuss my proposition and idea.
> >
> >[1] -  https://github.com/apache/ignite/pull/7845
> >
> >
> >
>
>
>
>

[jira] [Created] (IGNITE-13071) Improve test coverage for read-only cluster state

[Sergey Antonov (Jira)]

Sergey Antonov created IGNITE-13071:
---

 Summary: Improve test coverage for read-only cluster state
 Key: IGNITE-13071
 URL: https://issues.apache.org/jira/browse/IGNITE-13071
 Project: Ignite
  Issue Type: Improvement
Reporter: Sergey Antonov
Assignee: Sergey Antonov
 Fix For: 2.9


It would be nice to extend test coverage for this feature. 
We don't have tests for:
* data structures
* cache destroy/create
* cache clear
* DDL requests
* Cache updates from task/deployed service



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[DISCUSS] Extra test coverage for ACTIVE_READ_ONLY cluster state

[Sergey Antonov]

Hello, Igniters.

I introduced cluster read-only mode [1] and a new API for cluster state
change [2]. At the moment we don't have good test coverage for this
feature. I'm going to fix it and write tests and check that operations
are *denied
*in read-only mode:

   - data structures usage
   - cache create/clear/destroy
   - DDL requests
   - cache updates by task's execution / deployed service

And the following operations are *allowed *in read-only mode:

   - update of metastorage / distributed metastorage
   - updates to ignite-sys-cache
   - task's execution, but updates must be rejected
   - service deploy/undeploy, but updates must be rejected
   - data recovery on node join

I'll work under these tests in ticket [3].
Any objections?

[1] https://issues.apache.org/jira/browse/IGNITE-11256
[2] https://issues.apache.org/jira/browse/IGNITE-12225
[3] https://issues.apache.org/jira/browse/IGNITE-13071
-- 
BR, Sergey Antonov

[jira] [Created] (IGNITE-13072) Synchronization problems when different classloaders are used for deployment of same class

[Vladislav Pyatkov (Jira)]

Vladislav Pyatkov created IGNITE-13072:
--

 Summary: Synchronization problems when different classloaders are 
used for deployment of same class
 Key: IGNITE-13072
 URL: https://issues.apache.org/jira/browse/IGNITE-13072
 Project: Ignite
  Issue Type: Bug
Reporter: Vladislav Pyatkov
Assignee: Vladislav Pyatkov


If you concurrently deploy one class using different classloaders you can get 
error:

{noformat}

2020-04-28 
14:36:42.523[ERROR][sys-stripe-45-#46%GRID%GridNodeName%][o.a.i.i.m.d.GridDeploymentLocalStore]
 Found more than one active deployment for the same resource [cls=class 
org.some.class.old.InvokeIndexRemover, depMode=SHARED, dep=GridDeployment 
[ts=1588067100125, depMode=SHARED, 
clsLdr=org.some.class.factory.NodeClassLoader@14035d21, 
clsLdrId=85ab310c171-a9fad11c-9f8c-4d2a-8146-6c87254303e7, userVer=0, loc=true, 
sampleClsName=org.some.class.predicates.CompositePredicate, 
pendingUndeploy=false, undeployed=false, usage=0]]
 
2020-04-28 
14:36:42.544[ERROR][sys-stripe-45-#46%GRID%GridNodeName%][o.a.i.i.p.cache.GridCacheIoManager]
 Failed to process message [senderId=f104e069-9d80-4202-b50a-b3dc1804ac89, 
msg=GridNearAtomicSingleUpdateRequest [key=KeyCacheObject [hasValBytes=true], 
super=GridNearAtomicSingleUpdateRequest [key=KeyCacheObject [hasValBytes=true], 
parent=GridNearAtomicAbstractSingleUpdateRequest [nodeId=null, futId=1376257, 
topVer=AffinityTopologyVersion [topVer=35, minorTopVer=0], 
parent=GridNearAtomicAbstractUpdateRequest [res=null, flags=]
java.lang.AssertionError: null
 at 
org.apache.ignite.internal.managers.deployment.GridDeploymentLocalStore.getDeployment(GridDeploymentLocalStore.java:203)
 at 
org.apache.ignite.internal.managers.deployment.GridDeploymentManager.getLocalDeployment(GridDeploymentManager.java:383)
 at 
org.apache.ignite.internal.processors.cache.GridCacheDeploymentManager$CacheClassLoader.findClass(GridCacheDeploymentManager.java:802)
 at 
org.apache.ignite.internal.processors.cache.GridCacheDeploymentManager$CacheClassLoader.loadClass(GridCacheDeploymentManager.java:794)
 at org.apache.ignite.internal.util.IgniteUtils.forName(IgniteUtils.java:8561)
 at 
org.apache.ignite.internal.MarshallerContextImpl.getClass(MarshallerContextImpl.java:374)
 at 
org.apache.ignite.internal.binary.BinaryContext.descriptorForTypeId(BinaryContext.java:700)
 at 
org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize0(BinaryReaderExImpl.java:1757)
 at 
org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize(BinaryReaderExImpl.java:1716)
 at 
org.apache.ignite.internal.binary.GridBinaryMarshaller.deserialize(GridBinaryMarshaller.java:313)
 at 
org.apache.ignite.internal.binary.BinaryMarshaller.unmarshal0(BinaryMarshaller.java:99)
 at 
org.apache.ignite.marshaller.AbstractNodeNameAwareMarshaller.unmarshal(AbstractNodeNameAwareMarshaller.java:82)
 at org.apache.ignite.internal.util.IgniteUtils.unmarshal(IgniteUtils.java:9959)
 at 
org.apache.ignite.internal.util.IgniteUtils.unmarshal(IgniteUtils.java:10017)
 at 
org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridNearAtomicSingleUpdateInvokeRequest.finishUnmarshal(GridNearAtomicSingleUpdateInvokeRequest.java:200)
 at 
org.apache.ignite.internal.processors.cache.GridCacheIoManager.unmarshall(GridCacheIoManager.java:1560)
 at 
org.apache.ignite.internal.processors.cache.GridCacheIoManager.onMessage0(GridCacheIoManager.java:582)
 at 
org.apache.ignite.internal.processors.cache.GridCacheIoManager.handleMessage(GridCacheIoManager.java:386)
 at 
org.apache.ignite.internal.processors.cache.GridCacheIoManager.handleMessage(GridCacheIoManager.java:312)
 at 
org.apache.ignite.internal.processors.cache.GridCacheIoManager.access$100(GridCacheIoManager.java:102)
 at 
org.apache.ignite.internal.processors.cache.GridCacheIoManager$1.onMessage(GridCacheIoManager.java:301)
 at 
org.apache.ignite.internal.managers.communication.GridIoManager.invokeListener(GridIoManager.java:1556)
 at 
org.apache.ignite.internal.managers.communication.GridIoManager.processRegularMessage0(GridIoManager.java:1184)
 at 
org.apache.ignite.internal.managers.communication.GridIoManager.access$4200(GridIoManager.java:125)
 at 
org.apache.ignite.internal.managers.communication.GridIoManager$9.run(GridIoManager.java:1091)
 at 
org.apache.ignite.internal.util.StripedExecutor$Stripe.body(StripedExecutor.java:546)
 at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:120)
 at java.lang.Thread.run(Thread.java:748)

{noformat}

 

Looks like we lack synchronization for modifying 
{{LocalDeploymentSpi.ldrRsrcs}}.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Re: [DISCUSS] TC bot instructions for open source contributors

[Dmitriy Pavlov]

Hi Ivan, Igniters,

Yeah, our HTC is kind of a mile long. One try I've done to get a shorter
version is GitHub's Contributing.MD
https://github.com/apache/ignite/blob/master/CONTRIBUTING.md#contributing-to-apache-ignite


I used to share both versions - short and long to new contributors.

But I noticed we have checklist for GitHub. It is extremely essential to
have such small features, which simplifies understanding of process for
newcomers. Thank you for doing this.

Sincerely,
Dmitriy Pavlov

вт, 12 мая 2020 г. в 17:16, Ivan Pavlukhin :

> What worries me is that the page [1] is quite large. I would love to
> see a short (and sufficient enough) version for new contributors.
>
> [1] https://cwiki.apache.org/confluence/display/IGNITE/How+to+Contribute
>
> Best regards,
> Ivan Pavlukhin
>
> вт, 12 мая 2020 г. в 16:41, Dmitriy Pavlov :
> >
> > +1 for linking it from How to contribute  -
> > https://cwiki.apache.org/confluence/display/IGNITE/How+to+Contribute
> > to instructions related to the bot.
> >
> > May be eventually I would be able to contribute some automation into the
> > Bot to make this process easier and requiring less contributor
> > intervention.
> >
> > Sincerely,
> > Dmitriy Pavlov
> >
> > вт, 12 мая 2020 г. в 12:42, Ilya Kasnacheev :
> >
> > > Hello!
> > >
> > > I think it should be mentioned in [2] since this is the link we try to
> give
> > > to every new Ignite contributor.
> > >
> > > Regards.
> > > --
> > > Ilya Kasnacheev
> > >
> > >
> > > пн, 11 мая 2020 г. в 11:29, Ivan Pavlukhin :
> > >
> > > > Hi,
> > > >
> > > > Recently I faced problems explaining how a code contribution for
> > > > Ignite should be checked with TC bot. Initially I thought that this
> > > > process is well defined in our beloved contribution guidelines [1,
> 2].
> > > > But as I see neither of them explains the process directly. Finally I
> > > > found better instructions [3].
> > > >
> > > > So, the questions is should not we mention TC bot as earlier as
> > > > possible? Naively I would like to see some words about it in
> > > > CONTRIBUTING.md [1].
> > > >
> > > > Please, share your thoughts on this!
> > > >
> > > > [1] https://github.com/apache/ignite/blob/master/CONTRIBUTING.md
> > > > [2]
> https://cwiki.apache.org/confluence/display/IGNITE/How+to+Contribute
> > > > [3]
> > > >
> > >
> https://cwiki.apache.org/confluence/display/IGNITE/Apache+Ignite+Teamcity+Bot#ApacheIgniteTeamcityBot-HowtocheckaPRwiththeTCBot
> > > >
> > > > Best regards,
> > > > Ivan Pavlukhin
> > > >
> > >
>

[jira] [Created] (IGNITE-13073) NPE on closing node in communication SPI

[Nikita Tolstunov (Jira)]

Nikita Tolstunov created IGNITE-13073:
-

 Summary: NPE on closing node in communication SPI
 Key: IGNITE-13073
 URL: https://issues.apache.org/jira/browse/IGNITE-13073
 Project: Ignite
  Issue Type: Bug
Reporter: Nikita Tolstunov
Assignee: Nikita Tolstunov


I got this exception on TC. The issue can leads to stuck Thin Client Java suite.
I have reproduced it local, when ReliabilityTest#testFailover looped.

 

{{[19:44:27,889][SEVERE][sys-stripe-1-#39670%533fdeb2-8079-45a3-874b-368c89491fb4%][]
 Critical system error detected. Will be handled accordingly to configured 
handler [hnd=StopNodeOrHaltFailureHandler [tryStop=false, timeout=0, 
super=AbstractFailureHandler [ignoredFailureTypes=UnmodifiableSet 
[SYSTEM_WORKER_BLOCKED, SYSTEM_CRITICAL_OPERATION_TIMEOUT]]], 
failureCtx=FailureContext [type=SYSTEM_WORKER_TERMINATION, 
err=java.lang.NullPointerException]]
java.lang.NullPointerException
at 
org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.sendMessage0(TcpCommunicationSpi.java:2810)
at 
org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.sendMessage(TcpCommunicationSpi.java:2794)
at 
org.apache.ignite.internal.managers.communication.GridIoManager.send(GridIoManager.java:1724)
at 
org.apache.ignite.internal.managers.communication.GridIoManager.sendToGridTopic(GridIoManager.java:1830)
at 
org.apache.ignite.internal.processors.cache.GridCacheIoManager.send(GridCacheIoManager.java:1257)
at 
org.apache.ignite.internal.processors.cache.GridCacheIoManager.send(GridCacheIoManager.java:1296)
at 
org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.sendDeferredUpdateResponse(GridDhtAtomicCache.java:3556)
at 
org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.access$3300(GridDhtAtomicCache.java:138)
at 
org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache$DeferredUpdateTimeout.run(GridDhtAtomicCache.java:3802)
at 
org.apache.ignite.internal.util.StripedExecutor$Stripe.body(StripedExecutor.java:559)
at 
org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:119)
at java.lang.Thread.run(Thread.java:748)}}

 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Re: [jira] [Created] (IGNITE-13065) Ignite Web Console 2.7.0 not working when scanning cache Error: Cannot read property 'length' of null

[Ivan Pavlukhin]

Folks,

Indeed it seems that there is no web-console docker image for 2.8.0.

Can someone help here? Who knows what should be done to deploy new image?

2020-05-22 18:43 GMT+03:00, david (Jira) :
> david created IGNITE-13065:
> --
>
>  Summary: Ignite Web Console 2.7.0 not working when scanning
> cache Error: Cannot read property 'length' of null
>  Key: IGNITE-13065
>  URL: https://issues.apache.org/jira/browse/IGNITE-13065
>  Project: Ignite
>   Issue Type: Bug
> Reporter: david
>
>
> Hi,
>
> I am using docker to host the web console and have pulled the latest image
> from docker. I have upgraded my ignite server to 2.8.0 but co no longer
> perform scans of my cache
>
>
>
> Has a new docker image for the web console been pushed to docker to support
> ignite 2.8.0?
>
>
>
> thank you
>
> David
>
>
>
> --
> This message was sent by Atlassian Jira
> (v8.3.4#803005)
>


-- 

Best regards,
Ivan Pavlukhin

[jira] [Created] (IGNITE-13074) Thin client: Default compute cluster group should be "server nodes", but currently "all nodes"

[Aleksey Plekhanov (Jira)]

Aleksey Plekhanov created IGNITE-13074:
--

 Summary: Thin client: Default compute cluster group should be 
"server nodes", but currently "all nodes"
 Key: IGNITE-13074
 URL: https://issues.apache.org/jira/browse/IGNITE-13074
 Project: Ignite
  Issue Type: Bug
  Components: thin client
Reporter: Aleksey Plekhanov
Assignee: Aleksey Plekhanov


When executing compute tasks from thin client without specifying cluster group, 
tasks should be executed only on server nodes. But currently client nodes are 
also included. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Re: [DISCUSSION] Key and Value fields with same name and SQL DML

[Alexey Sasov]

Hello. Looks good. What whould be returned from the query like this "select
_key from cache" if we don't know the type of the key? ExpandoObject, Tuple,
BinaryObject? How would you change ICache interface which has this member?

/// Queries cache.
/// Query.
/// Cursor.
IQueryCursor> Query(QueryBase qry);



--
Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/

Re: [DISCUSSION] Key and Value fields with same name and SQL DML

[Nikolay Izhikov]

> Nikolay, btw, what's your thinking on the approach proposed by me?

Makes sense.

> 26 мая 2020 г., в 14:07, Alexey Sasov  написал(а):
> 
> Hello. Looks good. What whould be returned from the query like this "select
> _key from cache" if we don't know the type of the key? ExpandoObject, Tuple,
> BinaryObject? How would you change ICache interface which has this member?
> 
>/// Queries cache.
>/// Query.
>/// Cursor.
>IQueryCursor> Query(QueryBase qry);
> 
> 
> 
> --
> Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/

Re: [DISCUSSION] Ignite.C++ and CMake

[Alexey Kukushkin]

+1. I recently completed a cross-IDE (MS Visual Studio & GCC/GDB) Ignite
C++ project and CMake in Ignite C++ would save me a day of effort.

вт, 26 мая 2020 г. в 12:09, Pavel Tupitsyn :

> +1
>
> On Tue, May 26, 2020 at 12:02 PM Zhenya Stanilovsky
>  wrote:
>
> >
> > Ivan huge +1 with your proposal.
> > I had some problems with odbc tests building too, looks like cmake will
> > make it more easy.
> > >Hello Igniters.
> > >
> > >I’d like to discuss porting build process of Ignite.C++. I think that
> > there is time to change it.
> > >
> > >*Motivation*
> > >Currently, it is hard to build Ignite.C++. Different build process for
> > windows and linux, lack of building support on Mac OS X (quite popular OS
> > among developers), absolutely not IDE support, except windows and only
> > Visual Studio is supported.
> > >
> > >*Suggestion*
> > >I’d suggest to migrate to CMake build system. It is very popular among
> > open source projects, and in Apache Software Foundation too. Notable
> user:
> > Apache Mesos, Apache Zookeeper (C client offers CMake as an alternative
> to
> > autoconf and only option on windows), Apache Kafka (librdkafka - C/C++
> > client), Apache Thrift. Popular column-oriented database ClickHouse also
> > uses CMake.
> > >
> > >CMake is widely supported in many IDE’s on various platforms, notably
> > Visual Studio, CLion, Xcode, QtCreator, KDevelop.
> > >
> > >*Current status*
> > >
> > >Currently, the most of work is done (see [1]) and tested on Mac OS X
> > 10.15 (some C++ porting). All tests are run without any flaws, including
> > odbc (unixodbc), ssl, thin and thick client, installation, IDE
> integration
> > (CLion). Next steps is to test linux and windows.
> > >
> > >But full migration isn’t possible without agreement and help of
> > community. Even if most of all you agree, migration requires additional
> > efforts in TC agents tuning and so on (event though test running fully
> > automated by CMake CTest).
> > >
> > >Lets discuss my proposition and idea.
> > >
> > >[1] -  https://github.com/apache/ignite/pull/7845
> > >
> > >
> > >
> >
> >
> >
> >
>


-- 
Best regards,
Alexey

Re: Extended logging for rebalance performance analysis

[Alexei Scherbakov]

Hi, Kirill.

2. Ok for me.

3. We already have log message about rebalance cancellation for specific
rebalanceId and log message about new rebalancing has started with next
rebalanceId.

5. We already have summary information per group for each supplier added in
p.2
I would avoid duplication here or put it under debug logging.

Information about selected suppliers can already be obtained from logs:
[2020-05-06 20:56:37,034][INFO ][...] Prepared rebalancing [grp=cache1,
mode=ASYNC, supplier=94a3fcbc-18d5-4c64-b0ab-4313aba1,
partitionsCount=11, topVer=AffinityTopologyVersion [topVer=3,
minorTopVer=1]]
[2020-05-06 20:56:37,036][INFO ][...] Prepared rebalancing [grp=cache1,
mode=ASYNC, supplier=b3f3aeeb-5fa0-42f7-a74e-cf39fa50,
partitionsCount=10, topVer=AffinityTopologyVersion [topVer=3,
minorTopVer=1]]

so I still do not see any value in "detailed" message.

I think this is enough to understand rebalancing flow using proper grepping
by topVer and rebalanceId.

All additional aggregation should be done by external tools using
rebalancing metrics.
I'm on the same page with Maxim Muzafarov here.



ср, 20 мая 2020 г. в 23:08, ткаленко кирилл :

> Hello, Alexey! Unfortunately, my response was delayed.
>
> Point 2: You can do as you suggested, I think it is still worth specifying
> how many partitions were obtained.
>
> [2020-05-06 20:56:37,044][INFO ][...] Completed rebalancing [grp=cache1,
> supplier=94a3fcbc-18d5-4c64-b0ab-4313aba1, partitions=5, entries=100,
> duration=12ms,
> bytesRcvd=5M, topVer=AffinityTopologyVersion [topVer=3, minorTopVer=1],
> progress=1/2]
>
> Point 3: is It "rebalanceId"?
>
> Point 5: I think we can output a summary for each supplier, so as not to
> keep it in mind.
>
> [2020-05-06 20:56:36,999][INFO ][...] Completed rebalance chain:
> [rebalanceId=2, [supplier=94a3fcbc-18d5-4c64-b0ab-4313aba1,
> partitions=5, entries=100, duration=12ms, bytesRcvd=5M],
> [supplier=94a3fcbc-18d5-4c64-b0ab-4313aba2, partitions=5, entries=100,
> duration=12ms, bytesRcvd=5M]]
>
> I can add "rebalanceId" to each message that you gave at above.
>
> A detailed message will help us understand how correctly the suppliers
> were selected.
>
> 06.05.2020, 22:08, "Alexei Scherbakov" :
> > Hello.
> >
> > Let's look at existing rebalancing log for a single group:
> >
> > [2020-05-06 20:56:36,999][INFO ][...] Rebalancing scheduled
> > [order=[ignite-sys-cache, cache1, cache2, default],
> > top=AffinityTopologyVersion [topVer=3, minorTopVer=1],
> > evt=DISCOVERY_CUSTOM_EVT, node=9d9edb7b-eb01-47a1-8ff9-fef715d2]
> > ...
> > [2020-05-06 20:56:37,034][INFO ][...] Prepared rebalancing [grp=cache1,
> > mode=ASYNC, supplier=94a3fcbc-18d5-4c64-b0ab-4313aba1,
> > partitionsCount=11, topVer=AffinityTopologyVersion [topVer=3,
> > minorTopVer=1]]
> > [2020-05-06 20:56:37,036][INFO ][...] Prepared rebalancing [grp=cache1,
> > mode=ASYNC, supplier=b3f3aeeb-5fa0-42f7-a74e-cf39fa50,
> > partitionsCount=10, topVer=AffinityTopologyVersion [topVer=3,
> > minorTopVer=1]]
> > [2020-05-06 20:56:37,036][INFO ][...] Starting rebalance routine [cache1,
> > topVer=AffinityTopologyVersion [topVer=3, minorTopVer=1],
> > supplier=94a3fcbc-18d5-4c64-b0ab-4313aba1, fullPartitions=[1, 5, 7,
> 9,
> > 11, 13, 15, 23, 27, 29, 31], histPartitions=[]]
> > [2020-05-06 20:56:37,037][INFO ][...] Starting rebalance routine [cache1,
> > topVer=AffinityTopologyVersion [topVer=3, minorTopVer=1],
> > supplier=b3f3aeeb-5fa0-42f7-a74e-cf39fa50, fullPartitions=[6, 8, 10,
> > 16, 18, 20, 22, 24, 26, 28], histPartitions=[]]
> > [2020-05-06 20:56:37,044][INFO ][...] Completed rebalancing [grp=cache1,
> > supplier=94a3fcbc-18d5-4c64-b0ab-4313aba1,
> > topVer=AffinityTopologyVersion [topVer=3, minorTopVer=1], progress=1/2]
> > [2020-05-06 20:56:37,046][INFO ][...] Completed (final) rebalancing
> > [grp=cache1, supplier=b3f3aeeb-5fa0-42f7-a74e-cf39fa50,
> > topVer=AffinityTopologyVersion [topVer=3, minorTopVer=1], progress=2/2]
> > [2020-05-06 20:56:37,048][INFO ][...] Completed rebalance future:
> > RebalanceFuture [grp=CacheGroupContext [grp=cache1],
> > topVer=AffinityTopologyVersion [topVer=3, minorTopVer=1], rebalanceId=2,
> > routines=2]
> >
> > From these logs I'm already can get answers to 1 and 4.
> > The logs look concise and easy to read and understand, and should
> > remain what way.
> >
> > But I think some proposed improvements can be done here without harm.
> >
> > 2. OK, let's add it to supplier info per cache with additional info:
> >
> > [2020-05-06 20:56:37,044][INFO ][...] Completed rebalancing [grp=cache1,
> > supplier=94a3fcbc-18d5-4c64-b0ab-4313aba1, entries=100,
> duration=12ms,
> > bytesRcvd=5M, topVer=AffinityTopologyVersion [topVer=3, minorTopVer=1],
> > progress=1/2]
> >
> > 3. This information is already printed to log.
> >
> > 5. OK, let's add a line which is printed after all groups in the chain
> are
> > rebalanced or cancelled with a summary:
> >
> > [2020-05-06 20:56:36,999][INFO ][.

[jira] [Created] (IGNITE-13075) NPE on request JDBC metadata

[Konstantin Orlov (Jira)]

Konstantin Orlov created IGNITE-13075:
-

 Summary: NPE on request JDBC metadata
 Key: IGNITE-13075
 URL: https://issues.apache.org/jira/browse/IGNITE-13075
 Project: Ignite
  Issue Type: Bug
  Components: sql
Reporter: Konstantin Orlov
Assignee: Konstantin Orlov


NPE occure when metadata is requested and there is a table created through java 
API like a
{code:java}
ignite.createCache(
new CacheConfiguration<>(DEFAULT_CACHE_NAME)
.setIndexedTypes(Integer.class, String.class)
);{code}
Error here:
{code:java}
[2020-05-26 
15:52:24,502][ERROR][client-connector-#264%thin.JdbcThinMetadataSelfTest0%][JdbcRequestHandler]
 Failed to get columns metadata [reqId=15, req=JdbcMetaColumnsRequest 
[schemaName=testCache, tblName=null, colName=null]][2020-05-26 
15:52:24,502][ERROR][client-connector-#264%thin.JdbcThinMetadataSelfTest0%][JdbcRequestHandler]
 Failed to get columns metadata [reqId=15, req=JdbcMetaColumnsRequest 
[schemaName=testCache, tblName=null, 
colName=null]]java.lang.NullPointerException at 
org.apache.ignite.internal.processors.query.h2.IgniteH2Indexing.lambda$null$10(IgniteH2Indexing.java:1902)
 at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193) 
at java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:175) at 
java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:175) at 
java.util.Spliterators$ArraySpliterator.forEachRemaining(Spliterators.java:948) 
at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481) at 
java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471) at 
java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:151) 
at 
java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:174)
 at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) at 
java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:418) at 
java.util.stream.ReferencePipeline$7$1.accept(ReferencePipeline.java:270) at 
java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:175) at 
java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:175) at 
java.util.concurrent.ConcurrentHashMap$ValueSpliterator.forEachRemaining(ConcurrentHashMap.java:3566)
 at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481) at 
java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471) at 
java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:151) 
at 
java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:174)
 at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) at 
java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:418) at 
org.apache.ignite.internal.processors.query.h2.IgniteH2Indexing.columnsInformation(IgniteH2Indexing.java:1910)
 at 
org.apache.ignite.internal.processors.odbc.jdbc.JdbcMetadataInfo.getColumnsMeta(JdbcMetadataInfo.java:180)
 at 
org.apache.ignite.internal.processors.odbc.jdbc.JdbcRequestHandler.getColumnsMeta(JdbcRequestHandler.java:1128)
 at 
org.apache.ignite.internal.processors.odbc.jdbc.JdbcRequestHandler.doHandle(JdbcRequestHandler.java:348)
 at 
org.apache.ignite.internal.processors.odbc.jdbc.JdbcRequestHandler.handle(JdbcRequestHandler.java:257)
 at 
org.apache.ignite.internal.processors.odbc.ClientListenerNioListener.onMessage(ClientListenerNioListener.java:200)
 at 
org.apache.ignite.internal.processors.odbc.ClientListenerNioListener.onMessage(ClientListenerNioListener.java:54)
 at 
org.apache.ignite.internal.util.nio.GridNioFilterChain$TailFilter.onMessageReceived(GridNioFilterChain.java:279)
 at 
org.apache.ignite.internal.util.nio.GridNioFilterAdapter.proceedMessageReceived(GridNioFilterAdapter.java:109)
 at 
org.apache.ignite.internal.util.nio.GridNioAsyncNotifyFilter$3.body(GridNioAsyncNotifyFilter.java:97)
 at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:120) 
at 
org.apache.ignite.internal.util.worker.GridWorkerPool$1.run(GridWorkerPool.java:70)
 at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) 
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) 
at java.lang.Thread.run(Thread.java:748){code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Re: [DISCUSSION] Ignite.C++ and CMake

[Igor Sapego]

Great!

Let's start with creating a TC suite for it.

The only concern I have is that it is one more build system
to support. Should we get rid of autotools in 3.0?

Best Regards,
Igor


On Tue, May 26, 2020 at 2:44 PM Alexey Kukushkin 
wrote:

> +1. I recently completed a cross-IDE (MS Visual Studio & GCC/GDB) Ignite
> C++ project and CMake in Ignite C++ would save me a day of effort.
>
> вт, 26 мая 2020 г. в 12:09, Pavel Tupitsyn :
>
> > +1
> >
> > On Tue, May 26, 2020 at 12:02 PM Zhenya Stanilovsky
> >  wrote:
> >
> > >
> > > Ivan huge +1 with your proposal.
> > > I had some problems with odbc tests building too, looks like cmake will
> > > make it more easy.
> > > >Hello Igniters.
> > > >
> > > >I’d like to discuss porting build process of Ignite.C++. I think that
> > > there is time to change it.
> > > >
> > > >*Motivation*
> > > >Currently, it is hard to build Ignite.C++. Different build process for
> > > windows and linux, lack of building support on Mac OS X (quite popular
> OS
> > > among developers), absolutely not IDE support, except windows and only
> > > Visual Studio is supported.
> > > >
> > > >*Suggestion*
> > > >I’d suggest to migrate to CMake build system. It is very popular among
> > > open source projects, and in Apache Software Foundation too. Notable
> > user:
> > > Apache Mesos, Apache Zookeeper (C client offers CMake as an alternative
> > to
> > > autoconf and only option on windows), Apache Kafka (librdkafka - C/C++
> > > client), Apache Thrift. Popular column-oriented database ClickHouse
> also
> > > uses CMake.
> > > >
> > > >CMake is widely supported in many IDE’s on various platforms, notably
> > > Visual Studio, CLion, Xcode, QtCreator, KDevelop.
> > > >
> > > >*Current status*
> > > >
> > > >Currently, the most of work is done (see [1]) and tested on Mac OS X
> > > 10.15 (some C++ porting). All tests are run without any flaws,
> including
> > > odbc (unixodbc), ssl, thin and thick client, installation, IDE
> > integration
> > > (CLion). Next steps is to test linux and windows.
> > > >
> > > >But full migration isn’t possible without agreement and help of
> > > community. Even if most of all you agree, migration requires additional
> > > efforts in TC agents tuning and so on (event though test running fully
> > > automated by CMake CTest).
> > > >
> > > >Lets discuss my proposition and idea.
> > > >
> > > >[1] -  https://github.com/apache/ignite/pull/7845
> > > >
> > > >
> > > >
> > >
> > >
> > >
> > >
> >
>
>
> --
> Best regards,
> Alexey
>

[jira] [Created] (IGNITE-13076) Cluster read-only mode doesn't affect LOCAL caches

[Sergey Antonov (Jira)]

Sergey Antonov created IGNITE-13076:
---

 Summary: Cluster read-only mode doesn't affect LOCAL caches
 Key: IGNITE-13076
 URL: https://issues.apache.org/jira/browse/IGNITE-13076
 Project: Ignite
  Issue Type: Bug
Affects Versions: 2.9
Reporter: Sergey Antonov
 Fix For: 2.9


You can make data modification operations with LOCAL cache even if cluster in a 
{{ACTIVE_READ_ONLY}} state. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Re: [DISCUSSION] Ignite.C++ and CMake

[Nikolay Izhikov]

Hello, Igor.

I’m ++1 for this change.

How build system relates to major version of product?
I think we can change the way we build modules any time we want.


> 26 мая 2020 г., в 16:02, Igor Sapego  написал(а):
> 
> Great!
> 
> Let's start with creating a TC suite for it.
> 
> The only concern I have is that it is one more build system
> to support. Should we get rid of autotools in 3.0?
> 
> Best Regards,
> Igor
> 
> 
> On Tue, May 26, 2020 at 2:44 PM Alexey Kukushkin 
> wrote:
> 
>> +1. I recently completed a cross-IDE (MS Visual Studio & GCC/GDB) Ignite
>> C++ project and CMake in Ignite C++ would save me a day of effort.
>> 
>> вт, 26 мая 2020 г. в 12:09, Pavel Tupitsyn :
>> 
>>> +1
>>> 
>>> On Tue, May 26, 2020 at 12:02 PM Zhenya Stanilovsky
>>>  wrote:
>>> 
 
 Ivan huge +1 with your proposal.
 I had some problems with odbc tests building too, looks like cmake will
 make it more easy.
> Hello Igniters.
> 
> I’d like to discuss porting build process of Ignite.C++. I think that
 there is time to change it.
> 
> *Motivation*
> Currently, it is hard to build Ignite.C++. Different build process for
 windows and linux, lack of building support on Mac OS X (quite popular
>> OS
 among developers), absolutely not IDE support, except windows and only
 Visual Studio is supported.
> 
> *Suggestion*
> I’d suggest to migrate to CMake build system. It is very popular among
 open source projects, and in Apache Software Foundation too. Notable
>>> user:
 Apache Mesos, Apache Zookeeper (C client offers CMake as an alternative
>>> to
 autoconf and only option on windows), Apache Kafka (librdkafka - C/C++
 client), Apache Thrift. Popular column-oriented database ClickHouse
>> also
 uses CMake.
> 
> CMake is widely supported in many IDE’s on various platforms, notably
 Visual Studio, CLion, Xcode, QtCreator, KDevelop.
> 
> *Current status*
> 
> Currently, the most of work is done (see [1]) and tested on Mac OS X
 10.15 (some C++ porting). All tests are run without any flaws,
>> including
 odbc (unixodbc), ssl, thin and thick client, installation, IDE
>>> integration
 (CLion). Next steps is to test linux and windows.
> 
> But full migration isn’t possible without agreement and help of
 community. Even if most of all you agree, migration requires additional
 efforts in TC agents tuning and so on (event though test running fully
 automated by CMake CTest).
> 
> Lets discuss my proposition and idea.
> 
> [1] -  https://github.com/apache/ignite/pull/7845
> 
> 
> 
 
 
 
 
>>> 
>> 
>> 
>> --
>> Best regards,
>> Alexey
>> 

Re: [DISCUSSION] Ignite.C++ and CMake

[Igor Sapego]

Nikolay, removing support for a certain build system is a breaking change.

Ivan, in C++ world there are companies that still use VS 2012 or even 2010.

Best Regards,
Igor


On Tue, May 26, 2020 at 4:08 PM Ivan Daschinskiy 
wrote:

> Guys, thank you all for support
>
>  >> The only concern I have is that it is one more build system
> to support. Should we get rid of autotools in 3.0?
>
> I'd suggest leave CMake as an only build system. Cmake could generate VS
> projects more than 12 years ago flawlessly, moreover, VS since 2017 has
> native support of CMake, so even this step (generation of project) is no
> more necessary.
>
>
> On 26.05.2020 16:02, Igor Sapego wrote:
> > Great!
> >
> > Let's start with creating a TC suite for it.
> >
> > The only concern I have is that it is one more build system
> > to support. Should we get rid of autotools in 3.0?
> >
> > Best Regards,
> > Igor
> >
> >
> > On Tue, May 26, 2020 at 2:44 PM Alexey Kukushkin <
> kukushkinale...@gmail.com>
> > wrote:
> >
> >> +1. I recently completed a cross-IDE (MS Visual Studio & GCC/GDB) Ignite
> >> C++ project and CMake in Ignite C++ would save me a day of effort.
> >>
> >> вт, 26 мая 2020 г. в 12:09, Pavel Tupitsyn :
> >>
> >>> +1
> >>>
> >>> On Tue, May 26, 2020 at 12:02 PM Zhenya Stanilovsky
> >>>  wrote:
> >>>
>  Ivan huge +1 with your proposal.
>  I had some problems with odbc tests building too, looks like cmake
> will
>  make it more easy.
> > Hello Igniters.
> >
> > I’d like to discuss porting build process of Ignite.C++. I think that
>  there is time to change it.
> > *Motivation*
> > Currently, it is hard to build Ignite.C++. Different build process
> for
>  windows and linux, lack of building support on Mac OS X (quite popular
> >> OS
>  among developers), absolutely not IDE support, except windows and only
>  Visual Studio is supported.
> > *Suggestion*
> > I’d suggest to migrate to CMake build system. It is very popular
> among
>  open source projects, and in Apache Software Foundation too. Notable
> >>> user:
>  Apache Mesos, Apache Zookeeper (C client offers CMake as an
> alternative
> >>> to
>  autoconf and only option on windows), Apache Kafka (librdkafka - C/C++
>  client), Apache Thrift. Popular column-oriented database ClickHouse
> >> also
>  uses CMake.
> > CMake is widely supported in many IDE’s on various platforms, notably
>  Visual Studio, CLion, Xcode, QtCreator, KDevelop.
> > *Current status*
> >
> > Currently, the most of work is done (see [1]) and tested on Mac OS X
>  10.15 (some C++ porting). All tests are run without any flaws,
> >> including
>  odbc (unixodbc), ssl, thin and thick client, installation, IDE
> >>> integration
>  (CLion). Next steps is to test linux and windows.
> > But full migration isn’t possible without agreement and help of
>  community. Even if most of all you agree, migration requires
> additional
>  efforts in TC agents tuning and so on (event though test running fully
>  automated by CMake CTest).
> > Lets discuss my proposition and idea.
> >
> > [1] -  https://github.com/apache/ignite/pull/7845
> >
> >
> >
> 
> 
> 
> >>
> >> --
> >> Best regards,
> >> Alexey
> >>
>

Re: [DISCUSSION] Ignite.C++ and CMake

[Ivan Daschinsky]

Igor, I just said about native support from Visual Studio, not from cmake
itself.

Modern cmake can generate projects even for Visual Studio 2008 [1]

[1] --
https://cmake.org/cmake/help/v3.16/generator/Visual%20Studio%209%202008.html

вт, 26 мая 2020 г. в 16:16, Igor Sapego :

> Nikolay, removing support for a certain build system is a breaking change.
>
> Ivan, in C++ world there are companies that still use VS 2012 or even 2010.
>
> Best Regards,
> Igor
>
>
> On Tue, May 26, 2020 at 4:08 PM Ivan Daschinskiy 
> wrote:
>
> > Guys, thank you all for support
> >
> >  >> The only concern I have is that it is one more build system
> > to support. Should we get rid of autotools in 3.0?
> >
> > I'd suggest leave CMake as an only build system. Cmake could generate VS
> > projects more than 12 years ago flawlessly, moreover, VS since 2017 has
> > native support of CMake, so even this step (generation of project) is no
> > more necessary.
> >
> >
> > On 26.05.2020 16:02, Igor Sapego wrote:
> > > Great!
> > >
> > > Let's start with creating a TC suite for it.
> > >
> > > The only concern I have is that it is one more build system
> > > to support. Should we get rid of autotools in 3.0?
> > >
> > > Best Regards,
> > > Igor
> > >
> > >
> > > On Tue, May 26, 2020 at 2:44 PM Alexey Kukushkin <
> > kukushkinale...@gmail.com>
> > > wrote:
> > >
> > >> +1. I recently completed a cross-IDE (MS Visual Studio & GCC/GDB)
> Ignite
> > >> C++ project and CMake in Ignite C++ would save me a day of effort.
> > >>
> > >> вт, 26 мая 2020 г. в 12:09, Pavel Tupitsyn :
> > >>
> > >>> +1
> > >>>
> > >>> On Tue, May 26, 2020 at 12:02 PM Zhenya Stanilovsky
> > >>>  wrote:
> > >>>
> >  Ivan huge +1 with your proposal.
> >  I had some problems with odbc tests building too, looks like cmake
> > will
> >  make it more easy.
> > > Hello Igniters.
> > >
> > > I’d like to discuss porting build process of Ignite.C++. I think
> that
> >  there is time to change it.
> > > *Motivation*
> > > Currently, it is hard to build Ignite.C++. Different build process
> > for
> >  windows and linux, lack of building support on Mac OS X (quite
> popular
> > >> OS
> >  among developers), absolutely not IDE support, except windows and
> only
> >  Visual Studio is supported.
> > > *Suggestion*
> > > I’d suggest to migrate to CMake build system. It is very popular
> > among
> >  open source projects, and in Apache Software Foundation too. Notable
> > >>> user:
> >  Apache Mesos, Apache Zookeeper (C client offers CMake as an
> > alternative
> > >>> to
> >  autoconf and only option on windows), Apache Kafka (librdkafka -
> C/C++
> >  client), Apache Thrift. Popular column-oriented database ClickHouse
> > >> also
> >  uses CMake.
> > > CMake is widely supported in many IDE’s on various platforms,
> notably
> >  Visual Studio, CLion, Xcode, QtCreator, KDevelop.
> > > *Current status*
> > >
> > > Currently, the most of work is done (see [1]) and tested on Mac OS
> X
> >  10.15 (some C++ porting). All tests are run without any flaws,
> > >> including
> >  odbc (unixodbc), ssl, thin and thick client, installation, IDE
> > >>> integration
> >  (CLion). Next steps is to test linux and windows.
> > > But full migration isn’t possible without agreement and help of
> >  community. Even if most of all you agree, migration requires
> > additional
> >  efforts in TC agents tuning and so on (event though test running
> fully
> >  automated by CMake CTest).
> > > Lets discuss my proposition and idea.
> > >
> > > [1] -  https://github.com/apache/ignite/pull/7845
> > >
> > >
> > >
> > 
> > 
> > 
> > >>
> > >> --
> > >> Best regards,
> > >> Alexey
> > >>
> >
>


-- 
Sincerely yours, Ivan Daschinskiy

Re: [DISCUSSION] Ignite.C++ and CMake

[Ilya Kasnacheev]

Hello!

I don't see why we can't get rid of autotools in a minor release, provided
that cmake actually works. Removing native VS support may be a different
thing.

Build system and precise set of dependencies is not a part of public API in
my opinion.

Regards,
-- 
Ilya Kasnacheev


вт, 26 мая 2020 г. в 16:02, Igor Sapego :

> Great!
>
> Let's start with creating a TC suite for it.
>
> The only concern I have is that it is one more build system
> to support. Should we get rid of autotools in 3.0?
>
> Best Regards,
> Igor
>
>
> On Tue, May 26, 2020 at 2:44 PM Alexey Kukushkin <
> kukushkinale...@gmail.com>
> wrote:
>
> > +1. I recently completed a cross-IDE (MS Visual Studio & GCC/GDB) Ignite
> > C++ project and CMake in Ignite C++ would save me a day of effort.
> >
> > вт, 26 мая 2020 г. в 12:09, Pavel Tupitsyn :
> >
> > > +1
> > >
> > > On Tue, May 26, 2020 at 12:02 PM Zhenya Stanilovsky
> > >  wrote:
> > >
> > > >
> > > > Ivan huge +1 with your proposal.
> > > > I had some problems with odbc tests building too, looks like cmake
> will
> > > > make it more easy.
> > > > >Hello Igniters.
> > > > >
> > > > >I’d like to discuss porting build process of Ignite.C++. I think
> that
> > > > there is time to change it.
> > > > >
> > > > >*Motivation*
> > > > >Currently, it is hard to build Ignite.C++. Different build process
> for
> > > > windows and linux, lack of building support on Mac OS X (quite
> popular
> > OS
> > > > among developers), absolutely not IDE support, except windows and
> only
> > > > Visual Studio is supported.
> > > > >
> > > > >*Suggestion*
> > > > >I’d suggest to migrate to CMake build system. It is very popular
> among
> > > > open source projects, and in Apache Software Foundation too. Notable
> > > user:
> > > > Apache Mesos, Apache Zookeeper (C client offers CMake as an
> alternative
> > > to
> > > > autoconf and only option on windows), Apache Kafka (librdkafka -
> C/C++
> > > > client), Apache Thrift. Popular column-oriented database ClickHouse
> > also
> > > > uses CMake.
> > > > >
> > > > >CMake is widely supported in many IDE’s on various platforms,
> notably
> > > > Visual Studio, CLion, Xcode, QtCreator, KDevelop.
> > > > >
> > > > >*Current status*
> > > > >
> > > > >Currently, the most of work is done (see [1]) and tested on Mac OS X
> > > > 10.15 (some C++ porting). All tests are run without any flaws,
> > including
> > > > odbc (unixodbc), ssl, thin and thick client, installation, IDE
> > > integration
> > > > (CLion). Next steps is to test linux and windows.
> > > > >
> > > > >But full migration isn’t possible without agreement and help of
> > > > community. Even if most of all you agree, migration requires
> additional
> > > > efforts in TC agents tuning and so on (event though test running
> fully
> > > > automated by CMake CTest).
> > > > >
> > > > >Lets discuss my proposition and idea.
> > > > >
> > > > >[1] -  https://github.com/apache/ignite/pull/7845
> > > > >
> > > > >
> > > > >
> > > >
> > > >
> > > >
> > > >
> > >
> >
> >
> > --
> > Best regards,
> > Alexey
> >
>

Re: [DISCUSSION] Ignite.C++ and CMake

[Nikolay Izhikov]

Hello, Igor.

> Nikolay, removing support for a certain build system is a breaking change.

No, it’s not.
Why do you think so?

Development environment and build system is a subject to change in any project.
We can drop or add support of any build system any time we want.

> 26 мая 2020 г., в 16:35, Ilya Kasnacheev  
> написал(а):
> 
> Hello!
> 
> I don't see why we can't get rid of autotools in a minor release, provided
> that cmake actually works. Removing native VS support may be a different
> thing.
> 
> Build system and precise set of dependencies is not a part of public API in
> my opinion.
> 
> Regards,
> -- 
> Ilya Kasnacheev
> 
> 
> вт, 26 мая 2020 г. в 16:02, Igor Sapego :
> 
>> Great!
>> 
>> Let's start with creating a TC suite for it.
>> 
>> The only concern I have is that it is one more build system
>> to support. Should we get rid of autotools in 3.0?
>> 
>> Best Regards,
>> Igor
>> 
>> 
>> On Tue, May 26, 2020 at 2:44 PM Alexey Kukushkin <
>> kukushkinale...@gmail.com>
>> wrote:
>> 
>>> +1. I recently completed a cross-IDE (MS Visual Studio & GCC/GDB) Ignite
>>> C++ project and CMake in Ignite C++ would save me a day of effort.
>>> 
>>> вт, 26 мая 2020 г. в 12:09, Pavel Tupitsyn :
>>> 
 +1
 
 On Tue, May 26, 2020 at 12:02 PM Zhenya Stanilovsky
  wrote:
 
> 
> Ivan huge +1 with your proposal.
> I had some problems with odbc tests building too, looks like cmake
>> will
> make it more easy.
>> Hello Igniters.
>> 
>> I’d like to discuss porting build process of Ignite.C++. I think
>> that
> there is time to change it.
>> 
>> *Motivation*
>> Currently, it is hard to build Ignite.C++. Different build process
>> for
> windows and linux, lack of building support on Mac OS X (quite
>> popular
>>> OS
> among developers), absolutely not IDE support, except windows and
>> only
> Visual Studio is supported.
>> 
>> *Suggestion*
>> I’d suggest to migrate to CMake build system. It is very popular
>> among
> open source projects, and in Apache Software Foundation too. Notable
 user:
> Apache Mesos, Apache Zookeeper (C client offers CMake as an
>> alternative
 to
> autoconf and only option on windows), Apache Kafka (librdkafka -
>> C/C++
> client), Apache Thrift. Popular column-oriented database ClickHouse
>>> also
> uses CMake.
>> 
>> CMake is widely supported in many IDE’s on various platforms,
>> notably
> Visual Studio, CLion, Xcode, QtCreator, KDevelop.
>> 
>> *Current status*
>> 
>> Currently, the most of work is done (see [1]) and tested on Mac OS X
> 10.15 (some C++ porting). All tests are run without any flaws,
>>> including
> odbc (unixodbc), ssl, thin and thick client, installation, IDE
 integration
> (CLion). Next steps is to test linux and windows.
>> 
>> But full migration isn’t possible without agreement and help of
> community. Even if most of all you agree, migration requires
>> additional
> efforts in TC agents tuning and so on (event though test running
>> fully
> automated by CMake CTest).
>> 
>> Lets discuss my proposition and idea.
>> 
>> [1] -  https://github.com/apache/ignite/pull/7845
>> 
>> 
>> 
> 
> 
> 
> 
 
>>> 
>>> 
>>> --
>>> Best regards,
>>> Alexey
>>> 
>> 

Re: [DISCUSSION] Ignite.C++ and CMake

[Ivan Daschinsky]

Guys, I will certainly thoroughly test my fix not only unices, but on
windows too.
And I will describe it very thoroughly.

When I was C++ developer (more than 10 years ago), I have not any trouble
at all with CMake and Visual Studio 2005.
Everything works and works good. Moreover, you can build with NMake,
msbuild and generate solutions for development.

I suppose, for CI purposes, using NMake is a way better, than use vs
solutions.

вт, 26 мая 2020 г. в 16:42, Nikolay Izhikov :

> Hello, Igor.
>
> > Nikolay, removing support for a certain build system is a breaking
> change.
>
> No, it’s not.
> Why do you think so?
>
> Development environment and build system is a subject to change in any
> project.
> We can drop or add support of any build system any time we want.
>
> > 26 мая 2020 г., в 16:35, Ilya Kasnacheev 
> написал(а):
> >
> > Hello!
> >
> > I don't see why we can't get rid of autotools in a minor release,
> provided
> > that cmake actually works. Removing native VS support may be a different
> > thing.
> >
> > Build system and precise set of dependencies is not a part of public API
> in
> > my opinion.
> >
> > Regards,
> > --
> > Ilya Kasnacheev
> >
> >
> > вт, 26 мая 2020 г. в 16:02, Igor Sapego :
> >
> >> Great!
> >>
> >> Let's start with creating a TC suite for it.
> >>
> >> The only concern I have is that it is one more build system
> >> to support. Should we get rid of autotools in 3.0?
> >>
> >> Best Regards,
> >> Igor
> >>
> >>
> >> On Tue, May 26, 2020 at 2:44 PM Alexey Kukushkin <
> >> kukushkinale...@gmail.com>
> >> wrote:
> >>
> >>> +1. I recently completed a cross-IDE (MS Visual Studio & GCC/GDB)
> Ignite
> >>> C++ project and CMake in Ignite C++ would save me a day of effort.
> >>>
> >>> вт, 26 мая 2020 г. в 12:09, Pavel Tupitsyn :
> >>>
>  +1
> 
>  On Tue, May 26, 2020 at 12:02 PM Zhenya Stanilovsky
>   wrote:
> 
> >
> > Ivan huge +1 with your proposal.
> > I had some problems with odbc tests building too, looks like cmake
> >> will
> > make it more easy.
> >> Hello Igniters.
> >>
> >> I’d like to discuss porting build process of Ignite.C++. I think
> >> that
> > there is time to change it.
> >>
> >> *Motivation*
> >> Currently, it is hard to build Ignite.C++. Different build process
> >> for
> > windows and linux, lack of building support on Mac OS X (quite
> >> popular
> >>> OS
> > among developers), absolutely not IDE support, except windows and
> >> only
> > Visual Studio is supported.
> >>
> >> *Suggestion*
> >> I’d suggest to migrate to CMake build system. It is very popular
> >> among
> > open source projects, and in Apache Software Foundation too. Notable
>  user:
> > Apache Mesos, Apache Zookeeper (C client offers CMake as an
> >> alternative
>  to
> > autoconf and only option on windows), Apache Kafka (librdkafka -
> >> C/C++
> > client), Apache Thrift. Popular column-oriented database ClickHouse
> >>> also
> > uses CMake.
> >>
> >> CMake is widely supported in many IDE’s on various platforms,
> >> notably
> > Visual Studio, CLion, Xcode, QtCreator, KDevelop.
> >>
> >> *Current status*
> >>
> >> Currently, the most of work is done (see [1]) and tested on Mac OS X
> > 10.15 (some C++ porting). All tests are run without any flaws,
> >>> including
> > odbc (unixodbc), ssl, thin and thick client, installation, IDE
>  integration
> > (CLion). Next steps is to test linux and windows.
> >>
> >> But full migration isn’t possible without agreement and help of
> > community. Even if most of all you agree, migration requires
> >> additional
> > efforts in TC agents tuning and so on (event though test running
> >> fully
> > automated by CMake CTest).
> >>
> >> Lets discuss my proposition and idea.
> >>
> >> [1] -  https://github.com/apache/ignite/pull/7845
> >>
> >>
> >>
> >
> >
> >
> >
> 
> >>>
> >>>
> >>> --
> >>> Best regards,
> >>> Alexey
> >>>
> >>
>
>

-- 
Sincerely yours, Ivan Daschinskiy

[jira] [Created] (IGNITE-13077) .NET transaction gets overwritten for multiple Ignite trying to start transactions in one thread

[Sergey Stronchinskiy (Jira)]

Sergey Stronchinskiy created IGNITE-13077:
-

 Summary: .NET transaction gets overwritten for multiple Ignite 
trying to start transactions in one thread
 Key: IGNITE-13077
 URL: https://issues.apache.org/jira/browse/IGNITE-13077
 Project: Ignite
  Issue Type: Bug
  Components: platforms
Reporter: Sergey Stronchinskiy


{code:c#}
var trs1 = Ignition.Start(new IgniteConfiguration
     {
         IgniteInstanceName = "First"
     })
     .GetTransactions();
 trs1.TxStart();
 var trs2 = Ignition.Start(new IgniteConfiguration
     {
         IgniteInstanceName = "Second"
     })
     .GetTransactions();
 Assert.IsNull(trs2.Tx); // fails
{code}




--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Re: [DISCUSSION] Ignite.C++ and CMake

[Ilya Kasnacheev]

Hello!

I will assist with checking on Linux if you would contribute a patch.
Please start with a ticket (or even an IEP maybe?)

Regards,
-- 
Ilya Kasnacheev


вт, 26 мая 2020 г. в 16:47, Ivan Daschinsky :

> Guys, I will certainly thoroughly test my fix not only unices, but on
> windows too.
> And I will describe it very thoroughly.
>
> When I was C++ developer (more than 10 years ago), I have not any trouble
> at all with CMake and Visual Studio 2005.
> Everything works and works good. Moreover, you can build with NMake,
> msbuild and generate solutions for development.
>
> I suppose, for CI purposes, using NMake is a way better, than use vs
> solutions.
>
> вт, 26 мая 2020 г. в 16:42, Nikolay Izhikov :
>
> > Hello, Igor.
> >
> > > Nikolay, removing support for a certain build system is a breaking
> > change.
> >
> > No, it’s not.
> > Why do you think so?
> >
> > Development environment and build system is a subject to change in any
> > project.
> > We can drop or add support of any build system any time we want.
> >
> > > 26 мая 2020 г., в 16:35, Ilya Kasnacheev 
> > написал(а):
> > >
> > > Hello!
> > >
> > > I don't see why we can't get rid of autotools in a minor release,
> > provided
> > > that cmake actually works. Removing native VS support may be a
> different
> > > thing.
> > >
> > > Build system and precise set of dependencies is not a part of public
> API
> > in
> > > my opinion.
> > >
> > > Regards,
> > > --
> > > Ilya Kasnacheev
> > >
> > >
> > > вт, 26 мая 2020 г. в 16:02, Igor Sapego :
> > >
> > >> Great!
> > >>
> > >> Let's start with creating a TC suite for it.
> > >>
> > >> The only concern I have is that it is one more build system
> > >> to support. Should we get rid of autotools in 3.0?
> > >>
> > >> Best Regards,
> > >> Igor
> > >>
> > >>
> > >> On Tue, May 26, 2020 at 2:44 PM Alexey Kukushkin <
> > >> kukushkinale...@gmail.com>
> > >> wrote:
> > >>
> > >>> +1. I recently completed a cross-IDE (MS Visual Studio & GCC/GDB)
> > Ignite
> > >>> C++ project and CMake in Ignite C++ would save me a day of effort.
> > >>>
> > >>> вт, 26 мая 2020 г. в 12:09, Pavel Tupitsyn :
> > >>>
> >  +1
> > 
> >  On Tue, May 26, 2020 at 12:02 PM Zhenya Stanilovsky
> >   wrote:
> > 
> > >
> > > Ivan huge +1 with your proposal.
> > > I had some problems with odbc tests building too, looks like cmake
> > >> will
> > > make it more easy.
> > >> Hello Igniters.
> > >>
> > >> I’d like to discuss porting build process of Ignite.C++. I think
> > >> that
> > > there is time to change it.
> > >>
> > >> *Motivation*
> > >> Currently, it is hard to build Ignite.C++. Different build process
> > >> for
> > > windows and linux, lack of building support on Mac OS X (quite
> > >> popular
> > >>> OS
> > > among developers), absolutely not IDE support, except windows and
> > >> only
> > > Visual Studio is supported.
> > >>
> > >> *Suggestion*
> > >> I’d suggest to migrate to CMake build system. It is very popular
> > >> among
> > > open source projects, and in Apache Software Foundation too.
> Notable
> >  user:
> > > Apache Mesos, Apache Zookeeper (C client offers CMake as an
> > >> alternative
> >  to
> > > autoconf and only option on windows), Apache Kafka (librdkafka -
> > >> C/C++
> > > client), Apache Thrift. Popular column-oriented database ClickHouse
> > >>> also
> > > uses CMake.
> > >>
> > >> CMake is widely supported in many IDE’s on various platforms,
> > >> notably
> > > Visual Studio, CLion, Xcode, QtCreator, KDevelop.
> > >>
> > >> *Current status*
> > >>
> > >> Currently, the most of work is done (see [1]) and tested on Mac
> OS X
> > > 10.15 (some C++ porting). All tests are run without any flaws,
> > >>> including
> > > odbc (unixodbc), ssl, thin and thick client, installation, IDE
> >  integration
> > > (CLion). Next steps is to test linux and windows.
> > >>
> > >> But full migration isn’t possible without agreement and help of
> > > community. Even if most of all you agree, migration requires
> > >> additional
> > > efforts in TC agents tuning and so on (event though test running
> > >> fully
> > > automated by CMake CTest).
> > >>
> > >> Lets discuss my proposition and idea.
> > >>
> > >> [1] -  https://github.com/apache/ignite/pull/7845
> > >>
> > >>
> > >>
> > >
> > >
> > >
> > >
> > 
> > >>>
> > >>>
> > >>> --
> > >>> Best regards,
> > >>> Alexey
> > >>>
> > >>
> >
> >
>
> --
> Sincerely yours, Ivan Daschinskiy
>

Re: Prevent insertion of cache entry if the binary field type and the type of the query entity do not match.

[Ilya Kasnacheev]

Hello!

I'm not aware about any mechanism like this one built in Apache Ignite.

I advise you to wrap Ignite's APIs into ones of your own and avoid using
raw Ignite API. This way you will make sure to do these checks on your own.

Regards,
-- 
Ilya Kasnacheev


пн, 25 мая 2020 г. в 15:22, Pavel Pereslegin :

> Hello Igniters.
>
> If type of binary field does not match query entity field type we
> still able to insert such entry into cache, but can't query it.
> In the following example we have query entity with the UUID field
> "name", but we insert String field "name" using binary object.
>
> IgniteCache cache = grid(0).createCache(
> new CacheConfiguration<>("testCache").setQueryEntities(
> Collections.singletonList(
> new QueryEntity()
> .setKeyFieldName("id")
> .setValueType("Person")
> .setFields(new LinkedHashMap<>(
> F.asMap("id", "java.lang.Integer",
> "name", "java.util.UUID"));
>
> BinaryObject obj = grid(0).binary().builder("Person")
> .setField("id", 1)
> .setField("name", UUID.randomUUID().toString())
> .build();
>
> cache.put(1, obj);
> assertEquals(obj, cache.withKeepBinary().get(1));
>
> String sql = "select id, name from Person where id=1";
>
> grid(0).context().query()
> .querySqlFields(new
> SqlFieldsQuery(sql).setSchema("testCache"), true)
> .getAll(); // java.lang.ClassCastException:
> java.lang.String cannot be cast to java.util.UUID
>
> The object was successfully inserted, but the "name"-field cannot be
> read using sql.
>
> Should it be better to prevent insertion of cache entry if the binary
> field type and the type of the query entity field do not match?
>

[DISCUSSION] IEP-47 Native persistence defragmentation

[Ivan Bessonov]

Hello Igniters,

I'd like to discuss this new IEP with you: [1]. The main idea is to have a
procedure that relocates
pages to the top of the file as compact as possible which allows us to
trim the file and increase its
fill-factor. It will be configured manually and executed after the restart,
but before node joins
topology (it means any load would be impossible during defragmentation). It
is described in detail
in the IEP itself, please read it. This topic was also previously discussed
here on dev-list in [2].

Here I would like to list a few moments that are not as clear and require
your opinion.

 - what are your overall thoughts on the IEP? Any concerns?

 - maintenance mode - how do we communicate with the node that's not in
topology? What are
   the options? As far as I know, we have no current tools like this.

 - checkpointer refactoring - these changes will involve intensive writing
of pages to the storage.
   If we're going to reuse the offheap page model to perform
defragmentation then the
   checkpointing mechanism will have to be adapted in some form.
   Are you fine with this? Or we need a separate discussion?

[1]
https://cwiki.apache.org/confluence/display/IGNITE/IEP-47%3A+Native+persistence+defragmentation
[2]
http://apache-ignite-developers.2346864.n4.nabble.com/How-to-free-up-space-on-disc-after-removing-entries-from-IgniteCache-with-enabled-PDS-td39839.html


-- 
Sincerely yours,
Ivan Bessonov

Re: [jira] [Created] (IGNITE-13065) Ignite Web Console 2.7.0 not working when scanning cache Error: Cannot read property 'length' of null

[Alex Panchenko]

I have the same issue. Can anybody help with this?



--
Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/

Re: [DISCUSSION] Ignite.C++ and CMake

[Ivan Daschinsky]

I appreciate any help, thank you, Ilya.

Currently I have a small PR without ticket (link in first post),but I
decided not to file a jira issue before discussion.
Now I see, that this feature are of great interest to community. So I file
a ticket, test myself on my home laptop (ubuntu 20.04)
 and add detailed instructions to DEVNOTES.txt in a few days.

I would be happy if my someone can follow the instruction and write
possible issues.

I will notify about status update in this thread in next few days.

Thank you all very much for support!


вт, 26 мая 2020 г. в 17:50, Ilya Kasnacheev :

> Hello!
>
> I will assist with checking on Linux if you would contribute a patch.
> Please start with a ticket (or even an IEP maybe?)
>
> Regards,
> --
> Ilya Kasnacheev
>
>
> вт, 26 мая 2020 г. в 16:47, Ivan Daschinsky :
>
> > Guys, I will certainly thoroughly test my fix not only unices, but on
> > windows too.
> > And I will describe it very thoroughly.
> >
> > When I was C++ developer (more than 10 years ago), I have not any trouble
> > at all with CMake and Visual Studio 2005.
> > Everything works and works good. Moreover, you can build with NMake,
> > msbuild and generate solutions for development.
> >
> > I suppose, for CI purposes, using NMake is a way better, than use vs
> > solutions.
> >
> > вт, 26 мая 2020 г. в 16:42, Nikolay Izhikov :
> >
> > > Hello, Igor.
> > >
> > > > Nikolay, removing support for a certain build system is a breaking
> > > change.
> > >
> > > No, it’s not.
> > > Why do you think so?
> > >
> > > Development environment and build system is a subject to change in any
> > > project.
> > > We can drop or add support of any build system any time we want.
> > >
> > > > 26 мая 2020 г., в 16:35, Ilya Kasnacheev 
> > > написал(а):
> > > >
> > > > Hello!
> > > >
> > > > I don't see why we can't get rid of autotools in a minor release,
> > > provided
> > > > that cmake actually works. Removing native VS support may be a
> > different
> > > > thing.
> > > >
> > > > Build system and precise set of dependencies is not a part of public
> > API
> > > in
> > > > my opinion.
> > > >
> > > > Regards,
> > > > --
> > > > Ilya Kasnacheev
> > > >
> > > >
> > > > вт, 26 мая 2020 г. в 16:02, Igor Sapego :
> > > >
> > > >> Great!
> > > >>
> > > >> Let's start with creating a TC suite for it.
> > > >>
> > > >> The only concern I have is that it is one more build system
> > > >> to support. Should we get rid of autotools in 3.0?
> > > >>
> > > >> Best Regards,
> > > >> Igor
> > > >>
> > > >>
> > > >> On Tue, May 26, 2020 at 2:44 PM Alexey Kukushkin <
> > > >> kukushkinale...@gmail.com>
> > > >> wrote:
> > > >>
> > > >>> +1. I recently completed a cross-IDE (MS Visual Studio & GCC/GDB)
> > > Ignite
> > > >>> C++ project and CMake in Ignite C++ would save me a day of effort.
> > > >>>
> > > >>> вт, 26 мая 2020 г. в 12:09, Pavel Tupitsyn :
> > > >>>
> > >  +1
> > > 
> > >  On Tue, May 26, 2020 at 12:02 PM Zhenya Stanilovsky
> > >   wrote:
> > > 
> > > >
> > > > Ivan huge +1 with your proposal.
> > > > I had some problems with odbc tests building too, looks like
> cmake
> > > >> will
> > > > make it more easy.
> > > >> Hello Igniters.
> > > >>
> > > >> I’d like to discuss porting build process of Ignite.C++. I think
> > > >> that
> > > > there is time to change it.
> > > >>
> > > >> *Motivation*
> > > >> Currently, it is hard to build Ignite.C++. Different build
> process
> > > >> for
> > > > windows and linux, lack of building support on Mac OS X (quite
> > > >> popular
> > > >>> OS
> > > > among developers), absolutely not IDE support, except windows and
> > > >> only
> > > > Visual Studio is supported.
> > > >>
> > > >> *Suggestion*
> > > >> I’d suggest to migrate to CMake build system. It is very popular
> > > >> among
> > > > open source projects, and in Apache Software Foundation too.
> > Notable
> > >  user:
> > > > Apache Mesos, Apache Zookeeper (C client offers CMake as an
> > > >> alternative
> > >  to
> > > > autoconf and only option on windows), Apache Kafka (librdkafka -
> > > >> C/C++
> > > > client), Apache Thrift. Popular column-oriented database
> ClickHouse
> > > >>> also
> > > > uses CMake.
> > > >>
> > > >> CMake is widely supported in many IDE’s on various platforms,
> > > >> notably
> > > > Visual Studio, CLion, Xcode, QtCreator, KDevelop.
> > > >>
> > > >> *Current status*
> > > >>
> > > >> Currently, the most of work is done (see [1]) and tested on Mac
> > OS X
> > > > 10.15 (some C++ porting). All tests are run without any flaws,
> > > >>> including
> > > > odbc (unixodbc), ssl, thin and thick client, installation, IDE
> > >  integration
> > > > (CLion). Next steps is to test linux and windows.
> > > >>
> > > >> But full migration isn’t possible without agreement and help of
> > > > community. Even if most of all you agree, mi

Re: IGNITE-12359 Migrate RocketMQ module to ignite-extensions

[Ilya Kasnacheev]

Hello!

We still have a mention of it in assembly/libs/README.txt - I think we
should remove it alongside with some others. I'm not sure where this file
goes to.

Otherwise, LGTM.

Regards,
-- 
Ilya Kasnacheev


пн, 25 мая 2020 г. в 20:13, Saikat Maitra :

> Hi Ilya,
>
> Thank you for reviewing the changes.
>
> I have updated the PR.
>
> Please review and share your feedback.
>
> Regards,
> Saikat
>
> On Sat, May 16, 2020 at 7:30 PM Saikat Maitra 
> wrote:
>
> > Hi,
> >
> > I have raised PRs for the following issue:
> >
> > IGNITE-12359 Migrate RocketMQ module to ignite-extensions
> >
> > https://issues.apache.org/jira/browse/IGNITE-12359
> >
> > PR
> > https://github.com/apache/ignite-extensions/pull/14
> > https://github.com/apache/ignite/pull/7809
> >
> > Please take a look and share feedback.
> >
> > Regards,
> > Saikat
> >
>

[jira] [Created] (IGNITE-13078) С++: Add CMake build support

[Ivan Daschinskiy (Jira)]

Ivan Daschinskiy created IGNITE-13078:
-

 Summary: С++: Add CMake build support
 Key: IGNITE-13078
 URL: https://issues.apache.org/jira/browse/IGNITE-13078
 Project: Ignite
  Issue Type: Improvement
  Components: platforms
Reporter: Ivan Daschinskiy
Assignee: Ivan Daschinskiy
 Fix For: 2.9


Currently, it is hard to build Ignite.C++. Different build process for windows 
and linux, lack of building support on Mac OS X (quite popular OS among 
developers), absolutely not IDE support, except windows and only Visual Studio 
is supported.

I’d suggest to migrate to CMake build system. It is very popular among open 
source projects, and in Apache Software Foundation too. Notable user: Apache 
Mesos, Apache Zookeeper (C client offers CMake as an alternative to autoconf 
and only option on windows), Apache Kafka (librdkafka - C/C++ client), Apache 
Thrift. Popular column-oriented database ClickHouse also uses CMake.

CMake is widely supported in many IDE’s on various platforms, notably Visual 
Studio, CLion, Xcode, QtCreator, KDevelop.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Re: [DISCUSSION] Ignite.C++ and CMake

[Stephen Darlington]

Not sure if it’ll help, but I made some changes to get it working on a Mac with 
the current built system. There may be some code worth taking.

https://github.com/apache/ignite/pull/4872 


Regards,
Stephen

> On 26 May 2020, at 16:02, Ivan Daschinsky  wrote:
> 
> I appreciate any help, thank you, Ilya.
> 
> Currently I have a small PR without ticket (link in first post),but I
> decided not to file a jira issue before discussion.
> Now I see, that this feature are of great interest to community. So I file
> a ticket, test myself on my home laptop (ubuntu 20.04)
> and add detailed instructions to DEVNOTES.txt in a few days.
> 
> I would be happy if my someone can follow the instruction and write
> possible issues.
> 
> I will notify about status update in this thread in next few days.
> 
> Thank you all very much for support!
> 
> 
> вт, 26 мая 2020 г. в 17:50, Ilya Kasnacheev :
> 
>> Hello!
>> 
>> I will assist with checking on Linux if you would contribute a patch.
>> Please start with a ticket (or even an IEP maybe?)
>> 
>> Regards,
>> --
>> Ilya Kasnacheev
>> 
>> 
>> вт, 26 мая 2020 г. в 16:47, Ivan Daschinsky :
>> 
>>> Guys, I will certainly thoroughly test my fix not only unices, but on
>>> windows too.
>>> And I will describe it very thoroughly.
>>> 
>>> When I was C++ developer (more than 10 years ago), I have not any trouble
>>> at all with CMake and Visual Studio 2005.
>>> Everything works and works good. Moreover, you can build with NMake,
>>> msbuild and generate solutions for development.
>>> 
>>> I suppose, for CI purposes, using NMake is a way better, than use vs
>>> solutions.
>>> 
>>> вт, 26 мая 2020 г. в 16:42, Nikolay Izhikov :
>>> 
 Hello, Igor.
 
> Nikolay, removing support for a certain build system is a breaking
 change.
 
 No, it’s not.
 Why do you think so?
 
 Development environment and build system is a subject to change in any
 project.
 We can drop or add support of any build system any time we want.
 
> 26 мая 2020 г., в 16:35, Ilya Kasnacheev 
 написал(а):
> 
> Hello!
> 
> I don't see why we can't get rid of autotools in a minor release,
 provided
> that cmake actually works. Removing native VS support may be a
>>> different
> thing.
> 
> Build system and precise set of dependencies is not a part of public
>>> API
 in
> my opinion.
> 
> Regards,
> --
> Ilya Kasnacheev
> 
> 
> вт, 26 мая 2020 г. в 16:02, Igor Sapego :
> 
>> Great!
>> 
>> Let's start with creating a TC suite for it.
>> 
>> The only concern I have is that it is one more build system
>> to support. Should we get rid of autotools in 3.0?
>> 
>> Best Regards,
>> Igor
>> 
>> 
>> On Tue, May 26, 2020 at 2:44 PM Alexey Kukushkin <
>> kukushkinale...@gmail.com>
>> wrote:
>> 
>>> +1. I recently completed a cross-IDE (MS Visual Studio & GCC/GDB)
 Ignite
>>> C++ project and CMake in Ignite C++ would save me a day of effort.
>>> 
>>> вт, 26 мая 2020 г. в 12:09, Pavel Tupitsyn :
>>> 
 +1
 
 On Tue, May 26, 2020 at 12:02 PM Zhenya Stanilovsky
  wrote:
 
> 
> Ivan huge +1 with your proposal.
> I had some problems with odbc tests building too, looks like
>> cmake
>> will
> make it more easy.
>> Hello Igniters.
>> 
>> I’d like to discuss porting build process of Ignite.C++. I think
>> that
> there is time to change it.
>> 
>> *Motivation*
>> Currently, it is hard to build Ignite.C++. Different build
>> process
>> for
> windows and linux, lack of building support on Mac OS X (quite
>> popular
>>> OS
> among developers), absolutely not IDE support, except windows and
>> only
> Visual Studio is supported.
>> 
>> *Suggestion*
>> I’d suggest to migrate to CMake build system. It is very popular
>> among
> open source projects, and in Apache Software Foundation too.
>>> Notable
 user:
> Apache Mesos, Apache Zookeeper (C client offers CMake as an
>> alternative
 to
> autoconf and only option on windows), Apache Kafka (librdkafka -
>> C/C++
> client), Apache Thrift. Popular column-oriented database
>> ClickHouse
>>> also
> uses CMake.
>> 
>> CMake is widely supported in many IDE’s on various platforms,
>> notably
> Visual Studio, CLion, Xcode, QtCreator, KDevelop.
>> 
>> *Current status*
>> 
>> Currently, the most of work is done (see [1]) and tested on Mac
>>> OS X
> 10.15 (some C++ porting). All tests are run without any flaws,
>>> including
> odbc (unixodbc), ssl, thin and thick client, installation, IDE
 integration

Re: Re[2]: Proposal: set default transaction timeout to 5 minutes

[Ivan Rakov]

Zhenya,

Can you please elaborate?
Why we need to change default TX timeout via JMX? It looks feasible and
perhaps may work as a hotfix for live deployments experiencing issues with
long transactions, but it's definitely a separate issue.

On Fri, May 22, 2020 at 6:20 PM Zhenya Stanilovsky
 wrote:

>
> Ivan, does global timeout change through jmx in scope of this ticket ? If
> so, can you add it ? Opposite we need additional ticket, i hope ? We
> still have no somehow store for jmx changed params, every one need to
> remember that cluster restart will reset this setting to default, in this
> case system param need to be appended.
>
>
>
> >https://issues.apache.org/jira/browse/IGNITE-13064 is raised with label
> >"newbie".
> >
> >On Tue, May 19, 2020 at 4:10 PM Ivan Rakov < ivan.glu...@gmail.com >
> wrote:
> >
> >> Support this idea in general but why 5 minutes and not less?
> >>
> >> This value looks to me greater than any value that can possibly affect
> >> existing deployments (existing long transactions may suddenly start to
> >> rollback), but less than reaction time of users that are only starting
> to
> >> get along with Ignite and suddenly experience TX deadlock.
> >>
> >> --
> >> Best Regards,
> >> Ivan Rakov
> >>
> >> On Tue, May 19, 2020 at 10:31 AM Anton Vinogradov < a...@apache.org >
> wrote:
> >>
> >>> +1
> >>>
> >>> On Mon, May 18, 2020 at 9:45 PM Sergey Antonov <
> antonovserge...@gmail.com
> >>> >
> >>> wrote:
> >>>
> >>> > +1
> >>> >
> >>> > пн, 18 мая 2020 г. в 21:26, Andrey Mashenkov <
> >>>  andrey.mashen...@gmail.com >:
> >>> >
> >>> > > +1
> >>> > >
> >>> > > On Mon, May 18, 2020 at 9:19 PM Ivan Rakov < ivan.glu...@gmail.com
> >
> >>> > wrote:
> >>> > >
> >>> > > > Hi Igniters,
> >>> > > >
> >>> > > > I have a very simple proposal. Let's set default TX timeout to 5
> >>> > minutes
> >>> > > > (right now it's 0 = no timeout).
> >>> > > > Pros:
> >>> > > > 1. Deadlock detection procedure is triggered on timeout. In case
> >>> user
> >>> > > will
> >>> > > > get into key-level deadlock, he'll be able to discover root cause
> >>> from
> >>> > > the
> >>> > > > logs (even though load will hang for a while) and skip step with
> >>> > googling
> >>> > > > and debugging.
> >>> > > > 2. Almost every system with transactions has timeout enabled by
> >>> > default.
> >>> > > >
> >>> > > > WDYT?
> >>> > > >
> >>> > > > --
> >>> > > > Best Regards,
> >>> > > > Ivan Rakov
> >>> > > >
> >>> > >
> >>> > >
> >>> > > --
> >>> > > Best regards,
> >>> > > Andrey V. Mashenkov
> >>> > >
> >>> >
> >>> >
> >>> > --
> >>> > BR, Sergey Antonov
> >>> >
> >>>
> >>
>
>
>
>

Re: [DISCUSSION] Ignite.C++ and CMake

[Ivan Daschinsky]

Stephen, looks great! I do mostly the same things in C++ code. Thank you!

вт, 26 мая 2020 г. в 18:33, Stephen Darlington <
stephen.darling...@gridgain.com>:

> Not sure if it’ll help, but I made some changes to get it working on a Mac
> with the current built system. There may be some code worth taking.
>
> https://github.com/apache/ignite/pull/4872 <
> https://github.com/apache/ignite/pull/4872>
>
> Regards,
> Stephen
>
> > On 26 May 2020, at 16:02, Ivan Daschinsky  wrote:
> >
> > I appreciate any help, thank you, Ilya.
> >
> > Currently I have a small PR without ticket (link in first post),but I
> > decided not to file a jira issue before discussion.
> > Now I see, that this feature are of great interest to community. So I
> file
> > a ticket, test myself on my home laptop (ubuntu 20.04)
> > and add detailed instructions to DEVNOTES.txt in a few days.
> >
> > I would be happy if my someone can follow the instruction and write
> > possible issues.
> >
> > I will notify about status update in this thread in next few days.
> >
> > Thank you all very much for support!
> >
> >
> > вт, 26 мая 2020 г. в 17:50, Ilya Kasnacheev :
> >
> >> Hello!
> >>
> >> I will assist with checking on Linux if you would contribute a patch.
> >> Please start with a ticket (or even an IEP maybe?)
> >>
> >> Regards,
> >> --
> >> Ilya Kasnacheev
> >>
> >>
> >> вт, 26 мая 2020 г. в 16:47, Ivan Daschinsky :
> >>
> >>> Guys, I will certainly thoroughly test my fix not only unices, but on
> >>> windows too.
> >>> And I will describe it very thoroughly.
> >>>
> >>> When I was C++ developer (more than 10 years ago), I have not any
> trouble
> >>> at all with CMake and Visual Studio 2005.
> >>> Everything works and works good. Moreover, you can build with NMake,
> >>> msbuild and generate solutions for development.
> >>>
> >>> I suppose, for CI purposes, using NMake is a way better, than use vs
> >>> solutions.
> >>>
> >>> вт, 26 мая 2020 г. в 16:42, Nikolay Izhikov :
> >>>
>  Hello, Igor.
> 
> > Nikolay, removing support for a certain build system is a breaking
>  change.
> 
>  No, it’s not.
>  Why do you think so?
> 
>  Development environment and build system is a subject to change in any
>  project.
>  We can drop or add support of any build system any time we want.
> 
> > 26 мая 2020 г., в 16:35, Ilya Kasnacheev 
>  написал(а):
> >
> > Hello!
> >
> > I don't see why we can't get rid of autotools in a minor release,
>  provided
> > that cmake actually works. Removing native VS support may be a
> >>> different
> > thing.
> >
> > Build system and precise set of dependencies is not a part of public
> >>> API
>  in
> > my opinion.
> >
> > Regards,
> > --
> > Ilya Kasnacheev
> >
> >
> > вт, 26 мая 2020 г. в 16:02, Igor Sapego :
> >
> >> Great!
> >>
> >> Let's start with creating a TC suite for it.
> >>
> >> The only concern I have is that it is one more build system
> >> to support. Should we get rid of autotools in 3.0?
> >>
> >> Best Regards,
> >> Igor
> >>
> >>
> >> On Tue, May 26, 2020 at 2:44 PM Alexey Kukushkin <
> >> kukushkinale...@gmail.com>
> >> wrote:
> >>
> >>> +1. I recently completed a cross-IDE (MS Visual Studio & GCC/GDB)
>  Ignite
> >>> C++ project and CMake in Ignite C++ would save me a day of effort.
> >>>
> >>> вт, 26 мая 2020 г. в 12:09, Pavel Tupitsyn :
> >>>
>  +1
> 
>  On Tue, May 26, 2020 at 12:02 PM Zhenya Stanilovsky
>   wrote:
> 
> >
> > Ivan huge +1 with your proposal.
> > I had some problems with odbc tests building too, looks like
> >> cmake
> >> will
> > make it more easy.
> >> Hello Igniters.
> >>
> >> I’d like to discuss porting build process of Ignite.C++. I think
> >> that
> > there is time to change it.
> >>
> >> *Motivation*
> >> Currently, it is hard to build Ignite.C++. Different build
> >> process
> >> for
> > windows and linux, lack of building support on Mac OS X (quite
> >> popular
> >>> OS
> > among developers), absolutely not IDE support, except windows and
> >> only
> > Visual Studio is supported.
> >>
> >> *Suggestion*
> >> I’d suggest to migrate to CMake build system. It is very popular
> >> among
> > open source projects, and in Apache Software Foundation too.
> >>> Notable
>  user:
> > Apache Mesos, Apache Zookeeper (C client offers CMake as an
> >> alternative
>  to
> > autoconf and only option on windows), Apache Kafka (librdkafka -
> >> C/C++
> > client), Apache Thrift. Popular column-oriented database
> >> ClickHouse
> >>> also
> > uses CMake.
> >>
> >> CMake is widely supported in many IDE’

Re[4]: Proposal: set default transaction timeout to 5 minutes

[Zhenya Stanilovsky]

Of course, i just thinking about huge persistent installation and guys who not 
carefully reads Release Notes )
In case of long tx timeouts by design, they can easily fix default timeout with 
just one jmx call. 
 
>Zhenya,
>
>Can you please elaborate?
>Why we need to change default TX timeout via JMX? It looks feasible and
>perhaps may work as a hotfix for live deployments experiencing issues with
>long transactions, but it's definitely a separate issue.
>
>On Fri, May 22, 2020 at 6:20 PM Zhenya Stanilovsky
>< arzamas...@mail.ru.invalid > wrote:
> 
>>
>> Ivan, does global timeout change through jmx in scope of this ticket ? If
>> so, can you add it ? Opposite we need additional ticket, i hope ? We
>> still have no somehow store for jmx changed params, every one need to
>> remember that cluster restart will reset this setting to default, in this
>> case system param need to be appended.
>>
>>
>>
>> > https://issues.apache.org/jira/browse/IGNITE-13064 is raised with label
>> >"newbie".
>> >
>> >On Tue, May 19, 2020 at 4:10 PM Ivan Rakov <  ivan.glu...@gmail.com >
>> wrote:
>> >
>> >> Support this idea in general but why 5 minutes and not less?
>> >>
>> >> This value looks to me greater than any value that can possibly affect
>> >> existing deployments (existing long transactions may suddenly start to
>> >> rollback), but less than reaction time of users that are only starting
>> to
>> >> get along with Ignite and suddenly experience TX deadlock.
>> >>
>> >> --
>> >> Best Regards,
>> >> Ivan Rakov
>> >>
>> >> On Tue, May 19, 2020 at 10:31 AM Anton Vinogradov <  a...@apache.org >
>> wrote:
>> >>
>> >>> +1
>> >>>
>> >>> On Mon, May 18, 2020 at 9:45 PM Sergey Antonov <
>>  antonovserge...@gmail.com
>> >>> >
>> >>> wrote:
>> >>>
>> >>> > +1
>> >>> >
>> >>> > пн, 18 мая 2020 г. в 21:26, Andrey Mashenkov <
>> >>>  andrey.mashen...@gmail.com >:
>> >>> >
>> >>> > > +1
>> >>> > >
>> >>> > > On Mon, May 18, 2020 at 9:19 PM Ivan Rakov <  ivan.glu...@gmail.com
>> >
>> >>> > wrote:
>> >>> > >
>> >>> > > > Hi Igniters,
>> >>> > > >
>> >>> > > > I have a very simple proposal. Let's set default TX timeout to 5
>> >>> > minutes
>> >>> > > > (right now it's 0 = no timeout).
>> >>> > > > Pros:
>> >>> > > > 1. Deadlock detection procedure is triggered on timeout. In case
>> >>> user
>> >>> > > will
>> >>> > > > get into key-level deadlock, he'll be able to discover root cause
>> >>> from
>> >>> > > the
>> >>> > > > logs (even though load will hang for a while) and skip step with
>> >>> > googling
>> >>> > > > and debugging.
>> >>> > > > 2. Almost every system with transactions has timeout enabled by
>> >>> > default.
>> >>> > > >
>> >>> > > > WDYT?
>> >>> > > >
>> >>> > > > --
>> >>> > > > Best Regards,
>> >>> > > > Ivan Rakov
>> >>> > > >
>> >>> > >
>> >>> > >
>> >>> > > --
>> >>> > > Best regards,
>> >>> > > Andrey V. Mashenkov
>> >>> > >
>> >>> >
>> >>> >
>> >>> > --
>> >>> > BR, Sergey Antonov
>> >>> >
>> >>>
>> >>
>>
>>
>>
>> 
 
 
 
 

Re: [DISCUSS] Extra test coverage for ACTIVE_READ_ONLY cluster state

[Maxim Muzafarov]

Sergey,

Sounds good!
Should we consider removing the deprecated methods `active()`,
`active(boolean active)` from tests also?

On Tue, 26 May 2020 at 12:18, Sergey Antonov  wrote:
>
> Hello, Igniters.
>
> I introduced cluster read-only mode [1] and a new API for cluster state
> change [2]. At the moment we don't have good test coverage for this
> feature. I'm going to fix it and write tests and check that operations
> are *denied
> *in read-only mode:
>
>- data structures usage
>- cache create/clear/destroy
>- DDL requests
>- cache updates by task's execution / deployed service
>
> And the following operations are *allowed *in read-only mode:
>
>- update of metastorage / distributed metastorage
>- updates to ignite-sys-cache
>- task's execution, but updates must be rejected
>- service deploy/undeploy, but updates must be rejected
>- data recovery on node join
>
> I'll work under these tests in ticket [3].
> Any objections?
>
> [1] https://issues.apache.org/jira/browse/IGNITE-11256
> [2] https://issues.apache.org/jira/browse/IGNITE-12225
> [3] https://issues.apache.org/jira/browse/IGNITE-13071
> --
> BR, Sergey Antonov

Re: [DISCUSS] Extra test coverage for ACTIVE_READ_ONLY cluster state

[Sergey Antonov]

Maxim, I'd prefer to do this with a separate ticket.

вт, 26 мая 2020 г. в 19:59, Maxim Muzafarov :

> Sergey,
>
> Sounds good!
> Should we consider removing the deprecated methods `active()`,
> `active(boolean active)` from tests also?
>
> On Tue, 26 May 2020 at 12:18, Sergey Antonov 
> wrote:
> >
> > Hello, Igniters.
> >
> > I introduced cluster read-only mode [1] and a new API for cluster state
> > change [2]. At the moment we don't have good test coverage for this
> > feature. I'm going to fix it and write tests and check that operations
> > are *denied
> > *in read-only mode:
> >
> >- data structures usage
> >- cache create/clear/destroy
> >- DDL requests
> >- cache updates by task's execution / deployed service
> >
> > And the following operations are *allowed *in read-only mode:
> >
> >- update of metastorage / distributed metastorage
> >- updates to ignite-sys-cache
> >- task's execution, but updates must be rejected
> >- service deploy/undeploy, but updates must be rejected
> >- data recovery on node join
> >
> > I'll work under these tests in ticket [3].
> > Any objections?
> >
> > [1] https://issues.apache.org/jira/browse/IGNITE-11256
> > [2] https://issues.apache.org/jira/browse/IGNITE-12225
> > [3] https://issues.apache.org/jira/browse/IGNITE-13071
> > --
> > BR, Sergey Antonov
>


-- 
BR, Sergey Antonov

[jira] [Created] (IGNITE-13079) Replace deprecated cluster activation methods in tests.

[Sergey Antonov (Jira)]

Sergey Antonov created IGNITE-13079:
---

 Summary: Replace deprecated cluster activation methods in tests.
 Key: IGNITE-13079
 URL: https://issues.apache.org/jira/browse/IGNITE-13079
 Project: Ignite
  Issue Type: Improvement
Reporter: Sergey Antonov


After we IGNITE-12225 we have a new API for a changing cluster state. We must 
replace deprecated methods ({{IgniteCluster#active()}}, 
{{IgniteCluster#active(boolean)}} and etc) to a new API



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Re: Re[2]: [VOTE] Release Apache Ignite 2.8.1 RC1

[Maxim Muzafarov]

+1 (binding)

Download binary, run some examples.
Checked zookeeper dependencies.

On Tue, 26 May 2020 at 12:02, Zhenya Stanilovsky
 wrote:
>
>
> Nikolay, performance boost is always good news for all customers at all and 
> for me too )
> Can you give more info (looks like in different mail thread) ? What kind of 
> benchmarks have been done, is it about persistence or both ?
> thanks !
> >+1 (binding).
> >
> >We made some internal benchmarking and found performance boost in comparison 
> >with 2.8.0
> >
> >
> >> 22 мая 2020 г., в 22:42, Denis Magda < dma...@apache.org > написал(а):
> >>
> >> + 1 (binding)
> >>
> >> Downloaded a binary version, started a cluster and ran some samples.
> >>
> >> -
> >> Denis
> >>
> >>
> >> On Fri, May 22, 2020 at 3:57 AM Nikolay Izhikov < nizhi...@apache.org > 
> >> wrote:
> >>
> >>> Dear Community,
> >>>
> >>> I have uploaded release candidate to
> >>>  https://dist.apache.org/repos/dist/dev/ignite/2.8.1-rc1/
> >>>  https://dist.apache.org/repos/dist/dev/ignite/packages_2.8.1-rc1/
> >>>
> >>> The following staging can be used for testing:
> >>>  https://repository.apache.org/content/repositories/orgapacheignite-1481
> >>>
> >>> This is the first maintenance release for 2.8.x with a number of fixes.
> >>>
> >>> Tag name is 2.8.1-rc1:
> >>>  
> >>> https://gitbox.apache.org/repos/asf?p=ignite.git;a=tag;h=refs/tags/2.8.1-rc1
> >>>
> >>> RELEASE NOTES:
> >>>  
> >>> https://gitbox.apache.org/repos/asf?p=ignite.git;a=blob;f=RELEASE_NOTES.txt;h=531b2ff7d72bdec9e0d9130efb27bf64f2b4f3fd;hb=9e214ee8ce7b87780d904111d359c74e90613e3f#l5
> >>>
> >>> Complete list of resolved issues:
> >>>  
> >>> https://issues.apache.org/jira/issues/?jql=project%20%3D%20IGNITE%20AND%20fixVersion%20%3D%202.8.1
> >>>
> >>> DEVNOTES
> >>>  
> >>> https://gitbox.apache.org/repos/asf?p=ignite.git;a=blob;f=DEVNOTES.txt;h=307189059ae70834ead4c127cc18295ce4d0735a;hb=9e214ee8ce7b87780d904111d359c74e90613e3
> >>>
> >>> The vote is formal, see voting guidelines
> >>>  https://www.apache.org/foundation/voting.html
> >>>
> >>> +1 - to accept Apache Ignite 2.8.1-rc1
> >>> 0 - don't care either way
> >>> -1 - DO NOT accept Apache Ignite Ignite 2.8.1-rc1 (explain why)
> >>>
> >>> See notes on how to verify release here
> >>>  https://www.apache.org/info/verification.html
> >>> and
> >>>  
> >>> https://cwiki.apache.org/confluence/display/IGNITE/Release+Process#ReleaseProcess-P5.VotingonReleaseandReleaseVerification
> >>>
> >>> Additional links:
> >>>
> >>> NuGet staging -
> >>>  
> >>> https://www.myget.org/feed/apache-ignite-staging/package/nuget/Apache.Ignite
> >>> Comparsion of 2.8.0 and 2.8.1 -
> >>>  
> >>> https://ci.ignite.apache.org/repository/download/ApacheIgniteReleaseJava8_IgniteRelease72CheckFileConsistency/5329458:id/results/result.log
> >>>
> >>>
> >>> This vote will be open for at least 3 days till Sun May 22, 23:59:59 UTC.
> >>>  
> >>> https://www.timeanddate.com/countdown/generic?iso=20200525T235959&p0=%3A&msg=Ignite+2.8.1-rc1+Vote&font=cursive
>
>
>
>

Re: [DISCUSS] Extra test coverage for ACTIVE_READ_ONLY cluster state

[Sergey Antonov]

Maxim, I've created a ticket [1] for this change.

[1] https://issues.apache.org/jira/browse/IGNITE-13079

вт, 26 мая 2020 г. в 20:09, Sergey Antonov :

> Maxim, I'd prefer to do this with a separate ticket.
>
> вт, 26 мая 2020 г. в 19:59, Maxim Muzafarov :
>
>> Sergey,
>>
>> Sounds good!
>> Should we consider removing the deprecated methods `active()`,
>> `active(boolean active)` from tests also?
>>
>> On Tue, 26 May 2020 at 12:18, Sergey Antonov 
>> wrote:
>> >
>> > Hello, Igniters.
>> >
>> > I introduced cluster read-only mode [1] and a new API for cluster state
>> > change [2]. At the moment we don't have good test coverage for this
>> > feature. I'm going to fix it and write tests and check that operations
>> > are *denied
>> > *in read-only mode:
>> >
>> >- data structures usage
>> >- cache create/clear/destroy
>> >- DDL requests
>> >- cache updates by task's execution / deployed service
>> >
>> > And the following operations are *allowed *in read-only mode:
>> >
>> >- update of metastorage / distributed metastorage
>> >- updates to ignite-sys-cache
>> >- task's execution, but updates must be rejected
>> >- service deploy/undeploy, but updates must be rejected
>> >- data recovery on node join
>> >
>> > I'll work under these tests in ticket [3].
>> > Any objections?
>> >
>> > [1] https://issues.apache.org/jira/browse/IGNITE-11256
>> > [2] https://issues.apache.org/jira/browse/IGNITE-12225
>> > [3] https://issues.apache.org/jira/browse/IGNITE-13071
>> > --
>> > BR, Sergey Antonov
>>
>
>
> --
> BR, Sergey Antonov
>


-- 
BR, Sergey Antonov

Re: [DISCUSSION] Framework for SQL performance regressions detection.

[Denis Magda]

Roman,

Probably you are right that the SQL benchmarks need to be incorporated into
the existing 'ignite-benchmarks' module. Please disregard my suggestion of
setting a dedicated repository.

-
Denis


On Sat, May 23, 2020 at 6:12 AM Roman Kondakov 
wrote:

> Hi Denis,
>
> I'm not sure we need a separate repository for it. What would be the
> benefit of using a separate repo?
>
> BTW I noticed that Ignite has `ignite-benchmarks` module. It contains
> JMH/JOL benchmarks for now. We can also put the SQL benchmark to this
> module. What do you think?
>
>
> --
> Kind Regards
> Roman Kondakov
>
>
> On 22.05.2020 22:36, Denis Magda wrote:
> > Hi Roman,
> >
> > +1 for sure. On a side note, should we create a separate ASF/Git
> repository
> > for the project? Not sure we need to put the suite in the main Ignite
> repo.
> >
> > -
> > Denis
> >
> >
> > On Fri, May 22, 2020 at 8:54 AM Roman Kondakov
> 
> > wrote:
> >
> >> Hi everybody!
> >>
> >> Currently Ignite doesn't have an ability to detect SQL performance
> >> regressions between different versions. We have a Yardstick benchmark
> >> module, but it has several drawbacks:
> >> - it doesn't compare different Ignite versions
> >> - it doesn't check the query result
> >> - it doesn't have an ability to execute randomized SQL queries (aka
> >> fuzzy testing)
> >>
> >> So, Yardstick is not very helpful for detecting SQL performance
> >> regressions.
> >>
> >> I think we need a brand-new framework for this task and I propose to
> >> implement it by adopting the ideas taken from the Apollo tool paper [1].
> >> The Apollo tool pipeline works like like this:
> >>
> >> 1. Apollo start two different versions of databases simultaneously.
> >> 2. Then Apollo populates them with the same dataset
> >> 3. Apollo generates random SQL queries using external library (i.e.
> >> SQLSmith [2])
> >> 4. Each query is executed in both database versions. Execution time is
> >> measured by the framework.
> >> 5. If the execution time difference for the same query exceeds some
> >> threshold (say, 2x slower), the query is logged.
> >> 6. Apollo then tries to simplify the problematic queries in order to
> >> obtain the minimal reproducer.
> >> 7. Apollo also has an ability to automatically perform git history
> >> binary search to find the bad commit
> >> 8. It also can localize a root cause of the regression by carrying out
> >> the statistical debugging.
> >>
> >> I think we don't have to implement all these Apollo steps. First 4 steps
> >> will be enough for our needs.
> >>
> >> My proposal is to create a new module called 'sql-testing'. We need a
> >> separate module because it should be suitable for both query engines:
> >> H2-based and upcoming Calcite-based. This module will contain a test
> >> suite which works in the following way:
> >> 1. It starts two Ignite clusters with different versions (current
> >> version and the previous release version).
> >> 2. Framework then runs randomly generated queries in both clusters and
> >> checks the execution time for each cluster. We need to port SQLSmith [2]
> >> library from C++ to java for this step. But initially we can start with
> >> some set of hardcoded queries and postpone the SQLSmith port. Randomized
> >> queries can be added later.
> >> 3. All problematic queries are then reported as performance issues. In
> >> this way we can manually examine the problems.
> >>
> >> This tool will bring a certain amount of robustness to our SQL layer as
> >> well as some portion of confidence in absence of SQL query regressions.
> >>
> >> What do you think?
> >>
> >>
> >> [1] http://www.vldb.org/pvldb/vol13/p57-jung.pdf
> >> [2] https://github.com/anse1/sqlsmith
> >>
> >>
> >> --
> >> Kind Regards
> >> Roman Kondakov
> >>
> >>
> >
>

[RESULT] [VOTE] Release Apache Ignite 2.8.1 RC1

[Nikolay Izhikov]

Hello!
 
Apache Ignite 2.8.1 release RC1 has been accepted.
 
6 "+1" votes received.
 
Here are the votes received:
 
- Alexey Zinoviev (binding)
- Nikolay Izhikov (binding)
- Pavel Tupitsyn (binding)
- Ilya Kasnacheev (binding)
- Denis Magda (binding)
- Maxim Muzafarov (binding)

 
Here is the link to vote thread -
http://apache-ignite-developers.2346864.n4.nabble.com/VOTE-Release-Apache-Ignite-2-8-1-RC1-tc47622.html
  

Thanks!