Re: Discovery-based services deployment guarantees question
Vyacheslav, thanks for the explanation, makes sense to me. I was thinking though, should we make the behavior with the timeout default for all proxies? Just my opinion - I think for a user it would be hard to control which node deploys the service, especially if multiple nodes deploy it concurrently. Most likely users will end up always calling the second option of the proxy (with the timeout), so, perhaps, make it default? вс, 29 дек. 2019 г. в 21:05, Vyacheslav Daradur : > Alexey, > > I've prepared pr [1] to show our proxy invocation guarantees and to > avoid misunderstanding. > > Please, let me know if you think that we should improve our guaranties > in some cases. > > [1] https://github.com/apache/ignite/pull/7213 > > On Tue, Dec 24, 2019 at 7:27 PM Vyacheslav Daradur > wrote: > > > > > even the local deployment looks broken: if a compute job > > > is sent to a remote node after the service deployment > > > > This is a different case and covered by retries: > > * If you deploy a service from node A to node B, then take a proxy > > from node A (deployment initiator) it should NOT fail even if node B > > has not received yet a message that deployment finished successfully, > > because of proxy invocation retries. > > > > Look like It's better to describe all these cases on the wiki. > > > > > Should we schedule this ticket for the further work on Services IEP? > > > > If it is a frequent use-case we definitely should implement it. > > > > > > On Tue, Dec 24, 2019 at 6:55 PM Alexey Goncharuk > > wrote: > > > > > > Ok, got it. > > > > > > I agree that this is consistent with the old behavior, but this is the > kind > > > of errors we wanted to get rid of when we started the IEP. From the > > > user perspective, even the local deployment looks broken: if a compute > job > > > is sent to a remote node after the service deployment, the job > execution > > > may fail due to this error. > > > > > > Should we schedule this ticket for the further work on Services IEP? > > > > > > вт, 24 дек. 2019 г. в 18:49, Vyacheslav Daradur : > > > > > > > Not sure that "user fallback" is the right definition, it is not new > > > > behaviour in comparison with legacy implementation. > > > > > > > > Our synchronous deployment provides guaranties for a deployment > > > > initiator to be able to start work with service immediately after > > > > deployment finished successfully. > > > > For not the deployment initiator we can't provide such guarantees > now, > > > > because of unknown deployment result and possibly fail. > > > > > > > > In this case, a reasonable timeout might be an acceptable solution. > > > > > > > > We can improve guaranties in future releases, but there is an open > > > > question: > > > > - how long taking of proxy should wait? - deployment of "heavy" > > > > service may take a while > > > > > > > > On Tue, Dec 24, 2019 at 6:19 PM Alexey Goncharuk > > > > wrote: > > > > > > > > > > What should be the user fallback in this case? Retry infinitely? Is > > > > there a > > > > > way to wait for the proper deployment? > > > > > > > > > > вт, 24 дек. 2019 г. в 12:41, Vyacheslav Daradur < > daradu...@gmail.com>: > > > > > > > > > > > I’ll take a look at the end of the week. > > > > > > > > > > > > There is one more use-case: > > > > > > * if you initiate deployment from node A, but getting proxy on > node B > > > > > > (which isn’t deployment initiator) to call service on node A - > it may > > > > fail > > > > > > with "service not found", this is expected behaviour because we > didn't > > > > > > provide such guarantees. > > > > > > > > > > > > API of getting proxy with timeout should be used in this case: > > > > > > T serviceProxy(String name, Class svcItf, boolean > sticky, > > > > long > > > > > > timeout) > > > > > > > > > > > > > > > > > > вт, 24 дек. 2019 г. в 12:11, Alexey Goncharuk < > > > > alexey.goncha...@gmail.com > > > > > > >: > > > > > > > > > > > > > Well, this is exactly the case. The service is deployed from > node A, > > > > the > > > > > > > proxy is created on node B, and "service not found" exception > gets > > > > thrown > > > > > > > to a user anyway. Perhaps, the retry happens too fast? > > > > > > > > > > > > > > Created a ticket [1]. > > > > > > > > > > > > > > [1] https://issues.apache.org/jira/browse/IGNITE-12490 > > > > > > > > > > > > > > пн, 23 дек. 2019 г. в 22:08, Vyacheslav Daradur < > daradu...@gmail.com > > > > >: > > > > > > > > > > > > > > > Hi, Alexey > > > > > > > > > > > > > > > > Please attach a reproducer to the ticket. > > > > > > > > > > > > > > > > As far as I remember we have the following behaviour for the > > > > proxies: > > > > > > > > > > > > > > > > Let's assume you have deployed service from node A, then: > > > > > > > > * if you invoke service locally from node A - it is > guaranteed to > > > > > > > > service to be deployed and ready to work > > > > > > > > * if you take a proxy from node A to remote node B right > after > > > > deploy > > > > > > > >
[jira] [Created] (IGNITE-12511) Fix misleading javadoc.
Andrey Mashenkov created IGNITE-12511: - Summary: Fix misleading javadoc. Key: IGNITE-12511 URL: https://issues.apache.org/jira/browse/IGNITE-12511 Project: Ignite Issue Type: Task Reporter: Andrey Mashenkov Fix For: 2.9 -- This message was sent by Atlassian Jira (v8.3.4#803005)
Re: Discovery-based services deployment guarantees question
Alexey, I would not make it default in the current implementation. Waiting of proxies on non-deployment-initiator nodes should be improved - additional checks are required: 1) We should not wait if requested service has not been submitted to deploy (when there is no info about such service) 2) If service deployment failed - getting proxy should be failed or interrupted as well (do not wait for all available timeout) Let's schedule this improvement to next release, I'll try to find a time to implement it. What do you think? On Mon, Dec 30, 2019 at 12:05 PM Alexey Goncharuk wrote: > > Vyacheslav, thanks for the explanation, makes sense to me. > > I was thinking though, should we make the behavior with the timeout default > for all proxies? > > Just my opinion - I think for a user it would be hard to control which node > deploys the service, especially if multiple nodes deploy it concurrently. > Most likely users will end up always calling the second option of the proxy > (with the timeout), so, perhaps, make it default? > > вс, 29 дек. 2019 г. в 21:05, Vyacheslav Daradur : > > > Alexey, > > > > I've prepared pr [1] to show our proxy invocation guarantees and to > > avoid misunderstanding. > > > > Please, let me know if you think that we should improve our guaranties > > in some cases. > > > > [1] https://github.com/apache/ignite/pull/7213 > > > > On Tue, Dec 24, 2019 at 7:27 PM Vyacheslav Daradur > > wrote: > > > > > > > even the local deployment looks broken: if a compute job > > > > is sent to a remote node after the service deployment > > > > > > This is a different case and covered by retries: > > > * If you deploy a service from node A to node B, then take a proxy > > > from node A (deployment initiator) it should NOT fail even if node B > > > has not received yet a message that deployment finished successfully, > > > because of proxy invocation retries. > > > > > > Look like It's better to describe all these cases on the wiki. > > > > > > > Should we schedule this ticket for the further work on Services IEP? > > > > > > If it is a frequent use-case we definitely should implement it. > > > > > > > > > On Tue, Dec 24, 2019 at 6:55 PM Alexey Goncharuk > > > wrote: > > > > > > > > Ok, got it. > > > > > > > > I agree that this is consistent with the old behavior, but this is the > > kind > > > > of errors we wanted to get rid of when we started the IEP. From the > > > > user perspective, even the local deployment looks broken: if a compute > > job > > > > is sent to a remote node after the service deployment, the job > > execution > > > > may fail due to this error. > > > > > > > > Should we schedule this ticket for the further work on Services IEP? > > > > > > > > вт, 24 дек. 2019 г. в 18:49, Vyacheslav Daradur : > > > > > > > > > Not sure that "user fallback" is the right definition, it is not new > > > > > behaviour in comparison with legacy implementation. > > > > > > > > > > Our synchronous deployment provides guaranties for a deployment > > > > > initiator to be able to start work with service immediately after > > > > > deployment finished successfully. > > > > > For not the deployment initiator we can't provide such guarantees > > now, > > > > > because of unknown deployment result and possibly fail. > > > > > > > > > > In this case, a reasonable timeout might be an acceptable solution. > > > > > > > > > > We can improve guaranties in future releases, but there is an open > > > > > question: > > > > > - how long taking of proxy should wait? - deployment of "heavy" > > > > > service may take a while > > > > > > > > > > On Tue, Dec 24, 2019 at 6:19 PM Alexey Goncharuk > > > > > wrote: > > > > > > > > > > > > What should be the user fallback in this case? Retry infinitely? Is > > > > > there a > > > > > > way to wait for the proper deployment? > > > > > > > > > > > > вт, 24 дек. 2019 г. в 12:41, Vyacheslav Daradur < > > daradu...@gmail.com>: > > > > > > > > > > > > > I’ll take a look at the end of the week. > > > > > > > > > > > > > > There is one more use-case: > > > > > > > * if you initiate deployment from node A, but getting proxy on > > node B > > > > > > > (which isn’t deployment initiator) to call service on node A - > > it may > > > > > fail > > > > > > > with "service not found", this is expected behaviour because we > > didn't > > > > > > > provide such guarantees. > > > > > > > > > > > > > > API of getting proxy with timeout should be used in this case: > > > > > > > T serviceProxy(String name, Class svcItf, boolean > > sticky, > > > > > long > > > > > > > timeout) > > > > > > > > > > > > > > > > > > > > > вт, 24 дек. 2019 г. в 12:11, Alexey Goncharuk < > > > > > alexey.goncha...@gmail.com > > > > > > > >: > > > > > > > > > > > > > > > Well, this is exactly the case. The service is deployed from > > node A, > > > > > the > > > > > > > > proxy is created on node B, and "service not found" exception > > gets > > > > > thrown > > > > > > > > to a user anyway. Perhaps, the retry happens
Re: Discovery-based services deployment guarantees question
Agree, sounds like a plan, thanks for taking over! пн, 30 дек. 2019 г. в 13:00, Vyacheslav Daradur : > Alexey, > > I would not make it default in the current implementation. > > Waiting of proxies on non-deployment-initiator nodes should be > improved - additional checks are required: > 1) We should not wait if requested service has not been submitted to > deploy (when there is no info about such service) > 2) If service deployment failed - getting proxy should be failed or > interrupted as well (do not wait for all available timeout) > > Let's schedule this improvement to next release, I'll try to find a > time to implement it. > > What do you think? > > On Mon, Dec 30, 2019 at 12:05 PM Alexey Goncharuk > wrote: > > > > Vyacheslav, thanks for the explanation, makes sense to me. > > > > I was thinking though, should we make the behavior with the timeout > default > > for all proxies? > > > > Just my opinion - I think for a user it would be hard to control which > node > > deploys the service, especially if multiple nodes deploy it concurrently. > > Most likely users will end up always calling the second option of the > proxy > > (with the timeout), so, perhaps, make it default? > > > > вс, 29 дек. 2019 г. в 21:05, Vyacheslav Daradur : > > > > > Alexey, > > > > > > I've prepared pr [1] to show our proxy invocation guarantees and to > > > avoid misunderstanding. > > > > > > Please, let me know if you think that we should improve our guaranties > > > in some cases. > > > > > > [1] https://github.com/apache/ignite/pull/7213 > > > > > > On Tue, Dec 24, 2019 at 7:27 PM Vyacheslav Daradur < > daradu...@gmail.com> > > > wrote: > > > > > > > > > even the local deployment looks broken: if a compute job > > > > > is sent to a remote node after the service deployment > > > > > > > > This is a different case and covered by retries: > > > > * If you deploy a service from node A to node B, then take a proxy > > > > from node A (deployment initiator) it should NOT fail even if node B > > > > has not received yet a message that deployment finished successfully, > > > > because of proxy invocation retries. > > > > > > > > Look like It's better to describe all these cases on the wiki. > > > > > > > > > Should we schedule this ticket for the further work on Services > IEP? > > > > > > > > If it is a frequent use-case we definitely should implement it. > > > > > > > > > > > > On Tue, Dec 24, 2019 at 6:55 PM Alexey Goncharuk > > > > wrote: > > > > > > > > > > Ok, got it. > > > > > > > > > > I agree that this is consistent with the old behavior, but this is > the > > > kind > > > > > of errors we wanted to get rid of when we started the IEP. From the > > > > > user perspective, even the local deployment looks broken: if a > compute > > > job > > > > > is sent to a remote node after the service deployment, the job > > > execution > > > > > may fail due to this error. > > > > > > > > > > Should we schedule this ticket for the further work on Services > IEP? > > > > > > > > > > вт, 24 дек. 2019 г. в 18:49, Vyacheslav Daradur < > daradu...@gmail.com>: > > > > > > > > > > > Not sure that "user fallback" is the right definition, it is not > new > > > > > > behaviour in comparison with legacy implementation. > > > > > > > > > > > > Our synchronous deployment provides guaranties for a deployment > > > > > > initiator to be able to start work with service immediately after > > > > > > deployment finished successfully. > > > > > > For not the deployment initiator we can't provide such guarantees > > > now, > > > > > > because of unknown deployment result and possibly fail. > > > > > > > > > > > > In this case, a reasonable timeout might be an acceptable > solution. > > > > > > > > > > > > We can improve guaranties in future releases, but there is an > open > > > > > > question: > > > > > > - how long taking of proxy should wait? - deployment of "heavy" > > > > > > service may take a while > > > > > > > > > > > > On Tue, Dec 24, 2019 at 6:19 PM Alexey Goncharuk > > > > > > wrote: > > > > > > > > > > > > > > What should be the user fallback in this case? Retry > infinitely? Is > > > > > > there a > > > > > > > way to wait for the proper deployment? > > > > > > > > > > > > > > вт, 24 дек. 2019 г. в 12:41, Vyacheslav Daradur < > > > daradu...@gmail.com>: > > > > > > > > > > > > > > > I’ll take a look at the end of the week. > > > > > > > > > > > > > > > > There is one more use-case: > > > > > > > > * if you initiate deployment from node A, but getting proxy > on > > > node B > > > > > > > > (which isn’t deployment initiator) to call service on node A > - > > > it may > > > > > > fail > > > > > > > > with "service not found", this is expected behaviour because > we > > > didn't > > > > > > > > provide such guarantees. > > > > > > > > > > > > > > > > API of getting proxy with timeout should be used in this > case: > > > > > > > > T serviceProxy(String name, Class svcItf, boolean > > > sticky, > > > > > > long > > > > > > > > timeout) > > >
Re: Apache Ignite 2.8 RELEASE [Time, Scope, Manager]
Hello! I have checked that on master, and fallback to old behavior does not seem to work for pre-existing clusters: I am starting a cluster with two nodes with pre-existing PDS, and when I start client, which would do setBaselineTopology, I get: Caused by: org.apache.ignite.internal.processors.cluster.BaselineAdjustForbiddenException: Baseline auto-adjust is enabled, please turn-off it before try to adjust baseline manually at org.apache.ignite.internal.processors.cluster.GridClusterStateProcessor.changeGlobalState0 (GridClusterStateProcessor.java:996) at org.apache.ignite.internal.processors.cluster.GridClusterStateProcessor.changeGlobalState (GridClusterStateProcessor.java:916) at org.apache.ignite.internal.processors.cluster.GridClusterStateProcessor.changeGlobalState (GridClusterStateProcessor.java:895) at org.apache.ignite.internal.processors.cluster.GridClusterStateProcessor.changeGlobalState (GridClusterStateProcessor.java:855) at org.apache.ignite.internal.cluster.IgniteClusterImpl.setBaselineTopology (IgniteClusterImpl.java:387) You can try that on a reproducer referenced in https://issues.apache.org/jira/browse/IGNITE-12504 - start cluster in 2.6.0, run persistence-data-nodes/persistence, then upgrade (don't forget H2) and start cluster again. Regards, -- Ilya Kasnacheev пт, 27 дек. 2019 г. в 18:24, Anton Kalashnikov : > Hello. > > Ivan is right that "baseline auto-adjust" is disabled by default if you > start your node on > existing PDS. But "baseline auto-adjust" is enabled by default for > in-memory cluster due to in-memory nodes also have bound to baseline since > 2.8 version. > > Also, I want to note that after this ticket( > https://issues.apache.org/jira/projects/IGNITE/issues/IGNITE-12227). > "baseline auto-adjust" would be disabled by default for any persistent > cluster(empty and existed one) because current logic is a little confused > and can lead to some problems which described in the ticket. > > -- > Best regards, > Anton Kalashnikov > > > 27.12.2019, 17:58, "Ivan Bessonov" : > > Hello, > > > > "baseline auto-adjust" is disabled by default if you start your node on > > existing PDS. > > It's enabled on new clusters only. > > > > Existing installations should not be affected by the update. Is that ok? > > > > пт, 27 дек. 2019 г. в 14:46, Maxim Muzafarov : > > > >> Ilya, > >> > >> +1 from my side. > >> > >> On Fri, 27 Dec 2019 at 14:36, Ilya Kasnacheev < > ilya.kasnach...@gmail.com> > >> wrote: > >> > > >> > Hello! > >> > > >> > I have also noticed that we have baseline auto-adjust enabled by > default > >> in > >> > 2.8 builds, and it breaks existing code in runtime: > >> > https://issues.apache.org/jira/browse/IGNITE-12504 > >> > > >> > I propose to turn auto-adjust off by default in 2.8 release. What do > you > >> > think? > >> > > >> > Regards, > >> > -- > >> > Ilya Kasnacheev > >> > > >> > > >> > пт, 27 дек. 2019 г. в 12:40, Sergei Ryzhov : > >> > > >> > > Hello! > >> > > Task IGNITE-12470 is ready. > >> > > https://issues.apache.org/jira/browse/IGNITE-12470 > >> > > Please check this API. > >> > > > >> > > > >> > > Regards, > >> > > Ryzhov Sergei. > >> > > > >> > > чт, 26 дек. 2019 г. в 18:50, Maxim Muzafarov : > >> > > > >> > > > Ilya, > >> > > > > >> > > > > >> > > > I agree with you that there is no risk and spring-data-2.2 can be > >> > > > safely cherry-picked to the ignite-2.8 branch. I'm OK with it. > Will > >> > > > you do such merge or I should do it by myself? > >> > > > > >> > > > > >> > > > As for the second part of your email, you are proposing to bump > up a > >> > > > minor dependencies version (no API changes) for the whole > components > >> > > > mentioned in the parent/pom.xml file, right? From a point of the > >> > > > release view, it seems not a good thing since a scope test of the > >> > > > release becomes too wider. I don't think we will simplify thus > the > >> > > > year-long release test scope, so as for me, this sounds not good > but > >> > > > I'd like to hear thoughts of other community members on this > point. > >> > > > > >> > > > As an alternative, for instance, we can bump minor versions only > for > >> > > > those components which have security vulnerabilities. To find > such > >> > > > dependencies, I've run some local test with a maven > >> > > > dependency-check-maven [1] an open-source dependency check tool. > Here > >> > > > is a brief report (only a few modules tested): > >> > > > > >> > > > spring-core-4.3.18.RELEASE.jar : CVE-2018-15756 [2] > >> > > > h2-1.4.197.jar : CVE-2018-10054, CVE-2018-14335 (discussed also > [3]) > >> > > > ignite-shmem-1.0.0.jar : CVE-2017-14614 > >> > > > > >> > > > > >> > > > [1] https://jeremylong.github.io/DependencyCheck/index.html > >> > > > [2] https://nvd.nist.gov/vuln/detail/CVE-2018-15756 > >> > > > [3] https://issues.apache.org/jira/browse/IGNITE-10801 > >> > > > > >> > > > > >> > > > > >> > > > On Thu, 26
[jira] [Created] (IGNITE-12512) SQL: Compact logs for cancelled queries.
Andrey Mashenkov created IGNITE-12512: - Summary: SQL: Compact logs for cancelled queries. Key: IGNITE-12512 URL: https://issues.apache.org/jira/browse/IGNITE-12512 Project: Ignite Issue Type: Task Reporter: Andrey Mashenkov Assignee: Andrey Mashenkov Cancelled queries log stacktraces in error level. Let's log short info message on cancellation and log an additional stacktrace in debug level. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (IGNITE-12513) SQL: Improve QueryEntity field conflict message.
Andrey Mashenkov created IGNITE-12513:
-
Summary: SQL: Improve QueryEntity field conflict message.
Key: IGNITE-12513
URL: https://issues.apache.org/jira/browse/IGNITE-12513
Project: Ignite
Issue Type: Task
Reporter: Andrey Mashenkov
We have 2 objects that have same field (balance in this case)
While we start node there is lack of information about what is wrong - the only
thing we display is a conflicting field name
We need to add at least class names to this error (cache name?)
{code:java}
Caused by: org.springframework.beans.factory.BeanCreationException: Error
creating bean with name
'org.apache.ignite.configuration.CacheConfiguration#7e9a5fbe' defined in URL
[file:/storage/ssd/dsherstobitov/tiden/compatibility-190716-181722/test_rolling_upgrade/caches.xml]:
Error setting property values; nested exception is
org.springframework.beans.PropertyBatchUpdateException; nested
PropertyAccessExceptions (1) are:
PropertyAccessException 1: org.springframework.beans.MethodInvocationException:
Property 'indexedTypes' threw exception; nested exception is
javax.cache.CacheException: Property with name 'balance' already exists.
at
org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.applyPropertyValues(AbstractAutowireCapableBeanFactory.java:1568)
at
org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.populateBean(AbstractAutowireCapableBeanFactory.java:1276)
at
org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:553)
at
org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:483)
at
org.springframework.beans.factory.support.BeanDefinitionValueResolver.resolveInnerBean(BeanDefinitionValueResolver.java:299)
... 24 more
Caused by: org.springframework.beans.PropertyBatchUpdateException; nested
PropertyAccessExceptions (1) are:
PropertyAccessException 1: org.springframework.beans.MethodInvocationException:
Property 'indexedTypes' threw exception; nested exception is
javax.cache.CacheException: Property with name 'balance' already exists.
at
org.springframework.beans.AbstractPropertyAccessor.setPropertyValues(AbstractPropertyAccessor.java:121)
at
org.springframework.beans.AbstractPropertyAccessor.setPropertyValues(AbstractPropertyAccessor.java:75)
at
org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.applyPropertyValues(AbstractAutowireCapableBeanFactory.java:1564)
... 28 more
{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (IGNITE-12514) WAL don't flush several last records in LOG-ONLY/FSYNC mode if flush ptr=null
Ivan Bessonov created IGNITE-12514: -- Summary: WAL don't flush several last records in LOG-ONLY/FSYNC mode if flush ptr=null Key: IGNITE-12514 URL: https://issues.apache.org/jira/browse/IGNITE-12514 Project: Ignite Issue Type: Bug Reporter: Ivan Bessonov Assignee: Ivan Bessonov Fix For: 2.9 In the current implementation, last flush pointer dependent to thread-local. If some thread adds new records and another thread calls wal.flush(null), this flush may not be flushed records witch was added in thread one, because in case null flush pointer, thread flushed until the last record which was added in current thread. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (IGNITE-12515) GridMultinodeRedeploySharedModeSelfTest.testSharedMode fails sometimes
Ivan Bessonov created IGNITE-12515: -- Summary: GridMultinodeRedeploySharedModeSelfTest.testSharedMode fails sometimes Key: IGNITE-12515 URL: https://issues.apache.org/jira/browse/IGNITE-12515 Project: Ignite Issue Type: Bug Reporter: Ivan Bessonov Assignee: Ivan Bessonov Fix For: 2.9 Exception in org.apache.ignite.internal.managers.deployment.GridDeploymentPerVersionStore#searchDeploymentCache -- This message was sent by Atlassian Jira (v8.3.4#803005)
Re: SQL dialects supported by Calcite
Hello! I think we may eventually need at least two dialects. One is "H2 compatible" for users migrating from older versions of AI. We may need to add a few tweaks there to make most of existing SQL code work. Other is "modern" in which we will remove these tweaks to get the best, vanilla Ignite+Calcite SQL experience. I don't think we need "Oracle dialect", this will probably create confusion as well as testing burden. Regards, -- Ilya Kasnacheev сб, 28 дек. 2019 г. в 11:02, Seliverstov Igor : > I guess you are thinking about transparent migration from Oracle for > example. > > From user perspective it's really cool, but we will be forced to maintain > all these dialects and fully test them. Also I heard about several > inconsistencies between how it should and how it actually works. All these > issues become ours if we declare several dialects support. > > So, I think multi dialects support will bring more troubles than benefits. > > Regards, > Igor > > сб, 28 дек. 2019 г., 0:42 Denis Magda : > > > Igor, > > > > When you are saying that we should not allow the dialect changing, are > you > > referring to changes in runtime when a node is already up-and-running? > > Generally, it will be more than enough if a dialect can be set statically > > and globally -- the user selects the dialect for the entire cluster via a > > configuration parameter and starts the nodes after that. > > > > - > > Denis > > > > > > On Fri, Dec 27, 2019 at 7:05 AM Seliverstov Igor > > wrote: > > > > > Denis, > > > > > > > Is it true that Calcite can parse MySQL, Oracle > > > > > > Yes, it can. > > > > > > > Do I need to select a dialect globally or can it be set on a > per-query > > > > level? > > > > > > Technically a parser, a validator, a planner, other components are > > created > > > and configured for each query call (because they are stateful). So you > > may > > > configure it per query, or hardcode a desired dialect, or get it from > > > configuration - it’s up to you, but we expect parser configuration to > be > > > (more or less) static, it is a part of initial framework > configuration. I > > > think we should not allow such parameters (as a dialect) changing. > > > > > > Regards, > > > Igor > > > > > > > 27 дек. 2019 г., в 07:47, Denis Magda > написал(а): > > > > > > > > Igor S., Roman and others who involved in Calcite prototyping, > > > > > > > > Is it true that Calcite can parse MySQL, Oracle, ANSI-99 and other > > > dialects > > > > as suggested by this page? > > > > > > > > > > https://calcite.apache.org/apidocs/org/apache/calcite/sql/validate/SqlConformanceEnum.html > > > > > > > > Do I need to select a dialect globally or can it be set on a > per-query > > > > level? > > > > > > > > - > > > > Denis > > > > > > > > >
Re: SQL dialects supported by Calcite
My humble opinion here is that we should strive for one and only dialect. I understand concerns about supporting a backward compatible dialect. But I personally do not know whether it is practical with Calcite. Regarding multiples dialects my forecast is that supporting multiple dialects close by meaning to "drop-in replacement for nothing". пн, 30 дек. 2019 г. в 16:40, Ilya Kasnacheev : > > Hello! > > I think we may eventually need at least two dialects. > > One is "H2 compatible" for users migrating from older versions of AI. We > may need to add a few tweaks there to make most of existing SQL code work. > Other is "modern" in which we will remove these tweaks to get the best, > vanilla Ignite+Calcite SQL experience. > > I don't think we need "Oracle dialect", this will probably create confusion > as well as testing burden. > > Regards, > -- > Ilya Kasnacheev > > > сб, 28 дек. 2019 г. в 11:02, Seliverstov Igor : > > > I guess you are thinking about transparent migration from Oracle for > > example. > > > > From user perspective it's really cool, but we will be forced to maintain > > all these dialects and fully test them. Also I heard about several > > inconsistencies between how it should and how it actually works. All these > > issues become ours if we declare several dialects support. > > > > So, I think multi dialects support will bring more troubles than benefits. > > > > Regards, > > Igor > > > > сб, 28 дек. 2019 г., 0:42 Denis Magda : > > > > > Igor, > > > > > > When you are saying that we should not allow the dialect changing, are > > you > > > referring to changes in runtime when a node is already up-and-running? > > > Generally, it will be more than enough if a dialect can be set statically > > > and globally -- the user selects the dialect for the entire cluster via a > > > configuration parameter and starts the nodes after that. > > > > > > - > > > Denis > > > > > > > > > On Fri, Dec 27, 2019 at 7:05 AM Seliverstov Igor > > > wrote: > > > > > > > Denis, > > > > > > > > > Is it true that Calcite can parse MySQL, Oracle > > > > > > > > Yes, it can. > > > > > > > > > Do I need to select a dialect globally or can it be set on a > > per-query > > > > > level? > > > > > > > > Technically a parser, a validator, a planner, other components are > > > created > > > > and configured for each query call (because they are stateful). So you > > > may > > > > configure it per query, or hardcode a desired dialect, or get it from > > > > configuration - it’s up to you, but we expect parser configuration to > > be > > > > (more or less) static, it is a part of initial framework > > configuration. I > > > > think we should not allow such parameters (as a dialect) changing. > > > > > > > > Regards, > > > > Igor > > > > > > > > > 27 дек. 2019 г., в 07:47, Denis Magda > > написал(а): > > > > > > > > > > Igor S., Roman and others who involved in Calcite prototyping, > > > > > > > > > > Is it true that Calcite can parse MySQL, Oracle, ANSI-99 and other > > > > dialects > > > > > as suggested by this page? > > > > > > > > > > > > > > https://calcite.apache.org/apidocs/org/apache/calcite/sql/validate/SqlConformanceEnum.html > > > > > > > > > > Do I need to select a dialect globally or can it be set on a > > per-query > > > > > level? > > > > > > > > > > - > > > > > Denis > > > > > > > > > > > > > -- Best regards, Ivan Pavlukhin
[jira] [Created] (IGNITE-12516) Dump active transaction from near node does not work if transaction not follow first
Denis Chudov created IGNITE-12516:
-
Summary: Dump active transaction from near node does not work if
transaction not follow first
Key: IGNITE-12516
URL: https://issues.apache.org/jira/browse/IGNITE-12516
Project: Ignite
Issue Type: Bug
Reporter: Denis Chudov
Assignee: Denis Chudov
For this code:
{code:java}
for (IgniteInternalTx tx : tm.activeTransactions()) {
if (curTime - tx.startTime() > timeout) {
found = true;
if (warnings.canAddMessage()) {
warnings.add(longRunningTransactionWarning(tx, curTime));
if (ltrDumpLimiter.allowAction(tx))
dumpLongRunningTransaction(tx);
}
else
warnings.incTotal();
}
}
{code}
you can see, if transaction is not ACTIVE dumping closure will skipped.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
Re: PME speedup #2, TX recovery delay elimination.
Anton, > Since this fix made to speedup pme-free switch which prohibits the merges... Sounds as a bit of knowledge that I missed. Good to understand it now. > The "magic numbers" are always the "magic numbers" :) > We must get rid of them to see problems covered by them. Let's try =) чт, 26 дек. 2019 г. в 15:09, Anton Vinogradov : > > Ivan, > > The "magic numbers" are always the "magic numbers" :) > We must get rid of them to see problems covered by them. > > >> Was there any > >> performance measurements for such multiple left nodes cases? > Since this fix made to speedup pme-free switch which prohibits the merges, > the answer is "no". > > BTW, the fix was merged to master. > > On Thu, Dec 26, 2019 at 2:21 PM Ivan Pavlukhin wrote: > > > Anton, > > > > Thank you for your efforts! And sorry for a late reply. > > > > I am a little bit familiar with tx recovery. I personally like the > > idea of removing such "magic" logic from the code. I think a proper > > way is either justify and sustain (tests, documentation) some behavior > > or get rid of it. > > > > Regarding a delay before tx recovery. My understanding was that it > > might be useful when multiple (client) nodes leaves almost at the same > > time (perhaps due to some network connectivity issues). With a delay > > recovering multiple failed nodes will be grouped into one recovery > > round (+PME). Correct me if my understanding is wrong. Was there any > > performance measurements for such multiple left nodes cases? > > > > вт, 24 дек. 2019 г. в 16:22, Anton Vinogradov : > > > > > > Rechecked TC two more times. > > > Going to merge to master in case no objections here. > > > > > > On Mon, Dec 23, 2019 at 1:44 PM Anton Vinogradov wrote: > > > > > > > Igniters, > > > > > > > > One more PME optimization ready to be reviewed. > > > > I found a strange tx recovery delay caused by > > IGNITE_TX_SALVAGE_TIMEOUT. > > > > I've checked the code and tests and found no reason to delay recovery. > > > > > > > > So, the issue [1] is ready to be reviewed. > > > > > > > > Improvement checked with benchmark [2] and fix, obviously, 100 ms > > faster > > > > :) > > > > > > > > [1] https://issues.apache.org/jira/browse/IGNITE-12272 > > > > [2] > > > > > > https://github.com/anton-vinogradov/ignite/commit/f8c27253b0ecfe7381418f505aafe942efe5a0a8 > > > > > > > > > > > > -- > > Best regards, > > Ivan Pavlukhin > > -- Best regards, Ivan Pavlukhin
Ignite wallpapers for desktop & mobile - light your fire
Hi Igniters! Here [1] you can find wallpapers and add some blaze to your desktop or mobile! As they can be used in any month, probably I shall upload them to ignite wiki or to the project website? Please suggest what will be appropriate. Me and GG designer Nacho wish you happy holidays and a productive New Year! [1] https://www.dropbox.com/sh/0dphcy7swss4s18/AACz5_URq9ogx21pEseDuDHqa/2020 Kseniya GridGain DevRel
Re: [DISCUSS] Pub/Sub Streamer Implementation
Hi Ivan, Yes, we will need to remove flink module from main ignite repo. I can do it as part of this issue. IGNITE-12356 Migrate Flink module to ignite-extensions Regards, Saikat On Mon, Dec 30, 2019 at 12:54 AM Ivan Pavlukhin wrote: > Folks, > > Do not we need to remove flink integration sources from main ignite > repository [1]? Is this work in progress? > > [1] https://github.com/apache/ignite > > вс, 29 дек. 2019 г. в 22:19, Emmanouil Gkatziouras : > > > > Hi Saikat! > > > > Thank you for your time and assistance. > > > > Kind regards, > > Emmanouil > > > > *Emmanouil Gkatziouras* > > https://egkatzioura.com/ | > https://www.linkedin.com/in/gkatziourasemmanouil/ > > https://github.com/gkatzioura > > > > > > On Sun, 29 Dec 2019 at 18:35, Saikat Maitra > wrote: > > > > > Hi Emmanouil, > > > > > > Thank you for your contribution. I have merged your changes in master. > > > > > > Regards, > > > Saikat > > > > > > > > > On Sun, Dec 29, 2019 at 12:20 PM Saikat Maitra < > saikat.mai...@gmail.com> > > > wrote: > > > > > > > Thank you, Emmanouil > > > > > > > > I am reviewing the changes. > > > > > > > > Regards, > > > > Saikat > > > > > > > > On Sat, Dec 28, 2019 at 3:48 PM Emmanouil Gkatziouras < > > > > gkatzio...@gmail.com> wrote: > > > > > > > >> Hi Saikat! > > > >> > > > >> The dependencies have been updated, and the merge conflicts > resolved. > > > >> Kind regards > > > >> > > > >> On Sat, 28 Dec 2019, 23:27 Saikat Maitra, > > > >> wrote: > > > >> > > > >> > Hi Emmanouil, > > > >> > > > > >> > Thank you for making the changes. I have merged my PR changes in > > > master. > > > >> > Can you please take the latest from master branch and resolve the > > > >> conflicts > > > >> > in your PR. > > > >> > > > > >> > I have also reviewed the changes in your PR and requested minor > > > changes > > > >> > specific to dependencies version and license information. > > > >> > > > > >> > Can you please review and share your feedback? > > > >> > > > > >> > Regards, > > > >> > Saikat > > > >> > > > > >> > > > > >> > > > > >> > > > > >> > On Sat, Dec 28, 2019 at 4:38 AM Emmanouil Gkatziouras < > > > >> > gkatzio...@gmail.com> > > > >> > wrote: > > > >> > > > > >> > > Hi Saikat! > > > >> > > > > > >> > > Thank you on this one! I tried it and it worked as expected [1] > > > >> > > > > > >> > > Kind regards > > > >> > > Emmanouil > > > >> > > > > > >> > > [1] > > > >> > > > > > >> > > > > > >> > > > > >> > > > > https://ci.ignite.apache.org/viewLog.html?buildId=4888357&tab=buildLog&_focus=2046#_state=2046 > > > >> > > > > > >> > > *Emmanouil Gkatziouras* > > > >> > > https://egkatzioura.com/ | > > > >> > > https://www.linkedin.com/in/gkatziourasemmanouil/ > > > >> > > https://github.com/gkatzioura > > > >> > > > > > >> > > > > > >> > > On Thu, 26 Dec 2019 at 19:50, Saikat Maitra < > > > saikat.mai...@gmail.com> > > > >> > > wrote: > > > >> > > > > > >> > > > Hi Emmanouil, > > > >> > > > > > > >> > > > I have been able to resolve the build failures for missing > > > modules. > > > >> > > > > > > >> > > > > > > >> > > > > > > >> > > > > > >> > > > > >> > > > > https://ci.ignite.apache.org/viewLog.html?buildId=4883394&buildTypeId=IgniteExtensions_Build > > > >> > > > > > > >> > > > Will you please take the latest changes from my PR and run the > > > build > > > >> > for > > > >> > > > your PR. > > > >> > > > > > > >> > > > https://github.com/apache/ignite-extensions/pull/1/files > > > >> > > > > > > >> > > > The new build configurations should let your module build also > > > pass. > > > >> > > > > > > >> > > > The issue that was causing failure was that I had put > ignite.zip > > > as > > > >> > > > artifact dependency and it was downloading the artifacts > including > > > >> pom > > > >> > > file > > > >> > > > inside the build checkout folder of ignite-extensions and as a > > > >> result > > > >> > the > > > >> > > > pom.xml of ignite-extensions was getting overwritten by > ignite's > > > >> > pom.xml. > > > >> > > > This was resulting in missing flink-ext module error as it > was not > > > >> even > > > >> > > > present in the ignite main pom.xml. > > > >> > > > > > > >> > > > To resolve the issue I just had to download and extract > artifact > > > of > > > >> > > > ignite.zip in a separate directory. > > > >> > > > > > > >> > > > Regards, > > > >> > > > Saikat > > > >> > > > > > > >> > > > > > > >> > > > On Sat, Nov 30, 2019 at 5:53 PM Emmanouil Gkatziouras < > > > >> > > > gkatzio...@gmail.com> > > > >> > > > wrote: > > > >> > > > > > > >> > > > > Hi Saikat! > > > >> > > > > > > > >> > > > > Thank you for your assistance on that! > > > >> > > > > > > > >> > > > > Kind regards > > > >> > > > > > > > >> > > > > *Emmanouil Gkatziouras* > > > >> > > > > https://egkatzioura.com/ | > > > >> > > > > https://www.linkedin.com/in/gkatziourasemmanouil/ > > > >> > > > > https://github.com/gkatzioura > > > >> > > > > > > > >> > > > > > > > >> > > > > On Sat, 30 Nov 2019 at 23:51, Saikat Maitra < > > > >> saikat.mai...@gmai
IGNITE-12356 Migrate Flink module to ignite-extensions
Hi, I have raised a PR for removal of flink module from ignite main repo. I have already moved flink module in ignite-extensions repo ( https://github.com/apache/ignite-extensions). PR https://github.com/apache/ignite/pull/7222 jira : IGNITE-12356 Migrate Flink module to ignite-extensions https://issues.apache.org/jira/browse/IGNITE-12356 Please review and share feedback. Regards, Saikat
Re: Ignite wallpapers for desktop & mobile - light your fire
Kseniya, Nacho Thank you for sharing the wallpapers. They are very cool. IMO, we can add it in project website. Also, I have a query regarding Apache lgnite logo that can be used for external presentation. Can we add this also in Apache Ignite project website? http://apache.org/logos/#ignite Regards, Saikat On Mon, Dec 30, 2019 at 9:24 AM Kseniya Romanova wrote: > Hi Igniters! > > Here [1] you can find wallpapers and add some blaze to your desktop or > mobile! > > As they can be used in any month, probably I shall upload them to ignite > wiki or to the project website? Please suggest what will be appropriate. > > Me and GG designer Nacho wish you happy holidays and a productive New Year! > > [1] > https://www.dropbox.com/sh/0dphcy7swss4s18/AACz5_URq9ogx21pEseDuDHqa/2020 > > > Kseniya > GridGain DevRel >
[jira] [Created] (IGNITE-12517) Upgrade flink module version to 1.9.1 in ignite-extensions
Saikat Maitra created IGNITE-12517: -- Summary: Upgrade flink module version to 1.9.1 in ignite-extensions Key: IGNITE-12517 URL: https://issues.apache.org/jira/browse/IGNITE-12517 Project: Ignite Issue Type: Improvement Components: streaming Reporter: Saikat Maitra Fix For: 2.8 Upgrade flink module version to 1.9.1 in ignite-extensions [https://github.com/apache/ignite-extensions] -- This message was sent by Atlassian Jira (v8.3.4#803005)
IGNITE-12517 Upgrade flink module version to 1.9.1 in ignite-extensions
Hi I have raised a tiny PR to upgrade flink version to latest stable release. Jira : https://issues.apache.org/jira/browse/IGNITE-12517 PR https://github.com/apache/ignite-extensions/pull/3 Please review and provide approval. Regards, Saikat
