Re: Apache Ignite 3.0

2020-10-30 Thread Yakov Zhdanov
Alexey, Thanks for details! Common replication infra suggestion looks great! Agree with your points regarding per-page replication, but still have a feeling that this protocol can be made compact enough, e.g. by sending only deltas. As far as entry processors we can decide on what to send - if the

Re: Apache Ignite 3.0

2020-10-26 Thread Ivan Daschinsky
Hi! Alexey, > If we want to support etcd as a metastorage - let's do this as a concrete configuration option, a > first-class citizen of the system rather than an SPI implementation with a rigid interface. On one side this is quite reasonable. But on the other side, if someone wants to adopt, for e

Re: Apache Ignite 3.0

2020-10-23 Thread Alexey Goncharuk
Hello Ivan, Thanks for the feedback, see my comments inline: чт, 22 окт. 2020 г. в 17:59, Ivan Daschinsky : > Hi! > Alexey, your proposal looks great. Can I ask you some questions? > 1. Is nodes, that take part of metastorage replication group (raft > candidates and leader) are expected to also

Re: Apache Ignite 3.0

2020-10-22 Thread Ivan Daschinsky
Hi! Alexey, your proposal looks great. Can I ask you some questions? 1. Is nodes, that take part of metastorage replication group (raft candidates and leader) are expected to also bear cache data and participate in cache transactions? As for me, it seems quite dangerous to mix roles. For example

Re: Apache Ignite 3.0

2020-10-22 Thread Alexey Goncharuk
Hello Yakov, Glad to see you back! Hi! > I am back! > > Here are several ideas on top of my mind for Ignite 3.0 > 1. Client nodes should take the config from servers. Basically it should be > enough to provide some cluster identifier or any known IP address to start > a client. > This totally mak

Re: Apache Ignite 3.0

2020-10-17 Thread Yakov Zhdanov
Hey Valentin! Any design docs/wiki for 1, 4 and 5 so far? Yakov Zhdanov

Re: Apache Ignite 3.0

2020-10-13 Thread Valentin Kulichenko
Hi Yakov, Great to have you here! :) Thanks for the inputs. My comments are below. *Alexey*, could you please comment on the points 3 and 4? -Val On Sat, Oct 10, 2020 at 4:36 PM Yakov Zhdanov wrote: > Hi! > I am back! > > Here are several ideas on top of my mind for Ignite 3.0 > 1. Client nod

Re: Apache Ignite 3.0

2020-10-10 Thread Yakov Zhdanov
Hi! I am back! Here are several ideas on top of my mind for Ignite 3.0 1. Client nodes should take the config from servers. Basically it should be enough to provide some cluster identifier or any known IP address to start a client. 2. Thread per partition. Again. I strongly recommend taking a loo

Re: Apache Ignite 3.0

2020-09-17 Thread 18624049226
Hi Val, AFAIK, many developers use Ignite as a powerful development framework for distributed applications. The messaging function is easy to use. With it, many systems do not need to introduce a special message system such as Kafka. Although this function does not seem to coordinate with igni

Re: Apache Ignite 3.0

2020-09-17 Thread Raymond Wilson
Val, We use it in a mixture of server-server and server-client use cases. The common use case is variations of cache invalidation, particularly with respect to cached derivatives computed from information contained in the Ignite persistent store. I am aware of the Ignite cache events to detect ch

Re: Apache Ignite 3.0

2020-09-16 Thread Valentin Kulichenko
Raymond, Can you please describe the way you use messaging in a little bit more detail? Is it for communication between server nodes or more of a client-server? what are the typical patterns? -Val On Wed, Sep 16, 2020 at 4:05 PM Raymond Wilson wrote: > Hi Val, > > Possibly, but the core diffic

Re: Apache Ignite 3.0

2020-09-16 Thread Raymond Wilson
Hi Val, Possibly, but the core difficulty there is knowing when the fan-out to listeners has been completed. Before you know it you have re-implemented Kafka! :) I think we can leverage our use case knowledge to convert the messaging style of notifications to a compute style broadcast against a c

Re: Apache Ignite 3.0

2020-09-16 Thread Valentin Kulichenko
Hi Raymond, Do you think you could use services for your use case? https://apacheignite.readme.io/docs/service-grid -Val On Tue, Sep 15, 2020 at 4:32 PM Raymond Wilson wrote: > For what it is worth, we use messaging in Ignite to provide grid > notifications for various purposes. These might st

Re: Apache Ignite 3.0

2020-09-15 Thread Raymond Wilson
For what it is worth, we use messaging in Ignite to provide grid notifications for various purposes. These might stimulate compute operations in the message recipients, but they don't represent compute. If messaging is removed, how would Ignite provide the ability for a node to listen to events wi

Re: Apache Ignite 3.0

2020-09-15 Thread Kseniya Romanova
Just wanted to submit a link to the video about some offered changes. @valentin.kuliche...@gmail.com please add it to the wiki page. I created timestamps in the description, so one can browse the presentation fast or link the particular section: https://youtu.be/zPuLJgUfLaM Thanks a lot for the p

Re: Apache Ignite 3.0

2020-08-18 Thread Ilya Kasnacheev
Hello! Maybe we can implement messaging via compute? Share underlying code but have a different API for convenience (maybe in extras). Regards, -- Ilya Kasnacheev вт, 18 авг. 2020 г. в 11:59, Pavel Tupitsyn : > Val, > > > Is see the ".NET Modernization for Ignite 3.0" item in the roadmap, but

Re: Apache Ignite 3.0

2020-08-18 Thread Pavel Tupitsyn
Val, > Is see the ".NET Modernization for Ignite 3.0" item in the roadmap, but it > doesn't provide much detail. What are the actual changes in .NET that you > propose for 3.0? I've updated the roadmap with a brief description. We want to move away from an old, long unsupported .NET version, and

Re: Apache Ignite 3.0

2020-08-17 Thread Saikat Maitra
Hi Pavel, Awesome, thank you. Yes, I remember having .Net modernization as part of Apache Ignite 3.0 roadmap. Regards, Saikat On Sun, Aug 16, 2020 at 11:04 AM Pavel Tupitsyn wrote: > Saikat, yes, most definitely. > This is mentioned in the wishlist under ".NET: Target .NET Standard 2.0, > dis

Re: Apache Ignite 3.0

2020-08-17 Thread Valentin Kulichenko
Hi Pavel, Is see the ".NET Modernization for Ignite 3.0" item in the roadmap, but it doesn't provide much detail. What are the actual changes in .NET that you propose for 3.0? As for the messaging, so far I haven't seen use cases that would require this API. Sometimes users attempt to use it for

Re: Apache Ignite 3.0

2020-08-16 Thread Pavel Tupitsyn
Saikat, yes, most definitely. This is mentioned in the wishlist under ".NET: Target .NET Standard 2.0, discontinue .NET 4.0 support". I'm already working towards this goal by making more code and tests work properly under .NET Core, so when the time for breaking changes comes, it will be simpler.

Re: Apache Ignite 3.0

2020-08-15 Thread Saikat Maitra
Hi Val, Thank you for adding the Cleanup section and Removals list. Pavel, As part of Apache Ignite Roadmap we had mentioned we will add modernization of .NET. Are we still targeting it in Apache Ignite 3.0 release? https://cwiki.apache.org/confluence/display/IGNITE/Apache+Ignite+Roadmap Regard

Re: Apache Ignite 3.0

2020-08-14 Thread Carbone, Adam
If you want to make is simpler to have the components that you want, but have that be immutable at install time you could take an approach similar to the way spring does it with their initializer ( https://start.spring.io/ ) as an example... Basically the concept being something that produces a

Re: Apache Ignite 3.0

2020-08-14 Thread Nikolay Izhikov
I propose to add removals of the legacy JMX bean and Java metrics We should use new metrics framework and JMX exporter instead of it. > 14 авг. 2020 г., в 10:40, Pavel Tupitsyn написал(а): > > Val, > > The list of removals looks good to me, except Messaging. > I remember it had some implementat

Re: Apache Ignite 3.0

2020-08-14 Thread Pavel Tupitsyn
Val, The list of removals looks good to me, except Messaging. I remember it had some implementation issues, but the API itself seems to be rather useful - what are we going to offer instead? On Fri, Aug 14, 2020 at 3:16 AM Valentin Kulichenko < valentin.kuliche...@gmail.com> wrote: > Folks, > >

Re: Apache Ignite 3.0

2020-08-13 Thread Valentin Kulichenko
Folks, Since we all want 3.0 to be a "cleanup" release, I've added a section that lists potential API removals: https://cwiki.apache.org/confluence/display/IGNITE/Apache+Ignite+3.0#ApacheIgnite3.0-Removals Please take a look and let me know if there are any objections, or if there is anything tha

Re: Apache Ignite 3.0

2020-08-13 Thread Valentin Kulichenko
Hi Ilya, Can you please describe your vision of how it should work? Let's say, I want to set up a cluster of several standalone server nodes with a couple of optional modules enabled. What are my steps? -Val On Thu, Aug 13, 2020 at 6:03 AM Carbone, Adam wrote: > Good Morning from the EastCoas

Re: Apache Ignite 3.0

2020-08-13 Thread Carbone, Adam
Good Morning from the EastCoast I have to agree that the larger industry is tending towards immutability, and that you build once and test, then you promote/migrate that immutable binary object, be is a library or a docker image etc... however there are still patterns that allow you to determi

Re: Apache Ignite 3.0

2020-08-13 Thread Ilya Kasnacheev
Hello! On the contrary, I would suggest that apache2 way was outdated even at times when apache was all rage. Now the nginx approach is prevalent: on devops phase, assemble a custom bundle with all plugins included, store it somewhere, and ship it to production as a whole to remove any on-the-fly

Re: Apache Ignite 3.0

2020-08-13 Thread Petr Ivanov
RPM/DEB will be easy to refactor that way —internal dependency management will help with that. We can start with simple task — divide package into at least the following modules: — apache-ignite: slim package with only necessary runtime libs and scripts (just like slim binary) — apache-ignite

Re: Apache Ignite 3.0

2020-08-12 Thread Valentin Kulichenko
Hi Petr, Got it, that makes sense. I think we should rely on Maven for this. Basically, like the mvnw wrapper, but specific for Ignite purposes (enable/disable a module, start/stop a node, etc.). BTW, do you think RPM/DEB packages can use the same approach? Or they require that binaries are stored

Re: Apache Ignite 3.0

2020-08-11 Thread Petr Ivanov
Hi, Val. > On 12 Aug 2020, at 01:31, Valentin Kulichenko > wrote: > > Hi Petr, > > I agree -- we should better modularize the platform. The current way if very > error-prone, especially during upgrades -- any changes made within > IGNITE_HOME (configs, scripts, modules, etc.) must be merged

Re: Apache Ignite 3.0

2020-08-11 Thread Valentin Kulichenko
Denis, Agree. Added this requirement to the page along with everything else discussed so far. -Val On Tue, Aug 11, 2020 at 4:07 PM Denis Magda wrote: > Val, > > I remember some of us wanted to rework the transactional API. Is it still > relevant? > > Also, I would add the native image feature

Re: Apache Ignite 3.0

2020-08-11 Thread Denis Magda
Val, I remember some of us wanted to rework the transactional API. Is it still relevant? Also, I would add the native image feature of GraalVM to the scope of the release. Presently, even Java *thin* clients fail if an app is compiled and started as the native image. Turns out that the thin clien

Re: Apache Ignite 3.0

2020-08-11 Thread Valentin Kulichenko
Hi Saikat, Absolutely, we should revisit the existing APIs and decide which we want to keep as-is, which we want to remove, and which we want to rework and modernize. 3.0 is a rare opportunity to do such a cleanup :) -Val On Mon, Aug 10, 2020 at 6:54 PM Saikat Maitra wrote: > Hi Val, > > Thank

Re: Apache Ignite 3.0

2020-08-11 Thread Valentin Kulichenko
Hi Petr, I agree -- we should better modularize the platform. The current way if very error-prone, especially during upgrades -- any changes made within IGNITE_HOME (configs, scripts, modules, etc.) must be merged with a new version of the package. There is no standard way of doing this. However,

Re: Apache Ignite 3.0

2020-08-10 Thread Saikat Maitra
Hi Val, Thank you for sharing the page and your efforts in compiling the features list, I am thinking since it is a major version release then shall we include a section for deprecated features and add changes as mentioned in our Apache Ignite 3.0 Wishlist https://cwiki.apache.org/confluence/disp

Re: Apache Ignite 3.0

2020-08-09 Thread Petr Ivanov
Hi, Val! Thanks for your efforts on this endeavour! I would like to suggest deliveries changes in Apache Ignite 3.0: — modularised binary delivery — single minimal binary for starting Ignite and all other modules and parts of the project (benchmarks, examples, etc.) packed in their own binary

Re: Apache Ignite 3.0

2020-08-07 Thread Valentin Kulichenko
Igniters, I've created the page: https://cwiki.apache.org/confluence/display/IGNITE/Apache+Ignite+3.0 That's not everything I have in mind, but I believe there is already a lot to talk about :) Please take a look let me know if you have any concerns, objections, or questions. Once we reach the c

Re: Apache Ignite 3.0

2020-08-06 Thread Saikat Maitra
Hi Denis, Val Thank you for your reply and really appreciate it. It will be very cool to be able to connect and plan release together and learn more about Ignite in the process :) Regards Saikat On Thu, Aug 6, 2020 at 7:12 PM Valentin Kulichenko < valentin.kuliche...@gmail.com> wrote: > Hi Sa

Re: Apache Ignite 3.0

2020-08-06 Thread Valentin Kulichenko
Hi Saikat, That surely is a great idea. We will work together with Denis on setting this up in the nearest future. -Val On Thu, Aug 6, 2020 at 10:21 AM Denis Magda wrote: > Saikat, > > Fully support your idea on a virtual meetup! Once Val collects and outlines > the main changes with direction

Re: Apache Ignite 3.0

2020-08-06 Thread Denis Magda
Saikat, Fully support your idea on a virtual meetup! Once Val collects and outlines the main changes with directions on wiki, we’ll go ahead and schedule the meetup to talk things out in a bit more detail. We’ll use our new Virtual Ignite Meetup group for that inviting both Ignite contributors and

Re: Apache Ignite 3.0

2020-08-06 Thread Saikat Maitra
Hi Valentin Thank you for sharing and starting the thread. I am thinking if it will be a good idea to have a virtual meet setup to discuss on the release planning. It will help to learn more individual features to be added and also to understand about features that have been deprecated and schedu

Re: Apache Ignite 3.0

2020-08-06 Thread Ilya Kasnacheev
Hello! I hope to see Apache Ignite release 3.0 as API trimming release. Let us correct external and internal APIs for which we have better ideas now, as well as remove old and deprecated code. We may also introduce new configuration mechanisms and user-facing API (such as cache-less native SQL qu