I don’t understand how it possible. Are you talking about plugin that uses some kind of internal API?
> 3 дек. 2021 г., в 18:50, Вячеслав Коптилин <slava.kopti...@gmail.com> > написал(а): > > Hello Nikolay, > > If I am not mistaken, the method you mentioned, allows to create a "user" > cache that is available through public api for the user. This does not > cover the case when the plugin developer wants to hide such cache and > protect it form the end user (at least, via public api). > > Thanks, > S. > > пт, 3 дек. 2021 г., 14:15 Nikolay Izhikov <nizhi...@apache.org>: > >> Vyacheslav, Val, can you please clarify - What is the issue if third-party >> plugins will create «ignite-sys-cache» from the code? >> >> Like just replacing `Ignite#cache` with the `Ignite#getOrCreateCache`. >> >>> 2 дек. 2021 г., в 16:13, Вячеслав Коптилин <slava.kopti...@gmail.com> >> написал(а): >>> >>> Hello Maxim, >>> >>>> I don't understand why you are using *backwards compatibility* for >>> completely internal things. >>>> Why you are thinking in terms of users usage when are talking about >>> ignite-sys-cache but not thinking when refactoring >>> First of all, we are talking about all plugin developers. It does not >>> matter it is open-source or proprietary plugins. >>> Honestly, I don't have a list of all possible plugins that have already >>> been developed and available under the ASF license, for instance. >>> Do you have such a list? Can you be sure that this change will not affect >>> anyone? >>> >>>> I don't understand why you are using *backwards compatibility* for >>> completely internal things. >>>> Why you are thinking in terms of users usage when are talking about >>> ignite-sys-cache but not thinking when refactoring >>> The system cache was the crucial thing in the architecture of Apache >> Ignite >>> 1.x and 2.x (at least 2.1 - 2.11?) >>> >>>> All the internals have been reworked and nobody even mentioned that it >>> may affect some of the plugin developers. >>> Yep, perhaps, some internal parts of Apache Ignite were reworked in order >>> to avoid using the system cache. >>> However, it is still a part of Ignite and it works and can be used in >>> plugins. >>> Honestly, the mentioned alternative, I mean the distributed metastorage, >> is >>> the INTERNAL thing as well. >>> It means that plugin developer depends on INTERNAL entities. (it does not >>> matter system-cache/metastorage) >>> IMHO, such questions should be CAREFULLY discussed with no rush. >>> >>>> I do not propose to rush with the ignite-sys-cache removal. I'd like to >>> collect all the technical stuff that we depend on, fix all of them and >>> after everything will be ready do a final removal. >>> Good! We are on the same page! >>> >>>> 1. Add deprecation for the 2.12 release. I hope it is still possible. >>> If I am not mistaken, the code freeze was October 29. I think it is too >>> late to introduce this change. We can add deprecation for the 2.13 >> release. >>> >>>> Apache Ignite core still have some dependencies on ignite-sys-cache that >>> should fix for 2.13 >>> Ok, I agree. We need to try to find out all edge cases and add new >>> abilities to the metastorage in order to cover all known >>> issues/requirements. >>> >>>> Remove the system cache in 2.14. >>> It depends on our progress with improving the distributed metastorage. In >>> general, I hope it will be possible. >>> >>> Thanks, >>> S. >>> >>> чт, 2 дек. 2021 г. в 13:36, Maxim Muzafarov <mmu...@apache.org>: >>> >>>> Pavel, >>>> >>>> >>>> I don't understand why you are using *backwards compatibility* for >>>> completely internal things. Why you are thinking in terms of users >>>> usage when are talking about ignite-sys-cache but not thinking when >>>> refactoring, for instance, all the checkpoint classes? Take a look at >>>> the [1] issue. All the internals have been reworked and nobody even >>>> mentioned that it may affect some of the plugin developers. This is >>>> really strange, so I can't agree with you. >>>> >>>> >>>> I do not propose to rush with the ignite-sys-cache removal. I'd like >>>> to collect all the technical stuff that we depend on, fix all of them >>>> and after everything will be ready do a final removal. Slava already >>>> mentioned some crucial cases, so I hope you and Val also help with the >>>> rest of them. Let's be more precise in terms *what we need to fix*. >>>> >>>> >>>> I've made some investigations related to the ignite-sys-cache and here >>>> is my proposal: >>>> >>>> 1. Add deprecation for the 2.12 release. I hope it is still possible. >>>> >>>> 2. Apache Ignite core still have some dependencies on ignite-sys-cache >>>> that should fix for 2.13: >>>> >>>> 2.1. CLUSTER_NAME property: when it's not set the `deploymentId` of >>>> system cache is used. Let's move it to metastorage. >>>> 2.2. When the Security is enabled each compute task name (hash to be >>>> more precise) is written to the ignite-sys-cache [2]. From my point of >>>> view, we can remove it in 2.13. Can anyone check? >>>> 2.3. Slava mentioned some issues with the distributed metastorage that >>>> might be fixed. I think this is doable. >>>> >>>> 3. Remove the system cache in 2.14. >>>> >>>> >>>> >>>> [1] https://issues.apache.org/jira/browse/IGNITE-13143 >>>> [2] >>>> >> https://github.com/apache/ignite/blob/master/modules/core/src/main/java/org/apache/ignite/internal/processors/task/GridTaskProcessor.java#L201 >>>> >>>> On Thu, 2 Dec 2021 at 12:56, Pavel Tupitsyn <ptupit...@apache.org> >> wrote: >>>>> >>>>> Maxim, >>>>> >>>>>> I don't think that we should wait for 3-rd party plugins to be updated >>>>>> this is an awful practice when the open-source product releases depend >>>>>> on some of the proprietary plugins >>>>> >>>>> This makes no sense, sorry. >>>>> >>>>> It is not that we depend on 3rd party plugins. >>>>> It is that *our users depend on us to preserve backwards >> compatibility*. >>>>> No one likes when their app breaks suddenly because of a library >> update. >>>>> >>>>> We know about one use case for sys-cache in GridGain, but there may be >>>>> more, no one knows. >>>>> Every breaking change should be carried out carefully and gradually. >>>>> >>>>> >>>>> On Thu, Dec 2, 2021 at 12:34 PM Maxim Muzafarov <mmu...@apache.org> >>>> wrote: >>>>> >>>>>> Slava, >>>>>> >>>>>> Thank you for the comments. >>>>>> >>>>>>> Maxim, the community must provide a reasonable time interval in >>>> order to >>>>>> notify all contributors and wait for updating all 3-rd party plugins. >>>>>> >>>>>> This is not actually true. We must notify about changes as earlier as >>>>>> possible and not only users but all the developers also. However, I >>>>>> don't think that we should wait for 3-rd party plugins to be updated >>>>>> this is an awful practice when the open-source product releases depend >>>>>> on some of the proprietary plugins. There is no dependency between >>>>>> Apache Ignite releases and proprietary plugin releases. If someone >>>>>> desires to upgrade to a new version of Apache Ignite it can update its >>>>>> plugins any time he likes. >>>>>> >>>>>>> We just need to provide a window that is enough to cover all related >>>>>> issues. >>>>>> >>>>>> Can you share all the issues you know, please? >>>>>> >>>>>>> distributed meta storage does not provide the ability to atomically >>>>>> change several properties at the same time (there are no transactions >>>> on >>>>>> this API). >>>>>> >>>>>> Can you share an example of what kind of properties we intend to >>>>>> change at the same? Will it be enough to change them through a single >>>>>> thread (e.g. the discovery thread or the exchange thread)? I agree >>>>>> here that distributed metastorage should provide such an ability, >>>>>> however, I can't find any real examples for Apache Ignite internal >>>>>> needs. Please, share the details and let's fix it. >>>>>> >>>>>> On Wed, 1 Dec 2021 at 23:19, Valentin Kulichenko >>>>>> <valentin.kuliche...@gmail.com> wrote: >>>>>>> >>>>>>> Agree with Slava. Two months is not enough time, especially >>>> considering >>>>>>> that the system cache is not functionally equivalent to the >>>> metastorage. >>>>>> I >>>>>>> suggest we do the following: >>>>>>> >>>>>>> 1. Write down the differences between the system cache and the >>>>>> metastorage, >>>>>>> and provide a transition guide for plugin developers. >>>>>>> 2. Deprecate the system cache in 2.13. >>>>>>> 3. Remove the system cache in one of the further releases. I don't >>>> think >>>>>>> it's reasonable to do this earlier than mid next year (even that is >>>>>>> potentially too early). >>>>>>> >>>>>>> Removing the system cache is the right move, but let's be more >>>>>> considerate >>>>>>> to the users. There is absolutely no need to rush. >>>>>>> >>>>>>> -Val >>>>>>> >>>>>>> On Wed, Dec 1, 2021 at 7:28 AM Вячеслав Коптилин < >>>>>> slava.kopti...@gmail.com> >>>>>>> wrote: >>>>>>> >>>>>>>> Hi Maxim, >>>>>>>> >>>>>>>>> On the other hand, I don't see any valuable reason why the change >>>>>> can't >>>>>>>> be done and we should wait for the future that never comes. >>>>>>>> Maxim, the community must provide a reasonable time interval in >>>> order >>>>>> to >>>>>>>> notify all contributors and wait for updating all 3-rd party >>>> plugins. >>>>>>>> Honestly, it does not seem to me that two, three months (the >>>> current >>>>>>>> timeline - end of March is the release date, so the end of Feb is >>>> code >>>>>>>> freeze) are quite enough. >>>>>>>> >>>>>>>>> I don't see any reasons why we should keep something in the CORE >>>>>> module >>>>>>>> that is being used at PLUGINS only. This is a lack of architecture >>>>>> design >>>>>>>> that must be fixed for sure; >>>>>>>> I agree with you. We just need to provide a window that is enough >>>> to >>>>>> cover >>>>>>>> all related issues. >>>>>>>> For example, distributed meta storage does not provide the ability >>>> to >>>>>>>> atomically change several properties at the same time (there are no >>>>>>>> transactions on this API). >>>>>>>> >>>>>>>> Perhaps, it would be nice to plan to remove the system cache on >>>> 2.14 or >>>>>>>> even 2.15. What do you think? >>>>>>>> >>>>>>>> Thanks, >>>>>>>> S. >>>>>>>> >>>>>>>> >>>>>>>> вт, 30 нояб. 2021 г. в 13:58, Maxim Muzafarov <mmu...@apache.org>: >>>>>>>> >>>>>>>>> Hello Val, >>>>>>>>> >>>>>>>>> Thank you for sharing the details. On the one hand, I agree with >>>> you >>>>>>>>> that there is no need to haste with this change and it must be >>>>>>>>> prepared carefully. On the other hand, I don't see any valuable >>>>>> reason >>>>>>>>> why the change can't be done and we should wait for the future >>>> that >>>>>>>>> never comes. >>>>>>>>> >>>>>>>>> Please, consider the following my considerations: >>>>>>>>> >>>>>>>>> - The release 2.13 is not started yet. It will take months to be >>>>>>>>> released (e.g. the end of March as a possible release date). The >>>> time >>>>>>>>> is more than enough; >>>>>>>>> - The 2.13 release has breaking changes, so it is a good >>>> opportunity >>>>>>>>> to fix some gaps; >>>>>>>>> - I don't see any reasons why we should keep something in the >>>> CORE >>>>>>>>> module that is being used at PLUGINS only. This is a lack of >>>>>>>>> architecture design that must be fixed for sure; >>>>>>>>> - The cache affects cluster stability and require additional >>>> human >>>>>>>>> resources for the code maintenance; >>>>>>>>> - As a straightforward solution plugins can create their own >>>> internal >>>>>>>>> caches and use them ever they like (or use the metastorage as I >>>>>>>>> mentioned earlier). Moving system cache to plugin doesn't look so >>>>>>>>> complicated and harmful; >>>>>>>>> >>>>>>>>> On Mon, 29 Nov 2021 at 23:07, Valentin Kulichenko >>>>>>>>> <valentin.kuliche...@gmail.com> wrote: >>>>>>>>>> >>>>>>>>>> Maxim, >>>>>>>>>> >>>>>>>>>> You're right that the system cache is still likely to be used >>>> by >>>>>>>> plugins. >>>>>>>>>> We at GridGain use it for security features, for example. As >>>> far >>>>>> as I >>>>>>>>> know, >>>>>>>>>> that's not the only case. >>>>>>>>>> >>>>>>>>>> I also agree that the metastorage should be the preferred way >>>> for >>>>>> this >>>>>>>>> kind >>>>>>>>>> of purposes, but is there any harm in keeping the system cache >>>> for >>>>>> now? >>>>>>>>>> Unlike the legacy Service Grid, this is not a public feature >>>> that >>>>>> can >>>>>>>> be >>>>>>>>>> used directly by a regular user, so I'm not sure why the rush. >>>>>>>>>> >>>>>>>>>> How about we instead deprecate the system cache, clearly >>>> document >>>>>> how >>>>>>>> to >>>>>>>>>> use the metastorage as an alternative, and then complete the >>>>>> removal >>>>>>>>>> sometime in the future? >>>>>>>>>> >>>>>>>>>> -Val >>>>>>>>>> >>>>>>>>>> On Sun, Nov 28, 2021 at 6:49 AM Maxim Muzafarov < >>>> mmu...@apache.org >>>>>>> >>>>>>>>> wrote: >>>>>>>>>> >>>>>>>>>>> Igniters, >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Since the legacy service grid implementation was removed [2] >>>> I'd >>>>>> like >>>>>>>>>>> to remove the ignite-sys-cache also. It is still possible >>>> that >>>>>> some >>>>>>>> of >>>>>>>>>>> Ignite plugins (e.g. security plugin) are still using this >>>> cache, >>>>>>>>>>> however, AFAIK this is not the reason to keep the outdated >>>> system >>>>>>>>>>> cache and these plugins must be migrated to the metastorage >>>>>> instead. >>>>>>>>>>> >>>>>>>>>>> I've filed the issue [1] for removal. Any objections? >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> [1] https://issues.apache.org/jira/browse/IGNITE-16008 >>>>>>>>>>> [2] https://issues.apache.org/jira/browse/IGNITE-15758 >>>>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>> >>>> >> >>
