[DISCUSSION] Java 11 for Ignite 3.0 development
Igniters, We already had some discussion about using modern Java versions for Ignite 3.0 development [1] but we still don't have consensus. As I see from this discussion the strongest argument for Java 11 is the fact that Java 11 is the latest LTS release which will have premier support until September 2023. So I don't see any reason for preferring any other version of Java at this moment. The purpose of this thread is to gather opinions about using Java 11 in the Ignite 3.0 project and, eventually, reach a consensus on this. I want to share my several arguments in favor of abandoning Java 8 and preferring Java 11: * Java 8 has gone through the End of Public Updates process for legacy releases. So it doesn't make sense to start new development on Java 8. * Java 9+ brings Jigsaw modularization which allows us to have more fine-grained structure of Ignite modules and APIs in the future. * Ignite actively uses Unsafe functionality which, firstly, isn't public, and secondly, leads to problems with running Ignite under Java 9+ (modularization which requires dozens of command-line options in order to forcibly export corresponding packages) and GraalVM. Such a situation could be described as bad user experience and we should fix it. Var handles [2] could be used for solving described problems. * Java 9+ introduces Flight Recorder API [3] which could be used in the Ignite project for lightweight profiling of internal processes. Please, share your opinions, objections and ideas about this topic. I hope we will not have serious disagreements and the consensus will be reached quickly. 1. http://apache-ignite-developers.2346864.n4.nabble.com/DISCUSS-Ignite-3-0-development-approach-tp49922p50295.html 2. https://docs.oracle.com/javase/9/docs/api/java/lang/invoke/VarHandle.html 3. https://docs.oracle.com/en/java/javase/11/docs/api/jdk.jfr/jdk/jfr/FlightRecorder.html
Re: [DISCUSSION] Java 11 for Ignite 3.0 development
I totally support Java 11 for development. It's time to go forward вт, 8 дек. 2020 г. в 11:40, Andrey Gura : > Igniters, > > We already had some discussion about using modern Java versions for > Ignite 3.0 development [1] but we still don't have consensus. > As I see from this discussion the strongest argument for Java 11 is > the fact that Java 11 is the latest LTS release which will have > premier support until September 2023. So I don't see any reason for > preferring any other version of Java at this moment. > > The purpose of this thread is to gather opinions about using Java 11 > in the Ignite 3.0 project and, eventually, reach a consensus on this. > > I want to share my several arguments in favor of abandoning Java 8 and > preferring Java 11: > > * Java 8 has gone through the End of Public Updates process for legacy > releases. So it doesn't make sense to start new development on Java 8. > > * Java 9+ brings Jigsaw modularization which allows us to have more > fine-grained structure of Ignite modules and APIs in the future. > > * Ignite actively uses Unsafe functionality which, firstly, isn't > public, and secondly, leads to problems with running Ignite under Java > 9+ (modularization which requires dozens of command-line options in > order to forcibly export corresponding packages) and GraalVM. Such a > situation could be described as bad user experience and we should fix > it. Var handles [2] could be used for solving described problems. > > * Java 9+ introduces Flight Recorder API [3] which could be used in > the Ignite project for lightweight profiling of internal processes. > > Please, share your opinions, objections and ideas about this topic. I > hope we will not have serious disagreements and the consensus will be > reached quickly. > > > 1. > http://apache-ignite-developers.2346864.n4.nabble.com/DISCUSS-Ignite-3-0-development-approach-tp49922p50295.html > 2. > https://docs.oracle.com/javase/9/docs/api/java/lang/invoke/VarHandle.html > 3. > https://docs.oracle.com/en/java/javase/11/docs/api/jdk.jfr/jdk/jfr/FlightRecorder.html >
Re: [DISCUSSION] Java 11 for Ignite 3.0 development
+1, Java 11 seems to be the only right choice at the moment. On Tue, Dec 8, 2020 at 12:08 PM Alexey Zinoviev wrote: > I totally support Java 11 for development. It's time to go forward > > вт, 8 дек. 2020 г. в 11:40, Andrey Gura : > > > Igniters, > > > > We already had some discussion about using modern Java versions for > > Ignite 3.0 development [1] but we still don't have consensus. > > As I see from this discussion the strongest argument for Java 11 is > > the fact that Java 11 is the latest LTS release which will have > > premier support until September 2023. So I don't see any reason for > > preferring any other version of Java at this moment. > > > > The purpose of this thread is to gather opinions about using Java 11 > > in the Ignite 3.0 project and, eventually, reach a consensus on this. > > > > I want to share my several arguments in favor of abandoning Java 8 and > > preferring Java 11: > > > > * Java 8 has gone through the End of Public Updates process for legacy > > releases. So it doesn't make sense to start new development on Java 8. > > > > * Java 9+ brings Jigsaw modularization which allows us to have more > > fine-grained structure of Ignite modules and APIs in the future. > > > > * Ignite actively uses Unsafe functionality which, firstly, isn't > > public, and secondly, leads to problems with running Ignite under Java > > 9+ (modularization which requires dozens of command-line options in > > order to forcibly export corresponding packages) and GraalVM. Such a > > situation could be described as bad user experience and we should fix > > it. Var handles [2] could be used for solving described problems. > > > > * Java 9+ introduces Flight Recorder API [3] which could be used in > > the Ignite project for lightweight profiling of internal processes. > > > > Please, share your opinions, objections and ideas about this topic. I > > hope we will not have serious disagreements and the consensus will be > > reached quickly. > > > > > > 1. > > > http://apache-ignite-developers.2346864.n4.nabble.com/DISCUSS-Ignite-3-0-development-approach-tp49922p50295.html > > 2. > > > https://docs.oracle.com/javase/9/docs/api/java/lang/invoke/VarHandle.html > > 3. > > > https://docs.oracle.com/en/java/javase/11/docs/api/jdk.jfr/jdk/jfr/FlightRecorder.html > > >
Re: [DISCUSSION] Java 11 for Ignite 3.0 development
Hello! Andrey's arguments are solid. On Tue, Dec 8, 2020 at 12:23 PM Pavel Tupitsyn wrote: > +1, Java 11 seems to be the only right choice at the moment. > > On Tue, Dec 8, 2020 at 12:08 PM Alexey Zinoviev > wrote: > > > I totally support Java 11 for development. It's time to go forward > > > > вт, 8 дек. 2020 г. в 11:40, Andrey Gura : > > > > > Igniters, > > > > > > We already had some discussion about using modern Java versions for > > > Ignite 3.0 development [1] but we still don't have consensus. > > > As I see from this discussion the strongest argument for Java 11 is > > > the fact that Java 11 is the latest LTS release which will have > > > premier support until September 2023. So I don't see any reason for > > > preferring any other version of Java at this moment. > > > > > > The purpose of this thread is to gather opinions about using Java 11 > > > in the Ignite 3.0 project and, eventually, reach a consensus on this. > > > > > > I want to share my several arguments in favor of abandoning Java 8 and > > > preferring Java 11: > > > > > > * Java 8 has gone through the End of Public Updates process for legacy > > > releases. So it doesn't make sense to start new development on Java 8. > > > > > > * Java 9+ brings Jigsaw modularization which allows us to have more > > > fine-grained structure of Ignite modules and APIs in the future. > > > > > > * Ignite actively uses Unsafe functionality which, firstly, isn't > > > public, and secondly, leads to problems with running Ignite under Java > > > 9+ (modularization which requires dozens of command-line options in > > > order to forcibly export corresponding packages) and GraalVM. Such a > > > situation could be described as bad user experience and we should fix > > > it. Var handles [2] could be used for solving described problems. > > > > > > * Java 9+ introduces Flight Recorder API [3] which could be used in > > > the Ignite project for lightweight profiling of internal processes. > > > > > > Please, share your opinions, objections and ideas about this topic. I > > > hope we will not have serious disagreements and the consensus will be > > > reached quickly. > > > > > > > > > 1. > > > > > > http://apache-ignite-developers.2346864.n4.nabble.com/DISCUSS-Ignite-3-0-development-approach-tp49922p50295.html > > > 2. > > > > > > https://docs.oracle.com/javase/9/docs/api/java/lang/invoke/VarHandle.html > > > 3. > > > > > > https://docs.oracle.com/en/java/javase/11/docs/api/jdk.jfr/jdk/jfr/FlightRecorder.html > > > > > >
[DISCUSSION] Modules organization in Ignite 3
Igniters, I want to tackle the topic of modules structure in Ignite 3. So far, the modules in Ignite are mostly defined intuitively which leads to some complications: - Ignite public API is separated from the rest of the code only by package name. This leads to private classes leaking to public API which is very hard to catch even during the review process (we missed a bunch of such leaks for new metrics API [1] and I remember this happening for almost every SPI) - Classes from 'internal' packages are considered to be 'free for grabs' in every place of the code. This leads to tight coupling and abstraction leakage in the code. An example of such a case - an often cast of WALPointer to FileWALPointer, so that the community decided to get rid of the WALPointer interface altogether [2] - Overall code complexity. Because of the lack of inter-module interaction rules, we are free to add new methods and callbacks to any class, which leads to duplicating entities and verbose interfaces. A good example of this is the clear duplication of methods in IgniteCacheOffheapManager and IgniteCacheOffheapManager.DataStore [3] I think we need to work out some rules that will help us define and control both Ignite public API and module internal API which still defines a clear contract for other modules. Some ideas: - Perhaps we can move all user public classed and interfaces to an Ignite-API module which will have no dependencies on implementation modules. This will prevent private classes from leaking to the API module. - We need somehow define which classes from a module are exposed to other modules, and which classes are left for module-private usage. Maybe Java's jigsaw will help us here, but maybe we will be ok with just more strict java access modifiers usage :) The idea here is that a module should never touch a dependent module's private classes, ever. The exported classes and interfaces are still free to be modified between releases, as long as it is not a user public API. - A module should be logically complete, thus it may be beneficial if module name matches with the code package it provides (e.g. configuration -> org.apache.ignite.configuration, replication -> org.apache.ignite.replication, raft->org.apache.ignite.raft, etc) Any other principles/rules we can apply to make the code structure more concise? Thoughts? --AG [1] https://issues.apache.org/jira/browse/IGNITE-12552 [2] https://issues.apache.org/jira/browse/IGNITE-13513 [3] https://issues.apache.org/jira/browse/IGNITE-13220
Re: [DISCUSSION] Java 11 for Ignite 3.0 development
+1 08.12.2020, 12:48, "Philipp Masharov" : > Hello! > > Andrey's arguments are solid. > > On Tue, Dec 8, 2020 at 12:23 PM Pavel Tupitsyn wrote: > >> +1, Java 11 seems to be the only right choice at the moment. >> >> On Tue, Dec 8, 2020 at 12:08 PM Alexey Zinoviev >> wrote: >> >> > I totally support Java 11 for development. It's time to go forward >> > >> > вт, 8 дек. 2020 г. в 11:40, Andrey Gura : >> > >> > > Igniters, >> > > >> > > We already had some discussion about using modern Java versions for >> > > Ignite 3.0 development [1] but we still don't have consensus. >> > > As I see from this discussion the strongest argument for Java 11 is >> > > the fact that Java 11 is the latest LTS release which will have >> > > premier support until September 2023. So I don't see any reason for >> > > preferring any other version of Java at this moment. >> > > >> > > The purpose of this thread is to gather opinions about using Java 11 >> > > in the Ignite 3.0 project and, eventually, reach a consensus on this. >> > > >> > > I want to share my several arguments in favor of abandoning Java 8 and >> > > preferring Java 11: >> > > >> > > * Java 8 has gone through the End of Public Updates process for legacy >> > > releases. So it doesn't make sense to start new development on Java 8. >> > > >> > > * Java 9+ brings Jigsaw modularization which allows us to have more >> > > fine-grained structure of Ignite modules and APIs in the future. >> > > >> > > * Ignite actively uses Unsafe functionality which, firstly, isn't >> > > public, and secondly, leads to problems with running Ignite under Java >> > > 9+ (modularization which requires dozens of command-line options in >> > > order to forcibly export corresponding packages) and GraalVM. Such a >> > > situation could be described as bad user experience and we should fix >> > > it. Var handles [2] could be used for solving described problems. >> > > >> > > * Java 9+ introduces Flight Recorder API [3] which could be used in >> > > the Ignite project for lightweight profiling of internal processes. >> > > >> > > Please, share your opinions, objections and ideas about this topic. I >> > > hope we will not have serious disagreements and the consensus will be >> > > reached quickly. >> > > >> > > >> > > 1. >> > > >> > >> >> http://apache-ignite-developers.2346864.n4.nabble.com/DISCUSS-Ignite-3-0-development-approach-tp49922p50295.html >> > > 2. >> > > >> > >> https://docs.oracle.com/javase/9/docs/api/java/lang/invoke/VarHandle.html >> > > 3. >> > > >> > >> >> https://docs.oracle.com/en/java/javase/11/docs/api/jdk.jfr/jdk/jfr/FlightRecorder.html >> > > >> >
Pull request for minor fix in index page documentation
Hi team, I just made a minor change in apache ignite index page documentation. But I am not sure the pull request raised is sufficient or do I need to follow any additional steps. Please let me know/advice on the same. Also is there any way to verify these changes as these are related to documentation being loaded on the ignite website and not related to code? https://github.com/sumitdeshinge/ignite/pull/1 -- Regards, Sumit Deshinge
[jira] [Created] (IGNITE-13828) Change ducktape dependency to own fork
Sergei Ryzhov created IGNITE-13828: -- Summary: Change ducktape dependency to own fork Key: IGNITE-13828 URL: https://issues.apache.org/jira/browse/IGNITE-13828 Project: Ignite Issue Type: Task Reporter: Sergei Ryzhov Assignee: Sergei Ryzhov The ducktape repository doesn't contain everything we need, so we need to use our own fork with extensions -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (IGNITE-13829) Add log rotation to ducktape-tests
Sergei Ryzhov created IGNITE-13829: -- Summary: Add log rotation to ducktape-tests Key: IGNITE-13829 URL: https://issues.apache.org/jira/browse/IGNITE-13829 Project: Ignite Issue Type: Task Reporter: Sergei Ryzhov Assignee: Sergei Ryzhov Add log rotation to ducktape-tests -- This message was sent by Atlassian Jira (v8.3.4#803005)
Re: [DISCUSSION] Java 11 for Ignite 3.0 development
+1 for using java 11. > 8 дек. 2020 г., в 13:18, ткаленко кирилл написал(а): > > +1 > > 08.12.2020, 12:48, "Philipp Masharov" : >> Hello! >> >> Andrey's arguments are solid. >> >> On Tue, Dec 8, 2020 at 12:23 PM Pavel Tupitsyn wrote: >> >>> +1, Java 11 seems to be the only right choice at the moment. >>> >>> On Tue, Dec 8, 2020 at 12:08 PM Alexey Zinoviev >>> wrote: >>> >>> > I totally support Java 11 for development. It's time to go forward >>> > >>> > вт, 8 дек. 2020 г. в 11:40, Andrey Gura : >>> > >>> > > Igniters, >>> > > >>> > > We already had some discussion about using modern Java versions for >>> > > Ignite 3.0 development [1] but we still don't have consensus. >>> > > As I see from this discussion the strongest argument for Java 11 is >>> > > the fact that Java 11 is the latest LTS release which will have >>> > > premier support until September 2023. So I don't see any reason for >>> > > preferring any other version of Java at this moment. >>> > > >>> > > The purpose of this thread is to gather opinions about using Java 11 >>> > > in the Ignite 3.0 project and, eventually, reach a consensus on this. >>> > > >>> > > I want to share my several arguments in favor of abandoning Java 8 and >>> > > preferring Java 11: >>> > > >>> > > * Java 8 has gone through the End of Public Updates process for legacy >>> > > releases. So it doesn't make sense to start new development on Java 8. >>> > > >>> > > * Java 9+ brings Jigsaw modularization which allows us to have more >>> > > fine-grained structure of Ignite modules and APIs in the future. >>> > > >>> > > * Ignite actively uses Unsafe functionality which, firstly, isn't >>> > > public, and secondly, leads to problems with running Ignite under Java >>> > > 9+ (modularization which requires dozens of command-line options in >>> > > order to forcibly export corresponding packages) and GraalVM. Such a >>> > > situation could be described as bad user experience and we should fix >>> > > it. Var handles [2] could be used for solving described problems. >>> > > >>> > > * Java 9+ introduces Flight Recorder API [3] which could be used in >>> > > the Ignite project for lightweight profiling of internal processes. >>> > > >>> > > Please, share your opinions, objections and ideas about this topic. I >>> > > hope we will not have serious disagreements and the consensus will be >>> > > reached quickly. >>> > > >>> > > >>> > > 1. >>> > > >>> > >>> >>> http://apache-ignite-developers.2346864.n4.nabble.com/DISCUSS-Ignite-3-0-development-approach-tp49922p50295.html >>> > > 2. >>> > > >>> > >>> https://docs.oracle.com/javase/9/docs/api/java/lang/invoke/VarHandle.html >>> > > 3. >>> > > >>> > >>> >>> https://docs.oracle.com/en/java/javase/11/docs/api/jdk.jfr/jdk/jfr/FlightRecorder.html >>> > > >>> >
Re: [DISCUSSION] Modules organization in Ignite 3
Hello, Alexey. Think we can extend our @IgniteExperimental annotation. `@IgniteExperimental` - mark features that are truly experimental and can be completely removed in future releases. `@NotRecommended` - mark features that widely adopted by the users but implemented wrong or have known issues that can’t be fixed. `@NotStable` - mark features supported by community but API not stable and can be reworked in the next release. `@Stable` - mark features that are completely OK and here to stay. We should output notes about these annotations in the JavaDoc, also. What do you think? > 8 дек. 2020 г., в 12:49, Alexey Goncharuk > написал(а): > > Igniters, > > I want to tackle the topic of modules structure in Ignite 3. So far, the > modules in Ignite are mostly defined intuitively which leads to some > complications: > > - Ignite public API is separated from the rest of the code only by > package name. This leads to private classes leaking to public API which is > very hard to catch even during the review process (we missed a bunch of > such leaks for new metrics API [1] and I remember this happening for almost > every SPI) > - Classes from 'internal' packages are considered to be 'free for grabs' > in every place of the code. This leads to tight coupling and abstraction > leakage in the code. An example of such a case - an often cast of > WALPointer to FileWALPointer, so that the community decided to get rid of > the WALPointer interface altogether [2] > - Overall code complexity. Because of the lack of inter-module > interaction rules, we are free to add new methods and callbacks to any > class, which leads to duplicating entities and verbose interfaces. A good > example of this is the clear duplication of methods in > IgniteCacheOffheapManager and IgniteCacheOffheapManager.DataStore [3] > > I think we need to work out some rules that will help us define and control > both Ignite public API and module internal API which still defines a clear > contract for other modules. Some ideas: > > - Perhaps we can move all user public classed and interfaces to an > Ignite-API module which will have no dependencies on implementation > modules. This will prevent private classes from leaking to the API module. > - We need somehow define which classes from a module are exposed to > other modules, and which classes are left for module-private usage. Maybe > Java's jigsaw will help us here, but maybe we will be ok with just more > strict java access modifiers usage :) The idea here is that a module should > never touch a dependent module's private classes, ever. The exported > classes and interfaces are still free to be modified between releases, as > long as it is not a user public API. > - A module should be logically complete, thus it may be beneficial if > module name matches with the code package it provides (e.g. configuration > -> org.apache.ignite.configuration, replication -> > org.apache.ignite.replication, raft->org.apache.ignite.raft, etc) > > Any other principles/rules we can apply to make the code structure more > concise? Thoughts? > > --AG > > [1] https://issues.apache.org/jira/browse/IGNITE-12552 > [2] https://issues.apache.org/jira/browse/IGNITE-13513 > [3] https://issues.apache.org/jira/browse/IGNITE-13220
Re: [DISCUSSION] Modules organization in Ignite 3
Definitely agree with Alexey. Separating API declaration from implementation could really improve system design and avoid coupling. About extending @IgniteExperimental annotation. It doesn't look good to me. We should consider any API either experimental or stable. Third option is deprecated API. Merging API which is known in advance that it is unstable is a very bad idea. On Tue, Dec 8, 2020 at 3:48 PM Nikolay Izhikov wrote: > > Hello, Alexey. > > Think we can extend our @IgniteExperimental annotation. > > `@IgniteExperimental` - mark features that are truly experimental and can be > completely removed in future releases. > `@NotRecommended` - mark features that widely adopted by the users but > implemented wrong or have known issues that can’t be fixed. > `@NotStable` - mark features supported by community but API not stable and > can be reworked in the next release. > `@Stable` - mark features that are completely OK and here to stay. > > We should output notes about these annotations in the JavaDoc, also. > What do you think? > > > > 8 дек. 2020 г., в 12:49, Alexey Goncharuk > > написал(а): > > > > Igniters, > > > > I want to tackle the topic of modules structure in Ignite 3. So far, the > > modules in Ignite are mostly defined intuitively which leads to some > > complications: > > > > - Ignite public API is separated from the rest of the code only by > > package name. This leads to private classes leaking to public API which is > > very hard to catch even during the review process (we missed a bunch of > > such leaks for new metrics API [1] and I remember this happening for > > almost > > every SPI) > > - Classes from 'internal' packages are considered to be 'free for grabs' > > in every place of the code. This leads to tight coupling and abstraction > > leakage in the code. An example of such a case - an often cast of > > WALPointer to FileWALPointer, so that the community decided to get rid of > > the WALPointer interface altogether [2] > > - Overall code complexity. Because of the lack of inter-module > > interaction rules, we are free to add new methods and callbacks to any > > class, which leads to duplicating entities and verbose interfaces. A good > > example of this is the clear duplication of methods in > > IgniteCacheOffheapManager and IgniteCacheOffheapManager.DataStore [3] > > > > I think we need to work out some rules that will help us define and control > > both Ignite public API and module internal API which still defines a clear > > contract for other modules. Some ideas: > > > > - Perhaps we can move all user public classed and interfaces to an > > Ignite-API module which will have no dependencies on implementation > > modules. This will prevent private classes from leaking to the API module. > > - We need somehow define which classes from a module are exposed to > > other modules, and which classes are left for module-private usage. Maybe > > Java's jigsaw will help us here, but maybe we will be ok with just more > > strict java access modifiers usage :) The idea here is that a module > > should > > never touch a dependent module's private classes, ever. The exported > > classes and interfaces are still free to be modified between releases, as > > long as it is not a user public API. > > - A module should be logically complete, thus it may be beneficial if > > module name matches with the code package it provides (e.g. configuration > > -> org.apache.ignite.configuration, replication -> > > org.apache.ignite.replication, raft->org.apache.ignite.raft, etc) > > > > Any other principles/rules we can apply to make the code structure more > > concise? Thoughts? > > > > --AG > > > > [1] https://issues.apache.org/jira/browse/IGNITE-12552 > > [2] https://issues.apache.org/jira/browse/IGNITE-13513 > > [3] https://issues.apache.org/jira/browse/IGNITE-13220 >
Replace Future.get with Future.get(int timeout) in tests
Hello, Igniters. Currently, we have a lot of usages `Future.get` without a timeout in tests. In case the test that uses `Future.get` is flaky it can lead to the whole suite hang. Is there any reason to use the get method without a timeout? Can we a. Replace all invocation of get with the timeoutable companion. b. Introduce code style check that will prohibit to use non-timeout method in tests. What do you think?
Re: Replace Future.get with Future.get(int timeout) in tests
It seems to be a good topic, but it seems to be left to the reviewer's discretion. 08.12.2020, 18:36, "Nikolay Izhikov" : > Hello, Igniters. > > Currently, we have a lot of usages `Future.get` without a timeout in tests. > In case the test that uses `Future.get` is flaky it can lead to the whole > suite hang. > > Is there any reason to use the get method without a timeout? > Can we > > a. Replace all invocation of get with the timeoutable companion. > b. Introduce code style check that will prohibit to use non-timeout method in > tests. > > What do you think?
Re: [DISCUSSION] Modules organization in Ignite 3
+1 for modules 08.12.2020, 16:02, "Andrey Gura" : > Definitely agree with Alexey. Separating API declaration from > implementation could really improve system design and avoid coupling. > > About extending @IgniteExperimental annotation. It doesn't look good > to me. We should consider any API either experimental or stable. Third > option is deprecated API. Merging API which is known in advance that > it is unstable is a very bad idea. > > On Tue, Dec 8, 2020 at 3:48 PM Nikolay Izhikov wrote: >> Hello, Alexey. >> >> Think we can extend our @IgniteExperimental annotation. >> >> `@IgniteExperimental` - mark features that are truly experimental and can >> be completely removed in future releases. >> `@NotRecommended` - mark features that widely adopted by the users but >> implemented wrong or have known issues that can’t be fixed. >> `@NotStable` - mark features supported by community but API not stable and >> can be reworked in the next release. >> `@Stable` - mark features that are completely OK and here to stay. >> >> We should output notes about these annotations in the JavaDoc, also. >> What do you think? >> >> > 8 дек. 2020 г., в 12:49, Alexey Goncharuk >> написал(а): >> > >> > Igniters, >> > >> > I want to tackle the topic of modules structure in Ignite 3. So far, the >> > modules in Ignite are mostly defined intuitively which leads to some >> > complications: >> > >> > - Ignite public API is separated from the rest of the code only by >> > package name. This leads to private classes leaking to public API which is >> > very hard to catch even during the review process (we missed a bunch of >> > such leaks for new metrics API [1] and I remember this happening for >> almost >> > every SPI) >> > - Classes from 'internal' packages are considered to be 'free for grabs' >> > in every place of the code. This leads to tight coupling and abstraction >> > leakage in the code. An example of such a case - an often cast of >> > WALPointer to FileWALPointer, so that the community decided to get rid of >> > the WALPointer interface altogether [2] >> > - Overall code complexity. Because of the lack of inter-module >> > interaction rules, we are free to add new methods and callbacks to any >> > class, which leads to duplicating entities and verbose interfaces. A good >> > example of this is the clear duplication of methods in >> > IgniteCacheOffheapManager and IgniteCacheOffheapManager.DataStore [3] >> > >> > I think we need to work out some rules that will help us define and >> control >> > both Ignite public API and module internal API which still defines a clear >> > contract for other modules. Some ideas: >> > >> > - Perhaps we can move all user public classed and interfaces to an >> > Ignite-API module which will have no dependencies on implementation >> > modules. This will prevent private classes from leaking to the API module. >> > - We need somehow define which classes from a module are exposed to >> > other modules, and which classes are left for module-private usage. Maybe >> > Java's jigsaw will help us here, but maybe we will be ok with just more >> > strict java access modifiers usage :) The idea here is that a module >> should >> > never touch a dependent module's private classes, ever. The exported >> > classes and interfaces are still free to be modified between releases, as >> > long as it is not a user public API. >> > - A module should be logically complete, thus it may be beneficial if >> > module name matches with the code package it provides (e.g. configuration >> > -> org.apache.ignite.configuration, replication -> >> > org.apache.ignite.replication, raft->org.apache.ignite.raft, etc) >> > >> > Any other principles/rules we can apply to make the code structure more >> > concise? Thoughts? >> > >> > --AG >> > >> > [1] https://issues.apache.org/jira/browse/IGNITE-12552 >> > [2] https://issues.apache.org/jira/browse/IGNITE-13513 >> > [3] https://issues.apache.org/jira/browse/IGNITE-13220
Re: Replace Future.get with Future.get(int timeout) in tests
Not only futures, there are a lot of latches, barriers etc. with same problem. вт, 8 дек. 2020 г. в 18:41, ткаленко кирилл : > It seems to be a good topic, but it seems to be left to the reviewer's > discretion. > > 08.12.2020, 18:36, "Nikolay Izhikov" : > > Hello, Igniters. > > > > Currently, we have a lot of usages `Future.get` without a timeout in > tests. > > In case the test that uses `Future.get` is flaky it can lead to the > whole suite hang. > > > > Is there any reason to use the get method without a timeout? > > Can we > > > > a. Replace all invocation of get with the timeoutable companion. > > b. Introduce code style check that will prohibit to use non-timeout > method in tests. > > > > What do you think? > -- Sincerely yours, Ivan Daschinskiy
Removing MVCC public API
Hello Igniters, I want to start voting on removing the public API (and eventually all unused parts) related to the MVCC feature. This topic has already been discussed many times (at least, [1], [2]) and the community has agreed the feature implementation must be reapproached, because using coordinator node for transactions ordering and 2pc protocol is slow by design and will not scale well. [3] Moreover, the current implementation has critical issues [4], not supported by the community, and not well tested at all. Removing the public API first will allow us to clean up the code later step by step without rushing and keep intact useful improvements that are already in use or can be reused for other parts in the future. For instance, partition counters implementation is already adapted to fix tx caches protocol issues [5]. The future of MVCC is unclear for now, but, definitely, this feature is useful for a lot of user scenarios and can be scheduled for later Ignite versions. Also, the MVCC feature is in an experimental state, so it can be modified in any way, I think. +1 - to accept removing MVVC feature from public API 0 - don't care either way -1 - do not accept removing API (explain why) The vote will hold for 7 days and will end on Wednesday, December 16th at 19:00 UTC: https://www.timeanddate.com/countdown/generic?iso=20201216T19&p0=1440&font=cursive [1] http://apache-ignite-developers.2346864.n4.nabble.com/Mark-MVCC-with-IgniteExperimental-td45669.html [2] http://apache-ignite-developers.2346864.n4.nabble.com/Disable-MVCC-test-suites-td50416.html [3] http://apache-ignite-developers.2346864.n4.nabble.com/Mark-MVCC-with-IgniteExperimental-tp45669p45727.html [4] http://apache-ignite-developers.2346864.n4.nabble.com/Mark-MVCC-with-IgniteExperimental-tp45669p45716.html [5] http://apache-ignite-developers.2346864.n4.nabble.com/Mark-MVCC-with-IgniteExperimental-tp45669p45714.html Thanks, Slava.
Re: Pull request for minor fix in index page documentation
Hi Sumit, Thanks for catching the issue and preparing a pull-request! The pull-request merges changes to your Ignite master branch rather than to the Ignite master. Not a bid deal. I see where the problem is. @Nikita Safonov , @Viktor Chemodanov , could any of your correct the broken link on the page? As for the docs contribution process, it's here. There is a step that explains how to build the HTML version of the docs locally before pushing live: https://cwiki.apache.org/confluence/display/IGNITE/How+to+Document#HowtoDocument-ContributingtoDocumentation - Denis On Tue, Dec 8, 2020 at 2:21 AM Sumit Deshinge wrote: > Hi team, > > I just made a minor change in apache ignite index page documentation. > But I am not sure the pull request raised is sufficient or do I need to > follow any additional steps. Please let me know/advice on the same. > Also is there any way to verify these changes as these are related to > documentation being loaded on the ignite website and not related to code? > https://github.com/sumitdeshinge/ignite/pull/1 > > > -- > Regards, > Sumit Deshinge >
Re: [DISCUSSION] Java 11 for Ignite 3.0 development
+1 for sure. AFAIK, the only thing holding us back from using Java 11 is the dominance of Java 8, but I'm sure that by the time Ignite 3 is GA, there will be much fewer Java 8 users if any significant number at all. By the by, Ignite's sources need minimal effort to be able to be compiled with Java 11 as a target. 08.12.2020, 15:00, "Nikolay Izhikov" : > +1 for using java 11. > >> 8 дек. 2020 г., в 13:18, ткаленко кирилл написал(а): >> >> +1 >> >> 08.12.2020, 12:48, "Philipp Masharov" : >>> Hello! >>> >>> Andrey's arguments are solid. >>> >>> On Tue, Dec 8, 2020 at 12:23 PM Pavel Tupitsyn >>> wrote: >>> +1, Java 11 seems to be the only right choice at the moment. On Tue, Dec 8, 2020 at 12:08 PM Alexey Zinoviev wrote: > I totally support Java 11 for development. It's time to go forward > > вт, 8 дек. 2020 г. в 11:40, Andrey Gura : > > > Igniters, > > > > We already had some discussion about using modern Java versions for > > Ignite 3.0 development [1] but we still don't have consensus. > > As I see from this discussion the strongest argument for Java 11 is > > the fact that Java 11 is the latest LTS release which will have > > premier support until September 2023. So I don't see any reason for > > preferring any other version of Java at this moment. > > > > The purpose of this thread is to gather opinions about using Java 11 > > in the Ignite 3.0 project and, eventually, reach a consensus on this. > > > > I want to share my several arguments in favor of abandoning Java 8 and > > preferring Java 11: > > > > * Java 8 has gone through the End of Public Updates process for legacy > > releases. So it doesn't make sense to start new development on Java 8. > > > > * Java 9+ brings Jigsaw modularization which allows us to have more > > fine-grained structure of Ignite modules and APIs in the future. > > > > * Ignite actively uses Unsafe functionality which, firstly, isn't > > public, and secondly, leads to problems with running Ignite under Java > > 9+ (modularization which requires dozens of command-line options in > > order to forcibly export corresponding packages) and GraalVM. Such a > > situation could be described as bad user experience and we should fix > > it. Var handles [2] could be used for solving described problems. > > > > * Java 9+ introduces Flight Recorder API [3] which could be used in > > the Ignite project for lightweight profiling of internal processes. > > > > Please, share your opinions, objections and ideas about this topic. I > > hope we will not have serious disagreements and the consensus will be > > reached quickly. > > > > > > 1. > > > http://apache-ignite-developers.2346864.n4.nabble.com/DISCUSS-Ignite-3-0-development-approach-tp49922p50295.html > > 2. > > > https://docs.oracle.com/javase/9/docs/api/java/lang/invoke/VarHandle.html > > 3. > > > https://docs.oracle.com/en/java/javase/11/docs/api/jdk.jfr/jdk/jfr/FlightRecorder.html > > >
Re: Removing MVCC public API
+1 On Tue, Dec 8, 2020 at 8:31 AM Вячеслав Коптилин wrote: > Hello Igniters, > > I want to start voting on removing the public API (and eventually all > unused parts) related to the MVCC feature. > > This topic has already been discussed many times (at least, [1], [2]) and > the community has agreed the feature implementation must be reapproached, > because using coordinator node for transactions ordering and 2pc protocol > is slow by design and will not scale well. [3] > > Moreover, the current implementation has critical issues [4], not supported > by the community, and not well tested at all. > > Removing the public API first will allow us to clean up the code later step > by step without rushing and keep intact useful improvements that are > already in use or can be reused for other parts in the future. > For instance, partition counters implementation is already adapted to fix > tx caches protocol issues [5]. > > The future of MVCC is unclear for now, but, definitely, this feature is > useful for a lot of user scenarios and can be scheduled for later Ignite > versions. > Also, the MVCC feature is in an experimental state, so it can be modified > in any way, I think. > > +1 - to accept removing MVVC feature from public API > 0 - don't care either way > -1 - do not accept removing API (explain why) > > The vote will hold for 7 days and will end on Wednesday, December 16th at > 19:00 UTC: > > https://www.timeanddate.com/countdown/generic?iso=20201216T19&p0=1440&font=cursive > > [1] > > http://apache-ignite-developers.2346864.n4.nabble.com/Mark-MVCC-with-IgniteExperimental-td45669.html > [2] > > http://apache-ignite-developers.2346864.n4.nabble.com/Disable-MVCC-test-suites-td50416.html > [3] > > http://apache-ignite-developers.2346864.n4.nabble.com/Mark-MVCC-with-IgniteExperimental-tp45669p45727.html > [4] > > http://apache-ignite-developers.2346864.n4.nabble.com/Mark-MVCC-with-IgniteExperimental-tp45669p45716.html > [5] > > http://apache-ignite-developers.2346864.n4.nabble.com/Mark-MVCC-with-IgniteExperimental-tp45669p45714.html > > Thanks, > Slava. >
Re: Removing MVCC public API
+1 > 8 дек. 2020 г., в 21:54, Valentin Kulichenko > написал(а): > > +1 > > On Tue, Dec 8, 2020 at 8:31 AM Вячеслав Коптилин > wrote: > >> Hello Igniters, >> >> I want to start voting on removing the public API (and eventually all >> unused parts) related to the MVCC feature. >> >> This topic has already been discussed many times (at least, [1], [2]) and >> the community has agreed the feature implementation must be reapproached, >> because using coordinator node for transactions ordering and 2pc protocol >> is slow by design and will not scale well. [3] >> >> Moreover, the current implementation has critical issues [4], not supported >> by the community, and not well tested at all. >> >> Removing the public API first will allow us to clean up the code later step >> by step without rushing and keep intact useful improvements that are >> already in use or can be reused for other parts in the future. >> For instance, partition counters implementation is already adapted to fix >> tx caches protocol issues [5]. >> >> The future of MVCC is unclear for now, but, definitely, this feature is >> useful for a lot of user scenarios and can be scheduled for later Ignite >> versions. >> Also, the MVCC feature is in an experimental state, so it can be modified >> in any way, I think. >> >> +1 - to accept removing MVVC feature from public API >> 0 - don't care either way >> -1 - do not accept removing API (explain why) >> >> The vote will hold for 7 days and will end on Wednesday, December 16th at >> 19:00 UTC: >> >> https://www.timeanddate.com/countdown/generic?iso=20201216T19&p0=1440&font=cursive >> >> [1] >> >> http://apache-ignite-developers.2346864.n4.nabble.com/Mark-MVCC-with-IgniteExperimental-td45669.html >> [2] >> >> http://apache-ignite-developers.2346864.n4.nabble.com/Disable-MVCC-test-suites-td50416.html >> [3] >> >> http://apache-ignite-developers.2346864.n4.nabble.com/Mark-MVCC-with-IgniteExperimental-tp45669p45727.html >> [4] >> >> http://apache-ignite-developers.2346864.n4.nabble.com/Mark-MVCC-with-IgniteExperimental-tp45669p45716.html >> [5] >> >> http://apache-ignite-developers.2346864.n4.nabble.com/Mark-MVCC-with-IgniteExperimental-tp45669p45714.html >> >> Thanks, >> Slava. >>
Re: [DISCUSSION] Modules organization in Ignite 3
+1 to Alexey's ideas. As for annotations, I actually can see the difference between Expirimental and NonStable. The latter means that a feature works, but its API can still be changed (in other words, no backwards compatibility guarantees). The former is just something that is under development, can't be tried out, but can't be relied on; potentially, something that can be removed completely. I'm not sure if this can be applied to any of our current situations, but as a general rule - makes sense to me. I do not support adding the NotRecommended option though - such functionality should be just deprecated. -Val On Tue, Dec 8, 2020 at 7:41 AM ткаленко кирилл wrote: > +1 for modules > > 08.12.2020, 16:02, "Andrey Gura" : > > Definitely agree with Alexey. Separating API declaration from > > implementation could really improve system design and avoid coupling. > > > > About extending @IgniteExperimental annotation. It doesn't look good > > to me. We should consider any API either experimental or stable. Third > > option is deprecated API. Merging API which is known in advance that > > it is unstable is a very bad idea. > > > > On Tue, Dec 8, 2020 at 3:48 PM Nikolay Izhikov > wrote: > >> Hello, Alexey. > >> > >> Think we can extend our @IgniteExperimental annotation. > >> > >> `@IgniteExperimental` - mark features that are truly experimental and > can be completely removed in future releases. > >> `@NotRecommended` - mark features that widely adopted by the users but > implemented wrong or have known issues that can’t be fixed. > >> `@NotStable` - mark features supported by community but API not stable > and can be reworked in the next release. > >> `@Stable` - mark features that are completely OK and here to stay. > >> > >> We should output notes about these annotations in the JavaDoc, also. > >> What do you think? > >> > >> > 8 дек. 2020 г., в 12:49, Alexey Goncharuk < > alexey.goncha...@gmail.com> написал(а): > >> > > >> > Igniters, > >> > > >> > I want to tackle the topic of modules structure in Ignite 3. So far, > the > >> > modules in Ignite are mostly defined intuitively which leads to some > >> > complications: > >> > > >> > - Ignite public API is separated from the rest of the code only by > >> > package name. This leads to private classes leaking to public API > which is > >> > very hard to catch even during the review process (we missed a bunch > of > >> > such leaks for new metrics API [1] and I remember this happening for > almost > >> > every SPI) > >> > - Classes from 'internal' packages are considered to be 'free for > grabs' > >> > in every place of the code. This leads to tight coupling and > abstraction > >> > leakage in the code. An example of such a case - an often cast of > >> > WALPointer to FileWALPointer, so that the community decided to get > rid of > >> > the WALPointer interface altogether [2] > >> > - Overall code complexity. Because of the lack of inter-module > >> > interaction rules, we are free to add new methods and callbacks to > any > >> > class, which leads to duplicating entities and verbose interfaces. A > good > >> > example of this is the clear duplication of methods in > >> > IgniteCacheOffheapManager and IgniteCacheOffheapManager.DataStore [3] > >> > > >> > I think we need to work out some rules that will help us define and > control > >> > both Ignite public API and module internal API which still defines a > clear > >> > contract for other modules. Some ideas: > >> > > >> > - Perhaps we can move all user public classed and interfaces to an > >> > Ignite-API module which will have no dependencies on implementation > >> > modules. This will prevent private classes from leaking to the API > module. > >> > - We need somehow define which classes from a module are exposed to > >> > other modules, and which classes are left for module-private usage. > Maybe > >> > Java's jigsaw will help us here, but maybe we will be ok with just > more > >> > strict java access modifiers usage :) The idea here is that a module > should > >> > never touch a dependent module's private classes, ever. The exported > >> > classes and interfaces are still free to be modified between > releases, as > >> > long as it is not a user public API. > >> > - A module should be logically complete, thus it may be beneficial if > >> > module name matches with the code package it provides (e.g. > configuration > >> > -> org.apache.ignite.configuration, replication -> > >> > org.apache.ignite.replication, raft->org.apache.ignite.raft, etc) > >> > > >> > Any other principles/rules we can apply to make the code structure > more > >> > concise? Thoughts? > >> > > >> > --AG > >> > > >> > [1] https://issues.apache.org/jira/browse/IGNITE-12552 > >> > [2] https://issues.apache.org/jira/browse/IGNITE-13513 > >> > [3] https://issues.apache.org/jira/browse/IGNITE-13220 >
Re: Removing MVCC public API
+1 On Tue, Dec 8, 2020 at 10:02 PM Nikolay Izhikov wrote: > > +1 > > > 8 дек. 2020 г., в 21:54, Valentin Kulichenko > > написал(а): > > > > +1 > > > > On Tue, Dec 8, 2020 at 8:31 AM Вячеслав Коптилин > > wrote: > > > >> Hello Igniters, > >> > >> I want to start voting on removing the public API (and eventually all > >> unused parts) related to the MVCC feature. > >> > >> This topic has already been discussed many times (at least, [1], [2]) and > >> the community has agreed the feature implementation must be reapproached, > >> because using coordinator node for transactions ordering and 2pc protocol > >> is slow by design and will not scale well. [3] > >> > >> Moreover, the current implementation has critical issues [4], not supported > >> by the community, and not well tested at all. > >> > >> Removing the public API first will allow us to clean up the code later step > >> by step without rushing and keep intact useful improvements that are > >> already in use or can be reused for other parts in the future. > >> For instance, partition counters implementation is already adapted to fix > >> tx caches protocol issues [5]. > >> > >> The future of MVCC is unclear for now, but, definitely, this feature is > >> useful for a lot of user scenarios and can be scheduled for later Ignite > >> versions. > >> Also, the MVCC feature is in an experimental state, so it can be modified > >> in any way, I think. > >> > >> +1 - to accept removing MVVC feature from public API > >> 0 - don't care either way > >> -1 - do not accept removing API (explain why) > >> > >> The vote will hold for 7 days and will end on Wednesday, December 16th at > >> 19:00 UTC: > >> > >> https://www.timeanddate.com/countdown/generic?iso=20201216T19&p0=1440&font=cursive > >> > >> [1] > >> > >> http://apache-ignite-developers.2346864.n4.nabble.com/Mark-MVCC-with-IgniteExperimental-td45669.html > >> [2] > >> > >> http://apache-ignite-developers.2346864.n4.nabble.com/Disable-MVCC-test-suites-td50416.html > >> [3] > >> > >> http://apache-ignite-developers.2346864.n4.nabble.com/Mark-MVCC-with-IgniteExperimental-tp45669p45727.html > >> [4] > >> > >> http://apache-ignite-developers.2346864.n4.nabble.com/Mark-MVCC-with-IgniteExperimental-tp45669p45716.html > >> [5] > >> > >> http://apache-ignite-developers.2346864.n4.nabble.com/Mark-MVCC-with-IgniteExperimental-tp45669p45714.html > >> > >> Thanks, > >> Slava. > >> >
Re: Removing MVCC public API
+1 08.12.2020 22:38, Andrey Gura пишет: +1 On Tue, Dec 8, 2020 at 10:02 PM Nikolay Izhikov wrote: +1 8 дек. 2020 г., в 21:54, Valentin Kulichenko написал(а): +1 On Tue, Dec 8, 2020 at 8:31 AM Вячеслав Коптилин wrote: Hello Igniters, I want to start voting on removing the public API (and eventually all unused parts) related to the MVCC feature. This topic has already been discussed many times (at least, [1], [2]) and the community has agreed the feature implementation must be reapproached, because using coordinator node for transactions ordering and 2pc protocol is slow by design and will not scale well. [3] Moreover, the current implementation has critical issues [4], not supported by the community, and not well tested at all. Removing the public API first will allow us to clean up the code later step by step without rushing and keep intact useful improvements that are already in use or can be reused for other parts in the future. For instance, partition counters implementation is already adapted to fix tx caches protocol issues [5]. The future of MVCC is unclear for now, but, definitely, this feature is useful for a lot of user scenarios and can be scheduled for later Ignite versions. Also, the MVCC feature is in an experimental state, so it can be modified in any way, I think. +1 - to accept removing MVVC feature from public API 0 - don't care either way -1 - do not accept removing API (explain why) The vote will hold for 7 days and will end on Wednesday, December 16th at 19:00 UTC: https://www.timeanddate.com/countdown/generic?iso=20201216T19&p0=1440&font=cursive [1] http://apache-ignite-developers.2346864.n4.nabble.com/Mark-MVCC-with-IgniteExperimental-td45669.html [2] http://apache-ignite-developers.2346864.n4.nabble.com/Disable-MVCC-test-suites-td50416.html [3] http://apache-ignite-developers.2346864.n4.nabble.com/Mark-MVCC-with-IgniteExperimental-tp45669p45727.html [4] http://apache-ignite-developers.2346864.n4.nabble.com/Mark-MVCC-with-IgniteExperimental-tp45669p45716.html [5] http://apache-ignite-developers.2346864.n4.nabble.com/Mark-MVCC-with-IgniteExperimental-tp45669p45714.html Thanks, Slava.
Re: Removing MVCC public API
+1 On Tue, Dec 8, 2020 at 11:22 PM Igor Seliverstov wrote: > +1 > > 08.12.2020 22:38, Andrey Gura пишет: > > +1 > > > > On Tue, Dec 8, 2020 at 10:02 PM Nikolay Izhikov > wrote: > >> +1 > >> > >>> 8 дек. 2020 г., в 21:54, Valentin Kulichenko < > valentin.kuliche...@gmail.com> написал(а): > >>> > >>> +1 > >>> > >>> On Tue, Dec 8, 2020 at 8:31 AM Вячеслав Коптилин < > slava.kopti...@gmail.com> > >>> wrote: > >>> > Hello Igniters, > > I want to start voting on removing the public API (and eventually all > unused parts) related to the MVCC feature. > > This topic has already been discussed many times (at least, [1], [2]) > and > the community has agreed the feature implementation must be > reapproached, > because using coordinator node for transactions ordering and 2pc > protocol > is slow by design and will not scale well. [3] > > Moreover, the current implementation has critical issues [4], not > supported > by the community, and not well tested at all. > > Removing the public API first will allow us to clean up the code > later step > by step without rushing and keep intact useful improvements that are > already in use or can be reused for other parts in the future. > For instance, partition counters implementation is already adapted to > fix > tx caches protocol issues [5]. > > The future of MVCC is unclear for now, but, definitely, this feature > is > useful for a lot of user scenarios and can be scheduled for later > Ignite > versions. > Also, the MVCC feature is in an experimental state, so it can be > modified > in any way, I think. > > +1 - to accept removing MVVC feature from public API > 0 - don't care either way > -1 - do not accept removing API (explain why) > > The vote will hold for 7 days and will end on Wednesday, December > 16th at > 19:00 UTC: > > > https://www.timeanddate.com/countdown/generic?iso=20201216T19&p0=1440&font=cursive > > [1] > > > http://apache-ignite-developers.2346864.n4.nabble.com/Mark-MVCC-with-IgniteExperimental-td45669.html > [2] > > > http://apache-ignite-developers.2346864.n4.nabble.com/Disable-MVCC-test-suites-td50416.html > [3] > > > http://apache-ignite-developers.2346864.n4.nabble.com/Mark-MVCC-with-IgniteExperimental-tp45669p45727.html > [4] > > > http://apache-ignite-developers.2346864.n4.nabble.com/Mark-MVCC-with-IgniteExperimental-tp45669p45716.html > [5] > > > http://apache-ignite-developers.2346864.n4.nabble.com/Mark-MVCC-with-IgniteExperimental-tp45669p45714.html > > Thanks, > Slava. > > -- Best regards, Andrey V. Mashenkov
[jira] [Created] (IGNITE-13830) SQL COPY command: implement table and column names escaping
Roman created IGNITE-13830: -- Summary: SQL COPY command: implement table and column names escaping Key: IGNITE-13830 URL: https://issues.apache.org/jira/browse/IGNITE-13830 Project: Ignite Issue Type: Bug Components: sql Affects Versions: 2.9 Reporter: Roman It's impossible to escape table names and column names with [`] sign using sql copy command. It terminates with sql syntax errors if there are column names like *year* or *service* in the schema or if the column name contains the dash sign. -- This message was sent by Atlassian Jira (v8.3.4#803005)
Re: Removing MVCC public API
+1 08.12.2020, 23:47, "Andrey Mashenkov" : > +1 > > On Tue, Dec 8, 2020 at 11:22 PM Igor Seliverstov > wrote: > >> +1 >> >> 08.12.2020 22:38, Andrey Gura пишет: >> > +1 >> > >> > On Tue, Dec 8, 2020 at 10:02 PM Nikolay Izhikov >> wrote: >> >> +1 >> >> >> >>> 8 дек. 2020 г., в 21:54, Valentin Kulichenko < >> valentin.kuliche...@gmail.com> написал(а): >> >>> >> >>> +1 >> >>> >> >>> On Tue, Dec 8, 2020 at 8:31 AM Вячеслав Коптилин < >> slava.kopti...@gmail.com> >> >>> wrote: >> >>> >> Hello Igniters, >> >> I want to start voting on removing the public API (and eventually all >> unused parts) related to the MVCC feature. >> >> This topic has already been discussed many times (at least, [1], [2]) >> and >> the community has agreed the feature implementation must be >> reapproached, >> because using coordinator node for transactions ordering and 2pc >> protocol >> is slow by design and will not scale well. [3] >> >> Moreover, the current implementation has critical issues [4], not >> supported >> by the community, and not well tested at all. >> >> Removing the public API first will allow us to clean up the code >> later step >> by step without rushing and keep intact useful improvements that are >> already in use or can be reused for other parts in the future. >> For instance, partition counters implementation is already adapted to >> fix >> tx caches protocol issues [5]. >> >> The future of MVCC is unclear for now, but, definitely, this feature >> is >> useful for a lot of user scenarios and can be scheduled for later >> Ignite >> versions. >> Also, the MVCC feature is in an experimental state, so it can be >> modified >> in any way, I think. >> >> +1 - to accept removing MVVC feature from public API >> 0 - don't care either way >> -1 - do not accept removing API (explain why) >> >> The vote will hold for 7 days and will end on Wednesday, December >> 16th at >> 19:00 UTC: >> >> >> >> https://www.timeanddate.com/countdown/generic?iso=20201216T19&p0=1440&font=cursive >> >> [1] >> >> >> >> http://apache-ignite-developers.2346864.n4.nabble.com/Mark-MVCC-with-IgniteExperimental-td45669.html >> [2] >> >> >> >> http://apache-ignite-developers.2346864.n4.nabble.com/Disable-MVCC-test-suites-td50416.html >> [3] >> >> >> >> http://apache-ignite-developers.2346864.n4.nabble.com/Mark-MVCC-with-IgniteExperimental-tp45669p45727.html >> [4] >> >> >> >> http://apache-ignite-developers.2346864.n4.nabble.com/Mark-MVCC-with-IgniteExperimental-tp45669p45716.html >> [5] >> >> >> >> http://apache-ignite-developers.2346864.n4.nabble.com/Mark-MVCC-with-IgniteExperimental-tp45669p45714.html >> >> Thanks, >> Slava. >> > > -- > Best regards, > Andrey V. Mashenkov
Re: Removing MVCC public API
+1 ср, 9 дек. 2020 г. в 08:29, ткаленко кирилл : > > +1 > > > 08.12.2020, 23:47, "Andrey Mashenkov" : > > +1 > > > > On Tue, Dec 8, 2020 at 11:22 PM Igor Seliverstov > > wrote: > > > >> +1 > >> > >> 08.12.2020 22:38, Andrey Gura пишет: > >> > +1 > >> > > >> > On Tue, Dec 8, 2020 at 10:02 PM Nikolay Izhikov > >> wrote: > >> >> +1 > >> >> > >> >>> 8 дек. 2020 г., в 21:54, Valentin Kulichenko < > >> valentin.kuliche...@gmail.com> написал(а): > >> >>> > >> >>> +1 > >> >>> > >> >>> On Tue, Dec 8, 2020 at 8:31 AM Вячеслав Коптилин < > >> slava.kopti...@gmail.com> > >> >>> wrote: > >> >>> > >> Hello Igniters, > >> > >> I want to start voting on removing the public API (and eventually all > >> unused parts) related to the MVCC feature. > >> > >> This topic has already been discussed many times (at least, [1], [2]) > >> and > >> the community has agreed the feature implementation must be > >> reapproached, > >> because using coordinator node for transactions ordering and 2pc > >> protocol > >> is slow by design and will not scale well. [3] > >> > >> Moreover, the current implementation has critical issues [4], not > >> supported > >> by the community, and not well tested at all. > >> > >> Removing the public API first will allow us to clean up the code > >> later step > >> by step without rushing and keep intact useful improvements that are > >> already in use or can be reused for other parts in the future. > >> For instance, partition counters implementation is already adapted to > >> fix > >> tx caches protocol issues [5]. > >> > >> The future of MVCC is unclear for now, but, definitely, this feature > >> is > >> useful for a lot of user scenarios and can be scheduled for later > >> Ignite > >> versions. > >> Also, the MVCC feature is in an experimental state, so it can be > >> modified > >> in any way, I think. > >> > >> +1 - to accept removing MVVC feature from public API > >> 0 - don't care either way > >> -1 - do not accept removing API (explain why) > >> > >> The vote will hold for 7 days and will end on Wednesday, December > >> 16th at > >> 19:00 UTC: > >> > >> > >> > >> https://www.timeanddate.com/countdown/generic?iso=20201216T19&p0=1440&font=cursive > >> > >> [1] > >> > >> > >> > >> http://apache-ignite-developers.2346864.n4.nabble.com/Mark-MVCC-with-IgniteExperimental-td45669.html > >> [2] > >> > >> > >> > >> http://apache-ignite-developers.2346864.n4.nabble.com/Disable-MVCC-test-suites-td50416.html > >> [3] > >> > >> > >> > >> http://apache-ignite-developers.2346864.n4.nabble.com/Mark-MVCC-with-IgniteExperimental-tp45669p45727.html > >> [4] > >> > >> > >> > >> http://apache-ignite-developers.2346864.n4.nabble.com/Mark-MVCC-with-IgniteExperimental-tp45669p45716.html > >> [5] > >> > >> > >> > >> http://apache-ignite-developers.2346864.n4.nabble.com/Mark-MVCC-with-IgniteExperimental-tp45669p45714.html > >> > >> Thanks, > >> Slava. > >> > > > > -- > > Best regards, > > Andrey V. Mashenkov -- Best wishes, Amelchev Nikita
Re[2]: [DISCUSSION] Modules organization in Ignite 3
Hello Nikolay, if i find out introduced features structure in some project, i would prefer to choose different one ) > >> >>>Hello, Alexey. >>> >>>Think we can extend our @IgniteExperimental annotation. >>> >>>`@IgniteExperimental` - mark features that are truly experimental and can be >>>completely removed in future releases. >>>`@NotRecommended` - mark features that widely adopted by the users but >>>implemented wrong or have known issues that can’t be fixed. >>>`@NotStable` - mark features supported by community but API not stable and >>>can be reworked in the next release. >>>`@Stable` - mark features that are completely OK and here to stay. >>> >>>We should output notes about these annotations in the JavaDoc, also. >>>What do you think? >>> >>> 8 дек. 2020 г., в 12:49, Alexey Goncharuk < alexey.goncha...@gmail.com > написал(а): Igniters, I want to tackle the topic of modules structure in Ignite 3. So far, the modules in Ignite are mostly defined intuitively which leads to some complications: - Ignite public API is separated from the rest of the code only by package name. This leads to private classes leaking to public API which is very hard to catch even during the review process (we missed a bunch of such leaks for new metrics API [1] and I remember this happening for almost every SPI) - Classes from 'internal' packages are considered to be 'free for grabs' in every place of the code. This leads to tight coupling and abstraction leakage in the code. An example of such a case - an often cast of WALPointer to FileWALPointer, so that the community decided to get rid of the WALPointer interface altogether [2] - Overall code complexity. Because of the lack of inter-module interaction rules, we are free to add new methods and callbacks to any class, which leads to duplicating entities and verbose interfaces. A good example of this is the clear duplication of methods in IgniteCacheOffheapManager and IgniteCacheOffheapManager.DataStore [3] I think we need to work out some rules that will help us define and control both Ignite public API and module internal API which still defines a clear contract for other modules. Some ideas: - Perhaps we can move all user public classed and interfaces to an Ignite-API module which will have no dependencies on implementation modules. This will prevent private classes from leaking to the API module. - We need somehow define which classes from a module are exposed to other modules, and which classes are left for module-private usage. Maybe Java's jigsaw will help us here, but maybe we will be ok with just more strict java access modifiers usage :) The idea here is that a module should never touch a dependent module's private classes, ever. The exported classes and interfaces are still free to be modified between releases, as long as it is not a user public API. - A module should be logically complete, thus it may be beneficial if module name matches with the code package it provides (e.g. configuration -> org.apache.ignite.configuration, replication -> org.apache.ignite.replication, raft->org.apache.ignite.raft, etc) Any other principles/rules we can apply to make the code structure more concise? Thoughts? --AG [1] https://issues.apache.org/jira/browse/IGNITE-12552 [2] https://issues.apache.org/jira/browse/IGNITE-13513 [3] https://issues.apache.org/jira/browse/IGNITE-13220 >> >> >> >>
Re: Removing MVCC public API
+1 > On 9 Dec 2020, at 09:39, Nikita Amelchev wrote: > > +1 > > ср, 9 дек. 2020 г. в 08:29, ткаленко кирилл : >> >> +1 >> >> >> 08.12.2020, 23:47, "Andrey Mashenkov" : >>> +1 >>> >>> On Tue, Dec 8, 2020 at 11:22 PM Igor Seliverstov >>> wrote: >>> +1 08.12.2020 22:38, Andrey Gura пишет: > +1 > > On Tue, Dec 8, 2020 at 10:02 PM Nikolay Izhikov wrote: >> +1 >> >>> 8 дек. 2020 г., в 21:54, Valentin Kulichenko < valentin.kuliche...@gmail.com> написал(а): >>> >>> +1 >>> >>> On Tue, Dec 8, 2020 at 8:31 AM Вячеслав Коптилин < slava.kopti...@gmail.com> >>> wrote: >>> Hello Igniters, I want to start voting on removing the public API (and eventually all unused parts) related to the MVCC feature. This topic has already been discussed many times (at least, [1], [2]) and the community has agreed the feature implementation must be reapproached, because using coordinator node for transactions ordering and 2pc protocol is slow by design and will not scale well. [3] Moreover, the current implementation has critical issues [4], not supported by the community, and not well tested at all. Removing the public API first will allow us to clean up the code later step by step without rushing and keep intact useful improvements that are already in use or can be reused for other parts in the future. For instance, partition counters implementation is already adapted to fix tx caches protocol issues [5]. The future of MVCC is unclear for now, but, definitely, this feature is useful for a lot of user scenarios and can be scheduled for later Ignite versions. Also, the MVCC feature is in an experimental state, so it can be modified in any way, I think. +1 - to accept removing MVVC feature from public API 0 - don't care either way -1 - do not accept removing API (explain why) The vote will hold for 7 days and will end on Wednesday, December 16th at 19:00 UTC: https://www.timeanddate.com/countdown/generic?iso=20201216T19&p0=1440&font=cursive [1] http://apache-ignite-developers.2346864.n4.nabble.com/Mark-MVCC-with-IgniteExperimental-td45669.html [2] http://apache-ignite-developers.2346864.n4.nabble.com/Disable-MVCC-test-suites-td50416.html [3] http://apache-ignite-developers.2346864.n4.nabble.com/Mark-MVCC-with-IgniteExperimental-tp45669p45727.html [4] http://apache-ignite-developers.2346864.n4.nabble.com/Mark-MVCC-with-IgniteExperimental-tp45669p45716.html [5] http://apache-ignite-developers.2346864.n4.nabble.com/Mark-MVCC-with-IgniteExperimental-tp45669p45714.html Thanks, Slava. >>> >>> -- >>> Best regards, >>> Andrey V. Mashenkov > > > > -- > Best wishes, > Amelchev Nikita
Re: Re[2]: [DISCUSSION] Modules organization in Ignite 3
Conversation shifted into an unintended direction, but I agree. I think that if API can (or will) be changed then it should be deprecated. For that we can introduce @IgniteDeprecated that will contain Ignite version when API is planned to be removed. Otherwise it's either stable or experimental. Having officially "unstable" features doesn't sound good for product reputation. As for the modularization - I'm all for this idea. If we don't force ourselves to organize code properly then we'll end up with the same problems as we have in the current code base. And this way there's a hope of having good tests that can be completed in minutes, not hours. At least new ones. BTW, did we have any discussions about dependency injection and all this stuff? Seems like a related topic to me. ср, 9 дек. 2020 г. в 09:47, Zhenya Stanilovsky : > > > Hello Nikolay, if i find out introduced features structure in some > project, i would prefer to choose different one ) > > > > >> > >>>Hello, Alexey. > >>> > >>>Think we can extend our @IgniteExperimental annotation. > >>> > >>>`@IgniteExperimental` - mark features that are truly experimental and > can be completely removed in future releases. > >>>`@NotRecommended` - mark features that widely adopted by the users but > implemented wrong or have known issues that can’t be fixed. > >>>`@NotStable` - mark features supported by community but API not stable > and can be reworked in the next release. > >>>`@Stable` - mark features that are completely OK and here to stay. > >>> > >>>We should output notes about these annotations in the JavaDoc, also. > >>>What do you think? > >>> > >>> > 8 дек. 2020 г., в 12:49, Alexey Goncharuk < > alexey.goncha...@gmail.com > написал(а): > > Igniters, > > I want to tackle the topic of modules structure in Ignite 3. So far, > the > modules in Ignite are mostly defined intuitively which leads to some > complications: > > - Ignite public API is separated from the rest of the code only by > package name. This leads to private classes leaking to public API > which is > very hard to catch even during the review process (we missed a bunch > of > such leaks for new metrics API [1] and I remember this happening for > almost > every SPI) > - Classes from 'internal' packages are considered to be 'free for > grabs' > in every place of the code. This leads to tight coupling and > abstraction > leakage in the code. An example of such a case - an often cast of > WALPointer to FileWALPointer, so that the community decided to get > rid of > the WALPointer interface altogether [2] > - Overall code complexity. Because of the lack of inter-module > interaction rules, we are free to add new methods and callbacks to any > class, which leads to duplicating entities and verbose interfaces. A > good > example of this is the clear duplication of methods in > IgniteCacheOffheapManager and IgniteCacheOffheapManager.DataStore [3] > > I think we need to work out some rules that will help us define and > control > both Ignite public API and module internal API which still defines a > clear > contract for other modules. Some ideas: > > - Perhaps we can move all user public classed and interfaces to an > Ignite-API module which will have no dependencies on implementation > modules. This will prevent private classes from leaking to the API > module. > - We need somehow define which classes from a module are exposed to > other modules, and which classes are left for module-private usage. > Maybe > Java's jigsaw will help us here, but maybe we will be ok with just > more > strict java access modifiers usage :) The idea here is that a module > should > never touch a dependent module's private classes, ever. The exported > classes and interfaces are still free to be modified between > releases, as > long as it is not a user public API. > - A module should be logically complete, thus it may be beneficial if > module name matches with the code package it provides (e.g. > configuration > -> org.apache.ignite.configuration, replication -> > org.apache.ignite.replication, raft->org.apache.ignite.raft, etc) > > Any other principles/rules we can apply to make the code structure > more > concise? Thoughts? > > --AG > > [1] https://issues.apache.org/jira/browse/IGNITE-12552 > [2] https://issues.apache.org/jira/browse/IGNITE-13513 > [3] https://issues.apache.org/jira/browse/IGNITE-13220 > >> > >> > >> > >> -- Sincerely yours, Ivan Bessonov
Re: [DISCUSSION] Java 11 for Ignite 3.0 development
This is an awesome idea. Honestly, I can't come up with strong technical arguments for Java 11 as a source level, I had no chance to work with it long enough, but it feels like a proper time to move to a "modern" technology. Subjectively I can say that Java 11 has a lot of good optimization and Ignite should run better on it. So it makes no sense to compile for 8 but recommend 11, you know. вт, 8 дек. 2020 г. в 21:16, Данилов Семён : > +1 for sure. AFAIK, the only thing holding us back from using Java 11 is > the dominance of Java 8, but I'm sure that by the time Ignite 3 is GA, > there will be much fewer Java 8 users if any significant number at all. By > the by, Ignite's sources need minimal effort to be able to be compiled with > Java 11 as a target. > > 08.12.2020, 15:00, "Nikolay Izhikov" : > > +1 for using java 11. > > > >> 8 дек. 2020 г., в 13:18, ткаленко кирилл > написал(а): > >> > >> +1 > >> > >> 08.12.2020, 12:48, "Philipp Masharov" : > >>> Hello! > >>> > >>> Andrey's arguments are solid. > >>> > >>> On Tue, Dec 8, 2020 at 12:23 PM Pavel Tupitsyn > wrote: > >>> > +1, Java 11 seems to be the only right choice at the moment. > > On Tue, Dec 8, 2020 at 12:08 PM Alexey Zinoviev < > zaleslaw@gmail.com> > wrote: > > > I totally support Java 11 for development. It's time to go forward > > > > вт, 8 дек. 2020 г. в 11:40, Andrey Gura : > > > > > Igniters, > > > > > > We already had some discussion about using modern Java versions > for > > > Ignite 3.0 development [1] but we still don't have consensus. > > > As I see from this discussion the strongest argument for Java > 11 is > > > the fact that Java 11 is the latest LTS release which will have > > > premier support until September 2023. So I don't see any reason > for > > > preferring any other version of Java at this moment. > > > > > > The purpose of this thread is to gather opinions about using > Java 11 > > > in the Ignite 3.0 project and, eventually, reach a consensus on > this. > > > > > > I want to share my several arguments in favor of abandoning > Java 8 and > > > preferring Java 11: > > > > > > * Java 8 has gone through the End of Public Updates process for > legacy > > > releases. So it doesn't make sense to start new development on > Java 8. > > > > > > * Java 9+ brings Jigsaw modularization which allows us to have > more > > > fine-grained structure of Ignite modules and APIs in the future. > > > > > > * Ignite actively uses Unsafe functionality which, firstly, > isn't > > > public, and secondly, leads to problems with running Ignite > under Java > > > 9+ (modularization which requires dozens of command-line > options in > > > order to forcibly export corresponding packages) and GraalVM. > Such a > > > situation could be described as bad user experience and we > should fix > > > it. Var handles [2] could be used for solving described > problems. > > > > > > * Java 9+ introduces Flight Recorder API [3] which could be > used in > > > the Ignite project for lightweight profiling of internal > processes. > > > > > > Please, share your opinions, objections and ideas about this > topic. I > > > hope we will not have serious disagreements and the consensus > will be > > > reached quickly. > > > > > > > > > 1. > > > > > > > http://apache-ignite-developers.2346864.n4.nabble.com/DISCUSS-Ignite-3-0-development-approach-tp49922p50295.html > > > 2. > > > > > > > https://docs.oracle.com/javase/9/docs/api/java/lang/invoke/VarHandle.html > > > 3. > > > > > > > https://docs.oracle.com/en/java/javase/11/docs/api/jdk.jfr/jdk/jfr/FlightRecorder.html > > > > > > -- Sincerely yours, Ivan Bessonov
Re: Removing MVCC public API
+1 ср, 9 дек. 2020 г. в 10:03, Petr Ivanov : > +1 > > > > On 9 Dec 2020, at 09:39, Nikita Amelchev wrote: > > > > +1 > > > > ср, 9 дек. 2020 г. в 08:29, ткаленко кирилл : > >> > >> +1 > >> > >> > >> 08.12.2020, 23:47, "Andrey Mashenkov" : > >>> +1 > >>> > >>> On Tue, Dec 8, 2020 at 11:22 PM Igor Seliverstov > > >>> wrote: > >>> > +1 > > 08.12.2020 22:38, Andrey Gura пишет: > > +1 > > > > On Tue, Dec 8, 2020 at 10:02 PM Nikolay Izhikov > > wrote: > >> +1 > >> > >>> 8 дек. 2020 г., в 21:54, Valentin Kulichenko < > valentin.kuliche...@gmail.com> написал(а): > >>> > >>> +1 > >>> > >>> On Tue, Dec 8, 2020 at 8:31 AM Вячеслав Коптилин < > slava.kopti...@gmail.com> > >>> wrote: > >>> > Hello Igniters, > > I want to start voting on removing the public API (and eventually > all > unused parts) related to the MVCC feature. > > This topic has already been discussed many times (at least, [1], > [2]) > and > the community has agreed the feature implementation must be > reapproached, > because using coordinator node for transactions ordering and 2pc > protocol > is slow by design and will not scale well. [3] > > Moreover, the current implementation has critical issues [4], not > supported > by the community, and not well tested at all. > > Removing the public API first will allow us to clean up the code > later step > by step without rushing and keep intact useful improvements that > are > already in use or can be reused for other parts in the future. > For instance, partition counters implementation is already > adapted to > fix > tx caches protocol issues [5]. > > The future of MVCC is unclear for now, but, definitely, this > feature > is > useful for a lot of user scenarios and can be scheduled for later > Ignite > versions. > Also, the MVCC feature is in an experimental state, so it can be > modified > in any way, I think. > > +1 - to accept removing MVVC feature from public API > 0 - don't care either way > -1 - do not accept removing API (explain why) > > The vote will hold for 7 days and will end on Wednesday, December > 16th at > 19:00 UTC: > > > > https://www.timeanddate.com/countdown/generic?iso=20201216T19&p0=1440&font=cursive > > [1] > > > > http://apache-ignite-developers.2346864.n4.nabble.com/Mark-MVCC-with-IgniteExperimental-td45669.html > [2] > > > > http://apache-ignite-developers.2346864.n4.nabble.com/Disable-MVCC-test-suites-td50416.html > [3] > > > > http://apache-ignite-developers.2346864.n4.nabble.com/Mark-MVCC-with-IgniteExperimental-tp45669p45727.html > [4] > > > > http://apache-ignite-developers.2346864.n4.nabble.com/Mark-MVCC-with-IgniteExperimental-tp45669p45716.html > [5] > > > > http://apache-ignite-developers.2346864.n4.nabble.com/Mark-MVCC-with-IgniteExperimental-tp45669p45714.html > > Thanks, > Slava. > > >>> > >>> -- > >>> Best regards, > >>> Andrey V. Mashenkov > > > > > > > > -- > > Best wishes, > > Amelchev Nikita > > -- Best regards, Alexei Scherbakov
Re: [DISCUSSION] Java 11 for Ignite 3.0 development
I think we should move forward, so java11 seems like a proper choice for 3. ср, 9 дек. 2020 г. в 10:17, Ivan Bessonov : > This is an awesome idea. > > Honestly, I can't come up with strong technical arguments for Java 11 as a > source level, I had no chance to work with it long enough, but it feels > like a > proper time to move to a "modern" technology. Subjectively I can say that > Java 11 has a lot of good optimization and Ignite should run better on it. > So > it makes no sense to compile for 8 but recommend 11, you know. > > вт, 8 дек. 2020 г. в 21:16, Данилов Семён : > > > +1 for sure. AFAIK, the only thing holding us back from using Java 11 is > > the dominance of Java 8, but I'm sure that by the time Ignite 3 is GA, > > there will be much fewer Java 8 users if any significant number at all. > By > > the by, Ignite's sources need minimal effort to be able to be compiled > with > > Java 11 as a target. > > > > 08.12.2020, 15:00, "Nikolay Izhikov" : > > > +1 for using java 11. > > > > > >> 8 дек. 2020 г., в 13:18, ткаленко кирилл > > написал(а): > > >> > > >> +1 > > >> > > >> 08.12.2020, 12:48, "Philipp Masharov" : > > >>> Hello! > > >>> > > >>> Andrey's arguments are solid. > > >>> > > >>> On Tue, Dec 8, 2020 at 12:23 PM Pavel Tupitsyn < > ptupit...@apache.org> > > wrote: > > >>> > > +1, Java 11 seems to be the only right choice at the moment. > > > > On Tue, Dec 8, 2020 at 12:08 PM Alexey Zinoviev < > > zaleslaw@gmail.com> > > wrote: > > > > > I totally support Java 11 for development. It's time to go > forward > > > > > > вт, 8 дек. 2020 г. в 11:40, Andrey Gura : > > > > > > > Igniters, > > > > > > > > We already had some discussion about using modern Java > versions > > for > > > > Ignite 3.0 development [1] but we still don't have consensus. > > > > As I see from this discussion the strongest argument for Java > > 11 is > > > > the fact that Java 11 is the latest LTS release which will > have > > > > premier support until September 2023. So I don't see any > reason > > for > > > > preferring any other version of Java at this moment. > > > > > > > > The purpose of this thread is to gather opinions about using > > Java 11 > > > > in the Ignite 3.0 project and, eventually, reach a consensus > on > > this. > > > > > > > > I want to share my several arguments in favor of abandoning > > Java 8 and > > > > preferring Java 11: > > > > > > > > * Java 8 has gone through the End of Public Updates process > for > > legacy > > > > releases. So it doesn't make sense to start new development on > > Java 8. > > > > > > > > * Java 9+ brings Jigsaw modularization which allows us to have > > more > > > > fine-grained structure of Ignite modules and APIs in the > future. > > > > > > > > * Ignite actively uses Unsafe functionality which, firstly, > > isn't > > > > public, and secondly, leads to problems with running Ignite > > under Java > > > > 9+ (modularization which requires dozens of command-line > > options in > > > > order to forcibly export corresponding packages) and GraalVM. > > Such a > > > > situation could be described as bad user experience and we > > should fix > > > > it. Var handles [2] could be used for solving described > > problems. > > > > > > > > * Java 9+ introduces Flight Recorder API [3] which could be > > used in > > > > the Ignite project for lightweight profiling of internal > > processes. > > > > > > > > Please, share your opinions, objections and ideas about this > > topic. I > > > > hope we will not have serious disagreements and the consensus > > will be > > > > reached quickly. > > > > > > > > > > > > 1. > > > > > > > > > > > > http://apache-ignite-developers.2346864.n4.nabble.com/DISCUSS-Ignite-3-0-development-approach-tp49922p50295.html > > > > 2. > > > > > > > > > > > > https://docs.oracle.com/javase/9/docs/api/java/lang/invoke/VarHandle.html > > > > 3. > > > > > > > > > > > > https://docs.oracle.com/en/java/javase/11/docs/api/jdk.jfr/jdk/jfr/FlightRecorder.html > > > > > > > > > > > > -- > Sincerely yours, > Ivan Bessonov > -- Best regards, Alexei Scherbakov
Re: [DISCUSSION] Modules organization in Ignite 3
Hello, Zhenya, Ivan. > Hello Nikolay, if i find out introduced features structure in some project, i > would prefer to choose different one ) Many, of the real world users disagree with you. Please, take a look at some examples from widely used projects: Kafka - https://github.com/apache/kafka/blob/trunk/clients/src/main/java/org/apache/kafka/common/annotation/InterfaceStability.java#L28 - Stable, Evolving, Unstable Spark - https://github.com/apache/spark/tree/master/common/tags/src/main/java/org/apache/spark/annotation - AlphaComponent, DeveloperApi, Evolving, Experimental, Private, Stable, Unstable > Having officially "unstable" features doesn't sound good for product > reputation. Can’t agree with you. Forcing ourselves to make perfect API from the first try we just put too much pressure on every decision. Every developer making mistakes. The product is evolving and the API too - it’s totally OK. For every new feature time required to be adopted and used in real-world production. I believe, slight API changes is totally fine for early adopters. Moreover, I think, that we should warn our users that some feature is very fresh and can have issues. So, Why Kafka and Spark is good enough to have unstable API and Ignite not? :) > 9 дек. 2020 г., в 10:08, Ivan Bessonov написал(а): > > Conversation shifted into an unintended direction, but I agree. > > I think that if API can (or will) be changed then it should be deprecated. > For that > we can introduce @IgniteDeprecated that will contain Ignite version when > API is planned to be removed. Otherwise it's either stable or experimental. > Having officially "unstable" features doesn't sound good for product > reputation. > > As for the modularization - I'm all for this idea. If we don't force > ourselves to > organize code properly then we'll end up with the same problems as we have > in the current code base. And this way there's a hope of having good tests > that can be completed in minutes, not hours. At least new ones. > > BTW, did we have any discussions about dependency injection and all this > stuff? > Seems like a related topic to me. > > ср, 9 дек. 2020 г. в 09:47, Zhenya Stanilovsky : > >> >> >> Hello Nikolay, if i find out introduced features structure in some >> project, i would prefer to choose different one ) >> >>> > Hello, Alexey. > > Think we can extend our @IgniteExperimental annotation. > > `@IgniteExperimental` - mark features that are truly experimental and >> can be completely removed in future releases. > `@NotRecommended` - mark features that widely adopted by the users but >> implemented wrong or have known issues that can’t be fixed. > `@NotStable` - mark features supported by community but API not stable >> and can be reworked in the next release. > `@Stable` - mark features that are completely OK and here to stay. > > We should output notes about these annotations in the JavaDoc, also. > What do you think? > > >> 8 дек. 2020 г., в 12:49, Alexey Goncharuk < >> alexey.goncha...@gmail.com > написал(а): >> >> Igniters, >> >> I want to tackle the topic of modules structure in Ignite 3. So far, >> the >> modules in Ignite are mostly defined intuitively which leads to some >> complications: >> >> - Ignite public API is separated from the rest of the code only by >> package name. This leads to private classes leaking to public API >> which is >> very hard to catch even during the review process (we missed a bunch >> of >> such leaks for new metrics API [1] and I remember this happening for >> almost >> every SPI) >> - Classes from 'internal' packages are considered to be 'free for >> grabs' >> in every place of the code. This leads to tight coupling and >> abstraction >> leakage in the code. An example of such a case - an often cast of >> WALPointer to FileWALPointer, so that the community decided to get >> rid of >> the WALPointer interface altogether [2] >> - Overall code complexity. Because of the lack of inter-module >> interaction rules, we are free to add new methods and callbacks to any >> class, which leads to duplicating entities and verbose interfaces. A >> good >> example of this is the clear duplication of methods in >> IgniteCacheOffheapManager and IgniteCacheOffheapManager.DataStore [3] >> >> I think we need to work out some rules that will help us define and >> control >> both Ignite public API and module internal API which still defines a >> clear >> contract for other modules. Some ideas: >> >> - Perhaps we can move all user public classed and interfaces to an >> Ignite-API module which will have no dependencies on implementation >> modules. This will prevent private classes from leaking to the API >> module. >> - We need somehow define which classes from a module are exposed to >> oth
