> Nikolay, maybe is it better to discuss your question on a separate topic?
> Because looks like it is a pretty discussable topic.
Sorry for interrupting original discussion.
For me it was a overlapping topic in the beginning.
Agree that we we should have separate thread.
> 9 дек. 2020 г., в 16:15, Anton Kalashnikov <kaa....@yandex.ru> написал(а):
>
> Hello,
>
> I totally agree that we should start to think about the module organization.
> I suggest making the new confluence page where we define new rules on how to
> develop modules.
> In my opinion, at least, the following topics should be covered there(it
> makes sense to discuss every topic separately, not here):
> * In which cases new modules are required
> * The naming of modules, packages
> * Class dependency management ( inversion of control, no more context)
> * Test organization ( module contains only unit, module tests. All
> integration tests are in the extra module)
> * Feature/Module lifecycle - experimental, stable, unstable, deprecated
>
>
> Let's get back to the original topic. I agree with the package naming rule:
> if the module name is configuration the package name should be
> org.apache.ignite.configuration and only after that any other subpackages.
> Also, I don't sure that we need ignite- prefix in the module name because it
> doesn't have any extra information:
>
> <groupId>org.apache.ignite</groupId>
> <artifactId>ignite-configuration</artifactId>
>
> we don't lose anything if convert it to
>
> <groupId>org.apache.ignite</groupId>
> <artifactId>configuration</artifactId>
>
>
> I also hope that jigsaw can help us somehow with class visibility.
> But if not we can take agreement that for example 'internal' package -
> shouldn't be touched outside of the module -
> of course, using the proper class access level is also the solution(where it
> is possible)
> org.apache.ignite.configuration.Configurator // it has access to the internal
> package but it shouldn't return any class from the internal package only from
> the public one.
> org.apache.ignite.configuration.DynamicProperty //interface
> org.apache.ignite.configuration.internal.properties.IntDynamicProperty
>
>
> User API makes sense only for one end module - ignite(or ignite-core) which
> depends on all other modules
> and doing some integration(adapters) and provide final API for the user.
> So I agree that separated module Ignite-API with zero dependencies will be a
> good solution.
>
> configuration module:
> Configurator.baseline().enabled() -> DynamicProperties
>
> ignite-api module:
> BaselineConfiguration.enabled() -> boolean //interface
>
> ignite module:
> BaselineConfigurationImpl implements BaselineConfiguration{
> Configurator configurator;
> public boolean enabled(){
> return configurator.baseline().enabled().value();
> }
> }
>
> So maybe my example is not so good. But I want to show that end-user API will
> be defined only in ignite-api
> and you need to adapt it in ignite module which leads to some
> overhead(especially in my example)
> but it makes development pretty manageable/predictable -
> you can easily implement a new module without any worries that user starts to
> use it.
> It will be available only after making changes in ignite-api.
> The major advantage here is the small size of ignite-api which allows to
> carefully review every change
> which allows keeping ignite API in better quality(I hope at least)
>
> Nikolay, maybe is it better to discuss your question on a separate topic?
> Because looks like it is a pretty discussable topic.
>
> --
> Best regards,
> Anton Kalashnikov
>
>
>
> 09.12.2020, 10:31, "Nikolay Izhikov" <nizhi...@apache.org>:
>> Hello, Zhenya, Ivan.
>>
>>> Hello Nikolay, if i find out introduced features structure in some
>>> project, i would prefer to choose different one )
>>
>> Many, of the real world users disagree with you.
>> Please, take a look at some examples from widely used projects:
>>
>> Kafka -
>> https://github.com/apache/kafka/blob/trunk/clients/src/main/java/org/apache/kafka/common/annotation/InterfaceStability.java#L28
>> - Stable, Evolving, Unstable
>>
>> Spark -
>> https://github.com/apache/spark/tree/master/common/tags/src/main/java/org/apache/spark/annotation
>> - AlphaComponent, DeveloperApi, Evolving, Experimental, Private,
>> Stable, Unstable
>>
>>> Having officially "unstable" features doesn't sound good for product
>>> reputation.
>>
>> Can’t agree with you.
>>
>> Forcing ourselves to make perfect API from the first try we just put too
>> much pressure on every decision.
>> Every developer making mistakes.
>> The product is evolving and the API too - it’s totally OK.
>>
>> For every new feature time required to be adopted and used in real-world
>> production.
>> I believe, slight API changes is totally fine for early adopters.
>> Moreover, I think, that we should warn our users that some feature is very
>> fresh and can have issues.
>>
>> So, Why Kafka and Spark is good enough to have unstable API and Ignite not?
>> :)
>>
>>> 9 дек. 2020 г., в 10:08, Ivan Bessonov <bessonov...@gmail.com> написал(а):
>>>
>>> Conversation shifted into an unintended direction, but I agree.
>>>
>>> I think that if API can (or will) be changed then it should be deprecated.
>>> For that
>>> we can introduce @IgniteDeprecated that will contain Ignite version when
>>> API is planned to be removed. Otherwise it's either stable or experimental.
>>> Having officially "unstable" features doesn't sound good for product
>>> reputation.
>>>
>>> As for the modularization - I'm all for this idea. If we don't force
>>> ourselves to
>>> organize code properly then we'll end up with the same problems as we have
>>> in the current code base. And this way there's a hope of having good tests
>>> that can be completed in minutes, not hours. At least new ones.
>>>
>>> BTW, did we have any discussions about dependency injection and all this
>>> stuff?
>>> Seems like a related topic to me.
>>>
>>> ср, 9 дек. 2020 г. в 09:47, Zhenya Stanilovsky
>>> <arzamas...@mail.ru.invalid>:
>>>
>>>> Hello Nikolay, if i find out introduced features structure in some
>>>> project, i would prefer to choose different one )
>>>>
>>>>>>> Hello, Alexey.
>>>>>>>
>>>>>>> Think we can extend our @IgniteExperimental annotation.
>>>>>>>
>>>>>>> `@IgniteExperimental` - mark features that are truly experimental and
>>>> can be completely removed in future releases.
>>>>>>> `@NotRecommended` - mark features that widely adopted by the users but
>>>> implemented wrong or have known issues that can’t be fixed.
>>>>>>> `@NotStable` - mark features supported by community but API not stable
>>>> and can be reworked in the next release.
>>>>>>> `@Stable` - mark features that are completely OK and here to stay.
>>>>>>>
>>>>>>> We should output notes about these annotations in the JavaDoc, also.
>>>>>>> What do you think?
>>>>>>>
>>>>>>>> 8 дек. 2020 г., в 12:49, Alexey Goncharuk <
>>>> alexey.goncha...@gmail.com > написал(а):
>>>>>>>> Igniters,
>>>>>>>>
>>>>>>>> I want to tackle the topic of modules structure in Ignite 3. So far,
>>>> the
>>>>>>>> modules in Ignite are mostly defined intuitively which leads to some
>>>>>>>> complications:
>>>>>>>>
>>>>>>>> - Ignite public API is separated from the rest of the code only by
>>>>>>>> package name. This leads to private classes leaking to public API
>>>> which is
>>>>>>>> very hard to catch even during the review process (we missed a bunch
>>>> of
>>>>>>>> such leaks for new metrics API [1] and I remember this happening for
>>>> almost
>>>>>>>> every SPI)
>>>>>>>> - Classes from 'internal' packages are considered to be 'free for
>>>> grabs'
>>>>>>>> in every place of the code. This leads to tight coupling and
>>>> abstraction
>>>>>>>> leakage in the code. An example of such a case - an often cast of
>>>>>>>> WALPointer to FileWALPointer, so that the community decided to get
>>>> rid of
>>>>>>>> the WALPointer interface altogether [2]
>>>>>>>> - Overall code complexity. Because of the lack of inter-module
>>>>>>>> interaction rules, we are free to add new methods and callbacks to any
>>>>>>>> class, which leads to duplicating entities and verbose interfaces. A
>>>> good
>>>>>>>> example of this is the clear duplication of methods in
>>>>>>>> IgniteCacheOffheapManager and IgniteCacheOffheapManager.DataStore [3]
>>>>>>>>
>>>>>>>> I think we need to work out some rules that will help us define and
>>>> control
>>>>>>>> both Ignite public API and module internal API which still defines a
>>>> clear
>>>>>>>> contract for other modules. Some ideas:
>>>>>>>>
>>>>>>>> - Perhaps we can move all user public classed and interfaces to an
>>>>>>>> Ignite-API module which will have no dependencies on implementation
>>>>>>>> modules. This will prevent private classes from leaking to the API
>>>> module.
>>>>>>>> - We need somehow define which classes from a module are exposed to
>>>>>>>> other modules, and which classes are left for module-private usage.
>>>> Maybe
>>>>>>>> Java's jigsaw will help us here, but maybe we will be ok with just
>>>> more
>>>>>>>> strict java access modifiers usage :) The idea here is that a module
>>>> should
>>>>>>>> never touch a dependent module's private classes, ever. The exported
>>>>>>>> classes and interfaces are still free to be modified between
>>>> releases, as
>>>>>>>> long as it is not a user public API.
>>>>>>>> - A module should be logically complete, thus it may be beneficial if
>>>>>>>> module name matches with the code package it provides (e.g.
>>>> configuration
>>>>>>>> -> org.apache.ignite.configuration, replication ->
>>>>>>>> org.apache.ignite.replication, raft->org.apache.ignite.raft, etc)
>>>>>>>>
>>>>>>>> Any other principles/rules we can apply to make the code structure
>>>> more
>>>>>>>> concise? Thoughts?
>>>>>>>>
>>>>>>>> --AG
>>>>>>>>
>>>>>>>> [1] https://issues.apache.org/jira/browse/IGNITE-12552
>>>>>>>> [2] https://issues.apache.org/jira/browse/IGNITE-13513
>>>>>>>> [3] https://issues.apache.org/jira/browse/IGNITE-13220
>>>
>>> --
>>> Sincerely yours,
>>> Ivan Bessonov
