Ilya,
I think common sense is our only friend here :) The low number of
dependencies is indeed one of the advantages of Ignite. On the other hand,
the "zero dependency" approach we've been trying to follow for the last
several years sometimes forces us to implement, stabilize and maintain a
whole lot of code that we potentially could reuse from somewhere else.
We should always try to minimize the number of dependencies, especially for
the core part of the product. But we should also reuse existing code where
it is reasonable.
-Val
On Fri, Dec 11, 2020 at 3:26 AM Ilya Kasnacheev <ilya.kasnach...@gmail.com>
wrote:
> Hello!
>
> I think it is a strong side of Apache Ignite that it has a low number of
> JARs and minimal dependencies. It may be less relevant now than 10 years
> ago, but it's still a plus.
>
> Not sure how fast it would evaporate as the number of JARs and dependencies
> grows.
>
> Regards,
> --
> Ilya Kasnacheev
>
>
> чт, 10 дек. 2020 г. в 12:28, Anton Kalashnikov <kaa....@yandex.ru>:
>
> > Ilya, it is a good point about jar name. If it really unresolvable
> problem
> > we should leave modules name as is(I do not insist on renaming it was
> just
> > a proposal)
> >
> > I also not really got you point about 'zillion JARs'. Do you think it
> will
> > be a problem if the number of JARs increases a little? If so can you
> > clarify it with some specific examples?
> >
> > --
> > Best regards,
> > Anton Kalashnikov
> >
> >
> >
> > 09.12.2020, 18:12, "Ilya Kasnacheev" <ilya.kasnach...@gmail.com>:
> > > Hello!
> > >
> > > When you do mvn dependencies:copy, you will now end up with
> > > "ignite-core-X.Y.Z.jar", "ignite-indexing-X.Y.Z.jar"
> > > But if we remove "ignite" from artifact name, user will end up with
> > cryptic
> > > "configuration.jar", "raft.jar", etc, etc.
> > >
> > > Remember that group name is discarded in file names. Therefore, every
> > > project out there keeps stuff like "spring-" or "hibernate-" in
> artifact
> > > name.
> > >
> > > I also hope that our users will not end up with zillion JARs.
> > >
> > > Regards,
> > > --
> > > Ilya Kasnacheev
> > >
> > > ср, 9 дек. 2020 г. в 16:16, Anton Kalashnikov <kaa....@yandex.ru>:
> > >
> > >> Hello,
> > >>
> > >> I totally agree that we should start to think about the module
> > >> organization.
> > >> I suggest making the new confluence page where we define new rules on
> > how
> > >> to develop modules.
> > >> In my opinion, at least, the following topics should be covered
> > there(it
> > >> makes sense to discuss every topic separately, not here):
> > >> * In which cases new modules are required
> > >> * The naming of modules, packages
> > >> * Class dependency management ( inversion of control, no more
> context)
> > >> * Test organization ( module contains only unit, module tests. All
> > >> integration tests are in the extra module)
> > >> * Feature/Module lifecycle - experimental, stable, unstable,
> deprecated
> > >>
> > >> Let's get back to the original topic. I agree with the package naming
> > >> rule:
> > >> if the module name is configuration the package name should be
> > >> org.apache.ignite.configuration and only after that any other
> > subpackages.
> > >> Also, I don't sure that we need ignite- prefix in the module name
> > because
> > >> it doesn't have any extra information:
> > >>
> > >> <groupId>org.apache.ignite</groupId>
> > >> <artifactId>ignite-configuration</artifactId>
> > >>
> > >> we don't lose anything if convert it to
> > >>
> > >> <groupId>org.apache.ignite</groupId>
> > >> <artifactId>configuration</artifactId>
> > >>
> > >> I also hope that jigsaw can help us somehow with class visibility.
> > >> But if not we can take agreement that for example 'internal' package
> -
> > >> shouldn't be touched outside of the module -
> > >> of course, using the proper class access level is also the
> > solution(where
> > >> it is possible)
> > >> org.apache.ignite.configuration.Configurator // it has access to the
> > >> internal package but it shouldn't return any class from the internal
> > >> package only from the public one.
> > >> org.apache.ignite.configuration.DynamicProperty //interface
> > >>
> org.apache.ignite.configuration.internal.properties.IntDynamicProperty
> > >>
> > >> User API makes sense only for one end module - ignite(or ignite-core)
> > >> which depends on all other modules
> > >> and doing some integration(adapters) and provide final API for the
> > user.
> > >> So I agree that separated module Ignite-API with zero dependencies
> > will be
> > >> a good solution.
> > >>
> > >> configuration module:
> > >> Configurator.baseline().enabled() -> DynamicProperties
> > >>
> > >> ignite-api module:
> > >> BaselineConfiguration.enabled() -> boolean //interface
> > >>
> > >> ignite module:
> > >> BaselineConfigurationImpl implements BaselineConfiguration{
> > >> Configurator configurator;
> > >> public boolean enabled(){
> > >> return configurator.baseline().enabled().value();
> > >> }
> > >> }
> > >>
> > >> So maybe my example is not so good. But I want to show that end-user
> > API
> > >> will be defined only in ignite-api
> > >> and you need to adapt it in ignite module which leads to some
> > >> overhead(especially in my example)
> > >> but it makes development pretty manageable/predictable -
> > >> you can easily implement a new module without any worries that user
> > starts
> > >> to use it.
> > >> It will be available only after making changes in ignite-api.
> > >> The major advantage here is the small size of ignite-api which allows
> > to
> > >> carefully review every change
> > >> which allows keeping ignite API in better quality(I hope at least)
> > >>
> > >> Nikolay, maybe is it better to discuss your question on a separate
> > topic?
> > >> Because looks like it is a pretty discussable topic.
> > >>
> > >> --
> > >> Best regards,
> > >> Anton Kalashnikov
> > >>
> > >> 09.12.2020, 10:31, "Nikolay Izhikov" <nizhi...@apache.org>:
> > >> > Hello, Zhenya, Ivan.
> > >> >
> > >> >> Hello Nikolay, if i find out introduced features structure in some
> > >> project, i would prefer to choose different one )
> > >> >
> > >> > Many, of the real world users disagree with you.
> > >> > Please, take a look at some examples from widely used projects:
> > >> >
> > >> > Kafka -
> > >>
> >
> https://github.com/apache/kafka/blob/trunk/clients/src/main/java/org/apache/kafka/common/annotation/InterfaceStability.java#L28
> > >> > - Stable, Evolving, Unstable
> > >> >
> > >> > Spark -
> > >>
> >
> https://github.com/apache/spark/tree/master/common/tags/src/main/java/org/apache/spark/annotation
> > >> > - AlphaComponent, DeveloperApi, Evolving, Experimental, Private,
> > >> Stable, Unstable
> > >> >
> > >> >> Having officially "unstable" features doesn't sound good for
> product
> > >> reputation.
> > >> >
> > >> > Can’t agree with you.
> > >> >
> > >> > Forcing ourselves to make perfect API from the first try we just
> put
> > too
> > >> much pressure on every decision.
> > >> > Every developer making mistakes.
> > >> > The product is evolving and the API too - it’s totally OK.
> > >> >
> > >> > For every new feature time required to be adopted and used in
> > real-world
> > >> production.
> > >> > I believe, slight API changes is totally fine for early adopters.
> > >> > Moreover, I think, that we should warn our users that some feature
> is
> > >> very fresh and can have issues.
> > >> >
> > >> > So, Why Kafka and Spark is good enough to have unstable API and
> > Ignite
> > >> not? :)
> > >> >
> > >> >> 9 дек. 2020 г., в 10:08, Ivan Bessonov <bessonov...@gmail.com>
> > >> написал(а):
> > >> >>
> > >> >> Conversation shifted into an unintended direction, but I agree.
> > >> >>
> > >> >> I think that if API can (or will) be changed then it should be
> > >> deprecated.
> > >> >> For that
> > >> >> we can introduce @IgniteDeprecated that will contain Ignite
> version
> > >> when
> > >> >> API is planned to be removed. Otherwise it's either stable or
> > >> experimental.
> > >> >> Having officially "unstable" features doesn't sound good for
> product
> > >> >> reputation.
> > >> >>
> > >> >> As for the modularization - I'm all for this idea. If we don't
> force
> > >> >> ourselves to
> > >> >> organize code properly then we'll end up with the same problems as
> > we
> > >> have
> > >> >> in the current code base. And this way there's a hope of having
> good
> > >> tests
> > >> >> that can be completed in minutes, not hours. At least new ones.
> > >> >>
> > >> >> BTW, did we have any discussions about dependency injection and
> all
> > >> this
> > >> >> stuff?
> > >> >> Seems like a related topic to me.
> > >> >>
> > >> >> ср, 9 дек. 2020 г. в 09:47, Zhenya Stanilovsky
> > >> <arzamas...@mail.ru.invalid>:
> > >> >>
> > >> >>> Hello Nikolay, if i find out introduced features structure in
> some
> > >> >>> project, i would prefer to choose different one )
> > >> >>>
> > >> >>>>>> Hello, Alexey.
> > >> >>>>>>
> > >> >>>>>> Think we can extend our @IgniteExperimental annotation.
> > >> >>>>>>
> > >> >>>>>> `@IgniteExperimental` - mark features that are truly
> > experimental
> > >> and
> > >> >>> can be completely removed in future releases.
> > >> >>>>>> `@NotRecommended` - mark features that widely adopted by the
> > users
> > >> but
> > >> >>> implemented wrong or have known issues that can’t be fixed.
> > >> >>>>>> `@NotStable` - mark features supported by community but API
> not
> > >> stable
> > >> >>> and can be reworked in the next release.
> > >> >>>>>> `@Stable` - mark features that are completely OK and here to
> > stay.
> > >> >>>>>>
> > >> >>>>>> We should output notes about these annotations in the JavaDoc,
> > >> also.
> > >> >>>>>> What do you think?
> > >> >>>>>>
> > >> >>>>>>> 8 дек. 2020 г., в 12:49, Alexey Goncharuk <
> > >> >>> alexey.goncha...@gmail.com > написал(а):
> > >> >>>>>>> Igniters,
> > >> >>>>>>>
> > >> >>>>>>> I want to tackle the topic of modules structure in Ignite 3.
> So
> > >> far,
> > >> >>> the
> > >> >>>>>>> modules in Ignite are mostly defined intuitively which leads
> to
> > >> some
> > >> >>>>>>> complications:
> > >> >>>>>>>
> > >> >>>>>>> - Ignite public API is separated from the rest of the code
> > only by
> > >> >>>>>>> package name. This leads to private classes leaking to public
> > API
> > >> >>> which is
> > >> >>>>>>> very hard to catch even during the review process (we missed
> a
> > >> bunch
> > >> >>> of
> > >> >>>>>>> such leaks for new metrics API [1] and I remember this
> > happening
> > >> for
> > >> >>> almost
> > >> >>>>>>> every SPI)
> > >> >>>>>>> - Classes from 'internal' packages are considered to be 'free
> > for
> > >> >>> grabs'
> > >> >>>>>>> in every place of the code. This leads to tight coupling and
> > >> >>> abstraction
> > >> >>>>>>> leakage in the code. An example of such a case - an often
> cast
> > of
> > >> >>>>>>> WALPointer to FileWALPointer, so that the community decided
> to
> > get
> > >> >>> rid of
> > >> >>>>>>> the WALPointer interface altogether [2]
> > >> >>>>>>> - Overall code complexity. Because of the lack of
> inter-module
> > >> >>>>>>> interaction rules, we are free to add new methods and
> callbacks
> > >> to any
> > >> >>>>>>> class, which leads to duplicating entities and verbose
> > >> interfaces. A
> > >> >>> good
> > >> >>>>>>> example of this is the clear duplication of methods in
> > >> >>>>>>> IgniteCacheOffheapManager and
> > IgniteCacheOffheapManager.DataStore
> > >> [3]
> > >> >>>>>>>
> > >> >>>>>>> I think we need to work out some rules that will help us
> define
> > >> and
> > >> >>> control
> > >> >>>>>>> both Ignite public API and module internal API which still
> > >> defines a
> > >> >>> clear
> > >> >>>>>>> contract for other modules. Some ideas:
> > >> >>>>>>>
> > >> >>>>>>> - Perhaps we can move all user public classed and interfaces
> > to an
> > >> >>>>>>> Ignite-API module which will have no dependencies on
> > >> implementation
> > >> >>>>>>> modules. This will prevent private classes from leaking to
> the
> > API
> > >> >>> module.
> > >> >>>>>>> - We need somehow define which classes from a module are
> > exposed
> > >> to
> > >> >>>>>>> other modules, and which classes are left for module-private
> > >> usage.
> > >> >>> Maybe
> > >> >>>>>>> Java's jigsaw will help us here, but maybe we will be ok with
> > just
> > >> >>> more
> > >> >>>>>>> strict java access modifiers usage :) The idea here is that a
> > >> module
> > >> >>> should
> > >> >>>>>>> never touch a dependent module's private classes, ever. The
> > >> exported
> > >> >>>>>>> classes and interfaces are still free to be modified between
> > >> >>> releases, as
> > >> >>>>>>> long as it is not a user public API.
> > >> >>>>>>> - A module should be logically complete, thus it may be
> > >> beneficial if
> > >> >>>>>>> module name matches with the code package it provides (e.g.
> > >> >>> configuration
> > >> >>>>>>> -> org.apache.ignite.configuration, replication ->
> > >> >>>>>>> org.apache.ignite.replication, raft->org.apache.ignite.raft,
> > etc)
> > >> >>>>>>>
> > >> >>>>>>> Any other principles/rules we can apply to make the code
> > structure
> > >> >>> more
> > >> >>>>>>> concise? Thoughts?
> > >> >>>>>>>
> > >> >>>>>>> --AG
> > >> >>>>>>>
> > >> >>>>>>> [1] https://issues.apache.org/jira/browse/IGNITE-12552
> > >> >>>>>>> [2] https://issues.apache.org/jira/browse/IGNITE-13513
> > >> >>>>>>> [3] https://issues.apache.org/jira/browse/IGNITE-13220
> > >> >>
> > >> >> --
> > >> >> Sincerely yours,
> > >> >> Ivan Bessonov
> >
>
