Hello Maxim,
My replies inline
On 11.07.2022 15:57, Maxim Muzafarov wrote:
Hello Łukasz,
One of major problems which are hard to solve without cooperation is mentioned
"split package" which is currently found in the sources.
Can you clarify details of what is the issue of having such a case? I
my understanding it will be better to consider ignite-core +
ignite-indexing + ignite-calcite as a single feature. Thus having a
"split package" issue should not be a problem. It will be a quite
challenging to keep these modules thruly without package duplication
in future.
Split package happens when same package is defined by more than one
module. In such situation OSGi framework will stick with first provider
of a given package (ie. ignite-core), simply ignoring subsequent code
chunks provided by other modules (think of ignite-indexing).
Because OSGi validate wiring at a package level, it has some impact on
how code should be structured. Simply speaking module can not import a
foreign package and export it with additional contents. I attempted to
duplicate some of split package contents within ignite-indexing but it
should be seen as a temporary workaround.
I've also briefly looked at the integration with the Apache Karaf.
Will it be possible to remove a module dependencies from the
feature-file [4]? It seems it's a bit complicated to maintain these
dependencies and corresponding versions each time some module changed.
These dependencies are recommended as it will force proper build order
of modules. Without it you may end up with a situation that feature
validation involving ignite-core will be made before before ignite-core
is being built locally. Relying on sequence order of <module> elements
in reactor pom is often insufficient to assure that.
The issue IGNITE-17046 [1] you mentioned above was merged. I've also
moved the OSGi sub-modules to the Apache Ignite Extensions project
[2], so I hope we can proceed here. Please, note that I've removed the
ignite-paxloggin module due to it is completely out of date. The
ignite-log4j will be removed very soon [3], so we have to migrate
everything to the ignite-log4j2.
Fair enough, I can take it forward from that place and try to come with
a clear Karaf feature set also for some of ignite extensions. I noticed
some of them were refereed.
I've also take a look at the packages you mentioned above. Here is the
list of packages with my comments:
org.apache.ignite.internal.managers.systemview.walker
(will be moved soon from ignite-indexing to the ignite-core)
org.apache.ignite.internal.mxbean
(ignite-indexing - classes deprecated and will be removed here [5])
org.apache.ignite.internal.processors.cache.query
(ignite-indexing - I doubt it can be simply removed currently,
refactoring is required)
org.apache.ignite.internal.processors.query.stat
(ignite-indexing - I doubt we can do anything here)
org.apache.ignite.internal.visor.cache.index
(ignite-indexing - we can simply move everything to the ignite-core)
org.apache.ignite.internal.visor.verify
(ignite-indexing - we can simply move everything to the ignite-core)
org.apache.ignite.internal.managers.systemview
(will be moved soon from ignite-indexing to the ignite-core)
I understand that moving code is difficult and causes a friction to
dependent projects, so easiest path is moving dependencies up (to
ignite-core), making it even bigger.
We do not need a full functinality re-factoring to satisfy "singularity"
of a package. It is sufficient to have unique name for package
structure. For example:
org.apache.ignite.internal.processors.cache.query in indexing could be:
- org.apache.ignite.internal.processors.cache.indexing.query
- org.apache.ignite.indexing.internal.processors.cache.query
or similar.
Best,
Łukasz
[1] https://issues.apache.org/jira/browse/IGNITE-17046
[2] https://issues.apache.org/jira/browse/IGNITE-13570
[3] https://issues.apache.org/jira/browse/IGNITE-16650
[4]
https://github.com/apache/ignite-extensions/blob/master/modules/osgi-ext/osgi-karaf/src/main/resources/features.xml#L131
[5] https://issues.apache.org/jira/browse/IGNITE-15761
On Mon, 4 Jul 2022 at 21:42, Łukasz Dywicki <l...@code-house.org> wrote:
Hello Maxim,
Thank you for your answer. I haven't expected such quick feedback. I am
here to help you with OSGi, I believe that together we will be able to
get Ignite free of troubles it gives.
I abstained from creating new issues as there are two which already
outline end user problem: IGNITE-17216, IGNITE-17216.
I do not see a problem with moving ignite-osgi module to other
repository as long as we avoid dependency cycles.
Since igite-osgi functionality seem to be limited to bootstrapping
ignite and its classloading it does not depend per say on where code
itself is located. I suppose that osgi-karaf and osgi-paxlogging should
follow that move to keep all osgi related things in one place.
Major concern I can raise is how to keep it up so it does not sink over
time. Integration tests I mentioned will form a safety net, but they
need to be part of build pipeline to follow changes in main repository.
Some of recent changes I found about cleaning out some inner-dependency
paths (ie. indexing-h2 relation: IGNITE-17046), are very helpful to
solidify OSGi metadata.
One of major problems which are hard to solve without cooperation is
mentioned "split package" which is currently found in the sources. This
happens when same package is defined (and exported) from two places.
Currently both ignite-core and indexing bring contents of
org.apache.ignite.internal and spi packages:
- org.apache.ignite.internal.managers.systemview.walker
- org.apache.ignite.internal.mxbean
- org.apache.ignite.internal.processors.cache.query
- org.apache.ignite.internal.processors.query.stat
- org.apache.ignite.internal.visor.cache.index
- org.apache.ignite.internal.visor.verify
- org.apache.ignite.spi.systemview.view
To be fair, I don't know yet what is scope of these and that's where I
need most of your guidance. I don't think we will be able to stabilize
osgi integration to guarantee it will work reliably if these stay.
Kind regards,
Łukasz
On 4.07.2022 19:43, Maxim Muzafarov wrote:
Hello Łukasz,
Nice to meet you!
Currently there are some known issues with the Apache Ignite OSGi
integration. We have a lack of time and knowledg to support it, so any
help are very appreciated. I also have a a plan to move this
integration to the Apache Ignite Extensions project [1] and release it
when all the issues are resolved. Here is an umbrella ticket for it
[2].
What do you think if I move the OSGi to the Apache Ignite Extensions
first and after it we will resolve the issue you mentioned above? I
can help with a review and try to do the best I can here.
Anyway if this plan wouldn't work for you, please, feel free to create
an issue here [3] with your PR.
[1] https://github.com/apache/ignite-extensions
[2] https://issues.apache.org/jira/browse/IGNITE-17019
[3] https://issues.apache.org/jira/projects/IGNITE/
On Mon, 4 Jul 2022 at 20:26, Łukasz Dywicki <l...@code-house.org> wrote:
Hello everyone,
First of all thank you for awesome project and its maintenance. My name
is Łukasz and I am committer of Apache Karaf and Apache PLC4X projects
hence you most likely don't know me as I am coming from other part of
ASF ecosystem.
I was asked about looking at Apache Ignite OSGi metadata as project we
are trying to use Ignite with is bound to the OSGi runtime. After a bit
of look I found that metadata generation is partial and overall
infrastructure in this area haven't received any polishing since a while.
Since there are "split packages" and few additional troubles I wanted to
check with you if there is a will to move that part ahead. Is there a
will or plan to retain the metadata or rather throw it away?
Last week I made a draft PR with changes which allowed me to run ignite
inside Apache Karaf (4.3.x): https://github.com/apache/ignite/pull/10133
There are several maven pom updates and, so far, one code relocation in
kubernetes module. I based all changes on exiting properties:
osgi.import.package, osgi.export.package, osgi.private.package
I think there are few more places where "split package" is found,
however I did not look very carefully at these yet.
Looking forward to hear out your guidance in how you would like to get
these changes, if at all. If you would be up to move that part forward I
can sacrifice more time in making metadata generation and validation
more rebust through ie. basic OSGi integration tests. PR I linked above
automatically validates import and export constraints for Karaf feautes.
Kind regards,
Łukasz
